[Bug #13319] Page allocation failures with b43 and p54usb

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (18 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Andrew Morton]

On Sat, 16 May 2009 21:20:45 +0200 (CEST) "Rafael J. Wysocki" <rjw-KKrjLPT3xs0@public.gmane.org> wrote:

> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> Subject		: Page allocation failures with b43 and p54usb
> Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> Date		: 2009-04-29 21:01 (18 days old)
> References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
> 
> 

Well..  order-1 GFP_ATOMIC allocations are unreliable.  The networking
code should hanlde the situation and recover.  I assume that is
happening in this case?

Perhaps we did something in that code after 2.6.29 which increased the
frequency of the order-1 allocation attempts?  Maybe earlier kernels
used order-0 all the time?  Those are much more reliable.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Andrew Morton wrote:
> 
> Well..  order-1 GFP_ATOMIC allocations are unreliable.  The networking
> code should hanlde the situation and recover.  I assume that is
> happening in this case?

Yes, the driver has recovered in all cases so far.

> Perhaps we did something in that code after 2.6.29 which increased the
> frequency of the order-1 allocation attempts?  Maybe earlier kernels
> used order-0 all the time?  Those are much more reliable.

I think something happened to change the allocation as I never saw these O(1)
failures before with these particular drivers. I put in a few test printk's and
the buffers were 700-800 bytes long, and I would not expect them to require more
than an O(0) allocation.

I pushed 2.6.30-rc6 hard for ~12 hours without any recurrence of the problem.
Given the relative infrequency of the error, this certainly does not indicate a
fix in recent code. I will be trying to force it again.

Larry

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi Andrew,

On Sat, 16 May 2009 21:20:45 +0200 (CEST) "Rafael J. Wysocki"
<rjw-KKrjLPT3xs0@public.gmane.org> wrote:
>> This message has been generated automatically as a part of a report
>> of recent regressions.
>>
>> The following bug entry is on the current list of known regressions
>> from 2.6.29.  Please verify if it still should be listed and let me know
>> (either way).
>>
>>
>> Bug-Entry     : http://bugzilla.kernel.org/show_bug.cgi?id=13319
>> Subject               : Page allocation failures with b43 and p54usb
>> Submitter     : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
>> Date          : 2009-04-29 21:01 (18 days old)
>> References    : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>> Handled-By    : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

On Sun, May 17, 2009 at 2:36 AM, Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> wrote:
> Well..  order-1 GFP_ATOMIC allocations are unreliable.  The networking
> code should hanlde the situation and recover.  I assume that is
> happening in this case?
>
> Perhaps we did something in that code after 2.6.29 which increased the
> frequency of the order-1 allocation attempts?  Maybe earlier kernels
> used order-0 all the time?  Those are much more reliable.

I wonder if this is related:

http://bugzilla.kernel.org/show_bug.cgi?id=13069

Both point to post 2.6.29... Hmm.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> Subject		: Page allocation failures with b43 and p54usb
> Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> Date		: 2009-04-29 21:01 (18 days old)
> References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

I have been unable to repeat the problem with recent kernels - I am now running
2.6.30-rc6. This regression should probably be dropped even though it may show
up when 2.6.30 is released.

Larry

2.6.30-rc7: Reported regressions from 2.6.29

[Rafael J. Wysocki]

[NOTE:
 Bug entries for which I know there are working patches get the
 RESOLVED / PATCH_ALREADY_AVAILABLE status in the Bugzilla, which
 makes it easier to check if the patches have been merged before sending
 out the list.  For this reason, please let me know of working patches
 for any of the regression list items or (better) change the status of the bug
 entries in case you know of a working patch (in which case please also add
 a link to the patch to the appropriate bug entry).]

This message contains a list of some regressions from 2.6.29, for which there
are no fixes in the mainline I know of.  If any of them have been fixed already,
please let me know.

If you know of any other unresolved regressions from 2.6.29, please let me know
either and I'll add them to the list.  Also, please let me know if any of the
entries below are invalid.

Each entry from the list will be sent additionally in an automatic reply to
this message with CCs to the people involved in reporting and handling the
issue.


Listed regressions statistics:

  Date          Total  Pending  Unresolved
  ----------------------------------------
  2009-05-24       92       34          27
  2009-05-16       81       36          33
  2009-04-25       55       36          26
  2009-04-17       37       35          28


Unresolved regressions
----------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13377
Subject		: Microphone no longer works on Toshiba Satellite A100
Submitter	: M. Vefa Bicakci <bicave@superonline.com>
Date		: 2009-05-24 15:49 (1 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=23f0c048ba59ad5c2f3fd85ed98360b631dbf6f8
References	: http://marc.info/?l=linux-kernel&m=124318383910011&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13374
Subject		: reiserfs blocked for more than 120secs
Submitter	: Harald Dunkel <harald.dunkel@t-online.de>
Date		: 2009-05-23 8:52 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=124306880410811&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13373
Subject		: fbcon, intelfb, i915: INFO: possible circular locking dependency detected
Submitter	: Miles Lane <miles.lane@gmail.com>
Date		: 2009-05-23 5:08 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=124305538130702&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13372
Subject		: Oops in usb-serial with keyspan adapter
Submitter	: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date		: 2009-05-18 0:07 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=124260532924736&w=4
Handled-By	: Alan Stern <stern@rowland.harvard.edu>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13366
Subject		: About 80% of shutdowns fail (blocking)
Submitter	: Martin Bammer <mrb74@gmx.at>
Date		: 2009-05-23 00:58 (2 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13341
Subject		: Random Oops at boot at loading ip6tables rules
Submitter	:  <patrick@ostenberg.de>
Date		: 2009-05-19 09:08 (6 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13339
Subject		: rtable leak in ipv4/route.c
Submitter	: Alexander V. Lukyanov <lav@yar.ru>
Date		: 2009-05-18 14:10 (7 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13337
Subject		: [post 2.6.29 regression] hang during suspend of b44/b43 modules
Submitter	: Tomas Janousek <tomi@nomi.cz>
Date		: 2009-05-18 10:59 (7 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13331
Subject		: FUTEX_LOCK_PI kills kernel
Submitter	: Andreas Schwab <schwab@linux-m68k.org>
Date		: 2009-05-17 09:51 (8 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13330
Subject		: nfs4 NULL pointer dereference in _nfs4_do_setlk
Submitter	: Rich Ercolani <rercola@acm.jhu.edu>
Date		: 2009-05-17 04:44 (8 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13328
Subject		: b44: eth0: BUG! Timeout waiting for bit 00000002 of register 42c to clear.
Submitter	: Francis Moreau <francis.moro@gmail.com>
Date		: 2009-05-03 16:22 (22 days old)
References	: http://marc.info/?l=linux-kernel&m=124136778012280&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13326
Subject		: [PATCH]Null pointer dereference in rtc-cmos driver
Submitter	: Ozan Çağlayan <ozan@pardus.org.tr>
Date		: 2009-05-14 16:16 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=124231783704696&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13325
Subject		: 2.6.30-rc kills my box hard - and lockdep chains
Submitter	: Jonathan Corbet <corbet@lwn.net>
Date		: 2009-05-14 15:49 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=124231630701394&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13323
Subject		: 2.6.30-rc deadline scheduler performance regression for iozone over NFS
Submitter	: Jeff Moyer <jmoyer@redhat.com>
Date		: 2009-04-23 14:01 (32 days old)
References	: http://marc.info/?l=linux-kernel&m=124049547915450&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger@lwfinger.net>
Date		: 2009-04-29 21:01 (26 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
Handled-By	: Johannes Berg <johannes@sipsolutions.net>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13318
Subject		: AGP doesn't work anymore on nforce2
Submitter	: Karsten Mehrhoff <kawime@gmx.de>
Date		: 2009-04-30 8:51 (25 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59de2bebabc5027f93df999d59cc65df591c3e6e
References	: http://marc.info/?l=linux-kernel&m=124108156417560&w=4
Handled-By	: Shaohua Li <shaohua.li@intel.com>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13306
Subject		: hibernate slow on _second_ run
Submitter	: Johannes Berg <johannes@sipsolutions.net>
Date		: 2009-05-14 09:34 (11 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13285
Subject		: INTELFB: Colors display incorrectly
Submitter	: Dean Menezes <samanddeanus@yahoo.com>
Date		: 2009-05-12 01:40 (13 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13250
Subject		: Side channel of Intel HDA chip doesn't work anymore, did work with 2.6.29
Submitter	: Andreas Juch <kernel-bt@juch.cc>
Date		: 2009-05-05 10:14 (20 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13219
Subject		: Since kernel 2.6.30-rc1, computers hangs randomly ..
Submitter	: David Hill <hilld@binarystorm.net>
Date		: 2009-05-01 16:57 (24 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13180
Subject		: 2.6.30-rc2: WARNING at i915_gem.c for i915_gem_idle
Submitter	: Niel Lambrechts <niel.lambrechts@gmail.com>
Date		: 2009-04-21 21:35 (34 days old)
References	: http://marc.info/?l=linux-kernel&m=124034980819102&w=4
		  http://lkml.org/lkml/2009/4/27/290


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13179
Subject		: CD-R: wodim intermittent failures
Submitter	: Andy Isaacson <adi@hexapodia.org>
Date		: 2009-04-21 1:52 (34 days old)
References	: http://marc.info/?l=linux-kernel&m=124027879214231&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13171
Subject		: 2.6.30-rc2 + xorg-intel-2.7.0 + DRM_I915_KMS = corruption
Submitter	: Alex Bennee <kernel-hacker@bennee.com>
Date		: 2009-04-19 6:27 (36 days old)
References	: http://marc.info/?l=linux-kernel&m=124022460014812&w=4
		  https://bugs.freedesktop.org/show_bug.cgi?id=21480


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13148
Subject		: resume after suspend-to-ram broken on Sony Vaio VGN-SR19VN when sony-laptop driver present
Submitter	: fanderay <fanderay4@googlemail.com>
Date		: 2009-04-22 14:39 (33 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13126
Subject		: BUG: MAX_LOCKDEP_ENTRIES too low! when mounting rootfs
Submitter	: Alexander Beregalov <a.beregalov@gmail.com>
Date		: 2009-04-15 12:43 (40 days old)
References	: http://marc.info/?l=linux-kernel&m=123979949820538&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13119
Subject		: Trouble with make-install from a NFS mount
Submitter	: Gregory Haskins <ghaskins@novell.com>
Date		: 2009-04-14 21:32 (41 days old)
References	: http://marc.info/?l=linux-kernel&m=123974482327044&w=4
Handled-By	: H. Peter Anvin <hpa@zytor.com>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13116
Subject		: Can't boot with nosmp
Submitter	: Stephen Hemminger <shemminger@vyatta.com>
Date		: 2009-04-15 4:18 (40 days old)
References	: http://marc.info/?l=linux-kernel&m=123976917817920&w=4
Handled-By	: Dan Williams <dan.j.williams@intel.com>


Regressions with patches
------------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13378
Subject		: kernel/async.c broke pata_legacy.c
Submitter	: Mikael Pettersson <mikpe@it.uu.se>
Date		: 2009-05-24 16:13 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=124318186507210&w=4
Handled-By	: James Bottomley <James.Bottomley@hansenpartnership.com>
Patch		: http://patchwork.kernel.org/patch/25699/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13293
Subject		: Kernel BUG under network load with gianfar
Submitter	: Michael Guntsche <mike@it-loops.com>
Date		: 2009-05-03 13:36 (22 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=0fd56bb5be6455d0d42241e65aed057244665e5e
References	: http://marc.info/?l=linux-kernel&m=124135824600924&w=4
Handled-By	: Lennert Buytenhek <buytenh@wantstofly.org>
Patch		: http://patchwork.kernel.org/patch/25518/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13277
Subject		: Thinkpad X40 no longer resumes reliable since ff69f2bba67bd45514923aaedbf40fe351787c59
Submitter	: Daniel Vetter <daniel@ffwll.ch>
Date		: 2009-05-11 10:08 (14 days old)
Handled-By	: Len Brown <len.brown@intel.com>
Patch		: http://patchwork.kernel.org/patch/22499/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13245
Subject		: possible circular locking dependency detected
Submitter	: Miles Lane <miles.lane@gmail.com>
Date		: 2009-05-04 16:56 (21 days old)
Handled-By	: Rafael J. Wysocki <rjw@sisk.pl>
Patch		: http://patchwork.kernel.org/patch/25557/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13125
Subject		: active uvcvideo breaks over suspend
Submitter	: Alan Jenkins <alan-jenkins@tuffmail.co.uk>
Date		: 2009-04-15 10:12 (40 days old)
References	: http://marc.info/?l=linux-kernel&m=123979009508840&w=4
Handled-By	: Ming Lei <tom.leiming@gmail.com>
Patch		: http://lkml.org/lkml/2009/4/18/5


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13121
Subject		: Buggy _BCM - acer aspire 5720G, 5710Z, 5315
Submitter	: Maxim Levitsky <maximlevitsky@gmail.com>
Date		: 2009-04-16 11:37 (39 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=1a7c618a3f7bef1a20ae740df512eeba21397fa5
References	: http://marc.info/?l=linux-kernel&m=123988189401913&w=4
Handled-By	: Zhang Rui <rui.zhang@intel.com>
Patch		: http://patchwork.kernel.org/patch/19755/
		  http://bugzilla.kernel.org/attachment.cgi?id=21268


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13109
Subject		: High latency on /sys/class/thermal
Submitter	: Tiago Simões Batista <tiagosbatista@gmail.com>
Date		: 2009-04-11 14:56 (44 days old)
References	: http://marc.info/?l=linux-kernel&m=123946182301248&w=4
Handled-By	: Zhang Rui <rui.zhang@intel.com>
		  Alexey Starikovskiy <astarikovskiy@suse.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=21061
		  http://bugzilla.kernel.org/attachment.cgi?id=21282


For details, please visit the bug entries and follow the links given in
references.

As you can see, there is a Bugzilla entry for each of the listed regressions.
There also is a Bugzilla entry used for tracking the regressions from 2.6.29,
unresolved as well as resolved, at:

http://bugzilla.kernel.org/show_bug.cgi?id=13070

Please let me know if there are any Bugzilla entries that should be added to
the list in there.

Thanks,
Rafael

[Bug #13109] High latency on /sys/class/thermal

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13109
Subject		: High latency on /sys/class/thermal
Submitter	: Tiago Simões Batista <tiagosbatista-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-04-11 14:56 (44 days old)
References	: http://marc.info/?l=linux-kernel&m=123946182301248&w=4
Handled-By	: Zhang Rui <rui.zhang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
		  Alexey Starikovskiy <astarikovskiy-l3A5Bk7waGM@public.gmane.org>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=21061
		  http://bugzilla.kernel.org/attachment.cgi?id=21282

[Bug #13119] Trouble with make-install from a NFS mount

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13119
Subject		: Trouble with make-install from a NFS mount
Submitter	: Gregory Haskins <ghaskins-Et1tbQHTxzrQT0dZR+AlfA@public.gmane.org>
Date		: 2009-04-14 21:32 (41 days old)
References	: http://marc.info/?l=linux-kernel&m=123974482327044&w=4
Handled-By	: H. Peter Anvin <hpa-YMNOUZJC4hwAvxtiuMwx3w@public.gmane.org>

[Bug #13116] Can't boot with nosmp

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13116
Subject		: Can't boot with nosmp
Submitter	: Stephen Hemminger <shemminger-ZtmgI6mnKB3QT0dZR+AlfA@public.gmane.org>
Date		: 2009-04-15 4:18 (40 days old)
References	: http://marc.info/?l=linux-kernel&m=123976917817920&w=4
Handled-By	: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

[Bug #13121] Buggy _BCM - acer aspire 5720G, 5710Z, 5315

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13121
Subject		: Buggy _BCM - acer aspire 5720G, 5710Z, 5315
Submitter	: Maxim Levitsky <maximlevitsky-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-04-16 11:37 (39 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=1a7c618a3f7bef1a20ae740df512eeba21397fa5
References	: http://marc.info/?l=linux-kernel&m=123988189401913&w=4
Handled-By	: Zhang Rui <rui.zhang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/19755/
		  http://bugzilla.kernel.org/attachment.cgi?id=21268

[Bug #13125] active uvcvideo breaks over suspend

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13125
Subject		: active uvcvideo breaks over suspend
Submitter	: Alan Jenkins <alan-jenkins-cCz0Lq7MMjm9FHfhHBbuYA@public.gmane.org>
Date		: 2009-04-15 10:12 (40 days old)
References	: http://marc.info/?l=linux-kernel&m=123979009508840&w=4
Handled-By	: Ming Lei <tom.leiming-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Patch		: http://lkml.org/lkml/2009/4/18/5

[Bug #13126] BUG: MAX_LOCKDEP_ENTRIES too low! when mounting rootfs

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13126
Subject		: BUG: MAX_LOCKDEP_ENTRIES too low! when mounting rootfs
Submitter	: Alexander Beregalov <a.beregalov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-04-15 12:43 (40 days old)
References	: http://marc.info/?l=linux-kernel&m=123979949820538&w=4

[Bug #13171] 2.6.30-rc2 + xorg-intel-2.7.0 + DRM_I915_KMS = corruption

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13171
Subject		: 2.6.30-rc2 + xorg-intel-2.7.0 + DRM_I915_KMS = corruption
Submitter	: Alex Bennee <kernel-hacker-jriKODhXDevQT0dZR+AlfA@public.gmane.org>
Date		: 2009-04-19 6:27 (36 days old)
References	: http://marc.info/?l=linux-kernel&m=124022460014812&w=4
		  https://bugs.freedesktop.org/show_bug.cgi?id=21480

[Bug #13179] CD-R: wodim intermittent failures

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13179
Subject		: CD-R: wodim intermittent failures
Submitter	: Andy Isaacson <adi-3HqRAUrWAWyGglJvpFV4uA@public.gmane.org>
Date		: 2009-04-21 1:52 (34 days old)
References	: http://marc.info/?l=linux-kernel&m=124027879214231&w=4

[Bug #13148] resume after suspend-to-ram broken on Sony Vaio VGN-SR19VN when sony-laptop driver present

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13148
Subject		: resume after suspend-to-ram broken on Sony Vaio VGN-SR19VN when sony-laptop driver present
Submitter	: fanderay <fanderay4-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org>
Date		: 2009-04-22 14:39 (33 days old)

[Bug #13180] 2.6.30-rc2: WARNING at i915_gem.c for i915_gem_idle

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13180
Subject		: 2.6.30-rc2: WARNING at i915_gem.c for i915_gem_idle
Submitter	: Niel Lambrechts <niel.lambrechts-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-04-21 21:35 (34 days old)
References	: http://marc.info/?l=linux-kernel&m=124034980819102&w=4
		  http://lkml.org/lkml/2009/4/27/290

[Bug #13219] Since kernel 2.6.30-rc1, computers hangs randomly ..

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13219
Subject		: Since kernel 2.6.30-rc1, computers hangs randomly ..
Submitter	: David Hill <hilld-HTiBYHdybX7UkGsOFmftXw@public.gmane.org>
Date		: 2009-05-01 16:57 (24 days old)

[Bug #13245] possible circular locking dependency detected

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13245
Subject		: possible circular locking dependency detected
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-05-04 16:56 (21 days old)
Handled-By	: Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/25557/

[Bug #13250] Side channel of Intel HDA chip doesn't work anymore, did work with 2.6.29

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13250
Subject		: Side channel of Intel HDA chip doesn't work anymore, did work with 2.6.29
Submitter	: Andreas Juch <kernel-bt-Sk8biX5uX1I@public.gmane.org>
Date		: 2009-05-05 10:14 (20 days old)

[Bug #13293] Kernel BUG under network load with gianfar

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13293
Subject		: Kernel BUG under network load with gianfar
Submitter	: Michael Guntsche <mike-Z92qn3yYq0hWk0Htik3J/w@public.gmane.org>
Date		: 2009-05-03 13:36 (22 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=0fd56bb5be6455d0d42241e65aed057244665e5e
References	: http://marc.info/?l=linux-kernel&m=124135824600924&w=4
Handled-By	: Lennert Buytenhek <buytenh-OLH4Qvv75CYX/NnBR394Jw@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/25518/

[Bug #13285] INTELFB: Colors display incorrectly

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13285
Subject		: INTELFB: Colors display incorrectly
Submitter	: Dean Menezes <samanddeanus-/E1597aS9LQAvxtiuMwx3w@public.gmane.org>
Date		: 2009-05-12 01:40 (13 days old)

[Bug #13277] Thinkpad X40 no longer resumes reliable since ff69f2bba67bd45514923aaedbf40fe351787c59

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13277
Subject		: Thinkpad X40 no longer resumes reliable since ff69f2bba67bd45514923aaedbf40fe351787c59
Submitter	: Daniel Vetter <daniel-/w4YWyX8dFk@public.gmane.org>
Date		: 2009-05-11 10:08 (14 days old)
Handled-By	: Len Brown <len.brown-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/22499/

[Bug #13306] hibernate slow on _second_ run

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13306
Subject		: hibernate slow on _second_ run
Submitter	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
Date		: 2009-05-14 09:34 (11 days old)

[Bug #13318] AGP doesn't work anymore on nforce2

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13318
Subject		: AGP doesn't work anymore on nforce2
Submitter	: Karsten Mehrhoff <kawime-Mmb7MZpHnFY@public.gmane.org>
Date		: 2009-04-30 8:51 (25 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59de2bebabc5027f93df999d59cc65df591c3e6e
References	: http://marc.info/?l=linux-kernel&m=124108156417560&w=4
Handled-By	: Shaohua Li <shaohua.li-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (26 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

[Bug #13323] 2.6.30-rc deadline scheduler performance regression for iozone over NFS

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13323
Subject		: 2.6.30-rc deadline scheduler performance regression for iozone over NFS
Submitter	: Jeff Moyer <jmoyer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Date		: 2009-04-23 14:01 (32 days old)
References	: http://marc.info/?l=linux-kernel&m=124049547915450&w=4

[Bug #13325] 2.6.30-rc kills my box hard - and lockdep chains

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13325
Subject		: 2.6.30-rc kills my box hard - and lockdep chains
Submitter	: Jonathan Corbet <corbet-T1hC0tSOHrs@public.gmane.org>
Date		: 2009-05-14 15:49 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=124231630701394&w=4

[Bug #13326] [PATCH]Null pointer dereference in rtc-cmos driver

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13326
Subject		: [PATCH]Null pointer dereference in rtc-cmos driver
Submitter	: Ozan Çağlayan <ozan-caicS1wCkhO6A22drWdTBw@public.gmane.org>
Date		: 2009-05-14 16:16 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=124231783704696&w=4

[Bug #13328] b44: eth0: BUG! Timeout waiting for bit 00000002 of register 42c to clear.

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13328
Subject		: b44: eth0: BUG! Timeout waiting for bit 00000002 of register 42c to clear.
Submitter	: Francis Moreau <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-05-03 16:22 (22 days old)
References	: http://marc.info/?l=linux-kernel&m=124136778012280&w=4

[Bug #13331] FUTEX_LOCK_PI kills kernel

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13331
Subject		: FUTEX_LOCK_PI kills kernel
Submitter	: Andreas Schwab <schwab-Td1EMuHUCqxL1ZNQvxDV9g@public.gmane.org>
Date		: 2009-05-17 09:51 (8 days old)

[Bug #13339] rtable leak in ipv4/route.c

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13339
Subject		: rtable leak in ipv4/route.c
Submitter	: Alexander V. Lukyanov <lav-L+1EwoRT+D8@public.gmane.org>
Date		: 2009-05-18 14:10 (7 days old)

[Bug #13330] nfs4 NULL pointer dereference in _nfs4_do_setlk

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13330
Subject		: nfs4 NULL pointer dereference in _nfs4_do_setlk
Submitter	: Rich Ercolani <rercola-YxllIAoeIHiVc3sceRu5cw@public.gmane.org>
Date		: 2009-05-17 04:44 (8 days old)

[Bug #13337] [post 2.6.29 regression] hang during suspend of b44/b43 modules

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13337
Subject		: [post 2.6.29 regression] hang during suspend of b44/b43 modules
Submitter	: Tomas Janousek <tomi-YoqI/XImC7s@public.gmane.org>
Date		: 2009-05-18 10:59 (7 days old)

[Bug #13341] Random Oops at boot at loading ip6tables rules

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13341
Subject		: Random Oops at boot at loading ip6tables rules
Submitter	:  <patrick-nxAOmsU53hB6lmGzAMPh1A@public.gmane.org>
Date		: 2009-05-19 09:08 (6 days old)

[Bug #13373] fbcon, intelfb, i915: INFO: possible circular locking dependency detected

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13373
Subject		: fbcon, intelfb, i915: INFO: possible circular locking dependency detected
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-05-23 5:08 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=124305538130702&w=4

[Bug #13372] Oops in usb-serial with keyspan adapter

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13372
Subject		: Oops in usb-serial with keyspan adapter
Submitter	: Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
Date		: 2009-05-18 0:07 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=124260532924736&w=4
Handled-By	: Alan Stern <stern-nwvwT67g6+6dFdvTe/nMLpVzexx5G7lz@public.gmane.org>

[Bug #13366] About 80% of shutdowns fail (blocking)

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13366
Subject		: About 80% of shutdowns fail (blocking)
Submitter	: Martin Bammer <mrb74-RbZlAiThDcE@public.gmane.org>
Date		: 2009-05-23 00:58 (2 days old)

[Bug #13377] Microphone no longer works on Toshiba Satellite A100

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13377
Subject		: Microphone no longer works on Toshiba Satellite A100
Submitter	: M. Vefa Bicakci <bicave-qUEp0r/b3FNfOZc0+OmrVg@public.gmane.org>
Date		: 2009-05-24 15:49 (1 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=23f0c048ba59ad5c2f3fd85ed98360b631dbf6f8
References	: http://marc.info/?l=linux-kernel&m=124318383910011&w=4

[Bug #13374] reiserfs blocked for more than 120secs

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13374
Subject		: reiserfs blocked for more than 120secs
Submitter	: Harald Dunkel <harald.dunkel-zqRNUXuvxA0b1SvskN2V4Q@public.gmane.org>
Date		: 2009-05-23 8:52 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=124306880410811&w=4

[Bug #13378] kernel/async.c broke pata_legacy.c

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13378
Subject		: kernel/async.c broke pata_legacy.c
Submitter	: Mikael Pettersson <mikpe-1zs4UD6AkMk@public.gmane.org>
Date		: 2009-05-24 16:13 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=124318186507210&w=4
Handled-By	: James Bottomley <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/egYHeGw8Jk@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/25699/

Re: 2.6.30-rc7: Reported regressions from 2.6.29

[Ozan Çağlayan]

Rafael J. Wysocki wrote:
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13326
> Subject		: [PATCH]Null pointer dereference in rtc-cmos driver
> Submitter	: Ozan Çağlayan <ozan-caicS1wCkhO6A22drWdTBw@public.gmane.org>
> Date		: 2009-05-14 16:16 (11 days old)
> References	: http://marc.info/?l=linux-kernel&m=124231783704696&w=4
>   

Just to notice that the patch for this is at:

http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/patches/driver-core.current/driver-core-do-not-oops-when-driver_unregister-is-called-for-unregistered-drivers.patch

Can at least be marked as "has a patch".

Thanks,
Ozan Caglayan
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.6.30-rc7: Reported regressions from 2.6.29

[Rafael J. Wysocki]

On Monday 25 May 2009, Ozan Çağlayan wrote:
> Rafael J. Wysocki wrote:
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13326
> > Subject		: [PATCH]Null pointer dereference in rtc-cmos driver
> > Submitter	: Ozan Çağlayan <ozan-caicS1wCkhO6A22drWdTBw@public.gmane.org>
> > Date		: 2009-05-14 16:16 (11 days old)
> > References	: http://marc.info/?l=linux-kernel&m=124231783704696&w=4
> >   
> 
> Just to notice that the patch for this is at:
> 
> http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/patches/driver-core.current/driver-core-do-not-oops-when-driver_unregister-is-called-for-unregistered-drivers.patch
> 
> Can at least be marked as "has a patch".

Thanks, I added the patch to the bug entry and changed its status.

Best,
Rafael

Re: [Bug #13339] rtable leak in ipv4/route.c

[Eric Dumazet]

Rafael J. Wysocki a écrit :
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13339
> Subject		: rtable leak in ipv4/route.c
> Submitter	: Alexander V. Lukyanov <lav-L+1EwoRT+D8@public.gmane.org>
> Date		: 2009-05-18 14:10 (7 days old)
> 

Bug was present in 2.6.29, so its a regression from 2.6.28

It is solved and available in David tree (net-2.6), and scheduled for stable submission

commit 1ddbcb005c395518c2cd0df504cff3d4b5c85853
net: fix rtable leak in net/ipv4/route.c

Alexander V. Lukyanov found a regression in 2.6.29 and made a complete
analysis found in http://bugzilla.kernel.org/show_bug.cgi?id=13339
Quoted here because its a perfect one :

begin_of_quotation
 2.6.29 patch has introduced flexible route cache rebuilding. Unfortunately the
 patch has at least one critical flaw, and another problem.

 rt_intern_hash calculates rthi pointer, which is later used for new entry
 insertion. The same loop calculates cand pointer which is used to clean the
 list. If the pointers are the same, rtable leak occurs, as first the cand is
 removed then the new entry is appended to it.

 This leak leads to unregister_netdevice problem (usage count > 0).

 Another problem of the patch is that it tries to insert the entries in certain
 order, to facilitate counting of entries distinct by all but QoS parameters.
 Unfortunately, referencing an existing rtable entry moves it to list beginning,
 to speed up further lookups, so the carefully built order is destroyed.

 For the first problem the simplest patch it to set rthi=0 when rthi==cand, but
 it will also destroy the ordering.
end_of_quotation

Problematic commit is 1080d709fb9d8cd4392f93476ee46a9d6ea05a5b
(net: implement emergency route cache rebulds when gc_elasticity is exceeded)

Trying to keep dst_entries ordered is too complex and breaks the fact that
order should depend on the frequency of use for garbage collection.

A possible fix is to make rt_intern_hash() simpler, and only makes
rt_check_expire() a litle bit smarter, being able to cope with an arbitrary
entries order. The added loop is running on cache hot data, while cpu
is prefetching next object, so should be unnoticied.

Reported-and-analyzed-by: Alexander V. Lukyanov <lav-L+1EwoRT+D8@public.gmane.org>
Signed-off-by: Eric Dumazet <dada1-fPLkHRcR87vqlBn2x/YWAg@public.gmane.org>
Acked-by: Neil Horman <nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org>
Signed-off-by: David S. Miller <davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>

Thanks

Re: [Bug #13148] resume after suspend-to-ram broken on Sony Vaio VGN-SR19VN when sony-laptop driver present

[Mattia Dongili]

On Sun, May 24, 2009 at 09:11:50PM +0200, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13148
> Subject		: resume after suspend-to-ram broken on Sony Vaio VGN-SR19VN when sony-laptop driver present
> Submitter	: fanderay <fanderay4-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org>
> Date		: 2009-04-22 14:39 (33 days old)

sorry for not mentioning this before, but it looks like this regression
was introduced between .28 and .29

cheers
-- 
mattia
:wq!

Re: [Bug #13293] Kernel BUG under network load with gianfar

[Lennert Buytenhek]

On Sun, May 24, 2009 at 09:11:52PM +0200, Rafael J. Wysocki wrote:

> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13293
> Subject		: Kernel BUG under network load with gianfar
> Submitter	: Michael Guntsche <mike-Z92qn3yYq0hWk0Htik3J/w@public.gmane.org>
> Date		: 2009-05-03 13:36 (22 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=0fd56bb5be6455d0d42241e65aed057244665e5e
> References	: http://marc.info/?l=linux-kernel&m=124135824600924&w=4
> Handled-By	: Lennert Buytenhek <buytenh-OLH4Qvv75CYX/NnBR394Jw@public.gmane.org>
> Patch		: http://patchwork.kernel.org/patch/25518/

The proposed patch has yet to be ACKed by the maintainer/merged, so
yes, this regression should still be listed.

Re: [Bug #13126] BUG: MAX_LOCKDEP_ENTRIES too low! when mounting rootfs

[Peter Zijlstra]

On Sun, 2009-05-24 at 21:11 +0200, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13126
> Subject		: BUG: MAX_LOCKDEP_ENTRIES too low! when mounting rootfs
> Submitter	: Alexander Beregalov <a.beregalov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-04-15 12:43 (40 days old)
> References	: http://marc.info/?l=linux-kernel&m=123979949820538&w=4

-rc7 includes:

Ingo Molnar (1):
      lockdep: increase MAX_LOCKDEP_ENTRIES and MAX_LOCKDEP_CHAINS

Which can be found with commit id: 
   d80c19df5fcceb8c741e96f09f275c2da719efef

Re: [Bug #13331] FUTEX_LOCK_PI kills kernel

[Peter Zijlstra]

On Sun, 2009-05-24 at 21:11 +0200, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13331
> Subject		: FUTEX_LOCK_PI kills kernel
> Submitter	: Andreas Schwab <schwab-Td1EMuHUCqxL1ZNQvxDV9g@public.gmane.org>
> Date		: 2009-05-17 09:51 (8 days old)

commit 64d1304a64477629cb16b75491a77bafe6f86963
Author: Thomas Gleixner <tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org>
Date:   Mon May 18 21:20:10 2009 +0200

    futex: setup writeable mapping for futex ops which modify user space data

Re: [Bug #13171] 2.6.30-rc2 + xorg-intel-2.7.0 + DRM_I915_KMS = corruption

[Alex Bennee]

2009/5/24 Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=13171
> Subject         : 2.6.30-rc2 + xorg-intel-2.7.0 + DRM_I915_KMS = corruption
> Submitter       : Alex Bennee <kernel-hacker-jriKODhXDevQT0dZR+AlfA@public.gmane.org>
> Date            : 2009-04-19 6:27 (36 days old)
> References      : http://marc.info/?l=linux-kernel&m=124022460014812&w=4
>                  https://bugs.freedesktop.org/show_bug.cgi?id=21480
>

I've tested on the latest rc7 kernel and can now boot into the X with
KMS enabled. I believe that's probably good enough for this regression
to be cleared.

I still have some issues with KMS enabled which means it shouldn't be
made a default for intel just yet however I'm not sure where the
problem lie between kernel, X and apps. Compared to a none KMS system
(same xorg.conf, same kernel):

1. GDM gets confused about screen sizes and comes up 1440x900 instead
of stretching the screen to 1680x1050. However the X session itself
gets the right size.

2. Full screen video freezes X. At this point I cannot switch to VT
and need to ssh in to kill and restore X. The logs/dmesg are fairly
mute as to why though.

3. Performance seems to be lower. The compiz benchmark happily report
120fps without KMS on my normal display and around 60FPS with it
enabled. Some displays (like the big wall plugin) the difference is
less marked (14FPS to 9FPS).

I've also seen some oopses occur while running:

[  406.818468] [drm] TMDS-8: set mode  2e
[  478.370123] [drm] DAC-6: set mode 1440x900 1b
[  478.475342] [drm] TMDS-8: set mode 1680x1050 2d
[  660.644103] [drm] TMDS-8: set mode  2f
[  960.272242] INFO: task events/0:9 blocked for more than 120 seconds.
[  960.272245] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  960.272248] events/0      D ffff8800b70a58c0     0     9      2
[  960.272254]  ffff8800b70b5de0 0000000000000046 ffff8800b103dc41
ffff880001028980
[  960.272260]  0000000000004000 0000000000010dc0 000000000000ca38
ffff8800b0d31678
[  960.272265]  ffff8800b70b5d60 ffff8800b70a58c0 ffffffff80648350
ffff8800b70a5c48
[  960.272270] Call Trace:
[  960.272280]  [<ffffffff804bfce6>] ? thread_return+0x3e/0xaa
[  960.272285]  [<ffffffff804c0589>] __mutex_lock_slowpath+0xdf/0x129
[  960.272290]  [<ffffffff804c092d>] mutex_lock+0x23/0x3b
[  960.272295]  [<ffffffff803cecd3>] i915_gem_retire_work_handler+0x2d/0x6b
[  960.272301]  [<ffffffff80247666>] worker_thread+0x176/0x20f
[  960.272306]  [<ffffffff803ceca6>] ? i915_gem_retire_work_handler+0x0/0x6b
[  960.272311]  [<ffffffff8024b463>] ? autoremove_wake_function+0x0/0x3d
[  960.272315]  [<ffffffff802474f0>] ? worker_thread+0x0/0x20f
[  960.272319]  [<ffffffff802474f0>] ? worker_thread+0x0/0x20f
[  960.272323]  [<ffffffff8024b061>] kthread+0x5b/0x88
[  960.272329]  [<ffffffff8020c0ba>] child_rip+0xa/0x20
[  960.272333]  [<ffffffff8024b006>] ? kthread+0x0/0x88
[  960.272337]  [<ffffffff8020c0b0>] ? child_rip+0x0/0x20
[ 1200.272036] INFO: task events/0:9 blocked for more than 120 seconds.
[ 1200.272039] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 1200.272043] events/0      D ffff8800b70a58c0     0     9      2
[ 1200.272048]  ffff8800b70b5de0 0000000000000046 ffff8800b103dc41
ffff880001028980
[ 1200.272054]  0000000000004000 0000000000010dc0 000000000000ca38
ffff8800b0d31678
[ 1200.272059]  ffff8800b70b5d60 ffff8800b70a58c0 ffff8800a6c98000
ffff8800b70a5c48
[ 1200.272064] Call Trace:
[ 1200.272074]  [<ffffffff804bfce6>] ? thread_return+0x3e/0xaa
[ 1200.272079]  [<ffffffff804c0589>] __mutex_lock_slowpath+0xdf/0x129
[ 1200.272083]  [<ffffffff804c092d>] mutex_lock+0x23/0x3b
[ 1200.272089]  [<ffffffff803cecd3>] i915_gem_retire_work_handler+0x2d/0x6b
[ 1200.272095]  [<ffffffff80247666>] worker_thread+0x176/0x20f
[ 1200.272100]  [<ffffffff803ceca6>] ? i915_gem_retire_work_handler+0x0/0x6b
[ 1200.272105]  [<ffffffff8024b463>] ? autoremove_wake_function+0x0/0x3d
[ 1200.272109]  [<ffffffff802474f0>] ? worker_thread+0x0/0x20f
[ 1200.272114]  [<ffffffff802474f0>] ? worker_thread+0x0/0x20f
[ 1200.272117]  [<ffffffff8024b061>] kthread+0x5b/0x88
[ 1200.272123]  [<ffffffff8020c0ba>] child_rip+0xa/0x20
[ 1200.272127]  [<ffffffff8024b006>] ? kthread+0x0/0x88
[ 1200.272131]  [<ffffffff8020c0b0>] ? child_rip+0x0/0x20
[ 1200.272154] INFO: task bash:5541 blocked for more than 120 seconds.
[ 1200.272156] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 1200.272159] bash          D ffff88009cb01a90     0  5541      1
[ 1200.272163]  ffff88009cb01928 0000000000000046 ffff8800b6a03400
ffff88009cb01988
[ 1200.272168]  0000000000004000 0000000000010dc0 000000000000ca38
0000000000000000
[ 1200.272173]  ffff88009cb019e8 ffff88009e0058c0 ffff8800a9544290
ffff88009e005c48
[ 1200.272178] Call Trace:
[ 1200.272184]  [<ffffffff8027a5b8>] ? find_get_page+0x28/0x89
[ 1200.272189]  [<ffffffff8027e28e>] ? __rmqueue_smallest+0xb7/0x141
[ 1200.272193]  [<ffffffff804bfd60>] schedule+0xe/0x22
[ 1200.272197]  [<ffffffff804c007b>] schedule_timeout+0x29/0x162
[ 1200.272201]  [<ffffffff8027e552>] ? rmqueue_bulk+0x47/0x91
[ 1200.272205]  [<ffffffff804bff10>] wait_for_common+0xd4/0x13a
[ 1200.272210]  [<ffffffff80230c9f>] ? default_wake_function+0x0/0x14
[ 1200.272215]  [<ffffffff804c0019>] wait_for_completion+0x1d/0x1f
[ 1200.272219]  [<ffffffff8024794c>] flush_cpu_workqueue+0x72/0x7e
[ 1200.272223]  [<ffffffff802479bd>] ? wq_barrier_func+0x0/0x14
[ 1200.272228]  [<ffffffff80247f5d>] flush_workqueue+0x46/0x6c
[ 1200.272232]  [<ffffffff80247f98>] flush_scheduled_work+0x15/0x17
[ 1200.272237]  [<ffffffff8039fb26>] tty_ldisc_release+0x4f/0x1da
[ 1200.272242]  [<ffffffff8022a89a>] ? __wake_up+0x48/0x55
[ 1200.272253]  [<ffffffff8039b5d6>] tty_release_dev+0x4ae/0x4e3
[ 1200.272257]  [<ffffffff802a18cd>] ? discard_slab+0x43/0x45
[ 1200.272260]  [<ffffffff8039b629>] tty_release+0x1e/0x29
[ 1200.272263]  [<ffffffff802a87ed>] __fput+0xe4/0x19d
[ 1200.272266]  [<ffffffff802a88c3>] fput+0x1d/0x1f
[ 1200.272270]  [<ffffffff802a5b42>] filp_close+0x5e/0x68
[ 1200.272273]  [<ffffffff802392d7>] put_files_struct+0x70/0xc7
[ 1200.272276]  [<ffffffff8023937a>] exit_files+0x4c/0x55
[ 1200.272279]  [<ffffffff8023ac10>] do_exit+0x1c7/0x637
[ 1200.272282]  [<ffffffff80230cc3>] ? wake_up_state+0x10/0x12
[ 1200.272286]  [<ffffffff80242cc1>] ? signal_wake_up+0x3d/0x41
[ 1200.272289]  [<ffffffff8023b0f7>] do_group_exit+0x77/0xa4
[ 1200.272292]  [<ffffffff80244944>] get_signal_to_deliver+0x318/0x338
[ 1200.272296]  [<ffffffff8020a525>] do_notify_resume+0x90/0x6f9
[ 1200.272299]  [<ffffffff8024391c>] ? kill_pid_info+0x3f/0x4d
[ 1200.272302]  [<ffffffff80243b2e>] ? sys_kill+0x90/0x17b
[ 1200.272305]  [<ffffffff8024221a>] ? sys_rt_sigaction+0x7a/0x98
[ 1200.272308]  [<ffffffff8020b235>] sysret_signal+0x6d/0xb7
[ 1320.272017] INFO: task events/0:9 blocked for more than 120 seconds.
[ 1320.272020] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 1320.272023] events/0      D ffff8800b70a58c0     0     9      2
[ 1320.272029]  ffff8800b70b5de0 0000000000000046 ffff8800b103dc41
ffff880001028980
[ 1320.272034]  0000000000004000 0000000000010dc0 000000000000ca38
ffff8800b0d31678
[ 1320.272040]  ffff8800b70b5d60 ffff8800b70a58c0 ffff8800a6c98000
ffff8800b70a5c48
[ 1320.272045] Call Trace:
[ 1320.272053]  [<ffffffff804bfce6>] ? thread_return+0x3e/0xaa
[ 1320.272059]  [<ffffffff804c0589>] __mutex_lock_slowpath+0xdf/0x129
[ 1320.272063]  [<ffffffff804c092d>] mutex_lock+0x23/0x3b
[ 1320.272068]  [<ffffffff803cecd3>] i915_gem_retire_work_handler+0x2d/0x6b
[ 1320.272074]  [<ffffffff80247666>] worker_thread+0x176/0x20f
[ 1320.272078]  [<ffffffff803ceca6>] ? i915_gem_retire_work_handler+0x0/0x6b
[ 1320.272083]  [<ffffffff8024b463>] ? autoremove_wake_function+0x0/0x3d
[ 1320.272088]  [<ffffffff802474f0>] ? worker_thread+0x0/0x20f
[ 1320.272092]  [<ffffffff802474f0>] ? worker_thread+0x0/0x20f
[ 1320.272096]  [<ffffffff8024b061>] kthread+0x5b/0x88
[ 1320.272101]  [<ffffffff8020c0ba>] child_rip+0xa/0x20
[ 1320.272105]  [<ffffffff8024b006>] ? kthread+0x0/0x88
[ 1320.272110]  [<ffffffff8020c0b0>] ? child_rip+0x0/0x20

But I've been unable to replicate these reliably on a kernel with
mutex debugging enabled.

-- 
Alex, homepage: http://www.bennee.com/~alex/
CV: http://www.bennee.com/~alex/cv.php

Re: [Bug #13125] active uvcvideo breaks over suspend

[Alan Jenkins]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13125
> Subject		: active uvcvideo breaks over suspend
> Submitter	: Alan Jenkins <alan-jenkins-cCz0Lq7MMjm9FHfhHBbuYA@public.gmane.org>
> Date		: 2009-04-15 10:12 (40 days old)
> References	: http://marc.info/?l=linux-kernel&m=123979009508840&w=4
> Handled-By	: Ming Lei <tom.leiming-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Patch		: http://lkml.org/lkml/2009/4/18/5
>   

Seems to be fixed in RC7, thanks.

Re: [Bug #13125] active uvcvideo breaks over suspend

[Rafael J. Wysocki]

On Monday 25 May 2009, Alan Jenkins wrote:
> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.29.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13125
> > Subject		: active uvcvideo breaks over suspend
> > Submitter	: Alan Jenkins <alan-jenkins-cCz0Lq7MMjm9FHfhHBbuYA@public.gmane.org>
> > Date		: 2009-04-15 10:12 (40 days old)
> > References	: http://marc.info/?l=linux-kernel&m=123979009508840&w=4
> > Handled-By	: Ming Lei <tom.leiming-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Patch		: http://lkml.org/lkml/2009/4/18/5
> >   
> 
> Seems to be fixed in RC7, thanks.

Thanks, closed.

Rafael

Re: [Bug #13126] BUG: MAX_LOCKDEP_ENTRIES too low! when mounting rootfs

[Rafael J. Wysocki]

On Monday 25 May 2009, Peter Zijlstra wrote:
> On Sun, 2009-05-24 at 21:11 +0200, Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.29.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13126
> > Subject		: BUG: MAX_LOCKDEP_ENTRIES too low! when mounting rootfs
> > Submitter	: Alexander Beregalov <a.beregalov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date		: 2009-04-15 12:43 (40 days old)
> > References	: http://marc.info/?l=linux-kernel&m=123979949820538&w=4
> 
> -rc7 includes:
> 
> Ingo Molnar (1):
>       lockdep: increase MAX_LOCKDEP_ENTRIES and MAX_LOCKDEP_CHAINS
> 
> Which can be found with commit id: 
>    d80c19df5fcceb8c741e96f09f275c2da719efef

Thanks, closed.

Rafael

Re: [Bug #13148] resume after suspend-to-ram broken on Sony Vaio VGN-SR19VN when sony-laptop driver present

[Rafael J. Wysocki]

On Monday 25 May 2009, Mattia Dongili wrote:
> On Sun, May 24, 2009 at 09:11:50PM +0200, Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.29.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13148
> > Subject		: resume after suspend-to-ram broken on Sony Vaio VGN-SR19VN when sony-laptop driver present
> > Submitter	: fanderay <fanderay4-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org>
> > Date		: 2009-04-22 14:39 (33 days old)
> 
> sorry for not mentioning this before, but it looks like this regression
> was introduced between .28 and .29

Thanks, moved to the other list.

Rafael

Re: [Bug #13331] FUTEX_LOCK_PI kills kernel

[Rafael J. Wysocki]

On Monday 25 May 2009, Peter Zijlstra wrote:
> On Sun, 2009-05-24 at 21:11 +0200, Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.29.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13331
> > Subject		: FUTEX_LOCK_PI kills kernel
> > Submitter	: Andreas Schwab <schwab-Td1EMuHUCqxL1ZNQvxDV9g@public.gmane.org>
> > Date		: 2009-05-17 09:51 (8 days old)
> 
> commit 64d1304a64477629cb16b75491a77bafe6f86963
> Author: Thomas Gleixner <tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org>
> Date:   Mon May 18 21:20:10 2009 +0200
> 
>     futex: setup writeable mapping for futex ops which modify user space data

Thanks, closed.

Rafael

Re: [Bug #13339] rtable leak in ipv4/route.c

[Rafael J. Wysocki]

On Monday 25 May 2009, Eric Dumazet wrote:
> Rafael J. Wysocki a écrit :
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.29.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13339
> > Subject		: rtable leak in ipv4/route.c
> > Submitter	: Alexander V. Lukyanov <lav-L+1EwoRT+D8@public.gmane.org>
> > Date		: 2009-05-18 14:10 (7 days old)
> > 
> 
> Bug was present in 2.6.29, so its a regression from 2.6.28
> 
> It is solved and available in David tree (net-2.6), and scheduled for stable submission
> 
> commit 1ddbcb005c395518c2cd0df504cff3d4b5c85853
> net: fix rtable leak in net/ipv4/route.c
> 
> Alexander V. Lukyanov found a regression in 2.6.29 and made a complete
> analysis found in http://bugzilla.kernel.org/show_bug.cgi?id=13339
> Quoted here because its a perfect one :
> 
> begin_of_quotation
>  2.6.29 patch has introduced flexible route cache rebuilding. Unfortunately the
>  patch has at least one critical flaw, and another problem.
> 
>  rt_intern_hash calculates rthi pointer, which is later used for new entry
>  insertion. The same loop calculates cand pointer which is used to clean the
>  list. If the pointers are the same, rtable leak occurs, as first the cand is
>  removed then the new entry is appended to it.
> 
>  This leak leads to unregister_netdevice problem (usage count > 0).
> 
>  Another problem of the patch is that it tries to insert the entries in certain
>  order, to facilitate counting of entries distinct by all but QoS parameters.
>  Unfortunately, referencing an existing rtable entry moves it to list beginning,
>  to speed up further lookups, so the carefully built order is destroyed.
> 
>  For the first problem the simplest patch it to set rthi=0 when rthi==cand, but
>  it will also destroy the ordering.
> end_of_quotation
> 
> Problematic commit is 1080d709fb9d8cd4392f93476ee46a9d6ea05a5b
> (net: implement emergency route cache rebulds when gc_elasticity is exceeded)
> 
> Trying to keep dst_entries ordered is too complex and breaks the fact that
> order should depend on the frequency of use for garbage collection.
> 
> A possible fix is to make rt_intern_hash() simpler, and only makes
> rt_check_expire() a litle bit smarter, being able to cope with an arbitrary
> entries order. The added loop is running on cache hot data, while cpu
> is prefetching next object, so should be unnoticied.
> 
> Reported-and-analyzed-by: Alexander V. Lukyanov <lav-L+1EwoRT+D8@public.gmane.org>
> Signed-off-by: Eric Dumazet <dada1-fPLkHRcR87vqlBn2x/YWAg@public.gmane.org>
> Acked-by: Neil Horman <nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org>
> Signed-off-by: David S. Miller <davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>

Thanks, updated.

Rafael

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (32 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.29.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (40 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.29.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> Subject		: Page allocation failures with b43 and p54usb
> Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> Date		: 2009-04-29 21:01 (40 days old)
> References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

This bug is extremely difficult to pin down. I cannot reproduce it at
will. The system has to be up for a long time, which is difficult with
testing the late RC's of 2.6.30 and the code in wireless-testing so
that new bugs don't end up in 2.6.31-RCX. That said, it still was in
2.6.30-RC6 and I'm not aware of any changes since that would fix it.

My operating kernel is patched with additional diagnostics to help me
understand why a kmalloc request for a buffer of 1390 bytes suddenly
ends up as an O(1) request. Unfortunately, I don't have any answers.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi Larry,

On Sun, Jun 7, 2009 at 4:10 PM, Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org> wrote:
> Rafael J. Wysocki wrote:
>> This message has been generated automatically as a part of a report
>> of recent regressions.
>>
>> The following bug entry is on the current list of known regressions
>> from 2.6.29.  Please verify if it still should be listed and let me know
>> (either way).
>>
>>
>> Bug-Entry     : http://bugzilla.kernel.org/show_bug.cgi?id=13319
>> Subject               : Page allocation failures with b43 and p54usb
>> Submitter     : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
>> Date          : 2009-04-29 21:01 (40 days old)
>> References    : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>> Handled-By    : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
>
> This bug is extremely difficult to pin down. I cannot reproduce it at
> will. The system has to be up for a long time, which is difficult with
> testing the late RC's of 2.6.30 and the code in wireless-testing so
> that new bugs don't end up in 2.6.31-RCX. That said, it still was in
> 2.6.30-RC6 and I'm not aware of any changes since that would fix it.
>
> My operating kernel is patched with additional diagnostics to help me
> understand why a kmalloc request for a buffer of 1390 bytes suddenly
> ends up as an O(1) request. Unfortunately, I don't have any answers.

Looking at the out-of-memory trace, there's still memory available but
the pskb_expand_head() allocation is GFP_ATOMIC so there's not much
the page allocator can do here. The amount of memory consumed by
inactive_file is pretty high so maybe the problem is related to the
recent mm/vmscan.c changes. Lets copy some more mm developers and see
if they can help out.

                        Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Rik van Riel]

Pekka Enberg wrote:
> Hi Larry,
> 
> On Sun, Jun 7, 2009 at 4:10 PM, Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org> wrote:
>> Rafael J. Wysocki wrote:
>>> This message has been generated automatically as a part of a report
>>> of recent regressions.
>>>
>>> The following bug entry is on the current list of known regressions
>>> from 2.6.29.  Please verify if it still should be listed and let me know
>>> (either way).
>>>
>>>
>>> Bug-Entry     : http://bugzilla.kernel.org/show_bug.cgi?id=13319
>>> Subject               : Page allocation failures with b43 and p54usb
>>> Submitter     : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
>>> Date          : 2009-04-29 21:01 (40 days old)
>>> References    : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>>> Handled-By    : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
>> This bug is extremely difficult to pin down. I cannot reproduce it at
>> will. The system has to be up for a long time, which is difficult with
>> testing the late RC's of 2.6.30 and the code in wireless-testing so
>> that new bugs don't end up in 2.6.31-RCX. That said, it still was in
>> 2.6.30-RC6 and I'm not aware of any changes since that would fix it.
>>
>> My operating kernel is patched with additional diagnostics to help me
>> understand why a kmalloc request for a buffer of 1390 bytes suddenly
>> ends up as an O(1) request. Unfortunately, I don't have any answers.
> 
> Looking at the out-of-memory trace, there's still memory available but
> the pskb_expand_head() allocation is GFP_ATOMIC so there's not much
> the page allocator can do here. The amount of memory consumed by
> inactive_file is pretty high so maybe the problem is related to the
> recent mm/vmscan.c changes. Lets copy some more mm developers and see
> if they can help out.

That is a very strange trace.  The Mem-Info indicates
that the system has more than enough memory free, and
also enough memory in higher-order free blocks.

This would indicate a bug somewhere in the page
allocator - this memory should have been given to this
allocation request.

-- 
All rights reversed.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Sun, Jun 7, 2009 at 5:19 PM, Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> Pekka Enberg wrote:
>>
>> Hi Larry,
>>
>> On Sun, Jun 7, 2009 at 4:10 PM, Larry Finger <Larry.Finger@lwfinger.net>
>> wrote:
>>>
>>> Rafael J. Wysocki wrote:
>>>>
>>>> This message has been generated automatically as a part of a report
>>>> of recent regressions.
>>>>
>>>> The following bug entry is on the current list of known regressions
>>>> from 2.6.29.  Please verify if it still should be listed and let me know
>>>> (either way).
>>>>
>>>>
>>>> Bug-Entry     : http://bugzilla.kernel.org/show_bug.cgi?id=13319
>>>> Subject               : Page allocation failures with b43 and p54usb
>>>> Submitter     : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
>>>> Date          : 2009-04-29 21:01 (40 days old)
>>>> References    : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>>>> Handled-By    : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
>>>
>>> This bug is extremely difficult to pin down. I cannot reproduce it at
>>> will. The system has to be up for a long time, which is difficult with
>>> testing the late RC's of 2.6.30 and the code in wireless-testing so
>>> that new bugs don't end up in 2.6.31-RCX. That said, it still was in
>>> 2.6.30-RC6 and I'm not aware of any changes since that would fix it.
>>>
>>> My operating kernel is patched with additional diagnostics to help me
>>> understand why a kmalloc request for a buffer of 1390 bytes suddenly
>>> ends up as an O(1) request. Unfortunately, I don't have any answers.
>>
>> Looking at the out-of-memory trace, there's still memory available but
>> the pskb_expand_head() allocation is GFP_ATOMIC so there's not much
>> the page allocator can do here. The amount of memory consumed by
>> inactive_file is pretty high so maybe the problem is related to the
>> recent mm/vmscan.c changes. Lets copy some more mm developers and see
>> if they can help out.
>
> That is a very strange trace.  The Mem-Info indicates
> that the system has more than enough memory free, and
> also enough memory in higher-order free blocks.
>
> This would indicate a bug somewhere in the page
> allocator - this memory should have been given to this
> allocation request.

Aha, I always have difficulties deciphering the traces. But lets
invite Mel to the party then!

                        Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Pekka Enberg wrote:
> On Sun, Jun 7, 2009 at 5:19 PM, Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>> That is a very strange trace.  The Mem-Info indicates
>> that the system has more than enough memory free, and
>> also enough memory in higher-order free blocks.
>>
>> This would indicate a bug somewhere in the page
>> allocator - this memory should have been given to this
>> allocation request.
> 
> Aha, I always have difficulties deciphering the traces. But lets
> invite Mel to the party then!

I'm happy to see some action on this problem. As usual, I'm happy to
test patches and/or provide diagnostic output.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[KAMEZAWA Hiroyuki]

On Sun, 07 Jun 2009 11:35:27 -0500
Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org> wrote:

> Pekka Enberg wrote:
> > On Sun, Jun 7, 2009 at 5:19 PM, Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> >> That is a very strange trace.  The Mem-Info indicates
> >> that the system has more than enough memory free, and
> >> also enough memory in higher-order free blocks.
> >>
> >> This would indicate a bug somewhere in the page
> >> allocator - this memory should have been given to this
> >> allocation request.
> > 
> > Aha, I always have difficulties deciphering the traces. But lets
> > invite Mel to the party then!
> 
> I'm happy to see some action on this problem. As usual, I'm happy to
> test patches and/or provide diagnostic output.
> 
One question. 

Did your system fragmented in same way as to this
(see DMA32, 10052 of order-0 pages) in older kernel ? I think you can check
fragmentation status via /proc/buddyinfo.
=
kernel: Node 0 DMA: 3*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2100kB
kernel: Node 0 DMA32: 10062*4kB 1*8kB 1*16kB 0*32kB 1*64kB 1*128kB 0*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 40976kB
==

Thanks,
-Kame

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Mel Gorman]

On Sun, Jun 07, 2009 at 05:32:52PM +0300, Pekka Enberg wrote:
> On Sun, Jun 7, 2009 at 5:19 PM, Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > Pekka Enberg wrote:
> >>
> >> Hi Larry,
> >>
> >> On Sun, Jun 7, 2009 at 4:10 PM, Larry Finger <Larry.Finger@lwfinger.net>
> >> wrote:
> >>>
> >>> Rafael J. Wysocki wrote:
> >>>>
> >>>> This message has been generated automatically as a part of a report
> >>>> of recent regressions.
> >>>>
> >>>> The following bug entry is on the current list of known regressions
> >>>> from 2.6.29.  Please verify if it still should be listed and let me know
> >>>> (either way).
> >>>>
> >>>>
> >>>> Bug-Entry     : http://bugzilla.kernel.org/show_bug.cgi?id=13319
> >>>> Subject               : Page allocation failures with b43 and p54usb
> >>>> Submitter     : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> >>>> Date          : 2009-04-29 21:01 (40 days old)
> >>>> References    : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> >>>> Handled-By    : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
> >>>
> >>> This bug is extremely difficult to pin down. I cannot reproduce it at
> >>> will. The system has to be up for a long time, which is difficult with
> >>> testing the late RC's of 2.6.30 and the code in wireless-testing so
> >>> that new bugs don't end up in 2.6.31-RCX. That said, it still was in
> >>> 2.6.30-RC6 and I'm not aware of any changes since that would fix it.
> >>>
> >>> My operating kernel is patched with additional diagnostics to help me
> >>> understand why a kmalloc request for a buffer of 1390 bytes suddenly
> >>> ends up as an O(1) request. Unfortunately, I don't have any answers.
> >>
> >> Looking at the out-of-memory trace, there's still memory available but
> >> the pskb_expand_head() allocation is GFP_ATOMIC so there's not much
> >> the page allocator can do here. The amount of memory consumed by
> >> inactive_file is pretty high so maybe the problem is related to the
> >> recent mm/vmscan.c changes. Lets copy some more mm developers and see
> >> if they can help out.
> >
> > That is a very strange trace.  The Mem-Info indicates
> > that the system has more than enough memory free, and
> > also enough memory in higher-order free blocks.
> >
> > This would indicate a bug somewhere in the page
> > allocator - this memory should have been given to this
> > allocation request.
> 
> Aha, I always have difficulties deciphering the traces. But lets
> invite Mel to the party then!
> 

Nothing like a party on Monday morning to get the week started!

What we appear to have is

o Allocation failure is high-order, high-priority, compound and atomic.
o swap is mostly unused, but we cannot enter direct reclaim.
o ZONE_DMA32 can be used
o The allocation path is in the slub allocator
o Are way above the order-0 watermarks so kswapd is probably not awake
o The minimum watermark for an order-0 page was about 647 pages in ZONE_DMA32
o The minimum watermark for an order-1 page was about 323 pages in ZONE_DMA32
o There are 10244 pages free at the time of the failure
o With the order-0 pages taken out for watermark calculation, there are
  182 free pages which is below the watermark of 323 pages for an
  order-1 allocation

While there is enough free memory overall, the zone watermark calculation
takes into account the order of the request. As this is an order-1 allocation,
the free order-0 pages are taken out of consideration and so the allocation
fails.

We've encountered this before and the conclusion was that the current
adjustments for watermark calculations of high-order allocations is right,
or at least there is no better alternative. In other words, the page
allocator in this instance is behaving as expected. Do we want to
revisit that discussion as to whether the watermark calculations for
high-order allocation should change? I think we'll reach the same
conclusion or at least decide that allowing the order-1 atomic
allocation to succeed here would just postpone the problem.

So the question is why are we doing a high-order atomic allocation in
this path? According to an earlier discussion on this problemn

> I think something happened to change the allocation as I never saw these
> O(1) failures before with these particular drivers. I put in a few test
> printk's and the buffers were 700-800 bytes long, and I would not expect
> them to require more than an O(0) allocation.

So, SLUB is deciding to use order-1 pages for the slab allocation.
Ordinarily, it'll get away with that because order-1 pages will be
allocated from a path that can direct reclaim. However, if a slab is
being used for atomic allocations, there is a chance that it's the
atomic request that allocates a new page for the slab.

Larry, can you post the contents of /proc/slabinfo so we can see
what size pages are being used for the kmalloc() buckets please?

Larry, you say the buffer is 700-800 bytes. Can you confirm that 800 bytes
is roughly the request size being made by ieee80211_skb_resize()?

Pekka, assuming the request size is 800 bytes, and SLUB is using order-1
pages for allocations of that size, what happened order-1 allocations
falling back to order-0 allocations as necessary. That logic exists,
right? If so, could it be broken?

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Mon, Jun 8, 2009 at 1:17 PM, Mel Gorman<mel-wPRd99KPJ+uzQB+pC5nmwQ@public.gmane.org> wrote:
> Pekka, assuming the request size is 800 bytes, and SLUB is using order-1
> pages for allocations of that size, what happened order-1 allocations
> falling back to order-0 allocations as necessary. That logic exists,
> right? If so, could it be broken?

That logic is in allocate_slab() and if the higher order allocation
fails, we fall-back to struct kmem_cache ->min order. That in turn is
set up in calculate_sizes() to get_order(size) so it seems pretty
unlikely to me the allocation is 800 bytes. Of course, I could be
missing something here and there's a bug in oo_make() or oo_order().
Hmm.

                        Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Mel Gorman]

On Mon, Jun 08, 2009 at 01:52:05PM +0300, Pekka Enberg wrote:
> On Mon, Jun 8, 2009 at 1:17 PM, Mel Gorman<mel-wPRd99KPJ+uzQB+pC5nmwQ@public.gmane.org> wrote:
> > Pekka, assuming the request size is 800 bytes, and SLUB is using order-1
> > pages for allocations of that size, what happened order-1 allocations
> > falling back to order-0 allocations as necessary. That logic exists,
> > right? If so, could it be broken?
> 
> That logic is in allocate_slab() and if the higher order allocation
> fails, we fall-back to struct kmem_cache ->min order. That in turn is
> set up in calculate_sizes() to get_order(size) so it seems pretty
> unlikely to me the allocation is 800 bytes. Of course, I could be
> missing something here and there's a bug in oo_make() or oo_order().
> Hmm.

Is there any chance you could hatchet together a patch
slab-allocation-failure that reports on slab allocation failures similar
to what the page allocator does? Minimally, it should tell us what
the size of the allocation was but any other information such as the
same of the slab, the size of pages it normally uses are, etc. would
also be useful.

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Rik van Riel]

Mel Gorman wrote:

> We've encountered this before and the conclusion was that the current
> adjustments for watermark calculations of high-order allocations is right,
> or at least there is no better alternative. In other words, the page
> allocator in this instance is behaving as expected. Do we want to
> revisit that discussion as to whether the watermark calculations for
> high-order allocation should change? I think we'll reach the same
> conclusion or at least decide that allowing the order-1 atomic
> allocation to succeed here would just postpone the problem.

It would not just postpone the problem, it would also
bring the system closer to a state where kswapd does
something about the order-1 free areas.

This might postpone the problem indefinately.

Currently the system fails early, without kswapd
kicking in and freeing new order-1 areas.

-- 
All rights reversed.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Mel Gorman wrote:
>
> Larry, can you post the contents of /proc/slabinfo so we can see
> what size pages are being used for the kmalloc() buckets please?

The system is not generating the failures at the moment, but here is
the current state:

finger@larrylap:~/wireless-testing> cat /proc/slabinfo
slabinfo - version: 2.1
# name            <active_objs> <num_objs> <objsize> <objperslab>
<pagesperslab> : tunables <limit> <batchcount> <sharedfactor> :
slabdata <active_slabs> <num_slabs> <sharedavail>
nfs_direct_cache       0      0    288   14    1 : tunables    0    0
   0 : slabdata      0      0      0
nfs_write_data        42     42    768   21    4 : tunables    0    0
   0 : slabdata      2      2      0
nfs_read_data         42     42    768   21    4 : tunables    0    0
   0 : slabdata      2      2      0
nfs_inode_cache       20     20   1568   20    8 : tunables    0    0
   0 : slabdata      1      1      0
nfs_page               0      0    192   21    1 : tunables    0    0
   0 : slabdata      0      0      0
rpc_buffers           30     30   2176   15    8 : tunables    0    0
   0 : slabdata      2      2      0
rpc_tasks             42     42    384   21    2 : tunables    0    0
   0 : slabdata      2      2      0
rpc_inode_cache       23     23   1408   23    8 : tunables    0    0
   0 : slabdata      1      1      0
fuse_request         351    352    720   22    4 : tunables    0    0
   0 : slabdata     16     16      0
fuse_inode           325    325   1216   13    4 : tunables    0    0
   0 : slabdata     25     25      0
ext4_inode_cache   17180  17180   1568   20    8 : tunables    0    0
   0 : slabdata    859    859      0
ext4_xattr             0      0    160   25    1 : tunables    0    0
   0 : slabdata      0      0      0
ext4_free_block_extents      0      0    128   32    1 : tunables    0
   0    0 : slabdata      0      0      0
ext4_alloc_context      0      0    216   18    1 : tunables    0    0
   0 : slabdata      0      0      0
ext4_prealloc_space      0      0    216   18    1 : tunables    0
0    0 : slabdata      0      0      0
jbd2_journal_handle     68     68    120   34    1 : tunables    0
0    0 : slabdata      2      2      0
jbd2_journal_head   3734   3784    184   22    1 : tunables    0    0
   0 : slabdata    172    172      0
jbd2_revoke_table     46     46     88   46    1 : tunables    0    0
   0 : slabdata      1      1      0
jbd2_revoke_record      0      0    128   32    1 : tunables    0    0
   0 : slabdata      0      0      0
kcopyd_job             0      0    528   15    2 : tunables    0    0
   0 : slabdata      0      0      0
dm_rq_clone_bio_info      0      0     88   46    1 : tunables    0
 0    0 : slabdata      0      0      0
dm_rq_target_io        0      0    480   17    2 : tunables    0    0
   0 : slabdata      0      0      0
dm_target_io           0      0     96   42    1 : tunables    0    0
   0 : slabdata      0      0      0
dm_io                  0      0    104   39    1 : tunables    0    0
   0 : slabdata      0      0      0
uhci_urb_priv          0      0    128   32    1 : tunables    0    0
   0 : slabdata      0      0      0
ext3_inode_cache   69341  69345   1408   23    8 : tunables    0    0
   0 : slabdata   3015   3015      0
ext3_xattr           325    325    160   25    1 : tunables    0    0
   0 : slabdata     13     13      0
journal_handle        68     68    120   34    1 : tunables    0    0
   0 : slabdata      2      2      0
journal_head        2473   4642    184   22    1 : tunables    0    0
   0 : slabdata    211    211      0
revoke_table          46     46     88   46    1 : tunables    0    0
   0 : slabdata      1      1      0
revoke_record         64     64    128   32    1 : tunables    0    0
   0 : slabdata      2      2      0
scsi_sense_cache      46     63    192   21    1 : tunables    0    0
   0 : slabdata      3      3      0
scsi_cmd_cache        28     36    320   12    1 : tunables    0    0
   0 : slabdata      3      3      0
sgpool-128            16     21   4224    7    8 : tunables    0    0
   0 : slabdata      3      3      0
sgpool-64             30     30   2176   15    8 : tunables    0    0
   0 : slabdata      2      2      0
sgpool-32             28     28   1152   14    4 : tunables    0    0
   0 : slabdata      2      2      0
sgpool-16             24     24    640   12    2 : tunables    0    0
   0 : slabdata      2      2      0
sgpool-8              44     63    384   21    2 : tunables    0    0
   0 : slabdata      3      3      0
scsi_data_buffer       0      0     96   42    1 : tunables    0    0
   0 : slabdata      0      0      0
flow_cache             0      0    168   24    1 : tunables    0    0
   0 : slabdata      0      0      0
cfq_io_context        93    102    240   17    1 : tunables    0    0
   0 : slabdata      6      6      0
cfq_queue             97    102    240   17    1 : tunables    0    0
   0 : slabdata      6      6      0
mqueue_inode_cache     23     23   1408   23    8 : tunables    0    0
   0 : slabdata      1      1      0
isofs_inode_cache      0      0   1088   15    4 : tunables    0    0
   0 : slabdata      0      0      0
kioctx                 0      0    640   12    2 : tunables    0    0
   0 : slabdata      0      0      0
kiocb                  0      0    320   12    1 : tunables    0    0
   0 : slabdata      0      0      0
inotify_event_cache     72     72    112   36    1 : tunables    0
0    0 : slabdata      2      2      0
inotify_watch_cache    224    224    144   28    1 : tunables    0
0    0 : slabdata      8      8      0
fasync_cache          42     42     96   42    1 : tunables    0    0
   0 : slabdata      1      1      0
shmem_inode_cache   1485   1488   1344   12    4 : tunables    0    0
   0 : slabdata    124    124      0
nsproxy                0      0    120   34    1 : tunables    0    0
   0 : slabdata      0      0      0
posix_timers_cache     26     26    304   13    1 : tunables    0    0
   0 : slabdata      2      2      0
uid_cache             24     24    320   12    1 : tunables    0    0
   0 : slabdata      2      2      0
UNIX                 354    360   1344   12    4 : tunables    0    0
   0 : slabdata     30     30      0
ip_mrt_cache           0      0    192   21    1 : tunables    0    0
   0 : slabdata      0      0      0
UDP-Lite               0      0   1216   13    4 : tunables    0    0
   0 : slabdata      0      0      0
tcp_bind_bucket       64     64    128   32    1 : tunables    0    0
   0 : slabdata      2      2      0
inet_peer_cache       21     21    192   21    1 : tunables    0    0
   0 : slabdata      1      1      0
secpath_cache          0      0    128   32    1 : tunables    0    0
   0 : slabdata      0      0      0
xfrm_dst_cache         0      0    448   18    2 : tunables    0    0
   0 : slabdata      0      0      0
ip_fib_alias           0      0    104   39    1 : tunables    0    0
   0 : slabdata      0      0      0
ip_fib_hash           56     56    144   28    1 : tunables    0    0
   0 : slabdata      2      2      0
ip_dst_cache          36     36    448   18    2 : tunables    0    0
   0 : slabdata      2      2      0
arp_cache             36     36    448   18    2 : tunables    0    0
   0 : slabdata      2      2      0
RAW                   14     14   1152   14    4 : tunables    0    0
   0 : slabdata      1      1      0
UDP                   26     26   1216   13    4 : tunables    0    0
   0 : slabdata      2      2      0
tw_sock_TCP           32     32    256   16    1 : tunables    0    0
   0 : slabdata      2      2      0
request_sock_TCP      21     21    192   21    1 : tunables    0    0
   0 : slabdata      1      1      0
TCP                   34     45   2176   15    8 : tunables    0    0
   0 : slabdata      3      3      0
eventpoll_pwq        110    112    144   28    1 : tunables    0    0
   0 : slabdata      4      4      0
eventpoll_epi         94     96    256   16    1 : tunables    0    0
   0 : slabdata      6      6      0
blkdev_queue          22     22   2736   11    8 : tunables    0    0
   0 : slabdata      2      2      0
blkdev_requests       40     54    440   18    2 : tunables    0    0
   0 : slabdata      3      3      0
blkdev_ioc           101    105    192   21    1 : tunables    0    0
   0 : slabdata      5      5      0
bio-0                 32     32    256   16    1 : tunables    0    0
   0 : slabdata      2      2      0
biovec-256             7      7   4224    7    8 : tunables    0    0
   0 : slabdata      1      1      0
biovec-128            30     30   2176   15    8 : tunables    0    0
   0 : slabdata      2      2      0
biovec-64             28     28   1152   14    4 : tunables    0    0
   0 : slabdata      2      2      0
biovec-16             42     42    384   21    2 : tunables    0    0
   0 : slabdata      2      2      0
sock_inode_cache     397    406   1152   14    4 : tunables    0    0
   0 : slabdata     29     29      0
skbuff_fclone_cache     32     32    512   16    2 : tunables    0
0    0 : slabdata      2      2      0
skbuff_head_cache    593    600    320   12    1 : tunables    0    0
   0 : slabdata     50     50      0
file_lock_cache       39     42    288   14    1 : tunables    0    0
   0 : slabdata      3      3      0
Acpi-Operand        1301   1316    144   28    1 : tunables    0    0
   0 : slabdata     47     47      0
Acpi-ParseExt         56     56    144   28    1 : tunables    0    0
   0 : slabdata      2      2      0
Acpi-Parse            68     68    120   34    1 : tunables    0    0
   0 : slabdata      2      2      0
Acpi-State            52     52    152   26    1 : tunables    0    0
   0 : slabdata      2      2      0
Acpi-Namespace       897    897    104   39    1 : tunables    0    0
   0 : slabdata     23     23      0
task_delay_info      247    255    232   17    1 : tunables    0    0
   0 : slabdata     15     15      0
taskstats             40     40    400   20    2 : tunables    0    0
   0 : slabdata      2      2      0
proc_inode_cache    1709   1725   1088   15    4 : tunables    0    0
   0 : slabdata    115    115      0
sigqueue              34     34    232   17    1 : tunables    0    0
   0 : slabdata      2      2      0
radix_tree_node    22109  22126    624   13    2 : tunables    0    0
   0 : slabdata   1702   1702      0
bdev_cache            42     42   1536   21    8 : tunables    0    0
   0 : slabdata      2      2      0
sysfs_dir_cache    12246  12246    152   26    1 : tunables    0    0
   0 : slabdata    471    471      0
mnt_cache             47     48    320   12    1 : tunables    0    0
   0 : slabdata      4      4      0
filp                2971   3150    384   21    2 : tunables    0    0
   0 : slabdata    150    150      0
inode_cache         3185   3195   1040   15    4 : tunables    0    0
   0 : slabdata    213    213      0
dentry            274295 274300    312   13    1 : tunables    0    0
   0 : slabdata  21100  21100      0
names_cache           14     14   4224    7    8 : tunables    0    0
   0 : slabdata      2      2      0
key_jar                0      0    320   12    1 : tunables    0    0
   0 : slabdata      0      0      0
buffer_head       120232 120244    176   23    1 : tunables    0    0
   0 : slabdata   5228   5228      0
vm_area_struct     10385  10768    248   16    1 : tunables    0    0
   0 : slabdata    673    673      0
mm_struct            111    140   1152   14    4 : tunables    0    0
   0 : slabdata     10     10      0
fs_cache             125    147    192   21    1 : tunables    0    0
   0 : slabdata      7      7      0
files_cache          122    144    896   18    4 : tunables    0    0
   0 : slabdata      8      8      0
signal_cache         164    192   1024   16    4 : tunables    0    0
   0 : slabdata     12     12      0
sighand_cache        161    182   2240   14    8 : tunables    0    0
   0 : slabdata     13     13      0
task_xstate           66     72    640   12    2 : tunables    0    0
   0 : slabdata      6      6      0
task_struct          241    256   3872    8    8 : tunables    0    0
   0 : slabdata     32     32      0
cred_jar             366    560    256   16    1 : tunables    0    0
   0 : slabdata     35     35      0
anon_vma            2206   2310    136   30    1 : tunables    0    0
   0 : slabdata     77     77      0
pid                  257    273    192   21    1 : tunables    0    0
   0 : slabdata     13     13      0
shared_policy_node      0      0    120   34    1 : tunables    0    0
   0 : slabdata      0      0      0
numa_policy           42     42     96   42    1 : tunables    0    0
   0 : slabdata      1      1      0
idr_layer_cache      403    403    616   13    2 : tunables    0    0
   0 : slabdata     31     31      0
kmalloc-8192          28     30   8264    3    8 : tunables    0    0
   0 : slabdata     10     10      0
kmalloc-4096         661    665   4168    7    8 : tunables    0    0
   0 : slabdata     95     95      0
kmalloc-2048         335    360   2120   15    8 : tunables    0    0
   0 : slabdata     24     24      0
kmalloc-1024         479    609   1096   29    8 : tunables    0    0
   0 : slabdata     21     21      0
kmalloc-512          783    784    584   14    2 : tunables    0    0
   0 : slabdata     56     56      0
kmalloc-256          535    552    328   12    1 : tunables    0    0
   0 : slabdata     46     46      0
kmalloc-128          309    360    200   20    1 : tunables    0    0
   0 : slabdata     18     18      0
kmalloc-64          2430   2520    136   30    1 : tunables    0    0
   0 : slabdata     84     84      0
kmalloc-32           656    663    104   39    1 : tunables    0    0
   0 : slabdata     17     17      0
kmalloc-16          2250   2254     88   46    1 : tunables    0    0
   0 : slabdata     49     49      0
kmalloc-8           3619   3621     80   51    1 : tunables    0    0
   0 : slabdata     71     71      0
kmalloc-192         1449   1455    264   15    1 : tunables    0    0
   0 : slabdata     97     97      0
kmalloc-96           726    816    168   24    1 : tunables    0    0
   0 : slabdata     34     34      0
kmem_cache_node        0      0    176   23    1 : tunables    0    0
   0 : slabdata      0      0      0

>
> Larry, you say the buffer is 700-800 bytes. Can you confirm that 800
bytes
> is roughly the request size being made by ieee80211_skb_resize()?

For some of the failures, the size was in the 700-800 range, but the
ones I found in my logs called pskb_expand_head() with skb->data_len
of 1962. For those calls, head_need and tail_need were both 0.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Mel Gorman]

On Mon, Jun 08, 2009 at 09:20:23AM -0400, Rik van Riel wrote:
> Mel Gorman wrote:
>
>> We've encountered this before and the conclusion was that the current
>> adjustments for watermark calculations of high-order allocations is right,
>> or at least there is no better alternative. In other words, the page
>> allocator in this instance is behaving as expected. Do we want to
>> revisit that discussion as to whether the watermark calculations for
>> high-order allocation should change? I think we'll reach the same
>> conclusion or at least decide that allowing the order-1 atomic
>> allocation to succeed here would just postpone the problem.
>
> It would not just postpone the problem, it would also
> bring the system closer to a state where kswapd does
> something about the order-1 free areas.
>
> This might postpone the problem indefinately.
>

How do you figure it does not just postpone the problem? If there are a batch
of order-1 allocations that come in like this, it will eventually deplete
the higher-order pages and then fail because kswapd is not getting woken up.

Minimally, if we were to ignore the watermarks, there would need to be logic
that says

	"If a high-order allocation would fail due to high-order watermarks
	not being met, but the watermarks are ok from an order-0 perspective
	and the high-order page is available, then grant the allocation but
	wake up kswapd as if the order-1 allocation had failed to get the
	high-order watermarks back in shape"

> Currently the system fails early, without kswapd
> kicking in and freeing new order-1 areas.
>

If the allocation was granted, then kswapd will still not kick in.

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka J Enberg]

Hi Mel,

On Mon, 8 Jun 2009, Mel Gorman wrote:
> Is there any chance you could hatchet together a patch
> slab-allocation-failure that reports on slab allocation failures similar
> to what the page allocator does? Minimally, it should tell us what
> the size of the allocation was but any other information such as the
> same of the slab, the size of pages it normally uses are, etc. would
> also be useful.

Would something like this be sufficient? Figuring out the actual _size_ 
passed to kmalloc() is pretty difficult as then we would need to do the 
NULL test in fastpath code or pass the argument deeper in the call-chain.

			Pekka

diff --git a/mm/slub.c b/mm/slub.c
index 65ffda5..b5acf18 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1565,6 +1565,8 @@ new_slab:
 		c->page = new;
 		goto load_freelist;
 	}
+	printk(KERN_WARNING "SLUB: unable to satisfy allocation for cache %s (size=%d, node=%d, gfp=%x)\n",
+		s->name, s->size, node, gfpflags);
 	return NULL;
 debug:
 	if (!alloc_debug_processing(s, c->page, object, addr))

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Mel Gorman]

On Mon, Jun 08, 2009 at 04:58:10PM +0300, Pekka J Enberg wrote:
> Hi Mel,
> 
> On Mon, 8 Jun 2009, Mel Gorman wrote:
> > Is there any chance you could hatchet together a patch
> > slab-allocation-failure that reports on slab allocation failures similar
> > to what the page allocator does? Minimally, it should tell us what
> > the size of the allocation was but any other information such as the
> > same of the slab, the size of pages it normally uses are, etc. would
> > also be useful.
> 
> Would something like this be sufficient? Figuring out the actual _size_ 
> passed to kmalloc() is pretty difficult as then we would need to do the 
> NULL test in fastpath code or pass the argument deeper in the call-chain.
> 

It's much better than nothing. In the event of an allocation failure, we'll
know which kmalloc bucket it's coming out of so we'll have a limited range
of possible buffer sizes.

I have some suggestions on what we're outputting though.

> 			Pekka
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 65ffda5..b5acf18 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -1565,6 +1565,8 @@ new_slab:
>  		c->page = new;
>  		goto load_freelist;
>  	}
> +	printk(KERN_WARNING "SLUB: unable to satisfy allocation for cache %s (size=%d, node=%d, gfp=%x)\n",
> +		s->name, s->size, node, gfpflags);

size could be almost anything here for a casual reader. You are
outputting the size of the object plus its metadata so the name should
reflect that. I think it would be better to output objsize= and the
object size without the metadata overhead. What do you think?

In addition, include how many objects there are per-slab and include what
the order is being passed to the page allocator when allocating new slabs.
Would that be enough to determine if fallback-to-smaller orders occured?

>  	return NULL;
>  debug:
>  	if (!alloc_debug_processing(s, c->page, object, addr))
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Mon, 8 Jun 2009, Mel Gorman wrote:

> In addition, include how many objects there are per-slab and include what
> the order is being passed to the page allocator when allocating new slabs.
> Would that be enough to determine if fallback-to-smaller orders occured?

There is a per slab counter ORDER_FALLBACK that is increased for
allocations that required fallback.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

KAMEZAWA Hiroyuki wrote:
> On Sun, 07 Jun 2009 11:35:27 -0500
> Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org> wrote:
> 
>> Pekka Enberg wrote:
>>> On Sun, Jun 7, 2009 at 5:19 PM, Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>>>> That is a very strange trace.  The Mem-Info indicates
>>>> that the system has more than enough memory free, and
>>>> also enough memory in higher-order free blocks.
>>>>
>>>> This would indicate a bug somewhere in the page
>>>> allocator - this memory should have been given to this
>>>> allocation request.
>>> Aha, I always have difficulties deciphering the traces. But lets
>>> invite Mel to the party then!
>> I'm happy to see some action on this problem. As usual, I'm happy to
>> test patches and/or provide diagnostic output.
>>
> One question. 
> 
> Did your system fragmented in same way as to this
> (see DMA32, 10052 of order-0 pages) in older kernel ? I think you can check
> fragmentation status via /proc/buddyinfo.
> =
> kernel: Node 0 DMA: 3*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
> 1*1024kB 0*2048kB 0*4096kB = 2100kB
> kernel: Node 0 DMA32: 10062*4kB 1*8kB 1*16kB 0*32kB 1*64kB 1*128kB 0*256kB
> 1*512kB 0*1024kB 0*2048kB 0*4096kB = 40976kB
> ==

The current system has not been up very long and does not show the
fragmentation:

finger@larrylap:~/wireless-testing> cat /proc/buddyinfo
Node 0, zone      DMA      4      5      4      2      4      1      2
     0      1      0      0
Node 0, zone    DMA32    261     78     46     55     61     54     37
    17     14     12    262

After I did a git pull and a kernel build with the sources on an
NFS-mounted volume, the fragmentation increased:

Node 0, zone      DMA      4      5      4      2      4      1      2
     0      1      0      0
Node 0, zone    DMA32   2213   1924   1292    705    285     81     25
     8      5      4    141

After a git pull and a kernel build on a second NFS-mounted tree:

Node 0, zone      DMA      4      5      4      2      4      1      2
     0      1      0      0
Node 0, zone    DMA32   3127   3058   1989    756    401    142     56
    14      5      3     12

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi Mel,

On Mon, 2009-06-08 at 15:12 +0100, Mel Gorman wrote:
> > diff --git a/mm/slub.c b/mm/slub.c
> > index 65ffda5..b5acf18 100644
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -1565,6 +1565,8 @@ new_slab:
> >  		c->page = new;
> >  		goto load_freelist;
> >  	}
> > +	printk(KERN_WARNING "SLUB: unable to satisfy allocation for cache %s (size=%d, node=%d, gfp=%x)\n",
> > +		s->name, s->size, node, gfpflags);
> 
> size could be almost anything here for a casual reader. You are
> outputting the size of the object plus its metadata so the name should
> reflect that. I think it would be better to output objsize= and the
> object size without the metadata overhead. What do you think?
> 
> In addition, include how many objects there are per-slab and include what
> the order is being passed to the page allocator when allocating new slabs.
> Would that be enough to determine if fallback-to-smaller orders occured?

So how about something like this then?

			Pekka

diff --git a/mm/slub.c b/mm/slub.c
index 65ffda5..a03dbe8 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1484,6 +1484,58 @@ static inline int node_match(struct kmem_cache_cpu *c, int node)
 	return 1;
 }
 
+static int count_free(struct page *page)
+{
+	return page->objects - page->inuse;
+}
+
+static unsigned long count_partial(struct kmem_cache_node *n,
+					int (*get_count)(struct page *))
+{
+	unsigned long flags;
+	unsigned long x = 0;
+	struct page *page;
+
+	spin_lock_irqsave(&n->list_lock, flags);
+	list_for_each_entry(page, &n->partial, lru)
+		x += get_count(page);
+	spin_unlock_irqrestore(&n->list_lock, flags);
+	return x;
+}
+
+static noinline void
+slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
+{
+	int node;
+
+	printk(KERN_WARNING
+		"SLUB: Unable to allocate memory on node %d (gfp=%x)\n",
+		nid, gfpflags);
+	printk(KERN_WARNING "  cache: %s, object size: %d, buffer size: %d, "
+		"default order: %d, min order: %d\n", s->name, s->objsize,
+		s->size, oo_order(s->oo), oo_order(s->min));
+
+	for_each_online_node(node) {
+		struct kmem_cache_node *n = get_node(s, node);
+		unsigned long nr_partials;
+		unsigned long nr_slabs;
+		unsigned long nr_objs;
+		unsigned long nr_free;
+
+		if (!n)
+			continue;
+
+		nr_partials = n->nr_partial;
+		nr_slabs = atomic_long_read(&n->nr_slabs);
+		nr_objs = atomic_long_read(&n->total_objects);
+		nr_free = count_partial(n, count_free);
+
+		printk(KERN_WARNING
+			"  node %d: partials: %ld, slabs: %ld, objs: %ld, free: %ld\n",
+			node, nr_partials, nr_slabs, nr_objs, nr_free);
+	}
+}
+
 /*
  * Slow path. The lockless freelist is empty or we need to perform
  * debugging duties.
@@ -1565,6 +1617,7 @@ new_slab:
 		c->page = new;
 		goto load_freelist;
 	}
+	slab_out_of_memory(s, gfpflags, node);
 	return NULL;
 debug:
 	if (!alloc_debug_processing(s, c->page, object, addr))
@@ -3318,20 +3371,6 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t gfpflags,
 }
 
 #ifdef CONFIG_SLUB_DEBUG
-static unsigned long count_partial(struct kmem_cache_node *n,
-					int (*get_count)(struct page *))
-{
-	unsigned long flags;
-	unsigned long x = 0;
-	struct page *page;
-
-	spin_lock_irqsave(&n->list_lock, flags);
-	list_for_each_entry(page, &n->partial, lru)
-		x += get_count(page);
-	spin_unlock_irqrestore(&n->list_lock, flags);
-	return x;
-}
-
 static int count_inuse(struct page *page)
 {
 	return page->inuse;
@@ -3342,11 +3381,6 @@ static int count_total(struct page *page)
 	return page->objects;
 }
 
-static int count_free(struct page *page)
-{
-	return page->objects - page->inuse;
-}
-
 static int validate_slab(struct kmem_cache *s, struct page *page,
 						unsigned long *map)
 {

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Mon, Jun 8, 2009 at 5:12 PM, Mel Gorman<mel-wPRd99KPJ+uzQB+pC5nmwQ@public.gmane.org> wrote:
> In addition, include how many objects there are per-slab and include what
> the order is being passed to the page allocator when allocating new slabs.
> Would that be enough to determine if fallback-to-smaller orders occured?

Well, if the slab_out_of_memory() is called, we already know the
higher order allocation failed _and_ the fallback allocation failed.
So yes, it would be enough.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Tue, 9 Jun 2009, Pekka Enberg wrote:

> Hi Mel,
> 
> On Mon, 2009-06-08 at 15:12 +0100, Mel Gorman wrote:
> > > diff --git a/mm/slub.c b/mm/slub.c
> > > index 65ffda5..b5acf18 100644
> > > --- a/mm/slub.c
> > > +++ b/mm/slub.c
> > > @@ -1565,6 +1565,8 @@ new_slab:
> > >  		c->page = new;
> > >  		goto load_freelist;
> > >  	}
> > > +	printk(KERN_WARNING "SLUB: unable to satisfy allocation for cache %s (size=%d, node=%d, gfp=%x)\n",
> > > +		s->name, s->size, node, gfpflags);
> > 
> > size could be almost anything here for a casual reader. You are
> > outputting the size of the object plus its metadata so the name should
> > reflect that. I think it would be better to output objsize= and the
> > object size without the metadata overhead. What do you think?
> > 
> > In addition, include how many objects there are per-slab and include what
> > the order is being passed to the page allocator when allocating new slabs.
> > Would that be enough to determine if fallback-to-smaller orders occured?
> 
> So how about something like this then?
> 

Larry reported this stack trace:

kernel: git: page allocation failure. order:1, mode:0x4020
kernel: Pid: 3707, comm: git Not tainted 2.6.30-rc1-wl #115
kernel: Call Trace:
kernel:  [<ffffffff80292f84>] __alloc_pages_internal+0x43d/0x45d
kernel:  [<ffffffff802b2383>] alloc_pages_current+0xbe/0xc6
kernel:  [<ffffffff802b66a4>] new_slab+0xcf/0x28b

That's in the order fallback for new slab allocations; so this cache must 
have oo_order(s->min) of 1.

To diagnose whether its object size dictates a >0 slab order, you could 
enable CONFIG_SLUB_STATS (it's disabled in his .config) and check which 
/sys/kernel/slab/cache/order_fallback increased.  Once you have identified 
the cache, you can get this information via 
/sys/kernel/slab/cache/{objsize,order,size}.  I think this is what 
Christoph was getting at.

You could even boot with `slub_nomerge' to determine whether cache merging 
was the issue where the cache under consideration was unnecessarily merged 
with one that requires larger higher order minimums.

I don't quite understand how its necessary to print the partial lists for 
each node, they should be exhausted if we're allocating a new slab if the 
node doesn't matter (and can't in Larry's case, he only has one).

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi David,

On Tue, 2009-06-09 at 00:54 -0700, David Rientjes wrote:
> Larry reported this stack trace:
> 
> kernel: git: page allocation failure. order:1, mode:0x4020
> kernel: Pid: 3707, comm: git Not tainted 2.6.30-rc1-wl #115
> kernel: Call Trace:
> kernel:  [<ffffffff80292f84>] __alloc_pages_internal+0x43d/0x45d
> kernel:  [<ffffffff802b2383>] alloc_pages_current+0xbe/0xc6
> kernel:  [<ffffffff802b66a4>] new_slab+0xcf/0x28b
> 
> That's in the order fallback for new slab allocations; so this cache must 
> have oo_order(s->min) of 1.

Yes, agreed which is why I said it's unlikely that the allocated size is
800 bytes or so.

On Tue, 2009-06-09 at 00:54 -0700, David Rientjes wrote:
> To diagnose whether its object size dictates a >0 slab order, you could 
> enable CONFIG_SLUB_STATS (it's disabled in his .config) and check which 
> /sys/kernel/slab/cache/order_fallback increased.  Once you have identified 
> the cache, you can get this information via 
> /sys/kernel/slab/cache/{objsize,order,size}.  I think this is what 
> Christoph was getting at.
> 
> You could even boot with `slub_nomerge' to determine whether cache merging 
> was the issue where the cache under consideration was unnecessarily merged 
> with one that requires larger higher order minimums.

Sure. Applying my diagnostic patch will probably shed some light on the
subject too.

On Tue, 2009-06-09 at 00:54 -0700, David Rientjes wrote:
> I don't quite understand how its necessary to print the partial lists for 
> each node, they should be exhausted if we're allocating a new slab if the 
> node doesn't matter (and can't in Larry's case, he only has one).

It doesn't hurt either, does it? Yes, we expect the partial lists to be
exhausted but it's better to print that out just in case we have a bug
some day somewhere and that condition is not true. This is very
infrequent slow patch code here anyway.

			Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Tue, 9 Jun 2009, Pekka Enberg wrote:

> > To diagnose whether its object size dictates a >0 slab order, you could 
> > enable CONFIG_SLUB_STATS (it's disabled in his .config) and check which 
> > /sys/kernel/slab/cache/order_fallback increased.  Once you have identified 
> > the cache, you can get this information via 
> > /sys/kernel/slab/cache/{objsize,order,size}.  I think this is what 
> > Christoph was getting at.
> > 
> > You could even boot with `slub_nomerge' to determine whether cache merging 
> > was the issue where the cache under consideration was unnecessarily merged 
> > with one that requires larger higher order minimums.
> 
> Sure. Applying my diagnostic patch will probably shed some light on the
> subject too.
> 

I wasn't sure whether you were proposing the patch as an addition to slub 
or just to help with this issue.  I agree it would help in a hopefully 
ratelimited manner for general slab allocation failures and would have 
avoided some of the confusion for this issue from lack of diagnostics.

> > I don't quite understand how its necessary to print the partial lists for 
> > each node, they should be exhausted if we're allocating a new slab if the 
> > node doesn't matter (and can't in Larry's case, he only has one).
> 
> It doesn't hurt either, does it? Yes, we expect the partial lists to be
> exhausted but it's better to print that out just in case we have a bug
> some day somewhere and that condition is not true. This is very
> infrequent slow patch code here anyway.
> 

It will lead to false postiives since you can get a free to a full slab 
which moves it back to an allowed node's partial list before count_free() 
is printed.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi David,

On Tue, 2009-06-09 at 01:14 -0700, David Rientjes wrote:
> I wasn't sure whether you were proposing the patch as an addition to slub 
> or just to help with this issue.  I agree it would help in a hopefully 
> ratelimited manner for general slab allocation failures and would have 
> avoided some of the confusion for this issue from lack of diagnostics.

I am proposing it as a generic addition to SLUB.

On Tue, 2009-06-09 at 01:14 -0700, David Rientjes wrote:
> > It doesn't hurt either, does it? Yes, we expect the partial lists to be
> > exhausted but it's better to print that out just in case we have a bug
> > some day somewhere and that condition is not true. This is very
> > infrequent slow patch code here anyway.
> 
> It will lead to false postiives since you can get a free to a full slab 
> which moves it back to an allowed node's partial list before count_free() 
> is printed.

Fair enough, lets drop it then!

			Pekka

diff --git a/mm/slub.c b/mm/slub.c
index 65ffda5..2bbacfc 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1484,6 +1484,56 @@ static inline int node_match(struct kmem_cache_cpu *c, int node)
 	return 1;
 }
 
+static int count_free(struct page *page)
+{
+	return page->objects - page->inuse;
+}
+
+static unsigned long count_partial(struct kmem_cache_node *n,
+					int (*get_count)(struct page *))
+{
+	unsigned long flags;
+	unsigned long x = 0;
+	struct page *page;
+
+	spin_lock_irqsave(&n->list_lock, flags);
+	list_for_each_entry(page, &n->partial, lru)
+		x += get_count(page);
+	spin_unlock_irqrestore(&n->list_lock, flags);
+	return x;
+}
+
+static noinline void
+slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
+{
+	int node;
+
+	printk(KERN_WARNING
+		"SLUB: Unable to allocate memory on node %d (gfp=%x)\n",
+		nid, gfpflags);
+	printk(KERN_WARNING "  cache: %s, object size: %d, buffer size: %d, "
+		"default order: %d, min order: %d\n", s->name, s->objsize,
+		s->size, oo_order(s->oo), oo_order(s->min));
+
+	for_each_online_node(node) {
+		struct kmem_cache_node *n = get_node(s, node);
+		unsigned long nr_slabs;
+		unsigned long nr_objs;
+		unsigned long nr_free;
+
+		if (!n)
+			continue;
+
+		nr_slabs = atomic_long_read(&n->nr_slabs);
+		nr_objs = atomic_long_read(&n->total_objects);
+		nr_free = count_partial(n, count_free);
+
+		printk(KERN_WARNING
+			"  node %d: slabs: %ld, objs: %ld, free: %ld\n",
+			node, nr_slabs, nr_objs, nr_free);
+	}
+}
+
 /*
  * Slow path. The lockless freelist is empty or we need to perform
  * debugging duties.
@@ -1565,6 +1615,7 @@ new_slab:
 		c->page = new;
 		goto load_freelist;
 	}
+	slab_out_of_memory(s, gfpflags, node);
 	return NULL;
 debug:
 	if (!alloc_debug_processing(s, c->page, object, addr))
@@ -3318,20 +3369,6 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t gfpflags,
 }
 
 #ifdef CONFIG_SLUB_DEBUG
-static unsigned long count_partial(struct kmem_cache_node *n,
-					int (*get_count)(struct page *))
-{
-	unsigned long flags;
-	unsigned long x = 0;
-	struct page *page;
-
-	spin_lock_irqsave(&n->list_lock, flags);
-	list_for_each_entry(page, &n->partial, lru)
-		x += get_count(page);
-	spin_unlock_irqrestore(&n->list_lock, flags);
-	return x;
-}
-
 static int count_inuse(struct page *page)
 {
 	return page->inuse;
@@ -3342,11 +3379,6 @@ static int count_total(struct page *page)
 	return page->objects;
 }
 
-static int count_free(struct page *page)
-{
-	return page->objects - page->inuse;
-}
-
 static int validate_slab(struct kmem_cache *s, struct page *page,
 						unsigned long *map)
 {

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Pekka Enberg wrote:
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 65ffda5..2bbacfc 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -1484,6 +1484,56 @@ static inline int node_match(struct kmem_cache_cpu *c, int node)
>  	return 1;
>  }
>  
> +static int count_free(struct page *page)
> +{
> +	return page->objects - page->inuse;
> +}
> +
> +static unsigned long count_partial(struct kmem_cache_node *n,
> +					int (*get_count)(struct page *))
> +{
> +	unsigned long flags;
> +	unsigned long x = 0;
> +	struct page *page;
> +
> +	spin_lock_irqsave(&n->list_lock, flags);
> +	list_for_each_entry(page, &n->partial, lru)
> +		x += get_count(page);
> +	spin_unlock_irqrestore(&n->list_lock, flags);
> +	return x;
> +}
> +
> +static noinline void
> +slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
> +{
> +	int node;
> +
> +	printk(KERN_WARNING
> +		"SLUB: Unable to allocate memory on node %d (gfp=%x)\n",
> +		nid, gfpflags);
> +	printk(KERN_WARNING "  cache: %s, object size: %d, buffer size: %d, "
> +		"default order: %d, min order: %d\n", s->name, s->objsize,
> +		s->size, oo_order(s->oo), oo_order(s->min));
> +
> +	for_each_online_node(node) {
> +		struct kmem_cache_node *n = get_node(s, node);
> +		unsigned long nr_slabs;
> +		unsigned long nr_objs;
> +		unsigned long nr_free;
> +
> +		if (!n)
> +			continue;
> +
> +		nr_slabs = atomic_long_read(&n->nr_slabs);
> +		nr_objs = atomic_long_read(&n->total_objects);
> +		nr_free = count_partial(n, count_free);
> +
> +		printk(KERN_WARNING
> +			"  node %d: slabs: %ld, objs: %ld, free: %ld\n",
> +			node, nr_slabs, nr_objs, nr_free);
> +	}
> +}
> +
>  /*
>   * Slow path. The lockless freelist is empty or we need to perform
>   * debugging duties.
> @@ -1565,6 +1615,7 @@ new_slab:
>  		c->page = new;
>  		goto load_freelist;
>  	}
> +	slab_out_of_memory(s, gfpflags, node);
>  	return NULL;
>  debug:
>  	if (!alloc_debug_processing(s, c->page, object, addr))
> @@ -3318,20 +3369,6 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t gfpflags,
>  }
>  
>  #ifdef CONFIG_SLUB_DEBUG
> -static unsigned long count_partial(struct kmem_cache_node *n,
> -					int (*get_count)(struct page *))
> -{
> -	unsigned long flags;
> -	unsigned long x = 0;
> -	struct page *page;
> -
> -	spin_lock_irqsave(&n->list_lock, flags);
> -	list_for_each_entry(page, &n->partial, lru)
> -		x += get_count(page);
> -	spin_unlock_irqrestore(&n->list_lock, flags);
> -	return x;
> -}
> -
>  static int count_inuse(struct page *page)
>  {
>  	return page->inuse;
> @@ -3342,11 +3379,6 @@ static int count_total(struct page *page)
>  	return page->objects;
>  }
>  
> -static int count_free(struct page *page)
> -{
> -	return page->objects - page->inuse;
> -}
> -
>  static int validate_slab(struct kmem_cache *s, struct page *page,
>  						unsigned long *map)
>  {

With the above patch installed, I pushed my system hard enough to get
the O(1) allocation failures. This time they were triggered with a
'make -j8' on the kernel. No, I don't have that many CPUs, but I
figured that the extra make jobs might stress memory. My kernel is
2.6.30-rc8 from the wireless-testing tree. Everything matches Linus's
tree except drivers/net/wireless/, which contains what is essentially
2.6.31 code.

The dmesg output starting with the first allocation failure is:

cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291c0f>] ieee80211_master_start_xmit+0x298/0x319 [mac80211]
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffffa0291957>] ieee80211_subif_start_xmit+0x54b/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382252 pages shared
441407 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
phy0: failed to reallocate TX buffer
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382252 pages shared
441407 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382252 pages shared
441407 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382252 pages shared
441407 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382252 pages shared
441407 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382252 pages shared
441407 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382252 pages shared
441407 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422332kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382238 pages shared
441414 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355620 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422480kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382238 pages shared
441414 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b4038>] ? add_partial+0x1a/0x69
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  15
CPU    1: hi:  186, btch:  31 usd:  65
Active_anon:128724 active_file:123018 inactive_anon:47276
 inactive_file:355620 unevictable:8 dirty:18 writeback:0 unstable:0
 free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
active_anon:514896kB inactive_anon:189104kB active_file:492072kB
inactive_file:1422480kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
479694 total pagecache pages
969 pages in swap cache
Swap cache stats: add 4523, delete 3554, find 2913/3063
Free swap  = 2091884kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
382238 pages shared
441414 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
__ratelimit: 23 callbacks suppressed
rpciod/0: page allocation failure. order:1, mode:0x4020
Pid: 3085, comm: rpciod/0 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291815>] ieee80211_subif_start_xmit+0x409/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff8023fc6b>] ? local_bh_enable+0xc9/0xcf
 [<ffffffff80422e9d>] tcp_push_one+0x2f/0x31
 [<ffffffff80417439>] tcp_sendmsg+0x7a1/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff803e0f6e>] ? sock_sendmsg+0xdf/0xf8
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffff803e39b3>] sock_no_sendpage+0x9b/0xaa
 [<ffffffff804176de>] tcp_sendpage+0x48/0x5ec
 [<ffffffffa054c525>] xs_sendpages+0x12c/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bf9c3>] ? nfs3_xdr_writeargs+0x0/0x87 [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054faa1>] rpc_async_schedule+0x10/0x12 [sunrpc]
 [<ffffffff8024af33>] worker_thread+0x1fa/0x30a
 [<ffffffff8024aedc>] ? worker_thread+0x1a3/0x30a
 [<ffffffffa054fa91>] ? rpc_async_schedule+0x0/0x12 [sunrpc]
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff8024ad39>] ? worker_thread+0x0/0x30a
 [<ffffffff8024ad39>] ? worker_thread+0x0/0x30a
 [<ffffffff8024ec21>] kthread+0x56/0x83
 [<ffffffff8020cb7a>] child_rip+0xa/0x20
 [<ffffffff8020c57c>] ? restore_args+0x0/0x30
 [<ffffffff8024ebcb>] ? kthread+0x0/0x83
 [<ffffffff8020cb70>] ? child_rip+0x0/0x20
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd: 154
CPU    1: hi:  186, btch:  31 usd: 173
Active_anon:147694 active_file:116688 inactive_anon:47252
 inactive_file:344419 unevictable:8 dirty:5 writeback:0 unstable:0
 free:2692 slab:76878 mapped:19321 pagetables:4204 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:8664kB min:6904kB low:8628kB high:10356kB
active_anon:590776kB inactive_anon:189008kB active_file:466752kB
inactive_file:1377660kB unevictable:32kB present:2997292kB
pages_scanned:70 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 1898*4kB 12*8kB 8*16kB 1*32kB 0*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 8744kB
462221 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4555, delete 3589, find 2917/3067
Free swap  = 2091764kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
373616 pages shared
454599 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
cc1: page allocation failure. order:1, mode:0x4020
Pid: 8867, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291c0f>] ieee80211_master_start_xmit+0x298/0x319 [mac80211]
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffffa0291957>] ieee80211_subif_start_xmit+0x54b/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff803e33a8>] ? release_sock+0xcd/0xd6
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff80422e9d>] tcp_push_one+0x2f/0x31
 [<ffffffff80417439>] tcp_sendmsg+0x7a1/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff803e0f6e>] ? sock_sendmsg+0xdf/0xf8
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffff803e39b3>] sock_no_sendpage+0x9b/0xaa
 [<ffffffff804176de>] tcp_sendpage+0x48/0x5ec
 [<ffffffffa054c525>] xs_sendpages+0x12c/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bf9c3>] ? nfs3_xdr_writeargs+0x0/0x87 [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
kswapd0: page allocation failure. order:1, mode:0x4020
Pid: 229, comm: kswapd0 Not tainted 2.6.30-rc8-wl #164
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa05bb774>] nfs_write_rpcsetup+0x215/0x237 [nfs]
Call Trace:
 [<ffffffffa05bd257>] nfs_flush_one+0xa2/0xd9 [nfs]
 [<ffffffffa05b82d9>] nfs_pageio_doio+0x32/0x5b [nfs]
 [<ffffffffa05b83ec>] nfs_pageio_complete+0x9/0xb [nfs]
 [<ffffffffa05bbeae>] nfs_writepages+0x101/0x13a [nfs]
 [<ffffffffa05bd1b5>] ? nfs_flush_one+0x0/0xd9 [nfs]
 [<ffffffffa05bd043>] nfs_write_mapping+0x63/0x9e [nfs]
 [<ffffffffa05bd0a7>] nfs_wb_all+0x12/0x14 [nfs]
 [<ffffffffa05b0145>] nfs_file_flush+0x8a/0xb1 [nfs]
 [<ffffffff802bb18d>] filp_close+0x40/0x63
 [<ffffffff802bb255>] sys_close+0xa5/0xe4
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
 <IRQ> Node 0  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  85
Active_anon:151538 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606152kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB  [<ffffffff802b6362>] new_slab+0xcf/0x28b
5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB 1*1024kB 0*2048kB
0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB  [<ffffffff802b4d1f>] ?
unfreeze_slab+0x4c/0xbd
3*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB
0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa01aaee5>] ? ssb_pci_read32+0x46/0x54 [ssb]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff8023f988>] irq_exit+0x4e/0x88
 [<ffffffff8020de2d>] do_IRQ+0xac/0xc3
 [<ffffffff8020c4d3>] ret_from_intr+0x0/0xf
 <EOI>  [<ffffffff8046013e>] ? _spin_unlock_irq+0x2d/0x30
 [<ffffffff80296a35>] ? __remove_mapping+0xac/0xc6
 [<ffffffff802971b3>] ? shrink_page_list+0x558/0x69f
 [<ffffffff80296262>] ? isolate_pages_global+0x179/0x219
 [<ffffffff8046013c>] ? _spin_unlock_irq+0x2b/0x30
 [<ffffffff8025ce77>] ? trace_hardirqs_on_caller+0x10b/0x12f
 [<ffffffff80297937>] ? shrink_list+0x2a1/0x5b6
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffff80297ed7>] ? shrink_zone+0x28b/0x335
 [<ffffffff8033a0d4>] ? __up_read+0x92/0x9a
 [<ffffffff802980c3>] ? shrink_slab+0x142/0x154
 [<ffffffff80298837>] ? kswapd+0x4b1/0x692
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff802960e9>] ? isolate_pages_global+0x0/0x219
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff8024ec21>] ? kthread+0x56/0x83
 [<ffffffff8020cb7a>] ? child_rip+0xa/0x20
 [<ffffffff8020c57c>] ? restore_args+0x0/0x30
 [<ffffffff8024ebcb>] ? kthread+0x0/0x83
 [<ffffffff8020cb70>] ? child_rip+0x0/0x20
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  85
Active_anon:151538 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606152kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
769872 pages RAM
769872 pages RAM
21377 pages reserved
371529 pages shared
456955 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
21377 pages reserved
  node 0: slabs: 96, objs: 672, free: 0
371529 pages shared
456955 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
phy0: failed to reallocate TX buffer
kswapd0: page allocation failure. order:1, mode:0x4020
Pid: 229, comm: kswapd0 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff8025ce5d>] ? trace_hardirqs_on_caller+0xf1/0x12f
 [<ffffffffa01aaee5>] ? ssb_pci_read32+0x46/0x54 [ssb]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff80243f1f>] ? run_timer_softirq+0x259/0x268
 [<ffffffff80243dfc>] ? run_timer_softirq+0x136/0x268
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff8023f988>] irq_exit+0x4e/0x88
 [<ffffffff8020de2d>] do_IRQ+0xac/0xc3
 [<ffffffff8020c4d3>] ret_from_intr+0x0/0xf
 <EOI>  [<ffffffff8046013e>] ? _spin_unlock_irq+0x2d/0x30
 [<ffffffff80296a35>] ? __remove_mapping+0xac/0xc6
 [<ffffffff802971b3>] ? shrink_page_list+0x558/0x69f
 [<ffffffff80296262>] ? isolate_pages_global+0x179/0x219
 [<ffffffff8046013c>] ? _spin_unlock_irq+0x2b/0x30
 [<ffffffff8025ce77>] ? trace_hardirqs_on_caller+0x10b/0x12f
 [<ffffffff80297937>] ? shrink_list+0x2a1/0x5b6
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffff80297ed7>] ? shrink_zone+0x28b/0x335
 [<ffffffff8033a0d4>] ? __up_read+0x92/0x9a
 [<ffffffff802980c3>] ? shrink_slab+0x142/0x154
 [<ffffffff80298837>] ? kswapd+0x4b1/0x692
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff802960e9>] ? isolate_pages_global+0x0/0x219
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff8024ec21>] ? kthread+0x56/0x83
 [<ffffffff8020cb7a>] ? child_rip+0xa/0x20
 [<ffffffff8020c57c>] ? restore_args+0x0/0x30
 [<ffffffff8024ebcb>] ? kthread+0x0/0x83
 [<ffffffff8020cb70>] ? child_rip+0x0/0x20
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  84
Active_anon:151538 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606152kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
cc1: page allocation failure. order:1, mode:0x4020
Pid: 8867, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291c0f>] ieee80211_master_start_xmit+0x298/0x319 [mac80211]
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffffa0291957>] ieee80211_subif_start_xmit+0x54b/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff80422e9d>] tcp_push_one+0x2f/0x31
 [<ffffffff80417439>] tcp_sendmsg+0x7a1/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff803e0f6e>] ? sock_sendmsg+0xdf/0xf8
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffff803e39b3>] sock_no_sendpage+0x9b/0xaa
 [<ffffffff804176de>] tcp_sendpage+0x48/0x5ec
 [<ffffffffa054c525>] xs_sendpages+0x12c/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bf9c3>] ? nfs3_xdr_writeargs+0x0/0x87 [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa05bb774>] nfs_write_rpcsetup+0x215/0x237 [nfs]
 [<ffffffffa05bd257>] nfs_flush_one+0xa2/0xd9 [nfs]
 [<ffffffffa05b82d9>] nfs_pageio_doio+0x32/0x5b [nfs]
 [<ffffffffa05b83ec>] nfs_pageio_complete+0x9/0xb [nfs]
 [<ffffffffa05bbeae>] nfs_writepages+0x101/0x13a [nfs]
 [<ffffffffa05bd1b5>] ? nfs_flush_one+0x0/0xd9 [nfs]
 [<ffffffffa05bd043>] nfs_write_mapping+0x63/0x9e [nfs]
 [<ffffffffa05bd0a7>] nfs_wb_all+0x12/0x14 [nfs]
 [<ffffffffa05b0145>] nfs_file_flush+0x8a/0xb1 [nfs]
 [<ffffffff802bb18d>] filp_close+0x40/0x63
 [<ffffffff802bb255>] sys_close+0xa5/0xe4
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  51
Active_anon:151575 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606300kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
371534 pages shared
456983 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
kswapd0: page allocation failure. order:1, mode:0x4020
Pid: 229, comm: kswapd0 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa01aaee5>] ? ssb_pci_read32+0x46/0x54 [ssb]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff80243f1f>] ? run_timer_softirq+0x259/0x268
 [<ffffffff80243dfc>] ? run_timer_softirq+0x136/0x268
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff8023f988>] irq_exit+0x4e/0x88
 [<ffffffff8020de2d>] do_IRQ+0xac/0xc3
 [<ffffffff8020c4d3>] ret_from_intr+0x0/0xf
 <EOI>  [<ffffffff8046013e>] ? _spin_unlock_irq+0x2d/0x30
 [<ffffffff80296a35>] ? __remove_mapping+0xac/0xc6
 [<ffffffff802971b3>] ? shrink_page_list+0x558/0x69f
 [<ffffffff80296262>] ? isolate_pages_global+0x179/0x219
 [<ffffffff8046013c>] ? _spin_unlock_irq+0x2b/0x30
 [<ffffffff8025ce77>] ? trace_hardirqs_on_caller+0x10b/0x12f
 [<ffffffff80297937>] ? shrink_list+0x2a1/0x5b6
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffff80297ed7>] ? shrink_zone+0x28b/0x335
 [<ffffffff8033a0d4>] ? __up_read+0x92/0x9a
 [<ffffffff802980c3>] ? shrink_slab+0x142/0x154
 [<ffffffff80298837>] ? kswapd+0x4b1/0x692
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff802960e9>] ? isolate_pages_global+0x0/0x219
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff8024ec21>] ? kthread+0x56/0x83
 [<ffffffff8020cb7a>] ? child_rip+0xa/0x20
 [<ffffffff8020c57c>] ? restore_args+0x0/0x30
 [<ffffffff8024ebcb>] ? kthread+0x0/0x83
 [<ffffffff8020cb70>] ? child_rip+0x0/0x20
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  51
Active_anon:151575 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606300kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
371535 pages shared
456983 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
cc1: page allocation failure. order:1, mode:0x4020
Pid: 8867, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291815>] ieee80211_subif_start_xmit+0x409/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b40a2>] ? get_partial_node+0x1b/0x8a
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff803e33a8>] ? release_sock+0xcd/0xd6
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e3497>] ? lock_sock_nested+0xe6/0xf5
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff803e11f7>] ? kernel_sendmsg+0x34/0x49
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c583>] xs_sendpages+0x18a/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bf9c3>] ? nfs3_xdr_writeargs+0x0/0x87 [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa05bb774>] nfs_write_rpcsetup+0x215/0x237 [nfs]
 [<ffffffffa05bd257>] nfs_flush_one+0xa2/0xd9 [nfs]
 [<ffffffffa05b82d9>] nfs_pageio_doio+0x32/0x5b [nfs]
 [<ffffffffa05b83ec>] nfs_pageio_complete+0x9/0xb [nfs]
 [<ffffffffa05bbeae>] nfs_writepages+0x101/0x13a [nfs]
 [<ffffffffa05bd1b5>] ? nfs_flush_one+0x0/0xd9 [nfs]
 [<ffffffffa05bd043>] nfs_write_mapping+0x63/0x9e [nfs]
 [<ffffffffa05bd0a7>] nfs_wb_all+0x12/0x14 [nfs]
 [<ffffffffa05b0145>] nfs_file_flush+0x8a/0xb1 [nfs]
 [<ffffffff802bb18d>] filp_close+0x40/0x63
 [<ffffffff802bb255>] sys_close+0xa5/0xe4
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  51
Active_anon:151575 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606300kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
371535 pages shared
456983 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
kswapd0: page allocation failure. order:1, mode:0x4020
Pid: 229, comm: kswapd0 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa01aaee5>] ? ssb_pci_read32+0x46/0x54 [ssb]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff80243f1f>] ? run_timer_softirq+0x259/0x268
 [<ffffffff80243dfc>] ? run_timer_softirq+0x136/0x268
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff8023f988>] irq_exit+0x4e/0x88
 [<ffffffff8020de2d>] do_IRQ+0xac/0xc3
 [<ffffffff8020c4d3>] ret_from_intr+0x0/0xf
 <EOI>  [<ffffffff8046013e>] ? _spin_unlock_irq+0x2d/0x30
 [<ffffffff80296a35>] ? __remove_mapping+0xac/0xc6
 [<ffffffff802971b3>] ? shrink_page_list+0x558/0x69f
 [<ffffffff80296262>] ? isolate_pages_global+0x179/0x219
 [<ffffffff8046013c>] ? _spin_unlock_irq+0x2b/0x30
 [<ffffffff8025ce77>] ? trace_hardirqs_on_caller+0x10b/0x12f
 [<ffffffff80297937>] ? shrink_list+0x2a1/0x5b6
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffff80297ed7>] ? shrink_zone+0x28b/0x335
 [<ffffffff8033a0d4>] ? __up_read+0x92/0x9a
 [<ffffffff802980c3>] ? shrink_slab+0x142/0x154
 [<ffffffff80298837>] ? kswapd+0x4b1/0x692
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff802960e9>] ? isolate_pages_global+0x0/0x219
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff8024ec21>] ? kthread+0x56/0x83
 [<ffffffff8020cb7a>] ? child_rip+0xa/0x20
 [<ffffffff8020c57c>] ? restore_args+0x0/0x30
 [<ffffffff8024ebcb>] ? kthread+0x0/0x83
 [<ffffffff8020cb70>] ? child_rip+0x0/0x20
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  51
Active_anon:151575 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606300kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
371535 pages shared
456983 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
cc1: page allocation failure. order:1, mode:0x4020
Pid: 8867, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291815>] ieee80211_subif_start_xmit+0x409/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803ed179>] net_tx_action+0xd9/0x156
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 <EOI>  [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff803efc0e>] ? dev_queue_xmit+0x352/0x384
 [<ffffffff8023fc57>] local_bh_enable+0xb5/0xcf
 [<ffffffff803efc0e>] dev_queue_xmit+0x352/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff802b40a2>] ? get_partial_node+0x1b/0x8a
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff803e33a8>] ? release_sock+0xcd/0xd6
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e3497>] ? lock_sock_nested+0xe6/0xf5
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff803e11f7>] ? kernel_sendmsg+0x34/0x49
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c583>] xs_sendpages+0x18a/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bf9c3>] ? nfs3_xdr_writeargs+0x0/0x87 [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa05bb774>] nfs_write_rpcsetup+0x215/0x237 [nfs]
 [<ffffffffa05bd257>] nfs_flush_one+0xa2/0xd9 [nfs]
 [<ffffffffa05b82d9>] nfs_pageio_doio+0x32/0x5b [nfs]
 [<ffffffffa05b83ec>] nfs_pageio_complete+0x9/0xb [nfs]
 [<ffffffffa05bbeae>] nfs_writepages+0x101/0x13a [nfs]
 [<ffffffffa05bd1b5>] ? nfs_flush_one+0x0/0xd9 [nfs]
 [<ffffffffa05bd043>] nfs_write_mapping+0x63/0x9e [nfs]
 [<ffffffffa05bd0a7>] nfs_wb_all+0x12/0x14 [nfs]
 [<ffffffffa05b0145>] nfs_file_flush+0x8a/0xb1 [nfs]
 [<ffffffff802bb18d>] filp_close+0x40/0x63
 [<ffffffff802bb255>] sys_close+0xa5/0xe4
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  51
Active_anon:151575 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606300kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
371535 pages shared
456983 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
kswapd0: page allocation failure. order:1, mode:0x4020
Pid: 229, comm: kswapd0 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 <IRQ>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff802b78f5>] __kmalloc_node_track_caller+0xbd/0x144
 [<ffffffffa02d131d>] ? setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffff803e872d>] __alloc_skb+0x6f/0x143
 [<ffffffffa02d131d>] setup_rx_descbuffer+0x4b/0x2d7 [b43]
 [<ffffffffa01aaee5>] ? ssb_pci_read32+0x46/0x54 [ssb]
 [<ffffffffa02d192e>] b43_dma_rx+0x319/0x4ff [b43]
 [<ffffffffa02c55d3>] b43_interrupt_tasklet+0x699/0x7fe [b43]
 [<ffffffff80243f1f>] ? run_timer_softirq+0x259/0x268
 [<ffffffff80243dfc>] ? run_timer_softirq+0x136/0x268
 [<ffffffff8023f684>] ? tasklet_action+0x44/0xdb
 [<ffffffff8023f6c0>] tasklet_action+0x80/0xdb
 [<ffffffff8023fdc7>] __do_softirq+0xb1/0x186
 [<ffffffff8020cc7c>] call_softirq+0x1c/0x28
 [<ffffffff8020e54d>] do_softirq+0x39/0x8a
 [<ffffffff8023f988>] irq_exit+0x4e/0x88
 [<ffffffff8020de2d>] do_IRQ+0xac/0xc3
 [<ffffffff8020c4d3>] ret_from_intr+0x0/0xf
 <EOI>  [<ffffffff8046013e>] ? _spin_unlock_irq+0x2d/0x30
 [<ffffffff80296a35>] ? __remove_mapping+0xac/0xc6
 [<ffffffff802971b3>] ? shrink_page_list+0x558/0x69f
 [<ffffffff80296262>] ? isolate_pages_global+0x179/0x219
 [<ffffffff8046013c>] ? _spin_unlock_irq+0x2b/0x30
 [<ffffffff8025ce77>] ? trace_hardirqs_on_caller+0x10b/0x12f
 [<ffffffff80297937>] ? shrink_list+0x2a1/0x5b6
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffff80297ed7>] ? shrink_zone+0x28b/0x335
 [<ffffffff8033a0d4>] ? __up_read+0x92/0x9a
 [<ffffffff802980c3>] ? shrink_slab+0x142/0x154
 [<ffffffff80298837>] ? kswapd+0x4b1/0x692
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff802960e9>] ? isolate_pages_global+0x0/0x219
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff80298386>] ? kswapd+0x0/0x692
 [<ffffffff8024ec21>] ? kthread+0x56/0x83
 [<ffffffff8020cb7a>] ? child_rip+0xa/0x20
 [<ffffffff8020c57c>] ? restore_args+0x0/0x30
 [<ffffffff8024ebcb>] ? kthread+0x0/0x83
 [<ffffffff8020cb70>] ? child_rip+0x0/0x20
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  89
CPU    1: hi:  186, btch:  31 usd:  51
Active_anon:151575 active_file:114269 inactive_anon:47211
 inactive_file:340887 unevictable:8 dirty:36 writeback:0 unstable:2
 free:5246 slab:76536 mapped:19364 pagetables:4251 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:18880kB min:6904kB low:8628kB high:10356kB
active_anon:606300kB inactive_anon:188844kB active_file:457076kB
inactive_file:1363548kB unevictable:32kB present:2997292kB
pages_scanned:69 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 4459*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 18852kB
456290 total pagecache pages
966 pages in swap cache
Swap cache stats: add 4587, delete 3621, find 2934/3084
Free swap  = 2091636kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
371535 pages shared
456983 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
protocol 0008 is buggy, dev eth1
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
769872 pages RAM
21377 pages reserved
371535 pages shared
456983 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 96, objs: 672, free: 0
b43-phy0 debug: DMA RX: setup_rx_descbuffer() failed
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
phy0: failed to reallocate TX buffer
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
phy0: failed to reallocate TX buffer
__ratelimit: 73 callbacks suppressed
cc1: page allocation failure. order:1, mode:0x4020
Pid: 9042, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291815>] ieee80211_subif_start_xmit+0x409/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff802b40a2>] ? get_partial_node+0x1b/0x8a
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  13
CPU    1: hi:  186, btch:  31 usd: 173
Active_anon:163559 active_file:111927 inactive_anon:47119
 inactive_file:334673 unevictable:8 dirty:23 writeback:0 unstable:0
 free:2704 slab:75670 mapped:19336 pagetables:4281 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:8712kB min:6904kB low:8628kB high:10356kB
active_anon:654236kB inactive_anon:188476kB active_file:447708kB
inactive_file:1338692kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 1910*4kB 6*8kB 6*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 8744kB
447641 total pagecache pages
962 pages in swap cache
Swap cache stats: add 4651, delete 3689, find 2934/3084
Free swap  = 2091380kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
365043 pages shared
466540 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 97, objs: 679, free: 0
phy0: failed to reallocate TX buffer
cc1: page allocation failure. order:1, mode:0x4020
Pid: 10081, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291c0f>] ieee80211_master_start_xmit+0x298/0x319 [mac80211]
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffffa0291957>] ieee80211_subif_start_xmit+0x54b/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
 [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
 [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
 [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
 [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
 [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
 [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
 [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
 [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
 [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
 [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
 [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
 [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
 [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
 [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
 [<ffffffff802c6d1d>] path_walk+0x69/0xd4
 [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
 [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
 [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
 [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
 [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
 [<ffffffff802bb393>] sys_open+0x1b/0x1d
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  60
CPU    1: hi:  186, btch:  31 usd: 132
Active_anon:162603 active_file:111766 inactive_anon:47119
 inactive_file:332454 unevictable:8 dirty:11 writeback:0 unstable:0
 free:6493 slab:75317 mapped:19281 pagetables:4242 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:23868kB min:6904kB low:8628kB high:10356kB
active_anon:650412kB inactive_anon:188476kB active_file:447064kB
inactive_file:1329816kB unevictable:32kB present:2997292kB
pages_scanned:154 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 5688*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 23768kB
445272 total pagecache pages
946 pages in swap cache
Swap cache stats: add 4659, delete 3713, find 2934/3084
Free swap  = 2091348kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
357698 pages shared
466721 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 0
phy0: failed to reallocate TX buffer
cc1: page allocation failure. order:1, mode:0x4020
Pid: 10064, comm: cc1 Not tainted 2.6.30-rc8-wl #164
Call Trace:
 [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
 [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
 [<ffffffff802b6362>] new_slab+0xcf/0x28b
 [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
 [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
 [<ffffffff802b7e60>] __kmalloc+0x119/0x194
 [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
 [<ffffffff80460180>] ? _spin_unlock_irqrestore+0x3f/0x47
 [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
 [<ffffffffa0291815>] ieee80211_subif_start_xmit+0x409/0x56b [mac80211]
 [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
 [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
 [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
 [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
 [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
 [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
 [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
 [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
 [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
 [<ffffffff803e33a8>] ? release_sock+0xcd/0xd6
 [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
 [<ffffffff8040f093>] ip_local_out+0x20/0x24
 [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
 [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
 [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
 [<ffffffff80422e9d>] tcp_push_one+0x2f/0x31
 [<ffffffff80417439>] tcp_sendmsg+0x7a1/0x9fe
 [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
 [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff803e0f6e>] ? sock_sendmsg+0xdf/0xf8
 [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
 [<ffffffff803e39b3>] sock_no_sendpage+0x9b/0xaa
 [<ffffffff804176de>] tcp_sendpage+0x48/0x5ec
 [<ffffffffa054c525>] xs_sendpages+0x12c/0x1af [sunrpc]
 [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
 [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
 [<ffffffffa05bf9c3>] ? nfs3_xdr_writeargs+0x0/0x87 [nfs]
 [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
 [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
 [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
 [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
 [<ffffffffa05bb774>] nfs_write_rpcsetup+0x215/0x237 [nfs]
 [<ffffffffa05bd257>] nfs_flush_one+0xa2/0xd9 [nfs]
 [<ffffffffa05b82d9>] nfs_pageio_doio+0x32/0x5b [nfs]
 [<ffffffffa05b83ec>] nfs_pageio_complete+0x9/0xb [nfs]
 [<ffffffffa05bbeae>] nfs_writepages+0x101/0x13a [nfs]
 [<ffffffffa05bd1b5>] ? nfs_flush_one+0x0/0xd9 [nfs]
 [<ffffffffa05bd043>] nfs_write_mapping+0x63/0x9e [nfs]
 [<ffffffffa05bd0a7>] nfs_wb_all+0x12/0x14 [nfs]
 [<ffffffffa05b0145>] nfs_file_flush+0x8a/0xb1 [nfs]
 [<ffffffff802bb18d>] filp_close+0x40/0x63
 [<ffffffff802bb255>] sys_close+0xa5/0xe4
 [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  42
CPU    1: hi:  186, btch:  31 usd: 207
Active_anon:165674 active_file:111453 inactive_anon:47087
 inactive_file:331621 unevictable:8 dirty:11 writeback:0 unstable:0
 free:4632 slab:75221 mapped:19318 pagetables:4242 bounce:0
Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
present:15220kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2927 2927 2927
Node 0 DMA32 free:16424kB min:6904kB low:8628kB high:10356kB
active_anon:662696kB inactive_anon:188348kB active_file:445812kB
inactive_file:1326484kB unevictable:32kB present:2997292kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 2104kB
Node 0 DMA32: 3831*4kB 1*8kB 3*16kB 0*32kB 1*64kB 1*128kB 1*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 16340kB
444199 total pagecache pages
962 pages in swap cache
Swap cache stats: add 4683, delete 3721, find 2934/3084
Free swap  = 2091252kB
Total swap = 2104444kB
769872 pages RAM
21377 pages reserved
358924 pages shared
469842 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=20)
  cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
order: 3, min order: 1
  node 0: slabs: 95, objs: 665, free: 2
phy0: failed to reallocate TX buffer

If you need the rest of the dmesg output, or anything else, please let
me know.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Wed, 2009-06-10 at 09:41 -0500, Larry Finger wrote:
> With the above patch installed, I pushed my system hard enough to get
> the O(1) allocation failures. This time they were triggered with a
> 'make -j8' on the kernel. No, I don't have that many CPUs, but I
> figured that the extra make jobs might stress memory. My kernel is
> 2.6.30-rc8 from the wireless-testing tree. Everything matches Linus's
> tree except drivers/net/wireless/, which contains what is essentially
> 2.6.31 code.
> 
> The dmesg output starting with the first allocation failure is:
> 
> cc1: page allocation failure. order:1, mode:0x4020
> Pid: 6577, comm: cc1 Not tainted 2.6.30-rc8-wl #164
> Call Trace:
>  [<ffffffff80292a7b>] __alloc_pages_internal+0x43d/0x45e
>  [<ffffffff802b1f1f>] alloc_pages_current+0xbe/0xc6
>  [<ffffffff802b6362>] new_slab+0xcf/0x28b
>  [<ffffffff802b4d1f>] ? unfreeze_slab+0x4c/0xbd
>  [<ffffffff802b672e>] __slab_alloc+0x210/0x44c
>  [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
>  [<ffffffff803e7bee>] ? pskb_expand_head+0x52/0x166
>  [<ffffffff802b7e60>] __kmalloc+0x119/0x194
>  [<ffffffff803e7bee>] pskb_expand_head+0x52/0x166
>  [<ffffffffa02913d6>] ieee80211_skb_resize+0x91/0xc7 [mac80211]
>  [<ffffffffa0291c0f>] ieee80211_master_start_xmit+0x298/0x319 [mac80211]
>  [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
>  [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
>  [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
>  [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
>  [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
>  [<ffffffffa0291957>] ieee80211_subif_start_xmit+0x54b/0x56b [mac80211]
>  [<ffffffffa029162b>] ? ieee80211_subif_start_xmit+0x21f/0x56b [mac80211]
>  [<ffffffff8025cea8>] ? trace_hardirqs_on+0xd/0xf
>  [<ffffffff803e7790>] ? __kfree_skb+0x82/0x86
>  [<ffffffff803ef72a>] dev_hard_start_xmit+0x229/0x2a8
>  [<ffffffff803ef55c>] ? dev_hard_start_xmit+0x5b/0x2a8
>  [<ffffffff804005ee>] __qdisc_run+0xed/0x1fe
>  [<ffffffff803efb08>] dev_queue_xmit+0x24c/0x384
>  [<ffffffff803efa2f>] ? dev_queue_xmit+0x173/0x384
>  [<ffffffff8040fec9>] ip_finish_output+0x217/0x25c
>  [<ffffffff802b4038>] ? add_partial+0x1a/0x69
>  [<ffffffff8040ffaa>] ip_output+0x9c/0xa1
>  [<ffffffff8040f093>] ip_local_out+0x20/0x24
>  [<ffffffff8040f900>] ip_queue_xmit+0x2e0/0x337
>  [<ffffffff8042087e>] tcp_transmit_skb+0x5f7/0x63a
>  [<ffffffff802b790b>] ? __kmalloc_node_track_caller+0xd3/0x144
>  [<ffffffff80422d89>] tcp_write_xmit+0x83f/0x924
>  [<ffffffff803e872d>] ? __alloc_skb+0x6f/0x143
>  [<ffffffff80422ec9>] __tcp_push_pending_frames+0x2a/0x81
>  [<ffffffff80417590>] tcp_sendmsg+0x8f8/0x9fe
>  [<ffffffff803e0f6e>] sock_sendmsg+0xdf/0xf8
>  [<ffffffff8024efec>] ? autoremove_wake_function+0x0/0x38
>  [<ffffffff8023695c>] ? finish_task_switch+0x3b/0xdc
>  [<ffffffff803e11f7>] kernel_sendmsg+0x34/0x49
>  [<ffffffffa054c3f0>] xs_send_kvec+0x7a/0x83 [sunrpc]
>  [<ffffffffa054c486>] xs_sendpages+0x8d/0x1af [sunrpc]
>  [<ffffffffa054c6b1>] xs_tcp_send_request+0x52/0x149 [sunrpc]
>  [<ffffffffa054b470>] xprt_transmit+0x178/0x234 [sunrpc]
>  [<ffffffffa05bfc11>] ? nfs3_xdr_fhandle+0x0/0x2e [nfs]
>  [<ffffffffa0548d02>] call_transmit+0x20e/0x250 [sunrpc]
>  [<ffffffffa054f8a7>] __rpc_execute+0x86/0x244 [sunrpc]
>  [<ffffffffa054fa8d>] rpc_execute+0x28/0x2c [sunrpc]
>  [<ffffffffa054963c>] rpc_run_task+0x56/0x5e [sunrpc]
>  [<ffffffffa054972f>] rpc_call_sync+0x3f/0x5d [sunrpc]
>  [<ffffffffa05bdcd0>] nfs3_rpc_wrapper+0x22/0x5c [nfs]
>  [<ffffffffa05be40c>] nfs3_proc_getattr+0x5b/0x81 [nfs]
>  [<ffffffffa05b1e22>] __nfs_revalidate_inode+0xbd/0x1c9 [nfs]
>  [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
>  [<ffffffffa05d0529>] ? nfs_have_delegation+0x79/0x82 [nfs]
>  [<ffffffffa05d04b0>] ? nfs_have_delegation+0x0/0x82 [nfs]
>  [<ffffffffa05acb60>] nfs_lookup_revalidate+0x265/0x49c [nfs]
>  [<ffffffff802ccfa9>] ? __d_lookup+0xba/0x16a
>  [<ffffffff802cd047>] ? __d_lookup+0x158/0x16a
>  [<ffffffff802cceef>] ? __d_lookup+0x0/0x16a
>  [<ffffffffa0550992>] ? rpcauth_lookupcred+0x77/0x9f [sunrpc]
>  [<ffffffff802c49c6>] do_lookup+0x166/0x1bb
>  [<ffffffff802c66b7>] __link_path_walk+0x8f8/0xd58
>  [<ffffffff802c6d1d>] path_walk+0x69/0xd4
>  [<ffffffff802c6fb6>] do_path_lookup+0x187/0x1df
>  [<ffffffff802bdf80>] ? get_empty_filp+0xe9/0x14e
>  [<ffffffff802c7c4b>] do_filp_open+0x105/0x909
>  [<ffffffff802d0bb6>] ? alloc_fd+0x11d/0x12e
>  [<ffffffff802bb2ea>] do_sys_open+0x56/0xd6
>  [<ffffffff802bb393>] sys_open+0x1b/0x1d
>  [<ffffffff8020baab>] system_call_fastpath+0x16/0x1b
> Mem-Info:
> Node 0 DMA per-cpu:
> CPU    0: hi:    0, btch:   1 usd:   0
> CPU    1: hi:    0, btch:   1 usd:   0
> Node 0 DMA32 per-cpu:
> CPU    0: hi:  186, btch:  31 usd:  15
> CPU    1: hi:  186, btch:  31 usd:  65
> Active_anon:128724 active_file:123018 inactive_anon:47276
>  inactive_file:355583 unevictable:8 dirty:18 writeback:0 unstable:0
>  free:3621 slab:77881 mapped:18629 pagetables:4056 bounce:0
> Node 0 DMA free:2104kB min:32kB low:40kB high:48kB active_anon:0kB
> inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
> present:15220kB pages_scanned:0 all_unreclaimable? yes
> lowmem_reserve[]: 0 2927 2927 2927
> Node 0 DMA32 free:12380kB min:6904kB low:8628kB high:10356kB
> active_anon:514896kB inactive_anon:189104kB active_file:492072kB
> inactive_file:1422332kB unevictable:32kB present:2997292kB
> pages_scanned:0 all_unreclaimable? no
> lowmem_reserve[]: 0 0 0 0
> Node 0 DMA: 4*4kB 3*8kB 5*16kB 2*32kB 4*64kB 1*128kB 2*256kB 0*512kB
> 1*1024kB 0*2048kB 0*4096kB = 2104kB
> Node 0 DMA32: 2821*4kB 1*8kB 3*16kB 1*32kB 1*64kB 1*128kB 1*256kB
> 1*512kB 0*1024kB 0*2048kB 0*4096kB = 12332kB
> 479694 total pagecache pages
> 969 pages in swap cache
> Swap cache stats: add 4523, delete 3554, find 2913/3063
> Free swap  = 2091884kB
> Total swap = 2104444kB
> 769872 pages RAM
> 21377 pages reserved
> 382252 pages shared
> 441407 pages non-shared
> SLUB: Unable to allocate memory on node -1 (gfp=20)
>   cache: kmalloc-4096, object size: 4096, buffer size: 4168, default
> order: 3, min order: 1
>   node 0: slabs: 95, objs: 665, free: 0
> phy0: failed to reallocate TX buffer

Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
help. Christoph, are you okay with this patch?

			Pekka

diff --git a/mm/slub.c b/mm/slub.c
index 65ffda5..2c93c30 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2334,6 +2334,8 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
 
 }
 
+#define MAX_DEBUG_SIZE (3 * sizeof(void *) + 2 * sizeof(struct track))
+
 static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
 		const char *name, size_t size,
 		size_t align, unsigned long flags,
@@ -2346,6 +2348,9 @@ static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
 	s->align = align;
 	s->flags = kmem_cache_flags(size, flags, name, ctor);
 
+	if ((size + MAX_DEBUG_SIZE) >= PAGE_SIZE)
+		flags &= ~(SLAB_POISON|SLAB_RED_ZONE|SLAB_STORE_USER);
+
 	if (!calculate_sizes(s, -1))
 		goto error;
 

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Wed, 2009-06-10 at 18:44 +0300, Pekka Enberg wrote:

> Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
> CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
> help. Christoph, are you okay with this patch?
> 
> 			Pekka
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 65ffda5..2c93c30 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -2334,6 +2334,8 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
>  
>  }
>  
> +#define MAX_DEBUG_SIZE (3 * sizeof(void *) + 2 * sizeof(struct track))
> +
>  static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
>  		const char *name, size_t size,
>  		size_t align, unsigned long flags,
> @@ -2346,6 +2348,9 @@ static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
>  	s->align = align;
>  	s->flags = kmem_cache_flags(size, flags, name, ctor);
>  
> +	if ((size + MAX_DEBUG_SIZE) >= PAGE_SIZE)
> +		flags &= ~(SLAB_POISON|SLAB_RED_ZONE|SLAB_STORE_USER);
> +
>  	if (!calculate_sizes(s, -1))
>  		goto error;
>  
> 
Argh, that patch has a typo. Please try this one instead.

			Pekka

diff --git a/mm/slub.c b/mm/slub.c
index 65ffda5..cb0473c 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2334,6 +2334,8 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
 
 }
 
+#define MAX_DEBUG_SIZE (3 * sizeof(void *) + 2 * sizeof(struct track))
+
 static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
 		const char *name, size_t size,
 		size_t align, unsigned long flags,
@@ -2346,6 +2348,9 @@ static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
 	s->align = align;
 	s->flags = kmem_cache_flags(size, flags, name, ctor);
 
+	if ((size + MAX_DEBUG_SIZE) >= PAGE_SIZE)
+		s->flags &= ~(SLAB_POISON|SLAB_RED_ZONE|SLAB_STORE_USER);
+
 	if (!calculate_sizes(s, -1))
 		goto error;
 

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

On Wed, 2009-06-10 at 18:49 +0300, Pekka Enberg wrote:
> On Wed, 2009-06-10 at 18:44 +0300, Pekka Enberg wrote:
> 
> > Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
> > CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
> > help. Christoph, are you okay with this patch?


> +	if ((size + MAX_DEBUG_SIZE) >= PAGE_SIZE)

&& size <= PAGE_SIZE

? Or is this a path that only happens for small allocations?

> +		s->flags &= ~(SLAB_POISON|SLAB_RED_ZONE|SLAB_STORE_USER);
> +
>  	if (!calculate_sizes(s, -1))
>  		goto error;

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Mel Gorman]

On Tue, Jun 09, 2009 at 10:06:41AM +0300, Pekka Enberg wrote:
> Hi Mel,
> 
> On Mon, 2009-06-08 at 15:12 +0100, Mel Gorman wrote:
> > > diff --git a/mm/slub.c b/mm/slub.c
> > > index 65ffda5..b5acf18 100644
> > > --- a/mm/slub.c
> > > +++ b/mm/slub.c
> > > @@ -1565,6 +1565,8 @@ new_slab:
> > >  		c->page = new;
> > >  		goto load_freelist;
> > >  	}
> > > +	printk(KERN_WARNING "SLUB: unable to satisfy allocation for cache %s (size=%d, node=%d, gfp=%x)\n",
> > > +		s->name, s->size, node, gfpflags);
> > 
> > size could be almost anything here for a casual reader. You are
> > outputting the size of the object plus its metadata so the name should
> > reflect that. I think it would be better to output objsize= and the
> > object size without the metadata overhead. What do you think?
> > 
> > In addition, include how many objects there are per-slab and include what
> > the order is being passed to the page allocator when allocating new slabs.
> > Would that be enough to determine if fallback-to-smaller orders occured?
> 
> So how about something like this then?
> 
> 			Pekka
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 65ffda5..a03dbe8 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -1484,6 +1484,58 @@ static inline int node_match(struct kmem_cache_cpu *c, int node)
>  	return 1;
>  }
>  
> +static int count_free(struct page *page)
> +{
> +	return page->objects - page->inuse;
> +}
> +
> +static unsigned long count_partial(struct kmem_cache_node *n,
> +					int (*get_count)(struct page *))
> +{
> +	unsigned long flags;
> +	unsigned long x = 0;
> +	struct page *page;
> +
> +	spin_lock_irqsave(&n->list_lock, flags);
> +	list_for_each_entry(page, &n->partial, lru)
> +		x += get_count(page);
> +	spin_unlock_irqrestore(&n->list_lock, flags);
> +	return x;
> +}
> +
> +static noinline void
> +slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
> +{
> +	int node;
> +
> +	printk(KERN_WARNING
> +		"SLUB: Unable to allocate memory on node %d (gfp=%x)\n",
> +		nid, gfpflags);
> +	printk(KERN_WARNING "  cache: %s, object size: %d, buffer size: %d, "
> +		"default order: %d, min order: %d\n", s->name, s->objsize,
> +		s->size, oo_order(s->oo), oo_order(s->min));
> +

Much nicer. There is a clear division between the object size and the
size including the metadata. There is also now a good idea of what sort
of request it was, we know what cache it was so we can guess the size
passed to kmalloc() with reasonable accuracy.

> +	for_each_online_node(node) {
> +		struct kmem_cache_node *n = get_node(s, node);
> +		unsigned long nr_partials;
> +		unsigned long nr_slabs;
> +		unsigned long nr_objs;
> +		unsigned long nr_free;
> +
> +		if (!n)
> +			continue;
> +
> +		nr_partials = n->nr_partial;
> +		nr_slabs = atomic_long_read(&n->nr_slabs);
> +		nr_objs = atomic_long_read(&n->total_objects);
> +		nr_free = count_partial(n, count_free);
> +
> +		printk(KERN_WARNING
> +			"  node %d: partials: %ld, slabs: %ld, objs: %ld, free: %ld\n",
> +			node, nr_partials, nr_slabs, nr_objs, nr_free);
> +	}
> +}

That looks like it would generate easier-to-debug-with messages and to
not-expert-at-slub eye, it looks correct. Slap a changelog on it with an
example message and go with it.  It should make page allocation failures
messages that go through SLUB a lot easier to figure out.

Thanks

> +
>  /*
>   * Slow path. The lockless freelist is empty or we need to perform
>   * debugging duties.
> @@ -1565,6 +1617,7 @@ new_slab:
>  		c->page = new;
>  		goto load_freelist;
>  	}
> +	slab_out_of_memory(s, gfpflags, node);
>  	return NULL;
>  debug:
>  	if (!alloc_debug_processing(s, c->page, object, addr))
> @@ -3318,20 +3371,6 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t gfpflags,
>  }
>  
>  #ifdef CONFIG_SLUB_DEBUG
> -static unsigned long count_partial(struct kmem_cache_node *n,
> -					int (*get_count)(struct page *))
> -{
> -	unsigned long flags;
> -	unsigned long x = 0;
> -	struct page *page;
> -
> -	spin_lock_irqsave(&n->list_lock, flags);
> -	list_for_each_entry(page, &n->partial, lru)
> -		x += get_count(page);
> -	spin_unlock_irqrestore(&n->list_lock, flags);
> -	return x;
> -}
> -
>  static int count_inuse(struct page *page)
>  {
>  	return page->inuse;
> @@ -3342,11 +3381,6 @@ static int count_total(struct page *page)
>  	return page->objects;
>  }
>  
> -static int count_free(struct page *page)
> -{
> -	return page->objects - page->inuse;
> -}
> -
>  static int validate_slab(struct kmem_cache *s, struct page *page,
>  						unsigned long *map)
>  {
> 
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Wed, 2009-06-10 at 17:52 +0200, Johannes Berg wrote:
> On Wed, 2009-06-10 at 18:49 +0300, Pekka Enberg wrote:
> > On Wed, 2009-06-10 at 18:44 +0300, Pekka Enberg wrote:
> > 
> > > Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
> > > CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
> > > help. Christoph, are you okay with this patch?
> 
> 
> > +	if ((size + MAX_DEBUG_SIZE) >= PAGE_SIZE)
> 
> && size <= PAGE_SIZE
> 
> ? Or is this a path that only happens for small allocations?

Anything that's beyond PAGE_SIZE * 2 is passed straight to the page
allocator and the intent of this patch is to disable debugging for all
big caches like SLAB does.

			Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Pekka Enberg wrote:

> Argh, that patch has a typo. Please try this one instead.
> 
> 			Pekka

I installed this patch on top of the other one and have started
testing. It usually takes almost a day for it to occur.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Wed, 2009-06-10 at 18:49 +0300, Pekka Enberg wrote:
> > On Wed, 2009-06-10 at 18:44 +0300, Pekka Enberg wrote:
> > 
> > > Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
> > > CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
> > > help. Christoph, are you okay with this patch?

On Wed, 2009-06-10 at 17:52 +0200, Johannes Berg wrote:
> > +	if ((size + MAX_DEBUG_SIZE) >= PAGE_SIZE)
> 
> && size <= PAGE_SIZE
> 
> ? Or is this a path that only happens for small allocations?
> 
> > +		s->flags &= ~(SLAB_POISON|SLAB_RED_ZONE|SLAB_STORE_USER);
> > +
> >  	if (!calculate_sizes(s, -1))
> >  		goto error;

Although something like this would probably be even nicer.

			Pekka

diff --git a/mm/slub.c b/mm/slub.c
index 65ffda5..a4206ef 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2334,6 +2334,16 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
 
 }
 
+#define MAX_DEBUG_SIZE (3 * sizeof(void *) + 2 * sizeof(struct track))
+
+static bool must_disable_debug(size_t size)
+{
+	/*
+	 * Disable debugging if it increases the minimum page order.
+	 */
+	return get_order(size + MAX_DEBUG_SIZE) > get_order(size);
+}
+
 static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
 		const char *name, size_t size,
 		size_t align, unsigned long flags,
@@ -2346,6 +2356,9 @@ static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags,
 	s->align = align;
 	s->flags = kmem_cache_flags(size, flags, name, ctor);
 
+	if (must_disable_debug(size))
+		s->flags &= ~(SLAB_POISON|SLAB_RED_ZONE|SLAB_STORE_USER);
+
 	if (!calculate_sizes(s, -1))
 		goto error;
 

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Wed, Jun 10, 2009 at 6:56 PM, Mel Gorman<mel-wPRd99KPJ+uzQB+pC5nmwQ@public.gmane.org> wrote:
> That looks like it would generate easier-to-debug-with messages and to
> not-expert-at-slub eye, it looks correct. Slap a changelog on it with an
> example message and go with it.  It should make page allocation failures
> messages that go through SLUB a lot easier to figure out.

Thanks Mel! The patch is here and will be part of next slab pull
request to Linus:

http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commitdiff;h=4bc6e7858da5ea4ecc3e47538f7fabed331cc21b

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Wed, 10 Jun 2009, Pekka Enberg wrote:

> Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
> CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
> help. Christoph, are you okay with this patch?

He likely has CONFIG_SLUB_DEBUG_ON set which enables debugging and thus
needs more than the payload for metadata.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Wed, 10 Jun 2009, Pekka Enberg wrote:
> > Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
> > CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
> > help. Christoph, are you okay with this patch?

On Thu, 2009-06-11 at 10:41 -0400, Christoph Lameter wrote:
> He likely has CONFIG_SLUB_DEBUG_ON set which enables debugging and thus
> needs more than the payload for metadata.

Yup. I suspect a lot of people who are doing _testing_ enable that. If
you're unhappy with my patch (the get_order one which shouldn't affect
that many caches anyway), any suggestions how to fix this up? It seems
that the wireless stack at least does quite a few kmalloc(4096)
allocations.

We can probably switch back to page allocator pass-through in the near
future (when Mel's patches are in) but we need a fix for -stable.

			Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 892 bytes --]

On Thu, 2009-06-11 at 18:09 +0300, Pekka Enberg wrote:
> On Wed, 10 Jun 2009, Pekka Enberg wrote:
> > > Aha, SLUB thinks the minimum order for 4096 is 1. I guess you have
> > > CONFIG_SLUB_DEBUG enabled? If yes, something like to following should
> > > help. Christoph, are you okay with this patch?
> 
> On Thu, 2009-06-11 at 10:41 -0400, Christoph Lameter wrote:
> > He likely has CONFIG_SLUB_DEBUG_ON set which enables debugging and thus
> > needs more than the payload for metadata.
> 
> Yup. I suspect a lot of people who are doing _testing_ enable that. If
> you're unhappy with my patch (the get_order one which shouldn't affect
> that many caches anyway), any suggestions how to fix this up? It seems
> that the wireless stack at least does quite a few kmalloc(4096)
> allocations.

I think networking rounds up allocations, but it's not wireless per se.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of regressions introduced between 2.6.29 and 2.6.30.

The following bug entry is on the current list of known regressions
introduced between 2.6.29 and 2.6.30.  Please verify if it still should
be listed and let me know (either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (61 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
		  http://lkml.org/lkml/2009/6/7/136
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of regressions introduced between 2.6.29 and 2.6.30.
> 
> The following bug entry is on the current list of known regressions
> introduced between 2.6.29 and 2.6.30.  Please verify if it still should
> be listed and let me know (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> Subject		: Page allocation failures with b43 and p54usb
> Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> Date		: 2009-04-29 21:01 (61 days old)
> References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> 		  http://lkml.org/lkml/2009/6/7/136
> Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

The cause of these failures has been determined. The wireless
subsystem frequently requests buffers of size 4096, but when SLUB
debugging is enabled and the debug info is added, the request becomes
of order 1 and memory becomes fragmented.

A controversial "fix" in which SLUB debugging was disabled for
allocations where adding such debugging info would increase the order
was discussed and tried. With a quick look at the commit list for
Linus's tree, I don't see that such a patch is available, but I will
be corrected if I missed it.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

On Monday 29 June 2009, Larry Finger wrote:
> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of regressions introduced between 2.6.29 and 2.6.30.
> > 
> > The following bug entry is on the current list of known regressions
> > introduced between 2.6.29 and 2.6.30.  Please verify if it still should
> > be listed and let me know (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> > Subject		: Page allocation failures with b43 and p54usb
> > Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> > Date		: 2009-04-29 21:01 (61 days old)
> > References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> > 		  http://lkml.org/lkml/2009/6/7/136
> > Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
> 
> The cause of these failures has been determined. The wireless
> subsystem frequently requests buffers of size 4096, but when SLUB
> debugging is enabled and the debug info is added, the request becomes
> of order 1 and memory becomes fragmented.
> 
> A controversial "fix" in which SLUB debugging was disabled for
> allocations where adding such debugging info would increase the order
> was discussed and tried. With a quick look at the commit list for
> Linus's tree, I don't see that such a patch is available, but I will
> be corrected if I missed it.

Thanks for the update.

Hmm, isn't it suboptimal to use a slab allocator for allocations taking up an
entire page?  That's the case on some architectures and seems to be the root
cause of the issue at hand.

Best,
Rafael

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Mon, 29 Jun 2009, Larry Finger wrote:

> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> > Subject		: Page allocation failures with b43 and p54usb
> > Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> > Date		: 2009-04-29 21:01 (61 days old)
> > References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> > 		  http://lkml.org/lkml/2009/6/7/136
> > Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
> 
> The cause of these failures has been determined. The wireless
> subsystem frequently requests buffers of size 4096, but when SLUB
> debugging is enabled and the debug info is added, the request becomes
> of order 1 and memory becomes fragmented.
> 
> A controversial "fix" in which SLUB debugging was disabled for
> allocations where adding such debugging info would increase the order
> was discussed and tried. With a quick look at the commit list for
> Linus's tree, I don't see that such a patch is available, but I will
> be corrected if I missed it.
> 

I'd disagree with disabling slub debugging by default for caches where 
oo_order(s->min) increases as the result of using it.  This particular 
page allocation failure is happening for, presumably, kmalloc-4096, and 
the system has 4K pages.  Disabling debugging for that cache (and any of 
its aliases) implicitly will lead to errors going undiagnosed as a result.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

David Rientjes wrote:
> On Mon, 29 Jun 2009, Larry Finger wrote:
> 
>>> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
>>> Subject		: Page allocation failures with b43 and p54usb
>>> Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
>>> Date		: 2009-04-29 21:01 (61 days old)
>>> References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>>> 		  http://lkml.org/lkml/2009/6/7/136
>>> Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
>> The cause of these failures has been determined. The wireless
>> subsystem frequently requests buffers of size 4096, but when SLUB
>> debugging is enabled and the debug info is added, the request becomes
>> of order 1 and memory becomes fragmented.
>>
>> A controversial "fix" in which SLUB debugging was disabled for
>> allocations where adding such debugging info would increase the order
>> was discussed and tried. With a quick look at the commit list for
>> Linus's tree, I don't see that such a patch is available, but I will
>> be corrected if I missed it.
>>
> 
> I'd disagree with disabling slub debugging by default for caches where 
> oo_order(s->min) increases as the result of using it.  This particular 
> page allocation failure is happening for, presumably, kmalloc-4096, and 
> the system has 4K pages.  Disabling debugging for that cache (and any of 
> its aliases) implicitly will lead to errors going undiagnosed as a result.

If the current behavior is not changed, I will be forced to disable
SLUB debugging, which will explicitly lead to errors that are
undiagnosed. It seems better to me to debug when you can, but turn off
debugging in cases like this.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Mon, 29 Jun 2009, Larry Finger wrote:

> > I'd disagree with disabling slub debugging by default for caches where 
> > oo_order(s->min) increases as the result of using it.  This particular 
> > page allocation failure is happening for, presumably, kmalloc-4096, and 
> > the system has 4K pages.  Disabling debugging for that cache (and any of 
> > its aliases) implicitly will lead to errors going undiagnosed as a result.
> 
> If the current behavior is not changed, I will be forced to disable
> SLUB debugging, which will explicitly lead to errors that are
> undiagnosed.

You're buying debugging support at the cost of increased memory 
consumption when you enable CONFIG_SLUB_DEBUG_ON and that's causing the 
page allocation failures because of fragmentation.  To reduce the minimum 
order required for caches such as kmalloc-4096, you'd have to disable 
debugging for that particular cache.  It's my opinion that such a 
configuration should not be the default, however.

You could argue adding `slub_debug=-,kmalloc-4096' support from the 
command line, but CONFIG_SLUB_DEBUG_ON should not change its well-defined 
purpose of enabling debugging on all slab caches.  Otherwise the rest of 
us would be forced to add `slub_debug=,kmalloc-4096' for consistent 
behavior with older kernels.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi David,

On Tue, Jun 30, 2009 at 2:47 AM, David Rientjes<rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> On Mon, 29 Jun 2009, Larry Finger wrote:
>
>> > Bug-Entry   : http://bugzilla.kernel.org/show_bug.cgi?id=13319
>> > Subject             : Page allocation failures with b43 and p54usb
>> > Submitter   : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
>> > Date                : 2009-04-29 21:01 (61 days old)
>> > References  : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>> >               http://lkml.org/lkml/2009/6/7/136
>> > Handled-By  : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
>>
>> The cause of these failures has been determined. The wireless
>> subsystem frequently requests buffers of size 4096, but when SLUB
>> debugging is enabled and the debug info is added, the request becomes
>> of order 1 and memory becomes fragmented.
>>
>> A controversial "fix" in which SLUB debugging was disabled for
>> allocations where adding such debugging info would increase the order
>> was discussed and tried. With a quick look at the commit list for
>> Linus's tree, I don't see that such a patch is available, but I will
>> be corrected if I missed it.
>>
>
> I'd disagree with disabling slub debugging by default for caches where
> oo_order(s->min) increases as the result of using it.  This particular
> page allocation failure is happening for, presumably, kmalloc-4096, and
> the system has 4K pages.  Disabling debugging for that cache (and any of
> its aliases) implicitly will lead to errors going undiagnosed as a result.

Well, I obviously don't agree here because kmalloc-4096 debugging
causes problems in the real world. Furthermore, SLUB never supported
debugging for objects that big historically because of page allocator
passthrough. And with Mel Gorman's page allocator optimizations, we
might be going back to that.

So we should fix SLUB debugging as outlined by Mel Gorman and
Christoph Lameter. I simply haven't had the time to do it. Patches are
welcome!

                              Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2316 bytes --]

On Tue, 30 Jun 2009, Pekka Enberg wrote:

> > I'd disagree with disabling slub debugging by default for caches where
> > oo_order(s->min) increases as the result of using it.  This particular
> > page allocation failure is happening for, presumably, kmalloc-4096, and
> > the system has 4K pages.  Disabling debugging for that cache (and any of
> > its aliases) implicitly will lead to errors going undiagnosed as a result.
> 
> Well, I obviously don't agree here because kmalloc-4096 debugging
> causes problems in the real world.

I don't think CONFIG_SLUB_DEBUG_ON is generally the configuration used in 
the real world.

The option has a clear and well-defined purpose and that is to enable 
debugging on all slab caches.  If you modify its definition, users will 
generally ignore the warning about debugging being disabled when "the 
minimum possible order at which slab may be allocated is higher than 
without."  And unless they check the kernel log for such a warning to boot 
with `slab_debug=,kmalloc-4096', we lose testing coverage because we 
cannot enable redzoning or tracing after boot.

> Furthermore, SLUB never supported
> debugging for objects that big historically because of page allocator
> passthrough. And with Mel Gorman's page allocator optimizations, we
> might be going back to that.
> 

Even when page allocation is fast enough, it would still be helpful to 
configure slub to not do passthrough purely for the lightweight debugging 
opportunities.

> So we should fix SLUB debugging as outlined by Mel Gorman and
> Christoph Lameter. I simply haven't had the time to do it. Patches are
> welcome!
> 

You're referring to `slub_debug=A'?  I think CONFIG_SLUB_DEBUG_ON should 
continue to enable debugging on all slab caches and in instances where it 
causes page allocation failures such in Larry's case because 
oo_order(s->min) with debugging on is greater than oo_order(s->min) with 
debugging off, you can emit a friendly warning in your recently added 
slab_out_of_memory() about using `slab_debug=-,<cache>'.

We have a disagreement about which is the default behavior, but I would 
opt on the side of adding exemptions to a debug configuration option as 
opposed to requiring additional command line parameters to be fully 
enabled.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi David,

On Tue, 30 Jun 2009, Pekka Enberg wrote:
>> > I'd disagree with disabling slub debugging by default for caches where
>> > oo_order(s->min) increases as the result of using it.  This particular
>> > page allocation failure is happening for, presumably, kmalloc-4096, and
>> > the system has 4K pages.  Disabling debugging for that cache (and any of
>> > its aliases) implicitly will lead to errors going undiagnosed as a result.
>>
>> Well, I obviously don't agree here because kmalloc-4096 debugging
>> causes problems in the real world.

On Tue, Jun 30, 2009 at 10:47 AM, David Rientjes<rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> I don't think CONFIG_SLUB_DEBUG_ON is generally the configuration used in
> the real world.

It is, hence the epic bug report that's eaten too many man hours
already! Look, we encourage _testers_ to turn all as much as debugging
options as possible so we catch bugs early. That why the only sane
defaults are the ones that don't cause other problems!

I don't know why you want to argue this. It's simply not an option to
say "stupid user, fix your config" in core code like the slab
allocator. Enabling CONFIG_SLUB_DEBUG_ON is a very reasonable thing to
do when you are a tester looking for bugs.

On Tue, 30 Jun 2009, Pekka Enberg wrote:
>> So we should fix SLUB debugging as outlined by Mel Gorman and
>> Christoph Lameter. I simply haven't had the time to do it. Patches are
>> welcome!

On Tue, Jun 30, 2009 at 10:47 AM, David Rientjes<rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> You're referring to `slub_debug=A'?  I think CONFIG_SLUB_DEBUG_ON should
> continue to enable debugging on all slab caches and in instances where it
> causes page allocation failures such in Larry's case because
> oo_order(s->min) with debugging on is greater than oo_order(s->min) with
> debugging off, you can emit a friendly warning in your recently added
> slab_out_of_memory() about using `slab_debug=-,<cache>'.
>
> We have a disagreement about which is the default behavior, but I would
> opt on the side of adding exemptions to a debug configuration option as
> opposed to requiring additional command line parameters to be fully
> enabled.

Yup, I was referring to slub_debug=A and no, I don't agree with you
that it should be on by default. Only people who know what they're
doing should enable the option and a random tester by definition
doesn't (no offence to Mr. Random Tester).

                                                   Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Tue, 30 Jun 2009, Pekka Enberg wrote:

> Well, I obviously don't agree here because kmalloc-4096 debugging causes
> problems in the real world. Furthermore, SLUB never supported debugging
> for objects that big historically because of page allocator passthrough.
> And with Mel Gorman's page allocator optimizations, we might be going
> back to that.

SLUB for some period of time had passthrough. It did not start out like
that though.

kmalloc-4096 causes problems in the long run and so do other caches that
are of similar size. But it allows debugging to occur. Silently switching
it off is something I am not comfortable with.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Pekka Enberg wrote:
> 
> Yup, I was referring to slub_debug=A and no, I don't agree with you
> that it should be on by default. Only people who know what they're
> doing should enable the option and a random tester by definition
> doesn't (no offence to Mr. Random Tester).

None taken.

For me, the next step is clear. As I'm much more interested in finding
bugs in the wireless system than in the mechanics of SLUB allocation,
I need to disable CONFIG_SLUB_DEBUG_ON. BTW, I use SLAB on Linus's
mainline tree and SLUB on the wireless testing tree. I build and boot
the mainline kernels mostly to look for quick failures/regressions,
but run the w-t kernels looking for longer-term effects such as memory
fragmentation or slow memory leaks.

For Rafael's benefit, we do need to decide if this is a bug or merely
an unintended side effect. My sense is the latter and Bug #13319
should have a summary of this discussion added to the record, and then
the bug should be closed.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Tue, 2009-06-30 at 10:32 -0400, Christoph Lameter wrote:
> kmalloc-4096 causes problems in the long run and so do other caches that
> are of similar size. But it allows debugging to occur. Silently switching
> it off is something I am not comfortable with.

I suggested adding a

  printk(KERN_INFO ": debugging disabled for %s. Use slub_debug=a to "
                    "enable it blah blah blah\n");

Does that work for you?

			Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Tue, 30 Jun 2009, Pekka Enberg wrote:

>   printk(KERN_INFO ": debugging disabled for %s. Use slub_debug=a to "
>                     "enable it blah blah blah\n");
>
> Does that work for you?

Its definitely better.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Tue, 30 Jun 2009, Christoph Lameter wrote:

> >   printk(KERN_INFO ": debugging disabled for %s. Use slub_debug=a to "
> >                     "enable it blah blah blah\n");
> >
> > Does that work for you?
> 
> Its definitely better.
> 

I don't see how that's different from enabling debugging on all caches 
like CONFIG_SLAB_DEBUG_ON currently does and then warning at the time of 
slab allocation failure that it may be the result of the debugging 
metadata so the user can subsequently prevent it.  In other words, if we 
use MAX_DEBUG_SIZE as Pekka originally implemented as
(3 * sizeof(void *) + 2 * sizeof(struct track)), do this:

diff --git a/mm/slub.c b/mm/slub.c
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -142,6 +142,11 @@
 				SLAB_POISON | SLAB_STORE_USER)
 
 /*
+ * The maximum amount of metadata added to a slab when debugging is enabled.
+ */
+#define MAX_DEBUG_SIZE (3 * sizeof(void *) + 2 * sizeof(struct track))
+
+/*
  * Set of flags that will prevent slab merging
  */
 #define SLUB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \
@@ -1561,6 +1566,21 @@ slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
 		"default order: %d, min order: %d\n", s->name, s->objsize,
 		s->size, oo_order(s->oo), oo_order(s->min));
 
+	if (s->flags & (SLAB_POISON | SLAB_RED_ZONE | SLAB_STORE_USER)) {
+		int min_order;
+
+		/*
+		 * Debugging is enabled, which may increase oo_order(s->min), so
+		 * warn the user that allocation failures may be avoided if
+		 * debugging is enabled for this cache.
+		 */
+		min_order = get_order(s->size - MAX_DEBUG_SIZE);
+		if (min_order < oo_order(s->min))
+			printk(KERN_WARNING "  %s debugging increased min order "
+			       "from %d to %d, use slab_debug=-,%s to disable.",
+			       s->name, min_order, oo_order(s->min), s->name);
+	}
+
 	for_each_online_node(node) {
 		struct kmem_cache_node *n = get_node(s, node);
 		unsigned long nr_slabs;

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Tue, 30 Jun 2009, Pekka Enberg wrote:

> On Tue, Jun 30, 2009 at 10:47 AM, David Rientjes<rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> > I don't think CONFIG_SLUB_DEBUG_ON is generally the configuration used in
> > the real world.
> 
> It is, hence the epic bug report that's eaten too many man hours
> already! Look, we encourage _testers_ to turn all as much as debugging
> options as possible so we catch bugs early. That why the only sane
> defaults are the ones that don't cause other problems!
> 

I feel that asking a user to add a command line parameter such as 
`slub_debug=A' in addition to CONFIG_SLUB_DEBUG_ON will likely lead to 
less testing coverage and bugs going unreported.  CONFIG_SLUB_DEBUG_ON is 
not something that a distro is going to enable or would be used in a 
production environment, it's something that's used to debug slub and/or 
slab allocations either during the development of new kernel code or when 
an underlying problem is realized.

> I don't know why you want to argue this. It's simply not an option to
> say "stupid user, fix your config" in core code like the slab
> allocator. Enabling CONFIG_SLUB_DEBUG_ON is a very reasonable thing to
> do when you are a tester looking for bugs.
> 

Quite the contrary, I agree completely with the above, and that's why I'm 
arguing for full debugging to be enabled when a well-defined configuration 
option is enabled.  I simply don't believe that such debugging should be 
coupled with a command line option to be fully activated for all caches.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Tue, 30 Jun 2009, David Rientjes wrote:

> I don't see how that's different from enabling debugging on all caches
> like CONFIG_SLAB_DEBUG_ON currently does and then warning at the time of
> slab allocation failure that it may be the result of the debugging
> metadata so the user can subsequently prevent it.  In other words, if we
> use MAX_DEBUG_SIZE as Pekka originally implemented as
> (3 * sizeof(void *) + 2 * sizeof(struct track)), do this:

I like it.

> diff --git a/mm/slub.c b/mm/slub.c
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -142,6 +142,11 @@
>  				SLAB_POISON | SLAB_STORE_USER)
>
>  /*
> + * The maximum amount of metadata added to a slab when debugging is enabled.
> + */
> +#define MAX_DEBUG_SIZE (3 * sizeof(void *) + 2 * sizeof(struct track))
> +
> +/*
>   * Set of flags that will prevent slab merging
>   */
>  #define SLUB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \
> @@ -1561,6 +1566,21 @@ slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
>  		"default order: %d, min order: %d\n", s->name, s->objsize,
>  		s->size, oo_order(s->oo), oo_order(s->min));
>
> +	if (s->flags & (SLAB_POISON | SLAB_RED_ZONE | SLAB_STORE_USER)) {
> +		int min_order;
> +
> +		/*
> +		 * Debugging is enabled, which may increase oo_order(s->min), so
> +		 * warn the user that allocation failures may be avoided if
> +		 * debugging is enabled for this cache.
> +		 */
> +		min_order = get_order(s->size - MAX_DEBUG_SIZE);
> +		if (min_order < oo_order(s->min))
> +			printk(KERN_WARNING "  %s debugging increased min order "
> +			       "from %d to %d, use slab_debug=-,%s to disable.",
> +			       s->name, min_order, oo_order(s->min), s->name);

It may be easier to check the order of the initial size vs. the order of
the size with all metadata

if (get_order(s->size) > get_order(s->objsize)

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Tue, 30 Jun 2009, Christoph Lameter wrote:

> > diff --git a/mm/slub.c b/mm/slub.c
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -142,6 +142,11 @@
> >  				SLAB_POISON | SLAB_STORE_USER)
> >
> >  /*
> > + * The maximum amount of metadata added to a slab when debugging is enabled.
> > + */
> > +#define MAX_DEBUG_SIZE (3 * sizeof(void *) + 2 * sizeof(struct track))
> > +
> > +/*
> >   * Set of flags that will prevent slab merging
> >   */
> >  #define SLUB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \
> > @@ -1561,6 +1566,21 @@ slab_out_of_memory(struct kmem_cache *s, gfp_t gfpflags, int nid)
> >  		"default order: %d, min order: %d\n", s->name, s->objsize,
> >  		s->size, oo_order(s->oo), oo_order(s->min));
> >
> > +	if (s->flags & (SLAB_POISON | SLAB_RED_ZONE | SLAB_STORE_USER)) {
> > +		int min_order;
> > +
> > +		/*
> > +		 * Debugging is enabled, which may increase oo_order(s->min), so
> > +		 * warn the user that allocation failures may be avoided if
> > +		 * debugging is enabled for this cache.
> > +		 */
> > +		min_order = get_order(s->size - MAX_DEBUG_SIZE);
> > +		if (min_order < oo_order(s->min))
> > +			printk(KERN_WARNING "  %s debugging increased min order "
> > +			       "from %d to %d, use slab_debug=-,%s to disable.",
> > +			       s->name, min_order, oo_order(s->min), s->name);
> 
> It may be easier to check the order of the initial size vs. the order of
> the size with all metadata
> 
> if (get_order(s->size) > get_order(s->objsize)
> 

Ah, right.  Then we could simply eliminate the check on s->flags to begin 
with.

This patch is supposing that `slab_debug=-,<cache>' actually disables all 
debugging for <cache> which would need to be implemented first, but I 
think this is a better alternative than requiring slab_debug=A for full 
debugging after enabling CONFIG_SLUB_DEBUG_ON.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Tue, 30 Jun 2009, David Rientjes wrote:

> This patch is supposing that `slab_debug=-,<cache>' actually disables all
> debugging for <cache> which would need to be implemented first, but I
> think this is a better alternative than requiring slab_debug=A for full
> debugging after enabling CONFIG_SLUB_DEBUG_ON.

We could add an option that disables debugging for troublesome page
size slabs


slab_debug=p

or so

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Tue, 30 Jun 2009, Christoph Lameter wrote:

> We could add an option that disables debugging for troublesome page
> size slabs
> 
> 
> slab_debug=p
> 
> or so
> 

I definitely like that more than slab_debug=A, where we're requiring an 
added parameter for full debugging to be activated.

I'm curious whether there would ever be any use for disabling debugging on 
specific caches for reasons other than higher minimum orders for metadata, 
though, given that we already support things like slub_debug=FZ,cache, 
which should only enable free debugging and redzoning even with 
CONFIG_SLUB_DEBUG_ON enabled for cache.

I think the solution to this is really based on good software engineering 
and test practices, though, so hopefully there'll be a consensus on which 
direction to take before any time is spent in implementing and pushing it.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Tue, 30 Jun 2009, David Rientjes wrote:

> I'm curious whether there would ever be any use for disabling debugging on
> specific caches for reasons other than higher minimum orders for metadata,
> though, given that we already support things like slub_debug=FZ,cache,
> which should only enable free debugging and redzoning even with
> CONFIG_SLUB_DEBUG_ON enabled for cache.

One of the reasons for disabling debugging is to speed up the kernel. Race
conditions may vanish due to the additional latency added by the debugging
code. Ideally you know which slab cache has the race and you only would
enable it on that one.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi David,

On Wed, Jul 1, 2009 at 12:52 AM, David Rientjes<rientjes@google.com> wrote:
> I think the solution to this is really based on good software engineering
> and test practices, though, so hopefully there'll be a consensus on which
> direction to take before any time is spent in implementing and pushing it.

Lets go with the slab_out_of_memory() patch you outlined in a previous
post and implement the slub_debug=p thing Christoph suggested. I think
it's the best compromise at this point. When you guys finally see the
light, we can always change it to a reasonable default. ;)

So can you send a patch, please?

                        Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Wed, 1 Jul 2009, Pekka Enberg wrote:

> Lets go with the slab_out_of_memory() patch you outlined in a previous
> post and implement the slub_debug=p thing Christoph suggested. I think
> it's the best compromise at this point. When you guys finally see the
> light, we can always change it to a reasonable default. ;)
> 
> So can you send a patch, please?
> 

Sure, let me know if you think this is -rc material; otherwise, the bug 
will have to be deferred until 2.6.32 with the temporary workaround of 
disabling CONFIG_SLUB_DEBUG_ON.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi David,

On Wed, 1 Jul 2009, Pekka Enberg wrote:
>> Lets go with the slab_out_of_memory() patch you outlined in a previous
>> post and implement the slub_debug=p thing Christoph suggested. I think
>> it's the best compromise at this point. When you guys finally see the
>> light, we can always change it to a reasonable default. ;)
>>
>> So can you send a patch, please?

On Thu, Jul 2, 2009 at 8:18 PM, David Rientjes<rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> Sure, let me know if you think this is -rc material; otherwise, the bug
> will have to be deferred until 2.6.32 with the temporary workaround of
> disabling CONFIG_SLUB_DEBUG_ON.

We're at -rc2 so yes, I do think we should fix 2.6.31.

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of regressions introduced between 2.6.29 and 2.6.30.

The following bug entry is on the current list of known regressions
introduced between 2.6.29 and 2.6.30.  Please verify if it still should
be listed and let me know (either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (69 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
		  http://lkml.org/lkml/2009/6/7/136
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of regressions introduced between 2.6.29 and 2.6.30.
> 
> The following bug entry is on the current list of known regressions
> introduced between 2.6.29 and 2.6.30.  Please verify if it still should
> be listed and let me know (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> Subject		: Page allocation failures with b43 and p54usb
> Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> Date		: 2009-04-29 21:01 (69 days old)
> References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> 		  http://lkml.org/lkml/2009/6/7/136
> Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

I have not seen the fix yet, but one is being prepared. I think the
bug can be closed.

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Mon, 6 Jul 2009, Larry Finger wrote:

> I have not seen the fix yet, but one is being prepared. I think the
> bug can be closed.
> 

http://patchwork.kernel.org/patch/34385 should fix your issue (try with 
CONFIG_SLUB_DEBUG_ON still enabled and slub_debug=O on the command line).

Pekka indicated he wanted this fixed in 2.6.31, so he'll probably push it 
to Linus before the 2.6.31-rc3.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi,

On Mon, 6 Jul 2009, Larry Finger wrote:
>> I have not seen the fix yet, but one is being prepared. I think the
>> bug can be closed.

On Tue, Jul 7, 2009 at 9:29 AM, David Rientjes<rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> http://patchwork.kernel.org/patch/34385 should fix your issue (try with
> CONFIG_SLUB_DEBUG_ON still enabled and slub_debug=O on the command line).
>
> Pekka indicated he wanted this fixed in 2.6.31, so he'll probably push it
> to Linus before the 2.6.31-rc3.

Yup, if it's not too much trouble Larry, I'd appreciate if you gave it
a spin before I apply the sucker. Note to regression trackers, I don't
think this should go to -stable because it depends on other patches in
mainline and there's an obvious workaround for the problem (disable
CONFIG_SLUB_DEBUG_ON). Furthermore, it's very unlikely people will hit
this on production configurations anyway.

                        Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Pekka Enberg wrote:
> 
> Yup, if it's not too much trouble Larry, I'd appreciate if you gave it
> a spin before I apply the sucker. Note to regression trackers, I don't
> think this should go to -stable because it depends on other patches in
> mainline and there's an obvious workaround for the problem (disable
> CONFIG_SLUB_DEBUG_ON). Furthermore, it's very unlikely people will hit
> this on production configurations anyway.

Pekka,

I have been running the patch for ~24 hours with slub_debug=0 as a
boot option. So far, no problems. My workload has not involved as many
network operations as some times, but my memory has not fragmented.
The output of 'cat /proc/buddyinfo' is as follows:

Node 0, zone      DMA      8      5      5      4      5      5      5
     4      3      0      1
Node 0, zone    DMA32   1804   1585   3007     36      0      0      0
     0      1      0      0

As might be expected, the high-order fragments are gone, but the O(1)
pieces have not been depleted.

Larry

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of regressions introduced between 2.6.29 and 2.6.30.

The following bug entry is on the current list of known regressions
introduced between 2.6.29 and 2.6.30.  Please verify if it still should
be listed and let me know (either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger@lwfinger.net>
Date		: 2009-04-29 21:01 (89 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
		  http://lkml.org/lkml/2009/6/7/136
Handled-By	: Johannes Berg <johannes@sipsolutions.net>

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Larry Finger]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of regressions introduced between 2.6.29 and 2.6.30.
> 
> The following bug entry is on the current list of known regressions
> introduced between 2.6.29 and 2.6.30.  Please verify if it still should
> be listed and let me know (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> Subject		: Page allocation failures with b43 and p54usb
> Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> Date		: 2009-04-29 21:01 (89 days old)
> References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> 		  http://lkml.org/lkml/2009/6/7/136
> Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>

This bug was fixed by commit 781b2ba6eb5f22440afac9c79a89ebd6e3674a60
entitled "SLUB: Out-of-memory diagnostics" by Pekka Enberg
<penberg-bbCR+/B0CizivPeTLB3BmA@public.gmane.org> and dated Wed Jun 10 18:50:32 2009 +0300.

The fault occurred because when CONFIG_SLUB_DEBUG was set, many of the
memory allocations for wireless buffers were increased from O(0) to
O(1) causing memory fragmentation, and eventually there were no
remaining O(1) fragments. This problem is unlikely to affect most
users as none of the distos turn on the above debug option.

This bug should be marked "closed with patch".

Larry

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Sun, 26 Jul 2009, Larry Finger wrote:

> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of regressions introduced between 2.6.29 and 2.6.30.
> > 
> > The following bug entry is on the current list of known regressions
> > introduced between 2.6.29 and 2.6.30.  Please verify if it still should
> > be listed and let me know (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
> > Subject		: Page allocation failures with b43 and p54usb
> > Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> > Date		: 2009-04-29 21:01 (89 days old)
> > References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> > 		  http://lkml.org/lkml/2009/6/7/136
> > Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
> 
> This bug was fixed by commit 781b2ba6eb5f22440afac9c79a89ebd6e3674a60
> entitled "SLUB: Out-of-memory diagnostics" by Pekka Enberg
> <penberg-bbCR+/B0CizivPeTLB3BmA@public.gmane.org> and dated Wed Jun 10 18:50:32 2009 +0300.
> 

Hmm, I'm remembering differently.  I thought the root problem here has 
only been fixed in Pekka's slab-2.6.git tree with "slub: add option to 
disable higher order debugging slabs" and isn't currently in Linus' tree.

> The fault occurred because when CONFIG_SLUB_DEBUG was set, many of the
> memory allocations for wireless buffers were increased from O(0) to
> O(1) causing memory fragmentation, and eventually there were no
> remaining O(1) fragments. This problem is unlikely to affect most
> users as none of the distos turn on the above debug option.
> 

It only happens when CONFIG_SLUB_DEBUG_ON is enabled for caches with 
sizes that increase as the result of the added metadata.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

On Mon, Jul 27, 2009 at 3:24 AM, David Rientjes<rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> On Sun, 26 Jul 2009, Larry Finger wrote:
>
>> Rafael J. Wysocki wrote:
>> > This message has been generated automatically as a part of a report
>> > of regressions introduced between 2.6.29 and 2.6.30.
>> >
>> > The following bug entry is on the current list of known regressions
>> > introduced between 2.6.29 and 2.6.30.  Please verify if it still should
>> > be listed and let me know (either way).
>> >
>> >
>> > Bug-Entry   : http://bugzilla.kernel.org/show_bug.cgi?id=13319
>> > Subject             : Page allocation failures with b43 and p54usb
>> > Submitter   : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
>> > Date                : 2009-04-29 21:01 (89 days old)
>> > References  : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>> >               http://lkml.org/lkml/2009/6/7/136
>> > Handled-By  : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
>>
>> This bug was fixed by commit 781b2ba6eb5f22440afac9c79a89ebd6e3674a60
>> entitled "SLUB: Out-of-memory diagnostics" by Pekka Enberg
>> <penberg-bbCR+/B0CizivPeTLB3BmA@public.gmane.org> and dated Wed Jun 10 18:50:32 2009 +0300.
>>
>
> Hmm, I'm remembering differently.  I thought the root problem here has
> only been fixed in Pekka's slab-2.6.git tree with "slub: add option to
> disable higher order debugging slabs" and isn't currently in Linus' tree.

Yup, the fix is in slab.git and queued for 2.6.32. There was some
complaints from Christoph from the patch that need to be addressed
still.

                                        Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1169 bytes --]

On Mon, 27 Jul 2009, Pekka Enberg wrote:

> > Hmm, I'm remembering differently.  I thought the root problem here has
> > only been fixed in Pekka's slab-2.6.git tree with "slub: add option to
> > disable higher order debugging slabs" and isn't currently in Linus' tree.
> 
> Yup, the fix is in slab.git and queued for 2.6.32. There was some
> complaints from Christoph from the patch that need to be addressed
> still.
> 

From what I recall, he asked that calculate_sizes() be called twice, first 
to determine if get_order(s->size) increased as the result of the metadata 
and, if so, a second time with the flags disabled.

slab_debug=O only disables debugging options that increase the min order 
of slab as defined in DEBUG_FLAGS; it doesn't selectively disable some of 
them when get_order(s->size) grows.  So it's quite sane, like my patch 
does, to disable all DEBUG_FLAGS when

	get_order(s->objsize) + DEBUG_SIZE_FLAGS > get_order(s->objsize)

without calling calculate_sizes() twice.

We need DEBUG_FLAGS to determine which flags to mask off to reduce the 
minimum order, so I don't see DEBUG_FLAGS_SIZE as troublesome.

Christoph?

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Mon, 27 Jul 2009, David Rientjes wrote:

> We need DEBUG_FLAGS to determine which flags to mask off to reduce the
> minimum order, so I don't see DEBUG_FLAGS_SIZE as troublesome.
>
> Christoph?

Post a patch? Otherwise just go ahead.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Mon, 27 Jul 2009, Christoph Lameter wrote:

> On Mon, 27 Jul 2009, David Rientjes wrote:
> 
> > We need DEBUG_FLAGS to determine which flags to mask off to reduce the
> > minimum order, so I don't see DEBUG_FLAGS_SIZE as troublesome.
> >
> > Christoph?
> 
> Post a patch? Otherwise just go ahead.
> 
> 

My patch is already in Pekka's slab-2.6.git tree at 
http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commit;h=fa5ec8a1f66f3c2a3af723abcf8085509c9ee682

You had proposed http://marc.info/?l=linux-kernel&m=124725166205814, which 
moves the mask to kmem_cache_open() and calls calculate_sizes() twice.  
That eliminates DEBUG_FLAGS_SIZE, but I don't see that define as being 
troublesome since we must define DEBUG_FLAGS to specify what options add 
metdata anyway.

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Christoph Lameter]

On Mon, 27 Jul 2009, David Rientjes wrote:

> My patch is already in Pekka's slab-2.6.git tree at
> http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commit;h=fa5ec8a1f66f3c2a3af723abcf8085509c9ee682
>
> You had proposed http://marc.info/?l=linux-kernel&m=124725166205814, which
> moves the mask to kmem_cache_open() and calls calculate_sizes() twice.
> That eliminates DEBUG_FLAGS_SIZE, but I don't see that define as being
> troublesome since we must define DEBUG_FLAGS to specify what options add
> metdata anyway.

My prosal was to use the size and objsize parameters. You would only have
to call calculate_sizes() twice when the comparison of the order of size
and objsize would be different.

Doing so would simplify additing future flags. If you do your own
calculations (like in the patch) then you have to replicate the size
calculation from calculate_sizes() somehow. Is the duplicate calculation
really accurate regarding alignment and other special casing?

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[David Rientjes]

On Mon, 27 Jul 2009, Christoph Lameter wrote:

> My prosal was to use the size and objsize parameters. You would only have
> to call calculate_sizes() twice when the comparison of the order of size
> and objsize would be different.
> 
> Doing so would simplify additing future flags. If you do your own
> calculations (like in the patch) then you have to replicate the size
> calculation from calculate_sizes() somehow. Is the duplicate calculation
> really accurate regarding alignment and other special casing?
> 

Ok, fair enough.  It seems like a matter of taste in implementation but 
your proposal is also more extendable than mine, and I'm definitely not 
going to argue your taste vs. mine when it comes to slub :)

I'll write an incremental patch on top of Pekka's for-next branch to 
implement your idea.

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of regressions introduced between 2.6.29 and 2.6.30.

The following bug entry is on the current list of known regressions
introduced between 2.6.29 and 2.6.30.  Please verify if it still should
be listed and let me know (either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (96 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
		  http://lkml.org/lkml/2009/6/7/136
		  http://lkml.org/lkml/2009/7/26/213
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
		  David Rientjes <rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/37655/

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of regressions introduced between 2.6.29 and 2.6.30.

The following bug entry is on the current list of known regressions
introduced between 2.6.29 and 2.6.30.  Please verify if it still should
be listed and let me know (either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger@lwfinger.net>
Date		: 2009-04-29 21:01 (103 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
		  http://lkml.org/lkml/2009/6/7/136
		  http://lkml.org/lkml/2009/7/26/213
Handled-By	: Johannes Berg <johannes@sipsolutions.net>
		  David Rientjes <rientjes@google.com>
Patch		: http://patchwork.kernel.org/patch/37655/

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of regressions introduced between 2.6.29 and 2.6.30.

The following bug entry is on the current list of known regressions
introduced between 2.6.29 and 2.6.30.  Please verify if it still should
be listed and let me know (either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (113 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
		  http://lkml.org/lkml/2009/6/7/136
		  http://lkml.org/lkml/2009/7/26/213
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
		  David Rientjes <rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/37655/

[Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of regressions introduced between 2.6.29 and 2.6.30.

The following bug entry is on the current list of known regressions
introduced between 2.6.29 and 2.6.30.  Please verify if it still should
be listed and let me know (either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=13319
Subject		: Page allocation failures with b43 and p54usb
Submitter	: Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
Date		: 2009-04-29 21:01 (119 days old)
References	: http://marc.info/?l=linux-kernel&m=124103897101088&w=4
		  http://lkml.org/lkml/2009/6/7/136
		  http://lkml.org/lkml/2009/7/26/213
Handled-By	: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
		  David Rientjes <rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/37655/

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Pekka Enberg]

Hi Rafael,

On Wed, Aug 26, 2009 at 12:05 AM, Rafael J. Wysocki<rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> This message has been generated automatically as a part of a report
> of regressions introduced between 2.6.29 and 2.6.30.
>
> The following bug entry is on the current list of known regressions
> introduced between 2.6.29 and 2.6.30.  Please verify if it still should
> be listed and let me know (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=13319
> Subject         : Page allocation failures with b43 and p54usb
> Submitter       : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> Date            : 2009-04-29 21:01 (119 days old)
> References      : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
>                  http://lkml.org/lkml/2009/6/7/136
>                  http://lkml.org/lkml/2009/7/26/213
> Handled-By      : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
>                  David Rientjes <rientjes-hpIqsD4AKldhl2p70BpVqQ@public.gmane.orgm>
> Patch           : http://patchwork.kernel.org/patch/37655/

FYI, the fix (a new slub debugging option) is queued for 2.6.32 and we
don't have plans to backport it to -stable because it depends on SLUB
out-of-memory diagnostic patches and the problem doesn't affect
production configs.

                                Pekka

Re: [Bug #13319] Page allocation failures with b43 and p54usb

[Rafael J. Wysocki]

On Wednesday 26 August 2009, Pekka Enberg wrote:
> Hi Rafael,
> 
> On Wed, Aug 26, 2009 at 12:05 AM, Rafael J. Wysocki<rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> > This message has been generated automatically as a part of a report
> > of regressions introduced between 2.6.29 and 2.6.30.
> >
> > The following bug entry is on the current list of known regressions
> > introduced between 2.6.29 and 2.6.30.  Please verify if it still should
> > be listed and let me know (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=13319
> > Subject         : Page allocation failures with b43 and p54usb
> > Submitter       : Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
> > Date            : 2009-04-29 21:01 (119 days old)
> > References      : http://marc.info/?l=linux-kernel&m=124103897101088&w=4
> >                  http://lkml.org/lkml/2009/6/7/136
> >                  http://lkml.org/lkml/2009/7/26/213
> > Handled-By      : Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
> >                  David Rientjes <rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> > Patch           : http://patchwork.kernel.org/patch/37655/
> 
> FYI, the fix (a new slub debugging option) is queued for 2.6.32 and we
> don't have plans to backport it to -stable because it depends on SLUB
> out-of-memory diagnostic patches and the problem doesn't affect
> production configs.

Thanks for the info, I've closed the bug as "will fix later".

Rafael