2.6.32-rc5-git3: Reported regressions from 2.6.31

2.6.32-rc5-git3: Reported regressions from 2.6.31

[Rafael J. Wysocki]

This message contains a list of some regressions from 2.6.31, for which there
are no fixes in the mainline I know of.  If any of them have been fixed already,
please let me know.

If you know of any other unresolved regressions from 2.6.31, please let me know
either and I'll add them to the list.  Also, please let me know if any of the
entries below are invalid.

Each entry from the list will be sent additionally in an automatic reply to
this message with CCs to the people involved in reporting and handling the
issue.


Listed regressions statistics:

  Date          Total  Pending  Unresolved
  ----------------------------------------
  2009-10-26       66       42          37
  2009-10-12       48       31          27
  2009-10-02       22       15           9


Unresolved regressions
----------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14485
Subject		: System lockup running "cat /sys/kernel/debug/dri/0/i915_regs"
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-26 4:00 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125652968117713&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14484
Subject		: no video output after suspend
Submitter	: Riccardo Magliocchetti <riccardo.magliocchetti-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-25 20:57 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125650430123713&w=4
Handled-By	: Jesse Barnes <jbarnes-Y1mF5jBUw70BENJcbMCuUQ@public.gmane.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14483
Subject		: WARNING: at drivers/base/sys.c:353 __sysdev_resume+0x54/0xca()
Submitter	: Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-25 19:58 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125650070420168&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14482
Subject		: kernel BUG at fs/dcache.c:670 +lvm +md +ext3
Submitter	: Alexander Clouter <alex-L4GPcECwBoDe9xe1eoZjHA@public.gmane.org>
Date		: 2009-10-23 10:30 (4 days old)
References	: http://lkml.org/lkml/2009/10/23/50


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14481
Subject		: umount blocked for more than 120 seconds after USB drive removal
Submitter	: Robert Hancock <hancockrwd-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-21 5:26 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125610280532245&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14479
Subject		: nfs oops
Submitter	: Egon Alter <egon.alter-hi6Y0CQ0nG0@public.gmane.org>
Date		: 2009-10-19 16:03 (8 days old)
References	: http://marc.info/?l=linux-kernel&m=125596822630410&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14477
Subject		: possible circular locking dependency in ISDN PPP
Submitter	: Tilman Schmidt <tilman-ZTO5kqT2PaM@public.gmane.org>
Date		: 2009-10-18 22:16 (9 days old)
References	: http://marc.info/?l=linux-kernel&m=125590423416087&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14473
Subject		: ATA related kernel warning after resume
Submitter	: Tino Keitel <tino.keitel-rAwCM5oiXHA@public.gmane.org>
Date		: 2009-10-14 6:55 (13 days old)
References	: http://marc.info/?l=linux-kernel&m=125550466624678&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14472
Subject		: EXT4 corruption
Submitter	: Shawn Starr <shawn.starr-bJEeYj9oJeDQT0dZR+AlfA@public.gmane.org>
Date		: 2009-10-13 2:07 (14 days old)
References	: http://marc.info/?l=linux-kernel&m=125539997508256&w=4
Handled-By	: Theodore Tso <tytso-3s7WtUTddSA@public.gmane.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14467
Subject		: Linker errors on ia64 with NR_CPUS=4096
Submitter	: Jeff Mahoney <jeffm-IBi9RG/b67k@public.gmane.org>
Date		: 2009-10-18 22:28 (9 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=34d76c41554a05425613d16efebb3069c4c545f0
References	: http://marc.info/?l=linux-kernel&m=125590493116720&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14466
Subject		: EFI boot on x86 fails in .32
Submitter	: Matthew Garrett <mjg59-1xO5oi07KQx4cg9Nei1l7Q@public.gmane.org>
Date		: 2009-10-20 0:34 (7 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=7bd867dfb4e0357e06a3211ab2bd0e714110def3
References	: http://marc.info/?l=linux-kernel&m=125599887314290&w=4
Handled-By	: Feng Tang <feng.tang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14442
Subject		: resume after hibernate: /dev/sdb drops and returns as /dev/sde
Submitter	: Duncan <1i5t5.duncan-j9pdmedNgrk@public.gmane.org>
Date		: 2009-10-20 01:52 (7 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14430
Subject		: sync() hangs in bdi_sched_wait
Submitter	: Petr Vandrovec <petr-vPk2MGR0e28uaRcfnNAh7A@public.gmane.org>
Date		: 2009-10-17 19:14 (10 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14415
Subject		: Reboot on kernel load
Submitter	: Brian Beardall <brian-sVkzCUl/XCrR7s880joybQ@public.gmane.org>
Date		: 2009-10-15 23:57 (12 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14408
Subject		: sysctl check failed
Submitter	: Peter Teoh <htmldeveloper-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-14 22:59 (13 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14406
Subject		: uvcvideo stopped work on Toshiba
Submitter	: okias <d.okias-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-14 19:08 (13 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel-VkQ1JFuSMpfAbQlEx87xDw@public.gmane.org>
Date		: 2009-10-11 0:04 (16 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Date		: 2009-10-09 8:58 (18 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f@public.gmane.org>
Date		: 2009-10-07 3:00 (20 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14384
Subject		: tbench regression with 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Date		: 2009-10-09 9:51 (18 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59abf02644c45f1591e1374ee7bb45dc757fcb88
References	: http://marc.info/?l=linux-kernel&m=125508216713138&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra-/NLkJaSkS4VmR6Xm/wNWPw@public.gmane.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Date		: 2009-10-09 9:19 (18 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra-/NLkJaSkS4VmR6Xm/wNWPw@public.gmane.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
Subject		: iwlagn lost connection after s2ram (with warnings)
Submitter	: Carlos R. Mafra <crmafra2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-07 14:20 (20 days old)
References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
Date		: 2009-10-08 14:51 (19 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-06 14:38 (21 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-02 10:16 (25 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: ath5k wireless not working after suspend-resume - eeepc
Submitter	: Fabio Comolli <fabio.comolli-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-03 15:36 (24 days old)
References	: http://lkml.org/lkml/2009/10/3/91


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
Submitter	: Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
Date		: 2009-10-10 03:07 (17 days old)
References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
Subject		: Bad corruption with 2.6.32-rc1 and upwards
Submitter	: Holger Freyther <zecke-MQnelBtSfJRAfugRpC6u6w@public.gmane.org>
Date		: 2009-10-09 15:42 (18 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-05 3:39 (22 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 00:30 (19 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14334
Subject		: pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m
Submitter	: Jose Marino <braket-PkbjNfxxIARBDgjK7y7TUQ@public.gmane.org>
Date		: 2009-10-06 15:44 (21 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14331
Subject		: Radeon XPRESS 200M: System hang with radeon DRI and Fedora 10 userspace unless DRI=off
Submitter	: Alex Villacis Lasso <avillaci-x0m+Mc+nT7uljOmnV8AmnkElSqmLX1BE@public.gmane.org>
Date		: 2009-10-06 00:29 (21 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
Subject		: oops in wireless, iwl3945 related?
Submitter	: Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org>
Date		: 2009-09-29 17:12 (28 days old)
References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14298
Subject		: warning at manage.c:361 (set_irq_wake), matrix-keypad related?
Submitter	: Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org>
Date		: 2009-09-30 20:07 (27 days old)
References	: http://marc.info/?l=linux-kernel&m=125434130703538&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org>
Date		: 2009-09-30 15:11 (27 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14296
Subject		: spitz boots but suspend/resume is broken
Submitter	: Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org>
Date		: 2009-09-30 12:06 (27 days old)
References	: http://marc.info/?l=linux-kernel&m=125431244516449&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
Date		: 2009-09-30 18:06 (27 days old)


Regressions with patches
------------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14480
Subject		: 2 locks held by cat -- running "find /sys | head -c 4" --> system hang
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-20 16:11 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125605511728088&w=4
Handled-By	: Chris Wilson <chris-Y6uKTt2uX1cEflXRtASbqLVCufUGDwFn@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/54974/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso-3s7WtUTddSA@public.gmane.org>
Date		: 2009-10-02 22:40 (25 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes-Y1mF5jBUw70BENJcbMCuUQ@public.gmane.org>
Patch		: http://marc.info/?l=linux-kernel&m=125591495325000&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 21:46 (19 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
Handled-By	: Alexey Starikovskiy <astarikovskiy-l3A5Bk7waGM@public.gmane.org>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-02 9:46 (25 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/51808/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-01 3:23 (26 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/50813/


For details, please visit the bug entries and follow the links given in
references.

As you can see, there is a Bugzilla entry for each of the listed regressions.
There also is a Bugzilla entry used for tracking the regressions from 2.6.31,
unresolved as well as resolved, at:

http://bugzilla.kernel.org/show_bug.cgi?id=14230

Please let me know if there are any Bugzilla entries that should be added to
the list in there.

Thanks,
Rafael

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Bug #14277] Caught 8-bit read from freed memory in b43 driver at association

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian-GANU6spQydw@public.gmane.org>
Date		: 2009-09-30 18:06 (27 days old)

[Bug #14299] oops in wireless, iwl3945 related?

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
Subject		: oops in wireless, iwl3945 related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-29 17:12 (28 days old)
References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4

Re: [Bug #14299] oops in wireless, iwl3945 related?

[Pavel Machek]

On Mon 2009-10-26 19:55:50, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
> Subject		: oops in wireless, iwl3945 related?
> Submitter	: Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org>
> Date		: 2009-09-29 17:12 (28 days old)
> References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4

It did not happen again, and there were some fixes in that area. (It
happened 2 times total). Close it, I guess?
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [Bug #14299] oops in wireless, iwl3945 related?

[Rafael J. Wysocki]

On Tuesday 27 October 2009, Pavel Machek wrote:
> On Mon 2009-10-26 19:55:50, Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
> > Subject		: oops in wireless, iwl3945 related?
> > Submitter	: Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org>
> > Date		: 2009-09-29 17:12 (28 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4
> 
> It did not happen again, and there were some fixes in that area. (It
> happened 2 times total). Close it, I guess?

OK, closing.

Thanks,
Rafael

[Bug #14296] spitz boots but suspend/resume is broken

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14296
Subject		: spitz boots but suspend/resume is broken
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 12:06 (27 days old)
References	: http://marc.info/?l=linux-kernel&m=125431244516449&w=4

[Bug #14297] console resume broken since ba15ab0e8d

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer@pengutronix.de>
Date		: 2009-09-30 15:11 (27 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4

[Bug #14298] warning at manage.c:361 (set_irq_wake), matrix-keypad related?

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14298
Subject		: warning at manage.c:361 (set_irq_wake), matrix-keypad related?
Submitter	: Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org>
Date		: 2009-09-30 20:07 (27 days old)
References	: http://marc.info/?l=linux-kernel&m=125434130703538&w=4

[Bug #14331] Radeon XPRESS 200M: System hang with radeon DRI and Fedora 10 userspace unless DRI=off

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14331
Subject		: Radeon XPRESS 200M: System hang with radeon DRI and Fedora 10 userspace unless DRI=off
Submitter	: Alex Villacis Lasso <avillaci-x0m+Mc+nT7uljOmnV8AmnkElSqmLX1BE@public.gmane.org>
Date		: 2009-10-06 00:29 (21 days old)

[Bug #14302] Kernel panic on i386 machine when booting with profile=2

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi@intel.com>
Date		: 2009-10-01 3:23 (26 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi@intel.com>
Patch		: http://patchwork.kernel.org/patch/50813/

[Bug #14352] WARNING: at net/mac80211/scan.c:267

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 00:30 (19 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7

[Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14334
Subject		: pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m
Submitter	: Jose Marino <braket-PkbjNfxxIARBDgjK7y7TUQ@public.gmane.org>
Date		: 2009-10-06 15:44 (21 days old)

Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

Hi,

On Monday 26 October 2009, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).

Here's a puzzle for whoever likes hardware detective work.

Suspend to RAM worked just fine on the Jose's box before commit
53024df259e37ad49ee3d1f3721d4cecdd7bc357 (PM / yenta: Fix cardbus
suspend/resume regression, mainline commit
0c570cdeb8fdfcb354a3e9cd81bfc6a09c19de0c) that made the resume of CardBus
devices actually work.  After this commit, the box hangs during resume 100% of
the time.

We've done quite some debugging and here's the summary of findings:

1) Resume works if pcmcia_socket_dev_resume(dev) is moved to the "regular"
   resume phase, after resume_device_irqs().

2) Resume works if yenta_set_power() is not called during resume
   (from yenta_set_socket()).

3) It doesn't help to resume all ACPI devices before yenta.

4) It doesn't help to disable yenta events during resume (unconditionally).

5) There are two CardBus bridges in the box and according to the PM_TRACE
   information the _second_ one is the last device we attempt to wake up
   during a failing resume.

Please look into http://bugzilla.kernel.org/show_bug.cgi?id=14334 for details.

OK, any ideas anyone?

Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Linus Torvalds]

On Fri, 30 Oct 2009, Rafael J. Wysocki wrote:
> 
>
> 1) Resume works if pcmcia_socket_dev_resume(dev) is moved to the "regular"
>    resume phase, after resume_device_irqs().

Hmm. We really probably shouldn't call pcmcia_socket_dev_resume() in 
early_resume. It takes mutexes etc, and it calls "socket_resume()", which 
sleeps etc. That per se should be ok these days (since we don't actualyl 
disable CPU irq's, just device irqs), but it also does that whole card 
insertion events etc. And _that_ code I wouldn't trust at all.

The PCMCIA code is better than it used to be a long time ago, but some of 
it is still pretty crazy. 

I get the feeling that we should just revert that commit 0c570cdeb, and 
instead always do PCMCIA suspend as a "eject" event. That way we have no 
driver behind it to resume at resume time - and we'll see any plugged-in 
device as just a new insertion. 

		Linus

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Friday 30 October 2009, Linus Torvalds wrote:
> 
> On Fri, 30 Oct 2009, Rafael J. Wysocki wrote:
> > 
> >
> > 1) Resume works if pcmcia_socket_dev_resume(dev) is moved to the "regular"
> >    resume phase, after resume_device_irqs().
> 
> Hmm. We really probably shouldn't call pcmcia_socket_dev_resume() in 
> early_resume. It takes mutexes etc, and it calls "socket_resume()", which 
> sleeps etc. That per se should be ok these days (since we don't actualyl 
> disable CPU irq's, just device irqs), but it also does that whole card 
> insertion events etc. And _that_ code I wouldn't trust at all.

I thought so when I worked on commit 0c570cdeb, but then it turned out to
work just fine with a number of boxes.

> The PCMCIA code is better than it used to be a long time ago, but some of 
> it is still pretty crazy. 
> 
> I get the feeling that we should just revert that commit 0c570cdeb,

Well, there's nothing wrong with doing the PCI stuff and restoring the state at
the _noirq stage IMO, so instead of reverting it altogether, I'd add
yenta_dev_suspend|resume() that would just call
pcmcia_socket_dev_suspend|resume() during "regular" suspend|resume.

> and instead always do PCMCIA suspend as a "eject" event. That way we have no 
> driver behind it to resume at resume time - and we'll see any plugged-in 
> device as just a new insertion. 

In fact I thought about that.

It looks like I need to find a CardBus adapter somewhere and clean that thing up.

That said, I'd really like to know what's going on in there. :-)

Thanks,
Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Linus Torvalds]

On Fri, 30 Oct 2009, Rafael J. Wysocki wrote:

> On Friday 30 October 2009, Linus Torvalds wrote:

> > The PCMCIA code is better than it used to be a long time ago, but some of 
> > it is still pretty crazy. 
> > 
> > I get the feeling that we should just revert that commit 0c570cdeb,
> 
> Well, there's nothing wrong with doing the PCI stuff and restoring the state at
> the _noirq stage IMO, so instead of reverting it altogether, I'd add
> yenta_dev_suspend|resume() that would just call
> pcmcia_socket_dev_suspend|resume() during "regular" suspend|resume.

Oh, that sounds fine too. I didn't mean that we had to do an outright 
revert, since we'd still need to do something else to fix the original 
issue.

> > and instead always do PCMCIA suspend as a "eject" event. That way we have no 
> > driver behind it to resume at resume time - and we'll see any plugged-in 
> > device as just a new insertion. 
> 
> In fact I thought about that.
> 
> It looks like I need to find a CardBus adapter somewhere and clean that thing up.
> 
> That said, I'd really like to know what's going on in there. :-)

I have to admit that my last two laptops haven't even _had_ a PCMCIA slot, 
so I'm unlikely to be able to help much. But I could dig out an old one, 
and try to find the one wireless card I know I have in some corner.

And partly exactly _because_ even Cardbus is starting to be "legacy", I'd 
personally prefer to try to simplify the model to the point where we don't 
have to think about all the subtle interactions. Just making suspend act 
as an eject would mean that we'd never have to worry about how the CardBus 
bridge interacts with the PCI layer at suspend/resume time.

		Linus

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Linus Torvalds]

On Fri, 30 Oct 2009, Linus Torvalds wrote:
> 
> And partly exactly _because_ even Cardbus is starting to be "legacy", I'd 
> personally prefer to try to simplify the model to the point where we don't 
> have to think about all the subtle interactions. Just making suspend act 
> as an eject would mean that we'd never have to worry about how the CardBus 
> bridge interacts with the PCI layer at suspend/resume time.

Put another way: five years ago I would have felt that it could be 
important that people can suspend and resume while they have a CD-ROM 
mounted through a PCMCIA IDE card. Or something like that where you want 
to keep session information.

These days, that scenario is less interesting to begin with, and we're 
generally better at some of the hotplug issues anyway. Example: one of the 
reasons I used to like not causing an unplug event was because I had 
network cards, and hated setting up the connection again. These days, all 
distros come with networkmanager or similar, and hotplug networking just 
works (even if the "CD-ROM mounted" case probably still would cause 
problems).

So I think we used to have good reasons to try to maintain state over a 
suspend event, but many of those reasons have become weaker, while at the 
same time USB has meant that PCMCIA itself has become more of a 
"maintenance burden" rather than a "primary subsystem".

			Linus

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Friday 30 October 2009, Linus Torvalds wrote:
> 
> On Fri, 30 Oct 2009, Linus Torvalds wrote:
> > 
> > And partly exactly _because_ even Cardbus is starting to be "legacy", I'd 
> > personally prefer to try to simplify the model to the point where we don't 
> > have to think about all the subtle interactions. Just making suspend act 
> > as an eject would mean that we'd never have to worry about how the CardBus 
> > bridge interacts with the PCI layer at suspend/resume time.
> 
> Put another way: five years ago I would have felt that it could be 
> important that people can suspend and resume while they have a CD-ROM 
> mounted through a PCMCIA IDE card. Or something like that where you want 
> to keep session information.
> 
> These days, that scenario is less interesting to begin with, and we're 
> generally better at some of the hotplug issues anyway. Example: one of the 
> reasons I used to like not causing an unplug event was because I had 
> network cards, and hated setting up the connection again. These days, all 
> distros come with networkmanager or similar, and hotplug networking just 
> works (even if the "CD-ROM mounted" case probably still would cause 
> problems).
> 
> So I think we used to have good reasons to try to maintain state over a 
> suspend event, but many of those reasons have become weaker, while at the 
> same time USB has meant that PCMCIA itself has become more of a 
> "maintenance burden" rather than a "primary subsystem".

I agree.

Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Benjamin Herrenschmidt]

> Put another way: five years ago I would have felt that it could be 
> important that people can suspend and resume while they have a CD-ROM 
> mounted through a PCMCIA IDE card. Or something like that where you want 
> to keep session information.

I think you are a bit too much laptop centric :-) It's sadly still
common in the embedded space to use PCMCIA or Cardbus for ... root
filesystems.

Cheers,
Ben.

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Benjamin Herrenschmidt]

On Fri, 2009-10-30 at 12:47 -0700, Linus Torvalds wrote:
> 
> On Fri, 30 Oct 2009, Rafael J. Wysocki wrote:
> > 
> >
> > 1) Resume works if pcmcia_socket_dev_resume(dev) is moved to the "regular"
> >    resume phase, after resume_device_irqs().
> 
> Hmm. We really probably shouldn't call pcmcia_socket_dev_resume() in 
> early_resume. It takes mutexes etc, and it calls "socket_resume()", which 
> sleeps etc. That per se should be ok these days (since we don't actualyl 
> disable CPU irq's, just device irqs), but it also does that whole card 
> insertion events etc. And _that_ code I wouldn't trust at all.
> 
> The PCMCIA code is better than it used to be a long time ago, but some of 
> it is still pretty crazy. 
> 
> I get the feeling that we should just revert that commit 0c570cdeb, and 
> instead always do PCMCIA suspend as a "eject" event. That way we have no 
> driver behind it to resume at resume time - and we'll see any plugged-in 
> device as just a new insertion. 

To me the proper approach would be to split it so that

 - early_resume() restores power & config space etc... so that existing
devices can move on (might check for removal). There's no other hotplug
activity

 - normal resume() restarts handling of events such as insertions

Now, while I do have some cardbus & pcmcia stuff somewhere, I also don't
have much time to hack on this right now...

Cheers,
Ben.

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Saturday 31 October 2009, Benjamin Herrenschmidt wrote:
> On Fri, 2009-10-30 at 12:47 -0700, Linus Torvalds wrote:
> > 
> > On Fri, 30 Oct 2009, Rafael J. Wysocki wrote:
> > > 
> > >
> > > 1) Resume works if pcmcia_socket_dev_resume(dev) is moved to the "regular"
> > >    resume phase, after resume_device_irqs().
> > 
> > Hmm. We really probably shouldn't call pcmcia_socket_dev_resume() in 
> > early_resume. It takes mutexes etc, and it calls "socket_resume()", which 
> > sleeps etc. That per se should be ok these days (since we don't actualyl 
> > disable CPU irq's, just device irqs), but it also does that whole card 
> > insertion events etc. And _that_ code I wouldn't trust at all.
> > 
> > The PCMCIA code is better than it used to be a long time ago, but some of 
> > it is still pretty crazy. 
> > 
> > I get the feeling that we should just revert that commit 0c570cdeb, and 
> > instead always do PCMCIA suspend as a "eject" event. That way we have no 
> > driver behind it to resume at resume time - and we'll see any plugged-in 
> > device as just a new insertion. 
> 
> To me the proper approach would be to split it so that

Well, agreed, but ...

>  - early_resume() restores power & config space etc... so that existing
> devices can move on (might check for removal). There's no other hotplug
> activity

... that's exactly what doesn't work at the moment.

Thanks,
Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Benjamin Herrenschmidt]

On Sat, 2009-10-31 at 10:31 +0100, Rafael J. Wysocki wrote:

> > To me the proper approach would be to split it so that
> 
> Well, agreed, but ...
> 
> >  - early_resume() restores power & config space etc... so that existing
> > devices can move on (might check for removal). There's no other hotplug
> > activity
> 
> ... that's exactly what doesn't work at the moment.

BTW. This is a PCMCIA problem, a Cardbus or both ? I'll see if I can dig
something on monday ?

From the little email history I caught, it smells like pcmcia old style.

I don't see a problem per-se with the mutex usage with the new interrupt
masking style as Linus says. socket_resume() also looks reasonably sane,
it's the whole handling of removal that should be deferred. Maybe
instead of doing socket_remove_drivers()...send_event() etc.. in there,
we could simply just shut the socket down (PCMCIA drivers should cope
with sockets returning ffff's for a short amount of time), flag it dead
in skt->state and have the "late" resume  actually fire off the driver
removal and sending of the event.

BTW. Have we ever documented whether it's kosher to ->remove() a driver
before ->resume()'ing it ? (In which case obviously we wouldn't resume
it).

Cheers,  
Ben.

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Saturday 31 October 2009, Benjamin Herrenschmidt wrote:
> On Sat, 2009-10-31 at 10:31 +0100, Rafael J. Wysocki wrote:
> 
> > > To me the proper approach would be to split it so that
> > 
> > Well, agreed, but ...
> > 
> > >  - early_resume() restores power & config space etc... so that existing
> > > devices can move on (might check for removal). There's no other hotplug
> > > activity
> > 
> > ... that's exactly what doesn't work at the moment.
> 
> BTW. This is a PCMCIA problem, a Cardbus or both ?

I _think_ it's a CardBus problem.  Evidently, it only happens if the second
CardBus bridge is resumed right after the first one (which resumes entirely
correctly) and only if that happens in the early resume phase (ie. before
resume_device_irqs()).

> I'll see if I can dig something on monday ?

In the meantime I invented a patch that works, ie. apparently fixes the problem
and if there was a card in the socket during the suspend, it's standard config
space is restored correctly.  I tested it on one of my boxes with two different
CardBus adapters and Jose says it fixes the problem for him.

The patch is appended, please have a look.

> >From the little email history I caught, it smells like pcmcia old style.
> 
> I don't see a problem per-se with the mutex usage with the new interrupt
> masking style as Linus says. socket_resume() also looks reasonably sane,
> it's the whole handling of removal that should be deferred.

In the failing case we don't even get there.  There are two CardBus bridges in
the system, but both sockets are empty, so we take the socket_insert() code
path.

> Maybe instead of doing socket_remove_drivers()...send_event() etc.. in there,
> we could simply just shut the socket down (PCMCIA drivers should cope
> with sockets returning ffff's for a short amount of time), flag it dead
> in skt->state and have the "late" resume  actually fire off the driver
> removal and sending of the event.
> 
> BTW. Have we ever documented whether it's kosher to ->remove() a driver
> before ->resume()'ing it ? (In which case obviously we wouldn't resume
> it).

The PM core doesn't have a problem with that.  Whether or not it's sane from
the driver design point of view is a separate question, though.

Thanks,
Rafael

---
From: Rafael J. Wysocki <rjw@sisk.pl>
Subject: PM / yenta: Split resume into early and late parts

Commit 0c570cdeb8fdfcb354a3e9cd81bfc6a09c19de0c
(PM / yenta: Fix cardbus suspend/resume regression) caused resume to
fail on systems with two CardBus bridges.  While the exact nature
of the failure is not known at the moment, it can be worked around by
splitting the yenta resume into an early part, executed during the
early phase of resume, that will only resume the socket and power it
up if there was a card in it during suspend, and a late part,
executed during "regular" resume, that will carry out all of the
remaining yenta resume operations.

Fixes http://bugzilla.kernel.org/show_bug.cgi?id=14334, which is a
listed regression from 2.6.31.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
---
 drivers/pcmcia/cs.c           |   79 +++++++++++++++++++++++++++++-------------
 drivers/pcmcia/yenta_socket.c |   12 +++++-
 include/pcmcia/ss.h           |    2 +
 3 files changed, 68 insertions(+), 25 deletions(-)

Index: linux-2.6/drivers/pcmcia/yenta_socket.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/yenta_socket.c
+++ linux-2.6/drivers/pcmcia/yenta_socket.c
@@ -1275,16 +1275,26 @@ static int yenta_dev_resume_noirq(struct
 	if (socket->type && socket->type->restore_state)
 		socket->type->restore_state(socket);
 
-	return pcmcia_socket_dev_resume(dev);
+	pcmcia_socket_dev_early_resume(dev);
+	return 0;
+}
+
+static int yenta_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_late_resume(dev);
+	return 0;
 }
 
 static struct dev_pm_ops yenta_pm_ops = {
 	.suspend_noirq = yenta_dev_suspend_noirq,
 	.resume_noirq = yenta_dev_resume_noirq,
+	.resume = yenta_dev_resume,
 	.freeze_noirq = yenta_dev_suspend_noirq,
 	.thaw_noirq = yenta_dev_resume_noirq,
+	.thaw = yenta_dev_resume,
 	.poweroff_noirq = yenta_dev_suspend_noirq,
 	.restore_noirq = yenta_dev_resume_noirq,
+	.restore = yenta_dev_resume,
 };
 
 #define YENTA_PM_OPS	(&yenta_pm_ops)
Index: linux-2.6/drivers/pcmcia/cs.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/cs.c
+++ linux-2.6/drivers/pcmcia/cs.c
@@ -98,10 +98,13 @@ EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
  * These functions check for the appropriate struct pcmcia_soket arrays,
  * and pass them to the low-level functions pcmcia_{suspend,resume}_socket
  */
+static int socket_early_resume(struct pcmcia_socket *skt);
+static int socket_late_resume(struct pcmcia_socket *skt);
 static int socket_resume(struct pcmcia_socket *skt);
 static int socket_suspend(struct pcmcia_socket *skt);
 
-int pcmcia_socket_dev_suspend(struct device *dev)
+static void pcmcia_socket_dev_run(struct device *dev,
+				  int (*cb)(struct pcmcia_socket *))
 {
 	struct pcmcia_socket *socket;
 
@@ -110,29 +113,34 @@ int pcmcia_socket_dev_suspend(struct dev
 		if (socket->dev.parent != dev)
 			continue;
 		mutex_lock(&socket->skt_mutex);
-		socket_suspend(socket);
+		cb(socket);
 		mutex_unlock(&socket->skt_mutex);
 	}
 	up_read(&pcmcia_socket_list_rwsem);
+}
 
+int pcmcia_socket_dev_suspend(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_suspend);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_suspend);
 
-int pcmcia_socket_dev_resume(struct device *dev)
+void pcmcia_socket_dev_early_resume(struct device *dev)
 {
-	struct pcmcia_socket *socket;
+	pcmcia_socket_dev_run(dev, socket_early_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_early_resume);
 
-	down_read(&pcmcia_socket_list_rwsem);
-	list_for_each_entry(socket, &pcmcia_socket_list, socket_list) {
-		if (socket->dev.parent != dev)
-			continue;
-		mutex_lock(&socket->skt_mutex);
-		socket_resume(socket);
-		mutex_unlock(&socket->skt_mutex);
-	}
-	up_read(&pcmcia_socket_list_rwsem);
+void pcmcia_socket_dev_late_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_late_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_late_resume);
 
+int pcmcia_socket_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_resume);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_resume);
@@ -546,23 +554,30 @@ static int socket_suspend(struct pcmcia_
 	return 0;
 }
 
-/*
- * Resume a socket.  If a card is present, verify its CIS against
- * our cached copy.  If they are different, the card has been
- * replaced, and we need to tell the drivers.
- */
-static int socket_resume(struct pcmcia_socket *skt)
+static void socket_start_resume(struct pcmcia_socket *skt)
 {
-	int ret;
-
-	if (!(skt->state & SOCKET_SUSPEND))
-		return -EBUSY;
-
 	skt->socket = dead_socket;
 	skt->ops->init(skt);
 	skt->ops->set_socket(skt, &skt->socket);
+}
+
+static int socket_early_resume(struct pcmcia_socket *skt)
+{
+	if ((skt->state & SOCKET_SUSPEND) && (skt->state & SOCKET_PRESENT))
+		socket_start_resume(skt);
+
+	return 0;
+}
+
+static int socket_late_resume(struct pcmcia_socket *skt)
+{
+	int ret;
+
+	if (!(skt->state & SOCKET_SUSPEND))
+		return 0;
 
 	if (!(skt->state & SOCKET_PRESENT)) {
+		socket_start_resume(skt);
 		skt->state &= ~SOCKET_SUSPEND;
 		return socket_insert(skt);
 	}
@@ -596,6 +611,22 @@ static int socket_resume(struct pcmcia_s
 	return 0;
 }
 
+/*
+ * Resume a socket.  If a card is present, verify its CIS against
+ * our cached copy.  If they are different, the card has been
+ * replaced, and we need to tell the drivers.
+ */
+static int socket_resume(struct pcmcia_socket *skt)
+{
+	if (!(skt->state & SOCKET_SUSPEND))
+		return -EBUSY;
+
+	if (skt->state & SOCKET_PRESENT)
+		socket_start_resume(skt);
+
+	return socket_late_resume(skt);
+}
+
 static void socket_remove(struct pcmcia_socket *skt)
 {
 	dev_printk(KERN_NOTICE, &skt->dev,
Index: linux-2.6/include/pcmcia/ss.h
===================================================================
--- linux-2.6.orig/include/pcmcia/ss.h
+++ linux-2.6/include/pcmcia/ss.h
@@ -280,6 +280,8 @@ extern struct pccard_resource_ops pccard
 
 /* socket drivers are expected to use these callbacks in their .drv struct */
 extern int pcmcia_socket_dev_suspend(struct device *dev);
+extern void pcmcia_socket_dev_early_resume(struct device *dev);
+extern void pcmcia_socket_dev_late_resume(struct device *dev);
 extern int pcmcia_socket_dev_resume(struct device *dev);
 
 /* socket drivers use this callback in their IRQ handler */

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Linus Torvalds]

On Sat, 31 Oct 2009, Rafael J. Wysocki wrote:
> 
> The patch is appended, please have a look.

Looks sane to me. It does the actual real socket ops early, and does the 
crazy pcmcia resume late. 

And I like how you abstracted out that dev->socket thing in 
pcmcia_socket_dev_run().

The only thing that looks odd is how you do "socket_start_resume()" in the 
"late_resume" path too - that has already been done by the early_resume, 
and as far as I can see you're now initializing the socket twice.

Is there a reason for that? Or am I misreading the patch (I didn't 
actually apply it, I just read the patch itself).

		Linus

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Saturday 31 October 2009, Linus Torvalds wrote:
> 
> On Sat, 31 Oct 2009, Rafael J. Wysocki wrote:
> > 
> > The patch is appended, please have a look.
> 
> Looks sane to me. It does the actual real socket ops early, and does the 
> crazy pcmcia resume late. 
> 
> And I like how you abstracted out that dev->socket thing in 
> pcmcia_socket_dev_run().
> 
> The only thing that looks odd is how you do "socket_start_resume()" in the 
> "late_resume" path too - that has already been done by the early_resume, 
> and as far as I can see you're now initializing the socket twice.
> 
> Is there a reason for that? Or am I misreading the patch (I didn't 
> actually apply it, I just read the patch itself).

Yes, there is, because socket_early_resume() only does it in
the (skt->state & SOCKET_PRESENT) case.  If that bit is not set, the
initialization is entirely postponed.

Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Linus Torvalds]

On Sat, 31 Oct 2009, Rafael J. Wysocki wrote:
> 
> Yes, there is, because socket_early_resume() only does it in
> the (skt->state & SOCKET_PRESENT) case.  If that bit is not set, the
> initialization is entirely postponed.

Ahh, ok. And what's the reason for that? It seems like the 

	skt->socket = dead_socket;
	skt->ops->init(skt);
	skt->ops->set_socket(skt, &skt->socket);

thing should always be safe, whether there is something present or not.. ?

		Linus

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Saturday 31 October 2009, Linus Torvalds wrote:
> 
> On Sat, 31 Oct 2009, Rafael J. Wysocki wrote:
> > 
> > Yes, there is, because socket_early_resume() only does it in
> > the (skt->state & SOCKET_PRESENT) case.  If that bit is not set, the
> > initialization is entirely postponed.
> 
> Ahh, ok. And what's the reason for that? It seems like the 
> 
> 	skt->socket = dead_socket;
> 	skt->ops->init(skt);
> 	skt->ops->set_socket(skt, &skt->socket);
> 
> thing should always be safe, whether there is something present or not.. ?

It should, but I'm not sure given the reported behavior so far.

I guess I'll prepare another patch that does this unconditionally in the
early phase and let's see how that works.

Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Benjamin Herrenschmidt]

On Sat, 2009-10-31 at 22:27 +0100, Rafael J. Wysocki wrote:

> In the meantime I invented a patch that works, ie. apparently fixes the problem
> and if there was a card in the socket during the suspend, it's standard config
> space is restored correctly.  I tested it on one of my boxes with two different
> CardBus adapters and Jose says it fixes the problem for him.
> 
> The patch is appended, please have a look.

Base idea of the patch sounds good, quick browse through looks good too,

I'll try to band on it with various PCMCIA & CB gear tomorrow.

Cheers,
Ben.

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Saturday 31 October 2009, Benjamin Herrenschmidt wrote:
> On Sat, 2009-10-31 at 22:27 +0100, Rafael J. Wysocki wrote:
> 
> > In the meantime I invented a patch that works, ie. apparently fixes the problem
> > and if there was a card in the socket during the suspend, it's standard config
> > space is restored correctly.  I tested it on one of my boxes with two different
> > CardBus adapters and Jose says it fixes the problem for him.
> > 
> > The patch is appended, please have a look.
> 
> Base idea of the patch sounds good, quick browse through looks good too,
> 
> I'll try to band on it with various PCMCIA & CB gear tomorrow.

Wait, I made a mistake when testing it, didn't notice that my distro ejected
PCMCIA cards automatically before suspend (sigh).

Working on a better patch right now, will hopefully post it in a while.

Thanks,
Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Sunday 01 November 2009, Rafael J. Wysocki wrote:
> On Saturday 31 October 2009, Benjamin Herrenschmidt wrote:
> > On Sat, 2009-10-31 at 22:27 +0100, Rafael J. Wysocki wrote:
> > 
> > > In the meantime I invented a patch that works, ie. apparently fixes the problem
> > > and if there was a card in the socket during the suspend, it's standard config
> > > space is restored correctly.  I tested it on one of my boxes with two different
> > > CardBus adapters and Jose says it fixes the problem for him.
> > > 
> > > The patch is appended, please have a look.
> > 
> > Base idea of the patch sounds good, quick browse through looks good too,
> > 
> > I'll try to band on it with various PCMCIA & CB gear tomorrow.
> 
> Wait, I made a mistake when testing it, didn't notice that my distro ejected
> PCMCIA cards automatically before suspend (sigh).
> 
> Working on a better patch right now, will hopefully post it in a while.

Here you go.

Waiting for feedback from the reporters of BKO #14334, so there's no sign-off
for now.

---
 drivers/pcmcia/cs.c           |   79 ++++++++++++++++++++++++++++--------------
 drivers/pcmcia/yenta_socket.c |   12 +++++-
 include/pcmcia/ss.h           |    4 ++
 3 files changed, 68 insertions(+), 27 deletions(-)

Index: linux-2.6/drivers/pcmcia/yenta_socket.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/yenta_socket.c
+++ linux-2.6/drivers/pcmcia/yenta_socket.c
@@ -1275,16 +1275,26 @@ static int yenta_dev_resume_noirq(struct
 	if (socket->type && socket->type->restore_state)
 		socket->type->restore_state(socket);
 
-	return pcmcia_socket_dev_resume(dev);
+	pcmcia_socket_dev_early_resume(dev);
+	return 0;
+}
+
+static int yenta_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_late_resume(dev);
+	return 0;
 }
 
 static struct dev_pm_ops yenta_pm_ops = {
 	.suspend_noirq = yenta_dev_suspend_noirq,
 	.resume_noirq = yenta_dev_resume_noirq,
+	.resume = yenta_dev_resume,
 	.freeze_noirq = yenta_dev_suspend_noirq,
 	.thaw_noirq = yenta_dev_resume_noirq,
+	.thaw = yenta_dev_resume,
 	.poweroff_noirq = yenta_dev_suspend_noirq,
 	.restore_noirq = yenta_dev_resume_noirq,
+	.restore = yenta_dev_resume,
 };
 
 #define YENTA_PM_OPS	(&yenta_pm_ops)
Index: linux-2.6/drivers/pcmcia/cs.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/cs.c
+++ linux-2.6/drivers/pcmcia/cs.c
@@ -98,10 +98,13 @@ EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
  * These functions check for the appropriate struct pcmcia_soket arrays,
  * and pass them to the low-level functions pcmcia_{suspend,resume}_socket
  */
+static int socket_early_resume(struct pcmcia_socket *skt);
+static int socket_late_resume(struct pcmcia_socket *skt);
 static int socket_resume(struct pcmcia_socket *skt);
 static int socket_suspend(struct pcmcia_socket *skt);
 
-int pcmcia_socket_dev_suspend(struct device *dev)
+static void pcmcia_socket_dev_run(struct device *dev,
+				  int (*cb)(struct pcmcia_socket *))
 {
 	struct pcmcia_socket *socket;
 
@@ -110,29 +113,34 @@ int pcmcia_socket_dev_suspend(struct dev
 		if (socket->dev.parent != dev)
 			continue;
 		mutex_lock(&socket->skt_mutex);
-		socket_suspend(socket);
+		cb(socket);
 		mutex_unlock(&socket->skt_mutex);
 	}
 	up_read(&pcmcia_socket_list_rwsem);
+}
 
+int pcmcia_socket_dev_suspend(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_suspend);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_suspend);
 
-int pcmcia_socket_dev_resume(struct device *dev)
+void pcmcia_socket_dev_early_resume(struct device *dev)
 {
-	struct pcmcia_socket *socket;
+	pcmcia_socket_dev_run(dev, socket_early_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_early_resume);
 
-	down_read(&pcmcia_socket_list_rwsem);
-	list_for_each_entry(socket, &pcmcia_socket_list, socket_list) {
-		if (socket->dev.parent != dev)
-			continue;
-		mutex_lock(&socket->skt_mutex);
-		socket_resume(socket);
-		mutex_unlock(&socket->skt_mutex);
-	}
-	up_read(&pcmcia_socket_list_rwsem);
+void pcmcia_socket_dev_late_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_late_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_late_resume);
 
+int pcmcia_socket_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_resume);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_resume);
@@ -546,29 +554,34 @@ static int socket_suspend(struct pcmcia_
 	return 0;
 }
 
-/*
- * Resume a socket.  If a card is present, verify its CIS against
- * our cached copy.  If they are different, the card has been
- * replaced, and we need to tell the drivers.
- */
-static int socket_resume(struct pcmcia_socket *skt)
+static void socket_start_resume(struct pcmcia_socket *skt)
 {
-	int ret;
-
-	if (!(skt->state & SOCKET_SUSPEND))
-		return -EBUSY;
-
 	skt->socket = dead_socket;
 	skt->ops->init(skt);
 	skt->ops->set_socket(skt, &skt->socket);
+	if (skt->state & SOCKET_PRESENT)
+		skt->resume_status = socket_setup(skt, resume_delay);
+}
+
+static int socket_early_resume(struct pcmcia_socket *skt)
+{
+	if (skt->state & SOCKET_SUSPEND)
+		socket_start_resume(skt);
+
+	return 0;
+}
+
+static int socket_late_resume(struct pcmcia_socket *skt)
+{
+	if (!(skt->state & SOCKET_SUSPEND))
+		return 0;
 
 	if (!(skt->state & SOCKET_PRESENT)) {
 		skt->state &= ~SOCKET_SUSPEND;
 		return socket_insert(skt);
 	}
 
-	ret = socket_setup(skt, resume_delay);
-	if (ret == 0) {
+	if (skt->resume_status == 0) {
 		/*
 		 * FIXME: need a better check here for cardbus cards.
 		 */
@@ -596,6 +609,20 @@ static int socket_resume(struct pcmcia_s
 	return 0;
 }
 
+/*
+ * Resume a socket.  If a card is present, verify its CIS against
+ * our cached copy.  If they are different, the card has been
+ * replaced, and we need to tell the drivers.
+ */
+static int socket_resume(struct pcmcia_socket *skt)
+{
+	if (!(skt->state & SOCKET_SUSPEND))
+		return -EBUSY;
+
+	socket_start_resume(skt);
+	return socket_late_resume(skt);
+}
+
 static void socket_remove(struct pcmcia_socket *skt)
 {
 	dev_printk(KERN_NOTICE, &skt->dev,
Index: linux-2.6/include/pcmcia/ss.h
===================================================================
--- linux-2.6.orig/include/pcmcia/ss.h
+++ linux-2.6/include/pcmcia/ss.h
@@ -262,6 +262,8 @@ struct pcmcia_socket {
 	struct device			dev;
 	/* data internal to the socket driver */
 	void				*driver_data;
+	/* status of the card during resume from a system sleep state */
+	int				resume_status;
 };
 
 
@@ -280,6 +282,8 @@ extern struct pccard_resource_ops pccard
 
 /* socket drivers are expected to use these callbacks in their .drv struct */
 extern int pcmcia_socket_dev_suspend(struct device *dev);
+extern void pcmcia_socket_dev_early_resume(struct device *dev);
+extern void pcmcia_socket_dev_late_resume(struct device *dev);
 extern int pcmcia_socket_dev_resume(struct device *dev);
 
 /* socket drivers use this callback in their IRQ handler */

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Sunday 01 November 2009, Rafael J. Wysocki wrote:
> On Sunday 01 November 2009, Rafael J. Wysocki wrote:
> > On Saturday 31 October 2009, Benjamin Herrenschmidt wrote:
> > > On Sat, 2009-10-31 at 22:27 +0100, Rafael J. Wysocki wrote:
> > > 
> > > > In the meantime I invented a patch that works, ie. apparently fixes the problem
> > > > and if there was a card in the socket during the suspend, it's standard config
> > > > space is restored correctly.  I tested it on one of my boxes with two different
> > > > CardBus adapters and Jose says it fixes the problem for him.
> > > > 
> > > > The patch is appended, please have a look.
> > > 
> > > Base idea of the patch sounds good, quick browse through looks good too,
> > > 
> > > I'll try to band on it with various PCMCIA & CB gear tomorrow.
> > 
> > Wait, I made a mistake when testing it, didn't notice that my distro ejected
> > PCMCIA cards automatically before suspend (sigh).
> > 
> > Working on a better patch right now, will hopefully post it in a while.
> 
> Here you go.
> 
> Waiting for feedback from the reporters of BKO #14334, so there's no sign-off
> for now.

Jose confirms that the patch fixes the problem for him, so here it goes again
with full changelog.

If people don't object, I'll push it through the suspend-2.6 tree along with a
few other bug fixes.

---
From: Rafael J. Wysocki <rjw@sisk.pl>
Subject: PM / yenta: Split resume into early and late parts (rev. 2)

Commit 0c570cdeb8fdfcb354a3e9cd81bfc6a09c19de0c
(PM / yenta: Fix cardbus suspend/resume regression) caused resume to
fail on systems with two CardBus bridges.  While the exact nature
of the failure is not known at the moment, it can be worked around by
splitting the yenta resume into an early part, executed during the
early phase of resume, that will only resume the socket and power it
up if there was a card in it during suspend, and a late part,
executed during "regular" resume, that will carry out all of the
remaining yenta resume operations.

Fixes http://bugzilla.kernel.org/show_bug.cgi?id=14334, which is a
listed regression from 2.6.31.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
---
 drivers/pcmcia/cs.c           |   79 ++++++++++++++++++++++++++++--------------
 drivers/pcmcia/yenta_socket.c |   12 +++++-
 include/pcmcia/ss.h           |    4 ++
 3 files changed, 68 insertions(+), 27 deletions(-)

Index: linux-2.6/drivers/pcmcia/yenta_socket.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/yenta_socket.c
+++ linux-2.6/drivers/pcmcia/yenta_socket.c
@@ -1275,16 +1275,26 @@ static int yenta_dev_resume_noirq(struct
 	if (socket->type && socket->type->restore_state)
 		socket->type->restore_state(socket);
 
-	return pcmcia_socket_dev_resume(dev);
+	pcmcia_socket_dev_early_resume(dev);
+	return 0;
+}
+
+static int yenta_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_late_resume(dev);
+	return 0;
 }
 
 static struct dev_pm_ops yenta_pm_ops = {
 	.suspend_noirq = yenta_dev_suspend_noirq,
 	.resume_noirq = yenta_dev_resume_noirq,
+	.resume = yenta_dev_resume,
 	.freeze_noirq = yenta_dev_suspend_noirq,
 	.thaw_noirq = yenta_dev_resume_noirq,
+	.thaw = yenta_dev_resume,
 	.poweroff_noirq = yenta_dev_suspend_noirq,
 	.restore_noirq = yenta_dev_resume_noirq,
+	.restore = yenta_dev_resume,
 };
 
 #define YENTA_PM_OPS	(&yenta_pm_ops)
Index: linux-2.6/drivers/pcmcia/cs.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/cs.c
+++ linux-2.6/drivers/pcmcia/cs.c
@@ -98,10 +98,13 @@ EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
  * These functions check for the appropriate struct pcmcia_soket arrays,
  * and pass them to the low-level functions pcmcia_{suspend,resume}_socket
  */
+static int socket_early_resume(struct pcmcia_socket *skt);
+static int socket_late_resume(struct pcmcia_socket *skt);
 static int socket_resume(struct pcmcia_socket *skt);
 static int socket_suspend(struct pcmcia_socket *skt);
 
-int pcmcia_socket_dev_suspend(struct device *dev)
+static void pcmcia_socket_dev_run(struct device *dev,
+				  int (*cb)(struct pcmcia_socket *))
 {
 	struct pcmcia_socket *socket;
 
@@ -110,29 +113,34 @@ int pcmcia_socket_dev_suspend(struct dev
 		if (socket->dev.parent != dev)
 			continue;
 		mutex_lock(&socket->skt_mutex);
-		socket_suspend(socket);
+		cb(socket);
 		mutex_unlock(&socket->skt_mutex);
 	}
 	up_read(&pcmcia_socket_list_rwsem);
+}
 
+int pcmcia_socket_dev_suspend(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_suspend);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_suspend);
 
-int pcmcia_socket_dev_resume(struct device *dev)
+void pcmcia_socket_dev_early_resume(struct device *dev)
 {
-	struct pcmcia_socket *socket;
+	pcmcia_socket_dev_run(dev, socket_early_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_early_resume);
 
-	down_read(&pcmcia_socket_list_rwsem);
-	list_for_each_entry(socket, &pcmcia_socket_list, socket_list) {
-		if (socket->dev.parent != dev)
-			continue;
-		mutex_lock(&socket->skt_mutex);
-		socket_resume(socket);
-		mutex_unlock(&socket->skt_mutex);
-	}
-	up_read(&pcmcia_socket_list_rwsem);
+void pcmcia_socket_dev_late_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_late_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_late_resume);
 
+int pcmcia_socket_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_resume);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_resume);
@@ -546,29 +554,34 @@ static int socket_suspend(struct pcmcia_
 	return 0;
 }
 
-/*
- * Resume a socket.  If a card is present, verify its CIS against
- * our cached copy.  If they are different, the card has been
- * replaced, and we need to tell the drivers.
- */
-static int socket_resume(struct pcmcia_socket *skt)
+static void socket_start_resume(struct pcmcia_socket *skt)
 {
-	int ret;
-
-	if (!(skt->state & SOCKET_SUSPEND))
-		return -EBUSY;
-
 	skt->socket = dead_socket;
 	skt->ops->init(skt);
 	skt->ops->set_socket(skt, &skt->socket);
+	if (skt->state & SOCKET_PRESENT)
+		skt->resume_status = socket_setup(skt, resume_delay);
+}
+
+static int socket_early_resume(struct pcmcia_socket *skt)
+{
+	if (skt->state & SOCKET_SUSPEND)
+		socket_start_resume(skt);
+
+	return 0;
+}
+
+static int socket_late_resume(struct pcmcia_socket *skt)
+{
+	if (!(skt->state & SOCKET_SUSPEND))
+		return 0;
 
 	if (!(skt->state & SOCKET_PRESENT)) {
 		skt->state &= ~SOCKET_SUSPEND;
 		return socket_insert(skt);
 	}
 
-	ret = socket_setup(skt, resume_delay);
-	if (ret == 0) {
+	if (skt->resume_status == 0) {
 		/*
 		 * FIXME: need a better check here for cardbus cards.
 		 */
@@ -596,6 +609,20 @@ static int socket_resume(struct pcmcia_s
 	return 0;
 }
 
+/*
+ * Resume a socket.  If a card is present, verify its CIS against
+ * our cached copy.  If they are different, the card has been
+ * replaced, and we need to tell the drivers.
+ */
+static int socket_resume(struct pcmcia_socket *skt)
+{
+	if (!(skt->state & SOCKET_SUSPEND))
+		return -EBUSY;
+
+	socket_start_resume(skt);
+	return socket_late_resume(skt);
+}
+
 static void socket_remove(struct pcmcia_socket *skt)
 {
 	dev_printk(KERN_NOTICE, &skt->dev,
Index: linux-2.6/include/pcmcia/ss.h
===================================================================
--- linux-2.6.orig/include/pcmcia/ss.h
+++ linux-2.6/include/pcmcia/ss.h
@@ -262,6 +262,8 @@ struct pcmcia_socket {
 	struct device			dev;
 	/* data internal to the socket driver */
 	void				*driver_data;
+	/* status of the card during resume from a system sleep state */
+	int				resume_status;
 };
 
 
@@ -280,6 +282,8 @@ extern struct pccard_resource_ops pccard
 
 /* socket drivers are expected to use these callbacks in their .drv struct */
 extern int pcmcia_socket_dev_suspend(struct device *dev);
+extern void pcmcia_socket_dev_early_resume(struct device *dev);
+extern void pcmcia_socket_dev_late_resume(struct device *dev);
 extern int pcmcia_socket_dev_resume(struct device *dev);
 
 /* socket drivers use this callback in their IRQ handler */

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Dominik Brodowski]

Hey,

On Sun, Nov 01, 2009 at 09:36:10AM +0100, Rafael J. Wysocki wrote:
> Commit 0c570cdeb8fdfcb354a3e9cd81bfc6a09c19de0c
> (PM / yenta: Fix cardbus suspend/resume regression) caused resume to
> fail on systems with two CardBus bridges.  While the exact nature
> of the failure is not known at the moment, it can be worked around by
> splitting the yenta resume into an early part, executed during the
> early phase of resume, that will only resume the socket and power it
> up if there was a card in it during suspend, and a late part,
> executed during "regular" resume, that will carry out all of the
> remaining yenta resume operations.
> 
> Fixes http://bugzilla.kernel.org/show_bug.cgi?id=14334, which is a
> listed regression from 2.6.31.

The only issue I see is that we now return 0 unconditionally on the resume
callbacks. Otherwise, it's

Acked-by: Dominik Brodowski <linux@dominikbrodowski.net>

> Index: linux-2.6/drivers/pcmcia/yenta_socket.c
> ===================================================================
> --- linux-2.6.orig/drivers/pcmcia/yenta_socket.c
> +++ linux-2.6/drivers/pcmcia/yenta_socket.c
> @@ -1275,16 +1275,26 @@ static int yenta_dev_resume_noirq(struct
>  	if (socket->type && socket->type->restore_state)
>  		socket->type->restore_state(socket);
>  
> -	return pcmcia_socket_dev_resume(dev);
> +	pcmcia_socket_dev_early_resume(dev);
> +	return 0;
> +}
> +
> +static int yenta_dev_resume(struct device *dev)
> +{
> +	pcmcia_socket_dev_late_resume(dev);
> +	return 0;
>  }
>  
>  static struct dev_pm_ops yenta_pm_ops = {
>  	.suspend_noirq = yenta_dev_suspend_noirq,
>  	.resume_noirq = yenta_dev_resume_noirq,
> +	.resume = yenta_dev_resume,
>  	.freeze_noirq = yenta_dev_suspend_noirq,
>  	.thaw_noirq = yenta_dev_resume_noirq,
> +	.thaw = yenta_dev_resume,
>  	.poweroff_noirq = yenta_dev_suspend_noirq,
>  	.restore_noirq = yenta_dev_resume_noirq,
> +	.restore = yenta_dev_resume,
>  };
>  
>  #define YENTA_PM_OPS	(&yenta_pm_ops)
> Index: linux-2.6/drivers/pcmcia/cs.c
> ===================================================================
> --- linux-2.6.orig/drivers/pcmcia/cs.c
> +++ linux-2.6/drivers/pcmcia/cs.c
> @@ -98,10 +98,13 @@ EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
>   * These functions check for the appropriate struct pcmcia_soket arrays,
>   * and pass them to the low-level functions pcmcia_{suspend,resume}_socket
>   */
> +static int socket_early_resume(struct pcmcia_socket *skt);
> +static int socket_late_resume(struct pcmcia_socket *skt);
>  static int socket_resume(struct pcmcia_socket *skt);
>  static int socket_suspend(struct pcmcia_socket *skt);
>  
> -int pcmcia_socket_dev_suspend(struct device *dev)
> +static void pcmcia_socket_dev_run(struct device *dev,
> +				  int (*cb)(struct pcmcia_socket *))
>  {
>  	struct pcmcia_socket *socket;
>  
> @@ -110,29 +113,34 @@ int pcmcia_socket_dev_suspend(struct dev
>  		if (socket->dev.parent != dev)
>  			continue;
>  		mutex_lock(&socket->skt_mutex);
> -		socket_suspend(socket);
> +		cb(socket);
>  		mutex_unlock(&socket->skt_mutex);
>  	}
>  	up_read(&pcmcia_socket_list_rwsem);
> +}
>  
> +int pcmcia_socket_dev_suspend(struct device *dev)
> +{
> +	pcmcia_socket_dev_run(dev, socket_suspend);
>  	return 0;
>  }
>  EXPORT_SYMBOL(pcmcia_socket_dev_suspend);
>  
> -int pcmcia_socket_dev_resume(struct device *dev)
> +void pcmcia_socket_dev_early_resume(struct device *dev)
>  {
> -	struct pcmcia_socket *socket;
> +	pcmcia_socket_dev_run(dev, socket_early_resume);
> +}
> +EXPORT_SYMBOL(pcmcia_socket_dev_early_resume);
>  
> -	down_read(&pcmcia_socket_list_rwsem);
> -	list_for_each_entry(socket, &pcmcia_socket_list, socket_list) {
> -		if (socket->dev.parent != dev)
> -			continue;
> -		mutex_lock(&socket->skt_mutex);
> -		socket_resume(socket);
> -		mutex_unlock(&socket->skt_mutex);
> -	}
> -	up_read(&pcmcia_socket_list_rwsem);
> +void pcmcia_socket_dev_late_resume(struct device *dev)
> +{
> +	pcmcia_socket_dev_run(dev, socket_late_resume);
> +}
> +EXPORT_SYMBOL(pcmcia_socket_dev_late_resume);
>  
> +int pcmcia_socket_dev_resume(struct device *dev)
> +{
> +	pcmcia_socket_dev_run(dev, socket_resume);
>  	return 0;
>  }
>  EXPORT_SYMBOL(pcmcia_socket_dev_resume);
> @@ -546,29 +554,34 @@ static int socket_suspend(struct pcmcia_
>  	return 0;
>  }
>  
> -/*
> - * Resume a socket.  If a card is present, verify its CIS against
> - * our cached copy.  If they are different, the card has been
> - * replaced, and we need to tell the drivers.
> - */
> -static int socket_resume(struct pcmcia_socket *skt)
> +static void socket_start_resume(struct pcmcia_socket *skt)
>  {
> -	int ret;
> -
> -	if (!(skt->state & SOCKET_SUSPEND))
> -		return -EBUSY;
> -
>  	skt->socket = dead_socket;
>  	skt->ops->init(skt);
>  	skt->ops->set_socket(skt, &skt->socket);
> +	if (skt->state & SOCKET_PRESENT)
> +		skt->resume_status = socket_setup(skt, resume_delay);
> +}
> +
> +static int socket_early_resume(struct pcmcia_socket *skt)
> +{
> +	if (skt->state & SOCKET_SUSPEND)
> +		socket_start_resume(skt);
> +
> +	return 0;
> +}
> +
> +static int socket_late_resume(struct pcmcia_socket *skt)
> +{
> +	if (!(skt->state & SOCKET_SUSPEND))
> +		return 0;
>  
>  	if (!(skt->state & SOCKET_PRESENT)) {
>  		skt->state &= ~SOCKET_SUSPEND;
>  		return socket_insert(skt);
>  	}
>  
> -	ret = socket_setup(skt, resume_delay);
> -	if (ret == 0) {
> +	if (skt->resume_status == 0) {
>  		/*
>  		 * FIXME: need a better check here for cardbus cards.
>  		 */
> @@ -596,6 +609,20 @@ static int socket_resume(struct pcmcia_s
>  	return 0;
>  }
>  
> +/*
> + * Resume a socket.  If a card is present, verify its CIS against
> + * our cached copy.  If they are different, the card has been
> + * replaced, and we need to tell the drivers.
> + */
> +static int socket_resume(struct pcmcia_socket *skt)
> +{
> +	if (!(skt->state & SOCKET_SUSPEND))
> +		return -EBUSY;
> +
> +	socket_start_resume(skt);
> +	return socket_late_resume(skt);
> +}
> +
>  static void socket_remove(struct pcmcia_socket *skt)
>  {
>  	dev_printk(KERN_NOTICE, &skt->dev,
> Index: linux-2.6/include/pcmcia/ss.h
> ===================================================================
> --- linux-2.6.orig/include/pcmcia/ss.h
> +++ linux-2.6/include/pcmcia/ss.h
> @@ -262,6 +262,8 @@ struct pcmcia_socket {
>  	struct device			dev;
>  	/* data internal to the socket driver */
>  	void				*driver_data;
> +	/* status of the card during resume from a system sleep state */
> +	int				resume_status;
>  };
>  
>  
> @@ -280,6 +282,8 @@ extern struct pccard_resource_ops pccard
>  
>  /* socket drivers are expected to use these callbacks in their .drv struct */
>  extern int pcmcia_socket_dev_suspend(struct device *dev);
> +extern void pcmcia_socket_dev_early_resume(struct device *dev);
> +extern void pcmcia_socket_dev_late_resume(struct device *dev);
>  extern int pcmcia_socket_dev_resume(struct device *dev);
>  
>  /* socket drivers use this callback in their IRQ handler */

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Sunday 01 November 2009, Dominik Brodowski wrote:
> Hey,
> 
> On Sun, Nov 01, 2009 at 09:36:10AM +0100, Rafael J. Wysocki wrote:
> > Commit 0c570cdeb8fdfcb354a3e9cd81bfc6a09c19de0c
> > (PM / yenta: Fix cardbus suspend/resume regression) caused resume to
> > fail on systems with two CardBus bridges.  While the exact nature
> > of the failure is not known at the moment, it can be worked around by
> > splitting the yenta resume into an early part, executed during the
> > early phase of resume, that will only resume the socket and power it
> > up if there was a card in it during suspend, and a late part,
> > executed during "regular" resume, that will carry out all of the
> > remaining yenta resume operations.
> > 
> > Fixes http://bugzilla.kernel.org/show_bug.cgi?id=14334, which is a
> > listed regression from 2.6.31.
> 
> The only issue I see is that we now return 0 unconditionally on the resume
> callbacks.

Hmm. pcmcia_socket_dev_suspend() and pcmcia_socket_dev_resume() return 0
unconditionally even without the patch, so it doesn't change that.

> Otherwise, it's
> 
> Acked-by: Dominik Brodowski <linux-X3ehHDuj6sIIGcDfoQAp7OTW4wlIGRCZ@public.gmane.org>

Thanks!

Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Linus Torvalds]

On Sun, 1 Nov 2009, Rafael J. Wysocki wrote:
> 
> If people don't object, I'll push it through the suspend-2.6 tree along 
> with a few other bug fixes.

No objections, but a cleanup request:

> +static int socket_early_resume(struct pcmcia_socket *skt)
> +{
> +	if (skt->state & SOCKET_SUSPEND)
> +		socket_start_resume(skt);
> +
> +	return 0;
> +}
> +
> +static int socket_late_resume(struct pcmcia_socket *skt)
> +{
> +	if (!(skt->state & SOCKET_SUSPEND))
> +		return 0;

As far as I can tell, that "SOCKET_SUSPEND" test is totally pointless. 
That socket _is_ going to be suspended, and testing for it here just seems 
to confuse things. 

So I'd remove it from both early_resume and late_resume, and only keep it 
in the case of the legacy user-requested suspend/resume (do we even do 
that any more?).

The SOCKET_SUSPEND flag itself is still relevant, of course, since the 
state change handling will test it (in order to avoid insert/remove 
handlign while we have the suspend flag set). It's just that the suspend 
code shouldn't _test_ it, since the suspend code is what sets it in the 
first place.

		Linus

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Sunday 01 November 2009, Linus Torvalds wrote:
> 
> On Sun, 1 Nov 2009, Rafael J. Wysocki wrote:
> > 
> > If people don't object, I'll push it through the suspend-2.6 tree along 
> > with a few other bug fixes.
> 
> No objections, but a cleanup request:
> 
> > +static int socket_early_resume(struct pcmcia_socket *skt)
> > +{
> > +	if (skt->state & SOCKET_SUSPEND)
> > +		socket_start_resume(skt);
> > +
> > +	return 0;
> > +}
> > +
> > +static int socket_late_resume(struct pcmcia_socket *skt)
> > +{
> > +	if (!(skt->state & SOCKET_SUSPEND))
> > +		return 0;
> 
> As far as I can tell, that "SOCKET_SUSPEND" test is totally pointless. 

Right.

> That socket _is_ going to be suspended, and testing for it here just seems 
> to confuse things. 
> 
> So I'd remove it from both early_resume and late_resume, and only keep it 
> in the case of the legacy user-requested suspend/resume (do we even do 
> that any more?).

OK, updated patch is appended.

Thanks,
Rafael

---
From: Rafael J. Wysocki <rjw@sisk.pl>
Subject: PM / yenta: Split resume into early and late parts (rev. 3)

Commit 0c570cdeb8fdfcb354a3e9cd81bfc6a09c19de0c
(PM / yenta: Fix cardbus suspend/resume regression) caused resume to
fail on systems with two CardBus bridges.  While the exact nature
of the failure is not known at the moment, it can be worked around by
splitting the yenta resume into an early part, executed during the
early phase of resume, that will only resume the socket and power it
up if there was a card in it during suspend, and a late part,
executed during "regular" resume, that will carry out all of the
remaining yenta resume operations.

Fixes http://bugzilla.kernel.org/show_bug.cgi?id=14334, which is a
listed regression from 2.6.31.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
---
 drivers/pcmcia/cs.c           |   74 +++++++++++++++++++++++++++---------------
 drivers/pcmcia/yenta_socket.c |   12 ++++++
 include/pcmcia/ss.h           |    4 ++
 3 files changed, 63 insertions(+), 27 deletions(-)

Index: linux-2.6/drivers/pcmcia/yenta_socket.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/yenta_socket.c
+++ linux-2.6/drivers/pcmcia/yenta_socket.c
@@ -1275,16 +1275,26 @@ static int yenta_dev_resume_noirq(struct
 	if (socket->type && socket->type->restore_state)
 		socket->type->restore_state(socket);
 
-	return pcmcia_socket_dev_resume(dev);
+	pcmcia_socket_dev_early_resume(dev);
+	return 0;
+}
+
+static int yenta_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_late_resume(dev);
+	return 0;
 }
 
 static struct dev_pm_ops yenta_pm_ops = {
 	.suspend_noirq = yenta_dev_suspend_noirq,
 	.resume_noirq = yenta_dev_resume_noirq,
+	.resume = yenta_dev_resume,
 	.freeze_noirq = yenta_dev_suspend_noirq,
 	.thaw_noirq = yenta_dev_resume_noirq,
+	.thaw = yenta_dev_resume,
 	.poweroff_noirq = yenta_dev_suspend_noirq,
 	.restore_noirq = yenta_dev_resume_noirq,
+	.restore = yenta_dev_resume,
 };
 
 #define YENTA_PM_OPS	(&yenta_pm_ops)
Index: linux-2.6/drivers/pcmcia/cs.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/cs.c
+++ linux-2.6/drivers/pcmcia/cs.c
@@ -98,10 +98,13 @@ EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
  * These functions check for the appropriate struct pcmcia_soket arrays,
  * and pass them to the low-level functions pcmcia_{suspend,resume}_socket
  */
+static int socket_early_resume(struct pcmcia_socket *skt);
+static int socket_late_resume(struct pcmcia_socket *skt);
 static int socket_resume(struct pcmcia_socket *skt);
 static int socket_suspend(struct pcmcia_socket *skt);
 
-int pcmcia_socket_dev_suspend(struct device *dev)
+static void pcmcia_socket_dev_run(struct device *dev,
+				  int (*cb)(struct pcmcia_socket *))
 {
 	struct pcmcia_socket *socket;
 
@@ -110,29 +113,34 @@ int pcmcia_socket_dev_suspend(struct dev
 		if (socket->dev.parent != dev)
 			continue;
 		mutex_lock(&socket->skt_mutex);
-		socket_suspend(socket);
+		cb(socket);
 		mutex_unlock(&socket->skt_mutex);
 	}
 	up_read(&pcmcia_socket_list_rwsem);
+}
 
+int pcmcia_socket_dev_suspend(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_suspend);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_suspend);
 
-int pcmcia_socket_dev_resume(struct device *dev)
+void pcmcia_socket_dev_early_resume(struct device *dev)
 {
-	struct pcmcia_socket *socket;
+	pcmcia_socket_dev_run(dev, socket_early_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_early_resume);
 
-	down_read(&pcmcia_socket_list_rwsem);
-	list_for_each_entry(socket, &pcmcia_socket_list, socket_list) {
-		if (socket->dev.parent != dev)
-			continue;
-		mutex_lock(&socket->skt_mutex);
-		socket_resume(socket);
-		mutex_unlock(&socket->skt_mutex);
-	}
-	up_read(&pcmcia_socket_list_rwsem);
+void pcmcia_socket_dev_late_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_late_resume);
+}
+EXPORT_SYMBOL(pcmcia_socket_dev_late_resume);
 
+int pcmcia_socket_dev_resume(struct device *dev)
+{
+	pcmcia_socket_dev_run(dev, socket_resume);
 	return 0;
 }
 EXPORT_SYMBOL(pcmcia_socket_dev_resume);
@@ -546,29 +554,29 @@ static int socket_suspend(struct pcmcia_
 	return 0;
 }
 
-/*
- * Resume a socket.  If a card is present, verify its CIS against
- * our cached copy.  If they are different, the card has been
- * replaced, and we need to tell the drivers.
- */
-static int socket_resume(struct pcmcia_socket *skt)
+static void socket_start_resume(struct pcmcia_socket *skt)
 {
-	int ret;
-
-	if (!(skt->state & SOCKET_SUSPEND))
-		return -EBUSY;
-
 	skt->socket = dead_socket;
 	skt->ops->init(skt);
 	skt->ops->set_socket(skt, &skt->socket);
+	if (skt->state & SOCKET_PRESENT)
+		skt->resume_status = socket_setup(skt, resume_delay);
+}
 
+static int socket_early_resume(struct pcmcia_socket *skt)
+{
+	socket_start_resume(skt);
+	return 0;
+}
+
+static int socket_late_resume(struct pcmcia_socket *skt)
+{
 	if (!(skt->state & SOCKET_PRESENT)) {
 		skt->state &= ~SOCKET_SUSPEND;
 		return socket_insert(skt);
 	}
 
-	ret = socket_setup(skt, resume_delay);
-	if (ret == 0) {
+	if (skt->resume_status == 0) {
 		/*
 		 * FIXME: need a better check here for cardbus cards.
 		 */
@@ -596,6 +604,20 @@ static int socket_resume(struct pcmcia_s
 	return 0;
 }
 
+/*
+ * Resume a socket.  If a card is present, verify its CIS against
+ * our cached copy.  If they are different, the card has been
+ * replaced, and we need to tell the drivers.
+ */
+static int socket_resume(struct pcmcia_socket *skt)
+{
+	if (!(skt->state & SOCKET_SUSPEND))
+		return -EBUSY;
+
+	socket_start_resume(skt);
+	return socket_late_resume(skt);
+}
+
 static void socket_remove(struct pcmcia_socket *skt)
 {
 	dev_printk(KERN_NOTICE, &skt->dev,
Index: linux-2.6/include/pcmcia/ss.h
===================================================================
--- linux-2.6.orig/include/pcmcia/ss.h
+++ linux-2.6/include/pcmcia/ss.h
@@ -262,6 +262,8 @@ struct pcmcia_socket {
 	struct device			dev;
 	/* data internal to the socket driver */
 	void				*driver_data;
+	/* status of the card during resume from a system sleep state */
+	int				resume_status;
 };
 
 
@@ -280,6 +282,8 @@ extern struct pccard_resource_ops pccard
 
 /* socket drivers are expected to use these callbacks in their .drv struct */
 extern int pcmcia_socket_dev_suspend(struct device *dev);
+extern void pcmcia_socket_dev_early_resume(struct device *dev);
+extern void pcmcia_socket_dev_late_resume(struct device *dev);
 extern int pcmcia_socket_dev_resume(struct device *dev);
 
 /* socket drivers use this callback in their IRQ handler */

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Dominik Brodowski]

Hey,

just two minor nit-pick which we could handle post-2.6.32:

> +++ linux-2.6/drivers/pcmcia/cs.c
> @@ -98,10 +98,13 @@ EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
>   * These functions check for the appropriate struct pcmcia_soket arrays,
>   * and pass them to the low-level functions pcmcia_{suspend,resume}_socket

... some documentation of the new functions, especially whether other socket
drivers should be updated?

> -static int socket_resume(struct pcmcia_socket *skt)
> +static void socket_start_resume(struct pcmcia_socket *skt)
>  {
> -	int ret;
> -
> -	if (!(skt->state & SOCKET_SUSPEND))
> -		return -EBUSY;
> -
>  	skt->socket = dead_socket;
>  	skt->ops->init(skt);
>  	skt->ops->set_socket(skt, &skt->socket);
> +	if (skt->state & SOCKET_PRESENT)
> +		skt->resume_status = socket_setup(skt, resume_delay);
> +}
>  
> +static int socket_early_resume(struct pcmcia_socket *skt)
> +{
> +	socket_start_resume(skt);
> +	return 0;
> +}

Why do we need to have two functions doing the same? Wouldn't

static int socket_early_resume(...)

suffice, with the only call to socket_start_resume() being replaced with
socket_early_resume()?

Best,
	Dominik

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Rafael J. Wysocki]

On Monday 02 November 2009, Dominik Brodowski wrote:
> Hey,
> 
> just two minor nit-pick which we could handle post-2.6.32:
> 
> > +++ linux-2.6/drivers/pcmcia/cs.c
> > @@ -98,10 +98,13 @@ EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
> >   * These functions check for the appropriate struct pcmcia_soket arrays,
> >   * and pass them to the low-level functions pcmcia_{suspend,resume}_socket
> 
> ... some documentation of the new functions, especially whether other socket
> drivers should be updated?

OK, I'll post a separate patch for that for .33.

> > -static int socket_resume(struct pcmcia_socket *skt)
> > +static void socket_start_resume(struct pcmcia_socket *skt)
> >  {
> > -	int ret;
> > -
> > -	if (!(skt->state & SOCKET_SUSPEND))
> > -		return -EBUSY;
> > -
> >  	skt->socket = dead_socket;
> >  	skt->ops->init(skt);
> >  	skt->ops->set_socket(skt, &skt->socket);
> > +	if (skt->state & SOCKET_PRESENT)
> > +		skt->resume_status = socket_setup(skt, resume_delay);
> > +}
> >  
> > +static int socket_early_resume(struct pcmcia_socket *skt)
> > +{
> > +	socket_start_resume(skt);
> > +	return 0;
> > +}
> 
> Why do we need to have two functions doing the same? Wouldn't
> 
> static int socket_early_resume(...)
> 
> suffice, with the only call to socket_start_resume() being replaced with
> socket_early_resume()?

Yes, it would.  I'll do that in the final version of the patch.

Thanks,
Rafael

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Linus Torvalds]

On Mon, 2 Nov 2009, Rafael J. Wysocki wrote:
> 
> OK, updated patch is appended.

Looks good to me, feel free to push any time (assuming you've gotten 
testing confirmation from the people who reported it). Thanks,

		Linus

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Benjamin Herrenschmidt]

On Mon, 2009-11-02 at 14:39 +0100, Rafael J. Wysocki wrote:
> 
> > That socket _is_ going to be suspended, and testing for it here just seems 
> > to confuse things. 
> > 
> > So I'd remove it from both early_resume and late_resume, and only keep it 
> > in the case of the legacy user-requested suspend/resume (do we even do 
> > that any more?).
> 
> OK, updated patch is appended.

Test by me delayed to tomorrow ... hit another suspend/resume bug on
that laptop which took away the time I had to do that test yesterday and
today I'm off :-)

(Bug was simple but took a while to track down: machine was left in
storage for a while, battery ran out, RTC went back to Jan 1, 1904,
which means a negative xtime, and the new timekeeping code will do
horrible things including hanging at resume when that happens. Fix is to
make powerpc read_persistent_clock() to ignore the RTC when it contains
a date older than epoch).

Cheers,
Ben.

Re: Help needed, Re: [Bug #14334] pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m

[Pavel Machek]

Hi!

> (Bug was simple but took a while to track down: machine was left in
> storage for a while, battery ran out, RTC went back to Jan 1, 1904,
> which means a negative xtime, and the new timekeeping code will do
> horrible things including hanging at resume when that happens. Fix is to
> make powerpc read_persistent_clock() to ignore the RTC when it contains
> a date older than epoch).

Additionaly, it would be nice if the machine would not hang, no matter
what RTC says...
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[Bug #14353] BUG: sleeping function called from invalid context at kernel/mutex.c:280

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-05 3:39 (22 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4

Re: [Bug #14353] BUG: sleeping function called from invalid context at kernel/mutex.c:280

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 757 bytes --]

On Mon, 2009-10-26 at 19:55 +0100, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
> Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
> Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-10-05 3:39 (22 days old)
> References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4

This should be fixed by a160ee69c6a4622ed30c377a978554015e9931cb.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Bug #14353] BUG: sleeping function called from invalid context at kernel/mutex.c:280

[Rafael J. Wysocki]

On Monday 26 October 2009, Johannes Berg wrote:
> On Mon, 2009-10-26 at 19:55 +0100, Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
> > Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
> > Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date		: 2009-10-05 3:39 (22 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4
> 
> This should be fixed by a160ee69c6a4622ed30c377a978554015e9931cb.

OK, closing.

Thanks,
Rafael

[Bug #14354] Bad corruption with 2.6.32-rc1 and upwards

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
Subject		: Bad corruption with 2.6.32-rc1 and upwards
Submitter	: Holger Freyther <zecke-MQnelBtSfJRAfugRpC6u6w@public.gmane.org>
Date		: 2009-10-09 15:42 (18 days old)

[Bug #14375] Intel(R) I/OAT DMA Engine init failed

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov@gmail.com>
Date		: 2009-10-02 9:46 (25 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams@intel.com>
Patch		: http://patchwork.kernel.org/patch/51808/

Re: [Bug #14375] Intel(R) I/OAT DMA Engine init failed

[Alexander Beregalov]

2009/10/26 Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14375
> Subject         : Intel(R) I/OAT DMA Engine init failed
> Submitter       : Alexander Beregalov <a.beregalov@gmail.com>
> Date            : 2009-10-02 9:46 (25 days old)
> References      : http://marc.info/?l=linux-kernel&m=125447680016160&w=4
> Handled-By      : Dan Williams <dan.j.williams@intel.com>
> Patch           : http://patchwork.kernel.org/patch/51808/

When I patch reiserfs the bug goes to libata, it means ioatdma works
and libata does not. It looks like inaccurate work with memory
somewhere, but I do not know how to find it.
I cannot reproduce it since 2.6.32-rc3 and I do not see any bugs since then.

Re: [Bug #14375] Intel(R) I/OAT DMA Engine init failed

[Rafael J. Wysocki]

On Monday 26 October 2009, Alexander Beregalov wrote:
> 2009/10/26 Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org>:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14375
> > Subject         : Intel(R) I/OAT DMA Engine init failed
> > Submitter       : Alexander Beregalov <a.beregalov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date            : 2009-10-02 9:46 (25 days old)
> > References      : http://marc.info/?l=linux-kernel&m=125447680016160&w=4
> > Handled-By      : Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> > Patch           : http://patchwork.kernel.org/patch/51808/
> 
> When I patch reiserfs the bug goes to libata, it means ioatdma works
> and libata does not. It looks like inaccurate work with memory
> somewhere, but I do not know how to find it.
> I cannot reproduce it since 2.6.32-rc3 and I do not see any bugs since then.

Since you can't reproduce it with -rc3 and later, I'm going to close it now.
Please reopen if the problem reappears.

Best,
Rafael

Re: [Bug #14375] Intel(R) I/OAT DMA Engine init failed

[Dan Williams]

On Mon, 2009-10-26 at 14:57 -0700, Alexander Beregalov wrote:
> 2009/10/26 Rafael J. Wysocki <rjw@sisk.pl>:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14375
> > Subject         : Intel(R) I/OAT DMA Engine init failed
> > Submitter       : Alexander Beregalov <a.beregalov@gmail.com>
> > Date            : 2009-10-02 9:46 (25 days old)
> > References      : http://marc.info/?l=linux-kernel&m=125447680016160&w=4
> > Handled-By      : Dan Williams <dan.j.williams@intel.com>
> > Patch           : http://patchwork.kernel.org/patch/51808/
> 
> When I patch reiserfs the bug goes to libata, it means ioatdma works
> and libata does not. It looks like inaccurate work with memory
> somewhere, but I do not know how to find it.
> I cannot reproduce it since 2.6.32-rc3 and I do not see any bugs since then.

If it helps the debug, the symptom seems to be that all interrupts get
turned off.  Neither the normal device completion interrupt nor the
completion watchdog timer fire before the self test timeout.  It seems
something else re-enables interrupts eventually, but we know they were
at least disabled for 3 seconds.  An unmatched spin_unlock_irq in an
async initialization path perhaps?

--
Dan

[Bug #14355] USB serial regression after 2.6.31.1 with Huawei E169 GSM modem

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
Submitter	: Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
Date		: 2009-10-10 03:07 (17 days old)
References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4

Re: [Bug #14355] USB serial regression after 2.6.31.1 with Huawei E169 GSM modem

[Benjamin Herrenschmidt]

On Mon, 2009-10-26 at 19:55 +0100, Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).

Well, the E169 works, though there seem to be a few issues remaining:

 - There are reports that the E220 is still broken (by the same
regression, just that the patch that fixes the E169 doesn't fix the
E220, but since I don't have one of these, I'm waiting for users to send
dumps instead of just complaining :-)

 - There's a weird glitch where the E169 tend to only work after being
plugged the -second- time. The first time, the storage device generally
shows up but not the ttyS's devices for some strange reason. Not sure
what's up yet. Yanking it and plugging it back works. Could be timing
related due to the need to load the module.

None of that happens with 2.6.31.1

Cheers,
Ben.


> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
> Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
> Submitter	: Benjamin Herrenschmidt <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>
> Date		: 2009-10-10 03:07 (17 days old)
> References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [Bug #14355] USB serial regression after 2.6.31.1 with Huawei E169 GSM modem

[Rafael J. Wysocki]

On Tuesday 27 October 2009, Benjamin Herrenschmidt wrote:
> On Mon, 2009-10-26 at 19:55 +0100, Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> 
> Well, the E169 works, though there seem to be a few issues remaining:
> 
>  - There are reports that the E220 is still broken (by the same
> regression, just that the patch that fixes the E169 doesn't fix the
> E220, but since I don't have one of these, I'm waiting for users to send
> dumps instead of just complaining :-)
> 
>  - There's a weird glitch where the E169 tend to only work after being
> plugged the -second- time. The first time, the storage device generally
> shows up but not the ttyS's devices for some strange reason. Not sure
> what's up yet. Yanking it and plugging it back works. Could be timing
> related due to the need to load the module.
> 
> None of that happens with 2.6.31.1

Well, I guess it's better to leave the bug open in that case.

Thanks,
Rafael

[Bug #14373] Task blocked for more than 120 seconds

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-02 10:16 (25 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4

[Bug #14372] ath5k wireless not working after suspend-resume - eeepc

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: ath5k wireless not working after suspend-resume - eeepc
Submitter	: Fabio Comolli <fabio.comolli-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-03 15:36 (24 days old)
References	: http://lkml.org/lkml/2009/10/3/91

Re: [Bug #14372] ath5k wireless not working after suspend-resume - eeepc

[Fabio Comolli]

Still present in -rc5

On Mon, Oct 26, 2009 at 7:55 PM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14372
> Subject         : ath5k wireless not working after suspend-resume - eeepc
> Submitter       : Fabio Comolli <fabio.comolli@gmail.com>
> Date            : 2009-10-03 15:36 (24 days old)
> References      : http://lkml.org/lkml/2009/10/3/91
>
>
>

Re: [Bug #14372] ath5k wireless not working after suspend-resume - eeepc

[Rafael J. Wysocki]

On Monday 26 October 2009, Fabio Comolli wrote:
> Still present in -rc5

Thanks for the update.

> On Mon, Oct 26, 2009 at 7:55 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14372
> > Subject         : ath5k wireless not working after suspend-resume - eeepc
> > Submitter       : Fabio Comolli <fabio.comolli@gmail.com>
> > Date            : 2009-10-03 15:36 (24 days old)
> > References      : http://lkml.org/lkml/2009/10/3/91

Re: [Bug #14372] ath5k wireless not working after suspend-resume - eeepc

[Harald Arnesen]

Fabio Comolli <fabio.comolli-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> Still present in -rc5

I can confirm that.
-- 
Hilsen Harald.

Re: [Bug #14372] ath5k wireless not working after rfkill [BISECTED]

[Sitsofe Wheeler]

On Tue, Oct 27, 2009 at 10:42:46AM +0100, Harald Arnesen wrote:
> Fabio Comolli <fabio.comolli-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
> 
> > Still present in -rc5

Help. I've just bisected this down to the same commit
b56ab33d68638e6aafdbfc694025e8354a628f49 (that will teach me for not
reading the whole thread) as I have the same problem on my EeePC 900.

Still present in -rc6 and the latest git.

-- 
Sitsofe | http://sucs.org/~sits/

Re: [Bug #14372] ath5k wireless not working after rfkill [BISECTED]

[Fabio Comolli]

Thanks for reporting. I guess I'll wait for -rc7 then.

On Wed, Nov 4, 2009 at 9:07 PM, Sitsofe Wheeler <sitsofe@yahoo.com> wrote:
> On Tue, Oct 27, 2009 at 10:42:46AM +0100, Harald Arnesen wrote:
>> Fabio Comolli <fabio.comolli@gmail.com> writes:
>>
>> > Still present in -rc5
>
> Help. I've just bisected this down to the same commit
> b56ab33d68638e6aafdbfc694025e8354a628f49 (that will teach me for not
> reading the whole thread) as I have the same problem on my EeePC 900.
>
> Still present in -rc6 and the latest git.
>
> --
> Sitsofe | http://sucs.org/~sits/
>

Re: [Bug #14372] ath5k wireless not working after rfkill [BISECTED]

[John W. Linville]

On Wed, Nov 04, 2009 at 08:07:14PM +0000, Sitsofe Wheeler wrote:
> On Tue, Oct 27, 2009 at 10:42:46AM +0100, Harald Arnesen wrote:
> > Fabio Comolli <fabio.comolli-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
> > 
> > > Still present in -rc5
> 
> Help. I've just bisected this down to the same commit
> b56ab33d68638e6aafdbfc694025e8354a628f49 (that will teach me for not
> reading the whole thread) as I have the same problem on my EeePC 900.
> 
> Still present in -rc6 and the latest git.

Thank you for taking the time to bisect this!

I move that we revert it.  It appears to be a work-around for a staging
driver that causes problems for a non-staging one...

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org			might be all we have.  Be ready.

Re: [Bug #14372] ath5k wireless not working after rfkill [BISECTED]

[Darren Salt]

I demand that John W. Linville may or may not have written...

> On Wed, Nov 04, 2009 at 08:07:14PM +0000, Sitsofe Wheeler wrote:
>> On Tue, Oct 27, 2009 at 10:42:46AM +0100, Harald Arnesen wrote:
>>> Fabio Comolli <fabio.comolli@gmail.com> writes:
>>>> Still present in -rc5
>> Help. I've just bisected this down to the same commit
>> b56ab33d68638e6aafdbfc694025e8354a628f49 (that will teach me for not
>> reading the whole thread) as I have the same problem on my EeePC 900.
>> Still present in -rc6 and the latest git.

> Thank you for taking the time to bisect this!

> I move that we revert it.  It appears to be a work-around for a staging
> driver that causes problems for a non-staging one...

Already in hand: a reversion patch was sent by Corentin Chary, and I have
mail from Len Brown saying that it has been applied. (The correct fix is
present in -rc5 and -rc6, so once this reversion is out of the way, I see no
reason not to close bug 13390.)

-- 
| Darren Salt            | linux at youmustbejoking | nr. Ashington, | Doon
| using Debian GNU/Linux | or ds    ,demon,co,uk    | Northumberland | Army
|   <URL:http://www.youmustbejoking.demon.co.uk/> (PGP 2.6, GPG keys)

About all some men accomplish in life is to send a son to Oxbridge.

[Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-08 21:46 (19 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
Handled-By	: Alexey Starikovskiy <astarikovskiy-l3A5Bk7waGM@public.gmane.org>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> Submitter	: Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-10-08 21:46 (19 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> Handled-By	: Alexey Starikovskiy<astarikovskiy-l3A5Bk7waGM@public.gmane.org>
> Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
>
>
>
>    
The patch submitted gets rid of the warning
for people crying wolf!! but probably would not close
the bugreport until the patch makes it into the main
kernel.

Justin P. Mattock

Re: [Bug #14379] ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String

[Rafael J. Wysocki]

On Monday 26 October 2009, Justin P. Mattock wrote:
> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
> > Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
> > Submitter	: Justin Mattock<justinmattock@gmail.com>
> > Date		: 2009-10-08 21:46 (19 days old)
> > First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=d9adc2e031bd22d5d9607a53a8d3b30e0b675f39
> > References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4
> > Handled-By	: Alexey Starikovskiy<astarikovskiy@suse.de>
> > Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23347
> >
> >
> >
> >    
> The patch submitted gets rid of the warning
> for people crying wolf!! but probably would not close
> the bugreport until the patch makes it into the main
> kernel.

Yup.

Thanks,
Rafael

[Bug #14378] Problems with net/core/skbuff.c

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra@navynet.it>
Date		: 2009-10-08 14:51 (19 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4

Re: [Bug #14378] Problems with net/core/skbuff.c

[Eric Dumazet]

Rafael J. Wysocki a écrit :
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
> Subject		: Problems with net/core/skbuff.c
> Submitter	: Massimo Cetra <mcetra-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
> Date		: 2009-10-08 14:51 (19 days old)
> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4
> 
> 

This was corrected by commit ed79bab847d8e5a2986d8ff43c49c6fb8ee3265f

virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

Because netpoll can call netdevice start_xmit() method with
irqs disabled, drivers should not call kfree_skb() from
their start_xmit(), but use dev_kfree_skb_any() instead.

Oct  8 11:16:52 172.30.1.31 [113074.791813] ------------[ cut here ]------------
Oct  8 11:16:52 172.30.1.31 [113074.791813] WARNING: at net/core/skbuff.c:398 \
                skb_release_head_state+0x64/0xc8()
Oct  8 11:16:52 172.30.1.31 [113074.791813] Hardware name:
Oct  8 11:16:52 172.30.1.31 [113074.791813] Modules linked in: netconsole ocfs2 jbd2 quota_tree \
ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs crc32c drbd cn loop \
serio_raw psmouse snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net pcspkr parport_pc parport \
i2c_piix4 i2c_core button processor evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot \
dm_mod ide_cd_mod cdrom ata_generic ata_piix virtio_blk libata scsi_mod piix ide_pci_generic ide_core \
                virtio_pci virtio_ring virtio floppy thermal fan thermal_sys [last unloaded: netconsole]
Oct  8 11:16:52 172.30.1.31 [113074.791813] Pid: 11132, comm: php5-cgi Tainted: G        W  \
                2.6.31.2-vserver #1
Oct  8 11:16:52 172.30.1.31 [113074.791813] Call Trace:
Oct  8 11:16:52 172.30.1.31 [113074.791813] <IRQ>  [<ffffffff81253cd5>] ? \
                skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa01cb139>] ? free_old_xmit_skbs+0x51/0x6e \
                [virtio_net]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa01cbc85>] ? start_xmit+0x26/0xf2 [virtio_net]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffffa0429216>] ? write_msg+0x90/0xeb [netconsole]
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8106b142>] ? vx_update_load+0x18/0x13e
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81308309>] ? printk+0x4e/0x5d
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff810626b7>] ? ktime_get+0xc/0x41
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
Oct  8 11:16:52 172.30.1.31 [113074.791813] [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
Oct  8 11:16:52 172.30.1.31 [113074.791813] <EOI>  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31

Reported-and-tested-by: Massimo Cetra <mcetra-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
Signed-off-by: Eric Dumazet <eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Bug-Entry: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Signed-off-by: David S. Miller <davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>

Re: [Bug #14378] Problems with net/core/skbuff.c

[Rafael J. Wysocki]

On Monday 26 October 2009, Eric Dumazet wrote:
> Rafael J. Wysocki a écrit :
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
> > Subject		: Problems with net/core/skbuff.c
> > Submitter	: Massimo Cetra <mcetra-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
> > Date		: 2009-10-08 14:51 (19 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4
> > 
> > 
> 
> This was corrected by commit ed79bab847d8e5a2986d8ff43c49c6fb8ee3265f
> 
> virtio_net: use dev_kfree_skb_any() in free_old_xmit_skbs()

Thanks, closing.

Rafael

[Bug #14376] Kernel NULL pointer dereference/ kvm subsystem

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-06 14:38 (21 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4

[Bug #14380] Video tearing/glitching with T400 laptops

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso-3s7WtUTddSA@public.gmane.org>
Date		: 2009-10-02 22:40 (25 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes-Y1mF5jBUw70BENJcbMCuUQ@public.gmane.org>
Patch		: http://marc.info/?l=linux-kernel&m=125591495325000&w=4

[Bug #14381] iwlagn lost connection after s2ram (with warnings)

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
Subject		: iwlagn lost connection after s2ram (with warnings)
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-07 14:20 (20 days old)
References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4

Re: [Bug #14381] iwlagn lost connection after s2ram (with warnings)

[Carlos R. Mafra]

On Mon 26.Oct'09 at 19:55:54 +0100, Rafael J. Wysocki wrote:
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
> Subject		: iwlagn lost connection after s2ram (with warnings)
> Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
> Date		: 2009-10-07 14:20 (20 days old)
> References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4

Somehow this problem did not happen again for 2 weeks or so (ie since
a few days after I reported it) and I suspend to RAM around ~4 times
a day. 

The problem was not deterministic, so I am not sure if we should close it now.
But in case you close it, I can report again if it happens in the future.

[Bug #14383] hackbench regression with kernel 2.6.32-rc1

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:19 (18 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>

[Bug #14387] deadlock with fallocate

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann@users.sourceforge.net>
Date		: 2009-10-07 3:00 (20 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch@lst.de>

Re: [Bug #14387] deadlock with fallocate

[Christoph Hellwig]

I don't think this actually is a regression.

Re: [Bug #14387] deadlock with fallocate

[Rafael J. Wysocki]

On Monday 26 October 2009, Christoph Hellwig wrote:
> I don't think this actually is a regression.

OK, dropping from the list.

[Bug #14389] Build system issue

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz@infradead.org>
Date		: 2009-10-09 8:58 (18 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4

Re: [Bug #14389] Build system issue

[Ingo Molnar]

* Rafael J. Wysocki <rjw@sisk.pl> wrote:

> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
> Subject		: Build system issue
> Submitter	: Peter Zijlstra <peterz@infradead.org>
> Date		: 2009-10-09 8:58 (18 days old)
> First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
> References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4
> 

This should be fixed upstream by:

 2331d1a: kbuild: revert "save ARCH & CROSS_COMPILE ..."

Thanks,

	Ingo

Re: [Bug #14389] Build system issue

[Rafael J. Wysocki]

On Monday 26 October 2009, Ingo Molnar wrote:
> 
> * Rafael J. Wysocki <rjw@sisk.pl> wrote:
> 
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
> > Subject		: Build system issue
> > Submitter	: Peter Zijlstra <peterz@infradead.org>
> > Date		: 2009-10-09 8:58 (18 days old)
> > First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
> > References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4
> > 
> 
> This should be fixed upstream by:
> 
>  2331d1a: kbuild: revert "save ARCH & CROSS_COMPILE ..."

Thanks, closing.

Rafael

[Bug #14384] tbench regression with 2.6.32-rc1

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14384
Subject		: tbench regression with 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:51 (18 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59abf02644c45f1591e1374ee7bb45dc757fcb88
References	: http://marc.info/?l=linux-kernel&m=125508216713138&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>

[Bug #14430] sync() hangs in bdi_sched_wait

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14430
Subject		: sync() hangs in bdi_sched_wait
Submitter	: Petr Vandrovec <petr-vPk2MGR0e28uaRcfnNAh7A@public.gmane.org>
Date		: 2009-10-17 19:14 (10 days old)

[Bug #14390] "bind" a device to a driver doesn't not work anymore

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel@tremplin-utc.net>
Date		: 2009-10-11 0:04 (16 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4

[Bug #14406] uvcvideo stopped work on Toshiba

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14406
Subject		: uvcvideo stopped work on Toshiba
Submitter	: okias <d.okias-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-14 19:08 (13 days old)

[Bug #14408] sysctl check failed

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14408
Subject		: sysctl check failed
Submitter	: Peter Teoh <htmldeveloper@gmail.com>
Date		: 2009-10-14 22:59 (13 days old)

Re: [Bug #14408] sysctl check failed

[Eric W. Biederman]

"Rafael J. Wysocki" <rjw@sisk.pl> writes:

> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).

I believe Andrew has the fix in the -mm tree.

This is a bug but not a regression as the sysctl_check code had this bug from
day one.

Eric

Re: [Bug #14408] sysctl check failed

[Rafael J. Wysocki]

On Wednesday 28 October 2009, Eric W. Biederman wrote:
> "Rafael J. Wysocki" <rjw-KKrjLPT3xs0@public.gmane.org> writes:
> 
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> 
> I believe Andrew has the fix in the -mm tree.
> 
> This is a bug but not a regression as the sysctl_check code had this bug from
> day one.

OK, so I'm dropping this one from the list of recent regressions.

Thanks,
Rafael

[Bug #14415] Reboot on kernel load

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14415
Subject		: Reboot on kernel load
Submitter	: Brian Beardall <brian-sVkzCUl/XCrR7s880joybQ@public.gmane.org>
Date		: 2009-10-15 23:57 (12 days old)

[Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14467
Subject		: Linker errors on ia64 with NR_CPUS=4096
Submitter	: Jeff Mahoney <jeffm-IBi9RG/b67k@public.gmane.org>
Date		: 2009-10-18 22:28 (9 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=34d76c41554a05425613d16efebb3069c4c545f0
References	: http://marc.info/?l=linux-kernel&m=125590493116720&w=4

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Christoph Lameter]

The NR_CPUS array in the percpu section needs to be removed.

A simple fix would be to allocate the rq_weight per cpu array
dynamically.

The large per cpu array was introduced as a fix to a race condition. Maybe
there is another way of dealing with this issue that does not require
large per cpu arrays?

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Jiri Kosina]

On Tue, 27 Oct 2009, Christoph Lameter wrote:

> The NR_CPUS array in the percpu section needs to be removed.
> 
> A simple fix would be to allocate the rq_weight per cpu array
> dynamically.

There already are two patches for this acked by Ingo/Tejun, which Tejun is 
going to take through his tree tomorrow.

	http://lkml.org/lkml/2009/10/27/132
	http://lkml.org/lkml/2009/10/27/215

-- 
Jiri Kosina
SUSE Labs, Novell Inc.

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Christoph Lameter]

On Tue, 27 Oct 2009, Jiri Kosina wrote:

> On Tue, 27 Oct 2009, Christoph Lameter wrote:
>
> > The NR_CPUS array in the percpu section needs to be removed.
> >
> > A simple fix would be to allocate the rq_weight per cpu array
> > dynamically.
>
> There already are two patches for this acked by Ingo/Tejun, which Tejun is
> going to take through his tree tomorrow.
>
> 	http://lkml.org/lkml/2009/10/27/132

per cpu alloc from an atomic context without passing gfp flags through to
the page allocator? That does not look right. Sure wish that the percpu
allocator would be working from atomic contexts for other cases.

> 	http://lkml.org/lkml/2009/10/27/215

There is still heavy percpu use because of the patch for machines with
large numbers of processors. Another solution would be better that does
not require a N^2 sized array.

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Tejun Heo]

Christoph Lameter wrote:
>> There already are two patches for this acked by Ingo/Tejun, which Tejun is
>> going to take through his tree tomorrow.
>>
>> 	http://lkml.org/lkml/2009/10/27/132
> 
> per cpu alloc from an atomic context without passing gfp flags through to
> the page allocator? That does not look right. Sure wish that the percpu
> allocator would be working from atomic contexts for other cases.

It's just for sched_init() which has irq off but is not really in
atomic context and does GFP_KERNEL allocations.  The following comment
has been added to the first patch to explain it.

+ * allocations are done using GFP_KERNEL with pcpu_lock released.  In
+ * general, percpu memory can't be allocated with irq off but
+ * irqsave/restore are still used in alloc path so that it can be used
+ * from early init path - sched_init() specifically.

Thanks.

-- 
tejun

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Christoph Lameter]

On Thu, 29 Oct 2009, Tejun Heo wrote:

> It's just for sched_init() which has irq off but is not really in
> atomic context and does GFP_KERNEL allocations.  The following comment
> has been added to the first patch to explain it.

Uhmm.. Is the page allocator available at that point? If you are
constricted to the reserved per cpu area then IA64 can still run out of
space if its booted with 4096 actual cpus.

> + * allocations are done using GFP_KERNEL with pcpu_lock released.  In
> + * general, percpu memory can't be allocated with irq off but
> + * irqsave/restore are still used in alloc path so that it can be used
> + * from early init path - sched_init() specifically.
>
> Thanks.

Maybe make the patch a bit more general so that it can operate in an
atomic context and handles gfp flags nicely?

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Tejun Heo]

Christoph Lameter wrote:
> On Thu, 29 Oct 2009, Tejun Heo wrote:
> 
>> It's just for sched_init() which has irq off but is not really in
>> atomic context and does GFP_KERNEL allocations.  The following comment
>> has been added to the first patch to explain it.
> 
> Uhmm.. Is the page allocator available at that point? If you are
> constricted to the reserved per cpu area then IA64 can still run out of
> space if its booted with 4096 actual cpus.

sched_init() is after mm_init() so it should work.

>> + * allocations are done using GFP_KERNEL with pcpu_lock released.  In
>> + * general, percpu memory can't be allocated with irq off but
>> + * irqsave/restore are still used in alloc path so that it can be used
>> + * from early init path - sched_init() specifically.
> 
> Maybe make the patch a bit more general so that it can operate in an
> atomic context and handles gfp flags nicely?

That would be nice but currently, we have no user which needs anything
other than GFP_KERNEL although lack of users could have been caused by
the lack of the functionality and the non-atomic irq disabled state is
an exception case which is handled differently by might_sleep() too.
So, I'm not sure whether adding GFP_* parameter would worth the
effort.  Also, adding that is a bit too late for 2.6.32 at this point.

Thanks.

-- 
tejun

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Christoph Lameter]

On Thu, 29 Oct 2009, Tejun Heo wrote:

> That would be nice but currently, we have no user which needs anything
> other than GFP_KERNEL although lack of users could have been caused by
> the lack of the functionality and the non-atomic irq disabled state is
> an exception case which is handled differently by might_sleep() too.
> So, I'm not sure whether adding GFP_* parameter would worth the
> effort.  Also, adding that is a bit too late for 2.6.32 at this point.

As I have said repeately before: We need this for dynamic DMA slab
creation in SLUB. Since you got it already halfway in: Complete it and I
can get rid of the static percpu definitions that I had to add to be able
to use the new percpu allocator.

Re: [Bug #14467] Linker errors on ia64 with NR_CPUS=4096

[Tejun Heo]

Hello,

Christoph Lameter wrote:
> As I have said repeately before: We need this for dynamic DMA slab
> creation in SLUB. Since you got it already halfway in: Complete it and I
> can get rid of the static percpu definitions that I had to add to be able
> to use the new percpu allocator.

I completely forgot about that.  The biggest cause of my reluctance is
that it's gonna make locking more complex as I really like the current
dumb locking in alloc path.  Oh well, I'm putting it on my todo list.

Thanks.

-- 
tejun

[Bug #14466] EFI boot on x86 fails in .32

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14466
Subject		: EFI boot on x86 fails in .32
Submitter	: Matthew Garrett <mjg59-1xO5oi07KQx4cg9Nei1l7Q@public.gmane.org>
Date		: 2009-10-20 0:34 (7 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=7bd867dfb4e0357e06a3211ab2bd0e714110def3
References	: http://marc.info/?l=linux-kernel&m=125599887314290&w=4
Handled-By	: Feng Tang <feng.tang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

[Bug #14442] resume after hibernate: /dev/sdb drops and returns as /dev/sde

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14442
Subject		: resume after hibernate: /dev/sdb drops and returns as /dev/sde
Submitter	: Duncan <1i5t5.duncan-j9pdmedNgrk@public.gmane.org>
Date		: 2009-10-20 01:52 (7 days old)

[Bug #14472] EXT4 corruption

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14472
Subject		: EXT4 corruption
Submitter	: Shawn Starr <shawn.starr-bJEeYj9oJeDQT0dZR+AlfA@public.gmane.org>
Date		: 2009-10-13 2:07 (14 days old)
References	: http://marc.info/?l=linux-kernel&m=125539997508256&w=4
Handled-By	: Theodore Tso <tytso-3s7WtUTddSA@public.gmane.org>

Re: [Bug #14472] EXT4 corruption

[Andrew Lutomirski]

On Mon, Oct 26, 2009 at 2:55 PM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14472
> Subject         : EXT4 corruption
> Submitter       : Shawn Starr <shawn.starr-bJEeYj9oJeDQT0dZR+AlfA@public.gmane.org>
> Date            : 2009-10-13 2:07 (14 days old)
> References      : http://marc.info/?l=linux-kernel&m=125539997508256&w=4
> Handled-By      : Theodore Tso <tytso-3s7WtUTddSA@public.gmane.org>
>


This but is *not* fixed.  I just triggered it a few minutes ago by
abusing i915 and drm, which caused a panic.  This is slightly newer
than 2.6.32-rc5, with a couple of i915 bugfixes thrown in.

Photos are here:
http://web.mit.edu/luto/www/ext4_crashphotos/

This is a very nasty regression, for obvious reasons.

--Andy

Re: [Bug #14472] EXT4 corruption

[Rafael J. Wysocki]

On Thursday 29 October 2009, Andrew Lutomirski wrote:
> On Mon, Oct 26, 2009 at 2:55 PM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14472
> > Subject         : EXT4 corruption
> > Submitter       : Shawn Starr <shawn.starr-bJEeYj9oJeDQT0dZR+AlfA@public.gmane.org>
> > Date            : 2009-10-13 2:07 (14 days old)
> > References      : http://marc.info/?l=linux-kernel&m=125539997508256&w=4
> > Handled-By      : Theodore Tso <tytso-3s7WtUTddSA@public.gmane.org>
> >
> 
> 
> This but is *not* fixed.  I just triggered it a few minutes ago by
> abusing i915 and drm, which caused a panic.  This is slightly newer
> than 2.6.32-rc5, with a couple of i915 bugfixes thrown in.
> 
> Photos are here:
> http://web.mit.edu/luto/www/ext4_crashphotos/
> 
> This is a very nasty regression, for obvious reasons.

Thanks for the update.

Rafael

Re: [Bug #14472] EXT4 corruption

[Theodore Tso]

On Thu, Oct 29, 2009 at 03:57:32PM -0400, Andrew Lutomirski wrote:
> 
> This but is *not* fixed.  I just triggered it a few minutes ago by
> abusing i915 and drm, which caused a panic.  This is slightly newer
> than 2.6.32-rc5, with a couple of i915 bugfixes thrown in.

Andrew, can you test to see if this patch helps?

Thanks,

						- Ted

commit a8836b1d6f92273e001012c7705ae8f4c3d5fb65
Author: Aneesh Kumar K.V <aneesh.kumar-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date:   Tue Oct 27 15:36:38 2009 +0530

    ext4: discard preallocation during truncate
    
    We need to make sure when we drop and reacquire the inode's
    i_data_sem we discard the inode preallocation. Otherwise we
    could have blocks marked as free in bitmap but still belonging
    to prealloc space.
    
    Signed-off-by: Aneesh Kumar K.V <aneesh.kumar-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 5c5bc5d..a1ef1c3 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -209,6 +209,12 @@ static int try_to_extend_transaction(handle_t *handle, struct inode *inode)
 	up_write(&EXT4_I(inode)->i_data_sem);
 	ret = ext4_journal_restart(handle, blocks_for_truncate(inode));
 	down_write(&EXT4_I(inode)->i_data_sem);
+	/*
+	 * We have dropped i_data_sem. So somebody else could have done
+	 * block allocation. So discard the prealloc space created as a
+	 * part of block allocation
+	 */
+	ext4_discard_preallocations(inode);
 
 	return ret;
 }

Re: [Bug #14472] EXT4 corruption

[Andrew Lutomirski]

On Thu, Oct 29, 2009 at 6:23 PM, Theodore Tso <tytso-3s7WtUTddSA@public.gmane.org> wrote:
> On Thu, Oct 29, 2009 at 03:57:32PM -0400, Andrew Lutomirski wrote:
>>
>> This but is *not* fixed.  I just triggered it a few minutes ago by
>> abusing i915 and drm, which caused a panic.  This is slightly newer
>> than 2.6.32-rc5, with a couple of i915 bugfixes thrown in.
>
> Andrew, can you test to see if this patch helps?

I'm building a kernel with that patch now, and I'll keep running it
for awhile.  I only seem to trigger this bug once a month or so, so
I'll let you know if I see any more corruption.

Thanks,
Andy

Re: [Bug #14472] EXT4 corruption

[Andrew Lutomirski]

On Thu, Oct 29, 2009 at 5:23 PM, Theodore Tso <tytso-3s7WtUTddSA@public.gmane.org> wrote:
> On Thu, Oct 29, 2009 at 03:57:32PM -0400, Andrew Lutomirski wrote:
>>
>> This but is *not* fixed.  I just triggered it a few minutes ago by
>> abusing i915 and drm, which caused a panic.  This is slightly newer
>> than 2.6.32-rc5, with a couple of i915 bugfixes thrown in.
>
> Andrew, can you test to see if this patch helps?
>
> Thanks,
>
>                                                - Ted
>
> commit a8836b1d6f92273e001012c7705ae8f4c3d5fb65
> Author: Aneesh Kumar K.V <aneesh.kumar-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
> Date:   Tue Oct 27 15:36:38 2009 +0530
>
>    ext4: discard preallocation during truncate
>
>    We need to make sure when we drop and reacquire the inode's
>    i_data_sem we discard the inode preallocation. Otherwise we
>    could have blocks marked as free in bitmap but still belonging
>    to prealloc space.
>
>    Signed-off-by: Aneesh Kumar K.V <aneesh.kumar-23VcF4HTsmIX0ybBhKVfKQ@public.gmane.orgom>
>
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index 5c5bc5d..a1ef1c3 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -209,6 +209,12 @@ static int try_to_extend_transaction(handle_t *handle, struct inode *inode)
>        up_write(&EXT4_I(inode)->i_data_sem);
>        ret = ext4_journal_restart(handle, blocks_for_truncate(inode));
>        down_write(&EXT4_I(inode)->i_data_sem);
> +       /*
> +        * We have dropped i_data_sem. So somebody else could have done
> +        * block allocation. So discard the prealloc space created as a
> +        * part of block allocation
> +        */
> +       ext4_discard_preallocations(inode);
>
>        return ret;
>  }
>


It looks like 2.6.32-rc6 is supposed to fix this bug, but it also
looks like this patch didn't make it in.  Should I still be using this
patch?

Thanks,
Andy

Re: [Bug #14472] EXT4 corruption

[Theodore Tso]

On Tue, Nov 03, 2009 at 06:43:11PM -0500, Andrew Lutomirski wrote:
> It looks like 2.6.32-rc6 is supposed to fix this bug, but it also
> looks like this patch didn't make it in.  Should I still be using this
> patch?

This patch does fix a potential problem and I am planning on pushing
it to Linus; the chances of hitting the race is quite low, though; the
revert which Eric identified is probably what was affecting most of
the people who were seeing problems with ext4 in 2.6.32-rcX.

    	       	    	   	    	      	 - Ted

Re: [Bug #14472] EXT4 corruption

[Shawn Starr]

On October 29, 2009 06:34:45 pm Andrew Lutomirski wrote:
> On Thu, Oct 29, 2009 at 6:23 PM, Theodore Tso <tytso@mit.edu> wrote:
> > On Thu, Oct 29, 2009 at 03:57:32PM -0400, Andrew Lutomirski wrote:
> >> This but is *not* fixed.  I just triggered it a few minutes ago by
> >> abusing i915 and drm, which caused a panic.  This is slightly newer
> >> than 2.6.32-rc5, with a couple of i915 bugfixes thrown in.
> >
> > Andrew, can you test to see if this patch helps?
> 
> I'm building a kernel with that patch now, and I'll keep running it
> for awhile.  I only seem to trigger this bug once a month or so, so
> I'll let you know if I see any more corruption.
> 
> Thanks,
> Andy
> 

You should be able to trigger this using the same method I did. To mistakenly 
cause modprobe to spawn unlimited processes ending up in a swap storm.

Thanks, 
Shawn.

[Bug #14477] possible circular locking dependency in ISDN PPP

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14477
Subject		: possible circular locking dependency in ISDN PPP
Submitter	: Tilman Schmidt <tilman@imap.cc>
Date		: 2009-10-18 22:16 (9 days old)
References	: http://marc.info/?l=linux-kernel&m=125590423416087&w=4

Re: [Bug #14477] possible circular locking dependency in ISDN PPP

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

Rafael J. Wysocki schrieb:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14477
> Subject		: possible circular locking dependency in ISDN PPP
> Submitter	: Tilman Schmidt <tilman@imap.cc>
> Date		: 2009-10-18 22:16 (9 days old)
> References	: http://marc.info/?l=linux-kernel&m=125590423416087&w=4

Fixed by

http://git.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=2bd9af046fdc10703b266b0f3b25423f0b7d703e

Thanks,
Tilman

-- 
Tilman Schmidt                    E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

Re: [Bug #14477] possible circular locking dependency in ISDN PPP

[Rafael J. Wysocki]

On Monday 26 October 2009, Tilman Schmidt wrote:
> Rafael J. Wysocki schrieb:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14477
> > Subject		: possible circular locking dependency in ISDN PPP
> > Submitter	: Tilman Schmidt <tilman-ZTO5kqT2PaM@public.gmane.org>
> > Date		: 2009-10-18 22:16 (9 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125590423416087&w=4
> 
> Fixed by
> 
> http://git.kernel.org/?p=linux/kernel/git/davem/net-2.6.git;a=commit;h=2bd9af046fdc10703b266b0f3b25423f0b7d703e

Thanks, closing.

Rafael

[Bug #14473] ATA related kernel warning after resume

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14473
Subject		: ATA related kernel warning after resume
Submitter	: Tino Keitel <tino.keitel@tikei.de>
Date		: 2009-10-14 6:55 (13 days old)
References	: http://marc.info/?l=linux-kernel&m=125550466624678&w=4

[Bug #14479] nfs oops

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14479
Subject		: nfs oops
Submitter	: Egon Alter <egon.alter-hi6Y0CQ0nG0@public.gmane.org>
Date		: 2009-10-19 16:03 (8 days old)
References	: http://marc.info/?l=linux-kernel&m=125596822630410&w=4

Re: [Bug #14479] nfs oops

[Egon Alter]

Am Montag, 26. Oktober 2009 19:55:59 schrieb Rafael J. Wysocki:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14479
> Subject		: nfs oops
> Submitter	: Egon Alter <egon.alter-hi6Y0CQ0nG0@public.gmane.org>
> Date		: 2009-10-19 16:03 (8 days old)
> References	: http://marc.info/?l=linux-kernel&m=125596822630410&w=4 

I cannot reproduce this anymore with lastest git (with or without the patch 
from Trond). So it is in a closeable state. Will reopen when it reappears.

Thanks

Egon

Re: [Bug #14479] nfs oops

[Rafael J. Wysocki]

On Monday 26 October 2009, Egon Alter wrote:
> Am Montag, 26. Oktober 2009 19:55:59 schrieb Rafael J. Wysocki:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14479
> > Subject		: nfs oops
> > Submitter	: Egon Alter <egon.alter-hi6Y0CQ0nG0@public.gmane.org>
> > Date		: 2009-10-19 16:03 (8 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125596822630410&w=4 
> 
> I cannot reproduce this anymore with lastest git (with or without the patch 
> from Trond). So it is in a closeable state. Will reopen when it reappears.

Thanks, closing.

Rafael

[Bug #14480] 2 locks held by cat -- running "find /sys | head -c 4" --> system hang

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14480
Subject		: 2 locks held by cat -- running "find /sys | head -c 4" --> system hang
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-20 16:11 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125605511728088&w=4
Handled-By	: Chris Wilson <chris-Y6uKTt2uX1cEflXRtASbqLVCufUGDwFn@public.gmane.org>
Patch		: http://patchwork.kernel.org/patch/54974/

[Bug #14484] no video output after suspend

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14484
Subject		: no video output after suspend
Submitter	: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>
Date		: 2009-10-25 20:57 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125650430123713&w=4
Handled-By	: Jesse Barnes <jbarnes@virtuousgeek.org>

Re: [Bug #14484] no video output after suspend

[Riccardo Magliocchetti]

Hello Rafael,

Rafael J. Wysocki ha scritto:
> This message has been generated automatically as a part of a report
> of recent regressions.
> 
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
> 
> 
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14484
> Subject		: no video output after suspend
> Submitter	: Riccardo Magliocchetti <riccardo.magliocchetti-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-10-25 20:57 (2 days old)
> References	: http://marc.info/?l=linux-kernel&m=125650430123713&w=4
> Handled-By	: Jesse Barnes <jbarnes-Y1mF5jBUw70BENJcbMCuUQ@public.gmane.org>

Since the bad commit is around 2.6.31-rc9 i think this is a regression 
from 2.6.30 and not 2.6.31, it's me that have reported the issue so 
late. Then the patch that you added on the first comment is for another 
issue and not this one.

thanks,
riccardo

Re: [Bug #14484] no video output after suspend

[Rafael J. Wysocki]

On Tuesday 27 October 2009, Riccardo Magliocchetti wrote:
> Hello Rafael,
> 
> Rafael J. Wysocki ha scritto:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> > 
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> > 
> > 
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14484
> > Subject		: no video output after suspend
> > Submitter	: Riccardo Magliocchetti <riccardo.magliocchetti-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date		: 2009-10-25 20:57 (2 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125650430123713&w=4
> > Handled-By	: Jesse Barnes <jbarnes-Y1mF5jBUw70BENJcbMCuUQ@public.gmane.org>
> 
> Since the bad commit is around 2.6.31-rc9 i think this is a regression 
> from 2.6.30 and not 2.6.31, it's me that have reported the issue so 
> late. Then the patch that you added on the first comment is for another 
> issue and not this one.

Thanks, updated.

Rafael

[Bug #14483] WARNING: at drivers/base/sys.c:353 __sysdev_resume+0x54/0xca()

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14483
Subject		: WARNING: at drivers/base/sys.c:353 __sysdev_resume+0x54/0xca()
Submitter	: Justin Mattock <justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-25 19:58 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125650070420168&w=4

Re: [Bug #14483] WARNING: at drivers/base/sys.c:353 __sysdev_resume+0x54/0xca()

[Justin P. Mattock]

Rafael J. Wysocki wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14483
> Subject		: WARNING: at drivers/base/sys.c:353 __sysdev_resume+0x54/0xca()
> Submitter	: Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date		: 2009-10-25 19:58 (2 days old)
> References	: http://marc.info/?l=linux-kernel&m=125650070420168&w=4
>
>
>
>    
yeah this happens on the first go of
echo mem > /sys/power/state
for both of my imac's I have here.
(on the second  try, this message does not appear)

Justin P. Mattock

Re: [Bug #14483] WARNING: at drivers/base/sys.c:353 __sysdev_resume+0x54/0xca()

[Rafael J. Wysocki]

On Monday 26 October 2009, Justin P. Mattock wrote:
> Rafael J. Wysocki wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14483
> > Subject		: WARNING: at drivers/base/sys.c:353 __sysdev_resume+0x54/0xca()
> > Submitter	: Justin Mattock<justinmattock-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date		: 2009-10-25 19:58 (2 days old)
> > References	: http://marc.info/?l=linux-kernel&m=125650070420168&w=4
> >
> >
> >
> >    
> yeah this happens on the first go of
> echo mem > /sys/power/state
> for both of my imac's I have here.
> (on the second  try, this message does not appear)

Thanks for the update.

Rafael

[Bug #14481] umount blocked for more than 120 seconds after USB drive removal

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14481
Subject		: umount blocked for more than 120 seconds after USB drive removal
Submitter	: Robert Hancock <hancockrwd-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-21 5:26 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125610280532245&w=4

Re: [Bug #14481] umount blocked for more than 120 seconds after USB drive removal

[Robert Hancock]

Haven't seen the problem again, but I haven't really tried to
reproduce it, and haven't gotten any responses about it, so yes, it
should still be listed.

On Mon, Oct 26, 2009 at 12:56 PM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14481
> Subject         : umount blocked for more than 120 seconds after USB drive removal
> Submitter       : Robert Hancock <hancockrwd-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date            : 2009-10-21 5:26 (6 days old)
> References      : http://marc.info/?l=linux-kernel&m=125610280532245&w=4
>
>
>

Re: [Bug #14481] umount blocked for more than 120 seconds after USB drive removal

[Rafael J. Wysocki]

On Tuesday 27 October 2009, Robert Hancock wrote:
> Haven't seen the problem again, but I haven't really tried to
> reproduce it, and haven't gotten any responses about it, so yes, it
> should still be listed.

Thanks for the update.

Rafael

Re: [Bug #14481] umount blocked for more than 120 seconds after USB drive removal

[Yong Zhang]

On Tue, Oct 27, 2009 at 2:56 AM, Rafael J. Wysocki <rjw-KKrjLPT3xs0@public.gmane.org> wrote:
> This message has been generated automatically as a part of a report
> of recent regressions.
>
> The following bug entry is on the current list of known regressions
> from 2.6.31.  Please verify if it still should be listed and let me know
> (either way).
>
>
> Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14481
> Subject         : umount blocked for more than 120 seconds after USB drive removal
> Submitter       : Robert Hancock <hancockrwd-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> Date            : 2009-10-21 5:26 (6 days old)
> References      : http://marc.info/?l=linux-kernel&m=125610280532245&w=4
>
>

I met the same problem on 2.6.32-rc5

Thanks,
Yong

> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: [Bug #14481] umount blocked for more than 120 seconds after USB drive removal

[Rafael J. Wysocki]

On Tuesday 27 October 2009, Yong Zhang wrote:
> On Tue, Oct 27, 2009 at 2:56 AM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> > This message has been generated automatically as a part of a report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.31.  Please verify if it still should be listed and let me know
> > (either way).
> >
> >
> > Bug-Entry       : http://bugzilla.kernel.org/show_bug.cgi?id=14481
> > Subject         : umount blocked for more than 120 seconds after USB drive removal
> > Submitter       : Robert Hancock <hancockrwd@gmail.com>
> > Date            : 2009-10-21 5:26 (6 days old)
> > References      : http://marc.info/?l=linux-kernel&m=125610280532245&w=4
> >
> >
> 
> I met the same problem on 2.6.32-rc5

Thanks for the information.

Rafael

[Bug #14482] kernel BUG at fs/dcache.c:670 +lvm +md +ext3

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14482
Subject		: kernel BUG at fs/dcache.c:670 +lvm +md +ext3
Submitter	: Alexander Clouter <alex@digriz.org.uk>
Date		: 2009-10-23 10:30 (4 days old)
References	: http://lkml.org/lkml/2009/10/23/50

[Bug #14485] System lockup running "cat /sys/kernel/debug/dri/0/i915_regs"

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14485
Subject		: System lockup running "cat /sys/kernel/debug/dri/0/i915_regs"
Submitter	: Miles Lane <miles.lane-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date		: 2009-10-26 4:00 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125652968117713&w=4

2.6.32-rc4: Reported regressions from 2.6.31

[Rafael J. Wysocki]

[Note:
  We started to get more regression reports for 2.6.32-rc (100%+ jump in the
  last 10 days.  Nevertheless, we're still receiving new reports of regressions
  from 2.6.30.]

This message contains a list of some regressions from 2.6.31, for which there
are no fixes in the mainline I know of.  If any of them have been fixed already,
please let me know.

If you know of any other unresolved regressions from 2.6.31, please let me know
either and I'll add them to the list.  Also, please let me know if any of the
entries below are invalid.

Each entry from the list will be sent additionally in an automatic reply to
this message with CCs to the people involved in reporting and handling the
issue.


Listed regressions statistics:

  Date          Total  Pending  Unresolved
  ----------------------------------------
  2009-10-12       48       31          27
  2009-10-02       22       15           9


Unresolved regressions
----------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14392
Subject		: Touchpad "paste" stops working after suspend to RAM
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-11 16:21 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125527987316493&w=4
Handled-By	: Dmitry Torokhov <dmitry.torokhov@gmail.com>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14390
Subject		: "bind" a device to a driver doesn't not work anymore
Submitter	: Éric Piel <Eric.Piel@tremplin-utc.net>
Date		: 2009-10-11 0:04 (1 days old)
References	: http://marc.info/?l=linux-kernel&m=125521979921241&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14389
Subject		: Build system issue
Submitter	: Peter Zijlstra <peterz@infradead.org>
Date		: 2009-10-09 8:58 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=575543347b5baed0ca927cb90ba8807396fe9cc9
References	: http://marc.info/?l=linux-kernel&m=125507914909152&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14387
Subject		: deadlock with fallocate
Submitter	: Thomas Neumann <tneumann@users.sourceforge.net>
Date		: 2009-10-07 3:00 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125488495526471&w=4
Handled-By	: Christoph Hellwig <hch@lst.de>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14386
Subject		: GPF in snd_hda_intel
Submitter	: Luca Tettamanti <kronos.it@gmail.com>
Date		: 2009-10-10 13:01 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125517999408019&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14384
Subject		: tbench regression with 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:51 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=59abf02644c45f1591e1374ee7bb45dc757fcb88
References	: http://marc.info/?l=linux-kernel&m=125508216713138&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14383
Subject		: hackbench regression with kernel 2.6.32-rc1
Submitter	: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
Date		: 2009-10-09 9:19 (3 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=29cd8bae396583a2ee9a3340db8c5102acf9f6fd
References	: http://marc.info/?l=linux-kernel&m=125508007510274&w=4
Handled-By	: Peter Zijlstra <a.p.zijlstra@chello.nl>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14381
Subject		: iwlagn lost connection after s2ram (with warnings)
Submitter	: Carlos R. Mafra <crmafra2@gmail.com>
Date		: 2009-10-07 14:20 (5 days old)
References	: http://marc.info/?l=linux-kernel&m=125492569119947&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14380
Subject		: Video tearing/glitching with T400 laptops
Submitter	: Theodore Ts'o <tytso@mit.edu>
Date		: 2009-10-02 22:40 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125452324520623&w=4
Handled-By	: Jesse Barnes <jbarnes@virtuousgeek.org>


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14379
Subject		: ACPI Warning for _SB_.BAT0._BIF: Converted Buffer to expected String
Submitter	: Justin Mattock <justinmattock@gmail.com>
Date		: 2009-10-08 21:46 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125504031328941&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra@navynet.it>
Date		: 2009-10-08 14:51 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14376
Subject		: Kernel NULL pointer dereference/ kvm subsystem
Submitter	: Don Dupuis <dondster@gmail.com>
Date		: 2009-10-06 14:38 (6 days old)
References	: http://marc.info/?l=linux-kernel&m=125484025021737&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14374
Subject		: MCEs caused by commit db8be50c4307dac2b37305fc59c8dc0f978d09ea
Submitter	: Nick Piggin <npiggin@suse.de>
Date		: 2009-10-02 7:34 (10 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db8be50c4307dac2b37305fc59c8dc0f978d09ea
References	: http://marc.info/?l=linux-kernel&m=125446885705223&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14373
Subject		: Task blocked for more than 120 seconds
Submitter	: Zeno Davatz <zdavatz@gmail.com>
Date		: 2009-10-02 10:16 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447858618412&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14372
Subject		: Wireless not working after suspend-resume
Submitter	: Fabio Comolli <fabio.comolli@gmail.com>
Date		: 2009-10-03 15:36 (9 days old)
References	: http://lkml.org/lkml/2009/10/3/91


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14370
Subject		: ext4 corruptions
Submitter	: Alexey Fisher <bug-track@fisher-privat.net>
Date		: 2009-10-09 19:20 (3 days old)
References	: http://marc.info/?l=linux-kernel&m=125511643504864&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14355
Subject		: USB serial regression after 2.6.31.1 with Huawei E169 GSM modem
Submitter	: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date		: 2009-10-10 03:07 (2 days old)
References	: http://marc.info/?l=linux-kernel&m=125513456327542&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14354
Subject		: Bad corruption with 2.6.32-rc1 and upwards
Submitter	: Holger Freyther <zecke@selfish.org>
Date		: 2009-10-09 15:42 (3 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14353
Subject		: BUG: sleeping function called from invalid context at kernel/mutex.c:280
Submitter	: Miles Lane <miles.lane@gmail.com>
Date		: 2009-10-05 3:39 (7 days old)
References	: http://marc.info/?l=linux-kernel&m=125471432208671&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14352
Subject		: WARNING: at net/mac80211/scan.c:267
Submitter	: Maciej Rutecki <maciej.rutecki@gmail.com>
Date		: 2009-10-08 00:30 (4 days old)
References	: http://bugzilla.intellinuxwireless.org/show_bug.cgi?id=2089#c7


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14334
Subject		: pcmcia suspend regression from 2.6.31.1 to 2.6.31.2 - Dell Inspiron 600m
Submitter	: Jose Marino <braket@hotmail.com>
Date		: 2009-10-06 15:44 (6 days old)


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14299
Subject		: oops in wireless, iwl3945 related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-29 17:12 (13 days old)
References	: http://marc.info/?l=linux-kernel&m=125424439725743&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14298
Subject		: warning at manage.c:361 (set_irq_wake), matrix-keypad related?
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 20:07 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125434130703538&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14297
Subject		: console resume broken since ba15ab0e8d
Submitter	: Sascha Hauer <s.hauer@pengutronix.de>
Date		: 2009-09-30 15:11 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125432349404060&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14296
Subject		: spitz boots but suspend/resume is broken
Submitter	: Pavel Machek <pavel@ucw.cz>
Date		: 2009-09-30 12:06 (12 days old)
References	: http://marc.info/?l=linux-kernel&m=125431244516449&w=4


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14279
Subject		: Suspend to RAM freeze totally since 2.6.32-rc1 - Acer Aspire 1511Lmi laptop
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:14 (12 days old)
First-Bad-Commit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5f68563996e812f9ca35b3939ad2a42e5d254d66


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14277
Subject		: Caught 8-bit read from freed memory in b43 driver at association
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:06 (12 days old)


Regressions with patches
------------------------

Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14382
Subject		: Transmit failure in et131x.
Submitter	: Nick Bowler <nbowler@elliptictech.com>
Date		: 2009-10-08 14:08 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501117713744&w=4
Handled-By	: Alan Cox <alan@lxorguk.ukuu.org.uk>
Patch		: http://patchwork.kernel.org/patch/52698/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14375
Subject		: Intel(R) I/OAT DMA Engine init failed
Submitter	: Alexander Beregalov <a.beregalov@gmail.com>
Date		: 2009-10-02 9:46 (10 days old)
References	: http://marc.info/?l=linux-kernel&m=125447680016160&w=4
Handled-By	: Dan Williams <dan.j.williams@intel.com>
Patch		: http://patchwork.kernel.org/patch/51808/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14302
Subject		: Kernel panic on i386 machine when booting with profile=2
Submitter	: Shi, Alex <alex.shi@intel.com>
Date		: 2009-10-01 3:23 (11 days old)
References	: http://marc.info/?l=linux-kernel&m=125436749607199&w=4
Handled-By	: Alex Shi <alex.shi@intel.com>
Patch		: http://patchwork.kernel.org/patch/50813/


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14278
Subject		: New message "NOHZ: local_softirq_pending 08" at each ping request
Submitter	: Christian Casteyde <casteyde.christian@free.fr>
Date		: 2009-09-30 18:12 (12 days old)
Handled-By	: Michael Buesch <mb@bu3sch.de>
Patch		: http://bugzilla.kernel.org/attachment.cgi?id=23220


For details, please visit the bug entries and follow the links given in
references.

As you can see, there is a Bugzilla entry for each of the listed regressions.
There also is a Bugzilla entry used for tracking the regressions from 2.6.31,
unresolved as well as resolved, at:

http://bugzilla.kernel.org/show_bug.cgi?id=14230

Please let me know if there are any Bugzilla entries that should be added to
the list in there.

Thanks,
Rafael

_______________________________________________
linux-pm mailing list
linux-pm@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/linux-pm

[Bug #14378] Problems with net/core/skbuff.c

[Rafael J. Wysocki]

This message has been generated automatically as a part of a report
of recent regressions.

The following bug entry is on the current list of known regressions
from 2.6.31.  Please verify if it still should be listed and let me know
(either way).


Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
Subject		: Problems with net/core/skbuff.c
Submitter	: Massimo Cetra <mcetra-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
Date		: 2009-10-08 14:51 (4 days old)
References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4

Re: [Bug #14378] Problems with net/core/skbuff.c

[David Miller]

From: "Rafael J. Wysocki" <rjw@sisk.pl>
Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)

> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
> Subject		: Problems with net/core/skbuff.c
> Submitter	: Massimo Cetra <mcetra@navynet.it>
> Date		: 2009-10-08 14:51 (4 days old)
> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4

I don't know what to do about this one.

The user indicates that they have the vserver patches applied,
so maybe there is some interaction with that stuff.

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo "CtRiX" Cetra]

David Miller ha scritto:
> From: "Rafael J. Wysocki" <rjw@sisk.pl>
> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
> 
>> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
>> Subject		: Problems with net/core/skbuff.c
>> Submitter	: Massimo Cetra <mcetra@navynet.it>
>> Date		: 2009-10-08 14:51 (4 days old)
>> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4
> 
> I don't know what to do about this one.
> 
> The user indicates that they have the vserver patches applied,
> so maybe there is some interaction with that stuff.

Actually i don't think that it's Vserver related.

I'm not a kernel hacker but the vserver patch doesn't seems to have much 
relation with networking (for who don't know vserver, it's a sort of 
advanced chroot).

I'm using OCFS2 and drbd. Both of those 2 things make heavy use of 
networking to share data between two hosts.

The backtrace seems to begin in the "world" of OFCS2.
I'm CC'ing ocfs mailing list where someome may have some clues.

Thanks,
Max

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

David Miller ha scritto:
> From: "Rafael J. Wysocki" <rjw@sisk.pl>
> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
>
>   
>> Bug-Entry	: http://bugzilla.kernel.org/show_bug.cgi?id=14378
>> Subject		: Problems with net/core/skbuff.c
>> Submitter	: Massimo Cetra <mcetra@navynet.it>
>> Date		: 2009-10-08 14:51 (4 days old)
>> References	: http://marc.info/?l=linux-kernel&m=125501488220358&w=4
>>     
>
> I don't know what to do about this one.
>
> The user indicates that they have the vserver patches applied,
> so maybe there is some interaction with that stuff.
>   
Actually i found another oops which is very similar to the previous one.
Here, vserver is not involved, and the problem starts at drbd which 
lives in kernel space (the other oops started at ocfs2).

Both ocfs2 and drbd make heavy use of network I/O so i guess the problem 
is something in the network layer.

Anything i can do to help to debugging and solving this issue ?

Thanks
Max




[-- Attachment #2: oops2.txt --]
[-- Type: text/plain, Size: 122916 bytes --]

[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2c ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2d ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2e ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f2f ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f30 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f31 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f32 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f33 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f34 ]---
[60257.728500] ------------[ cut here ]------------
[60257.728500] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60257.728500] Hardware name: 
[60257.728500] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60257.728500] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60257.728500] Call Trace:
[60257.728500]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60257.728500]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60257.728500]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60257.728500]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60257.728500]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60257.728500]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60257.728500]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60257.728500]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60257.728500]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60257.728500]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60257.728500]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60257.728500]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60257.728500]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60257.728500]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60257.728500]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60257.728500]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60257.728500]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60257.728500]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60257.728500]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60257.728500]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60257.728500]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60257.728500]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60257.728500]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60257.728500]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60257.728500]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60257.728500]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60257.728500]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60257.728500]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60257.728500]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60257.728500]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60257.728500]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60257.728500]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60257.728500]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60257.728500]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60257.728500]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60257.728500]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60257.728500]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60257.728500]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60257.728500]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60257.728500]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60257.728500]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60257.728500]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60257.728500]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60257.728500]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60257.728500] ---[ end trace 694817acca794f35 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f36 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f37 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f38 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f39 ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f3a ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f3b ]---
[60258.036152] ------------[ cut here ]------------
[60258.036152] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60258.036152] Hardware name: 
[60258.036152] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60258.036152] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60258.036152] Call Trace:
[60258.036152]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60258.036152]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60258.036152]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60258.036152]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60258.036152]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60258.036152]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60258.036152]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60258.036152]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60258.036152]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60258.036152]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60258.036152]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60258.036152]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60258.036152]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60258.036152]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60258.036152]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60258.036152]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60258.036152]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60258.036152]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60258.036152]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60258.036152]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60258.036152]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60258.036152]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60258.036152]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60258.036152]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60258.036152]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60258.036152]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60258.036152]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60258.036152]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60258.036152]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60258.036152]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60258.036152]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60258.036152]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60258.036152]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60258.036152]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60258.036152]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60258.036152]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60258.036152]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60258.036152]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60258.036152]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60258.036152]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60258.036152]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60258.036152]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60258.036152]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60258.036152]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60258.036152] ---[ end trace 694817acca794f3c ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f3d ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f3e ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f3f ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f40 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f41 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f42 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f43 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f44 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f45 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f46 ]---
[60259.116162] ------------[ cut here ]------------
[60259.116162] WARNING: at net/core/skbuff.c:398 skb_release_head_state+0x64/0xc8()
[60259.116162] Hardware name: 
[60259.116162] Modules linked in: ocfs2 jbd2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue crc32c netconsole configfs drbd cn loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw virtio_net pcspkr psmouse virtio_balloon parport_pc parport button processor i2c_piix4 i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod ide_cd_mod cdrom virtio_blk ata_generic ata_piix libata scsi_mod virtio_pci virtio_ring virtio piix ide_pci_generic ide_core floppy thermal fan thermal_sys
[60259.116162] Pid: 1814, comm: drbd1_receiver Tainted: G        W  2.6.31.2-vserver-navynet #1
[60259.116162] Call Trace:
[60259.116162]  <IRQ>  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81049ae1>] ? warn_slowpath_common+0x77/0xa3
[60259.116162]  [<ffffffff81253cd5>] ? skb_release_head_state+0x64/0xc8
[60259.116162]  [<ffffffff81253a1a>] ? __kfree_skb+0x9/0x7d
[60259.116162]  [<ffffffffa01e2139>] ? free_old_xmit_skbs+0x51/0x6e [virtio_net]
[60259.116162]  [<ffffffffa01e2c85>] ? start_xmit+0x26/0xf2 [virtio_net]
[60259.116162]  [<ffffffff8126934f>] ? netpoll_send_skb+0xd2/0x205
[60259.116162]  [<ffffffffa02a2216>] ? write_msg+0x90/0xeb [netconsole]
[60259.116162]  [<ffffffff81049f06>] ? __call_console_drivers+0x5e/0x6f
[60259.116162]  [<ffffffff8104a082>] ? release_console_sem+0x115/0x1ba
[60259.116162]  [<ffffffff8104a632>] ? vprintk+0x2f2/0x34b
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81308309>] ? printk+0x4e/0x5d
[60259.116162]  [<ffffffff8102b49d>] ? kvm_clock_read+0x4d/0x52
[60259.116162]  [<ffffffff81070b62>] ? getnstimeofday+0x55/0xaf
[60259.116162]  [<ffffffff81062683>] ? ktime_get_ts+0x21/0x49
[60259.116162]  [<ffffffff810626b7>] ? ktime_get+0xc/0x41
[60259.116162]  [<ffffffff81062788>] ? hrtimer_interrupt+0x9c/0x146
[60259.116162]  [<ffffffff81024a4b>] ? smp_apic_timer_interrupt+0x80/0x93
[60259.116162]  [<ffffffff81011663>] ? apic_timer_interrupt+0x13/0x20
[60259.116162]  <EOI>  [<ffffffff81157811>] ? cap_socket_recvmsg+0x0/0x3
[60259.116162]  [<ffffffff8130a9eb>] ? _spin_unlock_irq+0xd/0x31
[60259.116162]  [<ffffffff8103e6de>] ? finish_task_switch+0x5b/0xec
[60259.116162]  [<ffffffff81308f73>] ? thread_return+0x47/0xd5
[60259.116162]  [<ffffffff81293cc8>] ? tcp_current_mss+0x3f/0x5a
[60259.116162]  [<ffffffff81309213>] ? schedule_timeout+0x21/0x197
[60259.116162]  [<ffffffff8104f482>] ? local_bh_disable+0xe/0x10
[60259.116162]  [<ffffffff8130a7bf>] ? _spin_lock_bh+0x13/0x29
[60259.116162]  [<ffffffff8124f6ba>] ? release_sock+0x19/0xc3
[60259.116162]  [<ffffffff8124fd9b>] ? sk_wait_data+0x85/0xca
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff812892ab>] ? tcp_prequeue_process+0x9d/0xac
[60259.116162]  [<ffffffff8128a4ae>] ? tcp_recvmsg+0x421/0xabd
[60259.116162]  [<ffffffff8118d7cb>] ? cfq_add_rq_rb+0xd2/0xe5
[60259.116162]  [<ffffffff8124eb4a>] ? sock_common_recvmsg+0x30/0x45
[60259.116162]  [<ffffffff8124cde9>] ? sock_recvmsg+0xf7/0x18d
[60259.116162]  [<ffffffff8105f762>] ? autoremove_wake_function+0x0/0x2e
[60259.116162]  [<ffffffff8130a825>] ? _spin_lock_irqsave+0x25/0x41
[60259.116162]  [<ffffffffa0271e4d>] ? _drbd_send_cmd+0x11d/0x1eb [drbd]
[60259.116162]  [<ffffffff811851cf>] ? blk_rq_map_sg+0x12d/0x276
[60259.116162]  [<ffffffffa025c7d8>] ? drbd_recv+0x74/0x147 [drbd]
[60259.116162]  [<ffffffffa025c001>] ? drbd_may_finish_epoch+0x2c4/0x46e [drbd]
[60259.116162]  [<ffffffffa025ee5d>] ? drbd_recv_header+0x18/0xc8 [drbd]
[60259.116162]  [<ffffffffa025ef39>] ? drbdd+0x2c/0x1d6 [drbd]
[60259.116162]  [<ffffffffa026212b>] ? drbdd_init+0xf2/0x14a [drbd]
[60259.116162]  [<ffffffffa0273275>] ? drbd_thread_setup+0x16f/0x231 [drbd]
[60259.116162]  [<ffffffff81011b9a>] ? child_rip+0xa/0x20
[60259.116162]  [<ffffffff811b1e73>] ? vgacon_cursor+0x0/0x1a4
[60259.116162]  [<ffffffffa0273106>] ? drbd_thread_setup+0x0/0x231 [drbd]
[60259.116162]  [<ffffffff81011b90>] ? child_rip+0x0/0x20
[60259.116162] ---[ end trace 694817acca794f47 ]---

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

Massimo Cetra ha scritto:
> David Miller ha scritto:
>> From: "Rafael J. Wysocki" <rjw@sisk.pl>
>> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
>>
>>  
>>> Bug-Entry    : http://bugzilla.kernel.org/show_bug.cgi?id=14378
>>> Subject        : Problems with net/core/skbuff.c
>>> Submitter    : Massimo Cetra <mcetra@navynet.it>
>>> Date        : 2009-10-08 14:51 (4 days old)
>>> References    : http://marc.info/?l=linux-kernel&m=125501488220358&w=4
>>>     
>>
>> I don't know what to do about this one.
>>
>> The user indicates that they have the vserver patches applied,
>> so maybe there is some interaction with that stuff.
>>   
> Actually i found another oops which is very similar to the previous one.

And here it is another one, this time triggered by postfix, where mor 
drbd nor vserver are involved.
This is not the same server where the other oopses were grabbed.

Max

Re: [Bug #14378] Problems with net/core/skbuff.c

[Eric Dumazet]

Massimo Cetra a écrit :
> David Miller ha scritto:
>> From: "Rafael J. Wysocki" <rjw@sisk.pl>
>> Date: Mon, 12 Oct 2009 00:22:04 +0200 (CEST)
>>
>>  
>>> Bug-Entry    : http://bugzilla.kernel.org/show_bug.cgi?id=14378
>>> Subject        : Problems with net/core/skbuff.c
>>> Submitter    : Massimo Cetra <mcetra@navynet.it>
>>> Date        : 2009-10-08 14:51 (4 days old)
>>> References    : http://marc.info/?l=linux-kernel&m=125501488220358&w=4
>>>     
>>
>> I don't know what to do about this one.
>>
>> The user indicates that they have the vserver patches applied,
>> so maybe there is some interaction with that stuff.
>>   
> Actually i found another oops which is very similar to the previous one.
> Here, vserver is not involved, and the problem starts at drbd which
> lives in kernel space (the other oops started at ocfs2).
> 
> Both ocfs2 and drbd make heavy use of network I/O so i guess the problem
> is something in the network layer.
> 
> Anything i can do to help to debugging and solving this issue ?
> 
> Thanks
> Max
> 

Problem is kfree_skb() is called from irq context, wich is not allowed.

static void skb_release_head_state(struct sk_buff *skb)
{
...
if (skb->destructor) {
	WARN_ON(in_irq());
	skb->destructor();
}
...
}

virtio_net start_xmit() function calls free_old_xmit_skbs() and 
free_old_xmit_skbs() ultimately calls kfree_skb()

Quick fix would be to use dev_kfree_skb_any() instead,
because netpoll can definitly calls start_xmit()
with irq disabled.

Could you please following patch ?

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 8d00976..54bf091 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -454,7 +454,7 @@ static unsigned int free_old_xmit_skbs(struct virtnet_info *vi)
 		vi->dev->stats.tx_bytes += skb->len;
 		vi->dev->stats.tx_packets++;
 		tot_sgs += skb_vnet_hdr(skb)->num_sg;
-		kfree_skb(skb);
+		dev_kfree_skb_any(skb);
 	}
 	return tot_sgs;
 }

Re: [Bug #14378] Problems with net/core/skbuff.c

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 13 Oct 2009 12:19:51 +0200

> Quick fix would be to use dev_kfree_skb_any() instead,
> because netpoll can definitly calls start_xmit()
> with irq disabled.

Indeed, because of netpoll(), that is something drivers
must be able to cope with.

> Could you please following patch ?

Indeed, let us know how Eric's patch works.

Thanks Eric!

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

Massimo Cetra ha scritto:
> Eric Dumazet ha scritto:
>> Could you please following patch ?
>>
>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>> index 8d00976..54bf091 100644
>> --- a/drivers/net/virtio_net.c
>> +++ b/drivers/net/virtio_net.c
>> @@ -454,7 +454,7 @@ static unsigned int free_old_xmit_skbs(struct 
>> virtnet_info *vi)
>>          vi->dev->stats.tx_bytes += skb->len;
>>          vi->dev->stats.tx_packets++;
>>          tot_sgs += skb_vnet_hdr(skb)->num_sg;
>> -        kfree_skb(skb);
>> +        dev_kfree_skb_any(skb);
>>      }
>>      return tot_sgs;
>>  }
>>
>>
>>   
> Thank you very much.
>
> Compiling.
> It' will be in production in a few minutes.
> I'll let you know if the problem arises again.
> give me a couple of days because this problem is randomly triggered.
> Sometimes it happens multiple times a day, sometimes none.
>
Eric,
thanks for the patch.
The problem didn't arise again and i haven't seen any warning like that 
on both servers where that problem was happening more frequently.

I would say that it's fixed and if it's not, i'll let you know as soon 
as it happens again.

Thanks again,
Max

Re: [Bug #14378] Problems with net/core/skbuff.c

[David Miller]

From: Massimo Cetra <ctrixk-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
Date: Tue, 13 Oct 2009 11:11:17 +0200

> Here, vserver is not involved, and the problem starts at drbd which
> lives in kernel space (the other oops started at ocfs2).
> 
> Both ocfs2 and drbd make heavy use of network I/O so i guess the
> problem is something in the network layer.
> 
> Anything i can do to help to debugging and solving this issue ?

Is all of your traffic going over virtio_net?

Re: [Bug #14378] Problems with net/core/skbuff.c

[Massimo Cetra]

David Miller ha scritto:
> From: Massimo Cetra <ctrixk-BBpJ+9iBSNKonA0d6jMUrA@public.gmane.org>
> Date: Tue, 13 Oct 2009 11:11:17 +0200
>
>   
>> Here, vserver is not involved, and the problem starts at drbd which
>> lives in kernel space (the other oops started at ocfs2).
>>
>> Both ocfs2 and drbd make heavy use of network I/O so i guess the
>> problem is something in the network layer.
>>
>> Anything i can do to help to debugging and solving this issue ?
>>     
>
> Is all of your traffic going over virtio_net?
>   
Yes, indeed.
I'm compiling the patch and i'll let you know.

Thanks,
Max