From: Andrew Morton <akpm@linux-foundation.org>
To: Stephen Rothwell <sfr@canb.auug.org.au>,
linux-next@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
linux-scsi@vger.kernel.org, Jens Axboe <axboe@kernel.dk>,
linux-ide@vger.kernel.org, x86@kernel.org,
Tejun Heo <tj@kernel.org>
Subject: Re: linux-next: Tree for Dec 21
Date: Thu, 22 Dec 2011 15:08:36 -0800 [thread overview]
Message-ID: <20111222150836.af172886.akpm@linux-foundation.org> (raw)
In-Reply-To: <20111221151503.4d78f94f.akpm@linux-foundation.org>
On Wed, 21 Dec 2011 15:15:03 -0800
Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 21 Dec 2011 17:47:33 +1100
> Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> > I have created today's linux-next tree at
> > git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > (patches at http://www.kernel.org/pub/linux/kernel/next/ ).
>
> This kernel is seriously busted.
Too much eggnog?
> Sometime between the Dec 15 tree and
> the Dec 31 tree someone added something which appears to be causing
> writes to the (ata_piix controlled) disk to get lost. Processes are
> getting stuck in D state, usually before it reaches a login prompt.
>
> Suspects would be block, ata, scsi and possibly x86/acpi interrupt
> handling.
>
> And no, I'd rather not bisect - it would take ages. If the maintainers
> of the relevant trees can suggest specific patches to revert then I can
> take a look at that tomorrow.
>
>
> [ 558.576528] SysRq : Show Blocked State
> [ 558.576633] task PC stack pid father
> [ 558.576738] sh D 0000000000000001 0 4701 4700 0x00000080
> [ 558.576882] ffff8802493f78b8 0000000000000046 000000014a1121c0 ffff8802493f6010
> [ 558.577109] ffff88024a1121c0 00000000001d1100 ffff8802493f7fd8 0000000000004000
> [ 558.577336] ffff8802493f7fd8 00000000001d1100 ffff880255db66c0 ffff88024a1121c0
> [ 558.577568] Call Trace:
> [ 558.577623] [<ffffffff81051592>] ? sched_clock_cpu+0xc3/0xd1
> [ 558.577679] [<ffffffff81051469>] ? sched_clock_local+0x1c/0x82
> [ 558.577736] [<ffffffff81051592>] ? sched_clock_cpu+0xc3/0xd1
> [ 558.577793] [<ffffffff810631ae>] ? trace_hardirqs_off+0xd/0xf
> [ 558.577848] [<ffffffff810515cb>] ? local_clock+0x2b/0x3c
> [ 558.577905] [<ffffffff813d2744>] schedule+0x55/0x57
> [ 558.577960] [<ffffffff813d27cd>] io_schedule+0x87/0xca
> [ 558.578017] [<ffffffff811a1e72>] get_request_wait+0xbd/0x19e
> [ 558.578073] [<ffffffff810446a7>] ? wake_up_bit+0x25/0x25
> [ 558.578127] [<ffffffff8119e225>] ? elv_merge+0x9a/0xaa
> [ 558.578182] [<ffffffff811a20cc>] blk_queue_bio+0x179/0x271
> [ 558.578238] [<ffffffff811a02a9>] generic_make_request+0x9c/0xde
> [ 558.578293] [<ffffffff811a03a4>] submit_bio+0xb9/0xc4
> [ 558.578348] [<ffffffff810ffcc6>] submit_bh+0xe6/0x108
> [ 558.578404] [<ffffffff8110273c>] __block_write_full_page+0x1ec/0x2e3
> [ 558.578462] [<ffffffff81101fbe>] ? end_buffer_async_read+0x14b/0x14b
> [ 558.578518] [<ffffffff811028fb>] block_write_full_page_endio+0xc8/0xcc
> [ 558.578573] [<ffffffff8110290f>] block_write_full_page+0x10/0x12
> [ 558.578631] [<ffffffff811417cd>] ext3_writeback_writepage+0xaa/0x11d
> [ 558.578690] [<ffffffff810a0ed0>] __writepage+0x15/0x34
> [ 558.578744] [<ffffffff810a1913>] write_cache_pages+0x240/0x33e
> [ 558.578799] [<ffffffff810a0ebb>] ? set_page_dirty+0x65/0x65
> [ 558.578855] [<ffffffff81064abf>] ? trace_hardirqs_on+0xd/0xf
> [ 558.578911] [<ffffffff810a1a54>] generic_writepages+0x43/0x5a
> [ 558.578967] [<ffffffff810a1a91>] do_writepages+0x26/0x28
> [ 558.579022] [<ffffffff8109a8cf>] __filemap_fdatawrite_range+0x4e/0x50
> [ 558.579078] [<ffffffff8109aee8>] filemap_flush+0x17/0x19
> [ 558.579134] [<ffffffff8113f2c2>] ext3_release_file+0x2e/0xa4
> [ 558.579190] [<ffffffff810dbdce>] fput+0x10f/0x1cd
> [ 558.579244] [<ffffffff810d86e0>] filp_close+0x70/0x7b
> [ 558.579300] [<ffffffff8102c09b>] put_files_struct+0x16c/0x2c1
> [ 558.579356] [<ffffffff8102bf7b>] ? put_files_struct+0x4c/0x2c1
> [ 558.579412] [<ffffffff8102c236>] exit_files+0x46/0x4e
> [ 558.579465] [<ffffffff8102dd2a>] do_exit+0x246/0x73c
> [ 558.579521] [<ffffffff813d3c4a>] ? retint_swapgs+0xe/0x13
> [ 558.579576] [<ffffffff8102e2a4>] do_group_exit+0x84/0xb2
> [ 558.579743] [<ffffffff8102e2e4>] sys_exit_group+0x12/0x16
> [ 558.579910] [<ffffffff813d9562>] system_call_fastpath+0x16/0x1b
A large amount of block core code was merged in the Dec 15 - Dec 21
window. Tejun...
I sucked all the patches out of git, reverted them in reverse order and
did a quilt bisect (having been repeatedly traumatised by git bisect,
due to all the bisection holes we're adding, due to that
never-rebase-your-tree thing).
revert-64c42998f14d5894ea3138625897d620b30c8e4e.patch
revert-274193224cdabd687d804a26e0150bb20f2dd52c.patch
revert-4a0b75c7d02c2bd46ed227d4ba5941ba8a0aba5d.patch
revert-ff56c895cf43c7896e5a5f31e2d55bb7fdbdb63e.patch
revert-4eabc941259f9d8c8fb71746d3f30c87e1d9e49b.patch
revert-f1f8cc94651738b418ba54c039df536303b91704.patch
revert-9b84cacd013996f244d85b3d873287c2a8f88658.patch
revert-7e5a8794492e43e9eebb68a98a23be055888ccd0.patch
revert-3d3c2379feb177a5fd55bb0ed76776dc9d4f3243.patch
revert-47fdd4ca96bf4b28ac4d05d7a6e382df31d3d758.patch
revert-a612fddf0d8090f2877305c9168b6c1a34fb5d90.patch
revert-c58698073218f2c8f2fc5982fa3938c2d3803b9f.patch
revert-22f746e235a5cbee2a6ca9887b1be2aa7d31fe71.patch
revert-f8fc877d3c1f10457d0d73d8540a0c51a1fa718a.patch
revert-f2dbd76a0a994bc1d5a3d0e7c844cc373832e86c.patch BAD
revert-1238033c79e92e5c315af12e45396f1a78c73dec.patch
revert-b50b636bce6293fa858cc7ff6c3ffe4920d90006.patch
revert-b9a1920837bc53430d339380e393a6e4c372939f.patch
revert-b2efa05265d62bc29f3a64400fad4b44340eedb8.patch
revert-f1a4f4d35ff30a328d5ea28f6cc826b2083111d2.patch
revert-216284c352a0061f5b20acff2c4e50fb43fea183.patch
revert-dc86900e0a8f665122de6faadd27fb4c6d2b3e4d.patch
revert-283287a52e3c3f7f8f9da747f4b8c5202740d776.patch
revert-09ac46c429464c919d04bb737b27edd84d944f02.patch BAD
revert-6e736be7f282fff705db7c34a15313281b372a76.patch GOOD
revert-42ec57a8f68311bbbf4ff96a5d33c8a2e90b9d05.patch GOOD
revert-a73f730d013ff2788389fd0c46ad3e5510f124e6.patch
revert-8ba61435d73f2274e12d4d823fde06735e8f6a54.patch GOOD
revert-481a7d64790cd7ca61a8bbcbd9d017ce58e6fe39.patch
revert-34f6055c80285e4efb3f602a9119db75239744dc.patch
revert-1ba64edef6051d2ec79bb2fbd3a0c8f0df00ab55.patch GOOD
At the f2dbd76a0a994bc1d5a3d0e7c844cc373832e86 pivot point the kernel
went odd, got stuck, slowly emitting "cfq: cic link failed!" messages.
So we've added yet another bisection hole in there somewhere.
And the bisection indicates that this (drop-dead show-stopping
box-killing) regression was added by
6e736be7f282fff705db7c34a15313281b372a76 ("block: make ioc get/put
interface more conventional and fix race on alloction").
Please don't add another bisection hole when fixing this.</troll>
next prev parent reply other threads:[~2011-12-22 23:08 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-21 6:47 linux-next: Tree for Dec 21 Stephen Rothwell
2011-12-21 20:11 ` linux-next: Tree for Dec 21 (xen) Randy Dunlap
2011-12-21 19:30 ` Konrad Rzeszutek Wilk
2011-12-21 20:32 ` [PATCH] usb: fix renesas mod_gadget printk format warning Randy Dunlap
2011-12-21 21:16 ` Felipe Balbi
2011-12-21 22:19 ` Randy Dunlap
2011-12-21 21:24 ` Greg KH
2011-12-21 23:15 ` linux-next: Tree for Dec 21 Andrew Morton
2011-12-22 23:08 ` Andrew Morton [this message]
2011-12-22 23:20 ` Tejun Heo
2011-12-22 23:24 ` Andrew Morton
2011-12-22 23:38 ` Tejun Heo
2011-12-22 23:44 ` Andrew Morton
2011-12-22 23:46 ` Tejun Heo
2011-12-23 0:42 ` Tejun Heo
2011-12-24 5:13 ` Hugh Dickins
2011-12-25 1:02 ` [PATCH block/for-3.3/core] block: an exiting task should be allowed to create io_context Tejun Heo
2011-12-25 13:29 ` Jens Axboe
2011-12-27 22:07 ` Andrew Morton
2011-12-28 8:33 ` Hugh Dickins
2011-12-28 16:48 ` Tejun Heo
2011-12-28 17:50 ` Hugh Dickins
2011-12-28 17:55 ` Tejun Heo
2011-12-28 21:19 ` Tejun Heo
2012-01-03 17:35 ` Tejun Heo
2012-01-03 17:59 ` Tejun Heo
2012-01-03 20:09 ` Tejun Heo
2012-01-03 20:20 ` Jens Axboe
2012-01-03 22:13 ` Tejun Heo
2012-01-03 22:35 ` Tejun Heo
2012-01-05 1:24 ` Tejun Heo
2012-01-05 18:36 ` Hugh Dickins
2012-01-05 18:38 ` Tejun Heo
2012-01-06 2:17 ` [PATCH block:for-3.3/core] cfq: merged request shouldn't jump to a different cfqq Tejun Heo
2012-01-06 2:36 ` Tejun Heo
2012-01-06 3:14 ` Shaohua Li
2012-01-06 3:04 ` Tejun Heo
2012-01-06 3:30 ` Tejun Heo
2012-01-06 3:52 ` [PATCH block:for-3.3/core] block: disable ELEVATOR_INSERT_SORT_MERGE Tejun Heo
2012-01-06 4:19 ` Shaohua Li
2012-01-06 4:38 ` Tejun Heo
2012-01-06 8:15 ` Shaohua Li
2012-01-06 15:34 ` Tejun Heo
2012-01-06 3:34 ` [PATCH block:for-3.3/core] cfq: merged request shouldn't jump to a different cfqq Shaohua Li
2012-01-06 3:22 ` Tejun Heo
2012-01-06 4:15 ` Shaohua Li
2012-01-06 4:40 ` Tejun Heo
2012-01-06 2:47 ` Shaohua Li
2011-12-25 16:08 ` linux-next: Tree for Dec 21 Sedat Dilek
2011-12-25 23:55 ` Stephen Rothwell
-- strict thread matches above, loose matches on Subject: below --
2012-12-21 3:25 Stephen Rothwell
2014-12-21 10:28 Stephen Rothwell
2015-12-21 8:05 Stephen Rothwell
2016-12-21 2:22 Stephen Rothwell
2017-12-21 4:41 Stephen Rothwell
2018-12-21 8:32 Stephen Rothwell
2018-12-21 16:21 ` Guenter Roeck
2018-12-21 16:33 ` Ard Biesheuvel
2018-12-21 20:57 ` Florian Westphal
2018-12-22 6:43 ` Chandan Rajendra
2018-12-24 8:20 ` Greg Kroah-Hartman
2018-12-31 14:01 ` Vinod Koul
2020-12-21 3:18 Stephen Rothwell
2021-12-21 12:27 Stephen Rothwell
2023-12-21 10:42 Stephen Rothwell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111222150836.af172886.akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=linux-ide@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-next@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=sfr@canb.auug.org.au \
--cc=tj@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).