From: Ming Lei <ming.lei@redhat.com>
To: "Theodore Y. Ts'o" <tytso@mit.edu>
Cc: Jens Axboe <axboe@kernel.dk>,
linux-ext4@vger.kernel.org,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
linux-block@vger.kernel.org,
Linus Torvalds <torvalds@linux-foundation.org>,
linux-mm@kvack.org
Subject: Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag
Date: Fri, 25 Sep 2020 15:31:45 +0800 [thread overview]
Message-ID: <20200925073145.GC2388140@T590> (raw)
In-Reply-To: <20200925011311.GJ482521@mit.edu>
On Thu, Sep 24, 2020 at 09:13:11PM -0400, Theodore Y. Ts'o wrote:
> On Thu, Sep 24, 2020 at 10:33:45AM -0400, Theodore Y. Ts'o wrote:
> > HOWEVER, thanks to a hint from a colleague at $WORK, and realizing
> > that one of the stack traces had virtio balloon in the trace, I
> > realized that when I switched the GCE VM type from e1-standard-2 to
> > n1-standard-2 (where e1 VM's are cheaper because they use
> > virtio-balloon to better manage host OS memory utilization), problem
> > has become, much, *much* rarer (and possibly has gone away, although
> > I'm going to want to run a lot more tests before I say that
> > conclusively) on my test setup. At the very least, using an n1 VM
> > (which doesn't have virtio-balloon enabled in the hypervisor) is
> > enough to unblock ext4 development.
>
> .... and I spoke too soon. A number of runs using -rc6 are now
> failing even with the n1-standard-2 VM, so virtio-ballon may not be an
> indicator.
>
> This is why debugging this is frustrating; it is very much a heisenbug
> --- although 5.8 seems to work completely reliably, as does commits
> before 37f4a24c2469. Anything after that point will show random
> failures. :-(
It does not make sense to mention 37f4a24c2469, which is reverted in
4e2f62e566b5. Later the patch in 37f4a24c2469 is fixed and re-commited
as 568f27006577.
However, I can _not_ reproduce the issue by running the same test on
kernel built from 568f27006577 directly.
Also you have confirmed that the issue can't be fixed after reverting
568f27006577 against v5.9-rc4.
Looks the real issue(slab list corruption) should be introduced between
568f27006577 and v5.9-rc4.
thanks,
Ming
next prev parent reply other threads:[~2020-09-25 7:32 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-03 23:02 [PATCH] ext4: flag as supporting buffered async reads Jens Axboe
2020-08-11 14:31 ` Jens Axboe
2020-08-18 18:11 ` Theodore Y. Ts'o
2020-08-18 18:12 ` Jens Axboe
2020-08-21 21:26 ` Jens Axboe
2020-08-22 14:33 ` Theodore Y. Ts'o
2020-08-22 15:48 ` Jens Axboe
2020-08-24 10:56 ` Jens Axboe
2020-08-25 14:18 ` Jens Axboe
2020-08-27 1:54 ` Jens Axboe
2020-09-04 0:10 ` Jens Axboe
2020-09-04 3:55 ` Theodore Y. Ts'o
2020-09-04 14:51 ` Jens Axboe
2020-09-04 15:25 ` Darrick J. Wong
2020-09-15 4:45 ` REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver_tag Theodore Y. Ts'o
2020-09-15 7:33 ` Ming Lei
2020-09-15 22:45 ` Theodore Y. Ts'o
2020-09-15 23:09 ` Ming Lei
2020-09-16 20:20 ` Theodore Y. Ts'o
2020-09-17 2:20 ` Ming Lei
2020-09-17 14:30 ` Theodore Y. Ts'o
2020-09-17 23:08 ` Ming Lei
2020-09-24 0:59 ` Ming Lei
2020-09-24 14:33 ` Theodore Y. Ts'o
2020-09-25 1:13 ` Theodore Y. Ts'o
2020-09-25 7:31 ` Ming Lei [this message]
2020-09-25 16:19 ` Ming Lei
2020-09-25 16:32 ` Shakeel Butt
2020-09-25 16:47 ` Shakeel Butt
2020-09-25 17:22 ` Roman Gushchin
2020-09-25 17:17 ` Linus Torvalds
2020-09-25 17:22 ` Shakeel Butt
2020-09-25 17:35 ` Shakeel Butt
2020-09-25 17:47 ` Roman Gushchin
2020-09-25 17:58 ` Shakeel Butt
2020-09-25 19:19 ` Shakeel Butt
2020-09-25 20:56 ` Roman Gushchin
2020-09-25 21:18 ` Shakeel Butt
2020-09-27 17:38 ` Theodore Y. Ts'o
2020-09-26 1:43 ` Ming Lei
2020-09-26 6:42 ` Roman Gushchin
2020-09-25 1:14 ` Ming Lei
2020-09-25 2:34 ` Ming Lei
2020-10-02 20:08 ` [PATCH] ext4: flag as supporting buffered async reads Theodore Y. Ts'o
2020-10-02 20:10 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200925073145.GC2388140@T590 \
--to=ming.lei@redhat.com \
--cc=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=torvalds@linux-foundation.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.