From: "Marek Marczykowski-Górecki" <marmarek@invisiblethingslab.com>
To: "Jürgen Groß" <jgross@suse.com>
Cc: "Sagi Grimberg" <sagi@grimberg.me>,
"Jason Andryuk" <jandryuk@gmail.com>,
linux-nvme@lists.infradead.org, "Jens Axboe" <axboe@fb.com>,
"Keith Busch" <kbusch@kernel.org>,
xen-devel <xen-devel@lists.xenproject.org>,
"Christoph Hellwig" <hch@lst.de>,
"Roger Pau Monné" <roger.pau@citrix.com>
Subject: Re: GPF on 0xdead000000000100 in nvme_map_data - Linux 5.9.9
Date: Mon, 7 Dec 2020 14:00:04 +0100 [thread overview]
Message-ID: <20201207130004.GG1244@mail-itl> (raw)
In-Reply-To: <9bf64b27-51e8-a734-e15e-8da6d2eda736@suse.com>
[-- Attachment #1.1: Type: text/plain, Size: 2973 bytes --]
On Mon, Dec 07, 2020 at 01:00:14PM +0100, Jürgen Groß wrote:
> On 07.12.20 12:48, Marek Marczykowski-Górecki wrote:
> > On Mon, Dec 07, 2020 at 11:55:01AM +0100, Jürgen Groß wrote:
> > > Marek,
> > >
> > > On 06.12.20 17:47, Jason Andryuk wrote:
> > > > On Sat, Dec 5, 2020 at 3:29 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> > > > >
> > > > > On Fri, Dec 04, 2020 at 01:20:54PM +0100, Marek Marczykowski-Górecki wrote:
> > > > > > On Fri, Dec 04, 2020 at 01:08:03PM +0100, Christoph Hellwig wrote:
> > > > > > > On Fri, Dec 04, 2020 at 12:08:47PM +0100, Marek Marczykowski-Górecki wrote:
> > > > > > > > culprit:
> > > > > > > >
> > > > > > > > commit 9e2369c06c8a181478039258a4598c1ddd2cadfa
> > > > > > > > Author: Roger Pau Monne <roger.pau@citrix.com>
> > > > > > > > Date: Tue Sep 1 10:33:26 2020 +0200
> > > > > > > >
> > > > > > > > xen: add helpers to allocate unpopulated memory
> > > > > > > >
> > > > > > > > I'm adding relevant people and xen-devel to the thread.
> > > > > > > > For completeness, here is the original crash message:
> > > > > > >
> > > > > > > That commit definitively adds a new ZONE_DEVICE user, so it does look
> > > > > > > related. But you are not running on Xen, are you?
> > > > > >
> > > > > > I am. It is Xen dom0.
> > > > >
> > > > > I'm afraid I'm on leave and won't be able to look into this until the
> > > > > beginning of January. I would guess it's some kind of bad
> > > > > interaction between blkback and NVMe drivers both using ZONE_DEVICE?
> > > > >
> > > > > Maybe the best is to revert this change and I will look into it when
> > > > > I get back, unless someone is willing to debug this further.
> > > >
> > > > Looking at commit 9e2369c06c8a and xen-blkback put_free_pages() , they
> > > > both use page->lru which is part of the anonymous union shared with
> > > > *pgmap. That matches Marek's suspicion that the ZONE_DEVICE memory is
> > > > being used as ZONE_NORMAL.
> > > >
> > > > memmap_init_zone_device() says:
> > > > * ZONE_DEVICE pages union ->lru with a ->pgmap back pointer
> > > > * and zone_device_data. It is a bug if a ZONE_DEVICE page is
> > > > * ever freed or placed on a driver-private list.
> > >
> > > Second try, now even tested to work on a test system (without NVMe).
> >
> > It doesn't work for me:
> >
> > [ 526.023340] xen-blkback: backend/vbd/1/51712: using 2 queues, protocol 1 (x86_64-abi) persistent grants
> > [ 526.030550] xen-blkback: backend/vbd/1/51728: using 2 queues, protocol 1 (x86_64-abi) persistent grants
> > [ 526.034810] BUG: kernel NULL pointer dereference, address: 0000000000000010
>
> Oh, indeed. Silly bug. My test was with qdisk as backend :-(
>
> 3rd try...
Now it works :)
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #2: Type: text/plain, Size: 158 bytes --]
_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
WARNING: multiple messages have this Message-ID (diff)
From: "Marek Marczykowski-Górecki" <marmarek@invisiblethingslab.com>
To: "Jürgen Groß" <jgross@suse.com>
Cc: "Jason Andryuk" <jandryuk@gmail.com>,
"Roger Pau Monné" <roger.pau@citrix.com>,
"Christoph Hellwig" <hch@lst.de>,
xen-devel <xen-devel@lists.xenproject.org>,
"Keith Busch" <kbusch@kernel.org>, "Jens Axboe" <axboe@fb.com>,
"Sagi Grimberg" <sagi@grimberg.me>,
linux-nvme@lists.infradead.org
Subject: Re: GPF on 0xdead000000000100 in nvme_map_data - Linux 5.9.9
Date: Mon, 7 Dec 2020 14:00:04 +0100 [thread overview]
Message-ID: <20201207130004.GG1244@mail-itl> (raw)
In-Reply-To: <9bf64b27-51e8-a734-e15e-8da6d2eda736@suse.com>
[-- Attachment #1: Type: text/plain, Size: 2973 bytes --]
On Mon, Dec 07, 2020 at 01:00:14PM +0100, Jürgen Groß wrote:
> On 07.12.20 12:48, Marek Marczykowski-Górecki wrote:
> > On Mon, Dec 07, 2020 at 11:55:01AM +0100, Jürgen Groß wrote:
> > > Marek,
> > >
> > > On 06.12.20 17:47, Jason Andryuk wrote:
> > > > On Sat, Dec 5, 2020 at 3:29 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> > > > >
> > > > > On Fri, Dec 04, 2020 at 01:20:54PM +0100, Marek Marczykowski-Górecki wrote:
> > > > > > On Fri, Dec 04, 2020 at 01:08:03PM +0100, Christoph Hellwig wrote:
> > > > > > > On Fri, Dec 04, 2020 at 12:08:47PM +0100, Marek Marczykowski-Górecki wrote:
> > > > > > > > culprit:
> > > > > > > >
> > > > > > > > commit 9e2369c06c8a181478039258a4598c1ddd2cadfa
> > > > > > > > Author: Roger Pau Monne <roger.pau@citrix.com>
> > > > > > > > Date: Tue Sep 1 10:33:26 2020 +0200
> > > > > > > >
> > > > > > > > xen: add helpers to allocate unpopulated memory
> > > > > > > >
> > > > > > > > I'm adding relevant people and xen-devel to the thread.
> > > > > > > > For completeness, here is the original crash message:
> > > > > > >
> > > > > > > That commit definitively adds a new ZONE_DEVICE user, so it does look
> > > > > > > related. But you are not running on Xen, are you?
> > > > > >
> > > > > > I am. It is Xen dom0.
> > > > >
> > > > > I'm afraid I'm on leave and won't be able to look into this until the
> > > > > beginning of January. I would guess it's some kind of bad
> > > > > interaction between blkback and NVMe drivers both using ZONE_DEVICE?
> > > > >
> > > > > Maybe the best is to revert this change and I will look into it when
> > > > > I get back, unless someone is willing to debug this further.
> > > >
> > > > Looking at commit 9e2369c06c8a and xen-blkback put_free_pages() , they
> > > > both use page->lru which is part of the anonymous union shared with
> > > > *pgmap. That matches Marek's suspicion that the ZONE_DEVICE memory is
> > > > being used as ZONE_NORMAL.
> > > >
> > > > memmap_init_zone_device() says:
> > > > * ZONE_DEVICE pages union ->lru with a ->pgmap back pointer
> > > > * and zone_device_data. It is a bug if a ZONE_DEVICE page is
> > > > * ever freed or placed on a driver-private list.
> > >
> > > Second try, now even tested to work on a test system (without NVMe).
> >
> > It doesn't work for me:
> >
> > [ 526.023340] xen-blkback: backend/vbd/1/51712: using 2 queues, protocol 1 (x86_64-abi) persistent grants
> > [ 526.030550] xen-blkback: backend/vbd/1/51728: using 2 queues, protocol 1 (x86_64-abi) persistent grants
> > [ 526.034810] BUG: kernel NULL pointer dereference, address: 0000000000000010
>
> Oh, indeed. Silly bug. My test was with qdisk as backend :-(
>
> 3rd try...
Now it works :)
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2020-12-07 13:00 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-29 3:56 GPF on 0xdead000000000100 in nvme_map_data - Linux 5.9.9 Marek Marczykowski-Górecki
2020-11-30 16:40 ` Keith Busch
2020-12-02 0:06 ` Marek Marczykowski-Górecki
2020-12-04 11:08 ` Marek Marczykowski-Górecki
2020-12-04 11:08 ` Marek Marczykowski-Górecki
2020-12-04 12:08 ` Christoph Hellwig
2020-12-04 12:08 ` Christoph Hellwig
2020-12-04 12:20 ` Marek Marczykowski-Górecki
2020-12-04 12:20 ` Marek Marczykowski-Górecki
2020-12-05 8:28 ` Roger Pau Monné
2020-12-05 8:28 ` Roger Pau Monné
2020-12-06 16:47 ` Jason Andryuk
2020-12-06 16:47 ` Jason Andryuk
2020-12-07 8:53 ` Jürgen Groß
2020-12-07 8:53 ` Jürgen Groß
2020-12-07 9:02 ` Jürgen Groß
2020-12-07 9:02 ` Jürgen Groß
2020-12-07 10:55 ` Jürgen Groß
2020-12-07 10:55 ` Jürgen Groß
2020-12-07 11:48 ` Marek Marczykowski-Górecki
2020-12-07 11:48 ` Marek Marczykowski-Górecki
2020-12-07 12:00 ` Jürgen Groß
2020-12-07 12:00 ` Jürgen Groß
2020-12-07 13:00 ` Marek Marczykowski-Górecki [this message]
2020-12-07 13:00 ` Marek Marczykowski-Górecki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201207130004.GG1244@mail-itl \
--to=marmarek@invisiblethingslab.com \
--cc=axboe@fb.com \
--cc=hch@lst.de \
--cc=jandryuk@gmail.com \
--cc=jgross@suse.com \
--cc=kbusch@kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=roger.pau@citrix.com \
--cc=sagi@grimberg.me \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.