From: Borislav Petkov <bp-Gina5bIWoIWzQB+pC5nmwQ@public.gmane.org>
To: Jiang Liu <jiang.liu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Cc: x86-ml <x86-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
"Rafael J . Wysocki"
<rjw-LthD3rsA81gm4RdzfppkhA@public.gmane.org>,
lkml <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
Thomas Gleixner <tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org>
Subject: Re: [PATCH] iommu/amd: Implement syscore_ops.shutdown()
Date: Mon, 4 Aug 2014 12:12:17 +0200 [thread overview]
Message-ID: <20140804101217.GA4808@pd.tnic> (raw)
In-Reply-To: <1406946339-4297-1-git-send-email-jiang.liu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
On Sat, Aug 02, 2014 at 10:25:39AM +0800, Jiang Liu wrote:
> During hibernation or shutdown, AMD iommu generates warnings on some
> platforms as below:
> [ 89.089832] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000080 flags=0x0020]
> [ 89.102239] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000000 flags=0x0000]
> [ 89.114684] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.127162] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.139576] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.152017] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.164481] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
>
> It may be caused by that the firmware takes back the device after the OS
> released it and now the legacy emulation tries to do DMA with it. But
> since there is an IOMMU the physical addresses it tries to DMA to is
> not mapped and it generated IO page faults. So explicitly shutdown
> IOMMU units during hibernation or poweroff.
Unfortunately, this doesn't help. The current observation is that I
don't get the IOPFs during *every* suspend-to-disk cycle but I do get
them from time to time. Here are all IOPFs from 10ish suspend/resume
cycles with this patch. The newlines between them show how the IOPFs
happen: sometimes it is the GPU (device 01:00.0) with a single IOPF...
[ 78.482122] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
[ 171.593660] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
and sometimes it is the GPU and OHCI0 with multiple ones.
[ 280.680624] IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
[ 280.694539] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000080 flags=0x0020]
[ 280.707917] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000000 flags=0x0000]
[ 280.721234] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.734565] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.747877] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.761227] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.774444] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 290.686170] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
[ 49.393307] IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
Other than that, the box is fine AFAICT.
--
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--
WARNING: multiple messages have this Message-ID (diff)
From: Borislav Petkov <bp@alien8.de>
To: Jiang Liu <jiang.liu@linux.intel.com>
Cc: "Rafael J . Wysocki" <rjw@rjwysocki.net>,
Thomas Gleixner <tglx@linutronix.de>,
Joerg Roedel <joro@8bytes.org>, x86-ml <x86@kernel.org>,
lkml <linux-kernel@vger.kernel.org>,
iommu@lists.linux-foundation.org
Subject: Re: [PATCH] iommu/amd: Implement syscore_ops.shutdown()
Date: Mon, 4 Aug 2014 12:12:17 +0200 [thread overview]
Message-ID: <20140804101217.GA4808@pd.tnic> (raw)
In-Reply-To: <1406946339-4297-1-git-send-email-jiang.liu@linux.intel.com>
On Sat, Aug 02, 2014 at 10:25:39AM +0800, Jiang Liu wrote:
> During hibernation or shutdown, AMD iommu generates warnings on some
> platforms as below:
> [ 89.089832] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000080 flags=0x0020]
> [ 89.102239] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000000 flags=0x0000]
> [ 89.114684] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.127162] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.139576] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.152017] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
> [ 89.164481] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
>
> It may be caused by that the firmware takes back the device after the OS
> released it and now the legacy emulation tries to do DMA with it. But
> since there is an IOMMU the physical addresses it tries to DMA to is
> not mapped and it generated IO page faults. So explicitly shutdown
> IOMMU units during hibernation or poweroff.
Unfortunately, this doesn't help. The current observation is that I
don't get the IOPFs during *every* suspend-to-disk cycle but I do get
them from time to time. Here are all IOPFs from 10ish suspend/resume
cycles with this patch. The newlines between them show how the IOPFs
happen: sometimes it is the GPU (device 01:00.0) with a single IOPF...
[ 78.482122] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
[ 171.593660] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
and sometimes it is the GPU and OHCI0 with multiple ones.
[ 280.680624] IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
[ 280.694539] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000080 flags=0x0020]
[ 280.707917] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x0000000000000000 flags=0x0000]
[ 280.721234] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.734565] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.747877] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.761227] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 280.774444] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:12.0 domain=0x0009 address=0x00000000ffffffc0 flags=0x0010]
[ 290.686170] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
[ 49.393307] IO_PAGE_FAULT device=01:00.0 domain=0x0014 address=0x0000000020001000 flags=0x0000]
Other than that, the box is fine AFAICT.
--
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--
next prev parent reply other threads:[~2014-08-04 10:12 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20140728175326.GA7100@pd.tnic>
2014-07-28 20:59 ` rc7 + tip/master suspend fun Rafael J. Wysocki
2014-07-28 21:02 ` Thomas Gleixner
2014-07-28 22:02 ` Rafael J. Wysocki
2014-07-29 2:17 ` Jiang Liu
2014-07-30 9:37 ` [PATCH] x86, irq: Keep IRQ assignment for PCI devices during suspend/hibernation Jiang Liu
2014-07-30 17:58 ` Borislav Petkov
2014-07-31 0:33 ` Jiang Liu
2014-07-31 10:39 ` Borislav Petkov
[not found] ` <20140731103941.GA4375-fF5Pk5pvG8Y@public.gmane.org>
2014-07-31 14:41 ` Jiang Liu
2014-07-31 14:41 ` Jiang Liu
[not found] ` <53DA55A0.6070005-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2014-07-31 15:21 ` Borislav Petkov
2014-07-31 15:21 ` Borislav Petkov
2014-07-31 16:36 ` Jiang Liu
2014-07-31 16:56 ` Borislav Petkov
2014-08-01 10:56 ` [PATCH] x86, irq: Keep IRQ assignment for PCI devices during suspend/hibernation, bisected Borislav Petkov
2014-08-01 12:27 ` Jiang Liu
2014-08-01 14:39 ` Borislav Petkov
2014-08-01 16:11 ` Borislav Petkov
2014-08-01 22:14 ` Jörg Rödel
2014-08-01 22:50 ` Borislav Petkov
[not found] ` <20140801225049.GC4553-fF5Pk5pvG8Y@public.gmane.org>
2014-08-02 2:25 ` [PATCH] iommu/amd: Implement syscore_ops.shutdown() Jiang Liu
2014-08-02 2:25 ` Jiang Liu
[not found] ` <1406946339-4297-1-git-send-email-jiang.liu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2014-08-04 10:12 ` Borislav Petkov [this message]
2014-08-04 10:12 ` Borislav Petkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140804101217.GA4808@pd.tnic \
--to=bp-gina5biwoiwzqb+pc5nmwq@public.gmane.org \
--cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
--cc=jiang.liu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=rjw-LthD3rsA81gm4RdzfppkhA@public.gmane.org \
--cc=tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org \
--cc=x86-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.