linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: David Hildenbrand <david@redhat.com>
To: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
	devel@linuxdriverproject.org, linux-fsdevel@vger.kernel.org,
	linux-pm@vger.kernel.org, xen-devel@lists.xenproject.org,
	kexec-ml <kexec@lists.infradead.org>,
	pv-drivers@vmware.com, David Hildenbrand <david@redhat.com>,
	Alexander Duyck <alexander.h.duyck@linux.intel.com>,
	Alexey Dobriyan <adobriyan@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Arnd Bergmann <arnd@arndb.de>, Baoquan He <bhe@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Christian Hansen <chansen3@cisco.com>,
	Dave Young <dyoung@redhat.com>,
	David Rientjes <rientjes@google.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Jonathan Corbet <corbet@lwn.net>, Juergen Gross <jgross@suse.com>,
	Julien Freche <jfreche@vmware.com>,
	Kairui Song <kasong@redhat.com>,
	Kazuhito Hagio <k-hagio@ab.jp.nec.com>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Konstantin Khlebnikov <koct9i@gmail.com>,
	"K. Y. Srinivasan" <kys@microsoft.com>,
	Len Brown <len.brown@intel.com>,
	Lianbo Jiang <lijiang@redhat.com>,
	Matthew Wilcox <willy@infradead.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Michal Hocko <mhocko@kernel.org>, Michal Hocko <mhocko@suse.com>,
	Mike Rapoport <rppt@linux.vnet.ibm.com>,
	Miles Chen <miles.chen@mediatek.com>,
	Nadav Amit <namit@vmware.com>,
	Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
	Omar Sandoval <osandov@fb.com>, Pavel Machek <pavel@ucw.cz>,
	Pavel Tatashin <pasha.tatashin@oracle.com>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Vlastimil Babka <vbabka@suse.cz>,
	Xavier Deguillard <xdeguillard@vmware.com>
Subject: [PATCH v1 0/8] mm/kdump: allow to exclude pages that are logically offline
Date: Mon, 19 Nov 2018 11:16:08 +0100	[thread overview]
Message-ID: <20181119101616.8901-1-david@redhat.com> (raw)

Right now, pages inflated as part of a balloon driver will be dumped
by dump tools like makedumpfile. While XEN is able to check in the
crash kernel whether a certain pfn is actuall backed by memory in the
hypervisor (see xen_oldmem_pfn_is_ram) and optimize this case, dumps of
virtio-balloon, hv-balloon and VMWare balloon inflated memory will
essentially result in zero pages getting allocated by the hypervisor and
the dump getting filled with this data.

The allocation and reading of zero pages can directly be avoided if a
dumping tool could know which pages only contain stale information not to
be dumped.

Also for XEN, calling into the kernel and asking the hypervisor if a
pfn is backed can be avoided if the duming tool would skip such pages
right from the beginning.

Dumping tools have no idea whether a given page is part of a balloon driver
and shall not be dumped. Esp. PG_reserved cannot be used for that purpose
as all memory allocated during early boot is also PG_reserved, see
discussion at [1]. So some other way of indication is required and a new
page flag is frowned upon.

We have PG_balloon (MAPCOUNT value), which is essentially unused now. I
suggest renaming it to something more generic (PG_offline) to mark pages as
logically offline. This flag can than e.g. also be used by virtio-mem in
the future to mark subsections as offline. Or by other code that wants to
put pages logically offline (e.g. later maybe poisoned pages that shall
no longer be used).

This series converts PG_balloon to PG_offline, allows dumping tools to
query the value to detect such pages and marks pages in the hv-balloon
and XEN balloon properly as PG_offline. Note that virtio-balloon already
set pages to PG_balloon (and now PG_offline).

Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while
onlining a section via online_page_callback) would under some condicions
result in a kernel panic when dumping them.

As I don't have access to neither XEN nor Hyper-V nor VMWare installations,
this was only tested with the virtio-balloon and pages were properly
skipped when dumping. I'll also attach the makedumpfile patch to this
series.

[1] https://lkml.org/lkml/2018/7/20/566

RFC -> v1:
- Add "PM / Hibernate: use pfn_to_online_page()"
- Add "vmw_balloon: mark inflated pages PG_offline"
- "mm: convert PG_balloon to PG_offline"
-- After discussions, also rename the UAPI bit name (KPF_BALLOON -> KPF_OFFLINE)

David Hildenbrand (8):
  mm: balloon: update comment about isolation/migration/compaction
  mm: convert PG_balloon to PG_offline
  kexec: export PG_offline to VMCOREINFO
  xen/balloon: mark inflated pages PG_offline
  hv_balloon: mark inflated pages PG_offline
  vmw_balloon: mark inflated pages PG_offline
  PM / Hibernate: use pfn_to_online_page()
  PM / Hibernate: exclude all PageOffline() pages

 Documentation/admin-guide/mm/pagemap.rst |  9 ++++---
 drivers/hv/hv_balloon.c                  | 14 ++++++++--
 drivers/misc/vmw_balloon.c               | 32 ++++++++++++++++++++++
 drivers/xen/balloon.c                    |  3 +++
 fs/proc/page.c                           |  4 +--
 include/linux/balloon_compaction.h       | 34 +++++++++---------------
 include/linux/page-flags.h               | 11 +++++---
 include/uapi/linux/kernel-page-flags.h   |  2 +-
 kernel/crash_core.c                      |  2 ++
 kernel/power/snapshot.c                  | 13 +++++----
 tools/vm/page-types.c                    |  2 +-
 11 files changed, 87 insertions(+), 39 deletions(-)

-- 
2.17.2

             reply	other threads:[~2018-11-19 10:16 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-19 10:16 David Hildenbrand [this message]
2018-11-19 10:16 ` [PATCH v1 1/8] mm: balloon: update comment about isolation/migration/compaction David Hildenbrand
2018-11-21  1:56   ` Michael S. Tsirkin
2018-11-19 10:16 ` [PATCH v1 2/8] mm: convert PG_balloon to PG_offline David Hildenbrand
2018-11-19 11:56   ` Konstantin Khlebnikov
2018-11-19 16:06   ` David Hildenbrand
2018-11-21  1:58   ` Michael S. Tsirkin
2018-11-21  2:44   ` Pankaj Gupta
2018-11-19 10:16 ` [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO David Hildenbrand
2018-11-21  1:59   ` Michael S. Tsirkin
2018-11-21  2:58   ` Dave Young
2018-11-21  6:04   ` Baoquan He
2018-11-21  8:50     ` David Hildenbrand
2018-11-19 10:16 ` [PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline David Hildenbrand
2018-11-19 12:22   ` Juergen Gross
2018-11-19 13:07     ` David Hildenbrand
2018-11-19 10:16 ` [PATCH v1 5/8] hv_balloon: " David Hildenbrand
2018-11-20  8:45   ` Pankaj Gupta
2018-11-20  8:49     ` David Hildenbrand
2018-11-20  9:37       ` Pankaj Gupta
2018-11-21  1:04         ` Pankaj Gupta
2018-11-19 10:16 ` [PATCH v1 6/8] vmw_balloon: " David Hildenbrand
2018-11-21  3:22   ` Nadav Amit
2018-11-21  8:54     ` David Hildenbrand
2018-11-19 10:16 ` [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page() David Hildenbrand
2018-11-19 12:13   ` Michal Hocko
2018-11-19 13:16   ` Pavel Machek
2018-11-20 21:23   ` Rafael J. Wysocki
2018-11-19 10:16 ` [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages David Hildenbrand
2018-11-20 21:22   ` Rafael J. Wysocki
2018-11-21 11:35   ` William Kucharski
2018-11-21 12:21     ` David Hildenbrand
2018-11-19 10:18 ` [PATCH v1] makedumpfile: exclude pages that are logically offline David Hildenbrand
2018-11-21 14:58   ` Kazuhito Hagio
2018-11-21 15:35     ` David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181119101616.8901-1-david@redhat.com \
    --to=david@redhat.com \
    --cc=adobriyan@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.h.duyck@linux.intel.com \
    --cc=arnd@arndb.de \
    --cc=bhe@redhat.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bp@alien8.de \
    --cc=chansen3@cisco.com \
    --cc=corbet@lwn.net \
    --cc=devel@linuxdriverproject.org \
    --cc=dyoung@redhat.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=haiyangz@microsoft.com \
    --cc=jfreche@vmware.com \
    --cc=jgross@suse.com \
    --cc=k-hagio@ab.jp.nec.com \
    --cc=kasong@redhat.com \
    --cc=kexec@lists.infradead.org \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=koct9i@gmail.com \
    --cc=kys@microsoft.com \
    --cc=len.brown@intel.com \
    --cc=lijiang@redhat.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=mhocko@kernel.org \
    --cc=mhocko@suse.com \
    --cc=miles.chen@mediatek.com \
    --cc=mst@redhat.com \
    --cc=n-horiguchi@ah.jp.nec.com \
    --cc=namit@vmware.com \
    --cc=osandov@fb.com \
    --cc=pasha.tatashin@oracle.com \
    --cc=pavel@ucw.cz \
    --cc=pv-drivers@vmware.com \
    --cc=rientjes@google.com \
    --cc=rjw@rjwysocki.net \
    --cc=rppt@linux.vnet.ibm.com \
    --cc=sfr@canb.auug.org.au \
    --cc=sstabellini@kernel.org \
    --cc=sthemmin@microsoft.com \
    --cc=vbabka@suse.cz \
    --cc=vkuznets@redhat.com \
    --cc=willy@infradead.org \
    --cc=xdeguillard@vmware.com \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).