From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <rrichter@cavium.com>
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Thu, 6 Oct 2016 11:52:07 +0200
Message-ID: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>, linux-efi@vger.kernel.org, David Daney <david.daney@cavium.com>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, linux-kernel@vger.kernel.org, Robert Richter <rrichter@cavium.com>, Hanjun Guo <hanjun.guo@linaro.org>, linux-arm-kernel@lists.infradead.org
List-Id: linux-efi@vger.kernel.org

There is a memory setup problem on ThunderX systems with certain
memory configurations. The symptom is

 kernel BUG at mm/page_alloc.c:1848!

This happens for some configs with 64k page size enabled. The bug
triggers for page zones with some pages in the zone not assigned to
this particular zone. In my case some pages that are marked as nomap
were not reassigned to the new zone of node 1, so those are still
assigned to node 0.

The reason for the mis-configuration is a change in pfn_valid() which
reports pages marked nomap as invalid:

 68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping

This causes pages marked as nomap being no long reassigned to the new
zone in memmap_init_zone() by calling __init_single_pfn().

Fixing this by restoring the old behavior of pfn_valid() to use
memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
to use memblock_is_map_memory() where necessary. This only affects
code in ioremap.c. The code in mmu.c still can use the new version of
pfn_valid().

Should be marked stable v4.5..

Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 arch/arm64/mm/init.c    | 2 +-
 arch/arm64/mm/ioremap.c | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index bbb7ee76e319..25b8659c2a9f 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -147,7 +147,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 #ifdef CONFIG_HAVE_ARCH_PFN_VALID
 int pfn_valid(unsigned long pfn)
 {
-	return memblock_is_map_memory(pfn << PAGE_SHIFT);
+	return memblock_is_memory(pfn << PAGE_SHIFT);
 }
 EXPORT_SYMBOL(pfn_valid);
 #endif
diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c
index 01e88c8bcab0..c17c220b0c48 100644
--- a/arch/arm64/mm/ioremap.c
+++ b/arch/arm64/mm/ioremap.c
@@ -21,6 +21,7 @@
  */
 
 #include <linux/export.h>
+#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <linux/vmalloc.h>
 #include <linux/io.h>
@@ -55,7 +56,7 @@ static void __iomem *__ioremap_caller(phys_addr_t phys_addr, size_t size,
 	/*
 	 * Don't allow RAM to be mapped.
 	 */
-	if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr))))
+	if (WARN_ON(memblock_is_map_memory(phys_addr)))
 		return NULL;
 
 	area = get_vm_area_caller(size, VM_IOREMAP, caller);
@@ -96,7 +97,7 @@ EXPORT_SYMBOL(__iounmap);
 void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size)
 {
 	/* For normal memory we already have a cacheable mapping. */
-	if (pfn_valid(__phys_to_pfn(phys_addr)))
+	if (memblock_is_map_memory(phys_addr))
 		return (void __iomem *)__phys_to_virt(phys_addr);
 
 	return __ioremap_caller(phys_addr, size, __pgprot(PROT_NORMAL),
-- 
2.7.0.rc3

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Thu, 6 Oct 2016 11:00:33 +0100
Message-ID: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

Hi Robert,

Apologies for only responding now. I did not quite manage to get my
head around your original email yet, but I don't think this patch is
the correct solution.

On 6 October 2016 at 10:52, Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
>
>  kernel BUG at mm/page_alloc.c:1848!
>
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
>
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
>
>  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
>

These pages are owned by the firmware, which may map it with
attributes that conflict with the attributes we use for the linear
mapping. This means they should not be covered by the linear mapping.

> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
>

This sounds like the root cause of your issue. Could we not fix that instead?

> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory().

This is incorrect imo. In general, pfn_valid() means ordinary memory
covered by the linear mapping and the struct page array. Returning
reserved ranges that the kernel should not even touch only to please
the NUMA code seems like an inappropriate way to deal with this issue.

> Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().
>
> Should be marked stable v4.5..
>
> Signed-off-by: Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> ---
>  arch/arm64/mm/init.c    | 2 +-
>  arch/arm64/mm/ioremap.c | 5 +++--
>  2 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index bbb7ee76e319..25b8659c2a9f 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -147,7 +147,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
>  #ifdef CONFIG_HAVE_ARCH_PFN_VALID
>  int pfn_valid(unsigned long pfn)
>  {
> -       return memblock_is_map_memory(pfn << PAGE_SHIFT);
> +       return memblock_is_memory(pfn << PAGE_SHIFT);
>  }
>  EXPORT_SYMBOL(pfn_valid);
>  #endif
> diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c
> index 01e88c8bcab0..c17c220b0c48 100644
> --- a/arch/arm64/mm/ioremap.c
> +++ b/arch/arm64/mm/ioremap.c
> @@ -21,6 +21,7 @@
>   */
>
>  #include <linux/export.h>
> +#include <linux/memblock.h>
>  #include <linux/mm.h>
>  #include <linux/vmalloc.h>
>  #include <linux/io.h>
> @@ -55,7 +56,7 @@ static void __iomem *__ioremap_caller(phys_addr_t phys_addr, size_t size,
>         /*
>          * Don't allow RAM to be mapped.
>          */
> -       if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr))))
> +       if (WARN_ON(memblock_is_map_memory(phys_addr)))
>                 return NULL;
>
>         area = get_vm_area_caller(size, VM_IOREMAP, caller);
> @@ -96,7 +97,7 @@ EXPORT_SYMBOL(__iounmap);
>  void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size)
>  {
>         /* For normal memory we already have a cacheable mapping. */
> -       if (pfn_valid(__phys_to_pfn(phys_addr)))
> +       if (memblock_is_map_memory(phys_addr))
>                 return (void __iomem *)__phys_to_virt(phys_addr);
>
>         return __ioremap_caller(phys_addr, size, __pgprot(PROT_NORMAL),
> --
> 2.7.0.rc3
>

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 6 Oct 2016 18:11:14 +0200
Message-ID: <20161006161114.GH22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Cc: Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

Ard,

thank you for your answer and you explanation.

On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> On 6 October 2016 at 10:52, Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> > There is a memory setup problem on ThunderX systems with certain
> > memory configurations. The symptom is
> >
> >  kernel BUG at mm/page_alloc.c:1848!
> >
> > This happens for some configs with 64k page size enabled. The bug
> > triggers for page zones with some pages in the zone not assigned to
> > this particular zone. In my case some pages that are marked as nomap
> > were not reassigned to the new zone of node 1, so those are still
> > assigned to node 0.
> >
> > The reason for the mis-configuration is a change in pfn_valid() which
> > reports pages marked nomap as invalid:
> >
> >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> >
> 
> These pages are owned by the firmware, which may map it with
> attributes that conflict with the attributes we use for the linear
> mapping. This means they should not be covered by the linear mapping.
> 
> > This causes pages marked as nomap being no long reassigned to the new
> > zone in memmap_init_zone() by calling __init_single_pfn().
> >
> 
> This sounds like the root cause of your issue. Could we not fix that instead?

Yes, this is proposal b) from my last mail that would work too: I
implemented an arm64 private early_pfn_valid() function that uses
memblock_is_memory() to setup all pages of a zone. Though, I think
this is the wrong way and thus I prefer this patch instead. I see
serveral reasons for this:

Inconsistent use of struct *page, it is initialized but never used
again.

Other archs only do a basic range check in pfn_valid(), the default
implementation just returns if the whole section is valid. As I
understand the code, if the mem range is not aligned to the section,
then there will be pfn's in the section that don't have physical mem
attached. The page is then just initialized, it's not marked reserved
nor the refcount is non-zero. It is then simply not used. This is how
no-map pages should be handled too.

I think pfn_valid() is just a quick check if the pfn's struct *page
can be used. There is a good description for this in include/linux/
mmzone.h. So there can be memory holes that have a valid pfn.

If the no-map memory needs special handling, then additional checks
need to be added to the particular code (as in ioremap.c). It's imo
wrong to (mis-)use pfn_valid for that.

Variant b) involves generic mm code to fix it for arm64, this patch is
an arm64 change only. This makes it harder to get a fix for it.
(Though maybe only a problem of patch logistics.)

> 
> > Fixing this by restoring the old behavior of pfn_valid() to use
> > memblock_is_memory().
> 
> This is incorrect imo. In general, pfn_valid() means ordinary memory
> covered by the linear mapping and the struct page array. Returning
> reserved ranges that the kernel should not even touch only to please
> the NUMA code seems like an inappropriate way to deal with this issue.

As said above, it is not marked as reserved, it is treated like
non-existing memory.

This has been observed for non-numa kernels too and can happen for
each zone that is only partly initialized.

I think the patch addresses your concerns. I can't see there the
kernel uses memory marked as nomap in a wrong way.

Thanks,

-Robert

> 
> > Also changing users of pfn_valid() in arm64 code
> > to use memblock_is_map_memory() where necessary. This only affects
> > code in ioremap.c. The code in mmu.c still can use the new version of
> > pfn_valid().
> >
> > Should be marked stable v4.5..
> >
> > Signed-off-by: Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> > ---
> >  arch/arm64/mm/init.c    | 2 +-
> >  arch/arm64/mm/ioremap.c | 5 +++--
> >  2 files changed, 4 insertions(+), 3 deletions(-)

From mboxrd@z Thu Jan  1 00:00:00 1970
From: David Daney <ddaney.cavm@gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Mon, 10 Oct 2016 08:33:50 -0700
Message-ID: <57FBB4DE.2090502@gmail.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Return-path: <linux-kernel-owner@vger.kernel.org>
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Sender: linux-kernel-owner@vger.kernel.org
To: Robert Richter <rrichter@cavium.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, Mark Rutland <mark.rutland@arm.com>, linux-efi@vger.kernel.org, David Daney <david.daney@cavium.com>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, linux-kernel@vger.kernel.org, Hanjun Guo <hanjun.guo@linaro.org>, linux-arm-kernel@lists.infradead.org
List-Id: linux-efi@vger.kernel.org

On 10/06/2016 02:52 AM, Robert Richter wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
>
>   kernel BUG at mm/page_alloc.c:1848!
>
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
>
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
>
>   68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
>
> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
>
> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().
>
> Should be marked stable v4.5..

In that case you should add:

Cc: <stable@vger.kernel.org> # 4.5.x-

here.


>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
[...]

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Mon, 17 Oct 2016 20:58:01 +0200
Message-ID: <20161017185801.GT25086@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <20161006161114.GH22012-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>
Cc: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

Mark, Will, any opinion here?

Thanks,

-Robert

On 06.10.16 18:11:14, Robert Richter wrote:
> Ard,
> 
> thank you for your answer and you explanation.
> 
> On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > On 6 October 2016 at 10:52, Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> > > There is a memory setup problem on ThunderX systems with certain
> > > memory configurations. The symptom is
> > >
> > >  kernel BUG at mm/page_alloc.c:1848!
> > >
> > > This happens for some configs with 64k page size enabled. The bug
> > > triggers for page zones with some pages in the zone not assigned to
> > > this particular zone. In my case some pages that are marked as nomap
> > > were not reassigned to the new zone of node 1, so those are still
> > > assigned to node 0.
> > >
> > > The reason for the mis-configuration is a change in pfn_valid() which
> > > reports pages marked nomap as invalid:
> > >
> > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > >
> > 
> > These pages are owned by the firmware, which may map it with
> > attributes that conflict with the attributes we use for the linear
> > mapping. This means they should not be covered by the linear mapping.
> > 
> > > This causes pages marked as nomap being no long reassigned to the new
> > > zone in memmap_init_zone() by calling __init_single_pfn().
> > >
> > 
> > This sounds like the root cause of your issue. Could we not fix that instead?
> 
> Yes, this is proposal b) from my last mail that would work too: I
> implemented an arm64 private early_pfn_valid() function that uses
> memblock_is_memory() to setup all pages of a zone. Though, I think
> this is the wrong way and thus I prefer this patch instead. I see
> serveral reasons for this:
> 
> Inconsistent use of struct *page, it is initialized but never used
> again.
> 
> Other archs only do a basic range check in pfn_valid(), the default
> implementation just returns if the whole section is valid. As I
> understand the code, if the mem range is not aligned to the section,
> then there will be pfn's in the section that don't have physical mem
> attached. The page is then just initialized, it's not marked reserved
> nor the refcount is non-zero. It is then simply not used. This is how
> no-map pages should be handled too.
> 
> I think pfn_valid() is just a quick check if the pfn's struct *page
> can be used. There is a good description for this in include/linux/
> mmzone.h. So there can be memory holes that have a valid pfn.
> 
> If the no-map memory needs special handling, then additional checks
> need to be added to the particular code (as in ioremap.c). It's imo
> wrong to (mis-)use pfn_valid for that.
> 
> Variant b) involves generic mm code to fix it for arm64, this patch is
> an arm64 change only. This makes it harder to get a fix for it.
> (Though maybe only a problem of patch logistics.)
> 
> > 
> > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > memblock_is_memory().
> > 
> > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > covered by the linear mapping and the struct page array. Returning
> > reserved ranges that the kernel should not even touch only to please
> > the NUMA code seems like an inappropriate way to deal with this issue.
> 
> As said above, it is not marked as reserved, it is treated like
> non-existing memory.
> 
> This has been observed for non-numa kernels too and can happen for
> each zone that is only partly initialized.
> 
> I think the patch addresses your concerns. I can't see there the
> kernel uses memory marked as nomap in a wrong way.
> 
> Thanks,
> 
> -Robert
> 
> > 
> > > Also changing users of pfn_valid() in arm64 code
> > > to use memblock_is_map_memory() where necessary. This only affects
> > > code in ioremap.c. The code in mmu.c still can use the new version of
> > > pfn_valid().
> > >
> > > Should be marked stable v4.5..
> > >
> > > Signed-off-by: Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> > > ---
> > >  arch/arm64/mm/init.c    | 2 +-
> > >  arch/arm64/mm/ioremap.c | 5 +++--
> > >  2 files changed, 4 insertions(+), 3 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-efi" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Tue, 18 Oct 2016 11:18:36 +0100
Message-ID: <20161018101715.GA15639@leverpostej>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <20161006161114.GH22012-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

Hi Robert, Ard,

Sorry for the delay in getting to this; I've been travelling a lot
lately and in the meantime this managed to get buried in my inbox.

On Thu, Oct 06, 2016 at 06:11:14PM +0200, Robert Richter wrote:
> On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > On 6 October 2016 at 10:52, Robert Richter <rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> > > There is a memory setup problem on ThunderX systems with certain
> > > memory configurations. The symptom is
> > >
> > >  kernel BUG at mm/page_alloc.c:1848!
> > >
> > > This happens for some configs with 64k page size enabled. The bug
> > > triggers for page zones with some pages in the zone not assigned to
> > > this particular zone. In my case some pages that are marked as nomap
> > > were not reassigned to the new zone of node 1, so those are still
> > > assigned to node 0.
> > >
> > > The reason for the mis-configuration is a change in pfn_valid() which
> > > reports pages marked nomap as invalid:
> > >
> > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > 
> > These pages are owned by the firmware, which may map it with
> > attributes that conflict with the attributes we use for the linear
> > mapping. This means they should not be covered by the linear mapping.
> > 
> > > This causes pages marked as nomap being no long reassigned to the new
> > > zone in memmap_init_zone() by calling __init_single_pfn().

Why do we have pages for a nomap region? Given the region shouldn't be
in the linear mapping, and isn't suitable for general allocation, I
don't believe it makes sense to have a struct page for any part of it.

Am I missing some reason that we require a struct page?

e.g. is it just easier to allocate an unused struct page than to carve
it out?

> > This sounds like the root cause of your issue. Could we not fix that instead?
> 
> Yes, this is proposal b) from my last mail that would work too: I
> implemented an arm64 private early_pfn_valid() function that uses
> memblock_is_memory() to setup all pages of a zone. Though, I think
> this is the wrong way and thus I prefer this patch instead. I see
> serveral reasons for this:
> 
> Inconsistent use of struct *page, it is initialized but never used
> again.

As above, I don't believe we should have a struct page to initialise in
the first place.

> Other archs only do a basic range check in pfn_valid(), the default
> implementation just returns if the whole section is valid. As I
> understand the code, if the mem range is not aligned to the section,
> then there will be pfn's in the section that don't have physical mem
> attached. The page is then just initialized, it's not marked reserved
> nor the refcount is non-zero. It is then simply not used. This is how
> no-map pages should be handled too.
> 
> I think pfn_valid() is just a quick check if the pfn's struct *page
> can be used. There is a good description for this in include/linux/
> mmzone.h. So there can be memory holes that have a valid pfn.

I take it you mean the comment in the CONFIG_ARCH_HAS_HOLES_MEMORYMODEL
ifdef (line 1266 in v4.9-rc1)?

I'm not sufficiently acquainted with the memmap code to follow; I'll
need to dig into that a bit further.

> If the no-map memory needs special handling, then additional checks
> need to be added to the particular code (as in ioremap.c). It's imo
> wrong to (mis-)use pfn_valid for that.
> 
> Variant b) involves generic mm code to fix it for arm64, this patch is
> an arm64 change only. This makes it harder to get a fix for it.
> (Though maybe only a problem of patch logistics.)
> 
> > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > memblock_is_memory().
> > 
> > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > covered by the linear mapping and the struct page array. Returning
> > reserved ranges that the kernel should not even touch only to please
> > the NUMA code seems like an inappropriate way to deal with this issue.
> 
> As said above, it is not marked as reserved, it is treated like
> non-existing memory.

I think Ard was using "reserved" in the more general sense than the
Linux-specific meaning. NOMAP is distinct from the Linux concept of
"reserved" memory, but is "reserved" in some sense.

Memory with NOMAP is meant to be treated as non-existent for the purpose
of the linear mapping (and thus for the purpose of struct page).

> This has been observed for non-numa kernels too and can happen for
> each zone that is only partly initialized.
> 
> I think the patch addresses your concerns. I can't see there the
> kernel uses memory marked as nomap in a wrong way.

I'll have to dig into this locally; I'm still not familiar enough with
this code to know what the right thing to do is.

Thanks,
Mark.

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Tue, 18 Oct 2016 17:02:40 +0200
Message-ID: <20161018150240.GU22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161018101715.GA15639@leverpostej>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
Content-Disposition: inline
In-Reply-To: <20161018101715.GA15639@leverpostej>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Mark Rutland <mark.rutland@arm.com>
Cc: "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, David Daney <david.daney@cavium.com>, Catalin Marinas <catalin.marinas@arm.com>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Will Deacon <will.deacon@arm.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
List-Id: linux-efi@vger.kernel.org

Mark,

thanks for your answer. See below. I also attached the full crash
dump.

On 18.10.16 11:18:36, Mark Rutland wrote:
> Hi Robert, Ard,
> 
> Sorry for the delay in getting to this; I've been travelling a lot
> lately and in the meantime this managed to get buried in my inbox.
> 
> On Thu, Oct 06, 2016 at 06:11:14PM +0200, Robert Richter wrote:
> > On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > > On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > > > There is a memory setup problem on ThunderX systems with certain
> > > > memory configurations. The symptom is
> > > >
> > > >  kernel BUG at mm/page_alloc.c:1848!
> > > >
> > > > This happens for some configs with 64k page size enabled. The bug
> > > > triggers for page zones with some pages in the zone not assigned to
> > > > this particular zone. In my case some pages that are marked as nomap
> > > > were not reassigned to the new zone of node 1, so those are still
> > > > assigned to node 0.
> > > >
> > > > The reason for the mis-configuration is a change in pfn_valid() which
> > > > reports pages marked nomap as invalid:
> > > >
> > > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > > 
> > > These pages are owned by the firmware, which may map it with
> > > attributes that conflict with the attributes we use for the linear
> > > mapping. This means they should not be covered by the linear mapping.
> > > 
> > > > This causes pages marked as nomap being no long reassigned to the new
> > > > zone in memmap_init_zone() by calling __init_single_pfn().
> 
> Why do we have pages for a nomap region? Given the region shouldn't be
> in the linear mapping, and isn't suitable for general allocation, I
> don't believe it makes sense to have a struct page for any part of it.
> 
> Am I missing some reason that we require a struct page?
> 
> e.g. is it just easier to allocate an unused struct page than to carve
> it out?

Pages are handled in blocks with size MAX_ORDER_NR_PAGES. The start
and end pfn of a memory region is aligned then to fit the mem block
size (see memmap_init_zone()). Therefore a memblock may contain pages
without underlying physical memory.

mm code requires the whole memmap to be initialized, this means that
for each page in the whole mem block there is a valid struct page. See
e.g. move_freepages_block() and move_freepages(), stuct page is
accessed even before pfn_valid() is used. I assume there are other
occurrences of that too.

My interpretation is that pfn_valid() checks for the existence of a
valid struct page, there must not necessarily phys memory mapped to
it. This is the reason why I changed pfn_valid() to use
memblock_is_memory() which is sufficient for generic mm code. Only in
arm64 mm code I additinally added the memblock_is_map_memory() check
where pfn_valid() was used.

> 
> > > This sounds like the root cause of your issue. Could we not fix that instead?
> > 
> > Yes, this is proposal b) from my last mail that would work too: I
> > implemented an arm64 private early_pfn_valid() function that uses
> > memblock_is_memory() to setup all pages of a zone. Though, I think
> > this is the wrong way and thus I prefer this patch instead. I see
> > serveral reasons for this:
> > 
> > Inconsistent use of struct *page, it is initialized but never used
> > again.
> 
> As above, I don't believe we should have a struct page to initialise in
> the first place.
> 
> > Other archs only do a basic range check in pfn_valid(), the default
> > implementation just returns if the whole section is valid. As I
> > understand the code, if the mem range is not aligned to the section,
> > then there will be pfn's in the section that don't have physical mem
> > attached. The page is then just initialized, it's not marked reserved
> > nor the refcount is non-zero. It is then simply not used. This is how
> > no-map pages should be handled too.
> > 
> > I think pfn_valid() is just a quick check if the pfn's struct *page
> > can be used. There is a good description for this in include/linux/
> > mmzone.h. So there can be memory holes that have a valid pfn.
> 
> I take it you mean the comment in the CONFIG_ARCH_HAS_HOLES_MEMORYMODEL
> ifdef (line 1266 in v4.9-rc1)?

Yes.

> 
> I'm not sufficiently acquainted with the memmap code to follow; I'll
> need to dig into that a bit further.
> 
> > If the no-map memory needs special handling, then additional checks
> > need to be added to the particular code (as in ioremap.c). It's imo
> > wrong to (mis-)use pfn_valid for that.
> > 
> > Variant b) involves generic mm code to fix it for arm64, this patch is
> > an arm64 change only. This makes it harder to get a fix for it.
> > (Though maybe only a problem of patch logistics.)
> > 
> > > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > > memblock_is_memory().
> > > 
> > > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > > covered by the linear mapping and the struct page array. Returning
> > > reserved ranges that the kernel should not even touch only to please
> > > the NUMA code seems like an inappropriate way to deal with this issue.
> > 
> > As said above, it is not marked as reserved, it is treated like
> > non-existing memory.
> 
> I think Ard was using "reserved" in the more general sense than the
> Linux-specific meaning. NOMAP is distinct from the Linux concept of
> "reserved" memory, but is "reserved" in some sense.
> 
> Memory with NOMAP is meant to be treated as non-existent for the purpose
> of the linear mapping (and thus for the purpose of struct page).

Yes, it's not marked reserved and can never be freed.

Thanks,

-Robert

> 
> > This has been observed for non-numa kernels too and can happen for
> > each zone that is only partly initialized.
> > 
> > I think the patch addresses your concerns. I can't see there the
> > kernel uses memory marked as nomap in a wrong way.
> 
> I'll have to dig into this locally; I'm still not familiar enough with
> this code to know what the right thing to do is.
> 
> Thanks,
> Mark.

EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
Booting Linux on physical CPU 0x0
Linux version 4.8.0-rc4-00267-g664e88c (root@crb2spass2rric.localdomain) (gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.1) ) #3 SMP Wed Sep 7 06:38:13 UTC 2016
Boot CPU: AArch64 Processor [431f0a10]
efi: Getting EFI parameters from FDT:
efi: EFI v2.40 by Cavium Thunder cn88xx EFI ThunderX-Firmware-Release-1.22.10-0-g4e85766 Aug 24 2016 15:59:03
efi:  ACPI=0xfffff000  ACPI 2.0=0xfffff014  SMBIOS 3.0=0x10ffafcf000 
cma: Reserved 512 MiB at 0x00000000c0000000
NUMA: Adding memblock [0x1400000 - 0xfffffffff] on node 0
NUMA: Adding memblock [0x10000400000 - 0x10fffffffff] on node 1
NUMA: parsing numa-distance-map-v1
NUMA: Initmem setup node 0 [mem 0x01400000-0xfffffffff]
NUMA: NODE_DATA [mem 0xfffff2580-0xfffffffff]
NUMA: Initmem setup node 1 [mem 0x10000400000-0x10fffffffff]
NUMA: NODE_DATA [mem 0x10ffffa2500-0x10ffffaff7f]
kmemleak: Kernel memory leak detector disabled
Zone ranges:
  DMA      [mem 0x0000000001400000-0x00000000ffffffff]
  Normal   [mem 0x0000000100000000-0x0000010fffffffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000001400000-0x00000000fffdffff]
  node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
  node   0: [mem 0x0000000100000000-0x0000000fffffffff]
  node   1: [mem 0x0000010000400000-0x0000010ff9e8ffff]
  node   1: [mem 0x0000010ff9e90000-0x0000010ff9f2ffff]
  node   1: [mem 0x0000010ff9f30000-0x0000010ffaeaffff]
  node   1: [mem 0x0000010ffaeb0000-0x0000010ffaffffff]
  node   1: [mem 0x0000010ffb000000-0x0000010ffffaffff]
  node   1: [mem 0x0000010ffffb0000-0x0000010fffffffff]
Initmem setup node 0 [mem 0x0000000001400000-0x0000000fffffffff]
Initmem setup node 1 [mem 0x0000010000400000-0x0000010fffffffff]
psci: probing for conduit method from DT.
psci: PSCIv0.2 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: Trusted OS resident on physical CPU 0x0
Number of cores (96) exceeds configured maximum of 8 - clipping
percpu: Embedded 3 pages/cpu @ffffff0fffce0000 s117704 r8192 d70712 u196608
Detected VIPT I-cache on CPU0
CPU features: enabling workaround for Cavium erratum 27456
Built 2 zonelists in Node order, mobility grouping on.  Total pages: 2094720
Policy zone: Normal
Kernel command line: BOOT_IMAGE=/vmlinuz root=UUID=9a418ae5-231a-40c7-b7ba-ca70cfadfc5e ro LANG=en_US.UTF-8
PID hash table entries: 4096 (order: -1, 32768 bytes)
software IO TLB [mem 0xfbfd0000-0xfffd0000] (64MB) mapped at [fffffe00fbfd0000-fffffe00fffcffff]
Memory: 133196160K/134193152K available (9084K kernel code, 1545K rwdata, 3712K rodata, 1536K init, 15826K bss, 472704K reserved, 524288K cma-reserved)
Virtual kernel memory layout:
    modules : 0xfffffc0000000000 - 0xfffffc0008000000   (   128 MB)
    vmalloc : 0xfffffc0008000000 - 0xfffffdff5fff0000   (  2045 GB)
      .text : 0xfffffc0008080000 - 0xfffffc0008960000   (  9088 KB)
    .rodata : 0xfffffc0008960000 - 0xfffffc0008d10000   (  3776 KB)
      .init : 0xfffffc0008d10000 - 0xfffffc0008e90000   (  1536 KB)
      .data : 0xfffffc0008e90000 - 0xfffffc0009012600   (  1546 KB)
       .bss : 0xfffffc0009012600 - 0xfffffc0009f86fc8   ( 15827 KB)
    fixed   : 0xfffffdff7e7d0000 - 0xfffffdff7ec00000   (  4288 KB)
    PCI I/O : 0xfffffdff7ee00000 - 0xfffffdff7fe00000   (    16 MB)
    vmemmap : 0xfffffdff80000000 - 0xfffffe0000000000   (     2 GB maximum)
              0xfffffdff80005000 - 0xfffffdffc4000000   (  1087 MB actual)
    memory  : 0xfffffe0001400000 - 0xffffff1000000000   (1114092 MB)
SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=8, Nodes=2
Running RCU self tests
Hierarchical RCU implementation.
	RCU lockdep checking is enabled.
	Build-time adjustment of leaf fanout to 64.
NR_IRQS:64 nr_irqs:64 0
GICv3: GIC: Using split EOI/Deactivate mode
ITS: /interrupt-controller@801000000000/gic-its@801000020000
ITS@0x0000801000020000: allocated 2097152 Devices @ff5000000 (flat, esz 8, psz 64K, shr 1)
ITS: /interrupt-controller@801000000000/gic-its@901000020000
ITS@0x0000901000020000: allocated 2097152 Devices @ffa000000 (flat, esz 8, psz 64K, shr 1)
GIC: using LPI property table @0x0000000ff4140000
ITS: Allocated 32512 chunks for LPIs
GICv3: CPU0: found redistributor 0 region 0:0x0000801080000000
CPU0: using LPI pending table @0x0000000ff4150000
arm_arch_timer: Architected cp15 timer(s) running at 100.00MHz (phys).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x171024e7e0, max_idle_ns: 440795205315 ns
sched_clock: 56 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns
Console: colour dummy device 80x25
console [tty0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:  8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:          4096
... MAX_LOCKDEP_ENTRIES:     32768
... MAX_LOCKDEP_CHAINS:      65536
... CHAINHASH_SIZE:          32768
 memory used by lock dependency info: 8159 kB
 per task-struct memory footprint: 1920 bytes
mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=100000)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Yama: becoming mindful.
SELinux:  Initializing.
Dentry cache hash table entries: 16777216 (order: 11, 134217728 bytes)
Inode-cache hash table entries: 8388608 (order: 10, 67108864 bytes)
Mount-cache hash table entries: 262144 (order: 5, 2097152 bytes)
Mountpoint-cache hash table entries: 262144 (order: 5, 2097152 bytes)
ftrace: allocating 29244 entries in 8 pages
Unable to find CPU node for /cpus/cpu@8
/cpus/cpu-map/cluster0/core8: Can't get CPU for leaf core
ASID allocator initialised with 65536 entries
PCI/MSI: /interrupt-controller@801000000000/gic-its@801000020000 domain created
PCI/MSI: /interrupt-controller@801000000000/gic-its@901000020000 domain created
Platform MSI: /interrupt-controller@801000000000/gic-its@801000020000 domain created
Platform MSI: /interrupt-controller@801000000000/gic-its@901000020000 domain created
Remapping and enabling EFI services.
UEFI Runtime regions are not aligned to 64 KB -- buggy firmware?
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at arch/arm64/kernel/efi.c:34 efi_create_mapping+0x5c/0x11c
Modules linked in:

CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.8.0-rc4-00267-g664e88c #3
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fffffe0fee55a200 task.stack: ffffff00004a4000
PC is at efi_create_mapping+0x5c/0x11c
LR is at efi_create_mapping+0x5c/0x11c
pc : [<fffffc0008d14f68>] lr : [<fffffc0008d14f68>] pstate: 60000045
sp : ffffff00004a7d00
x29: ffffff00004a7d00 x28: 0000000000000000 
x27: 0000000000000000 x26: 0000000000000000 
x25: 0000000000000000 x24: fffffc0008f9c730 
x23: fffffc0008f9c730 x22: fffffc0008c60ac0 
x21: fffffc0008f9c730 x20: 00c8000000000713 
x19: ffffff0ff76b0c08 x18: 0000000000000010 
x17: 00000000c9cbbfc7 x16: 0000000000000000 
x15: fffffc0089c956f7 x14: 3f657261776d7269 
x13: 6620796767756220 x12: 2d2d20424b203436 
x11: 206f742064656e67 x10: 0000000000000077 
x9 : 65726120736e6f69 x8 : 0000000000000001 
x7 : ffffff00004a4000 x6 : fffffc000814a288 
x5 : 0000000000000000 x4 : 0000000000000001 
x3 : 0000000000000001 x2 : ffffff00004a4000 
x1 : fffffe0fee55a200 x0 : 0000000000000040 

---[ end trace fcdd7f8cb96cdb3b ]---
Call trace:
Exception stack(0xffffff00004a7b20 to 0xffffff00004a7c50)
7b20: ffffff0ff76b0c08 0000040000000000 ffffff00004a7d00 fffffc0008d14f68
7b40: 0000000060000045 000000000000003d 0000000000000001 fffffc000814a9fc
7b60: ffffff00004a7c00 fffffc000814adcc ffffff00004a7c60 fffffc0008baf420
7b80: fffffc0008f9c730 fffffc0008c60ac0 fffffc0008f9c730 fffffc0008f9c730
7ba0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7bc0: fffffc00080d6d70 0000000000000000 0000000000000040 fffffe0fee55a200
7be0: ffffff00004a4000 0000000000000001 0000000000000001 0000000000000000
7c00: fffffc000814a288 ffffff00004a4000 0000000000000001 65726120736e6f69
7c20: 0000000000000077 206f742064656e67 2d2d20424b203436 6620796767756220
7c40: 3f657261776d7269 fffffc0089c956f7
[<fffffc0008d14f68>] efi_create_mapping+0x5c/0x11c
[<fffffc0008d5e1ec>] arm_enable_runtime_services+0x12c/0x210
[<fffffc0008082ff4>] do_one_initcall+0x44/0x138
[<fffffc0008d10d30>] kernel_init_freeable+0x17c/0x2e0
[<fffffc000893c888>] kernel_init+0x20/0xf8
[<fffffc0008082b80>] ret_from_fork+0x10/0x50
  EFI remap 0x0000010ff9e98000 => 0000000020008000
  EFI remap 0x0000010ffaeb6000 => 00000000200a6000
  EFI remap 0x0000010ffafc9000 => 00000000201b9000
  EFI remap 0x0000010ffafcd000 => 00000000201bd000
  EFI remap 0x0000010ffffb9000 => 00000000201f9000
  EFI remap 0x0000010ffffcd000 => 000000002020d000
  EFI remap 0x0000804000001000 => 0000000020241000
  EFI remap 0x000087e0d0001000 => 0000000020251000
Detected VIPT I-cache on CPU1
GICv3: CPU1: found redistributor 1 region 0:0x0000801080020000
CPU1: using LPI pending table @0x0000010014060000
CPU1: Booted secondary processor [431f0a10]
------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at ./include/linux/cpumask.h:121 gic_raise_softirq+0x14c/0x1e0
Modules linked in:

CPU: 1 PID: 0 Comm: swapper/1 Tainted: G        W       4.8.0-rc4-00267-g664e88c #3
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fffffe0fee57b600 task.stack: fffffe0fe4038000
PC is at gic_raise_softirq+0x14c/0x1e0
LR is at gic_raise_softirq+0xc0/0x1e0
pc : [<fffffc00084c07a4>] lr : [<fffffc00084c0718>] pstate: 600001c5
sp : fffffe0fe403bcd0
x29: fffffe0fe403bcd0 x28: 0000000000000001 
x27: fffffc0008fd7b1d x26: 0000000000000000 
x25: fffffc0008ec2000 x24: fffffc0008ecebc8 
x23: fffffc0008998688 x22: 0000000000000001 
x21: fffffc0008ec2e34 x20: 0000000000000000 
x19: 0000000000000008 x18: 0000000000000030 
x17: 0000000000000000 x16: 0000000000000000 
x15: fffffc0009c97f08 x14: 0000000000000002 
x13: 0000000000000332 x12: 0000000000000335 
x11: fffffc0009bf3de0 x10: 0000000000000332 
x9 : fffffe0fe4038000 x8 : 0000000000000000 
x7 : 00000000ffffffff x6 : 0000000000000000 
x5 : fffffffffffffffe x4 : 0000000000000040 
x3 : 0000000000000000 x2 : 0000000000000000 
x1 : 0000000000000008 x0 : 0000000000000001 

---[ end trace fcdd7f8cb96cdb3c ]---
Call trace:
Exception stack(0xfffffe0fe403baf0 to 0xfffffe0fe403bc20)
bae0:                                   0000000000000008 0000040000000000
bb00: fffffe0fe403bcd0 fffffc00084c07a4 00000000600001c5 000000000000003d
bb20: fffffe0fee57bec0 ffffffffffffffd8 0000000000000028 fffffe0fee57be70
bb40: fffffc000925fe88 0000000000000002 fffffc00080886c0 fffffe0fee57b600
bb60: fffffe0fe403bb90 fffffc0008088974 fffffc00099b4d60 fffffe0fee57b600
bb80: fffffc000925fe88 fffffe0fe403bd4c fffffe0fe403bbe0 fffffc00080889e8
bba0: 0000000000000001 0000000000000008 0000000000000000 0000000000000000
bbc0: 0000000000000040 fffffffffffffffe 0000000000000000 00000000ffffffff
bbe0: 0000000000000000 fffffe0fe4038000 0000000000000332 fffffc0009bf3de0
bc00: 0000000000000335 0000000000000332 0000000000000002 fffffc0009c97f08
[<fffffc00084c07a4>] gic_raise_softirq+0x14c/0x1e0
[<fffffc000808e028>] smp_cross_call+0x80/0x238
[<fffffc000808efa4>] smp_send_reschedule+0x3c/0x48
[<fffffc00081021a8>] resched_curr+0x58/0x90
[<fffffc0008103fd8>] check_preempt_curr+0x70/0xe8
[<fffffc0008104090>] ttwu_do_wakeup+0x40/0x2e8
[<fffffc00081043cc>] ttwu_do_activate+0x94/0xc0
[<fffffc00081058a8>] try_to_wake_up+0x200/0x580
[<fffffc0008105d3c>] default_wake_function+0x34/0x48
[<fffffc0008126afc>] __wake_up_common+0x64/0xa8
[<fffffc0008126bec>] __wake_up_locked+0x3c/0x50
[<fffffc0008127b58>] complete+0x48/0x68
[<fffffc000808e630>] secondary_start_kernel+0x180/0x208
[<0000000001482e08>] 0x1482e08
Detected VIPT I-cache on CPU2
GICv3: CPU2: found redistributor 2 region 0:0x0000801080040000
CPU2: using LPI pending table @0x0000010014070000
CPU2: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU3
GICv3: CPU3: found redistributor 3 region 0:0x0000801080060000
CPU3: using LPI pending table @0x0000010014080000
CPU3: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU4
GICv3: CPU4: found redistributor 4 region 0:0x0000801080080000
CPU4: using LPI pending table @0x0000010014090000
CPU4: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU5
GICv3: CPU5: found redistributor 5 region 0:0x00008010800a0000
CPU5: using LPI pending table @0x00000100140a0000
CPU5: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU6
GICv3: CPU6: found redistributor 6 region 0:0x00008010800c0000
CPU6: using LPI pending table @0x00000100140b0000
CPU6: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU7
GICv3: CPU7: found redistributor 7 region 0:0x00008010800e0000
CPU7: using LPI pending table @0x00000100140c0000
CPU7: Booted secondary processor [431f0a10]
Brought up 8 CPUs
SMP: Total of 8 processors activated.
CPU features: detected feature: GIC system register CPU interface
CPU features: detected feature: LSE atomic instructions
CPU features: detected feature: Software prefetching using PRFM
CPU: All CPU(s) started at EL2
alternatives: patching kernel code
devtmpfs: initialized
SMBIOS 3.0.0 present.
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
atomic64_test: passed
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
cpuidle: using governor menu
vdso: 2 pages (1 code @ fffffc0008980000, 1 data @ fffffc0008eb0000)
hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
DMA: preallocated 256 KiB pool for atomic allocations
Serial: AMBA PL011 UART driver
87e024000000.serial: ttyAMA0 at MMIO 0x87e024000000 (irq = 7, base_baud = 0) is a PL011 rev3
console [ttyAMA0] enabled
87e025000000.serial: ttyAMA1 at MMIO 0x87e025000000 (irq = 8, base_baud = 0) is a PL011 rev3
OF: /soc@0/pci@848000000000: arguments longer than property
OF: /soc@0/pci@849000000000: arguments longer than property
OF: /soc@0/pci@84a000000000: arguments longer than property
OF: /soc@0/pci@84b000000000: arguments longer than property
OF: /soc@0/pci@87e0c0000000: arguments longer than property
OF: /soc@100000000000/pci@848000000000: arguments longer than property
OF: /soc@100000000000/pci@849000000000: arguments longer than property
OF: /soc@100000000000/pci@84a000000000: arguments longer than property
OF: /soc@100000000000/pci@84b000000000: arguments longer than property
HugeTLB registered 2 MB page size, pre-allocated 0 pages
HugeTLB registered 512 MB page size, pre-allocated 0 pages
ACPI: Interpreter disabled.
arm-smmu 830000000000.smmu0: probing hardware configuration...
arm-smmu 830000000000.smmu0: SMMUv2 with:
arm-smmu 830000000000.smmu0: 	stage 1 translation
arm-smmu 830000000000.smmu0: 	stage 2 translation
arm-smmu 830000000000.smmu0: 	nested translation
arm-smmu 830000000000.smmu0: 	non-coherent table walk
arm-smmu 830000000000.smmu0: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 830000000000.smmu0: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 830000000000.smmu0: 	128 context banks (0 stage-2 only)
arm-smmu 830000000000.smmu0: 	Supported page sizes: 0x62215000
arm-smmu 830000000000.smmu0: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 830000000000.smmu0: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 830000000000.smmu0: registered 1 master devices
arm-smmu 831000000000.smmu1: probing hardware configuration...
arm-smmu 831000000000.smmu1: SMMUv2 with:
arm-smmu 831000000000.smmu1: 	stage 1 translation
arm-smmu 831000000000.smmu1: 	stage 2 translation
arm-smmu 831000000000.smmu1: 	nested translation
arm-smmu 831000000000.smmu1: 	non-coherent table walk
arm-smmu 831000000000.smmu1: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 831000000000.smmu1: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 831000000000.smmu1: 	128 context banks (0 stage-2 only)
arm-smmu 831000000000.smmu1: 	Supported page sizes: 0x62215000
arm-smmu 831000000000.smmu1: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 831000000000.smmu1: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 831000000000.smmu1: registered 2 master devices
arm-smmu 832000000000.smmu2: probing hardware configuration...
arm-smmu 832000000000.smmu2: SMMUv2 with:
arm-smmu 832000000000.smmu2: 	stage 1 translation
arm-smmu 832000000000.smmu2: 	stage 2 translation
arm-smmu 832000000000.smmu2: 	nested translation
arm-smmu 832000000000.smmu2: 	non-coherent table walk
arm-smmu 832000000000.smmu2: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 832000000000.smmu2: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 832000000000.smmu2: 	128 context banks (0 stage-2 only)
arm-smmu 832000000000.smmu2: 	Supported page sizes: 0x62215000
arm-smmu 832000000000.smmu2: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 832000000000.smmu2: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 832000000000.smmu2: registered 1 master devices
arm-smmu 833000000000.smmu3: probing hardware configuration...
arm-smmu 833000000000.smmu3: SMMUv2 with:
arm-smmu 833000000000.smmu3: 	stage 1 translation
arm-smmu 833000000000.smmu3: 	stage 2 translation
arm-smmu 833000000000.smmu3: 	nested translation
arm-smmu 833000000000.smmu3: 	non-coherent table walk
arm-smmu 833000000000.smmu3: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 833000000000.smmu3: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 833000000000.smmu3: 	128 context banks (0 stage-2 only)
arm-smmu 833000000000.smmu3: 	Supported page sizes: 0x62215000
arm-smmu 833000000000.smmu3: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 833000000000.smmu3: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 833000000000.smmu3: registered 1 master devices
arm-smmu 930000000000.smmu4: probing hardware configuration...
arm-smmu 930000000000.smmu4: SMMUv2 with:
arm-smmu 930000000000.smmu4: 	stage 1 translation
arm-smmu 930000000000.smmu4: 	stage 2 translation
arm-smmu 930000000000.smmu4: 	nested translation
arm-smmu 930000000000.smmu4: 	non-coherent table walk
arm-smmu 930000000000.smmu4: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 930000000000.smmu4: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 930000000000.smmu4: 	128 context banks (0 stage-2 only)
arm-smmu 930000000000.smmu4: 	Supported page sizes: 0x62215000
arm-smmu 930000000000.smmu4: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 930000000000.smmu4: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 930000000000.smmu4: registered 1 master devices
arm-smmu 931000000000.smmu5: probing hardware configuration...
arm-smmu 931000000000.smmu5: SMMUv2 with:
arm-smmu 931000000000.smmu5: 	stage 1 translation
arm-smmu 931000000000.smmu5: 	stage 2 translation
arm-smmu 931000000000.smmu5: 	nested translation
arm-smmu 931000000000.smmu5: 	non-coherent table walk
arm-smmu 931000000000.smmu5: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 931000000000.smmu5: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 931000000000.smmu5: 	128 context banks (0 stage-2 only)
arm-smmu 931000000000.smmu5: 	Supported page sizes: 0x62215000
arm-smmu 931000000000.smmu5: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 931000000000.smmu5: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 931000000000.smmu5: registered 1 master devices
arm-smmu 932000000000.smmu6: probing hardware configuration...
arm-smmu 932000000000.smmu6: SMMUv2 with:
arm-smmu 932000000000.smmu6: 	stage 1 translation
arm-smmu 932000000000.smmu6: 	stage 2 translation
arm-smmu 932000000000.smmu6: 	nested translation
arm-smmu 932000000000.smmu6: 	non-coherent table walk
arm-smmu 932000000000.smmu6: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 932000000000.smmu6: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 932000000000.smmu6: 	128 context banks (0 stage-2 only)
arm-smmu 932000000000.smmu6: 	Supported page sizes: 0x62215000
arm-smmu 932000000000.smmu6: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 932000000000.smmu6: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 932000000000.smmu6: registered 1 master devices
arm-smmu 933000000000.smmu7: probing hardware configuration...
arm-smmu 933000000000.smmu7: SMMUv2 with:
arm-smmu 933000000000.smmu7: 	stage 1 translation
arm-smmu 933000000000.smmu7: 	stage 2 translation
arm-smmu 933000000000.smmu7: 	nested translation
arm-smmu 933000000000.smmu7: 	non-coherent table walk
arm-smmu 933000000000.smmu7: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 933000000000.smmu7: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 933000000000.smmu7: 	128 context banks (0 stage-2 only)
arm-smmu 933000000000.smmu7: 	Supported page sizes: 0x62215000
arm-smmu 933000000000.smmu7: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 933000000000.smmu7: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 933000000000.smmu7: registered 1 master devices
iommu: Adding device 848000000000.pci to group 0
iommu: Adding device 849000000000.pci to group 1
iommu: Adding device 84a000000000.pci to group 2
iommu: Adding device 84b000000000.pci to group 3
iommu: Adding device 88001f000000.pci to group 4
iommu: Adding device 948000000000.pci to group 5
iommu: Adding device 949000000000.pci to group 6
iommu: Adding device 94a000000000.pci to group 7
iommu: Adding device 94b000000000.pci to group 8
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
DMA-API: preallocated 4096 debug entries
DMA-API: debugging enabled by kernel config
clocksource: Switched to clocksource arch_sys_counter
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 6, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 6, 4194304 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
UDP hash table entries: 65536 (order: 7, 10485760 bytes)
UDP-Lite hash table entries: 65536 (order: 7, 10485760 bytes)
NET: Registered protocol family 1
Unpacking initramfs...
------------[ cut here ]------------
kernel BUG at mm/page_alloc.c:1844!
Internal error: Oops - BUG: 0 [#1] SMP
Modules linked in:
CPU: 4 PID: 1 Comm: swapper/0 Tainted: G        W       4.8.0-rc4-00267-g664e88c #3
Hardware name: www.cavium.com ThunderX CRB-2S/ThunderX CRB-2S, BIOS 0.3 Aug 24 2016
task: fffffe0fee55a200 task.stack: ffffff00004a4000
PC is at move_freepages+0x158/0x168
LR is at move_freepages_block+0xac/0xc0
pc : [<fffffc0008227618>] lr : [<fffffc00082276d4>] pstate: 800000c5
sp : ffffff00004a7480
x29: ffffff00004a7480 x28: fffffdffc3fc0020 
x27: fffffdffc3fc0000 x26: 000000000000000a 
x25: 000000000000000a x24: ffffff0ffffa3190 
x23: ffffff0fffdbf530 x22: 0000000000000000 
x21: fffffdffc3ffffc0 x20: ffffff0ffffa2c80 
x19: fffffdffc3f80000 x18: fffffe0fdc0c2548 
x17: 0000000000000000 x16: 0000000100000000 
x15: 00000000000000c1 x14: 04000000001b2000 
x13: 00aedf01c7040000 x12: 00001b00000033f5 
x11: 01c604000000001b x10: 0000ae7401c50430 
x9 : 0000000000000000 x8 : fffffc0008f53000 
x7 : fffffc00082295f0 x6 : 0000000000000001 
x5 : 0000000000000000 x4 : fffffe0fffff2580 
x3 : ffffff0ffffa2500 x2 : 0000000000000001 
x1 : fffffe0fffff2580 x0 : ffffff0ffffa2c80 

Process swapper/0 (pid: 1, stack limit = 0xffffff00004a4020)
Stack: (0xffffff00004a7480 to 0xffffff00004a8000)
7480: ffffff00004a74e0 fffffc00082276d4 0000000000000000 ffffff0ffffa2c80
74a0: fffffdffc3f80000 0000000000000000 ffffff0fffdbf530 ffffff0ffffa3190
74c0: 000000000000000a 000000000000000a fffffdffc3fc0000 fffffdffc3fc0020
74e0: ffffff00004a7510 fffffc0008227f00 ffffff0ffffa2c80 0000000000000000
7500: fffffdffc3fc0000 0000000000000028 ffffff00004a75b0 fffffc0008229654
7520: ffffff0fffdbf530 ffffff0ffffa2c80 fffffc0008e5f520 0000000000000000
7540: ffffff0fffdbf530 ffffff0fffdbf520 0000030ff6f60000 0000000000000001
7560: 0000000000000000 0000000000000000 ffffff0ffffa3200 fffffc00082295f0
7580: fffffc0008e5f520 fffffc0008ec1000 0000000000000000 0000000000000001
75a0: 0000000000000010 01ffff0ffffa2c80 ffffff00004a76a0 fffffc000822a5ec
75c0: 00000000024200c2 0000000000000000 0000000000000000 00000000024200c2
75e0: fffffc0008ec4000 0000000000000001 0000000000000000 fffffc0008ec2000
7600: 00000000024200c2 fffffc00082886c8 ffffffffffffffff 0000000000000040
7620: 00000012004a7640 fffffc0008e587e8 0000000100000000 0000000000000000
7640: ffffff00024200c2 fffffc00081007bc ffffff0ffffa2c80 0000000000000010
7660: fffffc0008e5f520 0000000000000040 0000000000000000 ffffff0ffffa3200
7680: 0000000000000001 ffffff0ffffa3b80 ffffff0ffffa3b80 ffffff00004a77b8
76a0: ffffff00004a77e0 fffffc00082886c8 0000000000000000 fffffc0008ec0c48
76c0: ffffff0ffffa3b80 0000000000000000 0000000000000000 0000000000000001
76e0: ffffff00004a4000 0000000000000000 fffffe0fee55a200 00000000000053d4
7700: fffffe0fee55a200 0000000000000000 0000000000000000 fffffc0008eec160
7720: 0000000000000000 0000000000000002 fffffe0fee55aa98 fffffc0008ec2000
7740: fffffe0fee55a200 fffffc0009c95000 0000000000000000 fffffc0008eec160
7760: 0000000000000000 fffffc0008e587e8 0000000000000000 0000000100400000
7780: 0000000000000000 0000000000000000 0000000000000000 0000000000000002
77a0: 0000000000000000 0000000000000000 fffffc000821d070 ffffff0ffffa3b80
77c0: 0000000000000000 ffffff0ffffa3b80 0000000100000000 0000000000000000
77e0: ffffff00004a7810 fffffc0008288d20 fffffc0008e587e8 fffffe0fec01a8f0
7800: 00000000024200c2 fffffc000821e62c ffffff00004a7870 fffffc000821e62c
7820: 0000000000000000 ffffff00004a4000 00000000024200c2 fffffe0fe0249410
7840: fffffc000821e788 00000000024200c2 fffffc0008f01d28 0000000002493ee0
7860: 0000000000002c2c 0000000000000040 ffffff00004a78d0 fffffc000821e788
7880: 0000000000000000 000000000000000e 00000000024200c2 fffffe0fe0249410
78a0: 0000000000000001 fffffc0008fd76ac fffffc0008f01d28 0000000002493ee0
78c0: 0000000000002c2c fffffc000821e850 ffffff00004a7930 fffffc000821e9b0
78e0: fffffe0fe0249410 0000000000000001 0000000000000001 0000000000002c2c
7900: ffffff00004a7a20 fffffe0fcc140400 00000000000053d4 fffffc00089b8900
7920: 0000000000002c2c fffffc000821e440 ffffff00004a7960 fffffc00082eb758
7940: fffffe0fe0249410 0000000000000001 0000000000010000 ffffff00004a7b18
7960: ffffff00004a79a0 fffffc000821e3bc 0000000000010000 ffffff00004a7b18
7980: fffffe0fe0249410 0000000000010000 0000000000000000 fffffc000821e434
79a0: ffffff00004a7a30 fffffc00082200e0 0000000000000000 fffffe0fcc140400
79c0: fffffe0fe0249210 ffffff00004a7af0 fffffe0fe0249410 ffffff00004a7b18
79e0: fffffc0008d40e70 0000000000008000 fffffc0009040078 ffffff0f83f26000
7a00: 0000000000000000 fffffc0008bd0300 00002c2c00000001 ffffff00004a4000
7a20: fffffdff83f20840 0000000027e02140 ffffff00004a7a80 fffffc00082201fc
7a40: 0000000000008000 ffffff00004a7af0 fffffe0fe02492e8 ffffff00004a7b18
7a60: ffffff00004a7bb8 0000000000000007 fffffc0008d40e70 0000000000008000
7a80: ffffff00004a7ab0 fffffc00082ba89c fffffe0fcc140400 0000000000008000
7aa0: ffffff00004a7bb8 fffffe0fdd350000 ffffff00004a7b40 fffffc00082bb588
7ac0: 0000000000000000 0000000000008000 fffffe0fcc140400 fffffe0fdd350000
7ae0: fffffe0fdd350000 0000000000008000 fffffe0fcc140400 000000000000ac2c
7b00: 0000000000000000 0000000000000000 0000000000000000 fffffc0000000003
7b20: 00000000000053d4 0000000000002c2c ffffff00004a7ae0 0000000000000001
7b40: ffffff00004a7b80 fffffc00082bc5b4 fffffe0fcc140400 fffffe0fcc140400
7b60: fffffe0fdd350000 0000000000008000 fffffc0008babe88 ffffff0f8ea0e439
7b80: ffffff00004a7bc0 fffffc0008d120f0 0000000000000000 0000000000008000
7ba0: fffffe0fdd350000 0000000000000000 fffffe0fdc3fa280 000000000000ac2c
7bc0: ffffff00004a7bf0 fffffc0008d121d4 fffffc0008d75000 fffffc0008d75b60
7be0: fffffc0008d75bb8 fffffe0fdd350000 ffffff00004a7c10 fffffc0008d11eac
7c00: fffffc0008d75b60 0000000000008000 ffffff00004a7c40 fffffc0008d11f10
7c20: 0000000000008000 0000000000008000 fffffc0008d75b60 fffffc00089e42c0
7c40: ffffff00004a7c80 fffffc0008d410f4 fffffe0fdc3fa280 ffffff0f83f26000
7c60: 0000000000000000 fffffe0fdd350000 fffffc0008d11d74 fffffc0008d11ec8
7c80: ffffff00004a7cf0 fffffc0008d411b4 fffffc0008d75000 ffffff0f83f26000
7ca0: 000000000e7f20bc 0000000000000000 fffffc0008d4119c fffffc0008ff1f38
7cc0: fffffc0008babf90 fffffc0009040000 fffffc0008e3d9a0 0000000000000000
7ce0: 0000000008d75000 0000000000008000 ffffff00004a7d00 fffffc0008d127c4
7d00: ffffff00004a7d60 fffffc0008d1294c fffffc0009040000 fffffc0008d128dc
7d20: fffffc0009040000 fffffc0009040000 fffffc0009040000 fffffc0008d1045c
7d40: fffffc0008cfc468 fffffc0008d74290 0000000000001b57 fffffc0008c04958
7d60: ffffff00004a7d90 fffffc0008082ff4 ffffff00004a4000 fffffc0008d128dc
7d80: 0000000000000000 0000000000000005 ffffff00004a7e00 fffffc0008d10df4
7da0: 0000000000000101 fffffc0009040000 fffffc0008d742a8 0000000000000005
7dc0: fffffc0008e3d700 0000000000000000 fffffc0008ee2370 fffffc0008bb79f8
7de0: 0000000000000000 0000000500000005 0000000000000000 fffffc0008d1045c
7e00: ffffff00004a7ea0 fffffc000893c888 fffffc0009040000 fffffc0009040000
7e20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff00004a4000
7e60: 0000000000000003 0000000000000000 0000000000000000 0000000000000000
7e80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7ea0: 0000000000000000 fffffc0008082b80 fffffc000893c868 0000000000000000
7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000
7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
Call trace:
Exception stack(0xffffff00004a72a0 to 0xffffff00004a73d0)
72a0: fffffdffc3f80000 0000040000000000 ffffff00004a7480 fffffc0008227618
72c0: 00000000800000c5 000000000000003d fffffc0008ec4000 0000000000000030
72e0: ffffff00004a7300 fffffc00081593b0 fffffc0008f53000 0000000000000000
7300: ffffff00004a7320 fffffc000822abac fffffc0008fd76be ffffff00004a7438
7320: ffffff00004a7460 fffffc00082886c8 ffffff00004a7380 fffffc0008135d08
7340: 0000000000000001 0000000000000007 ffffff0ffffa2c80 fffffe0fffff2580
7360: 0000000000000001 ffffff0ffffa2500 fffffe0fffff2580 0000000000000000
7380: 0000000000000001 fffffc00082295f0 fffffc0008f53000 0000000000000000
73a0: 0000ae7401c50430 01c604000000001b 00001b00000033f5 00aedf01c7040000
73c0: 04000000001b2000 00000000000000c1
[<fffffc0008227618>] move_freepages+0x158/0x168
[<fffffc00082276d4>] move_freepages_block+0xac/0xc0
[<fffffc0008227f00>] __rmqueue+0x740/0x898
[<fffffc0008229654>] get_page_from_freelist+0x3e4/0xca0
[<fffffc000822a5ec>] __alloc_pages_nodemask+0x1ac/0xfd8
[<fffffc00082886c8>] alloc_page_interleave+0x60/0xb8
[<fffffc0008288d20>] alloc_pages_current+0x168/0x1c8
[<fffffc000821e62c>] __page_cache_alloc+0x17c/0x1c0
[<fffffc000821e788>] pagecache_get_page+0x118/0x2f8
[<fffffc000821e9b0>] grab_cache_page_write_begin+0x48/0x68
[<fffffc00082eb758>] simple_write_begin+0x40/0x168
[<fffffc000821e3bc>] generic_perform_write+0xc4/0x1b8
[<fffffc00082200e0>] __generic_file_write_iter+0x178/0x1c8
[<fffffc00082201fc>] generic_file_write_iter+0xcc/0x1c8
[<fffffc00082ba89c>] __vfs_write+0xcc/0x140
[<fffffc00082bb588>] vfs_write+0xa8/0x1c0
[<fffffc00082bc5b4>] SyS_write+0x54/0xb0
[<fffffc0008d120f0>] xwrite+0x34/0x7c
[<fffffc0008d121d4>] do_copy+0x9c/0xf4
[<fffffc0008d11eac>] write_buffer+0x34/0x50
[<fffffc0008d11f10>] flush_buffer+0x48/0xb8
[<fffffc0008d410f4>] __gunzip+0x27c/0x324
[<fffffc0008d411b4>] gunzip+0x18/0x20
[<fffffc0008d127c4>] unpack_to_rootfs+0x168/0x280
[<fffffc0008d1294c>] populate_rootfs+0x70/0x138
[<fffffc0008082ff4>] do_one_initcall+0x44/0x138
[<fffffc0008d10df4>] kernel_init_freeable+0x240/0x2e0
[<fffffc000893c888>] kernel_init+0x20/0xf8
[<fffffc0008082b80>] ret_from_fork+0x10/0x50
Code: 913dc021 9400d2db d4210000 d503201f (d4210000) 
---[ end trace fcdd7f8cb96cdb3d ]---
note: swapper/0[1] exited with preempt_count 1
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

SMP: stopping secondary CPUs
---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 27 Oct 2016 17:01:36 +0100
Message-ID: <20161027160136.GD24290@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <20161017185801.GT25086-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

Hi Robert,

On Mon, Oct 17, 2016 at 08:58:01PM +0200, Robert Richter wrote:
> Mark, Will, any opinion here?

Having looking at this, I'm inclined to agree with you; pfn_valid() is
all about whether the underlying mem_map (struct page *) entry exists,
not about whether the page is mappable or not.

That said, setting the zone for pages representing NOMAP memory feels
like a slippery slope to losing information about them being NOMAP in
the first place and the whole problem getting out-of-hand. Whilst I'm
happy for pfn_valid() to return true (in the sense that we're within
bounds of mem_map etc), I'm less happy that we're also saying that the
struct page contains useful information, such as the zone and the node
information, which is then subsequently used by the NUMA code.

On top of that, pfn_valid is used in other places as a coarse "is this
memory?" check, and will cause things like ioremap to fail whereas it
wouldn't at the moment. It feels to me like NOMAP memory is a new type
of memory where there *is* a struct page, but it shouldn't be used for
anything. I don't think pfn_valid can describe that, given the way it's
currently used, and flipping the logic is just likely to move the problem
elsewhere.

What options do we have for fixing this in the NUMA code?

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Fri, 28 Oct 2016 11:19:05 +0200
Message-ID: <20161028091905.GM22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <20161027160136.GD24290-5wv7dgnIgG8@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
Cc: Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On 27.10.16 17:01:36, Will Deacon wrote:
> Hi Robert,
> 
> On Mon, Oct 17, 2016 at 08:58:01PM +0200, Robert Richter wrote:
> > Mark, Will, any opinion here?
> 
> Having looking at this, I'm inclined to agree with you; pfn_valid() is
> all about whether the underlying mem_map (struct page *) entry exists,
> not about whether the page is mappable or not.
> 
> That said, setting the zone for pages representing NOMAP memory feels
> like a slippery slope to losing information about them being NOMAP in
> the first place and the whole problem getting out-of-hand. Whilst I'm
> happy for pfn_valid() to return true (in the sense that we're within
> bounds of mem_map etc), I'm less happy that we're also saying that the
> struct page contains useful information, such as the zone and the node
> information, which is then subsequently used by the NUMA code.

Let's see it in a different way, pfns and the struct page assigned to
each of it is about *physical* memory. The system knows all the
memory, some is free, some reserved and some marked NOMAP. Regardless
of the mapping of the page the mm code maintains and uses that
information.

There are assumptions on validity and checks in the code that now
cause problems due to partly or non-existing data about nomap pages.
This inconsistency is dangerous since a problem may occur any time
then the page area is accessed first, thus a system may crash randomly
depending on the memory access. Luckily, in my case it triggered
reproducible while loading initrd during boot.

I also think that this is not only NUMA related. E.g. the following
bug report is probably also related:

 https://bugzilla.redhat.com/show_bug.cgi?id=1387793

> On top of that, pfn_valid is used in other places as a coarse "is this
> memory?" check, and will cause things like ioremap to fail whereas it
> wouldn't at the moment.

IMO this is a misuse of pfn_valid() that needs to be fixed with
additional checks, e.g. traversing memblocks.

> It feels to me like NOMAP memory is a new type
> of memory where there *is* a struct page, but it shouldn't be used for
> anything.

IMO, a NOMAP page should just be handled like a reserved page except
that the page is marked reserved. See free_low_memory_core_early().
Thus, NOMAP pages are not in the free pages list or set to reserved.
It is simply not available for mapping at all. Isn't that exactly what
it should be?

I also did not yet understand the benefit of the differentiation
between NOMAP and reserved and the original motivation for its
implementation. I looked through the mail threads but could not find
any hint. The only difference I see now is that it is not listed as a
reserved page, but as long as it is not freed it should behave the
same. I remember the case to handle memory different (coherency,
etc.), but are not sure here. Ard, could you explain this?

> I don't think pfn_valid can describe that, given the way it's
> currently used, and flipping the logic is just likely to move the problem
> elsewhere.
> 
> What options do we have for fixing this in the NUMA code?

Out of my mind:

1) Treat NOMAP pages same as reserved pages (my patch).

2) Change mm code to allow arch specific early_pfn_valid().

3) Fix mm code to only access stuct page (of a zone) if pfn_valid() is
   true.

There can be more alternatives. IMO:

 * We shouldn't touch generic mm code.

 * We should maintain a valid struct page for all pages in a sections.

 * We should only traverse memblock where really necessary (arm64
   only).

 * I don't think this problem is numa specific.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Tue, 1 Nov 2016 17:55:44 +0100
Message-ID: <20161101165544.GP22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="veXX9dWIonWZEC6h"
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
Cc: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
List-Id: linux-efi@vger.kernel.org

--veXX9dWIonWZEC6h
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline

On 06.10.16 11:52:07, Robert Richter wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
> 
>  kernel BUG at mm/page_alloc.c:1848!
> 
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
> 
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
> 
>  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> 
> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
> 
> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().

Below a reproducer for non-numa systems. Note that invalidating the
node id just simulates a different node in reality.

The patch injects a (pageblock_order) unaligned NOMAP mem range at the
end of a memory block and then tries to free that area. This causes a
BUG_ON() (log attached).

-Robert



>>From 20d853e300c99be5420c7ee3f072c318804cac1b Mon Sep 17 00:00:00 2001
From: root <root-Vr5RZmvqObNE+7tKeQX6HA@public.gmane.org>
Date: Tue, 1 Nov 2016 15:04:43 +0000
Subject: [PATCH] mm-fault-reproducer

Signed-off-by: root <root-Vr5RZmvqObNE+7tKeQX6HA@public.gmane.org>
---
 arch/arm64/mm/init.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/page_alloc.c      |  4 ++-
 2 files changed, 81 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 21c489bdeb4e..feaa7ab97551 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -36,6 +36,7 @@
 #include <linux/efi.h>
 #include <linux/swiotlb.h>
 #include <linux/vmalloc.h>
+#include <linux/page-isolation.h>
 
 #include <asm/boot.h>
 #include <asm/fixmap.h>
@@ -301,6 +302,80 @@ void __init arm64_memblock_init(void)
 	memblock_allow_resize();
 }
 
+static struct page *inject_pageblock;
+
+static void __init inject_nomap_create(void)
+{
+	phys_addr_t start, end;
+	unsigned long start_pfn, end_pfn;
+	u64 i;
+	int ret = -ENOMEM;
+
+	pr_info("%s: PAGES_PER_SECTION=%08lx pageblock_nr_pages=%08lx PAGE_SIZE=%08lx\n",
+		__func__, PAGES_PER_SECTION, pageblock_nr_pages, PAGE_SIZE);
+
+	/*
+	 * find a mem range with a complet pageblock in it
+	 */
+	for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end, NULL) {
+		start_pfn = PFN_DOWN(start);
+		end_pfn = PFN_UP(end);
+		if  (end_pfn - (start_pfn & ~(pageblock_nr_pages-1)) > 2 * pageblock_nr_pages)
+			break;
+	}
+
+	if (i == ULLONG_MAX)
+		goto fail;
+
+	start = PFN_PHYS(start_pfn);
+	end = PFN_PHYS(end_pfn) - 1;
+
+	pr_info("%s: Injecting into range: [%pa-%pa]\n", __func__, &start, &end);
+
+	/* mark the upper 5 pages nomap of a complete pageblock */
+	start_pfn = end_pfn & ~(pageblock_nr_pages-1);
+	start_pfn -= 5;			/* unalign by 5 pages */
+
+	start = PFN_PHYS(start_pfn);
+	end = PFN_PHYS(end_pfn) - 1;
+
+	ret = memblock_mark_nomap(start, end - start + 1);
+	if (ret)
+		goto fail;
+
+	inject_pageblock = pfn_to_page(start_pfn & ~(pageblock_nr_pages-1));
+
+	pr_info("%s: Injected nomap range at: [%pa-%pa] zones: %p %p\n", __func__,
+		&start, &end, page_zone(inject_pageblock),
+		page_zone(inject_pageblock + pageblock_nr_pages - 1));
+
+	return;
+fail:
+	pr_err("%s: Could not inject_unaligned_range: %d\n", __func__, ret);
+}
+
+static void __init inject_nomap_move(void)
+{
+	phys_addr_t start, end;
+	int ret;
+
+	if (!inject_pageblock)
+		return;
+
+	start = PFN_PHYS(page_to_pfn(inject_pageblock));
+	end = PFN_PHYS(page_to_pfn(inject_pageblock) + pageblock_nr_pages) - 1;
+
+	pr_info("%s: Moving [%pa-%pa] zones: %p %p\n", __func__,
+		&start, &end, page_zone(inject_pageblock),
+		page_zone(inject_pageblock + pageblock_nr_pages - 1));
+
+	ret = move_freepages_block(page_zone(inject_pageblock),
+				inject_pageblock,
+				gfpflags_to_migratetype(GFP_KERNEL));
+
+	pr_info("%s: Moved %d pages\n", __func__, ret);
+}
+
 void __init bootmem_init(void)
 {
 	unsigned long min, max;
@@ -320,6 +395,7 @@ void __init bootmem_init(void)
 	arm64_memory_present();
 
 	sparse_init();
+	inject_nomap_create();
 	zone_sizes_init(min, max);
 
 	high_memory = __va((max << PAGE_SHIFT) - 1) + 1;
@@ -479,6 +555,8 @@ void __init mem_init(void)
 		 */
 		sysctl_overcommit_memory = OVERCOMMIT_ALWAYS;
 	}
+
+	inject_nomap_move();
 }
 
 void free_initmem(void)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2b3bf6767d54..19d74637e242 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5077,8 +5077,10 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
 		if (context != MEMMAP_EARLY)
 			goto not_early;
 
-		if (!early_pfn_valid(pfn))
+		if (!early_pfn_valid(pfn)) {
+			set_page_node(pfn_to_page(pfn), NUMA_NO_NODE);
 			continue;
+		}
 		if (!early_pfn_in_nid(pfn, nid))
 			continue;
 		if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
-- 
2.9.3


--veXX9dWIonWZEC6h
Content-Type: application/x-xz
Content-Disposition: attachment;
	filename="typescript-crb2s-test21-201611010941-trigger-mm-fault.xz"
Content-Transfer-Encoding: base64

/Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj6S5WKHRdACmYykarec1wWuOyuHWSQFx0n4ZrJjMw
RcfxlPmc5+LvpK5TnaSBKhmU4vVf1qgflmSthKe/WQi4nGAyl+3Rl274lFz7DCRnf7YXMnCX
K5kcyTcuk18ZTBIMu/SHQbbHyN7s5cJwO/+FIWOGgtAFJ6ZU5yulK/g7rPbWw9FW9fUzaDf8
PruYdTNHcfzD8yoeOjKjUX/b59MC0DT/vXIozsUbronvNogSvrEdaRaZEOrIcW9p8s2exz78
SZmCOhnjgrP7oHuIV0Y8P+ZaVymTDnTgMe55kJ+6hpb+4yBYdYbSHwjgoTzTzq8TZAknphPf
JlfhGfDGeIh3k7RBhWRS0yC4KSC21L4Q4CL1djTpns/hg0Ibbiy3tGcK1cPXa7Ol/jwXDdey
L0q/RhK4YUvAWhV6iHpe7oHCiHuAAYmZ/FWj+N1VlFFI/aLO8S+MK/kW2CIyLxDHVkkxCdfW
qFQsnBDIjl+ALctRthC6K/CT6M9vP3RYNwPSVPI7dgOdZt9D2qfRc+kSTkb8iLbh63Z8pVFX
WJos3ud6WL/A6gMIkcyyLxBMwb9T8FKNXoH2sng/rl//p/VIL9bipGiyNtyXYFGq9gJ3Qrk0
FvkXsKgP65Y7qwqX4B8kCg9Y06fKRCzr2dFv7nfqaLt2joeqXS6tlxc8uIfzeqmsx5KSx1Yr
HUShvEnsEjiMmbD60I9MFV/GlDf3KRHbT4RAi40HGYyvBtUehhufKycdA/iLR1FnVj1+MdJW
jBalEX2EJY4HW39Htg2u8oaMSWY/ZmpTyM88eYv5LYqIDc15tsisjojgPYyIP/R2ch95eZmk
2uaPxHDi0D57jVZdYPGzKq65TpnBsBlJU0Hft5x3DTW8Ec211UokWexSIXIznsCNLIszJE7F
ua8ZdUrbEP+TDHsnJlLjftu3SYSz08YU3D3ZRaAH1wHB1fEtCY1wsMD3WOJaZDi7Jed3mR16
7qNTGuPOqL7Z3IHNCQ4ROXP3++ovzNHqafISuzhtFkbTQZirymmHKEi8n8hAMlUEhhGpPp0r
gI+w49PYMQlRLKLQCzOb5+RpE+H1xlBmcMyTkW9X4GTrp4RY95XR4PcaJSJ66ROfEXXzLcUN
asw8TAXsVK9APEl03sguP3TX7deIexlmnJIVJft/cYoGXctrjJcoPkHszV7fTTpMP8Of+tHD
oM/JSJbLOBGPCuWXNeo5ckNkMqtfR5XLqJyMglZGHBZy+B2ARoPdexLqfF62uN2oeLqBjdKI
FzxQ6VD9M0D/U0WvtFfE0zr1KjnI0qTO105NsjzgLon+j6erDOtlrOs+t33EBWePlJEXT9GD
mOplR5xZr6qrg1vlBOuKT2RB0/kHXnujonBcLqwX3oG6BemY0CPV9ZFa8biqWo0Kq8zA9b42
UqioFzb3ktT7FkDUt2vXQ66oJ0YoIAR06MfEtfvgajS20aSP5SUvk/6WiNJhTLEPhXOyyKjv
PwdQAYTeExX27Fe8urP2zOhs0lbgVSdgKDwrstbQzDNrxrOq6ayQgL2saAO48VmoLu1jCPOx
fW2frn/NGZIeOxuwvoHviyxwOVC1G6yRz/OP1fvRbybNHlA45nw6iRqpyrYZ/NPWumjtrrPM
h9nqUpcJrvS5mumNzklMI5g7ab8BQ8rB8Bjr28VWVgLro5MTxDygp6BeDQlnJO14jj+02kY7
gwcvZSBvi0xW0W/9EJIWz9WrmPexlYLsoxpyR5UmO70pZ2jZwIqEWpKY9e0i7/SsWsx19Oev
A1DcptSKMsvptsKBXE8YCYmNK2Cx/i57Y849E7RqEhY7NlmVT5W85MmH3D7qEJCgLBc8LFuv
tbngYMbEnzAMG28jDhTUwOV0vqFRy90QQ7HWuH47+0M7ZGyKpOoUBg8sWBt3rymTQjaT3sbB
NHBO5g8LgWzkK0sHCa74ejWnu6LxtF8o9r8Cm2ssmU93lRIVypbTNzzfWDHwV4EnnJ/v4VTV
/epMd12gIEumWnAaRAhu5EBQdZRIc24mLOIl/zzYOOyph/M7arDxkj1r5K1D6ETpozigdChg
7GNpKmL21ColEZRaP0Ack+DiId99FNS5EOgEiSBFsFsld83mZFFTTx3qWh6NQcTLDetvm+Yu
aRLUb1wLDNykLEJYX5i4TH1shOA/zeoQ1L6JGDKmlpUyJox6pxGJ6DSmhseXqLsSCtHnTPkx
D65vM/J0OGTKKWZtFc0zWnv2AjVEC5Ysk3lqkPBPjYZseai07PTRHWeRVvUHLdtN3QuBrJ2r
QAjKC4rI9FRtjeMB5t9JZMLjxoqv4TVX8H1mxPRhyhMyMf/QJe7kSddJeGha6rt5ghnQENK9
u/gjSQH55MzAZf3hjTHmLncjHiYdjBzOLh6dnh6z+rApCtQbvjI3Lo2y2rIwkj4NBp7lyEMt
foRMEkDeujGIFmZA3Yd24v4pyLBpHw8vmztFHmrZnctsvIktUvc3AGhkDDBZDyaxlztL/MRX
X8AT9CzuGO6kFemiVumUl8GHV8F+u4yLO6eZ5nBNWoc16ghcpvKwekgK0b0yB1UDAgpykp8d
zrNcWlV4mgGnILPTHsfZMSapEDt+ZEeRlKozMCLJT4BPbk2Yvg3SD/4Wht93Ato60dbM8hww
wRjbRbmUyy+6uHY0sXUALNqnZL28ZGQQOna5Qbh2Yk3lh5jd2NAROukKb8Mvrtb8pqj0Q4wy
dhfa2pH2Y0ynzKtvYKApdVsP7kWRdyU8z84r6VrQNZ6GJ13Ib09vATmA20uz4ylQZCW2v+Cq
EzuI9AXr27/pQETjwowvCiSKfqS8yZHYuoXxtCtD73iNx49LLYSV1oWGnYo1cmFlRxcDw4e2
NgfopaAvFSFxnlCReBXGmhVxilUnw/Q8dDKYg8jp1G3JJ5+E4EyRTmlMbRpRMAmOPpdKH777
ZR30o7jdhhPjBgk7tjmU/WHnUclrJG4QXNWXJ9qlHCwpP7THdO71MixFXdTx+Nm1FPe9pa4Y
GGe1p8Me1d/96Nw0teh+Ty1G11Y2rd3jGVxvdIJ43xQpASO3AZrVo/paAOtKbEAZjaG64fjP
Ea0BgikNwA5NXpWMWWDvNFnMlUk3KG24Zr4n9s2bQALgde4GFBfYV7o5nM1IFgn847XmqSzm
x+ru2JRcOLpobEGsTb7BjUU0TMGLgzPKSAwQxzkWyG4kEMLrhHSnjMV81PeKeYTRF40TXSl1
wdVBzVkA/4n6a8dq8Dru5UgL/S7ee2cto2V0PQGf4JtApEiTnP/JUwivp8gY+EsTb2sGOEAW
KkI/6OZVVxNTZ8XEzmo3uQrGwJvSkzlBZC0Wz58vyZK88LZhYGRL2GwFOXUWXZgzOB7I8s8j
Tmq5vE10IfeVFjKgXJXXKH3NkRBc/y+n0SXB1jfNLafpNzurjHm+Y4WhP+NgnmxS+l6mZOFB
zXnXILc8o2U3UHe3KX1l17RUSJ1AeWdLlvcXtJy61IQzcS9bTAZvAbk521ntOsZSdRJZ1Eof
Jc2lvH6jOIdwWqTIoHnwNfzUz9SM5os4qPnwA/7gaSlLgHO6yxXaKadoGWorX/ZATtbBOg99
V2T1rWlbd5Qelgxn0bLd6+tGyhG9T4hGFVkbFBNkw+MQGgEC7A5UdFtRMeDKtwxFmi9H25Tf
wazddsB9C1KOUjUuOIbseb2CIuXfeTtboVc5ZgtdVR8xqbAIufoeLdluXojy3cAzx1MISoWS
+omyR3OpDB1qz/Hs7zbmdXEm6dtLChq+evt3/v6WQLgM8lR01q8PX5bSSfzMbc9qFdWY1P+d
2PmqZ3E87BORXCx/t1fFAETJr1V85LJlt0l3oWVYxUCd7e7wrJ1IX6kG5tVjm1P9NBNj/DrB
IO+XcAefdFRQVm7LiafARSuFocQz2wIpAiPuJfIaCH96dr6OzqsgkA5+K/FTW6rUUW7/cpho
l/tw7Dtkh4WLcZhsFk6UgPKzsNdra1V0ujOrt5wUjRFRl8hiJFWMkiwAnYn9ydv6L2tEDwO7
eZQc6nyx4lb0V94YRUqhOceyc48+UwRGORflJA89dFlZLlfHvpZsWrEE6I2U8D/YIVl/3n5c
W6vBNJdAKTsX47DruHlsr5VyhnvTphN7KnZ5Qpnxi6ta5dQsTQ1nvgcwHk/CyBM1MuRGi1/z
qT7gwZzEa14DHxmtGR/6pmqeRDul8cwyb9NITy4xYgMMXqp3Jwp9UfOEv3r2wOVP6hZ0O4dK
q7Ltf9FGlc38wsvSqUDrdjwaxn1fJ5Ld6i9hPpI+1WHLFM2A+75xFICYkn/JqLFtLZX3w2mW
FNvcBvJ+wkYTUIHDue+WDWcew6ZlADPoyTv2RVC+TCwcQrFnySMjXTHkfPTqaCdFldXj9NYq
GkavP5OiGFMtv1VVnDY33aTfS9G6v1iTN0cVFngDTkIy3VhSuf9FJrm0WC0TP3EZwQd/2cUf
oqDhGhkzX/qJTD95aucZe7gQ6Arqv/+TqWKZwUSpfnPkkw56VdCoIiHncrHiXJ/btBPdBVaR
Jxjda3390iEhwMO5QDie1jsNqbbkqBkEbTT3Ne0mwU4ATVxX3Dyws0oVxpQvV/vj7tvJMuIz
DZ/otjjPe2DBGOgH2VD3A+EeSS0hc0mHpPjXT3Oql2lgvjB4XMXY+SRS4QlB5ct5BybwJQe8
1+hZ4H/prJn8u65mz5GFDAz3XRfg2sRuyG2dqmdy+/DeQel1QeTH0UeUi3Tz52LK4CJavipb
YAGeA56rADx3IbFlsEb3lVC9rR+8w98tgU3v1INP6kgj7Syaij1zvrc3hiqaLqhPm8DimRJg
C9v+3UkMkiNBu+Ip/2HyGevBkU9cCaKwWoieK2+dXqtjVPloTuKIyavR336W1Vg8OpcOad5L
L21PsE6yztKu053+7YlRxaV41vhDo4F40Bk3+O+DuopeCMx1n1B2avW7V7yQmQ6qte8JoauR
+aM24CQuBPjOnBErj90I6x6y9MWRwZKO0PZmDUWlOvHEF2fXn9H0B/IzH8TEiWy5n7M3v0xr
gX5ylgCVC0Gc4xTkwZ5TQ720qv3wlJ86ImgknS/5Ofwq+vxMg8vzfJFZNHTCuUIxjwaJYEUP
l/4IyMNu2qO8qSIsqKiqoy6ou1q+yPN1VNcK0CxOr0zOPfde38Ac6bekUvxE2EI2t7LrchJT
/5abG3uqtSmswlqenXetyXai3HCFvW3AfvF5CJ2QmmrwK83+fpeJk3jl0cIsATimcs4YeHH8
LoKD/h3pIoRhHCVXjEBt4EX10XQCeC22OISopVxyFUH/mPekRLRVSyWjoQEbwjdNvt69LTw4
S4C3v1KV9O0qbA45mug+Q8H7TabiWo285arREMWJi+NojXIJCDjQjOCc9JQF/t3CKBVIJZk3
ONWacu57kO17pqbd7QPCTw2V4uLIApFVJrSseWkJ/k83UhYR6lVniK3rA14MAlrEaVp59brn
Xzgpg+C5MY0O7lt70szGbkSXV5+v5jnDyXGAsGZPEe45im8IVYIF6jIr+9IYeGC/K61CsA53
2u7Gb+AacCDKTE/w6tuHd+Rje6jg3INhQmKYAeoMrz07fUh/ZEi49M9nZCqnmdJ0VPAIt2EC
qK8MiA+QsK8idmx5/AFVXGX8oxyE0fxcYGEEZwkkMJpJncutdF2EA1M2FAlOM3S0W5rqn1Dx
dr+aJz093wAiJnTEpwwRLN7galgOXkMJ855RFEfQF8C1dGkxt7LxzADAm4nyUH7L42HRvhDr
nQ9B9Yi6I/Gp62k8AMTwVI/WTsJE27Aep+pKkwme1Uj/pxvRXTSvRRYK+1u3L1vFej8GHUSD
y3E3EhHHapknLrBYgITWFYctispmBolb6s3Y7l/j8uEn/vdIJL38hoZJQQs692YE1qSbBAwA
O+J5iun8ZeY/jLCryU4EiB5rtEFuWy34UG9by/TDHdLh/XTHkL7iZ+tXuUt+ySEhaEAzxq5Y
rzndejNkyWfSU+2BcdCoJH1L+cLWB1tQTzIbzGbrbfgMfSzzKJVrVkOqdyaopkim+ZaIAfQx
vpzu97B68Aam2EDtS8K/2x8ydZswi+saUTT39psqsc7jccl6ET9TZE84lAJTTp09T9Pp7N5w
M2BmdOjrfzSNIBmX6cDK0RmEfsV/97hicfCg8IRM3h6gGuz7wX5FiNIh7gHnNnZrplJWZWLc
xkonMiTgmbs6h/np4U0Fkrlq0sDE+gZh0ilSMzoYBLoYOcaR+y77LaZ5u1RL/1Ch3bZMtNED
EDSQSiQhpdbhFkzyD+anjslGTlI07Evc0RqXxsHiegBJ/VwN9IYrzU5NYrqzpZJrM+Cx3Nnt
VZOqoVUXhfAtTr8zc+3/Jwbdqef4BKI0+sPql9NJGvC7gkXDazUeC1ojhABUU6nwuv0zSr+Z
+u30uePttKeFO7RG0tQTDZ+HT/GJnak2X1pI5DMNE+CNFvErRQyCYyS651inYlVeiQk9ETJn
d6ie2O0Ko4MrbvtD5+F3PiQGi31cDSKDfW41CEcTTP6gEtr/rNzEXJfK5Afu0psVC4lO1nzS
xdY1IAfDIfoAPhEyMBRNNBPAD3yA9PWld5W2Ns0f+vQ/LbNWRvfMO/snzdLzf7ZZgQGaOPHr
Ad9T2A09xZ/DdG/PPouJ7Gc6wOuNLn3Hvm7rfnHgqaILBWK1HdWmCFp3ODyaQXFYHFzW2h3s
y/A9qtRUhplpQ8aJDgdhBWN7dv7h8jqUgx3e6sHU1hKii8OzQ3YutiDFolVe68OHBx9SUbAD
qfb0B40p9LxI4omY6rosFa4Dzl4w8yphIjol1+84YARHarSeFZzmIrjQsy1hhrK1ZtXjwGfm
3XrwnYAyJitGO7+20OlVac5KP09Samjbrl64ee+xBSK7aEKjvk5pdPd+fUd+tSgL/7YEA+rL
/b9DvphvH28EIaNfTq/GYp4jmVTzkt1Vn8MeuUXOV/UvoagXLL5gKGxhK3OoLx2U5jWvHSBb
4TibbGEVq1bsFzIdfS1TebpV3oN58ujWAalBqs7saH6wQwuhqUwrQLpmvXGZZB/+DeUXIXmQ
ocDYzP+sNEvSqSMhTTo+9EcEUj8n9zkzaQ+m/kICpfRudzcCx8vST8Ip6MvA2OeH/eZkRWLg
VKkmt4MHC6DloxmqWSn1seSY7dGaqkAiInGXd7j986cNNZBiFzvppvAwuMej25u/oEHFApSa
kmEbxtSJTrbKwx4lpqfaSG1L6v90dgINnpWYvgjpjkpsAr3/kJJTTysSG1cYmmgvVYE8AcE2
cO6jPx299asc+y690TfcS6yjEIrWJ4CG7IrYEk1OfBNFDPXSPbTKjhOF8GfqPxqxXZ0Jsf9j
GJuyg0kf4UePaPvUJUNy0WF7hgpqU1Nv4aLJ8KFxIpZ+LpCHoatC4FI2bEp+waAmKLpAQJeu
tcgw50CIvLBjo506rzM1Icr427m+txpjZBysxlrf5jyGlmC6WIRdcxuPSd7rP0KTXltG8kdG
f8VQXazs2VtBE81kG7SqF34F6qs/qfNuvacT9gmtaxvBKqtm21gLbFHyLAz5/euXPmnprC2D
UWuPxDgwAKfKyB0hjOJOmakTJO27NrToLVclnahLRFsbuPq558fYXWEeypQYXDAwCy6ulOY1
yodmMscJDZeSHii7nORWlBPQgO7vE7yuAyVP05sAaaos2IfIyOYzo87zQUSNCWoKz32pLRQH
fySoaKzOb5ZL4wj+KVhET2GWuUKrfoCx1tLDZU6wprNwUgyedj+ibukjh5ljnRkOt/uX85XW
qefcqU3qpjICHz3rIh6Gn5I8m37DOtrS4IHixhs9P2SwjBQMipdsCrdEoJ6ETnALKL2gYnFD
Evr7a+x7A/mz+oT4olKLlGQtLDyxyG6lgWd9tsVbyQSA5LT22TDJxYQPn7F7txcvj553hG5S
+aHuZ2EVDVt8YQ4E5LTJmailxvpAeHE27rGn/VguvHPeQUk4IvI2wyyUCfbUEklYsZuoZ01R
7u9hJe+A5jUYWNvx2i8dC1LmxOTDajNZVJepnxnYQVHsSz/r0V9RdezwIYiET4YGd/CuCpDV
bzJDSe6paFf4u7n+7V+OMJpwwJMC4L+CV2PcsycWSZNGWSLiO4WCesbh0dFJKuJEIT6xCo5c
YdIwNWZvnMlXTlfQWrojlxLjD12mdBWvEqccXzKYcJBfUJTv7kzB1SeO+RDWYa2S91EEhPHF
RecWr2b9XrjkziBxiMZuWJDjfeTxKUWv+u988l+s4NARjGYRtmtclTFdixbi3pnLQ489ceCr
neFZkZcl9gNbzUddcqziTqXseINS+YxvVUJE1KpmbXm9u1ei5Qy4lXXeKa4i1qABBdKnLGLO
NPsn6QjyIt2fQdPMnk2XsRTz+wRJjOtGFkcx/XPlMgkDcjUsylQ350wEAlr9JRbk1sh7mLym
kIu7B3OaG6RF0CfvlfLWIVI0HXa0qNdVCrg6Jvuz6haIcls3xB5Fyl8qjKHJFzi6jyiW9e3P
lhn9t3cVWLQqZ9o4wkMyOJqm8vpBRiCLvtSagcZ1v6FPTSp9QAwN3H9uBtRU+zJwyyz4zkKo
w2P7yEqMelpIyNAMdB/PlreIpE4h5oYRfzvDOmkh06hjJDSXqghCh7ZmAjt42zhw53xIo8aC
DvlQkUq1vVOeQjyTJP7iaY8r8fLdfRqnG0NUFbmB51JPmu+4Q0PEqKek/0M1/+aHgqmjCkWV
85k1fET9GDKabVV9k6jNtFkXqjL9Hy+iqAsIdj9SxldMeTwpOxIif+PAhQ7A+WZUXrltpwbY
gcgFoHZ8CYsQVFdJWtJcaMgNU0q8JRGxt8wTgtY2fnThaigcO+gZoY75mYZdjAaUwbuy+zLA
pzGMdZ2/Nm7lSJdV93gKrg+lA6byLmp3Qn11y6Iuk69/XCe3mr0rYbMxivEehbowuIg0jJ8y
a2SOZfhDB+daA1NYjdiBHQTdfASeHFaCx5Cuv9NUbEOkZfVOKT3I0MXqyLlSmpD8Xum1Afjm
e1P+TfA5D5SkH1zOJHVfkZ51QvUp/1NV3ZJBArdBiPVbkzUa2yWlLZGuw12CzqxIWUOECVPa
EQG+qtnWuao2dp2u74ktlt9JSMu+ux6xpEHtyK5Zb0Gp6BhFahgoxna0yve5rT+eUOveOsSC
V2i9JydK9EZcVCCIwt2kMGEaN/NldnEQ3xpGZ7uBqKaao3Q/IC+VQt8SveNTvEXfYZ+oWqqe
fFJ3I7T/hsMHl9974/J94urDPOdzzmy8UMBVWkvTv8h7FnSGIXOVBwREigaxLp2z1zvCcH/b
troDwCP37HQ5qNXrZpVgFkllnRPT8PRpekAsh5CVu8y7kypVLdx3CVkgFSCb+IZKA9IzGChW
uvIvtcgNbhs9FNgDxrDz+KiglAoTdbw40gvRgERlaGHM8QbERQmB/HZN0OB4xaTTCoRtDTd4
o38d8YtYKBfSiE5jSRSg0MN5hMfvBtrdBqM5J+2wy3fqo18RnmL0JK5DCyCka2LSgHxJ0Qr3
mWbUM5K+u69cP9wEQ7HxxTxgUIsnxkjEvAJiQQxf2Brq2GWIm5K9v9vBhkCwnD8JEvdSaNay
p5NP9Q8eDnZcnR3UlCQ1TGGZk2d6dq3VxOU+tlvmOaUAsWg+ob+nRysP4wy3NW4dxJo8/Xh2
j6dBv0IWiS/anK+nb1rHuTMkzW0ALvVGAEhI316zpol/oyzNFBrE0tcdPS/XocJcucYrZjp6
YT1rdwhiUrBUx46sojGRDn/MQewvkiEOeYVEOQpGG/v/BULokUTTCXj9ylVHXOgp65txhxur
be0nLHBrqRxhDizrFT3RiXqR3GV5lTueOKrzg7NXOCeerHJFDDuj6Z/yqhwRuvU80ZLiaumt
DLFctP8Gn27ezvSXoIwiHdPMAKQz7QYeEoEp0ro6pRcVdBCyg8OZrpIcLqMcrucquZ9V0Vsq
geBBi+DPPtoOJ+yQSCznOkOLXbFkvXVNHPMl7f4R19nSoFL23JPtiscZ4W6TtNq7JYaNHmiB
Irpmn+XxArpvHZ6qqeAePdegHEdj+VMbA97SsQFwEJLtTDyVW5KlVQJyHWSKd5+W4tIn6HoA
1dNlpQlSwcpb4StQ+2nGpRFt6YCMU3fTBIdLABR5SA7XoG2svtQRH9SaDuzaQ01jNUMJZ37n
BS56HqOIlalsIVbbawAPJBEUqfEBE4B9xGVlSmKJRj155ERPBR+Vefik/z9BfGVn1mjfU8pT
CNpzNhgu7qIFRZFymojxsAW3eZmrHb4jdnvr9KMmPKtJ3Vw+eq7vhqS0i1rlTE07h7Q+jWsu
jgA1DwlNERJ5CSJhUSoLkibzPuzJuO9KpPCIQ/CuOGACVoee41aVTQIzVf/72/CQiKIJ75Z1
RFykc/kVzQqAFhRlhV4oVZ/6lE5Xy4sAe8AAbKYRNhP/dKrudOS31QO7gfun8WungB/nYdC7
9FXzwMikG0U8v1we0lPIvLa/tQaLvD3Pduekv10JBncaJ4euA8ZezPjaDOjQc26HxAjbpK5A
B0QX/DZp86TkiKHaHnio17V8nJfincGnmrL5mZfT4CvV0DTGIt9I4ec3TVy3Wh7dGSaeEyzK
Vj1+yT3RA/FuGwXpk6WLSFg5SYHwbahk1RXqaVC7NcizI8A3G632zZQoSKXrWJs7G1bgxAz7
+uAYT/DnYJ8d9HPgRyVb8V0egO7Oa2qshwzy0XL9C/K01hIjXNpbfkgskzXBmnW6vP32pMtR
8TZ4YwJxf6rhaPAhgkf/tkUczB14eZc9moBTo4VKwDIpi7d/+mIw9kHr2JYyylSVdRISgUXX
q7MfAOqCKYjnrLJhO9040QoDNPU9k1NO4f9pWWGVBUmUfnCDJMokN0E0ENm+QwxgDZusRMHZ
ftjX4qRhAWn71IOjTlFr4pZK8V1+hxNJ9gK0lznc2R3vCm3KJE+QWuZbk0dQkhCmBUlQ//sw
kPcrq8wU53I+UVZpt+4UX+vqyYKYSm3oBRTOL5pmrTnpWhlai9q/VvCaWzNVwD1D3VJeHBdZ
v3t9nT/fcMhjK9Ojt/2SLEldVJGk8eG8ojnsXwgDH42vigCc8tmS7yPfqj5eyW/P+GpqTzdW
J9BPKmxE3lwrdlzKZtenuFWYla3P8Lh/MQeNytkYJuVGbv11DH2Ud3c4GFTA/6zurOp0CjDC
V1FemSDjIJFDyVZfMZnn3bp3jFKr/+6UoBgNx45hpm6sz8Hnr0CTeZzbT0csEdam0XnhcEA8
oMNv4WkH5JRl98Vn/vwdwTuwmrfK5XWSMXi7XNHIj8u9Tm0HRIWAoPM5Oorn1pN4X2bMpu4z
Q1jmvCA+4tstttYbC6EKSH+sM26FPvO2LQ2eb6LkI/Jb+8u4gsfMJmdomkbUP47dGdawTIBW
iTG+JlGiMmYauir7+NGrFqVODSfD51cFul/5WunZSXrwXxORv4yBtj9nX1DgPBm0Y9JwBtcD
ixHtCDq0+qI3bHZB57pcc8HqVeWEG+nF5TnWNwoVCTE8NoSMTvbtEMNvBSBxyp81bMqapzQL
sW1xAzjtpwCTjIDHvZHzrWJoQY9hHwR2FapcTondlqwagcHqfNhZEa5O+5RQMXgV2085x0e1
row/AtyJeT/Vq7311HABbIJRt239YYsACCsP2+u2bjOdpaRhvsjTgTMxCDb48JsxrtwY6kMj
jxHg0bnEVVQk54nUbF/wdQ34wUTwW9BFDsYwMMPqeTGfMpLKWLAQH1iA0gjsHiYyRq4xKMn6
/ohdEd3ZgvVbfE1sxT9Aj9xN+6EAXtqxZPEYi4guEEjxhaStQxe7TaE8x8jjTz/YGyYFZAKV
IB61xvr+w/ztfyK5Tbe+KlwSquLO0MgzkuHKwRjfg1VU3veU1x4vlTFzS/nmdIfBTubv3spe
yR6dAynhQO4uR4CseJbYQRr5D+QmovrmYmjU5Qg+UabKXObnjVqgMfnxr/Prxo02fnwINW57
ukVNO7VChYT3D4M43oByZwrr1DemagU0F2QsH0Nb5h09gaJWuBd7VyraKvU5S4llkXFO9X9h
YvTVhQe92wl54LwDH0pfnlNxch5nUrTOUbUdsoqImGHMvqmaihcZE8zFEZaawpYz1vRDbSDR
gcWrCpny0gYEUsnZZjSxhgq0tuNNrRZ3CRwfkwgn5Pc3xYHR8q6OblhJUTfvCvxR65/2a/Nm
VT+4y0r9eI9YjSypOj5H/7A5ASLTWmmK5yCOSXtbdlMfT5GyxxDLt8QCiioV0Uv8nYvSBzXD
sWfaQHJUGqVNJaLfi69bDJ8Ca3EG5BRHWFwj1bEWIXaFB9CC7V6blaFbP3QX1b1YsrpqgyEQ
48Xz5vXxIaxHSMgdgARsM0msdBWyYXsIiZGbDGkGOkHGtw40yx7R8e4y/GM/JfmSDp8M+Vk+
rTqjfjKz05TOTSoZdVm7dyAlRUMq+uD6x6dKXkQqnHtoX1U3IUUB7GVcClJE6i1ClvBXXM6l
zirRUIoc8d9u5xBnfR9bx1sW3AMW/77Q3vt4qOFxlXXwsJhF9egNx7ZFQy6dyWQmltnvV4/v
oaVtf5IDpo7W8PDmZo6ws3LldlR7S0OgNwiz+a1Z+a9XNf38wWoCA30aCKGGt1UW9D/OwBpc
kmxP60BWzyuuaECx05gj9BfTF68w7ulo9F9UJFfbQW8ei54yrXso3KdaIAav4MkmMpqG3U8K
ON636Wi3VOtVbQE4EaR5yLnFnfWAcZvb2RB+B0Uibwdl9PlM7m8tQsFNX4p/Ajidx6t/XQM2
70SmK45tvO8xo/cjWYuyEAuo6xmY4k9U6c2V0n9/15g6oVR618j2H/Oqbvroot36Z6Uzb8Ab
XSx0xql65LZ/fGUU0JsDBmu+I7XtGBi/Li6AJqWXwlcxlaJqBMH7unPMBB0mePrzjF5y+fJ3
2BrY0E9VSxCR8uPKiznlsT53Jo41hrbfDshftXb0AYZNZ31wi5N1ezhcBXjLxivyWKAwW5p/
ybb73uKJbDFO8amc/OclXJX7Nqs9Tt5RjRo/VG9tDSgbYLrvx2BO86EOd5Qam9R16HYwZQMr
JneT3kVx/RgUOABTytEIYJwvvVt/dFQKwG7McruKrTTk2QqZ84gVT59SRNDOECtkR+5CA41k
HgGaGTpXV41FOgSJvAroCkg1RI/JWr7G7v8g50VEz947OP6dpn7ZCv7S+llWrMFS8Sgu4erW
xG7GCI+3EPVQP4rXhGcmEjOvVsiJiHMrBzcBWPeEhXDZgAmlEusJhFQUrizeRtWg+e2QnLxr
7pkm59Hq3JYaU/oBk3rn0GP55oZ4tzmziR8liwegjzFGWB8bKr3l9KKYKyNAddJkB3TrCqm3
O1jFOydWBbbEzO+2RK1rytlwTZn0m4QHjifZtjHe870mmInHXXtmIXTwZ3bRn3G08FSQl3Lv
4+2BjaslVp+LJ+Jf+o1vzNyPSvFDYKhGv7qAN7zC+Kl9iOpJomkJzvSNVVC1blaQBEgrJnF0
qFVilw7md40xdNwyi+SBE6BWKToaW+FUbBoq8xyn24vzOBn7LCEazSbLkjvvJI9BopQzJimt
VraIg6e1gxH6XAqmDjJfjkZihU48Lqe4kfLc1shxih0AFMtlsAiaXmHVOuGG9bCm0NZTBGG5
XiOGZPuyoRSX1vy4lFTyA2QvckCp8qag780EFIDeITAOgoWDNiD0L9sjQr04hspvlh7jFQeC
zaygt4hAp7zUGEYrGJkb/4zuGECzWBifzcKXblLd9+g06eUce/5x5E0p+DChhD4Xfp7V9o2l
IW6AiHKK+FU0s7wmt63TcvY38KFX2QsXVOOna3xLRxkN1vR2r3bFlS1pgk2Cx1RCjgM2KLJ4
vRwtSrTjj8+bC3A3TSMkcFeO7wAl5MbP569H7QABkFHX3CQAN4i4FbHEZ/sCAAAAAARZWg==

--veXX9dWIonWZEC6h--

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Will Deacon <will.deacon@arm.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Mon, 7 Nov 2016 21:05:14 +0000
Message-ID: <20161107210514.GP20591@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Return-path: <linux-kernel-owner@vger.kernel.org>
Content-Disposition: inline
In-Reply-To: <20161028091905.GM22012@rric.localdomain>
Sender: linux-kernel-owner@vger.kernel.org
To: Robert Richter <robert.richter@cavium.com>
Cc: Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Catalin Marinas <catalin.marinas@arm.com>, David Daney <david.daney@cavium.com>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
List-Id: linux-efi@vger.kernel.org

On Fri, Oct 28, 2016 at 11:19:05AM +0200, Robert Richter wrote:
> On 27.10.16 17:01:36, Will Deacon wrote:
> > It feels to me like NOMAP memory is a new type
> > of memory where there *is* a struct page, but it shouldn't be used for
> > anything.
> 
> IMO, a NOMAP page should just be handled like a reserved page except
> that the page is marked reserved. See free_low_memory_core_early().
> Thus, NOMAP pages are not in the free pages list or set to reserved.
> It is simply not available for mapping at all. Isn't that exactly what
> it should be?
> 
> I also did not yet understand the benefit of the differentiation
> between NOMAP and reserved and the original motivation for its
> implementation. I looked through the mail threads but could not find
> any hint. The only difference I see now is that it is not listed as a
> reserved page, but as long as it is not freed it should behave the
> same. I remember the case to handle memory different (coherency,
> etc.), but are not sure here. Ard, could you explain this?
> 
> > I don't think pfn_valid can describe that, given the way it's
> > currently used, and flipping the logic is just likely to move the problem
> > elsewhere.
> > 
> > What options do we have for fixing this in the NUMA code?
> 
> Out of my mind:
> 
> 1) Treat NOMAP pages same as reserved pages (my patch).

Just to reiterate here, but your patch as it stands will break other parts
of the kernel. For example, acpi_os_ioremap relies on being able to ioremap
these regions afaict.

I think any solution involving pfn_valid is just going to move the crash
around.

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Wed, 9 Nov 2016 20:51:32 +0100
Message-ID: <20161109195132.GZ22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Return-path: <linux-kernel-owner@vger.kernel.org>
Content-Disposition: inline
In-Reply-To: <20161107210514.GP20591@arm.com>
Sender: linux-kernel-owner@vger.kernel.org
To: Will Deacon <will.deacon@arm.com>
Cc: Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Catalin Marinas <catalin.marinas@arm.com>, David Daney <david.daney@cavium.com>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
List-Id: linux-efi@vger.kernel.org

Will,

On 07.11.16 21:05:14, Will Deacon wrote:
> Just to reiterate here, but your patch as it stands will break other parts
> of the kernel. For example, acpi_os_ioremap relies on being able to ioremap
> these regions afaict.
> 
> I think any solution involving pfn_valid is just going to move the crash
> around.

Let me describe the crash more detailed.

Following range is marked nomap (full efi map below):

[    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*

The mem belongs to this nodes:

[    0.000000] NUMA: Adding memblock [0x1400000 - 0xfffffffff] on node 0
[    0.000000] NUMA: Adding memblock [0x10000400000 - 0x10fffffffff] on node 1

The following mem ranges are created:

[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000001400000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x0000010fffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000001400000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x0000000fffffffff]
[    0.000000]   node   1: [mem 0x0000010000400000-0x0000010ff9e7ffff]
[    0.000000]   node   1: [mem 0x0000010ff9e80000-0x0000010ff9f1ffff]
[    0.000000]   node   1: [mem 0x0000010ff9f20000-0x0000010ffaeaffff]
[    0.000000]   node   1: [mem 0x0000010ffaeb0000-0x0000010ffaffffff]
[    0.000000]   node   1: [mem 0x0000010ffb000000-0x0000010ffffaffff]
[    0.000000]   node   1: [mem 0x0000010ffffb0000-0x0000010fffffffff]

The last range 0x0000010ffffb0000-0x0000010fffffffff is correctly
marked as a nomap area.

Paging is then initalized in free_area_init_core() (mm/page_alloc.c)
which calls memmap_init_zone(). This initializes all pages (struct
page) of the zones for each node except for pfns from nomap memory. It
uses early_pfn_valid() which is bound to pfn_valid().

In 4.4 *all* pages of a zone were initialized. Now page from nomap
ranges are skipped, and e.g. pfn 0x10ffffb has an uninitalized struct
page.  Note that nomap mem ranges are still part of the memblock list
and also part of the zone ranges within start_pfn and end_pfn, but
those don't have a valid struct page now. IMO this is a bug.

Later on all mappable memory is reserved and all pages of it are freed
by adding them to the free-pages list except for the reserved
memblocks.

Now the initrd is loaded. This reserves free memory by calling
move_freepages_block(). A block has the size of pageblock_nr_pages
which is in my configuration 0x2000. The kernel choses the block

 0x0000010fe0000000-0x0000010fffffffff

that also contains the nomap region around 10ffb000000.

In move_freepages() the pages of the zone are freed. The code accesses
pfn #10fffff but the struct page is uninitialized and thus node is 0.
This is different to #10fe000 and the zone which is node 1. This
causes the BUG_ON() to trigger:

[    4.173521] Unpacking initramfs...
[    8.420710] ------------[ cut here ]------------
[    8.425344] kernel BUG at mm/page_alloc.c:1844!
[    8.429870] Internal error: Oops - BUG: 0 [#1] SMP

I believe this is not the only case where the mm code relies on a
valid struct page for all pfns of a zone, thus modifying mm code to
make this case work is not an option. (E.g. this could be done by
moving the early_pfn_valid() check at the beginning of the loop in
memmap_init_zone().)

So for a fix we need to change early_pfn_valid() which is mapped to
pfn_valid() for SPARSEMEM. Using a different function than pfn_valid()
for this does not look reasonable. The only option is to revert
pfn_valid() to it's original bahaviour.

My fix now uses again memblock_is_memory() for pfn_valid() instead of
memblock_is_map_memory(). So I needed to change some users of
pfn_valid() to use memblock_is_map_memory() there necessary. This is
only required for arch specific code, all other uses of pfn_valid()
are save to use memblock_is_memory().

Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
keeps the same behaviour as before since it still uses memblock_is_
memory(). Could you more describe your concerns why do you think this
patch breaks the kernel and moves the problem somewhere else? I
believe it fixes the problem at all.

Thanks,

-Robert



[    0.000000] efi: Processing EFI memory map:
[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x1ffe800000 reserved size = 0x10537
[    0.000000]  memory.cnt  = 0x2
[    0.000000]  memory[0x0]     [0x00000001400000-0x00000fffffffff], 0xffec00000 bytes flags: 0x0
[    0.000000]  memory[0x1]     [0x00010000400000-0x00010fffffffff], 0xfffc00000 bytes flags: 0x0
[    0.000000]  reserved.cnt  = 0x1
[    0.000000]  reserved[0x0]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
[    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000001400000-0x0000000147ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000001480000-0x0000024affff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000001480000-0x000000024affff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000024b0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x000000024b0000-0x000000211fffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000021200000-0x0000002121ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000021220000-0x000000fffeffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x000000fffe0000-0x000000ffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x000000ffff0000-0x000000ffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x000000ffff0000-0x000000ffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000100000000-0x00000ff7ffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000ff8000000-0x00000ff801ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000ff8020000-0x00000fffa9ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000fffa90000-0x00000fffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010000400000-0x010f812b3fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010000400000-0x00010f812bffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f812b4000-0x010f812b6fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f812b0000-0x00010f812bffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f812b7000-0x010f822e6fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f812b0000-0x00010f822effff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f822e7000-0x010f822f6fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f822e0000-0x00010f822fffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f822f7000-0x010f82342fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f822f0000-0x00010f8234ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f82343000-0x010f91e73fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f82340000-0x00010f91e7ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f91e74000-0x010f91e74fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f91e70000-0x00010f91e7ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f91e75000-0x010f92c98fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f91e70000-0x00010f92c9ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f92c99000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f92c90000-0x00010f9388ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f93880000-0x00010ff788ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff7880000-0x00010ff788ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff7880000-0x00010ff78affff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff78a0000-0x00010ff9e8ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ff9e80000-0x00010ff9f1ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff9f10000-0x00010ffaebffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffaeb0000-0x00010ffafcffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffafc0000-0x00010ffafcffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffafc0000-0x00010ffaffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ffaff0000-0x00010ffb00ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ffb000000-0x00010fffe2ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffe20000-0x00010fffe3ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffe40000-0x00010fffe5ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffe50000-0x00010ffffbffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffffb0000-0x00010ffffcffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffffc0000-0x00010fffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffff0000-0x00010fffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
[    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
[    0.000000] memblock_reserve: [0x00010f812b0000-0x00010f812bffff] flags 0x0 efi_init+0x208/0x2c8
[    0.000000] memblock_add: [0x00010f82340000-0x00010f91e7ffff] flags 0x0 arm64_memblock_init+0x16c/0x248
[    0.000000] memblock_reserve: [0x00010f82340000-0x00010f91e7ffff] flags 0x0 arm64_memblock_init+0x178/0x248
[    0.000000] memblock_reserve: [0x00000001480000-0x000000024affff] flags 0x0 arm64_memblock_init+0x1b0/0x248
[    0.000000] memblock_reserve: [0x00010f82343000-0x00010f91e73613] flags 0x0 arm64_memblock_init+0x1cc/0x248
[    0.000000] memblock_reserve: [0x000000c0000000-0x000000dfffffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] cma: Reserved 512 MiB at 0x00000000c0000000
[    0.000000] memblock_reserve: [0x00010ffffa0000-0x00010ffffaffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff90000-0x00010ffff9ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff80000-0x00010ffff8ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff70000-0x00010ffff7ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff60000-0x00010ffff6ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff50000-0x00010ffff5ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff40000-0x00010ffff4ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff30000-0x00010ffff3ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000]    memblock_free: [0x00010ffffa0000-0x00010ffffaffff] paging_init+0x5c0/0x610
[    0.000000]    memblock_free: [0x00000002490000-0x000000024affff] paging_init+0x5f4/0x610
[    0.000000] memblock_reserve: [0x00010fffefed80-0x00010ffff2fffb] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffffaffd0-0x00010ffffafffe] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffffaffa0-0x00010ffffaffce] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffffa0000-0x00010ffffa000f] flags 0x0 numa_init+0x88/0x3f0
[    0.000000] NUMA: Adding memblock [0x1400000 - 0xfffffffff] on node 0
[    0.000000] NUMA: Adding memblock [0x10000400000 - 0x10fffffffff] on node 1
[    0.000000] NUMA: parsing numa-distance-map-v1
[    0.000000] NUMA: Initmem setup node 0 [mem 0x01400000-0xfffffffff]
[    0.000000] memblock_reserve: [0x00000fffffe500-0x00000fffffffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] NUMA: NODE_DATA [mem 0xfffffe500-0xfffffffff]
[    0.000000] NUMA: Initmem setup node 1 [mem 0x10000400000-0x10fffffffff]
[    0.000000] memblock_reserve: [0x00010ffffae480-0x00010ffffaff7f] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] NUMA: NODE_DATA [mem 0x10ffffae480-0x10ffffaff7f]
...
[    8.420710] ------------[ cut here ]------------
[    8.425344] kernel BUG at mm/page_alloc.c:1844!
[    8.429870] Internal error: Oops - BUG: 0 [#1] SMP
[    8.434654] Modules linked in:
[    8.437712] CPU: 72 PID: 1 Comm: swapper/0 Tainted: G        W       4.8.1.4.vanilla10-00007-g9eb3a76b8b88 #111
[    8.447788] Hardware name: www.cavium.com ThunderX CRB-2S/ThunderX CRB-2S, BIOS 0.3 Sep 13 2016
[    8.456474] task: ffff800fee626800 task.stack: ffff800fec02c000
[    8.462404] PC is at move_freepages+0x198/0x1b0
[    8.466924] LR is at move_freepages_block+0xa8/0xb8
[    8.471791] pc : [<ffff0000081e5dd0>] lr : [<ffff0000081e5e90>] pstate: 000000c5
[    8.479174] sp : ffff800fec02f4d0
[    8.482478] x29: ffff800fec02f4d0 x28: 0000000000000001 
[    8.487786] x27: 0000000000000000 x26: 0000000000000000 
[    8.493093] x25: 000000000000000a x24: ffff7fe043fd0020 
[    8.498401] x23: ffff810ffffaec00 x22: 0000000000000000 
[    8.503708] x21: ffff7fe043ffffc0 x20: 0000000000000000 
[    8.509015] x19: ffff7fe043f80000 x18: ffff8100102f2548 
[    8.514322] x17: 0000000000000000 x16: 0000000100000000 
[    8.519630] x15: 0000000000000007 x14: 0000010200000000 
[    8.524938] x13: 0004a41400000000 x12: 00032c920000001a 
[    8.530245] x11: 0000010200000000 x10: 0004a40800000000 
[    8.535553] x9 : 0000000000000040 x8 : 0000000000000000 
[    8.540860] x7 : 0000000000000000 x6 : ffff810ffffaf120 
[    8.546168] x5 : 0000000000000001 x4 : ffff000008d70c48 
[    8.551475] x3 : ffff810ffffae480 x2 : 0000000000000000 
[    8.556782] x1 : 0000000000000001 x0 : ffff810ffffaec00 
[    8.562089] 
[    8.563571] Process swapper/0 (pid: 1, stack limit = 0xffff800fec02c020)
[    8.570261] Stack: (0xffff800fec02f4d0 to 0xffff800fec030000)
...
[    9.279652] Call trace:
[    9.282091] Exception stack(0xffff800fec02f300 to 0xffff800fec02f430)
[    9.288520] f300: ffff7fe043f80000 0001000000000000 ffff800fec02f4d0 ffff0000081e5dd0
[    9.296339] f320: 0000000000000001 ffff800fee626800 0000000000000000 00000000026012d0
[    9.304157] f340: 0000000100000000 ffffffffffffffff 0000000000000130 0000000000000040
[    9.311976] f360: 0000001200000000 ffff000008cf8198 0000000100000000 0000000000000001
[    9.319794] f380: 00000000026012d0 ffff810ff9a1e8e8 0000000000000000 0000000000000040
[    9.327613] f3a0: ffff810ffffaec00 0000000000000001 0000000000000000 ffff810ffffae480
[    9.335431] f3c0: ffff000008d70c48 0000000000000001 ffff810ffffaf120 0000000000000000
[    9.343250] f3e0: 0000000000000000 0000000000000040 0004a40800000000 0000010200000000
[    9.351068] f400: 00032c920000001a 0004a41400000000 0000010200000000 0000000000000007
[    9.358885] f420: 0000000100000000 0000000000000000
[    9.363756] [<ffff0000081e5dd0>] move_freepages+0x198/0x1b0
[    9.369318] [<ffff0000081e5e90>] move_freepages_block+0xa8/0xb8
[    9.375227] [<ffff0000081e657c>] __rmqueue+0x604/0x650
[    9.380355] [<ffff0000081e7898>] get_page_from_freelist+0x3f0/0xb88
[    9.386612] [<ffff0000081e860c>] __alloc_pages_nodemask+0x12c/0xce8
[    9.392876] [<ffff00000823dc1c>] alloc_page_interleave+0x64/0xc0
[    9.398873] [<ffff00000823e2f8>] alloc_pages_current+0x108/0x168
[    9.404870] [<ffff0000081de1cc>] __page_cache_alloc+0x104/0x140
[    9.410779] [<ffff0000081de31c>] pagecache_get_page+0x114/0x300
[    9.416688] [<ffff0000081de550>] grab_cache_page_write_begin+0x48/0x68
[    9.423207] [<ffff00000828e5f8>] simple_write_begin+0x40/0x150
[    9.429029] [<ffff0000081ddfe0>] generic_perform_write+0xb8/0x1a0
[    9.435112] [<ffff0000081df8c8>] __generic_file_write_iter+0x170/0x1b0
[    9.441628] [<ffff0000081df9d4>] generic_file_write_iter+0xcc/0x1c8
[    9.447893] [<ffff000008262eb4>] __vfs_write+0xcc/0x140
[    9.453108] [<ffff000008263b84>] vfs_write+0xa4/0x1c0
[    9.458150] [<ffff000008264bc4>] SyS_write+0x54/0xb0
[    9.463108] [<ffff000008be1fe4>] xwrite+0x34/0x7c
[    9.467801] [<ffff000008be20c8>] do_copy+0x9c/0xf4
[    9.472583] [<ffff000008be1da4>] write_buffer+0x34/0x50
[    9.477796] [<ffff000008be1e08>] flush_buffer+0x48/0xb4
[    9.483016] [<ffff000008c0fe08>] __gunzip+0x288/0x334
[    9.488057] [<ffff000008c0fecc>] gunzip+0x18/0x20
[    9.492750] [<ffff000008be26bc>] unpack_to_rootfs+0x168/0x284
[    9.498486] [<ffff000008be2848>] populate_rootfs+0x70/0x138
[    9.504051] [<ffff000008082ff4>] do_one_initcall+0x44/0x138
[    9.509613] [<ffff000008be0d04>] kernel_init_freeable+0x1ac/0x24c
[    9.515699] [<ffff000008847f20>] kernel_init+0x20/0xf8
[    9.520826] [<ffff000008082b80>] ret_from_fork+0x10/0x50
[    9.526129] Code: 910a0021 9400b47d d4210000 d503201f (d4210000) 
[    9.532233] ---[ end trace c3040dccdcf12d3a ]---
[    9.536889] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 17 Nov 2016 14:25:29 +0000
Message-ID: <20161117142528.GJ22855@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <20161109195132.GZ22012-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
> Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
> keeps the same behaviour as before since it still uses memblock_is_
> memory(). Could you more describe your concerns why do you think this
> patch breaks the kernel and moves the problem somewhere else? I
> believe it fixes the problem at all.

acpi_os_ioremap always ends up in __ioremap_caller, regardless of
memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 17 Nov 2016 16:18:05 +0100
Message-ID: <20161117151805.GJ2151@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
Content-Disposition: inline
In-Reply-To: <20161117142528.GJ22855@arm.com>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, Robert Richter <rric@kernel.org>, David Daney <david.daney@cavium.com>, Catalin Marinas <catalin.marinas@arm.com>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
List-Id: linux-efi@vger.kernel.org

Thanks for your answer.

On 17.11.16 14:25:29, Will Deacon wrote:
> On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
> > Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
> > keeps the same behaviour as before since it still uses memblock_is_
> > memory(). Could you more describe your concerns why do you think this
> > patch breaks the kernel and moves the problem somewhere else? I
> > believe it fixes the problem at all.
> 
> acpi_os_ioremap always ends up in __ioremap_caller, regardless of
> memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.

But that's the reason my patch changed the code to use memblock_is_
map_memory() instead. I was looking into the users of pfn_valid() esp.
in arm64 code and changed it where required.

This week I looked into the kernel again for code that might break by
a pfn_valid() change. I found try_ram_remap() in memremap.c that has
changed behaviour now, but this is explicit for MEMREMAP_WB, so it
should be fine.

Maybe it might be better to use page_is_ram() in addition to
pfn_valid() where necessary. This should work now after commit:

 e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem

I still think pfn_valid() is not the correct use to determine the mem
attributes for mappings, there are further checks required.

The risk of breaking something with my patch is small and limited only
to the mapping of efi reserved regions (which is the state of 4.4). If
something breaks anyway it can easily be fixed by adding more checks
to pfn_valid() as suggested above.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Sun, 20 Nov 2016 17:07:44 +0000
Message-ID: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain> <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com> <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com> <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com> <20161117151805.GJ2151@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
In-Reply-To: <20161117151805.GJ2151-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On 17 November 2016 at 15:18, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> Thanks for your answer.
>
> On 17.11.16 14:25:29, Will Deacon wrote:
>> On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
>> > Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
>> > keeps the same behaviour as before since it still uses memblock_is_
>> > memory(). Could you more describe your concerns why do you think this
>> > patch breaks the kernel and moves the problem somewhere else? I
>> > believe it fixes the problem at all.
>>
>> acpi_os_ioremap always ends up in __ioremap_caller, regardless of
>> memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.
>
> But that's the reason my patch changed the code to use memblock_is_
> map_memory() instead. I was looking into the users of pfn_valid() esp.
> in arm64 code and changed it where required.
>
> This week I looked into the kernel again for code that might break by
> a pfn_valid() change. I found try_ram_remap() in memremap.c that has
> changed behaviour now, but this is explicit for MEMREMAP_WB, so it
> should be fine.
>
> Maybe it might be better to use page_is_ram() in addition to
> pfn_valid() where necessary. This should work now after commit:
>
>  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>
> I still think pfn_valid() is not the correct use to determine the mem
> attributes for mappings, there are further checks required.
>
> The risk of breaking something with my patch is small and limited only
> to the mapping of efi reserved regions (which is the state of 4.4). If
> something breaks anyway it can easily be fixed by adding more checks
> to pfn_valid() as suggested above.
>

As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
the correct way to address this. However, doing that does uncover a
bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
page fields before the pfn_valid_within() check, so it seems those
need to be switched around.

Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
for sparsemem. Care to elaborate why?

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Wed, 23 Nov 2016 22:15:38 +0100
Message-ID: <20161123211538.GH10776@rric.localdomain>
References: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
Content-Disposition: inline
In-Reply-To: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, Robert Richter <rric@kernel.org>, David Daney <david.daney@cavium.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
List-Id: linux-efi@vger.kernel.org

On 20.11.16 17:07:44, Ard Biesheuvel wrote:
> On 17 November 2016 at 15:18, Robert Richter <robert.richter@cavium.com> wrote:

> > The risk of breaking something with my patch is small and limited only
> > to the mapping of efi reserved regions (which is the state of 4.4). If
> > something breaks anyway it can easily be fixed by adding more checks
> > to pfn_valid() as suggested above.
> >
> 
> As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
> the correct way to address this. However, doing that does uncover a
> bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
> page fields before the pfn_valid_within() check, so it seems those
> need to be switched around.
> 
> Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
> for sparsemem. Care to elaborate why?

HOLES_IN_ZONE is of rare use in the kernel. I think it was introduced
to save memory for the memmap and only some single systems enable it.
There is no architecture that enables it entirely. For good reasons...

It introduces additional checks. pfn_valid() is usually checked only
once for the whole memmap. There are a number of checks enabled, just
grep for pfn_valid_within. This will increase the number of
pfn_valid() calls by a factor of MAX_ORDER_NR_PAGES, in my config this
is 8k. So, this is not the direction to go.

My patch fixes a regression in the kernel that was introduced by the
nomap implementation. Some systems can not boot anymore, beside of
that the BUG_ON() may occur any time depending only on physical page
access, we need to fix 4.9. Here is a reproducer:

 https://patchwork.kernel.org/patch/9407677/

My patch also does not break memremap(). With my patch applied
try_ram_remap() would return a linear addr for nomap regions. But this
is only called if WB is explicitly requested, so it should not happen.
If you think pfn_valid() is wrong here, I am happy to send a patch
that fixes this by using page_is_ram(). In any case, the worst case
that may happen is to behave the same as v4.4, we might fix then the
wrong use of pfn_valid() where it is not correctly used to check for
ram.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Wed, 23 Nov 2016 21:25:06 +0000
Message-ID: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
References: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain> <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com> <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com> <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com> <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com> <20161123211538.GH10776@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
In-Reply-To: <20161123211538.GH10776-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On 23 November 2016 at 21:15, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> On 20.11.16 17:07:44, Ard Biesheuvel wrote:
>> On 17 November 2016 at 15:18, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
>
>> > The risk of breaking something with my patch is small and limited only
>> > to the mapping of efi reserved regions (which is the state of 4.4). If
>> > something breaks anyway it can easily be fixed by adding more checks
>> > to pfn_valid() as suggested above.
>> >
>>
>> As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
>> the correct way to address this. However, doing that does uncover a
>> bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
>> page fields before the pfn_valid_within() check, so it seems those
>> need to be switched around.
>>
>> Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
>> for sparsemem. Care to elaborate why?
>
> HOLES_IN_ZONE is of rare use in the kernel. I think it was introduced
> to save memory for the memmap and only some single systems enable it.
> There is no architecture that enables it entirely. For good reasons...
>
> It introduces additional checks. pfn_valid() is usually checked only
> once for the whole memmap. There are a number of checks enabled, just
> grep for pfn_valid_within. This will increase the number of
> pfn_valid() calls by a factor of MAX_ORDER_NR_PAGES, in my config this
> is 8k. So, this is not the direction to go.
>

That does sound like a potential issue. But does it cause any slowdown
in practice?

The reality is that CONFIG_HOLES_IN_ZONE perfectly describes the
situation, and so it is still my preferred option if the performance
hit is tolerable.

> My patch fixes a regression in the kernel that was introduced by the
> nomap implementation. Some systems can not boot anymore, beside of
> that the BUG_ON() may occur any time depending only on physical page
> access, we need to fix 4.9. Here is a reproducer:
>
>  https://patchwork.kernel.org/patch/9407677/
>
> My patch also does not break memremap(). With my patch applied
> try_ram_remap() would return a linear addr for nomap regions. But this
> is only called if WB is explicitly requested, so it should not happen.

Why? MEMREMAP_WB is used often, among other things for mapping
firmware tables, which are marked as NOMAP, so in these cases, the
linear address is not mapped.

> If you think pfn_valid() is wrong here, I am happy to send a patch
> that fixes this by using page_is_ram(). In any case, the worst case
> that may happen is to behave the same as v4.4, we might fix then the
> wrong use of pfn_valid() where it is not correctly used to check for
> ram.
>

page_is_ram() uses string comparisons to look for regions called
'System RAM'. Is that something we can tolerate for each pfn_valid()
calll?

Perhaps the solution is to reimplement page_is_ram() for arm64 using
memblock_is_memory() instead, But that still means we need to modify
the generic memremap() code first to switch to it before changing the
arm64 implementation of pfn_valid

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Thu, 24 Nov 2016 13:44:31 +0000
Message-ID: <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
References: <20161017185801.GT25086@rric.localdomain> <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain> <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain> <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain> <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain> <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
In-Reply-To: <20161124134238.GI10776-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On 24 November 2016 at 13:42, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> Why? MEMREMAP_WB is used often, among other things for mapping
>> firmware tables, which are marked as NOMAP, so in these cases, the
>> linear address is not mapped.
>
> If fw tables are mapped wb, that is wrong and needs a separate fix.
>

Why is that wrong?

>> > If you think pfn_valid() is wrong here, I am happy to send a patch
>> > that fixes this by using page_is_ram(). In any case, the worst case
>> > that may happen is to behave the same as v4.4, we might fix then the
>> > wrong use of pfn_valid() where it is not correctly used to check for
>> > ram.
>> >
>>
>> page_is_ram() uses string comparisons to look for regions called
>> 'System RAM'. Is that something we can tolerate for each pfn_valid()
>> calll?
>>
>> Perhaps the solution is to reimplement page_is_ram() for arm64 using
>> memblock_is_memory() instead, But that still means we need to modify
>> the generic memremap() code first to switch to it before changing the
>> arm64 implementation of pfn_valid
>
> No, that's not the solution. pfn_valid() should just check if there is
> a valid struct page, as other archs do. And if there is a mis-use of
> pfn_valid() to check for ram, only that calls should be fixed to use
> page_is_ram(), however this is implemented, or something appropriate.
> But I don't see any problematic code, and if so, I will fix that.
>

memremap() uses pfn_valid() to decide whether some address is covered
by the linear mapping. If we correct pfn_valid() to adhere to your
definition, we will need to fix memremap() first in any case.

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 24 Nov 2016 14:42:38 +0100
Message-ID: <20161124134238.GI10776@rric.localdomain>
References: <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
Content-Disposition: inline
In-Reply-To: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, Robert Richter <rric@kernel.org>, David Daney <david.daney@cavium.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
List-Id: linux-efi@vger.kernel.org

On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> Why? MEMREMAP_WB is used often, among other things for mapping
> firmware tables, which are marked as NOMAP, so in these cases, the
> linear address is not mapped.

If fw tables are mapped wb, that is wrong and needs a separate fix.

> > If you think pfn_valid() is wrong here, I am happy to send a patch
> > that fixes this by using page_is_ram(). In any case, the worst case
> > that may happen is to behave the same as v4.4, we might fix then the
> > wrong use of pfn_valid() where it is not correctly used to check for
> > ram.
> >
> 
> page_is_ram() uses string comparisons to look for regions called
> 'System RAM'. Is that something we can tolerate for each pfn_valid()
> calll?
> 
> Perhaps the solution is to reimplement page_is_ram() for arm64 using
> memblock_is_memory() instead, But that still means we need to modify
> the generic memremap() code first to switch to it before changing the
> arm64 implementation of pfn_valid

No, that's not the solution. pfn_valid() should just check if there is
a valid struct page, as other archs do. And if there is a mis-use of
pfn_valid() to check for ram, only that calls should be fixed to use
page_is_ram(), however this is implemented, or something appropriate.
But I don't see any problematic code, and if so, I will fix that.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 24 Nov 2016 14:51:51 +0100
Message-ID: <20161124135151.GJ10776@rric.localdomain>
References: <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Return-path: <linux-kernel-owner@vger.kernel.org>
Content-Disposition: inline
In-Reply-To: <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
Sender: linux-kernel-owner@vger.kernel.org
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, David Daney <david.daney@cavium.com>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
List-Id: linux-efi@vger.kernel.org

On 24.11.16 13:44:31, Ard Biesheuvel wrote:
> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> >> Why? MEMREMAP_WB is used often, among other things for mapping
> >> firmware tables, which are marked as NOMAP, so in these cases, the
> >> linear address is not mapped.
> >
> > If fw tables are mapped wb, that is wrong and needs a separate fix.
> >
> 
> Why is that wrong?

The whole issue with mapping acpi tables is not marking them cachable,
what wb does. Otherwise we could just use linear mapping for those mem
ranges.

> >> > If you think pfn_valid() is wrong here, I am happy to send a patch
> >> > that fixes this by using page_is_ram(). In any case, the worst case
> >> > that may happen is to behave the same as v4.4, we might fix then the
> >> > wrong use of pfn_valid() where it is not correctly used to check for
> >> > ram.
> >> >
> >>
> >> page_is_ram() uses string comparisons to look for regions called
> >> 'System RAM'. Is that something we can tolerate for each pfn_valid()
> >> calll?
> >>
> >> Perhaps the solution is to reimplement page_is_ram() for arm64 using
> >> memblock_is_memory() instead, But that still means we need to modify
> >> the generic memremap() code first to switch to it before changing the
> >> arm64 implementation of pfn_valid
> >
> > No, that's not the solution. pfn_valid() should just check if there is
> > a valid struct page, as other archs do. And if there is a mis-use of
> > pfn_valid() to check for ram, only that calls should be fixed to use
> > page_is_ram(), however this is implemented, or something appropriate.
> > But I don't see any problematic code, and if so, I will fix that.
> >
> 
> memremap() uses pfn_valid() to decide whether some address is covered
> by the linear mapping. If we correct pfn_valid() to adhere to your
> definition, we will need to fix memremap() first in any case.

As said, will fix that if needed. But I think the caller is wrong
then.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Thu, 24 Nov 2016 13:58:30 +0000
Message-ID: <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
References: <20161028091905.GM22012@rric.localdomain> <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain> <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain> <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain> <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain> <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
In-Reply-To: <20161124135151.GJ10776-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On 24 November 2016 at 13:51, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>> On 24 November 2016 at 13:42, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
>> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> >> Why? MEMREMAP_WB is used often, among other things for mapping
>> >> firmware tables, which are marked as NOMAP, so in these cases, the
>> >> linear address is not mapped.
>> >
>> > If fw tables are mapped wb, that is wrong and needs a separate fix.
>> >
>>
>> Why is that wrong?
>
> The whole issue with mapping acpi tables is not marking them cachable,
> what wb does.

What 'issue'?

> Otherwise we could just use linear mapping for those mem
> ranges.
>

Regions containing firmware tables are owned by the firmware, and it
is the firmware that tells us which memory attributes we are allowed
to use. If those attributes include WB, it is perfectly legal to use a
cacheable mapping. That does *not* mean they should be covered by the
linear mapping. The linear mapping is read-write-non-exec, for
instance, and we may prefer to use a read-only mapping and/or
executable mapping.

>> >> > If you think pfn_valid() is wrong here, I am happy to send a patch
>> >> > that fixes this by using page_is_ram(). In any case, the worst case
>> >> > that may happen is to behave the same as v4.4, we might fix then the
>> >> > wrong use of pfn_valid() where it is not correctly used to check for
>> >> > ram.
>> >> >
>> >>
>> >> page_is_ram() uses string comparisons to look for regions called
>> >> 'System RAM'. Is that something we can tolerate for each pfn_valid()
>> >> calll?
>> >>
>> >> Perhaps the solution is to reimplement page_is_ram() for arm64 using
>> >> memblock_is_memory() instead, But that still means we need to modify
>> >> the generic memremap() code first to switch to it before changing the
>> >> arm64 implementation of pfn_valid
>> >
>> > No, that's not the solution. pfn_valid() should just check if there is
>> > a valid struct page, as other archs do. And if there is a mis-use of
>> > pfn_valid() to check for ram, only that calls should be fixed to use
>> > page_is_ram(), however this is implemented, or something appropriate.
>> > But I don't see any problematic code, and if so, I will fix that.
>> >
>>
>> memremap() uses pfn_valid() to decide whether some address is covered
>> by the linear mapping. If we correct pfn_valid() to adhere to your
>> definition, we will need to fix memremap() first in any case.
>
> As said, will fix that if needed. But I think the caller is wrong
> then.
>

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 24 Nov 2016 15:11:49 +0100
Message-ID: <20161124141149.GE2213@rric.localdomain>
References: <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
Content-Disposition: inline
In-Reply-To: <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, Robert Richter <rric@kernel.org>, David Daney <david.daney@cavium.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
List-Id: linux-efi@vger.kernel.org

On 24.11.16 13:58:30, Ard Biesheuvel wrote:
> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
> >> >> linear address is not mapped.
> >> >
> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
> >> >
> >>
> >> Why is that wrong?
> >
> > The whole issue with mapping acpi tables is not marking them cachable,
> > what wb does.
> 
> What 'issue'?
> 
> > Otherwise we could just use linear mapping for those mem
> > ranges.
> >
> 
> Regions containing firmware tables are owned by the firmware, and it
> is the firmware that tells us which memory attributes we are allowed
> to use. If those attributes include WB, it is perfectly legal to use a
> cacheable mapping. That does *not* mean they should be covered by the
> linear mapping. The linear mapping is read-write-non-exec, for
> instance, and we may prefer to use a read-only mapping and/or
> executable mapping.

Ok, I am going to fix try_ram_remap().

Are there other concerns with this patch?

Thanks,

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 24 Nov 2016 14:23:16 +0000
Message-ID: <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
References: <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
In-Reply-To: <20161124141149.GE2213@rric.localdomain>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Robert Richter <robert.richter@cavium.com>
Cc: Mark Rutland <mark.rutland@arm.com>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, Robert Richter <rric@kernel.org>, David Daney <david.daney@cavium.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
List-Id: linux-efi@vger.kernel.org

On 24 November 2016 at 14:11, Robert Richter <robert.richter@cavium.com> wrote:
> On 24.11.16 13:58:30, Ard Biesheuvel wrote:
>> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
>> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
>> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
>> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
>> >> >> linear address is not mapped.
>> >> >
>> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
>> >> >
>> >>
>> >> Why is that wrong?
>> >
>> > The whole issue with mapping acpi tables is not marking them cachable,
>> > what wb does.
>>
>> What 'issue'?
>>
>> > Otherwise we could just use linear mapping for those mem
>> > ranges.
>> >
>>
>> Regions containing firmware tables are owned by the firmware, and it
>> is the firmware that tells us which memory attributes we are allowed
>> to use. If those attributes include WB, it is perfectly legal to use a
>> cacheable mapping. That does *not* mean they should be covered by the
>> linear mapping. The linear mapping is read-write-non-exec, for
>> instance, and we may prefer to use a read-only mapping and/or
>> executable mapping.
>
> Ok, I am going to fix try_ram_remap().
>

Thanks. Could you also add an arm64 version of page_is_ram() that uses
memblock_is_memory() while you're at it? I think using memblock
directly in try_ram_remap() may not be the best approach

> Are there other concerns with this patch?
>

I think we all agree that pfn_valid() should return whether a pfn has
a struct page associated with it, the debate is about whether it makes
sense to allocate struct pages for memory that the kernel does not
own. But given that it does not really hurt to do so for small holes,
I think your suggestion makes sense.

Should we be doing anything more to ensure that those pages are not
dereferenced inadvertently? Is there a page flag we should be setting?

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter@cavium.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 24 Nov 2016 16:09:18 +0100
Message-ID: <20161124150918.GF2213@rric.localdomain>
References: <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Return-path: <linux-kernel-owner@vger.kernel.org>
Content-Disposition: inline
In-Reply-To: <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
Sender: linux-kernel-owner@vger.kernel.org
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, David Daney <david.daney@cavium.com>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
List-Id: linux-efi@vger.kernel.org

On 24.11.16 14:23:16, Ard Biesheuvel wrote:
> On 24 November 2016 at 14:11, Robert Richter <robert.richter@cavium.com> wrote:
> > On 24.11.16 13:58:30, Ard Biesheuvel wrote:
> >> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> >> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
> >> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> >> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> >> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
> >> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
> >> >> >> linear address is not mapped.
> >> >> >
> >> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
> >> >> >
> >> >>
> >> >> Why is that wrong?
> >> >
> >> > The whole issue with mapping acpi tables is not marking them cachable,
> >> > what wb does.
> >>
> >> What 'issue'?
> >>
> >> > Otherwise we could just use linear mapping for those mem
> >> > ranges.
> >> >
> >>
> >> Regions containing firmware tables are owned by the firmware, and it
> >> is the firmware that tells us which memory attributes we are allowed
> >> to use. If those attributes include WB, it is perfectly legal to use a
> >> cacheable mapping. That does *not* mean they should be covered by the
> >> linear mapping. The linear mapping is read-write-non-exec, for
> >> instance, and we may prefer to use a read-only mapping and/or
> >> executable mapping.
> >
> > Ok, I am going to fix try_ram_remap().
> >
> 
> Thanks. Could you also add an arm64 version of page_is_ram() that uses
> memblock_is_memory() while you're at it? I think using memblock
> directly in try_ram_remap() may not be the best approach

Sure. I also want to mark the patches as stable.

> > Are there other concerns with this patch?
> >
> 
> I think we all agree that pfn_valid() should return whether a pfn has
> a struct page associated with it, the debate is about whether it makes
> sense to allocate struct pages for memory that the kernel does not
> own. But given that it does not really hurt to do so for small holes,
> I think your suggestion makes sense.

Thanks for your comments and the review.

> Should we be doing anything more to ensure that those pages are not
> dereferenced inadvertently? Is there a page flag we should be setting?

I don't think so. Boot mem is initialized in free_low_memory_core_
early(). The PageReserved flag is set for pages from reserved memory
ranges, and memory ranges not marked NOMAP is just freed. Since pages
are either reservered or in the free_list of pages, pages from other
memory ranges (NOMAP) is not visible to mm.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Thu, 24 Nov 2016 20:26:59 +0100
Message-ID: <20161124192659.GH2213@rric.localdomain>
References: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <20161124150918.GF2213-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

Ard,

> > >> On 24 November 2016 at 13:51, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> > >> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:

> > >> Regions containing firmware tables are owned by the firmware, and it
> > >> is the firmware that tells us which memory attributes we are allowed
> > >> to use. If those attributes include WB, it is perfectly legal to use a
> > >> cacheable mapping. That does *not* mean they should be covered by the
> > >> linear mapping. The linear mapping is read-write-non-exec, for
> > >> instance, and we may prefer to use a read-only mapping and/or
> > >> executable mapping.
> > >
> > > Ok, I am going to fix try_ram_remap().

I revisited the code and it is working well already since:

 e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem

Now, try_ram_remap() is only called if the region to be mapped is
entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
ranges and not NOMAP mem. region_intersects() then returns
REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
REGION_DISJOINT would be returned and thus arch_memremap_wb() being
called directly. Before the e7cd190385d1 change try_ram_remap() was
called also for nomap regions.

So we can leave memremap() as it is and just apply this patch
unmodified. What do you think? Please ack.

I am going to prepare the pfn_is_ram() change in addition to this
patch, but that should not be required for this fix to work correcly.

Thanks,

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Thu, 24 Nov 2016 19:42:47 +0000
Message-ID: <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
References: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain> <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain> <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain> <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain> <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain> <20161124192659.GH2213@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Return-path: <linux-kernel-owner@vger.kernel.org>
In-Reply-To: <20161124192659.GH2213@rric.localdomain>
Sender: linux-kernel-owner@vger.kernel.org
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, David Daney <david.daney@cavium.com>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
List-Id: linux-efi@vger.kernel.org

On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:
> Ard,
>
>> > >> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
>> > >> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>
>> > >> Regions containing firmware tables are owned by the firmware, and it
>> > >> is the firmware that tells us which memory attributes we are allowed
>> > >> to use. If those attributes include WB, it is perfectly legal to use a
>> > >> cacheable mapping. That does *not* mean they should be covered by the
>> > >> linear mapping. The linear mapping is read-write-non-exec, for
>> > >> instance, and we may prefer to use a read-only mapping and/or
>> > >> executable mapping.
>> > >
>> > > Ok, I am going to fix try_ram_remap().
>
> I revisited the code and it is working well already since:
>
>  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>
> Now, try_ram_remap() is only called if the region to be mapped is
> entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
> ranges and not NOMAP mem. region_intersects() then returns
> REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
> REGION_DISJOINT would be returned and thus arch_memremap_wb() being
> called directly. Before the e7cd190385d1 change try_ram_remap() was
> called also for nomap regions.
>
> So we can leave memremap() as it is and just apply this patch
> unmodified. What do you think?

I agree. The pfn_valid() check in try_ram_remap() is still appropriate
simply because the PageHighmem check requires a valid struct page. But
if we don't enter that code path anymore for NOMAP regions, I think
we're ok.

> Please ack.
>

I still don't fully understand how it is guaranteed that *all* memory
(i.e., all regions for which memblock_is_memory() returns true) is
covered by a struct page, but marked as reserved. Are we relying on
the fact that NOMAP memory is also memblock_reserve()'d?

> I am going to prepare the pfn_is_ram() change in addition to this
> patch, but that should not be required for this fix to work correcly.
>

I don't think you need to bother with page_is_ram() then. The only
place we use it is in devmem_is_allowed(), and there it makes sense to
allow NOMAP regions to be accessed (provided that you think /dev/mem
is a good idea in the first place).

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Fri, 25 Nov 2016 12:29:14 +0100
Message-ID: <20161125112914.GI2213@rric.localdomain>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
 <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Content-Disposition: inline
In-Reply-To: <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On 24.11.16 19:42:47, Ard Biesheuvel wrote:
> On 24 November 2016 at 19:26, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:

> > I revisited the code and it is working well already since:
> >
> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
> >
> > Now, try_ram_remap() is only called if the region to be mapped is
> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
> > ranges and not NOMAP mem. region_intersects() then returns
> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
> > called directly. Before the e7cd190385d1 change try_ram_remap() was
> > called also for nomap regions.
> >
> > So we can leave memremap() as it is and just apply this patch
> > unmodified. What do you think?
> 
> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
> simply because the PageHighmem check requires a valid struct page. But
> if we don't enter that code path anymore for NOMAP regions, I think
> we're ok.
> 
> > Please ack.
> >
> 
> I still don't fully understand how it is guaranteed that *all* memory
> (i.e., all regions for which memblock_is_memory() returns true) is
> covered by a struct page, but marked as reserved. Are we relying on
> the fact that NOMAP memory is also memblock_reserve()'d?

See free_low_memory_core_early():

----
	for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
        			NULL)
		count += __free_memory_core(start, end);
----

Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
also *not* marked reserved. So nothing at all from NOMAP mem is
reported to mm, it is not present (see below for a mem config, note
flags: 0x4 mem regions).

-Robert


[    0.000000] efi: Processing EFI memory map:
[    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000001480000-0x0000024bffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000024c0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010000400000-0x010f816aefff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f816af000-0x010f816b1fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f816b2000-0x010f826f1fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f826f2000-0x010f82701fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f82702000-0x010f82787fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f82788000-0x010f9276bfff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f9276c000-0x010f9276cfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f9276d000-0x010f935a8fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f935a9000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
[    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]

[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x1ffe800000 reserved size = 0x39146a21
[    0.000000]  memory.cnt  = 0x9
[    0.000000]  memory[0x0]     [0x00000001400000-0x000000fffdffff], 0xfebe0000 bytes on node 0 flags: 0x0
[    0.000000]  memory[0x1]     [0x000000fffe0000-0x000000ffffffff], 0x20000 bytes on node 0 flags: 0x4
[    0.000000]  memory[0x2]     [0x00000100000000-0x00000fffffffff], 0xf00000000 bytes on node 0 flags: 0x0
[    0.000000]  memory[0x3]     [0x00010000400000-0x00010ff9e7ffff], 0xff9a80000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x4]     [0x00010ff9e80000-0x00010ff9f1ffff], 0xa0000 bytes on node 1 flags: 0x4
[    0.000000]  memory[0x5]     [0x00010ff9f20000-0x00010ffaeaffff], 0xf90000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x6]     [0x00010ffaeb0000-0x00010ffaffffff], 0x150000 bytes on node 1 flags: 0x4
[    0.000000]  memory[0x7]     [0x00010ffb000000-0x00010ffffaffff], 0x4fb0000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x8]     [0x00010ffffb0000-0x00010fffffffff], 0x50000 bytes on node 1 flags: 0x4
[    0.000000]  reserved.cnt  = 0xd
[    0.000000]  reserved[0x0]   [0x00000001480000-0x0000000249ffff], 0x1020000 bytes flags: 0x0
[    0.000000]  reserved[0x1]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
[    0.000000]  reserved[0x2]   [0x000000c0000000-0x000000dfffffff], 0x20000000 bytes flags: 0x0
[    0.000000]  reserved[0x3]   [0x00000ffbfb8000-0x00000ffffdffff], 0x4028000 bytes flags: 0x0
[    0.000000]  reserved[0x4]   [0x00000ffffecb00-0x00000fffffffff], 0x13500 bytes flags: 0x0
[    0.000000]  reserved[0x5]   [0x00010f81780000-0x00010f8178ffff], 0x10000 bytes flags: 0x0
[    0.000000]  reserved[0x6]   [0x00010f82870000-0x00010f9286ffff], 0x10000000 bytes flags: 0x0
[    0.000000]  reserved[0x7]   [0x00010ffbce0000-0x00010fffceffff], 0x4010000 bytes flags: 0x0
[    0.000000]  reserved[0x8]   [0x00010fffee6d80-0x00010ffff2fffb], 0x4927c bytes flags: 0x0
[    0.000000]  reserved[0x9]   [0x00010ffff30000-0x00010ffffa000f], 0x70010 bytes flags: 0x0
[    0.000000]  reserved[0xa]   [0x00010ffffae280-0x00010ffffaff7f], 0x1d00 bytes flags: 0x0
[    0.000000]  reserved[0xb]   [0x00010ffffaffa0-0x00010ffffaffce], 0x2f bytes flags: 0x0
[    0.000000]  reserved[0xc]   [0x00010ffffaffd0-0x00010ffffafffe], 0x2f bytes flags: 0x0

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Fri, 25 Nov 2016 12:28:50 +0000
Message-ID: <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain> <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain> <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain> <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain> <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com> <20161125112914.GI2213@rric.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Return-path: <linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
In-Reply-To: <20161125112914.GI2213-vWBEXY7mpu582hYKe6nXyg@public.gmane.org>
Sender: linux-efi-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
To: Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>, Robert Richter <rric-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org>, Catalin Marinas <catalin.marinas-5wv7dgnIgG8@public.gmane.org>, David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>, Hanjun Guo <hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" <linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org>, "linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-efi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
List-Id: linux-efi@vger.kernel.org

On 25 November 2016 at 11:29, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
> On 24.11.16 19:42:47, Ard Biesheuvel wrote:
>> On 24 November 2016 at 19:26, Robert Richter <robert.richter-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org> wrote:
>
>> > I revisited the code and it is working well already since:
>> >
>> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>> >
>> > Now, try_ram_remap() is only called if the region to be mapped is
>> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
>> > ranges and not NOMAP mem. region_intersects() then returns
>> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
>> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
>> > called directly. Before the e7cd190385d1 change try_ram_remap() was
>> > called also for nomap regions.
>> >
>> > So we can leave memremap() as it is and just apply this patch
>> > unmodified. What do you think?
>>
>> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
>> simply because the PageHighmem check requires a valid struct page. But
>> if we don't enter that code path anymore for NOMAP regions, I think
>> we're ok.
>>
>> > Please ack.
>> >
>>
>> I still don't fully understand how it is guaranteed that *all* memory
>> (i.e., all regions for which memblock_is_memory() returns true) is
>> covered by a struct page, but marked as reserved. Are we relying on
>> the fact that NOMAP memory is also memblock_reserve()'d?
>
> See free_low_memory_core_early():
>
> ----
>         for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
>                                 NULL)
>                 count += __free_memory_core(start, end);
> ----
>
> Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
> also *not* marked reserved. So nothing at all from NOMAP mem is
> reported to mm, it is not present (see below for a mem config, note
> flags: 0x4 mem regions).
>

OK, thanks for clearing that up. But that still does not explain how
we can be certain that NOMAP regions are guaranteed to be covered by a
struct page, does it? Because that is ultimately what pfn_valid()
means, that it is safe to, e.g., look at the page flags.


> [    0.000000] efi: Processing EFI memory map:
> [    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000001480000-0x0000024bffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000024c0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010000400000-0x010f816aefff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f816af000-0x010f816b1fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f816b2000-0x010f826f1fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f826f2000-0x010f82701fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f82702000-0x010f82787fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f82788000-0x010f9276bfff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f9276c000-0x010f9276cfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f9276d000-0x010f935a8fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f935a9000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
> [    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
>
> [    0.000000] MEMBLOCK configuration:
> [    0.000000]  memory size = 0x1ffe800000 reserved size = 0x39146a21
> [    0.000000]  memory.cnt  = 0x9
> [    0.000000]  memory[0x0]     [0x00000001400000-0x000000fffdffff], 0xfebe0000 bytes on node 0 flags: 0x0
> [    0.000000]  memory[0x1]     [0x000000fffe0000-0x000000ffffffff], 0x20000 bytes on node 0 flags: 0x4
> [    0.000000]  memory[0x2]     [0x00000100000000-0x00000fffffffff], 0xf00000000 bytes on node 0 flags: 0x0
> [    0.000000]  memory[0x3]     [0x00010000400000-0x00010ff9e7ffff], 0xff9a80000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x4]     [0x00010ff9e80000-0x00010ff9f1ffff], 0xa0000 bytes on node 1 flags: 0x4
> [    0.000000]  memory[0x5]     [0x00010ff9f20000-0x00010ffaeaffff], 0xf90000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x6]     [0x00010ffaeb0000-0x00010ffaffffff], 0x150000 bytes on node 1 flags: 0x4
> [    0.000000]  memory[0x7]     [0x00010ffb000000-0x00010ffffaffff], 0x4fb0000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x8]     [0x00010ffffb0000-0x00010fffffffff], 0x50000 bytes on node 1 flags: 0x4
> [    0.000000]  reserved.cnt  = 0xd
> [    0.000000]  reserved[0x0]   [0x00000001480000-0x0000000249ffff], 0x1020000 bytes flags: 0x0
> [    0.000000]  reserved[0x1]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
> [    0.000000]  reserved[0x2]   [0x000000c0000000-0x000000dfffffff], 0x20000000 bytes flags: 0x0
> [    0.000000]  reserved[0x3]   [0x00000ffbfb8000-0x00000ffffdffff], 0x4028000 bytes flags: 0x0
> [    0.000000]  reserved[0x4]   [0x00000ffffecb00-0x00000fffffffff], 0x13500 bytes flags: 0x0
> [    0.000000]  reserved[0x5]   [0x00010f81780000-0x00010f8178ffff], 0x10000 bytes flags: 0x0
> [    0.000000]  reserved[0x6]   [0x00010f82870000-0x00010f9286ffff], 0x10000000 bytes flags: 0x0
> [    0.000000]  reserved[0x7]   [0x00010ffbce0000-0x00010fffceffff], 0x4010000 bytes flags: 0x0
> [    0.000000]  reserved[0x8]   [0x00010fffee6d80-0x00010ffff2fffb], 0x4927c bytes flags: 0x0
> [    0.000000]  reserved[0x9]   [0x00010ffff30000-0x00010ffffa000f], 0x70010 bytes flags: 0x0
> [    0.000000]  reserved[0xa]   [0x00010ffffae280-0x00010ffffaff7f], 0x1d00 bytes flags: 0x0
> [    0.000000]  reserved[0xb]   [0x00010ffffaffa0-0x00010ffffaffce], 0x2f bytes flags: 0x0
> [    0.000000]  reserved[0xc]   [0x00010ffffaffd0-0x00010ffffafffe], 0x2f bytes flags: 0x0

From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Date: Fri, 25 Nov 2016 17:01:01 +0000
Message-ID: <CAKv+Gu8Hbf1sY4zMGoosYjGa1esApqP8a4Zs2Zqj4nNZ8AdtSw@mail.gmail.com>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
 <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
 <20161125112914.GI2213@rric.localdomain>
 <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org>
In-Reply-To: <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
List-Unsubscribe: <http://lists.infradead.org/mailman/options/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=unsubscribe>
List-Archive: <http://lists.infradead.org/pipermail/linux-arm-kernel/>
List-Post: <mailto:linux-arm-kernel@lists.infradead.org>
List-Help: <mailto:linux-arm-kernel-request@lists.infradead.org?subject=help>
List-Subscribe: <http://lists.infradead.org/mailman/listinfo/linux-arm-kernel>,
 <mailto:linux-arm-kernel-request@lists.infradead.org?subject=subscribe>
Sender: "linux-arm-kernel" <linux-arm-kernel-bounces@lists.infradead.org>
Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org
To: Robert Richter <robert.richter@cavium.com>
Cc: Mark Rutland <mark.rutland@arm.com>, "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, Robert Richter <rric@kernel.org>, David Daney <david.daney@cavium.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will.deacon@arm.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Hanjun Guo <hanjun.guo@linaro.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
List-Id: linux-efi@vger.kernel.org

On 25 November 2016 at 12:28, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 25 November 2016 at 11:29, Robert Richter <robert.richter@cavium.com> wrote:
>> On 24.11.16 19:42:47, Ard Biesheuvel wrote:
>>> On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:
>>
>>> > I revisited the code and it is working well already since:
>>> >
>>> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>>> >
>>> > Now, try_ram_remap() is only called if the region to be mapped is
>>> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
>>> > ranges and not NOMAP mem. region_intersects() then returns
>>> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
>>> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
>>> > called directly. Before the e7cd190385d1 change try_ram_remap() was
>>> > called also for nomap regions.
>>> >
>>> > So we can leave memremap() as it is and just apply this patch
>>> > unmodified. What do you think?
>>>
>>> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
>>> simply because the PageHighmem check requires a valid struct page. But
>>> if we don't enter that code path anymore for NOMAP regions, I think
>>> we're ok.
>>>
>>> > Please ack.
>>> >
>>>
>>> I still don't fully understand how it is guaranteed that *all* memory
>>> (i.e., all regions for which memblock_is_memory() returns true) is
>>> covered by a struct page, but marked as reserved. Are we relying on
>>> the fact that NOMAP memory is also memblock_reserve()'d?
>>
>> See free_low_memory_core_early():
>>
>> ----
>>         for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
>>                                 NULL)
>>                 count += __free_memory_core(start, end);
>> ----
>>
>> Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
>> also *not* marked reserved. So nothing at all from NOMAP mem is
>> reported to mm, it is not present (see below for a mem config, note
>> flags: 0x4 mem regions).
>>
>
> OK, thanks for clearing that up. But that still does not explain how
> we can be certain that NOMAP regions are guaranteed to be covered by a
> struct page, does it? Because that is ultimately what pfn_valid()
> means, that it is safe to, e.g., look at the page flags.
>

Answering to self: arm64_memory_present() registers all memory as
present, which means that sparse_init() will allocate struct page
backing for all of memory, including NOMAP regions.

We could ask ourselves whether it makes sense to disregard NOMAP
memory here, but that probably buys us very little (but I do wonder
how it affects the occurrence of the bug).

In any case, it looks to me like your patch is safe, i.e., calling
pfn_valid() on NOMAP pages is safe, although I still find it debatable
whether the kernel should be tracking memory it does not own. However,
for performance reasons, it probably makes sense to apply your patch,
and if anyone ever comes up with a use case where this is problematic
(e.g., gigabytes of NOMAP memory), we can look into it then.

Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

From mboxrd@z Thu Jan  1 00:00:00 1970
From: rrichter@cavium.com (Robert Richter)
Date: Thu, 6 Oct 2016 11:52:07 +0200
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Message-ID: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

There is a memory setup problem on ThunderX systems with certain
memory configurations. The symptom is

 kernel BUG at mm/page_alloc.c:1848!

This happens for some configs with 64k page size enabled. The bug
triggers for page zones with some pages in the zone not assigned to
this particular zone. In my case some pages that are marked as nomap
were not reassigned to the new zone of node 1, so those are still
assigned to node 0.

The reason for the mis-configuration is a change in pfn_valid() which
reports pages marked nomap as invalid:

 68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping

This causes pages marked as nomap being no long reassigned to the new
zone in memmap_init_zone() by calling __init_single_pfn().

Fixing this by restoring the old behavior of pfn_valid() to use
memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
to use memblock_is_map_memory() where necessary. This only affects
code in ioremap.c. The code in mmu.c still can use the new version of
pfn_valid().

Should be marked stable v4.5..

Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 arch/arm64/mm/init.c    | 2 +-
 arch/arm64/mm/ioremap.c | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index bbb7ee76e319..25b8659c2a9f 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -147,7 +147,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 #ifdef CONFIG_HAVE_ARCH_PFN_VALID
 int pfn_valid(unsigned long pfn)
 {
-	return memblock_is_map_memory(pfn << PAGE_SHIFT);
+	return memblock_is_memory(pfn << PAGE_SHIFT);
 }
 EXPORT_SYMBOL(pfn_valid);
 #endif
diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c
index 01e88c8bcab0..c17c220b0c48 100644
--- a/arch/arm64/mm/ioremap.c
+++ b/arch/arm64/mm/ioremap.c
@@ -21,6 +21,7 @@
  */
 
 #include <linux/export.h>
+#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <linux/vmalloc.h>
 #include <linux/io.h>
@@ -55,7 +56,7 @@ static void __iomem *__ioremap_caller(phys_addr_t phys_addr, size_t size,
 	/*
 	 * Don't allow RAM to be mapped.
 	 */
-	if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr))))
+	if (WARN_ON(memblock_is_map_memory(phys_addr)))
 		return NULL;
 
 	area = get_vm_area_caller(size, VM_IOREMAP, caller);
@@ -96,7 +97,7 @@ EXPORT_SYMBOL(__iounmap);
 void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size)
 {
 	/* For normal memory we already have a cacheable mapping. */
-	if (pfn_valid(__phys_to_pfn(phys_addr)))
+	if (memblock_is_map_memory(phys_addr))
 		return (void __iomem *)__phys_to_virt(phys_addr);
 
 	return __ioremap_caller(phys_addr, size, __pgprot(PROT_NORMAL),
-- 
2.7.0.rc3

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Thu, 6 Oct 2016 11:00:33 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Message-ID: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Hi Robert,

Apologies for only responding now. I did not quite manage to get my
head around your original email yet, but I don't think this patch is
the correct solution.

On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
>
>  kernel BUG at mm/page_alloc.c:1848!
>
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
>
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
>
>  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
>

These pages are owned by the firmware, which may map it with
attributes that conflict with the attributes we use for the linear
mapping. This means they should not be covered by the linear mapping.

> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
>

This sounds like the root cause of your issue. Could we not fix that instead?

> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory().

This is incorrect imo. In general, pfn_valid() means ordinary memory
covered by the linear mapping and the struct page array. Returning
reserved ranges that the kernel should not even touch only to please
the NUMA code seems like an inappropriate way to deal with this issue.

> Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().
>
> Should be marked stable v4.5..
>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> ---
>  arch/arm64/mm/init.c    | 2 +-
>  arch/arm64/mm/ioremap.c | 5 +++--
>  2 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index bbb7ee76e319..25b8659c2a9f 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -147,7 +147,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
>  #ifdef CONFIG_HAVE_ARCH_PFN_VALID
>  int pfn_valid(unsigned long pfn)
>  {
> -       return memblock_is_map_memory(pfn << PAGE_SHIFT);
> +       return memblock_is_memory(pfn << PAGE_SHIFT);
>  }
>  EXPORT_SYMBOL(pfn_valid);
>  #endif
> diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c
> index 01e88c8bcab0..c17c220b0c48 100644
> --- a/arch/arm64/mm/ioremap.c
> +++ b/arch/arm64/mm/ioremap.c
> @@ -21,6 +21,7 @@
>   */
>
>  #include <linux/export.h>
> +#include <linux/memblock.h>
>  #include <linux/mm.h>
>  #include <linux/vmalloc.h>
>  #include <linux/io.h>
> @@ -55,7 +56,7 @@ static void __iomem *__ioremap_caller(phys_addr_t phys_addr, size_t size,
>         /*
>          * Don't allow RAM to be mapped.
>          */
> -       if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr))))
> +       if (WARN_ON(memblock_is_map_memory(phys_addr)))
>                 return NULL;
>
>         area = get_vm_area_caller(size, VM_IOREMAP, caller);
> @@ -96,7 +97,7 @@ EXPORT_SYMBOL(__iounmap);
>  void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size)
>  {
>         /* For normal memory we already have a cacheable mapping. */
> -       if (pfn_valid(__phys_to_pfn(phys_addr)))
> +       if (memblock_is_map_memory(phys_addr))
>                 return (void __iomem *)__phys_to_virt(phys_addr);
>
>         return __ioremap_caller(phys_addr, size, __pgprot(PROT_NORMAL),
> --
> 2.7.0.rc3
>

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Thu, 6 Oct 2016 18:11:14 +0200
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
Message-ID: <20161006161114.GH22012@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Ard,

thank you for your answer and you explanation.

On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > There is a memory setup problem on ThunderX systems with certain
> > memory configurations. The symptom is
> >
> >  kernel BUG at mm/page_alloc.c:1848!
> >
> > This happens for some configs with 64k page size enabled. The bug
> > triggers for page zones with some pages in the zone not assigned to
> > this particular zone. In my case some pages that are marked as nomap
> > were not reassigned to the new zone of node 1, so those are still
> > assigned to node 0.
> >
> > The reason for the mis-configuration is a change in pfn_valid() which
> > reports pages marked nomap as invalid:
> >
> >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> >
> 
> These pages are owned by the firmware, which may map it with
> attributes that conflict with the attributes we use for the linear
> mapping. This means they should not be covered by the linear mapping.
> 
> > This causes pages marked as nomap being no long reassigned to the new
> > zone in memmap_init_zone() by calling __init_single_pfn().
> >
> 
> This sounds like the root cause of your issue. Could we not fix that instead?

Yes, this is proposal b) from my last mail that would work too: I
implemented an arm64 private early_pfn_valid() function that uses
memblock_is_memory() to setup all pages of a zone. Though, I think
this is the wrong way and thus I prefer this patch instead. I see
serveral reasons for this:

Inconsistent use of struct *page, it is initialized but never used
again.

Other archs only do a basic range check in pfn_valid(), the default
implementation just returns if the whole section is valid. As I
understand the code, if the mem range is not aligned to the section,
then there will be pfn's in the section that don't have physical mem
attached. The page is then just initialized, it's not marked reserved
nor the refcount is non-zero. It is then simply not used. This is how
no-map pages should be handled too.

I think pfn_valid() is just a quick check if the pfn's struct *page
can be used. There is a good description for this in include/linux/
mmzone.h. So there can be memory holes that have a valid pfn.

If the no-map memory needs special handling, then additional checks
need to be added to the particular code (as in ioremap.c). It's imo
wrong to (mis-)use pfn_valid for that.

Variant b) involves generic mm code to fix it for arm64, this patch is
an arm64 change only. This makes it harder to get a fix for it.
(Though maybe only a problem of patch logistics.)

> 
> > Fixing this by restoring the old behavior of pfn_valid() to use
> > memblock_is_memory().
> 
> This is incorrect imo. In general, pfn_valid() means ordinary memory
> covered by the linear mapping and the struct page array. Returning
> reserved ranges that the kernel should not even touch only to please
> the NUMA code seems like an inappropriate way to deal with this issue.

As said above, it is not marked as reserved, it is treated like
non-existing memory.

This has been observed for non-numa kernels too and can happen for
each zone that is only partly initialized.

I think the patch addresses your concerns. I can't see there the
kernel uses memory marked as nomap in a wrong way.

Thanks,

-Robert

> 
> > Also changing users of pfn_valid() in arm64 code
> > to use memblock_is_map_memory() where necessary. This only affects
> > code in ioremap.c. The code in mmu.c still can use the new version of
> > pfn_valid().
> >
> > Should be marked stable v4.5..
> >
> > Signed-off-by: Robert Richter <rrichter@cavium.com>
> > ---
> >  arch/arm64/mm/init.c    | 2 +-
> >  arch/arm64/mm/ioremap.c | 5 +++--
> >  2 files changed, 4 insertions(+), 3 deletions(-)

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ddaney.cavm@gmail.com (David Daney)
Date: Mon, 10 Oct 2016 08:33:50 -0700
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Message-ID: <57FBB4DE.2090502@gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 10/06/2016 02:52 AM, Robert Richter wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
>
>   kernel BUG at mm/page_alloc.c:1848!
>
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
>
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
>
>   68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
>
> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
>
> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().
>
> Should be marked stable v4.5..

In that case you should add:

Cc: <stable@vger.kernel.org> # 4.5.x-

here.


>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
[...]

From mboxrd@z Thu Jan  1 00:00:00 1970
From: rric@kernel.org (Robert Richter)
Date: Mon, 17 Oct 2016 20:58:01 +0200
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161006161114.GH22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
Message-ID: <20161017185801.GT25086@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Mark, Will, any opinion here?

Thanks,

-Robert

On 06.10.16 18:11:14, Robert Richter wrote:
> Ard,
> 
> thank you for your answer and you explanation.
> 
> On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > > There is a memory setup problem on ThunderX systems with certain
> > > memory configurations. The symptom is
> > >
> > >  kernel BUG at mm/page_alloc.c:1848!
> > >
> > > This happens for some configs with 64k page size enabled. The bug
> > > triggers for page zones with some pages in the zone not assigned to
> > > this particular zone. In my case some pages that are marked as nomap
> > > were not reassigned to the new zone of node 1, so those are still
> > > assigned to node 0.
> > >
> > > The reason for the mis-configuration is a change in pfn_valid() which
> > > reports pages marked nomap as invalid:
> > >
> > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > >
> > 
> > These pages are owned by the firmware, which may map it with
> > attributes that conflict with the attributes we use for the linear
> > mapping. This means they should not be covered by the linear mapping.
> > 
> > > This causes pages marked as nomap being no long reassigned to the new
> > > zone in memmap_init_zone() by calling __init_single_pfn().
> > >
> > 
> > This sounds like the root cause of your issue. Could we not fix that instead?
> 
> Yes, this is proposal b) from my last mail that would work too: I
> implemented an arm64 private early_pfn_valid() function that uses
> memblock_is_memory() to setup all pages of a zone. Though, I think
> this is the wrong way and thus I prefer this patch instead. I see
> serveral reasons for this:
> 
> Inconsistent use of struct *page, it is initialized but never used
> again.
> 
> Other archs only do a basic range check in pfn_valid(), the default
> implementation just returns if the whole section is valid. As I
> understand the code, if the mem range is not aligned to the section,
> then there will be pfn's in the section that don't have physical mem
> attached. The page is then just initialized, it's not marked reserved
> nor the refcount is non-zero. It is then simply not used. This is how
> no-map pages should be handled too.
> 
> I think pfn_valid() is just a quick check if the pfn's struct *page
> can be used. There is a good description for this in include/linux/
> mmzone.h. So there can be memory holes that have a valid pfn.
> 
> If the no-map memory needs special handling, then additional checks
> need to be added to the particular code (as in ioremap.c). It's imo
> wrong to (mis-)use pfn_valid for that.
> 
> Variant b) involves generic mm code to fix it for arm64, this patch is
> an arm64 change only. This makes it harder to get a fix for it.
> (Though maybe only a problem of patch logistics.)
> 
> > 
> > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > memblock_is_memory().
> > 
> > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > covered by the linear mapping and the struct page array. Returning
> > reserved ranges that the kernel should not even touch only to please
> > the NUMA code seems like an inappropriate way to deal with this issue.
> 
> As said above, it is not marked as reserved, it is treated like
> non-existing memory.
> 
> This has been observed for non-numa kernels too and can happen for
> each zone that is only partly initialized.
> 
> I think the patch addresses your concerns. I can't see there the
> kernel uses memory marked as nomap in a wrong way.
> 
> Thanks,
> 
> -Robert
> 
> > 
> > > Also changing users of pfn_valid() in arm64 code
> > > to use memblock_is_map_memory() where necessary. This only affects
> > > code in ioremap.c. The code in mmu.c still can use the new version of
> > > pfn_valid().
> > >
> > > Should be marked stable v4.5..
> > >
> > > Signed-off-by: Robert Richter <rrichter@cavium.com>
> > > ---
> > >  arch/arm64/mm/init.c    | 2 +-
> > >  arch/arm64/mm/ioremap.c | 5 +++--
> > >  2 files changed, 4 insertions(+), 3 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-efi" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

From mboxrd@z Thu Jan  1 00:00:00 1970
From: mark.rutland@arm.com (Mark Rutland)
Date: Tue, 18 Oct 2016 11:18:36 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161006161114.GH22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
Message-ID: <20161018101715.GA15639@leverpostej>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Hi Robert, Ard,

Sorry for the delay in getting to this; I've been travelling a lot
lately and in the meantime this managed to get buried in my inbox.

On Thu, Oct 06, 2016 at 06:11:14PM +0200, Robert Richter wrote:
> On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > > There is a memory setup problem on ThunderX systems with certain
> > > memory configurations. The symptom is
> > >
> > >  kernel BUG at mm/page_alloc.c:1848!
> > >
> > > This happens for some configs with 64k page size enabled. The bug
> > > triggers for page zones with some pages in the zone not assigned to
> > > this particular zone. In my case some pages that are marked as nomap
> > > were not reassigned to the new zone of node 1, so those are still
> > > assigned to node 0.
> > >
> > > The reason for the mis-configuration is a change in pfn_valid() which
> > > reports pages marked nomap as invalid:
> > >
> > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > 
> > These pages are owned by the firmware, which may map it with
> > attributes that conflict with the attributes we use for the linear
> > mapping. This means they should not be covered by the linear mapping.
> > 
> > > This causes pages marked as nomap being no long reassigned to the new
> > > zone in memmap_init_zone() by calling __init_single_pfn().

Why do we have pages for a nomap region? Given the region shouldn't be
in the linear mapping, and isn't suitable for general allocation, I
don't believe it makes sense to have a struct page for any part of it.

Am I missing some reason that we require a struct page?

e.g. is it just easier to allocate an unused struct page than to carve
it out?

> > This sounds like the root cause of your issue. Could we not fix that instead?
> 
> Yes, this is proposal b) from my last mail that would work too: I
> implemented an arm64 private early_pfn_valid() function that uses
> memblock_is_memory() to setup all pages of a zone. Though, I think
> this is the wrong way and thus I prefer this patch instead. I see
> serveral reasons for this:
> 
> Inconsistent use of struct *page, it is initialized but never used
> again.

As above, I don't believe we should have a struct page to initialise in
the first place.

> Other archs only do a basic range check in pfn_valid(), the default
> implementation just returns if the whole section is valid. As I
> understand the code, if the mem range is not aligned to the section,
> then there will be pfn's in the section that don't have physical mem
> attached. The page is then just initialized, it's not marked reserved
> nor the refcount is non-zero. It is then simply not used. This is how
> no-map pages should be handled too.
> 
> I think pfn_valid() is just a quick check if the pfn's struct *page
> can be used. There is a good description for this in include/linux/
> mmzone.h. So there can be memory holes that have a valid pfn.

I take it you mean the comment in the CONFIG_ARCH_HAS_HOLES_MEMORYMODEL
ifdef (line 1266 in v4.9-rc1)?

I'm not sufficiently acquainted with the memmap code to follow; I'll
need to dig into that a bit further.

> If the no-map memory needs special handling, then additional checks
> need to be added to the particular code (as in ioremap.c). It's imo
> wrong to (mis-)use pfn_valid for that.
> 
> Variant b) involves generic mm code to fix it for arm64, this patch is
> an arm64 change only. This makes it harder to get a fix for it.
> (Though maybe only a problem of patch logistics.)
> 
> > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > memblock_is_memory().
> > 
> > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > covered by the linear mapping and the struct page array. Returning
> > reserved ranges that the kernel should not even touch only to please
> > the NUMA code seems like an inappropriate way to deal with this issue.
> 
> As said above, it is not marked as reserved, it is treated like
> non-existing memory.

I think Ard was using "reserved" in the more general sense than the
Linux-specific meaning. NOMAP is distinct from the Linux concept of
"reserved" memory, but is "reserved" in some sense.

Memory with NOMAP is meant to be treated as non-existent for the purpose
of the linear mapping (and thus for the purpose of struct page).

> This has been observed for non-numa kernels too and can happen for
> each zone that is only partly initialized.
> 
> I think the patch addresses your concerns. I can't see there the
> kernel uses memory marked as nomap in a wrong way.

I'll have to dig into this locally; I'm still not familiar enough with
this code to know what the right thing to do is.

Thanks,
Mark.

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Tue, 18 Oct 2016 17:02:40 +0200
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161018101715.GA15639@leverpostej>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161018101715.GA15639@leverpostej>
Message-ID: <20161018150240.GU22012@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Mark,

thanks for your answer. See below. I also attached the full crash
dump.

On 18.10.16 11:18:36, Mark Rutland wrote:
> Hi Robert, Ard,
> 
> Sorry for the delay in getting to this; I've been travelling a lot
> lately and in the meantime this managed to get buried in my inbox.
> 
> On Thu, Oct 06, 2016 at 06:11:14PM +0200, Robert Richter wrote:
> > On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > > On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > > > There is a memory setup problem on ThunderX systems with certain
> > > > memory configurations. The symptom is
> > > >
> > > >  kernel BUG at mm/page_alloc.c:1848!
> > > >
> > > > This happens for some configs with 64k page size enabled. The bug
> > > > triggers for page zones with some pages in the zone not assigned to
> > > > this particular zone. In my case some pages that are marked as nomap
> > > > were not reassigned to the new zone of node 1, so those are still
> > > > assigned to node 0.
> > > >
> > > > The reason for the mis-configuration is a change in pfn_valid() which
> > > > reports pages marked nomap as invalid:
> > > >
> > > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > > 
> > > These pages are owned by the firmware, which may map it with
> > > attributes that conflict with the attributes we use for the linear
> > > mapping. This means they should not be covered by the linear mapping.
> > > 
> > > > This causes pages marked as nomap being no long reassigned to the new
> > > > zone in memmap_init_zone() by calling __init_single_pfn().
> 
> Why do we have pages for a nomap region? Given the region shouldn't be
> in the linear mapping, and isn't suitable for general allocation, I
> don't believe it makes sense to have a struct page for any part of it.
> 
> Am I missing some reason that we require a struct page?
> 
> e.g. is it just easier to allocate an unused struct page than to carve
> it out?

Pages are handled in blocks with size MAX_ORDER_NR_PAGES. The start
and end pfn of a memory region is aligned then to fit the mem block
size (see memmap_init_zone()). Therefore a memblock may contain pages
without underlying physical memory.

mm code requires the whole memmap to be initialized, this means that
for each page in the whole mem block there is a valid struct page. See
e.g. move_freepages_block() and move_freepages(), stuct page is
accessed even before pfn_valid() is used. I assume there are other
occurrences of that too.

My interpretation is that pfn_valid() checks for the existence of a
valid struct page, there must not necessarily phys memory mapped to
it. This is the reason why I changed pfn_valid() to use
memblock_is_memory() which is sufficient for generic mm code. Only in
arm64 mm code I additinally added the memblock_is_map_memory() check
where pfn_valid() was used.

> 
> > > This sounds like the root cause of your issue. Could we not fix that instead?
> > 
> > Yes, this is proposal b) from my last mail that would work too: I
> > implemented an arm64 private early_pfn_valid() function that uses
> > memblock_is_memory() to setup all pages of a zone. Though, I think
> > this is the wrong way and thus I prefer this patch instead. I see
> > serveral reasons for this:
> > 
> > Inconsistent use of struct *page, it is initialized but never used
> > again.
> 
> As above, I don't believe we should have a struct page to initialise in
> the first place.
> 
> > Other archs only do a basic range check in pfn_valid(), the default
> > implementation just returns if the whole section is valid. As I
> > understand the code, if the mem range is not aligned to the section,
> > then there will be pfn's in the section that don't have physical mem
> > attached. The page is then just initialized, it's not marked reserved
> > nor the refcount is non-zero. It is then simply not used. This is how
> > no-map pages should be handled too.
> > 
> > I think pfn_valid() is just a quick check if the pfn's struct *page
> > can be used. There is a good description for this in include/linux/
> > mmzone.h. So there can be memory holes that have a valid pfn.
> 
> I take it you mean the comment in the CONFIG_ARCH_HAS_HOLES_MEMORYMODEL
> ifdef (line 1266 in v4.9-rc1)?

Yes.

> 
> I'm not sufficiently acquainted with the memmap code to follow; I'll
> need to dig into that a bit further.
> 
> > If the no-map memory needs special handling, then additional checks
> > need to be added to the particular code (as in ioremap.c). It's imo
> > wrong to (mis-)use pfn_valid for that.
> > 
> > Variant b) involves generic mm code to fix it for arm64, this patch is
> > an arm64 change only. This makes it harder to get a fix for it.
> > (Though maybe only a problem of patch logistics.)
> > 
> > > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > > memblock_is_memory().
> > > 
> > > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > > covered by the linear mapping and the struct page array. Returning
> > > reserved ranges that the kernel should not even touch only to please
> > > the NUMA code seems like an inappropriate way to deal with this issue.
> > 
> > As said above, it is not marked as reserved, it is treated like
> > non-existing memory.
> 
> I think Ard was using "reserved" in the more general sense than the
> Linux-specific meaning. NOMAP is distinct from the Linux concept of
> "reserved" memory, but is "reserved" in some sense.
> 
> Memory with NOMAP is meant to be treated as non-existent for the purpose
> of the linear mapping (and thus for the purpose of struct page).

Yes, it's not marked reserved and can never be freed.

Thanks,

-Robert

> 
> > This has been observed for non-numa kernels too and can happen for
> > each zone that is only partly initialized.
> > 
> > I think the patch addresses your concerns. I can't see there the
> > kernel uses memory marked as nomap in a wrong way.
> 
> I'll have to dig into this locally; I'm still not familiar enough with
> this code to know what the right thing to do is.
> 
> Thanks,
> Mark.

EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
Booting Linux on physical CPU 0x0
Linux version 4.8.0-rc4-00267-g664e88c (root at crb2spass2rric.localdomain) (gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.1) ) #3 SMP Wed Sep 7 06:38:13 UTC 2016
Boot CPU: AArch64 Processor [431f0a10]
efi: Getting EFI parameters from FDT:
efi: EFI v2.40 by Cavium Thunder cn88xx EFI ThunderX-Firmware-Release-1.22.10-0-g4e85766 Aug 24 2016 15:59:03
efi:  ACPI=0xfffff000  ACPI 2.0=0xfffff014  SMBIOS 3.0=0x10ffafcf000 
cma: Reserved 512 MiB at 0x00000000c0000000
NUMA: Adding memblock [0x1400000 - 0xfffffffff] on node 0
NUMA: Adding memblock [0x10000400000 - 0x10fffffffff] on node 1
NUMA: parsing numa-distance-map-v1
NUMA: Initmem setup node 0 [mem 0x01400000-0xfffffffff]
NUMA: NODE_DATA [mem 0xfffff2580-0xfffffffff]
NUMA: Initmem setup node 1 [mem 0x10000400000-0x10fffffffff]
NUMA: NODE_DATA [mem 0x10ffffa2500-0x10ffffaff7f]
kmemleak: Kernel memory leak detector disabled
Zone ranges:
  DMA      [mem 0x0000000001400000-0x00000000ffffffff]
  Normal   [mem 0x0000000100000000-0x0000010fffffffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000001400000-0x00000000fffdffff]
  node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
  node   0: [mem 0x0000000100000000-0x0000000fffffffff]
  node   1: [mem 0x0000010000400000-0x0000010ff9e8ffff]
  node   1: [mem 0x0000010ff9e90000-0x0000010ff9f2ffff]
  node   1: [mem 0x0000010ff9f30000-0x0000010ffaeaffff]
  node   1: [mem 0x0000010ffaeb0000-0x0000010ffaffffff]
  node   1: [mem 0x0000010ffb000000-0x0000010ffffaffff]
  node   1: [mem 0x0000010ffffb0000-0x0000010fffffffff]
Initmem setup node 0 [mem 0x0000000001400000-0x0000000fffffffff]
Initmem setup node 1 [mem 0x0000010000400000-0x0000010fffffffff]
psci: probing for conduit method from DT.
psci: PSCIv0.2 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: Trusted OS resident on physical CPU 0x0
Number of cores (96) exceeds configured maximum of 8 - clipping
percpu: Embedded 3 pages/cpu @ffffff0fffce0000 s117704 r8192 d70712 u196608
Detected VIPT I-cache on CPU0
CPU features: enabling workaround for Cavium erratum 27456
Built 2 zonelists in Node order, mobility grouping on.  Total pages: 2094720
Policy zone: Normal
Kernel command line: BOOT_IMAGE=/vmlinuz root=UUID=9a418ae5-231a-40c7-b7ba-ca70cfadfc5e ro LANG=en_US.UTF-8
PID hash table entries: 4096 (order: -1, 32768 bytes)
software IO TLB [mem 0xfbfd0000-0xfffd0000] (64MB) mapped at [fffffe00fbfd0000-fffffe00fffcffff]
Memory: 133196160K/134193152K available (9084K kernel code, 1545K rwdata, 3712K rodata, 1536K init, 15826K bss, 472704K reserved, 524288K cma-reserved)
Virtual kernel memory layout:
    modules : 0xfffffc0000000000 - 0xfffffc0008000000   (   128 MB)
    vmalloc : 0xfffffc0008000000 - 0xfffffdff5fff0000   (  2045 GB)
      .text : 0xfffffc0008080000 - 0xfffffc0008960000   (  9088 KB)
    .rodata : 0xfffffc0008960000 - 0xfffffc0008d10000   (  3776 KB)
      .init : 0xfffffc0008d10000 - 0xfffffc0008e90000   (  1536 KB)
      .data : 0xfffffc0008e90000 - 0xfffffc0009012600   (  1546 KB)
       .bss : 0xfffffc0009012600 - 0xfffffc0009f86fc8   ( 15827 KB)
    fixed   : 0xfffffdff7e7d0000 - 0xfffffdff7ec00000   (  4288 KB)
    PCI I/O : 0xfffffdff7ee00000 - 0xfffffdff7fe00000   (    16 MB)
    vmemmap : 0xfffffdff80000000 - 0xfffffe0000000000   (     2 GB maximum)
              0xfffffdff80005000 - 0xfffffdffc4000000   (  1087 MB actual)
    memory  : 0xfffffe0001400000 - 0xffffff1000000000   (1114092 MB)
SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=8, Nodes=2
Running RCU self tests
Hierarchical RCU implementation.
	RCU lockdep checking is enabled.
	Build-time adjustment of leaf fanout to 64.
NR_IRQS:64 nr_irqs:64 0
GICv3: GIC: Using split EOI/Deactivate mode
ITS: /interrupt-controller at 801000000000/gic-its at 801000020000
ITS at 0x0000801000020000: allocated 2097152 Devices @ff5000000 (flat, esz 8, psz 64K, shr 1)
ITS: /interrupt-controller at 801000000000/gic-its at 901000020000
ITS at 0x0000901000020000: allocated 2097152 Devices @ffa000000 (flat, esz 8, psz 64K, shr 1)
GIC: using LPI property table @0x0000000ff4140000
ITS: Allocated 32512 chunks for LPIs
GICv3: CPU0: found redistributor 0 region 0:0x0000801080000000
CPU0: using LPI pending table @0x0000000ff4150000
arm_arch_timer: Architected cp15 timer(s) running at 100.00MHz (phys).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x171024e7e0, max_idle_ns: 440795205315 ns
sched_clock: 56 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns
Console: colour dummy device 80x25
console [tty0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:  8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:          4096
... MAX_LOCKDEP_ENTRIES:     32768
... MAX_LOCKDEP_CHAINS:      65536
... CHAINHASH_SIZE:          32768
 memory used by lock dependency info: 8159 kB
 per task-struct memory footprint: 1920 bytes
mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=100000)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Yama: becoming mindful.
SELinux:  Initializing.
Dentry cache hash table entries: 16777216 (order: 11, 134217728 bytes)
Inode-cache hash table entries: 8388608 (order: 10, 67108864 bytes)
Mount-cache hash table entries: 262144 (order: 5, 2097152 bytes)
Mountpoint-cache hash table entries: 262144 (order: 5, 2097152 bytes)
ftrace: allocating 29244 entries in 8 pages
Unable to find CPU node for /cpus/cpu at 8
/cpus/cpu-map/cluster0/core8: Can't get CPU for leaf core
ASID allocator initialised with 65536 entries
PCI/MSI: /interrupt-controller at 801000000000/gic-its at 801000020000 domain created
PCI/MSI: /interrupt-controller at 801000000000/gic-its at 901000020000 domain created
Platform MSI: /interrupt-controller at 801000000000/gic-its at 801000020000 domain created
Platform MSI: /interrupt-controller at 801000000000/gic-its at 901000020000 domain created
Remapping and enabling EFI services.
UEFI Runtime regions are not aligned to 64 KB -- buggy firmware?
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at arch/arm64/kernel/efi.c:34 efi_create_mapping+0x5c/0x11c
Modules linked in:

CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.8.0-rc4-00267-g664e88c #3
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fffffe0fee55a200 task.stack: ffffff00004a4000
PC is at efi_create_mapping+0x5c/0x11c
LR is at efi_create_mapping+0x5c/0x11c
pc : [<fffffc0008d14f68>] lr : [<fffffc0008d14f68>] pstate: 60000045
sp : ffffff00004a7d00
x29: ffffff00004a7d00 x28: 0000000000000000 
x27: 0000000000000000 x26: 0000000000000000 
x25: 0000000000000000 x24: fffffc0008f9c730 
x23: fffffc0008f9c730 x22: fffffc0008c60ac0 
x21: fffffc0008f9c730 x20: 00c8000000000713 
x19: ffffff0ff76b0c08 x18: 0000000000000010 
x17: 00000000c9cbbfc7 x16: 0000000000000000 
x15: fffffc0089c956f7 x14: 3f657261776d7269 
x13: 6620796767756220 x12: 2d2d20424b203436 
x11: 206f742064656e67 x10: 0000000000000077 
x9 : 65726120736e6f69 x8 : 0000000000000001 
x7 : ffffff00004a4000 x6 : fffffc000814a288 
x5 : 0000000000000000 x4 : 0000000000000001 
x3 : 0000000000000001 x2 : ffffff00004a4000 
x1 : fffffe0fee55a200 x0 : 0000000000000040 

---[ end trace fcdd7f8cb96cdb3b ]---
Call trace:
Exception stack(0xffffff00004a7b20 to 0xffffff00004a7c50)
7b20: ffffff0ff76b0c08 0000040000000000 ffffff00004a7d00 fffffc0008d14f68
7b40: 0000000060000045 000000000000003d 0000000000000001 fffffc000814a9fc
7b60: ffffff00004a7c00 fffffc000814adcc ffffff00004a7c60 fffffc0008baf420
7b80: fffffc0008f9c730 fffffc0008c60ac0 fffffc0008f9c730 fffffc0008f9c730
7ba0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7bc0: fffffc00080d6d70 0000000000000000 0000000000000040 fffffe0fee55a200
7be0: ffffff00004a4000 0000000000000001 0000000000000001 0000000000000000
7c00: fffffc000814a288 ffffff00004a4000 0000000000000001 65726120736e6f69
7c20: 0000000000000077 206f742064656e67 2d2d20424b203436 6620796767756220
7c40: 3f657261776d7269 fffffc0089c956f7
[<fffffc0008d14f68>] efi_create_mapping+0x5c/0x11c
[<fffffc0008d5e1ec>] arm_enable_runtime_services+0x12c/0x210
[<fffffc0008082ff4>] do_one_initcall+0x44/0x138
[<fffffc0008d10d30>] kernel_init_freeable+0x17c/0x2e0
[<fffffc000893c888>] kernel_init+0x20/0xf8
[<fffffc0008082b80>] ret_from_fork+0x10/0x50
  EFI remap 0x0000010ff9e98000 => 0000000020008000
  EFI remap 0x0000010ffaeb6000 => 00000000200a6000
  EFI remap 0x0000010ffafc9000 => 00000000201b9000
  EFI remap 0x0000010ffafcd000 => 00000000201bd000
  EFI remap 0x0000010ffffb9000 => 00000000201f9000
  EFI remap 0x0000010ffffcd000 => 000000002020d000
  EFI remap 0x0000804000001000 => 0000000020241000
  EFI remap 0x000087e0d0001000 => 0000000020251000
Detected VIPT I-cache on CPU1
GICv3: CPU1: found redistributor 1 region 0:0x0000801080020000
CPU1: using LPI pending table @0x0000010014060000
CPU1: Booted secondary processor [431f0a10]
------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at ./include/linux/cpumask.h:121 gic_raise_softirq+0x14c/0x1e0
Modules linked in:

CPU: 1 PID: 0 Comm: swapper/1 Tainted: G        W       4.8.0-rc4-00267-g664e88c #3
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fffffe0fee57b600 task.stack: fffffe0fe4038000
PC is at gic_raise_softirq+0x14c/0x1e0
LR is at gic_raise_softirq+0xc0/0x1e0
pc : [<fffffc00084c07a4>] lr : [<fffffc00084c0718>] pstate: 600001c5
sp : fffffe0fe403bcd0
x29: fffffe0fe403bcd0 x28: 0000000000000001 
x27: fffffc0008fd7b1d x26: 0000000000000000 
x25: fffffc0008ec2000 x24: fffffc0008ecebc8 
x23: fffffc0008998688 x22: 0000000000000001 
x21: fffffc0008ec2e34 x20: 0000000000000000 
x19: 0000000000000008 x18: 0000000000000030 
x17: 0000000000000000 x16: 0000000000000000 
x15: fffffc0009c97f08 x14: 0000000000000002 
x13: 0000000000000332 x12: 0000000000000335 
x11: fffffc0009bf3de0 x10: 0000000000000332 
x9 : fffffe0fe4038000 x8 : 0000000000000000 
x7 : 00000000ffffffff x6 : 0000000000000000 
x5 : fffffffffffffffe x4 : 0000000000000040 
x3 : 0000000000000000 x2 : 0000000000000000 
x1 : 0000000000000008 x0 : 0000000000000001 

---[ end trace fcdd7f8cb96cdb3c ]---
Call trace:
Exception stack(0xfffffe0fe403baf0 to 0xfffffe0fe403bc20)
bae0:                                   0000000000000008 0000040000000000
bb00: fffffe0fe403bcd0 fffffc00084c07a4 00000000600001c5 000000000000003d
bb20: fffffe0fee57bec0 ffffffffffffffd8 0000000000000028 fffffe0fee57be70
bb40: fffffc000925fe88 0000000000000002 fffffc00080886c0 fffffe0fee57b600
bb60: fffffe0fe403bb90 fffffc0008088974 fffffc00099b4d60 fffffe0fee57b600
bb80: fffffc000925fe88 fffffe0fe403bd4c fffffe0fe403bbe0 fffffc00080889e8
bba0: 0000000000000001 0000000000000008 0000000000000000 0000000000000000
bbc0: 0000000000000040 fffffffffffffffe 0000000000000000 00000000ffffffff
bbe0: 0000000000000000 fffffe0fe4038000 0000000000000332 fffffc0009bf3de0
bc00: 0000000000000335 0000000000000332 0000000000000002 fffffc0009c97f08
[<fffffc00084c07a4>] gic_raise_softirq+0x14c/0x1e0
[<fffffc000808e028>] smp_cross_call+0x80/0x238
[<fffffc000808efa4>] smp_send_reschedule+0x3c/0x48
[<fffffc00081021a8>] resched_curr+0x58/0x90
[<fffffc0008103fd8>] check_preempt_curr+0x70/0xe8
[<fffffc0008104090>] ttwu_do_wakeup+0x40/0x2e8
[<fffffc00081043cc>] ttwu_do_activate+0x94/0xc0
[<fffffc00081058a8>] try_to_wake_up+0x200/0x580
[<fffffc0008105d3c>] default_wake_function+0x34/0x48
[<fffffc0008126afc>] __wake_up_common+0x64/0xa8
[<fffffc0008126bec>] __wake_up_locked+0x3c/0x50
[<fffffc0008127b58>] complete+0x48/0x68
[<fffffc000808e630>] secondary_start_kernel+0x180/0x208
[<0000000001482e08>] 0x1482e08
Detected VIPT I-cache on CPU2
GICv3: CPU2: found redistributor 2 region 0:0x0000801080040000
CPU2: using LPI pending table @0x0000010014070000
CPU2: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU3
GICv3: CPU3: found redistributor 3 region 0:0x0000801080060000
CPU3: using LPI pending table @0x0000010014080000
CPU3: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU4
GICv3: CPU4: found redistributor 4 region 0:0x0000801080080000
CPU4: using LPI pending table @0x0000010014090000
CPU4: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU5
GICv3: CPU5: found redistributor 5 region 0:0x00008010800a0000
CPU5: using LPI pending table @0x00000100140a0000
CPU5: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU6
GICv3: CPU6: found redistributor 6 region 0:0x00008010800c0000
CPU6: using LPI pending table @0x00000100140b0000
CPU6: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU7
GICv3: CPU7: found redistributor 7 region 0:0x00008010800e0000
CPU7: using LPI pending table @0x00000100140c0000
CPU7: Booted secondary processor [431f0a10]
Brought up 8 CPUs
SMP: Total of 8 processors activated.
CPU features: detected feature: GIC system register CPU interface
CPU features: detected feature: LSE atomic instructions
CPU features: detected feature: Software prefetching using PRFM
CPU: All CPU(s) started at EL2
alternatives: patching kernel code
devtmpfs: initialized
SMBIOS 3.0.0 present.
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
atomic64_test: passed
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
cpuidle: using governor menu
vdso: 2 pages (1 code @ fffffc0008980000, 1 data @ fffffc0008eb0000)
hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
DMA: preallocated 256 KiB pool for atomic allocations
Serial: AMBA PL011 UART driver
87e024000000.serial: ttyAMA0 at MMIO 0x87e024000000 (irq = 7, base_baud = 0) is a PL011 rev3
console [ttyAMA0] enabled
87e025000000.serial: ttyAMA1 at MMIO 0x87e025000000 (irq = 8, base_baud = 0) is a PL011 rev3
OF: /soc at 0/pci at 848000000000: arguments longer than property
OF: /soc at 0/pci at 849000000000: arguments longer than property
OF: /soc at 0/pci at 84a000000000: arguments longer than property
OF: /soc at 0/pci at 84b000000000: arguments longer than property
OF: /soc at 0/pci at 87e0c0000000: arguments longer than property
OF: /soc at 100000000000/pci at 848000000000: arguments longer than property
OF: /soc at 100000000000/pci at 849000000000: arguments longer than property
OF: /soc at 100000000000/pci at 84a000000000: arguments longer than property
OF: /soc at 100000000000/pci at 84b000000000: arguments longer than property
HugeTLB registered 2 MB page size, pre-allocated 0 pages
HugeTLB registered 512 MB page size, pre-allocated 0 pages
ACPI: Interpreter disabled.
arm-smmu 830000000000.smmu0: probing hardware configuration...
arm-smmu 830000000000.smmu0: SMMUv2 with:
arm-smmu 830000000000.smmu0: 	stage 1 translation
arm-smmu 830000000000.smmu0: 	stage 2 translation
arm-smmu 830000000000.smmu0: 	nested translation
arm-smmu 830000000000.smmu0: 	non-coherent table walk
arm-smmu 830000000000.smmu0: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 830000000000.smmu0: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 830000000000.smmu0: 	128 context banks (0 stage-2 only)
arm-smmu 830000000000.smmu0: 	Supported page sizes: 0x62215000
arm-smmu 830000000000.smmu0: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 830000000000.smmu0: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 830000000000.smmu0: registered 1 master devices
arm-smmu 831000000000.smmu1: probing hardware configuration...
arm-smmu 831000000000.smmu1: SMMUv2 with:
arm-smmu 831000000000.smmu1: 	stage 1 translation
arm-smmu 831000000000.smmu1: 	stage 2 translation
arm-smmu 831000000000.smmu1: 	nested translation
arm-smmu 831000000000.smmu1: 	non-coherent table walk
arm-smmu 831000000000.smmu1: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 831000000000.smmu1: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 831000000000.smmu1: 	128 context banks (0 stage-2 only)
arm-smmu 831000000000.smmu1: 	Supported page sizes: 0x62215000
arm-smmu 831000000000.smmu1: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 831000000000.smmu1: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 831000000000.smmu1: registered 2 master devices
arm-smmu 832000000000.smmu2: probing hardware configuration...
arm-smmu 832000000000.smmu2: SMMUv2 with:
arm-smmu 832000000000.smmu2: 	stage 1 translation
arm-smmu 832000000000.smmu2: 	stage 2 translation
arm-smmu 832000000000.smmu2: 	nested translation
arm-smmu 832000000000.smmu2: 	non-coherent table walk
arm-smmu 832000000000.smmu2: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 832000000000.smmu2: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 832000000000.smmu2: 	128 context banks (0 stage-2 only)
arm-smmu 832000000000.smmu2: 	Supported page sizes: 0x62215000
arm-smmu 832000000000.smmu2: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 832000000000.smmu2: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 832000000000.smmu2: registered 1 master devices
arm-smmu 833000000000.smmu3: probing hardware configuration...
arm-smmu 833000000000.smmu3: SMMUv2 with:
arm-smmu 833000000000.smmu3: 	stage 1 translation
arm-smmu 833000000000.smmu3: 	stage 2 translation
arm-smmu 833000000000.smmu3: 	nested translation
arm-smmu 833000000000.smmu3: 	non-coherent table walk
arm-smmu 833000000000.smmu3: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 833000000000.smmu3: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 833000000000.smmu3: 	128 context banks (0 stage-2 only)
arm-smmu 833000000000.smmu3: 	Supported page sizes: 0x62215000
arm-smmu 833000000000.smmu3: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 833000000000.smmu3: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 833000000000.smmu3: registered 1 master devices
arm-smmu 930000000000.smmu4: probing hardware configuration...
arm-smmu 930000000000.smmu4: SMMUv2 with:
arm-smmu 930000000000.smmu4: 	stage 1 translation
arm-smmu 930000000000.smmu4: 	stage 2 translation
arm-smmu 930000000000.smmu4: 	nested translation
arm-smmu 930000000000.smmu4: 	non-coherent table walk
arm-smmu 930000000000.smmu4: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 930000000000.smmu4: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 930000000000.smmu4: 	128 context banks (0 stage-2 only)
arm-smmu 930000000000.smmu4: 	Supported page sizes: 0x62215000
arm-smmu 930000000000.smmu4: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 930000000000.smmu4: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 930000000000.smmu4: registered 1 master devices
arm-smmu 931000000000.smmu5: probing hardware configuration...
arm-smmu 931000000000.smmu5: SMMUv2 with:
arm-smmu 931000000000.smmu5: 	stage 1 translation
arm-smmu 931000000000.smmu5: 	stage 2 translation
arm-smmu 931000000000.smmu5: 	nested translation
arm-smmu 931000000000.smmu5: 	non-coherent table walk
arm-smmu 931000000000.smmu5: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 931000000000.smmu5: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 931000000000.smmu5: 	128 context banks (0 stage-2 only)
arm-smmu 931000000000.smmu5: 	Supported page sizes: 0x62215000
arm-smmu 931000000000.smmu5: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 931000000000.smmu5: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 931000000000.smmu5: registered 1 master devices
arm-smmu 932000000000.smmu6: probing hardware configuration...
arm-smmu 932000000000.smmu6: SMMUv2 with:
arm-smmu 932000000000.smmu6: 	stage 1 translation
arm-smmu 932000000000.smmu6: 	stage 2 translation
arm-smmu 932000000000.smmu6: 	nested translation
arm-smmu 932000000000.smmu6: 	non-coherent table walk
arm-smmu 932000000000.smmu6: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 932000000000.smmu6: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 932000000000.smmu6: 	128 context banks (0 stage-2 only)
arm-smmu 932000000000.smmu6: 	Supported page sizes: 0x62215000
arm-smmu 932000000000.smmu6: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 932000000000.smmu6: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 932000000000.smmu6: registered 1 master devices
arm-smmu 933000000000.smmu7: probing hardware configuration...
arm-smmu 933000000000.smmu7: SMMUv2 with:
arm-smmu 933000000000.smmu7: 	stage 1 translation
arm-smmu 933000000000.smmu7: 	stage 2 translation
arm-smmu 933000000000.smmu7: 	nested translation
arm-smmu 933000000000.smmu7: 	non-coherent table walk
arm-smmu 933000000000.smmu7: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 933000000000.smmu7: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 933000000000.smmu7: 	128 context banks (0 stage-2 only)
arm-smmu 933000000000.smmu7: 	Supported page sizes: 0x62215000
arm-smmu 933000000000.smmu7: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 933000000000.smmu7: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 933000000000.smmu7: registered 1 master devices
iommu: Adding device 848000000000.pci to group 0
iommu: Adding device 849000000000.pci to group 1
iommu: Adding device 84a000000000.pci to group 2
iommu: Adding device 84b000000000.pci to group 3
iommu: Adding device 88001f000000.pci to group 4
iommu: Adding device 948000000000.pci to group 5
iommu: Adding device 949000000000.pci to group 6
iommu: Adding device 94a000000000.pci to group 7
iommu: Adding device 94b000000000.pci to group 8
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
DMA-API: preallocated 4096 debug entries
DMA-API: debugging enabled by kernel config
clocksource: Switched to clocksource arch_sys_counter
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 6, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 6, 4194304 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
UDP hash table entries: 65536 (order: 7, 10485760 bytes)
UDP-Lite hash table entries: 65536 (order: 7, 10485760 bytes)
NET: Registered protocol family 1
Unpacking initramfs...
------------[ cut here ]------------
kernel BUG at mm/page_alloc.c:1844!
Internal error: Oops - BUG: 0 [#1] SMP
Modules linked in:
CPU: 4 PID: 1 Comm: swapper/0 Tainted: G        W       4.8.0-rc4-00267-g664e88c #3
Hardware name: www.cavium.com ThunderX CRB-2S/ThunderX CRB-2S, BIOS 0.3 Aug 24 2016
task: fffffe0fee55a200 task.stack: ffffff00004a4000
PC is at move_freepages+0x158/0x168
LR is at move_freepages_block+0xac/0xc0
pc : [<fffffc0008227618>] lr : [<fffffc00082276d4>] pstate: 800000c5
sp : ffffff00004a7480
x29: ffffff00004a7480 x28: fffffdffc3fc0020 
x27: fffffdffc3fc0000 x26: 000000000000000a 
x25: 000000000000000a x24: ffffff0ffffa3190 
x23: ffffff0fffdbf530 x22: 0000000000000000 
x21: fffffdffc3ffffc0 x20: ffffff0ffffa2c80 
x19: fffffdffc3f80000 x18: fffffe0fdc0c2548 
x17: 0000000000000000 x16: 0000000100000000 
x15: 00000000000000c1 x14: 04000000001b2000 
x13: 00aedf01c7040000 x12: 00001b00000033f5 
x11: 01c604000000001b x10: 0000ae7401c50430 
x9 : 0000000000000000 x8 : fffffc0008f53000 
x7 : fffffc00082295f0 x6 : 0000000000000001 
x5 : 0000000000000000 x4 : fffffe0fffff2580 
x3 : ffffff0ffffa2500 x2 : 0000000000000001 
x1 : fffffe0fffff2580 x0 : ffffff0ffffa2c80 

Process swapper/0 (pid: 1, stack limit = 0xffffff00004a4020)
Stack: (0xffffff00004a7480 to 0xffffff00004a8000)
7480: ffffff00004a74e0 fffffc00082276d4 0000000000000000 ffffff0ffffa2c80
74a0: fffffdffc3f80000 0000000000000000 ffffff0fffdbf530 ffffff0ffffa3190
74c0: 000000000000000a 000000000000000a fffffdffc3fc0000 fffffdffc3fc0020
74e0: ffffff00004a7510 fffffc0008227f00 ffffff0ffffa2c80 0000000000000000
7500: fffffdffc3fc0000 0000000000000028 ffffff00004a75b0 fffffc0008229654
7520: ffffff0fffdbf530 ffffff0ffffa2c80 fffffc0008e5f520 0000000000000000
7540: ffffff0fffdbf530 ffffff0fffdbf520 0000030ff6f60000 0000000000000001
7560: 0000000000000000 0000000000000000 ffffff0ffffa3200 fffffc00082295f0
7580: fffffc0008e5f520 fffffc0008ec1000 0000000000000000 0000000000000001
75a0: 0000000000000010 01ffff0ffffa2c80 ffffff00004a76a0 fffffc000822a5ec
75c0: 00000000024200c2 0000000000000000 0000000000000000 00000000024200c2
75e0: fffffc0008ec4000 0000000000000001 0000000000000000 fffffc0008ec2000
7600: 00000000024200c2 fffffc00082886c8 ffffffffffffffff 0000000000000040
7620: 00000012004a7640 fffffc0008e587e8 0000000100000000 0000000000000000
7640: ffffff00024200c2 fffffc00081007bc ffffff0ffffa2c80 0000000000000010
7660: fffffc0008e5f520 0000000000000040 0000000000000000 ffffff0ffffa3200
7680: 0000000000000001 ffffff0ffffa3b80 ffffff0ffffa3b80 ffffff00004a77b8
76a0: ffffff00004a77e0 fffffc00082886c8 0000000000000000 fffffc0008ec0c48
76c0: ffffff0ffffa3b80 0000000000000000 0000000000000000 0000000000000001
76e0: ffffff00004a4000 0000000000000000 fffffe0fee55a200 00000000000053d4
7700: fffffe0fee55a200 0000000000000000 0000000000000000 fffffc0008eec160
7720: 0000000000000000 0000000000000002 fffffe0fee55aa98 fffffc0008ec2000
7740: fffffe0fee55a200 fffffc0009c95000 0000000000000000 fffffc0008eec160
7760: 0000000000000000 fffffc0008e587e8 0000000000000000 0000000100400000
7780: 0000000000000000 0000000000000000 0000000000000000 0000000000000002
77a0: 0000000000000000 0000000000000000 fffffc000821d070 ffffff0ffffa3b80
77c0: 0000000000000000 ffffff0ffffa3b80 0000000100000000 0000000000000000
77e0: ffffff00004a7810 fffffc0008288d20 fffffc0008e587e8 fffffe0fec01a8f0
7800: 00000000024200c2 fffffc000821e62c ffffff00004a7870 fffffc000821e62c
7820: 0000000000000000 ffffff00004a4000 00000000024200c2 fffffe0fe0249410
7840: fffffc000821e788 00000000024200c2 fffffc0008f01d28 0000000002493ee0
7860: 0000000000002c2c 0000000000000040 ffffff00004a78d0 fffffc000821e788
7880: 0000000000000000 000000000000000e 00000000024200c2 fffffe0fe0249410
78a0: 0000000000000001 fffffc0008fd76ac fffffc0008f01d28 0000000002493ee0
78c0: 0000000000002c2c fffffc000821e850 ffffff00004a7930 fffffc000821e9b0
78e0: fffffe0fe0249410 0000000000000001 0000000000000001 0000000000002c2c
7900: ffffff00004a7a20 fffffe0fcc140400 00000000000053d4 fffffc00089b8900
7920: 0000000000002c2c fffffc000821e440 ffffff00004a7960 fffffc00082eb758
7940: fffffe0fe0249410 0000000000000001 0000000000010000 ffffff00004a7b18
7960: ffffff00004a79a0 fffffc000821e3bc 0000000000010000 ffffff00004a7b18
7980: fffffe0fe0249410 0000000000010000 0000000000000000 fffffc000821e434
79a0: ffffff00004a7a30 fffffc00082200e0 0000000000000000 fffffe0fcc140400
79c0: fffffe0fe0249210 ffffff00004a7af0 fffffe0fe0249410 ffffff00004a7b18
79e0: fffffc0008d40e70 0000000000008000 fffffc0009040078 ffffff0f83f26000
7a00: 0000000000000000 fffffc0008bd0300 00002c2c00000001 ffffff00004a4000
7a20: fffffdff83f20840 0000000027e02140 ffffff00004a7a80 fffffc00082201fc
7a40: 0000000000008000 ffffff00004a7af0 fffffe0fe02492e8 ffffff00004a7b18
7a60: ffffff00004a7bb8 0000000000000007 fffffc0008d40e70 0000000000008000
7a80: ffffff00004a7ab0 fffffc00082ba89c fffffe0fcc140400 0000000000008000
7aa0: ffffff00004a7bb8 fffffe0fdd350000 ffffff00004a7b40 fffffc00082bb588
7ac0: 0000000000000000 0000000000008000 fffffe0fcc140400 fffffe0fdd350000
7ae0: fffffe0fdd350000 0000000000008000 fffffe0fcc140400 000000000000ac2c
7b00: 0000000000000000 0000000000000000 0000000000000000 fffffc0000000003
7b20: 00000000000053d4 0000000000002c2c ffffff00004a7ae0 0000000000000001
7b40: ffffff00004a7b80 fffffc00082bc5b4 fffffe0fcc140400 fffffe0fcc140400
7b60: fffffe0fdd350000 0000000000008000 fffffc0008babe88 ffffff0f8ea0e439
7b80: ffffff00004a7bc0 fffffc0008d120f0 0000000000000000 0000000000008000
7ba0: fffffe0fdd350000 0000000000000000 fffffe0fdc3fa280 000000000000ac2c
7bc0: ffffff00004a7bf0 fffffc0008d121d4 fffffc0008d75000 fffffc0008d75b60
7be0: fffffc0008d75bb8 fffffe0fdd350000 ffffff00004a7c10 fffffc0008d11eac
7c00: fffffc0008d75b60 0000000000008000 ffffff00004a7c40 fffffc0008d11f10
7c20: 0000000000008000 0000000000008000 fffffc0008d75b60 fffffc00089e42c0
7c40: ffffff00004a7c80 fffffc0008d410f4 fffffe0fdc3fa280 ffffff0f83f26000
7c60: 0000000000000000 fffffe0fdd350000 fffffc0008d11d74 fffffc0008d11ec8
7c80: ffffff00004a7cf0 fffffc0008d411b4 fffffc0008d75000 ffffff0f83f26000
7ca0: 000000000e7f20bc 0000000000000000 fffffc0008d4119c fffffc0008ff1f38
7cc0: fffffc0008babf90 fffffc0009040000 fffffc0008e3d9a0 0000000000000000
7ce0: 0000000008d75000 0000000000008000 ffffff00004a7d00 fffffc0008d127c4
7d00: ffffff00004a7d60 fffffc0008d1294c fffffc0009040000 fffffc0008d128dc
7d20: fffffc0009040000 fffffc0009040000 fffffc0009040000 fffffc0008d1045c
7d40: fffffc0008cfc468 fffffc0008d74290 0000000000001b57 fffffc0008c04958
7d60: ffffff00004a7d90 fffffc0008082ff4 ffffff00004a4000 fffffc0008d128dc
7d80: 0000000000000000 0000000000000005 ffffff00004a7e00 fffffc0008d10df4
7da0: 0000000000000101 fffffc0009040000 fffffc0008d742a8 0000000000000005
7dc0: fffffc0008e3d700 0000000000000000 fffffc0008ee2370 fffffc0008bb79f8
7de0: 0000000000000000 0000000500000005 0000000000000000 fffffc0008d1045c
7e00: ffffff00004a7ea0 fffffc000893c888 fffffc0009040000 fffffc0009040000
7e20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff00004a4000
7e60: 0000000000000003 0000000000000000 0000000000000000 0000000000000000
7e80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7ea0: 0000000000000000 fffffc0008082b80 fffffc000893c868 0000000000000000
7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000
7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
Call trace:
Exception stack(0xffffff00004a72a0 to 0xffffff00004a73d0)
72a0: fffffdffc3f80000 0000040000000000 ffffff00004a7480 fffffc0008227618
72c0: 00000000800000c5 000000000000003d fffffc0008ec4000 0000000000000030
72e0: ffffff00004a7300 fffffc00081593b0 fffffc0008f53000 0000000000000000
7300: ffffff00004a7320 fffffc000822abac fffffc0008fd76be ffffff00004a7438
7320: ffffff00004a7460 fffffc00082886c8 ffffff00004a7380 fffffc0008135d08
7340: 0000000000000001 0000000000000007 ffffff0ffffa2c80 fffffe0fffff2580
7360: 0000000000000001 ffffff0ffffa2500 fffffe0fffff2580 0000000000000000
7380: 0000000000000001 fffffc00082295f0 fffffc0008f53000 0000000000000000
73a0: 0000ae7401c50430 01c604000000001b 00001b00000033f5 00aedf01c7040000
73c0: 04000000001b2000 00000000000000c1
[<fffffc0008227618>] move_freepages+0x158/0x168
[<fffffc00082276d4>] move_freepages_block+0xac/0xc0
[<fffffc0008227f00>] __rmqueue+0x740/0x898
[<fffffc0008229654>] get_page_from_freelist+0x3e4/0xca0
[<fffffc000822a5ec>] __alloc_pages_nodemask+0x1ac/0xfd8
[<fffffc00082886c8>] alloc_page_interleave+0x60/0xb8
[<fffffc0008288d20>] alloc_pages_current+0x168/0x1c8
[<fffffc000821e62c>] __page_cache_alloc+0x17c/0x1c0
[<fffffc000821e788>] pagecache_get_page+0x118/0x2f8
[<fffffc000821e9b0>] grab_cache_page_write_begin+0x48/0x68
[<fffffc00082eb758>] simple_write_begin+0x40/0x168
[<fffffc000821e3bc>] generic_perform_write+0xc4/0x1b8
[<fffffc00082200e0>] __generic_file_write_iter+0x178/0x1c8
[<fffffc00082201fc>] generic_file_write_iter+0xcc/0x1c8
[<fffffc00082ba89c>] __vfs_write+0xcc/0x140
[<fffffc00082bb588>] vfs_write+0xa8/0x1c0
[<fffffc00082bc5b4>] SyS_write+0x54/0xb0
[<fffffc0008d120f0>] xwrite+0x34/0x7c
[<fffffc0008d121d4>] do_copy+0x9c/0xf4
[<fffffc0008d11eac>] write_buffer+0x34/0x50
[<fffffc0008d11f10>] flush_buffer+0x48/0xb8
[<fffffc0008d410f4>] __gunzip+0x27c/0x324
[<fffffc0008d411b4>] gunzip+0x18/0x20
[<fffffc0008d127c4>] unpack_to_rootfs+0x168/0x280
[<fffffc0008d1294c>] populate_rootfs+0x70/0x138
[<fffffc0008082ff4>] do_one_initcall+0x44/0x138
[<fffffc0008d10df4>] kernel_init_freeable+0x240/0x2e0
[<fffffc000893c888>] kernel_init+0x20/0xf8
[<fffffc0008082b80>] ret_from_fork+0x10/0x50
Code: 913dc021 9400d2db d4210000 d503201f (d4210000) 
---[ end trace fcdd7f8cb96cdb3d ]---
note: swapper/0[1] exited with preempt_count 1
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

SMP: stopping secondary CPUs
---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

From mboxrd@z Thu Jan  1 00:00:00 1970
From: will.deacon@arm.com (Will Deacon)
Date: Thu, 27 Oct 2016 17:01:36 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161017185801.GT25086@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
Message-ID: <20161027160136.GD24290@arm.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Hi Robert,

On Mon, Oct 17, 2016 at 08:58:01PM +0200, Robert Richter wrote:
> Mark, Will, any opinion here?

Having looking at this, I'm inclined to agree with you; pfn_valid() is
all about whether the underlying mem_map (struct page *) entry exists,
not about whether the page is mappable or not.

That said, setting the zone for pages representing NOMAP memory feels
like a slippery slope to losing information about them being NOMAP in
the first place and the whole problem getting out-of-hand. Whilst I'm
happy for pfn_valid() to return true (in the sense that we're within
bounds of mem_map etc), I'm less happy that we're also saying that the
struct page contains useful information, such as the zone and the node
information, which is then subsequently used by the NUMA code.

On top of that, pfn_valid is used in other places as a coarse "is this
memory?" check, and will cause things like ioremap to fail whereas it
wouldn't at the moment. It feels to me like NOMAP memory is a new type
of memory where there *is* a struct page, but it shouldn't be used for
anything. I don't think pfn_valid can describe that, given the way it's
currently used, and flipping the logic is just likely to move the problem
elsewhere.

What options do we have for fixing this in the NUMA code?

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Fri, 28 Oct 2016 11:19:05 +0200
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161027160136.GD24290@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
Message-ID: <20161028091905.GM22012@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 27.10.16 17:01:36, Will Deacon wrote:
> Hi Robert,
> 
> On Mon, Oct 17, 2016 at 08:58:01PM +0200, Robert Richter wrote:
> > Mark, Will, any opinion here?
> 
> Having looking at this, I'm inclined to agree with you; pfn_valid() is
> all about whether the underlying mem_map (struct page *) entry exists,
> not about whether the page is mappable or not.
> 
> That said, setting the zone for pages representing NOMAP memory feels
> like a slippery slope to losing information about them being NOMAP in
> the first place and the whole problem getting out-of-hand. Whilst I'm
> happy for pfn_valid() to return true (in the sense that we're within
> bounds of mem_map etc), I'm less happy that we're also saying that the
> struct page contains useful information, such as the zone and the node
> information, which is then subsequently used by the NUMA code.

Let's see it in a different way, pfns and the struct page assigned to
each of it is about *physical* memory. The system knows all the
memory, some is free, some reserved and some marked NOMAP. Regardless
of the mapping of the page the mm code maintains and uses that
information.

There are assumptions on validity and checks in the code that now
cause problems due to partly or non-existing data about nomap pages.
This inconsistency is dangerous since a problem may occur any time
then the page area is accessed first, thus a system may crash randomly
depending on the memory access. Luckily, in my case it triggered
reproducible while loading initrd during boot.

I also think that this is not only NUMA related. E.g. the following
bug report is probably also related:

 https://bugzilla.redhat.com/show_bug.cgi?id=1387793

> On top of that, pfn_valid is used in other places as a coarse "is this
> memory?" check, and will cause things like ioremap to fail whereas it
> wouldn't at the moment.

IMO this is a misuse of pfn_valid() that needs to be fixed with
additional checks, e.g. traversing memblocks.

> It feels to me like NOMAP memory is a new type
> of memory where there *is* a struct page, but it shouldn't be used for
> anything.

IMO, a NOMAP page should just be handled like a reserved page except
that the page is marked reserved. See free_low_memory_core_early().
Thus, NOMAP pages are not in the free pages list or set to reserved.
It is simply not available for mapping at all. Isn't that exactly what
it should be?

I also did not yet understand the benefit of the differentiation
between NOMAP and reserved and the original motivation for its
implementation. I looked through the mail threads but could not find
any hint. The only difference I see now is that it is not listed as a
reserved page, but as long as it is not freed it should behave the
same. I remember the case to handle memory different (coherency,
etc.), but are not sure here. Ard, could you explain this?

> I don't think pfn_valid can describe that, given the way it's
> currently used, and flipping the logic is just likely to move the problem
> elsewhere.
> 
> What options do we have for fixing this in the NUMA code?

Out of my mind:

1) Treat NOMAP pages same as reserved pages (my patch).

2) Change mm code to allow arch specific early_pfn_valid().

3) Fix mm code to only access stuct page (of a zone) if pfn_valid() is
   true.

There can be more alternatives. IMO:

 * We shouldn't touch generic mm code.

 * We should maintain a valid struct page for all pages in a sections.

 * We should only traverse memblock where really necessary (arm64
   only).

 * I don't think this problem is numa specific.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Tue, 1 Nov 2016 17:55:44 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
Message-ID: <20161101165544.GP22012@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 06.10.16 11:52:07, Robert Richter wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
> 
>  kernel BUG at mm/page_alloc.c:1848!
> 
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
> 
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
> 
>  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> 
> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
> 
> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().

Below a reproducer for non-numa systems. Note that invalidating the
node id just simulates a different node in reality.

The patch injects a (pageblock_order) unaligned NOMAP mem range at the
end of a memory block and then tries to free that area. This causes a
BUG_ON() (log attached).

-Robert



>>From 20d853e300c99be5420c7ee3f072c318804cac1b Mon Sep 17 00:00:00 2001
From: root <root@10.18.240.201>
Date: Tue, 1 Nov 2016 15:04:43 +0000
Subject: [PATCH] mm-fault-reproducer

Signed-off-by: root <root@10.18.240.201>
---
 arch/arm64/mm/init.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/page_alloc.c      |  4 ++-
 2 files changed, 81 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 21c489bdeb4e..feaa7ab97551 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -36,6 +36,7 @@
 #include <linux/efi.h>
 #include <linux/swiotlb.h>
 #include <linux/vmalloc.h>
+#include <linux/page-isolation.h>
 
 #include <asm/boot.h>
 #include <asm/fixmap.h>
@@ -301,6 +302,80 @@ void __init arm64_memblock_init(void)
 	memblock_allow_resize();
 }
 
+static struct page *inject_pageblock;
+
+static void __init inject_nomap_create(void)
+{
+	phys_addr_t start, end;
+	unsigned long start_pfn, end_pfn;
+	u64 i;
+	int ret = -ENOMEM;
+
+	pr_info("%s: PAGES_PER_SECTION=%08lx pageblock_nr_pages=%08lx PAGE_SIZE=%08lx\n",
+		__func__, PAGES_PER_SECTION, pageblock_nr_pages, PAGE_SIZE);
+
+	/*
+	 * find a mem range with a complet pageblock in it
+	 */
+	for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end, NULL) {
+		start_pfn = PFN_DOWN(start);
+		end_pfn = PFN_UP(end);
+		if  (end_pfn - (start_pfn & ~(pageblock_nr_pages-1)) > 2 * pageblock_nr_pages)
+			break;
+	}
+
+	if (i == ULLONG_MAX)
+		goto fail;
+
+	start = PFN_PHYS(start_pfn);
+	end = PFN_PHYS(end_pfn) - 1;
+
+	pr_info("%s: Injecting into range: [%pa-%pa]\n", __func__, &start, &end);
+
+	/* mark the upper 5 pages nomap of a complete pageblock */
+	start_pfn = end_pfn & ~(pageblock_nr_pages-1);
+	start_pfn -= 5;			/* unalign by 5 pages */
+
+	start = PFN_PHYS(start_pfn);
+	end = PFN_PHYS(end_pfn) - 1;
+
+	ret = memblock_mark_nomap(start, end - start + 1);
+	if (ret)
+		goto fail;
+
+	inject_pageblock = pfn_to_page(start_pfn & ~(pageblock_nr_pages-1));
+
+	pr_info("%s: Injected nomap range at: [%pa-%pa] zones: %p %p\n", __func__,
+		&start, &end, page_zone(inject_pageblock),
+		page_zone(inject_pageblock + pageblock_nr_pages - 1));
+
+	return;
+fail:
+	pr_err("%s: Could not inject_unaligned_range: %d\n", __func__, ret);
+}
+
+static void __init inject_nomap_move(void)
+{
+	phys_addr_t start, end;
+	int ret;
+
+	if (!inject_pageblock)
+		return;
+
+	start = PFN_PHYS(page_to_pfn(inject_pageblock));
+	end = PFN_PHYS(page_to_pfn(inject_pageblock) + pageblock_nr_pages) - 1;
+
+	pr_info("%s: Moving [%pa-%pa] zones: %p %p\n", __func__,
+		&start, &end, page_zone(inject_pageblock),
+		page_zone(inject_pageblock + pageblock_nr_pages - 1));
+
+	ret = move_freepages_block(page_zone(inject_pageblock),
+				inject_pageblock,
+				gfpflags_to_migratetype(GFP_KERNEL));
+
+	pr_info("%s: Moved %d pages\n", __func__, ret);
+}
+
 void __init bootmem_init(void)
 {
 	unsigned long min, max;
@@ -320,6 +395,7 @@ void __init bootmem_init(void)
 	arm64_memory_present();
 
 	sparse_init();
+	inject_nomap_create();
 	zone_sizes_init(min, max);
 
 	high_memory = __va((max << PAGE_SHIFT) - 1) + 1;
@@ -479,6 +555,8 @@ void __init mem_init(void)
 		 */
 		sysctl_overcommit_memory = OVERCOMMIT_ALWAYS;
 	}
+
+	inject_nomap_move();
 }
 
 void free_initmem(void)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2b3bf6767d54..19d74637e242 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5077,8 +5077,10 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
 		if (context != MEMMAP_EARLY)
 			goto not_early;
 
-		if (!early_pfn_valid(pfn))
+		if (!early_pfn_valid(pfn)) {
+			set_page_node(pfn_to_page(pfn), NUMA_NO_NODE);
 			continue;
+		}
 		if (!early_pfn_in_nid(pfn, nid))
 			continue;
 		if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
-- 
2.9.3

-------------- next part --------------
A non-text attachment was scrubbed...
Name: typescript-crb2s-test21-201611010941-trigger-mm-fault.xz
Type: application/x-xz
Size: 10420 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20161101/7eae9153/attachment-0001.xz>

From mboxrd@z Thu Jan  1 00:00:00 1970
From: will.deacon@arm.com (Will Deacon)
Date: Mon, 7 Nov 2016 21:05:14 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161028091905.GM22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
Message-ID: <20161107210514.GP20591@arm.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On Fri, Oct 28, 2016 at 11:19:05AM +0200, Robert Richter wrote:
> On 27.10.16 17:01:36, Will Deacon wrote:
> > It feels to me like NOMAP memory is a new type
> > of memory where there *is* a struct page, but it shouldn't be used for
> > anything.
> 
> IMO, a NOMAP page should just be handled like a reserved page except
> that the page is marked reserved. See free_low_memory_core_early().
> Thus, NOMAP pages are not in the free pages list or set to reserved.
> It is simply not available for mapping at all. Isn't that exactly what
> it should be?
> 
> I also did not yet understand the benefit of the differentiation
> between NOMAP and reserved and the original motivation for its
> implementation. I looked through the mail threads but could not find
> any hint. The only difference I see now is that it is not listed as a
> reserved page, but as long as it is not freed it should behave the
> same. I remember the case to handle memory different (coherency,
> etc.), but are not sure here. Ard, could you explain this?
> 
> > I don't think pfn_valid can describe that, given the way it's
> > currently used, and flipping the logic is just likely to move the problem
> > elsewhere.
> > 
> > What options do we have for fixing this in the NUMA code?
> 
> Out of my mind:
> 
> 1) Treat NOMAP pages same as reserved pages (my patch).

Just to reiterate here, but your patch as it stands will break other parts
of the kernel. For example, acpi_os_ioremap relies on being able to ioremap
these regions afaict.

I think any solution involving pfn_valid is just going to move the crash
around.

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Wed, 9 Nov 2016 20:51:32 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161107210514.GP20591@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
Message-ID: <20161109195132.GZ22012@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Will,

On 07.11.16 21:05:14, Will Deacon wrote:
> Just to reiterate here, but your patch as it stands will break other parts
> of the kernel. For example, acpi_os_ioremap relies on being able to ioremap
> these regions afaict.
> 
> I think any solution involving pfn_valid is just going to move the crash
> around.

Let me describe the crash more detailed.

Following range is marked nomap (full efi map below):

[    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*

The mem belongs to this nodes:

[    0.000000] NUMA: Adding memblock [0x1400000 - 0xfffffffff] on node 0
[    0.000000] NUMA: Adding memblock [0x10000400000 - 0x10fffffffff] on node 1

The following mem ranges are created:

[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000001400000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x0000010fffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000001400000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x0000000fffffffff]
[    0.000000]   node   1: [mem 0x0000010000400000-0x0000010ff9e7ffff]
[    0.000000]   node   1: [mem 0x0000010ff9e80000-0x0000010ff9f1ffff]
[    0.000000]   node   1: [mem 0x0000010ff9f20000-0x0000010ffaeaffff]
[    0.000000]   node   1: [mem 0x0000010ffaeb0000-0x0000010ffaffffff]
[    0.000000]   node   1: [mem 0x0000010ffb000000-0x0000010ffffaffff]
[    0.000000]   node   1: [mem 0x0000010ffffb0000-0x0000010fffffffff]

The last range 0x0000010ffffb0000-0x0000010fffffffff is correctly
marked as a nomap area.

Paging is then initalized in free_area_init_core() (mm/page_alloc.c)
which calls memmap_init_zone(). This initializes all pages (struct
page) of the zones for each node except for pfns from nomap memory. It
uses early_pfn_valid() which is bound to pfn_valid().

In 4.4 *all* pages of a zone were initialized. Now page from nomap
ranges are skipped, and e.g. pfn 0x10ffffb has an uninitalized struct
page.  Note that nomap mem ranges are still part of the memblock list
and also part of the zone ranges within start_pfn and end_pfn, but
those don't have a valid struct page now. IMO this is a bug.

Later on all mappable memory is reserved and all pages of it are freed
by adding them to the free-pages list except for the reserved
memblocks.

Now the initrd is loaded. This reserves free memory by calling
move_freepages_block(). A block has the size of pageblock_nr_pages
which is in my configuration 0x2000. The kernel choses the block

 0x0000010fe0000000-0x0000010fffffffff

that also contains the nomap region around 10ffb000000.

In move_freepages() the pages of the zone are freed. The code accesses
pfn #10fffff but the struct page is uninitialized and thus node is 0.
This is different to #10fe000 and the zone which is node 1. This
causes the BUG_ON() to trigger:

[    4.173521] Unpacking initramfs...
[    8.420710] ------------[ cut here ]------------
[    8.425344] kernel BUG at mm/page_alloc.c:1844!
[    8.429870] Internal error: Oops - BUG: 0 [#1] SMP

I believe this is not the only case where the mm code relies on a
valid struct page for all pfns of a zone, thus modifying mm code to
make this case work is not an option. (E.g. this could be done by
moving the early_pfn_valid() check at the beginning of the loop in
memmap_init_zone().)

So for a fix we need to change early_pfn_valid() which is mapped to
pfn_valid() for SPARSEMEM. Using a different function than pfn_valid()
for this does not look reasonable. The only option is to revert
pfn_valid() to it's original bahaviour.

My fix now uses again memblock_is_memory() for pfn_valid() instead of
memblock_is_map_memory(). So I needed to change some users of
pfn_valid() to use memblock_is_map_memory() there necessary. This is
only required for arch specific code, all other uses of pfn_valid()
are save to use memblock_is_memory().

Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
keeps the same behaviour as before since it still uses memblock_is_
memory(). Could you more describe your concerns why do you think this
patch breaks the kernel and moves the problem somewhere else? I
believe it fixes the problem at all.

Thanks,

-Robert



[    0.000000] efi: Processing EFI memory map:
[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x1ffe800000 reserved size = 0x10537
[    0.000000]  memory.cnt  = 0x2
[    0.000000]  memory[0x0]     [0x00000001400000-0x00000fffffffff], 0xffec00000 bytes flags: 0x0
[    0.000000]  memory[0x1]     [0x00010000400000-0x00010fffffffff], 0xfffc00000 bytes flags: 0x0
[    0.000000]  reserved.cnt  = 0x1
[    0.000000]  reserved[0x0]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
[    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000001400000-0x0000000147ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000001480000-0x0000024affff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000001480000-0x000000024affff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000024b0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x000000024b0000-0x000000211fffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000021200000-0x0000002121ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000021220000-0x000000fffeffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x000000fffe0000-0x000000ffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x000000ffff0000-0x000000ffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x000000ffff0000-0x000000ffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000100000000-0x00000ff7ffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000ff8000000-0x00000ff801ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000ff8020000-0x00000fffa9ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00000fffa90000-0x00000fffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010000400000-0x010f812b3fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010000400000-0x00010f812bffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f812b4000-0x010f812b6fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f812b0000-0x00010f812bffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f812b7000-0x010f822e6fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f812b0000-0x00010f822effff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f822e7000-0x010f822f6fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f822e0000-0x00010f822fffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f822f7000-0x010f82342fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f822f0000-0x00010f8234ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f82343000-0x010f91e73fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f82340000-0x00010f91e7ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f91e74000-0x010f91e74fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f91e70000-0x00010f91e7ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f91e75000-0x010f92c98fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f91e70000-0x00010f92c9ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f92c99000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f92c90000-0x00010f9388ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010f93880000-0x00010ff788ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff7880000-0x00010ff788ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff7880000-0x00010ff78affff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff78a0000-0x00010ff9e8ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ff9e80000-0x00010ff9f1ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ff9f10000-0x00010ffaebffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffaeb0000-0x00010ffafcffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffafc0000-0x00010ffafcffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffafc0000-0x00010ffaffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ffaff0000-0x00010ffb00ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010ffb000000-0x00010fffe2ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffe20000-0x00010fffe3ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffe40000-0x00010fffe5ffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffe50000-0x00010ffffbffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffffb0000-0x00010ffffcffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]*
[    0.000000] memblock_add: [0x00010ffffc0000-0x00010fffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] memblock_add: [0x00010fffff0000-0x00010fffffffff] flags 0x0 early_init_dt_add_memory_arch+0x54/0x5c
[    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
[    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
[    0.000000] memblock_reserve: [0x00010f812b0000-0x00010f812bffff] flags 0x0 efi_init+0x208/0x2c8
[    0.000000] memblock_add: [0x00010f82340000-0x00010f91e7ffff] flags 0x0 arm64_memblock_init+0x16c/0x248
[    0.000000] memblock_reserve: [0x00010f82340000-0x00010f91e7ffff] flags 0x0 arm64_memblock_init+0x178/0x248
[    0.000000] memblock_reserve: [0x00000001480000-0x000000024affff] flags 0x0 arm64_memblock_init+0x1b0/0x248
[    0.000000] memblock_reserve: [0x00010f82343000-0x00010f91e73613] flags 0x0 arm64_memblock_init+0x1cc/0x248
[    0.000000] memblock_reserve: [0x000000c0000000-0x000000dfffffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] cma: Reserved 512 MiB at 0x00000000c0000000
[    0.000000] memblock_reserve: [0x00010ffffa0000-0x00010ffffaffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff90000-0x00010ffff9ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff80000-0x00010ffff8ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff70000-0x00010ffff7ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff60000-0x00010ffff6ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff50000-0x00010ffff5ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff40000-0x00010ffff4ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffff30000-0x00010ffff3ffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000]    memblock_free: [0x00010ffffa0000-0x00010ffffaffff] paging_init+0x5c0/0x610
[    0.000000]    memblock_free: [0x00000002490000-0x000000024affff] paging_init+0x5f4/0x610
[    0.000000] memblock_reserve: [0x00010fffefed80-0x00010ffff2fffb] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffffaffd0-0x00010ffffafffe] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffffaffa0-0x00010ffffaffce] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] memblock_reserve: [0x00010ffffa0000-0x00010ffffa000f] flags 0x0 numa_init+0x88/0x3f0
[    0.000000] NUMA: Adding memblock [0x1400000 - 0xfffffffff] on node 0
[    0.000000] NUMA: Adding memblock [0x10000400000 - 0x10fffffffff] on node 1
[    0.000000] NUMA: parsing numa-distance-map-v1
[    0.000000] NUMA: Initmem setup node 0 [mem 0x01400000-0xfffffffff]
[    0.000000] memblock_reserve: [0x00000fffffe500-0x00000fffffffff] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] NUMA: NODE_DATA [mem 0xfffffe500-0xfffffffff]
[    0.000000] NUMA: Initmem setup node 1 [mem 0x10000400000-0x10fffffffff]
[    0.000000] memblock_reserve: [0x00010ffffae480-0x00010ffffaff7f] flags 0x0 memblock_alloc_range_nid+0x30/0x58
[    0.000000] NUMA: NODE_DATA [mem 0x10ffffae480-0x10ffffaff7f]
...
[    8.420710] ------------[ cut here ]------------
[    8.425344] kernel BUG at mm/page_alloc.c:1844!
[    8.429870] Internal error: Oops - BUG: 0 [#1] SMP
[    8.434654] Modules linked in:
[    8.437712] CPU: 72 PID: 1 Comm: swapper/0 Tainted: G        W       4.8.1.4.vanilla10-00007-g9eb3a76b8b88 #111
[    8.447788] Hardware name: www.cavium.com ThunderX CRB-2S/ThunderX CRB-2S, BIOS 0.3 Sep 13 2016
[    8.456474] task: ffff800fee626800 task.stack: ffff800fec02c000
[    8.462404] PC is at move_freepages+0x198/0x1b0
[    8.466924] LR is at move_freepages_block+0xa8/0xb8
[    8.471791] pc : [<ffff0000081e5dd0>] lr : [<ffff0000081e5e90>] pstate: 000000c5
[    8.479174] sp : ffff800fec02f4d0
[    8.482478] x29: ffff800fec02f4d0 x28: 0000000000000001 
[    8.487786] x27: 0000000000000000 x26: 0000000000000000 
[    8.493093] x25: 000000000000000a x24: ffff7fe043fd0020 
[    8.498401] x23: ffff810ffffaec00 x22: 0000000000000000 
[    8.503708] x21: ffff7fe043ffffc0 x20: 0000000000000000 
[    8.509015] x19: ffff7fe043f80000 x18: ffff8100102f2548 
[    8.514322] x17: 0000000000000000 x16: 0000000100000000 
[    8.519630] x15: 0000000000000007 x14: 0000010200000000 
[    8.524938] x13: 0004a41400000000 x12: 00032c920000001a 
[    8.530245] x11: 0000010200000000 x10: 0004a40800000000 
[    8.535553] x9 : 0000000000000040 x8 : 0000000000000000 
[    8.540860] x7 : 0000000000000000 x6 : ffff810ffffaf120 
[    8.546168] x5 : 0000000000000001 x4 : ffff000008d70c48 
[    8.551475] x3 : ffff810ffffae480 x2 : 0000000000000000 
[    8.556782] x1 : 0000000000000001 x0 : ffff810ffffaec00 
[    8.562089] 
[    8.563571] Process swapper/0 (pid: 1, stack limit = 0xffff800fec02c020)
[    8.570261] Stack: (0xffff800fec02f4d0 to 0xffff800fec030000)
...
[    9.279652] Call trace:
[    9.282091] Exception stack(0xffff800fec02f300 to 0xffff800fec02f430)
[    9.288520] f300: ffff7fe043f80000 0001000000000000 ffff800fec02f4d0 ffff0000081e5dd0
[    9.296339] f320: 0000000000000001 ffff800fee626800 0000000000000000 00000000026012d0
[    9.304157] f340: 0000000100000000 ffffffffffffffff 0000000000000130 0000000000000040
[    9.311976] f360: 0000001200000000 ffff000008cf8198 0000000100000000 0000000000000001
[    9.319794] f380: 00000000026012d0 ffff810ff9a1e8e8 0000000000000000 0000000000000040
[    9.327613] f3a0: ffff810ffffaec00 0000000000000001 0000000000000000 ffff810ffffae480
[    9.335431] f3c0: ffff000008d70c48 0000000000000001 ffff810ffffaf120 0000000000000000
[    9.343250] f3e0: 0000000000000000 0000000000000040 0004a40800000000 0000010200000000
[    9.351068] f400: 00032c920000001a 0004a41400000000 0000010200000000 0000000000000007
[    9.358885] f420: 0000000100000000 0000000000000000
[    9.363756] [<ffff0000081e5dd0>] move_freepages+0x198/0x1b0
[    9.369318] [<ffff0000081e5e90>] move_freepages_block+0xa8/0xb8
[    9.375227] [<ffff0000081e657c>] __rmqueue+0x604/0x650
[    9.380355] [<ffff0000081e7898>] get_page_from_freelist+0x3f0/0xb88
[    9.386612] [<ffff0000081e860c>] __alloc_pages_nodemask+0x12c/0xce8
[    9.392876] [<ffff00000823dc1c>] alloc_page_interleave+0x64/0xc0
[    9.398873] [<ffff00000823e2f8>] alloc_pages_current+0x108/0x168
[    9.404870] [<ffff0000081de1cc>] __page_cache_alloc+0x104/0x140
[    9.410779] [<ffff0000081de31c>] pagecache_get_page+0x114/0x300
[    9.416688] [<ffff0000081de550>] grab_cache_page_write_begin+0x48/0x68
[    9.423207] [<ffff00000828e5f8>] simple_write_begin+0x40/0x150
[    9.429029] [<ffff0000081ddfe0>] generic_perform_write+0xb8/0x1a0
[    9.435112] [<ffff0000081df8c8>] __generic_file_write_iter+0x170/0x1b0
[    9.441628] [<ffff0000081df9d4>] generic_file_write_iter+0xcc/0x1c8
[    9.447893] [<ffff000008262eb4>] __vfs_write+0xcc/0x140
[    9.453108] [<ffff000008263b84>] vfs_write+0xa4/0x1c0
[    9.458150] [<ffff000008264bc4>] SyS_write+0x54/0xb0
[    9.463108] [<ffff000008be1fe4>] xwrite+0x34/0x7c
[    9.467801] [<ffff000008be20c8>] do_copy+0x9c/0xf4
[    9.472583] [<ffff000008be1da4>] write_buffer+0x34/0x50
[    9.477796] [<ffff000008be1e08>] flush_buffer+0x48/0xb4
[    9.483016] [<ffff000008c0fe08>] __gunzip+0x288/0x334
[    9.488057] [<ffff000008c0fecc>] gunzip+0x18/0x20
[    9.492750] [<ffff000008be26bc>] unpack_to_rootfs+0x168/0x284
[    9.498486] [<ffff000008be2848>] populate_rootfs+0x70/0x138
[    9.504051] [<ffff000008082ff4>] do_one_initcall+0x44/0x138
[    9.509613] [<ffff000008be0d04>] kernel_init_freeable+0x1ac/0x24c
[    9.515699] [<ffff000008847f20>] kernel_init+0x20/0xf8
[    9.520826] [<ffff000008082b80>] ret_from_fork+0x10/0x50
[    9.526129] Code: 910a0021 9400b47d d4210000 d503201f (d4210000) 
[    9.532233] ---[ end trace c3040dccdcf12d3a ]---
[    9.536889] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

From mboxrd@z Thu Jan  1 00:00:00 1970
From: will.deacon@arm.com (Will Deacon)
Date: Thu, 17 Nov 2016 14:25:29 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161109195132.GZ22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
Message-ID: <20161117142528.GJ22855@arm.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
> Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
> keeps the same behaviour as before since it still uses memblock_is_
> memory(). Could you more describe your concerns why do you think this
> patch breaks the kernel and moves the problem somewhere else? I
> believe it fixes the problem at all.

acpi_os_ioremap always ends up in __ioremap_caller, regardless of
memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Thu, 17 Nov 2016 16:18:05 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161117142528.GJ22855@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
Message-ID: <20161117151805.GJ2151@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Thanks for your answer.

On 17.11.16 14:25:29, Will Deacon wrote:
> On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
> > Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
> > keeps the same behaviour as before since it still uses memblock_is_
> > memory(). Could you more describe your concerns why do you think this
> > patch breaks the kernel and moves the problem somewhere else? I
> > believe it fixes the problem at all.
> 
> acpi_os_ioremap always ends up in __ioremap_caller, regardless of
> memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.

But that's the reason my patch changed the code to use memblock_is_
map_memory() instead. I was looking into the users of pfn_valid() esp.
in arm64 code and changed it where required.

This week I looked into the kernel again for code that might break by
a pfn_valid() change. I found try_ram_remap() in memremap.c that has
changed behaviour now, but this is explicit for MEMREMAP_WB, so it
should be fine.

Maybe it might be better to use page_is_ram() in addition to
pfn_valid() where necessary. This should work now after commit:

 e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem

I still think pfn_valid() is not the correct use to determine the mem
attributes for mappings, there are further checks required.

The risk of breaking something with my patch is small and limited only
to the mapping of efi reserved regions (which is the state of 4.4). If
something breaks anyway it can easily be fixed by adding more checks
to pfn_valid() as suggested above.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Sun, 20 Nov 2016 17:07:44 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161117151805.GJ2151@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com> <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com> <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com> <20161117151805.GJ2151@rric.localdomain>
Message-ID: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 17 November 2016 at 15:18, Robert Richter <robert.richter@cavium.com> wrote:
> Thanks for your answer.
>
> On 17.11.16 14:25:29, Will Deacon wrote:
>> On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
>> > Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
>> > keeps the same behaviour as before since it still uses memblock_is_
>> > memory(). Could you more describe your concerns why do you think this
>> > patch breaks the kernel and moves the problem somewhere else? I
>> > believe it fixes the problem at all.
>>
>> acpi_os_ioremap always ends up in __ioremap_caller, regardless of
>> memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.
>
> But that's the reason my patch changed the code to use memblock_is_
> map_memory() instead. I was looking into the users of pfn_valid() esp.
> in arm64 code and changed it where required.
>
> This week I looked into the kernel again for code that might break by
> a pfn_valid() change. I found try_ram_remap() in memremap.c that has
> changed behaviour now, but this is explicit for MEMREMAP_WB, so it
> should be fine.
>
> Maybe it might be better to use page_is_ram() in addition to
> pfn_valid() where necessary. This should work now after commit:
>
>  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>
> I still think pfn_valid() is not the correct use to determine the mem
> attributes for mappings, there are further checks required.
>
> The risk of breaking something with my patch is small and limited only
> to the mapping of efi reserved regions (which is the state of 4.4). If
> something breaks anyway it can easily be fixed by adding more checks
> to pfn_valid() as suggested above.
>

As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
the correct way to address this. However, doing that does uncover a
bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
page fields before the pfn_valid_within() check, so it seems those
need to be switched around.

Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
for sparsemem. Care to elaborate why?

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Wed, 23 Nov 2016 22:15:38 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
References: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
Message-ID: <20161123211538.GH10776@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 20.11.16 17:07:44, Ard Biesheuvel wrote:
> On 17 November 2016 at 15:18, Robert Richter <robert.richter@cavium.com> wrote:

> > The risk of breaking something with my patch is small and limited only
> > to the mapping of efi reserved regions (which is the state of 4.4). If
> > something breaks anyway it can easily be fixed by adding more checks
> > to pfn_valid() as suggested above.
> >
> 
> As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
> the correct way to address this. However, doing that does uncover a
> bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
> page fields before the pfn_valid_within() check, so it seems those
> need to be switched around.
> 
> Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
> for sparsemem. Care to elaborate why?

HOLES_IN_ZONE is of rare use in the kernel. I think it was introduced
to save memory for the memmap and only some single systems enable it.
There is no architecture that enables it entirely. For good reasons...

It introduces additional checks. pfn_valid() is usually checked only
once for the whole memmap. There are a number of checks enabled, just
grep for pfn_valid_within. This will increase the number of
pfn_valid() calls by a factor of MAX_ORDER_NR_PAGES, in my config this
is 8k. So, this is not the direction to go.

My patch fixes a regression in the kernel that was introduced by the
nomap implementation. Some systems can not boot anymore, beside of
that the BUG_ON() may occur any time depending only on physical page
access, we need to fix 4.9. Here is a reproducer:

 https://patchwork.kernel.org/patch/9407677/

My patch also does not break memremap(). With my patch applied
try_ram_remap() would return a linear addr for nomap regions. But this
is only called if WB is explicitly requested, so it should not happen.
If you think pfn_valid() is wrong here, I am happy to send a patch
that fixes this by using page_is_ram(). In any case, the worst case
that may happen is to behave the same as v4.4, we might fix then the
wrong use of pfn_valid() where it is not correctly used to check for
ram.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Wed, 23 Nov 2016 21:25:06 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161123211538.GH10776@rric.localdomain>
References: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com> <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com> <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com> <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
Message-ID: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 23 November 2016 at 21:15, Robert Richter <robert.richter@cavium.com> wrote:
> On 20.11.16 17:07:44, Ard Biesheuvel wrote:
>> On 17 November 2016 at 15:18, Robert Richter <robert.richter@cavium.com> wrote:
>
>> > The risk of breaking something with my patch is small and limited only
>> > to the mapping of efi reserved regions (which is the state of 4.4). If
>> > something breaks anyway it can easily be fixed by adding more checks
>> > to pfn_valid() as suggested above.
>> >
>>
>> As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
>> the correct way to address this. However, doing that does uncover a
>> bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
>> page fields before the pfn_valid_within() check, so it seems those
>> need to be switched around.
>>
>> Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
>> for sparsemem. Care to elaborate why?
>
> HOLES_IN_ZONE is of rare use in the kernel. I think it was introduced
> to save memory for the memmap and only some single systems enable it.
> There is no architecture that enables it entirely. For good reasons...
>
> It introduces additional checks. pfn_valid() is usually checked only
> once for the whole memmap. There are a number of checks enabled, just
> grep for pfn_valid_within. This will increase the number of
> pfn_valid() calls by a factor of MAX_ORDER_NR_PAGES, in my config this
> is 8k. So, this is not the direction to go.
>

That does sound like a potential issue. But does it cause any slowdown
in practice?

The reality is that CONFIG_HOLES_IN_ZONE perfectly describes the
situation, and so it is still my preferred option if the performance
hit is tolerable.

> My patch fixes a regression in the kernel that was introduced by the
> nomap implementation. Some systems can not boot anymore, beside of
> that the BUG_ON() may occur any time depending only on physical page
> access, we need to fix 4.9. Here is a reproducer:
>
>  https://patchwork.kernel.org/patch/9407677/
>
> My patch also does not break memremap(). With my patch applied
> try_ram_remap() would return a linear addr for nomap regions. But this
> is only called if WB is explicitly requested, so it should not happen.

Why? MEMREMAP_WB is used often, among other things for mapping
firmware tables, which are marked as NOMAP, so in these cases, the
linear address is not mapped.

> If you think pfn_valid() is wrong here, I am happy to send a patch
> that fixes this by using page_is_ram(). In any case, the worst case
> that may happen is to behave the same as v4.4, we might fix then the
> wrong use of pfn_valid() where it is not correctly used to check for
> ram.
>

page_is_ram() uses string comparisons to look for regions called
'System RAM'. Is that something we can tolerate for each pfn_valid()
calll?

Perhaps the solution is to reimplement page_is_ram() for arm64 using
memblock_is_memory() instead, But that still means we need to modify
the generic memremap() code first to switch to it before changing the
arm64 implementation of pfn_valid

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Thu, 24 Nov 2016 14:42:38 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
References: <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
Message-ID: <20161124134238.GI10776@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> Why? MEMREMAP_WB is used often, among other things for mapping
> firmware tables, which are marked as NOMAP, so in these cases, the
> linear address is not mapped.

If fw tables are mapped wb, that is wrong and needs a separate fix.

> > If you think pfn_valid() is wrong here, I am happy to send a patch
> > that fixes this by using page_is_ram(). In any case, the worst case
> > that may happen is to behave the same as v4.4, we might fix then the
> > wrong use of pfn_valid() where it is not correctly used to check for
> > ram.
> >
> 
> page_is_ram() uses string comparisons to look for regions called
> 'System RAM'. Is that something we can tolerate for each pfn_valid()
> calll?
> 
> Perhaps the solution is to reimplement page_is_ram() for arm64 using
> memblock_is_memory() instead, But that still means we need to modify
> the generic memremap() code first to switch to it before changing the
> arm64 implementation of pfn_valid

No, that's not the solution. pfn_valid() should just check if there is
a valid struct page, as other archs do. And if there is a mis-use of
pfn_valid() to check for ram, only that calls should be fixed to use
page_is_ram(), however this is implemented, or something appropriate.
But I don't see any problematic code, and if so, I will fix that.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Thu, 24 Nov 2016 13:44:31 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161124134238.GI10776@rric.localdomain>
References: <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain> <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain> <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
Message-ID: <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> Why? MEMREMAP_WB is used often, among other things for mapping
>> firmware tables, which are marked as NOMAP, so in these cases, the
>> linear address is not mapped.
>
> If fw tables are mapped wb, that is wrong and needs a separate fix.
>

Why is that wrong?

>> > If you think pfn_valid() is wrong here, I am happy to send a patch
>> > that fixes this by using page_is_ram(). In any case, the worst case
>> > that may happen is to behave the same as v4.4, we might fix then the
>> > wrong use of pfn_valid() where it is not correctly used to check for
>> > ram.
>> >
>>
>> page_is_ram() uses string comparisons to look for regions called
>> 'System RAM'. Is that something we can tolerate for each pfn_valid()
>> calll?
>>
>> Perhaps the solution is to reimplement page_is_ram() for arm64 using
>> memblock_is_memory() instead, But that still means we need to modify
>> the generic memremap() code first to switch to it before changing the
>> arm64 implementation of pfn_valid
>
> No, that's not the solution. pfn_valid() should just check if there is
> a valid struct page, as other archs do. And if there is a mis-use of
> pfn_valid() to check for ram, only that calls should be fixed to use
> page_is_ram(), however this is implemented, or something appropriate.
> But I don't see any problematic code, and if so, I will fix that.
>

memremap() uses pfn_valid() to decide whether some address is covered
by the linear mapping. If we correct pfn_valid() to adhere to your
definition, we will need to fix memremap() first in any case.

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Thu, 24 Nov 2016 14:51:51 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
References: <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
Message-ID: <20161124135151.GJ10776@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24.11.16 13:44:31, Ard Biesheuvel wrote:
> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> >> Why? MEMREMAP_WB is used often, among other things for mapping
> >> firmware tables, which are marked as NOMAP, so in these cases, the
> >> linear address is not mapped.
> >
> > If fw tables are mapped wb, that is wrong and needs a separate fix.
> >
> 
> Why is that wrong?

The whole issue with mapping acpi tables is not marking them cachable,
what wb does. Otherwise we could just use linear mapping for those mem
ranges.

> >> > If you think pfn_valid() is wrong here, I am happy to send a patch
> >> > that fixes this by using page_is_ram(). In any case, the worst case
> >> > that may happen is to behave the same as v4.4, we might fix then the
> >> > wrong use of pfn_valid() where it is not correctly used to check for
> >> > ram.
> >> >
> >>
> >> page_is_ram() uses string comparisons to look for regions called
> >> 'System RAM'. Is that something we can tolerate for each pfn_valid()
> >> calll?
> >>
> >> Perhaps the solution is to reimplement page_is_ram() for arm64 using
> >> memblock_is_memory() instead, But that still means we need to modify
> >> the generic memremap() code first to switch to it before changing the
> >> arm64 implementation of pfn_valid
> >
> > No, that's not the solution. pfn_valid() should just check if there is
> > a valid struct page, as other archs do. And if there is a mis-use of
> > pfn_valid() to check for ram, only that calls should be fixed to use
> > page_is_ram(), however this is implemented, or something appropriate.
> > But I don't see any problematic code, and if so, I will fix that.
> >
> 
> memremap() uses pfn_valid() to decide whether some address is covered
> by the linear mapping. If we correct pfn_valid() to adhere to your
> definition, we will need to fix memremap() first in any case.

As said, will fix that if needed. But I think the caller is wrong
then.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Thu, 24 Nov 2016 13:58:30 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161124135151.GJ10776@rric.localdomain>
References: <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain> <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
Message-ID: <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
>> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> >> Why? MEMREMAP_WB is used often, among other things for mapping
>> >> firmware tables, which are marked as NOMAP, so in these cases, the
>> >> linear address is not mapped.
>> >
>> > If fw tables are mapped wb, that is wrong and needs a separate fix.
>> >
>>
>> Why is that wrong?
>
> The whole issue with mapping acpi tables is not marking them cachable,
> what wb does.

What 'issue'?

> Otherwise we could just use linear mapping for those mem
> ranges.
>

Regions containing firmware tables are owned by the firmware, and it
is the firmware that tells us which memory attributes we are allowed
to use. If those attributes include WB, it is perfectly legal to use a
cacheable mapping. That does *not* mean they should be covered by the
linear mapping. The linear mapping is read-write-non-exec, for
instance, and we may prefer to use a read-only mapping and/or
executable mapping.

>> >> > If you think pfn_valid() is wrong here, I am happy to send a patch
>> >> > that fixes this by using page_is_ram(). In any case, the worst case
>> >> > that may happen is to behave the same as v4.4, we might fix then the
>> >> > wrong use of pfn_valid() where it is not correctly used to check for
>> >> > ram.
>> >> >
>> >>
>> >> page_is_ram() uses string comparisons to look for regions called
>> >> 'System RAM'. Is that something we can tolerate for each pfn_valid()
>> >> calll?
>> >>
>> >> Perhaps the solution is to reimplement page_is_ram() for arm64 using
>> >> memblock_is_memory() instead, But that still means we need to modify
>> >> the generic memremap() code first to switch to it before changing the
>> >> arm64 implementation of pfn_valid
>> >
>> > No, that's not the solution. pfn_valid() should just check if there is
>> > a valid struct page, as other archs do. And if there is a mis-use of
>> > pfn_valid() to check for ram, only that calls should be fixed to use
>> > page_is_ram(), however this is implemented, or something appropriate.
>> > But I don't see any problematic code, and if so, I will fix that.
>> >
>>
>> memremap() uses pfn_valid() to decide whether some address is covered
>> by the linear mapping. If we correct pfn_valid() to adhere to your
>> definition, we will need to fix memremap() first in any case.
>
> As said, will fix that if needed. But I think the caller is wrong
> then.
>

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Thu, 24 Nov 2016 15:11:49 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
References: <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
Message-ID: <20161124141149.GE2213@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24.11.16 13:58:30, Ard Biesheuvel wrote:
> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
> >> >> linear address is not mapped.
> >> >
> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
> >> >
> >>
> >> Why is that wrong?
> >
> > The whole issue with mapping acpi tables is not marking them cachable,
> > what wb does.
> 
> What 'issue'?
> 
> > Otherwise we could just use linear mapping for those mem
> > ranges.
> >
> 
> Regions containing firmware tables are owned by the firmware, and it
> is the firmware that tells us which memory attributes we are allowed
> to use. If those attributes include WB, it is perfectly legal to use a
> cacheable mapping. That does *not* mean they should be covered by the
> linear mapping. The linear mapping is read-write-non-exec, for
> instance, and we may prefer to use a read-only mapping and/or
> executable mapping.

Ok, I am going to fix try_ram_remap().

Are there other concerns with this patch?

Thanks,

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Thu, 24 Nov 2016 14:23:16 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161124141149.GE2213@rric.localdomain>
References: <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
Message-ID: <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24 November 2016 at 14:11, Robert Richter <robert.richter@cavium.com> wrote:
> On 24.11.16 13:58:30, Ard Biesheuvel wrote:
>> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
>> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
>> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
>> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
>> >> >> linear address is not mapped.
>> >> >
>> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
>> >> >
>> >>
>> >> Why is that wrong?
>> >
>> > The whole issue with mapping acpi tables is not marking them cachable,
>> > what wb does.
>>
>> What 'issue'?
>>
>> > Otherwise we could just use linear mapping for those mem
>> > ranges.
>> >
>>
>> Regions containing firmware tables are owned by the firmware, and it
>> is the firmware that tells us which memory attributes we are allowed
>> to use. If those attributes include WB, it is perfectly legal to use a
>> cacheable mapping. That does *not* mean they should be covered by the
>> linear mapping. The linear mapping is read-write-non-exec, for
>> instance, and we may prefer to use a read-only mapping and/or
>> executable mapping.
>
> Ok, I am going to fix try_ram_remap().
>

Thanks. Could you also add an arm64 version of page_is_ram() that uses
memblock_is_memory() while you're at it? I think using memblock
directly in try_ram_remap() may not be the best approach

> Are there other concerns with this patch?
>

I think we all agree that pfn_valid() should return whether a pfn has
a struct page associated with it, the debate is about whether it makes
sense to allocate struct pages for memory that the kernel does not
own. But given that it does not really hurt to do so for small holes,
I think your suggestion makes sense.

Should we be doing anything more to ensure that those pages are not
dereferenced inadvertently? Is there a page flag we should be setting?

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Thu, 24 Nov 2016 16:09:18 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
References: <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
Message-ID: <20161124150918.GF2213@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24.11.16 14:23:16, Ard Biesheuvel wrote:
> On 24 November 2016 at 14:11, Robert Richter <robert.richter@cavium.com> wrote:
> > On 24.11.16 13:58:30, Ard Biesheuvel wrote:
> >> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> >> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
> >> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> >> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> >> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
> >> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
> >> >> >> linear address is not mapped.
> >> >> >
> >> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
> >> >> >
> >> >>
> >> >> Why is that wrong?
> >> >
> >> > The whole issue with mapping acpi tables is not marking them cachable,
> >> > what wb does.
> >>
> >> What 'issue'?
> >>
> >> > Otherwise we could just use linear mapping for those mem
> >> > ranges.
> >> >
> >>
> >> Regions containing firmware tables are owned by the firmware, and it
> >> is the firmware that tells us which memory attributes we are allowed
> >> to use. If those attributes include WB, it is perfectly legal to use a
> >> cacheable mapping. That does *not* mean they should be covered by the
> >> linear mapping. The linear mapping is read-write-non-exec, for
> >> instance, and we may prefer to use a read-only mapping and/or
> >> executable mapping.
> >
> > Ok, I am going to fix try_ram_remap().
> >
> 
> Thanks. Could you also add an arm64 version of page_is_ram() that uses
> memblock_is_memory() while you're at it? I think using memblock
> directly in try_ram_remap() may not be the best approach

Sure. I also want to mark the patches as stable.

> > Are there other concerns with this patch?
> >
> 
> I think we all agree that pfn_valid() should return whether a pfn has
> a struct page associated with it, the debate is about whether it makes
> sense to allocate struct pages for memory that the kernel does not
> own. But given that it does not really hurt to do so for small holes,
> I think your suggestion makes sense.

Thanks for your comments and the review.

> Should we be doing anything more to ensure that those pages are not
> dereferenced inadvertently? Is there a page flag we should be setting?

I don't think so. Boot mem is initialized in free_low_memory_core_
early(). The PageReserved flag is set for pages from reserved memory
ranges, and memory ranges not marked NOMAP is just freed. Since pages
are either reservered or in the free_list of pages, pages from other
memory ranges (NOMAP) is not visible to mm.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Thu, 24 Nov 2016 20:26:59 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161124150918.GF2213@rric.localdomain>
References: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
Message-ID: <20161124192659.GH2213@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

Ard,

> > >> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> > >> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:

> > >> Regions containing firmware tables are owned by the firmware, and it
> > >> is the firmware that tells us which memory attributes we are allowed
> > >> to use. If those attributes include WB, it is perfectly legal to use a
> > >> cacheable mapping. That does *not* mean they should be covered by the
> > >> linear mapping. The linear mapping is read-write-non-exec, for
> > >> instance, and we may prefer to use a read-only mapping and/or
> > >> executable mapping.
> > >
> > > Ok, I am going to fix try_ram_remap().

I revisited the code and it is working well already since:

 e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem

Now, try_ram_remap() is only called if the region to be mapped is
entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
ranges and not NOMAP mem. region_intersects() then returns
REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
REGION_DISJOINT would be returned and thus arch_memremap_wb() being
called directly. Before the e7cd190385d1 change try_ram_remap() was
called also for nomap regions.

So we can leave memremap() as it is and just apply this patch
unmodified. What do you think? Please ack.

I am going to prepare the pfn_is_ram() change in addition to this
patch, but that should not be required for this fix to work correcly.

Thanks,

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Thu, 24 Nov 2016 19:42:47 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161124192659.GH2213@rric.localdomain>
References: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
 <20161124192659.GH2213@rric.localdomain>
Message-ID: <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:
> Ard,
>
>> > >> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
>> > >> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>
>> > >> Regions containing firmware tables are owned by the firmware, and it
>> > >> is the firmware that tells us which memory attributes we are allowed
>> > >> to use. If those attributes include WB, it is perfectly legal to use a
>> > >> cacheable mapping. That does *not* mean they should be covered by the
>> > >> linear mapping. The linear mapping is read-write-non-exec, for
>> > >> instance, and we may prefer to use a read-only mapping and/or
>> > >> executable mapping.
>> > >
>> > > Ok, I am going to fix try_ram_remap().
>
> I revisited the code and it is working well already since:
>
>  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>
> Now, try_ram_remap() is only called if the region to be mapped is
> entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
> ranges and not NOMAP mem. region_intersects() then returns
> REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
> REGION_DISJOINT would be returned and thus arch_memremap_wb() being
> called directly. Before the e7cd190385d1 change try_ram_remap() was
> called also for nomap regions.
>
> So we can leave memremap() as it is and just apply this patch
> unmodified. What do you think?

I agree. The pfn_valid() check in try_ram_remap() is still appropriate
simply because the PageHighmem check requires a valid struct page. But
if we don't enter that code path anymore for NOMAP regions, I think
we're ok.

> Please ack.
>

I still don't fully understand how it is guaranteed that *all* memory
(i.e., all regions for which memblock_is_memory() returns true) is
covered by a struct page, but marked as reserved. Are we relying on
the fact that NOMAP memory is also memblock_reserve()'d?

> I am going to prepare the pfn_is_ram() change in addition to this
> patch, but that should not be required for this fix to work correcly.
>

I don't think you need to bother with page_is_ram() then. The only
place we use it is in devmem_is_allowed(), and there it makes sense to
allow NOMAP regions to be accessed (provided that you think /dev/mem
is a good idea in the first place).

From mboxrd@z Thu Jan  1 00:00:00 1970
From: robert.richter@cavium.com (Robert Richter)
Date: Fri, 25 Nov 2016 12:29:14 +0100
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
 <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
Message-ID: <20161125112914.GI2213@rric.localdomain>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 24.11.16 19:42:47, Ard Biesheuvel wrote:
> On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:

> > I revisited the code and it is working well already since:
> >
> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
> >
> > Now, try_ram_remap() is only called if the region to be mapped is
> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
> > ranges and not NOMAP mem. region_intersects() then returns
> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
> > called directly. Before the e7cd190385d1 change try_ram_remap() was
> > called also for nomap regions.
> >
> > So we can leave memremap() as it is and just apply this patch
> > unmodified. What do you think?
> 
> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
> simply because the PageHighmem check requires a valid struct page. But
> if we don't enter that code path anymore for NOMAP regions, I think
> we're ok.
> 
> > Please ack.
> >
> 
> I still don't fully understand how it is guaranteed that *all* memory
> (i.e., all regions for which memblock_is_memory() returns true) is
> covered by a struct page, but marked as reserved. Are we relying on
> the fact that NOMAP memory is also memblock_reserve()'d?

See free_low_memory_core_early():

----
	for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
        			NULL)
		count += __free_memory_core(start, end);
----

Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
also *not* marked reserved. So nothing at all from NOMAP mem is
reported to mm, it is not present (see below for a mem config, note
flags: 0x4 mem regions).

-Robert


[    0.000000] efi: Processing EFI memory map:
[    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000001480000-0x0000024bffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000024c0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010000400000-0x010f816aefff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f816af000-0x010f816b1fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f816b2000-0x010f826f1fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f826f2000-0x010f82701fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f82702000-0x010f82787fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f82788000-0x010f9276bfff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f9276c000-0x010f9276cfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f9276d000-0x010f935a8fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f935a9000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
[    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]

[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x1ffe800000 reserved size = 0x39146a21
[    0.000000]  memory.cnt  = 0x9
[    0.000000]  memory[0x0]     [0x00000001400000-0x000000fffdffff], 0xfebe0000 bytes on node 0 flags: 0x0
[    0.000000]  memory[0x1]     [0x000000fffe0000-0x000000ffffffff], 0x20000 bytes on node 0 flags: 0x4
[    0.000000]  memory[0x2]     [0x00000100000000-0x00000fffffffff], 0xf00000000 bytes on node 0 flags: 0x0
[    0.000000]  memory[0x3]     [0x00010000400000-0x00010ff9e7ffff], 0xff9a80000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x4]     [0x00010ff9e80000-0x00010ff9f1ffff], 0xa0000 bytes on node 1 flags: 0x4
[    0.000000]  memory[0x5]     [0x00010ff9f20000-0x00010ffaeaffff], 0xf90000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x6]     [0x00010ffaeb0000-0x00010ffaffffff], 0x150000 bytes on node 1 flags: 0x4
[    0.000000]  memory[0x7]     [0x00010ffb000000-0x00010ffffaffff], 0x4fb0000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x8]     [0x00010ffffb0000-0x00010fffffffff], 0x50000 bytes on node 1 flags: 0x4
[    0.000000]  reserved.cnt  = 0xd
[    0.000000]  reserved[0x0]   [0x00000001480000-0x0000000249ffff], 0x1020000 bytes flags: 0x0
[    0.000000]  reserved[0x1]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
[    0.000000]  reserved[0x2]   [0x000000c0000000-0x000000dfffffff], 0x20000000 bytes flags: 0x0
[    0.000000]  reserved[0x3]   [0x00000ffbfb8000-0x00000ffffdffff], 0x4028000 bytes flags: 0x0
[    0.000000]  reserved[0x4]   [0x00000ffffecb00-0x00000fffffffff], 0x13500 bytes flags: 0x0
[    0.000000]  reserved[0x5]   [0x00010f81780000-0x00010f8178ffff], 0x10000 bytes flags: 0x0
[    0.000000]  reserved[0x6]   [0x00010f82870000-0x00010f9286ffff], 0x10000000 bytes flags: 0x0
[    0.000000]  reserved[0x7]   [0x00010ffbce0000-0x00010fffceffff], 0x4010000 bytes flags: 0x0
[    0.000000]  reserved[0x8]   [0x00010fffee6d80-0x00010ffff2fffb], 0x4927c bytes flags: 0x0
[    0.000000]  reserved[0x9]   [0x00010ffff30000-0x00010ffffa000f], 0x70010 bytes flags: 0x0
[    0.000000]  reserved[0xa]   [0x00010ffffae280-0x00010ffffaff7f], 0x1d00 bytes flags: 0x0
[    0.000000]  reserved[0xb]   [0x00010ffffaffa0-0x00010ffffaffce], 0x2f bytes flags: 0x0
[    0.000000]  reserved[0xc]   [0x00010ffffaffd0-0x00010ffffafffe], 0x2f bytes flags: 0x0

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Fri, 25 Nov 2016 12:28:50 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <20161125112914.GI2213@rric.localdomain>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
 <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
 <20161125112914.GI2213@rric.localdomain>
Message-ID: <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 25 November 2016 at 11:29, Robert Richter <robert.richter@cavium.com> wrote:
> On 24.11.16 19:42:47, Ard Biesheuvel wrote:
>> On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:
>
>> > I revisited the code and it is working well already since:
>> >
>> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>> >
>> > Now, try_ram_remap() is only called if the region to be mapped is
>> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
>> > ranges and not NOMAP mem. region_intersects() then returns
>> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
>> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
>> > called directly. Before the e7cd190385d1 change try_ram_remap() was
>> > called also for nomap regions.
>> >
>> > So we can leave memremap() as it is and just apply this patch
>> > unmodified. What do you think?
>>
>> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
>> simply because the PageHighmem check requires a valid struct page. But
>> if we don't enter that code path anymore for NOMAP regions, I think
>> we're ok.
>>
>> > Please ack.
>> >
>>
>> I still don't fully understand how it is guaranteed that *all* memory
>> (i.e., all regions for which memblock_is_memory() returns true) is
>> covered by a struct page, but marked as reserved. Are we relying on
>> the fact that NOMAP memory is also memblock_reserve()'d?
>
> See free_low_memory_core_early():
>
> ----
>         for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
>                                 NULL)
>                 count += __free_memory_core(start, end);
> ----
>
> Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
> also *not* marked reserved. So nothing at all from NOMAP mem is
> reported to mm, it is not present (see below for a mem config, note
> flags: 0x4 mem regions).
>

OK, thanks for clearing that up. But that still does not explain how
we can be certain that NOMAP regions are guaranteed to be covered by a
struct page, does it? Because that is ultimately what pfn_valid()
means, that it is safe to, e.g., look at the page flags.


> [    0.000000] efi: Processing EFI memory map:
> [    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000001480000-0x0000024bffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000024c0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010000400000-0x010f816aefff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f816af000-0x010f816b1fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f816b2000-0x010f826f1fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f826f2000-0x010f82701fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f82702000-0x010f82787fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f82788000-0x010f9276bfff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f9276c000-0x010f9276cfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f9276d000-0x010f935a8fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f935a9000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
> [    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
>
> [    0.000000] MEMBLOCK configuration:
> [    0.000000]  memory size = 0x1ffe800000 reserved size = 0x39146a21
> [    0.000000]  memory.cnt  = 0x9
> [    0.000000]  memory[0x0]     [0x00000001400000-0x000000fffdffff], 0xfebe0000 bytes on node 0 flags: 0x0
> [    0.000000]  memory[0x1]     [0x000000fffe0000-0x000000ffffffff], 0x20000 bytes on node 0 flags: 0x4
> [    0.000000]  memory[0x2]     [0x00000100000000-0x00000fffffffff], 0xf00000000 bytes on node 0 flags: 0x0
> [    0.000000]  memory[0x3]     [0x00010000400000-0x00010ff9e7ffff], 0xff9a80000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x4]     [0x00010ff9e80000-0x00010ff9f1ffff], 0xa0000 bytes on node 1 flags: 0x4
> [    0.000000]  memory[0x5]     [0x00010ff9f20000-0x00010ffaeaffff], 0xf90000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x6]     [0x00010ffaeb0000-0x00010ffaffffff], 0x150000 bytes on node 1 flags: 0x4
> [    0.000000]  memory[0x7]     [0x00010ffb000000-0x00010ffffaffff], 0x4fb0000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x8]     [0x00010ffffb0000-0x00010fffffffff], 0x50000 bytes on node 1 flags: 0x4
> [    0.000000]  reserved.cnt  = 0xd
> [    0.000000]  reserved[0x0]   [0x00000001480000-0x0000000249ffff], 0x1020000 bytes flags: 0x0
> [    0.000000]  reserved[0x1]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
> [    0.000000]  reserved[0x2]   [0x000000c0000000-0x000000dfffffff], 0x20000000 bytes flags: 0x0
> [    0.000000]  reserved[0x3]   [0x00000ffbfb8000-0x00000ffffdffff], 0x4028000 bytes flags: 0x0
> [    0.000000]  reserved[0x4]   [0x00000ffffecb00-0x00000fffffffff], 0x13500 bytes flags: 0x0
> [    0.000000]  reserved[0x5]   [0x00010f81780000-0x00010f8178ffff], 0x10000 bytes flags: 0x0
> [    0.000000]  reserved[0x6]   [0x00010f82870000-0x00010f9286ffff], 0x10000000 bytes flags: 0x0
> [    0.000000]  reserved[0x7]   [0x00010ffbce0000-0x00010fffceffff], 0x4010000 bytes flags: 0x0
> [    0.000000]  reserved[0x8]   [0x00010fffee6d80-0x00010ffff2fffb], 0x4927c bytes flags: 0x0
> [    0.000000]  reserved[0x9]   [0x00010ffff30000-0x00010ffffa000f], 0x70010 bytes flags: 0x0
> [    0.000000]  reserved[0xa]   [0x00010ffffae280-0x00010ffffaff7f], 0x1d00 bytes flags: 0x0
> [    0.000000]  reserved[0xb]   [0x00010ffffaffa0-0x00010ffffaffce], 0x2f bytes flags: 0x0
> [    0.000000]  reserved[0xc]   [0x00010ffffaffd0-0x00010ffffafffe], 0x2f bytes flags: 0x0

From mboxrd@z Thu Jan  1 00:00:00 1970
From: ard.biesheuvel@linaro.org (Ard Biesheuvel)
Date: Fri, 25 Nov 2016 17:01:01 +0000
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
In-Reply-To: <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
 <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
 <20161125112914.GI2213@rric.localdomain>
 <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
Message-ID: <CAKv+Gu8Hbf1sY4zMGoosYjGa1esApqP8a4Zs2Zqj4nNZ8AdtSw@mail.gmail.com>
To: linux-arm-kernel@lists.infradead.org
List-Id: linux-arm-kernel.lists.infradead.org

On 25 November 2016 at 12:28, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 25 November 2016 at 11:29, Robert Richter <robert.richter@cavium.com> wrote:
>> On 24.11.16 19:42:47, Ard Biesheuvel wrote:
>>> On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:
>>
>>> > I revisited the code and it is working well already since:
>>> >
>>> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>>> >
>>> > Now, try_ram_remap() is only called if the region to be mapped is
>>> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
>>> > ranges and not NOMAP mem. region_intersects() then returns
>>> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
>>> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
>>> > called directly. Before the e7cd190385d1 change try_ram_remap() was
>>> > called also for nomap regions.
>>> >
>>> > So we can leave memremap() as it is and just apply this patch
>>> > unmodified. What do you think?
>>>
>>> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
>>> simply because the PageHighmem check requires a valid struct page. But
>>> if we don't enter that code path anymore for NOMAP regions, I think
>>> we're ok.
>>>
>>> > Please ack.
>>> >
>>>
>>> I still don't fully understand how it is guaranteed that *all* memory
>>> (i.e., all regions for which memblock_is_memory() returns true) is
>>> covered by a struct page, but marked as reserved. Are we relying on
>>> the fact that NOMAP memory is also memblock_reserve()'d?
>>
>> See free_low_memory_core_early():
>>
>> ----
>>         for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
>>                                 NULL)
>>                 count += __free_memory_core(start, end);
>> ----
>>
>> Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
>> also *not* marked reserved. So nothing at all from NOMAP mem is
>> reported to mm, it is not present (see below for a mem config, note
>> flags: 0x4 mem regions).
>>
>
> OK, thanks for clearing that up. But that still does not explain how
> we can be certain that NOMAP regions are guaranteed to be covered by a
> struct page, does it? Because that is ultimately what pfn_valid()
> means, that it is safe to, e.g., look at the page flags.
>

Answering to self: arm64_memory_present() registers all memory as
present, which means that sparse_init() will allocate struct page
backing for all of memory, including NOMAP regions.

We could ask ourselves whether it makes sense to disregard NOMAP
memory here, but that probably buys us very little (but I do wonder
how it affects the occurrence of the bug).

In any case, it looks to me like your patch is safe, i.e., calling
pfn_valid() on NOMAP pages is safe, although I still find it debatable
whether the kernel should be tracking memory it does not own. However,
for performance reasons, it probably makes sense to apply your patch,
and if anyone ever comes up with a use case where this is problematic
(e.g., gigabytes of NOMAP memory), we can look into it then.

Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1753090AbcJFKBR (ORCPT <rfc822;w@1wt.eu>);
        Thu, 6 Oct 2016 06:01:17 -0400
Received: from mail-it0-f44.google.com ([209.85.214.44]:38212 "EHLO
        mail-it0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1752110AbcJFKBN (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 6 Oct 2016 06:01:13 -0400
MIME-Version: 1.0
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Thu, 6 Oct 2016 11:00:33 +0100
Message-ID: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <rrichter@cavium.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will.deacon@arm.com>,
        David Daney <david.daney@cavium.com>,
        Mark Rutland <mark.rutland@arm.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Robert,

Apologies for only responding now. I did not quite manage to get my
head around your original email yet, but I don't think this patch is
the correct solution.

On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
>
>  kernel BUG at mm/page_alloc.c:1848!
>
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
>
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
>
>  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
>

These pages are owned by the firmware, which may map it with
attributes that conflict with the attributes we use for the linear
mapping. This means they should not be covered by the linear mapping.

> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
>

This sounds like the root cause of your issue. Could we not fix that instead?

> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory().

This is incorrect imo. In general, pfn_valid() means ordinary memory
covered by the linear mapping and the struct page array. Returning
reserved ranges that the kernel should not even touch only to please
the NUMA code seems like an inappropriate way to deal with this issue.

> Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().
>
> Should be marked stable v4.5..
>
> Signed-off-by: Robert Richter <rrichter@cavium.com>
> ---
>  arch/arm64/mm/init.c    | 2 +-
>  arch/arm64/mm/ioremap.c | 5 +++--
>  2 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index bbb7ee76e319..25b8659c2a9f 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -147,7 +147,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
>  #ifdef CONFIG_HAVE_ARCH_PFN_VALID
>  int pfn_valid(unsigned long pfn)
>  {
> -       return memblock_is_map_memory(pfn << PAGE_SHIFT);
> +       return memblock_is_memory(pfn << PAGE_SHIFT);
>  }
>  EXPORT_SYMBOL(pfn_valid);
>  #endif
> diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c
> index 01e88c8bcab0..c17c220b0c48 100644
> --- a/arch/arm64/mm/ioremap.c
> +++ b/arch/arm64/mm/ioremap.c
> @@ -21,6 +21,7 @@
>   */
>
>  #include <linux/export.h>
> +#include <linux/memblock.h>
>  #include <linux/mm.h>
>  #include <linux/vmalloc.h>
>  #include <linux/io.h>
> @@ -55,7 +56,7 @@ static void __iomem *__ioremap_caller(phys_addr_t phys_addr, size_t size,
>         /*
>          * Don't allow RAM to be mapped.
>          */
> -       if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr))))
> +       if (WARN_ON(memblock_is_map_memory(phys_addr)))
>                 return NULL;
>
>         area = get_vm_area_caller(size, VM_IOREMAP, caller);
> @@ -96,7 +97,7 @@ EXPORT_SYMBOL(__iounmap);
>  void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size)
>  {
>         /* For normal memory we already have a cacheable mapping. */
> -       if (pfn_valid(__phys_to_pfn(phys_addr)))
> +       if (memblock_is_map_memory(phys_addr))
>                 return (void __iomem *)__phys_to_virt(phys_addr);
>
>         return __ioremap_caller(phys_addr, size, __pgprot(PROT_NORMAL),
> --
> 2.7.0.rc3
>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1755634AbcJFO1O (ORCPT <rfc822;w@1wt.eu>);
        Thu, 6 Oct 2016 10:27:14 -0400
Received: from mail-by2nam03on0075.outbound.protection.outlook.com ([104.47.42.75]:63695
        "EHLO NAM03-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S1751055AbcJFO1L (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 6 Oct 2016 10:27:11 -0400
X-Greylist: delayed 14429 seconds by postgrey-1.27 at vger.kernel.org; Thu, 06 Oct 2016 10:27:11 EDT
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
From: Robert Richter <rrichter@cavium.com>
To: Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will.deacon@arm.com>
CC: Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        David Daney <david.daney@cavium.com>,
        Mark Rutland <mark.rutland@arm.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        <linux-arm-kernel@lists.infradead.org>, <linux-efi@vger.kernel.org>,
        Robert Richter <rrichter@cavium.com>, <linux-kernel@vger.kernel.org>
Subject: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
Date: Thu, 6 Oct 2016 11:52:07 +0200
Message-ID: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
X-Mailer: git-send-email 2.7.0.rc3
MIME-Version: 1.0
Content-Type: text/plain
X-Originating-IP: [92.229.86.66]
X-ClientProxiedBy: AM5PR0501CA0021.eurprd05.prod.outlook.com (10.164.187.31)
 To CY1PR07MB2345.namprd07.prod.outlook.com (10.166.194.144)
X-MS-Office365-Filtering-Correlation-Id: 1c9e846f-20a2-4fa5-c951-08d3edce91fd
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2345;2:HM7ar0lkJ2uR+6ijyP8kDMKKYnq/Yxlczevm034qC1nXwCUlQu1OSe3vR5oQoL24n9bD3+4GzHBeRJTp4vgH7/WsCJfOQXqkWtg48vxTGUuXRBZepfrHPATJOcsHEGMDC0qS1/LqKq39U3X5yAiWuiWfnq43TvF25MeO5etim1D5TwmHedtza5/6ua3ULh2C60h/yIY/sOC/atFnAJERHg==;3:V9LAPVkA6pNUULvL0efdK5j+/YLb4R5QoixW/HzTeEto/gS0jf8oeAwwrD6au6gJAVjCvAMfrefHofIbbngCFnKJYw+pHf9wrMKYni88po/kngprXtFs+ycWuVWxJoUwoMHbEciDU++HNwU7j3GMTg==;25:i2x1lBqebwaIaYKZgCtvUwoSQWhhEQDpJqCrTNE9Ee8YkK5iMnDc7uNVBnX4PtWcfxUjz/HeRiJW4zsXXkriX3sMC9s3YjTUOGCTdDz4ion3tAZjk1fpYBRCPrPLEDmH6IzPHRMmjWyibjCSyC7vxs33b5XU5iiCnfYbwXYiN2S1vODWqd5XNlm5N5VIlX7ejsTqjkWg/Mz4OmDJXl134AaIdw1cRaiQGr3wR5KVPc9hzDuFJRPcIDRMaj2JWApfBemTRBWI49kq3Zc93hloVJsRhfIIWVGBPCZaH2QDRJ6aVqld3d59NVILai6f2BabKFxJOz9L2oabqgV1RcorBOscAkaN39XAcaIfILoEnAZDfyZ5ifi/jY83FlbcT48fNrHDPW5gluxMp9/iR9IECQDON+P8fKLlqEEMYM0gKIupwymKe+cSiKL0w9zQsQhD
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2345;
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2345;31:YUnqy64VP6EnqTSxjTmtyHvFyR64L8fJmBx3kS9kLwxKe1mm2rjUGfX+r6K7ZkpmyL3mSWXUkVwYSfmqrtwdWm/jZpDmgcwglgEGaHGF4vdi//nlMUh1INIhSb0oLrkjzUYlW8BXIImkYE32MRw4XndKswaXxKw1AgZf81jjYZbmUIp8PiPV76ZlaLiZrGBIvmw9Qjy26ULLoxQHNle5UA63SrEW0WUej1i5voVOS+UyMNeJMYX+ISm/NSWF0A8F;20: 
X-Microsoft-Antispam-PRVS: <CY1PR07MB234559BFDAA42BD2BB3793D2F1C70@CY1PR07MB2345.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(20558992708506);
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);SRVR:CY1PR07MB2345;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2345;
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2345;4:bZ7o9+PXK/7EI6VgtRfdnoC8XF51elKMpCRxGY4HZRAn1nV4UWvUGnd90bEH5oMKdJbuDZjN85EpBLuFJeIXQUHDdNAlcLSNHxvcK6kNSahvWaoMy0Ijubou/GIvc+YPlL8ya1eL3TD8kaOXJqPVNpbdYcCCx0W1J7El39tc8Ebms/0CaYYQlqgjWqLE/Avx3DXEU33KsYSPc2sjf6JGA8Kqkd4JT56RU2FJqmwQ8xd/N8GMTk97bLyjW/tdRKkEap7ffnw9nliFmQCiGSnn+hGtzSz+s5wEsq7yopfWcg8z1+wiaWFrNlTPHmn7YJE0r9ijR5TieW9IQufwSaUd6lM911AsLPj4O+evSJs8ssNBesM/jp55YrcOPMED4tG7vZy9jRuVhvOpG1JsPMjwnbzze1JprL4ToN2mrRXeQ0oiyLP9Cr6UKlgMiPe/dupO5xWE/40oUL8SQCaFxRf81g==
X-Forefront-PRVS: 00872B689F
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(189002)(7846002)(7736002)(305945005)(81156014)(47776003)(66066001)(8676002)(4326007)(19580395003)(81166006)(2906002)(50466002)(48376002)(50986999)(101416001)(19580405001)(77096005)(575784001)(42186005)(3846002)(6116002)(36756003)(189998001)(97736004)(586003)(5003940100001)(5001770100001)(5660300001)(105586002)(229853001)(106356001)(92566002)(33646002)(50226002)(68736007);DIR:OUT;SFP:1101;SCL:1;SRVR:CY1PR07MB2345;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;CY1PR07MB2345;23:1FxatQmPhFaYagw/StmEyRoDQhkc6kGvHlyp8HEc0?=
 =?us-ascii?Q?awT8S3HZzLzzFpV6oxb/7+dVz9fle441DwJzVNpm41uN9dvWWMqVKlZP34SK?=
 =?us-ascii?Q?Sx3026wBn9X9t69/Pp8tb1Q8vjD43JxCr83sZYakP9o0iN+nvzHbN0x/WxhY?=
 =?us-ascii?Q?bBVvmLOKhXnu/s6WdAfQQW/6cO6K0yMRpa6zp64CJVERkXbb+hj40z3yHxRY?=
 =?us-ascii?Q?/YwSMNWGOtm33jeVyv1tMY45h82D3A/akdtG+h1LwFf2NjFv+fji0xuo2RTW?=
 =?us-ascii?Q?DCtnZGt4KJMfb5dPUtJL2kSLsjdQV7MDua/oiL+h0m6YMmlaE2BT+Nkv76e9?=
 =?us-ascii?Q?9zmjOs/uMjG2kLuqZnu7QYezmNDMDaBcMWBxRTTEQuBeELuEO58ykwpo3KvH?=
 =?us-ascii?Q?1hmxLmVUwdIF2e45oXGrUqLvRkyhEBcNJuCKPZzJSFZIgOYP3TcIZ5Xe3gai?=
 =?us-ascii?Q?6Z5VVn8oSzbKirpVZfexsCxddISl+gsj9VJY1pgSrLQzTRn8gCmGM4znuXMJ?=
 =?us-ascii?Q?JyPNZrsb574/Nc8HjTcBHRqMQS0stzUUo5T53Z1e3Hgf4FyjKF7FYP+2h5NF?=
 =?us-ascii?Q?EKInAgzktNOa1nhcYqKZjj0GXXHb2FIuE0kUFAQYRkxM02Sf+ANTW6CD6034?=
 =?us-ascii?Q?RWQhWeeRYy/YvpkxDW9JZ3yy7MDDkWGTL9Lsc2uqaiQHZwv4UDTRyOlJ8tZf?=
 =?us-ascii?Q?30MyZxTaYeLQM5ARBTRtjbNlYHzLL/WWEfRtoifEQSITVSqCt26QgomPV7zM?=
 =?us-ascii?Q?10wz2gPxJqcCCxiGkPntL1awBPZW88PqRVHiD3V7lgH2nqy9UMJeGfbExgZ7?=
 =?us-ascii?Q?pwKFFt55hTHf1V487x+YmwCjnsuGvL5HhU5QAmf+WA57R5QIqmzFgYddZBvV?=
 =?us-ascii?Q?+afEmzQ7XT2rQaOugDyReOVs+AOfDUcpSIVuhsg/2rhZSR0YDOMay5G1iG4E?=
 =?us-ascii?Q?u6ngwCJxgI0ia8ME+iKvI3OHdLKc9tUBkaJA3pDJZMaS/iGQJUi3GSvPI/LV?=
 =?us-ascii?Q?5ckqy6MpcYt5kRqjFcrEUiqYOd5tUmxrbHuE0saIUKLYA=3D=3D?=
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2345;6:GhDQ7HH7hKAoees/Ue6Vzzu6X4V8+yY4kFDEMdzuxqRV9OSKOrL5quUFuY/3PtJIPVv/dKjZgaEqSZnaJXiI+zmWLXWAtdQvF55Mk1mqNU57jyoTbtClZLTvlM26cRftTKYxTkbrvtsRM/M/C9m22+uFCV/OwqCvvZ9/ry/Lc54DZgX7ClTVo3b+EKGb5Ma8qAhmdPjzgoN53M5TlhfMOD0i5iJk7/1fgaMuwhjCT1ttAmhf75pjL3Pcgz+i7T46m6ovXNSLyHUaXPppefNQw5nfYOQVbGVOfqMLhT7YRCOfSQGKtGDOAJpFy2RtW9Rg;5:M8Luc2SsMPzuQL4VNu9jWQto9drWeSKWlIW9uanIJ+Jhib5Mcc4o7xuKtX0HuaB9BEEUrVLZPCy6uuzNDsF+Bzc4CuRPk6Wx7i4hmPFsw5pHxJM4C2V5gKRDtWH4kPpawUFg51foOs1PaYooFPnB7hkQw7b4JumUAcHpxq2O01U=;24:ByyRsRX0Ok1ldR6Ne5cgvZmv4ytVxfKeTVI1n+w0E9NhxqddrnXn6n0Dgt8vkANwFtzb4oHRZS/4VDGZtsNzvyLNTSnd/QAr22yBGYmE1Jc=;7:F0dtajPF7ROPOBoaEus0/mlKOnzQDeJ/4XT6ph6+7EAhWlhgR4NzCqshnaT0Cvct2I9jGSrC2cSbbzrALB2nLzAcd4xM1OQExFHGWecpAcCzTmmFXyv5+5aOrThJTQveNIbEmPDz4/zVPLKZ9wFoFVk2EdWjzpIE0tKEg7YkgMaPiV4HU/USaSCZmLXH1OTvDjGXSVGPKQvNSHhH2juPRxvgcDx3o+GeQB9TvI1FpuYoTmgBjHlLxkdl3dFl5630BtJWjfv4s28yqJcl6MBw19lANARFskB4HJsE3/QYlGfDUg1itQjJSmM3Q9kqGqhqfIi5SDJQikdPgvwW4l6clA==
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2016 09:53:03.7849 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2345
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

There is a memory setup problem on ThunderX systems with certain
memory configurations. The symptom is

 kernel BUG at mm/page_alloc.c:1848!

This happens for some configs with 64k page size enabled. The bug
triggers for page zones with some pages in the zone not assigned to
this particular zone. In my case some pages that are marked as nomap
were not reassigned to the new zone of node 1, so those are still
assigned to node 0.

The reason for the mis-configuration is a change in pfn_valid() which
reports pages marked nomap as invalid:

 68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping

This causes pages marked as nomap being no long reassigned to the new
zone in memmap_init_zone() by calling __init_single_pfn().

Fixing this by restoring the old behavior of pfn_valid() to use
memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
to use memblock_is_map_memory() where necessary. This only affects
code in ioremap.c. The code in mmu.c still can use the new version of
pfn_valid().

Should be marked stable v4.5..

Signed-off-by: Robert Richter <rrichter@cavium.com>
---
 arch/arm64/mm/init.c    | 2 +-
 arch/arm64/mm/ioremap.c | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index bbb7ee76e319..25b8659c2a9f 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -147,7 +147,7 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max)
 #ifdef CONFIG_HAVE_ARCH_PFN_VALID
 int pfn_valid(unsigned long pfn)
 {
-	return memblock_is_map_memory(pfn << PAGE_SHIFT);
+	return memblock_is_memory(pfn << PAGE_SHIFT);
 }
 EXPORT_SYMBOL(pfn_valid);
 #endif
diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c
index 01e88c8bcab0..c17c220b0c48 100644
--- a/arch/arm64/mm/ioremap.c
+++ b/arch/arm64/mm/ioremap.c
@@ -21,6 +21,7 @@
  */
 
 #include <linux/export.h>
+#include <linux/memblock.h>
 #include <linux/mm.h>
 #include <linux/vmalloc.h>
 #include <linux/io.h>
@@ -55,7 +56,7 @@ static void __iomem *__ioremap_caller(phys_addr_t phys_addr, size_t size,
 	/*
 	 * Don't allow RAM to be mapped.
 	 */
-	if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr))))
+	if (WARN_ON(memblock_is_map_memory(phys_addr)))
 		return NULL;
 
 	area = get_vm_area_caller(size, VM_IOREMAP, caller);
@@ -96,7 +97,7 @@ EXPORT_SYMBOL(__iounmap);
 void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size)
 {
 	/* For normal memory we already have a cacheable mapping. */
-	if (pfn_valid(__phys_to_pfn(phys_addr)))
+	if (memblock_is_map_memory(phys_addr))
 		return (void __iomem *)__phys_to_virt(phys_addr);
 
 	return __ioremap_caller(phys_addr, size, __pgprot(PROT_NORMAL),
-- 
2.7.0.rc3

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S942270AbcJFQLi (ORCPT <rfc822;w@1wt.eu>);
        Thu, 6 Oct 2016 12:11:38 -0400
Received: from mail-by2nam01on0088.outbound.protection.outlook.com ([104.47.34.88]:53836
        "EHLO NAM01-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S942199AbcJFQL2 (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 6 Oct 2016 12:11:28 -0400
X-Greylist: delayed 22700 seconds by postgrey-1.27 at vger.kernel.org; Thu, 06 Oct 2016 12:11:27 EDT
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Thu, 6 Oct 2016 18:11:14 +0200
From: Robert Richter <robert.richter@cavium.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
CC: Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will.deacon@arm.com>,
        David Daney <david.daney@cavium.com>,
        Mark Rutland <mark.rutland@arm.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161006161114.GH22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.86.66]
X-ClientProxiedBy: AM4PR01CA0009.eurprd01.prod.exchangelabs.com
 (10.164.74.147) To SN1PR07MB2351.namprd07.prod.outlook.com (10.169.127.17)
X-MS-Office365-Filtering-Correlation-Id: 2595d235-3211-44cd-ddd8-08d3ee036c0b
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;2:zJpyTOgrfmkArY5BVxnRKXqGeTXxZ0Wct78ZuuMhoH0q5ruqjpsVW1jlLcAj2CD92JREgQzRfgvzzBszCW6w1uZra6C7vbZtni01r0bg0MEyOgZR82JwpO62aNbdzNzmaxMd48vAOBHA+yLZtWAQuwQmXICB6AQxA7NbzG9l0DuynBgd6neYGCliD96yioU6aI9pNafYY72kFvr3QI0dTw==;3:umD2YvZyrOSmOdI3p2wgRo/50w/ZpZYxeMM87UhnjO0Y0fWu9dN28ot+e9VkIMoglkKnwD5lSC/Hr1lwjqzmCyAt/AVHphLW0h/Wa3RrKqEgFxxqDdb0BR6UErVsmq8bPQ+KDcLpzAyI9qHdkS4Hqw==;25:fcXMcKx8taHSElZCgS+R0MjwyyFZsKX9nzvQINxlIYzaBGt+YdIwHqNZm2SXTisoSKfwS4fORITTDcvvFyaSxISP1IxjhWyRGIraJNLYfKewMmUqepaUAUYSYw/C2rCHFBXVXNTJECZD5/coL/Z6zwJp1sPf6skMjpTug1ElTmejGLUkS/po+8fMSs9i9yJK43VxQtqQEw+Mz1PbPRjzKpSmxevrr43jW+uQbZXtgsXWc+sOqvfmjugSS/TOioWPGhwim//fs7jML3pNwa393waoXWYbJOAI7U4vS8hUSaA6tzS8i6HNuvWkQfWw7RiT+DX3KorpP/l2EXqYhT0g6gUu0LfjpUWAwZ4JhuL+eF4ziKv/1N2mftrtVKprIxybsnpcQCYjXqcovirNlSxl1qAHjv/VMz1xW1oaQeIalgDltunWw8lXsNumWLBcmic2
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2351;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;31:NAMQ0AoVmp+Y6uSGNEy4S3sOvrKB+uNDp1WpYd9NjxwHc1Fqk7cGtKWaidE+7CYnsvJVSyO2sliUv2vQBIgB5re2FoZr+aOWH4kO2wZDRE/pZovKOxu09nZetqwspf8wjj5ogZ1XqiPUVk5uIXh0Wp7AZIAKj17oQY53vJUc0aK8P3aV4Z9F3WTssGYw+IS6V2IItO4ev7zeu8PSiJkxGaXCYxr9LH5RtF2F5lkVW2vk7wdtMLiQHEjkT4YsI2/ERMl/FPiPRZM5DSxjG65WIw==;20: 
X-Microsoft-Antispam-PRVS: <SN1PR07MB2351634E4F68CF0EB8C7832DF1C70@SN1PR07MB2351.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(20558992708506);
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);SRVR:SN1PR07MB2351;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2351;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;4:/X9ZbHnm4qGi2Iq+Dx5eNg9vEcVoNflBJi3Pz97zZPbzncRfItwXWdYqfzQK+Mwp+/n0n/1K7IQLbGNwQCQebGov/rYiCPgvrGI38amJmyZdgqfGNkEQZ9romTU49/G2suqlrnjiiIJHN4ciTPspzv8hc14s1OyGf8EKJrFnsz8DYFZOjpH1BukNMt8digMiAAnNjqkJTrskIUjB3D+LDbwt615/mqWdu1c+qhHv03GWPmeWn/li9Q7LF+uqw/LWwZic6b+GO8qxZgx+2mqOk/ysOjTBPRjkKZVD8DmNC6vKJ305M460qkIy905FOvUNrGL3tTPRG3yzQMHvhrxLUI5P0U01InBFb1AbKwzv5PmhzPRqBC8bfNZCwO2LTm23mD8bPGvtFyjEBrrxfCy58h+0HMAGxH53zbDnLBrzAj7yxy1Bx9o0E+GE3C8EA1Ho
X-Forefront-PRVS: 00872B689F
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(189002)(24454002)(54356999)(46406003)(76176999)(19580395003)(19580405001)(81166006)(81156014)(8676002)(7846002)(305945005)(23726003)(7736002)(6666003)(2950100002)(6916009)(4001350100001)(97736004)(83506001)(5890100001)(77096005)(110136003)(86362001)(66066001)(47776003)(92566002)(68736007)(561944003)(101416001)(33656002)(105586002)(5660300001)(50986999)(106356001)(189998001)(586003)(50466002)(97756001)(3846002)(345774005)(42186005)(4326007)(9686002)(2906002)(1076002)(6116002)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR07MB2351;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;SN1PR07MB2351;23:vz0cWyLWXU4dHW19yWQA9PqAC/AedaIzf1wSxjYYd?=
 =?us-ascii?Q?6kBCe80XW7/JQexWN4Yizm9w84dnkqBTb9XfpY13u/CTDajOXUjJcLo8djsk?=
 =?us-ascii?Q?NEe7Y6aDLMhqah7EcxYTiLiOA5YVaOtDx9vn4+Ych/nqz5gAGKPLcmQAc6qX?=
 =?us-ascii?Q?PkwwkB8QxGwb/FmdFQqZJLegyTHhOPfCbvnBACZErzd6MG4eiRPOw+gzWuwf?=
 =?us-ascii?Q?txr3TVLbrrINp7FMBYT2AmjZouDn5suxFJfz0egVV/pDsxffffXKMJTs24cx?=
 =?us-ascii?Q?E1MPYPONks9SY2+E0BYZbmPZQnj4m9FwU6eU+v0WfLkaY/KgKSzs5mGt+2rY?=
 =?us-ascii?Q?ttcMFBKF6akDu0nenbKhP0RRDFUKXWDVDRNBLJfdo/V01VRuFCCiCHe5l8cp?=
 =?us-ascii?Q?N/dcWNdrR/zsqMVYpsAtjKQfwOO3ls9rxkDtXDhOdWurSfRlirmDqZOFJuaC?=
 =?us-ascii?Q?6JWQUDHM+HiWkVZJDb1TGqv9TUoqgKLrTGTx7EhU0sLFXiwww7IbQODi5y05?=
 =?us-ascii?Q?ZoXZf/pvn7NsZUj9TH91NdKgUh1GgK+6OjGH9MlwKC76qRoOOY5NpJzxxczu?=
 =?us-ascii?Q?zYUcr9WvXz9V8FjT67YpnNbaC3or38ZISZuxDimATgqgpasx6GYqs3zHfkQB?=
 =?us-ascii?Q?FOvVH4wZ/JnYvCbjPuaAIbF6Fkv8vqlpSGdjTkcA5qD/FgDESBxOlYnZdu+e?=
 =?us-ascii?Q?yHTEvCo3KTPxrTZ7ifObXfdekdC+6cyY2Xe/7T1mz//Y4zIat4EcegnJa8p8?=
 =?us-ascii?Q?zNwhdKI3YVc/PObGlJdKhBHvCoFgrSlraZF2IDjyeQq1aIo03v8uidPVOf4G?=
 =?us-ascii?Q?ZgDHQCyuNScXt26V9TvZeGfMqlSDKAdNDe3TerR+MKR0tiD9BETHbCt0Po3F?=
 =?us-ascii?Q?isy0YR5/vHeSYvAGDFYfI7m+uU0FJpaQ1efIe5XycEn62r8jf5SAM2xhcm0q?=
 =?us-ascii?Q?oMHkLvpSdllAE2QRMPF7Yuttc1/zhDUn2yyCndAErcXsNE9hgiBfAEvgzWd4?=
 =?us-ascii?Q?bmnLS7ri9z45CVqyZhNtF60e7YW4YBG3/DM4fygp1yywuxMKW8jYcOvvBfpf?=
 =?us-ascii?Q?/2qIqXzhLc8HQf/nxZxRbOGAMpg5d114Kh+EWO8rie2g6uuQV1wInMKZ3xT5?=
 =?us-ascii?Q?PgqGBkS0RQeZrlXFaU/I0sunorJpkLm2NjVUXkTCyLzi+bI2w71ftrRN2AUl?=
 =?us-ascii?Q?QNBMTiBBEQE7qfVGP0cvD8/Z/A5+yIcA1fC7KEsvkh7lk39FuwmvQOZHHsET?=
 =?us-ascii?Q?XQ4UFLjAuIgtj5BCiyGhnxYwrNKsNzLqxWbGobXZlzwN0xi/J4Ld5ho2OumS?=
 =?us-ascii?B?dz09?=
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;6:kMq64O9R4wS4z15Q/EKTS9js9bUaFaATN4SASVdDONx1Nfn6XLCp73Pd2d93jGsoma8a6L8gN5o46QYiHFx2qHFEJ7dYbiooDYLRsNFzpEdH+ZucFdjv9LNs8BIub14OyeB41pPHk4CLrFMo9cOCRSL/BMT1nH0IpQ+XFCY4qlkLqjYtFUCHQGLvIHnbRUlAYn/fEoTTuiOqOOl0hx/v3ezccJYK7HO+h+fPg+g8nYtcnkej71FZyjB77Ulk7IlsqZ4jCOWUVAvQgQsxiTbxg0tGwrXBWw5cAqCpQTaoS0U1vluZJ3LvSXH0Eqttf2zy;5:I2V6KheFqi2TDRMqRUaCL9lz+WzuQaF12WZqZqqDCyK9Cs5O48Sh/9P6/kXHNZLiDG098qDbIcDjumYaqXaN6VcvvsZDm/zDuOEXy5nFYnD0iDMGcz4/lclLTiWDKbZ7TDIpCAxJe4ziHGnoQwCjq5AVWnJtGD9nqIebWZntuj4=;24:V1b/I3gg3H9mcdjdhMq2CutWO61TL/MV0sBcno4QQeR0lE7le1km3lCGqy2Y7jWRsxXHQPpoXid1QUU2iuXNA025gY3VY7FqebeDke+1TKs=;7:sbs4TyNnhU66KFFOB5+f29BI67Kv6T4lzridGi7K1YbTWJ8Zs8wuaxMOsjFzfl0XF98GXnqyuS95e48CqEHPrjD5yq1OSZL+x3y/3J3lMFzuhqkgu12mU0EZXw2dwQKlL1lPR8JeELHSF8evLRijmNRfSVHq98lJoRQinpAAlz9jhbqaijkg14N+oRxiTx9nEY8evVV5dqmbrDjNa1+O3n4x/7PFOL+y7O+TTv2yv+px4NTZyFdSuXg+ralLSgbFl0BrGAI2qxqfM0aoCPz97UfmYbEQvtDGe20g1Pxqmtv/jcjdyWnZSHKqosuJrtCBpmVDRdWWP1Bcr+hpneqQGw==
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2016 16:11:23.7127 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR07MB2351
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Ard,

thank you for your answer and you explanation.

On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > There is a memory setup problem on ThunderX systems with certain
> > memory configurations. The symptom is
> >
> >  kernel BUG at mm/page_alloc.c:1848!
> >
> > This happens for some configs with 64k page size enabled. The bug
> > triggers for page zones with some pages in the zone not assigned to
> > this particular zone. In my case some pages that are marked as nomap
> > were not reassigned to the new zone of node 1, so those are still
> > assigned to node 0.
> >
> > The reason for the mis-configuration is a change in pfn_valid() which
> > reports pages marked nomap as invalid:
> >
> >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> >
> 
> These pages are owned by the firmware, which may map it with
> attributes that conflict with the attributes we use for the linear
> mapping. This means they should not be covered by the linear mapping.
> 
> > This causes pages marked as nomap being no long reassigned to the new
> > zone in memmap_init_zone() by calling __init_single_pfn().
> >
> 
> This sounds like the root cause of your issue. Could we not fix that instead?

Yes, this is proposal b) from my last mail that would work too: I
implemented an arm64 private early_pfn_valid() function that uses
memblock_is_memory() to setup all pages of a zone. Though, I think
this is the wrong way and thus I prefer this patch instead. I see
serveral reasons for this:

Inconsistent use of struct *page, it is initialized but never used
again.

Other archs only do a basic range check in pfn_valid(), the default
implementation just returns if the whole section is valid. As I
understand the code, if the mem range is not aligned to the section,
then there will be pfn's in the section that don't have physical mem
attached. The page is then just initialized, it's not marked reserved
nor the refcount is non-zero. It is then simply not used. This is how
no-map pages should be handled too.

I think pfn_valid() is just a quick check if the pfn's struct *page
can be used. There is a good description for this in include/linux/
mmzone.h. So there can be memory holes that have a valid pfn.

If the no-map memory needs special handling, then additional checks
need to be added to the particular code (as in ioremap.c). It's imo
wrong to (mis-)use pfn_valid for that.

Variant b) involves generic mm code to fix it for arm64, this patch is
an arm64 change only. This makes it harder to get a fix for it.
(Though maybe only a problem of patch logistics.)

> 
> > Fixing this by restoring the old behavior of pfn_valid() to use
> > memblock_is_memory().
> 
> This is incorrect imo. In general, pfn_valid() means ordinary memory
> covered by the linear mapping and the struct page array. Returning
> reserved ranges that the kernel should not even touch only to please
> the NUMA code seems like an inappropriate way to deal with this issue.

As said above, it is not marked as reserved, it is treated like
non-existing memory.

This has been observed for non-numa kernels too and can happen for
each zone that is only partly initialized.

I think the patch addresses your concerns. I can't see there the
kernel uses memory marked as nomap in a wrong way.

Thanks,

-Robert

> 
> > Also changing users of pfn_valid() in arm64 code
> > to use memblock_is_map_memory() where necessary. This only affects
> > code in ioremap.c. The code in mmu.c still can use the new version of
> > pfn_valid().
> >
> > Should be marked stable v4.5..
> >
> > Signed-off-by: Robert Richter <rrichter@cavium.com>
> > ---
> >  arch/arm64/mm/init.c    | 2 +-
> >  arch/arm64/mm/ioremap.c | 5 +++--
> >  2 files changed, 4 insertions(+), 3 deletions(-)

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S934725AbcJQS6O (ORCPT <rfc822;w@1wt.eu>);
        Mon, 17 Oct 2016 14:58:14 -0400
Received: from mail-qk0-f195.google.com ([209.85.220.195]:36309 "EHLO
        mail-qk0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S932712AbcJQS6F (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Mon, 17 Oct 2016 14:58:05 -0400
Date: Mon, 17 Oct 2016 20:58:01 +0200
From: Robert Richter <rric@kernel.org>
To: Will Deacon <will.deacon@arm.com>, Mark Rutland <mark.rutland@arm.com>
Cc: Robert Richter <robert.richter@cavium.com>,
        Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161017185801.GT25086@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161006161114.GH22012@rric.localdomain>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Mark, Will, any opinion here?

Thanks,

-Robert

On 06.10.16 18:11:14, Robert Richter wrote:
> Ard,
> 
> thank you for your answer and you explanation.
> 
> On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > > There is a memory setup problem on ThunderX systems with certain
> > > memory configurations. The symptom is
> > >
> > >  kernel BUG at mm/page_alloc.c:1848!
> > >
> > > This happens for some configs with 64k page size enabled. The bug
> > > triggers for page zones with some pages in the zone not assigned to
> > > this particular zone. In my case some pages that are marked as nomap
> > > were not reassigned to the new zone of node 1, so those are still
> > > assigned to node 0.
> > >
> > > The reason for the mis-configuration is a change in pfn_valid() which
> > > reports pages marked nomap as invalid:
> > >
> > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > >
> > 
> > These pages are owned by the firmware, which may map it with
> > attributes that conflict with the attributes we use for the linear
> > mapping. This means they should not be covered by the linear mapping.
> > 
> > > This causes pages marked as nomap being no long reassigned to the new
> > > zone in memmap_init_zone() by calling __init_single_pfn().
> > >
> > 
> > This sounds like the root cause of your issue. Could we not fix that instead?
> 
> Yes, this is proposal b) from my last mail that would work too: I
> implemented an arm64 private early_pfn_valid() function that uses
> memblock_is_memory() to setup all pages of a zone. Though, I think
> this is the wrong way and thus I prefer this patch instead. I see
> serveral reasons for this:
> 
> Inconsistent use of struct *page, it is initialized but never used
> again.
> 
> Other archs only do a basic range check in pfn_valid(), the default
> implementation just returns if the whole section is valid. As I
> understand the code, if the mem range is not aligned to the section,
> then there will be pfn's in the section that don't have physical mem
> attached. The page is then just initialized, it's not marked reserved
> nor the refcount is non-zero. It is then simply not used. This is how
> no-map pages should be handled too.
> 
> I think pfn_valid() is just a quick check if the pfn's struct *page
> can be used. There is a good description for this in include/linux/
> mmzone.h. So there can be memory holes that have a valid pfn.
> 
> If the no-map memory needs special handling, then additional checks
> need to be added to the particular code (as in ioremap.c). It's imo
> wrong to (mis-)use pfn_valid for that.
> 
> Variant b) involves generic mm code to fix it for arm64, this patch is
> an arm64 change only. This makes it harder to get a fix for it.
> (Though maybe only a problem of patch logistics.)
> 
> > 
> > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > memblock_is_memory().
> > 
> > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > covered by the linear mapping and the struct page array. Returning
> > reserved ranges that the kernel should not even touch only to please
> > the NUMA code seems like an inappropriate way to deal with this issue.
> 
> As said above, it is not marked as reserved, it is treated like
> non-existing memory.
> 
> This has been observed for non-numa kernels too and can happen for
> each zone that is only partly initialized.
> 
> I think the patch addresses your concerns. I can't see there the
> kernel uses memory marked as nomap in a wrong way.
> 
> Thanks,
> 
> -Robert
> 
> > 
> > > Also changing users of pfn_valid() in arm64 code
> > > to use memblock_is_map_memory() where necessary. This only affects
> > > code in ioremap.c. The code in mmu.c still can use the new version of
> > > pfn_valid().
> > >
> > > Should be marked stable v4.5..
> > >
> > > Signed-off-by: Robert Richter <rrichter@cavium.com>
> > > ---
> > >  arch/arm64/mm/init.c    | 2 +-
> > >  arch/arm64/mm/ioremap.c | 5 +++--
> > >  2 files changed, 4 insertions(+), 3 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-efi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1758784AbcJRKTR (ORCPT <rfc822;w@1wt.eu>);
        Tue, 18 Oct 2016 06:19:17 -0400
Received: from foss.arm.com ([217.140.101.70]:56072 "EHLO foss.arm.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1758088AbcJRKTI (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 18 Oct 2016 06:19:08 -0400
Date: Tue, 18 Oct 2016 11:18:36 +0100
From: Mark Rutland <mark.rutland@arm.com>
To: Robert Richter <robert.richter@cavium.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will.deacon@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161018101715.GA15639@leverpostej>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161006161114.GH22012@rric.localdomain>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Robert, Ard,

Sorry for the delay in getting to this; I've been travelling a lot
lately and in the meantime this managed to get buried in my inbox.

On Thu, Oct 06, 2016 at 06:11:14PM +0200, Robert Richter wrote:
> On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > > There is a memory setup problem on ThunderX systems with certain
> > > memory configurations. The symptom is
> > >
> > >  kernel BUG at mm/page_alloc.c:1848!
> > >
> > > This happens for some configs with 64k page size enabled. The bug
> > > triggers for page zones with some pages in the zone not assigned to
> > > this particular zone. In my case some pages that are marked as nomap
> > > were not reassigned to the new zone of node 1, so those are still
> > > assigned to node 0.
> > >
> > > The reason for the mis-configuration is a change in pfn_valid() which
> > > reports pages marked nomap as invalid:
> > >
> > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > 
> > These pages are owned by the firmware, which may map it with
> > attributes that conflict with the attributes we use for the linear
> > mapping. This means they should not be covered by the linear mapping.
> > 
> > > This causes pages marked as nomap being no long reassigned to the new
> > > zone in memmap_init_zone() by calling __init_single_pfn().

Why do we have pages for a nomap region? Given the region shouldn't be
in the linear mapping, and isn't suitable for general allocation, I
don't believe it makes sense to have a struct page for any part of it.

Am I missing some reason that we require a struct page?

e.g. is it just easier to allocate an unused struct page than to carve
it out?

> > This sounds like the root cause of your issue. Could we not fix that instead?
> 
> Yes, this is proposal b) from my last mail that would work too: I
> implemented an arm64 private early_pfn_valid() function that uses
> memblock_is_memory() to setup all pages of a zone. Though, I think
> this is the wrong way and thus I prefer this patch instead. I see
> serveral reasons for this:
> 
> Inconsistent use of struct *page, it is initialized but never used
> again.

As above, I don't believe we should have a struct page to initialise in
the first place.

> Other archs only do a basic range check in pfn_valid(), the default
> implementation just returns if the whole section is valid. As I
> understand the code, if the mem range is not aligned to the section,
> then there will be pfn's in the section that don't have physical mem
> attached. The page is then just initialized, it's not marked reserved
> nor the refcount is non-zero. It is then simply not used. This is how
> no-map pages should be handled too.
> 
> I think pfn_valid() is just a quick check if the pfn's struct *page
> can be used. There is a good description for this in include/linux/
> mmzone.h. So there can be memory holes that have a valid pfn.

I take it you mean the comment in the CONFIG_ARCH_HAS_HOLES_MEMORYMODEL
ifdef (line 1266 in v4.9-rc1)?

I'm not sufficiently acquainted with the memmap code to follow; I'll
need to dig into that a bit further.

> If the no-map memory needs special handling, then additional checks
> need to be added to the particular code (as in ioremap.c). It's imo
> wrong to (mis-)use pfn_valid for that.
> 
> Variant b) involves generic mm code to fix it for arm64, this patch is
> an arm64 change only. This makes it harder to get a fix for it.
> (Though maybe only a problem of patch logistics.)
> 
> > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > memblock_is_memory().
> > 
> > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > covered by the linear mapping and the struct page array. Returning
> > reserved ranges that the kernel should not even touch only to please
> > the NUMA code seems like an inappropriate way to deal with this issue.
> 
> As said above, it is not marked as reserved, it is treated like
> non-existing memory.

I think Ard was using "reserved" in the more general sense than the
Linux-specific meaning. NOMAP is distinct from the Linux concept of
"reserved" memory, but is "reserved" in some sense.

Memory with NOMAP is meant to be treated as non-existent for the purpose
of the linear mapping (and thus for the purpose of struct page).

> This has been observed for non-numa kernels too and can happen for
> each zone that is only partly initialized.
> 
> I think the patch addresses your concerns. I can't see there the
> kernel uses memory marked as nomap in a wrong way.

I'll have to dig into this locally; I'm still not familiar enough with
this code to know what the right thing to do is.

Thanks,
Mark.

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S936491AbcJRPfv (ORCPT <rfc822;w@1wt.eu>);
        Tue, 18 Oct 2016 11:35:51 -0400
Received: from mail-by2nam03on0062.outbound.protection.outlook.com ([104.47.42.62]:28928
        "EHLO NAM03-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S935019AbcJRPfk (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 18 Oct 2016 11:35:40 -0400
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Tue, 18 Oct 2016 17:02:40 +0200
From: Robert Richter <robert.richter@cavium.com>
To: Mark Rutland <mark.rutland@arm.com>
CC: Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will.deacon@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161018150240.GU22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161018101715.GA15639@leverpostej>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20161018101715.GA15639@leverpostej>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.89.64]
X-ClientProxiedBy: AM5PR0601CA0039.eurprd06.prod.outlook.com (10.173.86.153)
 To SN1PR07MB2351.namprd07.prod.outlook.com (10.169.127.17)
X-MS-Office365-Filtering-Correlation-Id: bf84c037-72fc-49b6-0d76-08d3f767d526
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;2:ov4g1dPguCEU+8hWO9jNMi42vr+2m2SoQfbpniUPqGiekWTFZ1G6bqh5Lsl+V63un5az8Cm40WOYdVVLiSKJKdVlUUObG6wIbUHuO2mishHjzvWpNVDLK0CVNwwo4HDkM51S/YjN95l3FO/UG5goBBMA+O9KIc2JpKnCpTrmz5urttqPgb4FBdgTfvFW2voBuoGZ5Z02mfNvMgt8oxmHbQ==;3:cddUKFlBO3JRlQaN6ZtjiA7QUd9E37YB/XI1tOHREdD6MGgDJ9UB6P0lEygf5Ulsq8Ds7qD1lxNZ1KyhZogC/MNZA06Vcknb2uCgwZml78mQFKrd05wJhT2WhBfeq0jKiExDba02niH+egM+Hd9pog==
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2351;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;25: 
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;31:ZQGazl6sbAK6URvxT60OklbhTIwqLjk4iVOpoekjx2E4QtNTq9mffRVW8/jCMfO9P1WsjJCMyVcV+D32v4f3LiYyctArfnWNxEzAquguak/PMA8yrTYB6x8HRxwWf2Q5qZKOMpgtCjnyXbLs3SemX6BvkuQtRkGO1PJanx2DXWN6zcegoAKp+pY2+K+NoerYaGo/4V9bp9cFvCFZMnzXulfpXscjEcFhqwpHT4cyYIlPQBpsF+K5GZ8+uK12lrjeGbNzvXfwbzc5aZRn/8OtxA==;20: 
X-Microsoft-Antispam-PRVS: <SN1PR07MB2351E71FE5014502347B2107F1D30@SN1PR07MB2351.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(20558992708506)(192374486261705);
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:SN1PR07MB2351;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2351;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;4:JIX1zgqPosBpgCS0hwOgBkO/fAmeGj4lowNk8h0cTkwxwvpvbDSwYD6uZxO+cSR157CQbfVSk61ZrBIC5iKmP6ZwNps1nMKclwnv4pL7WfPldLdqxyjAaBXArKV+fld0wtbcPODg42oYyjZhlFZIHb0JTXHdrzqbsCwX2FZmlHOAZ5HLL16LDDnkeTxEzC2yvWf/z/7zHe/EHnlCDAnFKaTSWzuhhsMCtHqfKuYjs5TtLkSN4bpqKsCx8FQbGDnwgRfd4uTSzdiqGw1WukBXmT6G6a0qh7Z9YlzF5QZeP+PjOaUyOeOz7xybLByewQe/toubl1afLfDwK84L08ndKxKB70BM1WwFlS5B4L91gn79AmOAXq35Px/zRbkT9sq9Q2mTdciPUOD79lNzzw/12excZvxezaMInjKGcvgWBoFRUw4KzuwqKefG23tNf71dRIvl/J+OOW0ryo65ZMCoKK44jGdeTANPAaGZEPP50bM=
X-Forefront-PRVS: 00997889E7
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(24454002)(199003)(189002)(55674003)(575784001)(86362001)(101416001)(19580405001)(23726003)(2906002)(1076002)(19580395003)(5890100001)(586003)(2950100002)(93886004)(7736002)(8676002)(6666003)(4326007)(305945005)(5660300001)(92566002)(81166006)(6916009)(83506001)(33656002)(106356001)(189998001)(54356999)(76176999)(50986999)(81156014)(68736007)(7846002)(6116002)(105586002)(3846002)(9686002)(110136003)(77096005)(4001350100001)(47776003)(42186005)(46406003)(561944003)(97736004)(66066001)(15974865002)(50466002)(97756001)(18370500001)(19627235001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR07MB2351;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;SN1PR07MB2351;23:/deQJloFKy5ND+eQazcYqHH2vwLySbcabzeTWoQKZ?=
 =?us-ascii?Q?E70miapL+/peRae+GtnEeIokqzsbg6fV11AQkO0NMYDEAIWuDFUYTsTY9JOa?=
 =?us-ascii?Q?WY8bjsJjgen8iCVYmMH7pAR9G14B9FKHnDi2leKf6DhMzGXySvSwjYZ3j8ey?=
 =?us-ascii?Q?fBQOBbYXsdncEv8qBq5K+hwaZAoqQsHZ3GvFt1Cqoq9MvxgkLAubQwV6odhO?=
 =?us-ascii?Q?Y0hGx2nD4da78NjhmWlCz86Hwo54HkSIfzjSqQI/EfAWdsLoqDOCtNd2Q+f5?=
 =?us-ascii?Q?xz0A2gejP1jgSPHxu57aMzMZp9qXKLGanKUjl+wCzFNkfDMfY9GMCAg6OGEF?=
 =?us-ascii?Q?tPBhEW5QUJH2tD2AQ2nMZ9NySqoGupVHZu1AtDs/YRlNfa+fwi0gJfLI2WBU?=
 =?us-ascii?Q?H6Bo7aSuGVhN48QeBPAEm1Wm6epPEcKk/KSL/ZhJfBV3Dqu2NleiRS69KWOd?=
 =?us-ascii?Q?uekrN5R/2YYf6wlpSwcKsYfOfQAcvsAda7lKdz0CZa9vQe0KsCbJRJyHPvD2?=
 =?us-ascii?Q?ZYrMv9N0Oz8tulgs8/sFU10jDHEl04Hdg+4pSx6/VU7vYbmYsVW2bbuDh/TG?=
 =?us-ascii?Q?5t3CacUICzsxd/Y1mpJ19+TQ4ahKoHcsiOTuHmLbXvimQTxLGVskYehs6qwP?=
 =?us-ascii?Q?w8vBVJhtScbL/nYCUr957e6cMo6oa4hqE/1KXLq4d11V5Zq8+GDPXV/Zl3rc?=
 =?us-ascii?Q?GZ/9XqhWMgp4wXgebzoarJmVmyKP4OBHVdPIIVQK1d2SzHwdGbj8ByuF3WKm?=
 =?us-ascii?Q?R05Qi3Xi2gLXkEVOmP3By0lgHeoTGZfDgtptUPAVXLRBpKpt1pxk7N+pXUkm?=
 =?us-ascii?Q?P/kkHWLm0EM8IEldonUSJK2EwApiHzpLFRvXiH1ji5ZbKtpPU1CyKqi0bpfI?=
 =?us-ascii?Q?l/RZR/NKJg51pbNv91xmv4viHrZHPbjs+U/iCoAdsrSez+BW3gzwhtX4oc7y?=
 =?us-ascii?Q?ur4Lrvk2gxQ9ZuOzbfmlVSKoOI8Y90ua1+BwMSaFOpzO159BuuEoEKG/kJIv?=
 =?us-ascii?Q?oSSrJZOoJ4oRkd91QC6nbfhCPVCTovAkRzxN7BJjWTjlTZaMuh1RJFalJ2/m?=
 =?us-ascii?Q?Rj53aMImtFABL2LkY9D7jL3viiQ6y5+eo8B+NflnpvxwdT69C38Cq48l7C2N?=
 =?us-ascii?Q?V0s3EWOsmaYK1tEE/el640uq4Q5Of8KFOHRT8ULdUd/nXAR04+ME8rN4xyJC?=
 =?us-ascii?Q?i+ZDg8MtU4IDu5SPKKhLhWnuzgLHy7RJMGymjm+E5yWlkd3m2SeKRRyLbzBv?=
 =?us-ascii?Q?d6J9O01E5gDbnYGJGgs2g0+cLY+3J3mx37Vtoy/mNhNrLfmqCh57iS1OadID?=
 =?us-ascii?Q?Fh45FyGLp1kZrydb4KqIUi483wVN3qFHynBY7xycK3HmXgoMC/j+vUD8SnoD?=
 =?us-ascii?Q?BUrw1+SnuZ1dFCtEhjliWusvDDoBpkC0RDWFdYCz/k9OiP/?=
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;6:gmIPkxScn1+XjpgDRoefMw8g8qMkR9ZVTjZbwOafu5NDBVPDd3SkQKBUL+gPB9A/ZuanKtOJlL1ixdoa2E217+zTl6DJjuDIpGizvf8NqT8tr2ioylCtQNqzFJSqMbtF09kbY/+YtH1XfBh5sKyAmppX0rcey4zkMUvpxknFXgrNc4ivXYNMHcp8fRRVuQS55kkt3/xUOjxfEdmf2VEIxh1LHWuhX/l86IWDfOPMRzvXb6G8YnarA1zap5ZhMVpDszqUKApXhEI8e/f1zNgev+idae9Vve7s16rGQheXTPHUTcYkfmfY34DZ+R2KyrdC;5:MhkG2hp69W2zhpsjC3HOwCB9YqB9gIGEJfaNekmvMHXfcNovEfpsaa5pD+nfbZ7zXMms76dxzK1IncwCryV7xxAyIsVRcNMovtKFClq5ItDItHBJbrk5z2vMIXyjqJ5ezZG/rZm/lCsa1/D4u75QKNUd7PHaNjmS8fUbg6OyMKc=;24:LdHvTILXp2eqzz/5cAb1Ywv9fJUpoXrSgny3xNICMrU0meMidVzdzl9LZvbVoLkZO3ZTkBYxzePPQiBBwEtKVnIv7ilV5KBzh0uLmpbTAMk=;7:EPlgx/h4zfYX6GXCBTEvtP+EEXxsFs2zRY4EFbAu7oh6l4NBqBksap8nsVqFnHiqwm7tShEqoNFkM703cuRbY1nME2KYoUMTw5nHEtUhfNqOXtJ46BaXqFXeNNhisSOjA3tFIwDdeiNx/UD7N3GjpjgWVhqzKmN1PJIn54jf6pQyVBB05LpRzpMIGq4qEW8ivYrd1SwcWOMQllLszjeFpZWCMj+VN6Tc0Xv5ceCldyrVEtVlV/3ehlMXv80amRmeia6amy/EFDkYTtU/gbHUGA/b3V1FqdbVfwH0u33jknf34HD9PnPKSaqAIDRdkmL90wkXupW/duOLczkdwh9IsDo3koe9WY4hg2xRjh2lbMk=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2016 15:02:50.1570 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR07MB2351
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Mark,

thanks for your answer. See below. I also attached the full crash
dump.

On 18.10.16 11:18:36, Mark Rutland wrote:
> Hi Robert, Ard,
> 
> Sorry for the delay in getting to this; I've been travelling a lot
> lately and in the meantime this managed to get buried in my inbox.
> 
> On Thu, Oct 06, 2016 at 06:11:14PM +0200, Robert Richter wrote:
> > On 06.10.16 11:00:33, Ard Biesheuvel wrote:
> > > On 6 October 2016 at 10:52, Robert Richter <rrichter@cavium.com> wrote:
> > > > There is a memory setup problem on ThunderX systems with certain
> > > > memory configurations. The symptom is
> > > >
> > > >  kernel BUG at mm/page_alloc.c:1848!
> > > >
> > > > This happens for some configs with 64k page size enabled. The bug
> > > > triggers for page zones with some pages in the zone not assigned to
> > > > this particular zone. In my case some pages that are marked as nomap
> > > > were not reassigned to the new zone of node 1, so those are still
> > > > assigned to node 0.
> > > >
> > > > The reason for the mis-configuration is a change in pfn_valid() which
> > > > reports pages marked nomap as invalid:
> > > >
> > > >  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> > > 
> > > These pages are owned by the firmware, which may map it with
> > > attributes that conflict with the attributes we use for the linear
> > > mapping. This means they should not be covered by the linear mapping.
> > > 
> > > > This causes pages marked as nomap being no long reassigned to the new
> > > > zone in memmap_init_zone() by calling __init_single_pfn().
> 
> Why do we have pages for a nomap region? Given the region shouldn't be
> in the linear mapping, and isn't suitable for general allocation, I
> don't believe it makes sense to have a struct page for any part of it.
> 
> Am I missing some reason that we require a struct page?
> 
> e.g. is it just easier to allocate an unused struct page than to carve
> it out?

Pages are handled in blocks with size MAX_ORDER_NR_PAGES. The start
and end pfn of a memory region is aligned then to fit the mem block
size (see memmap_init_zone()). Therefore a memblock may contain pages
without underlying physical memory.

mm code requires the whole memmap to be initialized, this means that
for each page in the whole mem block there is a valid struct page. See
e.g. move_freepages_block() and move_freepages(), stuct page is
accessed even before pfn_valid() is used. I assume there are other
occurrences of that too.

My interpretation is that pfn_valid() checks for the existence of a
valid struct page, there must not necessarily phys memory mapped to
it. This is the reason why I changed pfn_valid() to use
memblock_is_memory() which is sufficient for generic mm code. Only in
arm64 mm code I additinally added the memblock_is_map_memory() check
where pfn_valid() was used.

> 
> > > This sounds like the root cause of your issue. Could we not fix that instead?
> > 
> > Yes, this is proposal b) from my last mail that would work too: I
> > implemented an arm64 private early_pfn_valid() function that uses
> > memblock_is_memory() to setup all pages of a zone. Though, I think
> > this is the wrong way and thus I prefer this patch instead. I see
> > serveral reasons for this:
> > 
> > Inconsistent use of struct *page, it is initialized but never used
> > again.
> 
> As above, I don't believe we should have a struct page to initialise in
> the first place.
> 
> > Other archs only do a basic range check in pfn_valid(), the default
> > implementation just returns if the whole section is valid. As I
> > understand the code, if the mem range is not aligned to the section,
> > then there will be pfn's in the section that don't have physical mem
> > attached. The page is then just initialized, it's not marked reserved
> > nor the refcount is non-zero. It is then simply not used. This is how
> > no-map pages should be handled too.
> > 
> > I think pfn_valid() is just a quick check if the pfn's struct *page
> > can be used. There is a good description for this in include/linux/
> > mmzone.h. So there can be memory holes that have a valid pfn.
> 
> I take it you mean the comment in the CONFIG_ARCH_HAS_HOLES_MEMORYMODEL
> ifdef (line 1266 in v4.9-rc1)?

Yes.

> 
> I'm not sufficiently acquainted with the memmap code to follow; I'll
> need to dig into that a bit further.
> 
> > If the no-map memory needs special handling, then additional checks
> > need to be added to the particular code (as in ioremap.c). It's imo
> > wrong to (mis-)use pfn_valid for that.
> > 
> > Variant b) involves generic mm code to fix it for arm64, this patch is
> > an arm64 change only. This makes it harder to get a fix for it.
> > (Though maybe only a problem of patch logistics.)
> > 
> > > > Fixing this by restoring the old behavior of pfn_valid() to use
> > > > memblock_is_memory().
> > > 
> > > This is incorrect imo. In general, pfn_valid() means ordinary memory
> > > covered by the linear mapping and the struct page array. Returning
> > > reserved ranges that the kernel should not even touch only to please
> > > the NUMA code seems like an inappropriate way to deal with this issue.
> > 
> > As said above, it is not marked as reserved, it is treated like
> > non-existing memory.
> 
> I think Ard was using "reserved" in the more general sense than the
> Linux-specific meaning. NOMAP is distinct from the Linux concept of
> "reserved" memory, but is "reserved" in some sense.
> 
> Memory with NOMAP is meant to be treated as non-existent for the purpose
> of the linear mapping (and thus for the purpose of struct page).

Yes, it's not marked reserved and can never be freed.

Thanks,

-Robert

> 
> > This has been observed for non-numa kernels too and can happen for
> > each zone that is only partly initialized.
> > 
> > I think the patch addresses your concerns. I can't see there the
> > kernel uses memory marked as nomap in a wrong way.
> 
> I'll have to dig into this locally; I'm still not familiar enough with
> this code to know what the right thing to do is.
> 
> Thanks,
> Mark.

EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
Booting Linux on physical CPU 0x0
Linux version 4.8.0-rc4-00267-g664e88c (root@crb2spass2rric.localdomain) (gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.1) ) #3 SMP Wed Sep 7 06:38:13 UTC 2016
Boot CPU: AArch64 Processor [431f0a10]
efi: Getting EFI parameters from FDT:
efi: EFI v2.40 by Cavium Thunder cn88xx EFI ThunderX-Firmware-Release-1.22.10-0-g4e85766 Aug 24 2016 15:59:03
efi:  ACPI=0xfffff000  ACPI 2.0=0xfffff014  SMBIOS 3.0=0x10ffafcf000 
cma: Reserved 512 MiB at 0x00000000c0000000
NUMA: Adding memblock [0x1400000 - 0xfffffffff] on node 0
NUMA: Adding memblock [0x10000400000 - 0x10fffffffff] on node 1
NUMA: parsing numa-distance-map-v1
NUMA: Initmem setup node 0 [mem 0x01400000-0xfffffffff]
NUMA: NODE_DATA [mem 0xfffff2580-0xfffffffff]
NUMA: Initmem setup node 1 [mem 0x10000400000-0x10fffffffff]
NUMA: NODE_DATA [mem 0x10ffffa2500-0x10ffffaff7f]
kmemleak: Kernel memory leak detector disabled
Zone ranges:
  DMA      [mem 0x0000000001400000-0x00000000ffffffff]
  Normal   [mem 0x0000000100000000-0x0000010fffffffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000001400000-0x00000000fffdffff]
  node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
  node   0: [mem 0x0000000100000000-0x0000000fffffffff]
  node   1: [mem 0x0000010000400000-0x0000010ff9e8ffff]
  node   1: [mem 0x0000010ff9e90000-0x0000010ff9f2ffff]
  node   1: [mem 0x0000010ff9f30000-0x0000010ffaeaffff]
  node   1: [mem 0x0000010ffaeb0000-0x0000010ffaffffff]
  node   1: [mem 0x0000010ffb000000-0x0000010ffffaffff]
  node   1: [mem 0x0000010ffffb0000-0x0000010fffffffff]
Initmem setup node 0 [mem 0x0000000001400000-0x0000000fffffffff]
Initmem setup node 1 [mem 0x0000010000400000-0x0000010fffffffff]
psci: probing for conduit method from DT.
psci: PSCIv0.2 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: Trusted OS resident on physical CPU 0x0
Number of cores (96) exceeds configured maximum of 8 - clipping
percpu: Embedded 3 pages/cpu @ffffff0fffce0000 s117704 r8192 d70712 u196608
Detected VIPT I-cache on CPU0
CPU features: enabling workaround for Cavium erratum 27456
Built 2 zonelists in Node order, mobility grouping on.  Total pages: 2094720
Policy zone: Normal
Kernel command line: BOOT_IMAGE=/vmlinuz root=UUID=9a418ae5-231a-40c7-b7ba-ca70cfadfc5e ro LANG=en_US.UTF-8
PID hash table entries: 4096 (order: -1, 32768 bytes)
software IO TLB [mem 0xfbfd0000-0xfffd0000] (64MB) mapped at [fffffe00fbfd0000-fffffe00fffcffff]
Memory: 133196160K/134193152K available (9084K kernel code, 1545K rwdata, 3712K rodata, 1536K init, 15826K bss, 472704K reserved, 524288K cma-reserved)
Virtual kernel memory layout:
    modules : 0xfffffc0000000000 - 0xfffffc0008000000   (   128 MB)
    vmalloc : 0xfffffc0008000000 - 0xfffffdff5fff0000   (  2045 GB)
      .text : 0xfffffc0008080000 - 0xfffffc0008960000   (  9088 KB)
    .rodata : 0xfffffc0008960000 - 0xfffffc0008d10000   (  3776 KB)
      .init : 0xfffffc0008d10000 - 0xfffffc0008e90000   (  1536 KB)
      .data : 0xfffffc0008e90000 - 0xfffffc0009012600   (  1546 KB)
       .bss : 0xfffffc0009012600 - 0xfffffc0009f86fc8   ( 15827 KB)
    fixed   : 0xfffffdff7e7d0000 - 0xfffffdff7ec00000   (  4288 KB)
    PCI I/O : 0xfffffdff7ee00000 - 0xfffffdff7fe00000   (    16 MB)
    vmemmap : 0xfffffdff80000000 - 0xfffffe0000000000   (     2 GB maximum)
              0xfffffdff80005000 - 0xfffffdffc4000000   (  1087 MB actual)
    memory  : 0xfffffe0001400000 - 0xffffff1000000000   (1114092 MB)
SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=8, Nodes=2
Running RCU self tests
Hierarchical RCU implementation.
	RCU lockdep checking is enabled.
	Build-time adjustment of leaf fanout to 64.
NR_IRQS:64 nr_irqs:64 0
GICv3: GIC: Using split EOI/Deactivate mode
ITS: /interrupt-controller@801000000000/gic-its@801000020000
ITS@0x0000801000020000: allocated 2097152 Devices @ff5000000 (flat, esz 8, psz 64K, shr 1)
ITS: /interrupt-controller@801000000000/gic-its@901000020000
ITS@0x0000901000020000: allocated 2097152 Devices @ffa000000 (flat, esz 8, psz 64K, shr 1)
GIC: using LPI property table @0x0000000ff4140000
ITS: Allocated 32512 chunks for LPIs
GICv3: CPU0: found redistributor 0 region 0:0x0000801080000000
CPU0: using LPI pending table @0x0000000ff4150000
arm_arch_timer: Architected cp15 timer(s) running at 100.00MHz (phys).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x171024e7e0, max_idle_ns: 440795205315 ns
sched_clock: 56 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns
Console: colour dummy device 80x25
console [tty0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:  8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        8191
... CLASSHASH_SIZE:          4096
... MAX_LOCKDEP_ENTRIES:     32768
... MAX_LOCKDEP_CHAINS:      65536
... CHAINHASH_SIZE:          32768
 memory used by lock dependency info: 8159 kB
 per task-struct memory footprint: 1920 bytes
mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=100000)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Yama: becoming mindful.
SELinux:  Initializing.
Dentry cache hash table entries: 16777216 (order: 11, 134217728 bytes)
Inode-cache hash table entries: 8388608 (order: 10, 67108864 bytes)
Mount-cache hash table entries: 262144 (order: 5, 2097152 bytes)
Mountpoint-cache hash table entries: 262144 (order: 5, 2097152 bytes)
ftrace: allocating 29244 entries in 8 pages
Unable to find CPU node for /cpus/cpu@8
/cpus/cpu-map/cluster0/core8: Can't get CPU for leaf core
ASID allocator initialised with 65536 entries
PCI/MSI: /interrupt-controller@801000000000/gic-its@801000020000 domain created
PCI/MSI: /interrupt-controller@801000000000/gic-its@901000020000 domain created
Platform MSI: /interrupt-controller@801000000000/gic-its@801000020000 domain created
Platform MSI: /interrupt-controller@801000000000/gic-its@901000020000 domain created
Remapping and enabling EFI services.
UEFI Runtime regions are not aligned to 64 KB -- buggy firmware?
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at arch/arm64/kernel/efi.c:34 efi_create_mapping+0x5c/0x11c
Modules linked in:

CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.8.0-rc4-00267-g664e88c #3
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fffffe0fee55a200 task.stack: ffffff00004a4000
PC is at efi_create_mapping+0x5c/0x11c
LR is at efi_create_mapping+0x5c/0x11c
pc : [<fffffc0008d14f68>] lr : [<fffffc0008d14f68>] pstate: 60000045
sp : ffffff00004a7d00
x29: ffffff00004a7d00 x28: 0000000000000000 
x27: 0000000000000000 x26: 0000000000000000 
x25: 0000000000000000 x24: fffffc0008f9c730 
x23: fffffc0008f9c730 x22: fffffc0008c60ac0 
x21: fffffc0008f9c730 x20: 00c8000000000713 
x19: ffffff0ff76b0c08 x18: 0000000000000010 
x17: 00000000c9cbbfc7 x16: 0000000000000000 
x15: fffffc0089c956f7 x14: 3f657261776d7269 
x13: 6620796767756220 x12: 2d2d20424b203436 
x11: 206f742064656e67 x10: 0000000000000077 
x9 : 65726120736e6f69 x8 : 0000000000000001 
x7 : ffffff00004a4000 x6 : fffffc000814a288 
x5 : 0000000000000000 x4 : 0000000000000001 
x3 : 0000000000000001 x2 : ffffff00004a4000 
x1 : fffffe0fee55a200 x0 : 0000000000000040 

---[ end trace fcdd7f8cb96cdb3b ]---
Call trace:
Exception stack(0xffffff00004a7b20 to 0xffffff00004a7c50)
7b20: ffffff0ff76b0c08 0000040000000000 ffffff00004a7d00 fffffc0008d14f68
7b40: 0000000060000045 000000000000003d 0000000000000001 fffffc000814a9fc
7b60: ffffff00004a7c00 fffffc000814adcc ffffff00004a7c60 fffffc0008baf420
7b80: fffffc0008f9c730 fffffc0008c60ac0 fffffc0008f9c730 fffffc0008f9c730
7ba0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7bc0: fffffc00080d6d70 0000000000000000 0000000000000040 fffffe0fee55a200
7be0: ffffff00004a4000 0000000000000001 0000000000000001 0000000000000000
7c00: fffffc000814a288 ffffff00004a4000 0000000000000001 65726120736e6f69
7c20: 0000000000000077 206f742064656e67 2d2d20424b203436 6620796767756220
7c40: 3f657261776d7269 fffffc0089c956f7
[<fffffc0008d14f68>] efi_create_mapping+0x5c/0x11c
[<fffffc0008d5e1ec>] arm_enable_runtime_services+0x12c/0x210
[<fffffc0008082ff4>] do_one_initcall+0x44/0x138
[<fffffc0008d10d30>] kernel_init_freeable+0x17c/0x2e0
[<fffffc000893c888>] kernel_init+0x20/0xf8
[<fffffc0008082b80>] ret_from_fork+0x10/0x50
  EFI remap 0x0000010ff9e98000 => 0000000020008000
  EFI remap 0x0000010ffaeb6000 => 00000000200a6000
  EFI remap 0x0000010ffafc9000 => 00000000201b9000
  EFI remap 0x0000010ffafcd000 => 00000000201bd000
  EFI remap 0x0000010ffffb9000 => 00000000201f9000
  EFI remap 0x0000010ffffcd000 => 000000002020d000
  EFI remap 0x0000804000001000 => 0000000020241000
  EFI remap 0x000087e0d0001000 => 0000000020251000
Detected VIPT I-cache on CPU1
GICv3: CPU1: found redistributor 1 region 0:0x0000801080020000
CPU1: using LPI pending table @0x0000010014060000
CPU1: Booted secondary processor [431f0a10]
------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at ./include/linux/cpumask.h:121 gic_raise_softirq+0x14c/0x1e0
Modules linked in:

CPU: 1 PID: 0 Comm: swapper/1 Tainted: G        W       4.8.0-rc4-00267-g664e88c #3
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fffffe0fee57b600 task.stack: fffffe0fe4038000
PC is at gic_raise_softirq+0x14c/0x1e0
LR is at gic_raise_softirq+0xc0/0x1e0
pc : [<fffffc00084c07a4>] lr : [<fffffc00084c0718>] pstate: 600001c5
sp : fffffe0fe403bcd0
x29: fffffe0fe403bcd0 x28: 0000000000000001 
x27: fffffc0008fd7b1d x26: 0000000000000000 
x25: fffffc0008ec2000 x24: fffffc0008ecebc8 
x23: fffffc0008998688 x22: 0000000000000001 
x21: fffffc0008ec2e34 x20: 0000000000000000 
x19: 0000000000000008 x18: 0000000000000030 
x17: 0000000000000000 x16: 0000000000000000 
x15: fffffc0009c97f08 x14: 0000000000000002 
x13: 0000000000000332 x12: 0000000000000335 
x11: fffffc0009bf3de0 x10: 0000000000000332 
x9 : fffffe0fe4038000 x8 : 0000000000000000 
x7 : 00000000ffffffff x6 : 0000000000000000 
x5 : fffffffffffffffe x4 : 0000000000000040 
x3 : 0000000000000000 x2 : 0000000000000000 
x1 : 0000000000000008 x0 : 0000000000000001 

---[ end trace fcdd7f8cb96cdb3c ]---
Call trace:
Exception stack(0xfffffe0fe403baf0 to 0xfffffe0fe403bc20)
bae0:                                   0000000000000008 0000040000000000
bb00: fffffe0fe403bcd0 fffffc00084c07a4 00000000600001c5 000000000000003d
bb20: fffffe0fee57bec0 ffffffffffffffd8 0000000000000028 fffffe0fee57be70
bb40: fffffc000925fe88 0000000000000002 fffffc00080886c0 fffffe0fee57b600
bb60: fffffe0fe403bb90 fffffc0008088974 fffffc00099b4d60 fffffe0fee57b600
bb80: fffffc000925fe88 fffffe0fe403bd4c fffffe0fe403bbe0 fffffc00080889e8
bba0: 0000000000000001 0000000000000008 0000000000000000 0000000000000000
bbc0: 0000000000000040 fffffffffffffffe 0000000000000000 00000000ffffffff
bbe0: 0000000000000000 fffffe0fe4038000 0000000000000332 fffffc0009bf3de0
bc00: 0000000000000335 0000000000000332 0000000000000002 fffffc0009c97f08
[<fffffc00084c07a4>] gic_raise_softirq+0x14c/0x1e0
[<fffffc000808e028>] smp_cross_call+0x80/0x238
[<fffffc000808efa4>] smp_send_reschedule+0x3c/0x48
[<fffffc00081021a8>] resched_curr+0x58/0x90
[<fffffc0008103fd8>] check_preempt_curr+0x70/0xe8
[<fffffc0008104090>] ttwu_do_wakeup+0x40/0x2e8
[<fffffc00081043cc>] ttwu_do_activate+0x94/0xc0
[<fffffc00081058a8>] try_to_wake_up+0x200/0x580
[<fffffc0008105d3c>] default_wake_function+0x34/0x48
[<fffffc0008126afc>] __wake_up_common+0x64/0xa8
[<fffffc0008126bec>] __wake_up_locked+0x3c/0x50
[<fffffc0008127b58>] complete+0x48/0x68
[<fffffc000808e630>] secondary_start_kernel+0x180/0x208
[<0000000001482e08>] 0x1482e08
Detected VIPT I-cache on CPU2
GICv3: CPU2: found redistributor 2 region 0:0x0000801080040000
CPU2: using LPI pending table @0x0000010014070000
CPU2: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU3
GICv3: CPU3: found redistributor 3 region 0:0x0000801080060000
CPU3: using LPI pending table @0x0000010014080000
CPU3: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU4
GICv3: CPU4: found redistributor 4 region 0:0x0000801080080000
CPU4: using LPI pending table @0x0000010014090000
CPU4: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU5
GICv3: CPU5: found redistributor 5 region 0:0x00008010800a0000
CPU5: using LPI pending table @0x00000100140a0000
CPU5: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU6
GICv3: CPU6: found redistributor 6 region 0:0x00008010800c0000
CPU6: using LPI pending table @0x00000100140b0000
CPU6: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU7
GICv3: CPU7: found redistributor 7 region 0:0x00008010800e0000
CPU7: using LPI pending table @0x00000100140c0000
CPU7: Booted secondary processor [431f0a10]
Brought up 8 CPUs
SMP: Total of 8 processors activated.
CPU features: detected feature: GIC system register CPU interface
CPU features: detected feature: LSE atomic instructions
CPU features: detected feature: Software prefetching using PRFM
CPU: All CPU(s) started at EL2
alternatives: patching kernel code
devtmpfs: initialized
SMBIOS 3.0.0 present.
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
atomic64_test: passed
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
cpuidle: using governor menu
vdso: 2 pages (1 code @ fffffc0008980000, 1 data @ fffffc0008eb0000)
hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
DMA: preallocated 256 KiB pool for atomic allocations
Serial: AMBA PL011 UART driver
87e024000000.serial: ttyAMA0 at MMIO 0x87e024000000 (irq = 7, base_baud = 0) is a PL011 rev3
console [ttyAMA0] enabled
87e025000000.serial: ttyAMA1 at MMIO 0x87e025000000 (irq = 8, base_baud = 0) is a PL011 rev3
OF: /soc@0/pci@848000000000: arguments longer than property
OF: /soc@0/pci@849000000000: arguments longer than property
OF: /soc@0/pci@84a000000000: arguments longer than property
OF: /soc@0/pci@84b000000000: arguments longer than property
OF: /soc@0/pci@87e0c0000000: arguments longer than property
OF: /soc@100000000000/pci@848000000000: arguments longer than property
OF: /soc@100000000000/pci@849000000000: arguments longer than property
OF: /soc@100000000000/pci@84a000000000: arguments longer than property
OF: /soc@100000000000/pci@84b000000000: arguments longer than property
HugeTLB registered 2 MB page size, pre-allocated 0 pages
HugeTLB registered 512 MB page size, pre-allocated 0 pages
ACPI: Interpreter disabled.
arm-smmu 830000000000.smmu0: probing hardware configuration...
arm-smmu 830000000000.smmu0: SMMUv2 with:
arm-smmu 830000000000.smmu0: 	stage 1 translation
arm-smmu 830000000000.smmu0: 	stage 2 translation
arm-smmu 830000000000.smmu0: 	nested translation
arm-smmu 830000000000.smmu0: 	non-coherent table walk
arm-smmu 830000000000.smmu0: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 830000000000.smmu0: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 830000000000.smmu0: 	128 context banks (0 stage-2 only)
arm-smmu 830000000000.smmu0: 	Supported page sizes: 0x62215000
arm-smmu 830000000000.smmu0: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 830000000000.smmu0: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 830000000000.smmu0: registered 1 master devices
arm-smmu 831000000000.smmu1: probing hardware configuration...
arm-smmu 831000000000.smmu1: SMMUv2 with:
arm-smmu 831000000000.smmu1: 	stage 1 translation
arm-smmu 831000000000.smmu1: 	stage 2 translation
arm-smmu 831000000000.smmu1: 	nested translation
arm-smmu 831000000000.smmu1: 	non-coherent table walk
arm-smmu 831000000000.smmu1: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 831000000000.smmu1: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 831000000000.smmu1: 	128 context banks (0 stage-2 only)
arm-smmu 831000000000.smmu1: 	Supported page sizes: 0x62215000
arm-smmu 831000000000.smmu1: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 831000000000.smmu1: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 831000000000.smmu1: registered 2 master devices
arm-smmu 832000000000.smmu2: probing hardware configuration...
arm-smmu 832000000000.smmu2: SMMUv2 with:
arm-smmu 832000000000.smmu2: 	stage 1 translation
arm-smmu 832000000000.smmu2: 	stage 2 translation
arm-smmu 832000000000.smmu2: 	nested translation
arm-smmu 832000000000.smmu2: 	non-coherent table walk
arm-smmu 832000000000.smmu2: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 832000000000.smmu2: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 832000000000.smmu2: 	128 context banks (0 stage-2 only)
arm-smmu 832000000000.smmu2: 	Supported page sizes: 0x62215000
arm-smmu 832000000000.smmu2: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 832000000000.smmu2: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 832000000000.smmu2: registered 1 master devices
arm-smmu 833000000000.smmu3: probing hardware configuration...
arm-smmu 833000000000.smmu3: SMMUv2 with:
arm-smmu 833000000000.smmu3: 	stage 1 translation
arm-smmu 833000000000.smmu3: 	stage 2 translation
arm-smmu 833000000000.smmu3: 	nested translation
arm-smmu 833000000000.smmu3: 	non-coherent table walk
arm-smmu 833000000000.smmu3: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 833000000000.smmu3: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 833000000000.smmu3: 	128 context banks (0 stage-2 only)
arm-smmu 833000000000.smmu3: 	Supported page sizes: 0x62215000
arm-smmu 833000000000.smmu3: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 833000000000.smmu3: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 833000000000.smmu3: registered 1 master devices
arm-smmu 930000000000.smmu4: probing hardware configuration...
arm-smmu 930000000000.smmu4: SMMUv2 with:
arm-smmu 930000000000.smmu4: 	stage 1 translation
arm-smmu 930000000000.smmu4: 	stage 2 translation
arm-smmu 930000000000.smmu4: 	nested translation
arm-smmu 930000000000.smmu4: 	non-coherent table walk
arm-smmu 930000000000.smmu4: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 930000000000.smmu4: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 930000000000.smmu4: 	128 context banks (0 stage-2 only)
arm-smmu 930000000000.smmu4: 	Supported page sizes: 0x62215000
arm-smmu 930000000000.smmu4: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 930000000000.smmu4: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 930000000000.smmu4: registered 1 master devices
arm-smmu 931000000000.smmu5: probing hardware configuration...
arm-smmu 931000000000.smmu5: SMMUv2 with:
arm-smmu 931000000000.smmu5: 	stage 1 translation
arm-smmu 931000000000.smmu5: 	stage 2 translation
arm-smmu 931000000000.smmu5: 	nested translation
arm-smmu 931000000000.smmu5: 	non-coherent table walk
arm-smmu 931000000000.smmu5: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 931000000000.smmu5: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 931000000000.smmu5: 	128 context banks (0 stage-2 only)
arm-smmu 931000000000.smmu5: 	Supported page sizes: 0x62215000
arm-smmu 931000000000.smmu5: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 931000000000.smmu5: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 931000000000.smmu5: registered 1 master devices
arm-smmu 932000000000.smmu6: probing hardware configuration...
arm-smmu 932000000000.smmu6: SMMUv2 with:
arm-smmu 932000000000.smmu6: 	stage 1 translation
arm-smmu 932000000000.smmu6: 	stage 2 translation
arm-smmu 932000000000.smmu6: 	nested translation
arm-smmu 932000000000.smmu6: 	non-coherent table walk
arm-smmu 932000000000.smmu6: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 932000000000.smmu6: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 932000000000.smmu6: 	128 context banks (0 stage-2 only)
arm-smmu 932000000000.smmu6: 	Supported page sizes: 0x62215000
arm-smmu 932000000000.smmu6: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 932000000000.smmu6: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 932000000000.smmu6: registered 1 master devices
arm-smmu 933000000000.smmu7: probing hardware configuration...
arm-smmu 933000000000.smmu7: SMMUv2 with:
arm-smmu 933000000000.smmu7: 	stage 1 translation
arm-smmu 933000000000.smmu7: 	stage 2 translation
arm-smmu 933000000000.smmu7: 	nested translation
arm-smmu 933000000000.smmu7: 	non-coherent table walk
arm-smmu 933000000000.smmu7: 	(IDR0.CTTW overridden by dma-coherent property)
arm-smmu 933000000000.smmu7: 	stream matching with 128 register groups, mask 0x7fff
arm-smmu 933000000000.smmu7: 	128 context banks (0 stage-2 only)
arm-smmu 933000000000.smmu7: 	Supported page sizes: 0x62215000
arm-smmu 933000000000.smmu7: 	Stage-1: 48-bit VA -> 48-bit IPA
arm-smmu 933000000000.smmu7: 	Stage-2: 48-bit IPA -> 48-bit PA
arm-smmu 933000000000.smmu7: registered 1 master devices
iommu: Adding device 848000000000.pci to group 0
iommu: Adding device 849000000000.pci to group 1
iommu: Adding device 84a000000000.pci to group 2
iommu: Adding device 84b000000000.pci to group 3
iommu: Adding device 88001f000000.pci to group 4
iommu: Adding device 948000000000.pci to group 5
iommu: Adding device 949000000000.pci to group 6
iommu: Adding device 94a000000000.pci to group 7
iommu: Adding device 94b000000000.pci to group 8
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
DMA-API: preallocated 4096 debug entries
DMA-API: debugging enabled by kernel config
clocksource: Switched to clocksource arch_sys_counter
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 6, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 6, 4194304 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
UDP hash table entries: 65536 (order: 7, 10485760 bytes)
UDP-Lite hash table entries: 65536 (order: 7, 10485760 bytes)
NET: Registered protocol family 1
Unpacking initramfs...
------------[ cut here ]------------
kernel BUG at mm/page_alloc.c:1844!
Internal error: Oops - BUG: 0 [#1] SMP
Modules linked in:
CPU: 4 PID: 1 Comm: swapper/0 Tainted: G        W       4.8.0-rc4-00267-g664e88c #3
Hardware name: www.cavium.com ThunderX CRB-2S/ThunderX CRB-2S, BIOS 0.3 Aug 24 2016
task: fffffe0fee55a200 task.stack: ffffff00004a4000
PC is at move_freepages+0x158/0x168
LR is at move_freepages_block+0xac/0xc0
pc : [<fffffc0008227618>] lr : [<fffffc00082276d4>] pstate: 800000c5
sp : ffffff00004a7480
x29: ffffff00004a7480 x28: fffffdffc3fc0020 
x27: fffffdffc3fc0000 x26: 000000000000000a 
x25: 000000000000000a x24: ffffff0ffffa3190 
x23: ffffff0fffdbf530 x22: 0000000000000000 
x21: fffffdffc3ffffc0 x20: ffffff0ffffa2c80 
x19: fffffdffc3f80000 x18: fffffe0fdc0c2548 
x17: 0000000000000000 x16: 0000000100000000 
x15: 00000000000000c1 x14: 04000000001b2000 
x13: 00aedf01c7040000 x12: 00001b00000033f5 
x11: 01c604000000001b x10: 0000ae7401c50430 
x9 : 0000000000000000 x8 : fffffc0008f53000 
x7 : fffffc00082295f0 x6 : 0000000000000001 
x5 : 0000000000000000 x4 : fffffe0fffff2580 
x3 : ffffff0ffffa2500 x2 : 0000000000000001 
x1 : fffffe0fffff2580 x0 : ffffff0ffffa2c80 

Process swapper/0 (pid: 1, stack limit = 0xffffff00004a4020)
Stack: (0xffffff00004a7480 to 0xffffff00004a8000)
7480: ffffff00004a74e0 fffffc00082276d4 0000000000000000 ffffff0ffffa2c80
74a0: fffffdffc3f80000 0000000000000000 ffffff0fffdbf530 ffffff0ffffa3190
74c0: 000000000000000a 000000000000000a fffffdffc3fc0000 fffffdffc3fc0020
74e0: ffffff00004a7510 fffffc0008227f00 ffffff0ffffa2c80 0000000000000000
7500: fffffdffc3fc0000 0000000000000028 ffffff00004a75b0 fffffc0008229654
7520: ffffff0fffdbf530 ffffff0ffffa2c80 fffffc0008e5f520 0000000000000000
7540: ffffff0fffdbf530 ffffff0fffdbf520 0000030ff6f60000 0000000000000001
7560: 0000000000000000 0000000000000000 ffffff0ffffa3200 fffffc00082295f0
7580: fffffc0008e5f520 fffffc0008ec1000 0000000000000000 0000000000000001
75a0: 0000000000000010 01ffff0ffffa2c80 ffffff00004a76a0 fffffc000822a5ec
75c0: 00000000024200c2 0000000000000000 0000000000000000 00000000024200c2
75e0: fffffc0008ec4000 0000000000000001 0000000000000000 fffffc0008ec2000
7600: 00000000024200c2 fffffc00082886c8 ffffffffffffffff 0000000000000040
7620: 00000012004a7640 fffffc0008e587e8 0000000100000000 0000000000000000
7640: ffffff00024200c2 fffffc00081007bc ffffff0ffffa2c80 0000000000000010
7660: fffffc0008e5f520 0000000000000040 0000000000000000 ffffff0ffffa3200
7680: 0000000000000001 ffffff0ffffa3b80 ffffff0ffffa3b80 ffffff00004a77b8
76a0: ffffff00004a77e0 fffffc00082886c8 0000000000000000 fffffc0008ec0c48
76c0: ffffff0ffffa3b80 0000000000000000 0000000000000000 0000000000000001
76e0: ffffff00004a4000 0000000000000000 fffffe0fee55a200 00000000000053d4
7700: fffffe0fee55a200 0000000000000000 0000000000000000 fffffc0008eec160
7720: 0000000000000000 0000000000000002 fffffe0fee55aa98 fffffc0008ec2000
7740: fffffe0fee55a200 fffffc0009c95000 0000000000000000 fffffc0008eec160
7760: 0000000000000000 fffffc0008e587e8 0000000000000000 0000000100400000
7780: 0000000000000000 0000000000000000 0000000000000000 0000000000000002
77a0: 0000000000000000 0000000000000000 fffffc000821d070 ffffff0ffffa3b80
77c0: 0000000000000000 ffffff0ffffa3b80 0000000100000000 0000000000000000
77e0: ffffff00004a7810 fffffc0008288d20 fffffc0008e587e8 fffffe0fec01a8f0
7800: 00000000024200c2 fffffc000821e62c ffffff00004a7870 fffffc000821e62c
7820: 0000000000000000 ffffff00004a4000 00000000024200c2 fffffe0fe0249410
7840: fffffc000821e788 00000000024200c2 fffffc0008f01d28 0000000002493ee0
7860: 0000000000002c2c 0000000000000040 ffffff00004a78d0 fffffc000821e788
7880: 0000000000000000 000000000000000e 00000000024200c2 fffffe0fe0249410
78a0: 0000000000000001 fffffc0008fd76ac fffffc0008f01d28 0000000002493ee0
78c0: 0000000000002c2c fffffc000821e850 ffffff00004a7930 fffffc000821e9b0
78e0: fffffe0fe0249410 0000000000000001 0000000000000001 0000000000002c2c
7900: ffffff00004a7a20 fffffe0fcc140400 00000000000053d4 fffffc00089b8900
7920: 0000000000002c2c fffffc000821e440 ffffff00004a7960 fffffc00082eb758
7940: fffffe0fe0249410 0000000000000001 0000000000010000 ffffff00004a7b18
7960: ffffff00004a79a0 fffffc000821e3bc 0000000000010000 ffffff00004a7b18
7980: fffffe0fe0249410 0000000000010000 0000000000000000 fffffc000821e434
79a0: ffffff00004a7a30 fffffc00082200e0 0000000000000000 fffffe0fcc140400
79c0: fffffe0fe0249210 ffffff00004a7af0 fffffe0fe0249410 ffffff00004a7b18
79e0: fffffc0008d40e70 0000000000008000 fffffc0009040078 ffffff0f83f26000
7a00: 0000000000000000 fffffc0008bd0300 00002c2c00000001 ffffff00004a4000
7a20: fffffdff83f20840 0000000027e02140 ffffff00004a7a80 fffffc00082201fc
7a40: 0000000000008000 ffffff00004a7af0 fffffe0fe02492e8 ffffff00004a7b18
7a60: ffffff00004a7bb8 0000000000000007 fffffc0008d40e70 0000000000008000
7a80: ffffff00004a7ab0 fffffc00082ba89c fffffe0fcc140400 0000000000008000
7aa0: ffffff00004a7bb8 fffffe0fdd350000 ffffff00004a7b40 fffffc00082bb588
7ac0: 0000000000000000 0000000000008000 fffffe0fcc140400 fffffe0fdd350000
7ae0: fffffe0fdd350000 0000000000008000 fffffe0fcc140400 000000000000ac2c
7b00: 0000000000000000 0000000000000000 0000000000000000 fffffc0000000003
7b20: 00000000000053d4 0000000000002c2c ffffff00004a7ae0 0000000000000001
7b40: ffffff00004a7b80 fffffc00082bc5b4 fffffe0fcc140400 fffffe0fcc140400
7b60: fffffe0fdd350000 0000000000008000 fffffc0008babe88 ffffff0f8ea0e439
7b80: ffffff00004a7bc0 fffffc0008d120f0 0000000000000000 0000000000008000
7ba0: fffffe0fdd350000 0000000000000000 fffffe0fdc3fa280 000000000000ac2c
7bc0: ffffff00004a7bf0 fffffc0008d121d4 fffffc0008d75000 fffffc0008d75b60
7be0: fffffc0008d75bb8 fffffe0fdd350000 ffffff00004a7c10 fffffc0008d11eac
7c00: fffffc0008d75b60 0000000000008000 ffffff00004a7c40 fffffc0008d11f10
7c20: 0000000000008000 0000000000008000 fffffc0008d75b60 fffffc00089e42c0
7c40: ffffff00004a7c80 fffffc0008d410f4 fffffe0fdc3fa280 ffffff0f83f26000
7c60: 0000000000000000 fffffe0fdd350000 fffffc0008d11d74 fffffc0008d11ec8
7c80: ffffff00004a7cf0 fffffc0008d411b4 fffffc0008d75000 ffffff0f83f26000
7ca0: 000000000e7f20bc 0000000000000000 fffffc0008d4119c fffffc0008ff1f38
7cc0: fffffc0008babf90 fffffc0009040000 fffffc0008e3d9a0 0000000000000000
7ce0: 0000000008d75000 0000000000008000 ffffff00004a7d00 fffffc0008d127c4
7d00: ffffff00004a7d60 fffffc0008d1294c fffffc0009040000 fffffc0008d128dc
7d20: fffffc0009040000 fffffc0009040000 fffffc0009040000 fffffc0008d1045c
7d40: fffffc0008cfc468 fffffc0008d74290 0000000000001b57 fffffc0008c04958
7d60: ffffff00004a7d90 fffffc0008082ff4 ffffff00004a4000 fffffc0008d128dc
7d80: 0000000000000000 0000000000000005 ffffff00004a7e00 fffffc0008d10df4
7da0: 0000000000000101 fffffc0009040000 fffffc0008d742a8 0000000000000005
7dc0: fffffc0008e3d700 0000000000000000 fffffc0008ee2370 fffffc0008bb79f8
7de0: 0000000000000000 0000000500000005 0000000000000000 fffffc0008d1045c
7e00: ffffff00004a7ea0 fffffc000893c888 fffffc0009040000 fffffc0009040000
7e20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff00004a4000
7e60: 0000000000000003 0000000000000000 0000000000000000 0000000000000000
7e80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7ea0: 0000000000000000 fffffc0008082b80 fffffc000893c868 0000000000000000
7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000
7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
Call trace:
Exception stack(0xffffff00004a72a0 to 0xffffff00004a73d0)
72a0: fffffdffc3f80000 0000040000000000 ffffff00004a7480 fffffc0008227618
72c0: 00000000800000c5 000000000000003d fffffc0008ec4000 0000000000000030
72e0: ffffff00004a7300 fffffc00081593b0 fffffc0008f53000 0000000000000000
7300: ffffff00004a7320 fffffc000822abac fffffc0008fd76be ffffff00004a7438
7320: ffffff00004a7460 fffffc00082886c8 ffffff00004a7380 fffffc0008135d08
7340: 0000000000000001 0000000000000007 ffffff0ffffa2c80 fffffe0fffff2580
7360: 0000000000000001 ffffff0ffffa2500 fffffe0fffff2580 0000000000000000
7380: 0000000000000001 fffffc00082295f0 fffffc0008f53000 0000000000000000
73a0: 0000ae7401c50430 01c604000000001b 00001b00000033f5 00aedf01c7040000
73c0: 04000000001b2000 00000000000000c1
[<fffffc0008227618>] move_freepages+0x158/0x168
[<fffffc00082276d4>] move_freepages_block+0xac/0xc0
[<fffffc0008227f00>] __rmqueue+0x740/0x898
[<fffffc0008229654>] get_page_from_freelist+0x3e4/0xca0
[<fffffc000822a5ec>] __alloc_pages_nodemask+0x1ac/0xfd8
[<fffffc00082886c8>] alloc_page_interleave+0x60/0xb8
[<fffffc0008288d20>] alloc_pages_current+0x168/0x1c8
[<fffffc000821e62c>] __page_cache_alloc+0x17c/0x1c0
[<fffffc000821e788>] pagecache_get_page+0x118/0x2f8
[<fffffc000821e9b0>] grab_cache_page_write_begin+0x48/0x68
[<fffffc00082eb758>] simple_write_begin+0x40/0x168
[<fffffc000821e3bc>] generic_perform_write+0xc4/0x1b8
[<fffffc00082200e0>] __generic_file_write_iter+0x178/0x1c8
[<fffffc00082201fc>] generic_file_write_iter+0xcc/0x1c8
[<fffffc00082ba89c>] __vfs_write+0xcc/0x140
[<fffffc00082bb588>] vfs_write+0xa8/0x1c0
[<fffffc00082bc5b4>] SyS_write+0x54/0xb0
[<fffffc0008d120f0>] xwrite+0x34/0x7c
[<fffffc0008d121d4>] do_copy+0x9c/0xf4
[<fffffc0008d11eac>] write_buffer+0x34/0x50
[<fffffc0008d11f10>] flush_buffer+0x48/0xb8
[<fffffc0008d410f4>] __gunzip+0x27c/0x324
[<fffffc0008d411b4>] gunzip+0x18/0x20
[<fffffc0008d127c4>] unpack_to_rootfs+0x168/0x280
[<fffffc0008d1294c>] populate_rootfs+0x70/0x138
[<fffffc0008082ff4>] do_one_initcall+0x44/0x138
[<fffffc0008d10df4>] kernel_init_freeable+0x240/0x2e0
[<fffffc000893c888>] kernel_init+0x20/0xf8
[<fffffc0008082b80>] ret_from_fork+0x10/0x50
Code: 913dc021 9400d2db d4210000 d503201f (d4210000) 
---[ end trace fcdd7f8cb96cdb3d ]---
note: swapper/0[1] exited with preempt_count 1
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

SMP: stopping secondary CPUs
---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S942711AbcJ0QBr (ORCPT <rfc822;w@1wt.eu>);
        Thu, 27 Oct 2016 12:01:47 -0400
Received: from foss.arm.com ([217.140.101.70]:40506 "EHLO foss.arm.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S937770AbcJ0QBj (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 27 Oct 2016 12:01:39 -0400
Date: Thu, 27 Oct 2016 17:01:36 +0100
From: Will Deacon <will.deacon@arm.com>
To: Robert Richter <rric@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>,
        Robert Richter <robert.richter@cavium.com>,
        Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161027160136.GD24290@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161017185801.GT25086@rric.localdomain>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Robert,

On Mon, Oct 17, 2016 at 08:58:01PM +0200, Robert Richter wrote:
> Mark, Will, any opinion here?

Having looking at this, I'm inclined to agree with you; pfn_valid() is
all about whether the underlying mem_map (struct page *) entry exists,
not about whether the page is mappable or not.

That said, setting the zone for pages representing NOMAP memory feels
like a slippery slope to losing information about them being NOMAP in
the first place and the whole problem getting out-of-hand. Whilst I'm
happy for pfn_valid() to return true (in the sense that we're within
bounds of mem_map etc), I'm less happy that we're also saying that the
struct page contains useful information, such as the zone and the node
information, which is then subsequently used by the NUMA code.

On top of that, pfn_valid is used in other places as a coarse "is this
memory?" check, and will cause things like ioremap to fail whereas it
wouldn't at the moment. It feels to me like NOMAP memory is a new type
of memory where there *is* a struct page, but it shouldn't be used for
anything. I don't think pfn_valid can describe that, given the way it's
currently used, and flipping the logic is just likely to move the problem
elsewhere.

What options do we have for fixing this in the NUMA code?

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1756227AbcJ1JfI (ORCPT <rfc822;w@1wt.eu>);
        Fri, 28 Oct 2016 05:35:08 -0400
Received: from mail-bl2nam02on0066.outbound.protection.outlook.com ([104.47.38.66]:28236
        "EHLO NAM02-BL2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S1755671AbcJ1JfG (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 28 Oct 2016 05:35:06 -0400
X-Greylist: delayed 939 seconds by postgrey-1.27 at vger.kernel.org; Fri, 28 Oct 2016 05:35:05 EDT
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Fri, 28 Oct 2016 11:19:05 +0200
From: Robert Richter <robert.richter@cavium.com>
To: Will Deacon <will.deacon@arm.com>
CC: Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
        Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161028091905.GM22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20161027160136.GD24290@arm.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.90.219]
X-ClientProxiedBy: AM3PR07CA0023.eurprd07.prod.outlook.com (10.141.45.151) To
 BY2PR07MB2343.namprd07.prod.outlook.com (10.166.114.145)
X-MS-Office365-Filtering-Correlation-Id: 83854c1a-c87d-4fe4-9599-08d3ff1381d1
X-Microsoft-Exchange-Diagnostics: 1;BY2PR07MB2343;2:AxPYpizkgpudLAV4Z4O4dykMzem5tiKEX28HpOwAqgh9mhMZuyKBZcGxpWN0n8mSk2/WUaPXvnPzuu6rN65qMa9H27npBLD4ceNSakfn0/zJin3rUsf9iariDjdQ1okCUoxUEDxbQryfE66hCSUbUbPNDsR4x6SoUqKOb3bAo2YEuq2zZJpneZfpKP0VKQY5Ou7EgjcbQJk/FHYNitYBQA==;3:a3oC8g+OqMFELuC+42v4jGzfl4mZuE9LufXjo/JLrCQYERI1ZjJB39j8OgaVyadSGZlw3JP39EnRU3bCfw4o3VVFm4xN4tXRh59vt3lV/VkrK47HCJuvDEcLQbk5uyZ/SsqnJFcWk5+R5DnbnC884g==;25:sddl1l/KCHTsZoppgQKeFiesr8HTqPfD7azZIhUaH3YOBDsfEYlTR9KVUJ3QUb3dSc436JKeiFVCnaqsfG5oB3Ko23qmAVcaBPpK5FVdXZLf5AAEF1Vbh5j2JWRjCr2gVTEQYDZRrG8R3PcNzpKKpeLrleUzUvqhCshlRG3lgPuamHUuNus8663hzWFj6FawbzcNP3KAf4VtJtpwExE5sLUPkWjjR2Prz2NHhfk0CLCWqmwcgMDH9vxXVerOSrIoWS9dEulmZOT9dj9bEIyx+moF7GzfylfsEVrk1QNfYNj5/RFvqJwa1/I8tXGU49glyZaLbfj9+wTuozOVld+5hKnclwGSxBnqgUo6O+6+n3HPycW3OLL3Ct3Mtz8WbxJk3FB8uSULWcizL5OpgpWZWjwHm9Huhdnu6Ch7+LwlgFKY8u2fA46bwq9c2wQuAZSJ
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR07MB2343;
X-Microsoft-Exchange-Diagnostics: 1;BY2PR07MB2343;31:Sy4mfsdvhCnbLrtCT52NwWocn4Clt8G1Kzf+WAiC/IZHaQLBuTA1OJb39S5LbDaH7YH8A6e90HC2cwsQ0glfwpxR5fSTOXdcXt/ozsNUHn4bfSPBkKt652r/sTC0fvhcD84I7uEg5w8k9rW8sqz5/aEEx1ZffTKb0tN1LOsgTs9kFb9v85bHUnZLryOXwWJ2shwranRAAhmqV70SSV5MnRb+IqQYpwkFBOWofnOhYeGD+y45+k1lX0psSEup1Zioc6Uxr3gOW+fiWwJBbzQKDA==;20: 
X-Microsoft-Antispam-PRVS: <BY2PR07MB23437DDB263B03FFDBD83E7BF1AD0@BY2PR07MB2343.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(83566789882024);
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:BY2PR07MB2343;BCL:0;PCL:0;RULEID:;SRVR:BY2PR07MB2343;
X-Microsoft-Exchange-Diagnostics: 1;BY2PR07MB2343;4:c9uWwCXv1kskMe1NShrJ6r+Kk8D9P1oXydrddvPE+zATSiCbRvEkJPoMegLLnwz4auJBwPM3hxDPU5BkpFtjCj/ylYI9r7t/ebqaXsv54CqRupXA0tPIiH4FRG9MIwG4tU/gPo4nyEs/D71+emKBfasNBkD/qUf/RhAnXCrLCOg4OZxpkUwfxocJV9A0Ja/aSK9K9omEDcxxhiTWIWRIS2SplancbDKuiqq+dCHTpspwOyCem4OOvdYUGHM0sKBMS/79Wsaw0L8Jt8xRwtSMBOwSN+zPR/gz7a4hgTNFrgxKixFS3vCzL8mXN7xOJVwTGMYL7azCmcr1vSM7tHndj26y9oFmDZp2EewXd0IL89fJn6RsKHWQlUJP9VNMr4JE0KWmm0PCyH7M5phsSjp6mKhmz94BHqeEY5I12tCKPMLatiLEbxXl6vdnzgSyg8wzJSXJTYYZxmX+BP4gZr6r2A==
X-Forefront-PRVS: 0109D382B0
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(24454002)(199003)(101416001)(19580395003)(68736007)(76176999)(66066001)(2950100002)(47776003)(6916009)(6116002)(6666003)(42186005)(3846002)(106356001)(93886004)(46406003)(92566002)(50986999)(54356999)(23726003)(1076002)(586003)(7736002)(105586002)(4326007)(83506001)(15975445007)(77096005)(5660300001)(110136003)(7846002)(8676002)(81156014)(189998001)(9686002)(97756001)(86362001)(4001350100001)(81166006)(305945005)(50466002)(2906002)(33656002)(97736004)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:BY2PR07MB2343;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BY2PR07MB2343;23:jEHBUuAQMnqhnhzAIHAeOqtXoeCFot6Cat4StUUw6?=
 =?us-ascii?Q?yVofmn8ZnPaX09LEBVGYBC5xjfqZu7u+Xo7e7NmXGjSxinOVn4NXLIg0EP3D?=
 =?us-ascii?Q?Nyswmeoxf5nAyS2T+JxluAi+lsco1H0SYldzpRP7JbKYA1DcukerKu6Mwa0q?=
 =?us-ascii?Q?I6u0gWnzwOfxQXPB6V8dueYk+qhM9KBdk8IeiRyPPOzEEHgPA/c+gMn5XFeD?=
 =?us-ascii?Q?a93lnmHKOSsthzxNxgvCcCM3AFx7Hm/ko2wMDI1eQzTQmO3bebES8hfm9fy4?=
 =?us-ascii?Q?HK6BM/7bEOSAw0vyiauhRP0xeoeDRJmk1wI+nLVx1lo6FTcLSy5ctXsE9PNe?=
 =?us-ascii?Q?9/RETwG97N+pao55bJ1jhnSCdcrWHQfgn7T5+6/VoLV04ACDz8BB1WLOTcAS?=
 =?us-ascii?Q?SbDRtW6QD8YnpeoYseS6X8pZOJb7TCO+pxwzcyMMIJBL85m5N+E6SVB6c8OM?=
 =?us-ascii?Q?vXFk4IIEGncdLOf/lKciA1g2fwEyulKejyZgRUqPHvyYG72x0S6nM9hDb6k/?=
 =?us-ascii?Q?lTcc8TtWlNo/2f32YDglRNus9xsLQYYAon0liUVg1boIOe1F987kzSKHo1v+?=
 =?us-ascii?Q?k+pebZAoZZDmeJAlLOoeAfNs2O6Vb8BCnKdmlCrVb2dz3z2xAvJyHPgEf7N8?=
 =?us-ascii?Q?U3nzVR63YpH0/mEHpbtl9m4qMwwgcyd6qztrOLGXWSQ64WiUnWCihhtkIuyi?=
 =?us-ascii?Q?OduJrr9397Ekgpx66P0hM37jsTf778EC3T+iIXs3W8fJtxCF5MTpXa17wJII?=
 =?us-ascii?Q?5trzZ51svjF1GA+IwuKVlyqJ7Q279oIo0fN46BFPPo05UZiYThkOpUSvvNbD?=
 =?us-ascii?Q?u9AwZFQTVgHeX4JRkBWQyJscQu2srKYGvuZzLmcZP+2PfTQQ8NM0YJSJ/Dr+?=
 =?us-ascii?Q?3PypXDkLItPyHOWsbm23wx8LpfGoR/kMMZaUNlLYkEIpv+WMyiR+7BisjJm+?=
 =?us-ascii?Q?wTyp85/2r6QfFT0DA4xWv9n6llZbdh1g6Ujjiv6MuqMtJl5QlTYJHZ3ZOEX7?=
 =?us-ascii?Q?U+RkFfqnHwgMw3jHRtFdD8XmdKkGhINGvygZRHXFZKWc+D+S4jJLSZnxlEMv?=
 =?us-ascii?Q?Cj+DzZTx0FT1Nn7GmUS6J9MvwfmtAy5LlcZ0XTP6VYiCNY4NmyL9u07M4R2F?=
 =?us-ascii?Q?ErR0xHiqytWqfURObyK5gkQjcohaDSMjUYNvlnuGNs/ucqZdz/QK1GRdlf8e?=
 =?us-ascii?Q?s4FD28Z5G64j7fEMYPz4wCwb1wg13uBHUs3v0/pIrbZ/bP/KmgBvL32HwaEn?=
 =?us-ascii?Q?WLKcr675sjwqLGNWXY=3D?=
X-Microsoft-Exchange-Diagnostics: 1;BY2PR07MB2343;6:h/HrrTVHIb4eCOFH8fLxwOkPLcx4YIlnvd1lDEGpJYLYGU3p2Rva+ebeQHUwt7rrJHmf+O2F5x5J6QoL0IZ3La1RJ4Vy/xRJGI6S9ar4/Wc63Opty4XJT7XF6e4llhbMflwVhiJo5uOwdbcqL7wQAR0nwO4E+l4q+c8df6Im+WyCYNfqKYzD8EP63zjXVyRI4NvoVdP0losN5UdXeheB2JnfUWuggvARTDmax6Xh+iBCPrc4zJpztmbRCaTb9FddBli3xBq6wMNAjqMZ461SbPvy36DVCk3htHwkLFxuGduCbfTVQq9gP/Lny2L74gA+;5:zZulbnemsdzDvtIcxT/wwG53enO1FF3wMJMDzqZJSU/2TyVxK1VtR6zbIoFjp7ybQnOKOmlFFZcWmEzsEVozGqnaI8iWEVatrdyFfju1dGNJ0EX8ith7r4zH72oICQNFkrjvfYRfcz8xpsPvKvCKUa/nVdOI5Hy9xH1zS97BPU0=;24:sd0T5ltOAAWYit4HbbDdcYJ+2o8nvdKXLpZGTWfzPP9tO3aD+11Yf3vsRId+FUs79DKXtOYOtdJIDwZKFHpAnZDM6Ryoz4VGrsIhunBse/U=;7:uC7MFjFHcSB9Z7iy/0cWfaxlO2zWxzWGvTKLCqZK1GTpTRAEHiUi9fadX3NLKWCua6o5uppWLTspitllucCnNFBIQcclBdBtb2Bueta+4+rxyZyBCu03tihrByxl/EI5mkIE5b/O29z7he2iKDwEuhZYyLRQyurBfEXDIj88LpJU6xJ2A7LLcLfQs2385fhDZIvZtwwJlp3uBbS6eybtiIO9Gtcc1FCtQiBGspVJ6dJfg9LnCzXQI4GlLKoOhRtG59nwCFFRrHbGEe01UewEkQ0kspiCEqjbgz4oxMULqzyVvIUvP8oHuQzlA2uxQNFvwP39HLHXF/JAVKZh8A4eJ7gC0PyR1X98zXIvsz4kv5A=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Oct 2016 09:19:21.4642 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR07MB2343
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 27.10.16 17:01:36, Will Deacon wrote:
> Hi Robert,
> 
> On Mon, Oct 17, 2016 at 08:58:01PM +0200, Robert Richter wrote:
> > Mark, Will, any opinion here?
> 
> Having looking at this, I'm inclined to agree with you; pfn_valid() is
> all about whether the underlying mem_map (struct page *) entry exists,
> not about whether the page is mappable or not.
> 
> That said, setting the zone for pages representing NOMAP memory feels
> like a slippery slope to losing information about them being NOMAP in
> the first place and the whole problem getting out-of-hand. Whilst I'm
> happy for pfn_valid() to return true (in the sense that we're within
> bounds of mem_map etc), I'm less happy that we're also saying that the
> struct page contains useful information, such as the zone and the node
> information, which is then subsequently used by the NUMA code.

Let's see it in a different way, pfns and the struct page assigned to
each of it is about *physical* memory. The system knows all the
memory, some is free, some reserved and some marked NOMAP. Regardless
of the mapping of the page the mm code maintains and uses that
information.

There are assumptions on validity and checks in the code that now
cause problems due to partly or non-existing data about nomap pages.
This inconsistency is dangerous since a problem may occur any time
then the page area is accessed first, thus a system may crash randomly
depending on the memory access. Luckily, in my case it triggered
reproducible while loading initrd during boot.

I also think that this is not only NUMA related. E.g. the following
bug report is probably also related:

 https://bugzilla.redhat.com/show_bug.cgi?id=1387793

> On top of that, pfn_valid is used in other places as a coarse "is this
> memory?" check, and will cause things like ioremap to fail whereas it
> wouldn't at the moment.

IMO this is a misuse of pfn_valid() that needs to be fixed with
additional checks, e.g. traversing memblocks.

> It feels to me like NOMAP memory is a new type
> of memory where there *is* a struct page, but it shouldn't be used for
> anything.

IMO, a NOMAP page should just be handled like a reserved page except
that the page is marked reserved. See free_low_memory_core_early().
Thus, NOMAP pages are not in the free pages list or set to reserved.
It is simply not available for mapping at all. Isn't that exactly what
it should be?

I also did not yet understand the benefit of the differentiation
between NOMAP and reserved and the original motivation for its
implementation. I looked through the mail threads but could not find
any hint. The only difference I see now is that it is not listed as a
reserved page, but as long as it is not freed it should behave the
same. I remember the case to handle memory different (coherency,
etc.), but are not sure here. Ard, could you explain this?

> I don't think pfn_valid can describe that, given the way it's
> currently used, and flipping the logic is just likely to move the problem
> elsewhere.
> 
> What options do we have for fixing this in the NUMA code?

Out of my mind:

1) Treat NOMAP pages same as reserved pages (my patch).

2) Change mm code to allow arch specific early_pfn_valid().

3) Fix mm code to only access stuct page (of a zone) if pfn_valid() is
   true.

There can be more alternatives. IMO:

 * We shouldn't touch generic mm code.

 * We should maintain a valid struct page for all pages in a sections.

 * We should only traverse memblock where really necessary (arm64
   only).

 * I don't think this problem is numa specific.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1752922AbcKAQ4F (ORCPT <rfc822;w@1wt.eu>);
        Tue, 1 Nov 2016 12:56:05 -0400
Received: from mail-by2nam01on0082.outbound.protection.outlook.com ([104.47.34.82]:50304
        "EHLO NAM01-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S1750758AbcKAQ4D (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 1 Nov 2016 12:56:03 -0400
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Tue, 1 Nov 2016 17:55:44 +0100
From: Robert Richter <robert.richter@cavium.com>
To: Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will.deacon@arm.com>
CC: Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        David Daney <david.daney@cavium.com>,
        Mark Rutland <mark.rutland@arm.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        <linux-arm-kernel@lists.infradead.org>, <linux-efi@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161101165544.GP22012@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="veXX9dWIonWZEC6h"
Content-Disposition: inline
In-Reply-To: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.80.109]
X-ClientProxiedBy: HE1PR01CA0074.eurprd01.prod.exchangelabs.com
 (10.165.170.170) To SN1PR07MB2351.namprd07.prod.outlook.com (10.169.127.17)
X-MS-Office365-Filtering-Correlation-Id: 69680f12-642d-42a4-2299-08d40277f555
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;2:Mpy/zm4nM58njm9VLdJXJirLuo0SvDHNJ32E/mPL0FVy3nv256r7it6OZL8FfKMdU/9u7jxGnlIfP0IGzlY8Ze5EHwjfG5l+rUy27dAxgwxC3SYZoLbEJ/wlstk7sAm+Ad+3FPszdnjzjnybfiJIZUaENYD+0VX1TtYyQ6hgWjiKrB5yag11zyP9JaN7uMHl80H3cXJi1sD9TcomCCytgg==;3:ptlqlPtTgVbThd153bXDVd/X8WD7Y4jxZlCFANGofvcJvY8bcWI+ApeKRMTOdW7UOcbDN0MS4dbmGByyO82tXBEVHlK/bMFAk3IEZbMAY6A2JBsphvZooeYBKM2t4BhxwgG+rJeN5ky50XoMIW3D+A==
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2351;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;25: 
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;31:tozdjPRLBOGpxU5b5CkptXwADwxxAHd5nhsjikXA16LqEOuKeW+A1o5WWQfOChYQtVsiI8wwJ6NoFPLF+cYcAVNVXC9yoif2DoMVP3RY4lA4+oJuLeUvAG6mykemrRvDx0tHVO1uq64EnU4IT0gGXdQKyk6VpTZgUvybdPFazv7O3TSZ4rPK8VBA2tP3g3OxPo/V5R5D1ga5rThd40jAmw0qiVfkeThxRZk0PDjI8beDfIHiHwkZyGYEIpDXWD2dRfh0x3eGB6+PJjr/QyyDNUf605T8u8P2IYU5nnC7jU0=;20: 
X-Microsoft-Antispam-PRVS: <SN1PR07MB23510DA7D8F2B772A8EE6A33F1A10@SN1PR07MB2351.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(20558992708506);
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(102415321)(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:SN1PR07MB2351;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2351;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;4:O6LNJvQrGLnUeo0wbwKSFl74VzXe8u6hHeYtTqa1/ji0KRJKK1DC00TEbVLixrqh0eeWsBKjMpYfsTKJapVkAeersO9bvVpALxMhhXLkbz9OWxewbNAOJUsTE6p07S8L2/3225Uogif9pvAKji54nu3hC77dK8bB7z3wTAGIwnPssn1QbaBQ2w6odMln1jZbQKPdmkf5064+GF/vd2p9/pdaplR0AmkBHL2ZVKWkyVqJqi4uu/LUxqqUky5JpQuiNoKGftgZjO7PxYqkVS10fG0K60aJHMPTt8YCPdbNTU1bU2PER6GbNKVDIpevxOOQr9A6awn1gseFJHU26C23sYrrwP8xpBe/qt6+37ujfhuuStjUyf47TOFKn3pLTWxvH/W75qwWCUoawSPsGWSasMjbOsxYGIpLeafnV6AMHDlV4oMIns31nRDThVSCOKvJbTG7pZKfwL0RqV5DGHas4A==
X-Forefront-PRVS: 01136D2D90
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(24454002)(189002)(199003)(66066001)(84326002)(42186005)(86362001)(5001770100001)(54356999)(76176999)(97736004)(4326007)(189998001)(568964002)(4001350100001)(260700001)(50986999)(105586002)(106356001)(305945005)(68736007)(101416001)(19580395003)(19580405001)(7736002)(7846002)(5660300001)(77096005)(512954002)(81156014)(8676002)(81166006)(6666003)(33656002)(2950100002)(2476003)(92566002)(6116002)(1076002)(3846002)(586003)(5890100001)(2906002)(4610100001)(9686002)(83506001)(18370500001)(2700100001)(47845001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR07MB2351;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;SN1PR07MB2351;23:teYPHIPYnG9EBUhtzYuXAZgpNu4gbcIWz5mJPDpFa?=
 =?us-ascii?Q?kV+2m84WVsP7FFJlFILPgFkvU1kiYTMOZ5kvjCrg3qrUtxA8afD9bOLe/QW8?=
 =?us-ascii?Q?4Tjt6lkPnd9Yc4RNGtaoHJFRjyTYBp98AZIKopPnwkmdGJYw1W9ww4u+rI4L?=
 =?us-ascii?Q?Xg02J9r60vhskSVDRUzShnUW/nDIPeOkQ6IKCw7FBUJwbCM0sZWZoyhqzWcF?=
 =?us-ascii?Q?mMywTOKUsgbq8E8wH9DpKEYvZhuAP8YRa/gOMLpPKKlK2c3x6aWYHrVC/yup?=
 =?us-ascii?Q?23V7SP1G7p1m46nDQoMy3vZPUvsTX18T5Nijt0ZQgWQqRMM7m2sc5R2HPeh0?=
 =?us-ascii?Q?s06G92KnPcRYlRzTdnHaZAwHmY856qj5V0pnfozkg3+7//T8hBNJu8Kv6qeG?=
 =?us-ascii?Q?sttbGuzFiy8hMx4lPUqLcezDicnYSu7eHYNVtHDzXu4EHUwSMkWZosB2VKCz?=
 =?us-ascii?Q?9UbtBHegFBk9cni7f373wifVF1Rs2vSGzO5kcGiZAqldRGQYIuIDtuIcfqD3?=
 =?us-ascii?Q?f+eGzM/XDfherEP+fUjeaohoBQiSLwTqe9zmPbdi5lVhoreY9d7drFWpSJCy?=
 =?us-ascii?Q?Rk5DlHRtv3R0wcDuDOFK/AoBbwxxR0xSQkiDclSGrW07XRL5e2fRxRmdflHn?=
 =?us-ascii?Q?vF+gIMJyPm+Nrmpyr9bgnetsP9tmxoYv1uwx2HdE9WFuiBoi1d1YqSmgG3Bv?=
 =?us-ascii?Q?kmuIfNgar1wW2XcZYrMnFuWdrPkn9UC+c1cPXU4GUpDeBYSrjx1bW6xAUhnI?=
 =?us-ascii?Q?Ymruu5zsD7ioAkdwl6l5U54lhpytLun/utDjdgKDX/5RMrcybqfiSRM+gTKR?=
 =?us-ascii?Q?rodJ1Y/LzqgsRpAqMdfg30bMC6R4Ttuc0K/yJDtPeerFJu4jQoYeUDfJc1CU?=
 =?us-ascii?Q?k6GzZ9Bn7uxXFtFr2RUqycNM5Iny/jf0nk5rCY4OWO6nXaTbPauV8nqU1DJO?=
 =?us-ascii?Q?4LTLKggWbNGRMsTT9S0n+c84waRiLIjQezuRE7YimSXQEuSwzd6h2TqeAhwU?=
 =?us-ascii?Q?mbxvDymc1vZLdHvsMB/tBi1DD6KYPalNcxa+uxQlW/qjZMUY4GNps1lIADme?=
 =?us-ascii?Q?IKXmgnfziKKmeRZ7Iy18cq5jD8l+4Zhk0RfE14hIi+nV6Oy5Jj51Tc1VKI2s?=
 =?us-ascii?Q?PJTs7PjYfKQH1jmGHVRoL2yOKrIUAt0lmZ1UWrN5OUNZ2zN4jtsTKtT/b2Wn?=
 =?us-ascii?Q?uAkGTHGy+Ul495reDjDtaGvQyg/Rchbv8QQcuDjXgWWyWghC0qg6YvXgQbqe?=
 =?us-ascii?Q?0Qs8QcEe7EJNnv0DSEYNQEvFrkWHcXfELzGv8XkBLNMP5xsEUnLowIbnsLVC?=
 =?us-ascii?Q?oZwS7/DE+BTGzu+iqFyw0o=3D?=
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;6:RC9jFvinHdX9QMo2XyeT7yhBo0fdwbmR0yJ54w31a7ZFuUT9H6Yqdcg57DP68I8k0qQIA7stSbMLnGNDctTNgYnm+GMUXikdiPpiessi34GKj1OvtiGgVAeQFEmI0SIJk77LjiYWoRBGUoJar/QDEBHlNrmilZXv9LtwsatHVKa1ZeH2XJc9vbODGnlNt20QiRxcF/C05QxwhbU95OT2vkUCzrtX84T6L0NLFYOrQV3sL1AFZvzzvt2+jMQjajo1vQjouigCEfzBW0VlVKW9wGe6LDTstqfcfKJN7WGUkkWcMwHQpwylQXuMoTMhbk9W;5:IJUuN1++4RE99oomEJubH8RHjF0nGScTUUSJf5SBNbfld1bbx6n1JO9wicArCZaNMQRGD11QLmxs1zw2XdS9e53V/jHi2EA5aC1Vg1XdL6Z8sPDkT5M4qtd5AD6iN3OqlhDFWJgraHo5PcxPoAVvAA==;24:FGhJ3nrXFhSGgTdYiP9lmn8Zrx+535EIRCzkag+n9JnRP6GAd9XMdsYd41Xou60nbKzToYXRoPSqIzsHIJS80rXkdrChhfm7F2+HSPZWBxY=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2351;7:e2SXxoXIZbv8TIB2NhkQ481gWlWoPoFQVMS7X8bWIzaVIqsPuSqyhu6Dahw2yPnAjEPJW/4ZOr28i6xkhb4If4Sg8urvrPTPFLUhRR+zGa+4Fx5/G0XsEys3bDhf6Jhy4LR3XjeOr3ui6ZR8TrFXx+15X2mfb0b/JxNZ6GqNLXqmrnghmfXqofowj8f/txsf/hltU/OEcVw2vlWPNdQt6LJuDlpKhqEK8sAdbiaxdTx/LKe7F4dAHJ7uIPxwh0TyB9GAKdz2WCbFAnTpM0c6iq+PWvk+t1O07ew+79609qUoBBLOTte2vxNEefxicplOviN2Q/Wp/EhD8Jy0GYNQxbrnnm0xQEG4T8Lqtc5njns=
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Nov 2016 16:55:58.2964 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR07MB2351
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

--veXX9dWIonWZEC6h
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline

On 06.10.16 11:52:07, Robert Richter wrote:
> There is a memory setup problem on ThunderX systems with certain
> memory configurations. The symptom is
> 
>  kernel BUG at mm/page_alloc.c:1848!
> 
> This happens for some configs with 64k page size enabled. The bug
> triggers for page zones with some pages in the zone not assigned to
> this particular zone. In my case some pages that are marked as nomap
> were not reassigned to the new zone of node 1, so those are still
> assigned to node 0.
> 
> The reason for the mis-configuration is a change in pfn_valid() which
> reports pages marked nomap as invalid:
> 
>  68709f45385a arm64: only consider memblocks with NOMAP cleared for linear mapping
> 
> This causes pages marked as nomap being no long reassigned to the new
> zone in memmap_init_zone() by calling __init_single_pfn().
> 
> Fixing this by restoring the old behavior of pfn_valid() to use
> memblock_is_memory(). Also changing users of pfn_valid() in arm64 code
> to use memblock_is_map_memory() where necessary. This only affects
> code in ioremap.c. The code in mmu.c still can use the new version of
> pfn_valid().

Below a reproducer for non-numa systems. Note that invalidating the
node id just simulates a different node in reality.

The patch injects a (pageblock_order) unaligned NOMAP mem range at the
end of a memory block and then tries to free that area. This causes a
BUG_ON() (log attached).

-Robert



>>From 20d853e300c99be5420c7ee3f072c318804cac1b Mon Sep 17 00:00:00 2001
From: root <root@10.18.240.201>
Date: Tue, 1 Nov 2016 15:04:43 +0000
Subject: [PATCH] mm-fault-reproducer

Signed-off-by: root <root@10.18.240.201>
---
 arch/arm64/mm/init.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/page_alloc.c      |  4 ++-
 2 files changed, 81 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 21c489bdeb4e..feaa7ab97551 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -36,6 +36,7 @@
 #include <linux/efi.h>
 #include <linux/swiotlb.h>
 #include <linux/vmalloc.h>
+#include <linux/page-isolation.h>
 
 #include <asm/boot.h>
 #include <asm/fixmap.h>
@@ -301,6 +302,80 @@ void __init arm64_memblock_init(void)
 	memblock_allow_resize();
 }
 
+static struct page *inject_pageblock;
+
+static void __init inject_nomap_create(void)
+{
+	phys_addr_t start, end;
+	unsigned long start_pfn, end_pfn;
+	u64 i;
+	int ret = -ENOMEM;
+
+	pr_info("%s: PAGES_PER_SECTION=%08lx pageblock_nr_pages=%08lx PAGE_SIZE=%08lx\n",
+		__func__, PAGES_PER_SECTION, pageblock_nr_pages, PAGE_SIZE);
+
+	/*
+	 * find a mem range with a complet pageblock in it
+	 */
+	for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end, NULL) {
+		start_pfn = PFN_DOWN(start);
+		end_pfn = PFN_UP(end);
+		if  (end_pfn - (start_pfn & ~(pageblock_nr_pages-1)) > 2 * pageblock_nr_pages)
+			break;
+	}
+
+	if (i == ULLONG_MAX)
+		goto fail;
+
+	start = PFN_PHYS(start_pfn);
+	end = PFN_PHYS(end_pfn) - 1;
+
+	pr_info("%s: Injecting into range: [%pa-%pa]\n", __func__, &start, &end);
+
+	/* mark the upper 5 pages nomap of a complete pageblock */
+	start_pfn = end_pfn & ~(pageblock_nr_pages-1);
+	start_pfn -= 5;			/* unalign by 5 pages */
+
+	start = PFN_PHYS(start_pfn);
+	end = PFN_PHYS(end_pfn) - 1;
+
+	ret = memblock_mark_nomap(start, end - start + 1);
+	if (ret)
+		goto fail;
+
+	inject_pageblock = pfn_to_page(start_pfn & ~(pageblock_nr_pages-1));
+
+	pr_info("%s: Injected nomap range at: [%pa-%pa] zones: %p %p\n", __func__,
+		&start, &end, page_zone(inject_pageblock),
+		page_zone(inject_pageblock + pageblock_nr_pages - 1));
+
+	return;
+fail:
+	pr_err("%s: Could not inject_unaligned_range: %d\n", __func__, ret);
+}
+
+static void __init inject_nomap_move(void)
+{
+	phys_addr_t start, end;
+	int ret;
+
+	if (!inject_pageblock)
+		return;
+
+	start = PFN_PHYS(page_to_pfn(inject_pageblock));
+	end = PFN_PHYS(page_to_pfn(inject_pageblock) + pageblock_nr_pages) - 1;
+
+	pr_info("%s: Moving [%pa-%pa] zones: %p %p\n", __func__,
+		&start, &end, page_zone(inject_pageblock),
+		page_zone(inject_pageblock + pageblock_nr_pages - 1));
+
+	ret = move_freepages_block(page_zone(inject_pageblock),
+				inject_pageblock,
+				gfpflags_to_migratetype(GFP_KERNEL));
+
+	pr_info("%s: Moved %d pages\n", __func__, ret);
+}
+
 void __init bootmem_init(void)
 {
 	unsigned long min, max;
@@ -320,6 +395,7 @@ void __init bootmem_init(void)
 	arm64_memory_present();
 
 	sparse_init();
+	inject_nomap_create();
 	zone_sizes_init(min, max);
 
 	high_memory = __va((max << PAGE_SHIFT) - 1) + 1;
@@ -479,6 +555,8 @@ void __init mem_init(void)
 		 */
 		sysctl_overcommit_memory = OVERCOMMIT_ALWAYS;
 	}
+
+	inject_nomap_move();
 }
 
 void free_initmem(void)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2b3bf6767d54..19d74637e242 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5077,8 +5077,10 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
 		if (context != MEMMAP_EARLY)
 			goto not_early;
 
-		if (!early_pfn_valid(pfn))
+		if (!early_pfn_valid(pfn)) {
+			set_page_node(pfn_to_page(pfn), NUMA_NO_NODE);
 			continue;
+		}
 		if (!early_pfn_in_nid(pfn, nid))
 			continue;
 		if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
-- 
2.9.3


--veXX9dWIonWZEC6h
Content-Type: application/x-xz
Content-Disposition: attachment;
	filename="typescript-crb2s-test21-201611010941-trigger-mm-fault.xz"
Content-Transfer-Encoding: base64

/Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj6S5WKHRdACmYykarec1wWuOyuHWSQFx0n4ZrJjMw
RcfxlPmc5+LvpK5TnaSBKhmU4vVf1qgflmSthKe/WQi4nGAyl+3Rl274lFz7DCRnf7YXMnCX
K5kcyTcuk18ZTBIMu/SHQbbHyN7s5cJwO/+FIWOGgtAFJ6ZU5yulK/g7rPbWw9FW9fUzaDf8
PruYdTNHcfzD8yoeOjKjUX/b59MC0DT/vXIozsUbronvNogSvrEdaRaZEOrIcW9p8s2exz78
SZmCOhnjgrP7oHuIV0Y8P+ZaVymTDnTgMe55kJ+6hpb+4yBYdYbSHwjgoTzTzq8TZAknphPf
JlfhGfDGeIh3k7RBhWRS0yC4KSC21L4Q4CL1djTpns/hg0Ibbiy3tGcK1cPXa7Ol/jwXDdey
L0q/RhK4YUvAWhV6iHpe7oHCiHuAAYmZ/FWj+N1VlFFI/aLO8S+MK/kW2CIyLxDHVkkxCdfW
qFQsnBDIjl+ALctRthC6K/CT6M9vP3RYNwPSVPI7dgOdZt9D2qfRc+kSTkb8iLbh63Z8pVFX
WJos3ud6WL/A6gMIkcyyLxBMwb9T8FKNXoH2sng/rl//p/VIL9bipGiyNtyXYFGq9gJ3Qrk0
FvkXsKgP65Y7qwqX4B8kCg9Y06fKRCzr2dFv7nfqaLt2joeqXS6tlxc8uIfzeqmsx5KSx1Yr
HUShvEnsEjiMmbD60I9MFV/GlDf3KRHbT4RAi40HGYyvBtUehhufKycdA/iLR1FnVj1+MdJW
jBalEX2EJY4HW39Htg2u8oaMSWY/ZmpTyM88eYv5LYqIDc15tsisjojgPYyIP/R2ch95eZmk
2uaPxHDi0D57jVZdYPGzKq65TpnBsBlJU0Hft5x3DTW8Ec211UokWexSIXIznsCNLIszJE7F
ua8ZdUrbEP+TDHsnJlLjftu3SYSz08YU3D3ZRaAH1wHB1fEtCY1wsMD3WOJaZDi7Jed3mR16
7qNTGuPOqL7Z3IHNCQ4ROXP3++ovzNHqafISuzhtFkbTQZirymmHKEi8n8hAMlUEhhGpPp0r
gI+w49PYMQlRLKLQCzOb5+RpE+H1xlBmcMyTkW9X4GTrp4RY95XR4PcaJSJ66ROfEXXzLcUN
asw8TAXsVK9APEl03sguP3TX7deIexlmnJIVJft/cYoGXctrjJcoPkHszV7fTTpMP8Of+tHD
oM/JSJbLOBGPCuWXNeo5ckNkMqtfR5XLqJyMglZGHBZy+B2ARoPdexLqfF62uN2oeLqBjdKI
FzxQ6VD9M0D/U0WvtFfE0zr1KjnI0qTO105NsjzgLon+j6erDOtlrOs+t33EBWePlJEXT9GD
mOplR5xZr6qrg1vlBOuKT2RB0/kHXnujonBcLqwX3oG6BemY0CPV9ZFa8biqWo0Kq8zA9b42
UqioFzb3ktT7FkDUt2vXQ66oJ0YoIAR06MfEtfvgajS20aSP5SUvk/6WiNJhTLEPhXOyyKjv
PwdQAYTeExX27Fe8urP2zOhs0lbgVSdgKDwrstbQzDNrxrOq6ayQgL2saAO48VmoLu1jCPOx
fW2frn/NGZIeOxuwvoHviyxwOVC1G6yRz/OP1fvRbybNHlA45nw6iRqpyrYZ/NPWumjtrrPM
h9nqUpcJrvS5mumNzklMI5g7ab8BQ8rB8Bjr28VWVgLro5MTxDygp6BeDQlnJO14jj+02kY7
gwcvZSBvi0xW0W/9EJIWz9WrmPexlYLsoxpyR5UmO70pZ2jZwIqEWpKY9e0i7/SsWsx19Oev
A1DcptSKMsvptsKBXE8YCYmNK2Cx/i57Y849E7RqEhY7NlmVT5W85MmH3D7qEJCgLBc8LFuv
tbngYMbEnzAMG28jDhTUwOV0vqFRy90QQ7HWuH47+0M7ZGyKpOoUBg8sWBt3rymTQjaT3sbB
NHBO5g8LgWzkK0sHCa74ejWnu6LxtF8o9r8Cm2ssmU93lRIVypbTNzzfWDHwV4EnnJ/v4VTV
/epMd12gIEumWnAaRAhu5EBQdZRIc24mLOIl/zzYOOyph/M7arDxkj1r5K1D6ETpozigdChg
7GNpKmL21ColEZRaP0Ack+DiId99FNS5EOgEiSBFsFsld83mZFFTTx3qWh6NQcTLDetvm+Yu
aRLUb1wLDNykLEJYX5i4TH1shOA/zeoQ1L6JGDKmlpUyJox6pxGJ6DSmhseXqLsSCtHnTPkx
D65vM/J0OGTKKWZtFc0zWnv2AjVEC5Ysk3lqkPBPjYZseai07PTRHWeRVvUHLdtN3QuBrJ2r
QAjKC4rI9FRtjeMB5t9JZMLjxoqv4TVX8H1mxPRhyhMyMf/QJe7kSddJeGha6rt5ghnQENK9
u/gjSQH55MzAZf3hjTHmLncjHiYdjBzOLh6dnh6z+rApCtQbvjI3Lo2y2rIwkj4NBp7lyEMt
foRMEkDeujGIFmZA3Yd24v4pyLBpHw8vmztFHmrZnctsvIktUvc3AGhkDDBZDyaxlztL/MRX
X8AT9CzuGO6kFemiVumUl8GHV8F+u4yLO6eZ5nBNWoc16ghcpvKwekgK0b0yB1UDAgpykp8d
zrNcWlV4mgGnILPTHsfZMSapEDt+ZEeRlKozMCLJT4BPbk2Yvg3SD/4Wht93Ato60dbM8hww
wRjbRbmUyy+6uHY0sXUALNqnZL28ZGQQOna5Qbh2Yk3lh5jd2NAROukKb8Mvrtb8pqj0Q4wy
dhfa2pH2Y0ynzKtvYKApdVsP7kWRdyU8z84r6VrQNZ6GJ13Ib09vATmA20uz4ylQZCW2v+Cq
EzuI9AXr27/pQETjwowvCiSKfqS8yZHYuoXxtCtD73iNx49LLYSV1oWGnYo1cmFlRxcDw4e2
NgfopaAvFSFxnlCReBXGmhVxilUnw/Q8dDKYg8jp1G3JJ5+E4EyRTmlMbRpRMAmOPpdKH777
ZR30o7jdhhPjBgk7tjmU/WHnUclrJG4QXNWXJ9qlHCwpP7THdO71MixFXdTx+Nm1FPe9pa4Y
GGe1p8Me1d/96Nw0teh+Ty1G11Y2rd3jGVxvdIJ43xQpASO3AZrVo/paAOtKbEAZjaG64fjP
Ea0BgikNwA5NXpWMWWDvNFnMlUk3KG24Zr4n9s2bQALgde4GFBfYV7o5nM1IFgn847XmqSzm
x+ru2JRcOLpobEGsTb7BjUU0TMGLgzPKSAwQxzkWyG4kEMLrhHSnjMV81PeKeYTRF40TXSl1
wdVBzVkA/4n6a8dq8Dru5UgL/S7ee2cto2V0PQGf4JtApEiTnP/JUwivp8gY+EsTb2sGOEAW
KkI/6OZVVxNTZ8XEzmo3uQrGwJvSkzlBZC0Wz58vyZK88LZhYGRL2GwFOXUWXZgzOB7I8s8j
Tmq5vE10IfeVFjKgXJXXKH3NkRBc/y+n0SXB1jfNLafpNzurjHm+Y4WhP+NgnmxS+l6mZOFB
zXnXILc8o2U3UHe3KX1l17RUSJ1AeWdLlvcXtJy61IQzcS9bTAZvAbk521ntOsZSdRJZ1Eof
Jc2lvH6jOIdwWqTIoHnwNfzUz9SM5os4qPnwA/7gaSlLgHO6yxXaKadoGWorX/ZATtbBOg99
V2T1rWlbd5Qelgxn0bLd6+tGyhG9T4hGFVkbFBNkw+MQGgEC7A5UdFtRMeDKtwxFmi9H25Tf
wazddsB9C1KOUjUuOIbseb2CIuXfeTtboVc5ZgtdVR8xqbAIufoeLdluXojy3cAzx1MISoWS
+omyR3OpDB1qz/Hs7zbmdXEm6dtLChq+evt3/v6WQLgM8lR01q8PX5bSSfzMbc9qFdWY1P+d
2PmqZ3E87BORXCx/t1fFAETJr1V85LJlt0l3oWVYxUCd7e7wrJ1IX6kG5tVjm1P9NBNj/DrB
IO+XcAefdFRQVm7LiafARSuFocQz2wIpAiPuJfIaCH96dr6OzqsgkA5+K/FTW6rUUW7/cpho
l/tw7Dtkh4WLcZhsFk6UgPKzsNdra1V0ujOrt5wUjRFRl8hiJFWMkiwAnYn9ydv6L2tEDwO7
eZQc6nyx4lb0V94YRUqhOceyc48+UwRGORflJA89dFlZLlfHvpZsWrEE6I2U8D/YIVl/3n5c
W6vBNJdAKTsX47DruHlsr5VyhnvTphN7KnZ5Qpnxi6ta5dQsTQ1nvgcwHk/CyBM1MuRGi1/z
qT7gwZzEa14DHxmtGR/6pmqeRDul8cwyb9NITy4xYgMMXqp3Jwp9UfOEv3r2wOVP6hZ0O4dK
q7Ltf9FGlc38wsvSqUDrdjwaxn1fJ5Ld6i9hPpI+1WHLFM2A+75xFICYkn/JqLFtLZX3w2mW
FNvcBvJ+wkYTUIHDue+WDWcew6ZlADPoyTv2RVC+TCwcQrFnySMjXTHkfPTqaCdFldXj9NYq
GkavP5OiGFMtv1VVnDY33aTfS9G6v1iTN0cVFngDTkIy3VhSuf9FJrm0WC0TP3EZwQd/2cUf
oqDhGhkzX/qJTD95aucZe7gQ6Arqv/+TqWKZwUSpfnPkkw56VdCoIiHncrHiXJ/btBPdBVaR
Jxjda3390iEhwMO5QDie1jsNqbbkqBkEbTT3Ne0mwU4ATVxX3Dyws0oVxpQvV/vj7tvJMuIz
DZ/otjjPe2DBGOgH2VD3A+EeSS0hc0mHpPjXT3Oql2lgvjB4XMXY+SRS4QlB5ct5BybwJQe8
1+hZ4H/prJn8u65mz5GFDAz3XRfg2sRuyG2dqmdy+/DeQel1QeTH0UeUi3Tz52LK4CJavipb
YAGeA56rADx3IbFlsEb3lVC9rR+8w98tgU3v1INP6kgj7Syaij1zvrc3hiqaLqhPm8DimRJg
C9v+3UkMkiNBu+Ip/2HyGevBkU9cCaKwWoieK2+dXqtjVPloTuKIyavR336W1Vg8OpcOad5L
L21PsE6yztKu053+7YlRxaV41vhDo4F40Bk3+O+DuopeCMx1n1B2avW7V7yQmQ6qte8JoauR
+aM24CQuBPjOnBErj90I6x6y9MWRwZKO0PZmDUWlOvHEF2fXn9H0B/IzH8TEiWy5n7M3v0xr
gX5ylgCVC0Gc4xTkwZ5TQ720qv3wlJ86ImgknS/5Ofwq+vxMg8vzfJFZNHTCuUIxjwaJYEUP
l/4IyMNu2qO8qSIsqKiqoy6ou1q+yPN1VNcK0CxOr0zOPfde38Ac6bekUvxE2EI2t7LrchJT
/5abG3uqtSmswlqenXetyXai3HCFvW3AfvF5CJ2QmmrwK83+fpeJk3jl0cIsATimcs4YeHH8
LoKD/h3pIoRhHCVXjEBt4EX10XQCeC22OISopVxyFUH/mPekRLRVSyWjoQEbwjdNvt69LTw4
S4C3v1KV9O0qbA45mug+Q8H7TabiWo285arREMWJi+NojXIJCDjQjOCc9JQF/t3CKBVIJZk3
ONWacu57kO17pqbd7QPCTw2V4uLIApFVJrSseWkJ/k83UhYR6lVniK3rA14MAlrEaVp59brn
Xzgpg+C5MY0O7lt70szGbkSXV5+v5jnDyXGAsGZPEe45im8IVYIF6jIr+9IYeGC/K61CsA53
2u7Gb+AacCDKTE/w6tuHd+Rje6jg3INhQmKYAeoMrz07fUh/ZEi49M9nZCqnmdJ0VPAIt2EC
qK8MiA+QsK8idmx5/AFVXGX8oxyE0fxcYGEEZwkkMJpJncutdF2EA1M2FAlOM3S0W5rqn1Dx
dr+aJz093wAiJnTEpwwRLN7galgOXkMJ855RFEfQF8C1dGkxt7LxzADAm4nyUH7L42HRvhDr
nQ9B9Yi6I/Gp62k8AMTwVI/WTsJE27Aep+pKkwme1Uj/pxvRXTSvRRYK+1u3L1vFej8GHUSD
y3E3EhHHapknLrBYgITWFYctispmBolb6s3Y7l/j8uEn/vdIJL38hoZJQQs692YE1qSbBAwA
O+J5iun8ZeY/jLCryU4EiB5rtEFuWy34UG9by/TDHdLh/XTHkL7iZ+tXuUt+ySEhaEAzxq5Y
rzndejNkyWfSU+2BcdCoJH1L+cLWB1tQTzIbzGbrbfgMfSzzKJVrVkOqdyaopkim+ZaIAfQx
vpzu97B68Aam2EDtS8K/2x8ydZswi+saUTT39psqsc7jccl6ET9TZE84lAJTTp09T9Pp7N5w
M2BmdOjrfzSNIBmX6cDK0RmEfsV/97hicfCg8IRM3h6gGuz7wX5FiNIh7gHnNnZrplJWZWLc
xkonMiTgmbs6h/np4U0Fkrlq0sDE+gZh0ilSMzoYBLoYOcaR+y77LaZ5u1RL/1Ch3bZMtNED
EDSQSiQhpdbhFkzyD+anjslGTlI07Evc0RqXxsHiegBJ/VwN9IYrzU5NYrqzpZJrM+Cx3Nnt
VZOqoVUXhfAtTr8zc+3/Jwbdqef4BKI0+sPql9NJGvC7gkXDazUeC1ojhABUU6nwuv0zSr+Z
+u30uePttKeFO7RG0tQTDZ+HT/GJnak2X1pI5DMNE+CNFvErRQyCYyS651inYlVeiQk9ETJn
d6ie2O0Ko4MrbvtD5+F3PiQGi31cDSKDfW41CEcTTP6gEtr/rNzEXJfK5Afu0psVC4lO1nzS
xdY1IAfDIfoAPhEyMBRNNBPAD3yA9PWld5W2Ns0f+vQ/LbNWRvfMO/snzdLzf7ZZgQGaOPHr
Ad9T2A09xZ/DdG/PPouJ7Gc6wOuNLn3Hvm7rfnHgqaILBWK1HdWmCFp3ODyaQXFYHFzW2h3s
y/A9qtRUhplpQ8aJDgdhBWN7dv7h8jqUgx3e6sHU1hKii8OzQ3YutiDFolVe68OHBx9SUbAD
qfb0B40p9LxI4omY6rosFa4Dzl4w8yphIjol1+84YARHarSeFZzmIrjQsy1hhrK1ZtXjwGfm
3XrwnYAyJitGO7+20OlVac5KP09Samjbrl64ee+xBSK7aEKjvk5pdPd+fUd+tSgL/7YEA+rL
/b9DvphvH28EIaNfTq/GYp4jmVTzkt1Vn8MeuUXOV/UvoagXLL5gKGxhK3OoLx2U5jWvHSBb
4TibbGEVq1bsFzIdfS1TebpV3oN58ujWAalBqs7saH6wQwuhqUwrQLpmvXGZZB/+DeUXIXmQ
ocDYzP+sNEvSqSMhTTo+9EcEUj8n9zkzaQ+m/kICpfRudzcCx8vST8Ip6MvA2OeH/eZkRWLg
VKkmt4MHC6DloxmqWSn1seSY7dGaqkAiInGXd7j986cNNZBiFzvppvAwuMej25u/oEHFApSa
kmEbxtSJTrbKwx4lpqfaSG1L6v90dgINnpWYvgjpjkpsAr3/kJJTTysSG1cYmmgvVYE8AcE2
cO6jPx299asc+y690TfcS6yjEIrWJ4CG7IrYEk1OfBNFDPXSPbTKjhOF8GfqPxqxXZ0Jsf9j
GJuyg0kf4UePaPvUJUNy0WF7hgpqU1Nv4aLJ8KFxIpZ+LpCHoatC4FI2bEp+waAmKLpAQJeu
tcgw50CIvLBjo506rzM1Icr427m+txpjZBysxlrf5jyGlmC6WIRdcxuPSd7rP0KTXltG8kdG
f8VQXazs2VtBE81kG7SqF34F6qs/qfNuvacT9gmtaxvBKqtm21gLbFHyLAz5/euXPmnprC2D
UWuPxDgwAKfKyB0hjOJOmakTJO27NrToLVclnahLRFsbuPq558fYXWEeypQYXDAwCy6ulOY1
yodmMscJDZeSHii7nORWlBPQgO7vE7yuAyVP05sAaaos2IfIyOYzo87zQUSNCWoKz32pLRQH
fySoaKzOb5ZL4wj+KVhET2GWuUKrfoCx1tLDZU6wprNwUgyedj+ibukjh5ljnRkOt/uX85XW
qefcqU3qpjICHz3rIh6Gn5I8m37DOtrS4IHixhs9P2SwjBQMipdsCrdEoJ6ETnALKL2gYnFD
Evr7a+x7A/mz+oT4olKLlGQtLDyxyG6lgWd9tsVbyQSA5LT22TDJxYQPn7F7txcvj553hG5S
+aHuZ2EVDVt8YQ4E5LTJmailxvpAeHE27rGn/VguvHPeQUk4IvI2wyyUCfbUEklYsZuoZ01R
7u9hJe+A5jUYWNvx2i8dC1LmxOTDajNZVJepnxnYQVHsSz/r0V9RdezwIYiET4YGd/CuCpDV
bzJDSe6paFf4u7n+7V+OMJpwwJMC4L+CV2PcsycWSZNGWSLiO4WCesbh0dFJKuJEIT6xCo5c
YdIwNWZvnMlXTlfQWrojlxLjD12mdBWvEqccXzKYcJBfUJTv7kzB1SeO+RDWYa2S91EEhPHF
RecWr2b9XrjkziBxiMZuWJDjfeTxKUWv+u988l+s4NARjGYRtmtclTFdixbi3pnLQ489ceCr
neFZkZcl9gNbzUddcqziTqXseINS+YxvVUJE1KpmbXm9u1ei5Qy4lXXeKa4i1qABBdKnLGLO
NPsn6QjyIt2fQdPMnk2XsRTz+wRJjOtGFkcx/XPlMgkDcjUsylQ350wEAlr9JRbk1sh7mLym
kIu7B3OaG6RF0CfvlfLWIVI0HXa0qNdVCrg6Jvuz6haIcls3xB5Fyl8qjKHJFzi6jyiW9e3P
lhn9t3cVWLQqZ9o4wkMyOJqm8vpBRiCLvtSagcZ1v6FPTSp9QAwN3H9uBtRU+zJwyyz4zkKo
w2P7yEqMelpIyNAMdB/PlreIpE4h5oYRfzvDOmkh06hjJDSXqghCh7ZmAjt42zhw53xIo8aC
DvlQkUq1vVOeQjyTJP7iaY8r8fLdfRqnG0NUFbmB51JPmu+4Q0PEqKek/0M1/+aHgqmjCkWV
85k1fET9GDKabVV9k6jNtFkXqjL9Hy+iqAsIdj9SxldMeTwpOxIif+PAhQ7A+WZUXrltpwbY
gcgFoHZ8CYsQVFdJWtJcaMgNU0q8JRGxt8wTgtY2fnThaigcO+gZoY75mYZdjAaUwbuy+zLA
pzGMdZ2/Nm7lSJdV93gKrg+lA6byLmp3Qn11y6Iuk69/XCe3mr0rYbMxivEehbowuIg0jJ8y
a2SOZfhDB+daA1NYjdiBHQTdfASeHFaCx5Cuv9NUbEOkZfVOKT3I0MXqyLlSmpD8Xum1Afjm
e1P+TfA5D5SkH1zOJHVfkZ51QvUp/1NV3ZJBArdBiPVbkzUa2yWlLZGuw12CzqxIWUOECVPa
EQG+qtnWuao2dp2u74ktlt9JSMu+ux6xpEHtyK5Zb0Gp6BhFahgoxna0yve5rT+eUOveOsSC
V2i9JydK9EZcVCCIwt2kMGEaN/NldnEQ3xpGZ7uBqKaao3Q/IC+VQt8SveNTvEXfYZ+oWqqe
fFJ3I7T/hsMHl9974/J94urDPOdzzmy8UMBVWkvTv8h7FnSGIXOVBwREigaxLp2z1zvCcH/b
troDwCP37HQ5qNXrZpVgFkllnRPT8PRpekAsh5CVu8y7kypVLdx3CVkgFSCb+IZKA9IzGChW
uvIvtcgNbhs9FNgDxrDz+KiglAoTdbw40gvRgERlaGHM8QbERQmB/HZN0OB4xaTTCoRtDTd4
o38d8YtYKBfSiE5jSRSg0MN5hMfvBtrdBqM5J+2wy3fqo18RnmL0JK5DCyCka2LSgHxJ0Qr3
mWbUM5K+u69cP9wEQ7HxxTxgUIsnxkjEvAJiQQxf2Brq2GWIm5K9v9vBhkCwnD8JEvdSaNay
p5NP9Q8eDnZcnR3UlCQ1TGGZk2d6dq3VxOU+tlvmOaUAsWg+ob+nRysP4wy3NW4dxJo8/Xh2
j6dBv0IWiS/anK+nb1rHuTMkzW0ALvVGAEhI316zpol/oyzNFBrE0tcdPS/XocJcucYrZjp6
YT1rdwhiUrBUx46sojGRDn/MQewvkiEOeYVEOQpGG/v/BULokUTTCXj9ylVHXOgp65txhxur
be0nLHBrqRxhDizrFT3RiXqR3GV5lTueOKrzg7NXOCeerHJFDDuj6Z/yqhwRuvU80ZLiaumt
DLFctP8Gn27ezvSXoIwiHdPMAKQz7QYeEoEp0ro6pRcVdBCyg8OZrpIcLqMcrucquZ9V0Vsq
geBBi+DPPtoOJ+yQSCznOkOLXbFkvXVNHPMl7f4R19nSoFL23JPtiscZ4W6TtNq7JYaNHmiB
Irpmn+XxArpvHZ6qqeAePdegHEdj+VMbA97SsQFwEJLtTDyVW5KlVQJyHWSKd5+W4tIn6HoA
1dNlpQlSwcpb4StQ+2nGpRFt6YCMU3fTBIdLABR5SA7XoG2svtQRH9SaDuzaQ01jNUMJZ37n
BS56HqOIlalsIVbbawAPJBEUqfEBE4B9xGVlSmKJRj155ERPBR+Vefik/z9BfGVn1mjfU8pT
CNpzNhgu7qIFRZFymojxsAW3eZmrHb4jdnvr9KMmPKtJ3Vw+eq7vhqS0i1rlTE07h7Q+jWsu
jgA1DwlNERJ5CSJhUSoLkibzPuzJuO9KpPCIQ/CuOGACVoee41aVTQIzVf/72/CQiKIJ75Z1
RFykc/kVzQqAFhRlhV4oVZ/6lE5Xy4sAe8AAbKYRNhP/dKrudOS31QO7gfun8WungB/nYdC7
9FXzwMikG0U8v1we0lPIvLa/tQaLvD3Pduekv10JBncaJ4euA8ZezPjaDOjQc26HxAjbpK5A
B0QX/DZp86TkiKHaHnio17V8nJfincGnmrL5mZfT4CvV0DTGIt9I4ec3TVy3Wh7dGSaeEyzK
Vj1+yT3RA/FuGwXpk6WLSFg5SYHwbahk1RXqaVC7NcizI8A3G632zZQoSKXrWJs7G1bgxAz7
+uAYT/DnYJ8d9HPgRyVb8V0egO7Oa2qshwzy0XL9C/K01hIjXNpbfkgskzXBmnW6vP32pMtR
8TZ4YwJxf6rhaPAhgkf/tkUczB14eZc9moBTo4VKwDIpi7d/+mIw9kHr2JYyylSVdRISgUXX
q7MfAOqCKYjnrLJhO9040QoDNPU9k1NO4f9pWWGVBUmUfnCDJMokN0E0ENm+QwxgDZusRMHZ
ftjX4qRhAWn71IOjTlFr4pZK8V1+hxNJ9gK0lznc2R3vCm3KJE+QWuZbk0dQkhCmBUlQ//sw
kPcrq8wU53I+UVZpt+4UX+vqyYKYSm3oBRTOL5pmrTnpWhlai9q/VvCaWzNVwD1D3VJeHBdZ
v3t9nT/fcMhjK9Ojt/2SLEldVJGk8eG8ojnsXwgDH42vigCc8tmS7yPfqj5eyW/P+GpqTzdW
J9BPKmxE3lwrdlzKZtenuFWYla3P8Lh/MQeNytkYJuVGbv11DH2Ud3c4GFTA/6zurOp0CjDC
V1FemSDjIJFDyVZfMZnn3bp3jFKr/+6UoBgNx45hpm6sz8Hnr0CTeZzbT0csEdam0XnhcEA8
oMNv4WkH5JRl98Vn/vwdwTuwmrfK5XWSMXi7XNHIj8u9Tm0HRIWAoPM5Oorn1pN4X2bMpu4z
Q1jmvCA+4tstttYbC6EKSH+sM26FPvO2LQ2eb6LkI/Jb+8u4gsfMJmdomkbUP47dGdawTIBW
iTG+JlGiMmYauir7+NGrFqVODSfD51cFul/5WunZSXrwXxORv4yBtj9nX1DgPBm0Y9JwBtcD
ixHtCDq0+qI3bHZB57pcc8HqVeWEG+nF5TnWNwoVCTE8NoSMTvbtEMNvBSBxyp81bMqapzQL
sW1xAzjtpwCTjIDHvZHzrWJoQY9hHwR2FapcTondlqwagcHqfNhZEa5O+5RQMXgV2085x0e1
row/AtyJeT/Vq7311HABbIJRt239YYsACCsP2+u2bjOdpaRhvsjTgTMxCDb48JsxrtwY6kMj
jxHg0bnEVVQk54nUbF/wdQ34wUTwW9BFDsYwMMPqeTGfMpLKWLAQH1iA0gjsHiYyRq4xKMn6
/ohdEd3ZgvVbfE1sxT9Aj9xN+6EAXtqxZPEYi4guEEjxhaStQxe7TaE8x8jjTz/YGyYFZAKV
IB61xvr+w/ztfyK5Tbe+KlwSquLO0MgzkuHKwRjfg1VU3veU1x4vlTFzS/nmdIfBTubv3spe
yR6dAynhQO4uR4CseJbYQRr5D+QmovrmYmjU5Qg+UabKXObnjVqgMfnxr/Prxo02fnwINW57
ukVNO7VChYT3D4M43oByZwrr1DemagU0F2QsH0Nb5h09gaJWuBd7VyraKvU5S4llkXFO9X9h
YvTVhQe92wl54LwDH0pfnlNxch5nUrTOUbUdsoqImGHMvqmaihcZE8zFEZaawpYz1vRDbSDR
gcWrCpny0gYEUsnZZjSxhgq0tuNNrRZ3CRwfkwgn5Pc3xYHR8q6OblhJUTfvCvxR65/2a/Nm
VT+4y0r9eI9YjSypOj5H/7A5ASLTWmmK5yCOSXtbdlMfT5GyxxDLt8QCiioV0Uv8nYvSBzXD
sWfaQHJUGqVNJaLfi69bDJ8Ca3EG5BRHWFwj1bEWIXaFB9CC7V6blaFbP3QX1b1YsrpqgyEQ
48Xz5vXxIaxHSMgdgARsM0msdBWyYXsIiZGbDGkGOkHGtw40yx7R8e4y/GM/JfmSDp8M+Vk+
rTqjfjKz05TOTSoZdVm7dyAlRUMq+uD6x6dKXkQqnHtoX1U3IUUB7GVcClJE6i1ClvBXXM6l
zirRUIoc8d9u5xBnfR9bx1sW3AMW/77Q3vt4qOFxlXXwsJhF9egNx7ZFQy6dyWQmltnvV4/v
oaVtf5IDpo7W8PDmZo6ws3LldlR7S0OgNwiz+a1Z+a9XNf38wWoCA30aCKGGt1UW9D/OwBpc
kmxP60BWzyuuaECx05gj9BfTF68w7ulo9F9UJFfbQW8ei54yrXso3KdaIAav4MkmMpqG3U8K
ON636Wi3VOtVbQE4EaR5yLnFnfWAcZvb2RB+B0Uibwdl9PlM7m8tQsFNX4p/Ajidx6t/XQM2
70SmK45tvO8xo/cjWYuyEAuo6xmY4k9U6c2V0n9/15g6oVR618j2H/Oqbvroot36Z6Uzb8Ab
XSx0xql65LZ/fGUU0JsDBmu+I7XtGBi/Li6AJqWXwlcxlaJqBMH7unPMBB0mePrzjF5y+fJ3
2BrY0E9VSxCR8uPKiznlsT53Jo41hrbfDshftXb0AYZNZ31wi5N1ezhcBXjLxivyWKAwW5p/
ybb73uKJbDFO8amc/OclXJX7Nqs9Tt5RjRo/VG9tDSgbYLrvx2BO86EOd5Qam9R16HYwZQMr
JneT3kVx/RgUOABTytEIYJwvvVt/dFQKwG7McruKrTTk2QqZ84gVT59SRNDOECtkR+5CA41k
HgGaGTpXV41FOgSJvAroCkg1RI/JWr7G7v8g50VEz947OP6dpn7ZCv7S+llWrMFS8Sgu4erW
xG7GCI+3EPVQP4rXhGcmEjOvVsiJiHMrBzcBWPeEhXDZgAmlEusJhFQUrizeRtWg+e2QnLxr
7pkm59Hq3JYaU/oBk3rn0GP55oZ4tzmziR8liwegjzFGWB8bKr3l9KKYKyNAddJkB3TrCqm3
O1jFOydWBbbEzO+2RK1rytlwTZn0m4QHjifZtjHe870mmInHXXtmIXTwZ3bRn3G08FSQl3Lv
4+2BjaslVp+LJ+Jf+o1vzNyPSvFDYKhGv7qAN7zC+Kl9iOpJomkJzvSNVVC1blaQBEgrJnF0
qFVilw7md40xdNwyi+SBE6BWKToaW+FUbBoq8xyn24vzOBn7LCEazSbLkjvvJI9BopQzJimt
VraIg6e1gxH6XAqmDjJfjkZihU48Lqe4kfLc1shxih0AFMtlsAiaXmHVOuGG9bCm0NZTBGG5
XiOGZPuyoRSX1vy4lFTyA2QvckCp8qag780EFIDeITAOgoWDNiD0L9sjQr04hspvlh7jFQeC
zaygt4hAp7zUGEYrGJkb/4zuGECzWBifzcKXblLd9+g06eUce/5x5E0p+DChhD4Xfp7V9o2l
IW6AiHKK+FU0s7wmt63TcvY38KFX2QsXVOOna3xLRxkN1vR2r3bFlS1pgk2Cx1RCjgM2KLJ4
vRwtSrTjj8+bC3A3TSMkcFeO7wAl5MbP569H7QABkFHX3CQAN4i4FbHEZ/sCAAAAAARZWg==

--veXX9dWIonWZEC6h--

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1753053AbcKQOc2 (ORCPT <rfc822;w@1wt.eu>);
        Thu, 17 Nov 2016 09:32:28 -0500
Received: from foss.arm.com ([217.140.101.70]:52420 "EHLO foss.arm.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1751059AbcKQOcV (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 17 Nov 2016 09:32:21 -0500
Date: Thu, 17 Nov 2016 14:25:29 +0000
From: Will Deacon <will.deacon@arm.com>
To: Robert Richter <robert.richter@cavium.com>
Cc: Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
        Ard Biesheuvel <ard.biesheuvel@linaro.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161117142528.GJ22855@arm.com>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161109195132.GZ22012@rric.localdomain>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
> Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
> keeps the same behaviour as before since it still uses memblock_is_
> memory(). Could you more describe your concerns why do you think this
> patch breaks the kernel and moves the problem somewhere else? I
> believe it fixes the problem at all.

acpi_os_ioremap always ends up in __ioremap_caller, regardless of
memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.

Will

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1751287AbcKQQ6I (ORCPT <rfc822;w@1wt.eu>);
        Thu, 17 Nov 2016 11:58:08 -0500
Received: from mail-by2nam03on0083.outbound.protection.outlook.com ([104.47.42.83]:20138
        "EHLO NAM03-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S1752037AbcKQQ4w (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 17 Nov 2016 11:56:52 -0500
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Thu, 17 Nov 2016 16:18:05 +0100
From: Robert Richter <robert.richter@cavium.com>
To: Will Deacon <will.deacon@arm.com>
CC: Robert Richter <rric@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
        "Ard Biesheuvel" <ard.biesheuvel@linaro.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161117151805.GJ2151@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20161117142528.GJ22855@arm.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.80.42]
X-ClientProxiedBy: VI1PR06CA0005.eurprd06.prod.outlook.com (10.162.116.143) To
 BL2PR07MB2338.namprd07.prod.outlook.com (10.167.101.16)
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2338;2:QW0ExkVAuWHiSt5DLkFdorbuVNe2qvooxOQmSIBIxaDh5/SiCwXTNPwx96JP1hoe6Cr4THxWL5Jt4KPhJXvL6F7jekzPYRE/ojMKEIQYdIVtRY6nyLfOtPr2rz3Zg46bSnTDaRxdg2LxRcpgx3BHAMol60dx8bzE0mJRuMeYp7c=;3:Rv45vtfECrQVCA7qZWcbNmyW4n7MhUcx9ZX60DZCx09mjLT4/M4zUEeG5vlhoTu/FeMnK5jlGWDb4nZZP0gVkgsFWO697+UBTPLF9oDaNBqsDZJlHlqa9Us4Z4PjYtzWtJmDiP9j+e9xgWSx3KMF/M5PDlBg0iSiLugEPwAqC5Q=
X-MS-Office365-Filtering-Correlation-Id: a3cce394-bdc0-4a17-62f6-08d40efcf644
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:BL2PR07MB2338;
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2338;25: 
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2338;31:QwVQvE3MnK0jAGUY5aGfHot8qoDSgGB6GmC7VBPjQd5JlgugBafy2sHa5/zX7IrEQuxg6MLNDQJswXG9nXq/05ORl6Yf09TErRkpg13XpFG6zJ28YruDj1QPuuUb/VJ2QQmkf01JaB3x3yV8iZE0As6brejuA/vj8FezhV4cN3g/pgm140dn7zZlcJHR+VBc454w2HwAN0LfKSPmAKe6iPQ9AeFYEls3yjJsg7AKY7Yr9PgzZwRKkwde5DaCdrlRPFkXOm+tJcaSynuxjkCJQA==;20: 
X-Microsoft-Antispam-PRVS: <BL2PR07MB2338BFA135AABCE84BB5708FF1B10@BL2PR07MB2338.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040281)(6060326)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6061324)(6041223);SRVR:BL2PR07MB2338;BCL:0;PCL:0;RULEID:;SRVR:BL2PR07MB2338;
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2338;4:ask8ZqPJoKRdtEKUDh7Y4g9lom2dojpC0SgKadX2Y+qpFrAVUCgjeHQBV0Qq4SAPiIQ+WfIM70O7d7SsEtTAZ5Ssjc2OLNfJqYWDXrWlkyVLMVl1qKIidTRxUFrmgYwiuSclAEVHEL1tWEiat/IigqeHIQ5bj10KLaU0W5Azc1yUhfGx2rAOMfPRzuiLU2rBp7NoZ90QEt4WEvL0WRwhhWlDCowetbyWjVaGbVvX0DUZ/lBbC/lESOQt6sF+YCpkHCyUnGrYpqsoiAt4m3MfOxKCiRPopr2XBWIMYDHVIY7VzlsV2tCDBn/nwdwIyvwplF+7SsnRs7dy4Zu/7DxjAwkmtbe56IGXL8i1J/LTrQCfXYZJYuY5qtoLIyE7FqX7WpRR0eK/GoHZ+anIZmmPGsmOh53ldMJ0QMEzImC789fvyzAYuaTdC3N973Z5UERibP8qmBsy7YvHyj0bPQVaTw==
X-Forefront-PRVS: 01294F875B
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(24454002)(189002)(4001350100001)(47776003)(105586002)(6116002)(189998001)(92566002)(83506001)(97736004)(4326007)(50466002)(8676002)(46406003)(2906002)(66066001)(1076002)(3846002)(23726003)(7736002)(68736007)(106356001)(305945005)(42186005)(7846002)(110136003)(77096005)(101416001)(81166006)(81156014)(33656002)(6666003)(2950100002)(6916009)(86362001)(93886004)(5660300001)(76176999)(97756001)(9686002)(50986999)(54356999)(229853002)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:BL2PR07MB2338;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BL2PR07MB2338;23:315eyRw+ob7pzp8kefTt0Awwa2ZEQ2Rjb9EJxrDSs?=
 =?us-ascii?Q?EsnL94mNyQru5Sy+LJBJ//DSJlrIjXf3YSYKBxWrVLlIHrBCy5bg12YAriPq?=
 =?us-ascii?Q?Bah7ZFTxO0f5Rylo23dTQZbxtul2tNgEWBLJuf/lERjUOrDz0PT9FjvPHMje?=
 =?us-ascii?Q?6rV0CyWz7JzB+QQXS/9D0uVQ5sE10Rt3wbNPcroRooXY6+pTNtW6EgSV5tIX?=
 =?us-ascii?Q?7tCTgNd8hWgqmkSvqcq7WhACsfl8ROTARMfPDIgXa4arYwFiUFzsOZHL+rU6?=
 =?us-ascii?Q?0LK9qjRx6fyaqGiDYslaGP68Ms8kXdSkf0BBTYhB1/hncZnjofEfgKjCQTjK?=
 =?us-ascii?Q?RLfqcNu4cxO+RzZ6DbL/4wyjpwQUwilBIaVA0PDVvUk2NsxlNXcMmnzrQaaK?=
 =?us-ascii?Q?O/0xEVsPiYoZsWp6QNwibeZJX8PKwGURz7mmFRIKPYQvwMkuS9kkYHRydBbZ?=
 =?us-ascii?Q?coXD1c0XnEvHXqqM2vnyne7LFGfGGYB9xv93vK4EWTc8Jus7LEe1vPXald9M?=
 =?us-ascii?Q?F+Xx5Ot64t+WaQz7fgnf9a/Ht735URi3ITUPzZkTWKMZQKUsjhCOa2juyMoY?=
 =?us-ascii?Q?kUvGzN3Bgtnuqt++46sRd+6ggMmNGG4roiDrGuS8Ah7Qz3zX+S2TUqW5F8Yv?=
 =?us-ascii?Q?x3EFlSK3K078Q1fGgQwh/81ONhkf0RGQ7KdMbqLcssaDtSJhNKMarUAJxYOs?=
 =?us-ascii?Q?HtW1BM3xfToT0A0ACmVwljjfGoFHywepVGxfiqRrTPdYulhUIYeWJ+j7Cn2h?=
 =?us-ascii?Q?I2fKvl/9Y8ECMRX4upsBLK1zCgmd/QUNxT8JoG3Q4sXuho1XqmVndFU6N2NH?=
 =?us-ascii?Q?q3AieLdEnb/IePKXv2Bz6BK5DGXFUotEgGcwXWBayw6yVAD05zeyEmTkvrBB?=
 =?us-ascii?Q?QdCjww3Bhd6Ach18PoV4CHHYvAeRIe2317UJJsy/IFCHfGHR0i7aUaCFd6eB?=
 =?us-ascii?Q?3VdeOCtgPKt+L4OwFzgAg+b9x2Dw2N9B9IbdyJqNLfMVk6dASvyh1lwmH6PU?=
 =?us-ascii?Q?m0fLzXWT/dqpDC6yhOgi8PmvDpKL/F+ud5DzltmQeMeEt8S/mKV2q9LgCJOv?=
 =?us-ascii?Q?+CK2QtZtjeTEg3yQ3fT+vZeh1Z3etF+00QKVMKifkW5NeL7kqC3p0jHxLXFs?=
 =?us-ascii?Q?bGQa/uXIMtdJ47XLXKZ+92btLq0E+HVMq6r/NSEEYMH9F2RWY4ATHezWwANp?=
 =?us-ascii?Q?f1BVxWoS/lnVSs=3D?=
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2338;6:ejCV6tgvv18W1aWedlBu6K/fRmJ/jN+NiRfFZuO0/NBEAicH8kBP6qHCPpPpcAHhaYsWo6V/kLdJoppETMIbnfOBfAyvtmjwt48qHNJuiJp+xU62ooty6fXajBhhpdqhxTZHlc6s96qsywhaP6kwZNH7T8D1hVP6O4nf0rgIVGBtA9nPC60jfc0eP76LekVpKH56nq/HWzZUI6R3ljmwuAUZ9pdrwq6NYt8cj5TxeATfqOdUlGXyDVU9xl2pzM+1BNN0xw4ihseryd5FQhtRn5MI/Xs3mHkHfN3S2AT25+/ioYLueJOxN9g4LzOYgur32B4E3BY6G+rSZ2vttj2ImFLHE8ZqXkoA25Wyk9uW+i8=;5:/lhmjPTZZlRNguaebjy2B7BDH68QNJxMlDTm5xnDfb+lItZljeeEZfvwo8N28DFXerL/TWL4cLDj7T33spyuGjqaWPhLZJ9yIUbX3dfeGLCQrKekXQRlSoYNq+aF+50cxY9Fb14f0LrZi8IcBdmCoeG5Eaqe8pErTH/tqXSsb1s=;24:VtBTDnIuH0OC3YFyvVRJugfsG4ABxa0SfDfbpU7GC/xOOt4Hh3YQFJ/AXQyYwibcO4EUDT5O9vcPzPbWhsnu4ufTnkeGFFHtvvxUB82ewjM=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2338;7:3/gIvVJa0BhhEGt00neGFhrwtki62/l1QebmgFtXKnbUggsLzqFnEkp3/eQWfq8BUcz6JnyQtKPcQezYO6mYplM2jgCQNg96OjfJMez0P+SOuinwpm4qdHC2l2dlkgpraL7NLnqbcPI4h8EWO54kQ84lIIsJnugUvOfTk9y9LGHK2s/cWz5Se4X5w6u8fluAyFE2P2LpxCBWNXpIJs1O7aycmqAVE7siwr/hIT0btlxu8/49HAtyWklszTECKprz4CZJygE4dcyp8XSFSc1bTuUB9681wlv77dASkts6QdEgpgrYEKqyrmYsan/crSBeYgVXfhfPTvJXaGl9zyY07lfi3SOn3qyIrAF+Jney5zU=
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2016 15:18:17.4182 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR07MB2338
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Thanks for your answer.

On 17.11.16 14:25:29, Will Deacon wrote:
> On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
> > Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
> > keeps the same behaviour as before since it still uses memblock_is_
> > memory(). Could you more describe your concerns why do you think this
> > patch breaks the kernel and moves the problem somewhere else? I
> > believe it fixes the problem at all.
> 
> acpi_os_ioremap always ends up in __ioremap_caller, regardless of
> memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.

But that's the reason my patch changed the code to use memblock_is_
map_memory() instead. I was looking into the users of pfn_valid() esp.
in arm64 code and changed it where required.

This week I looked into the kernel again for code that might break by
a pfn_valid() change. I found try_ram_remap() in memremap.c that has
changed behaviour now, but this is explicit for MEMREMAP_WB, so it
should be fine.

Maybe it might be better to use page_is_ram() in addition to
pfn_valid() where necessary. This should work now after commit:

 e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem

I still think pfn_valid() is not the correct use to determine the mem
attributes for mappings, there are further checks required.

The risk of breaking something with my patch is small and limited only
to the mapping of efi reserved regions (which is the state of 4.4). If
something breaks anyway it can easily be fixed by adding more checks
to pfn_valid() as suggested above.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1753222AbcKTRHs (ORCPT <rfc822;w@1wt.eu>);
        Sun, 20 Nov 2016 12:07:48 -0500
Received: from mail-io0-f179.google.com ([209.85.223.179]:33985 "EHLO
        mail-io0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1752145AbcKTRHq (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Sun, 20 Nov 2016 12:07:46 -0500
MIME-Version: 1.0
In-Reply-To: <20161117151805.GJ2151@rric.localdomain>
References: <1475747527-32387-1-git-send-email-rrichter@cavium.com>
 <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain> <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com> <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com> <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com> <20161117151805.GJ2151@rric.localdomain>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Sun, 20 Nov 2016 17:07:44 +0000
Message-ID: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 17 November 2016 at 15:18, Robert Richter <robert.richter@cavium.com> wrote:
> Thanks for your answer.
>
> On 17.11.16 14:25:29, Will Deacon wrote:
>> On Wed, Nov 09, 2016 at 08:51:32PM +0100, Robert Richter wrote:
>> > Thus, I don't see where my patch breaks code. Even acpi_os_ioremap()
>> > keeps the same behaviour as before since it still uses memblock_is_
>> > memory(). Could you more describe your concerns why do you think this
>> > patch breaks the kernel and moves the problem somewhere else? I
>> > believe it fixes the problem at all.
>>
>> acpi_os_ioremap always ends up in __ioremap_caller, regardless of
>> memblock_is_memory(). __ioremap_caller then fails if pfn_valid is true.
>
> But that's the reason my patch changed the code to use memblock_is_
> map_memory() instead. I was looking into the users of pfn_valid() esp.
> in arm64 code and changed it where required.
>
> This week I looked into the kernel again for code that might break by
> a pfn_valid() change. I found try_ram_remap() in memremap.c that has
> changed behaviour now, but this is explicit for MEMREMAP_WB, so it
> should be fine.
>
> Maybe it might be better to use page_is_ram() in addition to
> pfn_valid() where necessary. This should work now after commit:
>
>  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>
> I still think pfn_valid() is not the correct use to determine the mem
> attributes for mappings, there are further checks required.
>
> The risk of breaking something with my patch is small and limited only
> to the mapping of efi reserved regions (which is the state of 4.4). If
> something breaks anyway it can easily be fixed by adding more checks
> to pfn_valid() as suggested above.
>

As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
the correct way to address this. However, doing that does uncover a
bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
page fields before the pfn_valid_within() check, so it seems those
need to be switched around.

Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
for sparsemem. Care to elaborate why?

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S933800AbcKWVZK (ORCPT <rfc822;w@1wt.eu>);
        Wed, 23 Nov 2016 16:25:10 -0500
Received: from mail-io0-f172.google.com ([209.85.223.172]:34652 "EHLO
        mail-io0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1755376AbcKWVZI (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 23 Nov 2016 16:25:08 -0500
MIME-Version: 1.0
In-Reply-To: <20161123211538.GH10776@rric.localdomain>
References: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain> <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com> <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com> <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com> <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com> <20161123211538.GH10776@rric.localdomain>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Wed, 23 Nov 2016 21:25:06 +0000
Message-ID: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 23 November 2016 at 21:15, Robert Richter <robert.richter@cavium.com> wrote:
> On 20.11.16 17:07:44, Ard Biesheuvel wrote:
>> On 17 November 2016 at 15:18, Robert Richter <robert.richter@cavium.com> wrote:
>
>> > The risk of breaking something with my patch is small and limited only
>> > to the mapping of efi reserved regions (which is the state of 4.4). If
>> > something breaks anyway it can easily be fixed by adding more checks
>> > to pfn_valid() as suggested above.
>> >
>>
>> As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
>> the correct way to address this. However, doing that does uncover a
>> bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
>> page fields before the pfn_valid_within() check, so it seems those
>> need to be switched around.
>>
>> Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
>> for sparsemem. Care to elaborate why?
>
> HOLES_IN_ZONE is of rare use in the kernel. I think it was introduced
> to save memory for the memmap and only some single systems enable it.
> There is no architecture that enables it entirely. For good reasons...
>
> It introduces additional checks. pfn_valid() is usually checked only
> once for the whole memmap. There are a number of checks enabled, just
> grep for pfn_valid_within. This will increase the number of
> pfn_valid() calls by a factor of MAX_ORDER_NR_PAGES, in my config this
> is 8k. So, this is not the direction to go.
>

That does sound like a potential issue. But does it cause any slowdown
in practice?

The reality is that CONFIG_HOLES_IN_ZONE perfectly describes the
situation, and so it is still my preferred option if the performance
hit is tolerable.

> My patch fixes a regression in the kernel that was introduced by the
> nomap implementation. Some systems can not boot anymore, beside of
> that the BUG_ON() may occur any time depending only on physical page
> access, we need to fix 4.9. Here is a reproducer:
>
>  https://patchwork.kernel.org/patch/9407677/
>
> My patch also does not break memremap(). With my patch applied
> try_ram_remap() would return a linear addr for nomap regions. But this
> is only called if WB is explicitly requested, so it should not happen.

Why? MEMREMAP_WB is used often, among other things for mapping
firmware tables, which are marked as NOMAP, so in these cases, the
linear address is not mapped.

> If you think pfn_valid() is wrong here, I am happy to send a patch
> that fixes this by using page_is_ram(). In any case, the worst case
> that may happen is to behave the same as v4.4, we might fix then the
> wrong use of pfn_valid() where it is not correctly used to check for
> ram.
>

page_is_ram() uses string comparisons to look for regions called
'System RAM'. Is that something we can tolerate for each pfn_valid()
calll?

Perhaps the solution is to reimplement page_is_ram() for arm64 using
memblock_is_memory() instead, But that still means we need to modify
the generic memremap() code first to switch to it before changing the
arm64 implementation of pfn_valid

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1756089AbcKWVs4 (ORCPT <rfc822;w@1wt.eu>);
        Wed, 23 Nov 2016 16:48:56 -0500
Received: from mail-by2nam03on0062.outbound.protection.outlook.com ([104.47.42.62]:33136
        "EHLO NAM03-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S1753807AbcKWVsy (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 23 Nov 2016 16:48:54 -0500
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Wed, 23 Nov 2016 22:15:38 +0100
From: Robert Richter <robert.richter@cavium.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
CC: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        "Mark Rutland" <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161123211538.GH10776@rric.localdomain>
References: <CAKv+Gu-oYzSn_eeqaX3QVFeQjBGkaX-Wh7+oPcQFcXv7J6GOFQ@mail.gmail.com>
 <20161006161114.GH22012@rric.localdomain>
 <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.81.63]
X-ClientProxiedBy: AM4PR01CA0012.eurprd01.prod.exchangelabs.com
 (10.164.74.150) To CY1PR07MB2346.namprd07.prod.outlook.com (10.166.194.145)
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2346;2:4NX+0lv/vAsDrIFUhsaK0CF8ilApT6+JYLxGgWZj/Twxuh0087ifToAYoEK88w2ph3uGoeeMWr9+FU2wrthIqhqKpcMO3AEEkzE4vGWBq6GAtddsz6VXkQHc5r+zAoTcng6LxLYcIh3hZt2oXRQ7dQbA75vTuGoSqH3MDaUoi7o=;3:Ibx9ulYVaJu8RVLwXcMrUIMISC3cbhMuAL4So/YXpiA07eIACTllSqJgDlMD5sebiaHBn0THBe5qNc1TTWlZ9aPGRSZZKgb3n0ZILwIU2Zl9/nI+C4TM1bMQDwTQa3swHHNlX8mLHu+XXq3ybWP38g1PKmPeqzHszGgdNXYzkWY=;25:nnGWu/PquoBwTF8RTi9LKfgkFq5Ey76Od4RPzON+pxdIEIZZJxvEwDqgEbx6gA5w0yB+J1+xhcpBJmzWk44Ep+QpApkUZViDCC5MXY6UGNDo4lMAtjDExfmW8sCkC4DdLmKOZtK1SHLsNxLPusdcqxbG8Kjhox/MZXg6HzZeJzAAusmmI5NWdozMqhE8Ay+V1MIDtzZcOz8RBHlzolEakPRSnRhYgw1GBhZqjE2OhWGIQiLF9mUfD1s2+1BVh1k4GUJ9ZkyFKj5ffWYdoF45B1xcOW0F6QsQQYf7GOt+G8c0Hac8PX4SYlbwBao6LKMlCj8RhnqaX3xT+vUOCx158wyhwm8RGtDcSLaF5QqRH7Wo6+cdwROPL2paCW8a2qSdHlZM1wWuLxDhYFHmzndlSd44GmvJlTkTOMc5TM9PxWth7G/MkJpeH7MXTRx28YI4+aLUQ4nXn/AFoGlfwM4a5g==
X-MS-Office365-Filtering-Correlation-Id: 2fe0559c-9bf2-47e8-133d-08d413e5e70c
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:CY1PR07MB2346;
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2346;31:PbPszuLZ1Orj7C2VqdaV391VDLMdb8bVtkcPfIueC3IMbcDPDI7YZalOXRkTj12WAJt9FQS3a1QvZB+eMqiKfkWC+DWd0n+3JfQv3vyxdPjY3zPXvw5UYoSWr/RsFHpOpKyKxxckwOaua1KICB18icDcqtbxmbnGQRXKTD7R41pHiHi+ChQQmfXkuayP3r7iNJHnkulZ4I5bEZtY7GuZI/hZOorc7Ev1muZnzkE+Q0khW6TJKQ3oKFfLGB6PZ5vt8UCIjnRBMLFcZoco+mWpaM84iet6bedPt5Es9ShLRaY=;20: 
X-Microsoft-Antispam-PRVS: <CY1PR07MB23468B937E07BFC6BE080893F1B70@CY1PR07MB2346.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040307)(6045199)(6060326)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6061324)(6041248);SRVR:CY1PR07MB2346;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2346;
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2346;4:anRNtLzZnLlH6WFTR3oa0DYVO0uhT5Dy9OPuF6aWI+pYbILGRGRErbYnyaq2UVUCxf5B8BIrFddWiIN4ULmKURsYhVbldCFUS+yegr0sVn4YKRMRoOKcVEcHiY4YfT8VksY6dv6KCCK55ZQUGjs/ZvQzBvj5RLjnbZmX0JsgDGW8ywF1EXpojLiiE07d76tCSfpiLTRYHXdP6WK0ZQRhvv1lnU4Hwp2ywAHAf3YN1ZmzJpJdYi6ooDJIFFah5Bl0AyJr9CgzNyyueY40GZP0RBv0lsnDaYeeR7jSBFKT1dyJ7fPY6MdoGrE8C0LltL+7bZosd8uEJ75sC8P3Opq9UEIcOHAG2B5qhCHdsm8t3JRlcvGychhrvGnnhtq2C4XqqMP4+z+dpC0dqYdFKfloknSbFs8JZLR8FD7z+rHkkk1tJfYGnJ74kVy2XMoSu58zYU0GZDKTxRhnlKUfgdzw/u6dcuKPdj/VwYRav+upY1E=
X-Forefront-PRVS: 013568035E
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(24454002)(189002)(50466002)(81156014)(68736007)(86362001)(83506001)(6116002)(81166006)(66066001)(4001350100001)(189998001)(23726003)(305945005)(97736004)(92566002)(77096005)(4326007)(3846002)(2906002)(46406003)(7736002)(47776003)(7846002)(9686002)(6666003)(101416001)(97756001)(93886004)(8676002)(1076002)(38730400001)(50986999)(33656002)(42186005)(76176999)(2950100002)(54356999)(5660300001)(229853002)(6916009)(105586002)(110136003)(106356001)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:CY1PR07MB2346;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;CY1PR07MB2346;23:hnbjcVPkXvQKEnf/Bsk7F5JhslgMiP8IN4ClfMfWI?=
 =?us-ascii?Q?bTC8OR3996uvmVGzRZZ0Bb3Si0EdNsBAfP66onWkSsZ+QJ+W+UAo30jiAzxS?=
 =?us-ascii?Q?f4VoVYkqg+j4DyoNXjLUUWUoGMvHeYzMzYzMzjszpGckBrcgJXvmDMp3HmHf?=
 =?us-ascii?Q?9OQq4oRN4ipEm5la5fnVkoSwY1/ObNH/9LgJ4ChGROacnd/6VePek+BK0gaE?=
 =?us-ascii?Q?sGRaPit68RRZfC8VATt6Wu0DzO+cpvXXGxMfjAKjbzDp/uaVo6WZm1e+H+Ob?=
 =?us-ascii?Q?Y908NNEszHUOBKbiB9TnT7TS8dBY/Cd2o312AKl3Y35679fcgHPb/sLkuj0G?=
 =?us-ascii?Q?b92rlbVtt0Dqw1Gj2nAVZxva+uy168RuzvdoPwlOjd38WnoNH0SdwlGr4PRs?=
 =?us-ascii?Q?WHNUFKyAlWblFgIbXQf0rZDKGLoo+kycnBNmnNR56/VhrhNCvknC8gwynFwC?=
 =?us-ascii?Q?rgc03OmfK1Qp/m6cKURYIJCYJXoCfmarTfSLz0rNv/vNa3Gs6LolBZPY5wOD?=
 =?us-ascii?Q?2MpcM1zaRvLKfoEjFd8EASMWaeQHruoMmJ1L4L9iSuHDPZJPfJzmwU7rrItP?=
 =?us-ascii?Q?LSR7ucKXrHzreAZaKnDr/RHMXcx3grhZJWEB5YlhknDwne1Gneea7YPA46bQ?=
 =?us-ascii?Q?Gx6CwM2Lnr9tfgeGhP3UpcUgKXFfB6UHzXn7cf21y6ogt0X9A5JfDNyKNpcq?=
 =?us-ascii?Q?01Av2BAD/Ll6hRDn0+mF6pey+9KHea5t+b14ijLWiHAhxbxaRrQcsVvd9xlK?=
 =?us-ascii?Q?0kz+8w8PyAQiUrsawr05cA8PR5M3W2s0jBDmWUZr4HHi28/LPIcqqKP7AUEY?=
 =?us-ascii?Q?LrbalH3bbwpl0vKtLRJ4ivWPSFqjSPOat/V4rrXEUGkI5unPMcM9yfA1YTXJ?=
 =?us-ascii?Q?6LJ2Tnh90YqbNx3kNo/O0DDQqnRk0vrATUyHjR8wU+C7H+WsZGJqW83TCuzZ?=
 =?us-ascii?Q?Zhy05ownosVEMnPNXUJPmnoyixnLtfc2GDZKKJfwyzOx9+Edzzf2zM15YQqX?=
 =?us-ascii?Q?OHXqhDYYvTxvJXddekI2AW1RXuLjh+XIgngxc9c6sYwWwJyOG7Ne8eY4do1+?=
 =?us-ascii?Q?paxtwbguuA3HmoY8GXvr9PTykvxYU2StQPal4PWX/24diOdFrZhFgZdJ+kpP?=
 =?us-ascii?Q?3ae6rRnM57B6bV7U5jU5EOmmstxfRxDXSFMhmCCAHny2NYQWHpGFgsEBWXkK?=
 =?us-ascii?Q?i8ibosuTe765PyGMW97yQMeqDGTNoYJXzVk3WJx7MPi2vP8yAUtQFqA7Q=3D?=
 =?us-ascii?Q?=3D?=
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2346;6:6Xt+RMBONF9K1c2ko4ZtyKeQmsoJubgpFDTCywUxvEmQpleuwKClLmXp9aFyZkeaiVwjJP1xtVnBzUNo0keFaLDW2NU0vMvy9DQxFvFXofqbQO6eWD/X01w6Acv4YBjp2EfHusHP0G8rMgXTuEMQhaJaBVMeuoovtrsVhjUB7qAc4e7Y5eFNm2fjmyB0tdjz8Y+jBmTyX+h46AqS8VBtDlzZrhbJ8S5vZCxvG1bZE7G2kCt/aw9rcI2/X/T6oviuyUI9IHOYzQtYufYxrpHc4C78YuNsHsbbj1jue11plGv0dScR6QWeZ4CV8S3Z6QyGZWPZdqdUbiu7ddteNmgnu6jJNSKzmVXJ+Pp0XGtkO8k=;5:NtrRlmUoOmmY5+jHlQpc0cp8Jy+TDdNXha6euzm4AP6NSBEVRNULSzvGpVRwZfbllrQDw/E3RQD2acoVyw7goHQKjVklVPo+5vcekLmm7i6c6rXhL/AAPE9XorAVdM1WxGeuanL8olkBVALu5gxRQWLmmtFWfBI+A/jtMyGsufU=;24:pk9pTzASBRw5SnQ0uXlEz/952XPpQ88DlCkiSNBXIBLoYC6N/UdpoYWcLkqcdIwxFjoDa1UP32olaiRC3BEfXMOZpEnBnbHX8EXnSQpDeQ4=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2346;7:Gl3+nW8ZHply9sybd4ie1ymE+xUlk2lQlUJsfQpME4AGuB2jqwZ2LAMzuJxTW/mirSrrZDnqGOl+pzlWZtpic3bTYRAPmMu4SuhZwrDpq9+M1KntVJ0P/HTZ/JRYI2T3x8VK8MM4Lwu1UI5POxKB6rWsRv439z4/e/2oPe0Jk+Xu89QjNDHwy7VqlWN362pStwN00hhXfzFd5CHHWg/7oO7+C9vMlb7KLTwNLkomXILMgr4zQEKP+tUkMjyKpxRyy+20a4vlpSKdXnPA21wyA92X6GxU4pBdRbvH3BCZCF3P8yMptkwNu5mdFgIOlKyK+179klLugj2DXCqf6vGs84ijbp2Q3UNnuqVSv866Lwo=
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Nov 2016 21:15:49.0993 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2346
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 20.11.16 17:07:44, Ard Biesheuvel wrote:
> On 17 November 2016 at 15:18, Robert Richter <robert.richter@cavium.com> wrote:

> > The risk of breaking something with my patch is small and limited only
> > to the mapping of efi reserved regions (which is the state of 4.4). If
> > something breaks anyway it can easily be fixed by adding more checks
> > to pfn_valid() as suggested above.
> >
> 
> As I noted before, it looks to me like setting CONFIG_HOLES_IN_ZONE is
> the correct way to address this. However, doing that does uncover a
> bug in move_freepages() where the VM_BUG_ON_PAGE() dereferences struct
> page fields before the pfn_valid_within() check, so it seems those
> need to be switched around.
> 
> Robert, you mentioned that CONFIG_HOLES_IN_ZONE seems inappropriate
> for sparsemem. Care to elaborate why?

HOLES_IN_ZONE is of rare use in the kernel. I think it was introduced
to save memory for the memmap and only some single systems enable it.
There is no architecture that enables it entirely. For good reasons...

It introduces additional checks. pfn_valid() is usually checked only
once for the whole memmap. There are a number of checks enabled, just
grep for pfn_valid_within. This will increase the number of
pfn_valid() calls by a factor of MAX_ORDER_NR_PAGES, in my config this
is 8k. So, this is not the direction to go.

My patch fixes a regression in the kernel that was introduced by the
nomap implementation. Some systems can not boot anymore, beside of
that the BUG_ON() may occur any time depending only on physical page
access, we need to fix 4.9. Here is a reproducer:

 https://patchwork.kernel.org/patch/9407677/

My patch also does not break memremap(). With my patch applied
try_ram_remap() would return a linear addr for nomap regions. But this
is only called if WB is explicitly requested, so it should not happen.
If you think pfn_valid() is wrong here, I am happy to send a patch
that fixes this by using page_is_ram(). In any case, the worst case
that may happen is to behave the same as v4.4, we might fix then the
wrong use of pfn_valid() where it is not correctly used to check for
ram.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S965638AbcKXNof (ORCPT <rfc822;w@1wt.eu>);
        Thu, 24 Nov 2016 08:44:35 -0500
Received: from mail-io0-f178.google.com ([209.85.223.178]:33798 "EHLO
        mail-io0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S965293AbcKXNod (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 24 Nov 2016 08:44:33 -0500
MIME-Version: 1.0
In-Reply-To: <20161124134238.GI10776@rric.localdomain>
References: <20161017185801.GT25086@rric.localdomain> <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain> <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain> <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain> <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain> <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Thu, 24 Nov 2016 13:44:31 +0000
Message-ID: <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> Why? MEMREMAP_WB is used often, among other things for mapping
>> firmware tables, which are marked as NOMAP, so in these cases, the
>> linear address is not mapped.
>
> If fw tables are mapped wb, that is wrong and needs a separate fix.
>

Why is that wrong?

>> > If you think pfn_valid() is wrong here, I am happy to send a patch
>> > that fixes this by using page_is_ram(). In any case, the worst case
>> > that may happen is to behave the same as v4.4, we might fix then the
>> > wrong use of pfn_valid() where it is not correctly used to check for
>> > ram.
>> >
>>
>> page_is_ram() uses string comparisons to look for regions called
>> 'System RAM'. Is that something we can tolerate for each pfn_valid()
>> calll?
>>
>> Perhaps the solution is to reimplement page_is_ram() for arm64 using
>> memblock_is_memory() instead, But that still means we need to modify
>> the generic memremap() code first to switch to it before changing the
>> arm64 implementation of pfn_valid
>
> No, that's not the solution. pfn_valid() should just check if there is
> a valid struct page, as other archs do. And if there is a mis-use of
> pfn_valid() to check for ram, only that calls should be fixed to use
> page_is_ram(), however this is implemented, or something appropriate.
> But I don't see any problematic code, and if so, I will fix that.
>

memremap() uses pfn_valid() to decide whether some address is covered
by the linear mapping. If we correct pfn_valid() to adhere to your
definition, we will need to fix memremap() first in any case.

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S965586AbcKXN6K (ORCPT <rfc822;w@1wt.eu>);
        Thu, 24 Nov 2016 08:58:10 -0500
Received: from mail-by2nam03on0067.outbound.protection.outlook.com ([104.47.42.67]:43040
        "EHLO NAM03-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S965266AbcKXN6G (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 24 Nov 2016 08:58:06 -0500
X-Greylist: delayed 58152 seconds by postgrey-1.27 at vger.kernel.org; Thu, 24 Nov 2016 08:58:06 EST
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Thu, 24 Nov 2016 14:42:38 +0100
From: Robert Richter <robert.richter@cavium.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
CC: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161124134238.GI10776@rric.localdomain>
References: <20161017185801.GT25086@rric.localdomain>
 <20161027160136.GD24290@arm.com>
 <20161028091905.GM22012@rric.localdomain>
 <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.88.216]
X-ClientProxiedBy: HE1PR0801CA0012.eurprd08.prod.outlook.com (10.167.184.22)
 To BL2PR07MB2339.namprd07.prod.outlook.com (10.167.101.17)
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2339;2:hhkTM7onvSC02YxVP2H/RFSMw+kRmnqaRRHD8LOOLgPB5Kp8z4oFsv9vJ3WsDmK6ERcEyCtxAqA05BTiUm3a5ZHZHvmTT0K0zVtYXShbKoq0JYmGIi6hcSbgf4Sg7ajHLBgP/jQC9X6B+LJC/5viUv4ddH5gBw3TTZ0Ge0nTY78=;3:UsPG2VF7lEmqcq58NJ2/DVkeEEAK9ux9sYIKTCahx2DiLR3x0qT1q5kBGhQ4/d6dIV/l0Fe9wWubKFxMqDEGymh2ij0gwTh5/Bab1nS5jfS7kssoE6RAlrEL7FuqfFCnwNSXlHXgN/RxBc7H/8kOhq1/osdWTmd4y5auHCr/Kao=
X-MS-Office365-Filtering-Correlation-Id: 73e079be-5cfc-44e2-cf83-08d4146fcbce
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:BL2PR07MB2339;
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2339;25: 
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2339;31:68kYBhZUvscvRatNNvDp1o+Ej3wuu1Hty9nSnnK6+ZLSpE/86FgQenq7uaM2tYa2XSWkGzFmEQ23M8ay6DpARW4SSaXkbAE0mZ3pHHCPrxN9ifuJacUH9r0kzCjE7OBQS1yRgapxtVr7E3sQFB5/VmrH+YdfOm3cVLQvB7P2Wf2R8kM7PMQhnI5grohqMQVg0y2ns63sCGCppJnZigeL6qgH/SMSMVDRZgFY78rWw83UP/AqZ4zYFtOYL6sRVmC6fKI908tPUyiUslR+4oOzSQ==;20: 
X-Microsoft-Antispam-PRVS: <BL2PR07MB23397C23E7368C6D9619AB3CF1B60@BL2PR07MB2339.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6060326)(6045199)(6040361)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6041248)(6061324)(20161123564025)(20161123562025)(20161123555025)(20161123558021)(20161123560025);SRVR:BL2PR07MB2339;BCL:0;PCL:0;RULEID:;SRVR:BL2PR07MB2339;
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2339;4: 
X-Forefront-PRVS: 0136C1DDA4
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(24454002)(189002)(8676002)(76176999)(54356999)(77096005)(97736004)(189998001)(97756001)(305945005)(33656002)(7846002)(101416001)(47776003)(3846002)(6116002)(66066001)(39390400001)(39400400001)(7736002)(1076002)(81156014)(38730400001)(2950100002)(4001350100001)(6916009)(81166006)(23726003)(50466002)(93886004)(50986999)(86362001)(6666003)(110136003)(9686002)(106356001)(39410400001)(92566002)(39380400001)(105586002)(42186005)(68736007)(83506001)(2906002)(5660300001)(46406003)(229853002)(4326007)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:BL2PR07MB2339;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BL2PR07MB2339;23:2uieBd1RXehjgpV9lstZOMRGbFHMYIVgMxn+wL5Uz?=
 =?us-ascii?Q?w1UTokILxB9t9K7pRYI11yF3PFwhUpCFNWaEwD1m/Zf3SFVSpdBIu50SwhPb?=
 =?us-ascii?Q?ABnTgu1p3D4pkcU+6tBNOIE1/y5PlFqOdu2t5sjprqq7Y1f5ryF7CSPp//Ok?=
 =?us-ascii?Q?QneuSk1huFn0Yd+dCLLZye1g4TRuSLEjqqDHtSQbDGjGMU7D4iBOzZynTD1a?=
 =?us-ascii?Q?t8joZqMTzJRSgqngt2thUpr4pq6dX6PksM5kZbPYFVmeDB5J3OswSSc42BNf?=
 =?us-ascii?Q?cXAacW2nIrD1xUW/Nv5ImamEeJ9qAMvw6p3IL9xblWh0JuDfCKgSYyaqBorv?=
 =?us-ascii?Q?heqT/693TwHGvDeDdD3o8wACdpdrhOjiOeeaSEEjdPsy6Wib+r7suvgw47lw?=
 =?us-ascii?Q?OOImr8dYYpBlx9I5Hujyv8PjoNG9XebmgAd9LP7E3m/wSV/9xNwRBT8nBZCj?=
 =?us-ascii?Q?L8yk7smsdV23v4pdj3v2uVyTIpQ3J9Jf/Tawyzh1EL522gUy16RYqWVvoUAm?=
 =?us-ascii?Q?NPKgz14vxSahUHkw0ttU8HUuEwCNhdKcJly7UmjqYM3V9yiUTZU7pc6XtGUa?=
 =?us-ascii?Q?60MIB3yzwmiQo8lMUA4zKwMFyO5mDqX2wEmzgnafFXiM03a9/54G3QwtSvCH?=
 =?us-ascii?Q?fvCftDHV7uUayI7R3oqaYyhEJ46i2Ji+3E43Nw50PkvVEu6Hqr5b92byzL4/?=
 =?us-ascii?Q?7Kyit7DPjgFPGbJ7Cc9fagyePtRNubvnBzJJxQPhKDhF3i/z+QQ3zjORlups?=
 =?us-ascii?Q?b5NU+KTUJ5L9BsFBnRljemnp0vPHNaTRjn8eOFDHTuaHyQ/01ruf/fs3NVyZ?=
 =?us-ascii?Q?brG6uvn1xc0DHzCxjQjDnkVWcnoiwOz0uSP8mtUmx6v9do5BUOK0lWAL+Cit?=
 =?us-ascii?Q?zcBHbDpuWewQ2wkcOilQ0TeyIIvxIG8t83c9Sy/mLXHdkWGU72RrzqZbvjzR?=
 =?us-ascii?Q?5dPYHKQloClUshtIoEVm5IBQ22FdjuOq7F0XdHHCF1Lulj0+6rtzCkqnk8HS?=
 =?us-ascii?Q?KIC/DigyhVpz/jlRqSZV1VI9jPVxUDLepuIgsk74qURwu6GfJigqf6Vfocm0?=
 =?us-ascii?Q?W9eVCnU+kIwLU02CKnngfvUpiPN42rugCMykj2upv/NlcOwrQfPtFSIHtvPo?=
 =?us-ascii?Q?cGmRpQi9TO/qv0SFUvaMSGpwQdgfJRjzvucKDw6I4mmwjabXLokFFZ/HM7VG?=
 =?us-ascii?Q?0cOOxLQVcyd5Gu/h2W8yiNTYHtLsB4+LIcSV5NPmblivGTeflEuPu17WinjO?=
 =?us-ascii?Q?k2c5RJEk+GmrrYPXzQ0WvvCae1PSuJSn+hgJ1esnODOKuz45NkluaeAi7GT1?=
 =?us-ascii?Q?FgUph/K+cVQWC4X9wEdjQvzmlGNIWYox2xeWbj6DT4+?=
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2339;6:5UJnYnYs0EDXQ+CNYmQtGfzy5NhIGLFsKJ7T6wzPYGvJddsdIRJhETPLDBmgG4uqD2rBawvuh8KkcFH7gSXYSwb+Q/I0zDFkqnHlH6CjbhvdnZTjgSEgDNQzLqwNcTTOYFpFd71qrS9wNCLVa4wOOSHA7Zp6SNLADaIYt4Qtxt1fOXFJVozZ8awmcaG1C094L6RwTKfueGi5urjcaH+sNSFgtotl1d9bABMwUXdTAn2MyDGXNI9jAvTCkGeiJt9cdd8A+nS2+noHfjOywawna1PWzVyAYGkbZamatdgJBZsZcnnyzxhETTMS8QsLQj9iz2kS+bD+zPzCObgpNRGQhBLrZIiMlpkRfTAhKuCCQCs=;5:J1ReoJUF3YKod7wodNGHDDHyFBpK8UiPlIPsJ8mxbPC9irK8/oTOiw1w/uTPQGqOWvqZKB7v/Ubz4XkQiM+LSZcsSIyFRuVwCiDzbz+T4AsBgeyTZGu9/b/WMmV4gTf1WpgTdFKJ1LdgaMpr3B8GPeIUhZAZfiP5xwD8e/iio2o=;24:MqBGsJqvWgf0sOvgdVtu1PG8Ju8q2qR3TRCxF0UCHVaUr4JZOVX69lTBy7X/3KPPgwLGPUIByRREl8MNs5V55RaK+BEU5n8RHciPqYTG+B4=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2339;7:kWReBnExyp2srGbsCmCO+9ppR1gZ/CSVfLePVim8mIN4N3sMusJi9iIgkOh9s34ibrn53gzxlkEolNhI1T5kAEIRlFHDLgErx5mhAgTwnHBIHkwRcHAQnmzjLg274LauuW4bDlPx4FwLlCpuVIZ5cUWO/4N8Mz42zjAqjshJq2KvrNVWGEXraOv/Iqcw7gG2X/zRVjbWKqDG3fY8yYwpNa6k7Mrk3aZLbf8SycIdDr1kfH7NH48k5B8Bsa9fEjRsh5GuWvk3Qani6XMvw12SEjBgEH7ku/qUA1v3ufFN2qNG3eIDeRvOa+OG8kpQbXrydKVh/xQTilE+I8HbSF7IAi0rwALtX7l/FMe0VN74so0=
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2016 13:42:53.8459 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR07MB2339
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> Why? MEMREMAP_WB is used often, among other things for mapping
> firmware tables, which are marked as NOMAP, so in these cases, the
> linear address is not mapped.

If fw tables are mapped wb, that is wrong and needs a separate fix.

> > If you think pfn_valid() is wrong here, I am happy to send a patch
> > that fixes this by using page_is_ram(). In any case, the worst case
> > that may happen is to behave the same as v4.4, we might fix then the
> > wrong use of pfn_valid() where it is not correctly used to check for
> > ram.
> >
> 
> page_is_ram() uses string comparisons to look for regions called
> 'System RAM'. Is that something we can tolerate for each pfn_valid()
> calll?
> 
> Perhaps the solution is to reimplement page_is_ram() for arm64 using
> memblock_is_memory() instead, But that still means we need to modify
> the generic memremap() code first to switch to it before changing the
> arm64 implementation of pfn_valid

No, that's not the solution. pfn_valid() should just check if there is
a valid struct page, as other archs do. And if there is a mis-use of
pfn_valid() to check for ram, only that calls should be fixed to use
page_is_ram(), however this is implemented, or something appropriate.
But I don't see any problematic code, and if so, I will fix that.

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S965642AbcKXN6f (ORCPT <rfc822;w@1wt.eu>);
        Thu, 24 Nov 2016 08:58:35 -0500
Received: from mail-io0-f181.google.com ([209.85.223.181]:34806 "EHLO
        mail-io0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S965600AbcKXN6c (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 24 Nov 2016 08:58:32 -0500
MIME-Version: 1.0
In-Reply-To: <20161124135151.GJ10776@rric.localdomain>
References: <20161028091905.GM22012@rric.localdomain> <20161107210514.GP20591@arm.com>
 <20161109195132.GZ22012@rric.localdomain> <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain> <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain> <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain> <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Thu, 24 Nov 2016 13:58:30 +0000
Message-ID: <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
>> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> >> Why? MEMREMAP_WB is used often, among other things for mapping
>> >> firmware tables, which are marked as NOMAP, so in these cases, the
>> >> linear address is not mapped.
>> >
>> > If fw tables are mapped wb, that is wrong and needs a separate fix.
>> >
>>
>> Why is that wrong?
>
> The whole issue with mapping acpi tables is not marking them cachable,
> what wb does.

What 'issue'?

> Otherwise we could just use linear mapping for those mem
> ranges.
>

Regions containing firmware tables are owned by the firmware, and it
is the firmware that tells us which memory attributes we are allowed
to use. If those attributes include WB, it is perfectly legal to use a
cacheable mapping. That does *not* mean they should be covered by the
linear mapping. The linear mapping is read-write-non-exec, for
instance, and we may prefer to use a read-only mapping and/or
executable mapping.

>> >> > If you think pfn_valid() is wrong here, I am happy to send a patch
>> >> > that fixes this by using page_is_ram(). In any case, the worst case
>> >> > that may happen is to behave the same as v4.4, we might fix then the
>> >> > wrong use of pfn_valid() where it is not correctly used to check for
>> >> > ram.
>> >> >
>> >>
>> >> page_is_ram() uses string comparisons to look for regions called
>> >> 'System RAM'. Is that something we can tolerate for each pfn_valid()
>> >> calll?
>> >>
>> >> Perhaps the solution is to reimplement page_is_ram() for arm64 using
>> >> memblock_is_memory() instead, But that still means we need to modify
>> >> the generic memremap() code first to switch to it before changing the
>> >> arm64 implementation of pfn_valid
>> >
>> > No, that's not the solution. pfn_valid() should just check if there is
>> > a valid struct page, as other archs do. And if there is a mis-use of
>> > pfn_valid() to check for ram, only that calls should be fixed to use
>> > page_is_ram(), however this is implemented, or something appropriate.
>> > But I don't see any problematic code, and if so, I will fix that.
>> >
>>
>> memremap() uses pfn_valid() to decide whether some address is covered
>> by the linear mapping. If we correct pfn_valid() to adhere to your
>> definition, we will need to fix memremap() first in any case.
>
> As said, will fix that if needed. But I think the caller is wrong
> then.
>

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S965835AbcKXO2f (ORCPT <rfc822;w@1wt.eu>);
        Thu, 24 Nov 2016 09:28:35 -0500
Received: from mail-by2nam03on0086.outbound.protection.outlook.com ([104.47.42.86]:11916
        "EHLO NAM03-BY2-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S965556AbcKXO2T (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 24 Nov 2016 09:28:19 -0500
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Thu, 24 Nov 2016 15:11:49 +0100
From: Robert Richter <robert.richter@cavium.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
CC: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161124141149.GE2213@rric.localdomain>
References: <20161109195132.GZ22012@rric.localdomain>
 <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain>
 <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.88.216]
X-ClientProxiedBy: HE1PR0101CA0024.eurprd01.prod.exchangelabs.com
 (10.168.141.162) To CY1PR07MB2347.namprd07.prod.outlook.com (10.166.194.146)
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2347;2:hnkeurLt5WMc3pkZPEgL2NwL3rYwwcWAUF+92o5tAb1xwu3kw2fSg0tiNseDIX8nrUB28Fd8SK2Z5m+aMTZ3m2ATgpMtzP1dm1x7ltDMN+Ds5UQksaD5bnV3gBIjZqvg8RNgZrHMBqW6qI1r3jf+XULD1IKFv0mVm5uGAPoiQ+w=;3:UcObEP5JmYu42/2RFiv/UsLTNvX96HprpXrtIEpP8vNl9aE0zbsSV79BHcGQteicvYnjJ/PBdLnYhOdP5ZKFFoSXinOTJCd3zhA8SHNn3RXDqjAUDGE1xaYIh714UEW196PvtRsVPIIWZi1cxSLwSpuMLb/XpaLwurx6qhohXJ0=;25:Wr+HXWFdhoC3qg3WB5CwUmZhqvglkGO4vDnMAltAPVf3N4FXtkxn+2gq+KN7jobPRLKwGtsKU9fxo/LsFgsd/Gi3cTI+04UW52V1cw2H1qrFapekFj3janGsWdo6GyAbrosXug0SOA0ZVVMKHvZydW7VSH8fH5lJ22kp5WqEcHoa394i6P8sgTHYmnbx6DbFnktcvUevOhFLmr2jxxx7/YwYJIfT80XKDEMGDHFjcbgO+Al/SyZU6Wq9qGqriQw9ivfpXJZu+G7QGhR+nTgpoYaTiifloOPLrLJNuYaAnHQpnVJGr4c8w2VZhBxeZSA8Vg/n3R05is/wfrvPQXIr1ntB/xvlferv3lcvuV0D6WVf2ISY+pho+NkSShimyNE45Q2KxSr5eb0houlev5n40e+sxGRQTmX9VtWpCK5yFf/sCl6XgJxthfyD97us4ID0xBfW8p1X48m/dnt/3ER09w==
X-MS-Office365-Filtering-Correlation-Id: f5cd493a-57ab-4f4a-05e7-08d41473e02f
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:CY1PR07MB2347;
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2347;31:3GJ2NKq4tvO1iorhqybbyaslSaYuvAejfzt6AByvWPt8yIYsPCcYmjmNu+qtO8aRXeaNQi1M4xxne64YKpLM6Mba1wl1Ifk0WS+6AGvwlrZW7eJXuYpaLhiBd+99nit+6n9eXiVi6XCxKwNXv/zsf1bvIyvfd/rxmGsevot2bf/+veG/S4nPoHb2gRJD5A+Q97blp3gFEvhlTFJRMqiaMGKUanY07tLkb1eshR/4pF8UkPR59BMOEGSu4GlrOzdY82DBUHRqb2AaqimK1vbRKQ==;20: 
X-Microsoft-Antispam-PRVS: <CY1PR07MB234798766400F33CD7223241F1B60@CY1PR07MB2347.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6045199)(6060326)(6040361)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6061324)(6041248)(20161123555025)(20161123564025)(20161123560025)(20161123562025);SRVR:CY1PR07MB2347;BCL:0;PCL:0;RULEID:;SRVR:CY1PR07MB2347;
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2347;4: 
X-Forefront-PRVS: 0136C1DDA4
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(24454002)(189002)(199003)(39380400001)(39410400001)(106356001)(23726003)(4001350100001)(105586002)(47776003)(8676002)(66066001)(1076002)(33656002)(38730400001)(81166006)(110136003)(189998001)(97736004)(97756001)(81156014)(2906002)(5660300001)(42186005)(4326007)(9686002)(101416001)(93886004)(92566002)(50466002)(83506001)(305945005)(2950100002)(6666003)(76176999)(229853002)(6116002)(54356999)(7846002)(46406003)(50986999)(77096005)(6916009)(86362001)(68736007)(3846002)(7736002)(39400400001)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:CY1PR07MB2347;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;CY1PR07MB2347;23:CLnpdUqMgMT/bb5NQkdke4Wmsp0sy7dfu7R1cPXNf?=
 =?us-ascii?Q?cS0r0ZNo29noqM5F9d0e3czvqeG0fUsVxxPIHpBS0syIV/8DYS0se2tmqq2Y?=
 =?us-ascii?Q?uJppg5lY4QtG3W2suQAtT264PIdbU8BBrI5WtsY9doNovNJc30GD4q8KdqRE?=
 =?us-ascii?Q?TPUWKK4KBafuTdHuWQy19PIMK2E/ygk7UvgOpSl2nke57J0AY6+zWdaV7eBt?=
 =?us-ascii?Q?2RTBl5B9dnvqGQdPQe1aV0A1WIWnIt2RIHcX8CSKIc2+HdVUd6u2apXDBuh1?=
 =?us-ascii?Q?DukBFcUudX7U2WJiHOzC294mejfd2KIwQ6xh86B7PltW6UWLOFFu0U7DRr7M?=
 =?us-ascii?Q?TVGeHX9LN97cHm/pvprDfctyoz8QlnSCdpofNgQcqSt+eDiRY2vL6qdE2UnX?=
 =?us-ascii?Q?xTdvmJ0UlevPPFdGYY+/pRpdLFHxmI1fkzRD29PPlGwPmk1RDMQ1vso7PnHt?=
 =?us-ascii?Q?zMV43a1RNC/TKXa3rhC8vWfmtaijSM+atJELVoePuYPnzGG2wp6ZsLLJErsN?=
 =?us-ascii?Q?AYfyF4KKx9l5fvQoPJX49xXGR4bjHDqpC4mo19UOuvIdDFLQVTzUXdBz1/NN?=
 =?us-ascii?Q?HIOP7UJ91UG3yelntnLDCVqplI4FBkzdwcfLeACuv/kMW19TBcIFxm0OTl0N?=
 =?us-ascii?Q?AjhphQebpsZjqKzR9RaVzvm82RRUSdf7gxXvcqHKpwjHfoGLkcuFhmcvR2BK?=
 =?us-ascii?Q?ZHU7UKiv39VWCT/3ooAA9LYzABtDcCPlR4HFl2WUXre3ClwV50cEGbDs6FjV?=
 =?us-ascii?Q?YhseYJ2a7o9ByFTJ4iwjW5OBADsJUBvZIHWIfkb+DvpHWRY/NRNSsYOiIT8+?=
 =?us-ascii?Q?8h/BSmDLen9dtoijLOh2YLK59JJ2XpqZMxlyG8XJdlmkBq4kNCSrxmhD11Kx?=
 =?us-ascii?Q?b9nHBAXses3r283Ifbuy/nj5a/8s4NJOeEt07UHJe8dF6aDtkefiDFJwDsd2?=
 =?us-ascii?Q?/9RmoLGxaLdSwxmBFwHWHoloH5P+AA9MrT85+tQIc51hwYnPKpbQfCmA1Wu8?=
 =?us-ascii?Q?NUXhi1IPKVdcwIi+EtrPCgDGdNXQvdy1/v6pb5AEmYLP+LQAGco5suJJIKzW?=
 =?us-ascii?Q?2FweLRLZW3H6o7AUlqfiJY9MdGLPrOds6ZBFJDUJq88Mx9YkBfw3FomzD1ZO?=
 =?us-ascii?Q?pEAL6viz64Ah05pAnFngdF03ctAfeBHfP0T3MADcZ+r7ixTEtadiD+KLv/tG?=
 =?us-ascii?Q?VKgngahFogc6R9KD8m/cSgvIW8MH7/+hFTBzSyygSVKlkriRVs7d5uMXcCFV?=
 =?us-ascii?Q?ziBTBFmRaReoML1kZMklqLhHvK/iLNZRru2K8x7/e8/OAGTc6o6bshw11OQh?=
 =?us-ascii?B?Zz09?=
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2347;6:fRqdbCzsuse16jFg4ZQyvMVR1s146AkSTBWdNkpR6HT1tvgpOkPK44KZiCns9Gp8s9hpLAGYXhi6KcEdvBSrdZ6PShw90SfVUruh4+8YABr+dp79DJ1QWpWNhVJ32psfX6pwvRnFh9SXuJzzxIi0+QTrcGedv57GJdfacMNjkDqPNM7SXDT4rqW73UyAcKUggQSJHJB/ZxwANL5vmtPwIpfwvYEGGNiuxwcZpf5bmGXsPD/M1rj7X/fFuT0kreSvNB4W9RuYH1/SS6ma+FhOiZzqcmSPFI1haeNhRycZ0ENoYptUkTiClK4NkZbN72jzElsVtABCi/rDuK1DxlrXO9ON99RwowL7k8UGgTnez1E=;5:pK7jGp2hnCJ3bVg6mMw3Q5fQrCrDufON3inoJa0FVNE8qxNeoyqPIOrcwcZIYGUbDN1i7s+KQ9iTnGkkc9XIMX4aD0CnJlvjwnDQjU4Tij9Lt2LjSXRaVHblbPhUXD5Cftioi7gTjCSOG8QyaprKiA==;24:NvnuA1/L7me+sGMTdtS4OkwKgJFITDvb/v4PJCqpa26F8EaYvXuD218Wwz8VxxVtI59r0LNSuvDX3aDBi9UqGuWepiD0KV05QkKwDj6oTQw=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1;CY1PR07MB2347;7:RG7CR7CTizNr7TEwLUKvOCb7aNO+JXizxUp8q2brxoZF9zDBAdSQdA3nxgRl+pM37gdbb4xJq/0frpMc15/MSFnvZzw5regLsSlMmDr4PRiHGS4xMgPRxlF4QlEjpYEQ0gCLwMQh0/UnePcrmq8ZU2IUhSQsMzm5dX8WRurhn26feccAVqTFg+VPKkzFXsM/CwlayCrFVsKzj/NhhvyR7QHM2/cwtpRYGBiAEVlLuhCYK/TAf91R8GiM54pbOnd/8Aj3cw3Dgs8f/9dMIMaSrImClQKoAoYTW3gyLKmZ6E5OTDd6QoHUgJh79/IGoKItKgt/tX++NlCFjBFrFfYCsNWpfBeKJayJg8LaIWY423A=
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2016 14:12:05.4693 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2347
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 24.11.16 13:58:30, Ard Biesheuvel wrote:
> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
> >> >> linear address is not mapped.
> >> >
> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
> >> >
> >>
> >> Why is that wrong?
> >
> > The whole issue with mapping acpi tables is not marking them cachable,
> > what wb does.
> 
> What 'issue'?
> 
> > Otherwise we could just use linear mapping for those mem
> > ranges.
> >
> 
> Regions containing firmware tables are owned by the firmware, and it
> is the firmware that tells us which memory attributes we are allowed
> to use. If those attributes include WB, it is perfectly legal to use a
> cacheable mapping. That does *not* mean they should be covered by the
> linear mapping. The linear mapping is read-write-non-exec, for
> instance, and we may prefer to use a read-only mapping and/or
> executable mapping.

Ok, I am going to fix try_ram_remap().

Are there other concerns with this patch?

Thanks,

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1757033AbcKXObd (ORCPT <rfc822;w@1wt.eu>);
        Thu, 24 Nov 2016 09:31:33 -0500
Received: from mail-io0-f171.google.com ([209.85.223.171]:33035 "EHLO
        mail-io0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1756801AbcKXObb (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 24 Nov 2016 09:31:31 -0500
MIME-Version: 1.0
In-Reply-To: <20161124141149.GE2213@rric.localdomain>
References: <20161109195132.GZ22012@rric.localdomain> <20161117142528.GJ22855@arm.com>
 <20161117151805.GJ2151@rric.localdomain> <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain> <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain> <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain> <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Thu, 24 Nov 2016 14:23:16 +0000
Message-ID: <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 24 November 2016 at 14:11, Robert Richter <robert.richter@cavium.com> wrote:
> On 24.11.16 13:58:30, Ard Biesheuvel wrote:
>> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
>> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:
>> >> On 24 November 2016 at 13:42, Robert Richter <robert.richter@cavium.com> wrote:
>> >> > On 23.11.16 21:25:06, Ard Biesheuvel wrote:
>> >> >> Why? MEMREMAP_WB is used often, among other things for mapping
>> >> >> firmware tables, which are marked as NOMAP, so in these cases, the
>> >> >> linear address is not mapped.
>> >> >
>> >> > If fw tables are mapped wb, that is wrong and needs a separate fix.
>> >> >
>> >>
>> >> Why is that wrong?
>> >
>> > The whole issue with mapping acpi tables is not marking them cachable,
>> > what wb does.
>>
>> What 'issue'?
>>
>> > Otherwise we could just use linear mapping for those mem
>> > ranges.
>> >
>>
>> Regions containing firmware tables are owned by the firmware, and it
>> is the firmware that tells us which memory attributes we are allowed
>> to use. If those attributes include WB, it is perfectly legal to use a
>> cacheable mapping. That does *not* mean they should be covered by the
>> linear mapping. The linear mapping is read-write-non-exec, for
>> instance, and we may prefer to use a read-only mapping and/or
>> executable mapping.
>
> Ok, I am going to fix try_ram_remap().
>

Thanks. Could you also add an arm64 version of page_is_ram() that uses
memblock_is_memory() while you're at it? I think using memblock
directly in try_ram_remap() may not be the best approach

> Are there other concerns with this patch?
>

I think we all agree that pfn_valid() should return whether a pfn has
a struct page associated with it, the debate is about whether it makes
sense to allocate struct pages for memory that the kernel does not
own. But given that it does not really hurt to do so for small holes,
I think your suggestion makes sense.

Should we be doing anything more to ensure that those pages are not
dereferenced inadvertently? Is there a page flag we should be setting?

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S941259AbcKXUBk (ORCPT <rfc822;w@1wt.eu>);
        Thu, 24 Nov 2016 15:01:40 -0500
Received: from mail-dm3nam03on0079.outbound.protection.outlook.com ([104.47.41.79]:38016
        "EHLO NAM03-DM3-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S934137AbcKXUAx (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 24 Nov 2016 15:00:53 -0500
X-Greylist: delayed 1046 seconds by postgrey-1.27 at vger.kernel.org; Thu, 24 Nov 2016 15:00:53 EST
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Thu, 24 Nov 2016 20:26:59 +0100
From: Robert Richter <robert.richter@cavium.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
CC: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        "Mark Rutland" <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161124192659.GH2213@rric.localdomain>
References: <CAKv+Gu-2uGJGovqQL-gbB54u=fQ9L9YUbDdRo6u3oWqF6znn6Q@mail.gmail.com>
 <20161123211538.GH10776@rric.localdomain>
 <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20161124150918.GF2213@rric.localdomain>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.88.216]
X-ClientProxiedBy: VI1PR07CA0096.eurprd07.prod.outlook.com (10.165.229.150) To
 SN1PR07MB2349.namprd07.prod.outlook.com (10.169.127.15)
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2349;2:dUNDIVg/ojDU045wzqRIiHi4wtmzOcOKY1rGicME6KU+n3H7FYyVLmzq63eWZbGN54JTNxkbq5ZwNVt2Kjq3i0mOcQMet35wumXBH7X1QehY8VKSh1SzmTAEv+2p2QXvN7CBwbxCyHlUOXn1d9xIx/ZD5fgZut9X0/j2DFP0fI8=;3:yfrSp3KpgNdZLnaRxBS0jgftlO0SOqX+FoeEn73qXQnZIpjtfp3D+aRLyapkjlUG/dDD1TFfy0JqIuRYf2D4DT8Ka0UgImLQtV5ET6feqZvYyA8uf9I985HykQtOaAHleDgeC3cnMxUluE44k8C8Ydqtb3HiGEdFnc1uCoqZu9M=;25:bq1zc8ZdxoikRUIi0SFIIH2BnyDZTBeEETUS/KHB7yhkGoUpo4i3SlQ0XmwW9FaeHicNqTIL2qc+UUYlwPGmAsudE5nM5eNDGGp0aInuyKM6/vG7tRjFUXWyaPCMm4peOqujxDkzmAqalBvNRMFPUgGlzyxkb69hXKExlr9l/7YeDSRJvBPebFTacN6+BQZY6OjtD8DAaPW0wEKIpwZEfJpTGSQxjLhhPCDCytZUr6oHMAJnje8cvzmxoohERFsHNquMZ6U8Rko+2YkeLqKvP44KbwfxzP9h17/EX2AWenFNkDFzuslzHWG8QlofTAXh6Y5MiA/u8nyEcVmmyffy8qMSLqBM4JEAynkP1pHe9VOCt0oGWnzXYi9SwCkT4VRAnWWJ05YUUiVmqdnOPRDyYypyjHi9gW+YXwDaBRKUleA//7q4LzMBrGnxS/fHYIfcV1tfUiWrrWydQRqh0LlVxw==
X-MS-Office365-Filtering-Correlation-Id: 3ee22989-333f-4d08-a533-08d4149fe5e6
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:SN1PR07MB2349;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2349;31:Hvwf+HtquRTsFO4ErxbGLQOH2/Yw74RLEYIbr2HhnujjvjgCqVYjvnmbN9KJhZprASJuplUpsRFTerrro/hsU0IWF59tEZrDF1f+BA7KFBWTPNIcs/kURubAAlsO4pYlKbI8kvt1uixc663vJk/yvcnsiPlSgLYL6jtuC3DqbxtGXgR4pF/vtiXQyWsyjRCtVcSZYjMyHa5tLOpaBopHXdRZUyrOKv0QsGR5tH64RbgkBvfLnMHyvDkh1dXjh/C5ywtozQ4JD9KORjtDvufdEnYjBDDTottl3/glKWsIUis=;20: 
X-Microsoft-Antispam-PRVS: <SN1PR07MB2349BC7FE7D4B83EACD15EF0F1B60@SN1PR07MB2349.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6060326)(6045199)(6040361)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6061324)(6041248)(20161123560025)(20161123555025)(20161123562025)(20161123564025)(6072148);SRVR:SN1PR07MB2349;BCL:0;PCL:0;RULEID:;SRVR:SN1PR07MB2349;
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2349;4: 
X-Forefront-PRVS: 0136C1DDA4
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(24454002)(189002)(199003)(76104003)(2906002)(68736007)(5660300001)(93886004)(47776003)(305945005)(66066001)(4326007)(38730400001)(97756001)(23726003)(3846002)(6116002)(46406003)(33656002)(50986999)(2950100002)(6916009)(54356999)(110136003)(76176999)(101416001)(92566002)(7736002)(7846002)(229853002)(6666003)(9686002)(39400400001)(106356001)(189998001)(4001350100001)(97736004)(83506001)(77096005)(42186005)(105586002)(8676002)(50466002)(81166006)(86362001)(1076002)(39380400001)(39410400001)(81156014)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR07MB2349;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;SN1PR07MB2349;23:8jC6N5+lWk4dwxkd5P6LBvdivdZx098nmQ4TxRDtJ?=
 =?us-ascii?Q?onVHzQf3iVbRB1LkeTwXte0Z3M9vREYTuBDVVjozBZtldLDDuTdVV71ur1Ge?=
 =?us-ascii?Q?+9TNUJ6ZyLM7SO3WUMBvuC7q45hVYfjTK6NU6RYllGxzVSB9EGKcGMpAEDfX?=
 =?us-ascii?Q?19QQXNX/Y4ePIRPBX6228x+y4/rkw2axU+XDdOF1ATXdqDyA4v63txR03JXg?=
 =?us-ascii?Q?XfP9Cg36djlu5w+GHs/5qs74kJqO9HwYjEdlYFhqU3adbbwM6MCcOZPsQ4Fd?=
 =?us-ascii?Q?M7ifkTuo8dk2Sy9BBs7LVQvx3H9bNbhA91iuM9zKoqoyH+FOD5ufayRjoMTd?=
 =?us-ascii?Q?2P3uThHkSSwjUKqLgM3GUE/DF5YNP7t6Oiag6Rou6DmILLg8vzxtU0w/zLFA?=
 =?us-ascii?Q?LyjdPAVW4bL7//jAEthCWHrcSHQnJvuIeCJlkSUrVZxFjoBfZGDB4wgjTmMv?=
 =?us-ascii?Q?1c0yslIBuiu2YLCSGpHhrbNf7X0M2iC509f+xbAccVJ8ZAWgb0VJmSlpFT9Y?=
 =?us-ascii?Q?kp/uYVksdbpXHk1BS9AK55dmHX3ex9+m9OT9MFXPMKGngMLxVk1DThqe508w?=
 =?us-ascii?Q?YduArS65bL2sevXF4huTYe7kaNLYYTeEjqU8UERqTwfFv4Xq2DpITqZrGH4N?=
 =?us-ascii?Q?LmPEG8HLzb5101BIwLuvJT7LXdPE+aV9XsjgmbDI4JSMBfnpCjHV2Jbg5RSe?=
 =?us-ascii?Q?f+V2gM+X4TdokHmturP/g3jEm5YZ973mHtZni4gYTDA0Rns7RlvLMa4MkjFn?=
 =?us-ascii?Q?mGAXlJOHpG7NN6l+pXjSGPo6xR4PTmeexp1JCQNOdmjBi64uMFYl11llU9VC?=
 =?us-ascii?Q?bAfsxh+/DFfK+3g8cPfYn6EL6inJN+ei6U/Fnw19bu7rRdaY6G86Q+zTaBvH?=
 =?us-ascii?Q?OEe9ZBdTLI1FrGrad6LLyB0mLpdHF2C4zeWxclXX2CDwnPEPxGqWVf5tyEV3?=
 =?us-ascii?Q?99FkotLNz11EgwsKwbDdIdSjy4+Zms/fKzC/YXS+IejsIVZm8/S3z1TUx8Js?=
 =?us-ascii?Q?sbKl5cPfx8HsdBqRNMyLC4sVS/HaZu4YNB5pirLTX1QOchdV0cf644PZRcq0?=
 =?us-ascii?Q?v2K3nfniWSj9zyPE5Sn4n9grpHrL88axgPufO9ebErco8uVqF+t/QcHhn9b6?=
 =?us-ascii?Q?cZFHHUSzpmq5Bk7+cRz1HOVAsm/wwuwiWQfjDyp/3NwFCLTxxql/PTwgakVk?=
 =?us-ascii?Q?iWOQw+nUtkq+UIAbPm5nyz5R4pEONLPHAby4GhN4Jvpj1760Zjtmni6wcKij?=
 =?us-ascii?Q?pjK/JDwfG34rqUThc/4Jd5sLBStQeij0R/IMG+hVGcxSq3Cdj25t6Zo3pFie?=
 =?us-ascii?Q?3dVHDXRNEDojtdXCE3CTyM=3D?=
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2349;6:3GCqLDNYedayDMt9OJNSFR+/rdBKWZyEPChWWakEcCGIgJplFDaDEPhx32G47xKnTrm3LwkHbCfzAnyor8dCNo6yDCg22hcT3Bulpy8eb3pwjQTkxzYzGRMEDJ8qJKxsUTYvzmFPtdw07lKQFQYbxzrDOKciv8cw7xy+cOlYYROjQ1G00sHS4NV5ueOspk6vL8kZi2p4M4hoEN1RY9bhupwsusenXfNhQkOqzwstYdYh6MIpYZD2MGhhSZoh8vzI+yiea+OB6r2Ny89LK0sIJvCMktbWdZvhCF03uwqmu9ar3poMRT4Q66sd6jUnCQx80+nr8xCSVfHmEWLhBbgEO9To2OZ4IL7qA03D2CwGCFY=;5:sZTGTXOFwm3SEY3nLrANYO2OvOx4TdPJM86PaIDtYDn1arS8+O5OTNNDD0cxCk2R+1N45qTRvTyQyBQKfPnogMIlR87dDR1vpBlkOjUw3AFSRC5sFu3effZTqdXel1Dzy4Pa/cV/COIHoItmVGar97PXGswMiCdnU7OI8n87Mc4=;24:+A9RvKY7H0dk8KQv01elqOXahsWsz9Mt4IufA9x/qfwjh+34yVhbhC7/BFrmtOnq/8ZDmFOjB2IJ77U2WiAJlE6q0tWJQ5g0vLw/OzrDI0A=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1;SN1PR07MB2349;7:+QnRwoMlnaRscdjHTAh/2NqRHn5zfqlfe+kWolGQ1nBi09DMYdRtAQ68bx8Lmf9Au4WkN/VQL+FHZrP7JfjhwpnAFsjVwGEdiuQskgLrWEqqIWpAXCycYvyf3SAN/knh0H2KXKReFfDLY3uu+rV/gN2yQucHyEdPA48V+Op0raH2rRbIzDMqUIcooEGbLLTtOSOGYVOkoSDF3pZJKjwyqiPKbmaxpbXUIltNRqruKNFF5M1ek/uOBDpYD6Gvl5aIa1+/PIcEvGOrXEsogA/nNnT9Q9mmjxAEbmxNTfRltI39YIPNjYW8+jjBabWfGI+OQkl0x2qRoCMs3i5PWYhxhsoHBsyy6RsrLR1ZQgdpbBA=
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2016 19:27:13.4648 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR07MB2349
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Ard,

> > >> On 24 November 2016 at 13:51, Robert Richter <robert.richter@cavium.com> wrote:
> > >> > On 24.11.16 13:44:31, Ard Biesheuvel wrote:

> > >> Regions containing firmware tables are owned by the firmware, and it
> > >> is the firmware that tells us which memory attributes we are allowed
> > >> to use. If those attributes include WB, it is perfectly legal to use a
> > >> cacheable mapping. That does *not* mean they should be covered by the
> > >> linear mapping. The linear mapping is read-write-non-exec, for
> > >> instance, and we may prefer to use a read-only mapping and/or
> > >> executable mapping.
> > >
> > > Ok, I am going to fix try_ram_remap().

I revisited the code and it is working well already since:

 e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem

Now, try_ram_remap() is only called if the region to be mapped is
entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
ranges and not NOMAP mem. region_intersects() then returns
REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
REGION_DISJOINT would be returned and thus arch_memremap_wb() being
called directly. Before the e7cd190385d1 change try_ram_remap() was
called also for nomap regions.

So we can leave memremap() as it is and just apply this patch
unmodified. What do you think? Please ack.

I am going to prepare the pfn_is_ram() change in addition to this
patch, but that should not be required for this fix to work correcly.

Thanks,

-Robert

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1753264AbcKYLaX (ORCPT <rfc822;w@1wt.eu>);
        Fri, 25 Nov 2016 06:30:23 -0500
Received: from mail-sn1nam01on0069.outbound.protection.outlook.com ([104.47.32.69]:51662
        "EHLO NAM01-SN1-obe.outbound.protection.outlook.com"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S1751497AbcKYLaC (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 25 Nov 2016 06:30:02 -0500
X-Greylist: delayed 137647 seconds by postgrey-1.27 at vger.kernel.org; Fri, 25 Nov 2016 06:30:02 EST
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Robert.Richter@cavium.com; 
Date: Fri, 25 Nov 2016 12:29:14 +0100
From: Robert Richter <robert.richter@cavium.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
CC: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire
 section
Message-ID: <20161125112914.GI2213@rric.localdomain>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain>
 <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain>
 <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain>
 <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain>
 <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Originating-IP: [92.229.95.127]
X-ClientProxiedBy: AM4PR0701CA0022.eurprd07.prod.outlook.com (10.165.102.32)
 To BL2PR07MB2340.namprd07.prod.outlook.com (10.167.101.18)
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2340;2:qpim9mA4xMxDc+NxDvEJT2LbIMMmBFtazeRiGQBd6EjCFYcF3a+in4aYp7LwvkZIYxgzPo3t10fGnMZVyaXEx1htj7X9BY+R4ZPyK6Uv57zsOB25v5uiN2H2YAUrsgdp4USRE8aAo8ZcyIsb3aCd2gtniLkV5RQvUVCCb8GIXWg=;3:26KzJtFuBBpwndeCDyE1KbjVdGUcosi5NWSJzWlUjjdjfY1c9Zuscpi8DMgyZvK2Wa0jmRE18QeWtvwfwN4WgYI+USiibcxuh5uybIy57PUw8+L2AvyYQ2rrBPBq7K6ZvWQMJqVNKctKlLpCeO80alyF4L2L7OIzT4I+8EVjalk=;25:ruNz4sc9aV36ZXGfTsN4n4CyoXRVyOphVuvquZJtk3wFl0mk/Atc6RNrkKH/KE1HVUZGx9QsTNbub+L5FkpP3NHjKtEYw+bF79PfxZdGKIRja1vDecPUqdRDGYo0FgMNCNB1QCYfEP+skc4AJ8VfFKJ1wDv2Wfo/XtHRlkw3E2l8FKZJby5jlZxFI46Bv5ZapNMdrjrcnSf/0+o01GQ2DbaPTkY3vTS6JkUcVgTjpk8XcacYNAlusEoolksc/0R7yA04YxF2yUN4P6aRzu4PV15uXpKU9UE9MasUZNCyz7Q5S6iBmjNPl8pv9SLDq8nkpbmlI+HiVjSpqOOHDiT0LISc6CJsYlO6CKWHb3HQaVCIm4lGyER0NybsRmodwE8Co/RlBEPX8Zihaw8IzVv9rIkpfWtV3qcUdwkzQz6UVjW3u3TnwqSKfzhoUO0uMASGCW/qN54uqTzHW3EZBHyY2g==
X-MS-Office365-Filtering-Correlation-Id: c3c75fd4-16a2-46f1-cd95-08d4152652e2
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:BL2PR07MB2340;
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2340;31:D/5wT1vbpm+AkjixA92bbkfepCODhYWolrzvDnrRcFwnoUXYPFMN49P/zWaCUEMHI5dr6ziyQcIVshT+kAUt2CJyxIArX+XyKJAjypL7y7fmdZPGrnJDaZkQiOmds766cW1Icl+W7wn5oRwa4RdWlqVTfg3sOJ8rt34li4iqkRUHWzNo5/S2VLWBTVTtAK7lfQYBt/Qu1bOPZpT/zHmnn+jQdE1kUA3g5vSz0I36mADakjjbALCLxsptpK2/fxtsCpPsaHQKRt2tnfDMyZ41mw==;20: 
X-Microsoft-Antispam-PRVS: <BL2PR07MB2340B9DBA4111525E4B8BFADF1890@BL2PR07MB2340.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6060326)(6045199)(6040361)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6041248)(6061324)(20161123555025)(20161123564025)(20161123560025)(20161123562025);SRVR:BL2PR07MB2340;BCL:0;PCL:0;RULEID:;SRVR:BL2PR07MB2340;
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2340;4: 
X-Forefront-PRVS: 01371B902F
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(7916002)(76104003)(24454002)(189002)(199003)(39410400001)(1076002)(5660300001)(39380400001)(6916009)(9686002)(110136003)(68736007)(6666003)(2950100002)(38730400001)(83506001)(4326007)(97756001)(7846002)(93886004)(105586002)(46406003)(39450400002)(50466002)(7736002)(81166006)(2906002)(81156014)(305945005)(101416001)(92566002)(106356001)(23726003)(47776003)(3846002)(189998001)(54356999)(50986999)(6116002)(8676002)(77096005)(66066001)(42186005)(229853002)(97736004)(76176999)(575784001)(33656002)(4001350100001)(86362001)(39400400001)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:BL2PR07MB2340;H:rric.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en;
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BL2PR07MB2340;23:hQUlm6dIIAUJq5bcBkD2d0f8GDGKxI2cT46RMNP6B?=
 =?us-ascii?Q?ti3cDv+DRHR0WVXdyegeQw2edtmvJfO4mCkVXksOxiF8gnJ3G6KAvyw7oNLb?=
 =?us-ascii?Q?9c7SkDUFnEKStpSCt2qbuqNmMW4vAkTMTkh0MBatdiH8um0MUKooMGrzGj2w?=
 =?us-ascii?Q?Yt05K20xayOzJAOMgcPcEsHJsh38A2UooB0JyoIGQLAq+2gFsV69nTuMP4/P?=
 =?us-ascii?Q?1blQfZbV04iBekuh7X6OeokOf5sg+SbZdVZi1UtLv/zVtj4dtWQK65Q/Vn4/?=
 =?us-ascii?Q?Mw1Fc21OTbIW/rphEWC+IXalehqcWIGLK55bQdFl+euuIML7aA7dOjEtBDfX?=
 =?us-ascii?Q?UgA7noJAjpCHi540H4oDohtroaNKfFnjohNS+LE8qGpj72ovEQv+S4bEJMLn?=
 =?us-ascii?Q?LJY1RLBaYxTNUZ2VLO6ERwxi7KWbSXyQi8/V5RQmEJBoPHSxmfZeOl2kqg70?=
 =?us-ascii?Q?B609TnWI1oVswMKN3i7vOBiiXRVO/HDLcH3IVTdFZ99zbkC7NDUxzKmIISW2?=
 =?us-ascii?Q?e+RhfFNEwM+8zfl1CYDtMEPXbhpUW5WZkiJSG8w1u+VHNBgJViqf/Gx9vZc1?=
 =?us-ascii?Q?IDSy7o9QfjC+r6cWmZsccI+uBpODmyw5Xkjsh5WDXnxumx05GK8l6K3Gpl25?=
 =?us-ascii?Q?qXLQ4NBNhraXX7IyLv4EC9HrgJgzA/7WDj732kDvwXi6paSycSx5wZA9syhH?=
 =?us-ascii?Q?MdZx6k4wkAaPT/WutbD3i2OUL8UlHJXYYR/ZJJKRdS+tGyBoHy1ZJi57SVO3?=
 =?us-ascii?Q?Jst6mwDKiuEbL1NbzsUdtGwuNbuk2CPpHA/vQwazY0ksIOpJqxN5qF2L6ZhV?=
 =?us-ascii?Q?pkauRTX6fpnm1bd/sM5S4RtROJoogiOgkvxavTcj6atX7DtmcVGc+mHhR7Sl?=
 =?us-ascii?Q?JR5Zsn3D24+fMLPFiojnTQzLAKFSH3XhpdDOaTXcQ6bqbqKxgxIYHGvj2KAt?=
 =?us-ascii?Q?rqZRnCcVUFU2nBbJvt6nWkaOAVALANweylBvhiVIAWjw+4Fjtv/bgvdxI7Sc?=
 =?us-ascii?Q?zozUK/BLBJEHo22NNBr9Kc2kS8SJ3IcQv13we8ag+xIAvDvYRJOWp+V4o72w?=
 =?us-ascii?Q?OhF20AAIBtK9RnCQpMzYsXUaT3Ev+XXHWgdAlNdG/6cwo3iYOg4jV6UtqpM1?=
 =?us-ascii?Q?vVzucwltMv90l93Q1+QqwK4QuzWh2CmTKSzPieV35mtdHzfcTxg+5/ICgqTm?=
 =?us-ascii?Q?+2FQGQwLWNWM9tYH6OA8W2+xpIbch9Mvxj9TwizwZ7LCGttjodyMjFHg0s5b?=
 =?us-ascii?Q?OQlbIWDbtUSKc3eT+eIcKgUPx8BG5zxDMEiaz8xaZYKpgXcF1Rvmi38yrb++?=
 =?us-ascii?Q?mtCJDsaxXlKpsD5xuraVkbPqpj4GcRz5fJ/oL8wZlk1c6/1t3BdYY8dCz3hw?=
 =?us-ascii?Q?MNCV6Tt0OkCIprrDQOapq7MllE=3D?=
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2340;6:pdQrvxdFyI6sxfMqUfpq67AMkLmo0Bo9gAc/Jk5XAb6+5ngrHB/k2krSz+oFacD+97atUC6xFj8jEWr/z6Qhc+Kcl3gR3ULq9zAl6GRI9XWNdSUIWIGLs0hsPBsMl0I87K/vf+Apa8cyll36eA3P37dXgXibAOBGKxO5xo4SNxZmoxWjxT6hnKwbg3KL6nTrTieVEbAQfwTzcnquUhaNr0Zidow8q8FBaDHViBDWCR1GE/jTA5o5FqjJUn8M0IoeHZvG7RxSrZGV4ZTfRXbFspC7u4U+mCiBseU0zdLBFHYXm2s4+n5jg2ELRDJF2bHPnrd08EjES19KIj8AVUUGZXy1xyk134IFeS39CXMFD5U=;5:F+gGLguhAQx9zR5LNYvB39VYQm1tVnlA/z7vMY23uqDTWt6LX0k1NjbxnSfN2tyrhgAR6olz3CJMk4WUOf5pfH+Y25zSwcTu1QqsT32qRRdllm77TZLGiAZlZV6KZwRuZnUhFMIH2Q9lSWoasK4PWNJ+6F02pKfg7vZOEaFRczk=;24:uirQEeU/mOdiFjRYgcLD7jnk3uIr1JFcpSlw5FUurBGGXPlWXZmL41dEyqb6U6A2uivHBVXjhCeQuGothJW4Qwt+aoYisYtmL/ybrYb6uTA=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1;BL2PR07MB2340;7:wtATvf3uYG0fcqlEI7c9McBDLUlwE2vbpBhdAschzEjyZus2oxQEo+UrF5x74mD68ljpyorUA6pjSSs5yF6blftsXkLZnqwNOofUr5Q0E0Og65kKn4DFzlGJJFjnkBgZuz5z+6WX5qEr/9GVZspf75SnQ/n4EnOfAHuBTF1pPX6+8wEmtW8j7gndG1nvrKzQlmax9R1NaTi9goJUFl9EX1LkfsSrh6xZ8v4pQTlW0DZgruUVQVHsCfcc0cbnOW6gNxik+2RwPjaB1j5im8lkEMdJF2YQJ1Zy3kzQ/ruiYtOiPfplahAOZytPBqLO4s0sB2tlsH2LTXJbanC5wRV7b7AchtRzyBtDiQa0M5pnnJM=
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Nov 2016 11:29:29.2284 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR07MB2340
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 24.11.16 19:42:47, Ard Biesheuvel wrote:
> On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:

> > I revisited the code and it is working well already since:
> >
> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
> >
> > Now, try_ram_remap() is only called if the region to be mapped is
> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
> > ranges and not NOMAP mem. region_intersects() then returns
> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
> > called directly. Before the e7cd190385d1 change try_ram_remap() was
> > called also for nomap regions.
> >
> > So we can leave memremap() as it is and just apply this patch
> > unmodified. What do you think?
> 
> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
> simply because the PageHighmem check requires a valid struct page. But
> if we don't enter that code path anymore for NOMAP regions, I think
> we're ok.
> 
> > Please ack.
> >
> 
> I still don't fully understand how it is guaranteed that *all* memory
> (i.e., all regions for which memblock_is_memory() returns true) is
> covered by a struct page, but marked as reserved. Are we relying on
> the fact that NOMAP memory is also memblock_reserve()'d?

See free_low_memory_core_early():

----
	for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
        			NULL)
		count += __free_memory_core(start, end);
----

Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
also *not* marked reserved. So nothing at all from NOMAP mem is
reported to mm, it is not present (see below for a mem config, note
flags: 0x4 mem regions).

-Robert


[    0.000000] efi: Processing EFI memory map:
[    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000001480000-0x0000024bffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000024c0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010000400000-0x010f816aefff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f816af000-0x010f816b1fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f816b2000-0x010f826f1fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f826f2000-0x010f82701fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f82702000-0x010f82787fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f82788000-0x010f9276bfff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f9276c000-0x010f9276cfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f9276d000-0x010f935a8fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f935a9000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
[    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
[    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]

[    0.000000] MEMBLOCK configuration:
[    0.000000]  memory size = 0x1ffe800000 reserved size = 0x39146a21
[    0.000000]  memory.cnt  = 0x9
[    0.000000]  memory[0x0]     [0x00000001400000-0x000000fffdffff], 0xfebe0000 bytes on node 0 flags: 0x0
[    0.000000]  memory[0x1]     [0x000000fffe0000-0x000000ffffffff], 0x20000 bytes on node 0 flags: 0x4
[    0.000000]  memory[0x2]     [0x00000100000000-0x00000fffffffff], 0xf00000000 bytes on node 0 flags: 0x0
[    0.000000]  memory[0x3]     [0x00010000400000-0x00010ff9e7ffff], 0xff9a80000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x4]     [0x00010ff9e80000-0x00010ff9f1ffff], 0xa0000 bytes on node 1 flags: 0x4
[    0.000000]  memory[0x5]     [0x00010ff9f20000-0x00010ffaeaffff], 0xf90000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x6]     [0x00010ffaeb0000-0x00010ffaffffff], 0x150000 bytes on node 1 flags: 0x4
[    0.000000]  memory[0x7]     [0x00010ffb000000-0x00010ffffaffff], 0x4fb0000 bytes on node 1 flags: 0x0
[    0.000000]  memory[0x8]     [0x00010ffffb0000-0x00010fffffffff], 0x50000 bytes on node 1 flags: 0x4
[    0.000000]  reserved.cnt  = 0xd
[    0.000000]  reserved[0x0]   [0x00000001480000-0x0000000249ffff], 0x1020000 bytes flags: 0x0
[    0.000000]  reserved[0x1]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
[    0.000000]  reserved[0x2]   [0x000000c0000000-0x000000dfffffff], 0x20000000 bytes flags: 0x0
[    0.000000]  reserved[0x3]   [0x00000ffbfb8000-0x00000ffffdffff], 0x4028000 bytes flags: 0x0
[    0.000000]  reserved[0x4]   [0x00000ffffecb00-0x00000fffffffff], 0x13500 bytes flags: 0x0
[    0.000000]  reserved[0x5]   [0x00010f81780000-0x00010f8178ffff], 0x10000 bytes flags: 0x0
[    0.000000]  reserved[0x6]   [0x00010f82870000-0x00010f9286ffff], 0x10000000 bytes flags: 0x0
[    0.000000]  reserved[0x7]   [0x00010ffbce0000-0x00010fffceffff], 0x4010000 bytes flags: 0x0
[    0.000000]  reserved[0x8]   [0x00010fffee6d80-0x00010ffff2fffb], 0x4927c bytes flags: 0x0
[    0.000000]  reserved[0x9]   [0x00010ffff30000-0x00010ffffa000f], 0x70010 bytes flags: 0x0
[    0.000000]  reserved[0xa]   [0x00010ffffae280-0x00010ffffaff7f], 0x1d00 bytes flags: 0x0
[    0.000000]  reserved[0xb]   [0x00010ffffaffa0-0x00010ffffaffce], 0x2f bytes flags: 0x0
[    0.000000]  reserved[0xc]   [0x00010ffffaffd0-0x00010ffffafffe], 0x2f bytes flags: 0x0

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1754193AbcKYM3C (ORCPT <rfc822;w@1wt.eu>);
        Fri, 25 Nov 2016 07:29:02 -0500
Received: from mail-io0-f175.google.com ([209.85.223.175]:34398 "EHLO
        mail-io0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1752955AbcKYM2y (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 25 Nov 2016 07:28:54 -0500
MIME-Version: 1.0
In-Reply-To: <20161125112914.GI2213@rric.localdomain>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain> <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain> <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain> <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain> <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com> <20161125112914.GI2213@rric.localdomain>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Fri, 25 Nov 2016 12:28:50 +0000
Message-ID: <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 25 November 2016 at 11:29, Robert Richter <robert.richter@cavium.com> wrote:
> On 24.11.16 19:42:47, Ard Biesheuvel wrote:
>> On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:
>
>> > I revisited the code and it is working well already since:
>> >
>> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>> >
>> > Now, try_ram_remap() is only called if the region to be mapped is
>> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
>> > ranges and not NOMAP mem. region_intersects() then returns
>> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
>> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
>> > called directly. Before the e7cd190385d1 change try_ram_remap() was
>> > called also for nomap regions.
>> >
>> > So we can leave memremap() as it is and just apply this patch
>> > unmodified. What do you think?
>>
>> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
>> simply because the PageHighmem check requires a valid struct page. But
>> if we don't enter that code path anymore for NOMAP regions, I think
>> we're ok.
>>
>> > Please ack.
>> >
>>
>> I still don't fully understand how it is guaranteed that *all* memory
>> (i.e., all regions for which memblock_is_memory() returns true) is
>> covered by a struct page, but marked as reserved. Are we relying on
>> the fact that NOMAP memory is also memblock_reserve()'d?
>
> See free_low_memory_core_early():
>
> ----
>         for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
>                                 NULL)
>                 count += __free_memory_core(start, end);
> ----
>
> Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
> also *not* marked reserved. So nothing at all from NOMAP mem is
> reported to mm, it is not present (see below for a mem config, note
> flags: 0x4 mem regions).
>

OK, thanks for clearing that up. But that still does not explain how
we can be certain that NOMAP regions are guaranteed to be covered by a
struct page, does it? Because that is ultimately what pfn_valid()
means, that it is safe to, e.g., look at the page flags.


> [    0.000000] efi: Processing EFI memory map:
> [    0.000000] efi:   0x000001400000-0x00000147ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000001480000-0x0000024bffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000024c0000-0x0000211fffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000021200000-0x00002121ffff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000021220000-0x0000fffebfff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000fffec000-0x0000ffff5fff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000ffff6000-0x0000ffff6fff [ACPI Memory NVS    |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x0000ffff7000-0x0000ffffffff [ACPI Reclaim Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000100000000-0x000ff7ffffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000ff8000000-0x000ff801ffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000ff8020000-0x000fffa9efff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x000fffa9f000-0x000fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010000400000-0x010f816aefff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f816af000-0x010f816b1fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f816b2000-0x010f826f1fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f826f2000-0x010f82701fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f82702000-0x010f82787fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f82788000-0x010f9276bfff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f9276c000-0x010f9276cfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f9276d000-0x010f935a8fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f935a9000-0x010f93880fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010f93881000-0x010ff7880fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff7881000-0x010ff7886fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff7887000-0x010ff78a3fff [Loader Code        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff78a4000-0x010ff9e8dfff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff9e8e000-0x010ff9f16fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ff9f17000-0x010ffaeb5fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffaeb6000-0x010ffafc8fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffafc9000-0x010ffafccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffafcd000-0x010ffaff4fff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffaff5000-0x010ffb008fff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffb009000-0x010fffe28fff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe29000-0x010fffe3ffff [Conventional Memory|   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe40000-0x010fffe53fff [Loader Data        |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010fffe54000-0x010ffffb8fff [Boot Code          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffb9000-0x010ffffccfff [Runtime Code       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffcd000-0x010fffffefff [Runtime Data       |RUN|  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x010ffffff000-0x010fffffffff [Boot Data          |   |  |  |  |  |  |  |   |WB|WT|WC|UC]
> [    0.000000] efi:   0x804000001000-0x804000001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
> [    0.000000] efi:   0x87e0d0001000-0x87e0d0001fff [Memory Mapped I/O  |RUN|  |  |  |  |  |  |   |  |  |  |UC]
>
> [    0.000000] MEMBLOCK configuration:
> [    0.000000]  memory size = 0x1ffe800000 reserved size = 0x39146a21
> [    0.000000]  memory.cnt  = 0x9
> [    0.000000]  memory[0x0]     [0x00000001400000-0x000000fffdffff], 0xfebe0000 bytes on node 0 flags: 0x0
> [    0.000000]  memory[0x1]     [0x000000fffe0000-0x000000ffffffff], 0x20000 bytes on node 0 flags: 0x4
> [    0.000000]  memory[0x2]     [0x00000100000000-0x00000fffffffff], 0xf00000000 bytes on node 0 flags: 0x0
> [    0.000000]  memory[0x3]     [0x00010000400000-0x00010ff9e7ffff], 0xff9a80000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x4]     [0x00010ff9e80000-0x00010ff9f1ffff], 0xa0000 bytes on node 1 flags: 0x4
> [    0.000000]  memory[0x5]     [0x00010ff9f20000-0x00010ffaeaffff], 0xf90000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x6]     [0x00010ffaeb0000-0x00010ffaffffff], 0x150000 bytes on node 1 flags: 0x4
> [    0.000000]  memory[0x7]     [0x00010ffb000000-0x00010ffffaffff], 0x4fb0000 bytes on node 1 flags: 0x0
> [    0.000000]  memory[0x8]     [0x00010ffffb0000-0x00010fffffffff], 0x50000 bytes on node 1 flags: 0x4
> [    0.000000]  reserved.cnt  = 0xd
> [    0.000000]  reserved[0x0]   [0x00000001480000-0x0000000249ffff], 0x1020000 bytes flags: 0x0
> [    0.000000]  reserved[0x1]   [0x00000021200000-0x00000021210536], 0x10537 bytes flags: 0x0
> [    0.000000]  reserved[0x2]   [0x000000c0000000-0x000000dfffffff], 0x20000000 bytes flags: 0x0
> [    0.000000]  reserved[0x3]   [0x00000ffbfb8000-0x00000ffffdffff], 0x4028000 bytes flags: 0x0
> [    0.000000]  reserved[0x4]   [0x00000ffffecb00-0x00000fffffffff], 0x13500 bytes flags: 0x0
> [    0.000000]  reserved[0x5]   [0x00010f81780000-0x00010f8178ffff], 0x10000 bytes flags: 0x0
> [    0.000000]  reserved[0x6]   [0x00010f82870000-0x00010f9286ffff], 0x10000000 bytes flags: 0x0
> [    0.000000]  reserved[0x7]   [0x00010ffbce0000-0x00010fffceffff], 0x4010000 bytes flags: 0x0
> [    0.000000]  reserved[0x8]   [0x00010fffee6d80-0x00010ffff2fffb], 0x4927c bytes flags: 0x0
> [    0.000000]  reserved[0x9]   [0x00010ffff30000-0x00010ffffa000f], 0x70010 bytes flags: 0x0
> [    0.000000]  reserved[0xa]   [0x00010ffffae280-0x00010ffffaff7f], 0x1d00 bytes flags: 0x0
> [    0.000000]  reserved[0xb]   [0x00010ffffaffa0-0x00010ffffaffce], 0x2f bytes flags: 0x0
> [    0.000000]  reserved[0xc]   [0x00010ffffaffd0-0x00010ffffafffe], 0x2f bytes flags: 0x0

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S932592AbcKYRHd (ORCPT <rfc822;w@1wt.eu>);
        Fri, 25 Nov 2016 12:07:33 -0500
Received: from mail-io0-f176.google.com ([209.85.223.176]:36854 "EHLO
        mail-io0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S932442AbcKYRHZ (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 25 Nov 2016 12:07:25 -0500
MIME-Version: 1.0
In-Reply-To: <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
References: <CAKv+Gu8aWpz5vXSWdKHwL5Xk2aWyQBOoR1WHOnVPaWZOwqojBQ@mail.gmail.com>
 <20161124134238.GI10776@rric.localdomain> <CAKv+Gu_Ug3XOD6jy5xziy1pS7WV=TC0RLD4j3DT6nytujQDzRQ@mail.gmail.com>
 <20161124135151.GJ10776@rric.localdomain> <CAKv+Gu9qB7P2epMGdwFJ3vJR3vt-bBxU0nKwmKdpaOpTKdqzzA@mail.gmail.com>
 <20161124141149.GE2213@rric.localdomain> <CAKv+Gu-SXX_Vv1Wi1Nm=5vdaXSS0MBp20QBpQVacSKPLfdH0-A@mail.gmail.com>
 <20161124150918.GF2213@rric.localdomain> <20161124192659.GH2213@rric.localdomain>
 <CAKv+Gu_C_17RtAiw2U0OOzVik3G7KkwUTo5eiGK-HDo-maQ-bA@mail.gmail.com>
 <20161125112914.GI2213@rric.localdomain> <CAKv+Gu9h2YWJd8-+FhSYSftUsddj1f-HTA2D-6nC=8Wk6U1iAg@mail.gmail.com>
From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Fri, 25 Nov 2016 17:01:01 +0000
Message-ID: <CAKv+Gu8Hbf1sY4zMGoosYjGa1esApqP8a4Zs2Zqj4nNZ8AdtSw@mail.gmail.com>
Subject: Re: [PATCH] arm64: mm: Fix memmap to be initialized for the entire section
To: Robert Richter <robert.richter@cavium.com>
Cc: Will Deacon <will.deacon@arm.com>, Robert Richter <rric@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        David Daney <david.daney@cavium.com>,
        Hanjun Guo <hanjun.guo@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 25 November 2016 at 12:28, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 25 November 2016 at 11:29, Robert Richter <robert.richter@cavium.com> wrote:
>> On 24.11.16 19:42:47, Ard Biesheuvel wrote:
>>> On 24 November 2016 at 19:26, Robert Richter <robert.richter@cavium.com> wrote:
>>
>>> > I revisited the code and it is working well already since:
>>> >
>>> >  e7cd190385d1 arm64: mark reserved memblock regions explicitly in iomem
>>> >
>>> > Now, try_ram_remap() is only called if the region to be mapped is
>>> > entirely in IORESOURCE_SYSTEM_RAM. This is only true for normal mem
>>> > ranges and not NOMAP mem. region_intersects() then returns
>>> > REGION_INTERSECTS and calls try_ram_remap(). For the NOMAP memory case
>>> > REGION_DISJOINT would be returned and thus arch_memremap_wb() being
>>> > called directly. Before the e7cd190385d1 change try_ram_remap() was
>>> > called also for nomap regions.
>>> >
>>> > So we can leave memremap() as it is and just apply this patch
>>> > unmodified. What do you think?
>>>
>>> I agree. The pfn_valid() check in try_ram_remap() is still appropriate
>>> simply because the PageHighmem check requires a valid struct page. But
>>> if we don't enter that code path anymore for NOMAP regions, I think
>>> we're ok.
>>>
>>> > Please ack.
>>> >
>>>
>>> I still don't fully understand how it is guaranteed that *all* memory
>>> (i.e., all regions for which memblock_is_memory() returns true) is
>>> covered by a struct page, but marked as reserved. Are we relying on
>>> the fact that NOMAP memory is also memblock_reserve()'d?
>>
>> See free_low_memory_core_early():
>>
>> ----
>>         for_each_free_mem_range(i, NUMA_NO_NODE, MEMBLOCK_NONE, &start, &end,
>>                                 NULL)
>>                 count += __free_memory_core(start, end);
>> ----
>>
>> Only mem with the MEMBLOCK_NONE flag is added. And NOMAP pages are
>> also *not* marked reserved. So nothing at all from NOMAP mem is
>> reported to mm, it is not present (see below for a mem config, note
>> flags: 0x4 mem regions).
>>
>
> OK, thanks for clearing that up. But that still does not explain how
> we can be certain that NOMAP regions are guaranteed to be covered by a
> struct page, does it? Because that is ultimately what pfn_valid()
> means, that it is safe to, e.g., look at the page flags.
>

Answering to self: arm64_memory_present() registers all memory as
present, which means that sparse_init() will allocate struct page
backing for all of memory, including NOMAP regions.

We could ask ourselves whether it makes sense to disregard NOMAP
memory here, but that probably buys us very little (but I do wonder
how it affects the occurrence of the bug).

In any case, it looks to me like your patch is safe, i.e., calling
pfn_valid() on NOMAP pages is safe, although I still find it debatable
whether the kernel should be tracking memory it does not own. However,
for performance reasons, it probably makes sense to apply your patch,
and if anyone ever comes up with a use case where this is problematic
(e.g., gigabytes of NOMAP memory), we can look into it then.

Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

