Re: [PATCH v2 0/6] ARM: Add support for large kernel page (from 8K to 64K)

From: Catalin Marinas <catalin.marinas@arm.com>
To: Arnd Bergmann <arnd@arndb.de>
Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>,
	Gregory CLEMENT <gregory.clement@bootlin.com>,
	Russell King - ARM Linux admin <linux@armlinux.org.uk>,
	Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Subject: Re: [PATCH v2 0/6] ARM: Add support for large kernel page (from 8K to 64K)
Date: Fri, 12 Jun 2020 13:21:27 +0100	[thread overview]
Message-ID: <20200612122126.GC19739@gaia> (raw)
In-Reply-To: <CAK8P3a0KZzR0_8HWXYZRgq8Cg83M0s9GZa5wV+yz9KhvaSWEDw@mail.gmail.com>

On Fri, Jun 12, 2020 at 11:23:11AM +0200, Arnd Bergmann wrote:
> On Thu, Jun 11, 2020 at 6:21 PM Russell King - ARM Linux admin
> <linux@armlinux.org.uk> wrote:
> 
> > If you increase the page size, then you need to increase the number
> > of tables in a page, or suffer a huge amount of wasted memory taken
> > for the page tables - going to an 8k page size means that the upper
> > 4k of each page will not be used.  Going to 16k means the upper 12k
> > won't be used.  And so on - as your software page size increases,
> > the amount of memory wasted for each PTE table will increase
> > unless you also increase the number of hardware 1st level entries
> > pointing to each PTE page.  With 64k pages, 60k of each PTE page
> > will remain unused.
> >
> > That isn't very efficient use of memory.
> 
> I think this could be addressed by using the full page to contain
> PTEs by making PTRS_PER_PTE larger and PTRS_PER_PGD
> smaller, but there is an even bigger problem in the added memory
> usage and I/O overhead for basically everything else: in any
> sparsely populated memory mapped file or anonymous mapping,
> the memory usage grows with the page size as well.
> 
> I think Synology's vendor kernels for their NAS boxes have a
> different hack to make large file systems work, by extending
> the internal data types (I forgot which ones) to 64 bit. That is
> probably more invasive to the generic kernel code, but should
> be much more efficient and less invasive to ARM architecture
> specific code.

IIUC from Gregory's cover letter, the problem is page->index which is a
pgoff_t, unsigned long. This limits us to a 32-bit page offsets, so a
44-bit actual file offset (16TB). It may be worth exploring this than
hacking the page tables to pretend we have bigger page sizes.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel