From: Andrew Morton <akpm@linux-foundation.org>
To: Dave Hansen <dave.hansen@intel.com>
Cc: Mel Gorman <mgorman@suse.de>, Nathan Zimmer <nzimmer@sgi.com>,
Waiman Long <waiman.long@hp.com>,
Scott Norton <scott.norton@hp.com>,
Daniel J Blueman <daniel@numascale.com>,
Linux-MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
Al Viro <viro@zeniv.linux.org.uk>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: 4.2-rc2: hitting "file-max limit 8192 reached"
Date: Tue, 14 Jul 2015 09:15:54 -0700 [thread overview]
Message-ID: <20150714091554.3d653316.akpm@linux-foundation.org> (raw)
In-Reply-To: <55A530A3.2080301@intel.com>
On Tue, 14 Jul 2015 08:54:11 -0700 Dave Hansen <dave.hansen@intel.com> wrote:
> My laptop has been behaving strangely with 4.2-rc2. Once I log in to my
> X session, I start getting all kinds of strange errors from applications
> and see this in my dmesg:
>
> VFS: file-max limit 8192 reached
>
> Could this be from CONFIG_DEFERRED_STRUCT_PAGE_INIT=y? files_init()
> seems top be sizing files_stat.max_files from memory sizes.
argh.
> vfs_caches_init() uses nr_free_pages() to figure out what the "current
> kernel size" is in early boot. *But* since we have not freed most of
> our memory, nr_free_pages() is low and makes us calculate the reserve as
> if the kernel we huge.
>
> Adding some printk's confirms this. Broken kernel:
>
> vfs_caches_init() mempages: 4026972
> vfs_caches_init() reserve: 4021629
> vfs_caches_init() mempages (after reserve minus): 5343
> files_init() n: 2137
> files_init() files_stat.max_files: 8192
>
> Working kernel:
>
> vfs_caches_init() mempages: 4026972
> vfs_caches_init() reserve: 375
> vfs_caches_init() mempages2: 4026597
> files_init() n: 1610638
> files_init() files_stat.max_files: 1610638
>
> Do we have an alternative to call instead of nr_free_pages() in
> vfs_caches_init()?
>
> I guess we could save off 'nr_initialized' in memmap_init_zone() and
> then use "nr_initialized - nr_free_pages()", but that seems a bit hackish.
There are a lot of things that might be affected this way. Callers of
nr_free_buffer_pages(), nr_free_pagecache_pages(), etc.
If we'd fully used the memory hotplug infrastructure then everything
would work - all those knobs which are sized off free-memory would get
themselves resized as more memory comes on line. But quite a few
things have been missed.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Dave Hansen <dave.hansen@intel.com>
Cc: Mel Gorman <mgorman@suse.de>, Nathan Zimmer <nzimmer@sgi.com>,
Waiman Long <waiman.long@hp.com>,
Scott Norton <scott.norton@hp.com>,
Daniel J Blueman <daniel@numascale.com>,
Linux-MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
Al Viro <viro@zeniv.linux.org.uk>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: 4.2-rc2: hitting "file-max limit 8192 reached"
Date: Tue, 14 Jul 2015 09:15:54 -0700 [thread overview]
Message-ID: <20150714091554.3d653316.akpm@linux-foundation.org> (raw)
In-Reply-To: <55A530A3.2080301@intel.com>
On Tue, 14 Jul 2015 08:54:11 -0700 Dave Hansen <dave.hansen@intel.com> wrote:
> My laptop has been behaving strangely with 4.2-rc2. Once I log in to my
> X session, I start getting all kinds of strange errors from applications
> and see this in my dmesg:
>
> VFS: file-max limit 8192 reached
>
> Could this be from CONFIG_DEFERRED_STRUCT_PAGE_INIT=y? files_init()
> seems top be sizing files_stat.max_files from memory sizes.
argh.
> vfs_caches_init() uses nr_free_pages() to figure out what the "current
> kernel size" is in early boot. *But* since we have not freed most of
> our memory, nr_free_pages() is low and makes us calculate the reserve as
> if the kernel we huge.
>
> Adding some printk's confirms this. Broken kernel:
>
> vfs_caches_init() mempages: 4026972
> vfs_caches_init() reserve: 4021629
> vfs_caches_init() mempages (after reserve minus): 5343
> files_init() n: 2137
> files_init() files_stat.max_files: 8192
>
> Working kernel:
>
> vfs_caches_init() mempages: 4026972
> vfs_caches_init() reserve: 375
> vfs_caches_init() mempages2: 4026597
> files_init() n: 1610638
> files_init() files_stat.max_files: 1610638
>
> Do we have an alternative to call instead of nr_free_pages() in
> vfs_caches_init()?
>
> I guess we could save off 'nr_initialized' in memmap_init_zone() and
> then use "nr_initialized - nr_free_pages()", but that seems a bit hackish.
There are a lot of things that might be affected this way. Callers of
nr_free_buffer_pages(), nr_free_pagecache_pages(), etc.
If we'd fully used the memory hotplug infrastructure then everything
would work - all those knobs which are sized off free-memory would get
themselves resized as more memory comes on line. But quite a few
things have been missed.
next prev parent reply other threads:[~2015-07-14 16:16 UTC|newest]
Thread overview: 168+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-28 14:36 [PATCH 0/13] Parallel struct page initialisation v4 Mel Gorman
2015-04-28 14:36 ` Mel Gorman
2015-04-28 14:36 ` [PATCH 01/13] memblock: Introduce a for_each_reserved_mem_region iterator Mel Gorman
2015-04-28 14:36 ` Mel Gorman
2015-04-28 14:36 ` [PATCH 02/13] mm: meminit: Move page initialization into a separate function Mel Gorman
2015-04-28 14:36 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 03/13] mm: meminit: Only set page reserved in the memblock region Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-05-22 20:31 ` Tony Luck
2015-05-22 20:31 ` Tony Luck
2015-05-26 10:22 ` Mel Gorman
2015-05-26 10:22 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 04/13] mm: page_alloc: Pass PFN to __free_pages_bootmem Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-05-01 9:20 ` [PATCH] mm: page_alloc: pass PFN to __free_pages_bootmem -fix Mel Gorman
2015-05-01 9:20 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 05/13] mm: meminit: Make __early_pfn_to_nid SMP-safe and introduce meminit_pfn_in_nid Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 06/13] mm: meminit: Inline some helper functions Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-30 21:53 ` Andrew Morton
2015-04-30 21:53 ` Andrew Morton
2015-04-30 21:55 ` Andrew Morton
2015-04-30 21:55 ` Andrew Morton
2015-05-04 8:33 ` Michal Hocko
2015-05-04 8:33 ` Michal Hocko
2015-05-04 8:38 ` Michal Hocko
2015-05-04 8:38 ` Michal Hocko
2015-04-28 14:37 ` [PATCH 07/13] mm: meminit: Initialise a subset of struct pages if CONFIG_DEFERRED_STRUCT_PAGE_INIT is set Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-29 21:19 ` Andrew Morton
2015-04-29 21:19 ` Andrew Morton
2015-04-30 8:45 ` Mel Gorman
2015-04-30 8:45 ` Mel Gorman
2015-05-01 9:21 ` [PATCH] mm: meminit: Initialise a subset of struct pages if CONFIG_DEFERRED_STRUCT_PAGE_INIT is set -fix Mel Gorman
2015-05-01 9:21 ` Mel Gorman
2015-07-14 15:54 ` 4.2-rc2: hitting "file-max limit 8192 reached" Dave Hansen
2015-07-14 15:54 ` Dave Hansen
2015-07-14 16:15 ` Andrew Morton [this message]
2015-07-14 16:15 ` Andrew Morton
2015-07-15 10:45 ` Mel Gorman
2015-07-15 10:45 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 08/13] mm: meminit: Initialise remaining struct pages in parallel with kswapd Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 09/13] mm: meminit: Minimise number of pfn->page lookups during initialisation Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 10/13] x86: mm: Enable deferred struct page initialisation on x86-64 Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 11/13] mm: meminit: Free pages in large chunks where possible Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 12/13] mm: meminit: Reduce number of times pageblocks are set during struct page init Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-05-01 9:23 ` [PATCH] mm: meminit: Reduce number of times pageblocks are set during struct page init -fix Mel Gorman
2015-05-01 9:23 ` Mel Gorman
2015-04-28 14:37 ` [PATCH 13/13] mm: meminit: Remove mminit_verify_page_links Mel Gorman
2015-04-28 14:37 ` Mel Gorman
2015-04-28 16:06 ` [PATCH 0/13] Parallel struct page initialisation v4 Pekka Enberg
2015-04-28 16:06 ` Pekka Enberg
2015-04-28 18:38 ` nzimmer
2015-04-28 18:38 ` nzimmer
2015-04-30 16:10 ` Daniel J Blueman
2015-04-30 16:10 ` Daniel J Blueman
2015-04-30 17:12 ` nzimmer
2015-04-30 17:12 ` nzimmer
2015-04-30 17:28 ` Mel Gorman
2015-04-30 17:28 ` Mel Gorman
2015-05-02 11:52 ` Elliott, Robert (Server Storage)
2015-05-02 11:52 ` Elliott, Robert (Server Storage)
2015-05-02 11:52 ` Elliott, Robert (Server Storage)
2015-04-29 1:16 ` Waiman Long
2015-04-29 1:16 ` Waiman Long
2015-05-01 22:02 ` Waiman Long
2015-05-01 22:02 ` Waiman Long
2015-05-02 0:09 ` Waiman Long
2015-05-02 0:09 ` Waiman Long
2015-05-02 8:52 ` Daniel J Blueman
2015-05-02 8:52 ` Daniel J Blueman
2015-05-02 16:05 ` Daniel J Blueman
2015-05-02 16:05 ` Daniel J Blueman
2015-05-04 21:30 ` Andrew Morton
2015-05-04 21:30 ` Andrew Morton
2015-05-05 3:32 ` Waiman Long
2015-05-05 3:32 ` Waiman Long
2015-05-05 10:45 ` Mel Gorman
2015-05-05 10:45 ` Mel Gorman
2015-05-05 13:55 ` Waiman Long
2015-05-05 13:55 ` Waiman Long
2015-05-05 14:31 ` Mel Gorman
2015-05-05 14:31 ` Mel Gorman
2015-05-05 15:01 ` Waiman Long
2015-05-05 15:01 ` Waiman Long
2015-05-06 3:39 ` Waiman Long
2015-05-06 3:39 ` Waiman Long
2015-05-06 0:55 ` Waiman Long
2015-05-06 0:55 ` Waiman Long
2015-05-05 20:02 ` Andrew Morton
2015-05-05 20:02 ` Andrew Morton
2015-05-05 22:13 ` Mel Gorman
2015-05-05 22:13 ` Mel Gorman
2015-05-05 22:25 ` Andrew Morton
2015-05-05 22:25 ` Andrew Morton
2015-05-06 7:12 ` Mel Gorman
2015-05-06 7:12 ` Mel Gorman
2015-05-06 10:22 ` Mel Gorman
2015-05-06 10:22 ` Mel Gorman
2015-05-06 12:05 ` Mel Gorman
2015-05-06 12:05 ` Mel Gorman
2015-05-06 17:58 ` Waiman Long
2015-05-06 17:58 ` Waiman Long
2015-05-07 2:37 ` Waiman Long
2015-05-07 2:37 ` Waiman Long
2015-05-07 7:21 ` Mel Gorman
2015-05-07 7:21 ` Mel Gorman
2015-05-06 1:21 ` Waiman Long
2015-05-06 1:21 ` Waiman Long
2015-05-06 2:01 ` Andrew Morton
2015-05-06 2:01 ` Andrew Morton
2015-05-07 7:25 ` [PATCH] mm: meminit: Finish initialisation of struct pages before basic setup Mel Gorman
2015-05-07 7:25 ` Mel Gorman
2015-05-07 22:09 ` Andrew Morton
2015-05-07 22:09 ` Andrew Morton
2015-05-07 22:52 ` Mel Gorman
2015-05-07 22:52 ` Mel Gorman
2015-05-07 23:02 ` Andrew Morton
2015-05-07 23:02 ` Andrew Morton
2015-05-13 15:53 ` nzimmer
2015-05-13 15:53 ` nzimmer
2015-05-13 16:31 ` Mel Gorman
2015-05-13 16:31 ` Mel Gorman
2015-05-14 10:03 ` Daniel J Blueman
2015-05-14 10:03 ` Daniel J Blueman
2015-05-14 15:47 ` nzimmer
2015-05-14 15:47 ` nzimmer
2015-05-19 18:31 ` nzimmer
2015-05-19 18:31 ` nzimmer
2015-05-19 19:06 ` Mel Gorman
2015-05-19 19:06 ` Mel Gorman
2015-05-22 6:30 ` Daniel J Blueman
2015-05-22 6:30 ` Daniel J Blueman
2015-05-22 9:33 ` Mel Gorman
2015-05-22 9:33 ` Mel Gorman
2015-05-22 17:14 ` Waiman Long
2015-05-22 17:14 ` Waiman Long
2015-05-22 21:43 ` Davidlohr Bueso
2015-05-22 21:43 ` Davidlohr Bueso
2015-05-23 3:49 ` Daniel J Blueman
2015-05-23 3:49 ` Daniel J Blueman
2015-06-24 22:50 ` Nathan Zimmer
2015-06-24 22:50 ` Nathan Zimmer
2015-06-25 20:48 ` Mel Gorman
2015-06-25 20:48 ` Mel Gorman
2015-06-25 20:57 ` Mel Gorman
2015-06-25 20:57 ` Mel Gorman
2015-06-25 21:37 ` Nathan Zimmer
2015-06-25 21:37 ` Nathan Zimmer
2015-06-25 21:34 ` Nathan Zimmer
2015-06-25 21:34 ` Nathan Zimmer
2015-06-25 21:44 ` [RFC] kthread_create_on_node is failing to honor the node choice Nathan Zimmer
2015-06-26 1:08 ` Lai Jiangshan
2015-07-09 22:12 ` Andrew Morton
2015-07-10 14:26 ` Mel Gorman
2015-07-10 17:34 ` Nathan Zimmer
2015-06-26 10:16 ` [PATCH] mm: meminit: Finish initialisation of struct pages before basic setup Mel Gorman
2015-06-26 10:16 ` Mel Gorman
2015-07-06 17:45 ` Daniel J Blueman
2015-07-06 17:45 ` Daniel J Blueman
2015-07-09 17:49 ` Nathan Zimmer
2015-07-09 17:49 ` Nathan Zimmer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150714091554.3d653316.akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=daniel@numascale.com \
--cc=dave.hansen@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=nzimmer@sgi.com \
--cc=scott.norton@hp.com \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
--cc=waiman.long@hp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.