* [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree @ 2017-02-09 13:21 peter enderborg 2017-02-09 15:42 ` [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller peter enderborg 2017-02-10 10:27 ` [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree Michal Hocko 0 siblings, 2 replies; 7+ messages in thread From: peter enderborg @ 2017-02-09 13:21 UTC (permalink / raw) To: devel, Greg Kroah-Hartman, linux-kernel, Arve Hjønnevåg, Riley Andrews, Linus Torvalds, linux-mm Lowmemorykiller efficiency problem and a solution. Lowmemorykiller in android has a severe efficiency problem. The basic problem is that the registered shrinker gets called very often without anything actually happening. This is in some cases not a problem as it is a simple calculation that returns a value. But when there is high pressure on memory and we get to start killing processes to free memory we get some heavy work that does lots of cpu processing and lock holding for no real benefit. This occurs when we are below the first threshold level in minfree. We call that waste. To see this problem we introduce a patch that collects statistics from lowmemorykiller. We collect the amount of kills, scans, counts and some other metrics. One of this metrics is called waste. These metrics are presented in procfs as /proc/lmkstats. Patchset: 0001-android-Collect-statistics-from-lowmemorykiller.patch 0002-oom-Add-notification-for-oom_score_adj.patch 0003-mm-Remove-RCU-and-tasklocks-from-lmk.patch Collect-statistics-from-lowmemorykiller.patch --------------------------------------------- This patch only adds metrics and is there to show behavour before and after and is a good way to see that the device is in waste zone. 0002-oom-Add-notification-for-oom_score_adj.patch ------------------------------------------------ This is the prerequisite patch to be able to do the lowmemorykiller change. It introduces notifiers for oom_score_adj. It generates notifier events for process creation and death, and when process values are changed. These patches are outside from stageing drivers and are applied to core functions in e.g. fork.c. 0003-mm-Remove-RCU-and-tasklocks-from-lmk.patch ----------------------------------------------- This patch is the change of lowmemorykiller. It builds a tree structure that works as cache for the task list, but only contains the tasks that are relevant for the lmk. The key thing here is that the cache is sorted based on the oom_score_adj value so the scan and count function can find the right task with only a tree first operation. Based on the right task the count can give a proper reply and give a right estimate of the amount it will free, and more important when it is not willing to free anything. This makes the shrinker not to call the scan function at all, and when it is called it actually do what it's supposed to do that is to free up some memory. I consider this as mm based on the behaviour changes for the shrinker even if the code is a driver. About testing. Reproduce the problem. For this the first patch is needed and enabeld. It does not change the lowmemory killer other than it add some metrics. One counter is called WASTE. This is what this patch-set is about. In android environment this can be tested directly. On other systems like fedora a method using the stress package can be used. Apply the patches. (First with only metrics) then in your shell: echo 400 > /proc/self/oom_score_adj Now you have created a shell that has something that can be killed. In the same shell use stress program. The parameters will be very dependent on your configuration, but you need to run out of memmory. Most of the wasted cpu cycles are accounted in kswapd0 task so a compare of the reduced waste can also be seen in the schedstat for that task. However activitymanager will get some more work done in kernel space. Finaly the new version also has the WASTE counter, but this one is the cost of only a rbtree search. Cost/Drawback The impact on the fork call is on a 2ghz arm64 is about 500ns for the notifier. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller 2017-02-09 13:21 [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree peter enderborg @ 2017-02-09 15:42 ` peter enderborg 2017-02-09 20:13 ` Greg Kroah-Hartman 2017-02-10 10:27 ` [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree Michal Hocko 1 sibling, 1 reply; 7+ messages in thread From: peter enderborg @ 2017-02-09 15:42 UTC (permalink / raw) To: devel, Greg Kroah-Hartman, linux-kernel, Arve Hjønnevåg, Riley Andrews, Linus Torvalds, linux-mm This collects stats for shrinker calls and how much waste work we do within the lowmemorykiller. Signed-off-by: Peter Enderborg <peter.enderborg@sonymobile.com> --- drivers/staging/android/Kconfig | 11 ++++ drivers/staging/android/Makefile | 1 + drivers/staging/android/lowmemorykiller.c | 9 ++- drivers/staging/android/lowmemorykiller_stats.c | 85 +++++++++++++++++++++++++ drivers/staging/android/lowmemorykiller_stats.h | 29 +++++++++ 5 files changed, 134 insertions(+), 1 deletion(-) create mode 100644 drivers/staging/android/lowmemorykiller_stats.c create mode 100644 drivers/staging/android/lowmemorykiller_stats.h diff --git a/drivers/staging/android/Kconfig b/drivers/staging/android/Kconfig index 6c00d6f..96e86c7 100644 --- a/drivers/staging/android/Kconfig +++ b/drivers/staging/android/Kconfig @@ -24,6 +24,17 @@ config ANDROID_LOW_MEMORY_KILLER scripts (/init.rc), and it defines priority values with minimum free memory size for each priority. +config ANDROID_LOW_MEMORY_KILLER_STATS + bool "Android Low Memory Killer: collect statistics" + depends on ANDROID_LOW_MEMORY_KILLER + default n + help + Create a file in /proc/lmkstats that includes + collected statistics about kills, scans and counts + and interaction with the shrinker. Its content + will be different depeding on lmk implementation used. + + source "drivers/staging/android/ion/Kconfig" endif # if ANDROID diff --git a/drivers/staging/android/Makefile b/drivers/staging/android/Makefile index 7ed1be7..d710eb2 100644 --- a/drivers/staging/android/Makefile +++ b/drivers/staging/android/Makefile @@ -4,3 +4,4 @@ obj-y += ion/ obj-$(CONFIG_ASHMEM) += ashmem.o obj-$(CONFIG_ANDROID_LOW_MEMORY_KILLER) += lowmemorykiller.o +obj-$(CONFIG_ANDROID_LOW_MEMORY_KILLER_STATS) += lowmemorykiller_stats.o diff --git a/drivers/staging/android/lowmemorykiller.c b/drivers/staging/android/lowmemorykiller.c index ec3b665..15c1b38 100644 --- a/drivers/staging/android/lowmemorykiller.c +++ b/drivers/staging/android/lowmemorykiller.c @@ -42,6 +42,7 @@ #include <linux/rcupdate.h> #include <linux/profile.h> #include <linux/notifier.h> +#include "lowmemorykiller_stats.h" static u32 lowmem_debug_level = 1; static short lowmem_adj[6] = { @@ -72,6 +73,7 @@ static unsigned long lowmem_deathpending_timeout; static unsigned long lowmem_count(struct shrinker *s, struct shrink_control *sc) { + lmk_inc_stats(LMK_COUNT); return global_node_page_state(NR_ACTIVE_ANON) + global_node_page_state(NR_ACTIVE_FILE) + global_node_page_state(NR_INACTIVE_ANON) + @@ -95,6 +97,7 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc) global_node_page_state(NR_SHMEM) - total_swapcache_pages(); + lmk_inc_stats(LMK_SCAN); if (lowmem_adj_size < array_size) array_size = lowmem_adj_size; if (lowmem_minfree_size < array_size) @@ -134,6 +137,7 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc) if (task_lmk_waiting(p) && time_before_eq(jiffies, lowmem_deathpending_timeout)) { task_unlock(p); + lmk_inc_stats(LMK_TIMEOUT); rcu_read_unlock(); return 0; } @@ -179,7 +183,9 @@ static unsigned long lowmem_scan(struct shrinker *s, struct shrink_control *sc) other_free * (long)(PAGE_SIZE / 1024)); lowmem_deathpending_timeout = jiffies + HZ; rem += selected_tasksize; - } + lmk_inc_stats(LMK_KILL); + } else + lmk_inc_stats(LMK_WASTE); lowmem_print(4, "lowmem_scan %lu, %x, return %lu\n", sc->nr_to_scan, sc->gfp_mask, rem); @@ -196,6 +202,7 @@ static struct shrinker lowmem_shrinker = { static int __init lowmem_init(void) { register_shrinker(&lowmem_shrinker); + init_procfs_lmk(); return 0; } device_initcall(lowmem_init); diff --git a/drivers/staging/android/lowmemorykiller_stats.c b/drivers/staging/android/lowmemorykiller_stats.c new file mode 100644 index 0000000..673691c --- /dev/null +++ b/drivers/staging/android/lowmemorykiller_stats.c @@ -0,0 +1,85 @@ +/* + * lowmemorykiller_stats + * + * Copyright (C) 2017 Sony Mobile Communications Inc. + * + * Author: Peter Enderborg <peter.enderborg@sonymobile.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ +/* This code is bookkeeping of statistical information + * from lowmemorykiller and provide a node in proc "/proc/lmkstats". + */ + +#include <linux/proc_fs.h> +#include <linux/seq_file.h> +#include "lowmemorykiller_stats.h" + +struct lmk_stats { + atomic_long_t scans; /* counter as in shrinker scans */ + atomic_long_t kills; /* the number of sigkills sent */ + atomic_long_t waste; /* the numer of extensive calls that did + * not lead to anything + */ + atomic_long_t timeout; /* counter for shrinker calls that needed + * to be cancelled due to pending kills + */ + atomic_long_t count; /* number of shrinker count calls */ + atomic_long_t unknown; /* internal */ +} st; + +void lmk_inc_stats(int key) +{ + switch (key) { + case LMK_SCAN: + atomic_long_inc(&st.scans); + break; + case LMK_KILL: + atomic_long_inc(&st.kills); + break; + case LMK_WASTE: + atomic_long_inc(&st.waste); + break; + case LMK_TIMEOUT: + atomic_long_inc(&st.timeout); + break; + case LMK_COUNT: + atomic_long_inc(&st.count); + break; + default: + atomic_long_inc(&st.unknown); + break; + } +} + +static int lmk_proc_show(struct seq_file *m, void *v) +{ + seq_printf(m, "kill: %ld\n", atomic_long_read(&st.kills)); + seq_printf(m, "scan: %ld\n", atomic_long_read(&st.scans)); + seq_printf(m, "waste: %ld\n", atomic_long_read(&st.waste)); + seq_printf(m, "timeout: %ld\n", atomic_long_read(&st.timeout)); + seq_printf(m, "count: %ld\n", atomic_long_read(&st.count)); + seq_printf(m, "unknown: %ld (internal)\n", + atomic_long_read(&st.unknown)); + + return 0; +} + +static int lmk_proc_open(struct inode *inode, struct file *file) +{ + return single_open(file, lmk_proc_show, PDE_DATA(inode)); +} + +static const struct file_operations lmk_proc_fops = { + .open = lmk_proc_open, + .read = seq_read, + .release = single_release +}; + +int __init init_procfs_lmk(void) +{ + proc_create_data(LMK_PROCFS_NAME, 0444, NULL, &lmk_proc_fops, NULL); + return 0; +} diff --git a/drivers/staging/android/lowmemorykiller_stats.h b/drivers/staging/android/lowmemorykiller_stats.h new file mode 100644 index 0000000..abeb6924 --- /dev/null +++ b/drivers/staging/android/lowmemorykiller_stats.h @@ -0,0 +1,29 @@ +/* + * lowmemorykiller_stats interface + * + * Copyright (C) 2017 Sony Mobile Communications Inc. + * + * Author: Peter Enderborg <peter.enderborg@sonymobile.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +enum lmk_kill_stats { + LMK_SCAN = 1, + LMK_KILL = 2, + LMK_WASTE = 3, + LMK_TIMEOUT = 4, + LMK_COUNT = 5 +}; + +#define LMK_PROCFS_NAME "lmkstats" + +#ifdef CONFIG_ANDROID_LOW_MEMORY_KILLER_STATS +void lmk_inc_stats(int key); +int __init init_procfs_lmk(void); +#else +static inline void lmk_inc_stats(int key) { return; }; +static inline int __init init_procfs_lmk(void) { return 0; }; +#endif -- 2.4.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller 2017-02-09 15:42 ` [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller peter enderborg @ 2017-02-09 20:13 ` Greg Kroah-Hartman 2017-02-10 7:12 ` peter enderborg 0 siblings, 1 reply; 7+ messages in thread From: Greg Kroah-Hartman @ 2017-02-09 20:13 UTC (permalink / raw) To: peter enderborg Cc: devel, linux-kernel, Arve Hjønnevåg, Riley Andrews, Linus Torvalds, linux-mm On Thu, Feb 09, 2017 at 04:42:35PM +0100, peter enderborg wrote: > This collects stats for shrinker calls and how much > waste work we do within the lowmemorykiller. > > Signed-off-by: Peter Enderborg <peter.enderborg@sonymobile.com> > --- > drivers/staging/android/Kconfig | 11 ++++ > drivers/staging/android/Makefile | 1 + > drivers/staging/android/lowmemorykiller.c | 9 ++- > drivers/staging/android/lowmemorykiller_stats.c | 85 +++++++++++++++++++++++++ > drivers/staging/android/lowmemorykiller_stats.h | 29 +++++++++ > 5 files changed, 134 insertions(+), 1 deletion(-) > create mode 100644 drivers/staging/android/lowmemorykiller_stats.c > create mode 100644 drivers/staging/android/lowmemorykiller_stats.h What changed from v1? > @@ -72,6 +73,7 @@ static unsigned long lowmem_deathpending_timeout; > static unsigned long lowmem_count(struct shrinker *s, > struct shrink_control *sc) > { > + lmk_inc_stats(LMK_COUNT); > return global_node_page_state(NR_ACTIVE_ANON) + > global_node_page_state(NR_ACTIVE_FILE) + > global_node_page_state(NR_INACTIVE_ANON) + Your email client is eating tabs and spitting out spaces, making this impossible to even consider being merged :( Please fix your email client, documentation for how to do so is in the kernel Documentation directory. thanks, greg k-h -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller 2017-02-09 20:13 ` Greg Kroah-Hartman @ 2017-02-10 7:12 ` peter enderborg 0 siblings, 0 replies; 7+ messages in thread From: peter enderborg @ 2017-02-10 7:12 UTC (permalink / raw) To: Greg Kroah-Hartman Cc: devel, linux-kernel, Arve Hjønnevåg, Riley Andrews, Linus Torvalds, linux-mm On 02/09/2017 09:13 PM, Greg Kroah-Hartman wrote: > On Thu, Feb 09, 2017 at 04:42:35PM +0100, peter enderborg wrote: >> This collects stats for shrinker calls and how much >> waste work we do within the lowmemorykiller. >> >> Signed-off-by: Peter Enderborg <peter.enderborg@sonymobile.com> >> --- >> drivers/staging/android/Kconfig | 11 ++++ >> drivers/staging/android/Makefile | 1 + >> drivers/staging/android/lowmemorykiller.c | 9 ++- >> drivers/staging/android/lowmemorykiller_stats.c | 85 +++++++++++++++++++++++++ >> drivers/staging/android/lowmemorykiller_stats.h | 29 +++++++++ >> 5 files changed, 134 insertions(+), 1 deletion(-) >> create mode 100644 drivers/staging/android/lowmemorykiller_stats.c >> create mode 100644 drivers/staging/android/lowmemorykiller_stats.h > What changed from v1? Nothing. I thought I found the reason why my tabs are replaced by spaces in transport. >> @@ -72,6 +73,7 @@ static unsigned long lowmem_deathpending_timeout; >> static unsigned long lowmem_count(struct shrinker *s, >> struct shrink_control *sc) >> { >> + lmk_inc_stats(LMK_COUNT); >> return global_node_page_state(NR_ACTIVE_ANON) + >> global_node_page_state(NR_ACTIVE_FILE) + >> global_node_page_state(NR_INACTIVE_ANON) + > Your email client is eating tabs and spitting out spaces, making this > impossible to even consider being merged :( > > Please fix your email client, documentation for how to do so is in the > kernel Documentation directory. > > thanks, > > greg k-h -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree 2017-02-09 13:21 [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree peter enderborg 2017-02-09 15:42 ` [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller peter enderborg @ 2017-02-10 10:27 ` Michal Hocko 2017-02-13 15:42 ` peter enderborg 1 sibling, 1 reply; 7+ messages in thread From: Michal Hocko @ 2017-02-10 10:27 UTC (permalink / raw) To: peter enderborg Cc: devel, Greg Kroah-Hartman, linux-kernel, Arve Hjønnevåg, Riley Andrews, Linus Torvalds, linux-mm [I have only now see this cover - it answers some of the questions I've had to specific patches. It would be really great if you could use git send-email to post patch series - it just does the right thing(tm)] On Thu 09-02-17 14:21:40, peter enderborg wrote: > Lowmemorykiller efficiency problem and a solution. > > Lowmemorykiller in android has a severe efficiency problem. The basic > problem is that the registered shrinker gets called very often without > anything actually happening. Which is an inherent problem because lkml doesn't belong to shrinkers infrastructure. -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree 2017-02-10 10:27 ` [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree Michal Hocko @ 2017-02-13 15:42 ` peter enderborg 2017-02-20 10:21 ` Michal Hocko 0 siblings, 1 reply; 7+ messages in thread From: peter enderborg @ 2017-02-13 15:42 UTC (permalink / raw) To: Michal Hocko Cc: devel, Greg Kroah-Hartman, linux-kernel, Arve Hjønnevåg, Riley Andrews, Linus Torvalds, linux-mm On 02/10/2017 11:27 AM, Michal Hocko wrote: > [I have only now see this cover - it answers some of the questions I've > had to specific patches. It would be really great if you could use git > send-email to post patch series - it just does the right thing(tm)] > > On Thu 09-02-17 14:21:40, peter enderborg wrote: >> Lowmemorykiller efficiency problem and a solution. >> >> Lowmemorykiller in android has a severe efficiency problem. The basic >> problem is that the registered shrinker gets called very often without >> anything actually happening. > Which is an inherent problem because lkml doesn't belong to shrinkers > infrastructure. Not really what this patch address. I see it as a problem with shrinker that there no slow-path-free (scan/count) where it should belong. This patch address a specific problem where lot of cpu are wasted in low memory conditions. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree 2017-02-13 15:42 ` peter enderborg @ 2017-02-20 10:21 ` Michal Hocko 0 siblings, 0 replies; 7+ messages in thread From: Michal Hocko @ 2017-02-20 10:21 UTC (permalink / raw) To: peter enderborg Cc: devel, Greg Kroah-Hartman, linux-kernel, Arve Hjønnevåg, Riley Andrews, Linus Torvalds, linux-mm On Mon 13-02-17 16:42:42, peter enderborg wrote: > On 02/10/2017 11:27 AM, Michal Hocko wrote: > > [I have only now see this cover - it answers some of the questions I've > > had to specific patches. It would be really great if you could use git > > send-email to post patch series - it just does the right thing(tm)] > > > > On Thu 09-02-17 14:21:40, peter enderborg wrote: > >> Lowmemorykiller efficiency problem and a solution. > >> > >> Lowmemorykiller in android has a severe efficiency problem. The basic > >> problem is that the registered shrinker gets called very often without > >> anything actually happening. > > Which is an inherent problem because lkml doesn't belong to shrinkers > > infrastructure. > > Not really what this patch address. I see it as a problem with shrinker > that there no slow-path-free (scan/count) where it should belong. > This patch address a specific problem where lot of cpu are wasted > in low memory conditions. Let me repeat. The specific problem you are trying to solve is _inherent_ to how the lmk is designed. Full stop. -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-02-20 10:21 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-02-09 13:21 [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree peter enderborg 2017-02-09 15:42 ` [PATCH 1/3 v2 staging-next] android: Collect statistics from lowmemorykiller peter enderborg 2017-02-09 20:13 ` Greg Kroah-Hartman 2017-02-10 7:12 ` peter enderborg 2017-02-10 10:27 ` [PATCH 0/3 staging-next] android: Lowmemmorykiller task tree Michal Hocko 2017-02-13 15:42 ` peter enderborg 2017-02-20 10:21 ` Michal Hocko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).