Re: Kernel 3.11 / 3.12 OOM killer and Xen ballooning

[Bob Liu <bob.liu@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 2611 bytes --]

On 12/10/2013 11:27 PM, Konrad Rzeszutek Wilk wrote:
> On Tue, Dec 10, 2013 at 02:52:40PM +0000, James Dingwall wrote:
>> Konrad Rzeszutek Wilk wrote:
>>> On Mon, Dec 09, 2013 at 05:50:29PM +0000, James Dingwall wrote:
>>>> Hi,
>>>>
>>>> Since 3.11 I have noticed that the OOM killer quite frequently
>>>> triggers in my Xen guest domains which use ballooning to
>>>> increase/decrease their memory allocation according to their
>>>> requirements.  One example domain I have has a maximum memory
>>>> setting of ~1.5Gb but it usually idles at ~300Mb, it is also
>>>> configured with 2Gb swap which is almost 100% free.
>>>>
>>>> # free
>>>>              total       used       free     shared    buffers cached
>>>> Mem:        272080     248108      23972          0 1448      63064
>>>> -/+ buffers/cache:     183596      88484
>>>> Swap:      2097148          8    2097140
>>>>
>>>> There is plenty of available free memory in the hypervisor to
>>>> balloon to the maximum size:
>>>> # xl info | grep free_mem
>>>> free_memory            : 14923
>>>>
>>>> An example trace (they are always the same) from the oom killer in
>>>> 3.12 is added below.  So far I have not been able to reproduce this
>>>> at will so it is difficult to start bisecting it to see if a
>>>> particular change introduced this.  However it does seem that the
>>>> behaviour is wrong because a) ballooning could give the guest more
>>>> memory, b) there is lots of swap available which could be used as a
>>>> fallback.
> 
> Keep in mind that swap with tmem is actually no more swap. Heh, that
> sounds odd -but basically pages that are destined for swap end up
> going in the tmem code which pipes them up to the hypervisor.
> 
>>>>
>>>> If other information could help or there are more tests that I could
>>>> run then please let me know.
>>> I presume you have enabled 'tmem' both in the hypervisor and in
>>> the guest right?
>> Yes, domU and dom0 both have the tmem module loaded and  tmem
>> tmem_dedup=on tmem_compress=on is given on the xen command line.
> 
> Excellent. The odd thing is that your swap is not used that much, but
> it should be (as that is part of what the self-balloon is suppose to do).
> 
> Bob, you had a patch for the logic of how self-balloon is suppose
> to account for the slab - would this be relevant to this problem?
> 

Perhaps, I have attached the patch.
James, could you please apply it and try your application again? You
have to rebuild the guest kernel.
Oh, and also take a look at whether frontswap is in use, you can check
it by watching "cat /sys/kernel/debug/frontswap/*".

Thanks,
-Bob


[-- Attachment #2: balloon.patch --]
[-- Type: text/x-patch, Size: 1670 bytes --]

diff --git a/drivers/xen/xen-selfballoon.c b/drivers/xen/xen-selfballoon.c
index 21e18c1..4814759 100644
--- a/drivers/xen/xen-selfballoon.c
+++ b/drivers/xen/xen-selfballoon.c
@@ -191,6 +191,8 @@ static void selfballoon_process(struct work_struct *work)
 		tgt_pages = cur_pages; /* default is no change */
 		goal_pages = vm_memory_committed() +
 				totalreserve_pages +
+				global_page_state(NR_SLAB_RECLAIMABLE) +
+				global_page_state(NR_SLAB_UNRECLAIMABLE) +
 				MB2PAGES(selfballoon_reserved_mb);
 #ifdef CONFIG_FRONTSWAP
 		/* allow space for frontswap pages to be repatriated */
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 580a5f0..863b05c 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -16,6 +16,7 @@
 
 #include <linux/stddef.h>
 #include <linux/mm.h>
+#include <linux/mman.h>
 #include <linux/swap.h>
 #include <linux/interrupt.h>
 #include <linux/pagemap.h>
@@ -3075,7 +3076,7 @@ void show_free_areas(unsigned int filter)
 		" dirty:%lu writeback:%lu unstable:%lu\n"
 		" free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n"
 		" mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n"
-		" free_cma:%lu\n",
+		" free_cma:%lu totalram:%lu balloontarget:%lu\n",
 		global_page_state(NR_ACTIVE_ANON),
 		global_page_state(NR_INACTIVE_ANON),
 		global_page_state(NR_ISOLATED_ANON),
@@ -3093,7 +3094,9 @@ void show_free_areas(unsigned int filter)
 		global_page_state(NR_SHMEM),
 		global_page_state(NR_PAGETABLE),
 		global_page_state(NR_BOUNCE),
-		global_page_state(NR_FREE_CMA_PAGES));
+		global_page_state(NR_FREE_CMA_PAGES),
+		totalram_pages,
+		vm_memory_committed() + totalreserve_pages);
 
 	for_each_populated_zone(zone) {
 		int i;

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel