From mboxrd@z Thu Jan 1 00:00:00 1970 From: "azurIt" Subject: =?utf-8?q?Re=3A_=5Bpatch_0=2F7=5D_improve_memcg_oom_killer_robustness_v2?= Date: Mon, 16 Sep 2013 22:52:46 +0200 Message-ID: <20130916225246.A633145B@pobox.sk> References: <20130911191150.GN856@cmpxchg.org>, <20130911214118.7CDF2E71@pobox.sk>, <20130911200426.GO856@cmpxchg.org>, <20130914124831.4DD20346@pobox.sk>, <20130916134014.GA3674@dhcp22.suse.cz>, <20130916160119.2E76C2A1@pobox.sk>, <20130916140607.GC3674@dhcp22.suse.cz>, <20130916161316.5113F6E7@pobox.sk>, <20130916145744.GE3674@dhcp22.suse.cz>, <20130916170543.77F1ECB4@pobox.sk> <20130916152548.GF3674@dhcp22.suse.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20130916152548.GF3674@dhcp22.suse.cz> Sender: owner-linux-mm@kvack.org To: =?utf-8?q?Michal_Hocko?= Cc: =?utf-8?q?Johannes_Weiner?= , =?utf-8?q?Andrew_Morton?= , =?utf-8?q?David_Rientjes?= , =?utf-8?q?KAMEZAWA_Hiroyuki?= , =?utf-8?q?KOSAKI_Motohiro?= , linux-mm@kvack.org, cgroups@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org List-Id: linux-arch.vger.kernel.org > CC: "Johannes Weiner" , "Andrew Morton" , "David Rientjes" , "KAMEZAWA Hiroy= uki" , "KOSAKI Motohiro" , linux-mm@kvack.org, cgroups@vger.kernel.org, x86@kernel= .org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org >On Mon 16-09-13 17:05:43, azurIt wrote: >> > CC: "Johannes Weiner" , "Andrew Morton" , "David Rientjes" , "KAMEZAWA Hi= royuki" , "KOSAKI Motohiro" , linux-mm@kvack.org, cgroups@vger.kernel.org, x86@ker= nel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org >> >On Mon 16-09-13 16:13:16, azurIt wrote: >> >[...] >> >> >You can use sysrq+l via serial console to see tasks hogging the CP= U or >> >> >sysrq+t to see all the existing tasks. >> >>=20 >> >>=20 >> >> Doesn't work here, it just prints 'l' resp. 't'. >> > >> >I am using telnet for accessing my serial consoles exported by >> >the multiplicator or KVM and it can send sysrq via ctrl+t (Send >> >Break). Check your serial console setup. >>=20 >>=20 >>=20 >> I'm using Raritan KVM and i created keyboard macro 'sysrq + l' resp. >> 'sysrq + t'. I'm also unable to use it on my local PC. Maybe it needs >> to be enabled somehow? > >Probably yes. echo 1 > /proc/sys/kernel/sysrq should enable all sysrq >commands. You can select also some of them (have a look at >Documentation/sysrq.txt for more information) Now it happens again and i was just looking on the server's htop. I'm sur= e that this time it was only one process (apache) running under user acco= unt (not root). It was taking about 100% CPU (about 100% of one core). I = was able to kill it by hand inside htop but everything was very slow, ser= ver load was immediately on 500. I'm sure it must be related to that Joha= nnes kernel patches because i'm also using i/o throttling in cgroups via = Block IO controller so users are unable to create such a huge I/O. I will= try to take stacks of processes but i'm not able to identify the problem= atic process so i will have to take them from *all* apache processes whil= e killing them. azur -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gmmr2.centrum.cz ([46.255.227.252]:58408 "EHLO gmmr2.centrum.cz" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750948Ab3IPUwt (ORCPT ); Mon, 16 Sep 2013 16:52:49 -0400 Subject: =?utf-8?q?Re=3A_=5Bpatch_0=2F7=5D_improve_memcg_oom_killer_robustness_v2?= Date: Mon, 16 Sep 2013 22:52:46 +0200 From: "azurIt" References: <20130911191150.GN856@cmpxchg.org>, <20130911214118.7CDF2E71@pobox.sk>, <20130911200426.GO856@cmpxchg.org>, <20130914124831.4DD20346@pobox.sk>, <20130916134014.GA3674@dhcp22.suse.cz>, <20130916160119.2E76C2A1@pobox.sk>, <20130916140607.GC3674@dhcp22.suse.cz>, <20130916161316.5113F6E7@pobox.sk>, <20130916145744.GE3674@dhcp22.suse.cz>, <20130916170543.77F1ECB4@pobox.sk> <20130916152548.GF3674@dhcp22.suse.cz> In-Reply-To: <20130916152548.GF3674@dhcp22.suse.cz> MIME-Version: 1.0 Message-ID: <20130916225246.A633145B@pobox.sk> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-arch-owner@vger.kernel.org List-ID: To: =?utf-8?q?Michal_Hocko?= Cc: =?utf-8?q?Johannes_Weiner?= , =?utf-8?q?Andrew_Morton?= , =?utf-8?q?David_Rientjes?= , =?utf-8?q?KAMEZAWA_Hiroyuki?= , =?utf-8?q?KOSAKI_Motohiro?= , linux-mm@kvack.org, cgroups@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org Message-ID: <20130916205246.q7yogNi5S6IYd1YEOUTHs1gX-RDmIwcqQijaUxnfPck@z> > CC: "Johannes Weiner" , "Andrew Morton" , "David Rientjes" , "KAMEZAWA Hiroyuki" , "KOSAKI Motohiro" , linux-mm@kvack.org, cgroups@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org >On Mon 16-09-13 17:05:43, azurIt wrote: >> > CC: "Johannes Weiner" , "Andrew Morton" , "David Rientjes" , "KAMEZAWA Hiroyuki" , "KOSAKI Motohiro" , linux-mm@kvack.org, cgroups@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org >> >On Mon 16-09-13 16:13:16, azurIt wrote: >> >[...] >> >> >You can use sysrq+l via serial console to see tasks hogging the CPU or >> >> >sysrq+t to see all the existing tasks. >> >> >> >> >> >> Doesn't work here, it just prints 'l' resp. 't'. >> > >> >I am using telnet for accessing my serial consoles exported by >> >the multiplicator or KVM and it can send sysrq via ctrl+t (Send >> >Break). Check your serial console setup. >> >> >> >> I'm using Raritan KVM and i created keyboard macro 'sysrq + l' resp. >> 'sysrq + t'. I'm also unable to use it on my local PC. Maybe it needs >> to be enabled somehow? > >Probably yes. echo 1 > /proc/sys/kernel/sysrq should enable all sysrq >commands. You can select also some of them (have a look at >Documentation/sysrq.txt for more information) Now it happens again and i was just looking on the server's htop. I'm sure that this time it was only one process (apache) running under user account (not root). It was taking about 100% CPU (about 100% of one core). I was able to kill it by hand inside htop but everything was very slow, server load was immediately on 500. I'm sure it must be related to that Johannes kernel patches because i'm also using i/o throttling in cgroups via Block IO controller so users are unable to create such a huge I/O. I will try to take stacks of processes but i'm not able to identify the problematic process so i will have to take them from *all* apache processes while killing them. azur