From mboxrd@z Thu Jan  1 00:00:00 1970
From: Sasha Levin <levinsasha928@gmail.com>
Subject: Re: [PATCH v2 6/8] kvm tools: Add rwlock wrapper
Date: Fri, 03 Jun 2011 10:27:58 +0300
Message-ID: <1307086079.13088.8.camel@lappy>
References: <BANLkTin7mAbLG_bbqxkUen0S6dzEc04NEQ@mail.gmail.com>
	 <1306748796.14564.62.camel@lappy>
	 <BANLkTinQ9_kJGUHBPaFgQFMkA57DtJbc4Q@mail.gmail.com>
	 <20110530095451.GB8461@elte.hu>
	 <20110530201110.f3bf20b5.yoshikawa.takuya@oss.ntt.co.jp>
	 <1306753954.14564.92.camel@lappy>
	 <20110530202646.eff0ea28.yoshikawa.takuya@oss.ntt.co.jp>
	 <4DE381DB.8040804@redhat.com> <20110530114949.GD22324@elte.hu>
	 <4DE387DA.3020307@redhat.com>  <20110530123602.GK22324@elte.hu>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Cc: Avi Kivity <avi@redhat.com>,
	Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>,
	Pekka Enberg <penberg@kernel.org>, kvm@vger.kernel.org,
	asias.hejun@gmail.com, gorcunov@gmail.com,
	prasadjoshi124@gmail.com,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	takuya.yoshikawa@gmail.com
To: Ingo Molnar <mingo@elte.hu>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mail-ww0-f42.google.com ([74.125.82.42]:51517 "EHLO
	mail-ww0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1752098Ab1FCH2E (ORCPT <rfc822;kvm@vger.kernel.org>);
	Fri, 3 Jun 2011 03:28:04 -0400
Received: by wwk4 with SMTP id 4so4846143wwk.1
        for <kvm@vger.kernel.org>; Fri, 03 Jun 2011 00:28:03 -0700 (PDT)
In-Reply-To: <20110530123602.GK22324@elte.hu>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Mon, 2011-05-30 at 14:36 +0200, Ingo Molnar wrote:
> Right now i can produce something similar to it: 42 vcpus on a single 
> CPU:
> 
> 	$ taskse 1 kvm run --cpus 42
> 
> And that hangs early on during bootup, around:
> 
> [    0.236000] Disabled fast string operations
> [    0.242000]  #4
> [    0.270000] Disabled fast string operations
> [    0.275000]  #5
> [    0.317000] Disabled fast string operations
> [    0.322000]  #6
> [    0.352000] Disabled fast string operations
> [    0.358000]  #7
> [    0.414000] Disabled fast string operations
> 
> The threads seem to be livelocked:
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                    
> 22227 mingo     20   0  471g  95m  904 R 12.9  0.8   0:06.39 kvm                                        
> 22230 mingo     20   0  471g  95m  904 R 12.9  0.8   0:06.36 kvm                                        
> 22226 mingo     20   0  471g  95m  904 R 11.9  0.8   0:07.04 kvm                                        
> 22228 mingo     20   0  471g  95m  904 R 11.9  0.8   0:06.38 kvm                                        
> 22229 mingo     20   0  471g  95m  904 R 11.9  0.8   0:06.37 kvm                                        
> 22231 mingo     20   0  471g  95m  904 R 11.9  0.8   0:06.37 kvm                                        
> 22232 mingo     20   0  471g  95m  904 R 11.9  0.8   0:06.36 kvm                                        
> 22233 mingo     20   0  471g  95m  904 R 11.9  0.8   0:06.33 kvm                                        
>     7 root      -2  19     0    0    0 S  2.0  0.0   1:12.53 rcuc0             
> 
> with no apparent progress being made.

Since it's something that worked in 2.6.37, I've looked into it to find
what might have caused this issue.

I've bisected guest kernels and found that the problem starts with:

a26ac2455ffcf3be5c6ef92bc6df7182700f2114 is the first bad commit
commit a26ac2455ffcf3be5c6ef92bc6df7182700f2114
Author: Paul E. McKenney <paul.mckenney@linaro.org>
Date:   Wed Jan 12 14:10:23 2011 -0800

    rcu: move TREE_RCU from softirq to kthread

Ingo, could you confirm that the problem goes away for you when you use
an earlier commit?
-- 

Sasha.