From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ryan Harper Subject: Re: Huge memory leak in virtio, see kvm-Bugs-2989366 Date: Tue, 20 Apr 2010 20:58:03 -0500 Message-ID: <20100421015803.GV24351@us.ibm.com> References: <20100420222956.GA13218@moo.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm@vger.kernel.org To: Leszek Urbanski Return-path: Received: from e6.ny.us.ibm.com ([32.97.182.146]:41932 "EHLO e6.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752638Ab0DUB6J (ORCPT ); Tue, 20 Apr 2010 21:58:09 -0400 Received: from d01relay05.pok.ibm.com (d01relay05.pok.ibm.com [9.56.227.237]) by e6.ny.us.ibm.com (8.14.3/8.13.1) with ESMTP id o3L1tmnP015291 for ; Tue, 20 Apr 2010 21:55:48 -0400 Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by d01relay05.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o3L1w832176844 for ; Tue, 20 Apr 2010 21:58:08 -0400 Received: from d01av01.pok.ibm.com (loopback [127.0.0.1]) by d01av01.pok.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id o3L1w7Kg023216 for ; Tue, 20 Apr 2010 21:58:08 -0400 Content-Disposition: inline In-Reply-To: <20100420222956.GA13218@moo.pl> Sender: kvm-owner@vger.kernel.org List-ID: * Leszek Urbanski [2010-04-20 17:37]: > Hi, > > this is a follow-up to bug 2989366: > > https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2989366&group_id=180599 > > after extensive debugging with the guys on #kvm it turns out that the leak is > in the qemu-kvm userland process, in virtio-blk. > > A summary of my setup is described in the bug report above. > > The affected guests have a common load profile: frequent sequential I/O on > large (~ 2 GB) files. > > I tried switching off or changing almost all options in my qemu command > line and the only option that makes a difference is -drive if=virtio. > > When an affected guest is run with virtio drives the qemu-kvm process starts > leaking immediately after startup and grows (for the most heavily leaking > guests) by ~ 1 GB RSS for every ten hours (and keeps growing until OOM). > > With -drive if=ide or scsi, it doesn't leak at all. > > A diff of /proc//maps of an affected qemu-kvm at startup and after > 1.5 hrs: > > -039b9000-5ccd0000 rw-p 00000000 00:00 0 > +039b9000-65803000 rw-p 00000000 00:00 0 > > (a heap leak?) > > I'm willing to debug further. The problem is 100% reproducible. Is that qemu-kvm 0.12.3 compiled from source? or using the distro package? If you drop the -smp 4 part, you could also try plain qemu to eliminate if there was a qemu-kvm merge issue. Also, if you switch to a different guest do you still see the same leak? This should help determine if the virtio-blk front end is part of the issue. -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh@us.ibm.com