From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52490) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WLGG5-0002z2-8s for qemu-devel@nongnu.org; Wed, 05 Mar 2014 13:09:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WLGFz-0006SH-IW for qemu-devel@nongnu.org; Wed, 05 Mar 2014 13:09:33 -0500 Received: from mx.ipv6.kamp.de ([2a02:248:0:51::16]:40649 helo=mx01.kamp.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WLGFz-0006Rd-83 for qemu-devel@nongnu.org; Wed, 05 Mar 2014 13:09:27 -0500 Message-ID: <53176849.10804@kamp.de> Date: Wed, 05 Mar 2014 19:09:13 +0100 From: Peter Lieven MIME-Version: 1.0 References: <530DBE6C.5030502@kamp.de> <20140226154154.GB20820@stefanha-thinkpad.muc.redhat.com> <530E0FF0.20501@kamp.de> <20140227085711.GC21749@stefanha-thinkpad.redhat.com> <53109E99.3020102@kamp.de> <20140303120349.GA21055@stefanha-thinkpad.redhat.com> <53147385.2090906@kamp.de> <20140304092456.GF25676@stefanha-thinkpad.redhat.com> <53173841.1020905@kamp.de> <53174877.90509@kamp.de> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] qemu-img convert cache mode for source List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Marcus Cc: Kevin Wolf , Stefan Hajnoczi , "qemu-devel@nongnu.org" , Stefan Hajnoczi , Paolo Bonzini Am 05.03.2014 18:38, schrieb Marcus: > On Wed, Mar 5, 2014 at 8:53 AM, Peter Lieven wrote: >> Am 05.03.2014 16:20, schrieb Marcus: >>> I think this is a more generic sysadmin problem. I've seen the same >>> thing in the past with simply snapshotting a logical volume or zfs >>> zvol and copying it off somewhere. Page cache bloats, the system >>> starts swapping. To avoid it, we wrote a small C program that calls >>> FADV_DONTNEED on a file, and fork off a process to call it on the >>> source file every X seconds in our backup scripts. >> I do not call FADV_DONTNEED on the whole file, but only >> on the block that has just been read. > Yes, I suppose that's one of the advantages of having it integrated > into the reader. > >>> It's a little strange to me to have qemu-img do this, just like it >>> would be strange if 'cp' did it, but I can see it as a very useful >>> shortcut if it's an optional flag. qemu-img to me is just an admin >>> tool, and the admin should decide if they want their tool's reads >>> cached. Some additional things that come to mind: >>> >>> * If you are running qemu-img on a running VM's source file, >>> FADV_DONTNEED may ruin the cache you wanted if the VM is not running >>> cache=none. >> You would normally not run it on the source directly. In my case >> I run it on a snapshot of an logical volume, but I see your point. > Totally depends on the situation, just thought it was worth consideration. Yes, and it was a good remark. > >> So you can confirm my oberservations and would be happy if >> this behaviour could be toggled with a cmdline switch? > Yes, I've seen the same behavior you mention just with 'cp'. It was > with a version of the CentOS 6.2 kernel, at least, before we added > FADV_DONTNEED into the backup scripts. Ok, Stefan would you be happy with it? > >>> * O_DIRECT I think will cause unexpected problems, for example the >>> zfsonlinux guys (and tmpfs as mentioned) don't yet support it. If it >>> is used, there has to be a fallback or a way to turn it off. >> I don't use O_DIRECT. Its an option for the destination file only at the >> moment. You can set it with -t none as qemu-img argument. > I just mentioned it because setting it on the source was suggested > originally and subsequently discussed. Yes, but it would break readahead and would not work and tmpfs and many other things we don't see know. Peter