From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: does anyone run guests for more than 5 minutes? (virtio-net perf anomaly) Date: Tue, 03 Mar 2009 13:13:41 -0700 Message-ID: <1236111221.13193.223.camel@bling> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit To: kvm-devel Return-path: Received: from g4t0015.houston.hp.com ([15.201.24.18]:1619 "EHLO g4t0015.houston.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750946AbZCCUNt (ORCPT ); Tue, 3 Mar 2009 15:13:49 -0500 Received: from g4t0009.houston.hp.com (g4t0009.houston.hp.com [16.234.32.26]) by g4t0015.houston.hp.com (Postfix) with ESMTP id 7E8BA8423 for ; Tue, 3 Mar 2009 20:13:47 +0000 (UTC) Received: from ldl.fc.hp.com (ldl.fc.hp.com [15.11.146.30]) by g4t0009.houston.hp.com (Postfix) with ESMTP id 710FAC082 for ; Tue, 3 Mar 2009 20:13:47 +0000 (UTC) Received: from localhost (ldl.fc.hp.com [127.0.0.1]) by ldl.fc.hp.com (Postfix) with ESMTP id 2347139C011 for ; Tue, 3 Mar 2009 13:13:47 -0700 (MST) Received: from ldl.fc.hp.com ([127.0.0.1]) by localhost (ldl.fc.hp.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3wqovf+Npp4K for ; Tue, 3 Mar 2009 13:13:45 -0700 (MST) Received: from [192.168.1.60] (g3t0027.americas.hpqcorp.net [16.232.34.131]) by ldl.fc.hp.com (Postfix) with ESMTP id 3DF1439C07C for ; Tue, 3 Mar 2009 13:13:43 -0700 (MST) Sender: kvm-owner@vger.kernel.org List-ID: It seems like something happens around the 5 minute uptime in the guest that causes virtio-net throughput to plummet. Here's the scenario: guest started as: taskset -c 4 /usr/local/bin/qemu-system-x86_64 -hda /dev/sdb -m 2048 -vnc :1 -net nic,macaddr=02:00:10:91:73:02,model=virtio -net tap,script=$HOME/bin/null-ifup -serial tcp::1234,server,nowait -mem-path /hugepages/ null-ifup looks like this: #!/bin/sh /sbin/ifconfig $1 192.168.0.1 /sbin/route add -net 192.168.0.0/24 gw 192.168.0.1 The guest gets a static IP of 192.168.0.2. netserver (part of netperf) in the host is pinned to CPU0, which shares cache with CPU4 from the above taskset. When the guest boots, I run: netperf -c -C -H 192.168.0.1 -t TCP_STREAM -- -m 64k This results in ~13.5Gbps (note you won't get close to this if you don't get the tasksets correct) Wait 5 minutes, retry. Now I get ~4Gbps. The only way I can get 13.5Gbps again is by rebooting the guest within the same qemu context, or of course restarting it completely. Any guesses as to what might be going on? Can anyone reproduce? I'm hoping that I'm doing something dumb, but can't figure out what it is. The system is running v2.6.29-rc6-121-g64e7130 in the guest, v2.6.29-rc6-123-gbd7b3b4 on the host, kvm module kvm-84-620-g5bffffc and userspace kvm-84-95-gea1b668. Thanks, Alex