From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45353) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WFi1e-0000RP-Ay for qemu-devel@nongnu.org; Tue, 18 Feb 2014 05:35:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WFi1W-0008TL-TD for qemu-devel@nongnu.org; Tue, 18 Feb 2014 05:35:42 -0500 Received: from mx1.redhat.com ([209.132.183.28]:39917) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WFi1W-0008TB-Ki for qemu-devel@nongnu.org; Tue, 18 Feb 2014 05:35:34 -0500 Date: Tue, 18 Feb 2014 12:38:28 +0200 From: "Michael S. Tsirkin" Message-ID: <20140218103828.GB8114@redhat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] hotplug: VM got stuck when attaching a pass-through device to the non-pass-through VM for the first time List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Zhanghaoyu (A)" Cc: "chenliang (T)" , Avi Kivity , "Huangweidong (C)" , KVM , Gleb Natapov , Luonengjun , "qemu-devel@nongnu.org" , "Gonglei (Arei)" , Paolo Bonzini , "Gaowei (UVP)" On Tue, Feb 18, 2014 at 02:38:40AM +0000, Zhanghaoyu (A) wrote: > Hi, all > > The VM will get stuck for a while(about 6s for a VM with 20GB memory) when attaching a pass-through PCI card to the non-pass-through VM for the first time. > The reason is that the host will build the whole VT-d GPA->HPA DMAR page-table, which needs a lot of time, and during this time, the qemu_global_mutex > lock is hold by the main-thread, if the vcpu thread IOCTL return, it will be blocked to waiting main-thread to release the qemu_global_mutex lock, > so the VM got stuck. > The race between qemu-main-thread and vcpu-thread is shown as below, > > QEMU-main-thread vcpu-thread > | | > qemu_mutex_lock_iothread qemu_mutex_lock(&qemu_global_mutex) > | | > +----loop- ->+ +----loop---->+ > | | | | > | qemu_mutex_unlock_iothread | qemu_mutex_unlock_iothread > | | | | > | poll | kvm_vcpu_ioctl(KVM_RUN) > | | | | > | qemu_mutex_lock_iothread | | > | | | | > -------------------------------------------------------------------------------------- > | | | qemu_mutex_lock_iothread > | kvm_device_pci_assign | | > | | | blocked to waiting main-thread to release the qemu lock > | about 6 sec for 20GB memory | | > | | | | > +------------+ +-------------+ > > > Any advises? > > Thanks, > Zhang Haoyu What if you detach and re-attach? Is it fast then? If yes this means the issue is COW breaking that occurs with get_user_pages, not translation as such. Try hugepages with prealloc - does it help? -- MST