From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35379) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YS1G4-0006yD-QB for qemu-devel@nongnu.org; Sun, 01 Mar 2015 05:38:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YS1G1-00076E-Ik for qemu-devel@nongnu.org; Sun, 01 Mar 2015 05:38:00 -0500 Received: from vps01.wiesinger.com ([46.36.37.179]:59852) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YS1G1-0006K2-87 for qemu-devel@nongnu.org; Sun, 01 Mar 2015 05:37:57 -0500 Message-ID: <54F2EBA5.4050907@wiesinger.com> Date: Sun, 01 Mar 2015 11:36:21 +0100 From: Gerhard Wiesinger MIME-Version: 1.0 References: <54AE87C1.2060907@wiesinger.com> <54AEBD43.2060705@redhat.com> <54AEC877.9080600@wiesinger.com> <54AECAF3.3060909@redhat.com> <54AF047D.8010009@wiesinger.com> <54B3B2F5.1090405@wiesinger.com> <54B57C51.7090002@wiesinger.com> <54B584AB.4090303@redhat.com> <54B58AC0.5080805@wiesinger.com> <54B58B18.9060205@redhat.com> <54B595C7.3080101@wiesinger.com> <54B5BF5F.9000805@redhat.com> <54B633CE.3040901@wiesinger.com> <54E05659.9050701@wiesinger.com> <54E1FC2B.3030805@redhat.com> <54E20812.4090006@wiesinger.com> <54E20CD5.3050909@redhat.com> In-Reply-To: <54E20CD5.3050909@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Fedora FC21 - Bug: 100% CPU and hangs in gettimeofday(&tp, NULL); forever List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , Laine Stump , qemu-devel@nongnu.org, Cole Robinson , virt@lists.fedoraproject.org On 16.02.2015 16:29, Paolo Bonzini wrote: > > On 16/02/2015 16:09, Gerhard Wiesinger wrote: >> On 16.02.2015 15:18, Paolo Bonzini wrote: >>> On 15/02/2015 09:18, Gerhard Wiesinger wrote: >>>> Can you grab some random backtraces ("thread apply all bt full") with >>>> gdb? >> Very low load on the machine, so I guess most will be sleeping and hard >> to catch non sleeping ones. See below > This likely means that the 100% threads are not the I/O (event loop) > threads, but the VCPU threads. ok, happened again, details below. Any further ideas from the stack traces? So far it happened only the PostgreSQL database VM. Kernel is alive (ping works well). ssh is not working. console window: after entering one character at login prompt, then crashed: [1438.384864] Out of memory: Kill process 10115 (pg_dump) score 112 or sacrifice child [1438.384990] Killed process 10115 (pg_dump) total-vm: 340548kB, anon-rss: 162712kB, file-rss: 220kB VM uptime is ~1 day, 2 cores 100%CPU. VM is very stripped down, nevertheless is should have enough memory (and also swap). Looks like it crashed at the nighlty backup (pg_dumpall), command is: ssh -x ${REMOTE_USER}@${REMOTE_HOST} "pg_dumpall" | bzip2 -9 > ${DEST_SQL} free total used free shared buff/cache available Mem: 372264 28536 214780 20884 128948 303360 Swap: 1081340 0 1081340 Ciao, Gerhard [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". 0x00007feee62412c1 in ppoll () from /lib64/libc.so.6 Thread 4 (Thread 0x7feed3fff700 (LWP 4636)): #0 0x00007feee6242977 in ioctl () at /lib64/libc.so.6 #1 0x00007feef11d1c35 in kvm_vcpu_ioctl () #2 0x00007feef11d1cec in kvm_cpu_exec () #3 0x00007feef11bfb02 in qemu_kvm_cpu_thread_fn () #4 0x00007feeefcb352a in start_thread () at /lib64/libpthread.so.0 #5 0x00007feee624c79d in clone () at /lib64/libc.so.6 Thread 3 (Thread 0x7feed37fe700 (LWP 4637)): #0 0x00007feee6242977 in ioctl () at /lib64/libc.so.6 #1 0x00007feef11d1c35 in kvm_vcpu_ioctl () #2 0x00007feef11d1cec in kvm_cpu_exec () #3 0x00007feef11bfb02 in qemu_kvm_cpu_thread_fn () #4 0x00007feeefcb352a in start_thread () at /lib64/libpthread.so.0 #5 0x00007feee624c79d in clone () at /lib64/libc.so.6 Thread 2 (Thread 0x7feed1bff700 (LWP 4653)): #0 0x00007feeefcb8590 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0 #1 0x00007feef1466d79 in qemu_cond_wait () #2 0x00007feef13eadd3 in vnc_worker_thread_loop () #3 0x00007feef13eb1b8 in vnc_worker_thread () #4 0x00007feeefcb352a in start_thread () at /lib64/libpthread.so.0 #5 0x00007feee624c79d in clone () at /lib64/libc.so.6 Thread 1 (Thread 0x7feef103fa80 (LWP 4563)): #0 0x00007feee62412c1 in ppoll () at /lib64/libc.so.6 #1 0x00007feef13fc89c in qemu_poll_ns () #2 0x00007feef13fc034 in main_loop_wait () #3 0x00007feef1197cdd in main ()