qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Gerhard Wiesinger <lists@wiesinger.com>
To: Paolo Bonzini <pbonzini@redhat.com>,
	Laine Stump <laine@redhat.com>,
	qemu-devel@nongnu.org, Cole Robinson <crobinso@redhat.com>,
	virt@lists.fedoraproject.org
Subject: Re: [Qemu-devel] Fedora FC21 - Bug: 100% CPU and hangs in gettimeofday(&tp, NULL); forever
Date: Mon, 02 Mar 2015 16:52:20 +0100	[thread overview]
Message-ID: <54F48734.7020800@wiesinger.com> (raw)
In-Reply-To: <54F42CC7.20504@redhat.com>

On 02.03.2015 10:26, Paolo Bonzini wrote:
>
> On 01/03/2015 11:36, Gerhard Wiesinger wrote:
>> So far it happened only the PostgreSQL database VM. Kernel is alive
>> (ping works well). ssh is not working.
>> console window: after entering one character at login prompt, then crashed:
>> [1438.384864] Out of memory: Kill process 10115 (pg_dump) score 112 or
>> sacrifice child
>> [1438.384990] Killed process 10115 (pg_dump) total-vm: 340548kB,
>> anon-rss: 162712kB, file-rss: 220kB
> Can you get a vmcore or at least sysrq-t output?

Yes, next time it happens I can analyze it.

I think there are 2 problems:
1.) OOM (Out of Memory) problem with the low memory settings and kernel 
settings (see below)
2.) Instability problem which might have a dependency to 1.)

What I've done so far (thanks to Andrey Korolyov for ideas and help):
a.) Updated maschine type from pc-0.15 to pc-i440fx-2.2
virsh dumpxml database | grep "<type"
     <type arch='x86_64' machine='pc-0.15'>hvm</type>

virsh edit database
virsh dumpxml database | grep "<type"
     <type arch='x86_64' machine='pc-i440fx-2.2'>hvm</type>

SMBIOS is updated therefore from 2.4 to 2.8:
dmesg|grep -i SMBIOS
[    0.000000] SMBIOS 2.8 present.
b.) Switched to tsc clock, kernel parameters: clocksource=tsc nohz=off 
highres=off
c.) Changed overcommit to 1
echo "vm.overcommit_memory = 1" > /etc/sysctl.d/overcommit.conf
d.) Tried 1 VCPU instead of 2
e.) Installed 512MB vRAM instead of 384MB
f.) Prepared for sysrq and vmcore
echo "kernel.sysrq = 1" > /etc/sysctl.d/sysrq.conf
sysctl -w kernel.sysrq=1
virsh send-key database KEY_LEFTALT KEY_SYSRQ KEY_T
virsh dump domain-name /tmp/dumpfile
g.) Further ideas, not yet done: disable memory balooning by 
blacklisting baloon driver or remove from virsh xml config

Summary:
1.) 512MB, tsc timer, 1VCPU, vm.overcommit_memory = 1: no OOM problem, 
no crash
2.) 512MB, kvm_clock, 2VCPU, vm.overcommit_memory = 1: no OOM problem, 
no crash

So the OOM problem seems to be solved (at least it didn't happen so far) 
by installing 512MB RAM and setting vm.overcommit_memory = 1 (I guess 
just setting overcommit would be fine, too).

Instability didn't occour so far. If I can't reproduce it, I'll revert 
the settings.

Ciao,
Gerhard

  reply	other threads:[~2015-03-02 15:53 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-01-08 13:36 [Qemu-devel] Fedora FC21 - Bug: 100% CPU and hangs in gettimeofday(&tp, NULL); forever Gerhard Wiesinger
2015-01-08 17:24 ` Paolo Bonzini
2015-01-08 18:12   ` Gerhard Wiesinger
2015-01-08 18:22     ` Paolo Bonzini
2015-01-08 22:28       ` Gerhard Wiesinger
2015-01-08 22:42         ` Paolo Bonzini
2015-01-12 11:41         ` Gerhard Wiesinger
2015-01-12 11:46           ` Paolo Bonzini
2015-01-12 11:53             ` Gerhard Wiesinger
2015-01-12 11:54               ` Paolo Bonzini
2015-01-13 20:13           ` Gerhard Wiesinger
2015-01-13 20:48             ` Paolo Bonzini
2015-01-13 21:08               ` Cole Robinson
2015-01-13 21:14               ` Gerhard Wiesinger
2015-01-13 21:16                 ` Paolo Bonzini
2015-01-13 22:01                   ` Gerhard Wiesinger
2015-01-14  0:59                     ` Laine Stump
2015-01-14  9:15                       ` Gerhard Wiesinger
2015-02-15  8:18                         ` Gerhard Wiesinger
2015-02-16 14:18                           ` Paolo Bonzini
2015-02-16 15:09                             ` Gerhard Wiesinger
2015-02-16 15:29                               ` Paolo Bonzini
2015-03-01 10:36                                 ` Gerhard Wiesinger
2015-03-02  9:26                                   ` Paolo Bonzini
2015-03-02 15:52                                     ` Gerhard Wiesinger [this message]
2015-03-02 17:15                                       ` Gerhard Wiesinger
2015-03-03  9:12                                         ` Gerhard Wiesinger
2015-03-03 10:12                                           ` Paolo Bonzini
2015-03-03 11:51                                           ` Gonglei
2015-03-03 12:28                                           ` Gerhard Wiesinger
2015-03-03 13:18                                             ` Gerhard Wiesinger
2015-03-03 20:50                                               ` Gerhard Wiesinger
2015-01-14 17:47             ` Juan Quintela
2015-01-14 17:52               ` Juan Quintela
2015-01-14 22:19                 ` Gerhard Wiesinger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=54F48734.7020800@wiesinger.com \
    --to=lists@wiesinger.com \
    --cc=crobinso@redhat.com \
    --cc=laine@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=virt@lists.fedoraproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).