linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Dr. David Alan Gilbert" <dave@treblig.org>
To: Marc Haber <mh+linux-kernel@zugschlus.de>
Cc: Borislav Petkov <bp@alien8.de>,
	Paolo Bonzini <pbonzini@redhat.com>,
	linux-kernel@vger.kernel.org, kvm ML <kvm@vger.kernel.org>
Subject: Re: transparent huge pages breaks KVM on AMD.
Date: Fri, 13 May 2016 09:35:45 +0100	[thread overview]
Message-ID: <20160513083545.GA1307@gallifrey> (raw)
In-Reply-To: <20160512202009.GZ9143@torres.zugschlus.de>

* Marc Haber (mh+linux-kernel@zugschlus.de) wrote:
> Hi David,
> 
> On Sat, Apr 23, 2016 at 07:52:46PM +0100, Dr. David Alan Gilbert wrote:
> > Hmm, your problem does sound like bad hardware, but....
> > If you've got a nice reliable crash, can you try turning transparent huge pages
> > off on the host;
> >    echo never > /sys/kernel/mm/transparent_hugepage/enabled
> 
> I must have missed this hint in the middle of the "your hardware is
> bad" avalance that came over me.
> 
> I spent two weeks bisecting "good" kernels since during the repeated
> reconfigurations, transparent huge pages got turned off in kernel
> configuration. After running each kernel for 24 hours, I eventually
> ended up with a working 4.5 kernel. The configuration diff was short,
> showing transparent huge pages, and - finally - upon re-reading the
> thread I found your hint.

OK, good.  When I sent that mail I'd hit a THP bug but in a
corner of migration and at the time we didn't know why and there was
no reason to think it would cause any other symptoms, but since it was
also between 4.4 and 4.5 it did seem worth mentioning as a long shot,
but it was no more than a long shot.

> I have now the result that 4.5, 4.5.1 and 4.5.4 corrupt KVM guest
> memory reliably in the first hour of running under disk load, causing
> the VM to either drop dead in the water, or to read randomness from
> disk. Rebooting fixes the VM. This happens as soon as transparent huge
> pages are turned on in the host.
> 
> Turning off transparent huge pages by echo never >
> /sys/kernel/mm/transparent_hugepage/enabled fixes the issue even
> without rebooting the host. Start up the VM again and it works just
> fine.
> 
> Is this an issue in (a) transparent huge pages, (b) KVM or (c) qemu?
> Where should this issue be forwarded? Or do we just accept it and turn
> transparent huge pages off?

Try Andrea's fix for (a).

Dave

> 
> Greetings
> Marc
> 
> -- 
> -----------------------------------------------------------------------------
> Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
> Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
> Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/

  parent reply	other threads:[~2016-05-13  8:36 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-17 16:54 Major KVM issues with kernel 4.5 on the host Marc Haber
2016-03-17 18:11 ` Borislav Petkov
2016-03-18 10:01   ` Paolo Bonzini
2016-04-13 18:37     ` Marc Haber
2016-04-13 20:36       ` Paolo Bonzini
2016-04-13 20:52         ` Marc Haber
2016-04-13 22:29         ` Marc Haber
2016-04-14  1:16           ` Paolo Bonzini
2016-04-14  5:22             ` Marc Haber
2016-04-21  8:39               ` Marc Haber
2016-04-21 12:37                 ` Borislav Petkov
2016-04-21 14:50                   ` Marc Haber
2016-04-21 16:51                     ` Borislav Petkov
2016-04-21 20:04                       ` Marc Haber
2016-04-23 16:04                         ` Borislav Petkov
2016-04-23 18:43                           ` Marc Haber
2016-04-23 18:52                             ` Dr. David Alan Gilbert
2016-05-12 20:20                               ` transparent huge pages breaks KVM on AMD Marc Haber
2016-05-12 20:24                                 ` Kirill A. Shutemov
2016-05-12 20:34                                   ` Marc Haber
2016-05-12 20:42                                     ` Kirill A. Shutemov
2016-05-13  5:23                                       ` Marc Haber
2016-05-13  8:07                                         ` Borislav Petkov
2016-05-13  8:09                                           ` Borislav Petkov
2016-05-13 13:21                                             ` Marc Haber
2016-05-13 16:08                                               ` Borislav Petkov
2016-05-14  6:19                                             ` Marc Haber
2016-05-13  9:08                                           ` Marc Haber
2016-05-13  9:19                                             ` Borislav Petkov
2016-05-13 14:59                                           ` Marc Haber
2016-05-13  8:35                                 ` Dr. David Alan Gilbert [this message]
2016-05-13 14:03                                   ` Marc Haber
2016-04-23 23:57                             ` Major KVM issues with kernel 4.5 on the host Borislav Petkov
2016-04-14  6:07             ` Marc Haber
2016-04-14 16:47             ` Marc Haber
2016-04-14 17:30               ` Paolo Bonzini
2016-04-14 17:47                 ` Marc Haber
2016-03-18 18:49   ` Marc Haber
2016-03-18 22:04     ` Borislav Petkov
2016-03-19  0:08       ` Marc Haber
2016-03-20 13:31         ` Borislav Petkov
2016-03-20 17:14           ` Andrey Korolyov
2016-03-20 18:25             ` Borislav Petkov
2016-03-20 18:42               ` Andrey Korolyov
2016-03-20 18:58                 ` Borislav Petkov
2016-04-13 18:22                   ` Marc Haber
2016-04-13 20:37                     ` Paolo Bonzini
2016-04-13 18:20           ` Marc Haber
2016-03-21  9:08         ` Paolo Bonzini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160513083545.GA1307@gallifrey \
    --to=dave@treblig.org \
    --cc=bp@alien8.de \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mh+linux-kernel@zugschlus.de \
    --cc=pbonzini@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).