From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ian Campbell <ian.campbell@citrix.com>
Subject: Re: Problems with merlot* AMD Opteron 6376 systems (Was
 Re: stable trees (was: [xen-4.2-testing test] 58584: regressions))
Date: Fri, 26 Jun 2015 13:37:41 +0100
Message-ID: <1435322261.32500.182.camel@citrix.com>
References: <osstest-58584-mainreport@xen.org>
	<558151930200007800085E5C@mail.emea.novell.com>
	<21889.19282.1828.571256@mariner.uk.xensource.com>
	<5582C989020000780008696A@mail.emea.novell.com>
	<1434637353.28264.37.camel@citrix.com>
	<5584024B0200007800086F10@mail.emea.novell.com>
	<1434712049.28264.100.camel@citrix.com>
	<1435136770.28264.252.camel@citrix.com>
	<1435138723.28264.279.camel@citrix.com>
	<1435315055.32500.161.camel@citrix.com>
	<558D4A6E020000780008A2E7@mail.emea.novell.com>
	<1435317382.32500.168.camel@citrix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Received: from mail6.bemta14.messagelabs.com ([193.109.254.103])
	by lists.xen.org with esmtp (Exim 4.72)
	(envelope-from <prvs=612dc5eed=Ian.Campbell@citrix.com>)
	id 1Z8StD-0005wj-Gu
	for xen-devel@lists.xenproject.org; Fri, 26 Jun 2015 12:37:51 +0000
In-Reply-To: <1435317382.32500.168.camel@citrix.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Jan Beulich <JBeulich@suse.com>
Cc: Lars Kurth <lars.kurth@citrix.com>, Stefano Stabellini <stefano.stabellini@eu.citrix.com>, Andrew Cooper <Andrew.Cooper3@citrix.com>, Dario Faggioli <dario.faggioli@citrix.com>, Ian Jackson <Ian.Jackson@eu.citrix.com>, Aravind Gopalakrishnan <Aravind.Gopalakrishnan@amd.com>, Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>, Anthony Perard <anthony.perard@citrix.com>, xen-devel <xen-devel@lists.xenproject.org>, Boris Ostrovsky <boris.ostrovsky@oracle.com>
List-Id: xen-devel@lists.xenproject.org

On Fri, 2015-06-26 at 12:16 +0100, Ian Campbell wrote:
> On Fri, 2015-06-26 at 11:49 +0100, Jan Beulich wrote:
> > >>> On 26.06.15 at 12:37, <ian.campbell@citrix.com> wrote:
> > > At Andy Cooper's request I ran a quick job with mtrr.show=true
> > > http://logs.test-lab.xenproject.org/osstest/logs/58909/ 
> > > 
> > > I think the relevant serial output is:
> > > Jun 26 09:57:42.325077 (XEN) MTRR default type: uncachable
> > > Jun 26 09:57:42.325111 (XEN) MTRR fixed ranges enabled:
> > > Jun 26 09:57:42.333068 (XEN)   00000-9ffff write-back
> > > Jun 26 09:57:42.333101 (XEN)   a0000-bffff uncachable
> > > Jun 26 09:57:42.333128 (XEN)   c0000-fffff write-back
> > > Jun 26 09:57:42.341077 (XEN) MTRR variable ranges enabled:
> > > Jun 26 09:57:42.341110 (XEN)   0 base 000000000000 mask ffff80000000 write-back
> > > Jun 26 09:57:42.349088 (XEN)   1 base 000080000000 mask ffffc0000000 write-back
> > > Jun 26 09:57:42.349124 (XEN)   2 disabled
> > > Jun 26 09:57:42.357068 (XEN)   3 disabled
> > > Jun 26 09:57:42.357098 (XEN)   4 disabled
> > > Jun 26 09:57:42.357122 (XEN)   5 disabled
> > > Jun 26 09:57:42.357147 (XEN)   6 disabled
> > > Jun 26 09:57:42.365063 (XEN)   7 disabled
> > 
> > This alone would mean UC for all memory above 4G. But I seem to
> > recall AMD having some mechanism to avoid using MTRRs for this
> > case. Let me try to dig this out once back from lunch.
> 
> While you do that it seems like I may as well try a run with
> "e820-mtrr-clip" given to Xen.

According to http://logs.test-lab.xenproject.org/osstest/logs/58914/ it
didn't make any difference to the end result.

It did seems to cause a huge number of
Jun 26 11:51:29.933067 (XEN) AMD-Vi: IO_PAGE_FAULT: domain = 0, device id = 0x92, fault address = 0xbdfe7000, flags = 0
messages which weren't there before, not sure if that is a clue or not.

Ian.