From mboxrd@z Thu Jan 1 00:00:00 1970 From: "M. Ivanov" Subject: Re: About Xen bridged pci devices and suspend/resume for the X10SAE motherboard (SuperMicro) Date: Sat, 15 Aug 2015 01:08:27 +0300 Message-ID: <1439590107.7011.39.camel@gmail.com> References: <1439161898.27696.190.camel@gmail.com> <20150810135836.GB11826@l.oracle.com> <1439216068.425.11.camel@gmail.com> <20150810144735.GI11826@l.oracle.com> <1439230895.425.22.camel@gmail.com> <20150813194906.GA25713@l.oracle.com> <1439561312.7011.26.camel@gmail.com> <20150814161556.GA5802@l.oracle.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0900651652785111412==" Return-path: In-Reply-To: <20150814161556.GA5802@l.oracle.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org --===============0900651652785111412== Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-sATKX7Yihn++HeyyPNhD" --=-sATKX7Yihn++HeyyPNhD Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, 2015-08-14 at 12:15 -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Aug 14, 2015 at 05:08:32PM +0300, M. Ivanov wrote: > > On Thu, 2015-08-13 at 15:49 -0400, Konrad Rzeszutek Wilk wrote: > > > On Mon, Aug 10, 2015 at 09:21:35PM +0300, M. Ivanov wrote: > > > > On Mon, 2015-08-10 at 10:47 -0400, Konrad Rzeszutek Wilk wrote: > > > > > On Mon, Aug 10, 2015 at 05:14:28PM +0300, M. Ivanov wrote: > > > > > > On Mon, 2015-08-10 at 09:58 -0400, Konrad Rzeszutek Wilk wrote: > > > > > > > On Mon, Aug 10, 2015 at 02:11:38AM +0300, M. Ivanov wrote: > > > > > > > > Hello, > > > > > > > >=20 > > > > > > > > excuse me for bothering you, but I've read an old thread on= a mailing > > > > > > > > list about X10SAE compatibility.=20 > > > > > > > > http://lists.xen.org/archives/html/xen-devel/2014-02/msg021= 11.html > > > > > > >=20 > > > > > > > CC-ing Xen devel. > > > > > > > >=20 > > > > > > > > Currently I own this board and am trying to use it with Xen= and be able > > > > > > > > to suspend and resume. > > > > > > > >=20 > > > > > > > > But I am getting errors from the USB 3 Renesas controller a= bout parity > > > > > > > > in my bios event log, and my system hangs on resume, > > > > > > > > so I was wondering if that is connected to the bridge(tundr= a) you've > > > > > > > > mentioned. > > > > > > >=20 > > > > > > > Did you update the BIOS to the latest version? > > > > > > Will updating to version 3 solve my issue? > > > > > > Can you do a suspend/resume on your X10SAE? > > > > >=20 > > > > > It did work at some point. I will find out when I am at home late= r today. > > > > >=20 > > > > Looking forward to your reply and am really thankful for your time, > > > > so far I've tried changing many of the settings in the bios, > > > > fiddling with Xen's kernel params, > > > > blacklisting the xhci driver, doing a xl detach. > > > >=20 > > > > The only thing I haven't done yet is updating the bios, > > > > but Supermicro's support couldn't give me a changelog: > > > >=20 > > > > "The primary objective for ver3.0 BIOS release is to support Intel > > > > Broadwell CPUs" > > > > "We do not know if BIOS update will fix the issue you are seeing as= we > > > > never tested it with Xen." > > >=20 > > > I did test it remotely and it did something very odd. It suspended an= d then > > > immediately resumed with tons of VT-d errors!? > > >=20 > > > I will try again but be actually right by it. > > Thanks for your effort, > > Can you suggest a way for me to log what is happening? >=20 > I usually have an serial cable attached to it and log that. > Cranking up the debug on everything gives me some idea. Reasonable enough, will do. > >=20 > > Since my machine just hangs up and I don't get a picture on the screen. > > Or when I do(sometimes) - it's just a cursor,(on a black screen with > > nothing else,no errors shown) and the machine doesn't react to any key > > combinations. >=20 > Yeah. That is frustrating. > >=20 > > On a side note: > >=20 > > I did try updating the bios, > > but got some really strange result. > > The first time I got checksums about everything OK, > > (erase,flash,verify), > > but then it said - FDT is locked! > > And I am at least 90% sure I've enabled reflash in=20 > > the BIOS setup prior to flashing. > > So I've restarted but didn't clear the CMOS(through the jumper on the > > board). And it said Bios v 3.0 in the setup, then I also did the > > Reset to optmized defaults. > >=20 > > Tried suspending and couldn't resume like always. > > But perhaps currently my BIOS is in a broken/corrupted state. > >=20 > > After that I've tried reflashing the bios again. But this time -=20 > > after the messages about Erasing,Flashing,Verifying: > > it just froze. I didn't get any message about FDT, restarting or > > whatsoever. > >=20 > > So I rebooted and it seems to work like before, > > says version 3.0 in the BIOS setup. > > I wonder if I should try clearing the CMOS and > > flashing again. >=20 > One test I hadn't done is to try to suspend/resume > under baremental Linux and see how that works. Does it work for you? >=20 Yep, without Xen I can suspend fine. > The DMI tells me: > [ 7.276963] Hardware name: Supermicro X10SAE/X10SAE, BIOS 2.00 04/21/2= 014 >=20 > So a bit older BIOS. So we've had the same bios versions. But now I have a broken BIOS 3.0.. (probably). Maybe I will try reflashing to 2.0 if you have hints about getting suspend/resume on it. > The last thing I see: >=20 >=20 > # dmesg | grep -i Super > [ 0.000000] DMI: Supermicro X10SAE/X10SAE, BIOS 2.00 04/21/2014 > [ 0.000000] ACPI: RSDP 0x00000000000F0490 000024 (v02-MB 01072009 AMI = 00010013) > [ 0.000000] ACPI: DSDT 0x000000009B9DA1E8 00CDDA (v02 SUPERM SMCI--MB = 00000000 INTL 20120711) > [ 0.000000] ACPI: APIC 0x000000009B9E70D8 000092 (v03 SUPERM SMCI--MB = 01072009 AMI 00010013) > [ 0.000000] ACPI: FPDT 0x000000009B9E7170 000044 (v01 SUPERM SMCI--MB = 01072009 AMI 00010013) > [ 0.000000] ACPI: MCFG 0x000000009B9E9488 00003C (v01 SUPERM SMCI--MB = 01072009 MSFT 00000097) > [ 0.000000] ACPI: HPET 0x000000009B9E94C8 000038 (v01 SUPERM SMCI--MB = 01072009 AMI. 00000005) > [ 7.623166] Hardware name: Supermicro X10SAE/X10SAE, BIOS 2.00 04/21/2= 014 > # =07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07= =07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07=07pm-sus=07= poe=07=08 =08=08 =08# =1B[Jcd /sys/power > # ls > disk pm_freeze_timeout pm_trace resume > image_size pm_print_times pm_trace_dev_match state > pm_async pm_test reserved_size wakeup_count > # c# =1B[Jcat state > freeze standby disk > # echo standby > state Since I am using CentOS in my experiments(version 6.7 has no systemd support and 7 has no Xen support...)=20 I do a echo mem > /sys/power/state. > [ 204.755164] PM: Syncing filesystems ... done. > [ 204.755223] PM: Preparing system for standby sleep > [ 204.755606] Freezing user space processes ... (elapsed 0.000 seconds) = done. > [ 204.756506] Freezing re=C3=BD+=C3=8B+=C3=8B=C2=95=C3=BFezable tasks ..= . (elapsed 0.001 seconds) done. > [ 204.757607] PM: Entering standby sleep > [ 204.757762] Suspending console(s affinity for irq 40 > (XEN) Broke affinity for irq 41 > (XEN) Broke affinity for irq 23 > (XEN) Broke affinity for irq 16 > (XEN) Broke affinity fouppress EOI broadcast on CPU#3 > (XEN) masked ExtINT on CPU#3 > (XEN) Suppress EOI broadcast on CPU#4 > (XEN) masked ExtINT on CPU#4 > (XEN) Suppress EOI broadcast on CPU#5 > (XEN) masked ExtINT on CPU#5 > (XEN) Suppress EOI broadcast on CPU#6 > (XEN) masked ExtINT on CPU#6 > (XEN) Suppress EOI broadcast on CPU#7 > (XEN) masked ExtINT on CPU#7 > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 00000000c0000081 from 0xe0= 23e00800000000 to 0x0023001000000000. > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 00000000c0000082 from 0xff= ff83082727b000 to 0xffffffff8172e910. > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 00000000c0000083 from 0xff= ff83082727b080 to 0xffffffff81730c40. > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 0000000000000174 from 0x00= 0000000000e008 to 0x0000000000000010. > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 0000000000000175 from 0xff= ff83082727ffc0 to 0x0000000000000000. > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 0000000000000176 from 0xff= ff82d08022c7d0 to 0xffffffff81730c70. > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 00000000c0000083 from 0xff= ff83082727b080 to 0xffffffff81730eb0. > (XEN) traps.c:2653:d0v1 Domain attempted WRMSR 00000000c0000084 from 0x00= 00000000074700 to 0x0000000000047700. > (XEN) d0 attempted to change d0v1's CR4 flags 00002660 -> 00042760 > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 00000000c0000081 from 0xe0= 23e00800000000 to 0x0023001000000000. > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 00000000c0000082 from 0xff= ff82d0803db000 to 0xffffffff8172e910. > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730c40. > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 0000000000000174 from 0x00= 0000000000e008 to 0x0000000000000010. > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 0000000000000175 from 0xff= ff82d0803dffc0 to 0x0000000000000000. > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 0000000000000176 from 0xff= ff82d08022c7d0 to 0xffffffff81730c70. > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730eb0. > (XEN) traps.c:2653:d0v2 Domain attempted WRMSR 00000000c0000084 from 0x00= 00000000074700 to 0x0000000000047700. > (XEN) d0 attempted to change d0v2's CR4 flags 00002660 -> 00042760 > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 00000000c0000081 from 0xe0= 23e00800000000 to 0x0023001000000000. > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 00000000c0000082 from 0xff= ff82d0803db000 to 0xffffffff8172e910. > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730c40. > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 0000000000000174 from 0x00= 0000000000e008 to 0x0000000000000010. > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 0000000000000175 from 0xff= ff82d0803dffc0 to 0x0000000000000000. > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 0000000000000176 from 0xff= ff82d08022c7d0 to 0xffffffff81730c70. > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730eb0. > (XEN) traps.c:2653:d0v3 Domain attempted WRMSR 00000000c0000084 from 0x00= 00000000074700 to 0x0000000000047700. > (XEN) d0 attempted to change d0v3's CR4 flags 00002660 -> 00042760 > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 00000000c0000081 from 0xe0= 23e00800000000 to 0x0023001000000000. > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 00000000c0000082 from 0xff= ff82d0803db000 to 0xffffffff8172e910. > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730c40. > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 0000000000000174 from 0x00= 0000000000e008 to 0x0000000000000010. > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 0000000000000175 from 0xff= ff82d0803dffc0 to 0x0000000000000000. > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 0000000000000176 from 0xff= ff82d08022c7d0 to 0xffffffff81730c70. > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730eb0. > (XEN) traps.c:2653:d0v4 Domain attempted WRMSR 00000000c0000084 from 0x00= 00000000074700 to 0x0000000000047700. > (XEN) d0 attempted to change d0v4's CR4 flags 00002660 -> 00042760 > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 00000000c0000081 from 0xe0= 23e00800000000 to 0x0023001000000000. > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 00000000c0000082 from 0xff= ff8308272f3000 to 0xffffffff8172e910. > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 00000000c0000083 from 0xff= ff8308272f3080 to 0xffffffff81730c40. > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 0000000000000174 from 0x00= 0000000000e008 to 0x0000000000000010. > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 0000000000000175 from 0xff= ff8308272f7fc0 to 0x0000000000000000. > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 0000000000000176 from 0xff= ff82d08022c7d0 to 0xffffffff81730c70. > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 00000000c0000083 from 0xff= ff8308272f3080 to 0xffffffff81730eb0. > (XEN) traps.c:2653:d0v5 Domain attempted WRMSR 00000000c0000084 from 0x00= 00000000074700 to 0x0000000000047700. > (XEN) d0 attempted to change d0v5's CR4 flags 00002660 -> 00042760 > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 00000000c0000081 from 0xe0= 23e00800000000 to 0x0023001000000000. > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 00000000c0000082 from 0xff= ff82d0803db000 to 0xffffffff8172e910. > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730c40. > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 0000000000000174 from 0x00= 0000000000e008 to 0x0000000000000010. > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 0000000000000175 from 0xff= ff82d0803dffc0 to 0x0000000000000000. > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 0000000000000176 from 0xff= ff82d08022c7d0 to 0xffffffff81730c70. > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 00000000c0000083 from 0xff= ff82d0803db080 to 0xffffffff81730eb0. > (XEN) traps.c:2653:d0v6 Domain attempted WRMSR 00000000c0000084 from 0x00= 00000000074700 to 0x0000000000047700. > (XEN) d0 attempted to change d0v6's CR4 flags 00002660 -> 00042760 > (XEN) event_fifo.c:50:d0v6 domain 0, port 50 already on a queue > (XEN) traps.c:2653:d0v7 Domain attempted WRMSR 00000000c0000081 from 0xe0= 23e00800000000 to 0x0023001000000000. > (XEN) traps.c:2653:d0v7 Domain attempted WRMSR 00000000c0000082 from 0xff= ff83082727b000 to 0xffffffff8172e910. > (XEN) traps.c:2653:d0v7 Domain attempted WRMSR 00000000c0000083 from 0xff= ff83082727b080 to 0xffffffff81730c40. > (XEN) traps.c:2653:d0v7 Domain attempted WRMSR 0000000000000174 from 0x00= 0000000000e008 to 0x0000000000000010. > (XEN) traps.c:2653:d0v7 Domain attempR 00000000c0000084 from 0x0000000000= 074700 to 0x0000000000047700. > (XEN) d0 attempted to change d0v7's CR4 flags 00002660 -> 00042760 > ^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G^G(XEN) Domain 0 cras= hed: rebooting machine in 5 seconds. >=20 > telnet> Connection closed. > [Connecting to system 50 ] >=20 > Great, so 'Domain 0 crashed'! >=20 > Pff - and oddly enough the same kernel suspend/resumes fine on my desktop= and laptop. And yes, it's not about the kernel, it's about Xen. I've tried like at least 7 Linux kernels from 3.11 to 3.19. Also did you get the parity error in your bios event log? About the Bus06 or Bus05(if you have nothing in your PCI-E slots). Which I think is the Renesas USB 3 controller. > >=20 > > > > > > > >=20 > > > > > > > > I will be very glad if you could share any information rega= rding this > > > > > > > > matter.=20 > > > > > > > >=20 > > > > > > > > Best regards, > > > > > > > > M. Ivanov > > > > > > >=20 > > > > > > >=20 > > > > > >=20 > > > > >=20 > > > > >=20 > > > >=20 > > >=20 > > >=20 > >=20 >=20 >=20 --=-sATKX7Yihn++HeyyPNhD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAABCgAGBQJVzmbbAAoJEExnsX0sda0osZkIALVT5qnGPphySa2NQPRCyZH+ mES27Yu6xo5MCTM44D2yi67rRKKFjiQO3H6MQwOZARPTBm3IrV8VUgyb3YrrXiav bcHleR6nTVBzcBUoHdkq/HiEdNAVw17G7riw3ZnwUlQjCB2qcEdGE6b3KNCkfx62 hogwRmtkw9ZJV3X4sPfc94uptFPDZo35PpNCz6v4Hz7qsdF5nrvCy8xeO6DiOsCo YXaacCaAem3nxSt1VBBKz3AWvsHjGg9jjTQEGn+k15UJY0kOwVwilYWIFaRDwWXQ eZ7bZ/8V9nejXrnYg0G/BokERgAwHkXuYydhcePC/7L++ARTRNkMExELtqDgVFM= =yBu1 -----END PGP SIGNATURE----- --=-sATKX7Yihn++HeyyPNhD-- --===============0900651652785111412== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============0900651652785111412==--