From: AP <apxeng@gmail.com>
To: Jan Beulich <JBeulich@suse.com>
Cc: "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
"Keir (Xen.org)" <keir@xen.org>,
Ian Campbell <Ian.Campbell@citrix.com>,
Andrew Cooper <andrew.cooper3@citrix.com>,
Ian Jackson <Ian.Jackson@eu.citrix.com>,
Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: Re: [xen-unstable test] 11946: regressions - FAIL
Date: Mon, 7 May 2012 18:29:17 +0000 [thread overview]
Message-ID: <CAGU+auuOa04LSn_ebNRXyJQ=bn4PpLiL2ejYaqaivFKLEdnabQ@mail.gmail.com> (raw)
In-Reply-To: <4FA7EB80020000780008204B@nat28.tlf.novell.com>
[-- Attachment #1.1: Type: text/plain, Size: 10773 bytes --]
On Mon, May 7, 2012 at 1:34 PM, Jan Beulich <JBeulich@suse.com> wrote:
>
> >>> On 07.05.12 at 13:50, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> > On 07/05/2012 09:10, Jan Beulich wrote:
> >>>>> On 05.05.12 at 02:21, AP <apxeng@gmail.com> wrote:
> >>> (XEN) *** IRQ BUG found ***
> >>> (XEN) CPU0 -Testing vector 236 from bitmap
> >> 236 = 0xec = FIRST_LEGACY_VECTOR + 0x0c, i.e. an IRQ12 coming
> >> in through the 8259A. Something fundamentally fishy must be going
> >> on here, and I would suppose the code in question shouldn't even be
> >> reached for legacy vectors.
> >>
> >> Furthermore, calling dump_irqs() from the debugging code with
> >> desc->lock still held makes it impossible to get full output, as that
> >> function wants to lock all initialized IRQ descriptors.
> >
> > Yes - it has been vector 236 on each of the 3 reported failures from AP,
> > and I believe it was also vector 236 in the one case I managed to
> > reproduce the issue.
> >
> > However, once we have set up the IO-APIC, the 8259A should not be used
> > any more. The boot dmeg shows that io_ack_method is indeed "old" (which
> > was going to be my first suggestion), and that EOI Broadcast Suppression
> > is enabled, which I have already identified as a source of problems for
> > some customers. As a 'fix', I provided the ability for
> > "io_ack_method=new" to prevent EOI Broadcast Suppression being enabled.
> > This was upstreamed in c/s 24870:9bf3ec036bef, but apparently has not
> > completely fixed the customer problems - just made it substantially more
> > rare.
> >
> > AP: Can you manually invoke the 'i' debug key and provide that - it will
> > help to see how Xen is setting up the IO-APIC(s) on your system.
(XEN) Guest interrupt information:
(XEN) IRQ: 0 affinity:01 vec:f0 type=IO-APIC-edge status=00000000
mapped, unbound
(XEN) IRQ: 1 affinity:02 vec:85 type=IO-APIC-edge status=00000030
in-flight=0 domain-list=0: 1(----),
(XEN) IRQ: 2 affinity:ff vec:e2 type=XT-PIC status=00000000
mapped, unbound
(XEN) IRQ: 3 affinity:01 vec:40 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 4 affinity:01 vec:48 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 5 affinity:01 vec:50 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 6 affinity:01 vec:58 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 7 affinity:01 vec:60 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 8 affinity:08 vec:29 type=IO-APIC-edge status=00000030
in-flight=0 domain-list=0: 8(----),
(XEN) IRQ: 9 affinity:02 vec:7f type=IO-APIC-level status=00000010
in-flight=0 domain-list=0: 9(----),
(XEN) IRQ: 10 affinity:01 vec:78 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 11 affinity:01 vec:88 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 12 affinity:08 vec:d4 type=IO-APIC-edge status=00000030
in-flight=0 domain-list=0: 12(----),
(XEN) IRQ: 13 affinity:0f vec:98 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 14 affinity:01 vec:a0 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 15 affinity:01 vec:a8 type=IO-APIC-edge status=00000002
mapped, unbound
(XEN) IRQ: 16 affinity:02 vec:a6 type=IO-APIC-level status=00000030
in-flight=0 domain-list=0: 16(----),
(XEN) IRQ: 17 affinity:0f vec:c0 type=IO-APIC-level status=00000002
mapped, unbound
(XEN) IRQ: 18 affinity:0f vec:c8 type=IO-APIC-level status=00000002
mapped, unbound
(XEN) IRQ: 19 affinity:0f vec:f1 type=IO-APIC-level status=00000000
mapped, unbound
(XEN) IRQ: 20 affinity:0f vec:61 type=IO-APIC-level status=00000002
mapped, unbound
(XEN) IRQ: 22 affinity:0f vec:32 type=IO-APIC-level status=00000002
mapped, unbound
(XEN) IRQ: 23 affinity:01 vec:ac type=IO-APIC-level status=00000030
in-flight=0 domain-list=0: 23(----),
(XEN) IRQ: 24 affinity:01 vec:28 type=DMA_MSI status=00000000
mapped, unbound
(XEN) IRQ: 25 affinity:01 vec:30 type=DMA_MSI status=00000000
mapped, unbound
(XEN) IRQ: 26 affinity:01 vec:31 type=PCI-MSI/-X status=00000030
in-flight=0 domain-list=0:279(----),
(XEN) IRQ: 27 affinity:01 vec:39 type=PCI-MSI/-X status=00000030
in-flight=0 domain-list=0:278(----),
(XEN) IRQ: 28 affinity:01 vec:41 type=PCI-MSI/-X status=00000030
in-flight=0 domain-list=0:277(----),
(XEN) IRQ: 29 affinity:01 vec:49 type=PCI-MSI/-X status=00000030
in-flight=0 domain-list=0:276(----),
(XEN) IRQ: 30 affinity:01 vec:51 type=PCI-MSI/-X status=00000030
in-flight=0 domain-list=0:275(----),
(XEN) IRQ: 31 affinity:04 vec:d7 type=PCI-MSI status=00000030
in-flight=0 domain-list=0:274(----),
(XEN) IRQ: 32 affinity:04 vec:df type=PCI-MSI status=00000030
in-flight=0 domain-list=0:273(----),
(XEN) IRQ: 33 affinity:02 vec:b0 type=PCI-MSI status=00000010
in-flight=0 domain-list=0:272(----),
(XEN) IRQ: 34 affinity:02 vec:a8 type=PCI-MSI status=00000010
in-flight=0 domain-list=0:271(----),
(XEN) IRQ: 35 affinity:04 vec:ad type=PCI-MSI status=00000030
in-flight=0 domain-list=0:270(----),
(XEN) IO-APIC interrupt information:
(XEN) IRQ 0 Vec240:
(XEN) Apic 0x00, Pin 2: vec=f0 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 1 Vec133:
(XEN) Apic 0x00, Pin 1: vec=85 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 3 Vec 64:
(XEN) Apic 0x00, Pin 3: vec=40 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 4 Vec 72:
(XEN) Apic 0x00, Pin 4: vec=48 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 5 Vec 80:
(XEN) Apic 0x00, Pin 5: vec=50 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 6 Vec 88:
(XEN) Apic 0x00, Pin 6: vec=58 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 7 Vec 96:
(XEN) Apic 0x00, Pin 7: vec=60 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 8 Vec 41:
(XEN) Apic 0x00, Pin 8: vec=29 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 9 Vec127:
(XEN) Apic 0x00, Pin 9: vec=7f delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=L mask=0 dest_id:0
(XEN) IRQ 10 Vec120:
(XEN) Apic 0x00, Pin 10: vec=78 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 11 Vec136:
(XEN) Apic 0x00, Pin 11: vec=88 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 12 Vec212:
(XEN) Apic 0x00, Pin 12: vec=d4 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 13 Vec152:
(XEN) Apic 0x00, Pin 13: vec=98 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=1 dest_id:0
(XEN) IRQ 14 Vec160:
(XEN) Apic 0x00, Pin 14: vec=a0 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 15 Vec168:
(XEN) Apic 0x00, Pin 15: vec=a8 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 16 Vec166:
(XEN) Apic 0x00, Pin 16: vec=a6 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN) IRQ 17 Vec192:
(XEN) Apic 0x00, Pin 17: vec=c0 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN) IRQ 18 Vec200:
(XEN) Apic 0x00, Pin 18: vec=c8 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN) IRQ 19 Vec241:
(XEN) Apic 0x00, Pin 19: vec=f1 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN) IRQ 20 Vec 97:
(XEN) Apic 0x00, Pin 20: vec=61 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN) IRQ 22 Vec 50:
(XEN) Apic 0x00, Pin 22: vec=32 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN) IRQ 23 Vec172:
(XEN) Apic 0x00, Pin 23: vec=ac delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=0 dest_id:0
> Seeing the 'z' output might also be helpful, especially to see whether
> any of the IO-APICs' RTEs is an ExtINT one.
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #2 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #2......
(XEN) .... register #00: 02000000
(XEN) ....... : physical APIC id: 02
(XEN) ....... : Delivery Type: 0
(XEN) ....... : LTS : 0
(XEN) .... register #01: 00170020
(XEN) ....... : max redirection entries: 0017
(XEN) ....... : PRQ implemented: 0
(XEN) ....... : IO APIC version: 0020
(XEN) .... IRQ redirection table:
(XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN) 00 000 00 1 0 0 0 0 0 0 00
(XEN) 01 000 00 0 0 0 0 0 1 1 85
(XEN) 02 000 00 0 0 0 0 0 1 1 F0
(XEN) 03 000 00 0 0 0 0 0 1 1 40
(XEN) 04 000 00 0 0 0 0 0 1 1 48
(XEN) 05 000 00 0 0 0 0 0 1 1 50
(XEN) 06 000 00 0 0 0 0 0 1 1 58
(XEN) 07 000 00 0 0 0 0 0 1 1 60
(XEN) 08 000 00 0 0 0 0 0 1 1 29
(XEN) 09 000 00 0 1 0 0 0 1 1 A7
(XEN) 0a 000 00 0 0 0 0 0 1 1 78
(XEN) 0b 000 00 0 0 0 0 0 1 1 88
(XEN) 0c 000 00 0 0 0 0 0 1 1 D4
(XEN) 0d 000 00 1 0 0 0 0 1 1 98
(XEN) 0e 000 00 0 0 0 0 0 1 1 A0
(XEN) 0f 000 00 0 0 0 0 0 1 1 A8
(XEN) 10 000 00 0 1 0 1 0 1 1 AE
(XEN) 11 000 00 1 1 0 1 0 1 1 C0
(XEN) 12 000 00 1 1 0 1 0 1 1 C8
(XEN) 13 000 00 0 1 0 1 0 1 1 F1
(XEN) 14 000 00 1 1 0 1 0 1 1 61
(XEN) 15 0CA 0A 1 0 0 0 0 1 2 71
(XEN) 16 000 00 1 1 0 1 0 1 1 32
(XEN) 17 000 00 0 1 0 1 0 1 1 AC
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ133 -> 0:1
(XEN) IRQ64 -> 0:3
(XEN) IRQ72 -> 0:4
(XEN) IRQ80 -> 0:5
(XEN) IRQ88 -> 0:6
(XEN) IRQ96 -> 0:7
(XEN) IRQ41 -> 0:8
(XEN) IRQ167 -> 0:9
(XEN) IRQ120 -> 0:10
(XEN) IRQ136 -> 0:11
(XEN) IRQ212 -> 0:12
(XEN) IRQ152 -> 0:13
(XEN) IRQ160 -> 0:14
(XEN) IRQ168 -> 0:15
(XEN) IRQ174 -> 0:16
(XEN) IRQ192 -> 0:17
(XEN) IRQ200 -> 0:18
(XEN) IRQ241 -> 0:19
(XEN) IRQ97 -> 0:20
(XEN) IRQ50 -> 0:22
(XEN) IRQ172 -> 0:23
(XEN) .................................... done.
[-- Attachment #1.2: Type: text/html, Size: 11923 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
next prev parent reply other threads:[~2012-05-07 18:29 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-02-13 20:16 [xen-unstable test] 11946: regressions - FAIL xen.org
2012-02-14 10:44 ` Ian Campbell
2012-02-14 19:17 ` Daniel De Graaf
2012-03-27 10:36 ` Ian Campbell
2012-03-27 10:52 ` Jan Beulich
2012-05-04 19:48 ` AP
2012-05-04 20:11 ` Andrew Cooper
2012-05-05 0:21 ` AP
2012-05-05 11:04 ` Andrew Cooper
2012-05-05 18:41 ` AP
2012-05-05 19:06 ` AP
2012-05-07 8:10 ` Jan Beulich
2012-05-07 11:50 ` Andrew Cooper
2012-05-07 13:34 ` Jan Beulich
2012-05-07 14:41 ` Andrew Cooper
2012-05-07 14:50 ` Jan Beulich
2012-05-07 15:40 ` Andrew Cooper
2012-05-07 15:43 ` Jan Beulich
2012-05-07 14:54 ` Jan Beulich
2012-05-07 15:51 ` Andrew Cooper
2012-05-07 18:29 ` AP [this message]
2012-05-08 6:37 ` Jan Beulich
2012-05-05 10:33 ` Ian Campbell
2012-05-05 11:11 ` Andrew Cooper
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGU+auuOa04LSn_ebNRXyJQ=bn4PpLiL2ejYaqaivFKLEdnabQ@mail.gmail.com' \
--to=apxeng@gmail.com \
--cc=Ian.Campbell@citrix.com \
--cc=Ian.Jackson@eu.citrix.com \
--cc=JBeulich@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=dgdegra@tycho.nsa.gov \
--cc=keir@xen.org \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).