From: "Thimo E." <abc@digithi.de>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "Zhang, Yang Z" <yang.z.zhang@intel.com>,
xen-devel <xen-devel@lists.xenproject.org>,
Keir Fraser <keir@xen.org>, Jan Beulich <JBeulich@suse.com>
Subject: Re: cpuidle and un-eoid interrupts at the local apic
Date: Sun, 08 Sep 2013 12:24:16 +0200 [thread overview]
Message-ID: <522C5050.9030508@digithi.de> (raw)
In-Reply-To: <522C491B.6000004@citrix.com>
[-- Attachment #1: Type: text/plain, Size: 3514 bytes --]
Ah, sorry. Output is attached.
Am 08.09.2013 11:53, schrieb Andrew Cooper:
> On 08/09/2013 00:37, Thimo E. wrote:
>> Hello Andrew,
>>
>> ok, thanks. This is what I assumed.
>>
>> The output of "xl debug-keys iMQ" is empty.
> Sorry - I should have been more clear. `xl debug-keys` dumps its
> information into the xen dmesg buffer, so `xl dmesg` will capture the
> results.
>
> ~Andrew
>
>> [root@localhost ~]# dmesg |grep arcmsr
>> [ 8.159321] arcmsr 0000:01:00.0: PCI INT A -> GSI 16 (level, low)
>> -> IRQ 16
>> [ 8.159413] arcmsr 0000:01:00.0: setting latency timer to 64
>> [ 8.170316] arcmsr 0000:01:00.0: get owner: 7ff0
>> [ 8.170414] arcmsr 0000:01:00.0: irq 1276 (276) for MSI/MSI-X
>> [ 8.170421] IRQ 1276/arcmsr: IRQF_DISABLED is not guaranteed on
>> shared IRQs
>> [ 8.170654] arcmsr0: msi enabled
>>
>> [root@localhost /]# cat /proc/irq/1276/spurious
>> count 61007
>> unhandled 8
>> last_unhandled 36736990 ms
>>
>> arcmsr is the driver of the Areca Storage Raid Controller. Used it
>> already before with Xenserver 6.0.2 for years, no problems.
>>
>> THe messages "IRQF_DISABLED is not guaranteed...." and "8 unhandled
>> interrupts" look interesting. I am not a kernel hacker but what I
>> interpret from
>> http://lxr.free-electrons.com/source/kernel/irq/manage.c?v=2.6.32:
>>
>> 1025 if ((irqflags & (IRQF_SHARED|IRQF_DISABLED)) ==
>> 1026 (IRQF_SHARED|IRQF_DISABLED)) {
>> 1027 pr_warning(
>> 1028 "IRQ %d/%s: IRQF_DISABLED is not guaranteed on
>> shared IRQs\n",
>> 1029 irq, devname);
>> ...
>> 738 * Force MSI interrupts to run with interrupts
>> 739 * disabled. The multi vector cards can cause stack
>> 740 * overflows due to nested interrupts when enough of
>> 741 * them are directed to a core and fire at the same
>> 742 * time.
>> 743 */
>> 744 if (desc->msi_desc)
>> 745 new->flags |= IRQF_DISABLED;
>>
>> --> "IRQF_DISABLED is not guaranteed on shared IRQs" warning is only
>> printed when irqflags IRQF_SHARED and IRQF_DISABLED are set
>> --> Is that what we see in the kernel oops the stack overflow the
>> comment in lines 738-742 is talking about ?!
>> --> IRQF_SHARED is set, so MSI interrupt 1276 is shared ?! I thought
>> that it is not possible that MSI interrupts are shared. Attached
>> you'll see my /proc/interrupts
>>
>> So what I do now is disabling MSI for the arcmsr driver. Could this be
>> the source of the problem ?! But why is 1276 shared ?!
>>
>> Best regards
>> Thimo
>>
>> Am 07.09.2013 19:02, schrieb Andrew Cooper:
>>> irq 29 is just an internal Xen number for accounting all interrupts. It
>>> doesn't mean anything specific regarding hardware etc. The vector and
>>> affinity would expect to change as dom0s vcpus are moved around by the
>>> scheduler.
>>>
>>> domain-list=0 means that this interrupt is targeted at dom0 (It is a
>>> list because certain interrupts have to be shared my more than 1
>>> domain). Helpfully, the keyhandler truncates the pirq field, so 276 is
>>> unlikely to be correct. As it is a dom0 MSI, I am guessing it actually
>>> matches up with interrupt 1276 in /proc/interrupts, if there is one.
>>>
>>> Can you provide the results of `xl debug-keys iMQ`, and attach
>>> /proc/interrupts to this email (just in case the setup has changed after
>>> playing with your BIOS)
>>>
>>> ~Andrew
>>>
[-- Attachment #2: xl_debus_keys20130908.txt --]
[-- Type: text/plain, Size: 6746 bytes --]
(XEN) Guest interrupt information:
(XEN) IRQ: 0 affinity:1 vec:f0 type=IO-APIC-edge status=00000000 mapped, unbound
(XEN) IRQ: 1 affinity:1 vec:38 type=IO-APIC-edge status=00000014 in-flight=0 domain-list=0: 1(----),
(XEN) IRQ: 2 affinity:f vec:00 type=XT-PIC status=00000000 mapped, unbound
(XEN) IRQ: 3 affinity:1 vec:40 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 4 affinity:1 vec:48 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 5 affinity:1 vec:50 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 5(----),
(XEN) IRQ: 6 affinity:1 vec:58 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 7 affinity:1 vec:60 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 8 affinity:1 vec:68 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 8(----),
(XEN) IRQ: 9 affinity:1 vec:70 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 9(----),
(XEN) IRQ: 10 affinity:1 vec:78 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 11 affinity:1 vec:88 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 12 affinity:1 vec:90 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 13 affinity:1 vec:98 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 14 affinity:1 vec:a0 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 15 affinity:1 vec:a8 type=IO-APIC-edge status=00000002 mapped, unbound
(XEN) IRQ: 16 affinity:1 vec:b0 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 16(----),
(XEN) IRQ: 17 affinity:1 vec:39 type=IO-APIC-level status=00000002 mapped, unbound
(XEN) IRQ: 18 affinity:1 vec:31 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 18(----),
(XEN) IRQ: 22 affinity:1 vec:29 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 22(----),
(XEN) IRQ: 23 affinity:1 vec:d0 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 23(----),
(XEN) IRQ: 24 affinity:1 vec:28 type=DMA_MSI status=00000000 mapped, unbound
(XEN) IRQ: 25 affinity:1 vec:30 type=DMA_MSI status=00000000 mapped, unbound
(XEN) IRQ: 26 affinity:f vec:b8 type=PCI-MSI status=00000002 mapped, unbound
(XEN) IRQ: 27 affinity:f vec:c0 type=PCI-MSI status=00000002 mapped, unbound
(XEN) IRQ: 28 affinity:f vec:c8 type=PCI-MSI status=00000002 mapped, unbound
(XEN) IRQ: 29 affinity:f vec:d8 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:276(----),
(XEN) IRQ: 30 affinity:f vec:51 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:275(----),
(XEN) IRQ: 31 affinity:f vec:61 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:274(----),
(XEN) IO-APIC interrupt information:
(XEN) IRQ 0 Vec240:
(XEN) Apic 0x00, Pin 2: vec=f0 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 1 Vec 56:
(XEN) Apic 0x00, Pin 1: vec=38 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 3 Vec 64:
(XEN) Apic 0x00, Pin 3: vec=40 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 4 Vec 72:
(XEN) Apic 0x00, Pin 4: vec=48 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 5 Vec 80:
(XEN) Apic 0x00, Pin 5: vec=50 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 6 Vec 88:
(XEN) Apic 0x00, Pin 6: vec=58 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 7 Vec 96:
(XEN) Apic 0x00, Pin 7: vec=60 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 8 Vec104:
(XEN) Apic 0x00, Pin 8: vec=68 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 9 Vec112:
(XEN) Apic 0x00, Pin 9: vec=70 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=L mask=0 dest_id:0
(XEN) IRQ 10 Vec120:
(XEN) Apic 0x00, Pin 10: vec=78 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 11 Vec136:
(XEN) Apic 0x00, Pin 11: vec=88 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 12 Vec144:
(XEN) Apic 0x00, Pin 12: vec=90 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 13 Vec152:
(XEN) Apic 0x00, Pin 13: vec=98 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 14 Vec160:
(XEN) Apic 0x00, Pin 14: vec=a0 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 15 Vec168:
(XEN) Apic 0x00, Pin 15: vec=a8 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN) IRQ 16 Vec176:
(XEN) Apic 0x00, Pin 16: vec=b0 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN) IRQ 17 Vec 57:
(XEN) Apic 0x00, Pin 17: vec=39 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN) IRQ 18 Vec 49:
(XEN) Apic 0x00, Pin 18: vec=31 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN) IRQ 22 Vec 41:
(XEN) Apic 0x00, Pin 22: vec=29 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN) Apic 0x00, Pin 23: vec=d0 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN) PCI-MSI interrupt information:
(XEN) MSI 26 vec=b8 lowest edge assert log lowest dest=00000001 mask=0/1/-1
(XEN) MSI 27 vec=c0 lowest edge assert log lowest dest=00000001 mask=0/1/-1
(XEN) MSI 28 vec=c8 lowest edge assert log lowest dest=00000001 mask=0/1/-1
(XEN) MSI 29 vec=d8 lowest edge assert log lowest dest=00000001 mask=0/0/-1
(XEN) MSI 30 vec=51 lowest edge assert log lowest dest=00000001 mask=0/0/-1
(XEN) MSI 31 vec=61 lowest edge assert log lowest dedmes st=00000001 mask=0/0/-1
(XEN) ==== PCI devices ====
(XEN) 04:00.1 - dom 0 - MSIs < 31 >
(XEN) 04:00.0 - dom 0 - MSIs < 30 >
(XEN) 03:02.0 - dom 0 - MSIs < >
(XEN) 02:00.0 - dom 0 - MSIs < >
(XEN) 01:00.0 - dom 0 - MSIs < 29 >
(XEN) 00:1f.3 - dom 0 - MSIs < >
(XEN) 00:1f.0 - dom 0 - MSIs < >
(XEN) 00:1d.0 - dom 0 - MSIs < >
(XEN) 00:1c.4 - dom 0 - MSIs < 28 >
(XEN) 00:1c.0 - dom 0 - MSIs < 27 >
(XEN) 00:1b.0 - dom 0 - MSIs < >
(XEN) 00:1a.0 - dom 0 - MSIs < >
(XEN) 00:16.0 - dom 0 - MSIs < >
(XEN) 00:14.0 - dom 0 - MSIs < >
(XEN) 00:02.0 - dom 0 - MSIs < >
(XEN) 00:01.0 - dom 0 - MSIs < 26 >
(XEN) 00:00.0 - dom 0 - MSIs < >
[-- Attachment #3: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
next prev parent reply other threads:[~2013-09-08 10:24 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-31 20:32 cpuidle and un-eoid interrupts at the local apic Andrew Cooper
2013-06-03 14:30 ` Jan Beulich
2013-07-31 8:30 ` Thimo E.
2013-07-31 9:47 ` Andrew Cooper
2013-08-02 22:50 ` Thimo E.
2013-08-02 23:32 ` Andrew Cooper
2013-08-05 12:45 ` Jan Beulich
2013-08-05 14:51 ` Andrew Cooper
2013-08-09 21:27 ` Thimo E.
2013-08-09 21:40 ` Andrew Cooper
2013-08-09 21:44 ` Andrew Cooper
2013-08-11 17:46 ` Thimo E.
2013-08-12 6:02 ` Zhang, Yang Z
2013-08-12 8:49 ` Zhang, Yang Z
2013-08-12 8:57 ` Jan Beulich
2013-08-12 11:52 ` Thimo E
2013-08-12 12:04 ` Andrew Cooper
2013-08-19 15:14 ` Thimo E.
2013-08-20 5:43 ` Thimo Eichstädt
2013-08-20 8:40 ` Jan Beulich
2013-08-20 8:50 ` Zhang, Yang Z
2013-08-23 7:22 ` Thimo Eichstädt
2013-08-23 7:30 ` Zhang, Yang Z
2013-08-27 1:03 ` Zhang, Yang Z
2013-09-04 18:32 ` Thimo E.
2013-09-04 18:55 ` Andrew Cooper
2013-09-04 19:56 ` Thimo E.
2013-09-04 20:54 ` Andrew Cooper
2013-09-05 1:45 ` Zhang, Yang Z
2013-09-05 7:20 ` Thimo E.
2013-09-05 1:15 ` Zhang, Yang Z
2013-09-17 2:09 ` Zhang, Yang Z
2013-09-17 7:39 ` Thimo E.
2013-09-17 7:43 ` Zhang, Yang Z
2013-09-17 21:04 ` Thimo E.
2013-09-18 1:18 ` Zhang, Xiantao
2013-09-18 17:24 ` Thimo E.
2013-09-18 12:06 ` Andrew Cooper
2013-08-12 13:54 ` Thimo E
2013-08-12 14:06 ` Andrew Cooper
2013-08-13 1:43 ` Zhang, Yang Z
2013-08-13 6:39 ` Thimo E.
2013-08-13 11:39 ` Wu, Feng
2013-08-13 12:46 ` Andrew Cooper
2013-08-12 9:10 ` Andrew Cooper
2013-08-12 5:50 ` Zhang, Yang Z
2013-08-12 8:20 ` Jan Beulich
2013-08-12 9:28 ` Andrew Cooper
2013-08-12 10:05 ` Jan Beulich
2013-08-12 10:27 ` Andrew Cooper
2013-08-14 2:53 ` Zhang, Yang Z
2013-08-14 7:51 ` Thimo E.
2013-08-14 9:52 ` Andrew Cooper
2013-09-07 13:27 ` Thimo E.
2013-09-07 17:02 ` Andrew Cooper
2013-09-07 23:37 ` Thimo E.
2013-09-08 9:53 ` Andrew Cooper
2013-09-08 10:24 ` Thimo E. [this message]
2013-09-09 13:16 ` Andrew Cooper
2013-09-09 14:48 ` Thimo Eichstädt
2013-09-09 15:12 ` Andrew Cooper
2013-09-09 7:59 ` Jan Beulich
2013-09-09 12:53 ` Andrew Cooper
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=522C5050.9030508@digithi.de \
--to=abc@digithi.de \
--cc=JBeulich@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=keir@xen.org \
--cc=xen-devel@lists.xenproject.org \
--cc=yang.z.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.