From: Håvard Bjerke <Havard.Bjerke@idi.ntnu.no>
To: Ian Pratt <Ian.Pratt@cl.cam.ac.uk>
Cc: "Rune Johan Andresen" <Rune.Johan.Andresen@idi.ntnu.no>,
"Xen Virtual Machine Monitor" <xen-devel@lists.sourceforge.net>,
=?unknown-8bit?Q?H=E5vard?= Bjerke <Havard.Bjerke@idi.ntnu.no>
Subject: Re: Xen & I/O in clusters - problems! New Information
Date: Wed, 27 Oct 2004 19:11:49 +0200 [thread overview]
Message-ID: <20041027171148.GK23334@idi.ntnu.no> (raw)
In-Reply-To: <E1CL3Vo-00067O-00@mta1.cl.cam.ac.uk>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=unknown-8bit, Size: 2917 bytes --]
On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote:
> >
> > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre
> > results between
> > two xen dom0 nodes. I am not sure if these interrupts tells anything:
>
> It's the different rate at which the eth0 interrupt counts go up
> during your bandwidth tests that is interesting. e.g. poll it
> once a second during the test.
>
We tried sending 1 MB and measured:
non-SMP native Linux:
~ 130k interrupts
114 kB/s
native Linux with compiled-in SMP support, single CPU:
~ 140k interrupts
114 kB/s
Xen0:
~ 180k interrupts
80 kB/s
> It looks like your native Linux is not using legacy PIC mode
> rather than using the ioapic. Have you tried an SMP native
> kernel to see if it gets the same interrupt layout as Xen?
>
Layout in native linux w/o SMP:
CPU0
0: 130710161 XT-PIC timer
1: 2 XT-PIC keyboard
2: 0 XT-PIC cascade
3: 12872557 XT-PIC eth0
7: 0 XT-PIC ehci-hcd
8: 1 XT-PIC rtc
10: 3 XT-PIC usb-uhci
11: 445263 XT-PIC aic7xxx, aic7xxx, usb-uhci
14: 1 XT-PIC ide0
15: 0 XT-PIC libata
NMI: 0
ERR: 0
Layout in native linux compiled with SMP:
CPU0
0: 121362 IO-APIC-edge timer
1: 2 IO-APIC-edge keyboard
2: 0 XT-PIC cascade
14: 5 IO-APIC-edge ide0
16: 0 IO-APIC-level usb-uhci
18: 177583 IO-APIC-level eth0
19: 0 IO-APIC-level usb-uhci
24: 7587 IO-APIC-level aic7xxx
25: 30 IO-APIC-level aic7xxx
NMI: 0
LOC: 121307
ERR: 0
MIS: 0
Layout in Xen0:
CPU0
1: 2 Phys-irq keyboard
14: 3 Phys-irq ide0
18: 267274 Phys-irq eth0
24: 15910 Phys-irq aic7xxx
25: 30 Phys-irq aic7xxx
128: 1 Dynamic-irq misdirect
129: 0 Dynamic-irq ctrl-if
130: 771917 Dynamic-irq timer
131: 0 Dynamic-irq timer_dbg, net-be-dbg
132: 0 Dynamic-irq console
NMI: 0
ERR: 0
Questions:
Do you think interrupt sharing is the problem?
Or the use of IO-APIC?
Is this an inherent problem with Xen?
Is it possible to change the interrupt scheme in Xen in order to achieve the same performance as in native linux?
Cheers,
Håvard
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_idU88&alloc_id\x12065&op=click
next prev parent reply other threads:[~2004-10-27 17:11 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2004-10-13 5:12 plan 9 status Ronald G. Minnich
2004-10-13 12:14 ` Mark A. Williamson
2004-10-13 22:18 ` Ronald G. Minnich
2004-10-14 0:52 ` Kip Macy
2004-10-14 8:10 ` Keir Fraser
2004-10-14 10:35 ` Wesley Parish
2004-10-14 11:04 ` Keir Fraser
2004-10-14 15:32 ` Ronald G. Minnich
2004-10-14 15:57 ` Keir Fraser
2004-10-21 16:06 ` Keir Fraser
2004-10-14 15:53 ` Mark A. Williamson
2004-10-14 17:45 ` Xen & I/O in clusters - problems! Rune Johan Andresen
2004-10-14 18:17 ` Ian Pratt
2004-10-14 23:24 ` Mark A. Williamson
2004-10-15 13:48 ` Rune Johan Andresen
2004-10-21 19:12 ` Xen & I/O in clusters - problems! New Information Rune Johan Andresen
2004-10-21 20:24 ` Ian Pratt
2004-10-22 16:36 ` Rune Johan Andresen
2004-10-22 17:47 ` Ian Pratt
2004-10-27 17:11 ` Håvard Bjerke [this message]
2004-10-27 18:26 ` Ian Pratt
2004-10-27 18:40 ` Mark A. Williamson
2004-10-28 11:51 ` Håvard Bjerke
2004-10-28 13:09 ` Mark A. Williamson
2004-10-28 11:32 ` Håvard Bjerke
2004-10-28 12:44 ` Ian Pratt
2004-10-28 16:45 ` Håvard Bjerke
2004-10-29 17:05 ` Xen & I/O in clusters - Single Vs. Dual CPU issue Rune Johan Andresen
2004-10-29 17:24 ` Ian Pratt
2004-10-29 18:39 ` Håvard Bjerke
2004-10-29 19:46 ` Ian Pratt
2004-11-02 16:13 ` Håvard Bjerke
2004-11-02 16:35 ` Mark A. Williamson
2004-11-02 16:51 ` Mark A. Williamson
2004-11-04 17:42 ` Rune Johan Andresen
2004-11-04 18:41 ` Bin Ren
2004-11-05 14:53 ` Rune Johan Andresen
2004-11-05 16:47 ` Keir Fraser
2004-11-05 17:33 ` Rune Johan Andresen
2004-11-06 8:04 ` Keir Fraser
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20041027171148.GK23334@idi.ntnu.no \
--to=havard.bjerke@idi.ntnu.no \
--cc=Ian.Pratt@cl.cam.ac.uk \
--cc=Rune.Johan.Andresen@idi.ntnu.no \
--cc=xen-devel@lists.sourceforge.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.