From: Alexander Graf <agraf@suse.de>
To: "Gabriel L. Somlo" <gsomlo@gmail.com>
Cc: "pbonzini@redhat.com" <pbonzini@redhat.com>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
"stefanha@redhat.com" <stefanha@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>
Subject: Re: [Qemu-devel] e1000 autoneg timing, piix/osx
Date: Wed, 02 Jul 2014 23:02:30 +0200 [thread overview]
Message-ID: <53B47366.6000101@suse.de> (raw)
In-Reply-To: <20140702204959.GY1688@ERROL.INI.CMU.EDU>
On 02.07.14 22:49, Gabriel L. Somlo wrote:
> On Wed, Jul 02, 2014 at 11:16:52AM +0200, Alexander Graf wrote:
>>>> Are you sure there's not just simply some irq unmasking event
>>>> after 5500ms we don't handle properly?
>>> I poked around a bit, and the e1000 interrupt mask register is NOT the
>>> problem (the LSC mask bit is clear at all times). If anything, maybe
>>> the PIIX southbridge (or something further up "north") is masking PCI
>>> interrupts (at least from e1000) until roughly 5500 ms into the boot
>>> process ? Any ideas on how I could go about verifying this (without
>>> access to the guest source, obviously :) ) would be very helpful...
>> Yeah, maybe the interrupt is masked and doesn't get delivered properly? See
>> if you can trace when the e1000 emulation starts kicking an interrupt and
>> when the guest tries to fetch it (there should be an ack register for IRQs
>> somewhere).
>>
>> If we kick it but the guest doesn't react, the problem is further down -
>> check whether the IRQ ever got injected into the guest with trace points.
>>
>> If we don't kick it, we mask it somewhere in the e1000 emulation and need to
>> make sure we do kick once we unmask :). I don't know whether the LSC mask is
>> the only one involved.
> So it turns out everything I thought I knew (which was little indeed)
> was more or less wrong. The problem, as far as I'm observing it now,
> is that on PIIX, the OS X guest obsessively reads the ICR in a tight
> loop. It reads the injected LSC (and probably discards it) before
> unmasking the corresponding interrupt bit; later on, when it unmasks
> LSC, giving the emulated e1000 hardware a chance to raise the irq
> line, the actual LSC event has been flushed from the ICR, and the
> driver does not detect the link coming up.
>
> Here's how things work normally on Q35 (with INTERRUPT and PHY debugging
> enabled, // my comments on the side):
>
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: Start link auto negotiation
> e1000: Auto negotiation is completed
> e1000: set_ics 4, ICR 0, IMR 0 // autoneg timer injects LSC
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ics 2, ICR 4, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ics 2, ICR 6, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ims df // guest unmasks interrupts
> e1000: set_ics 0, ICR 6, IMR df
> e1000: set_interrupt_cause: mit_irq_level=1 // first raising irq edge
> e1000: ICR read: 6 // guest receives LSC (+more)
>
> ... and things work nicely from here on out :)
>
>
> On PIIX, however, things look like this:
>
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
>
> ... <8155 "ICR read" repetitions deleted> ...
>
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc ffffffff
> e1000: set_ics 0, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: Start link auto negotiation
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
>
> ... <145 "ICR read" repetitions deleted> ...
>
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: Auto negotiation is completed
> e1000: set_ics 4, ICR 0, IMR 0 // autoneg timer injects LSC
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 4 // guest reads (and discards?) LSC
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
>
> ... <23 "ICR read" repetitions deleted> ...
>
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ics 2, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 2
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ics 2, ICR 0, IMR 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ims df // guest unmasks interrupts
> e1000: set_ics 0, ICR 2, IMR df
> e1000: set_interrupt_cause: mit_irq_level=1 // first raising irq edge
> e1000: ICR read: 2 // too late, LSC is gone :(
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc 2
> e1000: set_ics 0, ICR 0, IMR dd
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ims df
> e1000: set_ics 0, ICR 0, IMR df
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ics 80, ICR 0, IMR df
> e1000: set_interrupt_cause: mit_irq_level=1
> e1000: set_ics 3, ICR 80, IMR df
> e1000: set_interrupt_cause: mit_irq_level=1
> e1000: ICR read: 83
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_imc 82
> e1000: set_ics 0, ICR 0, IMR 5d
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: set_ims df
> e1000: set_ics 0, ICR 0, IMR df
> e1000: set_interrupt_cause: mit_irq_level=0
> e1000: ICR read: 0
> e1000: set_interrupt_cause: mit_irq_level=0
>
> ... and so on...
>
>
> Any clue as to why ICR gets read like that on PIIX, but not Q35 ?
Maybe it's using MSI on q35? :)
Maybe we also share the same IRQ line with another device on PIIX that
gets polled all the time? IDE maybe?
Either way, why does the bit get cleared even though it hasn't been
raised? What does real hardware do with interrupts that have been masked?
Alex
next prev parent reply other threads:[~2014-07-02 21:02 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-06-30 16:55 [Qemu-devel] [RFC PATCH v1 0/2] e1000: More link negotiation vs. OS X Gabriel L. Somlo
2014-06-30 16:55 ` [Qemu-devel] [RFC PATCH v1 1/2] e1000: clean up set_phy_ctrl function Gabriel L. Somlo
2014-06-30 18:04 ` Michael S. Tsirkin
2014-06-30 18:12 ` Gabriel L. Somlo
2014-06-30 19:29 ` Gabriel L. Somlo
2014-06-30 19:35 ` Gabriel L. Somlo
2014-06-30 16:55 ` [Qemu-devel] [RFC PATCH v1 2/2] e1000: adjust initial autoneg timing (for piix/osx) Gabriel L. Somlo
2014-06-30 17:55 ` Michael S. Tsirkin
2014-06-30 18:00 ` Paolo Bonzini
2014-06-30 18:21 ` Alexander Graf
2014-07-02 9:02 ` Gabriel L. Somlo
2014-07-02 9:16 ` Alexander Graf
2014-07-02 20:49 ` [Qemu-devel] e1000 autoneg timing, piix/osx Gabriel L. Somlo
2014-07-02 21:02 ` Alexander Graf [this message]
2014-07-02 21:14 ` Gabriel L. Somlo
2014-07-02 21:54 ` Alexander Graf
2014-07-02 22:02 ` Gabriel L. Somlo
2014-07-03 8:04 ` Alexander Graf
2014-07-03 13:17 ` Gabriel L. Somlo
2014-07-03 13:20 ` Alexander Graf
2014-07-03 13:58 ` Gabriel L. Somlo
2014-07-03 14:02 ` Alexander Graf
2014-07-03 14:14 ` Gabriel L. Somlo
2014-07-03 14:51 ` Alexander Graf
2014-07-03 15:25 ` Alexander Graf
2014-07-03 16:09 ` Paolo Bonzini
2014-07-03 16:43 ` Gabriel L. Somlo
2014-07-03 17:33 ` Alexander Graf
2014-07-02 9:33 ` [Qemu-devel] [RFC PATCH v1 2/2] e1000: adjust initial autoneg timing (for piix/osx) Michael S. Tsirkin
2014-07-02 12:05 ` Gabriel L. Somlo
2014-07-02 12:09 ` Michael S. Tsirkin
2014-07-02 14:21 ` Gabriel L. Somlo
2014-07-02 15:17 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=53B47366.6000101@suse.de \
--to=agraf@suse.de \
--cc=gsomlo@gmail.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).