From mboxrd@z Thu Jan  1 00:00:00 1970
From: David Miller <davem@davemloft.net>
Subject: Re: Experimental fix for MSI-X issue on r8169
Date: Tue, 21 Aug 2018 16:32:45 -0700 (PDT)
Message-ID: <20180821.163245.724091648801257903.davem@davemloft.net>
References: <9f605d51-c1b3-2f00-9d44-0f53dcb38848@gmail.com>
        <CAPpJ_ecsMOL23VYM2juZ9R8JLrSh1bjCet16XCSpv0mDaSYu6w@mail.gmail.com>
        <98df6b0a-44df-9c0d-ddf6-11b892ccd990@gmail.com>
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Cc: jian-hong@endlessm.com, steved424@gmail.com, gogen@disroot.org,
        netdev@vger.kernel.org
To: hkallweit1@gmail.com
Return-path: <netdev-owner@vger.kernel.org>
Received: from shards.monkeyblade.net ([23.128.96.9]:46888 "EHLO
        shards.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1726916AbeHVCzD (ORCPT
        <rfc822;netdev@vger.kernel.org>); Tue, 21 Aug 2018 22:55:03 -0400
In-Reply-To: <98df6b0a-44df-9c0d-ddf6-11b892ccd990@gmail.com>
Sender: netdev-owner@vger.kernel.org
List-ID: <netdev.vger.kernel.org>

From: Heiner Kallweit <hkallweit1@gmail.com>
Date: Tue, 21 Aug 2018 23:19:04 +0200

> That's what I get on my system (RTL8168E-VL). In your case you'll come
> only till the first suspend.
> 
> [    3.743404] r8169 0000:03:00.0: MSI-X entry: context probe: fee01004 0 40ef 1

On probe, MSI-X is masked (ie. disabled) and is configured to use:

	address: 0x00000000fee01004
	data:    0x00000000000040ef

> [   29.539250] r8169 0000:03:00.0: MSI-X entry: context suspend: fee02004 0 4028 0

At suspend time, MSI-X is unmasked (ie. enabled) and is configured to use:

	address: 0x00000000fee01004
	data:    0x0000000000004028

> [   29.837457] r8169 0000:03:00.0: MSI-X entry: context resume: fee01004 0 402b 0

At reume time, MSI-X is unmasked (ie. enabled) and is configured to use:

	address: 0x00000000fee01004
	data:    0x000000000000402b

> [   36.921370] r8169 0000:03:00.0: MSI-X entry: context suspend: fee01004 0 402b 0

Second suspend:

	address: 0x00000000fee01004
	data:    0x000000000000402b

> [   37.239407] r8169 0000:03:00.0: MSI-X entry: context resume: fee01004 0 402b 0

Second resume:

	address: 0x00000000fee01004
	data:    0x000000000000402b

And this all looks normal.  The data field is changing when you first up the device
and interrupts are enabled.  This is where the request_irq happens, the MSI
vector is allocated, and that vector number is written to the data field of the
MSI-X entry.

It looks like this (re-)allocation of MSI vectors happens on every resume as well.

And that's why the data field changes each resume.