pci_bus warnings about adjacent RW1C bits w/ iproc-pci

linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* pci_bus warnings about adjacent RW1C bits w/ iproc-pci
@ 2017-10-10  3:58 Florian Fainelli
  2017-10-10  5:33 ` Ray Jui
  0 siblings, 1 reply; 10+ messages in thread
From: Florian Fainelli @ 2017-10-10  3:58 UTC (permalink / raw)
  To: bcm-kernel-feedback-list, Ray Jui, Jon Mason, Hauke Mehrtens,
	Rafał Miłecki, Bjorn Helgaas, linux-pci

Hi,

With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
warnings using pcie_iproc; is this something to be worried about?

[    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00



[    2.632358] pci_bus 0000:00: root bus resource [mem
0x08000000-0x0fffffff]


[    2.639207] pci_bus 0000:00: No busn resource found for root bus,
will use [bus 00-ff]


[    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4 may corrupt adjacent RW1C bits


[    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4 may corrupt adjacent RW1C bits


[    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4c may corrupt adjacent RW1C bits


[    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits


[    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4 may corrupt adjacent RW1C bits


[    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
offset 0xc may corrupt adjacent RW1C bits


[    2.705531] PCI: bus0: Fast back to back transfers disabled



[    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
00-00]), reconfiguring


[    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits


[    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0xc8 may corrupt adjacent RW1C bits


[    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits


[    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits


[    2.758262] pci 0000:01:00.0: enabling Extended Tags



[    2.763577] PCI: bus1: Fast back to back transfers disabled



[    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
0x08000000-0x080fffff]


[    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
0x08000000-0x08003fff 64bit]


[    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]



[    2.788293] pci 0000:00:00.0:   bridge window [mem
0x08000000-0x080fffff]


[    2.911536] pcie_iproc_bcma bcma0:8: link: UP



[    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00



[    2.922349] pci_bus 0001:00: root bus resource [mem
0x40000000-0x47ffffff]


[    2.929200] pci_bus 0001:00: No busn resource found for root bus,
will use [bus 00-ff]


[    2.937482] PCI: bus0: Fast back to back transfers disabled



[    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
00-00]), reconfiguring


[    2.951713] PCI: bus1: Fast back to back transfers disabled



[    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
0x40000000-0x400fffff]


[    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
0x40000000-0x40007fff 64bit]


[    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]



[    2.976434] pci 0001:00:00.0:   bridge window [mem
0x40000000-0x400fffff]
-- 
Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10  3:58 pci_bus warnings about adjacent RW1C bits w/ iproc-pci Florian Fainelli
@ 2017-10-10  5:33 ` Ray Jui
  2017-10-10 11:10   ` Bjorn Helgaas
  0 siblings, 1 reply; 10+ messages in thread
From: Ray Jui @ 2017-10-10  5:33 UTC (permalink / raw)
  To: Florian Fainelli
  Cc: bcm-kernel-feedback-list, Ray Jui, Jon Mason, Hauke Mehrtens,
	Rafał Miłecki, Bjorn Helgaas, linux-pci

Hi Florian,

This is from the following commit:

commit fb26592301200dbbe4a9943fe188b57a46716900
Author: Bjorn Helgaas <bhelgaas@google.com>
Date:   Mon Oct 31 16:00:01 2016 -0500

    PCI: Warn on possible RW1C corruption for sub-32 bit config writes

    Hardware that supports only 32-bit config writes is not spec-compliant.
    For example, if software performs a 16-bit write, we must do a 32-bit read,
    merge in the 16 bits we intend to write, followed by a 32-bit write.  If
    the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
    to-clear) bits set, we just inadvertently cleared something we shouldn't
    have.

    Add a rate-limited warning when we do sub-32 bit config writes.  Remove
    similar probe-time warnings from some of the affected host bridge drivers.

    Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
    Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
    Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
    Acked-by: Thierry Reding <treding@nvidia.com>

This warning message will show up on every root complex that can only
support 32-bit config register write (instead of 8-bit, 16-bit writes)
and iProc based host controller happens to be one of them.

This limitation violates the PCIe spec as the 32-bit write may clear
RW1C bits of a config register. But I believe those registers are
usually used in AER which we do not currently support in the iProc
based hos controllers.

Regards,

Ray

On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
> Hi,
>
> With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
> warnings using pcie_iproc; is this something to be worried about?
>
> [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
>
>
>
> [    2.632358] pci_bus 0000:00: root bus resource [mem
> 0x08000000-0x0fffffff]
>
>
> [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
> will use [bus 00-ff]
>
>
> [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x4 may corrupt adjacent RW1C bits
>
>
> [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x4 may corrupt adjacent RW1C bits
>
>
> [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x4c may corrupt adjacent RW1C bits
>
>
> [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x3e may corrupt adjacent RW1C bits
>
>
> [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x4 may corrupt adjacent RW1C bits
>
>
> [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
> offset 0xc may corrupt adjacent RW1C bits
>
>
> [    2.705531] PCI: bus0: Fast back to back transfers disabled
>
>
>
> [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
> 00-00]), reconfiguring
>
>
> [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x3e may corrupt adjacent RW1C bits
>
>
> [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0xc8 may corrupt adjacent RW1C bits
>
>
> [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x3e may corrupt adjacent RW1C bits
>
>
> [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> offset 0x3e may corrupt adjacent RW1C bits
>
>
> [    2.758262] pci 0000:01:00.0: enabling Extended Tags
>
>
>
> [    2.763577] PCI: bus1: Fast back to back transfers disabled
>
>
>
> [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
> 0x08000000-0x080fffff]
>
>
> [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
> 0x08000000-0x08003fff 64bit]
>
>
> [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
>
>
>
> [    2.788293] pci 0000:00:00.0:   bridge window [mem
> 0x08000000-0x080fffff]
>
>
> [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
>
>
>
> [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
>
>
>
> [    2.922349] pci_bus 0001:00: root bus resource [mem
> 0x40000000-0x47ffffff]
>
>
> [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
> will use [bus 00-ff]
>
>
> [    2.937482] PCI: bus0: Fast back to back transfers disabled
>
>
>
> [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
> 00-00]), reconfiguring
>
>
> [    2.951713] PCI: bus1: Fast back to back transfers disabled
>
>
>
> [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
> 0x40000000-0x400fffff]
>
>
> [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
> 0x40000000-0x40007fff 64bit]
>
>
> [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
>
>
>
> [    2.976434] pci 0001:00:00.0:   bridge window [mem
> 0x40000000-0x400fffff]
> --
> Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10  5:33 ` Ray Jui
@ 2017-10-10 11:10   ` Bjorn Helgaas
  2017-10-10 14:56     ` Jim Quinlan
                       ` (2 more replies)
  0 siblings, 3 replies; 10+ messages in thread
From: Bjorn Helgaas @ 2017-10-10 11:10 UTC (permalink / raw)
  To: Ray Jui
  Cc: Florian Fainelli, bcm-kernel-feedback-list, Ray Jui, Jon Mason,
	Hauke Mehrtens, Rafał Miłecki, Bjorn Helgaas, linux-pci

On Mon, Oct 09, 2017 at 10:33:01PM -0700, Ray Jui wrote:
> Hi Florian,
> 
> This is from the following commit:
> 
> commit fb26592301200dbbe4a9943fe188b57a46716900
> Author: Bjorn Helgaas <bhelgaas@google.com>
> Date:   Mon Oct 31 16:00:01 2016 -0500
> 
>     PCI: Warn on possible RW1C corruption for sub-32 bit config writes
> 
>     Hardware that supports only 32-bit config writes is not spec-compliant.
>     For example, if software performs a 16-bit write, we must do a 32-bit read,
>     merge in the 16 bits we intend to write, followed by a 32-bit write.  If
>     the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
>     to-clear) bits set, we just inadvertently cleared something we shouldn't
>     have.
> 
>     Add a rate-limited warning when we do sub-32 bit config writes.  Remove
>     similar probe-time warnings from some of the affected host bridge drivers.
> 
>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>     Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
>     Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
>     Acked-by: Thierry Reding <treding@nvidia.com>
> 
> This warning message will show up on every root complex that can only
> support 32-bit config register write (instead of 8-bit, 16-bit writes)
> and iProc based host controller happens to be one of them.
> 
> This limitation violates the PCIe spec as the 32-bit write may clear
> RW1C bits of a config register. But I believe those registers are
> usually used in AER which we do not currently support in the iProc
> based hos controllers.

This warning is ratelimited, but maybe even that is overkill.  It
looks like we currently warn on every config write of less than 32
bits.  How many of those do you see?  Once per boot might be enough,
or maybe once per device (so we get a hint when loading a new driver).  

> On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
> > Hi,
> >
> > With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
> > warnings using pcie_iproc; is this something to be worried about?
> >
> > [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
> >
> >
> >
> > [    2.632358] pci_bus 0000:00: root bus resource [mem
> > 0x08000000-0x0fffffff]
> >
> >
> > [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
> > will use [bus 00-ff]
> >
> >
> > [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x4 may corrupt adjacent RW1C bits
> >
> >
> > [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x4 may corrupt adjacent RW1C bits
> >
> >
> > [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x4c may corrupt adjacent RW1C bits
> >
> >
> > [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x3e may corrupt adjacent RW1C bits
> >
> >
> > [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x4 may corrupt adjacent RW1C bits
> >
> >
> > [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
> > offset 0xc may corrupt adjacent RW1C bits
> >
> >
> > [    2.705531] PCI: bus0: Fast back to back transfers disabled
> >
> >
> >
> > [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
> > 00-00]), reconfiguring
> >
> >
> > [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x3e may corrupt adjacent RW1C bits
> >
> >
> > [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0xc8 may corrupt adjacent RW1C bits
> >
> >
> > [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x3e may corrupt adjacent RW1C bits
> >
> >
> > [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> > offset 0x3e may corrupt adjacent RW1C bits
> >
> >
> > [    2.758262] pci 0000:01:00.0: enabling Extended Tags
> >
> >
> >
> > [    2.763577] PCI: bus1: Fast back to back transfers disabled
> >
> >
> >
> > [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
> > 0x08000000-0x080fffff]
> >
> >
> > [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
> > 0x08000000-0x08003fff 64bit]
> >
> >
> > [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
> >
> >
> >
> > [    2.788293] pci 0000:00:00.0:   bridge window [mem
> > 0x08000000-0x080fffff]
> >
> >
> > [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
> >
> >
> >
> > [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
> >
> >
> >
> > [    2.922349] pci_bus 0001:00: root bus resource [mem
> > 0x40000000-0x47ffffff]
> >
> >
> > [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
> > will use [bus 00-ff]
> >
> >
> > [    2.937482] PCI: bus0: Fast back to back transfers disabled
> >
> >
> >
> > [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
> > 00-00]), reconfiguring
> >
> >
> > [    2.951713] PCI: bus1: Fast back to back transfers disabled
> >
> >
> >
> > [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
> > 0x40000000-0x400fffff]
> >
> >
> > [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
> > 0x40000000-0x40007fff 64bit]
> >
> >
> > [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
> >
> >
> >
> > [    2.976434] pci 0001:00:00.0:   bridge window [mem
> > 0x40000000-0x400fffff]
> > --
> > Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10 11:10   ` Bjorn Helgaas
@ 2017-10-10 14:56     ` Jim Quinlan
  2017-10-10 14:59     ` Jon Mason
  2017-10-10 20:01     ` Florian Fainelli
  2 siblings, 0 replies; 10+ messages in thread
From: Jim Quinlan @ 2017-10-10 14:56 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Ray Jui, Florian Fainelli, bcm-kernel-feedback-list, Ray Jui,
	Jon Mason, Hauke Mehrtens, Rafał Miłecki, Bjorn Helgaas,
	linux-pci

Hi, we get about 9 of those messages on the Broadcom Settop box boot.
I think there's more but it is rate limited.  We could do individual
8-bit/16-bit/32-bit writes for the RC itself (we don't currently) but
not for the downstream devices without changes to the HW.  --Jim

On Tue, Oct 10, 2017 at 7:10 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> On Mon, Oct 09, 2017 at 10:33:01PM -0700, Ray Jui wrote:
>> Hi Florian,
>>
>> This is from the following commit:
>>
>> commit fb26592301200dbbe4a9943fe188b57a46716900
>> Author: Bjorn Helgaas <bhelgaas@google.com>
>> Date:   Mon Oct 31 16:00:01 2016 -0500
>>
>>     PCI: Warn on possible RW1C corruption for sub-32 bit config writes
>>
>>     Hardware that supports only 32-bit config writes is not spec-compliant.
>>     For example, if software performs a 16-bit write, we must do a 32-bit read,
>>     merge in the 16 bits we intend to write, followed by a 32-bit write.  If
>>     the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
>>     to-clear) bits set, we just inadvertently cleared something we shouldn't
>>     have.
>>
>>     Add a rate-limited warning when we do sub-32 bit config writes.  Remove
>>     similar probe-time warnings from some of the affected host bridge drivers.
>>
>>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>     Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
>>     Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
>>     Acked-by: Thierry Reding <treding@nvidia.com>
>>
>> This warning message will show up on every root complex that can only
>> support 32-bit config register write (instead of 8-bit, 16-bit writes)
>> and iProc based host controller happens to be one of them.
>>
>> This limitation violates the PCIe spec as the 32-bit write may clear
>> RW1C bits of a config register. But I believe those registers are
>> usually used in AER which we do not currently support in the iProc
>> based hos controllers.
>
> This warning is ratelimited, but maybe even that is overkill.  It
> looks like we currently warn on every config write of less than 32
> bits.  How many of those do you see?  Once per boot might be enough,
> or maybe once per device (so we get a hint when loading a new driver).
>
>> On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
>> > Hi,
>> >
>> > With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
>> > warnings using pcie_iproc; is this something to be worried about?
>> >
>> > [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
>> >
>> >
>> >
>> > [    2.632358] pci_bus 0000:00: root bus resource [mem
>> > 0x08000000-0x0fffffff]
>> >
>> >
>> > [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
>> > will use [bus 00-ff]
>> >
>> >
>> > [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4c may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
>> > offset 0xc may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.705531] PCI: bus0: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
>> > 00-00]), reconfiguring
>> >
>> >
>> > [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0xc8 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.758262] pci 0000:01:00.0: enabling Extended Tags
>> >
>> >
>> >
>> > [    2.763577] PCI: bus1: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
>> > 0x08000000-0x080fffff]
>> >
>> >
>> > [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
>> > 0x08000000-0x08003fff 64bit]
>> >
>> >
>> > [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
>> >
>> >
>> >
>> > [    2.788293] pci 0000:00:00.0:   bridge window [mem
>> > 0x08000000-0x080fffff]
>> >
>> >
>> > [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
>> >
>> >
>> >
>> > [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
>> >
>> >
>> >
>> > [    2.922349] pci_bus 0001:00: root bus resource [mem
>> > 0x40000000-0x47ffffff]
>> >
>> >
>> > [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
>> > will use [bus 00-ff]
>> >
>> >
>> > [    2.937482] PCI: bus0: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
>> > 00-00]), reconfiguring
>> >
>> >
>> > [    2.951713] PCI: bus1: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
>> > 0x40000000-0x400fffff]
>> >
>> >
>> > [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
>> > 0x40000000-0x40007fff 64bit]
>> >
>> >
>> > [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
>> >
>> >
>> >
>> > [    2.976434] pci 0001:00:00.0:   bridge window [mem
>> > 0x40000000-0x400fffff]
>> > --
>> > Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10 11:10   ` Bjorn Helgaas
  2017-10-10 14:56     ` Jim Quinlan
@ 2017-10-10 14:59     ` Jon Mason
  2017-10-10 16:18       ` Jim Quinlan
  2017-10-10 21:00       ` Bjorn Helgaas
  2017-10-10 20:01     ` Florian Fainelli
  2 siblings, 2 replies; 10+ messages in thread
From: Jon Mason @ 2017-10-10 14:59 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Ray Jui, Florian Fainelli, bcm-kernel-feedback-list, Ray Jui,
	Hauke Mehrtens, Rafał Miłecki, Bjorn Helgaas, linux-pci

On Tue, Oct 10, 2017 at 7:10 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> On Mon, Oct 09, 2017 at 10:33:01PM -0700, Ray Jui wrote:
>> Hi Florian,
>>
>> This is from the following commit:
>>
>> commit fb26592301200dbbe4a9943fe188b57a46716900
>> Author: Bjorn Helgaas <bhelgaas@google.com>
>> Date:   Mon Oct 31 16:00:01 2016 -0500
>>
>>     PCI: Warn on possible RW1C corruption for sub-32 bit config writes
>>
>>     Hardware that supports only 32-bit config writes is not spec-compliant.
>>     For example, if software performs a 16-bit write, we must do a 32-bit read,
>>     merge in the 16 bits we intend to write, followed by a 32-bit write.  If
>>     the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
>>     to-clear) bits set, we just inadvertently cleared something we shouldn't
>>     have.
>>
>>     Add a rate-limited warning when we do sub-32 bit config writes.  Remove
>>     similar probe-time warnings from some of the affected host bridge drivers.
>>
>>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>     Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
>>     Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
>>     Acked-by: Thierry Reding <treding@nvidia.com>
>>
>> This warning message will show up on every root complex that can only
>> support 32-bit config register write (instead of 8-bit, 16-bit writes)
>> and iProc based host controller happens to be one of them.
>>
>> This limitation violates the PCIe spec as the 32-bit write may clear
>> RW1C bits of a config register. But I believe those registers are
>> usually used in AER which we do not currently support in the iProc
>> based hos controllers.
>
> This warning is ratelimited, but maybe even that is overkill.  It
> looks like we currently warn on every config write of less than 32
> bits.  How many of those do you see?  Once per boot might be enough,
> or maybe once per device (so we get a hint when loading a new driver).

It's fairly trivial to change the warning in question to
dev_warn_once().  If this is acceptable, I can crank it out.

Thanks,
Jon

>> On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
>> > Hi,
>> >
>> > With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
>> > warnings using pcie_iproc; is this something to be worried about?
>> >
>> > [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
>> >
>> >
>> >
>> > [    2.632358] pci_bus 0000:00: root bus resource [mem
>> > 0x08000000-0x0fffffff]
>> >
>> >
>> > [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
>> > will use [bus 00-ff]
>> >
>> >
>> > [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4c may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x4 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
>> > offset 0xc may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.705531] PCI: bus0: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
>> > 00-00]), reconfiguring
>> >
>> >
>> > [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0xc8 may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>> > offset 0x3e may corrupt adjacent RW1C bits
>> >
>> >
>> > [    2.758262] pci 0000:01:00.0: enabling Extended Tags
>> >
>> >
>> >
>> > [    2.763577] PCI: bus1: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
>> > 0x08000000-0x080fffff]
>> >
>> >
>> > [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
>> > 0x08000000-0x08003fff 64bit]
>> >
>> >
>> > [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
>> >
>> >
>> >
>> > [    2.788293] pci 0000:00:00.0:   bridge window [mem
>> > 0x08000000-0x080fffff]
>> >
>> >
>> > [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
>> >
>> >
>> >
>> > [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
>> >
>> >
>> >
>> > [    2.922349] pci_bus 0001:00: root bus resource [mem
>> > 0x40000000-0x47ffffff]
>> >
>> >
>> > [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
>> > will use [bus 00-ff]
>> >
>> >
>> > [    2.937482] PCI: bus0: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
>> > 00-00]), reconfiguring
>> >
>> >
>> > [    2.951713] PCI: bus1: Fast back to back transfers disabled
>> >
>> >
>> >
>> > [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
>> > 0x40000000-0x400fffff]
>> >
>> >
>> > [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
>> > 0x40000000-0x40007fff 64bit]
>> >
>> >
>> > [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
>> >
>> >
>> >
>> > [    2.976434] pci 0001:00:00.0:   bridge window [mem
>> > 0x40000000-0x400fffff]
>> > --
>> > Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10 14:59     ` Jon Mason
@ 2017-10-10 16:18       ` Jim Quinlan
  2017-10-10 16:43         ` David Laight
  2017-10-10 20:09         ` Bjorn Helgaas
  2017-10-10 21:00       ` Bjorn Helgaas
  1 sibling, 2 replies; 10+ messages in thread
From: Jim Quinlan @ 2017-10-10 16:18 UTC (permalink / raw)
  To: Jon Mason
  Cc: Bjorn Helgaas, Ray Jui, Florian Fainelli,
	bcm-kernel-feedback-list, Ray Jui, Hauke Mehrtens,
	Rafał Miłecki, Bjorn Helgaas, linux-pci

BTW, can someone point out to me what precise provision in the PCI
spec says that 8/16/32 bit config space writes must be provided?

On Tue, Oct 10, 2017 at 10:59 AM, Jon Mason <jon.mason@broadcom.com> wrote:
> On Tue, Oct 10, 2017 at 7:10 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
>> On Mon, Oct 09, 2017 at 10:33:01PM -0700, Ray Jui wrote:
>>> Hi Florian,
>>>
>>> This is from the following commit:
>>>
>>> commit fb26592301200dbbe4a9943fe188b57a46716900
>>> Author: Bjorn Helgaas <bhelgaas@google.com>
>>> Date:   Mon Oct 31 16:00:01 2016 -0500
>>>
>>>     PCI: Warn on possible RW1C corruption for sub-32 bit config writes
>>>
>>>     Hardware that supports only 32-bit config writes is not spec-compliant.
>>>     For example, if software performs a 16-bit write, we must do a 32-bit read,
>>>     merge in the 16 bits we intend to write, followed by a 32-bit write.  If
>>>     the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
>>>     to-clear) bits set, we just inadvertently cleared something we shouldn't
>>>     have.
>>>
>>>     Add a rate-limited warning when we do sub-32 bit config writes.  Remove
>>>     similar probe-time warnings from some of the affected host bridge drivers.
>>>
>>>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>>     Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
>>>     Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
>>>     Acked-by: Thierry Reding <treding@nvidia.com>
>>>
>>> This warning message will show up on every root complex that can only
>>> support 32-bit config register write (instead of 8-bit, 16-bit writes)
>>> and iProc based host controller happens to be one of them.
>>>
>>> This limitation violates the PCIe spec as the 32-bit write may clear
>>> RW1C bits of a config register. But I believe those registers are
>>> usually used in AER which we do not currently support in the iProc
>>> based hos controllers.
>>
>> This warning is ratelimited, but maybe even that is overkill.  It
>> looks like we currently warn on every config write of less than 32
>> bits.  How many of those do you see?  Once per boot might be enough,
>> or maybe once per device (so we get a hint when loading a new driver).
>
> It's fairly trivial to change the warning in question to
> dev_warn_once().  If this is acceptable, I can crank it out.
>
> Thanks,
> Jon
>
>>> On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
>>> > warnings using pcie_iproc; is this something to be worried about?
>>> >
>>> > [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
>>> >
>>> >
>>> >
>>> > [    2.632358] pci_bus 0000:00: root bus resource [mem
>>> > 0x08000000-0x0fffffff]
>>> >
>>> >
>>> > [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
>>> > will use [bus 00-ff]
>>> >
>>> >
>>> > [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x4 may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x4 may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x4c may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x3e may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x4 may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
>>> > offset 0xc may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.705531] PCI: bus0: Fast back to back transfers disabled
>>> >
>>> >
>>> >
>>> > [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
>>> > 00-00]), reconfiguring
>>> >
>>> >
>>> > [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x3e may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0xc8 may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x3e may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> > offset 0x3e may corrupt adjacent RW1C bits
>>> >
>>> >
>>> > [    2.758262] pci 0000:01:00.0: enabling Extended Tags
>>> >
>>> >
>>> >
>>> > [    2.763577] PCI: bus1: Fast back to back transfers disabled
>>> >
>>> >
>>> >
>>> > [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
>>> > 0x08000000-0x080fffff]
>>> >
>>> >
>>> > [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
>>> > 0x08000000-0x08003fff 64bit]
>>> >
>>> >
>>> > [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
>>> >
>>> >
>>> >
>>> > [    2.788293] pci 0000:00:00.0:   bridge window [mem
>>> > 0x08000000-0x080fffff]
>>> >
>>> >
>>> > [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
>>> >
>>> >
>>> >
>>> > [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
>>> >
>>> >
>>> >
>>> > [    2.922349] pci_bus 0001:00: root bus resource [mem
>>> > 0x40000000-0x47ffffff]
>>> >
>>> >
>>> > [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
>>> > will use [bus 00-ff]
>>> >
>>> >
>>> > [    2.937482] PCI: bus0: Fast back to back transfers disabled
>>> >
>>> >
>>> >
>>> > [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
>>> > 00-00]), reconfiguring
>>> >
>>> >
>>> > [    2.951713] PCI: bus1: Fast back to back transfers disabled
>>> >
>>> >
>>> >
>>> > [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
>>> > 0x40000000-0x400fffff]
>>> >
>>> >
>>> > [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
>>> > 0x40000000-0x40007fff 64bit]
>>> >
>>> >
>>> > [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
>>> >
>>> >
>>> >
>>> > [    2.976434] pci 0001:00:00.0:   bridge window [mem
>>> > 0x40000000-0x400fffff]
>>> > --
>>> > Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10 16:18       ` Jim Quinlan
@ 2017-10-10 16:43         ` David Laight
  2017-10-10 20:09         ` Bjorn Helgaas
  1 sibling, 0 replies; 10+ messages in thread
From: David Laight @ 2017-10-10 16:43 UTC (permalink / raw)
  To: 'Jim Quinlan', Jon Mason
  Cc: Bjorn Helgaas, Ray Jui, Florian Fainelli,
	bcm-kernel-feedback-list, Ray Jui, Hauke Mehrtens, Rafal Milecki,
	Bjorn Helgaas, linux-pci

PiBCVFcsIGNhbiBzb21lb25lIHBvaW50IG91dCB0byBtZSB3aGF0IHByZWNpc2UgcHJvdmlzaW9u
IGluIHRoZSBQQ0kNCj4gc3BlYyBzYXlzIHRoYXQgOC8xNi8zMiBiaXQgY29uZmlnIHNwYWNlIHdy
aXRlcyBtdXN0IGJlIHByb3ZpZGVkPw0KDQpGb3IgUENJZToNCg0KMi4yLjUuIEZpcnN0L0xhc3Qg
RFcgQnl0ZSBFbmFibGVzIFJ1bGVzDQpCeXRlIEVuYWJsZXMgYXJlIGluY2x1ZGVkIHdpdGggTWVt
b3J5LCBJL08sIGFuZCBDb25maWd1cmF0aW9uIFJlcXVlc3RzLiBUaGlzIHNlY3Rpb24gZGVmaW5l
cyB0aGUNCmNvcnJlc3BvbmRpbmcgcnVsZXMuIEJ5dGUgRW5hYmxlcywgd2hlbiBwcmVzZW50IGlu
IHRoZSBSZXF1ZXN0IGhlYWRlciwgYXJlIGxvY2F0ZWQgaW4gYnl0ZSA3IG9mIHRoZQ0KaGVhZGVy
IChzZWUgRmlndXJlIDItOSkuDQoNCglEYXZpZA0KDQo=

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10 16:18       ` Jim Quinlan
  2017-10-10 16:43         ` David Laight
@ 2017-10-10 20:09         ` Bjorn Helgaas
  1 sibling, 0 replies; 10+ messages in thread
From: Bjorn Helgaas @ 2017-10-10 20:09 UTC (permalink / raw)
  To: Jim Quinlan
  Cc: Jon Mason, Ray Jui, Florian Fainelli, bcm-kernel-feedback-list,
	Ray Jui, Hauke Mehrtens, Rafał Miłecki, Bjorn Helgaas,
	linux-pci

On Tue, Oct 10, 2017 at 12:18:58PM -0400, Jim Quinlan wrote:
> BTW, can someone point out to me what precise provision in the PCI
> spec says that 8/16/32 bit config space writes must be provided?

PCI r3.0, sec 3.2.2.3.2 ("Software Generation of Configuration
Transactions") says

  In both Type 0 and Type 1 translations, byte enables for the data
  transfers must be directly copied from the processor bus.

Maybe I should have mentioned that in the changelog; I guess I thought
it was obvious from the example in the changelog that we can't handle
RW1C bits in 8- and 16-bit registers correctly unless the hardware can
perform 8- and 16-bit config writes.

> On Tue, Oct 10, 2017 at 10:59 AM, Jon Mason <jon.mason@broadcom.com> wrote:
> > On Tue, Oct 10, 2017 at 7:10 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> >> On Mon, Oct 09, 2017 at 10:33:01PM -0700, Ray Jui wrote:
> >>> Hi Florian,
> >>>
> >>> This is from the following commit:
> >>>
> >>> commit fb26592301200dbbe4a9943fe188b57a46716900
> >>> Author: Bjorn Helgaas <bhelgaas@google.com>
> >>> Date:   Mon Oct 31 16:00:01 2016 -0500
> >>>
> >>>     PCI: Warn on possible RW1C corruption for sub-32 bit config writes
> >>>
> >>>     Hardware that supports only 32-bit config writes is not spec-compliant.
> >>>     For example, if software performs a 16-bit write, we must do a 32-bit read,
> >>>     merge in the 16 bits we intend to write, followed by a 32-bit write.  If
> >>>     the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
> >>>     to-clear) bits set, we just inadvertently cleared something we shouldn't
> >>>     have.
> >>>
> >>>     Add a rate-limited warning when we do sub-32 bit config writes.  Remove
> >>>     similar probe-time warnings from some of the affected host bridge drivers.
> >>>
> >>>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> >>>     Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
> >>>     Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
> >>>     Acked-by: Thierry Reding <treding@nvidia.com>
> >>>
> >>> This warning message will show up on every root complex that can only
> >>> support 32-bit config register write (instead of 8-bit, 16-bit writes)
> >>> and iProc based host controller happens to be one of them.
> >>>
> >>> This limitation violates the PCIe spec as the 32-bit write may clear
> >>> RW1C bits of a config register. But I believe those registers are
> >>> usually used in AER which we do not currently support in the iProc
> >>> based hos controllers.
> >>
> >> This warning is ratelimited, but maybe even that is overkill.  It
> >> looks like we currently warn on every config write of less than 32
> >> bits.  How many of those do you see?  Once per boot might be enough,
> >> or maybe once per device (so we get a hint when loading a new driver).
> >
> > It's fairly trivial to change the warning in question to
> > dev_warn_once().  If this is acceptable, I can crank it out.
> >
> > Thanks,
> > Jon
> >
> >>> On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
> >>> > Hi,
> >>> >
> >>> > With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
> >>> > warnings using pcie_iproc; is this something to be worried about?
> >>> >
> >>> > [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
> >>> >
> >>> >
> >>> >
> >>> > [    2.632358] pci_bus 0000:00: root bus resource [mem
> >>> > 0x08000000-0x0fffffff]
> >>> >
> >>> >
> >>> > [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
> >>> > will use [bus 00-ff]
> >>> >
> >>> >
> >>> > [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x4 may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x4 may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x4c may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x3e may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x4 may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
> >>> > offset 0xc may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.705531] PCI: bus0: Fast back to back transfers disabled
> >>> >
> >>> >
> >>> >
> >>> > [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
> >>> > 00-00]), reconfiguring
> >>> >
> >>> >
> >>> > [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x3e may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0xc8 may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x3e may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >>> > offset 0x3e may corrupt adjacent RW1C bits
> >>> >
> >>> >
> >>> > [    2.758262] pci 0000:01:00.0: enabling Extended Tags
> >>> >
> >>> >
> >>> >
> >>> > [    2.763577] PCI: bus1: Fast back to back transfers disabled
> >>> >
> >>> >
> >>> >
> >>> > [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
> >>> > 0x08000000-0x080fffff]
> >>> >
> >>> >
> >>> > [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
> >>> > 0x08000000-0x08003fff 64bit]
> >>> >
> >>> >
> >>> > [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
> >>> >
> >>> >
> >>> >
> >>> > [    2.788293] pci 0000:00:00.0:   bridge window [mem
> >>> > 0x08000000-0x080fffff]
> >>> >
> >>> >
> >>> > [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
> >>> >
> >>> >
> >>> >
> >>> > [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
> >>> >
> >>> >
> >>> >
> >>> > [    2.922349] pci_bus 0001:00: root bus resource [mem
> >>> > 0x40000000-0x47ffffff]
> >>> >
> >>> >
> >>> > [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
> >>> > will use [bus 00-ff]
> >>> >
> >>> >
> >>> > [    2.937482] PCI: bus0: Fast back to back transfers disabled
> >>> >
> >>> >
> >>> >
> >>> > [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
> >>> > 00-00]), reconfiguring
> >>> >
> >>> >
> >>> > [    2.951713] PCI: bus1: Fast back to back transfers disabled
> >>> >
> >>> >
> >>> >
> >>> > [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
> >>> > 0x40000000-0x400fffff]
> >>> >
> >>> >
> >>> > [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
> >>> > 0x40000000-0x40007fff 64bit]
> >>> >
> >>> >
> >>> > [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
> >>> >
> >>> >
> >>> >
> >>> > [    2.976434] pci 0001:00:00.0:   bridge window [mem
> >>> > 0x40000000-0x400fffff]
> >>> > --
> >>> > Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10 14:59     ` Jon Mason
  2017-10-10 16:18       ` Jim Quinlan
@ 2017-10-10 21:00       ` Bjorn Helgaas
  1 sibling, 0 replies; 10+ messages in thread
From: Bjorn Helgaas @ 2017-10-10 21:00 UTC (permalink / raw)
  To: Jon Mason
  Cc: Ray Jui, Florian Fainelli, bcm-kernel-feedback-list, Ray Jui,
	Hauke Mehrtens, Rafał Miłecki, Bjorn Helgaas, linux-pci

On Tue, Oct 10, 2017 at 10:59:53AM -0400, Jon Mason wrote:
> On Tue, Oct 10, 2017 at 7:10 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> > On Mon, Oct 09, 2017 at 10:33:01PM -0700, Ray Jui wrote:
> >> Hi Florian,
> >>
> >> This is from the following commit:
> >>
> >> commit fb26592301200dbbe4a9943fe188b57a46716900
> >> Author: Bjorn Helgaas <bhelgaas@google.com>
> >> Date:   Mon Oct 31 16:00:01 2016 -0500
> >>
> >>     PCI: Warn on possible RW1C corruption for sub-32 bit config writes
> >>
> >>     Hardware that supports only 32-bit config writes is not spec-compliant.
> >>     For example, if software performs a 16-bit write, we must do a 32-bit read,
> >>     merge in the 16 bits we intend to write, followed by a 32-bit write.  If
> >>     the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
> >>     to-clear) bits set, we just inadvertently cleared something we shouldn't
> >>     have.
> >>
> >>     Add a rate-limited warning when we do sub-32 bit config writes.  Remove
> >>     similar probe-time warnings from some of the affected host bridge drivers.
> >>
> >>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> >>     Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
> >>     Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
> >>     Acked-by: Thierry Reding <treding@nvidia.com>
> >>
> >> This warning message will show up on every root complex that can only
> >> support 32-bit config register write (instead of 8-bit, 16-bit writes)
> >> and iProc based host controller happens to be one of them.
> >>
> >> This limitation violates the PCIe spec as the 32-bit write may clear
> >> RW1C bits of a config register. But I believe those registers are
> >> usually used in AER which we do not currently support in the iProc
> >> based hos controllers.
> >
> > This warning is ratelimited, but maybe even that is overkill.  It
> > looks like we currently warn on every config write of less than 32
> > bits.  How many of those do you see?  Once per boot might be enough,
> > or maybe once per device (so we get a hint when loading a new driver).
> 
> It's fairly trivial to change the warning in question to
> dev_warn_once().  If this is acceptable, I can crank it out.

I think my rationale was to make dmesg greppable for the device, e.g.,
if we had a problem with device 0000:00:00.0, "dmesg | grep
0000:00:00.0" would show this warning.

If we do a dev_warn_once(), we'll only see the warning once per
system, so we may see the warning for "pci_bus 0000:00", but we could
have an RW1C problem with a device on a different bus.  So I'm a
little hesitant to do that.

I'd really like it if we had the pci_dev; then we could add a bit in
the pci_dev and print the warning once for each device that's
vulnerable.  But we don't.

We used to have the host bridge driver emit one warning, but that was
unreliable because several drivers with this hardware limitation
didn't emit a warning.  fb2659230120 only removed the warning from
hisi and rockchip.  At least mach-ks8695, mach-sa1100, tegra, xgene,
hisi, iproc, and qcom also have the limitation but didn't emit a
warning.

Ironically, rockchip still has the hardware limitation but doesn't
emit a warning because it doesn't use pci_generic_config_write32().

> >> On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
> >> > warnings using pcie_iproc; is this something to be worried about?
> >> >
> >> > [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
> >> >
> >> >
> >> >
> >> > [    2.632358] pci_bus 0000:00: root bus resource [mem
> >> > 0x08000000-0x0fffffff]
> >> >
> >> >
> >> > [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
> >> > will use [bus 00-ff]
> >> >
> >> >
> >> > [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x4 may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x4 may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x4c may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x3e may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x4 may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
> >> > offset 0xc may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.705531] PCI: bus0: Fast back to back transfers disabled
> >> >
> >> >
> >> >
> >> > [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
> >> > 00-00]), reconfiguring
> >> >
> >> >
> >> > [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x3e may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0xc8 may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x3e may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
> >> > offset 0x3e may corrupt adjacent RW1C bits
> >> >
> >> >
> >> > [    2.758262] pci 0000:01:00.0: enabling Extended Tags
> >> >
> >> >
> >> >
> >> > [    2.763577] PCI: bus1: Fast back to back transfers disabled
> >> >
> >> >
> >> >
> >> > [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
> >> > 0x08000000-0x080fffff]
> >> >
> >> >
> >> > [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
> >> > 0x08000000-0x08003fff 64bit]
> >> >
> >> >
> >> > [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
> >> >
> >> >
> >> >
> >> > [    2.788293] pci 0000:00:00.0:   bridge window [mem
> >> > 0x08000000-0x080fffff]
> >> >
> >> >
> >> > [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
> >> >
> >> >
> >> >
> >> > [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
> >> >
> >> >
> >> >
> >> > [    2.922349] pci_bus 0001:00: root bus resource [mem
> >> > 0x40000000-0x47ffffff]
> >> >
> >> >
> >> > [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
> >> > will use [bus 00-ff]
> >> >
> >> >
> >> > [    2.937482] PCI: bus0: Fast back to back transfers disabled
> >> >
> >> >
> >> >
> >> > [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
> >> > 00-00]), reconfiguring
> >> >
> >> >
> >> > [    2.951713] PCI: bus1: Fast back to back transfers disabled
> >> >
> >> >
> >> >
> >> > [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
> >> > 0x40000000-0x400fffff]
> >> >
> >> >
> >> > [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
> >> > 0x40000000-0x40007fff 64bit]
> >> >
> >> >
> >> > [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
> >> >
> >> >
> >> >
> >> > [    2.976434] pci 0001:00:00.0:   bridge window [mem
> >> > 0x40000000-0x400fffff]
> >> > --
> >> > Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pci_bus warnings about adjacent RW1C bits w/ iproc-pci
  2017-10-10 11:10   ` Bjorn Helgaas
  2017-10-10 14:56     ` Jim Quinlan
  2017-10-10 14:59     ` Jon Mason
@ 2017-10-10 20:01     ` Florian Fainelli
  2 siblings, 0 replies; 10+ messages in thread
From: Florian Fainelli @ 2017-10-10 20:01 UTC (permalink / raw)
  To: Bjorn Helgaas, Ray Jui
  Cc: Florian Fainelli, bcm-kernel-feedback-list, Ray Jui, Jon Mason,
	Hauke Mehrtens, Rafał Miłecki, Bjorn Helgaas, linux-pci

On 10/10/2017 04:10 AM, Bjorn Helgaas wrote:
> On Mon, Oct 09, 2017 at 10:33:01PM -0700, Ray Jui wrote:
>> Hi Florian,
>>
>> This is from the following commit:
>>
>> commit fb26592301200dbbe4a9943fe188b57a46716900
>> Author: Bjorn Helgaas <bhelgaas@google.com>
>> Date:   Mon Oct 31 16:00:01 2016 -0500
>>
>>     PCI: Warn on possible RW1C corruption for sub-32 bit config writes
>>
>>     Hardware that supports only 32-bit config writes is not spec-compliant.
>>     For example, if software performs a 16-bit write, we must do a 32-bit read,
>>     merge in the 16 bits we intend to write, followed by a 32-bit write.  If
>>     the 16 bits we *don't* intend to write happen to have any RW1C (write-one-
>>     to-clear) bits set, we just inadvertently cleared something we shouldn't
>>     have.
>>
>>     Add a rate-limited warning when we do sub-32 bit config writes.  Remove
>>     similar probe-time warnings from some of the affected host bridge drivers.
>>
>>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>     Enthusiastically-Acked-by: Russell King <rmk+kernel@armlinux.org.uk>
>>     Acked-by: Shawn Lin <shawn.lin@rock-chips.com>      # rockchip
>>     Acked-by: Thierry Reding <treding@nvidia.com>
>>
>> This warning message will show up on every root complex that can only
>> support 32-bit config register write (instead of 8-bit, 16-bit writes)
>> and iProc based host controller happens to be one of them.
>>
>> This limitation violates the PCIe spec as the 32-bit write may clear
>> RW1C bits of a config register. But I believe those registers are
>> usually used in AER which we do not currently support in the iProc
>> based hos controllers.
> 
> This warning is ratelimited, but maybe even that is overkill.  It
> looks like we currently warn on every config write of less than 32
> bits.  How many of those do you see?  Once per boot might be enough,
> or maybe once per device (so we get a hint when loading a new driver). 

These were the exact number that I saw but I did not bind the Wi-Fi
driver to the WLAN card, so there could have been more once the Wi-Fi
driver attempted to initialize the device.

Would it be helpful to differentiate e.g: new drivers being submitted
that may support 8/16/32 bit accesses from drivers like iProc PCIe which
works with silicon that does not support that?

Either way is fine with me really.

> 
>> On Mon, Oct 9, 2017 at 8:58 PM, Florian Fainelli <f.fainelli@gmail.com> wrote:
>>> Hi,
>>>
>>> With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core
>>> warnings using pcie_iproc; is this something to be worried about?
>>>
>>> [    2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
>>>
>>>
>>>
>>> [    2.632358] pci_bus 0000:00: root bus resource [mem
>>> 0x08000000-0x0fffffff]
>>>
>>>
>>> [    2.639207] pci_bus 0000:00: No busn resource found for root bus,
>>> will use [bus 00-ff]
>>>
>>>
>>> [    2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x4 may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x4 may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x4c may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x3e may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x4 may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
>>> offset 0xc may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.705531] PCI: bus0: Fast back to back transfers disabled
>>>
>>>
>>>
>>> [    2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
>>> 00-00]), reconfiguring
>>>
>>>
>>> [    2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x3e may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0xc8 may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x3e may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
>>> offset 0x3e may corrupt adjacent RW1C bits
>>>
>>>
>>> [    2.758262] pci 0000:01:00.0: enabling Extended Tags
>>>
>>>
>>>
>>> [    2.763577] PCI: bus1: Fast back to back transfers disabled
>>>
>>>
>>>
>>> [    2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
>>> 0x08000000-0x080fffff]
>>>
>>>
>>> [    2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
>>> 0x08000000-0x08003fff 64bit]
>>>
>>>
>>> [    2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
>>>
>>>
>>>
>>> [    2.788293] pci 0000:00:00.0:   bridge window [mem
>>> 0x08000000-0x080fffff]
>>>
>>>
>>> [    2.911536] pcie_iproc_bcma bcma0:8: link: UP
>>>
>>>
>>>
>>> [    2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
>>>
>>>
>>>
>>> [    2.922349] pci_bus 0001:00: root bus resource [mem
>>> 0x40000000-0x47ffffff]
>>>
>>>
>>> [    2.929200] pci_bus 0001:00: No busn resource found for root bus,
>>> will use [bus 00-ff]
>>>
>>>
>>> [    2.937482] PCI: bus0: Fast back to back transfers disabled
>>>
>>>
>>>
>>> [    2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
>>> 00-00]), reconfiguring
>>>
>>>
>>> [    2.951713] PCI: bus1: Fast back to back transfers disabled
>>>
>>>
>>>
>>> [    2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
>>> 0x40000000-0x400fffff]
>>>
>>>
>>> [    2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
>>> 0x40000000-0x40007fff 64bit]
>>>
>>>
>>> [    2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
>>>
>>>
>>>
>>> [    2.976434] pci 0001:00:00.0:   bridge window [mem
>>> 0x40000000-0x400fffff]
>>> --
>>> Florian


-- 
Florian

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2017-10-10 21:00 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-10-10  3:58 pci_bus warnings about adjacent RW1C bits w/ iproc-pci Florian Fainelli
2017-10-10  5:33 ` Ray Jui
2017-10-10 11:10   ` Bjorn Helgaas
2017-10-10 14:56     ` Jim Quinlan
2017-10-10 14:59     ` Jon Mason
2017-10-10 16:18       ` Jim Quinlan
2017-10-10 16:43         ` David Laight
2017-10-10 20:09         ` Bjorn Helgaas
2017-10-10 21:00       ` Bjorn Helgaas
2017-10-10 20:01     ` Florian Fainelli

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).