PCI resource problem on C360

PCI resource problem on C360

[Tom Bogendoerfer]

Hi,

while trying to make progress with supporting VGA cards, I stumbled
over following problem on a C360:

pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
pci_bus 0000:00: root bus resource [mem 0xfffffffff2800000-0xfffffffff3ffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: scanning bus
pci 0000:00:03.0: [1002:5654] type 00 class 0x030000
pci 0000:00:03.0: reg 0x10: [mem 0xf3000000-0xf3ffffff]
pci 0000:00:03.0: reg 0x14: [io  0xfd00-0xfdff]
pci 0000:00:03.0: reg 0x30: [mem 0xf2f80000-0xf2f8ffff pref]
pci 0000:00:03.0: calling quirk_no_pm_reset+0x0/0x30
pci 0000:00:13.0: [1000:000f] type 00 class 0x010000
pci 0000:00:13.0: reg 0x10: [io  0xfe00-0xfeff]
pci 0000:00:13.0: reg 0x14: [mem 0xf2ffd000-0xf2ffd0ff]
pci 0000:00:13.0: reg 0x18: [mem 0xf2ffe000-0xf2ffefff]
pci 0000:00:14.0: [1011:0019] type 00 class 0x020000
pci 0000:00:14.0: reg 0x10: [io  0xff00-0xff7f]
pci 0000:00:14.0: reg 0x14: [mem 0xf2fff000-0xf2fff07f]
pci 0000:00:14.0: reg 0x30: [mem 0xf2f80000-0xf2fbffff pref]
pci_bus 0000:00: fixups for bus
pci_bus 0000:00: bus scan returning with max=00
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
pci 0000:00:03.0: BAR 0: no space for [mem size 0x01000000]
pci 0000:00:03.0: BAR 0: failed to assign [mem size 0x01000000]
pci 0000:00:13.0: BAR 2: no space for [mem size 0x00001000]
pci 0000:00:13.0: BAR 2: failed to assign [mem size 0x00001000]
pci 0000:00:03.0: BAR 1: assigned [io  0x0100-0x01ff]
pci 0000:00:13.0: BAR 0: assigned [io  0x0200-0x02ff]
pci 0000:00:13.0: BAR 1: no space for [mem size 0x00000100]
pci 0000:00:13.0: BAR 1: failed to assign [mem size 0x00000100]
pci 0000:00:14.0: BAR 0: assigned [io  0x0080-0x00ff]
pci 0000:00:14.0: BAR 1: no space for [mem size 0x00000080]
pci 0000:00:14.0: BAR 1: failed to assign [mem size 0x0000008

Is this a known problem ? Source tree is current parisc git.

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

Re: PCI resource problem on C360

[John David Anglin]

On 2017-08-03, at 10:07 AM, Tom Bogendoerfer wrote:

> Is this a known problem ? Source tree is current parisc git.

I don't think so.  Doesn't happen on rp3440:

[    1.820224] LBA 0:0: PCI host bridge to bus 0000:00
[    1.820365] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    1.820528] pci_bus 0000:00: root bus resource [mem 0xffffffff80000000-0xffff
ffff8fffffff] (bus address [0x80000000-0x8fffffff])[    1.821090] pci_bus 0000:00: root bus resource [bus 00-07]
[    1.828085] pci 0000:00:01.0: [1033:0035] type 00 class 0x0c0310[    1.828130] pci 0000:00:01.0: reg 0x10: [mem 0xffffffff80002000-0xffffffff800
02fff]
[    1.828267] pci 0000:00:01.0: supports D1 D2
[    1.828273] pci 0000:00:01.0: PME# supported from D0 D1 D2 D3hot
[    1.830081] pci 0000:00:01.1: [1033:0035] type 00 class 0x0c0310[    1.830120] pci 0000:00:01.1: reg 0x10: [mem 0xffffffff80001000-0xffffffff800
01fff][    1.830247] pci 0000:00:01.1: supports D1 D2
[    1.830253] pci 0000:00:01.1: PME# supported from D0 D1 D2 D3hot
[    1.830371] pci 0000:00:01.2: [1033:00e0] type 00 class 0x0c0320
[    1.830409] pci 0000:00:01.2: reg 0x10: [mem 0xffffffff80000000-0xffffffff800000ff]
[    1.830534] pci 0000:00:01.2: supports D1 D2
[    1.830540] pci 0000:00:01.2: PME# supported from D0 D1 D2 D3hot
[    1.830666] pci 0000:00:02.0: [1095:0649] type 00 class 0x01018f
[    1.830703] pci 0000:00:02.0: reg 0x10: [io  0x0d18-0x0d1f]
[    1.830723] pci 0000:00:02.0: reg 0x14: [io  0x0d24-0x0d27]
[    1.830743] pci 0000:00:02.0: reg 0x18: [io  0x0d10-0x0d17]
[    1.830762] pci 0000:00:02.0: reg 0x1c: [io  0x0d20-0x0d23]
[    1.830782] pci 0000:00:02.0: reg 0x20: [io  0x0d00-0x0d0f]
[    1.830855] pci 0000:00:02.0: supports D1 D2
[    1.850047] Mercury version TR3.2 (0x32) found at 0xfffffffffed22000
[    1.910206] LBA 0:1: PCI host bridge to bus 0000:20
[    1.910349] pci_bus 0000:20: root bus resource [io  0x10000-0x1ffff] (bus address [0x0000-0xffff])
[    1.910808] pci_bus 0000:20: root bus resource [mem 0xffffffff90000000-0xffffffff9fffffff] (bus address [0x90000000-0x9fffffff])
[    1.918145] pci_bus 0000:20: root bus resource [bus 20-27]
[    1.918320] pci 0000:20:01.0: [1000:0021] type 00 class 0x010000
[    1.918356] pci 0000:20:01.0: reg 0x10: [io  0x12100-0x121ff]
[    1.918378] pci 0000:20:01.0: reg 0x14: [mem 0xffffffff90015000-0xffffffff900153ff 64bit]
[    1.918400] pci 0000:20:01.0: reg 0x1c: [mem 0xffffffff90012000-0xffffffff90013fff 64bit]
[    1.918464] pci 0000:20:01.0: supports D1 D2
[    1.918595] pci 0000:20:01.1: [1000:0021] type 00 class 0x010000
[    1.918623] pci 0000:20:01.1: reg 0x10: [io  0x12000-0x120ff]
[    1.918645] pci 0000:20:01.1: reg 0x14: [mem 0xffffffff90014000-0xffffffff900143ff 64bit]
[    1.918667] pci 0000:20:01.1: reg 0x1c: [mem 0xffffffff90010000-0xffffffff90011fff 64bit]
[    1.918721] pci 0000:20:01.1: supports D1 D2
[    1.918839] pci 0000:20:02.0: [14e4:1645] type 00 class 0x020000
[    1.918876] pci 0000:20:02.0: reg 0x10: [mem 0xffffffff90000000-0xffffffff9000ffff 64bit]
[    1.918981] pci 0000:20:02.0: PME# supported from D3hot D3cold
[    1.940046] Mercury version TR3.2 (0x32) found at 0xfffffffffed24000
[    1.998842] LBA 0:2: PCI host bridge to bus 0000:40
[    1.998985] pci_bus 0000:40: root bus resource [io  0x20000-0x2ffff] (bus address [0x0000-0xffff])
[    1.999459] pci_bus 0000:40: root bus resource [mem 0xffffffffa0000000-0xffffffffafffffff] (bus address [0xa0000000-0xafffffff])
[    2.000039] pci_bus 0000:40: root bus resource [bus 40-47]
[    2.028845] Mercury version TR3.2 (0x32) found at 0xfffffffffed26000
[    2.080237] LBA 0:3: PCI host bridge to bus 0000:60
[    2.080561] pci_bus 0000:60: root bus resource [io  0x30000-0x3ffff] (bus address [0x0000-0xffff])
[    2.081116] pci_bus 0000:60: root bus resource [mem 0xffffffffb0000000-0xffffffffbfffffff] (bus address [0xb0000000-0xbfffffff])
[    2.090055] pci_bus 0000:60: root bus resource [bus 60-67]
[    2.116809] Mercury version TR3.2 (0x32) found at 0xfffffffffed28000
[    2.150026] LBA: lmmio_space [0xffffffffc0000000-0xffffffffdfffffff] - new
[    2.170228] LBA 0:4: PCI host bridge to bus 0000:80
[    2.170371] pci_bus 0000:80: root bus resource [io  0x40000-0x4ffff] (bus address [0x0000-0xffff])
[    2.170826] pci_bus 0000:80: root bus resource [mem 0xffffffffc0000000-0xffffffffdfffffff] (bus address [0xc0000000-0xdfffffff])
[    2.180021] pci_bus 0000:80: root bus resource [bus 80-87]
[    2.200044] Mercury version TR3.2 (0x32) found at 0xfffffffffed2c000
[    2.260215] LBA 0:6: PCI host bridge to bus 0000:c0
[    2.260357] pci_bus 0000:c0: root bus resource [io  0x50000-0x5ffff] (bus address [0x0000-0xffff])
[    2.260806] pci_bus 0000:c0: root bus resource [mem 0xffffffffe0000000-0xffffffffefffffff] (bus address [0xe0000000-0xefffffff])
[    2.270021] pci_bus 0000:c0: root bus resource [bus c0-c7]
[    2.290047] Mercury version TR3.2 (0x32) found at 0xfffffffffed2e000
[    2.320027] LBA: lmmio_space [0xfffffffff0000000-0xfffffffffe77ffff] - new
[    2.350182] LBA 0:7: PCI host bridge to bus 0000:e0
[    2.350325] pci_bus 0000:e0: root bus resource [io  0x60000-0x6ffff] (bus address [0x0000-0xffff])
[    2.350789] pci_bus 0000:e0: root bus resource [mem 0xfffffffff0000000-0xfffffffffe77ffff] (bus address [0xf0000000-0xfe77ffff])
[    2.358052] pci_bus 0000:e0: root bus resource [bus e0-e7]
[    2.358246] pci 0000:e0:01.0: [103c:1290] type 00 class 0x078000
[    2.358329] pci 0000:e0:01.0: reg 0x18: [mem 0xfffffffff4051000-0xfffffffff405100f]
[    2.358589] pci 0000:e0:01.1: [103c:1048] type 00 class 0x070002
[    2.358634] pci 0000:e0:01.1: reg 0x10: [mem 0xfffffffff4050000-0xfffffffff4050fff]
[    2.358674] pci 0000:e0:01.1: reg 0x18: [mem 0xfffffffff4020000-0xfffffffff403ffff pref]
[    2.358921] pci 0000:e0:02.0: [1002:5159] type 00 class 0x030000
[    2.358965] pci 0000:e0:02.0: reg 0x10: [mem 0xfffffffff0000000-0xfffffffff3ffffff pref]
[    2.358987] pci 0000:e0:02.0: reg 0x14: [io  0x6e000-0x6e0ff]
[    2.359010] pci 0000:e0:02.0: reg 0x18: [mem 0xfffffffff4040000-0xfffffffff404ffff]
[    2.359080] pci 0000:e0:02.0: reg 0x30: [mem 0xfffffffff4000000-0xfffffffff401ffff pref]
[    2.359134] pci 0000:e0:02.0: supports D1 D2
[    2.360013] powersw: Soft power switch support not available.
[    2.370516] WARNING: workqueue cpumask: online intersect > possible intersect
[    2.391187] pci 0000:e0:02.0: vgaarb: setting as boot VGA device
[    2.391359] pci 0000:e0:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    2.391805] pci 0000:e0:02.0: vgaarb: bridge control possible
[    2.391954] vgaarb: loaded

Dave
--
John David Anglin	dave.anglin@bell.net

Re: PCI resource problem on C360

[Helge Deller]

On 03.08.2017 17:58, John David Anglin wrote:
> On 2017-08-03, at 10:07 AM, Tom Bogendoerfer wrote:
> 
>> Is this a known problem ? Source tree is current parisc git.
> 
> I don't think so.  Doesn't happen on rp3440:

I had lots of such problems when I tried various VGA and other
PCI cards in my c3000.

I think HP designed many of the PCI slots for special usage only,
e.g. some slots allow PCI graphic cards while others don't.

Maybe plug your cards one-by-one into another slot and check again?

Helge

Re: PCI resource problem on C360

[John David Anglin]

On 2017-08-03, at 6:45 PM, Helge Deller wrote:

> On 03.08.2017 17:58, John David Anglin wrote:
>> On 2017-08-03, at 10:07 AM, Tom Bogendoerfer wrote:
>> 
>>> Is this a known problem ? Source tree is current parisc git.
>> 
>> I don't think so.  Doesn't happen on rp3440:
> 
> I had lots of such problems when I tried various VGA and other
> PCI cards in my c3000.
> 
> I think HP designed many of the PCI slots for special usage only,
> e.g. some slots allow PCI graphic cards while others don't.
> 
> Maybe plug your cards one-by-one into another slot and check again?

There's something about the firmware disabling dual graphics adapters in slots 1 or 2
if another is plugged into 3 or 4 in the user guide.  I would try slot 4 first.

Some C series units had feeble power supplies and various combination of HP graphics
cards aren't supported.

The c3000 has some PCIX slots and maybe both 3.3 and 5V PCI slots.

I suggest 64bit kernel as it is best tested.

Dave
--
John David Anglin	dave.anglin@bell.net

Re: PCI resource problem on C360

[Tom Bogendoerfer]

On Fri, Aug 04, 2017 at 12:45:31AM +0200, Helge Deller wrote:
> I had lots of such problems when I tried various VGA and other
> PCI cards in my c3000.

I know, but the get a MACH64 card up to sync in a B2600, with a small
change in aty driver. And the change is because Astro/Elroy behave incorrect
in respect to PCI standards, IMHO. It looks like it colapses multiple PCI
mem accesses into one access.

        aty_st_8(CLOCK_CNTL_ADDR, ((offset << 2) & PLL_ADDR) | PLL_WR_EN, par);
        /* write the register value */
        aty_st_8(CLOCK_CNTL_DATA, val & PLL_DATA, par);
        aty_st_8(CLOCK_CNTL_ADDR, ((offset << 2) & PLL_ADDR) & ~PLL_WR_EN, par);

B2600 HPMCs in this code sequence, probably because the two accesses
to CNTL_ADDR end up in just the last access. Placing in aty_ld_8 after
the first aty_st_8 avoids the problem. In a B180 everything is fine,
so Dino doesn't do that. Next testbed is the C360, which shows the PCI
resourec issue before it even comes to aty drvier.

> I think HP designed many of the PCI slots for special usage only,
> e.g. some slots allow PCI graphic cards while others don't.
> 
> Maybe plug your cards one-by-one into another slot and check again?

will do that, but what looks strange to me is the fact that ethernet and
scsi also don't get all resources assigned. This happened without the VGA
card plugged in, too.

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]

Re: PCI resource problem on C360

[Helge Deller]

On 08.08.2017 15:40, Tom Bogendoerfer wrote:
> On Fri, Aug 04, 2017 at 12:45:31AM +0200, Helge Deller wrote:
>> I had lots of such problems when I tried various VGA and other
>> PCI cards in my c3000.
> 
> I know, but the get a MACH64 card up to sync in a B2600, with a small
> change in aty driver. And the change is because Astro/Elroy behave incorrect
> in respect to PCI standards, IMHO. It looks like it colapses multiple PCI
> mem accesses into one access.

Yes, I think that's true.
 
>         aty_st_8(CLOCK_CNTL_ADDR, ((offset << 2) & PLL_ADDR) | PLL_WR_EN, par);
>         /* write the register value */
>         aty_st_8(CLOCK_CNTL_DATA, val & PLL_DATA, par);
>         aty_st_8(CLOCK_CNTL_ADDR, ((offset << 2) & PLL_ADDR) & ~PLL_WR_EN, par);
> 
> B2600 HPMCs in this code sequence, probably because the two accesses
> to CNTL_ADDR end up in just the last access. Placing in aty_ld_8 after
> the first aty_st_8 avoids the problem.

There are quite some comments in drivers/parisc/lba_pci.c regarding this.
Esp. the comment about "Directed" ranges at line 1258 might be interesting
regarding graphic cards.

Sadly I'm no expert in this area at all.

> In a B180 everything is fine,> so Dino doesn't do that. Next testbed is the C360, which shows the PCI
> resourec issue before it even comes to aty drvier.
> 
>> I think HP designed many of the PCI slots for special usage only,
>> e.g. some slots allow PCI graphic cards while others don't.
>>
>> Maybe plug your cards one-by-one into another slot and check again?
> 
> will do that, but what looks strange to me is the fact that ethernet and
> scsi also don't get all resources assigned. This happened without the VGA
> card plugged in, too.

Maybe the drivers of such NIC/SCSI cards don't request all available resources? 

Helge

Re: PCI resource problem on C360

[John David Anglin]

On 2017-08-08 11:48 AM, Helge Deller wrote:
>> I know, but the get a MACH64 card up to sync in a B2600, with a small
>> change in aty driver. And the change is because Astro/Elroy behave incorrect
>> in respect to PCI standards, IMHO. It looks like it colapses multiple PCI
>> mem accesses into one access.
> Yes, I think that's true.
>   
>>          aty_st_8(CLOCK_CNTL_ADDR, ((offset << 2) & PLL_ADDR) | PLL_WR_EN, par);
>>          /* write the register value */
>>          aty_st_8(CLOCK_CNTL_DATA, val & PLL_DATA, par);
>>          aty_st_8(CLOCK_CNTL_ADDR, ((offset << 2) & PLL_ADDR) & ~PLL_WR_EN, par);
>>
>> B2600 HPMCs in this code sequence, probably because the two accesses
>> to CNTL_ADDR end up in just the last access. Placing in aty_ld_8 after
>> the first aty_st_8 avoids the problem.
> There are quite some comments in drivers/parisc/lba_pci.c regarding this.
> Esp. the comment about "Directed" ranges at line 1258 might be interesting
> regarding graphic cards.

It says:

/************************************
  * LBA register read and write support
  *
  * BE WARNED: register writes are posted.
  *  (ie follow writes which must reach HW with a read)
  */

It seems that aty stores likely need a following read when writes are 
posted.  I would hack
drivers/video/fbdev/aty/aty128fb.c as a test.

Dave

-- 
John David Anglin  dave.anglin@bell.net

Re: PCI resource problem on C360

[Tom Bogendoerfer]

On Tue, Aug 08, 2017 at 05:48:33PM +0200, Helge Deller wrote:
> Maybe the drivers of such NIC/SCSI cards don't request all available
> resources? 

log is clear that the pci probing stuff is not able to setup pci resources.
And that's nothing a driver fix up later (without dirty tricks of course).
The problem is that dino code doesn't set the space_offset of the host
bridge correctly for 64bit kernels. Will send a patch later.

Now aty driver is happy with the mach64 card:

atyfb 0000:00:03.0: runtime IRQ mapping not provided by arch
atyfb 0000:00:03.0: enabling device (0082 -> 0083)
atyfb 0000:00:03.0: enabling SERR and PARITY (0083 -> 01c3)
atyfb: ATI264VT2 (A4) (Mach64 VT) [0x5654 rev 0x40]
atyfb: 512K RESV, 14.31818 MHz XTAL, 200 MHz PLL, 67 Mhz MCLK, 67 MHz XCLK
Console: switching to colour frame buffer device 80x30
atyfb: fb0: ATY Mach64 frame buffer device on PCI

Thomas.

-- 
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]