Access to PCI Expansion ROMs on PPC

Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]

Following up from initial conversation with benh in #ppc64, I did a bit
of testing (and then a bit more).

Two PCIe cards for my testing:
a) sata_sil24 eSATA (x86 BIOS)
b) ATI X700 graphics (x86 BIOS)

I put the card into an amd64 box, found the relevant 'rom' node ($ROM) under
/sys/device/pci*, and dumped it as follows:
# echo 1>$ROM
# cat $ROM >/tmp/dump
# echo 0>$ROM
On amd64, this produced the ROM for the cards fine (contained
recognizable strings).

I then repeated the same on my G5 (PM11,2), while the $ROM files have a
non-zero size in sysfs (that corresponds to the size shown by lspci
-vv), reading them always results in a 0-byte output.

Is the PPC kernel broken? Does something in the PPC architecture forbid
this? Instructions to muck with PCI code welcome.

This was under 2.6.24_rc3-git1, kernel .config available if needed.

Why am I trying to access the PCI ROM? To make my newer X1900 card work
under at least one of the newer (ati,radeonhd,avivo) drivers, that all
try to access the ATI AtomBIOS for varying data.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: Access to PCI Expansion ROMs on PPC

[Jon Smirl]

On 11/24/07, Robin H. Johnson <robbat2@gentoo.org> wrote:
> Following up from initial conversation with benh in #ppc64, I did a bit
> of testing (and then a bit more).
>
> Two PCIe cards for my testing:
> a) sata_sil24 eSATA (x86 BIOS)
> b) ATI X700 graphics (x86 BIOS)
>
> I put the card into an amd64 box, found the relevant 'rom' node ($ROM) under
> /sys/device/pci*, and dumped it as follows:
> # echo 1>$ROM
> # cat $ROM >/tmp/dump
> # echo 0>$ROM
> On amd64, this produced the ROM for the cards fine (contained
> recognizable strings).
>
> I then repeated the same on my G5 (PM11,2), while the $ROM files have a
> non-zero size in sysfs (that corresponds to the size shown by lspci
> -vv), reading them always results in a 0-byte output.
>
> Is the PPC kernel broken? Does something in the PPC architecture forbid
> this? Instructions to muck with PCI code welcome.

The ROM is mapped in drivers/pci/rom.c

You could add some printks and see if there is an error and if the ROM
is accessible

        rom = ioremap(start, *size);
        if (!rom) {
                /* restore enable if ioremap fails */
                if (!(res->flags & (IORESOURCE_ROM_ENABLE |
                                    IORESOURCE_ROM_SHADOW |
                                    IORESOURCE_ROM_COPY)))
                        pci_disable_rom(pdev);
                return NULL;
        }



>
> This was under 2.6.24_rc3-git1, kernel .config available if needed.
>
> Why am I trying to access the PCI ROM? To make my newer X1900 card work
> under at least one of the newer (ati,radeonhd,avivo) drivers, that all
> try to access the ATI AtomBIOS for varying data.
>
> --
> Robin Hugh Johnson
> Gentoo Linux Developer & Infra Guy
> E-Mail     : robbat2@gentoo.org
> GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
>
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
>
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]

On Sat, Nov 24, 2007 at 09:13:40PM -0500, Jon Smirl wrote:
> The ROM is mapped in drivers/pci/rom.c
> 
> You could add some printks and see if there is an error and if the ROM
> is accessible
> 
>         rom = ioremap(start, *size);
>         if (!rom) {
>                 /* restore enable if ioremap fails */
>                 if (!(res->flags & (IORESOURCE_ROM_ENABLE |
>                                     IORESOURCE_ROM_SHADOW |
>                                     IORESOURCE_ROM_COPY)))
>                         pci_disable_rom(pdev);
>                 return NULL;
>         }
I started in there from that, and ended up in pci-sysfs.c...
The ROM memcpy in drivers/pci/pci-sysfs.c:pci_read_rom() is never running.

Relevant section of my debug output:
[  306.396743] drivers/pci/pci-sysfs.c:577:pci_read_rom: size=0x0 rom=0xd000080082580000 off=0x0
[  306.396764] drivers/pci/pci-sysfs.c:579:pci_read_rom: off >= size!
[  306.396768] drivers/pci/pci-sysfs.c:588pci_read_rom: unmapping

Relevant snippet of my debug patching:
-       if (off >= size)
+       printk(KERN_INFO "%s:%d:%s: size=0x%lx rom=0x%lx off=0x%lx\n", __FILE__,__LINE__,__FUNCTION__,size,rom,off);
+       if (off >= size) {
+               printk(KERN_INFO "%s:%d:%s: off >= size!\n", __FILE__,__LINE__,__FUNCTION__);
                count = 0;
-       else {
+       } else {

So next, why is it failing to decode the ROM size correctly?
pci_get_rom_size(), here I come.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1118 bytes --]

On Sun, Nov 25, 2007 at 03:15:35AM -0800, Robin H. Johnson wrote:
> I started in there from that, and ended up in pci-sysfs.c...
> The ROM memcpy in drivers/pci/pci-sysfs.c:pci_read_rom() is never running.
...
> [  306.396743] drivers/pci/pci-sysfs.c:577:pci_read_rom: size=0x0 rom=0xd000080082580000 off=0x0
> [  306.396764] drivers/pci/pci-sysfs.c:579:pci_read_rom: off >= size!
...
> So next, why is it failing to decode the ROM size correctly?
> pci_get_rom_size(), here I come.
Instead of the expected value of 0x55, 0xAA at the start of the ROM
size decode in pci_get_rom_size(), the first two readb() calls both
return either 0xFF or 0x00 so the size check fails right away.

The two cards with x86 firmware return 0xFF for those two readb()
instructions, while the X1900 with OF returns 0x00 for the readb().

Could one of the more knowledgeable folk for PPC intricacies suggest why
those readb calls are returning the wrong data?

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: Access to PCI Expansion ROMs on PPC

[Jon Smirl]

On 11/25/07, Robin H. Johnson <robbat2@gentoo.org> wrote:
> On Sun, Nov 25, 2007 at 03:15:35AM -0800, Robin H. Johnson wrote:
> > I started in there from that, and ended up in pci-sysfs.c...
> > The ROM memcpy in drivers/pci/pci-sysfs.c:pci_read_rom() is never running.
> ...
> > [  306.396743] drivers/pci/pci-sysfs.c:577:pci_read_rom: size=0x0 rom=0xd000080082580000 off=0x0
> > [  306.396764] drivers/pci/pci-sysfs.c:579:pci_read_rom: off >= size!
> ...
> > So next, why is it failing to decode the ROM size correctly?
> > pci_get_rom_size(), here I come.
> Instead of the expected value of 0x55, 0xAA at the start of the ROM
> size decode in pci_get_rom_size(), the first two readb() calls both
> return either 0xFF or 0x00 so the size check fails right away.
>
> The two cards with x86 firmware return 0xFF for those two readb()
> instructions, while the X1900 with OF returns 0x00 for the readb().
>
> Could one of the more knowledgeable folk for PPC intricacies suggest why
> those readb calls are returning the wrong data?

I don't know PPC at this low of level but it may be a problem with non
word-aligned access to memory. I thought readb() was supposed to work
on all archs and alignment issues are handled inside readb(). Also,
the readw() may have an endian bug.

It might be prudent to add this check at the end of the rom size routine.
if (image == rom)
    return size;
That would ensure a non-zero size is returned for non-standard ROMs.

If you add that at the end, can you read the ROM from user space?

BenH, has source code for an x86 emulator that will run on PPC. That
will let you run the ROMs. The original plan was for the kernel to
generate a uevent that would have triggered the x86 emulator to run.

Progress along that path was blocked by the X developers. The X server
contains code for enabling the PCI ROM and reading it from user space.
I wanted to move this code out of X and into the kernel.

Because the path was blocked things like the PCI ROM API were never
throughly tested. It works most of the time but the occasional problem
is still turning up. Once we identify the PPC problem we can fix it in
the kernel.

>
> --
> Robin Hugh Johnson
> Gentoo Linux Developer & Infra Guy
> E-Mail     : robbat2@gentoo.org
> GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
>
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1421 bytes --]

On Sun, Nov 25, 2007 at 08:30:58AM -0500, Jon Smirl wrote:
> I don't know PPC at this low of level but it may be a problem with non
> word-aligned access to memory. I thought readb() was supposed to work
> on all archs and alignment issues are handled inside readb(). Also,
> the readw() may have an endian bug.
I was looking around for a description of the ROM layout, and instead I
found this: http://developer.apple.com/technotes/tn/tn2000.html
It was relevant because it explicitly mentioned enabling the
PCI_COMMAND_MEMORY bit in the PCI_COMMAND register, which is NOT present
in any path of the sizing code.

By doing:
# dev="/sys/devices/pci0001:00/0001:00:03.0/0001:06:00.0/"
# echo 1 >${dev}/enable
# echo 1 >${dev}/rom
# cat ${dev}/rom >rom
The ROM is successfully returned for two of my 3 cards.
Namely, both of the ones containing x86 BIOS (sata_sil24, ATI X700).
The X1900 does still not return any ROM.

So the pci-sysfs.c/rom.c code path needs to change to enable the device
somewhere if it was not enabled before (and presumably disable
afterwards for suspend/resume?).

I broke my testing kernel temporarily, so I need a bit more testing to
see why the X1900 did not return a ROM. More on that in a couple of
hours.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 3107 bytes --]

On Sun, Nov 25, 2007 at 11:41:01AM -0800, Robin H. Johnson wrote:
> I was looking around for a description of the ROM layout, and instead I
> found this: http://developer.apple.com/technotes/tn/tn2000.html
> It was relevant because it explicitly mentioned enabling the
> PCI_COMMAND_MEMORY bit in the PCI_COMMAND register, which is NOT present
> in any path of the sizing code.
...
> By doing:
> # dev="/sys/devices/pci0001:00/0001:00:03.0/0001:06:00.0/"
> # echo 1 >${dev}/enable
> # echo 1 >${dev}/rom
> # cat ${dev}/rom >rom
> The ROM is successfully returned for two of my 3 cards.
> Namely, both of the ones containing x86 BIOS (sata_sil24, ATI X700).
> The X1900 does still not return any ROM.
...
> I broke my testing kernel temporarily, so I need a bit more testing to
> see why the X1900 did not return a ROM. More on that in a couple of
> hours.
(Part of this is from an IRC discussion with benh, summarized with
results here).

Thanks to some friends, I added some more PCIe cards to the test set:
The complete list is now:
x86-BIOS:
- Silicon Image, Inc. SiI 3132 Serial ATA Raid II (sata_sil24)
- ATI X700
OpenFirmware (all get relevant nodes in the device-tree after boot):
- ATI X1900
- Nvidia 6600
- Marvell SATA 7042 (sata_mv) [Sonnet Tempo SATA E4P]

For testing, I explicitly enable the device, and then try to read the
ROM. Additionally, I have some debug printk statements scattered about
rom.c and pci-sysfs.c, to detect various parts of the branching.
Additionaly per Jon's suggestion, I also have a bit of code at the end
of the sizing routine:
if (image == rom) { printk(...); return size; }

On the G5, the results are as follows:
- sata_sil24, X700, sata_mv - ROMS readable and valid.
- Nvidia 6600 - Returns junk data, changes between boots.
  pci_get_rom_size fails the size decode, the first two readb() both
  return 0x33
- ATI X1900 - Returns nulls. pci_get_rom_size fails here, the readbs()
  return 0x00.

On an x86_64 (Core2 Duo), the results are as follows:
- sata_sil24 , X700 - ROMS readable and valid.
- sata_mv - appears in lspci, and works, but lspci does NOT show an 'Expansion ROM' line.
- Nvidia 6600, ATI X1900 - do not show up the kernel at all, or in
  lspci. The PCIe Root port [8086:29a1] is entirely missing from the
  lspci output.

Regarding the sub-thread on x86 emulation, that is totally out of scope
for this. The 'AtomBIOS' of the ATI cards, consists of multiple parts (I
may have minor errors here, ask airlied if you need more
clarification):
a) Initialization code (I think arch-specific)
b) AtomBIOS script interpreter (I think arch-specific)
c) AtomBIOS scripts (card-specific, but not arch-specific)
d) Data tables (card-specific, but not arch-specific)

The AtomBIOS parts of the various drivers need data from c+d primarily,
and provide their own script interpreter, or not using the scripts, just
the data tables.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: Access to PCI Expansion ROMs on PPC

[Michel Dänzer]

On Mon, 2007-11-26 at 00:59 -0800, Robin H. Johnson wrote:
> 
> Regarding the sub-thread on x86 emulation, that is totally out of scope
> for this. The 'AtomBIOS' of the ATI cards, consists of multiple parts (I
> may have minor errors here, ask airlied if you need more
> clarification):
> a) Initialization code (I think arch-specific)
> b) AtomBIOS script interpreter (I think arch-specific)
> c) AtomBIOS scripts (card-specific, but not arch-specific)
> d) Data tables (card-specific, but not arch-specific)
> 
> The AtomBIOS parts of the various drivers need data from c+d primarily,
> and provide their own script interpreter, or not using the scripts, just
> the data tables.

Are you sure the X1900 Mac Edition ROM contains any ATOM data structures
though? I rather doubt it.


-- 
Earthling Michel Dänzer           |          http://tungstengraphics.com
Libre software enthusiast         |          Debian, X and DRI developer

Re: Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1749 bytes --]

On Mon, Nov 26, 2007 at 10:20:14AM +0100, Michel D?nzer wrote:
> > Regarding the sub-thread on x86 emulation, that is totally out of scope
> > for this. The 'AtomBIOS' of the ATI cards, consists of multiple parts (I
> > may have minor errors here, ask airlied if you need more
> > clarification):
> > a) Initialization code (I think arch-specific)
> > b) AtomBIOS script interpreter (I think arch-specific)
> > c) AtomBIOS scripts (card-specific, but not arch-specific)
> > d) Data tables (card-specific, but not arch-specific)
> > 
> > The AtomBIOS parts of the various drivers need data from c+d primarily,
> > and provide their own script interpreter, or not using the scripts, just
> > the data tables.
> Are you sure the X1900 Mac Edition ROM contains any ATOM data structures
> though? I rather doubt it.
That's what I'm trying to ascertain actually, because it directly
impacts what work needs to be done in the drivers.

If it was me writing the ROMs in the first place (but it's not), from
what I understand of the ATOM, I'd have OpenFirmware versions of a+b,
and then could re-use c+d.

The fact that both Nvidia and ATI OF-powered cards fail when trying to
access their expansion ROMs, I'm inclined to think there is something in
accessing them that we are missing, and that the ROMs must exist (the
sata_mv one existed and was accessible).

On actually accessing the rest of the ATI X1900 card, the avivotool from
git://people.freedesktop.org/~airlied/radeontool.git (avivo branch) can
actually access the rest of the rest of the card registers.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: Access to PCI Expansion ROMs on PPC

[Jon Smirl]

On 11/26/07, Robin H. Johnson <robbat2@gentoo.org> wrote:
> On Mon, Nov 26, 2007 at 10:20:14AM +0100, Michel D?nzer wrote:
> > > Regarding the sub-thread on x86 emulation, that is totally out of scope
> > > for this. The 'AtomBIOS' of the ATI cards, consists of multiple parts (I
> > > may have minor errors here, ask airlied if you need more
> > > clarification):
> > > a) Initialization code (I think arch-specific)
> > > b) AtomBIOS script interpreter (I think arch-specific)
> > > c) AtomBIOS scripts (card-specific, but not arch-specific)
> > > d) Data tables (card-specific, but not arch-specific)
> > >
> > > The AtomBIOS parts of the various drivers need data from c+d primarily,
> > > and provide their own script interpreter, or not using the scripts, just
> > > the data tables.
> > Are you sure the X1900 Mac Edition ROM contains any ATOM data structures
> > though? I rather doubt it.
> That's what I'm trying to ascertain actually, because it directly
> impacts what work needs to be done in the drivers.
>
> If it was me writing the ROMs in the first place (but it's not), from
> what I understand of the ATOM, I'd have OpenFirmware versions of a+b,
> and then could re-use c+d.
>
> The fact that both Nvidia and ATI OF-powered cards fail when trying to
> access their expansion ROMs, I'm inclined to think there is something in
> accessing them that we are missing, and that the ROMs must exist (the
> sata_mv one existed and was accessible).

There may well be bugs in the ROM access code on the PowerPC. I don't
own the appropriate hardware nedded to check it out. I'm not sure that
the code has been used on the PowerPC very much. People use it on the
x86 all of the time so it is working fairly well there. But the
generic PowerPC code seems to be working since you can read the
sata_mv ROM.

Did you check out the cards on x86 and ascertain that they have the
standard PCI header in them? 55 AA .... All PCI ROMs are supposed to
have that. If they are missing that the size code in the rom.c isn't
going to work right.

If these are OF ROM and you are booting on OF firmware, the ROM is
getting run. In that case it may not be so simple to turn them back on
if they have been hidden using a proprietary register. That's the
quirk BenH has referred to.


>
> On actually accessing the rest of the ATI X1900 card, the avivotool from
> git://people.freedesktop.org/~airlied/radeontool.git (avivo branch) can
> actually access the rest of the rest of the card registers.
>
> --
> Robin Hugh Johnson
> Gentoo Linux Developer & Infra Guy
> E-Mail     : robbat2@gentoo.org
> GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
>
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
>
>


-- 
Jon Smirl
jonsmirl@gmail.com

Re: Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

On Mon, Nov 26, 2007 at 11:33:00AM -0500, Jon Smirl wrote:
> Did you check out the cards on x86 and ascertain that they have the
> standard PCI header in them? 55 AA .... All PCI ROMs are supposed to
> have that. If they are missing that the size code in the rom.c isn't
> going to work right.
See my more recent testing summary post to the list, with the message-id
of 20071126085928.GE14557@curie-int.orbis-terrarum.net
(http://ozlabs.org/pipermail/linuxppc-dev/2007-November/046880.html)
The X1900 and Nvidia OF-based cards do not turn up on the x86_64 box at
all (the PCIe root node is entirely missing with them as well), and the
sata_mv claims to not have a ROM, but does otherwise work.

> If these are OF ROM and you are booting on OF firmware, the ROM is
> getting run. In that case it may not be so simple to turn them back on
> if they have been hidden using a proprietary register. That's the
> quirk BenH has referred to.
That's the path that I was investigating with the register stuff via
airlied's avivotool. There was one register he thought about, but it
didn't seem to do much.

I also found some instructions to try and view the ROMs from inside OF,
some I'm going to try that later today, as that will enable seeing if
Linux is changing something critical.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: Access to PCI Expansion ROMs on PPC

[Benjamin Herrenschmidt]

On Sun, 2007-11-25 at 08:30 -0500, Jon Smirl wrote:
> 
> > The two cards with x86 firmware return 0xFF for those two readb()
> > instructions, while the X1900 with OF returns 0x00 for the readb().
> >
> > Could one of the more knowledgeable folk for PPC intricacies suggest
> why
> > those readb calls are returning the wrong data?
> 
> I don't know PPC at this low of level but it may be a problem with non
> word-aligned access to memory. I thought readb() was supposed to work
> on all archs and alignment issues are handled inside readb(). Also,
> the readw() may have an endian bug.

Ugh ? Read be reads -bytes- ! Can you tell me how the hell can a byte
access be non-aligned ?


> BenH, has source code for an x86 emulator that will run on PPC. That
> will let you run the ROMs. The original plan was for the kernel to
> generate a uevent that would have triggered the x86 emulator to run.
> 
> Progress along that path was blocked by the X developers. The X server
> contains code for enabling the PCI ROM and reading it from user space.
> I wanted to move this code out of X and into the kernel.
> 
> Because the path was blocked things like the PCI ROM API were never
> throughly tested. It works most of the time but the occasional problem
> is still turning up. Once we identify the PPC problem we can fix it in
> the kernel.

Nobody blocked anything, you are free to develop an emulator triggered
by a uevent, nobody prevented you from doing so.

Now, regarding that user problem, this it totally unrelated as it's a
"mac" card.

It's possible that it's one of these radeons that disable ROM access via
a register in which case a quirk is needed to re-enable it.

Ben.

Re: Access to PCI Expansion ROMs on PPC

[Jon Smirl]

On 11/25/07, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
>
> On Sun, 2007-11-25 at 08:30 -0500, Jon Smirl wrote:
> >
> > > The two cards with x86 firmware return 0xFF for those two readb()
> > > instructions, while the X1900 with OF returns 0x00 for the readb().
> > >
> > > Could one of the more knowledgeable folk for PPC intricacies suggest
> > why
> > > those readb calls are returning the wrong data?
> >
> > I don't know PPC at this low of level but it may be a problem with non
> > word-aligned access to memory. I thought readb() was supposed to work
> > on all archs and alignment issues are handled inside readb(). Also,
> > the readw() may have an endian bug.
>
> Ugh ? Read be reads -bytes- ! Can you tell me how the hell can a byte
> access be non-aligned ?

readb() doesn't work on the dev board I'm currently working with
because the flash is wired in such a way that it only works for 16b
reads. Maybe calling that an alignment issue is the wrong term.

> > BenH, has source code for an x86 emulator that will run on PPC. That
> > will let you run the ROMs. The original plan was for the kernel to
> > generate a uevent that would have triggered the x86 emulator to run.
> >
> > Progress along that path was blocked by the X developers. The X server
> > contains code for enabling the PCI ROM and reading it from user space.
> > I wanted to move this code out of X and into the kernel.
> >
> > Because the path was blocked things like the PCI ROM API were never
> > throughly tested. It works most of the time but the occasional problem
> > is still turning up. Once we identify the PPC problem we can fix it in
> > the kernel.
>
> Nobody blocked anything, you are free to develop an emulator triggered
> by a uevent, nobody prevented you from doing so.

They blocked the X server changes, which rendered the kernel support
pointless. But that's all old news.

> Now, regarding that user problem, this it totally unrelated as it's a
> "mac" card.
>
> It's possible that it's one of these radeons that disable ROM access via
> a register in which case a quirk is needed to re-enable it.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Access to PCI Expansion ROMs on PPC

[Benjamin Herrenschmidt]

On Sun, 2007-11-25 at 15:51 -0500, Jon Smirl wrote:
> 
> They blocked the X server changes, which rendered the kernel support
> pointless. But that's all old news.

No, you are wrong, nothing was ever "blocked".

Ben.

Re: Access to PCI Expansion ROMs on PPC

[Jon Smirl]

On 11/25/07, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
> Now, regarding that user problem, this it totally unrelated as it's a
> "mac" card.
>
> It's possible that it's one of these radeons that disable ROM access via
> a register in which case a quirk is needed to re-enable it.

He says in the first email that when he puts the boards into an x86
box he can read them. The quick turns the ROM back on if the firmware
is turning them off.

In the PPC box the ROMs aren't being run so they would still be in
power on state. I'd expect them to be readable in power-on state.

You can dump the PCI config block in /sys with hexdump and see if the
PCI_COMMAND_MEMORY bit is set. I was also unable to locate where in
the kernel PCI_COMMAND_MEMORY is being set. It could be that it is set
by the BIOS at boot on the x86 and not on the ppc.

-- 
Jon Smirl
jonsmirl@gmail.com

Re: Access to PCI Expansion ROMs on PPC

[Benjamin Herrenschmidt]

> You can dump the PCI config block in /sys with hexdump and see if the
> PCI_COMMAND_MEMORY bit is set. I was also unable to locate where in
> the kernel PCI_COMMAND_MEMORY is being set. It could be that it is set
> by the BIOS at boot on the x86 and not on the ppc.

The kernel doesn't until somebody calls pci_enable_device(). So yes,
that's likely to be your problem.

Ben.

Re: Access to PCI Expansion ROMs on PPC

[Andreas Schwab]

"Robin H. Johnson" <robbat2@gentoo.org> writes:

> I put the card into an amd64 box, found the relevant 'rom' node ($ROM) under
> /sys/device/pci*, and dumped it as follows:
> # echo 1>$ROM

Did you run it exactly like this?  Because this will echo exactly one
newline to $ROM.  If you want to echo "1\n" you need to put a space
between "1" and ">".

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Access to PCI Expansion ROMs on PPC

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

On Sun, Nov 25, 2007 at 10:42:42AM +0100, Andreas Schwab wrote:
> "Robin H. Johnson" <robbat2@gentoo.org> writes:
> 
> > I put the card into an amd64 box, found the relevant 'rom' node ($ROM) under
> > /sys/device/pci*, and dumped it as follows:
> > # echo 1>$ROM
> Did you run it exactly like this?  Because this will echo exactly one
> newline to $ROM.  If you want to echo "1\n" you need to put a space
> between "1" and ">".
I did run the correct version, "echo 1 >$ROM", so that a "1\n" was
actually sent, and not a newline without a digit.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]