[parisc-linux] iotree thoughts

[parisc-linux] iotree thoughts

[Matthew Wilcox]

This is a summary and a design document for building an iotree in PA/Linux.
Grant, Ryan, if I mis-remembered / misunderstood something, do shout.

There are two sources of information about what devices are in the
machine.  There's an Architected Bus Walk, which basically says `probe
devices at the following addresses'.  Code to do this already exists
in do_native_bus_walk.  The other way of finding devices is to ask PDC.
All non-architected devices (ie devices which won't be discovered by an
ABW) are known to PDC.

PDC also knows about other devices, including some PCI devices and some
of the architected devices, so some devices may get discovered twice.
Also, PDC occasionally has bugs (see the J2240 Dino and 715/Scorpio
card-mode Dino entries for examples).  And there are three types of PDC
to deal with -- PAT PDC on some 64-bit boxes, PDC which supports the
PDC_SYSTEM_MAP call, and PDC which predates the PDC_SYSTEM_MAP call.

In a related matter, the ABW will pick up serial ports (and other things)
which aren't activated.  Grant said there was a PDC call to determine
whether a device is inactive -- I'm not sure which one that is yet.

Here's my plan:

 * From setup_arch, call PDC to get a list of devices.  Put them in a
   table (marked __init, will be freed later).
 * In the badly-named gsc_init, or perhaps in pcibios_init, start the
   ABW at 0xFFF80000 -- this seems to be the defined location for the
   central bus.
 * Start creating the tree with devices that we find in the ABW.
   Each device which is a bus adapter / converter will do an ABW for its
   children, and add in any PDC-discovered devices in the table which are
   its immediate children.  The add-in is done by copy, not by reference.
 * At some indeterminate time after this, the table of PDC-reported drivers
   gets freed along with all the other __init data & code.

We'll have to be careful with CPU `drivers' since they will have stale
pointers which must be updated.  Perhaps this is where a ->remove method
would come in handy; or perhaps we just magically make them cope with
their ->probe function being called a second time on the same device
(same HPA).

Feedback welcome.

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] iotree thoughts

[Grant Grundler]

Matthew Wilcox wrote:
> 
> This is a summary and a design document for building an iotree in PA/Linux.
> Grant, Ryan, if I mis-remembered / misunderstood something, do shout.

Cool!

...
> In a related matter, the ABW will pick up serial ports (and other things)
> which aren't activated.  Grant said there was a PDC call to determine
> whether a device is inactive -- I'm not sure which one that is yet.

Yes. AFAIK, LASI sub-devices are listed by PDC. For Dino HPUX makes
a special hvers PDC call. Need to look at the HPUX code or Raven PDC ERS.

> 
> Here's my plan:
> 
>  * From setup_arch, call PDC to get a list of devices.  Put them in a
>    table (marked __init, will be freed later).

Why not start building a tree with "place holder" nodes directly?
Do we need the intermediate table because of memory issues?

ie build a "sparse" tree and have Native Bus walk fill in (or update
wrong) info for devices it finds.

>  * In the badly-named gsc_init, or perhaps in pcibios_init, start the
>    ABW at 0xFFF80000 -- this seems to be the defined location for the
>    central bus.

My long term goal is to be able to build a kernel w/o CONFIG_PCI defined.
And renaming gsc_init to native_bus_init() sounds like a good idea.

>  * Start creating the tree with devices that we find in the ABW.
>    Each device which is a bus adapter / converter will do an ABW for its
>    children, and add in any PDC-discovered devices in the table which are
>    its immediate children.  The add-in is done by copy, not by reference.
>  * At some indeterminate time after this, the table of PDC-reported drivers
>    gets freed along with all the other __init data & code.
> 
> We'll have to be careful with CPU `drivers' since they will have stale
> pointers which must be updated.  Perhaps this is where a ->remove method
> would come in handy; or perhaps we just magically make them cope with
> their ->probe function being called a second time on the same device
> (same HPA).

The latter sounds easier but more like a hack...maybe it's ok though.

thanks,
grant

Grant Grundler
parisc-linux {PCI|IOMMU|SMP} hacker
+1.408.447.7253

Re: [parisc-linux] iotree thoughts

[Matthew Wilcox]

On Thu, Aug 09, 2001 at 09:06:43AM -0600, Grant Grundler wrote:
> Yes. AFAIK, LASI sub-devices are listed by PDC. For Dino HPUX makes
> a special hvers PDC call. Need to look at the HPUX code or Raven PDC ERS.

If you could dig those out for me, that'd be great.  I'm not familiar with
the layout of the HPUX source tree yet.

> > 
> > Here's my plan:
> > 
> >  * From setup_arch, call PDC to get a list of devices.  Put them in a
> >    table (marked __init, will be freed later).
> 
> Why not start building a tree with "place holder" nodes directly?
> Do we need the intermediate table because of memory issues?
> 
> ie build a "sparse" tree and have Native Bus walk fill in (or update
> wrong) info for devices it finds.

Two reasons: first, kmalloc isn't initialised at this point.  Changing
this would be moderately hard and would require a large reorganisation
of the initialisation code.  Secondly, building a sparse tree would take
more code :-)

> >  * In the badly-named gsc_init, or perhaps in pcibios_init, start the
> >    ABW at 0xFFF80000 -- this seems to be the defined location for the
> >    central bus.
> 
> My long term goal is to be able to build a kernel w/o CONFIG_PCI defined.
> And renaming gsc_init to native_bus_init() sounds like a good idea.

That's a good point.  A good step towards that would be..

Index: include/asm-parisc/io.h
===================================================================
RCS file: /home/cvs/parisc/linux/include/asm-parisc/io.h,v
retrieving revision 1.22
diff -u -p -r1.22 io.h
--- io.h        2001/07/15 22:30:29     1.22
+++ io.h        2001/08/14 15:59:37
@@ -43,9 +43,6 @@
 #endif /* USE_HPPA_IOREMAP */
 
 #if defined(CONFIG_PCI) || defined(CONFIG_ISA)
-/*
- *     So we get clear link errors 
- */
 extern unsigned char inb(unsigned long addr);
 extern unsigned short inw(unsigned long addr);
 extern unsigned int inl(unsigned long addr);
@@ -54,6 +51,13 @@ extern void outb(unsigned char b, unsign
 extern void outw(unsigned short b, unsigned long addr);
 extern void outl(unsigned int b, unsigned long addr);
 
+#else
+#define inb(a) BUG()
+#define inw(a) BUG()
+#define inl(a) BUG()
+#define outb(a) BUG()
+#define outw(a) BUG()
+#define outl(a) BUG()
 #endif
 
 extern void memcpy_fromio(void *dest, unsigned long src, int count);


-- 
Revolutions do not require corporate support.

Re: [parisc-linux] iotree thoughts

[Grant Grundler]

Matthew Wilcox wrote:
> On Thu, Aug 09, 2001 at 09:06:43AM -0600, Grant Grundler wrote:
> > Yes. AFAIK, LASI sub-devices are listed by PDC. For Dino HPUX makes
> > a special hvers PDC call. Need to look at the HPUX code or Raven PDC ERS.
> 
> If you could dig those out for me, that'd be great.  I'm not familiar with
> the layout of the HPUX source tree yet.

That sounds too much like work. ;^)
Raven PDC ERS is available from HP's firmware team website. (internal only).

> > My long term goal is to be able to build a kernel w/o CONFIG_PCI defined.
> > And renaming gsc_init to native_bus_init() sounds like a good idea.
> 
> That's a good point.  A good step towards that would be..

I disagree. I want the kernel to NOT link if CONFIG_PCI and
CONFIG_ISA are not enabled when inx/outx functions are used.

> +#else
> +#define inb(a) BUG()
> +#define inw(a) BUG()
> +#define inl(a) BUG()
> +#define outb(a) BUG()
> +#define outw(a) BUG()
> +#define outl(a) BUG()
>  #endif

This will allow the kernel to link and then fail after rebooting.
That seems like a step backwards to me.

grant

Grant Grundler
parisc-linux {PCI|IOMMU|SMP} hacker
+1.408.447.7253

Re: [parisc-linux] iotree thoughts

[Matthew Wilcox]

On Mon, Aug 20, 2001 at 11:44:20PM -0600, Grant Grundler wrote:
> That sounds too much like work. ;^)
> Raven PDC ERS is available from HP's firmware team website. (internal only).

Thanks, I'll look at that.

> I disagree. I want the kernel to NOT link if CONFIG_PCI and
> CONFIG_ISA are not enabled when inx/outx functions are used.

I discussed this with Ted Ts'o at OLS.  The difficulty is that there's
a fair bit of arch-independent code which does (example taken from serial.c):

static _INLINE_ unsigned int serial_in(struct async_struct *info, int offset)
{
        switch (info->io_type) {
        case SERIAL_IO_MEM:
                return readb((unsigned long) info->iomem_base +
                             (offset<<info->iomem_reg_shift));
        default:
                return inb(info->port + offset);
        }
}

so either we add the nasty:

        default:
#if defined(CONFIG_PCI) || defined(CONFIG_ISA)
                return inb(info->port + offset);
#else
                BUG();
#endif

or we just make inb() BUG().

> This will allow the kernel to link and then fail after rebooting.
> That seems like a step backwards to me.

Only if they're called.  The majority of places where these functions
exist can provably never be called.  The others probably won't be :-)

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] iotree thoughts

[Grant Grundler]

Matthew Wilcox wrote:
> I discussed this with Ted Ts'o at OLS.  The difficulty is that there's
> a fair bit of arch-independent code which does (example taken from serial.c):
> 
> static _INLINE_ unsigned int serial_in(struct async_struct *info, int offset)
....
> so either we add the nasty:
> 
>         default:
> #if defined(CONFIG_PCI) || defined(CONFIG_ISA)
>                 return inb(info->port + offset);
> #else
>                 BUG();
> #endif

The ifdef is only once in that source file.

> or we just make inb() BUG().

The problem is serial.c is written for x86 arch and wasn't ported
cleanly for other arches. Makeing inb() BUG() allows sloppy
porting.


> Only if they're called.  The majority of places where these functions
> exist can provably never be called.  The others probably won't be :-)

I don't bet on "probably".

If the kernel is designed to not link when a dependency is not met,
I can find problems much faster when they arise. Having to build/reboot
to find porting/coding issues seems harder than it needs to be.

grant

Grant Grundler
parisc-linux {PCI|IOMMU|SMP} hacker
+1.408.447.7253

Re: [parisc-linux] iotree thoughts

[Alan Cox]

> or we just make inb() BUG().

Its probably more useful to make it print the port numher and traceback then
continue. That way any minor escapees will get back some cookie you pick
(eg 0xFF) and you can chase them out easily

Re: [parisc-linux] iotree thoughts

[Helge Deller]

On Tuesday 21 August 2001 09:53, Grant Grundler wrote:
> Matthew Wilcox wrote:
> > Only if they're called.  The majority of places where these functions
> > exist can provably never be called.  The others probably won't be :-)
>
> I don't bet on "probably".
>
> If the kernel is designed to not link when a dependency is not met,
> I can find problems much faster when they arise. Having to build/reboot
> to find porting/coding issues seems harder than it needs to be.

Hi Grant & Matthew,

I still think Matthew's solution is a good idea. Why should the user be
bothered to compile in that PCI stuff if he won't need it at all ?
I tried Matthew's patch and it worked without problems on my 715/64 
and this is exactlly one of those machines for which the patch was 
designed. Those machines use LASI-style LAN, PS/2 or HIL Keyboard/Mice, 
STI(fb) and the harmony sound driver. Those are well-defined drivers which we
know that they work without PCI and which other drivers should we need ? 
GSC-cards ? They should work since I think they only have additional LASI's 
or somethink like that (You may correct me if I'm wrong.).
EISA-Cards ? We don't support any yet. Maybe in the future, but then the user
will also need to add PCI support anyway.
So I don't see any reason, why we shouldn't make it possible to link
a kernel without PCI support and call BUG() (and print out the port-number as 
Alan suggested) just in case we would miss one place.
Btw, I love a small kernel if I'm running it on such a slow machine which is often 
only equipped with 32-64 MB RAM.

Just my 0.02 cents,

Helge

Re: [parisc-linux] iotree thoughts

[Matthew Wilcox]

On Mon, Aug 20, 2001 at 11:44:20PM -0600, Grant Grundler wrote:
> Matthew Wilcox wrote:
> > On Thu, Aug 09, 2001 at 09:06:43AM -0600, Grant Grundler wrote:
> > > Yes. AFAIK, LASI sub-devices are listed by PDC. For Dino HPUX makes
> > > a special hvers PDC call. Need to look at the HPUX code or Raven PDC ERS.
> > 
> > If you could dig those out for me, that'd be great.  I'm not familiar with
> > the layout of the HPUX source tree yet.
> 
> That sounds too much like work. ;^)
> Raven PDC ERS is available from HP's firmware team website. (internal only).

I looked through the Raven ERS, it wasn't too helpful.  I managed to find
the Dino driver in the HPUX source tree.... it doesn't seem to make the
call itself, but instead:

    /*
     * Loop thru the fixed_mod_table looking for a fixed module
     * that is a child of Dino.  Start at whatever index we just
     * figured out from the hardware path of the last module found.
     */
    for(; idx < nfixed_modules; idx++) {
        entry = &fixed_mod_table[idx];
        
        if (is_child_of(entry, parent_path))
        {
#ifdef RDB
            /*
             * Ignore the serial port if RDB is using it.
             */
            if ((entry != NULL) && (!(IS_RDB_DEVICE(entry->reg_set))))
#endif
                    return entry;
        }
        /* else not a child of dino - loop again */
    }

    /*
     * If we exit the loop without returning anything, we must not
     * have found the rs232 entry in the FMT.  The rs232 is not to
     * be used in this system, so return NULL
     */
    return NULL;

So it seems to me that we shouldn't do bus-walks underneath Dino, but
rely on PDC telling us about the device if it's attached.

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] iotree thoughts

[Grant Grundler]

Helge Deller wrote:
> Why should the user be
> bothered to compile in that PCI stuff if he won't need it at all ?

That isn't an issue - we all agree CONFIG_PCI isn't needed on older machines.

> Those are well-defined drivers which we
> know that they work without PCI and which other drivers should we need ? 

This is a good point. I'll shup up. :^)

thanks,
grant

Grant Grundler
parisc-linux {PCI|IOMMU|SMP} hacker
+1.408.447.7253

Re: [parisc-linux] iotree thoughts

[Grant Grundler]

Matthew Wilcox wrote:
> I looked through the Raven ERS, it wasn't too helpful.  I managed to find
> the Dino driver in the HPUX source tree.... it doesn't seem to make the
> call itself, but instead:

[...]


I was confused. The Dino built-in discovery doesn't make PDC calls.

HPUX 11.01 (9905 release?) does make PDC calls in the boot initialization
sequence.  IIRC, This associates PCI device numbers with slot numbers
printed on the back of the machine. Useful for defective FRU ("Field
Replacable Unit") isolation/error reporting.  AFAIK, This only works
for platforms with bridge-mode Dino - ie B1XX/CXXX workstations.
(Calls are appended for interested folks).

...
> So it seems to me that we shouldn't do bus-walks underneath Dino, but
> rely on PDC telling us about the device if it's attached.

Uhm, "underneath" is ambiguous since both built-ins and PCI devices
are heirarchially below "dino" (iotree view - not HW view).
We need to do PCI bus walks for PCI devices.

I'll assume your statement only refers to Dino built-ins and agree.
FWIW, same is true for LASI devices.

grant


Grant Grundler
parisc-linux {PCI|IOMMU|SMP} hacker
+1.408.447.7253

    /*
     * Get physical platform information
     */
    status = hversion_pdc_call(
                 3,(u_int)PDC_ARG2_IS_RADDR, 10, (ulong_t)PDC_PCI,
                 (ulong_t)PDC_PCI_GET_IFACE_INFO, (ulong_t)&(DINO(this)->ifinfo)
,
                 (ulong_t)this->base_addr,(ulong_t)0,(ulong_t)0,(ulong_t)0,
                 (ulong_t)0,(ulong_t)0,(ulong_t)0);
...
    /*
     * Total number of devices to get "slot info" for - all the slots plus
     *   all the built-ins
     */
    num_devs = DINO(this)->ifinfo.num_slots + DINO(this)->ifinfo.num_built_ins;
...
    for (slot_idx = 0; slot_idx < num_devs; slot_idx++) {
        ulong_t slot_path[32];
        status = hversion_pdc_call(
                     4, (u_int)PDC_ARG2_IS_RADDR, 32, (ulong_t)PDC_PCI,
                     (ulong_t)PDC_PCI_GET_SLOT_INFO, (ulong_t) slot_path,
                     (ulong_t)this->base_addr, (ulong_t)slot_idx,(ulong_t)0,
                     (ulong_t)0, (ulong_t)0, (ulong_t)0, (ulong_t)0);
...

Re: [parisc-linux] iotree thoughts

[Matthew Wilcox]

On Fri, Aug 24, 2001 at 11:59:25PM -0600, Grant Grundler wrote:
> I was confused. The Dino built-in discovery doesn't make PDC calls.

OK, that clears that up :-)

> > So it seems to me that we shouldn't do bus-walks underneath Dino, but
> > rely on PDC telling us about the device if it's attached.
> 
> Uhm, "underneath" is ambiguous since both built-ins and PCI devices
> are heirarchially below "dino" (iotree view - not HW view).
> We need to do PCI bus walks for PCI devices.

I mean native buswalks.  PCI buswalks are outside the scope of this topic.

> I'll assume your statement only refers to Dino built-ins and agree.
> FWIW, same is true for LASI devices.

Hmm.  OK.

-- 
Revolutions do not require corporate support.