Re: [Pcihpd-discuss] Hot Plug I/O Node spec 0.2.1

[Hiroshi Aono <h-aono@ap.jp.nec.com>]

At Sun, 3 Feb 2002 00:19:15 -0800,
Greg KH wrote:

Hello Greg,

> On Mon, Jan 28, 2002 at 10:36:35AM +0900, Hiroshi Aono wrote:
> > Hello hotplug developers,
> > 
> > I'm working on Hot Plug I/O Node work that is on Atlas project.
> > I've been considering the specification of Hot Plug IO node and I've started
> > the development about this.
> > Now I post Hot Plug I/O Node specification I want to do.
> > This isn't fixed yet. If you're interested, please feel free to add your ideas.
> 
> This looks good, a few minor points below.

Thank you for your comments.

> 
> > 3.1. Functions overview
> > 
> > This is the IO node hot plug function block diagram.
> > 
> >  +---------------+                 +-------------+  
> >  |Hotplug        |      +---------/Configuration/
> >  |Application    |      |        / File        /
> >  +---------------+      |       +-------------+
> > ---------|--------------|------------------------------------
> >  +---------------+  +-------------------+                      [kernel]
> >  |  Hotplug      |  |Object Registration|
> >  |  Filesystem   |  |Interface          |         User API
> >  +---------------+  +-------------------+
> 
> What is the "Object Redistration Interface" going to look like?  What is
> needed to be sent from a configuration file to the IO Node resource
> manager?  Doesn't the IO Node resource manager get all of the
> information it needs from the BIOS/ACPI interface?

I expect that this mainly works for debugging.
I think the hardware like this may not work sufficiently at first.
or it may be used for non-ACPI machine.
This configuration file will be stored hardware information.
Of course IO Node resource manager should get all information from
BIOS/ACPI.
I think the IO node driver and the Object Registration
interface between IO node resource manager should have same interface.

> 
> > 3.3. Hotplug Filesystem
> > 
> > We'd like to use the hotplug filesystem. This is for PCI hotplug
> > filesystem and we feel this is most suitable for the generic hotplug
> > interface. The hotplug filesystem will be used for user interfaces.
> > We will improve this to treat a hierarchical structure for describing 
> > IO node objects.
> > 
> > IO node hotplug hierarchical structure will be as follows:
> > 
> >   --+ ionodeXX
> >          + bridgeXX
> >              + slotXX
> >              + slotXX
> >              + pic
> 
> "pic"?  What is this?

"pic" means programmable interrupt controller like an IOSAPIC.

> 
> > 3.4.2. Interface between IO node manager and PCI hotplug function
> > 
> > IO node hotplug uses the /sbin/hotplug script.
> 
> How?  Doesn't a IO node just show up as a PCI device being added to the
> system?

Our IO node doesn't show up as a PCI device.
Does IBM work so?
I can know the hardware structure through ACPI name space in advance.

> 
> > When an IO node is added, a script for IO node will be executed and the
> > PCI hotplug driver will be loaded.
> 
> Doesn't the PCI Hotplug driver have to be loaded to recognize that a IO
> node was inserted into the system?

If so, I think PCI hotplug driver should be loaded in advance.

> 
> > Conversely, when an IO node is removed, a script for IO node will be
> > executed and the PCI hotplug driver will stop all PCI devices under
> > the IO node.
> 
> Will the removal event fire for every PCI slot, or just once for the
> node?  If just once, I now understand why you need this IO node driver :)

Yes, I think it's just once.

> 
> > 3.4.3. Solutions for assignment of bus number, I/O address and
> > interruption
> > 
> > Our 1st solution is to use the resources that the BIOS has allocated
> > when booting. In this case, we expect that the BIOS supports the
> > reservation of IO/Memory space gap and the bus numbers for hotplugging.
> 
> I am not aware of any other method of getting this information.  I
> thought the BIOS _had_ to do this.  Is there another way of doing this?

This means we can specify the IO/Memory space gap and bus numbers on a BIOS
menu.
I think this is same rule for PCI hotplug. Is this right?

> 
> > 3.4.4. IRQ management
> > 
> > (We should discuss, later.)
> > 
> > 3.4.5. Advanced solution
> > 
> > For more advanced solutions, we should consider active configuration of
> > the IO node memory region and bus number region.
> > 
> > For example, when we want to expand region A because of insufficient IO
> > memory area, we should do the following things:
> > 
> >  o Narrow region B.
> >  o Expand region A.
> > 
> >    MEMORY MAP                           MEMORY MAP
> >  +-------------+                      +-------------+
> >  |             |                      |             |
> >  +-------------+--                    +-------------+--
> >  |             |A (IO node 1)         |             |A (IO node 1)
> >  |             |                ->    |             |
> >  +-------------+--                    |             |
> >  |             |B (IO node 2)         +-------------+
> >  |             |                      |             |B' (IO node 2) 
> >  +-------------+--                    +-------------+--
> >  |             |                      |             |
> > 
> >                        Figure 4 Advanced solution
> >                        --------------------------
> > 
> > In addition, when some PCI cards work on IO node B, we should consider
> > the following things:
> > 
> >  o Suspend all of the devices on IO node 1
> >  o Suspend the cards working on IO node 2
> >  o Move card resources to region B'
> >  o Re-program the configuration space for cards on IO node 2
> >  o Resume the cards on IO node 2
> >  o Resume all the devices on IO node 1
> 
> I can't see users tolerating their already working and configured
> devices shutting down, and then requiring to be reconfigured when you
> insert a new node.  Is this an acceptable thing to you?

As a matter of fact, I think this is not realistic now.
So I say this is "Advanced". At first, this is difficult for
hardware implementation.

> 
> > 3.8. State and transition for IO node
> > 
> > This is the outline for the hot plug procedure:
> 
> > 
> > 1 Not Present 
> > | (Insertion)
> > (1.1) FW: rises an interruption (SCI)       (Note: FW: Firmware)
> > |
> > v
> > 2 Present but not communicating
> > (2.1) OS: ACPI driver handles SCI interruption and 
> >           sends message "Inserted IO nodeXX" to the syslog. (for
> > debugging)
> > (2.2) FW: rises an interruption (SCI)
> > |
> > v
> > 3 Communicating
> > (3.1) OS: ACPI driver handles SCI interruption and
> >           sends message "Communicating" to the syslog. (for debugging)
> > | 
> > v
> > 4 Ready to Join
> > (4.1) FW: scans buses. 
> > (4.2) FW: maps IO spaces. 
> > (4.3) OS: scans buses and devices on IO node.
> > (4.4) OS: add IO node resources.
> > (4.5) OS: sets interrupt vectors.
> > (4.6) OS: call /sbin/hotplug with notification of the new devices seen.
> > |
> > v
> > 5 Running
> > | (Removal)
> > (5.1) FW: calls an interruption (SCI) or user request 
> > |
> > v
> > 6 Prepare to disable
> > (6.1) OS: ACPI driver handles SCI interruption.
> > (6.2) OS: ACPI driver or node manager calls IO node event handler
> > functions for pre-removal.
> > (6.3) OS: call remove method of port drivers on IO node and stops the IO
> >           requests.
> > (6.4) OS: changes IO node state to be unavailable.
> > (6.5) OS: executes _PS3, _EJ0 methods. (Power off and Eject)
> > (6.6) OS: executes _STA to verify the node ejected successfully.
> > |
> > v
> > 7 Ready to Remove (Present but not communicating)
> > |(Physical removal)
> > (7.1) User: removes the IO node.
> > (7.1) FW: calls an interruption (SCI)
> > (7.2) OS: ACPI driver handles SCI interruption.
> > (7.3) OS: deletes the IO node resource.
> > |
> > v
> > Go to 1
> 
> IBM's BIOS does not work this way.  Does this matter?  On insertion, it
> only notifies the OS when the card has been powered up and configured
> (skipping steps 2 and 3 above).  If an error occured when configuring
> (bus speed mismatch, etc.) this error is reported.  Then the OS can
> assign physical resources, call /sbin/hotplug with the notification that
> a new device has been seen, and allows the driver to be loaded.

Actually, hardware spec doesn't fix yet, so we can suggest this for our
hardware people.
Personally, I feel that we don't need steps 2 and 3.

> 
> Most of the removal steps are not present too.  The BIOS does most of
> it, and again, only notifies the OS when everything is done.
> 
> Because of this, a number of places where you will be notifing the OS
> user of something, the IBM driver can not.  Do you see this being a
> problem?

I feel those hardware dependency part should be implement as a local IO node
driver.
I think it will be ok if we decide the interface definitely between IO
node driver and IO node resource manager.

Regards,
---
Hiroshi Aono, NEC Solutions
(h-aono@ap.jp.nec.com)

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel