# Modular System Support - PCH IO Expander

Author: Chen, Yugang

## Problem Description
When multi modular 2 socket reference platforms (RP) are connected together,
the BMCs on each RP shall support the ability to manage the system as either
single 4S/8S system or standalone as 2S systems.

## Background and References
In 4S/8S mode, only one PCH works at DMI (Direct Media Interface) mode,
this PCH is called legacy PCH (PCH.L), the other PCHs are called PCH IO Expander
(PCH.IO). The BMC attached to the PCH.L works as a primary and the other BMCs
work as secondaries.

## Requirements
Check a modular system as either 4S/8S or standalone mode by GPIOs on BMC each
time system AC on.
Each BMC works independently in standalone mode.
In 4S/8S mode, only primary BMC is responsible for management of system level
resources, secondary BMCs need to collect its local management resources and report
them to primary BMC.

## Proposed Design
Each BMC will have a property to reveal its work mode. Users can know if BMC is in standalone mode or 4S/8S mode.
Each BMC has another property to show if the BMC mode is PCH Legacy BMC or
Non-Legacy BMC. PCH Legacy BMC is working as primary and Non-Legacy BMC working
as secondary.

Each time a BMC reboot, The BMC needs to check 3 GPIO Pins: FM_STANDALONE_MONDE_N / FM_4S_8S_N_MODE / FM_NODE_ID to get working mode as standalone mode or primary /secondary(4S/8S) role according to the GPIO values.
After confirming the mode and BMCs' role, BMCs should set properties according
to the correct configuration.
In 4S/8S mode only node id 0 will be primary BMC because only this node will be the PCH.L. Node id 1,2,3 will be the secondary nodes.

Once a BMC gets mode is in 4S/8S, node roles are configured by node
ID (GPIO Pins) and keep consistent once AC on. Once node role check is done,

In 4S/8S mode, Primary BMC needs to broadcast its role to make sure there is only
one primary BMC in the system.

Need a new feature to make sure secondary BMCs send local redfish events to primary
BMC. And primary BMC needs to add a tag to those events coming from secondary BMC so
that user can know the event logs happened on which node.

Even in 4S/8S mode, each BMC will collect its local management resources,
including sensors, fans and do FSC according to the values of local sensors.
PSU and Fans on each node will not be connected together. Configuration
settings of each secondary node will remain the same, and won't be synced across
the nodes.

In 4S/8S mode, PECI will only be connected to primary node. Primary BMC needs to
monitor all CPU and DIMM sensors and deliver the sensor values of the CPUs/DIMMs
on secondary nodes to secondary BMCs. So that secondary BMCs can use the sensor values to control their own FSCs. Primary BMC also needs to have a way to find how many
CPUs are in the whole system include Primary and Secondary nodes.

## Alternatives Considered
Primary node monitors all the IPMI sensors in secondary nodes and creates redfish log
by itself.

Instead of BMC reboot, only AC cycle will make BMCs check GPIO pins and set
Legacy BMC or Non-Legacy BMC mode.

Only primary BMC broadcast its role and secondary only waiting for the broadcast
from primary.

## Impacts
Only on the motherboard where legacy PCH is located, POST code/Front Panel/KCS
port/UART will work, while these interfaces on board with non-legacy PCH will
not work due to BIOS and HW design. And this will cause non-functioning of
SOL/KVM/Virtual media on secondary BMCs.