[PATCH 0/6] ACPI based physical cpu hotplug

[PATCH 0/6] ACPI based physical cpu hotplug

[Keshavamurthy Anil S]

Hi All,
	As everybody knows that we have logical cpu online/offline support in the kernel, the following set of patches extends this feature to provide ACPI based physical CPU hotplug and I am asking this set of patches to be included in the acpi-test tree. 

Please let me know if anyone see any issues or comments or even flames welcome:). 

Complete patches which applies cleanly on to 2.6.8.1-rc1-mm2 follows this mail.

Context:
--------
This set of patches supports physical CPU hotplug notification happening directly on ACPI_PROCESSOR_HID in which case the notification is services within the processor driver or  if the processor is described within the ACPI container object(i.e ACPI004 or PNP0A05 or PNP0A06) and hotplug notifications happens on the Container object then the container driver(which is a new additional driver, patch 6/6) will service this request.

The interaction between the container driver and the processor driver is implicit i.e when the container driver is notified about the hotadd as a result of notification happening on ACPI container object(i.e ACPI004 or PNP0A05 or PNP0A06), the container driver will call acpi_bus_scan() api which will add the individual devices within its namespace and the respective driver's .add routine will be called in this case acpi_processor_add() routine gets called which will handle the setting up of new processor.

For Hotadd case, the kernel mode just initializes the minimal data structures (like mapping between acpiid<->apicid<->logical_cpu_number) and populates the sysfs entries(/sys/devices/system/cpu/cpuX/online) and issues an /sbin/hotplug notification to user mode agent script at which point the hotadded CPU will be in logical offline state. User mode agent script will turn the CPU online. This is designed as kernel/usermode (i.e setting up things in kernel and actual onlining from usermode agent) because at some point in future for the Container device which contains both CPU and Memory then there needs to some order in which you can bring up the child devices of the container(i.e all memory devices first and then the cpu devices). Also if it is in user mode, the script can implement the policy whether to continue onlineing of the devices if one fails etc.

For Hotremove case, the kernel mode again just sends the notification to user land and the agent script is responsible for offlining the devices and then calling "echo "\_SB_.LSB0" > /sys/firmware/acpi/eject". This eject is a new interface which has been designed to handle this hotremove. With this kind of an interface user mode initiated hotremoval is also possible if that is required for some platform. Again in this user mode initiated hotremoval the agent script can offline the devices and then can echo the acpi_handle name onto the eject file.

Some of the contribution to this patches has come from multiple people from with in Intel and also from Fujitsu's who are also involved in this work and Fujitsu has also hosted lhns.sourceforge.net opensource project specifically to address hotplug work and I am cc'ing lhns mail-list from where I borrowed the initial design concepts.
Thanks to all for those who participated in bringing the code to this quality.

All the testing has been done in an emulation environment and for more information on setting up the emulation environment for the hotplug testing please visit lhns.sourceforge.net.

TBD - Works that needs to be done:
1) Full NUMA based systems
2) Arch specific acpi enhancements for IA32 platforms to support physical CPU hotplug.

Thanks,

-Anil Keshavamurthy
Sr. Software Engineer
Linux OS Technology Team
Intel Corp.
(w) 503-712-4476


-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click

Re: [Lhns-devel] [PATCH 0/6] ACPI based physical cpu hotplug

[Keiichiro Tokunaga]

On Wed, 08 Sep 2004 18:16:54 -0700, Keshavamurthy Anil S wrote:
> 
> Hi All,
> 	As everybody knows that we have logical cpu online/offline support in the kernel, the following set of patches extends this feature to provide ACPI based physical CPU hotplug and I am asking this set of patches to be included in the acpi-test tree. 
> 
> Please let me know if anyone see any issues or comments or even flames welcome:). 

I am attaching user mode agent scripts for container hotplug,
'container.agent' and 'acpi_container.agent'.  Please put them
under /etc/hotplug/ for working the container hotplug.

'container.agent' is called by the container driver.  It calls a platform
specific agent as appropriate.  'acpi_container.agent' is one of the
platform specific agents and does real works.

'acpi_container.agent' does logical hotplug operations like "echo 0 >
/sys/devices/system/cpu/cpu0/online".  The agent gets an ACPI
object name of the target container device (e.g. DEV1) as an
argument, searchs entries of CPUs contained in the container, and
echos to the entries.  This version supports only CPU for now.

Thanks,
Keiichiro Tokunaga

[container.agent]
#!/bin/sh
#
# Container hotplug agent
#
# Kernel params are:
#
#       ACTION=%s [add or remove]
#       CONTAINER=/sys/...
#       PLATFORM=%s [platform name]
#
# This calls platform specific sub-agent
#
# 09-Sep-2004   Initial version, ACPI support
#

logger "Container Hotplug Event:<CONTAINER> ACTION="$ACTION" CONTAINER="$CONTAIN
ER" SEQNUM="$SEQNUM" DEVPATH="$DEVPATH" PLATFORM="$PLATFORM

#
# Sysfs mount point
#
SYSFS_MOUNTPOINT=`mount | grep 'sysfs' | cut -d ' ' -f 3`

#
# Platform specific agent
#
if [ "$PLATFORM" = "ACPI" ]; then
    ARCH_CONTAINER=/etc/hotplug/acpi_container.agent
    TARGET_DIR=$SYSFS_MOUNTPOINT/firmware/acpi/namespace/ACPI/`echo "$CONTAINER"
 | sed 's/^[\]//' | sed 's/_\./\//g'`
else
    # Unsupported platform
    exit
fi

case $ACTION in
    add) $ARCH_CONTAINER -p $TARGET_DIR add;;
    remove) $ARCH_CONTAINER -p $TARGET_DIR remove;;
esac


[acpi-container.agent]
#!/bin/sh
#
# ACPI Container hotplug agent
#
# Kernel params are:
#
#       ACTION=%s [add or remove]
#       CONTAINER=/sys/...
#
# 10-Sep-2004   Initial version, CPU support
#

TMP=tmp$$
CPU_INFO_DIR=/proc/acpi/processor
SYSDEVICES=/sys/devices/system
CPU_HOTPLUG=$SYSDEVICES/cpu/cpu
MEMBLOCKSIZE=$SYSDEVICES/memory/blocksize
FIRM_DIR=/sys/firmware/acpi
EJECT_OP=$FIRM_DIR/eject
alias cpus_list='ls $CPU_INFO_DIR'

function cpu_hot_plug()
{
    cpus_list > $TMP
    for cpu_firm in `find $DEV_PATH -type d -print | grep -w -f $TMP`
      do
      cpuname=`basename $cpu_firm`
      if [ ! -f $CPU_INFO_DIR/$cpuname/info ]
          then
          echo "cannot access $CPU_INFO_DIR/$cpuname/info"
          continue
      fi
      cpuid=`awk '/processor id:/ {print $3}' $CPU_INFO_DIR/$cpuname/info`
      if [ ! -f ${CPU_HOTPLUG}${cpuid}/online ]
          then
          echo "cannot access ${CPU_HOTPLUG}${cpuid}/online"
          continue
      fi
      onlined=`cat ${CPU_HOTPLUG}${cpuid}/online`
      if [ $onlined == "0" ]
          then
          contine;
      fi
      echo $CPU_HOTPLUG_CODE > ${CPU_HOTPLUG}${cpuid}/online
      if [ $? != 0 ]
          then
          echo "cpu$cpuid hotplug failed."
          ERROR=1
      fi
    done
}

function eject_target()
{
    echo $DEV_PATH | sed '1s/\/sys\/firmware\/acpi\/namespace\/ACPI\/_SB\///' |
sed '1s/\//./g'
}

function usage()
{
    echo "Usage : acpi_container.agent (-t target-device-name| -p CONTAINER) (ad
d|remove)"
    exit 2
}


#
# Main routine
#
set -- `getopt p:t: $*`
if [ $? != 0 ]
then
    usage
fi

while [ $1 != -- ]
do
  case $1 in
      -t) shift; DEV_PATH=/sys/firmware/acpi/namespace/ACPI/_SB/$1;;
      -p) shift; DEV_PATH=$1;;
  esac
  shift;
done
shift;
act=$1

ERROR=0

case $act in
    add)
        CPU_HOTPLUG_CODE="1"

        cpu_hot_plug
        ;;

    remove)
        CPU_HOTPLUG_CODE="0"

        cpu_hot_plug

        # eject operation
        if [ $ERROR == 0 ]; then
            target=`eject_target`
            echo $target > $EJECT_OP
            if [ $? != 0 ]; then
                echo "Failed to eject device($target)."
                ERROR=1
            fi
        fi
        ;;
esac

rm -f $TMP > /dev/null 2>&1

exit $ERROR


-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM. 
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php

Re: [Lhns-devel] [PATCH 0/6] ACPI based physical cpu hotplug

[Keiichiro Tokunaga]

On Mon, 13 Sep 2004 02:48:17 +0900 Keiichiro Tokunaga wrote:
> On Wed, 08 Sep 2004 18:16:54 -0700, Keshavamurthy Anil S wrote:
> > 
> > Hi All,
> > 	As everybody knows that we have logical cpu online/offline support in the kernel, the following set of patches extends this feature to provide ACPI based physical CPU hotplug and I am asking this set of patches to be included in the acpi-test tree. 
> > 
> > Please let me know if anyone see any issues or comments or even flames welcome:). 
> 
> I am attaching user mode agent scripts for container hotplug,
> 'container.agent' and 'acpi_container.agent'.  Please put them
> under /etc/hotplug/ for working the container hotplug.
> 
> 'container.agent' is called by the container driver.  It calls a platform
> specific agent as appropriate.  'acpi_container.agent' is one of the
> platform specific agents and does real works.
> 
> 'acpi_container.agent' does logical hotplug operations like "echo 0 >
> /sys/devices/system/cpu/cpu0/online".  The agent gets an ACPI
> object name of the target container device (e.g. DEV1) as an
> argument, searchs entries of CPUs contained in the container, and
> echos to the entries.  This version supports only CPU for now.

I found a bug in 'acpi_container.agent'.  Here is a fixed version.

  - Fixed the bug in eject_target() function.  It did not return a proper
    string.
  - Replaced a term '/sys' with '$SYSFS_MOUNTPOINT'.

Thanks,
Keiichiro Tokunaga

[acpi_container.agent]
#!/bin/sh
#
# ACPI Container hotplug agent
#
# Kernel params are:
#	
#	ACTION=%s [add or remove]
#	CONTAINER=/sys/...
#
# 10-Sep-2004	Initial version, CPU support
#

TMP=tmp$$
SYSFS_MOUNTPOINT=`mount | grep 'sysfs' | cut -d ' ' -f 3`
CPU_INFO_DIR=/proc/acpi/processor
SYSDEVICES=$SYSFS_MOUNTPOINT/devices/system
CPU_HOTPLUG=$SYSDEVICES/cpu/cpu
MEMBLOCKSIZE=$SYSDEVICES/memory/blocksize
FIRM_DIR=$SYSFS_MOUNTPOINT/firmware/acpi
EJECT_OP=$FIRM_DIR/eject
alias cpus_list='ls $CPU_INFO_DIR'

function cpu_hot_plug()
{
    cpus_list > $TMP
    for cpu_firm in `find $DEV_PATH -type d -print | grep -w -f $TMP` 
      do
      cpuname=`basename $cpu_firm`
      if [ ! -f $CPU_INFO_DIR/$cpuname/info ]
	  then
	  echo "cannot access $CPU_INFO_DIR/$cpuname/info"
	  continue
      fi
      cpuid=`awk '/processor id:/ {print $3}' $CPU_INFO_DIR/$cpuname/info`
      if [ ! -f ${CPU_HOTPLUG}${cpuid}/online ]
	  then
	  echo "cannot access ${CPU_HOTPLUG}${cpuid}/online"
	  continue
      fi
      onlined=`cat ${CPU_HOTPLUG}${cpuid}/online`
      if [ $onlined == "0" ]
	  then
	  contine;
      fi
      echo $CPU_HOTPLUG_CODE > ${CPU_HOTPLUG}${cpuid}/online
      if [ $? != 0 ]
	  then
	  echo "cpu$cpuid hotplug failed."
	  ERROR=1
      fi
    done 
}

function eject_target()
{
    TARGET=`echo $DEV_PATH | sed "1s?${SYSFS_MOUNTPOINT}??" | sed '1s/\/firmware\/acpi\/namespace\/ACPI\/_SB\///' | sed '1s/\//\./g'`
    echo \\_SB_.$TARGET
}

function usage()
{
    echo "Usage : acpi_container.agent (-t target-device-name| -p CONTAINER) (add|remove)"
    exit 2
}


#
# Main routine
#
set -- `getopt p:t: $*`
if [ $? != 0 ]
then
    usage
fi

while [ $1 != -- ]
do
  case $1 in
      -t) shift; DEV_PATH=$SYSFS_MOUNTPOINT/firmware/acpi/namespace/ACPI/_SB/$1;;
      -p) shift; DEV_PATH=$1;;
  esac
  shift;
done
test=`eject_target`

shift;
act=$1

ERROR=0

case $act in
    add)
	CPU_HOTPLUG_CODE="1"

	cpu_hot_plug
	;;

    remove)
	CPU_HOTPLUG_CODE="0"

	cpu_hot_plug

	# eject operation
	if [ $ERROR == 0 ]; then
	    target=`eject_target` 
	    echo $target > $EJECT_OP
	    if [ $? != 0 ]; then
		echo "Failed to eject device($target)."
		ERROR=1
	    fi
	fi
	;;
esac

rm -f $TMP > /dev/null 2>&1 

exit $ERROR


-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM. 
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php