* Re: [PATCH v6 2/3] drivers/vfio: EEH support for VFIO PCI device
From: Gavin Shan @ 2014-05-23 4:37 UTC (permalink / raw)
To: Alex Williamson; +Cc: aik, Gavin Shan, kvm-ppc, agraf, qiudayu, linuxppc-dev
In-Reply-To: <1400814653.3289.428.camel@ul30vt.home>
On Thu, May 22, 2014 at 09:10:53PM -0600, Alex Williamson wrote:
>On Thu, 2014-05-22 at 18:23 +1000, Gavin Shan wrote:
>> The patch adds new IOCTL commands for VFIO PCI device to support
>> EEH functionality for PCI devices, which have been passed through
>> from host to somebody else via VFIO.
>>
>> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
>> ---
>> Documentation/vfio.txt | 88 ++++++++++-
>> arch/powerpc/include/asm/eeh.h | 17 +++
>> arch/powerpc/kernel/eeh.c | 321 +++++++++++++++++++++++++++++++++++++++++
>> drivers/vfio/pci/vfio_pci.c | 131 ++++++++++++++++-
>> include/uapi/linux/vfio.h | 53 +++++++
>> 5 files changed, 603 insertions(+), 7 deletions(-)
>
>Maybe a chicken and egg problem, but it seems like we could split the
>platform code and vfio code into separate patches.
>
Ok. I'll keep egg/chicken separated in next revision.
>>
>> diff --git a/Documentation/vfio.txt b/Documentation/vfio.txt
>> index b9ca023..dd13db6 100644
>> --- a/Documentation/vfio.txt
>> +++ b/Documentation/vfio.txt
>> @@ -305,7 +305,10 @@ faster, the map/unmap handling has been implemented in real mode which provides
>> an excellent performance which has limitations such as inability to do
>> locked pages accounting in real time.
>>
>> -So 3 additional ioctls have been added:
>> +4) PPC64 guests detect PCI errors and recover from them via EEH RTAS services,
>> +which works on the basis of additional ioctl commands.
>> +
>> +So 8 additional ioctls have been added:
>>
>> VFIO_IOMMU_SPAPR_TCE_GET_INFO - returns the size and the start
>> of the DMA window on the PCI bus.
>> @@ -316,6 +319,20 @@ So 3 additional ioctls have been added:
>>
>> VFIO_IOMMU_DISABLE - disables the container.
>>
>> + VFIO_EEH_PE_SET_OPTION - enables or disables EEH functinality on the
>> + specified device. Also, it can be used to remove IO or DMA
>> + stopped state on the frozen PE.
>> +
>> + VFIO_EEH_PE_GET_ADDR - retrieve the unique address of the specified
>> + PE or query PE sharing mode.
>> +
>> + VFIO_EEH_PE_GET_STATE - retrieve PE's state: frozen or normal state.
>> +
>> + VFIO_EEH_PE_RESET - do PE reset, which is one of the major steps for
>> + error recovering.
>> +
>> + VFIO_EEH_PE_CONFIGURE - configure the PCI bridges after PE reset. It's
>> + one of the major steps for error recoverying.
>>
>> The code flow from the example above should be slightly changed:
>>
>> @@ -346,6 +363,75 @@ The code flow from the example above should be slightly changed:
>> ioctl(container, VFIO_IOMMU_MAP_DMA, &dma_map);
>> .....
>>
>> +Based on the initial example we have, the following piece of code could be
>> +reference for EEH setup and error handling:
>> +
>> + struct vfio_eeh_pe_set_option option = { .argsz = sizeof(option) };
>> + struct vfio_eeh_pe_get_addr addr = { .argsz = sizeof(addr) };
>> + struct vfio_eeh_pe_get_state state = { .argsz = sizeof(state) };
>> + struct vfio_eeh_pe_reset reset = { .argsz = sizeof(reset) };
>> + struct vfio_eeh_pe_configure config = { .argsz = sizeof(config) };
>> +
>> + ....
>> +
>> + /* Get a file descriptor for the device */
>> + device = ioctl(group, VFIO_GROUP_GET_DEVICE_FD, "0000:06:0d.0");
>> +
>> + /* Enable the EEH functionality on the device */
>> + option.option = EEH_OPT_ENABLE;
>> + ioctl(device, VFIO_EEH_PE_SET_OPTION, &option);
>> +
>> + /* Retrieve PE address and create and maintain PE by yourself */
>> + addr.option = EEH_OPT_GET_PE_ADDR;
>> + ioctl(device, VFIO_EEH_PE_GET_ADDR, &addr);
>> +
>> + /* Assure EEH is supported on the PE and make PE functional */
>> + ioctl(device, VFIO_EEH_PE_GET_STATE, &state);
>> +
>> + /* Save device's state. pci_save_state() would be good enough
>> + * as an example.
>> + */
>> +
>> + /* Test and setup the device */
>> + ioctl(device, VFIO_DEVICE_GET_INFO, &device_info);
>> +
>> + ....
>> +
>> + /* When 0xFF's returned from reading PCI config space or IO BARs
>> + * of the PCI device. Check the PE state to see if that has been
>> + * frozen.
>> + */
>> + ioctl(device, VFIO_EEH_PE_GET_STATE, &state);
>
>There's no notification, the user needs to observe the return value an
>poll? Should we be enabling an eventfd to notify the user of the state
>change?
>
Yes. The user needs to monitor the return value. we should have one notification,
but it's for later as we discussed :-)
>> +
>> + /* Waiting for pending PCI transactions to be completed and don't
>> + * produce any more PCI traffic from/to the affected PE until
>> + * recovery is finished.
>> + */
>> +
>> + /* Enable IO for the affected PE and collect logs. Usually, the
>> + * standard part of PCI config space, AER registers are dumped
>> + * as logs for further analysis.
>> + */
>> + option.option = EEH_OPT_THAW_MMIO;
>> + ioctl(device, VFIO_EEH_PE_SET_OPTION, &option);
>
>How does the guest learn about the error? Does it need to?
When guest detects 0xFF's from reading PCI config space or IO, it's going
check the device (PE) state. If the device (PE) has been put into frozen
state, the recovery will be started.
>> +
>> + /* Issue PE reset */
>> + reset.option = EEH_RESET_HOT;
>> + ioctl(device, VFIO_EEH_PE_RESET, &reset);
>> +
>> + /* Configure the PCI bridges for the affected PE */
>> + ioctl(device, VFIO_EEH_PE_CONFIGURE, NULL);
>> +
>
>I'm not sure I see why we've split these into separate ioctls. FWIW,
>the one ioctl we currently have for reset that takes no options is
>probably going to be the first to get deprecated because of it.
>
>> + /* Restored state we saved at initialization time. pci_restore_state()
>> + * is good enough as an example.
>> + */
>> +
>> + /* Hopefully, error is recovered successfully. Now, you can resume to
>> + * start PCI traffic to/from the affected PE.
>> + */
>> +
>> + ....
>> +
>> -------------------------------------------------------------------------------
>>
>> [1] VFIO was originally an acronym for "Virtual Function I/O" in its
>> diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
>> index 34a2d83..dd5f1cf 100644
>> --- a/arch/powerpc/include/asm/eeh.h
>> +++ b/arch/powerpc/include/asm/eeh.h
>> @@ -191,6 +191,11 @@ enum {
>> #define EEH_OPT_ENABLE 1 /* EEH enable */
>> #define EEH_OPT_THAW_MMIO 2 /* MMIO enable */
>> #define EEH_OPT_THAW_DMA 3 /* DMA enable */
>> +#define EEH_OPT_GET_PE_ADDR 0 /* Get PE addr */
>> +#define EEH_OPT_GET_PE_MODE 1 /* Get PE mode */
>> +#define EEH_PE_MODE_NONE 0 /* Invalid mode */
>> +#define EEH_PE_MODE_NOT_SHARED 1 /* Not shared */
>> +#define EEH_PE_MODE_SHARED 2 /* Shared mode */
>> #define EEH_STATE_UNAVAILABLE (1 << 0) /* State unavailable */
>> #define EEH_STATE_NOT_SUPPORT (1 << 1) /* EEH not supported */
>> #define EEH_STATE_RESET_ACTIVE (1 << 2) /* Active reset */
>> @@ -198,6 +203,11 @@ enum {
>> #define EEH_STATE_DMA_ACTIVE (1 << 4) /* Active DMA */
>> #define EEH_STATE_MMIO_ENABLED (1 << 5) /* MMIO enabled */
>> #define EEH_STATE_DMA_ENABLED (1 << 6) /* DMA enabled */
>> +#define EEH_PE_STATE_NORMAL 0 /* Normal state */
>> +#define EEH_PE_STATE_RESET 1 /* PE reset */
>> +#define EEH_PE_STATE_STOPPED_IO_DMA 2 /* Stopped */
>> +#define EEH_PE_STATE_STOPPED_DMA 4 /* Stopped DMA */
>> +#define EEH_PE_STATE_UNAVAIL 5 /* Unavailable */
>> #define EEH_RESET_DEACTIVATE 0 /* Deactivate the PE reset */
>> #define EEH_RESET_HOT 1 /* Hot reset */
>> #define EEH_RESET_FUNDAMENTAL 3 /* Fundamental reset */
>> @@ -305,6 +315,13 @@ void eeh_add_device_late(struct pci_dev *);
>> void eeh_add_device_tree_late(struct pci_bus *);
>> void eeh_add_sysfs_files(struct pci_bus *);
>> void eeh_remove_device(struct pci_dev *);
>> +int eeh_dev_open(struct pci_dev *pdev);
>> +void eeh_dev_release(struct pci_dev *pdev);
>> +int eeh_pe_set_option(struct pci_dev *pdev, int option);
>> +int eeh_pe_get_addr(struct pci_dev *pdev, int option);
>> +int eeh_pe_get_state(struct pci_dev *pdev);
>> +int eeh_pe_reset(struct pci_dev *pdev, int option);
>> +int eeh_pe_configure(struct pci_dev *pdev);
>>
>> /**
>> * EEH_POSSIBLE_ERROR() -- test for possible MMIO failure.
>> diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
>> index 9c6b899..b90a474 100644
>> --- a/arch/powerpc/kernel/eeh.c
>> +++ b/arch/powerpc/kernel/eeh.c
>> @@ -108,6 +108,9 @@ struct eeh_ops *eeh_ops = NULL;
>> /* Lock to avoid races due to multiple reports of an error */
>> DEFINE_RAW_SPINLOCK(confirm_error_lock);
>>
>> +/* Lock to protect passed flags */
>> +static DEFINE_MUTEX(eeh_dev_mutex);
>> +
>> /* Buffer for reporting pci register dumps. Its here in BSS, and
>> * not dynamically alloced, so that it ends up in RMO where RTAS
>> * can access it.
>> @@ -1098,6 +1101,324 @@ void eeh_remove_device(struct pci_dev *dev)
>> edev->mode &= ~EEH_DEV_SYSFS;
>> }
>>
>> +/**
>> + * eeh_dev_open - Mark EEH device and PE as passed through
>> + * @pdev: PCI device
>> + *
>> + * Mark the indicated EEH device and PE as passed through.
>> + * In the result, the EEH errors detected on the PE won't be
>> + * reported. The owner of the device will be responsible for
>> + * detection and recovery.
>> + */
>> +int eeh_dev_open(struct pci_dev *pdev)
>> +{
>> + struct eeh_dev *edev;
>> +
>> + mutex_lock(&eeh_dev_mutex);
>> +
>> + /* No PCI device ? */
>> + if (!pdev) {
>> + mutex_unlock(&eeh_dev_mutex);
>> + return -ENODEV;
>> + }
>> +
>> + /* No EEH device ? */
>> + edev = pci_dev_to_eeh_dev(pdev);
>> + if (!edev || !edev->pe) {
>> + mutex_unlock(&eeh_dev_mutex);
>> + return -ENODEV;
>> + }
>> +
>> + eeh_dev_set_passed(edev, true);
>> + eeh_pe_set_passed(edev->pe, true);
>> + mutex_unlock(&eeh_dev_mutex);
>> +
>> + return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(eeh_dev_open);
>> +
>> +/**
>> + * eeh_dev_release - Reclaim the ownership of EEH device
>> + * @pdev: PCI device
>> + *
>> + * Reclaim ownership of EEH device, potentially the corresponding
>> + * PE. In the result, the EEH errors detected on the PE will be
>> + * reported and handled as usual.
>> + */
>> +void eeh_dev_release(struct pci_dev *pdev)
>> +{
>> + bool release_pe = true;
>> + struct eeh_pe *pe = NULL;
>> + struct eeh_dev *tmp, *edev;
>> +
>> + mutex_lock(&eeh_dev_mutex);
>> +
>> + /* No PCI device ? */
>> + if (!pdev) {
>> + mutex_unlock(&eeh_dev_mutex);
>> + return;
>> + }
>> +
>> + /* No EEH device ? */
>> + edev = pci_dev_to_eeh_dev(pdev);
>> + if (!edev || !eeh_dev_passed(edev) ||
>> + !edev->pe || !eeh_pe_passed(pe)) {
>> + mutex_unlock(&eeh_dev_mutex);
>> + return;
>> + }
>> +
>> + /* Release device */
>> + pe = edev->pe;
>> + eeh_dev_set_passed(edev, false);
>> +
>> + /* Release PE */
>> + eeh_pe_for_each_dev(pe, edev, tmp) {
>> + if (eeh_dev_passed(edev)) {
>> + release_pe = false;
>> + break;
>> + }
>> + }
>> +
>> + if (release_pe)
>> + eeh_pe_set_passed(pe, false);
>> +
>> + mutex_unlock(&eeh_dev_mutex);
>> +}
>> +EXPORT_SYMBOL(eeh_dev_release);
>> +
>> +static int eeh_dev_check(struct pci_dev *pdev,
>> + struct eeh_dev **pedev,
>> + struct eeh_pe **ppe)
>> +{
>> + struct eeh_dev *edev;
>> +
>> + /* No device ? */
>> + if (!pdev)
>> + return -ENODEV;
>> +
>> + edev = pci_dev_to_eeh_dev(pdev);
>> + if (!edev || !eeh_dev_passed(edev) ||
>> + !edev->pe || !eeh_pe_passed(edev->pe))
>> + return -ENODEV;
>> +
>> + if (pedev)
>> + *pedev = edev;
>> + if (ppe)
>> + *ppe = edev->pe;
>> +
>> + return 0;
>> +}
>> +
>> +/**
>> + * eeh_pe_set_option - Set options for the indicated PE
>> + * @pdev: PCI device
>> + * @option: requested option
>> + *
>> + * The routine is called to enable or disable EEH functionality
>> + * on the indicated PE, to enable IO or DMA for the frozen PE.
>> + */
>> +int eeh_pe_set_option(struct pci_dev *pdev, int option)
>> +{
>> + struct eeh_dev *edev;
>> + struct eeh_pe *pe;
>> + int ret = 0;
>> +
>> + /* Device existing ? */
>> + ret = eeh_dev_check(pdev, &edev, &pe);
>> + if (ret)
>> + return ret;
>> +
>> + switch (option) {
>> + case EEH_OPT_DISABLE:
>> + case EEH_OPT_ENABLE:
>> + break;
>> + case EEH_OPT_THAW_MMIO:
>> + case EEH_OPT_THAW_DMA:
>> + if (!eeh_ops || !eeh_ops->set_option) {
>> + ret = -ENOENT;
>> + break;
>> + }
>> +
>> + ret = eeh_ops->set_option(pe, option);
>> + break;
>> + default:
>> + pr_debug("%s: Option %d out of range (%d, %d)\n",
>> + __func__, option, EEH_OPT_DISABLE, EEH_OPT_THAW_DMA);
>> + ret = -EINVAL;
>> + }
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(eeh_pe_set_option);
>> +
>> +/**
>> + * eeh_pe_get_addr - Retrieve the PE address or sharing mode
>> + * @pdev: PCI device
>> + * @option: option
>> + *
>> + * Retrieve the PE address or sharing mode.
>> + */
>> +int eeh_pe_get_addr(struct pci_dev *pdev, int option)
>> +{
>> + struct pci_bus *bus;
>> + struct eeh_dev *edev;
>> + struct eeh_pe *pe;
>> + int ret = 0;
>> +
>> + /* Device existing ? */
>> + ret = eeh_dev_check(pdev, &edev, &pe);
>> + if (ret)
>> + return ret;
>> +
>> + switch (option) {
>> + case EEH_OPT_GET_PE_ADDR:
>> + bus = eeh_pe_bus_get(pe);
>> + if (!bus) {
>> + ret = -ENODEV;
>> + break;
>> + }
>> +
>> + /* PE address has format "00BBSS00" */
>> + ret = bus->number << 16;
>> + break;
>> + case EEH_OPT_GET_PE_MODE:
>> + /* Wa always have shared PE */
>> + ret = EEH_PE_MODE_SHARED;
>> + break;
>> + default:
>> + pr_debug("%s: option %d out of range (0, 1)\n",
>> + __func__, option);
>> + ret = -EINVAL;
>> + }
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(eeh_pe_get_addr);
>> +
>> +/**
>> + * eeh_pe_get_state - Retrieve PE's state
>> + * @pdev: PCI device
>> + *
>> + * Retrieve the PE's state, which includes 3 aspects: enabled
>> + * DMA, enabled IO and asserted reset.
>> + */
>> +int eeh_pe_get_state(struct pci_dev *pdev)
>> +{
>> + struct eeh_dev *edev;
>> + struct eeh_pe *pe;
>> + int result, ret = 0;
>> +
>> + /* Device existing ? */
>> + ret = eeh_dev_check(pdev, &edev, &pe);
>> + if (ret)
>> + return ret;
>> +
>> + if (!eeh_ops || !eeh_ops->get_state)
>> + return -ENOENT;
>> +
>> + result = eeh_ops->get_state(pe, NULL);
>> + if (!(result & EEH_STATE_RESET_ACTIVE) &&
>> + (result & EEH_STATE_DMA_ENABLED) &&
>> + (result & EEH_STATE_MMIO_ENABLED))
>> + ret = EEH_PE_STATE_NORMAL;
>> + else if (result & EEH_STATE_RESET_ACTIVE)
>> + ret = EEH_PE_STATE_RESET;
>> + else if (!(result & EEH_STATE_RESET_ACTIVE) &&
>> + !(result & EEH_STATE_DMA_ENABLED) &&
>> + !(result & EEH_STATE_MMIO_ENABLED))
>> + ret = EEH_PE_STATE_STOPPED_IO_DMA;
>> + else if (!(result & EEH_STATE_RESET_ACTIVE) &&
>> + (result & EEH_STATE_DMA_ENABLED) &&
>> + !(result & EEH_STATE_MMIO_ENABLED))
>> + ret = EEH_PE_STATE_STOPPED_DMA;
>> + else
>> + ret = EEH_PE_STATE_UNAVAIL;
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(eeh_pe_get_state);
>> +
>> +/**
>> + * eeh_pe_reset - Issue PE reset according to specified type
>> + * @pdev: PCI device
>> + * @option: reset type
>> + *
>> + * The routine is called to reset the specified PE with the
>> + * indicated type, either fundamental reset or hot reset.
>> + * PE reset is the most important part for error recovery.
>> + */
>> +int eeh_pe_reset(struct pci_dev *pdev, int option)
>> +{
>> + struct eeh_dev *edev;
>> + struct eeh_pe *pe;
>> + int ret = 0;
>> +
>> + /* Device existing ? */
>> + ret = eeh_dev_check(pdev, &edev, &pe);
>> + if (ret)
>> + return ret;
>> +
>> + if (!eeh_ops || !eeh_ops->set_option || !eeh_ops->reset)
>> + return -ENOENT;
>> +
>> + switch (option) {
>> + case EEH_RESET_DEACTIVATE:
>> + ret = eeh_ops->reset(pe, option);
>> + if (ret)
>> + break;
>> +
>> + /*
>> + * The PE is still in frozen state and we need clear
>> + * that. It's good to clear frozen state after deassert
>> + * to avoid messy IO access during reset, which might
>> + * cause recursive frozen PE.
>> + */
>> + ret = eeh_ops->set_option(pe, EEH_OPT_THAW_MMIO);
>> + if (!ret)
>> + ret = eeh_ops->set_option(pe, EEH_OPT_THAW_DMA);
>> + if (!ret)
>> + eeh_pe_state_clear(pe, EEH_PE_ISOLATED);
>> + break;
>> + case EEH_RESET_HOT:
>> + case EEH_RESET_FUNDAMENTAL:
>> + ret = eeh_ops->reset(pe, option);
>> + break;
>> + default:
>> + pr_debug("%s: Unsupported option %d\n",
>> + __func__, option);
>> + ret = -EINVAL;
>> + }
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(eeh_pe_reset);
>> +
>> +/**
>> + * eeh_pe_configure - Configure PCI bridges after PE reset
>> + * @pdev: PCI device
>> + *
>> + * The routine is called to restore the PCI config space for
>> + * those PCI devices, especially PCI bridges affected by PE
>> + * reset issued previously.
>> + */
>> +int eeh_pe_configure(struct pci_dev *pdev)
>> +{
>> + struct eeh_dev *edev;
>> + struct eeh_pe *pe;
>> + int ret = 0;
>> +
>> + /* Device existing ? */
>> + ret = eeh_dev_check(pdev, &edev, &pe);
>> + if (ret)
>> + return ret;
>> +
>> + /* Restore config space for the affected devices */
>> + eeh_pe_restore_bars(pe);
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(eeh_pe_configure);
>> +
>> static int proc_eeh_show(struct seq_file *m, void *v)
>> {
>> if (!eeh_enabled()) {
>> diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
>> index 7ba0424..301ac18 100644
>> --- a/drivers/vfio/pci/vfio_pci.c
>> +++ b/drivers/vfio/pci/vfio_pci.c
>> @@ -25,6 +25,9 @@
>> #include <linux/types.h>
>> #include <linux/uaccess.h>
>> #include <linux/vfio.h>
>> +#ifdef CONFIG_EEH
>> +#include <asm/eeh.h>
>> +#endif
>
>Can we make a vfio_pci_eeh file that properly stubs out anything called
>from common code? I don't want to see all these inline ifdefs.
>
well. do you want see drivers/vfio/vfio-pci/vfio_pci_eeh.c and export
following functins for vfio_pci.c to use?
vfio_pci_eeh_open()
vfio_pci_eeh_release()
vfio_pci_eeh_ioctl()
>>
>> #include "vfio_pci_private.h"
>>
>> @@ -152,32 +155,57 @@ static void vfio_pci_disable(struct vfio_pci_device *vdev)
>> pci_restore_state(pdev);
>> }
>>
>> +static void vfio_eeh_pci_release(struct pci_dev *pdev)
>> +{
>> +#ifdef CONFIG_EEH
>> + eeh_dev_release(pdev);
>> +#endif
>> +}
>> +
>> static void vfio_pci_release(void *device_data)
>> {
>> struct vfio_pci_device *vdev = device_data;
>>
>> - if (atomic_dec_and_test(&vdev->refcnt))
>> + if (atomic_dec_and_test(&vdev->refcnt)) {
>> + vfio_eeh_pci_release(vdev->pdev);
>> vfio_pci_disable(vdev);
>> + }
>>
>> module_put(THIS_MODULE);
>> }
>>
>> +static int vfio_eeh_pci_open(struct pci_dev *pdev)
>> +{
>> + int ret = 0;
>> +
>> +#ifdef CONFIG_EEH
>> + ret = eeh_dev_open(pdev);
>> +#endif
>> + return ret;
>> +}
>> +
>> static int vfio_pci_open(void *device_data)
>> {
>> struct vfio_pci_device *vdev = device_data;
>> + int ret;
>>
>> if (!try_module_get(THIS_MODULE))
>> return -ENODEV;
>>
>> if (atomic_inc_return(&vdev->refcnt) == 1) {
>> - int ret = vfio_pci_enable(vdev);
>> - if (ret) {
>> - module_put(THIS_MODULE);
>> - return ret;
>> - }
>> + ret = vfio_pci_enable(vdev);
>> + if (ret)
>> + goto error;
>> +
>> + ret = vfio_eeh_pci_open(vdev->pdev);
>> + if (ret)
>> + goto error;
>> }
>>
>> return 0;
>> +error:
>> + module_put(THIS_MODULE);
>> + return ret;
>> }
>>
>> static int vfio_pci_get_irq_count(struct vfio_pci_device *vdev, int irq_type)
>> @@ -321,6 +349,91 @@ static int vfio_pci_for_each_slot_or_bus(struct pci_dev *pdev,
>> return walk.ret;
>> }
>>
>> +static int vfio_eeh_pci_ioctl(struct pci_dev *pdev,
>> + unsigned int cmd,
>> + unsigned long arg)
>> +{
>> + unsigned long minsz;
>> + int ret = 0;
>> +
>> +#ifdef CONFIG_EEH
>> + switch (cmd) {
>> + case VFIO_EEH_PE_SET_OPTION: {
>> + struct vfio_eeh_pe_set_option option;
>> +
>> + minsz = offsetofend(struct vfio_eeh_pe_set_option, option);
>> + if (copy_from_user(&option, (void __user *)arg, minsz))
>> + return -EFAULT;
>> + if (option.argsz < minsz)
>> + return -EINVAL;
>> +
>> + ret = eeh_pe_set_option(pdev, option.option);
>> + break;
>> + }
>> + case VFIO_EEH_PE_GET_ADDR: {
>> + struct vfio_eeh_pe_get_addr addr;
>> +
>> + minsz = offsetofend(struct vfio_eeh_pe_get_addr, info);
>> + if (copy_from_user(&addr, (void __user *)arg, minsz))
>> + return -EFAULT;
>> + if (addr.argsz < minsz)
>> + return -EINVAL;
>> +
>> + ret = eeh_pe_get_addr(pdev, addr.option);
>> + if (ret >= 0) {
>> + addr.info = ret;
>> + if (copy_to_user((void __user *)arg, &addr, minsz))
>> + return -EFAULT;
>> + ret = 0;
>> + }
>> +
>> + break;
>> + }
>> + case VFIO_EEH_PE_GET_STATE: {
>> + struct vfio_eeh_pe_get_state state;
>> +
>> + minsz = offsetofend(struct vfio_eeh_pe_get_state, state);
>> + if (copy_from_user(&state, (void __user *)arg, minsz))
>> + return -EFAULT;
>> + if (state.argsz < minsz)
>> + return -EINVAL;
>> +
>> + ret = eeh_pe_get_state(pdev);
>> + if (ret >= 0) {
>> + state.state = ret;
>> + if (copy_to_user((void __user *)arg, &state, minsz))
>> + return -EFAULT;
>> + ret = 0;
>> + }
>> + break;
>> + }
>> + case VFIO_EEH_PE_RESET: {
>> + struct vfio_eeh_pe_reset reset;
>> +
>> + minsz = offsetofend(struct vfio_eeh_pe_reset, option);
>> + if (copy_from_user(&reset, (void __user *)arg, minsz))
>> + return -EFAULT;
>> + if (reset.argsz < minsz)
>> + return -EINVAL;
>> +
>> + ret = eeh_pe_reset(pdev, reset.option);
>> + break;
>> + }
>> + case VFIO_EEH_PE_CONFIGURE:
>> + ret = eeh_pe_configure(pdev);
>> + break;
>> + default:
>> + ret = -EINVAL;
>> + pr_debug("%s: Cannot handle command %d\n",
>> + __func__, cmd);
>> + }
>> +#else
>> + ret = -ENOENT;
>> +#endif
>> +
>> + return ret;
>> +}
>> +
>> static long vfio_pci_ioctl(void *device_data,
>> unsigned int cmd, unsigned long arg)
>> {
>> @@ -682,6 +795,12 @@ hot_reset_release:
>>
>> kfree(groups);
>> return ret;
>> + } else if (cmd == VFIO_EEH_PE_SET_OPTION ||
>> + cmd == VFIO_EEH_PE_GET_ADDR ||
>> + cmd == VFIO_EEH_PE_GET_STATE ||
>> + cmd == VFIO_EEH_PE_RESET ||
>> + cmd == VFIO_EEH_PE_CONFIGURE) {
>> + return vfio_eeh_pci_ioctl(vdev->pdev, cmd, arg);
>> }
>>
>> return -ENOTTY;
>> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
>> index cb9023d..ef55682 100644
>> --- a/include/uapi/linux/vfio.h
>> +++ b/include/uapi/linux/vfio.h
>> @@ -455,6 +455,59 @@ struct vfio_iommu_spapr_tce_info {
>>
>> #define VFIO_IOMMU_SPAPR_TCE_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
>>
>> +/*
>> + * EEH functionality can be enabled or disabled on one specific device.
>> + * Also, the DMA or IO frozen state can be removed from the frozen PE
>> + * if required.
>> + */
>> +struct vfio_eeh_pe_set_option {
>> + __u32 argsz;
>> + __u32 option;
>> +};
>> +
>> +#define VFIO_EEH_PE_SET_OPTION _IO(VFIO_TYPE, VFIO_BASE + 21)
>
>What proposed ioctls are making you jump to 21?
>
>argsz is probably not as useful without a flags field. Otherwise the
>caller can't indicate what the extra space is.
>
The QEMU patches are based on Alexey's additional feature ("ddw"), which
consumed several ioctl commands.
ok. So you also prefer to remove "argsz"?
>> +
>> +/*
>> + * Each EEH PE should have unique address to be identified. The command
>> + * helps to retrieve the address and the sharing mode of the PE.
>> + */
>> +struct vfio_eeh_pe_get_addr {
>> + __u32 argsz;
>> + __u32 option;
>> + __u32 info;
>> +};
>> +
>> +#define VFIO_EEH_PE_GET_ADDR _IO(VFIO_TYPE, VFIO_BASE + 22)
>> +
>> +/*
>> + * EEH PE might have been frozen because of PCI errors. Also, it might
>> + * be experiencing reset for error revoery. The following command helps
>> + * to get the state.
>> + */
>> +struct vfio_eeh_pe_get_state {
>> + __u32 argsz;
>> + __u32 state;
>> +};
>> +
>> +#define VFIO_EEH_PE_GET_STATE _IO(VFIO_TYPE, VFIO_BASE + 23)
>> +
>> +/*
>> + * Reset is the major step to recover problematic PE. The following
>> + * command helps on that.
>> + */
>> +struct vfio_eeh_pe_reset {
>> + __u32 argsz;
>> + __u32 option;
>> +};
>> +
>> +#define VFIO_EEH_PE_RESET _IO(VFIO_TYPE, VFIO_BASE + 24)
>> +
>> +/*
>> + * One of the steps for recovery after PE reset is to configure the
>> + * PCI bridges affected by the PE reset.
>> + */
>> +#define VFIO_EEH_PE_CONFIGURE _IO(VFIO_TYPE, VFIO_BASE + 25)
>
>What can the user do differently by making these separate ioctls?
>
hrm, I didn't understood as well. Alex.G could have the explaination.
>> +
>> /* ***************************************************************** */
>>
>> #endif /* _UAPIVFIO_H */
Thanks,
Gavin
^ permalink raw reply
* Re: [PATCH v6 2/3] drivers/vfio: EEH support for VFIO PCI device
From: Benjamin Herrenschmidt @ 2014-05-23 5:00 UTC (permalink / raw)
To: Gavin Shan, Alex Williamson; +Cc: aik, qiudayu, linuxppc-dev, agraf, kvm-ppc
In-Reply-To: <20140523043722.GA11572@shangw>
On Fri, 2014-05-23 at 14:37 +1000, Gavin Shan wrote:
> >There's no notification, the user needs to observe the return value an
> >poll? Should we be enabling an eventfd to notify the user of the state
> >change?
> >
>
> Yes. The user needs to monitor the return value. we should have one notification,
> but it's for later as we discussed :-)
../..
> >How does the guest learn about the error? Does it need to?
>
> When guest detects 0xFF's from reading PCI config space or IO, it's going
> check the device (PE) state. If the device (PE) has been put into frozen
> state, the recovery will be started.
Quick recap for Alex W (we discussed that with Alex G).
While a notification looks like a worthwhile addition in the long run, it
is not sufficient and not used today and I prefer that we keep that as something
to add later for those two main reasons:
- First, the kernel itself isn't always notified. For example, if we implement
on top of an RTAS backend (PR KVM under pHyp) or if we are on top of PowerNV but
the error is a PHB "fence" (the entire PCI Host bridge gets fenced out in hardware
due to an internal error), then we get no notification. Only polling of the
hardware or firmware will tell us. Since we don't want to have a polling timer
in the kernel, that means that the userspace client of VFIO (or alternatively
the KVM guest) is the one that polls.
- Second, this is how our primary user expects it: The primary (and only initial)
user of this will be qemu/KVM for PAPR guests and they don't have a notification
mechanism. Instead they query the EEH state after detecting an all 1's return from
MMIO or config space. This is how PAPR specifies it so we are just implementing the
spec here :-)
Because of these, I think we shouldn't worry too much about notification at
this stage.
Cheers,
Ben.
^ permalink raw reply
* Re: [PATCH v6 2/3] drivers/vfio: EEH support for VFIO PCI device
From: Alexander Graf @ 2014-05-23 6:52 UTC (permalink / raw)
To: Alex Williamson
Cc: aik@ozlabs.ru, Gavin Shan, kvm-ppc@vger.kernel.org,
qiudayu@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org
In-Reply-To: <1400815400.3289.436.camel@ul30vt.home>
> Am 23.05.2014 um 05:23 schrieb Alex Williamson <alex.williamson@redhat.com=
>:
>=20
>> On Fri, 2014-05-23 at 10:37 +1000, Gavin Shan wrote:
>>> On Fri, May 23, 2014 at 10:17:30AM +1000, Gavin Shan wrote:
>>>> On Thu, May 22, 2014 at 11:55:29AM +0200, Alexander Graf wrote:
>>>> On 22.05.14 10:23, Gavin Shan wrote:
>>=20
>> .../...
>>=20
>>>>> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
>>>>> index cb9023d..ef55682 100644
>>>>> --- a/include/uapi/linux/vfio.h
>>>>> +++ b/include/uapi/linux/vfio.h
>>>>> @@ -455,6 +455,59 @@ struct vfio_iommu_spapr_tce_info {
>>>>> #define VFIO_IOMMU_SPAPR_TCE_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12=
)
>>>>> +/*
>>>>> + * EEH functionality can be enabled or disabled on one specific devic=
e.
>>>>> + * Also, the DMA or IO frozen state can be removed from the frozen PE=
>>>>> + * if required.
>>>>> + */
>>>>> +struct vfio_eeh_pe_set_option {
>>>>> + __u32 argsz;
>>>>=20
>>>> What is this argsz thing? Is this your way of maintaining backwards
>>>> compatibility when we introduce new fields? A new field will change
>>>> the ioctl number, so I don't think that makes a lot of sense :).
>>>>=20
>>>> Just make the ioctl have a u32 as incoming argument. No fancy
>>>> structs, no complicated code.
>>>>=20
>>>> The same applies for a number of structs below.
>>>=20
>>> ok. Will do in next revision.
>>=20
>> Rechecked include/uapi/linux/vfio.h, the data struct for each ioctl comma=
nd
>> always has "argsz". I guess it was used as checker by Alex.W. Do you real=
ly
>> want remove "argsz" ?
>=20
>=20
> IIRC, this was actually a suggestion incorporated from David Gibson, but
> using _IO with an argsz and flags field we can maintain compatibility
> without bumping the ioctl number. It really only makes sense if we have
> a flags field so we can identify what additional information is being
> provided. Flags can be used as a bitmap of trailing structures or as
> revision if we want a set of trailing structures that may change over
> time. Unless you can come up with a good argument against it that would
> prevent us inventing a new ioctl as soon as we need a minor tweak, I'd
> prefer to keep it. As I noted in a previous comment, the one ioctl we
> have for reset that doesn't take any options is likely going to be the
> first ioctl that we need to entirely replace. If we don't keep argsz,
> it seems like we probably need a flags field and reserved structures.
>=20
>>>>> + __u32 option;
>>>>> +};
>>>>> +
>>>>> +#define VFIO_EEH_PE_SET_OPTION _IO(VFIO_TYPE, VFIO_BASE + 21)
>>>>> +
>>>>> +/*
>>>>> + * Each EEH PE should have unique address to be identified. The comma=
nd
>>>>> + * helps to retrieve the address and the sharing mode of the PE.
>>>>> + */
>>>>> +struct vfio_eeh_pe_get_addr {
>>>>> + __u32 argsz;
>>>>> + __u32 option;
>>>>> + __u32 info;
>>>>=20
>>>> Any particular reason you need the info field? Can't the return value
>>>> of the ioctl hold this? Then you only have a single u32 argument left
>>>> to the ioctl again.
>>>=20
>>> ok. Will do in next revision.
>>=20
>> If we eventually remove "argsz" and let ioctl() return value to hold
>> information (or negative number for errors), we don't need any data
>> struct because the 3rd parameter of ioctl() would be used as input
>> and I only need one input parameter. Do you want see this ?
>>=20
>> Hopefully, Alex.W saw this and hasn't objections :)
>=20
> I'm not sure why we're pushing for the minimal data set to pass to an
> ioctl. Seems like a recipe for dead, useless ioctls. Thanks,
>=20
The ioctl number includes sizeof(payload). So if a new parameter gets added,=
that would be a different ioctl number.
If you want to maintain backwards compatibility ioctl number wise in the ker=
nel, you'll have to have a "flags" field to indicate whether new data is ava=
ilable and a "pad" field, prefarably in a union, that ensures the size of th=
e struct doesn't change.
I'm not sure it's really necessary here to have identical ioctl numbers if w=
e add parameters, since we can always just define a new ioctl with a bigger p=
ayload that can then become the default handler and a shim backwards compati=
ble handler with the old number.
But if you think it is important, let's do it for real, not just halfway.
Alex
^ permalink raw reply
* Re: [PATCH v6 2/3] drivers/vfio: EEH support for VFIO PCI device
From: Alexander Graf @ 2014-05-23 6:55 UTC (permalink / raw)
To: Gavin Shan
Cc: aik@ozlabs.ru, Gavin Shan, kvm-ppc@vger.kernel.org,
Alex Williamson, qiudayu@linux.vnet.ibm.com,
linuxppc-dev@lists.ozlabs.org
In-Reply-To: <20140523043722.GA11572@shangw>
> Am 23.05.2014 um 06:37 schrieb Gavin Shan <gwshan@linux.vnet.ibm.com>:
>=20
>> On Thu, May 22, 2014 at 09:10:53PM -0600, Alex Williamson wrote:
>>> On Thu, 2014-05-22 at 18:23 +1000, Gavin Shan wrote:
>>> The patch adds new IOCTL commands for VFIO PCI device to support
>>> EEH functionality for PCI devices, which have been passed through
>>> from host to somebody else via VFIO.
>>>=20
>>> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
>>> ---
>>> Documentation/vfio.txt | 88 ++++++++++-
>>> arch/powerpc/include/asm/eeh.h | 17 +++
>>> arch/powerpc/kernel/eeh.c | 321 +++++++++++++++++++++++++++++++++++=
++++++
>>> drivers/vfio/pci/vfio_pci.c | 131 ++++++++++++++++-
>>> include/uapi/linux/vfio.h | 53 +++++++
>>> 5 files changed, 603 insertions(+), 7 deletions(-)
>>=20
>> Maybe a chicken and egg problem, but it seems like we could split the
>> platform code and vfio code into separate patches.
>=20
> Ok. I'll keep egg/chicken separated in next revision.
>=20
>>>=20
>>> diff --git a/Documentation/vfio.txt b/Documentation/vfio.txt
>>> index b9ca023..dd13db6 100644
>>> --- a/Documentation/vfio.txt
>>> +++ b/Documentation/vfio.txt
>>> @@ -305,7 +305,10 @@ faster, the map/unmap handling has been implemented=
in real mode which provides
>>> an excellent performance which has limitations such as inability to do
>>> locked pages accounting in real time.
>>>=20
>>> -So 3 additional ioctls have been added:
>>> +4) PPC64 guests detect PCI errors and recover from them via EEH RTAS se=
rvices,
>>> +which works on the basis of additional ioctl commands.
>>> +
>>> +So 8 additional ioctls have been added:
>>>=20
>>> VFIO_IOMMU_SPAPR_TCE_GET_INFO - returns the size and the start
>>> of the DMA window on the PCI bus.
>>> @@ -316,6 +319,20 @@ So 3 additional ioctls have been added:
>>>=20
>>> VFIO_IOMMU_DISABLE - disables the container.
>>>=20
>>> + VFIO_EEH_PE_SET_OPTION - enables or disables EEH functinality on th=
e
>>> + specified device. Also, it can be used to remove IO or DMA
>>> + stopped state on the frozen PE.
>>> +
>>> + VFIO_EEH_PE_GET_ADDR - retrieve the unique address of the specified=
>>> + PE or query PE sharing mode.
>>> +
>>> + VFIO_EEH_PE_GET_STATE - retrieve PE's state: frozen or normal state=
.
>>> +
>>> + VFIO_EEH_PE_RESET - do PE reset, which is one of the major steps fo=
r
>>> + error recovering.
>>> +
>>> + VFIO_EEH_PE_CONFIGURE - configure the PCI bridges after PE reset. I=
t's
>>> + one of the major steps for error recoverying.
>>>=20
>>> The code flow from the example above should be slightly changed:
>>>=20
>>> @@ -346,6 +363,75 @@ The code flow from the example above should be slig=
htly changed:
>>> ioctl(container, VFIO_IOMMU_MAP_DMA, &dma_map);
>>> .....
>>>=20
>>> +Based on the initial example we have, the following piece of code could=
be
>>> +reference for EEH setup and error handling:
>>> +
>>> + struct vfio_eeh_pe_set_option option =3D { .argsz =3D sizeof(option=
) };
>>> + struct vfio_eeh_pe_get_addr addr =3D { .argsz =3D sizeof(addr) };
>>> + struct vfio_eeh_pe_get_state state =3D { .argsz =3D sizeof(state) }=
;
>>> + struct vfio_eeh_pe_reset reset =3D { .argsz =3D sizeof(reset) };
>>> + struct vfio_eeh_pe_configure config =3D { .argsz =3D sizeof(config)=
};
>>> +
>>> + ....
>>> +
>>> + /* Get a file descriptor for the device */
>>> + device =3D ioctl(group, VFIO_GROUP_GET_DEVICE_FD, "0000:06:0d.0");
>>> +
>>> + /* Enable the EEH functionality on the device */
>>> + option.option =3D EEH_OPT_ENABLE;
>>> + ioctl(device, VFIO_EEH_PE_SET_OPTION, &option);
>>> +
>>> + /* Retrieve PE address and create and maintain PE by yourself */
>>> + addr.option =3D EEH_OPT_GET_PE_ADDR;
>>> + ioctl(device, VFIO_EEH_PE_GET_ADDR, &addr);
>>> +
>>> + /* Assure EEH is supported on the PE and make PE functional */
>>> + ioctl(device, VFIO_EEH_PE_GET_STATE, &state);
>>> +
>>> + /* Save device's state. pci_save_state() would be good enough
>>> + * as an example.
>>> + */
>>> +
>>> + /* Test and setup the device */
>>> + ioctl(device, VFIO_DEVICE_GET_INFO, &device_info);
>>> +
>>> + ....
>>> +
>>> + /* When 0xFF's returned from reading PCI config space or IO BARs
>>> + * of the PCI device. Check the PE state to see if that has been
>>> + * frozen.
>>> + */
>>> + ioctl(device, VFIO_EEH_PE_GET_STATE, &state);
>>=20
>> There's no notification, the user needs to observe the return value an
>> poll? Should we be enabling an eventfd to notify the user of the state
>> change?
>=20
> Yes. The user needs to monitor the return value. we should have one notifi=
cation,
> but it's for later as we discussed :-)
>=20
>>> +
>>> + /* Waiting for pending PCI transactions to be completed and don't
>>> + * produce any more PCI traffic from/to the affected PE until
>>> + * recovery is finished.
>>> + */
>>> +
>>> + /* Enable IO for the affected PE and collect logs. Usually, the
>>> + * standard part of PCI config space, AER registers are dumped
>>> + * as logs for further analysis.
>>> + */
>>> + option.option =3D EEH_OPT_THAW_MMIO;
>>> + ioctl(device, VFIO_EEH_PE_SET_OPTION, &option);
>>=20
>> How does the guest learn about the error? Does it need to?
>=20
> When guest detects 0xFF's from reading PCI config space or IO, it's going
> check the device (PE) state. If the device (PE) has been put into frozen
> state, the recovery will be started.
>=20
>>> +
>>> + /* Issue PE reset */
>>> + reset.option =3D EEH_RESET_HOT;
>>> + ioctl(device, VFIO_EEH_PE_RESET, &reset);
>>> +
>>> + /* Configure the PCI bridges for the affected PE */
>>> + ioctl(device, VFIO_EEH_PE_CONFIGURE, NULL);
>>> +
>>=20
>> I'm not sure I see why we've split these into separate ioctls. FWIW,
>> the one ioctl we currently have for reset that takes no options is
>> probably going to be the first to get deprecated because of it.
>>=20
>>> + /* Restored state we saved at initialization time. pci_restore_stat=
e()
>>> + * is good enough as an example.
>>> + */
>>> +
>>> + /* Hopefully, error is recovered successfully. Now, you can resume t=
o
>>> + * start PCI traffic to/from the affected PE.
>>> + */
>>> +
>>> + ....
>>> +
>>> ------------------------------------------------------------------------=
-------
>>>=20
>>> [1] VFIO was originally an acronym for "Virtual Function I/O" in its
>>> diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/e=
eh.h
>>> index 34a2d83..dd5f1cf 100644
>>> --- a/arch/powerpc/include/asm/eeh.h
>>> +++ b/arch/powerpc/include/asm/eeh.h
>>> @@ -191,6 +191,11 @@ enum {
>>> #define EEH_OPT_ENABLE 1 /* EEH enable */
>>> #define EEH_OPT_THAW_MMIO 2 /* MMIO enable */
>>> #define EEH_OPT_THAW_DMA 3 /* DMA enable */
>>> +#define EEH_OPT_GET_PE_ADDR 0 /* Get PE addr */
>>> +#define EEH_OPT_GET_PE_MODE 1 /* Get PE mode */
>>> +#define EEH_PE_MODE_NONE 0 /* Invalid mode */
>>> +#define EEH_PE_MODE_NOT_SHARED 1 /* Not shared */
>>> +#define EEH_PE_MODE_SHARED 2 /* Shared mode */
>>> #define EEH_STATE_UNAVAILABLE (1 << 0) /* State unavailable */
>>> #define EEH_STATE_NOT_SUPPORT (1 << 1) /* EEH not supported */
>>> #define EEH_STATE_RESET_ACTIVE (1 << 2) /* Active reset */
>>> @@ -198,6 +203,11 @@ enum {
>>> #define EEH_STATE_DMA_ACTIVE (1 << 4) /* Active DMA */
>>> #define EEH_STATE_MMIO_ENABLED (1 << 5) /* MMIO enabled */
>>> #define EEH_STATE_DMA_ENABLED (1 << 6) /* DMA enabled */
>>> +#define EEH_PE_STATE_NORMAL 0 /* Normal state */
>>> +#define EEH_PE_STATE_RESET 1 /* PE reset */
>>> +#define EEH_PE_STATE_STOPPED_IO_DMA 2 /* Stopped */
>>> +#define EEH_PE_STATE_STOPPED_DMA 4 /* Stopped DMA */
>>> +#define EEH_PE_STATE_UNAVAIL 5 /* Unavailable */
>>> #define EEH_RESET_DEACTIVATE 0 /* Deactivate the PE reset */
>>> #define EEH_RESET_HOT 1 /* Hot reset */
>>> #define EEH_RESET_FUNDAMENTAL 3 /* Fundamental reset */
>>> @@ -305,6 +315,13 @@ void eeh_add_device_late(struct pci_dev *);
>>> void eeh_add_device_tree_late(struct pci_bus *);
>>> void eeh_add_sysfs_files(struct pci_bus *);
>>> void eeh_remove_device(struct pci_dev *);
>>> +int eeh_dev_open(struct pci_dev *pdev);
>>> +void eeh_dev_release(struct pci_dev *pdev);
>>> +int eeh_pe_set_option(struct pci_dev *pdev, int option);
>>> +int eeh_pe_get_addr(struct pci_dev *pdev, int option);
>>> +int eeh_pe_get_state(struct pci_dev *pdev);
>>> +int eeh_pe_reset(struct pci_dev *pdev, int option);
>>> +int eeh_pe_configure(struct pci_dev *pdev);
>>>=20
>>> /**
>>> * EEH_POSSIBLE_ERROR() -- test for possible MMIO failure.
>>> diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
>>> index 9c6b899..b90a474 100644
>>> --- a/arch/powerpc/kernel/eeh.c
>>> +++ b/arch/powerpc/kernel/eeh.c
>>> @@ -108,6 +108,9 @@ struct eeh_ops *eeh_ops =3D NULL;
>>> /* Lock to avoid races due to multiple reports of an error */
>>> DEFINE_RAW_SPINLOCK(confirm_error_lock);
>>>=20
>>> +/* Lock to protect passed flags */
>>> +static DEFINE_MUTEX(eeh_dev_mutex);
>>> +
>>> /* Buffer for reporting pci register dumps. Its here in BSS, and
>>> * not dynamically alloced, so that it ends up in RMO where RTAS
>>> * can access it.
>>> @@ -1098,6 +1101,324 @@ void eeh_remove_device(struct pci_dev *dev)
>>> edev->mode &=3D ~EEH_DEV_SYSFS;
>>> }
>>>=20
>>> +/**
>>> + * eeh_dev_open - Mark EEH device and PE as passed through
>>> + * @pdev: PCI device
>>> + *
>>> + * Mark the indicated EEH device and PE as passed through.
>>> + * In the result, the EEH errors detected on the PE won't be
>>> + * reported. The owner of the device will be responsible for
>>> + * detection and recovery.
>>> + */
>>> +int eeh_dev_open(struct pci_dev *pdev)
>>> +{
>>> + struct eeh_dev *edev;
>>> +
>>> + mutex_lock(&eeh_dev_mutex);
>>> +
>>> + /* No PCI device ? */
>>> + if (!pdev) {
>>> + mutex_unlock(&eeh_dev_mutex);
>>> + return -ENODEV;
>>> + }
>>> +
>>> + /* No EEH device ? */
>>> + edev =3D pci_dev_to_eeh_dev(pdev);
>>> + if (!edev || !edev->pe) {
>>> + mutex_unlock(&eeh_dev_mutex);
>>> + return -ENODEV;
>>> + }
>>> +
>>> + eeh_dev_set_passed(edev, true);
>>> + eeh_pe_set_passed(edev->pe, true);
>>> + mutex_unlock(&eeh_dev_mutex);
>>> +
>>> + return 0;
>>> +}
>>> +EXPORT_SYMBOL_GPL(eeh_dev_open);
>>> +
>>> +/**
>>> + * eeh_dev_release - Reclaim the ownership of EEH device
>>> + * @pdev: PCI device
>>> + *
>>> + * Reclaim ownership of EEH device, potentially the corresponding
>>> + * PE. In the result, the EEH errors detected on the PE will be
>>> + * reported and handled as usual.
>>> + */
>>> +void eeh_dev_release(struct pci_dev *pdev)
>>> +{
>>> + bool release_pe =3D true;
>>> + struct eeh_pe *pe =3D NULL;
>>> + struct eeh_dev *tmp, *edev;
>>> +
>>> + mutex_lock(&eeh_dev_mutex);
>>> +
>>> + /* No PCI device ? */
>>> + if (!pdev) {
>>> + mutex_unlock(&eeh_dev_mutex);
>>> + return;
>>> + }
>>> +
>>> + /* No EEH device ? */
>>> + edev =3D pci_dev_to_eeh_dev(pdev);
>>> + if (!edev || !eeh_dev_passed(edev) ||
>>> + !edev->pe || !eeh_pe_passed(pe)) {
>>> + mutex_unlock(&eeh_dev_mutex);
>>> + return;
>>> + }
>>> +
>>> + /* Release device */
>>> + pe =3D edev->pe;
>>> + eeh_dev_set_passed(edev, false);
>>> +
>>> + /* Release PE */
>>> + eeh_pe_for_each_dev(pe, edev, tmp) {
>>> + if (eeh_dev_passed(edev)) {
>>> + release_pe =3D false;
>>> + break;
>>> + }
>>> + }
>>> +
>>> + if (release_pe)
>>> + eeh_pe_set_passed(pe, false);
>>> +
>>> + mutex_unlock(&eeh_dev_mutex);
>>> +}
>>> +EXPORT_SYMBOL(eeh_dev_release);
>>> +
>>> +static int eeh_dev_check(struct pci_dev *pdev,
>>> + struct eeh_dev **pedev,
>>> + struct eeh_pe **ppe)
>>> +{
>>> + struct eeh_dev *edev;
>>> +
>>> + /* No device ? */
>>> + if (!pdev)
>>> + return -ENODEV;
>>> +
>>> + edev =3D pci_dev_to_eeh_dev(pdev);
>>> + if (!edev || !eeh_dev_passed(edev) ||
>>> + !edev->pe || !eeh_pe_passed(edev->pe))
>>> + return -ENODEV;
>>> +
>>> + if (pedev)
>>> + *pedev =3D edev;
>>> + if (ppe)
>>> + *ppe =3D edev->pe;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +/**
>>> + * eeh_pe_set_option - Set options for the indicated PE
>>> + * @pdev: PCI device
>>> + * @option: requested option
>>> + *
>>> + * The routine is called to enable or disable EEH functionality
>>> + * on the indicated PE, to enable IO or DMA for the frozen PE.
>>> + */
>>> +int eeh_pe_set_option(struct pci_dev *pdev, int option)
>>> +{
>>> + struct eeh_dev *edev;
>>> + struct eeh_pe *pe;
>>> + int ret =3D 0;
>>> +
>>> + /* Device existing ? */
>>> + ret =3D eeh_dev_check(pdev, &edev, &pe);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + switch (option) {
>>> + case EEH_OPT_DISABLE:
>>> + case EEH_OPT_ENABLE:
>>> + break;
>>> + case EEH_OPT_THAW_MMIO:
>>> + case EEH_OPT_THAW_DMA:
>>> + if (!eeh_ops || !eeh_ops->set_option) {
>>> + ret =3D -ENOENT;
>>> + break;
>>> + }
>>> +
>>> + ret =3D eeh_ops->set_option(pe, option);
>>> + break;
>>> + default:
>>> + pr_debug("%s: Option %d out of range (%d, %d)\n",
>>> + __func__, option, EEH_OPT_DISABLE, EEH_OPT_THAW_DMA);
>>> + ret =3D -EINVAL;
>>> + }
>>> +
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(eeh_pe_set_option);
>>> +
>>> +/**
>>> + * eeh_pe_get_addr - Retrieve the PE address or sharing mode
>>> + * @pdev: PCI device
>>> + * @option: option
>>> + *
>>> + * Retrieve the PE address or sharing mode.
>>> + */
>>> +int eeh_pe_get_addr(struct pci_dev *pdev, int option)
>>> +{
>>> + struct pci_bus *bus;
>>> + struct eeh_dev *edev;
>>> + struct eeh_pe *pe;
>>> + int ret =3D 0;
>>> +
>>> + /* Device existing ? */
>>> + ret =3D eeh_dev_check(pdev, &edev, &pe);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + switch (option) {
>>> + case EEH_OPT_GET_PE_ADDR:
>>> + bus =3D eeh_pe_bus_get(pe);
>>> + if (!bus) {
>>> + ret =3D -ENODEV;
>>> + break;
>>> + }
>>> +
>>> + /* PE address has format "00BBSS00" */
>>> + ret =3D bus->number << 16;
>>> + break;
>>> + case EEH_OPT_GET_PE_MODE:
>>> + /* Wa always have shared PE */
>>> + ret =3D EEH_PE_MODE_SHARED;
>>> + break;
>>> + default:
>>> + pr_debug("%s: option %d out of range (0, 1)\n",
>>> + __func__, option);
>>> + ret =3D -EINVAL;
>>> + }
>>> +
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(eeh_pe_get_addr);
>>> +
>>> +/**
>>> + * eeh_pe_get_state - Retrieve PE's state
>>> + * @pdev: PCI device
>>> + *
>>> + * Retrieve the PE's state, which includes 3 aspects: enabled
>>> + * DMA, enabled IO and asserted reset.
>>> + */
>>> +int eeh_pe_get_state(struct pci_dev *pdev)
>>> +{
>>> + struct eeh_dev *edev;
>>> + struct eeh_pe *pe;
>>> + int result, ret =3D 0;
>>> +
>>> + /* Device existing ? */
>>> + ret =3D eeh_dev_check(pdev, &edev, &pe);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + if (!eeh_ops || !eeh_ops->get_state)
>>> + return -ENOENT;
>>> +
>>> + result =3D eeh_ops->get_state(pe, NULL);
>>> + if (!(result & EEH_STATE_RESET_ACTIVE) &&
>>> + (result & EEH_STATE_DMA_ENABLED) &&
>>> + (result & EEH_STATE_MMIO_ENABLED))
>>> + ret =3D EEH_PE_STATE_NORMAL;
>>> + else if (result & EEH_STATE_RESET_ACTIVE)
>>> + ret =3D EEH_PE_STATE_RESET;
>>> + else if (!(result & EEH_STATE_RESET_ACTIVE) &&
>>> + !(result & EEH_STATE_DMA_ENABLED) &&
>>> + !(result & EEH_STATE_MMIO_ENABLED))
>>> + ret =3D EEH_PE_STATE_STOPPED_IO_DMA;
>>> + else if (!(result & EEH_STATE_RESET_ACTIVE) &&
>>> + (result & EEH_STATE_DMA_ENABLED) &&
>>> + !(result & EEH_STATE_MMIO_ENABLED))
>>> + ret =3D EEH_PE_STATE_STOPPED_DMA;
>>> + else
>>> + ret =3D EEH_PE_STATE_UNAVAIL;
>>> +
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(eeh_pe_get_state);
>>> +
>>> +/**
>>> + * eeh_pe_reset - Issue PE reset according to specified type
>>> + * @pdev: PCI device
>>> + * @option: reset type
>>> + *
>>> + * The routine is called to reset the specified PE with the
>>> + * indicated type, either fundamental reset or hot reset.
>>> + * PE reset is the most important part for error recovery.
>>> + */
>>> +int eeh_pe_reset(struct pci_dev *pdev, int option)
>>> +{
>>> + struct eeh_dev *edev;
>>> + struct eeh_pe *pe;
>>> + int ret =3D 0;
>>> +
>>> + /* Device existing ? */
>>> + ret =3D eeh_dev_check(pdev, &edev, &pe);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + if (!eeh_ops || !eeh_ops->set_option || !eeh_ops->reset)
>>> + return -ENOENT;
>>> +
>>> + switch (option) {
>>> + case EEH_RESET_DEACTIVATE:
>>> + ret =3D eeh_ops->reset(pe, option);
>>> + if (ret)
>>> + break;
>>> +
>>> + /*
>>> + * The PE is still in frozen state and we need clear
>>> + * that. It's good to clear frozen state after deassert
>>> + * to avoid messy IO access during reset, which might
>>> + * cause recursive frozen PE.
>>> + */
>>> + ret =3D eeh_ops->set_option(pe, EEH_OPT_THAW_MMIO);
>>> + if (!ret)
>>> + ret =3D eeh_ops->set_option(pe, EEH_OPT_THAW_DMA);
>>> + if (!ret)
>>> + eeh_pe_state_clear(pe, EEH_PE_ISOLATED);
>>> + break;
>>> + case EEH_RESET_HOT:
>>> + case EEH_RESET_FUNDAMENTAL:
>>> + ret =3D eeh_ops->reset(pe, option);
>>> + break;
>>> + default:
>>> + pr_debug("%s: Unsupported option %d\n",
>>> + __func__, option);
>>> + ret =3D -EINVAL;
>>> + }
>>> +
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(eeh_pe_reset);
>>> +
>>> +/**
>>> + * eeh_pe_configure - Configure PCI bridges after PE reset
>>> + * @pdev: PCI device
>>> + *
>>> + * The routine is called to restore the PCI config space for
>>> + * those PCI devices, especially PCI bridges affected by PE
>>> + * reset issued previously.
>>> + */
>>> +int eeh_pe_configure(struct pci_dev *pdev)
>>> +{
>>> + struct eeh_dev *edev;
>>> + struct eeh_pe *pe;
>>> + int ret =3D 0;
>>> +
>>> + /* Device existing ? */
>>> + ret =3D eeh_dev_check(pdev, &edev, &pe);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + /* Restore config space for the affected devices */
>>> + eeh_pe_restore_bars(pe);
>>> +
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(eeh_pe_configure);
>>> +
>>> static int proc_eeh_show(struct seq_file *m, void *v)
>>> {
>>> if (!eeh_enabled()) {
>>> diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
>>> index 7ba0424..301ac18 100644
>>> --- a/drivers/vfio/pci/vfio_pci.c
>>> +++ b/drivers/vfio/pci/vfio_pci.c
>>> @@ -25,6 +25,9 @@
>>> #include <linux/types.h>
>>> #include <linux/uaccess.h>
>>> #include <linux/vfio.h>
>>> +#ifdef CONFIG_EEH
>>> +#include <asm/eeh.h>
>>> +#endif
>>=20
>> Can we make a vfio_pci_eeh file that properly stubs out anything called
>> from common code? I don't want to see all these inline ifdefs.
>=20
> well. do you want see drivers/vfio/vfio-pci/vfio_pci_eeh.c and export
> following functins for vfio_pci.c to use?
>=20
> vfio_pci_eeh_open()
> vfio_pci_eeh_release()
> vfio_pci_eeh_ioctl()
>=20
>=20
>>>=20
>>> #include "vfio_pci_private.h"
>>>=20
>>> @@ -152,32 +155,57 @@ static void vfio_pci_disable(struct vfio_pci_devic=
e *vdev)
>>> pci_restore_state(pdev);
>>> }
>>>=20
>>> +static void vfio_eeh_pci_release(struct pci_dev *pdev)
>>> +{
>>> +#ifdef CONFIG_EEH
>>> + eeh_dev_release(pdev);
>>> +#endif
>>> +}
>>> +
>>> static void vfio_pci_release(void *device_data)
>>> {
>>> struct vfio_pci_device *vdev =3D device_data;
>>>=20
>>> - if (atomic_dec_and_test(&vdev->refcnt))
>>> + if (atomic_dec_and_test(&vdev->refcnt)) {
>>> + vfio_eeh_pci_release(vdev->pdev);
>>> vfio_pci_disable(vdev);
>>> + }
>>>=20
>>> module_put(THIS_MODULE);
>>> }
>>>=20
>>> +static int vfio_eeh_pci_open(struct pci_dev *pdev)
>>> +{
>>> + int ret =3D 0;
>>> +
>>> +#ifdef CONFIG_EEH
>>> + ret =3D eeh_dev_open(pdev);
>>> +#endif
>>> + return ret;
>>> +}
>>> +
>>> static int vfio_pci_open(void *device_data)
>>> {
>>> struct vfio_pci_device *vdev =3D device_data;
>>> + int ret;
>>>=20
>>> if (!try_module_get(THIS_MODULE))
>>> return -ENODEV;
>>>=20
>>> if (atomic_inc_return(&vdev->refcnt) =3D=3D 1) {
>>> - int ret =3D vfio_pci_enable(vdev);
>>> - if (ret) {
>>> - module_put(THIS_MODULE);
>>> - return ret;
>>> - }
>>> + ret =3D vfio_pci_enable(vdev);
>>> + if (ret)
>>> + goto error;
>>> +
>>> + ret =3D vfio_eeh_pci_open(vdev->pdev);
>>> + if (ret)
>>> + goto error;
>>> }
>>>=20
>>> return 0;
>>> +error:
>>> + module_put(THIS_MODULE);
>>> + return ret;
>>> }
>>>=20
>>> static int vfio_pci_get_irq_count(struct vfio_pci_device *vdev, int irq_=
type)
>>> @@ -321,6 +349,91 @@ static int vfio_pci_for_each_slot_or_bus(struct pci=
_dev *pdev,
>>> return walk.ret;
>>> }
>>>=20
>>> +static int vfio_eeh_pci_ioctl(struct pci_dev *pdev,
>>> + unsigned int cmd,
>>> + unsigned long arg)
>>> +{
>>> + unsigned long minsz;
>>> + int ret =3D 0;
>>> +
>>> +#ifdef CONFIG_EEH
>>> + switch (cmd) {
>>> + case VFIO_EEH_PE_SET_OPTION: {
>>> + struct vfio_eeh_pe_set_option option;
>>> +
>>> + minsz =3D offsetofend(struct vfio_eeh_pe_set_option, option);
>>> + if (copy_from_user(&option, (void __user *)arg, minsz))
>>> + return -EFAULT;
>>> + if (option.argsz < minsz)
>>> + return -EINVAL;
>>> +
>>> + ret =3D eeh_pe_set_option(pdev, option.option);
>>> + break;
>>> + }
>>> + case VFIO_EEH_PE_GET_ADDR: {
>>> + struct vfio_eeh_pe_get_addr addr;
>>> +
>>> + minsz =3D offsetofend(struct vfio_eeh_pe_get_addr, info);
>>> + if (copy_from_user(&addr, (void __user *)arg, minsz))
>>> + return -EFAULT;
>>> + if (addr.argsz < minsz)
>>> + return -EINVAL;
>>> +
>>> + ret =3D eeh_pe_get_addr(pdev, addr.option);
>>> + if (ret >=3D 0) {
>>> + addr.info =3D ret;
>>> + if (copy_to_user((void __user *)arg, &addr, minsz))
>>> + return -EFAULT;
>>> + ret =3D 0;
>>> + }
>>> +
>>> + break;
>>> + }
>>> + case VFIO_EEH_PE_GET_STATE: {
>>> + struct vfio_eeh_pe_get_state state;
>>> +
>>> + minsz =3D offsetofend(struct vfio_eeh_pe_get_state, state);
>>> + if (copy_from_user(&state, (void __user *)arg, minsz))
>>> + return -EFAULT;
>>> + if (state.argsz < minsz)
>>> + return -EINVAL;
>>> +
>>> + ret =3D eeh_pe_get_state(pdev);
>>> + if (ret >=3D 0) {
>>> + state.state =3D ret;
>>> + if (copy_to_user((void __user *)arg, &state, minsz))
>>> + return -EFAULT;
>>> + ret =3D 0;
>>> + }
>>> + break;
>>> + }
>>> + case VFIO_EEH_PE_RESET: {
>>> + struct vfio_eeh_pe_reset reset;
>>> +
>>> + minsz =3D offsetofend(struct vfio_eeh_pe_reset, option);
>>> + if (copy_from_user(&reset, (void __user *)arg, minsz))
>>> + return -EFAULT;
>>> + if (reset.argsz < minsz)
>>> + return -EINVAL;
>>> +
>>> + ret =3D eeh_pe_reset(pdev, reset.option);
>>> + break;
>>> + }
>>> + case VFIO_EEH_PE_CONFIGURE:
>>> + ret =3D eeh_pe_configure(pdev);
>>> + break;
>>> + default:
>>> + ret =3D -EINVAL;
>>> + pr_debug("%s: Cannot handle command %d\n",
>>> + __func__, cmd);
>>> + }
>>> +#else
>>> + ret =3D -ENOENT;
>>> +#endif
>>> +
>>> + return ret;
>>> +}
>>> +
>>> static long vfio_pci_ioctl(void *device_data,
>>> unsigned int cmd, unsigned long arg)
>>> {
>>> @@ -682,6 +795,12 @@ hot_reset_release:
>>>=20
>>> kfree(groups);
>>> return ret;
>>> + } else if (cmd =3D=3D VFIO_EEH_PE_SET_OPTION ||
>>> + cmd =3D=3D VFIO_EEH_PE_GET_ADDR ||
>>> + cmd =3D=3D VFIO_EEH_PE_GET_STATE ||
>>> + cmd =3D=3D VFIO_EEH_PE_RESET ||
>>> + cmd =3D=3D VFIO_EEH_PE_CONFIGURE) {
>>> + return vfio_eeh_pci_ioctl(vdev->pdev, cmd, arg);
>>> }
>>>=20
>>> return -ENOTTY;
>>> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
>>> index cb9023d..ef55682 100644
>>> --- a/include/uapi/linux/vfio.h
>>> +++ b/include/uapi/linux/vfio.h
>>> @@ -455,6 +455,59 @@ struct vfio_iommu_spapr_tce_info {
>>>=20
>>> #define VFIO_IOMMU_SPAPR_TCE_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
>>>=20
>>> +/*
>>> + * EEH functionality can be enabled or disabled on one specific device.=
>>> + * Also, the DMA or IO frozen state can be removed from the frozen PE
>>> + * if required.
>>> + */
>>> +struct vfio_eeh_pe_set_option {
>>> + __u32 argsz;
>>> + __u32 option;
>>> +};
>>> +
>>> +#define VFIO_EEH_PE_SET_OPTION _IO(VFIO_TYPE, VFIO_BASE + 21)
>>=20
>> What proposed ioctls are making you jump to 21?
>>=20
>> argsz is probably not as useful without a flags field. Otherwise the
>> caller can't indicate what the extra space is.
>=20
> The QEMU patches are based on Alexey's additional feature ("ddw"), which
> consumed several ioctl commands.
>=20
> ok. So you also prefer to remove "argsz"?
>=20
>>> +
>>> +/*
>>> + * Each EEH PE should have unique address to be identified. The command=
>>> + * helps to retrieve the address and the sharing mode of the PE.
>>> + */
>>> +struct vfio_eeh_pe_get_addr {
>>> + __u32 argsz;
>>> + __u32 option;
>>> + __u32 info;
>>> +};
>>> +
>>> +#define VFIO_EEH_PE_GET_ADDR _IO(VFIO_TYPE, VFIO_BASE + 22)
>>> +
>>> +/*
>>> + * EEH PE might have been frozen because of PCI errors. Also, it might
>>> + * be experiencing reset for error revoery. The following command helps=
>>> + * to get the state.
>>> + */
>>> +struct vfio_eeh_pe_get_state {
>>> + __u32 argsz;
>>> + __u32 state;
>>> +};
>>> +
>>> +#define VFIO_EEH_PE_GET_STATE _IO(VFIO_TYPE, VFIO_BASE + 23)
>>> +
>>> +/*
>>> + * Reset is the major step to recover problematic PE. The following
>>> + * command helps on that.
>>> + */
>>> +struct vfio_eeh_pe_reset {
>>> + __u32 argsz;
>>> + __u32 option;
>>> +};
>>> +
>>> +#define VFIO_EEH_PE_RESET _IO(VFIO_TYPE, VFIO_BASE + 24)
>>> +
>>> +/*
>>> + * One of the steps for recovery after PE reset is to configure the
>>> + * PCI bridges affected by the PE reset.
>>> + */
>>> +#define VFIO_EEH_PE_CONFIGURE _IO(VFIO_TYPE, VFIO_BASE + 25)
>>=20
>> What can the user do differently by making these separate ioctls?
>=20
> hrm, I didn't understood as well. Alex.G could have the explaination.
Alex raised the same concern as me: why separate reset and configure? When w=
e want to recover a device, we need a reset call anyway, right?
Alex
>=20
>>> +
>>> /* ***************************************************************** */
>>>=20
>>> #endif /* _UAPIVFIO_H */
>=20
> Thanks,
> Gavin
>=20
^ permalink raw reply
* Re: powerpc: remove checks for CONFIG_BOOK3E_MMU_TLB_STATS
From: Paul Bolle @ 2014-05-23 7:33 UTC (permalink / raw)
To: Scott Wood; +Cc: Paul Mackerras, linuxppc-dev, linux-kernel
In-Reply-To: <20140522223758.GA20343@home.buserror.net>
Scott,
On Thu, 2014-05-22 at 17:37 -0500, Scott Wood wrote:
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S: Assembler messages:
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:89: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:238: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:269: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:281: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:441: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:510: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:881: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:918: Error: unrecognized opcode: `tlb_miss_prolog_stats'
Thanks for testing!
That's a bit surprising. The patch is intended to be a non event. Ie, it
only removes what the preprocessor would have removed anyway. Unless I
botched it, of course.
What exactly did you test there?
Paul Bolle
^ permalink raw reply
* Re: [PATCH v6 2/3] drivers/vfio: EEH support for VFIO PCI device
From: Gavin Shan @ 2014-05-23 7:37 UTC (permalink / raw)
To: Alexander Graf
Cc: aik@ozlabs.ru, Gavin Shan, kvm-ppc@vger.kernel.org,
Alex Williamson, qiudayu@linux.vnet.ibm.com,
linuxppc-dev@lists.ozlabs.org
In-Reply-To: <76F750C7-EAA1-455F-A64F-BAB826F66281@suse.de>
On Fri, May 23, 2014 at 08:55:15AM +0200, Alexander Graf wrote:
>> Am 23.05.2014 um 06:37 schrieb Gavin Shan <gwshan@linux.vnet.ibm.com>:
>>> On Thu, May 22, 2014 at 09:10:53PM -0600, Alex Williamson wrote:
>>>> On Thu, 2014-05-22 at 18:23 +1000, Gavin Shan wrote:
>>>> The patch adds new IOCTL commands for VFIO PCI device to support
>>>> EEH functionality for PCI devices, which have been passed through
>>>> from host to somebody else via VFIO.
.../...
>>>> +
>>>> +/*
>>>> + * Reset is the major step to recover problematic PE. The following
>>>> + * command helps on that.
>>>> + */
>>>> +struct vfio_eeh_pe_reset {
>>>> + __u32 argsz;
>>>> + __u32 option;
>>>> +};
>>>> +
>>>> +#define VFIO_EEH_PE_RESET _IO(VFIO_TYPE, VFIO_BASE + 24)
>>>> +
>>>> +/*
>>>> + * One of the steps for recovery after PE reset is to configure the
>>>> + * PCI bridges affected by the PE reset.
>>>> + */
>>>> +#define VFIO_EEH_PE_CONFIGURE _IO(VFIO_TYPE, VFIO_BASE + 25)
>>>
>>> What can the user do differently by making these separate ioctls?
>>
>> hrm, I didn't understood as well. Alex.G could have the explaination.
>
>Alex raised the same concern as me: why separate reset and configure? When we want to recover a device, we need a reset call anyway, right?
>
Ok. With current ioctl commands, "reset+configure" is required to do
error recovery. Before the recovery, we also need call "configure"
in order to retrieve error log correctly.
Also, they corresponds to 2 separate RTAS services: "ibm,set-slot-reset"
and "ibm,configure-pe".
Thanks,
Gavin
^ permalink raw reply
* RE: [2/2] powerpc/corenet64_smp_defconfig: enable RTC support
From: Shengzhou.Liu @ 2014-05-23 8:03 UTC (permalink / raw)
To: Scott Wood; +Cc: linuxppc-dev@lists.ozlabs.org
In-Reply-To: <20140522225144.GA1670@home.buserror.net>
> -----Original Message-----
> From: Wood Scott-B07421
> Sent: Friday, May 23, 2014 6:52 AM
> To: Liu Shengzhou-B36685
> Cc: linuxppc-dev@lists.ozlabs.org
> Subject: Re: [2/2] powerpc/corenet64_smp_defconfig: enable RTC support
>=20
> > +++ b/arch/powerpc/configs/corenet64_smp_defconfig
> > @@ -125,6 +125,11 @@ CONFIG_USB_EHCI_FSL=3Dy CONFIG_USB_STORAGE=3Dy
> > CONFIG_MMC=3Dy CONFIG_MMC_SDHCI=3Dy
> > +CONFIG_RTC_CLASS=3Dy
> > +CONFIG_RTC_DRV_CMOS=3Dy
> > +CONFIG_RTC_DRV_DS1307=3Dy
> > +CONFIG_RTC_DRV_DS1374=3Dy
> > +CONFIG_RTC_DRV_DS3232=3Dy
> > CONFIG_EDAC=3Dy
> > CONFIG_EDAC_MM_EDAC=3Dy
> > CONFIG_DMADEVICES=3Dy
>=20
> Why only corenet64 and not corenet32?
>=20
> -Scott
[Shengzhou] There is already RTC support in corenet32, only missing in core=
net64.
^ permalink raw reply
* [PATCH v2 0/6] Implement split core for POWER8
From: Michael Neuling @ 2014-05-23 8:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexander Graf
Cc: linuxppc-dev, Paul Mackerras, kvm-ppc, kvm
This patch series implements split core mode on POWER8. This enables up to 4
subcores per core which can each independently run guests (per guest SPRs like
SDR1, LPIDR etc are replicated per subcore). Lots more documentation on this
feature in the code and commit messages.
Most of this code is in the powernv platform but there's a couple of KVM
specific patches too.
Patch series authored by mpe and me with a few bug fixes from others.
v2:
There are some minor updates based on comments and I've added the Acks by
Paulus and Alex for the KVM code.
^ permalink raw reply
* [PATCH v2 1/6] KVM: PPC: Book3S HV: Rework the secondary inhibit code
From: Michael Neuling @ 2014-05-23 8:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexander Graf
Cc: Michael Neuling, kvm, kvm-ppc, Paul Mackerras, linuxppc-dev
In-Reply-To: <1400832930-17050-1-git-send-email-mikey@neuling.org>
From: Michael Ellerman <mpe@ellerman.id.au>
As part of the support for split core on POWER8, we want to be able to
block splitting of the core while KVM VMs are active.
The logic to do that would be exactly the same as the code we currently
have for inhibiting onlining of secondaries.
Instead of adding an identical mechanism to block split core, rework the
secondary inhibit code to be a "HV KVM is active" check. We can then use
that in both the cpu hotplug code and the upcoming split core code.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Michael Neuling <mikey@neuling.org>
Acked-by: Alexander Graf <agraf@suse.de>
Acked-by: Paul Mackerras <paulus@samba.org>
---
arch/powerpc/include/asm/kvm_ppc.h | 7 +++++++
arch/powerpc/include/asm/smp.h | 8 --------
arch/powerpc/kernel/smp.c | 34 +++-------------------------------
arch/powerpc/kvm/book3s_hv.c | 8 ++++----
arch/powerpc/kvm/book3s_hv_builtin.c | 31 +++++++++++++++++++++++++++++++
5 files changed, 45 insertions(+), 43 deletions(-)
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 4096f16..2c8e399 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -337,6 +337,10 @@ static inline void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu)
vcpu->kvm->arch.kvm_ops->fast_vcpu_kick(vcpu);
}
+extern void kvm_hv_vm_activated(void);
+extern void kvm_hv_vm_deactivated(void);
+extern bool kvm_hv_mode_active(void);
+
#else
static inline void __init kvm_cma_reserve(void)
{}
@@ -356,6 +360,9 @@ static inline void kvmppc_fast_vcpu_kick(struct kvm_vcpu *vcpu)
{
kvm_vcpu_kick(vcpu);
}
+
+static inline bool kvm_hv_mode_active(void) { return false; }
+
#endif
#ifdef CONFIG_KVM_XICS
diff --git a/arch/powerpc/include/asm/smp.h b/arch/powerpc/include/asm/smp.h
index ff51046..5a6614a 100644
--- a/arch/powerpc/include/asm/smp.h
+++ b/arch/powerpc/include/asm/smp.h
@@ -68,14 +68,6 @@ void generic_mach_cpu_die(void);
void generic_set_cpu_dead(unsigned int cpu);
void generic_set_cpu_up(unsigned int cpu);
int generic_check_cpu_restart(unsigned int cpu);
-
-extern void inhibit_secondary_onlining(void);
-extern void uninhibit_secondary_onlining(void);
-
-#else /* HOTPLUG_CPU */
-static inline void inhibit_secondary_onlining(void) {}
-static inline void uninhibit_secondary_onlining(void) {}
-
#endif
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index e2a4232..6edae3d 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -36,6 +36,7 @@
#include <linux/atomic.h>
#include <asm/irq.h>
#include <asm/hw_irq.h>
+#include <asm/kvm_ppc.h>
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/prom.h>
@@ -457,38 +458,9 @@ int generic_check_cpu_restart(unsigned int cpu)
return per_cpu(cpu_state, cpu) == CPU_UP_PREPARE;
}
-static atomic_t secondary_inhibit_count;
-
-/*
- * Don't allow secondary CPU threads to come online
- */
-void inhibit_secondary_onlining(void)
-{
- /*
- * This makes secondary_inhibit_count stable during cpu
- * online/offline operations.
- */
- get_online_cpus();
-
- atomic_inc(&secondary_inhibit_count);
- put_online_cpus();
-}
-EXPORT_SYMBOL_GPL(inhibit_secondary_onlining);
-
-/*
- * Allow secondary CPU threads to come online again
- */
-void uninhibit_secondary_onlining(void)
-{
- get_online_cpus();
- atomic_dec(&secondary_inhibit_count);
- put_online_cpus();
-}
-EXPORT_SYMBOL_GPL(uninhibit_secondary_onlining);
-
-static int secondaries_inhibited(void)
+static bool secondaries_inhibited(void)
{
- return atomic_read(&secondary_inhibit_count);
+ return kvm_hv_mode_active();
}
#else /* HOTPLUG_CPU */
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 8227dba..d7b74f8 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -2317,10 +2317,10 @@ static int kvmppc_core_init_vm_hv(struct kvm *kvm)
spin_lock_init(&kvm->arch.slot_phys_lock);
/*
- * Don't allow secondary CPU threads to come online
- * while any KVM VMs exist.
+ * Track that we now have a HV mode VM active. This blocks secondary
+ * CPU threads from coming online.
*/
- inhibit_secondary_onlining();
+ kvm_hv_vm_activated();
return 0;
}
@@ -2336,7 +2336,7 @@ static void kvmppc_free_vcores(struct kvm *kvm)
static void kvmppc_core_destroy_vm_hv(struct kvm *kvm)
{
- uninhibit_secondary_onlining();
+ kvm_hv_vm_deactivated();
kvmppc_free_vcores(kvm);
if (kvm->arch.rma) {
diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
index 8cd0dae..7cde8a6 100644
--- a/arch/powerpc/kvm/book3s_hv_builtin.c
+++ b/arch/powerpc/kvm/book3s_hv_builtin.c
@@ -6,6 +6,7 @@
* published by the Free Software Foundation.
*/
+#include <linux/cpu.h>
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/export.h>
@@ -181,3 +182,33 @@ void __init kvm_cma_reserve(void)
kvm_cma_declare_contiguous(selected_size, align_size);
}
}
+
+/*
+ * When running HV mode KVM we need to block certain operations while KVM VMs
+ * exist in the system. We use a counter of VMs to track this.
+ *
+ * One of the operations we need to block is onlining of secondaries, so we
+ * protect hv_vm_count with get/put_online_cpus().
+ */
+static atomic_t hv_vm_count;
+
+void kvm_hv_vm_activated(void)
+{
+ get_online_cpus();
+ atomic_inc(&hv_vm_count);
+ put_online_cpus();
+}
+EXPORT_SYMBOL_GPL(kvm_hv_vm_activated);
+
+void kvm_hv_vm_deactivated(void)
+{
+ get_online_cpus();
+ atomic_dec(&hv_vm_count);
+ put_online_cpus();
+}
+EXPORT_SYMBOL_GPL(kvm_hv_vm_deactivated);
+
+bool kvm_hv_mode_active(void)
+{
+ return atomic_read(&hv_vm_count) != 0;
+}
--
1.9.1
^ permalink raw reply related
* [PATCH v2 2/6] powerpc/powernv: Make it possible to skip the IRQHAPPENED check in power7_nap()
From: Michael Neuling @ 2014-05-23 8:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexander Graf
Cc: Michael Neuling, kvm, kvm-ppc, Paul Mackerras, linuxppc-dev
In-Reply-To: <1400832930-17050-1-git-send-email-mikey@neuling.org>
From: Michael Ellerman <mpe@ellerman.id.au>
To support split core we need to be able to force all secondaries into
nap, so the core can detect they are idle and do an unsplit.
Currently power7_nap() will return without napping if there is an irq
pending. We want to ignore the pending irq and nap anyway, we will deal
with the interrupt later.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Michael Neuling <mikey@neuling.org>
---
arch/powerpc/include/asm/processor.h | 2 +-
arch/powerpc/kernel/idle_power7.S | 9 +++++++++
arch/powerpc/platforms/powernv/smp.c | 2 +-
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
index d660dc3..6d59072 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -449,7 +449,7 @@ extern unsigned long cpuidle_disable;
enum idle_boot_override {IDLE_NO_OVERRIDE = 0, IDLE_POWERSAVE_OFF};
extern int powersave_nap; /* set if nap mode can be used in idle loop */
-extern void power7_nap(void);
+extern void power7_nap(int check_irq);
extern void power7_sleep(void);
extern void flush_instruction_cache(void);
extern void hard_reset_now(void);
diff --git a/arch/powerpc/kernel/idle_power7.S b/arch/powerpc/kernel/idle_power7.S
index dca6e16..2480256 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -39,6 +39,10 @@
* Pass requested state in r3:
* 0 - nap
* 1 - sleep
+ *
+ * To check IRQ_HAPPENED in r4
+ * 0 - don't check
+ * 1 - check
*/
_GLOBAL(power7_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -71,6 +75,8 @@ _GLOBAL(power7_powersave_common)
lbz r0,PACAIRQHAPPENED(r13)
cmpwi cr0,r0,0
beq 1f
+ cmpwi cr0,r4,0
+ beq 1f
addi r1,r1,INT_FRAME_SIZE
ld r0,16(r1)
mtlr r0
@@ -114,15 +120,18 @@ _GLOBAL(power7_idle)
lwz r4,ADDROFF(powersave_nap)(r3)
cmpwi 0,r4,0
beqlr
+ li r3, 1
/* fall through */
_GLOBAL(power7_nap)
+ mr r4,r3
li r3,0
b power7_powersave_common
/* No return */
_GLOBAL(power7_sleep)
li r3,1
+ li r4,0
b power7_powersave_common
/* No return */
diff --git a/arch/powerpc/platforms/powernv/smp.c b/arch/powerpc/platforms/powernv/smp.c
index 1601a1e..65faf99 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -159,7 +159,7 @@ static void pnv_smp_cpu_kill_self(void)
mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1);
while (!generic_check_cpu_restart(cpu)) {
ppc64_runlatch_off();
- power7_nap();
+ power7_nap(1);
ppc64_runlatch_on();
if (!generic_check_cpu_restart(cpu)) {
DBG("CPU%d Unexpected exit while offline !\n", cpu);
--
1.9.1
^ permalink raw reply related
* [PATCH v2 3/6] powerpc: Add threads_per_subcore
From: Michael Neuling @ 2014-05-23 8:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexander Graf
Cc: Michael Neuling, kvm, kvm-ppc, Paul Mackerras, linuxppc-dev
In-Reply-To: <1400832930-17050-1-git-send-email-mikey@neuling.org>
From: Michael Ellerman <mpe@ellerman.id.au>
On POWER8 we have a new concept of a subcore. This is what happens when
you take a regular core and split it. A subcore is a grouping of two or
four SMT threads, as well as a handfull of SPRs which allows the subcore
to appear as if it were a core from the point of view of a guest.
Unlike threads_per_core which is fixed at boot, threads_per_subcore can
change while the system is running. Most code will not want to use
threads_per_subcore.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Michael Neuling <mikey@neuling.org>
---
arch/powerpc/include/asm/cputhreads.h | 7 +++++++
arch/powerpc/kernel/setup-common.c | 4 +++-
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/cputhreads.h b/arch/powerpc/include/asm/cputhreads.h
index ac3eedb..2bf8e93 100644
--- a/arch/powerpc/include/asm/cputhreads.h
+++ b/arch/powerpc/include/asm/cputhreads.h
@@ -18,10 +18,12 @@
#ifdef CONFIG_SMP
extern int threads_per_core;
+extern int threads_per_subcore;
extern int threads_shift;
extern cpumask_t threads_core_mask;
#else
#define threads_per_core 1
+#define threads_per_subcore 1
#define threads_shift 0
#define threads_core_mask (CPU_MASK_CPU0)
#endif
@@ -74,6 +76,11 @@ static inline int cpu_thread_in_core(int cpu)
return cpu & (threads_per_core - 1);
}
+static inline int cpu_thread_in_subcore(int cpu)
+{
+ return cpu & (threads_per_subcore - 1);
+}
+
static inline int cpu_first_thread_sibling(int cpu)
{
return cpu & ~(threads_per_core - 1);
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index 3cf25c8..aa0f5ed 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -390,9 +390,10 @@ void __init check_for_initrd(void)
#ifdef CONFIG_SMP
-int threads_per_core, threads_shift;
+int threads_per_core, threads_per_subcore, threads_shift;
cpumask_t threads_core_mask;
EXPORT_SYMBOL_GPL(threads_per_core);
+EXPORT_SYMBOL_GPL(threads_per_subcore);
EXPORT_SYMBOL_GPL(threads_shift);
EXPORT_SYMBOL_GPL(threads_core_mask);
@@ -401,6 +402,7 @@ static void __init cpu_init_thread_core_maps(int tpc)
int i;
threads_per_core = tpc;
+ threads_per_subcore = tpc;
cpumask_clear(&threads_core_mask);
/* This implementation only supports power of 2 number of threads
--
1.9.1
^ permalink raw reply related
* [PATCH v2 4/6] powerpc: Check cpu_thread_in_subcore() in __cpu_up()
From: Michael Neuling @ 2014-05-23 8:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexander Graf
Cc: Michael Neuling, kvm, kvm-ppc, Paul Mackerras, linuxppc-dev
In-Reply-To: <1400832930-17050-1-git-send-email-mikey@neuling.org>
From: Michael Ellerman <mpe@ellerman.id.au>
To support split core we need to change the check in __cpu_up() that
determines if a cpu is allowed to come online.
Currently we refuse to online cpus which are not the primary thread
within their core.
On POWER8 with split core support this check needs to instead refuse to
online cpus which are not the primary thread within their *sub* core.
On POWER7 and other systems that do not support split core,
threads_per_subcore == threads_per_core and so the check is equivalent.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Michael Neuling <mikey@neuling.org>
---
arch/powerpc/kernel/smp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 6edae3d..b5222c4 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -489,7 +489,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *tidle)
* Don't allow secondary threads to come online if inhibited
*/
if (threads_per_core > 1 && secondaries_inhibited() &&
- cpu % threads_per_core != 0)
+ cpu_thread_in_subcore(cpu))
return -EBUSY;
if (smp_ops == NULL ||
--
1.9.1
^ permalink raw reply related
* [PATCH v2 5/6] KVM: PPC: Book3S HV: Use threads_per_subcore in KVM
From: Michael Neuling @ 2014-05-23 8:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexander Graf
Cc: Michael Neuling, kvm, kvm-ppc, Paul Mackerras, linuxppc-dev
In-Reply-To: <1400832930-17050-1-git-send-email-mikey@neuling.org>
From: Michael Ellerman <mpe@ellerman.id.au>
To support split core on POWER8 we need to modify various parts of the
KVM code to use threads_per_subcore instead of threads_per_core. On
systems that do not support split core threads_per_subcore ==
threads_per_core and these changes are a nop.
We use threads_per_subcore as the value reported by KVM_CAP_PPC_SMT.
This communicates to userspace that guests can only be created with
a value of threads_per_core that is less than or equal to the current
threads_per_subcore. This ensures that guests can only be created with a
thread configuration that we are able to run given the current split
core mode.
Although threads_per_subcore can change during the life of the system,
the commit that enables that will ensure that threads_per_subcore does
not change during the life of a KVM VM.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Michael Neuling <mikey@neuling.org>
Acked-by: Alexander Graf <agraf@suse.de>
Acked-by: Paul Mackerras <paulus@samba.org>
---
arch/powerpc/kvm/book3s_hv.c | 26 ++++++++++++++++----------
arch/powerpc/kvm/powerpc.c | 2 +-
2 files changed, 17 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index d7b74f8..5e86f28 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -1266,7 +1266,7 @@ static struct kvm_vcpu *kvmppc_core_vcpu_create_hv(struct kvm *kvm,
int core;
struct kvmppc_vcore *vcore;
- core = id / threads_per_core;
+ core = id / threads_per_subcore;
if (core >= KVM_MAX_VCORES)
goto out;
@@ -1305,7 +1305,7 @@ static struct kvm_vcpu *kvmppc_core_vcpu_create_hv(struct kvm *kvm,
init_waitqueue_head(&vcore->wq);
vcore->preempt_tb = TB_NIL;
vcore->lpcr = kvm->arch.lpcr;
- vcore->first_vcpuid = core * threads_per_core;
+ vcore->first_vcpuid = core * threads_per_subcore;
vcore->kvm = kvm;
}
kvm->arch.vcores[core] = vcore;
@@ -1495,16 +1495,19 @@ static void kvmppc_wait_for_nap(struct kvmppc_vcore *vc)
static int on_primary_thread(void)
{
int cpu = smp_processor_id();
- int thr = cpu_thread_in_core(cpu);
+ int thr;
- if (thr)
+ /* Are we on a primary subcore? */
+ if (cpu_thread_in_subcore(cpu))
return 0;
- while (++thr < threads_per_core)
+
+ thr = 0;
+ while (++thr < threads_per_subcore)
if (cpu_online(cpu + thr))
return 0;
/* Grab all hw threads so they can't go into the kernel */
- for (thr = 1; thr < threads_per_core; ++thr) {
+ for (thr = 1; thr < threads_per_subcore; ++thr) {
if (kvmppc_grab_hwthread(cpu + thr)) {
/* Couldn't grab one; let the others go */
do {
@@ -1563,15 +1566,18 @@ static void kvmppc_run_core(struct kvmppc_vcore *vc)
}
/*
- * Make sure we are running on thread 0, and that
- * secondary threads are offline.
+ * Make sure we are running on primary threads, and that secondary
+ * threads are offline. Also check if the number of threads in this
+ * guest are greater than the current system threads per guest.
*/
- if (threads_per_core > 1 && !on_primary_thread()) {
+ if ((threads_per_core > 1) &&
+ ((vc->num_threads > threads_per_subcore) || !on_primary_thread())) {
list_for_each_entry(vcpu, &vc->runnable_threads, arch.run_list)
vcpu->arch.ret = -EBUSY;
goto out;
}
+
vc->pcpu = smp_processor_id();
list_for_each_entry(vcpu, &vc->runnable_threads, arch.run_list) {
kvmppc_start_thread(vcpu);
@@ -1599,7 +1605,7 @@ static void kvmppc_run_core(struct kvmppc_vcore *vc)
/* wait for secondary threads to finish writing their state to memory */
if (vc->nap_count < vc->n_woken)
kvmppc_wait_for_nap(vc);
- for (i = 0; i < threads_per_core; ++i)
+ for (i = 0; i < threads_per_subcore; ++i)
kvmppc_release_hwthread(vc->pcpu + i);
/* prevent other vcpu threads from doing kvmppc_start_thread() now */
vc->vcore_state = VCORE_EXITING;
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 3cf541a..27919a8 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -384,7 +384,7 @@ int kvm_dev_ioctl_check_extension(long ext)
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
case KVM_CAP_PPC_SMT:
if (hv_enabled)
- r = threads_per_core;
+ r = threads_per_subcore;
else
r = 0;
break;
--
1.9.1
^ permalink raw reply related
* [PATCH v2 6/6] powerpc/powernv: Add support for POWER8 split core on powernv
From: Michael Neuling @ 2014-05-23 8:15 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexander Graf
Cc: Michael Neuling, kvm, kvm-ppc, Paul Mackerras, Srivatsa S. Bhat,
Mahesh Salgaonkar, linuxppc-dev
In-Reply-To: <1400832930-17050-1-git-send-email-mikey@neuling.org>
From: Michael Ellerman <mpe@ellerman.id.au>
Upcoming POWER8 chips support a concept called split core. This is where the
core can be split into subcores that although not full cores, are able to
appear as full cores to a guest.
The splitting & unsplitting procedure is mildly complicated, and explained at
length in the comments within the patch.
One notable detail is that when splitting or unsplitting we need to pull
offline cpus out of their offline state to do work as part of the procedure.
The interface for changing the split mode is via a sysfs file, eg:
$ echo 2 > /sys/devices/system/cpu/subcores_per_core
Currently supported values are '1', '2' and '4'. And indicate respectively that
the core should be unsplit, split in half, and split in quarters. These modes
correspond to threads_per_subcore of 8, 4 and 2.
We do not allow changing the split mode while KVM VMs are active. This is to
prevent the value changing while userspace is configuring the VM, and also to
prevent the mode being changed in such a way that existing guests are unable to
be run.
CPU hotplug fixes by Srivatsa. max_cpus fixes by Mahesh. cpuset fixes by
benh. Fix for irq race by paulus. The rest by mikey and mpe.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Michael Neuling <mikey@neuling.org>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
arch/powerpc/include/asm/reg.h | 9 +
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/powernv.h | 2 +
arch/powerpc/platforms/powernv/smp.c | 18 +-
arch/powerpc/platforms/powernv/subcore-asm.S | 95 +++++++
arch/powerpc/platforms/powernv/subcore.c | 392 +++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/subcore.h | 18 ++
7 files changed, 527 insertions(+), 9 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/subcore-asm.S
create mode 100644 arch/powerpc/platforms/powernv/subcore.c
create mode 100644 arch/powerpc/platforms/powernv/subcore.h
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index 29de015..2cd799b 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -225,6 +225,7 @@
#define CTRL_TE 0x00c00000 /* thread enable */
#define CTRL_RUNLATCH 0x1
#define SPRN_DAWR 0xB4
+#define SPRN_RPR 0xBA /* Relative Priority Register */
#define SPRN_CIABR 0xBB
#define CIABR_PRIV 0x3
#define CIABR_PRIV_USER 1
@@ -273,8 +274,10 @@
#define SPRN_HSRR1 0x13B /* Hypervisor Save/Restore 1 */
#define SPRN_IC 0x350 /* Virtual Instruction Count */
#define SPRN_VTB 0x351 /* Virtual Time Base */
+#define SPRN_LDBAR 0x352 /* LD Base Address Register */
#define SPRN_PMICR 0x354 /* Power Management Idle Control Reg */
#define SPRN_PMSR 0x355 /* Power Management Status Reg */
+#define SPRN_PMMAR 0x356 /* Power Management Memory Activity Register */
#define SPRN_PMCR 0x374 /* Power Management Control Register */
/* HFSCR and FSCR bit numbers are the same */
@@ -434,6 +437,12 @@
#define HID0_BTCD (1<<1) /* Branch target cache disable */
#define HID0_NOPDST (1<<1) /* No-op dst, dstt, etc. instr. */
#define HID0_NOPTI (1<<0) /* No-op dcbt and dcbst instr. */
+/* POWER8 HID0 bits */
+#define HID0_POWER8_4LPARMODE __MASK(61)
+#define HID0_POWER8_2LPARMODE __MASK(57)
+#define HID0_POWER8_1TO2LPAR __MASK(52)
+#define HID0_POWER8_1TO4LPAR __MASK(51)
+#define HID0_POWER8_DYNLPARDIS __MASK(48)
#define SPRN_HID1 0x3F1 /* Hardware Implementation Register 1 */
#ifdef CONFIG_6xx
diff --git a/arch/powerpc/platforms/powernv/Makefile b/arch/powerpc/platforms/powernv/Makefile
index 63cebb9..4ad0d34 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -1,7 +1,7 @@
obj-y += setup.o opal-takeover.o opal-wrappers.o opal.o opal-async.o
obj-y += opal-rtc.o opal-nvram.o opal-lpc.o opal-flash.o
obj-y += rng.o opal-elog.o opal-dump.o opal-sysparam.o opal-sensor.o
-obj-y += opal-msglog.o
+obj-y += opal-msglog.o subcore.o subcore-asm.o
obj-$(CONFIG_SMP) += smp.o
obj-$(CONFIG_PCI) += pci.o pci-p5ioc2.o pci-ioda.o
diff --git a/arch/powerpc/platforms/powernv/powernv.h b/arch/powerpc/platforms/powernv/powernv.h
index 0051e10..75501bf 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -25,4 +25,6 @@ static inline int pnv_pci_dma_set_mask(struct pci_dev *pdev, u64 dma_mask)
extern void pnv_lpc_init(void);
+bool cpu_core_split_required(void);
+
#endif /* _POWERNV_H */
diff --git a/arch/powerpc/platforms/powernv/smp.c b/arch/powerpc/platforms/powernv/smp.c
index 65faf99..0062a43 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -161,15 +161,17 @@ static void pnv_smp_cpu_kill_self(void)
ppc64_runlatch_off();
power7_nap(1);
ppc64_runlatch_on();
- if (!generic_check_cpu_restart(cpu)) {
+
+ /* Reenable IRQs briefly to clear the IPI that woke us */
+ local_irq_enable();
+ local_irq_disable();
+ mb();
+
+ if (cpu_core_split_required())
+ continue;
+
+ if (!generic_check_cpu_restart(cpu))
DBG("CPU%d Unexpected exit while offline !\n", cpu);
- /* We may be getting an IPI, so we re-enable
- * interrupts to process it, it will be ignored
- * since we aren't online (hopefully)
- */
- local_irq_enable();
- local_irq_disable();
- }
}
mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) | LPCR_PECE1);
DBG("CPU%d coming online...\n", cpu);
diff --git a/arch/powerpc/platforms/powernv/subcore-asm.S b/arch/powerpc/platforms/powernv/subcore-asm.S
new file mode 100644
index 0000000..39bb24a
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/subcore-asm.S
@@ -0,0 +1,95 @@
+/*
+ * Copyright 2013, Michael (Ellerman|Neuling), IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <asm/asm-offsets.h>
+#include <asm/ppc_asm.h>
+#include <asm/reg.h>
+
+#include "subcore.h"
+
+
+_GLOBAL(split_core_secondary_loop)
+ /*
+ * r3 = u8 *state, used throughout the routine
+ * r4 = temp
+ * r5 = temp
+ * ..
+ * r12 = MSR
+ */
+ mfmsr r12
+
+ /* Disable interrupts so SRR0/1 don't get trashed */
+ li r4,0
+ ori r4,r4,MSR_EE|MSR_SE|MSR_BE|MSR_RI
+ andc r4,r12,r4
+ sync
+ mtmsrd r4
+
+ /* Switch to real mode and leave interrupts off */
+ li r5, MSR_IR|MSR_DR
+ andc r5, r4, r5
+
+ LOAD_REG_ADDR(r4, real_mode)
+
+ mtspr SPRN_SRR0,r4
+ mtspr SPRN_SRR1,r5
+ rfid
+ b . /* prevent speculative execution */
+
+real_mode:
+ /* Grab values from unsplit SPRs */
+ mfspr r6, SPRN_LDBAR
+ mfspr r7, SPRN_PMMAR
+ mfspr r8, SPRN_PMCR
+ mfspr r9, SPRN_RPR
+ mfspr r10, SPRN_SDR1
+
+ /* Order reading the SPRs vs telling the primary we are ready to split */
+ sync
+
+ /* Tell thread 0 we are in real mode */
+ li r4, SYNC_STEP_REAL_MODE
+ stb r4, 0(r3)
+
+ li r5, (HID0_POWER8_4LPARMODE | HID0_POWER8_2LPARMODE)@highest
+ sldi r5, r5, 48
+
+ /* Loop until we see the split happen in HID0 */
+1: mfspr r4, SPRN_HID0
+ and. r4, r4, r5
+ beq 1b
+
+ /*
+ * We only need to initialise the below regs once for each subcore,
+ * but it's simpler and harmless to do it on each thread.
+ */
+
+ /* Make sure various SPRS have sane values */
+ li r4, 0
+ mtspr SPRN_LPID, r4
+ mtspr SPRN_PCR, r4
+ mtspr SPRN_HDEC, r4
+
+ /* Restore SPR values now we are split */
+ mtspr SPRN_LDBAR, r6
+ mtspr SPRN_PMMAR, r7
+ mtspr SPRN_PMCR, r8
+ mtspr SPRN_RPR, r9
+ mtspr SPRN_SDR1, r10
+
+ LOAD_REG_ADDR(r5, virtual_mode)
+
+ /* Get out of real mode */
+ mtspr SPRN_SRR0,r5
+ mtspr SPRN_SRR1,r12
+ rfid
+ b . /* prevent speculative execution */
+
+virtual_mode:
+ blr
diff --git a/arch/powerpc/platforms/powernv/subcore.c b/arch/powerpc/platforms/powernv/subcore.c
new file mode 100644
index 0000000..894ecb3
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/subcore.c
@@ -0,0 +1,392 @@
+/*
+ * Copyright 2013, Michael (Ellerman|Neuling), IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#define pr_fmt(fmt) "powernv: " fmt
+
+#include <linux/kernel.h>
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/device.h>
+#include <linux/gfp.h>
+#include <linux/smp.h>
+#include <linux/stop_machine.h>
+
+#include <asm/cputhreads.h>
+#include <asm/kvm_ppc.h>
+#include <asm/machdep.h>
+#include <asm/opal.h>
+#include <asm/smp.h>
+
+#include "subcore.h"
+
+
+/*
+ * Split/unsplit procedure:
+ *
+ * A core can be in one of three states, unsplit, 2-way split, and 4-way split.
+ *
+ * The mapping to subcores_per_core is simple:
+ *
+ * State | subcores_per_core
+ * ------------|------------------
+ * Unsplit | 1
+ * 2-way split | 2
+ * 4-way split | 4
+ *
+ * The core is split along thread boundaries, the mapping between subcores and
+ * threads is as follows:
+ *
+ * Unsplit:
+ * ----------------------------
+ * Subcore | 0 |
+ * ----------------------------
+ * Thread | 0 1 2 3 4 5 6 7 |
+ * ----------------------------
+ *
+ * 2-way split:
+ * -------------------------------------
+ * Subcore | 0 | 1 |
+ * -------------------------------------
+ * Thread | 0 1 2 3 | 4 5 6 7 |
+ * -------------------------------------
+ *
+ * 4-way split:
+ * -----------------------------------------
+ * Subcore | 0 | 1 | 2 | 3 |
+ * -----------------------------------------
+ * Thread | 0 1 | 2 3 | 4 5 | 6 7 |
+ * -----------------------------------------
+ *
+ *
+ * Transitions
+ * -----------
+ *
+ * It is not possible to transition between either of the split states, the
+ * core must first be unsplit. The legal transitions are:
+ *
+ * ----------- ---------------
+ * | | <----> | 2-way split |
+ * | | ---------------
+ * | Unsplit |
+ * | | ---------------
+ * | | <----> | 4-way split |
+ * ----------- ---------------
+ *
+ * Unsplitting
+ * -----------
+ *
+ * Unsplitting is the simpler procedure. It requires thread 0 to request the
+ * unsplit while all other threads NAP.
+ *
+ * Thread 0 clears HID0_POWER8_DYNLPARDIS (Dynamic LPAR Disable). This tells
+ * the hardware that if all threads except 0 are napping, the hardware should
+ * unsplit the core.
+ *
+ * Non-zero threads are sent to a NAP loop, they don't exit the loop until they
+ * see the core unsplit.
+ *
+ * Core 0 spins waiting for the hardware to see all the other threads napping
+ * and perform the unsplit.
+ *
+ * Once thread 0 sees the unsplit, it IPIs the secondary threads to wake them
+ * out of NAP. They will then see the core unsplit and exit the NAP loop.
+ *
+ * Splitting
+ * ---------
+ *
+ * The basic splitting procedure is fairly straight forward. However it is
+ * complicated by the fact that after the split occurs, the newly created
+ * subcores are not in a fully initialised state.
+ *
+ * Most notably the subcores do not have the correct value for SDR1, which
+ * means they must not be running in virtual mode when the split occurs. The
+ * subcores have separate timebases SPRs but these are pre-synchronised by
+ * opal.
+ *
+ * To begin with secondary threads are sent to an assembly routine. There they
+ * switch to real mode, so they are immune to the uninitialised SDR1 value.
+ * Once in real mode they indicate that they are in real mode, and spin waiting
+ * to see the core split.
+ *
+ * Thread 0 waits to see that all secondaries are in real mode, and then begins
+ * the splitting procedure. It firstly sets HID0_POWER8_DYNLPARDIS, which
+ * prevents the hardware from unsplitting. Then it sets the appropriate HID bit
+ * to request the split, and spins waiting to see that the split has happened.
+ *
+ * Concurrently the secondaries will notice the split. When they do they set up
+ * their SPRs, notably SDR1, and then they can return to virtual mode and exit
+ * the procedure.
+ */
+
+/* Initialised at boot by subcore_init() */
+static int subcores_per_core;
+
+/*
+ * Used to communicate to offline cpus that we want them to pop out of the
+ * offline loop and do a split or unsplit.
+ *
+ * 0 - no split happening
+ * 1 - unsplit in progress
+ * 2 - split to 2 in progress
+ * 4 - split to 4 in progress
+ */
+static int new_split_mode;
+
+static cpumask_var_t cpu_offline_mask;
+
+struct split_state {
+ u8 step;
+ u8 master;
+};
+
+static DEFINE_PER_CPU(struct split_state, split_state);
+
+static void wait_for_sync_step(int step)
+{
+ int i, cpu = smp_processor_id();
+
+ for (i = cpu + 1; i < cpu + threads_per_core; i++)
+ while(per_cpu(split_state, i).step < step)
+ barrier();
+
+ /* Order the wait loop vs any subsequent loads/stores. */
+ mb();
+}
+
+static void unsplit_core(void)
+{
+ u64 hid0, mask;
+ int i, cpu;
+
+ mask = HID0_POWER8_2LPARMODE | HID0_POWER8_4LPARMODE;
+
+ cpu = smp_processor_id();
+ if (cpu_thread_in_core(cpu) != 0) {
+ while (mfspr(SPRN_HID0) & mask)
+ power7_nap(0);
+
+ per_cpu(split_state, cpu).step = SYNC_STEP_UNSPLIT;
+ return;
+ }
+
+ hid0 = mfspr(SPRN_HID0);
+ hid0 &= ~HID0_POWER8_DYNLPARDIS;
+ mtspr(SPRN_HID0, hid0);
+
+ while (mfspr(SPRN_HID0) & mask)
+ cpu_relax();
+
+ /* Wake secondaries out of NAP */
+ for (i = cpu + 1; i < cpu + threads_per_core; i++)
+ smp_send_reschedule(i);
+
+ wait_for_sync_step(SYNC_STEP_UNSPLIT);
+}
+
+static void split_core(int new_mode)
+{
+ struct { u64 value; u64 mask; } split_parms[2] = {
+ { HID0_POWER8_1TO2LPAR, HID0_POWER8_2LPARMODE },
+ { HID0_POWER8_1TO4LPAR, HID0_POWER8_4LPARMODE }
+ };
+ int i, cpu;
+ u64 hid0;
+
+ /* Convert new_mode (2 or 4) into an index into our parms array */
+ i = (new_mode >> 1) - 1;
+ BUG_ON(i < 0 || i > 1);
+
+ cpu = smp_processor_id();
+ if (cpu_thread_in_core(cpu) != 0) {
+ split_core_secondary_loop(&per_cpu(split_state, cpu).step);
+ return;
+ }
+
+ wait_for_sync_step(SYNC_STEP_REAL_MODE);
+
+ /* Write new mode */
+ hid0 = mfspr(SPRN_HID0);
+ hid0 |= HID0_POWER8_DYNLPARDIS | split_parms[i].value;
+ mtspr(SPRN_HID0, hid0);
+
+ /* Wait for it to happen */
+ while (!(mfspr(SPRN_HID0) & split_parms[i].mask))
+ cpu_relax();
+}
+
+static void cpu_do_split(int new_mode)
+{
+ /*
+ * At boot subcores_per_core will be 0, so we will always unsplit at
+ * boot. In the usual case where the core is already unsplit it's a
+ * nop, and this just ensures the kernel's notion of the mode is
+ * consistent with the hardware.
+ */
+ if (subcores_per_core != 1)
+ unsplit_core();
+
+ if (new_mode != 1)
+ split_core(new_mode);
+
+ mb();
+ per_cpu(split_state, smp_processor_id()).step = SYNC_STEP_FINISHED;
+}
+
+bool cpu_core_split_required(void)
+{
+ smp_rmb();
+
+ if (!new_split_mode)
+ return false;
+
+ cpu_do_split(new_split_mode);
+
+ return true;
+}
+
+static int cpu_update_split_mode(void *data)
+{
+ int cpu, new_mode = *(int *)data;
+
+ if (this_cpu_ptr(&split_state)->master) {
+ new_split_mode = new_mode;
+ smp_wmb();
+
+ cpumask_andnot(cpu_offline_mask, cpu_present_mask,
+ cpu_online_mask);
+
+ /* This should work even though the cpu is offline */
+ for_each_cpu(cpu, cpu_offline_mask)
+ smp_send_reschedule(cpu);
+ }
+
+ cpu_do_split(new_mode);
+
+ if (this_cpu_ptr(&split_state)->master) {
+ /* Wait for all cpus to finish before we touch subcores_per_core */
+ for_each_present_cpu(cpu) {
+ if (cpu >= setup_max_cpus)
+ break;
+
+ while(per_cpu(split_state, cpu).step < SYNC_STEP_FINISHED)
+ barrier();
+ }
+
+ new_split_mode = 0;
+
+ /* Make the new mode public */
+ subcores_per_core = new_mode;
+ threads_per_subcore = threads_per_core / subcores_per_core;
+
+ /* Make sure the new mode is written before we exit */
+ mb();
+ }
+
+ return 0;
+}
+
+static int set_subcores_per_core(int new_mode)
+{
+ struct split_state *state;
+ int cpu;
+
+ if (kvm_hv_mode_active()) {
+ pr_err("Unable to change split core mode while KVM active.\n");
+ return -EBUSY;
+ }
+
+ /*
+ * We are only called at boot, or from the sysfs write. If that ever
+ * changes we'll need a lock here.
+ */
+ BUG_ON(new_mode < 1 || new_mode > 4 || new_mode == 3);
+
+ for_each_present_cpu(cpu) {
+ state = &per_cpu(split_state, cpu);
+ state->step = SYNC_STEP_INITIAL;
+ state->master = 0;
+ }
+
+ get_online_cpus();
+
+ /* This cpu will update the globals before exiting stop machine */
+ this_cpu_ptr(&split_state)->master = 1;
+
+ /* Ensure state is consistent before we call the other cpus */
+ mb();
+
+ stop_machine(cpu_update_split_mode, &new_mode, cpu_online_mask);
+
+ put_online_cpus();
+
+ return 0;
+}
+
+static ssize_t __used store_subcores_per_core(struct device *dev,
+ struct device_attribute *attr, const char *buf,
+ size_t count)
+{
+ unsigned long val;
+ int rc;
+
+ /* We are serialised by the attribute lock */
+
+ rc = sscanf(buf, "%lx", &val);
+ if (rc != 1)
+ return -EINVAL;
+
+ switch (val) {
+ case 1:
+ case 2:
+ case 4:
+ if (subcores_per_core == val)
+ /* Nothing to do */
+ goto out;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ rc = set_subcores_per_core(val);
+ if (rc)
+ return rc;
+
+out:
+ return count;
+}
+
+static ssize_t show_subcores_per_core(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return sprintf(buf, "%x\n", subcores_per_core);
+}
+
+static DEVICE_ATTR(subcores_per_core, 0644,
+ show_subcores_per_core, store_subcores_per_core);
+
+static int subcore_init(void)
+{
+ if (!cpu_has_feature(CPU_FTR_ARCH_207S))
+ return 0;
+
+ /*
+ * We need all threads in a core to be present to split/unsplit so
+ * continue only if max_cpus are aligned to threads_per_core.
+ */
+ if (setup_max_cpus % threads_per_core)
+ return 0;
+
+ BUG_ON(!alloc_cpumask_var(&cpu_offline_mask, GFP_KERNEL));
+
+ set_subcores_per_core(1);
+
+ return device_create_file(cpu_subsys.dev_root,
+ &dev_attr_subcores_per_core);
+}
+machine_device_initcall(powernv, subcore_init);
diff --git a/arch/powerpc/platforms/powernv/subcore.h b/arch/powerpc/platforms/powernv/subcore.h
new file mode 100644
index 0000000..148abc9
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/subcore.h
@@ -0,0 +1,18 @@
+/*
+ * Copyright 2013, Michael Ellerman, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+/* These are ordered and tested with <= */
+#define SYNC_STEP_INITIAL 0
+#define SYNC_STEP_UNSPLIT 1 /* Set by secondary when it sees unsplit */
+#define SYNC_STEP_REAL_MODE 2 /* Set by secondary when in real mode */
+#define SYNC_STEP_FINISHED 3 /* Set by secondary when split/unsplit is done */
+
+#ifndef __ASSEMBLY__
+void split_core_secondary_loop(u8 *state);
+#endif
--
1.9.1
^ permalink raw reply related
* RE: [PATCH 1/2] mtd/spi: support en25s64 device
From: Shengzhou.Liu @ 2014-05-23 8:20 UTC (permalink / raw)
To: Scott Wood; +Cc: linuxppc-dev@lists.ozlabs.org
In-Reply-To: <1400705405.18239.92.camel@snotra.buserror.net>
DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gLS0tIGEvZHJpdmVycy9tdGQvZGV2
aWNlcy9tMjVwODAuYw0KPiA+ICsrKyBiL2RyaXZlcnMvbXRkL2RldmljZXMvbTI1cDgwLmMNCj4g
PiBAQCAtNzQ1LDYgKzc0NSw3IEBAIHN0YXRpYyBjb25zdCBzdHJ1Y3Qgc3BpX2RldmljZV9pZCBt
MjVwX2lkc1tdID0gew0KPiA+ICAJeyAiZW4yNXEzMmIiLCAgIElORk8oMHgxYzMwMTYsIDAsIDY0
ICogMTAyNCwgICA2NCwgMCkgfSwNCj4gPiAgCXsgImVuMjVwNjQiLCAgICBJTkZPKDB4MWMyMDE3
LCAwLCA2NCAqIDEwMjQsICAxMjgsIDApIH0sDQo+ID4gIAl7ICJlbjI1cTY0IiwgICAgSU5GTygw
eDFjMzAxNywgMCwgNjQgKiAxMDI0LCAgMTI4LCBTRUNUXzRLKSB9LA0KPiA+ICsJeyAiZW4yNXM2
NCIsICAgIElORk8oMHgxYzM4MTcsIDAsIDY0ICogMTAyNCwgIDEyOCwgMCkgfSwNCj4gPiAgCXsg
ImVuMjVxaDI1NiIsICBJTkZPKDB4MWM3MDE5LCAwLCA2NCAqIDEwMjQsICA1MTIsIDApIH0sDQo+
ID4NCj4gPiAgCS8qIEVTTVQgKi8NCj4gDQo+IFRoaXMgbmVlZHMgdG8gYmUgc2VudCB0byB0aGUg
bXRkIGFuZC9vciBzcGkgbWFpbnRhaW5lcnMsIG5vdCBoZXJlLg0KPiANCj4gV2hhdCBkb2VzIHRo
aXMgaGF2ZSB0byBkbyB3aXRoIHBhdGNoIDIvMj8gIERvbid0IHB1dCB1bnJlbGF0ZWQgdGhpbmdz
IGluDQo+IHRoZSBzYW1lIHBhdGNoc2V0LCBlc3BlY2lhbGx5IHdoZW4gdGhleSdyZSBkZXN0aW5l
ZCBmb3IgZGlmZmVyZW50DQo+IG1haW50YWluZXJzLg0KPiANCj4gLVNjb3R0DQo+IA0KW1NoZW5n
emhvdV0gdGhhbmtzLCBzZW50IHRvIGxpbnV4LW10ZEBsaXN0cy5pbmZyYWRlYWQub3JnIGFscmVh
ZHkuDQoNCg==
^ permalink raw reply
* [PATCH 3/4] powerpc: Update comments for generic idle conversion
From: Geert Uytterhoeven @ 2014-05-23 8:54 UTC (permalink / raw)
To: Thomas Gleixner, Andrew Morton
Cc: linuxppc-dev, linux-kernel, Geert Uytterhoeven, linux-pm
In-Reply-To: <1400835265-21956-1-git-send-email-geert+renesas@glider.be>
As of commit 799fef06123f86ff69cf754f996219e6ad1678f8 ("powerpc: Use
generic idle loop"), this applies to arch_cpu_idle() instead of cpu_idle().
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: linuxppc-dev@lists.ozlabs.org
---
arch/powerpc/kernel/irq.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/irq.c b/arch/powerpc/kernel/irq.c
index ca1cd7459c4a..248ee7e5bebd 100644
--- a/arch/powerpc/kernel/irq.c
+++ b/arch/powerpc/kernel/irq.c
@@ -304,7 +304,7 @@ void notrace restore_interrupts(void)
* being re-enabled and generally sanitized the lazy irq state,
* and in the latter case it will leave with interrupts hard
* disabled and marked as such, so the local_irq_enable() call
- * in cpu_idle() will properly re-enable everything.
+ * in arch_cpu_idle() will properly re-enable everything.
*/
bool prep_irq_for_idle(void)
{
--
1.9.1
^ permalink raw reply related
* [PATCH 1/3 v2] powerpc/fsl-booke: Add support for T2080/T2081 SoC
From: Shengzhou Liu @ 2014-05-23 8:27 UTC (permalink / raw)
To: linuxppc-dev, scottwood; +Cc: Shengzhou Liu
Add support for T2080/T2081 SoC without DPAA components.
The T2080 SoC includes the following function and features:
- Four dual-threaded 64-bit Power architecture e6500 cores, up to 1.8GHz
- 2MB L2 cache and 512KB CoreNet platform cache (CPC)
- Hierarchical interconnect fabric
- One 32-/64-bit DDR3/3L SDRAM memory controllers with ECC and interleaving
- Data Path Acceleration Architecture (DPAA) incorporating acceleration
- 16 SerDes lanes up to 10.3125 GHz
- 8 Ethernet interfaces (multiple 1G/2.5G/10G MACs)
- High-speed peripheral interfaces
- Four PCI Express controllers (two PCIe 2.0 and two PCIe 3.0)
- Two Serial RapidIO 2.0 controllers/ports running at up to 5 GHz
- Additional peripheral interfaces
- Two serial ATA (SATA 2.0) controllers
- Two high-speed USB 2.0 controllers with integrated PHY
- Enhanced secure digital host controller (SD/SDXC/eMMC)
- Enhanced serial peripheral interface (eSPI)
- Four I2C controllers
- Four 2-pin UARTs or two 4-pin UARTs
- Integrated Flash Controller supporting NAND and NOR flash
- Three eight-channel DMA engines
- Support for hardware virtualization and partitioning enforcement
- QorIQ Platform's Trust Architecture 2.0
T2081 is a reduced personality of T2080 with following difference:
Feature T2080 T2081
1G Ethernet numbers: 8 6
10G Ethernet numbers: 4 2
SerDes lanes: 16 8
Serial RapidIO,RMan: 2 no
SATA Controller: 2 no
Aurora: yes no
SoC Package: 896-pins 780-pins
Signed-off-by: Shengzhou Liu <Shengzhou.Liu@freescale.com>
---
v2: updated with some comments.
arch/powerpc/boot/dts/fsl/t2080si-post.dtsi | 60 +++++
arch/powerpc/boot/dts/fsl/t2081si-post.dtsi | 384 ++++++++++++++++++++++++++++
arch/powerpc/boot/dts/fsl/t208xsi-pre.dtsi | 91 +++++++
arch/powerpc/include/asm/mpc85xx.h | 2 +
4 files changed, 537 insertions(+)
create mode 100644 arch/powerpc/boot/dts/fsl/t2080si-post.dtsi
create mode 100644 arch/powerpc/boot/dts/fsl/t2081si-post.dtsi
create mode 100644 arch/powerpc/boot/dts/fsl/t208xsi-pre.dtsi
diff --git a/arch/powerpc/boot/dts/fsl/t2080si-post.dtsi b/arch/powerpc/boot/dts/fsl/t2080si-post.dtsi
new file mode 100644
index 0000000..1a902fe
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/t2080si-post.dtsi
@@ -0,0 +1,60 @@
+/*
+ * T2080 Silicon/SoC Device Tree Source (post include)
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "t2081si-post.dtsi"
+
+&soc {
+/include/ "qoriq-sata2-0.dtsi"
+/include/ "qoriq-sata2-1.dtsi"
+};
+
+&rio {
+ compatible = "fsl,srio";
+ interrupts = <16 2 1 11>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges;
+
+ port1 {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ cell-index = <1>;
+ };
+
+ port2 {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ cell-index = <2>;
+ };
+};
diff --git a/arch/powerpc/boot/dts/fsl/t2081si-post.dtsi b/arch/powerpc/boot/dts/fsl/t2081si-post.dtsi
new file mode 100644
index 0000000..4311560
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/t2081si-post.dtsi
@@ -0,0 +1,384 @@
+/*
+ * T2081 Silicon/SoC Device Tree Source (post include)
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+&ifc {
+ #address-cells = <2>;
+ #size-cells = <1>;
+ compatible = "fsl,ifc", "simple-bus";
+ interrupts = <25 2 0 0>;
+};
+
+/* controller at 0x240000 */
+&pci0 {
+ compatible = "fsl,t2080-pcie", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0x0 0xff>;
+ interrupts = <20 2 0 0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <20 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 40 1 0 0
+ 0000 0 0 2 &mpic 1 1 0 0
+ 0000 0 0 3 &mpic 2 1 0 0
+ 0000 0 0 4 &mpic 3 1 0 0
+ >;
+ };
+};
+
+/* controller at 0x250000 */
+&pci1 {
+ compatible = "fsl,t2080-pcie", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0 0xff>;
+ interrupts = <21 2 0 0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <21 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 41 1 0 0
+ 0000 0 0 2 &mpic 5 1 0 0
+ 0000 0 0 3 &mpic 6 1 0 0
+ 0000 0 0 4 &mpic 7 1 0 0
+ >;
+ };
+};
+
+/* controller at 0x260000 */
+&pci2 {
+ compatible = "fsl,t2080-pcie", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0x0 0xff>;
+ interrupts = <22 2 0 0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <22 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 42 1 0 0
+ 0000 0 0 2 &mpic 9 1 0 0
+ 0000 0 0 3 &mpic 10 1 0 0
+ 0000 0 0 4 &mpic 11 1 0 0
+ >;
+ };
+};
+
+/* controller at 0x270000 */
+&pci3 {
+ compatible = "fsl,t2080-pcie", "fsl,qoriq-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0x0 0xff>;
+ interrupts = <23 2 0 0>;
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <23 2 0 0>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <
+ /* IDSEL 0x0 */
+ 0000 0 0 1 &mpic 43 1 0 0
+ 0000 0 0 2 &mpic 0 1 0 0
+ 0000 0 0 3 &mpic 4 1 0 0
+ 0000 0 0 4 &mpic 8 1 0 0
+ >;
+ };
+};
+
+&dcsr {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,dcsr", "simple-bus";
+
+ dcsr-epu@0 {
+ compatible = "fsl,t2080-dcsr-epu", "fsl,dcsr-epu";
+ interrupts = <52 2 0 0
+ 84 2 0 0
+ 85 2 0 0
+ 94 2 0 0
+ 95 2 0 0>;
+ reg = <0x0 0x1000>;
+ };
+ dcsr-npc {
+ compatible = "fsl,t2080-dcsr-cnpc", "fsl,dcsr-cnpc";
+ reg = <0x1000 0x1000 0x1002000 0x10000>;
+ };
+ dcsr-nxc@2000 {
+ compatible = "fsl,dcsr-nxc";
+ reg = <0x2000 0x1000>;
+ };
+ dcsr-corenet {
+ compatible = "fsl,dcsr-corenet";
+ reg = <0x8000 0x1000 0x1A000 0x1000>;
+ };
+ dcsr-ocn@11000 {
+ compatible = "fsl,t2080-dcsr-ocn", "fsl,dcsr-ocn";
+ reg = <0x11000 0x1000>;
+ };
+ dcsr-ddr@12000 {
+ compatible = "fsl,dcsr-ddr";
+ dev-handle = <&ddr1>;
+ reg = <0x12000 0x1000>;
+ };
+ dcsr-nal@18000 {
+ compatible = "fsl,t2080-dcsr-nal", "fsl,dcsr-nal";
+ reg = <0x18000 0x1000>;
+ };
+ dcsr-rcpm@22000 {
+ compatible = "fsl,t2080-dcsr-rcpm", "fsl,dcsr-rcpm";
+ reg = <0x22000 0x1000>;
+ };
+ dcsr-snpc@30000 {
+ compatible = "fsl,t2080-dcsr-snpc", "fsl,dcsr-snpc";
+ reg = <0x30000 0x1000 0x1022000 0x10000>;
+ };
+ dcsr-snpc@31000 {
+ compatible = "fsl,t2080-dcsr-snpc", "fsl,dcsr-snpc";
+ reg = <0x31000 0x1000 0x1042000 0x10000>;
+ };
+ dcsr-snpc@32000 {
+ compatible = "fsl,t2080-dcsr-snpc", "fsl,dcsr-snpc";
+ reg = <0x32000 0x1000 0x1062000 0x10000>;
+ };
+ dcsr-cpu-sb-proxy@100000 {
+ compatible = "fsl,dcsr-e6500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu0>;
+ reg = <0x100000 0x1000 0x101000 0x1000>;
+ };
+ dcsr-cpu-sb-proxy@108000 {
+ compatible = "fsl,dcsr-e6500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu1>;
+ reg = <0x108000 0x1000 0x109000 0x1000>;
+ };
+ dcsr-cpu-sb-proxy@110000 {
+ compatible = "fsl,dcsr-e6500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu2>;
+ reg = <0x110000 0x1000 0x111000 0x1000>;
+ };
+ dcsr-cpu-sb-proxy@118000 {
+ compatible = "fsl,dcsr-e6500-sb-proxy", "fsl,dcsr-cpu-sb-proxy";
+ cpu-handle = <&cpu3>;
+ reg = <0x118000 0x1000 0x119000 0x1000>;
+ };
+};
+
+&soc {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ device_type = "soc";
+ compatible = "simple-bus";
+
+ soc-sram-error {
+ compatible = "fsl,soc-sram-error";
+ interrupts = <16 2 1 29>;
+ };
+
+ corenet-law@0 {
+ compatible = "fsl,corenet-law";
+ reg = <0x0 0x1000>;
+ fsl,num-laws = <32>;
+ };
+
+ ddr1: memory-controller@8000 {
+ compatible = "fsl,qoriq-memory-controller-v4.7",
+ "fsl,qoriq-memory-controller";
+ reg = <0x8000 0x1000>;
+ interrupts = <16 2 1 23>;
+ };
+
+ cpc: l3-cache-controller@10000 {
+ compatible = "fsl,t2080-l3-cache-controller", "cache";
+ reg = <0x10000 0x1000
+ 0x11000 0x1000
+ 0x12000 0x1000>;
+ interrupts = <16 2 1 27
+ 16 2 1 26
+ 16 2 1 25>;
+ };
+
+ corenet-cf@18000 {
+ compatible = "fsl,corenet2-cf", "fsl,corenet-cf";
+ reg = <0x18000 0x1000>;
+ interrupts = <16 2 1 31>;
+ fsl,ccf-num-csdids = <32>;
+ fsl,ccf-num-snoopids = <32>;
+ };
+
+ iommu@20000 {
+ compatible = "fsl,pamu-v1.0", "fsl,pamu";
+ reg = <0x20000 0x6000>;
+ interrupts = <
+ 24 2 0 0
+ 16 2 1 30>;
+ };
+
+/include/ "qoriq-mpic4.3.dtsi"
+
+ guts: global-utilities@e0000 {
+ compatible = "fsl,t2080-device-config", "fsl,qoriq-device-config-2.0";
+ reg = <0xe0000 0xe00>;
+ fsl,has-rstcr;
+ fsl,liodn-bits = <12>;
+ };
+
+ clockgen: global-utilities@e1000 {
+ compatible = "fsl,t2080-clockgen", "fsl,qoriq-clockgen-2.0",
+ "fixed-clock";
+ reg = <0xe1000 0x1000>;
+ clock-output-names = "sysclk";
+ #clock-cells = <0>;
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pll0: pll0@800 {
+ #clock-cells = <1>;
+ reg = <0x800>;
+ compatible = "fsl,core-pll-clock";
+ clocks = <&clockgen>;
+ clock-output-names = "pll0", "pll0-div2", "pll0-div4";
+ };
+ pll1: pll1@820 {
+ #clock-cells = <1>;
+ reg = <0x820>;
+ compatible = "fsl,core-pll-clock";
+ clocks = <&clockgen>;
+ clock-output-names = "pll1", "pll1-div2", "pll1-div4";
+ };
+ mux0: mux0@0 {
+ #clock-cells = <0>;
+ reg = <0x0>;
+ compatible = "fsl,core-mux-clock";
+ clocks = <&pll0 0>, <&pll0 1>, <&pll0 2>,
+ <&pll1 0>, <&pll1 1>, <&pll1 2>;
+ clock-names = "pll0_0", "pll0_1", "pll0_2",
+ "pll1_0", "pll1_1", "pll1_2";
+ clock-output-names = "cmux0";
+ };
+ mux1: mux1@20 {
+ #clock-cells = <0>;
+ reg = <0x20>;
+ compatible = "fsl,core-mux-clock";
+ clocks = <&pll0 0>, <&pll0 1>, <&pll0 2>,
+ <&pll1 0>, <&pll1 1>, <&pll1 2>;
+ clock-names = "pll0_0", "pll0_1", "pll0_2",
+ "pll1_0", "pll1_1", "pll1_2";
+ clock-output-names = "cmux1";
+ };
+ };
+
+ rcpm: global-utilities@e2000 {
+ compatible = "fsl,t2080-rcpm", "fsl,qoriq-rcpm-2.0";
+ reg = <0xe2000 0x1000>;
+ };
+
+ sfp: sfp@e8000 {
+ compatible = "fsl,t2080-sfp";
+ reg = <0xe8000 0x1000>;
+ };
+
+ serdes: serdes@ea000 {
+ compatible = "fsl,t2080-serdes";
+ reg = <0xea000 0x4000>;
+ };
+
+/include/ "elo3-dma-0.dtsi"
+/include/ "elo3-dma-1.dtsi"
+/include/ "elo3-dma-2.dtsi"
+
+/include/ "qoriq-espi-0.dtsi"
+ spi@110000 {
+ fsl,espi-num-chipselects = <4>;
+ };
+
+/include/ "qoriq-esdhc-0.dtsi"
+ sdhc@114000 {
+ compatible = "fsl,t2080-esdhc", "fsl,esdhc";
+ sdhci,auto-cmd12;
+ };
+/include/ "qoriq-i2c-0.dtsi"
+/include/ "qoriq-i2c-1.dtsi"
+/include/ "qoriq-duart-0.dtsi"
+/include/ "qoriq-duart-1.dtsi"
+/include/ "qoriq-gpio-0.dtsi"
+/include/ "qoriq-gpio-1.dtsi"
+/include/ "qoriq-gpio-2.dtsi"
+/include/ "qoriq-gpio-3.dtsi"
+/include/ "qoriq-usb2-mph-0.dtsi"
+ usb0: usb@210000 {
+ compatible = "fsl-usb2-mph-v2.4", "fsl-usb2-mph";
+ phy_type = "utmi";
+ port0;
+ };
+/include/ "qoriq-usb2-dr-0.dtsi"
+ usb1: usb@211000 {
+ compatible = "fsl-usb2-dr-v2.4", "fsl-usb2-dr";
+ dr_mode = "host";
+ phy_type = "utmi";
+ };
+/include/ "qoriq-sec5.2-0.dtsi"
+
+ L2_1: l2-cache-controller@c20000 {
+ /* Cluster 0 L2 cache */
+ compatible = "fsl,t2080-l2-cache-controller";
+ reg = <0xc20000 0x40000>;
+ next-level-cache = <&cpc>;
+ };
+};
diff --git a/arch/powerpc/boot/dts/fsl/t208xsi-pre.dtsi b/arch/powerpc/boot/dts/fsl/t208xsi-pre.dtsi
new file mode 100644
index 0000000..d21b100
--- /dev/null
+++ b/arch/powerpc/boot/dts/fsl/t208xsi-pre.dtsi
@@ -0,0 +1,91 @@
+/*
+ * T2080/T2081 Silicon/SoC Device Tree Source (pre include)
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/dts-v1/;
+
+/include/ "e6500_power_isa.dtsi"
+
+/ {
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ aliases {
+ ccsr = &soc;
+ dcsr = &dcsr;
+
+ serial0 = &serial0;
+ serial1 = &serial1;
+ serial2 = &serial2;
+ serial3 = &serial3;
+
+ crypto = &crypto;
+ pci0 = &pci0;
+ pci1 = &pci1;
+ pci2 = &pci2;
+ pci3 = &pci3;
+ usb0 = &usb0;
+ usb1 = &usb1;
+ dma0 = &dma0;
+ dma1 = &dma1;
+ dma2 = &dma2;
+ sdhc = &sdhc;
+ };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu0: PowerPC,e6500@0 {
+ device_type = "cpu";
+ reg = <0 1>;
+ next-level-cache = <&L2_1>;
+ };
+ cpu1: PowerPC,e6500@2 {
+ device_type = "cpu";
+ reg = <2 3>;
+ next-level-cache = <&L2_1>;
+ };
+ cpu2: PowerPC,e6500@4 {
+ device_type = "cpu";
+ reg = <4 5>;
+ next-level-cache = <&L2_1>;
+ };
+ cpu3: PowerPC,e6500@6 {
+ device_type = "cpu";
+ reg = <6 7>;
+ next-level-cache = <&L2_1>;
+ };
+ };
+};
diff --git a/arch/powerpc/include/asm/mpc85xx.h b/arch/powerpc/include/asm/mpc85xx.h
index 736d4ac..3bef74a 100644
--- a/arch/powerpc/include/asm/mpc85xx.h
+++ b/arch/powerpc/include/asm/mpc85xx.h
@@ -77,6 +77,8 @@
#define SVR_T1020 0x852100
#define SVR_T1021 0x852101
#define SVR_T1022 0x852102
+#define SVR_T2080 0x853000
+#define SVR_T2081 0x853100
#define SVR_8610 0x80A000
#define SVR_8641 0x809000
--
1.8.0
^ permalink raw reply related
* [PATCH v2] powerpc/t2080rdb: Add T2080RDB board support
From: Shengzhou Liu @ 2014-05-23 8:27 UTC (permalink / raw)
To: linuxppc-dev, scottwood; +Cc: Shengzhou Liu
In-Reply-To: <1400833623-17547-1-git-send-email-Shengzhou.Liu@freescale.com>
T2080PCIe-RDB is a Freescale Reference Design Board that hosts T2080 SoC.
The board feature overview:
Processor:
- T2080 SoC integrating four 64-bit dual-threads e6500 cores up to 1.8GHz
DDR Memory:
- Single memory controller capable of supporting DDR3 and DDR3-LP devices
- 72bit 4GB DDR3-LP SODIMM in slot
Ethernet interfaces:
- Two 1Gbps RGMII ports on-board
- Two 10Gbps SFP+ ports on-board
- Two 10Gbps Base-T ports on-board
Accelerator:
- DPAA components consist of FMan, BMan, QMan, PME, DCE and SEC
SerDes 16 lanes configuration:
- SerDes-1 Lane A-B: to two 10G SFP+ (MAC9 & MAC10)
- SerDes-1 Lane C-D: to two 10G Base-T (MAC1 & MAC2)
- SerDes-1 Lane E-H: to PCIe slot (PEX4 Gen3 x4)
- SerDes-2 Lane A-D: to PCIe finger (PEX1 x4)
- SerDes-2 Lane E-F: to C293 secure co-processor (PEX2 x2)
- SerDes-2 Lane G-H: to SATA1 & SATA2
IFC/Local Bus
- NOR: 128MB 16-bit NOR flash
- NAND: 1GB 8-bit NAND flash
- CPLD: for system controlling with programable header on-board
eSPI:
- 64MB N25Q512 SPI flash
USB:
- Two USB2.0 ports with internal PHY (both Type-A)
PCIe:
- One PCIe x4 goldfinger
- One PCIe x4 slot
- One PCIe x2 end-point device (C293 crypto co-processor)
SATA:
- Two SATA 2.0 ports on-board
SDHC:
- support a MicroSD/TF card on-board
I2C:
- Four I2C controllers.
UART:
- Dual 4-pins UART serial ports
Signed-off-by: Shengzhou Liu <Shengzhou.Liu@freescale.com>
---
v2: updated with some comments.
arch/powerpc/boot/dts/t2080rdb.dts | 57 ++++++++
arch/powerpc/boot/dts/t208xrdb.dtsi | 197 ++++++++++++++++++++++++++
arch/powerpc/platforms/85xx/Kconfig | 2 +-
arch/powerpc/platforms/85xx/corenet_generic.c | 2 +
4 files changed, 257 insertions(+), 1 deletion(-)
create mode 100644 arch/powerpc/boot/dts/t2080rdb.dts
create mode 100644 arch/powerpc/boot/dts/t208xrdb.dtsi
diff --git a/arch/powerpc/boot/dts/t2080rdb.dts b/arch/powerpc/boot/dts/t2080rdb.dts
new file mode 100644
index 0000000..e889104
--- /dev/null
+++ b/arch/powerpc/boot/dts/t2080rdb.dts
@@ -0,0 +1,57 @@
+/*
+ * T2080PCIe-RDB Board Device Tree Source
+ *
+ * Copyright 2014 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "fsl/t208xsi-pre.dtsi"
+/include/ "t208xrdb.dtsi"
+
+/ {
+ model = "fsl,T2080RDB";
+ compatible = "fsl,T2080RDB";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ rio: rapidio@ffe0c0000 {
+ reg = <0xf 0xfe0c0000 0 0x11000>;
+
+ port1 {
+ ranges = <0 0 0xc 0x20000000 0 0x10000000>;
+ };
+ port2 {
+ ranges = <0 0 0xc 0x30000000 0 0x10000000>;
+ };
+ };
+};
+
+/include/ "fsl/t2080si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/t208xrdb.dtsi b/arch/powerpc/boot/dts/t208xrdb.dtsi
new file mode 100644
index 0000000..3b85985
--- /dev/null
+++ b/arch/powerpc/boot/dts/t208xrdb.dtsi
@@ -0,0 +1,197 @@
+/*
+ * T2080PCIe-RDB Board Device Tree Source
+ *
+ * Copyright 2014 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/ {
+ model = "fsl,T2080RDB";
+ compatible = "fsl,T2080RDB";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ ifc: localbus@ffe124000 {
+ reg = <0xf 0xfe124000 0 0x2000>;
+ ranges = <0 0 0xf 0xe8000000 0x08000000
+ 2 0 0xf 0xff800000 0x00010000
+ 3 0 0xf 0xffdf0000 0x00008000>;
+
+ nor@0,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "cfi-flash";
+ reg = <0x0 0x0 0x8000000>;
+
+ bank-width = <2>;
+ device-width = <1>;
+ };
+
+ nand@1,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,ifc-nand";
+ reg = <0x2 0x0 0x10000>;
+ };
+
+ boardctrl: board-control@2,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,t2080-cpld";
+ reg = <3 0 0x300>;
+ ranges = <0 3 0 0x300>;
+ };
+ };
+
+ memory {
+ device_type = "memory";
+ };
+
+ dcsr: dcsr@f00000000 {
+ ranges = <0x00000000 0xf 0x00000000 0x01072000>;
+ };
+
+ soc: soc@ffe000000 {
+ ranges = <0x00000000 0xf 0xfe000000 0x1000000>;
+ reg = <0xf 0xfe000000 0 0x00001000>;
+ spi@110000 {
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "micron,n25q512a";
+ reg = <0>;
+ spi-max-frequency = <10000000>; /* input clock */
+ };
+ };
+
+ i2c@118000 {
+ adt7481@4c {
+ compatible = "adi,adt7481";
+ reg = <0x4c>;
+ };
+
+ rtc@68 {
+ compatible = "dallas,ds1339";
+ reg = <0x68>;
+ interrupts = <0x1 0x1 0 0>;
+ };
+
+ eeprom@50 {
+ compatible = "at24,24c256";
+ reg = <0x50>;
+ };
+ };
+
+ i2c@118100 {
+ pca9546@77 {
+ compatible = "nxp,pca9546";
+ reg = <0x77>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ i2c@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x0>;
+
+ sfp@50 {
+ compatible = "optics,sfp";
+ reg = <0x50>;
+ };
+ };
+ };
+ };
+
+ sdhc@114000 {
+ voltage-ranges = <1800 1800 3300 3300>;
+ };
+ };
+
+ pci0: pcie@ffe240000 {
+ reg = <0xf 0xfe240000 0 0x10000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x00000000 0x0 0x20000000
+ 0x01000000 0 0x00000000 0xf 0xf8000000 0x0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci1: pcie@ffe250000 {
+ reg = <0xf 0xfe250000 0 0x10000>;
+ ranges = <0x02000000 0x0 0xe0000000 0xc 0x20000000 0x0 0x10000000
+ 0x01000000 0x0 0x00000000 0xf 0xf8010000 0x0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci2: pcie@ffe260000 {
+ reg = <0xf 0xfe260000 0 0x1000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x30000000 0 0x10000000
+ 0x01000000 0 0x00000000 0xf 0xf8020000 0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci3: pcie@ffe270000 {
+ reg = <0xf 0xfe270000 0 0x10000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x40000000 0 0x10000000
+ 0x01000000 0 0x00000000 0xf 0xf8030000 0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+};
diff --git a/arch/powerpc/platforms/85xx/Kconfig b/arch/powerpc/platforms/85xx/Kconfig
index e3578b7..a8a0f2f 100644
--- a/arch/powerpc/platforms/85xx/Kconfig
+++ b/arch/powerpc/platforms/85xx/Kconfig
@@ -259,7 +259,7 @@ config CORENET_GENERIC
For 32bit kernel, the following boards are supported:
P2041 RDB, P3041 DS and P4080 DS
For 64bit kernel, the following boards are supported:
- T208x QDS, T4240 QDS and B4 QDS
+ T208x QDS/RDB, T4240 QDS and B4 QDS
The following boards are supported for both 32bit and 64bit kernel:
P5020 DS and P5040 DS
diff --git a/arch/powerpc/platforms/85xx/corenet_generic.c b/arch/powerpc/platforms/85xx/corenet_generic.c
index d824454..a4fdb7e 100644
--- a/arch/powerpc/platforms/85xx/corenet_generic.c
+++ b/arch/powerpc/platforms/85xx/corenet_generic.c
@@ -103,6 +103,7 @@ static const char * const boards[] __initconst = {
"fsl,P5020DS",
"fsl,P5040DS",
"fsl,T2080QDS",
+ "fsl,T2080RDB",
"fsl,T2081QDS",
"fsl,T4240QDS",
"fsl,B4860QDS",
@@ -118,6 +119,7 @@ static const char * const hv_boards[] __initconst = {
"fsl,P5020DS-hv",
"fsl,P5040DS-hv",
"fsl,T2080QDS-hv",
+ "fsl,T2080RDB-hv",
"fsl,T2081QDS-hv",
"fsl,T4240QDS-hv",
"fsl,B4860QDS-hv",
--
1.8.0
^ permalink raw reply related
* [PATCH v2] powerpc/fsl-booke: Add initial T208x QDS board support
From: Shengzhou Liu @ 2014-05-23 8:27 UTC (permalink / raw)
To: linuxppc-dev, scottwood; +Cc: Shengzhou Liu
In-Reply-To: <1400833623-17547-1-git-send-email-Shengzhou.Liu@freescale.com>
Add support for Freescale T2080/T2081 QDS Development System Board.
The T2080QDS Development System is a high-performance computing,
evaluation, and development platform that supports T2080 QorIQ
Power Architecture processor, with following major features:
T2080QDS feature overview:
Processor:
- T2080 SoC integrating four 64-bit dual-threads e6500 cores up to 1.8GHz
Memory:
- Single memory controller capable of supporting DDR3 and DDR3-LP
- Dual DIMM slots up 2133MT/s with ECC
Ethernet interfaces:
- Two 1Gbps RGMII on-board ports
- Four 10Gbps XFI on-board cages
- 1Gbps/2.5Gbps SGMII Riser card
- 10Gbps XAUI Riser card
Accelerator:
- DPAA components consist of FMan, BMan, QMan, PME, DCE and SEC
SerDes:
- 16 lanes up to 10.3125GHz
- Supports Aurora debug, PEX, SATA, SGMII, sRIO, HiGig, XFI and XAUI
IFC:
- 128MB NOR Flash, 512MB NAND Flash, PromJet debug port and FPGA
eSPI:
- Three SPI flash (16MB N25Q128A + 8MB EN25S64 + 512KB SST25WF040)
USB:
- Two USB2.0 ports with internal PHY (one Type-A + one micro Type-AB)
PCIE:
- Four PCI Express controllers (two PCIe 2.0 and two PCIe 3.0, SR-IOV)
SATA:
- Two SATA 2.0 ports on-board
SRIO:
- Two Serial RapidIO 2.0 ports up to 5 GHz
eSDHC:
- Supports SD/MMC/eMMC Card
DMA:
- Three 8-channels DMA controllers
I2C:
- Four I2C controllers.
UART:
- Dual 4-pins UART serial ports
System Logic:
- QIXIS-II FPGA system controll
T2081QDS board shares the same PCB with T1040QDS with some differences.
Signed-off-by: Shengzhou Liu <Shengzhou.Liu@freescale.com>
---
v2: updated with some comments.
arch/powerpc/boot/dts/t2080qds.dts | 57 ++++++
arch/powerpc/boot/dts/t2081qds.dts | 46 +++++
arch/powerpc/boot/dts/t208xqds.dtsi | 239 ++++++++++++++++++++++++++
arch/powerpc/platforms/85xx/Kconfig | 2 +-
arch/powerpc/platforms/85xx/corenet_generic.c | 4 +
5 files changed, 347 insertions(+), 1 deletion(-)
create mode 100644 arch/powerpc/boot/dts/t2080qds.dts
create mode 100644 arch/powerpc/boot/dts/t2081qds.dts
create mode 100644 arch/powerpc/boot/dts/t208xqds.dtsi
diff --git a/arch/powerpc/boot/dts/t2080qds.dts b/arch/powerpc/boot/dts/t2080qds.dts
new file mode 100644
index 0000000..aa1d6d8
--- /dev/null
+++ b/arch/powerpc/boot/dts/t2080qds.dts
@@ -0,0 +1,57 @@
+/*
+ * T2080QDS Device Tree Source
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "fsl/t208xsi-pre.dtsi"
+/include/ "t208xqds.dtsi"
+
+/ {
+ model = "fsl,T2080QDS";
+ compatible = "fsl,T2080QDS";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ rio: rapidio@ffe0c0000 {
+ reg = <0xf 0xfe0c0000 0 0x11000>;
+
+ port1 {
+ ranges = <0 0 0xc 0x20000000 0 0x10000000>;
+ };
+ port2 {
+ ranges = <0 0 0xc 0x30000000 0 0x10000000>;
+ };
+ };
+};
+
+/include/ "fsl/t2080si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/t2081qds.dts b/arch/powerpc/boot/dts/t2081qds.dts
new file mode 100644
index 0000000..8ec80a7
--- /dev/null
+++ b/arch/powerpc/boot/dts/t2081qds.dts
@@ -0,0 +1,46 @@
+/*
+ * T2081QDS Device Tree Source
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/include/ "fsl/t208xsi-pre.dtsi"
+/include/ "t208xqds.dtsi"
+
+/ {
+ model = "fsl,T2081QDS";
+ compatible = "fsl,T2081QDS";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+};
+
+/include/ "fsl/t2081si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/t208xqds.dtsi b/arch/powerpc/boot/dts/t208xqds.dtsi
new file mode 100644
index 0000000..555dc6e
--- /dev/null
+++ b/arch/powerpc/boot/dts/t208xqds.dtsi
@@ -0,0 +1,239 @@
+/*
+ * T2080/T2081 QDS Device Tree Source
+ *
+ * Copyright 2013 Freescale Semiconductor Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Freescale Semiconductor nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/ {
+ model = "fsl,T2080QDS";
+ compatible = "fsl,T2080QDS";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&mpic>;
+
+ ifc: localbus@ffe124000 {
+ reg = <0xf 0xfe124000 0 0x2000>;
+ ranges = <0 0 0xf 0xe8000000 0x08000000
+ 2 0 0xf 0xff800000 0x00010000
+ 3 0 0xf 0xffdf0000 0x00008000>;
+
+ nor@0,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "cfi-flash";
+ reg = <0x0 0x0 0x8000000>;
+ bank-width = <2>;
+ device-width = <1>;
+ };
+
+ nand@2,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,ifc-nand";
+ reg = <0x2 0x0 0x10000>;
+ };
+
+ boardctrl: board-control@3,0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "fsl,fpga-qixis";
+ reg = <3 0 0x300>;
+ ranges = <0 3 0 0x300>;
+ };
+ };
+
+ memory {
+ device_type = "memory";
+ };
+
+ dcsr: dcsr@f00000000 {
+ ranges = <0x00000000 0xf 0x00000000 0x01072000>;
+ };
+
+ soc: soc@ffe000000 {
+ ranges = <0x00000000 0xf 0xfe000000 0x1000000>;
+ reg = <0xf 0xfe000000 0 0x00001000>;
+ spi@110000 {
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "micron,n25q128a11"; /* 16MB */
+ reg = <0>;
+ spi-max-frequency = <40000000>; /* input clock */
+ };
+
+ flash@1 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "sst,sst25wf040";
+ reg = <1>;
+ spi-max-frequency = <35000000>;
+ };
+
+ flash@2 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "eon,en25s64";
+ reg = <2>;
+ spi-max-frequency = <35000000>;
+ };
+ };
+
+ i2c@118000 {
+ pca9547@77 {
+ compatible = "nxp,pca9547";
+ reg = <0x77>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ i2c@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x0>;
+
+ eeprom@50 {
+ compatible = "at24,24c512";
+ reg = <0x50>;
+ };
+
+ eeprom@51 {
+ compatible = "at24,24c02";
+ reg = <0x51>;
+ };
+
+ eeprom@57 {
+ compatible = "at24,24c02";
+ reg = <0x57>;
+ };
+
+ rtc@68 {
+ compatible = "dallas,ds3232";
+ reg = <0x68>;
+ interrupts = <0x1 0x1 0 0>;
+ };
+ };
+
+ i2c@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x1>;
+
+ eeprom@55 {
+ compatible = "at24,24c02";
+ reg = <0x55>;
+ };
+ };
+
+ i2c@2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x2>;
+
+ ina220@40 {
+ compatible = "ti,ina220";
+ reg = <0x40>;
+ shunt-resistor = <1000>;
+ };
+
+ ina220@41 {
+ compatible = "ti,ina220";
+ reg = <0x41>;
+ shunt-resistor = <1000>;
+ };
+ };
+ };
+ };
+
+ sdhc@114000 {
+ voltage-ranges = <1800 1800 3300 3300>;
+ };
+ };
+
+ pci0: pcie@ffe240000 {
+ reg = <0xf 0xfe240000 0 0x10000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x00000000 0x0 0x20000000
+ 0x01000000 0 0x00000000 0xf 0xf8000000 0x0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci1: pcie@ffe250000 {
+ reg = <0xf 0xfe250000 0 0x10000>;
+ ranges = <0x02000000 0x0 0xe0000000 0xc 0x20000000 0x0 0x10000000
+ 0x01000000 0x0 0x00000000 0xf 0xf8010000 0x0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci2: pcie@ffe260000 {
+ reg = <0xf 0xfe260000 0 0x1000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x30000000 0 0x10000000
+ 0x01000000 0 0x00000000 0xf 0xf8020000 0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+
+ pci3: pcie@ffe270000 {
+ reg = <0xf 0xfe270000 0 0x10000>;
+ ranges = <0x02000000 0 0xe0000000 0xc 0x40000000 0 0x10000000
+ 0x01000000 0 0x00000000 0xf 0xf8030000 0 0x00010000>;
+ pcie@0 {
+ ranges = <0x02000000 0 0xe0000000
+ 0x02000000 0 0xe0000000
+ 0 0x20000000
+
+ 0x01000000 0 0x00000000
+ 0x01000000 0 0x00000000
+ 0 0x00010000>;
+ };
+ };
+};
diff --git a/arch/powerpc/platforms/85xx/Kconfig b/arch/powerpc/platforms/85xx/Kconfig
index 4d46349..e3578b7 100644
--- a/arch/powerpc/platforms/85xx/Kconfig
+++ b/arch/powerpc/platforms/85xx/Kconfig
@@ -259,7 +259,7 @@ config CORENET_GENERIC
For 32bit kernel, the following boards are supported:
P2041 RDB, P3041 DS and P4080 DS
For 64bit kernel, the following boards are supported:
- T4240 QDS and B4 QDS
+ T208x QDS, T4240 QDS and B4 QDS
The following boards are supported for both 32bit and 64bit kernel:
P5020 DS and P5040 DS
diff --git a/arch/powerpc/platforms/85xx/corenet_generic.c b/arch/powerpc/platforms/85xx/corenet_generic.c
index fbd871e..d824454 100644
--- a/arch/powerpc/platforms/85xx/corenet_generic.c
+++ b/arch/powerpc/platforms/85xx/corenet_generic.c
@@ -102,6 +102,8 @@ static const char * const boards[] __initconst = {
"fsl,P4080DS",
"fsl,P5020DS",
"fsl,P5040DS",
+ "fsl,T2080QDS",
+ "fsl,T2081QDS",
"fsl,T4240QDS",
"fsl,B4860QDS",
"fsl,B4420QDS",
@@ -115,6 +117,8 @@ static const char * const hv_boards[] __initconst = {
"fsl,P4080DS-hv",
"fsl,P5020DS-hv",
"fsl,P5040DS-hv",
+ "fsl,T2080QDS-hv",
+ "fsl,T2081QDS-hv",
"fsl,T4240QDS-hv",
"fsl,B4860QDS-hv",
"fsl,B4420QDS-hv",
--
1.8.0
^ permalink raw reply related
* [PATCH] tty/hvc/hvc_console: Fix wakeup of HVC thread on hvc_kick()
From: Benjamin Herrenschmidt @ 2014-05-23 9:41 UTC (permalink / raw)
To: Greg Kroah-Hartman; +Cc: linuxppc-dev list, Jiri Slaby, Linux Kernel list
Some backends call hvc_kick() to wakeup the HVC thread from its
slumber upon incoming characters. This however doesn't work
properly because it uses msleep_interruptible() which is mostly
immune to wake_up_process(). It will basically go back to sleep
until the timeout is expired (only signals can really wake it).
Replace it with a simple shedule_timeout_interruptible() instead,
which may wakeup earlier every now and then but we really don't
care in this case.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
drivers/tty/hvc/hvc_console.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/tty/hvc/hvc_console.c b/drivers/tty/hvc/hvc_console.c
index 94f9e3a..1094265 100644
--- a/drivers/tty/hvc/hvc_console.c
+++ b/drivers/tty/hvc/hvc_console.c
@@ -760,10 +760,17 @@ static int khvcd(void *unused)
if (poll_mask == 0)
schedule();
else {
+ unsigned long j_timeout;
+
if (timeout < MAX_TIMEOUT)
timeout += (timeout >> 6) + 1;
- msleep_interruptible(timeout);
+ /*
+ * We don't use msleep_interruptible otherwise
+ * "kick" will fail to wake us up
+ */
+ j_timeout = msecs_to_jiffies(timeout) + 1;
+ schedule_timeout_interruptible(j_timeout);
}
}
__set_current_state(TASK_RUNNING);
^ permalink raw reply related
* Re: [PATCH v2 0/6] Implement split core for POWER8
From: Alexander Graf @ 2014-05-23 9:53 UTC (permalink / raw)
To: Michael Neuling, Benjamin Herrenschmidt
Cc: linuxppc-dev, Paul Mackerras, kvm-ppc, kvm
In-Reply-To: <1400832930-17050-1-git-send-email-mikey@neuling.org>
On 23.05.14 10:15, Michael Neuling wrote:
> This patch series implements split core mode on POWER8. This enables up to 4
> subcores per core which can each independently run guests (per guest SPRs like
> SDR1, LPIDR etc are replicated per subcore). Lots more documentation on this
> feature in the code and commit messages.
>
> Most of this code is in the powernv platform but there's a couple of KVM
> specific patches too.
>
> Patch series authored by mpe and me with a few bug fixes from others.
>
> v2:
> There are some minor updates based on comments and I've added the Acks by
> Paulus and Alex for the KVM code.
I don't see changelogs inside the individual patches. Please make sure
to always mention what changed from one version to the next in a
particular patch, so that I have the chance to check whether that change
was good :).
Also, is there any performance penalty associated with split core mode?
If not, could we just always default to split-by-4 on POWER8 bare metal?
Alex
^ permalink raw reply
* Re: [PATCH v6 2/3] drivers/vfio: EEH support for VFIO PCI device
From: Alexander Graf @ 2014-05-23 9:58 UTC (permalink / raw)
To: Gavin Shan
Cc: aik@ozlabs.ru, kvm-ppc@vger.kernel.org, Alex Williamson,
qiudayu@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org
In-Reply-To: <20140523073720.GA5929@shangw>
On 23.05.14 09:37, Gavin Shan wrote:
> On Fri, May 23, 2014 at 08:55:15AM +0200, Alexander Graf wrote:
>>> Am 23.05.2014 um 06:37 schrieb Gavin Shan <gwshan@linux.vnet.ibm.com>:
>>>> On Thu, May 22, 2014 at 09:10:53PM -0600, Alex Williamson wrote:
>>>>> On Thu, 2014-05-22 at 18:23 +1000, Gavin Shan wrote:
>>>>> The patch adds new IOCTL commands for VFIO PCI device to support
>>>>> EEH functionality for PCI devices, which have been passed through
>>>>> from host to somebody else via VFIO.
> .../...
>
>>>>> +
>>>>> +/*
>>>>> + * Reset is the major step to recover problematic PE. The following
>>>>> + * command helps on that.
>>>>> + */
>>>>> +struct vfio_eeh_pe_reset {
>>>>> + __u32 argsz;
>>>>> + __u32 option;
>>>>> +};
>>>>> +
>>>>> +#define VFIO_EEH_PE_RESET _IO(VFIO_TYPE, VFIO_BASE + 24)
>>>>> +
>>>>> +/*
>>>>> + * One of the steps for recovery after PE reset is to configure the
>>>>> + * PCI bridges affected by the PE reset.
>>>>> + */
>>>>> +#define VFIO_EEH_PE_CONFIGURE _IO(VFIO_TYPE, VFIO_BASE + 25)
>>>> What can the user do differently by making these separate ioctls?
>>> hrm, I didn't understood as well. Alex.G could have the explaination.
>> Alex raised the same concern as me: why separate reset and configure? When we want to recover a device, we need a reset call anyway, right?
>>
> Ok. With current ioctl commands, "reset+configure" is required to do
> error recovery. Before the recovery, we also need call "configure"
> in order to retrieve error log correctly.
Well, the "configure" ioctl (which is a really bad name for what it does
btw) currently only restores the BARs which doesn't sound like error log
retrieval to me.
> Also, they corresponds to 2 separate RTAS services: "ibm,set-slot-reset"
> and "ibm,configure-pe".
Does a guest always issue both? What's the order it calls them in?
Alex
^ permalink raw reply
* Re: [PATCH v2 0/6] Implement split core for POWER8
From: Michael Neuling @ 2014-05-23 10:00 UTC (permalink / raw)
To: Alexander Graf; +Cc: kvm, kvm-ppc, Paul Mackerras, linuxppc-dev
In-Reply-To: <537F1A8D.9000704@suse.de>
On Fri, 2014-05-23 at 11:53 +0200, Alexander Graf wrote:
> On 23.05.14 10:15, Michael Neuling wrote:
> > This patch series implements split core mode on POWER8. This enables u=
p to 4
> > subcores per core which can each independently run guests (per guest SP=
Rs like
> > SDR1, LPIDR etc are replicated per subcore). Lots more documentation o=
n this
> > feature in the code and commit messages.
> >
> > Most of this code is in the powernv platform but there's a couple of KV=
M
> > specific patches too.
> >
> > Patch series authored by mpe and me with a few bug fixes from others.
> >
> > v2:
> > There are some minor updates based on comments and I've added the Ac=
ks by
> > Paulus and Alex for the KVM code.
>=20
> I don't see changelogs inside the individual patches. Please make sure=
=20
> to always mention what changed from one version to the next in a=20
> particular patch, so that I have the chance to check whether that change=
=20
> was good :).
Sure, that was a bit sloppy
Only the last patch was the only one that changed. I changed the sysfs
file from 600 permissions to 644 so that users can read it more easily
as requested by Joel.
The other change was to fix the possibility of a race when coming out of
nap and checking if we need to split. This fix was form paulus' (worked
offline).
> Also, is there any performance penalty associated with split core mode?=
=20
> If not, could we just always default to split-by-4 on POWER8 bare metal?
Yeah, there is a performance hit . When you are split (ie
subcores_per_core =3D 2 or 4), the core is stuck in SMT8 mode. So if you
only have 1 thread active (others napped), you won't get the benefit of
ST mode in the core (more register renames per HW thread, more FXUs,
more FPUs etc).
Mikey
^ permalink raw reply
* Re: [PATCH v2 0/6] Implement split core for POWER8
From: Alexander Graf @ 2014-05-23 10:05 UTC (permalink / raw)
To: Michael Neuling; +Cc: kvm, kvm-ppc, Paul Mackerras, linuxppc-dev
In-Reply-To: <1400839224.28224.5.camel@ale.ozlabs.ibm.com>
On 23.05.14 12:00, Michael Neuling wrote:
> On Fri, 2014-05-23 at 11:53 +0200, Alexander Graf wrote:
>> On 23.05.14 10:15, Michael Neuling wrote:
>>> This patch series implements split core mode on POWER8. This enables up to 4
>>> subcores per core which can each independently run guests (per guest SPRs like
>>> SDR1, LPIDR etc are replicated per subcore). Lots more documentation on this
>>> feature in the code and commit messages.
>>>
>>> Most of this code is in the powernv platform but there's a couple of KVM
>>> specific patches too.
>>>
>>> Patch series authored by mpe and me with a few bug fixes from others.
>>>
>>> v2:
>>> There are some minor updates based on comments and I've added the Acks by
>>> Paulus and Alex for the KVM code.
>> I don't see changelogs inside the individual patches. Please make sure
>> to always mention what changed from one version to the next in a
>> particular patch, so that I have the chance to check whether that change
>> was good :).
> Sure, that was a bit sloppy
>
> Only the last patch was the only one that changed. I changed the sysfs
> file from 600 permissions to 644 so that users can read it more easily
> as requested by Joel.
>
> The other change was to fix the possibility of a race when coming out of
> nap and checking if we need to split. This fix was form paulus' (worked
> offline).
>
>> Also, is there any performance penalty associated with split core mode?
>> If not, could we just always default to split-by-4 on POWER8 bare metal?
> Yeah, there is a performance hit . When you are split (ie
> subcores_per_core = 2 or 4), the core is stuck in SMT8 mode. So if you
> only have 1 thread active (others napped), you won't get the benefit of
> ST mode in the core (more register renames per HW thread, more FXUs,
> more FPUs etc).
Ok, imagine I have 1 core with SMT8. I have one process running at 100%
occupying one thread, the other 7 threads are idle.
Do I get performance benefits from having the other threads idle? Or do
I have to configure the system into SMT1 mode to get my ST benefits?
If it's the latter, we could just have ppc64_cpu --smt=x also set the
subcore amount in parallel to the thread count.
The reason I'm bringing this up is that I'm not quite sure who would be
the instance doing these performance tweaks. So I'd guess the majority
of users will simply miss out on them.
Alex
^ permalink raw reply
* Re: powerpc: remove checks for CONFIG_BOOK3E_MMU_TLB_STATS
From: Paul Bolle @ 2014-05-23 10:06 UTC (permalink / raw)
To: Scott Wood; +Cc: Paul Mackerras, linuxppc-dev, linux-kernel
In-Reply-To: <1400830406.31526.6.camel@x220>
On Fri, 2014-05-23 at 09:33 +0200, Paul Bolle wrote:
> Scott,
>
> On Thu, 2014-05-22 at 17:37 -0500, Scott Wood wrote:
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S: Assembler messages:
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:89: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:238: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:269: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:281: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:441: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:510: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:881: Error: unrecognized opcode: `tlb_miss_prolog_stats'
> > /home/scott/fsl/git/linux/upstream/arch/powerpc/mm/tlb_low_64e.S:918: Error: unrecognized opcode: `tlb_miss_prolog_stats'
>
> Thanks for testing!
>
> That's a bit surprising. The patch is intended to be a non event. Ie, it
> only removes what the preprocessor would have removed anyway. Unless I
> botched it, of course.
>
> What exactly did you test there?
For what it's worth: I can't reproduce this error with the cross
compiler now shipped with Fedora 20 (ie, powerpc64-linux-gnu-gcc (GCC)
4.8.1 20130717 (Red Hat 4.8.1-5)). It shows a nice and clean
AS arch/powerpc/mm/tlb_low_64e.o
in the output.
That's v3.15-rc6, with just this patch, and using a .config generated,
with "make oldconfig", from arch/powerpc/configs/ppc64e_defconfig.
Paul Bolle
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox