* [PATCH 3/3] powerpc/eeh: Emulate EEH recovery for VFIO devices
From: Gavin Shan @ 2014-09-23 1:09 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Gavin Shan
In-Reply-To: <1411434544-10025-1-git-send-email-gwshan@linux.vnet.ibm.com>
When enabling EEH functionality on passed through devices (PE)
with VFIO, the devices in the PE would be removed permanently
from guest side. In that case, the PE remains frozen state.
When returning PE to host, or restarting the guest again, we
had mechanism unfreezing the PE by clearing PESTA/B frozen
bits. However, that's not enough for some adapters, which are
indicated as following "lspci" shows. Those adapters require
hot reset on the parent bus to bring their firmware back to
workable state. Otherwise, those adaptrs won't be operative
and the host (for returning case) or the guest will fail to
load the drivers for those adapters without exception.
0000:01:00.0 Ethernet controller: Emulex Corporation OneConnect \
10Gb NIC (be3) (rev 02)
0000:01:00.0 0200: 19a2:0710 (rev 02)
0001:03:00.0 Ethernet controller: Emulex Corporation OneConnect \
NIC (Lancer) (rev 10)
0001:03:00.0 0200: 10df:e220 (rev 10)
The patch adds mechanism to emulate EEH recovery (for hot reset
on parent PCI bus) on 3 gates to fix the issue: open/release one
adapter of the PE, enable EEH functionality on one adapter of the
PE.
Reported-by: Murilo Fossa Vicentini <muvic@br.ibm.com>
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/kernel/eeh.c | 68 +++++++++++++++++++++++++++++-
arch/powerpc/kernel/eeh_driver.c | 90 ++++++++++++++++++++++++++++++++++++++--
3 files changed, 153 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 9bf4a5e..79c1952 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -282,6 +282,7 @@ void eeh_add_device_tree_late(struct pci_bus *);
void eeh_add_sysfs_files(struct pci_bus *);
void eeh_remove_device(struct pci_dev *);
int eeh_unfreeze_pe(struct eeh_pe *pe, bool sw_state);
+int eeh_pe_reset_and_recover(struct eeh_pe *pe);
int eeh_dev_open(struct pci_dev *pdev);
void eeh_dev_release(struct pci_dev *pdev);
struct eeh_pe *eeh_iommu_group_to_pe(struct iommu_group *group);
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index 059aa00..6c88d781 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -1193,6 +1193,69 @@ int eeh_unfreeze_pe(struct eeh_pe *pe, bool sw_state)
return ret;
}
+
+static struct pci_device_id eeh_reset_ids[] = {
+ { PCI_DEVICE(0x19a2, 0x0710) }, /* Emulex, BE */
+ { PCI_DEVICE(0x10df, 0xe220) }, /* Emulex, Lancer */
+ { 0 }
+};
+
+static int eeh_pe_change_owner(struct eeh_pe *pe)
+{
+ struct eeh_dev *edev, *tmp;
+ struct pci_dev *pdev;
+ struct pci_device_id *id;
+ int flags, ret;
+ bool need_reset = false;
+
+ /* Check PE state */
+ flags = (EEH_STATE_MMIO_ACTIVE | EEH_STATE_DMA_ACTIVE);
+ ret = eeh_ops->get_state(pe, NULL);
+ if (ret < 0 || ret == EEH_STATE_NOT_SUPPORT)
+ return 0;
+
+ /* Unfrozen PE, nothing to do */
+ if ((ret & flags) == flags)
+ return 0;
+
+ /* Frozen PE, check if it needs PE level reset */
+ eeh_pe_for_each_dev(pe, edev, tmp) {
+ pdev = eeh_dev_to_pci_dev(edev);
+ if (!pdev)
+ continue;
+
+ for (id = &eeh_reset_ids[0]; id->vendor != 0; id++) {
+ if (id->vendor != PCI_ANY_ID &&
+ id->vendor != pdev->vendor)
+ continue;
+ if (id->device != PCI_ANY_ID &&
+ id->device != pdev->device)
+ continue;
+ if (id->subvendor != PCI_ANY_ID &&
+ id->subvendor != pdev->subsystem_vendor)
+ continue;
+ if (id->subdevice != PCI_ANY_ID &&
+ id->subdevice != pdev->subsystem_device)
+ continue;
+
+ need_reset = true;
+ break;
+ }
+
+ /* Any one device in the PE requires PE reset,
+ * we should do that.
+ */
+ if (need_reset)
+ break;
+ }
+
+ /* Unfroze the PE if reset isn't required */
+ if (!need_reset)
+ return eeh_unfreeze_pe(pe, true);
+
+ return eeh_pe_reset_and_recover(pe);
+}
+
/**
* eeh_dev_open - Increase count of pass through devices for PE
* @pdev: PCI device
@@ -1224,7 +1287,7 @@ int eeh_dev_open(struct pci_dev *pdev)
* in frozen PE won't work properly. Clear the frozen state
* in advance.
*/
- ret = eeh_unfreeze_pe(edev->pe, true);
+ ret = eeh_pe_change_owner(edev->pe);
if (ret)
goto out;
@@ -1265,6 +1328,7 @@ void eeh_dev_release(struct pci_dev *pdev)
/* Decrease PE's pass through count */
atomic_dec(&edev->pe->pass_dev_cnt);
WARN_ON(atomic_read(&edev->pe->pass_dev_cnt) < 0);
+ eeh_pe_change_owner(edev->pe);
out:
mutex_unlock(&eeh_dev_mutex);
}
@@ -1345,7 +1409,7 @@ int eeh_pe_set_option(struct eeh_pe *pe, int option)
switch (option) {
case EEH_OPT_ENABLE:
if (eeh_enabled()) {
- ret = eeh_unfreeze_pe(pe, true);
+ ret = eeh_pe_change_owner(pe);
break;
}
ret = -EIO;
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 948e6f9..3fd514f 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -180,6 +180,22 @@ static bool eeh_dev_removed(struct eeh_dev *edev)
return false;
}
+static void *eeh_dev_save_state(void *data, void *userdata)
+{
+ struct eeh_dev *edev = data;
+ struct pci_dev *pdev;
+
+ if (!edev)
+ return NULL;
+
+ pdev = eeh_dev_to_pci_dev(edev);
+ if (!pdev)
+ return NULL;
+
+ pci_save_state(pdev);
+ return NULL;
+}
+
/**
* eeh_report_error - Report pci error to each device driver
* @data: eeh device
@@ -303,6 +319,22 @@ static void *eeh_report_reset(void *data, void *userdata)
return NULL;
}
+static void *eeh_dev_restore_state(void *data, void *userdata)
+{
+ struct eeh_dev *edev = data;
+ struct pci_dev *pdev;
+
+ if (!edev)
+ return NULL;
+
+ pdev = eeh_dev_to_pci_dev(edev);
+ if (!pdev)
+ return NULL;
+
+ pci_restore_state(pdev);
+ return NULL;
+}
+
/**
* eeh_report_resume - Tell device to resume normal operations
* @data: eeh device
@@ -450,10 +482,11 @@ static void *eeh_pe_detach_dev(void *data, void *userdata)
static void *__eeh_clear_pe_frozen_state(void *data, void *flag)
{
struct eeh_pe *pe = (struct eeh_pe *)data;
+ bool *clear_sw_state = flag;
int i, rc = 1;
for (i = 0; rc && i < 3; i++)
- rc = eeh_unfreeze_pe(pe, false);
+ rc = eeh_unfreeze_pe(pe, clear_sw_state);
/* Stop immediately on any errors */
if (rc) {
@@ -465,17 +498,66 @@ static void *__eeh_clear_pe_frozen_state(void *data, void *flag)
return NULL;
}
-static int eeh_clear_pe_frozen_state(struct eeh_pe *pe)
+static int eeh_clear_pe_frozen_state(struct eeh_pe *pe,
+ bool clear_sw_state)
{
void *rc;
- rc = eeh_pe_traverse(pe, __eeh_clear_pe_frozen_state, NULL);
+ rc = eeh_pe_traverse(pe, __eeh_clear_pe_frozen_state, &clear_sw_state);
if (!rc)
eeh_pe_state_clear(pe, EEH_PE_ISOLATED);
return rc ? -EIO : 0;
}
+int eeh_pe_reset_and_recover(struct eeh_pe *pe)
+{
+ int result, ret;
+
+ /* Bail if the PE is being recovered */
+ if (pe->state & EEH_PE_RECOVERING)
+ return 0;
+
+ /* Put the PE into recovery mode */
+ eeh_pe_state_mark(pe, EEH_PE_RECOVERING);
+
+ /* Save states */
+ eeh_pe_dev_traverse(pe, eeh_dev_save_state, NULL);
+
+ /* Report error */
+ eeh_pe_dev_traverse(pe, eeh_report_error, &result);
+
+ /* Issue reset */
+ eeh_pe_state_mark(pe, EEH_PE_RESET);
+ ret = eeh_reset_pe(pe);
+ if (ret) {
+ eeh_pe_state_clear(pe, EEH_PE_RECOVERING | EEH_PE_RESET);
+ return ret;
+ }
+ eeh_pe_state_clear(pe, EEH_PE_RESET);
+
+ /* Unfreeze the PE */
+ ret = eeh_clear_pe_frozen_state(pe, true);
+ if (ret) {
+ eeh_pe_state_clear(pe, EEH_PE_RECOVERING);
+ return ret;
+ }
+
+ /* Notify completion of reset */
+ eeh_pe_dev_traverse(pe, eeh_report_reset, &result);
+
+ /* Restore device state */
+ eeh_pe_dev_traverse(pe, eeh_dev_restore_state, NULL);
+
+ /* Resume */
+ eeh_pe_dev_traverse(pe, eeh_report_resume, NULL);
+
+ /* Clear recovery mode */
+ eeh_pe_state_clear(pe, EEH_PE_RECOVERING);
+
+ return 0;
+}
+
/**
* eeh_reset_device - Perform actual reset of a pci slot
* @pe: EEH PE
@@ -534,7 +616,7 @@ static int eeh_reset_device(struct eeh_pe *pe, struct pci_bus *bus)
eeh_pe_state_clear(pe, EEH_PE_RESET);
/* Clear frozen state */
- rc = eeh_clear_pe_frozen_state(pe);
+ rc = eeh_clear_pe_frozen_state(pe, false);
if (rc)
return rc;
--
1.8.3.2
^ permalink raw reply related
* [PATCH 2/3] powerpc/eeh: Tag reset state for user owned PE
From: Gavin Shan @ 2014-09-23 1:09 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Gavin Shan
In-Reply-To: <1411434544-10025-1-git-send-email-gwshan@linux.vnet.ibm.com>
PE would be owned by userland, which probably request PE reset
done in host side. During the reset, we should drop the PCI
config accesses to the PE with help of flag EEH_PE_RESET.
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index 545860f..059aa00 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -1463,6 +1463,7 @@ int eeh_pe_reset(struct eeh_pe *pe, int option)
switch (option) {
case EEH_RESET_DEACTIVATE:
ret = eeh_ops->reset(pe, option);
+ eeh_pe_state_clear(pe, EEH_PE_RESET);
if (ret)
break;
@@ -1477,6 +1478,7 @@ int eeh_pe_reset(struct eeh_pe *pe, int option)
*/
eeh_ops->set_option(pe, EEH_OPT_FREEZE_PE);
+ eeh_pe_state_mark(pe, EEH_PE_RESET);
ret = eeh_ops->reset(pe, option);
break;
default:
--
1.8.3.2
^ permalink raw reply related
* [PATCH 1/3] powerpc/powernv: Sync OpalPciResetScope with firmware
From: Gavin Shan @ 2014-09-23 1:09 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Gavin Shan
The names of PCI reset scopes aren't sychronized with firmware.
The patch fixes it.
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/opal.h | 9 ++++++---
arch/powerpc/platforms/powernv/eeh-ioda.c | 12 ++++++------
arch/powerpc/platforms/powernv/pci-ioda.c | 4 ++--
3 files changed, 14 insertions(+), 11 deletions(-)
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9113653..bdf9f61 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -385,9 +385,12 @@ enum OpalM64EnableAction {
};
enum OpalPciResetScope {
- OPAL_PHB_COMPLETE = 1, OPAL_PCI_LINK = 2, OPAL_PHB_ERROR = 3,
- OPAL_PCI_HOT_RESET = 4, OPAL_PCI_FUNDAMENTAL_RESET = 5,
- OPAL_PCI_IODA_TABLE_RESET = 6,
+ OPAL_RESET_PHB_COMPLETE = 1,
+ OPAL_RESET_PCI_LINK = 2,
+ OPAL_RESET_PHB_ERROR = 3,
+ OPAL_RESET_PCI_HOT = 4,
+ OPAL_RESET_PCI_FUNDAMENTAL = 5,
+ OPAL_RESET_PCI_IODA_TABLE = 6
};
enum OpalPciReinitScope {
diff --git a/arch/powerpc/platforms/powernv/eeh-ioda.c b/arch/powerpc/platforms/powernv/eeh-ioda.c
index 729e445..f248586 100644
--- a/arch/powerpc/platforms/powernv/eeh-ioda.c
+++ b/arch/powerpc/platforms/powernv/eeh-ioda.c
@@ -514,11 +514,11 @@ int ioda_eeh_phb_reset(struct pci_controller *hose, int option)
if (option == EEH_RESET_FUNDAMENTAL ||
option == EEH_RESET_HOT)
rc = opal_pci_reset(phb->opal_id,
- OPAL_PHB_COMPLETE,
+ OPAL_RESET_PHB_COMPLETE,
OPAL_ASSERT_RESET);
else if (option == EEH_RESET_DEACTIVATE)
rc = opal_pci_reset(phb->opal_id,
- OPAL_PHB_COMPLETE,
+ OPAL_RESET_PHB_COMPLETE,
OPAL_DEASSERT_RESET);
if (rc < 0)
goto out;
@@ -558,15 +558,15 @@ static int ioda_eeh_root_reset(struct pci_controller *hose, int option)
*/
if (option == EEH_RESET_FUNDAMENTAL)
rc = opal_pci_reset(phb->opal_id,
- OPAL_PCI_FUNDAMENTAL_RESET,
+ OPAL_RESET_PCI_FUNDAMENTAL,
OPAL_ASSERT_RESET);
else if (option == EEH_RESET_HOT)
rc = opal_pci_reset(phb->opal_id,
- OPAL_PCI_HOT_RESET,
+ OPAL_RESET_PCI_HOT,
OPAL_ASSERT_RESET);
else if (option == EEH_RESET_DEACTIVATE)
rc = opal_pci_reset(phb->opal_id,
- OPAL_PCI_HOT_RESET,
+ OPAL_RESET_PCI_HOT,
OPAL_DEASSERT_RESET);
if (rc < 0)
goto out;
@@ -697,7 +697,7 @@ static int ioda_eeh_reset(struct eeh_pe *pe, int option)
(option == EEH_RESET_HOT ||
option == EEH_RESET_FUNDAMENTAL)) {
rc = opal_pci_reset(phb->opal_id,
- OPAL_PHB_ERROR,
+ OPAL_RESET_PHB_ERROR,
OPAL_ASSERT_RESET);
if (rc != OPAL_SUCCESS) {
pr_warn("%s: Failure %lld clearing "
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index df241b1..36b1a7a 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1627,7 +1627,7 @@ static u32 pnv_ioda_bdfn_to_pe(struct pnv_phb *phb, struct pci_bus *bus,
static void pnv_pci_ioda_shutdown(struct pnv_phb *phb)
{
- opal_pci_reset(phb->opal_id, OPAL_PCI_IODA_TABLE_RESET,
+ opal_pci_reset(phb->opal_id, OPAL_RESET_PCI_IODA_TABLE,
OPAL_ASSERT_RESET);
}
@@ -1803,7 +1803,7 @@ void __init pnv_pci_init_ioda_phb(struct device_node *np,
pci_add_flags(PCI_REASSIGN_ALL_RSRC);
/* Reset IODA tables to a clean state */
- rc = opal_pci_reset(phb_id, OPAL_PCI_IODA_TABLE_RESET, OPAL_ASSERT_RESET);
+ rc = opal_pci_reset(phb_id, OPAL_RESET_PCI_IODA_TABLE, OPAL_ASSERT_RESET);
if (rc)
pr_warning(" OPAL Error %ld performing IODA table reset !\n", rc);
--
1.8.3.2
^ permalink raw reply related
* Re: [3/5] pseries: Create device hotplug entry point
From: Tyrel Datwyler @ 2014-09-23 1:15 UTC (permalink / raw)
To: Nathan Fontenot, Michael Ellerman; +Cc: linuxppc-dev
In-Reply-To: <5419DDB8.1070001@linux.vnet.ibm.com>
On 09/17/2014 12:15 PM, Nathan Fontenot wrote:
> On 09/17/2014 02:07 AM, Michael Ellerman wrote:
>>
>> On Mon, 2014-09-15 at 15:31 -0500, Nathan Fontenot wrote:
>>> For pseries system the kernel will be notified of hotplug requests in
>>> the form of rtas hotplug events.
>>
>> Can you flesh that design out a bit for me, I don't entirely get how it's going
>> to work.
>>
>> The kernel gets the rtas hotplug events (in rtasd.c) and spits them out to
>> userspace, which then writes them back in ?
>>
>>> This patch creates a common routine that can handle these requests in both
>>> the PowerVM anbd PowerKVM environments, handle_dlpar_errorlog(). This also
>> ^
>>> creates the initial memory hotplug request handling stub.
>>>
>>> For PowerVM this patch also creates a new /proc file that the drmgr
>>> command will use to write rtas hotplug events to.
>>
>> Why is this different between phyp and KVM?
>>
>>> For future PowerKVM handling the rtas check-exception code can pass
>>> any rtas hotplug events received to handle_dlpar_errorlog().
>>
>> Internally to the kernel you mean?
>>
>
> Perhaps a better explanation of how things work today and where I see
> them going is needed. I was trying to avoid a long explanation and I
> don't think my shortened explanation worked. I'll include this in v2
> of the patchset too.
>
> The current hotplug (or dlpar) of devices (the process is generally the
> same for memory, cpu, and pci) on PowerVM systems is initiated
> from the HMC, which communicates the request to the partitions through
> the RSCT framework. The RSCT framework then invokes the drmgr command.
> The drmgr command performs the hotplug operation by doing some pieces,
> such as most of the rtas calls and device tree parsing, in userspace
> and make requests to the kernel to online/offline the device, update the
> device tree and add/remove the device.
>
> For PowerKVM the approach is to follow what is currently being done for
> pci hotplug. A hotplug request is initiated from the host. QEMU then
> sends an EPOW interrupt to the guest which causes the guest to make the
> rtas,check-exception call. In QEMU, the rtas,check-exception call
> returns a rtas hotplug event to the guest. I was using this same framework
> to also enable memory (and next cpu) hotplug.
>
> You are correct that the current pci hotplug path for PowerKVM involves
> the kernel receiving the rtas event, passing it to rtas_errd in userspace,
> and having rtas_errd invoke drmgr. The drmgr command then handles the request
> as described above for PowerVM systems.
>
> There is no need for this circuitous route, we should just handle the entire
> hotplug of devices in the kernel. What I am hoping to do is to enable this
> by moving the code to handle hotplug from drmgr into the kernel and
> provide a single path for handling hotplug for PowerVM and PowerKVM. To
> make this work for PowerKVM we will update the kernel rtas code to
> recognize rtas hotplug events returned from rtas,check-exception calls
> and call handle_dlpar_errorlog(). The hotplug rtas event is never sent out
> to userspace.
Wouldn't we still want the event surfaced to userspace so that it can at
least be logged?
-Tyrel
>
> For PowerVM systems, I created the /proc/powerpc/dlpar file that a rtas
> hotplug event can be written to and passed to handle_dlpar_errorlog().
> There is no chance of updating how we receive hotplug requests on PowerVM
> systems.
>
> Hopefully that explains the design better.
>
>>> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
>>> index a2450b8..574ec73 100644
>>> --- a/arch/powerpc/platforms/pseries/dlpar.c
>>> +++ b/arch/powerpc/platforms/pseries/dlpar.c
>>> @@ -16,7 +16,9 @@
>>> #include <linux/cpu.h>
>>> #include <linux/slab.h>
>>> #include <linux/of.h>
>>> +#include <linux/proc_fs.h>
>>> #include "offline_states.h"
>>> +#include "pseries.h"
>>>
>>> #include <asm/prom.h>
>>> #include <asm/machdep.h>
>>> @@ -530,13 +532,72 @@ static ssize_t dlpar_cpu_release(const char *buf, size_t count)
>>> return count;
>>> }
>>>
>>> +#endif /* CONFIG_ARCH_CPU_PROBE_RELEASE */
>>
>> That is really confusing, but I think it's just a diff artifact?
>
> Yes, diff artifact.
>
>>
>>> +static int handle_dlpar_errorlog(struct rtas_error_log *error_log)
>>> +{
>>> + struct pseries_errorlog *pseries_log;
>>> + struct pseries_hp_errorlog *hp_elog;
>>> + int rc = -EINVAL;
>>> +
>>> + pseries_log = get_pseries_errorlog(error_log,
>>> + PSERIES_ELOG_SECT_ID_HOTPLUG);
>>> + if (!pseries_log)
>>> + return rc;
>>> +
>>> + hp_elog = (struct pseries_hp_errorlog *)pseries_log->data;
>>> + if (!hp_elog)
>>> + return rc;
>>
>> I don't see how that can happen?
>>
>> struct pseries_errorlog {
>> __be16 id; /* 0x00 2-byte ASCII section ID */
>> __be16 length; /* 0x02 Section length in bytes */
>> uint8_t version; /* 0x04 Section version */
>> uint8_t subtype; /* 0x05 Section subtype */
>> __be16 creator_component; /* 0x06 Creator component ID */
>> uint8_t data[]; /* 0x08 Start of section data */
>> };
>>
>> Should you be checking for length == 0 instead ?
>>
>
> You are correct.
>
>> Also I think the code will probably end up cleaner if you do the endian
>> conversions immediately when you read the hp_elog, rather than passing it
>> around in BE and having to remember to convert at all the usages.
>>
>
> Agreed. I'll clean that up.
>
>>> + switch (hp_elog->resource) {
>>> + case PSERIES_HP_ELOG_RESOURCE_MEM:
>>> + rc = dlpar_memory(hp_elog);
>>> + break;
>>
>> Please add:
>>
>> default:
>> pr_warn_ratelimited("Unknown resource ..")
>>
>> Or something.
>
> can do.
>
>>
>>
>>> + }
>>> +
>>> + return rc;
>>> +}
>>> +
>>> +static ssize_t dlpar_write(struct file *file, const char __user *buf,
>>> + size_t count, loff_t *offset)
>>> +{
>>> + char *event_buf;
>>> + int rc;
>>> +
>>> + event_buf = kmalloc(count + 1, GFP_KERNEL);
>>
>> Why + 1 ? It's not null-terminated AFAICS.
>
> I think that's just habit. You're correct, it's not needed.
>
>>
>>> + if (!event_buf)
>>> + return -ENOMEM;
>>> +
>>> + rc = copy_from_user(event_buf, buf, count);
>>> + if (rc) {
>>> + kfree(event_buf);
>>> + return rc;
>>> + }
>>> +
>>> + rc = handle_dlpar_errorlog((struct rtas_error_log *)event_buf);
>>
>> If you start with a struct rtas_error_log * you shouldn't need any casts.
>
> good point.
>
>>
>>> + kfree(event_buf);
>>> + return rc ? rc : count;
>>> +}
>>> +
>>> +static const struct file_operations dlpar_fops = {
>>> + .write = dlpar_write,
>>> + .llseek = noop_llseek,
>>> +};
>>> +
>>> static int __init pseries_dlpar_init(void)
>>> {
>>> + struct proc_dir_entry *proc_ent;
>>> +
>>> + proc_ent = proc_create("powerpc/dlpar", S_IWUSR, NULL, &dlpar_fops);
>>> + if (proc_ent)
>>> + proc_set_size(proc_ent, 0);
>>
>> else
>> error message at least please
>>
>> Why are we putting it in /proc, can't it go in /sys/kernel like the mobility
>> stuff?
>
> I have no personal preference, I'll move it to /sys/kernel and update the
> creation handling.
>
>>
>>> diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
>>> index 24abc5c..0e60e15 100644
>>> --- a/arch/powerpc/platforms/pseries/hotplug-memory.c
>>> +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
>>> @@ -20,6 +22,9 @@
>>> #include <asm/machdep.h>
>>> #include <asm/prom.h>
>>> #include <asm/sparsemem.h>
>>> +#include <asm/rtas.h>
>>> +
>>> +DEFINE_MUTEX(dlpar_mem_mutex);
>>
>> static ?
>
> yes, that should have been static.
>
>>
>>> diff --git a/arch/powerpc/platforms/pseries/pseries.h b/arch/powerpc/platforms/pseries/pseries.h
>>> index b94516b..28bd994 100644
>>> --- a/arch/powerpc/platforms/pseries/pseries.h
>>> +++ b/arch/powerpc/platforms/pseries/pseries.h
>>> @@ -62,6 +63,15 @@ extern int dlpar_detach_node(struct device_node *);
>>> extern int dlpar_acquire_drc(u32);
>>> extern int dlpar_release_drc(u32);
>>>
>>> +#ifdef CONFIG_MEMORY_HOTPLUG
>>> +extern int dlpar_memory(struct pseries_hp_errorlog *);
>>> +#else
>>> +static inline int dlpar_memory(struct pseries_hp_errorlog *hp_elog)
>>> +{
>>> + return -ENOTSUPP;
>>
>> EOPNOTSUPP is a bit more standard.
>
> ok.
>
> Thanks for all the feedback.
>
> -Nathan
>
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
>
^ permalink raw reply
* [PATCH v2] powerpc/iommu/ddw: Fix endianness
From: Alexey Kardashevskiy @ 2014-09-23 1:58 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: linuxppc-dev, Anton Blanchard, linux-kernel
rtas_call() accepts and returns values in CPU endianness.
The ddw_query_response and ddw_create_response structs members are
defined and treated as BE but as they are passed to rtas_call() as
(u32 *) and they get byteswapped automatically, the data is actually
CPU-endian. This fixes ddw_query_response and ddw_create_response
definitions and use.
of_read_number() is designed to work with device tree cells - it assumes
the input is big-endian and returns data in CPU-endian. However due
to the ddw_create_response struct fix, create.addr_hi/lo are already
CPU-endian so do not byteswap them.
ddw_avail is a pointer to the "ibm,ddw-applicable" property which contains
3 cells which are big-endian as it is a device tree. rtas_call() accepts
a RTAS token in CPU-endian. This converts RTAS tokens from big-endian to
CPU-endian. Since every token is used once till guest is rebooted, there is
no much sense in caching RTAS tokens in CPU-endian.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
v2:
* updated commit log
* fixed definition of ddw_query_response and ddw_create_response
---
arch/powerpc/platforms/pseries/iommu.c | 38 ++++++++++++++++++----------------
1 file changed, 20 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 4642d6a..f052cc8 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -329,16 +329,16 @@ struct direct_window {
/* Dynamic DMA Window support */
struct ddw_query_response {
- __be32 windows_available;
- __be32 largest_available_block;
- __be32 page_size;
- __be32 migration_capable;
+ u32 windows_available;
+ u32 largest_available_block;
+ u32 page_size;
+ u32 migration_capable;
};
struct ddw_create_response {
- __be32 liobn;
- __be32 addr_hi;
- __be32 addr_lo;
+ u32 liobn;
+ u32 addr_hi;
+ u32 addr_lo;
};
static LIST_HEAD(direct_window_list);
@@ -750,7 +750,7 @@ static void remove_ddw(struct device_node *np, bool remove_prop)
pr_debug("%s successfully cleared tces in window.\n",
np->full_name);
- ret = rtas_call(ddw_avail[2], 1, 1, NULL, liobn);
+ ret = rtas_call(be32_to_cpu(ddw_avail[2]), 1, 1, NULL, liobn);
if (ret)
pr_warning("%s: failed to remove direct window: rtas returned "
"%d to ibm,remove-pe-dma-window(%x) %llx\n",
@@ -841,7 +841,7 @@ static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail,
cfg_addr = edev->pe_config_addr;
buid = edev->phb->buid;
- ret = rtas_call(ddw_avail[0], 3, 5, (u32 *)query,
+ ret = rtas_call(be32_to_cpu(ddw_avail[0]), 3, 5, (u32 *)query,
cfg_addr, BUID_HI(buid), BUID_LO(buid));
dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %x"
" returned %d\n", ddw_avail[0], cfg_addr, BUID_HI(buid),
@@ -872,8 +872,9 @@ static int create_ddw(struct pci_dev *dev, const u32 *ddw_avail,
do {
/* extra outputs are LIOBN and dma-addr (hi, lo) */
- ret = rtas_call(ddw_avail[1], 5, 4, (u32 *)create, cfg_addr,
- BUID_HI(buid), BUID_LO(buid), page_shift, window_shift);
+ ret = rtas_call(be32_to_cpu(ddw_avail[1]), 5, 4, (u32 *)create,
+ cfg_addr, BUID_HI(buid), BUID_LO(buid),
+ page_shift, window_shift);
} while (rtas_busy_delay(ret));
dev_info(&dev->dev,
"ibm,create-pe-dma-window(%x) %x %x %x %x %x returned %d "
@@ -966,11 +967,11 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
dev_dbg(&dev->dev, "no free dynamic windows");
goto out_failed;
}
- if (be32_to_cpu(query.page_size) & 4) {
+ if (query.page_size & 4) {
page_shift = 24; /* 16MB */
- } else if (be32_to_cpu(query.page_size) & 2) {
+ } else if (query.page_size & 2) {
page_shift = 16; /* 64kB */
- } else if (be32_to_cpu(query.page_size) & 1) {
+ } else if (query.page_size & 1) {
page_shift = 12; /* 4kB */
} else {
dev_dbg(&dev->dev, "no supported direct page size in mask %x",
@@ -980,7 +981,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
/* verify the window * number of ptes will map the partition */
/* check largest block * page size > max memory hotplug addr */
max_addr = memory_hotplug_max();
- if (be32_to_cpu(query.largest_available_block) < (max_addr >> page_shift)) {
+ if (query.largest_available_block < (max_addr >> page_shift)) {
dev_dbg(&dev->dev, "can't map partiton max 0x%llx with %u "
"%llu-sized pages\n", max_addr, query.largest_available_block,
1ULL << page_shift);
@@ -1006,8 +1007,9 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
if (ret != 0)
goto out_free_prop;
- ddwprop->liobn = create.liobn;
- ddwprop->dma_base = cpu_to_be64(of_read_number(&create.addr_hi, 2));
+ ddwprop->liobn = cpu_to_be32(create.liobn);
+ ddwprop->dma_base = cpu_to_be64(((u64)create.addr_hi << 32) |
+ create.addr_lo);
ddwprop->tce_shift = cpu_to_be32(page_shift);
ddwprop->window_shift = cpu_to_be32(len);
@@ -1039,7 +1041,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
list_add(&window->list, &direct_window_list);
spin_unlock(&direct_window_list_lock);
- dma_addr = of_read_number(&create.addr_hi, 2);
+ dma_addr = be64_to_cpu(ddwprop->dma_base);
goto out_unlock;
out_free_window:
--
2.0.0
^ permalink raw reply related
* [PATCH v2 01/13] powerpc/iommu: Check that TCE page size is equal to it_page_size
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
This checks that the TCE table page size is not bigger that the size of
a page we just pinned and going to put its physical address to the table.
Otherwise the hardware gets unwanted access to physical memory between
the end of the actual page and the end of the aligned up TCE page.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/kernel/iommu.c | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index a10642a..b378f78 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -38,6 +38,7 @@
#include <linux/pci.h>
#include <linux/iommu.h>
#include <linux/sched.h>
+#include <linux/hugetlb.h>
#include <asm/io.h>
#include <asm/prom.h>
#include <asm/iommu.h>
@@ -1059,16 +1060,37 @@ int iommu_put_tce_user_mode(struct iommu_table *tbl, unsigned long entry,
tce, entry << tbl->it_page_shift, ret); */
return -EFAULT;
}
+
+ /*
+ * Check that the TCE table granularity is not bigger than the size of
+ * a page we just found. Otherwise the hardware can get access to
+ * a bigger memory chunk that it should.
+ */
+ if (PageHuge(page)) {
+ struct page *head = compound_head(page);
+ long shift = PAGE_SHIFT + compound_order(head);
+
+ if (shift < tbl->it_page_shift) {
+ ret = -EINVAL;
+ goto put_page_exit;
+ }
+
+ }
+
hwaddr = (unsigned long) page_address(page) + offset;
ret = iommu_tce_build(tbl, entry, hwaddr, direction);
if (ret)
- put_page(page);
+ goto put_page_exit;
- if (ret < 0)
- pr_err("iommu_tce: %s failed ioba=%lx, tce=%lx, ret=%d\n",
+ return 0;
+
+put_page_exit:
+ pr_err("iommu_tce: %s failed ioba=%lx, tce=%lx, ret=%d\n",
__func__, entry << tbl->it_page_shift, tce, ret);
+ put_page(page);
+
return ret;
}
EXPORT_SYMBOL_GPL(iommu_put_tce_user_mode);
--
2.0.0
^ permalink raw reply related
* [PATCH v2 02/13] powerpc/powernv: Make invalidate() a callback
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
At the moment pnv_pci_ioda_tce_invalidate() gets the PE pointer via
container_of(tbl). Since we are going to have to add Dynamic DMA windows
and that means having 2 IOMMU tables per PE, this is not going to work.
This implements pnv_pci_ioda(1|2)_tce_invalidate as a pnv_ioda_pe callback.
This adds a pnv_iommu_table wrapper around iommu_table and stores a pointer
to PE there. PNV's ppc_md.tce_build() call uses this to find PE and
do the invalidation. This will be used later for Dynamic DMA windows too.
This registers invalidate() callbacks for IODA1 and IODA2:
- pnv_pci_ioda1_tce_invalidate;
- pnv_pci_ioda2_tce_invalidate.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
v4:
* changed commit log to explain why this change is needed
---
arch/powerpc/platforms/powernv/pci-ioda.c | 35 ++++++++++++-------------------
arch/powerpc/platforms/powernv/pci.c | 31 ++++++++++++++++++++-------
arch/powerpc/platforms/powernv/pci.h | 13 +++++++++++-
3 files changed, 48 insertions(+), 31 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index df241b1..136e765 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -857,7 +857,7 @@ static void pnv_pci_ioda_dma_dev_setup(struct pnv_phb *phb, struct pci_dev *pdev
pe = &phb->ioda.pe_array[pdn->pe_number];
WARN_ON(get_dma_ops(&pdev->dev) != &dma_iommu_ops);
- set_iommu_table_base_and_group(&pdev->dev, &pe->tce32_table);
+ set_iommu_table_base_and_group(&pdev->dev, &pe->tce32.table);
}
static int pnv_pci_ioda_dma_set_mask(struct pnv_phb *phb,
@@ -884,7 +884,7 @@ static int pnv_pci_ioda_dma_set_mask(struct pnv_phb *phb,
} else {
dev_info(&pdev->dev, "Using 32-bit DMA via iommu\n");
set_dma_ops(&pdev->dev, &dma_iommu_ops);
- set_iommu_table_base(&pdev->dev, &pe->tce32_table);
+ set_iommu_table_base(&pdev->dev, &pe->tce32.table);
}
*pdev->dev.dma_mask = dma_mask;
return 0;
@@ -899,9 +899,9 @@ static void pnv_ioda_setup_bus_dma(struct pnv_ioda_pe *pe,
list_for_each_entry(dev, &bus->devices, bus_list) {
if (add_to_iommu_group)
set_iommu_table_base_and_group(&dev->dev,
- &pe->tce32_table);
+ &pe->tce32.table);
else
- set_iommu_table_base(&dev->dev, &pe->tce32_table);
+ set_iommu_table_base(&dev->dev, &pe->tce32.table);
if (dev->subordinate)
pnv_ioda_setup_bus_dma(pe, dev->subordinate,
@@ -988,19 +988,6 @@ static void pnv_pci_ioda2_tce_invalidate(struct pnv_ioda_pe *pe,
}
}
-void pnv_pci_ioda_tce_invalidate(struct iommu_table *tbl,
- __be64 *startp, __be64 *endp, bool rm)
-{
- struct pnv_ioda_pe *pe = container_of(tbl, struct pnv_ioda_pe,
- tce32_table);
- struct pnv_phb *phb = pe->phb;
-
- if (phb->type == PNV_PHB_IODA1)
- pnv_pci_ioda1_tce_invalidate(pe, tbl, startp, endp, rm);
- else
- pnv_pci_ioda2_tce_invalidate(pe, tbl, startp, endp, rm);
-}
-
static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
struct pnv_ioda_pe *pe, unsigned int base,
unsigned int segs)
@@ -1058,9 +1045,11 @@ static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
}
/* Setup linux iommu table */
- tbl = &pe->tce32_table;
+ tbl = &pe->tce32.table;
pnv_pci_setup_iommu_table(tbl, addr, TCE32_TABLE_SIZE * segs,
base << 28, IOMMU_PAGE_SHIFT_4K);
+ pe->tce32.pe = pe;
+ pe->tce32.invalidate_fn = pnv_pci_ioda1_tce_invalidate;
/* OPAL variant of P7IOC SW invalidated TCEs */
swinvp = of_get_property(phb->hose->dn, "ibm,opal-tce-kill", NULL);
@@ -1097,7 +1086,7 @@ static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
static void pnv_pci_ioda2_set_bypass(struct iommu_table *tbl, bool enable)
{
struct pnv_ioda_pe *pe = container_of(tbl, struct pnv_ioda_pe,
- tce32_table);
+ tce32.table);
uint16_t window_id = (pe->pe_number << 1 ) + 1;
int64_t rc;
@@ -1142,10 +1131,10 @@ static void pnv_pci_ioda2_setup_bypass_pe(struct pnv_phb *phb,
pe->tce_bypass_base = 1ull << 59;
/* Install set_bypass callback for VFIO */
- pe->tce32_table.set_bypass = pnv_pci_ioda2_set_bypass;
+ pe->tce32.table.set_bypass = pnv_pci_ioda2_set_bypass;
/* Enable bypass by default */
- pnv_pci_ioda2_set_bypass(&pe->tce32_table, true);
+ pnv_pci_ioda2_set_bypass(&pe->tce32.table, true);
}
static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
@@ -1193,9 +1182,11 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
}
/* Setup linux iommu table */
- tbl = &pe->tce32_table;
+ tbl = &pe->tce32.table;
pnv_pci_setup_iommu_table(tbl, addr, tce_table_size, 0,
IOMMU_PAGE_SHIFT_4K);
+ pe->tce32.pe = pe;
+ pe->tce32.invalidate_fn = pnv_pci_ioda2_tce_invalidate;
/* OPAL variant of PHB3 invalidated TCEs */
swinvp = of_get_property(phb->hose->dn, "ibm,opal-tce-kill", NULL);
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index b854b57..97895d4 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -599,6 +599,27 @@ struct pci_ops pnv_pci_ops = {
.write = pnv_pci_write_config,
};
+static void pnv_tce_invalidate(struct iommu_table *tbl, __be64 *startp,
+ __be64 *endp, bool rm)
+{
+ struct pnv_iommu_table *ptbl = container_of(tbl,
+ struct pnv_iommu_table, table);
+ struct pnv_ioda_pe *pe = ptbl->pe;
+
+ /*
+ * Some implementations won't cache invalid TCEs and thus may not
+ * need that flush. We'll probably turn it_type into a bit mask
+ * of flags if that becomes the case
+ */
+ if (!(tbl->it_type & TCE_PCI_SWINV_FREE))
+ return;
+
+ if (!pe || !ptbl->invalidate_fn)
+ return;
+
+ ptbl->invalidate_fn(pe, tbl, startp, endp, rm);
+}
+
static int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
unsigned long uaddr, enum dma_data_direction direction,
struct dma_attrs *attrs, bool rm)
@@ -619,12 +640,7 @@ static int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
*(tcep++) = cpu_to_be64(proto_tce |
(rpn++ << tbl->it_page_shift));
- /* Some implementations won't cache invalid TCEs and thus may not
- * need that flush. We'll probably turn it_type into a bit mask
- * of flags if that becomes the case
- */
- if (tbl->it_type & TCE_PCI_SWINV_CREATE)
- pnv_pci_ioda_tce_invalidate(tbl, tces, tcep - 1, rm);
+ pnv_tce_invalidate(tbl, tces, tcep - 1, rm);
return 0;
}
@@ -648,8 +664,7 @@ static void pnv_tce_free(struct iommu_table *tbl, long index, long npages,
while (npages--)
*(tcep++) = cpu_to_be64(0);
- if (tbl->it_type & TCE_PCI_SWINV_FREE)
- pnv_pci_ioda_tce_invalidate(tbl, tces, tcep - 1, rm);
+ pnv_tce_invalidate(tbl, tces, tcep - 1, rm);
}
static void pnv_tce_free_vm(struct iommu_table *tbl, long index, long npages)
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index 48494d4..095db43 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -24,6 +24,17 @@ enum pnv_phb_model {
#define PNV_IODA_PE_MASTER (1 << 3) /* Master PE in compound case */
#define PNV_IODA_PE_SLAVE (1 << 4) /* Slave PE in compound case */
+struct pnv_ioda_pe;
+typedef void (*pnv_invalidate_fn)(struct pnv_ioda_pe *pe,
+ struct iommu_table *tbl,
+ __be64 *startp, __be64 *endp, bool rm);
+
+struct pnv_iommu_table {
+ struct iommu_table table;
+ struct pnv_ioda_pe *pe;
+ pnv_invalidate_fn invalidate_fn;
+};
+
/* Data associated with a PE, including IOMMU tracking etc.. */
struct pnv_phb;
struct pnv_ioda_pe {
@@ -53,7 +64,7 @@ struct pnv_ioda_pe {
/* "Base" iommu table, ie, 4K TCEs, 32-bit DMA */
int tce32_seg;
int tce32_segcount;
- struct iommu_table tce32_table;
+ struct pnv_iommu_table tce32;
phys_addr_t tce_inval_reg_phys;
/* 64-bit TCE bypass region */
--
2.0.0
^ permalink raw reply related
* [PATCH v2 00/13] powerpc/iommu/vfio: Enable Dynamic DMA windows
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
This enables PAPR defined feature called Dynamic DMA windows (DDW).
Each Partitionable Endpoint (IOMMU group) has a separate DMA window on
a PCI bus where devices are allows to perform DMA. By default there is
1 or 2GB window allocated at the host boot time and these windows are
used when an IOMMU group is passed to the userspace (guest). These windows
are mapped at zero offset on a PCI bus.
Hi-speed devices may suffer from limited size of this window. On the host
side a TCE bypass mode is enabled on POWER8 CPU which implements
direct mapping of the host memory to a PCI bus at 1<<59.
For the guest, PAPR defines a DDW RTAS API which allows the pseries guest
to query the hypervisor if it supports DDW and what are the parameters
of possible windows.
Currently POWER8 supports 2 DMA windows per PE - already mentioned and used
small 32bit window and 64bit window which can only start from 1<<59 and
can support various page sizes.
This patchset reworks PPC IOMMU code and adds necessary structures
to extend it to support big windows.
When the guest detectes the feature and the PE is capable of 64bit DMA,
it does:
1. query to hypervisor about number of available windows and page masks;
2. creates a window with the biggest possible page size (current guests can do
64K or 16MB TCEs);
3. maps the entire guest RAM via H_PUT_TCE* hypercalls
4. switches dma_ops to direct_dma_ops on the selected PE.
Once this is done, H_PUT_TCE is not called anymore and the guest gets
maximum performance.
Please comment. Thanks!
Changes:
v2:
* added missing __pa() in "powerpc/powernv: Release replaced TCE"
* reposted to make some noise :)
Alexey Kardashevskiy (13):
powerpc/iommu: Check that TCE page size is equal to it_page_size
powerpc/powernv: Make invalidate() a callback
powerpc/spapr: vfio: Implement spapr_tce_iommu_ops
powerpc/powernv: Convert/move set_bypass() callback to
take_ownership()
powerpc/iommu: Fix IOMMU ownership control functions
powerpc/iommu: Move tce_xxx callbacks from ppc_md to iommu_table
powerpc/powernv: Do not set "read" flag if direction==DMA_NONE
powerpc/powernv: Release replaced TCE
powerpc/pseries/lpar: Enable VFIO
powerpc/powernv: Implement Dynamic DMA windows (DDW) for IODA
vfio: powerpc/spapr: Move locked_vm accounting to helpers
vfio: powerpc/spapr: Use it_page_size
vfio: powerpc/spapr: Enable Dynamic DMA windows
arch/powerpc/include/asm/iommu.h | 35 ++-
arch/powerpc/include/asm/machdep.h | 25 --
arch/powerpc/include/asm/tce.h | 37 +++
arch/powerpc/kernel/iommu.c | 213 +++++++++------
arch/powerpc/kernel/vio.c | 5 +-
arch/powerpc/platforms/cell/iommu.c | 9 +-
arch/powerpc/platforms/pasemi/iommu.c | 8 +-
arch/powerpc/platforms/powernv/pci-ioda.c | 233 +++++++++++++++--
arch/powerpc/platforms/powernv/pci-p5ioc2.c | 4 +-
arch/powerpc/platforms/powernv/pci.c | 113 +++++---
arch/powerpc/platforms/powernv/pci.h | 15 +-
arch/powerpc/platforms/pseries/iommu.c | 77 ++++--
arch/powerpc/sysdev/dart_iommu.c | 13 +-
drivers/vfio/vfio_iommu_spapr_tce.c | 384 +++++++++++++++++++++++-----
include/uapi/linux/vfio.h | 25 +-
15 files changed, 925 insertions(+), 271 deletions(-)
--
2.0.0
^ permalink raw reply
* [PATCH v2 04/13] powerpc/powernv: Convert/move set_bypass() callback to take_ownership()
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
At the moment the iommu_table struct has a set_bypass() which enables/
disables DMA bypass on IODA2 PHB. This is exposed to POWERPC IOMMU code
which calls this callback when external IOMMU users such as VFIO are
about to get over a PHB.
Since the set_bypass() is not really an iommu_table function but PE's
function, and we have an ops struct per IOMMU owner, let's move
set_bypass() to the spapr_tce_iommu_ops struct.
As arch/powerpc/kernel/iommu.c is more about POWERPC IOMMU tables and
has very little to do with PEs, this moves take_ownership() calls to
the VFIO SPAPR TCE driver.
This renames set_bypass() to take_ownership() as it is not necessarily
just enabling bypassing, it can be something else/more so let's give it
a generic name. The bool parameter is inverted.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/iommu.h | 1 -
arch/powerpc/include/asm/tce.h | 2 ++
arch/powerpc/kernel/iommu.c | 12 ------------
arch/powerpc/platforms/powernv/pci-ioda.c | 20 ++++++++++++--------
drivers/vfio/vfio_iommu_spapr_tce.c | 16 ++++++++++++++++
5 files changed, 30 insertions(+), 21 deletions(-)
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index 84ee339..2b0b01d 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -77,7 +77,6 @@ struct iommu_table {
#ifdef CONFIG_IOMMU_API
struct iommu_group *it_group;
#endif
- void (*set_bypass)(struct iommu_table *tbl, bool enable);
};
/* Pure 2^n version of get_order */
diff --git a/arch/powerpc/include/asm/tce.h b/arch/powerpc/include/asm/tce.h
index 9f159eb..e6355f9 100644
--- a/arch/powerpc/include/asm/tce.h
+++ b/arch/powerpc/include/asm/tce.h
@@ -56,6 +56,8 @@ struct spapr_tce_iommu_ops {
struct iommu_table *(*get_table)(
struct spapr_tce_iommu_group *data,
int num);
+ void (*take_ownership)(struct spapr_tce_iommu_group *data,
+ bool enable);
};
struct spapr_tce_iommu_group {
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index 1c5dae7..c2c8d9d 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -1139,14 +1139,6 @@ int iommu_take_ownership(struct iommu_table *tbl)
memset(tbl->it_map, 0xff, sz);
iommu_clear_tces_and_put_pages(tbl, tbl->it_offset, tbl->it_size);
- /*
- * Disable iommu bypass, otherwise the user can DMA to all of
- * our physical memory via the bypass window instead of just
- * the pages that has been explicitly mapped into the iommu
- */
- if (tbl->set_bypass)
- tbl->set_bypass(tbl, false);
-
return 0;
}
EXPORT_SYMBOL_GPL(iommu_take_ownership);
@@ -1161,10 +1153,6 @@ void iommu_release_ownership(struct iommu_table *tbl)
/* Restore bit#0 set by iommu_init_table() */
if (tbl->it_offset == 0)
set_bit(0, tbl->it_map);
-
- /* The kernel owns the device now, we can restore the iommu bypass */
- if (tbl->set_bypass)
- tbl->set_bypass(tbl, true);
}
EXPORT_SYMBOL_GPL(iommu_release_ownership);
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 2d32a1c..8cb2f31 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1105,10 +1105,8 @@ static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
__free_pages(tce_mem, get_order(TCE32_TABLE_SIZE * segs));
}
-static void pnv_pci_ioda2_set_bypass(struct iommu_table *tbl, bool enable)
+static void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable)
{
- struct pnv_ioda_pe *pe = container_of(tbl, struct pnv_ioda_pe,
- tce32.table);
uint16_t window_id = (pe->pe_number << 1 ) + 1;
int64_t rc;
@@ -1136,7 +1134,7 @@ static void pnv_pci_ioda2_set_bypass(struct iommu_table *tbl, bool enable)
* host side.
*/
if (pe->pdev)
- set_iommu_table_base(&pe->pdev->dev, tbl);
+ set_iommu_table_base(&pe->pdev->dev, &pe->tce32.table);
else
pnv_ioda_setup_bus_dma(pe, pe->pbus, false);
}
@@ -1152,15 +1150,21 @@ static void pnv_pci_ioda2_setup_bypass_pe(struct pnv_phb *phb,
/* TVE #1 is selected by PCI address bit 59 */
pe->tce_bypass_base = 1ull << 59;
- /* Install set_bypass callback for VFIO */
- pe->tce32.table.set_bypass = pnv_pci_ioda2_set_bypass;
-
/* Enable bypass by default */
- pnv_pci_ioda2_set_bypass(&pe->tce32.table, true);
+ pnv_pci_ioda2_set_bypass(pe, true);
+}
+
+static void pnv_ioda2_take_ownership(struct spapr_tce_iommu_group *data,
+ bool enable)
+{
+ struct pnv_ioda_pe *pe = data->iommu_owner;
+
+ pnv_pci_ioda2_set_bypass(pe, !enable);
}
static struct spapr_tce_iommu_ops pnv_pci_ioda2_ops = {
.get_table = pnv_ioda1_iommu_get_table,
+ .take_ownership = pnv_ioda2_take_ownership,
};
static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index a8adfbd..1c1a9c4 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -76,6 +76,13 @@ static struct iommu_table *spapr_tce_find_table(
return ret;
}
+static void tce_iommu_take_ownership_notify(struct spapr_tce_iommu_group *data,
+ bool enable)
+{
+ if (data && data->ops && data->ops->take_ownership)
+ data->ops->take_ownership(data, enable);
+}
+
static int tce_iommu_enable(struct tce_container *container)
{
int ret = 0;
@@ -413,6 +420,12 @@ static int tce_iommu_attach_group(void *iommu_data,
ret = iommu_take_ownership(tbl);
if (!ret)
container->grp = iommu_group;
+ /*
+ * Disable iommu bypass, otherwise the user can DMA to all of
+ * our physical memory via the bypass window instead of just
+ * the pages that has been explicitly mapped into the iommu
+ */
+ tce_iommu_take_ownership_notify(data, true);
}
mutex_unlock(&container->lock);
@@ -450,6 +463,9 @@ static void tce_iommu_detach_group(void *iommu_data,
BUG_ON(!tbl);
iommu_release_ownership(tbl);
+
+ /* Kernel owns the device now, we can restore bypass */
+ tce_iommu_take_ownership_notify(data, false);
}
mutex_unlock(&container->lock);
}
--
2.0.0
^ permalink raw reply related
* [PATCH v2 06/13] powerpc/iommu: Move tce_xxx callbacks from ppc_md to iommu_table
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
This adds a iommu_table_ops struct and puts pointer to it into
the iommu_table struct. This moves tce_build/tce_free/tce_get/tce_flush
callbacks from ppc_md to the new struct where they really belong to.
This adds an extra @ops parameter to iommu_init_table() to make sure
that we do not leave any IOMMU table without iommu_table_ops. @it_ops is
initialized in the very beginning as iommu_init_table() calls
iommu_table_clear() and the latter uses callbacks already.
This does s/tce_build/set/, s/tce_free/clear/ and removes "tce_" prefixes
for better readability.
This removes tce_xxx_rm handlers from ppc_md as well but does not add
them to iommu_table_ops, this will be done later if we decide to support
TCE hypercalls in real mode.
This always uses tce_buildmulti_pSeriesLP/tce_buildmulti_pSeriesLP as
callbacks for pseries. This changes "multi" callbacks to fall back to
tce_build_pSeriesLP/tce_free_pSeriesLP if FW_FEATURE_MULTITCE is not
present. The reason for this is we still have to support "multitce=off"
boot parameter in disable_multitce() and we do not want to walk through
all IOMMU tables in the system and replace "multi" callbacks with single
ones.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/include/asm/iommu.h | 20 +++++++++++-
arch/powerpc/include/asm/machdep.h | 25 ---------------
arch/powerpc/kernel/iommu.c | 50 ++++++++++++++++-------------
arch/powerpc/kernel/vio.c | 5 ++-
arch/powerpc/platforms/cell/iommu.c | 9 ++++--
arch/powerpc/platforms/pasemi/iommu.c | 8 +++--
arch/powerpc/platforms/powernv/pci-ioda.c | 4 +--
arch/powerpc/platforms/powernv/pci-p5ioc2.c | 3 +-
arch/powerpc/platforms/powernv/pci.c | 24 ++++----------
arch/powerpc/platforms/powernv/pci.h | 1 +
arch/powerpc/platforms/pseries/iommu.c | 42 +++++++++++++-----------
arch/powerpc/sysdev/dart_iommu.c | 13 ++++----
12 files changed, 102 insertions(+), 102 deletions(-)
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index 2b0b01d..c725e4a 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -43,6 +43,22 @@
extern int iommu_is_off;
extern int iommu_force_on;
+struct iommu_table_ops {
+ int (*set)(struct iommu_table *tbl,
+ long index, long npages,
+ unsigned long uaddr,
+ enum dma_data_direction direction,
+ struct dma_attrs *attrs);
+ void (*clear)(struct iommu_table *tbl,
+ long index, long npages);
+ unsigned long (*get)(struct iommu_table *tbl, long index);
+ void (*flush)(struct iommu_table *tbl);
+};
+
+/* These are used by VIO */
+extern struct iommu_table_ops iommu_table_lpar_multi_ops;
+extern struct iommu_table_ops iommu_table_pseries_ops;
+
/*
* IOMAP_MAX_ORDER defines the largest contiguous block
* of dma space we can get. IOMAP_MAX_ORDER = 13
@@ -77,6 +93,7 @@ struct iommu_table {
#ifdef CONFIG_IOMMU_API
struct iommu_group *it_group;
#endif
+ struct iommu_table_ops *it_ops;
};
/* Pure 2^n version of get_order */
@@ -106,7 +123,8 @@ extern void iommu_free_table(struct iommu_table *tbl, const char *node_name);
* structure
*/
extern struct iommu_table *iommu_init_table(struct iommu_table * tbl,
- int nid);
+ int nid,
+ struct iommu_table_ops *ops);
struct spapr_tce_iommu_ops;
#ifdef CONFIG_IOMMU_API
diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
index b125cea..1fc824d 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -65,31 +65,6 @@ struct machdep_calls {
* destroyed as well */
void (*hpte_clear_all)(void);
- int (*tce_build)(struct iommu_table *tbl,
- long index,
- long npages,
- unsigned long uaddr,
- enum dma_data_direction direction,
- struct dma_attrs *attrs);
- void (*tce_free)(struct iommu_table *tbl,
- long index,
- long npages);
- unsigned long (*tce_get)(struct iommu_table *tbl,
- long index);
- void (*tce_flush)(struct iommu_table *tbl);
-
- /* _rm versions are for real mode use only */
- int (*tce_build_rm)(struct iommu_table *tbl,
- long index,
- long npages,
- unsigned long uaddr,
- enum dma_data_direction direction,
- struct dma_attrs *attrs);
- void (*tce_free_rm)(struct iommu_table *tbl,
- long index,
- long npages);
- void (*tce_flush_rm)(struct iommu_table *tbl);
-
void __iomem * (*ioremap)(phys_addr_t addr, unsigned long size,
unsigned long flags, void *caller);
void (*iounmap)(volatile void __iomem *token);
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index cd80867..678fee8 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -323,11 +323,11 @@ static dma_addr_t iommu_alloc(struct device *dev, struct iommu_table *tbl,
ret = entry << tbl->it_page_shift; /* Set the return dma address */
/* Put the TCEs in the HW table */
- build_fail = ppc_md.tce_build(tbl, entry, npages,
+ build_fail = tbl->it_ops->set(tbl, entry, npages,
(unsigned long)page &
IOMMU_PAGE_MASK(tbl), direction, attrs);
- /* ppc_md.tce_build() only returns non-zero for transient errors.
+ /* tbl->it_ops->set() only returns non-zero for transient errors.
* Clean up the table bitmap in this case and return
* DMA_ERROR_CODE. For all other errors the functionality is
* not altered.
@@ -338,8 +338,8 @@ static dma_addr_t iommu_alloc(struct device *dev, struct iommu_table *tbl,
}
/* Flush/invalidate TLB caches if necessary */
- if (ppc_md.tce_flush)
- ppc_md.tce_flush(tbl);
+ if (tbl->it_ops->flush)
+ tbl->it_ops->flush(tbl);
/* Make sure updates are seen by hardware */
mb();
@@ -409,7 +409,7 @@ static void __iommu_free(struct iommu_table *tbl, dma_addr_t dma_addr,
if (!iommu_free_check(tbl, dma_addr, npages))
return;
- ppc_md.tce_free(tbl, entry, npages);
+ tbl->it_ops->clear(tbl, entry, npages);
spin_lock_irqsave(&(pool->lock), flags);
bitmap_clear(tbl->it_map, free_entry, npages);
@@ -425,8 +425,8 @@ static void iommu_free(struct iommu_table *tbl, dma_addr_t dma_addr,
* not do an mb() here on purpose, it is not needed on any of
* the current platforms.
*/
- if (ppc_md.tce_flush)
- ppc_md.tce_flush(tbl);
+ if (tbl->it_ops->flush)
+ tbl->it_ops->flush(tbl);
}
int iommu_map_sg(struct device *dev, struct iommu_table *tbl,
@@ -496,7 +496,7 @@ int iommu_map_sg(struct device *dev, struct iommu_table *tbl,
npages, entry, dma_addr);
/* Insert into HW table */
- build_fail = ppc_md.tce_build(tbl, entry, npages,
+ build_fail = tbl->it_ops->set(tbl, entry, npages,
vaddr & IOMMU_PAGE_MASK(tbl),
direction, attrs);
if(unlikely(build_fail))
@@ -535,8 +535,8 @@ int iommu_map_sg(struct device *dev, struct iommu_table *tbl,
}
/* Flush/invalidate TLB caches if necessary */
- if (ppc_md.tce_flush)
- ppc_md.tce_flush(tbl);
+ if (tbl->it_ops->flush)
+ tbl->it_ops->flush(tbl);
DBG("mapped %d elements:\n", outcount);
@@ -601,8 +601,8 @@ void iommu_unmap_sg(struct iommu_table *tbl, struct scatterlist *sglist,
* do not do an mb() here, the affected platforms do not need it
* when freeing.
*/
- if (ppc_md.tce_flush)
- ppc_md.tce_flush(tbl);
+ if (tbl->it_ops->flush)
+ tbl->it_ops->flush(tbl);
}
static void iommu_table_clear(struct iommu_table *tbl)
@@ -614,17 +614,17 @@ static void iommu_table_clear(struct iommu_table *tbl)
*/
if (!is_kdump_kernel() || is_fadump_active()) {
/* Clear the table in case firmware left allocations in it */
- ppc_md.tce_free(tbl, tbl->it_offset, tbl->it_size);
+ tbl->it_ops->clear(tbl, tbl->it_offset, tbl->it_size);
return;
}
#ifdef CONFIG_CRASH_DUMP
- if (ppc_md.tce_get) {
+ if (tbl->it_ops->get) {
unsigned long index, tceval, tcecount = 0;
/* Reserve the existing mappings left by the first kernel. */
for (index = 0; index < tbl->it_size; index++) {
- tceval = ppc_md.tce_get(tbl, index + tbl->it_offset);
+ tceval = tbl->it_ops->get(tbl, index + tbl->it_offset);
/*
* Freed TCE entry contains 0x7fffffffffffffff on JS20
*/
@@ -650,7 +650,8 @@ static void iommu_table_clear(struct iommu_table *tbl)
* Build a iommu_table structure. This contains a bit map which
* is used to manage allocation of the tce space.
*/
-struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid)
+struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid,
+ struct iommu_table_ops *ops)
{
unsigned long sz;
static int welcomed = 0;
@@ -658,6 +659,9 @@ struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid)
unsigned int i;
struct iommu_pool *p;
+ BUG_ON(!ops);
+ tbl->it_ops = ops;
+
/* number of bytes needed for the bitmap */
sz = BITS_TO_LONGS(tbl->it_size) * sizeof(unsigned long);
@@ -949,8 +953,8 @@ EXPORT_SYMBOL_GPL(iommu_tce_direction);
void iommu_flush_tce(struct iommu_table *tbl)
{
/* Flush/invalidate TLB caches if necessary */
- if (ppc_md.tce_flush)
- ppc_md.tce_flush(tbl);
+ if (tbl->it_ops->flush)
+ tbl->it_ops->flush(tbl);
/* Make sure updates are seen by hardware */
mb();
@@ -961,7 +965,7 @@ int iommu_tce_clear_param_check(struct iommu_table *tbl,
unsigned long ioba, unsigned long tce_value,
unsigned long npages)
{
- /* ppc_md.tce_free() does not support any value but 0 */
+ /* tbl->it_ops->clear() does not support any value but 0 */
if (tce_value)
return -EINVAL;
@@ -1009,9 +1013,9 @@ unsigned long iommu_clear_tce(struct iommu_table *tbl, unsigned long entry)
spin_lock(&(pool->lock));
- oldtce = ppc_md.tce_get(tbl, entry);
+ oldtce = tbl->it_ops->get(tbl, entry);
if (oldtce & (TCE_PCI_WRITE | TCE_PCI_READ))
- ppc_md.tce_free(tbl, entry, 1);
+ tbl->it_ops->clear(tbl, entry, 1);
else
oldtce = 0;
@@ -1058,10 +1062,10 @@ int iommu_tce_build(struct iommu_table *tbl, unsigned long entry,
spin_lock(&(pool->lock));
- oldtce = ppc_md.tce_get(tbl, entry);
+ oldtce = tbl->it_ops->get(tbl, entry);
/* Add new entry if it is not busy */
if (!(oldtce & (TCE_PCI_WRITE | TCE_PCI_READ)))
- ret = ppc_md.tce_build(tbl, entry, 1, hwaddr, direction, NULL);
+ ret = tbl->it_ops->set(tbl, entry, 1, hwaddr, direction, NULL);
spin_unlock(&(pool->lock));
diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
index 5bfdab9..c61ce7a 100644
--- a/arch/powerpc/kernel/vio.c
+++ b/arch/powerpc/kernel/vio.c
@@ -1196,7 +1196,10 @@ static struct iommu_table *vio_build_iommu_table(struct vio_dev *dev)
tbl->it_type = TCE_VB;
tbl->it_blocksize = 16;
- return iommu_init_table(tbl, -1);
+ return iommu_init_table(tbl, -1,
+ firmware_has_feature(FW_FEATURE_LPAR) ?
+ &iommu_table_lpar_multi_ops :
+ &iommu_table_pseries_ops);
}
/**
diff --git a/arch/powerpc/platforms/cell/iommu.c b/arch/powerpc/platforms/cell/iommu.c
index 2b90ff8..f6254f7 100644
--- a/arch/powerpc/platforms/cell/iommu.c
+++ b/arch/powerpc/platforms/cell/iommu.c
@@ -465,6 +465,11 @@ static inline u32 cell_iommu_get_ioid(struct device_node *np)
return *ioid;
}
+static struct iommu_table_ops cell_iommu_ops = {
+ .set = tce_build_cell,
+ .clear = tce_free_cell
+};
+
static struct iommu_window * __init
cell_iommu_setup_window(struct cbe_iommu *iommu, struct device_node *np,
unsigned long offset, unsigned long size,
@@ -492,7 +497,7 @@ cell_iommu_setup_window(struct cbe_iommu *iommu, struct device_node *np,
(offset >> window->table.it_page_shift) + pte_offset;
window->table.it_size = size >> window->table.it_page_shift;
- iommu_init_table(&window->table, iommu->nid);
+ iommu_init_table(&window->table, iommu->nid, &cell_iommu_ops);
pr_debug("\tioid %d\n", window->ioid);
pr_debug("\tblocksize %ld\n", window->table.it_blocksize);
@@ -1199,8 +1204,6 @@ static int __init cell_iommu_init(void)
/* Setup various ppc_md. callbacks */
ppc_md.pci_dma_dev_setup = cell_pci_dma_dev_setup;
ppc_md.dma_get_required_mask = cell_dma_get_required_mask;
- ppc_md.tce_build = tce_build_cell;
- ppc_md.tce_free = tce_free_cell;
if (!iommu_fixed_disabled && cell_iommu_fixed_mapping_init() == 0)
goto bail;
diff --git a/arch/powerpc/platforms/pasemi/iommu.c b/arch/powerpc/platforms/pasemi/iommu.c
index 2e576f2..eac33f4 100644
--- a/arch/powerpc/platforms/pasemi/iommu.c
+++ b/arch/powerpc/platforms/pasemi/iommu.c
@@ -132,6 +132,10 @@ static void iobmap_free(struct iommu_table *tbl, long index,
}
}
+static struct iommu_table_ops iommu_table_iobmap_ops = {
+ .set = iobmap_build,
+ .clear = iobmap_free
+};
static void iommu_table_iobmap_setup(void)
{
@@ -151,7 +155,7 @@ static void iommu_table_iobmap_setup(void)
* Should probably be 8 (64 bytes)
*/
iommu_table_iobmap.it_blocksize = 4;
- iommu_init_table(&iommu_table_iobmap, 0);
+ iommu_init_table(&iommu_table_iobmap, 0, &iommu_table_iobmap_ops);
pr_debug(" <- %s\n", __func__);
}
@@ -250,8 +254,6 @@ void __init iommu_init_early_pasemi(void)
ppc_md.pci_dma_dev_setup = pci_dma_dev_setup_pasemi;
ppc_md.pci_dma_bus_setup = pci_dma_bus_setup_pasemi;
- ppc_md.tce_build = iobmap_build;
- ppc_md.tce_free = iobmap_free;
set_pci_dma_ops(&dma_iommu_ops);
}
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 8cb2f31..296f49b 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1087,7 +1087,7 @@ static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
TCE_PCI_SWINV_FREE |
TCE_PCI_SWINV_PAIR);
}
- iommu_init_table(tbl, phb->hose->node);
+ iommu_init_table(tbl, phb->hose->node, &pnv_iommu_ops);
iommu_register_group(tbl, pe, &pnv_pci_ioda1_ops,
phb->hose->global_number, pe->pe_number);
@@ -1231,7 +1231,7 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
8);
tbl->it_type |= (TCE_PCI_SWINV_CREATE | TCE_PCI_SWINV_FREE);
}
- iommu_init_table(tbl, phb->hose->node);
+ iommu_init_table(tbl, phb->hose->node, &pnv_iommu_ops);
iommu_register_group(tbl, pe, &pnv_pci_ioda2_ops,
phb->hose->global_number, pe->pe_number);
diff --git a/arch/powerpc/platforms/powernv/pci-p5ioc2.c b/arch/powerpc/platforms/powernv/pci-p5ioc2.c
index b79066d..ea97414 100644
--- a/arch/powerpc/platforms/powernv/pci-p5ioc2.c
+++ b/arch/powerpc/platforms/powernv/pci-p5ioc2.c
@@ -87,7 +87,8 @@ static void pnv_pci_p5ioc2_dma_dev_setup(struct pnv_phb *phb,
struct pci_dev *pdev)
{
if (phb->p5ioc2.iommu_table.it_map == NULL) {
- iommu_init_table(&phb->p5ioc2.iommu_table, phb->hose->node);
+ iommu_init_table(&phb->p5ioc2.iommu_table, phb->hose->node,
+ &pnv_iommu_ops);
iommu_register_group(&phb->p5ioc2.iommu_table,
NULL, NULL,
pci_domain_nr(phb->hose->bus), phb->opal_id);
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 6ffac79..deddcad 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -677,18 +677,11 @@ static unsigned long pnv_tce_get(struct iommu_table *tbl, long index)
return ((u64 *)tbl->it_base)[index - tbl->it_offset];
}
-static int pnv_tce_build_rm(struct iommu_table *tbl, long index, long npages,
- unsigned long uaddr,
- enum dma_data_direction direction,
- struct dma_attrs *attrs)
-{
- return pnv_tce_build(tbl, index, npages, uaddr, direction, attrs, true);
-}
-
-static void pnv_tce_free_rm(struct iommu_table *tbl, long index, long npages)
-{
- pnv_tce_free(tbl, index, npages, true);
-}
+struct iommu_table_ops pnv_iommu_ops = {
+ .set = pnv_tce_build_vm,
+ .clear = pnv_tce_free_vm,
+ .get = pnv_tce_get,
+};
void pnv_pci_setup_iommu_table(struct iommu_table *tbl,
void *tce_mem, u64 tce_size,
@@ -722,7 +715,7 @@ static struct iommu_table *pnv_pci_setup_bml_iommu(struct pci_controller *hose)
return NULL;
pnv_pci_setup_iommu_table(tbl, __va(be64_to_cpup(basep)),
be32_to_cpup(sizep), 0, IOMMU_PAGE_SHIFT_4K);
- iommu_init_table(tbl, hose->node);
+ iommu_init_table(tbl, hose->node, &pnv_iommu_ops);
iommu_register_group(tbl, NULL, NULL, pci_domain_nr(hose->bus), 0);
/* Deal with SW invalidated TCEs when needed (BML way) */
@@ -865,11 +858,6 @@ void __init pnv_pci_init(void)
/* Configure IOMMU DMA hooks */
ppc_md.pci_dma_dev_setup = pnv_pci_dma_dev_setup;
- ppc_md.tce_build = pnv_tce_build_vm;
- ppc_md.tce_free = pnv_tce_free_vm;
- ppc_md.tce_build_rm = pnv_tce_build_rm;
- ppc_md.tce_free_rm = pnv_tce_free_rm;
- ppc_md.tce_get = pnv_tce_get;
ppc_md.pci_probe_mode = pnv_pci_probe_mode;
set_pci_dma_ops(&dma_iommu_ops);
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index 095db43..cf68c4b 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -223,6 +223,7 @@ extern struct pci_ops pnv_pci_ops;
#ifdef CONFIG_EEH
extern struct pnv_eeh_ops ioda_eeh_ops;
#endif
+extern struct iommu_table_ops pnv_iommu_ops;
void pnv_pci_dump_phb_diag_data(struct pci_controller *hose,
unsigned char *log_buff);
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index b95f8cf..9a7364f 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -193,7 +193,7 @@ static int tce_buildmulti_pSeriesLP(struct iommu_table *tbl, long tcenum,
int ret = 0;
unsigned long flags;
- if (npages == 1) {
+ if ((npages == 1) || !firmware_has_feature(FW_FEATURE_MULTITCE)) {
return tce_build_pSeriesLP(tbl, tcenum, npages, uaddr,
direction, attrs);
}
@@ -285,6 +285,9 @@ static void tce_freemulti_pSeriesLP(struct iommu_table *tbl, long tcenum, long n
{
u64 rc;
+ if (!firmware_has_feature(FW_FEATURE_MULTITCE))
+ return tce_free_pSeriesLP(tbl, tcenum, npages);
+
rc = plpar_tce_stuff((u64)tbl->it_index, (u64)tcenum << 12, 0, npages);
if (rc && printk_ratelimit()) {
@@ -460,7 +463,6 @@ static int tce_setrange_multi_pSeriesLP_walk(unsigned long start_pfn,
return tce_setrange_multi_pSeriesLP(start_pfn, num_pfn, arg);
}
-
#ifdef CONFIG_PCI
static void iommu_table_setparms(struct pci_controller *phb,
struct device_node *dn,
@@ -546,6 +548,12 @@ static void iommu_table_setparms_lpar(struct pci_controller *phb,
tbl->it_size = size >> tbl->it_page_shift;
}
+struct iommu_table_ops iommu_table_pseries_ops = {
+ .set = tce_build_pSeries,
+ .clear = tce_free_pSeries,
+ .get = tce_get_pseries
+};
+
static void pci_dma_bus_setup_pSeries(struct pci_bus *bus)
{
struct device_node *dn;
@@ -615,7 +623,8 @@ static void pci_dma_bus_setup_pSeries(struct pci_bus *bus)
pci->phb->node);
iommu_table_setparms(pci->phb, dn, tbl);
- pci->iommu_table = iommu_init_table(tbl, pci->phb->node);
+ pci->iommu_table = iommu_init_table(tbl, pci->phb->node,
+ &iommu_table_pseries_ops);
iommu_register_group(tbl, NULL, NULL, pci_domain_nr(bus), 0);
/* Divide the rest (1.75GB) among the children */
@@ -626,6 +635,11 @@ static void pci_dma_bus_setup_pSeries(struct pci_bus *bus)
pr_debug("ISA/IDE, window size is 0x%llx\n", pci->phb->dma_window_size);
}
+struct iommu_table_ops iommu_table_lpar_multi_ops = {
+ .set = tce_buildmulti_pSeriesLP,
+ .clear = tce_freemulti_pSeriesLP,
+ .get = tce_get_pSeriesLP
+};
static void pci_dma_bus_setup_pSeriesLP(struct pci_bus *bus)
{
@@ -660,7 +674,8 @@ static void pci_dma_bus_setup_pSeriesLP(struct pci_bus *bus)
tbl = kzalloc_node(sizeof(struct iommu_table), GFP_KERNEL,
ppci->phb->node);
iommu_table_setparms_lpar(ppci->phb, pdn, tbl, dma_window);
- ppci->iommu_table = iommu_init_table(tbl, ppci->phb->node);
+ ppci->iommu_table = iommu_init_table(tbl, ppci->phb->node,
+ &iommu_table_lpar_multi_ops);
iommu_register_group(tbl, NULL, NULL, pci_domain_nr(bus), 0);
pr_debug(" created table: %p\n", ppci->iommu_table);
}
@@ -687,7 +702,8 @@ static void pci_dma_dev_setup_pSeries(struct pci_dev *dev)
tbl = kzalloc_node(sizeof(struct iommu_table), GFP_KERNEL,
phb->node);
iommu_table_setparms(phb, dn, tbl);
- PCI_DN(dn)->iommu_table = iommu_init_table(tbl, phb->node);
+ PCI_DN(dn)->iommu_table = iommu_init_table(tbl, phb->node,
+ &iommu_table_pseries_ops);
iommu_register_group(tbl, NULL, NULL,
pci_domain_nr(phb->bus), 0);
set_iommu_table_base_and_group(&dev->dev,
@@ -1105,7 +1121,8 @@ static void pci_dma_dev_setup_pSeriesLP(struct pci_dev *dev)
tbl = kzalloc_node(sizeof(struct iommu_table), GFP_KERNEL,
pci->phb->node);
iommu_table_setparms_lpar(pci->phb, pdn, tbl, dma_window);
- pci->iommu_table = iommu_init_table(tbl, pci->phb->node);
+ pci->iommu_table = iommu_init_table(tbl, pci->phb->node,
+ &iommu_table_lpar_multi_ops);
iommu_register_group(tbl, NULL, NULL,
pci_domain_nr(pci->phb->bus), 0);
pr_debug(" created table: %p\n", pci->iommu_table);
@@ -1297,22 +1314,11 @@ void iommu_init_early_pSeries(void)
return;
if (firmware_has_feature(FW_FEATURE_LPAR)) {
- if (firmware_has_feature(FW_FEATURE_MULTITCE)) {
- ppc_md.tce_build = tce_buildmulti_pSeriesLP;
- ppc_md.tce_free = tce_freemulti_pSeriesLP;
- } else {
- ppc_md.tce_build = tce_build_pSeriesLP;
- ppc_md.tce_free = tce_free_pSeriesLP;
- }
- ppc_md.tce_get = tce_get_pSeriesLP;
ppc_md.pci_dma_bus_setup = pci_dma_bus_setup_pSeriesLP;
ppc_md.pci_dma_dev_setup = pci_dma_dev_setup_pSeriesLP;
ppc_md.dma_set_mask = dma_set_mask_pSeriesLP;
ppc_md.dma_get_required_mask = dma_get_required_mask_pSeriesLP;
} else {
- ppc_md.tce_build = tce_build_pSeries;
- ppc_md.tce_free = tce_free_pSeries;
- ppc_md.tce_get = tce_get_pseries;
ppc_md.pci_dma_bus_setup = pci_dma_bus_setup_pSeries;
ppc_md.pci_dma_dev_setup = pci_dma_dev_setup_pSeries;
}
@@ -1330,8 +1336,6 @@ static int __init disable_multitce(char *str)
firmware_has_feature(FW_FEATURE_LPAR) &&
firmware_has_feature(FW_FEATURE_MULTITCE)) {
printk(KERN_INFO "Disabling MULTITCE firmware feature\n");
- ppc_md.tce_build = tce_build_pSeriesLP;
- ppc_md.tce_free = tce_free_pSeriesLP;
powerpc_firmware_features &= ~FW_FEATURE_MULTITCE;
}
return 1;
diff --git a/arch/powerpc/sysdev/dart_iommu.c b/arch/powerpc/sysdev/dart_iommu.c
index 9e5353f..27721f5 100644
--- a/arch/powerpc/sysdev/dart_iommu.c
+++ b/arch/powerpc/sysdev/dart_iommu.c
@@ -286,6 +286,12 @@ static int __init dart_init(struct device_node *dart_node)
return 0;
}
+static struct iommu_table_ops iommu_dart_ops = {
+ .set = dart_build,
+ .clear = dart_free,
+ .flush = dart_flush,
+};
+
static void iommu_table_dart_setup(void)
{
iommu_table_dart.it_busno = 0;
@@ -298,7 +304,7 @@ static void iommu_table_dart_setup(void)
iommu_table_dart.it_base = (unsigned long)dart_vbase;
iommu_table_dart.it_index = 0;
iommu_table_dart.it_blocksize = 1;
- iommu_init_table(&iommu_table_dart, -1);
+ iommu_init_table(&iommu_table_dart, -1, &iommu_dart_ops);
/* Reserve the last page of the DART to avoid possible prefetch
* past the DART mapped area
@@ -386,11 +392,6 @@ void __init iommu_init_early_dart(void)
if (dart_init(dn) != 0)
goto bail;
- /* Setup low level TCE operations for the core IOMMU code */
- ppc_md.tce_build = dart_build;
- ppc_md.tce_free = dart_free;
- ppc_md.tce_flush = dart_flush;
-
/* Setup bypass if supported */
if (dart_is_u4)
ppc_md.dma_set_mask = dart_dma_set_mask;
--
2.0.0
^ permalink raw reply related
* [PATCH v2 03/13] powerpc/spapr: vfio: Implement spapr_tce_iommu_ops
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
Modern IBM POWERPC systems support multiple IOMMU tables per PE
so we need a more reliable way (compared to container_of()) to get
a PE pointer from the iommu_table struct pointer used in IOMMU functions.
At the moment IOMMU group data points to an iommu_table struct. This
introduces a spapr_tce_iommu_group struct which keeps an iommu_owner
and a spapr_tce_iommu_ops struct. For IODA, iommu_owner is a pointer to
the pnv_ioda_pe struct, for others it is still a pointer to
the iommu_table struct. The ops structs correspond to the type which
iommu_owner points to.
This defines a get_table() callback which returns an iommu_table
by its number.
As the IOMMU group data pointer points to variable type instead of
iommu_table, VFIO SPAPR TCE driver is updated to use the new type.
This changes the tce_container struct to store iommu_group instead of
iommu_table.
So, it was:
- iommu_table points to iommu_group via iommu_table::it_group;
- iommu_group points to iommu_table via iommu_group_get_iommudata();
now it is:
- iommu_table points to iommu_group via iommu_table::it_group;
- iommu_group points to spapr_tce_iommu_group via
iommu_group_get_iommudata();
- spapr_tce_iommu_group points to either (depending on .get_table()):
- iommu_table;
- pnv_ioda_pe;
This uses pnv_ioda1_iommu_get_table for both IODA1&2 but IODA2 will
have own pnv_ioda2_iommu_get_table soon and pnv_ioda1_iommu_get_table
will only be used for IODA1.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/include/asm/iommu.h | 6 ++
arch/powerpc/include/asm/tce.h | 13 +++
arch/powerpc/kernel/iommu.c | 35 ++++++-
arch/powerpc/platforms/powernv/pci-ioda.c | 31 +++++-
arch/powerpc/platforms/powernv/pci-p5ioc2.c | 1 +
arch/powerpc/platforms/powernv/pci.c | 2 +-
arch/powerpc/platforms/pseries/iommu.c | 10 +-
drivers/vfio/vfio_iommu_spapr_tce.c | 148 ++++++++++++++++++++++------
8 files changed, 208 insertions(+), 38 deletions(-)
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index 42632c7..84ee339 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -108,13 +108,19 @@ extern void iommu_free_table(struct iommu_table *tbl, const char *node_name);
*/
extern struct iommu_table *iommu_init_table(struct iommu_table * tbl,
int nid);
+
+struct spapr_tce_iommu_ops;
#ifdef CONFIG_IOMMU_API
extern void iommu_register_group(struct iommu_table *tbl,
+ void *iommu_owner,
+ struct spapr_tce_iommu_ops *ops,
int pci_domain_number, unsigned long pe_num);
extern int iommu_add_device(struct device *dev);
extern void iommu_del_device(struct device *dev);
#else
static inline void iommu_register_group(struct iommu_table *tbl,
+ void *iommu_owner,
+ struct spapr_tce_iommu_ops *ops,
int pci_domain_number,
unsigned long pe_num)
{
diff --git a/arch/powerpc/include/asm/tce.h b/arch/powerpc/include/asm/tce.h
index 743f36b..9f159eb 100644
--- a/arch/powerpc/include/asm/tce.h
+++ b/arch/powerpc/include/asm/tce.h
@@ -50,5 +50,18 @@
#define TCE_PCI_READ 0x1 /* read from PCI allowed */
#define TCE_VB_WRITE 0x1 /* write from VB allowed */
+struct spapr_tce_iommu_group;
+
+struct spapr_tce_iommu_ops {
+ struct iommu_table *(*get_table)(
+ struct spapr_tce_iommu_group *data,
+ int num);
+};
+
+struct spapr_tce_iommu_group {
+ void *iommu_owner;
+ struct spapr_tce_iommu_ops *ops;
+};
+
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_TCE_H */
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index b378f78..1c5dae7 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -878,24 +878,53 @@ void iommu_free_coherent(struct iommu_table *tbl, size_t size,
*/
static void group_release(void *iommu_data)
{
- struct iommu_table *tbl = iommu_data;
- tbl->it_group = NULL;
+ kfree(iommu_data);
}
+static struct iommu_table *spapr_tce_default_get_table(
+ struct spapr_tce_iommu_group *data, int num)
+{
+ struct iommu_table *tbl = data->iommu_owner;
+
+ switch (num) {
+ case 0:
+ if (tbl->it_size)
+ return tbl;
+ /* fallthru */
+ default:
+ return NULL;
+ }
+}
+
+static struct spapr_tce_iommu_ops spapr_tce_default_ops = {
+ .get_table = spapr_tce_default_get_table
+};
+
void iommu_register_group(struct iommu_table *tbl,
+ void *iommu_owner, struct spapr_tce_iommu_ops *ops,
int pci_domain_number, unsigned long pe_num)
{
struct iommu_group *grp;
char *name;
+ struct spapr_tce_iommu_group *data;
+
+ data = kzalloc(sizeof(*data), GFP_KERNEL);
+ if (!data)
+ return;
+
+ data->iommu_owner = iommu_owner ? iommu_owner : tbl;
+ data->ops = ops ? ops : &spapr_tce_default_ops;
grp = iommu_group_alloc();
if (IS_ERR(grp)) {
pr_warn("powerpc iommu api: cannot create new group, err=%ld\n",
PTR_ERR(grp));
+ kfree(data);
return;
}
+
tbl->it_group = grp;
- iommu_group_set_iommudata(grp, tbl, group_release);
+ iommu_group_set_iommudata(grp, data, group_release);
name = kasprintf(GFP_KERNEL, "domain%d-pe%lx",
pci_domain_number, pe_num);
if (!name)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 136e765..2d32a1c 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -23,6 +23,7 @@
#include <linux/io.h>
#include <linux/msi.h>
#include <linux/memblock.h>
+#include <linux/iommu.h>
#include <asm/sections.h>
#include <asm/io.h>
@@ -988,6 +989,26 @@ static void pnv_pci_ioda2_tce_invalidate(struct pnv_ioda_pe *pe,
}
}
+static struct iommu_table *pnv_ioda1_iommu_get_table(
+ struct spapr_tce_iommu_group *data,
+ int num)
+{
+ struct pnv_ioda_pe *pe = data->iommu_owner;
+
+ switch (num) {
+ case 0:
+ if (pe->tce32.table.it_size)
+ return &pe->tce32.table;
+ /* fallthru */
+ default:
+ return NULL;
+ }
+}
+
+static struct spapr_tce_iommu_ops pnv_pci_ioda1_ops = {
+ .get_table = pnv_ioda1_iommu_get_table,
+};
+
static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
struct pnv_ioda_pe *pe, unsigned int base,
unsigned int segs)
@@ -1067,7 +1088,8 @@ static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
TCE_PCI_SWINV_PAIR);
}
iommu_init_table(tbl, phb->hose->node);
- iommu_register_group(tbl, phb->hose->global_number, pe->pe_number);
+ iommu_register_group(tbl, pe, &pnv_pci_ioda1_ops,
+ phb->hose->global_number, pe->pe_number);
if (pe->pdev)
set_iommu_table_base_and_group(&pe->pdev->dev, tbl);
@@ -1137,6 +1159,10 @@ static void pnv_pci_ioda2_setup_bypass_pe(struct pnv_phb *phb,
pnv_pci_ioda2_set_bypass(&pe->tce32.table, true);
}
+static struct spapr_tce_iommu_ops pnv_pci_ioda2_ops = {
+ .get_table = pnv_ioda1_iommu_get_table,
+};
+
static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
struct pnv_ioda_pe *pe)
{
@@ -1202,7 +1228,8 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
tbl->it_type |= (TCE_PCI_SWINV_CREATE | TCE_PCI_SWINV_FREE);
}
iommu_init_table(tbl, phb->hose->node);
- iommu_register_group(tbl, phb->hose->global_number, pe->pe_number);
+ iommu_register_group(tbl, pe, &pnv_pci_ioda2_ops,
+ phb->hose->global_number, pe->pe_number);
if (pe->pdev)
set_iommu_table_base_and_group(&pe->pdev->dev, tbl);
diff --git a/arch/powerpc/platforms/powernv/pci-p5ioc2.c b/arch/powerpc/platforms/powernv/pci-p5ioc2.c
index 94ce348..b79066d 100644
--- a/arch/powerpc/platforms/powernv/pci-p5ioc2.c
+++ b/arch/powerpc/platforms/powernv/pci-p5ioc2.c
@@ -89,6 +89,7 @@ static void pnv_pci_p5ioc2_dma_dev_setup(struct pnv_phb *phb,
if (phb->p5ioc2.iommu_table.it_map == NULL) {
iommu_init_table(&phb->p5ioc2.iommu_table, phb->hose->node);
iommu_register_group(&phb->p5ioc2.iommu_table,
+ NULL, NULL,
pci_domain_nr(phb->hose->bus), phb->opal_id);
}
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 97895d4..6ffac79 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -723,7 +723,7 @@ static struct iommu_table *pnv_pci_setup_bml_iommu(struct pci_controller *hose)
pnv_pci_setup_iommu_table(tbl, __va(be64_to_cpup(basep)),
be32_to_cpup(sizep), 0, IOMMU_PAGE_SHIFT_4K);
iommu_init_table(tbl, hose->node);
- iommu_register_group(tbl, pci_domain_nr(hose->bus), 0);
+ iommu_register_group(tbl, NULL, NULL, pci_domain_nr(hose->bus), 0);
/* Deal with SW invalidated TCEs when needed (BML way) */
swinvp = of_get_property(hose->dn, "linux,tce-sw-invalidate-info",
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 4642d6a..b95f8cf 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -616,7 +616,7 @@ static void pci_dma_bus_setup_pSeries(struct pci_bus *bus)
iommu_table_setparms(pci->phb, dn, tbl);
pci->iommu_table = iommu_init_table(tbl, pci->phb->node);
- iommu_register_group(tbl, pci_domain_nr(bus), 0);
+ iommu_register_group(tbl, NULL, NULL, pci_domain_nr(bus), 0);
/* Divide the rest (1.75GB) among the children */
pci->phb->dma_window_size = 0x80000000ul;
@@ -661,7 +661,7 @@ static void pci_dma_bus_setup_pSeriesLP(struct pci_bus *bus)
ppci->phb->node);
iommu_table_setparms_lpar(ppci->phb, pdn, tbl, dma_window);
ppci->iommu_table = iommu_init_table(tbl, ppci->phb->node);
- iommu_register_group(tbl, pci_domain_nr(bus), 0);
+ iommu_register_group(tbl, NULL, NULL, pci_domain_nr(bus), 0);
pr_debug(" created table: %p\n", ppci->iommu_table);
}
}
@@ -688,7 +688,8 @@ static void pci_dma_dev_setup_pSeries(struct pci_dev *dev)
phb->node);
iommu_table_setparms(phb, dn, tbl);
PCI_DN(dn)->iommu_table = iommu_init_table(tbl, phb->node);
- iommu_register_group(tbl, pci_domain_nr(phb->bus), 0);
+ iommu_register_group(tbl, NULL, NULL,
+ pci_domain_nr(phb->bus), 0);
set_iommu_table_base_and_group(&dev->dev,
PCI_DN(dn)->iommu_table);
return;
@@ -1105,7 +1106,8 @@ static void pci_dma_dev_setup_pSeriesLP(struct pci_dev *dev)
pci->phb->node);
iommu_table_setparms_lpar(pci->phb, pdn, tbl, dma_window);
pci->iommu_table = iommu_init_table(tbl, pci->phb->node);
- iommu_register_group(tbl, pci_domain_nr(pci->phb->bus), 0);
+ iommu_register_group(tbl, NULL, NULL,
+ pci_domain_nr(pci->phb->bus), 0);
pr_debug(" created table: %p\n", pci->iommu_table);
} else {
pr_debug(" found DMA window, table: %p\n", pci->iommu_table);
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 730b4ef..a8adfbd 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -43,17 +43,51 @@ static void tce_iommu_detach_group(void *iommu_data,
*/
struct tce_container {
struct mutex lock;
- struct iommu_table *tbl;
+ struct iommu_group *grp;
+ long windows_num;
bool enabled;
};
+static struct iommu_table *spapr_tce_find_table(
+ struct tce_container *container,
+ struct spapr_tce_iommu_group *data,
+ phys_addr_t ioba)
+{
+ long i;
+ struct iommu_table *ret = NULL;
+
+ mutex_lock(&container->lock);
+ for (i = 0; i < container->windows_num; ++i) {
+ struct iommu_table *tbl = data->ops->get_table(data, i);
+
+ if (tbl) {
+ unsigned long entry = ioba >> tbl->it_page_shift;
+ unsigned long start = tbl->it_offset;
+ unsigned long end = start + tbl->it_size;
+
+ if ((start <= entry) && (entry < end)) {
+ ret = tbl;
+ break;
+ }
+ }
+ }
+ mutex_unlock(&container->lock);
+
+ return ret;
+}
+
static int tce_iommu_enable(struct tce_container *container)
{
int ret = 0;
unsigned long locked, lock_limit, npages;
- struct iommu_table *tbl = container->tbl;
+ struct iommu_table *tbl;
+ struct spapr_tce_iommu_group *data;
- if (!container->tbl)
+ if (!container->grp)
+ return -ENXIO;
+
+ data = iommu_group_get_iommudata(container->grp);
+ if (!data || !data->iommu_owner || !data->ops->get_table)
return -ENXIO;
if (!current->mm)
@@ -80,6 +114,10 @@ static int tce_iommu_enable(struct tce_container *container)
* that would effectively kill the guest at random points, much better
* enforcing the limit based on the max that the guest can map.
*/
+ tbl = data->ops->get_table(data, 0);
+ if (!tbl)
+ return -ENXIO;
+
down_write(¤t->mm->mmap_sem);
npages = (tbl->it_size << IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
locked = current->mm->locked_vm + npages;
@@ -89,7 +127,6 @@ static int tce_iommu_enable(struct tce_container *container)
rlimit(RLIMIT_MEMLOCK));
ret = -ENOMEM;
} else {
-
current->mm->locked_vm += npages;
container->enabled = true;
}
@@ -100,16 +137,27 @@ static int tce_iommu_enable(struct tce_container *container)
static void tce_iommu_disable(struct tce_container *container)
{
+ struct spapr_tce_iommu_group *data;
+ struct iommu_table *tbl;
+
if (!container->enabled)
return;
container->enabled = false;
- if (!container->tbl || !current->mm)
+ if (!container->grp || !current->mm)
+ return;
+
+ data = iommu_group_get_iommudata(container->grp);
+ if (!data || !data->iommu_owner || !data->ops->get_table)
+ return;
+
+ tbl = data->ops->get_table(data, 0);
+ if (!tbl)
return;
down_write(¤t->mm->mmap_sem);
- current->mm->locked_vm -= (container->tbl->it_size <<
+ current->mm->locked_vm -= (tbl->it_size <<
IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
up_write(¤t->mm->mmap_sem);
}
@@ -129,6 +177,8 @@ static void *tce_iommu_open(unsigned long arg)
mutex_init(&container->lock);
+ container->windows_num = 1;
+
return container;
}
@@ -136,11 +186,11 @@ static void tce_iommu_release(void *iommu_data)
{
struct tce_container *container = iommu_data;
- WARN_ON(container->tbl && !container->tbl->it_group);
+ WARN_ON(container->grp);
tce_iommu_disable(container);
- if (container->tbl && container->tbl->it_group)
- tce_iommu_detach_group(iommu_data, container->tbl->it_group);
+ if (container->grp)
+ tce_iommu_detach_group(iommu_data, container->grp);
mutex_destroy(&container->lock);
@@ -169,8 +219,17 @@ static long tce_iommu_ioctl(void *iommu_data,
case VFIO_IOMMU_SPAPR_TCE_GET_INFO: {
struct vfio_iommu_spapr_tce_info info;
- struct iommu_table *tbl = container->tbl;
+ struct iommu_table *tbl;
+ struct spapr_tce_iommu_group *data;
+ if (WARN_ON(!container->grp))
+ return -ENXIO;
+
+ data = iommu_group_get_iommudata(container->grp);
+ if (WARN_ON(!data || !data->iommu_owner || !data->ops))
+ return -ENXIO;
+
+ tbl = data->ops->get_table(data, 0);
if (WARN_ON(!tbl))
return -ENXIO;
@@ -194,13 +253,16 @@ static long tce_iommu_ioctl(void *iommu_data,
}
case VFIO_IOMMU_MAP_DMA: {
struct vfio_iommu_type1_dma_map param;
- struct iommu_table *tbl = container->tbl;
+ struct iommu_table *tbl;
+ struct spapr_tce_iommu_group *data;
unsigned long tce, i;
- if (!tbl)
+ if (WARN_ON(!container->grp))
return -ENXIO;
- BUG_ON(!tbl->it_group);
+ data = iommu_group_get_iommudata(container->grp);
+ if (WARN_ON(!data || !data->iommu_owner || !data->ops))
+ return -ENXIO;
minsz = offsetofend(struct vfio_iommu_type1_dma_map, size);
@@ -225,6 +287,11 @@ static long tce_iommu_ioctl(void *iommu_data,
if (param.flags & VFIO_DMA_MAP_FLAG_WRITE)
tce |= TCE_PCI_WRITE;
+ tbl = spapr_tce_find_table(container, data, param.iova);
+ if (!tbl)
+ return -ENXIO;
+ BUG_ON(!tbl->it_group);
+
ret = iommu_tce_put_param_check(tbl, param.iova, tce);
if (ret)
return ret;
@@ -247,9 +314,14 @@ static long tce_iommu_ioctl(void *iommu_data,
}
case VFIO_IOMMU_UNMAP_DMA: {
struct vfio_iommu_type1_dma_unmap param;
- struct iommu_table *tbl = container->tbl;
+ struct iommu_table *tbl;
+ struct spapr_tce_iommu_group *data;
- if (WARN_ON(!tbl))
+ if (WARN_ON(!container->grp))
+ return -ENXIO;
+
+ data = iommu_group_get_iommudata(container->grp);
+ if (WARN_ON(!data || !data->iommu_owner || !data->ops))
return -ENXIO;
minsz = offsetofend(struct vfio_iommu_type1_dma_unmap,
@@ -268,6 +340,12 @@ static long tce_iommu_ioctl(void *iommu_data,
if (param.size & ~IOMMU_PAGE_MASK_4K)
return -EINVAL;
+ tbl = spapr_tce_find_table(container, data, param.iova);
+ if (!tbl)
+ return -ENXIO;
+
+ BUG_ON(!tbl->it_group);
+
ret = iommu_tce_clear_param_check(tbl, param.iova, 0,
param.size >> IOMMU_PAGE_SHIFT_4K);
if (ret)
@@ -293,10 +371,10 @@ static long tce_iommu_ioctl(void *iommu_data,
mutex_unlock(&container->lock);
return 0;
case VFIO_EEH_PE_OP:
- if (!container->tbl || !container->tbl->it_group)
+ if (!container->grp)
return -ENODEV;
- return vfio_spapr_iommu_eeh_ioctl(container->tbl->it_group,
+ return vfio_spapr_iommu_eeh_ioctl(container->grp,
cmd, arg);
}
@@ -308,16 +386,16 @@ static int tce_iommu_attach_group(void *iommu_data,
{
int ret;
struct tce_container *container = iommu_data;
- struct iommu_table *tbl = iommu_group_get_iommudata(iommu_group);
+ struct iommu_table *tbl;
+ struct spapr_tce_iommu_group *data;
- BUG_ON(!tbl);
mutex_lock(&container->lock);
/* pr_debug("tce_vfio: Attaching group #%u to iommu %p\n",
iommu_group_id(iommu_group), iommu_group); */
- if (container->tbl) {
+ if (container->grp) {
pr_warn("tce_vfio: Only one group per IOMMU container is allowed, existing id=%d, attaching id=%d\n",
- iommu_group_id(container->tbl->it_group),
+ iommu_group_id(container->grp),
iommu_group_id(iommu_group));
ret = -EBUSY;
} else if (container->enabled) {
@@ -325,9 +403,16 @@ static int tce_iommu_attach_group(void *iommu_data,
iommu_group_id(iommu_group));
ret = -EBUSY;
} else {
+ data = iommu_group_get_iommudata(iommu_group);
+ if (WARN_ON(!data || !data->iommu_owner || !data->ops))
+ return -ENXIO;
+
+ tbl = data->ops->get_table(data, 0);
+ BUG_ON(!tbl);
+
ret = iommu_take_ownership(tbl);
if (!ret)
- container->tbl = tbl;
+ container->grp = iommu_group;
}
mutex_unlock(&container->lock);
@@ -339,24 +424,31 @@ static void tce_iommu_detach_group(void *iommu_data,
struct iommu_group *iommu_group)
{
struct tce_container *container = iommu_data;
- struct iommu_table *tbl = iommu_group_get_iommudata(iommu_group);
+ struct iommu_table *tbl;
+ struct spapr_tce_iommu_group *data;
- BUG_ON(!tbl);
mutex_lock(&container->lock);
- if (tbl != container->tbl) {
+ if (iommu_group != container->grp) {
pr_warn("tce_vfio: detaching group #%u, expected group is #%u\n",
iommu_group_id(iommu_group),
- iommu_group_id(tbl->it_group));
+ iommu_group_id(container->grp));
} else {
if (container->enabled) {
pr_warn("tce_vfio: detaching group #%u from enabled container, forcing disable\n",
- iommu_group_id(tbl->it_group));
+ iommu_group_id(container->grp));
tce_iommu_disable(container);
}
/* pr_debug("tce_vfio: detaching group #%u from iommu %p\n",
iommu_group_id(iommu_group), iommu_group); */
- container->tbl = NULL;
+ container->grp = NULL;
+
+ data = iommu_group_get_iommudata(iommu_group);
+ BUG_ON(!data || !data->iommu_owner || !data->ops);
+
+ tbl = data->ops->get_table(data, 0);
+ BUG_ON(!tbl);
+
iommu_release_ownership(tbl);
}
mutex_unlock(&container->lock);
--
2.0.0
^ permalink raw reply related
* [PATCH v2 05/13] powerpc/iommu: Fix IOMMU ownership control functions
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
This adds missing locks in iommu_take_ownership()/
iommu_release_ownership().
This marks all pages busy in iommu_table::it_map in order to catch
errors if there is an attempt to use this table while ownership over it
is taken.
This only clears TCE content if there is no page marked busy in it_map.
Clearing must be done outside of the table locks as iommu_clear_tce()
called from iommu_clear_tces_and_put_pages() does this.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/kernel/iommu.c | 36 +++++++++++++++++++++++++++++-------
1 file changed, 29 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index c2c8d9d..cd80867 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -1126,33 +1126,55 @@ EXPORT_SYMBOL_GPL(iommu_put_tce_user_mode);
int iommu_take_ownership(struct iommu_table *tbl)
{
- unsigned long sz = (tbl->it_size + 7) >> 3;
+ unsigned long flags, i, sz = (tbl->it_size + 7) >> 3;
+ int ret = 0, bit0 = 0;
+
+ spin_lock_irqsave(&tbl->large_pool.lock, flags);
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_lock(&tbl->pools[i].lock);
if (tbl->it_offset == 0)
- clear_bit(0, tbl->it_map);
+ bit0 = test_and_clear_bit(0, tbl->it_map);
if (!bitmap_empty(tbl->it_map, tbl->it_size)) {
pr_err("iommu_tce: it_map is not empty");
- return -EBUSY;
+ ret = -EBUSY;
+ if (bit0)
+ set_bit(0, tbl->it_map);
+ } else {
+ memset(tbl->it_map, 0xff, sz);
}
- memset(tbl->it_map, 0xff, sz);
- iommu_clear_tces_and_put_pages(tbl, tbl->it_offset, tbl->it_size);
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_unlock(&tbl->pools[i].lock);
+ spin_unlock_irqrestore(&tbl->large_pool.lock, flags);
- return 0;
+ if (!ret)
+ iommu_clear_tces_and_put_pages(tbl, tbl->it_offset,
+ tbl->it_size);
+ return ret;
}
EXPORT_SYMBOL_GPL(iommu_take_ownership);
void iommu_release_ownership(struct iommu_table *tbl)
{
- unsigned long sz = (tbl->it_size + 7) >> 3;
+ unsigned long flags, i, sz = (tbl->it_size + 7) >> 3;
iommu_clear_tces_and_put_pages(tbl, tbl->it_offset, tbl->it_size);
+
+ spin_lock_irqsave(&tbl->large_pool.lock, flags);
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_lock(&tbl->pools[i].lock);
+
memset(tbl->it_map, 0, sz);
/* Restore bit#0 set by iommu_init_table() */
if (tbl->it_offset == 0)
set_bit(0, tbl->it_map);
+
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_unlock(&tbl->pools[i].lock);
+ spin_unlock_irqrestore(&tbl->large_pool.lock, flags);
}
EXPORT_SYMBOL_GPL(iommu_release_ownership);
--
2.0.0
^ permalink raw reply related
* [PATCH v2 09/13] powerpc/pseries/lpar: Enable VFIO
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
The previous patch introduced iommu_table_ops::exchange() callback
which effectively disabled VFIO on pseries. This implements exchange()
for pseries/lpar so VFIO can work in nested guests.
Since exchaange() callback returns an old TCE, it has to call H_GET_TCE
for every TCE being put to the table so VFIO performance in guests
running under PR KVM is expected to be slower than in guests running under
HV KVM or bare metal hosts.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/platforms/pseries/iommu.c | 25 +++++++++++++++++++++++--
1 file changed, 23 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 9a7364f..ae15b5a 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -138,13 +138,14 @@ static void tce_freemulti_pSeriesLP(struct iommu_table*, long, long);
static int tce_build_pSeriesLP(struct iommu_table *tbl, long tcenum,
long npages, unsigned long uaddr,
+ unsigned long *old_tces,
enum dma_data_direction direction,
struct dma_attrs *attrs)
{
u64 rc = 0;
u64 proto_tce, tce;
u64 rpn;
- int ret = 0;
+ int ret = 0, i = 0;
long tcenum_start = tcenum, npages_start = npages;
rpn = __pa(uaddr) >> TCE_SHIFT;
@@ -154,6 +155,9 @@ static int tce_build_pSeriesLP(struct iommu_table *tbl, long tcenum,
while (npages--) {
tce = proto_tce | (rpn & TCE_RPN_MASK) << TCE_RPN_SHIFT;
+ if (old_tces)
+ plpar_tce_get((u64)tbl->it_index, (u64)tcenum << 12,
+ &old_tces[i++]);
rc = plpar_tce_put((u64)tbl->it_index, (u64)tcenum << 12, tce);
if (unlikely(rc == H_NOT_ENOUGH_RESOURCES)) {
@@ -179,8 +183,9 @@ static int tce_build_pSeriesLP(struct iommu_table *tbl, long tcenum,
static DEFINE_PER_CPU(__be64 *, tce_page);
-static int tce_buildmulti_pSeriesLP(struct iommu_table *tbl, long tcenum,
+static int tce_xchg_pSeriesLP(struct iommu_table *tbl, long tcenum,
long npages, unsigned long uaddr,
+ unsigned long *old_tces,
enum dma_data_direction direction,
struct dma_attrs *attrs)
{
@@ -195,6 +200,7 @@ static int tce_buildmulti_pSeriesLP(struct iommu_table *tbl, long tcenum,
if ((npages == 1) || !firmware_has_feature(FW_FEATURE_MULTITCE)) {
return tce_build_pSeriesLP(tbl, tcenum, npages, uaddr,
+ old_tces,
direction, attrs);
}
@@ -211,6 +217,7 @@ static int tce_buildmulti_pSeriesLP(struct iommu_table *tbl, long tcenum,
if (!tcep) {
local_irq_restore(flags);
return tce_build_pSeriesLP(tbl, tcenum, npages, uaddr,
+ old_tces,
direction, attrs);
}
__get_cpu_var(tce_page) = tcep;
@@ -232,6 +239,10 @@ static int tce_buildmulti_pSeriesLP(struct iommu_table *tbl, long tcenum,
for (l = 0; l < limit; l++) {
tcep[l] = cpu_to_be64(proto_tce | (rpn & TCE_RPN_MASK) << TCE_RPN_SHIFT);
rpn++;
+ if (old_tces)
+ plpar_tce_get((u64)tbl->it_index,
+ (u64)(tcenum + l) << 12,
+ &old_tces[tcenum + l]);
}
rc = plpar_tce_put_indirect((u64)tbl->it_index,
@@ -262,6 +273,15 @@ static int tce_buildmulti_pSeriesLP(struct iommu_table *tbl, long tcenum,
return ret;
}
+static int tce_buildmulti_pSeriesLP(struct iommu_table *tbl, long tcenum,
+ long npages, unsigned long uaddr,
+ enum dma_data_direction direction,
+ struct dma_attrs *attrs)
+{
+ return tce_xchg_pSeriesLP(tbl, tcenum, npages, uaddr, NULL,
+ direction, attrs);
+}
+
static void tce_free_pSeriesLP(struct iommu_table *tbl, long tcenum, long npages)
{
u64 rc;
@@ -637,6 +657,7 @@ static void pci_dma_bus_setup_pSeries(struct pci_bus *bus)
struct iommu_table_ops iommu_table_lpar_multi_ops = {
.set = tce_buildmulti_pSeriesLP,
+ .exchange = tce_xchg_pSeriesLP,
.clear = tce_freemulti_pSeriesLP,
.get = tce_get_pSeriesLP
};
--
2.0.0
^ permalink raw reply related
* [PATCH v2 12/13] vfio: powerpc/spapr: Use it_page_size
From: Alexey Kardashevskiy @ 2014-09-23 3:01 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
This makes use of the it_page_size from the iommu_table struct
as page size can differ.
This replaces missing IOMMU_PAGE_SHIFT macro in commented debug code
as recently introduced IOMMU_PAGE_XXX macros do not include
IOMMU_PAGE_SHIFT.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
drivers/vfio/vfio_iommu_spapr_tce.c | 36 ++++++++++++++++++------------------
1 file changed, 18 insertions(+), 18 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index c9fac97..0dccbc4 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -36,7 +36,7 @@ static long try_increment_locked_vm(struct iommu_table *tbl)
if (!current || !current->mm)
return -ESRCH; /* process exited */
- npages = (tbl->it_size << IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
+ npages = (tbl->it_size << tbl->it_page_shift) >> PAGE_SHIFT;
down_write(¤t->mm->mmap_sem);
locked = current->mm->locked_vm + npages;
@@ -60,7 +60,7 @@ static void decrement_locked_vm(struct iommu_table *tbl)
if (!current || !current->mm)
return; /* process exited */
- npages = (tbl->it_size << IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
+ npages = (tbl->it_size << tbl->it_page_shift) >> PAGE_SHIFT;
down_write(¤t->mm->mmap_sem);
if (npages > current->mm->locked_vm)
@@ -284,8 +284,8 @@ static long tce_iommu_ioctl(void *iommu_data,
if (info.argsz < minsz)
return -EINVAL;
- info.dma32_window_start = tbl->it_offset << IOMMU_PAGE_SHIFT_4K;
- info.dma32_window_size = tbl->it_size << IOMMU_PAGE_SHIFT_4K;
+ info.dma32_window_start = tbl->it_offset << tbl->it_page_shift;
+ info.dma32_window_size = tbl->it_size << tbl->it_page_shift;
info.flags = 0;
if (copy_to_user((void __user *)arg, &info, minsz))
@@ -318,10 +318,6 @@ static long tce_iommu_ioctl(void *iommu_data,
VFIO_DMA_MAP_FLAG_WRITE))
return -EINVAL;
- if ((param.size & ~IOMMU_PAGE_MASK_4K) ||
- (param.vaddr & ~IOMMU_PAGE_MASK_4K))
- return -EINVAL;
-
/* iova is checked by the IOMMU API */
tce = param.vaddr;
if (param.flags & VFIO_DMA_MAP_FLAG_READ)
@@ -334,21 +330,25 @@ static long tce_iommu_ioctl(void *iommu_data,
return -ENXIO;
BUG_ON(!tbl->it_group);
+ if ((param.size & ~IOMMU_PAGE_MASK(tbl)) ||
+ (param.vaddr & ~IOMMU_PAGE_MASK(tbl)))
+ return -EINVAL;
+
ret = iommu_tce_put_param_check(tbl, param.iova, tce);
if (ret)
return ret;
- for (i = 0; i < (param.size >> IOMMU_PAGE_SHIFT_4K); ++i) {
+ for (i = 0; i < (param.size >> tbl->it_page_shift); ++i) {
ret = iommu_put_tce_user_mode(tbl,
- (param.iova >> IOMMU_PAGE_SHIFT_4K) + i,
+ (param.iova >> tbl->it_page_shift) + i,
tce);
if (ret)
break;
- tce += IOMMU_PAGE_SIZE_4K;
+ tce += IOMMU_PAGE_SIZE(tbl);
}
if (ret)
iommu_clear_tces_and_put_pages(tbl,
- param.iova >> IOMMU_PAGE_SHIFT_4K, i);
+ param.iova >> tbl->it_page_shift, i);
iommu_flush_tce(tbl);
@@ -379,23 +379,23 @@ static long tce_iommu_ioctl(void *iommu_data,
if (param.flags)
return -EINVAL;
- if (param.size & ~IOMMU_PAGE_MASK_4K)
- return -EINVAL;
-
tbl = spapr_tce_find_table(container, data, param.iova);
if (!tbl)
return -ENXIO;
+ if (param.size & ~IOMMU_PAGE_MASK(tbl))
+ return -EINVAL;
+
BUG_ON(!tbl->it_group);
ret = iommu_tce_clear_param_check(tbl, param.iova, 0,
- param.size >> IOMMU_PAGE_SHIFT_4K);
+ param.size >> tbl->it_page_shift);
if (ret)
return ret;
ret = iommu_clear_tces_and_put_pages(tbl,
- param.iova >> IOMMU_PAGE_SHIFT_4K,
- param.size >> IOMMU_PAGE_SHIFT_4K);
+ param.iova >> tbl->it_page_shift,
+ param.size >> tbl->it_page_shift);
iommu_flush_tce(tbl);
return ret;
--
2.0.0
^ permalink raw reply related
* [PATCH v2 13/13] vfio: powerpc/spapr: Enable Dynamic DMA windows
From: Alexey Kardashevskiy @ 2014-09-23 3:01 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
This defines and implements VFIO IOMMU API which lets the userspace
create and remove DMA windows.
This updates VFIO_IOMMU_SPAPR_TCE_GET_INFO to return the number of
available windows and page mask.
This adds VFIO_IOMMU_SPAPR_TCE_CREATE and VFIO_IOMMU_SPAPR_TCE_REMOVE
to allow the user space to create and remove window(s).
The VFIO IOMMU driver does basic sanity checks and calls corresponding
SPAPR TCE functions. At the moment only IODA2 (POWER8 PCI host bridge)
implements them.
This advertises VFIO_IOMMU_SPAPR_TCE_FLAG_DDW capability via
VFIO_IOMMU_SPAPR_TCE_GET_INFO.
This calls platform DDW reset() callback when IOMMU is being disabled
to reset the DMA configuration to its original state.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
drivers/vfio/vfio_iommu_spapr_tce.c | 135 ++++++++++++++++++++++++++++++++++--
include/uapi/linux/vfio.h | 25 ++++++-
2 files changed, 153 insertions(+), 7 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 0dccbc4..b518891 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -190,18 +190,25 @@ static void tce_iommu_disable(struct tce_container *container)
container->enabled = false;
- if (!container->grp || !current->mm)
+ if (!container->grp)
return;
data = iommu_group_get_iommudata(container->grp);
if (!data || !data->iommu_owner || !data->ops->get_table)
return;
- tbl = data->ops->get_table(data, 0);
- if (!tbl)
- return;
+ if (current->mm) {
+ tbl = data->ops->get_table(data, 0);
+ if (tbl)
+ decrement_locked_vm(tbl);
- decrement_locked_vm(tbl);
+ tbl = data->ops->get_table(data, 1);
+ if (tbl)
+ decrement_locked_vm(tbl);
+ }
+
+ if (data->ops->reset)
+ data->ops->reset(data);
}
static void *tce_iommu_open(unsigned long arg)
@@ -243,7 +250,7 @@ static long tce_iommu_ioctl(void *iommu_data,
unsigned int cmd, unsigned long arg)
{
struct tce_container *container = iommu_data;
- unsigned long minsz;
+ unsigned long minsz, ddwsz;
long ret;
switch (cmd) {
@@ -288,6 +295,28 @@ static long tce_iommu_ioctl(void *iommu_data,
info.dma32_window_size = tbl->it_size << tbl->it_page_shift;
info.flags = 0;
+ ddwsz = offsetofend(struct vfio_iommu_spapr_tce_info,
+ page_size_mask);
+
+ if (info.argsz == ddwsz) {
+ if (data->ops->query && data->ops->create &&
+ data->ops->remove) {
+ info.flags |= VFIO_IOMMU_SPAPR_TCE_FLAG_DDW;
+
+ ret = data->ops->query(data,
+ &info.current_windows,
+ &info.windows_available,
+ &info.page_size_mask);
+ if (ret)
+ return ret;
+ } else {
+ info.current_windows = 0;
+ info.windows_available = 0;
+ info.page_size_mask = 0;
+ }
+ minsz = ddwsz;
+ }
+
if (copy_to_user((void __user *)arg, &info, minsz))
return -EFAULT;
@@ -412,12 +441,106 @@ static long tce_iommu_ioctl(void *iommu_data,
tce_iommu_disable(container);
mutex_unlock(&container->lock);
return 0;
+
case VFIO_EEH_PE_OP:
if (!container->grp)
return -ENODEV;
return vfio_spapr_iommu_eeh_ioctl(container->grp,
cmd, arg);
+
+ case VFIO_IOMMU_SPAPR_TCE_CREATE: {
+ struct vfio_iommu_spapr_tce_create create;
+ struct spapr_tce_iommu_group *data;
+ struct iommu_table *tbl;
+
+ if (WARN_ON(!container->grp))
+ return -ENXIO;
+
+ data = iommu_group_get_iommudata(container->grp);
+
+ minsz = offsetofend(struct vfio_iommu_spapr_tce_create,
+ start_addr);
+
+ if (copy_from_user(&create, (void __user *)arg, minsz))
+ return -EFAULT;
+
+ if (create.argsz < minsz)
+ return -EINVAL;
+
+ if (create.flags)
+ return -EINVAL;
+
+ if (!data->ops->create || !data->iommu_owner)
+ return -ENOSYS;
+
+ BUG_ON(!data || !data->ops || !data->ops->remove);
+
+ ret = data->ops->create(data, create.page_shift,
+ create.window_shift, &tbl);
+ if (ret)
+ return ret;
+
+ ret = try_increment_locked_vm(tbl);
+ if (ret) {
+ data->ops->remove(data, tbl);
+ return ret;
+ }
+
+ create.start_addr = tbl->it_offset << tbl->it_page_shift;
+
+ if (copy_to_user((void __user *)arg, &create, minsz)) {
+ data->ops->remove(data, tbl);
+ decrement_locked_vm(tbl);
+ return -EFAULT;
+ }
+ mutex_lock(&container->lock);
+ ++container->windows_num;
+ mutex_unlock(&container->lock);
+
+ return ret;
+ }
+ case VFIO_IOMMU_SPAPR_TCE_REMOVE: {
+ struct vfio_iommu_spapr_tce_remove remove;
+ struct spapr_tce_iommu_group *data;
+ struct iommu_table *tbl;
+
+ if (WARN_ON(!container->grp))
+ return -ENXIO;
+
+ data = iommu_group_get_iommudata(container->grp);
+
+ minsz = offsetofend(struct vfio_iommu_spapr_tce_remove,
+ start_addr);
+
+ if (copy_from_user(&remove, (void __user *)arg, minsz))
+ return -EFAULT;
+
+ if (remove.argsz < minsz)
+ return -EINVAL;
+
+ if (remove.flags)
+ return -EINVAL;
+
+ if (!data->ops->remove || !data->iommu_owner)
+ return -ENOSYS;
+
+ tbl = spapr_tce_find_table(container, data, remove.start_addr);
+ if (!tbl)
+ return -EINVAL;
+
+ ret = data->ops->remove(data, tbl);
+ if (ret)
+ return ret;
+
+ decrement_locked_vm(tbl);
+
+ mutex_lock(&container->lock);
+ --container->windows_num;
+ mutex_unlock(&container->lock);
+
+ return 0;
+ }
}
return -ENOTTY;
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 6612974..e71a6ef 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -451,9 +451,13 @@ struct vfio_iommu_type1_dma_unmap {
*/
struct vfio_iommu_spapr_tce_info {
__u32 argsz;
- __u32 flags; /* reserved for future use */
+ __u32 flags;
+#define VFIO_IOMMU_SPAPR_TCE_FLAG_DDW 1 /* Support dynamic windows */
__u32 dma32_window_start; /* 32 bit window start (bytes) */
__u32 dma32_window_size; /* 32 bit window size (bytes) */
+ __u32 current_windows;
+ __u32 windows_available;
+ __u32 page_size_mask;
};
#define VFIO_IOMMU_SPAPR_TCE_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
@@ -489,6 +493,25 @@ struct vfio_eeh_pe_op {
#define VFIO_EEH_PE_OP _IO(VFIO_TYPE, VFIO_BASE + 21)
+struct vfio_iommu_spapr_tce_create {
+ __u32 argsz;
+ __u32 flags;
+ /* in */
+ __u32 page_shift;
+ __u32 window_shift;
+ /* out */
+ __u64 start_addr;
+};
+#define VFIO_IOMMU_SPAPR_TCE_CREATE _IO(VFIO_TYPE, VFIO_BASE + 18)
+
+struct vfio_iommu_spapr_tce_remove {
+ __u32 argsz;
+ __u32 flags;
+ /* in */
+ __u64 start_addr;
+};
+#define VFIO_IOMMU_SPAPR_TCE_REMOVE _IO(VFIO_TYPE, VFIO_BASE + 19)
+
/* ***************************************************************** */
#endif /* _UAPIVFIO_H */
--
2.0.0
^ permalink raw reply related
* [PATCH v2 07/13] powerpc/powernv: Do not set "read" flag if direction==DMA_NONE
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
Normally a bitmap from the iommu_table is used to track what TCE entry
is in use. Since we are going to use iommu_table without its locks and
do xchg() instead, it becomes essential not to put bits which are not
implied in the direction flag.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/platforms/powernv/pci.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index deddcad..ab79e2d 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -628,10 +628,18 @@ static int pnv_tce_build(struct iommu_table *tbl, long index, long npages,
__be64 *tcep, *tces;
u64 rpn;
- proto_tce = TCE_PCI_READ; // Read allowed
-
- if (direction != DMA_TO_DEVICE)
- proto_tce |= TCE_PCI_WRITE;
+ switch (direction) {
+ case DMA_BIDIRECTIONAL:
+ case DMA_FROM_DEVICE:
+ proto_tce = TCE_PCI_READ | TCE_PCI_WRITE;
+ break;
+ case DMA_TO_DEVICE:
+ proto_tce = TCE_PCI_READ;
+ break;
+ default:
+ proto_tce = 0;
+ break;
+ }
tces = tcep = ((__be64 *)tbl->it_base) + index - tbl->it_offset;
rpn = __pa(uaddr) >> tbl->it_page_shift;
--
2.0.0
^ permalink raw reply related
* [PATCH v2 08/13] powerpc/powernv: Release replaced TCE
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
At the moment writing new TCE value to the IOMMU table fails with EBUSY
if there is a valid entry already. However PAPR specification allows
the guest to write new TCE value without clearing it first.
Another problem this patch is addressing is the use of pool locks for
external IOMMU users such as VFIO. The pool locks are to protect
DMA page allocator rather than entries and since the host kernel does
not control what pages are in use, there is no point in pool locks and
exchange()+put_page(oldtce) is sufficient to avoid possible races.
This adds an exchange() callback to iommu_table_ops which does the same
thing as set() plus it returns replaced TCE(s) so the caller can release
the pages afterwards.
This makes iommu_tce_build() put pages returned by exchange().
This replaces iommu_clear_tce() with iommu_tce_build which now
can call exchange() with TCE==NULL (i.e. clear).
This preserves permission bits in TCE in iommu_put_tce_user_mode().
This removes use of pool locks for external IOMMU uses.
This disables external IOMMU use (i.e. VFIO) for IOMMUs which do not
implement exchange() callback. Therefore the "powernv" platform is
the only supported one after this patch.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
v2:
* added missing __pa() for TCE which was read from the table
---
arch/powerpc/include/asm/iommu.h | 8 +++--
arch/powerpc/kernel/iommu.c | 62 ++++++++++++------------------------
arch/powerpc/platforms/powernv/pci.c | 40 +++++++++++++++++++++++
3 files changed, 67 insertions(+), 43 deletions(-)
diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h
index c725e4a..8e0537d 100644
--- a/arch/powerpc/include/asm/iommu.h
+++ b/arch/powerpc/include/asm/iommu.h
@@ -49,6 +49,12 @@ struct iommu_table_ops {
unsigned long uaddr,
enum dma_data_direction direction,
struct dma_attrs *attrs);
+ int (*exchange)(struct iommu_table *tbl,
+ long index, long npages,
+ unsigned long uaddr,
+ unsigned long *old_tces,
+ enum dma_data_direction direction,
+ struct dma_attrs *attrs);
void (*clear)(struct iommu_table *tbl,
long index, long npages);
unsigned long (*get)(struct iommu_table *tbl, long index);
@@ -209,8 +215,6 @@ extern int iommu_tce_put_param_check(struct iommu_table *tbl,
unsigned long ioba, unsigned long tce);
extern int iommu_tce_build(struct iommu_table *tbl, unsigned long entry,
unsigned long hwaddr, enum dma_data_direction direction);
-extern unsigned long iommu_clear_tce(struct iommu_table *tbl,
- unsigned long entry);
extern int iommu_clear_tces_and_put_pages(struct iommu_table *tbl,
unsigned long entry, unsigned long pages);
extern int iommu_put_tce_user_mode(struct iommu_table *tbl,
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index 678fee8..39ccce7 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -1006,43 +1006,11 @@ int iommu_tce_put_param_check(struct iommu_table *tbl,
}
EXPORT_SYMBOL_GPL(iommu_tce_put_param_check);
-unsigned long iommu_clear_tce(struct iommu_table *tbl, unsigned long entry)
-{
- unsigned long oldtce;
- struct iommu_pool *pool = get_pool(tbl, entry);
-
- spin_lock(&(pool->lock));
-
- oldtce = tbl->it_ops->get(tbl, entry);
- if (oldtce & (TCE_PCI_WRITE | TCE_PCI_READ))
- tbl->it_ops->clear(tbl, entry, 1);
- else
- oldtce = 0;
-
- spin_unlock(&(pool->lock));
-
- return oldtce;
-}
-EXPORT_SYMBOL_GPL(iommu_clear_tce);
-
int iommu_clear_tces_and_put_pages(struct iommu_table *tbl,
unsigned long entry, unsigned long pages)
{
- unsigned long oldtce;
- struct page *page;
-
for ( ; pages; --pages, ++entry) {
- oldtce = iommu_clear_tce(tbl, entry);
- if (!oldtce)
- continue;
-
- page = pfn_to_page(oldtce >> PAGE_SHIFT);
- WARN_ON(!page);
- if (page) {
- if (oldtce & TCE_PCI_WRITE)
- SetPageDirty(page);
- put_page(page);
- }
+ iommu_tce_build(tbl, entry, 0, DMA_NONE);
}
return 0;
@@ -1056,18 +1024,19 @@ EXPORT_SYMBOL_GPL(iommu_clear_tces_and_put_pages);
int iommu_tce_build(struct iommu_table *tbl, unsigned long entry,
unsigned long hwaddr, enum dma_data_direction direction)
{
- int ret = -EBUSY;
+ int ret;
unsigned long oldtce;
- struct iommu_pool *pool = get_pool(tbl, entry);
- spin_lock(&(pool->lock));
+ ret = tbl->it_ops->exchange(tbl, entry, 1, hwaddr, &oldtce,
+ direction, NULL);
- oldtce = tbl->it_ops->get(tbl, entry);
- /* Add new entry if it is not busy */
- if (!(oldtce & (TCE_PCI_WRITE | TCE_PCI_READ)))
- ret = tbl->it_ops->set(tbl, entry, 1, hwaddr, direction, NULL);
+ if (oldtce & (TCE_PCI_WRITE | TCE_PCI_READ)) {
+ struct page *pg = pfn_to_page(__pa(oldtce) >> PAGE_SHIFT);
- spin_unlock(&(pool->lock));
+ if (oldtce & TCE_PCI_WRITE)
+ SetPageDirty(pg);
+ put_page(pg);
+ }
/* if (unlikely(ret))
pr_err("iommu_tce: %s failed on hwaddr=%lx ioba=%lx kva=%lx ret=%d\n",
@@ -1111,6 +1080,7 @@ int iommu_put_tce_user_mode(struct iommu_table *tbl, unsigned long entry,
}
hwaddr = (unsigned long) page_address(page) + offset;
+ hwaddr |= tce & (TCE_PCI_READ | TCE_PCI_WRITE);
ret = iommu_tce_build(tbl, entry, hwaddr, direction);
if (ret)
@@ -1133,6 +1103,16 @@ int iommu_take_ownership(struct iommu_table *tbl)
unsigned long flags, i, sz = (tbl->it_size + 7) >> 3;
int ret = 0, bit0 = 0;
+ /*
+ * VFIO does not control TCE entries allocation and the guest
+ * can write new TCEs on top of existing ones so iommu_tce_build()
+ * must be able to release old pages. This functionality
+ * requires exchange() callback defined so if it is not
+ * implemented, we disallow taking ownership over the table.
+ */
+ if (!tbl->it_ops->exchange)
+ return -EINVAL;
+
spin_lock_irqsave(&tbl->large_pool.lock, flags);
for (i = 0; i < tbl->nr_pools; i++)
spin_lock(&tbl->pools[i].lock);
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index ab79e2d..052e503 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -662,6 +662,45 @@ static int pnv_tce_build_vm(struct iommu_table *tbl, long index, long npages,
false);
}
+static int pnv_tce_xchg_vm(struct iommu_table *tbl, long index,
+ long npages,
+ unsigned long uaddr, unsigned long *old_tces,
+ enum dma_data_direction direction,
+ struct dma_attrs *attrs)
+{
+ u64 rpn, proto_tce;
+ __be64 *tcep, *tces;
+ long i;
+
+ switch (direction) {
+ case DMA_BIDIRECTIONAL:
+ case DMA_FROM_DEVICE:
+ proto_tce = TCE_PCI_READ | TCE_PCI_WRITE;
+ break;
+ case DMA_TO_DEVICE:
+ proto_tce = TCE_PCI_READ;
+ break;
+ default:
+ proto_tce = 0;
+ break;
+ }
+
+ tces = tcep = ((__be64 *)tbl->it_base) + index - tbl->it_offset;
+ rpn = __pa(uaddr) >> tbl->it_page_shift;
+
+ for (i = 0; i < npages; i++) {
+ unsigned long oldtce = xchg(tcep, cpu_to_be64(proto_tce |
+ (rpn++ << tbl->it_page_shift)));
+
+ old_tces[i] = (unsigned long) __va(be64_to_cpu(oldtce));
+ tcep++;
+ }
+
+ pnv_tce_invalidate(tbl, tces, tcep - 1, false);
+
+ return 0;
+}
+
static void pnv_tce_free(struct iommu_table *tbl, long index, long npages,
bool rm)
{
@@ -687,6 +726,7 @@ static unsigned long pnv_tce_get(struct iommu_table *tbl, long index)
struct iommu_table_ops pnv_iommu_ops = {
.set = pnv_tce_build_vm,
+ .exchange = pnv_tce_xchg_vm,
.clear = pnv_tce_free_vm,
.get = pnv_tce_get,
};
--
2.0.0
^ permalink raw reply related
* [PATCH v2 10/13] powerpc/powernv: Implement Dynamic DMA windows (DDW) for IODA
From: Alexey Kardashevskiy @ 2014-09-23 3:00 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
SPAPR defines an interface to create additional DMA windows dynamically.
"Dynamically" means that the window is not allocated before the guest
even started, the guest can request it later. In practice, existing linux
guests check for the capability and if it is there, they create and map
a DMA window as big as the entire guest RAM.
This adds 4 callbacks to the spapr_tce_iommu_ops struct:
1. query - ibm,query-pe-dma-window - returns number/size of windows
which can be created (one, any page size);
2. create - ibm,create-pe-dma-window - creates a window;
3. remove - ibm,remove-pe-dma-window - removes a window; removing
the default 32bit window is not allowed by this patch, this will be added
later if needed;
4. reset - ibm,reset-pe-dma-window - reset the DMA windows configuration
to the default state; as the default window cannot be removed, it only
removes the additional window if it was created.
The next patch will add corresponding ioctls to VFIO SPAPR TCE driver to
provide necessary support to the userspace.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
arch/powerpc/include/asm/tce.h | 22 +++++
arch/powerpc/platforms/powernv/pci-ioda.c | 159 +++++++++++++++++++++++++++++-
arch/powerpc/platforms/powernv/pci.h | 1 +
3 files changed, 181 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/tce.h b/arch/powerpc/include/asm/tce.h
index e6355f9..23b0362 100644
--- a/arch/powerpc/include/asm/tce.h
+++ b/arch/powerpc/include/asm/tce.h
@@ -58,6 +58,28 @@ struct spapr_tce_iommu_ops {
int num);
void (*take_ownership)(struct spapr_tce_iommu_group *data,
bool enable);
+
+ /* Dynamic DMA window */
+ /* Page size flags for ibm,query-pe-dma-window */
+#define DDW_PGSIZE_4K 0x01
+#define DDW_PGSIZE_64K 0x02
+#define DDW_PGSIZE_16M 0x04
+#define DDW_PGSIZE_32M 0x08
+#define DDW_PGSIZE_64M 0x10
+#define DDW_PGSIZE_128M 0x20
+#define DDW_PGSIZE_256M 0x40
+#define DDW_PGSIZE_16G 0x80
+ long (*query)(struct spapr_tce_iommu_group *data,
+ __u32 *current_windows,
+ __u32 *windows_available,
+ __u32 *page_size_mask);
+ long (*create)(struct spapr_tce_iommu_group *data,
+ __u32 page_shift,
+ __u32 window_shift,
+ struct iommu_table **ptbl);
+ long (*remove)(struct spapr_tce_iommu_group *data,
+ struct iommu_table *tbl);
+ long (*reset)(struct spapr_tce_iommu_group *data);
};
struct spapr_tce_iommu_group {
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 296f49b..a6318cb 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1154,6 +1154,26 @@ static void pnv_pci_ioda2_setup_bypass_pe(struct pnv_phb *phb,
pnv_pci_ioda2_set_bypass(pe, true);
}
+static struct iommu_table *pnv_ioda2_iommu_get_table(
+ struct spapr_tce_iommu_group *data,
+ int num)
+{
+ struct pnv_ioda_pe *pe = data->iommu_owner;
+
+ switch (num) {
+ case 0:
+ if (pe->tce32.table.it_size)
+ return &pe->tce32.table;
+ return NULL;
+ case 1:
+ if (pe->tce64.table.it_size)
+ return &pe->tce64.table;
+ return NULL;
+ default:
+ return NULL;
+ }
+}
+
static void pnv_ioda2_take_ownership(struct spapr_tce_iommu_group *data,
bool enable)
{
@@ -1162,9 +1182,146 @@ static void pnv_ioda2_take_ownership(struct spapr_tce_iommu_group *data,
pnv_pci_ioda2_set_bypass(pe, !enable);
}
+static long pnv_pci_ioda2_ddw_query(struct spapr_tce_iommu_group *data,
+ __u32 *current_windows,
+ __u32 *windows_available, __u32 *page_size_mask)
+{
+ struct pnv_ioda_pe *pe = data->iommu_owner;
+
+ *windows_available = 2;
+ *current_windows = 0;
+ if (pe->tce32.table.it_size) {
+ --*windows_available;
+ ++*current_windows;
+ }
+ if (pe->tce64.table.it_size) {
+ --*windows_available;
+ ++*current_windows;
+ }
+ *page_size_mask =
+ DDW_PGSIZE_4K |
+ DDW_PGSIZE_64K |
+ DDW_PGSIZE_16M;
+
+ return 0;
+}
+
+static long pnv_pci_ioda2_ddw_create(struct spapr_tce_iommu_group *data,
+ __u32 page_shift, __u32 window_shift,
+ struct iommu_table **ptbl)
+{
+ struct pnv_ioda_pe *pe = data->iommu_owner;
+ struct pnv_phb *phb = pe->phb;
+ struct page *tce_mem = NULL;
+ void *addr;
+ long ret;
+ unsigned long tce_table_size =
+ (1ULL << (window_shift - page_shift)) * 8;
+ unsigned order;
+ struct iommu_table *tbl64 = &pe->tce64.table;
+
+ if ((page_shift != 12) && (page_shift != 16) && (page_shift != 24))
+ return -EINVAL;
+
+ if (window_shift > (memory_hotplug_max() >> page_shift))
+ return -EINVAL;
+
+ if (pe->tce64.table.it_size && pe->tce32.table.it_size)
+ return -EBUSY;
+
+ tce_table_size = max(0x1000UL, tce_table_size);
+ order = get_order(tce_table_size);
+
+ pe_info(pe, "Setting up DDW at %llx..%llx ws=0x%x ps=0x%x table_size=0x%lx order=0x%x\n",
+ pe->tce_bypass_base,
+ pe->tce_bypass_base + (1ULL << window_shift) - 1,
+ window_shift, page_shift, tce_table_size, order);
+
+ tce_mem = alloc_pages_node(phb->hose->node, GFP_KERNEL, order);
+ if (!tce_mem) {
+ pe_err(pe, " Failed to allocate a DDW\n");
+ return -EFAULT;
+ }
+ addr = page_address(tce_mem);
+ memset(addr, 0, tce_table_size);
+
+ /* Configure HW */
+ ret = opal_pci_map_pe_dma_window(phb->opal_id,
+ pe->pe_number,
+ (pe->pe_number << 1) + 1, /* Window number */
+ 1,
+ __pa(addr),
+ tce_table_size,
+ 1 << page_shift);
+ if (ret) {
+ pe_err(pe, " Failed to configure 32-bit TCE table, err %ld\n",
+ ret);
+ return -EFAULT;
+ }
+
+ /* Setup linux iommu table */
+ pnv_pci_setup_iommu_table(tbl64, addr, tce_table_size,
+ pe->tce_bypass_base, page_shift);
+ pe->tce64.pe = pe;
+ pe->tce64.invalidate_fn = pnv_pci_ioda2_tce_invalidate;
+
+ /* Copy "invalidate" register address */
+ tbl64->it_index = pe->tce32.table.it_index;
+ tbl64->it_group = pe->tce32.table.it_group;
+ tbl64->it_type = TCE_PCI_SWINV_CREATE | TCE_PCI_SWINV_FREE |
+ TCE_PCI_SWINV_PAIR;
+ tbl64->it_map = (void *) 0xDEADBEEF; /* poison */
+ tbl64->it_ops = pe->tce32.table.it_ops;
+
+ *ptbl = tbl64;
+
+ return 0;
+}
+
+static long pnv_pci_ioda2_ddw_remove(struct spapr_tce_iommu_group *data,
+ struct iommu_table *tbl)
+{
+ struct pnv_ioda_pe *pe = data->iommu_owner;
+ struct pnv_phb *phb = pe->phb;
+ long ret;
+
+ /* Only additional 64bit window removal is supported */
+ if ((tbl != &pe->tce64.table) || !pe->tce64.table.it_size)
+ return -EFAULT;
+
+ pe_info(pe, "Removing huge 64bit DMA window\n");
+
+ iommu_clear_tces_and_put_pages(tbl, tbl->it_offset, tbl->it_size);
+
+ ret = opal_pci_map_pe_dma_window(phb->opal_id,
+ pe->pe_number,
+ (pe->pe_number << 1) + 1,
+ 0/* levels */, 0/* table address */,
+ 0/* table size */, 0/* page size */);
+ if (ret)
+ pe_warn(pe, "Unmapping failed, ret = %ld\n", ret);
+
+ free_pages(tbl->it_base, get_order(tbl->it_size << 3));
+ memset(&pe->tce64, 0, sizeof(pe->tce64));
+
+ return ret;
+}
+
+static long pnv_pci_ioda2_ddw_reset(struct spapr_tce_iommu_group *data)
+{
+ struct pnv_ioda_pe *pe = data->iommu_owner;
+
+ pe_info(pe, "Reset DMA windows\n");
+ return pnv_pci_ioda2_ddw_remove(data, &pe->tce64.table);
+}
+
static struct spapr_tce_iommu_ops pnv_pci_ioda2_ops = {
- .get_table = pnv_ioda1_iommu_get_table,
+ .get_table = pnv_ioda2_iommu_get_table,
.take_ownership = pnv_ioda2_take_ownership,
+ .query = pnv_pci_ioda2_ddw_query,
+ .create = pnv_pci_ioda2_ddw_create,
+ .remove = pnv_pci_ioda2_ddw_remove,
+ .reset = pnv_pci_ioda2_ddw_reset
};
static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index cf68c4b..9941800 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -66,6 +66,7 @@ struct pnv_ioda_pe {
int tce32_segcount;
struct pnv_iommu_table tce32;
phys_addr_t tce_inval_reg_phys;
+ struct pnv_iommu_table tce64;
/* 64-bit TCE bypass region */
bool tce_bypass_enabled;
--
2.0.0
^ permalink raw reply related
* [PATCH v2 11/13] vfio: powerpc/spapr: Move locked_vm accounting to helpers
From: Alexey Kardashevskiy @ 2014-09-23 3:01 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Gavin Shan, linux-kernel,
Alex Williamson
In-Reply-To: <1411441262-11025-1-git-send-email-aik@ozlabs.ru>
There moves locked pages accounting to helpers.
Later they will be reused for Dynamic DMA windows (DDW).
While we are here, update the comment explaining why RLIMIT_MEMLOCK
might be required to be bigger than the guest RAM.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
drivers/vfio/vfio_iommu_spapr_tce.c | 71 +++++++++++++++++++++++++++----------
1 file changed, 53 insertions(+), 18 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 1c1a9c4..c9fac97 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -29,6 +29,46 @@
static void tce_iommu_detach_group(void *iommu_data,
struct iommu_group *iommu_group);
+static long try_increment_locked_vm(struct iommu_table *tbl)
+{
+ long ret = 0, locked, lock_limit, npages;
+
+ if (!current || !current->mm)
+ return -ESRCH; /* process exited */
+
+ npages = (tbl->it_size << IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
+
+ down_write(¤t->mm->mmap_sem);
+ locked = current->mm->locked_vm + npages;
+ lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
+ if (locked > lock_limit && !capable(CAP_IPC_LOCK)) {
+ pr_warn("RLIMIT_MEMLOCK (%ld) exceeded\n",
+ rlimit(RLIMIT_MEMLOCK));
+ ret = -ENOMEM;
+ } else {
+ current->mm->locked_vm += npages;
+ }
+ up_write(¤t->mm->mmap_sem);
+
+ return ret;
+}
+
+static void decrement_locked_vm(struct iommu_table *tbl)
+{
+ long npages;
+
+ if (!current || !current->mm)
+ return; /* process exited */
+
+ npages = (tbl->it_size << IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
+
+ down_write(¤t->mm->mmap_sem);
+ if (npages > current->mm->locked_vm)
+ npages = current->mm->locked_vm;
+ current->mm->locked_vm -= npages;
+ up_write(¤t->mm->mmap_sem);
+}
+
/*
* VFIO IOMMU fd for SPAPR_TCE IOMMU implementation
*
@@ -86,7 +126,6 @@ static void tce_iommu_take_ownership_notify(struct spapr_tce_iommu_group *data,
static int tce_iommu_enable(struct tce_container *container)
{
int ret = 0;
- unsigned long locked, lock_limit, npages;
struct iommu_table *tbl;
struct spapr_tce_iommu_group *data;
@@ -120,24 +159,23 @@ static int tce_iommu_enable(struct tce_container *container)
* Also we don't have a nice way to fail on H_PUT_TCE due to ulimits,
* that would effectively kill the guest at random points, much better
* enforcing the limit based on the max that the guest can map.
+ *
+ * Unfortunately at the moment it counts whole tables, no matter how
+ * much memory the guest has. I.e. for 4GB guest and 4 IOMMU groups
+ * each with 2GB DMA window, 8GB will be counted here. The reason for
+ * this is that we cannot tell here the amount of RAM used by the guest
+ * as this information is only available from KVM and VFIO is
+ * KVM agnostic.
*/
tbl = data->ops->get_table(data, 0);
if (!tbl)
return -ENXIO;
- down_write(¤t->mm->mmap_sem);
- npages = (tbl->it_size << IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
- locked = current->mm->locked_vm + npages;
- lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
- if (locked > lock_limit && !capable(CAP_IPC_LOCK)) {
- pr_warn("RLIMIT_MEMLOCK (%ld) exceeded\n",
- rlimit(RLIMIT_MEMLOCK));
- ret = -ENOMEM;
- } else {
- current->mm->locked_vm += npages;
- container->enabled = true;
- }
- up_write(¤t->mm->mmap_sem);
+ ret = try_increment_locked_vm(tbl);
+ if (ret)
+ return ret;
+
+ container->enabled = true;
return ret;
}
@@ -163,10 +201,7 @@ static void tce_iommu_disable(struct tce_container *container)
if (!tbl)
return;
- down_write(¤t->mm->mmap_sem);
- current->mm->locked_vm -= (tbl->it_size <<
- IOMMU_PAGE_SHIFT_4K) >> PAGE_SHIFT;
- up_write(¤t->mm->mmap_sem);
+ decrement_locked_vm(tbl);
}
static void *tce_iommu_open(unsigned long arg)
--
2.0.0
^ permalink raw reply related
* [PATCH] powerpc/book3s: Fix flush_tlb cpu_spec hook to take a generic argument.
From: Mahesh J Salgaonkar @ 2014-09-23 3:53 UTC (permalink / raw)
To: Michael Ellerman, linuxppc-dev, Paul Mackerras,
Benjamin Herrenschmidt
From: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
The flush_tlb hook in cpu_spec was introduced as a generic function hook
to invalidate TLBs. But the current implementation of flush_tlb hook
takes IS (invalidation selector) as an argument which is architecture
dependent. Hence, It is not right to have a generic routine where caller
has to pass non-generic argument.
This patch fixes this and makes flush_tlb hook as high level API.
The old code used to call flush_tlb hook with IS=0 (single page) resulting
partial invalidation of TLBs which is not right. This fix now makes
sure that whole TLB is invalidated to be able to successfully recover from
TLB and ERAT errors.
Reported-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/cputable.h | 2 +
arch/powerpc/include/asm/mmu-hash64.h | 5 +++
arch/powerpc/kernel/cpu_setup_power.S | 10 +-----
arch/powerpc/kernel/cputable.c | 4 +-
arch/powerpc/kernel/mce_power.c | 57 ++++++++++++++++++++++++++++++++-
arch/powerpc/kvm/book3s_hv_ras.c | 4 +-
6 files changed, 67 insertions(+), 15 deletions(-)
diff --git a/arch/powerpc/include/asm/cputable.h b/arch/powerpc/include/asm/cputable.h
index daa5af9..ae3e74f 100644
--- a/arch/powerpc/include/asm/cputable.h
+++ b/arch/powerpc/include/asm/cputable.h
@@ -100,7 +100,7 @@ struct cpu_spec {
/*
* Processor specific routine to flush tlbs.
*/
- void (*flush_tlb)(unsigned long inval_selector);
+ void (*flush_tlb)(unsigned int action);
};
diff --git a/arch/powerpc/include/asm/mmu-hash64.h b/arch/powerpc/include/asm/mmu-hash64.h
index d765144..068ac8b 100644
--- a/arch/powerpc/include/asm/mmu-hash64.h
+++ b/arch/powerpc/include/asm/mmu-hash64.h
@@ -112,6 +112,11 @@
#define TLBIEL_INVAL_SET_SHIFT 12
#define POWER7_TLB_SETS 128 /* # sets in POWER7 TLB */
+#define POWER8_TLB_SETS 512 /* # sets in POWER8 TLB */
+
+/* TLB flush actions. Used as argument to cpu_spec.flush_tlb() hook */
+#define FLUSH_TLB_ALL 0 /* invalidate all TLBs */
+#define FLUSH_TLB_LPID 1 /* invalidate TLBs for current LPID */
#ifndef __ASSEMBLY__
diff --git a/arch/powerpc/kernel/cpu_setup_power.S b/arch/powerpc/kernel/cpu_setup_power.S
index 4673353..9c9b741 100644
--- a/arch/powerpc/kernel/cpu_setup_power.S
+++ b/arch/powerpc/kernel/cpu_setup_power.S
@@ -137,15 +137,11 @@ __init_HFSCR:
/*
* Clear the TLB using the specified IS form of tlbiel instruction
* (invalidate by congruence class). P7 has 128 CCs., P8 has 512.
- *
- * r3 = IS field
*/
__init_tlb_power7:
- li r3,0xc00 /* IS field = 0b11 */
-_GLOBAL(__flush_tlb_power7)
li r6,128
mtctr r6
- mr r7,r3 /* IS field */
+ li r7,0xc00 /* IS field = 0b11 */
ptesync
2: tlbiel r7
addi r7,r7,0x1000
@@ -154,11 +150,9 @@ _GLOBAL(__flush_tlb_power7)
1: blr
__init_tlb_power8:
- li r3,0xc00 /* IS field = 0b11 */
-_GLOBAL(__flush_tlb_power8)
li r6,512
mtctr r6
- mr r7,r3 /* IS field */
+ li r7,0xc00 /* IS field = 0b11 */
ptesync
2: tlbiel r7
addi r7,r7,0x1000
diff --git a/arch/powerpc/kernel/cputable.c b/arch/powerpc/kernel/cputable.c
index 9b6dcaa..5cfffeb 100644
--- a/arch/powerpc/kernel/cputable.c
+++ b/arch/powerpc/kernel/cputable.c
@@ -71,8 +71,8 @@ extern void __restore_cpu_power7(void);
extern void __setup_cpu_power8(unsigned long offset, struct cpu_spec* spec);
extern void __restore_cpu_power8(void);
extern void __restore_cpu_a2(void);
-extern void __flush_tlb_power7(unsigned long inval_selector);
-extern void __flush_tlb_power8(unsigned long inval_selector);
+extern void __flush_tlb_power7(unsigned int action);
+extern void __flush_tlb_power8(unsigned int action);
extern long __machine_check_early_realmode_p7(struct pt_regs *regs);
extern long __machine_check_early_realmode_p8(struct pt_regs *regs);
#endif /* CONFIG_PPC64 */
diff --git a/arch/powerpc/kernel/mce_power.c b/arch/powerpc/kernel/mce_power.c
index aa9aff3..ffe22aa 100644
--- a/arch/powerpc/kernel/mce_power.c
+++ b/arch/powerpc/kernel/mce_power.c
@@ -28,6 +28,59 @@
#include <asm/mce.h>
#include <asm/machdep.h>
+static void _flush_tlb(uint32_t tlb_set, unsigned long inval_selector)
+{
+ unsigned long i, rb;
+
+ rb = inval_selector;
+ for (i = 0; i < tlb_set; i++) {
+ asm volatile("tlbiel %0" : : "r" (rb));
+ rb += 1 << TLBIEL_INVAL_SET_SHIFT;
+ }
+}
+
+/*
+ * Generic routine to flush TLB on power7. This routine is used as
+ * flush_tlb hook in cpu_spec for Power7 processor.
+ *
+ * action => FLUSH_TLB_ALL: Invalidate all TLBs.
+ * FLUSH_TLB_LPID: Invalidate TLB for current LPID.
+ */
+void __flush_tlb_power7(unsigned int action)
+{
+ switch (action) {
+ case FLUSH_TLB_ALL:
+ _flush_tlb(POWER7_TLB_SETS, TLBIEL_INVAL_SET);
+ break;
+ case FLUSH_TLB_LPID:
+ _flush_tlb(POWER7_TLB_SETS, TLBIEL_INVAL_SET_LPID);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Generic routine to flush TLB on power8. This routine is used as
+ * flush_tlb hook in cpu_spec for power8 processor.
+ *
+ * action => FLUSH_TLB_ALL: Invalidate all TLBs.
+ * FLUSH_TLB_LPID: Invalidate TLB for current LPID.
+ */
+void __flush_tlb_power8(unsigned int action)
+{
+ switch (action) {
+ case FLUSH_TLB_ALL:
+ _flush_tlb(POWER8_TLB_SETS, TLBIEL_INVAL_SET);
+ break;
+ case FLUSH_TLB_LPID:
+ _flush_tlb(POWER8_TLB_SETS, TLBIEL_INVAL_SET_LPID);
+ break;
+ default:
+ break;
+ }
+}
+
/* flush SLBs and reload */
static void flush_and_reload_slb(void)
{
@@ -79,7 +132,7 @@ static long mce_handle_derror(uint64_t dsisr, uint64_t slb_error_bits)
}
if (dsisr & P7_DSISR_MC_TLB_MULTIHIT_MFTLB) {
if (cur_cpu_spec && cur_cpu_spec->flush_tlb)
- cur_cpu_spec->flush_tlb(TLBIEL_INVAL_PAGE);
+ cur_cpu_spec->flush_tlb(FLUSH_TLB_ALL);
/* reset error bits */
dsisr &= ~P7_DSISR_MC_TLB_MULTIHIT_MFTLB;
}
@@ -110,7 +163,7 @@ static long mce_handle_common_ierror(uint64_t srr1)
break;
case P7_SRR1_MC_IFETCH_TLB_MULTIHIT:
if (cur_cpu_spec && cur_cpu_spec->flush_tlb) {
- cur_cpu_spec->flush_tlb(TLBIEL_INVAL_PAGE);
+ cur_cpu_spec->flush_tlb(FLUSH_TLB_ALL);
handled = 1;
}
break;
diff --git a/arch/powerpc/kvm/book3s_hv_ras.c b/arch/powerpc/kvm/book3s_hv_ras.c
index d562c8e..8246090 100644
--- a/arch/powerpc/kvm/book3s_hv_ras.c
+++ b/arch/powerpc/kvm/book3s_hv_ras.c
@@ -84,7 +84,7 @@ static long kvmppc_realmode_mc_power7(struct kvm_vcpu *vcpu)
}
if (dsisr & DSISR_MC_TLB_MULTI) {
if (cur_cpu_spec && cur_cpu_spec->flush_tlb)
- cur_cpu_spec->flush_tlb(TLBIEL_INVAL_SET_LPID);
+ cur_cpu_spec->flush_tlb(FLUSH_TLB_LPID);
dsisr &= ~DSISR_MC_TLB_MULTI;
}
/* Any other errors we don't understand? */
@@ -102,7 +102,7 @@ static long kvmppc_realmode_mc_power7(struct kvm_vcpu *vcpu)
break;
case SRR1_MC_IFETCH_TLBMULTI:
if (cur_cpu_spec && cur_cpu_spec->flush_tlb)
- cur_cpu_spec->flush_tlb(TLBIEL_INVAL_SET_LPID);
+ cur_cpu_spec->flush_tlb(FLUSH_TLB_LPID);
break;
default:
handled = 0;
^ permalink raw reply related
* Re: Pull request: scottwood/linux.git next
From: Bob Cochran @ 2014-09-23 3:52 UTC (permalink / raw)
To: Scott Wood, benh; +Cc: linuxppc-dev
In-Reply-To: <20140922222142.GA8525@home.buserror.net>
On 09/22/2014 06:21 PM, Scott Wood wrote:
> Highlights include DMA32 zone support (SATA, USB, etc now works on 64-bit
> FSL kernels), MSI changes, 8xx optimizations and cleanup, t104x board
> support, and PrPMC PCI enumeration.
>
> The following changes since commit 78eb9094ca08a40b8f9d3e113a2b88e0b7dbad1d:
>
> powerpc/t2080rdb: Add T2080RDB board support (2014-07-31 00:11:10 -0500)
>
> are available in the git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/scottwood/linux.git next
>
> for you to fetch changes up to cb0446c1b625326682ec4f9d1dd10779433646bc:
>
> Revert "powerpc/fsl_msi: spread msi ints across different MSIRs" (2014-09-19 15:20:42 -0500)
>
> ----------------------------------------------------------------
> Aaron Sierra (2):
> fsl_ifc: Fix csor_ext position in fsl_ifc_regs
> powerpc: fsl_pci: Add forced PCI Agent enumeration
>
> LEROY Christophe (7):
> powerpc/8xx: Declare SPRG2 as a SCRATCH register
> powerpc/8xx: Use SCRATCH0 and SCRATCH1 also for TLB handlers
> powerpc/8xx: Remove loading of r10 at end of FixupDAR
> powerpc/8xx: Fix comment about DIRTY update
> powerpc/8xx: No need to save r10 and r3 when not calling FixupDAR
> powerpc/8xx: Optimize verification in FixupDAR
> powerpc/8xx: Duplicate two insns instead of branching
>
> Nikhil Badola (3):
> powerpc: configs: Add VFAT file-system configs
> powerpc: dts: t4240: Change T4240 USB controller version
> powerpc: dts: t208x: Change T208x USB controller version
>
> Priyanka Jain (2):
> powerpc/fsl-booke: Add initial T1040/T1042 RDB board support
Hi Scott,
I would like to try your next branch with my t1040rdb along with DPAA
support / drivers. I don't see DPAA support in your repo. Can you
point me to a DPAA patch set somewhere, or am I on my own to merge what
I need from sdk or yocto?
Thanks,
Bob
> powerpc/fsl-booke: Add initial T1042RDB_PI board support
>
> Scott Wood (7):
> powerpc: Dynamic DMA zone limits
> powerpc/64: Honor swiotlb limit in coherent allocations
> powerpc/64: Limit ZONE_DMA32 to 4GiB in swiotlb_detect_4g()
> powerpc/fsl-pci: Limit ZONE_DMA32 to 2GiB on 64-bit platforms
> powerpc/85xx/defconfig: Remove duplicate CONFIG_RTC_DRV_DS1307
> powerpc/mm: Use common paging_init() for NUMA
> Revert "powerpc/fsl_msi: spread msi ints across different MSIRs"
>
> Tudor Laurentiu (6):
> powerpc/fsl_msi: support vmpic msi with mpic 4.3
> powerpc/fsl_msi: reorganize structs to improve clarity and flexibility
> powerpc/fsl_msi: change the irq handler from chained to normal
> powerpc/fsl_msi: show more meaningful names in /proc/interrupts
> powerpc/fsl_msi: spread msi ints across different MSIRs
> powerpc/fsl-booke64: add missing virtualization options in defconfig
>
> Documentation/devicetree/bindings/pci/fsl,pci.txt | 27 ++++
> arch/powerpc/Kconfig | 4 +
> arch/powerpc/boot/dts/fsl/t2081si-post.dtsi | 4 +-
> arch/powerpc/boot/dts/fsl/t4240si-post.dtsi | 4 +-
> arch/powerpc/boot/dts/t1040rdb.dts | 48 +++++++
> arch/powerpc/boot/dts/t1042rdb.dts | 48 +++++++
> arch/powerpc/boot/dts/t1042rdb_pi.dts | 57 ++++++++
> arch/powerpc/boot/dts/t104xrdb.dtsi | 156 ++++++++++++++++++++++
> arch/powerpc/configs/corenet32_smp_defconfig | 2 +
> arch/powerpc/configs/corenet64_smp_defconfig | 46 ++-----
> arch/powerpc/configs/mpc85xx_defconfig | 4 +-
> arch/powerpc/configs/mpc85xx_smp_defconfig | 4 +-
> arch/powerpc/configs/mpc86xx_defconfig | 3 +
> arch/powerpc/include/asm/pgtable.h | 3 +
> arch/powerpc/include/asm/reg.h | 3 +-
> arch/powerpc/kernel/dma-swiotlb.c | 8 +-
> arch/powerpc/kernel/dma.c | 33 +++++
> arch/powerpc/kernel/head_8xx.S | 150 ++++++++-------------
> arch/powerpc/mm/mem.c | 68 +++++++++-
> arch/powerpc/mm/numa.c | 8 --
> arch/powerpc/platforms/85xx/Kconfig | 2 +-
> arch/powerpc/platforms/85xx/corenet_generic.c | 14 ++
> arch/powerpc/platforms/85xx/qemu_e500.c | 10 ++
> arch/powerpc/sysdev/fsl_msi.c | 95 ++++++-------
> arch/powerpc/sysdev/fsl_msi.h | 4 +-
> arch/powerpc/sysdev/fsl_pci.c | 3 +-
> include/linux/fsl_ifc.h | 6 +-
> 27 files changed, 610 insertions(+), 204 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/pci/fsl,pci.txt
> create mode 100644 arch/powerpc/boot/dts/t1040rdb.dts
> create mode 100644 arch/powerpc/boot/dts/t1042rdb.dts
> create mode 100644 arch/powerpc/boot/dts/t1042rdb_pi.dts
> create mode 100644 arch/powerpc/boot/dts/t104xrdb.dtsi
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
>
^ permalink raw reply
* [PATCH v3 7/9] MSI/powerpc: Use __read_msi_msg() instead of read_msi_msg()
From: Yijing Wang @ 2014-09-23 5:27 UTC (permalink / raw)
To: Bjorn Helgaas; +Cc: Yijing Wang, linux-pci, linuxppc-dev
In-Reply-To: <1411450050-1510-1-git-send-email-wangyijing@huawei.com>
Read_msi_msg() only be called in rtas_setup_msi_irqs(),
use __read_msi_msg() instead of read_msi_msg for
simplification.
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Acked-by: Michael Ellerman <mpe@ellerman.id.au>
CC: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: linuxppc-dev@lists.ozlabs.org
---
arch/powerpc/platforms/pseries/msi.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
index 18ff462..dae71df 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -485,7 +485,7 @@ again:
irq_set_msi_desc(virq, entry);
/* Read config space back so we can restore after reset */
- read_msi_msg(virq, &msg);
+ __read_msi_msg(entry, &msg);
entry->msg = msg;
}
--
1.7.1
^ permalink raw reply related
* [PATCH] powerpc: Fix do_page_fault to check for HWPOISON flag.
From: Henish Patel @ 2014-09-23 5:42 UTC (permalink / raw)
To: michael, linuxppc-dev, paulus, benh
The current implementation of do_page_fault does not check whether the
page being accessed is marked hwpiosoned or not. Hence when an
application tries to access page that is marked hwpoisoned, it results
into Linux hypervisor crash and system goes into IPLing state.
This patch fixes this issue by adding a check for HWPOISON flag and send
SIGBUS to an application that is trying to access hwpoisoned page.
Signed-off-by: Henish Patel <hpatel@linux.vnet.ibm.com>
---
arch/powerpc/mm/fault.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index 51ab9e7..5e9b4fd 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -170,10 +170,10 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
return MM_FAULT_RETURN;
}
- /* Bus error. x86 handles HWPOISON here, we'll add this if/when
- * we support the feature in HW
+ /* Send SIGBUS to the application when it tries to access a page,
+ * which has been marked as HWPOISON.
*/
- if (fault & VM_FAULT_SIGBUS)
+ if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON))
return do_sigbus(regs, addr);
/* We don't understand the fault code, this is fatal */
^ permalink raw reply related
* Re: Pull request: scottwood/linux.git next
From: Scott Wood @ 2014-09-23 5:45 UTC (permalink / raw)
To: Bob Cochran; +Cc: linuxppc-dev
In-Reply-To: <5420EE93.7080006@mindchasers.com>
On Mon, 2014-09-22 at 23:52 -0400, Bob Cochran wrote:
> On 09/22/2014 06:21 PM, Scott Wood wrote:
> > Highlights include DMA32 zone support (SATA, USB, etc now works on 64-bit
> > FSL kernels), MSI changes, 8xx optimizations and cleanup, t104x board
> > support, and PrPMC PCI enumeration.
> >
> > The following changes since commit 78eb9094ca08a40b8f9d3e113a2b88e0b7dbad1d:
> >
> > powerpc/t2080rdb: Add T2080RDB board support (2014-07-31 00:11:10 -0500)
> >
> > are available in the git repository at:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/scottwood/linux.git next
> >
> > for you to fetch changes up to cb0446c1b625326682ec4f9d1dd10779433646bc:
> >
> > Revert "powerpc/fsl_msi: spread msi ints across different MSIRs" (2014-09-19 15:20:42 -0500)
> >
> > ----------------------------------------------------------------
> > Aaron Sierra (2):
> > fsl_ifc: Fix csor_ext position in fsl_ifc_regs
> > powerpc: fsl_pci: Add forced PCI Agent enumeration
> >
> > LEROY Christophe (7):
> > powerpc/8xx: Declare SPRG2 as a SCRATCH register
> > powerpc/8xx: Use SCRATCH0 and SCRATCH1 also for TLB handlers
> > powerpc/8xx: Remove loading of r10 at end of FixupDAR
> > powerpc/8xx: Fix comment about DIRTY update
> > powerpc/8xx: No need to save r10 and r3 when not calling FixupDAR
> > powerpc/8xx: Optimize verification in FixupDAR
> > powerpc/8xx: Duplicate two insns instead of branching
> >
> > Nikhil Badola (3):
> > powerpc: configs: Add VFAT file-system configs
> > powerpc: dts: t4240: Change T4240 USB controller version
> > powerpc: dts: t208x: Change T208x USB controller version
> >
> > Priyanka Jain (2):
> > powerpc/fsl-booke: Add initial T1040/T1042 RDB board support
>
>
> Hi Scott,
>
> I would like to try your next branch with my t1040rdb along with DPAA
> support / drivers. I don't see DPAA support in your repo. Can you
> point me to a DPAA patch set somewhere, or am I on my own to merge what
> I need from sdk or yocto?
Unfortunately DPAA support has not yet been upstreamed, though it is
being worked on. For now if you need DPAA with the latest kernel,
you'll need to merge it yourself based on the SDK code. It also
wouldn't hurt to let any sales/support contact you may have at
Freescale know that this is important to you.
-Scott
^ permalink raw reply
* Re: [PATCH v5 1/1] powerpc/perf: Adjust callchain based on DWARF debug info
From: Jiri Olsa @ 2014-09-23 6:59 UTC (permalink / raw)
To: Aaro Koskinen
Cc: Michael Ellerman, ulrich.weigand, Anton Blanchard, linux-kernel,
Arnaldo Carvalho de Melo, linuxppc-dev, Maynard Johnson,
Sukadev Bhattiprolu
In-Reply-To: <20140922213345.GB666@drone.musicnaut.iki.fi>
On Tue, Sep 23, 2014 at 12:33:45AM +0300, Aaro Koskinen wrote:
> Hi,
>
> On Wed, Jun 25, 2014 at 08:49:03AM -0700, Sukadev Bhattiprolu wrote:
> > powerpc/perf: Adjust callchain based on DWARF debug info
> >
> > When saving the callchain on Power, the kernel conservatively saves excess
> > entries in the callchain. A few of these entries are needed in some cases
> > but not others. We should use the DWARF debug information to determine
> > when the entries are needed.
>
> This patch breaks perf compilation if DWARF support is not present.
> DWARF support is auto-detected early in the build, so IMHO either user
> should be informed to install the needed stuff, or the build should
> succeed with limited functionality.
>
> arch/powerpc/util/skip-callchain-idx.c:13:19: fatal error: dwarf.h: No such file or directory #include <dwarf.h>
> ^
> compilation terminated.
looks like powerpc issue.. hows 'make -f tests/make' doing
on powerpc? It tests above scenario.
jirka
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox