* Re: [PATCH 3/3 v16] iommu/fsl: Freescale PAMU driver and iommu implementation.
From: Alex Williamson @ 2013-06-26 19:14 UTC (permalink / raw)
To: Sethi Varun-B16395
Cc: Wood Scott-B07421, Timur Tabi, joro@8bytes.org,
linux-kernel@vger.kernel.org, Yoder Stuart-B08248,
iommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org
In-Reply-To: <C5ECD7A89D1DC44195F34B25E172658D5DD70C@039-SN2MPN1-011.039d.mgd.msft.net>
On Wed, 2013-06-26 at 06:24 +0000, Sethi Varun-B16395 wrote:
>
> > -----Original Message-----
> > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > Sent: Tuesday, June 25, 2013 10:27 AM
> > To: Sethi Varun-B16395
> > Cc: joro@8bytes.org; iommu@lists.linux-foundation.org; linuxppc-
> > dev@lists.ozlabs.org; linux-kernel@vger.kernel.org;
> > benh@kernel.crashing.org; galak@kernel.crashing.org; Yoder Stuart-B08248;
> > Wood Scott-B07421; Timur Tabi
> > Subject: Re: [PATCH 3/3 v16] iommu/fsl: Freescale PAMU driver and iommu
> > implementation.
> >
> > On Thu, 2013-06-20 at 21:31 +0530, Varun Sethi wrote:
> >
> > > +#define REQ_ACS_FLAGS (PCI_ACS_SV | PCI_ACS_RR | PCI_ACS_CR |
> > PCI_ACS_UF)
> > > +
> > > +static struct iommu_group *get_device_iommu_group(struct device *dev)
> > > +{
> > > + struct iommu_group *group;
> > > +
> > > + group = iommu_group_get(dev);
> > > + if (!group)
> > > + group = iommu_group_alloc();
> > > +
> > > + return group;
> > > +}
> > > +
> > [snip]
> > > +
> >
> > This really gets parent or peer, right?
> >
> > > +static struct iommu_group *get_peer_pci_device_group(struct pci_dev
> > > +*pdev) {
> > > + struct iommu_group *group = NULL;
> > > +
> > > + /* check if this is the first device on the bus*/
> > > + if (pdev->bus_list.next == pdev->bus_list.prev) {
> >
> > It's a list_head, use list functions. The list implementation should be
> > treated as opaque.
> >
> > if (list_is_singular(&pdev->bus_list))
> >
> > > + struct pci_bus *bus = pdev->bus->parent;
> > > + /* Traverese the parent bus list to get
> > > + * pdev & dev for the sibling device.
> > > + */
> > > + while (bus) {
> > > + if (!list_empty(&bus->devices)) {
> > > + pdev = container_of(bus->devices.next,
> > > + struct pci_dev, bus_list);
> >
> > pdev = list_first_entry(&bus->devices, struct pci_dev, bus_list);
> >
> > > + group = iommu_group_get(&pdev->dev);
> > > + break;
> > > + } else
> > > + bus = bus->parent;
> >
> > Is this ever reached? Don't you always have bus->self?
> >
> [Sethi Varun-B16395] Not sure I understand. Trying to get the group
> information from the parent bus, if there are no sibling devices on
> the current bus.
I assume there's always a bridge on a bus, but maybe that bridge
(parent->self) is not in the list of parent->devices? Is that the case?
If not, then there's always a device on the bus, the bridge that created
it.
> > > + }
> > > + } else {
> > > + /*
> > > + * Get the pdev & dev for the sibling device
> > > + */
> > > + pdev = container_of(pdev->bus_list.prev,
> > > + struct pci_dev, bus_list);
> >
> > How do you know if you're at the head or tail of the list?
> >
> > struct pci_dev *tmp;
> > list_for_each_entry(tmp, &pdev->bus_list, bus_list) {
> > if (tmp == pdev)
> > continue;
> >
> > group = iommu_group_get(&tmp->dev);
> > break;
> > }
> >
> > > + group = iommu_group_get(&pdev->dev);
> > > + }
> > > +
> > > + return group;
> > > +}
> > > +
> > > +static struct iommu_group *get_pci_device_group(struct pci_dev *pdev)
> > > +{
> > > + struct iommu_group *group = NULL;
> > > + struct pci_dev *bridge, *dma_pdev = NULL;
> > > + struct pci_controller *pci_ctl;
> > > + bool pci_endpt_partioning;
> > > +
> > > + pci_ctl = pci_bus_to_host(pdev->bus);
> > > + pci_endpt_partioning = check_pci_ctl_endpt_part(pci_ctl);
> > > + /* We can partition PCIe devices so assign device group to the
> > device */
> > > + if (pci_endpt_partioning) {
> > > + bridge = pci_find_upstream_pcie_bridge(pdev);
> > > + if (bridge) {
> > > + if (pci_is_pcie(bridge))
> > > + dma_pdev = pci_get_domain_bus_and_slot(
> > > + pci_domain_nr(pdev->bus),
> > > + bridge->subordinate->number, 0);
> > > + if (!dma_pdev)
> > > + dma_pdev = pci_dev_get(bridge);
> > > + } else
> > > + dma_pdev = pci_dev_get(pdev);
> > > +
> > > + /* Account for quirked devices */
> > > + swap_pci_ref(&dma_pdev, pci_get_dma_source(dma_pdev));
> > > +
> > > + /*
> > > + * If it's a multifunction device that does not support our
> > > + * required ACS flags, add to the same group as function 0.
> > > + */
> >
> > See c14d2690 in Joerg's next tree, using function 0 was a poor
> > assumption.
> [Sethi Varun-B16395] ok.
>
> >
> > > + if (dma_pdev->multifunction &&
> > > + !pci_acs_enabled(dma_pdev, REQ_ACS_FLAGS))
> > > + swap_pci_ref(&dma_pdev,
> > > + pci_get_slot(dma_pdev->bus,
> > > + PCI_DEVFN(PCI_SLOT(dma_pdev-
> > >devfn),
> > > + 0)));
> > > +
> > > + group = get_device_iommu_group(&pdev->dev);
> > > + pci_dev_put(pdev);
> >
> > What was the point of all the above if we use pdev here instead of
> > dma_pdev? Wrong device and broken reference counting.
> [Sethi Varun-B16395] Will fix this
>
> This also isn't
> > testing ACS all the way up to the root complex or controller.
> [Sethi Varun-B16395] In our case the IOMMU can differentiate
> transactions based on the LIODN. The PCIe controller can generate a
> unique LIODN based on the bus,device,function number. I believe this
> would even be true for devices connected to a PCIe bridge (and not on
> the root bus). So, do we still need to check for ACS up to the root
> node?
ACS is the PCI capability that tells us whether a device allows
peer-to-peer or requires all DMA to be forwarded upstream. At any step
along the way from an endpoint to the IOMMU the transaction can be
re-routed. We therefore need to test the ACS property not only of the
endpoint, but every device between the endpoint and the IOMMU to form
the IOMMU groups. An IOMMU cannot isolate a device if a DMA access is
re-routed by a link in the topology that prevents the IOMMU from even
being part of the transaction. Thanks,
Alex
^ permalink raw reply
* Re: [PATCH v2 15/45] rcu: Use get/put_online_cpus_atomic() to prevent CPU offline
From: Tejun Heo @ 2013-06-26 21:34 UTC (permalink / raw)
To: Srivatsa S. Bhat
Cc: peterz, fweisbec, linux-kernel, walken, mingo, linux-arch,
vincent.guittot, xiaoguangrong, wangyun, paulmck, nikunj,
linux-pm, rusty, Steven Rostedt, namhyung, tglx, laijs, zhong,
netdev, oleg, sbw, David Laight, akpm, linuxppc-dev
In-Reply-To: <51CB32E0.3040106@linux.vnet.ibm.com>
Hey,
On Wed, Jun 26, 2013 at 11:58:48PM +0530, Srivatsa S. Bhat wrote:
> Yes, we were discussing hot-unplug latency for use-cases such as
> suspend/resume. We didn't want to make those operations slower in the
> process of removing stop_machine() from hotplug.
Can you please explain why tho? How much would it help in terms of
power-saving? Or are there other issues in taking longer to shut down
cpus?
Thanks.
--
tejun
^ permalink raw reply
* Re: [RFC] [PATCH] powerpc: Add MSR_DE to MSR_KERNEL
From: Scott Wood @ 2013-06-26 21:42 UTC (permalink / raw)
To: Joakim Tjernlund
Cc: Wood Scott-B07421, Zang Roy-R61911, Bob Cochran,
linuxppc-dev@ozlabs.org, Dan Malek, Support
In-Reply-To: <OF239E7E70.94A77B69-ONC1257B95.0020B40D-C1257B95.0020FEEE@transmode.se>
On 06/25/2013 01:00:23 AM, Joakim Tjernlund wrote:
> Scott Wood <scottwood@freescale.com> wrote on 2013/06/25 02:51:00:
> >
> > On Fri, Jul 20, 2012 at 10:37:17AM +0200, Joakim Tjernlund wrote:
> > > Zang Roy-R61911 <r61911@freescale.com> wrote on 2012/07/20 =20
> 10:27:52:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From:
> linuxppc-dev-bounces+tie-fei.zang=3Dfreescale.com@lists.ozlabs.org
> > > > > [
> mailto:linuxppc-dev-bounces+tie-fei.zang=3Dfreescale.com@lists.ozlabs.org=
]
> > > > > On Behalf Of Joakim Tjernlund
> > > > > Sent: Friday, June 01, 2012 6:36 AM
> > > > > To: Wood Scott-B07421
> > > > > Cc: linuxppc-dev@ozlabs.org; Dan Malek; Bob Cochran; Support
> > > > > Subject: Re: [RFC] [PATCH] powerpc: Add MSR_DE to MSR_KERNEL
> > > > >
> > > > > It just occurred to me that you guys have this already in your
> Linux SDK so
> > > > > it can't be that bad.
> > > > No. MSR_DE is ONLY added when using CW debug in SDK.
> > > > Roy
> > > >
> > >
> > > Yes, and I later found that user space debugging is busted if you =20
> turn
> on MSR_DE in
> > > kernel.
> >
> > So, how should we handle the CONFIG_BDI_SWITCH patch? It seems =20
> like it
> > should at least have a warning in the kconfig help text that it =20
> breaks
> > userspace debugging (to the point of causing a kernel oops if it's
> > tried). Or maybe it can deselect CONFIG_PPC_ADV_DEBUG_REGS?
> >
> > It'd also be nice to keep things like this, that are a consequence =20
> of
> how
> > external debug works on e500, separate from the Abatron-specific =20
> stuff.
> >
>=20
> I was hoping the kernel would grow per context handling of MSR_DE. =20
> Then
> one could have
> MSR_DE on in MSR_KERNEL but off in user space(unless gdb request it =20
> on a
> per process basis).
What if the external debugger wants debugging to work when in userspace =20
as well? Plus, you wouldn't be able to debug from the very beginning =20
of an exception handler if the exception came from userspace.
-Scott=
^ permalink raw reply
* Re: linux-next: manual merge of the akpm tree with the powerpc tree
From: Benjamin Herrenschmidt @ 2013-06-26 21:56 UTC (permalink / raw)
To: Oleg Nesterov
Cc: Stephen Rothwell, Michael Neuling, linux-kernel, linux-next,
Andrew Morton, linuxppc-dev
In-Reply-To: <20130626141916.GA30716@redhat.com>
On Wed, 2013-06-26 at 16:19 +0200, Oleg Nesterov wrote:
>
> You were cc'ed every time ;)
>
> > Why didn't it go through the powerpc tree ?
>
> Because this series needs to update any user of
> ptrace_get/put_breakpoints
> in arch/ (simply remove these calls), then change the core kernel
> code, then
> fix arch/86.
That's fine. I didn't realize that was a series, thought it was a
standalone patch I had missed.
Ben.
^ permalink raw reply
* Re: [1/1] Add a new platform tree for ib8315. Also add the DTS and the defconfig for that board.
From: Scott Wood @ 2013-06-27 0:12 UTC (permalink / raw)
To: Sergey Gerasimov; +Cc: linuxppc-dev, Paul Mackerras, linux-kernel
In-Reply-To: <1363614452-1109-1-git-send-email-Sergey.Gerasimov@astrosoft-development.com>
Please keep subject lines limited to 60-70 characters, and prefix with
"powerpc/83xx:".
On Mon, Mar 18, 2013 at 05:47:32PM +0400, Sergey Gerasimov wrote:
> Signed-off-by: Sergey Gerasimov <Sergey.Gerasimov@astrosoft-development.com>
>
> ---
> arch/powerpc/boot/dts/ib8315.dts | 490 +++++++
> arch/powerpc/configs/83xx/ib8315_defconfig | 2121 ++++++++++++++++++++++++++++
> arch/powerpc/platforms/83xx/Kconfig | 7 +
> arch/powerpc/platforms/83xx/Makefile | 1 +
> arch/powerpc/platforms/83xx/tqm8315.c | 137 ++
> 5 files changed, 2756 insertions(+)
> create mode 100644 arch/powerpc/boot/dts/ib8315.dts
> create mode 100644 arch/powerpc/configs/83xx/ib8315_defconfig
> create mode 100644 arch/powerpc/platforms/83xx/tqm8315.c
As Kumar asked, why is the device tree "ib8315" but the platform file
"tqm8315"?
Why does this board need its own defconfig, versus being added to
mpc83xx_defconfig? Also make sure that you run savedefconfig when you
add or modify a defconfig, to trim it down to options that deviate from
the default.
> diff --git a/arch/powerpc/boot/dts/ib8315.dts b/arch/powerpc/boot/dts/ib8315.dts
> new file mode 100644
> index 0000000..963caf2
> --- /dev/null
> +++ b/arch/powerpc/boot/dts/ib8315.dts
> @@ -0,0 +1,490 @@
> +/*
> + * IB8315 Device Tree Source based on:
> + * TQM8315 Device Tree Source
> + *
> + * Copyright 2009 TQ Components
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License as published by the
> + * Free Software Foundation; either version 2 of the License, or (at your
> + * option) any later version.
> + */
> +
> +/dts-v1/;
> +
> +/ {
> + compatible = "fsl,tqm8315";
And here's tqm8315 again. Just because you based this on tqm8315 doesn't
mean you can call it a tqm8315.
> + partition@2 {
> + label = "env2";
> + reg = <0xA0000 0x40000>; // 384 KiB
> + read-only;
> + };
Comment says 384 KiB. Code says 256 KiB.
> + partition@3 {
> + label ="dtb";
> + reg = <0x100000 0x100000>; // 1 MiB
> + };
Space after =
> + partition@5 {
> + label ="root";
> + reg = <0x500000 0x2800000>; // 40 MiB
> + };
> +
> + /*
> + * The remaining 19 MiB, e.g. for a file system.
> + * Requires MTD concatenation support
> + */
> + partition@6 {
> + label ="user";
> + reg = <0x2D00000 0x1300000>;
> + };
Nothing between 0x2800000 and 0x2d00000?
> + nand@1,0 {
> + #address-cells = <1>;
> + #size-cells = <1>;
> + compatible = "fsl,mpc8315-fcm-nand",
> + "fsl,elbc-fcm-nand";
> + reg = <0x1 0x0 0x8000>;
> +
> + partition@0 {
> + label = "filesystem";
> + reg = <0x0 0x20000000>;
> + /*read-only;*/
> + };
> + };
No need to define a partition at all if you're giving it all to one use.
> + immr@e0000000 {
> + #address-cells = <1>;
> + #size-cells = <1>;
> + device_type = "soc";
> + compatible = "fsl,mpc8315-immr", "simple-bus";
> + ranges = <0 0xe0000000 0x00100000>;
> + reg = <0xe0000000 0x00000200>;
> + bus-frequency = <0>; // from bootloader
Please consider factoring this stuff out into an mpc8315 include file.
> + /* Enable this to support sensors on STK85xxNG */
> + /*sensor@49 {
> + compatible = "national,lm75";
> + reg = <0x49>;
> + };
> + sensor@4A {
> + compatible = "national,lm75";
> + reg = <0x4A>;
> + };
> + sensor@4B {
> + compatible = "national,lm75";
> + reg = <0x4B>;
> + };*/
Does the board have them or not? The device tree describes the hardware,
not what you want to do with it.
> + pci0: pci@e0008500 {
> + interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
> + /* The values are calculated the following:
> + * The first thre values are address values as address-cells is 3
> + * The first value is the bus number in bits 31-16 (normal 0 because masked out)
> + * Bits 15-8 are the devfn (this is the IDSEL Line shift 3 bits to the left IDSEL AD15 is 0x78)
> + * Bis 7-0 are unused set to 0 and mask out with the interrupt-mask value
> + * The following two values should be 0 and masked out
> + * The fourth value is the interrupt bis from PCI configuration header (one value as interrupt-cells is 1)
> + * The last three values are the interrupt this interrupt is connected to.
> + * First interrupt controller node, then the number and last the flags (8 means level low) */
/*
* Linux multi-line
* comment style is
* like this
*/
Also keep line lengths under 80 columns.
"interrupt bis"?
> +# CONFIG_DEBUG_BUGVERBOSE is not set
Please reconsider this one. It makes BUG dumps much more readable, and
doesn't cost much.
> +#define MPC8315_SATA_PHYCTRL_REG_OFFSET 0x15C
> +#define PHYCTRLCFG_REFCLK_MASK 0x00000070
> +#define PHYCTRLCFG_REFCLK_50MHZ 0x00000050
> +#define PHYCTRLCFG_REFCLK_75MHZ 0x00000000
> +#define PHYCTRLCFG_REFCLK_100MHZ 0x00000060
> +#define PHYCTRLCFG_REFCLK_125MHZ 0x00000070
> +#define PHYCTRLCFG_REFCLK_150MHZ 0x00000020
> +
> +
> +#ifdef CONFIG_SATA_FSL
> +void init_mpc8315_sata_phy(void)
> +{
> + u32 val32;
> + void __iomem *immap;
> +
> + immap = ioremap(get_immrbase() + 0x18000, 0x1000);
> + if (immap == NULL)
> + return;
> +
> + /* Configure PHY for 125 MHz reference clock */
> + val32 = ioread32(immap + MPC8315_SATA_PHYCTRL_REG_OFFSET);
> + val32 &= ~PHYCTRLCFG_REFCLK_MASK;
> + val32 |= PHYCTRLCFG_REFCLK_125MHZ;
> + iowrite32(val32, immap + MPC8315_SATA_PHYCTRL_REG_OFFSET);
> +
> + iounmap(immap);
> +}
> +#endif
Please use the device tree to find the regs you want, rather than
get_immrbase(). If the regs aren't already in the device tree, add them.
In this case it's the SATA controller. Is this the right way to handle
this, or should platform code (or better, the device tree) be passing
information to the SATA driver about what PHY frequency to use? Would
this configuration survive if the SATA driver were to reset the SATA
block?
> +/*
> + * Setup the architecture
> + */
> +static void __init tqm8315_setup_arch(void)
> +{
> +#ifdef CONFIG_PCI
> + struct device_node *np;
> +#endif
> +
> + if (ppc_md.progress)
> + ppc_md.progress("tqm8315_setup_arch()", 0);
> +
> +#ifdef CONFIG_PCI
> + for_each_compatible_node(np, "pci", "fsl,mpc8349-pci")
> + mpc83xx_add_bridge(np);
> + for_each_compatible_node(np, "pci", "fsl,mpc8314-pcie")
> + mpc83xx_add_bridge(np);
> +#endif
Call mpc83xx_setup_pci().
> +static void __init tqm8315_init_IRQ(void)
> +{
> + struct device_node *np;
> +
> + np = of_find_node_by_type(NULL, "ipic");
> + if (!np)
> + return;
> +
> + ipic_init(np, 0);
> +
> + /* Initialize the default interrupt mapping priorities,
> + * in case the boot rom changed something on us.
> + */
> + ipic_set_default_priority();
> +}
Call mpc83xx_ipic_init_IRQ().
> +static struct of_device_id __initdata of_bus_ids[] = {
> + {.compatible = "simple-bus"},
> + {.compatible = "gianfar"},
> + {},
> +};
> +
> +static int __init declare_of_platform_devices(void)
> +{
> + of_platform_bus_probe(NULL, of_bus_ids, NULL);
> + return 0;
> +}
> +machine_device_initcall(tqm8315, declare_of_platform_devices);
Call mpc83xx_declare_platform_devices().
-Scott
^ permalink raw reply
* [PATCH] powerpc: don't flush/invalidate the d/icache for an unknown relocation type
From: Kevin Hao @ 2013-06-27 1:09 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: linuxppc, Suzuki Poulose
For an unknown relocation type since the value of r4 is just the 8bit
relocation type, the sum of r4 and r7 may yield an invalid memory
address. For example:
In normal case:
r4 = c00xxxxx
r7 = 40000000
r4 + r7 = 000xxxxx
For an unknown relocation type:
r4 = 000000xx
r7 = 40000000
r4 + r7 = 400000xx
400000xx is an invalid memory address for a board which has just
512M memory.
And for operations such as dcbst or icbi may cause bus error for an
invalid memory address on some platforms and then cause the board
reset. So we should skip the flush/invalidate the d/icache for
an unknown relocation type.
Signed-off-by: Kevin Hao <haokexin@gmail.com>
---
arch/powerpc/kernel/reloc_32.S | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/reloc_32.S b/arch/powerpc/kernel/reloc_32.S
index ef46ba6..f366fed 100644
--- a/arch/powerpc/kernel/reloc_32.S
+++ b/arch/powerpc/kernel/reloc_32.S
@@ -166,7 +166,7 @@ ha16:
/* R_PPC_ADDR16_LO */
lo16:
cmpwi r4, R_PPC_ADDR16_LO
- bne nxtrela
+ bne unknown_type
lwz r4, 0(r9) /* r_offset */
lwz r0, 8(r9) /* r_addend */
add r0, r0, r3
@@ -191,6 +191,7 @@ nxtrela:
dcbst r4,r7
sync /* Ensure the data is flushed before icbi */
icbi r4,r7
+unknown_type:
cmpwi r8, 0 /* relasz = 0 ? */
ble done
add r9, r9, r6 /* move to next entry in the .rela table */
--
1.8.1.4
^ permalink raw reply related
* [PATCH 1/2] powerpc: enable the relocatable support for the fsl booke 32bit kernel
From: Kevin Hao @ 2013-06-27 2:00 UTC (permalink / raw)
To: Kumar Gala; +Cc: Scott Wood, linuxppc
In-Reply-To: <1372298434-20220-1-git-send-email-haokexin@gmail.com>
This is based on the codes in the head_44x.S. Since we always align to
256M before mapping the PAGE_OFFSET for a relocatable kernel, we also
change the init tlb map to 256M size.
Signed-off-by: Kevin Hao <haokexin@gmail.com>
---
arch/powerpc/Kconfig | 2 +-
arch/powerpc/kernel/fsl_booke_entry_mapping.S | 8 ++-
arch/powerpc/kernel/head_fsl_booke.S | 92 +++++++++++++++++++++++++--
3 files changed, 94 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c33e3ad..9eb97ac 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -866,7 +866,7 @@ config DYNAMIC_MEMSTART
config RELOCATABLE
bool "Build a relocatable kernel"
- depends on ADVANCED_OPTIONS && FLATMEM && 44x
+ depends on ADVANCED_OPTIONS && FLATMEM && (44x || FSL_BOOKE)
select NONSTATIC_KERNEL
help
This builds a kernel image that is capable of running at the
diff --git a/arch/powerpc/kernel/fsl_booke_entry_mapping.S b/arch/powerpc/kernel/fsl_booke_entry_mapping.S
index a92c79b..32a4b38 100644
--- a/arch/powerpc/kernel/fsl_booke_entry_mapping.S
+++ b/arch/powerpc/kernel/fsl_booke_entry_mapping.S
@@ -165,10 +165,10 @@ skpinv: addi r6,r6,1 /* Increment */
lis r6,0x1000 /* Set MAS0(TLBSEL) = TLB1(1), ESEL = 0 */
mtspr SPRN_MAS0,r6
lis r6,(MAS1_VALID|MAS1_IPROT)@h
- ori r6,r6,(MAS1_TSIZE(BOOK3E_PAGESZ_64M))@l
+ ori r6,r6,(MAS1_TSIZE(BOOK3E_PAGESZ_256M))@l
mtspr SPRN_MAS1,r6
- lis r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_64M, M_IF_SMP)@h
- ori r6,r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_64M, M_IF_SMP)@l
+ lis r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_256M, M_IF_SMP)@h
+ ori r6,r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_256M, M_IF_SMP)@l
mtspr SPRN_MAS2,r6
mtspr SPRN_MAS3,r8
tlbwe
@@ -176,6 +176,8 @@ skpinv: addi r6,r6,1 /* Increment */
/* 7. Jump to KERNELBASE mapping */
lis r6,(KERNELBASE & ~0xfff)@h
ori r6,r6,(KERNELBASE & ~0xfff)@l
+ rlwinm r7,r25,0,4,31
+ add r6,r7,r6
#elif defined(ENTRY_MAPPING_KEXEC_SETUP)
/*
diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S
index d10a7ca..c3b4c8e53 100644
--- a/arch/powerpc/kernel/head_fsl_booke.S
+++ b/arch/powerpc/kernel/head_fsl_booke.S
@@ -83,10 +83,43 @@ _ENTRY(_start);
andc r31,r20,r18 /* r31 = page base */
or r31,r31,r19 /* r31 = devtree phys addr */
mfspr r30,SPRN_MAS7
-
- li r25,0 /* phys kernel start (low) */
li r24,0 /* CPU number */
- li r23,0 /* phys kernel start (high) */
+
+#ifdef CONFIG_RELOCATABLE
+ bl 0f /* Get our runtime address */
+0: mflr r3 /* Make it accessible */
+ addis r3,r3,(_stext - 0b)@ha
+ addi r3,r3,(_stext - 0b)@l /* Get our current runtime base */
+
+ /* Translate _stext address to physical, save in r23/r25 */
+ tlbsx 0,r3 /* must succeed */
+
+ mfspr r16,SPRN_MAS1
+ mfspr r20,SPRN_MAS3
+ rlwinm r17,r16,25,0x1f /* r17 = log2(page size) */
+ li r18,1024
+ slw r18,r18,r17 /* r18 = page size */
+ addi r18,r18,-1
+ and r19,r3,r18 /* r19 = page offset */
+ andc r25,r20,r18 /* r25 = page base */
+ or r25,r25,r19 /* r25 = _stext phys addr */
+ mfspr r23,SPRN_MAS7
+
+ /*
+ * We have the runtime (virutal) address of our base.
+ * We calculate our shift of offset from a 256M page.
+ * We could map the 256M page we belong to at PAGE_OFFSET and
+ * get going from there.
+ */
+ lis r4,KERNELBASE@h
+ ori r4,r4,KERNELBASE@l
+ rlwinm r6,r25,0,4,31 /* r6 = PHYS_START % 256M */
+ rlwinm r5,r4,0,4,31 /* r5 = KERNELBASE % 256M */
+ subf r3,r5,r6 /* r3 = r6 - r5 */
+ add r3,r4,r3 /* Required Virutal Address */
+
+ bl relocate
+#endif
/* We try to not make any assumptions about how the boot loader
* setup or used the TLBs. We invalidate all mappings from the
@@ -197,7 +230,58 @@ _ENTRY(__early_start)
bl early_init
-#ifdef CONFIG_DYNAMIC_MEMSTART
+#ifdef CONFIG_RELOCATABLE
+ /*
+ * Relocatable kernel support based on processing of dynamic
+ * relocation entries.
+ *
+ * r25/r23 will contain RPN/ERPN for the start address of memory
+ */
+ lis r3,kernstart_addr@ha
+ la r3,kernstart_addr@l(r3)
+
+#ifdef CONFIG_PHYS_64BIT
+ stw r23,0(r3)
+ stw r25,4(r3)
+#else
+ stw r25,0(r3)
+#endif
+
+ /*
+ * Compute the virt_phys_offset :
+ * virt_phys_offset = stext.run - kernstart_addr
+ *
+ * stext.run = (KERNELBASE & ~0xfffffff) + (kernstart_addr & 0xfffffff)
+ * When we relocate, we have :
+ *
+ * (kernstart_addr & 0xfffffff) = (stext.run & 0xfffffff)
+ *
+ * hence:
+ * virt_phys_offset = (KERNELBASE & ~0xfffffff) -
+ * (kernstart_addr & ~0xfffffff)
+ *
+ */
+
+ /* KERNELBASE&~0xfffffff => (r4,r5) */
+ li r4, 0 /* higer 32bit */
+ lis r5,KERNELBASE@h
+ rlwinm r5,r5,0,0,3 /* Align to 256M, lower 32bit */
+
+ rlwinm r7,r25,0,0,3
+ /*
+ * 64bit subtraction.
+ */
+ subfc r5,r7,r5
+ subfe r4,r23,r4
+
+ /* Store virt_phys_offset */
+ lis r3,virt_phys_offset@ha
+ la r3,virt_phys_offset@l(r3)
+
+ stw r4,0(r3)
+ stw r5,4(r3)
+
+#elif defined(CONFIG_DYNAMIC_MEMSTART)
lis r3,kernstart_addr@ha
la r3,kernstart_addr@l(r3)
#ifdef CONFIG_PHYS_64BIT
--
1.8.1.4
^ permalink raw reply related
* [PATCH 0/2] powerpc: enable the relocatable support for fsl booke 32bit kernel
From: Kevin Hao @ 2013-06-27 2:00 UTC (permalink / raw)
To: Kumar Gala; +Cc: Scott Wood, linuxppc
Currently the fsl booke 32bit kernel is using the DYNAMIC_MEMSTART relocation
method. But the RELOCATABLE method is more flexible and has less alignment
restriction. So enable this feature on this platform and use it by
default for the kdump kernel.
These patches have passed the kdump boot test on a p2020rdb board.
Kevin Hao (2):
powerpc: enable the relocatable support for the fsl booke 32bit kernel
powerpc/fsl_booke: enable the relocatable for the kdump kernel
arch/powerpc/Kconfig | 5 +-
arch/powerpc/include/asm/mmu-book3e.h | 5 ++
arch/powerpc/kernel/fsl_booke_entry_mapping.S | 8 ++-
arch/powerpc/kernel/head_fsl_booke.S | 92 +++++++++++++++++++++++++--
arch/powerpc/mm/fsl_booke_mmu.c | 30 ++++++++-
5 files changed, 127 insertions(+), 13 deletions(-)
--
1.8.1.4
^ permalink raw reply
* [PATCH 2/2] powerpc/fsl_booke: enable the relocatable for the kdump kernel
From: Kevin Hao @ 2013-06-27 2:00 UTC (permalink / raw)
To: Kumar Gala; +Cc: Scott Wood, linuxppc
In-Reply-To: <1372298434-20220-1-git-send-email-haokexin@gmail.com>
For a relocatable kdump kernel, we may create a tlb map which is
beyond the real memory allocated to the kdump kernel. For example,
when the boot kernel reserve 32M memory for the kdump kernel by
using 'crashkernel=32M@64M', we will have to create a 256M tlb
entry in the kdump kernel. So define PPC_PIN_SIZE for fsl ppc32 bit,
this will make sure that we still get the right VMALLOC_START in
this case.
Signed-off-by: Kevin Hao <haokexin@gmail.com>
---
arch/powerpc/Kconfig | 3 +--
arch/powerpc/include/asm/mmu-book3e.h | 5 +++++
arch/powerpc/mm/fsl_booke_mmu.c | 30 +++++++++++++++++++++++++++---
3 files changed, 33 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 9eb97ac..ca237f5 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -382,8 +382,7 @@ config KEXEC
config CRASH_DUMP
bool "Build a kdump crash kernel"
depends on PPC64 || 6xx || FSL_BOOKE || (44x && !SMP)
- select RELOCATABLE if PPC64 || 44x
- select DYNAMIC_MEMSTART if FSL_BOOKE
+ select RELOCATABLE if PPC64 || 44x || FSL_BOOKE
help
Build a kernel suitable for use as a kdump capture kernel.
The same kernel binary can be used as production kernel and dump
diff --git a/arch/powerpc/include/asm/mmu-book3e.h b/arch/powerpc/include/asm/mmu-book3e.h
index 936db36..bf422db 100644
--- a/arch/powerpc/include/asm/mmu-book3e.h
+++ b/arch/powerpc/include/asm/mmu-book3e.h
@@ -214,6 +214,11 @@
#define TLBILX_T_CLASS2 6
#define TLBILX_T_CLASS3 7
+#ifdef CONFIG_PPC32
+/* The max size that one tlb can map in a 32bit kernel. */
+#define PPC_PIN_SIZE (1 << 28) /* 256M */
+#endif
+
#ifndef __ASSEMBLY__
#include <asm/bug.h>
diff --git a/arch/powerpc/mm/fsl_booke_mmu.c b/arch/powerpc/mm/fsl_booke_mmu.c
index 07ba45b..59549b3 100644
--- a/arch/powerpc/mm/fsl_booke_mmu.c
+++ b/arch/powerpc/mm/fsl_booke_mmu.c
@@ -52,6 +52,7 @@
#include <asm/smp.h>
#include <asm/machdep.h>
#include <asm/setup.h>
+#include <asm/sections.h>
#include "mmu_decl.h"
@@ -177,11 +178,34 @@ unsigned long map_mem_in_cams(unsigned long ram, int max_cam_idx)
unsigned long virt = PAGE_OFFSET;
phys_addr_t phys = memstart_addr;
unsigned long amount_mapped = 0;
-
+ unsigned long cam_sz;
+
+#if defined(CONFIG_RELOCATABLE) && defined(CONFIG_PPC32)
+ /*
+ * For a relocatable kernel, we would not map from memstart_addr.
+ * We first align to PPC_PIN_SIZE (256M), then map the PAGE_OFFSET
+ * from there.
+ */
+ phys &= ~(PPC_PIN_SIZE - 1);
+ ram += memstart_addr & (PPC_PIN_SIZE - 1);
+
+ /*
+ * For a kdump kernel, we may use a memory area reserved by the boot
+ * kernel by using a kernel option like this 'crashkernel=32M@64M'.
+ * In this case, the ram is 96M. The kernel will try to map the first
+ * 64M in the first tlb entry. The kernel will definitely get stuck,
+ * since the kernel is running above the 64M. So we have to make sure
+ * that the first tlb cover the current kernel running address at least.
+ */
+ while (1) {
+ cam_sz = calc_cam_sz(ram, virt, phys);
+ if (cam_sz + phys > PHYSICAL_START + _end - _stext)
+ break;
+ ram = 1 << (ilog2(ram) + 1);
+ }
+#endif
/* Calculate CAM values */
for (i = 0; ram && i < max_cam_idx; i++) {
- unsigned long cam_sz;
-
cam_sz = calc_cam_sz(ram, virt, phys);
settlbcam(i, virt, phys, cam_sz, PAGE_KERNEL_X, 0);
--
1.8.1.4
^ permalink raw reply related
* Re: [PATCH 3/4] powerpc: Rename and flesh out the facility unavailable exception handler
From: Stephen Rothwell @ 2013-06-27 4:05 UTC (permalink / raw)
To: Michael Ellerman; +Cc: linuxppc-dev
In-Reply-To: <1372146477-4338-3-git-send-email-michael@ellerman.id.au>
[-- Attachment #1: Type: text/plain, Size: 620 bytes --]
Hi Michael,
On Tue, 25 Jun 2013 17:47:56 +1000 Michael Ellerman <michael@ellerman.id.au> wrote:
>
> -void tm_unavailable_exception(struct pt_regs *regs)
> +void facility_unavailable_exception(struct pt_regs *regs)
> {
> + static char *facility_strings[] = {
> + "FPU",
> + "VMX/VSX",
> + "DSCR",
> + "PMU SPRs",
> + "BHRB",
> + "TM",
> + "AT",
> + "EBB",
> + "TAR",
> + };
Are the indexes into this array defined somewhere? If not, can we do
that. Then, can we use explicit indexed initialisers for this array?
--
Cheers,
Stephen Rothwell sfr@canb.auug.org.au
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply
* Re: [PATCH 3/6 v5] powerpc: export debug registers save function for KVM
From: Stephen Rothwell @ 2013-06-27 4:47 UTC (permalink / raw)
To: Bharat Bhushan
Cc: mikey, kvm, agraf, kvm-ppc, linux-kernel, Bharat Bhushan,
tiejun.chen, scottwood, linuxppc-dev
In-Reply-To: <1372225346-5029-4-git-send-email-Bharat.Bhushan@freescale.com>
[-- Attachment #1: Type: text/plain, Size: 729 bytes --]
Hi,
On Wed, 26 Jun 2013 11:12:23 +0530 Bharat Bhushan <r65777@freescale.com> wrote:
>
> diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
> index 200d763..50b357f 100644
> --- a/arch/powerpc/include/asm/switch_to.h
> +++ b/arch/powerpc/include/asm/switch_to.h
> @@ -30,6 +30,10 @@ extern void enable_kernel_spe(void);
> extern void giveup_spe(struct task_struct *);
> extern void load_up_spe(struct task_struct *);
>
> +#ifdef CONFIG_PPC_ADV_DEBUG_REGS
> +extern void switch_booke_debug_regs(struct thread_struct *new_thread);
> +#endif
We usually don't bother guarding function declarations.
--
Cheers,
Stephen Rothwell sfr@canb.auug.org.au
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply
* [PATCH v3] powerpc: Rework iommu_table locks
From: Alexey Kardashevskiy @ 2013-06-27 4:53 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, linux-kernel, Paul Mackerras,
Anton Blanchard, David Gibson
The locks in arch/powerpc/kernel/iommu.c were initally added to protect
iommu_table::it_map so the patch just makes things consistent.
Specifically, it does:
1. add missing locks for it_map access during iommu_take_ownership/
iommu_release_ownership execution where the entire it_map is marked
busy/free in order to avoid allocation from it by some broken
driver on the host.
2. remove locks from functions being called by VFIO. The whole table
is given to the user space so it is responsible now for races.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
This is actually v3 of [PATCH v2] powerpc/vfio: Add missing locks for take/release TCE table ownership
---
arch/powerpc/kernel/iommu.c | 40 ++++++++++++++++++++++++++--------------
1 file changed, 26 insertions(+), 14 deletions(-)
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index b20ff17..1391ddd 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -975,9 +975,6 @@ EXPORT_SYMBOL_GPL(iommu_tce_put_param_check);
unsigned long iommu_clear_tce(struct iommu_table *tbl, unsigned long entry)
{
unsigned long oldtce;
- struct iommu_pool *pool = get_pool(tbl, entry);
-
- spin_lock(&(pool->lock));
oldtce = ppc_md.tce_get(tbl, entry);
if (oldtce & (TCE_PCI_WRITE | TCE_PCI_READ))
@@ -985,8 +982,6 @@ unsigned long iommu_clear_tce(struct iommu_table *tbl, unsigned long entry)
else
oldtce = 0;
- spin_unlock(&(pool->lock));
-
return oldtce;
}
EXPORT_SYMBOL_GPL(iommu_clear_tce);
@@ -1024,17 +1019,12 @@ int iommu_tce_build(struct iommu_table *tbl, unsigned long entry,
{
int ret = -EBUSY;
unsigned long oldtce;
- struct iommu_pool *pool = get_pool(tbl, entry);
-
- spin_lock(&(pool->lock));
oldtce = ppc_md.tce_get(tbl, entry);
/* Add new entry if it is not busy */
if (!(oldtce & (TCE_PCI_WRITE | TCE_PCI_READ)))
ret = ppc_md.tce_build(tbl, entry, 1, hwaddr, direction, NULL);
- spin_unlock(&(pool->lock));
-
/* if (unlikely(ret))
pr_err("iommu_tce: %s failed on hwaddr=%lx ioba=%lx kva=%lx ret=%d\n",
__func__, hwaddr, entry << IOMMU_PAGE_SHIFT,
@@ -1076,32 +1066,54 @@ EXPORT_SYMBOL_GPL(iommu_put_tce_user_mode);
int iommu_take_ownership(struct iommu_table *tbl)
{
unsigned long sz = (tbl->it_size + 7) >> 3;
+ unsigned long i, flags;
+ int ret = 0;
+
+ spin_lock_irqsave(&tbl->large_pool.lock, flags);
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_lock(&tbl->pools[i].lock);
if (tbl->it_offset == 0)
clear_bit(0, tbl->it_map);
if (!bitmap_empty(tbl->it_map, tbl->it_size)) {
pr_err("iommu_tce: it_map is not empty");
- return -EBUSY;
+ ret = -EBUSY;
+ } else {
+ memset(tbl->it_map, 0xff, sz);
}
- memset(tbl->it_map, 0xff, sz);
- iommu_clear_tces_and_put_pages(tbl, tbl->it_offset, tbl->it_size);
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_unlock(&tbl->pools[i].lock);
+ spin_unlock_irqrestore(&tbl->large_pool.lock, flags);
- return 0;
+ if (!ret)
+ iommu_clear_tces_and_put_pages(tbl, tbl->it_offset, tbl->it_size);
+
+ return ret;
}
EXPORT_SYMBOL_GPL(iommu_take_ownership);
void iommu_release_ownership(struct iommu_table *tbl)
{
unsigned long sz = (tbl->it_size + 7) >> 3;
+ unsigned long i, flags;
iommu_clear_tces_and_put_pages(tbl, tbl->it_offset, tbl->it_size);
+
+ spin_lock_irqsave(&tbl->large_pool.lock, flags);
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_lock(&tbl->pools[i].lock);
+
memset(tbl->it_map, 0, sz);
/* Restore bit#0 set by iommu_init_table() */
if (tbl->it_offset == 0)
set_bit(0, tbl->it_map);
+
+ for (i = 0; i < tbl->nr_pools; i++)
+ spin_unlock(&tbl->pools[i].lock);
+ spin_unlock_irqrestore(&tbl->large_pool.lock, flags);
}
EXPORT_SYMBOL_GPL(iommu_release_ownership);
--
1.7.10.4
^ permalink raw reply related
* [PATCH 0/8 v4] KVM: PPC: IOMMU in-kernel handling
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
The changes are:
1. rebased on v3.10-rc7
2. removed spinlocks from real mode
3. added security checks between KVM and VFIO
MOre details in the individual patch comments.
Alexey Kardashevskiy (8):
KVM: PPC: reserve a capability number for multitce support
KVM: PPC: reserve a capability and ioctl numbers for realmode VFIO
vfio: add external user support
hashtable: add hash_for_each_possible_rcu_notrace()
powerpc: Prepare to support kernel handling of IOMMU map/unmap
KVM: PPC: Add support for multiple-TCE hcalls
KVM: PPC: Add support for IOMMU in-kernel handling
KVM: PPC: Add hugepage support for IOMMU in-kernel handling
Documentation/virtual/kvm/api.txt | 51 +++
arch/powerpc/include/asm/kvm_host.h | 31 ++
arch/powerpc/include/asm/kvm_ppc.h | 18 +-
arch/powerpc/include/asm/pgtable-ppc64.h | 4 +
arch/powerpc/include/uapi/asm/kvm.h | 8 +
arch/powerpc/kvm/book3s_64_vio.c | 506 +++++++++++++++++++++++++++++-
arch/powerpc/kvm/book3s_64_vio_hv.c | 439 ++++++++++++++++++++++++--
arch/powerpc/kvm/book3s_hv.c | 41 ++-
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 ++-
arch/powerpc/kvm/powerpc.c | 15 +
arch/powerpc/mm/init_64.c | 78 ++++-
drivers/vfio/vfio.c | 53 ++++
include/linux/hashtable.h | 15 +
include/linux/page-flags.h | 4 +-
include/uapi/linux/kvm.h | 3 +
16 files changed, 1279 insertions(+), 30 deletions(-)
--
1.7.10.4
^ permalink raw reply
* [PATCH 1/8] KVM: PPC: reserve a capability number for multitce support
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
include/uapi/linux/kvm.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index d88c8ee..970b1f5 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -666,6 +666,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_IRQ_MPIC 90
#define KVM_CAP_PPC_RTAS 91
#define KVM_CAP_IRQ_XICS 92
+#define KVM_CAP_SPAPR_MULTITCE 93
#ifdef KVM_CAP_IRQ_ROUTING
--
1.7.10.4
^ permalink raw reply related
* [PATCH 2/8] KVM: PPC: reserve a capability and ioctl numbers for realmode VFIO
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
include/uapi/linux/kvm.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 970b1f5..0865c01 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -667,6 +667,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_PPC_RTAS 91
#define KVM_CAP_IRQ_XICS 92
#define KVM_CAP_SPAPR_MULTITCE 93
+#define KVM_CAP_SPAPR_TCE_IOMMU 94
#ifdef KVM_CAP_IRQ_ROUTING
@@ -923,6 +924,7 @@ struct kvm_s390_ucas_mapping {
/* Available with KVM_CAP_PPC_ALLOC_HTAB */
#define KVM_PPC_ALLOCATE_HTAB _IOWR(KVMIO, 0xa7, __u32)
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct kvm_create_spapr_tce)
+#define KVM_CREATE_SPAPR_TCE_IOMMU _IOW(KVMIO, 0xaf, struct kvm_create_spapr_tce_iommu)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
/* Available with KVM_CAP_PPC_HTAB_FD */
--
1.7.10.4
^ permalink raw reply related
* [PATCH 3/8] vfio: add external user support
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
VFIO is designed to be used via ioctls on file descriptors
returned by VFIO.
However in some situations support for an external user is required.
The first user is KVM on PPC64 (SPAPR TCE protocol) which is going to
use the existing VFIO groups for exclusive access in real/virtual mode
in the host kernel to avoid passing map/unmap requests to the user
space which would made things pretty slow.
The proposed protocol includes:
1. do normal VFIO init stuff such as opening a new container, attaching
group(s) to it, setting an IOMMU driver for a container. When IOMMU is
set for a container, all groups in it are considered ready to use by
an external user.
2. pass a fd of the group we want to accelerate to KVM. KVM calls
vfio_group_iommu_id_from_file() to verify if the group is initialized
and IOMMU is set for it. The current TCE IOMMU driver marks the whole
IOMMU table as busy when IOMMU is set for a container what this prevents
other DMA users from allocating from it so it is safe to pass the group
to the user space.
3. KVM increases the container users counter via
vfio_group_add_external_user(). This prevents the VFIO group from
being disposed prior to exiting KVM.
4. When KVM is finished and doing cleanup, it releases the group file
and decrements the container users counter. Everything gets released.
5. KVM also keeps the group file as otherwise its fd might have been
closed at the moment of KVM finish so vfio_group_del_external_user()
call will not be possible.
The "vfio: Limit group opens" patch is also required for the consistency.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
drivers/vfio/vfio.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index c488da5..54192b2 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -1370,6 +1370,59 @@ static const struct file_operations vfio_device_fops = {
};
/**
+ * External user API, exported by symbols to be linked dynamically.
+ */
+
+/* Allows an external user (for example, KVM) to lock an IOMMU group */
+static int vfio_group_add_external_user(struct file *filep)
+{
+ struct vfio_group *group = filep->private_data;
+
+ if (filep->f_op != &vfio_group_fops)
+ return -EINVAL;
+
+ if (!atomic_inc_not_zero(&group->container_users))
+ return -EINVAL;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_group_add_external_user);
+
+/* Allows an external user (for example, KVM) to unlock an IOMMU group */
+static void vfio_group_del_external_user(struct file *filep)
+{
+ struct vfio_group *group = filep->private_data;
+
+ BUG_ON(filep->f_op != &vfio_group_fops);
+
+ vfio_group_try_dissolve_container(group);
+}
+EXPORT_SYMBOL_GPL(vfio_group_del_external_user);
+
+/*
+ * Checks if a group for the specified file can be used by
+ * an external user and returns the IOMMU ID if external use is possible.
+ */
+static int vfio_group_iommu_id_from_file(struct file *filep)
+{
+ int ret;
+ struct vfio_group *group = filep->private_data;
+
+ if (WARN_ON(filep->f_op != &vfio_group_fops))
+ return -EINVAL;
+
+ if (0 == atomic_read(&group->container_users) ||
+ !group->container->iommu_driver ||
+ !vfio_group_viable(group))
+ return -EINVAL;
+
+ ret = iommu_group_id(group->iommu_group);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(vfio_group_iommu_id_from_file);
+
+/**
* Module/class support
*/
static char *vfio_devnode(struct device *dev, umode_t *mode)
--
1.7.10.4
^ permalink raw reply related
* [PATCH 4/8] hashtable: add hash_for_each_possible_rcu_notrace()
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
This adds hash_for_each_possible_rcu_notrace() which is basically
a notrace clone of hash_for_each_possible_rcu() which cannot be
used in real mode due to its tracing/debugging capability.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
include/linux/hashtable.h | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/include/linux/hashtable.h b/include/linux/hashtable.h
index a9df51f..af8b169 100644
--- a/include/linux/hashtable.h
+++ b/include/linux/hashtable.h
@@ -174,6 +174,21 @@ static inline void hash_del_rcu(struct hlist_node *node)
member)
/**
+ * hash_for_each_possible_rcu_notrace - iterate over all possible objects hashing
+ * to the same bucket in an rcu enabled hashtable in a rcu enabled hashtable
+ * @name: hashtable to iterate
+ * @obj: the type * to use as a loop cursor for each entry
+ * @member: the name of the hlist_node within the struct
+ * @key: the key of the objects to iterate over
+ *
+ * This is the same as hash_for_each_possible_rcu() except that it does
+ * not do any RCU debugging or tracing.
+ */
+#define hash_for_each_possible_rcu_notrace(name, obj, member, key) \
+ hlist_for_each_entry_rcu_notrace(obj, &name[hash_min(key, HASH_BITS(name))],\
+ member)
+
+/**
* hash_for_each_possible_safe - iterate over all possible objects hashing to the
* same bucket safe against removals
* @name: hashtable to iterate
--
1.7.10.4
^ permalink raw reply related
* [PATCH 5/8] powerpc: Prepare to support kernel handling of IOMMU map/unmap
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
The current VFIO-on-POWER implementation supports only user mode
driven mapping, i.e. QEMU is sending requests to map/unmap pages.
However this approach is really slow, so we want to move that to KVM.
Since H_PUT_TCE can be extremely performance sensitive (especially with
network adapters where each packet needs to be mapped/unmapped) we chose
to implement that as a "fast" hypercall directly in "real
mode" (processor still in the guest context but MMU off).
To be able to do that, we need to provide some facilities to
access the struct page count within that real mode environment as things
like the sparsemem vmemmap mappings aren't accessible.
This adds an API to increment/decrement page counter as
get_user_pages API used for user mode mapping does not work
in the real mode.
CONFIG_SPARSEMEM_VMEMMAP and CONFIG_FLATMEM are supported.
Reviewed-by: Paul Mackerras <paulus@samba.org>
Cc: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
2013/06/27:
* realmode_get_page() fixed to use get_page_unless_zero(). If failed,
the call will be passed from real to virtual mode and safely handled.
* added comment to PageCompound() in include/linux/page-flags.h.
2013/05/20:
* PageTail() is replaced by PageCompound() in order to have the same checks
for whether the page is huge in realmode_get_page() and realmode_put_page()
---
arch/powerpc/include/asm/pgtable-ppc64.h | 4 ++
arch/powerpc/mm/init_64.c | 78 +++++++++++++++++++++++++++++-
include/linux/page-flags.h | 4 +-
3 files changed, 84 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/pgtable-ppc64.h b/arch/powerpc/include/asm/pgtable-ppc64.h
index e3d55f6f..7b46e5f 100644
--- a/arch/powerpc/include/asm/pgtable-ppc64.h
+++ b/arch/powerpc/include/asm/pgtable-ppc64.h
@@ -376,6 +376,10 @@ static inline pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea,
}
#endif /* !CONFIG_HUGETLB_PAGE */
+struct page *realmode_pfn_to_page(unsigned long pfn);
+int realmode_get_page(struct page *page);
+int realmode_put_page(struct page *page);
+
#endif /* __ASSEMBLY__ */
#endif /* _ASM_POWERPC_PGTABLE_PPC64_H_ */
diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
index a90b9c4..7031be3 100644
--- a/arch/powerpc/mm/init_64.c
+++ b/arch/powerpc/mm/init_64.c
@@ -297,5 +297,81 @@ void vmemmap_free(unsigned long start, unsigned long end)
{
}
-#endif /* CONFIG_SPARSEMEM_VMEMMAP */
+/*
+ * We do not have access to the sparsemem vmemmap, so we fallback to
+ * walking the list of sparsemem blocks which we already maintain for
+ * the sake of crashdump. In the long run, we might want to maintain
+ * a tree if performance of that linear walk becomes a problem.
+ *
+ * Any of realmode_XXXX functions can fail due to:
+ * 1) As real sparsemem blocks do not lay in RAM continously (they
+ * are in virtual address space which is not available in the real mode),
+ * the requested page struct can be split between blocks so get_page/put_page
+ * may fail.
+ * 2) When huge pages are used, the get_page/put_page API will fail
+ * in real mode as the linked addresses in the page struct are virtual
+ * too.
+ * When 1) or 2) takes place, the API returns an error code to cause
+ * an exit to kernel virtual mode where the operation will be completed.
+ */
+struct page *realmode_pfn_to_page(unsigned long pfn)
+{
+ struct vmemmap_backing *vmem_back;
+ struct page *page;
+ unsigned long page_size = 1 << mmu_psize_defs[mmu_vmemmap_psize].shift;
+ unsigned long pg_va = (unsigned long) pfn_to_page(pfn);
+
+ for (vmem_back = vmemmap_list; vmem_back; vmem_back = vmem_back->list) {
+ if (pg_va < vmem_back->virt_addr)
+ continue;
+ /* Check that page struct is not split between real pages */
+ if ((pg_va + sizeof(struct page)) >
+ (vmem_back->virt_addr + page_size))
+ return NULL;
+
+ page = (struct page *) (vmem_back->phys + pg_va -
+ vmem_back->virt_addr);
+ return page;
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(realmode_pfn_to_page);
+
+#elif defined(CONFIG_FLATMEM)
+
+struct page *realmode_pfn_to_page(unsigned long pfn)
+{
+ struct page *page = pfn_to_page(pfn);
+ return page;
+}
+EXPORT_SYMBOL_GPL(realmode_pfn_to_page);
+
+#endif /* CONFIG_SPARSEMEM_VMEMMAP/CONFIG_FLATMEM */
+
+#if defined(CONFIG_SPARSEMEM_VMEMMAP) || defined(CONFIG_FLATMEM)
+int realmode_get_page(struct page *page)
+{
+ if (PageCompound(page))
+ return -EAGAIN;
+
+ if (!get_page_unless_zero(page))
+ return -EAGAIN;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(realmode_get_page);
+
+int realmode_put_page(struct page *page)
+{
+ if (PageCompound(page))
+ return -EAGAIN;
+
+ if (!atomic_add_unless(&page->_count, -1, 1))
+ return -EAGAIN;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(realmode_put_page);
+#endif
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 6d53675..98ada58 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -329,7 +329,9 @@ static inline void set_page_writeback(struct page *page)
* System with lots of page flags available. This allows separate
* flags for PageHead() and PageTail() checks of compound pages so that bit
* tests can be used in performance sensitive paths. PageCompound is
- * generally not used in hot code paths.
+ * generally not used in hot code paths except arch/powerpc/mm/init_64.c
+ * and arch/powerpc/kvm/book3s_64_vio_hv.c which use it to detect huge pages
+ * and avoid handling those in real mode.
*/
__PAGEFLAG(Head, head) CLEARPAGEFLAG(Head, head)
__PAGEFLAG(Tail, tail)
--
1.7.10.4
^ permalink raw reply related
* [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
This adds real mode handlers for the H_PUT_TCE_INDIRECT and
H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO
devices or emulated PCI. These calls allow adding multiple entries
(up to 512) into the TCE table in one call which saves time on
transition to/from real mode.
This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs
(copied from user and verified) before writing the whole list into
the TCE table. This cache will be utilized more in the upcoming
VFIO/IOMMU support to continue TCE list processing in the virtual
mode in the case if the real mode handler failed for some reason.
This adds a guest physical to host real address converter
and calls the existing H_PUT_TCE handler. The converting function
is going to be fully utilized by upcoming VFIO supporting patches.
This also implements the KVM_CAP_PPC_MULTITCE capability,
so in order to support the functionality of this patch, QEMU
needs to query for this capability and set the "hcall-multi-tce"
hypertas property only if the capability is present, otherwise
there will be serious performance degradation.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changelog:
2013/06/27:
* fixed clear of BUSY bit in kvmppc_lookup_pte()
* H_PUT_TCE_INDIRECT does realmode_get_page() now
* KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64
* updated doc
2013/06/05:
* fixed mistype about IBMVIO in the commit message
* updated doc and moved it to another section
* changed capability number
2013/05/21:
* added kvm_vcpu_arch::tce_tmp
* removed cleanup if put_indirect failed, instead we do not even start
writing to TCE table if we cannot get TCEs from the user and they are
invalid
* kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce
and kvmppc_emulated_validate_tce (for the previous item)
* fixed bug with failthrough for H_IPI
* removed all get_user() from real mode handlers
* kvmppc_lookup_pte() added (instead of making lookup_linux_pte public)
---
Documentation/virtual/kvm/api.txt | 25 +++
arch/powerpc/include/asm/kvm_host.h | 2 +
arch/powerpc/include/asm/kvm_ppc.h | 16 +-
arch/powerpc/kvm/book3s_64_vio.c | 123 ++++++++++++++
arch/powerpc/kvm/book3s_64_vio_hv.c | 270 +++++++++++++++++++++++++++----
arch/powerpc/kvm/book3s_hv.c | 41 ++++-
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 ++++-
arch/powerpc/kvm/powerpc.c | 3 +
9 files changed, 490 insertions(+), 33 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 6365fef..762c703 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be
handled.
+4.86 KVM_CAP_PPC_MULTITCE
+
+Capability: KVM_CAP_PPC_MULTITCE
+Architectures: ppc
+Type: vm
+
+This capability means the kernel is capable of handling hypercalls
+H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user
+space. This significanly accelerates DMA operations for PPC KVM guests.
+The user space should expect that its handlers for these hypercalls
+are not going to be called.
+
+In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest,
+the user space might have to advertise it for the guest. For example,
+IBM pSeries guest starts using them if "hcall-multi-tce" is present in
+the "ibm,hypertas-functions" device-tree property.
+
+Without this capability, only H_PUT_TCE is handled by the kernel and
+therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended
+unless the capability is present as passing hypercalls to the userspace
+slows operations a lot.
+
+Unlike other capabilities of this section, this one is always enabled.
+
+
5. The kvm_run structure
------------------------
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index af326cd..3bf407b 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -609,6 +609,8 @@ struct kvm_vcpu_arch {
spinlock_t tbacct_lock;
u64 busy_stolen;
u64 busy_preempt;
+
+ unsigned long *tce_tmp; /* TCE cache for TCE_PUT_INDIRECT hcall */
#endif
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index a5287fe..e852921b 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -133,8 +133,20 @@ extern int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu);
extern long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
struct kvm_create_spapr_tce *args);
-extern long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
- unsigned long ioba, unsigned long tce);
+extern struct kvmppc_spapr_tce_table *kvmppc_find_tce_table(
+ struct kvm_vcpu *vcpu, unsigned long liobn);
+extern long kvmppc_emulated_validate_tce(unsigned long tce);
+extern void kvmppc_emulated_put_tce(struct kvmppc_spapr_tce_table *tt,
+ unsigned long ioba, unsigned long tce);
+extern long kvmppc_virtmode_h_put_tce(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce);
+extern long kvmppc_virtmode_h_put_tce_indirect(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce_list, unsigned long npages);
+extern long kvmppc_virtmode_h_stuff_tce(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce_value, unsigned long npages);
extern long kvm_vm_ioctl_allocate_rma(struct kvm *kvm,
struct kvm_allocate_rma *rma);
extern struct kvmppc_linear_info *kvm_alloc_rma(void);
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index b2d3f3b..45ee05a 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -14,6 +14,7 @@
*
* Copyright 2010 Paul Mackerras, IBM Corp. <paulus@au1.ibm.com>
* Copyright 2011 David Gibson, IBM Corporation <dwg@au1.ibm.com>
+ * Copyright 2013 Alexey Kardashevskiy, IBM Corporation <aik@au1.ibm.com>
*/
#include <linux/types.h>
@@ -36,8 +37,11 @@
#include <asm/ppc-opcode.h>
#include <asm/kvm_host.h>
#include <asm/udbg.h>
+#include <asm/iommu.h>
+#include <asm/tce.h>
#define TCES_PER_PAGE (PAGE_SIZE / sizeof(u64))
+#define ERROR_ADDR ((void *)~(unsigned long)0x0)
static long kvmppc_stt_npages(unsigned long window_size)
{
@@ -148,3 +152,122 @@ fail:
}
return ret;
}
+
+/* Converts guest physical address to host virtual address */
+static void __user *kvmppc_virtmode_gpa_to_hva(struct kvm_vcpu *vcpu,
+ unsigned long gpa)
+{
+ unsigned long hva, gfn = gpa >> PAGE_SHIFT;
+ struct kvm_memory_slot *memslot;
+
+ memslot = search_memslots(kvm_memslots(vcpu->kvm), gfn);
+ if (!memslot)
+ return ERROR_ADDR;
+
+ hva = __gfn_to_hva_memslot(memslot, gfn) + (gpa & ~PAGE_MASK);
+ return (void *) hva;
+}
+
+long kvmppc_virtmode_h_put_tce(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce)
+{
+ long ret;
+ struct kvmppc_spapr_tce_table *tt;
+
+ tt = kvmppc_find_tce_table(vcpu, liobn);
+ /* Didn't find the liobn, put it to userspace */
+ if (!tt)
+ return H_TOO_HARD;
+
+ if (ioba >= tt->window_size)
+ return H_PARAMETER;
+
+ ret = kvmppc_emulated_validate_tce(tce);
+ if (ret)
+ return ret;
+
+ kvmppc_emulated_put_tce(tt, ioba, tce);
+
+ return H_SUCCESS;
+}
+
+long kvmppc_virtmode_h_put_tce_indirect(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce_list, unsigned long npages)
+{
+ struct kvmppc_spapr_tce_table *tt;
+ long i, ret = H_SUCCESS;
+ unsigned long __user *tces;
+ struct page *pg;
+
+ tt = kvmppc_find_tce_table(vcpu, liobn);
+ /* Didn't find the liobn, put it to userspace */
+ if (!tt)
+ return H_TOO_HARD;
+
+ /*
+ * The spec says that the maximum size of the list is 512 TCEs so
+ * so the whole table addressed resides in 4K page
+ */
+ if (npages > 512)
+ return H_PARAMETER;
+
+ if (tce_list & ~IOMMU_PAGE_MASK)
+ return H_PARAMETER;
+
+ if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
+ return H_PARAMETER;
+
+ tces = kvmppc_virtmode_gpa_to_hva(vcpu, tce_list);
+ if (tces == ERROR_ADDR)
+ return H_TOO_HARD;
+
+ if (get_user_pages_fast((unsigned long) tces & PAGE_MASK,
+ 1, 0, &pg) != 1)
+ return H_TOO_HARD;
+
+ for (i = 0; i < npages; ++i) {
+ if (get_user(vcpu->arch.tce_tmp[i], tces + i)) {
+ ret = H_PARAMETER;
+ goto put_list_page_exit;
+ }
+
+ ret = kvmppc_emulated_validate_tce(vcpu->arch.tce_tmp[i]);
+ if (ret)
+ goto put_list_page_exit;
+ }
+
+ for (i = 0; i < npages; ++i)
+ kvmppc_emulated_put_tce(tt, ioba + (i << IOMMU_PAGE_SHIFT),
+ vcpu->arch.tce_tmp[i]);
+put_list_page_exit:
+ put_page(pg);
+
+ return ret;
+}
+
+long kvmppc_virtmode_h_stuff_tce(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce_value, unsigned long npages)
+{
+ struct kvmppc_spapr_tce_table *tt;
+ long i, ret;
+
+ tt = kvmppc_find_tce_table(vcpu, liobn);
+ /* Didn't find the liobn, put it to userspace */
+ if (!tt)
+ return H_TOO_HARD;
+
+ if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
+ return H_PARAMETER;
+
+ ret = kvmppc_emulated_validate_tce(tce_value);
+ if (ret || (tce_value & (TCE_PCI_WRITE | TCE_PCI_READ)))
+ return H_PARAMETER;
+
+ for (i = 0; i < npages; ++i, ioba += IOMMU_PAGE_SIZE)
+ kvmppc_emulated_put_tce(tt, ioba, tce_value);
+
+ return H_SUCCESS;
+}
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 30c2f3b..108cc08 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -14,6 +14,7 @@
*
* Copyright 2010 Paul Mackerras, IBM Corp. <paulus@au1.ibm.com>
* Copyright 2011 David Gibson, IBM Corporation <dwg@au1.ibm.com>
+ * Copyright 2013 Alexey Kardashevskiy, IBM Corporation <aik@au1.ibm.com>
*/
#include <linux/types.h>
@@ -35,42 +36,253 @@
#include <asm/ppc-opcode.h>
#include <asm/kvm_host.h>
#include <asm/udbg.h>
+#include <asm/iommu.h>
+#include <asm/tce.h>
#define TCES_PER_PAGE (PAGE_SIZE / sizeof(u64))
+#define ERROR_ADDR (~(unsigned long)0x0)
-/* WARNING: This will be called in real-mode on HV KVM and virtual
- * mode on PR KVM
+/* Finds a TCE table descriptor by LIOBN */
+struct kvmppc_spapr_tce_table *kvmppc_find_tce_table(struct kvm_vcpu *vcpu,
+ unsigned long liobn)
+{
+ struct kvmppc_spapr_tce_table *tt;
+
+ list_for_each_entry(tt, &vcpu->kvm->arch.spapr_tce_tables, list) {
+ if (tt->liobn == liobn)
+ return tt;
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(kvmppc_find_tce_table);
+
+/*
+ * Validates TCE address.
+ * At the moment only flags are validated as other checks will significantly slow
+ * down or can make it even impossible to handle TCE requests in real mode.
+ */
+long kvmppc_emulated_validate_tce(unsigned long tce)
+{
+ if (tce & ~(IOMMU_PAGE_MASK | TCE_PCI_WRITE | TCE_PCI_READ))
+ return H_PARAMETER;
+
+ return H_SUCCESS;
+}
+EXPORT_SYMBOL_GPL(kvmppc_emulated_validate_tce);
+
+/*
+ * Handles TCE requests for QEMU emulated devices.
+ * Puts guest TCE values to the table and expects QEMU to convert them
+ * later in a QEMU device implementation.
+ * Called in both real and virtual modes.
+ * Cannot fail so kvmppc_emulated_validate_tce must be called before it.
+ */
+void kvmppc_emulated_put_tce(struct kvmppc_spapr_tce_table *tt,
+ unsigned long ioba, unsigned long tce)
+{
+ unsigned long idx = ioba >> SPAPR_TCE_SHIFT;
+ struct page *page;
+ u64 *tbl;
+
+ /*
+ * Note on the use of page_address() in real mode,
+ *
+ * It is safe to use page_address() in real mode on ppc64 because
+ * page_address() is always defined as lowmem_page_address()
+ * which returns __va(PFN_PHYS(page_to_pfn(page))) which is arithmetial
+ * operation and does not access page struct.
+ *
+ * Theoretically page_address() could be defined different
+ * but either WANT_PAGE_VIRTUAL or HASHED_PAGE_VIRTUAL
+ * should be enabled.
+ * WANT_PAGE_VIRTUAL is never enabled on ppc32/ppc64,
+ * HASHED_PAGE_VIRTUAL could be enabled for ppc32 only and only
+ * if CONFIG_HIGHMEM is defined. As CONFIG_SPARSEMEM_VMEMMAP
+ * is not expected to be enabled on ppc32, page_address()
+ * is safe for ppc32 as well.
+ */
+#if defined(HASHED_PAGE_VIRTUAL) || defined(WANT_PAGE_VIRTUAL)
+#error TODO: fix to avoid page_address() here
+#endif
+ page = tt->pages[idx / TCES_PER_PAGE];
+ tbl = (u64 *)page_address(page);
+
+ /* udbg_printf("tce @ %p\n", &tbl[idx % TCES_PER_PAGE]); */
+ tbl[idx % TCES_PER_PAGE] = tce;
+}
+EXPORT_SYMBOL_GPL(kvmppc_emulated_put_tce);
+
+#ifdef CONFIG_KVM_BOOK3S_64_HV
+/*
+ * Converts guest physical address to host physical address.
+ * Tries to increase page counter via realmode_get_page() and
+ * returns ERROR_ADDR if failed.
*/
+static unsigned long kvmppc_realmode_gpa_to_hpa(struct kvm_vcpu *vcpu,
+ unsigned long gpa)
+{
+ struct kvm_memory_slot *memslot;
+ pte_t *ptep, pte;
+ unsigned long hva, hpa = ERROR_ADDR;
+ unsigned long gfn = gpa >> PAGE_SHIFT;
+ unsigned shift = 0;
+ struct page *pg;
+
+ memslot = search_memslots(kvm_memslots(vcpu->kvm), gfn);
+ if (!memslot)
+ return ERROR_ADDR;
+
+ hva = __gfn_to_hva_memslot(memslot, gfn);
+
+ ptep = find_linux_pte_or_hugepte(vcpu->arch.pgdir, hva, &shift);
+ if (!ptep || !pte_present(*ptep))
+ return ERROR_ADDR;
+ pte = *ptep;
+
+ if (((gpa & TCE_PCI_WRITE) || pte_write(pte)) && !pte_dirty(pte))
+ return ERROR_ADDR;
+
+ if (!pte_young(pte))
+ return ERROR_ADDR;
+
+ if (!shift)
+ shift = PAGE_SHIFT;
+
+ /* Put huge pages handling to the virtual mode */
+ if (shift > PAGE_SHIFT)
+ return ERROR_ADDR;
+
+ pg = realmode_pfn_to_page(pte_pfn(pte));
+ if (!pg || realmode_get_page(pg))
+ return ERROR_ADDR;
+
+ /* pte_pfn(pte) returns address aligned to pg_size */
+ hpa = (pte_pfn(pte) << PAGE_SHIFT) +
+ (gpa & ((1 << shift) - 1));
+
+ if (unlikely(pte_val(pte) != pte_val(*ptep))) {
+ hpa = ERROR_ADDR;
+ realmode_put_page(pg);
+ }
+
+ return hpa;
+}
+
+static long kvmppc_realmode_put_hpa(unsigned long hpa, bool dirty)
+{
+ struct page *pg;
+
+ pg = realmode_pfn_to_page(hpa >> PAGE_SHIFT);
+ if (!pg)
+ return H_TOO_HARD;
+
+ if (dirty)
+ SetPageDirty(pg);
+
+ if (PageCompound(pg))
+ return H_SUCCESS;
+
+ if (realmode_put_page(pg))
+ return H_TOO_HARD;
+
+ return H_SUCCESS;
+}
+
long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
unsigned long ioba, unsigned long tce)
{
- struct kvm *kvm = vcpu->kvm;
- struct kvmppc_spapr_tce_table *stt;
-
- /* udbg_printf("H_PUT_TCE(): liobn=0x%lx ioba=0x%lx, tce=0x%lx\n", */
- /* liobn, ioba, tce); */
-
- list_for_each_entry(stt, &kvm->arch.spapr_tce_tables, list) {
- if (stt->liobn == liobn) {
- unsigned long idx = ioba >> SPAPR_TCE_SHIFT;
- struct page *page;
- u64 *tbl;
-
- /* udbg_printf("H_PUT_TCE: liobn 0x%lx => stt=%p window_size=0x%x\n", */
- /* liobn, stt, stt->window_size); */
- if (ioba >= stt->window_size)
- return H_PARAMETER;
-
- page = stt->pages[idx / TCES_PER_PAGE];
- tbl = (u64 *)page_address(page);
-
- /* FIXME: Need to validate the TCE itself */
- /* udbg_printf("tce @ %p\n", &tbl[idx % TCES_PER_PAGE]); */
- tbl[idx % TCES_PER_PAGE] = tce;
- return H_SUCCESS;
- }
+ long ret;
+ struct kvmppc_spapr_tce_table *tt;
+
+ tt = kvmppc_find_tce_table(vcpu, liobn);
+ /* Didn't find the liobn, put it to virtual space */
+ if (!tt)
+ return H_TOO_HARD;
+
+ if (ioba >= tt->window_size)
+ return H_PARAMETER;
+
+ ret = kvmppc_emulated_validate_tce(tce);
+ if (ret)
+ return ret;
+
+ kvmppc_emulated_put_tce(tt, ioba, tce);
+
+ return H_SUCCESS;
+}
+
+long kvmppc_h_put_tce_indirect(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce_list, unsigned long npages)
+{
+ struct kvmppc_spapr_tce_table *tt;
+ long i, ret = H_SUCCESS;
+ unsigned long *tces;
+
+ tt = kvmppc_find_tce_table(vcpu, liobn);
+ /* Didn't find the liobn, put it to virtual space */
+ if (!tt)
+ return H_TOO_HARD;
+
+ /*
+ * The spec says that the maximum size of the list is 512 TCEs so
+ * so the whole table addressed resides in 4K page
+ */
+ if (npages > 512)
+ return H_PARAMETER;
+
+ if (tce_list & ~IOMMU_PAGE_MASK)
+ return H_PARAMETER;
+
+ if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
+ return H_PARAMETER;
+
+ tces = (unsigned long *) kvmppc_realmode_gpa_to_hpa(vcpu, tce_list);
+ if ((unsigned long)tces == ERROR_ADDR)
+ return H_TOO_HARD;
+
+ for (i = 0; i < npages; ++i) {
+ ret = kvmppc_emulated_validate_tce(tces[i]);
+ if (ret)
+ goto put_list_page_exit;
+ }
+
+ for (i = 0; i < npages; ++i)
+ kvmppc_emulated_put_tce(tt, ioba + (i << IOMMU_PAGE_SHIFT),
+ tces[i]);
+
+put_list_page_exit:
+ if (kvmppc_realmode_put_hpa((unsigned long)tces, false)) {
+ vcpu->arch.tce_reason = H_TOO_HARD;
+ ret = H_TOO_HARD;
}
- /* Didn't find the liobn, punt it to userspace */
- return H_TOO_HARD;
+ return ret;
+}
+
+long kvmppc_h_stuff_tce(struct kvm_vcpu *vcpu,
+ unsigned long liobn, unsigned long ioba,
+ unsigned long tce_value, unsigned long npages)
+{
+ struct kvmppc_spapr_tce_table *tt;
+ long i, ret;
+
+ tt = kvmppc_find_tce_table(vcpu, liobn);
+ /* Didn't find the liobn, put it to virtual space */
+ if (!tt)
+ return H_TOO_HARD;
+
+ if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
+ return H_PARAMETER;
+
+ ret = kvmppc_emulated_validate_tce(tce_value);
+ if (ret || (tce_value & (TCE_PCI_WRITE | TCE_PCI_READ)))
+ return H_PARAMETER;
+
+ for (i = 0; i < npages; ++i, ioba += IOMMU_PAGE_SIZE)
+ kvmppc_emulated_put_tce(tt, ioba, tce_value);
+
+ return H_SUCCESS;
}
+#endif /* CONFIG_KVM_BOOK3S_64_HV */
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 550f592..b18b076 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -567,7 +567,31 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)
if (kvmppc_xics_enabled(vcpu)) {
ret = kvmppc_xics_hcall(vcpu, req);
break;
- } /* fallthrough */
+ }
+ return RESUME_HOST;
+ case H_PUT_TCE:
+ ret = kvmppc_virtmode_h_put_tce(vcpu, kvmppc_get_gpr(vcpu, 4),
+ kvmppc_get_gpr(vcpu, 5),
+ kvmppc_get_gpr(vcpu, 6));
+ if (ret == H_TOO_HARD)
+ return RESUME_HOST;
+ break;
+ case H_PUT_TCE_INDIRECT:
+ ret = kvmppc_virtmode_h_put_tce_indirect(vcpu, kvmppc_get_gpr(vcpu, 4),
+ kvmppc_get_gpr(vcpu, 5),
+ kvmppc_get_gpr(vcpu, 6),
+ kvmppc_get_gpr(vcpu, 7));
+ if (ret == H_TOO_HARD)
+ return RESUME_HOST;
+ break;
+ case H_STUFF_TCE:
+ ret = kvmppc_virtmode_h_stuff_tce(vcpu, kvmppc_get_gpr(vcpu, 4),
+ kvmppc_get_gpr(vcpu, 5),
+ kvmppc_get_gpr(vcpu, 6),
+ kvmppc_get_gpr(vcpu, 7));
+ if (ret == H_TOO_HARD)
+ return RESUME_HOST;
+ break;
default:
return RESUME_HOST;
}
@@ -958,6 +982,20 @@ struct kvm_vcpu *kvmppc_core_vcpu_create(struct kvm *kvm, unsigned int id)
vcpu->arch.cpu_type = KVM_CPU_3S_64;
kvmppc_sanity_check(vcpu);
+ /*
+ * As we want to minimize the chance of having H_PUT_TCE_INDIRECT
+ * half executed, we first read TCEs from the user, check them and
+ * return error if something went wrong and only then put TCEs into
+ * the TCE table.
+ *
+ * tce_tmp is a cache for TCEs to avoid stack allocation or
+ * kmalloc as the whole TCE list can take up to 512 items 8 bytes
+ * each (4096 bytes).
+ */
+ vcpu->arch.tce_tmp = kmalloc(4096, GFP_KERNEL);
+ if (!vcpu->arch.tce_tmp)
+ goto free_vcpu;
+
return vcpu;
free_vcpu:
@@ -980,6 +1018,7 @@ void kvmppc_core_vcpu_free(struct kvm_vcpu *vcpu)
unpin_vpa(vcpu->kvm, &vcpu->arch.slb_shadow);
unpin_vpa(vcpu->kvm, &vcpu->arch.vpa);
spin_unlock(&vcpu->arch.vpa_update_lock);
+ kfree(vcpu->arch.tce_tmp);
kvm_vcpu_uninit(vcpu);
kmem_cache_free(kvm_vcpu_cache, vcpu);
}
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index b02f91e..d35554e 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -1490,6 +1490,12 @@ hcall_real_table:
.long 0 /* 0x11c */
.long 0 /* 0x120 */
.long .kvmppc_h_bulk_remove - hcall_real_table
+ .long 0 /* 0x128 */
+ .long 0 /* 0x12c */
+ .long 0 /* 0x130 */
+ .long 0 /* 0x134 */
+ .long .kvmppc_h_stuff_tce - hcall_real_table
+ .long .kvmppc_h_put_tce_indirect - hcall_real_table
hcall_real_table_end:
ignore_hdec:
diff --git a/arch/powerpc/kvm/book3s_pr_papr.c b/arch/powerpc/kvm/book3s_pr_papr.c
index da0e0bc..91d4b45 100644
--- a/arch/powerpc/kvm/book3s_pr_papr.c
+++ b/arch/powerpc/kvm/book3s_pr_papr.c
@@ -220,7 +220,38 @@ static int kvmppc_h_pr_put_tce(struct kvm_vcpu *vcpu)
unsigned long tce = kvmppc_get_gpr(vcpu, 6);
long rc;
- rc = kvmppc_h_put_tce(vcpu, liobn, ioba, tce);
+ rc = kvmppc_virtmode_h_put_tce(vcpu, liobn, ioba, tce);
+ if (rc == H_TOO_HARD)
+ return EMULATE_FAIL;
+ kvmppc_set_gpr(vcpu, 3, rc);
+ return EMULATE_DONE;
+}
+
+static int kvmppc_h_pr_put_tce_indirect(struct kvm_vcpu *vcpu)
+{
+ unsigned long liobn = kvmppc_get_gpr(vcpu, 4);
+ unsigned long ioba = kvmppc_get_gpr(vcpu, 5);
+ unsigned long tce = kvmppc_get_gpr(vcpu, 6);
+ unsigned long npages = kvmppc_get_gpr(vcpu, 7);
+ long rc;
+
+ rc = kvmppc_virtmode_h_put_tce_indirect(vcpu, liobn, ioba,
+ tce, npages);
+ if (rc == H_TOO_HARD)
+ return EMULATE_FAIL;
+ kvmppc_set_gpr(vcpu, 3, rc);
+ return EMULATE_DONE;
+}
+
+static int kvmppc_h_pr_stuff_tce(struct kvm_vcpu *vcpu)
+{
+ unsigned long liobn = kvmppc_get_gpr(vcpu, 4);
+ unsigned long ioba = kvmppc_get_gpr(vcpu, 5);
+ unsigned long tce_value = kvmppc_get_gpr(vcpu, 6);
+ unsigned long npages = kvmppc_get_gpr(vcpu, 7);
+ long rc;
+
+ rc = kvmppc_virtmode_h_stuff_tce(vcpu, liobn, ioba, tce_value, npages);
if (rc == H_TOO_HARD)
return EMULATE_FAIL;
kvmppc_set_gpr(vcpu, 3, rc);
@@ -247,6 +278,10 @@ int kvmppc_h_pr(struct kvm_vcpu *vcpu, unsigned long cmd)
return kvmppc_h_pr_bulk_remove(vcpu);
case H_PUT_TCE:
return kvmppc_h_pr_put_tce(vcpu);
+ case H_PUT_TCE_INDIRECT:
+ return kvmppc_h_pr_put_tce_indirect(vcpu);
+ case H_STUFF_TCE:
+ return kvmppc_h_pr_stuff_tce(vcpu);
case H_CEDE:
vcpu->arch.shared->msr |= MSR_EE;
kvm_vcpu_block(vcpu);
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 6316ee3..ccb578b 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -394,6 +394,9 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_PPC_GET_SMMU_INFO:
r = 1;
break;
+ case KVM_CAP_SPAPR_MULTITCE:
+ r = 1;
+ break;
#endif
default:
r = 0;
--
1.7.10.4
^ permalink raw reply related
* [PATCH 7/8] KVM: PPC: Add support for IOMMU in-kernel handling
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
This allows the host kernel to handle H_PUT_TCE, H_PUT_TCE_INDIRECT
and H_STUFF_TCE requests without passing them to QEMU, which saves time
on switching to QEMU and back.
Both real and virtual modes are supported. First the kernel tries to
handle a TCE request in the real mode, if failed it passes it to
the virtual mode to complete the operation. If it a virtual mode
handler fails, a request is passed to the user mode.
This adds a new KVM_CAP_SPAPR_TCE_IOMMU ioctl to associate
a virtual PCI bus ID (LIOBN) with an IOMMU group which enables
in-kernel handling of IOMMU map/unmap. The external user API support
in VFIO is required.
Tests show that this patch increases transmission speed from 220MB/s
to 750..1020MB/s on 10Gb network (Chelsea CXGB3 10Gb ethernet card).
Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
2013/06/27:
* tce_list page is referenced now in order to protect it from accident
invalidation during H_PUT_TCE_INDIRECT execution
* added use of the external user VFIO API
2013/06/05:
* changed capability number
* changed ioctl number
* update the doc article number
2013/05/20:
* removed get_user() from real mode handlers
* kvm_vcpu_arch::tce_tmp usage extended. Now real mode handler puts there
translated TCEs, tries realmode_get_page() on those and if it fails, it
passes control over the virtual mode handler which tries to finish
the request handling
* kvmppc_lookup_pte() now does realmode_get_page() protected by BUSY bit
on a page
* The only reason to pass the request to user mode now is when the user mode
did not register TCE table in the kernel, in all other cases the virtual mode
handler is expected to do the job
---
Documentation/virtual/kvm/api.txt | 26 ++++
arch/powerpc/include/asm/kvm_host.h | 4 +
arch/powerpc/include/asm/kvm_ppc.h | 2 +
arch/powerpc/include/uapi/asm/kvm.h | 8 +
arch/powerpc/kvm/book3s_64_vio.c | 294 ++++++++++++++++++++++++++++++++++-
arch/powerpc/kvm/book3s_64_vio_hv.c | 165 ++++++++++++++++++++
arch/powerpc/kvm/powerpc.c | 12 ++
7 files changed, 509 insertions(+), 2 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 762c703..01b0dc2 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2387,6 +2387,32 @@ slows operations a lot.
Unlike other capabilities of this section, this one is always enabled.
+4.87 KVM_CREATE_SPAPR_TCE_IOMMU
+
+Capability: KVM_CAP_SPAPR_TCE_IOMMU
+Architectures: powerpc
+Type: vm ioctl
+Parameters: struct kvm_create_spapr_tce_iommu (in)
+Returns: 0 on success, -1 on error
+
+struct kvm_create_spapr_tce_iommu {
+ __u64 liobn;
+ __u32 iommu_id;
+ __u32 flags;
+};
+
+This creates a link between IOMMU group and a hardware TCE (translation
+control entry) table. This link lets the host kernel know what IOMMU
+group (i.e. TCE table) to use for the LIOBN number passed with
+H_PUT_TCE, H_PUT_TCE_INDIRECT, H_STUFF_TCE hypercalls.
+
+In response to a TCE hypercall, the kernel looks for a TCE table descriptor
+in the list and handles the hypercall in real or virtual modes if
+the descriptor is found. Otherwise the hypercall is passed to the user mode.
+
+No flag is supported at the moment.
+
+
5. The kvm_run structure
------------------------
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 3bf407b..716ab18 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -180,6 +180,8 @@ struct kvmppc_spapr_tce_table {
struct kvm *kvm;
u64 liobn;
u32 window_size;
+ struct iommu_group *grp; /* used for IOMMU groups */
+ struct file *vfio_filp; /* used for IOMMU groups */
struct page *pages[0];
};
@@ -611,6 +613,8 @@ struct kvm_vcpu_arch {
u64 busy_preempt;
unsigned long *tce_tmp; /* TCE cache for TCE_PUT_INDIRECT hcall */
+ unsigned long tce_tmp_num; /* Number of handled TCEs in the cache */
+ unsigned long tce_reason; /* The reason of switching to the virtmode */
#endif
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index e852921b..934e01d 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -133,6 +133,8 @@ extern int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu);
extern long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
struct kvm_create_spapr_tce *args);
+extern long kvm_vm_ioctl_create_spapr_tce_iommu(struct kvm *kvm,
+ struct kvm_create_spapr_tce_iommu *args);
extern struct kvmppc_spapr_tce_table *kvmppc_find_tce_table(
struct kvm_vcpu *vcpu, unsigned long liobn);
extern long kvmppc_emulated_validate_tce(unsigned long tce);
diff --git a/arch/powerpc/include/uapi/asm/kvm.h b/arch/powerpc/include/uapi/asm/kvm.h
index 0fb1a6e..7d0fc02 100644
--- a/arch/powerpc/include/uapi/asm/kvm.h
+++ b/arch/powerpc/include/uapi/asm/kvm.h
@@ -319,6 +319,14 @@ struct kvm_create_spapr_tce {
__u32 window_size;
};
+/* for KVM_CAP_SPAPR_TCE_IOMMU */
+struct kvm_create_spapr_tce_iommu {
+ __u64 liobn;
+ __u32 fd;
+ __u32 iommu_id;
+ __u32 flags;
+};
+
/* for KVM_ALLOCATE_RMA */
struct kvm_allocate_rma {
__u64 rma_size;
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index 45ee05a..a5d0195 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -27,6 +27,10 @@
#include <linux/hugetlb.h>
#include <linux/list.h>
#include <linux/anon_inodes.h>
+#include <linux/pci.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/file.h>
#include <asm/tlbflush.h>
#include <asm/kvm_ppc.h>
@@ -49,6 +53,47 @@ static long kvmppc_stt_npages(unsigned long window_size)
* sizeof(u64), PAGE_SIZE) / PAGE_SIZE;
}
+extern int vfio_group_add_external_user(struct file *filep);
+static int kvmppc_vfio_group_add_external_user(struct file *filep)
+{
+ int ret;
+ int (*proc)(struct file *) = symbol_get(vfio_group_add_external_user);
+
+ if (!proc)
+ return -EINVAL;
+
+ ret = proc(filep);
+ symbol_put(vfio_group_add_external_user);
+
+ return ret;
+}
+
+extern void vfio_group_del_external_user(struct file *filep);
+static void kvmppc_vfio_group_del_external_user(struct file *filep)
+{
+ void (*proc)(struct file *) = symbol_get(vfio_group_del_external_user);
+
+ if (!proc)
+ return;
+
+ proc(filep);
+ symbol_put(vfio_group_del_external_user);
+}
+
+extern int vfio_group_iommu_id_from_file(struct file *filep);
+static int kvmppc_vfio_group_iommu_id_from_file(struct file *filep)
+{
+ int ret;
+ int (*proc)(struct file *) = symbol_get(vfio_group_iommu_id_from_file);
+
+ if (!proc)
+ return -EINVAL;
+
+ ret = proc(filep);
+ symbol_put(vfio_group_iommu_id_from_file);
+
+ return ret;
+}
static void release_spapr_tce_table(struct kvmppc_spapr_tce_table *stt)
{
struct kvm *kvm = stt->kvm;
@@ -56,8 +101,17 @@ static void release_spapr_tce_table(struct kvmppc_spapr_tce_table *stt)
mutex_lock(&kvm->lock);
list_del(&stt->list);
- for (i = 0; i < kvmppc_stt_npages(stt->window_size); i++)
- __free_page(stt->pages[i]);
+#ifdef CONFIG_IOMMU_API
+ if (stt->grp) {
+ if (stt->vfio_filp) {
+ kvmppc_vfio_group_del_external_user(stt->vfio_filp);
+ fput(stt->vfio_filp);
+ }
+ iommu_group_put(stt->grp);
+ } else
+#endif
+ for (i = 0; i < kvmppc_stt_npages(stt->window_size); i++)
+ __free_page(stt->pages[i]);
kfree(stt);
mutex_unlock(&kvm->lock);
@@ -153,6 +207,99 @@ fail:
return ret;
}
+#ifdef CONFIG_IOMMU_API
+static const struct file_operations kvm_spapr_tce_iommu_fops = {
+ .release = kvm_spapr_tce_release,
+};
+
+long kvm_vm_ioctl_create_spapr_tce_iommu(struct kvm *kvm,
+ struct kvm_create_spapr_tce_iommu *args)
+{
+ struct kvmppc_spapr_tce_table *tt = NULL;
+ struct iommu_group *grp;
+ struct iommu_table *tbl;
+ struct file *vfio_filp;
+ int ret;
+
+ /* Check this LIOBN hasn't been previously allocated */
+ list_for_each_entry(tt, &kvm->arch.spapr_tce_tables, list) {
+ if (tt->liobn == args->liobn)
+ return -EBUSY;
+ }
+
+ vfio_filp = fget(args->fd);
+ if (!vfio_filp)
+ return -ENXIO;
+
+ /* Lock the group */
+ ret = kvmppc_vfio_group_add_external_user(vfio_filp);
+ if (ret)
+ goto fput_exit;
+
+ /* Get IOMMU ID. Fails if group is not attached to IOMMU */
+ ret = kvmppc_vfio_group_iommu_id_from_file(vfio_filp);
+ if (ret < 0)
+ goto del_fput_exit;
+
+ if (ret != args->iommu_id) {
+ ret = -EINVAL;
+ goto del_fput_exit;
+ }
+
+ ret = -ENXIO;
+ /* Find an IOMMU table for the given ID */
+ grp = iommu_group_get_by_id(args->iommu_id);
+ if (!grp)
+ goto del_fput_exit;
+
+ tbl = iommu_group_get_iommudata(grp);
+ if (!tbl)
+ goto del_fput_exit;
+
+ tt = kzalloc(sizeof(*tt), GFP_KERNEL);
+ if (!tt)
+ goto del_fput_exit;
+
+ tt->liobn = args->liobn;
+ tt->kvm = kvm;
+ tt->grp = grp;
+ tt->window_size = tbl->it_size << IOMMU_PAGE_SHIFT;
+ tt->vfio_filp = vfio_filp;
+
+ pr_debug("LIOBN=%llX fd=%d hooked to IOMMU %d, flags=%u\n",
+ args->liobn, args->fd, args->iommu_id, args->flags);
+
+ ret = anon_inode_getfd("kvm-spapr-tce-iommu",
+ &kvm_spapr_tce_iommu_fops, tt, O_RDWR);
+ if (ret < 0)
+ goto free_del_fput_exit;
+
+ kvm_get_kvm(kvm);
+
+ mutex_lock(&kvm->lock);
+ list_add(&tt->list, &kvm->arch.spapr_tce_tables);
+
+ mutex_unlock(&kvm->lock);
+
+ return ret;
+
+free_del_fput_exit:
+ kfree(tt);
+del_fput_exit:
+ kvmppc_vfio_group_del_external_user(vfio_filp);
+fput_exit:
+ fput(vfio_filp);
+
+ return ret;
+}
+#else
+long kvm_vm_ioctl_create_spapr_tce_iommu(struct kvm *kvm,
+ struct kvm_create_spapr_tce_iommu *args)
+{
+ return -ENOSYS;
+}
+#endif /* CONFIG_IOMMU_API */
+
/* Converts guest physical address to host virtual address */
static void __user *kvmppc_virtmode_gpa_to_hva(struct kvm_vcpu *vcpu,
unsigned long gpa)
@@ -180,6 +327,47 @@ long kvmppc_virtmode_h_put_tce(struct kvm_vcpu *vcpu,
if (!tt)
return H_TOO_HARD;
+#ifdef CONFIG_IOMMU_API
+ if (tt->grp) {
+ unsigned long entry = ioba >> IOMMU_PAGE_SHIFT;
+ struct iommu_table *tbl = iommu_group_get_iommudata(tt->grp);
+
+ /* Return error if the group is being destroyed */
+ if (!tbl)
+ return H_RESCINDED;
+
+ if (vcpu->arch.tce_reason == H_HARDWARE) {
+ iommu_clear_tces_and_put_pages(tbl, entry, 1);
+ return H_HARDWARE;
+
+ } else if (!(tce & (TCE_PCI_READ | TCE_PCI_WRITE))) {
+ if (iommu_tce_clear_param_check(tbl, ioba, 0, 1))
+ return H_PARAMETER;
+
+ ret = iommu_clear_tces_and_put_pages(tbl, entry, 1);
+ } else {
+ void *hva;
+
+ if (iommu_tce_put_param_check(tbl, ioba, tce))
+ return H_PARAMETER;
+
+ hva = kvmppc_virtmode_gpa_to_hva(vcpu, tce);
+ if (hva == ERROR_ADDR)
+ return H_HARDWARE;
+
+ ret = iommu_put_tce_user_mode(tbl,
+ ioba >> IOMMU_PAGE_SHIFT,
+ (unsigned long) hva);
+ }
+ iommu_flush_tce(tbl);
+
+ if (ret)
+ return H_HARDWARE;
+
+ return H_SUCCESS;
+ }
+#endif
+ /* Emulated IO */
if (ioba >= tt->window_size)
return H_PARAMETER;
@@ -192,6 +380,72 @@ long kvmppc_virtmode_h_put_tce(struct kvm_vcpu *vcpu,
return H_SUCCESS;
}
+static long kvmppc_virtmode_h_put_tce_indirect_iommu(struct kvm_vcpu *vcpu,
+ struct kvmppc_spapr_tce_table *tt, unsigned long ioba,
+ unsigned long *tces, unsigned long npages)
+{
+ int i;
+ struct iommu_table *tbl = iommu_group_get_iommudata(tt->grp);
+
+ /* Return error if the group is being destroyed */
+ if (!tbl)
+ return H_RESCINDED;
+
+ /* Something bad happened, do cleanup and exit */
+ if (vcpu->arch.tce_reason == H_HARDWARE) {
+ i = vcpu->arch.tce_tmp_num;
+ goto fail_clear_tce;
+ } else if (vcpu->arch.tce_reason != H_TOO_HARD) {
+ /*
+ * We get here in PR KVM mode or
+ * when realmode_get_page() failed on tce_list.
+ */
+ for (i = 0; i < npages; ++i) {
+ if (get_user(vcpu->arch.tce_tmp[i], tces + i))
+ return H_HARDWARE;
+
+ if (iommu_tce_put_param_check(tbl, ioba +
+ (i << IOMMU_PAGE_SHIFT),
+ vcpu->arch.tce_tmp[i]))
+ return H_PARAMETER;
+ }
+ } /* else: The real mode handler checked TCEs already */
+
+ /* Translate TCEs */
+ for (i = vcpu->arch.tce_tmp_num; i < npages; ++i) {
+ void *hva = kvmppc_virtmode_gpa_to_hva(vcpu,
+ vcpu->arch.tce_tmp[i]);
+
+ if (hva == ERROR_ADDR)
+ goto fail_clear_tce;
+
+ vcpu->arch.tce_tmp[i] = (unsigned long) hva;
+ }
+
+ /* Do get_page and put TCEs for all pages */
+ for (i = 0; i < npages; ++i) {
+ if (iommu_put_tce_user_mode(tbl,
+ (ioba >> IOMMU_PAGE_SHIFT) + i,
+ vcpu->arch.tce_tmp[i])) {
+ i = npages;
+ goto fail_clear_tce;
+ }
+ }
+
+ iommu_flush_tce(tbl);
+
+ return H_SUCCESS;
+
+fail_clear_tce:
+ /* Cannot complete the translation, clean up and exit */
+ iommu_clear_tces_and_put_pages(tbl, ioba >> IOMMU_PAGE_SHIFT, i);
+
+ iommu_flush_tce(tbl);
+
+ return H_HARDWARE;
+
+}
+
long kvmppc_virtmode_h_put_tce_indirect(struct kvm_vcpu *vcpu,
unsigned long liobn, unsigned long ioba,
unsigned long tce_list, unsigned long npages)
@@ -227,6 +481,14 @@ long kvmppc_virtmode_h_put_tce_indirect(struct kvm_vcpu *vcpu,
1, 0, &pg) != 1)
return H_TOO_HARD;
+#ifdef CONFIG_IOMMU_API
+ if (tt->grp) {
+ ret = kvmppc_virtmode_h_put_tce_indirect_iommu(vcpu,
+ tt, ioba, tces, npages);
+ goto put_list_page_exit;
+ }
+#endif
+ /* Emulated IO */
for (i = 0; i < npages; ++i) {
if (get_user(vcpu->arch.tce_tmp[i], tces + i)) {
ret = H_PARAMETER;
@@ -259,6 +521,34 @@ long kvmppc_virtmode_h_stuff_tce(struct kvm_vcpu *vcpu,
if (!tt)
return H_TOO_HARD;
+#ifdef CONFIG_IOMMU_API
+ if (tt->grp) {
+ struct iommu_table *tbl = iommu_group_get_iommudata(tt->grp);
+ unsigned long tmp, entry = ioba >> IOMMU_PAGE_SHIFT;
+
+ vcpu->arch.tce_tmp_num = 0;
+
+ /* Return error if the group is being destroyed */
+ if (!tbl)
+ return H_RESCINDED;
+
+ /* PR KVM? */
+ if (!vcpu->arch.tce_tmp_num &&
+ (vcpu->arch.tce_reason != H_TOO_HARD) &&
+ iommu_tce_clear_param_check(tbl, ioba,
+ tce_value, npages))
+ return H_PARAMETER;
+
+ /* Do actual cleanup */
+ tmp = vcpu->arch.tce_tmp_num;
+ if (iommu_clear_tces_and_put_pages(tbl, entry + tmp,
+ npages - tmp))
+ return H_PARAMETER;
+
+ return H_SUCCESS;
+ }
+#endif
+ /* Emulated IO */
if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
return H_PARAMETER;
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 108cc08..497daaf 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -26,6 +26,7 @@
#include <linux/slab.h>
#include <linux/hugetlb.h>
#include <linux/list.h>
+#include <linux/iommu.h>
#include <asm/tlbflush.h>
#include <asm/kvm_ppc.h>
@@ -189,6 +190,41 @@ static long kvmppc_realmode_put_hpa(unsigned long hpa, bool dirty)
return H_SUCCESS;
}
+#ifdef CONFIG_IOMMU_API
+static long kvmppc_realmode_clear_tce(struct kvm_vcpu *vcpu,
+ struct iommu_table *tbl, unsigned long ioba,
+ unsigned long tce_value, unsigned long npages)
+{
+ long ret = 0, i;
+ unsigned long entry = ioba >> IOMMU_PAGE_SHIFT;
+
+ if (iommu_tce_clear_param_check(tbl, ioba, tce_value, npages))
+ return H_PARAMETER;
+
+ for (i = 0; i < npages; ++i) {
+ unsigned long oldtce;
+
+ oldtce = iommu_clear_tce(tbl, entry + i);
+ if (!oldtce)
+ continue;
+
+ ret = kvmppc_realmode_put_hpa(oldtce, oldtce & TCE_PCI_WRITE);
+ if (ret)
+ break;
+ }
+
+ if (ret == H_TOO_HARD) {
+ vcpu->arch.tce_tmp_num = i;
+ vcpu->arch.tce_reason = H_TOO_HARD;
+ }
+ /* if (ret < 0)
+ pr_err("iommu_tce: %s failed ioba=%lx, tce_value=%lx ret=%ld\n",
+ __func__, ioba, tce_value, ret); */
+
+ return ret;
+}
+#endif /* CONFIG_IOMMU_API */
+
long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
unsigned long ioba, unsigned long tce)
{
@@ -200,6 +236,53 @@ long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
if (!tt)
return H_TOO_HARD;
+#ifdef CONFIG_IOMMU_API
+ if (tt->grp) {
+ struct iommu_table *tbl = iommu_group_get_iommudata(tt->grp);
+
+ /* Return error if the group is being destroyed */
+ if (!tbl)
+ return H_RESCINDED;
+
+ vcpu->arch.tce_reason = 0;
+
+ if (tce & (TCE_PCI_READ | TCE_PCI_WRITE)) {
+ unsigned long hpa, hva;
+
+ if (iommu_tce_put_param_check(tbl, ioba, tce))
+ return H_PARAMETER;
+
+ hpa = kvmppc_realmode_gpa_to_hpa(vcpu, tce);
+ if (hpa == ERROR_ADDR) {
+ vcpu->arch.tce_reason = H_TOO_HARD;
+ return H_TOO_HARD;
+ }
+
+ hva = (unsigned long) __va(hpa);
+ ret = iommu_tce_build(tbl,
+ ioba >> IOMMU_PAGE_SHIFT,
+ hva, iommu_tce_direction(hva));
+ if (unlikely(ret)) {
+ ret = kvmppc_realmode_put_hpa(hpa,
+ tce & TCE_PCI_WRITE);
+ if (ret) {
+ vcpu->arch.tce_reason = H_HARDWARE;
+ return H_TOO_HARD;
+ }
+ return H_HARDWARE;
+ }
+ } else {
+ ret = kvmppc_realmode_clear_tce(vcpu, tbl, ioba, 0, 1);
+ if (ret)
+ return ret;
+ }
+
+ iommu_flush_tce(tbl);
+
+ return H_SUCCESS;
+ }
+#endif
+ /* Emulated IO */
if (ioba >= tt->window_size)
return H_PARAMETER;
@@ -212,6 +295,56 @@ long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
return H_SUCCESS;
}
+static long kvmppc_h_put_tce_indirect_iommu(struct kvm_vcpu *vcpu,
+ struct kvmppc_spapr_tce_table *tt, unsigned long ioba,
+ unsigned long *tces, unsigned long npages)
+{
+ int i;
+ struct iommu_table *tbl = iommu_group_get_iommudata(tt->grp);
+
+ /* Return error if the group is being destroyed */
+ if (!tbl)
+ return H_RESCINDED;
+
+ /* Check all TCEs */
+ for (i = 0; i < npages; ++i) {
+ if (iommu_tce_put_param_check(tbl, ioba +
+ (i << IOMMU_PAGE_SHIFT), tces[i]))
+ return H_PARAMETER;
+
+ vcpu->arch.tce_tmp[i] = tces[i];
+ }
+
+ /* Translate TCEs and go get_page */
+ for (i = 0; i < npages; ++i) {
+ unsigned long hpa = kvmppc_realmode_gpa_to_hpa(vcpu,
+ vcpu->arch.tce_tmp[i]);
+ if (hpa == ERROR_ADDR) {
+ vcpu->arch.tce_tmp_num = i;
+ vcpu->arch.tce_reason = H_TOO_HARD;
+ return H_TOO_HARD;
+ }
+ vcpu->arch.tce_tmp[i] = hpa;
+ }
+
+ /* Put TCEs to the table */
+ for (i = 0; i < npages; ++i) {
+ unsigned long hva = (unsigned long)
+ __va(vcpu->arch.tce_tmp[i]);
+ if (iommu_tce_build(tbl,
+ (ioba >> IOMMU_PAGE_SHIFT) + i,
+ hva, iommu_tce_direction(hva))) {
+ /* All wrong, go virtmode and do cleanup */
+ vcpu->arch.tce_reason = H_HARDWARE;
+ return H_TOO_HARD;
+ }
+ }
+
+ iommu_flush_tce(tbl);
+
+ return H_SUCCESS;
+}
+
long kvmppc_h_put_tce_indirect(struct kvm_vcpu *vcpu,
unsigned long liobn, unsigned long ioba,
unsigned long tce_list, unsigned long npages)
@@ -238,10 +371,21 @@ long kvmppc_h_put_tce_indirect(struct kvm_vcpu *vcpu,
if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
return H_PARAMETER;
+ vcpu->arch.tce_tmp_num = 0;
+ vcpu->arch.tce_reason = 0;
+
tces = (unsigned long *) kvmppc_realmode_gpa_to_hpa(vcpu, tce_list);
if ((unsigned long)tces == ERROR_ADDR)
return H_TOO_HARD;
+#ifdef CONFIG_IOMMU_API
+ if (tt->grp) {
+ ret = kvmppc_h_put_tce_indirect_iommu(vcpu,
+ tt, ioba, tces, npages);
+ goto put_list_page_exit;
+ }
+#endif
+ /* Emulated IO */
for (i = 0; i < npages; ++i) {
ret = kvmppc_emulated_validate_tce(tces[i]);
if (ret)
@@ -273,6 +417,27 @@ long kvmppc_h_stuff_tce(struct kvm_vcpu *vcpu,
if (!tt)
return H_TOO_HARD;
+#ifdef CONFIG_IOMMU_API
+ if (tt->grp) {
+ struct iommu_table *tbl = iommu_group_get_iommudata(tt->grp);
+
+ /* Return error if the group is being destroyed */
+ if (!tbl)
+ return H_RESCINDED;
+
+ vcpu->arch.tce_reason = 0;
+
+ ret = kvmppc_realmode_clear_tce(vcpu, tbl, ioba,
+ tce_value, npages);
+ if (ret)
+ return ret;
+
+ iommu_flush_tce(tbl);
+
+ return H_SUCCESS;
+ }
+#endif
+ /* Emulated IO */
if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
return H_PARAMETER;
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index ccb578b..2909cfa 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -395,6 +395,7 @@ int kvm_dev_ioctl_check_extension(long ext)
r = 1;
break;
case KVM_CAP_SPAPR_MULTITCE:
+ case KVM_CAP_SPAPR_TCE_IOMMU:
r = 1;
break;
#endif
@@ -1025,6 +1026,17 @@ long kvm_arch_vm_ioctl(struct file *filp,
r = kvm_vm_ioctl_create_spapr_tce(kvm, &create_tce);
goto out;
}
+ case KVM_CREATE_SPAPR_TCE_IOMMU: {
+ struct kvm_create_spapr_tce_iommu create_tce_iommu;
+ struct kvm *kvm = filp->private_data;
+
+ r = -EFAULT;
+ if (copy_from_user(&create_tce_iommu, argp,
+ sizeof(create_tce_iommu)))
+ goto out;
+ r = kvm_vm_ioctl_create_spapr_tce_iommu(kvm, &create_tce_iommu);
+ goto out;
+ }
#endif /* CONFIG_PPC_BOOK3S_64 */
#ifdef CONFIG_KVM_BOOK3S_64_HV
--
1.7.10.4
^ permalink raw reply related
* [PATCH 8/8] KVM: PPC: Add hugepage support for IOMMU in-kernel handling
From: Alexey Kardashevskiy @ 2013-06-27 5:02 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, linux-doc, Alexey Kardashevskiy, Alexander Graf, kvm-ppc,
linux-kernel, Alex Williamson, Paul Mackerras, Paul E . McKenney,
David Gibson
In-Reply-To: <1372309356-28320-1-git-send-email-aik@ozlabs.ru>
This adds special support for huge pages (16MB). The reference
counting cannot be easily done for such pages in real mode (when
MMU is off) so we added a list of huge pages. It is populated in
virtual mode and get_page is called just once per a huge page.
Real mode handlers check if the requested page is huge and in the list,
then no reference counting is done, otherwise an exit to virtual mode
happens. The list is released at KVM exit. At the moment the fastest
card available for tests uses up to 9 huge pages so walking through this
list is not very expensive. However this can change and we may want
to optimize this.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
2013/06/27:
* list of huge pages replaces with hashtable for better performance
* spinlock removed from real mode and only protects insertion of new
huge [ages descriptors into the hashtable
2013/06/05:
* fixed compile error when CONFIG_IOMMU_API=n
2013/05/20:
* the real mode handler now searches for a huge page by gpa (used to be pte)
* the virtual mode handler prints warning if it is called twice for the same
huge page as the real mode handler is expected to fail just once - when a huge
page is not in the list yet.
* the huge page is refcounted twice - when added to the hugepage list and
when used in the virtual mode hcall handler (can be optimized but it will
make the patch less nice).
---
arch/powerpc/include/asm/kvm_host.h | 25 +++++++++
arch/powerpc/kvm/book3s_64_vio.c | 95 +++++++++++++++++++++++++++++++++--
arch/powerpc/kvm/book3s_64_vio_hv.c | 24 +++++++--
3 files changed, 138 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 716ab18..0ad6189 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -30,6 +30,7 @@
#include <linux/kvm_para.h>
#include <linux/list.h>
#include <linux/atomic.h>
+#include <linux/hashtable.h>
#include <asm/kvm_asm.h>
#include <asm/processor.h>
#include <asm/page.h>
@@ -182,9 +183,33 @@ struct kvmppc_spapr_tce_table {
u32 window_size;
struct iommu_group *grp; /* used for IOMMU groups */
struct file *vfio_filp; /* used for IOMMU groups */
+ DECLARE_HASHTABLE(hash_tab, ilog2(64)); /* used for IOMMU groups */
+ spinlock_t hugepages_write_lock; /* used for IOMMU groups */
struct page *pages[0];
};
+/*
+ * The KVM guest can be backed with 16MB pages.
+ * In this case, we cannot do page counting from the real mode
+ * as the compound pages are used - they are linked in a list
+ * with pointers as virtual addresses which are inaccessible
+ * in real mode.
+ *
+ * The code below keeps a 16MB pages list and uses page struct
+ * in real mode if it is already locked in RAM and inserted into
+ * the list or switches to the virtual mode where it can be
+ * handled in a usual manner.
+ */
+#define KVMPPC_HUGEPAGE_HASH(gpa) hash_32(gpa >> 24, 32)
+
+struct kvmppc_iommu_hugepage {
+ struct hlist_node hash_node;
+ unsigned long gpa; /* Guest physical address */
+ unsigned long hpa; /* Host physical address */
+ struct page *page; /* page struct of the very first subpage */
+ unsigned long size; /* Huge page size (always 16MB at the moment) */
+};
+
struct kvmppc_linear_info {
void *base_virt;
unsigned long base_pfn;
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index a5d0195..6cedfe9 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -47,6 +47,78 @@
#define TCES_PER_PAGE (PAGE_SIZE / sizeof(u64))
#define ERROR_ADDR ((void *)~(unsigned long)0x0)
+#ifdef CONFIG_IOMMU_API
+/* Adds a new huge page descriptor to the hashtable */
+static long kvmppc_iommu_hugepage_try_add(
+ struct kvmppc_spapr_tce_table *tt,
+ pte_t pte, unsigned long hva, unsigned long gpa,
+ unsigned long pg_size)
+{
+ long ret = 0;
+ struct kvmppc_iommu_hugepage *hp;
+ struct page *pg;
+ unsigned key = KVMPPC_HUGEPAGE_HASH(gpa);
+
+ spin_lock(&tt->hugepages_write_lock);
+ hash_for_each_possible_rcu(tt->hash_tab, hp, hash_node, key) {
+ if (KVMPPC_HUGEPAGE_HASH(hp->gpa) != key)
+ continue;
+ if ((gpa < hp->gpa) || (gpa >= hp->gpa + hp->size))
+ continue;
+ goto unlock_exit;
+ }
+
+ hva = hva & ~(pg_size - 1);
+ ret = get_user_pages_fast(hva, 1, true/*write*/, &pg);
+ if ((ret != 1) || !pg) {
+ ret = -EFAULT;
+ goto unlock_exit;
+ }
+ ret = 0;
+
+ hp = kzalloc(sizeof(*hp), GFP_KERNEL);
+ if (!hp) {
+ ret = -ENOMEM;
+ goto unlock_exit;
+ }
+
+ hp->page = pg;
+ hp->gpa = gpa & ~(pg_size - 1);
+ hp->hpa = (pte_pfn(pte) << PAGE_SHIFT);
+ hp->size = pg_size;
+
+ hash_add_rcu(tt->hash_tab, &hp->hash_node, key);
+
+unlock_exit:
+ spin_unlock(&tt->hugepages_write_lock);
+
+ return ret;
+}
+
+static void kvmppc_iommu_hugepages_init(struct kvmppc_spapr_tce_table *tt)
+{
+ spin_lock_init(&tt->hugepages_write_lock);
+ hash_init(tt->hash_tab);
+}
+
+static void kvmppc_iommu_hugepages_cleanup(struct kvmppc_spapr_tce_table *tt)
+{
+ int bkt;
+ struct kvmppc_iommu_hugepage *hp;
+ struct hlist_node *tmp;
+
+ spin_lock(&tt->hugepages_write_lock);
+ hash_for_each_safe(tt->hash_tab, bkt, tmp, hp, hash_node) {
+ hlist_del_rcu(&hp->hash_node);
+
+ put_page(hp->page); /* one for iommu_put_tce_user_mode */
+ put_page(hp->page); /* one for kvmppc_iommu_hugepage_try_add */
+ kfree(hp);
+ }
+ spin_unlock(&tt->hugepages_write_lock);
+}
+#endif /* CONFIG_IOMMU_API */
+
static long kvmppc_stt_npages(unsigned long window_size)
{
return ALIGN((window_size >> SPAPR_TCE_SHIFT)
@@ -108,6 +180,7 @@ static void release_spapr_tce_table(struct kvmppc_spapr_tce_table *stt)
fput(stt->vfio_filp);
}
iommu_group_put(stt->grp);
+ kvmppc_iommu_hugepages_cleanup(stt);
} else
#endif
for (i = 0; i < kvmppc_stt_npages(stt->window_size); i++)
@@ -277,6 +350,7 @@ long kvm_vm_ioctl_create_spapr_tce_iommu(struct kvm *kvm,
kvm_get_kvm(kvm);
mutex_lock(&kvm->lock);
+ kvmppc_iommu_hugepages_init(tt);
list_add(&tt->list, &kvm->arch.spapr_tce_tables);
mutex_unlock(&kvm->lock);
@@ -302,16 +376,31 @@ long kvm_vm_ioctl_create_spapr_tce_iommu(struct kvm *kvm,
/* Converts guest physical address to host virtual address */
static void __user *kvmppc_virtmode_gpa_to_hva(struct kvm_vcpu *vcpu,
+ struct kvmppc_spapr_tce_table *tt,
unsigned long gpa)
{
unsigned long hva, gfn = gpa >> PAGE_SHIFT;
struct kvm_memory_slot *memslot;
+ pte_t *ptep;
+ unsigned int shift = 0;
memslot = search_memslots(kvm_memslots(vcpu->kvm), gfn);
if (!memslot)
return ERROR_ADDR;
hva = __gfn_to_hva_memslot(memslot, gfn) + (gpa & ~PAGE_MASK);
+
+ ptep = find_linux_pte_or_hugepte(vcpu->arch.pgdir, hva, &shift);
+ WARN_ON(!ptep);
+ if (!ptep)
+ return ERROR_ADDR;
+#ifdef CONFIG_IOMMU_API
+ if (tt && (shift > PAGE_SHIFT)) {
+ if (kvmppc_iommu_hugepage_try_add(tt, *ptep,
+ hva, gpa, 1 << shift))
+ return ERROR_ADDR;
+ }
+#endif
return (void *) hva;
}
@@ -351,7 +440,7 @@ long kvmppc_virtmode_h_put_tce(struct kvm_vcpu *vcpu,
if (iommu_tce_put_param_check(tbl, ioba, tce))
return H_PARAMETER;
- hva = kvmppc_virtmode_gpa_to_hva(vcpu, tce);
+ hva = kvmppc_virtmode_gpa_to_hva(vcpu, tt, tce);
if (hva == ERROR_ADDR)
return H_HARDWARE;
@@ -414,7 +503,7 @@ static long kvmppc_virtmode_h_put_tce_indirect_iommu(struct kvm_vcpu *vcpu,
/* Translate TCEs */
for (i = vcpu->arch.tce_tmp_num; i < npages; ++i) {
void *hva = kvmppc_virtmode_gpa_to_hva(vcpu,
- vcpu->arch.tce_tmp[i]);
+ tt, vcpu->arch.tce_tmp[i]);
if (hva == ERROR_ADDR)
goto fail_clear_tce;
@@ -473,7 +562,7 @@ long kvmppc_virtmode_h_put_tce_indirect(struct kvm_vcpu *vcpu,
if ((ioba + (npages << IOMMU_PAGE_SHIFT)) > tt->window_size)
return H_PARAMETER;
- tces = kvmppc_virtmode_gpa_to_hva(vcpu, tce_list);
+ tces = kvmppc_virtmode_gpa_to_hva(vcpu, NULL, tce_list);
if (tces == ERROR_ADDR)
return H_TOO_HARD;
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index 497daaf..c1db019 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -121,6 +121,7 @@ EXPORT_SYMBOL_GPL(kvmppc_emulated_put_tce);
* returns ERROR_ADDR if failed.
*/
static unsigned long kvmppc_realmode_gpa_to_hpa(struct kvm_vcpu *vcpu,
+ struct kvmppc_spapr_tce_table *tt,
unsigned long gpa)
{
struct kvm_memory_slot *memslot;
@@ -129,6 +130,23 @@ static unsigned long kvmppc_realmode_gpa_to_hpa(struct kvm_vcpu *vcpu,
unsigned long gfn = gpa >> PAGE_SHIFT;
unsigned shift = 0;
struct page *pg;
+ struct kvmppc_iommu_hugepage *hp;
+
+ /* Try to find an already used hugepage */
+ if (tt) {
+ unsigned key = KVMPPC_HUGEPAGE_HASH(gpa);
+
+ hash_for_each_possible_rcu_notrace(tt->hash_tab, hp,
+ hash_node, key) {
+ if (KVMPPC_HUGEPAGE_HASH(hp->gpa) != key)
+ continue;
+
+ if ((gpa < hp->gpa) || (gpa >= hp->gpa + hp->size))
+ continue;
+
+ return hp->hpa + (gpa & (hp->size - 1));
+ }
+ }
memslot = search_memslots(kvm_memslots(vcpu->kvm), gfn);
if (!memslot)
@@ -252,7 +270,7 @@ long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
if (iommu_tce_put_param_check(tbl, ioba, tce))
return H_PARAMETER;
- hpa = kvmppc_realmode_gpa_to_hpa(vcpu, tce);
+ hpa = kvmppc_realmode_gpa_to_hpa(vcpu, tt, tce);
if (hpa == ERROR_ADDR) {
vcpu->arch.tce_reason = H_TOO_HARD;
return H_TOO_HARD;
@@ -318,7 +336,7 @@ static long kvmppc_h_put_tce_indirect_iommu(struct kvm_vcpu *vcpu,
/* Translate TCEs and go get_page */
for (i = 0; i < npages; ++i) {
unsigned long hpa = kvmppc_realmode_gpa_to_hpa(vcpu,
- vcpu->arch.tce_tmp[i]);
+ tt, vcpu->arch.tce_tmp[i]);
if (hpa == ERROR_ADDR) {
vcpu->arch.tce_tmp_num = i;
vcpu->arch.tce_reason = H_TOO_HARD;
@@ -374,7 +392,7 @@ long kvmppc_h_put_tce_indirect(struct kvm_vcpu *vcpu,
vcpu->arch.tce_tmp_num = 0;
vcpu->arch.tce_reason = 0;
- tces = (unsigned long *) kvmppc_realmode_gpa_to_hpa(vcpu, tce_list);
+ tces = (unsigned long *) kvmppc_realmode_gpa_to_hpa(vcpu, tt, tce_list);
if ((unsigned long)tces == ERROR_ADDR)
return H_TOO_HARD;
--
1.7.10.4
^ permalink raw reply related
* From: Gavin Shan <shangw@linux.vnet.ibm.com>
From: Gavin Shan @ 2013-06-27 5:46 UTC (permalink / raw)
To: linuxppc-dev
The series of patches are follow-up in order to make EEH workable for PowerNV
platform on Juno-IOC-L machine. Couple of issues have been fixed with help of
Ben:
- Check PCIe link after PHB complete reset
- Restore config space for bridges
- The EEH address cache wasn't built successfully
- Misc cleanup on output messages
- Misc cleanup on EEH flags maintained by "struct pnv_phb"
- Misc cleanup on properties of functions to avoid build warnings
- Let PCI config accessors rely on device node
- Do hotplug during reset for those devices whose drivers can't
support EEH
---
Trigger frozen PE:
echo 0x0000000002000000 > /sys/kernel/debug/powerpc/PCI0000/err_injct
sleep 1
echo 0x0 > /sys/kernel/debug/powerpc/PCI0000/err_injct
Trigger fenced PHB:
echo 0x8000000000000000 > /sys/kernel/debug/powerpc/PCI0000/err_injct
---
Changelog:
==========
v3 -> v4:
* Add more output messages in EEH core to let users know what the EEH
core is doing.
* Add one patch to use device node in the PCI config accessors since
the accessors used by EEH and it's not safe enough to refer PCI device
and bus. We instead fully utilize the information from PCI_DN.
* Add one patch to remove those deivces whose drivers can't support EEH
before reset, and add them to the system after reset.
v2 -> v3:
* Fix overwritten buffer while collecting data from PCI config space.
v1 -> v2:
* Remove the mechanism to block PCI-CFG and MMIO.
* Add one patch to do cleanup on output messages.
* Add one patch to avoid build warnings.
* Split functions to restore BARs for PCI devices and bridges separately.
---
arch/powerpc/include/asm/eeh.h | 8 +-
arch/powerpc/include/asm/pci.h | 1 +
arch/powerpc/kernel/eeh.c | 43 +++++--
arch/powerpc/kernel/eeh_cache.c | 4 +-
arch/powerpc/kernel/eeh_driver.c | 157 +++++++++++++++++++++++-
arch/powerpc/kernel/eeh_pe.c | 166 ++++++++++++++++++++++++--
arch/powerpc/kernel/pci_hotplug.c | 8 +-
arch/powerpc/platforms/powernv/eeh-ioda.c | 33 +++--
arch/powerpc/platforms/powernv/eeh-powernv.c | 44 +-------
arch/powerpc/platforms/powernv/pci-ioda.c | 1 +
arch/powerpc/platforms/powernv/pci.c | 124 ++++++++++++--------
arch/powerpc/platforms/powernv/pci.h | 11 ++-
drivers/pci/probe.c | 6 +-
13 files changed, 462 insertions(+), 144 deletions(-)
Thanks,
Gavin
^ permalink raw reply
* [PATCH 3/8] powerpc/powernv: Replace variables with flags
From: Gavin Shan @ 2013-06-27 5:46 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Gavin Shan
In-Reply-To: <1372312009-13710-1-git-send-email-shangw@linux.vnet.ibm.com>
We have 2 fields in "struct pnv_phb" to trace the states. The patch
replace the fields with one and introduces flags for that. The patch
doesn't impact the logic.
Signed-off-by: Gavin Shan <shangw@linux.vnet.ibm.com>
---
arch/powerpc/platforms/powernv/eeh-ioda.c | 8 ++++----
arch/powerpc/platforms/powernv/pci.c | 4 ++--
arch/powerpc/platforms/powernv/pci.h | 7 +++++--
3 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/eeh-ioda.c b/arch/powerpc/platforms/powernv/eeh-ioda.c
index 84f3036..85025d7 100644
--- a/arch/powerpc/platforms/powernv/eeh-ioda.c
+++ b/arch/powerpc/platforms/powernv/eeh-ioda.c
@@ -132,7 +132,7 @@ static int ioda_eeh_post_init(struct pci_controller *hose)
&ioda_eeh_dbgfs_ops);
#endif
- phb->eeh_enabled = 1;
+ phb->eeh_state |= PNV_EEH_STATE_ENABLED;
}
return 0;
@@ -815,7 +815,7 @@ static int ioda_eeh_next_error(struct eeh_pe **pe)
* removed, we needn't take care of it any more.
*/
phb = hose->private_data;
- if (phb->removed)
+ if (phb->eeh_state & PNV_EEH_STATE_REMOVED)
continue;
rc = opal_pci_next_error(phb->opal_id,
@@ -850,7 +850,7 @@ static int ioda_eeh_next_error(struct eeh_pe **pe)
list_for_each_entry_safe(hose, tmp,
&hose_list, list_node) {
phb = hose->private_data;
- phb->removed = 1;
+ phb->eeh_state |= PNV_EEH_STATE_REMOVED;
}
WARN(1, "EEH: dead IOC detected\n");
@@ -867,7 +867,7 @@ static int ioda_eeh_next_error(struct eeh_pe **pe)
WARN(1, "EEH: dead PHB#%x detected\n",
hose->global_number);
- phb->removed = 1;
+ phb->eeh_state |= PNV_EEH_STATE_REMOVED;
ret = 3;
goto out;
} else if (severity == OPAL_EEH_SEV_PHB_FENCED) {
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 6d9a506..1f31826 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -308,7 +308,7 @@ static int pnv_pci_read_config(struct pci_bus *bus,
if (phb_pe && (phb_pe->state & EEH_PE_ISOLATED))
return PCIBIOS_SUCCESSFUL;
- if (phb->eeh_enabled) {
+ if (phb->eeh_state & PNV_EEH_STATE_ENABLED) {
if (*val == EEH_IO_ERROR_VALUE(size)) {
busdn = pci_bus_to_OF_node(bus);
for (dn = busdn->child; dn; dn = dn->sibling) {
@@ -358,7 +358,7 @@ static int pnv_pci_write_config(struct pci_bus *bus,
/* Check if the PHB got frozen due to an error (no response) */
#ifdef CONFIG_EEH
- if (!phb->eeh_enabled)
+ if (!(phb->eeh_state & PNV_EEH_STATE_ENABLED))
pnv_pci_config_check_eeh(phb, bus, bdfn);
#else
pnv_pci_config_check_eeh(phb, bus, bdfn);
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index 43906e3..40bdf02 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -78,6 +78,10 @@ struct pnv_eeh_ops {
int (*configure_bridge)(struct eeh_pe *pe);
int (*next_error)(struct eeh_pe **pe);
};
+
+#define PNV_EEH_STATE_ENABLED (1 << 0) /* EEH enabled */
+#define PNV_EEH_STATE_REMOVED (1 << 1) /* PHB removed */
+
#endif /* CONFIG_EEH */
struct pnv_phb {
@@ -92,8 +96,7 @@ struct pnv_phb {
#ifdef CONFIG_EEH
struct pnv_eeh_ops *eeh_ops;
- int eeh_enabled;
- int removed;
+ int eeh_state;
#endif
#ifdef CONFIG_DEBUG_FS
--
1.7.5.4
^ permalink raw reply related
* [PATCH 1/8] powerpc/eeh: Don't collect PCI-CFG data on PHB
From: Gavin Shan @ 2013-06-27 5:46 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Gavin Shan
In-Reply-To: <1372312009-13710-1-git-send-email-shangw@linux.vnet.ibm.com>
When the PHB is fenced or dead, it's pointless to collect the data
from PCI config space of subordinate PCI devices since it should
return 0xFF's. The patch also fixes overwritten buffer while getting
PCI config data.
Signed-off-by: Gavin Shan <shangw@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh.c | 34 ++++++++++++++++++++++++----------
1 files changed, 24 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index 951a632..2dd0bd1 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -232,16 +232,30 @@ void eeh_slot_error_detail(struct eeh_pe *pe, int severity)
{
size_t loglen = 0;
struct eeh_dev *edev;
+ bool valid_cfg_log = true;
- eeh_pci_enable(pe, EEH_OPT_THAW_MMIO);
- eeh_ops->configure_bridge(pe);
- eeh_pe_restore_bars(pe);
-
- pci_regs_buf[0] = 0;
- eeh_pe_for_each_dev(pe, edev) {
- loglen += eeh_gather_pci_data(edev, pci_regs_buf,
- EEH_PCI_REGS_LOG_LEN);
- }
+ /*
+ * When the PHB is fenced or dead, it's pointless to collect
+ * the data from PCI config space because it should return
+ * 0xFF's. For ER, we still retrieve the data from the PCI
+ * config space.
+ */
+ if (eeh_probe_mode_dev() &&
+ (pe->type & EEH_PE_PHB) &&
+ (pe->state & (EEH_PE_ISOLATED | EEH_PE_PHB_DEAD)))
+ valid_cfg_log = false;
+
+ if (valid_cfg_log) {
+ eeh_pci_enable(pe, EEH_OPT_THAW_MMIO);
+ eeh_ops->configure_bridge(pe);
+ eeh_pe_restore_bars(pe);
+
+ pci_regs_buf[0] = 0;
+ eeh_pe_for_each_dev(pe, edev) {
+ loglen += eeh_gather_pci_data(edev, pci_regs_buf + loglen,
+ EEH_PCI_REGS_LOG_LEN - loglen);
+ }
+ }
eeh_ops->get_log(pe, severity, pci_regs_buf, loglen);
}
--
1.7.5.4
^ permalink raw reply related
* [PATCH 8/8] powernv/eeh: Do hotplug on devices without EEH aware driver
From: Gavin Shan @ 2013-06-27 5:46 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Gavin Shan
In-Reply-To: <1372312009-13710-1-git-send-email-shangw@linux.vnet.ibm.com>
During recovery for EEH errors, the device driver requires reset
explicitly (most of cases). The EEH core doesn't do hotplug during
reset. However, there might have some device drivers that can't
support EEH. So the deivce can't be put into quite state during
the reset and possibly requesting PCI config or MMIO access. That
would lead to the failure of the reset, and we don't expect that.
The patch intends to fix that by removing those devices whose drivers
can't support EEH before reset and added into the system after reset.
In the result, it would avoid the race condition mentioned as above.
The idea was proposed by Ben.
Signed-off-by: Gavin Shan <shangw@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 4 +-
arch/powerpc/include/asm/pci.h | 1 +
arch/powerpc/kernel/eeh_driver.c | 134 +++++++++++++++++++++++++++++++++++++-
3 files changed, 137 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 09a8743..dd62006 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -82,7 +82,8 @@ struct eeh_pe {
* another tree except the currently existing tree of PCI
* buses and PCI devices
*/
-#define EEH_DEV_IRQ_DISABLED (1<<0) /* Interrupt disabled */
+#define EEH_DEV_IRQ_DISABLED (1 << 0) /* Interrupt disabled */
+#define EEH_DEV_REMOVED (1 << 1) /* PCI device removed */
struct eeh_dev {
int mode; /* EEH mode */
@@ -95,6 +96,7 @@ struct eeh_dev {
struct pci_controller *phb; /* Associated PHB */
struct device_node *dn; /* Associated device node */
struct pci_dev *pdev; /* Associated PCI device */
+ struct pci_bus *bus; /* PCI bus used in hotplug */
};
static inline struct device_node *eeh_dev_to_of_node(struct eeh_dev *edev)
diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
index 6653f27..af10ec5 100644
--- a/arch/powerpc/include/asm/pci.h
+++ b/arch/powerpc/include/asm/pci.h
@@ -183,6 +183,7 @@ extern void pci_resource_to_user(const struct pci_dev *dev, int bar,
resource_size_t *start, resource_size_t *end);
extern resource_size_t pcibios_io_space_offset(struct pci_controller *hose);
+extern void pcibios_setup_device(struct pci_dev *dev);
extern void pcibios_setup_bus_devices(struct pci_bus *bus);
extern void pcibios_setup_bus_self(struct pci_bus *bus);
extern void pcibios_setup_phb_io_space(struct pci_controller *hose);
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 2b1ce17..cb3baab 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -244,6 +244,7 @@ static void *eeh_report_reset(void *data, void *userdata)
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
enum pci_ers_result rc, *res = userdata;
struct pci_driver *driver;
+ bool enable_irq = true;
if (!dev) return NULL;
dev->error_state = pci_channel_io_normal;
@@ -251,7 +252,21 @@ static void *eeh_report_reset(void *data, void *userdata)
driver = eeh_pcid_get(dev);
if (!driver) return NULL;
- eeh_enable_irq(dev);
+ /*
+ * For those PCI devices just added, we reloaded its driver
+ * and needn't to enable the interrupt. The driver should
+ * take care of that. Otherwise, complaint raised from IRQ
+ * subsystem.
+ */
+ if (eeh_probe_mode_dev() && (edev->mode & EEH_DEV_REMOVED)) {
+ edev->mode &= ~(EEH_DEV_REMOVED | EEH_DEV_IRQ_DISABLED);
+ edev->bus = NULL;
+ enable_irq = false;
+
+ }
+
+ if (enable_irq)
+ eeh_enable_irq(dev);
if (!driver->err_handler ||
!driver->err_handler->slot_reset) {
@@ -338,6 +353,115 @@ static void *eeh_report_failure(void *data, void *userdata)
return NULL;
}
+static void *eeh_rmv_device(void *data, void *userdata)
+{
+ struct pci_driver *driver;
+ struct eeh_dev *edev = (struct eeh_dev *)data;
+ struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
+ int *removed = (int *)userdata;
+
+ /*
+ * Actually, we should remove the PCI bridges as well.
+ * However, that's lots of complexity to do that,
+ * particularly some of devices under the bridge might
+ * support EEH. So we just care about PCI devices for
+ * simplicity here.
+ */
+ if (!dev || (dev->hdr_type & PCI_HEADER_TYPE_BRIDGE))
+ return NULL;
+ driver = eeh_pcid_get(dev);
+ if (driver && driver->err_handler)
+ return NULL;
+
+ /* If the driver doesn't support EEH, remove it */
+ pr_info("EEH: Removing device %s without EEH support\n",
+ pci_name(dev));
+
+ /* Detach EEH device from PCI device */
+ edev->pdev = NULL;
+ dev->dev.archdata.edev = NULL;
+ pci_dev_put(dev);
+
+ /* Remove and address cache */
+ eeh_addr_cache_rmv_dev(dev);
+ eeh_sysfs_remove_device(dev);
+
+ /* Remove it from PCI subsystem */
+ edev->mode |= EEH_DEV_REMOVED;
+ edev->bus = dev->bus;
+ pci_stop_and_remove_bus_device(dev);
+ (*removed)++;
+
+ return NULL;
+}
+
+static void *eeh_add_device(void *data, void *userdata)
+{
+ struct eeh_dev *edev = (struct eeh_dev *)data;
+ struct pci_dev *dev;
+ struct pci_bus *bus;
+ struct resource *r;
+ int *removed = (int *)userdata;
+ int devfn, i;
+
+ if (!edev || !(edev->mode & EEH_DEV_REMOVED))
+ return NULL;
+ if (*removed <= 0)
+ return edev;
+
+ /*
+ * We don't clear EEH_DEV_REMOVED flag here.
+ * Instead, do that before enabling IRQ to
+ * avoid complain from IRQ subsystem.
+ */
+ *removed -= 1;
+ bus = edev->bus;
+ devfn = edev->config_addr & 0xFF;
+ pr_info("EEH: Adding PCI device %04x:%02x:%02x.%01x\n",
+ edev->phb->global_number, bus->number,
+ PCI_SLOT(devfn), PCI_FUNC(devfn));
+
+ /* Scan PCI function */
+ dev = pci_scan_single_device(bus, devfn);
+ if (!dev) {
+ pr_err("%s: Can't scan PCI function %02x:%02x.%01x\n",
+ __func__, bus->number, PCI_SLOT(devfn),
+ PCI_FUNC(devfn));
+ return NULL;
+ }
+
+ /*
+ * Setup the PCI device. It's not enough to
+ * claim the resource and we need assign or
+ * reassign that.
+ */
+ pcibios_setup_device(dev);
+ for (i = 0; i < PCI_NUM_RESOURCES; i++) {
+ r = &dev->resource[i];
+ if (r->parent || !r->flags)
+ continue;
+ if (pci_assign_resource(dev, i)) {
+ pr_err("%s: Can't allocate %pR for %s\n",
+ __func__, r, pci_name(dev));
+ /* Clear it out */
+ r->start = 0;
+ r->end = 0;
+ r->flags = 0;
+ }
+ }
+
+ /* Associate EEH device with PCI device */
+ pci_dev_get(dev);
+ edev->pdev = dev;
+ dev->dev.archdata.edev = edev;
+ eeh_addr_cache_insert_dev(dev);
+
+ pci_bus_add_devices(bus);
+ eeh_sysfs_add_device(dev);
+
+ return NULL;
+}
+
/**
* eeh_reset_device - Perform actual reset of a pci slot
* @pe: EEH PE
@@ -351,6 +475,7 @@ static int eeh_reset_device(struct eeh_pe *pe, struct pci_bus *bus)
{
struct timeval tstamp;
int cnt, rc;
+ int removed = 0;
/* pcibios will clear the counter; save the value */
cnt = pe->freeze_count;
@@ -364,6 +489,8 @@ static int eeh_reset_device(struct eeh_pe *pe, struct pci_bus *bus)
*/
if (bus)
__pcibios_remove_pci_devices(bus, 0);
+ else if (eeh_probe_mode_dev())
+ eeh_pe_dev_traverse(pe, eeh_rmv_device, &removed);
/* Reset the pci controller. (Asserts RST#; resets config space).
* Reconfigure bridges and devices. Don't try to bring the system
@@ -384,8 +511,13 @@ static int eeh_reset_device(struct eeh_pe *pe, struct pci_bus *bus)
* potentially weird things happen.
*/
if (bus) {
+ pr_info("EEH: Hold for 5 seconds after reset\n");
ssleep(5);
pcibios_add_pci_devices(bus);
+ } else if (eeh_probe_mode_dev() && removed) {
+ pr_info("EEH: Hold for 5 seconds after reset\n");
+ ssleep(5);
+ eeh_pe_dev_traverse(pe, eeh_add_device, &removed);
}
pe->tstamp = tstamp;
--
1.7.5.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox