* Re: [PATCH v5 03/14] staging: typec: tcpci: add compatible string for nxp ptn5110
From: Mats Karrman @ 2018-05-11 19:51 UTC (permalink / raw)
To: Li Jun, robh+dt, gregkh, heikki.krogerus, linux
Cc: a.hajda, cw00.choi, shufan_lee, peter.chen, gsomlo, devicetree,
linux-usb, linux-imx
In-Reply-To: <1525307094-27402-4-git-send-email-jun.li@nxp.com>
Hi Li Jun,
On 2018-05-03 02:24, Li Jun wrote:
> Add nxp ptn5110 typec controller compatible string: usb-tcpci,ptn5110,
> which is a standard tcpci chip with power delivery support.
>
> Signed-off-by: Li Jun <jun.li@nxp.com>
> ---
> drivers/staging/typec/tcpci.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/staging/typec/tcpci.c b/drivers/staging/typec/tcpci.c
> index 076d97e..741a80a 100644
> --- a/drivers/staging/typec/tcpci.c
> +++ b/drivers/staging/typec/tcpci.c
> @@ -576,6 +576,7 @@ MODULE_DEVICE_TABLE(i2c, tcpci_id);
> #ifdef CONFIG_OF
> static const struct of_device_id tcpci_of_match[] = {
> { .compatible = "usb,tcpci", },
Either this line should go away, or a "generic TCPCI controller" line should be
added to the DT documentation.
BR // Mats
> + { .compatible = "nxp,ptn5110", },
> {},
> };
> MODULE_DEVICE_TABLE(of, tcpci_of_match);
>
^ permalink raw reply
* Re: [PATCH v5 01/14] dt-bindings: connector: add properties for typec
From: Mats Karrman @ 2018-05-11 19:49 UTC (permalink / raw)
To: Li Jun, robh+dt, gregkh, heikki.krogerus, linux
Cc: a.hajda, cw00.choi, shufan_lee, peter.chen, gsomlo, devicetree,
linux-usb, linux-imx
In-Reply-To: <1525307094-27402-2-git-send-email-jun.li@nxp.com>
Hi Li Jun,
On 2018-05-03 02:24, Li Jun wrote:
> Add bingdings supported by current typec driver, so user can pass
> all those properties via dt.
>
> Signed-off-by: Li Jun <jun.li@nxp.com>
> ---
> .../bindings/connector/usb-connector.txt | 44 +++++++++++++++
> include/dt-bindings/usb/pd.h | 62 ++++++++++++++++++++++
> 2 files changed, 106 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/connector/usb-connector.txt b/Documentation/devicetree/bindings/connector/usb-connector.txt
> index e1463f1..4b19de6d0 100644
> --- a/Documentation/devicetree/bindings/connector/usb-connector.txt
> +++ b/Documentation/devicetree/bindings/connector/usb-connector.txt
> @@ -15,6 +15,33 @@ Optional properties:
> - type: size of the connector, should be specified in case of USB-A, USB-B
> non-fullsize connectors: "mini", "micro".
>
> +Optional properties for usb-c-connector:
> +- power-role: should be one of "source", "sink" or "dual"(DRP) if typec
> + connector has power support.
> +- try-power-role: preferred power role if "dual"(DRP) can support Try.SNK
> + or Try.SRC, should be "sink" for Try.SNK or "source" for Try.SRC.
> +- data-role: should be one of "host", "device", "dual"(DRD) if typec
> + connector supports USB data.
> +
> +Required properties for usb-c-connector with power delivery support:
> +- source-pdos: An array of u32 with each entry providing supported power
> + source data object(PDO), the detailed bit definitions of PDO can be found
> + in "Universal Serial Bus Power Delivery Specification" chapter 6.4.1.2
> + Source_Capabilities Message, the order of each entry(PDO) should follow
> + the PD spec chapter 6.4.1. Required for power source and power dual role.
> + User can specify the source PDO array via PDO_FIXED/BATT/VAR() defined in
> + dt-bindings/usb/pd.h.
> +- sink-pdos: An array of u32 with each entry providing supported power
> + sink data object(PDO), the detailed bit definitions of PDO can be found
> + in "Universal Serial Bus Power Delivery Specification" chapter 6.4.1.3
> + Sink Capabilities Message, the order of each entry(PDO) should follow
> + the PD spec chapter 6.4.1. Required for power sink and power dual role.
> + User can specify the sink PDO array via PDO_FIXED/BATT/VAR() defined in
> + dt-bindings/usb/pd.h.
> +- op-sink-microwatt: Sink required operating power in microwatt, if source
> + can't offer the power, Capability Mismatch is set, required for power
...set. Required...
(new sentence, otherwise it's unclear what is required; op-sink-microwatt or
Capability Mismatch set)
BR // Mats
> + sink and power dual role.
> +
> Required nodes:
> - any data bus to the connector should be modeled using the OF graph bindings
> specified in bindings/graph.txt, unless the bus is between parent node and
> @@ -73,3 +100,20 @@ ccic: s2mm005@33 {
> };
> };
> };
> +
> +3. USB-C connector attached to a typec port controller(ptn5110), which has
> +power delivery support and enables drp.
> +
> +typec: ptn5110@50 {
> + ...
> + usb_con: connector {
> + compatible = "usb-c-connector";
> + label = "USB-C";
> + power-role = "dual";
> + try-power-role = "sink";
> + source-pdos = <PDO_FIXED(5000, 2000, PDO_FIXED_USB_COMM)>;
> + sink-pdos = <PDO_FIXED(5000, 2000, PDO_FIXED_USB_COMM)
> + PDO_VAR(5000, 12000, 2000)>;
> + op-sink-microwatt = <10000000>;
> + };
> +};
> diff --git a/include/dt-bindings/usb/pd.h b/include/dt-bindings/usb/pd.h
> new file mode 100644
> index 0000000..7b7a92f
> --- /dev/null
> +++ b/include/dt-bindings/usb/pd.h
> @@ -0,0 +1,62 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __DT_POWER_DELIVERY_H
> +#define __DT_POWER_DELIVERY_H
> +
> +/* Power delivery Power Data Object definitions */
> +#define PDO_TYPE_FIXED 0
> +#define PDO_TYPE_BATT 1
> +#define PDO_TYPE_VAR 2
> +#define PDO_TYPE_APDO 3
> +
> +#define PDO_TYPE_SHIFT 30
> +#define PDO_TYPE_MASK 0x3
> +
> +#define PDO_TYPE(t) ((t) << PDO_TYPE_SHIFT)
> +
> +#define PDO_VOLT_MASK 0x3ff
> +#define PDO_CURR_MASK 0x3ff
> +#define PDO_PWR_MASK 0x3ff
> +
> +#define PDO_FIXED_DUAL_ROLE (1 << 29) /* Power role swap supported */
> +#define PDO_FIXED_SUSPEND (1 << 28) /* USB Suspend supported (Source) */
> +#define PDO_FIXED_HIGHER_CAP (1 << 28) /* Requires more than vSafe5V (Sink) */
> +#define PDO_FIXED_EXTPOWER (1 << 27) /* Externally powered */
> +#define PDO_FIXED_USB_COMM (1 << 26) /* USB communications capable */
> +#define PDO_FIXED_DATA_SWAP (1 << 25) /* Data role swap supported */
> +#define PDO_FIXED_VOLT_SHIFT 10 /* 50mV units */
> +#define PDO_FIXED_CURR_SHIFT 0 /* 10mA units */
> +
> +#define PDO_FIXED_VOLT(mv) ((((mv) / 50) & PDO_VOLT_MASK) << PDO_FIXED_VOLT_SHIFT)
> +#define PDO_FIXED_CURR(ma) ((((ma) / 10) & PDO_CURR_MASK) << PDO_FIXED_CURR_SHIFT)
> +
> +#define PDO_FIXED(mv, ma, flags) \
> + (PDO_TYPE(PDO_TYPE_FIXED) | (flags) | \
> + PDO_FIXED_VOLT(mv) | PDO_FIXED_CURR(ma))
> +
> +#define VSAFE5V 5000 /* mv units */
> +
> +#define PDO_BATT_MAX_VOLT_SHIFT 20 /* 50mV units */
> +#define PDO_BATT_MIN_VOLT_SHIFT 10 /* 50mV units */
> +#define PDO_BATT_MAX_PWR_SHIFT 0 /* 250mW units */
> +
> +#define PDO_BATT_MIN_VOLT(mv) ((((mv) / 50) & PDO_VOLT_MASK) << PDO_BATT_MIN_VOLT_SHIFT)
> +#define PDO_BATT_MAX_VOLT(mv) ((((mv) / 50) & PDO_VOLT_MASK) << PDO_BATT_MAX_VOLT_SHIFT)
> +#define PDO_BATT_MAX_POWER(mw) ((((mw) / 250) & PDO_PWR_MASK) << PDO_BATT_MAX_PWR_SHIFT)
> +
> +#define PDO_BATT(min_mv, max_mv, max_mw) \
> + (PDO_TYPE(PDO_TYPE_BATT) | PDO_BATT_MIN_VOLT(min_mv) | \
> + PDO_BATT_MAX_VOLT(max_mv) | PDO_BATT_MAX_POWER(max_mw))
> +
> +#define PDO_VAR_MAX_VOLT_SHIFT 20 /* 50mV units */
> +#define PDO_VAR_MIN_VOLT_SHIFT 10 /* 50mV units */
> +#define PDO_VAR_MAX_CURR_SHIFT 0 /* 10mA units */
> +
> +#define PDO_VAR_MIN_VOLT(mv) ((((mv) / 50) & PDO_VOLT_MASK) << PDO_VAR_MIN_VOLT_SHIFT)
> +#define PDO_VAR_MAX_VOLT(mv) ((((mv) / 50) & PDO_VOLT_MASK) << PDO_VAR_MAX_VOLT_SHIFT)
> +#define PDO_VAR_MAX_CURR(ma) ((((ma) / 10) & PDO_CURR_MASK) << PDO_VAR_MAX_CURR_SHIFT)
> +
> +#define PDO_VAR(min_mv, max_mv, max_ma) \
> + (PDO_TYPE(PDO_TYPE_VAR) | PDO_VAR_MIN_VOLT(min_mv) | \
> + PDO_VAR_MAX_VOLT(max_mv) | PDO_VAR_MAX_CURR(max_ma))
> +
> + #endif /* __DT_POWER_DELIVERY_H */
^ permalink raw reply
* Re: [PATCH v3 1/2] vfio: platform: Fix reset module leak in error path
From: Alex Williamson @ 2018-05-11 19:45 UTC (permalink / raw)
To: Geert Uytterhoeven
Cc: Baptiste Reynal, Philipp Zabel, Rob Herring, Mark Rutland, kvm,
devicetree, linux-renesas-soc, linux-kernel
In-Reply-To: <1523438149-16433-3-git-send-email-geert+renesas@glider.be>
On Wed, 11 Apr 2018 11:15:48 +0200
Geert Uytterhoeven <geert+renesas@glider.be> wrote:
> If the IOMMU group setup fails, the reset module is not released.
>
> Fixes: b5add544d677d363 ("vfio, platform: make reset driver a requirement by default")
> Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Simon Horman <horms+renesas@verge.net.au>
> ---
> v3:
> - Add Reviewed-by,
>
> v2:
> - Add Reviewed-by.
> ---
> drivers/vfio/platform/vfio_platform_common.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
Applied to next branch for v4.18 with Eric's ack. Thanks,
Alex
^ permalink raw reply
* [linux-next PATCH 4/4] ARM: keystone: k2g: enable micrel and dp83867 phys
From: Murali Karicheri @ 2018-05-11 19:29 UTC (permalink / raw)
To: ssantosh, robh+dt, mark.rutland, linux, linux-arm-kernel,
devicetree, linux-kernel
In-Reply-To: <1526066952-5230-1-git-send-email-m-karicheri2@ti.com>
Enable micrel and dp83867 phys for K2G Boards in keystone_defconfig
to support Network driver on these boards (K2G ICE and GP EVM).
Signed-off-by: Murali Karicheri <m-karicheri2@ti.com>
---
arch/arm/configs/keystone_defconfig | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm/configs/keystone_defconfig b/arch/arm/configs/keystone_defconfig
index 2536c23..85cf2fc 100644
--- a/arch/arm/configs/keystone_defconfig
+++ b/arch/arm/configs/keystone_defconfig
@@ -235,3 +235,5 @@ CONFIG_INPUT_GPIO_DECODER=m
CONFIG_GPIO_PCA953X=m
CONFIG_LEDS_TRIGGER_ACTIVITY=y
CONFIG_LEDS_TRIGGER_CPU=y
+CONFIG_MICREL_PHY=y
+CONFIG_DP83867_PHY=y
--
1.9.1
^ permalink raw reply related
* [linux-next PATCH 3/4] ARM: dts: keystone-k2g-ice: Enable netcp network driver
From: Murali Karicheri @ 2018-05-11 19:29 UTC (permalink / raw)
To: ssantosh, robh+dt, mark.rutland, linux, linux-arm-kernel,
devicetree, linux-kernel
In-Reply-To: <1526066952-5230-1-git-send-email-m-karicheri2@ti.com>
This patch adds dt bindings to enable netcp network driver on K2G
ICE boards. This consists of enabling bindings for NSS qmss, pktdma,
2u ethss, mdio, pinmux and netcp devices as well as DP83867 phy.
EVM hardware spec recommends to add 0.25 nsec delay in the tx direction
and 2.25 nsec delay in the rx direction for internal delay in the clock
path to be on the safer side.
The board straps RX_DV/RX_CTRL pin of on board DP83867 phy in mode 1.
The phy data manual disallows this. Add ti,dp83867-rxctrl-strap-quirk
in the phy node to allow software to enable the workaround suggested for
this incorrect strap setting. This ensures proper operation of this PHY.
Signed-off-by: Murali Karicheri <m-karicheri2@ti.com>
---
arch/arm/boot/dts/keystone-k2g-ice.dts | 59 ++++++++++++++++++++++++++++++++++
1 file changed, 59 insertions(+)
diff --git a/arch/arm/boot/dts/keystone-k2g-ice.dts b/arch/arm/boot/dts/keystone-k2g-ice.dts
index d820ed2..2a2d38c 100644
--- a/arch/arm/boot/dts/keystone-k2g-ice.dts
+++ b/arch/arm/boot/dts/keystone-k2g-ice.dts
@@ -7,6 +7,7 @@
/dts-v1/;
#include "keystone-k2g.dtsi"
+#include <dt-bindings/net/ti-dp83867.h>
/ {
compatible = "ti,k2g-ice", "ti,k2g", "ti,keystone";
@@ -281,6 +282,30 @@
K2G_CORE_IOPAD(0x11bc) (BUFFER_CLASS_B | PIN_PULLUP | MUX_MODE3) /* spi2_scsn1.gpio0_102 */
>;
};
+
+ emac_pins: pinmux_emac_pins {
+ pinctrl-single,pins = <
+ K2G_CORE_IOPAD(0x113C) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD1.RGMII_RXD1 */
+ K2G_CORE_IOPAD(0x1138) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD2.RGMII_RXD2 */
+ K2G_CORE_IOPAD(0x1134) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD3.RGMII_RXD3 */
+ K2G_CORE_IOPAD(0x1140) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD0.RGMII_RXD0 */
+ K2G_CORE_IOPAD(0x1178) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD0.RGMII_TXD0 */
+ K2G_CORE_IOPAD(0x1174) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD1.RGMII_TXD1 */
+ K2G_CORE_IOPAD(0x1170) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD2.RGMII_TXD2 */
+ K2G_CORE_IOPAD(0x116C) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD3.RGMII_TXD3 */
+ K2G_CORE_IOPAD(0x1154) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXCLK.RGMII_TXC */
+ K2G_CORE_IOPAD(0x117C) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXEN.RGMII_TXCTL */
+ K2G_CORE_IOPAD(0x1120) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXCLK.RGMII_RXC */
+ K2G_CORE_IOPAD(0x1144) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXDV.RGMII_RXCTL */
+ >;
+ };
+
+ mdio_pins: pinmux_mdio_pins {
+ pinctrl-single,pins = <
+ K2G_CORE_IOPAD(0x118C) (BUFFER_CLASS_B | PULL_DISABLE | MUX_MODE0) /* MDIO_CLK.MDIO_CLK */
+ K2G_CORE_IOPAD(0x1188) (BUFFER_CLASS_B | PULL_DISABLE | MUX_MODE0) /* MDIO_DATA.MDIO_DATA */
+ >;
+ };
};
&uart0 {
@@ -386,3 +411,37 @@
vcc-supply = <&vdd_3v3>;
};
};
+
+&qmss {
+ status = "okay";
+};
+
+&knav_dmas {
+ status = "okay";
+};
+
+&netcp {
+ pinctrl-names = "default";
+ pinctrl-0 = <&emac_pins>;
+ status = "okay";
+};
+
+&mdio {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mdio_pins>;
+ status = "okay";
+ ethphy0: ethernet-phy@0 {
+ reg = <0>;
+ ti,rx-internal-delay = <DP83867_RGMIIDCTL_2_25_NS>;
+ ti,tx-internal-delay = <DP83867_RGMIIDCTL_250_PS>;
+ ti,fifo-depth = <DP83867_PHYCR_FIFO_DEPTH_8_B_NIB>;
+ ti,min-output-impedance;
+ ti,dp83867-rxctrl-strap-quirk;
+ };
+};
+
+&gbe0 {
+ phy-handle = <ðphy0>;
+ phy-mode = "rgmii-id";
+ status = "okay";
+};
--
1.9.1
^ permalink raw reply related
* [linux-next PATCH 2/4] ARM: dts: keystone-k2g-evm: Enable netcp network driver
From: Murali Karicheri @ 2018-05-11 19:29 UTC (permalink / raw)
To: ssantosh, robh+dt, mark.rutland, linux, linux-arm-kernel,
devicetree, linux-kernel
In-Reply-To: <1526066952-5230-1-git-send-email-m-karicheri2@ti.com>
Add dt bindings to enable netcp network driver on K2G GP EVM. This
consists of enabling bindings for nss qmss, pktdma, 2u ethss,
mdio, pinmux and netcp devices.
Signed-off-by: Murali Karicheri <m-karicheri2@ti.com>
---
arch/arm/boot/dts/keystone-k2g-evm.dts | 53 ++++++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/arch/arm/boot/dts/keystone-k2g-evm.dts b/arch/arm/boot/dts/keystone-k2g-evm.dts
index 154fdd7..f028a00 100644
--- a/arch/arm/boot/dts/keystone-k2g-evm.dts
+++ b/arch/arm/boot/dts/keystone-k2g-evm.dts
@@ -128,6 +128,30 @@
K2G_CORE_IOPAD(0x1228) (BUFFER_CLASS_B | PIN_PULLDOWN | MUX_MODE1) /* qspicsn3.dcan1rx */
>;
};
+
+ emac_pins: pinmux_emac_pins {
+ pinctrl-single,pins = <
+ K2G_CORE_IOPAD(0x113C) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD1.RGMII_RXD1 */
+ K2G_CORE_IOPAD(0x1138) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD2.RGMII_RXD2 */
+ K2G_CORE_IOPAD(0x1134) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD3.RGMII_RXD3 */
+ K2G_CORE_IOPAD(0x1140) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXD0.RGMII_RXD0 */
+ K2G_CORE_IOPAD(0x1178) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD0.RGMII_TXD0 */
+ K2G_CORE_IOPAD(0x1174) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD1.RGMII_TXD1 */
+ K2G_CORE_IOPAD(0x1170) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD2.RGMII_TXD2 */
+ K2G_CORE_IOPAD(0x116C) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXD3.RGMII_TXD3 */
+ K2G_CORE_IOPAD(0x1154) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXCLK.RGMII_TXC */
+ K2G_CORE_IOPAD(0x117C) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_TXEN.RGMII_TXCTL */
+ K2G_CORE_IOPAD(0x1120) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXCLK.RGMII_RXC */
+ K2G_CORE_IOPAD(0x1144) (BUFFER_CLASS_D | PULL_DISABLE | MUX_MODE1) /* MII_RXDV.RGMII_RXCTL */
+ >;
+ };
+
+ mdio_pins: pinmux_mdio_pins {
+ pinctrl-single,pins = <
+ K2G_CORE_IOPAD(0x118C) (BUFFER_CLASS_B | PULL_DISABLE | MUX_MODE0) /* MDIO_CLK.MDIO_CLK */
+ K2G_CORE_IOPAD(0x1188) (BUFFER_CLASS_B | PULL_DISABLE | MUX_MODE0) /* MDIO_DATA.MDIO_DATA */
+ >;
+ };
};
&uart0 {
@@ -294,3 +318,32 @@
pinctrl-0 = <&dcan1_pins>;
status = "okay";
};
+
+&qmss {
+ status = "okay";
+};
+
+&knav_dmas {
+ status = "okay";
+};
+
+&mdio {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mdio_pins>;
+ status = "okay";
+ ethphy0: ethernet-phy@0 {
+ reg = <0>;
+ };
+};
+
+&gbe0 {
+ phy-handle = <ðphy0>;
+ phy-mode = "rgmii-id";
+ status = "okay";
+};
+
+&netcp {
+ pinctrl-names = "default";
+ pinctrl-0 = <&emac_pins>;
+ status = "okay";
+};
--
1.9.1
^ permalink raw reply related
* [linux-next PATCH 1/4] ARM: dts: k2g: add dt bindings to support network driver
From: Murali Karicheri @ 2018-05-11 19:29 UTC (permalink / raw)
To: ssantosh, robh+dt, mark.rutland, linux, linux-arm-kernel,
devicetree, linux-kernel
In-Reply-To: <1526066952-5230-1-git-send-email-m-karicheri2@ti.com>
This patch add dt bindings to support network driver based on
Network Sub System (NSS) found on k2g SoC. This consists of
bindings for netcp node, nss qmss , pktdma, cpsw 2u version
of ethss and mdio.
In order to support transitioning between non-promiscuous
and promiscuous modes in K2G's ethernet switch, netcp ALE
needs to be enabled. Therefore this patch enables K2G's
netcp ALE for that purpose.
Signed-off-by: Murali Karicheri <m-karicheri2@ti.com>
Signed-off-by: WingMan Kwok <w-kwok2@ti.com>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
arch/arm/boot/dts/keystone-k2g-netcp.dtsi | 147 ++++++++++++++++++++++++++++++
arch/arm/boot/dts/keystone-k2g.dtsi | 13 +++
2 files changed, 160 insertions(+)
create mode 100644 arch/arm/boot/dts/keystone-k2g-netcp.dtsi
diff --git a/arch/arm/boot/dts/keystone-k2g-netcp.dtsi b/arch/arm/boot/dts/keystone-k2g-netcp.dtsi
new file mode 100644
index 0000000..d0e6a9a
--- /dev/null
+++ b/arch/arm/boot/dts/keystone-k2g-netcp.dtsi
@@ -0,0 +1,147 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Device Tree Source for K2G Netcp driver
+ *
+ * Copyright (C) 2018 Texas Instruments Incorporated - http://www.ti.com/
+ */
+
+qmss: qmss@4020000 {
+ compatible = "ti,66ak2g-navss-qm";
+ dma-coherent;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ power-domains = <&k2g_pds 0x0018>;
+ clocks = <&k2g_clks 0x0018 0>;
+ clock-names = "nss_vclk";
+ ranges;
+ queue-range = <0 0x80>;
+ linkram0 = <0x4020000 0x7ff>;
+ status = "disabled";
+
+ qmgrs {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ qmgr0 {
+ managed-queues = <0 0x80>;
+ reg = <0x4100000 0x800>,
+ <0x4040000 0x100>,
+ <0x4080000 0x800>,
+ <0x40c0000 0x800>;
+ reg-names = "peek", "config",
+ "region", "push";
+ };
+
+ };
+ queue-pools {
+ qpend {
+ qpend-0 {
+ qrange = <77 8>;
+ interrupts =<0 308 0xf04 0 309 0xf04 0 310 0xf04
+ 0 311 0xf04 0 312 0xf04 0 313 0xf04
+ 0 314 0xf04 0 315 0xf04>;
+ qalloc-by-id;
+ };
+ };
+ general-purpose {
+ gp-0 {
+ qrange = <112 8>;
+ };
+ netcp-tx {
+ qrange = <5 8>;
+ qalloc-by-id;
+ };
+ };
+ };
+
+ descriptor-regions {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ region-12 {
+ id = <12>;
+ region-spec = <1023 128>; /* num_desc desc_size */
+ link-index = <0x400>;
+ };
+ };
+}; /* qmss */
+
+knav_dmas: knav_dmas@0 {
+ compatible = "ti,keystone-navigator-dma";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ status = "disabled";
+ power-domains = <&k2g_pds 0x0018>;
+ clocks = <&k2g_clks 0x0018 0>;
+ clock-names = "nss_vclk";
+ ranges;
+ ti,navigator-cloud-address = <0x40c0000 0x40c0000 0x40c0000 0x40c0000>;
+
+ dma_gbe: dma_gbe@0 {
+ reg = <0x4010000 0x100>,
+ <0x4011000 0x2a0>, /* 21 Tx channels */
+ <0x4012000 0x400>, /* 32 Rx channels */
+ <0x4010100 0x80>,
+ <0x4013000 0x400>; /* 32 Rx flows */
+ reg-names = "global", "txchan", "rxchan",
+ "txsched", "rxflow";
+ };
+
+};
+
+netcp: netcp@4000000 {
+ reg = <0x2620110 0x8>;
+ reg-names = "efuse";
+ compatible = "ti,netcp-1.0";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ status = "disabled";
+ power-domains = <&k2g_pds 0x0018>;
+ clocks = <&k2g_clks 0x0018 3>, <&k2g_clks 0x0018 8>;
+ clock-names = "ethss_clk", "cpts";
+
+ /* NetCP address range */
+ ranges = <0 0x4000000 0x1000000>;
+
+ dma-coherent;
+
+ ti,navigator-dmas = <&dma_gbe 0>, <&dma_gbe 5>;
+ ti,navigator-dma-names = "netrx0", "nettx";
+
+ netcp-devices {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ gbe: gbe@200000 {
+ label = "netcp-gbe";
+ compatible = "ti,netcp-gbe-2";
+ reg = <0x200000 0x20>, <0x220000 0x20000>;
+ enable-ale;
+ tx-queue = <5>;
+ tx-channel = "nettx";
+ cpts-rftclk-sel = <0>;
+ cpts-ext-ts-inputs = <8>;
+
+ interfaces {
+ gbe0: interface-0 {
+ slave-port = <0>;
+ link-interface = <5>;
+ };
+ };
+ };
+ };
+
+ netcp-interfaces {
+ interface-0 {
+ rx-channel = "netrx0";
+ rx-pool = <512 12>;
+ tx-pool = <511 12>;
+ rx-queue-depth = <128 128 0 0>;
+ rx-buffer-size = <1518 4096 0 0>;
+ rx-queue = <77>;
+ tx-completion-queue = <78>;
+ efuse-mac = <1>;
+ netcp-gbe = <&gbe0>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/keystone-k2g.dtsi b/arch/arm/boot/dts/keystone-k2g.dtsi
index da78c00..a8324a7 100644
--- a/arch/arm/boot/dts/keystone-k2g.dtsi
+++ b/arch/arm/boot/dts/keystone-k2g.dtsi
@@ -609,5 +609,18 @@
reg = <0x21010000 0x200>;
interrupts = <GIC_SPI 123 IRQ_TYPE_EDGE_RISING>;
};
+
+ mdio: mdio@4200f00 {
+ compatible = "ti,keystone_mdio", "ti,davinci_mdio";
+ reg = <0x04200f00 0x100>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&k2g_clks 0x0018 3>;
+ clock-names = "fck";
+ power-domains = <&k2g_pds 0x0018>;
+ status = "disabled";
+ bus_freq = <2500000>;
+ };
+ #include "keystone-k2g-netcp.dtsi"
};
};
--
1.9.1
^ permalink raw reply related
* [linux-next PATCH 0/4] Enable network driver on K2G ICE and GP EVMs
From: Murali Karicheri @ 2018-05-11 19:29 UTC (permalink / raw)
To: ssantosh, robh+dt, mark.rutland, linux, linux-arm-kernel,
devicetree, linux-kernel
Now that NetCP driver patches for K2G SoC is merged to linux-next master
this series add patches to enable network driver on K2G ICE and GP EVMs.
Thanks
Applied the patches on top of latest linux-next master, built kernel and
booted up on both EVMs. The logs are below
K2G GP EVM: https://pastebin.ubuntu.com/p/ycZDnZXYPx/
K2G ICE EVM: https://pastebin.ubuntu.com/p/bdCpzgdrXr/
Murali Karicheri (4):
ARM: dts: k2g: add dt bindings to support network driver
ARM: dts: keystone-k2g-evm: Enable netcp network driver
ARM: dts: keystone-k2g-ice: Enable netcp network driver
ARM: keystone: k2g: enable micrel and dp83867 phys
arch/arm/boot/dts/keystone-k2g-evm.dts | 53 +++++++++++
arch/arm/boot/dts/keystone-k2g-ice.dts | 59 ++++++++++++
arch/arm/boot/dts/keystone-k2g-netcp.dtsi | 147 ++++++++++++++++++++++++++++++
arch/arm/boot/dts/keystone-k2g.dtsi | 13 +++
arch/arm/configs/keystone_defconfig | 2 +
5 files changed, 274 insertions(+)
create mode 100644 arch/arm/boot/dts/keystone-k2g-netcp.dtsi
--
1.9.1
^ permalink raw reply
* [PATCH v2 40/40] iommu/arm-smmu-v3: Add support for PCI PASID
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
Enable PASID for PCI devices that support it. Unlike PRI, we can't enable
PASID lazily in iommu_sva_device_init(), because it has to be enabled
before ATS, and because we have to allocate substream tables early.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3.c | 54 +++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 0edbb8d19579..ac6e69f25893 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2542,6 +2542,52 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
return sid < limit;
}
+static int arm_smmu_enable_pasid(struct arm_smmu_master_data *master)
+{
+ int ret;
+ int features;
+ u8 pasid_bits;
+ int num_pasids;
+ struct pci_dev *pdev;
+
+ if (!dev_is_pci(master->dev))
+ return -ENOSYS;
+
+ pdev = to_pci_dev(master->dev);
+
+ features = pci_pasid_features(pdev);
+ if (features < 0)
+ return -ENOSYS;
+
+ num_pasids = pci_max_pasids(pdev);
+ if (num_pasids <= 0)
+ return -ENOSYS;
+
+ pasid_bits = min_t(u8, ilog2(num_pasids), master->smmu->ssid_bits);
+
+ dev_dbg(&pdev->dev, "device supports %#x PASID bits [%s%s]\n", pasid_bits,
+ (features & PCI_PASID_CAP_EXEC) ? "x" : "",
+ (features & PCI_PASID_CAP_PRIV) ? "p" : "");
+
+ ret = pci_enable_pasid(pdev, features);
+ return ret ? ret : pasid_bits;
+}
+
+static void arm_smmu_disable_pasid(struct arm_smmu_master_data *master)
+{
+ struct pci_dev *pdev;
+
+ if (!dev_is_pci(master->dev))
+ return;
+
+ pdev = to_pci_dev(master->dev);
+
+ if (!pdev->pasid_enabled)
+ return;
+
+ pci_disable_pasid(pdev);
+}
+
static int arm_smmu_enable_ats(struct arm_smmu_master_data *master)
{
size_t stu;
@@ -2712,6 +2758,11 @@ static int arm_smmu_add_device(struct device *dev)
master->ste.can_stall = true;
}
+ /* PASID must be enabled before ATS */
+ ret = arm_smmu_enable_pasid(master);
+ if (ret > 0)
+ master->ssid_bits = ret;
+
arm_smmu_enable_ats(master);
ret = iommu_device_link(&smmu->iommu, dev);
@@ -2740,6 +2791,7 @@ static int arm_smmu_add_device(struct device *dev)
err_disable_ats:
arm_smmu_disable_ats(master);
+ arm_smmu_disable_pasid(master);
err_free_master:
kfree(master);
@@ -2769,7 +2821,9 @@ static void arm_smmu_remove_device(struct device *dev)
arm_smmu_remove_master(smmu, master);
iommu_device_unlink(&smmu->iommu, dev);
arm_smmu_disable_pri(master);
+ /* PASID must be disabled after ATS */
arm_smmu_disable_ats(master);
+ arm_smmu_disable_pasid(master);
kfree(master);
iommu_fwspec_free(dev);
}
--
2.17.0
^ permalink raw reply related
* [PATCH v2 39/40] iommu/arm-smmu-v3: Add support for PRI
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
For PCI devices that support it, enable the PRI capability and handle PRI
Page Requests with the generic fault handler. It is enabled on demand by
iommu_sva_device_init().
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
v1->v2:
* Terminate the page request and disable PRI if no handler is registered
* Enable and disable PRI in sva_device_init/shutdown, instead of
add/remove_device
---
drivers/iommu/arm-smmu-v3.c | 192 +++++++++++++++++++++++++++---------
1 file changed, 145 insertions(+), 47 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 6cb69ace371b..0edbb8d19579 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -248,6 +248,7 @@
#define STRTAB_STE_1_S1COR GENMASK_ULL(5, 4)
#define STRTAB_STE_1_S1CSH GENMASK_ULL(7, 6)
+#define STRTAB_STE_1_PPAR (1UL << 18)
#define STRTAB_STE_1_S1STALLD (1UL << 27)
#define STRTAB_STE_1_EATS GENMASK_ULL(29, 28)
@@ -309,6 +310,9 @@
#define CMDQ_PRI_0_SID GENMASK_ULL(63, 32)
#define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0)
#define CMDQ_PRI_1_RESP GENMASK_ULL(13, 12)
+#define CMDQ_PRI_1_RESP_FAILURE FIELD_PREP(CMDQ_PRI_1_RESP, 0UL)
+#define CMDQ_PRI_1_RESP_INVALID FIELD_PREP(CMDQ_PRI_1_RESP, 1UL)
+#define CMDQ_PRI_1_RESP_SUCCESS FIELD_PREP(CMDQ_PRI_1_RESP, 2UL)
#define CMDQ_RESUME_0_SID GENMASK_ULL(63, 32)
#define CMDQ_RESUME_0_ACTION_RETRY (1UL << 12)
@@ -383,12 +387,6 @@ module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO);
MODULE_PARM_DESC(disable_ats_check,
"By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check.");
-enum pri_resp {
- PRI_RESP_DENY = 0,
- PRI_RESP_FAIL = 1,
- PRI_RESP_SUCC = 2,
-};
-
enum arm_smmu_msi_index {
EVTQ_MSI_INDEX,
GERROR_MSI_INDEX,
@@ -471,7 +469,7 @@ struct arm_smmu_cmdq_ent {
u32 sid;
u32 ssid;
u16 grpid;
- enum pri_resp resp;
+ enum page_response_code resp;
} pri;
#define CMDQ_OP_RESUME 0x44
@@ -556,6 +554,7 @@ struct arm_smmu_strtab_ent {
struct arm_smmu_s2_cfg *s2_cfg;
bool can_stall;
+ bool prg_resp_needs_ssid;
};
struct arm_smmu_strtab_cfg {
@@ -907,14 +906,18 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent)
cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SID, ent->pri.sid);
cmd[1] |= FIELD_PREP(CMDQ_PRI_1_GRPID, ent->pri.grpid);
switch (ent->pri.resp) {
- case PRI_RESP_DENY:
- case PRI_RESP_FAIL:
- case PRI_RESP_SUCC:
+ case IOMMU_PAGE_RESP_FAILURE:
+ cmd[1] |= CMDQ_PRI_1_RESP_FAILURE;
+ break;
+ case IOMMU_PAGE_RESP_INVALID:
+ cmd[1] |= CMDQ_PRI_1_RESP_INVALID;
+ break;
+ case IOMMU_PAGE_RESP_SUCCESS:
+ cmd[1] |= CMDQ_PRI_1_RESP_SUCCESS;
break;
default:
return -EINVAL;
}
- cmd[1] |= FIELD_PREP(CMDQ_PRI_1_RESP, ent->pri.resp);
break;
case CMDQ_OP_RESUME:
cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_SID, ent->resume.sid);
@@ -1114,8 +1117,15 @@ static int arm_smmu_page_response(struct device *dev,
cmd.resume.sid = sid;
cmd.resume.stag = resp->page_req_group_id;
cmd.resume.resp = resp->resp_code;
+ } else if (master->can_fault) {
+ cmd.opcode = CMDQ_OP_PRI_RESP;
+ cmd.substream_valid = resp->pasid_present &&
+ master->ste.prg_resp_needs_ssid;
+ cmd.pri.sid = sid;
+ cmd.pri.ssid = resp->pasid;
+ cmd.pri.grpid = resp->page_req_group_id;
+ cmd.pri.resp = resp->resp_code;
} else {
- /* TODO: put PRI response here */
return -ENODEV;
}
@@ -1236,6 +1246,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid,
FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
FIELD_PREP(STRTAB_STE_1_STRW, strw));
+ if (ste->prg_resp_needs_ssid)
+ dst[1] |= STRTAB_STE_1_PPAR;
+
if (smmu->features & ARM_SMMU_FEAT_STALLS &&
!(smmu->features & ARM_SMMU_FEAT_STALL_FORCE) &&
!ste->can_stall)
@@ -1471,39 +1484,54 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
static void arm_smmu_handle_ppr(struct arm_smmu_device *smmu, u64 *evt)
{
- u32 sid, ssid;
- u16 grpid;
- bool ssv, last;
-
- sid = FIELD_GET(PRIQ_0_SID, evt[0]);
- ssv = FIELD_GET(PRIQ_0_SSID_V, evt[0]);
- ssid = ssv ? FIELD_GET(PRIQ_0_SSID, evt[0]) : 0;
- last = FIELD_GET(PRIQ_0_PRG_LAST, evt[0]);
- grpid = FIELD_GET(PRIQ_1_PRG_IDX, evt[1]);
-
- dev_info(smmu->dev, "unexpected PRI request received:\n");
- dev_info(smmu->dev,
- "\tsid 0x%08x.0x%05x: [%u%s] %sprivileged %s%s%s access at iova 0x%016llx\n",
- sid, ssid, grpid, last ? "L" : "",
- evt[0] & PRIQ_0_PERM_PRIV ? "" : "un",
- evt[0] & PRIQ_0_PERM_READ ? "R" : "",
- evt[0] & PRIQ_0_PERM_WRITE ? "W" : "",
- evt[0] & PRIQ_0_PERM_EXEC ? "X" : "",
- evt[1] & PRIQ_1_ADDR_MASK);
-
- if (last) {
- struct arm_smmu_cmdq_ent cmd = {
- .opcode = CMDQ_OP_PRI_RESP,
- .substream_valid = ssv,
- .pri = {
- .sid = sid,
- .ssid = ssid,
- .grpid = grpid,
- .resp = PRI_RESP_DENY,
- },
+ u32 sid = FIELD_PREP(PRIQ_0_SID, evt[0]);
+
+ struct arm_smmu_master_data *master;
+ struct iommu_fault_event fault = {
+ .type = IOMMU_FAULT_PAGE_REQ,
+ .last_req = FIELD_GET(PRIQ_0_PRG_LAST, evt[0]),
+ .pasid_valid = FIELD_GET(PRIQ_0_SSID_V, evt[0]),
+ .pasid = FIELD_GET(PRIQ_0_SSID, evt[0]),
+ .page_req_group_id = FIELD_GET(PRIQ_1_PRG_IDX, evt[1]),
+ .addr = evt[1] & PRIQ_1_ADDR_MASK,
+ };
+
+ if (evt[0] & PRIQ_0_PERM_READ)
+ fault.prot |= IOMMU_FAULT_READ;
+ if (evt[0] & PRIQ_0_PERM_WRITE)
+ fault.prot |= IOMMU_FAULT_WRITE;
+ if (evt[0] & PRIQ_0_PERM_EXEC)
+ fault.prot |= IOMMU_FAULT_EXEC;
+ if (evt[0] & PRIQ_0_PERM_PRIV)
+ fault.prot |= IOMMU_FAULT_PRIV;
+
+ /* Discard Stop PASID marker, it isn't used */
+ if (!(fault.prot & (IOMMU_FAULT_READ|IOMMU_FAULT_WRITE)) &&
+ fault.last_req)
+ return;
+
+ master = arm_smmu_find_master(smmu, sid);
+ if (WARN_ON(!master))
+ return;
+
+ if (iommu_report_device_fault(master->dev, &fault)) {
+ /*
+ * No handler registered, so subsequent faults won't produce
+ * better results. Try to disable PRI.
+ */
+ struct page_response_msg page_response = {
+ .addr = fault.addr,
+ .pasid = fault.pasid,
+ .pasid_present = fault.pasid_valid,
+ .page_req_group_id = fault.page_req_group_id,
+ .resp_code = IOMMU_PAGE_RESP_FAILURE,
};
- arm_smmu_cmdq_issue_cmd(smmu, &cmd);
+ dev_warn(master->dev,
+ "PPR 0x%x:0x%llx 0x%x: nobody cared, disabling PRI\n",
+ fault.pasid_valid ? fault.pasid : 0, fault.addr,
+ fault.prot);
+ arm_smmu_page_response(master->dev, &page_response);
}
}
@@ -1529,6 +1557,11 @@ static irqreturn_t arm_smmu_priq_thread(int irq, void *dev)
}
if (queue_sync_prod(q) == -EOVERFLOW)
+ /*
+ * TODO: flush pending faults, since the SMMU might have
+ * auto-responded to the Last request of a pending
+ * group
+ */
dev_err(smmu->dev, "PRIQ overflow detected -- requests lost\n");
} while (!queue_empty(q));
@@ -1577,7 +1610,8 @@ static int arm_smmu_flush_queues(void *cookie, struct device *dev)
master = dev->iommu_fwspec->iommu_priv;
if (master->ste.can_stall)
arm_smmu_flush_queue(smmu, &smmu->evtq.q, "evtq");
- /* TODO: add support for PRI */
+ else if (master->can_fault)
+ arm_smmu_flush_queue(smmu, &smmu->priq.q, "priq");
return 0;
}
@@ -2301,6 +2335,59 @@ arm_smmu_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova)
return ops->iova_to_phys(ops, iova);
}
+static int arm_smmu_enable_pri(struct arm_smmu_master_data *master)
+{
+ int ret, pos;
+ struct pci_dev *pdev;
+ /*
+ * TODO: find a good inflight PPR number. We should divide the PRI queue
+ * by the number of PRI-capable devices, but it's impossible to know
+ * about current and future (hotplugged) devices. So we're at risk of
+ * dropping PPRs (and leaking pending requests in the FQ).
+ */
+ size_t max_inflight_pprs = 16;
+ struct arm_smmu_device *smmu = master->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_PRI) || !dev_is_pci(master->dev))
+ return -ENOSYS;
+
+ pdev = to_pci_dev(master->dev);
+
+ ret = pci_reset_pri(pdev);
+ if (ret)
+ return ret;
+
+ ret = pci_enable_pri(pdev, max_inflight_pprs);
+ if (ret) {
+ dev_err(master->dev, "cannot enable PRI: %d\n", ret);
+ return ret;
+ }
+
+ master->can_fault = true;
+ master->ste.prg_resp_needs_ssid = pci_prg_resp_requires_prefix(pdev);
+
+ dev_dbg(master->dev, "enabled PRI\n");
+
+ return 0;
+}
+
+static void arm_smmu_disable_pri(struct arm_smmu_master_data *master)
+{
+ struct pci_dev *pdev;
+
+ if (!dev_is_pci(master->dev))
+ return;
+
+ pdev = to_pci_dev(master->dev);
+
+ if (!pdev->pri_enabled)
+ return;
+
+ pci_disable_pri(pdev);
+ dev_dbg(master->dev, "disabled PRI\n");
+ master->can_fault = false;
+}
+
static int arm_smmu_sva_init(struct device *dev, struct iommu_sva_param *param)
{
int ret;
@@ -2314,11 +2401,15 @@ static int arm_smmu_sva_init(struct device *dev, struct iommu_sva_param *param)
return -EINVAL;
if (param->features & IOMMU_SVA_FEAT_IOPF) {
- if (!master->can_fault)
- return -EINVAL;
+ arm_smmu_enable_pri(master);
+ if (!master->can_fault) {
+ ret = -ENODEV;
+ goto err_disable_pri;
+ }
+
ret = iopf_queue_add_device(master->smmu->iopf_queue, dev);
if (ret)
- return ret;
+ goto err_disable_pri;
}
if (!param->max_pasid)
@@ -2329,11 +2420,17 @@ static int arm_smmu_sva_init(struct device *dev, struct iommu_sva_param *param)
param->max_pasid = min(param->max_pasid, (1U << master->ssid_bits) - 1);
return 0;
+
+err_disable_pri:
+ arm_smmu_disable_pri(master);
+
+ return ret;
}
static void arm_smmu_sva_shutdown(struct device *dev,
struct iommu_sva_param *param)
{
+ arm_smmu_disable_pri(dev->iommu_fwspec->iommu_priv);
iopf_queue_remove_device(dev);
}
@@ -2671,6 +2768,7 @@ static void arm_smmu_remove_device(struct device *dev)
iommu_group_remove_device(dev);
arm_smmu_remove_master(smmu, master);
iommu_device_unlink(&smmu->iommu, dev);
+ arm_smmu_disable_pri(master);
arm_smmu_disable_ats(master);
kfree(master);
iommu_fwspec_free(dev);
--
2.17.0
^ permalink raw reply related
* [PATCH v2 38/40] PCI: Make "PRG Response PASID Required" handling common
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA, bhelgaas-hpIqsD4AKlfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
The PASID ECN to the PCIe spec added a bit in the PRI status register that
allows a Function to declare whether a PRG Response should contain the
PASID prefix or not.
Move the helper that accesses it from amd_iommu into the PCI subsystem,
renaming it to be consistent with the current PCI Express specification
(PRPR - PRG Response PASID Required).
Cc: bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org
Acked-by: Bjorn Helgaas <bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/amd_iommu.c | 19 +------------------
drivers/pci/ats.c | 17 +++++++++++++++++
include/linux/pci-ats.h | 8 ++++++++
include/uapi/linux/pci_regs.h | 1 +
4 files changed, 27 insertions(+), 18 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 8fb8c737fffe..ac0d97fa514f 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2043,23 +2043,6 @@ static int pdev_iommuv2_enable(struct pci_dev *pdev)
return ret;
}
-/* FIXME: Move this to PCI code */
-#define PCI_PRI_TLP_OFF (1 << 15)
-
-static bool pci_pri_tlp_required(struct pci_dev *pdev)
-{
- u16 status;
- int pos;
-
- pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI);
- if (!pos)
- return false;
-
- pci_read_config_word(pdev, pos + PCI_PRI_STATUS, &status);
-
- return (status & PCI_PRI_TLP_OFF) ? true : false;
-}
-
/*
* If a device is not yet associated with a domain, this function
* assigns it visible for the hardware
@@ -2088,7 +2071,7 @@ static int attach_device(struct device *dev,
dev_data->ats.enabled = true;
dev_data->ats.qdep = pci_ats_queue_depth(pdev);
- dev_data->pri_tlp = pci_pri_tlp_required(pdev);
+ dev_data->pri_tlp = pci_prg_resp_requires_prefix(pdev);
}
} else if (amd_iommu_iotlb_sup &&
pci_enable_ats(pdev, PAGE_SHIFT) == 0) {
diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
index 89305b569d3d..dc743e2fae7d 100644
--- a/drivers/pci/ats.c
+++ b/drivers/pci/ats.c
@@ -388,3 +388,20 @@ int pci_max_pasids(struct pci_dev *pdev)
}
EXPORT_SYMBOL_GPL(pci_max_pasids);
#endif /* CONFIG_PCI_PASID */
+
+#if defined(CONFIG_PCI_PASID) && defined(CONFIG_PCI_PRI)
+bool pci_prg_resp_requires_prefix(struct pci_dev *pdev)
+{
+ u16 status;
+ int pos;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI);
+ if (!pos)
+ return false;
+
+ pci_read_config_word(pdev, pos + PCI_PRI_STATUS, &status);
+
+ return !!(status & PCI_PRI_STATUS_PRPR);
+}
+EXPORT_SYMBOL_GPL(pci_prg_resp_requires_prefix);
+#endif /* CONFIG_PCI_PASID && CONFIG_PCI_PRI */
diff --git a/include/linux/pci-ats.h b/include/linux/pci-ats.h
index 7c4b8e27268c..1825ca2c9bf4 100644
--- a/include/linux/pci-ats.h
+++ b/include/linux/pci-ats.h
@@ -68,5 +68,13 @@ static inline int pci_max_pasids(struct pci_dev *pdev)
#endif /* CONFIG_PCI_PASID */
+#if defined(CONFIG_PCI_PASID) && defined(CONFIG_PCI_PRI)
+bool pci_prg_resp_requires_prefix(struct pci_dev *pdev);
+#else
+static inline bool pci_prg_resp_requires_prefix(struct pci_dev *pdev)
+{
+ return false;
+}
+#endif /* CONFIG_PCI_PASID && CONFIG_PCI_PRI */
#endif /* LINUX_PCI_ATS_H*/
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 103ba797a8f3..f9a11a13131c 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -871,6 +871,7 @@
#define PCI_PRI_STATUS_RF 0x001 /* Response Failure */
#define PCI_PRI_STATUS_UPRGI 0x002 /* Unexpected PRG index */
#define PCI_PRI_STATUS_STOPPED 0x100 /* PRI Stopped */
+#define PCI_PRI_STATUS_PRPR 0x8000 /* PRG Response requires PASID prefix */
#define PCI_PRI_MAX_REQ 0x08 /* PRI max reqs supported */
#define PCI_PRI_ALLOC_REQ 0x0c /* PRI max reqs allowed */
#define PCI_EXT_CAP_PRI_SIZEOF 16
--
2.17.0
^ permalink raw reply related
* [PATCH v2 37/40] iommu/arm-smmu-v3: Disable tagged pointers
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
The ARM architecture has a "Top Byte Ignore" (TBI) option that makes the
MMU mask out bits [63:56] of an address, allowing a userspace application
to store data in its pointers. This option is incompatible with PCI ATS.
If TBI is enabled in the SMMU and userspace triggers DMA transactions on
tagged pointers, the endpoint might create ATC entries for addresses that
include a tag. Software would then have to send ATC invalidation packets
for each 255 possible alias of an address, or just wipe the whole address
space. This is not a viable option, so disable TBI.
The impact of this change is unclear, since there are very few users of
tagged pointers, much less SVA. But the requirement introduced by this
patch doesn't seem excessive: a userspace application using both tagged
pointers and SVA should now sanitize addresses (clear the tag) before
using them for device DMA.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3-context.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/iommu/arm-smmu-v3-context.c b/drivers/iommu/arm-smmu-v3-context.c
index bdc9bfd1f35d..22e7b80a7682 100644
--- a/drivers/iommu/arm-smmu-v3-context.c
+++ b/drivers/iommu/arm-smmu-v3-context.c
@@ -200,7 +200,6 @@ static u64 arm_smmu_cpu_tcr_to_cd(struct arm_smmu_context_cfg *cfg, u64 tcr)
val |= ARM_SMMU_TCR2CD(tcr, EPD0);
val |= ARM_SMMU_TCR2CD(tcr, EPD1);
val |= ARM_SMMU_TCR2CD(tcr, IPS);
- val |= ARM_SMMU_TCR2CD(tcr, TBI0);
if (cfg->hw_access)
val |= ARM_SMMU_TCR2CD(tcr, HA);
--
2.17.0
^ permalink raw reply related
* [PATCH v2 36/40] iommu/arm-smmu-v3: Hook up ATC invalidation to mm ops
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
The core calls us when an mm is modified. Perform the required ATC
invalidations.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3.c | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 7034b0bdcbdf..6cb69ace371b 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -1735,6 +1735,15 @@ static int arm_smmu_atc_inv_master_all(struct arm_smmu_master_data *master,
return arm_smmu_atc_inv_master(master, &cmd);
}
+static int arm_smmu_atc_inv_master_range(struct arm_smmu_master_data *master,
+ int ssid, unsigned long iova, size_t size)
+{
+ struct arm_smmu_cmdq_ent cmd;
+
+ arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd);
+ return arm_smmu_atc_inv_master(master, &cmd);
+}
+
static size_t
arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid,
unsigned long iova, size_t size)
@@ -2389,11 +2398,12 @@ static void arm_smmu_mm_detach(struct iommu_domain *domain, struct device *dev,
struct arm_smmu_mm *smmu_mm = to_smmu_mm(io_mm);
struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
struct iommu_pasid_table_ops *ops = smmu_domain->s1_cfg.ops;
+ struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv;
if (detach_domain)
ops->clear_entry(ops, io_mm->pasid, smmu_mm->cd);
- /* TODO: Invalidate ATC. */
+ arm_smmu_atc_inv_master_all(master, io_mm->pasid);
/* TODO: Invalidate all mappings if last and not DVM. */
}
@@ -2401,8 +2411,10 @@ static void arm_smmu_mm_invalidate(struct iommu_domain *domain,
struct device *dev, struct io_mm *io_mm,
unsigned long iova, size_t size)
{
+ struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv;
+
+ arm_smmu_atc_inv_master_range(master, io_mm->pasid, iova, size);
/*
- * TODO: Invalidate ATC.
* TODO: Invalidate mapping if not DVM
*/
}
--
2.17.0
^ permalink raw reply related
* [PATCH v2 35/40] iommu/arm-smmu-v3: Add support for PCI ATS
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
PCIe devices can implement their own TLB, named Address Translation Cache
(ATC). Enable Address Translation Service (ATS) for devices that support
it and send them invalidation requests whenever we invalidate the IOTLBs.
Range calculation
-----------------
The invalidation packet itself is a bit awkward: range must be naturally
aligned, which means that the start address is a multiple of the range
size. In addition, the size must be a power of two number of 4k pages. We
have a few options to enforce this constraint:
(1) Find the smallest naturally aligned region that covers the requested
range. This is simple to compute and only takes one ATC_INV, but it
will spill on lots of neighbouring ATC entries.
(2) Align the start address to the region size (rounded up to a power of
two), and send a second invalidation for the next range of the same
size. Still not great, but reduces spilling.
(3) Cover the range exactly with the smallest number of naturally aligned
regions. This would be interesting to implement but as for (2),
requires multiple ATC_INV.
As I suspect ATC invalidation packets will be a very scarce resource, I'll
go with option (1) for now, and only send one big invalidation. We can
move to (2), which is both easier to read and more gentle with the ATC,
once we've observed on real systems that we can send multiple smaller
Invalidation Requests for roughly the same price as a single big one.
Note that with io-pgtable, the unmap function is called for each page, so
this doesn't matter. The problem shows up when sharing page tables with
the MMU.
Timeout
-------
ATC invalidation is allowed to take up to 90 seconds, according to the
PCIe spec, so it is possible to hit the SMMU command queue timeout during
normal operations.
Some SMMU implementations will raise a CERROR_ATC_INV_SYNC when a CMD_SYNC
fails because of an ATC invalidation. Some will just abort the CMD_SYNC.
Others might let CMD_SYNC complete and have an asynchronous IMPDEF
mechanism to record the error. When we receive a CERROR_ATC_INV_SYNC, we
could retry sending all ATC_INV since last successful CMD_SYNC. When a
CMD_SYNC fails without CERROR_ATC_INV_SYNC, we could retry sending *all*
commands since last successful CMD_SYNC.
We cannot afford to wait 90 seconds in iommu_unmap, let alone MMU
notifiers. So we'd have to introduce a more clever system if this timeout
becomes a problem, like keeping hold of mappings and invalidating in the
background. Implementing safe delayed invalidations is a very complex
problem and deserves a series of its own. We'll assess whether more work
is needed to properly handle ATC invalidation timeouts once this code runs
on real hardware.
Misc
----
I didn't put ATC and TLB invalidations in the same functions for three
reasons:
* TLB invalidation by range is batched and committed with a single sync.
Batching ATC invalidation is inconvenient, endpoints limit the number of
inflight invalidations. We'd have to count the number of invalidations
queued and send a sync periodically. In addition, I suspect we always
need a sync between TLB and ATC invalidation for the same page.
* Doing ATC invalidation outside tlb_inv_range also allows to send less
requests, since TLB invalidations are done per page or block, while ATC
invalidations target IOVA ranges.
* TLB invalidation by context is performed when freeing the domain, at
which point there isn't any device attached anymore.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
v1->v2: display error if ats is supported but cannot be enabled
---
drivers/iommu/arm-smmu-v3.c | 225 +++++++++++++++++++++++++++++++++++-
1 file changed, 219 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 8a6a799ba04a..7034b0bdcbdf 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -39,6 +39,7 @@
#include <linux/of_iommu.h>
#include <linux/of_platform.h>
#include <linux/pci.h>
+#include <linux/pci-ats.h>
#include <linux/platform_device.h>
#include <linux/sched/mm.h>
@@ -103,6 +104,7 @@
#define IDR5_VAX_52_BIT 1
#define ARM_SMMU_CR0 0x20
+#define CR0_ATSCHK (1 << 4)
#define CR0_CMDQEN (1 << 3)
#define CR0_EVTQEN (1 << 2)
#define CR0_PRIQEN (1 << 1)
@@ -277,6 +279,7 @@
#define CMDQ_ERR_CERROR_NONE_IDX 0
#define CMDQ_ERR_CERROR_ILL_IDX 1
#define CMDQ_ERR_CERROR_ABT_IDX 2
+#define CMDQ_ERR_CERROR_ATC_INV_IDX 3
#define CMDQ_0_OP GENMASK_ULL(7, 0)
#define CMDQ_0_SSV (1UL << 11)
@@ -296,6 +299,12 @@
#define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12)
#define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12)
+#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12)
+#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32)
+#define CMDQ_ATC_0_GLOBAL (1UL << 9)
+#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0)
+#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12)
+
#define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12)
#define CMDQ_PRI_0_SID GENMASK_ULL(63, 32)
#define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0)
@@ -369,6 +378,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO);
MODULE_PARM_DESC(disable_bypass,
"Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU.");
+static bool disable_ats_check;
+module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO);
+MODULE_PARM_DESC(disable_ats_check,
+ "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check.");
+
enum pri_resp {
PRI_RESP_DENY = 0,
PRI_RESP_FAIL = 1,
@@ -442,6 +456,16 @@ struct arm_smmu_cmdq_ent {
u64 addr;
} tlbi;
+ #define CMDQ_OP_ATC_INV 0x40
+ #define ATC_INV_SIZE_ALL 52
+ struct {
+ u32 sid;
+ u32 ssid;
+ u64 addr;
+ u8 size;
+ bool global;
+ } atc;
+
#define CMDQ_OP_PRI_RESP 0x41
struct {
u32 sid;
@@ -869,6 +893,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent)
case CMDQ_OP_TLBI_EL2_ASID:
cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_ASID, ent->tlbi.asid);
break;
+ case CMDQ_OP_ATC_INV:
+ cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid);
+ cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global);
+ cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid);
+ cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid);
+ cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size);
+ cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK;
+ break;
case CMDQ_OP_PRI_RESP:
cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid);
cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid);
@@ -922,6 +954,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu)
[CMDQ_ERR_CERROR_NONE_IDX] = "No error",
[CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command",
[CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch",
+ [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout",
};
int i;
@@ -941,6 +974,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu)
dev_err(smmu->dev, "retrying command fetch\n");
case CMDQ_ERR_CERROR_NONE_IDX:
return;
+ case CMDQ_ERR_CERROR_ATC_INV_IDX:
+ /*
+ * ATC Invalidation Completion timeout. CONS is still pointing
+ * at the CMD_SYNC. Attempt to complete other pending commands
+ * by repeating the CMD_SYNC, though we might well end up back
+ * here since the ATC invalidation may still be pending.
+ */
+ return;
case CMDQ_ERR_CERROR_ILL_IDX:
/* Fallthrough */
default:
@@ -1193,9 +1234,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid,
FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) |
FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) |
FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) |
-#ifdef CONFIG_PCI_ATS
- FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) |
-#endif
FIELD_PREP(STRTAB_STE_1_STRW, strw));
if (smmu->features & ARM_SMMU_FEAT_STALLS &&
@@ -1225,6 +1263,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid,
val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
}
+ if (IS_ENABLED(CONFIG_PCI_ATS))
+ dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS,
+ STRTAB_STE_1_EATS_TRANS));
+
arm_smmu_sync_ste_for_sid(smmu, sid);
dst[0] = cpu_to_le64(val);
arm_smmu_sync_ste_for_sid(smmu, sid);
@@ -1613,6 +1655,104 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev)
return IRQ_WAKE_THREAD;
}
+/* ATS invalidation */
+static bool arm_smmu_master_has_ats(struct arm_smmu_master_data *master)
+{
+ return dev_is_pci(master->dev) && to_pci_dev(master->dev)->ats_enabled;
+}
+
+static void
+arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size,
+ struct arm_smmu_cmdq_ent *cmd)
+{
+ size_t log2_span;
+ size_t span_mask;
+ /* ATC invalidates are always on 4096 bytes pages */
+ size_t inval_grain_shift = 12;
+ unsigned long page_start, page_end;
+
+ *cmd = (struct arm_smmu_cmdq_ent) {
+ .opcode = CMDQ_OP_ATC_INV,
+ .substream_valid = !!ssid,
+ .atc.ssid = ssid,
+ };
+
+ if (!size) {
+ cmd->atc.size = ATC_INV_SIZE_ALL;
+ return;
+ }
+
+ page_start = iova >> inval_grain_shift;
+ page_end = (iova + size - 1) >> inval_grain_shift;
+
+ /*
+ * Find the smallest power of two that covers the range. Most
+ * significant differing bit between start and end address indicates the
+ * required span, ie. fls(start ^ end). For example:
+ *
+ * We want to invalidate pages [8; 11]. This is already the ideal range:
+ * x = 0b1000 ^ 0b1011 = 0b11
+ * span = 1 << fls(x) = 4
+ *
+ * To invalidate pages [7; 10], we need to invalidate [0; 15]:
+ * x = 0b0111 ^ 0b1010 = 0b1101
+ * span = 1 << fls(x) = 16
+ */
+ log2_span = fls_long(page_start ^ page_end);
+ span_mask = (1ULL << log2_span) - 1;
+
+ page_start &= ~span_mask;
+
+ cmd->atc.addr = page_start << inval_grain_shift;
+ cmd->atc.size = log2_span;
+}
+
+static int arm_smmu_atc_inv_master(struct arm_smmu_master_data *master,
+ struct arm_smmu_cmdq_ent *cmd)
+{
+ int i;
+ struct iommu_fwspec *fwspec = master->dev->iommu_fwspec;
+
+ if (!arm_smmu_master_has_ats(master))
+ return 0;
+
+ for (i = 0; i < fwspec->num_ids; i++) {
+ cmd->atc.sid = fwspec->ids[i];
+ arm_smmu_cmdq_issue_cmd(master->smmu, cmd);
+ }
+
+ arm_smmu_cmdq_issue_sync(master->smmu);
+
+ return 0;
+}
+
+static int arm_smmu_atc_inv_master_all(struct arm_smmu_master_data *master,
+ int ssid)
+{
+ struct arm_smmu_cmdq_ent cmd;
+
+ arm_smmu_atc_inv_to_cmd(ssid, 0, 0, &cmd);
+ return arm_smmu_atc_inv_master(master, &cmd);
+}
+
+static size_t
+arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid,
+ unsigned long iova, size_t size)
+{
+ unsigned long flags;
+ struct arm_smmu_cmdq_ent cmd;
+ struct arm_smmu_master_data *master;
+
+ arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd);
+
+ spin_lock_irqsave(&smmu_domain->devices_lock, flags);
+ list_for_each_entry(master, &smmu_domain->devices, list)
+ arm_smmu_atc_inv_master(master, &cmd);
+ spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
+
+ return size;
+}
+
/* IO_PGTABLE API */
static void __arm_smmu_tlb_sync(struct arm_smmu_device *smmu)
{
@@ -2026,6 +2166,8 @@ static void arm_smmu_detach_dev(struct device *dev)
if (smmu_domain) {
__iommu_sva_unbind_dev_all(dev);
+ arm_smmu_atc_inv_master_all(master, 0);
+
spin_lock_irqsave(&smmu_domain->devices_lock, flags);
list_del(&master->list);
spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
@@ -2113,12 +2255,19 @@ static int arm_smmu_map(struct iommu_domain *domain, unsigned long iova,
static size_t
arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size)
{
- struct io_pgtable_ops *ops = to_smmu_domain(domain)->pgtbl_ops;
+ int ret;
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
if (!ops)
return 0;
- return ops->unmap(ops, iova, size);
+ ret = ops->unmap(ops, iova, size);
+
+ if (ret && smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)
+ ret = arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size);
+
+ return ret;
}
static void arm_smmu_iotlb_sync(struct iommu_domain *domain)
@@ -2284,6 +2433,54 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
return sid < limit;
}
+static int arm_smmu_enable_ats(struct arm_smmu_master_data *master)
+{
+ size_t stu;
+ int ret, pos;
+ struct pci_dev *pdev;
+ struct arm_smmu_device *smmu = master->smmu;
+ struct iommu_fwspec *fwspec = master->dev->iommu_fwspec;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) ||
+ (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS))
+ return -ENOSYS;
+
+ pdev = to_pci_dev(master->dev);
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_ATS);
+ if (!pos)
+ return -ENOSYS;
+
+ /* Smallest Translation Unit: log2 of the smallest supported granule */
+ stu = __ffs(smmu->pgsize_bitmap);
+
+ ret = pci_enable_ats(pdev, stu);
+ if (ret) {
+ dev_err(&pdev->dev, "could not enable ATS: %d\n", ret);
+ return ret;
+ }
+
+ dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu,
+ pci_ats_queue_depth(pdev));
+
+ return 0;
+}
+
+static void arm_smmu_disable_ats(struct arm_smmu_master_data *master)
+{
+ struct pci_dev *pdev;
+
+ if (!dev_is_pci(master->dev))
+ return;
+
+ pdev = to_pci_dev(master->dev);
+
+ if (!pdev->ats_enabled)
+ return;
+
+ pci_disable_ats(pdev);
+}
+
static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
struct arm_smmu_master_data *master)
{
@@ -2406,9 +2603,11 @@ static int arm_smmu_add_device(struct device *dev)
master->ste.can_stall = true;
}
+ arm_smmu_enable_ats(master);
+
ret = iommu_device_link(&smmu->iommu, dev);
if (ret)
- goto err_free_master;
+ goto err_disable_ats;
ret = arm_smmu_insert_master(smmu, master);
if (ret)
@@ -2430,6 +2629,9 @@ static int arm_smmu_add_device(struct device *dev)
err_unlink:
iommu_device_unlink(&smmu->iommu, dev);
+err_disable_ats:
+ arm_smmu_disable_ats(master);
+
err_free_master:
kfree(master);
fwspec->iommu_priv = NULL;
@@ -2457,6 +2659,7 @@ static void arm_smmu_remove_device(struct device *dev)
iommu_group_remove_device(dev);
arm_smmu_remove_master(smmu, master);
iommu_device_unlink(&smmu->iommu, dev);
+ arm_smmu_disable_ats(master);
kfree(master);
iommu_fwspec_free(dev);
}
@@ -3069,6 +3272,16 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass)
}
}
+ if (smmu->features & ARM_SMMU_FEAT_ATS && !disable_ats_check) {
+ enables |= CR0_ATSCHK;
+ ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0,
+ ARM_SMMU_CR0ACK);
+ if (ret) {
+ dev_err(smmu->dev, "failed to enable ATS check\n");
+ return ret;
+ }
+ }
+
ret = arm_smmu_setup_irqs(smmu);
if (ret) {
dev_err(smmu->dev, "failed to setup irqs\n");
--
2.17.0
^ permalink raw reply related
* [PATCH v2 34/40] ACPI/IORT: Check ATS capability in root complex nodes
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A, sudeep.holla-5wv7dgnIgG8,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
Root complex node in IORT has a bit telling whether it supports ATS or
not. Store this bit in the IOMMU fwspec when setting up a device, so it
can be accessed later by an IOMMU driver.
Use the negative version (NO_ATS) at the moment because it's not clear
if/how the bit needs to be integrated in other firmware descriptions. The
SMMU has a feature bit telling if it supports ATS, which might be
sufficient in most systems for deciding whether or not we should enable
the ATS capability in endpoints.
Cc: lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org
Cc: hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org
Cc: sudeep.holla-5wv7dgnIgG8@public.gmane.org
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/acpi/arm64/iort.c | 11 +++++++++++
include/linux/iommu.h | 4 ++++
2 files changed, 15 insertions(+)
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 7a3a541046ed..4f4907e58cc8 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1004,6 +1004,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size)
dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset);
}
+static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node)
+{
+ struct acpi_iort_root_complex *pci_rc;
+
+ pci_rc = (struct acpi_iort_root_complex *)node->node_data;
+ return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED;
+}
+
/**
* iort_iommu_configure - Set-up IOMMU configuration for a device.
*
@@ -1039,6 +1047,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
info.node = node;
err = pci_for_each_dma_alias(to_pci_dev(dev),
iort_pci_iommu_init, &info);
+
+ if (!err && !iort_pci_rc_supports_ats(node))
+ dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS;
} else {
int i = 0;
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index bcce44455117..d7e2f54086e4 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -655,12 +655,16 @@ struct iommu_fwspec {
const struct iommu_ops *ops;
struct fwnode_handle *iommu_fwnode;
void *iommu_priv;
+ u32 flags;
unsigned int num_ids;
unsigned int num_pasid_bits;
bool can_stall;
u32 ids[1];
};
+/* Firmware disabled ATS in the root complex */
+#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0)
+
int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode,
const struct iommu_ops *ops);
void iommu_fwspec_free(struct device *dev);
--
2.17.0
^ permalink raw reply related
* [PATCH v2 33/40] iommu/arm-smmu-v3: Add stall support for platform devices
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
The SMMU provides a Stall model for handling page faults in platform
devices. It is similar to PCI PRI, but doesn't require devices to have
their own translation cache. Instead, faulting transactions are parked and
the OS is given a chance to fix the page tables and retry the transaction.
Enable stall for devices that support it (opt-in by firmware). When an
event corresponds to a translation error, call the IOMMU fault handler. If
the fault is recoverable, it will call us back to terminate or continue
the stall.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3.c | 178 +++++++++++++++++++++++++++++++++++-
1 file changed, 173 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 0f2d8aa0deee..8a6a799ba04a 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -301,6 +301,11 @@
#define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0)
#define CMDQ_PRI_1_RESP GENMASK_ULL(13, 12)
+#define CMDQ_RESUME_0_SID GENMASK_ULL(63, 32)
+#define CMDQ_RESUME_0_ACTION_RETRY (1UL << 12)
+#define CMDQ_RESUME_0_ACTION_ABORT (1UL << 13)
+#define CMDQ_RESUME_1_STAG GENMASK_ULL(15, 0)
+
#define CMDQ_SYNC_0_CS GENMASK_ULL(13, 12)
#define CMDQ_SYNC_0_CS_NONE 0
#define CMDQ_SYNC_0_CS_IRQ 1
@@ -316,6 +321,25 @@
#define EVTQ_0_ID GENMASK_ULL(7, 0)
+#define EVT_ID_TRANSLATION_FAULT 0x10
+#define EVT_ID_ADDR_SIZE_FAULT 0x11
+#define EVT_ID_ACCESS_FAULT 0x12
+#define EVT_ID_PERMISSION_FAULT 0x13
+
+#define EVTQ_0_SSV (1UL << 11)
+#define EVTQ_0_SSID GENMASK_ULL(31, 12)
+#define EVTQ_0_SID GENMASK_ULL(63, 32)
+#define EVTQ_1_STAG GENMASK_ULL(15, 0)
+#define EVTQ_1_STALL (1UL << 31)
+#define EVTQ_1_PRIV (1UL << 33)
+#define EVTQ_1_EXEC (1UL << 34)
+#define EVTQ_1_READ (1UL << 35)
+#define EVTQ_1_S2 (1UL << 39)
+#define EVTQ_1_CLASS GENMASK_ULL(41, 40)
+#define EVTQ_1_TT_READ (1UL << 44)
+#define EVTQ_2_ADDR GENMASK_ULL(63, 0)
+#define EVTQ_3_IPA GENMASK_ULL(51, 12)
+
/* PRI queue */
#define PRIQ_ENT_DWORDS 2
#define PRIQ_MAX_SZ_SHIFT 8
@@ -426,6 +450,13 @@ struct arm_smmu_cmdq_ent {
enum pri_resp resp;
} pri;
+ #define CMDQ_OP_RESUME 0x44
+ struct {
+ u32 sid;
+ u16 stag;
+ enum page_response_code resp;
+ } resume;
+
#define CMDQ_OP_CMD_SYNC 0x46
struct {
u32 msidata;
@@ -499,6 +530,8 @@ struct arm_smmu_strtab_ent {
bool assigned;
struct arm_smmu_s1_cfg *s1_cfg;
struct arm_smmu_s2_cfg *s2_cfg;
+
+ bool can_stall;
};
struct arm_smmu_strtab_cfg {
@@ -851,6 +884,21 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent)
}
cmd[1] |= FIELD_PREP(CMDQ_PRI_1_RESP, ent->pri.resp);
break;
+ case CMDQ_OP_RESUME:
+ cmd[0] |= FIELD_PREP(CMDQ_RESUME_0_SID, ent->resume.sid);
+ cmd[1] |= FIELD_PREP(CMDQ_RESUME_1_STAG, ent->resume.stag);
+ switch (ent->resume.resp) {
+ case IOMMU_PAGE_RESP_INVALID:
+ case IOMMU_PAGE_RESP_FAILURE:
+ cmd[0] |= CMDQ_RESUME_0_ACTION_ABORT;
+ break;
+ case IOMMU_PAGE_RESP_SUCCESS:
+ cmd[0] |= CMDQ_RESUME_0_ACTION_RETRY;
+ break;
+ default:
+ return -EINVAL;
+ }
+ break;
case CMDQ_OP_CMD_SYNC:
if (ent->sync.msiaddr)
cmd[0] |= FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_IRQ);
@@ -1013,6 +1061,34 @@ static void arm_smmu_cmdq_issue_sync(struct arm_smmu_device *smmu)
dev_err_ratelimited(smmu->dev, "CMD_SYNC timeout\n");
}
+static int arm_smmu_page_response(struct device *dev,
+ struct page_response_msg *resp)
+{
+ int sid = dev->iommu_fwspec->ids[0];
+ struct arm_smmu_cmdq_ent cmd = {0};
+ struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv;
+
+ if (master->ste.can_stall) {
+ cmd.opcode = CMDQ_OP_RESUME;
+ cmd.resume.sid = sid;
+ cmd.resume.stag = resp->page_req_group_id;
+ cmd.resume.resp = resp->resp_code;
+ } else {
+ /* TODO: put PRI response here */
+ return -ENODEV;
+ }
+
+ arm_smmu_cmdq_issue_cmd(master->smmu, &cmd);
+ /*
+ * Don't send a SYNC, it doesn't do anything for RESUME or PRI_RESP.
+ * RESUME consumption guarantees that the stalled transaction will be
+ * terminated... at some point in the future. PRI_RESP is fire and
+ * forget.
+ */
+
+ return 0;
+}
+
/* Stream table manipulation functions */
static void
arm_smmu_write_strtab_l1_desc(__le64 *dst, struct arm_smmu_strtab_l1_desc *desc)
@@ -1123,7 +1199,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid,
FIELD_PREP(STRTAB_STE_1_STRW, strw));
if (smmu->features & ARM_SMMU_FEAT_STALLS &&
- !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
+ !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE) &&
+ !ste->can_stall)
dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
val |= (ste->s1_cfg->tables.base & STRTAB_STE_0_S1CTXPTR_MASK) |
@@ -1196,7 +1273,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
return 0;
}
-__maybe_unused
static struct arm_smmu_master_data *
arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
{
@@ -1222,10 +1298,86 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
return master;
}
+static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
+{
+ int ret;
+ struct arm_smmu_master_data *master;
+ u8 type = FIELD_GET(EVTQ_0_ID, evt[0]);
+ u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]);
+
+ struct iommu_fault_event fault = {
+ .page_req_group_id = FIELD_GET(EVTQ_1_STAG, evt[1]),
+ .addr = FIELD_GET(EVTQ_2_ADDR, evt[2]),
+ .last_req = true,
+ };
+
+ switch (type) {
+ case EVT_ID_TRANSLATION_FAULT:
+ case EVT_ID_ADDR_SIZE_FAULT:
+ case EVT_ID_ACCESS_FAULT:
+ fault.reason = IOMMU_FAULT_REASON_PTE_FETCH;
+ break;
+ case EVT_ID_PERMISSION_FAULT:
+ fault.reason = IOMMU_FAULT_REASON_PERMISSION;
+ break;
+ default:
+ /* TODO: report other unrecoverable faults. */
+ return -EFAULT;
+ }
+
+ /* Stage-2 is always pinned at the moment */
+ if (evt[1] & EVTQ_1_S2)
+ return -EFAULT;
+
+ master = arm_smmu_find_master(smmu, sid);
+ if (!master)
+ return -EINVAL;
+
+ /*
+ * The domain is valid until the fault returns, because detach() flushes
+ * the fault queue.
+ */
+ if (evt[1] & EVTQ_1_STALL)
+ fault.type = IOMMU_FAULT_PAGE_REQ;
+ else
+ fault.type = IOMMU_FAULT_DMA_UNRECOV;
+
+ if (evt[1] & EVTQ_1_READ)
+ fault.prot |= IOMMU_FAULT_READ;
+ else
+ fault.prot |= IOMMU_FAULT_WRITE;
+
+ if (evt[1] & EVTQ_1_EXEC)
+ fault.prot |= IOMMU_FAULT_EXEC;
+
+ if (evt[1] & EVTQ_1_PRIV)
+ fault.prot |= IOMMU_FAULT_PRIV;
+
+ if (evt[0] & EVTQ_0_SSV) {
+ fault.pasid_valid = true;
+ fault.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]);
+ }
+
+ ret = iommu_report_device_fault(master->dev, &fault);
+ if (ret && fault.type == IOMMU_FAULT_PAGE_REQ) {
+ /* Nobody cared, abort the access */
+ struct page_response_msg resp = {
+ .addr = fault.addr,
+ .pasid = fault.pasid,
+ .pasid_present = fault.pasid_valid,
+ .page_req_group_id = fault.page_req_group_id,
+ .resp_code = IOMMU_PAGE_RESP_FAILURE,
+ };
+ arm_smmu_page_response(master->dev, &resp);
+ }
+
+ return ret;
+}
+
/* IRQ and event handlers */
static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
{
- int i;
+ int i, ret;
int num_handled = 0;
struct arm_smmu_device *smmu = dev;
struct arm_smmu_queue *q = &smmu->evtq.q;
@@ -1237,12 +1389,19 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
while (!queue_remove_raw(q, evt)) {
u8 id = FIELD_GET(EVTQ_0_ID, evt[0]);
+ spin_unlock(&q->wq.lock);
+ ret = arm_smmu_handle_evt(smmu, evt);
+ spin_lock(&q->wq.lock);
+
if (++num_handled == queue_size) {
q->batch++;
wake_up_all_locked(&q->wq);
num_handled = 0;
}
+ if (!ret)
+ continue;
+
dev_info(smmu->dev, "event 0x%02x received:\n", id);
for (i = 0; i < ARRAY_SIZE(evt); ++i)
dev_info(smmu->dev, "\t0x%016llx\n",
@@ -1374,7 +1533,9 @@ static int arm_smmu_flush_queues(void *cookie, struct device *dev)
if (dev) {
master = dev->iommu_fwspec->iommu_priv;
- /* TODO: add support for PRI and Stall */
+ if (master->ste.can_stall)
+ arm_smmu_flush_queue(smmu, &smmu->evtq.q, "evtq");
+ /* TODO: add support for PRI */
return 0;
}
@@ -1688,7 +1849,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,
.order = master->ssid_bits,
.sync = &arm_smmu_ctx_sync,
.arm_smmu = {
- .stall = !!(smmu->features & ARM_SMMU_FEAT_STALL_FORCE),
+ .stall = !!(smmu->features & ARM_SMMU_FEAT_STALL_FORCE) ||
+ master->ste.can_stall,
.asid_bits = smmu->asid_bits,
.hw_access = !!(smmu->features & ARM_SMMU_FEAT_HA),
.hw_dirty = !!(smmu->features & ARM_SMMU_FEAT_HD),
@@ -2239,6 +2401,11 @@ static int arm_smmu_add_device(struct device *dev)
master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits);
+ if (fwspec->can_stall && smmu->features & ARM_SMMU_FEAT_STALLS) {
+ master->can_fault = true;
+ master->ste.can_stall = true;
+ }
+
ret = iommu_device_link(&smmu->iommu, dev);
if (ret)
goto err_free_master;
@@ -2403,6 +2570,7 @@ static struct iommu_ops arm_smmu_ops = {
.mm_attach = arm_smmu_mm_attach,
.mm_detach = arm_smmu_mm_detach,
.mm_invalidate = arm_smmu_mm_invalidate,
+ .page_response = arm_smmu_page_response,
.map = arm_smmu_map,
.unmap = arm_smmu_unmap,
.map_sg = default_iommu_map_sg,
--
2.17.0
^ permalink raw reply related
* [PATCH v2 32/40] iommu/arm-smmu-v3: Maintain a SID->device structure
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
When handling faults from the event or PRI queue, we need to find the
struct device associated to a SID. Add a rb_tree to keep track of SIDs.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3.c | 114 +++++++++++++++++++++++++++++++++++-
1 file changed, 113 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index d5f3875abfb9..0f2d8aa0deee 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -570,9 +570,18 @@ struct arm_smmu_device {
/* IOMMU core code handle */
struct iommu_device iommu;
+ struct rb_root streams;
+ struct mutex streams_mutex;
+
struct iopf_queue *iopf_queue;
};
+struct arm_smmu_stream {
+ u32 id;
+ struct arm_smmu_master_data *master;
+ struct rb_node node;
+};
+
/* SMMU private data for each master */
struct arm_smmu_master_data {
struct arm_smmu_device *smmu;
@@ -580,6 +589,7 @@ struct arm_smmu_master_data {
struct arm_smmu_domain *domain;
struct list_head list; /* domain->devices */
+ struct arm_smmu_stream *streams;
struct device *dev;
size_t ssid_bits;
@@ -1186,6 +1196,32 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
return 0;
}
+__maybe_unused
+static struct arm_smmu_master_data *
+arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
+{
+ struct rb_node *node;
+ struct arm_smmu_stream *stream;
+ struct arm_smmu_master_data *master = NULL;
+
+ mutex_lock(&smmu->streams_mutex);
+ node = smmu->streams.rb_node;
+ while (node) {
+ stream = rb_entry(node, struct arm_smmu_stream, node);
+ if (stream->id < sid) {
+ node = node->rb_right;
+ } else if (stream->id > sid) {
+ node = node->rb_left;
+ } else {
+ master = stream->master;
+ break;
+ }
+ }
+ mutex_unlock(&smmu->streams_mutex);
+
+ return master;
+}
+
/* IRQ and event handlers */
static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
{
@@ -2086,6 +2122,71 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
return sid < limit;
}
+static int arm_smmu_insert_master(struct arm_smmu_device *smmu,
+ struct arm_smmu_master_data *master)
+{
+ int i;
+ int ret = 0;
+ struct arm_smmu_stream *new_stream, *cur_stream;
+ struct rb_node **new_node, *parent_node = NULL;
+ struct iommu_fwspec *fwspec = master->dev->iommu_fwspec;
+
+ master->streams = kcalloc(fwspec->num_ids,
+ sizeof(struct arm_smmu_stream), GFP_KERNEL);
+ if (!master->streams)
+ return -ENOMEM;
+
+ mutex_lock(&smmu->streams_mutex);
+ for (i = 0; i < fwspec->num_ids && !ret; i++) {
+ new_stream = &master->streams[i];
+ new_stream->id = fwspec->ids[i];
+ new_stream->master = master;
+
+ new_node = &(smmu->streams.rb_node);
+ while (*new_node) {
+ cur_stream = rb_entry(*new_node, struct arm_smmu_stream,
+ node);
+ parent_node = *new_node;
+ if (cur_stream->id > new_stream->id) {
+ new_node = &((*new_node)->rb_left);
+ } else if (cur_stream->id < new_stream->id) {
+ new_node = &((*new_node)->rb_right);
+ } else {
+ dev_warn(master->dev,
+ "stream %u already in tree\n",
+ cur_stream->id);
+ ret = -EINVAL;
+ break;
+ }
+ }
+
+ if (!ret) {
+ rb_link_node(&new_stream->node, parent_node, new_node);
+ rb_insert_color(&new_stream->node, &smmu->streams);
+ }
+ }
+ mutex_unlock(&smmu->streams_mutex);
+
+ return ret;
+}
+
+static void arm_smmu_remove_master(struct arm_smmu_device *smmu,
+ struct arm_smmu_master_data *master)
+{
+ int i;
+ struct iommu_fwspec *fwspec = master->dev->iommu_fwspec;
+
+ if (!master->streams)
+ return;
+
+ mutex_lock(&smmu->streams_mutex);
+ for (i = 0; i < fwspec->num_ids; i++)
+ rb_erase(&master->streams[i].node, &smmu->streams);
+ mutex_unlock(&smmu->streams_mutex);
+
+ kfree(master->streams);
+}
+
static struct iommu_ops arm_smmu_ops;
static int arm_smmu_add_device(struct device *dev)
@@ -2142,16 +2243,23 @@ static int arm_smmu_add_device(struct device *dev)
if (ret)
goto err_free_master;
+ ret = arm_smmu_insert_master(smmu, master);
+ if (ret)
+ goto err_unlink;
+
group = iommu_group_get_for_dev(dev);
if (IS_ERR(group)) {
ret = PTR_ERR(group);
- goto err_unlink;
+ goto err_remove_master;
}
iommu_group_put(group);
return 0;
+err_remove_master:
+ arm_smmu_remove_master(smmu, master);
+
err_unlink:
iommu_device_unlink(&smmu->iommu, dev);
@@ -2180,6 +2288,7 @@ static void arm_smmu_remove_device(struct device *dev)
if (master->ste.assigned)
arm_smmu_detach_dev(dev);
iommu_group_remove_device(dev);
+ arm_smmu_remove_master(smmu, master);
iommu_device_unlink(&smmu->iommu, dev);
kfree(master);
iommu_fwspec_free(dev);
@@ -2483,6 +2592,9 @@ static int arm_smmu_init_structures(struct arm_smmu_device *smmu)
int ret;
atomic_set(&smmu->sync_nr, 0);
+ mutex_init(&smmu->streams_mutex);
+ smmu->streams = RB_ROOT;
+
ret = arm_smmu_init_queues(smmu);
if (ret)
return ret;
--
2.17.0
^ permalink raw reply related
* [PATCH v2 31/40] iommu/arm-smmu-v3: Improve add_device error handling
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
As add_device becomes more likely to fail when adding new features, let it
clean up behind itself. The iommu_bus_init function does call
remove_device on error, but other sites (e.g. of_iommu) do not.
Don't free level-2 stream tables because we'd have to track if we
allocated each of them or if they are used by other endpoints. It's not
worth the hassle since they are managed resources.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
v1->v2: new
---
drivers/iommu/arm-smmu-v3.c | 36 ++++++++++++++++++++++++++++--------
1 file changed, 28 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 5d57f41f79b4..d5f3875abfb9 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2123,26 +2123,43 @@ static int arm_smmu_add_device(struct device *dev)
for (i = 0; i < fwspec->num_ids; i++) {
u32 sid = fwspec->ids[i];
- if (!arm_smmu_sid_in_range(smmu, sid))
- return -ERANGE;
+ if (!arm_smmu_sid_in_range(smmu, sid)) {
+ ret = -ERANGE;
+ goto err_free_master;
+ }
/* Ensure l2 strtab is initialised */
if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
ret = arm_smmu_init_l2_strtab(smmu, sid);
if (ret)
- return ret;
+ goto err_free_master;
}
}
master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits);
+ ret = iommu_device_link(&smmu->iommu, dev);
+ if (ret)
+ goto err_free_master;
+
group = iommu_group_get_for_dev(dev);
- if (!IS_ERR(group)) {
- iommu_group_put(group);
- iommu_device_link(&smmu->iommu, dev);
+ if (IS_ERR(group)) {
+ ret = PTR_ERR(group);
+ goto err_unlink;
}
- return PTR_ERR_OR_ZERO(group);
+ iommu_group_put(group);
+
+ return 0;
+
+err_unlink:
+ iommu_device_unlink(&smmu->iommu, dev);
+
+err_free_master:
+ kfree(master);
+ fwspec->iommu_priv = NULL;
+
+ return ret;
}
static void arm_smmu_remove_device(struct device *dev)
@@ -2155,9 +2172,12 @@ static void arm_smmu_remove_device(struct device *dev)
return;
master = fwspec->iommu_priv;
+ if (!master)
+ return;
+
smmu = master->smmu;
iopf_queue_remove_device(dev);
- if (master && master->ste.assigned)
+ if (master->ste.assigned)
arm_smmu_detach_dev(dev);
iommu_group_remove_device(dev);
iommu_device_unlink(&smmu->iommu, dev);
--
2.17.0
^ permalink raw reply related
* [PATCH v2 30/40] iommu/arm-smmu-v3: Register I/O Page Fault queue
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
When using PRI or Stall, the PRI or event handler enqueues faults into the
core fault queue. Register it based on the SMMU features.
When the core stops using a PASID, it notifies the SMMU to flush all
instances of this PASID from the PRI queue. Add a way to flush the PRI and
event queue. PRI and event thread now take a spinlock while processing the
queue. The flush handler takes this lock to inspect the queue state.
We avoid livelock, where the SMMU adds fault to the queue faster than we
can consume them, by incrementing a 'batch' number on every cycle so the
flush handler only has to wait a complete cycle (two batch increments).
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
v1->v2: Use an iopf_queue for each SMMU
---
drivers/iommu/Kconfig | 1 +
drivers/iommu/arm-smmu-v3.c | 111 +++++++++++++++++++++++++++++++++++-
2 files changed, 110 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 70900670a9fa..41db49795c90 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -344,6 +344,7 @@ config ARM_SMMU_V3
depends on ARM64
select IOMMU_API
select IOMMU_SVA
+ select IOMMU_PAGE_FAULT
select IOMMU_IO_PGTABLE_LPAE
select ARM_SMMU_V3_CONTEXT
select GENERIC_MSI_IRQ_DOMAIN
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 7c839d305d97..5d57f41f79b4 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -448,6 +448,10 @@ struct arm_smmu_queue {
u32 __iomem *prod_reg;
u32 __iomem *cons_reg;
+
+ /* Event and PRI */
+ u64 batch;
+ wait_queue_head_t wq;
};
struct arm_smmu_cmdq {
@@ -565,6 +569,8 @@ struct arm_smmu_device {
/* IOMMU core code handle */
struct iommu_device iommu;
+
+ struct iopf_queue *iopf_queue;
};
/* SMMU private data for each master */
@@ -577,6 +583,7 @@ struct arm_smmu_master_data {
struct device *dev;
size_t ssid_bits;
+ bool can_fault;
};
/* SMMU private data for an IOMMU domain */
@@ -1183,14 +1190,23 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)
static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
{
int i;
+ int num_handled = 0;
struct arm_smmu_device *smmu = dev;
struct arm_smmu_queue *q = &smmu->evtq.q;
+ size_t queue_size = 1 << q->max_n_shift;
u64 evt[EVTQ_ENT_DWORDS];
+ spin_lock(&q->wq.lock);
do {
while (!queue_remove_raw(q, evt)) {
u8 id = FIELD_GET(EVTQ_0_ID, evt[0]);
+ if (++num_handled == queue_size) {
+ q->batch++;
+ wake_up_all_locked(&q->wq);
+ num_handled = 0;
+ }
+
dev_info(smmu->dev, "event 0x%02x received:\n", id);
for (i = 0; i < ARRAY_SIZE(evt); ++i)
dev_info(smmu->dev, "\t0x%016llx\n",
@@ -1208,6 +1224,11 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
/* Sync our overflow flag, as we believe we're up to speed */
q->cons = Q_OVF(q, q->prod) | Q_WRP(q, q->cons) | Q_IDX(q, q->cons);
+
+ q->batch++;
+ wake_up_all_locked(&q->wq);
+ spin_unlock(&q->wq.lock);
+
return IRQ_HANDLED;
}
@@ -1251,13 +1272,24 @@ static void arm_smmu_handle_ppr(struct arm_smmu_device *smmu, u64 *evt)
static irqreturn_t arm_smmu_priq_thread(int irq, void *dev)
{
+ int num_handled = 0;
struct arm_smmu_device *smmu = dev;
struct arm_smmu_queue *q = &smmu->priq.q;
+ size_t queue_size = 1 << q->max_n_shift;
u64 evt[PRIQ_ENT_DWORDS];
+ spin_lock(&q->wq.lock);
do {
- while (!queue_remove_raw(q, evt))
+ while (!queue_remove_raw(q, evt)) {
+ spin_unlock(&q->wq.lock);
arm_smmu_handle_ppr(smmu, evt);
+ spin_lock(&q->wq.lock);
+ if (++num_handled == queue_size) {
+ q->batch++;
+ wake_up_all_locked(&q->wq);
+ num_handled = 0;
+ }
+ }
if (queue_sync_prod(q) == -EOVERFLOW)
dev_err(smmu->dev, "PRIQ overflow detected -- requests lost\n");
@@ -1265,9 +1297,60 @@ static irqreturn_t arm_smmu_priq_thread(int irq, void *dev)
/* Sync our overflow flag, as we believe we're up to speed */
q->cons = Q_OVF(q, q->prod) | Q_WRP(q, q->cons) | Q_IDX(q, q->cons);
+
+ q->batch++;
+ wake_up_all_locked(&q->wq);
+ spin_unlock(&q->wq.lock);
+
return IRQ_HANDLED;
}
+/*
+ * arm_smmu_flush_queue - wait until all events/PPRs currently in the queue have
+ * been consumed.
+ *
+ * Wait until the queue thread finished a batch, or until the queue is empty.
+ * Note that we don't handle overflows on q->batch. If it occurs, just wait for
+ * the queue to be empty.
+ */
+static int arm_smmu_flush_queue(struct arm_smmu_device *smmu,
+ struct arm_smmu_queue *q, const char *name)
+{
+ int ret;
+ u64 batch;
+
+ spin_lock(&q->wq.lock);
+ if (queue_sync_prod(q) == -EOVERFLOW)
+ dev_err(smmu->dev, "%s overflow detected -- requests lost\n", name);
+
+ batch = q->batch;
+ ret = wait_event_interruptible_locked(q->wq, queue_empty(q) ||
+ q->batch >= batch + 2);
+ spin_unlock(&q->wq.lock);
+
+ return ret;
+}
+
+static int arm_smmu_flush_queues(void *cookie, struct device *dev)
+{
+ struct arm_smmu_master_data *master;
+ struct arm_smmu_device *smmu = cookie;
+
+ if (dev) {
+ master = dev->iommu_fwspec->iommu_priv;
+ /* TODO: add support for PRI and Stall */
+ return 0;
+ }
+
+ /* No target device, flush all queues. */
+ if (smmu->features & ARM_SMMU_FEAT_STALLS)
+ arm_smmu_flush_queue(smmu, &smmu->evtq.q, "evtq");
+ if (smmu->features & ARM_SMMU_FEAT_PRI)
+ arm_smmu_flush_queue(smmu, &smmu->priq.q, "priq");
+
+ return 0;
+}
+
static int arm_smmu_device_disable(struct arm_smmu_device *smmu);
static irqreturn_t arm_smmu_gerror_handler(int irq, void *dev)
@@ -1864,15 +1947,24 @@ arm_smmu_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova)
static int arm_smmu_sva_init(struct device *dev, struct iommu_sva_param *param)
{
+ int ret;
struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv;
/* SSID support is mandatory for the moment */
if (!master->ssid_bits)
return -EINVAL;
- if (param->features)
+ if (param->features & ~IOMMU_SVA_FEAT_IOPF)
return -EINVAL;
+ if (param->features & IOMMU_SVA_FEAT_IOPF) {
+ if (!master->can_fault)
+ return -EINVAL;
+ ret = iopf_queue_add_device(master->smmu->iopf_queue, dev);
+ if (ret)
+ return ret;
+ }
+
if (!param->max_pasid)
param->max_pasid = 0xfffffU;
@@ -1886,6 +1978,7 @@ static int arm_smmu_sva_init(struct device *dev, struct iommu_sva_param *param)
static void arm_smmu_sva_shutdown(struct device *dev,
struct iommu_sva_param *param)
{
+ iopf_queue_remove_device(dev);
}
static struct io_mm *arm_smmu_mm_alloc(struct iommu_domain *domain,
@@ -2063,6 +2156,7 @@ static void arm_smmu_remove_device(struct device *dev)
master = fwspec->iommu_priv;
smmu = master->smmu;
+ iopf_queue_remove_device(dev);
if (master && master->ste.assigned)
arm_smmu_detach_dev(dev);
iommu_group_remove_device(dev);
@@ -2222,6 +2316,10 @@ static int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,
q->q_base |= FIELD_PREP(Q_BASE_LOG2SIZE, q->max_n_shift);
q->prod = q->cons = 0;
+
+ init_waitqueue_head(&q->wq);
+ q->batch = 0;
+
return 0;
}
@@ -3128,6 +3226,14 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
if (ret)
return ret;
+ if (smmu->features & (ARM_SMMU_FEAT_STALLS | ARM_SMMU_FEAT_PRI)) {
+ smmu->iopf_queue = iopf_queue_alloc(dev_name(dev),
+ arm_smmu_flush_queues,
+ smmu);
+ if (!smmu->iopf_queue)
+ return -ENOMEM;
+ }
+
/* And we're up. Go go go! */
ret = iommu_device_sysfs_add(&smmu->iommu, dev, NULL,
"smmu3.%pa", &ioaddr);
@@ -3170,6 +3276,7 @@ static int arm_smmu_device_remove(struct platform_device *pdev)
{
struct arm_smmu_device *smmu = platform_get_drvdata(pdev);
+ iopf_queue_free(smmu->iopf_queue);
arm_smmu_device_disable(smmu);
return 0;
--
2.17.0
^ permalink raw reply related
* [PATCH v2 29/40] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
If the SMMU supports it and the kernel was built with HTTU support, enable
hardware update of access and dirty flags. This is essential for shared
page tables, to reduce the number of access faults on the fault queue.
We can enable HTTU even if CPUs don't support it, because the kernel
always checks for HW dirty bit and updates the PTE flags atomically.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3-context.c | 16 ++++++++++++++--
drivers/iommu/arm-smmu-v3.c | 12 ++++++++++++
drivers/iommu/iommu-pasid-table.h | 4 ++++
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3-context.c b/drivers/iommu/arm-smmu-v3-context.c
index 0e12f6804e16..bdc9bfd1f35d 100644
--- a/drivers/iommu/arm-smmu-v3-context.c
+++ b/drivers/iommu/arm-smmu-v3-context.c
@@ -52,6 +52,11 @@
#define CTXDESC_CD_0_TCR_TBI0 (1ULL << 38)
#define ARM64_TCR_TBI0 (1ULL << 37)
+#define CTXDESC_CD_0_TCR_HA (1UL << 43)
+#define ARM64_TCR_HA (1ULL << 39)
+#define CTXDESC_CD_0_TCR_HD (1UL << 42)
+#define ARM64_TCR_HD (1ULL << 40)
+
#define CTXDESC_CD_0_AA64 (1UL << 41)
#define CTXDESC_CD_0_S (1UL << 44)
#define CTXDESC_CD_0_R (1UL << 45)
@@ -182,7 +187,7 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_cd_tables *tbl, u32 ssid)
return l1_desc->ptr + idx * CTXDESC_CD_DWORDS;
}
-static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr)
+static u64 arm_smmu_cpu_tcr_to_cd(struct arm_smmu_context_cfg *cfg, u64 tcr)
{
u64 val = 0;
@@ -197,6 +202,12 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr)
val |= ARM_SMMU_TCR2CD(tcr, IPS);
val |= ARM_SMMU_TCR2CD(tcr, TBI0);
+ if (cfg->hw_access)
+ val |= ARM_SMMU_TCR2CD(tcr, HA);
+
+ if (cfg->hw_dirty)
+ val |= ARM_SMMU_TCR2CD(tcr, HD);
+
return val;
}
@@ -250,7 +261,7 @@ static int __arm_smmu_write_ctx_desc(struct arm_smmu_cd_tables *tbl, int ssid,
iommu_pasid_flush(&tbl->pasid, ssid, true);
- val = arm_smmu_cpu_tcr_to_cd(cd->tcr) |
+ val = arm_smmu_cpu_tcr_to_cd(cfg, cd->tcr) |
#ifdef __BIG_ENDIAN
CTXDESC_CD_0_ENDI |
#endif
@@ -455,6 +466,7 @@ arm_smmu_alloc_shared_cd(struct iommu_pasid_table_ops *ops, struct mm_struct *mm
reg = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);
par = cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR0_PARANGE_SHIFT);
tcr |= par << ARM_LPAE_TCR_IPS_SHIFT;
+ tcr |= TCR_HA | TCR_HD;
cd->ttbr = virt_to_phys(mm->pgd);
cd->tcr = tcr;
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index c2c96025ac3b..7c839d305d97 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -66,6 +66,8 @@
#define IDR0_ASID16 (1 << 12)
#define IDR0_ATS (1 << 10)
#define IDR0_HYP (1 << 9)
+#define IDR0_HD (1 << 7)
+#define IDR0_HA (1 << 6)
#define IDR0_BTM (1 << 5)
#define IDR0_COHACC (1 << 4)
#define IDR0_TTF GENMASK(3, 2)
@@ -528,6 +530,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_E2H (1 << 15)
#define ARM_SMMU_FEAT_BTM (1 << 16)
#define ARM_SMMU_FEAT_SVA (1 << 17)
+#define ARM_SMMU_FEAT_HA (1 << 18)
+#define ARM_SMMU_FEAT_HD (1 << 19)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
@@ -1567,6 +1571,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,
.arm_smmu = {
.stall = !!(smmu->features & ARM_SMMU_FEAT_STALL_FORCE),
.asid_bits = smmu->asid_bits,
+ .hw_access = !!(smmu->features & ARM_SMMU_FEAT_HA),
+ .hw_dirty = !!(smmu->features & ARM_SMMU_FEAT_HD),
},
};
@@ -2818,6 +2824,12 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
smmu->features |= ARM_SMMU_FEAT_E2H;
}
+ if (reg & (IDR0_HA | IDR0_HD)) {
+ smmu->features |= ARM_SMMU_FEAT_HA;
+ if (reg & IDR0_HD)
+ smmu->features |= ARM_SMMU_FEAT_HD;
+ }
+
/*
* If the CPU is using VHE, but the SMMU doesn't support it, the SMMU
* will create TLB entries for NH-EL1 world and will miss the
diff --git a/drivers/iommu/iommu-pasid-table.h b/drivers/iommu/iommu-pasid-table.h
index b84709e297bc..a7243579a4cb 100644
--- a/drivers/iommu/iommu-pasid-table.h
+++ b/drivers/iommu/iommu-pasid-table.h
@@ -78,12 +78,16 @@ struct iommu_pasid_sync_ops {
* SMMU properties:
* @stall: devices attached to the domain are allowed to stall.
* @asid_bits: number of ASID bits supported by the SMMU
+ * @hw_dirty: hardware may update dirty flag
+ * @hw_access: hardware may update access flag
*
* @s1fmt: PASID table format, chosen by the allocator.
*/
struct arm_smmu_context_cfg {
u8 stall:1;
u8 asid_bits;
+ u8 hw_dirty:1;
+ u8 hw_access:1;
#define ARM_SMMU_S1FMT_LINEAR 0x0
#define ARM_SMMU_S1FMT_4K_L2 0x1
--
2.17.0
^ permalink raw reply related
* [PATCH v2 28/40] iommu/arm-smmu-v3: Implement mm operations
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
Hook mm operations to support PASID and page table sharing with the
SMMUv3:
* mm_alloc allocates a context descriptor.
* mm_free releases the context descriptor.
* mm_attach checks device capabilities and writes the context descriptor.
* mm_detach clears the context descriptor and sends required
invalidations.
* mm_invalidate sends required invalidations.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/Kconfig | 1 +
drivers/iommu/arm-smmu-v3.c | 126 ++++++++++++++++++++++++++++++++++++
2 files changed, 127 insertions(+)
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 11c8492b3763..70900670a9fa 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -343,6 +343,7 @@ config ARM_SMMU_V3
bool "ARM Ltd. System MMU Version 3 (SMMUv3) Support"
depends on ARM64
select IOMMU_API
+ select IOMMU_SVA
select IOMMU_IO_PGTABLE_LPAE
select ARM_SMMU_V3_CONTEXT
select GENERIC_MSI_IRQ_DOMAIN
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 2716e4a4d3f7..c2c96025ac3b 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -31,6 +31,7 @@
#include <linux/interrupt.h>
#include <linux/iommu.h>
#include <linux/iopoll.h>
+#include <linux/mmu_context.h>
#include <linux/module.h>
#include <linux/msi.h>
#include <linux/of.h>
@@ -39,6 +40,7 @@
#include <linux/of_platform.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
+#include <linux/sched/mm.h>
#include <linux/amba/bus.h>
@@ -599,6 +601,11 @@ struct arm_smmu_domain {
spinlock_t devices_lock;
};
+struct arm_smmu_mm {
+ struct io_mm io_mm;
+ struct iommu_pasid_entry *cd;
+};
+
struct arm_smmu_option_prop {
u32 opt;
const char *prop;
@@ -625,6 +632,11 @@ static struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom)
return container_of(dom, struct arm_smmu_domain, domain);
}
+static struct arm_smmu_mm *to_smmu_mm(struct io_mm *io_mm)
+{
+ return container_of(io_mm, struct arm_smmu_mm, io_mm);
+}
+
static void parse_driver_options(struct arm_smmu_device *smmu)
{
int i = 0;
@@ -1725,6 +1737,8 @@ static void arm_smmu_detach_dev(struct device *dev)
struct arm_smmu_domain *smmu_domain = master->domain;
if (smmu_domain) {
+ __iommu_sva_unbind_dev_all(dev);
+
spin_lock_irqsave(&smmu_domain->devices_lock, flags);
list_del(&master->list);
spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);
@@ -1842,6 +1856,111 @@ arm_smmu_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova)
return ops->iova_to_phys(ops, iova);
}
+static int arm_smmu_sva_init(struct device *dev, struct iommu_sva_param *param)
+{
+ struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv;
+
+ /* SSID support is mandatory for the moment */
+ if (!master->ssid_bits)
+ return -EINVAL;
+
+ if (param->features)
+ return -EINVAL;
+
+ if (!param->max_pasid)
+ param->max_pasid = 0xfffffU;
+
+ /* SSID support in the SMMU requires at least one SSID bit */
+ param->min_pasid = max(param->min_pasid, 1U);
+ param->max_pasid = min(param->max_pasid, (1U << master->ssid_bits) - 1);
+
+ return 0;
+}
+
+static void arm_smmu_sva_shutdown(struct device *dev,
+ struct iommu_sva_param *param)
+{
+}
+
+static struct io_mm *arm_smmu_mm_alloc(struct iommu_domain *domain,
+ struct mm_struct *mm,
+ unsigned long flags)
+{
+ struct arm_smmu_mm *smmu_mm;
+ struct iommu_pasid_entry *cd;
+ struct iommu_pasid_table_ops *ops;
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return NULL;
+
+ smmu_mm = kzalloc(sizeof(*smmu_mm), GFP_KERNEL);
+ if (!smmu_mm)
+ return NULL;
+
+ ops = smmu_domain->s1_cfg.ops;
+ cd = ops->alloc_shared_entry(ops, mm);
+ if (IS_ERR(cd)) {
+ kfree(smmu_mm);
+ return ERR_CAST(cd);
+ }
+
+ smmu_mm->cd = cd;
+ return &smmu_mm->io_mm;
+}
+
+static void arm_smmu_mm_free(struct io_mm *io_mm)
+{
+ struct arm_smmu_mm *smmu_mm = to_smmu_mm(io_mm);
+
+ iommu_free_pasid_entry(smmu_mm->cd);
+ kfree(smmu_mm);
+}
+
+static int arm_smmu_mm_attach(struct iommu_domain *domain, struct device *dev,
+ struct io_mm *io_mm, bool attach_domain)
+{
+ struct arm_smmu_mm *smmu_mm = to_smmu_mm(io_mm);
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct iommu_pasid_table_ops *ops = smmu_domain->s1_cfg.ops;
+ struct arm_smmu_master_data *master = dev->iommu_fwspec->iommu_priv;
+
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (!(master->smmu->features & ARM_SMMU_FEAT_SVA))
+ return -ENODEV;
+
+ if (!attach_domain)
+ return 0;
+
+ return ops->set_entry(ops, io_mm->pasid, smmu_mm->cd);
+}
+
+static void arm_smmu_mm_detach(struct iommu_domain *domain, struct device *dev,
+ struct io_mm *io_mm, bool detach_domain)
+{
+ struct arm_smmu_mm *smmu_mm = to_smmu_mm(io_mm);
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct iommu_pasid_table_ops *ops = smmu_domain->s1_cfg.ops;
+
+ if (detach_domain)
+ ops->clear_entry(ops, io_mm->pasid, smmu_mm->cd);
+
+ /* TODO: Invalidate ATC. */
+ /* TODO: Invalidate all mappings if last and not DVM. */
+}
+
+static void arm_smmu_mm_invalidate(struct iommu_domain *domain,
+ struct device *dev, struct io_mm *io_mm,
+ unsigned long iova, size_t size)
+{
+ /*
+ * TODO: Invalidate ATC.
+ * TODO: Invalidate mapping if not DVM
+ */
+}
+
static struct platform_driver arm_smmu_driver;
static int arm_smmu_match_node(struct device *dev, void *data)
@@ -2048,6 +2167,13 @@ static struct iommu_ops arm_smmu_ops = {
.domain_alloc = arm_smmu_domain_alloc,
.domain_free = arm_smmu_domain_free,
.attach_dev = arm_smmu_attach_dev,
+ .sva_device_init = arm_smmu_sva_init,
+ .sva_device_shutdown = arm_smmu_sva_shutdown,
+ .mm_alloc = arm_smmu_mm_alloc,
+ .mm_free = arm_smmu_mm_free,
+ .mm_attach = arm_smmu_mm_attach,
+ .mm_detach = arm_smmu_mm_detach,
+ .mm_invalidate = arm_smmu_mm_invalidate,
.map = arm_smmu_map,
.unmap = arm_smmu_unmap,
.map_sg = default_iommu_map_sg,
--
2.17.0
^ permalink raw reply related
* [PATCH v2 27/40] iommu/arm-smmu-v3: Add SVA feature checking
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
Aggregate all sanity-checks for sharing CPU page tables with the SMMU
under a single ARM_SMMU_FEAT_SVA bit. For PCIe SVA, users also need to
check FEAT_ATS and FEAT_PRI. For platform SVM, they will most likely have
to check FEAT_STALLS.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
v1->v2: Add 52-bit PA cap and debug message
---
drivers/iommu/arm-smmu-v3.c | 72 +++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 073cba33ae6c..2716e4a4d3f7 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -525,6 +525,7 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_VAX (1 << 14)
#define ARM_SMMU_FEAT_E2H (1 << 15)
#define ARM_SMMU_FEAT_BTM (1 << 16)
+#define ARM_SMMU_FEAT_SVA (1 << 17)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
@@ -2565,6 +2566,74 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass)
return 0;
}
+static bool arm_smmu_supports_sva(struct arm_smmu_device *smmu)
+{
+ unsigned long reg, fld;
+ unsigned long oas;
+ unsigned long asid_bits;
+
+ u32 feat_mask = ARM_SMMU_FEAT_BTM | ARM_SMMU_FEAT_COHERENCY;
+
+ if ((smmu->features & feat_mask) != feat_mask)
+ return false;
+
+ if (!(smmu->pgsize_bitmap & PAGE_SIZE))
+ return false;
+
+ /*
+ * Get the smallest PA size of all CPUs (sanitized by cpufeature). We're
+ * not even pretending to support AArch32 here.
+ */
+ reg = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);
+ fld = cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR0_PARANGE_SHIFT);
+ switch (fld) {
+ case 0x0:
+ oas = 32;
+ break;
+ case 0x1:
+ oas = 36;
+ break;
+ case 0x2:
+ oas = 40;
+ break;
+ case 0x3:
+ oas = 42;
+ break;
+ case 0x4:
+ oas = 44;
+ break;
+ case 0x5:
+ oas = 48;
+ break;
+ case 0x6:
+ oas = 52;
+ break;
+ default:
+ return false;
+ }
+
+ /* abort if MMU outputs addresses greater than what we support. */
+ if (smmu->oas < oas)
+ return false;
+
+ /* We can support bigger ASIDs than the CPU, but not smaller */
+ fld = cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR0_ASID_SHIFT);
+ asid_bits = fld ? 16 : 8;
+ if (smmu->asid_bits < asid_bits)
+ return false;
+
+ /*
+ * See max_pinned_asids in arch/arm64/mm/context.c. The following is
+ * generally the maximum number of bindable processes.
+ */
+ if (IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0))
+ asid_bits--;
+ dev_dbg(smmu->dev, "%d shared contexts\n", (1 << asid_bits) -
+ num_possible_cpus() - 2);
+
+ return true;
+}
+
static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
{
u32 reg;
@@ -2766,6 +2835,9 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
smmu->ias = max(smmu->ias, smmu->oas);
+ if (arm_smmu_supports_sva(smmu))
+ smmu->features |= ARM_SMMU_FEAT_SVA;
+
dev_info(smmu->dev, "ias %lu-bit, oas %lu-bit (features 0x%08x)\n",
smmu->ias, smmu->oas, smmu->features);
return 0;
--
2.17.0
^ permalink raw reply related
* [PATCH v2 26/40] iommu/arm-smmu-v3: Enable broadcast TLB maintenance
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
The SMMUv3 can handle invalidation targeted at TLB entries with shared
ASIDs. If the implementation supports broadcast TLB maintenance, enable it
and keep track of it in a feature bit. The SMMU will then be affected by
inner-shareable TLB invalidations from other agents.
A major side-effect of this change is that stage-2 translation contexts
are now affected by all invalidations by VMID. VMIDs are all shared and
the only ways to prevent over-invalidation, since the stage-2 page tables
are not shared between CPU and SMMU, are to either disable BTM or allocate
different VMIDs. This patch does not address the problem.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3.c | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 280a5d9be839..073cba33ae6c 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -64,6 +64,7 @@
#define IDR0_ASID16 (1 << 12)
#define IDR0_ATS (1 << 10)
#define IDR0_HYP (1 << 9)
+#define IDR0_BTM (1 << 5)
#define IDR0_COHACC (1 << 4)
#define IDR0_TTF GENMASK(3, 2)
#define IDR0_TTF_AARCH64 2
@@ -523,6 +524,7 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_STALL_FORCE (1 << 13)
#define ARM_SMMU_FEAT_VAX (1 << 14)
#define ARM_SMMU_FEAT_E2H (1 << 15)
+#define ARM_SMMU_FEAT_BTM (1 << 16)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
@@ -2459,11 +2461,14 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass)
writel_relaxed(reg, smmu->base + ARM_SMMU_CR1);
/* CR2 (random crap) */
- reg = CR2_PTM | CR2_RECINVSID;
+ reg = CR2_RECINVSID;
if (smmu->features & ARM_SMMU_FEAT_E2H)
reg |= CR2_E2H;
+ if (!(smmu->features & ARM_SMMU_FEAT_BTM))
+ reg |= CR2_PTM;
+
writel_relaxed(reg, smmu->base + ARM_SMMU_CR2);
/* Stream table */
@@ -2564,6 +2569,7 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
{
u32 reg;
bool coherent = smmu->features & ARM_SMMU_FEAT_COHERENCY;
+ bool vhe = cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN);
/* IDR0 */
reg = readl_relaxed(smmu->base + ARM_SMMU_IDR0);
@@ -2613,10 +2619,19 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
if (reg & IDR0_HYP) {
smmu->features |= ARM_SMMU_FEAT_HYP;
- if (cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN))
+ if (vhe)
smmu->features |= ARM_SMMU_FEAT_E2H;
}
+ /*
+ * If the CPU is using VHE, but the SMMU doesn't support it, the SMMU
+ * will create TLB entries for NH-EL1 world and will miss the
+ * broadcasted TLB invalidations that target EL2-E2H world. Don't enable
+ * BTM in that case.
+ */
+ if (reg & IDR0_BTM && (!vhe || reg & IDR0_HYP))
+ smmu->features |= ARM_SMMU_FEAT_BTM;
+
/*
* The coherency feature as set by FW is used in preference to the ID
* register, but warn on mismatch.
--
2.17.0
^ permalink raw reply related
* [PATCH v2 25/40] iommu/arm-smmu-v3: Add support for VHE
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
ARMv8.1 extensions added Virtualization Host Extensions (VHE), which allow
to run a host kernel at EL2. When using normal DMA, Device and CPU address
spaces are dissociated, and do not need to implement the same
capabilities, so VHE hasn't been used in the SMMU until now.
With shared address spaces however, ASIDs are shared between MMU and SMMU,
and broadcast TLB invalidations issued by a CPU are taken into account by
the SMMU. TLB entries on both sides need to have identical exception level
in order to be cleared with a single invalidation.
When the CPU is using VHE, enable VHE in the SMMU for all STEs. Normal DMA
mappings will need to use TLBI_EL2 commands instead of TLBI_NH, but
shouldn't be otherwise affected by this change.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3.c | 32 ++++++++++++++++++++++++++------
1 file changed, 26 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 16b08f2fb8ac..280a5d9be839 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -24,6 +24,7 @@
#include <linux/acpi_iort.h>
#include <linux/bitfield.h>
#include <linux/bitops.h>
+#include <linux/cpufeature.h>
#include <linux/delay.h>
#include <linux/dma-iommu.h>
#include <linux/err.h>
@@ -400,6 +401,8 @@ struct arm_smmu_cmdq_ent {
#define CMDQ_OP_TLBI_NH_ASID 0x11
#define CMDQ_OP_TLBI_NH_VA 0x12
#define CMDQ_OP_TLBI_EL2_ALL 0x20
+ #define CMDQ_OP_TLBI_EL2_ASID 0x21
+ #define CMDQ_OP_TLBI_EL2_VA 0x22
#define CMDQ_OP_TLBI_S12_VMALL 0x28
#define CMDQ_OP_TLBI_S2_IPA 0x2a
#define CMDQ_OP_TLBI_NSNH_ALL 0x30
@@ -519,6 +522,7 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_HYP (1 << 12)
#define ARM_SMMU_FEAT_STALL_FORCE (1 << 13)
#define ARM_SMMU_FEAT_VAX (1 << 14)
+#define ARM_SMMU_FEAT_E2H (1 << 15)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
@@ -777,6 +781,7 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent)
cmd[1] |= FIELD_PREP(CMDQ_CFGI_1_RANGE, 31);
break;
case CMDQ_OP_TLBI_NH_VA:
+ case CMDQ_OP_TLBI_EL2_VA:
cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_ASID, ent->tlbi.asid);
cmd[1] |= FIELD_PREP(CMDQ_TLBI_1_LEAF, ent->tlbi.leaf);
cmd[1] |= ent->tlbi.addr & CMDQ_TLBI_1_VA_MASK;
@@ -792,6 +797,9 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent)
case CMDQ_OP_TLBI_S12_VMALL:
cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid);
break;
+ case CMDQ_OP_TLBI_EL2_ASID:
+ cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_ASID, ent->tlbi.asid);
+ break;
case CMDQ_OP_PRI_RESP:
cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid);
cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid);
@@ -1064,6 +1072,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid,
if (ste->s1_cfg) {
struct iommu_pasid_table_cfg *cfg = &ste->s1_cfg->tables;
+ int strw = smmu->features & ARM_SMMU_FEAT_E2H ?
+ STRTAB_STE_1_STRW_EL2 : STRTAB_STE_1_STRW_NSEL1;
BUG_ON(ste_live);
dst[1] = cpu_to_le64(
@@ -1074,7 +1084,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid,
#ifdef CONFIG_PCI_ATS
FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) |
#endif
- FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1));
+ FIELD_PREP(STRTAB_STE_1_STRW, strw));
if (smmu->features & ARM_SMMU_FEAT_STALLS &&
!(smmu->features & ARM_SMMU_FEAT_STALL_FORCE))
@@ -1325,7 +1335,8 @@ static void arm_smmu_tlb_inv_context(void *cookie)
if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
if (unlikely(!smmu_domain->s1_cfg.cd0))
return;
- cmd.opcode = CMDQ_OP_TLBI_NH_ASID;
+ cmd.opcode = smmu->features & ARM_SMMU_FEAT_E2H ?
+ CMDQ_OP_TLBI_EL2_ASID : CMDQ_OP_TLBI_NH_ASID;
cmd.tlbi.asid = smmu_domain->s1_cfg.cd0->tag;
cmd.tlbi.vmid = 0;
} else {
@@ -1352,7 +1363,8 @@ static void arm_smmu_tlb_inv_range_nosync(unsigned long iova, size_t size,
if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
if (unlikely(!smmu_domain->s1_cfg.cd0))
return;
- cmd.opcode = CMDQ_OP_TLBI_NH_VA;
+ cmd.opcode = smmu->features & ARM_SMMU_FEAT_E2H ?
+ CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA;
cmd.tlbi.asid = smmu_domain->s1_cfg.cd0->tag;
} else {
cmd.opcode = CMDQ_OP_TLBI_S2_IPA;
@@ -1422,7 +1434,8 @@ static void arm_smmu_tlb_inv_ssid(void *cookie, int ssid,
struct arm_smmu_domain *smmu_domain = cookie;
struct arm_smmu_device *smmu = smmu_domain->smmu;
struct arm_smmu_cmdq_ent cmd = {
- .opcode = CMDQ_OP_TLBI_NH_ASID,
+ .opcode = smmu->features & ARM_SMMU_FEAT_E2H ?
+ CMDQ_OP_TLBI_EL2_ASID : CMDQ_OP_TLBI_NH_ASID,
.tlbi.asid = entry->tag,
};
@@ -2446,7 +2459,11 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass)
writel_relaxed(reg, smmu->base + ARM_SMMU_CR1);
/* CR2 (random crap) */
- reg = CR2_PTM | CR2_RECINVSID | CR2_E2H;
+ reg = CR2_PTM | CR2_RECINVSID;
+
+ if (smmu->features & ARM_SMMU_FEAT_E2H)
+ reg |= CR2_E2H;
+
writel_relaxed(reg, smmu->base + ARM_SMMU_CR2);
/* Stream table */
@@ -2594,8 +2611,11 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
if (reg & IDR0_MSI)
smmu->features |= ARM_SMMU_FEAT_MSI;
- if (reg & IDR0_HYP)
+ if (reg & IDR0_HYP) {
smmu->features |= ARM_SMMU_FEAT_HYP;
+ if (cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN))
+ smmu->features |= ARM_SMMU_FEAT_E2H;
+ }
/*
* The coherency feature as set by FW is used in preference to the ID
--
2.17.0
^ permalink raw reply related
* [PATCH v2 24/40] iommu/arm-smmu-v3: Seize private ASID
From: Jean-Philippe Brucker @ 2018-05-11 19:06 UTC (permalink / raw)
To: linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-acpi-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
kvm-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg
Cc: xuzaibo-hv44wF8Li93QT0dZR+AlfA, will.deacon-5wv7dgnIgG8,
okaya-sgV2jX0FEOL9JmXXK+q4OQ, ashok.raj-ral2JQCrhuEAvxtiuMwx3w,
bharatku-gjFFaj9aHVfQT0dZR+AlfA, rfranz-YGCgFSpz5w/QT0dZR+AlfA,
rgummal-gjFFaj9aHVfQT0dZR+AlfA,
ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A,
dwmw2-wEGCiKHe2LqWVfeAwA7xHQ, christian.koenig-5C7GfCeVMHo
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
The SMMU has a single ASID space, the union of shared and private ASID
sets. This means that the context table module competes with the arch
allocator for ASIDs. Shared ASIDs are those of Linux processes, allocated
by the arch, and contribute in broadcast TLB maintenance. Private ASIDs
are allocated by the SMMU driver and used for "classic" map/unmap DMA.
They require explicit TLB invalidations.
When we pin down an mm_context and get an ASID that is already in use by
the SMMU, it belongs to a private context. We used to simply abort the
bind, but this is unfair to users that would be unable to bind a few
seemingly random processes. Try to allocate a new private ASID for the
context in use, and make the old ASID shared.
Introduce a new lock to prevent races when rewriting context descriptors.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>
---
drivers/iommu/arm-smmu-v3-context.c | 90 ++++++++++++++++++++++++++---
1 file changed, 83 insertions(+), 7 deletions(-)
diff --git a/drivers/iommu/arm-smmu-v3-context.c b/drivers/iommu/arm-smmu-v3-context.c
index 352cba3c1a62..0e12f6804e16 100644
--- a/drivers/iommu/arm-smmu-v3-context.c
+++ b/drivers/iommu/arm-smmu-v3-context.c
@@ -65,6 +65,8 @@
#define ARM_SMMU_TCR2CD(tcr, fld) FIELD_PREP(CTXDESC_CD_0_TCR_##fld, \
FIELD_GET(ARM64_TCR_##fld, tcr))
+#define ARM_SMMU_NO_PASID (-1)
+
struct arm_smmu_cd {
struct iommu_pasid_entry entry;
@@ -72,8 +74,14 @@ struct arm_smmu_cd {
u64 tcr;
u64 mair;
+ int pasid;
+
+ /* 'refs' tracks alloc/free */
refcount_t refs;
+ /* 'users' tracks attach/detach, and is only used for sanity checking */
+ unsigned int users;
struct mm_struct *mm;
+ struct arm_smmu_cd_tables *tbl;
};
#define pasid_entry_to_cd(entry) \
@@ -105,6 +113,7 @@ struct arm_smmu_cd_tables {
#define pasid_ops_to_tables(ops) \
pasid_to_cd_tables(iommu_pasid_table_ops_to_table(ops))
+static DEFINE_SPINLOCK(contexts_lock);
static DEFINE_SPINLOCK(asid_lock);
static DEFINE_IDR(asid_idr);
@@ -191,8 +200,8 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr)
return val;
}
-static int arm_smmu_write_ctx_desc(struct arm_smmu_cd_tables *tbl, int ssid,
- struct arm_smmu_cd *cd)
+static int __arm_smmu_write_ctx_desc(struct arm_smmu_cd_tables *tbl, int ssid,
+ struct arm_smmu_cd *cd)
{
u64 val;
bool cd_live;
@@ -262,6 +271,18 @@ static int arm_smmu_write_ctx_desc(struct arm_smmu_cd_tables *tbl, int ssid,
return 0;
}
+static int arm_smmu_write_ctx_desc(struct arm_smmu_cd_tables *tbl, int ssid,
+ struct arm_smmu_cd *cd)
+{
+ int ret;
+
+ spin_lock(&contexts_lock);
+ ret = __arm_smmu_write_ctx_desc(tbl, ssid, cd);
+ spin_unlock(&contexts_lock);
+
+ return ret;
+}
+
static bool arm_smmu_free_asid(struct arm_smmu_cd *cd)
{
bool free;
@@ -301,15 +322,26 @@ static struct arm_smmu_cd *arm_smmu_alloc_cd(struct arm_smmu_cd_tables *tbl)
if (!cd)
return NULL;
- cd->entry.release = arm_smmu_free_cd;
+ cd->pasid = ARM_SMMU_NO_PASID;
+ cd->tbl = tbl;
+ cd->entry.release = arm_smmu_free_cd;
refcount_set(&cd->refs, 1);
return cd;
}
+/*
+ * Try to reserve this ASID in the SMMU. If it is in use, try to steal it from
+ * the private entry. Careful here, we may be modifying the context tables of
+ * another SMMU!
+ */
static struct arm_smmu_cd *arm_smmu_share_asid(u16 asid)
{
+ int ret;
struct arm_smmu_cd *cd;
+ struct arm_smmu_cd_tables *tbl;
+ struct arm_smmu_context_cfg *cfg;
+ struct iommu_pasid_entry old_entry;
cd = idr_find(&asid_idr, asid);
if (!cd)
@@ -319,17 +351,47 @@ static struct arm_smmu_cd *arm_smmu_share_asid(u16 asid)
/*
* It's pretty common to find a stale CD when doing unbind-bind,
* given that the release happens after a RCU grace period.
- * Simply reuse it.
+ * Simply reuse it, but check that it isn't active, because it's
+ * going to be assigned a different PASID.
*/
+ if (WARN_ON(cd->users))
+ return ERR_PTR(-EINVAL);
+
refcount_inc(&cd->refs);
return cd;
}
+ tbl = cd->tbl;
+ cfg = &tbl->pasid.cfg.arm_smmu;
+
+ ret = idr_alloc_cyclic(&asid_idr, cd, 0, 1 << cfg->asid_bits,
+ GFP_ATOMIC);
+ if (ret < 0)
+ return ERR_PTR(-ENOSPC);
+
+ /* Save the previous ASID */
+ old_entry = cd->entry;
+
/*
- * Ouch, ASID is already in use for a private cd.
- * TODO: seize it, for the common good.
+ * Race with unmap; TLB invalidations will start targeting the new ASID,
+ * which isn't assigned yet. We'll do an invalidate-all on the old ASID
+ * later, so it doesn't matter.
*/
- return ERR_PTR(-EEXIST);
+ cd->entry.tag = ret;
+
+ /*
+ * Update ASID and invalidate CD in all associated masters. There will
+ * be some overlap between use of both ASIDs, until we invalidate the
+ * TLB.
+ */
+ arm_smmu_write_ctx_desc(tbl, cd->pasid, cd);
+
+ /* Invalidate TLB entries previously associated with that context */
+ iommu_pasid_flush_tlbs(&tbl->pasid, cd->pasid, &old_entry);
+
+ idr_remove(&asid_idr, asid);
+
+ return NULL;
}
static struct iommu_pasid_entry *
@@ -476,6 +538,15 @@ static int arm_smmu_set_cd(struct iommu_pasid_table_ops *ops, int pasid,
if (WARN_ON(pasid > (1 << tbl->pasid.cfg.order)))
return -EINVAL;
+ if (WARN_ON(cd->pasid != ARM_SMMU_NO_PASID && cd->pasid != pasid))
+ return -EEXIST;
+
+ /*
+ * There is a single cd structure for each address space, multiple
+ * devices may use the same in different tables.
+ */
+ cd->users++;
+ cd->pasid = pasid;
return arm_smmu_write_ctx_desc(tbl, pasid, cd);
}
@@ -488,6 +559,11 @@ static void arm_smmu_clear_cd(struct iommu_pasid_table_ops *ops, int pasid,
if (WARN_ON(pasid > (1 << tbl->pasid.cfg.order)))
return;
+ WARN_ON(cd->pasid != pasid);
+
+ if (!(--cd->users))
+ cd->pasid = ARM_SMMU_NO_PASID;
+
arm_smmu_write_ctx_desc(tbl, pasid, NULL);
/*
--
2.17.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox