From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
torvalds@linux-foundation.org, stable@vger.kernel.org
Cc: lwn@lwn.net, jslaby@suse.cz,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: Re: Linux 6.12.91
Date: Sat, 23 May 2026 13:46:19 +0200 [thread overview]
Message-ID: <2026052319-surreal-prewashed-de1a@gregkh> (raw)
In-Reply-To: <2026052319-chastity-viper-7530@gregkh>
diff --git a/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml b/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml
index 5f051c666cbe..9deaf132d0e9 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml
+++ b/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml
@@ -50,7 +50,7 @@ properties:
The 2nd cell contains the interrupt number for the interrupt type.
SPI interrupts are in the range [0-987]. PPI interrupts are in the
range [0-15]. Extended SPI interrupts are in the range [0-1023].
- Extended PPI interrupts are in the range [0-127].
+ Extended PPI interrupts are in the range [0-63].
The 3rd cell is the flags, encoded as follows:
bits[3:0] trigger type and level flags.
diff --git a/Documentation/devicetree/bindings/net/dsa/nxp,sja1105.yaml b/Documentation/devicetree/bindings/net/dsa/nxp,sja1105.yaml
index 9432565f4f5d..8bfa2ea579f0 100644
--- a/Documentation/devicetree/bindings/net/dsa/nxp,sja1105.yaml
+++ b/Documentation/devicetree/bindings/net/dsa/nxp,sja1105.yaml
@@ -131,8 +131,6 @@ allOf:
else:
properties:
spi-cpha: false
- required:
- - spi-cpol
unevaluatedProperties: false
diff --git a/Documentation/mm/hugetlbfs_reserv.rst b/Documentation/mm/hugetlbfs_reserv.rst
index 4914fbf07966..a49115db18c7 100644
--- a/Documentation/mm/hugetlbfs_reserv.rst
+++ b/Documentation/mm/hugetlbfs_reserv.rst
@@ -155,7 +155,7 @@ are enough free huge pages to accommodate the reservation. If there are,
the global reservation count resv_huge_pages is adjusted something like the
following::
- if (resv_needed <= (resv_huge_pages - free_huge_pages))
+ if (resv_needed <= (free_huge_pages - resv_huge_pages)
resv_huge_pages += resv_needed;
Note that the global lock hugetlb_lock is held when checking and adjusting
diff --git a/Documentation/networking/netconsole.rst b/Documentation/networking/netconsole.rst
index d55c2a22ec7a..56578486ff7e 100644
--- a/Documentation/networking/netconsole.rst
+++ b/Documentation/networking/netconsole.rst
@@ -45,7 +45,7 @@ following format::
r if present, prepend kernel version (release) to the message
src-port source for UDP packets (defaults to 6665)
src-ip source IP to use (interface address)
- dev network interface (eth0)
+ dev network interface name (eth0) or MAC address
tgt-port port for logging agent (6666)
tgt-ip IP address for logging agent
tgt-macaddr ethernet MAC address for logging agent (broadcast)
@@ -62,6 +62,10 @@ or using IPv6::
insmod netconsole netconsole=@/,@fd00:1:2:3::1/
+or using a MAC address to select the egress interface::
+
+ linux netconsole=4444@10.0.0.1/22:33:44:55:66:77,9353@10.0.0.2/12:34:56:78:9a:bc
+
It also supports logging to multiple remote agents by specifying
parameters for the multiple agents separated by semicolons and the
complete string enclosed in "quotes", thusly::
diff --git a/Makefile b/Makefile
index 701def9e6be4..8e8c4c74f6d2 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 6
PATCHLEVEL = 12
-SUBLEVEL = 90
+SUBLEVEL = 91
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arc/net/bpf_jit_arcv2.c b/arch/arc/net/bpf_jit_arcv2.c
index 6d989b6d88c6..7ee50aeae5a4 100644
--- a/arch/arc/net/bpf_jit_arcv2.c
+++ b/arch/arc/net/bpf_jit_arcv2.c
@@ -2427,7 +2427,7 @@ u8 arc_prologue(u8 *buf, u32 usage, u16 frame_size)
#ifdef ARC_BPF_JIT_DEBUG
if ((usage & BIT(ARC_R_FP)) && frame_size == 0) {
- pr_err("FP is being saved while there is no frame.");
+ pr_err("FP is being saved while there is no frame.\n");
BUG();
}
#endif
@@ -2454,7 +2454,7 @@ u8 arc_epilogue(u8 *buf, u32 usage, u16 frame_size)
#ifdef ARC_BPF_JIT_DEBUG
if ((usage & BIT(ARC_R_FP)) && frame_size == 0) {
- pr_err("FP is being saved while there is no frame.");
+ pr_err("FP is being saved while there is no frame.\n");
BUG();
}
#endif
@@ -2868,7 +2868,7 @@ u8 gen_jmp_64(u8 *buf, u8 rd, u8 rs, u8 cond, u32 curr_off, u32 targ_off)
break;
default:
#ifdef ARC_BPF_JIT_DEBUG
- pr_err("64-bit jump condition is not known.");
+ pr_err("64-bit jump condition is not known.\n");
BUG();
#endif
}
@@ -2948,7 +2948,7 @@ u8 gen_jmp_32(u8 *buf, u8 rd, u8 rs, u8 cond, u32 curr_off, u32 targ_off)
*/
if (cond >= ARC_CC_LAST) {
#ifdef ARC_BPF_JIT_DEBUG
- pr_err("32-bit jump condition is not known.");
+ pr_err("32-bit jump condition is not known.\n");
BUG();
#endif
return 0;
diff --git a/arch/arm/boot/dts/mediatek/mt7623.dtsi b/arch/arm/boot/dts/mediatek/mt7623.dtsi
index fd7a89cc337d..a60b1d6879ff 100644
--- a/arch/arm/boot/dts/mediatek/mt7623.dtsi
+++ b/arch/arm/boot/dts/mediatek/mt7623.dtsi
@@ -328,7 +328,7 @@ sysirq: interrupt-controller@10200100 {
efuse: efuse@10206000 {
compatible = "mediatek,mt7623-efuse",
- "mediatek,mt8173-efuse";
+ "mediatek,efuse";
reg = <0 0x10206000 0 0x1000>;
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm/boot/dts/nxp/imx/imx27-eukrea-cpuimx27.dtsi b/arch/arm/boot/dts/nxp/imx/imx27-eukrea-cpuimx27.dtsi
index c7e923584878..9f0e65526d5f 100644
--- a/arch/arm/boot/dts/nxp/imx/imx27-eukrea-cpuimx27.dtsi
+++ b/arch/arm/boot/dts/nxp/imx/imx27-eukrea-cpuimx27.dtsi
@@ -106,7 +106,7 @@ uart8250@3,200000 {
compatible = "ns8250";
clocks = <&clk14745600>;
fsl,weim-cs-timing = <0x0000d603 0x0d1d0d01 0x00d20000>;
- interrupts = <&gpio2 23 IRQ_TYPE_LEVEL_LOW>;
+ interrupts-extended = <&gpio2 23 IRQ_TYPE_LEVEL_LOW>;
reg = <3 0x200000 0x1000>;
reg-shift = <1>;
reg-io-width = <1>;
@@ -119,7 +119,7 @@ uart8250@3,400000 {
compatible = "ns8250";
clocks = <&clk14745600>;
fsl,weim-cs-timing = <0x0000d603 0x0d1d0d01 0x00d20000>;
- interrupts = <&gpio2 22 IRQ_TYPE_LEVEL_LOW>;
+ interrupts-extended = <&gpio2 22 IRQ_TYPE_LEVEL_LOW>;
reg = <3 0x400000 0x1000>;
reg-shift = <1>;
reg-io-width = <1>;
@@ -132,7 +132,7 @@ uart8250@3,800000 {
compatible = "ns8250";
clocks = <&clk14745600>;
fsl,weim-cs-timing = <0x0000d603 0x0d1d0d01 0x00d20000>;
- interrupts = <&gpio2 27 IRQ_TYPE_LEVEL_LOW>;
+ interrupts-extended = <&gpio2 27 IRQ_TYPE_LEVEL_LOW>;
reg = <3 0x800000 0x1000>;
reg-shift = <1>;
reg-io-width = <1>;
@@ -145,7 +145,7 @@ uart8250@3,1000000 {
compatible = "ns8250";
clocks = <&clk14745600>;
fsl,weim-cs-timing = <0x0000d603 0x0d1d0d01 0x00d20000>;
- interrupts = <&gpio2 30 IRQ_TYPE_LEVEL_LOW>;
+ interrupts-extended = <&gpio2 30 IRQ_TYPE_LEVEL_LOW>;
reg = <3 0x1000000 0x1000>;
reg-shift = <1>;
reg-io-width = <1>;
diff --git a/arch/arm/boot/dts/nxp/imx/imx27-eukrea-mbimxsd27-baseboard.dts b/arch/arm/boot/dts/nxp/imx/imx27-eukrea-mbimxsd27-baseboard.dts
index d78793601306..c71f80298330 100644
--- a/arch/arm/boot/dts/nxp/imx/imx27-eukrea-mbimxsd27-baseboard.dts
+++ b/arch/arm/boot/dts/nxp/imx/imx27-eukrea-mbimxsd27-baseboard.dts
@@ -76,7 +76,7 @@ ads7846@0 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_touch>;
reg = <0>;
- interrupts = <&gpio4 25 IRQ_TYPE_LEVEL_LOW>;
+ interrupts-extended = <&gpio4 25 IRQ_TYPE_LEVEL_LOW>;
spi-cpol;
spi-max-frequency = <1500000>;
ti,keep-vref-on;
diff --git a/arch/arm/mach-omap1/clock_data.c b/arch/arm/mach-omap1/clock_data.c
index c58d200e4816..5203b047deac 100644
--- a/arch/arm/mach-omap1/clock_data.c
+++ b/arch/arm/mach-omap1/clock_data.c
@@ -700,8 +700,8 @@ int __init omap1_clk_init(void)
/* Make sure UART clocks are enabled early */
if (cpu_is_omap16xx())
omap_writel(omap_readl(MOD_CONF_CTRL_0) |
- CONF_MOD_UART1_CLK_MODE_R |
- CONF_MOD_UART3_CLK_MODE_R, MOD_CONF_CTRL_0);
+ (1 << CONF_MOD_UART1_CLK_MODE_R) |
+ (1 << CONF_MOD_UART3_CLK_MODE_R), MOD_CONF_CTRL_0);
#endif
/* USB_REQ_EN will be disabled later if necessary (usb_dc_ck) */
diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index deeb8f292454..a900aa973885 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -1852,6 +1852,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx)
{
u64 val = (u32)imm | (u64)insn[1].imm << 32;
+ if (insn->src_reg == BPF_PSEUDO_FUNC)
+ goto notyet;
+
emit_a32_mov_i64(dst, val, ctx);
return 1;
@@ -2055,6 +2058,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx)
const s8 *r5 = bpf2a32[BPF_REG_5];
const u32 func = (u32)__bpf_call_base + (u32)imm;
+ if (insn->src_reg == BPF_PSEUDO_CALL)
+ goto notyet;
+
emit_a32_mov_r64(true, r0, r1, ctx);
emit_a32_mov_r64(true, r1, r2, ctx);
emit_push_r64(r5, ctx);
diff --git a/arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p230.dts b/arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p230.dts
index c1470416faad..36e97ed585ae 100644
--- a/arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p230.dts
+++ b/arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p230.dts
@@ -84,7 +84,8 @@ external_phy: ethernet-phy@0 {
reset-gpios = <&gpio GPIOZ_14 GPIO_ACTIVE_LOW>;
interrupt-parent = <&gpio_intc>;
- interrupts = <29 IRQ_TYPE_LEVEL_LOW>;
+ /* MAC_INTR on GPIOZ_15 */
+ interrupts = <25 IRQ_TYPE_LEVEL_LOW>;
eee-broken-1000t;
};
};
diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
index 97f2ed267d69..04c27ef91fb2 100644
--- a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi
@@ -750,9 +750,10 @@ i2c0: i2c@2000000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c0_scl>;
- pinctrl-1 = <&i2c0_scl_gpio>;
+ pinctrl-0 = <&i2c0_pins>;
+ pinctrl-1 = <&gpio0_3_2_pins>;
scl-gpios = <&gpio0 3 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio0 2 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -766,9 +767,10 @@ i2c1: i2c@2010000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c1_scl>;
- pinctrl-1 = <&i2c1_scl_gpio>;
+ pinctrl-0 = <&i2c1_pins>;
+ pinctrl-1 = <&gpio0_31_30_pins>;
scl-gpios = <&gpio0 31 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio0 30 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -782,9 +784,10 @@ i2c2: i2c@2020000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c2_scl>;
- pinctrl-1 = <&i2c2_scl_gpio>;
+ pinctrl-0 = <&i2c2_pins>;
+ pinctrl-1 = <&gpio0_29_28_pins>;
scl-gpios = <&gpio0 29 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio0 28 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -798,9 +801,10 @@ i2c3: i2c@2030000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c3_scl>;
- pinctrl-1 = <&i2c3_scl_gpio>;
+ pinctrl-0 = <&i2c3_pins>;
+ pinctrl-1 = <&gpio0_27_26_pins>;
scl-gpios = <&gpio0 27 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio0 26 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -814,9 +818,10 @@ i2c4: i2c@2040000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c4_scl>;
- pinctrl-1 = <&i2c4_scl_gpio>;
+ pinctrl-0 = <&i2c4_pins>;
+ pinctrl-1 = <&gpio0_25_24_pins>;
scl-gpios = <&gpio0 25 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio0 24 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -830,9 +835,10 @@ i2c5: i2c@2050000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c5_scl>;
- pinctrl-1 = <&i2c5_scl_gpio>;
+ pinctrl-0 = <&i2c5_pins>;
+ pinctrl-1 = <&gpio0_23_22_pins>;
scl-gpios = <&gpio0 23 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio0 22 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -846,9 +852,10 @@ i2c6: i2c@2060000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c6_scl>;
- pinctrl-1 = <&i2c6_scl_gpio>;
+ pinctrl-0 = <&i2c6_i2c7_pins>;
+ pinctrl-1 = <&gpio1_18_15_pins>;
scl-gpios = <&gpio1 16 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio1 15 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -862,9 +869,10 @@ i2c7: i2c@2070000 {
clocks = <&clockgen QORIQ_CLK_PLATFORM_PLL
QORIQ_CLK_PLL_DIV(16)>;
pinctrl-names = "default", "gpio";
- pinctrl-0 = <&i2c7_scl>;
- pinctrl-1 = <&i2c7_scl_gpio>;
+ pinctrl-0 = <&i2c6_i2c7_pins>;
+ pinctrl-1 = <&gpio1_18_15_pins>;
scl-gpios = <&gpio1 18 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
+ sda-gpios = <&gpio1 17 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
};
@@ -1709,92 +1717,159 @@ pinmux_i2crv: pinmux@70010012c {
pinctrl-single,register-width = <32>;
pinctrl-single,function-mask = <0x7>;
- i2c1_scl: i2c1-scl-pins {
- pinctrl-single,bits = <0x0 0 0x7>;
+ /* RCWSR12 */
+ i2c1_pins: iic2-i2c-pins {
+ pinctrl-single,bits = <0x0 0x0 0x7>;
};
- i2c1_scl_gpio: i2c1-scl-gpio-pins {
+ gpio0_31_30_pins: iic2-gpio-pins {
pinctrl-single,bits = <0x0 0x1 0x7>;
};
+ ftm0_ch10_pins: iic2-ftm-pins {
+ pinctrl-single,bits = <0x0 0x2 0x7>;
+ };
+
esdhc0_cd_wp_pins: iic2-sdhc-pins {
pinctrl-single,bits = <0x0 0x6 0x7>;
};
- i2c2_scl: i2c2-scl-pins {
- pinctrl-single,bits = <0x0 0 (0x7 << 3)>;
+ i2c2_pins: iic3-i2c-pins {
+ pinctrl-single,bits = <0x0 0x0 (0x7 << 3)>;
};
- i2c2_scl_gpio: i2c2-scl-gpio-pins {
+ gpio0_29_28_pins: iic3-gpio-pins {
pinctrl-single,bits = <0x0 (0x1 << 3) (0x7 << 3)>;
};
- i2c3_scl: i2c3-scl-pins {
- pinctrl-single,bits = <0x0 0 (0x7 << 6)>;
+ can0_pins: iic3-can-pins {
+ pinctrl-single,bits = <0x0 (0x2 << 3) (0x7 << 3)>;
+ };
+
+ event65_pins: iic3-event-pins {
+ pinctrl-single,bits = <0x0 (0x6 << 3) (0x7 << 3)>;
+ };
+
+ i2c3_pins: iic4-i2c-pins {
+ pinctrl-single,bits = <0x0 0x0 (0x7 << 6)>;
};
- i2c3_scl_gpio: i2c3-scl-gpio-pins {
+ gpio0_27_26_pins: iic4-gpio-pins {
pinctrl-single,bits = <0x0 (0x1 << 6) (0x7 << 6)>;
};
- i2c4_scl: i2c4-scl-pins {
- pinctrl-single,bits = <0x0 0 (0x7 << 9)>;
+ can1_pins: iic4-can-pins {
+ pinctrl-single,bits = <0x0 (0x2 << 6) (0x7 << 6)>;
+ };
+
+ event87_pins: iic4-event-pins {
+ pinctrl-single,bits = <0x0 (0x6 << 6) (0x7 << 6)>;
+ };
+
+ i2c4_pins: iic5-i2c-pins {
+ pinctrl-single,bits = <0x0 0x0 (0x7 << 9)>;
};
- i2c4_scl_gpio: i2c4-scl-gpio-pins {
+ gpio0_25_24_pins: iic5-gpio-pins {
pinctrl-single,bits = <0x0 (0x1 << 9) (0x7 << 9)>;
};
- i2c5_scl: i2c5-scl-pins {
- pinctrl-single,bits = <0x0 0 (0x7 << 12)>;
+ esdhc0_clksync_pins: iic5-sdhc-clk-pins {
+ pinctrl-single,bits = <0x0 (0x2 << 9) (0x7 << 9)>;
};
- i2c5_scl_gpio: i2c5-scl-gpio-pins {
+ dspi2_miso_mosi_pins: iic5-spi3-pins {
+ pinctrl-single,bits = <0x3 (0x2 << 9) (0x7 << 9)>;
+ };
+
+ i2c5_pins: iic6-i2c-pins {
+ pinctrl-single,bits = <0x0 0x0 (0x7 << 12)>;
+ };
+
+ gpio0_23_22_pins: iic6-gpio-pins {
pinctrl-single,bits = <0x0 (0x1 << 12) (0x7 << 12)>;
};
+ esdhc1_clksync_pins: iic6-sdhc-clk-pins {
+ pinctrl-single,bits = <0x0 (0x2 << 12) (0x7 << 12)>;
+ };
+
fspi_data74_pins: xspi1-data74-pins {
pinctrl-single,bits = <0x0 0x0 (0x7 << 15)>;
};
+ gpio1_31_28_pins: xspi1-data74-gpio-pins {
+ pinctrl-single,bits = <0x0 0x1 (0x7 << 15)>;
+ };
+
fspi_data30_pins: xspi1-data30-pins {
pinctrl-single,bits = <0x0 0x0 (0x7 << 18)>;
};
+ gpio1_27_24_pins: xspi1-data30-gpio-pins {
+ pinctrl-single,bits = <0x0 0x1 (0x7 << 18)>;
+ };
+
fspi_dqs_sck_cs10_pins: xspi1-base-pins {
pinctrl-single,bits = <0x0 0x0 (0x7 << 21)>;
};
+ gpio1_23_20_pins: xspi1-base-gpio-pins {
+ pinctrl-single,bits = <0x0 0x1 (0x7 << 21)>;
+ };
+
esdhc0_cmd_data30_clk_vsel_pins: sdhc1-base-sdhc-vsel-pins {
pinctrl-single,bits = <0x0 0x0 (0x7 << 24)>;
};
+ gpio0_21_15_pins: sdhc1-base-gpio-pins {
+ pinctrl-single,bits = <0x0 (0x1 << 24) (0x7 << 24)>;
+ };
+
+ dspi0_pins: sdhc1-base-spi1-pins {
+ pinctrl-single,bits = <0x0 (0x2 << 24) (0x7 << 24)>;
+ };
+
+ esdhc0_cmd_data30_clk_dspi2_cs0_pins: sdhc1-base-sdhc-spi3-pins {
+ pinctrl-single,bits = <0x0 (0x3 << 24) (0x7 << 24)>;
+ };
+
+ esdhc0_cmd_data30_clk_data4_pins: sdhc1-base-sdhc-data4-pins {
+ pinctrl-single,bits = <0x0 (0x4 << 24) (0x7 << 24)>;
+ };
+
+ esdhc0_dir_pins: sdhc1-dir-pins {
+ pinctrl-single,bits = <0x0 0x0 (0x7 << 27)>;
+ };
+
gpio0_14_12_pins: sdhc1-dir-gpio-pins {
pinctrl-single,bits = <0x0 (0x1 << 27) (0x7 << 27)>;
};
- i2c6_scl: i2c6-scl-pins {
- pinctrl-single,bits = <0x4 0x2 0x7>;
+ dspi2_cs31_pins: sdhc1-dir-spi3-pins {
+ pinctrl-single,bits = <0x0 (0x3 << 27) (0x7 << 27)>;
};
- i2c6_scl_gpio: i2c6-scl-gpio-pins {
- pinctrl-single,bits = <0x4 0x1 0x7>;
+ esdhc0_data75_pins: sdhc1-dir-sdhc-pins {
+ pinctrl-single,bits = <0x0 (0x4 << 27) (0x7 << 27)>;
};
- i2c7_scl: i2c7-scl-pins {
- pinctrl-single,bits = <0x4 0x2 0x7>;
+ /* RCWSR13 */
+ gpio1_18_15_pins: iic8-iic7-gpio-pins {
+ pinctrl-single,bits = <0x4 0x1 0x7>;
};
- i2c7_scl_gpio: i2c7-scl-gpio-pins {
- pinctrl-single,bits = <0x4 0x1 0x7>;
+ i2c6_i2c7_pins: iic8-iic7-i2c-pins {
+ pinctrl-single,bits = <0x4 0x2 0x7>;
};
- i2c0_scl: i2c0-scl-pins {
- pinctrl-single,bits = <0x8 0 (0x7 << 10)>;
+ /* RCWSR14 */
+ i2c0_pins: iic1-i2c-pins {
+ pinctrl-single,bits = <0x8 0x0 (0x1 << 10)>;
};
- i2c0_scl_gpio: i2c0-scl-gpio-pins {
- pinctrl-single,bits = <0x8 (0x1 << 10) (0x7 << 10)>;
+ gpio0_3_2_pins: iic1-gpio-pins {
+ pinctrl-single,bits = <0x8 (0x1 << 10) (0x1 << 10)>;
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.1.dtsi b/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.1.dtsi
index 5438923a905c..5dc12da72dd2 100644
--- a/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.1.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.1.dtsi
@@ -21,6 +21,7 @@ led-1 {
color = <LED_COLOR_ID_GREEN>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <1>;
gpios = <&lsio_gpio5 27 GPIO_ACTIVE_HIGH>;
};
@@ -29,6 +30,7 @@ led-2 {
color = <LED_COLOR_ID_RED>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <1>;
gpios = <&lsio_gpio5 29 GPIO_ACTIVE_HIGH>;
};
@@ -37,6 +39,7 @@ led-3 {
color = <LED_COLOR_ID_GREEN>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <2>;
gpios = <&lsio_gpio5 20 GPIO_ACTIVE_HIGH>;
};
@@ -45,6 +48,7 @@ led-4 {
color = <LED_COLOR_ID_RED>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <2>;
gpios = <&lsio_gpio5 21 GPIO_ACTIVE_HIGH>;
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.2.dtsi b/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.2.dtsi
index f6654fdcb147..f3111bf03a4d 100644
--- a/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.2.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8-apalis-ixora-v1.2.dtsi
@@ -21,6 +21,7 @@ led-1 {
color = <LED_COLOR_ID_GREEN>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <1>;
gpios = <&lsio_gpio5 27 GPIO_ACTIVE_HIGH>;
};
@@ -29,6 +30,7 @@ led-2 {
color = <LED_COLOR_ID_RED>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <1>;
gpios = <&lsio_gpio5 29 GPIO_ACTIVE_HIGH>;
};
@@ -37,6 +39,7 @@ led-3 {
color = <LED_COLOR_ID_GREEN>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <2>;
gpios = <&lsio_gpio5 20 GPIO_ACTIVE_HIGH>;
};
@@ -45,6 +48,7 @@ led-4 {
color = <LED_COLOR_ID_RED>;
default-state = "off";
function = LED_FUNCTION_STATUS;
+ function-enumerator = <2>;
gpios = <&lsio_gpio5 21 GPIO_ACTIVE_HIGH>;
};
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-emtop-som.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-emtop-som.dtsi
index 67d22d3768aa..507d1824d99d 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-emtop-som.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-emtop-som.dtsi
@@ -60,7 +60,7 @@ pmic@25 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pmic>;
interrupt-parent = <&gpio1>;
- interrupts = <3 IRQ_TYPE_EDGE_RISING>;
+ interrupts = <3 IRQ_TYPE_LEVEL_LOW>;
regulators {
buck1: BUCK1 {
@@ -194,7 +194,7 @@ MX8MM_IOMUXC_I2C1_SDA_I2C1_SDA 0x400001c3
pinctrl_pmic: emtop-pmic-grp {
fsl,pins = <
- MX8MM_IOMUXC_GPIO1_IO03_GPIO1_IO3 0x41
+ MX8MM_IOMUXC_GPIO1_IO03_GPIO1_IO3 0x141
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml.dtsi
index 8f58c84e14c8..d94a59715ee0 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml.dtsi
@@ -290,7 +290,7 @@ pinctrl_i2c1_gpio: i2c1gpiogrp {
};
pinctrl_pmic: pmicgrp {
- fsl,pins = <MX8MM_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x94>;
+ fsl,pins = <MX8MM_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x1d4>;
};
pinctrl_reg_usdhc2_vmmc: regusdhc2vmmcgrp {
diff --git a/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl.dtsi b/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl.dtsi
index 2d64b2c0b181..5cd81024269a 100644
--- a/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mn-tqma8mqnl.dtsi
@@ -293,7 +293,7 @@ pinctrl_i2c1_gpio: i2c1gpiogrp {
};
pinctrl_pmic: pmicgrp {
- fsl,pins = <MX8MN_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x84>;
+ fsl,pins = <MX8MN_IOMUXC_GPIO1_IO08_GPIO1_IO8 0x1c4>;
};
pinctrl_reg_usdhc2_vmmc: regusdhc2vmmcgrp {
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-data-modul-edm-sbc.dts b/arch/arm64/boot/dts/freescale/imx8mp-data-modul-edm-sbc.dts
index 16078ff60ef0..fa13662ca366 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-data-modul-edm-sbc.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-data-modul-edm-sbc.dts
@@ -900,7 +900,7 @@ MX8MP_IOMUXC_SAI3_RXFS__AUDIOMIX_PDM_BIT_STREAM00 0x0
pinctrl_pmic: pmic-grp {
fsl,pins = <
/* PMIC_nINT */
- MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x40000090
+ MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x1c0
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-debix-model-a.dts b/arch/arm64/boot/dts/freescale/imx8mp-debix-model-a.dts
index af02af9e5334..740cac4cb31d 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-debix-model-a.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-debix-model-a.dts
@@ -440,7 +440,7 @@ MX8MP_IOMUXC_SAI5_RXC__I2C6_SDA 0x400001c3
pinctrl_pmic: pmicirqgrp {
fsl,pins = <
- MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x41
+ MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x1c0
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a-bmb-08.dts b/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a-bmb-08.dts
index d241db3743a9..ed89d2ccb6ce 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a-bmb-08.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a-bmb-08.dts
@@ -452,7 +452,7 @@ MX8MP_IOMUXC_SAI1_RXD1__GPIO4_IO03 0x140
pinctrl_pmic: pmicgrp {
fsl,pins = <
- MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x41
+ MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x1c0
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a.dtsi
index 91094c227744..b31e8fe95ca7 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-debix-som-a.dtsi
@@ -241,7 +241,7 @@ MX8MP_IOMUXC_I2C4_SDA__I2C4_SDA 0x400001c3
pinctrl_pmic: pmicgrp {
fsl,pins = <
- MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x41
+ MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x1c0
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-dhcom-som.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-dhcom-som.dtsi
index 1141b26d6b6f..f2e6391ca3b1 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-dhcom-som.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-dhcom-som.dtsi
@@ -989,7 +989,7 @@ MX8MP_IOMUXC_SAI5_RXC__GPIO3_IO20 0x22
pinctrl_pmic: dhcom-pmic-grp {
fsl,pins = <
/* PMIC_nINT */
- MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x40000090
+ MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x1c0
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-evk.dts b/arch/arm64/boot/dts/freescale/imx8mp-evk.dts
index d26930f1a9e9..2f1ef1b188be 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-evk.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-evk.dts
@@ -947,7 +947,7 @@ MX8MP_IOMUXC_SD1_DATA5__GPIO2_IO07 0x40
pinctrl_pcie0_reg: pcie0reggrp {
fsl,pins = <
- MX8MP_IOMUXC_SD1_DATA4__GPIO2_IO06 0x40
+ MX8MP_IOMUXC_SD1_DATA4__GPIO2_IO06 0x140
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-icore-mx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-icore-mx8mp.dtsi
index a6319824ea2e..69558ffefa9a 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-icore-mx8mp.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-icore-mx8mp.dtsi
@@ -132,7 +132,7 @@ MX8MP_IOMUXC_I2C1_SDA__I2C1_SDA 0x400001c3
pinctrl_pmic: pmicgrp {
fsl,pins = <
- MX8MP_IOMUXC_NAND_CE0_B__GPIO3_IO01 0x41
+ MX8MP_IOMUXC_NAND_CE0_B__GPIO3_IO01 0x1c0
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-navqp.dts b/arch/arm64/boot/dts/freescale/imx8mp-navqp.dts
index 5fd1614982cd..128bc1e6dac5 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-navqp.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-navqp.dts
@@ -309,7 +309,7 @@ MX8MP_IOMUXC_I2C4_SDA__I2C4_SDA 0x400001c3
pinctrl_pmic: pmicgrp {
fsl,pins = <
- MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x41
+ MX8MP_IOMUXC_GPIO1_IO03__GPIO1_IO03 0x1c0
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mp-ras314.dts b/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mp-ras314.dts
index a122f2ed5f53..06c865c3a8cf 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mp-ras314.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mp-ras314.dts
@@ -833,8 +833,8 @@ pinctrl_tlv320aic3x04: tlv320aic3x04grp {
pinctrl_uart1: uart1grp {
fsl,pins = <MX8MP_IOMUXC_SAI2_RXFS__UART1_DCE_TX 0x14>,
<MX8MP_IOMUXC_SAI2_RXC__UART1_DCE_RX 0x14>,
- <MX8MP_IOMUXC_SAI2_RXD0__UART1_DTE_CTS 0x14>,
- <MX8MP_IOMUXC_SAI2_TXFS__UART1_DTE_RTS 0x14>;
+ <MX8MP_IOMUXC_SAI2_RXD0__UART1_DCE_RTS 0x14>,
+ <MX8MP_IOMUXC_SAI2_TXFS__UART1_DCE_CTS 0x14>;
};
pinctrl_uart1_gpio: uart1gpiogrp {
diff --git a/arch/arm64/boot/dts/freescale/imx8qm-mek.dts b/arch/arm64/boot/dts/freescale/imx8qm-mek.dts
index 19c8d7ce1d40..32a37e0d0b42 100644
--- a/arch/arm64/boot/dts/freescale/imx8qm-mek.dts
+++ b/arch/arm64/boot/dts/freescale/imx8qm-mek.dts
@@ -351,9 +351,17 @@ ptn5110: tcpc@51 {
usb_con1: connector {
compatible = "usb-c-connector";
label = "USB-C";
- power-role = "source";
+ power-role = "dual";
data-role = "dual";
+ try-power-role = "sink";
source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ /*
+ * Set operational current to 0mA as we don't want EN_SNK
+ * enable 12V VBUS switch when it work as a sink.
+ */
+ sink-pdos = <PDO_FIXED(5000, 0, PDO_FIXED_USB_COMM)>;
+ op-sink-microwatt = <0>;
+ self-powered;
ports {
#address-cells = <1>;
diff --git a/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts b/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts
index 936ba5ecdcac..447a6a4c3c7e 100644
--- a/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts
+++ b/arch/arm64/boot/dts/freescale/imx8qxp-mek.dts
@@ -176,9 +176,17 @@ ptn5110: tcpc@50 {
usb_con1: connector {
compatible = "usb-c-connector";
label = "USB-C";
- power-role = "source";
+ power-role = "dual";
data-role = "dual";
+ try-power-role = "sink";
source-pdos = <PDO_FIXED(5000, 3000, PDO_FIXED_USB_COMM)>;
+ /*
+ * Set operational current to 0mA as we don't want EN_SNK
+ * enable 12V VBUS switch when it work as a sink.
+ */
+ sink-pdos = <PDO_FIXED(5000, 0, PDO_FIXED_USB_COMM)>;
+ op-sink-microwatt = <0>;
+ self-powered;
ports {
#address-cells = <1>;
diff --git a/arch/arm64/boot/dts/marvell/armada-3720-uDPU.dtsi b/arch/arm64/boot/dts/marvell/armada-3720-uDPU.dtsi
index cd856c0aba71..12deacb741cc 100644
--- a/arch/arm64/boot/dts/marvell/armada-3720-uDPU.dtsi
+++ b/arch/arm64/boot/dts/marvell/armada-3720-uDPU.dtsi
@@ -161,7 +161,7 @@ ð1 {
&usb3 {
status = "okay";
phys = <&usb2_utmi_otg_phy>;
- phy-names = "usb2-utmi-otg-phy";
+ phy-names = "usb2-phy";
};
&uart0 {
diff --git a/arch/arm64/boot/dts/marvell/armada-37xx.dtsi b/arch/arm64/boot/dts/marvell/armada-37xx.dtsi
index 9603223dd761..16b2b7052950 100644
--- a/arch/arm64/boot/dts/marvell/armada-37xx.dtsi
+++ b/arch/arm64/boot/dts/marvell/armada-37xx.dtsi
@@ -375,7 +375,7 @@ usb3: usb@58000 {
interrupts = <GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&sb_periph_clk 12>;
phys = <&comphy0 0>, <&usb2_utmi_otg_phy>;
- phy-names = "usb3-phy", "usb2-utmi-otg-phy";
+ phy-names = "usb3-phy", "usb2-phy";
status = "disabled";
};
diff --git a/arch/arm64/boot/dts/mediatek/mt6795.dtsi b/arch/arm64/boot/dts/mediatek/mt6795.dtsi
index e5e269a660b1..5dd822d470e8 100644
--- a/arch/arm64/boot/dts/mediatek/mt6795.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt6795.dtsi
@@ -372,7 +372,7 @@ pio: pinctrl@10005000 {
<GIC_SPI 154 IRQ_TYPE_LEVEL_HIGH>;
gpio-controller;
#gpio-cells = <2>;
- gpio-ranges = <&pio 0 0 196>;
+ gpio-ranges = <&pio 0 0 197>;
interrupt-controller;
#interrupt-cells = <2>;
};
diff --git a/arch/arm64/boot/dts/mediatek/mt7981b.dtsi b/arch/arm64/boot/dts/mediatek/mt7981b.dtsi
index 5cbea9cd411f..63da296bebad 100644
--- a/arch/arm64/boot/dts/mediatek/mt7981b.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt7981b.dtsi
@@ -200,7 +200,7 @@ pio: pinctrl@11d00000 {
interrupt-controller;
interrupts = <GIC_SPI 225 IRQ_TYPE_LEVEL_HIGH>;
interrupt-parent = <&gic>;
- gpio-ranges = <&pio 0 0 56>;
+ gpio-ranges = <&pio 0 0 57>;
gpio-controller;
#gpio-cells = <2>;
#interrupt-cells = <2>;
diff --git a/arch/arm64/boot/dts/mediatek/mt7986a.dtsi b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
index 559990dcd1d1..05bd2938242f 100644
--- a/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
@@ -187,7 +187,7 @@ pio: pinctrl@1001f000 {
"iocfg_lb", "iocfg_tr", "iocfg_tl", "eint";
gpio-controller;
#gpio-cells = <2>;
- gpio-ranges = <&pio 0 0 100>;
+ gpio-ranges = <&pio 0 0 101>;
interrupt-controller;
interrupts = <GIC_SPI 225 IRQ_TYPE_LEVEL_HIGH>;
interrupt-parent = <&gic>;
diff --git a/arch/arm64/boot/dts/mediatek/mt8365.dtsi b/arch/arm64/boot/dts/mediatek/mt8365.dtsi
index 2bf8c9d02b6e..e9ec44ee72e2 100644
--- a/arch/arm64/boot/dts/mediatek/mt8365.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8365.dtsi
@@ -481,10 +481,9 @@ iommu: iommu@10205000 {
#iommu-cells = <1>;
};
- infracfg_nao: infracfg@1020e000 {
- compatible = "mediatek,mt8365-infracfg", "syscon";
+ infracfg_nao: syscon@1020e000 {
+ compatible = "mediatek,mt8365-infracfg-nao", "syscon";
reg = <0 0x1020e000 0 0x1000>;
- #clock-cells = <1>;
};
rng: rng@1020f000 {
diff --git a/arch/arm64/boot/dts/qcom/msm8953-xiaomi-daisy.dts b/arch/arm64/boot/dts/qcom/msm8953-xiaomi-daisy.dts
index 336b916729e4..4e59d7de9976 100644
--- a/arch/arm64/boot/dts/qcom/msm8953-xiaomi-daisy.dts
+++ b/arch/arm64/boot/dts/qcom/msm8953-xiaomi-daisy.dts
@@ -157,7 +157,7 @@ &pm8953_resin {
&pmi8950_wled {
qcom,current-limit-microamp = <20000>;
- qcom,num-strings = <2>;
+ qcom,num-strings = <3>;
status = "okay";
};
diff --git a/arch/arm64/boot/dts/qcom/msm8953-xiaomi-vince.dts b/arch/arm64/boot/dts/qcom/msm8953-xiaomi-vince.dts
index d46325e79917..c2a290bf493c 100644
--- a/arch/arm64/boot/dts/qcom/msm8953-xiaomi-vince.dts
+++ b/arch/arm64/boot/dts/qcom/msm8953-xiaomi-vince.dts
@@ -169,7 +169,7 @@ &pm8953_resin {
&pmi8950_wled {
qcom,current-limit-microamp = <20000>;
- qcom,ovp-millivolt = <29600>;
+ qcom,ovp-millivolt = <29500>;
qcom,num-strings = <2>;
qcom,external-pfet;
qcom,cabc;
diff --git a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi
index 617b17b2d7d9..c872d6442018 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi
@@ -147,6 +147,7 @@ vreg_l1a_0p875: ldo1 {
regulator-min-microvolt = <880000>;
regulator-max-microvolt = <880000>;
regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>;
+ regulator-boot-on;
};
vreg_l5a_0p8: ldo5 {
diff --git a/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts b/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
index 52b16a4fdc43..83dac3ca5331 100644
--- a/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
+++ b/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
@@ -946,12 +946,14 @@ &qup_uart1_cts {
* the Bluetooth module drives the pin in either
* direction or leaves the pin fully unpowered.
*/
+ /delete-property/ bias-disable;
bias-bus-hold;
};
&qup_uart1_rts {
/* We'll drive RTS, so no pull */
drive-strength = <2>;
+ /delete-property/ bias-pull-down;
bias-disable;
};
@@ -962,12 +964,14 @@ &qup_uart1_rx {
* in tri-state (module powered off or not driving the
* signal yet).
*/
+ /delete-property/ bias-disable;
bias-pull-up;
};
&qup_uart1_tx {
/* We'll drive TX, so no pull */
drive-strength = <2>;
+ /delete-property/ bias-pull-up;
bias-disable;
};
diff --git a/arch/arm64/boot/dts/qcom/sm8250.dtsi b/arch/arm64/boot/dts/qcom/sm8250.dtsi
index e17937f76806..9995fc515e11 100644
--- a/arch/arm64/boot/dts/qcom/sm8250.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8250.dtsi
@@ -664,6 +664,11 @@ cpu7_opp20: opp-2841600000 {
opp-hz = /bits/ 64 <2841600000>;
opp-peak-kBps = <8368000 51609600>;
};
+
+ cpu7_opp21: opp-3091200000 {
+ opp-hz = /bits/ 64 <3091200000>;
+ opp-peak-kBps = <8368000 51609600>;
+ };
};
firmware {
diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi
index 58ed68f534e5..f5be69e3be99 100644
--- a/arch/arm64/boot/dts/qcom/sm8450.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi
@@ -4274,7 +4274,7 @@ intc: interrupt-controller@17100000 {
gic_its: msi-controller@17140000 {
compatible = "arm,gic-v3-its";
- reg = <0x0 0x17140000 0x0 0x20000>;
+ reg = <0x0 0x17140000 0x0 0x40000>;
msi-controller;
#msi-cells = <1>;
};
@@ -4599,9 +4599,6 @@ sdhc_2: mmc@8804000 {
bus-width = <4>;
dma-coherent;
- /* Forbid SDR104/SDR50 - broken hw! */
- sdhci-caps-mask = <0x3 0x0>;
-
status = "disabled";
sdhc2_opp_table: opp-table {
diff --git a/arch/arm64/boot/dts/qcom/sm8550.dtsi b/arch/arm64/boot/dts/qcom/sm8550.dtsi
index cfdd30009015..38b15db0676c 100644
--- a/arch/arm64/boot/dts/qcom/sm8550.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8550.dtsi
@@ -2824,7 +2824,7 @@ sdhc_2: mmc@8804000 {
clocks = <&gcc GCC_SDCC2_AHB_CLK>,
<&gcc GCC_SDCC2_APPS_CLK>,
- <&rpmhcc RPMH_CXO_CLK>;
+ <&bi_tcxo_div2>;
clock-names = "iface", "core", "xo";
iommus = <&apps_smmu 0x540 0>;
qcom,dll-config = <0x0007642c>;
@@ -2838,9 +2838,6 @@ sdhc_2: mmc@8804000 {
bus-width = <4>;
dma-coherent;
- /* Forbid SDR104/SDR50 - broken hw! */
- sdhci-caps-mask = <0x3 0>;
-
status = "disabled";
sdhc2_opp_table: opp-table {
@@ -4418,7 +4415,7 @@ intc: interrupt-controller@17100000 {
gic_its: msi-controller@17140000 {
compatible = "arm,gic-v3-its";
- reg = <0 0x17140000 0 0x20000>;
+ reg = <0 0x17140000 0 0x40000>;
msi-controller;
#msi-cells = <1>;
};
diff --git a/arch/arm64/boot/dts/qcom/sm8650.dtsi b/arch/arm64/boot/dts/qcom/sm8650.dtsi
index 6763c750f680..6098206350f9 100644
--- a/arch/arm64/boot/dts/qcom/sm8650.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8650.dtsi
@@ -3423,7 +3423,7 @@ sdhc_2: mmc@8804000 {
clocks = <&gcc GCC_SDCC2_AHB_CLK>,
<&gcc GCC_SDCC2_APPS_CLK>,
- <&rpmhcc RPMH_CXO_CLK>;
+ <&bi_tcxo_div2>;
clock-names = "iface",
"core",
"xo";
@@ -3442,9 +3442,6 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
bus-width = <4>;
- /* Forbid SDR104/SDR50 - broken hw! */
- sdhci-caps-mask = <0x3 0>;
-
qcom,dll-config = <0x0007642c>;
qcom,ddr-config = <0x80040868>;
@@ -5142,7 +5139,7 @@ intc: interrupt-controller@17100000 {
gic_its: msi-controller@17140000 {
compatible = "arm,gic-v3-its";
- reg = <0 0x17140000 0 0x20000>;
+ reg = <0 0x17140000 0 0x40000>;
msi-controller;
#msi-cells = <1>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3566-lckfb-tspi.dts b/arch/arm64/boot/dts/rockchip/rk3566-lckfb-tspi.dts
index 7cd91f8000cb..419225a4806c 100644
--- a/arch/arm64/boot/dts/rockchip/rk3566-lckfb-tspi.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3566-lckfb-tspi.dts
@@ -635,10 +635,10 @@ &uart1 {
status = "okay";
bluetooth: bluetooth {
- compatible = "brcm,bcm43438-bt";
+ compatible = "brcm,bcm43430a1-bt";
clocks = <&rk809 1>;
clock-names = "lpo";
- max-speed = <3000000>;
+ max-speed = <1500000>;
pinctrl-names = "default";
pinctrl-0 = <&bt_host_wake_l &bt_wake_l &bt_enable_h>;
shutdown-gpios = <&gpio2 RK_PB7 GPIO_ACTIVE_HIGH>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3588-jaguar.dts b/arch/arm64/boot/dts/rockchip/rk3588-jaguar.dts
index e61c5731fb99..46834d9ae565 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588-jaguar.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3588-jaguar.dts
@@ -424,7 +424,7 @@ led1_pin: led1-pin {
pcie30x4 {
pcie30x4_clkreqn_m0: pcie30x4-clkreqn-m0 {
- rockchip,pins = <0 RK_PC6 RK_FUNC_GPIO &pcfg_pull_none>;
+ rockchip,pins = <0 RK_PC6 RK_FUNC_GPIO &pcfg_pull_up>;
};
pcie30x4_perstn_m0: pcie30x4-perstn-m0 {
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s-gameforce-ace.dts b/arch/arm64/boot/dts/rockchip/rk3588s-gameforce-ace.dts
index 467f69594089..b9a17108e1ac 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s-gameforce-ace.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3588s-gameforce-ace.dts
@@ -59,8 +59,8 @@ axis@0 {
reg = <0>;
abs-flat = <40>;
abs-fuzz = <30>;
- abs-range = <0 4095>;
- linux,code = <ABS_RX>;
+ abs-range = <4095 0>;
+ linux,code = <ABS_RY>;
};
axis@1 {
@@ -68,7 +68,7 @@ axis@1 {
abs-flat = <40>;
abs-fuzz = <30>;
abs-range = <0 4095>;
- linux,code = <ABS_RY>;
+ linux,code = <ABS_RX>;
};
axis@2 {
@@ -76,7 +76,7 @@ axis@2 {
abs-flat = <40>;
abs-fuzz = <30>;
abs-range = <0 4095>;
- linux,code = <ABS_Y>;
+ linux,code = <ABS_X>;
};
axis@3 {
@@ -84,7 +84,7 @@ axis@3 {
abs-flat = <40>;
abs-fuzz = <30>;
abs-range = <0 4095>;
- linux,code = <ABS_X>;
+ linux,code = <ABS_Y>;
};
};
@@ -303,7 +303,7 @@ pwm_fan: pwm-fan {
compatible = "pwm-fan";
#cooling-cells = <2>;
cooling-levels = <0 120 150 180 210 240 255>;
- fan-supply = <&vcc5v0_sys>;
+ fan-supply = <&vcc5v0_spk>;
interrupt-parent = <&gpio4>;
interrupts = <RK_PB2 IRQ_TYPE_EDGE_RISING>;
pulses-per-revolution = <4>;
diff --git a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
index 4609f366006e..f050886e67c6 100644
--- a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
@@ -80,13 +80,13 @@ main_mmc0_pins_default: main-mmc0-default-pins {
AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (V3) MMC0_CMD */
AM62X_IOPAD(0x218, PIN_INPUT, 0) /* (Y1) MMC0_CLK */
AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (V2) MMC0_DAT0 */
- AM62X_IOPAD(0x210, PIN_INPUT, 0) /* (V1) MMC0_DAT1 */
- AM62X_IOPAD(0x20c, PIN_INPUT, 0) /* (W2) MMC0_DAT2 */
- AM62X_IOPAD(0x208, PIN_INPUT, 0) /* (W1) MMC0_DAT3 */
- AM62X_IOPAD(0x204, PIN_INPUT, 0) /* (Y2) MMC0_DAT4 */
- AM62X_IOPAD(0x200, PIN_INPUT, 0) /* (W3) MMC0_DAT5 */
- AM62X_IOPAD(0x1fc, PIN_INPUT, 0) /* (W4) MMC0_DAT6 */
- AM62X_IOPAD(0x1f8, PIN_INPUT, 0) /* (V4) MMC0_DAT7 */
+ AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (V1) MMC0_DAT1 */
+ AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (W2) MMC0_DAT2 */
+ AM62X_IOPAD(0x208, PIN_INPUT_PULLUP, 0) /* (W1) MMC0_DAT3 */
+ AM62X_IOPAD(0x204, PIN_INPUT_PULLUP, 0) /* (Y2) MMC0_DAT4 */
+ AM62X_IOPAD(0x200, PIN_INPUT_PULLUP, 0) /* (W3) MMC0_DAT5 */
+ AM62X_IOPAD(0x1fc, PIN_INPUT_PULLUP, 0) /* (W4) MMC0_DAT6 */
+ AM62X_IOPAD(0x1f8, PIN_INPUT_PULLUP, 0) /* (V4) MMC0_DAT7 */
>;
};
diff --git a/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi b/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi
index 7c90a4e488a4..893834c8803b 100644
--- a/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi
@@ -272,7 +272,7 @@ AM62X_IOPAD(0x0018, PIN_INPUT, 7) /* (F24) OSPI0_D3.GPIO0_6 */ /* SODIMM 62 */
};
/* Verdin SPI_1 CS as GPIO */
- pinctrl_qspi1_io4_gpio: main-gpio0-7-default-pins {
+ pinctrl_spi1_cs_gpio: main-gpio0-7-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x001c, PIN_INPUT, 7) /* (J23) OSPI0_D4.GPIO0_7 */ /* SODIMM 202 */
>;
diff --git a/arch/arm64/boot/dts/ti/k3-am62p5-sk.dts b/arch/arm64/boot/dts/ti/k3-am62p5-sk.dts
index b94093a7a392..de3a999aab13 100644
--- a/arch/arm64/boot/dts/ti/k3-am62p5-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am62p5-sk.dts
@@ -224,9 +224,9 @@ main_mmc1_pins_default: main-mmc1-default-pins {
AM62PX_IOPAD(0x023c, PIN_INPUT, 0) /* (H20) MMC1_CMD */
AM62PX_IOPAD(0x0234, PIN_OUTPUT, 0) /* (J24) MMC1_CLK */
AM62PX_IOPAD(0x0230, PIN_INPUT, 0) /* (H21) MMC1_DAT0 */
- AM62PX_IOPAD(0x022c, PIN_INPUT_PULLUP, 0) /* (H23) MMC1_DAT1 */
- AM62PX_IOPAD(0x0228, PIN_INPUT_PULLUP, 0) /* (H22) MMC1_DAT2 */
- AM62PX_IOPAD(0x0224, PIN_INPUT_PULLUP, 0) /* (H25) MMC1_DAT3 */
+ AM62PX_IOPAD(0x022c, PIN_INPUT, 0) /* (H23) MMC1_DAT1 */
+ AM62PX_IOPAD(0x0228, PIN_INPUT, 0) /* (H22) MMC1_DAT2 */
+ AM62PX_IOPAD(0x0224, PIN_INPUT, 0) /* (H25) MMC1_DAT3 */
AM62PX_IOPAD(0x0240, PIN_INPUT, 0) /* (D23) MMC1_SDCD */
>;
bootph-all;
diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index bf05a77873a4..56fe12a6f7b0 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -69,7 +69,12 @@
#define KERNEL_SEGMENT_COUNT 5
#if SWAPPER_BLOCK_SIZE > SEGMENT_ALIGN
-#define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 1)
+/*
+ * KERNEL_SEGMENT_COUNT counts the permanent kernel VMAs. The early mapping
+ * has one additional split, [_text, _stext). Reserve one more page for the
+ * SWAPPER_BLOCK_SIZE-unaligned boundaries.
+ */
+#define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 2)
/*
* The initial ID map consists of the kernel image, mapped as two separate
* segments, and may appear misaligned wrt the swapper block size. This means
diff --git a/arch/arm64/include/asm/xor.h b/arch/arm64/include/asm/xor.h
index befcd8a7abc9..7c0320715719 100644
--- a/arch/arm64/include/asm/xor.h
+++ b/arch/arm64/include/asm/xor.h
@@ -13,7 +13,7 @@
#ifdef CONFIG_KERNEL_MODE_NEON
-extern struct xor_block_template const xor_block_inner_neon;
+extern struct xor_block_template xor_block_inner_neon __ro_after_init;
static void
xor_neon_2(unsigned long bytes, unsigned long * __restrict p1,
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5e68d65e675e..8246015fd2d7 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -533,7 +533,7 @@ static const struct arm64_ftr_bits ftr_id_aa64dfr0[] = {
* We can instantiate multiple PMU instances with different levels
* of support.
*/
- S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_EL1_PMUVer_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_AA64DFR0_EL1_PMUVer_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64DFR0_EL1_DebugVer_SHIFT, 4, 0x6),
ARM64_FTR_END,
};
@@ -677,7 +677,7 @@ static const struct arm64_ftr_bits ftr_id_pfr2[] = {
static const struct arm64_ftr_bits ftr_id_dfr0[] = {
/* [31:28] TraceFilt */
- S_ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_DFR0_EL1_PerfMon_SHIFT, 4, 0),
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_EXACT, ID_DFR0_EL1_PerfMon_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_EL1_MProfDbg_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_EL1_MMapTrc_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_DFR0_EL1_CopTrc_SHIFT, 4, 0),
diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index 6f121a0164a4..28df62051cc9 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -129,9 +129,6 @@ int machine_kexec_post_load(struct kimage *kimage)
}
/* Create a copy of the linear map */
- trans_pgd = kexec_page_alloc(kimage);
- if (!trans_pgd)
- return -ENOMEM;
rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END);
if (rc)
return rc;
diff --git a/arch/arm64/kernel/pi/patch-scs.c b/arch/arm64/kernel/pi/patch-scs.c
index be7050fdfbba..76e69df0085d 100644
--- a/arch/arm64/kernel/pi/patch-scs.c
+++ b/arch/arm64/kernel/pi/patch-scs.c
@@ -178,9 +178,9 @@ static int scs_handle_fde_frame(const struct eh_frame *frame,
loc += *opcode++ * code_alignment_factor;
loc += (*opcode++ << 8) * code_alignment_factor;
loc += (*opcode++ << 16) * code_alignment_factor;
- loc += (*opcode++ << 24) * code_alignment_factor;
+ loc += ((u64)*opcode++ << 24) * code_alignment_factor;
size -= 4;
- break;
+ break;
case DW_CFA_def_cfa:
case DW_CFA_offset_extended:
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 9310196e0a09..c852749405e0 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -33,8 +33,8 @@
#define ARENA_VM_START (MAX_BPF_JIT_REG + 5)
#define check_imm(bits, imm) do { \
- if ((((imm) > 0) && ((imm) >> (bits))) || \
- (((imm) < 0) && (~(imm) >> (bits)))) { \
+ if ((((imm) > 0) && ((imm) >> ((bits) - 1))) || \
+ (((imm) < 0) && (~(imm) >> ((bits) - 1)))) { \
pr_info("[%2d] imm=%d(0x%x) out of range\n", \
i, imm, imm); \
return -EINVAL; \
diff --git a/arch/loongarch/Kbuild b/arch/loongarch/Kbuild
index bfa21465d83a..604adaff2623 100644
--- a/arch/loongarch/Kbuild
+++ b/arch/loongarch/Kbuild
@@ -3,7 +3,7 @@ obj-y += mm/
obj-y += net/
obj-y += vdso/
-obj-$(CONFIG_KVM) += kvm/
+obj-$(subst m,y,$(CONFIG_KVM)) += kvm/
obj-$(CONFIG_BUILTIN_DTB) += boot/dts/
# for cleaning
diff --git a/arch/loongarch/include/asm/asm-prototypes.h b/arch/loongarch/include/asm/asm-prototypes.h
index 51f224bcfc65..50c66b2a817a 100644
--- a/arch/loongarch/include/asm/asm-prototypes.h
+++ b/arch/loongarch/include/asm/asm-prototypes.h
@@ -12,3 +12,23 @@ __int128_t __ashlti3(__int128_t a, int b);
__int128_t __ashrti3(__int128_t a, int b);
__int128_t __lshrti3(__int128_t a, int b);
#endif
+
+struct kvm_run;
+struct kvm_vcpu;
+struct loongarch_fpu;
+
+void kvm_exc_entry(void);
+int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu);
+
+void kvm_save_fpu(struct loongarch_fpu *fpu);
+void kvm_restore_fpu(struct loongarch_fpu *fpu);
+
+#ifdef CONFIG_CPU_HAS_LSX
+void kvm_save_lsx(struct loongarch_fpu *fpu);
+void kvm_restore_lsx(struct loongarch_fpu *fpu);
+#endif
+
+#ifdef CONFIG_CPU_HAS_LASX
+void kvm_save_lasx(struct loongarch_fpu *fpu);
+void kvm_restore_lasx(struct loongarch_fpu *fpu);
+#endif
diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index d6bb72424027..4f813fd89ac5 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -73,7 +73,6 @@ struct kvm_context {
struct kvm_world_switch {
int (*exc_entry)(void);
int (*enter_guest)(struct kvm_run *run, struct kvm_vcpu *vcpu);
- unsigned long page_order;
};
#define MAX_PGTABLE_LEVELS 4
@@ -317,8 +316,6 @@ void kvm_exc_entry(void);
int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu);
extern unsigned long vpid_mask;
-extern const unsigned long kvm_exception_size;
-extern const unsigned long kvm_enter_guest_size;
extern struct kvm_world_switch *kvm_loongarch_ops;
#define SW_GCSR (1 << 0)
diff --git a/arch/loongarch/kvm/Makefile b/arch/loongarch/kvm/Makefile
index 2e188e8f1468..cbc27edfd1a2 100644
--- a/arch/loongarch/kvm/Makefile
+++ b/arch/loongarch/kvm/Makefile
@@ -9,11 +9,12 @@ include $(srctree)/virt/kvm/Makefile.kvm
obj-$(CONFIG_KVM) += kvm.o
+obj-y += switch.o
+
kvm-y += exit.o
kvm-y += interrupt.o
kvm-y += main.o
kvm-y += mmu.o
-kvm-y += switch.o
kvm-y += timer.o
kvm-y += tlb.o
kvm-y += vcpu.o
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 34fad2c29ee6..550ce2b4df70 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -320,8 +320,7 @@ void kvm_arch_disable_virtualization_cpu(void)
static int kvm_loongarch_env_init(void)
{
- int cpu, order;
- void *addr;
+ int cpu;
struct kvm_context *context;
vmcs = alloc_percpu(struct kvm_context);
@@ -337,30 +336,8 @@ static int kvm_loongarch_env_init(void)
return -ENOMEM;
}
- /*
- * PGD register is shared between root kernel and kvm hypervisor.
- * So world switch entry should be in DMW area rather than TLB area
- * to avoid page fault reenter.
- *
- * In future if hardware pagetable walking is supported, we won't
- * need to copy world switch code to DMW area.
- */
- order = get_order(kvm_exception_size + kvm_enter_guest_size);
- addr = (void *)__get_free_pages(GFP_KERNEL, order);
- if (!addr) {
- free_percpu(vmcs);
- vmcs = NULL;
- kfree(kvm_loongarch_ops);
- kvm_loongarch_ops = NULL;
- return -ENOMEM;
- }
-
- memcpy(addr, kvm_exc_entry, kvm_exception_size);
- memcpy(addr + kvm_exception_size, kvm_enter_guest, kvm_enter_guest_size);
- flush_icache_range((unsigned long)addr, (unsigned long)addr + kvm_exception_size + kvm_enter_guest_size);
- kvm_loongarch_ops->exc_entry = addr;
- kvm_loongarch_ops->enter_guest = addr + kvm_exception_size;
- kvm_loongarch_ops->page_order = order;
+ kvm_loongarch_ops->exc_entry = (void *)kvm_exc_entry;
+ kvm_loongarch_ops->enter_guest = (void *)kvm_enter_guest;
vpid_mask = read_csr_gstat();
vpid_mask = (vpid_mask & CSR_GSTAT_GIDBIT) >> CSR_GSTAT_GIDBIT_SHIFT;
@@ -380,16 +357,10 @@ static int kvm_loongarch_env_init(void)
static void kvm_loongarch_env_exit(void)
{
- unsigned long addr;
-
if (vmcs)
free_percpu(vmcs);
if (kvm_loongarch_ops) {
- if (kvm_loongarch_ops->exc_entry) {
- addr = (unsigned long)kvm_loongarch_ops->exc_entry;
- free_pages(addr, kvm_loongarch_ops->page_order);
- }
kfree(kvm_loongarch_ops);
}
}
diff --git a/arch/loongarch/kvm/switch.S b/arch/loongarch/kvm/switch.S
index 42c9fc99dc7e..df8e5a705b71 100644
--- a/arch/loongarch/kvm/switch.S
+++ b/arch/loongarch/kvm/switch.S
@@ -7,6 +7,7 @@
#include <asm/asm.h>
#include <asm/asmmacro.h>
#include <asm/loongarch.h>
+#include <asm/page.h>
#include <asm/regdef.h>
#include <asm/unwind_hints.h>
@@ -108,8 +109,13 @@
* - is still in guest mode, such as pgd table/vmid registers etc,
* - will fix with hw page walk enabled in future
* load kvm_vcpu from reserved CSR KVM_VCPU_KS, and save a2 to KVM_TEMP_KS
+ *
+ * PGD register is shared between root kernel and kvm hypervisor.
+ * So world switch entry should be in DMW area rather than TLB area
+ * to avoid page fault re-enter.
*/
.text
+ .p2align PAGE_SHIFT
.cfi_sections .debug_frame
SYM_CODE_START(kvm_exc_entry)
UNWIND_HINT_END_OF_STACK
@@ -198,8 +204,8 @@ ret_to_host:
kvm_restore_host_gpr a2
jr ra
-SYM_INNER_LABEL(kvm_exc_entry_end, SYM_L_LOCAL)
SYM_CODE_END(kvm_exc_entry)
+EXPORT_SYMBOL_GPL(kvm_exc_entry)
/*
* int kvm_enter_guest(struct kvm_run *run, struct kvm_vcpu *vcpu)
@@ -223,8 +229,8 @@ SYM_FUNC_START(kvm_enter_guest)
/* Save kvm_vcpu to kscratch */
csrwr a1, KVM_VCPU_KS
kvm_switch_to_guest
-SYM_INNER_LABEL(kvm_enter_guest_end, SYM_L_LOCAL)
SYM_FUNC_END(kvm_enter_guest)
+EXPORT_SYMBOL_GPL(kvm_enter_guest)
SYM_FUNC_START(kvm_save_fpu)
fpu_save_csr a0 t1
@@ -232,6 +238,7 @@ SYM_FUNC_START(kvm_save_fpu)
fpu_save_cc a0 t1 t2
jr ra
SYM_FUNC_END(kvm_save_fpu)
+EXPORT_SYMBOL_GPL(kvm_save_fpu)
SYM_FUNC_START(kvm_restore_fpu)
fpu_restore_double a0 t1
@@ -239,6 +246,7 @@ SYM_FUNC_START(kvm_restore_fpu)
fpu_restore_cc a0 t1 t2
jr ra
SYM_FUNC_END(kvm_restore_fpu)
+EXPORT_SYMBOL_GPL(kvm_restore_fpu)
#ifdef CONFIG_CPU_HAS_LSX
SYM_FUNC_START(kvm_save_lsx)
@@ -247,6 +255,7 @@ SYM_FUNC_START(kvm_save_lsx)
lsx_save_data a0 t1
jr ra
SYM_FUNC_END(kvm_save_lsx)
+EXPORT_SYMBOL_GPL(kvm_save_lsx)
SYM_FUNC_START(kvm_restore_lsx)
lsx_restore_data a0 t1
@@ -254,6 +263,7 @@ SYM_FUNC_START(kvm_restore_lsx)
fpu_restore_csr a0 t1 t2
jr ra
SYM_FUNC_END(kvm_restore_lsx)
+EXPORT_SYMBOL_GPL(kvm_restore_lsx)
#endif
#ifdef CONFIG_CPU_HAS_LASX
@@ -263,6 +273,7 @@ SYM_FUNC_START(kvm_save_lasx)
lasx_save_data a0 t1
jr ra
SYM_FUNC_END(kvm_save_lasx)
+EXPORT_SYMBOL_GPL(kvm_save_lasx)
SYM_FUNC_START(kvm_restore_lasx)
lasx_restore_data a0 t1
@@ -270,10 +281,8 @@ SYM_FUNC_START(kvm_restore_lasx)
fpu_restore_csr a0 t1 t2
jr ra
SYM_FUNC_END(kvm_restore_lasx)
+EXPORT_SYMBOL_GPL(kvm_restore_lasx)
#endif
- .section ".rodata"
-SYM_DATA(kvm_exception_size, .quad kvm_exc_entry_end - kvm_exc_entry)
-SYM_DATA(kvm_enter_guest_size, .quad kvm_enter_guest_end - kvm_enter_guest)
#ifdef CONFIG_CPU_HAS_LBT
STACK_FRAME_NON_STANDARD kvm_restore_fpu
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index 601e569303e1..e6c0724109e3 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -66,11 +66,9 @@ void relocate_new_kernel(unsigned long indirection_page, unsigned long reboot_co
unsigned long start_address) __noreturn;
void kexec_copy_flush(struct kimage *image);
-#ifdef CONFIG_KEXEC_FILE
-extern const struct kexec_file_ops kexec_elf64_ops;
+#if defined(CONFIG_KEXEC_FILE) || defined(CONFIG_CRASH_DUMP)
#define ARCH_HAS_KIMAGE_ARCH
-
struct kimage_arch {
struct crash_mem *exclude_ranges;
@@ -78,6 +76,10 @@ struct kimage_arch {
void *backup_buf;
void *fdt;
};
+#endif
+
+#ifdef CONFIG_KEXEC_FILE
+extern const struct kexec_file_ops kexec_elf64_ops;
char *setup_kdump_cmdline(struct kimage *image, char *cmdline,
unsigned long cmdline_len);
@@ -141,6 +143,10 @@ int arch_crash_hotplug_support(struct kimage *image, unsigned long kexec_flags);
unsigned int arch_crash_get_elfcorehdr_size(void);
#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
+
+int machine_kexec_post_load(struct kimage *image);
+#define machine_kexec_post_load machine_kexec_post_load
+
#endif /* CONFIG_CRASH_HOTPLUG */
extern int crashing_cpu;
@@ -155,6 +161,8 @@ extern void default_machine_crash_shutdown(struct pt_regs *regs);
extern void crash_kexec_prepare(void);
extern void crash_kexec_secondary(struct pt_regs *regs);
+extern void sync_backup_region_phdr(struct kimage *image, Elf64_Ehdr *ehdr,
+ bool phdr_to_kimage);
static inline bool kdump_in_progress(void)
{
return crashing_cpu >= 0;
diff --git a/arch/powerpc/kexec/crash.c b/arch/powerpc/kexec/crash.c
index a325c1c02f96..e6539f213b3d 100644
--- a/arch/powerpc/kexec/crash.c
+++ b/arch/powerpc/kexec/crash.c
@@ -27,6 +27,7 @@
#include <asm/debug.h>
#include <asm/interrupt.h>
#include <asm/kexec_ranges.h>
+#include <asm/crashdump-ppc64.h>
/*
* The primary CPU waits a while for all secondary CPUs to enter. This is to
@@ -399,7 +400,68 @@ void default_machine_crash_shutdown(struct pt_regs *regs)
ppc_md.kexec_cpu_down(1, 0);
}
+#ifdef CONFIG_CRASH_DUMP
+/**
+ * sync_backup_region_phdr - synchronize backup region offset between
+ * kexec image and ELF core header.
+ * @image: Kexec image.
+ * @ehdr: ELF core header.
+ * @phdr_to_kimage: If true, read the offset from the ELF program header
+ * and update the kimage backup region. If false, update
+ * the ELF program header offset from the kimage backup
+ * region.
+ *
+ * Note: During kexec_load, this is called with phdr_to_kimage = true. For
+ * kexec_file_load and ELF core header recreation during memory hotplug
+ * events, it is called with phdr_to_kimage = false.
+ *
+ * Returns nothing.
+ */
+void sync_backup_region_phdr(struct kimage *image, Elf64_Ehdr *ehdr, bool phdr_to_kimage)
+{
+ Elf64_Phdr *phdr;
+ unsigned int i;
+
+ phdr = (Elf64_Phdr *)(ehdr + 1);
+ for (i = 0; i < ehdr->e_phnum; i++, phdr++) {
+ if (phdr->p_paddr == BACKUP_SRC_START) {
+ if (phdr_to_kimage)
+ image->arch.backup_start = phdr->p_offset;
+ else
+ phdr->p_offset = image->arch.backup_start;
+
+ kexec_dprintk("Backup region offset updated to 0x%lx\n",
+ image->arch.backup_start);
+ return;
+ }
+ }
+}
+#endif /* CONFIG_CRASH_DUMP */
+
#ifdef CONFIG_CRASH_HOTPLUG
+
+int machine_kexec_post_load(struct kimage *image)
+{
+ int i;
+ unsigned long mem;
+ unsigned char *ptr;
+
+ if (image->type != KEXEC_TYPE_CRASH)
+ return 0;
+
+ if (image->file_mode)
+ return 0;
+
+ for (i = 0; i < image->nr_segments; i++) {
+ mem = image->segment[i].mem;
+ ptr = (char *)__va(mem);
+
+ if (ptr && memcmp(ptr, ELFMAG, SELFMAG) == 0)
+ sync_backup_region_phdr(image, (Elf64_Ehdr *) ptr, true);
+ }
+ return 0;
+}
+
#undef pr_fmt
#define pr_fmt(fmt) "crash hp: " fmt
@@ -474,6 +536,8 @@ static void update_crash_elfcorehdr(struct kimage *image, struct memory_notify *
goto out;
}
+ sync_backup_region_phdr(image, (Elf64_Ehdr *) elfbuf, false);
+
ptr = __va(mem);
if (ptr) {
/* Temporarily invalidate the crash image while it is replaced */
diff --git a/arch/powerpc/kexec/file_load_64.c b/arch/powerpc/kexec/file_load_64.c
index 248a0f00a291..276233f73a53 100644
--- a/arch/powerpc/kexec/file_load_64.c
+++ b/arch/powerpc/kexec/file_load_64.c
@@ -557,33 +557,6 @@ static int load_backup_segment(struct kimage *image, struct kexec_buf *kbuf)
return 0;
}
-/**
- * update_backup_region_phdr - Update backup region's offset for the core to
- * export the region appropriately.
- * @image: Kexec image.
- * @ehdr: ELF core header.
- *
- * Assumes an exclusive program header is setup for the backup region
- * in the ELF headers
- *
- * Returns nothing.
- */
-static void update_backup_region_phdr(struct kimage *image, Elf64_Ehdr *ehdr)
-{
- Elf64_Phdr *phdr;
- unsigned int i;
-
- phdr = (Elf64_Phdr *)(ehdr + 1);
- for (i = 0; i < ehdr->e_phnum; i++) {
- if (phdr->p_paddr == BACKUP_SRC_START) {
- phdr->p_offset = image->arch.backup_start;
- kexec_dprintk("Backup region offset updated to 0x%lx\n",
- image->arch.backup_start);
- return;
- }
- }
-}
-
static unsigned int kdump_extra_elfcorehdr_size(struct crash_mem *cmem)
{
#if defined(CONFIG_CRASH_HOTPLUG) && defined(CONFIG_MEMORY_HOTPLUG)
@@ -628,7 +601,7 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf)
}
/* Fix the offset for backup region in the ELF header */
- update_backup_region_phdr(image, headers);
+ sync_backup_region_phdr(image, headers, false);
kbuf->buffer = headers;
kbuf->mem = KEXEC_BUF_MEM_UNKNOWN;
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index 34c0adb9fdbf..742aa58a7c7e 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -115,10 +115,9 @@ long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
struct iommu_table_group *table_group;
long i;
struct kvmppc_spapr_tce_iommu_table *stit;
- struct fd f;
+ CLASS(fd, f)(tablefd);
- f = fdget(tablefd);
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
rcu_read_lock();
@@ -130,16 +129,12 @@ long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
}
rcu_read_unlock();
- if (!found) {
- fdput(f);
+ if (!found)
return -EINVAL;
- }
table_group = iommu_group_get_iommudata(grp);
- if (WARN_ON(!table_group)) {
- fdput(f);
+ if (WARN_ON(!table_group))
return -EFAULT;
- }
for (i = 0; i < IOMMU_TABLE_GROUP_MAX_TABLES; ++i) {
struct iommu_table *tbltmp = table_group->tables[i];
@@ -160,10 +155,8 @@ long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
break;
}
}
- if (!tbl) {
- fdput(f);
+ if (!tbl)
return -EINVAL;
- }
rcu_read_lock();
list_for_each_entry_rcu(stit, &stt->iommu_tables, next) {
@@ -174,7 +167,6 @@ long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
/* stit is being destroyed */
iommu_tce_table_put(tbl);
rcu_read_unlock();
- fdput(f);
return -ENOTTY;
}
/*
@@ -182,7 +174,6 @@ long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
* its KVM reference counter and can return.
*/
rcu_read_unlock();
- fdput(f);
return 0;
}
rcu_read_unlock();
@@ -190,7 +181,6 @@ long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
stit = kzalloc(sizeof(*stit), GFP_KERNEL);
if (!stit) {
iommu_tce_table_put(tbl);
- fdput(f);
return -ENOMEM;
}
@@ -199,7 +189,6 @@ long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,
list_add_rcu(&stit->next, &stt->iommu_tables);
- fdput(f);
return 0;
}
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 4b6ce4f07bc2..5a95c3d473b0 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -1930,12 +1930,11 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
#endif
#ifdef CONFIG_KVM_MPIC
case KVM_CAP_IRQ_MPIC: {
- struct fd f;
+ CLASS(fd, f)(cap->args[0]);
struct kvm_device *dev;
r = -EBADF;
- f = fdget(cap->args[0]);
- if (!fd_file(f))
+ if (fd_empty(f))
break;
r = -EPERM;
@@ -1943,18 +1942,16 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
if (dev)
r = kvmppc_mpic_connect_vcpu(dev, vcpu, cap->args[1]);
- fdput(f);
break;
}
#endif
#ifdef CONFIG_KVM_XICS
case KVM_CAP_IRQ_XICS: {
- struct fd f;
+ CLASS(fd, f)(cap->args[0]);
struct kvm_device *dev;
r = -EBADF;
- f = fdget(cap->args[0]);
- if (!fd_file(f))
+ if (fd_empty(f))
break;
r = -EPERM;
@@ -1965,34 +1962,27 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
else
r = kvmppc_xics_connect_vcpu(dev, vcpu, cap->args[1]);
}
-
- fdput(f);
break;
}
#endif /* CONFIG_KVM_XICS */
#ifdef CONFIG_KVM_XIVE
case KVM_CAP_PPC_IRQ_XIVE: {
- struct fd f;
+ CLASS(fd, f)(cap->args[0]);
struct kvm_device *dev;
r = -EBADF;
- f = fdget(cap->args[0]);
- if (!fd_file(f))
+ if (fd_empty(f))
break;
r = -ENXIO;
- if (!xive_enabled()) {
- fdput(f);
+ if (!xive_enabled())
break;
- }
r = -EPERM;
dev = kvm_device_from_filp(fd_file(f));
if (dev)
r = kvmppc_xive_native_connect_vcpu(dev, vcpu,
cap->args[1]);
-
- fdput(f);
break;
}
#endif /* CONFIG_KVM_XIVE */
diff --git a/arch/powerpc/platforms/44x/warp.c b/arch/powerpc/platforms/44x/warp.c
index a5001d32f978..6f674f86dc85 100644
--- a/arch/powerpc/platforms/44x/warp.c
+++ b/arch/powerpc/platforms/44x/warp.c
@@ -293,6 +293,8 @@ static int pika_dtm_thread(void __iomem *fpga)
schedule_timeout(HZ);
}
+ put_device(&client->dev);
+
return 0;
}
diff --git a/arch/powerpc/platforms/cell/spu_syscalls.c b/arch/powerpc/platforms/cell/spu_syscalls.c
index cd7d42fc12a6..da4fad7fc8bf 100644
--- a/arch/powerpc/platforms/cell/spu_syscalls.c
+++ b/arch/powerpc/platforms/cell/spu_syscalls.c
@@ -64,12 +64,10 @@ SYSCALL_DEFINE4(spu_create, const char __user *, name, unsigned int, flags,
return -ENOSYS;
if (flags & SPU_CREATE_AFFINITY_SPU) {
- struct fd neighbor = fdget(neighbor_fd);
+ CLASS(fd, neighbor)(neighbor_fd);
ret = -EBADF;
- if (fd_file(neighbor)) {
+ if (!fd_empty(neighbor))
ret = calls->create_thread(name, flags, mode, fd_file(neighbor));
- fdput(neighbor);
- }
} else
ret = calls->create_thread(name, flags, mode, NULL);
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index bc65fa6dc155..6f1314490641 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -3329,8 +3329,7 @@ static void aen_host_forward(unsigned long si)
struct zpci_gaite *gaite;
struct kvm *kvm;
- gaite = (struct zpci_gaite *)aift->gait +
- (si * sizeof(struct zpci_gaite));
+ gaite = aift->gait + si;
if (gaite->count == 0)
return;
if (gaite->aisb != 0)
diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c
index ffa7739c7a28..e5d3c0793f61 100644
--- a/arch/s390/kvm/pci.c
+++ b/arch/s390/kvm/pci.c
@@ -292,8 +292,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,
phys_to_virt(fib->fmt0.aibv));
spin_lock_irq(&aift->gait_lock);
- gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb *
- sizeof(struct zpci_gaite));
+ gaite = aift->gait + zdev->aisb;
/* If assist not requested, host will get all alerts */
if (assist)
@@ -359,8 +358,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force)
if (zdev->kzdev->fib.fmt0.aibv == 0)
goto out;
spin_lock_irq(&aift->gait_lock);
- gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb *
- sizeof(struct zpci_gaite));
+ gaite = aift->gait + zdev->aisb;
isc = gaite->gisc;
gaite->count--;
if (gaite->count == 0) {
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index f305cb42070d..c3ad3cf86ca6 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -842,25 +842,34 @@ static int bpf_jit_probe_post(struct bpf_jit *jit, struct bpf_prog *fp,
}
/*
- * Sign-extend the register if necessary
+ * Sign- or zero-extend the register if necessary
*/
-static int sign_extend(struct bpf_jit *jit, int r, u8 size, u8 flags)
+static int sign_zero_extend(struct bpf_jit *jit, int r, u8 size, u8 flags)
{
- if (!(flags & BTF_FMODEL_SIGNED_ARG))
- return 0;
-
switch (size) {
case 1:
- /* lgbr %r,%r */
- EMIT4(0xb9060000, r, r);
+ if (flags & BTF_FMODEL_SIGNED_ARG)
+ /* lgbr %r,%r */
+ EMIT4(0xb9060000, r, r);
+ else
+ /* llgcr %r,%r */
+ EMIT4(0xb9840000, r, r);
return 0;
case 2:
- /* lghr %r,%r */
- EMIT4(0xb9070000, r, r);
+ if (flags & BTF_FMODEL_SIGNED_ARG)
+ /* lghr %r,%r */
+ EMIT4(0xb9070000, r, r);
+ else
+ /* llghr %r,%r */
+ EMIT4(0xb9850000, r, r);
return 0;
case 4:
- /* lgfr %r,%r */
- EMIT4(0xb9140000, r, r);
+ if (flags & BTF_FMODEL_SIGNED_ARG)
+ /* lgfr %r,%r */
+ EMIT4(0xb9140000, r, r);
+ else
+ /* llgfr %r,%r */
+ EMIT4(0xb9160000, r, r);
return 0;
case 8:
return 0;
@@ -1802,9 +1811,9 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
return -1;
for (j = 0; j < m->nr_args; j++) {
- if (sign_extend(jit, BPF_REG_1 + j,
- m->arg_size[j],
- m->arg_flags[j]))
+ if (sign_zero_extend(jit, BPF_REG_1 + j,
+ m->arg_size[j],
+ m->arg_flags[j]))
return -1;
}
}
@@ -2551,7 +2560,7 @@ static int invoke_bpf_prog(struct bpf_tramp_jit *tjit,
call_r1(jit);
/* stg %r2,retval_off(%r15) */
if (save_ret) {
- if (sign_extend(jit, REG_2, m->ret_size, m->ret_flags))
+ if (sign_zero_extend(jit, REG_2, m->ret_size, m->ret_flags))
return -1;
EMIT6_DISP_LH(0xe3000000, 0x0024, REG_2, REG_0, REG_15,
tjit->retval_off);
diff --git a/arch/sparc/vdso/Makefile b/arch/sparc/vdso/Makefile
index 243dbfc4609d..fc3ada4269f1 100644
--- a/arch/sparc/vdso/Makefile
+++ b/arch/sparc/vdso/Makefile
@@ -22,7 +22,7 @@ targets += $(foreach x, 32 64, vdso-image-$(x).c vdso$(x).so vdso$(x).so.dbg)
CPPFLAGS_vdso.lds += -P -C
-VDSO_LDFLAGS_vdso.lds = -m elf64_sparc -soname linux-vdso.so.1 --no-undefined \
+VDSO_LDFLAGS_vdso.lds = -m elf64_sparc -soname linux-vdso.so.1 \
-z max-page-size=8192
$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE
@@ -101,7 +101,6 @@ $(obj)/vdso32.so.dbg: FORCE \
quiet_cmd_vdso = VDSO $@
cmd_vdso = $(LD) -nostdlib -o $@ \
$(VDSO_LDFLAGS) $(VDSO_LDFLAGS_$(filter %.lds,$(^F))) \
- -T $(filter %.lds,$^) $(filter %.o,$^) && \
- sh $(src)/checkundef.sh '$(OBJDUMP)' '$@'
+ -T $(filter %.lds,$^) $(filter %.o,$^)
-VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 -Bsymbolic
+VDSO_LDFLAGS = -shared --hash-style=both --build-id=sha1 -Bsymbolic --no-undefined -z noexecstack
diff --git a/arch/sparc/vdso/checkundef.sh b/arch/sparc/vdso/checkundef.sh
deleted file mode 100644
index 2d85876ffc32..000000000000
--- a/arch/sparc/vdso/checkundef.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/sh
-objdump="$1"
-file="$2"
-$objdump -t "$file" | grep '*UUND*' | grep -v '#scratch' > /dev/null 2>&1
-if [ $? -eq 1 ]; then
- exit 0
-else
- echo "$file: undefined symbols found" >&2
- exit 1
-fi
diff --git a/arch/x86/Makefile.um b/arch/x86/Makefile.um
index c86cbd9cbba3..19c13afa474e 100644
--- a/arch/x86/Makefile.um
+++ b/arch/x86/Makefile.um
@@ -60,4 +60,6 @@ ELF_FORMAT := elf64-x86-64
LINK-$(CONFIG_LD_SCRIPT_DYN_RPATH) += -Wl,-rpath,/lib64
LINK-y += -m64
+vdso-install-y += arch/x86/um/vdso/vdso.so.dbg
+
endif
diff --git a/arch/x86/kernel/acpi/cppc.c b/arch/x86/kernel/acpi/cppc.c
index 147f0d8d54d8..12fc5f553405 100644
--- a/arch/x86/kernel/acpi/cppc.c
+++ b/arch/x86/kernel/acpi/cppc.c
@@ -86,19 +86,19 @@ static void amd_set_max_freq_ratio(void)
rc = cppc_get_perf_caps(0, &perf_caps);
if (rc) {
- pr_warn("Could not retrieve perf counters (%d)\n", rc);
+ pr_debug("Could not retrieve perf counters (%d)\n", rc);
return;
}
rc = amd_get_boost_ratio_numerator(0, &numerator);
if (rc) {
- pr_warn("Could not retrieve highest performance (%d)\n", rc);
+ pr_debug("Could not retrieve highest performance (%d)\n", rc);
return;
}
nominal_perf = perf_caps.nominal_perf;
if (!nominal_perf) {
- pr_warn("Could not retrieve nominal performance\n");
+ pr_debug("Could not retrieve nominal performance\n");
return;
}
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 147ea26dfdad..6fb1e7dd4595 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -903,19 +903,15 @@ static struct miscdevice sgx_dev_provision = {
int sgx_set_attribute(unsigned long *allowed_attributes,
unsigned int attribute_fd)
{
- struct fd f = fdget(attribute_fd);
+ CLASS(fd, f)(attribute_fd);
- if (!fd_file(f))
+ if (fd_empty(f))
return -EINVAL;
- if (fd_file(f)->f_op != &sgx_provision_fops) {
- fdput(f);
+ if (fd_file(f)->f_op != &sgx_provision_fops)
return -EINVAL;
- }
*allowed_attributes |= SGX_ATTR_PROVISIONKEY;
-
- fdput(f);
return 0;
}
EXPORT_SYMBOL_GPL(sgx_set_attribute);
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 7aedb0a06354..ad7590ec4065 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -533,17 +533,12 @@ static int sev_bind_asid(struct kvm *kvm, unsigned int handle, int *error)
static int __sev_issue_cmd(int fd, int id, void *data, int *error)
{
- struct fd f;
- int ret;
+ CLASS(fd, f)(fd);
- f = fdget(fd);
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
- ret = sev_issue_cmd_external_user(fd_file(f), id, data, error);
-
- fdput(f);
- return ret;
+ return sev_issue_cmd_external_user(fd_file(f), id, data, error);
}
static int sev_issue_cmd(struct kvm *kvm, int id, void *data, int *error)
@@ -2089,23 +2084,21 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
{
struct kvm_sev_info *dst_sev = &to_kvm_svm(kvm)->sev_info;
struct kvm_sev_info *src_sev, *cg_cleanup_sev;
- struct fd f = fdget(source_fd);
+ CLASS(fd, f)(source_fd);
struct kvm *source_kvm;
bool charged = false;
int ret;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
- if (!file_is_kvm(fd_file(f))) {
- ret = -EBADF;
- goto out_fput;
- }
+ if (!file_is_kvm(fd_file(f)))
+ return -EBADF;
source_kvm = fd_file(f)->private_data;
ret = sev_lock_two_vms(kvm, source_kvm);
if (ret)
- goto out_fput;
+ return ret;
if (kvm->arch.vm_type != source_kvm->arch.vm_type ||
sev_guest(kvm) || !sev_guest(source_kvm)) {
@@ -2152,8 +2145,6 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
cg_cleanup_sev->misc_cg = NULL;
out_unlock:
sev_unlock_two_vms(kvm, source_kvm);
-out_fput:
- fdput(f);
return ret;
}
@@ -2817,23 +2808,21 @@ int sev_mem_enc_unregister_region(struct kvm *kvm,
int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd)
{
- struct fd f = fdget(source_fd);
+ CLASS(fd, f)(source_fd);
struct kvm *source_kvm;
struct kvm_sev_info *source_sev, *mirror_sev;
int ret;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
- if (!file_is_kvm(fd_file(f))) {
- ret = -EBADF;
- goto e_source_fput;
- }
+ if (!file_is_kvm(fd_file(f)))
+ return -EBADF;
source_kvm = fd_file(f)->private_data;
ret = sev_lock_two_vms(kvm, source_kvm);
if (ret)
- goto e_source_fput;
+ return ret;
/*
* Mirrors of mirrors should work, but let's not get silly. Also
@@ -2876,8 +2865,6 @@ int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd)
e_unlock:
sev_unlock_two_vms(kvm, source_kvm);
-e_source_fput:
- fdput(f);
return ret;
}
diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
index d3aeffd6ae75..efa715d77e58 100644
--- a/arch/x86/kvm/trace.h
+++ b/arch/x86/kvm/trace.h
@@ -140,7 +140,7 @@ TRACE_EVENT(kvm_xen_hypercall,
__entry->a2 = a2;
__entry->a3 = a3;
__entry->a4 = a4;
- __entry->a4 = a5;
+ __entry->a5 = a5;
),
TP_printk("cpl %d nr 0x%lx a0 0x%lx a1 0x%lx a2 0x%lx a3 0x%lx a4 0x%lx a5 %lx",
diff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile
index 6a77ea6434ff..bde42fac402c 100644
--- a/arch/x86/um/vdso/Makefile
+++ b/arch/x86/um/vdso/Makefile
@@ -3,16 +3,11 @@
# Building vDSO images for x86.
#
-VDSO64-y := y
-
-vdso-install-$(VDSO64-y) += vdso.so
-
-
# files to link into the vdso
vobjs-y := vdso-note.o um_vdso.o
# files to link into kernel
-obj-$(VDSO64-y) += vdso.o vma.o
+obj-y += vdso.o vma.o
vobjs := $(foreach F,$(vobjs-y),$(obj)/$F)
diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index 3f7cb9d891aa..a0fbb427a7a6 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -24,6 +24,7 @@
#include <linux/backing-dev.h>
#include <linux/slab.h>
#include <linux/delay.h>
+#include <linux/wait_bit.h>
#include <linux/atomic.h>
#include <linux/ctype.h>
#include <linux/resume_user_mode.h>
@@ -611,6 +612,8 @@ static void blkg_destroy_all(struct gendisk *disk)
q->root_blkg = NULL;
spin_unlock_irq(&q->queue_lock);
+
+ wake_up_var(&q->root_blkg);
}
static void blkg_iostat_set(struct blkg_iostat *dst, struct blkg_iostat *src)
@@ -1451,6 +1454,18 @@ int blkcg_init_disk(struct gendisk *disk)
struct blkcg_gq *new_blkg, *blkg;
bool preloaded;
+ /*
+ * If the queue is shared across disk rebind (e.g., SCSI), the
+ * previous disk's blkcg state is cleaned up asynchronously via
+ * disk_release() -> blkcg_exit_disk(). Wait for that cleanup to
+ * finish (indicated by root_blkg becoming NULL) before setting up
+ * new blkcg state. Otherwise, we may overwrite q->root_blkg while
+ * the old one is still alive, and radix_tree_insert() in
+ * blkg_create() will fail with -EEXIST because the old entries
+ * still occupy the same queue id slot in blkcg->blkg_tree.
+ */
+ wait_var_event(&q->root_blkg, !READ_ONCE(q->root_blkg));
+
new_blkg = blkg_alloc(&blkcg_root, disk, GFP_KERNEL);
if (!new_blkg)
return -ENOMEM;
@@ -1970,6 +1985,7 @@ void blkcg_maybe_throttle_current(void)
return;
out:
rcu_read_unlock();
+ put_disk(disk);
}
/**
diff --git a/block/disk-events.c b/block/disk-events.c
index 2f697224386a..868823915bdc 100644
--- a/block/disk-events.c
+++ b/block/disk-events.c
@@ -290,13 +290,14 @@ EXPORT_SYMBOL(disk_check_media_change);
* Should be called when the media changes for @disk. Generates a uevent
* and attempts to free all dentries and inodes and invalidates all block
* device page cache entries in that case.
+ *
+ * Callers that need a partition re-scan should arrange for one explicitly.
*/
void disk_force_media_change(struct gendisk *disk)
{
disk_event_uevent(disk, DISK_EVENT_MEDIA_CHANGE);
inc_diskseq(disk);
bdev_mark_dead(disk->part0, true);
- set_bit(GD_NEED_PART_SCAN, &disk->state);
}
EXPORT_SYMBOL_GPL(disk_force_media_change);
diff --git a/crypto/af_alg.c b/crypto/af_alg.c
index b61c3ba126ed..0111d0754593 100644
--- a/crypto/af_alg.c
+++ b/crypto/af_alg.c
@@ -586,6 +586,8 @@ static int af_alg_cmsg_send(struct msghdr *msg, struct af_alg_control *con)
if (cmsg->cmsg_len < CMSG_LEN(sizeof(u32)))
return -EINVAL;
con->aead_assoclen = *(u32 *)CMSG_DATA(cmsg);
+ if (con->aead_assoclen >= 0x80000000u)
+ return -EINVAL;
break;
default:
diff --git a/crypto/jitterentropy-kcapi.c b/crypto/jitterentropy-kcapi.c
index 1266eb790708..a9e9b8fe1598 100644
--- a/crypto/jitterentropy-kcapi.c
+++ b/crypto/jitterentropy-kcapi.c
@@ -42,6 +42,7 @@
#include <linux/fips.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/time.h>
#include <crypto/internal/rng.h>
@@ -192,7 +193,7 @@ int jent_read_random_block(void *hash_state, char *dst, unsigned int dst_len)
***************************************************************************/
struct jitterentropy {
- spinlock_t jent_lock;
+ struct mutex jent_lock;
struct rand_data *entropy_collector;
struct crypto_shash *tfm;
struct shash_desc *sdesc;
@@ -202,7 +203,7 @@ static void jent_kcapi_cleanup(struct crypto_tfm *tfm)
{
struct jitterentropy *rng = crypto_tfm_ctx(tfm);
- spin_lock(&rng->jent_lock);
+ mutex_lock(&rng->jent_lock);
if (rng->sdesc) {
shash_desc_zero(rng->sdesc);
@@ -217,7 +218,7 @@ static void jent_kcapi_cleanup(struct crypto_tfm *tfm)
if (rng->entropy_collector)
jent_entropy_collector_free(rng->entropy_collector);
rng->entropy_collector = NULL;
- spin_unlock(&rng->jent_lock);
+ mutex_unlock(&rng->jent_lock);
}
static int jent_kcapi_init(struct crypto_tfm *tfm)
@@ -227,7 +228,7 @@ static int jent_kcapi_init(struct crypto_tfm *tfm)
struct shash_desc *sdesc;
int size, ret = 0;
- spin_lock_init(&rng->jent_lock);
+ mutex_init(&rng->jent_lock);
/*
* Use SHA3-256 as conditioner. We allocate only the generic
@@ -264,7 +265,6 @@ static int jent_kcapi_init(struct crypto_tfm *tfm)
goto err;
}
- spin_lock_init(&rng->jent_lock);
return 0;
err:
@@ -279,7 +279,7 @@ static int jent_kcapi_random(struct crypto_rng *tfm,
struct jitterentropy *rng = crypto_rng_ctx(tfm);
int ret = 0;
- spin_lock(&rng->jent_lock);
+ mutex_lock(&rng->jent_lock);
ret = jent_read_entropy(rng->entropy_collector, rdata, dlen);
@@ -305,7 +305,7 @@ static int jent_kcapi_random(struct crypto_rng *tfm,
ret = -EINVAL;
}
- spin_unlock(&rng->jent_lock);
+ mutex_unlock(&rng->jent_lock);
return ret;
}
diff --git a/drivers/acpi/arm64/agdi.c b/drivers/acpi/arm64/agdi.c
index f5f21dd0d277..dccbd1c4dafc 100644
--- a/drivers/acpi/arm64/agdi.c
+++ b/drivers/acpi/arm64/agdi.c
@@ -32,7 +32,7 @@ static int agdi_sdei_probe(struct platform_device *pdev,
err = sdei_event_register(adata->sdei_event, agdi_sdei_handler, pdev);
if (err) {
- dev_err(&pdev->dev, "Failed to register for SDEI event %d",
+ dev_err(&pdev->dev, "Failed to register for SDEI event %d\n",
adata->sdei_event);
return err;
}
diff --git a/drivers/acpi/x86/cmos_rtc.c b/drivers/acpi/x86/cmos_rtc.c
index 51643ff6fe5f..45db7e51cbe6 100644
--- a/drivers/acpi/x86/cmos_rtc.c
+++ b/drivers/acpi/x86/cmos_rtc.c
@@ -24,72 +24,91 @@ static const struct acpi_device_id acpi_cmos_rtc_ids[] = {
{}
};
-static acpi_status
-acpi_cmos_rtc_space_handler(u32 function, acpi_physical_address address,
- u32 bits, u64 *value64,
- void *handler_context, void *region_context)
+static bool cmos_rtc_space_handler_present __read_mostly;
+
+static acpi_status acpi_cmos_rtc_space_handler(u32 function,
+ acpi_physical_address address,
+ u32 bits, u64 *value64,
+ void *handler_context,
+ void *region_context)
{
- int i;
+ unsigned int i, bytes = DIV_ROUND_UP(bits, 8);
u8 *value = (u8 *)value64;
if (address > 0xff || !value64)
return AE_BAD_PARAMETER;
- if (function != ACPI_WRITE && function != ACPI_READ)
- return AE_BAD_PARAMETER;
+ guard(spinlock_irq)(&rtc_lock);
+
+ if (function == ACPI_WRITE) {
+ for (i = 0; i < bytes; i++, address++, value++)
+ CMOS_WRITE(*value, address);
- spin_lock_irq(&rtc_lock);
+ return AE_OK;
+ }
- for (i = 0; i < DIV_ROUND_UP(bits, 8); ++i, ++address, ++value)
- if (function == ACPI_READ)
+ if (function == ACPI_READ) {
+ for (i = 0; i < bytes; i++, address++, value++)
*value = CMOS_READ(address);
- else
- CMOS_WRITE(*value, address);
- spin_unlock_irq(&rtc_lock);
+ return AE_OK;
+ }
- return AE_OK;
+ return AE_BAD_PARAMETER;
}
int acpi_install_cmos_rtc_space_handler(acpi_handle handle)
{
acpi_status status;
+ if (cmos_rtc_space_handler_present)
+ return 0;
+
status = acpi_install_address_space_handler(handle,
- ACPI_ADR_SPACE_CMOS,
- &acpi_cmos_rtc_space_handler,
- NULL, NULL);
+ ACPI_ADR_SPACE_CMOS,
+ acpi_cmos_rtc_space_handler,
+ NULL, NULL);
if (ACPI_FAILURE(status)) {
- pr_err("Error installing CMOS-RTC region handler\n");
+ pr_err("Failed to install CMOS-RTC address space handler\n");
return -ENODEV;
}
+ cmos_rtc_space_handler_present = true;
+
return 1;
}
EXPORT_SYMBOL_GPL(acpi_install_cmos_rtc_space_handler);
void acpi_remove_cmos_rtc_space_handler(acpi_handle handle)
{
- if (ACPI_FAILURE(acpi_remove_address_space_handler(handle,
- ACPI_ADR_SPACE_CMOS, &acpi_cmos_rtc_space_handler)))
- pr_err("Error removing CMOS-RTC region handler\n");
+ acpi_status status;
+
+ if (cmos_rtc_space_handler_present)
+ return;
+
+ status = acpi_remove_address_space_handler(handle,
+ ACPI_ADR_SPACE_CMOS,
+ acpi_cmos_rtc_space_handler);
+ if (ACPI_FAILURE(status))
+ pr_err("Failed to remove CMOS-RTC address space handler\n");
}
EXPORT_SYMBOL_GPL(acpi_remove_cmos_rtc_space_handler);
-static int acpi_cmos_rtc_attach_handler(struct acpi_device *adev, const struct acpi_device_id *id)
+static int acpi_cmos_rtc_attach(struct acpi_device *adev,
+ const struct acpi_device_id *id)
{
- return acpi_install_cmos_rtc_space_handler(adev->handle);
-}
+ int ret;
-static void acpi_cmos_rtc_detach_handler(struct acpi_device *adev)
-{
- acpi_remove_cmos_rtc_space_handler(adev->handle);
+ ret = acpi_install_cmos_rtc_space_handler(adev->handle);
+ if (ret < 0)
+ return ret;
+
+ return 1;
}
static struct acpi_scan_handler cmos_rtc_handler = {
.ids = acpi_cmos_rtc_ids,
- .attach = acpi_cmos_rtc_attach_handler,
- .detach = acpi_cmos_rtc_detach_handler,
+ .attach = acpi_cmos_rtc_attach,
};
void __init acpi_cmos_rtc_init(void)
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index b55443e31f40..f3d0979082cb 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1705,7 +1705,7 @@ void ata_scsi_requeue_deferred_qc(struct ata_port *ap)
/*
* If we have a deferred qc when a reset occurs or NCQ commands fail,
* do not try to be smart about what to do with this deferred command
- * and simply retry it by completing it with DID_SOFT_ERROR.
+ * and simply requeue it by completing it with DID_REQUEUE.
*/
if (!qc)
return;
@@ -1714,7 +1714,7 @@ void ata_scsi_requeue_deferred_qc(struct ata_port *ap)
ap->deferred_qc = NULL;
cancel_work(&ap->deferred_qc_work);
ata_qc_free(qc);
- scmd->result = (DID_SOFT_ERROR << 16);
+ scmd->result = (DID_REQUEUE << 16);
scsi_done(scmd);
}
diff --git a/drivers/base/devres.c b/drivers/base/devres.c
index 68224f2f83ff..f229c5b3e785 100644
--- a/drivers/base/devres.c
+++ b/drivers/base/devres.c
@@ -913,6 +913,8 @@ void *devm_krealloc(struct device *dev, void *ptr, size_t new_size, gfp_t gfp)
if (!new_dr)
return NULL;
+ set_node_dbginfo(&new_dr->node, "devm_krealloc_release", new_size);
+
/*
* The spinlock protects the linked list against concurrent
* modifications but not the resource itself.
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 8c12bf1b2a0d..2e985101963e 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -3377,8 +3377,10 @@ int drbd_adm_dump_devices(struct sk_buff *skb, struct netlink_callback *cb)
if (resource_filter) {
retcode = ERR_RES_NOT_KNOWN;
resource = drbd_find_resource(nla_data(resource_filter));
- if (!resource)
+ if (!resource) {
+ rcu_read_lock();
goto put_result;
+ }
cb->args[0] = (long)resource;
}
}
@@ -3627,8 +3629,10 @@ int drbd_adm_dump_peer_devices(struct sk_buff *skb, struct netlink_callback *cb)
if (resource_filter) {
retcode = ERR_RES_NOT_KNOWN;
resource = drbd_find_resource(nla_data(resource_filter));
- if (!resource)
+ if (!resource) {
+ rcu_read_lock();
goto put_result;
+ }
}
cb->args[0] = (long)resource;
}
diff --git a/drivers/bluetooth/btmtk.c b/drivers/bluetooth/btmtk.c
index ca3e730fedda..98cb8529d8bc 100644
--- a/drivers/bluetooth/btmtk.c
+++ b/drivers/bluetooth/btmtk.c
@@ -678,8 +678,8 @@ static int btmtk_usb_hci_wmt_sync(struct hci_dev *hdev,
case BTMTK_WMT_FUNC_CTRL:
if (!skb_pull_data(data->evt_skb,
sizeof(wmt_evt_funcc->status))) {
- err = -EINVAL;
- goto err_free_skb;
+ status = BTMTK_WMT_ON_UNDONE;
+ break;
}
wmt_evt_funcc = (struct btmtk_hci_wmt_evt_funcc *)wmt_evt;
diff --git a/drivers/bluetooth/hci_ldisc.c b/drivers/bluetooth/hci_ldisc.c
index 436ee77d4bf2..0d06b83816d1 100644
--- a/drivers/bluetooth/hci_ldisc.c
+++ b/drivers/bluetooth/hci_ldisc.c
@@ -692,6 +692,9 @@ static int hci_uart_register_dev(struct hci_uart *hu)
if (hci_register_dev(hdev) < 0) {
BT_ERR("Can't register HCI device");
+ percpu_down_write(&hu->proto_lock);
+ clear_bit(HCI_UART_PROTO_INIT, &hu->flags);
+ percpu_up_write(&hu->proto_lock);
hu->proto->close(hu);
hu->hdev = NULL;
hci_free_dev(hdev);
diff --git a/drivers/bus/fsl-mc/fsl-mc-bus.c b/drivers/bus/fsl-mc/fsl-mc-bus.c
index 5543ba93e501..2810f3b6e2f6 100644
--- a/drivers/bus/fsl-mc/fsl-mc-bus.c
+++ b/drivers/bus/fsl-mc/fsl-mc-bus.c
@@ -86,12 +86,16 @@ static int fsl_mc_bus_match(struct device *dev, const struct device_driver *drv)
struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
const struct fsl_mc_driver *mc_drv = to_fsl_mc_driver(drv);
bool found = false;
+ int ret;
/* When driver_override is set, only bind to the matching driver */
- if (mc_dev->driver_override) {
- found = !strcmp(mc_dev->driver_override, mc_drv->driver.name);
+ ret = device_match_driver_override(dev, drv);
+ if (ret > 0) {
+ found = true;
goto out;
}
+ if (ret == 0)
+ goto out;
if (!mc_drv->match_id_table)
goto out;
@@ -180,39 +184,8 @@ static ssize_t modalias_show(struct device *dev, struct device_attribute *attr,
}
static DEVICE_ATTR_RO(modalias);
-static ssize_t driver_override_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
- int ret;
-
- if (WARN_ON(dev->bus != &fsl_mc_bus_type))
- return -EINVAL;
-
- ret = driver_set_override(dev, &mc_dev->driver_override, buf, count);
- if (ret)
- return ret;
-
- return count;
-}
-
-static ssize_t driver_override_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
- ssize_t len;
-
- device_lock(dev);
- len = sysfs_emit(buf, "%s\n", mc_dev->driver_override);
- device_unlock(dev);
- return len;
-}
-static DEVICE_ATTR_RW(driver_override);
-
static struct attribute *fsl_mc_dev_attrs[] = {
&dev_attr_modalias.attr,
- &dev_attr_driver_override.attr,
NULL,
};
@@ -315,6 +288,7 @@ ATTRIBUTE_GROUPS(fsl_mc_bus);
const struct bus_type fsl_mc_bus_type = {
.name = "fsl-mc",
+ .driver_override = true,
.match = fsl_mc_bus_match,
.uevent = fsl_mc_bus_uevent,
.dma_configure = fsl_mc_dma_configure,
@@ -924,9 +898,6 @@ static struct notifier_block fsl_mc_nb;
*/
void fsl_mc_device_remove(struct fsl_mc_device *mc_dev)
{
- kfree(mc_dev->driver_override);
- mc_dev->driver_override = NULL;
-
/*
* The device-specific remove callback will get invoked by device_del()
*/
diff --git a/drivers/bus/stm32_rifsc.c b/drivers/bus/stm32_rifsc.c
index 4cf1b60014b7..59872134c322 100644
--- a/drivers/bus/stm32_rifsc.c
+++ b/drivers/bus/stm32_rifsc.c
@@ -126,34 +126,6 @@ static int stm32_rifsc_grant_access(struct stm32_firewall_controller *ctrl, u32
sec_reg_value = readl(rifsc_controller->mmio + RIFSC_RISC_SECCFGR0 + 0x4 * reg_id);
cid_reg_value = readl(rifsc_controller->mmio + RIFSC_RISC_PER0_CIDCFGR + 0x8 * firewall_id);
- /* First check conditions for semaphore mode, which doesn't take into account static CID. */
- if ((cid_reg_value & CIDCFGR_SEMEN) && (cid_reg_value & CIDCFGR_CFEN)) {
- if (cid_reg_value & BIT(RIF_CID1 + SEMWL_SHIFT)) {
- /* Static CID is irrelevant if semaphore mode */
- goto skip_cid_check;
- } else {
- dev_dbg(rifsc_controller->dev,
- "Invalid bus semaphore configuration: index %d\n", firewall_id);
- return -EACCES;
- }
- }
-
- /*
- * Skip CID check if CID filtering isn't enabled or filtering is enabled on CID0, which
- * corresponds to whatever CID.
- */
- if (!(cid_reg_value & CIDCFGR_CFEN) ||
- FIELD_GET(RIFSC_RISC_SCID_MASK, cid_reg_value) == RIF_CID0)
- goto skip_cid_check;
-
- /* Coherency check with the CID configuration */
- if (FIELD_GET(RIFSC_RISC_SCID_MASK, cid_reg_value) != RIF_CID1) {
- dev_dbg(rifsc_controller->dev, "Invalid CID configuration for peripheral: %d\n",
- firewall_id);
- return -EACCES;
- }
-
-skip_cid_check:
/* Check security configuration */
if (sec_reg_value & BIT(reg_offset)) {
dev_dbg(rifsc_controller->dev,
@@ -161,19 +133,31 @@ static int stm32_rifsc_grant_access(struct stm32_firewall_controller *ctrl, u32
return -EACCES;
}
- /*
- * If the peripheral is in semaphore mode, take the semaphore so that
- * the CID1 has the ownership.
- */
- if ((cid_reg_value & CIDCFGR_SEMEN) && (cid_reg_value & CIDCFGR_CFEN)) {
+ /* Skip CID check if CID filtering isn't enabled */
+ if (!(cid_reg_value & CIDCFGR_CFEN))
+ goto skip_cid_check;
+
+ /* First check conditions for semaphore mode, which doesn't take into account static CID. */
+ if (cid_reg_value & CIDCFGR_SEMEN) {
+ if (!(cid_reg_value & BIT(RIF_CID1 + SEMWL_SHIFT))) {
+ dev_dbg(rifsc_controller->dev,
+ "Invalid bus semaphore configuration: index %d\n", firewall_id);
+ return -EACCES;
+ }
+
rc = stm32_rif_acquire_semaphore(rifsc_controller, firewall_id);
if (rc) {
- dev_err(rifsc_controller->dev,
+ dev_dbg(rifsc_controller->dev,
"Couldn't acquire semaphore for peripheral: %d\n", firewall_id);
return rc;
}
+ } else if (FIELD_GET(RIFSC_RISC_SCID_MASK, cid_reg_value) != RIF_CID1) {
+ dev_dbg(rifsc_controller->dev, "Invalid CID configuration for peripheral: %d\n",
+ firewall_id);
+ return -EACCES;
}
+skip_cid_check:
return 0;
}
diff --git a/drivers/cdrom/cdrom.c b/drivers/cdrom/cdrom.c
index 6a99a459b80b..19d6f9a069bd 100644
--- a/drivers/cdrom/cdrom.c
+++ b/drivers/cdrom/cdrom.c
@@ -634,6 +634,16 @@ int register_cdrom(struct gendisk *disk, struct cdrom_device_info *cdi)
WARN_ON(!cdo->generic_packet);
+ /*
+ * Propagate the drive's write support to the block layer so BLKROGET
+ * reflects actual write capability. Drivers that use GET CONFIGURATION
+ * features (CDC_MRW_W, CDC_RAM) must have called
+ * cdrom_probe_write_features() before register_cdrom() so the mask is
+ * complete here.
+ */
+ set_disk_ro(disk, !CDROM_CAN(CDC_DVD_RAM | CDC_MRW_W | CDC_RAM |
+ CDC_CD_RW));
+
cd_dbg(CD_REG_UNREG, "drive \"/dev/%s\" registered\n", cdi->name);
mutex_lock(&cdrom_mutex);
list_add(&cdi->list, &cdrom_list);
@@ -748,6 +758,44 @@ static int cdrom_is_random_writable(struct cdrom_device_info *cdi, int *write)
return 0;
}
+/*
+ * Probe write-related MMC features via GET CONFIGURATION and update
+ * cdi->mask accordingly. Drivers that populate cdi->mask from the MODE SENSE
+ * capabilities page (e.g. sr) should call this after those MODE SENSE bits
+ * have been set but before register_cdrom(), so that the full set of
+ * write-capability bits is known by the time register_cdrom() decides on the
+ * initial read-only state of the disk.
+ */
+void cdrom_probe_write_features(struct cdrom_device_info *cdi)
+{
+ int mrw, mrw_write, ram_write;
+
+ mrw = 0;
+ if (!cdrom_is_mrw(cdi, &mrw_write))
+ mrw = 1;
+
+ if (CDROM_CAN(CDC_MO_DRIVE))
+ ram_write = 1;
+ else
+ (void) cdrom_is_random_writable(cdi, &ram_write);
+
+ if (mrw)
+ cdi->mask &= ~CDC_MRW;
+ else
+ cdi->mask |= CDC_MRW;
+
+ if (mrw_write)
+ cdi->mask &= ~CDC_MRW_W;
+ else
+ cdi->mask |= CDC_MRW_W;
+
+ if (ram_write)
+ cdi->mask &= ~CDC_RAM;
+ else
+ cdi->mask |= CDC_RAM;
+}
+EXPORT_SYMBOL(cdrom_probe_write_features);
+
static int cdrom_media_erasable(struct cdrom_device_info *cdi)
{
disc_information di;
@@ -900,33 +948,8 @@ static int cdrom_is_dvd_rw(struct cdrom_device_info *cdi)
*/
static int cdrom_open_write(struct cdrom_device_info *cdi)
{
- int mrw, mrw_write, ram_write;
int ret = 1;
- mrw = 0;
- if (!cdrom_is_mrw(cdi, &mrw_write))
- mrw = 1;
-
- if (CDROM_CAN(CDC_MO_DRIVE))
- ram_write = 1;
- else
- (void) cdrom_is_random_writable(cdi, &ram_write);
-
- if (mrw)
- cdi->mask &= ~CDC_MRW;
- else
- cdi->mask |= CDC_MRW;
-
- if (mrw_write)
- cdi->mask &= ~CDC_MRW_W;
- else
- cdi->mask |= CDC_MRW_W;
-
- if (ram_write)
- cdi->mask &= ~CDC_RAM;
- else
- cdi->mask |= CDC_RAM;
-
if (CDROM_CAN(CDC_MRW_W))
ret = cdrom_mrw_open_write(cdi);
else if (CDROM_CAN(CDC_DVD_RAM))
diff --git a/drivers/char/ipmi/ssif_bmc.c b/drivers/char/ipmi/ssif_bmc.c
index 310f17dd9511..c5492e17ae57 100644
--- a/drivers/char/ipmi/ssif_bmc.c
+++ b/drivers/char/ipmi/ssif_bmc.c
@@ -163,6 +163,8 @@ static ssize_t ssif_bmc_read(struct file *file, char __user *buf, size_t count,
spin_unlock_irqrestore(&ssif_bmc->lock, flags);
ret = copy_to_user(buf, &msg, count);
+ if (ret > 0)
+ ret = -EFAULT;
}
return (ret < 0) ? ret : count;
@@ -455,6 +457,15 @@ static bool supported_write_cmd(u8 cmd)
return false;
}
+static bool supported_write_start_cmd(u8 cmd)
+{
+ if (cmd == SSIF_IPMI_SINGLEPART_WRITE ||
+ cmd == SSIF_IPMI_MULTIPART_WRITE_START)
+ return true;
+
+ return false;
+}
+
/* Process the IPMI response that will be read by master */
static void handle_read_processed(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
{
@@ -557,7 +568,7 @@ static void process_request_part(struct ssif_bmc_ctx *ssif_bmc)
len = ssif_bmc->request.len + part->length;
/* Do the bound check here, not allow the request len exceed 254 bytes */
if (len > IPMI_SSIF_PAYLOAD_MAX) {
- dev_warn(&ssif_bmc->client->dev,
+ dev_dbg(&ssif_bmc->client->dev,
"Warn: Request exceeded 254 bytes, aborting");
/* Request too long, aborting */
ssif_bmc->aborting = true;
@@ -603,7 +614,7 @@ static void on_read_requested_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
ssif_bmc->state == SSIF_START ||
ssif_bmc->state == SSIF_REQ_RECVING ||
ssif_bmc->state == SSIF_RES_SENDING) {
- dev_warn(&ssif_bmc->client->dev,
+ dev_dbg(&ssif_bmc->client->dev,
"Warn: %s unexpected READ REQUESTED in state=%s\n",
__func__, state_to_string(ssif_bmc->state));
ssif_bmc->state = SSIF_ABORTING;
@@ -612,7 +623,7 @@ static void on_read_requested_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
} else if (ssif_bmc->state == SSIF_SMBUS_CMD) {
if (!supported_read_cmd(ssif_bmc->part_buf.smbus_cmd)) {
- dev_warn(&ssif_bmc->client->dev, "Warn: Unknown SMBus read command=0x%x",
+ dev_dbg(&ssif_bmc->client->dev, "Warn: Unknown SMBus read command=0x%x",
ssif_bmc->part_buf.smbus_cmd);
ssif_bmc->aborting = true;
}
@@ -647,7 +658,7 @@ static void on_read_processed_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
ssif_bmc->state == SSIF_START ||
ssif_bmc->state == SSIF_REQ_RECVING ||
ssif_bmc->state == SSIF_SMBUS_CMD) {
- dev_warn(&ssif_bmc->client->dev,
+ dev_dbg(&ssif_bmc->client->dev,
"Warn: %s unexpected READ PROCESSED in state=%s\n",
__func__, state_to_string(ssif_bmc->state));
ssif_bmc->state = SSIF_ABORTING;
@@ -672,7 +683,7 @@ static void on_write_requested_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
} else if (ssif_bmc->state == SSIF_START ||
ssif_bmc->state == SSIF_REQ_RECVING ||
ssif_bmc->state == SSIF_RES_SENDING) {
- dev_warn(&ssif_bmc->client->dev,
+ dev_dbg(&ssif_bmc->client->dev,
"Warn: %s unexpected WRITE REQUEST in state=%s\n",
__func__, state_to_string(ssif_bmc->state));
ssif_bmc->state = SSIF_ABORTING;
@@ -687,7 +698,7 @@ static void on_write_received_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
{
if (ssif_bmc->state == SSIF_READY ||
ssif_bmc->state == SSIF_RES_SENDING) {
- dev_warn(&ssif_bmc->client->dev,
+ dev_dbg(&ssif_bmc->client->dev,
"Warn: %s unexpected WRITE RECEIVED in state=%s\n",
__func__, state_to_string(ssif_bmc->state));
ssif_bmc->state = SSIF_ABORTING;
@@ -697,7 +708,7 @@ static void on_write_received_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
} else if (ssif_bmc->state == SSIF_SMBUS_CMD) {
if (!supported_write_cmd(ssif_bmc->part_buf.smbus_cmd)) {
- dev_warn(&ssif_bmc->client->dev, "Warn: Unknown SMBus write command=0x%x",
+ dev_dbg(&ssif_bmc->client->dev, "Warn: Unknown SMBus write command=0x%x",
ssif_bmc->part_buf.smbus_cmd);
ssif_bmc->aborting = true;
}
@@ -706,6 +717,11 @@ static void on_write_received_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
ssif_bmc->state = SSIF_ABORTING;
else
ssif_bmc->state = SSIF_REQ_RECVING;
+ } else if (ssif_bmc->state == SSIF_ABORTING) {
+ if (supported_write_start_cmd(*val)) {
+ ssif_bmc->state = SSIF_SMBUS_CMD;
+ ssif_bmc->aborting = false;
+ }
}
/* This is response sending state */
@@ -721,7 +737,7 @@ static void on_stop_event(struct ssif_bmc_ctx *ssif_bmc, u8 *val)
ssif_bmc->state == SSIF_START ||
ssif_bmc->state == SSIF_SMBUS_CMD ||
ssif_bmc->state == SSIF_ABORTING) {
- dev_warn(&ssif_bmc->client->dev,
+ dev_dbg(&ssif_bmc->client->dev,
"Warn: %s unexpected SLAVE STOP in state=%s\n",
__func__, state_to_string(ssif_bmc->state));
ssif_bmc->state = SSIF_READY;
@@ -788,7 +804,7 @@ static int ssif_bmc_cb(struct i2c_client *client, enum i2c_slave_event event, u8
break;
default:
- dev_warn(&ssif_bmc->client->dev, "Warn: Unknown i2c slave event\n");
+ dev_dbg(&ssif_bmc->client->dev, "Warn: Unknown i2c slave event\n");
break;
}
diff --git a/drivers/clk/clk-qoriq.c b/drivers/clk/clk-qoriq.c
index 4dcde305944c..318acd176a2e 100644
--- a/drivers/clk/clk-qoriq.c
+++ b/drivers/clk/clk-qoriq.c
@@ -906,13 +906,11 @@ static const struct clockgen_pll_div *get_pll_div(struct clockgen *cg,
return &cg->pll[pll].div[div];
}
-static struct clk * __init create_mux_common(struct clockgen *cg,
- struct mux_hwclock *hwc,
- const struct clk_ops *ops,
- unsigned long min_rate,
- unsigned long max_rate,
- unsigned long pct80_rate,
- const char *fmt, int idx)
+static struct clk * __init __printf(7, 8)
+create_mux_common(struct clockgen *cg, struct mux_hwclock *hwc,
+ const struct clk_ops *ops, unsigned long min_rate,
+ unsigned long max_rate, unsigned long pct80_rate,
+ const char *fmt, ...)
{
struct clk_init_data init = {};
struct clk *clk;
@@ -920,8 +918,11 @@ static struct clk * __init create_mux_common(struct clockgen *cg,
const char *parent_names[NUM_MUX_PARENTS];
char name[32];
int i, j;
+ va_list args;
- snprintf(name, sizeof(name), fmt, idx);
+ va_start(args, fmt);
+ vsnprintf(name, sizeof(name), fmt, args);
+ va_end(args);
for (i = 0, j = 0; i < NUM_MUX_PARENTS; i++) {
unsigned long rate;
diff --git a/drivers/clk/clk-xgene.c b/drivers/clk/clk-xgene.c
index 0c3d0cee98c8..a542b78d9c73 100644
--- a/drivers/clk/clk-xgene.c
+++ b/drivers/clk/clk-xgene.c
@@ -187,6 +187,8 @@ static void xgene_pllclk_init(struct device_node *np, enum xgene_pll_type pll_ty
of_clk_add_provider(np, of_clk_src_simple_get, clk);
clk_register_clkdev(clk, clk_name, NULL);
pr_debug("Add %s clock PLL\n", clk_name);
+ } else {
+ iounmap(reg);
}
}
diff --git a/drivers/clk/imx/clk-imx6q.c b/drivers/clk/imx/clk-imx6q.c
index bf4c1d9c9928..048e2ddba490 100644
--- a/drivers/clk/imx/clk-imx6q.c
+++ b/drivers/clk/imx/clk-imx6q.c
@@ -188,9 +188,11 @@ static void of_assigned_ldb_sels(struct device_node *node,
}
if (clkspec.np != node || clkspec.args[0] >= IMX6QDL_CLK_END) {
pr_err("ccm: parent clock %d not in ccm\n", index);
+ of_node_put(clkspec.np);
return;
}
parent = clkspec.args[0];
+ of_node_put(clkspec.np);
rc = of_parse_phandle_with_args(node, "assigned-clocks",
"#clock-cells", index, &clkspec);
@@ -198,9 +200,11 @@ static void of_assigned_ldb_sels(struct device_node *node,
return;
if (clkspec.np != node || clkspec.args[0] >= IMX6QDL_CLK_END) {
pr_err("ccm: child clock %d not in ccm\n", index);
+ of_node_put(clkspec.np);
return;
}
child = clkspec.args[0];
+ of_node_put(clkspec.np);
if (child != IMX6QDL_CLK_LDB_DI0_SEL &&
child != IMX6QDL_CLK_LDB_DI1_SEL)
@@ -238,8 +242,11 @@ static bool pll6_bypassed(struct device_node *node)
return false;
if (clkspec.np == node &&
- clkspec.args[0] == IMX6QDL_PLL6_BYPASS)
+ clkspec.args[0] == IMX6QDL_PLL6_BYPASS) {
+ of_node_put(clkspec.np);
break;
+ }
+ of_node_put(clkspec.np);
}
/* PLL6 bypass is not part of the assigned clock list */
@@ -249,6 +256,9 @@ static bool pll6_bypassed(struct device_node *node)
ret = of_parse_phandle_with_args(node, "assigned-clock-parents",
"#clock-cells", index, &clkspec);
+ if (!ret)
+ of_node_put(clkspec.np);
+
if (clkspec.args[0] != IMX6QDL_CLK_PLL6)
return true;
diff --git a/drivers/clk/imx/clk-imx8mq.c b/drivers/clk/imx/clk-imx8mq.c
index f70ed231b92d..cedc8a02aa1f 100644
--- a/drivers/clk/imx/clk-imx8mq.c
+++ b/drivers/clk/imx/clk-imx8mq.c
@@ -237,7 +237,7 @@ static const char * const imx8mq_dsi_esc_sels[] = {"osc_25m", "sys2_pll_100m", "
static const char * const imx8mq_csi1_core_sels[] = {"osc_25m", "sys1_pll_266m", "sys2_pll_250m", "sys1_pll_800m",
"sys2_pll_1000m", "sys3_pll_out", "audio_pll2_out", "video_pll1_out", };
-static const char * const imx8mq_csi1_phy_sels[] = {"osc_25m", "sys2_pll_125m", "sys2_pll_100m", "sys1_pll_800m",
+static const char * const imx8mq_csi1_phy_sels[] = {"osc_25m", "sys2_pll_333m", "sys2_pll_100m", "sys1_pll_800m",
"sys2_pll_1000m", "clk_ext2", "audio_pll2_out", "video_pll1_out", };
static const char * const imx8mq_csi1_esc_sels[] = {"osc_25m", "sys2_pll_100m", "sys1_pll_80m", "sys1_pll_800m",
@@ -246,7 +246,7 @@ static const char * const imx8mq_csi1_esc_sels[] = {"osc_25m", "sys2_pll_100m",
static const char * const imx8mq_csi2_core_sels[] = {"osc_25m", "sys1_pll_266m", "sys2_pll_250m", "sys1_pll_800m",
"sys2_pll_1000m", "sys3_pll_out", "audio_pll2_out", "video_pll1_out", };
-static const char * const imx8mq_csi2_phy_sels[] = {"osc_25m", "sys2_pll_125m", "sys2_pll_100m", "sys1_pll_800m",
+static const char * const imx8mq_csi2_phy_sels[] = {"osc_25m", "sys2_pll_333m", "sys2_pll_100m", "sys1_pll_800m",
"sys2_pll_1000m", "clk_ext2", "audio_pll2_out", "video_pll1_out", };
static const char * const imx8mq_csi2_esc_sels[] = {"osc_25m", "sys2_pll_100m", "sys1_pll_80m", "sys1_pll_800m",
diff --git a/drivers/clk/qcom/dispcc-sc7180.c b/drivers/clk/qcom/dispcc-sc7180.c
index 4710247be530..ae98fe4dcfb2 100644
--- a/drivers/clk/qcom/dispcc-sc7180.c
+++ b/drivers/clk/qcom/dispcc-sc7180.c
@@ -16,6 +16,7 @@
#include "clk-regmap-divider.h"
#include "common.h"
#include "gdsc.h"
+#include "reset.h"
enum {
P_BI_TCXO,
@@ -635,6 +636,11 @@ static struct gdsc mdss_gdsc = {
.flags = HW_CTRL,
};
+static const struct qcom_reset_map disp_cc_sc7180_resets[] = {
+ [DISP_CC_MDSS_CORE_BCR] = { 0x2000 },
+ [DISP_CC_MDSS_RSCC_BCR] = { 0x4000 },
+};
+
static struct gdsc *disp_cc_sc7180_gdscs[] = {
[MDSS_GDSC] = &mdss_gdsc,
};
@@ -686,6 +692,8 @@ static const struct qcom_cc_desc disp_cc_sc7180_desc = {
.config = &disp_cc_sc7180_regmap_config,
.clks = disp_cc_sc7180_clocks,
.num_clks = ARRAY_SIZE(disp_cc_sc7180_clocks),
+ .resets = disp_cc_sc7180_resets,
+ .num_resets = ARRAY_SIZE(disp_cc_sc7180_resets),
.gdscs = disp_cc_sc7180_gdscs,
.num_gdscs = ARRAY_SIZE(disp_cc_sc7180_gdscs),
};
diff --git a/drivers/clk/qcom/dispcc-sc8280xp.c b/drivers/clk/qcom/dispcc-sc8280xp.c
index f1ca9ae0b33f..c23cbb983d29 100644
--- a/drivers/clk/qcom/dispcc-sc8280xp.c
+++ b/drivers/clk/qcom/dispcc-sc8280xp.c
@@ -1161,7 +1161,6 @@ static struct clk_regmap_div disp0_cc_mdss_byte0_div_clk_src = {
&disp0_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_regmap_div_ops,
},
};
@@ -1176,7 +1175,6 @@ static struct clk_regmap_div disp1_cc_mdss_byte0_div_clk_src = {
&disp1_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_regmap_div_ops,
},
};
@@ -1191,7 +1189,6 @@ static struct clk_regmap_div disp0_cc_mdss_byte1_div_clk_src = {
&disp0_cc_mdss_byte1_clk_src.clkr.hw,
},
.num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_regmap_div_ops,
},
};
@@ -1206,7 +1203,6 @@ static struct clk_regmap_div disp1_cc_mdss_byte1_div_clk_src = {
&disp1_cc_mdss_byte1_clk_src.clkr.hw,
},
.num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_regmap_div_ops,
},
};
diff --git a/drivers/clk/qcom/dispcc-sm4450.c b/drivers/clk/qcom/dispcc-sm4450.c
index 98ba016bc57f..398910f09a72 100644
--- a/drivers/clk/qcom/dispcc-sm4450.c
+++ b/drivers/clk/qcom/dispcc-sm4450.c
@@ -336,7 +336,6 @@ static struct clk_regmap_div disp_cc_mdss_byte0_div_clk_src = {
&disp_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_regmap_div_ops,
},
};
diff --git a/drivers/clk/qcom/dispcc-sm8250.c b/drivers/clk/qcom/dispcc-sm8250.c
index 884bbd3fb305..ea79d48b9a25 100644
--- a/drivers/clk/qcom/dispcc-sm8250.c
+++ b/drivers/clk/qcom/dispcc-sm8250.c
@@ -578,7 +578,7 @@ static struct clk_rcg2 disp_cc_mdss_pclk0_clk_src = {
.name = "disp_cc_mdss_pclk0_clk_src",
.parent_data = disp_cc_parent_data_6,
.num_parents = ARRAY_SIZE(disp_cc_parent_data_6),
- .flags = CLK_SET_RATE_PARENT,
+ .flags = CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
.ops = &clk_pixel_ops,
},
};
@@ -592,7 +592,7 @@ static struct clk_rcg2 disp_cc_mdss_pclk1_clk_src = {
.name = "disp_cc_mdss_pclk1_clk_src",
.parent_data = disp_cc_parent_data_6,
.num_parents = ARRAY_SIZE(disp_cc_parent_data_6),
- .flags = CLK_SET_RATE_PARENT,
+ .flags = CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
.ops = &clk_pixel_ops,
},
};
@@ -632,7 +632,7 @@ static struct clk_rcg2 disp_cc_mdss_vsync_clk_src = {
.parent_data = disp_cc_parent_data_1,
.num_parents = ARRAY_SIZE(disp_cc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
diff --git a/drivers/clk/qcom/dispcc-sm8450.c b/drivers/clk/qcom/dispcc-sm8450.c
index d1d3f60789ee..8baaf94a119e 100644
--- a/drivers/clk/qcom/dispcc-sm8450.c
+++ b/drivers/clk/qcom/dispcc-sm8450.c
@@ -364,7 +364,7 @@ static struct clk_rcg2 disp_cc_mdss_dptx1_aux_clk_src = {
.parent_data = disp_cc_parent_data_1,
.num_parents = ARRAY_SIZE(disp_cc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
- .ops = &clk_dp_ops,
+ .ops = &clk_rcg2_ops,
},
};
diff --git a/drivers/clk/qcom/gcc-sc8180x.c b/drivers/clk/qcom/gcc-sc8180x.c
index 31e788e22ab4..4095a1f54a09 100644
--- a/drivers/clk/qcom/gcc-sc8180x.c
+++ b/drivers/clk/qcom/gcc-sc8180x.c
@@ -4172,7 +4172,7 @@ static struct gdsc usb30_sec_gdsc = {
.pd = {
.name = "usb30_sec_gdsc",
},
- .pwrsts = PWRSTS_OFF_ON,
+ .pwrsts = PWRSTS_RET_ON,
.flags = POLL_CFG_GDSCR,
};
@@ -4190,7 +4190,7 @@ static struct gdsc usb30_prim_gdsc = {
.pd = {
.name = "usb30_prim_gdsc",
},
- .pwrsts = PWRSTS_OFF_ON,
+ .pwrsts = PWRSTS_RET_ON,
.flags = POLL_CFG_GDSCR,
};
@@ -4199,7 +4199,7 @@ static struct gdsc pcie_0_gdsc = {
.pd = {
.name = "pcie_0_gdsc",
},
- .pwrsts = PWRSTS_OFF_ON,
+ .pwrsts = PWRSTS_RET_ON,
.flags = POLL_CFG_GDSCR,
};
@@ -4226,7 +4226,7 @@ static struct gdsc pcie_1_gdsc = {
.pd = {
.name = "pcie_1_gdsc",
},
- .pwrsts = PWRSTS_OFF_ON,
+ .pwrsts = PWRSTS_RET_ON,
.flags = POLL_CFG_GDSCR,
};
@@ -4235,7 +4235,7 @@ static struct gdsc pcie_2_gdsc = {
.pd = {
.name = "pcie_2_gdsc",
},
- .pwrsts = PWRSTS_OFF_ON,
+ .pwrsts = PWRSTS_RET_ON,
.flags = POLL_CFG_GDSCR,
};
@@ -4253,7 +4253,7 @@ static struct gdsc pcie_3_gdsc = {
.pd = {
.name = "pcie_3_gdsc",
},
- .pwrsts = PWRSTS_OFF_ON,
+ .pwrsts = PWRSTS_RET_ON,
.flags = POLL_CFG_GDSCR,
};
@@ -4262,10 +4262,55 @@ static struct gdsc usb30_mp_gdsc = {
.pd = {
.name = "usb30_mp_gdsc",
},
- .pwrsts = PWRSTS_OFF_ON,
+ .pwrsts = PWRSTS_RET_ON,
.flags = POLL_CFG_GDSCR,
};
+static struct gdsc hlos1_vote_mmnoc_mmu_tbu_hf0_gdsc = {
+ .gdscr = 0x7d050,
+ .pd = {
+ .name = "hlos1_vote_mmnoc_mmu_tbu_hf0_gdsc",
+ },
+ .pwrsts = PWRSTS_OFF_ON,
+ .flags = VOTABLE,
+};
+
+static struct gdsc hlos1_vote_mmnoc_mmu_tbu_hf1_gdsc = {
+ .gdscr = 0x7d058,
+ .pd = {
+ .name = "hlos1_vote_mmnoc_mmu_tbu_hf1_gdsc",
+ },
+ .pwrsts = PWRSTS_OFF_ON,
+ .flags = VOTABLE,
+};
+
+static struct gdsc hlos1_vote_mmnoc_mmu_tbu_sf_gdsc = {
+ .gdscr = 0x7d054,
+ .pd = {
+ .name = "hlos1_vote_mmnoc_mmu_tbu_sf_gdsc",
+ },
+ .pwrsts = PWRSTS_OFF_ON,
+ .flags = VOTABLE,
+};
+
+static struct gdsc hlos1_vote_turing_mmu_tbu0_gdsc = {
+ .gdscr = 0x7d05c,
+ .pd = {
+ .name = "hlos1_vote_turing_mmu_tbu0_gdsc",
+ },
+ .pwrsts = PWRSTS_OFF_ON,
+ .flags = VOTABLE,
+};
+
+static struct gdsc hlos1_vote_turing_mmu_tbu1_gdsc = {
+ .gdscr = 0x7d060,
+ .pd = {
+ .name = "hlos1_vote_turing_mmu_tbu1_gdsc",
+ },
+ .pwrsts = PWRSTS_OFF_ON,
+ .flags = VOTABLE,
+};
+
static struct clk_regmap *gcc_sc8180x_clocks[] = {
[GCC_AGGRE_NOC_PCIE_TBU_CLK] = &gcc_aggre_noc_pcie_tbu_clk.clkr,
[GCC_AGGRE_UFS_CARD_AXI_CLK] = &gcc_aggre_ufs_card_axi_clk.clkr,
@@ -4595,6 +4640,11 @@ static struct gdsc *gcc_sc8180x_gdscs[] = {
[USB30_MP_GDSC] = &usb30_mp_gdsc,
[USB30_PRIM_GDSC] = &usb30_prim_gdsc,
[USB30_SEC_GDSC] = &usb30_sec_gdsc,
+ [HLOS1_VOTE_MMNOC_MMU_TBU_HF0_GDSC] = &hlos1_vote_mmnoc_mmu_tbu_hf0_gdsc,
+ [HLOS1_VOTE_MMNOC_MMU_TBU_HF1_GDSC] = &hlos1_vote_mmnoc_mmu_tbu_hf1_gdsc,
+ [HLOS1_VOTE_MMNOC_MMU_TBU_SF_GDSC] = &hlos1_vote_mmnoc_mmu_tbu_sf_gdsc,
+ [HLOS1_VOTE_TURING_MMU_TBU0_GDSC] = &hlos1_vote_turing_mmu_tbu0_gdsc,
+ [HLOS1_VOTE_TURING_MMU_TBU1_GDSC] = &hlos1_vote_turing_mmu_tbu1_gdsc,
};
static const struct regmap_config gcc_sc8180x_regmap_config = {
diff --git a/drivers/clk/qcom/gcc-x1e80100.c b/drivers/clk/qcom/gcc-x1e80100.c
index 0c49f0461ae3..e96faa529d3c 100644
--- a/drivers/clk/qcom/gcc-x1e80100.c
+++ b/drivers/clk/qcom/gcc-x1e80100.c
@@ -7413,6 +7413,7 @@ static int gcc_x1e80100_probe(struct platform_device *pdev)
qcom_branch_set_clk_en(regmap, 0x32004); /* GCC_VIDEO_AHB_CLK */
qcom_branch_set_clk_en(regmap, 0x32030); /* GCC_VIDEO_XO_CLK */
qcom_branch_set_clk_en(regmap, 0x71004); /* GCC_GPU_CFG_AHB_CLK */
+ qcom_branch_set_clk_en(regmap, 0x7d01c); /* GCC_HLOS1_VOTE_AGGRE_NOC_MMU_USB_QTB_CLK */
/* Clear GDSC_SLEEP_ENA_VOTE to stop votes being auto-removed in sleep. */
regmap_write(regmap, 0x52224, 0x0);
diff --git a/drivers/clk/visconti/pll.c b/drivers/clk/visconti/pll.c
index 3f929cf8dd2f..22930fd589b7 100644
--- a/drivers/clk/visconti/pll.c
+++ b/drivers/clk/visconti/pll.c
@@ -244,7 +244,7 @@ static struct clk_hw *visconti_register_pll(struct visconti_pll_provider *ctx,
const struct visconti_pll_rate_table *rate_table,
spinlock_t *lock)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct visconti_pll *pll;
struct clk_hw *pll_hw_clk;
size_t len;
diff --git a/drivers/crypto/atmel-aes.c b/drivers/crypto/atmel-aes.c
index 5f53936eb905..5d35f6de452b 100644
--- a/drivers/crypto/atmel-aes.c
+++ b/drivers/crypto/atmel-aes.c
@@ -2200,12 +2200,10 @@ static irqreturn_t atmel_aes_irq(int irq, void *dev_id)
static void atmel_aes_unregister_algs(struct atmel_aes_dev *dd)
{
- int i;
-
#if IS_ENABLED(CONFIG_CRYPTO_DEV_ATMEL_AUTHENC)
if (dd->caps.has_authenc)
- for (i = 0; i < ARRAY_SIZE(aes_authenc_algs); i++)
- crypto_unregister_aead(&aes_authenc_algs[i]);
+ crypto_unregister_aeads(aes_authenc_algs,
+ ARRAY_SIZE(aes_authenc_algs));
#endif
if (dd->caps.has_xts)
@@ -2214,8 +2212,7 @@ static void atmel_aes_unregister_algs(struct atmel_aes_dev *dd)
if (dd->caps.has_gcm)
crypto_unregister_aead(&aes_gcm_alg);
- for (i = 0; i < ARRAY_SIZE(aes_algs); i++)
- crypto_unregister_skcipher(&aes_algs[i]);
+ crypto_unregister_skciphers(aes_algs, ARRAY_SIZE(aes_algs));
}
static void atmel_aes_crypto_alg_init(struct crypto_alg *alg)
@@ -2228,7 +2225,7 @@ static void atmel_aes_crypto_alg_init(struct crypto_alg *alg)
static int atmel_aes_register_algs(struct atmel_aes_dev *dd)
{
- int err, i, j;
+ int err, i;
for (i = 0; i < ARRAY_SIZE(aes_algs); i++) {
atmel_aes_crypto_alg_init(&aes_algs[i].base);
@@ -2271,17 +2268,17 @@ static int atmel_aes_register_algs(struct atmel_aes_dev *dd)
#if IS_ENABLED(CONFIG_CRYPTO_DEV_ATMEL_AUTHENC)
/* i = ARRAY_SIZE(aes_authenc_algs); */
err_aes_authenc_alg:
- for (j = 0; j < i; j++)
- crypto_unregister_aead(&aes_authenc_algs[j]);
- crypto_unregister_skcipher(&aes_xts_alg);
+ crypto_unregister_aeads(aes_authenc_algs, i);
+ if (dd->caps.has_xts)
+ crypto_unregister_skcipher(&aes_xts_alg);
#endif
err_aes_xts_alg:
- crypto_unregister_aead(&aes_gcm_alg);
+ if (dd->caps.has_gcm)
+ crypto_unregister_aead(&aes_gcm_alg);
err_aes_gcm_alg:
i = ARRAY_SIZE(aes_algs);
err_aes_algs:
- for (j = 0; j < i; j++)
- crypto_unregister_skcipher(&aes_algs[j]);
+ crypto_unregister_skciphers(aes_algs, i);
return err;
}
diff --git a/drivers/crypto/atmel-sha.c b/drivers/crypto/atmel-sha.c
index 8cc57df25778..4f2f57bc3bb6 100644
--- a/drivers/crypto/atmel-sha.c
+++ b/drivers/crypto/atmel-sha.c
@@ -2416,27 +2416,23 @@ EXPORT_SYMBOL_GPL(atmel_sha_authenc_abort);
static void atmel_sha_unregister_algs(struct atmel_sha_dev *dd)
{
- int i;
-
if (dd->caps.has_hmac)
- for (i = 0; i < ARRAY_SIZE(sha_hmac_algs); i++)
- crypto_unregister_ahash(&sha_hmac_algs[i]);
+ crypto_unregister_ahashes(sha_hmac_algs,
+ ARRAY_SIZE(sha_hmac_algs));
- for (i = 0; i < ARRAY_SIZE(sha_1_256_algs); i++)
- crypto_unregister_ahash(&sha_1_256_algs[i]);
+ crypto_unregister_ahashes(sha_1_256_algs, ARRAY_SIZE(sha_1_256_algs));
if (dd->caps.has_sha224)
crypto_unregister_ahash(&sha_224_alg);
- if (dd->caps.has_sha_384_512) {
- for (i = 0; i < ARRAY_SIZE(sha_384_512_algs); i++)
- crypto_unregister_ahash(&sha_384_512_algs[i]);
- }
+ if (dd->caps.has_sha_384_512)
+ crypto_unregister_ahashes(sha_384_512_algs,
+ ARRAY_SIZE(sha_384_512_algs));
}
static int atmel_sha_register_algs(struct atmel_sha_dev *dd)
{
- int err, i, j;
+ int err, i;
for (i = 0; i < ARRAY_SIZE(sha_1_256_algs); i++) {
atmel_sha_alg_init(&sha_1_256_algs[i]);
@@ -2478,18 +2474,15 @@ static int atmel_sha_register_algs(struct atmel_sha_dev *dd)
/*i = ARRAY_SIZE(sha_hmac_algs);*/
err_sha_hmac_algs:
- for (j = 0; j < i; j++)
- crypto_unregister_ahash(&sha_hmac_algs[j]);
+ crypto_unregister_ahashes(sha_hmac_algs, i);
i = ARRAY_SIZE(sha_384_512_algs);
err_sha_384_512_algs:
- for (j = 0; j < i; j++)
- crypto_unregister_ahash(&sha_384_512_algs[j]);
+ crypto_unregister_ahashes(sha_384_512_algs, i);
crypto_unregister_ahash(&sha_224_alg);
err_sha_224_algs:
i = ARRAY_SIZE(sha_1_256_algs);
err_sha_1_256_algs:
- for (j = 0; j < i; j++)
- crypto_unregister_ahash(&sha_1_256_algs[j]);
+ crypto_unregister_ahashes(sha_1_256_algs, i);
return err;
}
diff --git a/drivers/crypto/atmel-tdes.c b/drivers/crypto/atmel-tdes.c
index 813ed47e60cc..f07109608a09 100644
--- a/drivers/crypto/atmel-tdes.c
+++ b/drivers/crypto/atmel-tdes.c
@@ -897,38 +897,25 @@ static irqreturn_t atmel_tdes_irq(int irq, void *dev_id)
return IRQ_NONE;
}
-static void atmel_tdes_unregister_algs(struct atmel_tdes_dev *dd)
-{
- int i;
-
- for (i = 0; i < ARRAY_SIZE(tdes_algs); i++)
- crypto_unregister_skcipher(&tdes_algs[i]);
-}
-
static int atmel_tdes_register_algs(struct atmel_tdes_dev *dd)
{
- int err, i, j;
+ int err, i;
for (i = 0; i < ARRAY_SIZE(tdes_algs); i++) {
atmel_tdes_skcipher_alg_init(&tdes_algs[i]);
err = crypto_register_skcipher(&tdes_algs[i]);
- if (err)
- goto err_tdes_algs;
+ if (err) {
+ crypto_unregister_skciphers(tdes_algs, i);
+ return err;
+ }
}
return 0;
-
-err_tdes_algs:
- for (j = 0; j < i; j++)
- crypto_unregister_skcipher(&tdes_algs[j]);
-
- return err;
}
static void atmel_tdes_get_cap(struct atmel_tdes_dev *dd)
{
-
dd->caps.has_dma = 0;
/* keep only major version number */
@@ -1061,7 +1048,7 @@ static void atmel_tdes_remove(struct platform_device *pdev)
list_del(&tdes_dd->list);
spin_unlock(&atmel_tdes.lock);
- atmel_tdes_unregister_algs(tdes_dd);
+ crypto_unregister_skciphers(tdes_algs, ARRAY_SIZE(tdes_algs));
tasklet_kill(&tdes_dd->done_task);
tasklet_kill(&tdes_dd->queue_task);
diff --git a/drivers/crypto/ccp/ccp-crypto-aes.c b/drivers/crypto/ccp/ccp-crypto-aes.c
index d11daaf47f06..871886826cf0 100644
--- a/drivers/crypto/ccp/ccp-crypto-aes.c
+++ b/drivers/crypto/ccp/ccp-crypto-aes.c
@@ -29,8 +29,11 @@ static int ccp_aes_complete(struct crypto_async_request *async_req, int ret)
if (ret)
return ret;
- if (ctx->u.aes.mode != CCP_AES_MODE_ECB)
- memcpy(req->iv, rctx->iv, AES_BLOCK_SIZE);
+ if (ctx->u.aes.mode != CCP_AES_MODE_ECB) {
+ size_t ivsize = crypto_skcipher_ivsize(crypto_skcipher_reqtfm(req));
+
+ memcpy(req->iv, rctx->iv, ivsize);
+ }
return 0;
}
diff --git a/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c b/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c
index ef5f03be4190..5d32f2958da8 100644
--- a/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c
+++ b/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c
@@ -98,9 +98,25 @@ static struct adf_hw_device_class adf_420xx_class = {
static u32 get_ae_mask(struct adf_hw_device_data *self)
{
- u32 me_disable = self->fuses;
+ unsigned long fuses = self->fuses[ADF_FUSECTL4];
+ u32 mask = ADF_420XX_ACCELENGINES_MASK;
- return ~me_disable & ADF_420XX_ACCELENGINES_MASK;
+ if (test_bit(0, &fuses))
+ mask &= ~ADF_AE_GROUP_0;
+
+ if (test_bit(4, &fuses))
+ mask &= ~ADF_AE_GROUP_1;
+
+ if (test_bit(8, &fuses))
+ mask &= ~ADF_AE_GROUP_2;
+
+ if (test_bit(12, &fuses))
+ mask &= ~ADF_AE_GROUP_3;
+
+ if (test_bit(16, &fuses))
+ mask &= ~ADF_AE_GROUP_4;
+
+ return mask;
}
static u32 uof_get_num_objs(struct adf_accel_dev *accel_dev)
diff --git a/drivers/crypto/intel/qat/qat_420xx/adf_drv.c b/drivers/crypto/intel/qat/qat_420xx/adf_drv.c
index 41420e349572..b24f0a55cf01 100644
--- a/drivers/crypto/intel/qat/qat_420xx/adf_drv.c
+++ b/drivers/crypto/intel/qat/qat_420xx/adf_drv.c
@@ -79,7 +79,7 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
adf_init_hw_data_420xx(accel_dev->hw_device, ent->device);
pci_read_config_byte(pdev, PCI_REVISION_ID, &accel_pci_dev->revid);
- pci_read_config_dword(pdev, ADF_GEN4_FUSECTL4_OFFSET, &hw_data->fuses);
+ pci_read_config_dword(pdev, ADF_GEN4_FUSECTL4_OFFSET, &hw_data->fuses[ADF_FUSECTL4]);
/* Get Accelerators and Accelerators Engines masks */
hw_data->accel_mask = hw_data->get_accel_mask(hw_data);
diff --git a/drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c b/drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c
index bbd92c017c28..579f92e466f6 100644
--- a/drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c
+++ b/drivers/crypto/intel/qat/qat_4xxx/adf_4xxx_hw_data.c
@@ -101,9 +101,19 @@ static struct adf_hw_device_class adf_4xxx_class = {
static u32 get_ae_mask(struct adf_hw_device_data *self)
{
- u32 me_disable = self->fuses;
+ unsigned long fuses = self->fuses[ADF_FUSECTL4];
+ u32 mask = ADF_4XXX_ACCELENGINES_MASK;
- return ~me_disable & ADF_4XXX_ACCELENGINES_MASK;
+ if (test_bit(0, &fuses))
+ mask &= ~ADF_AE_GROUP_0;
+
+ if (test_bit(4, &fuses))
+ mask &= ~ADF_AE_GROUP_1;
+
+ if (test_bit(8, &fuses))
+ mask &= ~ADF_AE_GROUP_2;
+
+ return mask;
}
static u32 get_accel_cap(struct adf_accel_dev *accel_dev)
diff --git a/drivers/crypto/intel/qat/qat_4xxx/adf_drv.c b/drivers/crypto/intel/qat/qat_4xxx/adf_drv.c
index 01b34eda83e9..6efbfed67c95 100644
--- a/drivers/crypto/intel/qat/qat_4xxx/adf_drv.c
+++ b/drivers/crypto/intel/qat/qat_4xxx/adf_drv.c
@@ -81,7 +81,7 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
adf_init_hw_data_4xxx(accel_dev->hw_device, ent->device);
pci_read_config_byte(pdev, PCI_REVISION_ID, &accel_pci_dev->revid);
- pci_read_config_dword(pdev, ADF_GEN4_FUSECTL4_OFFSET, &hw_data->fuses);
+ pci_read_config_dword(pdev, ADF_GEN4_FUSECTL4_OFFSET, &hw_data->fuses[ADF_FUSECTL4]);
/* Get Accelerators and Accelerators Engines masks */
hw_data->accel_mask = hw_data->get_accel_mask(hw_data);
diff --git a/drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c b/drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c
index 201f9412c582..e78f7bfd30b8 100644
--- a/drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c
+++ b/drivers/crypto/intel/qat/qat_c3xxx/adf_c3xxx_hw_data.c
@@ -27,8 +27,8 @@ static struct adf_hw_device_class c3xxx_class = {
static u32 get_accel_mask(struct adf_hw_device_data *self)
{
+ u32 fuses = self->fuses[ADF_FUSECTL0];
u32 straps = self->straps;
- u32 fuses = self->fuses;
u32 accel;
accel = ~(fuses | straps) >> ADF_C3XXX_ACCELERATORS_REG_OFFSET;
@@ -39,8 +39,8 @@ static u32 get_accel_mask(struct adf_hw_device_data *self)
static u32 get_ae_mask(struct adf_hw_device_data *self)
{
+ u32 fuses = self->fuses[ADF_FUSECTL0];
u32 straps = self->straps;
- u32 fuses = self->fuses;
unsigned long disabled;
u32 ae_disable;
int accel;
diff --git a/drivers/crypto/intel/qat/qat_c3xxx/adf_drv.c b/drivers/crypto/intel/qat/qat_c3xxx/adf_drv.c
index b776f7ea0dfb..fdbfcb0c8214 100644
--- a/drivers/crypto/intel/qat/qat_c3xxx/adf_drv.c
+++ b/drivers/crypto/intel/qat/qat_c3xxx/adf_drv.c
@@ -134,7 +134,7 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
adf_init_hw_data_c3xxx(accel_dev->hw_device);
pci_read_config_byte(pdev, PCI_REVISION_ID, &accel_pci_dev->revid);
pci_read_config_dword(pdev, ADF_DEVICE_FUSECTL_OFFSET,
- &hw_data->fuses);
+ &hw_data->fuses[ADF_FUSECTL0]);
pci_read_config_dword(pdev, ADF_C3XXX_SOFTSTRAP_CSR_OFFSET,
&hw_data->straps);
diff --git a/drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c b/drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c
index 6b5b0cf9c7c7..32ebe09477a8 100644
--- a/drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c
+++ b/drivers/crypto/intel/qat/qat_c62x/adf_c62x_hw_data.c
@@ -27,8 +27,8 @@ static struct adf_hw_device_class c62x_class = {
static u32 get_accel_mask(struct adf_hw_device_data *self)
{
+ u32 fuses = self->fuses[ADF_FUSECTL0];
u32 straps = self->straps;
- u32 fuses = self->fuses;
u32 accel;
accel = ~(fuses | straps) >> ADF_C62X_ACCELERATORS_REG_OFFSET;
@@ -39,8 +39,8 @@ static u32 get_accel_mask(struct adf_hw_device_data *self)
static u32 get_ae_mask(struct adf_hw_device_data *self)
{
+ u32 fuses = self->fuses[ADF_FUSECTL0];
u32 straps = self->straps;
- u32 fuses = self->fuses;
unsigned long disabled;
u32 ae_disable;
int accel;
diff --git a/drivers/crypto/intel/qat/qat_c62x/adf_drv.c b/drivers/crypto/intel/qat/qat_c62x/adf_drv.c
index 5310149c311e..e8d8a057bbce 100644
--- a/drivers/crypto/intel/qat/qat_c62x/adf_drv.c
+++ b/drivers/crypto/intel/qat/qat_c62x/adf_drv.c
@@ -134,7 +134,7 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
adf_init_hw_data_c62x(accel_dev->hw_device);
pci_read_config_byte(pdev, PCI_REVISION_ID, &accel_pci_dev->revid);
pci_read_config_dword(pdev, ADF_DEVICE_FUSECTL_OFFSET,
- &hw_data->fuses);
+ &hw_data->fuses[ADF_FUSECTL0]);
pci_read_config_dword(pdev, ADF_C62X_SOFTSTRAP_CSR_OFFSET,
&hw_data->straps);
@@ -177,7 +177,7 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
hw_data->accel_capabilities_mask = hw_data->get_accel_cap(accel_dev);
/* Find and map all the device's BARS */
- i = (hw_data->fuses & ADF_DEVICE_FUSECTL_MASK) ? 1 : 0;
+ i = (hw_data->fuses[ADF_FUSECTL0] & ADF_DEVICE_FUSECTL_MASK) ? 1 : 0;
bar_mask = pci_select_bars(pdev, IORESOURCE_MEM);
for_each_set_bit(bar_nr, &bar_mask, ADF_PCI_MAX_BARS * 2) {
struct adf_bar *bar = &accel_pci_dev->pci_bars[i++];
diff --git a/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h b/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h
index 7830ecb1a1f1..cfe5bb9f5f7f 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h
+++ b/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h
@@ -52,6 +52,16 @@ enum adf_accel_capabilities {
ADF_ACCEL_CAPABILITIES_RANDOM_NUMBER = 128
};
+enum adf_fuses {
+ ADF_FUSECTL0,
+ ADF_FUSECTL1,
+ ADF_FUSECTL2,
+ ADF_FUSECTL3,
+ ADF_FUSECTL4,
+ ADF_FUSECTL5,
+ ADF_MAX_FUSES
+};
+
struct adf_bar {
resource_size_t base_addr;
void __iomem *virt_addr;
@@ -343,7 +353,7 @@ struct adf_hw_device_data {
struct qat_migdev_ops vfmig_ops;
const char *fw_name;
const char *fw_mmp_name;
- u32 fuses;
+ u32 fuses[ADF_MAX_FUSES];
u32 straps;
u32 accel_capabilities_mask;
u32 extended_dc_capabilities;
diff --git a/drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c b/drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c
index 1f64bf49b221..2b263442c856 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_gen2_hw_data.c
@@ -115,8 +115,8 @@ u32 adf_gen2_get_accel_cap(struct adf_accel_dev *accel_dev)
{
struct adf_hw_device_data *hw_data = accel_dev->hw_device;
struct pci_dev *pdev = accel_dev->accel_pci_dev.pci_dev;
+ u32 fuses = hw_data->fuses[ADF_FUSECTL0];
u32 straps = hw_data->straps;
- u32 fuses = hw_data->fuses;
u32 legfuses;
u32 capabilities = ICP_ACCEL_CAPABILITIES_CRYPTO_SYMMETRIC |
ICP_ACCEL_CAPABILITIES_CRYPTO_ASYMMETRIC |
diff --git a/drivers/crypto/intel/qat/qat_common/adf_sysfs_ras_counters.c b/drivers/crypto/intel/qat/qat_common/adf_sysfs_ras_counters.c
index e97c67c87b3c..6abb57bfd328 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_sysfs_ras_counters.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_sysfs_ras_counters.c
@@ -13,14 +13,14 @@ static ssize_t errors_correctable_show(struct device *dev,
char *buf)
{
struct adf_accel_dev *accel_dev;
- unsigned long counter;
+ int counter;
accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
if (!accel_dev)
return -EINVAL;
counter = ADF_RAS_ERR_CTR_READ(accel_dev->ras_errors, ADF_RAS_CORR);
- return scnprintf(buf, PAGE_SIZE, "%ld\n", counter);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", counter);
}
static ssize_t errors_nonfatal_show(struct device *dev,
@@ -28,14 +28,14 @@ static ssize_t errors_nonfatal_show(struct device *dev,
char *buf)
{
struct adf_accel_dev *accel_dev;
- unsigned long counter;
+ int counter;
accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
if (!accel_dev)
return -EINVAL;
counter = ADF_RAS_ERR_CTR_READ(accel_dev->ras_errors, ADF_RAS_UNCORR);
- return scnprintf(buf, PAGE_SIZE, "%ld\n", counter);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", counter);
}
static ssize_t errors_fatal_show(struct device *dev,
@@ -43,14 +43,14 @@ static ssize_t errors_fatal_show(struct device *dev,
char *buf)
{
struct adf_accel_dev *accel_dev;
- unsigned long counter;
+ int counter;
accel_dev = adf_devmgr_pci_to_accel_dev(to_pci_dev(dev));
if (!accel_dev)
return -EINVAL;
counter = ADF_RAS_ERR_CTR_READ(accel_dev->ras_errors, ADF_RAS_FATAL);
- return scnprintf(buf, PAGE_SIZE, "%ld\n", counter);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", counter);
}
static ssize_t reset_error_counters_store(struct device *dev,
diff --git a/drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h b/drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h
index 7ea8962272f2..d28732225c9e 100644
--- a/drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h
+++ b/drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h
@@ -3,6 +3,8 @@
#ifndef _ICP_QAT_HW_20_COMP_H_
#define _ICP_QAT_HW_20_COMP_H_
+#include <linux/swab.h>
+
#include "icp_qat_hw_20_comp_defs.h"
#include "icp_qat_fw.h"
@@ -54,7 +56,7 @@ ICP_QAT_FW_COMP_20_BUILD_CONFIG_LOWER(struct icp_qat_hw_comp_20_config_csr_lower
QAT_FIELD_SET(val32, csr.abd, ICP_QAT_HW_COMP_20_CONFIG_CSR_ABD_BITPOS,
ICP_QAT_HW_COMP_20_CONFIG_CSR_ABD_MASK);
- return __builtin_bswap32(val32);
+ return swab32(val32);
}
struct icp_qat_hw_comp_20_config_csr_upper {
@@ -106,7 +108,7 @@ ICP_QAT_FW_COMP_20_BUILD_CONFIG_UPPER(struct icp_qat_hw_comp_20_config_csr_upper
ICP_QAT_HW_COMP_20_CONFIG_CSR_NICE_PARAM_BITPOS,
ICP_QAT_HW_COMP_20_CONFIG_CSR_NICE_PARAM_MASK);
- return __builtin_bswap32(val32);
+ return swab32(val32);
}
struct icp_qat_hw_decomp_20_config_csr_lower {
@@ -138,7 +140,7 @@ ICP_QAT_FW_DECOMP_20_BUILD_CONFIG_LOWER(struct icp_qat_hw_decomp_20_config_csr_l
ICP_QAT_HW_DECOMP_20_CONFIG_CSR_LZ4_BLOCK_CHECKSUM_PRESENT_BITPOS,
ICP_QAT_HW_DECOMP_20_CONFIG_CSR_LZ4_BLOCK_CHECKSUM_PRESENT_MASK);
- return __builtin_bswap32(val32);
+ return swab32(val32);
}
struct icp_qat_hw_decomp_20_config_csr_upper {
@@ -158,7 +160,7 @@ ICP_QAT_FW_DECOMP_20_BUILD_CONFIG_UPPER(struct icp_qat_hw_decomp_20_config_csr_u
ICP_QAT_HW_DECOMP_20_CONFIG_CSR_MINI_CAM_CONTROL_BITPOS,
ICP_QAT_HW_DECOMP_20_CONFIG_CSR_MINI_CAM_CONTROL_MASK);
- return __builtin_bswap32(val32);
+ return swab32(val32);
}
#endif
diff --git a/drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c b/drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c
index c0661ff5e929..e48bcf1818cd 100644
--- a/drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c
+++ b/drivers/crypto/intel/qat/qat_dh895xcc/adf_dh895xcc_hw_data.c
@@ -29,7 +29,7 @@ static struct adf_hw_device_class dh895xcc_class = {
static u32 get_accel_mask(struct adf_hw_device_data *self)
{
- u32 fuses = self->fuses;
+ u32 fuses = self->fuses[ADF_FUSECTL0];
return ~fuses >> ADF_DH895XCC_ACCELERATORS_REG_OFFSET &
ADF_DH895XCC_ACCELERATORS_MASK;
@@ -37,7 +37,7 @@ static u32 get_accel_mask(struct adf_hw_device_data *self)
static u32 get_ae_mask(struct adf_hw_device_data *self)
{
- u32 fuses = self->fuses;
+ u32 fuses = self->fuses[ADF_FUSECTL0];
return ~fuses & ADF_DH895XCC_ACCELENGINES_MASK;
}
@@ -99,7 +99,7 @@ static u32 get_accel_cap(struct adf_accel_dev *accel_dev)
static enum dev_sku_info get_sku(struct adf_hw_device_data *self)
{
- int sku = (self->fuses & ADF_DH895XCC_FUSECTL_SKU_MASK)
+ int sku = (self->fuses[ADF_FUSECTL0] & ADF_DH895XCC_FUSECTL_SKU_MASK)
>> ADF_DH895XCC_FUSECTL_SKU_SHIFT;
switch (sku) {
diff --git a/drivers/crypto/intel/qat/qat_dh895xcc/adf_drv.c b/drivers/crypto/intel/qat/qat_dh895xcc/adf_drv.c
index 5ddf567ffcad..9a29bb15ef15 100644
--- a/drivers/crypto/intel/qat/qat_dh895xcc/adf_drv.c
+++ b/drivers/crypto/intel/qat/qat_dh895xcc/adf_drv.c
@@ -134,7 +134,7 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
adf_init_hw_data_dh895xcc(accel_dev->hw_device);
pci_read_config_byte(pdev, PCI_REVISION_ID, &accel_pci_dev->revid);
pci_read_config_dword(pdev, ADF_DEVICE_FUSECTL_OFFSET,
- &hw_data->fuses);
+ &hw_data->fuses[ADF_FUSECTL0]);
/* Get Accelerators and Accelerators Engines masks */
hw_data->accel_mask = hw_data->get_accel_mask(hw_data);
diff --git a/drivers/crypto/sa2ul.c b/drivers/crypto/sa2ul.c
index 461eca40e878..d1c2d129ad41 100644
--- a/drivers/crypto/sa2ul.c
+++ b/drivers/crypto/sa2ul.c
@@ -1775,13 +1775,13 @@ static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
static int sa_cra_init_aead_sha1(struct crypto_aead *tfm)
{
return sa_cra_init_aead(tfm, "sha1",
- "authenc(hmac(sha1-ce),cbc(aes-ce))");
+ "authenc(hmac(sha1),cbc(aes))");
}
static int sa_cra_init_aead_sha256(struct crypto_aead *tfm)
{
return sa_cra_init_aead(tfm, "sha256",
- "authenc(hmac(sha256-ce),cbc(aes-ce))");
+ "authenc(hmac(sha256),cbc(aes))");
}
static void sa_exit_tfm_aead(struct crypto_aead *tfm)
diff --git a/drivers/crypto/tegra/tegra-se-aes.c b/drivers/crypto/tegra/tegra-se-aes.c
index 073431a110bd..30c78afe3dea 100644
--- a/drivers/crypto/tegra/tegra-se-aes.c
+++ b/drivers/crypto/tegra/tegra-se-aes.c
@@ -4,6 +4,7 @@
* Crypto driver to handle block cipher algorithms using NVIDIA Security Engine.
*/
+#include <linux/bottom_half.h>
#include <linux/clk.h>
#include <linux/dma-mapping.h>
#include <linux/module.h>
@@ -28,6 +29,9 @@ struct tegra_aes_ctx {
u32 ivsize;
u32 key1_id;
u32 key2_id;
+ u32 keylen;
+ u8 key1[AES_MAX_KEY_SIZE];
+ u8 key2[AES_MAX_KEY_SIZE];
};
struct tegra_aes_reqctx {
@@ -43,8 +47,9 @@ struct tegra_aead_ctx {
struct tegra_se *se;
unsigned int authsize;
u32 alg;
- u32 keylen;
u32 key_id;
+ u32 keylen;
+ u8 key[AES_MAX_KEY_SIZE];
};
struct tegra_aead_reqctx {
@@ -56,8 +61,8 @@ struct tegra_aead_reqctx {
unsigned int cryptlen;
unsigned int authsize;
bool encrypt;
- u32 config;
u32 crypto_config;
+ u32 config;
u32 key_id;
u32 iv[4];
u8 authdata[16];
@@ -67,6 +72,8 @@ struct tegra_cmac_ctx {
struct tegra_se *se;
unsigned int alg;
u32 key_id;
+ u32 keylen;
+ u8 key[AES_MAX_KEY_SIZE];
struct crypto_shash *fallback_tfm;
};
@@ -260,11 +267,13 @@ static int tegra_aes_do_one_req(struct crypto_engine *engine, void *areq)
struct tegra_aes_ctx *ctx = crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
struct tegra_aes_reqctx *rctx = skcipher_request_ctx(req);
struct tegra_se *se = ctx->se;
- unsigned int cmdlen;
+ unsigned int cmdlen, key1_id, key2_id;
int ret;
rctx->iv = (ctx->alg == SE_ALG_ECB) ? NULL : (u32 *)req->iv;
rctx->len = req->cryptlen;
+ key1_id = ctx->key1_id;
+ key2_id = ctx->key2_id;
/* Pad input to AES Block size */
if (ctx->alg != SE_ALG_XTS) {
@@ -275,11 +284,36 @@ static int tegra_aes_do_one_req(struct crypto_engine *engine, void *areq)
rctx->datbuf.size = rctx->len;
rctx->datbuf.buf = dma_alloc_coherent(se->dev, rctx->datbuf.size,
&rctx->datbuf.addr, GFP_KERNEL);
- if (!rctx->datbuf.buf)
- return -ENOMEM;
+ if (!rctx->datbuf.buf) {
+ ret = -ENOMEM;
+ goto out_finalize;
+ }
scatterwalk_map_and_copy(rctx->datbuf.buf, req->src, 0, req->cryptlen, 0);
+ rctx->config = tegra234_aes_cfg(ctx->alg, rctx->encrypt);
+ rctx->crypto_config = tegra234_aes_crypto_cfg(ctx->alg, rctx->encrypt);
+
+ if (!key1_id) {
+ ret = tegra_key_submit_reserved_aes(ctx->se, ctx->key1,
+ ctx->keylen, ctx->alg, &key1_id);
+ if (ret)
+ goto out;
+ }
+
+ rctx->crypto_config |= SE_AES_KEY_INDEX(key1_id);
+
+ if (ctx->alg == SE_ALG_XTS) {
+ if (!key2_id) {
+ ret = tegra_key_submit_reserved_xts(ctx->se, ctx->key2,
+ ctx->keylen, ctx->alg, &key2_id);
+ if (ret)
+ goto out;
+ }
+
+ rctx->crypto_config |= SE_AES_KEY2_INDEX(key2_id);
+ }
+
/* Prepare the command and submit for execution */
cmdlen = tegra_aes_prep_cmd(ctx, rctx);
ret = tegra_se_host1x_submit(se, se->cmdbuf, cmdlen);
@@ -288,11 +322,21 @@ static int tegra_aes_do_one_req(struct crypto_engine *engine, void *areq)
tegra_aes_update_iv(req, ctx);
scatterwalk_map_and_copy(rctx->datbuf.buf, req->dst, 0, req->cryptlen, 1);
+out:
/* Free the buffer */
dma_free_coherent(ctx->se->dev, rctx->datbuf.size,
rctx->datbuf.buf, rctx->datbuf.addr);
+ if (tegra_key_is_reserved(key1_id))
+ tegra_key_invalidate_reserved(ctx->se, key1_id, ctx->alg);
+
+ if (tegra_key_is_reserved(key2_id))
+ tegra_key_invalidate_reserved(ctx->se, key2_id, ctx->alg);
+
+out_finalize:
+ local_bh_disable();
crypto_finalize_skcipher_request(se->engine, req, ret);
+ local_bh_enable();
return 0;
}
@@ -313,6 +357,7 @@ static int tegra_aes_cra_init(struct crypto_skcipher *tfm)
ctx->se = se_alg->se_dev;
ctx->key1_id = 0;
ctx->key2_id = 0;
+ ctx->keylen = 0;
algname = crypto_tfm_alg_name(&tfm->base);
ret = se_algname_to_algid(algname);
@@ -341,13 +386,20 @@ static int tegra_aes_setkey(struct crypto_skcipher *tfm,
const u8 *key, u32 keylen)
{
struct tegra_aes_ctx *ctx = crypto_skcipher_ctx(tfm);
+ int ret;
if (aes_check_keylen(keylen)) {
dev_dbg(ctx->se->dev, "invalid key length (%d)\n", keylen);
return -EINVAL;
}
- return tegra_key_submit(ctx->se, key, keylen, ctx->alg, &ctx->key1_id);
+ ret = tegra_key_submit(ctx->se, key, keylen, ctx->alg, &ctx->key1_id);
+ if (ret) {
+ ctx->keylen = keylen;
+ memcpy(ctx->key1, key, keylen);
+ }
+
+ return 0;
}
static int tegra_xts_setkey(struct crypto_skcipher *tfm,
@@ -365,11 +417,17 @@ static int tegra_xts_setkey(struct crypto_skcipher *tfm,
ret = tegra_key_submit(ctx->se, key, len,
ctx->alg, &ctx->key1_id);
- if (ret)
- return ret;
+ if (ret) {
+ ctx->keylen = len;
+ memcpy(ctx->key1, key, len);
+ }
- return tegra_key_submit(ctx->se, key + len, len,
+ ret = tegra_key_submit(ctx->se, key + len, len,
ctx->alg, &ctx->key2_id);
+ if (ret) {
+ ctx->keylen = len;
+ memcpy(ctx->key2, key + len, len);
+ }
return 0;
}
@@ -444,12 +502,6 @@ static int tegra_aes_crypt(struct skcipher_request *req, bool encrypt)
return 0;
rctx->encrypt = encrypt;
- rctx->config = tegra234_aes_cfg(ctx->alg, encrypt);
- rctx->crypto_config = tegra234_aes_crypto_cfg(ctx->alg, encrypt);
- rctx->crypto_config |= SE_AES_KEY_INDEX(ctx->key1_id);
-
- if (ctx->key2_id)
- rctx->crypto_config |= SE_AES_KEY2_INDEX(ctx->key2_id);
return crypto_transfer_skcipher_request_to_engine(ctx->se->engine, req);
}
@@ -716,7 +768,7 @@ static int tegra_gcm_do_gmac(struct tegra_aead_ctx *ctx, struct tegra_aead_reqct
rctx->config = tegra234_aes_cfg(SE_ALG_GMAC, rctx->encrypt);
rctx->crypto_config = tegra234_aes_crypto_cfg(SE_ALG_GMAC, rctx->encrypt) |
- SE_AES_KEY_INDEX(ctx->key_id);
+ SE_AES_KEY_INDEX(rctx->key_id);
cmdlen = tegra_gmac_prep_cmd(ctx, rctx);
@@ -733,7 +785,7 @@ static int tegra_gcm_do_crypt(struct tegra_aead_ctx *ctx, struct tegra_aead_reqc
rctx->config = tegra234_aes_cfg(SE_ALG_GCM, rctx->encrypt);
rctx->crypto_config = tegra234_aes_crypto_cfg(SE_ALG_GCM, rctx->encrypt) |
- SE_AES_KEY_INDEX(ctx->key_id);
+ SE_AES_KEY_INDEX(rctx->key_id);
/* Prepare command and submit */
cmdlen = tegra_gcm_crypt_prep_cmd(ctx, rctx);
@@ -756,7 +808,7 @@ static int tegra_gcm_do_final(struct tegra_aead_ctx *ctx, struct tegra_aead_reqc
rctx->config = tegra234_aes_cfg(SE_ALG_GCM_FINAL, rctx->encrypt);
rctx->crypto_config = tegra234_aes_crypto_cfg(SE_ALG_GCM_FINAL, rctx->encrypt) |
- SE_AES_KEY_INDEX(ctx->key_id);
+ SE_AES_KEY_INDEX(rctx->key_id);
/* Prepare command and submit */
cmdlen = tegra_gcm_prep_final_cmd(se, cpuvaddr, rctx);
@@ -887,7 +939,7 @@ static int tegra_ccm_do_cbcmac(struct tegra_aead_ctx *ctx, struct tegra_aead_req
rctx->config = tegra234_aes_cfg(SE_ALG_CBC_MAC, rctx->encrypt);
rctx->crypto_config = tegra234_aes_crypto_cfg(SE_ALG_CBC_MAC,
rctx->encrypt) |
- SE_AES_KEY_INDEX(ctx->key_id);
+ SE_AES_KEY_INDEX(rctx->key_id);
/* Prepare command and submit */
cmdlen = tegra_cbcmac_prep_cmd(ctx, rctx);
@@ -1074,7 +1126,7 @@ static int tegra_ccm_do_ctr(struct tegra_aead_ctx *ctx, struct tegra_aead_reqctx
rctx->config = tegra234_aes_cfg(SE_ALG_CTR, rctx->encrypt);
rctx->crypto_config = tegra234_aes_crypto_cfg(SE_ALG_CTR, rctx->encrypt) |
- SE_AES_KEY_INDEX(ctx->key_id);
+ SE_AES_KEY_INDEX(rctx->key_id);
/* Copy authdata in the top of buffer for encryption/decryption */
if (rctx->encrypt)
@@ -1153,21 +1205,30 @@ static int tegra_ccm_do_one_req(struct crypto_engine *engine, void *areq)
ret = tegra_ccm_crypt_init(req, se, rctx);
if (ret)
- return ret;
+ goto out_finalize;
+
+ rctx->key_id = ctx->key_id;
/* Allocate buffers required */
rctx->inbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen + 100;
rctx->inbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->inbuf.size,
&rctx->inbuf.addr, GFP_KERNEL);
if (!rctx->inbuf.buf)
- return -ENOMEM;
+ goto out_finalize;
rctx->outbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen + 100;
rctx->outbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->outbuf.size,
&rctx->outbuf.addr, GFP_KERNEL);
if (!rctx->outbuf.buf) {
ret = -ENOMEM;
- goto outbuf_err;
+ goto out_free_inbuf;
+ }
+
+ if (!ctx->key_id) {
+ ret = tegra_key_submit_reserved_aes(ctx->se, ctx->key,
+ ctx->keylen, ctx->alg, &rctx->key_id);
+ if (ret)
+ goto out;
}
if (rctx->encrypt) {
@@ -1196,11 +1257,17 @@ static int tegra_ccm_do_one_req(struct crypto_engine *engine, void *areq)
dma_free_coherent(ctx->se->dev, rctx->inbuf.size,
rctx->outbuf.buf, rctx->outbuf.addr);
-outbuf_err:
+out_free_inbuf:
dma_free_coherent(ctx->se->dev, rctx->outbuf.size,
rctx->inbuf.buf, rctx->inbuf.addr);
+ if (tegra_key_is_reserved(rctx->key_id))
+ tegra_key_invalidate_reserved(ctx->se, rctx->key_id, ctx->alg);
+
+out_finalize:
+ local_bh_disable();
crypto_finalize_aead_request(ctx->se->engine, req, ret);
+ local_bh_enable();
return 0;
}
@@ -1226,19 +1293,30 @@ static int tegra_gcm_do_one_req(struct crypto_engine *engine, void *areq)
memcpy(rctx->iv, req->iv, GCM_AES_IV_SIZE);
rctx->iv[3] = (1 << 24);
+ rctx->key_id = ctx->key_id;
+
/* Allocate buffers required */
rctx->inbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen;
rctx->inbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->inbuf.size,
&rctx->inbuf.addr, GFP_KERNEL);
- if (!rctx->inbuf.buf)
- return -ENOMEM;
+ if (!rctx->inbuf.buf) {
+ ret = -ENOMEM;
+ goto out_finalize;
+ }
rctx->outbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen;
rctx->outbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->outbuf.size,
&rctx->outbuf.addr, GFP_KERNEL);
if (!rctx->outbuf.buf) {
ret = -ENOMEM;
- goto outbuf_err;
+ goto out_free_inbuf;
+ }
+
+ if (!ctx->key_id) {
+ ret = tegra_key_submit_reserved_aes(ctx->se, ctx->key,
+ ctx->keylen, ctx->alg, &rctx->key_id);
+ if (ret)
+ goto out;
}
/* If there is associated data perform GMAC operation */
@@ -1267,12 +1345,17 @@ static int tegra_gcm_do_one_req(struct crypto_engine *engine, void *areq)
dma_free_coherent(ctx->se->dev, rctx->outbuf.size,
rctx->outbuf.buf, rctx->outbuf.addr);
-outbuf_err:
+out_free_inbuf:
dma_free_coherent(ctx->se->dev, rctx->inbuf.size,
rctx->inbuf.buf, rctx->inbuf.addr);
- /* Finalize the request if there are no errors */
+ if (tegra_key_is_reserved(rctx->key_id))
+ tegra_key_invalidate_reserved(ctx->se, rctx->key_id, ctx->alg);
+
+out_finalize:
+ local_bh_disable();
crypto_finalize_aead_request(ctx->se->engine, req, ret);
+ local_bh_enable();
return 0;
}
@@ -1293,6 +1376,7 @@ static int tegra_aead_cra_init(struct crypto_aead *tfm)
ctx->se = se_alg->se_dev;
ctx->key_id = 0;
+ ctx->keylen = 0;
ret = se_algname_to_algid(algname);
if (ret < 0) {
@@ -1374,13 +1458,20 @@ static int tegra_aead_setkey(struct crypto_aead *tfm,
const u8 *key, u32 keylen)
{
struct tegra_aead_ctx *ctx = crypto_aead_ctx(tfm);
+ int ret;
if (aes_check_keylen(keylen)) {
dev_dbg(ctx->se->dev, "invalid key length (%d)\n", keylen);
return -EINVAL;
}
- return tegra_key_submit(ctx->se, key, keylen, ctx->alg, &ctx->key_id);
+ ret = tegra_key_submit(ctx->se, key, keylen, ctx->alg, &ctx->key_id);
+ if (ret) {
+ ctx->keylen = keylen;
+ memcpy(ctx->key, key, keylen);
+ }
+
+ return 0;
}
static unsigned int tegra_cmac_prep_cmd(struct tegra_cmac_ctx *ctx,
@@ -1454,6 +1545,35 @@ static void tegra_cmac_paste_result(struct tegra_se *se, struct tegra_cmac_reqct
se->base + se->hw->regs->result + (i * 4));
}
+static int tegra_cmac_do_init(struct ahash_request *req)
+{
+ struct tegra_cmac_reqctx *rctx = ahash_request_ctx(req);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct tegra_cmac_ctx *ctx = crypto_ahash_ctx(tfm);
+ struct tegra_se *se = ctx->se;
+ int i;
+
+ rctx->total_len = 0;
+ rctx->datbuf.size = 0;
+ rctx->residue.size = 0;
+ rctx->key_id = ctx->key_id;
+ rctx->task |= SHA_FIRST;
+ rctx->blk_size = crypto_ahash_blocksize(tfm);
+
+ rctx->residue.buf = dma_alloc_coherent(se->dev, rctx->blk_size * 2,
+ &rctx->residue.addr, GFP_KERNEL);
+ if (!rctx->residue.buf)
+ return -ENOMEM;
+
+ rctx->residue.size = 0;
+
+ /* Clear any previous result */
+ for (i = 0; i < CMAC_RESULT_REG_COUNT; i++)
+ writel(0, se->base + se->hw->regs->result + (i * 4));
+
+ return 0;
+}
+
static int tegra_cmac_do_update(struct ahash_request *req)
{
struct tegra_cmac_reqctx *rctx = ahash_request_ctx(req);
@@ -1481,7 +1601,7 @@ static int tegra_cmac_do_update(struct ahash_request *req)
rctx->datbuf.size = (req->nbytes + rctx->residue.size) - nresidue;
rctx->total_len += rctx->datbuf.size;
rctx->config = tegra234_aes_cfg(SE_ALG_CMAC, 0);
- rctx->crypto_config = SE_AES_KEY_INDEX(ctx->key_id);
+ rctx->crypto_config = SE_AES_KEY_INDEX(rctx->key_id);
/*
* Keep one block and residue bytes in residue and
@@ -1599,6 +1719,21 @@ static int tegra_cmac_do_one_req(struct crypto_engine *engine, void *areq)
struct tegra_se *se = ctx->se;
int ret = 0;
+ if (rctx->task & SHA_INIT) {
+ ret = tegra_cmac_do_init(req);
+ if (ret)
+ goto out;
+
+ rctx->task &= ~SHA_INIT;
+ }
+
+ if (!ctx->key_id) {
+ ret = tegra_key_submit_reserved_aes(ctx->se, ctx->key,
+ ctx->keylen, ctx->alg, &rctx->key_id);
+ if (ret)
+ goto out;
+ }
+
if (rctx->task & SHA_UPDATE) {
ret = tegra_cmac_do_update(req);
if (ret)
@@ -1615,7 +1750,12 @@ static int tegra_cmac_do_one_req(struct crypto_engine *engine, void *areq)
rctx->task &= ~SHA_FINAL;
}
out:
+ if (tegra_key_is_reserved(rctx->key_id))
+ tegra_key_invalidate_reserved(ctx->se, rctx->key_id, ctx->alg);
+
+ local_bh_disable();
crypto_finalize_hash_request(se->engine, req, ret);
+ local_bh_enable();
return 0;
}
@@ -1655,6 +1795,7 @@ static int tegra_cmac_cra_init(struct crypto_tfm *tfm)
ctx->se = se_alg->se_dev;
ctx->key_id = 0;
+ ctx->keylen = 0;
ret = se_algname_to_algid(algname);
if (ret < 0) {
@@ -1679,38 +1820,11 @@ static void tegra_cmac_cra_exit(struct crypto_tfm *tfm)
tegra_key_invalidate(ctx->se, ctx->key_id, ctx->alg);
}
-static int tegra_cmac_init(struct ahash_request *req)
-{
- struct tegra_cmac_reqctx *rctx = ahash_request_ctx(req);
- struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
- struct tegra_cmac_ctx *ctx = crypto_ahash_ctx(tfm);
- struct tegra_se *se = ctx->se;
- int i;
-
- rctx->total_len = 0;
- rctx->datbuf.size = 0;
- rctx->residue.size = 0;
- rctx->task = SHA_FIRST;
- rctx->blk_size = crypto_ahash_blocksize(tfm);
-
- rctx->residue.buf = dma_alloc_coherent(se->dev, rctx->blk_size * 2,
- &rctx->residue.addr, GFP_KERNEL);
- if (!rctx->residue.buf)
- return -ENOMEM;
-
- rctx->residue.size = 0;
-
- /* Clear any previous result */
- for (i = 0; i < CMAC_RESULT_REG_COUNT; i++)
- writel(0, se->base + se->hw->regs->result + (i * 4));
-
- return 0;
-}
-
static int tegra_cmac_setkey(struct crypto_ahash *tfm, const u8 *key,
unsigned int keylen)
{
struct tegra_cmac_ctx *ctx = crypto_ahash_ctx(tfm);
+ int ret;
if (aes_check_keylen(keylen)) {
dev_dbg(ctx->se->dev, "invalid key length (%d)\n", keylen);
@@ -1720,7 +1834,24 @@ static int tegra_cmac_setkey(struct crypto_ahash *tfm, const u8 *key,
if (ctx->fallback_tfm)
crypto_shash_setkey(ctx->fallback_tfm, key, keylen);
- return tegra_key_submit(ctx->se, key, keylen, ctx->alg, &ctx->key_id);
+ ret = tegra_key_submit(ctx->se, key, keylen, ctx->alg, &ctx->key_id);
+ if (ret) {
+ ctx->keylen = keylen;
+ memcpy(ctx->key, key, keylen);
+ }
+
+ return 0;
+}
+
+static int tegra_cmac_init(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct tegra_cmac_ctx *ctx = crypto_ahash_ctx(tfm);
+ struct tegra_cmac_reqctx *rctx = ahash_request_ctx(req);
+
+ rctx->task = SHA_INIT;
+
+ return crypto_transfer_hash_request_to_engine(ctx->se->engine, req);
}
static int tegra_cmac_update(struct ahash_request *req)
@@ -1761,13 +1892,9 @@ static int tegra_cmac_digest(struct ahash_request *req)
struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
struct tegra_cmac_ctx *ctx = crypto_ahash_ctx(tfm);
struct tegra_cmac_reqctx *rctx = ahash_request_ctx(req);
- int ret;
- ret = tegra_cmac_init(req);
- if (ret)
- return ret;
+ rctx->task |= SHA_INIT | SHA_UPDATE | SHA_FINAL;
- rctx->task |= SHA_UPDATE | SHA_FINAL;
return crypto_transfer_hash_request_to_engine(ctx->se->engine, req);
}
diff --git a/drivers/crypto/tegra/tegra-se-hash.c b/drivers/crypto/tegra/tegra-se-hash.c
index fb28b7ef726a..327f9c802f6d 100644
--- a/drivers/crypto/tegra/tegra-se-hash.c
+++ b/drivers/crypto/tegra/tegra-se-hash.c
@@ -4,6 +4,7 @@
* Crypto driver to handle HASH algorithms using NVIDIA Security Engine.
*/
+#include <linux/bottom_half.h>
#include <linux/clk.h>
#include <linux/dma-mapping.h>
#include <linux/module.h>
@@ -296,6 +297,44 @@ static void tegra_sha_paste_hash_result(struct tegra_se *se, struct tegra_sha_re
se->base + se->hw->regs->result + (i * 4));
}
+static int tegra_sha_do_init(struct ahash_request *req)
+{
+ struct tegra_sha_reqctx *rctx = ahash_request_ctx(req);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct tegra_sha_ctx *ctx = crypto_ahash_ctx(tfm);
+ struct tegra_se *se = ctx->se;
+
+ if (ctx->fallback)
+ return tegra_sha_fallback_init(req);
+
+ rctx->total_len = 0;
+ rctx->datbuf.size = 0;
+ rctx->residue.size = 0;
+ rctx->key_id = ctx->key_id;
+ rctx->task |= SHA_FIRST;
+ rctx->alg = ctx->alg;
+ rctx->blk_size = crypto_ahash_blocksize(tfm);
+ rctx->digest.size = crypto_ahash_digestsize(tfm);
+
+ rctx->digest.buf = dma_alloc_coherent(se->dev, rctx->digest.size,
+ &rctx->digest.addr, GFP_KERNEL);
+ if (!rctx->digest.buf)
+ goto digbuf_fail;
+
+ rctx->residue.buf = dma_alloc_coherent(se->dev, rctx->blk_size,
+ &rctx->residue.addr, GFP_KERNEL);
+ if (!rctx->residue.buf)
+ goto resbuf_fail;
+
+ return 0;
+
+resbuf_fail:
+ dma_free_coherent(se->dev, rctx->digest.size, rctx->digest.buf,
+ rctx->digest.addr);
+digbuf_fail:
+ return -ENOMEM;
+}
+
static int tegra_sha_do_update(struct ahash_request *req)
{
struct tegra_sha_ctx *ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(req));
@@ -435,6 +474,14 @@ static int tegra_sha_do_one_req(struct crypto_engine *engine, void *areq)
struct tegra_se *se = ctx->se;
int ret = 0;
+ if (rctx->task & SHA_INIT) {
+ ret = tegra_sha_do_init(req);
+ if (ret)
+ goto out;
+
+ rctx->task &= ~SHA_INIT;
+ }
+
if (rctx->task & SHA_UPDATE) {
ret = tegra_sha_do_update(req);
if (ret)
@@ -452,7 +499,9 @@ static int tegra_sha_do_one_req(struct crypto_engine *engine, void *areq)
}
out:
+ local_bh_disable();
crypto_finalize_hash_request(se->engine, req, ret);
+ local_bh_enable();
return 0;
}
@@ -525,44 +574,6 @@ static void tegra_sha_cra_exit(struct crypto_tfm *tfm)
tegra_key_invalidate(ctx->se, ctx->key_id, ctx->alg);
}
-static int tegra_sha_init(struct ahash_request *req)
-{
- struct tegra_sha_reqctx *rctx = ahash_request_ctx(req);
- struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
- struct tegra_sha_ctx *ctx = crypto_ahash_ctx(tfm);
- struct tegra_se *se = ctx->se;
-
- if (ctx->fallback)
- return tegra_sha_fallback_init(req);
-
- rctx->total_len = 0;
- rctx->datbuf.size = 0;
- rctx->residue.size = 0;
- rctx->key_id = ctx->key_id;
- rctx->task = SHA_FIRST;
- rctx->alg = ctx->alg;
- rctx->blk_size = crypto_ahash_blocksize(tfm);
- rctx->digest.size = crypto_ahash_digestsize(tfm);
-
- rctx->digest.buf = dma_alloc_coherent(se->dev, rctx->digest.size,
- &rctx->digest.addr, GFP_KERNEL);
- if (!rctx->digest.buf)
- goto digbuf_fail;
-
- rctx->residue.buf = dma_alloc_coherent(se->dev, rctx->blk_size,
- &rctx->residue.addr, GFP_KERNEL);
- if (!rctx->residue.buf)
- goto resbuf_fail;
-
- return 0;
-
-resbuf_fail:
- dma_free_coherent(se->dev, rctx->digest.size, rctx->digest.buf,
- rctx->digest.addr);
-digbuf_fail:
- return -ENOMEM;
-}
-
static int tegra_hmac_fallback_setkey(struct tegra_sha_ctx *ctx, const u8 *key,
unsigned int keylen)
{
@@ -593,6 +604,17 @@ static int tegra_hmac_setkey(struct crypto_ahash *tfm, const u8 *key,
return 0;
}
+static int tegra_sha_init(struct ahash_request *req)
+{
+ struct tegra_sha_reqctx *rctx = ahash_request_ctx(req);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct tegra_sha_ctx *ctx = crypto_ahash_ctx(tfm);
+
+ rctx->task = SHA_INIT;
+
+ return crypto_transfer_hash_request_to_engine(ctx->se->engine, req);
+}
+
static int tegra_sha_update(struct ahash_request *req)
{
struct tegra_sha_reqctx *rctx = ahash_request_ctx(req);
@@ -640,16 +662,12 @@ static int tegra_sha_digest(struct ahash_request *req)
struct tegra_sha_reqctx *rctx = ahash_request_ctx(req);
struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
struct tegra_sha_ctx *ctx = crypto_ahash_ctx(tfm);
- int ret;
if (ctx->fallback)
return tegra_sha_fallback_digest(req);
- ret = tegra_sha_init(req);
- if (ret)
- return ret;
+ rctx->task |= SHA_INIT | SHA_UPDATE | SHA_FINAL;
- rctx->task |= SHA_UPDATE | SHA_FINAL;
return crypto_transfer_hash_request_to_engine(ctx->se->engine, req);
}
diff --git a/drivers/crypto/tegra/tegra-se-key.c b/drivers/crypto/tegra/tegra-se-key.c
index 276b261fb6df..956fa9b4e9b1 100644
--- a/drivers/crypto/tegra/tegra-se-key.c
+++ b/drivers/crypto/tegra/tegra-se-key.c
@@ -141,6 +141,23 @@ void tegra_key_invalidate(struct tegra_se *se, u32 keyid, u32 alg)
tegra_keyslot_free(keyid);
}
+void tegra_key_invalidate_reserved(struct tegra_se *se, u32 keyid, u32 alg)
+{
+ u8 zkey[AES_MAX_KEY_SIZE] = {0};
+
+ if (!keyid)
+ return;
+
+ /* Overwrite the key with 0s */
+ tegra_key_insert(se, zkey, AES_MAX_KEY_SIZE, keyid, alg);
+}
+
+inline int tegra_key_submit_reserved(struct tegra_se *se, const u8 *key,
+ u32 keylen, u32 alg, u32 *keyid)
+{
+ return tegra_key_insert(se, key, keylen, *keyid, alg);
+}
+
int tegra_key_submit(struct tegra_se *se, const u8 *key, u32 keylen, u32 alg, u32 *keyid)
{
int ret;
@@ -149,7 +166,7 @@ int tegra_key_submit(struct tegra_se *se, const u8 *key, u32 keylen, u32 alg, u3
if (!tegra_key_in_kslt(*keyid)) {
*keyid = tegra_keyslot_alloc();
if (!(*keyid)) {
- dev_err(se->dev, "failed to allocate key slot\n");
+ dev_dbg(se->dev, "failed to allocate key slot\n");
return -ENOMEM;
}
}
diff --git a/drivers/crypto/tegra/tegra-se.h b/drivers/crypto/tegra/tegra-se.h
index e196a90eedb9..85674703a960 100644
--- a/drivers/crypto/tegra/tegra-se.h
+++ b/drivers/crypto/tegra/tegra-se.h
@@ -341,9 +341,13 @@
#define SE_MAX_KEYSLOT 15
#define SE_MAX_MEM_ALLOC SZ_4M
+#define TEGRA_AES_RESERVED_KSLT 14
+#define TEGRA_XTS_RESERVED_KSLT 15
+
#define SHA_FIRST BIT(0)
-#define SHA_UPDATE BIT(1)
-#define SHA_FINAL BIT(2)
+#define SHA_INIT BIT(1)
+#define SHA_UPDATE BIT(2)
+#define SHA_FINAL BIT(3)
/* Security Engine operation modes */
enum se_aes_alg {
@@ -500,9 +504,34 @@ void tegra_deinit_aes(struct tegra_se *se);
void tegra_deinit_hash(struct tegra_se *se);
int tegra_key_submit(struct tegra_se *se, const u8 *key,
u32 keylen, u32 alg, u32 *keyid);
+
+int tegra_key_submit_reserved(struct tegra_se *se, const u8 *key,
+ u32 keylen, u32 alg, u32 *keyid);
+
void tegra_key_invalidate(struct tegra_se *se, u32 keyid, u32 alg);
+void tegra_key_invalidate_reserved(struct tegra_se *se, u32 keyid, u32 alg);
int tegra_se_host1x_submit(struct tegra_se *se, struct tegra_se_cmdbuf *cmdbuf, u32 size);
+static inline int tegra_key_submit_reserved_aes(struct tegra_se *se, const u8 *key,
+ u32 keylen, u32 alg, u32 *keyid)
+{
+ *keyid = TEGRA_AES_RESERVED_KSLT;
+ return tegra_key_submit_reserved(se, key, keylen, alg, keyid);
+}
+
+static inline int tegra_key_submit_reserved_xts(struct tegra_se *se, const u8 *key,
+ u32 keylen, u32 alg, u32 *keyid)
+{
+ *keyid = TEGRA_XTS_RESERVED_KSLT;
+ return tegra_key_submit_reserved(se, key, keylen, alg, keyid);
+}
+
+static inline bool tegra_key_is_reserved(u32 keyid)
+{
+ return ((keyid == TEGRA_AES_RESERVED_KSLT) ||
+ (keyid == TEGRA_XTS_RESERVED_KSLT));
+}
+
/* HOST1x OPCODES */
static inline u32 host1x_opcode_setpayload(unsigned int payload)
{
diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
index 6e553b5752b1..09a2bc817d64 100644
--- a/drivers/cxl/pci.c
+++ b/drivers/cxl/pci.c
@@ -994,6 +994,9 @@ static void cxl_reset_done(struct pci_dev *pdev)
* that no longer exists.
*/
guard(device)(&cxlmd->dev);
+ if (!cxlmd->dev.driver)
+ return;
+
if (cxlmd->endpoint &&
cxl_endpoint_decoder_reset_detected(cxlmd->endpoint)) {
dev_crit(dev, "SBR happened without memory regions removal.\n");
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index fffafa86d964..adef111aca31 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -592,8 +592,6 @@ static void dw_axi_dma_set_hw_channel(struct axi_dma_chan *chan, bool set)
(chan->id * DMA_APB_HS_SEL_BIT_SIZE));
reg_value |= (val << (chan->id * DMA_APB_HS_SEL_BIT_SIZE));
lo_hi_writeq(reg_value, chip->apb_regs + DMAC_APB_HW_HS_SEL_0);
-
- return;
}
/*
diff --git a/drivers/dma/mxs-dma.c b/drivers/dma/mxs-dma.c
index cfb9962417ef..53f572b6b6fc 100644
--- a/drivers/dma/mxs-dma.c
+++ b/drivers/dma/mxs-dma.c
@@ -824,6 +824,7 @@ static int mxs_dma_probe(struct platform_device *pdev)
if (ret) {
dev_err(mxs_dma->dma_device.dev,
"failed to register controller\n");
+ return ret;
}
dev_info(mxs_dma->dma_device.dev, "initialized\n");
diff --git a/drivers/firmware/arm_ffa/driver.c b/drivers/firmware/arm_ffa/driver.c
index bec1fbaff7f3..15e71a53956e 100644
--- a/drivers/firmware/arm_ffa/driver.c
+++ b/drivers/firmware/arm_ffa/driver.c
@@ -1804,7 +1804,7 @@ static int __init ffa_init(void)
ret = ffa_rxtx_map(virt_to_phys(drv_info->tx_buffer),
virt_to_phys(drv_info->rx_buffer),
- rxtx_bufsz / FFA_PAGE_SIZE);
+ PAGE_ALIGN(rxtx_bufsz) / FFA_PAGE_SIZE);
if (ret) {
pr_err("failed to register FFA RxTx buffers\n");
goto free_pages;
diff --git a/drivers/firmware/efi/capsule-loader.c b/drivers/firmware/efi/capsule-loader.c
index 0c17bdd388e1..bbddeb6a0955 100644
--- a/drivers/firmware/efi/capsule-loader.c
+++ b/drivers/firmware/efi/capsule-loader.c
@@ -67,7 +67,7 @@ int __efi_capsule_setup_info(struct capsule_info *cap_info)
cap_info->pages = temp_page;
temp_page = krealloc(cap_info->phys,
- pages_needed * sizeof(phys_addr_t *),
+ pages_needed * sizeof(phys_addr_t),
GFP_KERNEL | __GFP_ZERO);
if (!temp_page)
return -ENOMEM;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 7edf8d67a0fa..b667da0ec68a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -388,6 +388,7 @@ struct amdgpu_ip_block_version {
struct amdgpu_ip_block {
struct amdgpu_ip_block_status status;
const struct amdgpu_ip_block_version *version;
+ struct amdgpu_device *adev;
};
int amdgpu_device_ip_block_version_cmp(struct amdgpu_device *adev,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c
index bf6c4a0d0525..c4ca59875679 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c
@@ -585,7 +585,7 @@ static int acp_resume(void *handle)
return 0;
}
-static int acp_early_init(void *handle)
+static int acp_early_init(struct amdgpu_ip_block *ip_block)
{
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
index 54067edb7747..47bef5e7747c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
@@ -1247,6 +1247,8 @@ static enum drm_mode_status amdgpu_connector_dvi_mode_valid(struct drm_connector
case CONNECTOR_OBJECT_ID_HDMI_TYPE_B:
max_digital_pixel_clock_khz = max_dvi_single_link_pixel_clock * 2;
break;
+ default:
+ return MODE_BAD;
}
/* When the display EDID claims that it's an HDMI display,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 12d7e45a4245..13e8bc2426f8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -2336,6 +2336,8 @@ int amdgpu_device_ip_block_add(struct amdgpu_device *adev,
DRM_INFO("add ip block number %d <%s>\n", adev->num_ip_blocks,
ip_block_version->funcs->name);
+ adev->ip_blocks[adev->num_ip_blocks].adev = adev;
+
adev->ip_blocks[adev->num_ip_blocks++].version = ip_block_version;
return 0;
@@ -2637,25 +2639,25 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
total = true;
for (i = 0; i < adev->num_ip_blocks; i++) {
+ ip_block = &adev->ip_blocks[i];
+
if ((amdgpu_ip_block_mask & (1 << i)) == 0) {
DRM_WARN("disabled ip block: %d <%s>\n",
i, adev->ip_blocks[i].version->funcs->name);
adev->ip_blocks[i].status.valid = false;
- } else {
- if (adev->ip_blocks[i].version->funcs->early_init) {
- r = adev->ip_blocks[i].version->funcs->early_init((void *)adev);
- if (r == -ENOENT) {
- adev->ip_blocks[i].status.valid = false;
- } else if (r) {
- DRM_ERROR("early_init of IP block <%s> failed %d\n",
- adev->ip_blocks[i].version->funcs->name, r);
- total = false;
- } else {
- adev->ip_blocks[i].status.valid = true;
- }
+ } else if (ip_block->version->funcs->early_init) {
+ r = ip_block->version->funcs->early_init(ip_block);
+ if (r == -ENOENT) {
+ adev->ip_blocks[i].status.valid = false;
+ } else if (r) {
+ DRM_ERROR("early_init of IP block <%s> failed %d\n",
+ adev->ip_blocks[i].version->funcs->name, r);
+ total = false;
} else {
adev->ip_blocks[i].status.valid = true;
}
+ } else {
+ adev->ip_blocks[i].status.valid = true;
}
/* get the vbios after the asic_funcs are set up */
if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_COMMON) {
@@ -5464,7 +5466,7 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
for (i = 0; i < tmp_adev->num_ip_blocks; i++)
if (tmp_adev->ip_blocks[i].version->funcs->dump_ip_state)
tmp_adev->ip_blocks[i].version->funcs
- ->dump_ip_state((void *)tmp_adev);
+ ->dump_ip_state((void *)&tmp_adev->ip_blocks[i]);
dev_info(tmp_adev->dev, "Dumping IP State Completed\n");
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index 564c68c9277b..f63b5a429b10 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -269,7 +269,7 @@ void amdgpu_gmc_sysvm_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc
* @mc: memory controller structure holding memory information
* @gart_placement: GART placement policy with respect to VRAM
*
- * Function will place try to place GART before or after VRAM.
+ * Function will try to place GART before or after VRAM.
* If GART size is bigger than space left then we ajust GART size.
* Thus function will never fails.
*/
@@ -297,7 +297,10 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
mc->gart_start = max_mc_address - mc->gart_size + 1;
break;
case AMDGPU_GART_PLACEMENT_LOW:
- mc->gart_start = 0;
+ if (size_bf >= mc->gart_size)
+ mc->gart_start = 0;
+ else
+ mc->gart_start = ALIGN(mc->fb_end, four_gb);
break;
case AMDGPU_GART_PLACEMENT_BEST_FIT:
default:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_isp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_isp.c
index 4766e99dd98f..7c1f17dc6b4b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_isp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_isp.c
@@ -122,9 +122,10 @@ static int isp_load_fw_by_psp(struct amdgpu_device *adev)
return r;
}
-static int isp_early_init(void *handle)
+static int isp_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ struct amdgpu_device *adev = ip_block->adev;
struct amdgpu_isp *isp = &adev->isp;
switch (amdgpu_ip_version(adev, ISP_HWIP, 0)) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index ba9a9adca0bf..70762c7fcfe4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -42,7 +42,7 @@ static void amdgpu_job_do_core_dump(struct amdgpu_device *adev,
for (i = 0; i < adev->num_ip_blocks; i++)
if (adev->ip_blocks[i].version->funcs->dump_ip_state)
adev->ip_blocks[i].version->funcs
- ->dump_ip_state((void *)adev);
+ ->dump_ip_state((void *)&adev->ip_blocks[i]);
dev_info(adev->dev, "Dumping IP State Completed\n");
amdgpu_coredump(adev, true, false, job);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index c626f66ded18..6788265157db 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -786,68 +786,59 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
? -EFAULT : 0;
}
case AMDGPU_INFO_READ_MMR_REG: {
- int ret = 0;
- unsigned int n, alloc_size;
- uint32_t *regs;
unsigned int se_num = (info->read_mmr_reg.instance >>
AMDGPU_INFO_MMR_SE_INDEX_SHIFT) &
AMDGPU_INFO_MMR_SE_INDEX_MASK;
unsigned int sh_num = (info->read_mmr_reg.instance >>
AMDGPU_INFO_MMR_SH_INDEX_SHIFT) &
AMDGPU_INFO_MMR_SH_INDEX_MASK;
-
- if (!down_read_trylock(&adev->reset_domain->sem))
- return -ENOENT;
+ unsigned int alloc_size;
+ uint32_t *regs;
+ int ret;
/* set full masks if the userspace set all bits
* in the bitfields
*/
- if (se_num == AMDGPU_INFO_MMR_SE_INDEX_MASK) {
+ if (se_num == AMDGPU_INFO_MMR_SE_INDEX_MASK)
se_num = 0xffffffff;
- } else if (se_num >= AMDGPU_GFX_MAX_SE) {
- ret = -EINVAL;
- goto out;
- }
+ else if (se_num >= AMDGPU_GFX_MAX_SE)
+ return -EINVAL;
- if (sh_num == AMDGPU_INFO_MMR_SH_INDEX_MASK) {
+ if (sh_num == AMDGPU_INFO_MMR_SH_INDEX_MASK)
sh_num = 0xffffffff;
- } else if (sh_num >= AMDGPU_GFX_MAX_SH_PER_SE) {
- ret = -EINVAL;
- goto out;
- }
+ else if (sh_num >= AMDGPU_GFX_MAX_SH_PER_SE)
+ return -EINVAL;
- if (info->read_mmr_reg.count > 128) {
- ret = -EINVAL;
- goto out;
- }
+ if (info->read_mmr_reg.count > 128)
+ return -EINVAL;
- regs = kmalloc_array(info->read_mmr_reg.count, sizeof(*regs), GFP_KERNEL);
- if (!regs) {
- ret = -ENOMEM;
- goto out;
- }
+ regs = kmalloc_array(info->read_mmr_reg.count, sizeof(*regs),
+ GFP_KERNEL);
+ if (!regs)
+ return -ENOMEM;
+ down_read(&adev->reset_domain->sem);
alloc_size = info->read_mmr_reg.count * sizeof(*regs);
-
amdgpu_gfx_off_ctrl(adev, false);
+ ret = 0;
for (i = 0; i < info->read_mmr_reg.count; i++) {
if (amdgpu_asic_read_register(adev, se_num, sh_num,
info->read_mmr_reg.dword_offset + i,
®s[i])) {
DRM_DEBUG_KMS("unallowed offset %#x\n",
info->read_mmr_reg.dword_offset + i);
- kfree(regs);
- amdgpu_gfx_off_ctrl(adev, true);
ret = -EFAULT;
- goto out;
+ break;
}
}
amdgpu_gfx_off_ctrl(adev, true);
- n = copy_to_user(out, regs, min(size, alloc_size));
- kfree(regs);
- ret = (n ? -EFAULT : 0);
-out:
up_read(&adev->reset_domain->sem);
+
+ if (!ret) {
+ ret = copy_to_user(out, regs, min(size, alloc_size))
+ ? -EFAULT : 0;
+ }
+ kfree(regs);
return ret;
}
case AMDGPU_INFO_DEV_INFO: {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 26260873f6a1..382125c64e4c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -159,9 +159,9 @@ static int psp_init_sriov_microcode(struct psp_context *psp)
return ret;
}
-static int psp_early_init(void *handle)
+static int psp_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
struct psp_context *psp = &adev->psp;
psp->autoload_supported = true;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sched.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sched.c
index b0a8abc7a8ec..341beec59537 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sched.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sched.c
@@ -35,21 +35,19 @@ static int amdgpu_sched_process_priority_override(struct amdgpu_device *adev,
int fd,
int32_t priority)
{
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
struct amdgpu_fpriv *fpriv;
struct amdgpu_ctx_mgr *mgr;
struct amdgpu_ctx *ctx;
uint32_t id;
int r;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EINVAL;
r = amdgpu_file_to_fpriv(fd_file(f), &fpriv);
- if (r) {
- fdput(f);
+ if (r)
return r;
- }
mgr = &fpriv->ctx_mgr;
mutex_lock(&mgr->lock);
@@ -57,7 +55,6 @@ static int amdgpu_sched_process_priority_override(struct amdgpu_device *adev,
amdgpu_ctx_priority_override(ctx, priority);
mutex_unlock(&mgr->lock);
- fdput(f);
return 0;
}
@@ -66,31 +63,25 @@ static int amdgpu_sched_context_priority_override(struct amdgpu_device *adev,
unsigned ctx_id,
int32_t priority)
{
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
struct amdgpu_fpriv *fpriv;
struct amdgpu_ctx *ctx;
int r;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EINVAL;
r = amdgpu_file_to_fpriv(fd_file(f), &fpriv);
- if (r) {
- fdput(f);
+ if (r)
return r;
- }
ctx = amdgpu_ctx_get(fpriv, ctx_id);
- if (!ctx) {
- fdput(f);
+ if (!ctx)
return -EINVAL;
- }
amdgpu_ctx_priority_override(ctx, priority);
amdgpu_ctx_put(ctx);
- fdput(f);
-
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_umsch_mm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_umsch_mm.c
index d5125523bfa7..2b9bf1c1951c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_umsch_mm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_umsch_mm.c
@@ -765,9 +765,9 @@ static int umsch_mm_init(struct amdgpu_device *adev)
}
-static int umsch_mm_early_init(void *handle)
+static int umsch_mm_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
switch (amdgpu_ip_version(adev, VCN_HWIP, 0)) {
case IP_VERSION(4, 0, 5):
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vpe.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vpe.c
index bf4d2e3f2395..9312b6a9e3be 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vpe.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vpe.c
@@ -295,9 +295,9 @@ int amdgpu_vpe_ring_fini(struct amdgpu_vpe *vpe)
return 0;
}
-static int vpe_early_init(void *handle)
+static int vpe_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
struct amdgpu_vpe *vpe = &adev->vpe;
switch (amdgpu_ip_version(adev, VPE_HWIP, 0)) {
diff --git a/drivers/gpu/drm/amd/amdgpu/cik.c b/drivers/gpu/drm/amd/amdgpu/cik.c
index cf1d5d462b67..0a0114de11b4 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik.c
@@ -1985,9 +1985,9 @@ static const struct amdgpu_asic_funcs cik_asic_funcs =
.query_video_codecs = &cik_query_video_codecs,
};
-static int cik_common_early_init(void *handle)
+static int cik_common_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->smc_rreg = &cik_smc_rreg;
adev->smc_wreg = &cik_smc_wreg;
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
index 576baa9dbb0e..5ccd7e2ebf67 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
@@ -283,9 +283,9 @@ static void cik_ih_set_rptr(struct amdgpu_device *adev,
WREG32(mmIH_RB_RPTR, ih->rptr);
}
-static int cik_ih_early_init(void *handle)
+static int cik_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int ret;
ret = amdgpu_irq_add_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
index 952737de9411..3565dbcf7e38 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
@@ -918,9 +918,9 @@ static void cik_enable_sdma_mgls(struct amdgpu_device *adev,
}
}
-static int cik_sdma_early_init(void *handle)
+static int cik_sdma_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
adev->sdma.num_instances = SDMA_MAX_INSTANCE;
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
index 072643787384..bbc50a8e3bc4 100644
--- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
@@ -274,9 +274,9 @@ static void cz_ih_set_rptr(struct amdgpu_device *adev,
WREG32(mmIH_RB_RPTR, ih->rptr);
}
-static int cz_ih_early_init(void *handle)
+static int cz_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int ret;
ret = amdgpu_irq_add_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index baafbb5c032a..daed4a8439cc 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -2733,9 +2733,9 @@ static int dce_v10_0_crtc_init(struct amdgpu_device *adev, int index)
return 0;
}
-static int dce_v10_0_early_init(void *handle)
+static int dce_v10_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->audio_endpt_rreg = &dce_v10_0_audio_endpt_rreg;
adev->audio_endpt_wreg = &dce_v10_0_audio_endpt_wreg;
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
index a67b6b20b677..b00be90eb981 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
@@ -2846,9 +2846,9 @@ static int dce_v11_0_crtc_init(struct amdgpu_device *adev, int index)
return 0;
}
-static int dce_v11_0_early_init(void *handle)
+static int dce_v11_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->audio_endpt_rreg = &dce_v11_0_audio_endpt_rreg;
adev->audio_endpt_wreg = &dce_v11_0_audio_endpt_wreg;
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
index 1036b7a37390..1e15348a5c7d 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
@@ -2628,9 +2628,9 @@ static int dce_v6_0_crtc_init(struct amdgpu_device *adev, int index)
return 0;
}
-static int dce_v6_0_early_init(void *handle)
+static int dce_v6_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->audio_endpt_rreg = &dce_v6_0_audio_endpt_rreg;
adev->audio_endpt_wreg = &dce_v6_0_audio_endpt_wreg;
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index 0b30b3ed9d4b..7e92f322bc33 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -2639,9 +2639,9 @@ static int dce_v8_0_crtc_init(struct amdgpu_device *adev, int index)
return 0;
}
-static int dce_v8_0_early_init(void *handle)
+static int dce_v8_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->audio_endpt_rreg = &dce_v8_0_audio_endpt_rreg;
adev->audio_endpt_wreg = &dce_v8_0_audio_endpt_wreg;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
index 7babb74caf6f..a6e22c897b9d 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
@@ -6601,7 +6601,7 @@ static void gfx_v10_0_gfx_mqd_set_priority(struct amdgpu_device *adev,
/* set up default queue priority level
* 0x0 = low priority, 0x1 = high priority
*/
- if (prop->hqd_pipe_priority == AMDGPU_GFX_PIPE_PRIO_HIGH)
+ if (prop->hqd_queue_priority == AMDGPU_GFX_QUEUE_PRIORITY_MAXIMUM)
priority = 1;
tmp = RREG32_SOC15(GC, 0, mmCP_GFX_HQD_QUEUE_PRIORITY);
@@ -7678,9 +7678,9 @@ static void gfx_v10_0_ring_emit_gds_switch(struct amdgpu_ring *ring,
(1 << (oa_size + oa_base)) - (1 << oa_base));
}
-static int gfx_v10_0_early_init(void *handle)
+static int gfx_v10_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.funcs = &gfx_v10_0_gfx_funcs;
@@ -9630,9 +9630,9 @@ static void gfx_v10_ip_print(void *handle, struct drm_printer *p)
}
}
-static void gfx_v10_ip_dump(void *handle)
+static void gfx_v10_ip_dump(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
uint32_t i, j, k, reg, index = 0;
uint32_t reg_count = ARRAY_SIZE(gc_reg_list_10_1);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
index 3c91c30edf2b..6b5b5fcdb988 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
@@ -3919,7 +3919,7 @@ static void gfx_v11_0_gfx_mqd_set_priority(struct amdgpu_device *adev,
/* set up default queue priority level
* 0x0 = low priority, 0x1 = high priority
*/
- if (prop->hqd_pipe_priority == AMDGPU_GFX_PIPE_PRIO_HIGH)
+ if (prop->hqd_queue_priority == AMDGPU_GFX_QUEUE_PRIORITY_MAXIMUM)
priority = 1;
tmp = regCP_GFX_HQD_QUEUE_PRIORITY_DEFAULT;
@@ -5037,9 +5037,9 @@ static void gfx_v11_0_ring_emit_gds_switch(struct amdgpu_ring *ring,
(1 << (oa_size + oa_base)) - (1 << oa_base));
}
-static int gfx_v11_0_early_init(void *handle)
+static int gfx_v11_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.funcs = &gfx_v11_0_gfx_funcs;
@@ -6746,9 +6746,9 @@ static void gfx_v11_ip_print(void *handle, struct drm_printer *p)
}
}
-static void gfx_v11_ip_dump(void *handle)
+static void gfx_v11_ip_dump(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
uint32_t i, j, k, reg, index = 0;
uint32_t reg_count = ARRAY_SIZE(gc_reg_list_11_0);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index 0f4896a5f82c..97116a92cd8b 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -3712,9 +3712,9 @@ static uint64_t gfx_v12_0_get_gpu_clock_counter(struct amdgpu_device *adev)
return clock;
}
-static int gfx_v12_0_early_init(void *handle)
+static int gfx_v12_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.funcs = &gfx_v12_0_gfx_funcs;
@@ -5096,9 +5096,9 @@ static void gfx_v12_ip_print(void *handle, struct drm_printer *p)
}
}
-static void gfx_v12_ip_dump(void *handle)
+static void gfx_v12_ip_dump(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
uint32_t i, j, k, reg, index = 0;
uint32_t reg_count = ARRAY_SIZE(gc_reg_list_12_0);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
index 564f0b9336b6..90c426ee877b 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
@@ -1553,6 +1553,71 @@ static void gfx_v6_0_setup_spi(struct amdgpu_device *adev)
mutex_unlock(&adev->grbm_idx_mutex);
}
+/**
+ * gfx_v6_0_setup_tcc() - setup which TCCs are used
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Verify whether the current GPU has any TCCs disabled,
+ * which can happen when the GPU is harvested and some
+ * memory channels are disabled, reducing the memory bus width.
+ * For example, on the Radeon HD 7870 XT (Tahiti LE).
+ *
+ * If some TCCs are disabled, we need to make sure that
+ * the disabled TCCs are not used, and the remaining TCCs
+ * are used optimally.
+ *
+ * TCP_CHAN_STEER_LO/HI control which TCC is used by TCP channels.
+ * TCP_ADDR_CONFIG.NUM_TCC_BANKS controls how many channels are used.
+ *
+ * For optimal performance:
+ * - Rely on the CHAN_STEER from the golden registers table,
+ * only skip disabled TCCs but keep the mapping order.
+ * - Limit NUM_TCC_BANKS to number of active TCCs to avoid thrashing,
+ * which performs better than using the same TCC twice.
+ */
+static void gfx_v6_0_setup_tcc(struct amdgpu_device *adev)
+{
+ u32 i, tcc, tcp_addr_config, num_active_tcc = 0;
+ u64 chan_steer, patched_chan_steer = 0;
+ const u32 num_max_tcc = adev->gfx.config.max_texture_channel_caches;
+ const u32 dis_tcc_mask =
+ amdgpu_gfx_create_bitmask(num_max_tcc) &
+ (REG_GET_FIELD(RREG32(mmCGTS_TCC_DISABLE),
+ CGTS_TCC_DISABLE, TCC_DISABLE) |
+ REG_GET_FIELD(RREG32(mmCGTS_USER_TCC_DISABLE),
+ CGTS_USER_TCC_DISABLE, TCC_DISABLE));
+
+ /* When no TCC is disabled, the golden registers table already has optimal TCC setup */
+ if (!dis_tcc_mask)
+ return;
+
+ /* Each 4-bit nibble contains the index of a TCC used by all TCPs */
+ chan_steer = RREG32(mmTCP_CHAN_STEER_LO) | ((u64)RREG32(mmTCP_CHAN_STEER_HI) << 32ull);
+
+ /* Patch the TCP to TCC mapping to skip disabled TCCs */
+ for (i = 0; i < num_max_tcc; ++i) {
+ tcc = (chan_steer >> (u64)(4 * i)) & 0xf;
+
+ if (!((1 << tcc) & dis_tcc_mask)) {
+ /* Copy enabled TCC indices to the patched register value. */
+ patched_chan_steer |= (u64)tcc << (u64)(4 * num_active_tcc);
+ ++num_active_tcc;
+ }
+ }
+
+ WARN_ON(num_active_tcc != num_max_tcc - hweight32(dis_tcc_mask));
+
+ /* Patch number of TCCs used by TCPs */
+ tcp_addr_config = REG_SET_FIELD(RREG32(mmTCP_ADDR_CONFIG),
+ TCP_ADDR_CONFIG, NUM_TCC_BANKS,
+ num_active_tcc - 1);
+
+ WREG32(mmTCP_ADDR_CONFIG, tcp_addr_config);
+ WREG32(mmTCP_CHAN_STEER_HI, upper_32_bits(patched_chan_steer));
+ WREG32(mmTCP_CHAN_STEER_LO, lower_32_bits(patched_chan_steer));
+}
+
static void gfx_v6_0_config_init(struct amdgpu_device *adev)
{
adev->gfx.config.double_offchip_lds_buf = 0;
@@ -1711,6 +1776,7 @@ static void gfx_v6_0_constants_init(struct amdgpu_device *adev)
gfx_v6_0_tiling_mode_table_init(adev);
gfx_v6_0_setup_rb(adev);
+ gfx_v6_0_setup_tcc(adev);
gfx_v6_0_setup_spi(adev);
@@ -3023,9 +3089,9 @@ static const struct amdgpu_rlc_funcs gfx_v6_0_rlc_funcs = {
.start = gfx_v6_0_rlc_start
};
-static int gfx_v6_0_early_init(void *handle)
+static int gfx_v6_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.xcc_mask = 1;
adev->gfx.num_gfx_rings = GFX6_NUM_GFX_RINGS;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
index f146806c4633..3babf5b5a9dd 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
@@ -4134,9 +4134,9 @@ static const struct amdgpu_rlc_funcs gfx_v7_0_rlc_funcs = {
.update_spm_vmid = gfx_v7_0_update_spm_vmid
};
-static int gfx_v7_0_early_init(void *handle)
+static int gfx_v7_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.xcc_mask = 1;
adev->gfx.num_gfx_rings = GFX7_NUM_GFX_RINGS;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
index 1f675d67a1a7..2e54fb63dd5b 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
@@ -5262,9 +5262,9 @@ static const struct amdgpu_gfx_funcs gfx_v8_0_gfx_funcs = {
.select_me_pipe_q = &gfx_v8_0_select_me_pipe_q
};
-static int gfx_v8_0_early_init(void *handle)
+static int gfx_v8_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.xcc_mask = 1;
adev->gfx.num_gfx_rings = GFX8_NUM_GFX_RINGS;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
index a081fe118c26..d16ac8669e07 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
@@ -4760,9 +4760,9 @@ static int gfx_v9_0_do_edc_gpr_workarounds(struct amdgpu_device *adev)
return r;
}
-static int gfx_v9_0_early_init(void *handle)
+static int gfx_v9_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.funcs = &gfx_v9_0_gfx_funcs;
@@ -7364,9 +7364,9 @@ static void gfx_v9_ip_print(void *handle, struct drm_printer *p)
}
-static void gfx_v9_ip_dump(void *handle)
+static void gfx_v9_ip_dump(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
uint32_t i, j, k, reg, index = 0;
uint32_t reg_count = ARRAY_SIZE(gc_reg_list_9);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
index 26c2d8d9e246..15d482990297 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c
@@ -2517,9 +2517,9 @@ static void gfx_v9_4_3_ring_emit_gds_switch(struct amdgpu_ring *ring,
(1 << (oa_size + oa_base)) - (1 << oa_base));
}
-static int gfx_v9_4_3_early_init(void *handle)
+static int gfx_v9_4_3_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->gfx.num_compute_rings = min(amdgpu_gfx_get_num_kcq(adev),
AMDGPU_MAX_COMPUTE_RINGS);
@@ -4644,9 +4644,9 @@ static void gfx_v9_4_3_ip_print(void *handle, struct drm_printer *p)
}
}
-static void gfx_v9_4_3_ip_dump(void *handle)
+static void gfx_v9_4_3_ip_dump(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
uint32_t i, j, k;
uint32_t num_xcc, reg, num_inst;
uint32_t xcc_id, xcc_offset, inst_offset;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
index c6e742921282..86a7261df8b7 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
@@ -630,9 +630,9 @@ static void gmc_v10_0_set_gfxhub_funcs(struct amdgpu_device *adev)
}
-static int gmc_v10_0_early_init(void *handle)
+static int gmc_v10_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
gmc_v10_0_set_mmhub_funcs(adev);
gmc_v10_0_set_gfxhub_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
index abbf49c90e57..789b4f531f31 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
@@ -628,9 +628,9 @@ static void gmc_v11_0_set_gfxhub_funcs(struct amdgpu_device *adev)
}
}
-static int gmc_v11_0_early_init(void *handle)
+static int gmc_v11_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
gmc_v11_0_set_gfxhub_funcs(adev);
gmc_v11_0_set_mmhub_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
index 729f343c17a7..aaa6307254ff 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
@@ -614,9 +614,9 @@ static void gmc_v12_0_set_gfxhub_funcs(struct amdgpu_device *adev)
}
}
-static int gmc_v12_0_early_init(void *handle)
+static int gmc_v12_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
gmc_v12_0_set_gfxhub_funcs(adev);
gmc_v12_0_set_mmhub_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
index d36725666b54..3a524319f31e 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
@@ -762,9 +762,9 @@ static int gmc_v6_0_convert_vram_type(int mc_seq_vram_type)
}
}
-static int gmc_v6_0_early_init(void *handle)
+static int gmc_v6_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
gmc_v6_0_set_gmc_funcs(adev);
gmc_v6_0_set_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index 8e2f73125650..ece404a738e0 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -921,9 +921,9 @@ static int gmc_v7_0_convert_vram_type(int mc_seq_vram_type)
}
}
-static int gmc_v7_0_early_init(void *handle)
+static int gmc_v7_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
gmc_v7_0_set_gmc_funcs(adev);
gmc_v7_0_set_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 5248832c04ad..52f5843f8a37 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -1027,9 +1027,9 @@ static int gmc_v8_0_convert_vram_type(int mc_seq_vram_type)
}
}
-static int gmc_v8_0_early_init(void *handle)
+static int gmc_v8_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
gmc_v8_0_set_gmc_funcs(adev);
gmc_v8_0_set_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index 91c6464efed2..d82f90f48024 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -1554,9 +1554,9 @@ static void gmc_v9_0_set_xgmi_ras_funcs(struct amdgpu_device *adev)
adev->gmc.xgmi.ras = &xgmi_ras;
}
-static int gmc_v9_0_early_init(void *handle)
+static int gmc_v9_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
/*
* 9.4.0, 9.4.1 and 9.4.3 don't have XGMI defined
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
index 07984f7c3ae7..87b29600cf1f 100644
--- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
@@ -273,9 +273,9 @@ static void iceland_ih_set_rptr(struct amdgpu_device *adev,
WREG32(mmIH_RB_RPTR, ih->rptr);
}
-static int iceland_ih_early_init(void *handle)
+static int iceland_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int ret;
ret = amdgpu_irq_add_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index 18a761d6ef33..fa6c7e5fbbe3 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -559,9 +559,9 @@ static void ih_v6_0_set_self_irq_funcs(struct amdgpu_device *adev)
adev->irq.self_irq.funcs = &ih_v6_0_self_irq_funcs;
}
-static int ih_v6_0_early_init(void *handle)
+static int ih_v6_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
ih_v6_0_set_interrupt_funcs(adev);
ih_v6_0_set_self_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
index 2e0469feca1e..ebe23630e8f6 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
@@ -532,9 +532,9 @@ static void ih_v6_1_set_self_irq_funcs(struct amdgpu_device *adev)
adev->irq.self_irq.funcs = &ih_v6_1_self_irq_funcs;
}
-static int ih_v6_1_early_init(void *handle)
+static int ih_v6_1_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int ret;
ret = amdgpu_irq_add_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
index 6852081fcff2..1619f0ba4d1b 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
@@ -528,9 +528,9 @@ static void ih_v7_0_set_self_irq_funcs(struct amdgpu_device *adev)
adev->irq.self_irq.funcs = &ih_v7_0_self_irq_funcs;
}
-static int ih_v7_0_early_init(void *handle)
+static int ih_v7_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
ih_v7_0_set_interrupt_funcs(adev);
ih_v7_0_set_self_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c
index 6e0e88076224..8effd6dc65d4 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c
@@ -462,9 +462,9 @@ static int jpeg_v1_0_process_interrupt(struct amdgpu_device *adev,
*
* Set ring and irq function pointers
*/
-int jpeg_v1_0_early_init(void *handle)
+int jpeg_v1_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->jpeg.num_jpeg_inst = 1;
adev->jpeg.num_jpeg_rings = 1;
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.h b/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.h
index 9654d22e0376..791de235cd8b 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.h
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.h
@@ -24,7 +24,7 @@
#ifndef __JPEG_V1_0_H__
#define __JPEG_V1_0_H__
-int jpeg_v1_0_early_init(void *handle);
+int jpeg_v1_0_early_init(struct amdgpu_ip_block *ip_block);
int jpeg_v1_0_sw_init(void *handle);
void jpeg_v1_0_sw_fini(void *handle);
void jpeg_v1_0_start(struct amdgpu_device *adev, int mode);
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
index 41c0f8750dc1..5c95cf6167d9 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
@@ -45,9 +45,9 @@ static int jpeg_v2_0_set_powergating_state(void *handle,
*
* Set ring and irq function pointers
*/
-static int jpeg_v2_0_early_init(void *handle)
+static int jpeg_v2_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->jpeg.num_jpeg_inst = 1;
adev->jpeg.num_jpeg_rings = 1;
@@ -766,6 +766,7 @@ static const struct amd_ip_funcs jpeg_v2_0_ip_funcs = {
static const struct amdgpu_ring_funcs jpeg_v2_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v2_0_dec_ring_get_rptr,
.get_wptr = jpeg_v2_0_dec_ring_get_wptr,
.set_wptr = jpeg_v2_0_dec_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
index eedb9a829d95..e4cfa6b7141f 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
@@ -54,9 +54,9 @@ static int amdgpu_ih_clientid_jpeg[] = {
*
* Set ring and irq function pointers
*/
-static int jpeg_v2_5_early_init(void *handle)
+static int jpeg_v2_5_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
u32 harvest;
int i;
@@ -659,6 +659,7 @@ static const struct amd_ip_funcs jpeg_v2_6_ip_funcs = {
static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v2_5_dec_ring_get_rptr,
.get_wptr = jpeg_v2_5_dec_ring_get_wptr,
.set_wptr = jpeg_v2_5_dec_ring_set_wptr,
@@ -689,6 +690,7 @@ static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = {
static const struct amdgpu_ring_funcs jpeg_v2_6_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v2_5_dec_ring_get_rptr,
.get_wptr = jpeg_v2_5_dec_ring_get_wptr,
.set_wptr = jpeg_v2_5_dec_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c
index b1e7fd25afbc..27a27de6c6c3 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c
@@ -46,9 +46,9 @@ static int jpeg_v3_0_set_powergating_state(void *handle,
*
* Set ring and irq function pointers
*/
-static int jpeg_v3_0_early_init(void *handle)
+static int jpeg_v3_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
u32 harvest;
@@ -557,6 +557,7 @@ static const struct amd_ip_funcs jpeg_v3_0_ip_funcs = {
static const struct amdgpu_ring_funcs jpeg_v3_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v3_0_dec_ring_get_rptr,
.get_wptr = jpeg_v3_0_dec_ring_get_wptr,
.set_wptr = jpeg_v3_0_dec_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c
index 6c5c1a68a9b7..90f64a46bff7 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.c
@@ -52,9 +52,9 @@ static void jpeg_v4_0_dec_ring_set_wptr(struct amdgpu_ring *ring);
*
* Set ring and irq function pointers
*/
-static int jpeg_v4_0_early_init(void *handle)
+static int jpeg_v4_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->jpeg.num_jpeg_inst = 1;
@@ -724,6 +724,7 @@ static const struct amd_ip_funcs jpeg_v4_0_ip_funcs = {
static const struct amdgpu_ring_funcs jpeg_v4_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v4_0_dec_ring_get_rptr,
.get_wptr = jpeg_v4_0_dec_ring_get_wptr,
.set_wptr = jpeg_v4_0_dec_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c
index ae9b95dd8602..c4f812939a85 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c
@@ -72,9 +72,9 @@ static inline bool jpeg_v4_0_3_normalizn_reqd(struct amdgpu_device *adev)
*
* Set ring and irq function pointers
*/
-static int jpeg_v4_0_3_early_init(void *handle)
+static int jpeg_v4_0_3_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->jpeg.num_jpeg_rings = AMDGPU_MAX_JPEG_RINGS;
@@ -1122,6 +1122,7 @@ static const struct amd_ip_funcs jpeg_v4_0_3_ip_funcs = {
static const struct amdgpu_ring_funcs jpeg_v4_0_3_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v4_0_3_dec_ring_get_rptr,
.get_wptr = jpeg_v4_0_3_dec_ring_get_wptr,
.set_wptr = jpeg_v4_0_3_dec_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c
index 44eeed445ea9..78a9fb26bce2 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c
@@ -65,9 +65,9 @@ static int amdgpu_ih_clientid_jpeg[] = {
*
* Set ring and irq function pointers
*/
-static int jpeg_v4_0_5_early_init(void *handle)
+static int jpeg_v4_0_5_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
switch (amdgpu_ip_version(adev, UVD_HWIP, 0)) {
case IP_VERSION(4, 0, 5):
@@ -765,6 +765,7 @@ static const struct amd_ip_funcs jpeg_v4_0_5_ip_funcs = {
static const struct amdgpu_ring_funcs jpeg_v4_0_5_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v4_0_5_dec_ring_get_rptr,
.get_wptr = jpeg_v4_0_5_dec_ring_get_wptr,
.set_wptr = jpeg_v4_0_5_dec_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c
index d662aa841f97..d0f800e7938d 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c
@@ -46,9 +46,9 @@ static int jpeg_v5_0_0_set_powergating_state(void *handle,
*
* Set ring and irq function pointers
*/
-static int jpeg_v5_0_0_early_init(void *handle)
+static int jpeg_v5_0_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->jpeg.num_jpeg_inst = 1;
adev->jpeg.num_jpeg_rings = 1;
@@ -644,6 +644,7 @@ static const struct amd_ip_funcs jpeg_v5_0_0_ip_funcs = {
static const struct amdgpu_ring_funcs jpeg_v5_0_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
+ .no_user_fence = true,
.get_rptr = jpeg_v5_0_0_dec_ring_get_rptr,
.get_wptr = jpeg_v5_0_0_dec_ring_get_wptr,
.set_wptr = jpeg_v5_0_0_dec_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index ccd9055360fc..236fedc5b183 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -1633,9 +1633,9 @@ static int mes_v11_0_resume(void *handle)
return amdgpu_mes_resume(adev);
}
-static int mes_v11_0_early_init(void *handle)
+static int mes_v11_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int pipe, r;
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
index 945016712157..c432a2a3405d 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
@@ -1628,9 +1628,9 @@ static int mes_v12_0_resume(void *handle)
return amdgpu_mes_resume(adev);
}
-static int mes_v12_0_early_init(void *handle)
+static int mes_v12_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int pipe, r;
for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) {
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index b281462093f1..17aab897f86b 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -542,9 +542,9 @@ static void navi10_ih_set_self_irq_funcs(struct amdgpu_device *adev)
adev->irq.self_irq.funcs = &navi10_ih_self_irq_funcs;
}
-static int navi10_ih_early_init(void *handle)
+static int navi10_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
navi10_ih_set_interrupt_funcs(adev);
navi10_ih_set_self_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/nv.c b/drivers/gpu/drm/amd/amdgpu/nv.c
index ab0eecbab412..ba8341c62255 100644
--- a/drivers/gpu/drm/amd/amdgpu/nv.c
+++ b/drivers/gpu/drm/amd/amdgpu/nv.c
@@ -634,9 +634,9 @@ static const struct amdgpu_asic_funcs nv_asic_funcs = {
.query_video_codecs = &nv_query_video_codecs,
};
-static int nv_common_early_init(void *handle)
+static int nv_common_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->nbio.funcs->set_reg_remap(adev);
adev->smc_rreg = NULL;
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
index 725392522267..5b8198558869 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
@@ -807,9 +807,9 @@ static void sdma_v2_4_ring_emit_wreg(struct amdgpu_ring *ring,
amdgpu_ring_write(ring, val);
}
-static int sdma_v2_4_early_init(void *handle)
+static int sdma_v2_4_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
adev->sdma.num_instances = SDMA_MAX_INSTANCE;
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
index e65194fe94af..37275b38bca8 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
@@ -1080,9 +1080,9 @@ static void sdma_v3_0_ring_emit_wreg(struct amdgpu_ring *ring,
amdgpu_ring_write(ring, val);
}
-static int sdma_v3_0_early_init(void *handle)
+static int sdma_v3_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
switch (adev->asic_type) {
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index 37bb0857d8f8..dafd2ecac8e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -1751,9 +1751,9 @@ static bool sdma_v4_0_fw_support_paging_queue(struct amdgpu_device *adev)
}
}
-static int sdma_v4_0_early_init(void *handle)
+static int sdma_v4_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
r = sdma_v4_0_init_microcode(adev);
@@ -2371,9 +2371,9 @@ static void sdma_v4_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void sdma_v4_0_dump_ip_state(void *handle)
+static void sdma_v4_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
uint32_t instance_offset;
uint32_t reg_count = ARRAY_SIZE(sdma_reg_list_4_0);
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
index 1e4ce06f5f2c..8c97a67f4c9f 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
@@ -1296,9 +1296,9 @@ static bool sdma_v4_4_2_fw_support_paging_queue(struct amdgpu_device *adev)
}
}
-static int sdma_v4_4_2_early_init(void *handle)
+static int sdma_v4_4_2_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
r = sdma_v4_4_2_init_microcode(adev);
@@ -1884,9 +1884,9 @@ static void sdma_v4_4_2_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void sdma_v4_4_2_dump_ip_state(void *handle)
+static void sdma_v4_4_2_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
uint32_t instance_offset;
uint32_t reg_count = ARRAY_SIZE(sdma_reg_list_4_4_2);
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
index 3e48ea38385d..34fcbcdd93b6 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
@@ -1366,9 +1366,9 @@ static void sdma_v5_0_ring_emit_reg_write_reg_wait(struct amdgpu_ring *ring,
amdgpu_ring_emit_reg_wait(ring, reg1, mask, mask);
}
-static int sdma_v5_0_early_init(void *handle)
+static int sdma_v5_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
r = sdma_v5_0_init_microcode(adev);
@@ -1799,9 +1799,9 @@ static void sdma_v5_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void sdma_v5_0_dump_ip_state(void *handle)
+static void sdma_v5_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
uint32_t instance_offset;
uint32_t reg_count = ARRAY_SIZE(sdma_reg_list_5_0);
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
index bc9b240a3488..241995252ff0 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
@@ -1216,9 +1216,9 @@ static void sdma_v5_2_ring_emit_reg_write_reg_wait(struct amdgpu_ring *ring,
amdgpu_ring_emit_reg_wait(ring, reg1, mask, mask);
}
-static int sdma_v5_2_early_init(void *handle)
+static int sdma_v5_2_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
r = amdgpu_sdma_init_microcode(adev, 0, true);
@@ -1757,9 +1757,9 @@ static void sdma_v5_2_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void sdma_v5_2_dump_ip_state(void *handle)
+static void sdma_v5_2_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
uint32_t instance_offset;
uint32_t reg_count = ARRAY_SIZE(sdma_reg_list_5_2);
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c
index 208a1fa9d4e7..74e79ddd714a 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c
@@ -1272,9 +1272,9 @@ static void sdma_v6_0_set_ras_funcs(struct amdgpu_device *adev)
}
}
-static int sdma_v6_0_early_init(void *handle)
+static int sdma_v6_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
r = amdgpu_sdma_init_microcode(adev, 0, true);
@@ -1577,9 +1577,9 @@ static void sdma_v6_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void sdma_v6_0_dump_ip_state(void *handle)
+static void sdma_v6_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
uint32_t instance_offset;
uint32_t reg_count = ARRAY_SIZE(sdma_reg_list_6_0);
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c
index 843e6b46deee..1d0131c172d6 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c
@@ -1259,9 +1259,9 @@ static void sdma_v7_0_ring_emit_reg_write_reg_wait(struct amdgpu_ring *ring,
amdgpu_ring_emit_reg_wait(ring, reg1, mask, mask);
}
-static int sdma_v7_0_early_init(void *handle)
+static int sdma_v7_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int r;
r = amdgpu_sdma_init_microcode(adev, 0, true);
@@ -1565,9 +1565,9 @@ static void sdma_v7_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void sdma_v7_0_dump_ip_state(void *handle)
+static void sdma_v7_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
uint32_t instance_offset;
uint32_t reg_count = ARRAY_SIZE(sdma_reg_list_7_0);
diff --git a/drivers/gpu/drm/amd/amdgpu/si.c b/drivers/gpu/drm/amd/amdgpu/si.c
index 85235470e872..93c68abf447a 100644
--- a/drivers/gpu/drm/amd/amdgpu/si.c
+++ b/drivers/gpu/drm/amd/amdgpu/si.c
@@ -2022,9 +2022,9 @@ static uint32_t si_get_rev_id(struct amdgpu_device *adev)
>> CC_DRM_ID_STRAPS__ATI_REV_ID__SHIFT;
}
-static int si_common_early_init(void *handle)
+static int si_common_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->smc_rreg = &si_smc_rreg;
adev->smc_wreg = &si_smc_wreg;
diff --git a/drivers/gpu/drm/amd/amdgpu/si_dma.c b/drivers/gpu/drm/amd/amdgpu/si_dma.c
index 11db5b755832..791d492e991d 100644
--- a/drivers/gpu/drm/amd/amdgpu/si_dma.c
+++ b/drivers/gpu/drm/amd/amdgpu/si_dma.c
@@ -457,9 +457,9 @@ static void si_dma_ring_emit_wreg(struct amdgpu_ring *ring,
amdgpu_ring_write(ring, val);
}
-static int si_dma_early_init(void *handle)
+static int si_dma_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->sdma.num_instances = 2;
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
index 5237395e4fab..bd2ae8255498 100644
--- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
@@ -156,9 +156,9 @@ static void si_ih_set_rptr(struct amdgpu_device *adev,
WREG32(IH_RB_RPTR, ih->rptr);
}
-static int si_ih_early_init(void *handle)
+static int si_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
si_ih_set_interrupt_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c
index c162149b5494..5b65b760cc91 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.c
@@ -928,9 +928,9 @@ static const struct amdgpu_asic_funcs aqua_vanjaram_asic_funcs =
.get_reg_state = &aqua_vanjaram_get_reg_state,
};
-static int soc15_common_early_init(void *handle)
+static int soc15_common_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->nbio.funcs->set_reg_remap(adev);
adev->smc_rreg = NULL;
diff --git a/drivers/gpu/drm/amd/amdgpu/soc21.c b/drivers/gpu/drm/amd/amdgpu/soc21.c
index 7d570325167e..1287dd875e3d 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc21.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc21.c
@@ -556,9 +556,9 @@ static const struct amdgpu_asic_funcs soc21_asic_funcs = {
.update_umd_stable_pstate = &soc21_update_umd_stable_pstate,
};
-static int soc21_common_early_init(void *handle)
+static int soc21_common_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->nbio.funcs->set_reg_remap(adev);
adev->smc_rreg = NULL;
diff --git a/drivers/gpu/drm/amd/amdgpu/soc24.c b/drivers/gpu/drm/amd/amdgpu/soc24.c
index 29a848f2466b..53d96edc2877 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc24.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc24.c
@@ -363,9 +363,9 @@ static const struct amdgpu_asic_funcs soc24_asic_funcs = {
.update_umd_stable_pstate = &soc24_update_umd_stable_pstate,
};
-static int soc24_common_early_init(void *handle)
+static int soc24_common_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->nbio.funcs->set_reg_remap(adev);
adev->smc_rreg = NULL;
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
index 24d49d813607..ae27dac94117 100644
--- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
@@ -283,9 +283,9 @@ static void tonga_ih_set_rptr(struct amdgpu_device *adev,
}
}
-static int tonga_ih_early_init(void *handle)
+static int tonga_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int ret;
ret = amdgpu_irq_add_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c b/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c
index 21376d98ee49..0344f74cc454 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c
@@ -98,7 +98,7 @@ static void uvd_v3_1_ring_emit_ib(struct amdgpu_ring *ring,
}
/**
- * uvd_v3_1_ring_emit_fence - emit an fence & trap command
+ * uvd_v3_1_ring_emit_fence - emit a fence & trap command
*
* @ring: amdgpu_ring pointer
* @addr: address
@@ -242,7 +242,11 @@ static void uvd_v3_1_mc_resume(struct amdgpu_device *adev)
uint64_t addr;
uint32_t size;
- /* programm the VCPU memory controller bits 0-27 */
+ /* When the keyselect is already set, don't perturb it. */
+ if (RREG32(mmUVD_FW_START))
+ return;
+
+ /* program the VCPU memory controller bits 0-27 */
addr = (adev->uvd.inst->gpu_addr + AMDGPU_UVD_FIRMWARE_OFFSET) >> 3;
size = AMDGPU_UVD_FIRMWARE_SIZE(adev) >> 3;
WREG32(mmUVD_VCPU_CACHE_OFFSET0, addr);
@@ -284,6 +288,12 @@ static int uvd_v3_1_fw_validate(struct amdgpu_device *adev)
int i;
uint32_t keysel = adev->uvd.keyselect;
+ if (RREG32(mmUVD_FW_START) & UVD_FW_STATUS__PASS_MASK) {
+ dev_dbg(adev->dev, "UVD keyselect already set: 0x%x (on CPU: 0x%x)\n",
+ RREG32(mmUVD_FW_START), adev->uvd.keyselect);
+ return 0;
+ }
+
WREG32(mmUVD_FW_START, keysel);
for (i = 0; i < 10; ++i) {
@@ -416,7 +426,7 @@ static int uvd_v3_1_start(struct amdgpu_device *adev)
/* Set the write pointer delay */
WREG32(mmUVD_RBC_RB_WPTR_CNTL, 0);
- /* programm the 4GB memory segment for rptr and ring buffer */
+ /* Program the 4GB memory segment for rptr and ring buffer */
WREG32(mmUVD_LMI_EXT40_ADDR, upper_32_bits(ring->gpu_addr) |
(0x7 << 16) | (0x1 << 31));
@@ -531,9 +541,9 @@ static void uvd_v3_1_set_irq_funcs(struct amdgpu_device *adev)
}
-static int uvd_v3_1_early_init(void *handle)
+static int uvd_v3_1_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->uvd.num_uvd_inst = 1;
uvd_v3_1_set_ring_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
index 3f19c606f4de..a1227867dc88 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
@@ -90,9 +90,14 @@ static void uvd_v4_2_ring_set_wptr(struct amdgpu_ring *ring)
WREG32(mmUVD_RBC_RB_WPTR, lower_32_bits(ring->wptr));
}
-static int uvd_v4_2_early_init(void *handle)
+static int uvd_v4_2_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
+
+ /* UVD doesn't work without DPM, it needs DPM to ungate it. */
+ if (!amdgpu_dpm)
+ return -ENOENT;
+
adev->uvd.num_uvd_inst = 1;
uvd_v4_2_set_ring_funcs(adev);
@@ -303,7 +308,7 @@ static int uvd_v4_2_start(struct amdgpu_device *adev)
/* enable VCPU clock */
WREG32(mmUVD_VCPU_CNTL, 1 << 9);
- /* disable interupt */
+ /* disable interrupt */
WREG32_P(mmUVD_MASTINT_EN, 0, ~(1 << 1));
#ifdef __BIG_ENDIAN
@@ -313,6 +318,7 @@ static int uvd_v4_2_start(struct amdgpu_device *adev)
#endif
WREG32(mmUVD_LMI_SWAP_CNTL, lmi_swap_cntl);
WREG32(mmUVD_MP_SWAP_CNTL, mp_swap_cntl);
+
/* initialize UVD memory controller */
WREG32(mmUVD_LMI_CTRL, 0x203108);
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
index efd903c21d48..fd4acb1300f9 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
@@ -88,9 +88,9 @@ static void uvd_v5_0_ring_set_wptr(struct amdgpu_ring *ring)
WREG32(mmUVD_RBC_RB_WPTR, lower_32_bits(ring->wptr));
}
-static int uvd_v5_0_early_init(void *handle)
+static int uvd_v5_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->uvd.num_uvd_inst = 1;
uvd_v5_0_set_ring_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
index 495de5068455..e05e81d6fbd4 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
@@ -354,9 +354,9 @@ static int uvd_v6_0_enc_ring_test_ib(struct amdgpu_ring *ring, long timeout)
return r;
}
-static int uvd_v6_0_early_init(void *handle)
+static int uvd_v6_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->uvd.num_uvd_inst = 1;
if (!(adev->flags & AMD_IS_APU) &&
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
index 9a30b8c10838..e3c20cbc7a00 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
@@ -361,9 +361,9 @@ static int uvd_v7_0_enc_ring_test_ib(struct amdgpu_ring *ring, long timeout)
return r;
}
-static int uvd_v7_0_early_init(void *handle)
+static int uvd_v7_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
if (adev->asic_type == CHIP_VEGA20) {
u32 harvest;
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
index 66fada199bda..8a6c24b98f1f 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
@@ -278,7 +278,7 @@ static int vce_v2_0_stop(struct amdgpu_device *adev)
int status;
if (vce_v2_0_lmi_clean(adev)) {
- DRM_INFO("vce is not idle \n");
+ DRM_INFO("VCE is not idle \n");
return 0;
}
@@ -398,9 +398,9 @@ static void vce_v2_0_enable_mgcg(struct amdgpu_device *adev, bool enable,
}
}
-static int vce_v2_0_early_init(void *handle)
+static int vce_v2_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->vce.num_rings = 2;
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
index 4bfba2931b08..31ca855a950a 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
@@ -396,9 +396,9 @@ static unsigned vce_v3_0_get_harvest_config(struct amdgpu_device *adev)
}
}
-static int vce_v3_0_early_init(void *handle)
+static int vce_v3_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->vce.harvest_config = vce_v3_0_get_harvest_config(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
index 0748bf44c880..14ead62ec57d 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
@@ -407,9 +407,9 @@ static int vce_v4_0_stop(struct amdgpu_device *adev)
return 0;
}
-static int vce_v4_0_early_init(void *handle)
+static int vce_v4_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
if (amdgpu_sriov_vf(adev)) /* currently only VCN0 support SRIOV */
adev->vce.num_rings = 1;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
index ecdfbfefd66a..563721c55163 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
@@ -100,9 +100,9 @@ static void vcn_v1_0_ring_begin_use(struct amdgpu_ring *ring);
* Set ring and irq function pointers
* Load microcode from filesystem
*/
-static int vcn_v1_0_early_init(void *handle)
+static int vcn_v1_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->vcn.num_enc_rings = 2;
@@ -110,7 +110,7 @@ static int vcn_v1_0_early_init(void *handle)
vcn_v1_0_set_enc_ring_funcs(adev);
vcn_v1_0_set_irq_funcs(adev);
- jpeg_v1_0_early_init(handle);
+ jpeg_v1_0_early_init(ip_block);
return amdgpu_vcn_early_init(adev);
}
@@ -1957,9 +1957,9 @@ static void vcn_v1_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v1_0_dump_ip_state(void *handle)
+static void vcn_v1_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c
index 9479bf9ea30f..29f5f6b1eb0a 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c
@@ -106,9 +106,9 @@ static int vcn_v2_0_start_sriov(struct amdgpu_device *adev);
* Set ring and irq function pointers
* Load microcode from filesystem
*/
-static int vcn_v2_0_early_init(void *handle)
+static int vcn_v2_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
if (amdgpu_sriov_vf(adev))
adev->vcn.num_enc_rings = 1;
@@ -2074,9 +2074,9 @@ static void vcn_v2_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v2_0_dump_ip_state(void *handle)
+static void vcn_v2_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off;
@@ -2127,6 +2127,7 @@ static const struct amd_ip_funcs vcn_v2_0_ip_funcs = {
static const struct amdgpu_ring_funcs vcn_v2_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_DEC,
.align_mask = 0xf,
+ .no_user_fence = true,
.secure_submission_supported = true,
.get_rptr = vcn_v2_0_dec_ring_get_rptr,
.get_wptr = vcn_v2_0_dec_ring_get_wptr,
@@ -2158,6 +2159,7 @@ static const struct amdgpu_ring_funcs vcn_v2_0_enc_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_ENC,
.align_mask = 0x3f,
.nop = VCN_ENC_CMD_NO_OP,
+ .no_user_fence = true,
.get_rptr = vcn_v2_0_enc_ring_get_rptr,
.get_wptr = vcn_v2_0_enc_ring_get_wptr,
.set_wptr = vcn_v2_0_enc_ring_set_wptr,
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
index 274d5063e9a2..42edc91f4a78 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
@@ -116,9 +116,9 @@ static int amdgpu_ih_clientid_vcns[] = {
* Set ring and irq function pointers
* Load microcode from filesystem
*/
-static int vcn_v2_5_early_init(void *handle)
+static int vcn_v2_5_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
if (amdgpu_sriov_vf(adev)) {
adev->vcn.num_vcn_inst = 2;
@@ -1628,6 +1628,7 @@ static void vcn_v2_5_dec_ring_set_wptr(struct amdgpu_ring *ring)
static const struct amdgpu_ring_funcs vcn_v2_5_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_DEC,
.align_mask = 0xf,
+ .no_user_fence = true,
.secure_submission_supported = true,
.get_rptr = vcn_v2_5_dec_ring_get_rptr,
.get_wptr = vcn_v2_5_dec_ring_get_wptr,
@@ -1728,6 +1729,7 @@ static const struct amdgpu_ring_funcs vcn_v2_5_enc_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_ENC,
.align_mask = 0x3f,
.nop = VCN_ENC_CMD_NO_OP,
+ .no_user_fence = true,
.get_rptr = vcn_v2_5_enc_ring_get_rptr,
.get_wptr = vcn_v2_5_enc_ring_get_wptr,
.set_wptr = vcn_v2_5_enc_ring_set_wptr,
@@ -1965,9 +1967,9 @@ static void vcn_v2_5_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v2_5_dump_ip_state(void *handle)
+static void vcn_v2_5_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
index f4ac8bcdb70a..09f8324fdb99 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
@@ -122,9 +122,9 @@ static void vcn_v3_0_enc_ring_set_wptr(struct amdgpu_ring *ring);
* Set ring and irq function pointers
* Load microcode from filesystem
*/
-static int vcn_v3_0_early_init(void *handle)
+static int vcn_v3_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
if (amdgpu_sriov_vf(adev)) {
adev->vcn.num_vcn_inst = VCN_INSTANCES_SIENNA_CICHLID;
@@ -1791,6 +1791,7 @@ static const struct amdgpu_ring_funcs vcn_v3_0_dec_sw_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_DEC,
.align_mask = 0x3f,
.nop = VCN_DEC_SW_CMD_NO_OP,
+ .no_user_fence = true,
.secure_submission_supported = true,
.get_rptr = vcn_v3_0_dec_ring_get_rptr,
.get_wptr = vcn_v3_0_dec_ring_get_wptr,
@@ -1972,6 +1973,7 @@ static int vcn_v3_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p,
static const struct amdgpu_ring_funcs vcn_v3_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_DEC,
.align_mask = 0xf,
+ .no_user_fence = true,
.secure_submission_supported = true,
.get_rptr = vcn_v3_0_dec_ring_get_rptr,
.get_wptr = vcn_v3_0_dec_ring_get_wptr,
@@ -2073,6 +2075,7 @@ static const struct amdgpu_ring_funcs vcn_v3_0_enc_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_ENC,
.align_mask = 0x3f,
.nop = VCN_ENC_CMD_NO_OP,
+ .no_user_fence = true,
.get_rptr = vcn_v3_0_enc_ring_get_rptr,
.get_wptr = vcn_v3_0_enc_ring_get_wptr,
.set_wptr = vcn_v3_0_enc_ring_set_wptr,
@@ -2312,9 +2315,9 @@ static void vcn_v3_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v3_0_dump_ip_state(void *handle)
+static void vcn_v3_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
index 2f8d07a7b60b..041531a799d4 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
@@ -112,9 +112,9 @@ static void vcn_v4_0_set_ras_funcs(struct amdgpu_device *adev);
* Set ring and irq function pointers
* Load microcode from filesystem
*/
-static int vcn_v4_0_early_init(void *handle)
+static int vcn_v4_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i;
if (amdgpu_sriov_vf(adev)) {
@@ -2210,9 +2210,9 @@ static void vcn_v4_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v4_0_dump_ip_state(void *handle)
+static void vcn_v4_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c
index 2094357a931c..7edb6da93fd2 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_3.c
@@ -103,9 +103,9 @@ static void vcn_v4_0_3_enable_ras(struct amdgpu_device *adev,
*
* Set ring and irq function pointers
*/
-static int vcn_v4_0_3_early_init(void *handle)
+static int vcn_v4_0_3_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
/* re-use enc ring as unified ring */
adev->vcn.num_enc_rings = 1;
@@ -1520,6 +1520,7 @@ static const struct amdgpu_ring_funcs vcn_v4_0_3_unified_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_ENC,
.align_mask = 0x3f,
.nop = VCN_ENC_CMD_NO_OP,
+ .no_user_fence = true,
.get_rptr = vcn_v4_0_3_unified_ring_get_rptr,
.get_wptr = vcn_v4_0_3_unified_ring_get_wptr,
.set_wptr = vcn_v4_0_3_unified_ring_set_wptr,
@@ -1787,9 +1788,9 @@ static void vcn_v4_0_3_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v4_0_3_dump_ip_state(void *handle)
+static void vcn_v4_0_3_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off, inst_id;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_5.c b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_5.c
index 48cb61a9c13f..f0354a7dfb46 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0_5.c
@@ -110,9 +110,9 @@ static void vcn_v4_0_5_unified_ring_set_wptr(struct amdgpu_ring *ring);
* Set ring and irq function pointers
* Load microcode from filesystem
*/
-static int vcn_v4_0_5_early_init(void *handle)
+static int vcn_v4_0_5_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
/* re-use enc ring as unified ring */
adev->vcn.num_enc_rings = 1;
@@ -1414,6 +1414,7 @@ static const struct amdgpu_ring_funcs vcn_v4_0_5_unified_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_ENC,
.align_mask = 0x3f,
.nop = VCN_ENC_CMD_NO_OP,
+ .no_user_fence = true,
.get_rptr = vcn_v4_0_5_unified_ring_get_rptr,
.get_wptr = vcn_v4_0_5_unified_ring_get_wptr,
.set_wptr = vcn_v4_0_5_unified_ring_set_wptr,
@@ -1663,9 +1664,9 @@ static void vcn_v4_0_5_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v4_0_5_dump_ip_state(void *handle)
+static void vcn_v4_0_5_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off;
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v5_0_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v5_0_0.c
index 3aa715830fbe..f8994e4bdf4b 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v5_0_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v5_0_0.c
@@ -92,9 +92,9 @@ static void vcn_v5_0_0_unified_ring_set_wptr(struct amdgpu_ring *ring);
* Set ring and irq function pointers
* Load microcode from filesystem
*/
-static int vcn_v5_0_0_early_init(void *handle)
+static int vcn_v5_0_0_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
/* re-use enc ring as unified ring */
adev->vcn.num_enc_rings = 1;
@@ -1137,6 +1137,7 @@ static const struct amdgpu_ring_funcs vcn_v5_0_0_unified_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_ENC,
.align_mask = 0x3f,
.nop = VCN_ENC_CMD_NO_OP,
+ .no_user_fence = true,
.get_rptr = vcn_v5_0_0_unified_ring_get_rptr,
.get_wptr = vcn_v5_0_0_unified_ring_get_wptr,
.set_wptr = vcn_v5_0_0_unified_ring_set_wptr,
@@ -1386,9 +1387,9 @@ static void vcn_v5_0_print_ip_state(void *handle, struct drm_printer *p)
}
}
-static void vcn_v5_0_dump_ip_state(void *handle)
+static void vcn_v5_0_dump_ip_state(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
int i, j;
bool is_powered;
uint32_t inst_off;
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
index bf68e18e3824..5b0c81d510e7 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
@@ -472,9 +472,9 @@ static void vega10_ih_set_self_irq_funcs(struct amdgpu_device *adev)
adev->irq.self_irq.funcs = &vega10_ih_self_irq_funcs;
}
-static int vega10_ih_early_init(void *handle)
+static int vega10_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
vega10_ih_set_interrupt_funcs(adev);
vega10_ih_set_self_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
index 16f5561fb86e..a96c7737d1b4 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
@@ -553,9 +553,9 @@ static void vega20_ih_set_self_irq_funcs(struct amdgpu_device *adev)
adev->irq.self_irq.funcs = &vega20_ih_self_irq_funcs;
}
-static int vega20_ih_early_init(void *handle)
+static int vega20_ih_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
vega20_ih_set_interrupt_funcs(adev);
vega20_ih_set_self_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vi.c b/drivers/gpu/drm/amd/amdgpu/vi.c
index 6e4f9c6108f6..80d06a7c1db7 100644
--- a/drivers/gpu/drm/amd/amdgpu/vi.c
+++ b/drivers/gpu/drm/amd/amdgpu/vi.c
@@ -1455,9 +1455,9 @@ static const struct amdgpu_asic_funcs vi_asic_funcs =
#define CZ_REV_BRISTOL(rev) \
((rev >= 0xC8 && rev <= 0xCE) || (rev >= 0xE1 && rev <= 0xE6))
-static int vi_common_early_init(void *handle)
+static int vi_common_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
if (adev->flags & AMD_IS_APU) {
adev->smc_rreg = &cz_smc_rreg;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 1ed631006e63..ad9125f3655f 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -975,7 +975,7 @@ static int dm_set_powergating_state(void *handle,
}
/* Prototypes of private functions */
-static int dm_early_init(void *handle);
+static int dm_early_init(struct amdgpu_ip_block *ip_block);
/* Allocate memory for FBC compressed data */
static void amdgpu_dm_fbc_init(struct drm_connector *connector)
@@ -5386,9 +5386,9 @@ static int dm_init_microcode(struct amdgpu_device *adev)
return r;
}
-static int dm_early_init(void *handle)
+static int dm_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
struct amdgpu_mode_info *mode_info = &adev->mode_info;
struct atom_context *ctx = mode_info->atom_context;
int index = GetIndexIntoMasterTable(DATA, Object_Header);
diff --git a/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c b/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c
index a523c5cfcd24..fad0129bf8b1 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/bios_parser.c
@@ -1215,6 +1215,60 @@ static enum bp_result bios_parser_get_embedded_panel_info(
return BP_RESULT_FAILURE;
}
+static enum bp_result get_embedded_panel_extra_info(
+ struct bios_parser *bp,
+ struct embedded_panel_info *info,
+ const uint32_t table_offset)
+{
+ uint8_t *record = bios_get_image(&bp->base, table_offset, 1);
+ ATOM_PANEL_RESOLUTION_PATCH_RECORD *panel_res_record;
+ ATOM_FAKE_EDID_PATCH_RECORD *fake_edid_record;
+
+ while (*record != ATOM_RECORD_END_TYPE) {
+ switch (*record) {
+ case LCD_MODE_PATCH_RECORD_MODE_TYPE:
+ record += sizeof(ATOM_PATCH_RECORD_MODE);
+ break;
+ case LCD_RTS_RECORD_TYPE:
+ record += sizeof(ATOM_LCD_RTS_RECORD);
+ break;
+ case LCD_CAP_RECORD_TYPE:
+ record += sizeof(ATOM_LCD_MODE_CONTROL_CAP);
+ break;
+ case LCD_FAKE_EDID_PATCH_RECORD_TYPE:
+ fake_edid_record = (ATOM_FAKE_EDID_PATCH_RECORD *)record;
+ if (fake_edid_record->ucFakeEDIDLength) {
+ if (fake_edid_record->ucFakeEDIDLength == 128)
+ info->fake_edid_size =
+ fake_edid_record->ucFakeEDIDLength;
+ else
+ info->fake_edid_size =
+ fake_edid_record->ucFakeEDIDLength * 128;
+
+ info->fake_edid = fake_edid_record->ucFakeEDIDString;
+
+ record += struct_size(fake_edid_record,
+ ucFakeEDIDString,
+ info->fake_edid_size);
+ } else {
+ /* empty fake edid record must be 3 bytes long */
+ record += sizeof(ATOM_FAKE_EDID_PATCH_RECORD) + 1;
+ }
+ break;
+ case LCD_PANEL_RESOLUTION_RECORD_TYPE:
+ panel_res_record = (ATOM_PANEL_RESOLUTION_PATCH_RECORD *)record;
+ info->panel_width_mm = panel_res_record->usHSize;
+ info->panel_height_mm = panel_res_record->usVSize;
+ record += sizeof(ATOM_PANEL_RESOLUTION_PATCH_RECORD);
+ break;
+ default:
+ return BP_RESULT_BADBIOSTABLE;
+ }
+ }
+
+ return BP_RESULT_OK;
+}
+
static enum bp_result get_embedded_panel_info_v1_2(
struct bios_parser *bp,
struct embedded_panel_info *info)
@@ -1331,6 +1385,10 @@ static enum bp_result get_embedded_panel_info_v1_2(
if (ATOM_PANEL_MISC_API_ENABLED & lvds->ucLVDS_Misc)
info->lcd_timing.misc_info.API_ENABLED = true;
+ if (lvds->usExtInfoTableOffset)
+ return get_embedded_panel_extra_info(bp, info,
+ le16_to_cpu(lvds->usExtInfoTableOffset) + DATA_TABLES(LCD_Info));
+
return BP_RESULT_OK;
}
@@ -1456,6 +1514,10 @@ static enum bp_result get_embedded_panel_info_v1_3(
(uint32_t) (ATOM_PANEL_MISC_V13_GREY_LEVEL &
lvds->ucLCD_Misc) >> ATOM_PANEL_MISC_V13_GREY_LEVEL_SHIFT;
+ if (lvds->usExtInfoTableOffset)
+ return get_embedded_panel_extra_info(bp, info,
+ le16_to_cpu(lvds->usExtInfoTableOffset) + DATA_TABLES(LCD_Info));
+
return BP_RESULT_OK;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c
index 0c50fe266c8a..4103213a572a 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c
@@ -989,7 +989,9 @@ void dce110_link_encoder_hw_init(
ASSERT(result == BP_RESULT_OK);
}
- aux_initialize(enc110);
+
+ if (enc110->aux_regs)
+ aux_initialize(enc110);
/* reinitialize HPD.
* hpd_initialize() will pass DIG_FE id to HW context.
diff --git a/drivers/gpu/drm/amd/display/include/grph_object_ctrl_defs.h b/drivers/gpu/drm/amd/display/include/grph_object_ctrl_defs.h
index 813463ffe15c..8e776c90d21b 100644
--- a/drivers/gpu/drm/amd/display/include/grph_object_ctrl_defs.h
+++ b/drivers/gpu/drm/amd/display/include/grph_object_ctrl_defs.h
@@ -153,6 +153,10 @@ struct embedded_panel_info {
uint32_t drr_enabled;
uint32_t min_drr_refresh_rate;
bool realtek_eDPToLVDS;
+ uint16_t panel_width_mm;
+ uint16_t panel_height_mm;
+ uint16_t fake_edid_size;
+ const uint8_t *fake_edid;
};
struct dc_firmware_info {
diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h
index 3f91926a50e9..847ca23a7c85 100644
--- a/drivers/gpu/drm/amd/include/amd_shared.h
+++ b/drivers/gpu/drm/amd/include/amd_shared.h
@@ -375,9 +375,11 @@ enum amd_dpm_forced_level;
* making calls to hooks from each IP block. This list is ordered to ensure
* that the driver initializes the IP blocks in a safe sequence.
*/
+struct amdgpu_ip_block;
+
struct amd_ip_funcs {
char *name;
- int (*early_init)(void *handle);
+ int (*early_init)(struct amdgpu_ip_block *ip_block);
int (*late_init)(void *handle);
int (*sw_init)(void *handle);
int (*sw_fini)(void *handle);
@@ -399,7 +401,7 @@ struct amd_ip_funcs {
int (*set_powergating_state)(void *handle,
enum amd_powergating_state state);
void (*get_clockgating_state)(void *handle, u64 *flags);
- void (*dump_ip_state)(void *handle);
+ void (*dump_ip_state)(struct amdgpu_ip_block *ip_block);
void (*print_ip_state)(void *handle, struct drm_printer *p);
};
diff --git a/drivers/gpu/drm/amd/pm/legacy-dpm/kv_dpm.c b/drivers/gpu/drm/amd/pm/legacy-dpm/kv_dpm.c
index 8cf7e517da84..e4820d8850ba 100644
--- a/drivers/gpu/drm/amd/pm/legacy-dpm/kv_dpm.c
+++ b/drivers/gpu/drm/amd/pm/legacy-dpm/kv_dpm.c
@@ -2954,9 +2954,9 @@ static int kv_dpm_get_temp(void *handle)
return actual_temp;
}
-static int kv_dpm_early_init(void *handle)
+static int kv_dpm_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->powerplay.pp_funcs = &kv_dpm_funcs;
adev->powerplay.pp_handle = adev;
diff --git a/drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c b/drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c
index 26defd72a36c..b5ecef3f75bb 100644
--- a/drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c
+++ b/drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c
@@ -7972,10 +7972,10 @@ static void si_dpm_print_power_state(void *handle,
amdgpu_dpm_print_ps_status(adev, rps);
}
-static int si_dpm_early_init(void *handle)
+static int si_dpm_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
adev->powerplay.pp_funcs = &si_dpm_funcs;
adev->powerplay.pp_handle = adev;
diff --git a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
index 24b25cddf0c1..78721bce42fd 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
@@ -80,11 +80,10 @@ static void amd_powerplay_destroy(struct amdgpu_device *adev)
hwmgr = NULL;
}
-static int pp_early_init(void *handle)
+static int pp_early_init(struct amdgpu_ip_block *ip_block)
{
int ret;
- struct amdgpu_device *adev = handle;
-
+ struct amdgpu_device *adev = ip_block->adev;
ret = amd_powerplay_create(adev);
if (ret != 0)
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c
index 2b5ac21fee39..1d6e30269d56 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c
@@ -104,6 +104,21 @@ int hwmgr_early_init(struct pp_hwmgr *hwmgr)
PP_GFXOFF_MASK);
hwmgr->pp_table_version = PP_TABLE_V0;
hwmgr->od_enabled = false;
+ switch (hwmgr->chip_id) {
+ case CHIP_BONAIRE:
+ /* R9 M380 in iMac 2015: SMU hangs when enabling MCLK DPM
+ * R7 260X cards with old MC ucode: MCLK DPM is unstable
+ */
+ if (adev->pdev->subsystem_vendor == 0x106B ||
+ adev->pdev->device == 0x6658) {
+ dev_info(adev->dev, "disabling MCLK DPM on quirky ASIC");
+ adev->pm.pp_feature &= ~PP_MCLK_DPM_MASK;
+ hwmgr->feature_mask &= ~PP_MCLK_DPM_MASK;
+ }
+ break;
+ default:
+ break;
+ }
smu7_init_function_pointers(hwmgr);
break;
case AMDGPU_FAMILY_CZ:
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
index 3018e294673a..80e34a7748d7 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
@@ -787,7 +787,7 @@ static int smu7_setup_dpm_tables_v0(struct pp_hwmgr *hwmgr)
hwmgr->dyn_state.vddc_dependency_on_mclk;
struct phm_cac_leakage_table *std_voltage_table =
hwmgr->dyn_state.cac_leakage_table;
- uint32_t i;
+ uint32_t i, clk;
PP_ASSERT_WITH_CODE(allowed_vdd_sclk_table != NULL,
"SCLK dependency table is missing. This table is mandatory", return -EINVAL);
@@ -804,10 +804,12 @@ static int smu7_setup_dpm_tables_v0(struct pp_hwmgr *hwmgr)
data->dpm_table.sclk_table.count = 0;
for (i = 0; i < allowed_vdd_sclk_table->count; i++) {
+ clk = min(allowed_vdd_sclk_table->entries[i].clk, data->sclk_cap);
+
if (i == 0 || data->dpm_table.sclk_table.dpm_levels[data->dpm_table.sclk_table.count-1].value !=
- allowed_vdd_sclk_table->entries[i].clk) {
+ clk) {
data->dpm_table.sclk_table.dpm_levels[data->dpm_table.sclk_table.count].value =
- allowed_vdd_sclk_table->entries[i].clk;
+ clk;
data->dpm_table.sclk_table.dpm_levels[data->dpm_table.sclk_table.count].enabled = (i == 0) ? 1 : 0;
data->dpm_table.sclk_table.count++;
}
@@ -2802,6 +2804,10 @@ static int smu7_patch_dependency_tables_with_leakage(struct pp_hwmgr *hwmgr)
if (tmp)
return -EINVAL;
+ tmp = smu7_patch_vddc(hwmgr, hwmgr->dyn_state.vddc_dependency_on_display_clock);
+ if (tmp)
+ return -EINVAL;
+
tmp = smu7_patch_vce_vddc(hwmgr, hwmgr->dyn_state.vce_clock_voltage_dependency_table);
if (tmp)
return -EINVAL;
@@ -2885,6 +2891,8 @@ static int smu7_hwmgr_backend_fini(struct pp_hwmgr *hwmgr)
{
kfree(hwmgr->dyn_state.vddc_dep_on_dal_pwrl);
hwmgr->dyn_state.vddc_dep_on_dal_pwrl = NULL;
+ kfree(hwmgr->dyn_state.vddc_dependency_on_display_clock);
+ hwmgr->dyn_state.vddc_dependency_on_display_clock = NULL;
kfree(hwmgr->backend);
hwmgr->backend = NULL;
@@ -2955,6 +2963,70 @@ static int smu7_update_edc_leakage_table(struct pp_hwmgr *hwmgr)
return ret;
}
+static int smu7_init_voltage_dependency_on_display_clock_table(struct pp_hwmgr *hwmgr)
+{
+ struct phm_clock_voltage_dependency_table *table;
+
+ if (!amdgpu_device_ip_get_ip_block(hwmgr->adev, AMD_IP_BLOCK_TYPE_DCE))
+ return 0;
+
+ table = kzalloc(struct_size(table, entries, 4), GFP_KERNEL);
+ if (!table)
+ return -ENOMEM;
+
+ if (hwmgr->chip_id >= CHIP_POLARIS10) {
+ table->entries[0].clk = 38918;
+ table->entries[1].clk = 45900;
+ table->entries[2].clk = 66700;
+ table->entries[3].clk = 113200;
+
+ table->entries[0].v = 700;
+ table->entries[1].v = 740;
+ table->entries[2].v = 800;
+ table->entries[3].v = 900;
+ } else {
+ if (hwmgr->chip_family == AMDGPU_FAMILY_CZ) {
+ table->entries[0].clk = 35200;
+ table->entries[1].clk = 35200;
+ table->entries[2].clk = 46700;
+ table->entries[3].clk = 64300;
+ } else {
+ table->entries[0].clk = 0;
+ table->entries[1].clk = 35200;
+ table->entries[2].clk = 54000;
+ table->entries[3].clk = 62500;
+ }
+
+ table->entries[0].v = 0;
+ table->entries[1].v = 720;
+ table->entries[2].v = 810;
+ table->entries[3].v = 900;
+ }
+
+ table->count = 4;
+ hwmgr->dyn_state.vddc_dependency_on_display_clock = table;
+ return 0;
+}
+
+static void smu7_set_sclk_cap(struct pp_hwmgr *hwmgr)
+{
+ struct amdgpu_device *adev = hwmgr->adev;
+ struct smu7_hwmgr *data = (struct smu7_hwmgr *)(hwmgr->backend);
+
+ data->sclk_cap = 0xffffffff;
+
+ if (hwmgr->od_enabled)
+ return;
+
+ /* R9 390X board: last sclk dpm level is unstable, use lower sclk */
+ if (adev->pdev->device == 0x67B0 &&
+ adev->pdev->subsystem_vendor == 0x1043)
+ data->sclk_cap = 104000; /* 1040 MHz */
+
+ if (data->sclk_cap != 0xffffffff)
+ dev_info(adev->dev, "sclk cap: %u kHz on quirky ASIC\n", data->sclk_cap * 10);
+}
+
static int smu7_hwmgr_backend_init(struct pp_hwmgr *hwmgr)
{
struct amdgpu_device *adev = hwmgr->adev;
@@ -2966,6 +3038,7 @@ static int smu7_hwmgr_backend_init(struct pp_hwmgr *hwmgr)
return -ENOMEM;
hwmgr->backend = data;
+ smu7_set_sclk_cap(hwmgr);
smu7_patch_voltage_workaround(hwmgr);
smu7_init_dpm_defaults(hwmgr);
@@ -2983,6 +3056,10 @@ static int smu7_hwmgr_backend_init(struct pp_hwmgr *hwmgr)
smu7_get_elb_voltages(hwmgr);
}
+ result = smu7_init_voltage_dependency_on_display_clock_table(hwmgr);
+ if (result)
+ goto fail;
+
if (hwmgr->pp_table_version == PP_TABLE_V1) {
smu7_complete_dependency_tables(hwmgr);
smu7_set_private_data_based_on_pptable_v1(hwmgr);
@@ -3079,13 +3156,40 @@ static int smu7_force_dpm_highest(struct pp_hwmgr *hwmgr)
return 0;
}
+static uint32_t smu7_lookup_vddc_from_dispclk(struct pp_hwmgr *hwmgr)
+{
+ const struct amd_pp_display_configuration *cfg = hwmgr->display_config;
+ const struct phm_clock_voltage_dependency_table *vddc_dep_on_dispclk =
+ hwmgr->dyn_state.vddc_dependency_on_display_clock;
+ uint32_t i;
+
+ if (!vddc_dep_on_dispclk || !vddc_dep_on_dispclk->count ||
+ !cfg || !cfg->num_display || !cfg->display_clk)
+ return 0;
+
+ /* Start from 1 because ClocksStateUltraLow should not be used according to DC. */
+ for (i = 1; i < vddc_dep_on_dispclk->count; ++i)
+ if (vddc_dep_on_dispclk->entries[i].clk >= cfg->display_clk)
+ return vddc_dep_on_dispclk->entries[i].v;
+
+ return vddc_dep_on_dispclk->entries[vddc_dep_on_dispclk->count - 1].v;
+}
+
+static void smu7_apply_minimum_dce_voltage_request(struct pp_hwmgr *hwmgr)
+{
+ uint32_t req_vddc = smu7_lookup_vddc_from_dispclk(hwmgr);
+
+ smum_send_msg_to_smc_with_parameter(hwmgr,
+ PPSMC_MSG_VddC_Request,
+ req_vddc * VOLTAGE_SCALE,
+ NULL);
+}
+
static int smu7_upload_dpm_level_enable_mask(struct pp_hwmgr *hwmgr)
{
struct smu7_hwmgr *data = (struct smu7_hwmgr *)(hwmgr->backend);
- if (hwmgr->pp_table_version == PP_TABLE_V1)
- phm_apply_dal_min_voltage_request(hwmgr);
-/* TO DO for v0 iceland and Ci*/
+ smu7_apply_minimum_dce_voltage_request(hwmgr);
if (!data->sclk_dpm_key_disabled) {
if (data->dpm_level_enable_mask.sclk_dpm_enable_mask)
@@ -3821,7 +3925,7 @@ static int smu7_get_pp_table_entry_callback_func_v0(struct pp_hwmgr *hwmgr,
/* Performance levels are arranged from low to high. */
performance_level->memory_clock = memory_clock;
- performance_level->engine_clock = engine_clock;
+ performance_level->engine_clock = min(engine_clock, data->sclk_cap);
pcie_gen_from_bios = visland_clk_info->ucPCIEGen;
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.h b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.h
index d9e8b386bd4d..66adabeab6a3 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.h
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.h
@@ -234,6 +234,7 @@ struct smu7_hwmgr {
uint32_t pcie_gen_cap;
uint32_t pcie_lane_cap;
uint32_t pcie_spc_cap;
+ uint32_t sclk_cap;
struct smu7_leakage_voltage vddc_leakage;
struct smu7_leakage_voltage vddci_leakage;
struct smu7_leakage_voltage vddcgfx_leakage;
diff --git a/drivers/gpu/drm/amd/pm/powerplay/inc/hwmgr.h b/drivers/gpu/drm/amd/pm/powerplay/inc/hwmgr.h
index 227bf0e84a13..d829121d29fb 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/inc/hwmgr.h
+++ b/drivers/gpu/drm/amd/pm/powerplay/inc/hwmgr.h
@@ -632,6 +632,7 @@ struct phm_dynamic_state_info {
struct phm_clock_voltage_dependency_table *vddci_dependency_on_mclk;
struct phm_clock_voltage_dependency_table *vddc_dependency_on_mclk;
struct phm_clock_voltage_dependency_table *mvdd_dependency_on_mclk;
+ struct phm_clock_voltage_dependency_table *vddc_dependency_on_display_clock;
struct phm_clock_voltage_dependency_table *vddc_dep_on_dal_pwrl;
struct phm_clock_array *valid_sclk_values;
struct phm_clock_array *valid_mclk_values;
diff --git a/drivers/gpu/drm/amd/pm/powerplay/smumgr/ci_smumgr.c b/drivers/gpu/drm/amd/pm/powerplay/smumgr/ci_smumgr.c
index 0cb7eaaba384..aea3ad523cc0 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/smumgr/ci_smumgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/smumgr/ci_smumgr.c
@@ -245,7 +245,7 @@ static void ci_initialize_power_tune_defaults(struct pp_hwmgr *hwmgr)
smu_data->power_tune_defaults = &defaults_hawaii_pro;
break;
case 0x67B8:
- case 0x66B0:
+ case 0x67B0:
smu_data->power_tune_defaults = &defaults_hawaii_xt;
break;
case 0x6640:
@@ -543,12 +543,11 @@ static int ci_populate_dw8(struct pp_hwmgr *hwmgr, uint32_t fuse_table_offset)
{
struct ci_smumgr *smu_data = (struct ci_smumgr *)(hwmgr->smu_backend);
const struct ci_pt_defaults *defaults = smu_data->power_tune_defaults;
- uint32_t temp;
if (ci_read_smc_sram_dword(hwmgr,
fuse_table_offset +
offsetof(SMU7_Discrete_PmFuses, TdcWaterfallCtl),
- (uint32_t *)&temp, SMC_RAM_END))
+ (uint32_t *)&smu_data->power_tune_table.TdcWaterfallCtl, SMC_RAM_END))
PP_ASSERT_WITH_CODE(false,
"Attempt to read PmFuses.DW6 (SviLoadLineEn) from SMC Failed!",
return -EINVAL);
@@ -1217,7 +1216,7 @@ static int ci_populate_single_memory_level(
}
memory_level->EnabledForThrottle = 1;
- memory_level->EnabledForActivity = 1;
+ memory_level->EnabledForActivity = 0;
memory_level->UpH = data->current_profile_setting.mclk_up_hyst;
memory_level->DownH = data->current_profile_setting.mclk_down_hyst;
memory_level->VoltageDownH = 0;
@@ -1322,6 +1321,14 @@ static int ci_populate_all_memory_levels(struct pp_hwmgr *hwmgr)
return result;
}
+ if (data->mclk_dpm_key_disabled && dpm_table->mclk_table.count) {
+ /* Populate the table with the highest MCLK level when MCLK DPM is disabled */
+ for (i = 0; i < dpm_table->mclk_table.count - 1; i++) {
+ levels[i] = levels[dpm_table->mclk_table.count - 1];
+ levels[i].DisplayWatermark = PPSMC_DISPLAY_WATERMARK_HIGH;
+ }
+ }
+
smu_data->smc_state_table.MemoryLevel[0].EnabledForActivity = 1;
dev_id = adev->pdev->device;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index dff21c1f7015..a6683a8cebf2 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -746,9 +746,9 @@ static int smu_set_funcs(struct amdgpu_device *adev)
return 0;
}
-static int smu_early_init(void *handle)
+static int smu_early_init(struct amdgpu_ip_block *ip_block)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ struct amdgpu_device *adev = ip_block->adev;
struct smu_context *smu;
int r;
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c b/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c
index df5da5a44755..b4f2b89651ff 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c
@@ -4,6 +4,8 @@
* Author: James.Qian.Wang <james.qian.wang@arm.com>
*
*/
+#include <linux/overflow.h>
+
#include <drm/drm_device.h>
#include <drm/drm_fb_dma_helper.h>
#include <drm/drm_gem.h>
@@ -92,7 +94,9 @@ komeda_fb_afbc_size_check(struct komeda_fb *kfb, struct drm_file *file,
kfb->afbc_size = kfb->offset_payload + n_blocks *
ALIGN(bpp * AFBC_SUPERBLK_PIXELS / 8,
AFBC_SUPERBLK_ALIGNMENT);
- min_size = kfb->afbc_size + fb->offsets[0];
+ if (check_add_overflow(kfb->afbc_size, fb->offsets[0], &min_size)) {
+ goto check_failed;
+ }
if (min_size > obj->size) {
DRM_DEBUG_KMS("afbc size check failed, obj_size: 0x%zx. min_size 0x%llx.\n",
obj->size, min_size);
diff --git a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c
index b18bdb2daddf..3c4ea9bdefbe 100644
--- a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c
+++ b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c
@@ -810,7 +810,7 @@ static void cdns_mhdp_fw_cb(const struct firmware *fw, void *context)
bridge_attached = mhdp->bridge_attached;
spin_unlock(&mhdp->start_lock);
if (bridge_attached) {
- if (mhdp->connector.dev)
+ if (mhdp->connector_ptr)
drm_kms_helper_hotplug_event(mhdp->bridge.dev);
else
drm_bridge_hpd_notify(&mhdp->bridge, cdns_mhdp_detect(mhdp));
@@ -1706,6 +1706,7 @@ static int cdns_mhdp_connector_init(struct cdns_mhdp_device *mhdp)
return ret;
}
+ mhdp->connector_ptr = conn;
drm_connector_helper_add(conn, &cdns_mhdp_conn_helper_funcs);
ret = drm_display_info_set_bus_formats(&conn->display_info,
@@ -1985,17 +1986,25 @@ static void cdns_mhdp_atomic_enable(struct drm_bridge *bridge,
struct drm_atomic_state *state = bridge_state->base.state;
struct cdns_mhdp_bridge_state *mhdp_state;
struct drm_crtc_state *crtc_state;
- struct drm_connector *connector;
struct drm_connector_state *conn_state;
struct drm_bridge_state *new_state;
const struct drm_display_mode *mode;
u32 resp;
- int ret;
+ int ret = 0;
dev_dbg(mhdp->dev, "bridge enable\n");
mutex_lock(&mhdp->link_mutex);
+ mhdp->connector_ptr = drm_atomic_get_new_connector_for_encoder(state,
+ bridge->encoder);
+ if (WARN_ON(!mhdp->connector_ptr))
+ goto out;
+
+ conn_state = drm_atomic_get_new_connector_state(state, mhdp->connector_ptr);
+ if (WARN_ON(!conn_state))
+ goto out;
+
if (mhdp->plugged && !mhdp->link_up) {
ret = cdns_mhdp_link_up(mhdp);
if (ret < 0)
@@ -2015,15 +2024,6 @@ static void cdns_mhdp_atomic_enable(struct drm_bridge *bridge,
cdns_mhdp_reg_write(mhdp, CDNS_DPTX_CAR,
resp | CDNS_VIF_CLK_EN | CDNS_VIF_CLK_RSTN);
- connector = drm_atomic_get_new_connector_for_encoder(state,
- bridge->encoder);
- if (WARN_ON(!connector))
- goto out;
-
- conn_state = drm_atomic_get_new_connector_state(state, connector);
- if (WARN_ON(!conn_state))
- goto out;
-
if (mhdp->hdcp_supported &&
mhdp->hw_state == MHDP_HW_READY &&
conn_state->content_protection ==
@@ -2100,6 +2100,7 @@ static void cdns_mhdp_atomic_disable(struct drm_bridge *bridge,
if (mhdp->info && mhdp->info->ops && mhdp->info->ops->disable)
mhdp->info->ops->disable(mhdp);
+ mhdp->connector_ptr = NULL;
mutex_unlock(&mhdp->link_mutex);
}
@@ -2192,6 +2193,10 @@ static int cdns_mhdp_atomic_check(struct drm_bridge *bridge,
{
struct cdns_mhdp_device *mhdp = bridge_to_mhdp(bridge);
const struct drm_display_mode *mode = &crtc_state->adjusted_mode;
+ struct drm_connector_state *old_state, *new_state;
+ struct drm_atomic_state *state = crtc_state->state;
+ struct drm_connector *conn = mhdp->connector_ptr;
+ u64 old_cp, new_cp;
mutex_lock(&mhdp->link_mutex);
@@ -2211,6 +2216,25 @@ static int cdns_mhdp_atomic_check(struct drm_bridge *bridge,
if (mhdp->info)
bridge_state->input_bus_cfg.flags = *mhdp->info->input_bus_flags;
+ if (conn && mhdp->hdcp_supported) {
+ old_state = drm_atomic_get_old_connector_state(state, conn);
+ new_state = drm_atomic_get_new_connector_state(state, conn);
+ old_cp = old_state->content_protection;
+ new_cp = new_state->content_protection;
+
+ if (old_state->hdcp_content_type != new_state->hdcp_content_type &&
+ new_cp != DRM_MODE_CONTENT_PROTECTION_UNDESIRED) {
+ new_state->content_protection = DRM_MODE_CONTENT_PROTECTION_DESIRED;
+ crtc_state = drm_atomic_get_new_crtc_state(state, new_state->crtc);
+ crtc_state->mode_changed = true;
+ }
+
+ if (!new_state->crtc) {
+ if (old_cp == DRM_MODE_CONTENT_PROTECTION_ENABLED)
+ new_state->content_protection = DRM_MODE_CONTENT_PROTECTION_DESIRED;
+ }
+ }
+
mutex_unlock(&mhdp->link_mutex);
return 0;
}
@@ -2230,6 +2254,25 @@ static const struct drm_edid *cdns_mhdp_bridge_edid_read(struct drm_bridge *brid
return cdns_mhdp_edid_read(mhdp, connector);
}
+static enum drm_mode_status
+cdns_mhdp_bridge_mode_valid(struct drm_bridge *bridge,
+ const struct drm_display_info *info,
+ const struct drm_display_mode *mode)
+{
+ struct cdns_mhdp_device *mhdp = bridge_to_mhdp(bridge);
+
+ mutex_lock(&mhdp->link_mutex);
+
+ if (!cdns_mhdp_bandwidth_ok(mhdp, mode, mhdp->link.num_lanes,
+ mhdp->link.rate)) {
+ mutex_unlock(&mhdp->link_mutex);
+ return MODE_CLOCK_HIGH;
+ }
+
+ mutex_unlock(&mhdp->link_mutex);
+ return MODE_OK;
+}
+
static const struct drm_bridge_funcs cdns_mhdp_bridge_funcs = {
.atomic_enable = cdns_mhdp_atomic_enable,
.atomic_disable = cdns_mhdp_atomic_disable,
@@ -2244,6 +2287,7 @@ static const struct drm_bridge_funcs cdns_mhdp_bridge_funcs = {
.edid_read = cdns_mhdp_bridge_edid_read,
.hpd_enable = cdns_mhdp_bridge_hpd_enable,
.hpd_disable = cdns_mhdp_bridge_hpd_disable,
+ .mode_valid = cdns_mhdp_bridge_mode_valid,
};
static bool cdns_mhdp_detect_hpd(struct cdns_mhdp_device *mhdp, bool *hpd_pulse)
@@ -2365,7 +2409,7 @@ static void cdns_mhdp_modeset_retry_fn(struct work_struct *work)
mhdp = container_of(work, typeof(*mhdp), modeset_retry_work);
- conn = &mhdp->connector;
+ conn = mhdp->connector_ptr;
/* Grab the locks before changing connector property */
mutex_lock(&conn->dev->mode_config.mutex);
@@ -2442,7 +2486,7 @@ static void cdns_mhdp_hpd_work(struct work_struct *work)
int ret;
ret = cdns_mhdp_update_link_status(mhdp);
- if (mhdp->connector.dev) {
+ if (mhdp->connector_ptr) {
if (ret < 0)
schedule_work(&mhdp->modeset_retry_work);
else
diff --git a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.h b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.h
index bad2fc0c7306..a76775c76895 100644
--- a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.h
+++ b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.h
@@ -376,6 +376,7 @@ struct cdns_mhdp_device {
struct mutex link_mutex;
struct drm_connector connector;
+ struct drm_connector *connector_ptr;
struct drm_bridge bridge;
struct cdns_mhdp_link link;
diff --git a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-hdcp.c b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-hdcp.c
index 31832ba4017f..3c8532d7f784 100644
--- a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-hdcp.c
+++ b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-hdcp.c
@@ -394,7 +394,7 @@ static int _cdns_mhdp_hdcp_disable(struct cdns_mhdp_device *mhdp)
int ret;
dev_dbg(mhdp->dev, "[%s:%d] HDCP is being disabled...\n",
- mhdp->connector.name, mhdp->connector.base.id);
+ mhdp->connector_ptr->name, mhdp->connector_ptr->base.id);
ret = cdns_mhdp_hdcp_set_config(mhdp, 0, false);
@@ -436,6 +436,10 @@ static int cdns_mhdp_hdcp_check_link(struct cdns_mhdp_device *mhdp)
int ret = 0;
mutex_lock(&mhdp->hdcp.mutex);
+
+ if (!mhdp->connector_ptr)
+ goto out;
+
if (mhdp->hdcp.value == DRM_MODE_CONTENT_PROTECTION_UNDESIRED)
goto out;
@@ -445,7 +449,7 @@ static int cdns_mhdp_hdcp_check_link(struct cdns_mhdp_device *mhdp)
dev_err(mhdp->dev,
"[%s:%d] HDCP link failed, retrying authentication\n",
- mhdp->connector.name, mhdp->connector.base.id);
+ mhdp->connector_ptr->name, mhdp->connector_ptr->base.id);
ret = _cdns_mhdp_hdcp_disable(mhdp);
if (ret) {
@@ -487,13 +491,19 @@ static void cdns_mhdp_hdcp_prop_work(struct work_struct *work)
struct cdns_mhdp_device *mhdp = container_of(hdcp,
struct cdns_mhdp_device,
hdcp);
- struct drm_device *dev = mhdp->connector.dev;
+ struct drm_device *dev = NULL;
struct drm_connector_state *state;
+ if (mhdp->connector_ptr)
+ dev = mhdp->connector_ptr->dev;
+
+ if (!dev)
+ return;
+
drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
mutex_lock(&mhdp->hdcp.mutex);
if (mhdp->hdcp.value != DRM_MODE_CONTENT_PROTECTION_UNDESIRED) {
- state = mhdp->connector.state;
+ state = mhdp->connector_ptr->state;
state->content_protection = mhdp->hdcp.value;
}
mutex_unlock(&mhdp->hdcp.mutex);
diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
index 8e3d2d7060f8..4f2ab8a7b50f 100644
--- a/drivers/gpu/drm/drm_syncobj.c
+++ b/drivers/gpu/drm/drm_syncobj.c
@@ -712,16 +712,14 @@ static int drm_syncobj_fd_to_handle(struct drm_file *file_private,
int fd, u32 *handle)
{
struct drm_syncobj *syncobj;
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
int ret;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EINVAL;
- if (fd_file(f)->f_op != &drm_syncobj_file_fops) {
- fdput(f);
+ if (fd_file(f)->f_op != &drm_syncobj_file_fops)
return -EINVAL;
- }
/* take a reference to put in the idr */
syncobj = fd_file(f)->private_data;
@@ -739,7 +737,6 @@ static int drm_syncobj_fd_to_handle(struct drm_file *file_private,
} else
drm_syncobj_put(syncobj);
- fdput(f);
return ret;
}
diff --git a/drivers/gpu/drm/gma500/oaktrail_hdmi.c b/drivers/gpu/drm/gma500/oaktrail_hdmi.c
index f0ae675581d9..fa7a504c9358 100644
--- a/drivers/gpu/drm/gma500/oaktrail_hdmi.c
+++ b/drivers/gpu/drm/gma500/oaktrail_hdmi.c
@@ -579,6 +579,7 @@ static int oaktrail_hdmi_get_modes(struct drm_connector *connector)
} else {
edid = (struct edid *)raw_edid;
/* FIXME ? edid = drm_get_edid(connector, i2c_adap); */
+ i2c_put_adapter(i2c_adap);
}
if (edid) {
diff --git a/drivers/gpu/drm/gma500/oaktrail_lvds.c b/drivers/gpu/drm/gma500/oaktrail_lvds.c
index 72191d6f0d06..cdaa9b5b78cb 100644
--- a/drivers/gpu/drm/gma500/oaktrail_lvds.c
+++ b/drivers/gpu/drm/gma500/oaktrail_lvds.c
@@ -292,7 +292,7 @@ void oaktrail_lvds_init(struct drm_device *dev,
{
struct gma_encoder *gma_encoder;
struct gma_connector *gma_connector;
- struct gma_i2c_chan *ddc_bus;
+ struct gma_i2c_chan *ddc_bus = NULL;
struct drm_connector *connector;
struct drm_encoder *encoder;
struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
@@ -366,6 +366,8 @@ void oaktrail_lvds_init(struct drm_device *dev,
if (edid == NULL && dev_priv->lpc_gpio_base) {
ddc_bus = oaktrail_lvds_i2c_init(dev);
if (!IS_ERR(ddc_bus)) {
+ if (i2c_adap)
+ i2c_put_adapter(i2c_adap);
i2c_adap = &ddc_bus->base;
edid = drm_get_edid(connector, i2c_adap);
}
@@ -420,7 +422,10 @@ void oaktrail_lvds_init(struct drm_device *dev,
err_unlock:
mutex_unlock(&dev->mode_config.mutex);
- gma_i2c_destroy(to_gma_i2c_chan(connector->ddc));
+ if (!IS_ERR_OR_NULL(ddc_bus))
+ gma_i2c_destroy(ddc_bus);
+ else if (i2c_adap)
+ i2c_put_adapter(i2c_adap);
drm_encoder_cleanup(encoder);
err_connector_cleanup:
drm_connector_cleanup(connector);
diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c
index eb4952be7bcc..756bb0b1c83b 100644
--- a/drivers/gpu/drm/i915/display/intel_dp.c
+++ b/drivers/gpu/drm/i915/display/intel_dp.c
@@ -2726,8 +2726,13 @@ static void intel_dp_compute_vsc_colorimetry(const struct intel_crtc_state *crtc
drm_WARN_ON(&dev_priv->drm,
vsc->bpc == 6 && vsc->pixelformat != DP_PIXELFORMAT_RGB);
- /* all YCbCr are always limited range */
- vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA;
+ /* All YCbCr formats are always limited range. */
+ if (vsc->pixelformat == DP_PIXELFORMAT_RGB)
+ vsc->dynamic_range = crtc_state->limited_color_range ?
+ DP_DYNAMIC_RANGE_CTA : DP_DYNAMIC_RANGE_VESA;
+ else
+ vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA;
+
vsc->content_type = DP_CONTENT_TYPE_NOT_DEFINED;
}
diff --git a/drivers/gpu/drm/i915/display/skl_watermark.c b/drivers/gpu/drm/i915/display/skl_watermark.c
index 045c7cac166b..0c868be53f2f 100644
--- a/drivers/gpu/drm/i915/display/skl_watermark.c
+++ b/drivers/gpu/drm/i915/display/skl_watermark.c
@@ -3039,202 +3039,6 @@ static void skl_wm_get_hw_state(struct drm_i915_private *i915)
dbuf_state->enabled_slices = i915->display.dbuf.enabled_slices;
}
-static bool skl_dbuf_is_misconfigured(struct drm_i915_private *i915)
-{
- const struct intel_dbuf_state *dbuf_state =
- to_intel_dbuf_state(i915->display.dbuf.obj.state);
- struct skl_ddb_entry entries[I915_MAX_PIPES] = {};
- struct intel_crtc *crtc;
-
- for_each_intel_crtc(&i915->drm, crtc) {
- const struct intel_crtc_state *crtc_state =
- to_intel_crtc_state(crtc->base.state);
-
- entries[crtc->pipe] = crtc_state->wm.skl.ddb;
- }
-
- for_each_intel_crtc(&i915->drm, crtc) {
- const struct intel_crtc_state *crtc_state =
- to_intel_crtc_state(crtc->base.state);
- u8 slices;
-
- slices = skl_compute_dbuf_slices(crtc, dbuf_state->active_pipes,
- dbuf_state->joined_mbus);
- if (dbuf_state->slices[crtc->pipe] & ~slices)
- return true;
-
- if (skl_ddb_allocation_overlaps(&crtc_state->wm.skl.ddb, entries,
- I915_MAX_PIPES, crtc->pipe))
- return true;
- }
-
- return false;
-}
-
-static void skl_wm_sanitize(struct drm_i915_private *i915)
-{
- struct intel_crtc *crtc;
-
- /*
- * On TGL/RKL (at least) the BIOS likes to assign the planes
- * to the wrong DBUF slices. This will cause an infinite loop
- * in skl_commit_modeset_enables() as it can't find a way to
- * transition between the old bogus DBUF layout to the new
- * proper DBUF layout without DBUF allocation overlaps between
- * the planes (which cannot be allowed or else the hardware
- * may hang). If we detect a bogus DBUF layout just turn off
- * all the planes so that skl_commit_modeset_enables() can
- * simply ignore them.
- */
- if (!skl_dbuf_is_misconfigured(i915))
- return;
-
- drm_dbg_kms(&i915->drm, "BIOS has misprogrammed the DBUF, disabling all planes\n");
-
- for_each_intel_crtc(&i915->drm, crtc) {
- struct intel_plane *plane = to_intel_plane(crtc->base.primary);
- const struct intel_plane_state *plane_state =
- to_intel_plane_state(plane->base.state);
- struct intel_crtc_state *crtc_state =
- to_intel_crtc_state(crtc->base.state);
-
- if (plane_state->uapi.visible)
- intel_plane_disable_noatomic(crtc, plane);
-
- drm_WARN_ON(&i915->drm, crtc_state->active_planes != 0);
-
- memset(&crtc_state->wm.skl.ddb, 0, sizeof(crtc_state->wm.skl.ddb));
- }
-}
-
-static void skl_wm_get_hw_state_and_sanitize(struct drm_i915_private *i915)
-{
- skl_wm_get_hw_state(i915);
- skl_wm_sanitize(i915);
-}
-
-void intel_wm_state_verify(struct intel_atomic_state *state,
- struct intel_crtc *crtc)
-{
- struct drm_i915_private *i915 = to_i915(state->base.dev);
- const struct intel_crtc_state *new_crtc_state =
- intel_atomic_get_new_crtc_state(state, crtc);
- struct skl_hw_state {
- struct skl_ddb_entry ddb[I915_MAX_PLANES];
- struct skl_ddb_entry ddb_y[I915_MAX_PLANES];
- struct skl_pipe_wm wm;
- } *hw;
- const struct skl_pipe_wm *sw_wm = &new_crtc_state->wm.skl.optimal;
- struct intel_plane *plane;
- u8 hw_enabled_slices;
- int level;
-
- if (DISPLAY_VER(i915) < 9 || !new_crtc_state->hw.active)
- return;
-
- hw = kzalloc(sizeof(*hw), GFP_KERNEL);
- if (!hw)
- return;
-
- skl_pipe_wm_get_hw_state(crtc, &hw->wm);
-
- skl_pipe_ddb_get_hw_state(crtc, hw->ddb, hw->ddb_y);
-
- hw_enabled_slices = intel_enabled_dbuf_slices_mask(i915);
-
- if (DISPLAY_VER(i915) >= 11 &&
- hw_enabled_slices != i915->display.dbuf.enabled_slices)
- drm_err(&i915->drm,
- "mismatch in DBUF Slices (expected 0x%x, got 0x%x)\n",
- i915->display.dbuf.enabled_slices,
- hw_enabled_slices);
-
- for_each_intel_plane_on_crtc(&i915->drm, crtc, plane) {
- const struct skl_ddb_entry *hw_ddb_entry, *sw_ddb_entry;
- const struct skl_wm_level *hw_wm_level, *sw_wm_level;
-
- /* Watermarks */
- for (level = 0; level < i915->display.wm.num_levels; level++) {
- hw_wm_level = &hw->wm.planes[plane->id].wm[level];
- sw_wm_level = skl_plane_wm_level(sw_wm, plane->id, level);
-
- if (skl_wm_level_equals(hw_wm_level, sw_wm_level))
- continue;
-
- drm_err(&i915->drm,
- "[PLANE:%d:%s] mismatch in WM%d (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
- plane->base.base.id, plane->base.name, level,
- sw_wm_level->enable,
- sw_wm_level->blocks,
- sw_wm_level->lines,
- hw_wm_level->enable,
- hw_wm_level->blocks,
- hw_wm_level->lines);
- }
-
- hw_wm_level = &hw->wm.planes[plane->id].trans_wm;
- sw_wm_level = skl_plane_trans_wm(sw_wm, plane->id);
-
- if (!skl_wm_level_equals(hw_wm_level, sw_wm_level)) {
- drm_err(&i915->drm,
- "[PLANE:%d:%s] mismatch in trans WM (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
- plane->base.base.id, plane->base.name,
- sw_wm_level->enable,
- sw_wm_level->blocks,
- sw_wm_level->lines,
- hw_wm_level->enable,
- hw_wm_level->blocks,
- hw_wm_level->lines);
- }
-
- hw_wm_level = &hw->wm.planes[plane->id].sagv.wm0;
- sw_wm_level = &sw_wm->planes[plane->id].sagv.wm0;
-
- if (HAS_HW_SAGV_WM(i915) &&
- !skl_wm_level_equals(hw_wm_level, sw_wm_level)) {
- drm_err(&i915->drm,
- "[PLANE:%d:%s] mismatch in SAGV WM (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
- plane->base.base.id, plane->base.name,
- sw_wm_level->enable,
- sw_wm_level->blocks,
- sw_wm_level->lines,
- hw_wm_level->enable,
- hw_wm_level->blocks,
- hw_wm_level->lines);
- }
-
- hw_wm_level = &hw->wm.planes[plane->id].sagv.trans_wm;
- sw_wm_level = &sw_wm->planes[plane->id].sagv.trans_wm;
-
- if (HAS_HW_SAGV_WM(i915) &&
- !skl_wm_level_equals(hw_wm_level, sw_wm_level)) {
- drm_err(&i915->drm,
- "[PLANE:%d:%s] mismatch in SAGV trans WM (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
- plane->base.base.id, plane->base.name,
- sw_wm_level->enable,
- sw_wm_level->blocks,
- sw_wm_level->lines,
- hw_wm_level->enable,
- hw_wm_level->blocks,
- hw_wm_level->lines);
- }
-
- /* DDB */
- hw_ddb_entry = &hw->ddb[PLANE_CURSOR];
- sw_ddb_entry = &new_crtc_state->wm.skl.plane_ddb[PLANE_CURSOR];
-
- if (!skl_ddb_entry_equal(hw_ddb_entry, sw_ddb_entry)) {
- drm_err(&i915->drm,
- "[PLANE:%d:%s] mismatch in DDB (expected (%u,%u), found (%u,%u))\n",
- plane->base.base.id, plane->base.name,
- sw_ddb_entry->start, sw_ddb_entry->end,
- hw_ddb_entry->start, hw_ddb_entry->end);
- }
- }
-
- kfree(hw);
-}
-
bool skl_watermark_ipc_enabled(struct drm_i915_private *i915)
{
return i915->display.wm.ipc_enabled;
@@ -3390,20 +3194,6 @@ static void skl_setup_wm_latency(struct drm_i915_private *i915)
intel_print_wm_latency(i915, "Gen9 Plane", i915->display.wm.skl_latency);
}
-static const struct intel_wm_funcs skl_wm_funcs = {
- .compute_global_watermarks = skl_compute_wm,
- .get_hw_state = skl_wm_get_hw_state_and_sanitize,
-};
-
-void skl_wm_init(struct drm_i915_private *i915)
-{
- intel_sagv_init(i915);
-
- skl_setup_wm_latency(i915);
-
- i915->display.funcs.wm = &skl_wm_funcs;
-}
-
static struct intel_global_state *intel_dbuf_duplicate_state(struct intel_global_obj *obj)
{
struct intel_dbuf_state *dbuf_state;
@@ -3747,6 +3537,216 @@ void intel_dbuf_post_plane_update(struct intel_atomic_state *state)
gen9_dbuf_slices_update(i915, new_slices);
}
+static bool skl_dbuf_is_misconfigured(struct drm_i915_private *i915)
+{
+ const struct intel_dbuf_state *dbuf_state =
+ to_intel_dbuf_state(i915->display.dbuf.obj.state);
+ struct skl_ddb_entry entries[I915_MAX_PIPES] = {};
+ struct intel_crtc *crtc;
+
+ for_each_intel_crtc(&i915->drm, crtc) {
+ const struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+
+ entries[crtc->pipe] = crtc_state->wm.skl.ddb;
+ }
+
+ for_each_intel_crtc(&i915->drm, crtc) {
+ const struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+ u8 slices;
+
+ slices = skl_compute_dbuf_slices(crtc, dbuf_state->active_pipes,
+ dbuf_state->joined_mbus);
+ if (dbuf_state->slices[crtc->pipe] & ~slices)
+ return true;
+
+ if (skl_ddb_allocation_overlaps(&crtc_state->wm.skl.ddb, entries,
+ I915_MAX_PIPES, crtc->pipe))
+ return true;
+ }
+
+ return false;
+}
+
+static void skl_wm_sanitize(struct drm_i915_private *i915)
+{
+ struct intel_crtc *crtc;
+
+ /*
+ * On TGL/RKL (at least) the BIOS likes to assign the planes
+ * to the wrong DBUF slices. This will cause an infinite loop
+ * in skl_commit_modeset_enables() as it can't find a way to
+ * transition between the old bogus DBUF layout to the new
+ * proper DBUF layout without DBUF allocation overlaps between
+ * the planes (which cannot be allowed or else the hardware
+ * may hang). If we detect a bogus DBUF layout just turn off
+ * all the planes so that skl_commit_modeset_enables() can
+ * simply ignore them.
+ */
+ if (!skl_dbuf_is_misconfigured(i915))
+ return;
+
+ drm_dbg_kms(&i915->drm, "BIOS has misprogrammed the DBUF, disabling all planes\n");
+
+ for_each_intel_crtc(&i915->drm, crtc) {
+ struct intel_plane *plane = to_intel_plane(crtc->base.primary);
+ const struct intel_plane_state *plane_state =
+ to_intel_plane_state(plane->base.state);
+ struct intel_crtc_state *crtc_state =
+ to_intel_crtc_state(crtc->base.state);
+
+ if (plane_state->uapi.visible)
+ intel_plane_disable_noatomic(crtc, plane);
+
+ drm_WARN_ON(&i915->drm, crtc_state->active_planes != 0);
+
+ memset(&crtc_state->wm.skl.ddb, 0, sizeof(crtc_state->wm.skl.ddb));
+ }
+}
+
+static void skl_wm_get_hw_state_and_sanitize(struct drm_i915_private *i915)
+{
+ skl_wm_get_hw_state(i915);
+ skl_wm_sanitize(i915);
+}
+
+void intel_wm_state_verify(struct intel_atomic_state *state,
+ struct intel_crtc *crtc)
+{
+ struct drm_i915_private *i915 = to_i915(state->base.dev);
+ const struct intel_crtc_state *new_crtc_state =
+ intel_atomic_get_new_crtc_state(state, crtc);
+ struct skl_hw_state {
+ struct skl_ddb_entry ddb[I915_MAX_PLANES];
+ struct skl_ddb_entry ddb_y[I915_MAX_PLANES];
+ struct skl_pipe_wm wm;
+ } *hw;
+ const struct skl_pipe_wm *sw_wm = &new_crtc_state->wm.skl.optimal;
+ struct intel_plane *plane;
+ u8 hw_enabled_slices;
+ int level;
+
+ if (DISPLAY_VER(i915) < 9 || !new_crtc_state->hw.active)
+ return;
+
+ hw = kzalloc(sizeof(*hw), GFP_KERNEL);
+ if (!hw)
+ return;
+
+ skl_pipe_wm_get_hw_state(crtc, &hw->wm);
+
+ skl_pipe_ddb_get_hw_state(crtc, hw->ddb, hw->ddb_y);
+
+ hw_enabled_slices = intel_enabled_dbuf_slices_mask(i915);
+
+ if (DISPLAY_VER(i915) >= 11 &&
+ hw_enabled_slices != i915->display.dbuf.enabled_slices)
+ drm_err(&i915->drm,
+ "mismatch in DBUF Slices (expected 0x%x, got 0x%x)\n",
+ i915->display.dbuf.enabled_slices,
+ hw_enabled_slices);
+
+ for_each_intel_plane_on_crtc(&i915->drm, crtc, plane) {
+ const struct skl_ddb_entry *hw_ddb_entry, *sw_ddb_entry;
+ const struct skl_wm_level *hw_wm_level, *sw_wm_level;
+
+ /* Watermarks */
+ for (level = 0; level < i915->display.wm.num_levels; level++) {
+ hw_wm_level = &hw->wm.planes[plane->id].wm[level];
+ sw_wm_level = skl_plane_wm_level(sw_wm, plane->id, level);
+
+ if (skl_wm_level_equals(hw_wm_level, sw_wm_level))
+ continue;
+
+ drm_err(&i915->drm,
+ "[PLANE:%d:%s] mismatch in WM%d (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
+ plane->base.base.id, plane->base.name, level,
+ sw_wm_level->enable,
+ sw_wm_level->blocks,
+ sw_wm_level->lines,
+ hw_wm_level->enable,
+ hw_wm_level->blocks,
+ hw_wm_level->lines);
+ }
+
+ hw_wm_level = &hw->wm.planes[plane->id].trans_wm;
+ sw_wm_level = skl_plane_trans_wm(sw_wm, plane->id);
+
+ if (!skl_wm_level_equals(hw_wm_level, sw_wm_level)) {
+ drm_err(&i915->drm,
+ "[PLANE:%d:%s] mismatch in trans WM (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
+ plane->base.base.id, plane->base.name,
+ sw_wm_level->enable,
+ sw_wm_level->blocks,
+ sw_wm_level->lines,
+ hw_wm_level->enable,
+ hw_wm_level->blocks,
+ hw_wm_level->lines);
+ }
+
+ hw_wm_level = &hw->wm.planes[plane->id].sagv.wm0;
+ sw_wm_level = &sw_wm->planes[plane->id].sagv.wm0;
+
+ if (HAS_HW_SAGV_WM(i915) &&
+ !skl_wm_level_equals(hw_wm_level, sw_wm_level)) {
+ drm_err(&i915->drm,
+ "[PLANE:%d:%s] mismatch in SAGV WM (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
+ plane->base.base.id, plane->base.name,
+ sw_wm_level->enable,
+ sw_wm_level->blocks,
+ sw_wm_level->lines,
+ hw_wm_level->enable,
+ hw_wm_level->blocks,
+ hw_wm_level->lines);
+ }
+
+ hw_wm_level = &hw->wm.planes[plane->id].sagv.trans_wm;
+ sw_wm_level = &sw_wm->planes[plane->id].sagv.trans_wm;
+
+ if (HAS_HW_SAGV_WM(i915) &&
+ !skl_wm_level_equals(hw_wm_level, sw_wm_level)) {
+ drm_err(&i915->drm,
+ "[PLANE:%d:%s] mismatch in SAGV trans WM (expected e=%d b=%u l=%u, got e=%d b=%u l=%u)\n",
+ plane->base.base.id, plane->base.name,
+ sw_wm_level->enable,
+ sw_wm_level->blocks,
+ sw_wm_level->lines,
+ hw_wm_level->enable,
+ hw_wm_level->blocks,
+ hw_wm_level->lines);
+ }
+
+ /* DDB */
+ hw_ddb_entry = &hw->ddb[plane->id];
+ sw_ddb_entry = &new_crtc_state->wm.skl.plane_ddb[plane->id];
+
+ if (!skl_ddb_entry_equal(hw_ddb_entry, sw_ddb_entry)) {
+ drm_err(&i915->drm,
+ "[PLANE:%d:%s] mismatch in DDB (expected (%u,%u), found (%u,%u))\n",
+ plane->base.base.id, plane->base.name,
+ sw_ddb_entry->start, sw_ddb_entry->end,
+ hw_ddb_entry->start, hw_ddb_entry->end);
+ }
+ }
+
+ kfree(hw);
+}
+
+static const struct intel_wm_funcs skl_wm_funcs = {
+ .compute_global_watermarks = skl_compute_wm,
+ .get_hw_state = skl_wm_get_hw_state_and_sanitize,
+};
+
+void skl_wm_init(struct drm_i915_private *i915)
+{
+ intel_sagv_init(i915);
+
+ skl_setup_wm_latency(i915);
+
+ i915->display.funcs.wm = &skl_wm_funcs;
+}
+
static int skl_watermark_ipc_status_show(struct seq_file *m, void *data)
{
struct drm_i915_private *i915 = m->private;
diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/gt/intel_reset.c
index 8f1ea95471ef..523b514c6a0d 100644
--- a/drivers/gpu/drm/i915/gt/intel_reset.c
+++ b/drivers/gpu/drm/i915/gt/intel_reset.c
@@ -135,7 +135,8 @@ void __i915_request_reset(struct i915_request *rq, bool guilty)
rcu_read_lock(); /* protect the GEM context */
if (guilty) {
i915_request_set_error_once(rq, -EIO);
- __i915_request_skip(rq);
+ if (!i915_request_signaled(rq))
+ __i915_request_skip(rq);
banned = mark_guilty(rq);
} else {
i915_request_set_error_once(rq, -EAGAIN);
diff --git a/drivers/gpu/drm/imagination/pvr_rogue_fwif.h b/drivers/gpu/drm/imagination/pvr_rogue_fwif.h
index 172886be4c82..5d590c4c2566 100644
--- a/drivers/gpu/drm/imagination/pvr_rogue_fwif.h
+++ b/drivers/gpu/drm/imagination/pvr_rogue_fwif.h
@@ -1347,8 +1347,12 @@ struct rogue_fwif_fwccb_cmd_freelists_reconstruction_data {
struct rogue_fwif_fwccb_cmd_context_reset_data {
/* Context affected by the reset */
u32 server_common_context_id;
- /* Reason for reset */
- enum rogue_context_reset_reason reset_reason;
+ /*
+ * Reason for reset
+ * The valid values for reset_reason are the ones from
+ * enum rogue_context_reset_reason
+ */
+ u32 reset_reason;
/* Data Master affected by the reset */
u32 dm;
/* Job ref running at the time of reset */
diff --git a/drivers/gpu/drm/imagination/pvr_rogue_fwif_shared.h b/drivers/gpu/drm/imagination/pvr_rogue_fwif_shared.h
index 6c09c15bf9bd..f95acd5a1f8e 100644
--- a/drivers/gpu/drm/imagination/pvr_rogue_fwif_shared.h
+++ b/drivers/gpu/drm/imagination/pvr_rogue_fwif_shared.h
@@ -249,7 +249,11 @@ enum rogue_context_reset_reason {
};
struct rogue_context_reset_reason_data {
- enum rogue_context_reset_reason reset_reason;
+ /*
+ * The valid values for reset_reason are the ones from
+ * enum rogue_context_reset_reason
+ */
+ u32 reset_reason;
u32 reset_ext_job_ref;
};
diff --git a/drivers/gpu/drm/loongson/lsdc_drv.c b/drivers/gpu/drm/loongson/lsdc_drv.c
index adc7344d2f80..49fd08e0d8a1 100644
--- a/drivers/gpu/drm/loongson/lsdc_drv.c
+++ b/drivers/gpu/drm/loongson/lsdc_drv.c
@@ -291,7 +291,7 @@ static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
vga_client_register(pdev, lsdc_vga_set_decode);
- drm_kms_helper_poll_init(ddev);
+ drmm_kms_helper_poll_init(ddev);
if (loongson_vblank) {
ret = drm_vblank_init(ddev, descp->num_of_crtc);
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
index 5d7d2f5a2a1f..dac88af9b3d0 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c
@@ -346,7 +346,7 @@ static void a6xx_get_debugbus_blocks(struct msm_gpu *gpu,
sizeof(*a6xx_state->debugbus));
if (a6xx_state->debugbus) {
- int i;
+ int i, j;
for (i = 0; i < ARRAY_SIZE(a6xx_debugbus_blocks); i++)
a6xx_get_debugbus_block(gpu,
@@ -354,8 +354,6 @@ static void a6xx_get_debugbus_blocks(struct msm_gpu *gpu,
&a6xx_debugbus_blocks[i],
&a6xx_state->debugbus[i]);
- a6xx_state->nr_debugbus = ARRAY_SIZE(a6xx_debugbus_blocks);
-
/*
* GBIF has same debugbus as of other GPU blocks, fall back to
* default path if GPU uses GBIF, also GBIF uses exactly same
@@ -366,17 +364,19 @@ static void a6xx_get_debugbus_blocks(struct msm_gpu *gpu,
&a6xx_gbif_debugbus_block,
&a6xx_state->debugbus[i]);
- a6xx_state->nr_debugbus += 1;
+ i++;
}
if (adreno_is_a650_family(to_adreno_gpu(gpu))) {
- for (i = 0; i < ARRAY_SIZE(a650_debugbus_blocks); i++)
+ for (j = 0; j < ARRAY_SIZE(a650_debugbus_blocks); i++, j++)
a6xx_get_debugbus_block(gpu,
a6xx_state,
- &a650_debugbus_blocks[i],
+ &a650_debugbus_blocks[j],
&a6xx_state->debugbus[i]);
}
+
+ a6xx_state->nr_debugbus = i;
}
}
@@ -996,7 +996,7 @@ static void a6xx_get_crashdumper_hlsq_registers(struct msm_gpu *gpu,
u64 out = dumper->iova + A6XX_CD_DATA_OFFSET;
int i, regcount = 0;
- in += CRASHDUMP_WRITE(in, REG_A6XX_HLSQ_DBG_READ_SEL, regs->val1);
+ in += CRASHDUMP_WRITE(in, REG_A6XX_HLSQ_DBG_READ_SEL, (regs->val1 & 0xff) << 8);
for (i = 0; i < regs->count; i += 2) {
u32 count = RANGE(regs->registers, i);
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c
index cdb3f6e74d3e..b858e740ac7e 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c
@@ -31,7 +31,7 @@ static int a6xx_hfi_queue_read(struct a6xx_gmu *gmu,
struct a6xx_hfi_queue_header *header = queue->header;
u32 i, hdr, index = header->read_index;
- if (header->read_index == header->write_index) {
+ if (header->read_index == READ_ONCE(header->write_index)) {
header->rx_request = 1;
return 0;
}
@@ -59,7 +59,10 @@ static int a6xx_hfi_queue_read(struct a6xx_gmu *gmu,
if (!gmu->legacy)
index = ALIGN(index, 4) % header->size;
- header->read_index = index;
+ /* Ensure all memory operations are complete before updating the read index */
+ dma_mb();
+
+ WRITE_ONCE(header->read_index, index);
return HFI_HEADER_SIZE(hdr);
}
@@ -71,7 +74,7 @@ static int a6xx_hfi_queue_write(struct a6xx_gmu *gmu,
spin_lock(&queue->lock);
- space = CIRC_SPACE(header->write_index, header->read_index,
+ space = CIRC_SPACE(header->write_index, READ_ONCE(header->read_index),
header->size);
if (space < dwords) {
header->dropped++;
@@ -92,7 +95,10 @@ static int a6xx_hfi_queue_write(struct a6xx_gmu *gmu,
queue->data[index] = 0xfafafafa;
}
- header->write_index = index;
+ /* Ensure all memory operations are complete before updating the write index */
+ dma_mb();
+
+ WRITE_ONCE(header->write_index, index);
spin_unlock(&queue->lock);
gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 0x01);
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
index 9bcae53c4f45..edc35638ce76 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c
@@ -1401,8 +1401,6 @@ static int __maybe_unused dpu_runtime_suspend(struct device *dev)
struct msm_drm_private *priv = platform_get_drvdata(pdev);
struct dpu_kms *dpu_kms = to_dpu_kms(priv->kms);
- /* Drop the performance state vote */
- dev_pm_opp_set_rate(dev, 0);
clk_bulk_disable_unprepare(dpu_kms->num_clocks, dpu_kms->clocks);
for (i = 0; i < dpu_kms->num_paths; i++)
diff --git a/drivers/gpu/drm/msm/dsi/dsi_cfg.c b/drivers/gpu/drm/msm/dsi/dsi_cfg.c
index 10ba7d153d1c..8bf3266d1383 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_cfg.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_cfg.c
@@ -278,10 +278,10 @@ static const struct msm_dsi_cfg_handler dsi_cfg_handlers[] = {
&msm8996_dsi_cfg, &msm_dsi_6g_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V1_4_2,
&msm8976_dsi_cfg, &msm_dsi_6g_host_ops},
+ {MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_0_0,
+ &msm8998_dsi_cfg, &msm_dsi_6g_v2_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_1_0,
&sdm660_dsi_cfg, &msm_dsi_6g_v2_host_ops},
- {MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_2_0,
- &msm8998_dsi_cfg, &msm_dsi_6g_v2_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_2_1,
&sdm845_dsi_cfg, &msm_dsi_6g_v2_host_ops},
{MSM_DSI_VER_MAJOR_6G, MSM_DSI_6G_VER_MINOR_V2_3_0,
diff --git a/drivers/gpu/drm/msm/dsi/dsi_cfg.h b/drivers/gpu/drm/msm/dsi/dsi_cfg.h
index 4c9b4b37681b..2c9ed626f790 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_cfg.h
+++ b/drivers/gpu/drm/msm/dsi/dsi_cfg.h
@@ -19,8 +19,8 @@
#define MSM_DSI_6G_VER_MINOR_V1_3_1 0x10030001
#define MSM_DSI_6G_VER_MINOR_V1_4_1 0x10040001
#define MSM_DSI_6G_VER_MINOR_V1_4_2 0x10040002
+#define MSM_DSI_6G_VER_MINOR_V2_0_0 0x20000000
#define MSM_DSI_6G_VER_MINOR_V2_1_0 0x20010000
-#define MSM_DSI_6G_VER_MINOR_V2_2_0 0x20000000
#define MSM_DSI_6G_VER_MINOR_V2_2_1 0x20020001
#define MSM_DSI_6G_VER_MINOR_V2_3_0 0x20030000
#define MSM_DSI_6G_VER_MINOR_V2_4_0 0x20040000
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index 0c360e790329..6f538c578f74 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -519,6 +519,7 @@ void dsi_link_clk_disable_v2(struct msm_dsi_host *msm_host)
* dsi_adjust_pclk_for_compression() - Adjust the pclk rate for compression case
* @mode: The selected mode for the DSI output
* @dsc: DRM DSC configuration for this DSI output
+ * @is_bonded_dsi: True if two DSI controllers are bonded
*
* Adjust the pclk rate by calculating a new hdisplay proportional to
* the compression ratio such that:
@@ -984,8 +985,9 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi)
/*
* DPU sends 3 bytes per pclk cycle to DSI. If widebus is
* enabled, MDP always sends out 48-bit compressed data per
- * pclk and on average, DSI consumes an amount of compressed
- * data equivalent to the uncompressed pixel depth per pclk.
+ * pclk and on average, for video mode, DSI consumes only an
+ * amount of compressed data equivalent to the uncompressed
+ * pixel depth per pclk.
*
* Calculate the number of pclks needed to transmit one line of
* the compressed data.
@@ -997,10 +999,14 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi)
* unused anyway.
*/
h_total -= hdisplay;
- if (wide_bus_enabled)
- bits_per_pclk = mipi_dsi_pixel_format_to_bpp(msm_host->format);
- else
+ if (wide_bus_enabled) {
+ if (msm_host->mode_flags & MIPI_DSI_MODE_VIDEO)
+ bits_per_pclk = dsc->bits_per_component * 3;
+ else
+ bits_per_pclk = 48;
+ } else {
bits_per_pclk = 24;
+ }
hdisplay = DIV_ROUND_UP(msm_dsc_get_bytes_per_line(msm_host->dsc) * 8, bits_per_pclk);
diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c
index 07ca4ddfe4e3..89c0ea8ddeaa 100644
--- a/drivers/gpu/drm/msm/msm_gem_shrinker.c
+++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c
@@ -26,9 +26,8 @@ static bool can_swap(void)
static bool can_block(struct shrink_control *sc)
{
- if (!(sc->gfp_mask & __GFP_DIRECT_RECLAIM))
- return false;
- return current_is_kswapd() || (sc->gfp_mask & __GFP_RECLAIM);
+ return (sc->gfp_mask & __GFP_DIRECT_RECLAIM) ||
+ (current_is_kswapd() && (sc->gfp_mask & __GFP_KSWAPD_RECLAIM));
}
static unsigned long
diff --git a/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c b/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c
index 729cbb0d8403..bc3e13b5a1f7 100644
--- a/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c
+++ b/drivers/gpu/drm/panel/panel-sharp-ls043t1le01.c
@@ -212,6 +212,7 @@ static int sharp_nt_panel_add(struct sharp_nt_panel *sharp_nt)
drm_panel_init(&sharp_nt->base, &sharp_nt->dsi->dev,
&sharp_nt_panel_funcs, DRM_MODE_CONNECTOR_DSI);
+ sharp_nt->base.prepare_prev_first = true;
ret = drm_panel_of_backlight(&sharp_nt->base);
if (ret)
diff --git a/drivers/gpu/drm/panel/panel-simple.c b/drivers/gpu/drm/panel/panel-simple.c
index 32d876f6684e..a0e2da295376 100644
--- a/drivers/gpu/drm/panel/panel-simple.c
+++ b/drivers/gpu/drm/panel/panel-simple.c
@@ -1260,7 +1260,7 @@ static const struct panel_desc auo_g190ean01 = {
.height = 301,
},
.delay = {
- .prepare = 50,
+ .prepare = 30,
.enable = 200,
.disable = 110,
.unprepare = 1000,
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 671eed4ad890..b6e5442f8728 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -325,6 +325,8 @@ panfrost_ioctl_wait_bo(struct drm_device *dev, void *data,
true, timeout);
if (!ret)
ret = timeout ? -ETIMEDOUT : -EBUSY;
+ else if (ret > 0)
+ ret = 0;
drm_gem_object_put(gem_obj);
diff --git a/drivers/gpu/drm/sun4i/sun4i_backend.c b/drivers/gpu/drm/sun4i/sun4i_backend.c
index e89eb96d3131..329e2a2384e8 100644
--- a/drivers/gpu/drm/sun4i/sun4i_backend.c
+++ b/drivers/gpu/drm/sun4i/sun4i_backend.c
@@ -490,6 +490,9 @@ static int sun4i_backend_atomic_check(struct sunxi_engine *engine,
drm_for_each_plane_mask(plane, drm, crtc_state->plane_mask) {
struct drm_plane_state *plane_state =
drm_atomic_get_plane_state(state, plane);
+ if (IS_ERR(plane_state))
+ return PTR_ERR(plane_state);
+
struct sun4i_layer_state *layer_state =
state_to_sun4i_layer_state(plane_state);
struct drm_framebuffer *fb = plane_state->fb;
@@ -877,7 +880,8 @@ static int sun4i_backend_bind(struct device *dev, struct device *master,
&sun4i_backend_regmap_config);
if (IS_ERR(backend->engine.regs)) {
dev_err(dev, "Couldn't create the backend regmap\n");
- return PTR_ERR(backend->engine.regs);
+ ret = PTR_ERR(backend->engine.regs);
+ goto err_disable_ram_clk;
}
list_add_tail(&backend->engine.list, &drv->engine_list);
diff --git a/drivers/gpu/drm/tiny/ofdrm.c b/drivers/gpu/drm/tiny/ofdrm.c
index 35996f7eedac..cc90e0035d03 100644
--- a/drivers/gpu/drm/tiny/ofdrm.c
+++ b/drivers/gpu/drm/tiny/ofdrm.c
@@ -350,6 +350,7 @@ static void ofdrm_pci_release(void *data)
struct pci_dev *pcidev = data;
pci_disable_device(pcidev);
+ pci_dev_put(pcidev);
}
static int ofdrm_device_init_pci(struct ofdrm_device *odev)
@@ -375,6 +376,7 @@ static int ofdrm_device_init_pci(struct ofdrm_device *odev)
if (ret) {
drm_err(dev, "pci_enable_device(%s) failed: %d\n",
dev_name(&pcidev->dev), ret);
+ pci_dev_put(pcidev);
return ret;
}
ret = devm_add_action_or_reset(&pdev->dev, ofdrm_pci_release, pcidev);
diff --git a/drivers/gpu/drm/v3d/v3d_drv.c b/drivers/gpu/drm/v3d/v3d_drv.c
index f45fdd7d542f..e0272acad2a8 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.c
+++ b/drivers/gpu/drm/v3d/v3d_drv.c
@@ -109,7 +109,7 @@ v3d_open(struct drm_device *dev, struct drm_file *file)
struct v3d_dev *v3d = to_v3d_dev(dev);
struct v3d_file_priv *v3d_priv;
struct drm_gpu_scheduler *sched;
- int i;
+ int i, ret;
v3d_priv = kzalloc(sizeof(*v3d_priv), GFP_KERNEL);
if (!v3d_priv)
@@ -119,9 +119,11 @@ v3d_open(struct drm_device *dev, struct drm_file *file)
for (i = 0; i < V3D_MAX_QUEUES; i++) {
sched = &v3d->queue[i].sched;
- drm_sched_entity_init(&v3d_priv->sched_entity[i],
- DRM_SCHED_PRIORITY_NORMAL, &sched,
- 1, NULL);
+ ret = drm_sched_entity_init(&v3d_priv->sched_entity[i],
+ DRM_SCHED_PRIORITY_NORMAL, &sched,
+ 1, NULL);
+ if (ret)
+ goto err_sched;
memset(&v3d_priv->stats[i], 0, sizeof(v3d_priv->stats[i]));
seqcount_init(&v3d_priv->stats[i].lock);
@@ -131,6 +133,12 @@ v3d_open(struct drm_device *dev, struct drm_file *file)
file->driver_priv = v3d_priv;
return 0;
+
+err_sched:
+ for (i--; i >= 0; i--)
+ drm_sched_entity_destroy(&v3d_priv->sched_entity[i]);
+ kfree(v3d_priv);
+ return ret;
}
static void
diff --git a/drivers/gpu/drm/v3d/v3d_submit.c b/drivers/gpu/drm/v3d/v3d_submit.c
index d607aa9c4ec2..ddc20191a1ce 100644
--- a/drivers/gpu/drm/v3d/v3d_submit.c
+++ b/drivers/gpu/drm/v3d/v3d_submit.c
@@ -389,6 +389,11 @@ v3d_get_multisync_submit_deps(struct drm_file *file_priv,
if (multisync.pad)
return -EINVAL;
+ if (!multisync.in_sync_count && !multisync.out_sync_count) {
+ DRM_DEBUG("Empty multisync extension\n");
+ return -EINVAL;
+ }
+
ret = v3d_get_multisync_post_deps(file_priv, se, multisync.out_sync_count,
multisync.out_syncs);
if (ret)
diff --git a/drivers/gpu/drm/xe/xe_dma_buf.c b/drivers/gpu/drm/xe/xe_dma_buf.c
index cedb5c911238..c422f7d60115 100644
--- a/drivers/gpu/drm/xe/xe_dma_buf.c
+++ b/drivers/gpu/drm/xe/xe_dma_buf.c
@@ -278,15 +278,25 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev,
}
}
- /*
- * Don't publish the bo until we have a valid attachment, and a
- * valid attachment needs the bo address. So pre-create a bo before
- * creating the attachment and publish.
- */
bo = xe_bo_alloc();
if (IS_ERR(bo))
return ERR_CAST(bo);
+ /*
+ * xe_dma_buf_init_obj() takes ownership of the raw bo, so do not touch
+ * on fail, since it will already take care of cleanup. On success we
+ * still need to drop the ref, if something later fails.
+ *
+ * In addition this needs to happen before the attach, since
+ * it will create a new attachment for this, and add it to the list of
+ * attachments, at which point it is globally visible, and at any point
+ * the export side can call into on invalidate_mappings callback, which
+ * require a working object.
+ */
+ obj = xe_dma_buf_init_obj(dev, bo, dma_buf);
+ if (IS_ERR(obj))
+ return obj;
+
attach_ops = &xe_dma_buf_attach_ops;
#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
if (test)
@@ -299,21 +309,12 @@ struct drm_gem_object *xe_gem_prime_import(struct drm_device *dev,
goto out_err;
}
- /*
- * xe_dma_buf_init_obj() takes ownership of bo on both success
- * and failure, so we must not touch bo after this call.
- */
- obj = xe_dma_buf_init_obj(dev, bo, dma_buf);
- if (IS_ERR(obj)) {
- dma_buf_detach(dma_buf, attach);
- return obj;
- }
get_dma_buf(dma_buf);
obj->import_attach = attach;
return obj;
out_err:
- xe_bo_free(bo);
+ xe_bo_put(bo);
return obj;
}
diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c
index 268cd3123be9..e6c3074d0a78 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue.c
+++ b/drivers/gpu/drm/xe/xe_exec_queue.c
@@ -638,7 +638,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,
if (q->vm && q->hwe->hw_engine_group) {
err = xe_hw_engine_group_add_exec_queue(q->hwe->hw_engine_group, q);
if (err)
- goto put_exec_queue;
+ goto kill_exec_queue;
}
}
@@ -647,12 +647,15 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,
/* user id alloc must always be last in ioctl to prevent UAF */
err = xa_alloc(&xef->exec_queue.xa, &id, q, xa_limit_32b, GFP_KERNEL);
if (err)
- goto kill_exec_queue;
+ goto del_hw_engine_group;
args->exec_queue_id = id;
return 0;
+del_hw_engine_group:
+ if (q->vm && q->hwe && q->hwe->hw_engine_group)
+ xe_hw_engine_group_del_exec_queue(q->hwe->hw_engine_group, q);
kill_exec_queue:
xe_exec_queue_kill(q);
put_exec_queue:
diff --git a/drivers/gpu/drm/xe/xe_gsc.c b/drivers/gpu/drm/xe/xe_gsc.c
index feb680d127e6..efc480d34c9d 100644
--- a/drivers/gpu/drm/xe/xe_gsc.c
+++ b/drivers/gpu/drm/xe/xe_gsc.c
@@ -163,7 +163,7 @@ static int query_compatibility_version(struct xe_gsc *gsc)
&rd_offset);
if (err) {
xe_gt_err(gt, "HuC: invalid GSC reply for version query (err=%d)\n", err);
- return err;
+ goto out_bo;
}
compat->major = version_query_rd(xe, &bo->vmap, rd_offset, proj_major);
diff --git a/drivers/gpu/drm/xe/xe_reg_whitelist.c b/drivers/gpu/drm/xe/xe_reg_whitelist.c
index 3996934974fa..3de0a867149d 100644
--- a/drivers/gpu/drm/xe/xe_reg_whitelist.c
+++ b/drivers/gpu/drm/xe/xe_reg_whitelist.c
@@ -137,7 +137,7 @@ void xe_reg_whitelist_print_entry(struct drm_printer *p, unsigned int indent,
}
range_start = reg & REG_GENMASK(25, range_bit);
- range_end = range_start | REG_GENMASK(range_bit, 0);
+ range_end = range_start | REG_GENMASK(range_bit - 1, 0);
switch (val & RING_FORCE_TO_NONPRIV_ACCESS_MASK) {
case RING_FORCE_TO_NONPRIV_ACCESS_RW:
diff --git a/drivers/hid/hid-asus.c b/drivers/hid/hid-asus.c
index b2788bb0477f..552a049c7d14 100644
--- a/drivers/hid/hid-asus.c
+++ b/drivers/hid/hid-asus.c
@@ -1092,7 +1092,8 @@ static int asus_start_multitouch(struct hid_device *hdev)
return 0;
}
-static int __maybe_unused asus_resume(struct hid_device *hdev) {
+static int __maybe_unused asus_resume(struct hid_device *hdev)
+{
struct asus_drvdata *drvdata = hid_get_drvdata(hdev);
int ret = 0;
@@ -1219,22 +1220,17 @@ static int asus_probe(struct hid_device *hdev, const struct hid_device_id *id)
* were freed during registration due to no usages being mapped,
* leaving drvdata->input pointing to freed memory.
*/
- if (!drvdata->input || !(hdev->claimed & HID_CLAIMED_INPUT)) {
- hid_err(hdev, "Asus input not registered\n");
- ret = -ENOMEM;
- goto err_stop_hw;
- }
-
- if (drvdata->tp) {
- drvdata->input->name = "Asus TouchPad";
- } else {
- drvdata->input->name = "Asus Keyboard";
- }
+ if (drvdata->input && (hdev->claimed & HID_CLAIMED_INPUT)) {
+ if (drvdata->tp)
+ drvdata->input->name = "Asus TouchPad";
+ else
+ drvdata->input->name = "Asus Keyboard";
- if (drvdata->tp) {
- ret = asus_start_multitouch(hdev);
- if (ret)
- goto err_stop_hw;
+ if (drvdata->tp) {
+ ret = asus_start_multitouch(hdev);
+ if (ret)
+ goto err_stop_hw;
+ }
}
return 0;
diff --git a/drivers/hid/usbhid/hid-core.c b/drivers/hid/usbhid/hid-core.c
index a2c5a31931f6..f14b46ce00cb 100644
--- a/drivers/hid/usbhid/hid-core.c
+++ b/drivers/hid/usbhid/hid-core.c
@@ -1552,7 +1552,7 @@ static int hid_post_reset(struct usb_interface *intf)
* configuration descriptors passed, we already know that
* the size of the HID report descriptor has not changed.
*/
- rdesc = kmalloc(hid->dev_rsize, GFP_KERNEL);
+ rdesc = kmalloc(hid->dev_rsize, GFP_NOIO);
if (!rdesc)
return -ENOMEM;
diff --git a/drivers/hte/Kconfig b/drivers/hte/Kconfig
index 641af722b555..f57bad67deef 100644
--- a/drivers/hte/Kconfig
+++ b/drivers/hte/Kconfig
@@ -16,13 +16,13 @@ if HTE
config HTE_TEGRA194
tristate "NVIDIA Tegra194 HTE Support"
- depends on (ARCH_TEGRA_194_SOC || COMPILE_TEST)
+ depends on (ARCH_TEGRA || COMPILE_TEST)
depends on GPIOLIB
help
Enable this option for integrated hardware timestamping engine also
known as generic timestamping engine (GTE) support on NVIDIA Tegra194
- systems-on-chip. The driver supports 352 LIC IRQs and 39 AON GPIOs
- lines for timestamping in realtime.
+ and later systems-on-chip. The driver supports 352 LIC IRQs and 39
+ AON GPIOs lines for timestamping in realtime.
config HTE_TEGRA194_TEST
tristate "NVIDIA Tegra194 HTE Test"
diff --git a/drivers/hwmon/abituguru.c b/drivers/hwmon/abituguru.c
index 93653ea05430..ba8c68ae4595 100644
--- a/drivers/hwmon/abituguru.c
+++ b/drivers/hwmon/abituguru.c
@@ -1531,7 +1531,7 @@ static struct platform_driver abituguru_driver = {
.pm = pm_sleep_ptr(&abituguru_pm),
},
.probe = abituguru_probe,
- .remove_new = abituguru_remove,
+ .remove = abituguru_remove,
};
static int __init abituguru_detect(void)
diff --git a/drivers/hwmon/abituguru3.c b/drivers/hwmon/abituguru3.c
index 4501f0e49efb..b70330dc2198 100644
--- a/drivers/hwmon/abituguru3.c
+++ b/drivers/hwmon/abituguru3.c
@@ -1147,12 +1147,12 @@ static int abituguru3_resume(struct device *dev)
static DEFINE_SIMPLE_DEV_PM_OPS(abituguru3_pm, abituguru3_suspend, abituguru3_resume);
static struct platform_driver abituguru3_driver = {
- .driver = {
+ .driver = {
.name = ABIT_UGURU3_NAME,
.pm = pm_sleep_ptr(&abituguru3_pm),
},
.probe = abituguru3_probe,
- .remove_new = abituguru3_remove,
+ .remove = abituguru3_remove,
};
static int __init abituguru3_dmi_detect(void)
diff --git a/drivers/hwmon/aspeed-g6-pwm-tach.c b/drivers/hwmon/aspeed-g6-pwm-tach.c
index 75eadda738ab..d1f7f4397482 100644
--- a/drivers/hwmon/aspeed-g6-pwm-tach.c
+++ b/drivers/hwmon/aspeed-g6-pwm-tach.c
@@ -517,13 +517,6 @@ static int aspeed_pwm_tach_probe(struct platform_device *pdev)
return 0;
}
-static void aspeed_pwm_tach_remove(struct platform_device *pdev)
-{
- struct aspeed_pwm_tach_data *priv = platform_get_drvdata(pdev);
-
- reset_control_assert(priv->reset);
-}
-
static const struct of_device_id aspeed_pwm_tach_match[] = {
{
.compatible = "aspeed,ast2600-pwm-tach",
@@ -534,7 +527,6 @@ MODULE_DEVICE_TABLE(of, aspeed_pwm_tach_match);
static struct platform_driver aspeed_pwm_tach_driver = {
.probe = aspeed_pwm_tach_probe,
- .remove_new = aspeed_pwm_tach_remove,
.driver = {
.name = "aspeed-g6-pwm-tach",
.of_match_table = aspeed_pwm_tach_match,
diff --git a/drivers/hwmon/da9052-hwmon.c b/drivers/hwmon/da9052-hwmon.c
index 7fb0c57dfef5..588e96790850 100644
--- a/drivers/hwmon/da9052-hwmon.c
+++ b/drivers/hwmon/da9052-hwmon.c
@@ -473,7 +473,7 @@ static void da9052_hwmon_remove(struct platform_device *pdev)
static struct platform_driver da9052_hwmon_driver = {
.probe = da9052_hwmon_probe,
- .remove_new = da9052_hwmon_remove,
+ .remove = da9052_hwmon_remove,
.driver = {
.name = "da9052-hwmon",
},
diff --git a/drivers/hwmon/dme1737.c b/drivers/hwmon/dme1737.c
index 1a9b28dc91e6..3d4057309950 100644
--- a/drivers/hwmon/dme1737.c
+++ b/drivers/hwmon/dme1737.c
@@ -2721,7 +2721,7 @@ static struct platform_driver dme1737_isa_driver = {
.name = "dme1737",
},
.probe = dme1737_isa_probe,
- .remove_new = dme1737_isa_remove,
+ .remove = dme1737_isa_remove,
};
/* ---------------------------------------------------------------------
diff --git a/drivers/hwmon/f71805f.c b/drivers/hwmon/f71805f.c
index 243c570dee4c..820f894d9ffd 100644
--- a/drivers/hwmon/f71805f.c
+++ b/drivers/hwmon/f71805f.c
@@ -1497,7 +1497,7 @@ static struct platform_driver f71805f_driver = {
.name = DRVNAME,
},
.probe = f71805f_probe,
- .remove_new = f71805f_remove,
+ .remove = f71805f_remove,
};
static int __init f71805f_device_add(unsigned short address,
diff --git a/drivers/hwmon/f71882fg.c b/drivers/hwmon/f71882fg.c
index 734df959276a..204059d2de6c 100644
--- a/drivers/hwmon/f71882fg.c
+++ b/drivers/hwmon/f71882fg.c
@@ -2660,7 +2660,7 @@ static struct platform_driver f71882fg_driver = {
.name = DRVNAME,
},
.probe = f71882fg_probe,
- .remove_new = f71882fg_remove,
+ .remove = f71882fg_remove,
};
static int __init f71882fg_init(void)
diff --git a/drivers/hwmon/i5k_amb.c b/drivers/hwmon/i5k_amb.c
index 02f5d35dd319..b22e0423e324 100644
--- a/drivers/hwmon/i5k_amb.c
+++ b/drivers/hwmon/i5k_amb.c
@@ -568,7 +568,7 @@ static struct platform_driver i5k_amb_driver = {
.name = DRVNAME,
},
.probe = i5k_amb_probe,
- .remove_new = i5k_amb_remove,
+ .remove = i5k_amb_remove,
};
static int __init i5k_amb_init(void)
diff --git a/drivers/hwmon/max197.c b/drivers/hwmon/max197.c
index bb30403f81ca..f0048ff37607 100644
--- a/drivers/hwmon/max197.c
+++ b/drivers/hwmon/max197.c
@@ -332,7 +332,7 @@ static struct platform_driver max197_driver = {
.name = "max197",
},
.probe = max197_probe,
- .remove_new = max197_remove,
+ .remove = max197_remove,
.id_table = max197_device_ids,
};
module_platform_driver(max197_driver);
diff --git a/drivers/hwmon/mc13783-adc.c b/drivers/hwmon/mc13783-adc.c
index 67471c9cd4d4..66304d48d33a 100644
--- a/drivers/hwmon/mc13783-adc.c
+++ b/drivers/hwmon/mc13783-adc.c
@@ -315,7 +315,7 @@ static const struct platform_device_id mc13783_adc_idtable[] = {
MODULE_DEVICE_TABLE(platform, mc13783_adc_idtable);
static struct platform_driver mc13783_adc_driver = {
- .remove_new = mc13783_adc_remove,
+ .remove = mc13783_adc_remove,
.driver = {
.name = DRIVER_NAME,
},
diff --git a/drivers/hwmon/occ/p9_sbe.c b/drivers/hwmon/occ/p9_sbe.c
index b5993c79c09e..89761a9c8892 100644
--- a/drivers/hwmon/occ/p9_sbe.c
+++ b/drivers/hwmon/occ/p9_sbe.c
@@ -192,8 +192,8 @@ static struct platform_driver p9_sbe_occ_driver = {
.name = "occ-hwmon",
.of_match_table = p9_sbe_occ_of_match,
},
- .probe = p9_sbe_occ_probe,
- .remove_new = p9_sbe_occ_remove,
+ .probe = p9_sbe_occ_probe,
+ .remove = p9_sbe_occ_remove,
};
module_platform_driver(p9_sbe_occ_driver);
diff --git a/drivers/hwmon/pc87360.c b/drivers/hwmon/pc87360.c
index 788b5d58f77e..0f8aa6b42164 100644
--- a/drivers/hwmon/pc87360.c
+++ b/drivers/hwmon/pc87360.c
@@ -1606,7 +1606,7 @@ static struct platform_driver pc87360_driver = {
.name = DRIVER_NAME,
},
.probe = pc87360_probe,
- .remove_new = pc87360_remove,
+ .remove = pc87360_remove,
};
/*
diff --git a/drivers/hwmon/pc87427.c b/drivers/hwmon/pc87427.c
index 7bca04eb4ee4..571402a89368 100644
--- a/drivers/hwmon/pc87427.c
+++ b/drivers/hwmon/pc87427.c
@@ -1129,7 +1129,7 @@ static struct platform_driver pc87427_driver = {
.name = DRVNAME,
},
.probe = pc87427_probe,
- .remove_new = pc87427_remove,
+ .remove = pc87427_remove,
};
static int __init pc87427_device_add(const struct pc87427_sio_data *sio_data)
diff --git a/drivers/hwmon/sch5636.c b/drivers/hwmon/sch5636.c
index a4b05ebb0546..d00bd5cc6b15 100644
--- a/drivers/hwmon/sch5636.c
+++ b/drivers/hwmon/sch5636.c
@@ -512,7 +512,7 @@ static struct platform_driver sch5636_driver = {
.name = DRVNAME,
},
.probe = sch5636_probe,
- .remove_new = sch5636_remove,
+ .remove = sch5636_remove,
.id_table = sch5636_device_id,
};
diff --git a/drivers/hwmon/sht15.c b/drivers/hwmon/sht15.c
index 494f9655f44f..3d55047e9baf 100644
--- a/drivers/hwmon/sht15.c
+++ b/drivers/hwmon/sht15.c
@@ -1051,7 +1051,7 @@ static struct platform_driver sht15_driver = {
.of_match_table = of_match_ptr(sht15_dt_match),
},
.probe = sht15_probe,
- .remove_new = sht15_remove,
+ .remove = sht15_remove,
.id_table = sht15_device_ids,
};
module_platform_driver(sht15_driver);
diff --git a/drivers/hwmon/sis5595.c b/drivers/hwmon/sis5595.c
index e73b1522f3ce..b7a7bcd6d3af 100644
--- a/drivers/hwmon/sis5595.c
+++ b/drivers/hwmon/sis5595.c
@@ -784,7 +784,7 @@ static struct platform_driver sis5595_driver = {
.name = DRIVER_NAME,
},
.probe = sis5595_probe,
- .remove_new = sis5595_remove,
+ .remove = sis5595_remove,
};
static int sis5595_pci_probe(struct pci_dev *dev,
diff --git a/drivers/hwmon/smsc47m1.c b/drivers/hwmon/smsc47m1.c
index 0d46edbcb144..595bceb78d76 100644
--- a/drivers/hwmon/smsc47m1.c
+++ b/drivers/hwmon/smsc47m1.c
@@ -858,7 +858,7 @@ static struct platform_driver smsc47m1_driver __refdata = {
.driver = {
.name = DRVNAME,
},
- .remove_new = __exit_p(smsc47m1_remove),
+ .remove = __exit_p(smsc47m1_remove),
};
static int __init smsc47m1_device_add(unsigned short address,
diff --git a/drivers/hwmon/ultra45_env.c b/drivers/hwmon/ultra45_env.c
index 2765d5f1b7f0..e4f1bb538628 100644
--- a/drivers/hwmon/ultra45_env.c
+++ b/drivers/hwmon/ultra45_env.c
@@ -317,7 +317,7 @@ static struct platform_driver env_driver = {
.of_match_table = env_match,
},
.probe = env_probe,
- .remove_new = env_remove,
+ .remove = env_remove,
};
module_platform_driver(env_driver);
diff --git a/drivers/hwmon/via-cputemp.c b/drivers/hwmon/via-cputemp.c
index 5abe95b683c0..823bff2871e1 100644
--- a/drivers/hwmon/via-cputemp.c
+++ b/drivers/hwmon/via-cputemp.c
@@ -197,7 +197,7 @@ static struct platform_driver via_cputemp_driver = {
.name = DRVNAME,
},
.probe = via_cputemp_probe,
- .remove_new = via_cputemp_remove,
+ .remove = via_cputemp_remove,
};
struct pdev_entry {
diff --git a/drivers/hwmon/via686a.c b/drivers/hwmon/via686a.c
index 3a002ad3c005..bbaeb808cc15 100644
--- a/drivers/hwmon/via686a.c
+++ b/drivers/hwmon/via686a.c
@@ -799,7 +799,7 @@ static struct platform_driver via686a_driver = {
.name = DRIVER_NAME,
},
.probe = via686a_probe,
- .remove_new = via686a_remove,
+ .remove = via686a_remove,
};
static const struct pci_device_id via686a_pci_ids[] = {
diff --git a/drivers/hwmon/vt1211.c b/drivers/hwmon/vt1211.c
index 2f3890463e18..386edea6b69e 100644
--- a/drivers/hwmon/vt1211.c
+++ b/drivers/hwmon/vt1211.c
@@ -1221,7 +1221,7 @@ static struct platform_driver vt1211_driver = {
.name = DRVNAME,
},
.probe = vt1211_probe,
- .remove_new = vt1211_remove,
+ .remove = vt1211_remove,
};
static int __init vt1211_device_add(unsigned short address)
diff --git a/drivers/hwmon/vt8231.c b/drivers/hwmon/vt8231.c
index dcdd14ccd115..3bf27c21845b 100644
--- a/drivers/hwmon/vt8231.c
+++ b/drivers/hwmon/vt8231.c
@@ -910,11 +910,11 @@ static void vt8231_remove(struct platform_device *pdev)
static struct platform_driver vt8231_driver = {
- .driver = {
+ .driver = {
.name = DRIVER_NAME,
},
.probe = vt8231_probe,
- .remove_new = vt8231_remove,
+ .remove = vt8231_remove,
};
static const struct pci_device_id vt8231_pci_ids[] = {
diff --git a/drivers/hwmon/w83627hf.c b/drivers/hwmon/w83627hf.c
index 2fc9b718e2ab..95115d7b863e 100644
--- a/drivers/hwmon/w83627hf.c
+++ b/drivers/hwmon/w83627hf.c
@@ -1844,7 +1844,7 @@ static struct platform_driver w83627hf_driver = {
.pm = W83627HF_DEV_PM_OPS,
},
.probe = w83627hf_probe,
- .remove_new = w83627hf_remove,
+ .remove = w83627hf_remove,
};
static int __init w83627hf_find(int sioaddr, unsigned short *addr,
diff --git a/drivers/hwmon/w83781d.c b/drivers/hwmon/w83781d.c
index b7957c84d235..076200ed2ec9 100644
--- a/drivers/hwmon/w83781d.c
+++ b/drivers/hwmon/w83781d.c
@@ -1828,7 +1828,7 @@ static struct platform_driver w83781d_isa_driver = {
.name = "w83781d",
},
.probe = w83781d_isa_probe,
- .remove_new = w83781d_isa_remove,
+ .remove = w83781d_isa_remove,
};
/* return 1 if a supported chip is found, 0 otherwise */
diff --git a/drivers/hwmon/xgene-hwmon.c b/drivers/hwmon/xgene-hwmon.c
index 4e05077e4256..2cdbd5f107a2 100644
--- a/drivers/hwmon/xgene-hwmon.c
+++ b/drivers/hwmon/xgene-hwmon.c
@@ -772,7 +772,7 @@ MODULE_DEVICE_TABLE(of, xgene_hwmon_of_match);
static struct platform_driver xgene_hwmon_driver = {
.probe = xgene_hwmon_probe,
- .remove_new = xgene_hwmon_remove,
+ .remove = xgene_hwmon_remove,
.driver = {
.name = "xgene-slimpro-hwmon",
.of_match_table = xgene_hwmon_of_match,
diff --git a/drivers/i3c/master/dw-i3c-master.c b/drivers/i3c/master/dw-i3c-master.c
index 3453431e49a2..a60eb86bddba 100644
--- a/drivers/i3c/master/dw-i3c-master.c
+++ b/drivers/i3c/master/dw-i3c-master.c
@@ -6,6 +6,7 @@
*/
#include <linux/bitops.h>
+#include <linux/cleanup.h>
#include <linux/clk.h>
#include <linux/completion.h>
#include <linux/err.h>
@@ -912,7 +913,6 @@ static int dw_i3c_master_priv_xfers(struct i3c_dev_desc *dev,
struct i3c_master_controller *m = i3c_dev_get_master(dev);
struct dw_i3c_master *master = to_dw_i3c_master(m);
unsigned int nrxwords = 0, ntxwords = 0;
- struct dw_i3c_xfer *xfer;
int i, ret = 0;
if (!i3c_nxfers)
@@ -932,7 +932,7 @@ static int dw_i3c_master_priv_xfers(struct i3c_dev_desc *dev,
nrxwords > master->caps.datafifodepth)
return -ENOTSUPP;
- xfer = dw_i3c_master_alloc_xfer(master, i3c_nxfers);
+ struct dw_i3c_xfer *xfer __free(kfree) = dw_i3c_master_alloc_xfer(master, i3c_nxfers);
if (!xfer)
return -ENOMEM;
@@ -983,7 +983,6 @@ static int dw_i3c_master_priv_xfers(struct i3c_dev_desc *dev,
}
ret = xfer->ret;
- dw_i3c_master_free_xfer(xfer);
pm_runtime_mark_last_busy(master->dev);
pm_runtime_put_autosuspend(master->dev);
@@ -1566,13 +1565,11 @@ int dw_i3c_common_probe(struct dw_i3c_master *master,
if (IS_ERR(master->pclk))
return PTR_ERR(master->pclk);
- master->core_rst = devm_reset_control_get_optional_exclusive(&pdev->dev,
- "core_rst");
+ master->core_rst = devm_reset_control_get_optional_exclusive_deasserted(&pdev->dev,
+ "core_rst");
if (IS_ERR(master->core_rst))
return PTR_ERR(master->core_rst);
- reset_control_deassert(master->core_rst);
-
spin_lock_init(&master->xferqueue.lock);
INIT_LIST_HEAD(&master->xferqueue.list);
@@ -1584,7 +1581,7 @@ int dw_i3c_common_probe(struct dw_i3c_master *master,
dw_i3c_master_irq_handler, 0,
dev_name(&pdev->dev), master);
if (ret)
- goto err_assert_rst;
+ return ret;
platform_set_drvdata(pdev, master);
@@ -1620,9 +1617,6 @@ int dw_i3c_common_probe(struct dw_i3c_master *master,
pm_runtime_set_suspended(&pdev->dev);
pm_runtime_dont_use_autosuspend(&pdev->dev);
-err_assert_rst:
- reset_control_assert(master->core_rst);
-
return ret;
}
EXPORT_SYMBOL_GPL(dw_i3c_common_probe);
diff --git a/drivers/i3c/master/mipi-i3c-hci/dma.c b/drivers/i3c/master/mipi-i3c-hci/dma.c
index 36a4c13ab757..b9496e8c4784 100644
--- a/drivers/i3c/master/mipi-i3c-hci/dma.c
+++ b/drivers/i3c/master/mipi-i3c-hci/dma.c
@@ -645,7 +645,10 @@ static void hci_dma_process_ibi(struct i3c_hci *hci, struct hci_rh_data *rh)
if (!(ibi_status & IBI_LAST_STATUS)) {
ibi_size += chunks * rh->ibi_chunk_sz;
} else {
- ibi_size += FIELD_GET(IBI_DATA_LENGTH, ibi_status);
+ if (chunks) {
+ ibi_size += (chunks - 1) * rh->ibi_chunk_sz;
+ ibi_size += FIELD_GET(IBI_DATA_LENGTH, ibi_status);
+ }
last_ptr = ptr;
break;
}
diff --git a/drivers/infiniband/core/iwpm_msg.c b/drivers/infiniband/core/iwpm_msg.c
index 3c9a9869212b..feb09008eb9c 100644
--- a/drivers/infiniband/core/iwpm_msg.c
+++ b/drivers/infiniband/core/iwpm_msg.c
@@ -365,9 +365,9 @@ int iwpm_remove_mapping(struct sockaddr_storage *local_addr, u8 nl_client)
/* netlink attribute policy for the received response to register pid request */
static const struct nla_policy resp_reg_policy[IWPM_NLA_RREG_PID_MAX] = {
[IWPM_NLA_RREG_PID_SEQ] = { .type = NLA_U32 },
- [IWPM_NLA_RREG_IBDEV_NAME] = { .type = NLA_STRING,
+ [IWPM_NLA_RREG_IBDEV_NAME] = { .type = NLA_NUL_STRING,
.len = IWPM_DEVNAME_SIZE - 1 },
- [IWPM_NLA_RREG_ULIB_NAME] = { .type = NLA_STRING,
+ [IWPM_NLA_RREG_ULIB_NAME] = { .type = NLA_NUL_STRING,
.len = IWPM_ULIBNAME_SIZE - 1 },
[IWPM_NLA_RREG_ULIB_VER] = { .type = NLA_U16 },
[IWPM_NLA_RREG_PID_ERR] = { .type = NLA_U16 }
@@ -677,7 +677,7 @@ int iwpm_remote_info_cb(struct sk_buff *skb, struct netlink_callback *cb)
/* netlink attribute policy for the received request for mapping info */
static const struct nla_policy resp_mapinfo_policy[IWPM_NLA_MAPINFO_REQ_MAX] = {
- [IWPM_NLA_MAPINFO_ULIB_NAME] = { .type = NLA_STRING,
+ [IWPM_NLA_MAPINFO_ULIB_NAME] = { .type = NLA_NUL_STRING,
.len = IWPM_ULIBNAME_SIZE - 1 },
[IWPM_NLA_MAPINFO_ULIB_VER] = { .type = NLA_U16 }
};
diff --git a/drivers/infiniband/hw/mana/cq.c b/drivers/infiniband/hw/mana/cq.c
index f04a679d2871..54cf3868b297 100644
--- a/drivers/infiniband/hw/mana/cq.c
+++ b/drivers/infiniband/hw/mana/cq.c
@@ -120,8 +120,9 @@ int mana_ib_install_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq)
if (cq->queue.id >= gc->max_num_cqs)
return -EINVAL;
- /* Create CQ table entry */
- WARN_ON(gc->cq_table[cq->queue.id]);
+ /* Create CQ table entry, sharing a CQ between WQs is not supported */
+ if (gc->cq_table[cq->queue.id])
+ return -EINVAL;
gdma_cq = kzalloc(sizeof(*gdma_cq), GFP_KERNEL);
if (!gdma_cq)
return -ENOMEM;
diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h
index df2aa1c4fafc..7f13b314abbc 100644
--- a/drivers/iommu/amd/amd_iommu_types.h
+++ b/drivers/iommu/amd/amd_iommu_types.h
@@ -426,9 +426,13 @@
#define DTE_GCR3_SHIFT_C 43
#define DTE_GPT_LEVEL_SHIFT 54
+#define DTE_GPT_LEVEL_MASK GENMASK_ULL(55, 54)
#define GCR3_VALID 0x01ULL
+/* DTE[128:179] | DTE[184:191] */
+#define DTE_DATA2_INTR_MASK ~GENMASK_ULL(55, 52)
+
#define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL)
#define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_PR)
#define IOMMU_PTE_DIRTY(pte) ((pte) & IOMMU_PTE_HD)
@@ -567,6 +571,12 @@ struct pdom_dev_data {
struct list_head list;
};
+/* Keeps track of the IOMMUs attached to protection domain */
+struct pdom_iommu_info {
+ struct amd_iommu *iommu; /* IOMMUs attach to protection domain */
+ u32 refcnt; /* Count of attached dev/pasid per domain/IOMMU */
+};
+
/*
* This structure contains generic data for IOMMU protection domains
* independent of their use.
@@ -580,8 +590,7 @@ struct protection_domain {
u16 id; /* the domain id written to the device table */
enum protection_domain_mode pd_mode; /* Track page table type */
bool dirty_tracking; /* dirty tracking is enabled in the domain */
- unsigned dev_cnt; /* devices assigned to this domain */
- unsigned dev_iommu[MAX_IOMMUS]; /* per-IOMMU reference count */
+ struct xarray iommu_array; /* per-IOMMU reference count */
struct mmu_notifier mn; /* mmu notifier for the SVA domain */
struct list_head dev_data_list; /* List of pdom_dev_data */
@@ -833,7 +842,8 @@ struct devid_map {
*/
struct iommu_dev_data {
/*Protect against attach/detach races */
- spinlock_t lock;
+ struct mutex mutex;
+ spinlock_t dte_lock; /* DTE lock for 256-bit access */
struct list_head list; /* For domain->dev_list */
struct llist_node dev_data_list; /* For global dev_data_list */
@@ -884,7 +894,10 @@ extern struct amd_iommu *amd_iommus[MAX_IOMMUS];
* Structure defining one entry in the device table
*/
struct dev_table_entry {
- u64 data[4];
+ union {
+ u64 data[4];
+ u128 data128[2];
+ };
};
/*
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index d0e53a03eff0..65d61b9c7382 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -77,12 +77,142 @@ static void detach_device(struct device *dev);
static void set_dte_entry(struct amd_iommu *iommu,
struct iommu_dev_data *dev_data);
+static void iommu_flush_dte_sync(struct amd_iommu *iommu, u16 devid);
+
+static struct iommu_dev_data *find_dev_data(struct amd_iommu *iommu, u16 devid);
+
/****************************************************************************
*
* Helper functions
*
****************************************************************************/
+static __always_inline void amd_iommu_atomic128_set(__int128 *ptr, __int128 val)
+{
+ /*
+ * Note:
+ * We use arch_cmpxchg128_local() because:
+ * - Need cmpxchg16b instruction mainly for 128-bit store to DTE
+ * (not necessary for cmpxchg since this function is already
+ * protected by a spin_lock for this DTE).
+ * - Neither need LOCK_PREFIX nor try loop because of the spin_lock.
+ */
+ arch_cmpxchg128_local(ptr, *ptr, val);
+}
+
+static void write_dte_upper128(struct dev_table_entry *ptr, struct dev_table_entry *new)
+{
+ struct dev_table_entry old;
+
+ old.data128[1] = ptr->data128[1];
+ /*
+ * Preserve DTE_DATA2_INTR_MASK. This needs to be
+ * done here since it requires to be inside
+ * spin_lock(&dev_data->dte_lock) context.
+ */
+ new->data[2] &= ~DTE_DATA2_INTR_MASK;
+ new->data[2] |= old.data[2] & DTE_DATA2_INTR_MASK;
+
+ amd_iommu_atomic128_set(&ptr->data128[1], new->data128[1]);
+}
+
+static void write_dte_lower128(struct dev_table_entry *ptr, struct dev_table_entry *new)
+{
+ amd_iommu_atomic128_set(&ptr->data128[0], new->data128[0]);
+}
+
+/*
+ * Note:
+ * IOMMU reads the entire Device Table entry in a single 256-bit transaction
+ * but the driver is programming DTE using 2 128-bit cmpxchg. So, the driver
+ * need to ensure the following:
+ * - DTE[V|GV] bit is being written last when setting.
+ * - DTE[V|GV] bit is being written first when clearing.
+ *
+ * This function is used only by code, which updates DMA translation part of the DTE.
+ * So, only consider control bits related to DMA when updating the entry.
+ */
+static void update_dte256(struct amd_iommu *iommu, struct iommu_dev_data *dev_data,
+ struct dev_table_entry *new)
+{
+ unsigned long flags;
+ struct dev_table_entry *dev_table = get_dev_table(iommu);
+ struct dev_table_entry *ptr = &dev_table[dev_data->devid];
+
+ spin_lock_irqsave(&dev_data->dte_lock, flags);
+
+ if (!(ptr->data[0] & DTE_FLAG_V)) {
+ /* Existing DTE is not valid. */
+ write_dte_upper128(ptr, new);
+ write_dte_lower128(ptr, new);
+ iommu_flush_dte_sync(iommu, dev_data->devid);
+ } else if (!(new->data[0] & DTE_FLAG_V)) {
+ /* Existing DTE is valid. New DTE is not valid. */
+ write_dte_lower128(ptr, new);
+ write_dte_upper128(ptr, new);
+ iommu_flush_dte_sync(iommu, dev_data->devid);
+ } else if (!FIELD_GET(DTE_FLAG_GV, ptr->data[0])) {
+ /*
+ * Both DTEs are valid.
+ * Existing DTE has no guest page table.
+ */
+ write_dte_upper128(ptr, new);
+ write_dte_lower128(ptr, new);
+ iommu_flush_dte_sync(iommu, dev_data->devid);
+ } else if (!FIELD_GET(DTE_FLAG_GV, new->data[0])) {
+ /*
+ * Both DTEs are valid.
+ * Existing DTE has guest page table,
+ * new DTE has no guest page table,
+ */
+ write_dte_lower128(ptr, new);
+ write_dte_upper128(ptr, new);
+ iommu_flush_dte_sync(iommu, dev_data->devid);
+ } else if (FIELD_GET(DTE_GPT_LEVEL_MASK, ptr->data[2]) !=
+ FIELD_GET(DTE_GPT_LEVEL_MASK, new->data[2])) {
+ /*
+ * Both DTEs are valid and have guest page table,
+ * but have different number of levels. So, we need
+ * to upadte both upper and lower 128-bit value, which
+ * require disabling and flushing.
+ */
+ struct dev_table_entry clear = {};
+
+ /* First disable DTE */
+ write_dte_lower128(ptr, &clear);
+ iommu_flush_dte_sync(iommu, dev_data->devid);
+
+ /* Then update DTE */
+ write_dte_upper128(ptr, new);
+ write_dte_lower128(ptr, new);
+ iommu_flush_dte_sync(iommu, dev_data->devid);
+ } else {
+ /*
+ * Both DTEs are valid and have guest page table,
+ * and same number of levels. We just need to only
+ * update the lower 128-bit. So no need to disable DTE.
+ */
+ write_dte_lower128(ptr, new);
+ }
+
+ spin_unlock_irqrestore(&dev_data->dte_lock, flags);
+}
+
+static void get_dte256(struct amd_iommu *iommu, struct iommu_dev_data *dev_data,
+ struct dev_table_entry *dte)
+{
+ unsigned long flags;
+ struct dev_table_entry *ptr;
+ struct dev_table_entry *dev_table = get_dev_table(iommu);
+
+ ptr = &dev_table[dev_data->devid];
+
+ spin_lock_irqsave(&dev_data->dte_lock, flags);
+ dte->data128[0] = ptr->data128[0];
+ dte->data128[1] = ptr->data128[1];
+ spin_unlock_irqrestore(&dev_data->dte_lock, flags);
+}
+
static inline bool pdom_is_v2_pgtbl_mode(struct protection_domain *pdom)
{
return (pdom && (pdom->pd_mode == PD_MODE_V2));
@@ -225,7 +355,8 @@ static struct iommu_dev_data *alloc_dev_data(struct amd_iommu *iommu, u16 devid)
if (!dev_data)
return NULL;
- spin_lock_init(&dev_data->lock);
+ mutex_init(&dev_data->mutex);
+ spin_lock_init(&dev_data->dte_lock);
dev_data->devid = devid;
ratelimit_default_init(&dev_data->rs);
@@ -251,11 +382,14 @@ static struct iommu_dev_data *search_dev_data(struct amd_iommu *iommu, u16 devid
return NULL;
}
-static int clone_alias(struct pci_dev *pdev, u16 alias, void *data)
+static int clone_alias(struct pci_dev *pdev_origin, u16 alias, void *data)
{
+ struct dev_table_entry new;
struct amd_iommu *iommu;
- struct dev_table_entry *dev_table;
+ struct iommu_dev_data *dev_data, *alias_data;
+ struct pci_dev *pdev = data;
u16 devid = pci_dev_id(pdev);
+ int ret = 0;
if (devid == alias)
return 0;
@@ -264,13 +398,27 @@ static int clone_alias(struct pci_dev *pdev, u16 alias, void *data)
if (!iommu)
return 0;
- amd_iommu_set_rlookup_table(iommu, alias);
- dev_table = get_dev_table(iommu);
- memcpy(dev_table[alias].data,
- dev_table[devid].data,
- sizeof(dev_table[alias].data));
+ /* Copy the data from pdev */
+ dev_data = dev_iommu_priv_get(&pdev->dev);
+ if (!dev_data) {
+ pr_err("%s : Failed to get dev_data for 0x%x\n", __func__, devid);
+ ret = -EINVAL;
+ goto out;
+ }
+ get_dte256(iommu, dev_data, &new);
- return 0;
+ /* Setup alias */
+ alias_data = find_dev_data(iommu, alias);
+ if (!alias_data) {
+ pr_err("%s : Failed to get alias dev_data for 0x%x\n", __func__, alias);
+ ret = -EINVAL;
+ goto out;
+ }
+ update_dte256(iommu, alias_data, &new);
+
+ amd_iommu_set_rlookup_table(iommu, alias);
+out:
+ return ret;
}
static void clone_aliases(struct amd_iommu *iommu, struct device *dev)
@@ -286,9 +434,9 @@ static void clone_aliases(struct amd_iommu *iommu, struct device *dev)
* part of the PCI DMA aliases if it's bus differs
* from the original device.
*/
- clone_alias(pdev, iommu->pci_seg->alias_table[pci_dev_id(pdev)], NULL);
+ clone_alias(pdev, iommu->pci_seg->alias_table[pci_dev_id(pdev)], pdev);
- pci_for_each_dma_alias(pdev, clone_alias, NULL);
+ pci_for_each_dma_alias(pdev, clone_alias, pdev);
}
static void setup_aliases(struct amd_iommu *iommu, struct device *dev)
@@ -543,6 +691,12 @@ static int iommu_init_device(struct amd_iommu *iommu, struct device *dev)
return -ENOMEM;
dev_data->dev = dev;
+
+ /*
+ * The dev_iommu_priv_set() needes to be called before setup_aliases.
+ * Otherwise, subsequent call to dev_iommu_priv_get() will fail.
+ */
+ dev_iommu_priv_set(dev, dev_data);
setup_aliases(iommu, dev);
/*
@@ -556,8 +710,6 @@ static int iommu_init_device(struct amd_iommu *iommu, struct device *dev)
dev_data->flags = pdev_get_caps(to_pci_dev(dev));
}
- dev_iommu_priv_set(dev, dev_data);
-
return 0;
}
@@ -604,10 +756,13 @@ static void amd_iommu_uninit_device(struct device *dev)
static void dump_dte_entry(struct amd_iommu *iommu, u16 devid)
{
int i;
- struct dev_table_entry *dev_table = get_dev_table(iommu);
+ struct dev_table_entry dte;
+ struct iommu_dev_data *dev_data = find_dev_data(iommu, devid);
+
+ get_dte256(iommu, dev_data, &dte);
for (i = 0; i < 4; ++i)
- pr_err("DTE[%d]: %016llx\n", i, dev_table[devid].data[i]);
+ pr_err("DTE[%d]: %016llx\n", i, dte.data[i]);
}
static void dump_command(unsigned long phys_addr)
@@ -1290,18 +1445,17 @@ static int iommu_completion_wait(struct amd_iommu *iommu)
static void domain_flush_complete(struct protection_domain *domain)
{
- int i;
+ struct pdom_iommu_info *pdom_iommu_info;
+ unsigned long i;
- for (i = 0; i < amd_iommu_get_num_iommus(); ++i) {
- if (domain && !domain->dev_iommu[i])
- continue;
+ lockdep_assert_held(&domain->lock);
- /*
- * Devices of this domain are behind this IOMMU
- * We need to wait for completion of all commands.
- */
- iommu_completion_wait(amd_iommus[i]);
- }
+ /*
+ * Devices of this domain are behind this IOMMU
+ * We need to wait for completion of all commands.
+ */
+ xa_for_each(&domain->iommu_array, i, pdom_iommu_info)
+ iommu_completion_wait(pdom_iommu_info->iommu);
}
static int iommu_flush_dte(struct amd_iommu *iommu, u16 devid)
@@ -1313,6 +1467,15 @@ static int iommu_flush_dte(struct amd_iommu *iommu, u16 devid)
return iommu_queue_command(iommu, &cmd);
}
+static void iommu_flush_dte_sync(struct amd_iommu *iommu, u16 devid)
+{
+ int ret;
+
+ ret = iommu_flush_dte(iommu, devid);
+ if (!ret)
+ iommu_completion_wait(iommu);
+}
+
static void amd_iommu_flush_dte_all(struct amd_iommu *iommu)
{
u32 devid;
@@ -1483,21 +1646,22 @@ static int domain_flush_pages_v2(struct protection_domain *pdom,
static int domain_flush_pages_v1(struct protection_domain *pdom,
u64 address, size_t size)
{
+ struct pdom_iommu_info *pdom_iommu_info;
struct iommu_cmd cmd;
- int ret = 0, i;
+ int ret = 0;
+ unsigned long i;
+
+ lockdep_assert_held(&pdom->lock);
build_inv_iommu_pages(&cmd, address, size,
pdom->id, IOMMU_NO_PASID, false);
- for (i = 0; i < amd_iommu_get_num_iommus(); ++i) {
- if (!pdom->dev_iommu[i])
- continue;
-
+ xa_for_each(&pdom->iommu_array, i, pdom_iommu_info) {
/*
* Devices of this domain are behind this IOMMU
* We need a TLB flush
*/
- ret |= iommu_queue_command(amd_iommus[i], &cmd);
+ ret |= iommu_queue_command(pdom_iommu_info->iommu, &cmd);
}
return ret;
@@ -1536,6 +1700,8 @@ static void __domain_flush_pages(struct protection_domain *domain,
void amd_iommu_domain_flush_pages(struct protection_domain *domain,
u64 address, size_t size)
{
+ lockdep_assert_held(&domain->lock);
+
if (likely(!amd_iommu_np_cache)) {
__domain_flush_pages(domain, address, size);
@@ -2051,57 +2217,69 @@ static void destroy_gcr3_table(struct iommu_dev_data *dev_data,
free_gcr3_table(gcr3_info);
}
-static int do_attach(struct iommu_dev_data *dev_data,
- struct protection_domain *domain)
+static int pdom_attach_iommu(struct amd_iommu *iommu,
+ struct protection_domain *pdom)
{
- struct amd_iommu *iommu = get_amd_iommu_from_dev_data(dev_data);
- struct io_pgtable_cfg *cfg = &domain->iop.pgtbl.cfg;
+ struct pdom_iommu_info *pdom_iommu_info, *curr;
+ struct io_pgtable_cfg *cfg = &pdom->iop.pgtbl.cfg;
+ unsigned long flags;
int ret = 0;
- /* Update data structures */
- dev_data->domain = domain;
- list_add(&dev_data->list, &domain->dev_list);
+ spin_lock_irqsave(&pdom->lock, flags);
- /* Update NUMA Node ID */
- if (cfg->amd.nid == NUMA_NO_NODE)
- cfg->amd.nid = dev_to_node(dev_data->dev);
+ pdom_iommu_info = xa_load(&pdom->iommu_array, iommu->index);
+ if (pdom_iommu_info) {
+ pdom_iommu_info->refcnt++;
+ goto out_unlock;
+ }
- /* Do reference counting */
- domain->dev_iommu[iommu->index] += 1;
- domain->dev_cnt += 1;
+ pdom_iommu_info = kzalloc(sizeof(*pdom_iommu_info), GFP_ATOMIC);
+ if (!pdom_iommu_info) {
+ ret = -ENOMEM;
+ goto out_unlock;
+ }
- /* Setup GCR3 table */
- if (pdom_is_sva_capable(domain)) {
- ret = init_gcr3_table(dev_data, domain);
- if (ret)
- return ret;
+ pdom_iommu_info->iommu = iommu;
+ pdom_iommu_info->refcnt = 1;
+
+ curr = xa_cmpxchg(&pdom->iommu_array, iommu->index,
+ NULL, pdom_iommu_info, GFP_ATOMIC);
+ if (curr) {
+ kfree(pdom_iommu_info);
+ ret = -ENOSPC;
+ goto out_unlock;
}
+ /* Update NUMA Node ID */
+ if (cfg->amd.nid == NUMA_NO_NODE)
+ cfg->amd.nid = dev_to_node(&iommu->dev->dev);
+
+out_unlock:
+ spin_unlock_irqrestore(&pdom->lock, flags);
return ret;
}
-static void do_detach(struct iommu_dev_data *dev_data)
+static void pdom_detach_iommu(struct amd_iommu *iommu,
+ struct protection_domain *pdom)
{
- struct protection_domain *domain = dev_data->domain;
- struct amd_iommu *iommu = get_amd_iommu_from_dev_data(dev_data);
-
- /* Clear DTE and flush the entry */
- dev_update_dte(dev_data, false);
+ struct pdom_iommu_info *pdom_iommu_info;
+ unsigned long flags;
- /* Flush IOTLB and wait for the flushes to finish */
- amd_iommu_domain_flush_all(domain);
+ spin_lock_irqsave(&pdom->lock, flags);
- /* Clear GCR3 table */
- if (pdom_is_sva_capable(domain))
- destroy_gcr3_table(dev_data, domain);
+ pdom_iommu_info = xa_load(&pdom->iommu_array, iommu->index);
+ if (!pdom_iommu_info) {
+ spin_unlock_irqrestore(&pdom->lock, flags);
+ return;
+ }
- /* Update data structures */
- dev_data->domain = NULL;
- list_del(&dev_data->list);
+ pdom_iommu_info->refcnt--;
+ if (pdom_iommu_info->refcnt == 0) {
+ xa_erase(&pdom->iommu_array, iommu->index);
+ kfree(pdom_iommu_info);
+ }
- /* decrease reference counters - needs to happen after the flushes */
- domain->dev_iommu[iommu->index] -= 1;
- domain->dev_cnt -= 1;
+ spin_unlock_irqrestore(&pdom->lock, flags);
}
/*
@@ -2111,28 +2289,60 @@ static void do_detach(struct iommu_dev_data *dev_data)
static int attach_device(struct device *dev,
struct protection_domain *domain)
{
- struct iommu_dev_data *dev_data;
+ struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
+ struct amd_iommu *iommu = get_amd_iommu_from_dev_data(dev_data);
+ struct pci_dev *pdev;
unsigned long flags;
int ret = 0;
- spin_lock_irqsave(&domain->lock, flags);
-
- dev_data = dev_iommu_priv_get(dev);
-
- spin_lock(&dev_data->lock);
+ mutex_lock(&dev_data->mutex);
if (dev_data->domain != NULL) {
ret = -EBUSY;
goto out;
}
- ret = do_attach(dev_data, domain);
+ /* Do reference counting */
+ ret = pdom_attach_iommu(iommu, domain);
+ if (ret)
+ goto out;
-out:
- spin_unlock(&dev_data->lock);
+ /* Setup GCR3 table */
+ if (pdom_is_sva_capable(domain)) {
+ ret = init_gcr3_table(dev_data, domain);
+ if (ret) {
+ pdom_detach_iommu(iommu, domain);
+ goto out;
+ }
+ }
+ pdev = dev_is_pci(dev_data->dev) ? to_pci_dev(dev_data->dev) : NULL;
+ if (pdev && pdom_is_sva_capable(domain)) {
+ pdev_enable_caps(pdev);
+
+ /*
+ * Device can continue to function even if IOPF
+ * enablement failed. Hence in error path just
+ * disable device PRI support.
+ */
+ if (amd_iommu_iopf_add_device(iommu, dev_data))
+ pdev_disable_cap_pri(pdev);
+ } else if (pdev) {
+ pdev_enable_cap_ats(pdev);
+ }
+
+ /* Update data structures */
+ dev_data->domain = domain;
+ spin_lock_irqsave(&domain->lock, flags);
+ list_add(&dev_data->list, &domain->dev_list);
spin_unlock_irqrestore(&domain->lock, flags);
+ /* Update device table */
+ dev_update_dte(dev_data, true);
+
+out:
+ mutex_unlock(&dev_data->mutex);
+
return ret;
}
@@ -2142,14 +2352,11 @@ static int attach_device(struct device *dev,
static void detach_device(struct device *dev)
{
struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
- struct protection_domain *domain = dev_data->domain;
struct amd_iommu *iommu = get_amd_iommu_from_dev_data(dev_data);
+ struct protection_domain *domain = dev_data->domain;
unsigned long flags;
- bool ppr = dev_data->ppr;
- spin_lock_irqsave(&domain->lock, flags);
-
- spin_lock(&dev_data->lock);
+ mutex_lock(&dev_data->mutex);
/*
* First check if the device is still attached. It might already
@@ -2160,27 +2367,36 @@ static void detach_device(struct device *dev)
if (WARN_ON(!dev_data->domain))
goto out;
- if (ppr) {
+ /* Remove IOPF handler */
+ if (dev_data->ppr) {
iopf_queue_flush_dev(dev);
-
- /* Updated here so that it gets reflected in DTE */
- dev_data->ppr = false;
+ amd_iommu_iopf_remove_device(iommu, dev_data);
}
- do_detach(dev_data);
+ if (dev_is_pci(dev))
+ pdev_disable_caps(to_pci_dev(dev));
-out:
- spin_unlock(&dev_data->lock);
+ /* Clear DTE and flush the entry */
+ dev_update_dte(dev_data, false);
+ /* Flush IOTLB and wait for the flushes to finish */
+ spin_lock_irqsave(&domain->lock, flags);
+ amd_iommu_domain_flush_all(domain);
+ list_del(&dev_data->list);
spin_unlock_irqrestore(&domain->lock, flags);
- /* Remove IOPF handler */
- if (ppr)
- amd_iommu_iopf_remove_device(iommu, dev_data);
+ /* Clear GCR3 table */
+ if (pdom_is_sva_capable(domain))
+ destroy_gcr3_table(dev_data, domain);
- if (dev_is_pci(dev))
- pdev_disable_caps(to_pci_dev(dev));
+ /* Update data structures */
+ dev_data->domain = NULL;
+ /* decrease reference counters - needs to happen after the flushes */
+ pdom_detach_iommu(iommu, domain);
+
+out:
+ mutex_unlock(&dev_data->mutex);
}
static struct iommu_device *amd_iommu_probe_device(struct device *dev)
@@ -2268,24 +2484,6 @@ static struct iommu_group *amd_iommu_device_group(struct device *dev)
*
*****************************************************************************/
-static void cleanup_domain(struct protection_domain *domain)
-{
- struct iommu_dev_data *entry;
-
- lockdep_assert_held(&domain->lock);
-
- if (!domain->dev_cnt)
- return;
-
- while (!list_empty(&domain->dev_list)) {
- entry = list_first_entry(&domain->dev_list,
- struct iommu_dev_data, list);
- BUG_ON(!entry->domain);
- do_detach(entry);
- }
- WARN_ON(domain->dev_cnt != 0);
-}
-
void protection_domain_free(struct protection_domain *domain)
{
WARN_ON(!list_empty(&domain->dev_list));
@@ -2312,6 +2510,7 @@ struct protection_domain *protection_domain_alloc(unsigned int type, int nid)
spin_lock_init(&domain->lock);
INIT_LIST_HEAD(&domain->dev_list);
INIT_LIST_HEAD(&domain->dev_data_list);
+ xa_init(&domain->iommu_array);
domain->iop.pgtbl.cfg.amd.nid = nid;
switch (type) {
@@ -2452,16 +2651,7 @@ amd_iommu_domain_alloc_user(struct device *dev, u32 flags,
void amd_iommu_domain_free(struct iommu_domain *dom)
{
- struct protection_domain *domain;
- unsigned long flags;
-
- domain = to_pdomain(dom);
-
- spin_lock_irqsave(&domain->lock, flags);
-
- cleanup_domain(domain);
-
- spin_unlock_irqrestore(&domain->lock, flags);
+ struct protection_domain *domain = to_pdomain(dom);
protection_domain_free(domain);
}
@@ -2475,9 +2665,9 @@ static int blocked_domain_attach_device(struct iommu_domain *domain,
detach_device(dev);
/* Clear DTE and flush the entry */
- spin_lock(&dev_data->lock);
+ mutex_lock(&dev_data->mutex);
dev_update_dte(dev_data, false);
- spin_unlock(&dev_data->lock);
+ mutex_unlock(&dev_data->mutex);
return 0;
}
@@ -2495,7 +2685,6 @@ static int amd_iommu_attach_device(struct iommu_domain *dom,
struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
struct protection_domain *domain = to_pdomain(dom);
struct amd_iommu *iommu = get_amd_iommu_from_dev(dev);
- struct pci_dev *pdev;
int ret;
/*
@@ -2528,24 +2717,6 @@ static int amd_iommu_attach_device(struct iommu_domain *dom,
}
#endif
- pdev = dev_is_pci(dev_data->dev) ? to_pci_dev(dev_data->dev) : NULL;
- if (pdev && pdom_is_sva_capable(domain)) {
- pdev_enable_caps(pdev);
-
- /*
- * Device can continue to function even if IOPF
- * enablement failed. Hence in error path just
- * disable device PRI support.
- */
- if (amd_iommu_iopf_add_device(iommu, dev_data))
- pdev_disable_cap_pri(pdev);
- } else if (pdev) {
- pdev_enable_cap_ats(pdev);
- }
-
- /* Update device table */
- dev_update_dte(dev_data, true);
-
return ret;
}
diff --git a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
index dd7d030d2e89..a76839cf571d 100644
--- a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
+++ b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
@@ -387,6 +387,10 @@ static int tegra241_vcmdq_hw_init(struct tegra241_vcmdq *vcmdq)
/* Reset VCMDQ */
tegra241_vcmdq_hw_deinit(vcmdq);
+ /* vintf->hyp_own is a HW state finalized in tegra241_vintf_hw_init() */
+ if (!vcmdq->vintf->hyp_own)
+ vcmdq->cmdq.supports_cmd = tegra241_guest_vcmdq_supports_cmd;
+
/* Configure and enable VCMDQ */
writeq_relaxed(vcmdq->cmdq.q.q_base, REG_VCMDQ_PAGE1(vcmdq, BASE));
@@ -514,9 +518,6 @@ static int tegra241_vcmdq_alloc_smmu_cmdq(struct tegra241_vcmdq *vcmdq)
q->q_base = q->base_dma & VCMDQ_ADDR;
q->q_base |= FIELD_PREP(VCMDQ_LOG2SIZE, q->llq.max_n_shift);
- if (!vcmdq->vintf->hyp_own)
- cmdq->supports_cmd = tegra241_guest_vcmdq_supports_cmd;
-
return arm_smmu_cmdq_init(smmu, cmdq);
}
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index d4f852f712aa..cce5a19b5d33 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -4684,6 +4684,9 @@ static void quirk_iommu_igfx(struct pci_dev *dev)
disable_igfx_iommu = 1;
}
+/* Q35 integrated gfx dmar support is totally busted. */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x29b2, quirk_iommu_igfx);
+
/* G4x/GM45 integrated gfx dmar support is totally busted. */
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2a40, quirk_iommu_igfx);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2e00, quirk_iommu_igfx);
diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c
index af39b2379d53..8226e28d79a5 100644
--- a/drivers/iommu/iommufd/fault.c
+++ b/drivers/iommu/iommufd/fault.c
@@ -317,9 +317,10 @@ static ssize_t iommufd_fault_fops_write(struct file *filep, const char __user *b
mutex_lock(&fault->mutex);
while (count > done) {
- rc = copy_from_user(&response, buf + done, response_size);
- if (rc)
+ if (copy_from_user(&response, buf + done, response_size)) {
+ rc = -EFAULT;
break;
+ }
static_assert((int)IOMMUFD_PAGE_RESP_SUCCESS ==
(int)IOMMU_PAGE_RESP_SUCCESS);
diff --git a/drivers/iommu/iommufd/vfio_compat.c b/drivers/iommu/iommufd/vfio_compat.c
index a3ad5f0b6c59..80a2f7faee9b 100644
--- a/drivers/iommu/iommufd/vfio_compat.c
+++ b/drivers/iommu/iommufd/vfio_compat.c
@@ -283,7 +283,7 @@ static int iommufd_vfio_check_extension(struct iommufd_ctx *ictx,
case VFIO_TYPE1_IOMMU:
case VFIO_TYPE1v2_IOMMU:
case VFIO_UNMAP_ALL:
- return 1;
+ return !ictx->no_iommu_mode;
case VFIO_NOIOMMU_IOMMU:
return IS_ENABLED(CONFIG_VFIO_NOIOMMU);
diff --git a/drivers/irqchip/irq-pic32-evic.c b/drivers/irqchip/irq-pic32-evic.c
index eb6ca516a166..9b309d8ec20c 100644
--- a/drivers/irqchip/irq-pic32-evic.c
+++ b/drivers/irqchip/irq-pic32-evic.c
@@ -196,7 +196,7 @@ static void __init pic32_ext_irq_of_init(struct irq_domain *domain)
of_property_for_each_u32(node, pname, hwirq) {
if (i >= ARRAY_SIZE(priv->ext_irqs)) {
- pr_warn("More than %d external irq, skip rest\n",
+ pr_warn("More than %zu external irq, skip rest\n",
ARRAY_SIZE(priv->ext_irqs));
break;
}
diff --git a/drivers/irqchip/irq-riscv-imsic-early.c b/drivers/irqchip/irq-riscv-imsic-early.c
index b5def6268936..dbd02dfe0397 100644
--- a/drivers/irqchip/irq-riscv-imsic-early.c
+++ b/drivers/irqchip/irq-riscv-imsic-early.c
@@ -139,6 +139,8 @@ static int imsic_dying_cpu(unsigned int cpu)
/* Cleanup IPIs */
imsic_ipi_dying_cpu();
+ imsic_local_sync_all(false);
+
/* Mark per-CPU IMSIC state as offline */
imsic_state_offline();
diff --git a/drivers/leds/blink/leds-lgm-sso.c b/drivers/leds/blink/leds-lgm-sso.c
index 7b04ea146260..330cd4619465 100644
--- a/drivers/leds/blink/leds-lgm-sso.c
+++ b/drivers/leds/blink/leds-lgm-sso.c
@@ -806,8 +806,6 @@ static int intel_sso_led_probe(struct platform_device *pdev)
priv->fpid_clkrate = clk_get_rate(priv->clocks[1].clk);
- priv->mmap = syscon_node_to_regmap(dev->of_node);
-
priv->mmap = syscon_node_to_regmap(dev->of_node);
if (IS_ERR(priv->mmap)) {
dev_err(dev, "Failed to map iomem!\n");
diff --git a/drivers/mailbox/mailbox-test.c b/drivers/mailbox/mailbox-test.c
index e416ce9e2d67..b4ed96be815d 100644
--- a/drivers/mailbox/mailbox-test.c
+++ b/drivers/mailbox/mailbox-test.c
@@ -28,8 +28,6 @@
#define MBOX_HEXDUMP_MAX_LEN (MBOX_HEXDUMP_LINE_LEN * \
(MBOX_MAX_MSG_LEN / MBOX_BYTES_PER_LINE))
-static bool mbox_data_ready;
-
struct mbox_test_device {
struct device *dev;
void __iomem *tx_mmio;
@@ -42,6 +40,7 @@ struct mbox_test_device {
spinlock_t lock;
struct mutex mutex;
wait_queue_head_t waitq;
+ bool data_ready;
struct fasync_struct *async_queue;
struct dentry *root_debugfs_dir;
};
@@ -162,7 +161,7 @@ static bool mbox_test_message_data_ready(struct mbox_test_device *tdev)
unsigned long flags;
spin_lock_irqsave(&tdev->lock, flags);
- data_ready = mbox_data_ready;
+ data_ready = tdev->data_ready;
spin_unlock_irqrestore(&tdev->lock, flags);
return data_ready;
@@ -227,7 +226,7 @@ static ssize_t mbox_test_message_read(struct file *filp, char __user *userbuf,
*(touser + l) = '\0';
memset(tdev->rx_buffer, 0, MBOX_MAX_MSG_LEN);
- mbox_data_ready = false;
+ tdev->data_ready = false;
spin_unlock_irqrestore(&tdev->lock, flags);
@@ -297,7 +296,7 @@ static void mbox_test_receive_message(struct mbox_client *client, void *message)
message, MBOX_MAX_MSG_LEN);
memcpy(tdev->rx_buffer, message, MBOX_MAX_MSG_LEN);
}
- mbox_data_ready = true;
+ tdev->data_ready = true;
spin_unlock_irqrestore(&tdev->lock, flags);
wake_up_interruptible(&tdev->waitq);
@@ -366,6 +365,12 @@ static int mbox_test_probe(struct platform_device *pdev)
if (!tdev)
return -ENOMEM;
+ tdev->dev = &pdev->dev;
+ spin_lock_init(&tdev->lock);
+ mutex_init(&tdev->mutex);
+ init_waitqueue_head(&tdev->waitq);
+ platform_set_drvdata(pdev, tdev);
+
/* It's okay for MMIO to be NULL */
tdev->tx_mmio = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
if (PTR_ERR(tdev->tx_mmio) == -EBUSY) {
@@ -395,27 +400,29 @@ static int mbox_test_probe(struct platform_device *pdev)
if (!tdev->rx_channel && (tdev->rx_mmio != tdev->tx_mmio))
tdev->rx_channel = tdev->tx_channel;
- tdev->dev = &pdev->dev;
- platform_set_drvdata(pdev, tdev);
-
- spin_lock_init(&tdev->lock);
- mutex_init(&tdev->mutex);
-
if (tdev->rx_channel) {
tdev->rx_buffer = devm_kzalloc(&pdev->dev,
MBOX_MAX_MSG_LEN, GFP_KERNEL);
- if (!tdev->rx_buffer)
- return -ENOMEM;
+ if (!tdev->rx_buffer) {
+ ret = -ENOMEM;
+ goto err_free_chans;
+ }
}
ret = mbox_test_add_debugfs(pdev, tdev);
if (ret)
- return ret;
+ goto err_free_chans;
- init_waitqueue_head(&tdev->waitq);
dev_info(&pdev->dev, "Successfully registered\n");
return 0;
+
+err_free_chans:
+ if (tdev->tx_channel)
+ mbox_free_channel(tdev->tx_channel);
+ if (tdev->rx_channel && tdev->rx_channel != tdev->tx_channel)
+ mbox_free_channel(tdev->rx_channel);
+ return ret;
}
static void mbox_test_remove(struct platform_device *pdev)
@@ -426,7 +433,7 @@ static void mbox_test_remove(struct platform_device *pdev)
if (tdev->tx_channel)
mbox_free_channel(tdev->tx_channel);
- if (tdev->rx_channel)
+ if (tdev->rx_channel && tdev->rx_channel != tdev->tx_channel)
mbox_free_channel(tdev->rx_channel);
}
diff --git a/drivers/mailbox/mailbox.c b/drivers/mailbox/mailbox.c
index b4d52b814055..39269359e3a6 100644
--- a/drivers/mailbox/mailbox.c
+++ b/drivers/mailbox/mailbox.c
@@ -520,8 +520,7 @@ int mbox_controller_register(struct mbox_controller *mbox)
{
int i, txdone;
- /* Sanity check */
- if (!mbox || !mbox->dev || !mbox->ops || !mbox->num_chans)
+ if (!mbox || !mbox->dev || !mbox->ops || !mbox->chans || !mbox->num_chans)
return -EINVAL;
if (mbox->txdone_irq)
diff --git a/drivers/mailbox/mtk-cmdq-mailbox.c b/drivers/mailbox/mtk-cmdq-mailbox.c
index 80a10361492b..f1159dabcd4e 100644
--- a/drivers/mailbox/mtk-cmdq-mailbox.c
+++ b/drivers/mailbox/mtk-cmdq-mailbox.c
@@ -434,14 +434,14 @@ static int cmdq_mbox_send_data(struct mbox_chan *chan, void *data)
if (curr_pa == end_pa - CMDQ_INST_SIZE ||
curr_pa == end_pa) {
/* set to this task directly */
- writel(task->pa_base >> cmdq->pdata->shift,
- thread->base + CMDQ_THR_CURR_ADDR);
+ gce_addr = cmdq_convert_gce_addr(task->pa_base, cmdq->pdata);
+ writel(gce_addr, thread->base + CMDQ_THR_CURR_ADDR);
} else {
cmdq_task_insert_into_thread(task);
smp_mb(); /* modify jump before enable thread */
}
- writel((task->pa_base + pkt->cmd_buf_size) >> cmdq->pdata->shift,
- thread->base + CMDQ_THR_END_ADDR);
+ gce_addr = cmdq_convert_gce_addr(task->pa_base + pkt->cmd_buf_size, cmdq->pdata);
+ writel(gce_addr, thread->base + CMDQ_THR_END_ADDR);
cmdq_thread_resume(thread);
}
list_move_tail(&task->list_entry, &thread->task_busy_list);
diff --git a/drivers/md/dm-cache-metadata.c b/drivers/md/dm-cache-metadata.c
index 24cd87fddf75..1328327d1d37 100644
--- a/drivers/md/dm-cache-metadata.c
+++ b/drivers/md/dm-cache-metadata.c
@@ -1023,6 +1023,12 @@ static bool cmd_write_lock(struct dm_cache_metadata *cmd)
return; \
} while (0)
+#define WRITE_LOCK_OR_GOTO(cmd, label) \
+ do { \
+ if (!cmd_write_lock((cmd))) \
+ goto label; \
+ } while (0)
+
#define WRITE_UNLOCK(cmd) \
up_write(&(cmd)->root_lock)
@@ -1747,17 +1753,6 @@ int dm_cache_write_hints(struct dm_cache_metadata *cmd, struct dm_cache_policy *
return r;
}
-int dm_cache_metadata_all_clean(struct dm_cache_metadata *cmd, bool *result)
-{
- int r;
-
- READ_LOCK(cmd);
- r = blocks_are_unmapped_or_clean(cmd, 0, cmd->cache_blocks, result);
- READ_UNLOCK(cmd);
-
- return r;
-}
-
void dm_cache_metadata_set_read_only(struct dm_cache_metadata *cmd)
{
WRITE_LOCK_VOID(cmd);
@@ -1824,11 +1819,8 @@ int dm_cache_metadata_abort(struct dm_cache_metadata *cmd)
new_bm = dm_block_manager_create(cmd->bdev, DM_CACHE_METADATA_BLOCK_SIZE << SECTOR_SHIFT,
CACHE_MAX_CONCURRENT_LOCKS);
- WRITE_LOCK(cmd);
- if (cmd->fail_io) {
- WRITE_UNLOCK(cmd);
- goto out;
- }
+ /* cmd_write_lock() already checks fail_io with cmd->root_lock held */
+ WRITE_LOCK_OR_GOTO(cmd, out);
__destroy_persistent_data_objects(cmd, false);
old_bm = cmd->bm;
diff --git a/drivers/md/dm-cache-metadata.h b/drivers/md/dm-cache-metadata.h
index 57afc7047947..24e4af14fcca 100644
--- a/drivers/md/dm-cache-metadata.h
+++ b/drivers/md/dm-cache-metadata.h
@@ -138,11 +138,6 @@ void dm_cache_dump(struct dm_cache_metadata *cmd);
*/
int dm_cache_write_hints(struct dm_cache_metadata *cmd, struct dm_cache_policy *p);
-/*
- * Query method. Are all the blocks in the cache clean?
- */
-int dm_cache_metadata_all_clean(struct dm_cache_metadata *cmd, bool *result);
-
int dm_cache_metadata_needs_check(struct dm_cache_metadata *cmd, bool *result);
int dm_cache_metadata_set_needs_check(struct dm_cache_metadata *cmd);
void dm_cache_metadata_set_read_only(struct dm_cache_metadata *cmd);
diff --git a/drivers/md/dm-cache-policy-smq.c b/drivers/md/dm-cache-policy-smq.c
index 2ed894155cab..d81a87142cac 100644
--- a/drivers/md/dm-cache-policy-smq.c
+++ b/drivers/md/dm-cache-policy-smq.c
@@ -1589,14 +1589,18 @@ static int smq_invalidate_mapping(struct dm_cache_policy *p, dm_cblock_t cblock)
{
struct smq_policy *mq = to_smq_policy(p);
struct entry *e = get_entry(&mq->cache_alloc, from_cblock(cblock));
+ unsigned long flags;
if (!e->allocated)
return -ENODATA;
+ spin_lock_irqsave(&mq->lock, flags);
// FIXME: what if this block has pending background work?
del_queue(mq, e);
h_remove(&mq->table, e);
free_entry(&mq->cache_alloc, e);
+ spin_unlock_irqrestore(&mq->lock, flags);
+
return 0;
}
diff --git a/drivers/md/dm-cache-target.c b/drivers/md/dm-cache-target.c
index 6aa4095dc587..3a7881c0a5b1 100644
--- a/drivers/md/dm-cache-target.c
+++ b/drivers/md/dm-cache-target.c
@@ -406,6 +406,12 @@ struct cache {
mempool_t migration_pool;
struct bio_set bs;
+
+ /*
+ * Cache_size entries. Set bits indicate blocks mapped beyond the
+ * target length, which are marked for invalidation.
+ */
+ unsigned long *invalid_bitset;
};
struct per_bio_data {
@@ -1456,11 +1462,19 @@ static void invalidate_complete(struct dm_cache_migration *mg, bool success)
struct cache *cache = mg->cache;
bio_list_init(&bios);
- if (dm_cell_unlock_v2(cache->prison, mg->cell, &bios))
- free_prison_cell(cache, mg->cell);
+ if (mg->cell) {
+ if (dm_cell_unlock_v2(cache->prison, mg->cell, &bios))
+ free_prison_cell(cache, mg->cell);
+ }
- if (!success && mg->overwrite_bio)
- bio_io_error(mg->overwrite_bio);
+ if (mg->overwrite_bio) {
+ // Set generic error if the bio hasn't been issued yet,
+ // e.g., invalidation or metadata commit failed before bio
+ // submission. Otherwise preserve the bio's own error status.
+ if (!success && !mg->overwrite_bio->bi_status)
+ mg->overwrite_bio->bi_status = BLK_STS_IOERR;
+ bio_endio(mg->overwrite_bio);
+ }
free_migration(mg);
defer_bios(cache, &bios);
@@ -1500,6 +1514,24 @@ static int invalidate_cblock(struct cache *cache, dm_cblock_t cblock)
return r;
}
+static void invalidate_committed(struct work_struct *ws)
+{
+ struct dm_cache_migration *mg = ws_to_mg(ws);
+ struct cache *cache = mg->cache;
+ struct bio *bio = mg->overwrite_bio;
+ struct per_bio_data *pb = get_per_bio_data(bio);
+
+ if (mg->k.input) {
+ invalidate_complete(mg, false);
+ return;
+ }
+
+ init_continuation(&mg->k, invalidate_completed);
+ remap_to_origin_clear_discard(cache, bio, mg->invalidate_oblock);
+ dm_hook_bio(&pb->hook_info, bio, overwrite_endio, mg);
+ dm_submit_bio_remap(bio, NULL);
+}
+
static void invalidate_remove(struct work_struct *ws)
{
int r;
@@ -1512,10 +1544,8 @@ static void invalidate_remove(struct work_struct *ws)
return;
}
- init_continuation(&mg->k, invalidate_completed);
+ init_continuation(&mg->k, invalidate_committed);
continue_after_commit(&cache->committer, &mg->k);
- remap_to_origin_clear_discard(cache, mg->overwrite_bio, mg->invalidate_oblock);
- mg->overwrite_bio = NULL;
schedule_commit(&cache->committer);
}
@@ -1533,6 +1563,15 @@ static int invalidate_lock(struct dm_cache_migration *mg)
READ_WRITE_LOCK_LEVEL, prealloc, &mg->cell);
if (r < 0) {
free_prison_cell(cache, prealloc);
+
+ /* Defer the bio for retrying the cell lock */
+ if (mg->overwrite_bio) {
+ struct bio *bio = mg->overwrite_bio;
+
+ mg->overwrite_bio = NULL;
+ defer_bio(cache, bio);
+ }
+
invalidate_complete(mg, false);
return r;
}
@@ -1695,6 +1734,7 @@ static int map_bio(struct cache *cache, struct bio *bio, dm_oblock_t block,
bio_drop_shared_lock(cache, bio);
atomic_inc(&cache->stats.demotion);
invalidate_start(cache, cblock, block, bio);
+ return DM_MAPIO_SUBMITTED;
} else
remap_to_origin_clear_discard(cache, bio, block);
} else {
@@ -1922,6 +1962,9 @@ static void __destroy(struct cache *cache)
if (cache->discard_bitset)
free_bitset(cache->discard_bitset);
+ if (cache->invalid_bitset)
+ free_bitset(cache->invalid_bitset);
+
if (cache->copier)
dm_kcopyd_client_destroy(cache->copier);
@@ -2465,23 +2508,8 @@ static int cache_create(struct cache_args *ca, struct cache **result)
goto bad;
}
- if (passthrough_mode(cache)) {
- bool all_clean;
-
- r = dm_cache_metadata_all_clean(cache->cmd, &all_clean);
- if (r) {
- *error = "dm_cache_metadata_all_clean() failed";
- goto bad;
- }
-
- if (!all_clean) {
- *error = "Cannot enter passthrough mode unless all blocks are clean";
- r = -EINVAL;
- goto bad;
- }
-
+ if (passthrough_mode(cache))
policy_allow_migrations(cache->policy, false);
- }
spin_lock_init(&cache->lock);
bio_list_init(&cache->deferred_bios);
@@ -2510,6 +2538,13 @@ static int cache_create(struct cache_args *ca, struct cache **result)
}
clear_bitset(cache->discard_bitset, from_dblock(cache->discard_nr_blocks));
+ cache->invalid_bitset = alloc_bitset(from_cblock(cache->cache_size));
+ if (!cache->invalid_bitset) {
+ *error = "could not allocate bitset for invalid blocks";
+ goto bad;
+ }
+ clear_bitset(cache->invalid_bitset, from_cblock(cache->cache_size));
+
cache->copier = dm_kcopyd_client_create(&dm_kcopyd_throttle);
if (IS_ERR(cache->copier)) {
*error = "could not create kcopyd client";
@@ -2800,6 +2835,12 @@ static int load_mapping(void *context, dm_oblock_t oblock, dm_cblock_t cblock,
struct cache *cache = context;
if (dirty) {
+ if (passthrough_mode(cache)) {
+ DMERR("%s: cannot enter passthrough mode unless all blocks are clean",
+ cache_device_name(cache));
+ return -EBUSY;
+ }
+
set_bit(from_cblock(cblock), cache->dirty_bitset);
atomic_inc(&cache->nr_dirty);
} else
@@ -2808,6 +2849,24 @@ static int load_mapping(void *context, dm_oblock_t oblock, dm_cblock_t cblock,
return policy_load_mapping(cache->policy, oblock, cblock, dirty, hint, hint_valid);
}
+static int load_filtered_mapping(void *context, dm_oblock_t oblock, dm_cblock_t cblock,
+ bool dirty, uint32_t hint, bool hint_valid)
+{
+ struct cache *cache = context;
+
+ if (from_oblock(oblock) >= from_oblock(cache->origin_blocks)) {
+ if (dirty) {
+ DMERR("%s: unable to shrink origin; cache block %u is dirty",
+ cache_device_name(cache), from_cblock(cblock));
+ return -EFBIG;
+ }
+ set_bit(from_cblock(cblock), cache->invalid_bitset);
+ return 0;
+ }
+
+ return load_mapping(context, oblock, cblock, dirty, hint, hint_valid);
+}
+
/*
* The discard block size in the on disk metadata is not
* necessarily the same as we're currently using. So we have to
@@ -2962,6 +3021,24 @@ static int resize_cache_dev(struct cache *cache, dm_cblock_t new_size)
return 0;
}
+static int truncate_oblocks(struct cache *cache)
+{
+ uint32_t nr_blocks = from_cblock(cache->cache_size);
+ uint32_t i;
+ int r;
+
+ for_each_set_bit(i, cache->invalid_bitset, nr_blocks) {
+ r = dm_cache_remove_mapping(cache->cmd, to_cblock(i));
+ if (r) {
+ DMERR_LIMIT("%s: invalidation failed; couldn't update on disk metadata",
+ cache_device_name(cache));
+ return r;
+ }
+ }
+
+ return 0;
+}
+
static int cache_preresume(struct dm_target *ti)
{
int r = 0;
@@ -2986,11 +3063,25 @@ static int cache_preresume(struct dm_target *ti)
}
if (!cache->loaded_mappings) {
+ /*
+ * The fast device could have been resized since the last
+ * failed preresume attempt. To be safe we start by a blank
+ * bitset for cache blocks.
+ */
+ clear_bitset(cache->invalid_bitset, from_cblock(cache->cache_size));
+
r = dm_cache_load_mappings(cache->cmd, cache->policy,
- load_mapping, cache);
+ load_filtered_mapping, cache);
if (r) {
DMERR("%s: could not load cache mappings", cache_device_name(cache));
- metadata_operation_failed(cache, "dm_cache_load_mappings", r);
+ if (r != -EFBIG && r != -EBUSY)
+ metadata_operation_failed(cache, "dm_cache_load_mappings", r);
+ return r;
+ }
+
+ r = truncate_oblocks(cache);
+ if (r) {
+ metadata_operation_failed(cache, "dm_cache_remove_mapping", r);
return r;
}
@@ -3450,7 +3541,7 @@ static void cache_io_hints(struct dm_target *ti, struct queue_limits *limits)
static struct target_type cache_target = {
.name = "cache",
- .version = {2, 2, 0},
+ .version = {2, 3, 0},
.module = THIS_MODULE,
.ctr = cache_ctr,
.dtr = cache_dtr,
diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
index b37bbe762500..423269cbdd2b 100644
--- a/drivers/md/dm-init.c
+++ b/drivers/md/dm-init.c
@@ -303,8 +303,10 @@ static int __init dm_init_init(void)
}
}
- if (waitfor[0])
+ if (waitfor[0]) {
+ wait_for_device_probe();
DMINFO("all devices available");
+ }
list_for_each_entry(dev, &devices, list) {
if (dm_early_create(&dev->dmi, dev->table,
diff --git a/drivers/md/dm-log.c b/drivers/md/dm-log.c
index bced5a783ee3..4a1369b8f44a 100644
--- a/drivers/md/dm-log.c
+++ b/drivers/md/dm-log.c
@@ -373,7 +373,7 @@ static int create_log_context(struct dm_dirty_log *log, struct dm_target *ti,
struct log_c *lc;
uint32_t region_size;
- unsigned int region_count;
+ sector_t region_count;
size_t bitset_size, buf_size;
int r;
char dummy;
@@ -401,6 +401,10 @@ static int create_log_context(struct dm_dirty_log *log, struct dm_target *ti,
}
region_count = dm_sector_div_up(ti->len, region_size);
+ if (region_count > UINT_MAX) {
+ DMWARN("region count exceeds limit of %u", UINT_MAX);
+ return -EINVAL;
+ }
lc = kmalloc(sizeof(*lc), GFP_KERNEL);
if (!lc) {
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 526390acd39e..1aff3e541ceb 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -470,6 +470,17 @@ int mddev_suspend(struct mddev *mddev, bool interruptible)
}
percpu_ref_kill(&mddev->active_io);
+
+ /*
+ * RAID456 IO can sleep in wait_for_reshape while still holding an
+ * active_io reference. If reshape is already interrupted or frozen,
+ * wake those waiters so they can abort and drop the reference instead
+ * of deadlocking suspend.
+ */
+ if (mddev->pers && mddev->pers->prepare_suspend &&
+ reshape_interrupted(mddev))
+ mddev->pers->prepare_suspend(mddev);
+
if (interruptible)
err = wait_event_interruptible(mddev->sb_wait,
percpu_ref_is_zero(&mddev->active_io));
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 093b04e6be67..eb583df45ecb 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -61,7 +61,7 @@ static int check_and_add_serial(struct md_rdev *rdev, struct r1bio *r1_bio,
unsigned long flags;
int ret = 0;
sector_t lo = r1_bio->sector;
- sector_t hi = lo + r1_bio->sectors;
+ sector_t hi = lo + r1_bio->sectors - 1;
struct serial_in_rdev *serial = &rdev->serial[idx];
spin_lock_irqsave(&serial->serial_lock, flags);
@@ -453,7 +453,7 @@ static void raid1_end_write_request(struct bio *bio)
int mirror = find_bio_disk(r1_bio, bio);
struct md_rdev *rdev = conf->mirrors[mirror].rdev;
sector_t lo = r1_bio->sector;
- sector_t hi = r1_bio->sector + r1_bio->sectors;
+ sector_t hi = r1_bio->sector + r1_bio->sectors - 1;
bool ignore_error = !raid1_should_handle_error(bio) ||
(bio->bi_status && bio_op(bio) == REQ_OP_DISCARD);
diff --git a/drivers/media/i2c/og01a1b.c b/drivers/media/i2c/og01a1b.c
index b7d0b677975d..68573122cd6e 100644
--- a/drivers/media/i2c/og01a1b.c
+++ b/drivers/media/i2c/og01a1b.c
@@ -1,7 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
// Copyright (c) 2022 Intel Corporation.
-#include <linux/unaligned.h>
#include <linux/acpi.h>
#include <linux/clk.h>
#include <linux/delay.h>
@@ -10,6 +9,8 @@
#include <linux/module.h>
#include <linux/pm_runtime.h>
#include <linux/regulator/consumer.h>
+#include <linux/unaligned.h>
+
#include <media/v4l2-ctrls.h>
#include <media/v4l2-device.h>
#include <media/v4l2-fwnode.h>
@@ -421,6 +422,7 @@ static const struct og01a1b_mode supported_modes[] = {
};
struct og01a1b {
+ struct device *dev;
struct clk *xvclk;
struct gpio_desc *reset_gpio;
struct regulator *avdd;
@@ -512,7 +514,6 @@ static int og01a1b_write_reg(struct og01a1b *og01a1b, u16 reg, u16 len, u32 val)
static int og01a1b_write_reg_list(struct og01a1b *og01a1b,
const struct og01a1b_reg_list *r_list)
{
- struct i2c_client *client = v4l2_get_subdevdata(&og01a1b->sd);
unsigned int i;
int ret;
@@ -520,7 +521,7 @@ static int og01a1b_write_reg_list(struct og01a1b *og01a1b,
ret = og01a1b_write_reg(og01a1b, r_list->regs[i].address, 1,
r_list->regs[i].val);
if (ret) {
- dev_err_ratelimited(&client->dev,
+ dev_err_ratelimited(og01a1b->dev,
"failed to write reg 0x%4.4x. error = %d",
r_list->regs[i].address, ret);
return ret;
@@ -544,7 +545,6 @@ static int og01a1b_set_ctrl(struct v4l2_ctrl *ctrl)
{
struct og01a1b *og01a1b = container_of(ctrl->handler,
struct og01a1b, ctrl_handler);
- struct i2c_client *client = v4l2_get_subdevdata(&og01a1b->sd);
s64 exposure_max;
int ret = 0;
@@ -560,7 +560,7 @@ static int og01a1b_set_ctrl(struct v4l2_ctrl *ctrl)
}
/* V4L2 controls values will be applied only when power is already up */
- if (!pm_runtime_get_if_in_use(&client->dev))
+ if (!pm_runtime_get_if_in_use(og01a1b->dev))
return 0;
switch (ctrl->id) {
@@ -596,7 +596,7 @@ static int og01a1b_set_ctrl(struct v4l2_ctrl *ctrl)
break;
}
- pm_runtime_put(&client->dev);
+ pm_runtime_put(og01a1b->dev);
return ret;
}
@@ -688,7 +688,6 @@ static void og01a1b_update_pad_format(const struct og01a1b_mode *mode,
static int og01a1b_start_streaming(struct og01a1b *og01a1b)
{
- struct i2c_client *client = v4l2_get_subdevdata(&og01a1b->sd);
const struct og01a1b_reg_list *reg_list;
int link_freq_index, ret;
@@ -697,14 +696,14 @@ static int og01a1b_start_streaming(struct og01a1b *og01a1b)
ret = og01a1b_write_reg_list(og01a1b, reg_list);
if (ret) {
- dev_err(&client->dev, "failed to set plls");
+ dev_err(og01a1b->dev, "failed to set plls");
return ret;
}
reg_list = &og01a1b->cur_mode->reg_list;
ret = og01a1b_write_reg_list(og01a1b, reg_list);
if (ret) {
- dev_err(&client->dev, "failed to set mode");
+ dev_err(og01a1b->dev, "failed to set mode");
return ret;
}
@@ -716,7 +715,7 @@ static int og01a1b_start_streaming(struct og01a1b *og01a1b)
OG01A1B_REG_VALUE_08BIT,
OG01A1B_MODE_STREAMING);
if (ret) {
- dev_err(&client->dev, "failed to set stream");
+ dev_err(og01a1b->dev, "failed to set stream");
return ret;
}
@@ -725,22 +724,19 @@ static int og01a1b_start_streaming(struct og01a1b *og01a1b)
static void og01a1b_stop_streaming(struct og01a1b *og01a1b)
{
- struct i2c_client *client = v4l2_get_subdevdata(&og01a1b->sd);
-
if (og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT,
OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STANDBY))
- dev_err(&client->dev, "failed to set stream");
+ dev_err(og01a1b->dev, "failed to set stream");
}
static int og01a1b_set_stream(struct v4l2_subdev *sd, int enable)
{
struct og01a1b *og01a1b = to_og01a1b(sd);
- struct i2c_client *client = v4l2_get_subdevdata(sd);
int ret = 0;
mutex_lock(&og01a1b->mutex);
if (enable) {
- ret = pm_runtime_resume_and_get(&client->dev);
+ ret = pm_runtime_resume_and_get(og01a1b->dev);
if (ret) {
mutex_unlock(&og01a1b->mutex);
return ret;
@@ -750,11 +746,11 @@ static int og01a1b_set_stream(struct v4l2_subdev *sd, int enable)
if (ret) {
enable = 0;
og01a1b_stop_streaming(og01a1b);
- pm_runtime_put(&client->dev);
+ pm_runtime_put(og01a1b->dev);
}
} else {
og01a1b_stop_streaming(og01a1b);
- pm_runtime_put(&client->dev);
+ pm_runtime_put(og01a1b->dev);
}
mutex_unlock(&og01a1b->mutex);
@@ -889,7 +885,6 @@ static const struct v4l2_subdev_internal_ops og01a1b_internal_ops = {
static int og01a1b_identify_module(struct og01a1b *og01a1b)
{
- struct i2c_client *client = v4l2_get_subdevdata(&og01a1b->sd);
int ret;
u32 val;
@@ -899,7 +894,7 @@ static int og01a1b_identify_module(struct og01a1b *og01a1b)
return ret;
if (val != OG01A1B_CHIP_ID) {
- dev_err(&client->dev, "chip id mismatch: %x!=%x",
+ dev_err(og01a1b->dev, "chip id mismatch: %x!=%x",
OG01A1B_CHIP_ID, val);
return -ENXIO;
}
@@ -909,8 +904,7 @@ static int og01a1b_identify_module(struct og01a1b *og01a1b)
static int og01a1b_check_hwcfg(struct og01a1b *og01a1b)
{
- struct i2c_client *client = v4l2_get_subdevdata(&og01a1b->sd);
- struct device *dev = &client->dev;
+ struct device *dev = og01a1b->dev;
struct fwnode_handle *ep;
struct fwnode_handle *fwnode = dev_fwnode(dev);
struct v4l2_fwnode_endpoint bus_cfg = {
@@ -1064,9 +1058,10 @@ static void og01a1b_remove(struct i2c_client *client)
struct og01a1b *og01a1b = to_og01a1b(sd);
v4l2_async_unregister_subdev(sd);
+ v4l2_subdev_cleanup(&og01a1b->sd);
media_entity_cleanup(&sd->entity);
v4l2_ctrl_handler_free(sd->ctrl_handler);
- pm_runtime_disable(&client->dev);
+ pm_runtime_disable(og01a1b->dev);
mutex_destroy(&og01a1b->mutex);
}
@@ -1079,34 +1074,36 @@ static int og01a1b_probe(struct i2c_client *client)
if (!og01a1b)
return -ENOMEM;
+ og01a1b->dev = &client->dev;
+
v4l2_i2c_subdev_init(&og01a1b->sd, client, &og01a1b_subdev_ops);
- og01a1b->xvclk = devm_clk_get_optional(&client->dev, NULL);
+ og01a1b->xvclk = devm_clk_get_optional(og01a1b->dev, NULL);
if (IS_ERR(og01a1b->xvclk)) {
ret = PTR_ERR(og01a1b->xvclk);
- dev_err(&client->dev, "failed to get xvclk clock: %d\n", ret);
+ dev_err(og01a1b->dev, "failed to get xvclk clock: %d\n", ret);
return ret;
}
ret = og01a1b_check_hwcfg(og01a1b);
if (ret) {
- dev_err(&client->dev, "failed to check HW configuration: %d",
+ dev_err(og01a1b->dev, "failed to check HW configuration: %d",
ret);
return ret;
}
- og01a1b->reset_gpio = devm_gpiod_get_optional(&client->dev, "reset",
+ og01a1b->reset_gpio = devm_gpiod_get_optional(og01a1b->dev, "reset",
GPIOD_OUT_LOW);
if (IS_ERR(og01a1b->reset_gpio)) {
- dev_err(&client->dev, "cannot get reset GPIO\n");
+ dev_err(og01a1b->dev, "cannot get reset GPIO\n");
return PTR_ERR(og01a1b->reset_gpio);
}
- og01a1b->avdd = devm_regulator_get_optional(&client->dev, "avdd");
+ og01a1b->avdd = devm_regulator_get_optional(og01a1b->dev, "avdd");
if (IS_ERR(og01a1b->avdd)) {
ret = PTR_ERR(og01a1b->avdd);
if (ret != -ENODEV) {
- dev_err_probe(&client->dev, ret,
+ dev_err_probe(og01a1b->dev, ret,
"Failed to get 'avdd' regulator\n");
return ret;
}
@@ -1114,11 +1111,11 @@ static int og01a1b_probe(struct i2c_client *client)
og01a1b->avdd = NULL;
}
- og01a1b->dovdd = devm_regulator_get_optional(&client->dev, "dovdd");
+ og01a1b->dovdd = devm_regulator_get_optional(og01a1b->dev, "dovdd");
if (IS_ERR(og01a1b->dovdd)) {
ret = PTR_ERR(og01a1b->dovdd);
if (ret != -ENODEV) {
- dev_err_probe(&client->dev, ret,
+ dev_err_probe(og01a1b->dev, ret,
"Failed to get 'dovdd' regulator\n");
return ret;
}
@@ -1126,11 +1123,11 @@ static int og01a1b_probe(struct i2c_client *client)
og01a1b->dovdd = NULL;
}
- og01a1b->dvdd = devm_regulator_get_optional(&client->dev, "dvdd");
+ og01a1b->dvdd = devm_regulator_get_optional(og01a1b->dev, "dvdd");
if (IS_ERR(og01a1b->dvdd)) {
ret = PTR_ERR(og01a1b->dvdd);
if (ret != -ENODEV) {
- dev_err_probe(&client->dev, ret,
+ dev_err_probe(og01a1b->dev, ret,
"Failed to get 'dvdd' regulator\n");
return ret;
}
@@ -1139,13 +1136,13 @@ static int og01a1b_probe(struct i2c_client *client)
}
/* The sensor must be powered on to read the CHIP_ID register */
- ret = og01a1b_power_on(&client->dev);
+ ret = og01a1b_power_on(og01a1b->dev);
if (ret)
return ret;
ret = og01a1b_identify_module(og01a1b);
if (ret) {
- dev_err(&client->dev, "failed to find sensor: %d", ret);
+ dev_err(og01a1b->dev, "failed to find sensor: %d", ret);
goto power_off;
}
@@ -1153,7 +1150,7 @@ static int og01a1b_probe(struct i2c_client *client)
og01a1b->cur_mode = &supported_modes[0];
ret = og01a1b_init_controls(og01a1b);
if (ret) {
- dev_err(&client->dev, "failed to init controls: %d", ret);
+ dev_err(og01a1b->dev, "failed to init controls: %d", ret);
goto probe_error_v4l2_ctrl_handler_free;
}
@@ -1164,24 +1161,34 @@ static int og01a1b_probe(struct i2c_client *client)
og01a1b->pad.flags = MEDIA_PAD_FL_SOURCE;
ret = media_entity_pads_init(&og01a1b->sd.entity, 1, &og01a1b->pad);
if (ret) {
- dev_err(&client->dev, "failed to init entity pads: %d", ret);
+ dev_err(og01a1b->dev, "failed to init entity pads: %d", ret);
goto probe_error_v4l2_ctrl_handler_free;
}
+ ret = v4l2_subdev_init_finalize(&og01a1b->sd);
+ if (ret < 0) {
+ dev_err_probe(og01a1b->dev, ret,
+ "failed to finalize subdevice init\n");
+ goto probe_error_media_entity_cleanup;
+ }
+
ret = v4l2_async_register_subdev_sensor(&og01a1b->sd);
if (ret < 0) {
- dev_err(&client->dev, "failed to register V4L2 subdev: %d",
+ dev_err(og01a1b->dev, "failed to register V4L2 subdev: %d",
ret);
- goto probe_error_media_entity_cleanup;
+ goto probe_error_v4l2_subdev_cleanup;
}
/* Enable runtime PM and turn off the device */
- pm_runtime_set_active(&client->dev);
- pm_runtime_enable(&client->dev);
- pm_runtime_idle(&client->dev);
+ pm_runtime_set_active(og01a1b->dev);
+ pm_runtime_enable(og01a1b->dev);
+ pm_runtime_idle(og01a1b->dev);
return 0;
+probe_error_v4l2_subdev_cleanup:
+ v4l2_subdev_cleanup(&og01a1b->sd);
+
probe_error_media_entity_cleanup:
media_entity_cleanup(&og01a1b->sd.entity);
@@ -1190,7 +1197,7 @@ static int og01a1b_probe(struct i2c_client *client)
mutex_destroy(&og01a1b->mutex);
power_off:
- og01a1b_power_off(&client->dev);
+ og01a1b_power_off(og01a1b->dev);
return ret;
}
diff --git a/drivers/media/rc/lirc_dev.c b/drivers/media/rc/lirc_dev.c
index 314f64420f62..7d4942925993 100644
--- a/drivers/media/rc/lirc_dev.c
+++ b/drivers/media/rc/lirc_dev.c
@@ -816,28 +816,23 @@ void __exit lirc_dev_exit(void)
struct rc_dev *rc_dev_get_from_fd(int fd, bool write)
{
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
struct lirc_fh *fh;
struct rc_dev *dev;
- if (!fd_file(f))
+ if (fd_empty(f))
return ERR_PTR(-EBADF);
- if (fd_file(f)->f_op != &lirc_fops) {
- fdput(f);
+ if (fd_file(f)->f_op != &lirc_fops)
return ERR_PTR(-EINVAL);
- }
- if (write && !(fd_file(f)->f_mode & FMODE_WRITE)) {
- fdput(f);
+ if (write && !(fd_file(f)->f_mode & FMODE_WRITE))
return ERR_PTR(-EPERM);
- }
fh = fd_file(f)->private_data;
dev = fh->rc;
get_device(&dev->dev);
- fdput(f);
return dev;
}
diff --git a/drivers/memory/tegra/tegra124-emc.c b/drivers/memory/tegra/tegra124-emc.c
index 03f1daa2d132..71d20b5916d8 100644
--- a/drivers/memory/tegra/tegra124-emc.c
+++ b/drivers/memory/tegra/tegra124-emc.c
@@ -608,7 +608,7 @@ static int tegra_emc_prepare_timing_change(struct tegra_emc *emc,
if ((last->emc_mode_1 & 0x1) == (timing->emc_mode_1 & 0x1))
dll_change = DLL_CHANGE_NONE;
- else if (timing->emc_mode_1 & 0x1)
+ else if (!(timing->emc_mode_1 & 0x1))
dll_change = DLL_CHANGE_ON;
else
dll_change = DLL_CHANGE_OFF;
diff --git a/drivers/memory/tegra/tegra30-emc.c b/drivers/memory/tegra/tegra30-emc.c
index 921dce1b8bc6..4981b7fa0f78 100644
--- a/drivers/memory/tegra/tegra30-emc.c
+++ b/drivers/memory/tegra/tegra30-emc.c
@@ -554,14 +554,14 @@ static int emc_prepare_timing_change(struct tegra_emc *emc, unsigned long rate)
emc->emc_cfg = readl_relaxed(emc->regs + EMC_CFG);
emc_dbg = readl_relaxed(emc->regs + EMC_DBG);
- if (emc->dll_on == !!(timing->emc_mode_1 & 0x1))
+ if (emc->dll_on == !(timing->emc_mode_1 & 0x1))
dll_change = DLL_CHANGE_NONE;
- else if (timing->emc_mode_1 & 0x1)
+ else if (!(timing->emc_mode_1 & 0x1))
dll_change = DLL_CHANGE_ON;
else
dll_change = DLL_CHANGE_OFF;
- emc->dll_on = !!(timing->emc_mode_1 & 0x1);
+ emc->dll_on = !(timing->emc_mode_1 & 0x1);
if (timing->data[80] && !readl_relaxed(emc->regs + EMC_ZCAL_INTERVAL))
emc->zcal_long = true;
diff --git a/drivers/mfd/mc13xxx-core.c b/drivers/mfd/mc13xxx-core.c
index 920797b806ce..786eab3b2d03 100644
--- a/drivers/mfd/mc13xxx-core.c
+++ b/drivers/mfd/mc13xxx-core.c
@@ -377,7 +377,7 @@ static int mc13xxx_add_subdevice_pdata(struct mc13xxx *mc13xxx,
if (snprintf(buf, sizeof(buf), format, name) > sizeof(buf))
return -E2BIG;
- cell.name = kmemdup(buf, strlen(buf) + 1, GFP_KERNEL);
+ cell.name = devm_kmemdup(mc13xxx->dev, buf, strlen(buf) + 1, GFP_KERNEL);
if (!cell.name)
return -ENOMEM;
diff --git a/drivers/mtd/maps/physmap-gemini.c b/drivers/mtd/maps/physmap-gemini.c
index 9d3b4bf84a1a..1c34b4ef77ea 100644
--- a/drivers/mtd/maps/physmap-gemini.c
+++ b/drivers/mtd/maps/physmap-gemini.c
@@ -181,7 +181,7 @@ int of_flash_probe_gemini(struct platform_device *pdev,
dev_err(dev, "no enabled pin control state\n");
gf->disabled_state = pinctrl_lookup_state(gf->p, "disabled");
- if (IS_ERR(gf->enabled_state)) {
+ if (IS_ERR(gf->disabled_state)) {
dev_err(dev, "no disabled pin control state\n");
} else {
ret = pinctrl_select_state(gf->p, gf->disabled_state);
diff --git a/drivers/mtd/nand/raw/sunxi_nand.c b/drivers/mtd/nand/raw/sunxi_nand.c
index ac887754b98e..136d67af9717 100644
--- a/drivers/mtd/nand/raw/sunxi_nand.c
+++ b/drivers/mtd/nand/raw/sunxi_nand.c
@@ -886,9 +886,9 @@ static void sunxi_nfc_hw_ecc_read_extra_oob(struct nand_chip *nand,
if (len <= 0)
return;
- if (!cur_off || *cur_off != offset)
- nand_change_read_column_op(nand, mtd->writesize, NULL, 0,
- false);
+ if (!cur_off || *cur_off != (offset + mtd->writesize))
+ nand_change_read_column_op(nand, mtd->writesize + offset,
+ NULL, 0, false);
if (!randomize)
sunxi_nfc_read_buf(nand, oob + offset, len);
diff --git a/drivers/mtd/parsers/ofpart_core.c b/drivers/mtd/parsers/ofpart_core.c
index 09961c6f3949..321002a1d0ca 100644
--- a/drivers/mtd/parsers/ofpart_core.c
+++ b/drivers/mtd/parsers/ofpart_core.c
@@ -71,7 +71,7 @@ static int parse_fixed_partitions(struct mtd_info *master,
dedicated = false;
}
} else { /* Partition */
- ofpart_node = mtd_node;
+ ofpart_node = of_node_get(mtd_node);
}
of_id = of_match_node(parse_ofpart_match_table, ofpart_node);
@@ -191,11 +191,11 @@ static int parse_fixed_partitions(struct mtd_info *master,
ofpart_fail:
pr_err("%s: error parsing ofpart partition %pOF (%pOF)\n",
master->name, pp, mtd_node);
+ of_node_put(pp);
ret = -EINVAL;
ofpart_none:
if (dedicated)
of_node_put(ofpart_node);
- of_node_put(pp);
kfree(parts);
return ret;
}
diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index a3e6a8c28dfb..d666e044b0c4 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -2416,7 +2416,7 @@ static int spi_nor_spimem_check_readop(struct spi_nor *nor,
/* convert the dummy cycles to the number of bytes */
op.dummy.nbytes = (read->num_mode_clocks + read->num_wait_states) *
op.dummy.buswidth / 8;
- if (spi_nor_protocol_is_dtr(nor->read_proto))
+ if (spi_nor_protocol_is_dtr(read->proto))
op.dummy.nbytes *= 2;
return spi_nor_spimem_check_op(nor, &op);
diff --git a/drivers/mtd/spi-nor/core.h b/drivers/mtd/spi-nor/core.h
index 1516b6d0dc37..df07ded38260 100644
--- a/drivers/mtd/spi-nor/core.h
+++ b/drivers/mtd/spi-nor/core.h
@@ -408,7 +408,11 @@ struct spi_nor_flash_parameter {
* flash parameters when information provided by the flash_info
* table is incomplete or wrong.
* @post_bfpt: called after the BFPT table has been parsed
- * @post_sfdp: called after SFDP has been parsed (is also called for SPI NORs
+ * @smpt_read_dummy: called during SMPT table is being parsed. Used to fix the
+ * number of dummy cycles in read register ops.
+ * @smpt_map_id: called after map ID in SMPT table has been determined for the
+ * case the map ID is wrong and needs to be fixed.
+ * @post_sfdp: called after SFDP has been parsed (is not called for SPI NORs
* that do not support RDSFDP). Typically used to tweak various
* parameters that could not be extracted by other means (i.e.
* when information provided by the SFDP/flash_info tables are
@@ -425,6 +429,8 @@ struct spi_nor_fixups {
int (*post_bfpt)(struct spi_nor *nor,
const struct sfdp_parameter_header *bfpt_header,
const struct sfdp_bfpt *bfpt);
+ void (*smpt_read_dummy)(const struct spi_nor *nor, u8 *read_dummy);
+ void (*smpt_map_id)(const struct spi_nor *nor, u8 *map_id);
int (*post_sfdp)(struct spi_nor *nor);
int (*late_init)(struct spi_nor *nor);
};
diff --git a/drivers/mtd/spi-nor/sfdp.c b/drivers/mtd/spi-nor/sfdp.c
index 5b1117265bd2..20b791568d5f 100644
--- a/drivers/mtd/spi-nor/sfdp.c
+++ b/drivers/mtd/spi-nor/sfdp.c
@@ -695,6 +695,17 @@ static u8 spi_nor_smpt_addr_nbytes(const struct spi_nor *nor, const u32 settings
}
}
+static void spi_nor_smpt_read_dummy_fixups(const struct spi_nor *nor,
+ u8 *read_dummy)
+{
+ if (nor->manufacturer && nor->manufacturer->fixups &&
+ nor->manufacturer->fixups->smpt_read_dummy)
+ nor->manufacturer->fixups->smpt_read_dummy(nor, read_dummy);
+
+ if (nor->info->fixups && nor->info->fixups->smpt_read_dummy)
+ nor->info->fixups->smpt_read_dummy(nor, read_dummy);
+}
+
/**
* spi_nor_smpt_read_dummy() - return the configuration detection command read
* latency, in clock cycles.
@@ -707,11 +718,24 @@ static u8 spi_nor_smpt_read_dummy(const struct spi_nor *nor, const u32 settings)
{
u8 read_dummy = SMPT_CMD_READ_DUMMY(settings);
- if (read_dummy == SMPT_CMD_READ_DUMMY_IS_VARIABLE)
- return nor->read_dummy;
+ if (read_dummy == SMPT_CMD_READ_DUMMY_IS_VARIABLE) {
+ read_dummy = nor->read_dummy;
+ spi_nor_smpt_read_dummy_fixups(nor, &read_dummy);
+ }
+
return read_dummy;
}
+static void spi_nor_smpt_map_id_fixups(const struct spi_nor *nor, u8 *map_id)
+{
+ if (nor->manufacturer && nor->manufacturer->fixups &&
+ nor->manufacturer->fixups->smpt_map_id)
+ nor->manufacturer->fixups->smpt_map_id(nor, map_id);
+
+ if (nor->info->fixups && nor->info->fixups->smpt_map_id)
+ nor->info->fixups->smpt_map_id(nor, map_id);
+}
+
/**
* spi_nor_get_map_in_use() - get the configuration map in use
* @nor: pointer to a 'struct spi_nor'
@@ -765,6 +789,8 @@ static const u32 *spi_nor_get_map_in_use(struct spi_nor *nor, const u32 *smpt,
map_id = map_id << 1 | !!(*buf & read_data_mask);
}
+ spi_nor_smpt_map_id_fixups(nor, &map_id);
+
/*
* If command descriptors are provided, they always precede map
* descriptors in the table. There is no need to start the iteration
diff --git a/drivers/mtd/spi-nor/swp.c b/drivers/mtd/spi-nor/swp.c
index fdc411f2a23c..7208be75d850 100644
--- a/drivers/mtd/spi-nor/swp.c
+++ b/drivers/mtd/spi-nor/swp.c
@@ -27,8 +27,10 @@ static u8 spi_nor_get_sr_tb_mask(struct spi_nor *nor)
{
if (nor->flags & SNOR_F_HAS_SR_TB_BIT6)
return SR_TB_BIT6;
- else
+ else if (nor->flags & SNOR_F_HAS_SR_TB)
return SR_TB_BIT5;
+ else
+ return 0;
}
static u64 spi_nor_get_min_prot_length_sr(struct spi_nor *nor)
diff --git a/drivers/net/bareudp.c b/drivers/net/bareudp.c
index fa2dd76ba3d9..8d6b632371f8 100644
--- a/drivers/net/bareudp.c
+++ b/drivers/net/bareudp.c
@@ -528,6 +528,9 @@ static int bareudp_fill_metadata_dst(struct net_device *dev,
struct in6_addr saddr;
struct socket *sock = rcu_dereference(bareudp->sock);
+ if (!sock)
+ return -ESHUTDOWN;
+
dst = udp_tunnel6_dst_lookup(skb, dev, bareudp->net, sock,
0, &saddr, &info->key,
sport, bareudp->port, info->key.tos,
diff --git a/drivers/net/dsa/realtek/rtl8365mb.c b/drivers/net/dsa/realtek/rtl8365mb.c
index 4cb986988f1a..9621fa4052f8 100644
--- a/drivers/net/dsa/realtek/rtl8365mb.c
+++ b/drivers/net/dsa/realtek/rtl8365mb.c
@@ -216,7 +216,7 @@
(_extint) == 2 ? RTL8365MB_DIGITAL_INTERFACE_SELECT_REG1 : \
0x0)
#define RTL8365MB_DIGITAL_INTERFACE_SELECT_MODE_MASK(_extint) \
- (0xF << (((_extint) % 2)))
+ (0xF << (((_extint) % 2) * 4))
#define RTL8365MB_DIGITAL_INTERFACE_SELECT_MODE_OFFSET(_extint) \
(((_extint) % 2) * 4)
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_pci_func.c b/drivers/net/ethernet/aquantia/atlantic/aq_pci_func.c
index 43c71f6b314f..eb22531be554 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_pci_func.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_pci_func.c
@@ -371,7 +371,7 @@ static void aq_pci_shutdown(struct pci_dev *pdev)
pci_disable_device(pdev);
if (system_state == SYSTEM_POWER_OFF) {
- pci_wake_from_d3(pdev, false);
+ pci_wake_from_d3(pdev, self->aq_hw->aq_nic_cfg->wol);
pci_set_power_state(pdev, PCI_D3hot);
}
}
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 49f6e83d6013..1263d0005873 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -41,15 +41,13 @@
#include "bcmgenet.h"
-/* Maximum number of hardware queues, downsized if needed */
-#define GENET_MAX_MQ_CNT 4
-
/* Default highest priority queue for multi queue support */
-#define GENET_Q0_PRIORITY 0
+#define GENET_Q1_PRIORITY 0
+#define GENET_Q0_PRIORITY 1
-#define GENET_Q16_RX_BD_CNT \
+#define GENET_Q0_RX_BD_CNT \
(TOTAL_DESC - priv->hw_params->rx_queues * priv->hw_params->rx_bds_per_q)
-#define GENET_Q16_TX_BD_CNT \
+#define GENET_Q0_TX_BD_CNT \
(TOTAL_DESC - priv->hw_params->tx_queues * priv->hw_params->tx_bds_per_q)
#define RX_BUF_LENGTH 2048
@@ -104,7 +102,7 @@ static inline void dmadesc_set_addr(struct bcmgenet_priv *priv,
* the platform is explicitly configured for 64-bits/LPAE.
*/
#ifdef CONFIG_PHYS_ADDR_T_64BIT
- if (priv->hw_params->flags & GENET_HAS_40BITS)
+ if (bcmgenet_has_40bits(priv))
bcmgenet_writel(upper_32_bits(addr), d + DMA_DESC_ADDRESS_HI);
#endif
}
@@ -585,7 +583,7 @@ static void bcmgenet_hfb_create_rxnfc_filter(struct bcmgenet_priv *priv,
u16 mask_16;
size_t size;
- f = fs->location;
+ f = fs->location + 1;
if (fs->flow_type & FLOW_MAC_EXT) {
bcmgenet_hfb_insert_data(priv, f, 0,
&fs->h_ext.h_dest, &fs->m_ext.h_dest,
@@ -667,19 +665,14 @@ static void bcmgenet_hfb_create_rxnfc_filter(struct bcmgenet_priv *priv,
}
bcmgenet_hfb_set_filter_length(priv, f, 2 * f_length);
- if (!fs->ring_cookie || fs->ring_cookie == RX_CLS_FLOW_WAKE) {
- /* Ring 0 flows can be handled by the default Descriptor Ring
- * We'll map them to ring 0, but don't enable the filter
- */
+ if (fs->ring_cookie == RX_CLS_FLOW_WAKE)
bcmgenet_hfb_set_filter_rx_queue_mapping(priv, f, 0);
- rule->state = BCMGENET_RXNFC_STATE_DISABLED;
- } else {
+ else
/* Other Rx rings are direct mapped here */
bcmgenet_hfb_set_filter_rx_queue_mapping(priv, f,
fs->ring_cookie);
- bcmgenet_hfb_enable_filter(priv, f);
- rule->state = BCMGENET_RXNFC_STATE_ENABLED;
- }
+ bcmgenet_hfb_enable_filter(priv, f);
+ rule->state = BCMGENET_RXNFC_STATE_ENABLED;
}
/* bcmgenet_hfb_clear
@@ -715,6 +708,10 @@ static void bcmgenet_hfb_clear(struct bcmgenet_priv *priv)
for (i = 0; i < priv->hw_params->hfb_filter_cnt; i++)
bcmgenet_hfb_clear_filter(priv, i);
+
+ /* Enable filter 0 to send default flow to ring 0 */
+ bcmgenet_hfb_set_filter_length(priv, 0, 4);
+ bcmgenet_hfb_enable_filter(priv, 0);
}
static void bcmgenet_hfb_init(struct bcmgenet_priv *priv)
@@ -819,20 +816,16 @@ static int bcmgenet_get_coalesce(struct net_device *dev,
unsigned int i;
ec->tx_max_coalesced_frames =
- bcmgenet_tdma_ring_readl(priv, DESC_INDEX,
- DMA_MBUF_DONE_THRESH);
+ bcmgenet_tdma_ring_readl(priv, 0, DMA_MBUF_DONE_THRESH);
ec->rx_max_coalesced_frames =
- bcmgenet_rdma_ring_readl(priv, DESC_INDEX,
- DMA_MBUF_DONE_THRESH);
+ bcmgenet_rdma_ring_readl(priv, 0, DMA_MBUF_DONE_THRESH);
ec->rx_coalesce_usecs =
- bcmgenet_rdma_readl(priv, DMA_RING16_TIMEOUT) * 8192 / 1000;
+ bcmgenet_rdma_readl(priv, DMA_RING0_TIMEOUT) * 8192 / 1000;
- for (i = 0; i < priv->hw_params->rx_queues; i++) {
+ for (i = 0; i <= priv->hw_params->rx_queues; i++) {
ring = &priv->rx_rings[i];
ec->use_adaptive_rx_coalesce |= ring->dim.use_dim;
}
- ring = &priv->rx_rings[DESC_INDEX];
- ec->use_adaptive_rx_coalesce |= ring->dim.use_dim;
return 0;
}
@@ -902,17 +895,13 @@ static int bcmgenet_set_coalesce(struct net_device *dev,
/* Program all TX queues with the same values, as there is no
* ethtool knob to do coalescing on a per-queue basis
*/
- for (i = 0; i < priv->hw_params->tx_queues; i++)
+ for (i = 0; i <= priv->hw_params->tx_queues; i++)
bcmgenet_tdma_ring_writel(priv, i,
ec->tx_max_coalesced_frames,
DMA_MBUF_DONE_THRESH);
- bcmgenet_tdma_ring_writel(priv, DESC_INDEX,
- ec->tx_max_coalesced_frames,
- DMA_MBUF_DONE_THRESH);
- for (i = 0; i < priv->hw_params->rx_queues; i++)
+ for (i = 0; i <= priv->hw_params->rx_queues; i++)
bcmgenet_set_ring_rx_coalesce(&priv->rx_rings[i], ec);
- bcmgenet_set_ring_rx_coalesce(&priv->rx_rings[DESC_INDEX], ec);
return 0;
}
@@ -961,12 +950,13 @@ static int bcmgenet_set_pauseparam(struct net_device *dev,
/* standard ethtool support functions. */
enum bcmgenet_stat_type {
- BCMGENET_STAT_NETDEV = -1,
+ BCMGENET_STAT_RTNL = -1,
BCMGENET_STAT_MIB_RX,
BCMGENET_STAT_MIB_TX,
BCMGENET_STAT_RUNT,
BCMGENET_STAT_MISC,
BCMGENET_STAT_SOFT,
+ BCMGENET_STAT_SOFT64,
};
struct bcmgenet_stats {
@@ -976,13 +966,15 @@ struct bcmgenet_stats {
enum bcmgenet_stat_type type;
/* reg offset from UMAC base for misc counters */
u16 reg_offset;
+ /* sync for u64 stats counters */
+ int syncp_offset;
};
-#define STAT_NETDEV(m) { \
+#define STAT_RTNL(m) { \
.stat_string = __stringify(m), \
- .stat_sizeof = sizeof(((struct net_device_stats *)0)->m), \
- .stat_offset = offsetof(struct net_device_stats, m), \
- .type = BCMGENET_STAT_NETDEV, \
+ .stat_sizeof = sizeof(((struct rtnl_link_stats64 *)0)->m), \
+ .stat_offset = offsetof(struct rtnl_link_stats64, m), \
+ .type = BCMGENET_STAT_RTNL, \
}
#define STAT_GENET_MIB(str, m, _type) { \
@@ -992,6 +984,14 @@ struct bcmgenet_stats {
.type = _type, \
}
+#define STAT_GENET_SOFT_MIB64(str, s, m) { \
+ .stat_string = str, \
+ .stat_sizeof = sizeof(((struct bcmgenet_priv *)0)->s.m), \
+ .stat_offset = offsetof(struct bcmgenet_priv, s.m), \
+ .type = BCMGENET_STAT_SOFT64, \
+ .syncp_offset = offsetof(struct bcmgenet_priv, s.syncp), \
+}
+
#define STAT_GENET_MIB_RX(str, m) STAT_GENET_MIB(str, m, BCMGENET_STAT_MIB_RX)
#define STAT_GENET_MIB_TX(str, m) STAT_GENET_MIB(str, m, BCMGENET_STAT_MIB_TX)
#define STAT_GENET_RUNT(str, m) STAT_GENET_MIB(str, m, BCMGENET_STAT_RUNT)
@@ -1006,18 +1006,18 @@ struct bcmgenet_stats {
}
#define STAT_GENET_Q(num) \
- STAT_GENET_SOFT_MIB("txq" __stringify(num) "_packets", \
- tx_rings[num].packets), \
- STAT_GENET_SOFT_MIB("txq" __stringify(num) "_bytes", \
- tx_rings[num].bytes), \
- STAT_GENET_SOFT_MIB("rxq" __stringify(num) "_bytes", \
- rx_rings[num].bytes), \
- STAT_GENET_SOFT_MIB("rxq" __stringify(num) "_packets", \
- rx_rings[num].packets), \
- STAT_GENET_SOFT_MIB("rxq" __stringify(num) "_errors", \
- rx_rings[num].errors), \
- STAT_GENET_SOFT_MIB("rxq" __stringify(num) "_dropped", \
- rx_rings[num].dropped)
+ STAT_GENET_SOFT_MIB64("txq" __stringify(num) "_packets", \
+ tx_rings[num].stats64, packets), \
+ STAT_GENET_SOFT_MIB64("txq" __stringify(num) "_bytes", \
+ tx_rings[num].stats64, bytes), \
+ STAT_GENET_SOFT_MIB64("rxq" __stringify(num) "_bytes", \
+ rx_rings[num].stats64, bytes), \
+ STAT_GENET_SOFT_MIB64("rxq" __stringify(num) "_packets", \
+ rx_rings[num].stats64, packets), \
+ STAT_GENET_SOFT_MIB64("rxq" __stringify(num) "_errors", \
+ rx_rings[num].stats64, errors), \
+ STAT_GENET_SOFT_MIB64("rxq" __stringify(num) "_dropped", \
+ rx_rings[num].stats64, dropped)
/* There is a 0xC gap between the end of RX and beginning of TX stats and then
* between the end of TX stats and the beginning of the RX RUNT
@@ -1029,15 +1029,15 @@ struct bcmgenet_stats {
*/
static const struct bcmgenet_stats bcmgenet_gstrings_stats[] = {
/* general stats */
- STAT_NETDEV(rx_packets),
- STAT_NETDEV(tx_packets),
- STAT_NETDEV(rx_bytes),
- STAT_NETDEV(tx_bytes),
- STAT_NETDEV(rx_errors),
- STAT_NETDEV(tx_errors),
- STAT_NETDEV(rx_dropped),
- STAT_NETDEV(tx_dropped),
- STAT_NETDEV(multicast),
+ STAT_RTNL(rx_packets),
+ STAT_RTNL(tx_packets),
+ STAT_RTNL(rx_bytes),
+ STAT_RTNL(tx_bytes),
+ STAT_RTNL(rx_errors),
+ STAT_RTNL(tx_errors),
+ STAT_RTNL(rx_dropped),
+ STAT_RTNL(tx_dropped),
+ STAT_RTNL(multicast),
/* UniMAC RSV counters */
STAT_GENET_MIB_RX("rx_64_octets", mib.rx.pkt_cnt.cnt_64),
STAT_GENET_MIB_RX("rx_65_127_oct", mib.rx.pkt_cnt.cnt_127),
@@ -1120,11 +1120,25 @@ static const struct bcmgenet_stats bcmgenet_gstrings_stats[] = {
STAT_GENET_Q(1),
STAT_GENET_Q(2),
STAT_GENET_Q(3),
- STAT_GENET_Q(16),
+ STAT_GENET_Q(4),
};
#define BCMGENET_STATS_LEN ARRAY_SIZE(bcmgenet_gstrings_stats)
+#define BCMGENET_STATS64_ADD(stats, m, v) \
+ do { \
+ u64_stats_update_begin(&stats->syncp); \
+ u64_stats_add(&stats->m, v); \
+ u64_stats_update_end(&stats->syncp); \
+ } while (0)
+
+#define BCMGENET_STATS64_INC(stats, m) \
+ do { \
+ u64_stats_update_begin(&stats->syncp); \
+ u64_stats_inc(&stats->m); \
+ u64_stats_update_end(&stats->syncp); \
+ } while (0)
+
static void bcmgenet_get_drvinfo(struct net_device *dev,
struct ethtool_drvinfo *info)
{
@@ -1208,8 +1222,9 @@ static void bcmgenet_update_mib_counters(struct bcmgenet_priv *priv)
s = &bcmgenet_gstrings_stats[i];
switch (s->type) {
- case BCMGENET_STAT_NETDEV:
+ case BCMGENET_STAT_RTNL:
case BCMGENET_STAT_SOFT:
+ case BCMGENET_STAT_SOFT64:
continue;
case BCMGENET_STAT_RUNT:
offset += BCMGENET_STAT_OFFSET;
@@ -1247,28 +1262,40 @@ static void bcmgenet_get_ethtool_stats(struct net_device *dev,
u64 *data)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
+ struct rtnl_link_stats64 stats64;
+ struct u64_stats_sync *syncp;
+ unsigned int start;
int i;
if (netif_running(dev))
bcmgenet_update_mib_counters(priv);
- dev->netdev_ops->ndo_get_stats(dev);
+ dev_get_stats(dev, &stats64);
for (i = 0; i < BCMGENET_STATS_LEN; i++) {
const struct bcmgenet_stats *s;
char *p;
s = &bcmgenet_gstrings_stats[i];
- if (s->type == BCMGENET_STAT_NETDEV)
- p = (char *)&dev->stats;
- else
- p = (char *)priv;
- p += s->stat_offset;
- if (sizeof(unsigned long) != sizeof(u32) &&
- s->stat_sizeof == sizeof(unsigned long))
- data[i] = *(unsigned long *)p;
- else
- data[i] = *(u32 *)p;
+ p = (char *)priv;
+
+ if (s->type == BCMGENET_STAT_SOFT64) {
+ syncp = (struct u64_stats_sync *)(p + s->syncp_offset);
+ do {
+ start = u64_stats_fetch_begin(syncp);
+ data[i] = u64_stats_read((u64_stats_t *)(p + s->stat_offset));
+ } while (u64_stats_fetch_retry(syncp, start));
+ } else {
+ if (s->type == BCMGENET_STAT_RTNL)
+ p = (char *)&stats64;
+
+ p += s->stat_offset;
+ if (sizeof(unsigned long) != sizeof(u32) &&
+ s->stat_sizeof == sizeof(unsigned long))
+ data[i] = *(unsigned long *)p;
+ else
+ data[i] = *(u32 *)p;
+ }
}
}
@@ -1465,10 +1492,10 @@ static int bcmgenet_insert_flow(struct net_device *dev,
loc_rule = &priv->rxnfc_rules[cmd->fs.location];
}
if (loc_rule->state == BCMGENET_RXNFC_STATE_ENABLED)
- bcmgenet_hfb_disable_filter(priv, cmd->fs.location);
+ bcmgenet_hfb_disable_filter(priv, cmd->fs.location + 1);
if (loc_rule->state != BCMGENET_RXNFC_STATE_UNUSED) {
list_del(&loc_rule->list);
- bcmgenet_hfb_clear_filter(priv, cmd->fs.location);
+ bcmgenet_hfb_clear_filter(priv, cmd->fs.location + 1);
}
loc_rule->state = BCMGENET_RXNFC_STATE_UNUSED;
memcpy(&loc_rule->fs, &cmd->fs,
@@ -1498,10 +1525,10 @@ static int bcmgenet_delete_flow(struct net_device *dev,
}
if (rule->state == BCMGENET_RXNFC_STATE_ENABLED)
- bcmgenet_hfb_disable_filter(priv, cmd->fs.location);
+ bcmgenet_hfb_disable_filter(priv, cmd->fs.location + 1);
if (rule->state != BCMGENET_RXNFC_STATE_UNUSED) {
list_del(&rule->list);
- bcmgenet_hfb_clear_filter(priv, cmd->fs.location);
+ bcmgenet_hfb_clear_filter(priv, cmd->fs.location + 1);
}
rule->state = BCMGENET_RXNFC_STATE_UNUSED;
memset(&rule->fs, 0, sizeof(struct ethtool_rx_flow_spec));
@@ -1644,9 +1671,9 @@ static int bcmgenet_power_down(struct bcmgenet_priv *priv,
case GENET_POWER_PASSIVE:
/* Power down LED */
- if (priv->hw_params->flags & GENET_HAS_EXT) {
+ if (bcmgenet_has_ext(priv)) {
reg = bcmgenet_ext_readl(priv, EXT_EXT_PWR_MGMT);
- if (GENET_IS_V5(priv) && !priv->ephy_16nm)
+ if (GENET_IS_V5(priv) && !bcmgenet_has_ephy_16nm(priv))
reg |= EXT_PWR_DOWN_PHY_EN |
EXT_PWR_DOWN_PHY_RD |
EXT_PWR_DOWN_PHY_SD |
@@ -1674,7 +1701,7 @@ static void bcmgenet_power_up(struct bcmgenet_priv *priv,
{
u32 reg;
- if (!(priv->hw_params->flags & GENET_HAS_EXT))
+ if (!bcmgenet_has_ext(priv))
return;
reg = bcmgenet_ext_readl(priv, EXT_EXT_PWR_MGMT);
@@ -1683,7 +1710,7 @@ static void bcmgenet_power_up(struct bcmgenet_priv *priv,
case GENET_POWER_PASSIVE:
reg &= ~(EXT_PWR_DOWN_DLL | EXT_PWR_DOWN_BIAS |
EXT_ENERGY_DET_MASK);
- if (GENET_IS_V5(priv) && !priv->ephy_16nm) {
+ if (GENET_IS_V5(priv) && !bcmgenet_has_ephy_16nm(priv)) {
reg &= ~(EXT_PWR_DOWN_PHY_EN |
EXT_PWR_DOWN_PHY_RD |
EXT_PWR_DOWN_PHY_SD |
@@ -1740,28 +1767,16 @@ static struct enet_cb *bcmgenet_put_txcb(struct bcmgenet_priv *priv,
{
struct enet_cb *tx_cb_ptr;
- tx_cb_ptr = ring->cbs;
- tx_cb_ptr += ring->write_ptr - ring->cb_ptr;
-
/* Rewinding local write pointer */
if (ring->write_ptr == ring->cb_ptr)
ring->write_ptr = ring->end_ptr;
else
ring->write_ptr--;
- return tx_cb_ptr;
-}
-
-static inline void bcmgenet_rx_ring16_int_disable(struct bcmgenet_rx_ring *ring)
-{
- bcmgenet_intrl2_0_writel(ring->priv, UMAC_IRQ_RXDMA_DONE,
- INTRL2_CPU_MASK_SET);
-}
+ tx_cb_ptr = ring->cbs;
+ tx_cb_ptr += ring->write_ptr - ring->cb_ptr;
-static inline void bcmgenet_rx_ring16_int_enable(struct bcmgenet_rx_ring *ring)
-{
- bcmgenet_intrl2_0_writel(ring->priv, UMAC_IRQ_RXDMA_DONE,
- INTRL2_CPU_MASK_CLEAR);
+ return tx_cb_ptr;
}
static inline void bcmgenet_rx_ring_int_disable(struct bcmgenet_rx_ring *ring)
@@ -1778,18 +1793,6 @@ static inline void bcmgenet_rx_ring_int_enable(struct bcmgenet_rx_ring *ring)
INTRL2_CPU_MASK_CLEAR);
}
-static inline void bcmgenet_tx_ring16_int_disable(struct bcmgenet_tx_ring *ring)
-{
- bcmgenet_intrl2_0_writel(ring->priv, UMAC_IRQ_TXDMA_DONE,
- INTRL2_CPU_MASK_SET);
-}
-
-static inline void bcmgenet_tx_ring16_int_enable(struct bcmgenet_tx_ring *ring)
-{
- bcmgenet_intrl2_0_writel(ring->priv, UMAC_IRQ_TXDMA_DONE,
- INTRL2_CPU_MASK_CLEAR);
-}
-
static inline void bcmgenet_tx_ring_int_enable(struct bcmgenet_tx_ring *ring)
{
bcmgenet_intrl2_1_writel(ring->priv, 1 << ring->index,
@@ -1861,6 +1864,7 @@ static struct sk_buff *bcmgenet_free_rx_cb(struct device *dev,
static unsigned int __bcmgenet_tx_reclaim(struct net_device *dev,
struct bcmgenet_tx_ring *ring)
{
+ struct bcmgenet_tx_stats64 *stats = &ring->stats64;
struct bcmgenet_priv *priv = netdev_priv(dev);
unsigned int txbds_processed = 0;
unsigned int bytes_compl = 0;
@@ -1870,12 +1874,7 @@ static unsigned int __bcmgenet_tx_reclaim(struct net_device *dev,
struct sk_buff *skb;
/* Clear status before servicing to reduce spurious interrupts */
- if (ring->index == DESC_INDEX)
- bcmgenet_intrl2_0_writel(priv, UMAC_IRQ_TXDMA_DONE,
- INTRL2_CPU_CLEAR);
- else
- bcmgenet_intrl2_1_writel(priv, (1 << ring->index),
- INTRL2_CPU_CLEAR);
+ bcmgenet_intrl2_1_writel(priv, (1 << ring->index), INTRL2_CPU_CLEAR);
/* Compute how many buffers are transmitted since last xmit call */
c_index = bcmgenet_tdma_ring_readl(priv, ring->index, TDMA_CONS_INDEX)
@@ -1906,22 +1905,53 @@ static unsigned int __bcmgenet_tx_reclaim(struct net_device *dev,
ring->free_bds += txbds_processed;
ring->c_index = c_index;
- ring->packets += pkts_compl;
- ring->bytes += bytes_compl;
+ u64_stats_update_begin(&stats->syncp);
+ u64_stats_add(&stats->packets, pkts_compl);
+ u64_stats_add(&stats->bytes, bytes_compl);
+ u64_stats_update_end(&stats->syncp);
- netdev_tx_completed_queue(netdev_get_tx_queue(dev, ring->queue),
+ netdev_tx_completed_queue(netdev_get_tx_queue(dev, ring->index),
pkts_compl, bytes_compl);
return txbds_processed;
}
static unsigned int bcmgenet_tx_reclaim(struct net_device *dev,
- struct bcmgenet_tx_ring *ring)
+ struct bcmgenet_tx_ring *ring,
+ bool all)
{
- unsigned int released;
+ struct bcmgenet_priv *priv = netdev_priv(dev);
+ struct device *kdev = &priv->pdev->dev;
+ unsigned int released, drop, wr_ptr;
+ struct enet_cb *cb_ptr;
+ struct sk_buff *skb;
spin_lock_bh(&ring->lock);
released = __bcmgenet_tx_reclaim(dev, ring);
+ if (all) {
+ skb = NULL;
+ drop = (ring->prod_index - ring->c_index) & DMA_C_INDEX_MASK;
+ released += drop;
+ ring->prod_index = ring->c_index & DMA_C_INDEX_MASK;
+ ring->free_bds += drop;
+ while (drop--) {
+ cb_ptr = bcmgenet_put_txcb(priv, ring);
+ skb = cb_ptr->skb;
+ bcmgenet_free_tx_cb(kdev, cb_ptr);
+ if (skb && cb_ptr == GENET_CB(skb)->first_cb) {
+ dev_consume_skb_any(skb);
+ skb = NULL;
+ }
+ }
+ if (skb)
+ dev_consume_skb_any(skb);
+ netdev_tx_reset_queue(netdev_get_tx_queue(dev, ring->index));
+ bcmgenet_tdma_ring_writel(priv, ring->index,
+ ring->prod_index, TDMA_PROD_INDEX);
+ wr_ptr = ring->write_ptr * WORDS_PER_BD(priv);
+ bcmgenet_tdma_ring_writel(priv, ring->index, wr_ptr,
+ TDMA_WRITE_PTR);
+ }
spin_unlock_bh(&ring->lock);
return released;
@@ -1937,14 +1967,14 @@ static int bcmgenet_tx_poll(struct napi_struct *napi, int budget)
spin_lock(&ring->lock);
work_done = __bcmgenet_tx_reclaim(ring->priv->dev, ring);
if (ring->free_bds > (MAX_SKB_FRAGS + 1)) {
- txq = netdev_get_tx_queue(ring->priv->dev, ring->queue);
+ txq = netdev_get_tx_queue(ring->priv->dev, ring->index);
netif_tx_wake_queue(txq);
}
spin_unlock(&ring->lock);
if (work_done == 0) {
napi_complete(napi);
- ring->int_enable(ring);
+ bcmgenet_tx_ring_int_enable(ring);
return 0;
}
@@ -1955,22 +1985,21 @@ static int bcmgenet_tx_poll(struct napi_struct *napi, int budget)
static void bcmgenet_tx_reclaim_all(struct net_device *dev)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
- int i;
-
- if (netif_is_multiqueue(dev)) {
- for (i = 0; i < priv->hw_params->tx_queues; i++)
- bcmgenet_tx_reclaim(dev, &priv->tx_rings[i]);
- }
+ int i = 0;
- bcmgenet_tx_reclaim(dev, &priv->tx_rings[DESC_INDEX]);
+ do {
+ bcmgenet_tx_reclaim(dev, &priv->tx_rings[i++], true);
+ } while (i <= priv->hw_params->tx_queues && netif_is_multiqueue(dev));
}
/* Reallocate the SKB to put enough headroom in front of it and insert
* the transmit checksum offsets in the descriptors
*/
static struct sk_buff *bcmgenet_add_tsb(struct net_device *dev,
- struct sk_buff *skb)
+ struct sk_buff *skb,
+ struct bcmgenet_tx_ring *ring)
{
+ struct bcmgenet_tx_stats64 *stats = &ring->stats64;
struct bcmgenet_priv *priv = netdev_priv(dev);
struct status_64 *status = NULL;
struct sk_buff *new_skb;
@@ -1987,7 +2016,7 @@ static struct sk_buff *bcmgenet_add_tsb(struct net_device *dev,
if (!new_skb) {
dev_kfree_skb_any(skb);
priv->mib.tx_realloc_tsb_failed++;
- dev->stats.tx_dropped++;
+ BCMGENET_STATS64_INC(stats, dropped);
return NULL;
}
dev_consume_skb_any(skb);
@@ -2050,19 +2079,14 @@ static netdev_tx_t bcmgenet_xmit(struct sk_buff *skb, struct net_device *dev)
index = skb_get_queue_mapping(skb);
/* Mapping strategy:
- * queue_mapping = 0, unclassified, packet xmited through ring16
- * queue_mapping = 1, goes to ring 0. (highest priority queue
- * queue_mapping = 2, goes to ring 1.
- * queue_mapping = 3, goes to ring 2.
- * queue_mapping = 4, goes to ring 3.
+ * queue_mapping = 0, unclassified, packet xmited through ring 0
+ * queue_mapping = 1, goes to ring 1. (highest priority queue)
+ * queue_mapping = 2, goes to ring 2.
+ * queue_mapping = 3, goes to ring 3.
+ * queue_mapping = 4, goes to ring 4.
*/
- if (index == 0)
- index = DESC_INDEX;
- else
- index -= 1;
-
ring = &priv->tx_rings[index];
- txq = netdev_get_tx_queue(dev, ring->queue);
+ txq = netdev_get_tx_queue(dev, index);
nr_frags = skb_shinfo(skb)->nr_frags;
@@ -2080,7 +2104,7 @@ static netdev_tx_t bcmgenet_xmit(struct sk_buff *skb, struct net_device *dev)
GENET_CB(skb)->bytes_sent = skb->len;
/* add the Transmit Status Block */
- skb = bcmgenet_add_tsb(dev, skb);
+ skb = bcmgenet_add_tsb(dev, skb, ring);
if (!skb) {
ret = NETDEV_TX_OK;
goto out;
@@ -2222,6 +2246,7 @@ static struct sk_buff *bcmgenet_rx_refill(struct bcmgenet_priv *priv,
static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
unsigned int budget)
{
+ struct bcmgenet_rx_stats64 *stats = &ring->stats64;
struct bcmgenet_priv *priv = ring->priv;
struct net_device *dev = priv->dev;
struct enet_cb *cb;
@@ -2235,15 +2260,8 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
unsigned int discards;
/* Clear status before servicing to reduce spurious interrupts */
- if (ring->index == DESC_INDEX) {
- bcmgenet_intrl2_0_writel(priv, UMAC_IRQ_RXDMA_DONE,
- INTRL2_CPU_CLEAR);
- } else {
- mask = 1 << (UMAC_IRQ1_RX_INTR_SHIFT + ring->index);
- bcmgenet_intrl2_1_writel(priv,
- mask,
- INTRL2_CPU_CLEAR);
- }
+ mask = 1 << (UMAC_IRQ1_RX_INTR_SHIFT + ring->index);
+ bcmgenet_intrl2_1_writel(priv, mask, INTRL2_CPU_CLEAR);
p_index = bcmgenet_rdma_ring_readl(priv, ring->index, RDMA_PROD_INDEX);
@@ -2251,7 +2269,7 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
DMA_P_INDEX_DISCARD_CNT_MASK;
if (discards > ring->old_discards) {
discards = discards - ring->old_discards;
- ring->errors += discards;
+ BCMGENET_STATS64_ADD(stats, errors, discards);
ring->old_discards += discards;
/* Clear HW register when we reach 75% of maximum 0xFFFF */
@@ -2277,7 +2295,7 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
skb = bcmgenet_rx_refill(priv, cb);
if (unlikely(!skb)) {
- ring->dropped++;
+ BCMGENET_STATS64_INC(stats, dropped);
goto next;
}
@@ -2304,8 +2322,7 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
if (unlikely(len > RX_BUF_LENGTH)) {
netif_err(priv, rx_status, dev, "oversized packet\n");
- dev->stats.rx_length_errors++;
- dev->stats.rx_errors++;
+ BCMGENET_STATS64_INC(stats, length_errors);
dev_kfree_skb_any(skb);
goto next;
}
@@ -2313,7 +2330,7 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
if (unlikely(!(dma_flag & DMA_EOP) || !(dma_flag & DMA_SOP))) {
netif_err(priv, rx_status, dev,
"dropping fragmented packet!\n");
- ring->errors++;
+ BCMGENET_STATS64_INC(stats, errors);
dev_kfree_skb_any(skb);
goto next;
}
@@ -2326,15 +2343,22 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
DMA_RX_RXER))) {
netif_err(priv, rx_status, dev, "dma_flag=0x%x\n",
(unsigned int)dma_flag);
+ u64_stats_update_begin(&stats->syncp);
if (dma_flag & DMA_RX_CRC_ERROR)
- dev->stats.rx_crc_errors++;
+ u64_stats_inc(&stats->crc_errors);
if (dma_flag & DMA_RX_OV)
- dev->stats.rx_over_errors++;
+ u64_stats_inc(&stats->over_errors);
if (dma_flag & DMA_RX_NO)
- dev->stats.rx_frame_errors++;
+ u64_stats_inc(&stats->frame_errors);
if (dma_flag & DMA_RX_LG)
- dev->stats.rx_length_errors++;
- dev->stats.rx_errors++;
+ u64_stats_inc(&stats->length_errors);
+ if ((dma_flag & (DMA_RX_CRC_ERROR |
+ DMA_RX_OV |
+ DMA_RX_NO |
+ DMA_RX_LG |
+ DMA_RX_RXER)) == DMA_RX_RXER)
+ u64_stats_inc(&stats->errors);
+ u64_stats_update_end(&stats->syncp);
dev_kfree_skb_any(skb);
goto next;
} /* error packet */
@@ -2354,10 +2378,13 @@ static unsigned int bcmgenet_desc_rx(struct bcmgenet_rx_ring *ring,
/*Finish setting up the received SKB and send it to the kernel*/
skb->protocol = eth_type_trans(skb, priv->dev);
- ring->packets++;
- ring->bytes += len;
+
+ u64_stats_update_begin(&stats->syncp);
+ u64_stats_inc(&stats->packets);
+ u64_stats_add(&stats->bytes, len);
if (dma_flag & DMA_RX_MULT)
- dev->stats.multicast++;
+ u64_stats_inc(&stats->multicast);
+ u64_stats_update_end(&stats->syncp);
/* Notify kernel */
napi_gro_receive(&ring->napi, skb);
@@ -2392,7 +2419,7 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
if (work_done < budget) {
napi_complete_done(napi, work_done);
- ring->int_enable(ring);
+ bcmgenet_rx_ring_int_enable(ring);
}
if (ring->dim.use_dim) {
@@ -2516,7 +2543,7 @@ static void bcmgenet_link_intr_enable(struct bcmgenet_priv *priv)
} else if (priv->ext_phy) {
int0_enable |= UMAC_IRQ_LINK_EVENT;
} else if (priv->phy_interface == PHY_INTERFACE_MODE_MOCA) {
- if (priv->hw_params->flags & GENET_HAS_MOCA_LINK_DET)
+ if (bcmgenet_has_moca_link_det(priv))
int0_enable |= UMAC_IRQ_LINK_EVENT;
}
bcmgenet_intrl2_0_writel(priv, int0_enable, INTRL2_CPU_MASK_CLEAR);
@@ -2581,7 +2608,7 @@ static void init_umac(struct bcmgenet_priv *priv)
}
/* Enable MDIO interrupts on GENET v3+ */
- if (priv->hw_params->flags & GENET_HAS_MDIO_INTR)
+ if (bcmgenet_has_mdio_intr(priv))
int0_enable |= (UMAC_IRQ_MDIO_DONE | UMAC_IRQ_MDIO_ERROR);
bcmgenet_intrl2_0_writel(priv, int0_enable, INTRL2_CPU_MASK_CLEAR);
@@ -2632,15 +2659,6 @@ static void bcmgenet_init_tx_ring(struct bcmgenet_priv *priv,
spin_lock_init(&ring->lock);
ring->priv = priv;
ring->index = index;
- if (index == DESC_INDEX) {
- ring->queue = 0;
- ring->int_enable = bcmgenet_tx_ring16_int_enable;
- ring->int_disable = bcmgenet_tx_ring16_int_disable;
- } else {
- ring->queue = index + 1;
- ring->int_enable = bcmgenet_tx_ring_int_enable;
- ring->int_disable = bcmgenet_tx_ring_int_disable;
- }
ring->cbs = priv->tx_cbs + start_ptr;
ring->size = size;
ring->clean_ptr = start_ptr;
@@ -2651,8 +2669,8 @@ static void bcmgenet_init_tx_ring(struct bcmgenet_priv *priv,
ring->end_ptr = end_ptr - 1;
ring->prod_index = 0;
- /* Set flow period for ring != 16 */
- if (index != DESC_INDEX)
+ /* Set flow period for ring != 0 */
+ if (index)
flow_period_val = ENET_MAX_MTU_SIZE << 16;
bcmgenet_tdma_ring_writel(priv, index, 0, TDMA_PROD_INDEX);
@@ -2690,13 +2708,6 @@ static int bcmgenet_init_rx_ring(struct bcmgenet_priv *priv,
ring->priv = priv;
ring->index = index;
- if (index == DESC_INDEX) {
- ring->int_enable = bcmgenet_rx_ring16_int_enable;
- ring->int_disable = bcmgenet_rx_ring16_int_disable;
- } else {
- ring->int_enable = bcmgenet_rx_ring_int_enable;
- ring->int_disable = bcmgenet_rx_ring_int_disable;
- }
ring->cbs = priv->rx_cbs + start_ptr;
ring->size = size;
ring->c_index = 0;
@@ -2742,15 +2753,11 @@ static void bcmgenet_enable_tx_napi(struct bcmgenet_priv *priv)
unsigned int i;
struct bcmgenet_tx_ring *ring;
- for (i = 0; i < priv->hw_params->tx_queues; ++i) {
+ for (i = 0; i <= priv->hw_params->tx_queues; ++i) {
ring = &priv->tx_rings[i];
napi_enable(&ring->napi);
- ring->int_enable(ring);
+ bcmgenet_tx_ring_int_enable(ring);
}
-
- ring = &priv->tx_rings[DESC_INDEX];
- napi_enable(&ring->napi);
- ring->int_enable(ring);
}
static void bcmgenet_disable_tx_napi(struct bcmgenet_priv *priv)
@@ -2758,13 +2765,10 @@ static void bcmgenet_disable_tx_napi(struct bcmgenet_priv *priv)
unsigned int i;
struct bcmgenet_tx_ring *ring;
- for (i = 0; i < priv->hw_params->tx_queues; ++i) {
+ for (i = 0; i <= priv->hw_params->tx_queues; ++i) {
ring = &priv->tx_rings[i];
napi_disable(&ring->napi);
}
-
- ring = &priv->tx_rings[DESC_INDEX];
- napi_disable(&ring->napi);
}
static void bcmgenet_fini_tx_napi(struct bcmgenet_priv *priv)
@@ -2772,33 +2776,31 @@ static void bcmgenet_fini_tx_napi(struct bcmgenet_priv *priv)
unsigned int i;
struct bcmgenet_tx_ring *ring;
- for (i = 0; i < priv->hw_params->tx_queues; ++i) {
+ for (i = 0; i <= priv->hw_params->tx_queues; ++i) {
ring = &priv->tx_rings[i];
netif_napi_del(&ring->napi);
}
-
- ring = &priv->tx_rings[DESC_INDEX];
- netif_napi_del(&ring->napi);
}
/* Initialize Tx queues
*
- * Queues 0-3 are priority-based, each one has 32 descriptors,
- * with queue 0 being the highest priority queue.
+ * Queues 1-4 are priority-based, each one has 32 descriptors,
+ * with queue 1 being the highest priority queue.
*
- * Queue 16 is the default Tx queue with
- * GENET_Q16_TX_BD_CNT = 256 - 4 * 32 = 128 descriptors.
+ * Queue 0 is the default Tx queue with
+ * GENET_Q0_TX_BD_CNT = 256 - 4 * 32 = 128 descriptors.
*
* The transmit control block pool is then partitioned as follows:
- * - Tx queue 0 uses tx_cbs[0..31]
- * - Tx queue 1 uses tx_cbs[32..63]
- * - Tx queue 2 uses tx_cbs[64..95]
- * - Tx queue 3 uses tx_cbs[96..127]
- * - Tx queue 16 uses tx_cbs[128..255]
+ * - Tx queue 0 uses tx_cbs[0..127]
+ * - Tx queue 1 uses tx_cbs[128..159]
+ * - Tx queue 2 uses tx_cbs[160..191]
+ * - Tx queue 3 uses tx_cbs[192..223]
+ * - Tx queue 4 uses tx_cbs[224..255]
*/
static void bcmgenet_init_tx_queues(struct net_device *dev)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
+ unsigned int start = 0, end = GENET_Q0_TX_BD_CNT;
u32 i, dma_enable;
u32 dma_ctrl, ring_cfg;
u32 dma_priority[3] = {0, 0, 0};
@@ -2815,27 +2817,17 @@ static void bcmgenet_init_tx_queues(struct net_device *dev)
bcmgenet_tdma_writel(priv, DMA_ARBITER_SP, DMA_ARB_CTRL);
/* Initialize Tx priority queues */
- for (i = 0; i < priv->hw_params->tx_queues; i++) {
- bcmgenet_init_tx_ring(priv, i, priv->hw_params->tx_bds_per_q,
- i * priv->hw_params->tx_bds_per_q,
- (i + 1) * priv->hw_params->tx_bds_per_q);
+ for (i = 0; i <= priv->hw_params->tx_queues; i++) {
+ bcmgenet_init_tx_ring(priv, i, end - start, start, end);
+ start = end;
+ end += priv->hw_params->tx_bds_per_q;
ring_cfg |= (1 << i);
dma_ctrl |= (1 << (i + DMA_RING_BUF_EN_SHIFT));
dma_priority[DMA_PRIO_REG_INDEX(i)] |=
- ((GENET_Q0_PRIORITY + i) << DMA_PRIO_REG_SHIFT(i));
+ (i ? GENET_Q1_PRIORITY : GENET_Q0_PRIORITY)
+ << DMA_PRIO_REG_SHIFT(i);
}
- /* Initialize Tx default queue 16 */
- bcmgenet_init_tx_ring(priv, DESC_INDEX, GENET_Q16_TX_BD_CNT,
- priv->hw_params->tx_queues *
- priv->hw_params->tx_bds_per_q,
- TOTAL_DESC);
- ring_cfg |= (1 << DESC_INDEX);
- dma_ctrl |= (1 << (DESC_INDEX + DMA_RING_BUF_EN_SHIFT));
- dma_priority[DMA_PRIO_REG_INDEX(DESC_INDEX)] |=
- ((GENET_Q0_PRIORITY + priv->hw_params->tx_queues) <<
- DMA_PRIO_REG_SHIFT(DESC_INDEX));
-
/* Set Tx queue priorities */
bcmgenet_tdma_writel(priv, dma_priority[0], DMA_PRIORITY_0);
bcmgenet_tdma_writel(priv, dma_priority[1], DMA_PRIORITY_1);
@@ -2855,15 +2847,11 @@ static void bcmgenet_enable_rx_napi(struct bcmgenet_priv *priv)
unsigned int i;
struct bcmgenet_rx_ring *ring;
- for (i = 0; i < priv->hw_params->rx_queues; ++i) {
+ for (i = 0; i <= priv->hw_params->rx_queues; ++i) {
ring = &priv->rx_rings[i];
napi_enable(&ring->napi);
- ring->int_enable(ring);
+ bcmgenet_rx_ring_int_enable(ring);
}
-
- ring = &priv->rx_rings[DESC_INDEX];
- napi_enable(&ring->napi);
- ring->int_enable(ring);
}
static void bcmgenet_disable_rx_napi(struct bcmgenet_priv *priv)
@@ -2871,15 +2859,11 @@ static void bcmgenet_disable_rx_napi(struct bcmgenet_priv *priv)
unsigned int i;
struct bcmgenet_rx_ring *ring;
- for (i = 0; i < priv->hw_params->rx_queues; ++i) {
+ for (i = 0; i <= priv->hw_params->rx_queues; ++i) {
ring = &priv->rx_rings[i];
napi_disable(&ring->napi);
cancel_work_sync(&ring->dim.dim.work);
}
-
- ring = &priv->rx_rings[DESC_INDEX];
- napi_disable(&ring->napi);
- cancel_work_sync(&ring->dim.dim.work);
}
static void bcmgenet_fini_rx_napi(struct bcmgenet_priv *priv)
@@ -2887,13 +2871,10 @@ static void bcmgenet_fini_rx_napi(struct bcmgenet_priv *priv)
unsigned int i;
struct bcmgenet_rx_ring *ring;
- for (i = 0; i < priv->hw_params->rx_queues; ++i) {
+ for (i = 0; i <= priv->hw_params->rx_queues; ++i) {
ring = &priv->rx_rings[i];
netif_napi_del(&ring->napi);
}
-
- ring = &priv->rx_rings[DESC_INDEX];
- netif_napi_del(&ring->napi);
}
/* Initialize Rx queues
@@ -2901,15 +2882,13 @@ static void bcmgenet_fini_rx_napi(struct bcmgenet_priv *priv)
* Queues 0-15 are priority queues. Hardware Filtering Block (HFB) can be
* used to direct traffic to these queues.
*
- * Queue 16 is the default Rx queue with GENET_Q16_RX_BD_CNT descriptors.
+ * Queue 0 is also the default Rx queue with GENET_Q0_RX_BD_CNT descriptors.
*/
static int bcmgenet_init_rx_queues(struct net_device *dev)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
- u32 i;
- u32 dma_enable;
- u32 dma_ctrl;
- u32 ring_cfg;
+ unsigned int start = 0, end = GENET_Q0_RX_BD_CNT;
+ u32 i, dma_enable, dma_ctrl = 0, ring_cfg = 0;
int ret;
dma_ctrl = bcmgenet_rdma_readl(priv, DMA_CTRL);
@@ -2921,34 +2900,21 @@ static int bcmgenet_init_rx_queues(struct net_device *dev)
ring_cfg = 0;
/* Initialize Rx priority queues */
- for (i = 0; i < priv->hw_params->rx_queues; i++) {
- ret = bcmgenet_init_rx_ring(priv, i,
- priv->hw_params->rx_bds_per_q,
- i * priv->hw_params->rx_bds_per_q,
- (i + 1) *
- priv->hw_params->rx_bds_per_q);
+ for (i = 0; i <= priv->hw_params->rx_queues; i++) {
+ ret = bcmgenet_init_rx_ring(priv, i, end - start, start, end);
if (ret)
return ret;
+ start = end;
+ end += priv->hw_params->rx_bds_per_q;
ring_cfg |= (1 << i);
dma_ctrl |= (1 << (i + DMA_RING_BUF_EN_SHIFT));
}
- /* Initialize Rx default queue 16 */
- ret = bcmgenet_init_rx_ring(priv, DESC_INDEX, GENET_Q16_RX_BD_CNT,
- priv->hw_params->rx_queues *
- priv->hw_params->rx_bds_per_q,
- TOTAL_DESC);
- if (ret)
- return ret;
-
- ring_cfg |= (1 << DESC_INDEX);
- dma_ctrl |= (1 << (DESC_INDEX + DMA_RING_BUF_EN_SHIFT));
-
- /* Enable rings */
+ /* Configure Rx queues as descriptor rings */
bcmgenet_rdma_writel(priv, ring_cfg, DMA_RING_CFG);
- /* Configure ring as descriptor ring and re-enable DMA if enabled */
+ /* Enable Rx rings */
if (dma_enable)
dma_ctrl |= DMA_EN;
bcmgenet_rdma_writel(priv, dma_ctrl, DMA_CTRL);
@@ -3007,14 +2973,14 @@ static int bcmgenet_dma_teardown(struct bcmgenet_priv *priv)
}
dma_ctrl = 0;
- for (i = 0; i < priv->hw_params->rx_queues; i++)
+ for (i = 0; i <= priv->hw_params->rx_queues; i++)
dma_ctrl |= (1 << (i + DMA_RING_BUF_EN_SHIFT));
reg = bcmgenet_rdma_readl(priv, DMA_CTRL);
reg &= ~dma_ctrl;
bcmgenet_rdma_writel(priv, reg, DMA_CTRL);
dma_ctrl = 0;
- for (i = 0; i < priv->hw_params->tx_queues; i++)
+ for (i = 0; i <= priv->hw_params->tx_queues; i++)
dma_ctrl |= (1 << (i + DMA_RING_BUF_EN_SHIFT));
reg = bcmgenet_tdma_readl(priv, DMA_CTRL);
reg &= ~dma_ctrl;
@@ -3031,18 +2997,11 @@ static void bcmgenet_fini_dma(struct bcmgenet_priv *priv)
bcmgenet_fini_rx_napi(priv);
bcmgenet_fini_tx_napi(priv);
- for (i = 0; i < priv->num_tx_bds; i++)
- dev_kfree_skb(bcmgenet_free_tx_cb(&priv->pdev->dev,
- priv->tx_cbs + i));
-
- for (i = 0; i < priv->hw_params->tx_queues; i++) {
- txq = netdev_get_tx_queue(priv->dev, priv->tx_rings[i].queue);
+ for (i = 0; i <= priv->hw_params->tx_queues; i++) {
+ txq = netdev_get_tx_queue(priv->dev, i);
netdev_tx_reset_queue(txq);
}
- txq = netdev_get_tx_queue(priv->dev, priv->tx_rings[DESC_INDEX].queue);
- netdev_tx_reset_queue(txq);
-
bcmgenet_free_rx_buffers(priv);
kfree(priv->rx_cbs);
kfree(priv->tx_cbs);
@@ -3135,7 +3094,7 @@ static void bcmgenet_irq_task(struct work_struct *work)
}
-/* bcmgenet_isr1: handle Rx and Tx priority queues */
+/* bcmgenet_isr1: handle Rx and Tx queues */
static irqreturn_t bcmgenet_isr1(int irq, void *dev_id)
{
struct bcmgenet_priv *priv = dev_id;
@@ -3154,7 +3113,7 @@ static irqreturn_t bcmgenet_isr1(int irq, void *dev_id)
"%s: IRQ=0x%x\n", __func__, status);
/* Check Rx priority queue interrupts */
- for (index = 0; index < priv->hw_params->rx_queues; index++) {
+ for (index = 0; index <= priv->hw_params->rx_queues; index++) {
if (!(status & BIT(UMAC_IRQ1_RX_INTR_SHIFT + index)))
continue;
@@ -3162,20 +3121,20 @@ static irqreturn_t bcmgenet_isr1(int irq, void *dev_id)
rx_ring->dim.event_ctr++;
if (likely(napi_schedule_prep(&rx_ring->napi))) {
- rx_ring->int_disable(rx_ring);
+ bcmgenet_rx_ring_int_disable(rx_ring);
__napi_schedule_irqoff(&rx_ring->napi);
}
}
/* Check Tx priority queue interrupts */
- for (index = 0; index < priv->hw_params->tx_queues; index++) {
+ for (index = 0; index <= priv->hw_params->tx_queues; index++) {
if (!(status & BIT(index)))
continue;
tx_ring = &priv->tx_rings[index];
if (likely(napi_schedule_prep(&tx_ring->napi))) {
- tx_ring->int_disable(tx_ring);
+ bcmgenet_tx_ring_int_disable(tx_ring);
__napi_schedule_irqoff(&tx_ring->napi);
}
}
@@ -3183,12 +3142,10 @@ static irqreturn_t bcmgenet_isr1(int irq, void *dev_id)
return IRQ_HANDLED;
}
-/* bcmgenet_isr0: handle Rx and Tx default queues + other stuff */
+/* bcmgenet_isr0: handle other stuff */
static irqreturn_t bcmgenet_isr0(int irq, void *dev_id)
{
struct bcmgenet_priv *priv = dev_id;
- struct bcmgenet_rx_ring *rx_ring;
- struct bcmgenet_tx_ring *tx_ring;
unsigned int status;
unsigned long flags;
@@ -3202,26 +3159,7 @@ static irqreturn_t bcmgenet_isr0(int irq, void *dev_id)
netif_dbg(priv, intr, priv->dev,
"IRQ=0x%x\n", status);
- if (status & UMAC_IRQ_RXDMA_DONE) {
- rx_ring = &priv->rx_rings[DESC_INDEX];
- rx_ring->dim.event_ctr++;
-
- if (likely(napi_schedule_prep(&rx_ring->napi))) {
- rx_ring->int_disable(rx_ring);
- __napi_schedule_irqoff(&rx_ring->napi);
- }
- }
-
- if (status & UMAC_IRQ_TXDMA_DONE) {
- tx_ring = &priv->tx_rings[DESC_INDEX];
-
- if (likely(napi_schedule_prep(&tx_ring->napi))) {
- tx_ring->int_disable(tx_ring);
- __napi_schedule_irqoff(&tx_ring->napi);
- }
- }
-
- if ((priv->hw_params->flags & GENET_HAS_MDIO_INTR) &&
+ if (bcmgenet_has_mdio_intr(priv) &&
status & (UMAC_IRQ_MDIO_DONE | UMAC_IRQ_MDIO_ERROR)) {
wake_up(&priv->wq);
}
@@ -3286,15 +3224,15 @@ static u32 bcmgenet_dma_disable(struct bcmgenet_priv *priv, bool flush_rx)
u32 dma_ctrl;
/* disable DMA */
- dma_ctrl = 1 << (DESC_INDEX + DMA_RING_BUF_EN_SHIFT) | DMA_EN;
- for (i = 0; i < priv->hw_params->tx_queues; i++)
+ dma_ctrl = DMA_EN;
+ for (i = 0; i <= priv->hw_params->tx_queues; i++)
dma_ctrl |= (1 << (i + DMA_RING_BUF_EN_SHIFT));
reg = bcmgenet_tdma_readl(priv, DMA_CTRL);
reg &= ~dma_ctrl;
bcmgenet_tdma_writel(priv, reg, DMA_CTRL);
- dma_ctrl = 1 << (DESC_INDEX + DMA_RING_BUF_EN_SHIFT) | DMA_EN;
- for (i = 0; i < priv->hw_params->rx_queues; i++)
+ dma_ctrl = DMA_EN;
+ for (i = 0; i <= priv->hw_params->rx_queues; i++)
dma_ctrl |= (1 << (i + DMA_RING_BUF_EN_SHIFT));
reg = bcmgenet_rdma_readl(priv, DMA_CTRL);
reg &= ~dma_ctrl;
@@ -3377,6 +3315,9 @@ static int bcmgenet_open(struct net_device *dev)
bcmgenet_set_hw_addr(priv, dev->dev_addr);
+ /* HFB init */
+ bcmgenet_hfb_init(priv);
+
/* Disable RX/TX DMA and flush TX and RX queues */
dma_ctrl = bcmgenet_dma_disable(priv, true);
@@ -3387,12 +3328,8 @@ static int bcmgenet_open(struct net_device *dev)
goto err_clk_disable;
}
- /* Always enable ring 16 - descriptor ring */
bcmgenet_enable_dma(priv, dma_ctrl);
- /* HFB init */
- bcmgenet_hfb_init(priv);
-
ret = request_irq(priv->irq0, bcmgenet_isr0, IRQF_SHARED,
dev->name, priv);
if (ret < 0) {
@@ -3499,16 +3436,11 @@ static void bcmgenet_dump_tx_queue(struct bcmgenet_tx_ring *ring)
if (!netif_msg_tx_err(priv))
return;
- txq = netdev_get_tx_queue(priv->dev, ring->queue);
+ txq = netdev_get_tx_queue(priv->dev, ring->index);
spin_lock(&ring->lock);
- if (ring->index == DESC_INDEX) {
- intsts = ~bcmgenet_intrl2_0_readl(priv, INTRL2_CPU_MASK_STATUS);
- intmsk = UMAC_IRQ_TXDMA_DONE | UMAC_IRQ_TXDMA_MBDONE;
- } else {
- intsts = ~bcmgenet_intrl2_1_readl(priv, INTRL2_CPU_MASK_STATUS);
- intmsk = 1 << ring->index;
- }
+ intsts = ~bcmgenet_intrl2_1_readl(priv, INTRL2_CPU_MASK_STATUS);
+ intmsk = 1 << ring->index;
c_index = bcmgenet_tdma_ring_readl(priv, ring->index, TDMA_CONS_INDEX);
p_index = bcmgenet_tdma_ring_readl(priv, ring->index, TDMA_PROD_INDEX);
txq_stopped = netif_tx_queue_stopped(txq);
@@ -3522,7 +3454,7 @@ static void bcmgenet_dump_tx_queue(struct bcmgenet_tx_ring *ring)
"(sw)c_index: %d (hw)c_index: %d\n"
"(sw)clean_p: %d (sw)write_p: %d\n"
"(sw)cb_ptr: %d (sw)end_ptr: %d\n",
- ring->index, ring->queue,
+ ring->index, ring->index,
txq_stopped ? "stopped" : "active",
intsts & intmsk ? "enabled" : "disabled",
free_bds, ring->size,
@@ -3535,32 +3467,23 @@ static void bcmgenet_dump_tx_queue(struct bcmgenet_tx_ring *ring)
static void bcmgenet_timeout(struct net_device *dev, unsigned int txqueue)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
- u32 int0_enable = 0;
- u32 int1_enable = 0;
- unsigned int q;
+ struct bcmgenet_tx_ring *ring = &priv->tx_rings[txqueue];
+ struct netdev_queue *txq = netdev_get_tx_queue(dev, txqueue);
netif_dbg(priv, tx_err, dev, "bcmgenet_timeout\n");
- for (q = 0; q < priv->hw_params->tx_queues; q++)
- bcmgenet_dump_tx_queue(&priv->tx_rings[q]);
- bcmgenet_dump_tx_queue(&priv->tx_rings[DESC_INDEX]);
-
- bcmgenet_tx_reclaim_all(dev);
-
- for (q = 0; q < priv->hw_params->tx_queues; q++)
- int1_enable |= (1 << q);
+ bcmgenet_dump_tx_queue(ring);
- int0_enable = UMAC_IRQ_TXDMA_DONE;
+ bcmgenet_tx_reclaim(dev, ring, true);
- /* Re-enable TX interrupts if disabled */
- bcmgenet_intrl2_0_writel(priv, int0_enable, INTRL2_CPU_MASK_CLEAR);
- bcmgenet_intrl2_1_writel(priv, int1_enable, INTRL2_CPU_MASK_CLEAR);
+ /* Re-enable the TX interrupt for this ring */
+ bcmgenet_intrl2_1_writel(priv, 1 << txqueue, INTRL2_CPU_MASK_CLEAR);
- netif_trans_update(dev);
+ txq_trans_cond_update(txq);
- dev->stats.tx_errors++;
+ BCMGENET_STATS64_INC((&ring->stats64), errors);
- netif_tx_wake_all_queues(dev);
+ netif_tx_wake_queue(txq);
}
#define MAX_MDF_FILTER 17
@@ -3647,47 +3570,68 @@ static int bcmgenet_set_mac_addr(struct net_device *dev, void *p)
return 0;
}
-static struct net_device_stats *bcmgenet_get_stats(struct net_device *dev)
+static void bcmgenet_get_stats64(struct net_device *dev,
+ struct rtnl_link_stats64 *stats)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
- unsigned long tx_bytes = 0, tx_packets = 0;
- unsigned long rx_bytes = 0, rx_packets = 0;
- unsigned long rx_errors = 0, rx_dropped = 0;
- struct bcmgenet_tx_ring *tx_ring;
- struct bcmgenet_rx_ring *rx_ring;
+ struct bcmgenet_tx_stats64 *tx_stats;
+ struct bcmgenet_rx_stats64 *rx_stats;
+ u64 rx_length_errors, rx_over_errors;
+ u64 rx_crc_errors, rx_frame_errors;
+ u64 tx_errors, tx_dropped;
+ u64 rx_errors, rx_dropped;
+ u64 tx_bytes, tx_packets;
+ u64 rx_bytes, rx_packets;
+ unsigned int start;
unsigned int q;
-
- for (q = 0; q < priv->hw_params->tx_queues; q++) {
- tx_ring = &priv->tx_rings[q];
- tx_bytes += tx_ring->bytes;
- tx_packets += tx_ring->packets;
+ u64 multicast;
+
+ for (q = 0; q <= priv->hw_params->tx_queues; q++) {
+ tx_stats = &priv->tx_rings[q].stats64;
+ do {
+ start = u64_stats_fetch_begin(&tx_stats->syncp);
+ tx_bytes = u64_stats_read(&tx_stats->bytes);
+ tx_packets = u64_stats_read(&tx_stats->packets);
+ tx_errors = u64_stats_read(&tx_stats->errors);
+ tx_dropped = u64_stats_read(&tx_stats->dropped);
+ } while (u64_stats_fetch_retry(&tx_stats->syncp, start));
+
+ stats->tx_bytes += tx_bytes;
+ stats->tx_packets += tx_packets;
+ stats->tx_errors += tx_errors;
+ stats->tx_dropped += tx_dropped;
}
- tx_ring = &priv->tx_rings[DESC_INDEX];
- tx_bytes += tx_ring->bytes;
- tx_packets += tx_ring->packets;
- for (q = 0; q < priv->hw_params->rx_queues; q++) {
- rx_ring = &priv->rx_rings[q];
-
- rx_bytes += rx_ring->bytes;
- rx_packets += rx_ring->packets;
- rx_errors += rx_ring->errors;
- rx_dropped += rx_ring->dropped;
+ for (q = 0; q <= priv->hw_params->rx_queues; q++) {
+ rx_stats = &priv->rx_rings[q].stats64;
+ do {
+ start = u64_stats_fetch_begin(&rx_stats->syncp);
+ rx_bytes = u64_stats_read(&rx_stats->bytes);
+ rx_packets = u64_stats_read(&rx_stats->packets);
+ rx_errors = u64_stats_read(&rx_stats->errors);
+ rx_dropped = u64_stats_read(&rx_stats->dropped);
+ rx_length_errors = u64_stats_read(&rx_stats->length_errors);
+ rx_over_errors = u64_stats_read(&rx_stats->over_errors);
+ rx_crc_errors = u64_stats_read(&rx_stats->crc_errors);
+ rx_frame_errors = u64_stats_read(&rx_stats->frame_errors);
+ multicast = u64_stats_read(&rx_stats->multicast);
+ } while (u64_stats_fetch_retry(&rx_stats->syncp, start));
+
+ rx_errors += rx_length_errors;
+ rx_errors += rx_crc_errors;
+ rx_errors += rx_frame_errors;
+
+ stats->rx_bytes += rx_bytes;
+ stats->rx_packets += rx_packets;
+ stats->rx_errors += rx_errors;
+ stats->rx_dropped += rx_dropped;
+ stats->rx_missed_errors += rx_errors;
+ stats->rx_length_errors += rx_length_errors;
+ stats->rx_over_errors += rx_over_errors;
+ stats->rx_crc_errors += rx_crc_errors;
+ stats->rx_frame_errors += rx_frame_errors;
+ stats->multicast += multicast;
}
- rx_ring = &priv->rx_rings[DESC_INDEX];
- rx_bytes += rx_ring->bytes;
- rx_packets += rx_ring->packets;
- rx_errors += rx_ring->errors;
- rx_dropped += rx_ring->dropped;
-
- dev->stats.tx_bytes = tx_bytes;
- dev->stats.tx_packets = tx_packets;
- dev->stats.rx_bytes = rx_bytes;
- dev->stats.rx_packets = rx_packets;
- dev->stats.rx_errors = rx_errors;
- dev->stats.rx_missed_errors = rx_errors;
- dev->stats.rx_dropped = rx_dropped;
- return &dev->stats;
}
static int bcmgenet_change_carrier(struct net_device *dev, bool new_carrier)
@@ -3715,7 +3659,7 @@ static const struct net_device_ops bcmgenet_netdev_ops = {
.ndo_set_mac_address = bcmgenet_set_mac_addr,
.ndo_eth_ioctl = phy_do_ioctl_running,
.ndo_set_features = bcmgenet_set_features,
- .ndo_get_stats = bcmgenet_get_stats,
+ .ndo_get_stats64 = bcmgenet_get_stats64,
.ndo_change_carrier = bcmgenet_change_carrier,
};
@@ -3891,7 +3835,7 @@ static void bcmgenet_set_hw_params(struct bcmgenet_priv *priv)
}
#ifdef CONFIG_PHYS_ADDR_T_64BIT
- if (!(params->flags & GENET_HAS_40BITS))
+ if (!bcmgenet_has_40bits(priv))
pr_warn("GENET does not support 40-bits PA\n");
#endif
@@ -4070,7 +4014,7 @@ static int bcmgenet_probe(struct platform_device *pdev)
bcmgenet_set_hw_params(priv);
err = -EIO;
- if (priv->hw_params->flags & GENET_HAS_40BITS)
+ if (bcmgenet_has_40bits(priv))
err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(40));
if (err)
err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
@@ -4125,16 +4069,19 @@ static int bcmgenet_probe(struct platform_device *pdev)
if (err)
goto err_clk_disable;
- /* setup number of real queues + 1 (GENET_V1 has 0 hardware queues
- * just the ring 16 descriptor based TX
- */
+ /* setup number of real queues + 1 */
netif_set_real_num_tx_queues(priv->dev, priv->hw_params->tx_queues + 1);
netif_set_real_num_rx_queues(priv->dev, priv->hw_params->rx_queues + 1);
/* Set default coalescing parameters */
- for (i = 0; i < priv->hw_params->rx_queues; i++)
+ for (i = 0; i <= priv->hw_params->rx_queues; i++)
priv->rx_rings[i].rx_max_coalesced_frames = 1;
- priv->rx_rings[DESC_INDEX].rx_max_coalesced_frames = 1;
+
+ /* Initialize u64 stats seq counter for 32bit machines */
+ for (i = 0; i <= priv->hw_params->rx_queues; i++)
+ u64_stats_init(&priv->rx_rings[i].stats64.syncp);
+ for (i = 0; i <= priv->hw_params->tx_queues; i++)
+ u64_stats_init(&priv->tx_rings[i].stats64.syncp);
/* libphy will determine the link state */
netif_carrier_off(dev);
@@ -4257,7 +4204,6 @@ static int bcmgenet_resume(struct device *d)
goto out_clk_disable;
}
- /* Always enable ring 16 - descriptor ring */
bcmgenet_enable_dma(priv, dma_ctrl);
if (!device_may_wakeup(d))
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.h b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
index c0005a0fff56..89b071da3114 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.h
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
- * Copyright (c) 2014-2024 Broadcom
+ * Copyright (c) 2014-2025 Broadcom
*/
#ifndef __BCMGENET_H__
@@ -18,6 +18,9 @@
#include "../unimac.h"
+/* Maximum number of hardware queues, downsized if needed */
+#define GENET_MAX_MQ_CNT 4
+
/* total number of Buffer Descriptors, same for Rx/Tx */
#define TOTAL_DESC 256
@@ -152,6 +155,27 @@ struct bcmgenet_mib_counters {
u32 tx_realloc_tsb_failed;
};
+struct bcmgenet_tx_stats64 {
+ struct u64_stats_sync syncp;
+ u64_stats_t packets;
+ u64_stats_t bytes;
+ u64_stats_t errors;
+ u64_stats_t dropped;
+};
+
+struct bcmgenet_rx_stats64 {
+ struct u64_stats_sync syncp;
+ u64_stats_t bytes;
+ u64_stats_t packets;
+ u64_stats_t errors;
+ u64_stats_t dropped;
+ u64_stats_t multicast;
+ u64_stats_t length_errors;
+ u64_stats_t over_errors;
+ u64_stats_t crc_errors;
+ u64_stats_t frame_errors;
+};
+
#define UMAC_MIB_START 0x400
#define UMAC_MDIO_CMD 0x614
@@ -510,10 +534,8 @@ struct bcmgenet_skb_cb {
struct bcmgenet_tx_ring {
spinlock_t lock; /* ring lock */
struct napi_struct napi; /* NAPI per tx queue */
- unsigned long packets;
- unsigned long bytes;
+ struct bcmgenet_tx_stats64 stats64;
unsigned int index; /* ring index */
- unsigned int queue; /* queue index */
struct enet_cb *cbs; /* tx ring buffer control block*/
unsigned int size; /* size of each tx ring */
unsigned int clean_ptr; /* Tx ring clean pointer */
@@ -523,8 +545,6 @@ struct bcmgenet_tx_ring {
unsigned int prod_index; /* Tx ring producer index SW copy */
unsigned int cb_ptr; /* Tx ring initial CB ptr */
unsigned int end_ptr; /* Tx ring end CB ptr */
- void (*int_enable)(struct bcmgenet_tx_ring *);
- void (*int_disable)(struct bcmgenet_tx_ring *);
struct bcmgenet_priv *priv;
};
@@ -538,10 +558,7 @@ struct bcmgenet_net_dim {
struct bcmgenet_rx_ring {
struct napi_struct napi; /* Rx NAPI struct */
- unsigned long bytes;
- unsigned long packets;
- unsigned long errors;
- unsigned long dropped;
+ struct bcmgenet_rx_stats64 stats64;
unsigned int index; /* Rx ring index */
struct enet_cb *cbs; /* Rx ring buffer control block */
unsigned int size; /* Rx ring size */
@@ -553,8 +570,6 @@ struct bcmgenet_rx_ring {
struct bcmgenet_net_dim dim;
u32 rx_max_coalesced_frames;
u32 rx_coalesce_usecs;
- void (*int_enable)(struct bcmgenet_rx_ring *);
- void (*int_disable)(struct bcmgenet_rx_ring *);
struct bcmgenet_priv *priv;
};
@@ -583,7 +598,7 @@ struct bcmgenet_priv {
struct enet_cb *tx_cbs;
unsigned int num_tx_bds;
- struct bcmgenet_tx_ring tx_rings[DESC_INDEX + 1];
+ struct bcmgenet_tx_ring tx_rings[GENET_MAX_MQ_CNT + 1];
/* receive variables */
void __iomem *rx_bds;
@@ -593,7 +608,7 @@ struct bcmgenet_priv {
struct bcmgenet_rxnfc_rule rxnfc_rules[MAX_NUM_OF_FS_RULES];
struct list_head rxnfc_list;
- struct bcmgenet_rx_ring rx_rings[DESC_INDEX + 1];
+ struct bcmgenet_rx_ring rx_rings[GENET_MAX_MQ_CNT + 1];
/* other misc variables */
struct bcmgenet_hw_params *hw_params;
@@ -648,6 +663,31 @@ struct bcmgenet_priv {
struct bcmgenet_mib_counters mib;
};
+static inline bool bcmgenet_has_40bits(struct bcmgenet_priv *priv)
+{
+ return !!(priv->hw_params->flags & GENET_HAS_40BITS);
+}
+
+static inline bool bcmgenet_has_ext(struct bcmgenet_priv *priv)
+{
+ return !!(priv->hw_params->flags & GENET_HAS_EXT);
+}
+
+static inline bool bcmgenet_has_mdio_intr(struct bcmgenet_priv *priv)
+{
+ return !!(priv->hw_params->flags & GENET_HAS_MDIO_INTR);
+}
+
+static inline bool bcmgenet_has_moca_link_det(struct bcmgenet_priv *priv)
+{
+ return !!(priv->hw_params->flags & GENET_HAS_MOCA_LINK_DET);
+}
+
+static inline bool bcmgenet_has_ephy_16nm(struct bcmgenet_priv *priv)
+{
+ return priv->ephy_16nm;
+}
+
#define GENET_IO_MACRO(name, offset) \
static inline u32 bcmgenet_##name##_readl(struct bcmgenet_priv *priv, \
u32 off) \
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c b/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c
index 2033fb9d893e..98358b71cef5 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c
@@ -2,7 +2,7 @@
/*
* Broadcom GENET (Gigabit Ethernet) Wake-on-LAN support
*
- * Copyright (c) 2014-2024 Broadcom
+ * Copyright (c) 2014-2025 Broadcom
*/
#define pr_fmt(fmt) "bcmgenet_wol: " fmt
@@ -180,7 +180,7 @@ int bcmgenet_wol_power_down_cfg(struct bcmgenet_priv *priv,
if (priv->wolopts & WAKE_FILTER) {
list_for_each_entry(rule, &priv->rxnfc_list, list)
if (rule->fs.ring_cookie == RX_CLS_FLOW_WAKE)
- hfb_enable |= (1 << rule->fs.location);
+ hfb_enable |= (1 << (rule->fs.location + 1));
reg = (hfb_ctrl_reg & ~RBUF_HFB_EN) | RBUF_ACPI_EN;
bcmgenet_hfb_reg_writel(priv, reg, HFB_CTRL);
}
diff --git a/drivers/net/ethernet/broadcom/genet/bcmmii.c b/drivers/net/ethernet/broadcom/genet/bcmmii.c
index 9beb65e6d0a9..eeb2aa75efda 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmmii.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmmii.c
@@ -2,7 +2,7 @@
/*
* Broadcom GENET MDIO routines
*
- * Copyright (c) 2014-2024 Broadcom
+ * Copyright (c) 2014-2025 Broadcom
*/
#include <linux/acpi.h>
@@ -151,7 +151,7 @@ void bcmgenet_phy_power_set(struct net_device *dev, bool enable)
u32 reg = 0;
/* EXT_GPHY_CTRL is only valid for GENETv4 and onward */
- if (GENET_IS_V4(priv) || priv->ephy_16nm) {
+ if (GENET_IS_V4(priv) || bcmgenet_has_ephy_16nm(priv)) {
reg = bcmgenet_ext_readl(priv, EXT_GPHY_CTRL);
if (enable) {
reg &= ~EXT_CK25_DIS;
@@ -181,7 +181,7 @@ void bcmgenet_phy_power_set(struct net_device *dev, bool enable)
static void bcmgenet_moca_phy_setup(struct bcmgenet_priv *priv)
{
- if (priv->hw_params->flags & GENET_HAS_MOCA_LINK_DET)
+ if (bcmgenet_has_moca_link_det(priv))
fixed_phy_set_link_update(priv->dev->phydev,
bcmgenet_fixed_phy_link_update);
}
diff --git a/drivers/net/ethernet/freescale/Makefile b/drivers/net/ethernet/freescale/Makefile
index de7b31842233..d0a259e47960 100644
--- a/drivers/net/ethernet/freescale/Makefile
+++ b/drivers/net/ethernet/freescale/Makefile
@@ -22,6 +22,5 @@ ucc_geth_driver-objs := ucc_geth.o ucc_geth_ethtool.o
obj-$(CONFIG_FSL_FMAN) += fman/
obj-$(CONFIG_FSL_DPAA_ETH) += dpaa/
-obj-$(CONFIG_FSL_DPAA2_ETH) += dpaa2/
-
+obj-y += dpaa2/
obj-y += enetc/
diff --git a/drivers/net/ethernet/freescale/dpaa2/Kconfig b/drivers/net/ethernet/freescale/dpaa2/Kconfig
index d029b69c3f18..36280e5d99e1 100644
--- a/drivers/net/ethernet/freescale/dpaa2/Kconfig
+++ b/drivers/net/ethernet/freescale/dpaa2/Kconfig
@@ -34,6 +34,10 @@ config FSL_DPAA2_SWITCH
tristate "Freescale DPAA2 Ethernet Switch"
depends on BRIDGE || BRIDGE=n
depends on NET_SWITCHDEV
+ depends on FSL_MC_BUS && FSL_MC_DPIO
+ select PHYLINK
+ select PCS_LYNX
+ select FSL_XGMAC_MDIO
help
Driver for Freescale DPAA2 Ethernet Switch. This driver manages
switch objects discovered on the Freeescale MC bus.
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 4d9dcb0001d2..3f0dd19ba399 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -7700,6 +7700,7 @@ static int e1000_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
err_register:
if (!(adapter->flags & FLAG_HAS_AMT))
e1000e_release_hw_control(adapter);
+ e1000e_ptp_remove(adapter);
err_eeprom:
if (hw->phy.ops.check_reset_block && !hw->phy.ops.check_reset_block(hw))
e1000_phy_hw_reset(&adapter->hw);
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index de3d5e5b8306..6fc911ec809e 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -1310,6 +1310,7 @@ void i40e_ptp_restore_hw_time(struct i40e_pf *pf);
void i40e_ptp_init(struct i40e_pf *pf);
void i40e_ptp_stop(struct i40e_pf *pf);
int i40e_ptp_alloc_pins(struct i40e_pf *pf);
+void i40e_ptp_free_pins(struct i40e_pf *pf);
int i40e_update_adq_vsi_queues(struct i40e_vsi *vsi, int vsi_offset);
int i40e_is_vsi_uplink_mode_veb(struct i40e_vsi *vsi);
int i40e_get_partition_bw_setting(struct i40e_pf *pf);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index e7a06db26c91..059cf9376ffc 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -13883,7 +13883,6 @@ static int i40e_config_netdev(struct i40e_vsi *vsi)
netdev->neigh_priv_len = sizeof(u32) * 4;
netdev->priv_flags |= IFF_UNICAST_FLT;
- netdev->priv_flags |= IFF_SUPP_NOFCS;
/* Setup netdev TC information */
i40e_vsi_config_netdev_tc(vsi, vsi->tc_config.enabled_tc);
@@ -16227,6 +16226,7 @@ static int i40e_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
i40e_clear_interrupt_scheme(pf);
kfree(pf->vsi);
err_switch_setup:
+ i40e_ptp_free_pins(pf);
i40e_reset_interrupt_capability(pf);
timer_shutdown_sync(&pf->service_timer);
err_mac_addr:
diff --git a/drivers/net/ethernet/intel/i40e/i40e_ptp.c b/drivers/net/ethernet/intel/i40e/i40e_ptp.c
index b72a4b5d76b9..38c1bfb93ad0 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ptp.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ptp.c
@@ -937,12 +937,13 @@ int i40e_ptp_get_ts_config(struct i40e_pf *pf, struct ifreq *ifr)
*
* Release memory allocated for PTP pins.
**/
-static void i40e_ptp_free_pins(struct i40e_pf *pf)
+void i40e_ptp_free_pins(struct i40e_pf *pf)
{
if (i40e_is_ptp_pin_dev(&pf->hw)) {
kfree(pf->ptp_pins);
kfree(pf->ptp_caps.pin_config);
pf->ptp_pins = NULL;
+ pf->ptp_caps.pin_config = NULL;
}
}
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 48cd1d06761c..8cd742c4da91 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -158,11 +158,10 @@ struct iavf_vlan {
enum iavf_vlan_state_t {
IAVF_VLAN_INVALID,
IAVF_VLAN_ADD, /* filter needs to be added */
- IAVF_VLAN_IS_NEW, /* filter is new, wait for PF answer */
- IAVF_VLAN_ACTIVE, /* filter is accepted by PF */
- IAVF_VLAN_DISABLE, /* filter needs to be deleted by PF, then marked INACTIVE */
- IAVF_VLAN_INACTIVE, /* filter is inactive, we are in IFF_DOWN */
- IAVF_VLAN_REMOVE, /* filter needs to be removed from list */
+ IAVF_VLAN_ADDING, /* ADD sent to PF, waiting for response */
+ IAVF_VLAN_ACTIVE, /* PF confirmed, filter is in HW */
+ IAVF_VLAN_REMOVE, /* filter queued for DEL from PF */
+ IAVF_VLAN_REMOVING, /* DEL sent to PF, waiting for response */
};
struct iavf_vlan_filter {
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 5f07f37933a0..383e015a6f4e 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -774,10 +774,10 @@ iavf_vlan_filter *iavf_add_vlan(struct iavf_adapter *adapter,
adapter->num_vlan_filters++;
iavf_schedule_aq_request(adapter, IAVF_FLAG_AQ_ADD_VLAN_FILTER);
} else if (f->state == IAVF_VLAN_REMOVE) {
- /* Re-add the filter since we cannot tell whether the
- * pending delete has already been processed by the PF.
- * A duplicate add is harmless.
- */
+ /* DEL not yet sent to PF, cancel it */
+ f->state = IAVF_VLAN_ACTIVE;
+ } else if (f->state == IAVF_VLAN_REMOVING) {
+ /* DEL already sent to PF, re-add after completion */
f->state = IAVF_VLAN_ADD;
iavf_schedule_aq_request(adapter,
IAVF_FLAG_AQ_ADD_VLAN_FILTER);
@@ -808,37 +808,19 @@ static void iavf_del_vlan(struct iavf_adapter *adapter, struct iavf_vlan vlan)
list_del(&f->list);
kfree(f);
adapter->num_vlan_filters--;
- } else {
+ } else if (f->state != IAVF_VLAN_REMOVING) {
f->state = IAVF_VLAN_REMOVE;
iavf_schedule_aq_request(adapter,
IAVF_FLAG_AQ_DEL_VLAN_FILTER);
}
+ /* If REMOVING, DEL is already sent to PF; completion
+ * handler will free the filter when PF confirms.
+ */
}
spin_unlock_bh(&adapter->mac_vlan_list_lock);
}
-/**
- * iavf_restore_filters
- * @adapter: board private structure
- *
- * Restore existing non MAC filters when VF netdev comes back up
- **/
-static void iavf_restore_filters(struct iavf_adapter *adapter)
-{
- struct iavf_vlan_filter *f;
-
- /* re-add all VLAN filters */
- spin_lock_bh(&adapter->mac_vlan_list_lock);
-
- list_for_each_entry(f, &adapter->vlan_filter_list, list) {
- if (f->state == IAVF_VLAN_INACTIVE)
- f->state = IAVF_VLAN_ADD;
- }
-
- spin_unlock_bh(&adapter->mac_vlan_list_lock);
- adapter->aq_required |= IAVF_FLAG_AQ_ADD_VLAN_FILTER;
-}
/**
* iavf_get_num_vlans_added - get number of VLANs added
@@ -1257,13 +1239,12 @@ static void iavf_up_complete(struct iavf_adapter *adapter)
}
/**
- * iavf_clear_mac_vlan_filters - Remove mac and vlan filters not sent to PF
- * yet and mark other to be removed.
+ * iavf_clear_mac_filters - Remove MAC filters not sent to PF yet and mark
+ * others to be removed.
* @adapter: board private structure
**/
-static void iavf_clear_mac_vlan_filters(struct iavf_adapter *adapter)
+static void iavf_clear_mac_filters(struct iavf_adapter *adapter)
{
- struct iavf_vlan_filter *vlf, *vlftmp;
struct iavf_mac_filter *f, *ftmp;
spin_lock_bh(&adapter->mac_vlan_list_lock);
@@ -1282,11 +1263,6 @@ static void iavf_clear_mac_vlan_filters(struct iavf_adapter *adapter)
}
}
- /* disable all VLAN filters */
- list_for_each_entry_safe(vlf, vlftmp, &adapter->vlan_filter_list,
- list)
- vlf->state = IAVF_VLAN_DISABLE;
-
spin_unlock_bh(&adapter->mac_vlan_list_lock);
}
@@ -1382,7 +1358,7 @@ void iavf_down(struct iavf_adapter *adapter)
iavf_napi_disable_all(adapter);
iavf_irq_disable(adapter);
- iavf_clear_mac_vlan_filters(adapter);
+ iavf_clear_mac_filters(adapter);
iavf_clear_cloud_filters(adapter);
iavf_clear_fdir_filters(adapter);
iavf_clear_adv_rss_conf(adapter);
@@ -1399,8 +1375,6 @@ void iavf_down(struct iavf_adapter *adapter)
*/
if (!list_empty(&adapter->mac_filter_list))
adapter->aq_required |= IAVF_FLAG_AQ_DEL_MAC_FILTER;
- if (!list_empty(&adapter->vlan_filter_list))
- adapter->aq_required |= IAVF_FLAG_AQ_DEL_VLAN_FILTER;
if (!list_empty(&adapter->cloud_filter_list))
adapter->aq_required |= IAVF_FLAG_AQ_DEL_CLOUD_FILTER;
if (!list_empty(&adapter->fdir_list_head))
@@ -4363,8 +4337,6 @@ static int iavf_open(struct net_device *netdev)
spin_unlock_bh(&adapter->mac_vlan_list_lock);
- /* Restore filters that were removed with IFF_DOWN */
- iavf_restore_filters(adapter);
iavf_restore_fdir_filters(adapter);
iavf_configure(adapter);
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index 7e810b65380c..3e6bdd6ba537 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -636,7 +636,7 @@ static void iavf_vlan_add_reject(struct iavf_adapter *adapter)
spin_lock_bh(&adapter->mac_vlan_list_lock);
list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list, list) {
- if (f->state == IAVF_VLAN_IS_NEW) {
+ if (f->state == IAVF_VLAN_ADDING) {
list_del(&f->list);
kfree(f);
adapter->num_vlan_filters--;
@@ -701,7 +701,7 @@ void iavf_add_vlans(struct iavf_adapter *adapter)
if (f->state == IAVF_VLAN_ADD) {
vvfl->vlan_id[i] = f->vlan.vid;
i++;
- f->state = IAVF_VLAN_IS_NEW;
+ f->state = IAVF_VLAN_ADDING;
if (i == count)
break;
}
@@ -762,7 +762,7 @@ void iavf_add_vlans(struct iavf_adapter *adapter)
vlan->tpid = f->vlan.tpid;
i++;
- f->state = IAVF_VLAN_IS_NEW;
+ f->state = IAVF_VLAN_ADDING;
}
}
@@ -799,22 +799,12 @@ void iavf_del_vlans(struct iavf_adapter *adapter)
spin_lock_bh(&adapter->mac_vlan_list_lock);
list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list, list) {
- /* since VLAN capabilities are not allowed, we dont want to send
- * a VLAN delete request because it will most likely fail and
- * create unnecessary errors/noise, so just free the VLAN
- * filters marked for removal to enable bailing out before
- * sending a virtchnl message
- */
if (f->state == IAVF_VLAN_REMOVE &&
!VLAN_FILTERING_ALLOWED(adapter)) {
list_del(&f->list);
kfree(f);
adapter->num_vlan_filters--;
- } else if (f->state == IAVF_VLAN_DISABLE &&
- !VLAN_FILTERING_ALLOWED(adapter)) {
- f->state = IAVF_VLAN_INACTIVE;
- } else if (f->state == IAVF_VLAN_REMOVE ||
- f->state == IAVF_VLAN_DISABLE) {
+ } else if (f->state == IAVF_VLAN_REMOVE) {
count++;
}
}
@@ -845,18 +835,10 @@ void iavf_del_vlans(struct iavf_adapter *adapter)
vvfl->vsi_id = adapter->vsi_res->vsi_id;
vvfl->num_elements = count;
- list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list, list) {
- if (f->state == IAVF_VLAN_DISABLE) {
- vvfl->vlan_id[i] = f->vlan.vid;
- f->state = IAVF_VLAN_INACTIVE;
- i++;
- if (i == count)
- break;
- } else if (f->state == IAVF_VLAN_REMOVE) {
+ list_for_each_entry(f, &adapter->vlan_filter_list, list) {
+ if (f->state == IAVF_VLAN_REMOVE) {
vvfl->vlan_id[i] = f->vlan.vid;
- list_del(&f->list);
- kfree(f);
- adapter->num_vlan_filters--;
+ f->state = IAVF_VLAN_REMOVING;
i++;
if (i == count)
break;
@@ -892,9 +874,8 @@ void iavf_del_vlans(struct iavf_adapter *adapter)
vvfl_v2->vport_id = adapter->vsi_res->vsi_id;
vvfl_v2->num_elements = count;
- list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list, list) {
- if (f->state == IAVF_VLAN_DISABLE ||
- f->state == IAVF_VLAN_REMOVE) {
+ list_for_each_entry(f, &adapter->vlan_filter_list, list) {
+ if (f->state == IAVF_VLAN_REMOVE) {
struct virtchnl_vlan_supported_caps *filtering_support =
&adapter->vlan_v2_caps.filtering.filtering_support;
struct virtchnl_vlan *vlan;
@@ -908,13 +889,7 @@ void iavf_del_vlans(struct iavf_adapter *adapter)
vlan->tci = f->vlan.vid;
vlan->tpid = f->vlan.tpid;
- if (f->state == IAVF_VLAN_DISABLE) {
- f->state = IAVF_VLAN_INACTIVE;
- } else {
- list_del(&f->list);
- kfree(f);
- adapter->num_vlan_filters--;
- }
+ f->state = IAVF_VLAN_REMOVING;
i++;
if (i == count)
break;
@@ -2061,10 +2036,6 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
ether_addr_copy(adapter->hw.mac.addr, netdev->dev_addr);
wake_up(&adapter->vc_waitqueue);
break;
- case VIRTCHNL_OP_DEL_VLAN:
- dev_err(&adapter->pdev->dev, "Failed to delete VLAN filter, error %s\n",
- iavf_stat_str(&adapter->hw, v_retval));
- break;
case VIRTCHNL_OP_DEL_ETH_ADDR:
dev_err(&adapter->pdev->dev, "Failed to delete MAC filter, error %s\n",
iavf_stat_str(&adapter->hw, v_retval));
@@ -2544,17 +2515,42 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
spin_unlock_bh(&adapter->adv_rss_lock);
}
break;
+ case VIRTCHNL_OP_ADD_VLAN:
case VIRTCHNL_OP_ADD_VLAN_V2: {
struct iavf_vlan_filter *f;
+ if (v_retval)
+ break;
+
spin_lock_bh(&adapter->mac_vlan_list_lock);
list_for_each_entry(f, &adapter->vlan_filter_list, list) {
- if (f->state == IAVF_VLAN_IS_NEW)
+ if (f->state == IAVF_VLAN_ADDING)
f->state = IAVF_VLAN_ACTIVE;
}
spin_unlock_bh(&adapter->mac_vlan_list_lock);
}
break;
+ case VIRTCHNL_OP_DEL_VLAN:
+ case VIRTCHNL_OP_DEL_VLAN_V2: {
+ struct iavf_vlan_filter *f, *ftmp;
+
+ spin_lock_bh(&adapter->mac_vlan_list_lock);
+ list_for_each_entry_safe(f, ftmp, &adapter->vlan_filter_list,
+ list) {
+ if (f->state == IAVF_VLAN_REMOVING) {
+ if (v_retval) {
+ /* PF rejected DEL, keep filter */
+ f->state = IAVF_VLAN_ACTIVE;
+ } else {
+ list_del(&f->list);
+ kfree(f);
+ adapter->num_vlan_filters--;
+ }
+ }
+ }
+ spin_unlock_bh(&adapter->mac_vlan_list_lock);
+ }
+ break;
case VIRTCHNL_OP_ENABLE_VLAN_STRIPPING:
/* PF enabled vlan strip on this VF.
* Update netdev->features if needed to be in sync with ethtool.
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 66ae0352c6bc..d8013818da96 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -1347,7 +1347,7 @@ struct ice_aqc_get_link_status_data {
#define ICE_AQ_LINK_PWR_QSFP_CLASS_3 2
#define ICE_AQ_LINK_PWR_QSFP_CLASS_4 3
__le16 link_speed;
-#define ICE_AQ_LINK_SPEED_M 0x7FF
+#define ICE_AQ_LINK_SPEED_M GENMASK(11, 0)
#define ICE_AQ_LINK_SPEED_10MB BIT(0)
#define ICE_AQ_LINK_SPEED_100MB BIT(1)
#define ICE_AQ_LINK_SPEED_1000MB BIT(2)
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_consts.h b/drivers/net/ethernet/intel/ice/ice_ptp_consts.h
index bdb1020147d1..91cc7df5cab5 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp_consts.h
+++ b/drivers/net/ethernet/intel/ice/ice_ptp_consts.h
@@ -123,14 +123,14 @@ struct ice_eth56g_mac_reg_cfg eth56g_mac_cfg[NUM_ICE_ETH56G_LNK_SPD] = {
.blktime = 0x666, /* 3.2 */
.tx_offset = {
.serdes = 0x234c, /* 17.6484848 */
- .no_fec = 0x8e80, /* 71.25 */
+ .no_fec = 0x93d9, /* 73 */
.fc = 0xb4a4, /* 90.32 */
.sfd = 0x4a4, /* 2.32 */
.onestep = 0x4ccd /* 38.4 */
},
.rx_offset = {
.serdes = 0xffffeb27, /* -10.42424 */
- .no_fec = 0xffffcccd, /* -25.6 */
+ .no_fec = 0xffffc7b6, /* -28 */
.fc = 0xfffc557b, /* -469.26 */
.sfd = 0x4a4, /* 2.32 */
.bs_ds = 0x32 /* 0.0969697 */
@@ -163,17 +163,17 @@ struct ice_eth56g_mac_reg_cfg eth56g_mac_cfg[NUM_ICE_ETH56G_LNK_SPD] = {
.mktime = 0x147b, /* 10.24, only if RS-FEC enabled */
.tx_offset = {
.serdes = 0xe1e, /* 7.0593939 */
- .no_fec = 0x3857, /* 28.17 */
+ .no_fec = 0x4266, /* 33 */
.fc = 0x48c3, /* 36.38 */
- .rs = 0x8100, /* 64.5 */
+ .rs = 0x8a00, /* 69 */
.sfd = 0x1dc, /* 0.93 */
.onestep = 0x1eb8 /* 15.36 */
},
.rx_offset = {
.serdes = 0xfffff7a9, /* -4.1697 */
- .no_fec = 0xffffe71a, /* -12.45 */
+ .no_fec = 0xffffe700, /* -12 */
.fc = 0xfffe894d, /* -187.35 */
- .rs = 0xfffff8cd, /* -3.6 */
+ .rs = 0xfffff8cc, /* -3 */
.sfd = 0x1dc, /* 0.93 */
.bs_ds = 0x14 /* 0.0387879, RS-FEC 0 */
}
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c
index 7190fde16c86..478ee1c54014 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c
+++ b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c
@@ -2340,6 +2340,8 @@ static int ice_phy_cfg_mac_eth56g(struct ice_hw *hw, u8 port)
* @ena: enable or disable interrupt
* @threshold: interrupt threshold
*
+ * The threshold cannot be 0 while the interrupt is enabled.
+ *
* Configure TX timestamp interrupt for the specified port
*
* Return:
@@ -2351,19 +2353,45 @@ int ice_phy_cfg_intr_eth56g(struct ice_hw *hw, u8 port, bool ena, u8 threshold)
int err;
u32 val;
+ if (ena && !threshold)
+ return -EINVAL;
+
err = ice_read_ptp_reg_eth56g(hw, port, PHY_REG_TS_INT_CONFIG, &val);
if (err)
return err;
+ val &= ~PHY_TS_INT_CONFIG_ENA_M;
if (ena) {
- val |= PHY_TS_INT_CONFIG_ENA_M;
val &= ~PHY_TS_INT_CONFIG_THRESHOLD_M;
val |= FIELD_PREP(PHY_TS_INT_CONFIG_THRESHOLD_M, threshold);
- } else {
- val &= ~PHY_TS_INT_CONFIG_ENA_M;
+ err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TS_INT_CONFIG,
+ val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP,
+ "Failed to update 'threshold' PHY_REG_TS_INT_CONFIG port=%u ena=%u threshold=%u\n",
+ port, !!ena, threshold);
+ return err;
+ }
+ val |= PHY_TS_INT_CONFIG_ENA_M;
}
- return ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TS_INT_CONFIG, val);
+ err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TS_INT_CONFIG, val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP,
+ "Failed to update 'ena' PHY_REG_TS_INT_CONFIG port=%u ena=%u threshold=%u\n",
+ port, !!ena, threshold);
+ return err;
+ }
+
+ err = ice_read_ptp_reg_eth56g(hw, port, PHY_REG_TS_INT_CONFIG, &val);
+ if (err) {
+ ice_debug(hw, ICE_DBG_PTP,
+ "Failed to read PHY_REG_TS_INT_CONFIG port=%u ena=%u threshold=%u\n",
+ port, !!ena, threshold);
+ return err;
+ }
+
+ return 0;
}
/**
@@ -2654,13 +2682,19 @@ int ice_ptp_read_tx_hwtstamp_status_eth56g(struct ice_hw *hw, u32 *ts_status)
*ts_status = 0;
for (phy = 0; phy < params->num_phys; phy++) {
+ u8 port;
int err;
- err = ice_read_phy_eth56g(hw, phy, PHY_PTP_INT_STATUS, &status);
+ /* ice_read_phy_eth56g expects a port index, so use the first
+ * port of the PHY
+ */
+ port = phy * hw->ptp.ports_per_phy;
+
+ err = ice_read_phy_eth56g(hw, port, PHY_PTP_INT_STATUS, &status);
if (err)
return err;
- *ts_status |= (status & mask) << (phy * hw->ptp.ports_per_phy);
+ *ts_status |= (status & mask) << port;
}
ice_debug(hw, ICE_DBG_PTP, "PHY interrupt err: %x\n", *ts_status);
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index 431a6ed498a4..48434a79869c 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -2346,8 +2346,8 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)
ice_trace(xmit_frame_ring, tx_ring, skb);
- if (unlikely(ipv6_hopopt_jumbo_remove(skb)))
- goto out_drop;
+ /* record the location of the first descriptor for this packet */
+ first = &tx_ring->tx_buf[tx_ring->next_to_use];
count = ice_xmit_desc_count(skb);
if (ice_chk_linearize(skb, count)) {
@@ -2374,8 +2374,6 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)
offload.tx_ring = tx_ring;
- /* record the location of the first descriptor for this packet */
- first = &tx_ring->tx_buf[tx_ring->next_to_use];
first->skb = skb;
first->type = ICE_TX_BUF_SKB;
first->bytecount = max_t(unsigned int, skb->len, ETH_ZLEN);
@@ -2439,6 +2437,7 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)
out_drop:
ice_trace(xmit_frame_ring_drop, tx_ring, skb);
dev_kfree_skb_any(skb);
+ first->type = ICE_TX_BUF_EMPTY;
return NETDEV_TX_OK;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
index 815ad0bfe832..5267b08011fc 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
@@ -803,7 +803,12 @@ void ice_reset_all_vfs(struct ice_pf *pf)
ice_vf_ctrl_invalidate_vsi(vf);
ice_vf_pre_vsi_rebuild(vf);
- ice_vf_rebuild_vsi(vf);
+ if (ice_vf_rebuild_vsi(vf)) {
+ dev_err(dev, "VF %u VSI rebuild failed, leaving VF disabled\n",
+ vf->vf_id);
+ mutex_unlock(&vf->cfg_lock);
+ continue;
+ }
ice_vf_post_vsi_rebuild(vf);
ice_eswitch_attach_vf(pf, vf);
diff --git a/drivers/net/ethernet/mediatek/airoha_eth.c b/drivers/net/ethernet/mediatek/airoha_eth.c
index 7a85550e5ecb..7b929b20fe64 100644
--- a/drivers/net/ethernet/mediatek/airoha_eth.c
+++ b/drivers/net/ethernet/mediatek/airoha_eth.c
@@ -1574,14 +1574,18 @@ static int airoha_qdma_init_rx_queue(struct airoha_queue *q,
dma_addr_t dma_addr;
q->buf_size = PAGE_SIZE / 2;
- q->ndesc = ndesc;
q->qdma = qdma;
- q->entry = devm_kzalloc(eth->dev, q->ndesc * sizeof(*q->entry),
+ q->entry = devm_kzalloc(eth->dev, ndesc * sizeof(*q->entry),
GFP_KERNEL);
if (!q->entry)
return -ENOMEM;
+ q->desc = dmam_alloc_coherent(eth->dev, ndesc * sizeof(*q->desc),
+ &dma_addr, GFP_KERNEL);
+ if (!q->desc)
+ return -ENOMEM;
+
q->page_pool = page_pool_create(&pp_params);
if (IS_ERR(q->page_pool)) {
int err = PTR_ERR(q->page_pool);
@@ -1590,11 +1594,7 @@ static int airoha_qdma_init_rx_queue(struct airoha_queue *q,
return err;
}
- q->desc = dmam_alloc_coherent(eth->dev, q->ndesc * sizeof(*q->desc),
- &dma_addr, GFP_KERNEL);
- if (!q->desc)
- return -ENOMEM;
-
+ q->ndesc = ndesc;
netif_napi_add(eth->napi_dev, &q->napi, airoha_qdma_rx_napi_poll);
airoha_qdma_wr(qdma, REG_RX_RING_BASE(qid), dma_addr);
@@ -1641,6 +1641,11 @@ static void airoha_qdma_cleanup_rx_queue(struct airoha_queue *q)
}
q->head = q->tail;
+ /* Set RX_DMA_IDX to RX_CPU_IDX to notify the hw the QDMA RX ring is
+ * empty.
+ */
+ airoha_qdma_rmw(qdma, REG_RX_CPU_IDX(qid), RX_RING_CPU_IDX_MASK,
+ FIELD_PREP(RX_RING_CPU_IDX_MASK, q->head));
airoha_qdma_rmw(qdma, REG_RX_DMA_IDX(qid), RX_RING_DMA_IDX_MASK,
FIELD_PREP(RX_RING_DMA_IDX_MASK, q->tail));
}
@@ -1729,9 +1734,11 @@ static int airoha_qdma_tx_napi_poll(struct napi_struct *napi, int budget)
WRITE_ONCE(desc->msg1, 0);
if (skb) {
+ u16 queue = skb_get_queue_mapping(skb);
struct netdev_queue *txq;
- txq = netdev_get_tx_queue(skb->dev, qid);
+ txq = netdev_get_tx_queue(skb->dev, queue);
+ netdev_tx_completed_queue(txq, 1, skb->len);
if (netif_tx_queue_stopped(txq) &&
q->ndesc - q->queued >= q->free_thr)
netif_tx_wake_queue(txq);
@@ -2510,7 +2517,9 @@ static netdev_tx_t airoha_dev_xmit(struct sk_buff *skb,
q->queued += i;
skb_tx_timestamp(skb);
- if (!netdev_xmit_more())
+ netdev_tx_sent_queue(txq, skb->len);
+
+ if (netif_xmit_stopped(txq) || !netdev_xmit_more())
airoha_qdma_rmw(qdma, REG_TX_CPU_IDX(qid),
TX_RING_CPU_IDX_MASK,
FIELD_PREP(TX_RING_CPU_IDX_MASK, q->head));
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
index 40fe3d1e2342..8f22559e373c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/* Copyright (c) 2017, Mellanox Technologies inc. All rights reserved. */
+#include <linux/iopoll.h>
+
#include "mlx5_core.h"
#include "en.h"
#include "ipsec.h"
@@ -593,7 +595,6 @@ int mlx5e_ipsec_aso_query(struct mlx5e_ipsec_sa_entry *sa_entry,
struct mlx5_wqe_aso_ctrl_seg *ctrl;
struct mlx5e_hw_objs *res;
struct mlx5_aso_wqe *wqe;
- unsigned long expires;
u8 ds_cnt;
int ret;
@@ -615,13 +616,8 @@ int mlx5e_ipsec_aso_query(struct mlx5e_ipsec_sa_entry *sa_entry,
mlx5e_ipsec_aso_copy(ctrl, data);
mlx5_aso_post_wqe(aso->aso, false, &wqe->ctrl);
- expires = jiffies + msecs_to_jiffies(10);
- do {
- ret = mlx5_aso_poll_cq(aso->aso, false);
- if (ret)
- /* We are in atomic context */
- udelay(10);
- } while (ret && time_is_after_jiffies(expires));
+ read_poll_timeout_atomic(mlx5_aso_poll_cq, ret, !ret, 10,
+ 10 * USEC_PER_MSEC, false, aso->aso, false);
if (!ret)
memcpy(sa_entry->ctx, aso->ctx, MLX5_ST_SZ_BYTES(ipsec_aso));
spin_unlock_bh(&aso->lock);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 5736ed61e6eb..dbfb900980de 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -6466,6 +6466,14 @@ static int _mlx5e_probe(struct auxiliary_device *adev)
goto err_resume;
}
+ /* mlx5e_fix_features() returns early when the device is not present
+ * to avoid dereferencing cleared priv during profile changes.
+ * This also causes it to be a no-op during register_netdev(), where
+ * the device is not yet present.
+ * Trigger an additional features update that will actually work.
+ */
+ mlx5e_update_features(netdev);
+
mlx5e_dcbnl_init_app(priv);
mlx5_core_uplink_netdev_set(mdev, netdev);
mlx5e_params_print_info(mdev, &priv->channels.params);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 8856949fbe6a..4d8295249c42 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -1885,7 +1885,7 @@ int mlx5_mdev_init(struct mlx5_core_dev *dev, int profile_idx)
err = mlx5_notifiers_init(dev);
if (err)
- goto err_hca_caps;
+ goto err_notifiers_init;
/* The conjunction of sw_vhca_id with sw_owner_id will be a global
* unique id per function which uses mlx5_core.
@@ -1901,6 +1901,8 @@ int mlx5_mdev_init(struct mlx5_core_dev *dev, int profile_idx)
return 0;
+err_notifiers_init:
+ mlx5_hca_caps_free(dev);
err_hca_caps:
mlx5_adev_cleanup(dev);
err_adev_init:
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
index 72bdc6c76c0c..53bb1d691cc0 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
@@ -137,7 +137,7 @@ void fbnic_up(struct fbnic_net *fbn)
/* Enable Tx/Rx processing */
fbnic_napi_enable(fbn);
- netif_tx_start_all_queues(fbn->netdev);
+ netif_tx_wake_all_queues(fbn->netdev);
fbnic_service_task_start(fbn);
}
diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_target.c b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_target.c
index 79470f198a62..9cf19446657c 100644
--- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_target.c
+++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_target.c
@@ -435,12 +435,17 @@ static int nfp_encode_basic_qdr(u64 addr, int dest_island, int cpp_tgt,
/* Full Island ID and channel bits overlap? */
ret = nfp_decode_basic(addr, &v, cpp_tgt, mode, addr40, isld1, isld0);
- if (ret)
+ if (ret) {
+ pr_warn("%s: decode dest_island failed: %d\n", __func__, ret);
return ret;
+ }
/* The current address won't go where expected? */
- if (dest_island != -1 && dest_island != v)
+ if (dest_island != -1 && dest_island != v) {
+ pr_warn("%s: dest_island mismatch: current (%d) != decoded (%d)\n",
+ __func__, dest_island, v);
return -EINVAL;
+ }
/* If dest_island was -1, we don't care where it goes. */
return 0;
@@ -493,7 +498,7 @@ static int nfp_encode_basic(u64 *addr, int dest_island, int cpp_tgt,
* the address but we can verify if the existing
* contents will point to a valid island.
*/
- return nfp_encode_basic_qdr(*addr, cpp_tgt, dest_island,
+ return nfp_encode_basic_qdr(*addr, dest_island, cpp_tgt,
mode, addr40, isld1, isld0);
iid_lsb = addr40 ? 34 : 26;
@@ -504,7 +509,7 @@ static int nfp_encode_basic(u64 *addr, int dest_island, int cpp_tgt,
return 0;
case 1:
if (cpp_tgt == NFP_CPP_TARGET_QDR && !addr40)
- return nfp_encode_basic_qdr(*addr, cpp_tgt, dest_island,
+ return nfp_encode_basic_qdr(*addr, dest_island, cpp_tgt,
mode, addr40, isld1, isld0);
idx_lsb = addr40 ? 39 : 31;
@@ -530,7 +535,7 @@ static int nfp_encode_basic(u64 *addr, int dest_island, int cpp_tgt,
* be set before hand and with them select an island.
* So we need to confirm that it's at least plausible.
*/
- return nfp_encode_basic_qdr(*addr, cpp_tgt, dest_island,
+ return nfp_encode_basic_qdr(*addr, dest_island, cpp_tgt,
mode, addr40, isld1, isld0);
/* Make sure we compare against isldN values
@@ -551,7 +556,7 @@ static int nfp_encode_basic(u64 *addr, int dest_island, int cpp_tgt,
* iid<1> = addr<30> = channel<0>
* channel<1> = addr<31> = Index
*/
- return nfp_encode_basic_qdr(*addr, cpp_tgt, dest_island,
+ return nfp_encode_basic_qdr(*addr, dest_island, cpp_tgt,
mode, addr40, isld1, isld0);
isld[0] &= ~3;
diff --git a/drivers/net/ethernet/sfc/efx_devlink.c b/drivers/net/ethernet/sfc/efx_devlink.c
index 3cd750820fdd..d5a4b3cf9454 100644
--- a/drivers/net/ethernet/sfc/efx_devlink.c
+++ b/drivers/net/ethernet/sfc/efx_devlink.c
@@ -530,7 +530,7 @@ static int efx_devlink_info_running_versions(struct efx_nic *efx,
if (rc || outlength < MC_CMD_GET_VERSION_OUT_LEN) {
netif_err(efx, drv, efx->net_dev,
"mcdi MC_CMD_GET_VERSION failed\n");
- return rc;
+ return rc ?: -EIO;
}
/* Handle previous output */
diff --git a/drivers/net/hamradio/6pack.c b/drivers/net/hamradio/6pack.c
index 0c766c9c3195..437e5e391a77 100644
--- a/drivers/net/hamradio/6pack.c
+++ b/drivers/net/hamradio/6pack.c
@@ -391,7 +391,6 @@ static void sixpack_receive_buf(struct tty_struct *tty, const u8 *cp,
const u8 *fp, size_t count)
{
struct sixpack *sp;
- size_t count1;
if (!count)
return;
@@ -401,16 +400,16 @@ static void sixpack_receive_buf(struct tty_struct *tty, const u8 *cp,
return;
/* Read the characters out of the buffer */
- count1 = count;
- while (count) {
- count--;
+ while (count--) {
if (fp && *fp++) {
if (!test_and_set_bit(SIXPF_ERROR, &sp->flags))
sp->dev->stats.rx_errors++;
+ cp++;
continue;
}
+ sixpack_decode(sp, cp, 1);
+ cp++;
}
- sixpack_decode(sp, cp, count1);
tty_unthrottle(tty);
}
diff --git a/drivers/net/ipa/gsi.c b/drivers/net/ipa/gsi.c
index 4c3227e77898..624649484d62 100644
--- a/drivers/net/ipa/gsi.c
+++ b/drivers/net/ipa/gsi.c
@@ -2044,6 +2044,7 @@ static int gsi_ring_setup(struct gsi *gsi)
count = reg_decode(reg, NUM_EV_PER_EE, val);
} else {
reg = gsi_reg(gsi, HW_PARAM_4);
+ val = ioread32(gsi->virt + reg_offset(reg));
count = reg_decode(reg, EV_PER_EE, val);
}
if (!count) {
diff --git a/drivers/net/ipa/ipa_main.c b/drivers/net/ipa/ipa_main.c
index 5f3dd5a2dcf4..f374a590d1c5 100644
--- a/drivers/net/ipa/ipa_main.c
+++ b/drivers/net/ipa/ipa_main.c
@@ -361,7 +361,7 @@ static void ipa_qtime_config(struct ipa *ipa)
{
const struct reg *reg;
u32 offset;
- u32 val;
+ u32 val = 0;
/* Timer clock divider must be disabled when we change the rate */
reg = ipa_reg(ipa, TIMERS_XO_CLK_DIV_CFG);
@@ -374,8 +374,8 @@ static void ipa_qtime_config(struct ipa *ipa)
val |= reg_bit(reg, DPL_TIMESTAMP_SEL);
}
/* Configure tag and NAT Qtime timestamp resolution as well */
- val = reg_encode(reg, TAG_TIMESTAMP_LSB, TAG_TIMESTAMP_SHIFT);
- val = reg_encode(reg, NAT_TIMESTAMP_LSB, NAT_TIMESTAMP_SHIFT);
+ val |= reg_encode(reg, TAG_TIMESTAMP_LSB, TAG_TIMESTAMP_SHIFT);
+ val |= reg_encode(reg, NAT_TIMESTAMP_LSB, NAT_TIMESTAMP_SHIFT);
iowrite32(val, ipa->reg_virt + reg_offset(reg));
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index 3770bc84a944..e778367c1d29 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -351,6 +351,7 @@ static void macvlan_broadcast_enqueue(struct macvlan_port *port,
const struct macvlan_dev *src,
struct sk_buff *skb)
{
+ u32 bc_queue_len_used = READ_ONCE(port->bc_queue_len_used);
struct sk_buff *nskb;
int err = -ENOMEM;
@@ -361,7 +362,7 @@ static void macvlan_broadcast_enqueue(struct macvlan_port *port,
MACVLAN_SKB_CB(nskb)->src = src;
spin_lock(&port->bc_queue.lock);
- if (skb_queue_len(&port->bc_queue) < port->bc_queue_len_used) {
+ if (skb_queue_len(&port->bc_queue) < bc_queue_len_used) {
if (src)
dev_hold(src->dev);
__skb_queue_tail(&port->bc_queue, nskb);
@@ -1677,6 +1678,7 @@ static size_t macvlan_get_size(const struct net_device *dev)
+ macvlan_get_size_mac(vlan) /* IFLA_MACVLAN_MACADDR */
+ nla_total_size(4) /* IFLA_MACVLAN_BC_QUEUE_LEN */
+ nla_total_size(4) /* IFLA_MACVLAN_BC_QUEUE_LEN_USED */
+ + nla_total_size(4) /* IFLA_MACVLAN_BC_CUTOFF */
);
}
@@ -1723,7 +1725,8 @@ static int macvlan_fill_info(struct sk_buff *skb,
}
if (nla_put_u32(skb, IFLA_MACVLAN_BC_QUEUE_LEN, vlan->bc_queue_len_req))
goto nla_put_failure;
- if (nla_put_u32(skb, IFLA_MACVLAN_BC_QUEUE_LEN_USED, port->bc_queue_len_used))
+ if (nla_put_u32(skb, IFLA_MACVLAN_BC_QUEUE_LEN_USED,
+ READ_ONCE(port->bc_queue_len_used)))
goto nla_put_failure;
if (port->bc_cutoff != 1 &&
nla_put_s32(skb, IFLA_MACVLAN_BC_CUTOFF, port->bc_cutoff))
@@ -1783,7 +1786,7 @@ static void update_port_bc_queue_len(struct macvlan_port *port)
if (vlan->bc_queue_len_req > max_bc_queue_len_req)
max_bc_queue_len_req = vlan->bc_queue_len_req;
}
- port->bc_queue_len_used = max_bc_queue_len_req;
+ WRITE_ONCE(port->bc_queue_len_used, max_bc_queue_len_req);
}
static int macvlan_device_event(struct notifier_block *unused,
diff --git a/drivers/net/mctp/mctp-i2c.c b/drivers/net/mctp/mctp-i2c.c
index f8f83fe424e5..a939bc084fe8 100644
--- a/drivers/net/mctp/mctp-i2c.c
+++ b/drivers/net/mctp/mctp-i2c.c
@@ -497,8 +497,6 @@ static void mctp_i2c_xmit(struct mctp_i2c_dev *midev, struct sk_buff *skb)
u8 *pecp;
int rc;
- fs = mctp_i2c_get_tx_flow_state(midev, skb);
-
hdr = (void *)skb_mac_header(skb);
/* Sanity check that packet contents matches skb length,
* and can't exceed MCTP_I2C_BUFSZ
@@ -510,6 +508,8 @@ static void mctp_i2c_xmit(struct mctp_i2c_dev *midev, struct sk_buff *skb)
return;
}
+ fs = mctp_i2c_get_tx_flow_state(midev, skb);
+
if (skb_tailroom(skb) >= 1) {
/* Linear case with space, we can just append the PEC */
skb_put(skb, 1);
diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c
index 2f20f9ed3a0d..60375bb814a1 100644
--- a/drivers/net/netconsole.c
+++ b/drivers/net/netconsole.c
@@ -502,6 +502,13 @@ static ssize_t dev_name_store(struct config_item *item, const char *buf,
size_t count)
{
struct netconsole_target *nt = to_target(item);
+ size_t len = count;
+
+ /* Account for a trailing newline appended by tools like echo */
+ if (len && buf[len - 1] == '\n')
+ len--;
+ if (len >= IFNAMSIZ)
+ return -ENAMETOOLONG;
mutex_lock(&dynamic_netconsole_mutex);
if (nt->enabled) {
@@ -652,7 +659,7 @@ static ssize_t remote_mac_store(struct config_item *item, const char *buf,
if (!mac_pton(buf, remote_mac))
goto out_unlock;
- if (buf[3 * ETH_ALEN - 1] && buf[3 * ETH_ALEN - 1] != '\n')
+ if (buf[MAC_ADDR_STR_LEN] && buf[MAC_ADDR_STR_LEN] != '\n')
goto out_unlock;
memcpy(nt->np.remote_mac, remote_mac, ETH_ALEN);
diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index 2614d6509954..daec92570c2e 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -758,7 +758,7 @@ static struct sk_buff *nsim_dev_trap_skb_build(void)
skb->protocol = htons(ETH_P_IP);
skb_set_network_header(skb, skb->len);
- iph = skb_put(skb, sizeof(struct iphdr));
+ iph = skb_put_zero(skb, sizeof(struct iphdr));
iph->protocol = IPPROTO_UDP;
iph->saddr = in_aton("192.0.2.1");
iph->daddr = in_aton("198.51.100.1");
diff --git a/drivers/net/phy/dp83869.c b/drivers/net/phy/dp83869.c
index b6b38caf9c0e..96e5b8b03083 100644
--- a/drivers/net/phy/dp83869.c
+++ b/drivers/net/phy/dp83869.c
@@ -31,6 +31,7 @@
#define DP83869_RGMIICTL 0x0032
#define DP83869_STRAP_STS1 0x006e
#define DP83869_RGMIIDCTL 0x0086
+#define DP83869_ANA_PLL_PROG_PI 0x00c6
#define DP83869_RXFCFG 0x0134
#define DP83869_RXFPMD1 0x0136
#define DP83869_RXFPMD2 0x0137
@@ -827,12 +828,22 @@ static int dp83869_config_init(struct phy_device *phydev)
dp83869_config_port_mirroring(phydev);
/* Clock output selection if muxing property is set */
- if (dp83869->clk_output_sel != DP83869_CLK_O_SEL_REF_CLK)
+ if (dp83869->clk_output_sel != DP83869_CLK_O_SEL_REF_CLK) {
+ /*
+ * Table 7-121 in datasheet says we have to set register 0xc6
+ * to value 0x10 before CLK_O_SEL can be modified.
+ */
+ ret = phy_write_mmd(phydev, DP83869_DEVADDR,
+ DP83869_ANA_PLL_PROG_PI, 0x10);
+ if (ret)
+ return ret;
+
ret = phy_modify_mmd(phydev,
DP83869_DEVADDR, DP83869_IO_MUX_CFG,
DP83869_IO_MUX_CFG_CLK_O_SEL_MASK,
dp83869->clk_output_sel <<
DP83869_IO_MUX_CFG_CLK_O_SEL_SHIFT);
+ }
if (phy_interface_is_rgmii(phydev)) {
ret = phy_write_mmd(phydev, DP83869_DEVADDR, DP83869_RGMIIDCTL,
diff --git a/drivers/net/phy/phy_device.c b/drivers/net/phy/phy_device.c
index 7f995d0e51f7..eb478e4961cb 100644
--- a/drivers/net/phy/phy_device.c
+++ b/drivers/net/phy/phy_device.c
@@ -873,8 +873,8 @@ static int get_phy_c45_ids(struct mii_bus *bus, int addr,
/* returning -ENODEV doesn't stop bus
* scanning
*/
- return (phy_reg == -EIO ||
- phy_reg == -ENODEV) ? -ENODEV : -EIO;
+ return (ret == -EIO ||
+ ret == -ENODEV) ? -ENODEV : -EIO;
if (!ret)
continue;
diff --git a/drivers/net/phy/qcom/at803x.c b/drivers/net/phy/qcom/at803x.c
index ac909ad8a87b..11b7540b6958 100644
--- a/drivers/net/phy/qcom/at803x.c
+++ b/drivers/net/phy/qcom/at803x.c
@@ -474,7 +474,7 @@ static int at803x_config_init(struct phy_device *phydev)
* behaviour but we still need to accommodate it. XNP is only needed
* for 10Gbps support, so disable XNP.
*/
- return phy_modify(phydev, MII_ADVERTISE, MDIO_AN_CTRL1_XNP, 0);
+ return phy_modify(phydev, MII_ADVERTISE, ADVERTISE_XNP, 0);
}
static void at803x_link_change_notify(struct phy_device *phydev)
diff --git a/drivers/net/ppp/ppp_generic.c b/drivers/net/ppp/ppp_generic.c
index afc1566488b3..c70994c6a265 100644
--- a/drivers/net/ppp/ppp_generic.c
+++ b/drivers/net/ppp/ppp_generic.c
@@ -1064,6 +1064,9 @@ static int ppp_unattached_ioctl(struct net *net, struct ppp_file *pf,
struct ppp_net *pn;
int __user *p = (int __user *)arg;
+ if (!ns_capable(net->user_ns, CAP_NET_ADMIN))
+ return -EPERM;
+
switch (cmd) {
case PPPIOCNEWUNIT:
/* Create a new ppp unit */
@@ -2242,7 +2245,7 @@ ppp_do_recv(struct ppp *ppp, struct sk_buff *skb, struct channel *pch)
*/
static void __ppp_decompress_proto(struct sk_buff *skb)
{
- if (skb->data[0] & 0x01)
+ if (ppp_skb_is_compressed_proto(skb))
*(u8 *)skb_push(skb, 1) = 0x00;
}
diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c
index 2ea4f4890d23..937cf9b17f9a 100644
--- a/drivers/net/ppp/pppoe.c
+++ b/drivers/net/ppp/pppoe.c
@@ -425,7 +425,7 @@ static int pppoe_rcv(struct sk_buff *skb, struct net_device *dev,
if (skb_mac_header_len(skb) < ETH_HLEN)
goto drop;
- if (!pskb_may_pull(skb, sizeof(struct pppoe_hdr)))
+ if (!pskb_may_pull(skb, PPPOE_SES_HLEN))
goto drop;
ph = pppoe_hdr(skb);
@@ -435,6 +435,12 @@ static int pppoe_rcv(struct sk_buff *skb, struct net_device *dev,
if (skb->len < len)
goto drop;
+ /* skb->data points to the PPP protocol header after skb_pull_rcsum.
+ * Drop PFC frames.
+ */
+ if (ppp_skb_is_compressed_proto(skb))
+ goto drop;
+
if (pskb_trim_rcsum(skb, len))
goto drop;
diff --git a/drivers/net/slip/slhc.c b/drivers/net/slip/slhc.c
index ee9fd3a94b96..daf086c28342 100644
--- a/drivers/net/slip/slhc.c
+++ b/drivers/net/slip/slhc.c
@@ -80,9 +80,9 @@
#include <linux/unaligned.h>
static unsigned char *encode(unsigned char *cp, unsigned short n);
-static long decode(unsigned char **cpp);
+static long decode(unsigned char **cpp, const unsigned char *end);
static unsigned char * put16(unsigned char *cp, unsigned short x);
-static unsigned short pull16(unsigned char **cpp);
+static long pull16(unsigned char **cpp, const unsigned char *end);
/* Allocate compression data structure
* slots must be in range 0 to 255 (zero meaning no compression)
@@ -190,30 +190,34 @@ encode(unsigned char *cp, unsigned short n)
return cp;
}
-/* Pull a 16-bit integer in host order from buffer in network byte order */
-static unsigned short
-pull16(unsigned char **cpp)
+/* Pull a 16-bit integer in host order from buffer in network byte order.
+ * Returns -1 if the buffer is exhausted, otherwise the 16-bit value.
+ */
+static long
+pull16(unsigned char **cpp, const unsigned char *end)
{
- short rval;
+ long rval;
+ if (*cpp + 2 > end)
+ return -1;
rval = *(*cpp)++;
rval <<= 8;
rval |= *(*cpp)++;
return rval;
}
-/* Decode a number */
+/* Decode a number. Returns -1 if the buffer is exhausted. */
static long
-decode(unsigned char **cpp)
+decode(unsigned char **cpp, const unsigned char *end)
{
int x;
+ if (*cpp >= end)
+ return -1;
x = *(*cpp)++;
- if(x == 0){
- return pull16(cpp) & 0xffff; /* pull16 returns -1 on error */
- } else {
- return x & 0xff; /* -1 if PULLCHAR returned error */
- }
+ if (x == 0)
+ return pull16(cpp, end);
+ return x & 0xff;
}
/*
@@ -499,6 +503,7 @@ slhc_uncompress(struct slcompress *comp, unsigned char *icp, int isize)
struct cstate *cs;
int len, hdrlen;
unsigned char *cp = icp;
+ const unsigned char *end = icp + isize;
/* We've got a compressed packet; read the change byte */
comp->sls_i_compressed++;
@@ -506,6 +511,8 @@ slhc_uncompress(struct slcompress *comp, unsigned char *icp, int isize)
comp->sls_i_error++;
return 0;
}
+ if (!comp->rstate)
+ goto bad;
changes = *cp++;
if(changes & NEW_C){
/* Make sure the state index is in range, then grab the state.
@@ -534,6 +541,8 @@ slhc_uncompress(struct slcompress *comp, unsigned char *icp, int isize)
thp = &cs->cs_tcp;
ip = &cs->cs_ip;
+ if (cp + 2 > end)
+ goto bad;
thp->check = *(__sum16 *)cp;
cp += 2;
@@ -564,26 +573,26 @@ slhc_uncompress(struct slcompress *comp, unsigned char *icp, int isize)
default:
if(changes & NEW_U){
thp->urg = 1;
- if((x = decode(&cp)) == -1) {
+ if((x = decode(&cp, end)) == -1) {
goto bad;
}
thp->urg_ptr = htons(x);
} else
thp->urg = 0;
if(changes & NEW_W){
- if((x = decode(&cp)) == -1) {
+ if((x = decode(&cp, end)) == -1) {
goto bad;
}
thp->window = htons( ntohs(thp->window) + x);
}
if(changes & NEW_A){
- if((x = decode(&cp)) == -1) {
+ if((x = decode(&cp, end)) == -1) {
goto bad;
}
thp->ack_seq = htonl( ntohl(thp->ack_seq) + x);
}
if(changes & NEW_S){
- if((x = decode(&cp)) == -1) {
+ if((x = decode(&cp, end)) == -1) {
goto bad;
}
thp->seq = htonl( ntohl(thp->seq) + x);
@@ -591,7 +600,7 @@ slhc_uncompress(struct slcompress *comp, unsigned char *icp, int isize)
break;
}
if(changes & NEW_I){
- if((x = decode(&cp)) == -1) {
+ if((x = decode(&cp, end)) == -1) {
goto bad;
}
ip->id = htons (ntohs (ip->id) + x);
@@ -649,6 +658,10 @@ slhc_remember(struct slcompress *comp, unsigned char *icp, int isize)
struct cstate *cs;
unsigned int ihl;
+ if (!comp->rstate) {
+ comp->sls_i_error++;
+ return slhc_toss(comp);
+ }
/* The packet is shorter than a legal IP header.
* Also make sure isize is positive.
*/
diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
index 6ce25673e4cc..1c36816405f1 100644
--- a/drivers/net/usb/r8152.c
+++ b/drivers/net/usb/r8152.c
@@ -3894,7 +3894,7 @@ static void r8156_ups_en(struct r8152 *tp, bool enable)
case RTL_VER_15:
ocp_data = ocp_read_word(tp, MCU_TYPE_USB, USB_UPHY_XTAL);
ocp_data &= ~OOBS_POLLING;
- ocp_write_byte(tp, MCU_TYPE_USB, USB_UPHY_XTAL, ocp_data);
+ ocp_write_word(tp, MCU_TYPE_USB, USB_UPHY_XTAL, ocp_data);
break;
default:
break;
diff --git a/drivers/net/usb/rtl8150.c b/drivers/net/usb/rtl8150.c
index e40b0669d9f4..647f28b367b9 100644
--- a/drivers/net/usb/rtl8150.c
+++ b/drivers/net/usb/rtl8150.c
@@ -685,6 +685,7 @@ static netdev_tx_t rtl8150_start_xmit(struct sk_buff *skb,
struct net_device *netdev)
{
rtl8150_t *dev = netdev_priv(netdev);
+ unsigned int skb_len;
int count, res;
/* pad the frame and ensure terminating USB packet, datasheet 9.2.3 */
@@ -696,6 +697,8 @@ static netdev_tx_t rtl8150_start_xmit(struct sk_buff *skb,
return NETDEV_TX_OK;
}
+ skb_len = skb->len;
+
netif_stop_queue(netdev);
dev->tx_skb = skb;
usb_fill_bulk_urb(dev->tx_urb, dev->udev, usb_sndbulkpipe(dev->udev, 2),
@@ -709,9 +712,16 @@ static netdev_tx_t rtl8150_start_xmit(struct sk_buff *skb,
netdev->stats.tx_errors++;
netif_start_queue(netdev);
}
+ /*
+ * The URB was not submitted, so write_bulk_callback() will
+ * never run to free dev->tx_skb. Drop the skb here and
+ * clear tx_skb to avoid leaving a stale pointer.
+ */
+ dev->tx_skb = NULL;
+ dev_kfree_skb_any(skb);
} else {
netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += skb->len;
+ netdev->stats.tx_bytes += skb_len;
netif_trans_update(netdev);
}
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 5c83983f0eb3..324802cef40b 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -365,24 +365,7 @@ struct receive_queue {
bool do_dma;
};
-/* This structure can contain rss message with maximum settings for indirection table and keysize
- * Note, that default structure that describes RSS configuration virtio_net_rss_config
- * contains same info but can't handle table values.
- * In any case, structure would be passed to virtio hw through sg_buf split by parts
- * because table sizes may be differ according to the device configuration.
- */
#define VIRTIO_NET_RSS_MAX_KEY_SIZE 40
-struct virtio_net_ctrl_rss {
- u32 hash_types;
- u16 indirection_table_mask;
- u16 unclassified_queue;
- u16 hash_cfg_reserved; /* for HASH_CONFIG (see virtio_net_hash_config for details) */
- u16 max_tx_vq;
- u8 hash_key_length;
- u8 key[VIRTIO_NET_RSS_MAX_KEY_SIZE];
-
- u16 *indirection_table;
-};
/* Control VQ buffers: protected by the rtnl lock */
struct control_buf {
@@ -426,7 +409,9 @@ struct virtnet_info {
u16 rss_indir_table_size;
u32 rss_hash_types_supported;
u32 rss_hash_types_saved;
- struct virtio_net_ctrl_rss rss;
+ struct virtio_net_rss_config_hdr *rss_hdr;
+ struct virtio_net_rss_config_trailer rss_trailer;
+ u8 rss_hash_key_data[VIRTIO_NET_RSS_MAX_KEY_SIZE];
/* Has control virtqueue */
bool has_cvq;
@@ -520,23 +505,16 @@ static struct sk_buff *virtnet_skb_append_frag(struct sk_buff *head_skb,
struct page *page, void *buf,
int len, int truesize);
-static int rss_indirection_table_alloc(struct virtio_net_ctrl_rss *rss, u16 indir_table_size)
+static size_t virtnet_rss_hdr_size(const struct virtnet_info *vi)
{
- if (!indir_table_size) {
- rss->indirection_table = NULL;
- return 0;
- }
+ u16 indir_table_size = vi->has_rss ? vi->rss_indir_table_size : 1;
- rss->indirection_table = kmalloc_array(indir_table_size, sizeof(u16), GFP_KERNEL);
- if (!rss->indirection_table)
- return -ENOMEM;
-
- return 0;
+ return struct_size(vi->rss_hdr, indirection_table, indir_table_size);
}
-static void rss_indirection_table_free(struct virtio_net_ctrl_rss *rss)
+static size_t virtnet_rss_trailer_size(const struct virtnet_info *vi)
{
- kfree(rss->indirection_table);
+ return struct_size(&vi->rss_trailer, hash_key_data, vi->rss_key_size);
}
static bool is_xdp_frame(void *ptr)
@@ -3477,15 +3455,16 @@ static void virtnet_rss_update_by_qpairs(struct virtnet_info *vi, u16 queue_pair
for (; i < vi->rss_indir_table_size; ++i) {
indir_val = ethtool_rxfh_indir_default(i, queue_pairs);
- vi->rss.indirection_table[i] = indir_val;
+ vi->rss_hdr->indirection_table[i] = cpu_to_le16(indir_val);
}
- vi->rss.max_tx_vq = queue_pairs;
+ vi->rss_trailer.max_tx_vq = cpu_to_le16(queue_pairs);
}
static int virtnet_set_queues(struct virtnet_info *vi, u16 queue_pairs)
{
struct virtio_net_ctrl_mq *mq __free(kfree) = NULL;
- struct virtio_net_ctrl_rss old_rss;
+ struct virtio_net_rss_config_hdr *old_rss_hdr;
+ struct virtio_net_rss_config_trailer old_rss_trailer;
struct net_device *dev = vi->dev;
struct scatterlist sg;
@@ -3500,24 +3479,28 @@ static int virtnet_set_queues(struct virtnet_info *vi, u16 queue_pairs)
* update (VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET below) and return directly.
*/
if (vi->has_rss && !netif_is_rxfh_configured(dev)) {
- memcpy(&old_rss, &vi->rss, sizeof(old_rss));
- if (rss_indirection_table_alloc(&vi->rss, vi->rss_indir_table_size)) {
- vi->rss.indirection_table = old_rss.indirection_table;
+ old_rss_hdr = vi->rss_hdr;
+ old_rss_trailer = vi->rss_trailer;
+ vi->rss_hdr = kzalloc(virtnet_rss_hdr_size(vi), GFP_KERNEL);
+ if (!vi->rss_hdr) {
+ vi->rss_hdr = old_rss_hdr;
return -ENOMEM;
}
+ *vi->rss_hdr = *old_rss_hdr;
virtnet_rss_update_by_qpairs(vi, queue_pairs);
if (!virtnet_commit_rss_command(vi)) {
/* restore ctrl_rss if commit_rss_command failed */
- rss_indirection_table_free(&vi->rss);
- memcpy(&vi->rss, &old_rss, sizeof(old_rss));
+ kfree(vi->rss_hdr);
+ vi->rss_hdr = old_rss_hdr;
+ vi->rss_trailer = old_rss_trailer;
dev_warn(&dev->dev, "Fail to set num of queue pairs to %d, because committing RSS failed\n",
queue_pairs);
return -EINVAL;
}
- rss_indirection_table_free(&old_rss);
+ kfree(old_rss_hdr);
goto succ;
}
@@ -3534,6 +3517,12 @@ static int virtnet_set_queues(struct virtnet_info *vi, u16 queue_pairs)
queue_pairs);
return -EINVAL;
}
+
+ /* Keep max_tx_vq in sync so that a later RSS command does not
+ * revert queue_pairs to a stale value.
+ */
+ if (vi->has_rss)
+ vi->rss_trailer.max_tx_vq = cpu_to_le16(queue_pairs);
succ:
vi->curr_queue_pairs = queue_pairs;
/* virtnet_open() will refill when device is going to up. */
@@ -3960,28 +3949,12 @@ static int virtnet_set_ringparam(struct net_device *dev,
static bool virtnet_commit_rss_command(struct virtnet_info *vi)
{
struct net_device *dev = vi->dev;
- struct scatterlist sgs[4];
- unsigned int sg_buf_size;
+ struct scatterlist sgs[2];
/* prepare sgs */
- sg_init_table(sgs, 4);
-
- sg_buf_size = offsetof(struct virtio_net_ctrl_rss, hash_cfg_reserved);
- sg_set_buf(&sgs[0], &vi->rss, sg_buf_size);
-
- if (vi->has_rss) {
- sg_buf_size = sizeof(uint16_t) * vi->rss_indir_table_size;
- sg_set_buf(&sgs[1], vi->rss.indirection_table, sg_buf_size);
- } else {
- sg_set_buf(&sgs[1], &vi->rss.hash_cfg_reserved, sizeof(uint16_t));
- }
-
- sg_buf_size = offsetof(struct virtio_net_ctrl_rss, key)
- - offsetof(struct virtio_net_ctrl_rss, max_tx_vq);
- sg_set_buf(&sgs[2], &vi->rss.max_tx_vq, sg_buf_size);
-
- sg_buf_size = vi->rss_key_size;
- sg_set_buf(&sgs[3], vi->rss.key, sg_buf_size);
+ sg_init_table(sgs, 2);
+ sg_set_buf(&sgs[0], vi->rss_hdr, virtnet_rss_hdr_size(vi));
+ sg_set_buf(&sgs[1], &vi->rss_trailer, virtnet_rss_trailer_size(vi));
if (!virtnet_send_command(vi, VIRTIO_NET_CTRL_MQ,
vi->has_rss ? VIRTIO_NET_CTRL_MQ_RSS_CONFIG
@@ -3998,17 +3971,17 @@ static bool virtnet_commit_rss_command(struct virtnet_info *vi)
static void virtnet_init_default_rss(struct virtnet_info *vi)
{
- vi->rss.hash_types = vi->rss_hash_types_supported;
+ vi->rss_hdr->hash_types = cpu_to_le32(vi->rss_hash_types_supported);
vi->rss_hash_types_saved = vi->rss_hash_types_supported;
- vi->rss.indirection_table_mask = vi->rss_indir_table_size
- ? vi->rss_indir_table_size - 1 : 0;
- vi->rss.unclassified_queue = 0;
+ vi->rss_hdr->indirection_table_mask = vi->rss_indir_table_size
+ ? cpu_to_le16(vi->rss_indir_table_size - 1) : 0;
+ vi->rss_hdr->unclassified_queue = 0;
virtnet_rss_update_by_qpairs(vi, vi->curr_queue_pairs);
- vi->rss.hash_key_length = vi->rss_key_size;
+ vi->rss_trailer.hash_key_length = vi->rss_key_size;
- netdev_rss_key_fill(vi->rss.key, vi->rss_key_size);
+ netdev_rss_key_fill(vi->rss_hash_key_data, vi->rss_key_size);
}
static void virtnet_get_hashflow(const struct virtnet_info *vi, struct ethtool_rxnfc *info)
@@ -4119,7 +4092,7 @@ static bool virtnet_set_hashflow(struct virtnet_info *vi, struct ethtool_rxnfc *
if (new_hashtypes != vi->rss_hash_types_saved) {
vi->rss_hash_types_saved = new_hashtypes;
- vi->rss.hash_types = vi->rss_hash_types_saved;
+ vi->rss_hdr->hash_types = cpu_to_le32(vi->rss_hash_types_saved);
if (vi->dev->features & NETIF_F_RXHASH)
return virtnet_commit_rss_command(vi);
}
@@ -5291,11 +5264,11 @@ static int virtnet_get_rxfh(struct net_device *dev,
if (rxfh->indir) {
for (i = 0; i < vi->rss_indir_table_size; ++i)
- rxfh->indir[i] = vi->rss.indirection_table[i];
+ rxfh->indir[i] = le16_to_cpu(vi->rss_hdr->indirection_table[i]);
}
if (rxfh->key)
- memcpy(rxfh->key, vi->rss.key, vi->rss_key_size);
+ memcpy(rxfh->key, vi->rss_hash_key_data, vi->rss_key_size);
rxfh->hfunc = ETH_RSS_HASH_TOP;
@@ -5319,7 +5292,7 @@ static int virtnet_set_rxfh(struct net_device *dev,
return -EOPNOTSUPP;
for (i = 0; i < vi->rss_indir_table_size; ++i)
- vi->rss.indirection_table[i] = rxfh->indir[i];
+ vi->rss_hdr->indirection_table[i] = cpu_to_le16(rxfh->indir[i]);
update = true;
}
@@ -5331,7 +5304,7 @@ static int virtnet_set_rxfh(struct net_device *dev,
if (!vi->has_rss && !vi->has_rss_hash_report)
return -EOPNOTSUPP;
- memcpy(vi->rss.key, rxfh->key, vi->rss_key_size);
+ memcpy(vi->rss_hash_key_data, rxfh->key, vi->rss_key_size);
update = true;
}
@@ -5945,9 +5918,9 @@ static int virtnet_set_features(struct net_device *dev,
if ((dev->features ^ features) & NETIF_F_RXHASH) {
if (features & NETIF_F_RXHASH)
- vi->rss.hash_types = vi->rss_hash_types_saved;
+ vi->rss_hdr->hash_types = cpu_to_le32(vi->rss_hash_types_saved);
else
- vi->rss.hash_types = VIRTIO_NET_HASH_REPORT_NONE;
+ vi->rss_hdr->hash_types = cpu_to_le32(VIRTIO_NET_HASH_REPORT_NONE);
if (!virtnet_commit_rss_command(vi))
return -EINVAL;
@@ -6619,9 +6592,11 @@ static int virtnet_probe(struct virtio_device *vdev)
virtio_cread16(vdev, offsetof(struct virtio_net_config,
rss_max_indirection_table_length));
}
- err = rss_indirection_table_alloc(&vi->rss, vi->rss_indir_table_size);
- if (err)
+ vi->rss_hdr = kzalloc(virtnet_rss_hdr_size(vi), GFP_KERNEL);
+ if (!vi->rss_hdr) {
+ err = -ENOMEM;
goto free;
+ }
if (vi->has_rss || vi->has_rss_hash_report) {
vi->rss_key_size =
@@ -6900,7 +6875,7 @@ static void virtnet_remove(struct virtio_device *vdev)
remove_vq_common(vi);
- rss_indirection_table_free(&vi->rss);
+ kfree(vi->rss_hdr);
free_netdev(vi->dev);
}
diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c
index b62462d8eff2..93ecd9577f4d 100644
--- a/drivers/net/vrf.c
+++ b/drivers/net/vrf.c
@@ -1104,6 +1104,7 @@ static int do_vrf_add_slave(struct net_device *dev, struct net_device *port_dev,
err:
port_dev->priv_flags &= ~IFF_L3MDEV_SLAVE;
+ synchronize_net();
return ret;
}
@@ -1123,10 +1124,16 @@ static int vrf_add_slave(struct net_device *dev, struct net_device *port_dev,
}
/* inverse of do_vrf_add_slave */
-static int do_vrf_del_slave(struct net_device *dev, struct net_device *port_dev)
+static int do_vrf_del_slave(struct net_device *dev, struct net_device *port_dev,
+ bool needs_sync)
{
netdev_upper_dev_unlink(port_dev, dev);
port_dev->priv_flags &= ~IFF_L3MDEV_SLAVE;
+ /* Make sure that concurrent RCU readers that identified the device
+ * as a VRF port see a VRF master or no master at all.
+ */
+ if (needs_sync)
+ synchronize_net();
cycle_netdev(port_dev, NULL);
@@ -1135,7 +1142,7 @@ static int do_vrf_del_slave(struct net_device *dev, struct net_device *port_dev)
static int vrf_del_slave(struct net_device *dev, struct net_device *port_dev)
{
- return do_vrf_del_slave(dev, port_dev);
+ return do_vrf_del_slave(dev, port_dev, true);
}
static void vrf_dev_uninit(struct net_device *dev)
@@ -1691,7 +1698,7 @@ static void vrf_dellink(struct net_device *dev, struct list_head *head)
struct list_head *iter;
netdev_for_each_lower_dev(dev, port_dev, iter)
- vrf_del_slave(dev, port_dev);
+ do_vrf_del_slave(dev, port_dev, false);
vrf_map_unregister_dev(dev);
@@ -1822,7 +1829,7 @@ static int vrf_device_event(struct notifier_block *unused,
goto out;
vrf_dev = netdev_master_upper_dev_get(dev);
- vrf_del_slave(vrf_dev, dev);
+ do_vrf_del_slave(vrf_dev, dev, false);
}
out:
return NOTIFY_DONE;
diff --git a/drivers/net/wireless/ath/ath10k/wmi-tlv.c b/drivers/net/wireless/ath/ath10k/wmi-tlv.c
index 16d07d619b4d..ba1294c8ee39 100644
--- a/drivers/net/wireless/ath/ath10k/wmi-tlv.c
+++ b/drivers/net/wireless/ath/ath10k/wmi-tlv.c
@@ -3,7 +3,7 @@
* Copyright (c) 2005-2011 Atheros Communications Inc.
* Copyright (c) 2011-2017 Qualcomm Atheros, Inc.
* Copyright (c) 2018-2019, The Linux Foundation. All rights reserved.
- * Copyright (c) 2024 Qualcomm Innovation Center, Inc. All rights reserved.
+ * Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries.
*/
#include "core.h"
#include "debug.h"
@@ -14,6 +14,7 @@
#include "wmi-tlv.h"
#include "p2p.h"
#include "testmode.h"
+#include "txrx.h"
#include <linux/bitfield.h>
/***************/
@@ -224,8 +225,9 @@ static int ath10k_wmi_tlv_parse_peer_stats_info(struct ath10k *ar, u16 tag, u16
const void *ptr, void *data)
{
const struct wmi_tlv_peer_stats_info *stat = ptr;
- struct ieee80211_sta *sta;
+ u32 vdev_id = *(u32 *)data;
struct ath10k_sta *arsta;
+ struct ath10k_peer *peer;
if (tag != WMI_TLV_TAG_STRUCT_PEER_STATS_INFO)
return -EPROTO;
@@ -241,20 +243,20 @@ static int ath10k_wmi_tlv_parse_peer_stats_info(struct ath10k *ar, u16 tag, u16
__le32_to_cpu(stat->last_tx_rate_code),
__le32_to_cpu(stat->last_tx_bitrate_kbps));
- rcu_read_lock();
- sta = ieee80211_find_sta_by_ifaddr(ar->hw, stat->peer_macaddr.addr, NULL);
- if (!sta) {
- rcu_read_unlock();
- ath10k_warn(ar, "not found station for peer stats\n");
+ guard(spinlock_bh)(&ar->data_lock);
+
+ peer = ath10k_peer_find(ar, vdev_id, stat->peer_macaddr.addr);
+ if (!peer || !peer->sta) {
+ ath10k_warn(ar, "not found %s with vdev id %u mac addr %pM for peer stats\n",
+ peer ? "sta" : "peer", vdev_id, stat->peer_macaddr.addr);
return -EINVAL;
}
- arsta = (struct ath10k_sta *)sta->drv_priv;
+ arsta = (struct ath10k_sta *)peer->sta->drv_priv;
arsta->rx_rate_code = __le32_to_cpu(stat->last_rx_rate_code);
arsta->rx_bitrate_kbps = __le32_to_cpu(stat->last_rx_bitrate_kbps);
arsta->tx_rate_code = __le32_to_cpu(stat->last_tx_rate_code);
arsta->tx_bitrate_kbps = __le32_to_cpu(stat->last_tx_bitrate_kbps);
- rcu_read_unlock();
return 0;
}
@@ -266,6 +268,7 @@ static int ath10k_wmi_tlv_op_pull_peer_stats_info(struct ath10k *ar,
const struct wmi_tlv_peer_stats_info_ev *ev;
const void *data;
u32 num_peer_stats;
+ u32 vdev_id;
int ret;
tb = ath10k_wmi_tlv_parse_alloc(ar, skb->data, skb->len, GFP_ATOMIC);
@@ -284,15 +287,16 @@ static int ath10k_wmi_tlv_op_pull_peer_stats_info(struct ath10k *ar,
}
num_peer_stats = __le32_to_cpu(ev->num_peers);
+ vdev_id = __le32_to_cpu(ev->vdev_id);
ath10k_dbg(ar, ATH10K_DBG_WMI,
"wmi tlv peer stats info update peer vdev id %d peers %i more data %d\n",
- __le32_to_cpu(ev->vdev_id),
+ vdev_id,
num_peer_stats,
__le32_to_cpu(ev->more_data));
ret = ath10k_wmi_tlv_iter(ar, data, ath10k_wmi_tlv_len(data),
- ath10k_wmi_tlv_parse_peer_stats_info, NULL);
+ ath10k_wmi_tlv_parse_peer_stats_info, &vdev_id);
if (ret)
ath10k_warn(ar, "failed to parse stats info tlv: %d\n", ret);
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
index 2ef92ef25517..4dbb1898f306 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
@@ -1006,18 +1006,33 @@ static int brcmf_chip_recognition(struct brcmf_chip_priv *ci)
core = brcmf_chip_add_core(ci, BCMA_CORE_CHIPCOMMON,
SI_ENUM_BASE_DEFAULT, 0);
+ if (IS_ERR(core))
+ return PTR_ERR(core);
+
brcmf_chip_sb_corerev(ci, core);
core = brcmf_chip_add_core(ci, BCMA_CORE_SDIO_DEV,
BCM4329_CORE_BUS_BASE, 0);
+ if (IS_ERR(core))
+ return PTR_ERR(core);
+
brcmf_chip_sb_corerev(ci, core);
core = brcmf_chip_add_core(ci, BCMA_CORE_INTERNAL_MEM,
BCM4329_CORE_SOCRAM_BASE, 0);
+ if (IS_ERR(core))
+ return PTR_ERR(core);
+
brcmf_chip_sb_corerev(ci, core);
core = brcmf_chip_add_core(ci, BCMA_CORE_ARM_CM3,
BCM4329_CORE_ARM_BASE, 0);
+ if (IS_ERR(core))
+ return PTR_ERR(core);
+
brcmf_chip_sb_corerev(ci, core);
core = brcmf_chip_add_core(ci, BCMA_CORE_80211, 0x18001000, 0);
+ if (IS_ERR(core))
+ return PTR_ERR(core);
+
brcmf_chip_sb_corerev(ci, core);
} else if (socitype == SOCI_AI) {
ci->iscoreup = brcmf_chip_ai_iscoreup;
diff --git a/drivers/net/wireless/marvell/mwifiex/11n_aggr.c b/drivers/net/wireless/marvell/mwifiex/11n_aggr.c
index 34b4b34276d6..042b1fe5f0d6 100644
--- a/drivers/net/wireless/marvell/mwifiex/11n_aggr.c
+++ b/drivers/net/wireless/marvell/mwifiex/11n_aggr.c
@@ -203,6 +203,7 @@ mwifiex_11n_aggregate_pkt(struct mwifiex_private *priv,
if (!mwifiex_is_ralist_valid(priv, pra_list, ptrindex)) {
spin_unlock_bh(&priv->wmm.ra_list_spinlock);
+ mwifiex_write_data_complete(adapter, skb_aggr, 1, -1);
return -1;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
index 7ba789834e8d..1e473f490b4b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
@@ -1174,21 +1174,6 @@ void mt7615_mac_set_rates(struct mt7615_phy *phy, struct mt7615_sta *sta,
}
EXPORT_SYMBOL_GPL(mt7615_mac_set_rates);
-void mt7615_mac_enable_rtscts(struct mt7615_dev *dev,
- struct ieee80211_vif *vif, bool enable)
-{
- struct mt7615_vif *mvif = (struct mt7615_vif *)vif->drv_priv;
- u32 addr;
-
- addr = mt7615_mac_wtbl_addr(dev, mvif->sta.wcid.idx) + 3 * 4;
-
- if (enable)
- mt76_set(dev, addr, MT_WTBL_W3_RTS);
- else
- mt76_clear(dev, addr, MT_WTBL_W3_RTS);
-}
-EXPORT_SYMBOL_GPL(mt7615_mac_enable_rtscts);
-
static int
mt7615_mac_wtbl_update_key(struct mt7615_dev *dev, struct mt76_wcid *wcid,
struct ieee80211_key_conf *key,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/main.c b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
index 4f0c840ef93d..f747285a1a5d 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
@@ -584,9 +584,6 @@ static void mt7615_bss_info_changed(struct ieee80211_hw *hw,
}
}
- if (changed & BSS_CHANGED_ERP_CTS_PROT)
- mt7615_mac_enable_rtscts(dev, vif, info->use_cts_prot);
-
if (changed & BSS_CHANGED_BEACON_ENABLED && info->enable_beacon) {
mt7615_mcu_add_bss_info(phy, vif, NULL, true);
mt7615_mcu_sta_add(phy, vif, NULL, true);
@@ -599,6 +596,10 @@ static void mt7615_bss_info_changed(struct ieee80211_hw *hw,
BSS_CHANGED_BEACON_ENABLED))
mt7615_mcu_add_beacon(dev, hw, vif, info->enable_beacon);
+ if (changed & BSS_CHANGED_HT || changed & BSS_CHANGED_ERP_CTS_PROT)
+ mt7615_mcu_set_protection(phy, vif, info->ht_operation_mode,
+ info->use_cts_prot);
+
if (changed & BSS_CHANGED_PS)
mt76_connac_mcu_set_vif_ps(&dev->mt76, vif);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
index 40e15a0ba95a..ac831422d4c7 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
@@ -2564,3 +2564,50 @@ int mt7615_mcu_set_roc(struct mt7615_phy *phy, struct ieee80211_vif *vif,
return mt76_mcu_send_msg(&dev->mt76, MCU_CE_CMD(SET_ROC),
&req, sizeof(req), false);
}
+
+int mt7615_mcu_set_protection(struct mt7615_phy *phy, struct ieee80211_vif *vif,
+ u8 ht_mode, bool use_cts_prot)
+{
+ struct mt7615_dev *dev = phy->dev;
+ struct {
+ u8 prot_idx;
+ u8 band;
+ u8 rsv[2];
+
+ bool long_nav;
+ bool prot_mm;
+ bool prot_gf;
+ bool prot_bw40;
+ bool prot_rifs;
+ bool prot_bw80;
+ bool prot_bw160;
+ u8 prot_erp_mask;
+ } __packed req = {
+ .prot_idx = 0x2,
+ .band = phy != &dev->phy,
+ };
+
+ switch (ht_mode & IEEE80211_HT_OP_MODE_PROTECTION) {
+ case IEEE80211_HT_OP_MODE_PROTECTION_NONMEMBER:
+ case IEEE80211_HT_OP_MODE_PROTECTION_NONHT_MIXED:
+ req.prot_mm = true;
+ req.prot_gf = true;
+ fallthrough;
+ case IEEE80211_HT_OP_MODE_PROTECTION_20MHZ:
+ req.prot_bw40 = true;
+ break;
+ }
+
+ if (ht_mode & IEEE80211_HT_OP_MODE_NON_GF_STA_PRSNT)
+ req.prot_gf = true;
+
+ if (use_cts_prot) {
+ struct mt7615_vif *mvif = (struct mt7615_vif *)vif->drv_priv;
+ u8 i = mvif->mt76.omac_idx > HW_BSSID_MAX ? HW_BSSID_0 : mvif->mt76.omac_idx;
+
+ req.prot_erp_mask = BIT(i);
+ }
+
+ return mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD(PROTECT_CTRL), &req,
+ sizeof(req), true);
+}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h b/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
index 530da48ce3ea..aac748fee257 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
@@ -466,8 +466,6 @@ void mt7615_mac_reset_counters(struct mt7615_phy *phy);
void mt7615_mac_cca_stats_reset(struct mt7615_phy *phy);
void mt7615_mac_set_scs(struct mt7615_phy *phy, bool enable);
void mt7615_mac_enable_nf(struct mt7615_dev *dev, bool ext_phy);
-void mt7615_mac_enable_rtscts(struct mt7615_dev *dev,
- struct ieee80211_vif *vif, bool enable);
void mt7615_mac_sta_poll(struct mt7615_dev *dev);
int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
struct sk_buff *skb, struct mt76_wcid *wcid,
@@ -522,7 +520,8 @@ int mt7615_mcu_set_sku_en(struct mt7615_phy *phy, bool enable);
int mt7615_mcu_apply_rx_dcoc(struct mt7615_phy *phy);
int mt7615_mcu_apply_tx_dpd(struct mt7615_phy *phy);
int mt7615_dfs_init_radar_detector(struct mt7615_phy *phy);
-
+int mt7615_mcu_set_protection(struct mt7615_phy *phy, struct ieee80211_vif *vif,
+ u8 ht_mode, bool use_cts_prot);
int mt7615_mcu_set_roc(struct mt7615_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_channel *chan, int duration);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/regs.h b/drivers/net/wireless/mediatek/mt76/mt7615/regs.h
index 806b3887c541..9e6d55c91b26 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/regs.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/regs.h
@@ -455,8 +455,6 @@ enum mt7615_reg_base {
#define MT_WTBL_RIUCR3_RATE6 GENMASK(19, 8)
#define MT_WTBL_RIUCR3_RATE7 GENMASK(31, 20)
-#define MT_WTBL_W3_RTS BIT(22)
-
#define MT_WTBL_W5_CHANGE_BW_RATE GENMASK(7, 5)
#define MT_WTBL_W5_SHORT_GI_20 BIT(8)
#define MT_WTBL_W5_SHORT_GI_40 BIT(9)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
index 268f414f0a02..05f2ff8e012b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c
@@ -1135,8 +1135,10 @@ void mt76_connac2_tx_check_aggr(struct ieee80211_sta *sta, __le32 *txwi)
return;
wcid = (struct mt76_wcid *)sta->drv_priv;
- if (!test_and_set_bit(tid, &wcid->ampdu_state))
- ieee80211_start_tx_ba_session(sta, tid, 0);
+ if (!test_and_set_bit(tid, &wcid->ampdu_state)) {
+ if (ieee80211_start_tx_ba_session(sta, tid, 0))
+ clear_bit(tid, &wcid->ampdu_state);
+ }
}
EXPORT_SYMBOL_GPL(mt76_connac2_tx_check_aggr);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/init.c b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
index 83f062fb95d5..b2220ff7d675 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
@@ -1275,6 +1275,7 @@ int mt7915_register_device(struct mt7915_dev *dev)
void mt7915_unregister_device(struct mt7915_dev *dev)
{
+ cancel_work_sync(&dev->dump_work);
mt7915_unregister_ext_phy(dev);
mt7915_coredump_unregister(dev);
mt7915_unregister_thermal(&dev->phy);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
index 799e8d2cc7e6..ab31a350ca73 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
@@ -235,19 +235,6 @@ static void mt7915_mac_sta_poll(struct mt7915_dev *dev)
rcu_read_unlock();
}
-void mt7915_mac_enable_rtscts(struct mt7915_dev *dev,
- struct ieee80211_vif *vif, bool enable)
-{
- struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
- u32 addr;
-
- addr = mt7915_mac_wtbl_lmac_addr(dev, mvif->sta.wcid.idx, 5);
- if (enable)
- mt76_set(dev, addr, BIT(5));
- else
- mt76_clear(dev, addr, BIT(5));
-}
-
static void
mt7915_wed_check_ppe(struct mt7915_dev *dev, struct mt76_queue *q,
struct mt7915_sta *msta, struct sk_buff *skb,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/main.c b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
index 8c0d63cebf3e..f49dcdde3f2c 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
@@ -68,7 +68,7 @@ int mt7915_run(struct ieee80211_hw *hw)
if (ret)
goto out;
- ret = mt76_connac_mcu_set_rts_thresh(&dev->mt76, 0x92b,
+ ret = mt76_connac_mcu_set_rts_thresh(&dev->mt76, MT7915_RTS_LEN_THRES,
phy->mt76->band_idx);
if (ret)
goto out;
@@ -630,8 +630,9 @@ static void mt7915_bss_info_changed(struct ieee80211_hw *hw,
if (set_sta == 1)
mt7915_mcu_add_sta(dev, vif, NULL, CONN_STATE_PORT_SECURE, false);
- if (changed & BSS_CHANGED_ERP_CTS_PROT)
- mt7915_mac_enable_rtscts(dev, vif, info->use_cts_prot);
+ if (changed & BSS_CHANGED_HT || changed & BSS_CHANGED_ERP_CTS_PROT)
+ mt7915_mcu_set_protection(phy, vif, info->ht_operation_mode,
+ info->use_cts_prot);
if (changed & BSS_CHANGED_ERP_SLOT) {
int slottime = info->use_short_slot ? 9 : 20;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
index 7b481aea76b6..2489364110d6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
@@ -3799,6 +3799,68 @@ int mt7915_mcu_get_rx_rate(struct mt7915_phy *phy, struct ieee80211_vif *vif,
return ret;
}
+int mt7915_mcu_set_protection(struct mt7915_phy *phy, struct ieee80211_vif *vif,
+ u8 ht_mode, bool use_cts_prot)
+{
+ struct mt7915_dev *dev = phy->dev;
+ int len = sizeof(struct sta_req_hdr) + sizeof(struct bss_info_prot);
+ struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv;
+ struct bss_info_prot *prot;
+ struct sk_buff *skb;
+ struct tlv *tlv;
+ enum {
+ PROT_NONMEMBER = BIT(1),
+ PROT_20MHZ = BIT(2),
+ PROT_NONHT_MIXED = BIT(3),
+ PROT_LEGACY_ERP = BIT(5),
+ PROT_NONGF_STA = BIT(7),
+ };
+ u32 rts_threshold;
+
+ skb = __mt76_connac_mcu_alloc_sta_req(&dev->mt76, &mvif->mt76,
+ NULL, len);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ tlv = mt76_connac_mcu_add_tlv(skb, BSS_INFO_PROTECT_INFO,
+ sizeof(*prot));
+ prot = (struct bss_info_prot *)tlv;
+
+ switch (ht_mode & IEEE80211_HT_OP_MODE_PROTECTION) {
+ case IEEE80211_HT_OP_MODE_PROTECTION_NONMEMBER:
+ prot->prot_mode = cpu_to_le32(PROT_NONMEMBER);
+ break;
+ case IEEE80211_HT_OP_MODE_PROTECTION_20MHZ:
+ prot->prot_mode = cpu_to_le32(PROT_20MHZ);
+ break;
+ case IEEE80211_HT_OP_MODE_PROTECTION_NONHT_MIXED:
+ prot->prot_mode = cpu_to_le32(PROT_NONHT_MIXED);
+ break;
+ }
+
+ if (ht_mode & IEEE80211_HT_OP_MODE_NON_GF_STA_PRSNT)
+ prot->prot_mode |= cpu_to_le32(PROT_NONGF_STA);
+
+ if (use_cts_prot)
+ prot->prot_mode |= cpu_to_le32(PROT_LEGACY_ERP);
+
+ /* reuse current RTS setting */
+ rts_threshold = phy->mt76->hw->wiphy->rts_threshold;
+ if (rts_threshold == (u32)-1)
+ prot->rts_len_thres = cpu_to_le32(MT7915_RTS_LEN_THRES);
+ else
+ prot->rts_len_thres = cpu_to_le32(rts_threshold);
+
+ prot->rts_pkt_thres = 0x2;
+
+ prot->he_rts_thres = cpu_to_le16(vif->bss_conf.frame_time_rts_th);
+ if (!prot->he_rts_thres)
+ prot->he_rts_thres = cpu_to_le16(DEFAULT_HE_DURATION_RTS_THRES);
+
+ return mt76_mcu_skb_send_msg(&dev->mt76, skb,
+ MCU_EXT_CMD(BSS_INFO_UPDATE), true);
+}
+
int mt7915_mcu_update_bss_color(struct mt7915_dev *dev, struct ieee80211_vif *vif,
struct cfg80211_he_bss_color *he_bss_color)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
index 49476a4182fd..b7e26c240944 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
@@ -399,6 +399,17 @@ struct bss_info_inband_discovery {
__le16 prob_rsp_len;
} __packed __aligned(4);
+struct bss_info_prot {
+ __le16 tag;
+ __le16 len;
+ __le32 prot_type;
+ __le32 prot_mode;
+ __le32 rts_len_thres;
+ __le16 he_rts_thres;
+ u8 rts_pkt_thres;
+ u8 rsv[5];
+} __packed;
+
enum {
BSS_INFO_BCN_CSA,
BSS_INFO_BCN_BCC,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
index 5fe872ef2e93..f49d6b2f81e4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
@@ -83,6 +83,8 @@
#define MT7915_CRIT_TEMP 110
#define MT7915_MAX_TEMP 120
+#define MT7915_RTS_LEN_THRES 0x92b
+
struct mt7915_vif;
struct mt7915_sta;
struct mt7915_dfs_pulse;
@@ -458,6 +460,8 @@ int mt7915_mcu_add_inband_discov(struct mt7915_dev *dev, struct ieee80211_vif *v
u32 changed);
int mt7915_mcu_add_beacon(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
int enable, u32 changed);
+int mt7915_mcu_set_protection(struct mt7915_phy *phy, struct ieee80211_vif *vif,
+ u8 ht_mode, bool use_cts_prot);
int mt7915_mcu_add_obss_spr(struct mt7915_phy *phy, struct ieee80211_vif *vif,
struct ieee80211_he_obss_pd *he_obss_pd);
int mt7915_mcu_add_rate_ctrl(struct mt7915_dev *dev, struct ieee80211_vif *vif,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
index bc823a7c09bb..a93ae4e44f16 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
@@ -791,7 +791,8 @@ mt7921_regd_set_6ghz_power_type(struct ieee80211_vif *vif, bool is_add)
}
out:
- mt7921_mcu_set_clc(dev, dev->mt76.alpha2, dev->country_ie_env);
+ if (vif->bss_conf.chanreq.oper.chan->band == NL80211_BAND_6GHZ)
+ mt7921_regd_update(dev);
}
int mt7921_mac_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,
@@ -802,6 +803,9 @@ int mt7921_mac_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct mt792x_vif *mvif = (struct mt792x_vif *)vif->drv_priv;
int ret, idx;
+ if (sta->aid > MT7921_MAX_AID)
+ return -ENOENT;
+
idx = mt76_wcid_alloc(dev->mt76.wcid_mask, MT792x_WTBL_STA - 1);
if (idx < 0)
return -ENOSPC;
@@ -845,6 +849,9 @@ int mt7921_mac_sta_event(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct mt792x_sta *msta = (struct mt792x_sta *)sta->drv_priv;
struct mt792x_vif *mvif = (struct mt792x_vif *)vif->drv_priv;
+ if (sta->aid > MT7921_MAX_AID)
+ return -ENOENT;
+
if (ev != MT76_STA_EVENT_ASSOC)
return 0;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
index 16c89815c0b8..7a7e09fcf4b5 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h
@@ -7,6 +7,8 @@
#include "../mt792x.h"
#include "regs.h"
+#define MT7921_MAX_AID 20
+
#define MT7921_TX_RING_SIZE 2048
#define MT7921_TX_MCU_RING_SIZE 256
#define MT7921_TX_FWDL_RING_SIZE 128
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
index 18b7479cee79..e5de05a91aee 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
@@ -807,8 +807,8 @@ mt7925_mac_write_txwi(struct mt76_dev *dev, __le32 *txwi,
txwi[5] = cpu_to_le32(val);
val = MT_TXD6_DAS | FIELD_PREP(MT_TXD6_MSDU_CNT, 1);
- if (!ieee80211_vif_is_mld(vif) ||
- (q_idx >= MT_LMAC_ALTX0 && q_idx <= MT_LMAC_BCN0))
+ if (vif && (!ieee80211_vif_is_mld(vif) ||
+ (q_idx >= MT_LMAC_ALTX0 && q_idx <= MT_LMAC_BCN0)))
val |= MT_TXD6_DIS_MAT;
txwi[6] = cpu_to_le32(val);
txwi[7] = 0;
@@ -849,11 +849,14 @@ static void mt7925_tx_check_aggr(struct ieee80211_sta *sta, struct sk_buff *skb,
bool is_8023;
u16 fc, tid;
+ if (!sta)
+ return;
+
link_sta = rcu_dereference(sta->link[wcid->link_id]);
if (!link_sta)
return;
- if (!sta || !(link_sta->ht_cap.ht_supported || link_sta->he_cap.has_he))
+ if (!(link_sta->ht_cap.ht_supported || link_sta->he_cap.has_he))
return;
tid = skb->priority & IEEE80211_QOS_CTL_TID_MASK;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
index 59d4357819ed..5fc95c862364 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
@@ -535,7 +535,7 @@ static int mt7925_set_mlo_roc(struct mt792x_phy *phy,
phy->roc_grant = false;
- err = mt7925_mcu_set_mlo_roc(mconf, sel_links, 5, ++phy->roc_token_id);
+ err = mt7925_mcu_set_mlo_roc(phy, mconf, sel_links, 5, ++phy->roc_token_id);
if (err < 0) {
clear_bit(MT76_STATE_ROC, &phy->mt76->state);
goto out;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c
index 0e7ea02574de..1f113a618515 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c
@@ -1245,8 +1245,8 @@ int mt7925_mcu_add_key(struct mt76_dev *dev, struct ieee80211_vif *vif,
return mt76_mcu_skb_send_msg(dev, skb, mcu_cmd, true);
}
-int mt7925_mcu_set_mlo_roc(struct mt792x_bss_conf *mconf, u16 sel_links,
- int duration, u8 token_id)
+int mt7925_mcu_set_mlo_roc(struct mt792x_phy *phy, struct mt792x_bss_conf *mconf,
+ u16 sel_links, int duration, u8 token_id)
{
struct mt792x_vif *mvif = mconf->vif;
struct ieee80211_vif *vif = container_of((void *)mvif,
@@ -1281,6 +1281,8 @@ int mt7925_mcu_set_mlo_roc(struct mt792x_bss_conf *mconf, u16 sel_links,
.roc[1].len = cpu_to_le16(sizeof(struct roc_acquire_tlv))
};
+ struct wiphy *wiphy = phy->mt76->hw->wiphy;
+
if (!mconf || hweight16(vif->valid_links) < 2 ||
hweight16(sel_links) != 2)
return -EPERM;
@@ -1303,7 +1305,8 @@ int mt7925_mcu_set_mlo_roc(struct mt792x_bss_conf *mconf, u16 sel_links,
is_AG_band |= links[i].chan->band == NL80211_BAND_2GHZ;
}
- if (vif->cfg.eml_cap & IEEE80211_EML_CAP_EMLSR_SUPP)
+ if (!(wiphy->iftype_ext_capab[0].mld_capa_and_ops &
+ IEEE80211_MLD_CAP_OP_MAX_SIMUL_LINKS))
type = is_AG_band ? MT7925_ROC_REQ_MLSR_AG :
MT7925_ROC_REQ_MLSR_AA;
else
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mt7925.h b/drivers/net/wireless/mediatek/mt76/mt7925/mt7925.h
index 27680ad28b60..bc8d38782a81 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/mt7925.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/mt7925.h
@@ -324,8 +324,8 @@ int mt7925_set_tx_sar_pwr(struct ieee80211_hw *hw,
int mt7925_mcu_regval(struct mt792x_dev *dev, u32 regidx, u32 *val, bool set);
int mt7925_mcu_set_clc(struct mt792x_dev *dev, u8 *alpha2,
enum environment_cap env_cap);
-int mt7925_mcu_set_mlo_roc(struct mt792x_bss_conf *mconf, u16 sel_links,
- int duration, u8 token_id);
+int mt7925_mcu_set_mlo_roc(struct mt792x_phy *phy, struct mt792x_bss_conf *mconf,
+ u16 sel_links, int duration, u8 token_id);
int mt7925_mcu_set_roc(struct mt792x_phy *phy, struct mt792x_bss_conf *mconf,
struct ieee80211_channel *chan, int duration,
enum mt7925_roc_req type, u8 token_id);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/init.c b/drivers/net/wireless/mediatek/mt76/mt7996/init.c
index 5cd2fb7d9835..fc2d46b10b72 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/init.c
@@ -1399,6 +1399,7 @@ int mt7996_register_device(struct mt7996_dev *dev)
void mt7996_unregister_device(struct mt7996_dev *dev)
{
+ cancel_work_sync(&dev->dump_work);
cancel_work_sync(&dev->wed_rro.work);
mt7996_unregister_phy(mt7996_phy3(dev), MT_BAND2);
mt7996_unregister_phy(mt7996_phy2(dev), MT_BAND1);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
index b7a5426c933d..6f8167bb8613 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
@@ -501,7 +501,7 @@ mt7996_mac_fill_rx(struct mt7996_dev *dev, enum mt76_rxq_id q,
!(csum_status & (BIT(0) | BIT(2) | BIT(3))))
skb->ip_summed = CHECKSUM_UNNECESSARY;
- if (rxd1 & MT_RXD3_NORMAL_FCS_ERR)
+ if (rxd3 & MT_RXD3_NORMAL_FCS_ERR)
status->flag |= RX_FLAG_FAILED_FCS_CRC;
if (rxd1 & MT_RXD1_NORMAL_TKIP_MIC_ERR)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
index 8738e4b64542..54567a589344 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
@@ -218,7 +218,7 @@ mt7996_mcu_parse_response(struct mt76_dev *mdev, int cmd,
event = (struct mt7996_mcu_uni_event *)skb->data;
ret = le32_to_cpu(event->status);
/* skip invalid event */
- if (mcu_cmd != event->cid)
+ if (mcu_cmd != le16_to_cpu(event->cid))
ret = -EAGAIN;
} else {
skb_pull(skb, sizeof(struct mt7996_mcu_rxd));
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
index a75e1c9435bb..ded2fe721068 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
@@ -25,8 +25,8 @@ struct mt7996_mcu_rxd {
};
struct mt7996_mcu_uni_event {
- u8 cid;
- u8 __rsv[3];
+ __le16 cid;
+ u8 __rsv[2];
__le32 status; /* 0: success, others: fail */
} __packed;
diff --git a/drivers/net/wireless/realtek/rtlwifi/pci.c b/drivers/net/wireless/realtek/rtlwifi/pci.c
index d080469264cf..f0010336e78c 100644
--- a/drivers/net/wireless/realtek/rtlwifi/pci.c
+++ b/drivers/net/wireless/realtek/rtlwifi/pci.c
@@ -1674,6 +1674,7 @@ static void rtl_pci_deinit(struct ieee80211_hw *hw)
synchronize_irq(rtlpci->pdev->irq);
tasklet_kill(&rtlpriv->works.irq_tasklet);
+ tasklet_kill(&rtlpriv->works.irq_prepare_bcn_tasklet);
cancel_work_sync(&rtlpriv->works.lps_change_work);
}
diff --git a/drivers/net/wireless/realtek/rtw89/phy.c b/drivers/net/wireless/realtek/rtw89/phy.c
index b473e02ecd9e..d544077ec298 100644
--- a/drivers/net/wireless/realtek/rtw89/phy.c
+++ b/drivers/net/wireless/realtek/rtw89/phy.c
@@ -3893,7 +3893,7 @@ static void rtw89_phy_cfo_set_crystal_cap(struct rtw89_dev *rtwdev,
{
struct rtw89_cfo_tracking_info *cfo = &rtwdev->cfo_tracking;
const struct rtw89_chip_info *chip = rtwdev->chip;
- u8 sc_xi_val, sc_xo_val;
+ u8 sc_xi_val = 0, sc_xo_val = 0;
if (!force && cfo->crystal_cap == crystal_cap)
return;
diff --git a/drivers/nfc/trf7970a.c b/drivers/nfc/trf7970a.c
index 9e1a34e23af2..6b8311f526a5 100644
--- a/drivers/nfc/trf7970a.c
+++ b/drivers/nfc/trf7970a.c
@@ -311,6 +311,7 @@
#define TRF7970A_RSSI_OSC_STATUS_RSSI_MASK (BIT(2) | BIT(1) | BIT(0))
#define TRF7970A_RSSI_OSC_STATUS_RSSI_X_MASK (BIT(5) | BIT(4) | BIT(3))
#define TRF7970A_RSSI_OSC_STATUS_RSSI_OSC_OK BIT(6)
+#define TRF7970A_RSSI_OSC_STATUS_RSSI_NOISE_LEVEL 1
#define TRF7970A_SPECIAL_FCN_REG1_COL_7_6 BIT(0)
#define TRF7970A_SPECIAL_FCN_REG1_14_ANTICOLL BIT(1)
@@ -1253,7 +1254,7 @@ static int trf7970a_is_rf_field(struct trf7970a *trf, bool *is_rf_field)
if (ret)
return ret;
- if (rssi & TRF7970A_RSSI_OSC_STATUS_RSSI_MASK)
+ if ((rssi & TRF7970A_RSSI_OSC_STATUS_RSSI_MASK) > TRF7970A_RSSI_OSC_STATUS_RSSI_NOISE_LEVEL)
*is_rf_field = true;
else
*is_rf_field = false;
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 8eb1e4d48c43..758a187a8ab3 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1816,6 +1816,7 @@ static int nvme_create_queue(struct nvme_queue *nvmeq, int qid, bool polled)
static const struct blk_mq_ops nvme_mq_admin_ops = {
.queue_rq = nvme_queue_rq,
.complete = nvme_pci_complete_rq,
+ .commit_rqs = nvme_commit_rqs,
.init_hctx = nvme_admin_init_hctx,
.init_request = nvme_pci_init_request,
.timeout = nvme_timeout,
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index 0e9219881305..6c64af6a7c07 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -359,7 +359,7 @@ static void nvmet_tcp_free_cmd_buffers(struct nvmet_tcp_cmd *cmd)
static void nvmet_tcp_fatal_error(struct nvmet_tcp_queue *queue);
-static void nvmet_tcp_build_pdu_iovec(struct nvmet_tcp_cmd *cmd)
+static int nvmet_tcp_build_pdu_iovec(struct nvmet_tcp_cmd *cmd)
{
struct bio_vec *iov = cmd->iov;
struct scatterlist *sg;
@@ -372,22 +372,19 @@ static void nvmet_tcp_build_pdu_iovec(struct nvmet_tcp_cmd *cmd)
offset = cmd->rbytes_done;
cmd->sg_idx = offset / PAGE_SIZE;
sg_offset = offset % PAGE_SIZE;
- if (!cmd->req.sg_cnt || cmd->sg_idx >= cmd->req.sg_cnt) {
- nvmet_tcp_fatal_error(cmd->queue);
- return;
- }
+ if (!cmd->req.sg_cnt || cmd->sg_idx >= cmd->req.sg_cnt)
+ return -EPROTO;
+
sg = &cmd->req.sg[cmd->sg_idx];
sg_remaining = cmd->req.sg_cnt - cmd->sg_idx;
while (length) {
- if (!sg_remaining) {
- nvmet_tcp_fatal_error(cmd->queue);
- return;
- }
- if (!sg->length || sg->length <= sg_offset) {
- nvmet_tcp_fatal_error(cmd->queue);
- return;
- }
+ if (!sg_remaining)
+ return -EPROTO;
+
+ if (!sg->length || sg->length <= sg_offset)
+ return -EPROTO;
+
u32 iov_len = min_t(u32, length, sg->length - sg_offset);
bvec_set_page(iov, sg_page(sg), iov_len,
@@ -402,6 +399,7 @@ static void nvmet_tcp_build_pdu_iovec(struct nvmet_tcp_cmd *cmd)
iov_iter_bvec(&cmd->recv_msg.msg_iter, ITER_DEST, cmd->iov,
nr_pages, cmd->pdu_len);
+ return 0;
}
static void nvmet_tcp_fatal_error(struct nvmet_tcp_queue *queue)
@@ -997,7 +995,7 @@ static int nvmet_tcp_handle_icreq(struct nvmet_tcp_queue *queue)
return 0;
}
-static void nvmet_tcp_handle_req_failure(struct nvmet_tcp_queue *queue,
+static int nvmet_tcp_handle_req_failure(struct nvmet_tcp_queue *queue,
struct nvmet_tcp_cmd *cmd, struct nvmet_req *req)
{
size_t data_len = le32_to_cpu(req->cmd->common.dptr.sgl.length);
@@ -1013,19 +1011,23 @@ static void nvmet_tcp_handle_req_failure(struct nvmet_tcp_queue *queue,
if (!nvme_is_write(cmd->req.cmd) || !data_len ||
data_len > cmd->req.port->inline_data_size) {
nvmet_prepare_receive_pdu(queue);
- return;
+ return 0;
}
ret = nvmet_tcp_map_data(cmd);
if (unlikely(ret)) {
pr_err("queue %d: failed to map data\n", queue->idx);
nvmet_tcp_fatal_error(queue);
- return;
+ return -EPROTO;
}
queue->rcv_state = NVMET_TCP_RECV_DATA;
- nvmet_tcp_build_pdu_iovec(cmd);
cmd->flags |= NVMET_TCP_F_INIT_FAILED;
+ ret = nvmet_tcp_build_pdu_iovec(cmd);
+ if (unlikely(ret))
+ pr_err("queue %d: failed to build PDU iovec\n", queue->idx);
+
+ return ret;
}
static int nvmet_tcp_handle_h2c_data_pdu(struct nvmet_tcp_queue *queue)
@@ -1077,7 +1079,10 @@ static int nvmet_tcp_handle_h2c_data_pdu(struct nvmet_tcp_queue *queue)
goto err_proto;
}
cmd->pdu_recv = 0;
- nvmet_tcp_build_pdu_iovec(cmd);
+ if (unlikely(nvmet_tcp_build_pdu_iovec(cmd))) {
+ pr_err("queue %d: failed to build PDU iovec\n", queue->idx);
+ goto err_proto;
+ }
queue->cmd = cmd;
queue->rcv_state = NVMET_TCP_RECV_DATA;
@@ -1140,8 +1145,7 @@ static int nvmet_tcp_done_recv_pdu(struct nvmet_tcp_queue *queue)
req->cmd->common.opcode,
le32_to_cpu(req->cmd->common.dptr.sgl.length));
- nvmet_tcp_handle_req_failure(queue, queue->cmd, req);
- return 0;
+ return nvmet_tcp_handle_req_failure(queue, queue->cmd, req);
}
ret = nvmet_tcp_map_data(queue->cmd);
@@ -1158,8 +1162,11 @@ static int nvmet_tcp_done_recv_pdu(struct nvmet_tcp_queue *queue)
if (nvmet_tcp_need_data_in(queue->cmd)) {
if (nvmet_tcp_has_inline_data(queue->cmd)) {
queue->rcv_state = NVMET_TCP_RECV_DATA;
- nvmet_tcp_build_pdu_iovec(queue->cmd);
- return 0;
+ ret = nvmet_tcp_build_pdu_iovec(queue->cmd);
+ if (unlikely(ret))
+ pr_err("queue %d: failed to build PDU iovec\n",
+ queue->idx);
+ return ret;
}
/* send back R2T */
nvmet_tcp_queue_response(&queue->cmd->req);
diff --git a/drivers/nvmem/brcm_nvram.c b/drivers/nvmem/brcm_nvram.c
index 3d8c87835f4d..65f458af3a19 100644
--- a/drivers/nvmem/brcm_nvram.c
+++ b/drivers/nvmem/brcm_nvram.c
@@ -100,7 +100,7 @@ static int brcm_nvram_read_post_process_macaddr(void *context, const char *id, i
{
u8 mac[ETH_ALEN];
- if (bytes != 3 * ETH_ALEN - 1)
+ if (bytes != MAC_ADDR_STR_LEN)
return -EINVAL;
if (!mac_pton(buf, mac))
diff --git a/drivers/nvmem/layouts/u-boot-env.c b/drivers/nvmem/layouts/u-boot-env.c
index 21f6dcf905dd..8571aac56295 100644
--- a/drivers/nvmem/layouts/u-boot-env.c
+++ b/drivers/nvmem/layouts/u-boot-env.c
@@ -37,7 +37,7 @@ static int u_boot_env_read_post_process_ethaddr(void *context, const char *id, i
{
u8 mac[ETH_ALEN];
- if (bytes != 3 * ETH_ALEN - 1)
+ if (bytes != MAC_ADDR_STR_LEN)
return -EINVAL;
if (!mac_pton(buf, mac))
diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
index f700e8c49082..55bd13a2496e 100644
--- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
+++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
@@ -285,21 +285,19 @@ static int cdns_pcie_ep_get_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no)
}
static int cdns_pcie_ep_set_msix(struct pci_epc *epc, u8 fn, u8 vfn,
- u16 interrupts, enum pci_barno bir,
- u32 offset)
+ u16 nr_irqs, enum pci_barno bir, u32 offset)
{
struct cdns_pcie_ep *ep = epc_get_drvdata(epc);
struct cdns_pcie *pcie = &ep->pcie;
u32 cap = CDNS_PCIE_EP_FUNC_MSIX_CAP_OFFSET;
u32 val, reg;
- u16 actual_interrupts = interrupts + 1;
fn = cdns_pcie_get_fn_from_vfn(pcie, fn, vfn);
reg = cap + PCI_MSIX_FLAGS;
val = cdns_pcie_ep_fn_readw(pcie, fn, reg);
val &= ~PCI_MSIX_FLAGS_QSIZE;
- val |= interrupts; /* 0's based value */
+ val |= nr_irqs - 1; /* encoded as N-1 */
cdns_pcie_ep_fn_writew(pcie, fn, reg, val);
/* Set MSIX BAR and offset */
@@ -309,7 +307,7 @@ static int cdns_pcie_ep_set_msix(struct pci_epc *epc, u8 fn, u8 vfn,
/* Set PBA BAR and offset. BAR must match MSIX BAR */
reg = cap + PCI_MSIX_PBA;
- val = (offset + (actual_interrupts * PCI_MSIX_ENTRY_SIZE)) | bir;
+ val = (offset + (nr_irqs * PCI_MSIX_ENTRY_SIZE)) | bir;
cdns_pcie_ep_fn_writel(pcie, fn, reg, val);
return 0;
diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
index 189675747b2b..ec306406959a 100644
--- a/drivers/pci/controller/dwc/pcie-designware-ep.c
+++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
@@ -406,13 +406,12 @@ static int dw_pcie_ep_get_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no)
}
static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
- u16 interrupts, enum pci_barno bir, u32 offset)
+ u16 nr_irqs, enum pci_barno bir, u32 offset)
{
struct dw_pcie_ep *ep = epc_get_drvdata(epc);
struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
struct dw_pcie_ep_func *ep_func;
u32 val, reg;
- u16 actual_interrupts = interrupts + 1;
ep_func = dw_pcie_ep_get_func_from_ep(ep, func_no);
if (!ep_func || !ep_func->msix_cap)
@@ -423,15 +422,15 @@ static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
reg = ep_func->msix_cap + PCI_MSIX_FLAGS;
val = dw_pcie_ep_readw_dbi(ep, func_no, reg);
val &= ~PCI_MSIX_FLAGS_QSIZE;
- val |= interrupts; /* 0's based value */
- dw_pcie_writew_dbi(pci, reg, val);
+ val |= nr_irqs - 1; /* encoded as N-1 */
+ dw_pcie_ep_writew_dbi(ep, func_no, reg, val);
reg = ep_func->msix_cap + PCI_MSIX_TABLE;
val = offset | bir;
dw_pcie_ep_writel_dbi(ep, func_no, reg, val);
reg = ep_func->msix_cap + PCI_MSIX_PBA;
- val = (offset + (actual_interrupts * PCI_MSIX_ENTRY_SIZE)) | bir;
+ val = (offset + (nr_irqs * PCI_MSIX_ENTRY_SIZE)) | bir;
dw_pcie_ep_writel_dbi(ep, func_no, reg, val);
dw_pcie_dbi_ro_wr_dis(pci);
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 3e3168204e30..deda5b040d7a 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -984,11 +984,23 @@ int dw_pcie_resume_noirq(struct dw_pcie *pci)
ret = dw_pcie_start_link(pci);
if (ret)
- return ret;
+ goto err_deinit;
ret = dw_pcie_wait_for_link(pci);
- if (ret)
- return ret;
+ if (ret == -ETIMEDOUT)
+ goto err_stop_link;
+
+ if (pci->pp.ops->post_init)
+ pci->pp.ops->post_init(&pci->pp);
+
+ return 0;
+
+err_stop_link:
+ dw_pcie_stop_link(pci);
+
+err_deinit:
+ if (pci->pp.ops->deinit)
+ pci->pp.ops->deinit(&pci->pp);
return ret;
}
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index f7d10cb788e0..bc3d6269f33b 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -429,13 +429,13 @@ static inline void dw_pcie_writel_atu_ob(struct dw_pcie *pci, u32 index, u32 reg
static inline u32 dw_pcie_enable_ecrc(u32 val)
{
/*
- * DesignWare core version 4.90A has a design issue where the 'TD'
- * bit in the Control register-1 of the ATU outbound region acts
- * like an override for the ECRC setting, i.e., the presence of TLP
- * Digest (ECRC) in the outgoing TLPs is solely determined by this
- * bit. This is contrary to the PCIe spec which says that the
- * enablement of the ECRC is solely determined by the AER
- * registers.
+ * DWC versions 0x3530302a and 0x3536322a have a design issue where
+ * the 'TD' bit in the Control register-1 of the ATU outbound
+ * region acts like an override for the ECRC setting, i.e., the
+ * presence of TLP Digest (ECRC) in the outgoing TLPs is solely
+ * determined by this bit. This is contrary to the PCIe spec which
+ * says that the enablement of the ECRC is solely determined by the
+ * AER registers.
*
* Because of this, even when the ECRC is enabled through AER
* registers, the transactions going through ATU won't have TLP
@@ -505,7 +505,7 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
if (upper_32_bits(limit_addr) > upper_32_bits(cpu_addr) &&
dw_pcie_ver_is_ge(pci, 460A))
val |= PCIE_ATU_INCREASE_REGION_SIZE;
- if (dw_pcie_ver_is(pci, 490A))
+ if (dw_pcie_ver_is(pci, 490A) || dw_pcie_ver_is(pci, 500A))
val = dw_pcie_enable_ecrc(val);
dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL1, val);
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 0fad7751490f..a5f58ac8ea94 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -31,8 +31,10 @@
#define DW_PCIE_VER_470A 0x3437302a
#define DW_PCIE_VER_480A 0x3438302a
#define DW_PCIE_VER_490A 0x3439302a
+#define DW_PCIE_VER_500A 0x3530302a
#define DW_PCIE_VER_520A 0x3532302a
#define DW_PCIE_VER_540A 0x3534302a
+#define DW_PCIE_VER_562A 0x3536322a
#define __dw_pcie_ver_cmp(_pci, _ver, _op) \
((_pci)->version _op DW_PCIE_VER_ ## _ver)
diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
index 5d27cd149f51..ae0f36e270ba 100644
--- a/drivers/pci/controller/dwc/pcie-qcom.c
+++ b/drivers/pci/controller/dwc/pcie-qcom.c
@@ -329,15 +329,20 @@ static void qcom_pcie_clear_aspm_l0s(struct dw_pcie *pci)
dw_pcie_dbi_ro_wr_dis(pci);
}
-static void qcom_pcie_clear_hpc(struct dw_pcie *pci)
+static void qcom_pcie_set_slot_nccs(struct dw_pcie *pci)
{
u16 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
u32 val;
dw_pcie_dbi_ro_wr_en(pci);
+ /*
+ * Qcom PCIe Root Ports do not support generating command completion
+ * notifications for the Hot-Plug commands. So set the NCCS field to
+ * avoid waiting for the completions.
+ */
val = readl(pci->dbi_base + offset + PCI_EXP_SLTCAP);
- val &= ~PCI_EXP_SLTCAP_HPC;
+ val |= PCI_EXP_SLTCAP_NCCS;
writel(val, pci->dbi_base + offset + PCI_EXP_SLTCAP);
dw_pcie_dbi_ro_wr_dis(pci);
@@ -532,7 +537,7 @@ static int qcom_pcie_post_init_2_1_0(struct qcom_pcie *pcie)
writel(CFG_BRIDGE_SB_INIT,
pci->dbi_base + AXI_MSTR_RESP_COMP_CTRL1);
- qcom_pcie_clear_hpc(pcie->pci);
+ qcom_pcie_set_slot_nccs(pcie->pci);
return 0;
}
@@ -612,7 +617,7 @@ static int qcom_pcie_post_init_1_0_0(struct qcom_pcie *pcie)
writel(val, pcie->parf + PARF_AXI_MSTR_WR_ADDR_HALT);
}
- qcom_pcie_clear_hpc(pcie->pci);
+ qcom_pcie_set_slot_nccs(pcie->pci);
return 0;
}
@@ -705,7 +710,7 @@ static int qcom_pcie_post_init_2_3_2(struct qcom_pcie *pcie)
val |= EN;
writel(val, pcie->parf + PARF_AXI_MSTR_WR_ADDR_HALT_V2);
- qcom_pcie_clear_hpc(pcie->pci);
+ qcom_pcie_set_slot_nccs(pcie->pci);
return 0;
}
@@ -1009,7 +1014,7 @@ static int qcom_pcie_post_init_2_7_0(struct qcom_pcie *pcie)
writel(WR_NO_SNOOP_OVERIDE_EN | RD_NO_SNOOP_OVERIDE_EN,
pcie->parf + PARF_NO_SNOOP_OVERIDE);
- qcom_pcie_clear_hpc(pcie->pci);
+ qcom_pcie_set_slot_nccs(pcie->pci);
return 0;
}
diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
index 397c2f9477a1..8dcb63f813be 100644
--- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -427,7 +427,7 @@ static const struct pci_epc_features rcar_gen4_pcie_epc_features = {
.bar[BAR_3] = { .type = BAR_RESERVED, },
.bar[BAR_4] = { .type = BAR_FIXED, .fixed_size = 256 },
.bar[BAR_5] = { .type = BAR_RESERVED, },
- .align = SZ_1M,
+ .align = SZ_4K,
};
static const struct pci_epc_features*
diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index c2d626b090e3..368c50abd4fc 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -35,8 +35,8 @@
#include <soc/tegra/bpmp-abi.h>
#include "../../pci.h"
-#define TEGRA194_DWC_IP_VER 0x490A
-#define TEGRA234_DWC_IP_VER 0x562A
+#define TEGRA194_DWC_IP_VER DW_PCIE_VER_500A
+#define TEGRA234_DWC_IP_VER DW_PCIE_VER_562A
#define APPL_PINMUX 0x0
#define APPL_PINMUX_PEX_RST BIT(0)
@@ -137,7 +137,11 @@
#define APPL_DEBUG_PM_LINKST_IN_L0 0x11
#define APPL_DEBUG_LTSSM_STATE_MASK GENMASK(8, 3)
#define APPL_DEBUG_LTSSM_STATE_SHIFT 3
-#define LTSSM_STATE_PRE_DETECT 5
+#define LTSSM_STATE_DETECT_QUIET 0x00
+#define LTSSM_STATE_DETECT_ACT 0x08
+#define LTSSM_STATE_PRE_DETECT_QUIET 0x28
+#define LTSSM_STATE_DETECT_WAIT 0x30
+#define LTSSM_STATE_L2_IDLE 0xa8
#define APPL_RADM_STATUS 0xE4
#define APPL_PM_XMT_TURNOFF_STATE BIT(0)
@@ -198,9 +202,8 @@
#define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_MASK GENMASK(11, 8)
#define CAP_SPCIE_CAP_OFF_USP_TX_PRESET0_SHIFT 8
-#define PME_ACK_TIMEOUT 10000
-
-#define LTSSM_TIMEOUT 50000 /* 50ms */
+#define LTSSM_DELAY_US 10000 /* 10 ms */
+#define LTSSM_TIMEOUT_US 120000 /* 120 ms */
#define GEN3_GEN4_EQ_PRESET_INIT 5
@@ -484,15 +487,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
if (val & PCI_COMMAND_MASTER) {
ktime_t timeout;
- /* 110us for both snoop and no-snoop */
- val = FIELD_PREP(PCI_LTR_VALUE_MASK, 110) |
- FIELD_PREP(PCI_LTR_SCALE_MASK, 2) |
- LTR_MSG_REQ |
- FIELD_PREP(PCI_LTR_NOSNOOP_VALUE, 110) |
- FIELD_PREP(PCI_LTR_NOSNOOP_SCALE, 2) |
- LTR_NOSNOOP_MSG_REQ;
- appl_writel(pcie, val, APPL_LTR_MSG_1);
-
/* Send LTR upstream */
val = appl_readl(pcie, APPL_LTR_MSG_2);
val |= APPL_LTR_MSG_2_LTR_MSG_REQ_STATE;
@@ -1039,7 +1033,8 @@ static void tegra_pcie_dw_stop_link(struct dw_pcie *pci)
{
struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
- disable_irq(pcie->pex_rst_irq);
+ if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+ disable_irq(pcie->pex_rst_irq);
}
static const struct dw_pcie_ops tegra_dw_pcie_ops = {
@@ -1180,9 +1175,9 @@ static int tegra_pcie_dw_parse_dt(struct tegra_pcie_dw *pcie)
return err;
}
- pcie->pex_refclk_sel_gpiod = devm_gpiod_get(pcie->dev,
- "nvidia,refclk-select",
- GPIOD_OUT_HIGH);
+ pcie->pex_refclk_sel_gpiod = devm_gpiod_get_optional(pcie->dev,
+ "nvidia,refclk-select",
+ GPIOD_OUT_HIGH);
if (IS_ERR(pcie->pex_refclk_sel_gpiod)) {
int err = PTR_ERR(pcie->pex_refclk_sel_gpiod);
const char *level = KERN_ERR;
@@ -1272,44 +1267,6 @@ static int tegra_pcie_bpmp_set_pll_state(struct tegra_pcie_dw *pcie,
return 0;
}
-static void tegra_pcie_downstream_dev_to_D0(struct tegra_pcie_dw *pcie)
-{
- struct dw_pcie_rp *pp = &pcie->pci.pp;
- struct pci_bus *child, *root_bus = NULL;
- struct pci_dev *pdev;
-
- /*
- * link doesn't go into L2 state with some of the endpoints with Tegra
- * if they are not in D0 state. So, need to make sure that immediate
- * downstream devices are in D0 state before sending PME_TurnOff to put
- * link into L2 state.
- * This is as per PCI Express Base r4.0 v1.0 September 27-2017,
- * 5.2 Link State Power Management (Page #428).
- */
-
- list_for_each_entry(child, &pp->bridge->bus->children, node) {
- /* Bring downstream devices to D0 if they are not already in */
- if (child->parent == pp->bridge->bus) {
- root_bus = child;
- break;
- }
- }
-
- if (!root_bus) {
- dev_err(pcie->dev, "Failed to find downstream devices\n");
- return;
- }
-
- list_for_each_entry(pdev, &root_bus->devices, bus_list) {
- if (PCI_SLOT(pdev->devfn) == 0) {
- if (pci_set_power_state(pdev, PCI_D0))
- dev_err(pcie->dev,
- "Failed to transition %s to D0 state\n",
- dev_name(&pdev->dev));
- }
- }
-}
-
static int tegra_pcie_get_slot_regulators(struct tegra_pcie_dw *pcie)
{
pcie->slot_ctl_3v3 = devm_regulator_get_optional(pcie->dev, "vpcie3v3");
@@ -1570,9 +1527,10 @@ static int tegra_pcie_try_link_l2(struct tegra_pcie_dw *pcie)
val |= APPL_PM_XMT_TURNOFF_STATE;
appl_writel(pcie, val, APPL_RADM_STATUS);
- return readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG, val,
- val & APPL_DEBUG_PM_LINKST_IN_L2_LAT,
- 1, PME_ACK_TIMEOUT);
+ return readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
+ val & APPL_DEBUG_PM_LINKST_IN_L2_LAT,
+ PCIE_PME_TO_L2_TIMEOUT_US/10,
+ PCIE_PME_TO_L2_TIMEOUT_US);
}
static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
@@ -1607,23 +1565,22 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
data &= ~APPL_PINMUX_PEX_RST;
appl_writel(pcie, data, APPL_PINMUX);
+ err = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, data,
+ ((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
+ ((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||
+ ((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_PRE_DETECT_QUIET) ||
+ ((data & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_WAIT),
+ LTSSM_DELAY_US, LTSSM_TIMEOUT_US);
+ if (err)
+ dev_info(pcie->dev, "LTSSM state: 0x%x detect timeout: %d\n", data, err);
+
/*
- * Some cards do not go to detect state even after de-asserting
- * PERST#. So, de-assert LTSSM to bring link to detect state.
+ * Deassert LTSSM state to stop the state toggling between
+ * Polling and Detect.
*/
data = readl(pcie->appl_base + APPL_CTRL);
data &= ~APPL_CTRL_LTSSM_EN;
writel(data, pcie->appl_base + APPL_CTRL);
-
- err = readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG,
- data,
- ((data &
- APPL_DEBUG_LTSSM_STATE_MASK) >>
- APPL_DEBUG_LTSSM_STATE_SHIFT) ==
- LTSSM_STATE_PRE_DETECT,
- 1, LTSSM_TIMEOUT);
- if (err)
- dev_info(pcie->dev, "Link didn't go to detect state\n");
}
/*
* DBI registers may not be accessible after this as PLL-E would be
@@ -1639,7 +1596,6 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
static void tegra_pcie_deinit_controller(struct tegra_pcie_dw *pcie)
{
- tegra_pcie_downstream_dev_to_D0(pcie);
dw_pcie_host_deinit(&pcie->pci.pp);
tegra_pcie_dw_pme_turnoff(pcie);
tegra_pcie_unconfig_controller(pcie);
@@ -1705,19 +1661,24 @@ static void pex_ep_event_pex_rst_assert(struct tegra_pcie_dw *pcie)
if (pcie->ep_state == EP_STATE_DISABLED)
return;
- /* Disable LTSSM */
+ ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
+ ((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_QUIET) ||
+ ((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_ACT) ||
+ ((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_PRE_DETECT_QUIET) ||
+ ((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_DETECT_WAIT) ||
+ ((val & APPL_DEBUG_LTSSM_STATE_MASK) == LTSSM_STATE_L2_IDLE),
+ LTSSM_DELAY_US, LTSSM_TIMEOUT_US);
+ if (ret)
+ dev_info(pcie->dev, "LTSSM state: 0x%x detect timeout: %d\n", val, ret);
+
+ /*
+ * Deassert LTSSM state to stop the state toggling between
+ * Polling and Detect.
+ */
val = appl_readl(pcie, APPL_CTRL);
val &= ~APPL_CTRL_LTSSM_EN;
appl_writel(pcie, val, APPL_CTRL);
- ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
- ((val & APPL_DEBUG_LTSSM_STATE_MASK) >>
- APPL_DEBUG_LTSSM_STATE_SHIFT) ==
- LTSSM_STATE_PRE_DETECT,
- 1, LTSSM_TIMEOUT);
- if (ret)
- dev_err(pcie->dev, "Failed to go Detect state: %d\n", ret);
-
reset_control_assert(pcie->core_rst);
tegra_pcie_disable_phy(pcie);
@@ -1796,10 +1757,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
goto fail_phy;
}
- /* Perform cleanup that requires refclk */
- pci_epc_deinit_notify(pcie->pci.ep.epc);
- dw_pcie_ep_cleanup(&pcie->pci.ep);
-
/* Clear any stale interrupt statuses */
appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L0);
appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_0_0);
@@ -1858,8 +1815,25 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
val |= APPL_INTR_EN_L1_0_0_RDLH_LINK_UP_INT_EN;
appl_writel(pcie, val, APPL_INTR_EN_L1_0_0);
+ /* 110us for both snoop and no-snoop */
+ val = FIELD_PREP(PCI_LTR_VALUE_MASK, 110) |
+ FIELD_PREP(PCI_LTR_SCALE_MASK, 2) |
+ LTR_MSG_REQ |
+ FIELD_PREP(PCI_LTR_NOSNOOP_VALUE, 110) |
+ FIELD_PREP(PCI_LTR_NOSNOOP_SCALE, 2) |
+ LTR_NOSNOOP_MSG_REQ;
+ appl_writel(pcie, val, APPL_LTR_MSG_1);
+
reset_control_deassert(pcie->core_rst);
+ /* Perform cleanup that requires refclk and core reset deasserted */
+ pci_epc_deinit_notify(pcie->pci.ep.epc);
+ dw_pcie_ep_cleanup(&pcie->pci.ep);
+
+ val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL);
+ val &= ~PORT_LOGIC_SPEED_CHANGE;
+ dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val);
+
if (pcie->update_fc_fixup) {
val = dw_pcie_readl_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF);
val |= 0x1 << CFG_TIMER_CTRL_ACK_NAK_SHIFT;
@@ -2308,6 +2282,7 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)
static void tegra_pcie_dw_remove(struct platform_device *pdev)
{
struct tegra_pcie_dw *pcie = platform_get_drvdata(pdev);
+ struct dw_pcie_ep *ep = &pcie->pci.ep;
if (pcie->of_data->mode == DW_PCIE_RC_TYPE) {
if (!pcie->link_state)
@@ -2319,6 +2294,7 @@ static void tegra_pcie_dw_remove(struct platform_device *pdev)
} else {
disable_irq(pcie->pex_rst_irq);
pex_ep_event_pex_rst_assert(pcie);
+ dw_pcie_ep_deinit(ep);
}
pm_runtime_disable(pcie->dev);
@@ -2327,16 +2303,28 @@ static void tegra_pcie_dw_remove(struct platform_device *pdev)
gpiod_set_value(pcie->pex_refclk_sel_gpiod, 0);
}
-static int tegra_pcie_dw_suspend_late(struct device *dev)
+static int tegra_pcie_dw_suspend(struct device *dev)
{
struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
- u32 val;
if (pcie->of_data->mode == DW_PCIE_EP_TYPE) {
- dev_err(dev, "Failed to Suspend as Tegra PCIe is in EP mode\n");
- return -EPERM;
+ if (pcie->ep_state == EP_STATE_ENABLED) {
+ dev_err(dev, "Tegra PCIe is in EP mode, suspend not allowed\n");
+ return -EPERM;
+ }
+
+ disable_irq(pcie->pex_rst_irq);
+ return 0;
}
+ return 0;
+}
+
+static int tegra_pcie_dw_suspend_late(struct device *dev)
+{
+ struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
+ u32 val;
+
if (!pcie->link_state)
return 0;
@@ -2356,10 +2344,12 @@ static int tegra_pcie_dw_suspend_noirq(struct device *dev)
{
struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
+ if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+ return 0;
+
if (!pcie->link_state)
return 0;
- tegra_pcie_downstream_dev_to_D0(pcie);
tegra_pcie_dw_pme_turnoff(pcie);
tegra_pcie_unconfig_controller(pcie);
@@ -2371,6 +2361,9 @@ static int tegra_pcie_dw_resume_noirq(struct device *dev)
struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
int ret;
+ if (pcie->of_data->mode == DW_PCIE_EP_TYPE)
+ return 0;
+
if (!pcie->link_state)
return 0;
@@ -2403,8 +2396,8 @@ static int tegra_pcie_dw_resume_early(struct device *dev)
u32 val;
if (pcie->of_data->mode == DW_PCIE_EP_TYPE) {
- dev_err(dev, "Suspend is not supported in EP mode");
- return -ENOTSUPP;
+ enable_irq(pcie->pex_rst_irq);
+ return 0;
}
if (!pcie->link_state)
@@ -2433,7 +2426,6 @@ static void tegra_pcie_dw_shutdown(struct platform_device *pdev)
return;
debugfs_remove_recursive(pcie->debugfs);
- tegra_pcie_downstream_dev_to_D0(pcie);
disable_irq(pcie->pci.pp.irq);
if (IS_ENABLED(CONFIG_PCI_MSI))
@@ -2510,6 +2502,7 @@ static const struct of_device_id tegra_pcie_dw_of_match[] = {
};
static const struct dev_pm_ops tegra_pcie_dw_pm_ops = {
+ .suspend = tegra_pcie_dw_suspend,
.suspend_late = tegra_pcie_dw_suspend_late,
.suspend_noirq = tegra_pcie_dw_suspend_noirq,
.resume_noirq = tegra_pcie_dw_resume_noirq,
diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c
index 66ce4b5d309b..b373ece9542c 100644
--- a/drivers/pci/controller/pcie-mediatek-gen3.c
+++ b/drivers/pci/controller/pcie-mediatek-gen3.c
@@ -794,14 +794,14 @@ static int mtk_pcie_setup_irq(struct mtk_gen3_pcie *pcie)
struct platform_device *pdev = to_platform_device(dev);
int err;
- err = mtk_pcie_init_irq_domains(pcie);
- if (err)
- return err;
-
pcie->irq = platform_get_irq(pdev, 0);
if (pcie->irq < 0)
return pcie->irq;
+ err = mtk_pcie_init_irq_domains(pcie);
+ if (err)
+ return err;
+
irq_set_chained_handler_and_data(pcie->irq, mtk_pcie_irq_handler, pcie);
return 0;
diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
index 75c668829003..03d694944714 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -382,29 +382,28 @@ EXPORT_SYMBOL_GPL(pci_epc_get_msix);
* @epc: the EPC device on which MSI-X has to be configured
* @func_no: the physical endpoint function number in the EPC device
* @vfunc_no: the virtual endpoint function number in the physical function
- * @interrupts: number of MSI-X interrupts required by the EPF
+ * @nr_irqs: number of MSI-X interrupts required by the EPF
* @bir: BAR where the MSI-X table resides
* @offset: Offset pointing to the start of MSI-X table
*
* Invoke to set the required number of MSI-X interrupts.
*/
-int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
- u16 interrupts, enum pci_barno bir, u32 offset)
+int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u16 nr_irqs,
+ enum pci_barno bir, u32 offset)
{
int ret;
if (!pci_epc_function_is_valid(epc, func_no, vfunc_no))
return -EINVAL;
- if (interrupts < 1 || interrupts > 2048)
+ if (nr_irqs < 1 || nr_irqs > 2048)
return -EINVAL;
if (!epc->ops->set_msix)
return 0;
mutex_lock(&epc->lock);
- ret = epc->ops->set_msix(epc, func_no, vfunc_no, interrupts - 1, bir,
- offset);
+ ret = epc->ops->set_msix(epc, func_no, vfunc_no, nr_irqs, bir, offset);
mutex_unlock(&epc->lock);
return ret;
diff --git a/drivers/pci/npem.c b/drivers/pci/npem.c
index 97507e0df769..b5d012edebf3 100644
--- a/drivers/pci/npem.c
+++ b/drivers/pci/npem.c
@@ -504,7 +504,7 @@ static int pci_npem_set_led_classdev(struct npem *npem, struct npem_led *nled)
led->brightness_get = brightness_get;
led->max_brightness = 1;
led->default_trigger = "none";
- led->flags = 0;
+ led->flags = LED_HW_PLUGGABLE;
ret = led_classdev_register(&npem->dev->dev, led);
if (ret)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index a00a2ce01045..f0fbe45bfb9a 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -138,9 +138,11 @@ static const struct pci_device_id *pci_match_device(struct pci_driver *drv,
{
struct pci_dynid *dynid;
const struct pci_device_id *found_id = NULL, *ids;
+ int ret;
/* When driver_override is set, only bind to the matching driver */
- if (dev->driver_override && strcmp(dev->driver_override, drv->name))
+ ret = device_match_driver_override(&dev->dev, &drv->driver);
+ if (ret == 0)
return NULL;
/* Look at the dynamic ids first, before the static ones */
@@ -164,7 +166,7 @@ static const struct pci_device_id *pci_match_device(struct pci_driver *drv,
* matching.
*/
if (found_id->override_only) {
- if (dev->driver_override)
+ if (ret > 0)
return found_id;
} else {
return found_id;
@@ -172,11 +174,16 @@ static const struct pci_device_id *pci_match_device(struct pci_driver *drv,
}
/* driver_override will always match, send a dummy id */
- if (dev->driver_override)
+ if (ret > 0)
return &pci_device_id_any;
return NULL;
}
+static void _pci_free_device(struct device *dev)
+{
+ kfree(to_pci_dev(dev));
+}
+
/**
* new_id_store - sysfs frontend to pci_add_dynid()
* @driver: target device driver
@@ -212,11 +219,13 @@ static ssize_t new_id_store(struct device_driver *driver, const char *buf,
pdev->subsystem_vendor = subvendor;
pdev->subsystem_device = subdevice;
pdev->class = class;
+ pdev->dev.release = _pci_free_device;
+ device_initialize(&pdev->dev);
if (pci_match_device(pdrv, pdev))
retval = -EEXIST;
- kfree(pdev);
+ put_device(&pdev->dev);
if (retval)
return retval;
@@ -423,7 +432,7 @@ static int __pci_device_probe(struct pci_driver *drv, struct pci_dev *pci_dev)
static inline bool pci_device_can_probe(struct pci_dev *pdev)
{
return (!pdev->is_virtfn || pdev->physfn->sriov->drivers_autoprobe ||
- pdev->driver_override);
+ device_has_driver_override(&pdev->dev));
}
#else
static inline bool pci_device_can_probe(struct pci_dev *pdev)
@@ -1677,6 +1686,7 @@ static void pci_dma_cleanup(struct device *dev)
const struct bus_type pci_bus_type = {
.name = "pci",
+ .driver_override = true,
.match = pci_bus_match,
.uevent = pci_uevent,
.probe = pci_device_probe,
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 96f9cf9f8d64..122c182229b3 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -606,33 +606,6 @@ static ssize_t devspec_show(struct device *dev,
static DEVICE_ATTR_RO(devspec);
#endif
-static ssize_t driver_override_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct pci_dev *pdev = to_pci_dev(dev);
- int ret;
-
- ret = driver_set_override(dev, &pdev->driver_override, buf, count);
- if (ret)
- return ret;
-
- return count;
-}
-
-static ssize_t driver_override_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct pci_dev *pdev = to_pci_dev(dev);
- ssize_t len;
-
- device_lock(dev);
- len = sysfs_emit(buf, "%s\n", pdev->driver_override);
- device_unlock(dev);
- return len;
-}
-static DEVICE_ATTR_RW(driver_override);
-
static struct attribute *pci_dev_attrs[] = {
&dev_attr_power_state.attr,
&dev_attr_resource.attr,
@@ -660,7 +633,6 @@ static struct attribute *pci_dev_attrs[] = {
#ifdef CONFIG_OF
&dev_attr_devspec.attr,
#endif
- &dev_attr_driver_override.attr,
&dev_attr_ari_enabled.attr,
NULL,
};
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 2c8d0c1c317e..d44c07bb0a22 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3850,8 +3850,7 @@ int pci_rebar_set_size(struct pci_dev *pdev, int bar, int size)
*/
int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)
{
- struct pci_bus *bus = dev->bus;
- struct pci_dev *bridge;
+ struct pci_dev *root, *bridge;
u32 cap, ctl2;
/*
@@ -3881,35 +3880,35 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)
return -EINVAL;
}
- while (bus->parent) {
- bridge = bus->self;
+ root = pcie_find_root_port(dev);
+ if (!root)
+ return -EINVAL;
- pcie_capability_read_dword(bridge, PCI_EXP_DEVCAP2, &cap);
+ pcie_capability_read_dword(root, PCI_EXP_DEVCAP2, &cap);
+ if ((cap & cap_mask) != cap_mask)
+ return -EINVAL;
+ bridge = pci_upstream_bridge(dev);
+ while (bridge != root) {
switch (pci_pcie_type(bridge)) {
- /* Ensure switch ports support AtomicOp routing */
case PCI_EXP_TYPE_UPSTREAM:
- case PCI_EXP_TYPE_DOWNSTREAM:
- if (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))
- return -EINVAL;
- break;
-
- /* Ensure root port supports all the sizes we care about */
- case PCI_EXP_TYPE_ROOT_PORT:
- if ((cap & cap_mask) != cap_mask)
- return -EINVAL;
- break;
- }
-
- /* Ensure upstream ports don't block AtomicOps on egress */
- if (pci_pcie_type(bridge) == PCI_EXP_TYPE_UPSTREAM) {
+ /* Upstream ports must not block AtomicOps on egress */
pcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2,
&ctl2);
if (ctl2 & PCI_EXP_DEVCTL2_ATOMIC_EGRESS_BLOCK)
return -EINVAL;
+ fallthrough;
+
+ /* All switch ports need to route AtomicOps */
+ case PCI_EXP_TYPE_DOWNSTREAM:
+ pcie_capability_read_dword(bridge, PCI_EXP_DEVCAP2,
+ &cap);
+ if (!(cap & PCI_EXP_DEVCAP2_ATOMIC_ROUTE))
+ return -EINVAL;
+ break;
}
- bus = bus->parent;
+ bridge = pci_upstream_bridge(bridge);
}
pcie_capability_set_word(dev, PCI_EXP_DEVCTL2,
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 9e71eb4d1010..d8c5a957b70e 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2373,7 +2373,6 @@ static void pci_release_dev(struct device *dev)
pci_release_of_node(pci_dev);
pcibios_release_device(pci_dev);
pci_bus_put(pci_dev->bus);
- kfree(pci_dev->driver_override);
bitmap_free(pci_dev->dma_alias_mask);
dev_dbg(dev, "device released\n");
kfree(pci_dev);
diff --git a/drivers/pcmcia/rsrc_nonstatic.c b/drivers/pcmcia/rsrc_nonstatic.c
index da494fe451ba..efc439c74886 100644
--- a/drivers/pcmcia/rsrc_nonstatic.c
+++ b/drivers/pcmcia/rsrc_nonstatic.c
@@ -188,7 +188,7 @@ static void do_io_probe(struct pcmcia_socket *s, unsigned int base,
int any;
u_char *b, hole, most;
- dev_info(&s->dev, "cs: IO port probe %#x-%#x:", base, base+num-1);
+ pr_info("%s: cs: IO port probe %#x-%#x:", dev_name(&s->dev), base, base+num-1);
/* First, what does a floating port look like? */
b = kzalloc(256, GFP_KERNEL);
@@ -410,8 +410,8 @@ static int do_mem_probe(struct pcmcia_socket *s, u_long base, u_long num,
struct socket_data *s_data = s->resource_data;
u_long i, j, bad, fail, step;
- dev_info(&s->dev, "cs: memory probe 0x%06lx-0x%06lx:",
- base, base+num-1);
+ pr_info("%s: cs: memory probe 0x%06lx-0x%06lx:",
+ dev_name(&s->dev), base, base+num-1);
bad = fail = 0;
step = (num < 0x20000) ? 0x2000 : ((num>>4) & ~0x1fff);
/* don't allow too large steps */
diff --git a/drivers/pinctrl/nomadik/pinctrl-abx500.c b/drivers/pinctrl/nomadik/pinctrl-abx500.c
index 68750b6f8e57..22ef6c2d16a6 100644
--- a/drivers/pinctrl/nomadik/pinctrl-abx500.c
+++ b/drivers/pinctrl/nomadik/pinctrl-abx500.c
@@ -855,7 +855,7 @@ static int abx500_pin_config_set(struct pinctrl_dev *pctldev,
int ret = -EINVAL;
int i;
enum pin_config_param param;
- enum pin_config_param argument;
+ unsigned int argument;
for (i = 0; i < num_configs; i++) {
param = pinconf_to_config_param(configs[i]);
diff --git a/drivers/pinctrl/pinctrl-cy8c95x0.c b/drivers/pinctrl/pinctrl-cy8c95x0.c
index 7a6a1434ae7f..c7948e2a0802 100644
--- a/drivers/pinctrl/pinctrl-cy8c95x0.c
+++ b/drivers/pinctrl/pinctrl-cy8c95x0.c
@@ -1345,6 +1345,7 @@ static int cy8c95x0_irq_setup(struct cy8c95x0_pinctrl *chip, int irq)
{
struct gpio_irq_chip *girq = &chip->gpio_chip.irq;
DECLARE_BITMAP(pending_irqs, MAX_LINE);
+ struct device *dev = chip->dev;
int ret;
mutex_init(&chip->irq_lock);
@@ -1353,10 +1354,8 @@ static int cy8c95x0_irq_setup(struct cy8c95x0_pinctrl *chip, int irq)
/* Read IRQ status register to clear all pending interrupts */
ret = cy8c95x0_irq_pending(chip, pending_irqs);
- if (ret) {
- dev_err(chip->dev, "failed to clear irq status register\n");
- return ret;
- }
+ if (ret)
+ return dev_err_probe(dev, -EBUSY, "failed to clear irq status register\n");
/* Mask all interrupts */
bitmap_fill(chip->irq_mask, MAX_LINE);
@@ -1371,17 +1370,9 @@ static int cy8c95x0_irq_setup(struct cy8c95x0_pinctrl *chip, int irq)
girq->handler = handle_simple_irq;
girq->threaded = true;
- ret = devm_request_threaded_irq(chip->dev, irq,
- NULL, cy8c95x0_irq_handler,
- IRQF_ONESHOT | IRQF_SHARED,
- dev_name(chip->dev), chip);
- if (ret) {
- dev_err(chip->dev, "failed to request irq %d\n", irq);
- return ret;
- }
- dev_info(chip->dev, "Registered threaded IRQ\n");
-
- return 0;
+ return devm_request_threaded_irq(dev, irq, NULL, cy8c95x0_irq_handler,
+ IRQF_ONESHOT | IRQF_SHARED,
+ dev_name(chip->dev), chip);
}
static int cy8c95x0_setup_pinctrl(struct cy8c95x0_pinctrl *chip)
@@ -1397,11 +1388,7 @@ static int cy8c95x0_setup_pinctrl(struct cy8c95x0_pinctrl *chip)
pd->owner = THIS_MODULE;
chip->pctldev = devm_pinctrl_register(chip->dev, pd, chip);
- if (IS_ERR(chip->pctldev))
- return dev_err_probe(chip->dev, PTR_ERR(chip->pctldev),
- "can't register controller\n");
-
- return 0;
+ return PTR_ERR_OR_ZERO(chip->pctldev);
}
static int cy8c95x0_detect(struct i2c_client *client,
diff --git a/drivers/pinctrl/pinctrl-pic32.c b/drivers/pinctrl/pinctrl-pic32.c
index bf827ab081a1..f87b5964e8b3 100644
--- a/drivers/pinctrl/pinctrl-pic32.c
+++ b/drivers/pinctrl/pinctrl-pic32.c
@@ -2173,16 +2173,10 @@ static int pic32_pinctrl_probe(struct platform_device *pdev)
if (IS_ERR(pctl->reg_base))
return PTR_ERR(pctl->reg_base);
- pctl->clk = devm_clk_get(&pdev->dev, NULL);
+ pctl->clk = devm_clk_get_enabled(&pdev->dev, NULL);
if (IS_ERR(pctl->clk)) {
ret = PTR_ERR(pctl->clk);
- dev_err(&pdev->dev, "clk get failed\n");
- return ret;
- }
-
- ret = clk_prepare_enable(pctl->clk);
- if (ret) {
- dev_err(&pdev->dev, "clk enable failed\n");
+ dev_err(&pdev->dev, "Failed to get and enable clock\n");
return ret;
}
@@ -2238,16 +2232,10 @@ static int pic32_gpio_probe(struct platform_device *pdev)
if (irq < 0)
return irq;
- bank->clk = devm_clk_get(&pdev->dev, NULL);
+ bank->clk = devm_clk_get_enabled(&pdev->dev, NULL);
if (IS_ERR(bank->clk)) {
ret = PTR_ERR(bank->clk);
- dev_err(&pdev->dev, "clk get failed\n");
- return ret;
- }
-
- ret = clk_prepare_enable(bank->clk);
- if (ret) {
- dev_err(&pdev->dev, "clk enable failed\n");
+ dev_err(&pdev->dev, "Failed to get and enable clock\n");
return ret;
}
diff --git a/drivers/pinctrl/realtek/pinctrl-rtd.c b/drivers/pinctrl/realtek/pinctrl-rtd.c
index 244060486332..4c876d1f6ad5 100644
--- a/drivers/pinctrl/realtek/pinctrl-rtd.c
+++ b/drivers/pinctrl/realtek/pinctrl-rtd.c
@@ -279,7 +279,7 @@ static const struct rtd_pin_sconfig_desc *rtd_pinctrl_find_sconfig(struct rtd_pi
static int rtd_pconf_parse_conf(struct rtd_pinctrl *data,
unsigned int pinnr,
enum pin_config_param param,
- enum pin_config_param arg)
+ unsigned int arg)
{
const struct rtd_pin_config_desc *config_desc;
const struct rtd_pin_sconfig_desc *sconfig_desc;
diff --git a/drivers/pinctrl/renesas/pinctrl-rzg2l.c b/drivers/pinctrl/renesas/pinctrl-rzg2l.c
index 8a7eb11df902..bcb0c39369e0 100644
--- a/drivers/pinctrl/renesas/pinctrl-rzg2l.c
+++ b/drivers/pinctrl/renesas/pinctrl-rzg2l.c
@@ -2812,6 +2812,13 @@ static void rzg2l_pinctrl_pm_setup_regs(struct rzg2l_pinctrl *pctrl, bool suspen
off = RZG2L_PIN_CFG_TO_PORT_OFFSET(cfg);
pincnt = hweight8(FIELD_GET(PIN_CFG_PIN_MAP_MASK, cfg));
+ if (cfg & RZG2L_VARIABLE_CFG) {
+ unsigned int pin = port * RZG2L_PINS_PER_PORT;
+
+ for (unsigned int i = 0; i < RZG2L_PINS_PER_PORT; i++)
+ cfg |= *(u64 *)pctrl->desc.pins[pin + i].drv_data;
+ }
+
caps = FIELD_GET(PIN_CFG_MASK, cfg);
has_iolh = !!(caps & (PIN_CFG_IOLH_A | PIN_CFG_IOLH_B | PIN_CFG_IOLH_C));
has_ien = !!(caps & PIN_CFG_IEN);
diff --git a/drivers/platform/chrome/chromeos_tbmc.c b/drivers/platform/chrome/chromeos_tbmc.c
index d1cf8f3463ce..e248567c0a18 100644
--- a/drivers/platform/chrome/chromeos_tbmc.c
+++ b/drivers/platform/chrome/chromeos_tbmc.c
@@ -95,6 +95,11 @@ static int chromeos_tbmc_add(struct acpi_device *adev)
return 0;
}
+static void chromeos_tbmc_remove(struct acpi_device *adev)
+{
+ device_init_wakeup(&adev->dev, false);
+}
+
static const struct acpi_device_id chromeos_tbmc_acpi_device_ids[] = {
{ ACPI_DRV_NAME, 0 },
{ }
@@ -110,6 +115,7 @@ static struct acpi_driver chromeos_tbmc_driver = {
.ids = chromeos_tbmc_acpi_device_ids,
.ops = {
.add = chromeos_tbmc_add,
+ .remove = chromeos_tbmc_remove,
.notify = chromeos_tbmc_notify,
},
.drv.pm = &chromeos_tbmc_pm_ops,
diff --git a/drivers/platform/surface/surfacepro3_button.c b/drivers/platform/surface/surfacepro3_button.c
index 2755601f979c..7c7622f8f871 100644
--- a/drivers/platform/surface/surfacepro3_button.c
+++ b/drivers/platform/surface/surfacepro3_button.c
@@ -243,6 +243,7 @@ static void surface_button_remove(struct acpi_device *device)
{
struct surface_button *button = acpi_driver_data(device);
+ device_init_wakeup(&device->dev, false);
input_unregister_device(button->input);
kfree(button);
}
diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c
index 92ce975d900d..5d701fde07df 100644
--- a/drivers/platform/x86/asus-wmi.c
+++ b/drivers/platform/x86/asus-wmi.c
@@ -127,7 +127,6 @@ module_param(fnlock_default, bool, 0444);
#define NVIDIA_TEMP_MIN 75
#define NVIDIA_TEMP_MAX 87
-#define ASUS_SCREENPAD_BRIGHT_MIN 20
#define ASUS_SCREENPAD_BRIGHT_MAX 255
#define ASUS_SCREENPAD_BRIGHT_DEFAULT 60
@@ -4119,43 +4118,35 @@ static int read_screenpad_brightness(struct backlight_device *bd)
return err;
/* The device brightness can only be read if powered, so return stored */
if (err == BACKLIGHT_POWER_OFF)
- return asus->driver->screenpad_brightness - ASUS_SCREENPAD_BRIGHT_MIN;
+ return bd->props.brightness;
err = asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_SCREENPAD_LIGHT, &retval);
if (err < 0)
return err;
- return (retval & ASUS_WMI_DSTS_BRIGHTNESS_MASK) - ASUS_SCREENPAD_BRIGHT_MIN;
+ return retval & ASUS_WMI_DSTS_BRIGHTNESS_MASK;
}
static int update_screenpad_bl_status(struct backlight_device *bd)
{
- struct asus_wmi *asus = bl_get_data(bd);
- int power, err = 0;
- u32 ctrl_param;
+ u32 ctrl_param = bd->props.brightness;
+ int err = 0;
- power = read_screenpad_backlight_power(asus);
- if (power < 0)
- return power;
+ if (bd->props.power) {
+ err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_POWER, 1, NULL);
+ if (err < 0)
+ return err;
- if (bd->props.power != power) {
- if (power != BACKLIGHT_POWER_ON) {
- /* Only brightness > 0 can power it back on */
- ctrl_param = asus->driver->screenpad_brightness - ASUS_SCREENPAD_BRIGHT_MIN;
- err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_LIGHT,
- ctrl_param, NULL);
- } else {
- err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_POWER, 0, NULL);
- }
- } else if (power == BACKLIGHT_POWER_ON) {
- /* Only set brightness if powered on or we get invalid/unsync state */
- ctrl_param = bd->props.brightness + ASUS_SCREENPAD_BRIGHT_MIN;
err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_LIGHT, ctrl_param, NULL);
+ if (err < 0)
+ return err;
}
- /* Ensure brightness is stored to turn back on with */
- if (err == 0)
- asus->driver->screenpad_brightness = bd->props.brightness + ASUS_SCREENPAD_BRIGHT_MIN;
+ if (!bd->props.power) {
+ err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_POWER, 0, NULL);
+ if (err < 0)
+ return err;
+ }
return err;
}
@@ -4173,22 +4164,19 @@ static int asus_screenpad_init(struct asus_wmi *asus)
int err, power;
int brightness = 0;
- power = read_screenpad_backlight_power(asus);
+ power = asus_wmi_get_devstate_simple(asus, ASUS_WMI_DEVID_SCREENPAD_POWER);
if (power < 0)
return power;
- if (power != BACKLIGHT_POWER_OFF) {
+ if (power) {
err = asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_SCREENPAD_LIGHT, &brightness);
if (err < 0)
return err;
}
- /* default to an acceptable min brightness on boot if too low */
- if (brightness < ASUS_SCREENPAD_BRIGHT_MIN)
- brightness = ASUS_SCREENPAD_BRIGHT_DEFAULT;
memset(&props, 0, sizeof(struct backlight_properties));
props.type = BACKLIGHT_RAW; /* ensure this bd is last to be picked */
- props.max_brightness = ASUS_SCREENPAD_BRIGHT_MAX - ASUS_SCREENPAD_BRIGHT_MIN;
+ props.max_brightness = ASUS_SCREENPAD_BRIGHT_MAX;
bd = backlight_device_register("asus_screenpad",
&asus->platform_device->dev, asus,
&asus_screenpad_bl_ops, &props);
@@ -4199,7 +4187,7 @@ static int asus_screenpad_init(struct asus_wmi *asus)
asus->screenpad_backlight_device = bd;
asus->driver->screenpad_brightness = brightness;
- bd->props.brightness = brightness - ASUS_SCREENPAD_BRIGHT_MIN;
+ bd->props.brightness = brightness;
bd->props.power = power;
backlight_update_status(bd);
diff --git a/drivers/platform/x86/dell/dell-wmi-sysman/enum-attributes.c b/drivers/platform/x86/dell/dell-wmi-sysman/enum-attributes.c
index fc2f58b4cbc6..7e44ba301562 100644
--- a/drivers/platform/x86/dell/dell-wmi-sysman/enum-attributes.c
+++ b/drivers/platform/x86/dell/dell-wmi-sysman/enum-attributes.c
@@ -6,10 +6,32 @@
* Copyright (c) 2020 Dell Inc.
*/
+#include <linux/bug.h>
+
#include "dell-wmi-sysman.h"
get_instance_id(enumeration);
+static int append_enum_string(char *dest, const char *src)
+{
+ size_t dest_len = strlen(dest);
+ ssize_t copied;
+
+ if (WARN_ON_ONCE(dest_len >= MAX_BUFF))
+ return -EINVAL;
+
+ copied = strscpy(dest + dest_len, src, MAX_BUFF - dest_len);
+ if (copied < 0)
+ return -EINVAL;
+
+ dest_len += copied;
+ copied = strscpy(dest + dest_len, ";", MAX_BUFF - dest_len);
+ if (copied < 0)
+ return -EINVAL;
+
+ return 0;
+}
+
static ssize_t current_value_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf)
{
int instance_id = get_enumeration_instance_id(kobj);
@@ -176,9 +198,9 @@ int populate_enum_data(union acpi_object *enumeration_obj, int instance_id,
return -EINVAL;
if (check_property_type(enumeration, next_obj, ACPI_TYPE_STRING))
return -EINVAL;
- strcat(wmi_priv.enumeration_data[instance_id].dell_value_modifier,
- enumeration_obj[next_obj++].string.pointer);
- strcat(wmi_priv.enumeration_data[instance_id].dell_value_modifier, ";");
+ if (append_enum_string(wmi_priv.enumeration_data[instance_id].dell_value_modifier,
+ enumeration_obj[next_obj++].string.pointer))
+ return -EINVAL;
}
if (next_obj >= enum_property_count)
@@ -193,9 +215,9 @@ int populate_enum_data(union acpi_object *enumeration_obj, int instance_id,
return -EINVAL;
if (check_property_type(enumeration, next_obj, ACPI_TYPE_STRING))
return -EINVAL;
- strcat(wmi_priv.enumeration_data[instance_id].possible_values,
- enumeration_obj[next_obj++].string.pointer);
- strcat(wmi_priv.enumeration_data[instance_id].possible_values, ";");
+ if (append_enum_string(wmi_priv.enumeration_data[instance_id].possible_values,
+ enumeration_obj[next_obj++].string.pointer))
+ return -EINVAL;
}
return sysfs_create_group(attr_name_kobj, &enumeration_attr_group);
diff --git a/drivers/platform/x86/dell/dell_rbu.c b/drivers/platform/x86/dell/dell_rbu.c
index fee20866b41e..9039e494131f 100644
--- a/drivers/platform/x86/dell/dell_rbu.c
+++ b/drivers/platform/x86/dell/dell_rbu.c
@@ -30,6 +30,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/init.h>
+#include <linux/kstrtox.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/string.h>
@@ -617,9 +618,12 @@ static ssize_t packet_size_write(struct file *filp, struct kobject *kobj,
char *buffer, loff_t pos, size_t count)
{
unsigned long temp;
+
+ if (kstrtoul(buffer, 10, &temp))
+ return -EINVAL;
+
spin_lock(&rbu_data.lock);
packet_empty_list();
- sscanf(buffer, "%lu", &temp);
if (temp < 0xffffffff)
rbu_data.packetsize = temp;
diff --git a/drivers/platform/x86/panasonic-laptop.c b/drivers/platform/x86/panasonic-laptop.c
index 851f0f92219d..1b544953858c 100644
--- a/drivers/platform/x86/panasonic-laptop.c
+++ b/drivers/platform/x86/panasonic-laptop.c
@@ -1093,9 +1093,10 @@ static int acpi_pcc_hotkey_add(struct acpi_device *device)
}
result = device_create_file(&pcc->platform->dev,
&dev_attr_cdpower);
- pcc_register_optd_notifier(pcc, "\\_SB.PCI0.EHCI.ERHB.OPTD");
if (result)
goto out_platform;
+
+ pcc_register_optd_notifier(pcc, "\\_SB.PCI0.EHCI.ERHB.OPTD");
} else {
pcc->platform = NULL;
}
@@ -1129,10 +1130,10 @@ static void acpi_pcc_hotkey_remove(struct acpi_device *device)
i8042_remove_filter(panasonic_i8042_filter);
if (pcc->platform) {
+ pcc_unregister_optd_notifier(pcc, "\\_SB.PCI0.EHCI.ERHB.OPTD");
device_remove_file(&pcc->platform->dev, &dev_attr_cdpower);
platform_device_unregister(pcc->platform);
}
- pcc_unregister_optd_notifier(pcc, "\\_SB.PCI0.EHCI.ERHB.OPTD");
sysfs_remove_group(&device->dev.kobj, &pcc_attr_group);
diff --git a/drivers/platform/x86/wmi.c b/drivers/platform/x86/wmi.c
index 3cbe180c3fc0..f13173eb070e 100644
--- a/drivers/platform/x86/wmi.c
+++ b/drivers/platform/x86/wmi.c
@@ -696,39 +696,11 @@ static ssize_t expensive_show(struct device *dev,
}
static DEVICE_ATTR_RO(expensive);
-static ssize_t driver_override_show(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- struct wmi_device *wdev = to_wmi_device(dev);
- ssize_t ret;
-
- device_lock(dev);
- ret = sysfs_emit(buf, "%s\n", wdev->driver_override);
- device_unlock(dev);
-
- return ret;
-}
-
-static ssize_t driver_override_store(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct wmi_device *wdev = to_wmi_device(dev);
- int ret;
-
- ret = driver_set_override(dev, &wdev->driver_override, buf, count);
- if (ret < 0)
- return ret;
-
- return count;
-}
-static DEVICE_ATTR_RW(driver_override);
-
static struct attribute *wmi_attrs[] = {
&dev_attr_modalias.attr,
&dev_attr_guid.attr,
&dev_attr_instance_count.attr,
&dev_attr_expensive.attr,
- &dev_attr_driver_override.attr,
NULL
};
ATTRIBUTE_GROUPS(wmi);
@@ -797,7 +769,6 @@ static void wmi_dev_release(struct device *dev)
{
struct wmi_block *wblock = dev_to_wblock(dev);
- kfree(wblock->dev.driver_override);
kfree(wblock);
}
@@ -806,10 +777,12 @@ static int wmi_dev_match(struct device *dev, const struct device_driver *driver)
const struct wmi_driver *wmi_driver = drv_to_wdrv(driver);
struct wmi_block *wblock = dev_to_wblock(dev);
const struct wmi_device_id *id = wmi_driver->id_table;
+ int ret;
/* When driver_override is set, only bind to the matching driver */
- if (wblock->dev.driver_override)
- return !strcmp(wblock->dev.driver_override, driver->name);
+ ret = device_match_driver_override(dev, driver);
+ if (ret >= 0)
+ return ret;
if (id == NULL)
return 0;
@@ -891,6 +864,7 @@ static struct class wmi_bus_class = {
static const struct bus_type wmi_bus_type = {
.name = "wmi",
.dev_groups = wmi_groups,
+ .driver_override = true,
.match = wmi_dev_match,
.uevent = wmi_dev_uevent,
.probe = wmi_dev_probe,
diff --git a/drivers/pmdomain/imx/scu-pd.c b/drivers/pmdomain/imx/scu-pd.c
index 01d465d88f60..3ec33667a308 100644
--- a/drivers/pmdomain/imx/scu-pd.c
+++ b/drivers/pmdomain/imx/scu-pd.c
@@ -326,6 +326,7 @@ static void imx_sc_pd_get_console_rsrc(void)
return;
imx_con_rsrc = specs.args[0];
+ of_node_put(specs.np);
}
static int imx_sc_get_pd_power(struct device *dev, u32 rsrc)
diff --git a/drivers/pmdomain/ti/omap_prm.c b/drivers/pmdomain/ti/omap_prm.c
index b8ceb3c2b81c..f4e52e92dcbf 100644
--- a/drivers/pmdomain/ti/omap_prm.c
+++ b/drivers/pmdomain/ti/omap_prm.c
@@ -651,6 +651,7 @@ static int omap_prm_domain_attach_dev(struct generic_pm_domain *domain,
if (pd_args.args_count != 0)
dev_warn(dev, "%s: unusupported #power-domain-cells: %i\n",
prmd->pd.name, pd_args.args_count);
+ of_node_put(pd_args.np);
genpd_data = dev_gpd_data(dev);
genpd_data->data = NULL;
diff --git a/drivers/pwm/pwm-atmel-tcb.c b/drivers/pwm/pwm-atmel-tcb.c
index 5ee4254d1e48..9554deb413ab 100644
--- a/drivers/pwm/pwm-atmel-tcb.c
+++ b/drivers/pwm/pwm-atmel-tcb.c
@@ -50,6 +50,8 @@ struct atmel_tcb_pwm_chip {
spinlock_t lock;
u8 channel;
u8 width;
+ unsigned long rate;
+ unsigned long slow_rate;
struct regmap *regmap;
struct clk *clk;
struct clk *gclk;
@@ -266,7 +268,7 @@ static int atmel_tcb_pwm_config(struct pwm_chip *chip, struct pwm_device *pwm,
int slowclk = 0;
unsigned period;
unsigned duty;
- unsigned rate = clk_get_rate(tcbpwmc->clk);
+ unsigned long rate = tcbpwmc->rate;
unsigned long long min;
unsigned long long max;
@@ -294,7 +296,7 @@ static int atmel_tcb_pwm_config(struct pwm_chip *chip, struct pwm_device *pwm,
*/
if (i == ARRAY_SIZE(atmel_tcb_divisors)) {
i = slowclk;
- rate = clk_get_rate(tcbpwmc->slow_clk);
+ rate = tcbpwmc->slow_rate;
min = div_u64(NSEC_PER_SEC, rate);
max = min << tcbpwmc->width;
@@ -431,24 +433,49 @@ static int atmel_tcb_pwm_probe(struct platform_device *pdev)
}
chip->ops = &atmel_tcb_pwm_ops;
+ chip->atomic = true;
tcbpwmc->channel = channel;
tcbpwmc->width = config->counter_width;
- err = clk_prepare_enable(tcbpwmc->slow_clk);
+ err = clk_prepare_enable(tcbpwmc->clk);
if (err)
goto err_gclk;
+ err = clk_prepare_enable(tcbpwmc->slow_clk);
+ if (err)
+ goto err_disable_clk;;
+
+ err = clk_rate_exclusive_get(tcbpwmc->clk);
+ if (err)
+ goto err_disable_slow_clk;
+
+ err = clk_rate_exclusive_get(tcbpwmc->slow_clk);
+ if (err)
+ goto err_clk_unlock;
+
+ tcbpwmc->rate = clk_get_rate(tcbpwmc->clk);
+ tcbpwmc->slow_rate = clk_get_rate(tcbpwmc->slow_clk);
+
spin_lock_init(&tcbpwmc->lock);
err = pwmchip_add(chip);
if (err < 0)
- goto err_disable_clk;
+ goto err_slow_clk_unlock;
platform_set_drvdata(pdev, chip);
return 0;
+err_slow_clk_unlock:
+ clk_rate_exclusive_put(tcbpwmc->slow_clk);
+
+err_clk_unlock:
+ clk_rate_exclusive_put(tcbpwmc->clk);
+
err_disable_clk:
+ clk_disable_unprepare(tcbpwmc->clk);
+
+err_disable_slow_clk:
clk_disable_unprepare(tcbpwmc->slow_clk);
err_gclk:
@@ -470,6 +497,9 @@ static void atmel_tcb_pwm_remove(struct platform_device *pdev)
pwmchip_remove(chip);
+ clk_rate_exclusive_put(tcbpwmc->slow_clk);
+ clk_rate_exclusive_put(tcbpwmc->clk);
+ clk_disable_unprepare(tcbpwmc->clk);
clk_disable_unprepare(tcbpwmc->slow_clk);
clk_put(tcbpwmc->gclk);
clk_put(tcbpwmc->clk);
diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
index 6a64e5909f6a..d1c069704da8 100644
--- a/drivers/remoteproc/xlnx_r5_remoteproc.c
+++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
@@ -990,7 +990,7 @@ static int zynqmp_r5_get_sram_banks(struct zynqmp_r5_core *r5_core)
}
/* Get SRAM device address */
- ret = of_property_read_reg(sram_np, i, &abs_addr, &size);
+ ret = of_property_read_reg(sram_np, 0, &abs_addr, &size);
if (ret) {
dev_err(dev, "failed to get reg property\n");
goto fail_sram_get;
diff --git a/drivers/reset/core.c b/drivers/reset/core.c
index 4d509d41456a..22f67fc77ae5 100644
--- a/drivers/reset/core.c
+++ b/drivers/reset/core.c
@@ -773,12 +773,19 @@ EXPORT_SYMBOL_GPL(reset_control_bulk_release);
static struct reset_control *
__reset_control_get_internal(struct reset_controller_dev *rcdev,
- unsigned int index, bool shared, bool acquired)
+ unsigned int index, enum reset_control_flags flags)
{
+ bool shared = flags & RESET_CONTROL_FLAGS_BIT_SHARED;
+ bool acquired = flags & RESET_CONTROL_FLAGS_BIT_ACQUIRED;
struct reset_control *rstc;
lockdep_assert_held(&reset_list_mutex);
+ /* Expect callers to filter out OPTIONAL and DEASSERTED bits */
+ if (WARN_ON(flags & ~(RESET_CONTROL_FLAGS_BIT_SHARED |
+ RESET_CONTROL_FLAGS_BIT_ACQUIRED)))
+ return ERR_PTR(-EINVAL);
+
list_for_each_entry(rstc, &rcdev->reset_control_head, list) {
if (rstc->id == index) {
/*
@@ -994,8 +1001,9 @@ static struct reset_controller_dev *__reset_find_rcdev(const struct of_phandle_a
struct reset_control *
__of_reset_control_get(struct device_node *node, const char *id, int index,
- bool shared, bool optional, bool acquired)
+ enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
bool gpio_fallback = false;
struct reset_control *rstc;
struct reset_controller_dev *rcdev;
@@ -1059,8 +1067,10 @@ __of_reset_control_get(struct device_node *node, const char *id, int index,
goto out_unlock;
}
+ flags &= ~RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
/* reset_list_mutex also protects the rcdev's reset_control list */
- rstc = __reset_control_get_internal(rcdev, rstc_id, shared, acquired);
+ rstc = __reset_control_get_internal(rcdev, rstc_id, flags);
out_unlock:
mutex_unlock(&reset_list_mutex);
@@ -1091,8 +1101,9 @@ __reset_controller_by_name(const char *name)
static struct reset_control *
__reset_control_get_from_lookup(struct device *dev, const char *con_id,
- bool shared, bool optional, bool acquired)
+ enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
const struct reset_control_lookup *lookup;
struct reset_controller_dev *rcdev;
const char *dev_id = dev_name(dev);
@@ -1116,9 +1127,11 @@ __reset_control_get_from_lookup(struct device *dev, const char *con_id,
return ERR_PTR(-EPROBE_DEFER);
}
+ flags &= ~RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
rstc = __reset_control_get_internal(rcdev,
lookup->index,
- shared, acquired);
+ flags);
mutex_unlock(&reset_list_mutex);
break;
}
@@ -1133,30 +1146,29 @@ __reset_control_get_from_lookup(struct device *dev, const char *con_id,
}
struct reset_control *__reset_control_get(struct device *dev, const char *id,
- int index, bool shared, bool optional,
- bool acquired)
+ int index, enum reset_control_flags flags)
{
+ bool shared = flags & RESET_CONTROL_FLAGS_BIT_SHARED;
+ bool acquired = flags & RESET_CONTROL_FLAGS_BIT_ACQUIRED;
+
if (WARN_ON(shared && acquired))
return ERR_PTR(-EINVAL);
if (dev->of_node)
- return __of_reset_control_get(dev->of_node, id, index, shared,
- optional, acquired);
+ return __of_reset_control_get(dev->of_node, id, index, flags);
- return __reset_control_get_from_lookup(dev, id, shared, optional,
- acquired);
+ return __reset_control_get_from_lookup(dev, id, flags);
}
EXPORT_SYMBOL_GPL(__reset_control_get);
int __reset_control_bulk_get(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs,
- bool shared, bool optional, bool acquired)
+ enum reset_control_flags flags)
{
int ret, i;
for (i = 0; i < num_rstcs; i++) {
- rstcs[i].rstc = __reset_control_get(dev, rstcs[i].id, 0,
- shared, optional, acquired);
+ rstcs[i].rstc = __reset_control_get(dev, rstcs[i].id, 0, flags);
if (IS_ERR(rstcs[i].rstc)) {
ret = PTR_ERR(rstcs[i].rstc);
goto err;
@@ -1224,23 +1236,46 @@ static void devm_reset_control_release(struct device *dev, void *res)
reset_control_put(*(struct reset_control **)res);
}
+static void devm_reset_control_release_deasserted(struct device *dev, void *res)
+{
+ struct reset_control *rstc = *(struct reset_control **)res;
+
+ reset_control_assert(rstc);
+ reset_control_put(rstc);
+}
+
struct reset_control *
__devm_reset_control_get(struct device *dev, const char *id, int index,
- bool shared, bool optional, bool acquired)
+ enum reset_control_flags flags)
{
struct reset_control **ptr, *rstc;
+ bool deasserted = flags & RESET_CONTROL_FLAGS_BIT_DEASSERTED;
- ptr = devres_alloc(devm_reset_control_release, sizeof(*ptr),
+ ptr = devres_alloc(deasserted ? devm_reset_control_release_deasserted :
+ devm_reset_control_release, sizeof(*ptr),
GFP_KERNEL);
if (!ptr)
return ERR_PTR(-ENOMEM);
- rstc = __reset_control_get(dev, id, index, shared, optional, acquired);
+ flags &= ~RESET_CONTROL_FLAGS_BIT_DEASSERTED;
+
+ rstc = __reset_control_get(dev, id, index, flags);
if (IS_ERR_OR_NULL(rstc)) {
devres_free(ptr);
return rstc;
}
+ if (deasserted) {
+ int ret;
+
+ ret = reset_control_deassert(rstc);
+ if (ret) {
+ reset_control_put(rstc);
+ devres_free(ptr);
+ return ERR_PTR(ret);
+ }
+ }
+
*ptr = rstc;
devres_add(dev, ptr);
@@ -1260,24 +1295,45 @@ static void devm_reset_control_bulk_release(struct device *dev, void *res)
reset_control_bulk_put(devres->num_rstcs, devres->rstcs);
}
+static void devm_reset_control_bulk_release_deasserted(struct device *dev, void *res)
+{
+ struct reset_control_bulk_devres *devres = res;
+
+ reset_control_bulk_assert(devres->num_rstcs, devres->rstcs);
+ reset_control_bulk_put(devres->num_rstcs, devres->rstcs);
+}
+
int __devm_reset_control_bulk_get(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs,
- bool shared, bool optional, bool acquired)
+ enum reset_control_flags flags)
{
struct reset_control_bulk_devres *ptr;
+ bool deasserted = flags & RESET_CONTROL_FLAGS_BIT_DEASSERTED;
int ret;
- ptr = devres_alloc(devm_reset_control_bulk_release, sizeof(*ptr),
+ ptr = devres_alloc(deasserted ? devm_reset_control_bulk_release_deasserted :
+ devm_reset_control_bulk_release, sizeof(*ptr),
GFP_KERNEL);
if (!ptr)
return -ENOMEM;
- ret = __reset_control_bulk_get(dev, num_rstcs, rstcs, shared, optional, acquired);
+ flags &= ~RESET_CONTROL_FLAGS_BIT_DEASSERTED;
+
+ ret = __reset_control_bulk_get(dev, num_rstcs, rstcs, flags);
if (ret < 0) {
devres_free(ptr);
return ret;
}
+ if (deasserted) {
+ ret = reset_control_bulk_deassert(num_rstcs, rstcs);
+ if (ret) {
+ reset_control_bulk_put(num_rstcs, rstcs);
+ devres_free(ptr);
+ return ret;
+ }
+ }
+
ptr->num_rstcs = num_rstcs;
ptr->rstcs = rstcs;
devres_add(dev, ptr);
@@ -1298,6 +1354,7 @@ EXPORT_SYMBOL_GPL(__devm_reset_control_bulk_get);
*/
int __device_reset(struct device *dev, bool optional)
{
+ enum reset_control_flags flags;
struct reset_control *rstc;
int ret;
@@ -1313,7 +1370,8 @@ int __device_reset(struct device *dev, bool optional)
}
#endif
- rstc = __reset_control_get(dev, NULL, 0, 0, optional, true);
+ flags = optional ? RESET_CONTROL_OPTIONAL_EXCLUSIVE : RESET_CONTROL_EXCLUSIVE;
+ rstc = __reset_control_get(dev, NULL, 0, flags);
if (IS_ERR(rstc))
return PTR_ERR(rstc);
@@ -1356,17 +1414,14 @@ static int of_reset_control_get_count(struct device_node *node)
* device node.
*
* @np: device node for the device that requests the reset controls array
- * @shared: whether reset controls are shared or not
- * @optional: whether it is optional to get the reset controls
- * @acquired: only one reset control may be acquired for a given controller
- * and ID
+ * @flags: whether reset controls are shared, optional, acquired
*
* Returns pointer to allocated reset_control on success or error on failure
*/
struct reset_control *
-of_reset_control_array_get(struct device_node *np, bool shared, bool optional,
- bool acquired)
+of_reset_control_array_get(struct device_node *np, enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
struct reset_control_array *resets;
struct reset_control *rstc;
int num, i;
@@ -1381,8 +1436,7 @@ of_reset_control_array_get(struct device_node *np, bool shared, bool optional,
resets->num_rstcs = num;
for (i = 0; i < num; i++) {
- rstc = __of_reset_control_get(np, NULL, i, shared, optional,
- acquired);
+ rstc = __of_reset_control_get(np, NULL, i, flags);
if (IS_ERR(rstc))
goto err_rst;
resets->rstc[i] = rstc;
@@ -1407,8 +1461,7 @@ EXPORT_SYMBOL_GPL(of_reset_control_array_get);
* devm_reset_control_array_get - Resource managed reset control array get
*
* @dev: device that requests the list of reset controls
- * @shared: whether reset controls are shared or not
- * @optional: whether it is optional to get the reset controls
+ * @flags: whether reset controls are shared, optional, acquired
*
* The reset control array APIs are intended for a list of resets
* that just have to be asserted or deasserted, without any
@@ -1417,7 +1470,7 @@ EXPORT_SYMBOL_GPL(of_reset_control_array_get);
* Returns pointer to allocated reset_control on success or error on failure
*/
struct reset_control *
-devm_reset_control_array_get(struct device *dev, bool shared, bool optional)
+devm_reset_control_array_get(struct device *dev, enum reset_control_flags flags)
{
struct reset_control **ptr, *rstc;
@@ -1426,7 +1479,7 @@ devm_reset_control_array_get(struct device *dev, bool shared, bool optional)
if (!ptr)
return ERR_PTR(-ENOMEM);
- rstc = of_reset_control_array_get(dev->of_node, shared, optional, true);
+ rstc = of_reset_control_array_get(dev->of_node, flags);
if (IS_ERR_OR_NULL(rstc)) {
devres_free(ptr);
return rstc;
diff --git a/drivers/rtc/rtc-abx80x.c b/drivers/rtc/rtc-abx80x.c
index 3fee27914ba8..5f3a3e60a19d 100644
--- a/drivers/rtc/rtc-abx80x.c
+++ b/drivers/rtc/rtc-abx80x.c
@@ -933,6 +933,8 @@ static int abx80x_probe(struct i2c_client *client)
client->irq = 0;
}
}
+ if (client->irq <= 0)
+ clear_bit(RTC_FEATURE_ALARM, priv->rtc->features);
err = rtc_add_group(priv->rtc, &rtc_calib_attr_group);
if (err) {
diff --git a/drivers/s390/cio/cio.h b/drivers/s390/cio/cio.h
index a9057a5b670a..23374604c6b0 100644
--- a/drivers/s390/cio/cio.h
+++ b/drivers/s390/cio/cio.h
@@ -103,11 +103,6 @@ struct subchannel {
struct work_struct todo_work;
struct schib_config config;
u64 dma_mask;
- /*
- * Driver name to force a match. Do not set directly, because core
- * frees it. Use driver_set_override() to set or clear it.
- */
- const char *driver_override;
} __attribute__ ((aligned(8)));
DECLARE_PER_CPU_ALIGNED(struct irb, cio_irb);
diff --git a/drivers/s390/cio/css.c b/drivers/s390/cio/css.c
index 61be7c0550bc..b0ddbad1ebf3 100644
--- a/drivers/s390/cio/css.c
+++ b/drivers/s390/cio/css.c
@@ -160,7 +160,6 @@ static void css_subchannel_release(struct device *dev)
sch->config.intparm = 0;
cio_commit_config(sch);
- kfree(sch->driver_override);
kfree(sch);
}
@@ -324,37 +323,9 @@ static ssize_t modalias_show(struct device *dev, struct device_attribute *attr,
static DEVICE_ATTR_RO(modalias);
-static ssize_t driver_override_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct subchannel *sch = to_subchannel(dev);
- int ret;
-
- ret = driver_set_override(dev, &sch->driver_override, buf, count);
- if (ret)
- return ret;
-
- return count;
-}
-
-static ssize_t driver_override_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct subchannel *sch = to_subchannel(dev);
- ssize_t len;
-
- device_lock(dev);
- len = sysfs_emit(buf, "%s\n", sch->driver_override);
- device_unlock(dev);
- return len;
-}
-static DEVICE_ATTR_RW(driver_override);
-
static struct attribute *subch_attrs[] = {
&dev_attr_type.attr,
&dev_attr_modalias.attr,
- &dev_attr_driver_override.attr,
NULL,
};
@@ -1358,9 +1329,11 @@ static int css_bus_match(struct device *dev, const struct device_driver *drv)
struct subchannel *sch = to_subchannel(dev);
const struct css_driver *driver = to_cssdriver(drv);
struct css_device_id *id;
+ int ret;
/* When driver_override is set, only bind to the matching driver */
- if (sch->driver_override && strcmp(sch->driver_override, drv->name))
+ ret = device_match_driver_override(dev, drv);
+ if (ret == 0)
return 0;
for (id = driver->subchannel_type; id->match_flags; id++) {
@@ -1417,6 +1390,7 @@ static int css_uevent(const struct device *dev, struct kobj_uevent_env *env)
static const struct bus_type css_bus_type = {
.name = "css",
+ .driver_override = true,
.match = css_bus_match,
.probe = css_probe,
.remove = css_remove,
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 53dd46150849..f85a52e9a7a7 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1622,10 +1622,35 @@ sg_remove_device(struct device *cl_dev)
}
module_param_named(scatter_elem_sz, scatter_elem_sz, int, S_IRUGO | S_IWUSR);
-module_param_named(def_reserved_size, def_reserved_size, int,
- S_IRUGO | S_IWUSR);
module_param_named(allow_dio, sg_allow_dio, int, S_IRUGO | S_IWUSR);
+static int def_reserved_size_set(const char *val, const struct kernel_param *kp)
+{
+ int size, ret;
+
+ if (!val)
+ return -EINVAL;
+
+ ret = kstrtoint(val, 0, &size);
+ if (ret)
+ return ret;
+
+ /* limit to 1 MB */
+ if (size < 0 || size > 1048576)
+ return -ERANGE;
+
+ def_reserved_size = size;
+ return 0;
+}
+
+static const struct kernel_param_ops def_reserved_size_ops = {
+ .set = def_reserved_size_set,
+ .get = param_get_int,
+};
+
+module_param_cb(def_reserved_size, &def_reserved_size_ops, &def_reserved_size,
+ S_IRUGO | S_IWUSR);
+
MODULE_AUTHOR("Douglas Gilbert");
MODULE_DESCRIPTION("SCSI generic (sg) driver");
MODULE_LICENSE("GPL");
@@ -1691,13 +1716,13 @@ init_sg(void)
sg_sysfs_valid = 1;
rc = scsi_register_interface(&sg_interface);
if (0 == rc) {
+ register_sg_sysctls();
#ifdef CONFIG_SCSI_PROC_FS
sg_proc_init();
#endif /* CONFIG_SCSI_PROC_FS */
return 0;
}
class_unregister(&sg_sysfs_class);
- register_sg_sysctls();
err_out:
unregister_chrdev_region(MKDEV(SCSI_GENERIC_MAJOR, 0), SG_MAX_DEVS);
return rc;
diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c
index add13e306898..803fc9c13229 100644
--- a/drivers/scsi/sr.c
+++ b/drivers/scsi/sr.c
@@ -395,7 +395,7 @@ static blk_status_t sr_init_command(struct scsi_cmnd *SCpnt)
switch (req_op(rq)) {
case REQ_OP_WRITE:
- if (!cd->writeable)
+ if (get_disk_ro(cd->disk))
goto out;
SCpnt->cmnd[0] = WRITE_10;
cd->cdi.media_written = 1;
@@ -681,6 +681,7 @@ static int sr_probe(struct device *dev)
error = -ENOMEM;
if (get_capabilities(cd))
goto fail_minor;
+ cdrom_probe_write_features(&cd->cdi);
sr_vendor_init(cd);
set_capacity(disk, cd->capacity);
@@ -899,14 +900,6 @@ static int get_capabilities(struct scsi_cd *cd)
/*else I don't think it can close its tray
cd->cdi.mask |= CDC_CLOSE_TRAY; */
- /*
- * if DVD-RAM, MRW-W or CD-RW, we are randomly writable
- */
- if ((cd->cdi.mask & (CDC_DVD_RAM | CDC_MRW_W | CDC_RAM | CDC_CD_RW)) !=
- (CDC_DVD_RAM | CDC_MRW_W | CDC_RAM | CDC_CD_RW)) {
- cd->writeable = 1;
- }
-
kfree(buffer);
return 0;
}
diff --git a/drivers/scsi/sr.h b/drivers/scsi/sr.h
index dc899277b3a4..2d92f9cb6fec 100644
--- a/drivers/scsi/sr.h
+++ b/drivers/scsi/sr.h
@@ -35,7 +35,6 @@ typedef struct scsi_cd {
struct scsi_device *device;
unsigned int vendor; /* vendor code, see sr_vendor.c */
unsigned long ms_offset; /* for reading multisession-CD's */
- unsigned writeable : 1;
unsigned use:1; /* is this device still supportable */
unsigned xa_flag:1; /* CD has XA sectors ? */
unsigned readcd_known:1; /* drive supports READ_CD (0xbe) */
diff --git a/drivers/soc/qcom/llcc-qcom.c b/drivers/soc/qcom/llcc-qcom.c
index 0278e1854af0..5de1a72c077c 100644
--- a/drivers/soc/qcom/llcc-qcom.c
+++ b/drivers/soc/qcom/llcc-qcom.c
@@ -2584,7 +2584,7 @@ static const struct llcc_slice_config x1e80100_data[] = {
static const struct llcc_edac_reg_offset llcc_v1_edac_reg_offset = {
.trp_ecc_error_status0 = 0x20344,
.trp_ecc_error_status1 = 0x20348,
- .trp_ecc_sb_err_syn0 = 0x2304c,
+ .trp_ecc_sb_err_syn0 = 0x2034c,
.trp_ecc_db_err_syn0 = 0x20370,
.trp_ecc_error_cntr_clear = 0x20440,
.trp_interrupt_0_status = 0x20480,
diff --git a/drivers/soc/qcom/ocmem.c b/drivers/soc/qcom/ocmem.c
index 71130a2f62e9..37ea6b86aebc 100644
--- a/drivers/soc/qcom/ocmem.c
+++ b/drivers/soc/qcom/ocmem.c
@@ -196,17 +196,16 @@ struct ocmem *of_get_ocmem(struct device *dev)
}
pdev = of_find_device_by_node(devnode->parent);
- if (!pdev) {
- dev_err(dev, "Cannot find device node %s\n", devnode->name);
- return ERR_PTR(-EPROBE_DEFER);
- }
+ if (!pdev)
+ return dev_err_ptr_probe(dev, -EPROBE_DEFER,
+ "Cannot find device node %s\n",
+ devnode->name);
ocmem = platform_get_drvdata(pdev);
put_device(&pdev->dev);
- if (!ocmem) {
- dev_err(dev, "Cannot get ocmem\n");
- return ERR_PTR(-ENODEV);
- }
+ if (!ocmem)
+ return dev_err_ptr_probe(dev, -EPROBE_DEFER, "Cannot get ocmem\n");
+
return ocmem;
}
EXPORT_SYMBOL_GPL(of_get_ocmem);
@@ -308,7 +307,7 @@ static int ocmem_dev_probe(struct platform_device *pdev)
ocmem->dev = dev;
ocmem->config = device_get_match_data(dev);
- ocmem->core_clk = devm_clk_get(dev, "core");
+ ocmem->core_clk = devm_clk_get_optional(dev, "core");
if (IS_ERR(ocmem->core_clk))
return dev_err_probe(dev, PTR_ERR(ocmem->core_clk),
"Unable to get core clock\n");
diff --git a/drivers/soc/qcom/qcom_aoss.c b/drivers/soc/qcom/qcom_aoss.c
index 0320ad3b9148..d7cd449c57b6 100644
--- a/drivers/soc/qcom/qcom_aoss.c
+++ b/drivers/soc/qcom/qcom_aoss.c
@@ -354,7 +354,7 @@ static int qmp_cdev_set_cur_state(struct thermal_cooling_device *cdev,
/* Normalize state */
cdev_state = !!state;
- if (qmp_cdev->state == state)
+ if (qmp_cdev->state == cdev_state)
return 0;
ret = qmp_send(qmp_cdev->qmp, "{class: volt_flr, event:zero_temp, res:%s, value:%s}",
diff --git a/drivers/soc/tegra/cbb/tegra234-cbb.c b/drivers/soc/tegra/cbb/tegra234-cbb.c
index e8cc46874c72..eace89ed1617 100644
--- a/drivers/soc/tegra/cbb/tegra234-cbb.c
+++ b/drivers/soc/tegra/cbb/tegra234-cbb.c
@@ -1176,6 +1176,10 @@ static int __maybe_unused tegra234_cbb_resume_noirq(struct device *dev)
{
struct tegra234_cbb *cbb = dev_get_drvdata(dev);
+ /* set ERD bit to mask SError and generate interrupt to report error */
+ if (cbb->fabric->off_mask_erd)
+ tegra234_cbb_mask_serror(cbb);
+
tegra234_cbb_error_enable(&cbb->base);
dev_dbg(dev, "%s resumed\n", cbb->fabric->name);
diff --git a/drivers/soundwire/bus.c b/drivers/soundwire/bus.c
index 6c7989e2079e..25efafd89c03 100644
--- a/drivers/soundwire/bus.c
+++ b/drivers/soundwire/bus.c
@@ -1866,8 +1866,8 @@ int sdw_handle_slave_status(struct sdw_bus *bus,
if (status[i] == SDW_SLAVE_UNATTACHED &&
slave->status != SDW_SLAVE_UNATTACHED) {
- dev_warn(&slave->dev, "Slave %d state check1: UNATTACHED, status was %d\n",
- i, slave->status);
+ dev_dbg(&slave->dev, "Slave %d state check1: UNATTACHED, status was %d\n",
+ i, slave->status);
sdw_modify_slave_status(slave, SDW_SLAVE_UNATTACHED);
/* Ensure driver knows that peripheral unattached */
@@ -1918,8 +1918,8 @@ int sdw_handle_slave_status(struct sdw_bus *bus,
if (slave->status == SDW_SLAVE_UNATTACHED)
break;
- dev_warn(&slave->dev, "Slave %d state check2: UNATTACHED, status was %d\n",
- i, slave->status);
+ dev_dbg(&slave->dev, "Slave %d state check2: UNATTACHED, status was %d\n",
+ i, slave->status);
sdw_modify_slave_status(slave, SDW_SLAVE_UNATTACHED);
break;
diff --git a/drivers/soundwire/cadence_master.c b/drivers/soundwire/cadence_master.c
index 6f2b5ec5c87c..a503ef606a62 100644
--- a/drivers/soundwire/cadence_master.c
+++ b/drivers/soundwire/cadence_master.c
@@ -931,6 +931,14 @@ irqreturn_t sdw_cdns_irq(int irq, void *dev_id)
cdns_read_response(cdns);
+ /*
+ * Clear interrupt before signalling the completion to avoid
+ * a race between this thread and the main thread starting
+ * another TX.
+ */
+ cdns_writel(cdns, CDNS_MCP_INTSTAT, CDNS_MCP_INT_RX_WL);
+ int_status &= ~CDNS_MCP_INT_RX_WL;
+
if (defer && defer->msg) {
cdns_fill_msg_resp(cdns, defer->msg,
defer->length, 0);
diff --git a/drivers/soundwire/debugfs.c b/drivers/soundwire/debugfs.c
index c30f571934ee..93c93a128fb2 100644
--- a/drivers/soundwire/debugfs.c
+++ b/drivers/soundwire/debugfs.c
@@ -295,8 +295,8 @@ void sdw_slave_debugfs_init(struct sdw_slave *slave)
debugfs_create_file("go", 0200, d, slave, &cmd_go_fops);
debugfs_create_file("read_buffer", 0400, d, slave, &read_buffer_fops);
- firmware_file = NULL;
- debugfs_create_str("firmware_file", 0200, d, &firmware_file);
+ if (firmware_file)
+ debugfs_create_str("firmware_file", 0200, d, &firmware_file);
slave->debugfs = d;
}
@@ -308,10 +308,15 @@ void sdw_slave_debugfs_exit(struct sdw_slave *slave)
void sdw_debugfs_init(void)
{
+ if (!firmware_file)
+ firmware_file = kstrdup("", GFP_KERNEL);
+
sdw_debugfs_root = debugfs_create_dir("soundwire", NULL);
}
void sdw_debugfs_exit(void)
{
debugfs_remove_recursive(sdw_debugfs_root);
+ kfree(firmware_file);
+ firmware_file = NULL;
}
diff --git a/drivers/spi/spi-fsl-qspi.c b/drivers/spi/spi-fsl-qspi.c
index 21e357966d2a..cb2e5413dd32 100644
--- a/drivers/spi/spi-fsl-qspi.c
+++ b/drivers/spi/spi-fsl-qspi.c
@@ -606,7 +606,7 @@ static int fsl_qspi_do_op(struct fsl_qspi *q, const struct spi_mem_op *op)
void __iomem *base = q->iobase;
int err = 0;
- init_completion(&q->c);
+ reinit_completion(&q->c);
/*
* Always start the sequence at the same index since we update
@@ -924,6 +924,7 @@ static int fsl_qspi_probe(struct platform_device *pdev)
if (ret < 0)
goto err_disable_clk;
+ init_completion(&q->c);
ret = devm_request_irq(dev, ret,
fsl_qspi_irq_handler, 0, pdev->name, q);
if (ret) {
diff --git a/drivers/spi/spi-hisi-kunpeng.c b/drivers/spi/spi-hisi-kunpeng.c
index f0a50f40a3ba..77526f794068 100644
--- a/drivers/spi/spi-hisi-kunpeng.c
+++ b/drivers/spi/spi-hisi-kunpeng.c
@@ -196,8 +196,18 @@ static void hisi_spi_flush_fifo(struct hisi_spi *hs)
unsigned long limit = loops_per_jiffy << 1;
do {
- while (hisi_spi_rx_not_empty(hs))
+ unsigned long inner_limit = loops_per_jiffy;
+
+ while (hisi_spi_rx_not_empty(hs) && --inner_limit) {
readl(hs->regs + HISI_SPI_DOUT);
+ cpu_relax();
+ }
+
+ if (!inner_limit) {
+ dev_warn_ratelimited(hs->dev, "RX FIFO flush timeout\n");
+ break;
+ }
+
} while (hisi_spi_busy(hs) && limit--);
}
diff --git a/drivers/spi/spi-mtk-snfi.c b/drivers/spi/spi-mtk-snfi.c
index c5677fd94e5e..8234064921f3 100644
--- a/drivers/spi/spi-mtk-snfi.c
+++ b/drivers/spi/spi-mtk-snfi.c
@@ -1307,6 +1307,13 @@ static const struct spi_controller_mem_caps mtk_snand_mem_caps = {
.ecc = true,
};
+static void mtk_unregister_ecc_engine(void *data)
+{
+ struct nand_ecc_engine *eng = data;
+
+ nand_ecc_unregister_on_host_hw_engine(eng);
+}
+
static irqreturn_t mtk_snand_irq(int irq, void *id)
{
struct mtk_snand *snf = id;
@@ -1447,6 +1454,13 @@ static int mtk_snand_probe(struct platform_device *pdev)
goto release_ecc;
}
+ ret = devm_add_action_or_reset(&pdev->dev, mtk_unregister_ecc_engine,
+ &ms->ecc_eng);
+ if (ret) {
+ dev_err_probe(&pdev->dev, ret, "failed to add ECC unregister action\n");
+ goto release_ecc;
+ }
+
ctlr->num_chipselect = 1;
ctlr->mem_ops = &mtk_snand_mem_ops;
ctlr->mem_caps = &mtk_snand_mem_caps;
diff --git a/drivers/spi/spi-nxp-fspi.c b/drivers/spi/spi-nxp-fspi.c
index a43540d7995e..b7dbc015d88f 100644
--- a/drivers/spi/spi-nxp-fspi.c
+++ b/drivers/spi/spi-nxp-fspi.c
@@ -48,6 +48,8 @@
#include <linux/mutex.h>
#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/pm_runtime.h>
#include <linux/pm_qos.h>
#include <linux/regmap.h>
#include <linux/sizes.h>
@@ -57,6 +59,9 @@
#include <linux/spi/spi.h>
#include <linux/spi/spi-mem.h>
+/* runtime pm timeout */
+#define FSPI_RPM_TIMEOUT 50 /* 50ms */
+
/* Registers used by the driver */
#define FSPI_MCR0 0x00
#define FSPI_MCR0_AHB_TIMEOUT(x) ((x) << 24)
@@ -396,6 +401,8 @@ struct nxp_fspi {
struct mutex lock;
struct pm_qos_request pm_qos_req;
int selected;
+#define FSPI_NEED_INIT (1 << 0)
+ int flags;
};
static inline int needs_ip_only(struct nxp_fspi *f)
@@ -900,7 +907,7 @@ static int nxp_fspi_do_op(struct nxp_fspi *f, const struct spi_mem_op *op)
reg = reg | FSPI_IPRXFCR_CLR;
fspi_writel(f, reg, base + FSPI_IPRXFCR);
- init_completion(&f->c);
+ reinit_completion(&f->c);
fspi_writel(f, op->addr.val, base + FSPI_IPCR0);
/*
@@ -935,6 +942,13 @@ static int nxp_fspi_exec_op(struct spi_mem *mem, const struct spi_mem_op *op)
mutex_lock(&f->lock);
+ err = pm_runtime_get_sync(f->dev);
+ if (err < 0) {
+ mutex_unlock(&f->lock);
+ dev_err(f->dev, "Failed to enable clock %d\n", __LINE__);
+ return err;
+ }
+
/* Wait for controller being ready. */
err = fspi_readl_poll_tout(f, f->iobase + FSPI_STS0,
FSPI_STS0_ARB_IDLE, 1, POLL_TOUT, true);
@@ -963,8 +977,10 @@ static int nxp_fspi_exec_op(struct spi_mem *mem, const struct spi_mem_op *op)
/* Invalidate the data in the AHB buffer. */
nxp_fspi_invalid(f);
- mutex_unlock(&f->lock);
+ pm_runtime_mark_last_busy(f->dev);
+ pm_runtime_put_autosuspend(f->dev);
+ mutex_unlock(&f->lock);
return err;
}
@@ -1231,9 +1247,14 @@ static int nxp_fspi_probe(struct platform_device *pdev)
if (irq < 0)
return dev_err_probe(dev, irq, "Failed to get irq source");
- ret = nxp_fspi_clk_prep_enable(f);
- if (ret)
- return dev_err_probe(dev, ret, "Can't enable the clock\n");
+ pm_runtime_enable(dev);
+ pm_runtime_set_autosuspend_delay(dev, FSPI_RPM_TIMEOUT);
+ pm_runtime_use_autosuspend(dev);
+
+ /* enable clock */
+ ret = pm_runtime_get_sync(f->dev);
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "Failed to enable clock");
/* Clear potential interrupts */
reg = fspi_readl(f, f->iobase + FSPI_INTR);
@@ -1242,12 +1263,15 @@ static int nxp_fspi_probe(struct platform_device *pdev)
nxp_fspi_default_setup(f);
+ ret = pm_runtime_put_sync(dev);
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "Failed to disable clock");
+
+ init_completion(&f->c);
ret = devm_request_irq(dev, irq,
nxp_fspi_irq_handler, 0, pdev->name, f);
- if (ret) {
- nxp_fspi_clk_disable_unprep(f);
+ if (ret)
return dev_err_probe(dev, ret, "Failed to request irq\n");
- }
ret = devm_mutex_init(dev, &f->lock);
if (ret)
@@ -1271,29 +1295,70 @@ static void nxp_fspi_remove(struct platform_device *pdev)
{
struct nxp_fspi *f = platform_get_drvdata(pdev);
+ /* enable clock first since there is reigster access */
+ pm_runtime_get_sync(f->dev);
+
/* disable the hardware */
fspi_writel(f, FSPI_MCR0_MDIS, f->iobase + FSPI_MCR0);
+ pm_runtime_disable(f->dev);
+ pm_runtime_put_noidle(f->dev);
nxp_fspi_clk_disable_unprep(f);
if (f->ahb_addr)
iounmap(f->ahb_addr);
}
-static int nxp_fspi_suspend(struct device *dev)
+static int nxp_fspi_runtime_suspend(struct device *dev)
{
+ struct nxp_fspi *f = dev_get_drvdata(dev);
+
+ nxp_fspi_clk_disable_unprep(f);
+
return 0;
}
-static int nxp_fspi_resume(struct device *dev)
+static int nxp_fspi_runtime_resume(struct device *dev)
{
struct nxp_fspi *f = dev_get_drvdata(dev);
+ int ret;
- nxp_fspi_default_setup(f);
+ ret = nxp_fspi_clk_prep_enable(f);
+ if (ret)
+ return ret;
- return 0;
+ if (f->flags & FSPI_NEED_INIT) {
+ nxp_fspi_default_setup(f);
+ ret = pinctrl_pm_select_default_state(dev);
+ if (ret)
+ dev_err(dev, "select flexspi default pinctrl failed!\n");
+ f->flags &= ~FSPI_NEED_INIT;
+ }
+
+ return ret;
+}
+
+static int nxp_fspi_suspend(struct device *dev)
+{
+ struct nxp_fspi *f = dev_get_drvdata(dev);
+ int ret;
+
+ ret = pinctrl_pm_select_sleep_state(dev);
+ if (ret) {
+ dev_err(dev, "select flexspi sleep pinctrl failed!\n");
+ return ret;
+ }
+
+ f->flags |= FSPI_NEED_INIT;
+
+ return pm_runtime_force_suspend(dev);
}
+static const struct dev_pm_ops nxp_fspi_pm_ops = {
+ RUNTIME_PM_OPS(nxp_fspi_runtime_suspend, nxp_fspi_runtime_resume, NULL)
+ SYSTEM_SLEEP_PM_OPS(nxp_fspi_suspend, pm_runtime_force_resume)
+};
+
static const struct of_device_id nxp_fspi_dt_ids[] = {
{ .compatible = "nxp,lx2160a-fspi", .data = (void *)&lx2160a_data, },
{ .compatible = "nxp,imx8mm-fspi", .data = (void *)&imx8mm_data, },
@@ -1313,17 +1378,12 @@ static const struct acpi_device_id nxp_fspi_acpi_ids[] = {
MODULE_DEVICE_TABLE(acpi, nxp_fspi_acpi_ids);
#endif
-static const struct dev_pm_ops nxp_fspi_pm_ops = {
- .suspend = nxp_fspi_suspend,
- .resume = nxp_fspi_resume,
-};
-
static struct platform_driver nxp_fspi_driver = {
.driver = {
.name = "nxp-fspi",
.of_match_table = nxp_fspi_dt_ids,
.acpi_match_table = ACPI_PTR(nxp_fspi_acpi_ids),
- .pm = &nxp_fspi_pm_ops,
+ .pm = pm_ptr(&nxp_fspi_pm_ops),
},
.probe = nxp_fspi_probe,
.remove_new = nxp_fspi_remove,
diff --git a/drivers/spi/spi-rockchip.c b/drivers/spi/spi-rockchip.c
index b480408c812f..ccb35a2e39fb 100644
--- a/drivers/spi/spi-rockchip.c
+++ b/drivers/spi/spi-rockchip.c
@@ -357,7 +357,8 @@ static irqreturn_t rockchip_spi_isr(int irq, void *dev_id)
struct rockchip_spi *rs = spi_controller_get_devdata(ctlr);
/* When int_cs_inactive comes, spi target abort */
- if (rs->cs_inactive && readl_relaxed(rs->regs + ROCKCHIP_SPI_IMR) & INT_CS_INACTIVE) {
+ if (rs->cs_inactive &&
+ (readl_relaxed(rs->regs + ROCKCHIP_SPI_ISR) & INT_CS_INACTIVE)) {
ctlr->target_abort(ctlr);
writel_relaxed(0, rs->regs + ROCKCHIP_SPI_IMR);
writel_relaxed(0xffffffff, rs->regs + ROCKCHIP_SPI_ICR);
diff --git a/drivers/spi/spi-sifive.c b/drivers/spi/spi-sifive.c
index cfd17bbb2202..683dce55ef30 100644
--- a/drivers/spi/spi-sifive.c
+++ b/drivers/spi/spi-sifive.c
@@ -312,7 +312,8 @@ static int sifive_spi_probe(struct platform_device *pdev)
goto put_host;
}
- spi->clk = devm_clk_get(&pdev->dev, NULL);
+ /* Spin up the bus clock before hitting registers */
+ spi->clk = devm_clk_get_enabled(&pdev->dev, NULL);
if (IS_ERR(spi->clk)) {
dev_err(&pdev->dev, "Unable to find bus clock\n");
ret = PTR_ERR(spi->clk);
@@ -342,13 +343,6 @@ static int sifive_spi_probe(struct platform_device *pdev)
goto put_host;
}
- /* Spin up the bus clock before hitting registers */
- ret = clk_prepare_enable(spi->clk);
- if (ret) {
- dev_err(&pdev->dev, "Unable to enable bus clock\n");
- goto put_host;
- }
-
/* probe the number of CS lines */
spi->cs_inactive = sifive_spi_read(spi, SIFIVE_SPI_REG_CSDEF);
sifive_spi_write(spi, SIFIVE_SPI_REG_CSDEF, 0xffffffffU);
@@ -357,14 +351,14 @@ static int sifive_spi_probe(struct platform_device *pdev)
if (!cs_bits) {
dev_err(&pdev->dev, "Could not auto probe CS lines\n");
ret = -EINVAL;
- goto disable_clk;
+ goto put_host;
}
num_cs = ilog2(cs_bits) + 1;
if (num_cs > SIFIVE_SPI_MAX_CS) {
dev_err(&pdev->dev, "Invalid number of spi targets\n");
ret = -EINVAL;
- goto disable_clk;
+ goto put_host;
}
/* Define our host */
@@ -393,22 +387,20 @@ static int sifive_spi_probe(struct platform_device *pdev)
dev_name(&pdev->dev), spi);
if (ret) {
dev_err(&pdev->dev, "Unable to bind to interrupt\n");
- goto disable_clk;
+ goto put_host;
}
dev_info(&pdev->dev, "mapped; irq=%d, cs=%d\n",
irq, host->num_chipselect);
- ret = devm_spi_register_controller(&pdev->dev, host);
+ ret = spi_register_controller(host);
if (ret < 0) {
dev_err(&pdev->dev, "spi_register_host failed\n");
- goto disable_clk;
+ goto put_host;
}
return 0;
-disable_clk:
- clk_disable_unprepare(spi->clk);
put_host:
spi_controller_put(host);
@@ -420,9 +412,14 @@ static void sifive_spi_remove(struct platform_device *pdev)
struct spi_controller *host = platform_get_drvdata(pdev);
struct sifive_spi *spi = spi_controller_get_devdata(host);
+ spi_controller_get(host);
+
+ spi_unregister_controller(host);
+
/* Disable all the interrupts just in case */
sifive_spi_write(spi, SIFIVE_SPI_REG_IE, 0);
- clk_disable_unprepare(spi->clk);
+
+ spi_controller_put(host);
}
static int sifive_spi_suspend(struct device *dev)
diff --git a/drivers/target/target_core_sbc.c b/drivers/target/target_core_sbc.c
index fe8beb7dbab1..4c828a3ac18c 100644
--- a/drivers/target/target_core_sbc.c
+++ b/drivers/target/target_core_sbc.c
@@ -1136,7 +1136,8 @@ sbc_execute_unmap(struct se_cmd *cmd)
goto err;
}
- if (lba + range > dev->transport->get_blocks(dev) + 1) {
+ if (lba + range < lba ||
+ lba + range > dev->transport->get_blocks(dev) + 1) {
ret = TCM_ADDRESS_OUT_OF_RANGE;
goto err;
}
diff --git a/drivers/thermal/spear_thermal.c b/drivers/thermal/spear_thermal.c
index 60a871998b07..19b37f9b093f 100644
--- a/drivers/thermal/spear_thermal.c
+++ b/drivers/thermal/spear_thermal.c
@@ -93,7 +93,7 @@ static int spear_thermal_probe(struct platform_device *pdev)
struct device_node *np = pdev->dev.of_node;
int ret = 0, val;
- if (!np || !of_property_read_u32(np, "st,thermal-flags", &val)) {
+ if (!np || of_property_read_u32(np, "st,thermal-flags", &val)) {
dev_err(&pdev->dev, "Failed: DT Pdata not passed\n");
return -EINVAL;
}
diff --git a/drivers/tty/hvc/hvc_iucv.c b/drivers/tty/hvc/hvc_iucv.c
index ed4bf40278a7..a88722afa373 100644
--- a/drivers/tty/hvc/hvc_iucv.c
+++ b/drivers/tty/hvc/hvc_iucv.c
@@ -130,7 +130,7 @@ static struct iucv_handler hvc_iucv_handler = {
*/
static struct hvc_iucv_private *hvc_iucv_get_private(uint32_t num)
{
- if (num > hvc_iucv_devices)
+ if (num >= hvc_iucv_devices)
return NULL;
return hvc_iucv_table[num];
}
diff --git a/drivers/tty/serial/ip22zilog.c b/drivers/tty/serial/ip22zilog.c
index 6e19c6713849..a12101dc0554 100644
--- a/drivers/tty/serial/ip22zilog.c
+++ b/drivers/tty/serial/ip22zilog.c
@@ -1025,7 +1025,7 @@ static struct uart_driver ip22zilog_reg = {
#endif
};
-static void __init ip22zilog_prepare(struct uart_ip22zilog_port *up)
+static void ip22zilog_prepare(struct uart_ip22zilog_port *up)
{
unsigned char sysrq_on = IS_ENABLED(CONFIG_SERIAL_IP22_ZILOG_CONSOLE);
int brg;
diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
index f65d91c01f2e..03600872c480 100644
--- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c
+++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
@@ -430,9 +430,7 @@ static int vfio_fsl_mc_bus_notifier(struct notifier_block *nb,
if (action == BUS_NOTIFY_ADD_DEVICE &&
vdev->mc_dev == mc_cont) {
- mc_dev->driver_override = kasprintf(GFP_KERNEL, "%s",
- vfio_fsl_mc_ops.name);
- if (!mc_dev->driver_override)
+ if (device_set_driver_override(dev, vfio_fsl_mc_ops.name))
dev_warn(dev, "VFIO_FSL_MC: Setting driver override for device in dprc %s failed\n",
dev_name(&mc_cont->dev));
else
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 5f545b45078f..dd6e73de2e2a 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2014,9 +2014,8 @@ static int vfio_pci_bus_notifier(struct notifier_block *nb,
pdev->is_virtfn && physfn == vdev->pdev) {
pci_info(vdev->pdev, "Captured SR-IOV VF %s driver_override\n",
pci_name(pdev));
- pdev->driver_override = kasprintf(GFP_KERNEL, "%s",
- vdev->vdev.ops->name);
- WARN_ON(!pdev->driver_override);
+ WARN_ON(device_set_driver_override(&pdev->dev,
+ vdev->vdev.ops->name));
} else if (action == BUS_NOTIFY_BOUND_DRIVER &&
pdev->is_virtfn && physfn == vdev->pdev) {
struct pci_driver *drv = pci_dev_driver(pdev);
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index aff4ec783562..3ad1a6f4ef96 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -549,7 +549,7 @@ static void vhost_net_busy_poll(struct vhost_net *net,
busyloop_timeout = poll_rx ? rvq->busyloop_timeout:
tvq->busyloop_timeout;
- preempt_disable();
+ migrate_disable();
endtime = busy_clock() + busyloop_timeout;
while (vhost_can_busy_poll(endtime)) {
@@ -566,7 +566,7 @@ static void vhost_net_busy_poll(struct vhost_net *net,
cpu_relax();
}
- preempt_enable();
+ migrate_enable();
if (poll_rx || sock_has_rx_data(sock))
vhost_net_busy_poll_try_queue(net, vq);
diff --git a/drivers/video/backlight/sky81452-backlight.c b/drivers/video/backlight/sky81452-backlight.c
index 935043b67786..3a5ff1b6f432 100644
--- a/drivers/video/backlight/sky81452-backlight.c
+++ b/drivers/video/backlight/sky81452-backlight.c
@@ -202,6 +202,9 @@ static struct sky81452_bl_platform_data *sky81452_bl_parse_dt(
pdata->dpwm_mode = of_property_read_bool(np, "skyworks,dpwm-mode");
pdata->phase_shift = of_property_read_bool(np, "skyworks,phase-shift");
pdata->gpiod_enable = devm_gpiod_get_optional(dev, NULL, GPIOD_OUT_HIGH);
+ if (IS_ERR(pdata->gpiod_enable))
+ return dev_err_cast_probe(dev, pdata->gpiod_enable,
+ "failed to get gpio\n");
ret = of_property_count_u32_elems(np, "led-sources");
if (ret < 0) {
diff --git a/drivers/video/fbdev/matrox/g450_pll.c b/drivers/video/fbdev/matrox/g450_pll.c
index ff8e321a22ce..b2d3f7328ea8 100644
--- a/drivers/video/fbdev/matrox/g450_pll.c
+++ b/drivers/video/fbdev/matrox/g450_pll.c
@@ -407,7 +407,7 @@ static int __g450_setclk(struct matrox_fb_info *minfo, unsigned int fout,
case M_VIDEO_PLL:
{
u_int8_t tmp;
- unsigned int mnp;
+ unsigned int mnp __maybe_unused;
unsigned long flags;
matroxfb_DAC_lock_irqsave(flags);
diff --git a/drivers/video/fbdev/offb.c b/drivers/video/fbdev/offb.c
index f85428e13996..166b2dff36f5 100644
--- a/drivers/video/fbdev/offb.c
+++ b/drivers/video/fbdev/offb.c
@@ -640,8 +640,13 @@ static void offb_init_nodriver(struct platform_device *parent, struct device_nod
vid = be32_to_cpup(vidp);
did = be32_to_cpup(didp);
pdev = pci_get_device(vid, did, NULL);
- if (!pdev || pci_enable_device(pdev))
+ if (!pdev)
return;
+
+ if (pci_enable_device(pdev)) {
+ pci_dev_put(pdev);
+ return;
+ }
}
#endif
/* kludge for valkyrie */
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index b616b7768c3b..8b3006078d00 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -618,6 +618,8 @@ static int pcistub_seize(struct pci_dev *dev,
return err;
}
+static struct pci_driver xen_pcibk_pci_driver;
+
/* Called when 'bind'. This means we must _NOT_ call pci_reset_function or
* other functions that take the sysfs lock. */
static int pcistub_probe(struct pci_dev *dev, const struct pci_device_id *id)
@@ -629,8 +631,8 @@ static int pcistub_probe(struct pci_dev *dev, const struct pci_device_id *id)
match = pcistub_match(dev);
- if ((dev->driver_override &&
- !strcmp(dev->driver_override, PCISTUB_DRIVER_NAME)) ||
+ if (device_match_driver_override(&dev->dev,
+ &xen_pcibk_pci_driver.driver) > 0 ||
match) {
if (dev->hdr_type != PCI_HEADER_TYPE_NORMAL
diff --git a/fs/adfs/super.c b/fs/adfs/super.c
index f0b999a4961b..998ea8a2f833 100644
--- a/fs/adfs/super.c
+++ b/fs/adfs/super.c
@@ -343,6 +343,9 @@ static int adfs_validate_bblk(struct super_block *sb, struct buffer_head *bh,
if (adfs_checkdiscrecord(dr))
return -EILSEQ;
+ if ((dr->nzones | dr->nzones_high << 8) == 0)
+ return -EILSEQ;
+
*drp = dr;
return 0;
}
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 28024c827b75..c79dd6ff03dc 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -1085,6 +1085,12 @@ static void compress_file_range(struct btrfs_work *work)
mapping_set_error(mapping, -EIO);
goto free_pages;
}
+ /*
+ * If a single block at file offset 0 cannot be inlined, fall back to
+ * regular writes without marking the file incompressible.
+ */
+ if (start == 0 && end <= blocksize)
+ goto cleanup_and_bail_uncompressed;
/*
* We aren't doing an inline extent. Round the compressed size up to a
@@ -1288,7 +1294,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
NULL, &cached,
EXTENT_LOCKED | EXTENT_DELALLOC |
EXTENT_DELALLOC_NEW |
- EXTENT_DEFRAG | EXTENT_DO_ACCOUNTING,
+ EXTENT_DEFRAG | EXTENT_CLEAR_META_RESV,
PAGE_UNLOCK | PAGE_START_WRITEBACK |
PAGE_END_WRITEBACK);
free_async_extent_pages(async_extent);
@@ -4743,32 +4749,33 @@ int btrfs_delete_subvolume(struct btrfs_inode *dir, struct dentry *dentry)
return ret;
}
-static int btrfs_rmdir(struct inode *dir, struct dentry *dentry)
+static int btrfs_rmdir(struct inode *vfs_dir, struct dentry *dentry)
{
- struct inode *inode = d_inode(dentry);
- struct btrfs_fs_info *fs_info = BTRFS_I(inode)->root->fs_info;
+ struct btrfs_inode *dir = BTRFS_I(vfs_dir);
+ struct btrfs_inode *inode = BTRFS_I(d_inode(dentry));
+ struct btrfs_fs_info *fs_info = inode->root->fs_info;
int ret = 0;
struct btrfs_trans_handle *trans;
struct fscrypt_name fname;
- if (inode->i_size > BTRFS_EMPTY_DIR_SIZE)
+ if (inode->vfs_inode.i_size > BTRFS_EMPTY_DIR_SIZE)
return -ENOTEMPTY;
- if (btrfs_ino(BTRFS_I(inode)) == BTRFS_FIRST_FREE_OBJECTID) {
+ if (btrfs_ino(inode) == BTRFS_FIRST_FREE_OBJECTID) {
if (unlikely(btrfs_fs_incompat(fs_info, EXTENT_TREE_V2))) {
btrfs_err(fs_info,
"extent tree v2 doesn't support snapshot deletion yet");
return -EOPNOTSUPP;
}
- return btrfs_delete_subvolume(BTRFS_I(dir), dentry);
+ return btrfs_delete_subvolume(dir, dentry);
}
- ret = fscrypt_setup_filename(dir, &dentry->d_name, 1, &fname);
+ ret = fscrypt_setup_filename(vfs_dir, &dentry->d_name, 1, &fname);
if (ret)
return ret;
/* This needs to handle no-key deletions later on */
- trans = __unlink_start_trans(BTRFS_I(dir));
+ trans = __unlink_start_trans(dir);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
goto out_notrans;
@@ -4788,23 +4795,24 @@ static int btrfs_rmdir(struct inode *dir, struct dentry *dentry)
* This is because we can't unlink other roots when replaying the dir
* deletes for directory foo.
*/
- if (BTRFS_I(inode)->last_unlink_trans >= trans->transid)
- btrfs_record_snapshot_destroy(trans, BTRFS_I(dir));
+ if (inode->last_unlink_trans >= trans->transid)
+ btrfs_record_snapshot_destroy(trans, dir);
- if (unlikely(btrfs_ino(BTRFS_I(inode)) == BTRFS_EMPTY_SUBVOL_DIR_OBJECTID)) {
- ret = btrfs_unlink_subvol(trans, BTRFS_I(dir), dentry);
+ if (unlikely(btrfs_ino(inode) == BTRFS_EMPTY_SUBVOL_DIR_OBJECTID)) {
+ ret = btrfs_unlink_subvol(trans, dir, dentry);
goto out;
}
- ret = btrfs_orphan_add(trans, BTRFS_I(inode));
+ ret = btrfs_orphan_add(trans, inode);
if (ret)
goto out;
+ btrfs_record_unlink_dir(trans, dir, inode, false);
+
/* now the directory is empty */
- ret = btrfs_unlink_inode(trans, BTRFS_I(dir), BTRFS_I(d_inode(dentry)),
- &fname.disk_name);
+ ret = btrfs_unlink_inode(trans, dir, inode, &fname.disk_name);
if (!ret)
- btrfs_i_size_write(BTRFS_I(inode), 0);
+ btrfs_i_size_write(inode, 0);
out:
btrfs_end_transaction(trans);
out_notrans:
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index a61022182f45..b6152b36f81e 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -1315,9 +1315,9 @@ static noinline int __btrfs_ioctl_snap_create(struct file *file,
ret = btrfs_mksubvol(&file->f_path, idmap, name,
namelen, NULL, readonly, inherit);
} else {
- struct fd src = fdget(fd);
+ CLASS(fd, src)(fd);
struct inode *src_inode;
- if (!fd_file(src)) {
+ if (fd_empty(src)) {
ret = -EINVAL;
goto out_drop_write;
}
@@ -1348,7 +1348,6 @@ static noinline int __btrfs_ioctl_snap_create(struct file *file,
BTRFS_I(src_inode)->root,
readonly, inherit);
}
- fdput(src);
}
out_drop_write:
mnt_drop_write_file(file);
diff --git a/fs/btrfs/reflink.c b/fs/btrfs/reflink.c
index f0824c948cb7..e86df9c83aba 100644
--- a/fs/btrfs/reflink.c
+++ b/fs/btrfs/reflink.c
@@ -165,7 +165,7 @@ static int copy_inline_to_page(struct btrfs_inode *inode,
* the source inode to destination inode when possible. When not possible we
* copy the inline extent's data into the respective page of the inode.
*/
-static int clone_copy_inline_extent(struct inode *dst,
+static int clone_copy_inline_extent(struct btrfs_inode *inode,
struct btrfs_path *path,
struct btrfs_key *new_key,
const u64 drop_start,
@@ -175,8 +175,8 @@ static int clone_copy_inline_extent(struct inode *dst,
char *inline_data,
struct btrfs_trans_handle **trans_out)
{
- struct btrfs_fs_info *fs_info = inode_to_fs_info(dst);
- struct btrfs_root *root = BTRFS_I(dst)->root;
+ struct btrfs_root *root = inode->root;
+ struct btrfs_fs_info *fs_info = root->fs_info;
const u64 aligned_end = ALIGN(new_key->offset + datal,
fs_info->sectorsize);
struct btrfs_trans_handle *trans = NULL;
@@ -185,12 +185,12 @@ static int clone_copy_inline_extent(struct inode *dst,
struct btrfs_key key;
if (new_key->offset > 0) {
- ret = copy_inline_to_page(BTRFS_I(dst), new_key->offset,
+ ret = copy_inline_to_page(inode, new_key->offset,
inline_data, size, datal, comp_type);
goto out;
}
- key.objectid = btrfs_ino(BTRFS_I(dst));
+ key.objectid = btrfs_ino(inode);
key.type = BTRFS_EXTENT_DATA_KEY;
key.offset = 0;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
@@ -205,7 +205,7 @@ static int clone_copy_inline_extent(struct inode *dst,
goto copy_inline_extent;
}
btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]);
- if (key.objectid == btrfs_ino(BTRFS_I(dst)) &&
+ if (key.objectid == btrfs_ino(inode) &&
key.type == BTRFS_EXTENT_DATA_KEY) {
/*
* There's an implicit hole at file offset 0, copy the
@@ -214,7 +214,7 @@ static int clone_copy_inline_extent(struct inode *dst,
ASSERT(key.offset > 0);
goto copy_to_page;
}
- } else if (i_size_read(dst) <= datal) {
+ } else if (i_size_read(&inode->vfs_inode) <= datal) {
struct btrfs_file_extent_item *ei;
ei = btrfs_item_ptr(path->nodes[0], path->slots[0],
@@ -236,7 +236,7 @@ static int clone_copy_inline_extent(struct inode *dst,
* We have no extent items, or we have an extent at offset 0 which may
* or may not be inlined. All these cases are dealt the same way.
*/
- if (i_size_read(dst) > datal) {
+ if (i_size_read(&inode->vfs_inode) > datal) {
/*
* At the destination offset 0 we have either a hole, a regular
* extent or an inline extent larger then the one we want to
@@ -270,7 +270,7 @@ static int clone_copy_inline_extent(struct inode *dst,
drop_args.start = drop_start;
drop_args.end = aligned_end;
drop_args.drop_cache = true;
- ret = btrfs_drop_extents(trans, root, BTRFS_I(dst), &drop_args);
+ ret = btrfs_drop_extents(trans, root, inode, &drop_args);
if (ret)
goto out;
ret = btrfs_insert_empty_item(trans, root, path, new_key, size);
@@ -281,9 +281,9 @@ static int clone_copy_inline_extent(struct inode *dst,
btrfs_item_ptr_offset(path->nodes[0],
path->slots[0]),
size);
- btrfs_update_inode_bytes(BTRFS_I(dst), datal, drop_args.bytes_found);
- btrfs_set_inode_full_sync(BTRFS_I(dst));
- ret = btrfs_inode_set_file_extent_range(BTRFS_I(dst), 0, aligned_end);
+ btrfs_update_inode_bytes(inode, datal, drop_args.bytes_found);
+ btrfs_set_inode_full_sync(inode);
+ ret = btrfs_inode_set_file_extent_range(inode, 0, aligned_end);
out:
if (!ret && !trans) {
/*
@@ -318,8 +318,53 @@ static int clone_copy_inline_extent(struct inode *dst,
*/
btrfs_release_path(path);
- ret = copy_inline_to_page(BTRFS_I(dst), new_key->offset,
+ ret = copy_inline_to_page(inode, new_key->offset,
inline_data, size, datal, comp_type);
+
+ /*
+ * If we copied the inline extent data to a page/folio beyond the i_size
+ * of the destination inode, then we need to increase the i_size before
+ * we start a transaction to update the inode item. This is to prevent a
+ * deadlock when the flushoncommit mount option is used, which happens
+ * like this:
+ *
+ * 1) Task A clones an inline extent from inode X to an offset of inode
+ * Y that is beyond Y's current i_size. This means we copied the
+ * inline extent's data to a folio of inode Y that is beyond its EOF,
+ * using the call above to copy_inline_to_page();
+ *
+ * 2) Task B starts a transaction commit and calls
+ * btrfs_start_delalloc_flush() to flush delalloc;
+ *
+ * 3) The delalloc flushing sees the new dirty folio of inode Y and when
+ * it attempts to flush it, it ends up at extent_writepage() and sees
+ * that the offset of the folio is beyond the i_size of inode Y, so
+ * it attempts to invalidate the folio by calling folio_invalidate(),
+ * which ends up at btrfs' folio invalidate callback -
+ * btrfs_invalidate_folio(). There it tries to lock the folio's range
+ * in inode Y's extent io tree, but it blocks since it's currently
+ * locked by task A - during reflink we lock the inodes and the
+ * source and destination ranges after flushing all delalloc and
+ * waiting for ordered extent completion - after that we don't expect
+ * to have dirty folios in the ranges, the exception is if we have to
+ * copy an inline extent's data (because the destination offset is
+ * not zero);
+ *
+ * 4) Task A then does the 'goto out' below and attempts to start a
+ * transaction to update the inode item, and then it's blocked since
+ * the current transaction is in the TRANS_STATE_COMMIT_START state.
+ * Therefore task A has to wait for the current transaction to become
+ * unblocked (its state >= TRANS_STATE_UNBLOCKED).
+ *
+ * This leads to a deadlock - the task committing the transaction
+ * waiting for the delalloc flushing which is blocked during folio
+ * invalidation on the inode's extent lock and the reflink task waiting
+ * for the current transaction to be unblocked so that it can start a
+ * a new one to update the inode item (while holding the extent lock).
+ */
+ if (ret == 0 && new_key->offset + datal > i_size_read(&inode->vfs_inode))
+ i_size_write(&inode->vfs_inode, new_key->offset + datal);
+
goto out;
}
@@ -526,7 +571,7 @@ static int btrfs_clone(struct inode *src, struct inode *inode,
goto out;
}
- ret = clone_copy_inline_extent(inode, path, &new_key,
+ ret = clone_copy_inline_extent(BTRFS_I(inode), path, &new_key,
drop_start, datal, size,
comp, buf, &trans);
if (ret)
diff --git a/fs/ceph/xattr.c b/fs/ceph/xattr.c
index e066a556eccb..6b17fb311060 100644
--- a/fs/ceph/xattr.c
+++ b/fs/ceph/xattr.c
@@ -1256,6 +1256,22 @@ int __ceph_setxattr(struct inode *inode, const char *name,
ceph_vinop(inode), name, ceph_cap_string(issued));
__build_xattrs(inode);
+ /*
+ * __build_xattrs() may have released and reacquired i_ceph_lock,
+ * during which handle_cap_grant() could have replaced i_xattrs.blob
+ * with a newer MDS-provided blob and bumped i_xattrs.version. If that
+ * caused __build_xattrs() to rebuild the rb-tree from the new blob,
+ * count/names_size/vals_size may now be larger than when
+ * required_blob_size was computed above. Recompute it here so the
+ * prealloc_blob size check below reflects the current tree state.
+ */
+ required_blob_size = __get_required_blob_size(ci, name_len, val_len);
+ if (required_blob_size > mdsc->mdsmap->m_max_xattr_size) {
+ doutc(cl, "sync (size too large): %d > %llu\n",
+ required_blob_size, mdsc->mdsmap->m_max_xattr_size);
+ goto do_sync;
+ }
+
if (!ci->i_xattrs.prealloc_blob ||
required_blob_size > ci->i_xattrs.prealloc_blob->alloc_len) {
struct ceph_buffer *blob;
@@ -1296,6 +1312,7 @@ int __ceph_setxattr(struct inode *inode, const char *name,
do_sync:
spin_unlock(&ci->i_ceph_lock);
+ ceph_buffer_put(old_blob);
do_sync_unlocked:
if (lock_snap_rwsem)
up_read(&mdsc->snap_rwsem);
diff --git a/fs/debugfs/file.c b/fs/debugfs/file.c
index 67299e8b734e..cc9587a1c9a9 100644
--- a/fs/debugfs/file.c
+++ b/fs/debugfs/file.c
@@ -991,7 +991,6 @@ ssize_t debugfs_read_file_str(struct file *file, char __user *user_buf,
return ret;
}
-EXPORT_SYMBOL_GPL(debugfs_create_str);
static ssize_t debugfs_write_file_str(struct file *file, const char __user *user_buf,
size_t count, loff_t *ppos)
@@ -1071,7 +1070,7 @@ static const struct file_operations fops_str_wo = {
* directory dentry if set. If this parameter is %NULL, then the
* file will be created in the root of the debugfs filesystem.
* @value: a pointer to the variable that the file should read to and write
- * from.
+ * from. This pointer and the string it points to must not be %NULL.
*
* This function creates a file in debugfs with the given name that
* contains the value of the variable @value. If the @mode variable is so
@@ -1080,9 +1079,13 @@ static const struct file_operations fops_str_wo = {
void debugfs_create_str(const char *name, umode_t mode,
struct dentry *parent, char **value)
{
+ if (WARN_ON(!value || !*value))
+ return;
+
debugfs_create_mode_unsafe(name, mode, parent, value, &fops_str,
&fops_str_ro, &fops_str_wo);
}
+EXPORT_SYMBOL_GPL(debugfs_create_str);
static ssize_t read_file_blob(struct file *file, char __user *user_buf,
size_t count, loff_t *ppos)
diff --git a/fs/erofs/erofs_fs.h b/fs/erofs/erofs_fs.h
index c8f2ae845bd2..5b237148cb64 100644
--- a/fs/erofs/erofs_fs.h
+++ b/fs/erofs/erofs_fs.h
@@ -336,21 +336,20 @@ struct z_erofs_zstd_cfgs {
#define Z_EROFS_ZSTD_MAX_DICT_SIZE Z_EROFS_PCLUSTER_MAX_SIZE
/*
- * bit 0 : COMPACTED_2B indexes (0 - off; 1 - on)
- * e.g. for 4k logical cluster size, 4B if compacted 2B is off;
- * (4B) + 2B + (4B) if compacted 2B is on.
- * bit 1 : HEAD1 big pcluster (0 - off; 1 - on)
- * bit 2 : HEAD2 big pcluster (0 - off; 1 - on)
- * bit 3 : tailpacking inline pcluster (0 - off; 1 - on)
- * bit 4 : interlaced plain pcluster (0 - off; 1 - on)
- * bit 5 : fragment pcluster (0 - off; 1 - on)
+ * Enable COMPACTED_2B for EROFS_INODE_COMPRESSED_COMPACT inodes:
+ * 4B (disabled) vs 4B+2B+4B (enabled)
*/
#define Z_EROFS_ADVISE_COMPACTED_2B 0x0001
+/* Enable extent metadata for EROFS_INODE_COMPRESSED_FULL inodes */
+#define Z_EROFS_ADVISE_EXTENTS 0x0001
#define Z_EROFS_ADVISE_BIG_PCLUSTER_1 0x0002
#define Z_EROFS_ADVISE_BIG_PCLUSTER_2 0x0004
#define Z_EROFS_ADVISE_INLINE_PCLUSTER 0x0008
#define Z_EROFS_ADVISE_INTERLACED_PCLUSTER 0x0010
#define Z_EROFS_ADVISE_FRAGMENT_PCLUSTER 0x0020
+/* Indicate the record size for each extent if extent metadata is used */
+#define Z_EROFS_ADVISE_EXTRECSZ_BIT 1
+#define Z_EROFS_ADVISE_EXTRECSZ_MASK 0x3
#define Z_EROFS_FRAGMENT_INODE_BIT 7
struct z_erofs_map_header {
@@ -362,45 +361,24 @@ struct z_erofs_map_header {
/* indicates the encoded size of tailpacking data */
__le16 h_idata_size;
};
+ __le32 h_extents_lo; /* extent count LSB */
};
__le16 h_advise;
- /*
- * bit 0-3 : algorithm type of head 1 (logical cluster type 01);
- * bit 4-7 : algorithm type of head 2 (logical cluster type 11).
- */
- __u8 h_algorithmtype;
- /*
- * bit 0-2 : logical cluster bits - 12, e.g. 0 for 4096;
- * bit 3-6 : reserved;
- * bit 7 : move the whole file into packed inode or not.
- */
- __u8 h_clusterbits;
+ union {
+ struct {
+ /* algorithm type (bit 0-3: HEAD1; bit 4-7: HEAD2) */
+ __u8 h_algorithmtype;
+ /*
+ * bit 0-3 : logical cluster bits - blkszbits
+ * bit 4-6 : reserved
+ * bit 7 : pack the whole file into packed inode
+ */
+ __u8 h_clusterbits;
+ };
+ __le16 h_extents_hi; /* extent count MSB */
+ };
};
-/*
- * On-disk logical cluster type:
- * 0 - literal (uncompressed) lcluster
- * 1,3 - compressed lcluster (for HEAD lclusters)
- * 2 - compressed lcluster (for NONHEAD lclusters)
- *
- * In detail,
- * 0 - literal (uncompressed) lcluster,
- * di_advise = 0
- * di_clusterofs = the literal data offset of the lcluster
- * di_blkaddr = the blkaddr of the literal pcluster
- *
- * 1,3 - compressed lcluster (for HEAD lclusters)
- * di_advise = 1 or 3
- * di_clusterofs = the decompressed data offset of the lcluster
- * di_blkaddr = the blkaddr of the compressed pcluster
- *
- * 2 - compressed lcluster (for NONHEAD lclusters)
- * di_advise = 2
- * di_clusterofs =
- * the decompressed data offset in its own HEAD lcluster
- * di_u.delta[0] = distance to this HEAD lcluster
- * di_u.delta[1] = distance to the next HEAD lcluster
- */
enum {
Z_EROFS_LCLUSTER_TYPE_PLAIN = 0,
Z_EROFS_LCLUSTER_TYPE_HEAD1 = 1,
@@ -414,11 +392,7 @@ enum {
/* (noncompact only, HEAD) This pcluster refers to partial decompressed data */
#define Z_EROFS_LI_PARTIAL_REF (1 << 15)
-/*
- * D0_CBLKCNT will be marked _only_ at the 1st non-head lcluster to store the
- * compressed block count of a compressed extent (in logical clusters, aka.
- * block count of a pcluster).
- */
+/* Set on 1st non-head lcluster to store compressed block counti (in blocks) */
#define Z_EROFS_LI_D0_CBLKCNT (1 << 11)
struct z_erofs_lcluster_index {
@@ -427,19 +401,36 @@ struct z_erofs_lcluster_index {
__le16 di_clusterofs;
union {
- /* for the HEAD lclusters */
- __le32 blkaddr;
+ __le32 blkaddr; /* for the HEAD lclusters */
/*
- * for the NONHEAD lclusters
* [0] - distance to its HEAD lcluster
* [1] - distance to the next HEAD lcluster
*/
- __le16 delta[2];
+ __le16 delta[2]; /* for the NONHEAD lclusters */
} di_u;
};
-#define Z_EROFS_FULL_INDEX_ALIGN(end) \
- (ALIGN(end, 8) + sizeof(struct z_erofs_map_header) + 8)
+#define Z_EROFS_MAP_HEADER_END(end) \
+ (ALIGN(end, 8) + sizeof(struct z_erofs_map_header))
+#define Z_EROFS_FULL_INDEX_START(end) (Z_EROFS_MAP_HEADER_END(end) + 8)
+
+#define Z_EROFS_EXTENT_PLEN_PARTIAL BIT(27)
+#define Z_EROFS_EXTENT_PLEN_FMT_BIT 28
+#define Z_EROFS_EXTENT_PLEN_MASK ((Z_EROFS_PCLUSTER_MAX_SIZE << 1) - 1)
+struct z_erofs_extent {
+ __le32 plen; /* encoded length */
+ __le32 pstart_lo; /* physical offset */
+ __le32 pstart_hi; /* physical offset MSB */
+ __le32 lstart_lo; /* logical offset */
+ __le32 lstart_hi; /* logical offset MSB (>= 4GiB inodes) */
+ __u8 reserved[12]; /* for future use */
+};
+
+static inline int z_erofs_extent_recsize(unsigned int advise)
+{
+ return 4 << ((advise >> Z_EROFS_ADVISE_EXTRECSZ_BIT) &
+ Z_EROFS_ADVISE_EXTRECSZ_MASK);
+}
/* check the EROFS on-disk layout strictly at compile time */
static inline void erofs_check_ondisk_layout_definitions(void)
diff --git a/fs/erofs/internal.h b/fs/erofs/internal.h
index 856463a702b2..1c003412677e 100644
--- a/fs/erofs/internal.h
+++ b/fs/erofs/internal.h
@@ -275,7 +275,7 @@ struct erofs_inode {
struct {
unsigned short z_advise;
unsigned char z_algorithmtype[2];
- unsigned char z_logical_clusterbits;
+ unsigned char z_lclusterbits;
unsigned long z_tailextent_headlcn;
erofs_off_t z_fragmentoff;
unsigned short z_idata_size;
diff --git a/fs/erofs/zmap.c b/fs/erofs/zmap.c
index 25a4b82c183c..af1046908c62 100644
--- a/fs/erofs/zmap.c
+++ b/fs/erofs/zmap.c
@@ -10,7 +10,7 @@
struct z_erofs_maprecorder {
struct inode *inode;
struct erofs_map_blocks *map;
- unsigned long lcn;
+ u64 lcn;
/* compression extent information gathered */
u8 type, headtype;
u16 clusterofs;
@@ -20,12 +20,11 @@ struct z_erofs_maprecorder {
bool partialref;
};
-static int z_erofs_load_full_lcluster(struct z_erofs_maprecorder *m,
- unsigned long lcn)
+static int z_erofs_load_full_lcluster(struct z_erofs_maprecorder *m, u64 lcn)
{
struct inode *const inode = m->inode;
struct erofs_inode *const vi = EROFS_I(inode);
- const erofs_off_t pos = Z_EROFS_FULL_INDEX_ALIGN(erofs_iloc(inode) +
+ const erofs_off_t pos = Z_EROFS_FULL_INDEX_START(erofs_iloc(inode) +
vi->inode_isize + vi->xattr_isize) +
lcn * sizeof(struct z_erofs_lcluster_index);
struct z_erofs_lcluster_index *di;
@@ -40,7 +39,7 @@ static int z_erofs_load_full_lcluster(struct z_erofs_maprecorder *m,
advise = le16_to_cpu(di->di_advise);
m->type = advise & Z_EROFS_LI_LCLUSTER_TYPE_MASK;
if (m->type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
- m->clusterofs = 1 << vi->z_logical_clusterbits;
+ m->clusterofs = 1 << vi->z_lclusterbits;
m->delta[0] = le16_to_cpu(di->di_u.delta[0]);
if (m->delta[0] & Z_EROFS_LI_D0_CBLKCNT) {
if (!(vi->z_advise & (Z_EROFS_ADVISE_BIG_PCLUSTER_1 |
@@ -55,10 +54,6 @@ static int z_erofs_load_full_lcluster(struct z_erofs_maprecorder *m,
} else {
m->partialref = !!(advise & Z_EROFS_LI_PARTIAL_REF);
m->clusterofs = le16_to_cpu(di->di_clusterofs);
- if (m->clusterofs >= 1 << vi->z_logical_clusterbits) {
- DBG_BUGON(1);
- return -EFSCORRUPTED;
- }
m->pblk = le32_to_cpu(di->di_u.blkaddr);
}
return 0;
@@ -98,13 +93,13 @@ static int get_compacted_la_distance(unsigned int lobits,
}
static int z_erofs_load_compact_lcluster(struct z_erofs_maprecorder *m,
- unsigned long lcn, bool lookahead)
+ u64 lcn, bool lookahead)
{
struct inode *const inode = m->inode;
struct erofs_inode *const vi = EROFS_I(inode);
- const erofs_off_t ebase = sizeof(struct z_erofs_map_header) +
- ALIGN(erofs_iloc(inode) + vi->inode_isize + vi->xattr_isize, 8);
- const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ const erofs_off_t ebase = Z_EROFS_MAP_HEADER_END(erofs_iloc(inode) +
+ vi->inode_isize + vi->xattr_isize);
+ const unsigned int lclusterbits = vi->z_lclusterbits;
const unsigned int totalidx = erofs_iblks(inode);
unsigned int compacted_4b_initial, compacted_2b, amortizedshift;
unsigned int vcnt, lo, lobits, encodebits, nblk, bytes;
@@ -238,16 +233,31 @@ static int z_erofs_load_compact_lcluster(struct z_erofs_maprecorder *m,
}
static int z_erofs_load_lcluster_from_disk(struct z_erofs_maprecorder *m,
- unsigned int lcn, bool lookahead)
+ u64 lcn, bool lookahead)
{
- switch (EROFS_I(m->inode)->datalayout) {
- case EROFS_INODE_COMPRESSED_FULL:
- return z_erofs_load_full_lcluster(m, lcn);
- case EROFS_INODE_COMPRESSED_COMPACT:
- return z_erofs_load_compact_lcluster(m, lcn, lookahead);
- default:
- return -EINVAL;
+ struct erofs_inode *vi = EROFS_I(m->inode);
+ int err;
+
+ if (vi->datalayout == EROFS_INODE_COMPRESSED_COMPACT) {
+ err = z_erofs_load_compact_lcluster(m, lcn, lookahead);
+ } else {
+ DBG_BUGON(vi->datalayout != EROFS_INODE_COMPRESSED_FULL);
+ err = z_erofs_load_full_lcluster(m, lcn);
+ }
+ if (err)
+ return err;
+
+ if (m->type >= Z_EROFS_LCLUSTER_TYPE_MAX) {
+ erofs_err(m->inode->i_sb, "unknown type %u @ lcn %llu of nid %llu",
+ m->type, lcn, EROFS_I(m->inode)->nid);
+ DBG_BUGON(1);
+ return -EOPNOTSUPP;
+ } else if (m->type != Z_EROFS_LCLUSTER_TYPE_NONHEAD &&
+ m->clusterofs >= (1 << vi->z_lclusterbits)) {
+ DBG_BUGON(1);
+ return -EFSCORRUPTED;
}
+ return 0;
}
static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
@@ -255,22 +265,17 @@ static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
{
struct super_block *sb = m->inode->i_sb;
struct erofs_inode *const vi = EROFS_I(m->inode);
- const unsigned int lclusterbits = vi->z_logical_clusterbits;
+ const unsigned int lclusterbits = vi->z_lclusterbits;
while (m->lcn >= lookback_distance) {
- unsigned long lcn = m->lcn - lookback_distance;
+ u64 lcn = m->lcn - lookback_distance;
int err;
err = z_erofs_load_lcluster_from_disk(m, lcn, false);
if (err)
return err;
- if (m->type >= Z_EROFS_LCLUSTER_TYPE_MAX) {
- erofs_err(sb, "unknown type %u @ lcn %lu of nid %llu",
- m->type, lcn, vi->nid);
- DBG_BUGON(1);
- return -EOPNOTSUPP;
- } else if (m->type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
+ if (m->type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
lookback_distance = m->delta[0];
if (!lookback_distance)
break;
@@ -281,7 +286,7 @@ static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
return 0;
}
}
- erofs_err(sb, "bogus lookback distance %u @ lcn %lu of nid %llu",
+ erofs_err(sb, "bogus lookback distance %u @ lcn %llu of nid %llu",
lookback_distance, m->lcn, vi->nid);
DBG_BUGON(1);
return -EFSCORRUPTED;
@@ -295,7 +300,7 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m,
struct erofs_inode *vi = EROFS_I(inode);
bool bigpcl1 = vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_1;
bool bigpcl2 = vi->z_advise & Z_EROFS_ADVISE_BIG_PCLUSTER_2;
- unsigned long lcn = m->lcn + 1;
+ u64 lcn = m->lcn + 1;
int err;
DBG_BUGON(m->type == Z_EROFS_LCLUSTER_TYPE_NONHEAD);
@@ -304,7 +309,7 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m,
if ((m->headtype == Z_EROFS_LCLUSTER_TYPE_HEAD1 && !bigpcl1) ||
((m->headtype == Z_EROFS_LCLUSTER_TYPE_PLAIN ||
m->headtype == Z_EROFS_LCLUSTER_TYPE_HEAD2) && !bigpcl2) ||
- (lcn << vi->z_logical_clusterbits) >= inode->i_size)
+ (lcn << vi->z_lclusterbits) >= inode->i_size)
m->compressedblks = 1;
if (m->compressedblks)
@@ -325,25 +330,18 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m,
DBG_BUGON(lcn == initial_lcn &&
m->type == Z_EROFS_LCLUSTER_TYPE_NONHEAD);
- if (m->type == Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
- if (m->delta[0] != 1) {
- erofs_err(sb, "bogus CBLKCNT @ lcn %lu of nid %llu", lcn, vi->nid);
- DBG_BUGON(1);
- return -EFSCORRUPTED;
- }
- if (m->compressedblks)
- goto out;
- } else if (m->type < Z_EROFS_LCLUSTER_TYPE_MAX) {
- /*
- * if the 1st NONHEAD lcluster is actually PLAIN or HEAD type
- * rather than CBLKCNT, it's a 1 block-sized pcluster.
- */
- m->compressedblks = 1;
- goto out;
+ if (m->type == Z_EROFS_LCLUSTER_TYPE_NONHEAD && m->delta[0] != 1) {
+ erofs_err(sb, "bogus CBLKCNT @ lcn %llu of nid %llu", lcn, vi->nid);
+ DBG_BUGON(1);
+ return -EFSCORRUPTED;
}
- erofs_err(sb, "cannot found CBLKCNT @ lcn %lu of nid %llu", lcn, vi->nid);
- DBG_BUGON(1);
- return -EFSCORRUPTED;
+
+ /*
+ * if the 1st NONHEAD lcluster is actually PLAIN or HEAD type rather
+ * than CBLKCNT, it's a 1 block-sized pcluster.
+ */
+ if (m->type != Z_EROFS_LCLUSTER_TYPE_NONHEAD || !m->compressedblks)
+ m->compressedblks = 1;
out:
m->map->m_plen = erofs_pos(sb, m->compressedblks);
return 0;
@@ -354,7 +352,7 @@ static int z_erofs_get_extent_decompressedlen(struct z_erofs_maprecorder *m)
struct inode *inode = m->inode;
struct erofs_inode *vi = EROFS_I(inode);
struct erofs_map_blocks *map = m->map;
- unsigned int lclusterbits = vi->z_logical_clusterbits;
+ unsigned int lclusterbits = vi->z_lclusterbits;
u64 lcn = m->lcn, headlcn = map->m_la >> lclusterbits;
int err;
@@ -379,11 +377,6 @@ static int z_erofs_get_extent_decompressedlen(struct z_erofs_maprecorder *m)
if (lcn != headlcn)
break; /* ends at the next HEAD lcluster */
m->delta[1] = 1;
- } else {
- erofs_err(inode->i_sb, "unknown type %u @ lcn %llu of nid %llu",
- m->type, lcn, vi->nid);
- DBG_BUGON(1);
- return -EOPNOTSUPP;
}
lcn += m->delta[1];
}
@@ -398,16 +391,16 @@ static int z_erofs_do_map_blocks(struct inode *inode,
struct super_block *sb = inode->i_sb;
bool fragment = vi->z_advise & Z_EROFS_ADVISE_FRAGMENT_PCLUSTER;
bool ztailpacking = vi->z_idata_size;
+ unsigned int lclusterbits = vi->z_lclusterbits;
struct z_erofs_maprecorder m = {
.inode = inode,
.map = map,
};
int err = 0;
- unsigned int lclusterbits, endoff, afmt;
+ unsigned int endoff, afmt;
unsigned long initial_lcn;
unsigned long long ofs, end;
- lclusterbits = vi->z_logical_clusterbits;
ofs = flags & EROFS_GET_BLOCKS_FINDTAIL ? inode->i_size - 1 : map->m_la;
initial_lcn = ofs >> lclusterbits;
endoff = ofs & ((1 << lclusterbits) - 1);
@@ -421,44 +414,33 @@ static int z_erofs_do_map_blocks(struct inode *inode,
map->m_flags = EROFS_MAP_MAPPED | EROFS_MAP_ENCODED;
end = (m.lcn + 1ULL) << lclusterbits;
- switch (m.type) {
- case Z_EROFS_LCLUSTER_TYPE_PLAIN:
- case Z_EROFS_LCLUSTER_TYPE_HEAD1:
- case Z_EROFS_LCLUSTER_TYPE_HEAD2:
- if (endoff >= m.clusterofs) {
- m.headtype = m.type;
- map->m_la = (m.lcn << lclusterbits) | m.clusterofs;
- /*
- * For ztailpacking files, in order to inline data more
- * effectively, special EOF lclusters are now supported
- * which can have three parts at most.
- */
- if (ztailpacking && end > inode->i_size)
- end = inode->i_size;
- break;
- }
- /* m.lcn should be >= 1 if endoff < m.clusterofs */
- if (!m.lcn) {
- erofs_err(sb, "invalid logical cluster 0 at nid %llu",
- vi->nid);
- err = -EFSCORRUPTED;
- goto unmap_out;
+ if (m.type != Z_EROFS_LCLUSTER_TYPE_NONHEAD && endoff >= m.clusterofs) {
+ m.headtype = m.type;
+ map->m_la = (m.lcn << lclusterbits) | m.clusterofs;
+ /*
+ * For ztailpacking files, in order to inline data more
+ * effectively, special EOF lclusters are now supported
+ * which can have three parts at most.
+ */
+ if (ztailpacking && end > inode->i_size)
+ end = inode->i_size;
+ } else {
+ if (m.type != Z_EROFS_LCLUSTER_TYPE_NONHEAD) {
+ /* m.lcn should be >= 1 if endoff < m.clusterofs */
+ if (!m.lcn) {
+ erofs_err(sb, "invalid logical cluster 0 at nid %llu",
+ vi->nid);
+ err = -EFSCORRUPTED;
+ goto unmap_out;
+ }
+ end = (m.lcn << lclusterbits) | m.clusterofs;
+ map->m_flags |= EROFS_MAP_FULL_MAPPED;
+ m.delta[0] = 1;
}
- end = (m.lcn << lclusterbits) | m.clusterofs;
- map->m_flags |= EROFS_MAP_FULL_MAPPED;
- m.delta[0] = 1;
- fallthrough;
- case Z_EROFS_LCLUSTER_TYPE_NONHEAD:
/* get the corresponding first chunk */
err = z_erofs_extent_lookback(&m, m.delta[0]);
if (err)
goto unmap_out;
- break;
- default:
- erofs_err(sb, "unknown type %u @ offset %llu of nid %llu",
- m.type, ofs, vi->nid);
- err = -EOPNOTSUPP;
- goto unmap_out;
}
if (m.partialref)
map->m_flags |= EROFS_MAP_PARTIAL_REF;
@@ -569,6 +551,7 @@ static int z_erofs_fill_inode_lazy(struct inode *inode)
goto done;
}
vi->z_advise = le16_to_cpu(h->h_advise);
+ vi->z_lclusterbits = sb->s_blocksize_bits + (h->h_clusterbits & 15);
vi->z_algorithmtype[0] = h->h_algorithmtype & 15;
vi->z_algorithmtype[1] = h->h_algorithmtype >> 4;
if (vi->z_advise & Z_EROFS_ADVISE_FRAGMENT_PCLUSTER)
@@ -585,7 +568,6 @@ static int z_erofs_fill_inode_lazy(struct inode *inode)
goto out_put_metabuf;
}
- vi->z_logical_clusterbits = sb->s_blocksize_bits + (h->h_clusterbits & 7);
if (!erofs_sb_has_big_pcluster(EROFS_SB(sb)) &&
vi->z_advise & (Z_EROFS_ADVISE_BIG_PCLUSTER_1 |
Z_EROFS_ADVISE_BIG_PCLUSTER_2)) {
diff --git a/fs/eventfd.c b/fs/eventfd.c
index 22c934f3a080..76129bfcd663 100644
--- a/fs/eventfd.c
+++ b/fs/eventfd.c
@@ -347,13 +347,10 @@ EXPORT_SYMBOL_GPL(eventfd_fget);
*/
struct eventfd_ctx *eventfd_ctx_fdget(int fd)
{
- struct eventfd_ctx *ctx;
- struct fd f = fdget(fd);
- if (!fd_file(f))
+ CLASS(fd, f)(fd);
+ if (fd_empty(f))
return ERR_PTR(-EBADF);
- ctx = eventfd_ctx_fileget(fd_file(f));
- fdput(f);
- return ctx;
+ return eventfd_ctx_fileget(fd_file(f));
}
EXPORT_SYMBOL_GPL(eventfd_ctx_fdget);
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 075aa8793aaa..a860cb54658a 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -2207,25 +2207,22 @@ int do_epoll_ctl(int epfd, int op, int fd, struct epoll_event *epds,
{
int error;
int full_check = 0;
- struct fd f, tf;
struct eventpoll *ep;
struct epitem *epi;
struct eventpoll *tep = NULL;
- error = -EBADF;
- f = fdget(epfd);
- if (!fd_file(f))
- goto error_return;
+ CLASS(fd, f)(epfd);
+ if (fd_empty(f))
+ return -EBADF;
/* Get the "struct file *" for the target file */
- tf = fdget(fd);
- if (!fd_file(tf))
- goto error_fput;
+ CLASS(fd, tf)(fd);
+ if (fd_empty(tf))
+ return -EBADF;
/* The target file descriptor must support poll */
- error = -EPERM;
if (!file_can_poll(fd_file(tf)))
- goto error_tgt_fput;
+ return -EPERM;
/* Check if EPOLLWAKEUP is allowed */
if (ep_op_has_event(op))
@@ -2344,12 +2341,6 @@ int do_epoll_ctl(int epfd, int op, int fd, struct epoll_event *epds,
loop_check_gen++;
mutex_unlock(&epnested_mutex);
}
-
- fdput(tf);
-error_fput:
- fdput(f);
-error_return:
-
return error;
}
diff --git a/fs/ext4/mballoc-test.c b/fs/ext4/mballoc-test.c
index 0f81094fc0db..6f506118d18e 100644
--- a/fs/ext4/mballoc-test.c
+++ b/fs/ext4/mballoc-test.c
@@ -362,7 +362,6 @@ static int mbt_kunit_init(struct kunit *test)
return ret;
}
- test->priv = sb;
kunit_activate_static_stub(test,
ext4_read_block_bitmap_nowait,
ext4_read_block_bitmap_nowait_stub);
@@ -383,6 +382,8 @@ static int mbt_kunit_init(struct kunit *test)
return -ENOMEM;
}
+ test->priv = sb;
+
return 0;
}
@@ -390,6 +391,9 @@ static void mbt_kunit_exit(struct kunit *test)
{
struct super_block *sb = (struct super_block *)test->priv;
+ if (!sb)
+ return;
+
mbt_mb_release(sb);
mbt_ctx_release(sb);
mbt_ext4_free_super_block(sb);
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index ab0d9ed02092..b094fdaf318d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1858,6 +1858,9 @@ struct f2fs_sb_info {
spinlock_t iostat_lat_lock;
struct iostat_lat_info *iostat_io_lat;
#endif
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ struct lock_class_key cp_global_sem_key;
+#endif
};
/* Definitions to access f2fs_sb_info */
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index 3b91a95d4276..758ec6d3ae84 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -790,7 +790,7 @@ int f2fs_read_inline_dir(struct file *file, struct dir_context *ctx,
int f2fs_inline_data_fiemap(struct inode *inode,
struct fiemap_extent_info *fieinfo, __u64 start, __u64 len)
{
- __u64 byteaddr, ilen;
+ __u64 byteaddr = 0, ilen;
__u32 flags = FIEMAP_EXTENT_DATA_INLINE | FIEMAP_EXTENT_NOT_ALIGNED |
FIEMAP_EXTENT_LAST;
struct node_info ni;
@@ -823,9 +823,14 @@ int f2fs_inline_data_fiemap(struct inode *inode,
if (err)
goto out;
- byteaddr = (__u64)ni.blk_addr << inode->i_sb->s_blocksize_bits;
- byteaddr += (char *)inline_data_addr(inode, ipage) -
- (char *)F2FS_INODE(ipage);
+ if (__is_valid_data_blkaddr(ni.blk_addr)) {
+ byteaddr = (__u64)ni.blk_addr << inode->i_sb->s_blocksize_bits;
+ byteaddr += (char *)inline_data_addr(inode, ipage) -
+ (char *)F2FS_INODE(ipage);
+ } else {
+ f2fs_bug_on(F2FS_I_SB(inode), ni.blk_addr != NEW_ADDR);
+ flags |= FIEMAP_EXTENT_DELALLOC | FIEMAP_EXTENT_UNKNOWN;
+ }
err = fiemap_fill_next_extent(fieinfo, start, byteaddr, ilen, flags);
trace_f2fs_fiemap(inode, start, byteaddr, ilen, flags, err);
out:
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index f25a259f37f1..1bce35d6f4e2 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4490,6 +4490,11 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent)
init_f2fs_rwsem(&sbi->gc_lock);
mutex_init(&sbi->writepages);
init_f2fs_rwsem(&sbi->cp_global_sem);
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ lockdep_register_key(&sbi->cp_global_sem_key);
+ lockdep_set_class(&sbi->cp_global_sem.internal_rwsem,
+ &sbi->cp_global_sem_key);
+#endif
init_f2fs_rwsem(&sbi->node_write);
init_f2fs_rwsem(&sbi->node_change);
spin_lock_init(&sbi->stat_lock);
@@ -4963,6 +4968,9 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent)
free_sbi:
if (sbi->s_chksum_driver)
crypto_free_shash(sbi->s_chksum_driver);
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ lockdep_unregister_key(&sbi->cp_global_sem_key);
+#endif
kfree(sbi);
sb->s_fs_info = NULL;
@@ -5015,6 +5023,9 @@ static void kill_f2fs_super(struct super_block *sb)
/* Release block devices last, after fscrypt_destroy_keyring(). */
if (sbi) {
destroy_device_list(sbi);
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+ lockdep_unregister_key(&sbi->cp_global_sem_key);
+#endif
kfree(sbi);
sb->s_fs_info = NULL;
}
diff --git a/fs/f2fs/sysfs.c b/fs/f2fs/sysfs.c
index 7b490242bd05..4e1021eb372e 100644
--- a/fs/f2fs/sysfs.c
+++ b/fs/f2fs/sysfs.c
@@ -358,10 +358,12 @@ static ssize_t f2fs_sbi_show(struct f2fs_attr *a,
if (!strcmp(a->attr.name, "extension_list")) {
__u8 (*extlist)[F2FS_EXTENSION_LEN] =
sbi->raw_super->extension_list;
- int cold_count = le32_to_cpu(sbi->raw_super->extension_count);
- int hot_count = sbi->raw_super->hot_ext_count;
+ int cold_count, hot_count;
int len = 0, i;
+ f2fs_down_read(&sbi->sb_lock);
+ cold_count = le32_to_cpu(sbi->raw_super->extension_count);
+ hot_count = sbi->raw_super->hot_ext_count;
len += sysfs_emit_at(buf, len, "cold file extension:\n");
for (i = 0; i < cold_count; i++)
len += sysfs_emit_at(buf, len, "%s\n", extlist[i]);
@@ -369,6 +371,7 @@ static ssize_t f2fs_sbi_show(struct f2fs_attr *a,
len += sysfs_emit_at(buf, len, "hot file extension:\n");
for (i = cold_count; i < cold_count + hot_count; i++)
len += sysfs_emit_at(buf, len, "%s\n", extlist[i]);
+ f2fs_up_read(&sbi->sb_lock);
return len;
}
diff --git a/fs/fhandle.c b/fs/fhandle.c
index ff90f8203015..38d803a28ab9 100644
--- a/fs/fhandle.c
+++ b/fs/fhandle.c
@@ -139,12 +139,11 @@ static int get_path_from_fd(int fd, struct path *root)
path_get(root);
spin_unlock(&fs->lock);
} else {
- struct fd f = fdget(fd);
- if (!fd_file(f))
+ CLASS(fd, f)(fd);
+ if (fd_empty(f))
return -EBADF;
*root = fd_file(f)->f_path;
path_get(root);
- fdput(f);
}
return 0;
diff --git a/fs/gfs2/inode.c b/fs/gfs2/inode.c
index c37079718fdd..e6fe1a95d930 100644
--- a/fs/gfs2/inode.c
+++ b/fs/gfs2/inode.c
@@ -893,7 +893,7 @@ static int gfs2_create_inode(struct inode *dir, struct dentry *dentry,
goto fail_gunlock4;
mark_inode_dirty(inode);
- d_instantiate(dentry, inode);
+ d_instantiate_new(dentry, inode);
/* After instantiate, errors should result in evict which will destroy
* both inode and iopen glocks properly. */
if (file) {
@@ -905,7 +905,6 @@ static int gfs2_create_inode(struct inode *dir, struct dentry *dentry,
gfs2_glock_dq_uninit(&gh);
gfs2_glock_put(io_gl);
gfs2_qa_put(dip);
- unlock_new_inode(inode);
return error;
fail_gunlock4:
diff --git a/fs/gfs2/log.c b/fs/gfs2/log.c
index 115c4ac457e9..ecc5c59b8700 100644
--- a/fs/gfs2/log.c
+++ b/fs/gfs2/log.c
@@ -471,8 +471,9 @@ void gfs2_log_release(struct gfs2_sbd *sdp, unsigned int blks)
{
atomic_add(blks, &sdp->sd_log_blks_free);
trace_gfs2_log_blocks(sdp, blks);
- gfs2_assert_withdraw(sdp, atomic_read(&sdp->sd_log_blks_free) <=
- sdp->sd_jdesc->jd_blocks);
+ gfs2_assert_withdraw(sdp, !sdp->sd_jdesc ||
+ atomic_read(&sdp->sd_log_blks_free) <=
+ sdp->sd_jdesc->jd_blocks);
if (atomic_read(&sdp->sd_log_blks_needed))
wake_up(&sdp->sd_log_waitq);
}
@@ -1027,14 +1028,15 @@ static void trans_drain(struct gfs2_trans *tr)
}
/**
- * gfs2_log_flush - flush incore transaction(s)
+ * __gfs2_log_flush - flush incore transaction(s)
* @sdp: The filesystem
* @gl: The glock structure to flush. If NULL, flush the whole incore log
* @flags: The log header flags: GFS2_LOG_HEAD_FLUSH_* and debug flags
*
*/
-void gfs2_log_flush(struct gfs2_sbd *sdp, struct gfs2_glock *gl, u32 flags)
+static void __gfs2_log_flush(struct gfs2_sbd *sdp, struct gfs2_glock *gl,
+ u32 flags)
{
struct gfs2_trans *tr = NULL;
unsigned int reserved_blocks = 0, used_blocks = 0;
@@ -1042,7 +1044,6 @@ void gfs2_log_flush(struct gfs2_sbd *sdp, struct gfs2_glock *gl, u32 flags)
unsigned int first_log_head;
unsigned int reserved_revokes = 0;
- down_write(&sdp->sd_log_flush_lock);
trace_gfs2_log_flush(sdp, 1, flags);
repeat:
@@ -1154,7 +1155,6 @@ void gfs2_log_flush(struct gfs2_sbd *sdp, struct gfs2_glock *gl, u32 flags)
gfs2_assert_withdraw_delayed(sdp, used_blocks < reserved_blocks);
gfs2_log_release(sdp, reserved_blocks - used_blocks);
}
- up_write(&sdp->sd_log_flush_lock);
gfs2_trans_free(sdp, tr);
if (gfs2_withdrawing(sdp))
gfs2_withdraw(sdp);
@@ -1177,6 +1177,13 @@ void gfs2_log_flush(struct gfs2_sbd *sdp, struct gfs2_glock *gl, u32 flags)
goto out_end;
}
+void gfs2_log_flush(struct gfs2_sbd *sdp, struct gfs2_glock *gl, u32 flags)
+{
+ down_write(&sdp->sd_log_flush_lock);
+ __gfs2_log_flush(sdp, gl, flags);
+ up_write(&sdp->sd_log_flush_lock);
+}
+
/**
* gfs2_merge_trans - Merge a new transaction into a cached transaction
* @sdp: the filesystem
@@ -1319,19 +1326,25 @@ int gfs2_logd(void *data)
}
if (gfs2_jrnl_flush_reqd(sdp) || t == 0) {
+ down_write(&sdp->sd_log_flush_lock);
gfs2_ail1_empty(sdp, 0);
- gfs2_log_flush(sdp, NULL, GFS2_LOG_HEAD_FLUSH_NORMAL |
- GFS2_LFC_LOGD_JFLUSH_REQD);
+ __gfs2_log_flush(sdp, NULL,
+ GFS2_LOG_HEAD_FLUSH_NORMAL |
+ GFS2_LFC_LOGD_JFLUSH_REQD);
+ up_write(&sdp->sd_log_flush_lock);
}
if (test_bit(SDF_FORCE_AIL_FLUSH, &sdp->sd_flags) ||
gfs2_ail_flush_reqd(sdp)) {
clear_bit(SDF_FORCE_AIL_FLUSH, &sdp->sd_flags);
+ down_write(&sdp->sd_log_flush_lock);
gfs2_ail1_start(sdp);
gfs2_ail1_wait(sdp);
gfs2_ail1_empty(sdp, 0);
- gfs2_log_flush(sdp, NULL, GFS2_LOG_HEAD_FLUSH_NORMAL |
- GFS2_LFC_LOGD_AIL_FLUSH_REQD);
+ __gfs2_log_flush(sdp, NULL,
+ GFS2_LOG_HEAD_FLUSH_NORMAL |
+ GFS2_LFC_LOGD_AIL_FLUSH_REQD);
+ up_write(&sdp->sd_log_flush_lock);
}
t = gfs2_tune_get(sdp, gt_logd_secs) * HZ;
diff --git a/fs/ioctl.c b/fs/ioctl.c
index 6e0c954388d4..638a36be31c1 100644
--- a/fs/ioctl.c
+++ b/fs/ioctl.c
@@ -231,11 +231,11 @@ static int ioctl_fiemap(struct file *filp, struct fiemap __user *ufiemap)
static long ioctl_file_clone(struct file *dst_file, unsigned long srcfd,
u64 off, u64 olen, u64 destoff)
{
- struct fd src_file = fdget(srcfd);
+ CLASS(fd, src_file)(srcfd);
loff_t cloned;
int ret;
- if (!fd_file(src_file))
+ if (fd_empty(src_file))
return -EBADF;
cloned = vfs_clone_file_range(fd_file(src_file), off, dst_file, destoff,
olen, 0);
@@ -245,7 +245,6 @@ static long ioctl_file_clone(struct file *dst_file, unsigned long srcfd,
ret = -EINVAL;
else
ret = 0;
- fdput(src_file);
return ret;
}
@@ -892,22 +891,20 @@ static int do_vfs_ioctl(struct file *filp, unsigned int fd,
SYSCALL_DEFINE3(ioctl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
{
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
int error;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
error = security_file_ioctl(fd_file(f), cmd, arg);
if (error)
- goto out;
+ return error;
error = do_vfs_ioctl(fd_file(f), fd, cmd, arg);
if (error == -ENOIOCTLCMD)
error = vfs_ioctl(fd_file(f), cmd, arg);
-out:
- fdput(f);
return error;
}
@@ -950,15 +947,15 @@ EXPORT_SYMBOL(compat_ptr_ioctl);
COMPAT_SYSCALL_DEFINE3(ioctl, unsigned int, fd, unsigned int, cmd,
compat_ulong_t, arg)
{
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
int error;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
error = security_file_ioctl_compat(fd_file(f), cmd, arg);
if (error)
- goto out;
+ return error;
switch (cmd) {
/* FICLONE takes an int argument, so don't use compat_ptr() */
@@ -1009,10 +1006,6 @@ COMPAT_SYSCALL_DEFINE3(ioctl, unsigned int, fd, unsigned int, cmd,
error = -ENOTTY;
break;
}
-
- out:
- fdput(f);
-
return error;
}
#endif
diff --git a/fs/kernel_read_file.c b/fs/kernel_read_file.c
index 9ff37ae650ea..de32c95d823d 100644
--- a/fs/kernel_read_file.c
+++ b/fs/kernel_read_file.c
@@ -175,15 +175,11 @@ ssize_t kernel_read_file_from_fd(int fd, loff_t offset, void **buf,
size_t buf_size, size_t *file_size,
enum kernel_read_file_id id)
{
- struct fd f = fdget(fd);
- ssize_t ret = -EBADF;
+ CLASS(fd, f)(fd);
- if (!fd_file(f) || !(fd_file(f)->f_mode & FMODE_READ))
- goto out;
+ if (fd_empty(f) || !(fd_file(f)->f_mode & FMODE_READ))
+ return -EBADF;
- ret = kernel_read_file(fd_file(f), offset, buf, buf_size, file_size, id);
-out:
- fdput(f);
- return ret;
+ return kernel_read_file(fd_file(f), offset, buf, buf_size, file_size, id);
}
EXPORT_SYMBOL_GPL(kernel_read_file_from_fd);
diff --git a/fs/mbcache.c b/fs/mbcache.c
index e60a840999aa..90b0564c62d0 100644
--- a/fs/mbcache.c
+++ b/fs/mbcache.c
@@ -408,6 +408,7 @@ void mb_cache_destroy(struct mb_cache *cache)
{
struct mb_cache_entry *entry, *next;
+ cancel_work_sync(&cache->c_shrink_work);
shrinker_free(cache->c_shrink);
/*
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 154a14bb2d7f..429e4396e1b0 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -22,7 +22,7 @@
*
* Extract the page fragments from the given amount of the source iterator and
* build up a second iterator that refers to all of those bits. This allows
- * the original iterator to disposed of.
+ * the original iterator to be disposed of.
*
* @extraction_flags can have ITER_ALLOW_P2PDMA set to request peer-to-peer DMA be
* allowed on the pages extracted.
@@ -43,7 +43,7 @@ ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
unsigned int max_pages;
unsigned int npages = 0;
unsigned int i;
- ssize_t ret;
+ ssize_t ret = 0;
size_t count = orig_len, offset, len;
size_t bv_size, pg_size;
@@ -67,8 +67,8 @@ ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
ret = iov_iter_extract_pages(orig, &pages, count,
max_pages - npages, extraction_flags,
&offset);
- if (ret < 0) {
- pr_err("Couldn't get user pages (rc=%zd)\n", ret);
+ if (unlikely(ret <= 0)) {
+ ret = ret ?: -EIO;
break;
}
@@ -97,6 +97,13 @@ ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
npages += cur_npages;
}
+ if (ret < 0 && (ret == -ENOMEM || npages == 0)) {
+ for (i = 0; i < npages; i++)
+ unpin_user_page(bv[i].bv_page);
+ kvfree(bv);
+ return ret;
+ }
+
iov_iter_bvec(new, orig->data_source, bv, npages, orig_len - count);
return npages;
}
diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c
index 5d6edafbed20..d1889f608b5d 100644
--- a/fs/nfs/blocklayout/blocklayout.c
+++ b/fs/nfs/blocklayout/blocklayout.c
@@ -381,14 +381,13 @@ bl_write_pagelist(struct nfs_pgio_header *header, int sync)
sector_t isect, extent_length = 0;
struct parallel_io *par = NULL;
loff_t offset = header->args.offset;
- size_t count = header->args.count;
struct page **pages = header->args.pages;
int pg_index = header->args.pgbase >> PAGE_SHIFT;
unsigned int pg_len;
struct blk_plug plug;
int i;
- dprintk("%s enter, %zu@%lld\n", __func__, count, offset);
+ dprintk("%s enter, %u@%lld\n", __func__, header->args.count, offset);
/* At this point, header->page_aray is a (sequential) list of nfs_pages.
* We want to write each, and if there is an error set pnfs_error
@@ -429,7 +428,6 @@ bl_write_pagelist(struct nfs_pgio_header *header, int sync)
}
offset += pg_len;
- count -= pg_len;
isect += (pg_len >> SECTOR_SHIFT);
extent_length -= (pg_len >> SECTOR_SHIFT);
}
diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c
index fa77f78df681..4c66f6f99b2b 100644
--- a/fs/nilfs2/ioctl.c
+++ b/fs/nilfs2/ioctl.c
@@ -765,6 +765,12 @@ static int nilfs_ioctl_mark_blocks_dirty(struct the_nilfs *nilfs,
int ret, i;
for (i = 0; i < nmembs; i++) {
+ /*
+ * bd_oblocknr must never be 0 as block 0
+ * is never a valid GC target block
+ */
+ if (unlikely(!bdescs[i].bd_oblocknr))
+ return -EINVAL;
/* XXX: use macro or inline func to check liveness */
ret = nilfs_bmap_lookup_at_level(bmap,
bdescs[i].bd_offset,
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index 8e2d43fc6f7c..93c1619cdad6 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -1014,22 +1014,18 @@ static int fanotify_find_path(int dfd, const char __user *filename,
dfd, filename, flags);
if (filename == NULL) {
- struct fd f = fdget(dfd);
+ CLASS(fd, f)(dfd);
- ret = -EBADF;
- if (!fd_file(f))
- goto out;
+ if (fd_empty(f))
+ return -EBADF;
- ret = -ENOTDIR;
if ((flags & FAN_MARK_ONLYDIR) &&
- !(S_ISDIR(file_inode(fd_file(f))->i_mode))) {
- fdput(f);
- goto out;
- }
+ !(S_ISDIR(file_inode(fd_file(f))->i_mode)))
+ return -ENOTDIR;
*path = fd_file(f)->f_path;
path_get(path);
- fdput(f);
+ ret = 0;
} else {
unsigned int lookup_flags = 0;
@@ -1039,22 +1035,7 @@ static int fanotify_find_path(int dfd, const char __user *filename,
lookup_flags |= LOOKUP_DIRECTORY;
ret = user_path_at(dfd, filename, lookup_flags, path);
- if (ret)
- goto out;
- }
-
- /* you can only watch an inode if you have read permissions on it */
- ret = path_permission(path, MAY_READ);
- if (ret) {
- path_put(path);
- goto out;
}
-
- ret = security_path_notify(path, mask, obj_type);
- if (ret)
- path_put(path);
-
-out:
return ret;
}
@@ -1846,6 +1827,15 @@ static int do_fanotify_mark(int fanotify_fd, unsigned int flags, __u64 mask,
goto path_put_and_out;
}
+ /* you can only watch an inode if you have read permissions on it */
+ ret = path_permission(&path, MAY_READ);
+ if (ret)
+ goto path_put_and_out;
+
+ ret = security_path_notify(&path, mask, obj_type);
+ if (ret)
+ goto path_put_and_out;
+
if (fid_mode) {
ret = fanotify_test_fsid(path.dentry, flags, &__fsid);
if (ret)
diff --git a/fs/notify/inotify/inotify_user.c b/fs/notify/inotify/inotify_user.c
index 26839972f609..0ea4e99dc449 100644
--- a/fs/notify/inotify/inotify_user.c
+++ b/fs/notify/inotify/inotify_user.c
@@ -795,33 +795,26 @@ SYSCALL_DEFINE2(inotify_rm_watch, int, fd, __s32, wd)
{
struct fsnotify_group *group;
struct inotify_inode_mark *i_mark;
- struct fd f;
- int ret = -EINVAL;
+ CLASS(fd, f)(fd);
- f = fdget(fd);
- if (unlikely(!fd_file(f)))
+ if (fd_empty(f))
return -EBADF;
/* verify that this is indeed an inotify instance */
if (unlikely(fd_file(f)->f_op != &inotify_fops))
- goto out;
+ return -EINVAL;
group = fd_file(f)->private_data;
i_mark = inotify_idr_find(group, wd);
if (unlikely(!i_mark))
- goto out;
-
- ret = 0;
+ return -EINVAL;
fsnotify_destroy_mark(&i_mark->fsn_mark, group);
/* match ref taken by inotify_idr_find */
fsnotify_put_mark(&i_mark->fsn_mark);
-
-out:
- fdput(f);
- return ret;
+ return 0;
}
/*
diff --git a/fs/notify/mark.c b/fs/notify/mark.c
index 09e50fe5757c..a263305ed7c8 100644
--- a/fs/notify/mark.c
+++ b/fs/notify/mark.c
@@ -233,7 +233,12 @@ static struct inode *fsnotify_update_iref(struct fsnotify_mark_connector *conn,
return inode;
}
-static void *__fsnotify_recalc_mask(struct fsnotify_mark_connector *conn)
+/*
+ * Calculate mask of events for a list of marks.
+ *
+ * Return true if any of the attached marks want to hold an inode reference.
+ */
+static bool __fsnotify_recalc_mask(struct fsnotify_mark_connector *conn)
{
u32 new_mask = 0;
bool want_iref = false;
@@ -257,6 +262,34 @@ static void *__fsnotify_recalc_mask(struct fsnotify_mark_connector *conn)
*/
WRITE_ONCE(*fsnotify_conn_mask_p(conn), new_mask);
+ return want_iref;
+}
+
+/*
+ * Calculate mask of events for a list of marks after attach/modify mark
+ * and get an inode reference for the connector if needed.
+ *
+ * A concurrent add of evictable mark and detach of non-evictable mark can
+ * lead to __fsnotify_recalc_mask() returning false want_iref, but in this
+ * case we defer clearing iref to fsnotify_recalc_mask_clear_iref() called
+ * from fsnotify_put_mark().
+ */
+static void fsnotify_recalc_mask_set_iref(struct fsnotify_mark_connector *conn)
+{
+ bool has_iref = conn->flags & FSNOTIFY_CONN_FLAG_HAS_IREF;
+ bool want_iref = __fsnotify_recalc_mask(conn) || has_iref;
+
+ (void) fsnotify_update_iref(conn, want_iref);
+}
+
+/*
+ * Calculate mask of events for a list of marks after detach mark
+ * and return the inode object if its reference is no longer needed.
+ */
+static void *fsnotify_recalc_mask_clear_iref(struct fsnotify_mark_connector *conn)
+{
+ bool want_iref = __fsnotify_recalc_mask(conn);
+
return fsnotify_update_iref(conn, want_iref);
}
@@ -293,7 +326,7 @@ void fsnotify_recalc_mask(struct fsnotify_mark_connector *conn)
spin_lock(&conn->lock);
update_children = !fsnotify_conn_watches_children(conn);
- __fsnotify_recalc_mask(conn);
+ fsnotify_recalc_mask_set_iref(conn);
update_children &= fsnotify_conn_watches_children(conn);
spin_unlock(&conn->lock);
/*
@@ -408,7 +441,7 @@ void fsnotify_put_mark(struct fsnotify_mark *mark)
/* Update watched objects after detaching mark */
if (sb)
fsnotify_update_sb_watchers(sb, conn);
- objp = __fsnotify_recalc_mask(conn);
+ objp = fsnotify_recalc_mask_clear_iref(conn);
type = conn->type;
}
WRITE_ONCE(mark->connector, NULL);
diff --git a/fs/ntfs3/dir.c b/fs/ntfs3/dir.c
index 600e66035c1b..522ebc14b1fb 100644
--- a/fs/ntfs3/dir.c
+++ b/fs/ntfs3/dir.c
@@ -425,8 +425,7 @@ static int ntfs_readdir(struct file *file, struct dir_context *ctx)
if (!dir_emit_dots(file, ctx))
return 0;
- /* Allocate PATH_MAX bytes. */
- name = __getname();
+ name = kmalloc(PATH_MAX, GFP_KERNEL);
if (!name)
return -ENOMEM;
@@ -504,7 +503,7 @@ static int ntfs_readdir(struct file *file, struct dir_context *ctx)
out:
- __putname(name);
+ kfree(name);
put_indx_node(node);
if (err == 1) {
diff --git a/fs/ntfs3/fsntfs.c b/fs/ntfs3/fsntfs.c
index 37c5d9a1f77b..5972f160e566 100644
--- a/fs/ntfs3/fsntfs.c
+++ b/fs/ntfs3/fsntfs.c
@@ -2670,7 +2670,7 @@ int ntfs_set_label(struct ntfs_sb_info *sbi, u8 *label, int len)
u32 uni_bytes;
struct ntfs_inode *ni = sbi->volume.ni;
/* Allocate PATH_MAX bytes. */
- struct cpu_str *uni = __getname();
+ struct cpu_str *uni = kmalloc(PATH_MAX, GFP_KERNEL);
if (!uni)
return -ENOMEM;
@@ -2714,6 +2714,6 @@ int ntfs_set_label(struct ntfs_sb_info *sbi, u8 *label, int len)
err = _ni_write_inode(&ni->vfs_inode, 0);
out:
- __putname(uni);
+ kfree(uni);
return err;
}
\ No newline at end of file
diff --git a/fs/ntfs3/inode.c b/fs/ntfs3/inode.c
index 8113d47b0ceb..b50c9dff327d 100644
--- a/fs/ntfs3/inode.c
+++ b/fs/ntfs3/inode.c
@@ -1301,7 +1301,7 @@ int ntfs_create_inode(struct mnt_idmap *idmap, struct inode *dir,
fa |= FILE_ATTRIBUTE_READONLY;
/* Allocate PATH_MAX bytes. */
- new_de = kmem_cache_zalloc(names_cachep, GFP_KERNEL);
+ new_de = kzalloc(PATH_MAX, GFP_KERNEL);
if (!new_de) {
err = -ENOMEM;
goto out1;
@@ -1713,7 +1713,7 @@ int ntfs_create_inode(struct mnt_idmap *idmap, struct inode *dir,
ntfs_mark_rec_free(sbi, ino, false);
out2:
- __putname(new_de);
+ kfree(new_de);
kfree(rp);
out1:
@@ -1734,7 +1734,7 @@ int ntfs_link_inode(struct inode *inode, struct dentry *dentry)
struct NTFS_DE *de;
/* Allocate PATH_MAX bytes. */
- de = kmem_cache_zalloc(names_cachep, GFP_KERNEL);
+ de = kzalloc(PATH_MAX, GFP_KERNEL);
if (!de)
return -ENOMEM;
@@ -1748,7 +1748,7 @@ int ntfs_link_inode(struct inode *inode, struct dentry *dentry)
err = ni_add_name(ntfs_i(d_inode(dentry->d_parent)), ni, de);
out:
- __putname(de);
+ kfree(de);
return err;
}
@@ -1771,8 +1771,7 @@ int ntfs_unlink_inode(struct inode *dir, const struct dentry *dentry)
if (ntfs_is_meta_file(sbi, ni->mi.rno))
return -EINVAL;
- /* Allocate PATH_MAX bytes. */
- de = kmem_cache_zalloc(names_cachep, GFP_KERNEL);
+ de = kzalloc(PATH_MAX, GFP_KERNEL);
if (!de)
return -ENOMEM;
@@ -1808,7 +1807,7 @@ int ntfs_unlink_inode(struct inode *dir, const struct dentry *dentry)
out:
ni_unlock(ni);
- __putname(de);
+ kfree(de);
return err;
}
diff --git a/fs/ntfs3/namei.c b/fs/ntfs3/namei.c
index 71a5a959a48c..fa4f7d9f3845 100644
--- a/fs/ntfs3/namei.c
+++ b/fs/ntfs3/namei.c
@@ -68,7 +68,7 @@ static struct dentry *ntfs_lookup(struct inode *dir, struct dentry *dentry,
u32 flags)
{
struct ntfs_inode *ni = ntfs_i(dir);
- struct cpu_str *uni = __getname();
+ struct cpu_str *uni = kmalloc(PATH_MAX, GFP_KERNEL);
struct inode *inode;
int err;
@@ -85,7 +85,7 @@ static struct dentry *ntfs_lookup(struct inode *dir, struct dentry *dentry,
inode = dir_search_u(dir, uni, NULL);
ni_unlock(ni);
}
- __putname(uni);
+ kfree(uni);
}
/*
@@ -287,8 +287,7 @@ static int ntfs_rename(struct mnt_idmap *idmap, struct inode *dir,
return err;
}
- /* Allocate PATH_MAX bytes. */
- de = __getname();
+ de = kmalloc(PATH_MAX, GFP_KERNEL);
if (!de)
return -ENOMEM;
@@ -333,7 +332,7 @@ static int ntfs_rename(struct mnt_idmap *idmap, struct inode *dir,
ni_unlock(ni);
ni_unlock(dir_ni);
out:
- __putname(de);
+ kfree(de);
return err;
}
@@ -391,7 +390,7 @@ static int ntfs_d_hash(const struct dentry *dentry, struct qstr *name)
/*
* Try slow way with current upcase table
*/
- uni = kmem_cache_alloc(names_cachep, GFP_NOWAIT);
+ uni = kmalloc(PATH_MAX, GFP_NOWAIT);
if (!uni)
return -ENOMEM;
@@ -413,7 +412,7 @@ static int ntfs_d_hash(const struct dentry *dentry, struct qstr *name)
err = 0;
out:
- kmem_cache_free(names_cachep, uni);
+ kfree(uni);
return err;
}
@@ -452,7 +451,7 @@ static int ntfs_d_compare(const struct dentry *dentry, unsigned int len1,
* Try slow way with current upcase table
*/
sbi = dentry->d_sb->s_fs_info;
- uni1 = __getname();
+ uni1 = kmalloc(PATH_MAX, GFP_NOWAIT);
if (!uni1)
return -ENOMEM;
@@ -482,7 +481,7 @@ static int ntfs_d_compare(const struct dentry *dentry, unsigned int len1,
ret = !ntfs_cmp_names_cpu(uni1, uni2, sbi->upcase, false) ? 0 : 1;
out:
- __putname(uni1);
+ kfree(uni1);
return ret;
}
diff --git a/fs/ntfs3/super.c b/fs/ntfs3/super.c
index 89d126c155c7..1af1500ec24b 100644
--- a/fs/ntfs3/super.c
+++ b/fs/ntfs3/super.c
@@ -1235,8 +1235,13 @@ static int ntfs_fill_super(struct super_block *sb, struct fs_context *fc)
le32_to_cpu(attr->res.data_size) >> 1,
UTF16_LITTLE_ENDIAN, sbi->volume.label,
sizeof(sbi->volume.label));
- if (err < 0)
+ if (err < 0) {
sbi->volume.label[0] = 0;
+ } else if (err >= sizeof(sbi->volume.label)) {
+ sbi->volume.label[sizeof(sbi->volume.label) - 1] = 0;
+ } else {
+ sbi->volume.label[err] = 0;
+ }
} else {
/* Should we break mounting here? */
//err = -EINVAL;
diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c
index e0055dcf8fe3..6861c09d66d7 100644
--- a/fs/ntfs3/xattr.c
+++ b/fs/ntfs3/xattr.c
@@ -552,8 +552,7 @@ struct posix_acl *ntfs_get_acl(struct mnt_idmap *idmap, struct dentry *dentry,
int err;
void *buf;
- /* Allocate PATH_MAX bytes. */
- buf = __getname();
+ buf = kmalloc(PATH_MAX, GFP_KERNEL);
if (!buf)
return ERR_PTR(-ENOMEM);
@@ -584,7 +583,7 @@ struct posix_acl *ntfs_get_acl(struct mnt_idmap *idmap, struct dentry *dentry,
if (!IS_ERR(acl))
set_cached_acl(inode, type, acl);
- __putname(buf);
+ kfree(buf);
return acl;
}
diff --git a/fs/ocfs2/dlm/dlmdomain.c b/fs/ocfs2/dlm/dlmdomain.c
index 2018501b2249..fe17f1e548c1 100644
--- a/fs/ocfs2/dlm/dlmdomain.c
+++ b/fs/ocfs2/dlm/dlmdomain.c
@@ -980,6 +980,14 @@ static int dlm_match_regions(struct dlm_ctxt *dlm,
goto bail;
}
+ if (qr->qr_numregions > O2NM_MAX_REGIONS) {
+ mlog(ML_ERROR, "Domain %s: Joining node %d has invalid "
+ "number of heartbeat regions %u\n",
+ qr->qr_domain, qr->qr_node, qr->qr_numregions);
+ status = -EINVAL;
+ goto bail;
+ }
+
r = remote;
for (i = 0; i < qr->qr_numregions; ++i) {
mlog(0, "Region %.*s\n", O2HB_MAX_REGION_NAME_LEN, r);
@@ -994,7 +1002,7 @@ static int dlm_match_regions(struct dlm_ctxt *dlm,
for (i = 0; i < localnr; ++i) {
foundit = 0;
r = remote;
- for (j = 0; j <= qr->qr_numregions; ++j) {
+ for (j = 0; j < qr->qr_numregions; ++j) {
if (!memcmp(l, r, O2HB_MAX_REGION_NAME_LEN)) {
foundit = 1;
break;
diff --git a/fs/ocfs2/ioctl.c b/fs/ocfs2/ioctl.c
index 71beef7f8a60..6bafb17aa9fb 100644
--- a/fs/ocfs2/ioctl.c
+++ b/fs/ocfs2/ioctl.c
@@ -443,13 +443,16 @@ static int ocfs2_info_freefrag_scan_chain(struct ocfs2_super *osb,
struct buffer_head *bh = NULL;
struct ocfs2_group_desc *bg = NULL;
- unsigned int max_bits, num_clusters;
+ unsigned int max_bits, max_bitmap_bits, num_clusters;
unsigned int offset = 0, cluster, chunk;
unsigned int chunk_free, last_chunksize = 0;
if (!le32_to_cpu(rec->c_free))
goto bail;
+ max_bitmap_bits = 8 * ocfs2_group_bitmap_size(osb->sb, 0,
+ osb->s_feature_incompat);
+
do {
if (!bg)
blkno = le64_to_cpu(rec->c_blkno);
@@ -481,6 +484,19 @@ static int ocfs2_info_freefrag_scan_chain(struct ocfs2_super *osb,
continue;
max_bits = le16_to_cpu(bg->bg_bits);
+
+ /*
+ * Non-coherent scans read raw blocks and do not get the
+ * bg_bits validation from
+ * ocfs2_read_group_descriptor().
+ */
+ if (max_bits > max_bitmap_bits) {
+ mlog(ML_ERROR,
+ "Group desc #%llu has %u bits, max bitmap bits %u\n",
+ (unsigned long long)blkno, max_bits, max_bitmap_bits);
+ max_bits = max_bitmap_bits;
+ }
+
offset = 0;
for (chunk = 0; chunk < chunks_in_group; chunk++) {
diff --git a/fs/ocfs2/resize.c b/fs/ocfs2/resize.c
index ed7ed15ad9a7..583a411557ab 100644
--- a/fs/ocfs2/resize.c
+++ b/fs/ocfs2/resize.c
@@ -508,14 +508,14 @@ int ocfs2_group_add(struct inode *inode, struct ocfs2_new_group_input *input)
goto out_unlock;
}
- ocfs2_set_new_buffer_uptodate(INODE_CACHE(inode), group_bh);
-
ret = ocfs2_verify_group_and_input(main_bm_inode, fe, input, group_bh);
if (ret) {
mlog_errno(ret);
goto out_free_group_bh;
}
+ ocfs2_set_new_buffer_uptodate(INODE_CACHE(main_bm_inode), group_bh);
+
trace_ocfs2_group_add((unsigned long long)input->group,
input->chain, input->clusters, input->frees);
@@ -523,7 +523,7 @@ int ocfs2_group_add(struct inode *inode, struct ocfs2_new_group_input *input)
if (IS_ERR(handle)) {
mlog_errno(PTR_ERR(handle));
ret = -EINVAL;
- goto out_free_group_bh;
+ goto out_remove_cache;
}
cl_bpc = le16_to_cpu(fe->id2.i_chain.cl_bpc);
@@ -577,9 +577,11 @@ int ocfs2_group_add(struct inode *inode, struct ocfs2_new_group_input *input)
out_commit:
ocfs2_commit_trans(osb, handle);
-out_free_group_bh:
+out_remove_cache:
if (ret < 0)
- ocfs2_remove_from_cache(INODE_CACHE(inode), group_bh);
+ ocfs2_remove_from_cache(INODE_CACHE(main_bm_inode), group_bh);
+
+out_free_group_bh:
brelse(group_bh);
out_unlock:
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index b1a18a944c85..069732e657d7 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -907,8 +907,8 @@ static int ocfs2_xattr_list_entry(struct super_block *sb,
total_len = prefix_len + name_len + 1;
*result += total_len;
- /* we are just looking for how big our buffer needs to be */
- if (!size)
+ /* No buffer means we are only looking for the required size. */
+ if (!buffer)
return 0;
if (*result > size)
diff --git a/fs/omfs/inode.c b/fs/omfs/inode.c
index d6cd81163030..eb5a5fb26a79 100644
--- a/fs/omfs/inode.c
+++ b/fs/omfs/inode.c
@@ -512,6 +512,12 @@ static int omfs_fill_super(struct super_block *sb, void *data, int silent)
goto out_brelse_bh;
}
+ if (sbi->s_sys_blocksize < OMFS_DIR_START) {
+ printk(KERN_ERR "omfs: sysblock size (%d) is too small\n",
+ sbi->s_sys_blocksize);
+ goto out_brelse_bh;
+ }
+
if (sbi->s_blocksize < sbi->s_sys_blocksize ||
sbi->s_blocksize > OMFS_MAX_BLOCK_SIZE) {
printk(KERN_ERR "omfs: block size (%d) is out of range\n",
diff --git a/fs/open.c b/fs/open.c
index de1ea1b2f6ef..be7b55260a75 100644
--- a/fs/open.c
+++ b/fs/open.c
@@ -349,14 +349,12 @@ EXPORT_SYMBOL_GPL(vfs_fallocate);
int ksys_fallocate(int fd, int mode, loff_t offset, loff_t len)
{
- struct fd f = fdget(fd);
- int error = -EBADF;
+ CLASS(fd, f)(fd);
- if (fd_file(f)) {
- error = vfs_fallocate(fd_file(f), mode, offset, len);
- fdput(f);
- }
- return error;
+ if (fd_empty(f))
+ return -EBADF;
+
+ return vfs_fallocate(fd_file(f), mode, offset, len);
}
SYSCALL_DEFINE4(fallocate, int, fd, int, mode, loff_t, offset, loff_t, len)
@@ -671,14 +669,12 @@ int vfs_fchmod(struct file *file, umode_t mode)
SYSCALL_DEFINE2(fchmod, unsigned int, fd, umode_t, mode)
{
- struct fd f = fdget(fd);
- int err = -EBADF;
+ CLASS(fd, f)(fd);
- if (fd_file(f)) {
- err = vfs_fchmod(fd_file(f), mode);
- fdput(f);
- }
- return err;
+ if (fd_empty(f))
+ return -EBADF;
+
+ return vfs_fchmod(fd_file(f), mode);
}
static int do_fchmodat(int dfd, const char __user *filename, umode_t mode,
@@ -865,14 +861,12 @@ int vfs_fchown(struct file *file, uid_t user, gid_t group)
int ksys_fchown(unsigned int fd, uid_t user, gid_t group)
{
- struct fd f = fdget(fd);
- int error = -EBADF;
+ CLASS(fd, f)(fd);
- if (fd_file(f)) {
- error = vfs_fchown(fd_file(f), user, group);
- fdput(f);
- }
- return error;
+ if (fd_empty(f))
+ return -EBADF;
+
+ return vfs_fchown(fd_file(f), user, group);
}
SYSCALL_DEFINE3(fchown, unsigned int, fd, uid_t, user, gid_t, group)
diff --git a/fs/pstore/ram_core.c b/fs/pstore/ram_core.c
index 7b6d6378a3b8..95675d4bab14 100644
--- a/fs/pstore/ram_core.c
+++ b/fs/pstore/ram_core.c
@@ -489,6 +489,10 @@ static void *persistent_ram_iomap(phys_addr_t start, size_t size,
else
va = ioremap_wc(start, size);
+ /* We must release the mem region if ioremap fails. */
+ if (!va)
+ release_mem_region(start, size);
+
/*
* Since request_mem_region() and ioremap() are byte-granularity
* there is no need handle anything special like we do when the
diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
index 94825180385a..7c3095622872 100644
--- a/fs/quota/dquot.c
+++ b/fs/quota/dquot.c
@@ -364,6 +364,31 @@ static inline int dquot_active(struct dquot *dquot)
return test_bit(DQ_ACTIVE_B, &dquot->dq_flags);
}
+static struct dquot *__dqgrab(struct dquot *dquot)
+{
+ lockdep_assert_held(&dq_list_lock);
+ if (!atomic_read(&dquot->dq_count))
+ remove_free_dquot(dquot);
+ atomic_inc(&dquot->dq_count);
+ return dquot;
+}
+
+/*
+ * Get reference to dquot when we got pointer to it by some other means. The
+ * dquot has to be active and the caller has to make sure it cannot get
+ * deactivated under our hands.
+ */
+struct dquot *dqgrab(struct dquot *dquot)
+{
+ spin_lock(&dq_list_lock);
+ WARN_ON_ONCE(!dquot_active(dquot));
+ dquot = __dqgrab(dquot);
+ spin_unlock(&dq_list_lock);
+
+ return dquot;
+}
+EXPORT_SYMBOL_GPL(dqgrab);
+
static inline int dquot_dirty(struct dquot *dquot)
{
return test_bit(DQ_MOD_B, &dquot->dq_flags);
@@ -642,15 +667,14 @@ int dquot_scan_active(struct super_block *sb,
continue;
if (dquot->dq_sb != sb)
continue;
- /* Now we have active dquot so we can just increase use count */
- atomic_inc(&dquot->dq_count);
+ __dqgrab(dquot);
spin_unlock(&dq_list_lock);
dqput(old_dquot);
old_dquot = dquot;
/*
* ->release_dquot() can be racing with us. Our reference
- * protects us from new calls to it so just wait for any
- * outstanding call and recheck the DQ_ACTIVE_B after that.
+ * protects us from dquot_release() proceeding so just wait for
+ * any outstanding call and recheck the DQ_ACTIVE_B after that.
*/
wait_on_dquot(dquot);
if (dquot_active(dquot)) {
@@ -718,7 +742,7 @@ int dquot_writeback_dquots(struct super_block *sb, int type)
/* Now we have active dquot from which someone is
* holding reference so we can safely just increase
* use count */
- dqgrab(dquot);
+ __dqgrab(dquot);
spin_unlock(&dq_list_lock);
err = dquot_write_dquot(dquot);
if (err && !ret)
@@ -964,9 +988,7 @@ struct dquot *dqget(struct super_block *sb, struct kqid qid)
spin_unlock(&dq_list_lock);
dqstats_inc(DQST_LOOKUPS);
} else {
- if (!atomic_read(&dquot->dq_count))
- remove_free_dquot(dquot);
- atomic_inc(&dquot->dq_count);
+ __dqgrab(dquot);
spin_unlock(&dq_list_lock);
dqstats_inc(DQST_CACHE_HITS);
dqstats_inc(DQST_LOOKUPS);
diff --git a/fs/read_write.c b/fs/read_write.c
index 46408bab9238..430c06993b75 100644
--- a/fs/read_write.c
+++ b/fs/read_write.c
@@ -1675,36 +1675,32 @@ SYSCALL_DEFINE6(copy_file_range, int, fd_in, loff_t __user *, off_in,
{
loff_t pos_in;
loff_t pos_out;
- struct fd f_in;
- struct fd f_out;
ssize_t ret = -EBADF;
- f_in = fdget(fd_in);
- if (!fd_file(f_in))
- goto out2;
+ CLASS(fd, f_in)(fd_in);
+ if (fd_empty(f_in))
+ return -EBADF;
- f_out = fdget(fd_out);
- if (!fd_file(f_out))
- goto out1;
+ CLASS(fd, f_out)(fd_out);
+ if (fd_empty(f_out))
+ return -EBADF;
- ret = -EFAULT;
if (off_in) {
if (copy_from_user(&pos_in, off_in, sizeof(loff_t)))
- goto out;
+ return -EFAULT;
} else {
pos_in = fd_file(f_in)->f_pos;
}
if (off_out) {
if (copy_from_user(&pos_out, off_out, sizeof(loff_t)))
- goto out;
+ return -EFAULT;
} else {
pos_out = fd_file(f_out)->f_pos;
}
- ret = -EINVAL;
if (flags != 0)
- goto out;
+ return -EINVAL;
ret = vfs_copy_file_range(fd_file(f_in), pos_in, fd_file(f_out), pos_out, len,
flags);
@@ -1726,12 +1722,6 @@ SYSCALL_DEFINE6(copy_file_range, int, fd_in, loff_t __user *, off_in,
fd_file(f_out)->f_pos = pos_out;
}
}
-
-out:
- fdput(f_out);
-out1:
- fdput(f_in);
-out2:
return ret;
}
diff --git a/fs/signalfd.c b/fs/signalfd.c
index 736bebf93591..d1a5f43ce466 100644
--- a/fs/signalfd.c
+++ b/fs/signalfd.c
@@ -288,20 +288,17 @@ static int do_signalfd4(int ufd, sigset_t *mask, int flags)
fd_install(ufd, file);
} else {
- struct fd f = fdget(ufd);
- if (!fd_file(f))
+ CLASS(fd, f)(ufd);
+ if (fd_empty(f))
return -EBADF;
ctx = fd_file(f)->private_data;
- if (fd_file(f)->f_op != &signalfd_fops) {
- fdput(f);
+ if (fd_file(f)->f_op != &signalfd_fops)
return -EINVAL;
- }
spin_lock_irq(¤t->sighand->siglock);
ctx->sigmask = *mask;
spin_unlock_irq(¤t->sighand->siglock);
wake_up(¤t->sighand->signalfd_wqh);
- fdput(f);
}
return ufd;
diff --git a/fs/smb/client/ioctl.c b/fs/smb/client/ioctl.c
index 2ce193609d8b..200af8bad57d 100644
--- a/fs/smb/client/ioctl.c
+++ b/fs/smb/client/ioctl.c
@@ -300,7 +300,7 @@ static int cifs_dump_full_key(struct cifs_tcon *tcon, struct smb3_full_key_debug
break;
case SMB2_ENCRYPTION_AES256_CCM:
case SMB2_ENCRYPTION_AES256_GCM:
- out.session_key_length = CIFS_SESS_KEY_SIZE;
+ out.session_key_length = ses->auth_key.len;
out.server_in_key_length = out.server_out_key_length = SMB3_GCM256_CRYPTKEY_SIZE;
break;
default:
diff --git a/fs/smb/client/smb2file.c b/fs/smb/client/smb2file.c
index b7ab18d4bedc..279aaa755106 100644
--- a/fs/smb/client/smb2file.c
+++ b/fs/smb/client/smb2file.c
@@ -27,10 +27,11 @@ static struct smb2_symlink_err_rsp *symlink_data(const struct kvec *iov)
{
struct smb2_err_rsp *err = iov->iov_base;
struct smb2_symlink_err_rsp *sym = ERR_PTR(-EINVAL);
+ u8 *end = (u8 *)err + iov->iov_len;
u32 len;
if (err->ErrorContextCount) {
- struct smb2_error_context_rsp *p, *end;
+ struct smb2_error_context_rsp *p;
len = (u32)err->ErrorContextCount * (offsetof(struct smb2_error_context_rsp,
ErrorContextData) +
@@ -39,25 +40,29 @@ static struct smb2_symlink_err_rsp *symlink_data(const struct kvec *iov)
return ERR_PTR(-EINVAL);
p = (struct smb2_error_context_rsp *)err->ErrorData;
- end = (struct smb2_error_context_rsp *)((u8 *)err + iov->iov_len);
- do {
+ while ((u8 *)p + sizeof(*p) <= end) {
if (le32_to_cpu(p->ErrorId) == SMB2_ERROR_ID_DEFAULT) {
- sym = (struct smb2_symlink_err_rsp *)&p->ErrorContextData;
+ sym = (struct smb2_symlink_err_rsp *)p->ErrorContextData;
break;
}
cifs_dbg(FYI, "%s: skipping unhandled error context: 0x%x\n",
__func__, le32_to_cpu(p->ErrorId));
len = ALIGN(le32_to_cpu(p->ErrorDataLength), 8);
- p = (struct smb2_error_context_rsp *)((u8 *)&p->ErrorContextData + len);
- } while (p < end);
+ if (len > end - ((u8 *)p + sizeof(*p)))
+ return ERR_PTR(-EINVAL);
+
+ p = (struct smb2_error_context_rsp *)(p->ErrorContextData + len);
+ }
} else if (le32_to_cpu(err->ByteCount) >= sizeof(*sym) &&
iov->iov_len >= SMB2_SYMLINK_STRUCT_SIZE) {
sym = (struct smb2_symlink_err_rsp *)err->ErrorData;
}
- if (!IS_ERR(sym) && (le32_to_cpu(sym->SymLinkErrorTag) != SYMLINK_ERROR_TAG ||
- le32_to_cpu(sym->ReparseTag) != IO_REPARSE_TAG_SYMLINK))
+ if (!IS_ERR(sym) &&
+ ((u8 *)sym + sizeof(*sym) > end ||
+ le32_to_cpu(sym->SymLinkErrorTag) != SYMLINK_ERROR_TAG ||
+ le32_to_cpu(sym->ReparseTag) != IO_REPARSE_TAG_SYMLINK))
sym = ERR_PTR(-EINVAL);
return sym;
@@ -82,8 +87,10 @@ int smb2_parse_symlink_response(struct cifs_sb_info *cifs_sb, const struct kvec
print_len = le16_to_cpu(sym->PrintNameLength);
print_offs = le16_to_cpu(sym->PrintNameOffset);
- if (iov->iov_len < SMB2_SYMLINK_STRUCT_SIZE + sub_offs + sub_len ||
- iov->iov_len < SMB2_SYMLINK_STRUCT_SIZE + print_offs + print_len)
+ if ((char *)sym->PathBuffer + sub_offs + sub_len >
+ (char *)iov->iov_base + iov->iov_len ||
+ (char *)sym->PathBuffer + print_offs + print_len >
+ (char *)iov->iov_base + iov->iov_len)
return -EINVAL;
return smb2_parse_native_symlink(path,
diff --git a/fs/smb/client/smb2pdu.h b/fs/smb/client/smb2pdu.h
index 076d9e83e1a0..3c09a58dfd07 100644
--- a/fs/smb/client/smb2pdu.h
+++ b/fs/smb/client/smb2pdu.h
@@ -79,7 +79,7 @@ struct smb2_symlink_err_rsp {
struct smb2_error_context_rsp {
__le32 ErrorDataLength;
__le32 ErrorId;
- __u8 ErrorContextData; /* ErrorDataLength long array */
+ __u8 ErrorContextData[] __counted_by_le(ErrorDataLength);
} __packed;
/* ErrorId values */
diff --git a/fs/smb/client/smb2transport.c b/fs/smb/client/smb2transport.c
index 87f189894b1e..6d7e1b4d2d55 100644
--- a/fs/smb/client/smb2transport.c
+++ b/fs/smb/client/smb2transport.c
@@ -334,7 +334,8 @@ smb2_calc_signature(struct smb_rqst *rqst, struct TCP_Server_Info *server,
}
static int generate_key(struct cifs_ses *ses, struct kvec label,
- struct kvec context, __u8 *key, unsigned int key_size)
+ struct kvec context, __u8 *key, unsigned int key_size,
+ unsigned int full_key_size)
{
unsigned char zero = 0x0;
__u8 i[4] = {0, 0, 0, 1};
@@ -355,7 +356,7 @@ static int generate_key(struct cifs_ses *ses, struct kvec label,
}
rc = crypto_shash_setkey(server->secmech.hmacsha256->tfm,
- ses->auth_key.response, SMB2_NTLMV2_SESSKEY_SIZE);
+ ses->auth_key.response, full_key_size);
if (rc) {
cifs_server_dbg(VFS, "%s: Could not set with session key\n", __func__);
goto smb3signkey_ret;
@@ -430,6 +431,7 @@ generate_smb3signingkey(struct cifs_ses *ses,
struct TCP_Server_Info *server,
const struct derivation_triplet *ptriplet)
{
+ unsigned int full_key_size = SMB2_NTLMV2_SESSKEY_SIZE;
int rc;
bool is_binding = false;
int chan_index = 0;
@@ -464,17 +466,31 @@ generate_smb3signingkey(struct cifs_ses *ses,
rc = generate_key(ses, ptriplet->signing.label,
ptriplet->signing.context,
ses->chans[chan_index].signkey,
- SMB3_SIGN_KEY_SIZE);
+ SMB3_SIGN_KEY_SIZE,
+ SMB2_NTLMV2_SESSKEY_SIZE);
if (rc)
return rc;
} else {
rc = generate_key(ses, ptriplet->signing.label,
ptriplet->signing.context,
ses->smb3signingkey,
- SMB3_SIGN_KEY_SIZE);
+ SMB3_SIGN_KEY_SIZE,
+ SMB2_NTLMV2_SESSKEY_SIZE);
if (rc)
return rc;
+ /*
+ * Per MS-SMB2 3.2.5.3.1, signing key always uses Session.SessionKey
+ * (first 16 bytes). Encryption/decryption keys use
+ * Session.FullSessionKey when dialect is 3.1.1 and cipher is
+ * AES-256-CCM or AES-256-GCM, otherwise Session.SessionKey.
+ */
+
+ if (server->dialect == SMB311_PROT_ID &&
+ (server->cipher_type == SMB2_ENCRYPTION_AES256_CCM ||
+ server->cipher_type == SMB2_ENCRYPTION_AES256_GCM))
+ full_key_size = ses->auth_key.len;
+
/* safe to access primary channel, since it will never go away */
spin_lock(&ses->chan_lock);
memcpy(ses->chans[chan_index].signkey, ses->smb3signingkey,
@@ -484,13 +500,15 @@ generate_smb3signingkey(struct cifs_ses *ses,
rc = generate_key(ses, ptriplet->encryption.label,
ptriplet->encryption.context,
ses->smb3encryptionkey,
- SMB3_ENC_DEC_KEY_SIZE);
+ SMB3_ENC_DEC_KEY_SIZE,
+ full_key_size);
if (rc)
return rc;
rc = generate_key(ses, ptriplet->decryption.label,
ptriplet->decryption.context,
ses->smb3decryptionkey,
- SMB3_ENC_DEC_KEY_SIZE);
+ SMB3_ENC_DEC_KEY_SIZE,
+ full_key_size);
if (rc)
return rc;
}
@@ -505,7 +523,7 @@ generate_smb3signingkey(struct cifs_ses *ses,
&ses->Suid);
cifs_dbg(VFS, "Cipher type %d\n", server->cipher_type);
cifs_dbg(VFS, "Session Key %*ph\n",
- SMB2_NTLMV2_SESSKEY_SIZE, ses->auth_key.response);
+ (int)ses->auth_key.len, ses->auth_key.response);
cifs_dbg(VFS, "Signing Key %*ph\n",
SMB3_SIGN_KEY_SIZE, ses->smb3signingkey);
if ((server->cipher_type == SMB2_ENCRYPTION_AES256_CCM) ||
diff --git a/fs/smb/server/auth.c b/fs/smb/server/auth.c
index c12dcb0a47dd..fea62d1cc473 100644
--- a/fs/smb/server/auth.c
+++ b/fs/smb/server/auth.c
@@ -1111,6 +1111,7 @@ int ksmbd_crypt_message(struct ksmbd_work *work, struct kvec *iov,
struct smb2_transform_hdr *tr_hdr = smb2_get_msg(iov[0].iov_base);
unsigned int assoc_data_len = sizeof(struct smb2_transform_hdr) - 20;
int rc;
+ DECLARE_CRYPTO_WAIT(wait);
struct scatterlist *sg;
u8 sign[SMB2_SIGNATURE_SIZE] = {};
u8 key[SMB3_ENC_DEC_KEY_SIZE];
@@ -1197,12 +1198,12 @@ int ksmbd_crypt_message(struct ksmbd_work *work, struct kvec *iov,
aead_request_set_crypt(req, sg, sg, crypt_len, iv);
aead_request_set_ad(req, assoc_data_len);
- aead_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP, NULL, NULL);
+ aead_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG |
+ CRYPTO_TFM_REQ_MAY_SLEEP,
+ crypto_req_done, &wait);
- if (enc)
- rc = crypto_aead_encrypt(req);
- else
- rc = crypto_aead_decrypt(req);
+ rc = crypto_wait_req(enc ? crypto_aead_encrypt(req) :
+ crypto_aead_decrypt(req), &wait);
if (rc)
goto free_iv;
diff --git a/fs/smb/server/connection.c b/fs/smb/server/connection.c
index 8470aba1233a..1610b4d2fd41 100644
--- a/fs/smb/server/connection.c
+++ b/fs/smb/server/connection.c
@@ -41,6 +41,15 @@ void ksmbd_conn_free(struct ksmbd_conn *conn)
kfree(conn->preauth_info);
kfree(conn->mechToken);
if (atomic_dec_and_test(&conn->refcnt)) {
+ /*
+ * async_ida is embedded in struct ksmbd_conn, so pair
+ * ida_destroy() with the final kfree() rather than with
+ * the unconditional field teardown above. This keeps
+ * the IDA valid for the entire lifetime of the struct,
+ * even while other refcount holders (oplock / vfs
+ * durable handles) still reference the connection.
+ */
+ ida_destroy(&conn->async_ida);
conn->transport->ops->free_transport(conn->transport);
kfree(conn);
}
diff --git a/fs/smb/server/mgmt/user_session.c b/fs/smb/server/mgmt/user_session.c
index 352cf9e47ebe..151248e02e9e 100644
--- a/fs/smb/server/mgmt/user_session.c
+++ b/fs/smb/server/mgmt/user_session.c
@@ -171,6 +171,7 @@ void ksmbd_session_destroy(struct ksmbd_session *sess)
free_channel_list(sess);
kfree(sess->Preauth_HashValue);
ksmbd_release_id(&session_ida, sess->id);
+ ida_destroy(&sess->tree_conn_ida);
kfree(sess);
}
@@ -327,8 +328,13 @@ struct ksmbd_session *ksmbd_session_lookup_all(struct ksmbd_conn *conn,
struct ksmbd_session *sess;
sess = ksmbd_session_lookup(conn, id);
- if (!sess && conn->binding)
+ if (!sess && conn->binding) {
sess = ksmbd_session_lookup_slowpath(id);
+ if (sess && !xa_load(&sess->ksmbd_chann_list, (long)conn)) {
+ ksmbd_user_session_put(sess);
+ sess = NULL;
+ }
+ }
if (sess && sess->state != SMB2_SESSION_VALID) {
ksmbd_user_session_put(sess);
sess = NULL;
@@ -445,6 +451,8 @@ static struct ksmbd_session *__session_create(int protocol)
if (!sess)
return NULL;
+ ida_init(&sess->tree_conn_ida);
+
if (ksmbd_init_file_table(&sess->file_table))
goto error;
@@ -464,8 +472,6 @@ static struct ksmbd_session *__session_create(int protocol)
if (ret)
goto error;
- ida_init(&sess->tree_conn_ida);
-
down_write(&sessions_table_lock);
hash_add(sessions_table, &sess->hlist, sess->id);
up_write(&sessions_table_lock);
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index 29fbdada7259..700d9da3c65a 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -2849,6 +2849,8 @@ static int parse_durable_handle_context(struct ksmbd_work *work,
dh_info->reconnected = true;
goto out;
}
+ ksmbd_put_durable_fd(dh_info->fp);
+ dh_info->fp = NULL;
}
if ((lc && (lc->req_state & SMB2_LEASE_HANDLE_CACHING_LE)) ||
diff --git a/fs/sync.c b/fs/sync.c
index 67df255eb189..2955cd4c77a3 100644
--- a/fs/sync.c
+++ b/fs/sync.c
@@ -148,11 +148,11 @@ void emergency_sync(void)
*/
SYSCALL_DEFINE1(syncfs, int, fd)
{
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
struct super_block *sb;
int ret, ret2;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
sb = fd_file(f)->f_path.dentry->d_sb;
@@ -162,7 +162,6 @@ SYSCALL_DEFINE1(syncfs, int, fd)
ret2 = errseq_check_and_advance(&sb->s_wb_err, &fd_file(f)->f_sb_err);
- fdput(f);
return ret ? ret : ret2;
}
@@ -205,14 +204,12 @@ EXPORT_SYMBOL(vfs_fsync);
static int do_fsync(unsigned int fd, int datasync)
{
- struct fd f = fdget(fd);
- int ret = -EBADF;
+ CLASS(fd, f)(fd);
- if (fd_file(f)) {
- ret = vfs_fsync(fd_file(f), datasync);
- fdput(f);
- }
- return ret;
+ if (fd_empty(f))
+ return -EBADF;
+
+ return vfs_fsync(fd_file(f), datasync);
}
SYSCALL_DEFINE1(fsync, unsigned int, fd)
@@ -355,16 +352,12 @@ int sync_file_range(struct file *file, loff_t offset, loff_t nbytes,
int ksys_sync_file_range(int fd, loff_t offset, loff_t nbytes,
unsigned int flags)
{
- int ret;
- struct fd f;
+ CLASS(fd, f)(fd);
- ret = -EBADF;
- f = fdget(fd);
- if (fd_file(f))
- ret = sync_file_range(fd_file(f), offset, nbytes, flags);
+ if (fd_empty(f))
+ return -EBADF;
- fdput(f);
- return ret;
+ return sync_file_range(fd_file(f), offset, nbytes, flags);
}
SYSCALL_DEFINE4(sync_file_range, int, fd, loff_t, offset, loff_t, nbytes,
diff --git a/fs/tracefs/event_inode.c b/fs/tracefs/event_inode.c
index 0d2bc92b760f..02d56ed6ad20 100644
--- a/fs/tracefs/event_inode.c
+++ b/fs/tracefs/event_inode.c
@@ -732,7 +732,7 @@ struct eventfs_inode *eventfs_create_dir(const char *name, struct eventfs_inode
mutex_lock(&eventfs_mutex);
if (!parent->is_freed)
- list_add_tail(&ei->list, &parent->children);
+ list_add_tail_rcu(&ei->list, &parent->children);
mutex_unlock(&eventfs_mutex);
/* Was the parent freed? */
diff --git a/include/dt-bindings/clock/qcom,dispcc-sc7180.h b/include/dt-bindings/clock/qcom,dispcc-sc7180.h
index b9b51617a335..070510306074 100644
--- a/include/dt-bindings/clock/qcom,dispcc-sc7180.h
+++ b/include/dt-bindings/clock/qcom,dispcc-sc7180.h
@@ -6,6 +6,7 @@
#ifndef _DT_BINDINGS_CLK_QCOM_DISP_CC_SC7180_H
#define _DT_BINDINGS_CLK_QCOM_DISP_CC_SC7180_H
+/* Clocks */
#define DISP_CC_PLL0 0
#define DISP_CC_PLL0_OUT_EVEN 1
#define DISP_CC_MDSS_AHB_CLK 2
@@ -40,7 +41,11 @@
#define DISP_CC_MDSS_VSYNC_CLK_SRC 31
#define DISP_CC_XO_CLK 32
-/* DISP_CC GDSCR */
+/* Resets */
+#define DISP_CC_MDSS_CORE_BCR 0
+#define DISP_CC_MDSS_RSCC_BCR 1
+
+/* GDSCs */
#define MDSS_GDSC 0
#endif
diff --git a/include/dt-bindings/clock/qcom,gcc-sc8180x.h b/include/dt-bindings/clock/qcom,gcc-sc8180x.h
index e364006aa6ea..99c97b2033fc 100644
--- a/include/dt-bindings/clock/qcom,gcc-sc8180x.h
+++ b/include/dt-bindings/clock/qcom,gcc-sc8180x.h
@@ -312,5 +312,10 @@
#define USB30_MP_GDSC 8
#define USB30_PRIM_GDSC 9
#define USB30_SEC_GDSC 10
+#define HLOS1_VOTE_MMNOC_MMU_TBU_HF0_GDSC 11
+#define HLOS1_VOTE_MMNOC_MMU_TBU_HF1_GDSC 12
+#define HLOS1_VOTE_MMNOC_MMU_TBU_SF_GDSC 13
+#define HLOS1_VOTE_TURING_MMU_TBU0_GDSC 14
+#define HLOS1_VOTE_TURING_MMU_TBU1_GDSC 15
#endif
diff --git a/include/linux/cdrom.h b/include/linux/cdrom.h
index fdfb61ccf55a..b4f2b2374441 100644
--- a/include/linux/cdrom.h
+++ b/include/linux/cdrom.h
@@ -109,6 +109,7 @@ int cdrom_ioctl(struct cdrom_device_info *cdi, struct block_device *bdev,
extern unsigned int cdrom_check_events(struct cdrom_device_info *cdi,
unsigned int clearing);
+extern void cdrom_probe_write_features(struct cdrom_device_info *cdi);
extern int register_cdrom(struct gendisk *disk, struct cdrom_device_info *cdi);
extern void unregister_cdrom(struct cdrom_device_info *cdi);
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 2361ed4d2b15..6645faf1cc1d 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -94,7 +94,6 @@ enum cpuhp_state {
CPUHP_PCI_XGENE_DEAD,
CPUHP_IOMMU_IOVA_DEAD,
CPUHP_AP_ARM_CACHE_B15_RAC_DEAD,
- CPUHP_PADATA_DEAD,
CPUHP_AP_DTPM_CPU_DEAD,
CPUHP_RANDOM_PREPARE,
CPUHP_WORKQUEUE_PREP,
diff --git a/include/linux/dmi.h b/include/linux/dmi.h
index 927f8a8b7a1d..2eedf44e6801 100644
--- a/include/linux/dmi.h
+++ b/include/linux/dmi.h
@@ -60,6 +60,7 @@ enum dmi_entry_type {
DMI_ENTRY_OOB_REMOTE_ACCESS,
DMI_ENTRY_BIS_ENTRY,
DMI_ENTRY_SYSTEM_BOOT,
+ DMI_ENTRY_64_MEM_ERROR,
DMI_ENTRY_MGMT_DEV,
DMI_ENTRY_MGMT_DEV_COMPONENT,
DMI_ENTRY_MGMT_DEV_THRES,
@@ -69,6 +70,10 @@ enum dmi_entry_type {
DMI_ENTRY_ADDITIONAL,
DMI_ENTRY_ONBOARD_DEV_EXT,
DMI_ENTRY_MGMT_CONTROLLER_HOST,
+ DMI_ENTRY_TPM_DEVICE,
+ DMI_ENTRY_PROCESSOR_ADDITIONAL,
+ DMI_ENTRY_FIRMWARE_INVENTORY,
+ DMI_ENTRY_STRING_PROPERTY,
DMI_ENTRY_INACTIVE = 126,
DMI_ENTRY_END_OF_TABLE = 127,
};
diff --git a/include/linux/file.h b/include/linux/file.h
index f98de143245a..b49a92295b3f 100644
--- a/include/linux/file.h
+++ b/include/linux/file.h
@@ -30,12 +30,6 @@ extern struct file *alloc_file_pseudo_noaccount(struct inode *, struct vfsmount
extern struct file *alloc_file_clone(struct file *, int flags,
const struct file_operations *);
-static inline void fput_light(struct file *file, int fput_needed)
-{
- if (fput_needed)
- fput(file);
-}
-
/* either a reference to struct file + flags
* (cloned vs. borrowed, pos locked), with
* flags stored in lower bits of value,
diff --git a/include/linux/fsl/mc.h b/include/linux/fsl/mc.h
index c90ec889bfc2..b5f64a904689 100644
--- a/include/linux/fsl/mc.h
+++ b/include/linux/fsl/mc.h
@@ -178,9 +178,6 @@ struct fsl_mc_obj_desc {
* @regions: pointer to array of MMIO region entries
* @irqs: pointer to array of pointers to interrupts allocated to this device
* @resource: generic resource associated with this MC object device, if any.
- * @driver_override: driver name to force a match; do not set directly,
- * because core frees it; use driver_set_override() to
- * set or clear it.
*
* Generic device object for MC object devices that are "attached" to a
* MC bus.
@@ -214,7 +211,6 @@ struct fsl_mc_device {
struct fsl_mc_device_irq **irqs;
struct fsl_mc_resource *resource;
struct device_link *consumer_link;
- const char *driver_override;
};
#define to_fsl_mc_device(_dev) \
diff --git a/include/linux/if_ether.h b/include/linux/if_ether.h
index 47a0feffc121..ca9afa824aa4 100644
--- a/include/linux/if_ether.h
+++ b/include/linux/if_ether.h
@@ -19,6 +19,9 @@
#include <linux/skbuff.h>
#include <uapi/linux/if_ether.h>
+/* XX:XX:XX:XX:XX:XX */
+#define MAC_ADDR_STR_LEN (3 * ETH_ALEN - 1)
+
static inline struct ethhdr *eth_hdr(const struct sk_buff *skb)
{
return (struct ethhdr *)skb_mac_header(skb);
diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h
index 110e9d09de24..4c0b436f2092 100644
--- a/include/linux/moduleparam.h
+++ b/include/linux/moduleparam.h
@@ -396,14 +396,9 @@ extern char *parse_args(const char *name,
void *arg, parse_unknown_fn unknown);
/* Called by module remove. */
-#ifdef CONFIG_SYSFS
-extern void destroy_params(const struct kernel_param *params, unsigned num);
-#else
-static inline void destroy_params(const struct kernel_param *params,
- unsigned num)
-{
-}
-#endif /* !CONFIG_SYSFS */
+#ifdef CONFIG_MODULES
+void module_destroy_params(const struct kernel_param *params, unsigned int num);
+#endif
/* All the helper functions */
/* The macros to do compile-time type checking stolen from Jakub
diff --git a/include/linux/netpoll.h b/include/linux/netpoll.h
index b34301650c47..ca88b1b87059 100644
--- a/include/linux/netpoll.h
+++ b/include/linux/netpoll.h
@@ -25,7 +25,13 @@ union inet_addr {
struct netpoll {
struct net_device *dev;
netdevice_tracker dev_tracker;
+ /*
+ * Either dev_name or dev_mac can be used to specify the local
+ * interface - dev_name is used if it is a nonempty string, else
+ * dev_mac is used.
+ */
char dev_name[IFNAMSIZ];
+ u8 dev_mac[ETH_ALEN];
const char *name;
union inet_addr local_ip, remote_ip;
diff --git a/include/linux/padata.h b/include/linux/padata.h
index 765f2778e264..b6232bea6edf 100644
--- a/include/linux/padata.h
+++ b/include/linux/padata.h
@@ -149,23 +149,23 @@ struct padata_mt_job {
/**
* struct padata_instance - The overall control structure.
*
- * @cpu_online_node: Linkage for CPU online callback.
- * @cpu_dead_node: Linkage for CPU offline callback.
+ * @cpuhp_node: Linkage for CPU hotplug callbacks.
* @parallel_wq: The workqueue used for parallel work.
* @serial_wq: The workqueue used for serial work.
* @pslist: List of padata_shell objects attached to this instance.
* @cpumask: User supplied cpumasks for parallel and serial works.
+ * @validate_cpumask: Internal cpumask used to validate @cpumask during hotplug.
* @kobj: padata instance kernel object.
* @lock: padata instance lock.
* @flags: padata flags.
*/
struct padata_instance {
- struct hlist_node cpu_online_node;
- struct hlist_node cpu_dead_node;
+ struct hlist_node cpuhp_node;
struct workqueue_struct *parallel_wq;
struct workqueue_struct *serial_wq;
struct list_head pslist;
struct padata_cpumask cpumask;
+ cpumask_var_t validate_cpumask;
struct kobject kobj;
struct mutex lock;
u8 flags;
diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h
index de8cc3658220..8a275df496fb 100644
--- a/include/linux/pci-epc.h
+++ b/include/linux/pci-epc.h
@@ -103,7 +103,7 @@ struct pci_epc_ops {
u8 interrupts);
int (*get_msi)(struct pci_epc *epc, u8 func_no, u8 vfunc_no);
int (*set_msix)(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
- u16 interrupts, enum pci_barno, u32 offset);
+ u16 nr_irqs, enum pci_barno, u32 offset);
int (*get_msix)(struct pci_epc *epc, u8 func_no, u8 vfunc_no);
int (*raise_irq)(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
unsigned int type, u16 interrupt_num);
@@ -283,8 +283,8 @@ void pci_epc_unmap_addr(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
int pci_epc_set_msi(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
u8 interrupts);
int pci_epc_get_msi(struct pci_epc *epc, u8 func_no, u8 vfunc_no);
-int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
- u16 interrupts, enum pci_barno, u32 offset);
+int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no, u16 nr_irqs,
+ enum pci_barno, u32 offset);
int pci_epc_get_msix(struct pci_epc *epc, u8 func_no, u8 vfunc_no);
int pci_epc_map_msi_irq(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
phys_addr_t phys_addr, u8 interrupt_num,
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 242ee3843e10..825e6b3056f1 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -540,12 +540,6 @@ struct pci_dev {
u8 supported_speeds; /* Supported Link Speeds Vector */
phys_addr_t rom; /* Physical address if not from BAR */
size_t romlen; /* Length if not from BAR */
- /*
- * Driver name to force a match. Do not set directly, because core
- * frees it. Use driver_set_override() to set or clear it.
- */
- const char *driver_override;
-
unsigned long priv_flags; /* Private flags for the PCI driver */
/* These methods index pci_reset_fn_methods[] */
diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h
index c6716f474ba4..908e20bbfcae 100644
--- a/include/linux/pm_domain.h
+++ b/include/linux/pm_domain.h
@@ -36,8 +36,8 @@
struct dev_pm_domain_attach_data {
const char * const *pd_names;
- const u32 num_pd_names;
- const u32 pd_flags;
+ u32 num_pd_names;
+ u32 pd_flags;
};
struct dev_pm_domain_list {
diff --git a/include/linux/ppp_defs.h b/include/linux/ppp_defs.h
index b7e57fdbd413..b1d1f46d7d3b 100644
--- a/include/linux/ppp_defs.h
+++ b/include/linux/ppp_defs.h
@@ -8,6 +8,7 @@
#define _PPP_DEFS_H_
#include <linux/crc-ccitt.h>
+#include <linux/skbuff.h>
#include <uapi/linux/ppp_defs.h>
#define PPP_FCS(fcs, c) crc_ccitt_byte(fcs, c)
@@ -25,4 +26,19 @@ static inline bool ppp_proto_is_valid(u16 proto)
return !!((proto & 0x0101) == 0x0001);
}
+/**
+ * ppp_skb_is_compressed_proto - checks if PPP protocol in a skb is compressed
+ * @skb: skb to check
+ *
+ * Check if the PPP protocol field is compressed (the least significant
+ * bit of the most significant octet is 1). skb->data must point to the PPP
+ * protocol header.
+ *
+ * Return: Whether the PPP protocol field is compressed.
+ */
+static inline bool ppp_skb_is_compressed_proto(const struct sk_buff *skb)
+{
+ return unlikely(skb->data[0] & 0x01);
+}
+
#endif /* _PPP_DEFS_H_ */
diff --git a/include/linux/printk.h b/include/linux/printk.h
index f9498e9cb8ba..a6c6fd107805 100644
--- a/include/linux/printk.h
+++ b/include/linux/printk.h
@@ -800,7 +800,8 @@ static inline void print_hex_dump_devel(const char *prefix_str, int prefix_type,
#endif
/**
- * print_hex_dump_bytes - shorthand form of print_hex_dump() with default params
+ * print_hex_dump_bytes - shorthand form of print_hex_dump_debug() with default
+ * params
* @prefix_str: string to prefix each line with;
* caller supplies trailing spaces for alignment if desired
* @prefix_type: controls whether prefix of an offset, address, or none
@@ -808,7 +809,7 @@ static inline void print_hex_dump_devel(const char *prefix_str, int prefix_type,
* @buf: data blob to dump
* @len: number of bytes in the @buf
*
- * Calls print_hex_dump(), with log level of KERN_DEBUG,
+ * Calls print_hex_dump_debug(), with log level of KERN_DEBUG,
* rowsize of 16, groupsize of 1, and ASCII output included.
*/
#define print_hex_dump_bytes(prefix_str, prefix_type, buf, len) \
diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h
index 06cc8888199e..2334a02c3014 100644
--- a/include/linux/quotaops.h
+++ b/include/linux/quotaops.h
@@ -44,14 +44,7 @@ int dquot_initialize(struct inode *inode);
bool dquot_initialize_needed(struct inode *inode);
void dquot_drop(struct inode *inode);
struct dquot *dqget(struct super_block *sb, struct kqid qid);
-static inline struct dquot *dqgrab(struct dquot *dquot)
-{
- /* Make sure someone else has active reference to dquot */
- WARN_ON_ONCE(!atomic_read(&dquot->dq_count));
- WARN_ON_ONCE(!test_bit(DQ_ACTIVE_B, &dquot->dq_flags));
- atomic_inc(&dquot->dq_count);
- return dquot;
-}
+struct dquot *dqgrab(struct dquot *dquot);
static inline bool dquot_is_busy(struct dquot *dquot)
{
diff --git a/include/linux/reset.h b/include/linux/reset.h
index 4b31d683776e..0ae6e9030d34 100644
--- a/include/linux/reset.h
+++ b/include/linux/reset.h
@@ -26,6 +26,48 @@ struct reset_control_bulk_data {
struct reset_control *rstc;
};
+#define RESET_CONTROL_FLAGS_BIT_SHARED BIT(0) /* not exclusive */
+#define RESET_CONTROL_FLAGS_BIT_OPTIONAL BIT(1)
+#define RESET_CONTROL_FLAGS_BIT_ACQUIRED BIT(2) /* iff exclusive, not released */
+#define RESET_CONTROL_FLAGS_BIT_DEASSERTED BIT(3)
+
+/**
+ * enum reset_control_flags - Flags that can be passed to the reset_control_get functions
+ * to determine the type of reset control.
+ * These values cannot be OR'd.
+ *
+ * @RESET_CONTROL_EXCLUSIVE: exclusive, acquired,
+ * @RESET_CONTROL_EXCLUSIVE_DEASSERTED: exclusive, acquired, deasserted
+ * @RESET_CONTROL_EXCLUSIVE_RELEASED: exclusive, released,
+ * @RESET_CONTROL_SHARED: shared
+ * @RESET_CONTROL_SHARED_DEASSERTED: shared, deasserted
+ * @RESET_CONTROL_OPTIONAL_EXCLUSIVE: optional, exclusive, acquired
+ * @RESET_CONTROL_OPTIONAL_EXCLUSIVE_DEASSERTED: optional, exclusive, acquired, deasserted
+ * @RESET_CONTROL_OPTIONAL_EXCLUSIVE_RELEASED: optional, exclusive, released
+ * @RESET_CONTROL_OPTIONAL_SHARED: optional, shared
+ * @RESET_CONTROL_OPTIONAL_SHARED_DEASSERTED: optional, shared, deasserted
+ */
+enum reset_control_flags {
+ RESET_CONTROL_EXCLUSIVE = RESET_CONTROL_FLAGS_BIT_ACQUIRED,
+ RESET_CONTROL_EXCLUSIVE_DEASSERTED = RESET_CONTROL_FLAGS_BIT_ACQUIRED |
+ RESET_CONTROL_FLAGS_BIT_DEASSERTED,
+ RESET_CONTROL_EXCLUSIVE_RELEASED = 0,
+ RESET_CONTROL_SHARED = RESET_CONTROL_FLAGS_BIT_SHARED,
+ RESET_CONTROL_SHARED_DEASSERTED = RESET_CONTROL_FLAGS_BIT_SHARED |
+ RESET_CONTROL_FLAGS_BIT_DEASSERTED,
+ RESET_CONTROL_OPTIONAL_EXCLUSIVE = RESET_CONTROL_FLAGS_BIT_OPTIONAL |
+ RESET_CONTROL_FLAGS_BIT_ACQUIRED,
+ RESET_CONTROL_OPTIONAL_EXCLUSIVE_DEASSERTED = RESET_CONTROL_FLAGS_BIT_OPTIONAL |
+ RESET_CONTROL_FLAGS_BIT_ACQUIRED |
+ RESET_CONTROL_FLAGS_BIT_DEASSERTED,
+ RESET_CONTROL_OPTIONAL_EXCLUSIVE_RELEASED = RESET_CONTROL_FLAGS_BIT_OPTIONAL,
+ RESET_CONTROL_OPTIONAL_SHARED = RESET_CONTROL_FLAGS_BIT_OPTIONAL |
+ RESET_CONTROL_FLAGS_BIT_SHARED,
+ RESET_CONTROL_OPTIONAL_SHARED_DEASSERTED = RESET_CONTROL_FLAGS_BIT_OPTIONAL |
+ RESET_CONTROL_FLAGS_BIT_SHARED |
+ RESET_CONTROL_FLAGS_BIT_DEASSERTED,
+};
+
#ifdef CONFIG_RESET_CONTROLLER
int reset_control_reset(struct reset_control *rstc);
@@ -43,30 +85,25 @@ int reset_control_bulk_acquire(int num_rstcs, struct reset_control_bulk_data *rs
void reset_control_bulk_release(int num_rstcs, struct reset_control_bulk_data *rstcs);
struct reset_control *__of_reset_control_get(struct device_node *node,
- const char *id, int index, bool shared,
- bool optional, bool acquired);
+ const char *id, int index, enum reset_control_flags flags);
struct reset_control *__reset_control_get(struct device *dev, const char *id,
- int index, bool shared,
- bool optional, bool acquired);
+ int index, enum reset_control_flags flags);
void reset_control_put(struct reset_control *rstc);
int __reset_control_bulk_get(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs,
- bool shared, bool optional, bool acquired);
+ enum reset_control_flags flags);
void reset_control_bulk_put(int num_rstcs, struct reset_control_bulk_data *rstcs);
int __device_reset(struct device *dev, bool optional);
struct reset_control *__devm_reset_control_get(struct device *dev,
- const char *id, int index, bool shared,
- bool optional, bool acquired);
+ const char *id, int index, enum reset_control_flags flags);
int __devm_reset_control_bulk_get(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs,
- bool shared, bool optional, bool acquired);
+ enum reset_control_flags flags);
struct reset_control *devm_reset_control_array_get(struct device *dev,
- bool shared, bool optional);
-struct reset_control *of_reset_control_array_get(struct device_node *np,
- bool shared, bool optional,
- bool acquired);
+ enum reset_control_flags flags);
+struct reset_control *of_reset_control_array_get(struct device_node *np, enum reset_control_flags);
int reset_control_get_count(struct device *dev);
@@ -117,17 +154,19 @@ static inline int __device_reset(struct device *dev, bool optional)
static inline struct reset_control *__of_reset_control_get(
struct device_node *node,
- const char *id, int index, bool shared,
- bool optional, bool acquired)
+ const char *id, int index, enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
return optional ? NULL : ERR_PTR(-ENOTSUPP);
}
static inline struct reset_control *__reset_control_get(
struct device *dev, const char *id,
- int index, bool shared, bool optional,
- bool acquired)
+ int index, enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
return optional ? NULL : ERR_PTR(-ENOTSUPP);
}
@@ -163,8 +202,10 @@ reset_control_bulk_release(int num_rstcs, struct reset_control_bulk_data *rstcs)
static inline int
__reset_control_bulk_get(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs,
- bool shared, bool optional, bool acquired)
+ enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
return optional ? 0 : -EOPNOTSUPP;
}
@@ -175,30 +216,36 @@ reset_control_bulk_put(int num_rstcs, struct reset_control_bulk_data *rstcs)
static inline struct reset_control *__devm_reset_control_get(
struct device *dev, const char *id,
- int index, bool shared, bool optional,
- bool acquired)
+ int index, enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
return optional ? NULL : ERR_PTR(-ENOTSUPP);
}
static inline int
__devm_reset_control_bulk_get(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs,
- bool shared, bool optional, bool acquired)
+ enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
return optional ? 0 : -EOPNOTSUPP;
}
static inline struct reset_control *
-devm_reset_control_array_get(struct device *dev, bool shared, bool optional)
+devm_reset_control_array_get(struct device *dev, enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
return optional ? NULL : ERR_PTR(-ENOTSUPP);
}
static inline struct reset_control *
-of_reset_control_array_get(struct device_node *np, bool shared, bool optional,
- bool acquired)
+of_reset_control_array_get(struct device_node *np, enum reset_control_flags flags)
{
+ bool optional = flags & RESET_CONTROL_FLAGS_BIT_OPTIONAL;
+
return optional ? NULL : ERR_PTR(-ENOTSUPP);
}
@@ -237,7 +284,7 @@ static inline int device_reset_optional(struct device *dev)
static inline struct reset_control *
__must_check reset_control_get_exclusive(struct device *dev, const char *id)
{
- return __reset_control_get(dev, id, 0, false, false, true);
+ return __reset_control_get(dev, id, 0, RESET_CONTROL_EXCLUSIVE);
}
/**
@@ -254,7 +301,7 @@ static inline int __must_check
reset_control_bulk_get_exclusive(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __reset_control_bulk_get(dev, num_rstcs, rstcs, false, false, true);
+ return __reset_control_bulk_get(dev, num_rstcs, rstcs, RESET_CONTROL_EXCLUSIVE);
}
/**
@@ -275,7 +322,7 @@ static inline struct reset_control *
__must_check reset_control_get_exclusive_released(struct device *dev,
const char *id)
{
- return __reset_control_get(dev, id, 0, false, false, false);
+ return __reset_control_get(dev, id, 0, RESET_CONTROL_EXCLUSIVE_RELEASED);
}
/**
@@ -296,7 +343,7 @@ static inline int __must_check
reset_control_bulk_get_exclusive_released(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __reset_control_bulk_get(dev, num_rstcs, rstcs, false, false, false);
+ return __reset_control_bulk_get(dev, num_rstcs, rstcs, RESET_CONTROL_EXCLUSIVE_RELEASED);
}
/**
@@ -317,7 +364,8 @@ static inline int __must_check
reset_control_bulk_get_optional_exclusive_released(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __reset_control_bulk_get(dev, num_rstcs, rstcs, false, true, false);
+ return __reset_control_bulk_get(dev, num_rstcs, rstcs,
+ RESET_CONTROL_OPTIONAL_EXCLUSIVE_RELEASED);
}
/**
@@ -345,7 +393,7 @@ reset_control_bulk_get_optional_exclusive_released(struct device *dev, int num_r
static inline struct reset_control *reset_control_get_shared(
struct device *dev, const char *id)
{
- return __reset_control_get(dev, id, 0, true, false, false);
+ return __reset_control_get(dev, id, 0, RESET_CONTROL_SHARED);
}
/**
@@ -362,7 +410,7 @@ static inline int __must_check
reset_control_bulk_get_shared(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __reset_control_bulk_get(dev, num_rstcs, rstcs, true, false, false);
+ return __reset_control_bulk_get(dev, num_rstcs, rstcs, RESET_CONTROL_SHARED);
}
/**
@@ -379,7 +427,7 @@ reset_control_bulk_get_shared(struct device *dev, int num_rstcs,
static inline struct reset_control *reset_control_get_optional_exclusive(
struct device *dev, const char *id)
{
- return __reset_control_get(dev, id, 0, false, true, true);
+ return __reset_control_get(dev, id, 0, RESET_CONTROL_OPTIONAL_EXCLUSIVE);
}
/**
@@ -399,7 +447,7 @@ static inline int __must_check
reset_control_bulk_get_optional_exclusive(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __reset_control_bulk_get(dev, num_rstcs, rstcs, false, true, true);
+ return __reset_control_bulk_get(dev, num_rstcs, rstcs, RESET_CONTROL_OPTIONAL_EXCLUSIVE);
}
/**
@@ -416,7 +464,7 @@ reset_control_bulk_get_optional_exclusive(struct device *dev, int num_rstcs,
static inline struct reset_control *reset_control_get_optional_shared(
struct device *dev, const char *id)
{
- return __reset_control_get(dev, id, 0, true, true, false);
+ return __reset_control_get(dev, id, 0, RESET_CONTROL_OPTIONAL_SHARED);
}
/**
@@ -436,7 +484,7 @@ static inline int __must_check
reset_control_bulk_get_optional_shared(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __reset_control_bulk_get(dev, num_rstcs, rstcs, true, true, false);
+ return __reset_control_bulk_get(dev, num_rstcs, rstcs, RESET_CONTROL_OPTIONAL_SHARED);
}
/**
@@ -452,7 +500,7 @@ reset_control_bulk_get_optional_shared(struct device *dev, int num_rstcs,
static inline struct reset_control *of_reset_control_get_exclusive(
struct device_node *node, const char *id)
{
- return __of_reset_control_get(node, id, 0, false, false, true);
+ return __of_reset_control_get(node, id, 0, RESET_CONTROL_EXCLUSIVE);
}
/**
@@ -472,7 +520,7 @@ static inline struct reset_control *of_reset_control_get_exclusive(
static inline struct reset_control *of_reset_control_get_optional_exclusive(
struct device_node *node, const char *id)
{
- return __of_reset_control_get(node, id, 0, false, true, true);
+ return __of_reset_control_get(node, id, 0, RESET_CONTROL_OPTIONAL_EXCLUSIVE);
}
/**
@@ -497,7 +545,7 @@ static inline struct reset_control *of_reset_control_get_optional_exclusive(
static inline struct reset_control *of_reset_control_get_shared(
struct device_node *node, const char *id)
{
- return __of_reset_control_get(node, id, 0, true, false, false);
+ return __of_reset_control_get(node, id, 0, RESET_CONTROL_SHARED);
}
/**
@@ -514,7 +562,7 @@ static inline struct reset_control *of_reset_control_get_shared(
static inline struct reset_control *of_reset_control_get_exclusive_by_index(
struct device_node *node, int index)
{
- return __of_reset_control_get(node, NULL, index, false, false, true);
+ return __of_reset_control_get(node, NULL, index, RESET_CONTROL_EXCLUSIVE);
}
/**
@@ -542,7 +590,7 @@ static inline struct reset_control *of_reset_control_get_exclusive_by_index(
static inline struct reset_control *of_reset_control_get_shared_by_index(
struct device_node *node, int index)
{
- return __of_reset_control_get(node, NULL, index, true, false, false);
+ return __of_reset_control_get(node, NULL, index, RESET_CONTROL_SHARED);
}
/**
@@ -561,7 +609,26 @@ static inline struct reset_control *
__must_check devm_reset_control_get_exclusive(struct device *dev,
const char *id)
{
- return __devm_reset_control_get(dev, id, 0, false, false, true);
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_EXCLUSIVE);
+}
+
+/**
+ * devm_reset_control_get_exclusive_deasserted - resource managed
+ * reset_control_get_exclusive() +
+ * reset_control_deassert()
+ * @dev: device to be reset by the controller
+ * @id: reset line name
+ *
+ * Managed reset_control_get_exclusive() + reset_control_deassert(). For reset
+ * controllers returned from this function, reset_control_assert() +
+ * reset_control_put() is called automatically on driver detach.
+ *
+ * See reset_control_get_exclusive() for more information.
+ */
+static inline struct reset_control * __must_check
+devm_reset_control_get_exclusive_deasserted(struct device *dev, const char *id)
+{
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_EXCLUSIVE_DEASSERTED);
}
/**
@@ -581,7 +648,8 @@ static inline int __must_check
devm_reset_control_bulk_get_exclusive(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, false, false, true);
+ return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs,
+ RESET_CONTROL_EXCLUSIVE);
}
/**
@@ -600,7 +668,7 @@ static inline struct reset_control *
__must_check devm_reset_control_get_exclusive_released(struct device *dev,
const char *id)
{
- return __devm_reset_control_get(dev, id, 0, false, false, false);
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_EXCLUSIVE_RELEASED);
}
/**
@@ -620,7 +688,8 @@ static inline int __must_check
devm_reset_control_bulk_get_exclusive_released(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, false, false, false);
+ return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs,
+ RESET_CONTROL_EXCLUSIVE_RELEASED);
}
/**
@@ -639,7 +708,7 @@ static inline struct reset_control *
__must_check devm_reset_control_get_optional_exclusive_released(struct device *dev,
const char *id)
{
- return __devm_reset_control_get(dev, id, 0, false, true, false);
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_OPTIONAL_EXCLUSIVE_RELEASED);
}
/**
@@ -659,7 +728,8 @@ static inline int __must_check
devm_reset_control_bulk_get_optional_exclusive_released(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, false, true, false);
+ return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs,
+ RESET_CONTROL_OPTIONAL_EXCLUSIVE_RELEASED);
}
/**
@@ -674,7 +744,26 @@ devm_reset_control_bulk_get_optional_exclusive_released(struct device *dev, int
static inline struct reset_control *devm_reset_control_get_shared(
struct device *dev, const char *id)
{
- return __devm_reset_control_get(dev, id, 0, true, false, false);
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_SHARED);
+}
+
+/**
+ * devm_reset_control_get_shared_deasserted - resource managed
+ * reset_control_get_shared() +
+ * reset_control_deassert()
+ * @dev: device to be reset by the controller
+ * @id: reset line name
+ *
+ * Managed reset_control_get_shared() + reset_control_deassert(). For reset
+ * controllers returned from this function, reset_control_assert() +
+ * reset_control_put() is called automatically on driver detach.
+ *
+ * See devm_reset_control_get_shared() for more information.
+ */
+static inline struct reset_control * __must_check
+devm_reset_control_get_shared_deasserted(struct device *dev, const char *id)
+{
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_SHARED_DEASSERTED);
}
/**
@@ -694,7 +783,29 @@ static inline int __must_check
devm_reset_control_bulk_get_shared(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, true, false, false);
+ return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, RESET_CONTROL_SHARED);
+}
+
+/**
+ * devm_reset_control_bulk_get_shared_deasserted - resource managed
+ * reset_control_bulk_get_shared() +
+ * reset_control_bulk_deassert()
+ * @dev: device to be reset by the controller
+ * @num_rstcs: number of entries in rstcs array
+ * @rstcs: array of struct reset_control_bulk_data with reset line names set
+ *
+ * Managed reset_control_bulk_get_shared() + reset_control_bulk_deassert(). For
+ * reset controllers returned from this function, reset_control_bulk_assert() +
+ * reset_control_bulk_put() are called automatically on driver detach.
+ *
+ * See devm_reset_control_bulk_get_shared() for more information.
+ */
+static inline int __must_check
+devm_reset_control_bulk_get_shared_deasserted(struct device *dev, int num_rstcs,
+ struct reset_control_bulk_data *rstcs)
+{
+ return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs,
+ RESET_CONTROL_SHARED_DEASSERTED);
}
/**
@@ -712,7 +823,26 @@ devm_reset_control_bulk_get_shared(struct device *dev, int num_rstcs,
static inline struct reset_control *devm_reset_control_get_optional_exclusive(
struct device *dev, const char *id)
{
- return __devm_reset_control_get(dev, id, 0, false, true, true);
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_OPTIONAL_EXCLUSIVE);
+}
+
+/**
+ * devm_reset_control_get_optional_exclusive_deasserted - resource managed
+ * reset_control_get_optional_exclusive() +
+ * reset_control_deassert()
+ * @dev: device to be reset by the controller
+ * @id: reset line name
+ *
+ * Managed reset_control_get_optional_exclusive() + reset_control_deassert().
+ * For reset controllers returned from this function, reset_control_assert() +
+ * reset_control_put() is called automatically on driver detach.
+ *
+ * See devm_reset_control_get_optional_exclusive() for more information.
+ */
+static inline struct reset_control *
+devm_reset_control_get_optional_exclusive_deasserted(struct device *dev, const char *id)
+{
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_OPTIONAL_EXCLUSIVE_DEASSERTED);
}
/**
@@ -732,7 +862,8 @@ static inline int __must_check
devm_reset_control_bulk_get_optional_exclusive(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, false, true, true);
+ return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs,
+ RESET_CONTROL_OPTIONAL_EXCLUSIVE);
}
/**
@@ -750,7 +881,26 @@ devm_reset_control_bulk_get_optional_exclusive(struct device *dev, int num_rstcs
static inline struct reset_control *devm_reset_control_get_optional_shared(
struct device *dev, const char *id)
{
- return __devm_reset_control_get(dev, id, 0, true, true, false);
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_OPTIONAL_SHARED);
+}
+
+/**
+ * devm_reset_control_get_optional_shared_deasserted - resource managed
+ * reset_control_get_optional_shared() +
+ * reset_control_deassert()
+ * @dev: device to be reset by the controller
+ * @id: reset line name
+ *
+ * Managed reset_control_get_optional_shared() + reset_control_deassert(). For
+ * reset controllers returned from this function, reset_control_assert() +
+ * reset_control_put() is called automatically on driver detach.
+ *
+ * See devm_reset_control_get_optional_shared() for more information.
+ */
+static inline struct reset_control *
+devm_reset_control_get_optional_shared_deasserted(struct device *dev, const char *id)
+{
+ return __devm_reset_control_get(dev, id, 0, RESET_CONTROL_OPTIONAL_SHARED_DEASSERTED);
}
/**
@@ -770,7 +920,7 @@ static inline int __must_check
devm_reset_control_bulk_get_optional_shared(struct device *dev, int num_rstcs,
struct reset_control_bulk_data *rstcs)
{
- return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, true, true, false);
+ return __devm_reset_control_bulk_get(dev, num_rstcs, rstcs, RESET_CONTROL_OPTIONAL_SHARED);
}
/**
@@ -788,7 +938,7 @@ devm_reset_control_bulk_get_optional_shared(struct device *dev, int num_rstcs,
static inline struct reset_control *
devm_reset_control_get_exclusive_by_index(struct device *dev, int index)
{
- return __devm_reset_control_get(dev, NULL, index, false, false, true);
+ return __devm_reset_control_get(dev, NULL, index, RESET_CONTROL_EXCLUSIVE);
}
/**
@@ -804,7 +954,7 @@ devm_reset_control_get_exclusive_by_index(struct device *dev, int index)
static inline struct reset_control *
devm_reset_control_get_shared_by_index(struct device *dev, int index)
{
- return __devm_reset_control_get(dev, NULL, index, true, false, false);
+ return __devm_reset_control_get(dev, NULL, index, RESET_CONTROL_SHARED);
}
/*
@@ -852,54 +1002,54 @@ static inline struct reset_control *devm_reset_control_get_by_index(
static inline struct reset_control *
devm_reset_control_array_get_exclusive(struct device *dev)
{
- return devm_reset_control_array_get(dev, false, false);
+ return devm_reset_control_array_get(dev, RESET_CONTROL_EXCLUSIVE);
}
static inline struct reset_control *
devm_reset_control_array_get_shared(struct device *dev)
{
- return devm_reset_control_array_get(dev, true, false);
+ return devm_reset_control_array_get(dev, RESET_CONTROL_SHARED);
}
static inline struct reset_control *
devm_reset_control_array_get_optional_exclusive(struct device *dev)
{
- return devm_reset_control_array_get(dev, false, true);
+ return devm_reset_control_array_get(dev, RESET_CONTROL_OPTIONAL_EXCLUSIVE);
}
static inline struct reset_control *
devm_reset_control_array_get_optional_shared(struct device *dev)
{
- return devm_reset_control_array_get(dev, true, true);
+ return devm_reset_control_array_get(dev, RESET_CONTROL_OPTIONAL_SHARED);
}
static inline struct reset_control *
of_reset_control_array_get_exclusive(struct device_node *node)
{
- return of_reset_control_array_get(node, false, false, true);
+ return of_reset_control_array_get(node, RESET_CONTROL_EXCLUSIVE);
}
static inline struct reset_control *
of_reset_control_array_get_exclusive_released(struct device_node *node)
{
- return of_reset_control_array_get(node, false, false, false);
+ return of_reset_control_array_get(node, RESET_CONTROL_EXCLUSIVE_RELEASED);
}
static inline struct reset_control *
of_reset_control_array_get_shared(struct device_node *node)
{
- return of_reset_control_array_get(node, true, false, true);
+ return of_reset_control_array_get(node, RESET_CONTROL_SHARED);
}
static inline struct reset_control *
of_reset_control_array_get_optional_exclusive(struct device_node *node)
{
- return of_reset_control_array_get(node, false, true, true);
+ return of_reset_control_array_get(node, RESET_CONTROL_OPTIONAL_EXCLUSIVE);
}
static inline struct reset_control *
of_reset_control_array_get_optional_shared(struct device_node *node)
{
- return of_reset_control_array_get(node, true, true, true);
+ return of_reset_control_array_get(node, RESET_CONTROL_OPTIONAL_SHARED);
}
#endif
diff --git a/include/linux/spinlock_up.h b/include/linux/spinlock_up.h
index c87204247592..a132fc562297 100644
--- a/include/linux/spinlock_up.h
+++ b/include/linux/spinlock_up.h
@@ -48,16 +48,6 @@ static inline void arch_spin_unlock(arch_spinlock_t *lock)
lock->slock = 1;
}
-/*
- * Read-write spinlocks. No debug version.
- */
-#define arch_read_lock(lock) do { barrier(); (void)(lock); } while (0)
-#define arch_write_lock(lock) do { barrier(); (void)(lock); } while (0)
-#define arch_read_trylock(lock) ({ barrier(); (void)(lock); 1; })
-#define arch_write_trylock(lock) ({ barrier(); (void)(lock); 1; })
-#define arch_read_unlock(lock) do { barrier(); (void)(lock); } while (0)
-#define arch_write_unlock(lock) do { barrier(); (void)(lock); } while (0)
-
#else /* DEBUG_SPINLOCK */
#define arch_spin_is_locked(lock) ((void)(lock), 0)
/* for sched/core.c and kernel_lock.c: */
@@ -68,4 +58,14 @@ static inline void arch_spin_unlock(arch_spinlock_t *lock)
#define arch_spin_is_contended(lock) (((void)(lock), 0))
+/*
+ * Read-write spinlocks. No debug version.
+ */
+#define arch_read_lock(lock) do { barrier(); (void)(lock); } while (0)
+#define arch_write_lock(lock) do { barrier(); (void)(lock); } while (0)
+#define arch_read_trylock(lock) ({ barrier(); (void)(lock); 1; })
+#define arch_write_trylock(lock) ({ barrier(); (void)(lock); 1; })
+#define arch_read_unlock(lock) do { barrier(); (void)(lock); } while (0)
+#define arch_write_unlock(lock) do { barrier(); (void)(lock); } while (0)
+
#endif /* __LINUX_SPINLOCK_UP_H */
diff --git a/include/linux/wmi.h b/include/linux/wmi.h
index 3275470b5531..63cca3b58d6d 100644
--- a/include/linux/wmi.h
+++ b/include/linux/wmi.h
@@ -16,16 +16,12 @@
* struct wmi_device - WMI device structure
* @dev: Device associated with this WMI device
* @setable: True for devices implementing the Set Control Method
- * @driver_override: Driver name to force a match; do not set directly,
- * because core frees it; use driver_set_override() to
- * set or clear it.
*
* This represents WMI devices discovered by the WMI driver core.
*/
struct wmi_device {
struct device dev;
bool setable;
- const char *driver_override;
};
/**
diff --git a/include/net/page_pool/memory_provider.h b/include/net/page_pool/memory_provider.h
new file mode 100644
index 000000000000..e49d0a52629d
--- /dev/null
+++ b/include/net/page_pool/memory_provider.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _NET_PAGE_POOL_MEMORY_PROVIDER_H
+#define _NET_PAGE_POOL_MEMORY_PROVIDER_H
+
+#include <net/netmem.h>
+#include <net/page_pool/types.h>
+
+struct memory_provider_ops {
+ netmem_ref (*alloc_netmems)(struct page_pool *pool, gfp_t gfp);
+ bool (*release_netmem)(struct page_pool *pool, netmem_ref netmem);
+ int (*init)(struct page_pool *pool);
+ void (*destroy)(struct page_pool *pool);
+};
+
+#endif
diff --git a/include/net/page_pool/types.h b/include/net/page_pool/types.h
index f53e2c90b686..7fae0d4eef5e 100644
--- a/include/net/page_pool/types.h
+++ b/include/net/page_pool/types.h
@@ -156,8 +156,11 @@ struct page_pool_stats {
*/
#define PAGE_POOL_FRAG_GROUP_ALIGN (4 * sizeof(long))
+struct memory_provider_ops;
+
struct pp_memory_provider_params {
void *mp_priv;
+ const struct memory_provider_ops *mp_ops;
};
struct page_pool {
@@ -219,6 +222,7 @@ struct page_pool {
struct ptr_ring ring;
void *mp_priv;
+ const struct memory_provider_ops *mp_ops;
struct xarray dma_mapped;
diff --git a/include/net/pie.h b/include/net/pie.h
index 01cbc66825a4..1f3db0c35514 100644
--- a/include/net/pie.h
+++ b/include/net/pie.h
@@ -104,7 +104,7 @@ static inline void pie_vars_init(struct pie_vars *vars)
vars->dq_tstamp = DTIME_INVALID;
vars->accu_prob = 0;
vars->dq_count = DQCOUNT_INVALID;
- vars->avg_dq_rate = 0;
+ WRITE_ONCE(vars->avg_dq_rate, 0);
}
static inline struct pie_skb_cb *get_pie_cb(const struct sk_buff *skb)
diff --git a/include/sound/soc-dai.h b/include/sound/soc-dai.h
index 0d1b215f24f4..4a3505674f2f 100644
--- a/include/sound/soc-dai.h
+++ b/include/sound/soc-dai.h
@@ -450,9 +450,9 @@ struct snd_soc_dai {
struct snd_soc_dai_stream stream[SNDRV_PCM_STREAM_LAST + 1];
/* Symmetry data - only valid if symmetry is being enforced */
- unsigned int rate;
- unsigned int channels;
- unsigned int sample_bits;
+ unsigned int symmetric_rate;
+ unsigned int symmetric_channels;
+ unsigned int symmetric_sample_bits;
/* parent platform/codec */
struct snd_soc_component *component;
diff --git a/include/sound/soc.h b/include/sound/soc.h
index db3b464a91c7..cd467f8babdb 100644
--- a/include/sound/soc.h
+++ b/include/sound/soc.h
@@ -1452,6 +1452,9 @@ struct snd_soc_dai *snd_soc_find_dai(
struct snd_soc_dai *snd_soc_find_dai_with_mutex(
const struct snd_soc_dai_link_component *dlc);
+void soc_pcm_set_dai_params(struct snd_soc_dai *dai,
+ struct snd_pcm_hw_params *params);
+
#include <sound/soc-dai.h>
static inline
diff --git a/include/trace/events/timer.h b/include/trace/events/timer.h
index 1ef58a04fc57..19c975ffea17 100644
--- a/include/trace/events/timer.h
+++ b/include/trace/events/timer.h
@@ -218,12 +218,13 @@ TRACE_EVENT(hrtimer_init,
* hrtimer_start - called when the hrtimer is started
* @hrtimer: pointer to struct hrtimer
* @mode: the hrtimers mode
+ * @was_armed: Was armed when hrtimer_start*() was invoked
*/
TRACE_EVENT(hrtimer_start,
- TP_PROTO(struct hrtimer *hrtimer, enum hrtimer_mode mode),
+ TP_PROTO(struct hrtimer *hrtimer, enum hrtimer_mode mode, bool was_armed),
- TP_ARGS(hrtimer, mode),
+ TP_ARGS(hrtimer, mode, was_armed),
TP_STRUCT__entry(
__field( void *, hrtimer )
@@ -231,6 +232,7 @@ TRACE_EVENT(hrtimer_start,
__field( s64, expires )
__field( s64, softexpires )
__field( enum hrtimer_mode, mode )
+ __field( bool, was_armed )
),
TP_fast_assign(
@@ -239,13 +241,14 @@ TRACE_EVENT(hrtimer_start,
__entry->expires = hrtimer_get_expires(hrtimer);
__entry->softexpires = hrtimer_get_softexpires(hrtimer);
__entry->mode = mode;
+ __entry->was_armed = was_armed;
),
TP_printk("hrtimer=%p function=%ps expires=%llu softexpires=%llu "
- "mode=%s", __entry->hrtimer, __entry->function,
+ "mode=%s was_armed=%d", __entry->hrtimer, __entry->function,
(unsigned long long) __entry->expires,
(unsigned long long) __entry->softexpires,
- decode_hrtimer_mode(__entry->mode))
+ decode_hrtimer_mode(__entry->mode), __entry->was_armed)
);
/**
diff --git a/include/uapi/linux/mii.h b/include/uapi/linux/mii.h
index 39f7c44baf53..61d6edad4b94 100644
--- a/include/uapi/linux/mii.h
+++ b/include/uapi/linux/mii.h
@@ -82,7 +82,8 @@
#define ADVERTISE_100BASE4 0x0200 /* Try for 100mbps 4k packets */
#define ADVERTISE_PAUSE_CAP 0x0400 /* Try for pause */
#define ADVERTISE_PAUSE_ASYM 0x0800 /* Try for asymetric pause */
-#define ADVERTISE_RESV 0x1000 /* Unused... */
+#define ADVERTISE_XNP 0x1000 /* Extended Next Page */
+#define ADVERTISE_RESV ADVERTISE_XNP /* Used to be reserved */
#define ADVERTISE_RFAULT 0x2000 /* Say we can detect faults */
#define ADVERTISE_LPACK 0x4000 /* Ack link partners response */
#define ADVERTISE_NPAGE 0x8000 /* Next page bit */
diff --git a/include/uapi/linux/virtio_net.h b/include/uapi/linux/virtio_net.h
index ac9174717ef1..963540deae66 100644
--- a/include/uapi/linux/virtio_net.h
+++ b/include/uapi/linux/virtio_net.h
@@ -327,6 +327,19 @@ struct virtio_net_rss_config {
__u8 hash_key_data[/* hash_key_length */];
};
+struct virtio_net_rss_config_hdr {
+ __le32 hash_types;
+ __le16 indirection_table_mask;
+ __le16 unclassified_queue;
+ __le16 indirection_table[/* 1 + indirection_table_mask */];
+};
+
+struct virtio_net_rss_config_trailer {
+ __le16 max_tx_vq;
+ __u8 hash_key_length;
+ __u8 hash_key_data[/* hash_key_length */];
+};
+
#define VIRTIO_NET_CTRL_MQ_RSS_CONFIG 1
/*
diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c
index faa00f163e23..0aa32ec6de63 100644
--- a/io_uring/io-wq.c
+++ b/io_uring/io-wq.c
@@ -1044,7 +1044,8 @@ static inline void io_wq_remove_pending(struct io_wq *wq,
if (io_wq_is_hashed(work) && work == wq->hash_tail[hash]) {
if (prev)
prev_work = container_of(prev, struct io_wq_work, list);
- if (prev_work && io_get_work_hash(prev_work) == hash)
+ if (prev_work && io_wq_is_hashed(prev_work) &&
+ io_get_work_hash(prev_work) == hash)
wq->hash_tail[hash] = prev_work;
else
wq->hash_tail[hash] = NULL;
diff --git a/io_uring/kbuf.c b/io_uring/kbuf.c
index bd6e5c0f683a..f67ecacd2543 100644
--- a/io_uring/kbuf.c
+++ b/io_uring/kbuf.c
@@ -705,8 +705,7 @@ int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
if (copy_from_user(®, arg, sizeof(reg)))
return -EFAULT;
-
- if (reg.resv[0] || reg.resv[1] || reg.resv[2])
+ if (!mem_is_zero(reg.resv, sizeof(reg.resv)))
return -EINVAL;
if (reg.flags & ~(IOU_PBUF_RING_MMAP | IOU_PBUF_RING_INC))
return -EINVAL;
@@ -773,9 +772,7 @@ int io_unregister_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
if (copy_from_user(®, arg, sizeof(reg)))
return -EFAULT;
- if (reg.resv[0] || reg.resv[1] || reg.resv[2])
- return -EINVAL;
- if (reg.flags)
+ if (!mem_is_zero(reg.resv, sizeof(reg.resv)) || reg.flags)
return -EINVAL;
bl = io_buffer_get_list(ctx, reg.bgid);
@@ -793,14 +790,11 @@ int io_register_pbuf_status(struct io_ring_ctx *ctx, void __user *arg)
{
struct io_uring_buf_status buf_status;
struct io_buffer_list *bl;
- int i;
if (copy_from_user(&buf_status, arg, sizeof(buf_status)))
return -EFAULT;
-
- for (i = 0; i < ARRAY_SIZE(buf_status.resv); i++)
- if (buf_status.resv[i])
- return -EINVAL;
+ if (!mem_is_zero(buf_status.resv, sizeof(buf_status.resv)))
+ return -EINVAL;
bl = io_buffer_get_list(ctx, buf_status.buf_group);
if (!bl)
diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
index 44e7959b52d9..b39067a049fd 100644
--- a/io_uring/sqpoll.c
+++ b/io_uring/sqpoll.c
@@ -115,29 +115,21 @@ static struct io_sq_data *io_attach_sq_data(struct io_uring_params *p)
{
struct io_ring_ctx *ctx_attach;
struct io_sq_data *sqd;
- struct fd f;
+ CLASS(fd, f)(p->wq_fd);
- f = fdget(p->wq_fd);
- if (!fd_file(f))
+ if (fd_empty(f))
return ERR_PTR(-ENXIO);
- if (!io_is_uring_fops(fd_file(f))) {
- fdput(f);
+ if (!io_is_uring_fops(fd_file(f)))
return ERR_PTR(-EINVAL);
- }
ctx_attach = fd_file(f)->private_data;
sqd = ctx_attach->sq_data;
- if (!sqd) {
- fdput(f);
+ if (!sqd)
return ERR_PTR(-EINVAL);
- }
- if (sqd->task_tgid != current->tgid) {
- fdput(f);
+ if (sqd->task_tgid != current->tgid)
return ERR_PTR(-EPERM);
- }
refcount_inc(&sqd->refs);
- fdput(f);
return sqd;
}
@@ -456,16 +448,11 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx,
/* Retain compatibility with failing for an invalid attach attempt */
if ((ctx->flags & (IORING_SETUP_ATTACH_WQ | IORING_SETUP_SQPOLL)) ==
IORING_SETUP_ATTACH_WQ) {
- struct fd f;
-
- f = fdget(p->wq_fd);
- if (!fd_file(f))
+ CLASS(fd, f)(p->wq_fd);
+ if (fd_empty(f))
return -ENXIO;
- if (!io_is_uring_fops(fd_file(f))) {
- fdput(f);
+ if (!io_is_uring_fops(fd_file(f)))
return -EINVAL;
- }
- fdput(f);
}
if (ctx->flags & IORING_SETUP_SQPOLL) {
struct task_struct *tsk;
diff --git a/kernel/audit.c b/kernel/audit.c
index 1edaa4846a47..4a3dcce7bdc9 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -1427,6 +1427,8 @@ static int audit_receive_msg(struct sk_buff *skb, struct nlmsghdr *nlh,
err = audit_list_rules_send(skb, seq);
break;
case AUDIT_TRIM:
+ if (audit_enabled == AUDIT_LOCKED)
+ return -EPERM;
audit_trim_trees();
audit_log_common_recv_msg(audit_context(), &ab,
AUDIT_CONFIG_CHANGE);
@@ -1439,6 +1441,8 @@ static int audit_receive_msg(struct sk_buff *skb, struct nlmsghdr *nlh,
size_t msglen = data_len;
char *old, *new;
+ if (audit_enabled == AUDIT_LOCKED)
+ return -EPERM;
err = -EINVAL;
if (msglen < 2 * sizeof(u32))
break;
diff --git a/kernel/auditsc.c b/kernel/auditsc.c
index dae80e4dfcce..3b3ca3734349 100644
--- a/kernel/auditsc.c
+++ b/kernel/auditsc.c
@@ -2845,7 +2845,7 @@ void __audit_log_capset(const struct cred *new, const struct cred *old)
context->capset.pid = task_tgid_nr(current);
context->capset.cap.effective = new->cap_effective;
- context->capset.cap.inheritable = new->cap_effective;
+ context->capset.cap.inheritable = new->cap_inheritable;
context->capset.cap.permitted = new->cap_permitted;
context->capset.cap.ambient = new->cap_ambient;
context->type = AUDIT_CAPSET;
diff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c
index 4ce6786d3935..187e4871b74b 100644
--- a/kernel/bpf/arena.c
+++ b/kernel/bpf/arena.c
@@ -438,6 +438,10 @@ static long arena_alloc_pages(struct bpf_arena *arena, long uaddr, long page_cnt
u32 uaddr32;
int ret, i;
+ if (node_id != NUMA_NO_NODE &&
+ ((unsigned int)node_id >= nr_node_ids || !node_online(node_id)))
+ return 0;
+
if (page_cnt > page_cnt_max)
return 0;
diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c
index 6cdbb4c33d31..7ec69545fe05 100644
--- a/kernel/bpf/arraymap.c
+++ b/kernel/bpf/arraymap.c
@@ -998,8 +998,10 @@ static void bpf_fd_array_map_clear(struct bpf_map *map, bool need_defer)
struct bpf_array *array = container_of(map, struct bpf_array, map);
int i;
- for (i = 0; i < array->map.max_entries; i++)
+ for (i = 0; i < array->map.max_entries; i++) {
__fd_array_map_delete_elem(map, &i, need_defer);
+ cond_resched();
+ }
}
static void prog_array_map_seq_show_elem(struct bpf_map *map, void *key,
diff --git a/kernel/bpf/bpf_lsm.c b/kernel/bpf/bpf_lsm.c
index 3bc61628ab25..0849453b3617 100644
--- a/kernel/bpf/bpf_lsm.c
+++ b/kernel/bpf/bpf_lsm.c
@@ -355,8 +355,6 @@ BTF_ID(func, bpf_lsm_sb_umount)
BTF_ID(func, bpf_lsm_settime)
#ifdef CONFIG_SECURITY_NETWORK
-BTF_ID(func, bpf_lsm_inet_conn_established)
-
BTF_ID(func, bpf_lsm_socket_accept)
BTF_ID(func, bpf_lsm_socket_bind)
BTF_ID(func, bpf_lsm_socket_connect)
@@ -379,7 +377,6 @@ BTF_ID(func, bpf_lsm_current_getsecid_subj)
BTF_ID(func, bpf_lsm_task_getsecid_obj)
BTF_ID(func, bpf_lsm_task_prctl)
BTF_ID(func, bpf_lsm_task_setscheduler)
-BTF_ID(func, bpf_lsm_task_to_inode)
BTF_ID(func, bpf_lsm_userns_create)
BTF_SET_END(sleepable_lsm_hooks)
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index b58833e99969..517710c89fa5 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1489,6 +1489,8 @@ void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
* know whether fp here is the clone or the original.
*/
fp->aux->prog = fp;
+ if (fp->aux->offload)
+ fp->aux->offload->prog = fp;
bpf_prog_clone_free(fp_other);
}
diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c
index 39b7efa396b8..17f8c9d6e95c 100644
--- a/kernel/bpf/devmap.c
+++ b/kernel/bpf/devmap.c
@@ -653,7 +653,7 @@ int dev_map_enqueue_multi(struct xdp_frame *xdpf, struct net_device *dev_rx,
for (i = 0; i < dtab->n_buckets; i++) {
head = dev_map_index_hash(dtab, i);
hlist_for_each_entry_rcu(dst, head, index_hlist,
- lockdep_is_held(&dtab->index_lock)) {
+ rcu_read_lock_bh_held()) {
if (!is_valid_dst(dst, xdpf))
continue;
@@ -735,7 +735,6 @@ int dev_map_redirect_multi(struct net_device *dev, struct sk_buff *skb,
struct bpf_dtab_netdev *dst, *last_dst = NULL;
int excluded_devices[1+MAX_NEST_DEV];
struct hlist_head *head;
- struct hlist_node *next;
int num_excluded = 0;
unsigned int i;
int err;
@@ -775,7 +774,7 @@ int dev_map_redirect_multi(struct net_device *dev, struct sk_buff *skb,
} else { /* BPF_MAP_TYPE_DEVMAP_HASH */
for (i = 0; i < dtab->n_buckets; i++) {
head = dev_map_index_hash(dtab, i);
- hlist_for_each_entry_safe(dst, next, head, index_hlist) {
+ hlist_for_each_entry_rcu(dst, head, index_hlist, rcu_read_lock_bh_held()) {
if (is_ifindex_excluded(excluded_devices, num_excluded,
dst->dev->ifindex))
continue;
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 26883a997e71..9f9026a619d5 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -995,7 +995,7 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
for_each_possible_cpu(cpu) {
if (cpu == current_cpu)
- copy_map_value_long(&htab->map, per_cpu_ptr(pptr, cpu), value);
+ copy_map_value(&htab->map, per_cpu_ptr(pptr, cpu), value);
else /* Since elem is preallocated, we cannot touch special fields */
zero_map_value(&htab->map, per_cpu_ptr(pptr, cpu));
}
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index db4739951702..8aa7ca9c34c7 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -847,7 +847,13 @@ int bpf_bprintf_prepare(char *fmt, u32 fmt_size, const u64 *raw_args,
data->buf = buffers->buf;
for (i = 0; i < fmt_size; i++) {
- if ((!isprint(fmt[i]) && !isspace(fmt[i])) || !isascii(fmt[i])) {
+ unsigned char c = fmt[i];
+
+ /*
+ * Permit bytes >= 0x80 in plain text so UTF-8 literals can pass
+ * through unchanged, while still rejecting ASCII control bytes.
+ */
+ if (isascii(c) && !isprint(c) && !isspace(c)) {
err = -EINVAL;
goto out;
}
@@ -869,6 +875,15 @@ int bpf_bprintf_prepare(char *fmt, u32 fmt_size, const u64 *raw_args,
* always access fmt[i + 1], in the worst case it will be a 0
*/
i++;
+ c = fmt[i];
+ /*
+ * The format parser below only understands ASCII conversion
+ * specifiers and modifiers, so reject non-ASCII after '%'.
+ */
+ if (!isascii(c)) {
+ err = -EINVAL;
+ goto out;
+ }
/* skip optional "[0 +-][num]" width formatting field */
while (fmt[i] == '0' || fmt[i] == '+' || fmt[i] == '-' ||
diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c
index 3969eb0382af..cfb4ff261051 100644
--- a/kernel/bpf/local_storage.c
+++ b/kernel/bpf/local_storage.c
@@ -259,7 +259,7 @@ static int cgroup_storage_get_next_key(struct bpf_map *_map, void *key,
goto enoent;
storage = list_next_entry(storage, list_map);
- if (!storage)
+ if (list_entry_is_head(storage, &map->list, list_map))
goto enoent;
} else {
storage = list_first_entry(&map->list,
diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c
index 5af9e130e500..fc5f463ca529 100644
--- a/kernel/bpf/task_iter.c
+++ b/kernel/bpf/task_iter.c
@@ -10,6 +10,8 @@
#include <linux/bpf_mem_alloc.h>
#include <linux/btf_ids.h>
#include <linux/mm_types.h>
+#include <linux/mmap_lock.h>
+#include <linux/sched/mm.h>
#include "mmap_unlock_work.h"
static const char * const iter_task_type_names[] = {
@@ -798,11 +800,20 @@ const struct bpf_func_proto bpf_find_vma_proto = {
.arg5_type = ARG_ANYTHING,
};
+static inline void bpf_iter_mmput_async(struct mm_struct *mm)
+{
+#ifdef CONFIG_MMU
+ mmput_async(mm);
+#else
+ mmput(mm);
+#endif
+}
+
struct bpf_iter_task_vma_kern_data {
struct task_struct *task;
struct mm_struct *mm;
- struct mmap_unlock_irq_work *work;
- struct vma_iterator vmi;
+ struct vm_area_struct snapshot;
+ u64 next_addr;
};
struct bpf_iter_task_vma {
@@ -823,12 +834,28 @@ __bpf_kfunc int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it,
struct task_struct *task, u64 addr)
{
struct bpf_iter_task_vma_kern *kit = (void *)it;
- bool irq_work_busy = false;
int err;
BUILD_BUG_ON(sizeof(struct bpf_iter_task_vma_kern) != sizeof(struct bpf_iter_task_vma));
BUILD_BUG_ON(__alignof__(struct bpf_iter_task_vma_kern) != __alignof__(struct bpf_iter_task_vma));
+ if (!IS_ENABLED(CONFIG_PER_VMA_LOCK)) {
+ kit->data = NULL;
+ return -EOPNOTSUPP;
+ }
+
+ /*
+ * Reject irqs-disabled contexts including NMI. Operations used
+ * by _next() and _destroy() (vma_end_read, fput, bpf_iter_mmput_async)
+ * can take spinlocks with IRQs disabled (pi_lock, pool->lock).
+ * Running from NMI or from a tracepoint that fires with those
+ * locks held could deadlock.
+ */
+ if (irqs_disabled()) {
+ kit->data = NULL;
+ return -EBUSY;
+ }
+
/* is_iter_reg_valid_uninit guarantees that kit hasn't been initialized
* before, so non-NULL kit->data doesn't point to previously
* bpf_mem_alloc'd bpf_iter_task_vma_kern_data
@@ -838,38 +865,131 @@ __bpf_kfunc int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it,
return -ENOMEM;
kit->data->task = get_task_struct(task);
+ /*
+ * Safely read task->mm and acquire an mm reference.
+ *
+ * Cannot use get_task_mm() because its task_lock() is a
+ * blocking spin_lock that would deadlock if the target task
+ * already holds alloc_lock on this CPU (e.g. a softirq BPF
+ * program iterating a task interrupted while holding its
+ * alloc_lock).
+ */
+ if (!spin_trylock(&task->alloc_lock)) {
+ err = -EBUSY;
+ goto err_cleanup_iter;
+ }
kit->data->mm = task->mm;
+ if (kit->data->mm && !(task->flags & PF_KTHREAD))
+ mmget(kit->data->mm);
+ else
+ kit->data->mm = NULL;
+ spin_unlock(&task->alloc_lock);
if (!kit->data->mm) {
err = -ENOENT;
goto err_cleanup_iter;
}
- /* kit->data->work == NULL is valid after bpf_mmap_unlock_get_irq_work */
- irq_work_busy = bpf_mmap_unlock_get_irq_work(&kit->data->work);
- if (irq_work_busy || !mmap_read_trylock(kit->data->mm)) {
- err = -EBUSY;
- goto err_cleanup_iter;
- }
-
- vma_iter_init(&kit->data->vmi, kit->data->mm, addr);
+ kit->data->snapshot.vm_file = NULL;
+ kit->data->next_addr = addr;
return 0;
err_cleanup_iter:
- if (kit->data->task)
- put_task_struct(kit->data->task);
+ put_task_struct(kit->data->task);
bpf_mem_free(&bpf_global_ma, kit->data);
/* NULL kit->data signals failed bpf_iter_task_vma initialization */
kit->data = NULL;
return err;
}
+/*
+ * Find and lock the next VMA at or after data->next_addr.
+ *
+ * lock_vma_under_rcu() is a point lookup (mas_walk): it finds the VMA
+ * containing a given address but cannot iterate. An RCU-protected
+ * maple tree walk with vma_next() (mas_find) is needed first to locate
+ * the next VMA's vm_start across any gap.
+ *
+ * Between the RCU walk and the lock, the VMA may be removed, shrunk,
+ * or write-locked. On failure, advance past it using vm_end from the
+ * RCU walk. SLAB_TYPESAFE_BY_RCU can make vm_end stale, so fall back
+ * to PAGE_SIZE advancement to guarantee forward progress.
+ */
+static struct vm_area_struct *
+bpf_iter_task_vma_find_next(struct bpf_iter_task_vma_kern_data *data)
+{
+ struct vm_area_struct *vma;
+ struct vma_iterator vmi;
+ unsigned long start, end;
+
+retry:
+ rcu_read_lock();
+ vma_iter_init(&vmi, data->mm, data->next_addr);
+ vma = vma_next(&vmi);
+ if (!vma) {
+ rcu_read_unlock();
+ return NULL;
+ }
+ start = vma->vm_start;
+ end = vma->vm_end;
+ rcu_read_unlock();
+
+ vma = lock_vma_under_rcu(data->mm, start);
+ if (!vma) {
+ if (end <= data->next_addr)
+ data->next_addr += PAGE_SIZE;
+ else
+ data->next_addr = end;
+ goto retry;
+ }
+
+ if (unlikely(vma->vm_end <= data->next_addr)) {
+ data->next_addr += PAGE_SIZE;
+ vma_end_read(vma);
+ goto retry;
+ }
+
+ return vma;
+}
+
+static void bpf_iter_task_vma_snapshot_reset(struct vm_area_struct *snap)
+{
+ if (snap->vm_file) {
+ fput(snap->vm_file);
+ snap->vm_file = NULL;
+ }
+}
+
__bpf_kfunc struct vm_area_struct *bpf_iter_task_vma_next(struct bpf_iter_task_vma *it)
{
struct bpf_iter_task_vma_kern *kit = (void *)it;
+ struct vm_area_struct *snap, *vma;
if (!kit->data) /* bpf_iter_task_vma_new failed */
return NULL;
- return vma_next(&kit->data->vmi);
+
+ snap = &kit->data->snapshot;
+
+ bpf_iter_task_vma_snapshot_reset(snap);
+
+ vma = bpf_iter_task_vma_find_next(kit->data);
+ if (!vma)
+ return NULL;
+
+ memcpy(snap, vma, sizeof(*snap));
+
+ /*
+ * The verifier only trusts vm_mm and vm_file (see
+ * BTF_TYPE_SAFE_TRUSTED_OR_NULL in verifier.c). Take a reference
+ * on vm_file; vm_mm is already correct because lock_vma_under_rcu()
+ * verifies vma->vm_mm == mm. All other pointers are untrusted by
+ * the verifier and left as-is.
+ */
+ if (snap->vm_file)
+ get_file(snap->vm_file);
+
+ kit->data->next_addr = vma->vm_end;
+ vma_end_read(vma);
+ return snap;
}
__bpf_kfunc void bpf_iter_task_vma_destroy(struct bpf_iter_task_vma *it)
@@ -877,8 +997,9 @@ __bpf_kfunc void bpf_iter_task_vma_destroy(struct bpf_iter_task_vma *it)
struct bpf_iter_task_vma_kern *kit = (void *)it;
if (kit->data) {
- bpf_mmap_unlock_mm(kit->data->work, kit->data->mm);
+ bpf_iter_task_vma_snapshot_reset(&kit->data->snapshot);
put_task_struct(kit->data->task);
+ bpf_iter_mmput_async(kit->data->mm);
bpf_mem_free(&bpf_global_ma, kit->data);
}
}
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 96a04cd904a1..64a6ec8eb847 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -4724,6 +4724,18 @@ static void check_fastcall_stack_contract(struct bpf_verifier_env *env,
}
}
+static void scrub_special_slot(struct bpf_func_state *state, int spi)
+{
+ int i;
+
+ /* regular write of data into stack destroys any spilled ptr */
+ state->stack[spi].spilled_ptr.type = NOT_INIT;
+ /* Mark slots as STACK_MISC if they belonged to spilled ptr/dynptr/iter. */
+ if (is_stack_slot_special(&state->stack[spi]))
+ for (i = 0; i < BPF_REG_SIZE; i++)
+ scrub_spilled_slot(&state->stack[spi].slot_type[i]);
+}
+
/* check_stack_{read,write}_fixed_off functions track spill/fill of registers,
* stack boundary and alignment are checked in check_mem_access()
*/
@@ -4809,12 +4821,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,
} else {
u8 type = STACK_MISC;
- /* regular write of data into stack destroys any spilled ptr */
- state->stack[spi].spilled_ptr.type = NOT_INIT;
- /* Mark slots as STACK_MISC if they belonged to spilled ptr/dynptr/iter. */
- if (is_stack_slot_special(&state->stack[spi]))
- for (i = 0; i < BPF_REG_SIZE; i++)
- scrub_spilled_slot(&state->stack[spi].slot_type[i]);
+ scrub_special_slot(state, spi);
/* only mark the slot as written if all 8 bytes were written
* otherwise read propagation may incorrectly stop too soon
@@ -4949,8 +4956,13 @@ static int check_stack_write_var_off(struct bpf_verifier_env *env,
}
}
- /* Erase all other spilled pointers. */
- state->stack[spi].spilled_ptr.type = NOT_INIT;
+ /*
+ * Scrub slots if variable-offset stack write goes over spilled pointers.
+ * Otherwise is_spilled_reg() may == true && spilled_ptr.type == NOT_INIT
+ * and valid program is rejected by check_stack_read_fixed_off()
+ * with obscure "invalid size of register fill" message.
+ */
+ scrub_special_slot(state, spi);
/* Update the slot type. */
new_type = STACK_MISC;
@@ -5461,6 +5473,9 @@ static int map_kptr_match_type(struct bpf_verifier_env *env,
int perm_flags;
const char *reg_name = "";
+ if (base_type(reg->type) != PTR_TO_BTF_ID)
+ goto bad_type;
+
if (btf_is_kernel(reg->btf)) {
perm_flags = PTR_MAYBE_NULL | PTR_TRUSTED | MEM_RCU;
@@ -5473,7 +5488,7 @@ static int map_kptr_match_type(struct bpf_verifier_env *env,
perm_flags |= MEM_PERCPU;
}
- if (base_type(reg->type) != PTR_TO_BTF_ID || (type_flag(reg->type) & ~perm_flags))
+ if (type_flag(reg->type) & ~perm_flags)
goto bad_type;
/* We need to verify reg->type and reg->btf, before accessing reg->btf */
@@ -14422,11 +14437,20 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
int err;
dst_reg = ®s[insn->dst_reg];
- src_reg = NULL;
+ if (BPF_SRC(insn->code) == BPF_X)
+ src_reg = ®s[insn->src_reg];
+ else
+ src_reg = NULL;
- if (dst_reg->type == PTR_TO_ARENA) {
+ /* Case where at least one operand is an arena. */
+ if (dst_reg->type == PTR_TO_ARENA || (src_reg && src_reg->type == PTR_TO_ARENA)) {
struct bpf_insn_aux_data *aux = cur_aux(env);
+ if (dst_reg->type != PTR_TO_ARENA)
+ *dst_reg = *src_reg;
+
+ dst_reg->subreg_def = env->insn_idx + 1;
+
if (BPF_CLASS(insn->code) == BPF_ALU64)
/*
* 32-bit operations zero upper bits automatically.
@@ -14442,7 +14466,6 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
ptr_reg = dst_reg;
if (BPF_SRC(insn->code) == BPF_X) {
- src_reg = ®s[insn->src_reg];
if (src_reg->type != SCALAR_VALUE) {
if (dst_reg->type != SCALAR_VALUE) {
/* Combining two pointers by any ALU op yields
@@ -17413,16 +17436,47 @@ static bool check_ids(u32 old_id, u32 cur_id, struct bpf_idmap *idmap)
return false;
}
-/* Similar to check_ids(), but allocate a unique temporary ID
- * for 'old_id' or 'cur_id' of zero.
- * This makes pairs like '0 vs unique ID', 'unique ID vs 0' valid.
+/*
+ * Compare scalar register IDs for state equivalence.
+ *
+ * When old_id == 0, the old register is independent - not linked to any
+ * other register. Any linking in the current state only adds constraints,
+ * making it more restrictive. Since the old state didn't rely on any ID
+ * relationships for this register, it's always safe to accept cur regardless
+ * of its ID. Hence, return true immediately.
+ *
+ * When old_id != 0 but cur_id == 0, we need to ensure that different
+ * independent registers in cur don't incorrectly satisfy the ID matching
+ * requirements of linked registers in old.
+ *
+ * Example: if old has r6.id=X and r7.id=X (linked), but cur has r6.id=0
+ * and r7.id=0 (both independent), without temp IDs both would map old_id=X
+ * to cur_id=0 and pass. With temp IDs: r6 maps X->temp1, r7 tries to map
+ * X->temp2, but X is already mapped to temp1, so the check fails correctly.
+ *
+ * When old_id has BPF_ADD_CONST set, the compound id (base | flag) and the
+ * base id (flag stripped) must both map consistently. Example: old has
+ * r2.id=A, r3.id=A|flag (r3 = r2 + delta), cur has r2.id=B, r3.id=C|flag
+ * (r3 derived from unrelated r4). Without the base check, idmap gets two
+ * independent entries A->B and A|flag->C|flag, missing that A->C conflicts
+ * with A->B. The base ID cross-check catches this.
*/
static bool check_scalar_ids(u32 old_id, u32 cur_id, struct bpf_idmap *idmap)
{
- old_id = old_id ? old_id : ++idmap->tmp_id_gen;
+ if (!old_id)
+ return true;
+
cur_id = cur_id ? cur_id : ++idmap->tmp_id_gen;
- return check_ids(old_id, cur_id, idmap);
+ if (!check_ids(old_id, cur_id, idmap))
+ return false;
+ if (old_id & BPF_ADD_CONST) {
+ old_id &= ~BPF_ADD_CONST;
+ cur_id &= ~BPF_ADD_CONST;
+ if (!check_ids(old_id, cur_id, idmap))
+ return false;
+ }
+ return true;
}
static void clean_func_state(struct bpf_verifier_env *env,
@@ -17588,11 +17642,21 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold,
}
if (!rold->precise && exact == NOT_EXACT)
return true;
- if ((rold->id & BPF_ADD_CONST) != (rcur->id & BPF_ADD_CONST))
- return false;
- if ((rold->id & BPF_ADD_CONST) && (rold->off != rcur->off))
- return false;
- /* Why check_ids() for scalar registers?
+ /*
+ * Linked register tracking uses rold->id to detect relationships.
+ * When rold->id == 0, the register is independent and any linking
+ * in rcur only adds constraints. When rold->id != 0, we must verify
+ * id mapping and (for BPF_ADD_CONST) offset consistency.
+ *
+ * +------------------+-----------+------------------+---------------+
+ * | | rold->id | rold + ADD_CONST | rold->id == 0 |
+ * |------------------+-----------+------------------+---------------|
+ * | rcur->id | range,ids | false | range |
+ * | rcur + ADD_CONST | false | range,ids,off | range |
+ * | rcur->id == 0 | range,ids | false | range |
+ * +------------------+-----------+------------------+---------------+
+ *
+ * Why check_ids() for scalar registers?
*
* Consider the following BPF code:
* 1: r6 = ... unbound scalar, ID=a ...
@@ -17616,9 +17680,22 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold,
* ---
* Also verify that new value satisfies old value range knowledge.
*/
- return range_within(rold, rcur) &&
- tnum_in(rold->var_off, rcur->var_off) &&
- check_scalar_ids(rold->id, rcur->id, idmap);
+
+ /* ADD_CONST mismatch: different linking semantics */
+ if ((rold->id & BPF_ADD_CONST) && !(rcur->id & BPF_ADD_CONST))
+ return false;
+
+ if (rold->id && !(rold->id & BPF_ADD_CONST) && (rcur->id & BPF_ADD_CONST))
+ return false;
+
+ /* Both have offset linkage: offsets must match */
+ if ((rold->id & BPF_ADD_CONST) && rold->off != rcur->off)
+ return false;
+
+ if (!check_scalar_ids(rold->id, rcur->id, idmap))
+ return false;
+
+ return range_within(rold, rcur) && tnum_in(rold->var_off, rcur->var_off);
case PTR_TO_MAP_KEY:
case PTR_TO_MAP_VALUE:
case PTR_TO_MEM:
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 046f671532b0..3c7d466f13df 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -5646,16 +5646,6 @@ static void offline_css(struct cgroup_subsys_state *css)
RCU_INIT_POINTER(css->cgroup->subsys[ss->id], NULL);
wake_up_all(&css->cgroup->offline_waitq);
-
- css->cgroup->nr_dying_subsys[ss->id]++;
- /*
- * Parent css and cgroup cannot be freed until after the freeing
- * of child css, see css_free_rwork_fn().
- */
- while ((css = css->parent)) {
- css->nr_descendants--;
- css->cgroup->nr_dying_subsys[ss->id]++;
- }
}
/**
@@ -5957,6 +5947,8 @@ static void css_killed_ref_fn(struct percpu_ref *ref)
*/
static void kill_css(struct cgroup_subsys_state *css)
{
+ struct cgroup_subsys *ss = css->ss;
+
lockdep_assert_held(&cgroup_mutex);
if (css->flags & CSS_DYING)
@@ -5993,6 +5985,16 @@ static void kill_css(struct cgroup_subsys_state *css)
* css is confirmed to be seen as killed on all CPUs.
*/
percpu_ref_kill_and_confirm(&css->refcnt, css_killed_ref_fn);
+
+ css->cgroup->nr_dying_subsys[ss->id]++;
+ /*
+ * Parent css and cgroup cannot be freed until after the freeing
+ * of child css, see css_free_rwork_fn().
+ */
+ while ((css = css->parent)) {
+ css->nr_descendants--;
+ css->cgroup->nr_dying_subsys[ss->id]++;
+ }
}
/**
diff --git a/kernel/cgroup/rdma.c b/kernel/cgroup/rdma.c
index ef5878fb2005..d544a747f395 100644
--- a/kernel/cgroup/rdma.c
+++ b/kernel/cgroup/rdma.c
@@ -283,7 +283,7 @@ int rdmacg_try_charge(struct rdma_cgroup **rdmacg,
ret = PTR_ERR(rpool);
goto err;
} else {
- new = rpool->resources[index].usage + 1;
+ new = (s64)rpool->resources[index].usage + 1;
if (new > rpool->resources[index].max) {
ret = -EAGAIN;
goto err;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index bcedf9611cf4..6fce2bac6dae 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -984,22 +984,20 @@ static inline int perf_cgroup_connect(int fd, struct perf_event *event,
{
struct perf_cgroup *cgrp;
struct cgroup_subsys_state *css;
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
int ret = 0;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
css = css_tryget_online_from_dir(fd_file(f)->f_path.dentry,
&perf_event_cgrp_subsys);
- if (IS_ERR(css)) {
- ret = PTR_ERR(css);
- goto out;
- }
+ if (IS_ERR(css))
+ return PTR_ERR(css);
ret = perf_cgroup_ensure_storage(event, css);
if (ret)
- goto out;
+ return ret;
cgrp = container_of(css, struct perf_cgroup, css);
event->cgrp = cgrp;
@@ -1013,8 +1011,6 @@ static inline int perf_cgroup_connect(int fd, struct perf_event *event,
perf_detach_cgroup(event);
ret = -EINVAL;
}
-out:
- fdput(f);
return ret;
}
diff --git a/kernel/fork.c b/kernel/fork.c
index c6415bb0abf5..c4955cffcb6f 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -3348,11 +3348,10 @@ int ksys_unshare(unsigned long unshare_flags)
new_cred, new_fs);
if (err)
goto bad_unshare_cleanup_cred;
-
if (new_cred) {
err = set_cred_ucounts(new_cred);
if (err)
- goto bad_unshare_cleanup_cred;
+ goto bad_unshare_cleanup_nsproxy;
}
if (new_fs || new_fd || do_sysvsem || new_cred || new_nsproxy) {
@@ -3368,8 +3367,10 @@ int ksys_unshare(unsigned long unshare_flags)
shm_init_task(current);
}
- if (new_nsproxy)
+ if (new_nsproxy) {
switch_task_namespaces(current, new_nsproxy);
+ new_nsproxy = NULL;
+ }
task_lock(current);
@@ -3398,13 +3399,15 @@ int ksys_unshare(unsigned long unshare_flags)
perf_event_namespaces(current);
+bad_unshare_cleanup_nsproxy:
+ if (new_nsproxy)
+ put_nsproxy(new_nsproxy);
bad_unshare_cleanup_cred:
if (new_cred)
put_cred(new_cred);
bad_unshare_cleanup_fd:
if (new_fd)
put_files_struct(new_fd);
-
bad_unshare_cleanup_fs:
if (new_fs)
free_fs_struct(new_fs);
diff --git a/kernel/futex/requeue.c b/kernel/futex/requeue.c
index 559aae55792c..f4a69942780f 100644
--- a/kernel/futex/requeue.c
+++ b/kernel/futex/requeue.c
@@ -309,8 +309,11 @@ futex_proxy_trylock_atomic(u32 __user *pifutex, struct futex_hash_bucket *hb1,
return -EINVAL;
/* Ensure that this does not race against an early wakeup */
- if (!futex_requeue_pi_prepare(top_waiter, NULL))
+ if (!futex_requeue_pi_prepare(top_waiter, NULL)) {
+ plist_del(&top_waiter->list, &hb1->chain);
+ futex_hb_waiters_dec(hb1);
return -EAGAIN;
+ }
/*
* Try to take the lock for top_waiter and set the FUTEX_WAITERS bit
@@ -711,10 +714,12 @@ int handle_early_requeue_pi_wakeup(struct futex_hash_bucket *hb,
/*
* We were woken prior to requeue by a timeout or a signal.
- * Unqueue the futex_q and determine which it was.
+ * Conditionally unqueue the futex_q and determine which it was.
*/
- plist_del(&q->list, &hb->chain);
- futex_hb_waiters_dec(hb);
+ if (!plist_node_empty(&q->list)) {
+ plist_del(&q->list, &hb->chain);
+ futex_hb_waiters_dec(hb);
+ }
/* Handle spurious wakeups gracefully */
ret = -EWOULDBLOCK;
diff --git a/kernel/module/main.c b/kernel/module/main.c
index 915a9cf33dd0..ad58c44fb74f 100644
--- a/kernel/module/main.c
+++ b/kernel/module/main.c
@@ -1286,7 +1286,7 @@ static void free_module(struct module *mod)
module_unload_free(mod);
/* Free any allocated parameters. */
- destroy_params(mod->kp, mod->num_kp);
+ module_destroy_params(mod->kp, mod->num_kp);
if (is_livepatch_module(mod))
free_module_elf(mod);
@@ -3022,7 +3022,7 @@ static int load_module(struct load_info *info, const char __user *uargs,
mod_sysfs_teardown(mod);
coming_cleanup:
mod->state = MODULE_STATE_GOING;
- destroy_params(mod->kp, mod->num_kp);
+ module_destroy_params(mod->kp, mod->num_kp);
blocking_notifier_call_chain(&module_notify_list,
MODULE_STATE_GOING, mod);
klp_module_going(mod);
diff --git a/kernel/nsproxy.c b/kernel/nsproxy.c
index dc952c3b05af..c9d97ed20122 100644
--- a/kernel/nsproxy.c
+++ b/kernel/nsproxy.c
@@ -545,12 +545,12 @@ static void commit_nsset(struct nsset *nsset)
SYSCALL_DEFINE2(setns, int, fd, int, flags)
{
- struct fd f = fdget(fd);
+ CLASS(fd, f)(fd);
struct ns_common *ns = NULL;
struct nsset nsset = {};
int err = 0;
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
if (proc_ns_file(fd_file(f))) {
@@ -580,7 +580,6 @@ SYSCALL_DEFINE2(setns, int, fd, int, flags)
}
put_nsset(&nsset);
out:
- fdput(f);
return err;
}
diff --git a/kernel/padata.c b/kernel/padata.c
index e61bdc248551..483239c7fe82 100644
--- a/kernel/padata.c
+++ b/kernel/padata.c
@@ -558,7 +558,8 @@ static void padata_init_reorder_list(struct parallel_data *pd)
}
/* Allocate and initialize the internal cpumask dependend resources. */
-static struct parallel_data *padata_alloc_pd(struct padata_shell *ps)
+static struct parallel_data *padata_alloc_pd(struct padata_shell *ps,
+ int offlining_cpu)
{
struct padata_instance *pinst = ps->pinst;
struct parallel_data *pd;
@@ -584,6 +585,10 @@ static struct parallel_data *padata_alloc_pd(struct padata_shell *ps)
cpumask_and(pd->cpumask.pcpu, pinst->cpumask.pcpu, cpu_online_mask);
cpumask_and(pd->cpumask.cbcpu, pinst->cpumask.cbcpu, cpu_online_mask);
+ if (offlining_cpu >= 0) {
+ __cpumask_clear_cpu(offlining_cpu, pd->cpumask.pcpu);
+ __cpumask_clear_cpu(offlining_cpu, pd->cpumask.cbcpu);
+ }
padata_init_reorder_list(pd);
padata_init_squeues(pd);
@@ -630,11 +635,11 @@ static void __padata_stop(struct padata_instance *pinst)
}
/* Replace the internal control structure with a new one. */
-static int padata_replace_one(struct padata_shell *ps)
+static int padata_replace_one(struct padata_shell *ps, int offlining_cpu)
{
struct parallel_data *pd_new;
- pd_new = padata_alloc_pd(ps);
+ pd_new = padata_alloc_pd(ps, offlining_cpu);
if (!pd_new)
return -ENOMEM;
@@ -644,7 +649,7 @@ static int padata_replace_one(struct padata_shell *ps)
return 0;
}
-static int padata_replace(struct padata_instance *pinst)
+static int padata_replace(struct padata_instance *pinst, int offlining_cpu)
{
struct padata_shell *ps;
int err = 0;
@@ -652,7 +657,7 @@ static int padata_replace(struct padata_instance *pinst)
pinst->flags |= PADATA_RESET;
list_for_each_entry(ps, &pinst->pslist, list) {
- err = padata_replace_one(ps);
+ err = padata_replace_one(ps, offlining_cpu);
if (err)
break;
}
@@ -669,9 +674,21 @@ static int padata_replace(struct padata_instance *pinst)
/* If cpumask contains no active cpu, we mark the instance as invalid. */
static bool padata_validate_cpumask(struct padata_instance *pinst,
- const struct cpumask *cpumask)
+ const struct cpumask *cpumask,
+ int offlining_cpu)
{
- if (!cpumask_intersects(cpumask, cpu_online_mask)) {
+ cpumask_copy(pinst->validate_cpumask, cpu_online_mask);
+
+ /*
+ * @offlining_cpu is still in cpu_online_mask, so remove it here for
+ * validation. Using a sub-CPUHP_TEARDOWN_CPU hotplug state where
+ * @offlining_cpu wouldn't be in the online mask doesn't work because
+ * padata_cpu_offline() can fail but such a state doesn't allow failure.
+ */
+ if (offlining_cpu >= 0)
+ __cpumask_clear_cpu(offlining_cpu, pinst->validate_cpumask);
+
+ if (!cpumask_intersects(cpumask, pinst->validate_cpumask)) {
pinst->flags |= PADATA_INVALID;
return false;
}
@@ -687,13 +704,13 @@ static int __padata_set_cpumasks(struct padata_instance *pinst,
int valid;
int err;
- valid = padata_validate_cpumask(pinst, pcpumask);
+ valid = padata_validate_cpumask(pinst, pcpumask, -1);
if (!valid) {
__padata_stop(pinst);
goto out_replace;
}
- valid = padata_validate_cpumask(pinst, cbcpumask);
+ valid = padata_validate_cpumask(pinst, cbcpumask, -1);
if (!valid)
__padata_stop(pinst);
@@ -701,7 +718,7 @@ static int __padata_set_cpumasks(struct padata_instance *pinst,
cpumask_copy(pinst->cpumask.pcpu, pcpumask);
cpumask_copy(pinst->cpumask.cbcpu, cbcpumask);
- err = padata_setup_cpumasks(pinst) ?: padata_replace(pinst);
+ err = padata_setup_cpumasks(pinst) ?: padata_replace(pinst, -1);
if (valid)
__padata_start(pinst);
@@ -753,36 +770,6 @@ EXPORT_SYMBOL(padata_set_cpumask);
#ifdef CONFIG_HOTPLUG_CPU
-static int __padata_add_cpu(struct padata_instance *pinst, int cpu)
-{
- int err = 0;
-
- if (cpumask_test_cpu(cpu, cpu_online_mask)) {
- err = padata_replace(pinst);
-
- if (padata_validate_cpumask(pinst, pinst->cpumask.pcpu) &&
- padata_validate_cpumask(pinst, pinst->cpumask.cbcpu))
- __padata_start(pinst);
- }
-
- return err;
-}
-
-static int __padata_remove_cpu(struct padata_instance *pinst, int cpu)
-{
- int err = 0;
-
- if (!cpumask_test_cpu(cpu, cpu_online_mask)) {
- if (!padata_validate_cpumask(pinst, pinst->cpumask.pcpu) ||
- !padata_validate_cpumask(pinst, pinst->cpumask.cbcpu))
- __padata_stop(pinst);
-
- err = padata_replace(pinst);
- }
-
- return err;
-}
-
static inline int pinst_has_cpu(struct padata_instance *pinst, int cpu)
{
return cpumask_test_cpu(cpu, pinst->cpumask.pcpu) ||
@@ -794,27 +781,39 @@ static int padata_cpu_online(unsigned int cpu, struct hlist_node *node)
struct padata_instance *pinst;
int ret;
- pinst = hlist_entry_safe(node, struct padata_instance, cpu_online_node);
+ pinst = hlist_entry_safe(node, struct padata_instance, cpuhp_node);
if (!pinst_has_cpu(pinst, cpu))
return 0;
mutex_lock(&pinst->lock);
- ret = __padata_add_cpu(pinst, cpu);
+
+ ret = padata_replace(pinst, -1);
+
+ if (padata_validate_cpumask(pinst, pinst->cpumask.pcpu, -1) &&
+ padata_validate_cpumask(pinst, pinst->cpumask.cbcpu, -1))
+ __padata_start(pinst);
+
mutex_unlock(&pinst->lock);
return ret;
}
-static int padata_cpu_dead(unsigned int cpu, struct hlist_node *node)
+static int padata_cpu_offline(unsigned int cpu, struct hlist_node *node)
{
struct padata_instance *pinst;
int ret;
- pinst = hlist_entry_safe(node, struct padata_instance, cpu_dead_node);
+ pinst = hlist_entry_safe(node, struct padata_instance, cpuhp_node);
if (!pinst_has_cpu(pinst, cpu))
return 0;
mutex_lock(&pinst->lock);
- ret = __padata_remove_cpu(pinst, cpu);
+
+ if (!padata_validate_cpumask(pinst, pinst->cpumask.pcpu, cpu) ||
+ !padata_validate_cpumask(pinst, pinst->cpumask.cbcpu, cpu))
+ __padata_stop(pinst);
+
+ ret = padata_replace(pinst, cpu);
+
mutex_unlock(&pinst->lock);
return ret;
}
@@ -825,15 +824,14 @@ static enum cpuhp_state hp_online;
static void __padata_free(struct padata_instance *pinst)
{
#ifdef CONFIG_HOTPLUG_CPU
- cpuhp_state_remove_instance_nocalls(CPUHP_PADATA_DEAD,
- &pinst->cpu_dead_node);
- cpuhp_state_remove_instance_nocalls(hp_online, &pinst->cpu_online_node);
+ cpuhp_state_remove_instance_nocalls(hp_online, &pinst->cpuhp_node);
#endif
WARN_ON(!list_empty(&pinst->pslist));
free_cpumask_var(pinst->cpumask.pcpu);
free_cpumask_var(pinst->cpumask.cbcpu);
+ free_cpumask_var(pinst->validate_cpumask);
destroy_workqueue(pinst->serial_wq);
destroy_workqueue(pinst->parallel_wq);
kfree(pinst);
@@ -993,10 +991,10 @@ struct padata_instance *padata_alloc(const char *name)
if (!alloc_cpumask_var(&pinst->cpumask.pcpu, GFP_KERNEL))
goto err_free_serial_wq;
- if (!alloc_cpumask_var(&pinst->cpumask.cbcpu, GFP_KERNEL)) {
- free_cpumask_var(pinst->cpumask.pcpu);
- goto err_free_serial_wq;
- }
+ if (!alloc_cpumask_var(&pinst->cpumask.cbcpu, GFP_KERNEL))
+ goto err_free_p_mask;
+ if (!alloc_cpumask_var(&pinst->validate_cpumask, GFP_KERNEL))
+ goto err_free_cb_mask;
INIT_LIST_HEAD(&pinst->pslist);
@@ -1004,7 +1002,7 @@ struct padata_instance *padata_alloc(const char *name)
cpumask_copy(pinst->cpumask.cbcpu, cpu_possible_mask);
if (padata_setup_cpumasks(pinst))
- goto err_free_masks;
+ goto err_free_v_mask;
__padata_start(pinst);
@@ -1013,18 +1011,19 @@ struct padata_instance *padata_alloc(const char *name)
#ifdef CONFIG_HOTPLUG_CPU
cpuhp_state_add_instance_nocalls_cpuslocked(hp_online,
- &pinst->cpu_online_node);
- cpuhp_state_add_instance_nocalls_cpuslocked(CPUHP_PADATA_DEAD,
- &pinst->cpu_dead_node);
+ &pinst->cpuhp_node);
#endif
cpus_read_unlock();
return pinst;
-err_free_masks:
- free_cpumask_var(pinst->cpumask.pcpu);
+err_free_v_mask:
+ free_cpumask_var(pinst->validate_cpumask);
+err_free_cb_mask:
free_cpumask_var(pinst->cpumask.cbcpu);
+err_free_p_mask:
+ free_cpumask_var(pinst->cpumask.pcpu);
err_free_serial_wq:
destroy_workqueue(pinst->serial_wq);
err_put_cpus:
@@ -1067,7 +1066,7 @@ struct padata_shell *padata_alloc_shell(struct padata_instance *pinst)
ps->pinst = pinst;
cpus_read_lock();
- pd = padata_alloc_pd(ps);
+ pd = padata_alloc_pd(ps, -1);
cpus_read_unlock();
if (!pd)
@@ -1116,32 +1115,25 @@ void __init padata_init(void)
int ret;
ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "padata:online",
- padata_cpu_online, NULL);
+ padata_cpu_online, padata_cpu_offline);
if (ret < 0)
goto err;
hp_online = ret;
-
- ret = cpuhp_setup_state_multi(CPUHP_PADATA_DEAD, "padata:dead",
- NULL, padata_cpu_dead);
- if (ret < 0)
- goto remove_online_state;
#endif
possible_cpus = num_possible_cpus();
padata_works = kmalloc_array(possible_cpus, sizeof(struct padata_work),
GFP_KERNEL);
if (!padata_works)
- goto remove_dead_state;
+ goto remove_online_state;
for (i = 0; i < possible_cpus; ++i)
list_add(&padata_works[i].pw_list, &padata_free_works);
return;
-remove_dead_state:
-#ifdef CONFIG_HOTPLUG_CPU
- cpuhp_remove_multi_state(CPUHP_PADATA_DEAD);
remove_online_state:
+#ifdef CONFIG_HOTPLUG_CPU
cpuhp_remove_multi_state(hp_online);
err:
#endif
diff --git a/kernel/params.c b/kernel/params.c
index 9935ff599356..4495038fc2e7 100644
--- a/kernel/params.c
+++ b/kernel/params.c
@@ -595,12 +595,6 @@ static ssize_t param_attr_store(struct module_attribute *mattr,
}
#endif
-#ifdef CONFIG_MODULES
-#define __modinit
-#else
-#define __modinit __init
-#endif
-
#ifdef CONFIG_SYSFS
void kernel_param_lock(struct module *mod)
{
@@ -625,9 +619,9 @@ EXPORT_SYMBOL(kernel_param_unlock);
* create file in sysfs. Returns an error on out of memory. Always cleans up
* if there's an error.
*/
-static __modinit int add_sysfs_param(struct module_kobject *mk,
- const struct kernel_param *kp,
- const char *name)
+static __init_or_module int add_sysfs_param(struct module_kobject *mk,
+ const struct kernel_param *kp,
+ const char *name)
{
struct module_param_attrs *new_mp;
struct attribute **new_attrs;
@@ -754,16 +748,8 @@ void module_param_sysfs_remove(struct module *mod)
}
#endif
-void destroy_params(const struct kernel_param *params, unsigned num)
-{
- unsigned int i;
-
- for (i = 0; i < num; i++)
- if (params[i].ops->free)
- params[i].ops->free(params[i].arg);
-}
-
-struct module_kobject __modinit * lookup_or_create_module_kobject(const char *name)
+struct module_kobject * __init_or_module
+lookup_or_create_module_kobject(const char *name)
{
struct module_kobject *mk;
struct kobject *kobj;
@@ -996,3 +982,21 @@ static int __init param_sysfs_builtin_init(void)
late_initcall(param_sysfs_builtin_init);
#endif /* CONFIG_SYSFS */
+
+#ifdef CONFIG_MODULES
+
+/*
+ * module_destroy_params - free all parameters for one module
+ * @params: module parameters (array)
+ * @num: number of module parameters
+ */
+void module_destroy_params(const struct kernel_param *params, unsigned int num)
+{
+ unsigned int i;
+
+ for (i = 0; i < num; i++)
+ if (params[i].ops->free)
+ params[i].ops->free(params[i].arg);
+}
+
+#endif /* CONFIG_MODULES */
diff --git a/kernel/pid.c b/kernel/pid.c
index b80c3bfb58d0..c5650ea80a2b 100644
--- a/kernel/pid.c
+++ b/kernel/pid.c
@@ -536,11 +536,10 @@ EXPORT_SYMBOL_GPL(find_ge_pid);
struct pid *pidfd_get_pid(unsigned int fd, unsigned int *flags)
{
- struct fd f;
+ CLASS(fd, f)(fd);
struct pid *pid;
- f = fdget(fd);
- if (!fd_file(f))
+ if (fd_empty(f))
return ERR_PTR(-EBADF);
pid = pidfd_pid(fd_file(f));
@@ -548,8 +547,6 @@ struct pid *pidfd_get_pid(unsigned int fd, unsigned int *flags)
get_pid(pid);
*flags = fd_file(f)->f_flags;
}
-
- fdput(f);
return pid;
}
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index df76b32a013f..9b238c9c71c6 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4453,6 +4453,7 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p)
p->se.nr_migrations = 0;
p->se.vruntime = 0;
p->se.vlag = 0;
+ p->se.rel_deadline = 0;
INIT_LIST_HEAD(&p->se.group_node);
/* A delayed task cannot be in clone(). */
diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 25df16aed142..f24579675cf3 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -6346,6 +6346,14 @@ static bool scx_dispatch_from_dsq(struct bpf_iter_scx_dsq_kern *kit,
bool in_balance;
unsigned long flags;
+ /*
+ * The verifier considers an iterator slot initialized on any
+ * KF_ITER_NEW return, so a BPF program may legally reach here after
+ * bpf_iter_scx_dsq_new() failed and left @kit->dsq NULL.
+ */
+ if (unlikely(!src_dsq))
+ return false;
+
if (!scx_kf_allowed_if_unlocked() && !scx_kf_allowed(SCX_KF_DISPATCH))
return false;
diff --git a/kernel/sys.c b/kernel/sys.c
index 35990f0796bc..8283e35c9eeb 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -1929,12 +1929,11 @@ SYSCALL_DEFINE1(umask, int, mask)
static int prctl_set_mm_exe_file(struct mm_struct *mm, unsigned int fd)
{
- struct fd exe;
+ CLASS(fd, exe)(fd);
struct inode *inode;
int err;
- exe = fdget(fd);
- if (!fd_file(exe))
+ if (fd_empty(exe))
return -EBADF;
inode = file_inode(fd_file(exe));
@@ -1944,18 +1943,14 @@ static int prctl_set_mm_exe_file(struct mm_struct *mm, unsigned int fd)
* sure that this one is executable as well, to avoid breaking an
* overall picture.
*/
- err = -EACCES;
if (!S_ISREG(inode->i_mode) || path_noexec(&fd_file(exe)->f_path))
- goto exit;
+ return -EACCES;
err = file_permission(fd_file(exe), MAY_EXEC);
if (err)
- goto exit;
+ return err;
- err = replace_mm_exe_file(mm, fd_file(exe));
-exit:
- fdput(exe);
- return err;
+ return replace_mm_exe_file(mm, fd_file(exe));
}
/*
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 640d2ea4bd1f..5d1b880de171 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -489,17 +489,10 @@ debug_init(struct hrtimer *timer, clockid_t clockid,
trace_hrtimer_init(timer, clockid, mode);
}
-static inline void debug_activate(struct hrtimer *timer,
- enum hrtimer_mode mode)
+static inline void debug_activate(struct hrtimer *timer, enum hrtimer_mode mode, bool was_armed)
{
debug_hrtimer_activate(timer, mode);
- trace_hrtimer_start(timer, mode);
-}
-
-static inline void debug_deactivate(struct hrtimer *timer)
-{
- debug_hrtimer_deactivate(timer);
- trace_hrtimer_cancel(timer);
+ trace_hrtimer_start(timer, mode, was_armed);
}
static struct hrtimer_clock_base *
@@ -1091,13 +1084,12 @@ EXPORT_SYMBOL_GPL(hrtimer_forward);
* The timer is inserted in expiry order. Insertion into the
* red black tree is O(log(n)). Must hold the base lock.
*
- * Returns 1 when the new timer is the leftmost timer in the tree.
+ * Returns true when the new timer is the leftmost timer in the tree.
*/
-static int enqueue_hrtimer(struct hrtimer *timer,
- struct hrtimer_clock_base *base,
- enum hrtimer_mode mode)
+static bool enqueue_hrtimer(struct hrtimer *timer, struct hrtimer_clock_base *base,
+ enum hrtimer_mode mode, bool was_armed)
{
- debug_activate(timer, mode);
+ debug_activate(timer, mode, was_armed);
WARN_ON_ONCE(!base->cpu_base->online);
base->cpu_base->active_bases |= 1 << base->index;
@@ -1157,6 +1149,8 @@ remove_hrtimer(struct hrtimer *timer, struct hrtimer_clock_base *base,
if (state & HRTIMER_STATE_ENQUEUED) {
bool reprogram;
+ debug_hrtimer_deactivate(timer);
+
/*
* Remove the timer and force reprogramming when high
* resolution mode is active and the timer is on the current
@@ -1165,7 +1159,6 @@ remove_hrtimer(struct hrtimer *timer, struct hrtimer_clock_base *base,
* reprogramming happens in the interrupt handler. This is a
* rare case and less expensive than a smp call.
*/
- debug_deactivate(timer);
reprogram = base->cpu_base == this_cpu_ptr(&hrtimer_bases);
/*
@@ -1232,15 +1225,15 @@ static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
{
struct hrtimer_cpu_base *this_cpu_base = this_cpu_ptr(&hrtimer_bases);
struct hrtimer_clock_base *new_base;
- bool force_local, first;
+ bool force_local, first, was_armed;
/*
* If the timer is on the local cpu base and is the first expiring
* timer then this might end up reprogramming the hardware twice
- * (on removal and on enqueue). To avoid that by prevent the
- * reprogram on removal, keep the timer local to the current CPU
- * and enforce reprogramming after it is queued no matter whether
- * it is the new first expiring timer again or not.
+ * (on removal and on enqueue). To avoid that prevent the reprogram
+ * on removal, keep the timer local to the current CPU and enforce
+ * reprogramming after it is queued no matter whether it is the new
+ * first expiring timer again or not.
*/
force_local = base->cpu_base == this_cpu_base;
force_local &= base->cpu_base->next_timer == timer;
@@ -1262,7 +1255,7 @@ static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
* avoids programming the underlying clock event twice (once at
* removal and once after enqueue).
*/
- remove_hrtimer(timer, base, true, force_local);
+ was_armed = remove_hrtimer(timer, base, true, force_local);
if (mode & HRTIMER_MODE_REL)
tim = ktime_add_safe(tim, base->get_time());
@@ -1279,7 +1272,15 @@ static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim,
new_base = base;
}
- first = enqueue_hrtimer(timer, new_base, mode);
+ first = enqueue_hrtimer(timer, new_base, mode, was_armed);
+
+ /*
+ * If the hrtimer interrupt is running, then it will reevaluate the
+ * clock bases and reprogram the clock event device.
+ */
+ if (new_base->cpu_base->in_hrtirq)
+ return false;
+
if (!force_local) {
/*
* If the current CPU base is online, then the timer is
@@ -1375,8 +1376,11 @@ int hrtimer_try_to_cancel(struct hrtimer *timer)
base = lock_hrtimer_base(timer, &flags);
- if (!hrtimer_callback_running(timer))
+ if (!hrtimer_callback_running(timer)) {
ret = remove_hrtimer(timer, base, false, false);
+ if (ret)
+ trace_hrtimer_cancel(timer);
+ }
unlock_hrtimer_base(timer, &flags);
@@ -1763,7 +1767,7 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,
*/
if (restart != HRTIMER_NORESTART &&
!(timer->state & HRTIMER_STATE_ENQUEUED))
- enqueue_hrtimer(timer, base, HRTIMER_MODE_ABS);
+ enqueue_hrtimer(timer, base, HRTIMER_MODE_ABS, false);
/*
* Separate the ->running assignment from the ->state assignment.
@@ -2245,7 +2249,7 @@ static void migrate_hrtimer_list(struct hrtimer_clock_base *old_base,
while ((node = timerqueue_getnext(&old_base->active))) {
timer = container_of(node, struct hrtimer, node);
BUG_ON(hrtimer_callback_running(timer));
- debug_deactivate(timer);
+ debug_hrtimer_deactivate(timer);
/*
* Mark it as ENQUEUED not INACTIVE otherwise the
@@ -2262,7 +2266,7 @@ static void migrate_hrtimer_list(struct hrtimer_clock_base *old_base,
* sort out already expired timers and reprogram the
* event device.
*/
- enqueue_hrtimer(timer, new_base, HRTIMER_MODE_ABS);
+ enqueue_hrtimer(timer, new_base, HRTIMER_MODE_ABS, true);
}
}
diff --git a/kernel/trace/trace_branch.c b/kernel/trace/trace_branch.c
index e47fdb4c92fb..30f72e0ecb5d 100644
--- a/kernel/trace/trace_branch.c
+++ b/kernel/trace/trace_branch.c
@@ -379,10 +379,10 @@ __init static int init_annotated_branch_stats(void)
int ret;
ret = register_stat_tracer(&annotated_branch_stats);
- if (!ret) {
+ if (ret) {
printk(KERN_WARNING "Warning: could not register "
"annotated branches stats\n");
- return 1;
+ return ret;
}
return 0;
}
@@ -444,10 +444,10 @@ __init static int all_annotated_branch_stats(void)
int ret;
ret = register_stat_tracer(&all_branch_stats);
- if (!ret) {
+ if (ret) {
printk(KERN_WARNING "Warning: could not register "
"all branches stats\n");
- return 1;
+ return ret;
}
return 0;
}
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index 83de1a196a4a..2d085115afde 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -1342,12 +1342,14 @@ static const char *hist_field_name(struct hist_field *field,
field->flags & HIST_FIELD_FL_VAR_REF) {
if (field->system) {
static char full_name[MAX_FILTER_STR_VAL];
+ int len;
+
+ len = snprintf(full_name, sizeof(full_name), "%s.%s.%s",
+ field->system, field->event_name,
+ field->name);
+ if (len >= sizeof(full_name))
+ return NULL;
- strcat(full_name, field->system);
- strcat(full_name, ".");
- strcat(full_name, field->event_name);
- strcat(full_name, ".");
- strcat(full_name, field->name);
field_name = full_name;
} else
field_name = field->name;
diff --git a/kernel/watch_queue.c b/kernel/watch_queue.c
index e55f9810b91a..6d1936fb8ff0 100644
--- a/kernel/watch_queue.c
+++ b/kernel/watch_queue.c
@@ -672,16 +672,14 @@ struct watch_queue *get_watch_queue(int fd)
{
struct pipe_inode_info *pipe;
struct watch_queue *wqueue = ERR_PTR(-EINVAL);
- struct fd f;
+ CLASS(fd, f)(fd);
- f = fdget(fd);
- if (fd_file(f)) {
+ if (!fd_empty(f)) {
pipe = get_pipe_info(fd_file(f), false);
if (pipe && pipe->watch_queue) {
wqueue = pipe->watch_queue;
kref_get(&wqueue->usage);
}
- fdput(f);
}
return wqueue;
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index e6954e9409f9..2fdf9174ccec 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -5568,7 +5568,9 @@ static int alloc_and_link_pwqs(struct workqueue_struct *wq)
ret = apply_workqueue_attrs_locked(wq, unbound_std_wq_attrs[highpri]);
}
- return ret;
+ if (ret)
+ goto enomem;
+ return 0;
enomem:
if (wq->cpu_pwq) {
diff --git a/lib/net_utils.c b/lib/net_utils.c
index 42bb0473fb22..215cda672fee 100644
--- a/lib/net_utils.c
+++ b/lib/net_utils.c
@@ -7,11 +7,9 @@
bool mac_pton(const char *s, u8 *mac)
{
- size_t maxlen = 3 * ETH_ALEN - 1;
int i;
- /* XX:XX:XX:XX:XX:XX */
- if (strnlen(s, maxlen) < maxlen)
+ if (strnlen(s, MAC_ADDR_STR_LEN) < MAC_ADDR_STR_LEN)
return false;
/* Don't dirty result unless string is valid MAC. */
diff --git a/mm/fadvise.c b/mm/fadvise.c
index 532dee205c6e..588fe76c5a14 100644
--- a/mm/fadvise.c
+++ b/mm/fadvise.c
@@ -190,16 +190,12 @@ EXPORT_SYMBOL(vfs_fadvise);
int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice)
{
- struct fd f = fdget(fd);
- int ret;
+ CLASS(fd, f)(fd);
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
- ret = vfs_fadvise(fd_file(f), offset, len, advice);
-
- fdput(f);
- return ret;
+ return vfs_fadvise(fd_file(f), offset, len, advice);
}
SYSCALL_DEFINE4(fadvise64_64, int, fd, loff_t, offset, loff_t, len, int, advice)
diff --git a/mm/readahead.c b/mm/readahead.c
index bf79275060f3..32f57d0db9ac 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -676,29 +676,22 @@ EXPORT_SYMBOL_GPL(page_cache_async_ra);
ssize_t ksys_readahead(int fd, loff_t offset, size_t count)
{
- ssize_t ret;
- struct fd f;
+ CLASS(fd, f)(fd);
- ret = -EBADF;
- f = fdget(fd);
- if (!fd_file(f) || !(fd_file(f)->f_mode & FMODE_READ))
- goto out;
+ if (fd_empty(f) || !(fd_file(f)->f_mode & FMODE_READ))
+ return -EBADF;
/*
* The readahead() syscall is intended to run only on files
* that can execute readahead. If readahead is not possible
* on this file, then we must return -EINVAL.
*/
- ret = -EINVAL;
if (!fd_file(f)->f_mapping || !fd_file(f)->f_mapping->a_ops ||
(!S_ISREG(file_inode(fd_file(f))->i_mode) &&
!S_ISBLK(file_inode(fd_file(f))->i_mode)))
- goto out;
+ return -EINVAL;
- ret = vfs_fadvise(fd_file(f), offset, count, POSIX_FADV_WILLNEED);
-out:
- fdput(f);
- return ret;
+ return vfs_fadvise(fd_file(f), offset, count, POSIX_FADV_WILLNEED);
}
SYSCALL_DEFINE3(readahead, int, fd, loff_t, offset, size_t, count)
diff --git a/net/bluetooth/hci_event.c b/net/bluetooth/hci_event.c
index 8a14c00ad727..92270b99ee0f 100644
--- a/net/bluetooth/hci_event.c
+++ b/net/bluetooth/hci_event.c
@@ -3284,8 +3284,6 @@ static void hci_conn_request_evt(struct hci_dev *hdev, void *data,
memcpy(conn->dev_class, ev->dev_class, 3);
- hci_dev_unlock(hdev);
-
if (ev->link_type == ACL_LINK ||
(!(flags & HCI_PROTO_DEFER) && !lmp_esco_capable(hdev))) {
struct hci_cp_accept_conn_req cp;
@@ -3319,7 +3317,6 @@ static void hci_conn_request_evt(struct hci_dev *hdev, void *data,
hci_connect_cfm(conn, 0);
}
- return;
unlock:
hci_dev_unlock(hdev);
}
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index 307f7fe975b5..b01107370cbc 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -5399,7 +5399,13 @@ static inline int l2cap_ecred_reconf_rsp(struct l2cap_conn *conn,
if (chan->ident != cmd->ident)
continue;
+ l2cap_chan_hold(chan);
+ l2cap_chan_lock(chan);
+
l2cap_chan_del(chan, ECONNRESET);
+
+ l2cap_chan_unlock(chan);
+ l2cap_chan_put(chan);
}
return 0;
@@ -6657,7 +6663,7 @@ static int l2cap_ecred_data_rcv(struct l2cap_chan *chan, struct sk_buff *skb)
if (sdu_len > chan->imtu) {
BT_ERR("Too big LE L2CAP SDU length: len %u > %u",
- skb->len, sdu_len);
+ sdu_len, chan->imtu);
l2cap_send_disconn_req(chan, ECONNRESET);
err = -EMSGSIZE;
goto failed;
diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
index ded0c52ccf0b..d915db52db22 100644
--- a/net/bluetooth/sco.c
+++ b/net/bluetooth/sco.c
@@ -978,7 +978,8 @@ static int sco_sock_setsockopt(struct socket *sock, int level, int optname,
codecs = (void *)buffer;
- if (codecs->num_codecs > 1) {
+ if (codecs->num_codecs != 1 ||
+ optlen < struct_size(codecs, codecs, codecs->num_codecs)) {
hci_dev_put(hdev);
err = -EINVAL;
break;
diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
index c885a3942a16..7f12abc8a80c 100644
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -1061,19 +1061,23 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
switch (skb->protocol) {
case htons(ETH_P_IP):
- sk->sk_family = AF_INET;
- if (sizeof(struct iphdr) <= skb_headlen(skb)) {
- sk->sk_rcv_saddr = ip_hdr(skb)->saddr;
- sk->sk_daddr = ip_hdr(skb)->daddr;
+ if (skb_headlen(skb) < sizeof(struct iphdr)) {
+ ret = -EINVAL;
+ goto out;
}
+ sk->sk_family = AF_INET;
+ sk->sk_rcv_saddr = ip_hdr(skb)->saddr;
+ sk->sk_daddr = ip_hdr(skb)->daddr;
break;
#if IS_ENABLED(CONFIG_IPV6)
case htons(ETH_P_IPV6):
- sk->sk_family = AF_INET6;
- if (sizeof(struct ipv6hdr) <= skb_headlen(skb)) {
- sk->sk_v6_rcv_saddr = ipv6_hdr(skb)->saddr;
- sk->sk_v6_daddr = ipv6_hdr(skb)->daddr;
+ if (skb_headlen(skb) < sizeof(struct ipv6hdr)) {
+ ret = -EINVAL;
+ goto out;
}
+ sk->sk_family = AF_INET6;
+ sk->sk_v6_rcv_saddr = ipv6_hdr(skb)->saddr;
+ sk->sk_v6_daddr = ipv6_hdr(skb)->daddr;
break;
#endif
default:
@@ -1097,6 +1101,21 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
skb->ip_summed = CHECKSUM_COMPLETE;
}
+ if (prog->type == BPF_PROG_TYPE_LWT_XMIT) {
+ if (!ipv6_bpf_stub) {
+ pr_warn_once("Please test this program with the IPv6 module loaded\n");
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+#if IS_ENABLED(CONFIG_IPV6)
+ /* For CONFIG_IPV6=n, ipv6_bpf_stub is NULL which is
+ * handled by the above if statement.
+ */
+ dst_hold(&net->ipv6.ip6_null_entry->dst);
+ skb_dst_set(skb, &net->ipv6.ip6_null_entry->dst);
+#endif
+ }
+
ret = bpf_test_run(prog, skb, repeat, &retval, &duration, false);
if (ret)
goto out;
diff --git a/net/ceph/crush/crush.c b/net/ceph/crush/crush.c
index 254ded0b05f6..521aec1d5fc0 100644
--- a/net/ceph/crush/crush.c
+++ b/net/ceph/crush/crush.c
@@ -47,7 +47,6 @@ int crush_get_bucket_item_weight(const struct crush_bucket *b, int p)
void crush_destroy_bucket_uniform(struct crush_bucket_uniform *b)
{
kfree(b->h.items);
- kfree(b);
}
void crush_destroy_bucket_list(struct crush_bucket_list *b)
@@ -55,14 +54,12 @@ void crush_destroy_bucket_list(struct crush_bucket_list *b)
kfree(b->item_weights);
kfree(b->sum_weights);
kfree(b->h.items);
- kfree(b);
}
void crush_destroy_bucket_tree(struct crush_bucket_tree *b)
{
kfree(b->h.items);
kfree(b->node_weights);
- kfree(b);
}
void crush_destroy_bucket_straw(struct crush_bucket_straw *b)
@@ -70,14 +67,12 @@ void crush_destroy_bucket_straw(struct crush_bucket_straw *b)
kfree(b->straws);
kfree(b->item_weights);
kfree(b->h.items);
- kfree(b);
}
void crush_destroy_bucket_straw2(struct crush_bucket_straw2 *b)
{
kfree(b->item_weights);
kfree(b->h.items);
- kfree(b);
}
void crush_destroy_bucket(struct crush_bucket *b)
@@ -99,6 +94,7 @@ void crush_destroy_bucket(struct crush_bucket *b)
crush_destroy_bucket_straw2((struct crush_bucket_straw2 *)b);
break;
}
+ kfree(b);
}
/**
diff --git a/net/ceph/osdmap.c b/net/ceph/osdmap.c
index 7c76eb9d6cee..c34a5bf86831 100644
--- a/net/ceph/osdmap.c
+++ b/net/ceph/osdmap.c
@@ -390,11 +390,15 @@ static int decode_choose_args(void **p, void *end, struct crush_map *c)
goto fail;
if (arg->ids_size &&
- arg->ids_size != c->buckets[bucket_index]->size)
+ (!c->buckets[bucket_index] ||
+ arg->ids_size != c->buckets[bucket_index]->size))
goto e_inval;
}
- insert_choose_arg_map(&c->choose_args, arg_map);
+ if (!__insert_choose_arg_map(&c->choose_args, arg_map)) {
+ ret = -EEXIST;
+ goto fail;
+ }
}
return 0;
@@ -517,6 +521,10 @@ static struct crush_map *crush_decode(void *pbyval, void *end)
b->id = ceph_decode_32(p);
b->type = ceph_decode_16(p);
b->alg = ceph_decode_8(p);
+ if (b->alg != alg) {
+ b->alg = 0;
+ goto bad;
+ }
b->hash = ceph_decode_8(p);
b->weight = ceph_decode_32(p);
b->size = ceph_decode_32(p);
@@ -1703,7 +1711,7 @@ static int osdmap_decode(void **p, void *end, bool msgr2,
ceph_decode_need(p, end, 3*sizeof(u32) +
map->max_osd*(struct_v >= 5 ? sizeof(u32) :
sizeof(u8)) +
- sizeof(*map->osd_weight), e_inval);
+ map->max_osd*sizeof(*map->osd_weight), e_inval);
if (ceph_decode_32(p) != map->max_osd)
goto e_inval;
diff --git a/net/core/devmem.c b/net/core/devmem.c
index 17f8a83a5ee7..55f27a9ed121 100644
--- a/net/core/devmem.c
+++ b/net/core/devmem.c
@@ -15,6 +15,7 @@
#include <net/netdev_queues.h>
#include <net/netdev_rx_queue.h>
#include <net/page_pool/helpers.h>
+#include <net/page_pool/memory_provider.h>
#include <trace/events/page_pool.h>
#include "devmem.h"
@@ -26,6 +27,8 @@
/* Protected by rtnl_lock() */
static DEFINE_XARRAY_FLAGS(net_devmem_dmabuf_bindings, XA_FLAGS_ALLOC1);
+static const struct memory_provider_ops dmabuf_devmem_ops;
+
static void net_devmem_dmabuf_free_chunk_owner(struct gen_pool *genpool,
struct gen_pool_chunk *chunk,
void *not_used)
@@ -117,6 +120,7 @@ void net_devmem_unbind_dmabuf(struct net_devmem_dmabuf_binding *binding)
WARN_ON(rxq->mp_params.mp_priv != binding);
rxq->mp_params.mp_priv = NULL;
+ rxq->mp_params.mp_ops = NULL;
rxq_idx = get_netdev_rx_queue_index(rxq);
@@ -143,7 +147,7 @@ int net_devmem_bind_dmabuf_to_queue(struct net_device *dev, u32 rxq_idx,
}
rxq = __netif_get_rx_queue(dev, rxq_idx);
- if (rxq->mp_params.mp_priv) {
+ if (rxq->mp_params.mp_ops) {
NL_SET_ERR_MSG(extack, "designated queue already memory provider bound");
return -EEXIST;
}
@@ -161,6 +165,7 @@ int net_devmem_bind_dmabuf_to_queue(struct net_device *dev, u32 rxq_idx,
return err;
rxq->mp_params.mp_priv = binding;
+ rxq->mp_params.mp_ops = &dmabuf_devmem_ops;
err = netdev_rx_queue_restart(dev, rxq_idx);
if (err)
@@ -170,6 +175,7 @@ int net_devmem_bind_dmabuf_to_queue(struct net_device *dev, u32 rxq_idx,
err_xa_erase:
rxq->mp_params.mp_priv = NULL;
+ rxq->mp_params.mp_ops = NULL;
xa_erase(&binding->bound_rxqs, xa_idx);
return err;
@@ -333,11 +339,10 @@ int mp_dmabuf_devmem_init(struct page_pool *pool)
if (!binding)
return -EINVAL;
- if (!pool->dma_map)
- return -EOPNOTSUPP;
-
- if (pool->dma_sync)
- return -EOPNOTSUPP;
+ /* dma-buf dma addresses do not need and should not be used with
+ * dma_sync_for_cpu/device. Force disable dma_sync.
+ */
+ pool->dma_sync = false;
if (pool->p.order != 0)
return -E2BIG;
@@ -389,3 +394,10 @@ bool mp_dmabuf_devmem_release_page(struct page_pool *pool, netmem_ref netmem)
/* We don't want the page pool put_page()ing our net_iovs. */
return false;
}
+
+static const struct memory_provider_ops dmabuf_devmem_ops = {
+ .init = mp_dmabuf_devmem_init,
+ .destroy = mp_dmabuf_devmem_destroy,
+ .alloc_netmems = mp_dmabuf_devmem_alloc_netmems,
+ .release_netmem = mp_dmabuf_devmem_release_page,
+};
diff --git a/net/core/filter.c b/net/core/filter.c
index 1f96c3aa01ca..1d7467601a32 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -499,7 +499,7 @@ static bool convert_bpf_ld_abs(struct sock_filter *fp, struct bpf_insn **insnp)
((unaligned_ok && offset >= 0) ||
(!unaligned_ok && offset >= 0 &&
offset + ip_align >= 0 &&
- offset + ip_align % size == 0))) {
+ (offset + ip_align) % size == 0))) {
bool ldx_off_ok = offset <= S16_MAX;
*insn++ = BPF_MOV64_REG(BPF_REG_TMP, BPF_REG_H);
@@ -4388,6 +4388,8 @@ u32 xdp_master_redirect(struct xdp_buff *xdp)
struct net_device *master, *slave;
master = netdev_master_upper_dev_get_rcu(xdp->rxq->dev);
+ if (unlikely(!(master->flags & IFF_UP)))
+ return XDP_ABORTED;
slave = master->netdev_ops->ndo_xdp_get_xmit_slave(master, xdp);
if (slave && slave != xdp->rxq->dev) {
/* The target device is different from the receiving device, so
diff --git a/net/core/gro.c b/net/core/gro.c
index ac498c9f82cf..f5c80c2f69df 100644
--- a/net/core/gro.c
+++ b/net/core/gro.c
@@ -214,10 +214,12 @@ int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb)
p->data_len += len;
p->truesize += delta_truesize;
p->len += len;
+ skb_shinfo(p)->flags |= skbinfo->flags & SKBFL_SHARED_FRAG;
if (lp != p) {
lp->data_len += len;
lp->truesize += delta_truesize;
lp->len += len;
+ skb_shinfo(lp)->flags |= skbinfo->flags & SKBFL_SHARED_FRAG;
}
NAPI_GRO_CB(skb)->same_flow = 1;
return 0;
@@ -245,6 +247,8 @@ int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb)
p->truesize += skb->truesize;
p->len += skb->len;
+ skb_shinfo(p)->flags |= skb_shinfo(skb)->flags & SKBFL_SHARED_FRAG;
+
NAPI_GRO_CB(skb)->same_flow = 1;
return 0;
diff --git a/net/core/neighbour.c b/net/core/neighbour.c
index 96786016dbb4..bf07438d6dfa 100644
--- a/net/core/neighbour.c
+++ b/net/core/neighbour.c
@@ -3161,8 +3161,10 @@ int neigh_xmit(int index, struct net_device *dev,
rcu_read_lock();
tbl = rcu_dereference(neigh_tables[index]);
- if (!tbl)
- goto out_unlock;
+ if (!tbl) {
+ rcu_read_unlock();
+ goto out_kfree_skb;
+ }
if (index == NEIGH_ARP_TABLE) {
u32 key = *((u32 *)addr);
@@ -3178,7 +3180,6 @@ int neigh_xmit(int index, struct net_device *dev,
goto out_kfree_skb;
}
err = READ_ONCE(neigh->output)(neigh, skb);
-out_unlock:
rcu_read_unlock();
}
else if (index == NEIGH_LINK_TABLE) {
@@ -3188,11 +3189,10 @@ int neigh_xmit(int index, struct net_device *dev,
goto out_kfree_skb;
err = dev_queue_xmit(skb);
}
-out:
return err;
out_kfree_skb:
kfree_skb(skb);
- goto out;
+ return err;
}
EXPORT_SYMBOL(neigh_xmit);
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index ee3c1b37d06c..bb4ae3864ad2 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -712,20 +712,18 @@ EXPORT_SYMBOL_GPL(get_net_ns);
struct net *get_net_ns_by_fd(int fd)
{
- struct fd f = fdget(fd);
- struct net *net = ERR_PTR(-EINVAL);
+ CLASS(fd, f)(fd);
- if (!fd_file(f))
+ if (fd_empty(f))
return ERR_PTR(-EBADF);
if (proc_ns_file(fd_file(f))) {
struct ns_common *ns = get_proc_ns(file_inode(fd_file(f)));
if (ns->ops == &netns_operations)
- net = get_net(container_of(ns, struct net, ns));
+ return get_net(container_of(ns, struct net, ns));
}
- fdput(f);
- return net;
+ return ERR_PTR(-EINVAL);
}
EXPORT_SYMBOL_GPL(get_net_ns_by_fd);
#endif
diff --git a/net/core/netpoll.c b/net/core/netpoll.c
index 11b2a841b748..c48a47601c2f 100644
--- a/net/core/netpoll.c
+++ b/net/core/netpoll.c
@@ -506,7 +506,8 @@ void netpoll_print_options(struct netpoll *np)
np_info(np, "local IPv6 address %pI6c\n", &np->local_ip.in6);
else
np_info(np, "local IPv4 address %pI4\n", &np->local_ip.ip);
- np_info(np, "interface '%s'\n", np->dev_name);
+ np_info(np, "interface name '%s'\n", np->dev_name);
+ np_info(np, "local ethernet address '%pM'\n", np->dev_mac);
np_info(np, "remote port %d\n", np->remote_port);
if (np->ipv6)
np_info(np, "remote IPv6 address %pI6c\n", &np->remote_ip.in6);
@@ -575,11 +576,18 @@ int netpoll_parse_options(struct netpoll *np, char *opt)
cur++;
if (*cur != ',') {
- /* parse out dev name */
+ /* parse out dev_name or dev_mac */
if ((delim = strchr(cur, ',')) == NULL)
goto parse_failed;
*delim = 0;
- strscpy(np->dev_name, cur, sizeof(np->dev_name));
+
+ np->dev_name[0] = '\0';
+ eth_broadcast_addr(np->dev_mac);
+ if (!strchr(cur, ':'))
+ strscpy(np->dev_name, cur, sizeof(np->dev_name));
+ else if (!mac_pton(cur, np->dev_mac))
+ goto parse_failed;
+
cur = delim;
}
cur++;
@@ -684,110 +692,179 @@ int __netpoll_setup(struct netpoll *np, struct net_device *ndev)
}
EXPORT_SYMBOL_GPL(__netpoll_setup);
+/*
+ * Returns a pointer to a string representation of the identifier used
+ * to select the egress interface for the given netpoll instance. buf
+ * is used to format np->dev_mac when np->dev_name is empty; bufsz must
+ * be at least MAC_ADDR_STR_LEN + 1 to fit the formatted MAC address
+ * and its NUL terminator.
+ */
+static char *egress_dev(struct netpoll *np, char *buf, size_t bufsz)
+{
+ if (np->dev_name[0])
+ return np->dev_name;
+
+ snprintf(buf, bufsz, "%pM", np->dev_mac);
+ return buf;
+}
+
+static void netpoll_wait_carrier(struct netpoll *np, struct net_device *ndev,
+ unsigned int timeout)
+{
+ unsigned long atmost;
+
+ atmost = jiffies + timeout * HZ;
+ while (!netif_carrier_ok(ndev)) {
+ if (time_after(jiffies, atmost)) {
+ np_notice(np, "timeout waiting for carrier\n");
+ break;
+ }
+ msleep(1);
+ }
+}
+
+/*
+ * Take the IPv6 from ndev and populate local_ip structure in netpoll
+ */
+static int netpoll_take_ipv6(struct netpoll *np, struct net_device *ndev)
+{
+ char buf[MAC_ADDR_STR_LEN + 1];
+ int err = -EDESTADDRREQ;
+ struct inet6_dev *idev;
+
+ if (!IS_ENABLED(CONFIG_IPV6)) {
+ np_err(np, "IPv6 is not supported %s, aborting\n",
+ egress_dev(np, buf, sizeof(buf)));
+ return -EINVAL;
+ }
+
+ idev = __in6_dev_get(ndev);
+ if (idev) {
+ struct inet6_ifaddr *ifp;
+
+ read_lock_bh(&idev->lock);
+ list_for_each_entry(ifp, &idev->addr_list, if_list) {
+ if (!!(ipv6_addr_type(&ifp->addr) & IPV6_ADDR_LINKLOCAL) !=
+ !!(ipv6_addr_type(&np->remote_ip.in6) & IPV6_ADDR_LINKLOCAL))
+ continue;
+ /* Got the IP, let's return */
+ np->local_ip.in6 = ifp->addr;
+ err = 0;
+ break;
+ }
+ read_unlock_bh(&idev->lock);
+ }
+ if (err) {
+ np_err(np, "no IPv6 address for %s, aborting\n",
+ egress_dev(np, buf, sizeof(buf)));
+ return err;
+ }
+
+ np_info(np, "local IPv6 %pI6c\n", &np->local_ip.in6);
+ return 0;
+}
+
+/*
+ * Take the IPv4 from ndev and populate local_ip structure in netpoll
+ */
+static int netpoll_take_ipv4(struct netpoll *np, struct net_device *ndev)
+{
+ char buf[MAC_ADDR_STR_LEN + 1];
+ const struct in_ifaddr *ifa;
+ struct in_device *in_dev;
+
+ in_dev = __in_dev_get_rtnl(ndev);
+ if (!in_dev) {
+ np_err(np, "no IP address for %s, aborting\n",
+ egress_dev(np, buf, sizeof(buf)));
+ return -EDESTADDRREQ;
+ }
+
+ ifa = rtnl_dereference(in_dev->ifa_list);
+ if (!ifa) {
+ np_err(np, "no IP address for %s, aborting\n",
+ egress_dev(np, buf, sizeof(buf)));
+ return -EDESTADDRREQ;
+ }
+
+ np->local_ip.ip = ifa->ifa_local;
+ np_info(np, "local IP %pI4\n", &np->local_ip.ip);
+
+ return 0;
+}
+
+/*
+ * Test whether the caller left np->local_ip unset, so that
+ * netpoll_setup() should auto-populate it from the egress device.
+ *
+ * np->local_ip is a union of __be32 (IPv4) and struct in6_addr (IPv6),
+ * so an IPv6 address whose first 4 bytes are zero (e.g. ::1, ::2,
+ * IPv4-mapped ::ffff:a.b.c.d) must not be tested via the IPv4 arm —
+ * doing so would misclassify a caller-supplied address as unset and
+ * silently overwrite it with whatever address the device exposes.
+ */
+static bool netpoll_local_ip_unset(const struct netpoll *np)
+{
+ if (np->ipv6)
+ return ipv6_addr_any(&np->local_ip.in6);
+ return !np->local_ip.ip;
+}
+
int netpoll_setup(struct netpoll *np)
{
+ struct net *net = current->nsproxy->net_ns;
+ char buf[MAC_ADDR_STR_LEN + 1];
struct net_device *ndev = NULL;
bool ip_overwritten = false;
- struct in_device *in_dev;
int err;
rtnl_lock();
- if (np->dev_name[0]) {
- struct net *net = current->nsproxy->net_ns;
+ if (np->dev_name[0])
ndev = __dev_get_by_name(net, np->dev_name);
- }
+ else if (is_valid_ether_addr(np->dev_mac))
+ ndev = dev_getbyhwaddr(net, ARPHRD_ETHER, np->dev_mac);
+
if (!ndev) {
- np_err(np, "%s doesn't exist, aborting\n", np->dev_name);
+ np_err(np, "%s doesn't exist, aborting\n",
+ egress_dev(np, buf, sizeof(buf)));
err = -ENODEV;
goto unlock;
}
netdev_hold(ndev, &np->dev_tracker, GFP_KERNEL);
if (netdev_master_upper_dev_get(ndev)) {
- np_err(np, "%s is a slave device, aborting\n", np->dev_name);
+ np_err(np, "%s is a slave device, aborting\n",
+ egress_dev(np, buf, sizeof(buf)));
err = -EBUSY;
goto put;
}
if (!netif_running(ndev)) {
- unsigned long atmost;
-
- np_info(np, "device %s not up yet, forcing it\n", np->dev_name);
+ np_info(np, "device %s not up yet, forcing it\n",
+ egress_dev(np, buf, sizeof(buf)));
err = dev_open(ndev, NULL);
-
if (err) {
np_err(np, "failed to open %s\n", ndev->name);
goto put;
}
rtnl_unlock();
- atmost = jiffies + carrier_timeout * HZ;
- while (!netif_carrier_ok(ndev)) {
- if (time_after(jiffies, atmost)) {
- np_notice(np, "timeout waiting for carrier\n");
- break;
- }
- msleep(1);
- }
-
+ netpoll_wait_carrier(np, ndev, carrier_timeout);
rtnl_lock();
}
- if (!np->local_ip.ip) {
+ if (netpoll_local_ip_unset(np)) {
if (!np->ipv6) {
- const struct in_ifaddr *ifa;
-
- in_dev = __in_dev_get_rtnl(ndev);
- if (!in_dev)
- goto put_noaddr;
-
- ifa = rtnl_dereference(in_dev->ifa_list);
- if (!ifa) {
-put_noaddr:
- np_err(np, "no IP address for %s, aborting\n",
- np->dev_name);
- err = -EDESTADDRREQ;
+ err = netpoll_take_ipv4(np, ndev);
+ if (err)
goto put;
- }
-
- np->local_ip.ip = ifa->ifa_local;
- ip_overwritten = true;
- np_info(np, "local IP %pI4\n", &np->local_ip.ip);
} else {
-#if IS_ENABLED(CONFIG_IPV6)
- struct inet6_dev *idev;
-
- err = -EDESTADDRREQ;
- idev = __in6_dev_get(ndev);
- if (idev) {
- struct inet6_ifaddr *ifp;
-
- read_lock_bh(&idev->lock);
- list_for_each_entry(ifp, &idev->addr_list, if_list) {
- if (!!(ipv6_addr_type(&ifp->addr) & IPV6_ADDR_LINKLOCAL) !=
- !!(ipv6_addr_type(&np->remote_ip.in6) & IPV6_ADDR_LINKLOCAL))
- continue;
- np->local_ip.in6 = ifp->addr;
- ip_overwritten = true;
- err = 0;
- break;
- }
- read_unlock_bh(&idev->lock);
- }
- if (err) {
- np_err(np, "no IPv6 address for %s, aborting\n",
- np->dev_name);
+ err = netpoll_take_ipv6(np, ndev);
+ if (err)
goto put;
- } else
- np_info(np, "local IPv6 %pI6c\n", &np->local_ip.in6);
-#else
- np_err(np, "IPv6 is not supported %s, aborting\n",
- np->dev_name);
- err = -EINVAL;
- goto put;
-#endif
}
+ ip_overwritten = true;
}
err = __netpoll_setup(np, ndev);
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 458b040a8655..97ad4cc87be8 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -13,6 +13,7 @@
#include <net/netdev_rx_queue.h>
#include <net/page_pool/helpers.h>
+#include <net/page_pool/memory_provider.h>
#include <net/xdp.h>
#include <linux/dma-direction.h>
@@ -280,10 +281,21 @@ static int page_pool_init(struct page_pool *pool,
rxq = __netif_get_rx_queue(pool->slow.netdev,
pool->slow.queue_idx);
pool->mp_priv = rxq->mp_params.mp_priv;
+ pool->mp_ops = rxq->mp_params.mp_ops;
}
- if (pool->mp_priv) {
- err = mp_dmabuf_devmem_init(pool);
+ if (pool->mp_ops) {
+ if (!pool->dma_map || !pool->dma_sync) {
+ err = -EOPNOTSUPP;
+ goto free_ptr_ring;
+ }
+
+ if (WARN_ON(!is_kernel_rodata((unsigned long)pool->mp_ops))) {
+ err = -EFAULT;
+ goto free_ptr_ring;
+ }
+
+ err = pool->mp_ops->init(pool);
if (err) {
pr_warn("%s() mem-provider init failed %d\n", __func__,
err);
@@ -313,6 +325,11 @@ static void page_pool_uninit(struct page_pool *pool)
if (!pool->system)
free_percpu(pool->recycle_stats);
#endif
+
+ if (pool->mp_ops) {
+ pool->mp_ops->destroy(pool);
+ static_branch_dec(&page_pool_mem_providers);
+ }
}
/**
@@ -648,8 +665,8 @@ netmem_ref page_pool_alloc_netmem(struct page_pool *pool, gfp_t gfp)
return netmem;
/* Slow-path: cache empty, do real allocation */
- if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_priv)
- netmem = mp_dmabuf_devmem_alloc_netmems(pool, gfp);
+ if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops)
+ netmem = pool->mp_ops->alloc_netmems(pool, gfp);
else
netmem = __page_pool_alloc_pages_slow(pool, gfp);
return netmem;
@@ -743,8 +760,8 @@ void page_pool_return_page(struct page_pool *pool, netmem_ref netmem)
bool put;
put = true;
- if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_priv)
- put = mp_dmabuf_devmem_release_page(pool, netmem);
+ if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops)
+ put = pool->mp_ops->release_netmem(pool, netmem);
else
__page_pool_release_page_dma(pool, netmem);
@@ -1078,11 +1095,6 @@ static void __page_pool_destroy(struct page_pool *pool)
page_pool_unlist(pool);
page_pool_uninit(pool);
- if (pool->mp_priv) {
- mp_dmabuf_devmem_destroy(pool);
- static_branch_dec(&page_pool_mem_providers);
- }
-
kfree(pool);
}
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index a753d01b587b..aa9e91488473 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2214,6 +2214,7 @@ struct sk_buff *__pskb_copy_fclone(struct sk_buff *skb, int headroom,
skb_frag_ref(skb, i);
}
skb_shinfo(n)->nr_frags = i;
+ skb_shinfo(n)->flags |= skb_shinfo(skb)->flags & SKBFL_SHARED_FRAG;
}
if (skb_has_frag_list(skb)) {
@@ -4289,6 +4290,8 @@ int skb_shift(struct sk_buff *tgt, struct sk_buff *skb, int shiftlen)
tgt->ip_summed = CHECKSUM_PARTIAL;
skb->ip_summed = CHECKSUM_PARTIAL;
+ skb_shinfo(tgt)->flags |= skb_shinfo(skb)->flags & SKBFL_SHARED_FRAG;
+
skb_len_add(skb, -shiftlen);
skb_len_add(tgt, shiftlen);
@@ -4899,7 +4902,8 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,
skb_copy_from_linear_data_offset(head_skb, offset,
skb_put(nskb, hsize), hsize);
- skb_shinfo(nskb)->flags |= skb_shinfo(head_skb)->flags &
+ skb_shinfo(nskb)->flags |= (skb_shinfo(head_skb)->flags |
+ skb_shinfo(frag_skb)->flags) &
SKBFL_SHARED_FRAG;
if (skb_zerocopy_clone(nskb, frag_skb, GFP_ATOMIC))
@@ -4916,6 +4920,9 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,
nfrags = skb_shinfo(list_skb)->nr_frags;
frag = skb_shinfo(list_skb)->frags;
frag_skb = list_skb;
+
+ skb_shinfo(nskb)->flags |= skb_shinfo(frag_skb)->flags & SKBFL_SHARED_FRAG;
+
if (!skb_headlen(list_skb)) {
BUG_ON(!nfrags);
} else {
@@ -6066,6 +6073,8 @@ bool skb_try_coalesce(struct sk_buff *to, struct sk_buff *from,
from_shinfo->frags,
from_shinfo->nr_frags * sizeof(skb_frag_t));
to_shinfo->nr_frags += from_shinfo->nr_frags;
+ if (from_shinfo->nr_frags)
+ to_shinfo->flags |= from_shinfo->flags & SKBFL_SHARED_FRAG;
if (!skb_cloned(from))
from_shinfo->nr_frags = 0;
diff --git a/net/ipv4/netfilter/arp_tables.c b/net/ipv4/netfilter/arp_tables.c
index 1cdd9c28ab2d..97ead883e4a1 100644
--- a/net/ipv4/netfilter/arp_tables.c
+++ b/net/ipv4/netfilter/arp_tables.c
@@ -110,13 +110,25 @@ static inline int arp_packet_match(const struct arphdr *arphdr,
arpptr += dev->addr_len;
memcpy(&src_ipaddr, arpptr, sizeof(u32));
arpptr += sizeof(u32);
- tgt_devaddr = arpptr;
- arpptr += dev->addr_len;
+
+ if (IS_ENABLED(CONFIG_FIREWIRE_NET) && dev->type == ARPHRD_IEEE1394) {
+ if (unlikely(memchr_inv(arpinfo->tgt_devaddr.mask, 0,
+ sizeof(arpinfo->tgt_devaddr.mask))))
+ return 0;
+
+ tgt_devaddr = NULL;
+ } else {
+ tgt_devaddr = arpptr;
+ arpptr += dev->addr_len;
+ }
memcpy(&tgt_ipaddr, arpptr, sizeof(u32));
if (NF_INVF(arpinfo, ARPT_INV_SRCDEVADDR,
arp_devaddr_compare(&arpinfo->src_devaddr, src_devaddr,
- dev->addr_len)) ||
+ dev->addr_len)))
+ return 0;
+
+ if (tgt_devaddr &&
NF_INVF(arpinfo, ARPT_INV_TGTDEVADDR,
arp_devaddr_compare(&arpinfo->tgt_devaddr, tgt_devaddr,
dev->addr_len)))
diff --git a/net/ipv4/netfilter/arpt_mangle.c b/net/ipv4/netfilter/arpt_mangle.c
index a4e07e5e9c11..f65dd339208e 100644
--- a/net/ipv4/netfilter/arpt_mangle.c
+++ b/net/ipv4/netfilter/arpt_mangle.c
@@ -40,6 +40,10 @@ target(struct sk_buff *skb, const struct xt_action_param *par)
}
arpptr += pln;
if (mangle->flags & ARPT_MANGLE_TDEV) {
+ if (unlikely(IS_ENABLED(CONFIG_FIREWIRE_NET) &&
+ skb->dev->type == ARPHRD_IEEE1394))
+ return NF_DROP;
+
if (ARPT_DEV_ADDR_LEN_MAX < hln ||
(arpptr + hln > skb_tail_pointer(skb)))
return NF_DROP;
@@ -47,6 +51,10 @@ target(struct sk_buff *skb, const struct xt_action_param *par)
}
arpptr += hln;
if (mangle->flags & ARPT_MANGLE_TIP) {
+ if (unlikely(IS_ENABLED(CONFIG_FIREWIRE_NET) &&
+ skb->dev->type == ARPHRD_IEEE1394))
+ return NF_DROP;
+
if (ARPT_MANGLE_ADDR_LEN_MAX < pln ||
(arpptr + pln > skb_tail_pointer(skb)))
return NF_DROP;
diff --git a/net/ipv4/nexthop.c b/net/ipv4/nexthop.c
index f1d499a3e274..52d4890be83e 100644
--- a/net/ipv4/nexthop.c
+++ b/net/ipv4/nexthop.c
@@ -2445,10 +2445,10 @@ static int replace_nexthop_single(struct net *net, struct nexthop *old,
goto err_notify;
}
- /* When replacing an IPv4 nexthop with an IPv6 nexthop, potentially
+ /* When replacing a nexthop with one of a different family, potentially
* update IPv4 indication in all the groups using the nexthop.
*/
- if (oldi->family == AF_INET && newi->family == AF_INET6) {
+ if (oldi->family != newi->family) {
list_for_each_entry(nhge, &old->grp_list, nh_list) {
struct nexthop *nhp = nhge->nh_parent;
struct nh_group *nhg;
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 1948d15f1f28..640fc3b54277 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -284,7 +284,6 @@ static int cookie_tcp_reqsk_init(struct sock *sk, struct sk_buff *skb,
treq->rcv_isn = ntohl(th->seq) - 1;
treq->snt_isn = ntohl(th->ack_seq) - 1;
treq->syn_tos = TCP_SKB_CB(skb)->ip_dsfield;
- treq->req_usec_ts = false;
#if IS_ENABLED(CONFIG_MPTCP)
treq->is_mptcp = sk_is_mptcp(sk);
@@ -347,6 +346,7 @@ struct request_sock *cookie_tcp_reqsk_alloc(const struct request_sock_ops *ops,
ireq->wscale_ok = tcp_opt->wscale_ok;
ireq->ecn_ok = !!(tcp_opt->rcv_tsecr & TS_OPT_ECN);
+ treq->req_usec_ts = false;
treq->ts_off = tsoff;
return req;
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 9c5fc4464783..8b90665245b2 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4276,9 +4276,9 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk,
nla_put_u64_64bit(stats, TCP_NLA_SNDBUF_LIMITED,
info.tcpi_sndbuf_limited, TCP_NLA_PAD);
nla_put_u64_64bit(stats, TCP_NLA_DATA_SEGS_OUT,
- tp->data_segs_out, TCP_NLA_PAD);
+ READ_ONCE(tp->data_segs_out), TCP_NLA_PAD);
nla_put_u64_64bit(stats, TCP_NLA_TOTAL_RETRANS,
- tp->total_retrans, TCP_NLA_PAD);
+ READ_ONCE(tp->total_retrans), TCP_NLA_PAD);
rate = READ_ONCE(sk->sk_pacing_rate);
rate64 = (rate != ~0UL) ? rate : ~0ULL;
@@ -4297,26 +4297,30 @@ struct sk_buff *tcp_get_timestamping_opt_stats(const struct sock *sk,
nla_put_u32(stats, TCP_NLA_DELIVERED, tp->delivered);
nla_put_u32(stats, TCP_NLA_DELIVERED_CE, tp->delivered_ce);
- nla_put_u32(stats, TCP_NLA_SNDQ_SIZE, tp->write_seq - tp->snd_una);
+ nla_put_u32(stats, TCP_NLA_SNDQ_SIZE,
+ max_t(int, 0,
+ READ_ONCE(tp->write_seq) - READ_ONCE(tp->snd_una)));
nla_put_u8(stats, TCP_NLA_CA_STATE, inet_csk(sk)->icsk_ca_state);
- nla_put_u64_64bit(stats, TCP_NLA_BYTES_SENT, tp->bytes_sent,
+ nla_put_u64_64bit(stats, TCP_NLA_BYTES_SENT, READ_ONCE(tp->bytes_sent),
TCP_NLA_PAD);
- nla_put_u64_64bit(stats, TCP_NLA_BYTES_RETRANS, tp->bytes_retrans,
- TCP_NLA_PAD);
- nla_put_u32(stats, TCP_NLA_DSACK_DUPS, tp->dsack_dups);
+ nla_put_u64_64bit(stats, TCP_NLA_BYTES_RETRANS,
+ READ_ONCE(tp->bytes_retrans), TCP_NLA_PAD);
+ nla_put_u32(stats, TCP_NLA_DSACK_DUPS, READ_ONCE(tp->dsack_dups));
nla_put_u32(stats, TCP_NLA_REORD_SEEN, tp->reord_seen);
nla_put_u32(stats, TCP_NLA_SRTT, tp->srtt_us >> 3);
nla_put_u16(stats, TCP_NLA_TIMEOUT_REHASH, tp->timeout_rehash);
nla_put_u32(stats, TCP_NLA_BYTES_NOTSENT,
- max_t(int, 0, tp->write_seq - tp->snd_nxt));
+ max_t(int, 0,
+ READ_ONCE(tp->write_seq) - READ_ONCE(tp->snd_nxt)));
nla_put_u64_64bit(stats, TCP_NLA_EDT, orig_skb->skb_mstamp_ns,
TCP_NLA_PAD);
if (ack_skb)
nla_put_u8(stats, TCP_NLA_TTL,
tcp_skb_ttl_or_hop_limit(ack_skb));
- nla_put_u32(stats, TCP_NLA_REHASH, tp->plb_rehash + tp->timeout_rehash);
+ nla_put_u32(stats, TCP_NLA_REHASH,
+ READ_ONCE(tp->plb_rehash) + READ_ONCE(tp->timeout_rehash));
return stats;
}
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index c498588c021d..60c42d612d18 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -1059,7 +1059,7 @@ static u32 tcp_dsack_seen(struct tcp_sock *tp, u32 start_seq,
else if (tp->tlp_high_seq && tp->tlp_high_seq == end_seq)
state->flag |= FLAG_DSACK_TLP;
- tp->dsack_dups += dup_segs;
+ WRITE_ONCE(tp->dsack_dups, tp->dsack_dups + dup_segs);
/* Skip the DSACK if dup segs weren't retransmitted by sender */
if (tp->dsack_dups > tp->total_retrans)
return 0;
@@ -3670,7 +3670,7 @@ static void tcp_snd_una_update(struct tcp_sock *tp, u32 ack)
sock_owned_by_me((struct sock *)tp);
tp->bytes_acked += delta;
tcp_snd_sne_update(tp, ack);
- tp->snd_una = ack;
+ WRITE_ONCE(tp->snd_una, ack);
}
static void tcp_rcv_sne_update(struct tcp_sock *tp, u32 seq)
@@ -6877,7 +6877,7 @@ tcp_rcv_state_process(struct sock *sk, struct sk_buff *skb)
if (sk->sk_socket)
sk_wake_async(sk, SOCK_WAKE_IO, POLL_OUT);
- tp->snd_una = TCP_SKB_CB(skb)->ack_seq;
+ WRITE_ONCE(tp->snd_una, TCP_SKB_CB(skb)->ack_seq);
tp->snd_wnd = ntohs(th->window) << tp->rx_opt.snd_wscale;
tcp_init_wl(tp, TCP_SKB_CB(skb)->seq);
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 5e37dc45639d..c76672f544be 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -1446,8 +1446,10 @@ static int __tcp_transmit_skb(struct sock *sk, struct sk_buff *skb,
if (skb->len != tcp_header_size) {
tcp_event_data_sent(tp, sk);
- tp->data_segs_out += tcp_skb_pcount(skb);
- tp->bytes_sent += skb->len - tcp_header_size;
+ WRITE_ONCE(tp->data_segs_out,
+ tp->data_segs_out + tcp_skb_pcount(skb));
+ WRITE_ONCE(tp->bytes_sent,
+ tp->bytes_sent + skb->len - tcp_header_size);
}
if (after(tcb->end_seq, tp->snd_nxt) || tcb->seq == tcb->end_seq)
@@ -2389,6 +2391,7 @@ static int tcp_clone_payload(struct sock *sk, struct sk_buff *to,
todo = min_t(int, skb_frag_size(fragfrom),
probe_size - len);
len += todo;
+ skb_shinfo(to)->flags |= skb_shinfo(skb)->flags & SKBFL_SHARED_FRAG;
if (lastfrag &&
skb_frag_page(fragfrom) == skb_frag_page(lastfrag) &&
skb_frag_off(fragfrom) == skb_frag_off(lastfrag) +
@@ -3411,8 +3414,8 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs)
TCP_ADD_STATS(sock_net(sk), TCP_MIB_RETRANSSEGS, segs);
if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_SYN)
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPSYNRETRANS);
- tp->total_retrans += segs;
- tp->bytes_retrans += skb->len;
+ WRITE_ONCE(tp->total_retrans, tp->total_retrans + segs);
+ WRITE_ONCE(tp->bytes_retrans, tp->bytes_retrans + skb->len);
/* make sure skb->data is aligned on arches that require it
* and check if ack-trimming & collapsing extended the headroom
@@ -3944,7 +3947,7 @@ static void tcp_connect_init(struct sock *sk)
tp->snd_wnd = 0;
tcp_init_wl(tp, 0);
tcp_write_queue_purge(sk);
- tp->snd_una = tp->write_seq;
+ WRITE_ONCE(tp->snd_una, tp->write_seq);
tp->snd_sml = tp->write_seq;
tp->snd_up = tp->write_seq;
WRITE_ONCE(tp->snd_nxt, tp->write_seq);
@@ -4433,7 +4436,8 @@ int tcp_rtx_synack(const struct sock *sk, struct request_sock *req)
* However in this case, we are dealing with a passive fastopen
* socket thus we can change total_retrans value.
*/
- tcp_sk_rw(sk)->total_retrans++;
+ WRITE_ONCE(tcp_sk_rw(sk)->total_retrans,
+ tcp_sk_rw(sk)->total_retrans + 1);
}
trace_tcp_retransmit_synack(sk, req);
}
diff --git a/net/ipv4/tcp_plb.c b/net/ipv4/tcp_plb.c
index 4bcf7eff95e3..b7f9b60d8991 100644
--- a/net/ipv4/tcp_plb.c
+++ b/net/ipv4/tcp_plb.c
@@ -79,7 +79,7 @@ void tcp_plb_check_rehash(struct sock *sk, struct tcp_plb_state *plb)
sk_rethink_txhash(sk);
plb->consec_cong_rounds = 0;
- tcp_sk(sk)->plb_rehash++;
+ WRITE_ONCE(tcp_sk(sk)->plb_rehash, tcp_sk(sk)->plb_rehash + 1);
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPPLBREHASH);
}
EXPORT_SYMBOL_GPL(tcp_plb_check_rehash);
diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
index 79064580c8c0..0cc8f19bc102 100644
--- a/net/ipv4/tcp_timer.c
+++ b/net/ipv4/tcp_timer.c
@@ -49,7 +49,8 @@ static u32 tcp_clamp_rto_to_user_timeout(const struct sock *sk)
u32 tcp_clamp_probe0_to_user_timeout(const struct sock *sk, u32 when)
{
const struct inet_connection_sock *icsk = inet_csk(sk);
- u32 remaining, user_timeout;
+ u32 user_timeout;
+ s32 remaining;
s32 elapsed;
user_timeout = READ_ONCE(icsk->icsk_user_timeout);
@@ -60,7 +61,7 @@ u32 tcp_clamp_probe0_to_user_timeout(const struct sock *sk, u32 when)
if (unlikely(elapsed < 0))
elapsed = 0;
remaining = msecs_to_jiffies(user_timeout) - elapsed;
- remaining = max_t(u32, remaining, TCP_TIMEOUT_MIN);
+ remaining = max_t(int, remaining, TCP_TIMEOUT_MIN);
return min_t(u32, remaining, when);
}
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 27694334e410..865803caed74 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -68,7 +68,7 @@
* YOSHIFUJI Hideaki @USAGI and: Support IPV6_V6ONLY socket option, which
* Alexey Kuznetsov: allow both IPv4 and IPv6 sockets to bind
* a single port at the same time.
- * Derek Atkins <derek@ihtfp.com>: Add Encapulation Support
+ * Derek Atkins <derek@ihtfp.com>: Add Encapsulation Support
* James Chapman : Add L2TP encapsulation type.
*/
@@ -366,10 +366,10 @@ int udp_v4_get_port(struct sock *sk, unsigned short snum)
return udp_lib_get_port(sk, snum, hash2_nulladdr);
}
-static int compute_score(struct sock *sk, const struct net *net,
- __be32 saddr, __be16 sport,
- __be32 daddr, unsigned short hnum,
- int dif, int sdif)
+static __always_inline int
+compute_score(struct sock *sk, const struct net *net,
+ __be32 saddr, __be16 sport, __be32 daddr,
+ unsigned short hnum, int dif, int sdif)
{
int score;
struct inet_sock *inet;
@@ -509,8 +509,8 @@ static struct sock *udp4_lib_lookup2(const struct net *net,
continue;
/* compute_score is too long of a function to be
- * inlined, and calling it again here yields
- * measureable overhead for some
+ * inlined twice here, and calling it uninlined
+ * here yields measurable overhead for some
* workloads. Work around it by jumping
* backwards to rescore 'result'.
*/
@@ -2414,7 +2414,7 @@ static inline int udp4_csum_init(struct sk_buff *skb, struct udphdr *uh,
return 0;
}
-/* wrapper for udp_queue_rcv_skb tacking care of csum conversion and
+/* wrapper for udp_queue_rcv_skb taking care of csum conversion and
* return code conversion for ip layer consumption
*/
static int udp_unicast_rcv_skb(struct sock *sk, struct sk_buff *skb,
diff --git a/net/ipv6/icmp.c b/net/ipv6/icmp.c
index 387400829b20..229ae205450d 100644
--- a/net/ipv6/icmp.c
+++ b/net/ipv6/icmp.c
@@ -910,7 +910,6 @@ static int icmpv6_rcv(struct sk_buff *skb)
struct net *net = dev_net_rcu(skb->dev);
struct net_device *dev = icmp6_dev(skb);
struct inet6_dev *idev = __in6_dev_get(dev);
- const struct in6_addr *saddr, *daddr;
struct icmp6hdr *hdr;
u8 type;
@@ -941,12 +940,10 @@ static int icmpv6_rcv(struct sk_buff *skb)
__ICMP6_INC_STATS(dev_net_rcu(dev), idev, ICMP6_MIB_INMSGS);
- saddr = &ipv6_hdr(skb)->saddr;
- daddr = &ipv6_hdr(skb)->daddr;
-
if (skb_checksum_validate(skb, IPPROTO_ICMPV6, ip6_compute_pseudo)) {
net_dbg_ratelimited("ICMPv6 checksum failed [%pI6c > %pI6c]\n",
- saddr, daddr);
+ &ipv6_hdr(skb)->saddr,
+ &ipv6_hdr(skb)->daddr);
goto csum_error;
}
@@ -1029,7 +1026,8 @@ static int icmpv6_rcv(struct sk_buff *skb)
break;
net_dbg_ratelimited("icmpv6: msg of unknown type [%pI6c > %pI6c]\n",
- saddr, daddr);
+ &ipv6_hdr(skb)->saddr,
+ &ipv6_hdr(skb)->daddr);
/*
* error of unknown type.
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 9b93df668025..f6717b0d037d 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -114,10 +114,11 @@ void udp_v6_rehash(struct sock *sk)
udp_lib_rehash(sk, new_hash);
}
-static int compute_score(struct sock *sk, const struct net *net,
- const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, unsigned short hnum,
- int dif, int sdif)
+static __always_inline int
+compute_score(struct sock *sk, const struct net *net,
+ const struct in6_addr *saddr, __be16 sport,
+ const struct in6_addr *daddr, unsigned short hnum,
+ int dif, int sdif)
{
int bound_dev_if, score;
struct inet_sock *inet;
@@ -247,8 +248,8 @@ static struct sock *udp6_lib_lookup2(const struct net *net,
continue;
/* compute_score is too long of a function to be
- * inlined, and calling it again here yields
- * measureable overhead for some
+ * inlined twice here, and calling it uninlined
+ * here yields measurable overhead for some
* workloads. Work around it by jumping
* backwards to rescore 'result'.
*/
@@ -364,7 +365,7 @@ struct sock *udp6_lib_lookup(const struct net *net, const struct in6_addr *saddr
EXPORT_SYMBOL_GPL(udp6_lib_lookup);
#endif
-/* do not use the scratch area len for jumbogram: their length execeeds the
+/* do not use the scratch area len for jumbogram: their length exceeds the
* scratch area space; note that the IP6CB flags is still in the first
* cacheline, so checking for jumbograms is cheap
*/
@@ -964,7 +965,7 @@ static void udp6_sk_rx_dst_set(struct sock *sk, struct dst_entry *dst)
sk->sk_rx_dst_cookie = rt6_get_cookie(dst_rt6_info(dst));
}
-/* wrapper for udp_queue_rcv_skb tacking care of csum conversion and
+/* wrapper for udp_queue_rcv_skb taking care of csum conversion and
* return code conversion for ip layer consumption
*/
static int udp6_unicast_rcv_skb(struct sock *sk, struct sk_buff *skb,
diff --git a/net/mac80211/debugfs_sta.c b/net/mac80211/debugfs_sta.c
index e6f937cfedcf..3df6725ab00e 100644
--- a/net/mac80211/debugfs_sta.c
+++ b/net/mac80211/debugfs_sta.c
@@ -454,11 +454,12 @@ static ssize_t link_sta_addr_read(struct file *file, char __user *userbuf,
size_t count, loff_t *ppos)
{
struct link_sta_info *link_sta = file->private_data;
- u8 mac[3 * ETH_ALEN + 1];
+ u8 mac[MAC_ADDR_STR_LEN + 2];
snprintf(mac, sizeof(mac), "%pM\n", link_sta->pub->addr);
- return simple_read_from_buffer(userbuf, count, ppos, mac, 3 * ETH_ALEN);
+ return simple_read_from_buffer(userbuf, count, ppos, mac,
+ MAC_ADDR_STR_LEN + 1);
}
LINK_STA_OPS(addr);
@@ -1237,7 +1238,7 @@ void ieee80211_sta_debugfs_add(struct sta_info *sta)
struct ieee80211_local *local = sta->local;
struct ieee80211_sub_if_data *sdata = sta->sdata;
struct dentry *stations_dir = sta->sdata->debugfs.subdir_stations;
- u8 mac[3*ETH_ALEN];
+ u8 mac[MAC_ADDR_STR_LEN + 1];
if (!stations_dir)
return;
diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c
index 835316fd3cd7..20e5f513a27a 100644
--- a/net/mac80211/mlme.c
+++ b/net/mac80211/mlme.c
@@ -5304,7 +5304,8 @@ ieee80211_determine_our_sta_mode(struct ieee80211_sub_if_data *sdata,
if (is_5ghz &&
!(vht_cap.cap & (IEEE80211_VHT_CAP_SUPP_CHAN_WIDTH_160MHZ |
- IEEE80211_VHT_CAP_SUPP_CHAN_WIDTH_160_80PLUS80MHZ))) {
+ IEEE80211_VHT_CAP_SUPP_CHAN_WIDTH_160_80PLUS80MHZ |
+ IEEE80211_VHT_CAP_EXT_NSS_BW_MASK))) {
conn->bw_limit = IEEE80211_CONN_BW_LIMIT_80;
mlme_link_id_dbg(sdata, link_id,
"no VHT 160 MHz capability on 5 GHz, limiting to 80 MHz");
diff --git a/net/mptcp/fastopen.c b/net/mptcp/fastopen.c
index a29ff901df75..d4dbdd3d5679 100644
--- a/net/mptcp/fastopen.c
+++ b/net/mptcp/fastopen.c
@@ -12,6 +12,7 @@ void mptcp_fastopen_subflow_synack_set_params(struct mptcp_subflow_context *subf
struct sock *sk, *ssk;
struct sk_buff *skb;
struct tcp_sock *tp;
+ bool has_rxtstamp;
/* on early fallback the subflow context is deleted by
* subflow_syn_recv_sock()
@@ -39,14 +40,14 @@ void mptcp_fastopen_subflow_synack_set_params(struct mptcp_subflow_context *subf
*/
tp->copied_seq += skb->len;
subflow->ssn_offset += skb->len;
+ has_rxtstamp = TCP_SKB_CB(skb)->has_rxtstamp;
- /* initialize a dummy sequence number, we will update it at MPC
- * completion, if needed
- */
+ /* Only the sequence delta is relevant */
MPTCP_SKB_CB(skb)->map_seq = -skb->len;
MPTCP_SKB_CB(skb)->end_seq = 0;
MPTCP_SKB_CB(skb)->offset = 0;
- MPTCP_SKB_CB(skb)->has_rxtstamp = TCP_SKB_CB(skb)->has_rxtstamp;
+ MPTCP_SKB_CB(skb)->has_rxtstamp = has_rxtstamp;
+ MPTCP_SKB_CB(skb)->cant_coalesce = 1;
mptcp_data_lock(sk);
@@ -58,22 +59,3 @@ void mptcp_fastopen_subflow_synack_set_params(struct mptcp_subflow_context *subf
mptcp_data_unlock(sk);
}
-
-void __mptcp_fastopen_gen_msk_ackseq(struct mptcp_sock *msk, struct mptcp_subflow_context *subflow,
- const struct mptcp_options_received *mp_opt)
-{
- struct sock *sk = (struct sock *)msk;
- struct sk_buff *skb;
-
- skb = skb_peek_tail(&sk->sk_receive_queue);
- if (skb) {
- WARN_ON_ONCE(MPTCP_SKB_CB(skb)->end_seq);
- pr_debug("msk %p moving seq %llx -> %llx end_seq %llx -> %llx\n", sk,
- MPTCP_SKB_CB(skb)->map_seq, MPTCP_SKB_CB(skb)->map_seq + msk->ack_seq,
- MPTCP_SKB_CB(skb)->end_seq, MPTCP_SKB_CB(skb)->end_seq + msk->ack_seq);
- MPTCP_SKB_CB(skb)->map_seq += msk->ack_seq;
- MPTCP_SKB_CB(skb)->end_seq += msk->ack_seq;
- }
-
- pr_debug("msk=%p ack_seq=%llx\n", msk, msk->ack_seq);
-}
diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index 3ac09bfe6e4b..5d892583ab4e 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -307,8 +307,15 @@ static void mptcp_pm_add_timer(struct timer_list *timer)
if (!entry->addr.id)
return;
+ bh_lock_sock(sk);
+ if (sock_owned_by_user(sk)) {
+ /* Try again later. */
+ sk_reset_timer(sk, timer, jiffies + HZ / 20);
+ goto out;
+ }
+
if (mptcp_pm_should_add_signal_addr(msk)) {
- sk_reset_timer(sk, timer, jiffies + TCP_RTO_MAX / 8);
+ sk_reset_timer(sk, timer, jiffies + HZ);
goto out;
}
@@ -335,6 +342,7 @@ static void mptcp_pm_add_timer(struct timer_list *timer)
mptcp_pm_subflow_established(msk);
out:
+ bh_unlock_sock(sk);
__sock_put(sk);
}
@@ -590,6 +598,8 @@ static void mptcp_pm_create_subflow_or_signal_addr(struct mptcp_sock *msk)
/* check first for announce */
if (msk->pm.add_addr_signaled < add_addr_signal_max) {
+ u8 endp_id;
+
/* due to racing events on both ends we can reach here while
* previous add address is still running: if we invoke now
* mptcp_pm_announce_addr(), that will fail and the
@@ -603,19 +613,20 @@ static void mptcp_pm_create_subflow_or_signal_addr(struct mptcp_sock *msk)
if (!select_signal_address(pernet, msk, &local))
goto subflow;
+ /* Special case for ID0: set the correct ID */
+ endp_id = local.addr.id;
+ if (endp_id == msk->mpc_endpoint_id)
+ local.addr.id = 0;
+
/* If the alloc fails, we are on memory pressure, not worth
* continuing, and trying to create subflows.
*/
if (!mptcp_pm_alloc_anno_list(msk, &local.addr))
return;
- __clear_bit(local.addr.id, msk->pm.id_avail_bitmap);
+ __clear_bit(endp_id, msk->pm.id_avail_bitmap);
msk->pm.add_addr_signaled++;
- /* Special case for ID0: set the correct ID */
- if (local.addr.id == msk->mpc_endpoint_id)
- local.addr.id = 0;
-
mptcp_pm_announce_addr(msk, &local.addr, false);
mptcp_pm_nl_addr_send_ack(msk);
@@ -920,6 +931,9 @@ int mptcp_pm_nl_mp_prio_send_ack(struct mptcp_sock *msk,
struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
struct mptcp_addr_info local, remote;
+ if (!__mptcp_subflow_active(subflow))
+ continue;
+
mptcp_local_address((struct sock_common *)ssk, &local);
if (!mptcp_addresses_equal(&local, addr, addr->port))
continue;
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index f9dd5c3d2d50..7dbb666c72c3 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -137,7 +137,8 @@ static bool mptcp_try_coalesce(struct sock *sk, struct sk_buff *to,
bool fragstolen;
int delta;
- if (MPTCP_SKB_CB(from)->offset ||
+ if (unlikely(MPTCP_SKB_CB(to)->cant_coalesce) ||
+ MPTCP_SKB_CB(from)->offset ||
((to->len + from->len) > (sk->sk_rcvbuf >> 3)) ||
!skb_try_coalesce(to, from, &fragstolen, &delta))
return false;
@@ -368,6 +369,7 @@ static bool __mptcp_move_skb(struct mptcp_sock *msk, struct sock *ssk,
MPTCP_SKB_CB(skb)->end_seq = MPTCP_SKB_CB(skb)->map_seq + copy_len;
MPTCP_SKB_CB(skb)->offset = offset;
MPTCP_SKB_CB(skb)->has_rxtstamp = has_rxtstamp;
+ MPTCP_SKB_CB(skb)->cant_coalesce = 0;
if (MPTCP_SKB_CB(skb)->map_seq == msk->ack_seq) {
/* in sequence */
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 391a8026cb48..9ed9cb36e9bb 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -129,7 +129,8 @@ struct mptcp_skb_cb {
u64 map_seq;
u64 end_seq;
u32 offset;
- u8 has_rxtstamp:1;
+ u8 has_rxtstamp;
+ u8 cant_coalesce;
};
#define MPTCP_SKB_CB(__skb) ((struct mptcp_skb_cb *)&((__skb)->cb[0]))
@@ -1069,8 +1070,6 @@ void mptcp_event_pm_listener(const struct sock *ssk,
enum mptcp_event_type event);
bool mptcp_userspace_pm_active(const struct mptcp_sock *msk);
-void __mptcp_fastopen_gen_msk_ackseq(struct mptcp_sock *msk, struct mptcp_subflow_context *subflow,
- const struct mptcp_options_received *mp_opt);
void mptcp_fastopen_subflow_synack_set_params(struct mptcp_subflow_context *subflow,
struct request_sock *req);
int mptcp_nl_fill_addr(struct sk_buff *skb,
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 3843d3a80f4f..10e945f5fa0f 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -802,9 +802,6 @@ void __mptcp_subflow_fully_established(struct mptcp_sock *msk,
subflow_set_remote_key(msk, subflow, mp_opt);
WRITE_ONCE(subflow->fully_established, true);
WRITE_ONCE(msk->fully_established, true);
-
- if (subflow->is_mptfo)
- __mptcp_fastopen_gen_msk_ackseq(msk, subflow, mp_opt);
}
static struct sock *subflow_syn_recv_sock(const struct sock *sk,
diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c
index 8892f261451e..ed8b2616cf17 100644
--- a/net/netfilter/ipvs/ip_vs_xmit.c
+++ b/net/netfilter/ipvs/ip_vs_xmit.c
@@ -103,6 +103,18 @@ __ip_vs_dst_check(struct ip_vs_dest *dest)
return dest_dst;
}
+/* Based on ip_exceeds_mtu(). */
+static bool ip_vs_exceeds_mtu(const struct sk_buff *skb, unsigned int mtu)
+{
+ if (skb->len <= mtu)
+ return false;
+
+ if (skb_is_gso(skb) && skb_gso_validate_network_len(skb, mtu))
+ return false;
+
+ return true;
+}
+
static inline bool
__mtu_check_toobig_v6(const struct sk_buff *skb, u32 mtu)
{
@@ -112,10 +124,9 @@ __mtu_check_toobig_v6(const struct sk_buff *skb, u32 mtu)
*/
if (IP6CB(skb)->frag_max_size > mtu)
return true; /* largest fragment violate MTU */
- }
- else if (skb->len > mtu && !skb_is_gso(skb)) {
+ } else if (ip_vs_exceeds_mtu(skb, mtu))
return true; /* Packet size violate MTU size */
- }
+
return false;
}
@@ -233,7 +244,7 @@ static inline bool ensure_mtu_is_adequate(struct netns_ipvs *ipvs, int skb_af,
return true;
if (unlikely(ip_hdr(skb)->frag_off & htons(IP_DF) &&
- skb->len > mtu && !skb_is_gso(skb) &&
+ ip_vs_exceeds_mtu(skb, mtu) &&
!ip_vs_iph_icmp(ipvsh))) {
icmp_send(skb, ICMP_DEST_UNREACH, ICMP_FRAG_NEEDED,
htonl(mtu));
diff --git a/net/netfilter/nf_conntrack_proto_sctp.c b/net/netfilter/nf_conntrack_proto_sctp.c
index fabb2c1ca00a..0dd55d3fba38 100644
--- a/net/netfilter/nf_conntrack_proto_sctp.c
+++ b/net/netfilter/nf_conntrack_proto_sctp.c
@@ -471,9 +471,13 @@ int nf_conntrack_sctp_packet(struct nf_conn *ct,
if (!ih)
goto out_unlock;
- if (ct->proto.sctp.init[dir] && ct->proto.sctp.init[!dir])
- ct->proto.sctp.init[!dir] = 0;
- ct->proto.sctp.init[dir] = 1;
+ /* Do not record INIT matching peer vtag (stale or retransmitted INIT). */
+ if (old_state == SCTP_CONNTRACK_NONE ||
+ ct->proto.sctp.vtag[!dir] != ih->init_tag) {
+ if (ct->proto.sctp.init[dir] && ct->proto.sctp.init[!dir])
+ ct->proto.sctp.init[!dir] = 0;
+ ct->proto.sctp.init[dir] = 1;
+ }
pr_debug("Setting vtag %x for dir %d\n", ih->init_tag, !dir);
ct->proto.sctp.vtag[!dir] = ih->init_tag;
diff --git a/net/netfilter/nf_conntrack_sip.c b/net/netfilter/nf_conntrack_sip.c
index fda6fc1fc4c5..ec31611b7a29 100644
--- a/net/netfilter/nf_conntrack_sip.c
+++ b/net/netfilter/nf_conntrack_sip.c
@@ -181,6 +181,57 @@ static int sip_parse_addr(const struct nf_conn *ct, const char *cp,
return 1;
}
+/* Parse optional port number after IP address.
+ * Returns false on malformed input, true otherwise.
+ * If port is non-NULL, stores parsed port in network byte order.
+ * If no port is present, sets *port to default SIP port.
+ */
+static bool sip_parse_port(const char *dptr, const char **endp,
+ const char *limit, __be16 *port)
+{
+ unsigned int p = 0;
+ int len = 0;
+
+ if (dptr >= limit)
+ return false;
+
+ if (*dptr != ':') {
+ if (port)
+ *port = htons(SIP_PORT);
+ if (endp)
+ *endp = dptr;
+ return true;
+ }
+
+ dptr++; /* skip ':' */
+
+ while (dptr < limit && isdigit(*dptr)) {
+ p = p * 10 + (*dptr - '0');
+ dptr++;
+ len++;
+ if (len > 5) /* max "65535" */
+ return false;
+ }
+
+ if (len == 0)
+ return false;
+
+ /* reached limit while parsing port */
+ if (dptr >= limit)
+ return false;
+
+ if (p < 1024 || p > 65535)
+ return false;
+
+ if (port)
+ *port = htons(p);
+
+ if (endp)
+ *endp = dptr;
+
+ return true;
+}
+
/* skip ip address. returns its length. */
static int epaddr_len(const struct nf_conn *ct, const char *dptr,
const char *limit, int *shift)
@@ -193,11 +244,8 @@ static int epaddr_len(const struct nf_conn *ct, const char *dptr,
return 0;
}
- /* Port number */
- if (*dptr == ':') {
- dptr++;
- dptr += digits_len(ct, dptr, limit, shift);
- }
+ if (!sip_parse_port(dptr, &dptr, limit, NULL))
+ return 0;
return dptr - aux;
}
@@ -228,6 +276,51 @@ static int skp_epaddr_len(const struct nf_conn *ct, const char *dptr,
return epaddr_len(ct, dptr, limit, shift);
}
+/* simple_strtoul stops after first non-number character.
+ * But as we're not dealing with c-strings, we can't rely on
+ * hitting \r,\n,\0 etc. before moving past end of buffer.
+ *
+ * This is a variant of simple_strtoul, but doesn't require
+ * a c-string.
+ *
+ * If value exceeds UINT_MAX, 0 is returned.
+ */
+static unsigned int sip_strtouint(const char *cp, unsigned int len, char **endp)
+{
+ const unsigned int max = sizeof("4294967295");
+ unsigned int olen = len;
+ const char *s = cp;
+ u64 result = 0;
+
+ if (len > max)
+ len = max;
+
+ while (olen > 0 && isdigit(*s)) {
+ unsigned int value;
+
+ if (len == 0)
+ goto err;
+
+ value = *s - '0';
+ result = result * 10 + value;
+
+ if (result > UINT_MAX)
+ goto err;
+ s++;
+ len--;
+ olen--;
+ }
+
+ if (endp)
+ *endp = (char *)s;
+
+ return result;
+err:
+ if (endp)
+ *endp = (char *)cp;
+ return 0;
+}
+
/* Parse a SIP request line of the form:
*
* Request-Line = Method SP Request-URI SP SIP-Version CRLF
@@ -241,7 +334,6 @@ int ct_sip_parse_request(const struct nf_conn *ct,
{
const char *start = dptr, *limit = dptr + datalen, *end;
unsigned int mlen;
- unsigned int p;
int shift = 0;
/* Skip method and following whitespace */
@@ -267,14 +359,8 @@ int ct_sip_parse_request(const struct nf_conn *ct,
if (!sip_parse_addr(ct, dptr, &end, addr, limit, true))
return -1;
- if (end < limit && *end == ':') {
- end++;
- p = simple_strtoul(end, (char **)&end, 10);
- if (p < 1024 || p > 65535)
- return -1;
- *port = htons(p);
- } else
- *port = htons(SIP_PORT);
+ if (!sip_parse_port(end, &end, limit, port))
+ return -1;
if (end == dptr)
return 0;
@@ -509,7 +595,6 @@ int ct_sip_parse_header_uri(const struct nf_conn *ct, const char *dptr,
union nf_inet_addr *addr, __be16 *port)
{
const char *c, *limit = dptr + datalen;
- unsigned int p;
int ret;
ret = ct_sip_walk_headers(ct, dptr, dataoff ? *dataoff : 0, datalen,
@@ -520,14 +605,8 @@ int ct_sip_parse_header_uri(const struct nf_conn *ct, const char *dptr,
if (!sip_parse_addr(ct, dptr + *matchoff, &c, addr, limit, true))
return -1;
- if (*c == ':') {
- c++;
- p = simple_strtoul(c, (char **)&c, 10);
- if (p < 1024 || p > 65535)
- return -1;
- *port = htons(p);
- } else
- *port = htons(SIP_PORT);
+ if (!sip_parse_port(c, &c, limit, port))
+ return -1;
if (dataoff)
*dataoff = c - dptr;
@@ -609,7 +688,7 @@ int ct_sip_parse_numerical_param(const struct nf_conn *ct, const char *dptr,
return 0;
start += strlen(name);
- *val = simple_strtoul(start, &end, 0);
+ *val = sip_strtouint(start, limit - start, (char **)&end);
if (start == end)
return -1;
if (matchoff && matchlen) {
@@ -1065,6 +1144,8 @@ static int process_sdp(struct sk_buff *skb, unsigned int protoff,
mediaoff = sdpoff;
for (i = 0; i < ARRAY_SIZE(sdp_media_types); ) {
+ char *end;
+
if (ct_sip_get_sdp_header(ct, *dptr, mediaoff, *datalen,
SDP_HDR_MEDIA, SDP_HDR_UNSPEC,
&mediaoff, &medialen) <= 0)
@@ -1080,8 +1161,8 @@ static int process_sdp(struct sk_buff *skb, unsigned int protoff,
mediaoff += t->len;
medialen -= t->len;
- port = simple_strtoul(*dptr + mediaoff, NULL, 10);
- if (port == 0)
+ port = sip_strtouint(*dptr + mediaoff, *datalen - mediaoff, (char **)&end);
+ if (port == 0 || *dptr + mediaoff == end)
continue;
if (port < 1024 || port > 65535) {
nf_ct_helper_log(skb, ct, "wrong port %u", port);
@@ -1255,7 +1336,7 @@ static int process_register_request(struct sk_buff *skb, unsigned int protoff,
*/
if (ct_sip_get_header(ct, *dptr, 0, *datalen, SIP_HDR_EXPIRES,
&matchoff, &matchlen) > 0)
- expires = simple_strtoul(*dptr + matchoff, NULL, 10);
+ expires = sip_strtouint(*dptr + matchoff, *datalen - matchoff, NULL);
ret = ct_sip_parse_header_uri(ct, *dptr, NULL, *datalen,
SIP_HDR_CONTACT, NULL,
@@ -1286,6 +1367,10 @@ static int process_register_request(struct sk_buff *skb, unsigned int protoff,
goto store_cseq;
}
+ helper = rcu_dereference(nfct_help(ct)->helper);
+ if (!helper)
+ return NF_DROP;
+
exp = nf_ct_expect_alloc(ct);
if (!exp) {
nf_ct_helper_log(skb, ct, "cannot alloc expectation");
@@ -1296,10 +1381,6 @@ static int process_register_request(struct sk_buff *skb, unsigned int protoff,
if (sip_direct_signalling)
saddr = &ct->tuplehash[!dir].tuple.src.u3;
- helper = rcu_dereference(nfct_help(ct)->helper);
- if (!helper)
- return NF_DROP;
-
nf_ct_expect_init(exp, SIP_EXPECT_SIGNALLING, nf_ct_l3num(ct),
saddr, &daddr, proto, NULL, &port);
exp->timeout.expires = sip_timeout * HZ;
@@ -1359,7 +1440,7 @@ static int process_register_response(struct sk_buff *skb, unsigned int protoff,
if (ct_sip_get_header(ct, *dptr, 0, *datalen, SIP_HDR_EXPIRES,
&matchoff, &matchlen) > 0)
- expires = simple_strtoul(*dptr + matchoff, NULL, 10);
+ expires = sip_strtouint(*dptr + matchoff, *datalen - matchoff, NULL);
while (1) {
unsigned int c_expires = expires;
@@ -1419,10 +1500,12 @@ static int process_sip_response(struct sk_buff *skb, unsigned int protoff,
struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
unsigned int matchoff, matchlen, matchend;
unsigned int code, cseq, i;
+ char *end;
if (*datalen < strlen("SIP/2.0 200"))
return NF_ACCEPT;
- code = simple_strtoul(*dptr + strlen("SIP/2.0 "), NULL, 10);
+ code = sip_strtouint(*dptr + strlen("SIP/2.0 "),
+ *datalen - strlen("SIP/2.0 "), NULL);
if (!code) {
nf_ct_helper_log(skb, ct, "cannot get code");
return NF_DROP;
@@ -1433,8 +1516,8 @@ static int process_sip_response(struct sk_buff *skb, unsigned int protoff,
nf_ct_helper_log(skb, ct, "cannot parse cseq");
return NF_DROP;
}
- cseq = simple_strtoul(*dptr + matchoff, NULL, 10);
- if (!cseq && *(*dptr + matchoff) != '0') {
+ cseq = sip_strtouint(*dptr + matchoff, *datalen - matchoff, (char **)&end);
+ if (*dptr + matchoff == end) {
nf_ct_helper_log(skb, ct, "cannot get cseq");
return NF_DROP;
}
@@ -1483,6 +1566,7 @@ static int process_sip_request(struct sk_buff *skb, unsigned int protoff,
for (i = 0; i < ARRAY_SIZE(sip_handlers); i++) {
const struct sip_handler *handler;
+ char *end;
handler = &sip_handlers[i];
if (handler->request == NULL)
@@ -1499,8 +1583,8 @@ static int process_sip_request(struct sk_buff *skb, unsigned int protoff,
nf_ct_helper_log(skb, ct, "cannot parse cseq");
return NF_DROP;
}
- cseq = simple_strtoul(*dptr + matchoff, NULL, 10);
- if (!cseq && *(*dptr + matchoff) != '0') {
+ cseq = sip_strtouint(*dptr + matchoff, *datalen - matchoff, (char **)&end);
+ if (*dptr + matchoff == end) {
nf_ct_helper_log(skb, ct, "cannot get cseq");
return NF_DROP;
}
@@ -1576,7 +1660,7 @@ static int sip_help_tcp(struct sk_buff *skb, unsigned int protoff,
&matchoff, &matchlen) <= 0)
break;
- clen = simple_strtoul(dptr + matchoff, (char **)&end, 10);
+ clen = sip_strtouint(dptr + matchoff, datalen - matchoff, (char **)&end);
if (dptr + matchoff == end)
break;
diff --git a/net/netfilter/nf_nat_amanda.c b/net/netfilter/nf_nat_amanda.c
index 98deef6cde69..8f1054920a85 100644
--- a/net/netfilter/nf_nat_amanda.c
+++ b/net/netfilter/nf_nat_amanda.c
@@ -50,7 +50,7 @@ static unsigned int help(struct sk_buff *skb,
return NF_DROP;
}
- sprintf(buffer, "%u", port);
+ snprintf(buffer, sizeof(buffer), "%u", port);
if (!nf_nat_mangle_udp_packet(skb, exp->master, ctinfo,
protoff, matchoff, matchlen,
buffer, strlen(buffer))) {
diff --git a/net/netfilter/nf_nat_sip.c b/net/netfilter/nf_nat_sip.c
index cf4aeb299bde..9fbfc6bff0c2 100644
--- a/net/netfilter/nf_nat_sip.c
+++ b/net/netfilter/nf_nat_sip.c
@@ -68,25 +68,27 @@ static unsigned int mangle_packet(struct sk_buff *skb, unsigned int protoff,
}
static int sip_sprintf_addr(const struct nf_conn *ct, char *buffer,
+ size_t size,
const union nf_inet_addr *addr, bool delim)
{
if (nf_ct_l3num(ct) == NFPROTO_IPV4)
- return sprintf(buffer, "%pI4", &addr->ip);
+ return scnprintf(buffer, size, "%pI4", &addr->ip);
else {
if (delim)
- return sprintf(buffer, "[%pI6c]", &addr->ip6);
+ return scnprintf(buffer, size, "[%pI6c]", &addr->ip6);
else
- return sprintf(buffer, "%pI6c", &addr->ip6);
+ return scnprintf(buffer, size, "%pI6c", &addr->ip6);
}
}
static int sip_sprintf_addr_port(const struct nf_conn *ct, char *buffer,
+ size_t size,
const union nf_inet_addr *addr, u16 port)
{
if (nf_ct_l3num(ct) == NFPROTO_IPV4)
- return sprintf(buffer, "%pI4:%u", &addr->ip, port);
+ return scnprintf(buffer, size, "%pI4:%u", &addr->ip, port);
else
- return sprintf(buffer, "[%pI6c]:%u", &addr->ip6, port);
+ return scnprintf(buffer, size, "[%pI6c]:%u", &addr->ip6, port);
}
static int map_addr(struct sk_buff *skb, unsigned int protoff,
@@ -119,7 +121,7 @@ static int map_addr(struct sk_buff *skb, unsigned int protoff,
if (nf_inet_addr_cmp(&newaddr, addr) && newport == port)
return 1;
- buflen = sip_sprintf_addr_port(ct, buffer, &newaddr, ntohs(newport));
+ buflen = sip_sprintf_addr_port(ct, buffer, sizeof(buffer), &newaddr, ntohs(newport));
return mangle_packet(skb, protoff, dataoff, dptr, datalen,
matchoff, matchlen, buffer, buflen);
}
@@ -212,7 +214,7 @@ static unsigned int nf_nat_sip(struct sk_buff *skb, unsigned int protoff,
&addr, true) > 0 &&
nf_inet_addr_cmp(&addr, &ct->tuplehash[dir].tuple.src.u3) &&
!nf_inet_addr_cmp(&addr, &ct->tuplehash[!dir].tuple.dst.u3)) {
- buflen = sip_sprintf_addr(ct, buffer,
+ buflen = sip_sprintf_addr(ct, buffer, sizeof(buffer),
&ct->tuplehash[!dir].tuple.dst.u3,
true);
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
@@ -229,7 +231,7 @@ static unsigned int nf_nat_sip(struct sk_buff *skb, unsigned int protoff,
&addr, false) > 0 &&
nf_inet_addr_cmp(&addr, &ct->tuplehash[dir].tuple.dst.u3) &&
!nf_inet_addr_cmp(&addr, &ct->tuplehash[!dir].tuple.src.u3)) {
- buflen = sip_sprintf_addr(ct, buffer,
+ buflen = sip_sprintf_addr(ct, buffer, sizeof(buffer),
&ct->tuplehash[!dir].tuple.src.u3,
false);
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
@@ -244,10 +246,11 @@ static unsigned int nf_nat_sip(struct sk_buff *skb, unsigned int protoff,
if (ct_sip_parse_numerical_param(ct, *dptr, matchend, *datalen,
"rport=", &poff, &plen,
&n) > 0 &&
+ n >= 1024 && n <= 65535 &&
htons(n) == ct->tuplehash[dir].tuple.dst.u.udp.port &&
htons(n) != ct->tuplehash[!dir].tuple.src.u.udp.port) {
__be16 p = ct->tuplehash[!dir].tuple.src.u.udp.port;
- buflen = sprintf(buffer, "%u", ntohs(p));
+ buflen = scnprintf(buffer, sizeof(buffer), "%u", ntohs(p));
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
poff, plen, buffer, buflen)) {
nf_ct_helper_log(skb, ct, "cannot mangle rport");
@@ -418,7 +421,8 @@ static unsigned int nf_nat_sip_expect(struct sk_buff *skb, unsigned int protoff,
if (!nf_inet_addr_cmp(&exp->tuple.dst.u3, &exp->saved_addr) ||
exp->tuple.dst.u.udp.port != exp->saved_proto.udp.port) {
- buflen = sip_sprintf_addr_port(ct, buffer, &newaddr, port);
+ buflen = sip_sprintf_addr_port(ct, buffer, sizeof(buffer),
+ &newaddr, port);
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
matchoff, matchlen, buffer, buflen)) {
nf_ct_helper_log(skb, ct, "cannot mangle packet");
@@ -438,8 +442,8 @@ static int mangle_content_len(struct sk_buff *skb, unsigned int protoff,
{
enum ip_conntrack_info ctinfo;
struct nf_conn *ct = nf_ct_get(skb, &ctinfo);
+ char buffer[sizeof("4294967295")];
unsigned int matchoff, matchlen;
- char buffer[sizeof("65536")];
int buflen, c_len;
/* Get actual SDP length */
@@ -454,7 +458,7 @@ static int mangle_content_len(struct sk_buff *skb, unsigned int protoff,
&matchoff, &matchlen) <= 0)
return 0;
- buflen = sprintf(buffer, "%u", c_len);
+ buflen = scnprintf(buffer, sizeof(buffer), "%u", c_len);
return mangle_packet(skb, protoff, dataoff, dptr, datalen,
matchoff, matchlen, buffer, buflen);
}
@@ -491,7 +495,7 @@ static unsigned int nf_nat_sdp_addr(struct sk_buff *skb, unsigned int protoff,
char buffer[INET6_ADDRSTRLEN];
unsigned int buflen;
- buflen = sip_sprintf_addr(ct, buffer, addr, false);
+ buflen = sip_sprintf_addr(ct, buffer, sizeof(buffer), addr, false);
if (mangle_sdp_packet(skb, protoff, dataoff, dptr, datalen,
sdpoff, type, term, buffer, buflen))
return 0;
@@ -509,7 +513,7 @@ static unsigned int nf_nat_sdp_port(struct sk_buff *skb, unsigned int protoff,
char buffer[sizeof("nnnnn")];
unsigned int buflen;
- buflen = sprintf(buffer, "%u", port);
+ buflen = scnprintf(buffer, sizeof(buffer), "%u", port);
if (!mangle_packet(skb, protoff, dataoff, dptr, datalen,
matchoff, matchlen, buffer, buflen))
return 0;
@@ -529,7 +533,7 @@ static unsigned int nf_nat_sdp_session(struct sk_buff *skb, unsigned int protoff
unsigned int buflen;
/* Mangle session description owner and contact addresses */
- buflen = sip_sprintf_addr(ct, buffer, addr, false);
+ buflen = sip_sprintf_addr(ct, buffer, sizeof(buffer), addr, false);
if (mangle_sdp_packet(skb, protoff, dataoff, dptr, datalen, sdpoff,
SDP_HDR_OWNER, SDP_HDR_MEDIA, buffer, buflen))
return 0;
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index e373afdf0f07..838c9f49e4e0 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -6981,6 +6981,7 @@ static int nft_add_set_elem(struct nft_ctx *ctx, struct nft_set *set,
enum nft_registers dreg;
struct nft_trans *trans;
u8 update_flags;
+ bool set_full = false;
u64 expiration;
u64 timeout;
int err, i;
@@ -7267,10 +7268,18 @@ static int nft_add_set_elem(struct nft_ctx *ctx, struct nft_set *set,
if (err < 0)
goto err_elem_free;
+ if (!(flags & NFT_SET_ELEM_CATCHALL)) {
+ unsigned int max = nft_set_maxsize(set), nelems;
+
+ nelems = atomic_inc_return(&set->nelems);
+ if (nelems > max)
+ set_full = true;
+ }
+
trans = nft_trans_elem_alloc(ctx, NFT_MSG_NEWSETELEM, set);
if (trans == NULL) {
err = -ENOMEM;
- goto err_elem_free;
+ goto err_set_size;
}
ext->genmask = nft_genmask_cur(ctx->net);
@@ -7312,7 +7321,7 @@ static int nft_add_set_elem(struct nft_ctx *ctx, struct nft_set *set,
nft_trans_elem_priv(trans) = elem_priv;
nft_trans_elem_update_flags(trans) = update_flags;
nft_trans_commit_list_add_tail(ctx->net, trans);
- goto err_elem_free;
+ goto err_set_size;
}
}
}
@@ -7330,23 +7339,16 @@ static int nft_add_set_elem(struct nft_ctx *ctx, struct nft_set *set,
goto err_element_clash;
}
- if (!(flags & NFT_SET_ELEM_CATCHALL)) {
- unsigned int max = nft_set_maxsize(set);
-
- if (!atomic_add_unless(&set->nelems, 1, max)) {
- err = -ENFILE;
- goto err_set_full;
- }
- }
-
nft_trans_elem_priv(trans) = elem.priv;
nft_trans_commit_list_add_tail(ctx->net, trans);
- return 0;
-err_set_full:
- nft_setelem_remove(ctx->net, set, elem.priv);
+ return set_full ? -ENFILE : 0;
+
err_element_clash:
kfree(trans);
+err_set_size:
+ if (!(flags & NFT_SET_ELEM_CATCHALL))
+ atomic_dec(&set->nelems);
err_elem_free:
nf_tables_set_elem_destroy(ctx, set, elem.priv);
err_parse_data:
diff --git a/net/netfilter/nfnetlink_osf.c b/net/netfilter/nfnetlink_osf.c
index 9fc9544d4bc5..c89efb951994 100644
--- a/net/netfilter/nfnetlink_osf.c
+++ b/net/netfilter/nfnetlink_osf.c
@@ -31,26 +31,18 @@ EXPORT_SYMBOL_GPL(nf_osf_fingers);
static inline int nf_osf_ttl(const struct sk_buff *skb,
int ttl_check, unsigned char f_ttl)
{
- struct in_device *in_dev = __in_dev_get_rcu(skb->dev);
const struct iphdr *ip = ip_hdr(skb);
- const struct in_ifaddr *ifa;
- int ret = 0;
- if (ttl_check == NF_OSF_TTL_TRUE)
+ switch (ttl_check) {
+ case NF_OSF_TTL_TRUE:
return ip->ttl == f_ttl;
- if (ttl_check == NF_OSF_TTL_NOCHECK)
- return 1;
- else if (ip->ttl <= f_ttl)
+ break;
+ case NF_OSF_TTL_NOCHECK:
return 1;
-
- in_dev_for_each_ifa_rcu(ifa, in_dev) {
- if (inet_ifa_match(ip->saddr, ifa)) {
- ret = (ip->ttl == f_ttl);
- break;
- }
+ case NF_OSF_TTL_LESS:
+ default:
+ return ip->ttl <= f_ttl;
}
-
- return ret;
}
struct nf_osf_hdr_ctx {
@@ -64,9 +56,9 @@ struct nf_osf_hdr_ctx {
static bool nf_osf_match_one(const struct sk_buff *skb,
const struct nf_osf_user_finger *f,
int ttl_check,
- struct nf_osf_hdr_ctx *ctx)
+ const struct nf_osf_hdr_ctx *ctx)
{
- const __u8 *optpinit = ctx->optp;
+ const __u8 *optp = ctx->optp;
unsigned int check_WSS = 0;
int fmatch = FMATCH_WRONG;
int foptsize, optnum;
@@ -95,17 +87,17 @@ static bool nf_osf_match_one(const struct sk_buff *skb,
check_WSS = f->wss.wc;
for (optnum = 0; optnum < f->opt_num; ++optnum) {
- if (f->opt[optnum].kind == *ctx->optp) {
+ if (f->opt[optnum].kind == *optp) {
__u32 len = f->opt[optnum].length;
- const __u8 *optend = ctx->optp + len;
+ const __u8 *optend = optp + len;
fmatch = FMATCH_OK;
- switch (*ctx->optp) {
+ switch (*optp) {
case OSFOPT_MSS:
- mss = ctx->optp[3];
+ mss = optp[3];
mss <<= 8;
- mss |= ctx->optp[2];
+ mss |= optp[2];
mss = ntohs((__force __be16)mss);
break;
@@ -113,7 +105,7 @@ static bool nf_osf_match_one(const struct sk_buff *skb,
break;
}
- ctx->optp = optend;
+ optp = optend;
} else
fmatch = FMATCH_OPT_WRONG;
@@ -156,9 +148,6 @@ static bool nf_osf_match_one(const struct sk_buff *skb,
}
}
- if (fmatch != FMATCH_OK)
- ctx->optp = optpinit;
-
return fmatch == FMATCH_OK;
}
@@ -320,6 +309,10 @@ static int nfnl_osf_add_callback(struct sk_buff *skb,
if (f->opt_num > ARRAY_SIZE(f->opt))
return -EINVAL;
+ if (f->wss.wc >= OSF_WSS_MAX ||
+ (f->wss.wc == OSF_WSS_MODULO && f->wss.val == 0))
+ return -EINVAL;
+
for (i = 0; i < f->opt_num; i++) {
if (!f->opt[i].length || f->opt[i].length > MAX_IPOPTLEN)
return -EINVAL;
diff --git a/net/netfilter/nft_ct.c b/net/netfilter/nft_ct.c
index 134f4d3d5b22..5310c3dca832 100644
--- a/net/netfilter/nft_ct.c
+++ b/net/netfilter/nft_ct.c
@@ -1363,6 +1363,8 @@ static void nft_ct_expect_obj_eval(struct nft_object *obj,
if (nf_ct_expect_related(exp, 0) != 0)
regs->verdict.code = NF_DROP;
+
+ nf_ct_expect_put(exp);
}
static const struct nla_policy nft_ct_expect_policy[NFTA_CT_EXPECT_MAX + 1] = {
diff --git a/net/netfilter/nft_fwd_netdev.c b/net/netfilter/nft_fwd_netdev.c
index 152a9fb4d23a..256e832f1bb9 100644
--- a/net/netfilter/nft_fwd_netdev.c
+++ b/net/netfilter/nft_fwd_netdev.c
@@ -116,6 +116,11 @@ static void nft_fwd_neigh_eval(const struct nft_expr *expr,
goto out;
}
iph = ip_hdr(skb);
+ if (iph->ttl <= 1) {
+ verdict = NF_DROP;
+ goto out;
+ }
+
ip_decrease_ttl(iph);
neigh_table = NEIGH_ARP_TABLE;
break;
@@ -132,6 +137,11 @@ static void nft_fwd_neigh_eval(const struct nft_expr *expr,
goto out;
}
ip6h = ipv6_hdr(skb);
+ if (ip6h->hop_limit <= 1) {
+ verdict = NF_DROP;
+ goto out;
+ }
+
ip6h->hop_limit--;
neigh_table = NEIGH_ND_TABLE;
break;
diff --git a/net/netfilter/nft_osf.c b/net/netfilter/nft_osf.c
index 1c0b493ef0a9..bdc2f6c90e2f 100644
--- a/net/netfilter/nft_osf.c
+++ b/net/netfilter/nft_osf.c
@@ -28,6 +28,11 @@ static void nft_osf_eval(const struct nft_expr *expr, struct nft_regs *regs,
struct nf_osf_data data;
struct tcphdr _tcph;
+ if (nft_pf(pkt) != NFPROTO_IPV4) {
+ regs->verdict.code = NFT_BREAK;
+ return;
+ }
+
if (pkt->tprot != IPPROTO_TCP) {
regs->verdict.code = NFT_BREAK;
return;
@@ -114,7 +119,6 @@ static int nft_osf_validate(const struct nft_ctx *ctx,
switch (ctx->family) {
case NFPROTO_IPV4:
- case NFPROTO_IPV6:
case NFPROTO_INET:
hooks = (1 << NF_INET_LOCAL_IN) |
(1 << NF_INET_PRE_ROUTING) |
diff --git a/net/netfilter/xt_mac.c b/net/netfilter/xt_mac.c
index 81649da57ba5..bd2354760895 100644
--- a/net/netfilter/xt_mac.c
+++ b/net/netfilter/xt_mac.c
@@ -38,25 +38,37 @@ static bool mac_mt(const struct sk_buff *skb, struct xt_action_param *par)
return ret;
}
-static struct xt_match mac_mt_reg __read_mostly = {
- .name = "mac",
- .revision = 0,
- .family = NFPROTO_UNSPEC,
- .match = mac_mt,
- .matchsize = sizeof(struct xt_mac_info),
- .hooks = (1 << NF_INET_PRE_ROUTING) | (1 << NF_INET_LOCAL_IN) |
- (1 << NF_INET_FORWARD),
- .me = THIS_MODULE,
+static struct xt_match mac_mt_reg[] __read_mostly = {
+ {
+ .name = "mac",
+ .family = NFPROTO_IPV4,
+ .match = mac_mt,
+ .matchsize = sizeof(struct xt_mac_info),
+ .hooks = (1 << NF_INET_PRE_ROUTING) |
+ (1 << NF_INET_LOCAL_IN) |
+ (1 << NF_INET_FORWARD),
+ .me = THIS_MODULE,
+ },
+ {
+ .name = "mac",
+ .family = NFPROTO_IPV6,
+ .match = mac_mt,
+ .matchsize = sizeof(struct xt_mac_info),
+ .hooks = (1 << NF_INET_PRE_ROUTING) |
+ (1 << NF_INET_LOCAL_IN) |
+ (1 << NF_INET_FORWARD),
+ .me = THIS_MODULE,
+ },
};
static int __init mac_mt_init(void)
{
- return xt_register_match(&mac_mt_reg);
+ return xt_register_matches(mac_mt_reg, ARRAY_SIZE(mac_mt_reg));
}
static void __exit mac_mt_exit(void)
{
- xt_unregister_match(&mac_mt_reg);
+ xt_unregister_matches(mac_mt_reg, ARRAY_SIZE(mac_mt_reg));
}
module_init(mac_mt_init);
diff --git a/net/netfilter/xt_owner.c b/net/netfilter/xt_owner.c
index 50332888c8d2..7be2fe22b067 100644
--- a/net/netfilter/xt_owner.c
+++ b/net/netfilter/xt_owner.c
@@ -127,26 +127,39 @@ owner_mt(const struct sk_buff *skb, struct xt_action_param *par)
return true;
}
-static struct xt_match owner_mt_reg __read_mostly = {
- .name = "owner",
- .revision = 1,
- .family = NFPROTO_UNSPEC,
- .checkentry = owner_check,
- .match = owner_mt,
- .matchsize = sizeof(struct xt_owner_match_info),
- .hooks = (1 << NF_INET_LOCAL_OUT) |
- (1 << NF_INET_POST_ROUTING),
- .me = THIS_MODULE,
+static struct xt_match owner_mt_reg[] __read_mostly = {
+ {
+ .name = "owner",
+ .revision = 1,
+ .family = NFPROTO_IPV4,
+ .checkentry = owner_check,
+ .match = owner_mt,
+ .matchsize = sizeof(struct xt_owner_match_info),
+ .hooks = (1 << NF_INET_LOCAL_OUT) |
+ (1 << NF_INET_POST_ROUTING),
+ .me = THIS_MODULE,
+ },
+ {
+ .name = "owner",
+ .revision = 1,
+ .family = NFPROTO_IPV6,
+ .checkentry = owner_check,
+ .match = owner_mt,
+ .matchsize = sizeof(struct xt_owner_match_info),
+ .hooks = (1 << NF_INET_LOCAL_OUT) |
+ (1 << NF_INET_POST_ROUTING),
+ .me = THIS_MODULE,
+ }
};
static int __init owner_mt_init(void)
{
- return xt_register_match(&owner_mt_reg);
+ return xt_register_matches(owner_mt_reg, ARRAY_SIZE(owner_mt_reg));
}
static void __exit owner_mt_exit(void)
{
- xt_unregister_match(&owner_mt_reg);
+ xt_unregister_matches(owner_mt_reg, ARRAY_SIZE(owner_mt_reg));
}
module_init(owner_mt_init);
diff --git a/net/netfilter/xt_physdev.c b/net/netfilter/xt_physdev.c
index 343e65f377d4..130842c35c6f 100644
--- a/net/netfilter/xt_physdev.c
+++ b/net/netfilter/xt_physdev.c
@@ -115,24 +115,33 @@ static int physdev_mt_check(const struct xt_mtchk_param *par)
return 0;
}
-static struct xt_match physdev_mt_reg __read_mostly = {
- .name = "physdev",
- .revision = 0,
- .family = NFPROTO_UNSPEC,
- .checkentry = physdev_mt_check,
- .match = physdev_mt,
- .matchsize = sizeof(struct xt_physdev_info),
- .me = THIS_MODULE,
+static struct xt_match physdev_mt_reg[] __read_mostly = {
+ {
+ .name = "physdev",
+ .family = NFPROTO_IPV4,
+ .checkentry = physdev_mt_check,
+ .match = physdev_mt,
+ .matchsize = sizeof(struct xt_physdev_info),
+ .me = THIS_MODULE,
+ },
+ {
+ .name = "physdev",
+ .family = NFPROTO_IPV6,
+ .checkentry = physdev_mt_check,
+ .match = physdev_mt,
+ .matchsize = sizeof(struct xt_physdev_info),
+ .me = THIS_MODULE,
+ },
};
static int __init physdev_mt_init(void)
{
- return xt_register_match(&physdev_mt_reg);
+ return xt_register_matches(physdev_mt_reg, ARRAY_SIZE(physdev_mt_reg));
}
static void __exit physdev_mt_exit(void)
{
- xt_unregister_match(&physdev_mt_reg);
+ xt_unregister_matches(physdev_mt_reg, ARRAY_SIZE(physdev_mt_reg));
}
module_init(physdev_mt_init);
diff --git a/net/netfilter/xt_policy.c b/net/netfilter/xt_policy.c
index cb6e8279010a..b5fa65558318 100644
--- a/net/netfilter/xt_policy.c
+++ b/net/netfilter/xt_policy.c
@@ -63,7 +63,7 @@ match_policy_in(const struct sk_buff *skb, const struct xt_policy_info *info,
return 0;
for (i = sp->len - 1; i >= 0; i--) {
- pos = strict ? i - sp->len + 1 : 0;
+ pos = strict ? sp->len - i - 1 : 0;
if (pos >= info->len)
return 0;
e = &info->pol[pos];
diff --git a/net/netfilter/xt_realm.c b/net/netfilter/xt_realm.c
index 6df485f4403d..61b2f1e58d15 100644
--- a/net/netfilter/xt_realm.c
+++ b/net/netfilter/xt_realm.c
@@ -33,7 +33,7 @@ static struct xt_match realm_mt_reg __read_mostly = {
.matchsize = sizeof(struct xt_realm_info),
.hooks = (1 << NF_INET_POST_ROUTING) | (1 << NF_INET_FORWARD) |
(1 << NF_INET_LOCAL_OUT) | (1 << NF_INET_LOCAL_IN),
- .family = NFPROTO_UNSPEC,
+ .family = NFPROTO_IPV4,
.me = THIS_MODULE
};
diff --git a/net/netfilter/xt_socket.c b/net/netfilter/xt_socket.c
index 76e01f292aaf..811e53bee408 100644
--- a/net/netfilter/xt_socket.c
+++ b/net/netfilter/xt_socket.c
@@ -168,52 +168,41 @@ static int socket_mt_enable_defrag(struct net *net, int family)
static int socket_mt_v1_check(const struct xt_mtchk_param *par)
{
const struct xt_socket_mtinfo1 *info = (struct xt_socket_mtinfo1 *) par->matchinfo;
- int err;
-
- err = socket_mt_enable_defrag(par->net, par->family);
- if (err)
- return err;
if (info->flags & ~XT_SOCKET_FLAGS_V1) {
pr_info_ratelimited("unknown flags 0x%x\n",
info->flags & ~XT_SOCKET_FLAGS_V1);
return -EINVAL;
}
- return 0;
+
+ return socket_mt_enable_defrag(par->net, par->family);
}
static int socket_mt_v2_check(const struct xt_mtchk_param *par)
{
const struct xt_socket_mtinfo2 *info = (struct xt_socket_mtinfo2 *) par->matchinfo;
- int err;
-
- err = socket_mt_enable_defrag(par->net, par->family);
- if (err)
- return err;
if (info->flags & ~XT_SOCKET_FLAGS_V2) {
pr_info_ratelimited("unknown flags 0x%x\n",
info->flags & ~XT_SOCKET_FLAGS_V2);
return -EINVAL;
}
- return 0;
+
+ return socket_mt_enable_defrag(par->net, par->family);
}
static int socket_mt_v3_check(const struct xt_mtchk_param *par)
{
const struct xt_socket_mtinfo3 *info =
(struct xt_socket_mtinfo3 *)par->matchinfo;
- int err;
- err = socket_mt_enable_defrag(par->net, par->family);
- if (err)
- return err;
if (info->flags & ~XT_SOCKET_FLAGS_V3) {
pr_info_ratelimited("unknown flags 0x%x\n",
info->flags & ~XT_SOCKET_FLAGS_V3);
return -EINVAL;
}
- return 0;
+
+ return socket_mt_enable_defrag(par->net, par->family);
}
static void socket_mt_destroy(const struct xt_mtdtor_param *par)
diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c
index 8d3c01f0e2aa..607b5ca70ea5 100644
--- a/net/openvswitch/datapath.c
+++ b/net/openvswitch/datapath.c
@@ -2157,9 +2157,40 @@ static int ovs_vport_cmd_fill_info(struct vport *vport, struct sk_buff *skb,
return err;
}
+static size_t ovs_vport_cmd_msg_size(void)
+{
+ size_t msgsize = NLMSG_ALIGN(sizeof(struct ovs_header));
+
+ msgsize += nla_total_size(sizeof(u32)); /* OVS_VPORT_ATTR_PORT_NO */
+ msgsize += nla_total_size(sizeof(u32)); /* OVS_VPORT_ATTR_TYPE */
+ msgsize += nla_total_size(IFNAMSIZ); /* OVS_VPORT_ATTR_NAME */
+ msgsize += nla_total_size(sizeof(u32)); /* OVS_VPORT_ATTR_IFINDEX */
+ msgsize += nla_total_size(sizeof(s32)); /* OVS_VPORT_ATTR_NETNSID */
+
+ /* OVS_VPORT_ATTR_STATS */
+ msgsize += nla_total_size_64bit(sizeof(struct ovs_vport_stats));
+
+ /* OVS_VPORT_ATTR_UPCALL_STATS(OVS_VPORT_UPCALL_ATTR_SUCCESS +
+ * OVS_VPORT_UPCALL_ATTR_FAIL)
+ */
+ msgsize += nla_total_size(nla_total_size_64bit(sizeof(u64)) +
+ nla_total_size_64bit(sizeof(u64)));
+
+ /* OVS_VPORT_ATTR_UPCALL_PID */
+ msgsize += nla_total_size(nr_cpu_ids * sizeof(u32));
+
+ /* OVS_VPORT_ATTR_OPTIONS(OVS_TUNNEL_ATTR_DST_PORT +
+ * OVS_TUNNEL_ATTR_EXTENSION(OVS_VXLAN_EXT_GBP))
+ */
+ msgsize += nla_total_size(nla_total_size(sizeof(u16)) +
+ nla_total_size(nla_total_size(0)));
+
+ return msgsize;
+}
+
static struct sk_buff *ovs_vport_cmd_alloc_info(void)
{
- return nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ return genlmsg_new(ovs_vport_cmd_msg_size(), GFP_KERNEL);
}
/* Called with ovs_mutex, only via ovs_dp_notify_wq(). */
@@ -2169,7 +2200,7 @@ struct sk_buff *ovs_vport_cmd_build_info(struct vport *vport, struct net *net,
struct sk_buff *skb;
int retval;
- skb = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ skb = ovs_vport_cmd_alloc_info();
if (!skb)
return ERR_PTR(-ENOMEM);
diff --git a/net/openvswitch/vport.c b/net/openvswitch/vport.c
index 2a996858a914..469bc1fda726 100644
--- a/net/openvswitch/vport.c
+++ b/net/openvswitch/vport.c
@@ -407,6 +407,9 @@ int ovs_vport_set_upcall_portids(struct vport *vport, const struct nlattr *ids)
if (!nla_len(ids) || nla_len(ids) % sizeof(u32))
return -EINVAL;
+ if (nla_len(ids) / sizeof(u32) > nr_cpu_ids)
+ return -EINVAL;
+
old = ovsl_dereference(vport->upcall_portids);
vport_portids = kmalloc(sizeof(*vport_portids) + nla_len(ids),
diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c
index 8435a20968ef..f0840169d5e3 100644
--- a/net/rds/af_rds.c
+++ b/net/rds/af_rds.c
@@ -357,7 +357,8 @@ static int rds_cong_monitor(struct rds_sock *rs, sockptr_t optval, int optlen)
return ret;
}
-static int rds_set_transport(struct rds_sock *rs, sockptr_t optval, int optlen)
+static int rds_set_transport(struct net *net, struct rds_sock *rs,
+ sockptr_t optval, int optlen)
{
int t_type;
@@ -373,6 +374,10 @@ static int rds_set_transport(struct rds_sock *rs, sockptr_t optval, int optlen)
if (t_type < 0 || t_type >= RDS_TRANS_COUNT)
return -EINVAL;
+ /* RDS/IB is restricted to the initial network namespace */
+ if (t_type != RDS_TRANS_TCP && !net_eq(net, &init_net))
+ return -EPROTOTYPE;
+
rs->rs_transport = rds_trans_get(t_type);
return rs->rs_transport ? 0 : -ENOPROTOOPT;
@@ -433,6 +438,7 @@ static int rds_setsockopt(struct socket *sock, int level, int optname,
sockptr_t optval, unsigned int optlen)
{
struct rds_sock *rs = rds_sk_to_rs(sock->sk);
+ struct net *net = sock_net(sock->sk);
int ret;
if (level != SOL_RDS) {
@@ -461,7 +467,7 @@ static int rds_setsockopt(struct socket *sock, int level, int optname,
break;
case SO_RDS_TRANSPORT:
lock_sock(sock->sk);
- ret = rds_set_transport(rs, optval, optlen);
+ ret = rds_set_transport(net, rs, optval, optlen);
release_sock(sock->sk);
break;
case SO_TIMESTAMP_OLD:
diff --git a/net/rds/connection.c b/net/rds/connection.c
index 3a1b548dcdcb..d6ee386fd820 100644
--- a/net/rds/connection.c
+++ b/net/rds/connection.c
@@ -673,6 +673,13 @@ void rds_for_each_conn_info(struct socket *sock, unsigned int len,
i++, head++) {
hlist_for_each_entry_rcu(conn, head, c_hash_node) {
+ /* Zero the per-item buffer before handing it to the
+ * visitor so any field the visitor does not write -
+ * including implicit alignment padding - cannot leak
+ * stack contents to user space via rds_info_copy().
+ */
+ memset(buffer, 0, item_len);
+
/* XXX no c_lock usage.. */
if (!visitor(conn, buffer))
continue;
@@ -722,6 +729,13 @@ static void rds_walk_conn_path_info(struct socket *sock, unsigned int len,
*/
cp = conn->c_path;
+ /* Zero the per-item buffer for the same reason as
+ * rds_for_each_conn_info(): any byte the visitor
+ * does not write (including alignment padding) must
+ * not leak stack contents via rds_info_copy().
+ */
+ memset(buffer, 0, item_len);
+
/* XXX no cp_lock usage.. */
if (!visitor(cp, buffer))
continue;
diff --git a/net/rds/ib.c b/net/rds/ib.c
index 9826fe7f9d00..ce5be43c5fba 100644
--- a/net/rds/ib.c
+++ b/net/rds/ib.c
@@ -403,8 +403,8 @@ static void rds6_ib_ic_info(struct socket *sock, unsigned int len,
* allowed to influence which paths have priority. We could call userspace
* asserting this policy "routing".
*/
-static int rds_ib_laddr_check(struct net *net, const struct in6_addr *addr,
- __u32 scope_id)
+static int rds_ib_laddr_check_cm(struct net *net, const struct in6_addr *addr,
+ __u32 scope_id)
{
int ret;
struct rdma_cm_id *cm_id;
@@ -489,6 +489,26 @@ static int rds_ib_laddr_check(struct net *net, const struct in6_addr *addr,
return ret;
}
+static int rds_ib_laddr_check(struct net *net, const struct in6_addr *addr,
+ __u32 scope_id)
+{
+ struct rds_ib_device *rds_ibdev = NULL;
+
+ /* RDS/IB is restricted to the initial network namespace */
+ if (!net_eq(net, &init_net))
+ return -EPROTOTYPE;
+
+ if (ipv6_addr_v4mapped(addr)) {
+ rds_ibdev = rds_ib_get_device(addr->s6_addr32[3]);
+ if (rds_ibdev) {
+ rds_ib_dev_put(rds_ibdev);
+ return 0;
+ }
+ }
+
+ return rds_ib_laddr_check_cm(net, addr, scope_id);
+}
+
static void rds_ib_unregister_client(void)
{
ib_unregister_client(&rds_ib_client);
diff --git a/net/rds/ib.h b/net/rds/ib.h
index 8ef3178ed4d6..5ff346a1e8ba 100644
--- a/net/rds/ib.h
+++ b/net/rds/ib.h
@@ -381,6 +381,7 @@ void rds_ib_cm_connect_complete(struct rds_connection *conn,
__rds_ib_conn_error(conn, KERN_WARNING "RDS/IB: " fmt)
/* ib_rdma.c */
+struct rds_ib_device *rds_ib_get_device(__be32 ipaddr);
int rds_ib_update_ipaddr(struct rds_ib_device *rds_ibdev,
struct in6_addr *ipaddr);
void rds_ib_add_conn(struct rds_ib_device *rds_ibdev, struct rds_connection *conn);
diff --git a/net/rds/ib_rdma.c b/net/rds/ib_rdma.c
index 30fca2169aa7..468fd60d818f 100644
--- a/net/rds/ib_rdma.c
+++ b/net/rds/ib_rdma.c
@@ -47,7 +47,7 @@ struct rds_ib_dereg_odp_mr {
static void rds_ib_odp_mr_worker(struct work_struct *work);
-static struct rds_ib_device *rds_ib_get_device(__be32 ipaddr)
+struct rds_ib_device *rds_ib_get_device(__be32 ipaddr)
{
struct rds_ib_device *rds_ibdev;
struct rds_ib_ipaddr *i_ipaddr;
diff --git a/net/rds/message.c b/net/rds/message.c
index 921d89973b93..9824e79e057f 100644
--- a/net/rds/message.c
+++ b/net/rds/message.c
@@ -408,6 +408,7 @@ static int rds_message_zcopy_from_user(struct rds_message *rm, struct iov_iter *
for (i = 0; i < rm->data.op_nents; i++)
put_page(sg_page(&rm->data.op_sg[i]));
+ rm->data.op_nents = 0;
mmp = &rm->data.op_mmp_znotifier->z_mmp;
mm_unaccount_pinned_pages(mmp);
ret = -EFAULT;
diff --git a/net/sched/act_ct.c b/net/sched/act_ct.c
index 945b64be4c1f..c82755749211 100644
--- a/net/sched/act_ct.c
+++ b/net/sched/act_ct.c
@@ -326,9 +326,13 @@ static int tcf_ct_flow_table_get(struct net *net, struct tcf_ct_params *params)
int err = -ENOMEM;
mutex_lock(&zones_mutex);
- ct_ft = rhashtable_lookup_fast(&zones_ht, &key, zones_params);
- if (ct_ft && refcount_inc_not_zero(&ct_ft->ref))
+ rcu_read_lock();
+ ct_ft = rhashtable_lookup(&zones_ht, &key, zones_params);
+ if (ct_ft && refcount_inc_not_zero(&ct_ft->ref)) {
+ rcu_read_unlock();
goto out_unlock;
+ }
+ rcu_read_unlock();
ct_ft = kzalloc(sizeof(*ct_ft), GFP_KERNEL);
if (!ct_ft)
diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
index 5b3814365924..b1b0049d7a0e 100644
--- a/net/sched/act_mirred.c
+++ b/net/sched/act_mirred.c
@@ -348,7 +348,7 @@ static int tcf_blockcast_redir(struct sk_buff *skb, struct tcf_mirred *m,
goto assign_prev;
tcf_mirred_to_dev(skb, m, dev_prev,
- dev_is_mac_header_xmit(dev),
+ dev_is_mac_header_xmit(dev_prev),
mirred_eaction, retval);
assign_prev:
dev_prev = dev;
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 8024b6503cd9..ba1a5c15e6ca 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -603,7 +603,7 @@ static bool cake_update_flowkeys(struct flow_keys *keys,
}
port = rev ? tuple.src.u.all : tuple.dst.u.all;
if (port != keys->ports.dst) {
- port = keys->ports.dst;
+ keys->ports.dst = port;
upd = true;
}
}
@@ -2297,10 +2297,11 @@ static void cake_set_rate(struct cake_tin_data *b, u64 rate, u32 mtu,
byte_target_ns = (byte_target * rate_ns) >> rate_shft;
- b->cparams.target = max((byte_target_ns * 3) / 2, target_ns);
- b->cparams.interval = max(rtt_est_ns +
- b->cparams.target - target_ns,
- b->cparams.target * 2);
+ WRITE_ONCE(b->cparams.target,
+ max((byte_target_ns * 3) / 2, target_ns));
+ WRITE_ONCE(b->cparams.interval,
+ max(rtt_est_ns + b->cparams.target - target_ns,
+ b->cparams.target * 2));
b->cparams.mtu_time = byte_target_ns;
b->cparams.p_inc = 1 << 24; /* 1/256 */
b->cparams.p_dec = 1 << 20; /* 1/4096 */
@@ -2930,9 +2931,9 @@ static int cake_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
PUT_TSTAT_U32(BACKLOG_BYTES, b->tin_backlog);
PUT_TSTAT_U32(TARGET_US,
- ktime_to_us(ns_to_ktime(b->cparams.target)));
+ ktime_to_us(ns_to_ktime(READ_ONCE(b->cparams.target))));
PUT_TSTAT_U32(INTERVAL_US,
- ktime_to_us(ns_to_ktime(b->cparams.interval)));
+ ktime_to_us(ns_to_ktime(READ_ONCE(b->cparams.interval))));
PUT_TSTAT_U32(SENT_PACKETS, b->packets);
PUT_TSTAT_U32(DROPPED_PACKETS, b->tin_dropped);
diff --git a/net/sched/sch_choke.c b/net/sched/sch_choke.c
index 757b89292e7e..87d92b2a7444 100644
--- a/net/sched/sch_choke.c
+++ b/net/sched/sch_choke.c
@@ -229,7 +229,7 @@ static int choke_enqueue(struct sk_buff *skb, struct Qdisc *sch,
/* Draw a packet at random from queue and compare flow */
if (choke_match_random(q, skb, &idx)) {
- q->stats.matched++;
+ WRITE_ONCE(q->stats.matched, q->stats.matched + 1);
choke_drop_by_idx(sch, idx, to_free);
goto congestion_drop;
}
@@ -241,11 +241,13 @@ static int choke_enqueue(struct sk_buff *skb, struct Qdisc *sch,
qdisc_qstats_overlimit(sch);
if (use_harddrop(q) || !use_ecn(q) ||
!INET_ECN_set_ce(skb)) {
- q->stats.forced_drop++;
+ WRITE_ONCE(q->stats.forced_drop,
+ q->stats.forced_drop + 1);
goto congestion_drop;
}
- q->stats.forced_mark++;
+ WRITE_ONCE(q->stats.forced_mark,
+ q->stats.forced_mark + 1);
} else if (++q->vars.qcount) {
if (red_mark_probability(p, &q->vars, q->vars.qavg)) {
q->vars.qcount = 0;
@@ -253,11 +255,13 @@ static int choke_enqueue(struct sk_buff *skb, struct Qdisc *sch,
qdisc_qstats_overlimit(sch);
if (!use_ecn(q) || !INET_ECN_set_ce(skb)) {
- q->stats.prob_drop++;
+ WRITE_ONCE(q->stats.prob_drop,
+ q->stats.prob_drop + 1);
goto congestion_drop;
}
- q->stats.prob_mark++;
+ WRITE_ONCE(q->stats.prob_mark,
+ q->stats.prob_mark + 1);
}
} else
q->vars.qR = red_random(p);
@@ -272,7 +276,7 @@ static int choke_enqueue(struct sk_buff *skb, struct Qdisc *sch,
return NET_XMIT_SUCCESS;
}
- q->stats.pdrop++;
+ WRITE_ONCE(q->stats.pdrop, q->stats.pdrop + 1);
return qdisc_drop(skb, sch, to_free);
congestion_drop:
@@ -461,10 +465,12 @@ static int choke_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
{
struct choke_sched_data *q = qdisc_priv(sch);
struct tc_choke_xstats st = {
- .early = q->stats.prob_drop + q->stats.forced_drop,
- .marked = q->stats.prob_mark + q->stats.forced_mark,
- .pdrop = q->stats.pdrop,
- .matched = q->stats.matched,
+ .early = READ_ONCE(q->stats.prob_drop) +
+ READ_ONCE(q->stats.forced_drop),
+ .marked = READ_ONCE(q->stats.prob_mark) +
+ READ_ONCE(q->stats.forced_mark),
+ .pdrop = READ_ONCE(q->stats.pdrop),
+ .matched = READ_ONCE(q->stats.matched),
};
return gnet_stats_copy_app(d, &st, sizeof(st));
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index 551b7cbdae90..3f797ec4b0c2 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -581,6 +581,8 @@ static int fq_codel_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
};
struct list_head *pos;
+ sch_tree_lock(sch);
+
st.qdisc_stats.maxpacket = q->cstats.maxpacket;
st.qdisc_stats.drop_overlimit = q->drop_overlimit;
st.qdisc_stats.ecn_mark = q->cstats.ecn_mark;
@@ -589,7 +591,6 @@ static int fq_codel_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
st.qdisc_stats.memory_usage = q->memory_usage;
st.qdisc_stats.drop_overmemory = q->drop_overmemory;
- sch_tree_lock(sch);
list_for_each(pos, &q->new_flows)
st.qdisc_stats.new_flows_len++;
diff --git a/net/sched/sch_fq_pie.c b/net/sched/sch_fq_pie.c
index 6ed08b705f8a..ceba15465662 100644
--- a/net/sched/sch_fq_pie.c
+++ b/net/sched/sch_fq_pie.c
@@ -506,18 +506,19 @@ static int fq_pie_dump(struct Qdisc *sch, struct sk_buff *skb)
static int fq_pie_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
{
struct fq_pie_sched_data *q = qdisc_priv(sch);
- struct tc_fq_pie_xstats st = {
- .packets_in = q->stats.packets_in,
- .overlimit = q->stats.overlimit,
- .overmemory = q->overmemory,
- .dropped = q->stats.dropped,
- .ecn_mark = q->stats.ecn_mark,
- .new_flow_count = q->new_flow_count,
- .memory_usage = q->memory_usage,
- };
+ struct tc_fq_pie_xstats st = { 0 };
struct list_head *pos;
sch_tree_lock(sch);
+
+ st.packets_in = q->stats.packets_in;
+ st.overlimit = q->stats.overlimit;
+ st.overmemory = q->overmemory;
+ st.dropped = q->stats.dropped;
+ st.ecn_mark = q->stats.ecn_mark;
+ st.new_flow_count = q->new_flow_count;
+ st.memory_usage = q->memory_usage;
+
list_for_each(pos, &q->new_flows)
st.new_flows_len++;
diff --git a/net/sched/sch_hhf.c b/net/sched/sch_hhf.c
index 5aa434b46707..914e98542774 100644
--- a/net/sched/sch_hhf.c
+++ b/net/sched/sch_hhf.c
@@ -198,7 +198,8 @@ static struct hh_flow_state *seek_list(const u32 hash,
return NULL;
list_del(&flow->flowchain);
kfree(flow);
- q->hh_flows_current_cnt--;
+ WRITE_ONCE(q->hh_flows_current_cnt,
+ q->hh_flows_current_cnt - 1);
} else if (flow->hash_id == hash) {
return flow;
}
@@ -226,7 +227,7 @@ static struct hh_flow_state *alloc_new_hh(struct list_head *head,
}
if (q->hh_flows_current_cnt >= q->hh_flows_limit) {
- q->hh_flows_overlimit++;
+ WRITE_ONCE(q->hh_flows_overlimit, q->hh_flows_overlimit + 1);
return NULL;
}
/* Create new entry. */
@@ -234,7 +235,7 @@ static struct hh_flow_state *alloc_new_hh(struct list_head *head,
if (!flow)
return NULL;
- q->hh_flows_current_cnt++;
+ WRITE_ONCE(q->hh_flows_current_cnt, q->hh_flows_current_cnt + 1);
INIT_LIST_HEAD(&flow->flowchain);
list_add_tail(&flow->flowchain, head);
@@ -309,7 +310,7 @@ static enum wdrr_bucket_idx hhf_classify(struct sk_buff *skb, struct Qdisc *sch)
return WDRR_BUCKET_FOR_NON_HH;
flow->hash_id = hash;
flow->hit_timestamp = now;
- q->hh_flows_total_cnt++;
+ WRITE_ONCE(q->hh_flows_total_cnt, q->hh_flows_total_cnt + 1);
/* By returning without updating counters in q->hhf_arrays,
* we implicitly implement "shielding" (see Optimization O1).
@@ -403,7 +404,7 @@ static int hhf_enqueue(struct sk_buff *skb, struct Qdisc *sch,
return NET_XMIT_SUCCESS;
prev_backlog = sch->qstats.backlog;
- q->drop_overlimit++;
+ WRITE_ONCE(q->drop_overlimit, q->drop_overlimit + 1);
/* Return Congestion Notification only if we dropped a packet from this
* bucket.
*/
@@ -685,10 +686,10 @@ static int hhf_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
{
struct hhf_sched_data *q = qdisc_priv(sch);
struct tc_hhf_xstats st = {
- .drop_overlimit = q->drop_overlimit,
- .hh_overlimit = q->hh_flows_overlimit,
- .hh_tot_count = q->hh_flows_total_cnt,
- .hh_cur_count = q->hh_flows_current_cnt,
+ .drop_overlimit = READ_ONCE(q->drop_overlimit),
+ .hh_overlimit = READ_ONCE(q->hh_flows_overlimit),
+ .hh_tot_count = READ_ONCE(q->hh_flows_total_cnt),
+ .hh_cur_count = READ_ONCE(q->hh_flows_current_cnt),
};
return gnet_stats_copy_app(d, &st, sizeof(st));
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 825c398aa123..498c18d7d9c3 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -226,10 +226,10 @@ static bool loss_4state(struct netem_sched_data *q)
if (rnd < clg->a4) {
clg->state = LOST_IN_GAP_PERIOD;
return true;
- } else if (clg->a4 < rnd && rnd < clg->a1 + clg->a4) {
+ } else if (rnd < clg->a1 + clg->a4) {
clg->state = LOST_IN_BURST_PERIOD;
return true;
- } else if (clg->a1 + clg->a4 < rnd) {
+ } else {
clg->state = TX_IN_GAP_PERIOD;
}
@@ -246,9 +246,9 @@ static bool loss_4state(struct netem_sched_data *q)
case LOST_IN_BURST_PERIOD:
if (rnd < clg->a3)
clg->state = TX_IN_BURST_PERIOD;
- else if (clg->a3 < rnd && rnd < clg->a2 + clg->a3) {
+ else if (rnd < clg->a2 + clg->a3) {
clg->state = TX_IN_GAP_PERIOD;
- } else if (clg->a2 + clg->a3 < rnd) {
+ } else {
clg->state = LOST_IN_BURST_PERIOD;
return true;
}
@@ -522,7 +522,7 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
1 << get_random_u32_below(8);
}
- if (unlikely(q->t_len >= sch->limit)) {
+ if (unlikely(sch->q.qlen >= sch->limit)) {
/* re-link segs, so that qdisc_drop_all() frees them all */
skb->next = segs;
qdisc_drop_all(skb, sch, to_free);
@@ -657,9 +657,8 @@ static void get_slot_next(struct netem_sched_data *q, u64 now)
if (!q->slot_dist)
next_delay = q->slot_config.min_delay +
- (get_random_u32() *
- (q->slot_config.max_delay -
- q->slot_config.min_delay) >> 32);
+ mul_u64_u32_shr(q->slot_config.max_delay - q->slot_config.min_delay,
+ get_random_u32(), 32);
else
next_delay = tabledist(q->slot_config.dist_delay,
(s32)(q->slot_config.dist_jitter),
@@ -825,6 +824,39 @@ static int get_dist_table(struct disttable **tbl, const struct nlattr *attr)
return 0;
}
+static int validate_time(const struct nlattr *attr, const char *name,
+ struct netlink_ext_ack *extack)
+{
+ if (nla_get_s64(attr) < 0) {
+ NL_SET_ERR_MSG_ATTR_FMT(extack, attr, "negative %s", name);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static int validate_slot(const struct nlattr *attr, struct netlink_ext_ack *extack)
+{
+ const struct tc_netem_slot *c = nla_data(attr);
+
+ if (c->min_delay < 0 || c->max_delay < 0) {
+ NL_SET_ERR_MSG_ATTR(extack, attr, "negative slot delay");
+ return -EINVAL;
+ }
+ if (c->min_delay > c->max_delay) {
+ NL_SET_ERR_MSG_ATTR(extack, attr, "slot min delay greater than max delay");
+ return -EINVAL;
+ }
+ if (c->dist_delay < 0 || c->dist_jitter < 0) {
+ NL_SET_ERR_MSG_ATTR(extack, attr, "negative dist delay");
+ return -EINVAL;
+ }
+ if (c->max_packets < 0 || c->max_bytes < 0) {
+ NL_SET_ERR_MSG_ATTR(extack, attr, "negative slot limit");
+ return -EINVAL;
+ }
+ return 0;
+}
+
static void get_slot(struct netem_sched_data *q, const struct nlattr *attr)
{
const struct tc_netem_slot *c = nla_data(attr);
@@ -1038,6 +1070,24 @@ static int netem_change(struct Qdisc *sch, struct nlattr *opt,
goto table_free;
}
+ if (tb[TCA_NETEM_SLOT]) {
+ ret = validate_slot(tb[TCA_NETEM_SLOT], extack);
+ if (ret)
+ goto table_free;
+ }
+
+ if (tb[TCA_NETEM_LATENCY64]) {
+ ret = validate_time(tb[TCA_NETEM_LATENCY64], "latency", extack);
+ if (ret)
+ goto table_free;
+ }
+
+ if (tb[TCA_NETEM_JITTER64]) {
+ ret = validate_time(tb[TCA_NETEM_JITTER64], "jitter", extack);
+ if (ret)
+ goto table_free;
+ }
+
sch_tree_lock(sch);
/* backup q->clg and q->loss_model */
old_clg = q->clg;
@@ -1110,11 +1160,10 @@ static int netem_change(struct Qdisc *sch, struct nlattr *opt,
/* capping jitter to the range acceptable by tabledist() */
q->jitter = min_t(s64, abs(q->jitter), INT_MAX);
- if (tb[TCA_NETEM_PRNG_SEED])
+ if (tb[TCA_NETEM_PRNG_SEED]) {
q->prng.seed = nla_get_u64(tb[TCA_NETEM_PRNG_SEED]);
- else
- q->prng.seed = get_random_u64();
- prandom_seed_state(&q->prng.prng_state, q->prng.seed);
+ prandom_seed_state(&q->prng.prng_state, q->prng.seed);
+ }
unlock:
sch_tree_unlock(sch);
@@ -1137,6 +1186,9 @@ static int netem_init(struct Qdisc *sch, struct nlattr *opt,
return -EINVAL;
q->loss_model = CLG_RANDOM;
+ q->prng.seed = get_random_u64();
+ prandom_seed_state(&q->prng.prng_state, q->prng.seed);
+
ret = netem_change(sch, opt, extack);
if (ret)
pr_info("netem: change failed\n");
diff --git a/net/sched/sch_pie.c b/net/sched/sch_pie.c
index db61cbc21b13..a7c847781010 100644
--- a/net/sched/sch_pie.c
+++ b/net/sched/sch_pie.c
@@ -89,7 +89,7 @@ static int pie_qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
bool enqueue = false;
if (unlikely(qdisc_qlen(sch) >= sch->limit)) {
- q->stats.overlimit++;
+ WRITE_ONCE(q->stats.overlimit, q->stats.overlimit + 1);
goto out;
}
@@ -101,7 +101,7 @@ static int pie_qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
/* If packet is ecn capable, mark it if drop probability
* is lower than 10%, else drop it.
*/
- q->stats.ecn_mark++;
+ WRITE_ONCE(q->stats.ecn_mark, q->stats.ecn_mark + 1);
enqueue = true;
}
@@ -111,15 +111,15 @@ static int pie_qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
if (!q->params.dq_rate_estimator)
pie_set_enqueue_time(skb);
- q->stats.packets_in++;
+ WRITE_ONCE(q->stats.packets_in, q->stats.packets_in + 1);
if (qdisc_qlen(sch) > q->stats.maxq)
- q->stats.maxq = qdisc_qlen(sch);
+ WRITE_ONCE(q->stats.maxq, qdisc_qlen(sch));
return qdisc_enqueue_tail(skb, sch);
}
out:
- q->stats.dropped++;
+ WRITE_ONCE(q->stats.dropped, q->stats.dropped + 1);
q->vars.accu_prob = 0;
return qdisc_drop(skb, sch, to_free);
}
@@ -214,16 +214,14 @@ void pie_process_dequeue(struct sk_buff *skb, struct pie_params *params,
* packet timestamp.
*/
if (!params->dq_rate_estimator) {
- vars->qdelay = now - pie_get_enqueue_time(skb);
+ WRITE_ONCE(vars->qdelay,
+ backlog ? now - pie_get_enqueue_time(skb) : 0);
if (vars->dq_tstamp != DTIME_INVALID)
dtime = now - vars->dq_tstamp;
vars->dq_tstamp = now;
- if (backlog == 0)
- vars->qdelay = 0;
-
if (dtime == 0)
return;
@@ -262,11 +260,11 @@ void pie_process_dequeue(struct sk_buff *skb, struct pie_params *params,
count = count / dtime;
if (vars->avg_dq_rate == 0)
- vars->avg_dq_rate = count;
+ WRITE_ONCE(vars->avg_dq_rate, count);
else
- vars->avg_dq_rate =
+ WRITE_ONCE(vars->avg_dq_rate,
(vars->avg_dq_rate -
- (vars->avg_dq_rate >> 3)) + (count >> 3);
+ (vars->avg_dq_rate >> 3)) + (count >> 3));
/* If the queue has receded below the threshold, we hold
* on to the last drain rate calculated, else we reset
@@ -371,12 +369,12 @@ void pie_calculate_probability(struct pie_params *params, struct pie_vars *vars,
if (qdelay > (PSCHED_NS2TICKS(250 * NSEC_PER_MSEC)))
delta += MAX_PROB / (100 / 2);
- vars->prob += delta;
+ WRITE_ONCE(vars->prob, vars->prob + delta);
if (delta > 0) {
/* prevent overflow */
if (vars->prob < oldprob) {
- vars->prob = MAX_PROB;
+ WRITE_ONCE(vars->prob, MAX_PROB);
/* Prevent normalization error. If probability is at
* maximum value already, we normalize it here, and
* skip the check to do a non-linear drop in the next
@@ -387,7 +385,7 @@ void pie_calculate_probability(struct pie_params *params, struct pie_vars *vars,
} else {
/* prevent underflow */
if (vars->prob > oldprob)
- vars->prob = 0;
+ WRITE_ONCE(vars->prob, 0);
}
/* Non-linear drop in probability: Reduce drop probability quickly if
@@ -396,9 +394,9 @@ void pie_calculate_probability(struct pie_params *params, struct pie_vars *vars,
if (qdelay == 0 && qdelay_old == 0 && update_prob)
/* Reduce drop probability to 98.4% */
- vars->prob -= vars->prob / 64;
+ WRITE_ONCE(vars->prob, vars->prob - vars->prob / 64);
- vars->qdelay = qdelay;
+ WRITE_ONCE(vars->qdelay, qdelay);
vars->backlog_old = backlog;
/* We restart the measurement cycle if the following conditions are met
@@ -496,22 +494,22 @@ static int pie_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
{
struct pie_sched_data *q = qdisc_priv(sch);
struct tc_pie_xstats st = {
- .prob = q->vars.prob << BITS_PER_BYTE,
- .delay = ((u32)PSCHED_TICKS2NS(q->vars.qdelay)) /
+ .prob = READ_ONCE(q->vars.prob) << BITS_PER_BYTE,
+ .delay = ((u32)PSCHED_TICKS2NS(READ_ONCE(q->vars.qdelay))) /
NSEC_PER_USEC,
- .packets_in = q->stats.packets_in,
- .overlimit = q->stats.overlimit,
- .maxq = q->stats.maxq,
- .dropped = q->stats.dropped,
- .ecn_mark = q->stats.ecn_mark,
+ .packets_in = READ_ONCE(q->stats.packets_in),
+ .overlimit = READ_ONCE(q->stats.overlimit),
+ .maxq = READ_ONCE(q->stats.maxq),
+ .dropped = READ_ONCE(q->stats.dropped),
+ .ecn_mark = READ_ONCE(q->stats.ecn_mark),
};
/* avg_dq_rate is only valid if dq_rate_estimator is enabled */
- st.dq_rate_estimating = q->params.dq_rate_estimator;
+ st.dq_rate_estimating = READ_ONCE(q->params.dq_rate_estimator);
/* unscale and return dq_rate in bytes per sec */
- if (q->params.dq_rate_estimator)
- st.avg_dq_rate = q->vars.avg_dq_rate *
+ if (st.dq_rate_estimating)
+ st.avg_dq_rate = READ_ONCE(q->vars.avg_dq_rate) *
(PSCHED_TICKS_PER_SEC) >> PIE_SCALE;
return gnet_stats_copy_app(d, &st, sizeof(st));
diff --git a/net/sched/sch_red.c b/net/sched/sch_red.c
index a745d429a814..6f53a0dbd057 100644
--- a/net/sched/sch_red.c
+++ b/net/sched/sch_red.c
@@ -89,17 +89,20 @@ static int red_enqueue(struct sk_buff *skb, struct Qdisc *sch,
case RED_PROB_MARK:
qdisc_qstats_overlimit(sch);
if (!red_use_ecn(q)) {
- q->stats.prob_drop++;
+ WRITE_ONCE(q->stats.prob_drop,
+ q->stats.prob_drop + 1);
goto congestion_drop;
}
if (INET_ECN_set_ce(skb)) {
- q->stats.prob_mark++;
+ WRITE_ONCE(q->stats.prob_mark,
+ q->stats.prob_mark + 1);
skb = tcf_qevent_handle(&q->qe_mark, sch, skb, to_free, &ret);
if (!skb)
return NET_XMIT_CN | ret;
} else if (!red_use_nodrop(q)) {
- q->stats.prob_drop++;
+ WRITE_ONCE(q->stats.prob_drop,
+ q->stats.prob_drop + 1);
goto congestion_drop;
}
@@ -109,17 +112,20 @@ static int red_enqueue(struct sk_buff *skb, struct Qdisc *sch,
case RED_HARD_MARK:
qdisc_qstats_overlimit(sch);
if (red_use_harddrop(q) || !red_use_ecn(q)) {
- q->stats.forced_drop++;
+ WRITE_ONCE(q->stats.forced_drop,
+ q->stats.forced_drop + 1);
goto congestion_drop;
}
if (INET_ECN_set_ce(skb)) {
- q->stats.forced_mark++;
+ WRITE_ONCE(q->stats.forced_mark,
+ q->stats.forced_mark + 1);
skb = tcf_qevent_handle(&q->qe_mark, sch, skb, to_free, &ret);
if (!skb)
return NET_XMIT_CN | ret;
} else if (!red_use_nodrop(q)) {
- q->stats.forced_drop++;
+ WRITE_ONCE(q->stats.forced_drop,
+ q->stats.forced_drop + 1);
goto congestion_drop;
}
@@ -133,7 +139,8 @@ static int red_enqueue(struct sk_buff *skb, struct Qdisc *sch,
sch->qstats.backlog += len;
sch->q.qlen++;
} else if (net_xmit_drop_count(ret)) {
- q->stats.pdrop++;
+ WRITE_ONCE(q->stats.pdrop,
+ q->stats.pdrop + 1);
qdisc_qstats_drop(sch);
}
return ret;
@@ -461,9 +468,13 @@ static int red_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_RED,
&hw_stats_request);
}
- st.early = q->stats.prob_drop + q->stats.forced_drop;
- st.pdrop = q->stats.pdrop;
- st.marked = q->stats.prob_mark + q->stats.forced_mark;
+ st.early = READ_ONCE(q->stats.prob_drop) +
+ READ_ONCE(q->stats.forced_drop);
+
+ st.pdrop = READ_ONCE(q->stats.pdrop);
+
+ st.marked = READ_ONCE(q->stats.prob_mark) +
+ READ_ONCE(q->stats.forced_mark);
return gnet_stats_copy_app(d, &st, sizeof(st));
}
diff --git a/net/sched/sch_sfb.c b/net/sched/sch_sfb.c
index b717e15a3a17..c36725f0870d 100644
--- a/net/sched/sch_sfb.c
+++ b/net/sched/sch_sfb.c
@@ -130,7 +130,7 @@ static void increment_one_qlen(u32 sfbhash, u32 slot, struct sfb_sched_data *q)
sfbhash >>= SFB_BUCKET_SHIFT;
if (b[hash].qlen < 0xFFFF)
- b[hash].qlen++;
+ WRITE_ONCE(b[hash].qlen, b[hash].qlen + 1);
b += SFB_NUMBUCKETS; /* next level */
}
}
@@ -159,7 +159,7 @@ static void decrement_one_qlen(u32 sfbhash, u32 slot,
sfbhash >>= SFB_BUCKET_SHIFT;
if (b[hash].qlen > 0)
- b[hash].qlen--;
+ WRITE_ONCE(b[hash].qlen, b[hash].qlen - 1);
b += SFB_NUMBUCKETS; /* next level */
}
}
@@ -179,12 +179,12 @@ static void decrement_qlen(const struct sk_buff *skb, struct sfb_sched_data *q)
static void decrement_prob(struct sfb_bucket *b, struct sfb_sched_data *q)
{
- b->p_mark = prob_minus(b->p_mark, q->decrement);
+ WRITE_ONCE(b->p_mark, prob_minus(b->p_mark, q->decrement));
}
static void increment_prob(struct sfb_bucket *b, struct sfb_sched_data *q)
{
- b->p_mark = prob_plus(b->p_mark, q->increment);
+ WRITE_ONCE(b->p_mark, prob_plus(b->p_mark, q->increment));
}
static void sfb_zero_all_buckets(struct sfb_sched_data *q)
@@ -202,11 +202,14 @@ static u32 sfb_compute_qlen(u32 *prob_r, u32 *avgpm_r, const struct sfb_sched_da
const struct sfb_bucket *b = &q->bins[q->slot].bins[0][0];
for (i = 0; i < SFB_LEVELS * SFB_NUMBUCKETS; i++) {
- if (qlen < b->qlen)
- qlen = b->qlen;
- totalpm += b->p_mark;
- if (prob < b->p_mark)
- prob = b->p_mark;
+ u32 b_qlen = READ_ONCE(b->qlen);
+ u32 b_mark = READ_ONCE(b->p_mark);
+
+ if (qlen < b_qlen)
+ qlen = b_qlen;
+ totalpm += b_mark;
+ if (prob < b_mark)
+ prob = b_mark;
b++;
}
*prob_r = prob;
@@ -294,7 +297,8 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch,
if (unlikely(sch->q.qlen >= q->limit)) {
qdisc_qstats_overlimit(sch);
- q->stats.queuedrop++;
+ WRITE_ONCE(q->stats.queuedrop,
+ q->stats.queuedrop + 1);
goto drop;
}
@@ -347,7 +351,8 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch,
if (unlikely(minqlen >= q->max)) {
qdisc_qstats_overlimit(sch);
- q->stats.bucketdrop++;
+ WRITE_ONCE(q->stats.bucketdrop,
+ q->stats.bucketdrop + 1);
goto drop;
}
@@ -373,7 +378,8 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch,
}
if (sfb_rate_limit(skb, q)) {
qdisc_qstats_overlimit(sch);
- q->stats.penaltydrop++;
+ WRITE_ONCE(q->stats.penaltydrop,
+ q->stats.penaltydrop + 1);
goto drop;
}
goto enqueue;
@@ -388,14 +394,17 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch,
* In either case, we want to start dropping packets.
*/
if (r < (p_min - SFB_MAX_PROB / 2) * 2) {
- q->stats.earlydrop++;
+ WRITE_ONCE(q->stats.earlydrop,
+ q->stats.earlydrop + 1);
goto drop;
}
}
if (INET_ECN_set_ce(skb)) {
- q->stats.marked++;
+ WRITE_ONCE(q->stats.marked,
+ q->stats.marked + 1);
} else {
- q->stats.earlydrop++;
+ WRITE_ONCE(q->stats.earlydrop,
+ q->stats.earlydrop + 1);
goto drop;
}
}
@@ -408,7 +417,8 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch,
sch->q.qlen++;
increment_qlen(&cb, q);
} else if (net_xmit_drop_count(ret)) {
- q->stats.childdrop++;
+ WRITE_ONCE(q->stats.childdrop,
+ q->stats.childdrop + 1);
qdisc_qstats_drop(sch);
}
return ret;
@@ -597,12 +607,12 @@ static int sfb_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
{
struct sfb_sched_data *q = qdisc_priv(sch);
struct tc_sfb_xstats st = {
- .earlydrop = q->stats.earlydrop,
- .penaltydrop = q->stats.penaltydrop,
- .bucketdrop = q->stats.bucketdrop,
- .queuedrop = q->stats.queuedrop,
- .childdrop = q->stats.childdrop,
- .marked = q->stats.marked,
+ .earlydrop = READ_ONCE(q->stats.earlydrop),
+ .penaltydrop = READ_ONCE(q->stats.penaltydrop),
+ .bucketdrop = READ_ONCE(q->stats.bucketdrop),
+ .queuedrop = READ_ONCE(q->stats.queuedrop),
+ .childdrop = READ_ONCE(q->stats.childdrop),
+ .marked = READ_ONCE(q->stats.marked),
};
st.maxqlen = sfb_compute_qlen(&st.maxprob, &st.avgprob, q);
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 1620f0fd78ce..f1709efb5f04 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -633,7 +633,7 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
queue = skb_get_queue_mapping(skb);
child = q->qdiscs[queue];
- if (unlikely(!child))
+ if (unlikely(child == &noop_qdisc))
return qdisc_drop(skb, sch, to_free);
if (taprio_skb_exceeds_queue_max_sdu(sch, skb)) {
@@ -716,7 +716,7 @@ static struct sk_buff *taprio_dequeue_from_txq(struct Qdisc *sch, int txq,
int len;
u8 tc;
- if (unlikely(!child))
+ if (unlikely(child == &noop_qdisc))
return NULL;
if (TXTIME_ASSIST_IS_ENABLED(q->flags))
@@ -971,11 +971,12 @@ static enum hrtimer_restart advance_sched(struct hrtimer *timer)
}
if (should_change_schedules(admin, oper, end_time)) {
- /* Set things so the next time this runs, the new
- * schedule runs.
- */
- end_time = sched_base_time(admin);
switch_schedules(q, &admin, &oper);
+ /* After changing schedules, the next entry is the first one
+ * in the new schedule, with a pre-calculated end_time.
+ */
+ next = list_first_entry(&oper->entries, struct sched_entry, list);
+ end_time = next->end_time;
}
next->end_time = end_time;
@@ -2190,6 +2191,9 @@ static int taprio_graft(struct Qdisc *sch, unsigned long cl,
if (!dev_queue)
return -EINVAL;
+ if (!new)
+ new = &noop_qdisc;
+
if (dev->flags & IFF_UP)
dev_deactivate(dev);
@@ -2203,14 +2207,14 @@ static int taprio_graft(struct Qdisc *sch, unsigned long cl,
*old = q->qdiscs[cl - 1];
if (FULL_OFFLOAD_IS_ENABLED(q->flags)) {
WARN_ON_ONCE(dev_graft_qdisc(dev_queue, new) != *old);
- if (new)
+ if (new != &noop_qdisc)
qdisc_refcount_inc(new);
- if (*old)
+ if (*old && *old != &noop_qdisc)
qdisc_put(*old);
}
q->qdiscs[cl - 1] = new;
- if (new)
+ if (new != &noop_qdisc)
new->flags |= TCQ_F_ONETXQUEUE | TCQ_F_NOPARENT;
if (dev->flags & IFF_UP)
diff --git a/net/sctp/inqueue.c b/net/sctp/inqueue.c
index f5a7d5a38755..a024c0843247 100644
--- a/net/sctp/inqueue.c
+++ b/net/sctp/inqueue.c
@@ -201,6 +201,7 @@ struct sctp_chunk *sctp_inq_pop(struct sctp_inq *queue)
cb->chunk = head_cb->chunk;
cb->af = head_cb->af;
+ cb->encap_port = head_cb->encap_port;
}
}
diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c
index 966bd6a44594..376d4ce5ebb3 100644
--- a/net/sctp/sm_statefuns.c
+++ b/net/sctp/sm_statefuns.c
@@ -1556,6 +1556,12 @@ static enum sctp_disposition sctp_sf_do_unexpected_init(
/* Tag the variable length parameters. */
chunk->param_hdr.v = skb_pull(chunk->skb, sizeof(struct sctp_inithdr));
+ if (asoc->state >= SCTP_STATE_ESTABLISHED) {
+ /* Discard INIT matching peer vtag after handshake completion (stale INIT). */
+ if (ntohl(chunk->subh.init_hdr->init_tag) == asoc->peer.i.init_tag)
+ return sctp_sf_pdiscard(net, ep, asoc, type, arg, commands);
+ }
+
/* Verify the INIT chunk before processing it. */
err_chunk = NULL;
if (!sctp_verify_init(net, ep, asoc, chunk->chunk_hdr->type,
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index c8038b4b67c7..6b562dd1aae1 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -7003,7 +7003,7 @@ static int sctp_getsockopt_peer_auth_chunks(struct sock *sk, int len,
/* See if the user provided enough room for all the data */
num_chunks = ntohs(ch->param_hdr.length) - sizeof(struct sctp_paramhdr);
- if (len < num_chunks)
+ if (len < sizeof(struct sctp_authchunks) + num_chunks)
return -EINVAL;
if (copy_to_user(to, ch->chunks, num_chunks))
diff --git a/net/socket.c b/net/socket.c
index a0f6f8b3376d..878155076bc0 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -509,7 +509,7 @@ static int sock_map_fd(struct socket *sock, int flags)
struct socket *sock_from_file(struct file *file)
{
- if (file->f_op == &socket_file_ops)
+ if (likely(file->f_op == &socket_file_ops))
return file->private_data; /* set in sock_alloc_file */
return NULL;
@@ -549,24 +549,6 @@ struct socket *sockfd_lookup(int fd, int *err)
}
EXPORT_SYMBOL(sockfd_lookup);
-static struct socket *sockfd_lookup_light(int fd, int *err, int *fput_needed)
-{
- struct fd f = fdget(fd);
- struct socket *sock;
-
- *err = -EBADF;
- if (fd_file(f)) {
- sock = sock_from_file(fd_file(f));
- if (likely(sock)) {
- *fput_needed = f.word & FDPUT_FPUT;
- return sock;
- }
- *err = -ENOTSOCK;
- fdput(f);
- }
- return NULL;
-}
-
static ssize_t sockfs_listxattr(struct dentry *dentry, char *buffer,
size_t size)
{
@@ -1857,16 +1839,20 @@ int __sys_bind(int fd, struct sockaddr __user *umyaddr, int addrlen)
{
struct socket *sock;
struct sockaddr_storage address;
- int err, fput_needed;
-
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (sock) {
- err = move_addr_to_kernel(umyaddr, addrlen, &address);
- if (!err)
- err = __sys_bind_socket(sock, &address, addrlen);
- fput_light(sock->file, fput_needed);
- }
- return err;
+ CLASS(fd, f)(fd);
+ int err;
+
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
+
+ err = move_addr_to_kernel(umyaddr, addrlen, &address);
+ if (unlikely(err))
+ return err;
+
+ return __sys_bind_socket(sock, &address, addrlen);
}
SYSCALL_DEFINE3(bind, int, fd, struct sockaddr __user *, umyaddr, int, addrlen)
@@ -1895,15 +1881,16 @@ int __sys_listen_socket(struct socket *sock, int backlog)
int __sys_listen(int fd, int backlog)
{
+ CLASS(fd, f)(fd);
struct socket *sock;
- int err, fput_needed;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (sock) {
- err = __sys_listen_socket(sock, backlog);
- fput_light(sock->file, fput_needed);
- }
- return err;
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
+
+ return __sys_listen_socket(sock, backlog);
}
SYSCALL_DEFINE2(listen, int, fd, int, backlog)
@@ -2013,17 +2000,12 @@ static int __sys_accept4_file(struct file *file, struct sockaddr __user *upeer_s
int __sys_accept4(int fd, struct sockaddr __user *upeer_sockaddr,
int __user *upeer_addrlen, int flags)
{
- int ret = -EBADF;
- struct fd f;
+ CLASS(fd, f)(fd);
- f = fdget(fd);
- if (fd_file(f)) {
- ret = __sys_accept4_file(fd_file(f), upeer_sockaddr,
+ if (fd_empty(f))
+ return -EBADF;
+ return __sys_accept4_file(fd_file(f), upeer_sockaddr,
upeer_addrlen, flags);
- fdput(f);
- }
-
- return ret;
}
SYSCALL_DEFINE4(accept4, int, fd, struct sockaddr __user *, upeer_sockaddr,
@@ -2075,20 +2057,18 @@ int __sys_connect_file(struct file *file, struct sockaddr_storage *address,
int __sys_connect(int fd, struct sockaddr __user *uservaddr, int addrlen)
{
- int ret = -EBADF;
- struct fd f;
+ struct sockaddr_storage address;
+ CLASS(fd, f)(fd);
+ int ret;
- f = fdget(fd);
- if (fd_file(f)) {
- struct sockaddr_storage address;
+ if (fd_empty(f))
+ return -EBADF;
- ret = move_addr_to_kernel(uservaddr, addrlen, &address);
- if (!ret)
- ret = __sys_connect_file(fd_file(f), &address, addrlen, 0);
- fdput(f);
- }
+ ret = move_addr_to_kernel(uservaddr, addrlen, &address);
+ if (ret)
+ return ret;
- return ret;
+ return __sys_connect_file(fd_file(f), &address, addrlen, 0);
}
SYSCALL_DEFINE3(connect, int, fd, struct sockaddr __user *, uservaddr,
@@ -2107,26 +2087,25 @@ int __sys_getsockname(int fd, struct sockaddr __user *usockaddr,
{
struct socket *sock;
struct sockaddr_storage address;
- int err, fput_needed;
+ CLASS(fd, f)(fd);
+ int err;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- goto out;
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
err = security_socket_getsockname(sock);
if (err)
- goto out_put;
+ return err;
err = READ_ONCE(sock->ops)->getname(sock, (struct sockaddr *)&address, 0);
if (err < 0)
- goto out_put;
- /* "err" is actually length in this case */
- err = move_addr_to_user(&address, err, usockaddr, usockaddr_len);
+ return err;
-out_put:
- fput_light(sock->file, fput_needed);
-out:
- return err;
+ /* "err" is actually length in this case */
+ return move_addr_to_user(&address, err, usockaddr, usockaddr_len);
}
SYSCALL_DEFINE3(getsockname, int, fd, struct sockaddr __user *, usockaddr,
@@ -2145,26 +2124,25 @@ int __sys_getpeername(int fd, struct sockaddr __user *usockaddr,
{
struct socket *sock;
struct sockaddr_storage address;
- int err, fput_needed;
+ CLASS(fd, f)(fd);
+ int err;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (sock != NULL) {
- const struct proto_ops *ops = READ_ONCE(sock->ops);
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
- err = security_socket_getpeername(sock);
- if (err) {
- fput_light(sock->file, fput_needed);
- return err;
- }
+ err = security_socket_getpeername(sock);
+ if (err)
+ return err;
- err = ops->getname(sock, (struct sockaddr *)&address, 1);
- if (err >= 0)
- /* "err" is actually length in this case */
- err = move_addr_to_user(&address, err, usockaddr,
- usockaddr_len);
- fput_light(sock->file, fput_needed);
- }
- return err;
+ err = READ_ONCE(sock->ops)->getname(sock, (struct sockaddr *)&address, 1);
+ if (err < 0)
+ return err;
+
+ /* "err" is actually length in this case */
+ return move_addr_to_user(&address, err, usockaddr, usockaddr_len);
}
SYSCALL_DEFINE3(getpeername, int, fd, struct sockaddr __user *, usockaddr,
@@ -2185,14 +2163,17 @@ int __sys_sendto(int fd, void __user *buff, size_t len, unsigned int flags,
struct sockaddr_storage address;
int err;
struct msghdr msg;
- int fput_needed;
err = import_ubuf(ITER_SOURCE, buff, len, &msg.msg_iter);
if (unlikely(err))
return err;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- goto out;
+
+ CLASS(fd, f)(fd);
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
msg.msg_name = NULL;
msg.msg_control = NULL;
@@ -2202,7 +2183,7 @@ int __sys_sendto(int fd, void __user *buff, size_t len, unsigned int flags,
if (addr) {
err = move_addr_to_kernel(addr, addr_len, &address);
if (err < 0)
- goto out_put;
+ return err;
msg.msg_name = (struct sockaddr *)&address;
msg.msg_namelen = addr_len;
}
@@ -2210,12 +2191,7 @@ int __sys_sendto(int fd, void __user *buff, size_t len, unsigned int flags,
if (sock->file->f_flags & O_NONBLOCK)
flags |= MSG_DONTWAIT;
msg.msg_flags = flags;
- err = __sock_sendmsg(sock, &msg);
-
-out_put:
- fput_light(sock->file, fput_needed);
-out:
- return err;
+ return __sock_sendmsg(sock, &msg);
}
SYSCALL_DEFINE6(sendto, int, fd, void __user *, buff, size_t, len,
@@ -2250,14 +2226,18 @@ int __sys_recvfrom(int fd, void __user *ubuf, size_t size, unsigned int flags,
};
struct socket *sock;
int err, err2;
- int fput_needed;
err = import_ubuf(ITER_DEST, ubuf, size, &msg.msg_iter);
if (unlikely(err))
return err;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- goto out;
+
+ CLASS(fd, f)(fd);
+
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
if (sock->file->f_flags & O_NONBLOCK)
flags |= MSG_DONTWAIT;
@@ -2269,9 +2249,6 @@ int __sys_recvfrom(int fd, void __user *ubuf, size_t size, unsigned int flags,
if (err2 < 0)
err = err2;
}
-
- fput_light(sock->file, fput_needed);
-out:
return err;
}
@@ -2346,17 +2323,16 @@ int __sys_setsockopt(int fd, int level, int optname, char __user *user_optval,
{
sockptr_t optval = USER_SOCKPTR(user_optval);
bool compat = in_compat_syscall();
- int err, fput_needed;
struct socket *sock;
+ CLASS(fd, f)(fd);
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- return err;
-
- err = do_sock_setsockopt(sock, compat, level, optname, optval, optlen);
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
- fput_light(sock->file, fput_needed);
- return err;
+ return do_sock_setsockopt(sock, compat, level, optname, optval, optlen);
}
SYSCALL_DEFINE5(setsockopt, int, fd, int, level, int, optname,
@@ -2412,20 +2388,17 @@ EXPORT_SYMBOL(do_sock_getsockopt);
int __sys_getsockopt(int fd, int level, int optname, char __user *optval,
int __user *optlen)
{
- int err, fput_needed;
struct socket *sock;
- bool compat;
+ CLASS(fd, f)(fd);
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- return err;
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
- compat = in_compat_syscall();
- err = do_sock_getsockopt(sock, compat, level, optname,
+ return do_sock_getsockopt(sock, in_compat_syscall(), level, optname,
USER_SOCKPTR(optval), USER_SOCKPTR(optlen));
-
- fput_light(sock->file, fput_needed);
- return err;
}
SYSCALL_DEFINE5(getsockopt, int, fd, int, level, int, optname,
@@ -2451,15 +2424,16 @@ int __sys_shutdown_sock(struct socket *sock, int how)
int __sys_shutdown(int fd, int how)
{
- int err, fput_needed;
struct socket *sock;
+ CLASS(fd, f)(fd);
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (sock != NULL) {
- err = __sys_shutdown_sock(sock, how);
- fput_light(sock->file, fput_needed);
- }
- return err;
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
+
+ return __sys_shutdown_sock(sock, how);
}
SYSCALL_DEFINE2(shutdown, int, fd, int, how)
@@ -2675,22 +2649,21 @@ long __sys_sendmsg_sock(struct socket *sock, struct msghdr *msg,
long __sys_sendmsg(int fd, struct user_msghdr __user *msg, unsigned int flags,
bool forbid_cmsg_compat)
{
- int fput_needed, err;
struct msghdr msg_sys;
struct socket *sock;
if (forbid_cmsg_compat && (flags & MSG_CMSG_COMPAT))
return -EINVAL;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- goto out;
+ CLASS(fd, f)(fd);
- err = ___sys_sendmsg(sock, msg, &msg_sys, flags, NULL, 0);
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
- fput_light(sock->file, fput_needed);
-out:
- return err;
+ return ___sys_sendmsg(sock, msg, &msg_sys, flags, NULL, 0);
}
SYSCALL_DEFINE3(sendmsg, int, fd, struct user_msghdr __user *, msg, unsigned int, flags)
@@ -2705,7 +2678,7 @@ SYSCALL_DEFINE3(sendmsg, int, fd, struct user_msghdr __user *, msg, unsigned int
int __sys_sendmmsg(int fd, struct mmsghdr __user *mmsg, unsigned int vlen,
unsigned int flags, bool forbid_cmsg_compat)
{
- int fput_needed, err, datagrams;
+ int err, datagrams;
struct socket *sock;
struct mmsghdr __user *entry;
struct compat_mmsghdr __user *compat_entry;
@@ -2721,9 +2694,13 @@ int __sys_sendmmsg(int fd, struct mmsghdr __user *mmsg, unsigned int vlen,
datagrams = 0;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- return err;
+ CLASS(fd, f)(fd);
+
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
used_address.name_len = UINT_MAX;
entry = mmsg;
@@ -2760,8 +2737,6 @@ int __sys_sendmmsg(int fd, struct mmsghdr __user *mmsg, unsigned int vlen,
cond_resched();
}
- fput_light(sock->file, fput_needed);
-
/* We only return an error if no datagrams were able to be sent */
if (datagrams != 0)
return datagrams;
@@ -2883,22 +2858,21 @@ long __sys_recvmsg_sock(struct socket *sock, struct msghdr *msg,
long __sys_recvmsg(int fd, struct user_msghdr __user *msg, unsigned int flags,
bool forbid_cmsg_compat)
{
- int fput_needed, err;
struct msghdr msg_sys;
struct socket *sock;
if (forbid_cmsg_compat && (flags & MSG_CMSG_COMPAT))
return -EINVAL;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- goto out;
+ CLASS(fd, f)(fd);
- err = ___sys_recvmsg(sock, msg, &msg_sys, flags, 0);
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
- fput_light(sock->file, fput_needed);
-out:
- return err;
+ return ___sys_recvmsg(sock, msg, &msg_sys, flags, 0);
}
SYSCALL_DEFINE3(recvmsg, int, fd, struct user_msghdr __user *, msg,
@@ -2915,7 +2889,7 @@ static int do_recvmmsg(int fd, struct mmsghdr __user *mmsg,
unsigned int vlen, unsigned int flags,
struct timespec64 *timeout)
{
- int fput_needed, err, datagrams;
+ int err = 0, datagrams;
struct socket *sock;
struct mmsghdr __user *entry;
struct compat_mmsghdr __user *compat_entry;
@@ -2930,16 +2904,18 @@ static int do_recvmmsg(int fd, struct mmsghdr __user *mmsg,
datagrams = 0;
- sock = sockfd_lookup_light(fd, &err, &fput_needed);
- if (!sock)
- return err;
+ CLASS(fd, f)(fd);
+
+ if (fd_empty(f))
+ return -EBADF;
+ sock = sock_from_file(fd_file(f));
+ if (unlikely(!sock))
+ return -ENOTSOCK;
if (likely(!(flags & MSG_ERRQUEUE))) {
err = sock_error(sock->sk);
- if (err) {
- datagrams = err;
- goto out_put;
- }
+ if (err)
+ return err;
}
entry = mmsg;
@@ -2996,12 +2972,10 @@ static int do_recvmmsg(int fd, struct mmsghdr __user *mmsg,
}
if (err == 0)
- goto out_put;
+ return datagrams;
- if (datagrams == 0) {
- datagrams = err;
- goto out_put;
- }
+ if (datagrams == 0)
+ return err;
/*
* We may return less entries than requested (vlen) if the
@@ -3016,9 +2990,6 @@ static int do_recvmmsg(int fd, struct mmsghdr __user *mmsg,
*/
WRITE_ONCE(sock->sk->sk_err, -err);
}
-out_put:
- fput_light(sock->file, fput_needed);
-
return datagrams;
}
diff --git a/net/tipc/msg.c b/net/tipc/msg.c
index 76284fc538eb..b0bba0feef56 100644
--- a/net/tipc/msg.c
+++ b/net/tipc/msg.c
@@ -177,8 +177,20 @@ int tipc_buf_append(struct sk_buff **headbuf, struct sk_buff **buf)
if (fragid == LAST_FRAGMENT) {
TIPC_SKB_CB(head)->validated = 0;
- if (unlikely(!tipc_msg_validate(&head)))
+
+ /* If the reassembled skb has been freed in
+ * tipc_msg_validate() because of an invalid truesize,
+ * then head will point to a newly allocated reassembled
+ * skb, while *headbuf points to freed reassembled skb.
+ * In such cases, correct *headbuf for freeing the newly
+ * allocated reassembled skb later.
+ */
+ if (unlikely(!tipc_msg_validate(&head))) {
+ if (head != *headbuf)
+ *headbuf = head;
goto err;
+ }
+
*buf = head;
TIPC_SKB_CB(head)->tail = NULL;
*headbuf = NULL;
diff --git a/net/tls/tls.h b/net/tls/tls.h
index fca0c0e17004..97eba6f6ab65 100644
--- a/net/tls/tls.h
+++ b/net/tls/tls.h
@@ -186,6 +186,7 @@ int tls_strp_dev_init(void);
void tls_strp_dev_exit(void);
void tls_strp_done(struct tls_strparser *strp);
+void __tls_strp_done(struct tls_strparser *strp);
void tls_strp_stop(struct tls_strparser *strp);
int tls_strp_init(struct tls_strparser *strp, struct sock *sk);
void tls_strp_data_ready(struct tls_strparser *strp);
diff --git a/net/tls/tls_strp.c b/net/tls/tls_strp.c
index 98e12f0ff57e..c72e88317627 100644
--- a/net/tls/tls_strp.c
+++ b/net/tls/tls_strp.c
@@ -624,6 +624,12 @@ void tls_strp_done(struct tls_strparser *strp)
WARN_ON(!strp->stopped);
cancel_work_sync(&strp->work);
+ __tls_strp_done(strp);
+}
+
+/* For setup error paths where the strparser was initialized but never armed. */
+void __tls_strp_done(struct tls_strparser *strp)
+{
tls_strp_anchor_free(strp);
}
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index 36351942903b..4550f15d052d 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -2591,8 +2591,12 @@ void tls_sw_free_ctx_rx(struct tls_context *tls_ctx)
void tls_sw_free_resources_rx(struct sock *sk)
{
struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context_rx *ctx;
+
+ ctx = tls_sw_ctx_rx(tls_ctx);
tls_sw_release_resources_rx(sk);
+ __tls_strp_done(&ctx->strp);
tls_sw_free_ctx_rx(tls_ctx);
}
diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
index fd7f161e6e39..77976f36c4aa 100644
--- a/net/unix/af_unix.c
+++ b/net/unix/af_unix.c
@@ -3621,15 +3621,15 @@ static int bpf_iter_unix_seq_show(struct seq_file *seq, void *v)
struct bpf_prog *prog;
struct sock *sk = v;
uid_t uid;
- bool slow;
int ret;
if (v == SEQ_START_TOKEN)
return 0;
- slow = lock_sock_fast(sk);
+ lock_sock(sk);
+ unix_state_lock(sk);
- if (unlikely(sk_unhashed(sk))) {
+ if (unlikely(sock_flag(sk, SOCK_DEAD))) {
ret = SEQ_SKIP;
goto unlock;
}
@@ -3639,7 +3639,8 @@ static int bpf_iter_unix_seq_show(struct seq_file *seq, void *v)
prog = bpf_iter_get_info(&meta, false);
ret = unix_prog_seq_show(prog, &meta, v, uid);
unlock:
- unlock_sock_fast(sk, slow);
+ unix_state_unlock(sk);
+ release_sock(sk);
return ret;
}
diff --git a/net/unix/unix_bpf.c b/net/unix/unix_bpf.c
index bca2d86ba97d..976e035053e5 100644
--- a/net/unix/unix_bpf.c
+++ b/net/unix/unix_bpf.c
@@ -184,6 +184,9 @@ int unix_stream_bpf_update_proto(struct sock *sk, struct sk_psock *psock, bool r
*/
if (!psock->sk_pair) {
sk_pair = unix_peer(sk);
+ if (unlikely(!sk_pair))
+ return -EINVAL;
+
sock_hold(sk_pair);
psock->sk_pair = sk_pair;
}
diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index fb686fd3266f..1d7d4838eebd 100755
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -125,7 +125,13 @@ install_kernel_headers () {
pdir=debian/$1
version=${1#linux-headers-}
- CC="${DEB_HOST_GNU_TYPE}-gcc" "${srctree}/scripts/package/install-extmod-build" "${pdir}/usr/src/linux-headers-${version}"
+ # Override $CC only for cross-compiles, to not unnecessarily rebuild
+ # scripts/ including plugins, which may lead to a full kernel rebuild.
+ if [ -n "${CROSS_COMPILE}" ]; then
+ CC="${DEB_HOST_GNU_TYPE}-gcc" "${srctree}/scripts/package/install-extmod-build" "${pdir}/usr/src/linux-headers-${version}"
+ else
+ "${srctree}/scripts/package/install-extmod-build" "${pdir}/usr/src/linux-headers-${version}"
+ fi
mkdir -p $pdir/lib/modules/$version/
ln -s /usr/src/linux-headers-$version $pdir/lib/modules/$version/build
diff --git a/security/integrity/ima/ima_crypto.c b/security/integrity/ima/ima_crypto.c
index 6f5696d999d0..8ae7821a65c2 100644
--- a/security/integrity/ima/ima_crypto.c
+++ b/security/integrity/ima/ima_crypto.c
@@ -832,7 +832,7 @@ static int ima_calc_boot_aggregate_tfm(char *digest, u16 alg_id,
}
}
if (!rc)
- crypto_shash_final(shash, digest);
+ rc = crypto_shash_final(shash, digest);
return rc;
}
diff --git a/security/integrity/ima/ima_fs.c b/security/integrity/ima/ima_fs.c
index e4a79a9b2d58..25970867f594 100644
--- a/security/integrity/ima/ima_fs.c
+++ b/security/integrity/ima/ima_fs.c
@@ -116,28 +116,6 @@ void ima_putc(struct seq_file *m, void *data, int datalen)
seq_putc(m, *(char *)data++);
}
-static struct dentry **ascii_securityfs_measurement_lists __ro_after_init;
-static struct dentry **binary_securityfs_measurement_lists __ro_after_init;
-static int securityfs_measurement_list_count __ro_after_init;
-
-static void lookup_template_data_hash_algo(int *algo_idx, enum hash_algo *algo,
- struct seq_file *m,
- struct dentry **lists)
-{
- struct dentry *dentry;
- int i;
-
- dentry = file_dentry(m->file);
-
- for (i = 0; i < securityfs_measurement_list_count; i++) {
- if (dentry == lists[i]) {
- *algo_idx = i;
- *algo = ima_algo_array[i].algo;
- break;
- }
- }
-}
-
/* print format:
* 32bit-le=pcr#
* char[n]=template digest
@@ -160,9 +138,10 @@ int ima_measurements_show(struct seq_file *m, void *v)
algo_idx = ima_sha1_idx;
algo = HASH_ALGO_SHA1;
- if (m->file != NULL)
- lookup_template_data_hash_algo(&algo_idx, &algo, m,
- binary_securityfs_measurement_lists);
+ if (m->file != NULL) {
+ algo_idx = (unsigned long)file_inode(m->file)->i_private;
+ algo = ima_algo_array[algo_idx].algo;
+ }
/* get entry */
e = qe->entry;
@@ -256,9 +235,10 @@ static int ima_ascii_measurements_show(struct seq_file *m, void *v)
algo_idx = ima_sha1_idx;
algo = HASH_ALGO_SHA1;
- if (m->file != NULL)
- lookup_template_data_hash_algo(&algo_idx, &algo, m,
- ascii_securityfs_measurement_lists);
+ if (m->file != NULL) {
+ algo_idx = (unsigned long)file_inode(m->file)->i_private;
+ algo = ima_algo_array[algo_idx].algo;
+ }
/* get entry */
e = qe->entry;
@@ -396,11 +376,6 @@ static ssize_t ima_write_policy(struct file *file, const char __user *buf,
static struct dentry *ima_dir;
static struct dentry *ima_symlink;
-static struct dentry *binary_runtime_measurements;
-static struct dentry *ascii_runtime_measurements;
-static struct dentry *runtime_measurements_count;
-static struct dentry *violations;
-static struct dentry *ima_policy;
enum ima_fs_flags {
IMA_FS_BUSY,
@@ -417,64 +392,41 @@ static const struct seq_operations ima_policy_seqops = {
};
#endif
-static void __init remove_securityfs_measurement_lists(struct dentry **lists)
-{
- int i;
-
- if (lists) {
- for (i = 0; i < securityfs_measurement_list_count; i++)
- securityfs_remove(lists[i]);
-
- kfree(lists);
- }
-}
-
static int __init create_securityfs_measurement_lists(void)
{
- char file_name[NAME_MAX + 1];
- struct dentry *dentry;
- u16 algo;
- int i;
-
- securityfs_measurement_list_count = NR_BANKS(ima_tpm_chip);
+ int count = NR_BANKS(ima_tpm_chip);
if (ima_sha1_idx >= NR_BANKS(ima_tpm_chip))
- securityfs_measurement_list_count++;
-
- ascii_securityfs_measurement_lists =
- kcalloc(securityfs_measurement_list_count, sizeof(struct dentry *),
- GFP_KERNEL);
- if (!ascii_securityfs_measurement_lists)
- return -ENOMEM;
-
- binary_securityfs_measurement_lists =
- kcalloc(securityfs_measurement_list_count, sizeof(struct dentry *),
- GFP_KERNEL);
- if (!binary_securityfs_measurement_lists)
- return -ENOMEM;
-
- for (i = 0; i < securityfs_measurement_list_count; i++) {
- algo = ima_algo_array[i].algo;
-
- sprintf(file_name, "ascii_runtime_measurements_%s",
- hash_algo_name[algo]);
+ count++;
+
+ for (int i = 0; i < count; i++) {
+ u16 algo = ima_algo_array[i].algo;
+ char file_name[NAME_MAX + 1];
+ struct dentry *dentry;
+
+ if (algo == HASH_ALGO__LAST)
+ sprintf(file_name, "ascii_runtime_measurements_tpm_alg_%x",
+ ima_tpm_chip->allocated_banks[i].alg_id);
+ else
+ sprintf(file_name, "ascii_runtime_measurements_%s",
+ hash_algo_name[algo]);
dentry = securityfs_create_file(file_name, S_IRUSR | S_IRGRP,
- ima_dir, NULL,
+ ima_dir, (void *)(uintptr_t)i,
&ima_ascii_measurements_ops);
if (IS_ERR(dentry))
return PTR_ERR(dentry);
- ascii_securityfs_measurement_lists[i] = dentry;
-
- sprintf(file_name, "binary_runtime_measurements_%s",
- hash_algo_name[algo]);
+ if (algo == HASH_ALGO__LAST)
+ sprintf(file_name, "binary_runtime_measurements_tpm_alg_%x",
+ ima_tpm_chip->allocated_banks[i].alg_id);
+ else
+ sprintf(file_name, "binary_runtime_measurements_%s",
+ hash_algo_name[algo]);
dentry = securityfs_create_file(file_name, S_IRUSR | S_IRGRP,
- ima_dir, NULL,
+ ima_dir, (void *)(uintptr_t)i,
&ima_measurements_ops);
if (IS_ERR(dentry))
return PTR_ERR(dentry);
-
- binary_securityfs_measurement_lists[i] = dentry;
}
return 0;
@@ -533,8 +485,7 @@ static int ima_release_policy(struct inode *inode, struct file *file)
ima_update_policy();
#if !defined(CONFIG_IMA_WRITE_POLICY) && !defined(CONFIG_IMA_READ_POLICY)
- securityfs_remove(ima_policy);
- ima_policy = NULL;
+ securityfs_remove(file->f_path.dentry);
#elif defined(CONFIG_IMA_WRITE_POLICY)
clear_bit(IMA_FS_BUSY, &ima_fs_flags);
#elif defined(CONFIG_IMA_READ_POLICY)
@@ -553,11 +504,9 @@ static const struct file_operations ima_measure_policy_ops = {
int __init ima_fs_init(void)
{
+ struct dentry *dentry;
int ret;
- ascii_securityfs_measurement_lists = NULL;
- binary_securityfs_measurement_lists = NULL;
-
ima_dir = securityfs_create_dir("ima", integrity_dir);
if (IS_ERR(ima_dir))
return PTR_ERR(ima_dir);
@@ -573,57 +522,45 @@ int __init ima_fs_init(void)
if (ret != 0)
goto out;
- binary_runtime_measurements =
- securityfs_create_symlink("binary_runtime_measurements", ima_dir,
+ dentry = securityfs_create_symlink("binary_runtime_measurements", ima_dir,
"binary_runtime_measurements_sha1", NULL);
- if (IS_ERR(binary_runtime_measurements)) {
- ret = PTR_ERR(binary_runtime_measurements);
+ if (IS_ERR(dentry)) {
+ ret = PTR_ERR(dentry);
goto out;
}
- ascii_runtime_measurements =
- securityfs_create_symlink("ascii_runtime_measurements", ima_dir,
+ dentry = securityfs_create_symlink("ascii_runtime_measurements", ima_dir,
"ascii_runtime_measurements_sha1", NULL);
- if (IS_ERR(ascii_runtime_measurements)) {
- ret = PTR_ERR(ascii_runtime_measurements);
+ if (IS_ERR(dentry)) {
+ ret = PTR_ERR(dentry);
goto out;
}
- runtime_measurements_count =
- securityfs_create_file("runtime_measurements_count",
+ dentry = securityfs_create_file("runtime_measurements_count",
S_IRUSR | S_IRGRP, ima_dir, NULL,
&ima_measurements_count_ops);
- if (IS_ERR(runtime_measurements_count)) {
- ret = PTR_ERR(runtime_measurements_count);
+ if (IS_ERR(dentry)) {
+ ret = PTR_ERR(dentry);
goto out;
}
- violations =
- securityfs_create_file("violations", S_IRUSR | S_IRGRP,
+ dentry = securityfs_create_file("violations", S_IRUSR | S_IRGRP,
ima_dir, NULL, &ima_htable_violations_ops);
- if (IS_ERR(violations)) {
- ret = PTR_ERR(violations);
+ if (IS_ERR(dentry)) {
+ ret = PTR_ERR(dentry);
goto out;
}
- ima_policy = securityfs_create_file("policy", POLICY_FILE_FLAGS,
+ dentry = securityfs_create_file("policy", POLICY_FILE_FLAGS,
ima_dir, NULL,
&ima_measure_policy_ops);
- if (IS_ERR(ima_policy)) {
- ret = PTR_ERR(ima_policy);
+ if (IS_ERR(dentry)) {
+ ret = PTR_ERR(dentry);
goto out;
}
return 0;
out:
- securityfs_remove(ima_policy);
- securityfs_remove(violations);
- securityfs_remove(runtime_measurements_count);
- securityfs_remove(ascii_runtime_measurements);
- securityfs_remove(binary_runtime_measurements);
- remove_securityfs_measurement_lists(ascii_securityfs_measurement_lists);
- remove_securityfs_measurement_lists(binary_securityfs_measurement_lists);
- securityfs_measurement_list_count = 0;
securityfs_remove(ima_symlink);
securityfs_remove(ima_dir);
diff --git a/security/landlock/syscalls.c b/security/landlock/syscalls.c
index 4fa2d09f657a..25f9f3e0d397 100644
--- a/security/landlock/syscalls.c
+++ b/security/landlock/syscalls.c
@@ -255,31 +255,21 @@ const int landlock_abi_version = LANDLOCK_ABI_VERSION;
static struct landlock_ruleset *get_ruleset_from_fd(const int fd,
const fmode_t mode)
{
- struct fd ruleset_f;
+ CLASS(fd, ruleset_f)(fd);
struct landlock_ruleset *ruleset;
- ruleset_f = fdget(fd);
- if (!fd_file(ruleset_f))
+ if (fd_empty(ruleset_f))
return ERR_PTR(-EBADF);
/* Checks FD type and access right. */
- if (fd_file(ruleset_f)->f_op != &ruleset_fops) {
- ruleset = ERR_PTR(-EBADFD);
- goto out_fdput;
- }
- if (!(fd_file(ruleset_f)->f_mode & mode)) {
- ruleset = ERR_PTR(-EPERM);
- goto out_fdput;
- }
+ if (fd_file(ruleset_f)->f_op != &ruleset_fops)
+ return ERR_PTR(-EBADFD);
+ if (!(fd_file(ruleset_f)->f_mode & mode))
+ return ERR_PTR(-EPERM);
ruleset = fd_file(ruleset_f)->private_data;
- if (WARN_ON_ONCE(ruleset->num_layers != 1)) {
- ruleset = ERR_PTR(-EINVAL);
- goto out_fdput;
- }
+ if (WARN_ON_ONCE(ruleset->num_layers != 1))
+ return ERR_PTR(-EINVAL);
landlock_get_ruleset(ruleset);
-
-out_fdput:
- fdput(ruleset_f);
return ruleset;
}
diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c
index bdf1d78de833..d81a890b60c6 100644
--- a/sound/core/compress_offload.c
+++ b/sound/core/compress_offload.c
@@ -40,13 +40,6 @@
#define COMPR_CODEC_CAPS_OVERFLOW
#endif
-/* TODO:
- * - add substream support for multiple devices in case of
- * SND_DYNAMIC_MINORS is not used
- * - Multiple node representation
- * driver should be able to register multiple nodes
- */
-
struct snd_compr_file {
unsigned long caps;
struct snd_compr_stream stream;
diff --git a/sound/core/sound.c b/sound/core/sound.c
index 6531a67f13b3..7980b60f4ba0 100644
--- a/sound/core/sound.c
+++ b/sound/core/sound.c
@@ -216,9 +216,16 @@ static int snd_find_free_minor(int type, struct snd_card *card, int dev)
case SNDRV_DEVICE_TYPE_RAWMIDI:
case SNDRV_DEVICE_TYPE_PCM_PLAYBACK:
case SNDRV_DEVICE_TYPE_PCM_CAPTURE:
+ if (snd_BUG_ON(!card))
+ return -EINVAL;
+ minor = SNDRV_MINOR(card->number, type + dev);
+ break;
case SNDRV_DEVICE_TYPE_COMPRESS:
if (snd_BUG_ON(!card))
return -EINVAL;
+ if (dev < 0 ||
+ dev >= SNDRV_MINOR_HWDEP - SNDRV_MINOR_COMPRESS)
+ return -EINVAL;
minor = SNDRV_MINOR(card->number, type + dev);
break;
default:
diff --git a/sound/isa/sc6000.c b/sound/isa/sc6000.c
index 3115c32b4061..4066b68a102e 100644
--- a/sound/isa/sc6000.c
+++ b/sound/isa/sc6000.c
@@ -100,6 +100,15 @@ MODULE_PARM_DESC(joystick, "Enable gameport.");
#define PFX "sc6000: "
#define DRV_NAME "SC-6000"
+struct snd_sc6000 {
+ char __iomem *vport;
+ char __iomem *vmss_port;
+ u8 mss_config;
+ u8 config;
+ u8 hw_cfg[2];
+ bool old_dsp;
+};
+
/* hardware dependent functions */
/*
@@ -267,7 +276,7 @@ static int sc6000_dsp_reset(char __iomem *vport)
/* detection and initialization */
static int sc6000_hw_cfg_write(struct device *devptr,
- char __iomem *vport, const int *cfg)
+ char __iomem *vport, const u8 *cfg)
{
if (sc6000_write(devptr, vport, COMMAND_6C) < 0) {
dev_warn(devptr, "CMD 0x%x: failed!\n", COMMAND_6C);
@@ -353,8 +362,7 @@ static int sc6000_init_mss(struct device *devptr,
return 0;
}
-static void sc6000_hw_cfg_encode(struct device *devptr,
- char __iomem *vport, int *cfg,
+static void sc6000_hw_cfg_encode(struct device *devptr, u8 *cfg,
long xport, long xmpu,
long xmss_port, int joystick)
{
@@ -376,27 +384,83 @@ static void sc6000_hw_cfg_encode(struct device *devptr,
dev_dbg(devptr, "hw cfg %x, %x\n", cfg[0], cfg[1]);
}
-static int sc6000_init_board(struct device *devptr,
- char __iomem *vport,
- char __iomem *vmss_port, int dev)
+static void sc6000_prepare_board(struct device *devptr,
+ struct snd_sc6000 *sc6000,
+ unsigned int dev, int xirq, int xdma)
+{
+ sc6000->mss_config = sc6000_irq_to_softcfg(xirq) |
+ sc6000_dma_to_softcfg(xdma);
+ sc6000->config = sc6000->mss_config |
+ sc6000_mpu_irq_to_softcfg(mpu_irq[dev]);
+ sc6000_hw_cfg_encode(devptr, sc6000->hw_cfg, port[dev], mpu_port[dev],
+ mss_port[dev], joystick[dev]);
+}
+
+static void sc6000_detect_old_dsp(struct device *devptr,
+ struct snd_sc6000 *sc6000)
+{
+ sc6000_write(devptr, sc6000->vport, COMMAND_5C);
+ sc6000->old_dsp = sc6000_read(sc6000->vport) < 0;
+}
+
+static int sc6000_program_board(struct device *devptr,
+ struct snd_sc6000 *sc6000)
+{
+ int err;
+
+ if (!sc6000->old_dsp) {
+ if (sc6000_hw_cfg_write(devptr, sc6000->vport,
+ sc6000->hw_cfg) < 0) {
+ dev_err(devptr, "sc6000_hw_cfg_write: failed!\n");
+ return -EIO;
+ }
+ }
+
+ err = sc6000_setup_board(devptr, sc6000->vport, sc6000->config);
+ if (err < 0) {
+ dev_err(devptr, "sc6000_setup_board: failed!\n");
+ return -ENODEV;
+ }
+
+ sc6000_dsp_reset(sc6000->vport);
+
+ if (!sc6000->old_dsp) {
+ sc6000_write(devptr, sc6000->vport, COMMAND_60);
+ sc6000_write(devptr, sc6000->vport, 0x02);
+ sc6000_dsp_reset(sc6000->vport);
+ }
+
+ err = sc6000_setup_board(devptr, sc6000->vport, sc6000->config);
+ if (err < 0) {
+ dev_err(devptr, "sc6000_setup_board: failed!\n");
+ return -ENODEV;
+ }
+
+ err = sc6000_init_mss(devptr, sc6000->vport, sc6000->config,
+ sc6000->vmss_port, sc6000->mss_config);
+ if (err < 0) {
+ dev_err(devptr, "Cannot initialize Microsoft Sound System mode.\n");
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static int sc6000_init_board(struct device *devptr, struct snd_sc6000 *sc6000)
{
char answer[15];
char version[2];
- int mss_config = sc6000_irq_to_softcfg(irq[dev]) |
- sc6000_dma_to_softcfg(dma[dev]);
- int config = mss_config |
- sc6000_mpu_irq_to_softcfg(mpu_irq[dev]);
int err;
- int old = 0;
- err = sc6000_dsp_reset(vport);
+ err = sc6000_dsp_reset(sc6000->vport);
if (err < 0) {
dev_err(devptr, "sc6000_dsp_reset: failed!\n");
return err;
}
memset(answer, 0, sizeof(answer));
- err = sc6000_dsp_get_answer(devptr, vport, GET_DSP_COPYRIGHT, answer, 15);
+ err = sc6000_dsp_get_answer(devptr, sc6000->vport, GET_DSP_COPYRIGHT,
+ answer, 15);
if (err <= 0) {
dev_err(devptr, "sc6000_dsp_copyright: failed!\n");
return -ENODEV;
@@ -408,54 +472,17 @@ static int sc6000_init_board(struct device *devptr,
if (strncmp("SC-6000", answer, 7))
dev_warn(devptr, "Warning: non SC-6000 audio card!\n");
- if (sc6000_dsp_get_answer(devptr, vport, GET_DSP_VERSION, version, 2) < 2) {
+ if (sc6000_dsp_get_answer(devptr, sc6000->vport,
+ GET_DSP_VERSION, version, 2) < 2) {
dev_err(devptr, "sc6000_dsp_version: failed!\n");
return -ENODEV;
}
dev_info(devptr, "Detected model: %s, DSP version %d.%d\n",
answer, version[0], version[1]);
- /* set configuration */
- sc6000_write(devptr, vport, COMMAND_5C);
- if (sc6000_read(vport) < 0)
- old = 1;
-
- if (!old) {
- int cfg[2];
- sc6000_hw_cfg_encode(devptr,
- vport, &cfg[0], port[dev], mpu_port[dev],
- mss_port[dev], joystick[dev]);
- if (sc6000_hw_cfg_write(devptr, vport, cfg) < 0) {
- dev_err(devptr, "sc6000_hw_cfg_write: failed!\n");
- return -EIO;
- }
- }
- err = sc6000_setup_board(devptr, vport, config);
- if (err < 0) {
- dev_err(devptr, "sc6000_setup_board: failed!\n");
- return -ENODEV;
- }
-
- sc6000_dsp_reset(vport);
-
- if (!old) {
- sc6000_write(devptr, vport, COMMAND_60);
- sc6000_write(devptr, vport, 0x02);
- sc6000_dsp_reset(vport);
- }
+ sc6000_detect_old_dsp(devptr, sc6000);
- err = sc6000_setup_board(devptr, vport, config);
- if (err < 0) {
- dev_err(devptr, "sc6000_setup_board: failed!\n");
- return -ENODEV;
- }
- err = sc6000_init_mss(devptr, vport, config, vmss_port, mss_config);
- if (err < 0) {
- dev_err(devptr, "Cannot initialize Microsoft Sound System mode.\n");
- return -ENODEV;
- }
-
- return 0;
+ return sc6000_program_board(devptr, sc6000);
}
static int snd_sc6000_mixer(struct snd_wss *chip)
@@ -538,10 +565,10 @@ static int snd_sc6000_match(struct device *devptr, unsigned int dev)
static void snd_sc6000_free(struct snd_card *card)
{
- char __iomem *vport = (char __force __iomem *)card->private_data;
+ struct snd_sc6000 *sc6000 = card->private_data;
- if (vport)
- sc6000_setup_board(card->dev, vport, 0);
+ if (sc6000->vport)
+ sc6000_setup_board(card->dev, sc6000->vport, 0);
}
static int __snd_sc6000_probe(struct device *devptr, unsigned int dev)
@@ -552,15 +579,17 @@ static int __snd_sc6000_probe(struct device *devptr, unsigned int dev)
int xirq = irq[dev];
int xdma = dma[dev];
struct snd_card *card;
+ struct snd_sc6000 *sc6000;
struct snd_wss *chip;
struct snd_opl3 *opl3;
char __iomem *vport;
char __iomem *vmss_port;
err = snd_devm_card_new(devptr, index[dev], id[dev], THIS_MODULE,
- 0, &card);
+ sizeof(*sc6000), &card);
if (err < 0)
return err;
+ sc6000 = card->private_data;
if (xirq == SNDRV_AUTO_IRQ) {
xirq = snd_legacy_find_free_irq(possible_irqs);
@@ -587,7 +616,7 @@ static int __snd_sc6000_probe(struct device *devptr, unsigned int dev)
dev_err(devptr, "I/O port cannot be iomapped.\n");
return -EBUSY;
}
- card->private_data = (void __force *)vport;
+ sc6000->vport = vport;
/* to make it marked as used */
if (!devm_request_region(devptr, mss_port[dev], 4, DRV_NAME)) {
@@ -600,12 +629,15 @@ static int __snd_sc6000_probe(struct device *devptr, unsigned int dev)
dev_err(devptr, "MSS port I/O cannot be iomapped.\n");
return -EBUSY;
}
+ sc6000->vmss_port = vmss_port;
dev_dbg(devptr, "Initializing BASE[0x%lx] IRQ[%d] DMA[%d] MIRQ[%d]\n",
port[dev], xirq, xdma,
mpu_irq[dev] == SNDRV_AUTO_IRQ ? 0 : mpu_irq[dev]);
- err = sc6000_init_board(devptr, vport, vmss_port, dev);
+ sc6000_prepare_board(devptr, sc6000, dev, xirq, xdma);
+
+ err = sc6000_init_board(devptr, sc6000);
if (err < 0)
return err;
card->private_free = snd_sc6000_free;
diff --git a/sound/pci/hda/cs35l56_hda.c b/sound/pci/hda/cs35l56_hda.c
index c86817771286..ee5387140ae4 100644
--- a/sound/pci/hda/cs35l56_hda.c
+++ b/sound/pci/hda/cs35l56_hda.c
@@ -888,6 +888,7 @@ static int cs35l56_hda_system_resume(struct device *dev)
static int cs35l56_hda_fixup_yoga9(struct cs35l56_hda *cs35l56, int *bus_addr)
{
/* The cirrus,dev-index property has the wrong values */
+ cs35l56->num_amps = 2;
switch (*bus_addr) {
case 0x30:
cs35l56->index = 1;
@@ -937,7 +938,6 @@ static int cs35l56_hda_read_acpi(struct cs35l56_hda *cs35l56, int hid, int id)
char hid_string[8];
struct acpi_device *adev;
const char *property, *sub;
- size_t nval;
int i, ret;
/*
@@ -973,13 +973,14 @@ static int cs35l56_hda_read_acpi(struct cs35l56_hda *cs35l56, int hid, int id)
ret = -EINVAL;
goto err;
}
- nval = ret;
+ cs35l56->num_amps = ret;
- ret = device_property_read_u32_array(cs35l56->base.dev, property, values, nval);
+ ret = device_property_read_u32_array(cs35l56->base.dev, property, values,
+ cs35l56->num_amps);
if (ret)
goto err;
- for (i = 0; i < nval; i++) {
+ for (i = 0; i < cs35l56->num_amps; i++) {
if (values[i] == id) {
cs35l56->index = i;
break;
@@ -1002,7 +1003,8 @@ static int cs35l56_hda_read_acpi(struct cs35l56_hda *cs35l56, int hid, int id)
"Read ACPI _SUB failed(%ld): fallback to generic firmware\n",
PTR_ERR(sub));
} else {
- ret = cirrus_scodec_get_speaker_id(cs35l56->base.dev, cs35l56->index, nval, -1);
+ ret = cirrus_scodec_get_speaker_id(cs35l56->base.dev, cs35l56->index,
+ cs35l56->num_amps, -1);
if (ret == -ENOENT) {
cs35l56->system_name = sub;
} else if (ret >= 0) {
diff --git a/sound/pci/hda/cs35l56_hda.h b/sound/pci/hda/cs35l56_hda.h
index 38d94fb213a5..0074e8f5f18c 100644
--- a/sound/pci/hda/cs35l56_hda.h
+++ b/sound/pci/hda/cs35l56_hda.h
@@ -25,6 +25,7 @@ struct cs35l56_hda {
struct work_struct dsp_work;
int index;
+ int num_amps;
const char *system_name;
const char *amp_name;
diff --git a/sound/pci/hda/patch_conexant.c b/sound/pci/hda/patch_conexant.c
index b7c9eba9236d..0296777bb380 100644
--- a/sound/pci/hda/patch_conexant.c
+++ b/sound/pci/hda/patch_conexant.c
@@ -42,7 +42,7 @@ struct conexant_spec {
unsigned int gpio_led;
unsigned int gpio_mute_led_mask;
unsigned int gpio_mic_led_mask;
- bool is_cx8070_sn6140;
+ bool is_cx11880_sn6140;
};
@@ -195,7 +195,7 @@ static int cx_auto_init(struct hda_codec *codec)
cxt_init_gpio_led(codec);
snd_hda_apply_fixup(codec, HDA_FIXUP_ACT_INIT);
- if (spec->is_cx8070_sn6140)
+ if (spec->is_cx11880_sn6140)
cx_fixup_headset_recog(codec);
return 0;
@@ -247,7 +247,7 @@ static void cx_update_headset_mic_vref(struct hda_codec *codec, struct hda_jack_
{
unsigned int mic_present;
- /* In cx8070 and sn6140, the node 16 can only be configured to headphone or disabled,
+ /* In cx11880 and sn6140, the node 16 can only be configured to headphone or disabled,
* the node 19 can only be configured to microphone or disabled.
* Check hp&mic tag to process headset plugin & plugout.
*/
@@ -1182,6 +1182,7 @@ static void add_cx5051_fake_mutes(struct hda_codec *codec)
static int patch_conexant_auto(struct hda_codec *codec)
{
struct conexant_spec *spec;
+ struct hda_jack_callback *callback;
int err;
codec_info(codec, "%s: BIOS auto-probing.\n", codec->core.chip_name);
@@ -1193,12 +1194,17 @@ static int patch_conexant_auto(struct hda_codec *codec)
codec->spec = spec;
codec->patch_ops = cx_auto_patch_ops;
- /* init cx8070/sn6140 flag and reset headset_present_flag */
+ /* init cx11880/sn6140 flag and reset headset_present_flag */
switch (codec->core.vendor_id) {
case 0x14f11f86:
case 0x14f11f87:
- spec->is_cx8070_sn6140 = true;
- snd_hda_jack_detect_enable_callback(codec, 0x19, cx_update_headset_mic_vref);
+ spec->is_cx11880_sn6140 = true;
+ callback = snd_hda_jack_detect_enable_callback(codec, 0x19,
+ cx_update_headset_mic_vref);
+ if (IS_ERR(callback)) {
+ err = PTR_ERR(callback);
+ goto error;
+ }
break;
}
@@ -1285,7 +1291,7 @@ static int patch_conexant_auto(struct hda_codec *codec)
*/
static const struct hda_device_id snd_hda_id_conexant[] = {
- HDA_CODEC_ENTRY(0x14f11f86, "CX8070", patch_conexant_auto),
+ HDA_CODEC_ENTRY(0x14f11f86, "CX11880", patch_conexant_auto),
HDA_CODEC_ENTRY(0x14f11f87, "SN6140", patch_conexant_auto),
HDA_CODEC_ENTRY(0x14f12008, "CX8200", patch_conexant_auto),
HDA_CODEC_ENTRY(0x14f120d0, "CX11970", patch_conexant_auto),
diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c
index 4cab9696fdab..b0ee9b58e057 100644
--- a/sound/pci/hda/patch_realtek.c
+++ b/sound/pci/hda/patch_realtek.c
@@ -6428,9 +6428,9 @@ static void alc_fixup_headset_mode_alc255_no_hp_mic(struct hda_codec *codec,
struct alc_spec *spec = codec->spec;
spec->parse_flags |= HDA_PINCFG_HEADSET_MIC;
alc255_set_default_jack_type(codec);
- }
- else
+ } else {
alc_fixup_headset_mode(codec, fix, action);
+ }
}
static void alc288_update_headset_jack_cb(struct hda_codec *codec,
@@ -7881,22 +7881,11 @@ static void alc287_alc1318_playback_pcm_hook(struct hda_pcm_stream *hinfo,
struct snd_pcm_substream *substream,
int action)
{
- static const struct coef_fw dis_coefs[] = {
- WRITE_COEF(0x24, 0x0013), WRITE_COEF(0x25, 0x0000), WRITE_COEF(0x26, 0xC203),
- WRITE_COEF(0x28, 0x0004), WRITE_COEF(0x29, 0xb023),
- }; /* Disable AMP silence detection */
- static const struct coef_fw en_coefs[] = {
- WRITE_COEF(0x24, 0x0013), WRITE_COEF(0x25, 0x0000), WRITE_COEF(0x26, 0xC203),
- WRITE_COEF(0x28, 0x0084), WRITE_COEF(0x29, 0xb023),
- }; /* Enable AMP silence detection */
-
switch (action) {
case HDA_GEN_PCM_ACT_OPEN:
- alc_process_coef_fw(codec, dis_coefs);
alc_write_coefex_idx(codec, 0x5a, 0x00, 0x954f); /* write gpio3 to high */
break;
case HDA_GEN_PCM_ACT_CLOSE:
- alc_process_coef_fw(codec, en_coefs);
alc_write_coefex_idx(codec, 0x5a, 0x00, 0x554f); /* write gpio3 as default value */
break;
}
@@ -7919,10 +7908,15 @@ static void alc287_fixup_lenovo_thinkpad_with_alc1318(struct hda_codec *codec,
WRITE_COEF(0x24, 0x0013), WRITE_COEF(0x25, 0x0000), WRITE_COEF(0x26, 0xC301),
WRITE_COEF(0x28, 0x0001), WRITE_COEF(0x29, 0xb023),
};
+ static const struct coef_fw dis_coefs[] = {
+ WRITE_COEF(0x24, 0x0013), WRITE_COEF(0x25, 0x0000), WRITE_COEF(0x26, 0xC203),
+ WRITE_COEF(0x28, 0x0004), WRITE_COEF(0x29, 0xb023),
+ }; /* Disable AMP silence detection */
if (action != HDA_FIXUP_ACT_PRE_PROBE)
return;
alc_update_coef_idx(codec, 0x10, 1<<11, 1<<11);
+ alc_process_coef_fw(codec, dis_coefs);
alc_process_coef_fw(codec, coefs);
spec->power_hook = alc287_s4_power_gpio3_default;
spec->gen.pcm_playback_hook = alc287_alc1318_playback_pcm_hook;
diff --git a/sound/soc/amd/acp/acp-legacy-mach.c b/sound/soc/amd/acp/acp-legacy-mach.c
index d104f7e8fdcd..4221fc0f081b 100644
--- a/sound/soc/amd/acp/acp-legacy-mach.c
+++ b/sound/soc/amd/acp/acp-legacy-mach.c
@@ -174,7 +174,7 @@ static int acp_asoc_probe(struct platform_device *pdev)
acp_card_drvdata->platform = *((int *)dev->platform_data);
dmi_id = dmi_first_match(acp_quirk_table);
- if (dmi_id && dmi_id->driver_data)
+ if (dmi_id && dmi_id->driver_data == (void *)QUIRK_TDM_MODE_ENABLE)
acp_card_drvdata->tdm_mode = dmi_id->driver_data;
ret = acp_legacy_dai_links_create(card);
diff --git a/sound/soc/amd/acp/acp-mach-common.c b/sound/soc/amd/acp/acp-mach-common.c
index e9ff4815c12c..6c0a92d76b54 100644
--- a/sound/soc/amd/acp/acp-mach-common.c
+++ b/sound/soc/amd/acp/acp-mach-common.c
@@ -20,6 +20,7 @@
#include <sound/soc.h>
#include <linux/input.h>
#include <linux/module.h>
+#include <linux/dmi.h>
#include "../../codecs/rt5682.h"
#include "../../codecs/rt1019.h"
@@ -37,15 +38,21 @@
#define NAU8821_FREQ_OUT 12288000
#define MAX98388_CODEC_DAI "max98388-aif1"
-#define TDM_MODE_ENABLE 1
-
const struct dmi_system_id acp_quirk_table[] = {
{
/* Google skyrim proto-0 */
.matches = {
DMI_EXACT_MATCH(DMI_PRODUCT_FAMILY, "Google_Skyrim"),
},
- .driver_data = (void *)TDM_MODE_ENABLE,
+ .driver_data = (void *)QUIRK_TDM_MODE_ENABLE,
+ },
+ {
+ /* Valve Steam Deck OLED */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Valve"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Galileo"),
+ },
+ .driver_data = (void *)QUIRK_REMAP_DMIC_BT,
},
{}
};
@@ -1385,6 +1392,7 @@ int acp_sofdsp_dai_links_create(struct snd_soc_card *card)
struct snd_soc_dai_link *links;
struct device *dev = card->dev;
struct acp_card_drvdata *drv_data = card->drvdata;
+ const struct dmi_system_id *dmi_id = dmi_first_match(acp_quirk_table);
int i = 0, num_links = 0;
if (drv_data->hs_cpu_id)
@@ -1562,6 +1570,9 @@ int acp_sofdsp_dai_links_create(struct snd_soc_card *card)
links[i].codecs = &snd_soc_dummy_dlc;
links[i].num_codecs = 1;
}
+
+ if (dmi_id && dmi_id->driver_data == (void *)QUIRK_REMAP_DMIC_BT)
+ links[i].id = DMIC_BE_ID;
i++;
}
@@ -1577,6 +1588,11 @@ int acp_sofdsp_dai_links_create(struct snd_soc_card *card)
links[i].dpcm_capture = 1;
links[i].nonatomic = true;
links[i].no_pcm = 1;
+
+ if (dmi_id && dmi_id->driver_data == (void *)QUIRK_REMAP_DMIC_BT) {
+ links[i].id = BT_BE_ID;
+ dev_dbg(dev, "quirk REMAP_DMIC_BT enabled\n");
+ }
}
card->dai_link = links;
diff --git a/sound/soc/amd/acp/acp-mach.h b/sound/soc/amd/acp/acp-mach.h
index 93d9e3886b7e..4b255cbde9ff 100644
--- a/sound/soc/amd/acp/acp-mach.h
+++ b/sound/soc/amd/acp/acp-mach.h
@@ -24,6 +24,10 @@
#define acp_get_drvdata(card) ((struct acp_card_drvdata *)(card)->drvdata)
+/* List of DMI quirks - check acp-mach-common.c for usage. */
+#define QUIRK_TDM_MODE_ENABLE 1
+#define QUIRK_REMAP_DMIC_BT 2
+
enum be_id {
HEADSET_BE_ID = 0,
AMP_BE_ID,
diff --git a/sound/soc/amd/acp/acp-sof-mach.c b/sound/soc/amd/acp/acp-sof-mach.c
index f36750167fa2..4c069a34fbe1 100644
--- a/sound/soc/amd/acp/acp-sof-mach.c
+++ b/sound/soc/amd/acp/acp-sof-mach.c
@@ -113,7 +113,7 @@ static int acp_sof_probe(struct platform_device *pdev)
acp_card_drvdata = card->drvdata;
dmi_id = dmi_first_match(acp_quirk_table);
- if (dmi_id && dmi_id->driver_data)
+ if (dmi_id && dmi_id->driver_data == (void *)QUIRK_TDM_MODE_ENABLE)
acp_card_drvdata->tdm_mode = dmi_id->driver_data;
ret = acp_sofdsp_dai_links_create(card);
diff --git a/sound/soc/codecs/ab8500-codec.c b/sound/soc/codecs/ab8500-codec.c
index 04b5e1d5a653..32d9cdbc8c31 100644
--- a/sound/soc/codecs/ab8500-codec.c
+++ b/sound/soc/codecs/ab8500-codec.c
@@ -2496,13 +2496,13 @@ static int ab8500_codec_probe(struct snd_soc_component *component)
return status;
}
fc = (struct filter_control *)
- &ab8500_filter_controls[AB8500_FILTER_ANC_FIR].private_value;
+ ab8500_filter_controls[AB8500_FILTER_ANC_FIR].private_value;
drvdata->anc_fir_values = (long *)fc->value;
fc = (struct filter_control *)
- &ab8500_filter_controls[AB8500_FILTER_ANC_IIR].private_value;
+ ab8500_filter_controls[AB8500_FILTER_ANC_IIR].private_value;
drvdata->anc_iir_values = (long *)fc->value;
fc = (struct filter_control *)
- &ab8500_filter_controls[AB8500_FILTER_SID_FIR].private_value;
+ ab8500_filter_controls[AB8500_FILTER_SID_FIR].private_value;
drvdata->sid_fir_values = (long *)fc->value;
snd_soc_dapm_disable_pin(dapm, "ANC Configure Input");
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index 5461a8b44a09..e4079e154202 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -54,6 +54,9 @@ static int fsl_easrc_iec958_put_bits(struct snd_kcontrol *kcontrol,
unsigned int regval = ucontrol->value.integer.value[0];
int ret;
+ if (regval < EASRC_WIDTH_16_BIT || regval > EASRC_WIDTH_24_BIT)
+ return -EINVAL;
+
ret = (easrc_priv->bps_iec958[mc->regbase] != regval);
easrc_priv->bps_iec958[mc->regbase] = regval;
@@ -70,8 +73,16 @@ static int fsl_easrc_iec958_get_bits(struct snd_kcontrol *kcontrol,
struct soc_mreg_control *mc =
(struct soc_mreg_control *)kcontrol->private_value;
- ucontrol->value.enumerated.item[0] = easrc_priv->bps_iec958[mc->regbase];
+ ucontrol->value.integer.value[0] = easrc_priv->bps_iec958[mc->regbase];
+
+ return 0;
+}
+static int fsl_easrc_iec958_info(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_info *uinfo)
+{
+ uinfo->type = SNDRV_CTL_ELEM_TYPE_IEC958;
+ uinfo->count = 1;
return 0;
}
@@ -81,11 +92,33 @@ static int fsl_easrc_get_reg(struct snd_kcontrol *kcontrol,
struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
struct soc_mreg_control *mc =
(struct soc_mreg_control *)kcontrol->private_value;
- unsigned int regval;
+ struct fsl_asrc *easrc = snd_soc_component_get_drvdata(component);
+ unsigned int *regval = (unsigned int *)ucontrol->value.iec958.status;
+ int ret;
+
+ ret = regmap_read(easrc->regmap, REG_EASRC_CS0(mc->regbase), ®val[0]);
+ if (ret)
+ return ret;
- regval = snd_soc_component_read(component, mc->regbase);
+ ret = regmap_read(easrc->regmap, REG_EASRC_CS1(mc->regbase), ®val[1]);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(easrc->regmap, REG_EASRC_CS2(mc->regbase), ®val[2]);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(easrc->regmap, REG_EASRC_CS3(mc->regbase), ®val[3]);
+ if (ret)
+ return ret;
+
+ ret = regmap_read(easrc->regmap, REG_EASRC_CS4(mc->regbase), ®val[4]);
+ if (ret)
+ return ret;
- ucontrol->value.integer.value[0] = regval;
+ ret = regmap_read(easrc->regmap, REG_EASRC_CS5(mc->regbase), ®val[5]);
+ if (ret)
+ return ret;
return 0;
}
@@ -97,22 +130,62 @@ static int fsl_easrc_set_reg(struct snd_kcontrol *kcontrol,
struct soc_mreg_control *mc =
(struct soc_mreg_control *)kcontrol->private_value;
struct fsl_asrc *easrc = snd_soc_component_get_drvdata(component);
- unsigned int regval = ucontrol->value.integer.value[0];
- bool changed;
+ unsigned int *regval = (unsigned int *)ucontrol->value.iec958.status;
+ bool changed, changed_all = false;
int ret;
- ret = regmap_update_bits_check(easrc->regmap, mc->regbase,
- GENMASK(31, 0), regval, &changed);
- if (ret != 0)
+ ret = pm_runtime_resume_and_get(component->dev);
+ if (ret)
return ret;
- return changed;
+ ret = regmap_update_bits_check(easrc->regmap, REG_EASRC_CS0(mc->regbase),
+ GENMASK(31, 0), regval[0], &changed);
+ if (ret != 0)
+ goto err;
+ changed_all |= changed;
+
+ ret = regmap_update_bits_check(easrc->regmap, REG_EASRC_CS1(mc->regbase),
+ GENMASK(31, 0), regval[1], &changed);
+ if (ret != 0)
+ goto err;
+ changed_all |= changed;
+
+ ret = regmap_update_bits_check(easrc->regmap, REG_EASRC_CS2(mc->regbase),
+ GENMASK(31, 0), regval[2], &changed);
+ if (ret != 0)
+ goto err;
+ changed_all |= changed;
+
+ ret = regmap_update_bits_check(easrc->regmap, REG_EASRC_CS3(mc->regbase),
+ GENMASK(31, 0), regval[3], &changed);
+ if (ret != 0)
+ goto err;
+ changed_all |= changed;
+
+ ret = regmap_update_bits_check(easrc->regmap, REG_EASRC_CS4(mc->regbase),
+ GENMASK(31, 0), regval[4], &changed);
+ if (ret != 0)
+ goto err;
+ changed_all |= changed;
+
+ ret = regmap_update_bits_check(easrc->regmap, REG_EASRC_CS5(mc->regbase),
+ GENMASK(31, 0), regval[5], &changed);
+ if (ret != 0)
+ goto err;
+ changed_all |= changed;
+err:
+ pm_runtime_put_autosuspend(component->dev);
+
+ if (ret != 0)
+ return ret;
+ else
+ return changed_all;
}
#define SOC_SINGLE_REG_RW(xname, xreg) \
{ .iface = SNDRV_CTL_ELEM_IFACE_PCM, .name = (xname), \
.access = SNDRV_CTL_ELEM_ACCESS_READWRITE, \
- .info = snd_soc_info_xr_sx, .get = fsl_easrc_get_reg, \
+ .info = fsl_easrc_iec958_info, .get = fsl_easrc_get_reg, \
.put = fsl_easrc_set_reg, \
.private_value = (unsigned long)&(struct soc_mreg_control) \
{ .regbase = xreg, .regcount = 1, .nbits = 32, \
@@ -143,30 +216,10 @@ static const struct snd_kcontrol_new fsl_easrc_snd_controls[] = {
SOC_SINGLE_VAL_RW("Context 2 IEC958 Bits Per Sample", 2),
SOC_SINGLE_VAL_RW("Context 3 IEC958 Bits Per Sample", 3),
- SOC_SINGLE_REG_RW("Context 0 IEC958 CS0", REG_EASRC_CS0(0)),
- SOC_SINGLE_REG_RW("Context 1 IEC958 CS0", REG_EASRC_CS0(1)),
- SOC_SINGLE_REG_RW("Context 2 IEC958 CS0", REG_EASRC_CS0(2)),
- SOC_SINGLE_REG_RW("Context 3 IEC958 CS0", REG_EASRC_CS0(3)),
- SOC_SINGLE_REG_RW("Context 0 IEC958 CS1", REG_EASRC_CS1(0)),
- SOC_SINGLE_REG_RW("Context 1 IEC958 CS1", REG_EASRC_CS1(1)),
- SOC_SINGLE_REG_RW("Context 2 IEC958 CS1", REG_EASRC_CS1(2)),
- SOC_SINGLE_REG_RW("Context 3 IEC958 CS1", REG_EASRC_CS1(3)),
- SOC_SINGLE_REG_RW("Context 0 IEC958 CS2", REG_EASRC_CS2(0)),
- SOC_SINGLE_REG_RW("Context 1 IEC958 CS2", REG_EASRC_CS2(1)),
- SOC_SINGLE_REG_RW("Context 2 IEC958 CS2", REG_EASRC_CS2(2)),
- SOC_SINGLE_REG_RW("Context 3 IEC958 CS2", REG_EASRC_CS2(3)),
- SOC_SINGLE_REG_RW("Context 0 IEC958 CS3", REG_EASRC_CS3(0)),
- SOC_SINGLE_REG_RW("Context 1 IEC958 CS3", REG_EASRC_CS3(1)),
- SOC_SINGLE_REG_RW("Context 2 IEC958 CS3", REG_EASRC_CS3(2)),
- SOC_SINGLE_REG_RW("Context 3 IEC958 CS3", REG_EASRC_CS3(3)),
- SOC_SINGLE_REG_RW("Context 0 IEC958 CS4", REG_EASRC_CS4(0)),
- SOC_SINGLE_REG_RW("Context 1 IEC958 CS4", REG_EASRC_CS4(1)),
- SOC_SINGLE_REG_RW("Context 2 IEC958 CS4", REG_EASRC_CS4(2)),
- SOC_SINGLE_REG_RW("Context 3 IEC958 CS4", REG_EASRC_CS4(3)),
- SOC_SINGLE_REG_RW("Context 0 IEC958 CS5", REG_EASRC_CS5(0)),
- SOC_SINGLE_REG_RW("Context 1 IEC958 CS5", REG_EASRC_CS5(1)),
- SOC_SINGLE_REG_RW("Context 2 IEC958 CS5", REG_EASRC_CS5(2)),
- SOC_SINGLE_REG_RW("Context 3 IEC958 CS5", REG_EASRC_CS5(3)),
+ SOC_SINGLE_REG_RW("Context 0 IEC958 CS", 0),
+ SOC_SINGLE_REG_RW("Context 1 IEC958 CS", 1),
+ SOC_SINGLE_REG_RW("Context 2 IEC958 CS", 2),
+ SOC_SINGLE_REG_RW("Context 3 IEC958 CS", 3),
};
/*
diff --git a/sound/soc/fsl/fsl_micfil.c b/sound/soc/fsl/fsl_micfil.c
index 7cfe77b57b3c..6998d30af4c4 100644
--- a/sound/soc/fsl/fsl_micfil.c
+++ b/sound/soc/fsl/fsl_micfil.c
@@ -181,10 +181,34 @@ static int micfil_quality_set(struct snd_kcontrol *kcontrol,
{
struct snd_soc_component *cmpnt = snd_soc_kcontrol_component(kcontrol);
struct fsl_micfil *micfil = snd_soc_component_get_drvdata(cmpnt);
+ int val = ucontrol->value.integer.value[0];
+ bool change = false;
+ int old_val;
+ int ret;
+
+ if (val < QUALITY_HIGH || val > QUALITY_VLOW2)
+ return -EINVAL;
+
+ if (micfil->quality != val) {
+ ret = pm_runtime_resume_and_get(cmpnt->dev);
+ if (ret)
+ return ret;
+
+ old_val = micfil->quality;
+ micfil->quality = val;
+ ret = micfil_set_quality(micfil);
- micfil->quality = ucontrol->value.integer.value[0];
+ pm_runtime_put_autosuspend(cmpnt->dev);
- return micfil_set_quality(micfil);
+ if (ret) {
+ micfil->quality = old_val;
+ return ret;
+ }
+
+ change = true;
+ }
+
+ return change;
}
static const char * const micfil_hwvad_enable[] = {
@@ -243,6 +267,10 @@ static int micfil_put_dc_remover_state(struct snd_kcontrol *kcontrol,
if (val < 0 || val > 3)
return -EINVAL;
+ ret = pm_runtime_resume_and_get(comp->dev);
+ if (ret)
+ return ret;
+
micfil->dc_remover = val;
/* Calculate total value for all channels */
@@ -252,10 +280,10 @@ static int micfil_put_dc_remover_state(struct snd_kcontrol *kcontrol,
/* Update DC Remover mode for all channels */
ret = snd_soc_component_update_bits(comp, REG_MICFIL_DC_CTRL,
MICFIL_DC_CTRL_CONFIG, reg_val);
- if (ret < 0)
- return ret;
- return 0;
+ pm_runtime_put_autosuspend(comp->dev);
+
+ return ret;
}
static int micfil_get_dc_remover_state(struct snd_kcontrol *kcontrol,
@@ -277,10 +305,15 @@ static int hwvad_put_enable(struct snd_kcontrol *kcontrol,
unsigned int *item = ucontrol->value.enumerated.item;
struct fsl_micfil *micfil = snd_soc_component_get_drvdata(comp);
int val = snd_soc_enum_item_to_val(e, item[0]);
+ bool change = false;
+ if (val < 0 || val > 1)
+ return -EINVAL;
+
+ change = (micfil->vad_enabled != val);
micfil->vad_enabled = val;
- return 0;
+ return change;
}
static int hwvad_get_enable(struct snd_kcontrol *kcontrol,
@@ -302,13 +335,18 @@ static int hwvad_put_init_mode(struct snd_kcontrol *kcontrol,
unsigned int *item = ucontrol->value.enumerated.item;
struct fsl_micfil *micfil = snd_soc_component_get_drvdata(comp);
int val = snd_soc_enum_item_to_val(e, item[0]);
+ bool change = false;
+
+ if (val < MICFIL_HWVAD_ENVELOPE_MODE || val > MICFIL_HWVAD_ENERGY_MODE)
+ return -EINVAL;
/* 0 - Envelope-based Mode
* 1 - Energy-based Mode
*/
+ change = (micfil->vad_init_mode != val);
micfil->vad_init_mode = val;
- return 0;
+ return change;
}
static int hwvad_get_init_mode(struct snd_kcontrol *kcontrol,
@@ -395,7 +433,13 @@ static const struct snd_kcontrol_new fsl_micfil_snd_controls[] = {
SOC_SINGLE("HWVAD ZCD Adjustment", REG_MICFIL_VAD0_ZCD, 8, 15, 0),
SOC_SINGLE("HWVAD ZCD And Behavior Switch",
REG_MICFIL_VAD0_ZCD, 4, 1, 0),
- SOC_SINGLE_BOOL_EXT("VAD Detected", 0, hwvad_detected, NULL),
+ {
+ .iface = SNDRV_CTL_ELEM_IFACE_MIXER,
+ .access = SNDRV_CTL_ELEM_ACCESS_READ | SNDRV_CTL_ELEM_ACCESS_VOLATILE,
+ .name = "VAD Detected",
+ .info = snd_soc_info_bool_ext,
+ .get = hwvad_detected,
+ },
};
static int fsl_micfil_use_verid(struct device *dev)
diff --git a/sound/soc/fsl/fsl_xcvr.c b/sound/soc/fsl/fsl_xcvr.c
index 656a4d619cdf..a042bae63ab3 100644
--- a/sound/soc/fsl/fsl_xcvr.c
+++ b/sound/soc/fsl/fsl_xcvr.c
@@ -108,10 +108,17 @@ static int fsl_xcvr_arc_mode_put(struct snd_kcontrol *kcontrol,
struct fsl_xcvr *xcvr = snd_soc_dai_get_drvdata(dai);
struct soc_enum *e = (struct soc_enum *)kcontrol->private_value;
unsigned int *item = ucontrol->value.enumerated.item;
+ int val = snd_soc_enum_item_to_val(e, item[0]);
+ int ret;
- xcvr->arc_mode = snd_soc_enum_item_to_val(e, item[0]);
+ if (val < 0 || val > 1)
+ return -EINVAL;
- return 0;
+ ret = (xcvr->arc_mode != val);
+
+ xcvr->arc_mode = val;
+
+ return ret;
}
static int fsl_xcvr_arc_mode_get(struct snd_kcontrol *kcontrol,
@@ -211,10 +218,17 @@ static int fsl_xcvr_mode_put(struct snd_kcontrol *kcontrol,
struct fsl_xcvr *xcvr = snd_soc_dai_get_drvdata(dai);
struct soc_enum *e = (struct soc_enum *)kcontrol->private_value;
unsigned int *item = ucontrol->value.enumerated.item;
+ int val = snd_soc_enum_item_to_val(e, item[0]);
struct snd_soc_card *card = dai->component->card;
struct snd_soc_pcm_runtime *rtd;
+ int ret;
+
+ if (val < FSL_XCVR_MODE_SPDIF || val > FSL_XCVR_MODE_EARC)
+ return -EINVAL;
- xcvr->mode = snd_soc_enum_item_to_val(e, item[0]);
+ ret = (xcvr->mode != val);
+
+ xcvr->mode = val;
fsl_xcvr_activate_ctl(dai, fsl_xcvr_arc_mode_kctl.name,
(xcvr->mode == FSL_XCVR_MODE_ARC));
@@ -224,7 +238,7 @@ static int fsl_xcvr_mode_put(struct snd_kcontrol *kcontrol,
rtd = snd_soc_get_pcm_runtime(card, card->dai_link);
rtd->pcm->streams[SNDRV_PCM_STREAM_PLAYBACK].substream_count =
(xcvr->mode == FSL_XCVR_MODE_SPDIF ? 1 : 0);
- return 0;
+ return ret;
}
static int fsl_xcvr_mode_get(struct snd_kcontrol *kcontrol,
diff --git a/sound/soc/mediatek/mt8188/mt8188-dai-pcm.c b/sound/soc/mediatek/mt8188/mt8188-dai-pcm.c
index 5bc854a8f3df..8ca7cc75e21d 100644
--- a/sound/soc/mediatek/mt8188/mt8188-dai-pcm.c
+++ b/sound/soc/mediatek/mt8188/mt8188-dai-pcm.c
@@ -128,7 +128,7 @@ static int mtk_dai_pcm_configure(struct snd_pcm_substream *substream,
unsigned int lrck_inv;
unsigned int bck_inv;
unsigned int fmt;
- unsigned int bit_width = dai->sample_bits;
+ unsigned int bit_width = dai->symmetric_sample_bits;
unsigned int val = 0;
unsigned int mask = 0;
int fs = 0;
diff --git a/sound/soc/mediatek/mt8195/mt8195-dai-pcm.c b/sound/soc/mediatek/mt8195/mt8195-dai-pcm.c
index 6d6d79300d51..cdc16057d50e 100644
--- a/sound/soc/mediatek/mt8195/mt8195-dai-pcm.c
+++ b/sound/soc/mediatek/mt8195/mt8195-dai-pcm.c
@@ -127,7 +127,7 @@ static int mtk_dai_pcm_configure(struct snd_pcm_substream *substream,
unsigned int lrck_inv;
unsigned int bck_inv;
unsigned int fmt;
- unsigned int bit_width = dai->sample_bits;
+ unsigned int bit_width = dai->symmetric_sample_bits;
unsigned int val = 0;
unsigned int mask = 0;
int fs = 0;
diff --git a/sound/soc/mediatek/mt8365/mt8365-dai-dmic.c b/sound/soc/mediatek/mt8365/mt8365-dai-dmic.c
index f9945c2a2cd1..0bac143b48bf 100644
--- a/sound/soc/mediatek/mt8365/mt8365-dai-dmic.c
+++ b/sound/soc/mediatek/mt8365/mt8365-dai-dmic.c
@@ -118,13 +118,13 @@ static int mt8365_dai_configure_dmic(struct mtk_base_afe *afe,
unsigned int clk_phase_sel_ch1 = dmic_data->clk_phase_sel_ch1;
unsigned int clk_phase_sel_ch2 = dmic_data->clk_phase_sel_ch2;
unsigned int val = 0;
- unsigned int rate = dai->rate;
- int reg = get_chan_reg(dai->channels);
+ unsigned int rate = dai->symmetric_rate;
+ int reg = get_chan_reg(dai->symmetric_channels);
if (reg < 0)
return -EINVAL;
- dmic_data->dmic_channel = dai->channels;
+ dmic_data->dmic_channel = dai->symmetric_channels;
val |= DMIC_TOP_CON_SDM3_LEVEL_MODE;
diff --git a/sound/soc/mediatek/mt8365/mt8365-dai-pcm.c b/sound/soc/mediatek/mt8365/mt8365-dai-pcm.c
index f85ec07249c3..3373b88da28e 100644
--- a/sound/soc/mediatek/mt8365/mt8365-dai-pcm.c
+++ b/sound/soc/mediatek/mt8365/mt8365-dai-pcm.c
@@ -44,7 +44,7 @@ static int mt8365_dai_configure_pcm1(struct snd_pcm_substream *substream,
bool lrck_inv = pcm_priv->lrck_inv;
bool bck_inv = pcm_priv->bck_inv;
unsigned int fmt = pcm_priv->format;
- unsigned int bit_width = dai->sample_bits;
+ unsigned int bit_width = dai->symmetric_sample_bits;
unsigned int val = 0;
if (!slave_mode) {
diff --git a/sound/soc/qcom/qdsp6/topology.c b/sound/soc/qcom/qdsp6/topology.c
index 01bb1bdee5ce..6f5534b8092b 100644
--- a/sound/soc/qcom/qdsp6/topology.c
+++ b/sound/soc/qcom/qdsp6/topology.c
@@ -930,9 +930,6 @@ static int audioreach_widget_unload(struct snd_soc_component *scomp,
struct audioreach_container *cont;
struct audioreach_module *mod;
- mod = dobj->private;
- cont = mod->container;
-
if (w->id == snd_soc_dapm_mixer) {
/* virtual widget */
struct snd_ar_control *scontrol = dobj->private;
@@ -941,6 +938,11 @@ static int audioreach_widget_unload(struct snd_soc_component *scomp,
kfree(scontrol);
return 0;
}
+ mod = dobj->private;
+ if (!mod)
+ return 0;
+
+ cont = mod->container;
mutex_lock(&apm->lock);
idr_remove(&apm->modules_idr, mod->instance_id);
diff --git a/sound/soc/sh/rcar/core.c b/sound/soc/sh/rcar/core.c
index e3ef9104b411..c1d5f9b4d934 100644
--- a/sound/soc/sh/rcar/core.c
+++ b/sound/soc/sh/rcar/core.c
@@ -1978,7 +1978,7 @@ static int rsnd_probe(struct platform_device *pdev)
* asoc register
*/
ci = 0;
- for (i = 0; priv->component_dais[i] > 0; i++) {
+ for (i = 0; i < RSND_MAX_COMPONENT && priv->component_dais[i] > 0; i++) {
int nr = priv->component_dais[i];
ret = devm_snd_soc_register_component(dev, &rsnd_soc_component,
diff --git a/sound/soc/soc-compress.c b/sound/soc/soc-compress.c
index e692aa3b8b22..249cafe54f56 100644
--- a/sound/soc/soc-compress.c
+++ b/sound/soc/soc-compress.c
@@ -69,10 +69,10 @@ static int soc_compr_clean(struct snd_compr_stream *cstream, int rollback)
snd_soc_dai_digital_mute(codec_dai, 1, stream);
if (!snd_soc_dai_active(cpu_dai))
- cpu_dai->rate = 0;
+ soc_pcm_set_dai_params(cpu_dai, NULL);
if (!snd_soc_dai_active(codec_dai))
- codec_dai->rate = 0;
+ soc_pcm_set_dai_params(codec_dai, NULL);
snd_soc_link_compr_shutdown(cstream, rollback);
diff --git a/sound/soc/soc-pcm.c b/sound/soc/soc-pcm.c
index 0e21ff9f7b74..628322790c87 100644
--- a/sound/soc/soc-pcm.c
+++ b/sound/soc/soc-pcm.c
@@ -458,17 +458,17 @@ int dpcm_dapm_stream_event(struct snd_soc_pcm_runtime *fe, int dir,
return 0;
}
-static void soc_pcm_set_dai_params(struct snd_soc_dai *dai,
- struct snd_pcm_hw_params *params)
+void soc_pcm_set_dai_params(struct snd_soc_dai *dai,
+ struct snd_pcm_hw_params *params)
{
if (params) {
- dai->rate = params_rate(params);
- dai->channels = params_channels(params);
- dai->sample_bits = snd_pcm_format_physical_width(params_format(params));
+ dai->symmetric_rate = params_rate(params);
+ dai->symmetric_channels = params_channels(params);
+ dai->symmetric_sample_bits = snd_pcm_format_physical_width(params_format(params));
} else {
- dai->rate = 0;
- dai->channels = 0;
- dai->sample_bits = 0;
+ dai->symmetric_rate = 0;
+ dai->symmetric_channels = 0;
+ dai->symmetric_sample_bits = 0;
}
}
@@ -482,14 +482,14 @@ static int soc_pcm_apply_symmetry(struct snd_pcm_substream *substream,
return 0;
#define __soc_pcm_apply_symmetry(name, NAME) \
- if (soc_dai->name && (soc_dai->driver->symmetric_##name || \
- rtd->dai_link->symmetric_##name)) { \
+ if (soc_dai->symmetric_##name && \
+ (soc_dai->driver->symmetric_##name || rtd->dai_link->symmetric_##name)) { \
dev_dbg(soc_dai->dev, "ASoC: Symmetry forces %s to %d\n",\
- #name, soc_dai->name); \
+ #name, soc_dai->symmetric_##name); \
\
ret = snd_pcm_hw_constraint_single(substream->runtime, \
SNDRV_PCM_HW_PARAM_##NAME,\
- soc_dai->name); \
+ soc_dai->symmetric_##name); \
if (ret < 0) { \
dev_err(soc_dai->dev, \
"ASoC: Unable to apply %s constraint: %d\n",\
@@ -525,9 +525,11 @@ static int soc_pcm_params_symmetry(struct snd_pcm_substream *substream,
if (symmetry) \
for_each_rtd_cpu_dais(rtd, i, cpu_dai) \
if (!snd_soc_dai_is_dummy(cpu_dai) && \
- cpu_dai->xxx && cpu_dai->xxx != d.xxx) { \
+ cpu_dai->symmetric_##xxx && \
+ cpu_dai->symmetric_##xxx != d.symmetric_##xxx) { \
dev_err(rtd->dev, "ASoC: unmatched %s symmetry: %s:%d - %s:%d\n", \
- #xxx, cpu_dai->name, cpu_dai->xxx, d.name, d.xxx); \
+ #xxx, cpu_dai->name, cpu_dai->symmetric_##xxx, \
+ d.name, d.symmetric_##xxx); \
return -EINVAL; \
}
@@ -798,8 +800,7 @@ static int soc_pcm_clean(struct snd_soc_pcm_runtime *rtd,
/* Make sure DAI parameters cleared if the DAI becomes inactive */
for_each_rtd_dais(rtd, i, dai) {
- if (snd_soc_dai_active(dai) == 0 &&
- (dai->rate || dai->channels || dai->sample_bits))
+ if (snd_soc_dai_active(dai) == 0)
soc_pcm_set_dai_params(dai, NULL);
}
}
diff --git a/sound/soc/sof/compress.c b/sound/soc/sof/compress.c
index c469bb706e4a..b64a5d99fe60 100644
--- a/sound/soc/sof/compress.c
+++ b/sound/soc/sof/compress.c
@@ -247,6 +247,7 @@ static int sof_compr_set_params(struct snd_soc_component *component,
sstream->sampling_rate = params->codec.sample_rate;
sstream->channels = params->codec.ch_out;
sstream->sample_container_bytes = pcm->params.sample_container_bytes;
+ sstream->codec_params = params->codec;
spcm->prepared[cstream->direction] = true;
@@ -259,9 +260,10 @@ static int sof_compr_set_params(struct snd_soc_component *component,
static int sof_compr_get_params(struct snd_soc_component *component,
struct snd_compr_stream *cstream, struct snd_codec *params)
{
- /* TODO: we don't query the supported codecs for now, if the
- * application asks for an unsupported codec the set_params() will fail.
- */
+ struct sof_compr_stream *sstream = cstream->runtime->private_data;
+
+ *params = sstream->codec_params;
+
return 0;
}
diff --git a/sound/soc/sof/intel/hda-stream.c b/sound/soc/sof/intel/hda-stream.c
index 2be0d02f9cf9..d6a0e78b5ce7 100644
--- a/sound/soc/sof/intel/hda-stream.c
+++ b/sound/soc/sof/intel/hda-stream.c
@@ -419,16 +419,20 @@ int hda_dsp_iccmax_stream_hw_params(struct snd_sof_dev *sdev, struct hdac_ext_st
struct snd_dma_buffer *dmab,
struct snd_pcm_hw_params *params)
{
- struct hdac_stream *hstream = &hext_stream->hstream;
- int sd_offset = SOF_STREAM_SD_OFFSET(hstream);
+ struct hdac_stream *hstream;
+ int sd_offset;
int ret;
- u32 mask = 0x1 << hstream->index;
+ u32 mask;
if (!hext_stream) {
dev_err(sdev->dev, "error: no stream available\n");
return -ENODEV;
}
+ hstream = &hext_stream->hstream;
+ sd_offset = SOF_STREAM_SD_OFFSET(hstream);
+ mask = 0x1 << hstream->index;
+
if (!dmab) {
dev_err(sdev->dev, "error: no dma buffer allocated!\n");
return -ENODEV;
diff --git a/sound/soc/sof/ipc3.c b/sound/soc/sof/ipc3.c
index 83c22d4a4830..7de5e3d285e7 100644
--- a/sound/soc/sof/ipc3.c
+++ b/sound/soc/sof/ipc3.c
@@ -226,7 +226,7 @@ static inline void ipc3_log_header(struct device *dev, u8 *text, u32 cmd)
static void sof_ipc3_dump_payload(struct snd_sof_dev *sdev,
void *ipc_data, size_t size)
{
- printk(KERN_DEBUG "Size of payload following the header: %zu\n", size);
+ dev_dbg(sdev->dev, "Size of payload following the header: %zu\n", size);
print_hex_dump_debug("Message payload: ", DUMP_PREFIX_OFFSET,
16, 4, ipc_data, size, false);
}
diff --git a/sound/soc/sof/sof-priv.h b/sound/soc/sof/sof-priv.h
index 843be3b6415d..53bcc627ae8b 100644
--- a/sound/soc/sof/sof-priv.h
+++ b/sound/soc/sof/sof-priv.h
@@ -17,6 +17,7 @@
#include <sound/sof/info.h>
#include <sound/sof/pm.h>
#include <sound/sof/trace.h>
+#include <sound/compress_params.h>
#include <uapi/sound/sof/fw.h>
#include <sound/sof/ext_manifest.h>
@@ -119,6 +120,7 @@ struct sof_compr_stream {
u32 sampling_rate;
u16 channels;
u16 sample_container_bytes;
+ struct snd_codec codec_params;
size_t posn_offset;
};
diff --git a/sound/soc/sti/uniperif_player.c b/sound/soc/sti/uniperif_player.c
index 6d1ce030963c..45d35b887e4e 100644
--- a/sound/soc/sti/uniperif_player.c
+++ b/sound/soc/sti/uniperif_player.c
@@ -1028,8 +1028,13 @@ static int uni_player_parse_dt_audio_glue(struct platform_device *pdev,
return PTR_ERR(regmap);
}
- player->clk_sel = regmap_field_alloc(regmap, regfield[0]);
- player->valid_sel = regmap_field_alloc(regmap, regfield[1]);
+ player->clk_sel = devm_regmap_field_alloc(&pdev->dev, regmap, regfield[0]);
+ if (IS_ERR(player->clk_sel))
+ return PTR_ERR(player->clk_sel);
+
+ player->valid_sel = devm_regmap_field_alloc(&pdev->dev, regmap, regfield[1]);
+ if (IS_ERR(player->valid_sel))
+ return PTR_ERR(player->valid_sel);
return 0;
}
diff --git a/sound/usb/midi.c b/sound/usb/midi.c
index 461e183680da..5917f4da9776 100644
--- a/sound/usb/midi.c
+++ b/sound/usb/midi.c
@@ -1959,15 +1959,17 @@ static struct usb_ms_endpoint_descriptor *find_usb_ms_endpoint_descriptor(
while (extralen > 3) {
struct usb_ms_endpoint_descriptor *ms_ep =
(struct usb_ms_endpoint_descriptor *)extra;
+ int length = ms_ep->bLength;
- if (ms_ep->bLength > 3 &&
+ if (!length || length > extralen)
+ break;
+
+ if (length > 3 &&
ms_ep->bDescriptorType == USB_DT_CS_ENDPOINT &&
ms_ep->bDescriptorSubtype == UAC_MS_GENERAL)
return ms_ep;
- if (!extra[0])
- break;
- extralen -= extra[0];
- extra += extra[0];
+ extralen -= length;
+ extra += length;
}
return NULL;
}
diff --git a/sound/usb/midi2.c b/sound/usb/midi2.c
index caade4406f52..6f04aa7f685b 100644
--- a/sound/usb/midi2.c
+++ b/sound/usb/midi2.c
@@ -504,15 +504,17 @@ static void *find_usb_ms_endpoint_descriptor(struct usb_host_endpoint *hostep,
while (extralen > 3) {
struct usb_ms_endpoint_descriptor *ms_ep =
(struct usb_ms_endpoint_descriptor *)extra;
+ int length = ms_ep->bLength;
- if (ms_ep->bLength > 3 &&
+ if (!length || length > extralen)
+ break;
+
+ if (length > 3 &&
ms_ep->bDescriptorType == USB_DT_CS_ENDPOINT &&
ms_ep->bDescriptorSubtype == subtype)
return ms_ep;
- if (!extra[0])
- break;
- extralen -= extra[0];
- extra += extra[0];
+ extralen -= length;
+ extra += length;
}
return NULL;
}
diff --git a/sound/usb/mixer_scarlett2.c b/sound/usb/mixer_scarlett2.c
index fe1d6e512699..ef5945aa40e4 100644
--- a/sound/usb/mixer_scarlett2.c
+++ b/sound/usb/mixer_scarlett2.c
@@ -2221,7 +2221,7 @@ static const struct scarlett2_device_entry scarlett2_devices[] = {
{ USB_ID(0x1235, 0x820c), &clarett_8pre_info, "Clarett+" },
/* End of list */
- { 0, NULL },
+ { 0, NULL, NULL },
};
/* get the starting port index number for a given port type/direction */
diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c
index fb81dcd6ca2a..489dd84e2096 100644
--- a/sound/usb/quirks.c
+++ b/sound/usb/quirks.c
@@ -122,7 +122,7 @@ static int add_audio_stream_from_fixed_fmt(struct snd_usb_audio *chip,
snd_usb_audioformat_set_sync_ep(chip, fp);
- err = snd_usb_add_audio_stream(chip, stream, fp);
+ err = snd_usb_add_audio_stream(chip, stream, fp, NULL);
if (err < 0)
return err;
diff --git a/sound/usb/stream.c b/sound/usb/stream.c
index 8e8c99f21abf..08e2ad14aa6d 100644
--- a/sound/usb/stream.c
+++ b/sound/usb/stream.c
@@ -79,7 +79,7 @@ static void snd_usb_audio_pcm_free(struct snd_pcm *pcm)
static void snd_usb_init_substream(struct snd_usb_stream *as,
int stream,
struct audioformat *fp,
- struct snd_usb_power_domain *pd)
+ struct snd_usb_power_domain **pdptr)
{
struct snd_usb_substream *subs = &as->substream[stream];
@@ -105,10 +105,11 @@ static void snd_usb_init_substream(struct snd_usb_stream *as,
if (fp->channels > subs->channels_max)
subs->channels_max = fp->channels;
- if (pd) {
- subs->str_pd = pd;
+ if (pdptr && *pdptr) {
+ subs->str_pd = *pdptr;
+ *pdptr = NULL; /* assigned */
/* Initialize Power Domain to idle status D1 */
- snd_usb_power_domain_set(subs->stream->chip, pd,
+ snd_usb_power_domain_set(subs->stream->chip, subs->str_pd,
UAC3_PD_STATE_D1);
}
@@ -486,11 +487,14 @@ snd_pcm_chmap_elem *convert_chmap_v3(struct uac3_cluster_header_descriptor
* if not, create a new pcm stream. note, fp is added to the substream
* fmt_list and will be freed on the chip instance release. do not free
* fp or do remove it from the substream fmt_list to avoid double-free.
+ *
+ * pdptr is optional and can be NULL. When it's non-NULL and the PD gets
+ * assigned to the stream, *pdptr is cleared to NULL upon return.
*/
-static int __snd_usb_add_audio_stream(struct snd_usb_audio *chip,
- int stream,
- struct audioformat *fp,
- struct snd_usb_power_domain *pd)
+int snd_usb_add_audio_stream(struct snd_usb_audio *chip,
+ int stream,
+ struct audioformat *fp,
+ struct snd_usb_power_domain **pdptr)
{
struct snd_usb_stream *as;
@@ -523,7 +527,7 @@ static int __snd_usb_add_audio_stream(struct snd_usb_audio *chip,
err = snd_pcm_new_stream(as->pcm, stream, 1);
if (err < 0)
return err;
- snd_usb_init_substream(as, stream, fp, pd);
+ snd_usb_init_substream(as, stream, fp, pdptr);
return add_chmap(as->pcm, stream, subs);
}
@@ -551,7 +555,7 @@ static int __snd_usb_add_audio_stream(struct snd_usb_audio *chip,
else
strcpy(pcm->name, "USB Audio");
- snd_usb_init_substream(as, stream, fp, pd);
+ snd_usb_init_substream(as, stream, fp, pdptr);
/*
* Keep using head insertion for M-Audio Audiophile USB (tm) which has a
@@ -569,21 +573,6 @@ static int __snd_usb_add_audio_stream(struct snd_usb_audio *chip,
return add_chmap(pcm, stream, &as->substream[stream]);
}
-int snd_usb_add_audio_stream(struct snd_usb_audio *chip,
- int stream,
- struct audioformat *fp)
-{
- return __snd_usb_add_audio_stream(chip, stream, fp, NULL);
-}
-
-static int snd_usb_add_audio_stream_v3(struct snd_usb_audio *chip,
- int stream,
- struct audioformat *fp,
- struct snd_usb_power_domain *pd)
-{
- return __snd_usb_add_audio_stream(chip, stream, fp, pd);
-}
-
static int parse_uac_endpoint_attributes(struct snd_usb_audio *chip,
struct usb_host_interface *alts,
int protocol, int iface_no)
@@ -1108,8 +1097,7 @@ snd_usb_get_audioformat_uac3(struct snd_usb_audio *chip,
}
}
- if (pd)
- *pd_out = pd;
+ *pd_out = pd;
return fp;
}
@@ -1124,7 +1112,6 @@ static int __snd_usb_parse_audio_interface(struct snd_usb_audio *chip,
struct usb_interface_descriptor *altsd;
int i, altno, err, stream;
struct audioformat *fp = NULL;
- struct snd_usb_power_domain *pd = NULL;
bool set_iface_first;
int num, protocol;
@@ -1166,6 +1153,12 @@ static int __snd_usb_parse_audio_interface(struct snd_usb_audio *chip,
if (snd_usb_apply_interface_quirk(chip, iface_no, altno))
continue;
+ /* pd may be allocated at snd_usb_get_audioformat_uac3() and
+ * assigned at snd_usb_add_audio_stream(); otherwise it'll be
+ * freed automatically by cleanup at each loop.
+ */
+ struct snd_usb_power_domain *pd __free(kfree) = NULL;
+
/*
* Roland audio streaming interfaces are marked with protocols
* 0/1/2, but are UAC 1 compatible.
@@ -1221,23 +1214,16 @@ static int __snd_usb_parse_audio_interface(struct snd_usb_audio *chip,
*has_non_pcm = true;
if ((fp->fmt_type == UAC_FORMAT_TYPE_I) == non_pcm) {
audioformat_free(fp);
- kfree(pd);
fp = NULL;
- pd = NULL;
continue;
}
snd_usb_audioformat_set_sync_ep(chip, fp);
dev_dbg(&dev->dev, "%u:%d: add audio endpoint %#x\n", iface_no, altno, fp->endpoint);
- if (protocol == UAC_VERSION_3)
- err = snd_usb_add_audio_stream_v3(chip, stream, fp, pd);
- else
- err = snd_usb_add_audio_stream(chip, stream, fp);
-
+ err = snd_usb_add_audio_stream(chip, stream, fp, &pd);
if (err < 0) {
audioformat_free(fp);
- kfree(pd);
return err;
}
diff --git a/sound/usb/stream.h b/sound/usb/stream.h
index d92e18d5818f..61b9a133da01 100644
--- a/sound/usb/stream.h
+++ b/sound/usb/stream.h
@@ -7,7 +7,8 @@ int snd_usb_parse_audio_interface(struct snd_usb_audio *chip,
int snd_usb_add_audio_stream(struct snd_usb_audio *chip,
int stream,
- struct audioformat *fp);
+ struct audioformat *fp,
+ struct snd_usb_power_domain **pdptr);
#endif /* __USBAUDIO_STREAM_H */
diff --git a/tools/lib/bpf/relo_core.c b/tools/lib/bpf/relo_core.c
index 63a4d5ad12d1..04c8febfc0aa 100644
--- a/tools/lib/bpf/relo_core.c
+++ b/tools/lib/bpf/relo_core.c
@@ -293,6 +293,8 @@ int bpf_core_parse_spec(const char *prog_name, const struct btf *btf,
++spec_str;
if (sscanf(spec_str, "%d%n", &access_idx, &parsed_len) != 1)
return -EINVAL;
+ if (access_idx < 0)
+ return -EINVAL;
if (spec->raw_len == BPF_CORE_SPEC_MAX_LEN)
return -E2BIG;
spec_str += parsed_len;
diff --git a/tools/perf/builtin-diff.c b/tools/perf/builtin-diff.c
index 23326dd20333..82fb7773e03e 100644
--- a/tools/perf/builtin-diff.c
+++ b/tools/perf/builtin-diff.c
@@ -469,13 +469,13 @@ static int diff__process_sample_event(const struct perf_tool *tool,
static struct perf_diff pdiff;
-static struct evsel *evsel_match(struct evsel *evsel,
- struct evlist *evlist)
+static struct evsel *evsel_match(struct evsel *evsel, struct evlist *evlist)
{
struct evsel *e;
evlist__for_each_entry(evlist, e) {
- if (evsel__match2(evsel, e))
+ if ((evsel->core.attr.type == e->core.attr.type) &&
+ (evsel->core.attr.config == e->core.attr.config))
return e;
}
diff --git a/tools/perf/builtin-list.c b/tools/perf/builtin-list.c
index c5331721dfee..9e7fdfcdd7ff 100644
--- a/tools/perf/builtin-list.c
+++ b/tools/perf/builtin-list.c
@@ -19,6 +19,7 @@
#include "util/string2.h"
#include "util/strlist.h"
#include "util/strbuf.h"
+#include "util/tool_pmu.h"
#include <subcmd/pager.h>
#include <subcmd/parse-options.h>
#include <linux/zalloc.h>
@@ -614,9 +615,18 @@ int cmd_list(int argc, const char **argv)
event_symbols_hw, PERF_COUNT_HW_MAX);
else if (strcmp(argv[i], "sw") == 0 ||
strcmp(argv[i], "software") == 0) {
+ char *old_pmu_glob = default_ps.pmu_glob;
+
print_symbol_events(&print_cb, ps, PERF_TYPE_SOFTWARE,
event_symbols_sw, PERF_COUNT_SW_MAX);
- print_tool_events(&print_cb, ps);
+ default_ps.pmu_glob = strdup("tool");
+ if (!default_ps.pmu_glob) {
+ ret = -1;
+ goto out;
+ }
+ perf_pmus__print_pmu_events(&print_cb, ps);
+ zfree(&default_ps.pmu_glob);
+ default_ps.pmu_glob = old_pmu_glob;
} else if (strcmp(argv[i], "cache") == 0 ||
strcmp(argv[i], "hwcache") == 0)
print_hwcache_events(&print_cb, ps);
@@ -664,7 +674,6 @@ int cmd_list(int argc, const char **argv)
event_symbols_hw, PERF_COUNT_HW_MAX);
print_symbol_events(&print_cb, ps, PERF_TYPE_SOFTWARE,
event_symbols_sw, PERF_COUNT_SW_MAX);
- print_tool_events(&print_cb, ps);
print_hwcache_events(&print_cb, ps);
perf_pmus__print_pmu_events(&print_cb, ps);
print_tracepoint_events(&print_cb, ps);
diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c
index 33a456980664..3d490ea46551 100644
--- a/tools/perf/builtin-lock.c
+++ b/tools/perf/builtin-lock.c
@@ -2300,7 +2300,7 @@ static int parse_map_entry(const struct option *opt, const char *str,
static int parse_max_stack(const struct option *opt, const char *str,
int unset __maybe_unused)
{
- unsigned long *len = (unsigned long *)opt->value;
+ int *len = opt->value;
long val;
char *endptr;
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index b578930ed76a..e8708f785e7f 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -46,6 +46,7 @@
#include "util/parse-events.h"
#include "util/pmus.h"
#include "util/pmu.h"
+#include "util/tool_pmu.h"
#include "util/event.h"
#include "util/evlist.h"
#include "util/evsel.h"
@@ -171,7 +172,7 @@ struct opt_aggr_mode {
};
/* Turn command line option into most generic aggregation mode setting. */
-static enum aggr_mode opt_aggr_mode_to_aggr_mode(struct opt_aggr_mode *opt_mode)
+static enum aggr_mode opt_aggr_mode_to_aggr_mode(const struct opt_aggr_mode *opt_mode)
{
enum aggr_mode mode = AGGR_GLOBAL;
@@ -1154,8 +1155,8 @@ static int parse_cache_level(const struct option *opt,
int unset __maybe_unused)
{
int level;
- struct opt_aggr_mode *opt_aggr_mode = (struct opt_aggr_mode *)opt->value;
- u32 *aggr_level = (u32 *)opt->data;
+ bool *per_cache = opt->value;
+ u32 *aggr_level = opt->data;
/*
* If no string is specified, aggregate based on the topology of
@@ -1193,7 +1194,7 @@ static int parse_cache_level(const struct option *opt,
return -EINVAL;
}
out:
- opt_aggr_mode->cache = true;
+ *per_cache = true;
*aggr_level = level;
return 0;
}
@@ -2316,24 +2317,23 @@ static struct perf_stat perf_stat = {
static int __cmd_report(int argc, const char **argv)
{
struct perf_session *session;
+ struct opt_aggr_mode opt_mode = {};
const struct option options[] = {
OPT_STRING('i', "input", &input_name, "file", "input file name"),
- OPT_SET_UINT(0, "per-socket", &perf_stat.aggr_mode,
- "aggregate counts per processor socket", AGGR_SOCKET),
- OPT_SET_UINT(0, "per-die", &perf_stat.aggr_mode,
- "aggregate counts per processor die", AGGR_DIE),
- OPT_SET_UINT(0, "per-cluster", &perf_stat.aggr_mode,
- "aggregate counts perf processor cluster", AGGR_CLUSTER),
- OPT_CALLBACK_OPTARG(0, "per-cache", &perf_stat.aggr_mode, &perf_stat.aggr_level,
- "cache level",
- "aggregate count at this cache level (Default: LLC)",
+ OPT_BOOLEAN(0, "per-thread", &opt_mode.thread, "aggregate counts per thread"),
+ OPT_BOOLEAN(0, "per-socket", &opt_mode.socket,
+ "aggregate counts per processor socket"),
+ OPT_BOOLEAN(0, "per-die", &opt_mode.die, "aggregate counts per processor die"),
+ OPT_BOOLEAN(0, "per-cluster", &opt_mode.cluster,
+ "aggregate counts per processor cluster"),
+ OPT_CALLBACK_OPTARG(0, "per-cache", &opt_mode.cache, &perf_stat.aggr_level,
+ "cache level", "aggregate count at this cache level (Default: LLC)",
parse_cache_level),
- OPT_SET_UINT(0, "per-core", &perf_stat.aggr_mode,
- "aggregate counts per physical processor core", AGGR_CORE),
- OPT_SET_UINT(0, "per-node", &perf_stat.aggr_mode,
- "aggregate counts per numa node", AGGR_NODE),
- OPT_SET_UINT('A', "no-aggr", &perf_stat.aggr_mode,
- "disable CPU count aggregation", AGGR_NONE),
+ OPT_BOOLEAN(0, "per-core", &opt_mode.core,
+ "aggregate counts per physical processor core"),
+ OPT_BOOLEAN(0, "per-node", &opt_mode.node, "aggregate counts per numa node"),
+ OPT_BOOLEAN('A', "no-aggr", &opt_mode.no_aggr,
+ "disable aggregation across CPUs or PMUs"),
OPT_END()
};
struct stat st;
@@ -2341,6 +2341,10 @@ static int __cmd_report(int argc, const char **argv)
argc = parse_options(argc, argv, options, stat_report_usage, 0);
+ perf_stat.aggr_mode = opt_aggr_mode_to_aggr_mode(&opt_mode);
+ if (perf_stat.aggr_mode == AGGR_GLOBAL)
+ perf_stat.aggr_mode = AGGR_UNSET; /* No option found so leave unset. */
+
if (!input_name || !strlen(input_name)) {
if (!fstat(STDIN_FILENO, &st) && S_ISFIFO(st.st_mode))
input_name = "-";
@@ -2490,7 +2494,7 @@ int cmd_stat(int argc, const char **argv)
OPT_BOOLEAN(0, "per-die", &opt_mode.die, "aggregate counts per processor die"),
OPT_BOOLEAN(0, "per-cluster", &opt_mode.cluster,
"aggregate counts per processor cluster"),
- OPT_CALLBACK_OPTARG(0, "per-cache", &opt_mode, &stat_config.aggr_level,
+ OPT_CALLBACK_OPTARG(0, "per-cache", &opt_mode.cache, &stat_config.aggr_level,
"cache level", "aggregate count at this cache level (Default: LLC)",
parse_cache_level),
OPT_BOOLEAN(0, "per-core", &opt_mode.core,
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index dc616292b2dd..fa508e113dd0 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -83,6 +83,7 @@ perf-util-y += pmu.o
perf-util-y += pmus.o
perf-util-y += pmu-flex.o
perf-util-y += pmu-bison.o
+perf-util-y += tool_pmu.o
perf-util-y += svghelper.o
perf-util-$(CONFIG_LIBTRACEEVENT) += trace-event-info.o
perf-util-y += trace-event-scripting.o
diff --git a/tools/perf/util/branch.h b/tools/perf/util/branch.h
index b80c12c74bbb..90f76910b23f 100644
--- a/tools/perf/util/branch.h
+++ b/tools/perf/util/branch.h
@@ -65,6 +65,9 @@ static inline struct branch_entry *perf_sample__branch_entries(struct perf_sampl
{
u64 *entry = (u64 *)sample->branch_stack;
+ if (entry == NULL)
+ return NULL;
+
entry++;
if (sample->no_hw_idx)
return (struct branch_entry *)entry;
diff --git a/tools/perf/util/cgroup.c b/tools/perf/util/cgroup.c
index 0f759dd96db7..e172bcdf7fcb 100644
--- a/tools/perf/util/cgroup.c
+++ b/tools/perf/util/cgroup.c
@@ -417,7 +417,6 @@ int evlist__expand_cgroup(struct evlist *evlist, const char *str,
struct rblist *metric_events, bool open_cgroup)
{
struct evlist *orig_list, *tmp_list;
- struct evsel *pos, *evsel, *leader;
struct rblist orig_metric_events;
struct cgroup *cgrp = NULL;
struct cgroup_name *cn;
@@ -456,6 +455,7 @@ int evlist__expand_cgroup(struct evlist *evlist, const char *str,
goto out_err;
list_for_each_entry(cn, &cgroup_list, list) {
+ struct evsel *pos;
char *name;
if (!cn->used)
@@ -471,21 +471,37 @@ int evlist__expand_cgroup(struct evlist *evlist, const char *str,
if (cgrp == NULL)
continue;
- leader = NULL;
+ /* copy the list and set to the new cgroup. */
evlist__for_each_entry(orig_list, pos) {
- evsel = evsel__clone(pos);
+ struct evsel *evsel = evsel__clone(/*dest=*/NULL, pos);
+
if (evsel == NULL)
goto out_err;
+ /* stash the copy during the copying. */
+ pos->priv = evsel;
cgroup__put(evsel->cgrp);
evsel->cgrp = cgroup__get(cgrp);
- if (evsel__is_group_leader(pos))
- leader = evsel;
- evsel__set_leader(evsel, leader);
-
evlist__add(tmp_list, evsel);
}
+ /* update leader information using stashed pointer to copy. */
+ evlist__for_each_entry(orig_list, pos) {
+ struct evsel *evsel = pos->priv;
+
+ if (evsel__leader(pos))
+ evsel__set_leader(evsel, evsel__leader(pos)->priv);
+
+ if (pos->metric_leader)
+ evsel->metric_leader = pos->metric_leader->priv;
+
+ if (pos->first_wildcard_match)
+ evsel->first_wildcard_match = pos->first_wildcard_match->priv;
+ }
+ /* the stashed copy is no longer used. */
+ evlist__for_each_entry(orig_list, pos)
+ pos->priv = NULL;
+
/* cgroup__new() has a refcount, release it here */
cgroup__put(cgrp);
nr_cgroups++;
diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c
index b78ef0262135..9bf09a856b44 100644
--- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c
+++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c
@@ -237,46 +237,24 @@ cs_etm_decoder__init_def_logger_printing(struct cs_etm_decoder_params *d_params,
(void *)decoder,
cs_etm_decoder__print_str_cb);
if (ret != 0)
- ret = -1;
-
- return 0;
-}
+ return -1;
#ifdef CS_LOG_RAW_FRAMES
-static void
-cs_etm_decoder__init_raw_frame_logging(struct cs_etm_decoder_params *d_params,
- struct cs_etm_decoder *decoder)
-{
- /* Only log these during a --dump operation */
- if (d_params->operation == CS_ETM_OPERATION_PRINT) {
- /* set up a library default logger to process the
- * raw frame printer we add later
- */
- ocsd_def_errlog_init(OCSD_ERR_SEV_ERROR, 1);
-
- /* no stdout / err / file output */
- ocsd_def_errlog_config_output(C_API_MSGLOGOUT_FLG_NONE, NULL);
-
- /* set the string CB for the default logger,
- * passes strings to perf print logger.
- */
- ocsd_def_errlog_set_strprint_cb(decoder->dcd_tree,
- (void *)decoder,
- cs_etm_decoder__print_str_cb);
-
+ /*
+ * Only log raw frames if --dump operation and hardware is actually
+ * generating formatted CoreSight trace frames
+ */
+ if ((d_params->operation == CS_ETM_OPERATION_PRINT) &&
+ (d_params->formatted == true)) {
/* use the built in library printer for the raw frames */
- ocsd_dt_set_raw_frame_printer(decoder->dcd_tree,
- CS_RAW_DEBUG_FLAGS);
+ ret = ocsd_dt_set_raw_frame_printer(decoder->dcd_tree,
+ CS_RAW_DEBUG_FLAGS);
+ if (ret != 0)
+ return -1;
}
-}
-#else
-static void
-cs_etm_decoder__init_raw_frame_logging(
- struct cs_etm_decoder_params *d_params __maybe_unused,
- struct cs_etm_decoder *decoder __maybe_unused)
-{
-}
#endif
+ return 0;
+}
static ocsd_datapath_resp_t
cs_etm_decoder__do_soft_timestamp(struct cs_etm_queue *etmq,
@@ -755,9 +733,6 @@ cs_etm_decoder__new(int decoders, struct cs_etm_decoder_params *d_params,
if (ret != 0)
goto err_free_decoder;
- /* init raw frame logging if required */
- cs_etm_decoder__init_raw_frame_logging(d_params, decoder);
-
for (i = 0; i < decoders; i++) {
ret = cs_etm_decoder__create_etm_decoder(d_params,
&t_params[i],
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index dda107b12b8c..45a7ed5c7a47 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -10,7 +10,6 @@
#include <errno.h>
#include <inttypes.h>
#include <linux/bitops.h>
-#include <api/io.h>
#include <api/fs/fs.h>
#include <api/fs/tracing_path.h>
#include <linux/hw_breakpoint.h>
@@ -51,6 +50,7 @@
#include "off_cpu.h"
#include "pmu.h"
#include "pmus.h"
+#include "tool_pmu.h"
#include "rlimit.h"
#include "../perf-sys.h"
#include "util/parse-branch-options.h"
@@ -71,33 +71,6 @@ struct perf_missing_features perf_missing_features;
static clockid_t clockid;
-static const char *const perf_tool_event__tool_names[PERF_TOOL_MAX] = {
- NULL,
- "duration_time",
- "user_time",
- "system_time",
-};
-
-const char *perf_tool_event__to_str(enum perf_tool_event ev)
-{
- if (ev > PERF_TOOL_NONE && ev < PERF_TOOL_MAX)
- return perf_tool_event__tool_names[ev];
-
- return NULL;
-}
-
-enum perf_tool_event perf_tool_event__from_str(const char *str)
-{
- int i;
-
- perf_tool_event__for_each_event(i) {
- if (!strcmp(str, perf_tool_event__tool_names[i]))
- return i;
- }
- return PERF_TOOL_NONE;
-}
-
-
static int evsel__no_extra_init(struct evsel *evsel __maybe_unused)
{
return 0;
@@ -299,6 +272,7 @@ void evsel__init(struct evsel *evsel,
evsel->pmu_name = NULL;
evsel->group_pmu_name = NULL;
evsel->skippable = false;
+ evsel->alternate_hw_config = PERF_COUNT_HW_MAX;
}
struct evsel *evsel__new_idx(struct perf_event_attr *attr, int idx)
@@ -358,7 +332,7 @@ static int evsel__copy_config_terms(struct evsel *dst, struct evsel *src)
* The assumption is that @orig is not configured nor opened yet.
* So we only care about the attributes that can be set while it's parsed.
*/
-struct evsel *evsel__clone(struct evsel *orig)
+struct evsel *evsel__clone(struct evsel *dest, struct evsel *orig)
{
struct evsel *evsel;
@@ -371,7 +345,11 @@ struct evsel *evsel__clone(struct evsel *orig)
if (orig->bpf_obj)
return NULL;
- evsel = evsel__new(&orig->core.attr);
+ if (dest)
+ evsel = dest;
+ else
+ evsel = evsel__new(&orig->core.attr);
+
if (evsel == NULL)
return NULL;
@@ -421,12 +399,12 @@ struct evsel *evsel__clone(struct evsel *orig)
evsel->core.leader = orig->core.leader;
evsel->max_events = orig->max_events;
- evsel->tool_event = orig->tool_event;
- free((char *)evsel->unit);
- evsel->unit = strdup(orig->unit);
- if (evsel->unit == NULL)
- goto out_err;
-
+ zfree(&evsel->unit);
+ if (orig->unit) {
+ evsel->unit = strdup(orig->unit);
+ if (evsel->unit == NULL)
+ goto out_err;
+ }
evsel->scale = orig->scale;
evsel->snapshot = orig->snapshot;
evsel->per_pkg = orig->per_pkg;
@@ -445,6 +423,8 @@ struct evsel *evsel__clone(struct evsel *orig)
if (evsel__copy_config_terms(evsel, orig) < 0)
goto out_err;
+ evsel->alternate_hw_config = orig->alternate_hw_config;
+
return evsel;
out_err:
@@ -617,11 +597,6 @@ static int evsel__sw_name(struct evsel *evsel, char *bf, size_t size)
return r + evsel__add_modifiers(evsel, bf + r, size - r);
}
-static int evsel__tool_name(enum perf_tool_event ev, char *bf, size_t size)
-{
- return scnprintf(bf, size, "%s", perf_tool_event__to_str(ev));
-}
-
static int __evsel__bp_name(char *bf, size_t size, u64 addr, u64 type)
{
int r;
@@ -772,10 +747,7 @@ const char *evsel__name(struct evsel *evsel)
break;
case PERF_TYPE_SOFTWARE:
- if (evsel__is_tool(evsel))
- evsel__tool_name(evsel__tool_event(evsel), bf, sizeof(bf));
- else
- evsel__sw_name(evsel, bf, sizeof(bf));
+ evsel__sw_name(evsel, bf, sizeof(bf));
break;
case PERF_TYPE_TRACEPOINT:
@@ -786,6 +758,10 @@ const char *evsel__name(struct evsel *evsel)
evsel__bp_name(evsel, bf, sizeof(bf));
break;
+ case PERF_PMU_TYPE_TOOL:
+ scnprintf(bf, sizeof(bf), "%s", evsel__tool_pmu_event_name(evsel));
+ break;
+
default:
scnprintf(bf, sizeof(bf), "unknown attr type: %d",
evsel->core.attr.type);
@@ -811,7 +787,7 @@ const char *evsel__metric_id(const struct evsel *evsel)
return evsel->metric_id;
if (evsel__is_tool(evsel))
- return perf_tool_event__to_str(evsel__tool_event(evsel));
+ return evsel__tool_pmu_event_name(evsel);
return "unknown";
}
@@ -1695,165 +1671,22 @@ static int evsel__read_group(struct evsel *leader, int cpu_map_idx, int thread)
return evsel__process_group_data(leader, cpu_map_idx, thread, data);
}
-static bool read_until_char(struct io *io, char e)
-{
- int c;
-
- do {
- c = io__get_char(io);
- if (c == -1)
- return false;
- } while (c != e);
- return true;
-}
-
-static int read_stat_field(int fd, struct perf_cpu cpu, int field, __u64 *val)
-{
- char buf[256];
- struct io io;
- int i;
-
- io__init(&io, fd, buf, sizeof(buf));
-
- /* Skip lines to relevant CPU. */
- for (i = -1; i < cpu.cpu; i++) {
- if (!read_until_char(&io, '\n'))
- return -EINVAL;
- }
- /* Skip to "cpu". */
- if (io__get_char(&io) != 'c') return -EINVAL;
- if (io__get_char(&io) != 'p') return -EINVAL;
- if (io__get_char(&io) != 'u') return -EINVAL;
-
- /* Skip N of cpuN. */
- if (!read_until_char(&io, ' '))
- return -EINVAL;
-
- i = 1;
- while (true) {
- if (io__get_dec(&io, val) != ' ')
- break;
- if (field == i)
- return 0;
- i++;
- }
- return -EINVAL;
-}
-
-static int read_pid_stat_field(int fd, int field, __u64 *val)
-{
- char buf[256];
- struct io io;
- int c, i;
-
- io__init(&io, fd, buf, sizeof(buf));
- if (io__get_dec(&io, val) != ' ')
- return -EINVAL;
- if (field == 1)
- return 0;
-
- /* Skip comm. */
- if (io__get_char(&io) != '(' || !read_until_char(&io, ')'))
- return -EINVAL;
- if (field == 2)
- return -EINVAL; /* String can't be returned. */
-
- /* Skip state */
- if (io__get_char(&io) != ' ' || io__get_char(&io) == -1)
- return -EINVAL;
- if (field == 3)
- return -EINVAL; /* String can't be returned. */
-
- /* Loop over numeric fields*/
- if (io__get_char(&io) != ' ')
- return -EINVAL;
-
- i = 4;
- while (true) {
- c = io__get_dec(&io, val);
- if (c == -1)
- return -EINVAL;
- if (c == -2) {
- /* Assume a -ve was read */
- c = io__get_dec(&io, val);
- *val *= -1;
- }
- if (c != ' ')
- return -EINVAL;
- if (field == i)
- return 0;
- i++;
- }
- return -EINVAL;
-}
-
-static int evsel__read_tool(struct evsel *evsel, int cpu_map_idx, int thread)
+bool __evsel__match(const struct evsel *evsel, u32 type, u64 config)
{
- __u64 *start_time, cur_time, delta_start;
- int fd, err = 0;
- struct perf_counts_values *count;
- bool adjust = false;
- count = perf_counts(evsel->counts, cpu_map_idx, thread);
+ u32 e_type = evsel->core.attr.type;
+ u64 e_config = evsel->core.attr.config;
- switch (evsel__tool_event(evsel)) {
- case PERF_TOOL_DURATION_TIME:
- /*
- * Pretend duration_time is only on the first CPU and thread, or
- * else aggregation will scale duration_time by the number of
- * CPUs/threads.
- */
- start_time = &evsel->start_time;
- if (cpu_map_idx == 0 && thread == 0)
- cur_time = rdclock();
- else
- cur_time = *start_time;
- break;
- case PERF_TOOL_USER_TIME:
- case PERF_TOOL_SYSTEM_TIME: {
- bool system = evsel__tool_event(evsel) == PERF_TOOL_SYSTEM_TIME;
-
- start_time = xyarray__entry(evsel->start_times, cpu_map_idx, thread);
- fd = FD(evsel, cpu_map_idx, thread);
- lseek(fd, SEEK_SET, 0);
- if (evsel->pid_stat) {
- /* The event exists solely on 1 CPU. */
- if (cpu_map_idx == 0)
- err = read_pid_stat_field(fd, system ? 15 : 14, &cur_time);
- else
- cur_time = 0;
- } else {
- /* The event is for all threads. */
- if (thread == 0) {
- struct perf_cpu cpu = perf_cpu_map__cpu(evsel->core.cpus,
- cpu_map_idx);
-
- err = read_stat_field(fd, cpu, system ? 3 : 1, &cur_time);
- } else {
- cur_time = 0;
- }
- }
- adjust = true;
- break;
- }
- case PERF_TOOL_NONE:
- case PERF_TOOL_MAX:
- default:
- err = -EINVAL;
+ if (e_type != type) {
+ return type == PERF_TYPE_HARDWARE && evsel->pmu && evsel->pmu->is_core &&
+ evsel->alternate_hw_config == config;
}
- if (err)
- return err;
- delta_start = cur_time - *start_time;
- if (adjust) {
- __u64 ticks_per_sec = sysconf(_SC_CLK_TCK);
+ if ((type == PERF_TYPE_HARDWARE || type == PERF_TYPE_HW_CACHE) &&
+ perf_pmus__supports_extended_type())
+ e_config &= PERF_HW_EVENT_MASK;
- delta_start *= 1000000000 / ticks_per_sec;
- }
- count->val = delta_start;
- count->ena = count->run = delta_start;
- count->lost = 0;
- return 0;
+ return e_config == config;
}
int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread)
@@ -2053,6 +1886,7 @@ static struct perf_thread_map *empty_thread_map;
static int __evsel__prepare_open(struct evsel *evsel, struct perf_cpu_map *cpus,
struct perf_thread_map *threads)
{
+ int ret = 0;
int nthreads = perf_thread_map__nr(threads);
if ((perf_missing_features.write_backward && evsel->core.attr.write_backward) ||
@@ -2083,19 +1917,14 @@ static int __evsel__prepare_open(struct evsel *evsel, struct perf_cpu_map *cpus,
perf_evsel__alloc_fd(&evsel->core, perf_cpu_map__nr(cpus), nthreads) < 0)
return -ENOMEM;
- if ((evsel__tool_event(evsel) == PERF_TOOL_SYSTEM_TIME ||
- evsel__tool_event(evsel) == PERF_TOOL_USER_TIME) &&
- !evsel->start_times) {
- evsel->start_times = xyarray__new(perf_cpu_map__nr(cpus), nthreads, sizeof(__u64));
- if (!evsel->start_times)
- return -ENOMEM;
- }
+ if (evsel__is_tool(evsel))
+ ret = evsel__tool_pmu_prepare_open(evsel, cpus, nthreads);
evsel->open_flags = PERF_FLAG_FD_CLOEXEC;
if (evsel->cgrp)
evsel->open_flags |= PERF_FLAG_PID_CGROUP;
- return 0;
+ return ret;
}
static void evsel__disable_missing_features(struct evsel *evsel)
@@ -2273,13 +2102,6 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
int pid = -1, err, old_errno;
enum rlimit_action set_rlimit = NO_CHANGE;
- if (evsel__tool_event(evsel) == PERF_TOOL_DURATION_TIME) {
- if (evsel->core.attr.sample_period) /* no sampling */
- return -EINVAL;
- evsel->start_time = rdclock();
- return 0;
- }
-
if (evsel__is_retire_lat(evsel))
return tpebs_start(evsel->evlist);
@@ -2304,6 +2126,12 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
pr_debug3("Opening: %s\n", evsel__name(evsel));
display_attr(&evsel->core.attr);
+ if (evsel__is_tool(evsel)) {
+ return evsel__tool_pmu_open(evsel, threads,
+ start_cpu_map_idx,
+ end_cpu_map_idx);
+ }
+
for (idx = start_cpu_map_idx; idx < end_cpu_map_idx; idx++) {
for (thread = 0; thread < nthreads; thread++) {
@@ -2315,46 +2143,6 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
if (!evsel->cgrp && !evsel->core.system_wide)
pid = perf_thread_map__pid(threads, thread);
- if (evsel__tool_event(evsel) == PERF_TOOL_USER_TIME ||
- evsel__tool_event(evsel) == PERF_TOOL_SYSTEM_TIME) {
- bool system = evsel__tool_event(evsel) == PERF_TOOL_SYSTEM_TIME;
- __u64 *start_time = NULL;
-
- if (evsel->core.attr.sample_period) {
- /* no sampling */
- err = -EINVAL;
- goto out_close;
- }
- if (pid > -1) {
- char buf[64];
-
- snprintf(buf, sizeof(buf), "/proc/%d/stat", pid);
- fd = open(buf, O_RDONLY);
- evsel->pid_stat = true;
- } else {
- fd = open("/proc/stat", O_RDONLY);
- }
- FD(evsel, idx, thread) = fd;
- if (fd < 0) {
- err = -errno;
- goto out_close;
- }
- start_time = xyarray__entry(evsel->start_times, idx, thread);
- if (pid > -1) {
- err = read_pid_stat_field(fd, system ? 15 : 14,
- start_time);
- } else {
- struct perf_cpu cpu;
-
- cpu = perf_cpu_map__cpu(evsel->core.cpus, idx);
- err = read_stat_field(fd, cpu, system ? 3 : 1,
- start_time);
- }
- if (err)
- goto out_close;
- continue;
- }
-
group_fd = get_group_fd(evsel, idx, thread);
if (group_fd == -2) {
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 26574a33a725..0e64b9f17f0a 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -11,6 +11,7 @@
#include <perf/evsel.h>
#include "symbol_conf.h"
#include "pmus.h"
+#include "pmu.h"
struct bpf_object;
struct cgroup;
@@ -22,25 +23,9 @@ struct target;
struct hashmap;
struct bperf_leader_bpf;
struct bperf_follower_bpf;
-struct perf_pmu;
typedef int (evsel__sb_cb_t)(union perf_event *event, void *data);
-enum perf_tool_event {
- PERF_TOOL_NONE = 0,
- PERF_TOOL_DURATION_TIME = 1,
- PERF_TOOL_USER_TIME = 2,
- PERF_TOOL_SYSTEM_TIME = 3,
-
- PERF_TOOL_MAX,
-};
-
-const char *perf_tool_event__to_str(enum perf_tool_event ev);
-enum perf_tool_event perf_tool_event__from_str(const char *str);
-
-#define perf_tool_event__for_each_event(ev) \
- for ((ev) = PERF_TOOL_DURATION_TIME; (ev) < PERF_TOOL_MAX; ev++)
-
/** struct evsel - event selector
*
* @evlist - evlist this evsel is in, if it is in one.
@@ -83,7 +68,6 @@ struct evsel {
const char *unit;
struct cgroup *cgrp;
const char *metric_id;
- enum perf_tool_event tool_event;
/* parse modifier helper */
int exclude_GH;
int sample_read;
@@ -102,6 +86,7 @@ struct evsel {
int bpf_fd;
struct bpf_object *bpf_obj;
struct list_head config_terms;
+ u64 alternate_hw_config;
};
/*
@@ -256,7 +241,7 @@ static inline struct evsel *evsel__new(struct perf_event_attr *attr)
return evsel__new_idx(attr, 0);
}
-struct evsel *evsel__clone(struct evsel *orig);
+struct evsel *evsel__clone(struct evsel *dest, struct evsel *orig);
int copy_config_terms(struct list_head *dst, struct list_head *src);
void free_config_terms(struct list_head *config_terms);
@@ -322,21 +307,11 @@ const char *evsel__name(struct evsel *evsel);
bool evsel__name_is(struct evsel *evsel, const char *name);
const char *evsel__metric_id(const struct evsel *evsel);
-static inline bool evsel__is_tool(const struct evsel *evsel)
-{
- return evsel->tool_event != PERF_TOOL_NONE;
-}
-
static inline bool evsel__is_retire_lat(const struct evsel *evsel)
{
return evsel->retire_lat;
}
-static inline enum perf_tool_event evsel__tool_event(const struct evsel *evsel)
-{
- return evsel->tool_event;
-}
-
const char *evsel__group_name(struct evsel *evsel);
int evsel__group_desc(struct evsel *evsel, char *buf, size_t size);
@@ -395,26 +370,10 @@ u64 format_field__intval(struct tep_format_field *field, struct perf_sample *sam
struct tep_format_field *evsel__field(struct evsel *evsel, const char *name);
struct tep_format_field *evsel__common_field(struct evsel *evsel, const char *name);
-static inline bool __evsel__match(const struct evsel *evsel, u32 type, u64 config)
-{
- if (evsel->core.attr.type != type)
- return false;
-
- if ((type == PERF_TYPE_HARDWARE || type == PERF_TYPE_HW_CACHE) &&
- perf_pmus__supports_extended_type())
- return (evsel->core.attr.config & PERF_HW_EVENT_MASK) == config;
-
- return evsel->core.attr.config == config;
-}
+bool __evsel__match(const struct evsel *evsel, u32 type, u64 config);
#define evsel__match(evsel, t, c) __evsel__match(evsel, PERF_TYPE_##t, PERF_COUNT_##c)
-static inline bool evsel__match2(struct evsel *e1, struct evsel *e2)
-{
- return (e1->core.attr.type == e2->core.attr.type) &&
- (e1->core.attr.config == e2->core.attr.config);
-}
-
int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread);
int __evsel__read_on_cpu(struct evsel *evsel, int cpu_map_idx, int thread, bool scale);
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index 90c6ce2212e4..7ceefed276b7 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -373,7 +373,8 @@ int expr__find_ids(const char *expr, const char *one,
if (one)
expr__del_id(ctx, one);
- return ret;
+ /* A positive value means syntax error, convert to -EINVAL */
+ return ret > 0 ? -EINVAL : ret;
}
double expr_id_data__value(const struct expr_id_data *data)
diff --git a/tools/perf/util/maps.c b/tools/perf/util/maps.c
index 67133b60b03c..7b8f48677c31 100644
--- a/tools/perf/util/maps.c
+++ b/tools/perf/util/maps.c
@@ -989,16 +989,9 @@ int maps__copy_from(struct maps *dest, struct maps *parent)
map__put(new);
}
maps__set_maps_by_address_sorted(dest, maps__maps_by_address_sorted(parent));
- if (!err) {
- RC_CHK_ACCESS(dest)->last_search_by_name_idx =
- RC_CHK_ACCESS(parent)->last_search_by_name_idx;
- maps__set_maps_by_name_sorted(dest,
- dest_maps_by_name &&
- maps__maps_by_name_sorted(parent));
- } else {
- RC_CHK_ACCESS(dest)->last_search_by_name_idx = 0;
- maps__set_maps_by_name_sorted(dest, false);
- }
+ RC_CHK_ACCESS(dest)->last_search_by_name_idx = 0;
+ /* Values were copied into the name array in address order. */
+ maps__set_maps_by_name_sorted(dest, false);
} else {
/* Unexpected copying to a maps containing entries. */
for (unsigned int i = 0; !err && i < n; i++) {
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 4dff3e925a47..9181548e8881 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -14,6 +14,7 @@
#include "pmus.h"
#include "print-events.h"
#include "smt.h"
+#include "tool_pmu.h"
#include "expr.h"
#include "rblist.h"
#include <string.h>
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 9a8be1e46d67..ba84a451c70a 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -228,7 +228,7 @@ __add_event(struct list_head *list, int *idx,
bool init_attr,
const char *name, const char *metric_id, struct perf_pmu *pmu,
struct list_head *config_terms, bool auto_merge_stats,
- struct perf_cpu_map *cpu_list)
+ struct perf_cpu_map *cpu_list, u64 alternate_hw_config)
{
struct evsel *evsel;
struct perf_cpu_map *cpus = perf_cpu_map__is_empty(cpu_list) && pmu ? pmu->cpus : cpu_list;
@@ -264,6 +264,7 @@ __add_event(struct list_head *list, int *idx,
evsel->auto_merge_stats = auto_merge_stats;
evsel->pmu = pmu;
evsel->pmu_name = pmu ? strdup(pmu->name) : NULL;
+ evsel->alternate_hw_config = alternate_hw_config;
if (name)
evsel->name = strdup(name);
@@ -286,47 +287,19 @@ struct evsel *parse_events__add_event(int idx, struct perf_event_attr *attr,
{
return __add_event(/*list=*/NULL, &idx, attr, /*init_attr=*/false, name,
metric_id, pmu, /*config_terms=*/NULL,
- /*auto_merge_stats=*/false, /*cpu_list=*/NULL);
+ /*auto_merge_stats=*/false, /*cpu_list=*/NULL,
+ /*alternate_hw_config=*/PERF_COUNT_HW_MAX);
}
static int add_event(struct list_head *list, int *idx,
struct perf_event_attr *attr, const char *name,
- const char *metric_id, struct list_head *config_terms)
+ const char *metric_id, struct list_head *config_terms,
+ u64 alternate_hw_config)
{
return __add_event(list, idx, attr, /*init_attr*/true, name, metric_id,
/*pmu=*/NULL, config_terms,
- /*auto_merge_stats=*/false, /*cpu_list=*/NULL) ? 0 : -ENOMEM;
-}
-
-static int add_event_tool(struct list_head *list, int *idx,
- enum perf_tool_event tool_event)
-{
- struct evsel *evsel;
- struct perf_event_attr attr = {
- .type = PERF_TYPE_SOFTWARE,
- .config = PERF_COUNT_SW_DUMMY,
- };
- struct perf_cpu_map *cpu_list = NULL;
-
- if (tool_event == PERF_TOOL_DURATION_TIME) {
- /* Duration time is gathered globally, pretend it is only on CPU0. */
- cpu_list = perf_cpu_map__new("0");
- }
- evsel = __add_event(list, idx, &attr, /*init_attr=*/true, /*name=*/NULL,
- /*metric_id=*/NULL, /*pmu=*/NULL,
- /*config_terms=*/NULL, /*auto_merge_stats=*/false,
- cpu_list);
- perf_cpu_map__put(cpu_list);
- if (!evsel)
- return -ENOMEM;
- evsel->tool_event = tool_event;
- if (tool_event == PERF_TOOL_DURATION_TIME
- || tool_event == PERF_TOOL_USER_TIME
- || tool_event == PERF_TOOL_SYSTEM_TIME) {
- free((char *)evsel->unit);
- evsel->unit = strdup("ns");
- }
- return 0;
+ /*auto_merge_stats=*/false, /*cpu_list=*/NULL,
+ alternate_hw_config) ? 0 : -ENOMEM;
}
/**
@@ -450,7 +423,7 @@ bool parse_events__filter_pmu(const struct parse_events_state *parse_state,
static int parse_events_add_pmu(struct parse_events_state *parse_state,
struct list_head *list, struct perf_pmu *pmu,
const struct parse_events_terms *const_parsed_terms,
- bool auto_merge_stats);
+ bool auto_merge_stats, u64 alternate_hw_config);
int parse_events_add_cache(struct list_head *list, int *idx, const char *name,
struct parse_events_state *parse_state,
@@ -476,7 +449,8 @@ int parse_events_add_cache(struct list_head *list, int *idx, const char *name,
*/
ret = parse_events_add_pmu(parse_state, list, pmu,
parsed_terms,
- perf_pmu__auto_merge_stats(pmu));
+ perf_pmu__auto_merge_stats(pmu),
+ /*alternate_hw_config=*/PERF_COUNT_HW_MAX);
if (ret)
return ret;
continue;
@@ -507,7 +481,8 @@ int parse_events_add_cache(struct list_head *list, int *idx, const char *name,
if (__add_event(list, idx, &attr, /*init_attr*/true, config_name ?: name,
metric_id, pmu, &config_terms, /*auto_merge_stats=*/false,
- /*cpu_list=*/NULL) == NULL)
+ /*cpu_list=*/NULL,
+ /*alternate_hw_config=*/PERF_COUNT_HW_MAX) == NULL)
return -ENOMEM;
free_config_terms(&config_terms);
@@ -772,7 +747,7 @@ int parse_events_add_breakpoint(struct parse_events_state *parse_state,
name = get_config_name(head_config);
return add_event(list, &parse_state->idx, &attr, name, /*mertic_id=*/NULL,
- &config_terms);
+ &config_terms, /*alternate_hw_config=*/PERF_COUNT_HW_MAX);
}
static int check_type_val(struct parse_events_term *term,
@@ -1072,6 +1047,7 @@ static int config_term_pmu(struct perf_event_attr *attr,
if (perf_pmu__have_event(pmu, term->config)) {
term->type_term = PARSE_EVENTS__TERM_TYPE_USER;
term->no_value = true;
+ term->alternate_hw_config = true;
} else {
attr->type = PERF_TYPE_HARDWARE;
attr->config = term->val.num;
@@ -1384,8 +1360,9 @@ static int __parse_events_add_numeric(struct parse_events_state *parse_state,
name = get_config_name(head_config);
metric_id = get_config_metric_id(head_config);
ret = __add_event(list, &parse_state->idx, &attr, /*init_attr*/true, name,
- metric_id, pmu, &config_terms, /*auto_merge_stats=*/false,
- /*cpu_list=*/NULL) ? 0 : -ENOMEM;
+ metric_id, pmu, &config_terms, /*auto_merge_stats=*/false,
+ /*cpu_list=*/NULL, /*alternate_hw_config=*/PERF_COUNT_HW_MAX
+ ) == NULL ? -ENOMEM : 0;
free_config_terms(&config_terms);
return ret;
}
@@ -1421,13 +1398,6 @@ int parse_events_add_numeric(struct parse_events_state *parse_state,
type, /*extended_type=*/0, config, head_config);
}
-int parse_events_add_tool(struct parse_events_state *parse_state,
- struct list_head *list,
- int tool_event)
-{
- return add_event_tool(list, &parse_state->idx, tool_event);
-}
-
static bool config_term_percore(struct list_head *config_terms)
{
struct evsel_config_term *term;
@@ -1443,7 +1413,7 @@ static bool config_term_percore(struct list_head *config_terms)
static int parse_events_add_pmu(struct parse_events_state *parse_state,
struct list_head *list, struct perf_pmu *pmu,
const struct parse_events_terms *const_parsed_terms,
- bool auto_merge_stats)
+ bool auto_merge_stats, u64 alternate_hw_config)
{
struct perf_event_attr attr;
struct perf_pmu_info info;
@@ -1480,7 +1450,7 @@ static int parse_events_add_pmu(struct parse_events_state *parse_state,
/*init_attr=*/true, /*name=*/NULL,
/*metric_id=*/NULL, pmu,
/*config_terms=*/NULL, auto_merge_stats,
- /*cpu_list=*/NULL);
+ /*cpu_list=*/NULL, alternate_hw_config);
return evsel ? 0 : -ENOMEM;
}
@@ -1501,7 +1471,8 @@ static int parse_events_add_pmu(struct parse_events_state *parse_state,
/* Look for event names in the terms and rewrite into format based terms. */
if (perf_pmu__check_alias(pmu, &parsed_terms,
- &info, &alias_rewrote_terms, err)) {
+ &info, &alias_rewrote_terms,
+ &alternate_hw_config, err)) {
parse_events_terms__exit(&parsed_terms);
return -EINVAL;
}
@@ -1546,7 +1517,8 @@ static int parse_events_add_pmu(struct parse_events_state *parse_state,
evsel = __add_event(list, &parse_state->idx, &attr, /*init_attr=*/true,
get_config_name(&parsed_terms),
get_config_metric_id(&parsed_terms), pmu,
- &config_terms, auto_merge_stats, /*cpu_list=*/NULL);
+ &config_terms, auto_merge_stats, /*cpu_list=*/NULL,
+ alternate_hw_config);
if (!evsel) {
parse_events_terms__exit(&parsed_terms);
return -ENOMEM;
@@ -1567,7 +1539,7 @@ static int parse_events_add_pmu(struct parse_events_state *parse_state,
}
int parse_events_multi_pmu_add(struct parse_events_state *parse_state,
- const char *event_name,
+ const char *event_name, u64 hw_config,
const struct parse_events_terms *const_parsed_terms,
struct list_head **listp, void *loc_)
{
@@ -1620,7 +1592,7 @@ int parse_events_multi_pmu_add(struct parse_events_state *parse_state,
auto_merge_stats = perf_pmu__auto_merge_stats(pmu);
if (!parse_events_add_pmu(parse_state, list, pmu,
- &parsed_terms, auto_merge_stats)) {
+ &parsed_terms, auto_merge_stats, hw_config)) {
struct strbuf sb;
strbuf_init(&sb, /*hint=*/ 0);
@@ -1633,7 +1605,7 @@ int parse_events_multi_pmu_add(struct parse_events_state *parse_state,
if (parse_state->fake_pmu) {
if (!parse_events_add_pmu(parse_state, list, perf_pmus__fake_pmu(), &parsed_terms,
- /*auto_merge_stats=*/true)) {
+ /*auto_merge_stats=*/true, hw_config)) {
struct strbuf sb;
strbuf_init(&sb, /*hint=*/ 0);
@@ -1674,13 +1646,15 @@ int parse_events_multi_pmu_add_or_add_pmu(struct parse_events_state *parse_state
/* Attempt to add to list assuming event_or_pmu is a PMU name. */
pmu = perf_pmus__find(event_or_pmu);
if (pmu && !parse_events_add_pmu(parse_state, *listp, pmu, const_parsed_terms,
- /*auto_merge_stats=*/false))
+ /*auto_merge_stats=*/false,
+ /*alternate_hw_config=*/PERF_COUNT_HW_MAX))
return 0;
if (parse_state->fake_pmu) {
if (!parse_events_add_pmu(parse_state, *listp, perf_pmus__fake_pmu(),
const_parsed_terms,
- /*auto_merge_stats=*/false))
+ /*auto_merge_stats=*/false,
+ /*alternate_hw_config=*/PERF_COUNT_HW_MAX))
return 0;
}
@@ -1693,7 +1667,8 @@ int parse_events_multi_pmu_add_or_add_pmu(struct parse_events_state *parse_state
if (!parse_events_add_pmu(parse_state, *listp, pmu,
const_parsed_terms,
- auto_merge_stats)) {
+ auto_merge_stats,
+ /*alternate_hw_config=*/PERF_COUNT_HW_MAX)) {
ok++;
parse_state->wild_card_pmus = true;
}
@@ -1704,7 +1679,8 @@ int parse_events_multi_pmu_add_or_add_pmu(struct parse_events_state *parse_state
/* Failure to add, assume event_or_pmu is an event name. */
zfree(listp);
- if (!parse_events_multi_pmu_add(parse_state, event_or_pmu, const_parsed_terms, listp, loc))
+ if (!parse_events_multi_pmu_add(parse_state, event_or_pmu, PERF_COUNT_HW_MAX,
+ const_parsed_terms, listp, loc))
return 0;
if (asprintf(&help, "Unable to find PMU or event on a PMU of '%s'", event_or_pmu) < 0)
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 10cc9c433116..e9f59de2304b 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -127,6 +127,12 @@ struct parse_events_term {
* value is assumed to be 1. An event name also has no value.
*/
bool no_value;
+ /**
+ * @alternate_hw_config: config is the event name but num is an
+ * alternate PERF_TYPE_HARDWARE config value which is often nice for the
+ * sake of quick matching.
+ */
+ bool alternate_hw_config;
};
struct parse_events_error {
@@ -221,9 +227,6 @@ int parse_events_add_numeric(struct parse_events_state *parse_state,
u32 type, u64 config,
const struct parse_events_terms *head_config,
bool wildcard);
-int parse_events_add_tool(struct parse_events_state *parse_state,
- struct list_head *list,
- int tool_event);
int parse_events_add_cache(struct list_head *list, int *idx, const char *name,
struct parse_events_state *parse_state,
struct parse_events_terms *parsed_terms);
@@ -238,7 +241,7 @@ struct evsel *parse_events__add_event(int idx, struct perf_event_attr *attr,
struct perf_pmu *pmu);
int parse_events_multi_pmu_add(struct parse_events_state *parse_state,
- const char *event_name,
+ const char *event_name, u64 hw_config,
const struct parse_events_terms *const_parsed_terms,
struct list_head **listp, void *loc);
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index 5a0bcd7f166a..14e5bd856a18 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -121,14 +121,6 @@ static int sym(yyscan_t scanner, int type, int config)
return type == PERF_TYPE_HARDWARE ? PE_VALUE_SYM_HW : PE_VALUE_SYM_SW;
}
-static int tool(yyscan_t scanner, enum perf_tool_event event)
-{
- YYSTYPE *yylval = parse_events_get_lval(scanner);
-
- yylval->num = event;
- return PE_VALUE_SYM_TOOL;
-}
-
static int term(yyscan_t scanner, enum parse_events__term_type type)
{
YYSTYPE *yylval = parse_events_get_lval(scanner);
@@ -404,9 +396,6 @@ cpu-migrations|migrations { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COU
alignment-faults { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_ALIGNMENT_FAULTS); }
emulation-faults { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_EMULATION_FAULTS); }
dummy { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_DUMMY); }
-duration_time { return tool(yyscanner, PERF_TOOL_DURATION_TIME); }
-user_time { return tool(yyscanner, PERF_TOOL_USER_TIME); }
-system_time { return tool(yyscanner, PERF_TOOL_SYSTEM_TIME); }
bpf-output { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_BPF_OUTPUT); }
cgroup-switches { return sym(yyscanner, PERF_TYPE_SOFTWARE, PERF_COUNT_SW_CGROUP_SWITCHES); }
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index b3c51f06cbdc..f888cbb076d6 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -56,7 +56,6 @@ static void free_list_evsel(struct list_head* list_evsel)
%token PE_START_EVENTS PE_START_TERMS
%token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_TERM
-%token PE_VALUE_SYM_TOOL
%token PE_EVENT_NAME
%token PE_RAW PE_NAME
%token PE_MODIFIER_EVENT PE_MODIFIER_BP PE_BP_COLON PE_BP_SLASH
@@ -68,7 +67,6 @@ static void free_list_evsel(struct list_head* list_evsel)
%type <num> PE_VALUE
%type <num> PE_VALUE_SYM_HW
%type <num> PE_VALUE_SYM_SW
-%type <num> PE_VALUE_SYM_TOOL
%type <mod> PE_MODIFIER_EVENT
%type <term_type> PE_TERM
%type <num> value_sym
@@ -292,7 +290,7 @@ PE_NAME sep_dc
struct list_head *list;
int err;
- err = parse_events_multi_pmu_add(_parse_state, $1, NULL, &list, &@1);
+ err = parse_events_multi_pmu_add(_parse_state, $1, PERF_COUNT_HW_MAX, NULL, &list, &@1);
if (err < 0) {
struct parse_events_state *parse_state = _parse_state;
struct parse_events_error *error = parse_state->error;
@@ -350,20 +348,6 @@ value_sym sep_slash_slash_dc
PE_ABORT(err);
$$ = list;
}
-|
-PE_VALUE_SYM_TOOL sep_slash_slash_dc
-{
- struct list_head *list;
- int err;
-
- list = alloc_list();
- if (!list)
- YYNOMEM;
- err = parse_events_add_tool(_parse_state, list, $1);
- if (err)
- YYNOMEM;
- $$ = list;
-}
event_legacy_cache:
PE_LEGACY_CACHE opt_event_config
diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
index 8b4e346808b4..15fb144e890f 100644
--- a/tools/perf/util/pmu.c
+++ b/tools/perf/util/pmu.c
@@ -19,6 +19,7 @@
#include "evsel.h"
#include "pmu.h"
#include "pmus.h"
+#include "tool_pmu.h"
#include <util/pmu-bison.h>
#include <util/pmu-flex.h>
#include "parse-events.h"
@@ -1511,6 +1512,9 @@ int perf_pmu__config(struct perf_pmu *pmu, struct perf_event_attr *attr,
{
bool zero = !!pmu->perf_event_attr_init_default;
+ if (perf_pmu__is_tool(pmu))
+ return tool_pmu__config_terms(attr, head_terms, err);
+
/* Fake PMU doesn't have proper terms so nothing to configure in attr. */
if (perf_pmu__is_fake(pmu))
return 0;
@@ -1606,7 +1610,7 @@ static int check_info_data(struct perf_pmu *pmu,
*/
int perf_pmu__check_alias(struct perf_pmu *pmu, struct parse_events_terms *head_terms,
struct perf_pmu_info *info, bool *rewrote_terms,
- struct parse_events_error *err)
+ u64 *alternate_hw_config, struct parse_events_error *err)
{
struct parse_events_term *term, *h;
struct perf_pmu_alias *alias;
@@ -1623,8 +1627,8 @@ int perf_pmu__check_alias(struct perf_pmu *pmu, struct parse_events_terms *head_
info->scale = 0.0;
info->snapshot = false;
- /* Fake PMU doesn't rewrite terms. */
- if (perf_pmu__is_fake(pmu))
+ /* Tool/fake PMU doesn't rewrite terms. */
+ if (perf_pmu__is_tool(pmu) || perf_pmu__is_fake(pmu))
goto out;
list_for_each_entry_safe(term, h, &head_terms->terms, list) {
@@ -1638,6 +1642,7 @@ int perf_pmu__check_alias(struct perf_pmu *pmu, struct parse_events_terms *head_
NULL);
return ret;
}
+
*rewrote_terms = true;
ret = check_info_data(pmu, alias, info, err, term->err_term);
if (ret)
@@ -1646,6 +1651,9 @@ int perf_pmu__check_alias(struct perf_pmu *pmu, struct parse_events_terms *head_
if (alias->per_pkg)
info->per_pkg = true;
+ if (term->alternate_hw_config)
+ *alternate_hw_config = term->val.num;
+
list_del_init(&term->list);
parse_events_term__delete(term);
}
@@ -1790,6 +1798,8 @@ bool perf_pmu__have_event(struct perf_pmu *pmu, const char *name)
{
if (!name)
return false;
+ if (perf_pmu__is_tool(pmu))
+ return perf_tool_event__from_str(name) != PERF_TOOL_NONE;
if (perf_pmu__find_alias(pmu, name, /*load=*/ true) != NULL)
return true;
if (pmu->cpu_aliases_added || !pmu->events_table)
@@ -1801,6 +1811,9 @@ size_t perf_pmu__num_events(struct perf_pmu *pmu)
{
size_t nr;
+ if (perf_pmu__is_tool(pmu))
+ return tool_pmu__num_events();
+
pmu_aliases_parse(pmu);
nr = pmu->sysfs_aliases + pmu->sys_json_aliases;
@@ -1862,6 +1875,9 @@ int perf_pmu__for_each_event(struct perf_pmu *pmu, bool skip_duplicate_pmus,
int ret = 0;
struct strbuf sb;
+ if (perf_pmu__is_tool(pmu))
+ return tool_pmu__for_each_event_cb(pmu, state, cb);
+
strbuf_init(&sb, /*hint=*/ 0);
pmu_aliases_parse(pmu);
pmu_add_cpu_aliases(pmu);
@@ -1950,6 +1966,7 @@ bool perf_pmu__is_software(const struct perf_pmu *pmu)
case PERF_TYPE_HW_CACHE: return false;
case PERF_TYPE_RAW: return false;
case PERF_TYPE_BREAKPOINT: return true;
+ case PERF_PMU_TYPE_TOOL: return true;
default: break;
}
for (size_t i = 0; i < ARRAY_SIZE(known_sw_pmus); i++) {
@@ -2277,6 +2294,9 @@ const char *perf_pmu__name_from_config(struct perf_pmu *pmu, u64 config)
if (!pmu)
return NULL;
+ if (perf_pmu__is_tool(pmu))
+ return perf_tool_event__to_str(config);
+
pmu_aliases_parse(pmu);
pmu_add_cpu_aliases(pmu);
list_for_each_entry(event, &pmu->aliases, list) {
diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h
index bcd278b9b546..2dba1cfa20dd 100644
--- a/tools/perf/util/pmu.h
+++ b/tools/perf/util/pmu.h
@@ -37,6 +37,7 @@ struct perf_pmu_caps {
};
enum {
+ PERF_PMU_TYPE_TOOL = 0xFFFFFFFE,
PERF_PMU_TYPE_FAKE = 0xFFFFFFFF,
};
@@ -220,7 +221,7 @@ __u64 perf_pmu__format_bits(struct perf_pmu *pmu, const char *name);
int perf_pmu__format_type(struct perf_pmu *pmu, const char *name);
int perf_pmu__check_alias(struct perf_pmu *pmu, struct parse_events_terms *head_terms,
struct perf_pmu_info *info, bool *rewrote_terms,
- struct parse_events_error *err);
+ u64 *alternate_hw_config, struct parse_events_error *err);
int perf_pmu__find_event(struct perf_pmu *pmu, const char *event, void *state, pmu_event_callback cb);
void perf_pmu_format__set_value(void *format, int config, unsigned long *bits);
@@ -285,6 +286,7 @@ struct perf_pmu *perf_pmu__lookup(struct list_head *pmus, int dirfd, const char
struct perf_pmu *perf_pmu__create_placeholder_core_pmu(struct list_head *core_pmus);
void perf_pmu__delete(struct perf_pmu *pmu);
struct perf_pmu *perf_pmus__find_core_pmu(void);
+
const char *perf_pmu__name_from_config(struct perf_pmu *pmu, u64 config);
#endif /* __PMU_H */
diff --git a/tools/perf/util/pmus.c b/tools/perf/util/pmus.c
index 362596ed2729..5af26a08fb91 100644
--- a/tools/perf/util/pmus.c
+++ b/tools/perf/util/pmus.c
@@ -15,6 +15,7 @@
#include "evsel.h"
#include "pmus.h"
#include "pmu.h"
+#include "tool_pmu.h"
#include "print-events.h"
#include "strbuf.h"
@@ -200,6 +201,7 @@ static void pmu_read_sysfs(bool core_only)
int fd;
DIR *dir;
struct dirent *dent;
+ struct perf_pmu *tool_pmu;
if (read_sysfs_all_pmus || (core_only && read_sysfs_core_pmus))
return;
@@ -229,6 +231,10 @@ static void pmu_read_sysfs(bool core_only)
pr_err("Failure to set up any core PMUs\n");
}
list_sort(NULL, &core_pmus, pmus_cmp);
+ if (!core_only) {
+ tool_pmu = perf_pmus__tool_pmu();
+ list_add_tail(&tool_pmu->list, &other_pmus);
+ }
list_sort(NULL, &other_pmus, pmus_cmp);
if (!list_empty(&core_pmus)) {
read_sysfs_core_pmus = true;
@@ -584,6 +590,9 @@ void perf_pmus__print_raw_pmu_events(const struct print_callbacks *print_cb, voi
int len = pmu_name_len_no_suffix(pmu->name);
const char *desc = "(see 'man perf-list' or 'man perf-record' on how to encode it)";
+ if (perf_pmu__is_tool(pmu))
+ continue;
+
if (!pmu->is_core)
desc = NULL;
diff --git a/tools/perf/util/print-events.c b/tools/perf/util/print-events.c
index a1c71d9793bd..83aaf7cda635 100644
--- a/tools/perf/util/print-events.c
+++ b/tools/perf/util/print-events.c
@@ -29,6 +29,7 @@
#include "tracepoint.h"
#include "pfm.h"
#include "thread_map.h"
+#include "tool_pmu.h"
#include "util.h"
#define MAX_NAME_LEN 100
@@ -43,21 +44,6 @@ static const char * const event_type_descriptors[] = {
"Hardware breakpoint",
};
-static const struct event_symbol event_symbols_tool[PERF_TOOL_MAX] = {
- [PERF_TOOL_DURATION_TIME] = {
- .symbol = "duration_time",
- .alias = "",
- },
- [PERF_TOOL_USER_TIME] = {
- .symbol = "user_time",
- .alias = "",
- },
- [PERF_TOOL_SYSTEM_TIME] = {
- .symbol = "system_time",
- .alias = "",
- },
-};
-
/*
* Print the events from <debugfs_mount_point>/tracing/events
*/
@@ -342,24 +328,6 @@ int print_hwcache_events(const struct print_callbacks *print_cb, void *print_sta
return 0;
}
-void print_tool_events(const struct print_callbacks *print_cb, void *print_state)
-{
- // Start at 1 because the first enum entry means no tool event.
- for (int i = 1; i < PERF_TOOL_MAX; ++i) {
- print_cb->print_event(print_state,
- "tool",
- /*pmu_name=*/NULL,
- event_symbols_tool[i].symbol,
- event_symbols_tool[i].alias,
- /*scale_unit=*/NULL,
- /*deprecated=*/false,
- "Tool event",
- /*desc=*/NULL,
- /*long_desc=*/NULL,
- /*encoding_desc=*/NULL);
- }
-}
-
void print_symbol_events(const struct print_callbacks *print_cb, void *print_state,
unsigned int type, const struct event_symbol *syms,
unsigned int max)
@@ -423,8 +391,6 @@ void print_events(const struct print_callbacks *print_cb, void *print_state)
print_symbol_events(print_cb, print_state, PERF_TYPE_SOFTWARE,
event_symbols_sw, PERF_COUNT_SW_MAX);
- print_tool_events(print_cb, print_state);
-
print_hwcache_events(print_cb, print_state);
perf_pmus__print_pmu_events(print_cb, print_state);
diff --git a/tools/perf/util/print-events.h b/tools/perf/util/print-events.h
index bf4290bef0cd..445efa1636c1 100644
--- a/tools/perf/util/print-events.h
+++ b/tools/perf/util/print-events.h
@@ -36,7 +36,6 @@ void print_sdt_events(const struct print_callbacks *print_cb, void *print_state)
void print_symbol_events(const struct print_callbacks *print_cb, void *print_state,
unsigned int type, const struct event_symbol *syms,
unsigned int max);
-void print_tool_events(const struct print_callbacks *print_cb, void *print_state);
void print_tracepoint_events(const struct print_callbacks *print_cb, void *print_state);
bool is_event_supported(u8 type, u64 config);
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index e7f36ea9e2fa..5060dc801ded 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -13,6 +13,7 @@
#include "evsel.h"
#include "event.h"
#include "print_binary.h"
+#include "strbuf.h"
#include "thread_map.h"
#include "trace-event.h"
#include "mmap.h"
@@ -1247,6 +1248,60 @@ static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
#endif // HAVE_LIBTRACEEVENT
}
+static PyObject *pyrf_evsel__from_evsel(struct evsel *evsel)
+{
+ struct pyrf_evsel *pevsel = PyObject_New(struct pyrf_evsel, &pyrf_evsel__type);
+
+ if (!pevsel)
+ return NULL;
+
+ memset(&pevsel->evsel, 0, sizeof(pevsel->evsel));
+ evsel__init(&pevsel->evsel, &evsel->core.attr, evsel->core.idx);
+
+ evsel__clone(&pevsel->evsel, evsel);
+ return (PyObject *)pevsel;
+}
+
+static PyObject *pyrf_evlist__from_evlist(struct evlist *evlist)
+{
+ struct pyrf_evlist *pevlist = PyObject_New(struct pyrf_evlist, &pyrf_evlist__type);
+ struct evsel *pos;
+
+ if (!pevlist)
+ return NULL;
+
+ memset(&pevlist->evlist, 0, sizeof(pevlist->evlist));
+ evlist__init(&pevlist->evlist, evlist->core.all_cpus, evlist->core.threads);
+ evlist__for_each_entry(evlist, pos) {
+ struct pyrf_evsel *pevsel = (void *)pyrf_evsel__from_evsel(pos);
+
+ evlist__add(&pevlist->evlist, &pevsel->evsel);
+ }
+ return (PyObject *)pevlist;
+}
+
+static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)
+{
+ const char *input;
+ struct evlist evlist = {};
+ struct parse_events_error err;
+ PyObject *result;
+
+ if (!PyArg_ParseTuple(args, "s", &input))
+ return NULL;
+
+ parse_events_error__init(&err);
+ evlist__init(&evlist, NULL, NULL);
+ if (parse_events(&evlist, input, &err)) {
+ parse_events_error__print(&err, input);
+ PyErr_SetFromErrno(PyExc_OSError);
+ return NULL;
+ }
+ result = pyrf_evlist__from_evlist(&evlist);
+ evlist__exit(&evlist);
+ return result;
+}
+
static PyMethodDef perf__methods[] = {
{
.ml_name = "tracepoint",
@@ -1254,6 +1309,12 @@ static PyMethodDef perf__methods[] = {
.ml_flags = METH_VARARGS | METH_KEYWORDS,
.ml_doc = PyDoc_STR("Get tracepoint config.")
},
+ {
+ .ml_name = "parse_events",
+ .ml_meth = (PyCFunction) pyrf__parse_events,
+ .ml_flags = METH_VARARGS,
+ .ml_doc = PyDoc_STR("Parse a string of events and return an evlist.")
+ },
{ .ml_name = NULL, }
};
diff --git a/tools/perf/util/stat-display.c b/tools/perf/util/stat-display.c
index ea96e4ebad8c..a82a8ec79b39 100644
--- a/tools/perf/util/stat-display.c
+++ b/tools/perf/util/stat-display.c
@@ -21,6 +21,7 @@
#include "iostat.h"
#include "pmu.h"
#include "pmus.h"
+#include "tool_pmu.h"
#define CNTR_NOT_SUPPORTED "<not supported>"
#define CNTR_NOT_COUNTED "<not counted>"
@@ -946,7 +947,10 @@ static bool should_skip_zero_counter(struct perf_stat_config *config,
if (config->aggr_mode == AGGR_THREAD && config->system_wide)
return true;
- /* Tool events have the software PMU but are only gathered on 1. */
+ /*
+ * Many tool events are only gathered on the first index, skip other
+ * zero values.
+ */
if (evsel__is_tool(counter))
return true;
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 7c49997fab3a..caffdaa8be9a 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -15,6 +15,7 @@
#include <linux/zalloc.h>
#include "iostat.h"
#include "util/hashmap.h"
+#include "tool_pmu.h"
struct stats walltime_nsecs_stats;
struct rusage_stats ru_stats;
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index e398abfd13a0..61f931997f2e 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -1429,8 +1429,12 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
char dso_name[PATH_MAX];
/* Adjust symbol to map to file offset */
- if (adjust_kernel_syms)
- sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+ if (adjust_kernel_syms) {
+ if (dso__rel(dso))
+ sym->st_value += shdr->sh_offset;
+ else
+ sym->st_value -= shdr->sh_addr - shdr->sh_offset;
+ }
if (strcmp(section_name, (dso__short_name(curr_dso) + dso__short_name_len(dso))) == 0)
return 0;
diff --git a/tools/perf/util/tool_pmu.c b/tools/perf/util/tool_pmu.c
new file mode 100644
index 000000000000..3d1d6b3352ec
--- /dev/null
+++ b/tools/perf/util/tool_pmu.c
@@ -0,0 +1,417 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include "cgroup.h"
+#include "counts.h"
+#include "evsel.h"
+#include "pmu.h"
+#include "print-events.h"
+#include "time-utils.h"
+#include "tool_pmu.h"
+#include <api/io.h>
+#include <internal/threadmap.h>
+#include <perf/threadmap.h>
+#include <fcntl.h>
+#include <strings.h>
+
+static const char *const tool_pmu__event_names[PERF_TOOL_MAX] = {
+ NULL,
+ "duration_time",
+ "user_time",
+ "system_time",
+};
+
+
+const char *perf_tool_event__to_str(enum perf_tool_event ev)
+{
+ if (ev > PERF_TOOL_NONE && ev < PERF_TOOL_MAX)
+ return tool_pmu__event_names[ev];
+
+ return NULL;
+}
+
+enum perf_tool_event perf_tool_event__from_str(const char *str)
+{
+ int i;
+
+ perf_tool_event__for_each_event(i) {
+ if (!strcasecmp(str, tool_pmu__event_names[i]))
+ return i;
+ }
+ return PERF_TOOL_NONE;
+}
+
+static int tool_pmu__config_term(struct perf_event_attr *attr,
+ struct parse_events_term *term,
+ struct parse_events_error *err)
+{
+ if (term->type_term == PARSE_EVENTS__TERM_TYPE_USER) {
+ enum perf_tool_event ev = perf_tool_event__from_str(term->config);
+
+ if (ev == PERF_TOOL_NONE)
+ goto err_out;
+
+ attr->config = ev;
+ return 0;
+ }
+err_out:
+ if (err) {
+ char *err_str;
+
+ parse_events_error__handle(err, term->err_val,
+ asprintf(&err_str,
+ "unexpected tool event term (%s) %s",
+ parse_events__term_type_str(term->type_term),
+ term->config) < 0
+ ? strdup("unexpected tool event term")
+ : err_str,
+ NULL);
+ }
+ return -EINVAL;
+}
+
+int tool_pmu__config_terms(struct perf_event_attr *attr,
+ struct parse_events_terms *terms,
+ struct parse_events_error *err)
+{
+ struct parse_events_term *term;
+
+ list_for_each_entry(term, &terms->terms, list) {
+ if (tool_pmu__config_term(attr, term, err))
+ return -EINVAL;
+ }
+
+ return 0;
+
+}
+
+int tool_pmu__for_each_event_cb(struct perf_pmu *pmu, void *state, pmu_event_callback cb)
+{
+ struct pmu_event_info info = {
+ .pmu = pmu,
+ .event_type_desc = "Tool event",
+ };
+ int i;
+
+ perf_tool_event__for_each_event(i) {
+ int ret;
+
+ info.name = perf_tool_event__to_str(i);
+ info.alias = NULL;
+ info.scale_unit = NULL;
+ info.desc = NULL;
+ info.long_desc = NULL;
+ info.encoding_desc = NULL;
+ info.topic = NULL;
+ info.pmu_name = pmu->name;
+ info.deprecated = false;
+ ret = cb(state, &info);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+bool perf_pmu__is_tool(const struct perf_pmu *pmu)
+{
+ return pmu && pmu->type == PERF_PMU_TYPE_TOOL;
+}
+
+bool evsel__is_tool(const struct evsel *evsel)
+{
+ return perf_pmu__is_tool(evsel->pmu);
+}
+
+enum perf_tool_event evsel__tool_event(const struct evsel *evsel)
+{
+ if (!evsel__is_tool(evsel))
+ return PERF_TOOL_NONE;
+
+ return (enum perf_tool_event)evsel->core.attr.config;
+}
+
+const char *evsel__tool_pmu_event_name(const struct evsel *evsel)
+{
+ return perf_tool_event__to_str(evsel->core.attr.config);
+}
+
+static bool read_until_char(struct io *io, char e)
+{
+ int c;
+
+ do {
+ c = io__get_char(io);
+ if (c == -1)
+ return false;
+ } while (c != e);
+ return true;
+}
+
+static int read_stat_field(int fd, struct perf_cpu cpu, int field, __u64 *val)
+{
+ char buf[256];
+ struct io io;
+ int i;
+
+ io__init(&io, fd, buf, sizeof(buf));
+
+ /* Skip lines to relevant CPU. */
+ for (i = -1; i < cpu.cpu; i++) {
+ if (!read_until_char(&io, '\n'))
+ return -EINVAL;
+ }
+ /* Skip to "cpu". */
+ if (io__get_char(&io) != 'c') return -EINVAL;
+ if (io__get_char(&io) != 'p') return -EINVAL;
+ if (io__get_char(&io) != 'u') return -EINVAL;
+
+ /* Skip N of cpuN. */
+ if (!read_until_char(&io, ' '))
+ return -EINVAL;
+
+ i = 1;
+ while (true) {
+ if (io__get_dec(&io, val) != ' ')
+ break;
+ if (field == i)
+ return 0;
+ i++;
+ }
+ return -EINVAL;
+}
+
+static int read_pid_stat_field(int fd, int field, __u64 *val)
+{
+ char buf[256];
+ struct io io;
+ int c, i;
+
+ io__init(&io, fd, buf, sizeof(buf));
+ if (io__get_dec(&io, val) != ' ')
+ return -EINVAL;
+ if (field == 1)
+ return 0;
+
+ /* Skip comm. */
+ if (io__get_char(&io) != '(' || !read_until_char(&io, ')'))
+ return -EINVAL;
+ if (field == 2)
+ return -EINVAL; /* String can't be returned. */
+
+ /* Skip state */
+ if (io__get_char(&io) != ' ' || io__get_char(&io) == -1)
+ return -EINVAL;
+ if (field == 3)
+ return -EINVAL; /* String can't be returned. */
+
+ /* Loop over numeric fields*/
+ if (io__get_char(&io) != ' ')
+ return -EINVAL;
+
+ i = 4;
+ while (true) {
+ c = io__get_dec(&io, val);
+ if (c == -1)
+ return -EINVAL;
+ if (c == -2) {
+ /* Assume a -ve was read */
+ c = io__get_dec(&io, val);
+ *val *= -1;
+ }
+ if (c != ' ')
+ return -EINVAL;
+ if (field == i)
+ return 0;
+ i++;
+ }
+ return -EINVAL;
+}
+
+int evsel__tool_pmu_prepare_open(struct evsel *evsel,
+ struct perf_cpu_map *cpus,
+ int nthreads)
+{
+ if ((evsel__tool_event(evsel) == PERF_TOOL_SYSTEM_TIME ||
+ evsel__tool_event(evsel) == PERF_TOOL_USER_TIME) &&
+ !evsel->start_times) {
+ evsel->start_times = xyarray__new(perf_cpu_map__nr(cpus),
+ nthreads,
+ sizeof(__u64));
+ if (!evsel->start_times)
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+#define FD(e, x, y) (*(int *)xyarray__entry(e->core.fd, x, y))
+
+int evsel__tool_pmu_open(struct evsel *evsel,
+ struct perf_thread_map *threads,
+ int start_cpu_map_idx, int end_cpu_map_idx)
+{
+ enum perf_tool_event ev = evsel__tool_event(evsel);
+ int pid = -1, idx = 0, thread = 0, nthreads, err = 0, old_errno;
+
+ if (ev == PERF_TOOL_DURATION_TIME) {
+ if (evsel->core.attr.sample_period) /* no sampling */
+ return -EINVAL;
+ evsel->start_time = rdclock();
+ return 0;
+ }
+
+ if (evsel->cgrp)
+ pid = evsel->cgrp->fd;
+
+ nthreads = perf_thread_map__nr(threads);
+ for (idx = start_cpu_map_idx; idx < end_cpu_map_idx; idx++) {
+ for (thread = 0; thread < nthreads; thread++) {
+ if (thread >= nthreads)
+ break;
+
+ if (!evsel->cgrp && !evsel->core.system_wide)
+ pid = perf_thread_map__pid(threads, thread);
+
+ if (ev == PERF_TOOL_USER_TIME || ev == PERF_TOOL_SYSTEM_TIME) {
+ bool system = ev == PERF_TOOL_SYSTEM_TIME;
+ __u64 *start_time = NULL;
+ int fd;
+
+ if (evsel->core.attr.sample_period) {
+ /* no sampling */
+ err = -EINVAL;
+ goto out_close;
+ }
+ if (pid > -1) {
+ char buf[64];
+
+ snprintf(buf, sizeof(buf), "/proc/%d/stat", pid);
+ fd = open(buf, O_RDONLY);
+ evsel->pid_stat = true;
+ } else {
+ fd = open("/proc/stat", O_RDONLY);
+ }
+ FD(evsel, idx, thread) = fd;
+ if (fd < 0) {
+ err = -errno;
+ goto out_close;
+ }
+ start_time = xyarray__entry(evsel->start_times, idx, thread);
+ if (pid > -1) {
+ err = read_pid_stat_field(fd, system ? 15 : 14,
+ start_time);
+ } else {
+ struct perf_cpu cpu;
+
+ cpu = perf_cpu_map__cpu(evsel->core.cpus, idx);
+ err = read_stat_field(fd, cpu, system ? 3 : 1,
+ start_time);
+ }
+ if (err)
+ goto out_close;
+ }
+
+ }
+ }
+ return 0;
+out_close:
+ if (err)
+ threads->err_thread = thread;
+
+ old_errno = errno;
+ do {
+ while (--thread >= 0) {
+ if (FD(evsel, idx, thread) >= 0)
+ close(FD(evsel, idx, thread));
+ FD(evsel, idx, thread) = -1;
+ }
+ thread = nthreads;
+ } while (--idx >= 0);
+ errno = old_errno;
+ return err;
+}
+
+int evsel__read_tool(struct evsel *evsel, int cpu_map_idx, int thread)
+{
+ __u64 *start_time, cur_time, delta_start;
+ int fd, err = 0;
+ struct perf_counts_values *count;
+ bool adjust = false;
+
+ count = perf_counts(evsel->counts, cpu_map_idx, thread);
+
+ switch (evsel__tool_event(evsel)) {
+ case PERF_TOOL_DURATION_TIME:
+ /*
+ * Pretend duration_time is only on the first CPU and thread, or
+ * else aggregation will scale duration_time by the number of
+ * CPUs/threads.
+ */
+ start_time = &evsel->start_time;
+ if (cpu_map_idx == 0 && thread == 0)
+ cur_time = rdclock();
+ else
+ cur_time = *start_time;
+ break;
+ case PERF_TOOL_USER_TIME:
+ case PERF_TOOL_SYSTEM_TIME: {
+ bool system = evsel__tool_event(evsel) == PERF_TOOL_SYSTEM_TIME;
+
+ start_time = xyarray__entry(evsel->start_times, cpu_map_idx, thread);
+ fd = FD(evsel, cpu_map_idx, thread);
+ lseek(fd, SEEK_SET, 0);
+ if (evsel->pid_stat) {
+ /* The event exists solely on 1 CPU. */
+ if (cpu_map_idx == 0)
+ err = read_pid_stat_field(fd, system ? 15 : 14, &cur_time);
+ else
+ cur_time = 0;
+ } else {
+ /* The event is for all threads. */
+ if (thread == 0) {
+ struct perf_cpu cpu = perf_cpu_map__cpu(evsel->core.cpus,
+ cpu_map_idx);
+
+ err = read_stat_field(fd, cpu, system ? 3 : 1, &cur_time);
+ } else {
+ cur_time = 0;
+ }
+ }
+ adjust = true;
+ break;
+ }
+ case PERF_TOOL_NONE:
+ case PERF_TOOL_MAX:
+ default:
+ err = -EINVAL;
+ }
+ if (err)
+ return err;
+
+ delta_start = cur_time - *start_time;
+ if (adjust) {
+ __u64 ticks_per_sec = sysconf(_SC_CLK_TCK);
+
+ delta_start *= 1000000000 / ticks_per_sec;
+ }
+ count->val = delta_start;
+ count->lost = 0;
+ /*
+ * The values of enabled and running must make a ratio of 100%. The
+ * exact values don't matter as long as they are non-zero to avoid
+ * issues with evsel__count_has_error.
+ */
+ count->ena++;
+ count->run++;
+ return 0;
+}
+
+struct perf_pmu *perf_pmus__tool_pmu(void)
+{
+ static struct perf_pmu tool = {
+ .name = "tool",
+ .type = PERF_PMU_TYPE_TOOL,
+ .aliases = LIST_HEAD_INIT(tool.aliases),
+ .caps = LIST_HEAD_INIT(tool.caps),
+ .format = LIST_HEAD_INIT(tool.format),
+ };
+
+ return &tool;
+}
diff --git a/tools/perf/util/tool_pmu.h b/tools/perf/util/tool_pmu.h
new file mode 100644
index 000000000000..05a4052c8b9d
--- /dev/null
+++ b/tools/perf/util/tool_pmu.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __TOOL_PMU_H
+#define __TOOL_PMU_H
+
+#include "pmu.h"
+
+struct evsel;
+struct perf_thread_map;
+struct print_callbacks;
+
+enum perf_tool_event {
+ PERF_TOOL_NONE = 0,
+ PERF_TOOL_DURATION_TIME = 1,
+ PERF_TOOL_USER_TIME = 2,
+ PERF_TOOL_SYSTEM_TIME = 3,
+
+ PERF_TOOL_MAX,
+};
+
+#define perf_tool_event__for_each_event(ev) \
+ for ((ev) = PERF_TOOL_DURATION_TIME; (ev) < PERF_TOOL_MAX; ev++)
+
+static inline size_t tool_pmu__num_events(void)
+{
+ return PERF_TOOL_MAX - 1;
+}
+
+const char *perf_tool_event__to_str(enum perf_tool_event ev);
+enum perf_tool_event perf_tool_event__from_str(const char *str);
+int tool_pmu__config_terms(struct perf_event_attr *attr,
+ struct parse_events_terms *terms,
+ struct parse_events_error *err);
+int tool_pmu__for_each_event_cb(struct perf_pmu *pmu, void *state, pmu_event_callback cb);
+
+bool perf_pmu__is_tool(const struct perf_pmu *pmu);
+
+
+bool evsel__is_tool(const struct evsel *evsel);
+enum perf_tool_event evsel__tool_event(const struct evsel *evsel);
+const char *evsel__tool_pmu_event_name(const struct evsel *evsel);
+int evsel__tool_pmu_prepare_open(struct evsel *evsel,
+ struct perf_cpu_map *cpus,
+ int nthreads);
+int evsel__tool_pmu_open(struct evsel *evsel,
+ struct perf_thread_map *threads,
+ int start_cpu_map_idx, int end_cpu_map_idx);
+int evsel__read_tool(struct evsel *evsel, int cpu_map_idx, int thread);
+
+struct perf_pmu *perf_pmus__tool_pmu(void);
+
+#endif /* __TOOL_PMU_H */
diff --git a/tools/perf/util/util.h b/tools/perf/util/util.h
index 9966c21aaf04..bc30d038817d 100644
--- a/tools/perf/util/util.h
+++ b/tools/perf/util/util.h
@@ -26,7 +26,6 @@ extern bool perf_guest;
/* General helper functions */
void usage(const char *err) __noreturn;
-void die(const char *err, ...) __noreturn __printf(1, 2);
struct dirent;
struct strlist;
diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 3242a216af9e..c61adcfa837a 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -98,6 +98,7 @@ my $test_type;
my $build_type;
my $build_options;
my $final_post_ktest;
+my $post_ktest_done = 0;
my $pre_ktest;
my $post_ktest;
my $pre_test;
@@ -1550,6 +1551,24 @@ sub get_test_name() {
return $name;
}
+sub run_post_ktest {
+ my $cmd;
+
+ return if ($post_ktest_done);
+
+ if (defined($final_post_ktest)) {
+ $cmd = $final_post_ktest;
+ } elsif (defined($post_ktest)) {
+ $cmd = $post_ktest;
+ } else {
+ return;
+ }
+
+ my $cp_post_ktest = eval_kernel_version($cmd);
+ run_command $cp_post_ktest;
+ $post_ktest_done = 1;
+}
+
sub dodie {
# avoid recursion
return if ($in_die);
@@ -1609,6 +1628,7 @@ sub dodie {
if (defined($post_test)) {
run_command $post_test;
}
+ run_post_ktest;
die @_, "\n";
}
@@ -2483,7 +2503,7 @@ sub check_buildlog {
my $save_no_reboot = $no_reboot;
$no_reboot = 1;
- if (-f $warnings_file) {
+ if (defined($warnings_file) && -f $warnings_file) {
open(IN, $warnings_file) or
dodie "Error opening $warnings_file";
@@ -4127,7 +4147,8 @@ sub __set_test_option {
my $option = "$name\[$i\]";
- if (option_defined($option)) {
+ if (exists($opt{$option})) {
+ return undef if (!option_defined($option));
return $opt{$option};
}
@@ -4135,7 +4156,8 @@ sub __set_test_option {
if ($i >= $test &&
$i < $test + $repeat_tests{$test}) {
$option = "$name\[$test\]";
- if (option_defined($option)) {
+ if (exists($opt{$option})) {
+ return undef if (!option_defined($option));
return $opt{$option};
}
}
@@ -4242,6 +4264,7 @@ sub cancel_test {
send_email("KTEST: Your [$name] test was cancelled",
"Your test started at $script_start_time was cancelled: sig int");
}
+ run_post_ktest;
die "\nCaught Sig Int, test interrupted: $!\n"
}
@@ -4552,11 +4575,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
success $i;
}
-if (defined($final_post_ktest)) {
-
- my $cp_final_post_ktest = eval_kernel_version $final_post_ktest;
- run_command $cp_final_post_ktest;
-}
+run_post_ktest;
if ($opt{"POWEROFF_ON_SUCCESS"}) {
halt;
diff --git a/tools/testing/selftests/bpf/prog_tests/snprintf.c b/tools/testing/selftests/bpf/prog_tests/snprintf.c
index 4be6fdb78c6a..20a3c622bd28 100644
--- a/tools/testing/selftests/bpf/prog_tests/snprintf.c
+++ b/tools/testing/selftests/bpf/prog_tests/snprintf.c
@@ -114,7 +114,8 @@ static void test_snprintf_negative(void)
ASSERT_ERR(load_single_snprintf("%--------"), "invalid specifier 5");
ASSERT_ERR(load_single_snprintf("%lc"), "invalid specifier 6");
ASSERT_ERR(load_single_snprintf("%llc"), "invalid specifier 7");
- ASSERT_ERR(load_single_snprintf("\x80"), "non ascii character");
+ ASSERT_OK(load_single_snprintf("\x80"), "non ascii plain text");
+ ASSERT_ERR(load_single_snprintf("%\x80"), "non ascii in specifier");
ASSERT_ERR(load_single_snprintf("\x1"), "non printable character");
}
diff --git a/tools/testing/selftests/bpf/progs/bpf_misc.h b/tools/testing/selftests/bpf/progs/bpf_misc.h
index eccaf955e394..9c0d71f0b3cc 100644
--- a/tools/testing/selftests/bpf/progs/bpf_misc.h
+++ b/tools/testing/selftests/bpf/progs/bpf_misc.h
@@ -115,7 +115,7 @@
#define __description(desc) __attribute__((btf_decl_tag("comment:test_description=" desc)))
#define __msg_unpriv(msg) __attribute__((btf_decl_tag("comment:test_expect_msg_unpriv=" XSTR(__COUNTER__) "=" msg)))
#define __xlated_unpriv(msg) __attribute__((btf_decl_tag("comment:test_expect_xlated_unpriv=" XSTR(__COUNTER__) "=" msg)))
-#define __jited_unpriv(msg) __attribute__((btf_decl_tag("comment:test_jited=" XSTR(__COUNTER__) "=" msg)))
+#define __jited_unpriv(msg) __attribute__((btf_decl_tag("comment:test_jited_unpriv=" XSTR(__COUNTER__) "=" msg)))
#define __failure_unpriv __attribute__((btf_decl_tag("comment:test_expect_failure_unpriv")))
#define __success_unpriv __attribute__((btf_decl_tag("comment:test_expect_success_unpriv")))
#define __log_level(lvl) __attribute__((btf_decl_tag("comment:test_log_level="#lvl)))
diff --git a/tools/testing/selftests/bpf/progs/verifier_scalar_ids.c b/tools/testing/selftests/bpf/progs/verifier_scalar_ids.c
index 7c5e5e6d10eb..dc65218e93c4 100644
--- a/tools/testing/selftests/bpf/progs/verifier_scalar_ids.c
+++ b/tools/testing/selftests/bpf/progs/verifier_scalar_ids.c
@@ -723,9 +723,9 @@ __success __log_level(2)
/* The exit instruction should be reachable from two states,
* use two matches and "processed .. insns" to ensure this.
*/
-__msg("13: (95) exit")
-__msg("13: (95) exit")
-__msg("processed 18 insns")
+__msg("15: (95) exit")
+__msg("15: (95) exit")
+__msg("processed 20 insns")
__flag(BPF_F_TEST_STATE_FREQ)
__naked void two_old_ids_one_cur_id(void)
{
@@ -734,9 +734,11 @@ __naked void two_old_ids_one_cur_id(void)
"call %[bpf_ktime_get_ns];"
"r0 &= 0xff;"
"r6 = r0;"
+ "r8 = r0;"
"call %[bpf_ktime_get_ns];"
"r0 &= 0xff;"
"r7 = r0;"
+ "r9 = r0;"
"r0 = 0;"
/* Maybe make r{6,7} IDs identical */
"if r6 > r7 goto l0_%=;"
diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c
index 16f5d74ae762..7a44d221b8c4 100644
--- a/tools/testing/selftests/cgroup/test_memcontrol.c
+++ b/tools/testing/selftests/cgroup/test_memcontrol.c
@@ -1190,8 +1190,11 @@ static int tcp_server(const char *cgroup, void *arg)
saddr.sin6_port = htons(srv_args->port);
sk = socket(AF_INET6, SOCK_STREAM, 0);
- if (sk < 0)
+ if (sk < 0) {
+ /* Pass back errno to the ctl_fd */
+ write(ctl_fd, &errno, sizeof(errno));
return ret;
+ }
if (setsockopt(sk, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes)) < 0)
goto cleanup;
@@ -1321,6 +1324,12 @@ static int test_memcg_sock(const char *root)
goto cleanup;
close(args.ctl[0]);
+ /* Skip if address family not supported by protocol */
+ if (err == EAFNOSUPPORT) {
+ ret = KSFT_SKIP;
+ goto cleanup;
+ }
+
if (!err)
break;
if (err != EADDRINUSE)
diff --git a/tools/testing/selftests/mm/migration.c b/tools/testing/selftests/mm/migration.c
index 64bcbb7151cf..c883ef420d3b 100644
--- a/tools/testing/selftests/mm/migration.c
+++ b/tools/testing/selftests/mm/migration.c
@@ -33,7 +33,8 @@ FIXTURE_SETUP(migration)
{
int n;
- ASSERT_EQ(numa_available(), 0);
+ if (numa_available() < 0)
+ SKIP(return, "NUMA not available");
self->nthreads = numa_num_task_cpus() - 1;
self->n1 = -1;
self->n2 = -1;
diff --git a/tools/testing/selftests/net/netfilter/nft_tproxy_udp.sh b/tools/testing/selftests/net/netfilter/nft_tproxy_udp.sh
index d16de13fe5a7..1dc7b0450145 100755
--- a/tools/testing/selftests/net/netfilter/nft_tproxy_udp.sh
+++ b/tools/testing/selftests/net/netfilter/nft_tproxy_udp.sh
@@ -190,13 +190,13 @@ table inet filter {
}
EOF
- timeout "$timeout" ip netns exec "$nsrouter" socat -u "$socat_ipproto" udp-listen:12345,fork,ip-transparent,reuseport udp:"$ns1_ip_port",ip-transparent,reuseport,bind="$ns2_ip_port" 2>/dev/null &
+ timeout "$timeout" ip netns exec "$nsrouter" socat -u "$socat_ipproto" udp-listen:12345,fork,ip-transparent,reuseport,shut-none udp:"$ns1_ip_port",ip-transparent,reuseport,bind="$ns2_ip_port",shut-none 2>/dev/null &
local tproxy_pid=$!
- timeout "$timeout" ip netns exec "$ns2" socat "$socat_ipproto" udp-listen:8080,fork SYSTEM:"echo PONG_NS2" 2>/dev/null &
+ timeout "$timeout" ip netns exec "$ns2" socat "$socat_ipproto" udp-listen:8080,fork,shut-none SYSTEM:"echo PONG_NS2" 2>/dev/null &
local server2_pid=$!
- timeout "$timeout" ip netns exec "$ns3" socat "$socat_ipproto" udp-listen:8080,fork SYSTEM:"echo PONG_NS3" 2>/dev/null &
+ timeout "$timeout" ip netns exec "$ns3" socat "$socat_ipproto" udp-listen:8080,fork,shut-none SYSTEM:"echo PONG_NS3" 2>/dev/null &
local server3_pid=$!
busywait "$BUSYWAIT_TIMEOUT" listener_ready "$nsrouter" 12345 "-u"
@@ -205,7 +205,7 @@ EOF
local result
# request from ns1 to ns2 (forwarded traffic)
- result=$(echo I_M_PROXIED | ip netns exec "$ns1" socat -t 2 -T 2 STDIO udp:"$ns2_ip_port",sourceport=18888)
+ result=$(echo I_M_PROXIED | ip netns exec "$ns1" socat -t 2 -T 2 STDIO udp:"$ns2_ip_port",sourceport=18888,shut-none)
if [ "$result" == "$expect_ns1_ns2" ] ;then
echo "PASS: tproxy test $testname: ns1 got reply \"$result\" connecting to ns2"
else
@@ -214,7 +214,7 @@ EOF
fi
# request from ns1 to ns3 (forwarded traffic)
- result=$(echo I_M_PROXIED | ip netns exec "$ns1" socat -t 2 -T 2 STDIO udp:"$ns3_ip_port")
+ result=$(echo I_M_PROXIED | ip netns exec "$ns1" socat -t 2 -T 2 STDIO udp:"$ns3_ip_port",shut-none)
if [ "$result" = "$expect_ns1_ns3" ] ;then
echo "PASS: tproxy test $testname: ns1 got reply \"$result\" connecting to ns3"
else
@@ -223,7 +223,7 @@ EOF
fi
# request from nsrouter to ns2 (localy originated traffic)
- result=$(echo I_M_PROXIED | ip netns exec "$nsrouter" socat -t 2 -T 2 STDIO udp:"$ns2_ip_port")
+ result=$(echo I_M_PROXIED | ip netns exec "$nsrouter" socat -t 2 -T 2 STDIO udp:"$ns2_ip_port",shut-none)
if [ "$result" == "$expect_nsrouter_ns2" ] ;then
echo "PASS: tproxy test $testname: nsrouter got reply \"$result\" connecting to ns2"
else
@@ -232,7 +232,7 @@ EOF
fi
# request from nsrouter to ns3 (localy originated traffic)
- result=$(echo I_M_PROXIED | ip netns exec "$nsrouter" socat -t 2 -T 2 STDIO udp:"$ns3_ip_port")
+ result=$(echo I_M_PROXIED | ip netns exec "$nsrouter" socat -t 2 -T 2 STDIO udp:"$ns3_ip_port",shut-none)
if [ "$result" = "$expect_nsrouter_ns3" ] ;then
echo "PASS: tproxy test $testname: nsrouter got reply \"$result\" connecting to ns3"
else
diff --git a/tools/testing/selftests/powerpc/vphn/Makefile b/tools/testing/selftests/powerpc/vphn/Makefile
index 61d519a076c6..778fc396340d 100644
--- a/tools/testing/selftests/powerpc/vphn/Makefile
+++ b/tools/testing/selftests/powerpc/vphn/Makefile
@@ -5,7 +5,7 @@ top_srcdir = ../../../../..
include ../../lib.mk
include ../flags.mk
-CFLAGS += -m64 -I$(CURDIR)
+CFLAGS += -m64 -I$(CURDIR) -fno-strict-aliasing
$(TEST_GEN_PROGS): ../harness.c
diff --git a/tools/testing/selftests/sched_ext/exit.c b/tools/testing/selftests/sched_ext/exit.c
index 2c084ded2968..b4a8dd630b55 100644
--- a/tools/testing/selftests/sched_ext/exit.c
+++ b/tools/testing/selftests/sched_ext/exit.c
@@ -32,7 +32,7 @@ static enum scx_test_status run(void *ctx)
skel = exit__open();
skel->rodata->exit_point = tc;
- exit__load(skel);
+ SCX_FAIL_IF(exit__load(skel), "Failed to load skel");
link = bpf_map__attach_struct_ops(skel->maps.exit_ops);
if (!link) {
SCX_ERR("Failed to attach scheduler");
diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c
index 7bc74969a819..9cbed8140e04 100644
--- a/virt/kvm/dirty_ring.c
+++ b/virt/kvm/dirty_ring.c
@@ -66,7 +66,8 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask)
memslot = id_to_memslot(__kvm_memslots(kvm, as_id), id);
- if (!memslot || (offset + __fls(mask)) >= memslot->npages)
+ if (!memslot || offset >= memslot->npages ||
+ offset + __fls(mask) >= memslot->npages)
return;
KVM_MMU_LOCK(kvm);
diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c
index 388ae471d258..53262b8a7656 100644
--- a/virt/kvm/vfio.c
+++ b/virt/kvm/vfio.c
@@ -190,11 +190,10 @@ static int kvm_vfio_file_del(struct kvm_device *dev, unsigned int fd)
{
struct kvm_vfio *kv = dev->private;
struct kvm_vfio_file *kvf;
- struct fd f;
+ CLASS(fd, f)(fd);
int ret;
- f = fdget(fd);
- if (!fd_file(f))
+ if (fd_empty(f))
return -EBADF;
ret = -ENOENT;
@@ -220,9 +219,6 @@ static int kvm_vfio_file_del(struct kvm_device *dev, unsigned int fd)
kvm_vfio_update_coherency(dev);
mutex_unlock(&kv->lock);
-
- fdput(f);
-
return ret;
}
next prev parent reply other threads:[~2026-05-23 11:46 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-23 11:46 Linux 6.12.91 Greg Kroah-Hartman
2026-05-23 11:46 ` Greg Kroah-Hartman [this message]
2026-05-24 0:56 ` Wang Yugui
2026-05-26 11:35 ` Sasha Levin
2026-05-27 19:48 ` Linux 6.12.91 - perf build break: undefined parse_events__term_type_str / first_wildcard_match Sasha Levin
2026-05-29 5:18 ` Wang Yugui
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2026052319-surreal-prewashed-de1a@gregkh \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=lwn@lwn.net \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.