From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
torvalds@linux-foundation.org, stable@vger.kernel.org
Cc: lwn@lwn.net, jslaby@suse.cz,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: Re: Linux 6.12.118
Date: Tue, 17 Nov 2026 11:45:24 +0100 [thread overview]
Message-ID: <2026111724-snowbound-swear-ef83@gregkh> (raw)
In-Reply-To: <2026111724-motivate-lifting-94f1@gregkh>
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 886095f45290..c0748a3c5aa6 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4214,6 +4214,11 @@
This can be set from sysctl after boot.
See Documentation/admin-guide/sysctl/vm.rst for details.
+ nvme_tcp.tls_handshake_timeout=
+ [NVME] NVMe/TCP host TLS handshake timeout in seconds.
+ Valid values range from 0 to the smaller of
+ UINT_MAX / 1000 and MAX_JIFFY_OFFSET / HZ. Default: 10
+
ohci1394_dma=early [HW,EARLY] enable debugging via the ohci1394 driver.
See Documentation/core-api/debugging-via-ohci1394.rst for more
info.
@@ -7260,6 +7265,11 @@
q = USB_QUIRK_FORCE_ONE_CONFIG (Device
claims zero configurations,
forcing to 1);
+ r = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE (Device
+ fails during initialization when asked for
+ 9-bytes configuration descriptor request.
+ Ask for 255-bytes request instead to mirror
+ Windows' behavior);
Example: quirks=0781:5580:bk,0a5c:5834:gij
usbhid.mousepoll=
diff --git a/Documentation/filesystems/fsverity.rst b/Documentation/filesystems/fsverity.rst
index 0e2fac7a16da..f879e052d4ec 100644
--- a/Documentation/filesystems/fsverity.rst
+++ b/Documentation/filesystems/fsverity.rst
@@ -170,6 +170,7 @@ fatal signal), no changes are made to the file.
FS_IOC_ENABLE_VERITY can fail with the following errors:
- ``EACCES``: the process does not have write access to the file
+- ``EBADF``: the file descriptor is not open for reading
- ``EBADMSG``: the builtin signature is malformed
- ``EBUSY``: this ioctl is already running on the file
- ``EEXIST``: the file already has verity enabled
@@ -233,6 +234,7 @@ regardless of the size of the file.
FS_IOC_MEASURE_VERITY can fail with the following errors:
+- ``EBADF``: the file descriptor is not open for reading
- ``EFAULT``: the caller provided inaccessible memory
- ``ENODATA``: the file is not a verity file
- ``ENOTTY``: this type of filesystem does not implement fs-verity
@@ -305,6 +307,7 @@ blocks from disk without actually verifying the path to the root node.
FS_IOC_READ_VERITY_METADATA can fail with the following errors:
+- ``EBADF``: the file descriptor is not open for reading
- ``EFAULT``: the caller provided inaccessible memory
- ``EINTR``: the ioctl was interrupted before any data was read
- ``EINVAL``: reserved fields were set, or ``offset + length``
diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index c51fb6f2651f..0e01bfc75fce 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -678,7 +678,8 @@ When decoding an overlay file handle, the following steps are followed:
1. Find underlying layer by UUID and path type information.
2. Decode the underlying filesystem file handle to underlying dentry.
- 3. For a lower file handle, lookup the handle in index directory by name.
+ 3. For a lower file handle, encode the decoded underlying inode again and
+ lookup that handle in index directory by name.
4. If a whiteout is found in index, return ESTALE. This represents an
overlay object that was deleted after its file handle was encoded.
5. For a non-directory, instantiate a disconnected overlay dentry from the
diff --git a/Documentation/hwmon/coretemp.rst b/Documentation/hwmon/coretemp.rst
index c609329e3bc4..6089bee1159f 100644
--- a/Documentation/hwmon/coretemp.rst
+++ b/Documentation/hwmon/coretemp.rst
@@ -47,6 +47,9 @@ All Sysfs entries are named with their core_id (represented here by 'X').
================= ========================================================
tempX_input Core temperature (in millidegrees Celsius).
+ Reading fails with -ENODATA while the CPU reports the
+ temperature as invalid. Some CPUs do this when the
+ temperature is too low to be measured.
tempX_max All cooling devices should be turned on (on Core2).
tempX_crit Maximum junction temperature (in millidegrees Celsius).
tempX_crit_alarm Set when Out-of-spec bit is set, never clears.
diff --git a/Makefile b/Makefile
index 1464c3c0e2a5..c0d66e826498 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 6
PATCHLEVEL = 12
-SUBLEVEL = 117
+SUBLEVEL = 118
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arm64/boot/dts/qcom/ipq9574.dtsi b/arch/arm64/boot/dts/qcom/ipq9574.dtsi
index 81ccd0600c5a..31f97309c533 100644
--- a/arch/arm64/boot/dts/qcom/ipq9574.dtsi
+++ b/arch/arm64/boot/dts/qcom/ipq9574.dtsi
@@ -315,7 +315,7 @@ gcc: clock-controller@1800000 {
<0>,
<0>,
<0>,
- <0>;
+ <&usb_0_qmpphy>;
#clock-cells = <1>;
#reset-cells = <1>;
#interconnect-cells = <1>;
diff --git a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
index cb5e8364c973..fc79641d7bb5 100644
--- a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
@@ -7,12 +7,20 @@
/dts-v1/;
+#include "k3-am625.dtsi"
#include "k3-am62x-sk-common.dtsi"
/ {
compatible = "ti,am62-lp-sk", "ti,am625";
model = "Texas Instruments AM62x LP SK";
+ memory@80000000 {
+ /* 2G RAM */
+ reg = <0x00000000 0x80000000 0x00000000 0x80000000>;
+ device_type = "memory";
+ bootph-pre-ram;
+ };
+
vmain_pd: regulator-0 {
/* TPS65988 PD CONTROLLER OUTPUT */
compatible = "regulator-fixed";
diff --git a/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi b/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi
new file mode 100644
index 000000000000..db85d354133f
--- /dev/null
+++ b/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi
@@ -0,0 +1,304 @@
+// SPDX-License-Identifier: GPL-2.0-only OR MIT
+/*
+ * Common dtsi for AM625 SK and derivatives
+ *
+ * Copyright (C) 2021-2024 Texas Instruments Incorporated - https://www.ti.com/
+ */
+
+#include "k3-am62x-sk-common.dtsi"
+
+/ {
+ opp-table {
+ /* Add 1.4GHz OPP for am625-sk board. Requires VDD_CORE to be at 0.85V */
+ opp-1400000000 {
+ opp-hz = /bits/ 64 <1400000000>;
+ opp-supported-hw = <0x01 0x0004>;
+ clock-latency-ns = <6000000>;
+ };
+ };
+
+ vmain_pd: regulator-0 {
+ /* TPS65988 PD CONTROLLER OUTPUT */
+ bootph-all;
+ compatible = "regulator-fixed";
+ regulator-name = "vmain_pd";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vcc_5v0: regulator-1 {
+ /* Output of LM34936 */
+ bootph-all;
+ compatible = "regulator-fixed";
+ regulator-name = "vcc_5v0";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ vin-supply = <&vmain_pd>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vcc_3v3_sys: regulator-2 {
+ /* output of LM61460-Q1 */
+ bootph-all;
+ compatible = "regulator-fixed";
+ regulator-name = "vcc_3v3_sys";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ vin-supply = <&vmain_pd>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+
+ vdd_mmc1: regulator-3 {
+ /* TPS22918DBVR */
+ bootph-all;
+ compatible = "regulator-fixed";
+ regulator-name = "vdd_mmc1";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ enable-active-high;
+ vin-supply = <&vcc_3v3_sys>;
+ gpio = <&exp1 3 GPIO_ACTIVE_HIGH>;
+ };
+
+ vdd_sd_dv: regulator-4 {
+ /* Output of TLV71033 */
+ bootph-all;
+ compatible = "regulator-gpio";
+ regulator-name = "tlv71033";
+ pinctrl-names = "default";
+ pinctrl-0 = <&vdd_sd_dv_pins_default>;
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-boot-on;
+ vin-supply = <&vcc_5v0>;
+ gpios = <&main_gpio0 31 GPIO_ACTIVE_HIGH>;
+ states = <1800000 0x0>,
+ <3300000 0x1>;
+ };
+
+ vcc_1v8: regulator-5 {
+ /* output of TPS6282518DMQ */
+ compatible = "regulator-fixed";
+ regulator-name = "vcc_1v8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ vin-supply = <&vcc_3v3_sys>;
+ regulator-always-on;
+ regulator-boot-on;
+ };
+};
+
+&main_pmx0 {
+ main_mmc0_pins_default: main-mmc0-default-pins {
+ bootph-all;
+ pinctrl-single,pins = <
+ AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */
+ AM62X_IOPAD(0x218, PIN_OUTPUT, 0) /* (AB1) MMC0_CLK */
+ AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */
+ AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */
+ AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */
+ AM62X_IOPAD(0x208, PIN_INPUT_PULLUP, 0) /* (Y4) MMC0_DAT3 */
+ AM62X_IOPAD(0x204, PIN_INPUT_PULLUP, 0) /* (AB2) MMC0_DAT4 */
+ AM62X_IOPAD(0x200, PIN_INPUT_PULLUP, 0) /* (AC1) MMC0_DAT5 */
+ AM62X_IOPAD(0x1fc, PIN_INPUT_PULLUP, 0) /* (AD2) MMC0_DAT6 */
+ AM62X_IOPAD(0x1f8, PIN_INPUT_PULLUP, 0) /* (AC2) MMC0_DAT7 */
+ >;
+ };
+
+ main_rgmii2_pins_default: main-rgmii2-default-pins {
+ bootph-all;
+ pinctrl-single,pins = <
+ AM62X_IOPAD(0x184, PIN_INPUT, 0) /* (AE23) RGMII2_RD0 */
+ AM62X_IOPAD(0x188, PIN_INPUT, 0) /* (AB20) RGMII2_RD1 */
+ AM62X_IOPAD(0x18c, PIN_INPUT, 0) /* (AC21) RGMII2_RD2 */
+ AM62X_IOPAD(0x190, PIN_INPUT, 0) /* (AE22) RGMII2_RD3 */
+ AM62X_IOPAD(0x180, PIN_INPUT, 0) /* (AD23) RGMII2_RXC */
+ AM62X_IOPAD(0x17c, PIN_INPUT, 0) /* (AD22) RGMII2_RX_CTL */
+ AM62X_IOPAD(0x16c, PIN_OUTPUT, 0) /* (Y18) RGMII2_TD0 */
+ AM62X_IOPAD(0x170, PIN_OUTPUT, 0) /* (AA18) RGMII2_TD1 */
+ AM62X_IOPAD(0x174, PIN_OUTPUT, 0) /* (AD21) RGMII2_TD2 */
+ AM62X_IOPAD(0x178, PIN_OUTPUT, 0) /* (AC20) RGMII2_TD3 */
+ AM62X_IOPAD(0x168, PIN_OUTPUT, 0) /* (AE21) RGMII2_TXC */
+ AM62X_IOPAD(0x164, PIN_OUTPUT, 0) /* (AA19) RGMII2_TX_CTL */
+ >;
+ };
+
+ ospi0_pins_default: ospi0-default-pins {
+ bootph-all;
+ pinctrl-single,pins = <
+ AM62X_IOPAD(0x000, PIN_OUTPUT, 0) /* (H24) OSPI0_CLK */
+ AM62X_IOPAD(0x02c, PIN_OUTPUT, 0) /* (F23) OSPI0_CSn0 */
+ AM62X_IOPAD(0x00c, PIN_INPUT, 0) /* (E25) OSPI0_D0 */
+ AM62X_IOPAD(0x010, PIN_INPUT, 0) /* (G24) OSPI0_D1 */
+ AM62X_IOPAD(0x014, PIN_INPUT, 0) /* (F25) OSPI0_D2 */
+ AM62X_IOPAD(0x018, PIN_INPUT, 0) /* (F24) OSPI0_D3 */
+ AM62X_IOPAD(0x01c, PIN_INPUT, 0) /* (J23) OSPI0_D4 */
+ AM62X_IOPAD(0x020, PIN_INPUT, 0) /* (J25) OSPI0_D5 */
+ AM62X_IOPAD(0x024, PIN_INPUT, 0) /* (H25) OSPI0_D6 */
+ AM62X_IOPAD(0x028, PIN_INPUT, 0) /* (J22) OSPI0_D7 */
+ AM62X_IOPAD(0x008, PIN_INPUT, 0) /* (J24) OSPI0_DQS */
+ >;
+ };
+
+ vdd_sd_dv_pins_default: vdd-sd-dv-default-pins {
+ bootph-all;
+ pinctrl-single,pins = <
+ AM62X_IOPAD(0x07c, PIN_OUTPUT, 7) /* (P25) GPMC0_CLK.GPIO0_31 */
+ >;
+ };
+
+ main_gpio1_ioexp_intr_pins_default: main-gpio1-ioexp-intr-default-pins {
+ bootph-all;
+ pinctrl-single,pins = <
+ AM62X_IOPAD(0x01d4, PIN_INPUT, 7) /* (B15) UART0_RTSn.GPIO1_23 */
+ >;
+ };
+};
+
+&main_gpio0 {
+ bootph-all;
+};
+
+&main_gpio1 {
+ bootph-all;
+};
+
+&main_i2c1 {
+ bootph-all;
+ exp1: gpio@22 {
+ bootph-all;
+ compatible = "ti,tca6424";
+ reg = <0x22>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ gpio-line-names = "GPIO_CPSW2_RST", "GPIO_CPSW1_RST",
+ "PRU_DETECT", "MMC1_SD_EN",
+ "VPP_LDO_EN", "EXP_PS_3V3_En",
+ "EXP_PS_5V0_En", "EXP_HAT_DETECT",
+ "GPIO_AUD_RSTn", "GPIO_eMMC_RSTn",
+ "UART1_FET_BUF_EN", "WL_LT_EN",
+ "GPIO_HDMI_RSTn", "CSI_GPIO1",
+ "CSI_GPIO2", "PRU_3V3_EN",
+ "HDMI_INTn", "PD_I2C_IRQ",
+ "MCASP1_FET_EN", "MCASP1_BUF_BT_EN",
+ "MCASP1_FET_SEL", "UART1_FET_SEL",
+ "TSINT#", "IO_EXP_TEST_LED";
+
+ interrupt-parent = <&main_gpio1>;
+ interrupts = <23 IRQ_TYPE_EDGE_FALLING>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_gpio1_ioexp_intr_pins_default>;
+ };
+};
+
+&sdhci0 {
+ bootph-all;
+ non-removable;
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_mmc0_pins_default>;
+ status = "okay";
+};
+
+&sdhci1 {
+ vmmc-supply = <&vdd_mmc1>;
+ vqmmc-supply = <&vdd_sd_dv>;
+};
+
+&cpsw3g {
+ pinctrl-names = "default";
+ pinctrl-0 = <&main_rgmii1_pins_default>, <&main_rgmii2_pins_default>;
+};
+
+&cpsw_port2 {
+ phy-mode = "rgmii-rxid";
+ phy-handle = <&cpsw3g_phy1>;
+};
+
+&cpsw3g_mdio {
+ cpsw3g_phy1: ethernet-phy@1 {
+ reg = <1>;
+ ti,rx-internal-delay = <DP83867_RGMIIDCTL_2_00_NS>;
+ ti,fifo-depth = <DP83867_PHYCR_FIFO_DEPTH_4_B_NIB>;
+ ti,min-output-impedance;
+ };
+};
+
+&fss {
+ bootph-all;
+};
+
+&ospi0 {
+ bootph-all;
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&ospi0_pins_default>;
+
+ flash@0 {
+ bootph-all;
+ compatible = "jedec,spi-nor";
+ reg = <0x0>;
+ spi-tx-bus-width = <8>;
+ spi-rx-bus-width = <8>;
+ spi-max-frequency = <25000000>;
+ cdns,tshsl-ns = <60>;
+ cdns,tsd2d-ns = <60>;
+ cdns,tchsh-ns = <60>;
+ cdns,tslch-ns = <60>;
+ cdns,read-delay = <4>;
+
+ partitions {
+ bootph-all;
+ compatible = "fixed-partitions";
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ partition@0 {
+ label = "ospi.tiboot3";
+ reg = <0x0 0x80000>;
+ };
+
+ partition@80000 {
+ label = "ospi.tispl";
+ reg = <0x80000 0x200000>;
+ };
+
+ partition@280000 {
+ label = "ospi.u-boot";
+ reg = <0x280000 0x400000>;
+ };
+
+ partition@680000 {
+ label = "ospi.env";
+ reg = <0x680000 0x40000>;
+ };
+
+ partition@6c0000 {
+ label = "ospi.env.backup";
+ reg = <0x6c0000 0x40000>;
+ };
+
+ partition@800000 {
+ label = "ospi.rootfs";
+ reg = <0x800000 0x37c0000>;
+ };
+
+ partition@3fc0000 {
+ bootph-pre-ram;
+ label = "ospi.phypattern";
+ reg = <0x3fc0000 0x40000>;
+ };
+ };
+ };
+};
+
+&tlv320aic3106 {
+ DVDD-supply = <&vcc_1v8>;
+};
diff --git a/arch/arm64/boot/dts/ti/k3-am625-sk.dts b/arch/arm64/boot/dts/ti/k3-am625-sk.dts
index 0fa11d3aa71e..52954c77df80 100644
--- a/arch/arm64/boot/dts/ti/k3-am625-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am625-sk.dts
@@ -7,317 +7,17 @@
/dts-v1/;
-#include "k3-am62x-sk-common.dtsi"
+#include "k3-am625.dtsi"
+#include "k3-am625-sk-common.dtsi"
/ {
compatible = "ti,am625-sk", "ti,am625";
model = "Texas Instruments AM625 SK";
- opp-table {
- /* Add 1.4GHz OPP for am625-sk board. Requires VDD_CORE to be at 0.85V */
- opp-1400000000 {
- opp-hz = /bits/ 64 <1400000000>;
- opp-supported-hw = <0x01 0x0004>;
- clock-latency-ns = <6000000>;
- };
- };
-
memory@80000000 {
- device_type = "memory";
/* 2G RAM */
reg = <0x00000000 0x80000000 0x00000000 0x80000000>;
-
- };
-
- vmain_pd: regulator-0 {
- /* TPS65988 PD CONTROLLER OUTPUT */
- bootph-all;
- compatible = "regulator-fixed";
- regulator-name = "vmain_pd";
- regulator-min-microvolt = <5000000>;
- regulator-max-microvolt = <5000000>;
- regulator-always-on;
- regulator-boot-on;
- };
-
- vcc_5v0: regulator-1 {
- /* Output of LM34936 */
- bootph-all;
- compatible = "regulator-fixed";
- regulator-name = "vcc_5v0";
- regulator-min-microvolt = <5000000>;
- regulator-max-microvolt = <5000000>;
- vin-supply = <&vmain_pd>;
- regulator-always-on;
- regulator-boot-on;
- };
-
- vcc_3v3_sys: regulator-2 {
- /* output of LM61460-Q1 */
- bootph-all;
- compatible = "regulator-fixed";
- regulator-name = "vcc_3v3_sys";
- regulator-min-microvolt = <3300000>;
- regulator-max-microvolt = <3300000>;
- vin-supply = <&vmain_pd>;
- regulator-always-on;
- regulator-boot-on;
- };
-
- vdd_mmc1: regulator-3 {
- /* TPS22918DBVR */
- bootph-all;
- compatible = "regulator-fixed";
- regulator-name = "vdd_mmc1";
- regulator-min-microvolt = <3300000>;
- regulator-max-microvolt = <3300000>;
- regulator-boot-on;
- enable-active-high;
- vin-supply = <&vcc_3v3_sys>;
- gpio = <&exp1 3 GPIO_ACTIVE_HIGH>;
- };
-
- vdd_sd_dv: regulator-4 {
- /* Output of TLV71033 */
- bootph-all;
- compatible = "regulator-gpio";
- regulator-name = "tlv71033";
- pinctrl-names = "default";
- pinctrl-0 = <&vdd_sd_dv_pins_default>;
- regulator-min-microvolt = <1800000>;
- regulator-max-microvolt = <3300000>;
- regulator-boot-on;
- vin-supply = <&vcc_5v0>;
- gpios = <&main_gpio0 31 GPIO_ACTIVE_HIGH>;
- states = <1800000 0x0>,
- <3300000 0x1>;
- };
-
- vcc_1v8: regulator-5 {
- /* output of TPS6282518DMQ */
- compatible = "regulator-fixed";
- regulator-name = "vcc_1v8";
- regulator-min-microvolt = <1800000>;
- regulator-max-microvolt = <1800000>;
- vin-supply = <&vcc_3v3_sys>;
- regulator-always-on;
- regulator-boot-on;
- };
-};
-
-&main_pmx0 {
- main_mmc0_pins_default: main-mmc0-default-pins {
- bootph-all;
- pinctrl-single,pins = <
- AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */
- AM62X_IOPAD(0x218, PIN_INPUT, 0) /* (AB1) MMC0_CLK */
- AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */
- AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */
- AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */
- AM62X_IOPAD(0x208, PIN_INPUT_PULLUP, 0) /* (Y4) MMC0_DAT3 */
- AM62X_IOPAD(0x204, PIN_INPUT_PULLUP, 0) /* (AB2) MMC0_DAT4 */
- AM62X_IOPAD(0x200, PIN_INPUT_PULLUP, 0) /* (AC1) MMC0_DAT5 */
- AM62X_IOPAD(0x1fc, PIN_INPUT_PULLUP, 0) /* (AD2) MMC0_DAT6 */
- AM62X_IOPAD(0x1f8, PIN_INPUT_PULLUP, 0) /* (AC2) MMC0_DAT7 */
- >;
- };
-
- main_rgmii2_pins_default: main-rgmii2-default-pins {
- bootph-all;
- pinctrl-single,pins = <
- AM62X_IOPAD(0x184, PIN_INPUT, 0) /* (AE23) RGMII2_RD0 */
- AM62X_IOPAD(0x188, PIN_INPUT, 0) /* (AB20) RGMII2_RD1 */
- AM62X_IOPAD(0x18c, PIN_INPUT, 0) /* (AC21) RGMII2_RD2 */
- AM62X_IOPAD(0x190, PIN_INPUT, 0) /* (AE22) RGMII2_RD3 */
- AM62X_IOPAD(0x180, PIN_INPUT, 0) /* (AD23) RGMII2_RXC */
- AM62X_IOPAD(0x17c, PIN_INPUT, 0) /* (AD22) RGMII2_RX_CTL */
- AM62X_IOPAD(0x16c, PIN_OUTPUT, 0) /* (Y18) RGMII2_TD0 */
- AM62X_IOPAD(0x170, PIN_OUTPUT, 0) /* (AA18) RGMII2_TD1 */
- AM62X_IOPAD(0x174, PIN_OUTPUT, 0) /* (AD21) RGMII2_TD2 */
- AM62X_IOPAD(0x178, PIN_OUTPUT, 0) /* (AC20) RGMII2_TD3 */
- AM62X_IOPAD(0x168, PIN_OUTPUT, 0) /* (AE21) RGMII2_TXC */
- AM62X_IOPAD(0x164, PIN_OUTPUT, 0) /* (AA19) RGMII2_TX_CTL */
- >;
- };
-
- ospi0_pins_default: ospi0-default-pins {
- bootph-all;
- pinctrl-single,pins = <
- AM62X_IOPAD(0x000, PIN_OUTPUT, 0) /* (H24) OSPI0_CLK */
- AM62X_IOPAD(0x02c, PIN_OUTPUT, 0) /* (F23) OSPI0_CSn0 */
- AM62X_IOPAD(0x00c, PIN_INPUT, 0) /* (E25) OSPI0_D0 */
- AM62X_IOPAD(0x010, PIN_INPUT, 0) /* (G24) OSPI0_D1 */
- AM62X_IOPAD(0x014, PIN_INPUT, 0) /* (F25) OSPI0_D2 */
- AM62X_IOPAD(0x018, PIN_INPUT, 0) /* (F24) OSPI0_D3 */
- AM62X_IOPAD(0x01c, PIN_INPUT, 0) /* (J23) OSPI0_D4 */
- AM62X_IOPAD(0x020, PIN_INPUT, 0) /* (J25) OSPI0_D5 */
- AM62X_IOPAD(0x024, PIN_INPUT, 0) /* (H25) OSPI0_D6 */
- AM62X_IOPAD(0x028, PIN_INPUT, 0) /* (J22) OSPI0_D7 */
- AM62X_IOPAD(0x008, PIN_INPUT, 0) /* (J24) OSPI0_DQS */
- >;
- };
-
- vdd_sd_dv_pins_default: vdd-sd-dv-default-pins {
- bootph-all;
- pinctrl-single,pins = <
- AM62X_IOPAD(0x07c, PIN_OUTPUT, 7) /* (P25) GPMC0_CLK.GPIO0_31 */
- >;
- };
-
- main_gpio1_ioexp_intr_pins_default: main-gpio1-ioexp-intr-default-pins {
- bootph-all;
- pinctrl-single,pins = <
- AM62X_IOPAD(0x01d4, PIN_INPUT, 7) /* (B15) UART0_RTSn.GPIO1_23 */
- >;
- };
-};
-
-&main_gpio0 {
- bootph-all;
-};
-
-&main_gpio1 {
- bootph-all;
-};
-
-&main_i2c1 {
- bootph-all;
- exp1: gpio@22 {
- bootph-all;
- compatible = "ti,tca6424";
- reg = <0x22>;
- gpio-controller;
- #gpio-cells = <2>;
- gpio-line-names = "GPIO_CPSW2_RST", "GPIO_CPSW1_RST",
- "PRU_DETECT", "MMC1_SD_EN",
- "VPP_LDO_EN", "EXP_PS_3V3_En",
- "EXP_PS_5V0_En", "EXP_HAT_DETECT",
- "GPIO_AUD_RSTn", "GPIO_eMMC_RSTn",
- "UART1_FET_BUF_EN", "WL_LT_EN",
- "GPIO_HDMI_RSTn", "CSI_GPIO1",
- "CSI_GPIO2", "PRU_3V3_EN",
- "HDMI_INTn", "PD_I2C_IRQ",
- "MCASP1_FET_EN", "MCASP1_BUF_BT_EN",
- "MCASP1_FET_SEL", "UART1_FET_SEL",
- "TSINT#", "IO_EXP_TEST_LED";
-
- interrupt-parent = <&main_gpio1>;
- interrupts = <23 IRQ_TYPE_EDGE_FALLING>;
- interrupt-controller;
- #interrupt-cells = <2>;
-
- pinctrl-names = "default";
- pinctrl-0 = <&main_gpio1_ioexp_intr_pins_default>;
- };
-};
-
-&sdhci0 {
- bootph-all;
- non-removable;
- pinctrl-names = "default";
- pinctrl-0 = <&main_mmc0_pins_default>;
- status = "okay";
-};
-
-&sdhci1 {
- vmmc-supply = <&vdd_mmc1>;
- vqmmc-supply = <&vdd_sd_dv>;
-};
-
-&cpsw3g {
- pinctrl-names = "default";
- pinctrl-0 = <&main_rgmii1_pins_default>, <&main_rgmii2_pins_default>;
-};
-
-&cpsw_port2 {
- phy-mode = "rgmii-rxid";
- phy-handle = <&cpsw3g_phy1>;
-};
-
-&cpsw3g_mdio {
- cpsw3g_phy1: ethernet-phy@1 {
- reg = <1>;
- ti,rx-internal-delay = <DP83867_RGMIIDCTL_2_00_NS>;
- ti,fifo-depth = <DP83867_PHYCR_FIFO_DEPTH_4_B_NIB>;
- ti,min-output-impedance;
- };
-};
-
-&mailbox0_cluster0 {
- mbox_m4_0: mbox-m4-0 {
- ti,mbox-rx = <0 0 0>;
- ti,mbox-tx = <1 0 0>;
- };
-};
-
-&fss {
- bootph-all;
-};
-
-&ospi0 {
- bootph-all;
- status = "okay";
- pinctrl-names = "default";
- pinctrl-0 = <&ospi0_pins_default>;
-
- flash@0 {
- bootph-all;
- compatible = "jedec,spi-nor";
- reg = <0x0>;
- spi-tx-bus-width = <8>;
- spi-rx-bus-width = <8>;
- spi-max-frequency = <25000000>;
- cdns,tshsl-ns = <60>;
- cdns,tsd2d-ns = <60>;
- cdns,tchsh-ns = <60>;
- cdns,tslch-ns = <60>;
- cdns,read-delay = <4>;
-
- partitions {
- bootph-all;
- compatible = "fixed-partitions";
- #address-cells = <1>;
- #size-cells = <1>;
-
- partition@0 {
- label = "ospi.tiboot3";
- reg = <0x0 0x80000>;
- };
-
- partition@80000 {
- label = "ospi.tispl";
- reg = <0x80000 0x200000>;
- };
-
- partition@280000 {
- label = "ospi.u-boot";
- reg = <0x280000 0x400000>;
- };
-
- partition@680000 {
- label = "ospi.env";
- reg = <0x680000 0x40000>;
- };
-
- partition@6c0000 {
- label = "ospi.env.backup";
- reg = <0x6c0000 0x40000>;
- };
-
- partition@800000 {
- label = "ospi.rootfs";
- reg = <0x800000 0x37c0000>;
- };
-
- partition@3fc0000 {
- bootph-pre-ram;
- label = "ospi.phypattern";
- reg = <0x3fc0000 0x40000>;
- };
- };
+ device_type = "memory";
+ bootph-pre-ram;
};
};
-
-&tlv320aic3106 {
- DVDD-supply = <&vcc_1v8>;
-};
diff --git a/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi b/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi
index 9d45d8dad381..be5cc73cf2e4 100644
--- a/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi
@@ -8,7 +8,6 @@
#include <dt-bindings/leds/common.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/net/ti-dp83867.h>
-#include "k3-am625.dtsi"
/ {
aliases {
@@ -27,13 +26,6 @@ chosen {
stdout-path = "serial2:115200n8";
};
- memory@80000000 {
- bootph-pre-ram;
- device_type = "memory";
- /* 2G RAM */
- reg = <0x00000000 0x80000000 0x00000000 0x80000000>;
- };
-
reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h
index 484cb6972e99..3b689185f371 100644
--- a/arch/arm64/include/asm/sdei.h
+++ b/arch/arm64/include/asm/sdei.h
@@ -44,7 +44,8 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
struct sdei_registered_event *arg);
unsigned long do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg);
+ struct sdei_registered_event *arg,
+ unsigned long vbar);
unsigned long sdei_arch_get_entry_point(int conduit);
#define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x)
diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h
index eb60c9735553..2cc5fb405769 100644
--- a/arch/arm64/include/asm/suspend.h
+++ b/arch/arm64/include/asm/suspend.h
@@ -2,7 +2,7 @@
#ifndef __ASM_SUSPEND_H
#define __ASM_SUSPEND_H
-#define NR_CTX_REGS 14
+#define NR_CTX_REGS 15
#define NR_CALLEE_SAVED_REGS 12
/*
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index ea3876d99c2e..4033ee7005a4 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -27,6 +27,7 @@
#include <asm/stacktrace.h>
#include <asm/sysreg.h>
#include <asm/system_misc.h>
+#include <asm/vectors.h>
/*
* Handle IRQ/context state management when entering from kernel mode.
@@ -1087,6 +1088,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
asmlinkage noinstr unsigned long
__sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
{
+ unsigned long elr, spsr, esr, far, vbar;
unsigned long ret;
/*
@@ -1111,10 +1113,46 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
else if (cpu_has_pan())
set_pstate_pan(0);
+ /*
+ * Nothing saved the interrupted context's exception registers on the
+ * way in, and it may be in the middle of exception entry or return
+ * (SDEI ignores DAIF). Save them before anything can take an exception
+ * and overwrite them.
+ */
+ elr = read_sysreg(elr_el1);
+ spsr = read_sysreg(spsr_el1);
+ esr = read_sysreg(esr_el1);
+ far = read_sysreg(far_el1);
+
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
+ * handle exceptions from EL0: an exception taken here would run their
+ * empty current-EL half and never come back. Switch to the kernel
+ * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
+ * still goes through the trampoline.
+ */
+ vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0() &&
+ vbar == (unsigned long)raw_cpu_read(this_cpu_vector)) {
+ write_sysreg((unsigned long)vectors, vbar_el1);
+ isb();
+ }
+
arm64_enter_nmi(regs);
- ret = do_sdei_event(regs, arg);
+ ret = do_sdei_event(regs, arg, vbar);
arm64_exit_nmi(regs);
+ /*
+ * Put them back unconditionally. An unchanged ELR does not rule out a
+ * nested exception: one may have returned to the same address.
+ */
+ write_sysreg(elr, elr_el1);
+ write_sysreg(spsr, spsr_el1);
+ write_sysreg(esr, esr_el1);
+ write_sysreg(far, far_el1);
+ write_sysreg(vbar, vbar_el1);
+
return ret;
}
#endif /* CONFIG_ARM_SDE_INTERFACE */
diff --git a/arch/arm64/kernel/machine_kexec_file.c b/arch/arm64/kernel/machine_kexec_file.c
index af1ca875c52c..dc6948967ceb 100644
--- a/arch/arm64/kernel/machine_kexec_file.c
+++ b/arch/arm64/kernel/machine_kexec_file.c
@@ -128,6 +128,8 @@ int load_other_segments(struct kimage *image,
vfree(headers);
goto out_err;
}
+
+ vfree(image->elf_headers);
image->elf_headers = headers;
image->elf_load_addr = kbuf.mem;
image->elf_headers_sz = headers_sz;
diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c
index 255d12f881c2..a6d78ceb1364 100644
--- a/arch/arm64/kernel/sdei.c
+++ b/arch/arm64/kernel/sdei.c
@@ -203,21 +203,49 @@ unsigned long sdei_arch_get_entry_point(int conduit)
return 0;
}
+/*
+ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
+ * restored the interrupted context's exception registers. Their handlers are
+ * safe in NMI context:
+ * - EL1 data aborts on translation, permission and access-flag faults:
+ * faulthandler_disabled() is true in NMI context, so do_page_fault() never
+ * takes the mmap lock. This covers the _nofault() accessors used by the
+ * generic dump_stack()/show_regs() code (e.g. print_worker_info()).
+ * - BRK: kprobes, WARN/BUG, kCFI and UBSAN traps, all NMI-capable.
+ * Anything else taken at EL1 is fatal on its own, or unexpected here.
+ */
+static bool sdei_nested_exception_ok(unsigned long esr)
+{
+ switch (ESR_ELx_EC(esr)) {
+ case ESR_ELx_EC_DABT_CUR:
+ return esr_fsc_is_translation_fault(esr) ||
+ esr_fsc_is_permission_fault(esr) ||
+ esr_fsc_is_access_flag_fault(esr);
+ case ESR_ELx_EC_BRK64:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* do_sdei_event() returns one of:
* SDEI_EV_HANDLED - success, return to the interrupted context.
* SDEI_EV_FAILED - failure, return this error code to firmare.
* virtual-address - success, return to this address.
+ *
+ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
*/
unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg)
+ struct sdei_registered_event *arg,
+ unsigned long vbar)
{
u32 mode;
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
+ u64 nested_elr, nested_esr, nested_far;
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
- unsigned long vbar = read_sysreg(vbar_el1);
if (arm64_kernel_unmapped_at_el0())
clobbered_registers++;
@@ -229,18 +257,27 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
}
err = sdei_event_handler(regs, arg);
- if (err)
- return SDEI_EV_FAILED;
- if (elr != read_sysreg(elr_el1)) {
- /*
- * We took a synchronous exception from the SDEI handler.
- * This could deadlock, and if you interrupt KVM it will
- * hyp-panic instead.
- */
- pr_warn("unsafe: exception during handler\n");
+ /*
+ * A synchronous exception taken by the handler has overwritten the
+ * exception registers; __sdei_handler() puts them back. Only the last
+ * one's syndrome survives; report it if it is not one known to be
+ * harmless.
+ *
+ * If the event interrupted a KVM world switch, VBAR points at the KVM
+ * hyp vectors and any such exception is a hyp panic instead.
+ */
+ nested_elr = read_sysreg(elr_el1);
+ nested_esr = read_sysreg(esr_el1);
+ nested_far = read_sysreg(far_el1);
+ if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
+ pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
+ nested_esr, nested_far, (void *)nested_elr);
}
+ if (err)
+ return SDEI_EV_FAILED;
+
mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
/*
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 052e9b9945fe..326cf60d8d58 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -289,9 +289,7 @@ int counters_read_on_cpu(int cpu, smp_call_func_t func, u64 *val)
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
- smp_call_function_single(cpu, func, val, 1);
-
- return 0;
+ return smp_call_function_single(cpu, func, val, 1);
}
/*
@@ -318,10 +316,19 @@ bool cpc_ffh_supported(void)
return true;
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->bit_width && reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
{
int ret = -EOPNOTSUPP;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
switch ((u64)reg->address) {
case 0x0:
ret = counters_read_on_cpu(cpu, cpu_read_corecnt, val);
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 754914d9ec68..89ad3aee4715 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -506,8 +506,6 @@ static void timer_save_state(struct arch_timer_context *ctx)
goto out;
switch (index) {
- u64 cval;
-
case TIMER_VTIMER:
case TIMER_HVTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTV_CTL));
@@ -537,11 +535,12 @@ static void timer_save_state(struct arch_timer_context *ctx)
case TIMER_PTIMER:
case TIMER_HPTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL));
- cval = read_sysreg_el0(SYS_CNTP_CVAL);
-
- cval -= timer_get_offset(ctx);
-
- timer_set_cval(ctx, cval);
+ /*
+ * With an offset, memory already holds the guest's CVAL (the
+ * trap handler or __deactivate_traps() wrote it).
+ */
+ if (!timer_get_offset(ctx))
+ timer_set_cval(ctx, read_sysreg_el0(SYS_CNTP_CVAL));
/* Disable the timer */
write_sysreg_el0(0, SYS_CNTP_CTL);
@@ -626,7 +625,8 @@ static void timer_restore_state(struct arch_timer_context *ctx)
cval = timer_get_cval(ctx);
offset = timer_get_offset(ctx);
set_cntpoff(offset);
- cval += offset;
+ if (offset)
+ cval = timer_apply_offset(cval, offset, kvm_phys_timer_read());
write_sysreg_el0(cval, SYS_CNTP_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 97d993f52b57..ec59faa70f97 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -1189,30 +1189,33 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
local_irq_disable();
- kvm_vgic_flush_hwstate(vcpu);
-
- kvm_pmu_update_vcpu_events(vcpu);
-
/*
* Ensure we set mode to IN_GUEST_MODE after we disable
* interrupts and before the final VCPU requests check.
* See the comment in kvm_vcpu_exiting_guest_mode() and
* Documentation/virt/kvm/vcpu-requests.rst
+ *
+ * Requests to self must be posted before this point, or
+ * won't be taken into account until the next exit. No state
+ * loading other than what happens at vcpu_load() time can
+ * be done before we have checked for pending requests.
*/
smp_store_mb(vcpu->mode, IN_GUEST_MODE);
if (ret <= 0 || kvm_vcpu_exit_request(vcpu, &ret)) {
- isb(); /* Ensure work in x_flush_hwstate is committed */
- kvm_pmu_sync_hwstate(vcpu);
if (unlikely(!irqchip_in_kernel(vcpu->kvm)))
kvm_timer_sync_user(vcpu);
- kvm_vgic_sync_hwstate(vcpu);
smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
local_irq_enable();
preempt_enable();
continue;
}
+ /* From here onwards, we are commited to entering the guest */
+ kvm_vgic_flush_hwstate(vcpu);
+
+ kvm_pmu_update_vcpu_events(vcpu);
+
kvm_arm_setup_debug(vcpu);
kvm_arch_vcpu_ctxflush_fp(vcpu);
diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
index 424a5107cddb..e20979138305 100644
--- a/arch/arm64/kvm/hyp/exception.c
+++ b/arch/arm64/kvm/hyp/exception.c
@@ -92,6 +92,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
enum exception_type type)
{
unsigned long sctlr, vbar, old, new, mode;
+ struct kvm *kvm;
u64 exc_offset;
mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
@@ -131,8 +132,10 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
new |= (old & PSR_C_BIT);
new |= (old & PSR_V_BIT);
- if (kvm_has_mte(kern_hyp_va(vcpu->kvm)))
+ kvm = vcpu_get_kvm(vcpu);
+ if (kvm && kvm_has_mte(kvm))
new |= PSR_TCO_BIT;
+ vcpu_put_kvm(vcpu, kvm);
new |= (old & PSR_DIT_BIT);
diff --git a/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h b/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h
index 4fdfeabefeb4..f55950ee2a7e 100644
--- a/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h
+++ b/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h
@@ -13,6 +13,34 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_host.h>
+#ifdef __KVM_NVHE_HYPERVISOR__
+#include <nvhe/pkvm.h>
+
+/* Under pKVM a host vCPU's ->kvm is host-writable. */
+static inline struct kvm *vcpu_get_kvm(struct kvm_vcpu *vcpu)
+{
+ if (is_protected_kvm_enabled())
+ return pkvm_vcpu_get_kvm(vcpu);
+
+ return kern_hyp_va(vcpu->kvm);
+}
+
+static inline void vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm)
+{
+ if (is_protected_kvm_enabled())
+ pkvm_vcpu_put_kvm(vcpu, kvm);
+}
+#else
+static inline struct kvm *vcpu_get_kvm(struct kvm_vcpu *vcpu)
+{
+ return vcpu->kvm;
+}
+
+static inline void vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm)
+{
+}
+#endif
+
static inline void kvm_skip_instr(struct kvm_vcpu *vcpu)
{
if (vcpu_mode_is_32bit(vcpu)) {
diff --git a/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h b/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h
index f957890c7e38..3a9909f565f2 100644
--- a/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h
+++ b/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h
@@ -92,7 +92,8 @@
*/
#define PVM_ID_AA64MMFR0_RESTRICT_UNSIGNED (\
FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64MMFR0_EL1_PARANGE), ID_AA64MMFR0_EL1_PARANGE_40) | \
- FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64MMFR0_EL1_ASIDBITS), ID_AA64MMFR0_EL1_ASIDBITS_16) \
+ FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64MMFR0_EL1_ASIDBITS), ID_AA64MMFR0_EL1_ASIDBITS_16) | \
+ SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN16, IMP) \
)
/*
diff --git a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h
index 24a9a8330d19..1a9dc0c3b7db 100644
--- a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h
+++ b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h
@@ -69,5 +69,7 @@ int __pkvm_teardown_vm(pkvm_handle_t handle);
struct pkvm_hyp_vcpu *pkvm_load_hyp_vcpu(pkvm_handle_t handle,
unsigned int vcpu_idx);
void pkvm_put_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu);
+struct kvm *pkvm_vcpu_get_kvm(struct kvm_vcpu *vcpu);
+void pkvm_vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm);
#endif /* __ARM64_KVM_NVHE_PKVM_H__ */
diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 7c464340bcd0..ba7d6280efa4 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -262,6 +262,48 @@ void pkvm_put_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
hyp_spin_unlock(&vm_table_lock);
}
+static struct pkvm_hyp_vm *pkvm_get_loaded_hyp_vm(struct kvm_vcpu *vcpu)
+{
+ struct kvm_vcpu *running_vcpu = host_data_ptr(host_ctxt)->__hyp_running_vcpu;
+ struct pkvm_hyp_vcpu *hyp_vcpu;
+
+ if (!running_vcpu)
+ return NULL;
+
+ hyp_vcpu = container_of(running_vcpu, struct pkvm_hyp_vcpu, vcpu);
+
+ if (vcpu == &hyp_vcpu->vcpu || vcpu == hyp_vcpu->host_vcpu)
+ return pkvm_hyp_vcpu_to_hyp_vm(hyp_vcpu);
+
+ return NULL;
+}
+
+/*
+ * A loaded vCPU's VM is the hyp VM. An unloaded host vCPU's is mapped at
+ * EL2 only while pinned, so it's read once and pinned; the pin fails for
+ * memory the host isn't sharing.
+ */
+struct kvm *pkvm_vcpu_get_kvm(struct kvm_vcpu *vcpu)
+{
+ struct pkvm_hyp_vm *hyp_vm = pkvm_get_loaded_hyp_vm(vcpu);
+ struct kvm *kvm;
+
+ if (hyp_vm)
+ return &hyp_vm->kvm;
+
+ kvm = kern_hyp_va(READ_ONCE(vcpu->kvm));
+ if (hyp_pin_shared_mem(kvm, kvm + 1))
+ return NULL;
+
+ return kvm;
+}
+
+void pkvm_vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm)
+{
+ if (kvm && !pkvm_get_loaded_hyp_vm(vcpu))
+ hyp_unpin_shared_mem(kvm, kvm + 1);
+}
+
static void unpin_host_vcpu(struct kvm_vcpu *host_vcpu)
{
if (host_vcpu)
diff --git a/arch/arm64/kvm/hyp/nvhe/sys_regs.c b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
index 2860548d4250..039badc77bf1 100644
--- a/arch/arm64/kvm/hyp/nvhe/sys_regs.c
+++ b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
@@ -191,6 +191,15 @@ static u64 get_pvm_id_aa64mmfr0(const struct kvm_vcpu *vcpu)
set_mask = get_restricted_features_unsigned(id_aa64mmfr0_el1_sys_val,
PVM_ID_AA64MMFR0_RESTRICT_UNSIGNED);
+ set_mask |= SYS_FIELD_PREP(ID_AA64MMFR0_EL1, TGRAN4,
+ min_t(int, cpuid_feature_extract_signed_field(id_aa64mmfr0_el1_sys_val,
+ ID_AA64MMFR0_EL1_TGRAN4_SHIFT),
+ ID_AA64MMFR0_EL1_TGRAN4_IMP));
+ set_mask |= SYS_FIELD_PREP(ID_AA64MMFR0_EL1, TGRAN64,
+ min_t(int, cpuid_feature_extract_signed_field(id_aa64mmfr0_el1_sys_val,
+ ID_AA64MMFR0_EL1_TGRAN64_SHIFT),
+ ID_AA64MMFR0_EL1_TGRAN64_IMP));
+
return (id_aa64mmfr0_el1_sys_val & PVM_ID_AA64MMFR0_ALLOW) | set_mask;
}
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 496abfd3646b..b364d9319cea 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -195,7 +195,8 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)
offset = read_sysreg_s(SYS_CNTPOFF_EL2);
if (map.direct_ptimer && offset) {
- write_sysreg_el0(val + offset, SYS_CNTP_CVAL);
+ val = timer_apply_offset(val, offset, arch_timer_read_cntpct_el0());
+ write_sysreg_el0(val, SYS_CNTP_CVAL);
isb();
}
}
diff --git a/arch/arm64/kvm/ptdump.c b/arch/arm64/kvm/ptdump.c
index e4a342e903e2..ce9a64715bcd 100644
--- a/arch/arm64/kvm/ptdump.c
+++ b/arch/arm64/kvm/ptdump.c
@@ -97,22 +97,35 @@ static struct kvm_ptdump_guest_state *kvm_ptdump_parser_create(struct kvm *kvm)
{
struct kvm_ptdump_guest_state *st;
struct kvm_s2_mmu *mmu = &kvm->arch.mmu;
- struct kvm_pgtable *pgtable = mmu->pgt;
+ s8 start_level;
+ u32 ia_bits;
int ret;
+ /*
+ * We only need the pgt start level and IPA size to initialize the ptdump,
+ * get them while holding the mmu_lock. It's fine if the pgt gets freed
+ * afterwards, we'll check again when doing the actual dump.
+ */
+ scoped_guard(read_lock, &kvm->mmu_lock) {
+ if (!mmu->pgt)
+ return ERR_PTR(-ENOENT);
+ start_level = mmu->pgt->start_level;
+ ia_bits = mmu->pgt->ia_bits;
+ }
+
st = kzalloc(sizeof(struct kvm_ptdump_guest_state), GFP_KERNEL_ACCOUNT);
if (!st)
return ERR_PTR(-ENOMEM);
- ret = kvm_ptdump_build_levels(&st->level[0], pgtable->start_level);
+ ret = kvm_ptdump_build_levels(&st->level[0], start_level);
if (ret) {
kfree(st);
return ERR_PTR(ret);
}
st->ipa_marker[0].name = "Guest IPA";
- st->ipa_marker[1].start_address = BIT(pgtable->ia_bits);
- st->range[0].end = BIT(pgtable->ia_bits);
+ st->ipa_marker[1].start_address = BIT(ia_bits);
+ st->range[0].end = BIT(ia_bits);
st->kvm = kvm;
st->parser_state = (struct ptdump_pg_state) {
@@ -141,9 +154,11 @@ static int kvm_ptdump_guest_show(struct seq_file *m, void *unused)
parser_state->seq = m;
- write_lock(&kvm->mmu_lock);
+ guard(write_lock)(&kvm->mmu_lock);
+ if (!mmu->pgt)
+ return 0;
+
ret = kvm_pgtable_walk(mmu->pgt, 0, BIT(mmu->pgt->ia_bits), &walker);
- write_unlock(&kvm->mmu_lock);
return ret;
}
@@ -193,17 +208,27 @@ static const struct file_operations kvm_ptdump_guest_fops = {
static int kvm_pgtable_range_show(struct seq_file *m, void *unused)
{
- struct kvm_pgtable *pgtable = m->private;
+ struct kvm_s2_mmu *mmu = m->private;
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+
+ guard(read_lock)(&kvm->mmu_lock);
+
+ if (mmu->pgt)
+ seq_printf(m, "%2u\n", mmu->pgt->ia_bits);
- seq_printf(m, "%2u\n", pgtable->ia_bits);
return 0;
}
static int kvm_pgtable_levels_show(struct seq_file *m, void *unused)
{
- struct kvm_pgtable *pgtable = m->private;
+ struct kvm_s2_mmu *mmu = m->private;
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+
+ guard(read_lock)(&kvm->mmu_lock);
+
+ if (mmu->pgt)
+ seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - mmu->pgt->start_level);
- seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - pgtable->start_level);
return 0;
}
@@ -211,15 +236,12 @@ static int kvm_pgtable_debugfs_open(struct inode *m, struct file *file,
int (*show)(struct seq_file *, void *))
{
struct kvm *kvm = m->i_private;
- struct kvm_pgtable *pgtable;
int ret;
if (!kvm_get_kvm_safe(kvm))
return -ENOENT;
- pgtable = kvm->arch.mmu.pgt;
-
- ret = single_open(file, show, pgtable);
+ ret = single_open(file, show, &kvm->arch.mmu);
if (ret < 0)
kvm_put_kvm(kvm);
return ret;
diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index 6e157b0b21ad..c62666adc2f8 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -276,9 +276,19 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu,
if (ctlr != GICR_CTLR_ENABLE_LPIS)
return;
+ /*
+ * Yes, disabling LPIs is painful, since it can be done from
+ * a *remote* vcpu! So let's not take any chance, and make
+ * sure that the target vCPU has written their LRs back to the
+ * irq structures, and release any reference they would have.
+ *
+ * If it hurts, don't do it.
+ */
+ kvm_arm_halt_guest(vcpu->kvm);
vgic_flush_pending_lpis(vcpu);
vgic_its_invalidate_all_caches(vcpu->kvm);
atomic_set_release(&vgic_cpu->ctlr, 0);
+ kvm_arm_resume_guest(vcpu->kvm);
} else {
ctlr = atomic_cmpxchg_acquire(&vgic_cpu->ctlr, 0,
GICR_CTLR_ENABLE_LPIS);
diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c
index 7e3ad97e27d8..61da56ed0b5a 100644
--- a/arch/arm64/mm/mmap.c
+++ b/arch/arm64/mm/mmap.c
@@ -59,7 +59,9 @@ int valid_phys_addr_range(phys_addr_t addr, size_t size)
*/
int valid_mmap_phys_addr_range(unsigned long pfn, size_t size)
{
- return !(((pfn << PAGE_SHIFT) + size) & ~PHYS_MASK);
+ const phys_addr_t limit = PHYS_MASK + 1;
+
+ return size <= limit && pfn <= (limit - size) >> PAGE_SHIFT;
}
static int __init adjust_protection_map(void)
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 7afbece885a3..c7b7fa6491c3 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -103,6 +103,10 @@ SYM_FUNC_START(cpu_do_suspend)
alternative_if ARM64_HAS_TCR2
mrs x2, REG_TCR2_EL1
str x2, [x0, #104]
+alternative_else_nop_endif
+alternative_if ARM64_SME
+ mrs_s x2, SYS_TPIDR2_EL0
+ str x2, [x0, #112]
alternative_else_nop_endif
ret
SYM_FUNC_END(cpu_do_suspend)
@@ -142,6 +146,10 @@ alternative_if ARM64_HAS_TCR2
ldr x2, [x0, #104]
msr REG_TCR2_EL1, x2
alternative_else_nop_endif
+alternative_if ARM64_SME
+ ldr x2, [x0, #112]
+ msr_s SYS_TPIDR2_EL0, x2
+alternative_else_nop_endif
msr sctlr_el1, x12
set_this_cpu_offset x13
diff --git a/arch/loongarch/include/asm/hw_breakpoint.h b/arch/loongarch/include/asm/hw_breakpoint.h
index d202052df8a1..f4478936787d 100644
--- a/arch/loongarch/include/asm/hw_breakpoint.h
+++ b/arch/loongarch/include/asm/hw_breakpoint.h
@@ -121,12 +121,12 @@ bool watchpoint_handler(struct pt_regs *regs);
#ifdef CONFIG_HAVE_HW_BREAKPOINT
extern void ptrace_hw_copy_thread(struct task_struct *task);
-extern void hw_breakpoint_thread_switch(struct task_struct *next);
+extern void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next);
#else
static inline void ptrace_hw_copy_thread(struct task_struct *task)
{
}
-static inline void hw_breakpoint_thread_switch(struct task_struct *next)
+static inline void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next)
{
}
#endif
diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 8e6366a21fdd..f42080a20861 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -1069,6 +1069,9 @@
#define CSR_FWPS_SKIP_SHIFT 16
#define CSR_FWPS_SKIP (_ULCAST_(1) << CSR_FWPS_SKIP_SHIFT)
+#define CSR_MWPS_SKIP_SHIFT 16
+#define CSR_MWPS_SKIP (_ULCAST_(1) << CSR_MWPS_SKIP_SHIFT)
+
/*
* CSR_ECFG IM
*/
diff --git a/arch/loongarch/include/asm/processor.h b/arch/loongarch/include/asm/processor.h
index c3bc44b5f5b3..81af48cbee7b 100644
--- a/arch/loongarch/include/asm/processor.h
+++ b/arch/loongarch/include/asm/processor.h
@@ -138,6 +138,7 @@ struct thread_struct {
struct loongarch_lbt lbt; /* Also conditionally copied */
/* Hardware breakpoints pinned to this task. */
+ bool hbp_break_skip, hbp_watch_skip;
struct perf_event *hbp_break[LOONGARCH_MAX_BRP];
struct perf_event *hbp_watch[LOONGARCH_MAX_WRP];
};
diff --git a/arch/loongarch/include/asm/switch_to.h b/arch/loongarch/include/asm/switch_to.h
index 5b225aff3ba2..3b662b18e19a 100644
--- a/arch/loongarch/include/asm/switch_to.h
+++ b/arch/loongarch/include/asm/switch_to.h
@@ -36,7 +36,7 @@ extern asmlinkage struct task_struct *__switch_to(struct task_struct *prev,
do { \
lose_fpu_inatomic(1, prev); \
lose_lbt_inatomic(1, prev); \
- hw_breakpoint_thread_switch(next); \
+ hw_breakpoint_thread_switch(prev, next); \
(last) = __switch_to(prev, next, task_thread_info(next), \
__builtin_return_address(0), __builtin_frame_address(0)); \
} while (0)
diff --git a/arch/loongarch/kernel/hw_breakpoint.c b/arch/loongarch/kernel/hw_breakpoint.c
index 1a05350159f7..df149ff06dd5 100644
--- a/arch/loongarch/kernel/hw_breakpoint.c
+++ b/arch/loongarch/kernel/hw_breakpoint.c
@@ -156,6 +156,9 @@ static int hw_breakpoint_slot_setup(struct perf_event **slots, int max_slots,
void ptrace_hw_copy_thread(struct task_struct *tsk)
{
+ tsk->thread.hbp_break_skip = 0;
+ tsk->thread.hbp_watch_skip = 0;
+
memset(tsk->thread.hbp_break, 0, sizeof(tsk->thread.hbp_break));
memset(tsk->thread.hbp_watch, 0, sizeof(tsk->thread.hbp_watch));
}
@@ -485,6 +488,7 @@ NOKPROBE_SYMBOL(update_bp_registers);
bool breakpoint_handler(struct pt_regs *regs)
{
int i;
+ unsigned int clear_mask = 0;
bool need_sigtrap = false;
struct perf_event *bp, **slots;
@@ -500,11 +504,13 @@ bool breakpoint_handler(struct pt_regs *regs)
if (bp->attr.sigtrap)
need_sigtrap = true;
- csr_write32(0x1 << i, LOONGARCH_CSR_FWPS);
- update_bp_registers(regs, 0, 0);
+ clear_mask |= (0x1 << i);
}
}
+ if (clear_mask)
+ csr_write32(clear_mask | CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
+
return need_sigtrap;
}
NOKPROBE_SYMBOL(breakpoint_handler);
@@ -512,6 +518,7 @@ NOKPROBE_SYMBOL(breakpoint_handler);
bool watchpoint_handler(struct pt_regs *regs)
{
int i;
+ unsigned int clear_mask = 0;
bool need_sigtrap = false;
struct perf_event *wp, **slots;
@@ -527,11 +534,13 @@ bool watchpoint_handler(struct pt_regs *regs)
if (wp->attr.sigtrap)
need_sigtrap = true;
- csr_write32(0x1 << i, LOONGARCH_CSR_MWPS);
- update_bp_registers(regs, 0, 1);
+ clear_mask |= (0x1 << i);
}
}
+ if (clear_mask)
+ csr_write32(clear_mask | CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS);
+
return need_sigtrap;
}
NOKPROBE_SYMBOL(watchpoint_handler);
@@ -555,7 +564,7 @@ static int __init arch_hw_breakpoint_init(void)
}
arch_initcall(arch_hw_breakpoint_init);
-void hw_breakpoint_thread_switch(struct task_struct *next)
+void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next)
{
u64 addr, mask;
struct pt_regs *regs = task_pt_regs(next);
@@ -567,6 +576,26 @@ void hw_breakpoint_thread_switch(struct task_struct *next)
csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
regs->csr_prmd |= CSR_PRMD_PWE;
} else {
+ unsigned int fwps = csr_read32(LOONGARCH_CSR_FWPS);
+ unsigned int mwps = csr_read32(LOONGARCH_CSR_MWPS);
+
+ prev->thread.hbp_break_skip = !!(fwps & CSR_FWPS_SKIP);
+ prev->thread.hbp_watch_skip = !!(mwps & CSR_MWPS_SKIP);
+
+ if (!next->thread.hbp_break_skip)
+ csr_write32(0, LOONGARCH_CSR_FWPS);
+ else {
+ csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
+ next->thread.hbp_break_skip = 0;
+ }
+
+ if (!next->thread.hbp_watch_skip)
+ csr_write32(0, LOONGARCH_CSR_MWPS);
+ else {
+ csr_write32(CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS);
+ next->thread.hbp_watch_skip = 0;
+ }
+
/* Update breakpoints */
update_bp_registers(regs, 1, 0);
/* Update watchpoints */
diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c
index 84fe7f854820..fc62ec78a840 100644
--- a/arch/loongarch/kernel/numa.c
+++ b/arch/loongarch/kernel/numa.c
@@ -318,6 +318,8 @@ static int __init fake_numa_init(void)
phys_addr_t start = memblock_start_of_DRAM();
phys_addr_t end = memblock_end_of_DRAM() - 1;
+ set_cpuid_to_node(0, 0);
+ early_numa_add_cpu(0, 0);
node_set(0, numa_nodes_parsed);
pr_info("Faking a node at [mem %pap-%pap]\n", &start, &end);
@@ -353,8 +355,10 @@ int __init init_numa_memory(void)
return -EINVAL;
for_each_node_mask(node, node_possible_map) {
- node_mem_init(node);
- node_set_online(node);
+ if (!cpumask_empty(&cpus_on_node[node])) {
+ node_mem_init(node);
+ node_set_online(node);
+ }
}
max_low_pfn = PHYS_PFN(memblock_end_of_DRAM());
diff --git a/arch/m68k/68000/entry.S b/arch/m68k/68000/entry.S
index 72e95663b62f..0d3cbededc43 100644
--- a/arch/m68k/68000/entry.S
+++ b/arch/m68k/68000/entry.S
@@ -18,6 +18,13 @@
.text
+/* get thread_info pointer into a2 */
+ .macro getthreadinfo
+ movel %sp,%d1
+ andl #-THREAD_SIZE,%d1
+ movel %d1,%a2
+ .endm
+
.globl system_call
.globl resume
.globl ret_from_exception
@@ -70,10 +77,10 @@ ENTRY(system_call)
movel %sp@(PT_OFF_ORIG_D0),%d0
- movel %sp,%d1 /* get thread_info pointer */
- andl #-THREAD_SIZE,%d1
- movel %d1,%a2
- btst #(TIF_SYSCALL_TRACE%8),%a2@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8)
+ /* Doing a trace ? */
+ getthreadinfo
+ moveb %a2@(TINFO_FLAGS+2),%d1
+ andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d1
jne do_trace
cmpl #NR_syscalls,%d0
jcc badsys
@@ -96,16 +103,15 @@ Luser_return:
/* heavy interrupt load*/
andw #ALLOWINT,%sr
- movel %sp,%d1 /* get thread_info pointer */
- andl #-THREAD_SIZE,%d1
- movel %d1,%a2
+ getthreadinfo
1:
- move %a2@(TINFO_FLAGS),%d1 /* thread_info->flags */
+ /* check if any of the flags are set */
+ moveb %a2@(TINFO_FLAGS + 3),%d1 /* thread_info->flags (low 8 bits) */
jne Lwork_to_do
RESTORE_ALL
Lwork_to_do:
- movel %a2@(TINFO_FLAGS),%d1 /* thread_info->flags */
+ /* check if reschedule needs to be called */
btst #TIF_NEED_RESCHED,%d1
jne reschedule
diff --git a/arch/m68k/coldfire/entry.S b/arch/m68k/coldfire/entry.S
index 4ea08336e2fb..d658abdc5207 100644
--- a/arch/m68k/coldfire/entry.S
+++ b/arch/m68k/coldfire/entry.S
@@ -72,7 +72,8 @@ ENTRY(system_call)
movel %d2,%a0
movel %a0@,%a1 /* save top of frame */
movel %sp,%a1@(TASK_THREAD+THREAD_ESP0)
- btst #(TIF_SYSCALL_TRACE%8),%a0@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8)
+ moveb %a0@(TINFO_FLAGS+2),%d2
+ andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d2
bnes 1f
movel %d3,%a0
diff --git a/arch/microblaze/include/asm/entry.h b/arch/microblaze/include/asm/entry.h
index 6c42bed41166..74c5a99a37f8 100644
--- a/arch/microblaze/include/asm/entry.h
+++ b/arch/microblaze/include/asm/entry.h
@@ -21,6 +21,19 @@
#define PER_CPU(var) var
+/*
+ * The MicroBlaze ABI has the caller reserve an argument home area:
+ * REG_PARM_STACK_SPACE is 24 and OUTGOING_REG_PARM_STACK_SPACE is 1, so a
+ * callee may write to [caller_sp + 4, caller_sp + 28). The kernel calls C
+ * with r1 at the frame base, so reserve that area below pt_regs and reach
+ * the saved registers through PTO. This restores what 6e83557c38b4
+ * ("microblaze: Remove r0_ram pointer and PTO alignment") removed; the old
+ * value of 24 was one word short and left the last argument slot
+ * overlapping pt_regs' r0.
+ */
+#define PTO 28
+#define STATE_SAVE_SIZE (PT_SIZE + PTO)
+
# ifndef __ASSEMBLY__
DECLARE_PER_CPU(unsigned int, KSP); /* Saved kernel stack pointer */
DECLARE_PER_CPU(unsigned int, KM); /* Kernel/user mode */
diff --git a/arch/microblaze/include/asm/processor.h b/arch/microblaze/include/asm/processor.h
index 4e193c7550df..fe82fdb49f27 100644
--- a/arch/microblaze/include/asm/processor.h
+++ b/arch/microblaze/include/asm/processor.h
@@ -73,7 +73,7 @@ unsigned long __get_wchan(struct task_struct *p);
# define task_regs(task) ((struct pt_regs *)task_tos(task) - 1)
# define task_pt_regs_plus_args(tsk) \
- ((void *)task_pt_regs(tsk))
+ (((void *)task_pt_regs(tsk)) - PTO)
# define task_sp(task) (task_regs(task)->r1)
# define task_pc(task) (task_regs(task)->pc)
diff --git a/arch/microblaze/kernel/entry.S b/arch/microblaze/kernel/entry.S
index efa4ab056fb5..ebc8bcb98c52 100644
--- a/arch/microblaze/kernel/entry.S
+++ b/arch/microblaze/kernel/entry.S
@@ -177,78 +177,78 @@ syscall_debug_table:
1:
#define SAVE_REGS \
- swi r2, r1, PT_R2; /* Save SDA */ \
- swi r3, r1, PT_R3; \
- swi r4, r1, PT_R4; \
- swi r5, r1, PT_R5; \
- swi r6, r1, PT_R6; \
- swi r7, r1, PT_R7; \
- swi r8, r1, PT_R8; \
- swi r9, r1, PT_R9; \
- swi r10, r1, PT_R10; \
- swi r11, r1, PT_R11; /* save clobbered regs after rval */\
- swi r12, r1, PT_R12; \
- swi r13, r1, PT_R13; /* Save SDA2 */ \
- swi r14, r1, PT_PC; /* PC, before IRQ/trap */ \
- swi r15, r1, PT_R15; /* Save LP */ \
- swi r16, r1, PT_R16; \
- swi r17, r1, PT_R17; \
- swi r18, r1, PT_R18; /* Save asm scratch reg */ \
- swi r19, r1, PT_R19; \
- swi r20, r1, PT_R20; \
- swi r21, r1, PT_R21; \
- swi r22, r1, PT_R22; \
- swi r23, r1, PT_R23; \
- swi r24, r1, PT_R24; \
- swi r25, r1, PT_R25; \
- swi r26, r1, PT_R26; \
- swi r27, r1, PT_R27; \
- swi r28, r1, PT_R28; \
- swi r29, r1, PT_R29; \
- swi r30, r1, PT_R30; \
- swi r31, r1, PT_R31; /* Save current task reg */ \
+ swi r2, r1, PTO+PT_R2; /* Save SDA */ \
+ swi r3, r1, PTO+PT_R3; \
+ swi r4, r1, PTO+PT_R4; \
+ swi r5, r1, PTO+PT_R5; \
+ swi r6, r1, PTO+PT_R6; \
+ swi r7, r1, PTO+PT_R7; \
+ swi r8, r1, PTO+PT_R8; \
+ swi r9, r1, PTO+PT_R9; \
+ swi r10, r1, PTO+PT_R10; \
+ swi r11, r1, PTO+PT_R11; /* save clobbered regs after rval */\
+ swi r12, r1, PTO+PT_R12; \
+ swi r13, r1, PTO+PT_R13; /* Save SDA2 */ \
+ swi r14, r1, PTO+PT_PC; /* PC, before IRQ/trap */ \
+ swi r15, r1, PTO+PT_R15; /* Save LP */ \
+ swi r16, r1, PTO+PT_R16; \
+ swi r17, r1, PTO+PT_R17; \
+ swi r18, r1, PTO+PT_R18; /* Save asm scratch reg */ \
+ swi r19, r1, PTO+PT_R19; \
+ swi r20, r1, PTO+PT_R20; \
+ swi r21, r1, PTO+PT_R21; \
+ swi r22, r1, PTO+PT_R22; \
+ swi r23, r1, PTO+PT_R23; \
+ swi r24, r1, PTO+PT_R24; \
+ swi r25, r1, PTO+PT_R25; \
+ swi r26, r1, PTO+PT_R26; \
+ swi r27, r1, PTO+PT_R27; \
+ swi r28, r1, PTO+PT_R28; \
+ swi r29, r1, PTO+PT_R29; \
+ swi r30, r1, PTO+PT_R30; \
+ swi r31, r1, PTO+PT_R31; /* Save current task reg */ \
mfs r11, rmsr; /* save MSR */ \
- swi r11, r1, PT_MSR;
+ swi r11, r1, PTO+PT_MSR;
#define RESTORE_REGS_GP \
- lwi r2, r1, PT_R2; /* restore SDA */ \
- lwi r3, r1, PT_R3; \
- lwi r4, r1, PT_R4; \
- lwi r5, r1, PT_R5; \
- lwi r6, r1, PT_R6; \
- lwi r7, r1, PT_R7; \
- lwi r8, r1, PT_R8; \
- lwi r9, r1, PT_R9; \
- lwi r10, r1, PT_R10; \
- lwi r11, r1, PT_R11; /* restore clobbered regs after rval */\
- lwi r12, r1, PT_R12; \
- lwi r13, r1, PT_R13; /* restore SDA2 */ \
- lwi r14, r1, PT_PC; /* RESTORE_LINK PC, before IRQ/trap */\
- lwi r15, r1, PT_R15; /* restore LP */ \
- lwi r16, r1, PT_R16; \
- lwi r17, r1, PT_R17; \
- lwi r18, r1, PT_R18; /* restore asm scratch reg */ \
- lwi r19, r1, PT_R19; \
- lwi r20, r1, PT_R20; \
- lwi r21, r1, PT_R21; \
- lwi r22, r1, PT_R22; \
- lwi r23, r1, PT_R23; \
- lwi r24, r1, PT_R24; \
- lwi r25, r1, PT_R25; \
- lwi r26, r1, PT_R26; \
- lwi r27, r1, PT_R27; \
- lwi r28, r1, PT_R28; \
- lwi r29, r1, PT_R29; \
- lwi r30, r1, PT_R30; \
- lwi r31, r1, PT_R31; /* Restore cur task reg */
+ lwi r2, r1, PTO+PT_R2; /* restore SDA */ \
+ lwi r3, r1, PTO+PT_R3; \
+ lwi r4, r1, PTO+PT_R4; \
+ lwi r5, r1, PTO+PT_R5; \
+ lwi r6, r1, PTO+PT_R6; \
+ lwi r7, r1, PTO+PT_R7; \
+ lwi r8, r1, PTO+PT_R8; \
+ lwi r9, r1, PTO+PT_R9; \
+ lwi r10, r1, PTO+PT_R10; \
+ lwi r11, r1, PTO+PT_R11; /* restore clobbered regs after rval */\
+ lwi r12, r1, PTO+PT_R12; \
+ lwi r13, r1, PTO+PT_R13; /* restore SDA2 */ \
+ lwi r14, r1, PTO+PT_PC; /* RESTORE_LINK PC, before IRQ/trap */\
+ lwi r15, r1, PTO+PT_R15; /* restore LP */ \
+ lwi r16, r1, PTO+PT_R16; \
+ lwi r17, r1, PTO+PT_R17; \
+ lwi r18, r1, PTO+PT_R18; /* restore asm scratch reg */ \
+ lwi r19, r1, PTO+PT_R19; \
+ lwi r20, r1, PTO+PT_R20; \
+ lwi r21, r1, PTO+PT_R21; \
+ lwi r22, r1, PTO+PT_R22; \
+ lwi r23, r1, PTO+PT_R23; \
+ lwi r24, r1, PTO+PT_R24; \
+ lwi r25, r1, PTO+PT_R25; \
+ lwi r26, r1, PTO+PT_R26; \
+ lwi r27, r1, PTO+PT_R27; \
+ lwi r28, r1, PTO+PT_R28; \
+ lwi r29, r1, PTO+PT_R29; \
+ lwi r30, r1, PTO+PT_R30; \
+ lwi r31, r1, PTO+PT_R31; /* Restore cur task reg */
#define RESTORE_REGS \
- lwi r11, r1, PT_MSR; \
+ lwi r11, r1, PTO+PT_MSR; \
mts rmsr , r11; \
RESTORE_REGS_GP
#define RESTORE_REGS_RTBD \
- lwi r11, r1, PT_MSR; \
+ lwi r11, r1, PTO+PT_MSR; \
andni r11, r11, MSR_EIP; /* clear EIP */ \
ori r11, r11, MSR_EE | MSR_BIP; /* set EE and BIP */ \
mts rmsr , r11; \
@@ -265,11 +265,11 @@ syscall_debug_table:
lwi r1, r0, TOPHYS(PER_CPU(ENTRY_SP)); \
/* FIXME: I can add these two lines to one */ \
/* tophys(r1,r1); */ \
- /* addik r1, r1, -PT_SIZE; */ \
- addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE; \
+ /* addik r1, r1, -STATE_SAVE_SIZE; */ \
+ addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - STATE_SAVE_SIZE; \
SAVE_REGS \
brid 2f; \
- swi r1, r1, PT_MODE; \
+ swi r1, r1, PTO+PT_MODE; \
1: /* User-mode state save. */ \
lwi r1, r0, TOPHYS(PER_CPU(CURRENT_SAVE)); /* get saved current */\
tophys(r1,r1); \
@@ -277,12 +277,13 @@ syscall_debug_table:
/* MS these three instructions can be added to one */ \
/* addik r1, r1, THREAD_SIZE; */ \
/* tophys(r1,r1); */ \
- /* addik r1, r1, -PT_SIZE; */ \
- addik r1, r1, THREAD_SIZE + CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE; \
+ /* addik r1, r1, -STATE_SAVE_SIZE; */ \
+ addik r1, r1, THREAD_SIZE + CONFIG_KERNEL_BASE_ADDR \
+ - CONFIG_KERNEL_START - STATE_SAVE_SIZE; \
SAVE_REGS \
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP)); \
- swi r11, r1, PT_R1; /* Store user SP. */ \
- swi r0, r1, PT_MODE; /* Was in user-mode. */ \
+ swi r11, r1, PTO+PT_R1; /* Store user SP. */ \
+ swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */ \
/* MS: I am clearing UMS even in case when I come from kernel space */ \
clear_ums; \
2: lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
@@ -292,7 +293,7 @@ syscall_debug_table:
.extern cpuinfo
C_ENTRY(mb_flush_dcache):
- addik r1, r1, -PT_SIZE
+ addik r1, r1, -STATE_SAVE_SIZE
SAVE_REGS
addik r3, r0, cpuinfo
@@ -305,12 +306,12 @@ C_ENTRY(mb_flush_dcache):
addk r9, r9, r8
RESTORE_REGS
- addik r1, r1, PT_SIZE
+ addik r1, r1, STATE_SAVE_SIZE
rtsd r15, 8
nop
C_ENTRY(mb_invalidate_icache):
- addik r1, r1, -PT_SIZE
+ addik r1, r1, -STATE_SAVE_SIZE
SAVE_REGS
addik r3, r0, cpuinfo
@@ -323,7 +324,7 @@ C_ENTRY(mb_invalidate_icache):
addk r9, r9, r8
RESTORE_REGS
- addik r1, r1, PT_SIZE
+ addik r1, r1, STATE_SAVE_SIZE
rtsd r15, 8
nop
@@ -350,18 +351,18 @@ C_ENTRY(_user_exception):
addik r1, r1, THREAD_SIZE;
tophys(r1,r1);
- addik r1, r1, -PT_SIZE; /* Make room on the stack. */
+ addik r1, r1, -STATE_SAVE_SIZE; /* Make room on the stack. */
SAVE_REGS
- swi r0, r1, PT_R3
- swi r0, r1, PT_R4
+ swi r0, r1, PTO+PT_R3
+ swi r0, r1, PTO+PT_R4
- swi r0, r1, PT_MODE; /* Was in user-mode. */
+ swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
- swi r11, r1, PT_R1; /* Store user SP. */
+ swi r11, r1, PTO+PT_R1; /* Store user SP. */
clear_ums;
2: lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
/* Save away the syscall number. */
- swi r12, r1, PT_R0;
+ swi r12, r1, PTO+PT_R0;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8*/
@@ -380,18 +381,18 @@ C_ENTRY(_user_exception):
beqi r11, 4f
addik r3, r0, -ENOSYS
- swi r3, r1, PT_R3
+ swi r3, r1, PTO+PT_R3
brlid r15, do_syscall_trace_enter
- addik r5, r1, PT_R0
+ addik r5, r1, PTO+PT_R0
# do_syscall_trace_enter returns the new syscall nr.
addk r12, r0, r3
- lwi r5, r1, PT_R5;
- lwi r6, r1, PT_R6;
- lwi r7, r1, PT_R7;
- lwi r8, r1, PT_R8;
- lwi r9, r1, PT_R9;
- lwi r10, r1, PT_R10;
+ lwi r5, r1, PTO+PT_R5;
+ lwi r6, r1, PTO+PT_R6;
+ lwi r7, r1, PTO+PT_R7;
+ lwi r8, r1, PTO+PT_R8;
+ lwi r9, r1, PTO+PT_R9;
+ lwi r10, r1, PTO+PT_R10;
4:
/* Jump to the appropriate function for the system call number in r12
* (r12 is not preserved), or return an error if r12 is not valid.
@@ -432,8 +433,8 @@ C_ENTRY(_user_exception):
/* Entry point used to return from a syscall/trap */
/* We re-enable BIP bit before state restore */
C_ENTRY(ret_from_trap):
- swi r3, r1, PT_R3
- swi r4, r1, PT_R4
+ swi r3, r1, PTO+PT_R3
+ swi r4, r1, PTO+PT_R4
/*
* Entry point for returns that must not store r3/r4 back into pt_regs,
* i.e. rt_sigreturn, which has already restored them from the signal
@@ -441,7 +442,7 @@ C_ENTRY(ret_from_trap):
* so it stays correct if the number of stores above ever changes.
*/
ret_from_trap_no_rval:
- lwi r11, r1, PT_MODE;
+ lwi r11, r1, PTO+PT_MODE;
/* See if returning to kernel mode, if so, skip resched &c. */
bnei r11, 2f;
/* We're returning to user mode, so check for various conditions that
@@ -453,7 +454,7 @@ ret_from_trap_no_rval:
beqi r11, 1f
brlid r15, do_syscall_trace_leave
- addik r5, r1, PT_R0
+ addik r5, r1, PTO+PT_R0
1:
/* We're returning to user mode, so check for various conditions that
* trigger rescheduling. */
@@ -472,7 +473,7 @@ ret_from_trap_no_rval:
andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME;
beqi r11, 4f; /* Signals to handle, handle them */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
add r6, r30, r0; /* Arg 2: int in_syscall */
add r30, r0, r0 /* no more restarts */
@@ -484,7 +485,11 @@ ret_from_trap_no_rval:
VM_OFF;
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
+ /*
+ * Deliberately PT_SIZE, not STATE_SAVE_SIZE: r1 is back at the stack
+ * top, so PTO cancels and the saved SP sits at PT_R1 - PT_SIZE.
+ */
lwi r1, r1, PT_R1 - PT_SIZE;/* Restore user stack pointer. */
bri 6f;
@@ -493,7 +498,7 @@ ret_from_trap_no_rval:
VM_OFF;
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
tovirt(r1,r1);
6:
TRAP_return: /* Make global symbol for debugging */
@@ -534,7 +539,7 @@ C_ENTRY(sys_rt_sigreturn_wrapper):
*/
addik r15, r0, ret_from_trap_no_rval - 8
brid sys_rt_sigreturn /* Do real work */
- addik r5, r1, 0; /* add user context as 1st arg */
+ addik r5, r1, PTO; /* add user context as 1st arg */
/*
* HW EXCEPTION rutine start
@@ -545,7 +550,7 @@ C_ENTRY(full_exception_trap):
addik r17, r17, -4
SAVE_STATE /* Save registers */
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* FIXME this can be store directly in PT_ESR reg.
* I tested it but there is a fault */
@@ -555,7 +560,7 @@ C_ENTRY(full_exception_trap):
mfs r7, rfsr; /* save FSR */
mts rfsr, r0; /* Clear sticky fsr */
rted r0, full_exception
- addik r5, r1, 0 /* parameter struct pt_regs * regs */
+ addik r5, r1, PTO /* parameter struct pt_regs * regs */
/*
* Unaligned data trap.
@@ -581,14 +586,14 @@ C_ENTRY(unaligned_data_trap):
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
SAVE_STATE /* Save registers.*/
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8 */
addik r15, r0, ret_from_exc-8
mfs r3, resr /* ESR */
mfs r4, rear /* EAR */
rtbd r0, _unaligned_data_exception
- addik r7, r1, 0 /* parameter struct pt_regs * regs */
+ addik r7, r1, PTO /* parameter struct pt_regs * regs */
/*
* Page fault traps.
@@ -611,30 +616,30 @@ C_ENTRY(unaligned_data_trap):
C_ENTRY(page_fault_data_trap):
SAVE_STATE /* Save registers.*/
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8 */
addik r15, r0, ret_from_exc-8
mfs r6, rear /* parameter unsigned long address */
mfs r7, resr /* parameter unsigned long error_code */
rted r0, do_page_fault
- addik r5, r1, 0 /* parameter struct pt_regs * regs */
+ addik r5, r1, PTO /* parameter struct pt_regs * regs */
C_ENTRY(page_fault_instr_trap):
SAVE_STATE /* Save registers.*/
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8 */
addik r15, r0, ret_from_exc-8
mfs r6, rear /* parameter unsigned long address */
ori r7, r0, 0 /* parameter unsigned long error_code */
rted r0, do_page_fault
- addik r5, r1, 0 /* parameter struct pt_regs * regs */
+ addik r5, r1, PTO /* parameter struct pt_regs * regs */
/* Entry point used to return from an exception. */
C_ENTRY(ret_from_exc):
- lwi r11, r1, PT_MODE;
+ lwi r11, r1, PTO+PT_MODE;
bnei r11, 2f; /* See if returning to kernel mode, */
/* ... if so, skip resched &c. */
@@ -666,7 +671,7 @@ C_ENTRY(ret_from_exc):
* complete register state. Here we save anything not saved by
* the normal entry sequence, so that it may be safely restored
* (in a possibly modified form) after do_notify_resume returns. */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
addi r6, r0, 0; /* Arg 2: int in_syscall */
bri 1b
@@ -678,7 +683,7 @@ C_ENTRY(ret_from_exc):
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
lwi r1, r1, PT_R1 - PT_SIZE; /* Restore user stack pointer. */
bri 6f;
@@ -687,7 +692,7 @@ C_ENTRY(ret_from_exc):
VM_OFF;
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
tovirt(r1,r1);
6:
@@ -720,10 +725,10 @@ C_ENTRY(_interrupt):
tophys(r1,r1); /* MS: I have in r1 physical address where stack is */
/* save registers */
/* MS: Make room on the stack -> activation record */
- addik r1, r1, -PT_SIZE;
+ addik r1, r1, -STATE_SAVE_SIZE;
SAVE_REGS
brid 2f;
- swi r1, r1, PT_MODE; /* 0 - user mode, 1 - kernel mode */
+ swi r1, r1, PTO+PT_MODE; /* 0 - user mode, 1 - kernel mode */
1:
/* User-mode state save. */
/* MS: get the saved current */
@@ -733,23 +738,23 @@ C_ENTRY(_interrupt):
addik r1, r1, THREAD_SIZE;
tophys(r1,r1);
/* save registers */
- addik r1, r1, -PT_SIZE;
+ addik r1, r1, -STATE_SAVE_SIZE;
SAVE_REGS
/* calculate mode */
- swi r0, r1, PT_MODE;
+ swi r0, r1, PTO+PT_MODE;
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
- swi r11, r1, PT_R1;
+ swi r11, r1, PTO+PT_R1;
clear_ums;
2:
lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
tovirt(r1,r1)
addik r15, r0, irq_call;
irq_call:rtbd r0, do_IRQ;
- addik r5, r1, 0;
+ addik r5, r1, PTO;
/* MS: we are in virtual mode */
ret_from_irq:
- lwi r11, r1, PT_MODE;
+ lwi r11, r1, PTO+PT_MODE;
bnei r11, 2f;
1:
@@ -765,7 +770,7 @@ ret_from_irq:
5: andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME;
beqid r11, no_intr_resched
/* Handle a signal return; Pending signals should be in r18. */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
addi r6, r0, 0; /* Arg 2: int in_syscall */
bri 1b
@@ -778,7 +783,7 @@ no_intr_resched:
VM_OFF;
tophys(r1,r1);
RESTORE_REGS
- addik r1, r1, PT_SIZE /* MS: Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* MS: Clean up stack space. */
lwi r1, r1, PT_R1 - PT_SIZE;
bri 6f;
/* MS: Return to kernel state. */
@@ -801,7 +806,7 @@ restore:
VM_OFF /* MS: turn off MMU */
tophys(r1,r1)
RESTORE_REGS
- addik r1, r1, PT_SIZE /* MS: Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* MS: Clean up stack space. */
tovirt(r1,r1);
6:
IRQ_return: /* MS: Make global symbol for debugging */
@@ -820,29 +825,29 @@ C_ENTRY(_xtmr_manager_reset):
lwi r1, r0, xmb_manager_stackpointer
/* Restore MSR */
- lwi r2, r1, PT_MSR
+ lwi r2, r1, PTO+PT_MSR
mts rmsr, r2
bri 4
/* restore Special purpose registers */
- lwi r2, r1, PT_PID
+ lwi r2, r1, PTO+PT_PID
mts rpid, r2
- lwi r2, r1, PT_TLBI
+ lwi r2, r1, PTO+PT_TLBI
mts rtlbx, r2
- lwi r2, r1, PT_ZPR
+ lwi r2, r1, PTO+PT_ZPR
mts rzpr, r2
#if CONFIG_XILINX_MICROBLAZE0_USE_FPU
- lwi r2, r1, PT_FSR
+ lwi r2, r1, PTO+PT_FSR
mts rfsr, r2
#endif
/* restore all the tlb's */
addik r3, r0, TOPHYS(tlb_skip)
- addik r6, r0, PT_TLBL0
- addik r7, r0, PT_TLBH0
+ addik r6, r0, PTO+PT_TLBL0
+ addik r7, r0, PTO+PT_TLBH0
restore_tlb:
add r6, r6, r1
add r7, r7, r1
@@ -868,9 +873,9 @@ ret_from_reset:
VM_OFF
/* MS: Restore all regs */
RESTORE_REGS
- lwi r14, r1, PT_R14
- lwi r16, r1, PT_PC
- addik r1, r1, PT_SIZE + 36
+ lwi r14, r1, PTO+PT_R14
+ lwi r16, r1, PTO+PT_PC
+ addik r1, r1, STATE_SAVE_SIZE + 36
rtbd r16, 0
nop
@@ -885,11 +890,11 @@ C_ENTRY(_xmb_manager_break):
* Reserve memory in the stack for context store/restore
* (which includes memory for storing tlbs (max two tlbs))
*/
- addik r1, r1, -PT_SIZE - 36
+ addik r1, r1, -STATE_SAVE_SIZE - 36
swi r1, r0, xmb_manager_stackpointer
SAVE_REGS
- swi r14, r1, PT_R14 /* rewrite saved R14 value */
- swi r16, r1, PT_PC; /* PC and r16 are the same */
+ swi r14, r1, PTO+PT_R14 /* rewrite saved R14 value */
+ swi r16, r1, PTO+PT_PC; /* PC and r16 are the same */
lwi r6, r0, TOPHYS(xmb_manager_baseaddr)
lwi r7, r0, TOPHYS(xmb_manager_crval)
@@ -903,25 +908,25 @@ C_ENTRY(_xmb_manager_break):
/* Save the special purpose registers */
mfs r2, rpid
- swi r2, r1, PT_PID
+ swi r2, r1, PTO+PT_PID
mfs r2, rtlbx
- swi r2, r1, PT_TLBI
+ swi r2, r1, PTO+PT_TLBI
mfs r2, rzpr
- swi r2, r1, PT_ZPR
+ swi r2, r1, PTO+PT_ZPR
#if CONFIG_XILINX_MICROBLAZE0_USE_FPU
mfs r2, rfsr
- swi r2, r1, PT_FSR
+ swi r2, r1, PTO+PT_FSR
#endif
mfs r2, rmsr
- swi r2, r1, PT_MSR
+ swi r2, r1, PTO+PT_MSR
/* Save all the tlb's */
addik r3, r0, TOPHYS(tlb_skip)
- addik r6, r0, PT_TLBL0
- addik r7, r0, PT_TLBH0
+ addik r6, r0, PTO+PT_TLBL0
+ addik r7, r0, PTO+PT_TLBH0
save_tlb:
add r6, r6, r1
add r7, r7, r1
@@ -978,28 +983,28 @@ C_ENTRY(_debug_exception):
lwi r1, r0, TOPHYS(PER_CPU(ENTRY_SP)); /* Reload kernel stack-ptr*/
/* BIP bit is set on entry, no interrupts can occur */
- addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE;
+ addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - STATE_SAVE_SIZE;
SAVE_REGS;
/* save all regs to pt_reg structure */
- swi r0, r1, PT_R0; /* R0 must be saved too */
- swi r14, r1, PT_R14 /* rewrite saved R14 value */
- swi r16, r1, PT_PC; /* PC and r16 are the same */
+ swi r0, r1, PTO+PT_R0; /* R0 must be saved too */
+ swi r14, r1, PTO+PT_R14 /* rewrite saved R14 value */
+ swi r16, r1, PTO+PT_PC; /* PC and r16 are the same */
/* save special purpose registers to pt_regs */
mfs r11, rear;
- swi r11, r1, PT_EAR;
+ swi r11, r1, PTO+PT_EAR;
mfs r11, resr;
- swi r11, r1, PT_ESR;
+ swi r11, r1, PTO+PT_ESR;
mfs r11, rfsr;
- swi r11, r1, PT_FSR;
+ swi r11, r1, PTO+PT_FSR;
/* stack pointer is in physical address at it is decrease
- * by PT_SIZE but we need to get correct R1 value */
- addik r11, r1, CONFIG_KERNEL_START - CONFIG_KERNEL_BASE_ADDR + PT_SIZE;
- swi r11, r1, PT_R1
+ * by STATE_SAVE_SIZE but we need to get correct R1 value */
+ addik r11, r1, CONFIG_KERNEL_START - CONFIG_KERNEL_BASE_ADDR + STATE_SAVE_SIZE;
+ swi r11, r1, PTO+PT_R1
/* MS: r31 - current pointer isn't changed */
tovirt(r1,r1)
#ifdef CONFIG_KGDB
- addi r5, r1, 0 /* pass pt_reg address as the first arg */
+ addi r5, r1, PTO /* pass pt_reg address as the first arg */
addik r15, r0, dbtrap_call; /* return address */
rtbd r0, microblaze_kgdb_break
nop;
@@ -1015,16 +1020,16 @@ C_ENTRY(_debug_exception):
addik r1, r1, THREAD_SIZE; /* calculate kernel stack pointer */
tophys(r1,r1);
- addik r1, r1, -PT_SIZE; /* Make room on the stack. */
+ addik r1, r1, -STATE_SAVE_SIZE; /* Make room on the stack. */
SAVE_REGS;
- swi r16, r1, PT_PC; /* Save LP */
- swi r0, r1, PT_MODE; /* Was in user-mode. */
+ swi r16, r1, PTO+PT_PC; /* Save LP */
+ swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
- swi r11, r1, PT_R1; /* Store user SP. */
+ swi r11, r1, PTO+PT_R1; /* Store user SP. */
lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
tovirt(r1,r1)
set_vms;
- addik r5, r1, 0;
+ addik r5, r1, PTO;
addik r15, r0, dbtrap_call;
dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
rtbd r0, sw_exception
@@ -1032,7 +1037,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
/* MS: The first instruction for the second part of the gdb/kgdb */
set_bip; /* Ints masked for state restore */
- lwi r11, r1, PT_MODE;
+ lwi r11, r1, PTO+PT_MODE;
bnei r11, 2f;
/* MS: Return to user space - gdb */
1:
@@ -1051,7 +1056,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
5: andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME;
beqi r11, 4f; /* Signals to handle, handle them */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
addi r6, r0, 0; /* Arg 2: int in_syscall */
bri 1b
@@ -1062,7 +1067,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
tophys(r1,r1);
/* MS: Restore all regs */
RESTORE_REGS_RTBD
- addik r1, r1, PT_SIZE /* Clean up stack space */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space */
lwi r1, r1, PT_R1 - PT_SIZE; /* Restore user stack pointer */
DBTRAP_return_user: /* MS: Make global symbol for debugging */
rtbd r16, 0; /* MS: Instructions to return from a debug trap */
@@ -1073,9 +1078,9 @@ DBTRAP_return_user: /* MS: Make global symbol for debugging */
tophys(r1,r1);
/* MS: Restore all regs */
RESTORE_REGS_RTBD
- lwi r14, r1, PT_R14;
- lwi r16, r1, PT_PC;
- addik r1, r1, PT_SIZE; /* MS: Clean up stack space */
+ lwi r14, r1, PTO+PT_R14;
+ lwi r16, r1, PTO+PT_PC;
+ addik r1, r1, STATE_SAVE_SIZE; /* MS: Clean up stack space */
tovirt(r1,r1);
DBTRAP_return_kernel: /* MS: Make global symbol for debugging */
rtbd r16, 0; /* MS: Instructions to return from a debug trap */
@@ -1173,7 +1178,7 @@ ENTRY(_switch_to)
.ent xmb_inject_err
.type xmb_inject_err, @function
xmb_inject_err:
- addik r1, r1, -PT_SIZE
+ addik r1, r1, -STATE_SAVE_SIZE
SAVE_REGS
/* Switch to real mode */
@@ -1197,7 +1202,7 @@ xmb_inject_err:
nop;
1:
RESTORE_REGS
- addik r1, r1, PT_SIZE
+ addik r1, r1, STATE_SAVE_SIZE
rtsd r15, 8;
nop;
.end xmb_inject_err
diff --git a/arch/microblaze/kernel/hw_exception_handler.S b/arch/microblaze/kernel/hw_exception_handler.S
index 07ea23965f81..4693916be526 100644
--- a/arch/microblaze/kernel/hw_exception_handler.S
+++ b/arch/microblaze/kernel/hw_exception_handler.S
@@ -313,6 +313,11 @@ _MB_HW_ExceptionVectorTable:
.align 4
.ent _hw_exception_handler
_hw_exception_handler:
+ /*
+ * This handler saves into the static pt_pool_space buffer, not into a
+ * stack frame, so the plain PT_* offsets are right here -- do not add
+ * the PTO of entry.S's frames.
+ */
swi r1, r0, TOPHYS(pt_pool_space + PT_R1); /* GET_SP */
/* Save date to kernel memory. Here is the problem
* when you came from user space */
diff --git a/arch/microblaze/kernel/process.c b/arch/microblaze/kernel/process.c
index 56342e11442d..504e73854bc2 100644
--- a/arch/microblaze/kernel/process.c
+++ b/arch/microblaze/kernel/process.c
@@ -19,6 +19,7 @@
#include <linux/bitops.h>
#include <linux/ptrace.h>
#include <asm/cacheflush.h>
+#include <asm/entry.h>
void show_regs(struct pt_regs *regs)
{
@@ -65,7 +66,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
* the registers. That's OK for a brand new thread.*/
memset(childregs, 0, sizeof(struct pt_regs));
memset(&ti->cpu_context, 0, sizeof(struct cpu_context));
- ti->cpu_context.r1 = (unsigned long)childregs;
+ ti->cpu_context.r1 = (unsigned long)childregs - PTO;
ti->cpu_context.r20 = (unsigned long)args->fn;
ti->cpu_context.r19 = (unsigned long)args->fn_arg;
childregs->pt_mode = 1;
@@ -79,7 +80,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
childregs->r1 = usp;
memset(&ti->cpu_context, 0, sizeof(struct cpu_context));
- ti->cpu_context.r1 = (unsigned long)childregs;
+ ti->cpu_context.r1 = (unsigned long)childregs - PTO;
childregs->msr |= MSR_UMS;
/* we should consider the fact that childregs is a copy of the parent
diff --git a/arch/mips/crypto/Kconfig b/arch/mips/crypto/Kconfig
index ee9604fd2037..f951a75dfef3 100644
--- a/arch/mips/crypto/Kconfig
+++ b/arch/mips/crypto/Kconfig
@@ -34,7 +34,7 @@ config CRYPTO_MD5_OCTEON
config CRYPTO_SHA1_OCTEON
tristate "Hash functions: SHA-1 (OCTEON)"
- depends on CPU_CAVIUM_OCTEON
+ depends on CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
select CRYPTO_SHA1
select CRYPTO_HASH
help
@@ -44,7 +44,7 @@ config CRYPTO_SHA1_OCTEON
config CRYPTO_SHA256_OCTEON
tristate "Hash functions: SHA-224 and SHA-256 (OCTEON)"
- depends on CPU_CAVIUM_OCTEON
+ depends on CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
select CRYPTO_SHA256
select CRYPTO_HASH
help
@@ -54,7 +54,7 @@ config CRYPTO_SHA256_OCTEON
config CRYPTO_SHA512_OCTEON
tristate "Hash functions: SHA-384 and SHA-512 (OCTEON)"
- depends on CPU_CAVIUM_OCTEON
+ depends on CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
select CRYPTO_SHA512
select CRYPTO_HASH
help
diff --git a/arch/mips/mm/cex-gen.S b/arch/mips/mm/cex-gen.S
index e528583d1331..6989fe274aab 100644
--- a/arch/mips/mm/cex-gen.S
+++ b/arch/mips/mm/cex-gen.S
@@ -8,6 +8,7 @@
*
* Cache error handler
*/
+#include <asm/addrspace.h>
#include <asm/asm.h>
#include <asm/regdef.h>
#include <asm/mipsregs.h>
@@ -37,6 +38,21 @@
nop
nop
+#if defined(CONFIG_64BIT) && VMLINUX_LOAD_ADDRESS >= XKPHYS && \
+ VMLINUX_LOAD_ADDRESS < XKSEG
+ /*
+ * An XKPHYS kernel may lie outside the 256MB region a j from the
+ * vector reaches, so jump to the handler's uncached XKPHYS address.
+ */
+ PTR_LA k0, cache_parity_error
+ PTR_LI k1, TO_PHYS_MASK
+ and k0, k0, k1
+ PTR_LI k1, UNCAC_BASE
+ or k0, k0, k1
+ jr k0
+ nop
+#else
j cache_parity_error
nop
+#endif
END(except_vec2_generic)
diff --git a/arch/mips/pci/ops-mace.c b/arch/mips/pci/ops-mace.c
index 951d8070fb48..8827e5f2ea46 100644
--- a/arch/mips/pci/ops-mace.c
+++ b/arch/mips/pci/ops-mace.c
@@ -35,15 +35,27 @@ static inline int mkaddr(struct pci_bus *bus, unsigned int devfn,
(reg & 0xfc);
}
+/*
+ * On CRIME 1.1, a MACE PCI register read followed by two register writes
+ * can hang the machine under DMA load, so every register write in this
+ * file is followed by a read. Config cycles to empty slots end in a
+ * master abort; ack it with interrupts still off so the bridge error
+ * interrupt does not report it.
+ */
+static void mace_pci_ack_abort(void)
+{
+ u32 error = mace->pci.error;
+
+ if (error & MACEPCI_ERROR_MASTER_ABORT) {
+ mace->pci.error = error & ~MACEPCI_ERROR_MASTER_ABORT;
+ mace->pci.error;
+ }
+}
static int
mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 *val)
{
- u32 control = mace->pci.control;
-
- /* disable master aborts interrupts during config read */
- mace->pci.control = control & ~MACEPCI_CONTROL_MAR_INT;
mace->pci.config_addr = mkaddr(bus, devfn, reg);
switch (size) {
case 1:
@@ -56,9 +68,7 @@ mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
*val = mace->pci.config_data.l;
break;
}
- /* ack possible master abort */
- mace->pci.error &= ~MACEPCI_ERROR_MASTER_ABORT;
- mace->pci.control = control;
+ mace_pci_ack_abort();
/*
* someone forgot to set the ultra bit for the onboard
* scsi chips; we fake it here
@@ -77,6 +87,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 val)
{
mace->pci.config_addr = mkaddr(bus, devfn, reg);
+ mace->pci.error;
switch (size) {
case 1:
mace->pci.config_data.b[(reg & 3) ^ 3] = val;
@@ -88,6 +99,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
mace->pci.config_data.l = val;
break;
}
+ mace_pci_ack_abort();
DPRINTK("write%d: reg=%08x,val=%02x\n", size * 8, reg, val);
diff --git a/arch/mips/pci/pci-ip32.c b/arch/mips/pci/pci-ip32.c
index 7ae89d0c7099..41ec8453c865 100644
--- a/arch/mips/pci/pci-ip32.c
+++ b/arch/mips/pci/pci-ip32.c
@@ -77,6 +77,8 @@ static irqreturn_t macepci_error(int irq, void *dev)
}
mace->pci.error = flags;
+ /* follow the write with a read, see mace_pci_ack_abort() */
+ mace->pci.error;
return IRQ_HANDLED;
}
@@ -125,10 +127,13 @@ static int __init mace_init(void)
{
PCIBIOS_MIN_IO = 0x1000;
- /* Clear any outstanding errors and enable interrupts */
+ /*
+ * Clear any outstanding errors and enable interrupts. PCI interrupts
+ * are masked in CRIME, so the per-slot enables stay on.
+ */
mace->pci.error_addr = 0;
mace->pci.error = 0;
- mace->pci.control = 0xff008500;
+ mace->pci.control = 0xff0085ff;
printk("MACE PCI rev %d\n", mace->pci.rev);
diff --git a/arch/mips/sgi-ip32/ip32-irq.c b/arch/mips/sgi-ip32/ip32-irq.c
index 29d04468a06b..b9c72af481ed 100644
--- a/arch/mips/sgi-ip32/ip32-irq.c
+++ b/arch/mips/sgi-ip32/ip32-irq.c
@@ -160,38 +160,6 @@ static struct irq_chip crime_edge_interrupt = {
.irq_unmask = crime_enable_irq,
};
-/*
- * This is for MACE PCI interrupts. We can decrease bus traffic by masking
- * as close to the source as possible. This also means we can take the
- * next chunk of the CRIME register in one piece.
- */
-
-static unsigned long macepci_mask;
-
-static void enable_macepci_irq(struct irq_data *d)
-{
- macepci_mask |= MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- crime_mask |= 1 << (d->irq - CRIME_IRQ_BASE);
- crime->imask = crime_mask;
-}
-
-static void disable_macepci_irq(struct irq_data *d)
-{
- crime_mask &= ~(1 << (d->irq - CRIME_IRQ_BASE));
- crime->imask = crime_mask;
- flush_crime_bus();
- macepci_mask &= ~MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- flush_mace_bus();
-}
-
-static struct irq_chip ip32_macepci_interrupt = {
- .name = "IP32 MACE PCI",
- .irq_mask = disable_macepci_irq,
- .irq_unmask = enable_macepci_irq,
-};
-
/* This is used for MACE ISA interrupts. That means bits 4-6 in the
* CRIME register.
*/
@@ -299,8 +267,8 @@ static struct irq_chip ip32_maceisa_edge_interrupt = {
.irq_unmask = enable_maceisa_irq,
};
-/* This is used for regular non-ISA, non-PCI MACE interrupts. That means
- * bits 0-3 and 7 in the CRIME register.
+/* This is used for regular non-ISA MACE interrupts. That means bits 0-3
+ * and 7-15 in the CRIME register.
*/
static void enable_mace_irq(struct irq_data *d)
@@ -436,20 +404,13 @@ void __init arch_init_irq(void)
mips_cpu_irq_init();
for (irq = CRIME_IRQ_BASE; irq <= IP32_IRQ_MAX; irq++) {
switch (irq) {
- case MACE_VID_IN1_IRQ ... MACE_PCI_BRIDGE_IRQ:
+ case MACE_VID_IN1_IRQ ... MACEPCI_SHARED2_IRQ:
irq_set_chip_and_handler_name(irq,
&ip32_mace_interrupt,
handle_level_irq,
"level");
break;
- case MACEPCI_SCSI0_IRQ ... MACEPCI_SHARED2_IRQ:
- irq_set_chip_and_handler_name(irq,
- &ip32_macepci_interrupt,
- handle_level_irq,
- "level");
- break;
-
case CRIME_CPUERR_IRQ:
case CRIME_MEMERR_IRQ:
irq_set_chip_and_handler_name(irq,
diff --git a/arch/openrisc/kernel/or32_ksyms.c b/arch/openrisc/kernel/or32_ksyms.c
index 212e5f85004c..c63d7f25bc57 100644
--- a/arch/openrisc/kernel/or32_ksyms.c
+++ b/arch/openrisc/kernel/or32_ksyms.c
@@ -35,6 +35,7 @@ DECLARE_EXPORT(__divsi3);
DECLARE_EXPORT(__umodsi3);
DECLARE_EXPORT(__modsi3);
DECLARE_EXPORT(__muldi3);
+DECLARE_EXPORT(__mulsi3);
DECLARE_EXPORT(__ashrdi3);
DECLARE_EXPORT(__ashldi3);
DECLARE_EXPORT(__lshrdi3);
diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index cdea5dccaefe..02a1ea9334d1 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -13,6 +13,13 @@
#include <asm/types.h>
#include <asm/ppc-opcode.h>
+/*
+ * We need at least 2 passes for proper code generation, and may need
+ * additional passes if code size changes between passes.
+ */
+#define CODEGEN_MIN_PASSES 2
+#define CODEGEN_MAX_PASSES 3
+
#ifdef CONFIG_PPC64_ELF_ABI_V1
#define FUNCTION_DESCR_SIZE 24
#else
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 180175e072ad..2d8cecc36b73 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -29,11 +29,10 @@ static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr)
{
- if (!exit_addr || is_offset_in_branch_range(exit_addr - (ctx->idx * 4))) {
+ if (exit_addr && is_offset_in_branch_range(exit_addr - (long)(ctx->idx * 4))) {
PPC_JMP(exit_addr);
- } else if (ctx->alt_exit_addr) {
- if (WARN_ON(!is_offset_in_branch_range((long)ctx->alt_exit_addr - (ctx->idx * 4))))
- return -1;
+ } else if (ctx->alt_exit_addr && is_offset_in_branch_range(
+ (long)(ctx->alt_exit_addr) - (long)(ctx->idx * 4))) {
PPC_JMP(ctx->alt_exit_addr);
} else {
ctx->alt_exit_addr = ctx->idx * 4;
@@ -151,6 +150,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
+ cgctx.alt_exit_addr = 0;
if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
fp = org_fp;
goto out_addrs;
@@ -187,10 +187,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
code_base = (u32 *)(image + FUNCTION_DESCR_SIZE);
fcode_base = (u32 *)(fimage + FUNCTION_DESCR_SIZE);
- /* Code generation passes 1-2 */
- for (pass = 1; pass < 3; pass++) {
+ /* Code generation passes 1-2+, loop until program size converges. */
+ for (pass = 1; pass <= CODEGEN_MAX_PASSES; pass++) {
+ u32 prev_proglen = proglen;
+
/* Now build the prologue, body code & epilogue for real. */
cgctx.idx = 0;
+ cgctx.exentry_idx = 0;
cgctx.alt_exit_addr = 0;
bpf_jit_build_prologue(code_base, &cgctx);
if (bpf_jit_build_body(fp, code_base, fcode_base, &cgctx, addrs, pass,
@@ -200,11 +203,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fp = org_fp;
goto out_addrs;
}
+ addrs[fp->len] = cgctx.idx * 4;
bpf_jit_build_epilogue(code_base, &cgctx);
+ proglen = cgctx.idx * 4;
+
if (bpf_jit_enable > 1)
pr_info("Pass %d: shrink = %d, seen = 0x%x\n", pass,
- proglen - (cgctx.idx * 4), cgctx.seen);
+ prev_proglen - proglen, cgctx.seen);
+
+ /* Check if program size has converged, but ensure minimum passes */
+ if (pass >= CODEGEN_MIN_PASSES && proglen == prev_proglen)
+ break;
+
+ if (pass == CODEGEN_MAX_PASSES && proglen != prev_proglen) {
+ pr_err("BPF JIT: Program did not converge after %d passes\n",
+ CODEGEN_MAX_PASSES);
+ bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
+ bpf_jit_binary_pack_free(fhdr, hdr);
+ fp = org_fp;
+ goto out_addrs;
+ }
}
if (bpf_jit_enable > 1)
@@ -264,7 +283,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass
u32 *fixup;
/* Populate extable entries only in the last pass */
- if (pass != 2)
+ if (pass < CODEGEN_MIN_PASSES)
return 0;
if (!fp->aux->extable ||
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 4bcb6f222065..aad5200b7bc1 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -69,6 +69,8 @@ static struct iommu_table *iommu_pseries_alloc_table(int node)
return tbl;
}
+static phys_addr_t pseries_ddw_max_ram __ro_after_init;
+
#ifdef CONFIG_IOMMU_API
static struct iommu_table_group_ops spapr_tce_table_group_ops;
#endif
@@ -1284,15 +1286,19 @@ struct failed_ddw_pdn {
static LIST_HEAD(failed_ddw_pdn_list);
-static phys_addr_t ddw_memory_hotplug_max(void)
+static phys_addr_t __init ddw_memory_hotplug_max(void)
{
- resource_size_t max_addr;
+ resource_size_t max_addr = memory_hotplug_max();
+ struct device_node *memory;
-#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
- max_addr = hot_add_drconf_memory_max();
-#else
- max_addr = memblock_end_of_DRAM();
-#endif
+ for_each_node_by_type(memory, "memory") {
+ struct resource res;
+
+ if (of_address_to_resource(memory, 0, &res))
+ continue;
+
+ max_addr = max_t(resource_size_t, max_addr, res.end + 1);
+ }
return max_addr;
}
@@ -1447,7 +1453,7 @@ static struct property *ddw_property_create(const char *propname, u32 liobn, u64
static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mask)
{
int len = 0, ret;
- int max_ram_len = order_base_2(ddw_memory_hotplug_max());
+ int max_ram_len = order_base_2(pseries_ddw_max_ram);
struct ddw_query_response query;
struct ddw_create_response create;
int page_shift;
@@ -1617,12 +1623,12 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
if (default_win_removed && pmem_present && !direct_mapping) {
/* DDW is big enough to be split */
if ((1ULL << ddw_sz) >=
- MIN_DDW_VPMEM_DMA_WINDOW + (1ULL << max_ram_len)) {
+ MIN_DDW_VPMEM_DMA_WINDOW + pseries_ddw_max_ram) {
direct_mapping = true;
/* offset of the Dynamic part of DDW */
- dynamic_offset = 1ULL << max_ram_len;
+ dynamic_offset = pseries_ddw_max_ram;
}
/* DDW will at least have dynamic allocation */
@@ -1669,7 +1675,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
if (direct_mapping) {
/* DDW maps the whole partition, so enable direct DMA mapping */
- ret = walk_system_ram_range(0, ddw_memory_hotplug_max() >> PAGE_SHIFT,
+ ret = walk_system_ram_range(0, pseries_ddw_max_ram >> PAGE_SHIFT,
win64->value, tce_setrange_multi_pSeriesLP_walk);
if (ret) {
dev_info(&dev->dev, "failed to map DMA window for %pOF: %d\n",
@@ -1774,8 +1780,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
* to directly map both RAM and vPMEM, then we need to set DMA limit.
*/
if (pmem_present && direct_mapping && len != MAX_PHYSMEM_BITS)
- dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset +
- (1ULL << max_ram_len);
+ dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + pseries_ddw_max_ram;
dev_info(&dev->dev, "lsa_required: %x, lsa_enabled: %x, direct mapping: %x\n",
limited_addr_req, limited_addr_enabled, direct_mapping);
@@ -2422,23 +2427,35 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action,
{
struct dma_win *window;
struct memory_notify *arg = data;
+ unsigned long limit = arg->nr_pages;
+ unsigned long max_ram_pages = pseries_ddw_max_ram >> PAGE_SHIFT;
int ret = 0;
/* This notifier can get called when onlining persistent memory as well.
* TCEs are not pre-mapped for persistent memory. Persistent memory will
- * always be above ddw_memory_hotplug_max()
+ * always be above pseries_ddw_max_ram
*/
+ if (arg->start_pfn >= max_ram_pages)
+ return NOTIFY_OK;
+
+ /* RAM is being DLPAR'ed. The range should never exceed max ram.
+ * Just in case, clamp the range and throw a warning.
+ */
+ if (arg->start_pfn + limit > max_ram_pages) {
+ limit = max_ram_pages - arg->start_pfn;
+ WARN_ONCE(1, "Limiting Page Range %lx - %lx to Max Mem Pages: %lx\n",
+ arg->start_pfn, arg->start_pfn + arg->nr_pages,
+ max_ram_pages);
+ }
switch (action) {
case MEM_GOING_ONLINE:
spin_lock(&dma_win_list_lock);
list_for_each_entry(window, &dma_win_list, list) {
- if (window->direct && (arg->start_pfn << PAGE_SHIFT) <
- ddw_memory_hotplug_max()) {
+ if (window->direct) {
ret |= tce_setrange_multi_pSeriesLP(arg->start_pfn,
- arg->nr_pages, window->prop);
+ limit, window->prop);
}
- /* XXX log error */
}
spin_unlock(&dma_win_list_lock);
break;
@@ -2446,12 +2463,10 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action,
case MEM_OFFLINE:
spin_lock(&dma_win_list_lock);
list_for_each_entry(window, &dma_win_list, list) {
- if (window->direct && (arg->start_pfn << PAGE_SHIFT) <
- ddw_memory_hotplug_max()) {
+ if (window->direct) {
ret |= tce_clearrange_multi_pSeriesLP(arg->start_pfn,
- arg->nr_pages, window->prop);
+ limit, window->prop);
}
- /* XXX log error */
}
spin_unlock(&dma_win_list_lock);
break;
@@ -2535,6 +2550,14 @@ void __init iommu_init_early_pSeries(void)
register_memory_notifier(&iommu_mem_nb);
set_pci_dma_ops(&dma_iommu_ops);
+
+ /* During init determine the max memory an LPAR can have and set it. This
+ * will be used for pre-mapping RAM in DDW. memblock_end_of_DRAM() can
+ * change during the running of LPAR - daxctl can add pmemory as
+ * "system-ram". This memory range should not be pre-mapped in DDW since
+ * the address of pmemory can be much higher than the DDW size.
+ */
+ pseries_ddw_max_ram = ddw_memory_hotplug_max();
}
static int __init disable_multitce(char *str)
diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c
index e73be96ce5fe..836db17f4e8f 100644
--- a/arch/s390/pci/pci_irq.c
+++ b/arch/s390/pci/pci_irq.c
@@ -450,15 +450,10 @@ static int __init zpci_directed_irq_init(void)
if (!zpci_sbv)
return -ENOMEM;
- iib.diib.isc = PCI_ISC;
- iib.diib.nr_cpus = num_possible_cpus();
- iib.diib.disb_addr = virt_to_phys(zpci_sbv->vector);
- zpci_set_irq_ctrl(SIC_IRQ_MODE_DIRECT, 0, &iib);
-
zpci_ibv = kcalloc(num_possible_cpus(), sizeof(*zpci_ibv),
GFP_KERNEL);
if (!zpci_ibv)
- return -ENOMEM;
+ goto out_free_sbv;
for_each_possible_cpu(cpu) {
/*
@@ -470,13 +465,30 @@ static int __init zpci_directed_irq_init(void)
AIRQ_IV_CACHELINE |
(!cpu ? AIRQ_IV_ALLOC : 0), NULL);
if (!zpci_ibv[cpu])
- return -ENOMEM;
+ goto out_free_dibv;
}
+
+ iib.diib.isc = PCI_ISC;
+ iib.diib.nr_cpus = num_possible_cpus();
+ iib.diib.disb_addr = virt_to_phys(zpci_sbv->vector);
+ zpci_set_irq_ctrl(SIC_IRQ_MODE_DIRECT, 0, &iib);
on_each_cpu(cpu_enable_directed_irq, NULL, 1);
zpci_irq_chip.irq_set_affinity = zpci_set_irq_affinity;
return 0;
+
+out_free_dibv:
+ for_each_possible_cpu(cpu) {
+ if (zpci_ibv[cpu])
+ airq_iv_release(zpci_ibv[cpu]);
+ }
+ kfree(zpci_ibv);
+ zpci_ibv = NULL;
+out_free_sbv:
+ airq_iv_release(zpci_sbv);
+ zpci_sbv = NULL;
+ return -ENOMEM;
}
static int __init zpci_floating_irq_init(void)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 383d2bb218de..d73d6cc24922 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -483,6 +483,11 @@
#define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */
#define X86_FEATURE_FAST_CPPC (21*32 + 5) /* AMD Fast CPPC */
#define X86_FEATURE_INDIRECT_THUNK_ITS (21*32 + 6) /* Use thunk for indirect branches in lower half of cacheline */
+/*
+ * Upstream defines X86_FEATURE_ZEN6 as (3*32+6). That bit is still
+ * X86_FEATURE_P3 in this kernel, so use a free bit in this word.
+ */
+#define X86_FEATURE_ZEN6 (21*32+7) /* CPU based on Zen6 microarchitecture */
#define X86_FEATURE_TSA_SQ_NO (21*32+11) /* AMD CPU not vulnerable to TSA-SQ */
#define X86_FEATURE_TSA_L1_NO (21*32+12) /* AMD CPU not vulnerable to TSA-L1 */
diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index 861d080ed4c6..63bdbd15efcd 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -78,6 +78,7 @@ KVM_X86_OP(interrupt_allowed)
KVM_X86_OP(nmi_allowed)
KVM_X86_OP(get_nmi_mask)
KVM_X86_OP(set_nmi_mask)
+KVM_X86_OP_OPTIONAL(clear_hlt)
KVM_X86_OP(enable_nmi_window)
KVM_X86_OP(enable_irq_window)
KVM_X86_OP_OPTIONAL(update_cr8_intercept)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index cec497aed149..93d3f8010a1f 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1735,6 +1735,13 @@ struct kvm_x86_ops {
int (*nmi_allowed)(struct kvm_vcpu *vcpu, bool for_injection);
bool (*get_nmi_mask)(struct kvm_vcpu *vcpu);
void (*set_nmi_mask)(struct kvm_vcpu *vcpu, bool masked);
+ /*
+ * Force the vCPU out of any hardware-tracked halted/inactive state so
+ * that it will fetch and execute instructions on the next VM-Enter.
+ * Only needed by VMX, where VMCS.GUEST_ACTIVITY_STATE persists across
+ * VM-Exit/VM-Enter; SVM has no equivalent VMCB field.
+ */
+ void (*clear_hlt)(struct kvm_vcpu *vcpu);
/* Whether or not a virtual NMI is pending in hardware. */
bool (*is_vnmi_pending)(struct kvm_vcpu *vcpu);
/*
diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 4220dae14a2d..fb011cef01e7 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -276,20 +276,22 @@ unsigned long vmware_hypercall_hb_out(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep outsb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
@@ -304,20 +306,22 @@ unsigned long vmware_hypercall_hb_in(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep insb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
diff --git a/arch/x86/include/asm/xor_avx.h b/arch/x86/include/asm/xor_avx.h
index 7f81dd5897f4..c6450acd055d 100644
--- a/arch/x86/include/asm/xor_avx.h
+++ b/arch/x86/include/asm/xor_avx.h
@@ -50,6 +50,7 @@ do { \
p1 = (unsigned long *)((uintptr_t)p1 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -81,6 +82,7 @@ do { \
p2 = (unsigned long *)((uintptr_t)p2 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -116,6 +118,7 @@ do { \
p3 = (unsigned long *)((uintptr_t)p3 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -155,6 +158,7 @@ do { \
p4 = (unsigned long *)((uintptr_t)p4 + 512);
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/arch/x86/kernel/acpi/cppc.c b/arch/x86/kernel/acpi/cppc.c
index 12fc5f553405..d9cfed143b79 100644
--- a/arch/x86/kernel/acpi/cppc.c
+++ b/arch/x86/kernel/acpi/cppc.c
@@ -4,6 +4,8 @@
* Copyright (c) 2016, Intel Corporation.
*/
+#include <linux/limits.h>
+
#include <acpi/cppc_acpi.h>
#include <asm/msr.h>
#include <asm/processor.h>
@@ -43,10 +45,20 @@ bool cpc_ffh_supported(void)
return true;
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->address <= U32_MAX && reg->bit_width &&
+ reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val)
{
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrl_safe_on_cpu(cpunum, reg->address, val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
@@ -63,6 +75,9 @@ int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val)
u64 rd_val;
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrl_safe_on_cpu(cpunum, reg->address, &rd_val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index f06966220602..686e0d1df92f 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -531,6 +531,12 @@ static void bsp_init_amd(struct cpuinfo_x86 *c)
case 0xd0 ... 0xd7:
setup_force_cpu_cap(X86_FEATURE_ZEN5);
break;
+ case 0x50 ... 0x5f:
+ case 0x80 ... 0xaf:
+ case 0xc0 ... 0xcf:
+ case 0xd8 ... 0xef:
+ setup_force_cpu_cap(X86_FEATURE_ZEN6);
+ break;
default:
goto warn;
}
diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c
index 9a95d00f1423..49b3cc215a6a 100644
--- a/arch/x86/kernel/nmi.c
+++ b/arch/x86/kernel/nmi.c
@@ -120,6 +120,14 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration)
{
int remainder_ns, decimal_msecs;
+ /* sched_clock() went backwards? */
+ if ((s64)duration < 0) {
+ pr_warn_once("NMI handler (%ps): sched_clock() went backwards by %lld ns on CPU%d\n",
+ action->handler, -(s64)duration,
+ smp_processor_id());
+ return;
+ }
+
if (duration < nmi_longest_ns || duration < action->max_duration)
return;
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 0a2bbd674a6d..c1e20abc7fe5 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -36,7 +36,7 @@ static void iommu_shutdown_noop(void) { }
bool __init bool_x86_init_noop(void) { return false; }
void x86_op_int_noop(int cpu) { }
int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; }
-void get_rtc_noop(struct timespec64 *now) { }
+void get_rtc_noop(struct timespec64 *now) { *now = (struct timespec64){0}; }
static __initconst const struct of_device_id of_cmos_match[] = {
{ .compatible = "motorola,mc146818" },
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 3d9e8fcc5589..c4f33dc6dabc 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2156,6 +2156,16 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(struct kvm *kvm,
continue;
}
+ /*
+ * Note! Unlike fully direct MMUs, direct SPs in indirect MMUs
+ * can have an arbitrary number of parents, e.g if the guest is
+ * mapping a huge PTE at many points in its page tables.
+ */
+ if (pte_list_count(&sp->parent_ptes) >= RMAP_RECYCLE_THRESHOLD) {
+ kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list);
+ break;
+ }
+
/* unsync and write-flooding only apply to indirect SPs. */
if (sp->role.direct)
goto out;
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index 10e9e87f1420..aa6abaa60117 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -321,6 +321,13 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
gpa_t real_gpa;
gfn_t gfn;
+ /*
+ * KVM might sleep/resched when retrying the walk if the updating A/D
+ * fails because the gPTE changed between reading the gPTE and trying
+ * the CMPXHG to set A/D bits.
+ */
+ might_sleep();
+
trace_kvm_mmu_pagetable_walk(addr, access);
retry_walk:
walker->level = mmu->cpu_role.base.level;
@@ -471,8 +478,10 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
addr, write_fault);
if (unlikely(ret < 0))
goto error;
- else if (ret)
+ else if (ret) {
+ cond_resched();
goto retry_walk;
+ }
}
return 1;
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index e2e9ab085885..3c9c20d7855f 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1763,6 +1763,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
if (!(save->cr0 & X86_CR0_PG) ||
!(save->cr0 & X86_CR0_PE) ||
(save->rflags & X86_EFLAGS_VM) ||
+ ((save->efer & EFER_LMA) && !(save->efer & EFER_LME)) ||
!__nested_vmcb_check_save(vcpu, &save_cached))
goto out_free;
diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
index 3f83e36a657b..8a922ce0d19c 100644
--- a/arch/x86/kvm/vmx/main.c
+++ b/arch/x86/kvm/vmx/main.c
@@ -90,6 +90,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
.nmi_allowed = vmx_nmi_allowed,
.get_nmi_mask = vmx_get_nmi_mask,
.set_nmi_mask = vmx_set_nmi_mask,
+ .clear_hlt = vmx_clear_hlt,
.enable_nmi_window = vmx_enable_nmi_window,
.enable_irq_window = vmx_enable_irq_window,
.update_cr8_intercept = vmx_update_cr8_intercept,
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index c2aae9db372d..88654327a707 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -4968,6 +4968,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)
__func__, j, h.index, h.value);
goto vmabort;
}
+ break;
}
}
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index a36d9f504ba5..8e23e901a759 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -1814,7 +1814,7 @@ int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu)
return skip_emulated_instruction(vcpu);
}
-static void vmx_clear_hlt(struct kvm_vcpu *vcpu)
+void vmx_clear_hlt(struct kvm_vcpu *vcpu)
{
/*
* Ensure that we clear the HLT state in the VMCS. We don't need to
diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h
index 5e4ce13ab305..4348674de9f3 100644
--- a/arch/x86/kvm/vmx/x86_ops.h
+++ b/arch/x86/kvm/vmx/x86_ops.h
@@ -96,6 +96,7 @@ int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection);
int vmx_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection);
bool vmx_get_nmi_mask(struct kvm_vcpu *vcpu);
void vmx_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked);
+void vmx_clear_hlt(struct kvm_vcpu *vcpu);
void vmx_enable_nmi_window(struct kvm_vcpu *vcpu);
void vmx_enable_irq_window(struct kvm_vcpu *vcpu);
void vmx_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 68599667907c..ee3b528ec1e2 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8835,10 +8835,12 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
} else if (vcpu->arch.complete_userspace_io) {
writeback = false;
r = 0;
- } else if (r == EMULATION_RESTART)
+ } else if (r == EMULATION_RESTART) {
+ cond_resched();
goto restart;
- else
+ } else {
r = 1;
+ }
writeback:
if (writeback) {
@@ -11281,6 +11283,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
set_bit(KVM_APIC_SIPI, &vcpu->arch.apic->pending_events);
} else
vcpu->arch.mp_state = mp_state->mp_state;
+
+ /*
+ * Force the vCPU out of any hardware-tracked halted state, e.g. VMX's
+ * GUEST_ACTIVITY_STATE=HLT, when userspace puts the vCPU into a state
+ * other than HALTED. The hardware state is sticky across VM-Exit and
+ * VM-Enter and is not touched by any other ioctl, so a vCPU that halted
+ * with HLT-exiting disabled stays wedged even after userspace rewrites
+ * its registers, e.g. when a VMM emulates a machine reset.
+ */
+ if (kvm_hlt_in_guest(vcpu->kvm) &&
+ mp_state->mp_state != KVM_MP_STATE_HALTED)
+ kvm_x86_call(clear_hlt)(vcpu);
+
kvm_make_request(KVM_REQ_EVENT, vcpu);
ret = 0;
@@ -11781,18 +11796,23 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
kvm_x86_call(set_efer)(vcpu, 0);
kvm_x86_call(update_exception_bitmap)(vcpu);
+ if (old_cr0 & X86_CR0_PG)
+ kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
+
/*
- * On the standard CR0/CR4/EFER modification paths, there are several
- * complex conditions determining whether the MMU has to be reset and/or
- * which PCIDs have to be flushed. However, CR0.WP and the paging-related
- * bits in CR4 and EFER are irrelevant if CR0.PG was '0'; and a reset+flush
- * is needed anyway if CR0.PG was '1' (which can only happen for INIT, as
- * CR0 will be '0' prior to RESET). So we only need to check CR0.PG here.
+ * Unconditionally reset the MMU, even though super strictly speaking,
+ * a MMU reset is only necessary if and only one of the page role bits
+ * is changing. All of the CR0/CR4/EFER bits effectively get covered
+ * by checking CR0.PG=1, as CR0.WP and the paging-related bits in CR4
+ * and EFER are irrelevant if CR0.PG was '0'. Subtly, if the vCPU was
+ * in guest mode (L2) or SMM (although emulating INIT in SMM _should_
+ * be impossible), an MMU reset is also necessary as "guest mode" and
+ * "smm" are both encoded in the shadow page role. As a result, the
+ * only scenarios where an MMU reset is *not* needed is if the vCPU is
+ * in Real Mode or Unpaged Protected Mode, and micro-optimizing those
+ * paths are not worth the risk of running with a stale MMU context.
*/
- if (old_cr0 & X86_CR0_PG) {
- kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
- kvm_mmu_reset_context(vcpu);
- }
+ kvm_mmu_reset_context(vcpu);
/*
* Intel's SDM states that all TLB entries are flushed on INIT. AMD's
diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c
index b922b9fea6b6..1bbc117f91ab 100644
--- a/arch/x86/mm/mem_encrypt_identity.c
+++ b/arch/x86/mm/mem_encrypt_identity.c
@@ -101,7 +101,7 @@ static void __head sme_clear_pgd(struct sme_populate_pgd_data *ppd)
pgd_t *pgd_p;
pgd_start = ppd->vaddr & PGDIR_MASK;
- pgd_end = ppd->vaddr_end & PGDIR_MASK;
+ pgd_end = (ppd->vaddr_end - 1) & PGDIR_MASK;
pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1) * sizeof(pgd_t);
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 4707b9349305..b33df932ea1b 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -2819,6 +2819,9 @@ static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq)
if (!ioc->enabled || !rq->alloc_time_ns || !rq->start_time_ns)
return;
+ if (unlikely(rq->rq_flags & RQF_FLUSH_SEQ))
+ return;
+
switch (req_op(rq)) {
case REQ_OP_READ:
pidx = QOS_RLAT;
diff --git a/block/blk-mq.c b/block/blk-mq.c
index e01437e64c74..c5be2e8578a7 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3122,17 +3122,15 @@ void blk_mq_submit_bio(struct bio *bio)
blk_mq_bio_to_request(rq, bio, nr_segs);
+ if (bio_zone_write_plugging(bio))
+ blk_zone_write_plug_init_request(rq);
+
ret = blk_crypto_rq_get_keyslot(rq);
if (ret != BLK_STS_OK) {
- bio->bi_status = ret;
- bio_endio(bio);
- blk_mq_free_request(rq);
+ blk_mq_end_request(rq, ret);
return;
}
- if (bio_zone_write_plugging(bio))
- blk_zone_write_plug_init_request(rq);
-
if (op_is_flush(bio->bi_opf) && blk_insert_flush(rq))
return;
diff --git a/block/bsg.c b/block/bsg.c
index 72157a59b788..7231b1de663f 100644
--- a/block/bsg.c
+++ b/block/bsg.c
@@ -130,7 +130,10 @@ static long bsg_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
case SG_SET_TIMEOUT:
if (get_user(val, intp))
return -EFAULT;
- bd->timeout = clock_t_to_jiffies(val);
+ if (val < 0)
+ return -EIO;
+ bd->timeout = min(clock_t_to_jiffies(val),
+ min(UINT_MAX, MAX_JIFFY_OFFSET));
return 0;
case SG_GET_TIMEOUT:
return jiffies_to_clock_t(bd->timeout);
diff --git a/block/ioctl.c b/block/ioctl.c
index 919066b4bb49..f801bddf6fd1 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -736,8 +736,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg)
return compat_put_ulong(argp, bdev_nr_sectors(bdev));
/* The data is compatible, but the command number is different */
- case BLKBSZGET_32: /* get the logical block size (cf. BLKSSZGET) */
- return put_int(argp, bdev_logical_block_size(bdev));
+ case BLKBSZGET_32: /* get the soft block size (cf. BLKSSZGET) */
+ return put_int(argp, block_size(bdev));
case BLKBSZSET_32:
return blkdev_bszset(file, mode, argp);
case BLKGETSIZE64_32:
diff --git a/crypto/ecc.c b/crypto/ecc.c
index f97ff234086f..a289d47d5651 100644
--- a/crypto/ecc.c
+++ b/crypto/ecc.c
@@ -1410,9 +1410,22 @@ static void ecc_point_add(const struct ecc_point *result,
vli_set(result->x, q->x, ndigits);
vli_set(result->y, q->y, ndigits);
vli_mod_sub(z, result->x, p->x, curve->p, ndigits);
+ if (vli_is_zero(z, ndigits)) {
+ if (vli_cmp(p->y, q->y, ndigits)) {
+ /* P + (-P) is the point at infinity. */
+ vli_clear(result->x, ndigits);
+ vli_clear(result->y, ndigits);
+ return;
+ }
+ /* The co-Z addition formula does not handle P == Q. */
+ z[0] = 1;
+ ecc_point_double_jacobian(result->x, result->y, z, curve);
+ goto out;
+ }
vli_set(px, p->x, ndigits);
vli_set(py, p->y, ndigits);
xycz_add(px, py, result->x, result->y, curve);
+out:
vli_mod_inv(z, z, curve->p, ndigits);
apply_z(result->x, result->y, z, curve);
}
@@ -1452,22 +1465,38 @@ void ecc_point_mult_shamir(const struct ecc_point *result,
vli_set(rx, point->x, ndigits);
vli_set(ry, point->y, ndigits);
vli_clear(z + 1, ndigits - 1);
- z[0] = 1;
+ z[0] = !ecc_point_is_zero(point);
for (--i; i >= 0; i--) {
ecc_point_double_jacobian(rx, ry, z, curve);
idx = !!vli_test_bit(u1, i);
idx |= (!!vli_test_bit(u2, i)) << 1;
point = points[idx];
- if (point) {
+ if (point && !ecc_point_is_zero(point)) {
u64 tx[ECC_MAX_DIGITS];
u64 ty[ECC_MAX_DIGITS];
u64 tz[ECC_MAX_DIGITS];
+ if (vli_is_zero(z, ndigits)) {
+ /* Adding to infinity starts a new accumulator. */
+ vli_set(rx, point->x, ndigits);
+ vli_set(ry, point->y, ndigits);
+ z[0] = 1;
+ continue;
+ }
vli_set(tx, point->x, ndigits);
vli_set(ty, point->y, ndigits);
apply_z(tx, ty, z, curve);
vli_mod_sub(tz, rx, tx, curve->p, ndigits);
+ if (vli_is_zero(tz, ndigits)) {
+ if (!vli_cmp(ry, ty, ndigits))
+ ecc_point_double_jacobian(rx, ry, z,
+ curve);
+ else
+ /* Adding opposite points yields infinity. */
+ vli_clear(z, ndigits);
+ continue;
+ }
xycz_add(tx, ty, rx, ry, curve);
vli_mod_mult_fast(z, z, tz, curve);
}
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 2f5f6b52b2d4..e7848ea55ca1 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -4209,8 +4209,10 @@ static int test_akcipher_one(struct crypto_akcipher *tfm,
crypto_akcipher_verify(req) :
/* Run asymmetric encrypt */
crypto_akcipher_encrypt(req), &wait);
- if (err) {
- pr_err("alg: akcipher: %s test failed. err %d\n", op, err);
+ if (err != vecs->verify_error) {
+ pr_err("alg: akcipher: %s test failed: expected %d, got %d\n",
+ op, vecs->verify_error, err);
+ err = err ?: -EINVAL;
goto free_all;
}
if (!vecs->siggen_sigver_test && c) {
@@ -4232,9 +4234,9 @@ static int test_akcipher_one(struct crypto_akcipher *tfm,
/*
* Don't invoke (decrypt or sign) test which require a private key
- * for vectors with only a public key.
+ * for vectors with only a public key or an invalid signature.
*/
- if (vecs->public_key_vec) {
+ if (vecs->public_key_vec || vecs->verify_error) {
err = 0;
goto free_all;
}
diff --git a/crypto/testmgr.h b/crypto/testmgr.h
index 9b38501a17b2..2f0aa6f4454f 100644
--- a/crypto/testmgr.h
+++ b/crypto/testmgr.h
@@ -157,6 +157,7 @@ struct akcipher_testvec {
unsigned int param_len;
unsigned int m_size;
unsigned int c_size;
+ int verify_error;
bool public_key_vec;
bool siggen_sigver_test;
enum OID algo;
@@ -778,6 +779,146 @@ static const struct akcipher_testvec ecdsa_nist_p192_tv_template[] = {
};
static const struct akcipher_testvec ecdsa_nist_p256_tv_template[] = {
+ /* Invalid signature: Q = G, r = s, e = 3r mod n. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\x4f\xe3\x42\xe2\xfe\x1a\x7f\x9b\x8e\xe7\xeb\x4a\x7c\x0f\x9e"
+ "\x16\x2b\xce\x33\x57\x6b\x31\x5e\xce\xcb\xb6\x40\x68\x37\xbf\x51"
+ "\xf5",
+ .key_len = 65,
+ .m =
+ "\xb2\xc3\x09\x72\x03\x17\x9d\xaa\x5c\x4a\xd8\xff\x02\xfe\xcb\x15"
+ "\x21\x53\x81\x77\x45\xec\x45\x24\x3d\x68\x10\xb9\xf2\xa5\x88\x3a",
+ .m_size = 32,
+ .c =
+ "\x30\x46\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e\x74\x18\xf2"
+ "\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3\x10\x60\x9e"
+ "\x7e\xfa\x58\x39\xd9\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e"
+ "\x74\x18\xf2\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3"
+ "\x10\x60\x9e\x7e\xfa\x58\x39\xd9",
+ .c_size = 72,
+ .verify_error = -EKEYREJECTED,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* Invalid signature: Q = -G, r = s, e = 3r mod n. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\xb0\x1c\xbd\x1c\x01\xe5\x80\x65\x71\x18\x14\xb5\x83\xf0\x61"
+ "\xe9\xd4\x31\xcc\xa9\x94\xce\xa1\x31\x34\x49\xbf\x97\xc8\x40\xae"
+ "\x0a",
+ .key_len = 65,
+ .m =
+ "\xb2\xc3\x09\x72\x03\x17\x9d\xaa\x5c\x4a\xd8\xff\x02\xfe\xcb\x15"
+ "\x21\x53\x81\x77\x45\xec\x45\x24\x3d\x68\x10\xb9\xf2\xa5\x88\x3a",
+ .m_size = 32,
+ .c =
+ "\x30\x46\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e\x74\x18\xf2"
+ "\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3\x10\x60\x9e"
+ "\x7e\xfa\x58\x39\xd9\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e"
+ "\x74\x18\xf2\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3"
+ "\x10\x60\x9e\x7e\xfa\x58\x39\xd9",
+ .c_size = 72,
+ .verify_error = -EKEYREJECTED,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* Wycheproof secp256r1/SHA-256 tcId 60. */
+ {
+ .key =
+ "\x04\x29\x27\xb1\x05\x12\xba\xe3\xed\xdc\xfe\x46\x78\x28\x12\x8b"
+ "\xad\x29\x03\x26\x99\x19\xf7\x08\x60\x69\xc8\xc4\xdf\x6c\x73\x28"
+ "\x38\xc7\x78\x79\x64\xea\xac\x00\xe5\x92\x1f\xb1\x49\x8a\x60\xf4"
+ "\x60\x67\x66\xb3\xd9\x68\x50\x01\x55\x8d\x1a\x97\x4e\x73\x41\x51"
+ "\x3e",
+ .key_len = 65,
+ .m =
+ "\x70\x23\x9d\xd8\x77\xf7\xc9\x44\xc4\x22\xf4\x4d\xea\x4e\xd1\xa5"
+ "\x2f\x26\x27\x41\x6f\xaf\x2f\x07\x2f\xa5\x0c\x77\x2e\xd6\xf8\x07",
+ .m_size = 32,
+ .c =
+ "\x30\x44\x02\x20\x64\xa1\xaa\xb5\x00\x0d\x0e\x80\x4f\x3e\x2f\xc0"
+ "\x2b\xde\xe9\xbe\x8f\xf3\x12\x33\x4e\x2b\xa1\x6d\x11\x54\x7c\x97"
+ "\x71\x1c\x89\x8e\x02\x20\x6a\xf0\x15\x97\x1c\xc3\x0b\xe6\xd1\xa2"
+ "\x06\xd4\xe0\x13\xe0\x99\x77\x72\xa2\xf9\x1d\x73\x28\x6f\xfd\x68"
+ "\x3b\x9b\xb2\xcf\x4f\x1b",
+ .c_size = 70,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* Wycheproof secp256r1/SHA-256 tcId 210. */
+ {
+ .key =
+ "\x04\xc6\xa7\x71\x52\x70\x24\x22\x77\x92\x17\x0a\x6f\x8e\xee\x73"
+ "\x5b\xf3\x2b\x7f\x98\xaf\x66\x9e\xad\x29\x98\x02\xe3\x2d\x7c\x31"
+ "\x07\xbc\x3b\x4b\x5e\x65\xab\x88\x7b\xbd\x34\x35\x72\xb3\xe5\x61"
+ "\x92\x61\xfe\x3a\x07\x3e\x2f\xfd\x78\x41\x2f\x72\x68\x67\xdb\x58"
+ "\x9e",
+ .key_len = 65,
+ .m =
+ "\xbb\x5a\x52\xf4\x2f\x9c\x92\x61\xed\x43\x61\xf5\x94\x22\xa1\xe3"
+ "\x00\x36\xe7\xc3\x2b\x27\x0c\x88\x07\xa4\x19\xfe\xca\x60\x50\x23",
+ .m_size = 32,
+ .c =
+ "\x30\x45\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03"
+ "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc"
+ "\x47\x66\x99\x78\x02\x21\x00\xb6\xdb\x6d\xb6\x24\x92\x49\x25\x49"
+ "\x24\x92\x49\x24\x92\x49\x24\x62\x5b\xd7\xa0\x9b\xec\x4c\xa8\x1b"
+ "\xcd\xd9\xf8\xfd\x6b\x63\xcc",
+ .c_size = 71,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* Q = G, d = 1, k = 2. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\x4f\xe3\x42\xe2\xfe\x1a\x7f\x9b\x8e\xe7\xeb\x4a\x7c\x0f\x9e"
+ "\x16\x2b\xce\x33\x57\x6b\x31\x5e\xce\xcb\xb6\x40\x68\x37\xbf\x51"
+ "\xf5",
+ .key_len = 65,
+ .m =
+ "\x35\x9d\x17\x29\xb9\x1a\x52\xe9\x8b\xb5\x95\xc3\x86\x28\x3c\x10"
+ "\x30\x44\x95\x1a\xa1\x08\x20\xf0\xa0\x22\xc4\x18\xc2\x71\xa0\xcc",
+ .m_size = 32,
+ .c =
+ "\x30\x44\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03"
+ "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc"
+ "\x47\x66\x99\x78\x02\x20\x59\x47\xc9\x21\x23\x0e\xd1\x34\x0b\x03"
+ "\xe6\xe3\x45\x6e\xab\x69\xf8\x66\xff\x7e\x8c\x7d\x1e\x13\x23\x17"
+ "\x06\x8a\x84\xec\x1d\x22",
+ .c_size = 70,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* Q = -G, d = n - 1, k = 2. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\xb0\x1c\xbd\x1c\x01\xe5\x80\x65\x71\x18\x14\xb5\x83\xf0\x61"
+ "\xe9\xd4\x31\xcc\xa9\x94\xce\xa1\x31\x34\x49\xbf\x97\xc8\x40\xae"
+ "\x0a",
+ .key_len = 65,
+ .m =
+ "\x35\x9d\x17\x29\xb9\x1a\x52\xe9\x8b\xb5\x95\xc3\x86\x28\x3c\x10"
+ "\x30\x44\x95\x1a\xa1\x08\x20\xf0\xa0\x22\xc4\x18\xc2\x71\xa0\xcc",
+ .m_size = 32,
+ .c =
+ "\x30\x45\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03"
+ "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc"
+ "\x47\x66\x99\x78\x02\x21\x00\xdc\x55\x4e\x07\x96\x0b\x81\xb6\x80"
+ "\xb1\xae\xe0\x40\xb9\x90\xa5\xf4\xc4\x90\x49\xbb\xa2\xa1\x62\x70"
+ "\xc5\x88\x51\x39\xe8\xa8\xfb",
+ .c_size = 71,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
{
.key =
"\x04\xb9\x7b\xbb\xd7\x17\x64\xd2\x7e\xfc\x81\x5d\x87\x06\x83\x41"
@@ -1218,7 +1359,8 @@ static const struct akcipher_testvec ecdsa_nist_p521_tv_template[] = {
};
/*
- * EC-RDSA test vectors are generated by gost-engine.
+ * EC-RDSA test vectors generated by gost-engine, followed by constructed
+ * exceptional-point regression vectors.
*/
static const struct akcipher_testvec ecrdsa_tv_template[] = {
{
@@ -1369,6 +1511,201 @@ static const struct akcipher_testvec ecrdsa_tv_template[] = {
.public_key_vec = true,
.siggen_sigver_test = true,
},
+ /*
+ * Constructed EC-RDSA exceptional-point vectors.
+ * With Q = dG, choose k = z1 + d*z2 (mod n), r = x(kG) mod n,
+ * e = -r/z2 (mod n), and s = r*d + k*e (mod n).
+ * The input digest is e, encoded little-endian.
+ */
+ /* cp256a: Q=G, z1 = 3, z2 = 1. */
+ {
+ .key =
+ "\x04\x40\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x14\x1e\x9f\x9e\x9c\xc9\xac\x22\xb1\xe3\x23\xdf\x2d\x4f"
+ "\x29\x35\x76\x2b\x3f\x45\x5a\x50\xdf\x27\xda\x9c\x98\xe0\x71\xe4"
+ "\x91\x8d",
+ .key_len = 66,
+ .params = /* OID_gostCPSignA */
+ "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03"
+ "\x07\x01\x01\x02\x02",
+ .param_len = 21,
+ .c =
+ "\x1a\xed\x44\xae\xd4\x4a\xed\x44\xae\xd4\x4a\xed\x44\xae\xd4\x49"
+ "\x32\x67\xe0\x26\x16\xfd\xb7\xaf\xa4\xd7\x3e\x61\xd4\xf9\x7b\x94"
+ "\xf7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7"
+ "\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe4\xb7",
+ .c_size = 64,
+ .m =
+ "\xdc\xd3\xfd\x46\xcb\x14\x9d\xe1\x8f\x92\x54\xb2\x0c\xa0\x22\x66"
+ "\x18\x9c\x8f\xc1\xf9\x18\x9c\x8f\xc1\xf9\x18\x9c\x8f\xc1\xf9\x08",
+ .m_size = 32,
+ .algo = OID_gost2012PKey256,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* cp256a: Q=-G, z1 = 7, z2 = 6. */
+ {
+ .key =
+ "\x04\x40\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x83\xdf\x60\x61\x63\x36\x53\xdd\x4e\x1c\xdc\x20\xd2\xb0"
+ "\xd6\xca\x89\xd4\xc0\xba\xa5\xaf\x20\xd8\x25\x63\x67\x1f\x8e\x1b"
+ "\x6e\x72",
+ .key_len = 66,
+ .params = /* OID_gostCPSignA */
+ "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03"
+ "\x07\x01\x01\x02\x02",
+ .param_len = 21,
+ .c =
+ "\x2a\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa"
+ "\x92\x10\x2d\x68\x19\x8f\x22\xd5\x60\xeb\x59\xd6\xf3\xe5\x9e\xc2"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01",
+ .c_size = 64,
+ .m =
+ "\xc3\x9e\xe5\xf3\xd6\x59\xeb\x60\xd5\x22\x8f\x19\x68\x2d\x10\x92"
+ "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\x2a",
+ .m_size = 32,
+ .algo = OID_gost2012PKey256,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* cp256a: Q=-2G, z1 = 5, z2 = 2. */
+ {
+ .key =
+ "\x04\x40\x95\xfd\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\x14\x1e\x9f\x9e\x9c\xc9\xac\x22\xb1\xe3\x23\xdf\x2d\x4f"
+ "\x29\x35\x76\x2b\x3f\x45\x5a\x50\xdf\x27\xda\x9c\x98\xe0\x71\xe4"
+ "\x91\x8d",
+ .key_len = 66,
+ .params = /* OID_gostCPSignA */
+ "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03"
+ "\x07\x01\x01\x02\x02",
+ .param_len = 21,
+ .c =
+ "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xb6\x30\x88\x38\x4c\xad\x68\x80\x22\xc2\x0d\x84\xdb\xb0\xdc\x47"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01",
+ .c_size = 64,
+ .m =
+ "\x49\xdc\xb0\xdb\x84\x0d\xc2\x22\x80\x68\xad\x4c\x38\x88\x30\xb6"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f",
+ .m_size = 32,
+ .algo = OID_gost2012PKey256,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* tc512a: Q=G, z1 = 3, z2 = 1. */
+ {
+ .key =
+ "\x04\x81\x80\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\xa4\xf2\x15\x52\xcb\x89\xa5\x89\xb8\xf5\x35\xc2\x5f"
+ "\xfe\x28\x80\xe9\x41\x3a\x0e\xa5\xe6\x75\x3d\xe9\x36\xd0\x4f\xbe"
+ "\x26\x16\xdf\x21\xa9\xef\xcb\xfd\x64\x80\x77\xc1\xab\xf1\xac\x93"
+ "\x1c\x5e\xce\xe6\x50\x54\xe2\x16\x88\x1b\xa6\xe3\x6a\x83\x7a\xe8"
+ "\xcf\x03\x75",
+ .key_len = 131,
+ .params = /* OID_gostTC26Sign512A */
+ "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01",
+ .param_len = 13,
+ .c =
+ "\xd8\xc0\xd7\xe3\xfb\xa6\xca\xff\x6a\xd1\xb4\xb4\x82\x1d\x15\x6f"
+ "\xa7\xc7\xbd\x63\x95\xb8\x29\x0d\xd4\xfa\xb1\x1c\x36\xbd\x32\x8d"
+ "\x1a\xba\xf1\x2e\xbc\x7b\xf2\x02\x96\xf8\xef\x2a\x5e\x72\x64\x86"
+ "\x67\x62\xfb\x62\x8e\x83\x53\x63\xb7\xe6\x78\x3c\x42\xd7\x0d\x1d"
+ "\xb7\xbf\xb8\x09\x56\xc8\x67\x00\x31\xba\x19\x19\x29\xf6\x4e\x30"
+ "\x1d\x68\x16\x34\x23\x6d\x47\xa6\x0e\x57\x1a\x4b\xed\xc0\xef\x25"
+ "\x74\x52\xef\x78\xb5\xb9\x8d\xbb\x3d\x9f\x31\x29\xd9\x34\x94\x33"
+ "\xce\x2a\x3a\x35\xcb\x51\x9c\x91\xe2\xd6\x33\xd7\xb3\x73\xae\x16",
+ .c_size = 128,
+ .m =
+ "\x5f\x04\x9d\x6b\x69\x7d\xf7\xe7\xcb\x1b\x81\x2f\x76\xfe\x20\xcd"
+ "\x2c\xcc\xd0\x74\x63\xfa\x52\x32\x56\xfb\xd3\x3e\xba\xa5\x93\xb3"
+ "\xd9\x10\x3f\x12\xb4\xe5\xa8\xf1\x59\xb8\x92\xdc\xcb\xe9\x97\xe2"
+ "\xcf\xb1\x09\xd6\xe6\xe6\x45\xce\xff\x98\x37\xa9\xf6\x47\x40\x48",
+ .m_size = 64,
+ .algo = OID_gost2012PKey512,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* tc512a: Q=-G, z1 = 7, z2 = 6. */
+ {
+ .key =
+ "\x04\x81\x80\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x23\x0b\xea\xad\x34\x76\x5a\x76\x47\x0a\xca\x3d\xa0"
+ "\x01\xd7\x7f\x16\xbe\xc5\xf1\x5a\x19\x8a\xc2\x16\xc9\x2f\xb0\x41"
+ "\xd9\xe9\x20\xde\x56\x10\x34\x02\x9b\x7f\x88\x3e\x54\x0e\x53\x6c"
+ "\xe3\xa1\x31\x19\xaf\xab\x1d\xe9\x77\xe4\x59\x1c\x95\x7c\x85\x17"
+ "\x30\xfc\x8a",
+ .key_len = 131,
+ .params = /* OID_gostTC26Sign512A */
+ "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01",
+ .param_len = 13,
+ .c =
+ "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\x93\xf3\x4a\x99\x7a\x46\xc4\x88\xb7\xf9\x15\xc6\xa7\x02\xb0\x30"
+ "\x4d\xa5\x9c\x55\xfd\x69\x5c\x2e\xe5\x66\xd8\xa0\x8f\x88\x59\x37"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03",
+ .c_size = 128,
+ .m =
+ "\x3a\x59\x88\x8f\xa0\xd8\x66\xe5\x2e\x5c\x69\xfd\x55\x9c\xa5\x4d"
+ "\x30\xb0\x02\xa7\xc6\x15\xf9\xb7\x88\xc4\x46\x7a\x99\x4a\xf3\x93"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f",
+ .m_size = 64,
+ .algo = OID_gost2012PKey512,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
+ /* tc512a: Q=-2G, z1 = 5, z2 = 2. */
+ {
+ .key =
+ "\x04\x81\x80\x7c\x13\xea\xd2\xd9\xaf\x82\xb9\x1f\xb2\xfd\xe1\x8d"
+ "\xf4\x5b\x37\xe4\xf2\x04\x6a\x2a\x1d\x15\x16\x95\x2a\x5d\x40\xcd"
+ "\xbb\x34\x44\x2f\x2b\x13\x0a\x47\xb0\xf6\xe5\xa5\x58\x3a\xf4\x54"
+ "\x09\xb0\x1d\xf5\x5c\xf1\xbf\x9d\x86\xa5\x97\xab\x96\x29\x62\xfc"
+ "\xdc\x89\x3b\x76\x3d\x50\x6e\x8a\xcf\x02\x4c\x74\x0e\x6e\xe0\x1f"
+ "\x28\xab\x27\x9b\x20\x02\xc8\xc4\x12\xcb\x8a\xe3\xa4\x27\xce\x39"
+ "\x62\xdf\x35\x6d\xc1\x65\x7f\x7b\x48\x90\x11\x8f\x19\x7c\x1e\x67"
+ "\x91\x97\xeb\x90\x85\x25\xfc\x86\xb3\x88\x6e\x5c\x57\x1f\x1d\x1d"
+ "\x3b\xec\x37",
+ .key_len = 131,
+ .params = /* OID_gostTC26Sign512A */
+ "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01",
+ .param_len = 13,
+ .c =
+ "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\x93\xf3\x4a\x99\x7a\x46\xc4\x88\xb7\xf9\x15\xc6\xa7\x02\xb0\x30"
+ "\x4d\xa5\x9c\x55\xfd\x69\x5c\x2e\xe5\x66\xd8\xa0\x8f\x88\x59\x33"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03",
+ .c_size = 128,
+ .m =
+ "\x39\x59\x88\x8f\xa0\xd8\x66\xe5\x2e\x5c\x69\xfd\x55\x9c\xa5\x4d"
+ "\x30\xb0\x02\xa7\xc6\x15\xf9\xb7\x88\xc4\x46\x7a\x99\x4a\xf3\x93"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f",
+ .m_size = 64,
+ .algo = OID_gost2012PKey512,
+ .public_key_vec = true,
+ .siggen_sigver_test = true,
+ },
};
/*
diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
index 03cf3d63e609..045acf13c131 100644
--- a/drivers/acpi/apei/ghes.c
+++ b/drivers/acpi/apei/ghes.c
@@ -583,7 +583,7 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
* and don't filter out 'corrected' error here.
*/
if (is_cache && has_pa) {
- queued = ghes_do_memory_failure(err_info->physical_fault_addr, flags);
+ queued |= ghes_do_memory_failure(err_info->physical_fault_addr, flags);
p += err_info->length;
continue;
}
@@ -809,11 +809,11 @@ static bool ghes_do_proc(struct ghes *ghes,
atomic_notifier_call_chain(&ghes_report_chain, sev, mem_err);
arch_apei_report_mem_error(sev, mem_err);
- queued = ghes_handle_memory_failure(gdata, sev, sync);
+ queued |= ghes_handle_memory_failure(gdata, sev, sync);
} else if (guid_equal(sec_type, &CPER_SEC_PCIE)) {
ghes_handle_aer(gdata);
} else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) {
- queued = ghes_handle_arm_hw_error(gdata, sev, sync);
+ queued |= ghes_handle_arm_hw_error(gdata, sev, sync);
} else if (guid_equal(sec_type, &CPER_SEC_CXL_GEN_MEDIA_GUID)) {
struct cxl_cper_event_rec *rec = acpi_hest_get_payload(gdata);
diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
index 8ad6d12b7772..95ab801679e8 100644
--- a/drivers/acpi/cppc_acpi.c
+++ b/drivers/acpi/cppc_acpi.c
@@ -36,6 +36,7 @@
#include <linux/delay.h>
#include <linux/iopoll.h>
#include <linux/ktime.h>
+#include <linux/mutex.h>
#include <linux/rwsem.h>
#include <linux/wait.h>
#include <linux/topology.h>
@@ -94,6 +95,16 @@ static DEFINE_PER_CPU(int, cpu_pcc_subspace_idx);
*/
static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr);
+/* Protect immutable capability queries against descriptor removal. */
+static DEFINE_MUTEX(cpc_desc_lock);
+
+static void cpc_set_desc(unsigned int cpu, struct cpc_desc *desc)
+{
+ mutex_lock(&cpc_desc_lock);
+ per_cpu(cpc_desc_ptr, cpu) = desc;
+ mutex_unlock(&cpc_desc_lock);
+}
+
/* pcc mapped address + header size + offset within PCC subspace */
#define GET_PCC_VADDR(offs, pcc_ss_id) (pcc_data[pcc_ss_id]->pcc_comm_addr + \
0x8 + (offs))
@@ -910,12 +921,12 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr)
}
/* Plug PSD data into this CPU's CPC descriptor. */
- per_cpu(cpc_desc_ptr, pr->id) = cpc_ptr;
+ cpc_set_desc(pr->id, cpc_ptr);
ret = kobject_init_and_add(&cpc_ptr->kobj, &cppc_ktype, &cpu_dev->kobj,
"acpi_cppc");
if (ret) {
- per_cpu(cpc_desc_ptr, pr->id) = NULL;
+ cpc_set_desc(pr->id, NULL);
kobject_put(&cpc_ptr->kobj);
goto out_free;
}
@@ -967,6 +978,8 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr)
if (!cpc_ptr)
return;
+ cpc_set_desc(pr->id, NULL);
+
/* Free all the mapped sys mem areas for this CPU */
for (i = 2; i < cpc_ptr->num_entries; i++) {
addr = cpc_ptr->cpc_regs[i-2].sys_mem_vaddr;
@@ -1028,6 +1041,9 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
*val = 0;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_read_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
@@ -1054,8 +1070,6 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_read_ffh(cpu, reg, val);
else
return acpi_os_read_memory((acpi_physical_address)reg->address,
val, size);
@@ -1101,6 +1115,9 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
struct cpc_desc *cpc_desc;
unsigned long flags;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_write_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
@@ -1125,8 +1142,6 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_write_ffh(cpu, reg, val);
else
return acpi_os_write_memory((acpi_physical_address)reg->address,
val, size);
@@ -1276,6 +1291,11 @@ static int cppc_set_reg_val(int cpu, enum cppc_regs reg_idx, u64 val)
reg = &cpc_desc->cpc_regs[reg_idx];
+ /* Integer 1 describes autonomous selection that is always enabled. */
+ if (reg_idx == AUTO_SEL_ENABLE && reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1)
+ return val == 1 ? 0 : -EOPNOTSUPP;
+
/* if a register is writeable, it must be a buffer and not null */
if ((reg->type != ACPI_TYPE_BUFFER) || IS_NULL_REG(®->cpc_entry.reg)) {
pr_debug("CPC register is not supported\n");
@@ -1632,6 +1652,32 @@ int cppc_get_auto_sel(int cpu, bool *enable)
}
EXPORT_SYMBOL_GPL(cppc_get_auto_sel);
+/**
+ * cppc_auto_sel_is_immutable - Check for always-enabled autonomous selection.
+ * @cpu: CPU whose _CPC descriptor to check.
+ *
+ * Context: Process context.
+ * Return: true for Integer 1, false for a register or an absent descriptor.
+ */
+bool cppc_auto_sel_is_immutable(int cpu)
+{
+ struct cpc_desc *cpc_desc;
+ struct cpc_register_resource *reg;
+ bool immutable = false;
+
+ mutex_lock(&cpc_desc_lock);
+ cpc_desc = per_cpu(cpc_desc_ptr, cpu);
+ if (cpc_desc) {
+ reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE];
+ immutable = reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1;
+ }
+ mutex_unlock(&cpc_desc_lock);
+
+ return immutable;
+}
+EXPORT_SYMBOL_GPL(cppc_auto_sel_is_immutable);
+
/**
* cppc_set_auto_sel - Write autonomous selection register.
* @cpu : CPU to which to write register.
diff --git a/drivers/acpi/riscv/cppc.c b/drivers/acpi/riscv/cppc.c
index 10a2b554b3cc..ef3b052f9dad 100644
--- a/drivers/acpi/riscv/cppc.c
+++ b/drivers/acpi/riscv/cppc.c
@@ -99,6 +99,7 @@ bool cpc_ffh_supported(void)
int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
{
struct sbi_cppc_data data;
+ int ret;
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
@@ -109,19 +110,27 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
data.reg = FFH_CPPC_SBI_REG(reg->address);
- smp_call_function_single(cpu, sbi_cppc_read, &data, 1);
+ ret = smp_call_function_single(cpu, sbi_cppc_read, &data, 1);
+ if (ret)
+ return ret;
+ if (data.ret.error)
+ return sbi_err_map_linux_errno(data.ret.error);
*val = data.ret.value;
- return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0;
+ return 0;
} else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) {
data.reg = FFH_CPPC_CSR_NUM(reg->address);
- smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1);
+ ret = smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1);
+ if (ret)
+ return ret;
+ if (data.ret.error)
+ return data.ret.error;
*val = data.ret.value;
- return data.ret.error;
+ return 0;
}
return -EINVAL;
@@ -130,6 +139,7 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val)
{
struct sbi_cppc_data data;
+ int ret;
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
@@ -141,14 +151,18 @@ int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val)
data.reg = FFH_CPPC_SBI_REG(reg->address);
data.val = val;
- smp_call_function_single(cpu, sbi_cppc_write, &data, 1);
+ ret = smp_call_function_single(cpu, sbi_cppc_write, &data, 1);
+ if (ret)
+ return ret;
return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0;
} else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) {
data.reg = FFH_CPPC_CSR_NUM(reg->address);
data.val = val;
- smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1);
+ ret = smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1);
+ if (ret)
+ return ret;
return data.ret.error;
}
diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig
index 120a2b7067fc..ff81f707d0b2 100644
--- a/drivers/ata/Kconfig
+++ b/drivers/ata/Kconfig
@@ -329,13 +329,6 @@ config SATA_AHCI_SEATTLE
If unsure, say N
-config SATA_INIC162X
- tristate "Initio 162x SATA support (Very Experimental)"
- depends on PCI
- select SATA_HOST
- help
- This option enables support for Initio 162x Serial ATA.
-
config SATA_ACARD_AHCI
tristate "ACard AHCI variant (ATP 8620)"
depends on PCI
diff --git a/drivers/ata/Makefile b/drivers/ata/Makefile
index 20e6645ab737..81e04e96b807 100644
--- a/drivers/ata/Makefile
+++ b/drivers/ata/Makefile
@@ -9,7 +9,6 @@ obj-$(CONFIG_SATA_AHCI_SEATTLE) += ahci_seattle.o libahci.o libahci_platform.o
obj-$(CONFIG_SATA_AHCI_PLATFORM) += ahci_platform.o libahci.o libahci_platform.o
obj-$(CONFIG_SATA_FSL) += sata_fsl.o
obj-$(CONFIG_SATA_GEMINI) += sata_gemini.o
-obj-$(CONFIG_SATA_INIC162X) += sata_inic162x.o
obj-$(CONFIG_SATA_SIL24) += sata_sil24.o
obj-$(CONFIG_SATA_DWC) += sata_dwc_460ex.o
obj-$(CONFIG_SATA_HIGHBANK) += sata_highbank.o libahci.o
diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 85b7bd605d7d..bd543a5ee8c4 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -111,17 +111,17 @@ static const struct scsi_host_template ahci_sht = {
static struct ata_port_operations ahci_vt8251_ops = {
.inherits = &ahci_ops,
- .hardreset = ahci_vt8251_hardreset,
+ .reset.hardreset = ahci_vt8251_hardreset,
};
static struct ata_port_operations ahci_p5wdh_ops = {
.inherits = &ahci_ops,
- .hardreset = ahci_p5wdh_hardreset,
+ .reset.hardreset = ahci_p5wdh_hardreset,
};
static struct ata_port_operations ahci_avn_ops = {
.inherits = &ahci_ops,
- .hardreset = ahci_avn_hardreset,
+ .reset.hardreset = ahci_avn_hardreset,
};
static const struct ata_port_info ahci_port_info[] = {
diff --git a/drivers/ata/ahci_da850.c b/drivers/ata/ahci_da850.c
index 55a6627d5450..2b331770ac52 100644
--- a/drivers/ata/ahci_da850.c
+++ b/drivers/ata/ahci_da850.c
@@ -137,13 +137,13 @@ static int ahci_da850_hardreset(struct ata_link *link,
static struct ata_port_operations ahci_da850_port_ops = {
.inherits = &ahci_platform_ops,
- .softreset = ahci_da850_softreset,
+ .reset.softreset = ahci_da850_softreset,
/*
* No need to override .pmp_softreset - it's only used for actual
* PMP-enabled ports.
*/
- .hardreset = ahci_da850_hardreset,
- .pmp_hardreset = ahci_da850_hardreset,
+ .reset.hardreset = ahci_da850_hardreset,
+ .pmp_reset.hardreset = ahci_da850_hardreset,
};
static const struct ata_port_info ahci_da850_port_info = {
diff --git a/drivers/ata/ahci_dm816.c b/drivers/ata/ahci_dm816.c
index 4cb70064fb99..188d45716d3a 100644
--- a/drivers/ata/ahci_dm816.c
+++ b/drivers/ata/ahci_dm816.c
@@ -124,7 +124,7 @@ static int ahci_dm816_softreset(struct ata_link *link,
static struct ata_port_operations ahci_dm816_port_ops = {
.inherits = &ahci_platform_ops,
- .softreset = ahci_dm816_softreset,
+ .reset.softreset = ahci_dm816_softreset,
};
static const struct ata_port_info ahci_dm816_port_info = {
diff --git a/drivers/ata/ahci_imx.c b/drivers/ata/ahci_imx.c
index 6f955e9105e8..91c72a58f91c 100644
--- a/drivers/ata/ahci_imx.c
+++ b/drivers/ata/ahci_imx.c
@@ -642,18 +642,19 @@ static int ahci_imx_softreset(struct ata_link *link, unsigned int *class,
int ret;
if (imxpriv->type == AHCI_IMX53)
- ret = ahci_pmp_retry_srst_ops.softreset(link, class, deadline);
+ ret = ahci_pmp_retry_srst_ops.reset.softreset(link, class,
+ deadline);
else
- ret = ahci_ops.softreset(link, class, deadline);
+ ret = ahci_ops.reset.softreset(link, class, deadline);
return ret;
}
static struct ata_port_operations ahci_imx_ops = {
- .inherits = &ahci_ops,
- .host_stop = ahci_imx_host_stop,
- .error_handler = ahci_imx_error_handler,
- .softreset = ahci_imx_softreset,
+ .inherits = &ahci_ops,
+ .host_stop = ahci_imx_host_stop,
+ .error_handler = ahci_imx_error_handler,
+ .reset.softreset = ahci_imx_softreset,
};
static const struct ata_port_info ahci_imx_port_info = {
diff --git a/drivers/ata/ahci_qoriq.c b/drivers/ata/ahci_qoriq.c
index b1a4e57578e2..36e367a17d59 100644
--- a/drivers/ata/ahci_qoriq.c
+++ b/drivers/ata/ahci_qoriq.c
@@ -146,8 +146,8 @@ static int ahci_qoriq_hardreset(struct ata_link *link, unsigned int *class,
}
static struct ata_port_operations ahci_qoriq_ops = {
- .inherits = &ahci_ops,
- .hardreset = ahci_qoriq_hardreset,
+ .inherits = &ahci_ops,
+ .reset.hardreset = ahci_qoriq_hardreset,
};
static const struct ata_port_info ahci_qoriq_port_info = {
diff --git a/drivers/ata/ahci_xgene.c b/drivers/ata/ahci_xgene.c
index 81a1d838c0fc..d51cd210be13 100644
--- a/drivers/ata/ahci_xgene.c
+++ b/drivers/ata/ahci_xgene.c
@@ -613,11 +613,11 @@ static irqreturn_t xgene_ahci_irq_intr(int irq, void *dev_instance)
static struct ata_port_operations xgene_ahci_v1_ops = {
.inherits = &ahci_ops,
.host_stop = xgene_ahci_host_stop,
- .hardreset = xgene_ahci_hardreset,
+ .reset.hardreset = xgene_ahci_hardreset,
+ .reset.softreset = xgene_ahci_softreset,
+ .pmp_reset.softreset = xgene_ahci_pmp_softreset,
.read_id = xgene_ahci_read_id,
.qc_issue = xgene_ahci_qc_issue,
- .softreset = xgene_ahci_softreset,
- .pmp_softreset = xgene_ahci_pmp_softreset
};
static const struct ata_port_info xgene_ahci_v1_port_info = {
@@ -630,7 +630,7 @@ static const struct ata_port_info xgene_ahci_v1_port_info = {
static struct ata_port_operations xgene_ahci_v2_ops = {
.inherits = &ahci_ops,
.host_stop = xgene_ahci_host_stop,
- .hardreset = xgene_ahci_hardreset,
+ .reset.hardreset = xgene_ahci_hardreset,
.read_id = xgene_ahci_read_id,
};
diff --git a/drivers/ata/ata_piix.c b/drivers/ata/ata_piix.c
index d3cda803ae06..a78b662d3322 100644
--- a/drivers/ata/ata_piix.c
+++ b/drivers/ata/ata_piix.c
@@ -1074,7 +1074,7 @@ static struct ata_port_operations piix_pata_ops = {
.cable_detect = ata_cable_40wire,
.set_piomode = piix_set_piomode,
.set_dmamode = piix_set_dmamode,
- .prereset = piix_pata_prereset,
+ .reset.prereset = piix_pata_prereset,
};
static struct ata_port_operations piix_vmw_ops = {
@@ -1103,7 +1103,7 @@ static const struct scsi_host_template piix_sidpr_sht = {
static struct ata_port_operations piix_sidpr_sata_ops = {
.inherits = &piix_sata_ops,
- .hardreset = sata_std_hardreset,
+ .reset.hardreset = sata_std_hardreset,
.scr_read = piix_sidpr_scr_read,
.scr_write = piix_sidpr_scr_write,
.set_lpm = piix_sidpr_set_lpm,
diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
index 3fd193fcbd6e..cefb5013186e 100644
--- a/drivers/ata/libahci.c
+++ b/drivers/ata/libahci.c
@@ -163,10 +163,10 @@ struct ata_port_operations ahci_ops = {
.freeze = ahci_freeze,
.thaw = ahci_thaw,
- .softreset = ahci_softreset,
- .hardreset = ahci_hardreset,
- .postreset = ahci_postreset,
- .pmp_softreset = ahci_softreset,
+ .reset.softreset = ahci_softreset,
+ .reset.hardreset = ahci_hardreset,
+ .reset.postreset = ahci_postreset,
+ .pmp_reset.softreset = ahci_softreset,
.error_handler = ahci_error_handler,
.post_internal_cmd = ahci_post_internal_cmd,
.dev_config = ahci_dev_config,
@@ -193,7 +193,7 @@ EXPORT_SYMBOL_GPL(ahci_ops);
struct ata_port_operations ahci_pmp_retry_srst_ops = {
.inherits = &ahci_ops,
- .softreset = ahci_pmp_retry_softreset,
+ .reset.softreset = ahci_pmp_retry_softreset,
};
EXPORT_SYMBOL_GPL(ahci_pmp_retry_srst_ops);
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 0db6821cc3aa..65fb2cfc0e29 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -65,8 +65,8 @@
#include "libata-transport.h"
const struct ata_port_operations ata_base_port_ops = {
- .prereset = ata_std_prereset,
- .postreset = ata_std_postreset,
+ .reset.prereset = ata_std_prereset,
+ .reset.postreset = ata_std_postreset,
.error_handler = ata_std_error_handler,
.sched_eh = ata_std_sched_eh,
.end_eh = ata_std_end_eh,
diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c
index 32ec1cfd648e..b78aa5371c14 100644
--- a/drivers/ata/libata-eh.c
+++ b/drivers/ata/libata-eh.c
@@ -2632,14 +2632,16 @@ static int ata_eh_followup_srst_needed(struct ata_link *link, int rc)
return 0;
}
-int ata_eh_reset(struct ata_link *link, int classify,
- ata_prereset_fn_t prereset, ata_reset_fn_t softreset,
- ata_reset_fn_t hardreset, ata_postreset_fn_t postreset)
+int ata_eh_reset(struct ata_port *ap, struct ata_link *link, int classify,
+ struct ata_reset_operations *reset_ops)
{
- struct ata_port *ap = link->ap;
struct ata_link *slave = ap->slave_link;
struct ata_eh_context *ehc = &link->eh_context;
struct ata_eh_context *sehc = slave ? &slave->eh_context : NULL;
+ ata_reset_fn_t hardreset = reset_ops->hardreset;
+ ata_reset_fn_t softreset = reset_ops->softreset;
+ ata_prereset_fn_t prereset = reset_ops->prereset;
+ ata_postreset_fn_t postreset = reset_ops->postreset;
unsigned int *classes = ehc->classes;
unsigned int lflags = link->flags;
int verbose = !(ehc->i.flags & ATA_EHI_QUIET);
@@ -3248,12 +3250,12 @@ static int ata_eh_revalidate_and_attach(struct ata_link *link,
}
/**
- * ata_set_mode - Program timings and issue SET FEATURES - XFER
+ * ata_eh_set_mode - Program timings and issue SET FEATURES - XFER
* @link: link on which timings will be programmed
* @r_failed_dev: out parameter for failed device
*
* Set ATA device disk transfer mode (PIO3, UDMA6, etc.). If
- * ata_set_mode() fails, pointer to the failing device is
+ * ata_eh_set_mode() fails, pointer to the failing device is
* returned in @r_failed_dev.
*
* LOCKING:
@@ -3262,7 +3264,8 @@ static int ata_eh_revalidate_and_attach(struct ata_link *link,
* RETURNS:
* 0 on success, negative errno otherwise
*/
-int ata_set_mode(struct ata_link *link, struct ata_device **r_failed_dev)
+static int ata_eh_set_mode(struct ata_link *link,
+ struct ata_device **r_failed_dev)
{
struct ata_port *ap = link->ap;
struct ata_device *dev;
@@ -3729,10 +3732,7 @@ static int ata_eh_handle_dev_fail(struct ata_device *dev, int err)
/**
* ata_eh_recover - recover host port after error
* @ap: host port to recover
- * @prereset: prereset method (can be NULL)
- * @softreset: softreset method (can be NULL)
- * @hardreset: hardreset method (can be NULL)
- * @postreset: postreset method (can be NULL)
+ * @reset_ops: The set of reset operations to use
* @r_failed_link: out parameter for failed link
*
* This is the alpha and omega, eum and yang, heart and soul of
@@ -3748,9 +3748,7 @@ static int ata_eh_handle_dev_fail(struct ata_device *dev, int err)
* RETURNS:
* 0 on success, -errno on failure.
*/
-int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset,
- ata_reset_fn_t softreset, ata_reset_fn_t hardreset,
- ata_postreset_fn_t postreset,
+int ata_eh_recover(struct ata_port *ap, struct ata_reset_operations *reset_ops,
struct ata_link **r_failed_link)
{
struct ata_link *link;
@@ -3818,8 +3816,8 @@ int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset,
if (!(ehc->i.action & ATA_EH_RESET))
continue;
- rc = ata_eh_reset(link, ata_link_nr_vacant(link),
- prereset, softreset, hardreset, postreset);
+ rc = ata_eh_reset(ap, link, ata_link_nr_vacant(link),
+ reset_ops);
if (rc) {
ata_link_err(link, "reset failed, giving up\n");
goto out;
@@ -3900,7 +3898,7 @@ int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset,
/* configure transfer mode if necessary */
if (ehc->i.flags & ATA_EHI_SETMODE) {
- rc = ata_set_mode(link, &dev);
+ rc = ata_eh_set_mode(link, &dev);
if (rc)
goto rest_fail;
ehc->i.flags &= ~ATA_EHI_SETMODE;
@@ -4039,59 +4037,39 @@ void ata_eh_finish(struct ata_port *ap)
}
/**
- * ata_do_eh - do standard error handling
+ * ata_std_error_handler - standard error handler
* @ap: host port to handle error for
*
- * @prereset: prereset method (can be NULL)
- * @softreset: softreset method (can be NULL)
- * @hardreset: hardreset method (can be NULL)
- * @postreset: postreset method (can be NULL)
- *
* Perform standard error handling sequence.
*
* LOCKING:
* Kernel thread context (may sleep).
*/
-void ata_do_eh(struct ata_port *ap, ata_prereset_fn_t prereset,
- ata_reset_fn_t softreset, ata_reset_fn_t hardreset,
- ata_postreset_fn_t postreset)
+void ata_std_error_handler(struct ata_port *ap)
{
- struct ata_device *dev;
+ struct ata_reset_operations *reset_ops = &ap->ops->reset;
+ struct ata_link *link = &ap->link;
int rc;
+ /* Ignore built-in hardresets if SCR access is not available */
+ if ((reset_ops->hardreset == sata_std_hardreset ||
+ reset_ops->hardreset == sata_sff_hardreset) &&
+ !sata_scr_valid(link))
+ link->flags |= ATA_LFLAG_NO_HRST;
+
ata_eh_autopsy(ap);
ata_eh_report(ap);
- rc = ata_eh_recover(ap, prereset, softreset, hardreset, postreset,
- NULL);
+ rc = ata_eh_recover(ap, reset_ops, NULL);
if (rc) {
- ata_for_each_dev(dev, &ap->link, ALL)
+ struct ata_device *dev;
+
+ ata_for_each_dev(dev, link, ALL)
ata_dev_disable(dev);
}
ata_eh_finish(ap);
}
-
-/**
- * ata_std_error_handler - standard error handler
- * @ap: host port to handle error for
- *
- * Standard error handler
- *
- * LOCKING:
- * Kernel thread context (may sleep).
- */
-void ata_std_error_handler(struct ata_port *ap)
-{
- struct ata_port_operations *ops = ap->ops;
- ata_reset_fn_t hardreset = ops->hardreset;
-
- /* ignore built-in hardreset if SCR access is not available */
- if (hardreset == sata_std_hardreset && !sata_scr_valid(&ap->link))
- hardreset = NULL;
-
- ata_do_eh(ap, ops->prereset, ops->softreset, hardreset, ops->postreset);
-}
EXPORT_SYMBOL_GPL(ata_std_error_handler);
#ifdef CONFIG_PM
diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c
index 20dd3a5a818a..c3b3af28d17c 100644
--- a/drivers/ata/libata-pmp.c
+++ b/drivers/ata/libata-pmp.c
@@ -15,9 +15,9 @@
const struct ata_port_operations sata_pmp_port_ops = {
.inherits = &sata_port_ops,
- .pmp_prereset = ata_std_prereset,
- .pmp_hardreset = sata_std_hardreset,
- .pmp_postreset = ata_std_postreset,
+ .pmp_reset.prereset = ata_std_prereset,
+ .pmp_reset.hardreset = sata_std_hardreset,
+ .pmp_reset.postreset = ata_std_postreset,
.error_handler = sata_pmp_error_handler,
};
@@ -482,6 +482,20 @@ static void sata_pmp_quirks(struct ata_port *ap)
if (link->pmp == 4)
link->flags |= ATA_LFLAG_DISABLED;
}
+ } else if (vendor == 0x1b4b && devid == 0x9705) {
+ /*
+ * Marvell 88SM9705 quirks
+ *
+ * 1-to-5 PMP with an extra SEMB port. It reports 6 ports and
+ * port 5 is the SEMB, which never answers SRST: EH spends about
+ * a minute per round retrying it, and disk discovery waits for
+ * that. Seen behind a Marvell 88SE9235 in the Synology DX1215
+ * and DX1222.
+ */
+ ata_for_each_link(link, ap, EDGE) {
+ if (link->pmp == 5)
+ link->flags |= ATA_LFLAG_DISABLED;
+ }
}
}
@@ -746,10 +760,7 @@ static int sata_pmp_revalidate_quick(struct ata_device *dev)
/**
* sata_pmp_eh_recover_pmp - recover PMP
* @ap: ATA port PMP is attached to
- * @prereset: prereset method (can be NULL)
- * @softreset: softreset method
- * @hardreset: hardreset method
- * @postreset: postreset method (can be NULL)
+ * @reset_ops: The set of reset operations to use
*
* Recover PMP attached to @ap. Recovery procedure is somewhat
* similar to that of ata_eh_recover() except that reset should
@@ -763,8 +774,7 @@ static int sata_pmp_revalidate_quick(struct ata_device *dev)
* 0 on success, -errno on failure.
*/
static int sata_pmp_eh_recover_pmp(struct ata_port *ap,
- ata_prereset_fn_t prereset, ata_reset_fn_t softreset,
- ata_reset_fn_t hardreset, ata_postreset_fn_t postreset)
+ struct ata_reset_operations *reset_ops)
{
struct ata_link *link = &ap->link;
struct ata_eh_context *ehc = &link->eh_context;
@@ -786,8 +796,7 @@ static int sata_pmp_eh_recover_pmp(struct ata_port *ap,
struct ata_link *tlink;
/* reset */
- rc = ata_eh_reset(link, 0, prereset, softreset, hardreset,
- postreset);
+ rc = ata_eh_reset(ap, link, 0, reset_ops);
if (rc) {
ata_link_err(link, "failed to reset PMP, giving up\n");
goto fail;
@@ -951,8 +960,7 @@ static int sata_pmp_eh_recover(struct ata_port *ap)
retry:
/* PMP attached? */
if (!sata_pmp_attached(ap)) {
- rc = ata_eh_recover(ap, ops->prereset, ops->softreset,
- ops->hardreset, ops->postreset, NULL);
+ rc = ata_eh_recover(ap, &ops->reset, NULL);
if (rc) {
ata_for_each_dev(dev, &ap->link, ALL)
ata_dev_disable(dev);
@@ -970,8 +978,7 @@ static int sata_pmp_eh_recover(struct ata_port *ap)
}
/* recover pmp */
- rc = sata_pmp_eh_recover_pmp(ap, ops->prereset, ops->softreset,
- ops->hardreset, ops->postreset);
+ rc = sata_pmp_eh_recover_pmp(ap, &ops->reset);
if (rc)
goto pmp_fail;
@@ -997,8 +1004,7 @@ static int sata_pmp_eh_recover(struct ata_port *ap)
goto pmp_fail;
/* recover links */
- rc = ata_eh_recover(ap, ops->pmp_prereset, ops->pmp_softreset,
- ops->pmp_hardreset, ops->pmp_postreset, &link);
+ rc = ata_eh_recover(ap, &ops->pmp_reset, &link);
if (rc)
goto link_fail;
diff --git a/drivers/ata/libata-sata.c b/drivers/ata/libata-sata.c
index fc64fae9e23f..0b6cfdfc2759 100644
--- a/drivers/ata/libata-sata.c
+++ b/drivers/ata/libata-sata.c
@@ -1735,6 +1735,6 @@ const struct ata_port_operations sata_port_ops = {
.inherits = &ata_base_port_ops,
.qc_defer = ata_std_qc_defer,
- .hardreset = sata_std_hardreset,
+ .reset.hardreset = sata_std_hardreset,
};
EXPORT_SYMBOL_GPL(sata_port_ops);
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 619819262414..0e309707f882 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1989,6 +1989,18 @@ static int ata_scsi_translate(struct ata_device *dev, struct scsi_cmnd *cmd,
if (xlat_func(qc))
goto done;
+ /*
+ * A data-out request may contain pages pinned without write access.
+ * Reject a translated data-in command before issuing it, since its
+ * completion handler may write to the buffer even if the device fails.
+ */
+ if (unlikely(cmd->sc_data_direction == DMA_TO_DEVICE &&
+ ata_is_data(qc->tf.protocol) &&
+ !(qc->tf.flags & ATA_TFLAG_WRITE))) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ goto done;
+ }
+
return ata_scsi_qc_issue(ap, qc);
done:
@@ -2020,6 +2032,12 @@ static void ata_scsi_rbuf_fill(struct ata_device *dev, struct scsi_cmnd *cmd,
unsigned long flags;
unsigned int len;
+ /* A data-out request may contain pages pinned without write access. */
+ if (cmd->sc_data_direction == DMA_TO_DEVICE) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return;
+ }
+
spin_lock_irqsave(&ata_scsi_rbuf_lock, flags);
memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE);
@@ -3077,6 +3095,13 @@ static unsigned int atapi_xlat(struct ata_queued_cmd *qc)
qc->complete_fn = atapi_qc_complete;
+ /* atapi_fixup_inquiry() writes to the buffer on successful completion. */
+ if (scmd->sc_data_direction == DMA_TO_DEVICE &&
+ scmd->cmnd[0] == INQUIRY) {
+ ata_scsi_set_sense(dev, scmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return 1;
+ }
+
qc->tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
if (scmd->sc_data_direction == DMA_TO_DEVICE) {
qc->tf.flags |= ATA_TFLAG_WRITE;
diff --git a/drivers/ata/libata-sff.c b/drivers/ata/libata-sff.c
index 83cb44ca5a97..95a3ddf36eff 100644
--- a/drivers/ata/libata-sff.c
+++ b/drivers/ata/libata-sff.c
@@ -31,10 +31,10 @@ const struct ata_port_operations ata_sff_port_ops = {
.freeze = ata_sff_freeze,
.thaw = ata_sff_thaw,
- .prereset = ata_sff_prereset,
- .softreset = ata_sff_softreset,
- .hardreset = sata_sff_hardreset,
- .postreset = ata_sff_postreset,
+ .reset.prereset = ata_sff_prereset,
+ .reset.softreset = ata_sff_softreset,
+ .reset.hardreset = sata_sff_hardreset,
+ .reset.postreset = ata_sff_postreset,
.error_handler = ata_sff_error_handler,
.sff_dev_select = ata_sff_dev_select,
@@ -2054,8 +2054,6 @@ EXPORT_SYMBOL_GPL(ata_sff_drain_fifo);
*/
void ata_sff_error_handler(struct ata_port *ap)
{
- ata_reset_fn_t softreset = ap->ops->softreset;
- ata_reset_fn_t hardreset = ap->ops->hardreset;
struct ata_queued_cmd *qc;
unsigned long flags;
@@ -2077,13 +2075,7 @@ void ata_sff_error_handler(struct ata_port *ap)
spin_unlock_irqrestore(ap->lock, flags);
- /* ignore built-in hardresets if SCR access is not available */
- if ((hardreset == sata_std_hardreset ||
- hardreset == sata_sff_hardreset) && !sata_scr_valid(&ap->link))
- hardreset = NULL;
-
- ata_do_eh(ap, ap->ops->prereset, softreset, hardreset,
- ap->ops->postreset);
+ ata_std_error_handler(ap);
}
EXPORT_SYMBOL_GPL(ata_sff_error_handler);
diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h
index 4b3e246227e6..f98a3eb5d51e 100644
--- a/drivers/ata/libata.h
+++ b/drivers/ata/libata.h
@@ -196,13 +196,10 @@ extern void ata_eh_done(struct ata_link *link, struct ata_device *dev,
extern void ata_eh_autopsy(struct ata_port *ap);
const char *ata_get_cmd_name(u8 command);
extern void ata_eh_report(struct ata_port *ap);
-extern int ata_eh_reset(struct ata_link *link, int classify,
- ata_prereset_fn_t prereset, ata_reset_fn_t softreset,
- ata_reset_fn_t hardreset, ata_postreset_fn_t postreset);
-extern int ata_set_mode(struct ata_link *link, struct ata_device **r_failed_dev);
-extern int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset,
- ata_reset_fn_t softreset, ata_reset_fn_t hardreset,
- ata_postreset_fn_t postreset,
+extern int ata_eh_reset(struct ata_port *ap, struct ata_link *link,
+ int classify, struct ata_reset_operations *reset_ops);
+extern int ata_eh_recover(struct ata_port *ap,
+ struct ata_reset_operations *reset_ops,
struct ata_link **r_failed_disk);
extern void ata_eh_finish(struct ata_port *ap);
extern int ata_ering_map(struct ata_ering *ering,
diff --git a/drivers/ata/pata_acpi.c b/drivers/ata/pata_acpi.c
index ab38871b5e00..23fff10af2ac 100644
--- a/drivers/ata/pata_acpi.c
+++ b/drivers/ata/pata_acpi.c
@@ -216,7 +216,7 @@ static struct ata_port_operations pacpi_ops = {
.mode_filter = pacpi_mode_filter,
.set_piomode = pacpi_set_piomode,
.set_dmamode = pacpi_set_dmamode,
- .prereset = pacpi_pre_reset,
+ .reset.prereset = pacpi_pre_reset,
.port_start = pacpi_port_start,
};
diff --git a/drivers/ata/pata_ali.c b/drivers/ata/pata_ali.c
index bb790edd6036..9d5cb9c34c52 100644
--- a/drivers/ata/pata_ali.c
+++ b/drivers/ata/pata_ali.c
@@ -392,11 +392,11 @@ static struct ata_port_operations ali_20_port_ops = {
* Port operations for DMA capable ALi with cable detect
*/
static struct ata_port_operations ali_c2_port_ops = {
- .inherits = &ali_dma_base_ops,
- .check_atapi_dma = ali_check_atapi_dma,
- .cable_detect = ali_c2_cable_detect,
- .dev_config = ali_lock_sectors,
- .postreset = ali_c2_c3_postreset,
+ .inherits = &ali_dma_base_ops,
+ .check_atapi_dma = ali_check_atapi_dma,
+ .cable_detect = ali_c2_cable_detect,
+ .dev_config = ali_lock_sectors,
+ .reset.postreset = ali_c2_c3_postreset,
};
/*
diff --git a/drivers/ata/pata_amd.c b/drivers/ata/pata_amd.c
index 5b02b89748b7..a2fecadc927d 100644
--- a/drivers/ata/pata_amd.c
+++ b/drivers/ata/pata_amd.c
@@ -394,7 +394,7 @@ static const struct scsi_host_template amd_sht = {
static const struct ata_port_operations amd_base_port_ops = {
.inherits = &ata_bmdma32_port_ops,
- .prereset = amd_pre_reset,
+ .reset.prereset = amd_pre_reset,
};
static struct ata_port_operations amd33_port_ops = {
@@ -429,7 +429,7 @@ static const struct ata_port_operations nv_base_port_ops = {
.inherits = &ata_bmdma_port_ops,
.cable_detect = ata_cable_ignore,
.mode_filter = nv_mode_filter,
- .prereset = nv_pre_reset,
+ .reset.prereset = nv_pre_reset,
.host_stop = nv_host_stop,
};
diff --git a/drivers/ata/pata_artop.c b/drivers/ata/pata_artop.c
index 40544282f455..6160414172a3 100644
--- a/drivers/ata/pata_artop.c
+++ b/drivers/ata/pata_artop.c
@@ -301,7 +301,7 @@ static struct ata_port_operations artop6210_ops = {
.cable_detect = ata_cable_40wire,
.set_piomode = artop6210_set_piomode,
.set_dmamode = artop6210_set_dmamode,
- .prereset = artop62x0_pre_reset,
+ .reset.prereset = artop62x0_pre_reset,
.qc_defer = artop6210_qc_defer,
};
@@ -310,7 +310,7 @@ static struct ata_port_operations artop6260_ops = {
.cable_detect = artop6260_cable_detect,
.set_piomode = artop6260_set_piomode,
.set_dmamode = artop6260_set_dmamode,
- .prereset = artop62x0_pre_reset,
+ .reset.prereset = artop62x0_pre_reset,
};
static void atp8xx_fixup(struct pci_dev *pdev)
diff --git a/drivers/ata/pata_atiixp.c b/drivers/ata/pata_atiixp.c
index 8c5cc803aab3..4c612f9543f6 100644
--- a/drivers/ata/pata_atiixp.c
+++ b/drivers/ata/pata_atiixp.c
@@ -264,7 +264,7 @@ static struct ata_port_operations atiixp_port_ops = {
.bmdma_start = atiixp_bmdma_start,
.bmdma_stop = atiixp_bmdma_stop,
- .prereset = atiixp_prereset,
+ .reset.prereset = atiixp_prereset,
.cable_detect = atiixp_cable_detect,
.set_piomode = atiixp_set_piomode,
.set_dmamode = atiixp_set_dmamode,
diff --git a/drivers/ata/pata_efar.c b/drivers/ata/pata_efar.c
index 2e6eccf2902f..6fe49b303fee 100644
--- a/drivers/ata/pata_efar.c
+++ b/drivers/ata/pata_efar.c
@@ -243,7 +243,7 @@ static struct ata_port_operations efar_ops = {
.cable_detect = efar_cable_detect,
.set_piomode = efar_set_piomode,
.set_dmamode = efar_set_dmamode,
- .prereset = efar_pre_reset,
+ .reset.prereset = efar_pre_reset,
};
diff --git a/drivers/ata/pata_ep93xx.c b/drivers/ata/pata_ep93xx.c
index 3ee975309791..f469c40a8916 100644
--- a/drivers/ata/pata_ep93xx.c
+++ b/drivers/ata/pata_ep93xx.c
@@ -891,8 +891,8 @@ static const struct scsi_host_template ep93xx_pata_sht = {
static struct ata_port_operations ep93xx_pata_port_ops = {
.inherits = &ata_bmdma_port_ops,
- .softreset = ep93xx_pata_softreset,
- .hardreset = ATA_OP_NULL,
+ .reset.softreset = ep93xx_pata_softreset,
+ .reset.hardreset = ATA_OP_NULL,
.sff_dev_select = ep93xx_pata_dev_select,
.sff_set_devctl = ep93xx_pata_set_devctl,
diff --git a/drivers/ata/pata_hpt366.c b/drivers/ata/pata_hpt366.c
index 5280e9960025..b96e8bd2a3f8 100644
--- a/drivers/ata/pata_hpt366.c
+++ b/drivers/ata/pata_hpt366.c
@@ -322,7 +322,7 @@ static const struct scsi_host_template hpt36x_sht = {
static struct ata_port_operations hpt366_port_ops = {
.inherits = &ata_bmdma_port_ops,
- .prereset = hpt366_prereset,
+ .reset.prereset = hpt366_prereset,
.cable_detect = hpt36x_cable_detect,
.mode_filter = hpt366_filter,
.set_piomode = hpt366_set_piomode,
diff --git a/drivers/ata/pata_hpt37x.c b/drivers/ata/pata_hpt37x.c
index 4af22b819416..07e3a984cbb1 100644
--- a/drivers/ata/pata_hpt37x.c
+++ b/drivers/ata/pata_hpt37x.c
@@ -543,7 +543,7 @@ static struct ata_port_operations hpt370_port_ops = {
.cable_detect = hpt37x_cable_detect,
.set_piomode = hpt37x_set_piomode,
.set_dmamode = hpt37x_set_dmamode,
- .prereset = hpt37x_pre_reset,
+ .reset.prereset = hpt37x_pre_reset,
};
/*
@@ -567,7 +567,7 @@ static struct ata_port_operations hpt302_port_ops = {
.cable_detect = hpt37x_cable_detect,
.set_piomode = hpt37x_set_piomode,
.set_dmamode = hpt37x_set_dmamode,
- .prereset = hpt37x_pre_reset,
+ .reset.prereset = hpt37x_pre_reset,
};
/*
diff --git a/drivers/ata/pata_hpt3x2n.c b/drivers/ata/pata_hpt3x2n.c
index 5b1ecccf3c83..2cc57fcf2c46 100644
--- a/drivers/ata/pata_hpt3x2n.c
+++ b/drivers/ata/pata_hpt3x2n.c
@@ -356,7 +356,7 @@ static struct ata_port_operations hpt3xxn_port_ops = {
.cable_detect = hpt3x2n_cable_detect,
.set_piomode = hpt3x2n_set_piomode,
.set_dmamode = hpt3x2n_set_dmamode,
- .prereset = hpt3x2n_pre_reset,
+ .reset.prereset = hpt3x2n_pre_reset,
};
/*
diff --git a/drivers/ata/pata_icside.c b/drivers/ata/pata_icside.c
index 61d8760f09d9..70f056e47e6b 100644
--- a/drivers/ata/pata_icside.c
+++ b/drivers/ata/pata_icside.c
@@ -336,7 +336,7 @@ static struct ata_port_operations pata_icside_port_ops = {
.cable_detect = ata_cable_40wire,
.set_dmamode = pata_icside_set_dmamode,
- .postreset = pata_icside_postreset,
+ .reset.postreset = pata_icside_postreset,
.port_start = ATA_OP_NULL, /* don't need PRD table */
};
diff --git a/drivers/ata/pata_it8213.c b/drivers/ata/pata_it8213.c
index b7ac56103c8a..5ea75933cb8b 100644
--- a/drivers/ata/pata_it8213.c
+++ b/drivers/ata/pata_it8213.c
@@ -238,7 +238,7 @@ static struct ata_port_operations it8213_ops = {
.cable_detect = it8213_cable_detect,
.set_piomode = it8213_set_piomode,
.set_dmamode = it8213_set_dmamode,
- .prereset = it8213_pre_reset,
+ .reset.prereset = it8213_pre_reset,
};
diff --git a/drivers/ata/pata_jmicron.c b/drivers/ata/pata_jmicron.c
index f51fb8219762..b885f33e8980 100644
--- a/drivers/ata/pata_jmicron.c
+++ b/drivers/ata/pata_jmicron.c
@@ -113,7 +113,7 @@ static const struct scsi_host_template jmicron_sht = {
static struct ata_port_operations jmicron_ops = {
.inherits = &ata_bmdma_port_ops,
- .prereset = jmicron_pre_reset,
+ .reset.prereset = jmicron_pre_reset,
};
diff --git a/drivers/ata/pata_marvell.c b/drivers/ata/pata_marvell.c
index 8119caaad605..deab67328388 100644
--- a/drivers/ata/pata_marvell.c
+++ b/drivers/ata/pata_marvell.c
@@ -99,7 +99,7 @@ static const struct scsi_host_template marvell_sht = {
static struct ata_port_operations marvell_ops = {
.inherits = &ata_bmdma_port_ops,
.cable_detect = marvell_cable_detect,
- .prereset = marvell_pre_reset,
+ .reset.prereset = marvell_pre_reset,
};
diff --git a/drivers/ata/pata_mpiix.c b/drivers/ata/pata_mpiix.c
index 69e4baf27d72..ce310ae7c93a 100644
--- a/drivers/ata/pata_mpiix.c
+++ b/drivers/ata/pata_mpiix.c
@@ -145,7 +145,7 @@ static struct ata_port_operations mpiix_port_ops = {
.qc_issue = mpiix_qc_issue,
.cable_detect = ata_cable_40wire,
.set_piomode = mpiix_set_piomode,
- .prereset = mpiix_pre_reset,
+ .reset.prereset = mpiix_pre_reset,
.sff_data_xfer = ata_sff_data_xfer32,
};
diff --git a/drivers/ata/pata_ns87410.c b/drivers/ata/pata_ns87410.c
index 44cc24d21d5f..bdb55c1a3280 100644
--- a/drivers/ata/pata_ns87410.c
+++ b/drivers/ata/pata_ns87410.c
@@ -123,7 +123,7 @@ static struct ata_port_operations ns87410_port_ops = {
.qc_issue = ns87410_qc_issue,
.cable_detect = ata_cable_40wire,
.set_piomode = ns87410_set_piomode,
- .prereset = ns87410_pre_reset,
+ .reset.prereset = ns87410_pre_reset,
};
static int ns87410_init_one(struct pci_dev *dev, const struct pci_device_id *id)
diff --git a/drivers/ata/pata_octeon_cf.c b/drivers/ata/pata_octeon_cf.c
index 0bb9607e7348..dc61a3004ba9 100644
--- a/drivers/ata/pata_octeon_cf.c
+++ b/drivers/ata/pata_octeon_cf.c
@@ -942,7 +942,7 @@ static int octeon_cf_probe(struct platform_device *pdev)
/* 16 bit but not True IDE */
base = cs0 + 0x800;
octeon_cf_ops.sff_data_xfer = octeon_cf_data_xfer16;
- octeon_cf_ops.softreset = octeon_cf_softreset16;
+ octeon_cf_ops.reset.softreset = octeon_cf_softreset16;
octeon_cf_ops.sff_check_status = octeon_cf_check_status16;
octeon_cf_ops.sff_tf_read = octeon_cf_tf_read16;
octeon_cf_ops.sff_tf_load = octeon_cf_tf_load16;
diff --git a/drivers/ata/pata_oldpiix.c b/drivers/ata/pata_oldpiix.c
index dca82d92b004..659f27aa3464 100644
--- a/drivers/ata/pata_oldpiix.c
+++ b/drivers/ata/pata_oldpiix.c
@@ -214,7 +214,7 @@ static struct ata_port_operations oldpiix_pata_ops = {
.cable_detect = ata_cable_40wire,
.set_piomode = oldpiix_set_piomode,
.set_dmamode = oldpiix_set_dmamode,
- .prereset = oldpiix_pre_reset,
+ .reset.prereset = oldpiix_pre_reset,
};
diff --git a/drivers/ata/pata_opti.c b/drivers/ata/pata_opti.c
index 3d23f57eb128..3db1b95d1404 100644
--- a/drivers/ata/pata_opti.c
+++ b/drivers/ata/pata_opti.c
@@ -156,7 +156,7 @@ static struct ata_port_operations opti_port_ops = {
.inherits = &ata_sff_port_ops,
.cable_detect = ata_cable_40wire,
.set_piomode = opti_set_piomode,
- .prereset = opti_pre_reset,
+ .reset.prereset = opti_pre_reset,
};
static int opti_init_one(struct pci_dev *dev, const struct pci_device_id *id)
diff --git a/drivers/ata/pata_optidma.c b/drivers/ata/pata_optidma.c
index dfc36b4ec9c6..50124e205ece 100644
--- a/drivers/ata/pata_optidma.c
+++ b/drivers/ata/pata_optidma.c
@@ -344,7 +344,7 @@ static struct ata_port_operations optidma_port_ops = {
.set_piomode = optidma_set_pio_mode,
.set_dmamode = optidma_set_dma_mode,
.set_mode = optidma_set_mode,
- .prereset = optidma_pre_reset,
+ .reset.prereset = optidma_pre_reset,
};
static struct ata_port_operations optiplus_port_ops = {
diff --git a/drivers/ata/pata_parport/pata_parport.c b/drivers/ata/pata_parport/pata_parport.c
index 93ebf566b54e..22bd3ff6b7ae 100644
--- a/drivers/ata/pata_parport/pata_parport.c
+++ b/drivers/ata/pata_parport/pata_parport.c
@@ -321,8 +321,8 @@ static void pata_parport_drain_fifo(struct ata_queued_cmd *qc)
static struct ata_port_operations pata_parport_port_ops = {
.inherits = &ata_sff_port_ops,
- .softreset = pata_parport_softreset,
- .hardreset = NULL,
+ .reset.softreset = pata_parport_softreset,
+ .reset.hardreset = NULL,
.sff_dev_select = pata_parport_dev_select,
.sff_set_devctl = pata_parport_set_devctl,
diff --git a/drivers/ata/pata_pdc2027x.c b/drivers/ata/pata_pdc2027x.c
index 6820c5597b14..29df7d501b94 100644
--- a/drivers/ata/pata_pdc2027x.c
+++ b/drivers/ata/pata_pdc2027x.c
@@ -130,7 +130,7 @@ static struct ata_port_operations pdc2027x_pata100_ops = {
.inherits = &ata_bmdma_port_ops,
.check_atapi_dma = pdc2027x_check_atapi_dma,
.cable_detect = pdc2027x_cable_detect,
- .prereset = pdc2027x_prereset,
+ .reset.prereset = pdc2027x_prereset,
};
static struct ata_port_operations pdc2027x_pata133_ops = {
diff --git a/drivers/ata/pata_rdc.c b/drivers/ata/pata_rdc.c
index 0a9689862f71..6d8b276c979d 100644
--- a/drivers/ata/pata_rdc.c
+++ b/drivers/ata/pata_rdc.c
@@ -276,7 +276,7 @@ static struct ata_port_operations rdc_pata_ops = {
.cable_detect = rdc_pata_cable_detect,
.set_piomode = rdc_set_piomode,
.set_dmamode = rdc_set_dmamode,
- .prereset = rdc_pata_prereset,
+ .reset.prereset = rdc_pata_prereset,
};
static const struct ata_port_info rdc_port_info = {
diff --git a/drivers/ata/pata_sis.c b/drivers/ata/pata_sis.c
index 31de06b66221..2b751e393771 100644
--- a/drivers/ata/pata_sis.c
+++ b/drivers/ata/pata_sis.c
@@ -552,7 +552,7 @@ static struct ata_port_operations sis_133_for_sata_ops = {
static struct ata_port_operations sis_base_ops = {
.inherits = &ata_bmdma_port_ops,
- .prereset = sis_pre_reset,
+ .reset.prereset = sis_pre_reset,
};
static struct ata_port_operations sis_133_ops = {
diff --git a/drivers/ata/pata_sl82c105.c b/drivers/ata/pata_sl82c105.c
index 2a5ce46af5a0..25bce3fe2a10 100644
--- a/drivers/ata/pata_sl82c105.c
+++ b/drivers/ata/pata_sl82c105.c
@@ -248,7 +248,7 @@ static struct ata_port_operations sl82c105_port_ops = {
.bmdma_stop = sl82c105_bmdma_stop,
.cable_detect = ata_cable_40wire,
.set_piomode = sl82c105_set_piomode,
- .prereset = sl82c105_pre_reset,
+ .reset.prereset = sl82c105_pre_reset,
.sff_irq_check = sl82c105_sff_irq_check,
};
diff --git a/drivers/ata/pata_triflex.c b/drivers/ata/pata_triflex.c
index 26d448a869e2..596e86a031b3 100644
--- a/drivers/ata/pata_triflex.c
+++ b/drivers/ata/pata_triflex.c
@@ -170,7 +170,7 @@ static struct ata_port_operations triflex_port_ops = {
.bmdma_stop = triflex_bmdma_stop,
.cable_detect = ata_cable_40wire,
.set_piomode = triflex_set_piomode,
- .prereset = triflex_prereset,
+ .reset.prereset = triflex_prereset,
};
static int triflex_init_one(struct pci_dev *dev, const struct pci_device_id *id)
diff --git a/drivers/ata/pata_via.c b/drivers/ata/pata_via.c
index bb80e7800dcb..a8c9cf685b4b 100644
--- a/drivers/ata/pata_via.c
+++ b/drivers/ata/pata_via.c
@@ -451,7 +451,7 @@ static struct ata_port_operations via_port_ops = {
.cable_detect = via_cable_detect,
.set_piomode = via_set_piomode,
.set_dmamode = via_set_dmamode,
- .prereset = via_pre_reset,
+ .reset.prereset = via_pre_reset,
.sff_tf_load = via_tf_load,
.port_start = via_port_start,
.mode_filter = via_mode_filter,
diff --git a/drivers/ata/pdc_adma.c b/drivers/ata/pdc_adma.c
index 8e6b2599f0d5..17a5a59861c3 100644
--- a/drivers/ata/pdc_adma.c
+++ b/drivers/ata/pdc_adma.c
@@ -140,7 +140,7 @@ static struct ata_port_operations adma_ata_ops = {
.freeze = adma_freeze,
.thaw = adma_thaw,
- .prereset = adma_prereset,
+ .reset.prereset = adma_prereset,
.port_start = adma_port_start,
.port_stop = adma_port_stop,
diff --git a/drivers/ata/sata_dwc_460ex.c b/drivers/ata/sata_dwc_460ex.c
index 66db8dd8ec26..23753c4ae1b5 100644
--- a/drivers/ata/sata_dwc_460ex.c
+++ b/drivers/ata/sata_dwc_460ex.c
@@ -1087,7 +1087,7 @@ static struct ata_port_operations sata_dwc_ops = {
.inherits = &ata_sff_port_ops,
.error_handler = sata_dwc_error_handler,
- .hardreset = sata_dwc_hardreset,
+ .reset.hardreset = sata_dwc_hardreset,
.qc_issue = sata_dwc_qc_issue,
diff --git a/drivers/ata/sata_fsl.c b/drivers/ata/sata_fsl.c
index c64c9d99cd6c..56247db5d1d2 100644
--- a/drivers/ata/sata_fsl.c
+++ b/drivers/ata/sata_fsl.c
@@ -1395,9 +1395,9 @@ static struct ata_port_operations sata_fsl_ops = {
.freeze = sata_fsl_freeze,
.thaw = sata_fsl_thaw,
- .softreset = sata_fsl_softreset,
- .hardreset = sata_fsl_hardreset,
- .pmp_softreset = sata_fsl_softreset,
+ .reset.softreset = sata_fsl_softreset,
+ .reset.hardreset = sata_fsl_hardreset,
+ .pmp_reset.softreset = sata_fsl_softreset,
.error_handler = sata_fsl_error_handler,
.post_internal_cmd = sata_fsl_post_internal_cmd,
diff --git a/drivers/ata/sata_highbank.c b/drivers/ata/sata_highbank.c
index 596c6d294da9..6349026ed027 100644
--- a/drivers/ata/sata_highbank.c
+++ b/drivers/ata/sata_highbank.c
@@ -428,7 +428,7 @@ static int ahci_highbank_hardreset(struct ata_link *link, unsigned int *class,
static struct ata_port_operations ahci_highbank_ops = {
.inherits = &ahci_ops,
- .hardreset = ahci_highbank_hardreset,
+ .reset.hardreset = ahci_highbank_hardreset,
.transmit_led_message = ecx_transmit_led_message,
};
diff --git a/drivers/ata/sata_inic162x.c b/drivers/ata/sata_inic162x.c
deleted file mode 100644
index db9c255dc9f2..000000000000
--- a/drivers/ata/sata_inic162x.c
+++ /dev/null
@@ -1,902 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * sata_inic162x.c - Driver for Initio 162x SATA controllers
- *
- * Copyright 2006 SUSE Linux Products GmbH
- * Copyright 2006 Tejun Heo <teheo@novell.com>
- *
- * **** WARNING ****
- *
- * This driver never worked properly and unfortunately data corruption is
- * relatively common. There isn't anyone working on the driver and there's
- * no support from the vendor. Do not use this driver in any production
- * environment.
- *
- * http://thread.gmane.org/gmane.linux.debian.devel.bugs.rc/378525/focus=54491
- * https://bugzilla.kernel.org/show_bug.cgi?id=60565
- *
- * *****************
- *
- * This controller is eccentric and easily locks up if something isn't
- * right. Documentation is available at initio's website but it only
- * documents registers (not programming model).
- *
- * This driver has interesting history. The first version was written
- * from the documentation and a 2.4 IDE driver posted on a Taiwan
- * company, which didn't use any IDMA features and couldn't handle
- * LBA48. The resulting driver couldn't handle LBA48 devices either
- * making it pretty useless.
- *
- * After a while, initio picked the driver up, renamed it to
- * sata_initio162x, updated it to use IDMA for ATA DMA commands and
- * posted it on their website. It only used ATA_PROT_DMA for IDMA and
- * attaching both devices and issuing IDMA and !IDMA commands
- * simultaneously broke it due to PIRQ masking interaction but it did
- * show how to use the IDMA (ADMA + some initio specific twists)
- * engine.
- *
- * Then, I picked up their changes again and here's the usable driver
- * which uses IDMA for everything. Everything works now including
- * LBA48, CD/DVD burning, suspend/resume and hotplug. There are some
- * issues tho. Result Tf is not resported properly, NCQ isn't
- * supported yet and CD/DVD writing works with DMA assisted PIO
- * protocol (which, for native SATA devices, shouldn't cause any
- * noticeable difference).
- *
- * Anyways, so, here's finally a working driver for inic162x. Enjoy!
- *
- * initio: If you guys wanna improve the driver regarding result TF
- * access and other stuff, please feel free to contact me. I'll be
- * happy to assist.
- */
-
-#include <linux/gfp.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/pci.h>
-#include <scsi/scsi_host.h>
-#include <linux/libata.h>
-#include <linux/blkdev.h>
-#include <scsi/scsi_device.h>
-
-#define DRV_NAME "sata_inic162x"
-#define DRV_VERSION "0.4"
-
-enum {
- MMIO_BAR_PCI = 5,
- MMIO_BAR_CARDBUS = 1,
-
- NR_PORTS = 2,
-
- IDMA_CPB_TBL_SIZE = 4 * 32,
-
- INIC_DMA_BOUNDARY = 0xffffff,
-
- HOST_ACTRL = 0x08,
- HOST_CTL = 0x7c,
- HOST_STAT = 0x7e,
- HOST_IRQ_STAT = 0xbc,
- HOST_IRQ_MASK = 0xbe,
-
- PORT_SIZE = 0x40,
-
- /* registers for ATA TF operation */
- PORT_TF_DATA = 0x00,
- PORT_TF_FEATURE = 0x01,
- PORT_TF_NSECT = 0x02,
- PORT_TF_LBAL = 0x03,
- PORT_TF_LBAM = 0x04,
- PORT_TF_LBAH = 0x05,
- PORT_TF_DEVICE = 0x06,
- PORT_TF_COMMAND = 0x07,
- PORT_TF_ALT_STAT = 0x08,
- PORT_IRQ_STAT = 0x09,
- PORT_IRQ_MASK = 0x0a,
- PORT_PRD_CTL = 0x0b,
- PORT_PRD_ADDR = 0x0c,
- PORT_PRD_XFERLEN = 0x10,
- PORT_CPB_CPBLAR = 0x18,
- PORT_CPB_PTQFIFO = 0x1c,
-
- /* IDMA register */
- PORT_IDMA_CTL = 0x14,
- PORT_IDMA_STAT = 0x16,
-
- PORT_RPQ_FIFO = 0x1e,
- PORT_RPQ_CNT = 0x1f,
-
- PORT_SCR = 0x20,
-
- /* HOST_CTL bits */
- HCTL_LEDEN = (1 << 3), /* enable LED operation */
- HCTL_IRQOFF = (1 << 8), /* global IRQ off */
- HCTL_FTHD0 = (1 << 10), /* fifo threshold 0 */
- HCTL_FTHD1 = (1 << 11), /* fifo threshold 1*/
- HCTL_PWRDWN = (1 << 12), /* power down PHYs */
- HCTL_SOFTRST = (1 << 13), /* global reset (no phy reset) */
- HCTL_RPGSEL = (1 << 15), /* register page select */
-
- HCTL_KNOWN_BITS = HCTL_IRQOFF | HCTL_PWRDWN | HCTL_SOFTRST |
- HCTL_RPGSEL,
-
- /* HOST_IRQ_(STAT|MASK) bits */
- HIRQ_PORT0 = (1 << 0),
- HIRQ_PORT1 = (1 << 1),
- HIRQ_SOFT = (1 << 14),
- HIRQ_GLOBAL = (1 << 15), /* STAT only */
-
- /* PORT_IRQ_(STAT|MASK) bits */
- PIRQ_OFFLINE = (1 << 0), /* device unplugged */
- PIRQ_ONLINE = (1 << 1), /* device plugged */
- PIRQ_COMPLETE = (1 << 2), /* completion interrupt */
- PIRQ_FATAL = (1 << 3), /* fatal error */
- PIRQ_ATA = (1 << 4), /* ATA interrupt */
- PIRQ_REPLY = (1 << 5), /* reply FIFO not empty */
- PIRQ_PENDING = (1 << 7), /* port IRQ pending (STAT only) */
-
- PIRQ_ERR = PIRQ_OFFLINE | PIRQ_ONLINE | PIRQ_FATAL,
- PIRQ_MASK_DEFAULT = PIRQ_REPLY | PIRQ_ATA,
- PIRQ_MASK_FREEZE = 0xff,
-
- /* PORT_PRD_CTL bits */
- PRD_CTL_START = (1 << 0),
- PRD_CTL_WR = (1 << 3),
- PRD_CTL_DMAEN = (1 << 7), /* DMA enable */
-
- /* PORT_IDMA_CTL bits */
- IDMA_CTL_RST_ATA = (1 << 2), /* hardreset ATA bus */
- IDMA_CTL_RST_IDMA = (1 << 5), /* reset IDMA machinery */
- IDMA_CTL_GO = (1 << 7), /* IDMA mode go */
- IDMA_CTL_ATA_NIEN = (1 << 8), /* ATA IRQ disable */
-
- /* PORT_IDMA_STAT bits */
- IDMA_STAT_PERR = (1 << 0), /* PCI ERROR MODE */
- IDMA_STAT_CPBERR = (1 << 1), /* ADMA CPB error */
- IDMA_STAT_LGCY = (1 << 3), /* ADMA legacy */
- IDMA_STAT_UIRQ = (1 << 4), /* ADMA unsolicited irq */
- IDMA_STAT_STPD = (1 << 5), /* ADMA stopped */
- IDMA_STAT_PSD = (1 << 6), /* ADMA pause */
- IDMA_STAT_DONE = (1 << 7), /* ADMA done */
-
- IDMA_STAT_ERR = IDMA_STAT_PERR | IDMA_STAT_CPBERR,
-
- /* CPB Control Flags*/
- CPB_CTL_VALID = (1 << 0), /* CPB valid */
- CPB_CTL_QUEUED = (1 << 1), /* queued command */
- CPB_CTL_DATA = (1 << 2), /* data, rsvd in datasheet */
- CPB_CTL_IEN = (1 << 3), /* PCI interrupt enable */
- CPB_CTL_DEVDIR = (1 << 4), /* device direction control */
-
- /* CPB Response Flags */
- CPB_RESP_DONE = (1 << 0), /* ATA command complete */
- CPB_RESP_REL = (1 << 1), /* ATA release */
- CPB_RESP_IGNORED = (1 << 2), /* CPB ignored */
- CPB_RESP_ATA_ERR = (1 << 3), /* ATA command error */
- CPB_RESP_SPURIOUS = (1 << 4), /* ATA spurious interrupt error */
- CPB_RESP_UNDERFLOW = (1 << 5), /* APRD deficiency length error */
- CPB_RESP_OVERFLOW = (1 << 6), /* APRD exccess length error */
- CPB_RESP_CPB_ERR = (1 << 7), /* CPB error flag */
-
- /* PRD Control Flags */
- PRD_DRAIN = (1 << 1), /* ignore data excess */
- PRD_CDB = (1 << 2), /* atapi packet command pointer */
- PRD_DIRECT_INTR = (1 << 3), /* direct interrupt */
- PRD_DMA = (1 << 4), /* data transfer method */
- PRD_WRITE = (1 << 5), /* data dir, rsvd in datasheet */
- PRD_IOM = (1 << 6), /* io/memory transfer */
- PRD_END = (1 << 7), /* APRD chain end */
-};
-
-/* Comman Parameter Block */
-struct inic_cpb {
- u8 resp_flags; /* Response Flags */
- u8 error; /* ATA Error */
- u8 status; /* ATA Status */
- u8 ctl_flags; /* Control Flags */
- __le32 len; /* Total Transfer Length */
- __le32 prd; /* First PRD pointer */
- u8 rsvd[4];
- /* 16 bytes */
- u8 feature; /* ATA Feature */
- u8 hob_feature; /* ATA Ex. Feature */
- u8 device; /* ATA Device/Head */
- u8 mirctl; /* Mirror Control */
- u8 nsect; /* ATA Sector Count */
- u8 hob_nsect; /* ATA Ex. Sector Count */
- u8 lbal; /* ATA Sector Number */
- u8 hob_lbal; /* ATA Ex. Sector Number */
- u8 lbam; /* ATA Cylinder Low */
- u8 hob_lbam; /* ATA Ex. Cylinder Low */
- u8 lbah; /* ATA Cylinder High */
- u8 hob_lbah; /* ATA Ex. Cylinder High */
- u8 command; /* ATA Command */
- u8 ctl; /* ATA Control */
- u8 slave_error; /* Slave ATA Error */
- u8 slave_status; /* Slave ATA Status */
- /* 32 bytes */
-} __packed;
-
-/* Physical Region Descriptor */
-struct inic_prd {
- __le32 mad; /* Physical Memory Address */
- __le16 len; /* Transfer Length */
- u8 rsvd;
- u8 flags; /* Control Flags */
-} __packed;
-
-struct inic_pkt {
- struct inic_cpb cpb;
- struct inic_prd prd[LIBATA_MAX_PRD + 1]; /* + 1 for cdb */
- u8 cdb[ATAPI_CDB_LEN];
-} __packed;
-
-struct inic_host_priv {
- void __iomem *mmio_base;
- u16 cached_hctl;
-};
-
-struct inic_port_priv {
- struct inic_pkt *pkt;
- dma_addr_t pkt_dma;
- u32 *cpb_tbl;
- dma_addr_t cpb_tbl_dma;
-};
-
-static const struct scsi_host_template inic_sht = {
- ATA_BASE_SHT(DRV_NAME),
- .sg_tablesize = LIBATA_MAX_PRD, /* maybe it can be larger? */
-
- /*
- * This controller is braindamaged. dma_boundary is 0xffff like others
- * but it will lock up the whole machine HARD if 65536 byte PRD entry
- * is fed. Reduce maximum segment size.
- */
- .dma_boundary = INIC_DMA_BOUNDARY,
- .max_segment_size = 65536 - 512,
-};
-
-static const int scr_map[] = {
- [SCR_STATUS] = 0,
- [SCR_ERROR] = 1,
- [SCR_CONTROL] = 2,
-};
-
-static void __iomem *inic_port_base(struct ata_port *ap)
-{
- struct inic_host_priv *hpriv = ap->host->private_data;
-
- return hpriv->mmio_base + ap->port_no * PORT_SIZE;
-}
-
-static void inic_reset_port(void __iomem *port_base)
-{
- void __iomem *idma_ctl = port_base + PORT_IDMA_CTL;
-
- /* stop IDMA engine */
- readw(idma_ctl); /* flush */
- msleep(1);
-
- /* mask IRQ and assert reset */
- writew(IDMA_CTL_RST_IDMA, idma_ctl);
- readw(idma_ctl); /* flush */
- msleep(1);
-
- /* release reset */
- writew(0, idma_ctl);
-
- /* clear irq */
- writeb(0xff, port_base + PORT_IRQ_STAT);
-}
-
-static int inic_scr_read(struct ata_link *link, unsigned sc_reg, u32 *val)
-{
- void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR;
-
- if (unlikely(sc_reg >= ARRAY_SIZE(scr_map)))
- return -EINVAL;
-
- *val = readl(scr_addr + scr_map[sc_reg] * 4);
-
- /* this controller has stuck DIAG.N, ignore it */
- if (sc_reg == SCR_ERROR)
- *val &= ~SERR_PHYRDY_CHG;
- return 0;
-}
-
-static int inic_scr_write(struct ata_link *link, unsigned sc_reg, u32 val)
-{
- void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR;
-
- if (unlikely(sc_reg >= ARRAY_SIZE(scr_map)))
- return -EINVAL;
-
- writel(val, scr_addr + scr_map[sc_reg] * 4);
- return 0;
-}
-
-static void inic_stop_idma(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- readb(port_base + PORT_RPQ_FIFO);
- readb(port_base + PORT_RPQ_CNT);
- writew(0, port_base + PORT_IDMA_CTL);
-}
-
-static void inic_host_err_intr(struct ata_port *ap, u8 irq_stat, u16 idma_stat)
-{
- struct ata_eh_info *ehi = &ap->link.eh_info;
- struct inic_port_priv *pp = ap->private_data;
- struct inic_cpb *cpb = &pp->pkt->cpb;
- bool freeze = false;
-
- ata_ehi_clear_desc(ehi);
- ata_ehi_push_desc(ehi, "irq_stat=0x%x idma_stat=0x%x",
- irq_stat, idma_stat);
-
- inic_stop_idma(ap);
-
- if (irq_stat & (PIRQ_OFFLINE | PIRQ_ONLINE)) {
- ata_ehi_push_desc(ehi, "hotplug");
- ata_ehi_hotplugged(ehi);
- freeze = true;
- }
-
- if (idma_stat & IDMA_STAT_PERR) {
- ata_ehi_push_desc(ehi, "PCI error");
- freeze = true;
- }
-
- if (idma_stat & IDMA_STAT_CPBERR) {
- ata_ehi_push_desc(ehi, "CPB error");
-
- if (cpb->resp_flags & CPB_RESP_IGNORED) {
- __ata_ehi_push_desc(ehi, " ignored");
- ehi->err_mask |= AC_ERR_INVALID;
- freeze = true;
- }
-
- if (cpb->resp_flags & CPB_RESP_ATA_ERR)
- ehi->err_mask |= AC_ERR_DEV;
-
- if (cpb->resp_flags & CPB_RESP_SPURIOUS) {
- __ata_ehi_push_desc(ehi, " spurious-intr");
- ehi->err_mask |= AC_ERR_HSM;
- freeze = true;
- }
-
- if (cpb->resp_flags &
- (CPB_RESP_UNDERFLOW | CPB_RESP_OVERFLOW)) {
- __ata_ehi_push_desc(ehi, " data-over/underflow");
- ehi->err_mask |= AC_ERR_HSM;
- freeze = true;
- }
- }
-
- if (freeze)
- ata_port_freeze(ap);
- else
- ata_port_abort(ap);
-}
-
-static void inic_host_intr(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
- struct ata_queued_cmd *qc = ata_qc_from_tag(ap, ap->link.active_tag);
- u8 irq_stat;
- u16 idma_stat;
-
- /* read and clear IRQ status */
- irq_stat = readb(port_base + PORT_IRQ_STAT);
- writeb(irq_stat, port_base + PORT_IRQ_STAT);
- idma_stat = readw(port_base + PORT_IDMA_STAT);
-
- if (unlikely((irq_stat & PIRQ_ERR) || (idma_stat & IDMA_STAT_ERR)))
- inic_host_err_intr(ap, irq_stat, idma_stat);
-
- if (unlikely(!qc))
- goto spurious;
-
- if (likely(idma_stat & IDMA_STAT_DONE)) {
- inic_stop_idma(ap);
-
- /* Depending on circumstances, device error
- * isn't reported by IDMA, check it explicitly.
- */
- if (unlikely(readb(port_base + PORT_TF_COMMAND) &
- (ATA_DF | ATA_ERR)))
- qc->err_mask |= AC_ERR_DEV;
-
- ata_qc_complete(qc);
- return;
- }
-
- spurious:
- ata_port_warn(ap, "unhandled interrupt: cmd=0x%x irq_stat=0x%x idma_stat=0x%x\n",
- qc ? qc->tf.command : 0xff, irq_stat, idma_stat);
-}
-
-static irqreturn_t inic_interrupt(int irq, void *dev_instance)
-{
- struct ata_host *host = dev_instance;
- struct inic_host_priv *hpriv = host->private_data;
- u16 host_irq_stat;
- int i, handled = 0;
-
- host_irq_stat = readw(hpriv->mmio_base + HOST_IRQ_STAT);
-
- if (unlikely(!(host_irq_stat & HIRQ_GLOBAL)))
- goto out;
-
- spin_lock(&host->lock);
-
- for (i = 0; i < NR_PORTS; i++)
- if (host_irq_stat & (HIRQ_PORT0 << i)) {
- inic_host_intr(host->ports[i]);
- handled++;
- }
-
- spin_unlock(&host->lock);
-
- out:
- return IRQ_RETVAL(handled);
-}
-
-static int inic_check_atapi_dma(struct ata_queued_cmd *qc)
-{
- /* For some reason ATAPI_PROT_DMA doesn't work for some
- * commands including writes and other misc ops. Use PIO
- * protocol instead, which BTW is driven by the DMA engine
- * anyway, so it shouldn't make much difference for native
- * SATA devices.
- */
- if (atapi_cmd_type(qc->cdb[0]) == READ)
- return 0;
- return 1;
-}
-
-static void inic_fill_sg(struct inic_prd *prd, struct ata_queued_cmd *qc)
-{
- struct scatterlist *sg;
- unsigned int si;
- u8 flags = 0;
-
- if (qc->tf.flags & ATA_TFLAG_WRITE)
- flags |= PRD_WRITE;
-
- if (ata_is_dma(qc->tf.protocol))
- flags |= PRD_DMA;
-
- for_each_sg(qc->sg, sg, qc->n_elem, si) {
- prd->mad = cpu_to_le32(sg_dma_address(sg));
- prd->len = cpu_to_le16(sg_dma_len(sg));
- prd->flags = flags;
- prd++;
- }
-
- WARN_ON(!si);
- prd[-1].flags |= PRD_END;
-}
-
-static enum ata_completion_errors inic_qc_prep(struct ata_queued_cmd *qc)
-{
- struct inic_port_priv *pp = qc->ap->private_data;
- struct inic_pkt *pkt = pp->pkt;
- struct inic_cpb *cpb = &pkt->cpb;
- struct inic_prd *prd = pkt->prd;
- bool is_atapi = ata_is_atapi(qc->tf.protocol);
- bool is_data = ata_is_data(qc->tf.protocol);
- unsigned int cdb_len = 0;
-
- if (is_atapi)
- cdb_len = qc->dev->cdb_len;
-
- /* prepare packet, based on initio driver */
- memset(pkt, 0, sizeof(struct inic_pkt));
-
- cpb->ctl_flags = CPB_CTL_VALID | CPB_CTL_IEN;
- if (is_atapi || is_data)
- cpb->ctl_flags |= CPB_CTL_DATA;
-
- cpb->len = cpu_to_le32(qc->nbytes + cdb_len);
- cpb->prd = cpu_to_le32(pp->pkt_dma + offsetof(struct inic_pkt, prd));
-
- cpb->device = qc->tf.device;
- cpb->feature = qc->tf.feature;
- cpb->nsect = qc->tf.nsect;
- cpb->lbal = qc->tf.lbal;
- cpb->lbam = qc->tf.lbam;
- cpb->lbah = qc->tf.lbah;
-
- if (qc->tf.flags & ATA_TFLAG_LBA48) {
- cpb->hob_feature = qc->tf.hob_feature;
- cpb->hob_nsect = qc->tf.hob_nsect;
- cpb->hob_lbal = qc->tf.hob_lbal;
- cpb->hob_lbam = qc->tf.hob_lbam;
- cpb->hob_lbah = qc->tf.hob_lbah;
- }
-
- cpb->command = qc->tf.command;
- /* don't load ctl - dunno why. it's like that in the initio driver */
-
- /* setup PRD for CDB */
- if (is_atapi) {
- memcpy(pkt->cdb, qc->cdb, ATAPI_CDB_LEN);
- prd->mad = cpu_to_le32(pp->pkt_dma +
- offsetof(struct inic_pkt, cdb));
- prd->len = cpu_to_le16(cdb_len);
- prd->flags = PRD_CDB | PRD_WRITE;
- if (!is_data)
- prd->flags |= PRD_END;
- prd++;
- }
-
- /* setup sg table */
- if (is_data)
- inic_fill_sg(prd, qc);
-
- pp->cpb_tbl[0] = pp->pkt_dma;
-
- return AC_ERR_OK;
-}
-
-static unsigned int inic_qc_issue(struct ata_queued_cmd *qc)
-{
- struct ata_port *ap = qc->ap;
- void __iomem *port_base = inic_port_base(ap);
-
- /* fire up the ADMA engine */
- writew(HCTL_FTHD0 | HCTL_LEDEN, port_base + HOST_CTL);
- writew(IDMA_CTL_GO, port_base + PORT_IDMA_CTL);
- writeb(0, port_base + PORT_CPB_PTQFIFO);
-
- return 0;
-}
-
-static void inic_tf_read(struct ata_port *ap, struct ata_taskfile *tf)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- tf->error = readb(port_base + PORT_TF_FEATURE);
- tf->nsect = readb(port_base + PORT_TF_NSECT);
- tf->lbal = readb(port_base + PORT_TF_LBAL);
- tf->lbam = readb(port_base + PORT_TF_LBAM);
- tf->lbah = readb(port_base + PORT_TF_LBAH);
- tf->device = readb(port_base + PORT_TF_DEVICE);
- tf->status = readb(port_base + PORT_TF_COMMAND);
-}
-
-static void inic_qc_fill_rtf(struct ata_queued_cmd *qc)
-{
- struct ata_taskfile *rtf = &qc->result_tf;
- struct ata_taskfile tf;
-
- /* FIXME: Except for status and error, result TF access
- * doesn't work. I tried reading from BAR0/2, CPB and BAR5.
- * None works regardless of which command interface is used.
- * For now return true iff status indicates device error.
- * This means that we're reporting bogus sector for RW
- * failures. Eeekk....
- */
- inic_tf_read(qc->ap, &tf);
-
- if (tf.status & ATA_ERR) {
- rtf->status = tf.status;
- rtf->error = tf.error;
- }
-}
-
-static void inic_freeze(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- writeb(PIRQ_MASK_FREEZE, port_base + PORT_IRQ_MASK);
- writeb(0xff, port_base + PORT_IRQ_STAT);
-}
-
-static void inic_thaw(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- writeb(0xff, port_base + PORT_IRQ_STAT);
- writeb(PIRQ_MASK_DEFAULT, port_base + PORT_IRQ_MASK);
-}
-
-static int inic_check_ready(struct ata_link *link)
-{
- void __iomem *port_base = inic_port_base(link->ap);
-
- return ata_check_ready(readb(port_base + PORT_TF_COMMAND));
-}
-
-/*
- * SRST and SControl hardreset don't give valid signature on this
- * controller. Only controller specific hardreset mechanism works.
- */
-static int inic_hardreset(struct ata_link *link, unsigned int *class,
- unsigned long deadline)
-{
- struct ata_port *ap = link->ap;
- void __iomem *port_base = inic_port_base(ap);
- void __iomem *idma_ctl = port_base + PORT_IDMA_CTL;
- const unsigned int *timing = sata_ehc_deb_timing(&link->eh_context);
- int rc;
-
- /* hammer it into sane state */
- inic_reset_port(port_base);
-
- writew(IDMA_CTL_RST_ATA, idma_ctl);
- readw(idma_ctl); /* flush */
- ata_msleep(ap, 1);
- writew(0, idma_ctl);
-
- rc = sata_link_resume(link, timing, deadline);
- if (rc) {
- ata_link_warn(link,
- "failed to resume link after reset (errno=%d)\n",
- rc);
- return rc;
- }
-
- *class = ATA_DEV_NONE;
- if (ata_link_online(link)) {
- struct ata_taskfile tf;
-
- /* wait for link to become ready */
- rc = ata_wait_after_reset(link, deadline, inic_check_ready);
- /* link occupied, -ENODEV too is an error */
- if (rc) {
- ata_link_warn(link,
- "device not ready after hardreset (errno=%d)\n",
- rc);
- return rc;
- }
-
- inic_tf_read(ap, &tf);
- *class = ata_port_classify(ap, &tf);
- }
-
- return 0;
-}
-
-static void inic_error_handler(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- inic_reset_port(port_base);
- ata_std_error_handler(ap);
-}
-
-static void inic_post_internal_cmd(struct ata_queued_cmd *qc)
-{
- /* make DMA engine forget about the failed command */
- if (qc->flags & ATA_QCFLAG_EH)
- inic_reset_port(inic_port_base(qc->ap));
-}
-
-static void init_port(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
- struct inic_port_priv *pp = ap->private_data;
-
- /* clear packet and CPB table */
- memset(pp->pkt, 0, sizeof(struct inic_pkt));
- memset(pp->cpb_tbl, 0, IDMA_CPB_TBL_SIZE);
-
- /* setup CPB lookup table addresses */
- writel(pp->cpb_tbl_dma, port_base + PORT_CPB_CPBLAR);
-}
-
-static int inic_port_resume(struct ata_port *ap)
-{
- init_port(ap);
- return 0;
-}
-
-static int inic_port_start(struct ata_port *ap)
-{
- struct device *dev = ap->host->dev;
- struct inic_port_priv *pp;
-
- /* alloc and initialize private data */
- pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
- if (!pp)
- return -ENOMEM;
- ap->private_data = pp;
-
- /* Alloc resources */
- pp->pkt = dmam_alloc_coherent(dev, sizeof(struct inic_pkt),
- &pp->pkt_dma, GFP_KERNEL);
- if (!pp->pkt)
- return -ENOMEM;
-
- pp->cpb_tbl = dmam_alloc_coherent(dev, IDMA_CPB_TBL_SIZE,
- &pp->cpb_tbl_dma, GFP_KERNEL);
- if (!pp->cpb_tbl)
- return -ENOMEM;
-
- init_port(ap);
-
- return 0;
-}
-
-static struct ata_port_operations inic_port_ops = {
- .inherits = &sata_port_ops,
-
- .check_atapi_dma = inic_check_atapi_dma,
- .qc_prep = inic_qc_prep,
- .qc_issue = inic_qc_issue,
- .qc_fill_rtf = inic_qc_fill_rtf,
-
- .freeze = inic_freeze,
- .thaw = inic_thaw,
- .hardreset = inic_hardreset,
- .error_handler = inic_error_handler,
- .post_internal_cmd = inic_post_internal_cmd,
-
- .scr_read = inic_scr_read,
- .scr_write = inic_scr_write,
-
- .port_resume = inic_port_resume,
- .port_start = inic_port_start,
-};
-
-static const struct ata_port_info inic_port_info = {
- .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA,
- .pio_mask = ATA_PIO4,
- .mwdma_mask = ATA_MWDMA2,
- .udma_mask = ATA_UDMA6,
- .port_ops = &inic_port_ops
-};
-
-static int init_controller(void __iomem *mmio_base, u16 hctl)
-{
- int i;
- u16 val;
-
- hctl &= ~HCTL_KNOWN_BITS;
-
- /* Soft reset whole controller. Spec says reset duration is 3
- * PCI clocks, be generous and give it 10ms.
- */
- writew(hctl | HCTL_SOFTRST, mmio_base + HOST_CTL);
- readw(mmio_base + HOST_CTL); /* flush */
-
- for (i = 0; i < 10; i++) {
- msleep(1);
- val = readw(mmio_base + HOST_CTL);
- if (!(val & HCTL_SOFTRST))
- break;
- }
-
- if (val & HCTL_SOFTRST)
- return -EIO;
-
- /* mask all interrupts and reset ports */
- for (i = 0; i < NR_PORTS; i++) {
- void __iomem *port_base = mmio_base + i * PORT_SIZE;
-
- writeb(0xff, port_base + PORT_IRQ_MASK);
- inic_reset_port(port_base);
- }
-
- /* port IRQ is masked now, unmask global IRQ */
- writew(hctl & ~HCTL_IRQOFF, mmio_base + HOST_CTL);
- val = readw(mmio_base + HOST_IRQ_MASK);
- val &= ~(HIRQ_PORT0 | HIRQ_PORT1);
- writew(val, mmio_base + HOST_IRQ_MASK);
-
- return 0;
-}
-
-#ifdef CONFIG_PM_SLEEP
-static int inic_pci_device_resume(struct pci_dev *pdev)
-{
- struct ata_host *host = pci_get_drvdata(pdev);
- struct inic_host_priv *hpriv = host->private_data;
- int rc;
-
- rc = ata_pci_device_do_resume(pdev);
- if (rc)
- return rc;
-
- if (pdev->dev.power.power_state.event == PM_EVENT_SUSPEND) {
- rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl);
- if (rc)
- return rc;
- }
-
- ata_host_resume(host);
-
- return 0;
-}
-#endif
-
-static int inic_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
-{
- const struct ata_port_info *ppi[] = { &inic_port_info, NULL };
- struct ata_host *host;
- struct inic_host_priv *hpriv;
- void __iomem * const *iomap;
- int mmio_bar;
- int i, rc;
-
- ata_print_version_once(&pdev->dev, DRV_VERSION);
-
- dev_alert(&pdev->dev, "inic162x support is broken with common data corruption issues and will be disabled by default, contact linux-ide@vger.kernel.org if in production use\n");
-
- /* alloc host */
- host = ata_host_alloc_pinfo(&pdev->dev, ppi, NR_PORTS);
- hpriv = devm_kzalloc(&pdev->dev, sizeof(*hpriv), GFP_KERNEL);
- if (!host || !hpriv)
- return -ENOMEM;
-
- host->private_data = hpriv;
-
- /* Acquire resources and fill host. Note that PCI and cardbus
- * use different BARs.
- */
- rc = pcim_enable_device(pdev);
- if (rc)
- return rc;
-
- if (pci_resource_flags(pdev, MMIO_BAR_PCI) & IORESOURCE_MEM)
- mmio_bar = MMIO_BAR_PCI;
- else
- mmio_bar = MMIO_BAR_CARDBUS;
-
- rc = pcim_iomap_regions(pdev, 1 << mmio_bar, DRV_NAME);
- if (rc)
- return rc;
- host->iomap = iomap = pcim_iomap_table(pdev);
- hpriv->mmio_base = iomap[mmio_bar];
- hpriv->cached_hctl = readw(hpriv->mmio_base + HOST_CTL);
-
- for (i = 0; i < NR_PORTS; i++) {
- struct ata_port *ap = host->ports[i];
-
- ata_port_pbar_desc(ap, mmio_bar, -1, "mmio");
- ata_port_pbar_desc(ap, mmio_bar, i * PORT_SIZE, "port");
- }
-
- /* Set dma_mask. This devices doesn't support 64bit addressing. */
- rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (rc) {
- dev_err(&pdev->dev, "32-bit DMA enable failed\n");
- return rc;
- }
-
- rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl);
- if (rc) {
- dev_err(&pdev->dev, "failed to initialize controller\n");
- return rc;
- }
-
- pci_set_master(pdev);
- return ata_host_activate(host, pdev->irq, inic_interrupt, IRQF_SHARED,
- &inic_sht);
-}
-
-static const struct pci_device_id inic_pci_tbl[] = {
- { PCI_VDEVICE(INIT, 0x1622), },
- { },
-};
-
-static struct pci_driver inic_pci_driver = {
- .name = DRV_NAME,
- .id_table = inic_pci_tbl,
-#ifdef CONFIG_PM_SLEEP
- .suspend = ata_pci_device_suspend,
- .resume = inic_pci_device_resume,
-#endif
- .probe = inic_init_one,
- .remove = ata_pci_remove_one,
-};
-
-module_pci_driver(inic_pci_driver);
-
-MODULE_AUTHOR("Tejun Heo");
-MODULE_DESCRIPTION("low-level driver for Initio 162x SATA");
-MODULE_LICENSE("GPL v2");
-MODULE_DEVICE_TABLE(pci, inic_pci_tbl);
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 40fdc422940d..7e1bd5610c3c 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -687,7 +687,7 @@ static struct ata_port_operations mv5_ops = {
.freeze = mv_eh_freeze,
.thaw = mv_eh_thaw,
- .hardreset = mv_hardreset,
+ .reset.hardreset = mv_hardreset,
.scr_read = mv5_scr_read,
.scr_write = mv5_scr_write,
@@ -709,10 +709,10 @@ static struct ata_port_operations mv6_ops = {
.freeze = mv_eh_freeze,
.thaw = mv_eh_thaw,
- .hardreset = mv_hardreset,
- .softreset = mv_softreset,
- .pmp_hardreset = mv_pmp_hardreset,
- .pmp_softreset = mv_softreset,
+ .reset.hardreset = mv_hardreset,
+ .reset.softreset = mv_softreset,
+ .pmp_reset.hardreset = mv_pmp_hardreset,
+ .pmp_reset.softreset = mv_softreset,
.error_handler = mv_pmp_error_handler,
.scr_read = mv_scr_read,
diff --git a/drivers/ata/sata_nv.c b/drivers/ata/sata_nv.c
index 36d99043ef50..d1e6e80d0103 100644
--- a/drivers/ata/sata_nv.c
+++ b/drivers/ata/sata_nv.c
@@ -462,7 +462,7 @@ static struct ata_port_operations nv_generic_ops = {
.lost_interrupt = ATA_OP_NULL,
.scr_read = nv_scr_read,
.scr_write = nv_scr_write,
- .hardreset = nv_hardreset,
+ .reset.hardreset = nv_hardreset,
};
static struct ata_port_operations nv_nf2_ops = {
diff --git a/drivers/ata/sata_promise.c b/drivers/ata/sata_promise.c
index 2df1a070b25a..2a005aede123 100644
--- a/drivers/ata/sata_promise.c
+++ b/drivers/ata/sata_promise.c
@@ -188,7 +188,7 @@ static struct ata_port_operations pdc_sata_ops = {
.scr_read = pdc_sata_scr_read,
.scr_write = pdc_sata_scr_write,
.port_start = pdc_sata_port_start,
- .hardreset = pdc_sata_hardreset,
+ .reset.hardreset = pdc_sata_hardreset,
};
/* First-generation chips need a more restrictive ->check_atapi_dma op,
@@ -206,7 +206,7 @@ static struct ata_port_operations pdc_pata_ops = {
.freeze = pdc_freeze,
.thaw = pdc_thaw,
.port_start = pdc_common_port_start,
- .softreset = pdc_pata_softreset,
+ .reset.softreset = pdc_pata_softreset,
};
static const struct ata_port_info pdc_port_info[] = {
diff --git a/drivers/ata/sata_qstor.c b/drivers/ata/sata_qstor.c
index 90289560e7cd..b2be1ce87878 100644
--- a/drivers/ata/sata_qstor.c
+++ b/drivers/ata/sata_qstor.c
@@ -123,8 +123,8 @@ static struct ata_port_operations qs_ata_ops = {
.freeze = qs_freeze,
.thaw = qs_thaw,
- .prereset = qs_prereset,
- .softreset = ATA_OP_NULL,
+ .reset.prereset = qs_prereset,
+ .reset.softreset = ATA_OP_NULL,
.error_handler = qs_error_handler,
.lost_interrupt = ATA_OP_NULL,
diff --git a/drivers/ata/sata_rcar.c b/drivers/ata/sata_rcar.c
index c1469d076880..146a231f8fe4 100644
--- a/drivers/ata/sata_rcar.c
+++ b/drivers/ata/sata_rcar.c
@@ -624,7 +624,7 @@ static struct ata_port_operations sata_rcar_port_ops = {
.freeze = sata_rcar_freeze,
.thaw = sata_rcar_thaw,
- .softreset = sata_rcar_softreset,
+ .reset.softreset = sata_rcar_softreset,
.scr_read = sata_rcar_scr_read,
.scr_write = sata_rcar_scr_write,
diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
index ca24c294bf46..92bb693c32d7 100644
--- a/drivers/ata/sata_sil24.c
+++ b/drivers/ata/sata_sil24.c
@@ -394,10 +394,10 @@ static struct ata_port_operations sil24_ops = {
.freeze = sil24_freeze,
.thaw = sil24_thaw,
- .softreset = sil24_softreset,
- .hardreset = sil24_hardreset,
- .pmp_softreset = sil24_softreset,
- .pmp_hardreset = sil24_pmp_hardreset,
+ .reset.softreset = sil24_softreset,
+ .reset.hardreset = sil24_hardreset,
+ .pmp_reset.softreset = sil24_softreset,
+ .pmp_reset.hardreset = sil24_pmp_hardreset,
.error_handler = sil24_error_handler,
.post_internal_cmd = sil24_post_internal_cmd,
.dev_config = sil24_dev_config,
diff --git a/drivers/ata/sata_svw.c b/drivers/ata/sata_svw.c
index 598a872f6a08..c5d6aa36c9c3 100644
--- a/drivers/ata/sata_svw.c
+++ b/drivers/ata/sata_svw.c
@@ -340,8 +340,8 @@ static const struct scsi_host_template k2_sata_sht = {
static struct ata_port_operations k2_sata_ops = {
.inherits = &ata_bmdma_port_ops,
- .softreset = k2_sata_softreset,
- .hardreset = k2_sata_hardreset,
+ .reset.softreset = k2_sata_softreset,
+ .reset.hardreset = k2_sata_hardreset,
.sff_tf_load = k2_sata_tf_load,
.sff_tf_read = k2_sata_tf_read,
.sff_check_status = k2_stat_check_status,
diff --git a/drivers/ata/sata_sx4.c b/drivers/ata/sata_sx4.c
index c3042eca6332..1c2ba6d30c2d 100644
--- a/drivers/ata/sata_sx4.c
+++ b/drivers/ata/sata_sx4.c
@@ -241,7 +241,7 @@ static struct ata_port_operations pdc_20621_ops = {
.freeze = pdc_freeze,
.thaw = pdc_thaw,
- .softreset = pdc_softreset,
+ .reset.softreset = pdc_softreset,
.error_handler = pdc_error_handler,
.lost_interrupt = ATA_OP_NULL,
.post_internal_cmd = pdc_post_internal_cmd,
diff --git a/drivers/ata/sata_uli.c b/drivers/ata/sata_uli.c
index 60ea45926cd1..6ad4c00688df 100644
--- a/drivers/ata/sata_uli.c
+++ b/drivers/ata/sata_uli.c
@@ -67,7 +67,7 @@ static struct ata_port_operations uli_ops = {
.inherits = &ata_bmdma_port_ops,
.scr_read = uli_scr_read,
.scr_write = uli_scr_write,
- .hardreset = ATA_OP_NULL,
+ .reset.hardreset = ATA_OP_NULL,
};
static const struct ata_port_info uli_port_info = {
diff --git a/drivers/ata/sata_via.c b/drivers/ata/sata_via.c
index 57cbf2cef618..59fb2c1ebcb0 100644
--- a/drivers/ata/sata_via.c
+++ b/drivers/ata/sata_via.c
@@ -119,7 +119,7 @@ static struct ata_port_operations svia_base_ops = {
static struct ata_port_operations vt6420_sata_ops = {
.inherits = &svia_base_ops,
.freeze = svia_noop_freeze,
- .prereset = vt6420_prereset,
+ .reset.prereset = vt6420_prereset,
.bmdma_start = vt6420_bmdma_start,
};
@@ -139,7 +139,7 @@ static struct ata_port_operations vt6421_sata_ops = {
static struct ata_port_operations vt8251_ops = {
.inherits = &svia_base_ops,
- .hardreset = sata_std_hardreset,
+ .reset.hardreset = sata_std_hardreset,
.scr_read = vt8251_scr_read,
.scr_write = vt8251_scr_write,
};
diff --git a/drivers/block/amiflop.c b/drivers/block/amiflop.c
index 49ced65bef4c..7a53ee1d18d8 100644
--- a/drivers/block/amiflop.c
+++ b/drivers/block/amiflop.c
@@ -265,6 +265,8 @@ static struct fd_data_type data_types[] = {
static struct amiga_floppy_struct unit[FD_MAX_UNITS];
static struct timer_list flush_track_timer[FD_MAX_UNITS];
+static DEFINE_SPINLOCK(flush_track_timer_lock);
+static bool flush_track_timer_stopped[FD_MAX_UNITS];
static struct timer_list post_write_timer;
static unsigned long post_write_timer_drive;
static struct timer_list motor_on_timer;
@@ -1370,11 +1372,22 @@ static void flush_track_callback(struct timer_list *timer)
sizeof(flush_track_timer[0]);
nr&=3;
+ spin_lock(&flush_track_timer_lock);
+ if (flush_track_timer_stopped[nr]) {
+ spin_unlock(&flush_track_timer_lock);
+ return;
+ }
+ spin_unlock(&flush_track_timer_lock);
+
writefromint = 1;
if (!try_fdc(nr)) {
/* we might block in an interrupt, so try again later */
- flush_track_timer[nr].expires = jiffies + 1;
- add_timer(flush_track_timer + nr);
+ spin_lock(&flush_track_timer_lock);
+ if (!flush_track_timer_stopped[nr]) {
+ flush_track_timer[nr].expires = jiffies + 1;
+ add_timer(flush_track_timer + nr);
+ }
+ spin_unlock(&flush_track_timer_lock);
return;
}
get_fdc(nr);
@@ -1421,6 +1434,35 @@ static int non_int_flush_track (unsigned long nr)
return 1;
}
+static void flush_track_timer_stop(unsigned int drive)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&flush_track_timer_lock, flags);
+ flush_track_timer_stopped[drive] = true;
+ spin_unlock_irqrestore(&flush_track_timer_lock, flags);
+
+ timer_delete_sync(flush_track_timer + drive);
+}
+
+static void flush_track_timer_start(unsigned int drive)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&flush_track_timer_lock, flags);
+ flush_track_timer_stopped[drive] = false;
+ if (unit[drive].dirty)
+ mod_timer(flush_track_timer + drive, jiffies + 1);
+ spin_unlock_irqrestore(&flush_track_timer_lock, flags);
+}
+
+static void flush_track_timer_sync(unsigned int drive)
+{
+ flush_track_timer_stop(drive);
+ non_int_flush_track(drive);
+ flush_track_timer_start(drive);
+}
+
static int get_track(int drive, int track)
{
int error, errcnt;
@@ -1435,8 +1477,7 @@ static int get_track(int drive, int track)
}
if (unit[drive].dirty == 1) {
- del_timer (flush_track_timer + drive);
- non_int_flush_track (drive);
+ flush_track_timer_sync(drive);
}
errcnt = 0;
while (errcnt < MAX_ERRORS) {
@@ -1562,6 +1603,12 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode,
case FDFMTTRK:
if (param < p->type->tracks * p->type->heads)
{
+ flush_track_timer_stop(drive);
+ if (unit[drive].dirty == 1 &&
+ !non_int_flush_track(drive)) {
+ flush_track_timer_start(drive);
+ return -EIO;
+ }
get_fdc(drive);
if (fd_seek(drive,param) != 0){
memset(p->trackbuf, FD_FILL_BYTE,
@@ -1570,6 +1617,7 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode,
}
floppy_off(drive);
rel_fdc();
+ flush_track_timer_start(drive);
}
else
return -EINVAL;
@@ -1591,8 +1639,7 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode,
case FDDEFPRM:
return -EINVAL;
case FDFLUSH: /* unconditionally, even if not needed */
- del_timer (flush_track_timer + drive);
- non_int_flush_track(drive);
+ flush_track_timer_sync(drive);
break;
#ifdef RAW_IOCTL
case IOCTL_RAW_TRACK:
@@ -1611,9 +1658,14 @@ static int fd_ioctl(struct block_device *bdev, blk_mode_t mode,
unsigned int cmd, unsigned long param)
{
int ret;
+ bool freeze_queue = cmd == FDFMTTRK || cmd == FDFLUSH;
mutex_lock(&amiflop_mutex);
+ if (freeze_queue)
+ blk_mq_freeze_queue(bdev->bd_disk->queue);
ret = fd_locked_ioctl(bdev, mode, cmd, param);
+ if (freeze_queue)
+ blk_mq_unfreeze_queue(bdev->bd_disk->queue);
mutex_unlock(&amiflop_mutex);
return ret;
@@ -1713,9 +1765,9 @@ static void floppy_release(struct gendisk *disk)
int drive = p - unit;
mutex_lock(&amiflop_mutex);
+ blk_mq_freeze_queue(disk->queue);
if (unit[drive].dirty == 1) {
- del_timer (flush_track_timer + drive);
- non_int_flush_track (drive);
+ flush_track_timer_sync(drive);
}
if (!fd_ref[drive]--) {
@@ -1725,6 +1777,7 @@ static void floppy_release(struct gendisk *disk)
#ifdef MODULE
floppy_off (drive);
#endif
+ blk_mq_unfreeze_queue(disk->queue);
mutex_unlock(&amiflop_mutex);
}
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 085a4d624659..f89b36017d97 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -1589,8 +1589,10 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, unsigned tag,
io->flags |= UBLK_IO_FLAG_CANCELED;
spin_unlock(&ubq->cancel_lock);
+ /* dependency ordered against smp_wmb() in ublk_prep_cancel() */
if (!done)
- io_uring_cmd_done(io->cmd, UBLK_IO_RES_ABORT, 0, issue_flags);
+ io_uring_cmd_done(READ_ONCE(io->cmd), UBLK_IO_RES_ABORT, 0,
+ issue_flags);
}
/*
@@ -1807,6 +1809,12 @@ static inline void ublk_fill_io_cmd(struct ublk_io *io,
io->addr = buf_addr;
}
+/*
+ * Call before ublk_fill_io_cmd() publishes @cmd in io->cmd: a control-path
+ * cancel may complete any command found there, and io_uring_cmd_done() only
+ * takes it off the cancelable list if it is marked already. The handlers
+ * hold uring_lock, so marking takes no lock.
+ */
static inline void ublk_prep_cancel(struct io_uring_cmd *cmd,
unsigned int issue_flags,
struct ublk_queue *ubq, unsigned int tag)
@@ -1820,6 +1828,8 @@ static inline void ublk_prep_cancel(struct io_uring_cmd *cmd,
pdu->ubq = ubq;
pdu->tag = tag;
io_uring_cmd_mark_cancelable(cmd, issue_flags);
+ /* pairs with the cancel loading cmd from io->cmd, then cmd->flags */
+ smp_wmb();
}
static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd,
@@ -1899,6 +1909,7 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd,
goto out;
}
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
ublk_fill_io_cmd(io, cmd, ub_cmd->addr);
ublk_mark_io_ready(ub, ubq);
break;
@@ -1925,19 +1936,20 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd,
goto out;
}
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
ublk_fill_io_cmd(io, cmd, ub_cmd->addr);
ublk_commit_completion(ub, ub_cmd);
break;
case UBLK_IO_NEED_GET_DATA:
if (!(io->flags & UBLK_IO_FLAG_OWNED_BY_SRV))
goto out;
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
ublk_fill_io_cmd(io, cmd, ub_cmd->addr);
ublk_handle_need_get_data(ub, ub_cmd->q_id, ub_cmd->tag);
break;
default:
goto out;
}
- ublk_prep_cancel(cmd, issue_flags, ubq, tag);
return -EIOCBQUEUED;
out:
@@ -3037,6 +3049,7 @@ static int ublk_ctrl_uring_cmd_permission(struct ublk_device *ub,
case UBLK_CMD_STOP_DEV:
case UBLK_CMD_ADD_DEV:
case UBLK_CMD_DEL_DEV:
+ case UBLK_CMD_DEL_DEV_ASYNC:
case UBLK_CMD_SET_PARAMS:
case UBLK_CMD_START_USER_RECOVERY:
case UBLK_CMD_END_USER_RECOVERY:
diff --git a/drivers/char/hw_random/virtio-rng.c b/drivers/char/hw_random/virtio-rng.c
index 2933f8cb769e..fcb60d4deaac 100644
--- a/drivers/char/hw_random/virtio-rng.c
+++ b/drivers/char/hw_random/virtio-rng.c
@@ -187,6 +187,7 @@ static int probe_common(struct virtio_device *vdev)
err_find:
ida_free(&rng_index_ida, index);
err_ida:
+ vdev->priv = NULL;
kfree(vi);
return err;
}
@@ -195,6 +196,11 @@ static void remove_common(struct virtio_device *vdev)
{
struct virtrng_info *vi = vdev->priv;
+ if (!vi) {
+ virtio_reset_device(vdev);
+ return;
+ }
+
vi->hwrng_removed = true;
vi->data_avail = 0;
vi->data_idx = 0;
@@ -204,6 +210,7 @@ static void remove_common(struct virtio_device *vdev)
virtio_reset_device(vdev);
vdev->config->del_vqs(vdev);
ida_free(&rng_index_ida, vi->index);
+ vdev->priv = NULL;
kfree(vi);
}
diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c
index 63d26aebf55d..2e4652d43bca 100644
--- a/drivers/char/tpm/tpm2-cmd.c
+++ b/drivers/char/tpm/tpm2-cmd.c
@@ -551,9 +551,10 @@ int tpm2_probe(struct tpm_chip *chip)
out = (struct tpm_header *)buf.data;
if (be16_to_cpu(out->tag) == TPM2_ST_NO_SESSIONS)
chip->flags |= TPM_CHIP_FLAG_TPM2;
+ rc = 0;
}
tpm_buf_destroy(&buf);
- return 0;
+ return rc;
}
EXPORT_SYMBOL_GPL(tpm2_probe);
diff --git a/drivers/clk/nuvoton/clk-ma35d1.c b/drivers/clk/nuvoton/clk-ma35d1.c
index 9f65d0623517..c914079cee2d 100644
--- a/drivers/clk/nuvoton/clk-ma35d1.c
+++ b/drivers/clk/nuvoton/clk-ma35d1.c
@@ -17,6 +17,7 @@
static DEFINE_SPINLOCK(ma35d1_lock);
#define PLL_MAX_NUM 5
+#define MA35D1_CLK_COUNT 238
/* Clock Control Registers Offset */
#define REG_CLK_PWRCTL 0x00
@@ -467,12 +468,12 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
int ret;
ma35d1_hw_data = devm_kzalloc(dev,
- struct_size(ma35d1_hw_data, hws, CLK_MAX_IDX),
+ struct_size(ma35d1_hw_data, hws, MA35D1_CLK_COUNT),
GFP_KERNEL);
if (!ma35d1_hw_data)
return -ENOMEM;
- ma35d1_hw_data->num = CLK_MAX_IDX;
+ ma35d1_hw_data->num = MA35D1_CLK_COUNT;
hws = ma35d1_hw_data->hws;
clk_base = devm_platform_ioremap_resource(pdev, 0);
@@ -543,6 +544,8 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
hws[HCLK3] = ma35d1_clk_fixed_factor(dev, "hclk3", "sysclk1_mux", 1, 2);
hws[PCLK3] = ma35d1_clk_fixed_factor(dev, "pclk3", "sysclk1_mux", 1, 2);
hws[PCLK4] = ma35d1_clk_fixed_factor(dev, "pclk4", "sysclk1_mux", 1, 2);
+ hws[PCLK3_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk3_div4096", "pclk3", 1, 4096);
+ hws[PCLK4_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk4_div4096", "pclk4", 1, 4096);
hws[USBPHY0] = ma35d1_clk_fixed("usbphy0", 480000000);
hws[USBPHY1] = ma35d1_clk_fixed("usbphy1", 480000000);
diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index 88dffbd65fcc..7d51877c536f 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -1285,7 +1285,7 @@ static ssize_t amd_pstate_show_status(char *buf)
int amd_pstate_update_status(const char *buf, size_t size)
{
- int mode_idx;
+ int cpu, mode_idx, ret = 0;
if (size > strlen("passive") || size < strlen("active"))
return -EINVAL;
@@ -1295,10 +1295,25 @@ int amd_pstate_update_status(const char *buf, size_t size)
if (mode_idx < 0 || mode_idx >= AMD_PSTATE_MAX)
return -EINVAL;
- if (mode_state_machine[cppc_state][mode_idx])
- return mode_state_machine[cppc_state][mode_idx](mode_idx);
+ if (!mode_state_machine[cppc_state][mode_idx])
+ return 0;
- return 0;
+ if (mode_idx == AMD_PSTATE_PASSIVE &&
+ !cpu_feature_enabled(X86_FEATURE_CPPC)) {
+ cpus_read_lock();
+ /* Check offline CPUs too, before changing or removing the driver. */
+ for_each_present_cpu(cpu) {
+ if (cppc_auto_sel_is_immutable(cpu)) {
+ ret = -EOPNOTSUPP;
+ break;
+ }
+ }
+ cpus_read_unlock();
+ if (ret)
+ return ret;
+ }
+
+ return mode_state_machine[cppc_state][mode_idx](mode_idx);
}
EXPORT_SYMBOL_GPL(amd_pstate_update_status);
diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c
index bad1adacbc84..df2fc4e1c31d 100644
--- a/drivers/crypto/omap-aes.c
+++ b/drivers/crypto/omap-aes.c
@@ -33,6 +33,7 @@
#include <linux/pm_runtime.h>
#include <linux/scatterlist.h>
#include <linux/string.h>
+#include <linux/workqueue.h>
#include "omap-crypto.h"
#include "omap-aes.h"
@@ -222,7 +223,7 @@ static void omap_aes_dma_out_callback(void *data)
struct omap_aes_dev *dd = data;
/* dma_lch_out - completed */
- tasklet_schedule(&dd->done_task);
+ queue_work(system_bh_wq, &dd->done_task);
}
static int omap_aes_dma_init(struct omap_aes_dev *dd)
@@ -496,9 +497,9 @@ static void omap_aes_copy_ivout(struct omap_aes_dev *dd, u8 *ivbuf)
((u32 *)ivbuf)[i] = omap_aes_read(dd, AES_REG_IV(dd, i));
}
-static void omap_aes_done_task(unsigned long data)
+static void omap_aes_done_task(struct work_struct *t)
{
- struct omap_aes_dev *dd = (struct omap_aes_dev *)data;
+ struct omap_aes_dev *dd = from_work(dd, t, done_task);
pr_debug("enter done_task\n");
@@ -932,7 +933,7 @@ static irqreturn_t omap_aes_irq(int irq, void *dev_id)
if (!dd->total)
/* All bytes read! */
- tasklet_schedule(&dd->done_task);
+ queue_work(system_bh_wq, &dd->done_task);
else
/* Enable DATA_IN interrupt for next block */
omap_aes_write(dd, AES_REG_IRQ_ENABLE(dd), 0x2);
@@ -1150,7 +1151,7 @@ static int omap_aes_probe(struct platform_device *pdev)
(reg & dd->pdata->major_mask) >> dd->pdata->major_shift,
(reg & dd->pdata->minor_mask) >> dd->pdata->minor_shift);
- tasklet_init(&dd->done_task, omap_aes_done_task, (unsigned long)dd);
+ INIT_WORK(&dd->done_task, omap_aes_done_task);
err = omap_aes_dma_init(dd);
if (err == -EPROBE_DEFER) {
@@ -1245,7 +1246,7 @@ static int omap_aes_probe(struct platform_device *pdev)
omap_aes_dma_cleanup(dd);
err_irq:
- tasklet_kill(&dd->done_task);
+ cancel_work_sync(&dd->done_task);
err_pm_disable:
pm_runtime_disable(dev);
err_res:
@@ -1280,7 +1281,7 @@ static void omap_aes_remove(struct platform_device *pdev)
crypto_engine_exit(dd->engine);
- tasklet_kill(&dd->done_task);
+ cancel_work_sync(&dd->done_task);
omap_aes_dma_cleanup(dd);
pm_runtime_disable(dd->dev);
diff --git a/drivers/crypto/omap-aes.h b/drivers/crypto/omap-aes.h
index 0f35c9164764..d999d94e2314 100644
--- a/drivers/crypto/omap-aes.h
+++ b/drivers/crypto/omap-aes.h
@@ -161,7 +161,7 @@ struct omap_aes_dev {
unsigned long flags;
int err;
- struct tasklet_struct done_task;
+ struct work_struct done_task;
struct aead_queue aead_queue;
spinlock_t lock;
diff --git a/drivers/crypto/omap-des.c b/drivers/crypto/omap-des.c
index d2c016eafd46..cccd275ace67 100644
--- a/drivers/crypto/omap-des.c
+++ b/drivers/crypto/omap-des.c
@@ -33,6 +33,7 @@
#include <linux/pm_runtime.h>
#include <linux/scatterlist.h>
#include <linux/string.h>
+#include <linux/workqueue.h>
#include "omap-crypto.h"
@@ -133,7 +134,7 @@ struct omap_des_dev {
unsigned long flags;
int err;
- struct tasklet_struct done_task;
+ struct work_struct done_task;
struct skcipher_request *req;
struct crypto_engine *engine;
@@ -328,7 +329,7 @@ static void omap_des_dma_out_callback(void *data)
struct omap_des_dev *dd = data;
/* dma_lch_out - completed */
- tasklet_schedule(&dd->done_task);
+ queue_work(system_bh_wq, &dd->done_task);
}
static int omap_des_dma_init(struct omap_des_dev *dd)
@@ -584,9 +585,9 @@ static int omap_des_crypt_req(struct crypto_engine *engine,
omap_des_crypt_dma_start(dd);
}
-static void omap_des_done_task(unsigned long data)
+static void omap_des_done_task(struct work_struct *t)
{
- struct omap_des_dev *dd = (struct omap_des_dev *)data;
+ struct omap_des_dev *dd = from_work(dd, t, done_task);
int i;
pr_debug("enter done_task\n");
@@ -899,7 +900,7 @@ static irqreturn_t omap_des_irq(int irq, void *dev_id)
if (!dd->total)
/* All bytes read! */
- tasklet_schedule(&dd->done_task);
+ queue_work(system_bh_wq, &dd->done_task);
else
/* Enable DATA_IN interrupt for next block */
omap_des_write(dd, DES_REG_IRQ_ENABLE(dd), 0x2);
@@ -1009,7 +1010,7 @@ static int omap_des_probe(struct platform_device *pdev)
(reg & dd->pdata->major_mask) >> dd->pdata->major_shift,
(reg & dd->pdata->minor_mask) >> dd->pdata->minor_shift);
- tasklet_init(&dd->done_task, omap_des_done_task, (unsigned long)dd);
+ INIT_WORK(&dd->done_task, omap_des_done_task);
err = omap_des_dma_init(dd);
if (err == -EPROBE_DEFER) {
@@ -1072,7 +1073,7 @@ static int omap_des_probe(struct platform_device *pdev)
omap_des_dma_cleanup(dd);
err_irq:
- tasklet_kill(&dd->done_task);
+ cancel_work_sync(&dd->done_task);
err_get:
pm_runtime_disable(dev);
err_res:
@@ -1092,7 +1093,9 @@ static void omap_des_remove(struct platform_device *pdev)
omap_des_unregister_algs(dd->pdata);
- tasklet_kill(&dd->done_task);
+ crypto_engine_exit(dd->engine);
+
+ cancel_work_sync(&dd->done_task);
omap_des_dma_cleanup(dd);
pm_runtime_disable(dd->dev);
}
diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index 5bcd9ab0f72a..4f5e36197d3d 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -37,6 +37,7 @@
#include <linux/scatterlist.h>
#include <linux/slab.h>
#include <linux/string.h>
+#include <linux/workqueue.h>
#define MD5_DIGEST_SIZE 16
@@ -217,7 +218,7 @@ struct omap_sham_dev {
int irq;
int err;
struct dma_chan *dma_lch;
- struct tasklet_struct done_task;
+ struct work_struct done_task;
u8 polling_mode;
u8 xmit_buf[BUFLEN] OMAP_ALIGNED;
@@ -561,7 +562,7 @@ static void omap_sham_dma_callback(void *param)
struct omap_sham_dev *dd = param;
set_bit(FLAGS_DMA_READY, &dd->flags);
- tasklet_schedule(&dd->done_task);
+ queue_work(system_bh_wq, &dd->done_task);
}
static int omap_sham_xmit_dma(struct omap_sham_dev *dd, size_t length,
@@ -1704,9 +1705,9 @@ static struct ahash_engine_alg algs_sha384_sha512[] = {
},
};
-static void omap_sham_done_task(unsigned long data)
+static void omap_sham_done_task(struct work_struct *t)
{
- struct omap_sham_dev *dd = (struct omap_sham_dev *)data;
+ struct omap_sham_dev *dd = from_work(dd, t, done_task);
int err = 0;
dev_dbg(dd->dev, "%s: flags=%lx\n", __func__, dd->flags);
@@ -1740,7 +1741,7 @@ static void omap_sham_done_task(unsigned long data)
static irqreturn_t omap_sham_irq_common(struct omap_sham_dev *dd)
{
set_bit(FLAGS_OUTPUT_READY, &dd->flags);
- tasklet_schedule(&dd->done_task);
+ queue_work(system_bh_wq, &dd->done_task);
return IRQ_HANDLED;
}
@@ -2044,6 +2045,20 @@ static const struct attribute_group omap_sham_attr_group = {
.attrs = omap_sham_attrs,
};
+static void omap_sham_unregister_algs(const struct omap_sham_pdata *pdata)
+{
+ struct omap_sham_algs_info *alg_info;
+ int i;
+
+ for (i = pdata->algs_info_size - 1; i >= 0; i--) {
+ alg_info = &pdata->algs_info[i];
+
+ crypto_engine_unregister_ahashes(alg_info->algs_list,
+ alg_info->registered);
+ alg_info->registered = 0;
+ }
+}
+
static int omap_sham_probe(struct platform_device *pdev)
{
struct omap_sham_dev *dd;
@@ -2063,7 +2078,7 @@ static int omap_sham_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, dd);
INIT_LIST_HEAD(&dd->list);
- tasklet_init(&dd->done_task, omap_sham_done_task, (unsigned long)dd);
+ INIT_WORK(&dd->done_task, omap_sham_done_task);
crypto_init_queue(&dd->queue, OMAP_SHAM_QUEUE_LENGTH);
err = (dev->of_node) ? omap_sham_get_res_of(dd, dev, &res) :
@@ -2167,10 +2182,7 @@ static int omap_sham_probe(struct platform_device *pdev)
return 0;
err_algs:
- for (i = dd->pdata->algs_info_size - 1; i >= 0; i--)
- for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--)
- crypto_engine_unregister_ahash(
- &dd->pdata->algs_info[i].algs_list[j]);
+ omap_sham_unregister_algs(dd->pdata);
err_engine_start:
crypto_engine_exit(dd->engine);
err_engine:
@@ -2191,20 +2203,15 @@ static int omap_sham_probe(struct platform_device *pdev)
static void omap_sham_remove(struct platform_device *pdev)
{
struct omap_sham_dev *dd;
- int i, j;
dd = platform_get_drvdata(pdev);
spin_lock_bh(&sham.lock);
list_del(&dd->list);
spin_unlock_bh(&sham.lock);
- for (i = dd->pdata->algs_info_size - 1; i >= 0; i--)
- for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--) {
- crypto_engine_unregister_ahash(
- &dd->pdata->algs_info[i].algs_list[j]);
- dd->pdata->algs_info[i].registered--;
- }
- tasklet_kill(&dd->done_task);
+ omap_sham_unregister_algs(dd->pdata);
+ crypto_engine_exit(dd->engine);
+ cancel_work_sync(&dd->done_task);
pm_runtime_dont_use_autosuspend(&pdev->dev);
pm_runtime_disable(&pdev->dev);
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index cf647662ae81..d06f95f7868c 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -1322,8 +1322,8 @@ static int bam_dma_probe(struct platform_device *pdev)
for (i = 0; i < bdev->num_channels; i++)
bam_channel_init(bdev, &bdev->channels[i], i);
- ret = devm_request_irq(bdev->dev, bdev->irq, bam_dma_irq,
- IRQF_TRIGGER_HIGH, "bam_dma", bdev);
+ ret = request_irq(bdev->irq, bam_dma_irq, IRQF_TRIGGER_NONE,
+ "bam_dma", bdev);
if (ret)
goto err_bam_channel_exit;
@@ -1356,7 +1356,7 @@ static int bam_dma_probe(struct platform_device *pdev)
ret = dma_async_device_register(&bdev->common);
if (ret) {
dev_err(bdev->dev, "failed to register dma async device\n");
- goto err_bam_channel_exit;
+ goto err_free_irq;
}
ret = of_dma_controller_register(pdev->dev.of_node, bam_dma_xlate,
@@ -1375,6 +1375,8 @@ static int bam_dma_probe(struct platform_device *pdev)
err_unregister_dma:
dma_async_device_unregister(&bdev->common);
+err_free_irq:
+ free_irq(bdev->irq, bdev);
err_bam_channel_exit:
for (i = 0; i < bdev->num_channels; i++)
tasklet_kill(&bdev->channels[i].vc.task);
@@ -1400,7 +1402,7 @@ static void bam_dma_remove(struct platform_device *pdev)
/* mask all interrupts for this execution environment */
writel_relaxed(0, bam_addr(bdev, 0, BAM_IRQ_SRCS_MSK_EE));
- devm_free_irq(bdev->dev, bdev->irq, bdev);
+ free_irq(bdev->irq, bdev);
for (i = 0; i < bdev->num_channels; i++) {
bam_dma_terminate_all(&bdev->channels[i].vc.chan);
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index 1141cd06011f..d33c02e58717 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -22,7 +22,7 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ -std=gnu11 \
# arm64 uses the full KBUILD_CFLAGS so it's necessary to explicitly
# disable the stackleak plugin
-cflags-$(CONFIG_ARM64) += -fpie $(DISABLE_STACKLEAK_PLUGIN) \
+cflags-$(CONFIG_ARM64) += -fpie $(DISABLE_STACKLEAK_PLUGIN) -ffixed-x18 \
-fno-unwind-tables -fno-asynchronous-unwind-tables
cflags-$(CONFIG_ARM) += -DEFI_HAVE_STRLEN -DEFI_HAVE_STRNLEN \
-DEFI_HAVE_MEMCHR -DEFI_HAVE_STRRCHR \
diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c
index 72a1819713ed..db29342a7676 100644
--- a/drivers/gpio/gpio-pca953x.c
+++ b/drivers/gpio/gpio-pca953x.c
@@ -742,13 +742,18 @@ static int pca953x_irq_set_wake(struct irq_data *d, unsigned int on)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct pca953x_chip *chip = gpiochip_get_data(gc);
+ int ret;
+
+ ret = irq_set_irq_wake(chip->client->irq, on);
+ if (ret)
+ return ret;
if (on)
atomic_inc(&chip->wakeup_path);
else
atomic_dec(&chip->wakeup_path);
- return irq_set_irq_wake(chip->client->irq, on);
+ return 0;
}
static void pca953x_irq_bus_lock(struct irq_data *d)
diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
index b85ba9989b14..e7416a018a54 100644
--- a/drivers/gpio/gpiolib-cdev.c
+++ b/drivers/gpio/gpiolib-cdev.c
@@ -1367,11 +1367,6 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags,
assign_bit(FLAG_ACTIVE_LOW, flagsp,
flags & GPIO_V2_LINE_FLAG_ACTIVE_LOW);
- if (flags & GPIO_V2_LINE_FLAG_OUTPUT)
- set_bit(FLAG_IS_OUT, flagsp);
- else if (flags & GPIO_V2_LINE_FLAG_INPUT)
- clear_bit(FLAG_IS_OUT, flagsp);
-
assign_bit(FLAG_EDGE_RISING, flagsp,
flags & GPIO_V2_LINE_FLAG_EDGE_RISING);
assign_bit(FLAG_EDGE_FALLING, flagsp,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index d79c431de223..b921587d428c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -103,6 +103,7 @@ struct amdgpu_kfd_dev {
struct kfd_dev *dev;
int64_t vram_used[MAX_XCP];
uint64_t vram_used_aligned[MAX_XCP];
+ atomic64_t vram_pinned;
bool init_complete;
struct work_struct reset_work;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index bc284939cf66..55834719e984 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1500,6 +1500,11 @@ static int amdgpu_amdkfd_gpuvm_pin_bo(struct amdgpu_bo *bo, u32 domain)
pr_err("Error in Pinning BO to domain: %d\n", domain);
amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, false);
+
+ if (!ret && bo->tbo.resource->mem_type == TTM_PL_VRAM)
+ atomic64_add(amdgpu_bo_size(bo),
+ &amdgpu_ttm_adev(bo->tbo.bdev)->kfd.vram_pinned);
+
out:
amdgpu_bo_unreserve(bo);
return ret;
@@ -1522,6 +1527,11 @@ static void amdgpu_amdkfd_gpuvm_unpin_bo(struct amdgpu_bo *bo)
return;
amdgpu_bo_unpin(bo);
+
+ if (bo->tbo.resource->mem_type == TTM_PL_VRAM)
+ atomic64_sub(amdgpu_bo_size(bo),
+ &amdgpu_ttm_adev(bo->tbo.bdev)->kfd.vram_pinned);
+
amdgpu_bo_unreserve(bo);
}
@@ -1681,6 +1691,7 @@ size_t amdgpu_amdkfd_get_available_memory(struct amdgpu_device *adev,
vram_available = KFD_XCP_MEMORY_SIZE(adev, xcp_id)
- adev->kfd.vram_used_aligned[xcp_id]
- atomic64_read(&adev->vram_pin_size)
+ + atomic64_read(&adev->kfd.vram_pinned)
- reserved_for_pt
- reserved_for_ras;
@@ -2441,6 +2452,9 @@ static int import_obj_create(struct amdgpu_device *adev,
err_remove_mem:
remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info);
+ amdgpu_sync_free(&(*mem)->sync);
+ mutex_destroy(&(*mem)->lock);
+ dma_buf_put(dma_buf);
drm_vma_node_revoke(&obj->vma_node, drm_priv);
err_free_mem:
kfree(*mem);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 528f0aeac36c..c49aabb343a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -160,7 +160,12 @@ static int amdgpu_cs_p1_user_fence(struct amdgpu_cs_parser *p,
if (size != PAGE_SIZE || data->offset > (size - 8))
return -EINVAL;
- if (amdgpu_ttm_tt_get_usermm(p->uf_bo->tbo.ttm))
+ if (amdgpu_ttm_tt_is_userptr(p->uf_bo->tbo.ttm))
+ return -EINVAL;
+
+ /* The GPU writes the user fence through the GART */
+ if (p->uf_bo->preferred_domains != AMDGPU_GEM_DOMAIN_GTT &&
+ p->uf_bo->allowed_domains != AMDGPU_GEM_DOMAIN_GTT)
return -EINVAL;
*offset = data->offset;
@@ -198,25 +203,17 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
struct amdgpu_fpriv *fpriv = p->filp->driver_priv;
unsigned int num_ibs[AMDGPU_CS_GANG_SIZE] = { };
struct amdgpu_vm *vm = &fpriv->vm;
- uint64_t *chunk_array_user;
uint64_t *chunk_array;
uint32_t uf_offset = 0;
size_t size;
int ret;
int i;
- chunk_array = kvmalloc_array(cs->in.num_chunks, sizeof(uint64_t),
- GFP_KERNEL);
- if (!chunk_array)
- return -ENOMEM;
-
- /* get chunks */
- chunk_array_user = u64_to_user_ptr(cs->in.chunks);
- if (copy_from_user(chunk_array, chunk_array_user,
- sizeof(uint64_t)*cs->in.num_chunks)) {
- ret = -EFAULT;
- goto free_chunk;
- }
+ chunk_array = memdup_array_user(u64_to_user_ptr(cs->in.chunks),
+ cs->in.num_chunks,
+ sizeof(uint64_t));
+ if (IS_ERR(chunk_array))
+ return PTR_ERR(chunk_array);
p->nchunks = cs->in.num_chunks;
p->chunks = kvmalloc_array(p->nchunks, sizeof(struct amdgpu_cs_chunk),
@@ -229,7 +226,6 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
for (i = 0; i < p->nchunks; i++) {
struct drm_amdgpu_cs_chunk __user *chunk_ptr = NULL;
struct drm_amdgpu_cs_chunk user_chunk;
- uint32_t __user *cdata;
chunk_ptr = u64_to_user_ptr(chunk_array[i]);
if (copy_from_user(&user_chunk, chunk_ptr,
@@ -242,20 +238,22 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
p->chunks[i].length_dw = user_chunk.length_dw;
size = p->chunks[i].length_dw;
- cdata = u64_to_user_ptr(user_chunk.chunk_data);
- p->chunks[i].kdata = kvmalloc_array(size, sizeof(uint32_t),
- GFP_KERNEL);
- if (p->chunks[i].kdata == NULL) {
- ret = -ENOMEM;
+ if (size > INT_MAX / sizeof(uint32_t)) {
+ ret = -EINVAL;
i--;
goto free_partial_kdata;
}
- size *= sizeof(uint32_t);
- if (copy_from_user(p->chunks[i].kdata, cdata, size)) {
- ret = -EFAULT;
+
+ p->chunks[i].kdata = vmemdup_array_user(u64_to_user_ptr(user_chunk.chunk_data),
+ size,
+ sizeof(uint32_t));
+ if (IS_ERR(p->chunks[i].kdata)) {
+ ret = PTR_ERR(p->chunks[i].kdata);
+ i--;
goto free_partial_kdata;
}
+ size *= sizeof(uint32_t);
/* Assume the worst on the following checks */
ret = -EINVAL;
@@ -1086,7 +1084,7 @@ static int amdgpu_cs_patch_ibs(struct amdgpu_cs_parser *p,
if (r)
return r;
- kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE);
+ kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE) + m->offset;
if (ring->funcs->parse_cs) {
memcpy(ib->ptr, kptr, ib->length_dw * 4);
@@ -1320,10 +1318,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
fence = &p->jobs[i]->base.s_fence->scheduled;
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&leader->base, fence);
- if (r) {
- dma_fence_put(fence);
+ if (r)
return r;
- }
}
if (p->gang_size > 1) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index a8f8bb342884..037cb9ce69e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -2680,6 +2680,13 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
break;
}
+ /* Check for IP version 9.4.3 with A0 hardware */
+ if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3) &&
+ !amdgpu_device_get_rev_id(adev)) {
+ dev_err(adev->dev, "Unsupported A0 hardware\n");
+ return -ENODEV; /* device unsupported - no device error */
+ }
+
if (amdgpu_has_atpx() &&
(amdgpu_is_atpx_hybrid() ||
amdgpu_has_atpx_dgpu_power_cntl()) &&
@@ -2692,7 +2699,6 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
adev->has_pr3 = parent ? pci_pr3_present(parent) : false;
}
-
adev->pm.pp_feature = amdgpu_pp_feature_mask;
if (amdgpu_sriov_vf(adev) || sched_policy == KFD_SCHED_POLICY_NO_HWS)
adev->pm.pp_feature &= ~PP_GFXOFF_MASK;
@@ -3301,6 +3307,12 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE);
+ r = amdgpu_dpm_reapply_umd_profile(adev);
+ if (r) {
+ dev_err(adev->dev, "Failed to restore UMD profile state: %d\n", r);
+ return r;
+ }
+
amdgpu_device_fill_reset_magic(adev);
r = amdgpu_device_enable_mgpu_fan_boost();
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 1a112a361757..7c9fee6a2edb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -2838,19 +2838,6 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev)
return 0;
}
-static void amdgpu_restore_umd_profile_pstate_after_runpm(struct amdgpu_device *adev)
-{
- enum amd_dpm_forced_level level;
- uint32_t profile_mode_mask = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD |
- AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK |
- AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK |
- AMD_DPM_FORCED_LEVEL_PROFILE_PEAK;
-
- level = amdgpu_dpm_get_performance_level(adev);
- if (level & profile_mode_mask)
- amdgpu_asic_update_umd_stable_pstate(adev, true);
-}
-
static int amdgpu_pmops_runtime_resume(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
@@ -2896,7 +2883,6 @@ static int amdgpu_pmops_runtime_resume(struct device *dev)
if (adev->pm.rpm_mode == AMDGPU_RUNPM_PX)
drm_dev->switch_power_state = DRM_SWITCH_POWER_ON;
- amdgpu_restore_umd_profile_pstate_after_runpm(adev);
adev->in_runpm = false;
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index 86c17a8946f5..8fe497411601 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -423,10 +423,8 @@ int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job)
dma_fence_get(f);
r = drm_sched_job_add_dependency(&job->base, f);
- if (r) {
- dma_fence_put(f);
+ if (r)
return r;
- }
}
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index 902d0a538e41..dd72891624eb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -234,7 +234,6 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&p->job->base, fence);
if (r) {
- dma_fence_put(fence);
dma_resv_iter_end(&cursor);
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index daed4a8439cc..8bfa0b07dec7 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -1736,8 +1736,9 @@ static void dce_v10_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + dig->afmt->offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + dig->afmt->offset, 0x00000001);
- /* enable audio after to setting up hw */
- dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v10_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
index 1e15348a5c7d..9b39518a3e11 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
@@ -1721,8 +1721,9 @@ static void dce_v6_0_afmt_setmode(struct drm_encoder *encoder,
dce_v6_0_audio_dp_enable(encoder, 1);
}
- /* enable audio after setting up hw */
- dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v6_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index 7e92f322bc33..c22bb261d0ff 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -1683,8 +1683,9 @@ static void dce_v8_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + offset, 0x00000001);
- /* enable audio after setting up hw */
- dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v8_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index a076a314dc67..5282c2de1ec8 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -3828,6 +3828,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
{
u32 reg, data;
+ amdgpu_gfx_off_ctrl(adev, false);
reg = SOC15_REG_OFFSET(GC, 0, regRLC_SPM_MC_CNTL);
if (amdgpu_sriov_is_pp_one_vf(adev))
data = RREG32_NO_KIQ(reg);
@@ -3841,6 +3842,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
WREG32_SOC15_NO_KIQ(GC, 0, regRLC_SPM_MC_CNTL, data);
else
WREG32_SOC15(GC, 0, regRLC_SPM_MC_CNTL, data);
+ amdgpu_gfx_off_ctrl(adev, true);
if (ring
&& amdgpu_sriov_is_pp_one_vf(adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index d82f90f48024..06e4b393d1ce 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -1139,12 +1139,13 @@ static void gmc_v9_0_get_coherence_flags(struct amdgpu_device *adev,
bool uncached = bo->flags & AMDGPU_GEM_CREATE_UNCACHED;
struct amdgpu_vm *vm = mapping->bo_va->base.vm;
unsigned int mtype_local, mtype;
+ uint32_t gc_ip_version = amdgpu_ip_version(adev, GC_HWIP, 0);
bool snoop = false;
bool is_local;
dma_resv_assert_held(bo->tbo.base.resv);
- switch (amdgpu_ip_version(adev, GC_HWIP, 0)) {
+ switch (gc_ip_version) {
case IP_VERSION(9, 4, 1):
case IP_VERSION(9, 4, 2):
if (is_vram) {
@@ -1158,10 +1159,7 @@ static void gmc_v9_0_get_coherence_flags(struct amdgpu_device *adev,
/* FIXME: is this still needed? Or does
* amdgpu_ttm_tt_pde_flags already handle this?
*/
- if ((amdgpu_ip_version(adev, GC_HWIP, 0) ==
- IP_VERSION(9, 4, 2) ||
- amdgpu_ip_version(adev, GC_HWIP, 0) ==
- IP_VERSION(9, 4, 3)) &&
+ if (gc_ip_version == IP_VERSION(9, 4, 2) &&
adev->gmc.xgmi.connected_to_cpu)
snoop = true;
} else {
@@ -1209,20 +1207,17 @@ static void gmc_v9_0_get_coherence_flags(struct amdgpu_device *adev,
if (uncached) {
mtype = MTYPE_UC;
} else if (ext_coherent) {
- if (adev->rev_id)
- mtype = is_local ? MTYPE_CC : MTYPE_UC;
- else
- mtype = MTYPE_UC;
+ mtype = is_local ? MTYPE_CC : MTYPE_UC;
} else if (adev->flags & AMD_IS_APU) {
mtype = is_local ? mtype_local : MTYPE_NC;
} else {
/* dGPU */
if (is_local)
mtype = mtype_local;
- else if (is_vram)
- mtype = MTYPE_NC;
- else
+ else if (gc_ip_version < IP_VERSION(9, 5, 0) && !is_vram)
mtype = MTYPE_UC;
+ else
+ mtype = MTYPE_NC;
}
break;
@@ -1332,7 +1327,7 @@ static void gmc_v9_0_override_vm_pte_flags(struct amdgpu_device *adev,
mtype_local = MTYPE_CC;
*flags = AMDGPU_PTE_MTYPE_VG10(*flags, mtype_local);
- } else if (adev->rev_id) {
+ } else {
/* MTYPE_UC case */
*flags = AMDGPU_PTE_MTYPE_VG10(*flags, MTYPE_CC);
}
@@ -2329,17 +2324,22 @@ static int gmc_v9_0_hw_init(void *handle)
* (type 2), which flushes both. Due to a race condition with
* concurrent memory accesses using the same TLB cache line, we still
* need a second TLB flush after this.
+ *
+ * GC 9.4.3, 9.4.4 and 9.5.0 hit a similar race.
*/
- adev->gmc.flush_tlb_needs_extra_type_2 =
- amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 0) &&
- adev->gmc.xgmi.num_physical_nodes;
- /*
- * TODO: This workaround is badly documented and had a buggy
- * implementation. We should probably verify what we do here.
- */
- adev->gmc.flush_tlb_needs_extra_type_0 =
- amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3) &&
- adev->rev_id == 0;
+ switch (amdgpu_ip_version(adev, GC_HWIP, 0)) {
+ case IP_VERSION(9, 4, 0):
+ adev->gmc.flush_tlb_needs_extra_type_2 =
+ adev->gmc.xgmi.num_physical_nodes;
+ break;
+ case IP_VERSION(9, 4, 3):
+ case IP_VERSION(9, 4, 4):
+ case IP_VERSION(9, 5, 0):
+ adev->gmc.flush_tlb_needs_extra_type_2 = true;
+ break;
+ default:
+ break;
+ }
/* The sequence of these two function calls matters.*/
gmc_v9_0_init_golden_registers(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index fa6c7e5fbbe3..78775b8cfbc6 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -168,8 +168,19 @@ static int ih_v6_0_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index 236fedc5b183..63762ea6f87c 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -201,8 +201,10 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
index c432a2a3405d..98c706fa97fe 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
@@ -187,8 +187,10 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index 17aab897f86b..454a161a189f 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -161,13 +161,52 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = RREG32(ih_regs->ih_rb_cntl);
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0));
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1);
+
+ if (enable) {
+ /* Unset the CLEAR_OVERFLOW bit to make sure the next step
+ * is switching the bit from 0 to 1
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Clear RB_OVERFLOW bit */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 1);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Unset the CLEAR_OVERFLOW bit immediately so new overflows
+ * can be detected.
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ }
+
/* enable_intr field is only valid in ring0 */
if (ih == &adev->irq.ih)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/si.c b/drivers/gpu/drm/amd/amdgpu/si.c
index 93c68abf447a..dc0102db4633 100644
--- a/drivers/gpu/drm/amd/amdgpu/si.c
+++ b/drivers/gpu/drm/amd/amdgpu/si.c
@@ -1509,6 +1509,16 @@ static bool si_need_full_reset(struct amdgpu_device *adev)
static bool si_need_reset_on_init(struct amdgpu_device *adev)
{
+ /*
+ * MacBookPros with switchable graphics put VRAM at 0 when the iGPU
+ * is enabled. Any later post (S3 resume, GPU reset) moves it back to
+ * the vbios default, so reset and post on load to start with the
+ * vbios layout.
+ */
+ if (adev->pdev->subsystem_vendor == PCI_VENDOR_ID_APPLE &&
+ !(RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF))
+ return true;
+
return false;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index fa95bde45f3c..297148b68b86 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -347,11 +347,6 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
f2g = &aldebaran_kfd2kgd;
break;
case IP_VERSION(9, 4, 3):
- gfx_target_version = adev->rev_id >= 1 ? 90402
- : adev->flags & AMD_IS_APU ? 90400
- : 90401;
- f2g = &gc_9_4_3_kfd2kgd;
- break;
case IP_VERSION(9, 4, 4):
gfx_target_version = 90402;
f2g = &gc_9_4_3_kfd2kgd;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c
index e64eba0a9263..7dd458dcab5d 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c
@@ -424,7 +424,7 @@ static u32 kfd_get_vgpr_size_per_cu(u32 gfxv)
{
u32 vgpr_size = 0x40000;
- if ((gfxv / 100 * 100) == 90400 || /* GFX_VERSION_AQUA_VANJARAM */
+ if (gfxv == 90402 || /* GFX_VERSION_AQUA_VANJARAM */
gfxv == 90010 || /* GFX_VERSION_ALDEBARAN */
gfxv == 90008) /* GFX_VERSION_ARCTURUS */
vgpr_size = 0x80000;
@@ -483,7 +483,7 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev)
if (gfxv == 80002) /* GFX_VERSION_TONGA */
props->eop_buffer_size = 0x8000;
- else if ((gfxv / 100 * 100) == 90400) /* GFX_VERSION_AQUA_VANJARAM */
+ else if (gfxv == 90402) /* GFX_VERSION_AQUA_VANJARAM */
props->eop_buffer_size = 4096;
else if (gfxv >= 80000)
props->eop_buffer_size = 4096;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index 7bbeb9768a56..8f114e85ffbb 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -1224,6 +1224,7 @@ svm_range_get_pte_flags(struct kfd_node *node,
struct kfd_node *bo_node;
uint32_t flags = prange->flags;
uint32_t mapping_flags = 0;
+ uint32_t gc_ip_version = KFD_GC_VERSION(node);
uint64_t pte_flags;
bool snoop = (domain != SVM_RANGE_VRAM_DOMAIN);
bool coherent = flags & (KFD_IOCTL_SVM_FLAG_COHERENT | KFD_IOCTL_SVM_FLAG_EXT_COHERENT);
@@ -1233,7 +1234,7 @@ svm_range_get_pte_flags(struct kfd_node *node,
if (domain == SVM_RANGE_VRAM_DOMAIN)
bo_node = prange->svm_bo->node;
- switch (amdgpu_ip_version(node->adev, GC_HWIP, 0)) {
+ switch (gc_ip_version) {
case IP_VERSION(9, 4, 1):
if (domain == SVM_RANGE_VRAM_DOMAIN) {
if (bo_node == node) {
@@ -1271,7 +1272,7 @@ svm_range_get_pte_flags(struct kfd_node *node,
case IP_VERSION(9, 4, 3):
case IP_VERSION(9, 4, 4):
if (ext_coherent)
- mtype_local = node->adev->rev_id ? AMDGPU_VM_MTYPE_CC : AMDGPU_VM_MTYPE_UC;
+ mtype_local = AMDGPU_VM_MTYPE_CC;
else
mtype_local = amdgpu_mtype_local == 1 ? AMDGPU_VM_MTYPE_NC :
amdgpu_mtype_local == 2 ? AMDGPU_VM_MTYPE_CC : AMDGPU_VM_MTYPE_RW;
@@ -1286,9 +1287,13 @@ svm_range_get_pte_flags(struct kfd_node *node,
*/
else if (svm_nodes_in_same_hive(bo_node, node) && !ext_coherent)
mapping_flags |= AMDGPU_VM_MTYPE_NC;
- /* PCIe P2P or extended system scope coherence */
- else
+ /* PCIe P2P on GPUs pre-9.5.0 */
+ else if (gc_ip_version < IP_VERSION(9, 5, 0) &&
+ !svm_nodes_in_same_hive(bo_node, node))
mapping_flags |= AMDGPU_VM_MTYPE_UC;
+ /* Other remote memory */
+ else
+ mapping_flags |= ext_coherent ? AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC;
/* system memory accessed by the APU */
} else if (node->adev->flags & AMD_IS_APU) {
/* On NUMA systems, locality is determined per-page
@@ -1300,7 +1305,10 @@ svm_range_get_pte_flags(struct kfd_node *node,
mapping_flags |= ext_coherent ? AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC;
/* system memory accessed by the dGPU */
} else {
- mapping_flags |= AMDGPU_VM_MTYPE_UC;
+ if (gc_ip_version < IP_VERSION(9, 5, 0))
+ mapping_flags |= AMDGPU_VM_MTYPE_UC;
+ else
+ mapping_flags |= AMDGPU_VM_MTYPE_NC;
}
break;
case IP_VERSION(12, 0, 0):
@@ -1322,7 +1330,7 @@ svm_range_get_pte_flags(struct kfd_node *node,
pte_flags = AMDGPU_PTE_VALID;
pte_flags |= (domain == SVM_RANGE_VRAM_DOMAIN) ? 0 : AMDGPU_PTE_SYSTEM;
pte_flags |= snoop ? AMDGPU_PTE_SNOOPED : 0;
- if (KFD_GC_VERSION(node) >= IP_VERSION(12, 0, 0))
+ if (gc_ip_version >= IP_VERSION(12, 0, 0))
pte_flags |= AMDGPU_PTE_IS_PTE;
pte_flags |= amdgpu_gem_va_map_flags(node->adev, mapping_flags);
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
index 2ee034879f9f..24b1103c88c8 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
@@ -511,6 +511,9 @@ static void dcn401_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr
otg_master = resource_get_otg_master_for_stream(
&context->res_ctx, context->streams[i]);
ASSERT(otg_master);
+ if (!otg_master)
+ continue;
+
ASSERT(otg_master->clock_source);
ASSERT(otg_master->clock_source->funcs->program_pix_clk);
ASSERT(otg_master->stream_res.pix_clk_params.controller_id >= CONTROLLER_ID_D0);
diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
index fa29eb177331..896c69352482 100644
--- a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
+++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
@@ -2328,12 +2328,8 @@ bool dp_is_sink_present(struct dc_link *link)
* which indicates we need additional delay
*/
- if (dal_ddc_open(ddc, GPIO_MODE_INPUT,
- GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) {
- dal_ddc_close(ddc);
-
+ if (dal_ddc_open(ddc, GPIO_MODE_INPUT, GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK)
return present;
- }
/*
* Read GPIO: DP sink is present if both clock and data pins are zero
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
index b4694985a40a..d70b14f54519 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
@@ -452,6 +452,13 @@ void optc2_lock_doublebuffer_disable(struct timing_generator *optc)
void optc2_setup_manual_trigger(struct timing_generator *optc)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
+ uint32_t v_total, v_total_max;
+
+ REG_GET(OTG_V_TOTAL_MAX, OTG_V_TOTAL_MAX, &v_total_max);
+ if (!v_total_max) {
+ REG_GET(OTG_V_TOTAL, OTG_V_TOTAL, &v_total);
+ optc->funcs->set_vtotal_min_max(optc, v_total, v_total);
+ }
/* Set the min/max selectors unconditionally so that
* DMCUB fw may change OTG timings when necessary
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
index 1fa4d32ce0be..c79a175d4d9a 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
@@ -1775,8 +1775,21 @@ static bool dml1_validate(struct dc *dc, struct dc_state *context, bool fast_val
out = dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, fast_validate);
DC_FP_END();
- if (pipe_cnt == 0)
+ if (pipe_cnt == 0) {
+ /* The context is a copy of the previous state, so without this the
+ * clocks the last display needed stay applied with all displays off.
+ */
+ context->bw_ctx.bw.dcn.clk.socclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dramclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.fclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.fclk_p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.num_ways = 0;
goto validate_out;
+ }
if (!out)
goto validate_fail;
diff --git a/drivers/gpu/drm/amd/include/kgd_pp_interface.h b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
index a97b03232c0b..a42e6c8710a2 100644
--- a/drivers/gpu/drm/amd/include/kgd_pp_interface.h
+++ b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
@@ -372,6 +372,8 @@ struct amd_pm_funcs {
int (*print_clock_levels)(void *handle, enum pp_clock_type type, char *buf);
int (*emit_clock_levels)(void *handle, enum pp_clock_type type, char *buf, int *offset);
int (*force_performance_level)(void *handle, enum amd_dpm_forced_level level);
+ int (*reapply_performance_level)(void *handle,
+ enum amd_dpm_forced_level level);
int (*get_sclk_od)(void *handle);
int (*set_sclk_od)(void *handle, uint32_t value);
int (*get_mclk_od)(void *handle);
diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
index 0f37d5ccf246..e096efd77e73 100644
--- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
+++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
@@ -974,6 +974,109 @@ enum amd_dpm_forced_level amdgpu_dpm_get_performance_level(struct amdgpu_device
return level;
}
+static void amdgpu_dpm_enter_umd_state(struct amdgpu_device *adev)
+{
+ /* enter UMD Pstate */
+ amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_PG_STATE_UNGATE);
+ amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_CG_STATE_UNGATE);
+}
+
+static void amdgpu_dpm_exit_umd_state(struct amdgpu_device *adev)
+{
+ /* exit UMD Pstate */
+ amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_CG_STATE_GATE);
+ amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_PG_STATE_GATE);
+}
+
+/**
+ * amdgpu_dpm_reapply_umd_profile - Restore the active UMD profile
+ * @adev: amdgpu device
+ *
+ * Resume and reset retain the selected DPM level in software, but the
+ * underlying GFX and PMFW state is lost. Reapply a retained UMD profile
+ * after the common resume path has restored its default CG and PG state.
+ */
+int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev)
+{
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
+ enum amd_dpm_forced_level level;
+ int ret, fallback_ret;
+
+ if (!pp_funcs || !pp_funcs->reapply_performance_level ||
+ !pp_funcs->force_performance_level)
+ return 0;
+
+ /* Serialize the retained level with the GFX and PMFW reprogramming. */
+ mutex_lock(&adev->pm.mutex);
+ if (adev->pm.dpm.thermal_active) {
+ dev_warn(adev->dev,
+ "Cannot restore UMD profile while thermal control is active\n");
+ ret = 0;
+ goto unlock;
+ }
+
+ if (pp_funcs->get_performance_level)
+ level = pp_funcs->get_performance_level(adev->powerplay.pp_handle);
+ else
+ level = adev->pm.dpm.forced_level;
+
+ switch (level) {
+ case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
+ break;
+ default:
+ ret = 0;
+ goto unlock;
+ }
+
+ /* The active profile already owns the GFX PG reference. */
+ ret = amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_CG_STATE_UNGATE);
+ if (ret)
+ goto fallback_auto;
+
+ ret = amdgpu_asic_update_umd_stable_pstate(adev, true);
+ if (ret)
+ goto fallback_auto;
+
+ ret = pp_funcs->reapply_performance_level(adev->powerplay.pp_handle,
+ level);
+ if (ret)
+ goto fallback_auto;
+
+unlock:
+ mutex_unlock(&adev->pm.mutex);
+
+ return ret;
+
+fallback_auto:
+ dev_warn(adev->dev, "Failed to restore UMD profile state: %d\n", ret);
+
+ /* Restore normal CG before releasing the profile's PG reference. */
+ amdgpu_asic_update_umd_stable_pstate(adev, false);
+ amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_CG_STATE_GATE);
+ mutex_unlock(&adev->pm.mutex);
+
+ mutex_lock(&adev->pm.stable_pstate_ctx_lock);
+ fallback_ret = amdgpu_dpm_force_performance_level(adev,
+ AMD_DPM_FORCED_LEVEL_AUTO);
+ if (fallback_ret)
+ dev_warn(adev->dev, "Failed to fall back to AUTO performance level: %d\n",
+ fallback_ret);
+ else
+ adev->pm.stable_pstate_ctx = NULL;
+ mutex_unlock(&adev->pm.stable_pstate_ctx_lock);
+
+ return 0;
+}
+
int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
enum amd_dpm_forced_level level)
{
@@ -994,6 +1097,10 @@ int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
if (current_level == level)
return 0;
+ if (!(current_level & profile_mode_mask) &&
+ (level == AMD_DPM_FORCED_LEVEL_PROFILE_EXIT))
+ return -EINVAL;
+
if (adev->asic_type == CHIP_RAVEN) {
if (!(adev->apu_flags & AMD_APU_IS_RAVEN2)) {
if (current_level != AMD_DPM_FORCED_LEVEL_MANUAL &&
@@ -1005,35 +1112,25 @@ int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
}
}
- if (!(current_level & profile_mode_mask) &&
- (level == AMD_DPM_FORCED_LEVEL_PROFILE_EXIT))
- return -EINVAL;
-
- if (!(current_level & profile_mode_mask) &&
- (level & profile_mode_mask)) {
- /* enter UMD Pstate */
- amdgpu_device_ip_set_powergating_state(adev,
- AMD_IP_BLOCK_TYPE_GFX,
- AMD_PG_STATE_UNGATE);
- amdgpu_device_ip_set_clockgating_state(adev,
- AMD_IP_BLOCK_TYPE_GFX,
- AMD_CG_STATE_UNGATE);
- } else if ((current_level & profile_mode_mask) &&
- !(level & profile_mode_mask)) {
- /* exit UMD Pstate */
- amdgpu_device_ip_set_clockgating_state(adev,
- AMD_IP_BLOCK_TYPE_GFX,
- AMD_CG_STATE_GATE);
- amdgpu_device_ip_set_powergating_state(adev,
- AMD_IP_BLOCK_TYPE_GFX,
- AMD_PG_STATE_GATE);
- }
+ if (!(current_level & profile_mode_mask) && (level & profile_mode_mask))
+ amdgpu_dpm_enter_umd_state(adev);
+ else if ((current_level & profile_mode_mask) &&
+ !(level & profile_mode_mask))
+ amdgpu_dpm_exit_umd_state(adev);
mutex_lock(&adev->pm.mutex);
if (pp_funcs->force_performance_level(adev->powerplay.pp_handle,
level)) {
mutex_unlock(&adev->pm.mutex);
+ /* If new level failed, retain the umd state as before */
+ if (!(current_level & profile_mode_mask) &&
+ (level & profile_mode_mask))
+ amdgpu_dpm_exit_umd_state(adev);
+ else if ((current_level & profile_mode_mask) &&
+ !(level & profile_mode_mask))
+ amdgpu_dpm_enter_umd_state(adev);
+
return -EINVAL;
}
diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
index 3e27050ccd12..ff49e03626c6 100644
--- a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
+++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
@@ -480,6 +480,7 @@ void amdgpu_dpm_set_power_state(struct amdgpu_device *adev,
enum amd_dpm_forced_level amdgpu_dpm_get_performance_level(struct amdgpu_device *adev);
int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
enum amd_dpm_forced_level level);
+int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev);
int amdgpu_dpm_get_pp_num_states(struct amdgpu_device *adev,
struct pp_states_info *states);
int amdgpu_dpm_dispatch_task(struct amdgpu_device *adev,
diff --git a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
index b1526f80c972..58dc4a5920a5 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
@@ -383,24 +383,46 @@ static void pp_dpm_en_umd_pstate(struct pp_hwmgr *hwmgr,
}
}
-static int pp_dpm_force_performance_level(void *handle,
- enum amd_dpm_forced_level level)
+static int pp_dpm_apply_performance_level(struct pp_hwmgr *hwmgr,
+ enum amd_dpm_forced_level level,
+ bool reapply)
{
- struct pp_hwmgr *hwmgr = handle;
+ int ret;
if (!hwmgr || !hwmgr->pm_en)
return -EINVAL;
- if (level == hwmgr->dpm_level)
+ if (!reapply && level == hwmgr->dpm_level)
return 0;
pp_dpm_en_umd_pstate(hwmgr, &level);
hwmgr->request_dpm_level = level;
- hwmgr_handle_task(hwmgr, AMD_PP_TASK_READJUST_POWER_STATE, NULL);
+ ret = hwmgr_handle_task(hwmgr, AMD_PP_TASK_READJUST_POWER_STATE, NULL);
+ if (ret)
+ return ret;
+
+ ret = phm_force_dpm_levels(hwmgr, level);
+ if (ret)
+ return ret;
+
+ hwmgr->dpm_level = level;
return 0;
}
+static int pp_dpm_force_performance_level(void *handle,
+ enum amd_dpm_forced_level level)
+{
+ return pp_dpm_apply_performance_level(handle, level, false);
+}
+
+static int
+pp_dpm_reapply_performance_level(void *handle,
+ enum amd_dpm_forced_level level)
+{
+ return pp_dpm_apply_performance_level(handle, level, true);
+}
+
static enum amd_dpm_forced_level pp_dpm_get_performance_level(
void *handle)
{
@@ -1583,6 +1605,7 @@ static const struct amd_pm_funcs pp_dpm_funcs = {
.load_firmware = pp_dpm_load_fw,
.wait_for_fw_loading_complete = pp_dpm_fw_loading_complete,
.force_performance_level = pp_dpm_force_performance_level,
+ .reapply_performance_level = pp_dpm_reapply_performance_level,
.get_performance_level = pp_dpm_get_performance_level,
.get_current_power_state = pp_dpm_get_current_power_state,
.dispatch_tasks = pp_dpm_dispatch_tasks,
diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index 83f666dfe7bd..1963653a5c60 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -2214,6 +2214,13 @@ static int smu_set_powergating_state(void *handle,
return 0;
}
+static void smu_enter_umd_pstate(struct smu_context *smu)
+{
+ smu_gpo_control(smu, false);
+ smu_gfx_ulv_control(smu, false);
+ smu_deep_sleep_control(smu, false);
+}
+
static int smu_enable_umd_pstate(void *handle,
enum amd_dpm_forced_level *level)
{
@@ -2232,9 +2239,7 @@ static int smu_enable_umd_pstate(void *handle,
/* enter umd pstate, save current level, disable gfx cg*/
if (*level & profile_mode_mask) {
smu_dpm_ctx->saved_dpm_level = smu_dpm_ctx->dpm_level;
- smu_gpo_control(smu, false);
- smu_gfx_ulv_control(smu, false);
- smu_deep_sleep_control(smu, false);
+ smu_enter_umd_pstate(smu);
amdgpu_asic_update_umd_stable_pstate(smu->adev, true);
}
} else {
@@ -2293,7 +2298,8 @@ static void smu_power_profile_mode_put(struct smu_context *smu,
static int smu_adjust_power_state_dynamic(struct smu_context *smu,
enum amd_dpm_forced_level level,
- bool skip_display_settings)
+ bool skip_display_settings,
+ bool force_performance_level)
{
int ret = 0;
struct smu_dpm_context *smu_dpm_ctx = &(smu->smu_dpm);
@@ -2320,7 +2326,7 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu,
}
}
- if (smu_dpm_ctx->dpm_level != level) {
+ if (force_performance_level || smu_dpm_ctx->dpm_level != level) {
ret = smu_asic_set_performance_level(smu, level);
if (ret) {
dev_err(smu->adev->dev, "Failed to set performance level!");
@@ -2338,6 +2344,39 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu,
return ret;
}
+static int
+smu_reapply_power_state_dynamic(struct smu_context *smu,
+ enum amd_dpm_forced_level level)
+{
+ struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm;
+ int ret;
+
+ ret = smu_apply_clocks_adjust_rules(smu);
+ if (ret) {
+ dev_err(smu->adev->dev, "Failed to apply clocks adjust rules!");
+ return ret;
+ }
+
+ ret = smu_asic_set_performance_level(smu, level);
+ if (ret) {
+ if (ret == -EOPNOTSUPP)
+ dev_info(smu->adev->dev, "set performance level %d not supported",
+ level);
+ else
+ dev_err(smu->adev->dev, "Failed to set performance level %d",
+ level);
+ return ret;
+ }
+
+ smu_dpm_ctx->dpm_level = level;
+
+ if (smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_MANUAL ||
+ smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM)
+ return 0;
+
+ return smu_bump_power_profile_mode(smu, NULL, 0);
+}
+
static int smu_handle_task(struct smu_context *smu,
enum amd_dpm_forced_level level,
enum amd_pp_task task_id)
@@ -2352,13 +2391,13 @@ static int smu_handle_task(struct smu_context *smu,
ret = smu_pre_display_config_changed(smu);
if (ret)
return ret;
- ret = smu_adjust_power_state_dynamic(smu, level, false);
+ ret = smu_adjust_power_state_dynamic(smu, level, false, false);
break;
case AMD_PP_TASK_COMPLETE_INIT:
- ret = smu_adjust_power_state_dynamic(smu, level, true);
+ ret = smu_adjust_power_state_dynamic(smu, level, true, false);
break;
case AMD_PP_TASK_READJUST_POWER_STATE:
- ret = smu_adjust_power_state_dynamic(smu, level, true);
+ ret = smu_adjust_power_state_dynamic(smu, level, true, false);
break;
default:
break;
@@ -2454,6 +2493,37 @@ static int smu_force_performance_level(void *handle,
return ret;
}
+static int
+smu_reapply_performance_level(void *handle,
+ enum amd_dpm_forced_level level)
+{
+ struct smu_context *smu = handle;
+ struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm;
+
+ if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
+ return -EOPNOTSUPP;
+
+ if (!smu->is_apu && !smu_dpm_ctx->dpm_context)
+ return -EINVAL;
+
+ switch (level) {
+ case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (smu_dpm_ctx->dpm_level != level)
+ return -EINVAL;
+
+ smu_enter_umd_pstate(smu);
+
+ return smu_reapply_power_state_dynamic(smu, level);
+}
+
static int smu_set_display_count(void *handle, uint32_t count)
{
struct smu_context *smu = handle;
@@ -3711,6 +3781,7 @@ static const struct amd_pm_funcs swsmu_pm_funcs = {
.print_clock_levels = smu_print_ppclk_levels,
.emit_clock_levels = smu_emit_ppclk_levels,
.force_performance_level = smu_force_performance_level,
+ .reapply_performance_level = smu_reapply_performance_level,
.read_sensor = smu_read_sensor,
.get_apu_thermal_limit = smu_get_apu_thermal_limit,
.set_apu_thermal_limit = smu_set_apu_thermal_limit,
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
index 0145aff45c38..9521be67373a 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
@@ -358,6 +358,8 @@ struct smu_table_context {
void *boot_overdrive_table;
void *user_overdrive_table;
+ uint32_t user_od_feature_mask;
+
uint32_t gpu_metrics_table_size;
void *gpu_metrics_table;
};
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
index 6d178621ade1..5793f85d16fe 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
@@ -1610,7 +1610,6 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
int i, ret = 0;
switch (type) {
@@ -1890,21 +1889,16 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ table_context->user_od_feature_mask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
- /*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
- */
+ table_context->user_od_feature_mask =
+ od_table->OverDriveTable.FeatureCtrlMask;
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
if (memcmp((u8 *)od_table + offset_of_voltageoffset,
table_context->user_overdrive_table + offset_of_voltageoffset,
@@ -1917,7 +1911,6 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
return ret;
}
- od_table->OverDriveTable.FeatureCtrlMask = 0;
memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
@@ -2191,6 +2184,7 @@ static int smu_v13_0_0_set_default_od_settings(struct smu_context *smu)
boot_od_table,
sizeof(OverDriveTableExternal_t));
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2236,11 +2230,8 @@ static int smu_v13_0_0_restore_user_od_settings(struct smu_context *smu)
OverDriveTableExternal_t *od_table = table_context->overdrive_table;
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
int res;
-
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ user_od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
res = smu_v13_0_0_upload_overdrive_table(smu, user_od_table);
user_od_table->OverDriveTable.FeatureCtrlMask = 0;
if (res == 0)
@@ -3060,8 +3051,10 @@ static int smu_v13_0_0_set_power_limit(struct smu_context *smu,
if (limit <= msg_limit) {
if (smu->current_power_limit > msg_limit) {
od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
+ table_context->user_od_feature_mask &=
+ ~(1U << PP_OD_FEATURE_PPT_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_0_upload_overdrive_table(smu, od_table);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
@@ -3075,8 +3068,9 @@ static int smu_v13_0_0_set_power_limit(struct smu_context *smu,
return ret;
od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
+ table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_0_upload_overdrive_table(smu, od_table);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
index 7a0e82a96a1c..9467db88969c 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
@@ -1598,7 +1598,6 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
int i, ret = 0;
switch (type) {
@@ -1878,22 +1877,17 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ table_context->user_od_feature_mask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
- /*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
- */
+ table_context->user_od_feature_mask =
+ od_table->OverDriveTable.FeatureCtrlMask;
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
if (memcmp((u8 *)od_table + offset_of_voltageoffset,
table_context->user_overdrive_table + offset_of_voltageoffset,
@@ -1906,7 +1900,6 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
return ret;
}
- od_table->OverDriveTable.FeatureCtrlMask = 0;
memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
@@ -2174,6 +2167,7 @@ static int smu_v13_0_7_set_default_od_settings(struct smu_context *smu)
boot_od_table,
sizeof(OverDriveTableExternal_t));
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2220,10 +2214,8 @@ static int smu_v13_0_7_restore_user_od_settings(struct smu_context *smu)
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
int res;
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ user_od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
res = smu_v13_0_7_upload_overdrive_table(smu, user_od_table);
user_od_table->OverDriveTable.FeatureCtrlMask = 0;
if (res == 0)
@@ -2641,7 +2633,10 @@ static int smu_v13_0_7_set_power_limit(struct smu_context *smu,
if (limit <= msg_limit) {
if (smu->current_power_limit > msg_limit) {
od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask &=
+ ~(1U << PP_OD_FEATURE_PPT_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_7_upload_overdrive_table(smu, od_table);
if (ret) {
@@ -2656,7 +2651,9 @@ static int smu_v13_0_7_set_power_limit(struct smu_context *smu,
return ret;
od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v13_0_7_upload_overdrive_table(smu, od_table);
if (ret) {
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
index b2c32b2d2e51..33da78b53a86 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
@@ -2350,6 +2350,7 @@ static int smu_v14_0_2_set_default_od_settings(struct smu_context *smu)
boot_od_table,
sizeof(OverDriveTableExternal_t));
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2394,11 +2395,8 @@ static int smu_v14_0_2_restore_user_od_settings(struct smu_context *smu)
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
int res;
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT) |
- BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
+ user_od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
res = smu_v14_0_2_upload_overdrive_table(smu, user_od_table);
user_od_table->OverDriveTable.FeatureCtrlMask = 0;
if (res == 0)
@@ -2473,7 +2471,6 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
int i, ret = 0;
switch (type) {
@@ -2720,21 +2717,26 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ table_context->user_od_feature_mask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
/*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
+ * PMFW interprets FeatureCtrlMask as the complete set of OD
+ * features that are currently overridden, not as a delta
+ * describing this single operation. Any feature whose bit is
+ * clear gets reset to its default on import. od_table's mask
+ * has accumulated every feature the user touched since the
+ * last full restore, so record it and keep restating it on
+ * subsequent uploads.
*/
+ table_context->user_od_feature_mask =
+ od_table->OverDriveTable.FeatureCtrlMask;
+
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
if (memcmp((u8 *)od_table + offset_of_voltageoffset,
table_context->user_overdrive_table + offset_of_voltageoffset,
@@ -2747,7 +2749,6 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
return ret;
}
- od_table->OverDriveTable.FeatureCtrlMask = 0;
memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
@@ -2785,7 +2786,10 @@ static int smu_v14_0_2_set_power_limit(struct smu_context *smu,
if (limit <= msg_limit) {
if (smu->current_power_limit > msg_limit) {
od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask &=
+ ~(1U << PP_OD_FEATURE_PPT_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v14_0_2_upload_overdrive_table(smu, od_table);
if (ret) {
@@ -2800,7 +2804,9 @@ static int smu_v14_0_2_set_power_limit(struct smu_context *smu,
return ret;
od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT;
+ od_table->OverDriveTable.FeatureCtrlMask =
+ table_context->user_od_feature_mask;
ret = smu_v14_0_2_upload_overdrive_table(smu, od_table);
if (ret) {
diff --git a/drivers/gpu/drm/bridge/nwl-dsi.c b/drivers/gpu/drm/bridge/nwl-dsi.c
index 5f05647a3bea..00622febf01c 100644
--- a/drivers/gpu/drm/bridge/nwl-dsi.c
+++ b/drivers/gpu/drm/bridge/nwl-dsi.c
@@ -110,6 +110,8 @@ struct nwl_dsi {
int error;
struct nwl_dsi_transfer *xfer;
+
+ unsigned int endpoint;
};
static const struct regmap_config nwl_dsi_regmap_config = {
@@ -822,10 +824,13 @@ static int nwl_dsi_bridge_atomic_check(struct drm_bridge *bridge,
struct drm_connector_state *conn_state)
{
struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
+ struct nwl_dsi *dsi = bridge_to_dsi(bridge);
/* At least LCDIF + NWL needs active high sync */
- adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
- adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ if (dsi->endpoint == NWL_DSI_ENDPOINT_LCDIF) {
+ adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
+ adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ }
/*
* Do a full modeset if crtc_state->active is changed to be true.
@@ -1089,13 +1094,12 @@ static int nwl_dsi_parse_dt(struct nwl_dsi *dsi)
static int nwl_dsi_select_input(struct nwl_dsi *dsi)
{
struct device_node *remote;
- u32 use_dcss = 1;
int ret;
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_LCDIF);
if (remote) {
- use_dcss = 0;
+ dsi->endpoint = NWL_DSI_ENDPOINT_LCDIF;
} else {
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_DCSS);
@@ -1104,11 +1108,12 @@ static int nwl_dsi_select_input(struct nwl_dsi *dsi)
"No valid input endpoint found\n");
return -EINVAL;
}
+ dsi->endpoint = NWL_DSI_ENDPOINT_DCSS;
}
DRM_DEV_INFO(dsi->dev, "Using %s as input source\n",
- (use_dcss) ? "DCSS" : "LCDIF");
- ret = mux_control_try_select(dsi->mux, use_dcss);
+ (dsi->endpoint == NWL_DSI_ENDPOINT_DCSS) ? "DCSS" : "LCDIF");
+ ret = mux_control_try_select(dsi->mux, dsi->endpoint);
if (ret < 0)
DRM_DEV_ERROR(dsi->dev, "Failed to select input: %d\n", ret);
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index 3488ff067c69..fdeed38cec03 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -476,14 +476,23 @@ EXPORT_SYMBOL(__drmm_crtc_alloc_with_planes);
* drm_crtc_cleanup - Clean up the core crtc usage
* @crtc: CRTC to cleanup
*
- * This function cleans up @crtc and removes it from the DRM mode setting
- * core. Note that the function does *not* free the crtc structure itself,
- * this is the responsibility of the caller.
+ * This function cleans up @crtc and removes it from the DRM mode setting core,
+ * after first waiting an RCU grace period to ensure @crtc->dev can safely be
+ * dereferenced by our dma_fence_ops.
+ *
+ * Note that the function does *not* free the crtc structure itself, this is the
+ * responsibility of the caller.
*/
void drm_crtc_cleanup(struct drm_crtc *crtc)
{
struct drm_device *dev = crtc->dev;
+ /* Ensure our dma_fence_ops remain valid for an RCU grace period after
+ * the fence is signaled. This is necessary because our dma_fence_ops
+ * dereference crtc->dev.
+ */
+ synchronize_rcu();
+
/* Note that the crtc_list is considered to be static; should we
* remove the drm_crtc at runtime we would have to decrement all
* the indices on the drm_crtc after us in the crtc_list.
diff --git a/drivers/gpu/drm/drm_panel.c b/drivers/gpu/drm/drm_panel.c
index fd8fa2e0ef6f..7443aaacdd6c 100644
--- a/drivers/gpu/drm/drm_panel.c
+++ b/drivers/gpu/drm/drm_panel.c
@@ -316,6 +316,31 @@ int drm_panel_get_modes(struct drm_panel *panel,
}
EXPORT_SYMBOL(drm_panel_get_modes);
+void *__devm_drm_panel_alloc(struct device *dev, size_t size, size_t offset,
+ const struct drm_panel_funcs *funcs,
+ int connector_type)
+{
+ void *container;
+ struct drm_panel *panel;
+
+ if (!funcs) {
+ dev_warn(dev, "Missing funcs pointer\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ container = devm_kzalloc(dev, size, GFP_KERNEL);
+ if (!container)
+ return ERR_PTR(-ENOMEM);
+
+ panel = container + offset;
+ panel->funcs = funcs;
+
+ drm_panel_init(panel, dev, funcs, connector_type);
+
+ return container;
+}
+EXPORT_SYMBOL(__devm_drm_panel_alloc);
+
#ifdef CONFIG_OF
/**
* of_drm_find_panel - look up a panel using a device tree node
diff --git a/drivers/gpu/drm/drm_panic.c b/drivers/gpu/drm/drm_panic.c
index 0aa87eafdacd..f4662f88e217 100644
--- a/drivers/gpu/drm/drm_panic.c
+++ b/drivers/gpu/drm/drm_panic.c
@@ -395,7 +395,8 @@ static void draw_panic_static_kmsg(struct drm_scanout_buffer *sb)
const struct font_desc *font = get_default_font(sb->width, sb->height, NULL, NULL);
struct drm_rect r_screen = DRM_RECT_INIT(0, 0, sb->width, sb->height);
struct kmsg_dump_iter iter;
- char kmsg_buf[512];
+ /* the size printk formats one record into, too big for the stack */
+ static char kmsg_buf[2048];
size_t kmsg_len;
struct drm_panic_line line;
int yoffset;
@@ -413,6 +414,10 @@ static void draw_panic_static_kmsg(struct drm_scanout_buffer *sb)
char *start;
char *end;
+ /* a record that does not fit comes back empty, every time */
+ if (!kmsg_len)
+ break;
+
/* ignore terminating NUL and newline */
start = kmsg_buf + kmsg_len - 2;
end = kmsg_buf + kmsg_len - 1;
@@ -502,7 +507,8 @@ static int drm_panic_get_qr_code_url(u8 **qr_image)
char *kmsg;
int max_qr_data_size, url_len;
- url_len = snprintf(url, sizeof(url), CONFIG_DRM_PANIC_SCREEN_QR_CODE_URL "?a=%s&v=%s&zl=",
+ url_len = snprintf(url, sizeof(url), "%s?a=%s&v=%s&zl=",
+ CONFIG_DRM_PANIC_SCREEN_QR_CODE_URL,
utsname()->machine, utsname()->release);
max_qr_data_size = drm_panic_qr_max_data_size(panic_qr_version, url_len);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
index 88385dc3b30d..4f22e373ed34 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
@@ -230,6 +230,17 @@ u16 etnaviv_buffer_config_pta(struct etnaviv_gpu *gpu, unsigned short id)
return buffer->user_size / 8;
}
+static void etnaviv_cmd_flush_ts(struct etnaviv_cmdbuf *buffer, bool has_blt)
+{
+ CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, VIVS_TS_FLUSH_CACHE_FLUSH);
+
+ if (has_blt) {
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
+ }
+}
+
void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
{
struct etnaviv_cmdbuf *buffer = &gpu->buffer;
@@ -253,7 +264,7 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
unsigned int dwords = 7;
if (has_blt)
- dwords += 10;
+ dwords += 11;
link_target = etnaviv_buffer_reserve(gpu, buffer, dwords);
@@ -266,16 +277,8 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
}
CMD_LOAD_STATE(buffer, VIVS_GL_FLUSH_CACHE, flush);
- if (gpu->exec_state == ETNA_PIPE_3D) {
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
- }
+ if (gpu->exec_state == ETNA_PIPE_3D)
+ etnaviv_cmd_flush_ts(buffer, has_blt);
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
if (has_blt) {
@@ -462,12 +465,12 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
return_dwords = 7;
/*
- * When the BLT engine is present we need 6 more dwords in the return
+ * When the BLT engine is present we need 7 more dwords in the return
* target: 3 enable/flush/disable + 4 enable/semaphore stall/disable,
- * but we don't need the normal TS flush state.
+ * on top of the normal TS flush state.
*/
if (has_blt)
- return_dwords += 6;
+ return_dwords += 7;
return_target = etnaviv_buffer_reserve(gpu, buffer, return_dwords);
CMD_LINK(cmdbuf, return_dwords, return_target);
@@ -484,14 +487,7 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
VIVS_GL_FLUSH_CACHE_DEPTH |
VIVS_GL_FLUSH_CACHE_COLOR |
VIVS_GL_FLUSH_CACHE_SHADER_L1);
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
+ etnaviv_cmd_flush_ts(buffer, has_blt);
}
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c
index df16587ea964..97f82949a3a5 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.c
+++ b/drivers/gpu/drm/i915/display/intel_psr.c
@@ -2406,23 +2406,25 @@ static u32 psr2_pipe_srcsz_early_tpt_calc(struct intel_crtc_state *crtc_state,
}
static void clip_area_update(struct drm_rect *overlap_damage_area,
- struct drm_rect *damage_area,
- struct drm_rect *display_area)
+ const struct drm_rect *damage_area,
+ const struct drm_rect *display_area)
{
- if (!drm_rect_intersect(damage_area, display_area))
+ struct drm_rect inter = *damage_area;
+
+ if (!drm_rect_intersect(&inter, display_area))
return;
if (overlap_damage_area->y1 == -1) {
- overlap_damage_area->y1 = damage_area->y1;
- overlap_damage_area->y2 = damage_area->y2;
+ overlap_damage_area->y1 = inter.y1;
+ overlap_damage_area->y2 = inter.y2;
return;
}
- if (damage_area->y1 < overlap_damage_area->y1)
- overlap_damage_area->y1 = damage_area->y1;
+ if (inter.y1 < overlap_damage_area->y1)
+ overlap_damage_area->y1 = inter.y1;
- if (damage_area->y2 > overlap_damage_area->y2)
- overlap_damage_area->y2 = damage_area->y2;
+ if (inter.y2 > overlap_damage_area->y2)
+ overlap_damage_area->y2 = inter.y2;
}
static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index dbe9fc1e8a10..a89e61dd3940 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -2789,7 +2789,6 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
u64 __user *user_values;
struct eb_fence *f;
u64 nfences;
- int err = 0;
nfences = timeline_fences->fence_count;
if (!nfences)
@@ -2827,6 +2826,7 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
struct drm_syncobj *syncobj;
struct dma_fence *fence = NULL;
u64 point;
+ int err = 0;
if (__copy_from_user(&user_fence,
user_fences++,
diff --git a/drivers/gpu/drm/loongson/lsdc_gem.c b/drivers/gpu/drm/loongson/lsdc_gem.c
index a720d8f53209..5eb6b4f92302 100644
--- a/drivers/gpu/drm/loongson/lsdc_gem.c
+++ b/drivers/gpu/drm/loongson/lsdc_gem.c
@@ -155,14 +155,18 @@ struct drm_gem_object *lsdc_gem_object_create(struct drm_device *ddev,
return ERR_PTR(ret);
}
+ gobj = &lbo->tbo.base;
+ gobj->funcs = &lsdc_gem_object_funcs;
+
if (!sg) {
/* VRAM is filled with random data */
- lsdc_bo_clear(lbo);
+ ret = lsdc_bo_clear(lbo);
+ if (ret) {
+ drm_gem_object_put(gobj);
+ return ERR_PTR(ret);
+ }
}
- gobj = &lbo->tbo.base;
- gobj->funcs = &lsdc_gem_object_funcs;
-
/* tracking the BOs we created */
mutex_lock(&ldev->gem.mutex);
list_add_tail(&lbo->list, &ldev->gem.objects);
diff --git a/drivers/gpu/drm/loongson/lsdc_plane.c b/drivers/gpu/drm/loongson/lsdc_plane.c
index d227a2c1dcf1..df6cc5d11d79 100644
--- a/drivers/gpu/drm/loongson/lsdc_plane.c
+++ b/drivers/gpu/drm/loongson/lsdc_plane.c
@@ -227,7 +227,7 @@ static void lsdc_cursor_plane_atomic_async_update(struct drm_plane *plane,
new_state = drm_atomic_get_new_plane_state(state, plane);
- new_fb = plane->state->fb;
+ new_fb = new_state->fb;
plane->state->crtc_x = new_state->crtc_x;
plane->state->crtc_y = new_state->crtc_y;
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.c b/drivers/gpu/drm/loongson/lsdc_ttm.c
index 2e42c6970c9f..17b1998254e2 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.c
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.c
@@ -387,9 +387,13 @@ void lsdc_bo_kunmap(struct lsdc_bo *lbo)
ttm_bo_kunmap(&lbo->kmap);
}
-void lsdc_bo_clear(struct lsdc_bo *lbo)
+int lsdc_bo_clear(struct lsdc_bo *lbo)
{
- lsdc_bo_kmap(lbo);
+ int ret;
+
+ ret = lsdc_bo_kmap(lbo);
+ if (ret)
+ return ret;
if (lbo->is_iomem)
memset_io((void __iomem *)lbo->kptr, 0, lbo->size);
@@ -397,6 +401,8 @@ void lsdc_bo_clear(struct lsdc_bo *lbo)
memset(lbo->kptr, 0, lbo->size);
lsdc_bo_kunmap(lbo);
+
+ return 0;
}
int lsdc_bo_evict_vram(struct drm_device *ddev)
@@ -474,10 +480,8 @@ struct lsdc_bo *lsdc_bo_create(struct drm_device *ddev,
ret = ttm_bo_init_validate(bdev, tbo, bo_type, &lbo->placement, 0,
false, sg, resv, lsdc_bo_destroy);
- if (ret) {
- kfree(lbo);
+ if (ret)
return ERR_PTR(ret);
- }
return lbo;
}
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.h b/drivers/gpu/drm/loongson/lsdc_ttm.h
index 843e1475064e..df47d9bb0d03 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.h
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.h
@@ -89,7 +89,7 @@ size_t lsdc_bo_size(struct lsdc_bo *lbo);
int lsdc_bo_kmap(struct lsdc_bo *lbo);
void lsdc_bo_kunmap(struct lsdc_bo *lbo);
-void lsdc_bo_clear(struct lsdc_bo *lbo);
+int lsdc_bo_clear(struct lsdc_bo *lbo);
int lsdc_bo_evict_vram(struct drm_device *ddev);
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index cd605c75cef8..ebd9dfca6459 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -1205,6 +1205,8 @@ int dsi_tx_buf_alloc_6g(struct msm_dsi_host *msm_host, int size)
if (IS_ERR(data)) {
msm_host->tx_gem_obj = NULL;
+ msm_gem_address_space_put(msm_host->aspace);
+ msm_host->aspace = NULL;
return PTR_ERR(data);
}
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
index 3c103101d5fc..f21d6f1cf4fe 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
@@ -46,6 +46,7 @@ int nv4c_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct n
int g84_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int g92_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int g94_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
+int mcp79_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int gf100_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int gf106_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int gk104_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
index 6c20e827a069..3ceca50fd8d4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
@@ -145,6 +145,7 @@ nvkm_subdev_init_(struct nvkm_subdev *subdev)
ret = subdev->func->init(subdev);
if (ret) {
nvkm_error(subdev, "init failed, %d\n", ret);
+ subdev->use.enabled = false;
return ret;
}
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
index 9093d89b16f3..6f13b8b0fc77 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
@@ -1237,7 +1237,7 @@ nvac_chipset = {
.mc = { 0x00000001, g98_mc_new },
.mmu = { 0x00000001, mcp77_mmu_new },
.mxm = { 0x00000001, nv50_mxm_new },
- .pci = { 0x00000001, g94_pci_new },
+ .pci = { 0x00000001, mcp79_pci_new },
.therm = { 0x00000001, g84_therm_new },
.timer = { 0x00000001, nv41_timer_new },
.volt = { 0x00000001, nv40_volt_new },
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
index 5368e705e7fd..186e09ea0cb1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
@@ -27,6 +27,7 @@
#include <subdev/bios/init.h>
#include <subdev/bios/pll.h>
#include <subdev/clk/pll.h>
+#include <subdev/vga.h>
int
gf100_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq)
@@ -99,6 +100,25 @@ gf100_devinit_preinit(struct nvkm_devinit *base)
* can use it as a reliable way to know whether we should run devinit.
*/
base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0);
+
+ /*
+ * However some VBIOS init sequences miss this bit. So fall back to
+ * the nv50 method of checking if the bit is not set. Running the
+ * VBIOS multiple times may have detrimental effects.
+ */
+ if (base->post) {
+ u64 disable = nvkm_devinit_disable(base);
+ /* magic to detect whether or not x86 vbios code has
+ * executed the devinit scripts to initialise the
+ * board. only works if there's a display engine.
+ */
+ if (!(disable & (1ULL << NVKM_ENGINE_DISP))) {
+ if (nvkm_rdvgac(device, 0, 0x00) ||
+ nvkm_rdvgac(device, 0, 0x1a)) {
+ base->post = false;
+ }
+ }
+ }
}
static const struct nvkm_devinit_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild
index 174bdf995271..c5911d7d08ea 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild
@@ -9,6 +9,7 @@ nvkm-y += nvkm/subdev/pci/nv4c.o
nvkm-y += nvkm/subdev/pci/g84.o
nvkm-y += nvkm/subdev/pci/g92.o
nvkm-y += nvkm/subdev/pci/g94.o
+nvkm-y += nvkm/subdev/pci/mcp79.o
nvkm-y += nvkm/subdev/pci/gf100.o
nvkm-y += nvkm/subdev/pci/gf106.o
nvkm-y += nvkm/subdev/pci/gk104.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c
new file mode 100644
index 000000000000..98d85a856b88
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright 2026 Marek Czernohous
+ *
+ * MCP79/MCP7A (NVAC): like g94, but MSI re-arm goes through real PCI
+ * config space. The MMIO-mirror re-arm is unreliable on this IGP and a
+ * missed re-arm kills the interrupt line (see the nv46 comment; g92
+ * already re-arms through config space for the same reason).
+ */
+#include "priv.h"
+
+static const struct nvkm_pci_func
+mcp79_pci_func = {
+ .init = g84_pci_init,
+ .rd32 = nv40_pci_rd32,
+ .wr08 = nv40_pci_wr08,
+ .wr32 = nv40_pci_wr32,
+ .msi_rearm = nv46_pci_msi_rearm,
+
+ .pcie.init = g84_pcie_init,
+ .pcie.set_link = g84_pcie_set_link,
+
+ .pcie.max_speed = g84_pcie_max_speed,
+ .pcie.cur_speed = g84_pcie_cur_speed,
+
+ .pcie.set_version = g84_pcie_set_version,
+ .pcie.version = g84_pcie_version,
+ .pcie.version_supported = g92_pcie_version_supported,
+};
+
+int
+mcp79_pci_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_pci **ppci)
+{
+ return nvkm_pci_new_(&mcp79_pci_func, device, type, inst, ppci);
+}
diff --git a/drivers/gpu/drm/panel/panel-ebbg-ft8719.c b/drivers/gpu/drm/panel/panel-ebbg-ft8719.c
index e85d63a176d0..c97825177693 100644
--- a/drivers/gpu/drm/panel/panel-ebbg-ft8719.c
+++ b/drivers/gpu/drm/panel/panel-ebbg-ft8719.c
@@ -198,9 +198,11 @@ static int ebbg_ft8719_probe(struct mipi_dsi_device *dsi)
struct ebbg_ft8719 *ctx;
int i, ret;
- ctx = devm_kzalloc(dev, sizeof(*ctx), GFP_KERNEL);
- if (!ctx)
- return -ENOMEM;
+ ctx = devm_drm_panel_alloc(dev, struct ebbg_ft8719, panel,
+ &ebbg_ft8719_panel_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
for (i = 0; i < ARRAY_SIZE(ctx->supplies); i++)
ctx->supplies[i].supply = regulator_names[i];
@@ -231,8 +233,7 @@ static int ebbg_ft8719_probe(struct mipi_dsi_device *dsi)
dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
MIPI_DSI_CLOCK_NON_CONTINUOUS;
- drm_panel_init(&ctx->panel, dev, &ebbg_ft8719_panel_funcs,
- DRM_MODE_CONNECTOR_DSI);
+ ctx->panel.prepare_prev_first = true;
ret = drm_panel_of_backlight(&ctx->panel);
if (ret)
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index 70bd566cf6d4..2dd678fd8607 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -395,6 +395,8 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
dma_fence_put(job->base.irq_fence);
job->base.irq_fence = dma_fence_get(fence);
+ job->timedout_batches = job->args.cfg[4];
+
trace_v3d_submit_csd(dev, to_v3d_fence(fence)->seqno);
v3d_job_start_stats(&job->base, V3D_CSD);
diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c
index bec799d41d62..f7df302e5c67 100644
--- a/drivers/gpu/drm/vc4/vc4_gem.c
+++ b/drivers/gpu/drm/vc4/vc4_gem.c
@@ -706,8 +706,11 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
fence->seqno = exec->seqno;
exec->fence = &fence->base;
+ /* The job can complete and drop exec->fence as soon as job_lock is
+ * released, so hold our own reference.
+ */
if (out_sync)
- drm_syncobj_replace_fence(out_sync, exec->fence);
+ dma_fence_get(&fence->base);
vc4_update_bo_seqnos(exec, seqno);
@@ -729,6 +732,11 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
spin_unlock_irqrestore(&vc4->job_lock, irqflags);
+ if (out_sync) {
+ drm_syncobj_replace_fence(out_sync, &fence->base);
+ dma_fence_put(&fence->base);
+ }
+
return 0;
}
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c
index 7ea1622377b5..2fa0e0f4bcd2 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.c
@@ -806,8 +806,7 @@ static void vc4_hdmi_disable_scrambling(struct drm_encoder *encoder)
vc4_hdmi->scdc_enabled = false;
- if (delayed_work_pending(&vc4_hdmi->scrambling_work))
- cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
+ cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
if (!drm_dev_enter(drm, &idx))
return;
@@ -870,8 +869,6 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder,
HDMI_READ(HDMI_VID_CTL) & ~VC4_HD_VID_CTL_ENABLE);
spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
- vc4_hdmi_disable_scrambling(encoder);
-
drm_dev_exit(idx);
out:
@@ -900,6 +897,8 @@ static void vc4_hdmi_encoder_post_crtc_powerdown(struct drm_encoder *encoder,
if (vc4_hdmi->variant->phy_disable)
vc4_hdmi->variant->phy_disable(vc4_hdmi);
+ vc4_hdmi_disable_scrambling(encoder);
+
clk_disable_unprepare(vc4_hdmi->pixel_bvb_clock);
clk_disable_unprepare(vc4_hdmi->pixel_clock);
@@ -1510,6 +1509,8 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder,
goto err_disable_pixel_clock;
}
+ vc4_hdmi_enable_scrambling(encoder);
+
if (vc4_hdmi->variant->phy_init)
vc4_hdmi->variant->phy_init(vc4_hdmi, conn_state);
@@ -1641,7 +1642,6 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
}
vc4_hdmi_recenter_fifo(vc4_hdmi);
- vc4_hdmi_enable_scrambling(encoder);
drm_dev_exit(idx);
diff --git a/drivers/hte/hte-tegra194-test.c b/drivers/hte/hte-tegra194-test.c
index df631b5100d2..0bec696774c1 100644
--- a/drivers/hte/hte-tegra194-test.c
+++ b/drivers/hte/hte-tegra194-test.c
@@ -218,10 +218,10 @@ static void tegra_hte_test_remove(struct platform_device *pdev)
{
(void)pdev;
+ timer_shutdown_sync(&hte.timer);
free_irq(hte.gpio_in_irq, &hte);
gpiod_put(hte.gpio_in);
gpiod_put(hte.gpio_out);
- del_timer_sync(&hte.timer);
}
static struct platform_driver tegra_hte_test_driver = {
diff --git a/drivers/hwmon/acpi_power_meter.c b/drivers/hwmon/acpi_power_meter.c
index 6c8a9c863528..eedcfc9d96c6 100644
--- a/drivers/hwmon/acpi_power_meter.c
+++ b/drivers/hwmon/acpi_power_meter.c
@@ -10,6 +10,7 @@
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
#include <linux/jiffies.h>
+#include <linux/limits.h>
#include <linux/mutex.h>
#include <linux/dmi.h>
#include <linux/slab.h>
@@ -31,7 +32,12 @@
#define POWER_METER_CAN_NOTIFY (1 << 3)
#define POWER_METER_IS_BATTERY (1 << 8)
#define UNKNOWN_HYSTERESIS 0xFFFFFFFF
-#define UNKNOWN_POWER 0xFFFFFFFF
+/*
+ * ACPI integers can either be 32-bit or 64-bit wide, so
+ * we have to check for both values.
+ */
+#define UNKNOWN_POWER_32 U32_MAX
+#define UNKNOWN_POWER_64 U64_MAX
#define METER_NOTIFY_CONFIG 0x80
#define METER_NOTIFY_TRIP 0x81
@@ -349,7 +355,7 @@ static ssize_t show_power(struct device *dev,
update_meter(resource);
mutex_unlock(&resource->lock);
- if (resource->power == UNKNOWN_POWER)
+ if (resource->power == UNKNOWN_POWER_32 || resource->power == UNKNOWN_POWER_64)
return -ENODATA;
return sprintf(buf, "%llu\n", resource->power * 1000);
diff --git a/drivers/hwmon/aquacomputer_d5next.c b/drivers/hwmon/aquacomputer_d5next.c
index 0dcb8a3a691d..95ec74c6e67e 100644
--- a/drivers/hwmon/aquacomputer_d5next.c
+++ b/drivers/hwmon/aquacomputer_d5next.c
@@ -19,6 +19,7 @@
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/ktime.h>
+#include <linux/limits.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/seq_file.h>
@@ -133,6 +134,7 @@ static u8 aquaero_secondary_ctrl_report[] = {
#define AQUAERO_FAN_CURRENT_OFFSET 0x06
#define AQUAERO_FAN_POWER_OFFSET 0x08
#define AQUAERO_FAN_SPEED_OFFSET 0x00
+#define AQUAERO_MIN_STATUS_REPORT_SIZE (0x18B + AQUAERO_FAN_POWER_OFFSET + sizeof(u16))
static u16 aquaero_sensor_fan_offsets[] = { 0x167, 0x173, 0x17f, 0x18B };
/* Control report offsets for the Aquaero fan controllers */
@@ -156,6 +158,8 @@ static u16 aquaero_ctrl_fan_offsets[] = { 0x20c, 0x220, 0x234, 0x248 };
#define D5NEXT_5V_VOLTAGE 0x39
#define D5NEXT_12V_VOLTAGE 0x37
#define D5NEXT_VIRTUAL_SENSORS_START 0x3f
+#define D5NEXT_MIN_STATUS_REPORT_SIZE (D5NEXT_PUMP_OFFSET + AQC_FAN_SPEED_OFFSET + \
+ sizeof(u16))
static u16 d5next_sensor_fan_offsets[] = { D5NEXT_PUMP_OFFSET, D5NEXT_FAN_OFFSET };
/* Control report offsets for the D5 Next pump */
@@ -180,11 +184,14 @@ static u16 d5next_ctrl_fan_offsets[] = { 0x97, 0x42 }; /* Pump and fan speed (fr
#define AQUASTREAMULT_FAN_CURRENT_OFFSET 0x00
#define AQUASTREAMULT_FAN_POWER_OFFSET 0x04
#define AQUASTREAMULT_FAN_SPEED_OFFSET 0x06
+#define AQUASTREAMULT_MIN_STATUS_REPORT_SIZE (AQUASTREAMULT_PRESSURE_OFFSET + sizeof(u16))
static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Spec and sensor report offset for the Farbwerk RGB controller */
#define FARBWERK_NUM_SENSORS 4
#define FARBWERK_SENSOR_START 0x2f
+#define FARBWERK_MIN_STATUS_REPORT_SIZE (FARBWERK_SENSOR_START + \
+ FARBWERK_NUM_SENSORS * AQC_SENSOR_SIZE)
/* Specs of the Farbwerk 360 RGB controller */
#define FARBWERK360_NUM_SENSORS 4
@@ -194,6 +201,8 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Sensor report offsets for the Farbwerk 360 */
#define FARBWERK360_SENSOR_START 0x32
#define FARBWERK360_VIRTUAL_SENSORS_START 0x3a
+#define FARBWERK360_MIN_STATUS_REPORT_SIZE (FARBWERK360_VIRTUAL_SENSORS_START + \
+ FARBWERK360_NUM_VIRTUAL_SENSORS * AQC_SENSOR_SIZE)
/* Control report offsets for the Farbwerk 360 */
#define FARBWERK360_TEMP_CTRL_OFFSET 0x8
@@ -210,6 +219,7 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
#define OCTO_SENSOR_START 0x3D
#define OCTO_VIRTUAL_SENSORS_START 0x45
#define OCTO_FLOW_SENSOR_OFFSET 0x7B
+#define OCTO_MIN_STATUS_REPORT_SIZE (0xD8 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 octo_sensor_fan_offsets[] = { 0x7D, 0x8A, 0x97, 0xA4, 0xB1, 0xBE, 0xCB, 0xD8 };
/* Control report offsets for the Octo */
@@ -230,6 +240,7 @@ static u16 octo_ctrl_fan_offsets[] = { 0x5B, 0xB0, 0x105, 0x15A, 0x1AF, 0x204, 0
#define QUADRO_SENSOR_START 0x34
#define QUADRO_VIRTUAL_SENSORS_START 0x3c
#define QUADRO_FLOW_SENSOR_OFFSET 0x6e
+#define QUADRO_MIN_STATUS_REPORT_SIZE (0x97 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 quadro_sensor_fan_offsets[] = { 0x70, 0x7D, 0x8A, 0x97 };
/* Control report offsets for the Quadro */
@@ -249,6 +260,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define HIGHFLOWNEXT_CONDUCTIVITY 95
#define HIGHFLOWNEXT_5V_VOLTAGE 97
#define HIGHFLOWNEXT_5V_VOLTAGE_USB 99
+#define HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE (HIGHFLOWNEXT_5V_VOLTAGE_USB + sizeof(u16))
/* Specs of the Leakshield */
#define LEAKSHIELD_NUM_SENSORS 2
@@ -264,6 +276,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define LEAKSHIELD_FLOW_IN 111
#define LEAKSHIELD_RESERVOIR_VOLUME 313
#define LEAKSHIELD_RESERVOIR_FILLED 311
+#define LEAKSHIELD_MIN_STATUS_REPORT_SIZE (LEAKSHIELD_RESERVOIR_VOLUME + sizeof(u16))
/* Specs of the Aquastream XT pump */
#define AQUASTREAMXT_SERIAL_START 0x3a
@@ -556,6 +569,7 @@ struct aqc_data {
const char *name;
int status_report_id; /* Used for legacy devices, report is stored in buffer */
+ u16 min_report_size;
int ctrl_report_id;
int secondary_ctrl_report_id;
int secondary_ctrl_report_size;
@@ -601,9 +615,9 @@ struct aqc_data {
u32 speed_input_min[1];
u32 speed_input_target[1];
u32 speed_input_max[1];
- u32 power_input[8];
+ long power_input[8];
u16 voltage_input[8];
- u16 current_input[8];
+ u32 current_input[8];
/* Label values */
const char *const *temp_label;
@@ -996,7 +1010,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
/* Calculation derived from linear regression */
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_CURR_OFFSET);
- priv->current_input[0] = DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52;
+ priv->current_input[0] = max(DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52, 0);
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_VOLTAGE_OFFSET);
priv->voltage_input[0] = DIV_ROUND_CLOSEST(sensor_value * 1000, 61);
@@ -1029,6 +1043,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
int channel, long *val)
{
+ long power;
int ret;
struct aqc_data *priv = dev_get_drvdata(dev);
@@ -1093,7 +1108,10 @@ static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
}
break;
case hwmon_power:
- *val = priv->power_input[channel];
+ power = READ_ONCE(priv->power_input[channel]);
+ if (power == -ENODATA)
+ return -ENODATA;
+ *val = power;
break;
case hwmon_pwm:
switch (priv->kind) {
@@ -1356,6 +1374,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
return 0;
priv = hid_get_drvdata(hdev);
+ if (!priv->min_report_size || size < priv->min_report_size)
+ return 0;
/* Info provided with every report */
priv->serial_number[0] = get_unaligned_be16(data + priv->serial_number_start_offset);
@@ -1391,9 +1411,9 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->speed);
- priv->power_input[i] =
- get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
- priv->fan_structure->power) * 10000;
+ WRITE_ONCE(priv->power_input[i],
+ get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
+ priv->fan_structure->power) * 10000);
priv->voltage_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->voltage) * 10;
@@ -1433,7 +1453,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[2] = get_unaligned_be16(data + AQUASTREAMULT_PRESSURE_OFFSET);
priv->speed_input[3] = get_unaligned_be16(data + AQUASTREAMULT_FLOW_SENSOR_OFFSET);
- priv->power_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000;
+ WRITE_ONCE(priv->power_input[1],
+ get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000);
priv->voltage_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_VOLTAGE) * 10;
@@ -1446,10 +1467,11 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
case highflownext:
/* If external temp sensor is not connected, its power reading is also N/A */
if (priv->temp_input[1] == -ENODATA)
- priv->power_input[0] = -ENODATA;
+ WRITE_ONCE(priv->power_input[0], -ENODATA);
else
- priv->power_input[0] =
- get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * 1000000;
+ WRITE_ONCE(priv->power_input[0],
+ min_t(u64, get_unaligned_be16(data + HIGHFLOWNEXT_POWER) *
+ 1000000ULL, LONG_MAX));
priv->voltage_input[0] = get_unaligned_be16(data + HIGHFLOWNEXT_5V_VOLTAGE) * 10;
priv->voltage_input[1] =
@@ -1541,6 +1563,7 @@ static void aqc_debugfs_init(struct aqc_data *priv)
static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
{
struct aqc_data *priv;
+ u16 min_report_size = 0;
int ret;
priv = devm_kzalloc(&hdev->dev, sizeof(*priv), GFP_KERNEL);
@@ -1577,7 +1600,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
* they present. The two other devices have the type of the second element in
* their respective collections set to 1, while the real device has it set to 0.
*/
- if (hdev->collection[1].type != 0) {
+ if (hdev->maxcollection < 2 || hdev->collection[1].type != 0) {
ret = -ENODEV;
goto fail_and_close;
}
@@ -1608,6 +1631,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = AQUAERO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_D5NEXT:
priv->kind = d5next;
@@ -1633,6 +1657,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_d5next_power;
priv->voltage_label = label_d5next_voltages;
priv->current_label = label_d5next_current;
+ min_report_size = D5NEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK:
priv->kind = farbwerk;
@@ -1643,6 +1668,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_sensor_start_offset = FARBWERK_SENSOR_START;
priv->temp_label = label_temp_sensors;
+ min_report_size = FARBWERK_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK360:
priv->kind = farbwerk360;
@@ -1659,6 +1685,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_temp_sensors;
priv->virtual_temp_label = label_virtual_temp_sensors;
+ min_report_size = FARBWERK360_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_OCTO:
priv->kind = octo;
@@ -1688,6 +1715,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = OCTO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_QUADRO:
priv->kind = quadro;
@@ -1717,6 +1745,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = QUADRO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_HIGHFLOWNEXT:
priv->kind = highflownext;
@@ -1734,6 +1763,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->speed_label = label_highflownext_fan_speed;
priv->power_label = label_highflownext_power;
priv->voltage_label = label_highflownext_voltage;
+ min_report_size = HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_LEAKSHIELD:
/*
@@ -1753,6 +1783,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_leakshield_temp_sensors;
priv->speed_label = label_leakshield_fan_speed;
+ min_report_size = LEAKSHIELD_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_AQUASTREAMXT:
priv->kind = aquastreamxt;
@@ -1783,6 +1814,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_aquastreamult_power;
priv->voltage_label = label_aquastreamult_voltages;
priv->current_label = label_aquastreamult_current;
+ min_report_size = AQUASTREAMULT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_POWERADJUST3:
priv->kind = poweradjust3;
@@ -1856,6 +1888,8 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
break;
}
+ priv->min_report_size = min_report_size;
+
if (priv->buffer_size != 0) {
priv->checksum_start = 0x01;
priv->checksum_length = priv->buffer_size - 3;
diff --git a/drivers/hwmon/asus_wmi_sensors.c b/drivers/hwmon/asus_wmi_sensors.c
index c2dd7ff882f2..502996e97cf1 100644
--- a/drivers/hwmon/asus_wmi_sensors.c
+++ b/drivers/hwmon/asus_wmi_sensors.c
@@ -164,6 +164,7 @@ struct asus_wmi_wmi_info {
struct asus_wmi_sensors {
struct asus_wmi_wmi_info wmi;
+ struct hwmon_chip_info chip_info;
/* lock access to internal cache */
struct mutex lock;
};
@@ -496,11 +497,6 @@ static const struct hwmon_ops asus_wmi_hwmon_ops = {
.read_string = asus_wmi_hwmon_read_string,
};
-static struct hwmon_chip_info asus_wmi_chip_info = {
- .ops = &asus_wmi_hwmon_ops,
- .info = NULL,
-};
-
static int asus_wmi_configure_sensor_setup(struct device *dev,
struct asus_wmi_sensors *sensor_data)
{
@@ -508,7 +504,7 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
struct hwmon_channel_info *asus_wmi_hwmon_chan;
int nr_count[hwmon_max] = {}, nr_types = 0;
struct asus_wmi_sensor_info *temp_sensor;
- const struct hwmon_chip_info *chip_info;
+ struct hwmon_chip_info *chip_info;
enum hwmon_sensor_types type;
struct device *hwdev;
int i, idx;
@@ -549,8 +545,9 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
if (!ptr_asus_wmi_ci)
return -ENOMEM;
- asus_wmi_chip_info.info = ptr_asus_wmi_ci;
- chip_info = &asus_wmi_chip_info;
+ chip_info = &sensor_data->chip_info;
+ chip_info->ops = &asus_wmi_hwmon_ops;
+ chip_info->info = ptr_asus_wmi_ci;
sensor_data->wmi.info_by_id = devm_kcalloc(dev, sensor_data->wmi.sensor_count,
sizeof(*sensor_data->wmi.info_by_id),
diff --git a/drivers/hwmon/chipcap2.c b/drivers/hwmon/chipcap2.c
index d9adb9e1f9ab..46a3e22f43ad 100644
--- a/drivers/hwmon/chipcap2.c
+++ b/drivers/hwmon/chipcap2.c
@@ -20,6 +20,7 @@
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/module.h>
+#include <linux/unaligned.h>
#include <linux/regulator/consumer.h>
#define CC2_START_CM 0xA0
@@ -87,6 +88,7 @@ struct cc2_data {
int irq_low;
int irq_high;
bool process_irqs;
+ bool regulator_enabled;
};
/* %RH as a per cent mille from a register value */
@@ -116,8 +118,10 @@ static int cc2_enable(struct cc2_data *data)
int ret;
/* exclusive regulator, check in case a disable failed */
- if (regulator_is_enabled(data->regulator))
+ if (data->regulator_enabled) {
+ data->process_irqs = true;
return 0;
+ }
/* clear any pending completion */
try_wait_for_completion(&data->complete);
@@ -126,6 +130,8 @@ static int cc2_enable(struct cc2_data *data)
if (ret < 0)
return ret;
+ data->regulator_enabled = true;
+
usleep_range(CC2_STARTUP_TIME_US, CC2_STARTUP_TIME_US + 125);
data->process_irqs = true;
@@ -137,15 +143,27 @@ static void cc2_disable(struct cc2_data *data)
{
int err;
+ /* let the alarms already raised by the last measurement be processed */
+ if (data->irq_low > 0)
+ synchronize_irq(data->irq_low);
+
+ if (data->irq_high > 0)
+ synchronize_irq(data->irq_high);
+
/* ignore alarms triggered by voltage toggling when powering up */
data->process_irqs = false;
/* exclusive regulator, check in case an enable failed */
- if (regulator_is_enabled(data->regulator)) {
- err = regulator_disable(data->regulator);
- if (err)
- dev_dbg(&data->client->dev, "Failed to disable device");
+ if (!data->regulator_enabled)
+ return;
+
+ err = regulator_disable(data->regulator);
+ if (err) {
+ dev_dbg(&data->client->dev, "Failed to disable device");
+ return;
}
+
+ data->regulator_enabled = false;
}
static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
@@ -164,13 +182,13 @@ static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
case CC2_RESPONSE_BUSY:
return -EBUSY;
case CC2_RESPONSE_NACK:
- if (resp & CC2_ERR_CORR_EEPROM)
+ if (status & CC2_ERR_CORR_EEPROM)
dev_dbg(dev, "Command failed: corrected EEPROM\n");
- if (resp & CC2_ERR_UNCORR_EEPROM)
+ if (status & CC2_ERR_UNCORR_EEPROM)
dev_dbg(dev, "Command failed: uncorrected EEPROM\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_RAM_PARITY)
dev_dbg(dev, "Command failed: RAM parity\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_CONFIG_LOAD)
dev_dbg(dev, "Command failed: configuration error\n");
return -ENODATA;
default:
@@ -223,7 +241,7 @@ static int cc2_command_mode_start(struct cc2_data *data)
2 * CC2_RESP_START_CM_US);
}
ret = cc2_read_command_status(data->client);
- if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES)
+ if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES - 1)
break;
/* command window missed, prepare for a retry */
@@ -257,8 +275,7 @@ static int cc2_write_reg(struct cc2_data *data, u8 reg, u16 val)
if (ret < 0)
goto disable;
- cpu_to_be16s(&val);
- ret = i2c_smbus_write_word_data(data->client, reg, val);
+ ret = i2c_smbus_write_word_swapped(data->client, reg, val);
if (ret < 0)
goto disable;
@@ -308,7 +325,7 @@ static int cc2_read_reg(struct cc2_data *data, u8 reg, u16 *val)
if (ret != CC2_EEPROM_DATA_LEN)
return ret < 0 ? ret : -EIO;
- *val = be16_to_cpup((__be16 *)&buf[1]);
+ *val = get_unaligned_be16(&buf[1]);
return cc2_read_command_status(data->client);
}
@@ -348,10 +365,10 @@ static int cc2_data_fetch(struct i2c_client *client,
switch (type) {
case hwmon_humidity:
- *val = cc2_rh_convert(be16_to_cpup((__be16 *)&data[0]));
+ *val = cc2_rh_convert(get_unaligned_be16(&data[0]));
break;
case hwmon_temp:
- *val = cc2_temp_convert(be16_to_cpup((__be16 *)&data[2]));
+ *val = cc2_temp_convert(get_unaligned_be16(&data[2]));
break;
default:
return -EINVAL;
@@ -651,6 +668,8 @@ static int cc2_request_ready_irq(struct cc2_data *data, struct device *dev)
int ret = 0;
data->irq_ready = fwnode_irq_get_byname(dev_fwnode(dev), "ready");
+ if (data->irq_ready == -EPROBE_DEFER)
+ return data->irq_ready;
if (data->irq_ready > 0) {
init_completion(&data->complete);
ret = devm_request_threaded_irq(dev, data->irq_ready, NULL,
@@ -668,6 +687,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
int ret = 0;
data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low");
+ if (data->irq_low == -EPROBE_DEFER)
+ return data->irq_low;
if (data->irq_low > 0) {
ret = devm_request_threaded_irq(dev, data->irq_low, NULL,
cc2_low_interrupt,
@@ -681,6 +702,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
}
data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high");
+ if (data->irq_high == -EPROBE_DEFER)
+ return data->irq_high;
if (data->irq_high > 0) {
ret = devm_request_threaded_irq(dev, data->irq_high, NULL,
cc2_high_interrupt,
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index 04b20cf272ac..cc81b849c09e 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -61,6 +61,8 @@ enum coretemp_attr_index {
/*
* Per-Core Temperature Data
+ * @temp_valid: Whether the CPU reported @temp as valid. @temp is not
+ * reported when this is false.
* @tjmax: The static tjmax value when tjmax cannot be retrieved from
* IA32_TEMPERATURE_TARGET MSR.
* @last_updated: The time when the current temperature value was updated
@@ -73,6 +75,7 @@ enum coretemp_attr_index {
*/
struct temp_data {
int temp;
+ bool temp_valid;
int tjmax;
unsigned long last_updated;
unsigned int cpu;
@@ -397,7 +400,8 @@ static ssize_t show_temp(struct device *dev,
{
u32 eax, edx;
struct temp_data *tdata = container_of(devattr, struct temp_data, sd_attrs[ATTR_TEMP]);
- int tjmax;
+ int tjmax, temp;
+ bool valid;
mutex_lock(&tdata->update_lock);
@@ -406,17 +410,22 @@ static ssize_t show_temp(struct device *dev,
if (time_after(jiffies, tdata->last_updated + HZ)) {
rdmsr_on_cpu(tdata->cpu, tdata->status_reg, &eax, &edx);
/*
- * Ignore the valid bit. In all observed cases the register
- * value is either low or zero if the valid bit is 0.
- * Return it instead of reporting an error which doesn't
- * really help at all.
+ * Bit 31 indicates whether the reading is valid. Some CPUs
+ * clear it while the temperature is too low to be measured.
+ * Such a reading is not a temperature. Do not report it.
*/
+ tdata->temp_valid = (eax >> 31) & 1;
tdata->temp = tjmax - ((eax >> 16) & 0xff) * 1000;
tdata->last_updated = jiffies;
}
+ valid = tdata->temp_valid;
+ temp = tdata->temp;
mutex_unlock(&tdata->update_lock);
- return sprintf(buf, "%d\n", tdata->temp);
+
+ if (!valid)
+ return -ENODATA;
+ return sprintf(buf, "%d\n", temp);
}
static int create_core_attrs(struct temp_data *tdata, struct device *dev)
@@ -692,6 +701,9 @@ static int coretemp_cpu_online(unsigned int cpu)
if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
return 0;
+ if (!pdev)
+ return 0;
+
pdata = platform_get_drvdata(pdev);
if (!pdata->hwmon_dev) {
struct device *hwmon;
@@ -743,6 +755,9 @@ static int coretemp_cpu_offline(unsigned int cpu)
return 0;
/* If the physical CPU device does not exist, just return */
+ if (!pdev)
+ return 0;
+
pd = platform_get_drvdata(pdev);
if (!pd->hwmon_dev)
return 0;
diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c
index 17bb080eb9bb..1c152a6db380 100644
--- a/drivers/hwmon/corsair-cpro.c
+++ b/drivers/hwmon/corsair-cpro.c
@@ -215,10 +215,11 @@ static int set_target(struct ccp_device *ccp, int channel, long val)
int ret;
val = clamp_val(val, 0, 0xFFFF);
- ccp->target[channel] = val;
mutex_lock(&ccp->mutex);
ret = send_usb_cmd(ccp, CTL_SET_FAN_TARGET, channel, val >> 8, val);
+ if (!ret)
+ ccp->target[channel] = val;
mutex_unlock(&ccp->mutex);
return ret;
diff --git a/drivers/hwmon/hp-wmi-sensors.c b/drivers/hwmon/hp-wmi-sensors.c
index 82210fef828f..d2f8f93ee96d 100644
--- a/drivers/hwmon/hp-wmi-sensors.c
+++ b/drivers/hwmon/hp-wmi-sensors.c
@@ -390,6 +390,7 @@ struct hp_wmi_info {
*/
struct hp_wmi_sensors {
struct wmi_device *wdev;
+ struct hwmon_chip_info chip_info;
struct hp_wmi_info **info_map[hwmon_max];
u8 channel_count[hwmon_max];
bool has_intrusion;
@@ -724,15 +725,21 @@ static long scale_numeric_sensor(const struct hp_wmi_numeric_sensor *nsensor)
val = current_reading;
- for (; unit_modifier < target_modifier; unit_modifier++)
+ for (; val && unit_modifier < target_modifier; unit_modifier++)
val = DIV_ROUND_CLOSEST(val, 10);
- for (; unit_modifier > target_modifier; unit_modifier--) {
- if (val > LONG_MAX / 10) {
- val = LONG_MAX;
- break;
+ if (val) {
+ for (; unit_modifier > target_modifier; unit_modifier--) {
+ if (val > LONG_MAX / 10) {
+ val = LONG_MAX;
+ break;
+ }
+ if (val < LONG_MIN / 10) {
+ val = LONG_MIN;
+ break;
+ }
+ val *= 10;
}
- val *= 10;
}
if (sensor_type == HP_WMI_TYPE_TEMPERATURE) {
@@ -1517,11 +1524,6 @@ static const struct hwmon_ops hp_wmi_hwmon_ops = {
.write = hp_wmi_hwmon_write,
};
-static struct hwmon_chip_info hp_wmi_chip_info = {
- .ops = &hp_wmi_hwmon_ops,
- .info = NULL,
-};
-
static struct hp_wmi_info *match_fan_event(struct hp_wmi_sensors *state,
const char *event_description)
{
@@ -1948,7 +1950,8 @@ static int make_chip_info(struct hp_wmi_sensors *state, bool has_events)
if (!ptr_channel_info)
return -ENOMEM;
- hp_wmi_chip_info.info = ptr_channel_info;
+ state->chip_info.ops = &hp_wmi_hwmon_ops;
+ state->chip_info.info = ptr_channel_info;
for (type = hwmon_chip; type < hwmon_max; type++) {
count = channel_count[type];
@@ -2039,7 +2042,7 @@ static int hp_wmi_sensors_init(struct hp_wmi_sensors *state)
return err;
hwdev = devm_hwmon_device_register_with_info(dev, "hp_wmi_sensors",
- state, &hp_wmi_chip_info,
+ state, &state->chip_info,
NULL);
return PTR_ERR_OR_ZERO(hwdev);
}
diff --git a/drivers/hwmon/lm70.c b/drivers/hwmon/lm70.c
index 0d5a250cb672..ee06c5641870 100644
--- a/drivers/hwmon/lm70.c
+++ b/drivers/hwmon/lm70.c
@@ -97,21 +97,21 @@ static ssize_t temp1_input_show(struct device *dev,
*/
switch (p_lm70->chip) {
case LM70_CHIP_LM70:
- val = ((int)raw / 32) * 250;
+ val = ((int)raw >> 5) * 250;
break;
case LM70_CHIP_TMP121:
case LM70_CHIP_TMP122:
case LM70_CHIP_LM74:
- val = ((int)raw / 8) * 625 / 10;
+ val = ((int)raw >> 3) * 625 / 10;
break;
case LM70_CHIP_LM71:
- val = ((int)raw / 4) * 3125 / 100;
+ val = ((int)raw >> 2) * 3125 / 100;
break;
case LM70_CHIP_TMP125:
- val = (sign_extend32(raw, 14) / 32) * 250;
+ val = (sign_extend32(raw, 14) >> 5) * 250;
break;
}
diff --git a/drivers/hwmon/lm87.c b/drivers/hwmon/lm87.c
index d2d970e73c61..bdd633feb2ba 100644
--- a/drivers/hwmon/lm87.c
+++ b/drivers/hwmon/lm87.c
@@ -600,13 +600,16 @@ static DEVICE_ATTR_RO(cpu0_vid);
static ssize_t vrm_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
+
return sprintf(buf, "%d\n", data->vrm);
}
static ssize_t vrm_store(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
unsigned long val;
int err;
diff --git a/drivers/hwmon/nct6683.c b/drivers/hwmon/nct6683.c
index f71615e06a8f..e091551688a0 100644
--- a/drivers/hwmon/nct6683.c
+++ b/drivers/hwmon/nct6683.c
@@ -914,7 +914,7 @@ static ssize_t
show_pwm(struct device *dev, struct device_attribute *attr, char *buf)
{
struct nct6683_data *data = nct6683_update_device(dev);
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
int index = sattr->index;
return sprintf(buf, "%d\n", data->pwm[index]);
@@ -924,7 +924,7 @@ static ssize_t
store_pwm(struct device *dev, struct device_attribute *attr, const char *buf,
size_t count)
{
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
struct nct6683_data *data = dev_get_drvdata(dev);
int index = sattr->index;
unsigned long val;
diff --git a/drivers/hwmon/nct6775-platform.c b/drivers/hwmon/nct6775-platform.c
index ece183a0d4e9..4b0003280f11 100644
--- a/drivers/hwmon/nct6775-platform.c
+++ b/drivers/hwmon/nct6775-platform.c
@@ -1158,6 +1158,7 @@ static const char * const asus_wmi_boards[] = {
"Pro A520M-C",
"Pro A520M-C II",
"Pro B550M-C",
+ "Pro WS W480-ACE",
"Pro WS X570-ACE",
"ProArt B550-CREATOR",
"ProArt X570-CREATOR WIFI",
@@ -1257,6 +1258,7 @@ static const char * const asus_wmi_boards[] = {
"TUF Z390-PRO GAMING",
"TUF Z390M-PRO GAMING",
"TUF Z390M-PRO GAMING (WI-FI)",
+ "W480/SYS",
"WS Z390 PRO",
"Z490-GUNDAM (WI-FI)",
};
@@ -1269,6 +1271,7 @@ static const char * const asus_msi_boards[] = {
"EX-B760M-V5 D4",
"EX-H510M-V3",
"EX-H610M-V3 D4",
+ "G15CE",
"G15CF",
"PRIME A620M-A",
"PRIME B560-PLUS",
@@ -1319,6 +1322,8 @@ static const char * const asus_msi_boards[] = {
"PRIME X670-P",
"PRIME X670-P WIFI",
"PRIME X670E-PRO WIFI",
+ "PRIME X870-P",
+ "PRIME X870-P WIFI",
"PRIME Z590-A",
"PRIME Z590-P",
"PRIME Z590-P WIFI",
@@ -1361,11 +1366,18 @@ static const char * const asus_msi_boards[] = {
"ProArt B660-CREATOR D4",
"ProArt B760-CREATOR D4",
"ProArt X670E-CREATOR WIFI",
+ "ProArt X870E-CREATOR WIFI",
"ProArt Z690-CREATOR WIFI",
"ProArt Z790-CREATOR WIFI",
"ROG CROSSHAIR X670E EXTREME",
"ROG CROSSHAIR X670E GENE",
"ROG CROSSHAIR X670E HERO",
+ "ROG CROSSHAIR X870E APEX",
+ "ROG CROSSHAIR X870E DARK HERO",
+ "ROG CROSSHAIR X870E EXTREME",
+ "ROG CROSSHAIR X870E GLACIAL",
+ "ROG CROSSHAIR X870E HERO",
+ "ROG CROSSHAIR X870E HERO BTF",
"ROG MAXIMUS XIII APEX",
"ROG MAXIMUS XIII EXTREME",
"ROG MAXIMUS XIII EXTREME GLACIAL",
@@ -1403,6 +1415,12 @@ static const char * const asus_msi_boards[] = {
"ROG STRIX X670E-E GAMING WIFI",
"ROG STRIX X670E-F GAMING WIFI",
"ROG STRIX X670E-I GAMING WIFI",
+ "ROG STRIX X870-A GAMING WIFI",
+ "ROG STRIX X870-F GAMING WIFI",
+ "ROG STRIX X870-I GAMING WIFI",
+ "ROG STRIX X870E-E GAMING WIFI",
+ "ROG STRIX X870E-E GAMING WIFI7 R2",
+ "ROG STRIX X870E-H GAMING WIFI7",
"ROG STRIX Z590-A GAMING WIFI",
"ROG STRIX Z590-A GAMING WIFI II",
"ROG STRIX Z590-E GAMING WIFI",
@@ -1449,6 +1467,9 @@ static const char * const asus_msi_boards[] = {
"TUF GAMING H770-PRO WIFI",
"TUF GAMING X670E-PLUS",
"TUF GAMING X670E-PLUS WIFI",
+ "TUF GAMING X870-PLUS WIFI",
+ "TUF GAMING X870-PRO WIFI7 W NEO",
+ "TUF GAMING X870E-PLUS WIFI7",
"TUF GAMING Z590-PLUS",
"TUF GAMING Z590-PLUS WIFI",
"TUF GAMING Z690-PLUS",
@@ -1458,6 +1479,9 @@ static const char * const asus_msi_boards[] = {
"TUF GAMING Z790-PLUS D4",
"TUF GAMING Z790-PLUS WIFI",
"TUF GAMING Z790-PLUS WIFI D4",
+ "X870 AYW GAMING WIFI W",
+ "X870 MAX GAMING WIFI7",
+ "X870 MAX GAMING WIFI7 W",
"Z590 WIFI GUNDAM EDITION",
};
diff --git a/drivers/hwmon/peci/dimmtemp.c b/drivers/hwmon/peci/dimmtemp.c
index b7b09780c7b0..671ea8d3b8ad 100644
--- a/drivers/hwmon/peci/dimmtemp.c
+++ b/drivers/hwmon/peci/dimmtemp.c
@@ -4,7 +4,6 @@
#include <linux/auxiliary_bus.h>
#include <linux/bitfield.h>
#include <linux/bitops.h>
-#include <linux/devm-helpers.h>
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/module.h>
@@ -331,11 +330,13 @@ static int create_dimm_temp_info(struct peci_dimmtemp *priv)
if (ret == -ENODEV) {
dev_dbg(priv->dev, "No DIMMs found\n");
return 0;
- } else if (ret) {
+ }
+ if (ret == -EAGAIN) {
schedule_delayed_work(&priv->detect_work, DIMM_MASK_CHECK_DELAY_JIFFIES);
dev_dbg(priv->dev, "Deferred populating DIMM temp info\n");
- return ret;
}
+ if (ret)
+ return ret;
channels = priv->gen_info->chan_rank_max * priv->gen_info->dimm_idx_max;
@@ -405,10 +406,8 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
"Unexpected PECI revision %#x, some features may be unavailable\n",
peci_dev->info.peci_revision);
- ret = devm_delayed_work_autocancel(priv->dev, &priv->detect_work,
- create_dimm_temp_info_delayed);
- if (ret)
- return ret;
+ auxiliary_set_drvdata(adev, priv);
+ INIT_DELAYED_WORK(&priv->detect_work, create_dimm_temp_info_delayed);
ret = create_dimm_temp_info(priv);
if (ret && ret != -EAGAIN) {
@@ -419,6 +418,13 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
return 0;
}
+static void peci_dimmtemp_remove(struct auxiliary_device *adev)
+{
+ struct peci_dimmtemp *priv = auxiliary_get_drvdata(adev);
+
+ cancel_delayed_work_sync(&priv->detect_work);
+}
+
static int
read_thresholds_hsx(struct peci_dimmtemp *priv, int dimm_order, int chan_rank, u32 *data)
{
@@ -655,6 +661,7 @@ MODULE_DEVICE_TABLE(auxiliary, peci_dimmtemp_ids);
static struct auxiliary_driver peci_dimmtemp_driver = {
.probe = peci_dimmtemp_probe,
+ .remove = peci_dimmtemp_remove,
.id_table = peci_dimmtemp_ids,
};
diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c
index ea79cff0e8d0..596b7fd568bf 100644
--- a/drivers/hwmon/pmbus/pmbus_core.c
+++ b/drivers/hwmon/pmbus/pmbus_core.c
@@ -1420,11 +1420,14 @@ static int pmbus_thermal_add_sensor(struct pmbus_data *pmbus_data,
tzd = devm_thermal_of_zone_register(dev, index, tdata,
&pmbus_thermal_ops);
/*
- * If CONFIG_THERMAL_OF is disabled, this returns -ENODEV,
- * so ignore that error but forward any other error.
+ * If CONFIG_THERMAL_OF is disabled, devm_thermal_of_zone_register()
+ * returns -ENOTSUPP. If no thermal zone is configured for the sensor,
+ * it returns -ENODEV. Ignore those errors but forward any other error.
*/
- if (IS_ERR(tzd) && (PTR_ERR(tzd) != -ENODEV))
+ if (IS_ERR(tzd) && PTR_ERR(tzd) != -ENODEV && PTR_ERR(tzd) != -ENOTSUPP) {
+ devm_kfree(dev, tdata);
return PTR_ERR(tzd);
+ }
return 0;
}
@@ -1439,10 +1442,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
{
struct sensor_device_attribute *a;
struct pmbus_sensor *sensor;
+ int err;
sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL);
if (!sensor)
- return NULL;
+ return ERR_PTR(-ENOMEM);
a = &sensor->attribute;
if (type)
@@ -1467,14 +1471,17 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
pmbus_set_sensor, -1);
if (pmbus_add_attribute(data, &a->dev_attr.attr))
- return NULL;
+ return ERR_PTR(-ENOMEM);
sensor->next = data->sensors;
data->sensors = sensor;
/* temperature sensors with _input values are registered with thermal */
- if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0)
- pmbus_thermal_add_sensor(data, sensor, seq);
+ if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) {
+ err = pmbus_thermal_add_sensor(data, sensor, seq);
+ if (err)
+ return ERR_PTR(err);
+ }
return sensor;
}
@@ -1576,8 +1583,8 @@ static int pmbus_add_limit_attrs(struct i2c_client *client,
page, 0xff, l->reg, attr->class,
attr->update || l->update,
l->readonly, l->writeonly, true);
- if (!curr)
- return -ENOMEM;
+ if (IS_ERR(curr))
+ return PTR_ERR(curr);
if (l->sbit && (info->func[page] & attr->sfunc)) {
ret = pmbus_add_boolean(data, name,
l->alarm, index,
@@ -1616,8 +1623,8 @@ static int pmbus_add_sensor_attrs_one(struct i2c_client *client,
}
base = pmbus_add_sensor(data, name, "input", index, page, phase,
attr->reg, attr->class, true, true, false, true);
- if (!base)
- return -ENOMEM;
+ if (IS_ERR(base))
+ return PTR_ERR(base);
/* No limit and alarm attributes for phase specific sensors */
if (attr->sfunc && phase == 0xff) {
ret = pmbus_add_limit_attrs(client, data, info, name,
@@ -2290,8 +2297,8 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_FAN_TARGET_1 + id, PSC_FAN,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
if (!((data->info->func[page] & PMBUS_HAVE_PWM12) ||
(data->info->func[page] & PMBUS_HAVE_PWM34)))
@@ -2301,15 +2308,15 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_PWM_1 + id, PSC_PWM,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
sensor = pmbus_add_sensor(data, "pwm", "enable", index, page,
0xff, PMBUS_VIRT_PWM_ENABLE_1 + id, PSC_PWM,
true, false, false, false);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
return 0;
}
@@ -2326,6 +2333,7 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
int f;
for (f = 0; f < ARRAY_SIZE(pmbus_fan_registers); f++) {
+ struct pmbus_sensor *sensor;
int regval;
if (!(info->func[page] & pmbus_fan_flags[f]))
@@ -2346,10 +2354,11 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
(!(regval & (PB_FAN_1_INSTALLED >> ((f & 1) * 4)))))
continue;
- if (pmbus_add_sensor(data, "fan", "input", index,
- page, 0xff, pmbus_fan_registers[f],
- PSC_FAN, true, true, false, true) == NULL)
- return -ENOMEM;
+ sensor = pmbus_add_sensor(data, "fan", "input", index,
+ page, 0xff, pmbus_fan_registers[f],
+ PSC_FAN, true, true, false, true);
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
/* Fan control */
if (pmbus_check_word_register(client, page,
diff --git a/drivers/hwmon/pmbus/q54sj108a2.c b/drivers/hwmon/pmbus/q54sj108a2.c
index d9fd1095ae7e..857bf6b6141c 100644
--- a/drivers/hwmon/pmbus/q54sj108a2.c
+++ b/drivers/hwmon/pmbus/q54sj108a2.c
@@ -343,6 +343,8 @@ static int q54sj108a2_probe(struct i2c_client *client)
psu->client = client;
debugfs = pmbus_get_debugfs_dir(client);
+ if (!debugfs)
+ return 0;
q54sj108a2_dir = debugfs_create_dir(client->name, debugfs);
diff --git a/drivers/hwmon/tmp102.c b/drivers/hwmon/tmp102.c
index 8af44a33055f..967d2e7e8ccf 100644
--- a/drivers/hwmon/tmp102.c
+++ b/drivers/hwmon/tmp102.c
@@ -54,7 +54,7 @@
struct tmp102 {
struct regmap *regmap;
u16 config_orig;
- unsigned long ready_time;
+ u64 ready_time;
};
/* convert left adjusted 13-bit TMP102 register value to milliCelsius */
@@ -79,7 +79,7 @@ static int tmp102_read(struct device *dev, enum hwmon_sensor_types type,
switch (attr) {
case hwmon_temp_input:
/* Is it too early to return a conversion ? */
- if (time_before(jiffies, tmp102->ready_time)) {
+ if (time_before64(get_jiffies_64(), tmp102->ready_time)) {
dev_dbg(dev, "%s: Conversion not ready yet..\n", __func__);
return -EAGAIN;
}
@@ -245,7 +245,7 @@ static int tmp102_probe(struct i2c_client *client)
* Mark that we are not ready with data until the first
* conversion is complete
*/
- tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS);
+ tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS);
hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name,
tmp102,
@@ -278,7 +278,7 @@ static int tmp102_resume(struct device *dev)
err = regmap_update_bits(tmp102->regmap, TMP102_CONF_REG,
TMP102_CONF_SD, 0);
- tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS);
+ tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/hwmon/tmp108.c b/drivers/hwmon/tmp108.c
index a82bbc959eb1..18e7c9d9bcbb 100644
--- a/drivers/hwmon/tmp108.c
+++ b/drivers/hwmon/tmp108.c
@@ -73,7 +73,7 @@
struct tmp108 {
struct regmap *regmap;
u16 orig_config;
- unsigned long ready_time;
+ u64 ready_time;
};
/* convert 12-bit TMP108 register value to milliCelsius */
@@ -124,7 +124,7 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type,
switch (attr) {
case hwmon_temp_input:
/* Is it too early to return a conversion ? */
- if (time_before(jiffies, tmp108->ready_time)) {
+ if (time_before64(get_jiffies_64(), tmp108->ready_time)) {
dev_dbg(dev, "%s: Conversion not ready yet..\n",
__func__);
return -EAGAIN;
@@ -371,7 +371,7 @@ static int tmp108_probe(struct i2c_client *client)
return err;
}
- tmp108->ready_time = jiffies;
+ tmp108->ready_time = get_jiffies_64();
if ((tmp108->orig_config & TMP108_CONF_MODE_MASK) ==
TMP108_MODE_SHUTDOWN)
tmp108->ready_time +=
@@ -405,7 +405,7 @@ static int tmp108_resume(struct device *dev)
err = regmap_update_bits(tmp108->regmap, TMP108_REG_CONF,
TMP108_CONF_MODE_MASK, TMP108_MODE_CONTINUOUS);
- tmp108->ready_time = jiffies +
+ tmp108->ready_time = get_jiffies_64() +
msecs_to_jiffies(TMP108_CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index 9127bc025552..41307719f144 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -1082,6 +1082,9 @@ static void etm4_disable_sysfs(struct coresight_device *csdev)
smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1);
spin_unlock(&drvdata->spinlock);
+
+ cscfg_csdev_disable_active_config(csdev);
+
cpus_read_unlock();
/*
diff --git a/drivers/iio/adc/ad7192.c b/drivers/iio/adc/ad7192.c
index 6fe32f866765..ef8268ad353e 100644
--- a/drivers/iio/adc/ad7192.c
+++ b/drivers/iio/adc/ad7192.c
@@ -1395,7 +1395,7 @@ static int ad7192_probe(struct spi_device *spi)
st->chip_info = spi_get_device_match_data(spi);
if (!st->chip_info)
- return -ENODEV;
+ return -ENODATA;
indio_dev->name = st->chip_info->name;
indio_dev->modes = INDIO_DIRECT_MODE;
diff --git a/drivers/iio/adc/ti-ads1100.c b/drivers/iio/adc/ti-ads1100.c
index 1e46f07a9ca6..bea727ff9d6e 100644
--- a/drivers/iio/adc/ti-ads1100.c
+++ b/drivers/iio/adc/ti-ads1100.c
@@ -14,10 +14,12 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/i2c.h>
+#include <linux/iopoll.h>
#include <linux/mutex.h>
#include <linux/property.h>
#include <linux/pm_runtime.h>
#include <linux/regulator/consumer.h>
+#include <linux/time.h>
#include <linux/units.h>
#include <linux/iio/iio.h>
@@ -42,6 +44,9 @@
static const int ads1100_data_rate[] = { 128, 32, 16, 8 };
static const int ads1100_data_rate_bits[] = { 12, 14, 15, 16 };
+/* Timeout based on the minimum sample rate of 8 SPS (7500ms) */
+#define ADS1100_MAX_DRDY_TIMEOUT_US (7500 * USEC_PER_MSEC)
+
struct ads1100_data {
struct i2c_client *client;
struct regulator *reg_vdd;
@@ -123,10 +128,86 @@ static int ads1100_get_adc_result(struct ads1100_data *data, int chan, int *val)
return 0;
}
+static int ads1100_conversion_busy(struct ads1100_data *data)
+{
+ u8 buffer[3];
+ int ret;
+
+ ret = i2c_master_recv(data->client, (char *)&buffer, sizeof(buffer));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "I2C read fail: %d\n", ret);
+ return ret;
+ }
+
+ return FIELD_GET(ADS1100_CFG_ST_BSY, buffer[2]);
+}
+
+static int ads1100_wait_single_conversion(struct ads1100_data *data)
+{
+ int data_rate_index = FIELD_GET(ADS1100_DR_MASK, data->config);
+ int data_rate_Hz = ads1100_data_rate[data_rate_index];
+ unsigned long poll_us = DIV_ROUND_CLOSEST(USEC_PER_SEC, data_rate_Hz) / 4;
+ int busy;
+ int ret;
+
+ ret = readx_poll_timeout(ads1100_conversion_busy, data,
+ busy, busy <= 0,
+ poll_us, ADS1100_MAX_DRDY_TIMEOUT_US);
+ if (busy < 0)
+ return busy;
+
+ return ret;
+}
+
+static int ads1100_start_single_conversion(struct ads1100_data *data)
+{
+ u8 config = data->config | ADS1100_CFG_SC;
+ int ret;
+
+ ret = i2c_master_send(data->client, &config, sizeof(config));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "I2C write fail: %d\n", ret);
+ return ret;
+ }
+ /* Need to wait because of change from continuous to single mode */
+ ret = ads1100_wait_single_conversion(data);
+ if (ret)
+ return ret;
+
+ config |= ADS1100_CFG_ST_BSY;
+
+ ret = i2c_master_send(data->client, &config, sizeof(config));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "I2C write fail: %d\n", ret);
+ return ret;
+ }
+
+ /* No need to cache it, it's status bit */
+ data->config = config & ~ADS1100_CFG_ST_BSY;
+
+ return 0;
+}
+
+static int ads1100_poll_data_ready(struct ads1100_data *data)
+{
+ int ret;
+
+ ret = ads1100_start_single_conversion(data);
+ if (ret)
+ return ret;
+
+ ret = ads1100_wait_single_conversion(data);
+ if (ret)
+ return ret;
+
+ return ads1100_set_config_bits(data, ADS1100_CFG_SC, ADS1100_CONTINUOUS);
+}
+
static int ads1100_set_scale(struct ads1100_data *data, int val, int val2)
{
int microvolts;
int gain;
+ int ret;
/* With Vdd between 2.7 and 5V, the scale is always below 1 */
if (val)
@@ -135,6 +216,10 @@ static int ads1100_set_scale(struct ads1100_data *data, int val, int val2)
if (!val2)
return -EINVAL;
+ ret = pm_runtime_resume_and_get(&data->client->dev);
+ if (ret)
+ return ret;
+
microvolts = regulator_get_voltage(data->reg_vdd);
/*
* val2 is in 'micro' units, n = val2 / 1000000
@@ -144,27 +229,51 @@ static int ads1100_set_scale(struct ads1100_data *data, int val, int val2)
* bitshift so everything fits in 32-bits yields this formula.
*/
gain = DIV_ROUND_CLOSEST(microvolts, BIT(15)) * MILLI / val2;
- if (gain < BIT(0) || gain > BIT(3))
- return -EINVAL;
+ if (gain < BIT(0) || gain > BIT(3)) {
+ ret = -EINVAL;
+ goto out;
+ }
ads1100_set_config_bits(data, ADS1100_PGA_MASK, ffs(gain) - 1);
- return 0;
+ ret = ads1100_poll_data_ready(data);
+out:
+ pm_runtime_mark_last_busy(&data->client->dev);
+ pm_runtime_put_autosuspend(&data->client->dev);
+
+ return ret;
}
static int ads1100_set_data_rate(struct ads1100_data *data, int chan, int rate)
{
unsigned int i;
unsigned int size;
+ int ret;
size = data->supports_data_rate ? ARRAY_SIZE(ads1100_data_rate) : 1;
for (i = 0; i < size; i++) {
if (ads1100_data_rate[i] == rate)
- return ads1100_set_config_bits(data, ADS1100_DR_MASK,
- FIELD_PREP(ADS1100_DR_MASK, i));
+ break;
}
- return -EINVAL;
+ if (i == size)
+ return -EINVAL;
+
+ ret = pm_runtime_resume_and_get(&data->client->dev);
+ if (ret)
+ return ret;
+
+ ret = ads1100_set_config_bits(data, ADS1100_DR_MASK,
+ FIELD_PREP(ADS1100_DR_MASK, i));
+ if (ret)
+ goto out;
+
+ ret = ads1100_poll_data_ready(data);
+out:
+ pm_runtime_mark_last_busy(&data->client->dev);
+ pm_runtime_put_autosuspend(&data->client->dev);
+
+ return ret;
}
static int ads1100_get_vdd_millivolts(struct ads1100_data *data)
diff --git a/drivers/iio/dac/mcp4821.c b/drivers/iio/dac/mcp4821.c
index c1a59bbbba3c..9e4a3cb2a712 100644
--- a/drivers/iio/dac/mcp4821.c
+++ b/drivers/iio/dac/mcp4821.c
@@ -172,9 +172,13 @@ static const struct iio_info mcp4821_info = {
static int mcp4821_probe(struct spi_device *spi)
{
+ const struct mcp4821_chip_info *info;
struct iio_dev *indio_dev;
struct mcp4821_state *state;
- const struct mcp4821_chip_info *info;
+
+ info = spi_get_device_match_data(spi);
+ if (!info)
+ return -ENODATA;
indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*state));
if (indio_dev == NULL)
@@ -183,7 +187,6 @@ static int mcp4821_probe(struct spi_device *spi)
state = iio_priv(indio_dev);
state->spi = spi;
- info = spi_get_device_match_data(spi);
indio_dev->name = info->name;
indio_dev->info = &mcp4821_info;
indio_dev->modes = INDIO_DIRECT_MODE;
diff --git a/drivers/iio/light/isl29028.c b/drivers/iio/light/isl29028.c
index a23b2af287d5..12708bc35150 100644
--- a/drivers/iio/light/isl29028.c
+++ b/drivers/iio/light/isl29028.c
@@ -407,14 +407,9 @@ static int isl29028_write_raw(struct iio_dev *indio_dev,
mutex_unlock(&chip->lock);
- if (ret < 0)
- return ret;
+ isl29028_set_pm_runtime_busy(chip, false);
- ret = isl29028_set_pm_runtime_busy(chip, false);
- if (ret < 0)
- return ret;
-
- return 0;
+ return ret;
}
static int isl29028_read_raw(struct iio_dev *indio_dev,
@@ -423,7 +418,7 @@ static int isl29028_read_raw(struct iio_dev *indio_dev,
{
struct isl29028_chip *chip = iio_priv(indio_dev);
struct device *dev = regmap_get_device(chip->regmap);
- int ret, pm_ret;
+ int ret;
ret = isl29028_set_pm_runtime_busy(chip, true);
if (ret < 0)
@@ -476,17 +471,7 @@ static int isl29028_read_raw(struct iio_dev *indio_dev,
mutex_unlock(&chip->lock);
- if (ret < 0)
- return ret;
-
- /**
- * Preserve the ret variable if the call to
- * isl29028_set_pm_runtime_busy() is successful so the reading
- * (if applicable) is returned to user space.
- */
- pm_ret = isl29028_set_pm_runtime_busy(chip, false);
- if (pm_ret < 0)
- return pm_ret;
+ isl29028_set_pm_runtime_busy(chip, false);
return ret;
}
diff --git a/drivers/iio/light/ltr501.c b/drivers/iio/light/ltr501.c
index 8c0b616815b2..3c9e1dc01951 100644
--- a/drivers/iio/light/ltr501.c
+++ b/drivers/iio/light/ltr501.c
@@ -1558,7 +1558,7 @@ static int ltr501_probe(struct i2c_client *client)
if (ret) {
dev_err(&client->dev, "request irq (%d) failed\n",
client->irq);
- return ret;
+ goto powerdown_on_error;
}
} else {
indio_dev->info = data->chip_info->info_no_irq;
diff --git a/drivers/iio/pressure/dps310.c b/drivers/iio/pressure/dps310.c
index 5a986d5d6b66..aa2284b38878 100644
--- a/drivers/iio/pressure/dps310.c
+++ b/drivers/iio/pressure/dps310.c
@@ -50,9 +50,9 @@
#define DPS310_CFG_REG 0x09
#define DPS310_INT_HL BIT(7)
#define DPS310_TMP_SHIFT_EN BIT(3)
-#define DPS310_PRS_SHIFT_EN BIT(4)
-#define DPS310_FIFO_EN BIT(5)
-#define DPS310_SPI_EN BIT(6)
+#define DPS310_PRS_SHIFT_EN BIT(2)
+#define DPS310_FIFO_EN BIT(1)
+#define DPS310_SPI_EN BIT(0)
#define DPS310_RESET 0x0c
#define DPS310_RESET_MAGIC 0x09
#define DPS310_COEF_BASE 0x10
diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index db96a7547fb7..721f0d367281 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -5576,7 +5576,7 @@ static int init_rcverr(struct hfi1_devdata *dd)
static void free_rcverr(struct hfi1_devdata *dd)
{
if (dd->rcverr_timer.function)
- del_timer_sync(&dd->rcverr_timer);
+ timer_shutdown_sync(&dd->rcverr_timer);
}
static void handle_rxe_err(struct hfi1_devdata *dd, u32 unused, u64 reg)
@@ -12308,7 +12308,7 @@ static void free_cntrs(struct hfi1_devdata *dd)
int i;
if (dd->synth_stats_timer.function)
- del_timer_sync(&dd->synth_stats_timer);
+ timer_shutdown_sync(&dd->synth_stats_timer);
cancel_work_sync(&dd->update_cntr_work);
ppd = (struct hfi1_pportdata *)(dd + 1);
for (i = 0; i < dd->num_pports; i++, ppd++) {
diff --git a/drivers/input/input.c b/drivers/input/input.c
index c51858f1cdc5..05366b82f722 100644
--- a/drivers/input/input.c
+++ b/drivers/input/input.c
@@ -392,7 +392,7 @@ void input_event(struct input_dev *dev,
{
unsigned long flags;
- if (is_event_supported(type, dev->evbit, EV_MAX)) {
+ if (type == EV_SYN || is_event_supported(type, dev->evbit, EV_MAX)) {
spin_lock_irqsave(&dev->event_lock, flags);
input_handle_event(dev, type, code, value);
@@ -419,7 +419,7 @@ void input_inject_event(struct input_handle *handle,
struct input_handle *grab;
unsigned long flags;
- if (is_event_supported(type, dev->evbit, EV_MAX)) {
+ if (type == EV_SYN || is_event_supported(type, dev->evbit, EV_MAX)) {
spin_lock_irqsave(&dev->event_lock, flags);
rcu_read_lock();
diff --git a/drivers/input/joystick/xpad.c b/drivers/input/joystick/xpad.c
index 3086fc4a0642..5e83766a308f 100644
--- a/drivers/input/joystick/xpad.c
+++ b/drivers/input/joystick/xpad.c
@@ -432,6 +432,7 @@ static const struct xpad_device {
{ 0x3537, 0x1004, "GameSir T4 Kaleid", 0, XTYPE_XBOX360 },
{ 0x3537, 0x100f, "GameSir Nova 2 Lite", 0, XTYPE_XBOX360 },
{ 0x3537, 0x1010, "GameSir G7 SE", 0, XTYPE_XBOXONE },
+ { 0x3537, 0x1040, "GameSir Nova Lite", 0, XTYPE_XBOX360 },
{ 0x3767, 0x0101, "Fanatec Speedster 3 Forceshock Wheel", 0, XTYPE_XBOX },
{ 0x413d, 0x2104, "Black Shark Green Ghost Gamepad", 0, XTYPE_XBOX360 },
{ 0xffff, 0xffff, "Chinese-made Xbox Controller", 0, XTYPE_XBOX },
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 54aa09e1174d..835b24524aa8 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -1579,6 +1579,21 @@ static int __init init_iommu_from_acpi(struct amd_iommu *iommu,
return 0;
}
+/*
+ * Apply workaround for Zen1 to Zen5(BRH) generation of processors.
+ * References: CVE-2024-21963
+ */
+static bool __init amd_iommu_force_max_dev_table(void)
+{
+ if (boot_cpu_data.x86 >= 0x17 && boot_cpu_data.x86 <= 0x1a) {
+ if (cpu_feature_enabled(X86_FEATURE_ZEN6))
+ return false;
+ else
+ return true;
+ }
+ return false;
+}
+
/* Allocate PCI segment data structure */
static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
struct acpi_table_header *ivrs_base)
@@ -1601,7 +1616,13 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
pci_seg->last_bdf = last_bdf;
DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
- pci_seg->dev_table_size = tbl_size(DEV_TABLE_ENTRY_SIZE, last_bdf);
+ if (amd_iommu_force_max_dev_table()) {
+ pr_info("Force device table to maximum size\n");
+ pci_seg->dev_table_size = SZ_2M;
+ } else {
+ pci_seg->dev_table_size = tbl_size(DEV_TABLE_ENTRY_SIZE,
+ last_bdf);
+ }
pci_seg->alias_table_size = tbl_size(ALIAS_TABLE_ENTRY_SIZE, last_bdf);
pci_seg->rlookup_table_size = tbl_size(RLOOKUP_TABLE_ENTRY_SIZE, last_bdf);
@@ -2570,6 +2591,15 @@ static int __init init_unity_map_range(struct ivmd_header *m,
return 0;
}
+static bool ivrs_block_type_ivmd(u8 type)
+{
+ if (type == ACPI_IVMD_TYPE ||
+ type == ACPI_IVMD_TYPE_ALL ||
+ type == ACPI_IVMD_TYPE_RANGE)
+ return true;
+ return false;
+}
+
/* iterates over all memory definitions we find in the ACPI table */
static int __init init_memory_definitions(struct acpi_table_header *table)
{
@@ -2581,10 +2611,11 @@ static int __init init_memory_definitions(struct acpi_table_header *table)
while (p < end) {
m = (struct ivmd_header *)p;
+ p += m->length;
+ if (!ivrs_block_type_ivmd(m->type))
+ continue;
if (m->flags & (IVMD_FLAG_UNITY_MAP | IVMD_FLAG_EXCL_RANGE))
init_unity_map_range(m, table);
-
- p += m->length;
}
return 0;
diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 177e8e12f189..382150147f4d 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -505,7 +505,7 @@ iommu_read_and_clear_dirty(struct iommu_domain *domain,
arg.iopt = iopt;
arg.domain = domain;
arg.dirty = &dirty;
- iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
+ ret = iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
if (!(flags & IOMMU_DIRTY_NO_CLEAR))
iommu_iotlb_sync(domain, &gather);
diff --git a/drivers/iommu/iommufd/iova_bitmap.c b/drivers/iommu/iommufd/iova_bitmap.c
index 0943a351fb91..e12a36bf6aed 100644
--- a/drivers/iommu/iommufd/iova_bitmap.c
+++ b/drivers/iommu/iommufd/iova_bitmap.c
@@ -116,6 +116,9 @@ struct iova_bitmap {
/* length of the IOVA range for the whole bitmap */
size_t length;
+
+ /* first error encountered while recording dirty IOVAs */
+ int err;
};
/*
@@ -384,7 +387,9 @@ static int iova_bitmap_advance_to(struct iova_bitmap *bitmap,
int iova_bitmap_for_each(struct iova_bitmap *bitmap, void *opaque,
iova_bitmap_fn_t fn)
{
- return fn(bitmap, bitmap->iova, bitmap->length, opaque);
+ int ret = fn(bitmap, bitmap->iova, bitmap->length, opaque);
+
+ return ret ? ret : bitmap->err;
}
EXPORT_SYMBOL_NS_GPL(iova_bitmap_for_each, IOMMUFD);
@@ -404,6 +409,18 @@ void iova_bitmap_set(struct iova_bitmap *bitmap,
struct iova_bitmap_map *mapped = &bitmap->mapped;
unsigned long cur_bit, last_bit, last_page_idx;
+ /*
+ * If the range cannot be recorded then the system call returns failure.
+ * This is to prevent loosing dirties if userspace is expecting to step
+ * the bitmap through the IOVA space.
+ */
+ if (!length || length > bitmap->length || iova < bitmap->iova ||
+ iova - bitmap->iova > bitmap->length - length) {
+ if (!bitmap->err)
+ bitmap->err = -EPROTO;
+ return;
+ }
+
update_indexes:
if (unlikely(!iova_bitmap_mapped_range(mapped, iova, length))) {
diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c
index b85ce6310ddb..f84a50596f4d 100644
--- a/drivers/iommu/virtio-iommu.c
+++ b/drivers/iommu/virtio-iommu.c
@@ -1191,12 +1191,12 @@ static int viommu_probe(struct virtio_device *vdev)
/* Populate the event queue with buffers */
ret = viommu_fill_evtq(viommu);
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
ret = iommu_device_sysfs_add(&viommu->iommu, dev, NULL, "%s",
virtio_bus_name(vdev));
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
iommu_device_register(&viommu->iommu, &viommu_ops, parent_dev);
@@ -1208,6 +1208,8 @@ static int viommu_probe(struct virtio_device *vdev)
return 0;
+err_reset_vdev:
+ virtio_reset_device(vdev);
err_free_vqs:
vdev->config->del_vqs(vdev);
diff --git a/drivers/irqchip/irq-atmel-aic.c b/drivers/irqchip/irq-atmel-aic.c
index 3839ad79ad31..46c1b380c604 100644
--- a/drivers/irqchip/irq-atmel-aic.c
+++ b/drivers/irqchip/irq-atmel-aic.c
@@ -125,11 +125,12 @@ static void aic_resume(struct irq_data *d)
static void aic_pm_shutdown(struct irq_data *d)
{
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
+ unsigned long flags;
- irq_gc_lock(gc);
+ irq_gc_lock_irqsave(gc, flags);
irq_reg_writel(gc, 0xffffffff, AT91_AIC_IDCR);
irq_reg_writel(gc, 0xffffffff, AT91_AIC_ICCR);
- irq_gc_unlock(gc);
+ irq_gc_unlock_irqrestore(gc, flags);
}
#else
#define aic_suspend NULL
diff --git a/drivers/irqchip/irq-atmel-aic5.c b/drivers/irqchip/irq-atmel-aic5.c
index c0f55dc7b050..28dd53df0d7e 100644
--- a/drivers/irqchip/irq-atmel-aic5.c
+++ b/drivers/irqchip/irq-atmel-aic5.c
@@ -223,15 +223,16 @@ static void aic5_pm_shutdown(struct irq_data *d)
struct irq_domain_chip_generic *dgc = domain->gc;
struct irq_chip_generic *bgc = irq_get_domain_generic_chip(domain, 0);
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
+ unsigned long flags;
int i;
- irq_gc_lock(bgc);
+ irq_gc_lock_irqsave(bgc, flags);
for (i = 0; i < dgc->irqs_per_chip; i++) {
irq_reg_writel(bgc, i + gc->irq_base, AT91_AIC5_SSR);
irq_reg_writel(bgc, 1, AT91_AIC5_IDCR);
irq_reg_writel(bgc, 1, AT91_AIC5_ICCR);
}
- irq_gc_unlock(bgc);
+ irq_gc_unlock_irqrestore(bgc, flags);
}
#else
#define aic5_suspend NULL
diff --git a/drivers/mailbox/mailbox-altera.c b/drivers/mailbox/mailbox-altera.c
index afb320e9d69c..e81c8bac651a 100644
--- a/drivers/mailbox/mailbox-altera.c
+++ b/drivers/mailbox/mailbox-altera.c
@@ -270,7 +270,7 @@ static void altera_mbox_shutdown(struct mbox_chan *chan)
writel_relaxed(~0, mbox->mbox_base + MAILBOX_INTMASK_REG);
free_irq(mbox->irq, chan);
} else if (!mbox->is_sender) {
- del_timer_sync(&mbox->rxpoll_timer);
+ timer_shutdown_sync(&mbox->rxpoll_timer);
}
}
diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
index 601c3e3fe56b..d2a4fc35c838 100644
--- a/drivers/md/dm-bufio.c
+++ b/drivers/md/dm-bufio.c
@@ -2526,6 +2526,13 @@ struct dm_bufio_client *dm_bufio_client_create(struct block_device *bdev, unsign
init_waitqueue_head(&c->free_buffer_wait);
c->async_write_error = 0;
+ /*
+ * A client with no buffers yet is not holding the oldest buffer in
+ * the system; leaving this zeroed would claim exactly that and keep
+ * the client pinned at the front of dm_bufio_all_clients.
+ */
+ c->oldest_buffer = jiffies;
+
c->dm_io = dm_io_client_create();
if (IS_ERR(c->dm_io)) {
r = PTR_ERR(c->dm_io);
@@ -2861,9 +2868,23 @@ static unsigned long __evict_a_few(unsigned long nr_buffers)
count = __evict_many(c, ¶ms, LIST_CLEAN, nr_buffers);
dm_bufio_unlock(c);
- if (count)
+ if (count) {
c->oldest_buffer = params.last_accessed;
- __insert_client(c);
+ __insert_client(c);
+ } else {
+ /*
+ * We evicted nothing, so 'oldest_buffer' tells us nothing new
+ * and the client keeps its old, possibly very stale, value.
+ * Requeue it at the back rather than by that value, otherwise
+ * __insert_client() would put it straight back at the front
+ * and every subsequent call would pop this same client again,
+ * never reaching any of the others.
+ *
+ * This leaves the list slightly out of order, which is fine -
+ * it is only ever an approximation.
+ */
+ list_add_tail(&c->client_list, &dm_bufio_all_clients);
+ }
return count;
}
@@ -2888,11 +2909,23 @@ static void evict_old(void)
{
unsigned long threshold = dm_bufio_cache_size -
dm_bufio_cache_size / DM_BUFIO_LOW_WATERMARK_RATIO;
+ int unproductive = 0;
mutex_lock(&dm_bufio_clients_lock);
while (dm_bufio_current_allocated > threshold) {
- if (!__evict_a_few(64))
- break;
+ if (__evict_a_few(64)) {
+ unproductive = 0;
+ } else {
+ /*
+ * This client had nothing to give. Try the others
+ * before giving up: one idle client, or one whose
+ * buffers are all held, must not stop us evicting
+ * from everybody else. Stop once a whole sweep of
+ * the client list has freed nothing.
+ */
+ if (++unproductive >= dm_bufio_client_count)
+ break;
+ }
cond_resched();
}
mutex_unlock(&dm_bufio_clients_lock);
diff --git a/drivers/media/common/cypress_firmware.c b/drivers/media/common/cypress_firmware.c
index cdc7050ed3ac..3468eff508b0 100644
--- a/drivers/media/common/cypress_firmware.c
+++ b/drivers/media/common/cypress_firmware.c
@@ -59,6 +59,8 @@ static int cypress_get_hexline(const struct firmware *fw,
if (hx->type == 0x04) {
/* b[4] and b[5] are the Extended linear address record data
* field */
+ if (hx->len != 2)
+ return -EINVAL;
hx->addr |= (b[4] << 24) | (b[5] << 16);
}
diff --git a/drivers/media/i2c/adv748x/adv748x-core.c b/drivers/media/i2c/adv748x/adv748x-core.c
index 3eb6d5e8f082..70594d001804 100644
--- a/drivers/media/i2c/adv748x/adv748x-core.c
+++ b/drivers/media/i2c/adv748x/adv748x-core.c
@@ -675,9 +675,6 @@ static int adv748x_parse_dt(struct adv748x_state *state)
continue;
}
- of_node_get(ep_np);
- state->endpoints[ep.port] = ep_np;
-
/*
* At least one input endpoint and one output endpoint shall
* be defined.
@@ -689,8 +686,13 @@ static int adv748x_parse_dt(struct adv748x_state *state)
/* Store number of CSI-2 lanes used for TXA and TXB. */
ret = adv748x_parse_csi2_lanes(state, ep.port, ep_np);
- if (ret)
+ if (ret) {
+ of_node_put(ep_np);
return ret;
+ }
+
+ of_node_get(ep_np);
+ state->endpoints[ep.port] = ep_np;
}
return in_found && out_found ? 0 : -ENODEV;
@@ -739,7 +741,7 @@ static int adv748x_probe(struct i2c_client *client)
ret = adv748x_parse_dt(state);
if (ret) {
adv_err(state, "Failed to parse device tree");
- goto err_free_mutex;
+ goto err_cleanup_dt;
}
/* Configure IO Regmap region */
@@ -809,7 +811,6 @@ static int adv748x_probe(struct i2c_client *client)
adv748x_unregister_clients(state);
err_cleanup_dt:
adv748x_dt_cleanup(state);
-err_free_mutex:
mutex_destroy(&state->mutex);
return ret;
diff --git a/drivers/media/i2c/og01a1b.c b/drivers/media/i2c/og01a1b.c
index 68573122cd6e..cc8ff6ffd29a 100644
--- a/drivers/media/i2c/og01a1b.c
+++ b/drivers/media/i2c/og01a1b.c
@@ -442,9 +442,6 @@ struct og01a1b {
/* Current mode */
const struct og01a1b_mode *cur_mode;
-
- /* To serialize asynchronus callbacks */
- struct mutex mutex;
};
static u64 to_pixel_rate(u32 f_index)
@@ -616,7 +613,6 @@ static int og01a1b_init_controls(struct og01a1b *og01a1b)
if (ret)
return ret;
- ctrl_hdlr->lock = &og01a1b->mutex;
og01a1b->link_freq = v4l2_ctrl_new_int_menu(ctrl_hdlr,
&og01a1b_ctrl_ops,
V4L2_CID_LINK_FREQ,
@@ -686,74 +682,66 @@ static void og01a1b_update_pad_format(const struct og01a1b_mode *mode,
fmt->field = V4L2_FIELD_NONE;
}
-static int og01a1b_start_streaming(struct og01a1b *og01a1b)
+static int og01a1b_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
{
+ struct og01a1b *og01a1b = to_og01a1b(sd);
+ unsigned int link_freq_index = og01a1b->cur_mode->link_freq_index;
const struct og01a1b_reg_list *reg_list;
- int link_freq_index, ret;
+ int ret;
- link_freq_index = og01a1b->cur_mode->link_freq_index;
- reg_list = &link_freq_configs[link_freq_index].reg_list;
+ ret = pm_runtime_resume_and_get(og01a1b->dev);
+ if (ret)
+ return ret;
+ reg_list = &link_freq_configs[link_freq_index].reg_list;
ret = og01a1b_write_reg_list(og01a1b, reg_list);
if (ret) {
- dev_err(og01a1b->dev, "failed to set plls");
- return ret;
+ dev_err(og01a1b->dev, "failed to set plls: %d\n", ret);
+ goto error;
}
reg_list = &og01a1b->cur_mode->reg_list;
ret = og01a1b_write_reg_list(og01a1b, reg_list);
if (ret) {
- dev_err(og01a1b->dev, "failed to set mode");
- return ret;
+ dev_err(og01a1b->dev, "failed to set mode: %d\n", ret);
+ goto error;
}
ret = __v4l2_ctrl_handler_setup(og01a1b->sd.ctrl_handler);
if (ret)
- return ret;
+ goto error;
ret = og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT,
OG01A1B_REG_VALUE_08BIT,
OG01A1B_MODE_STREAMING);
if (ret) {
- dev_err(og01a1b->dev, "failed to set stream");
- return ret;
+ dev_err(og01a1b->dev, "failed to start streaming: %d\n", ret);
+ goto error;
}
return 0;
-}
-static void og01a1b_stop_streaming(struct og01a1b *og01a1b)
-{
- if (og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT,
- OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STANDBY))
- dev_err(og01a1b->dev, "failed to set stream");
+error:
+ pm_runtime_put_autosuspend(og01a1b->dev);
+
+ return ret;
}
-static int og01a1b_set_stream(struct v4l2_subdev *sd, int enable)
+static int og01a1b_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
{
struct og01a1b *og01a1b = to_og01a1b(sd);
- int ret = 0;
-
- mutex_lock(&og01a1b->mutex);
- if (enable) {
- ret = pm_runtime_resume_and_get(og01a1b->dev);
- if (ret) {
- mutex_unlock(&og01a1b->mutex);
- return ret;
- }
+ int ret;
- ret = og01a1b_start_streaming(og01a1b);
- if (ret) {
- enable = 0;
- og01a1b_stop_streaming(og01a1b);
- pm_runtime_put(og01a1b->dev);
- }
- } else {
- og01a1b_stop_streaming(og01a1b);
- pm_runtime_put(og01a1b->dev);
- }
+ ret = og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT,
+ OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STANDBY);
+ if (ret)
+ dev_err(og01a1b->dev, "failed to stop streaming: %d\n", ret);
- mutex_unlock(&og01a1b->mutex);
+ pm_runtime_put_autosuspend(og01a1b->dev);
return ret;
}
@@ -771,7 +759,6 @@ static int og01a1b_set_format(struct v4l2_subdev *sd,
height, fmt->format.width,
fmt->format.height);
- mutex_lock(&og01a1b->mutex);
og01a1b_update_pad_format(mode, &fmt->format);
if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
*v4l2_subdev_state_get_format(sd_state, fmt->pad) = fmt->format;
@@ -794,8 +781,6 @@ static int og01a1b_set_format(struct v4l2_subdev *sd,
h_blank);
}
- mutex_unlock(&og01a1b->mutex);
-
return 0;
}
@@ -805,15 +790,12 @@ static int og01a1b_get_format(struct v4l2_subdev *sd,
{
struct og01a1b *og01a1b = to_og01a1b(sd);
- mutex_lock(&og01a1b->mutex);
if (fmt->which == V4L2_SUBDEV_FORMAT_TRY)
fmt->format = *v4l2_subdev_state_get_format(sd_state,
fmt->pad);
else
og01a1b_update_pad_format(og01a1b->cur_mode, &fmt->format);
- mutex_unlock(&og01a1b->mutex);
-
return 0;
}
@@ -849,18 +831,14 @@ static int og01a1b_enum_frame_size(struct v4l2_subdev *sd,
static int og01a1b_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
{
- struct og01a1b *og01a1b = to_og01a1b(sd);
-
- mutex_lock(&og01a1b->mutex);
og01a1b_update_pad_format(&supported_modes[0],
v4l2_subdev_state_get_format(fh->state, 0));
- mutex_unlock(&og01a1b->mutex);
return 0;
}
static const struct v4l2_subdev_video_ops og01a1b_video_ops = {
- .s_stream = og01a1b_set_stream,
+ .s_stream = v4l2_subdev_s_stream_helper,
};
static const struct v4l2_subdev_pad_ops og01a1b_pad_ops = {
@@ -868,6 +846,8 @@ static const struct v4l2_subdev_pad_ops og01a1b_pad_ops = {
.get_fmt = og01a1b_get_format,
.enum_mbus_code = og01a1b_enum_mbus_code,
.enum_frame_size = og01a1b_enum_frame_size,
+ .enable_streams = og01a1b_enable_streams,
+ .disable_streams = og01a1b_disable_streams,
};
static const struct v4l2_subdev_ops og01a1b_subdev_ops = {
@@ -1062,7 +1042,11 @@ static void og01a1b_remove(struct i2c_client *client)
media_entity_cleanup(&sd->entity);
v4l2_ctrl_handler_free(sd->ctrl_handler);
pm_runtime_disable(og01a1b->dev);
- mutex_destroy(&og01a1b->mutex);
+
+ if (!pm_runtime_status_suspended(og01a1b->dev)) {
+ og01a1b_power_off(og01a1b->dev);
+ pm_runtime_set_suspended(og01a1b->dev);
+ }
}
static int og01a1b_probe(struct i2c_client *client)
@@ -1146,7 +1130,6 @@ static int og01a1b_probe(struct i2c_client *client)
goto power_off;
}
- mutex_init(&og01a1b->mutex);
og01a1b->cur_mode = &supported_modes[0];
ret = og01a1b_init_controls(og01a1b);
if (ret) {
@@ -1154,6 +1137,7 @@ static int og01a1b_probe(struct i2c_client *client)
goto probe_error_v4l2_ctrl_handler_free;
}
+ og01a1b->sd.state_lock = og01a1b->ctrl_handler.lock;
og01a1b->sd.internal_ops = &og01a1b_internal_ops;
og01a1b->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
og01a1b->sd.entity.ops = &og01a1b_subdev_entity_ops;
@@ -1194,7 +1178,6 @@ static int og01a1b_probe(struct i2c_client *client)
probe_error_v4l2_ctrl_handler_free:
v4l2_ctrl_handler_free(og01a1b->sd.ctrl_handler);
- mutex_destroy(&og01a1b->mutex);
power_off:
og01a1b_power_off(og01a1b->dev);
diff --git a/drivers/media/i2c/ov5647.c b/drivers/media/i2c/ov5647.c
index 305677913a0c..3e7fd8f28df3 100644
--- a/drivers/media/i2c/ov5647.c
+++ b/drivers/media/i2c/ov5647.c
@@ -1481,6 +1481,10 @@ static void ov5647_remove(struct i2c_client *client)
v4l2_ctrl_handler_free(&sensor->ctrls);
v4l2_device_unregister_subdev(sd);
pm_runtime_disable(&client->dev);
+ if (!pm_runtime_status_suspended(&client->dev)) {
+ ov5647_power_off(&client->dev);
+ pm_runtime_set_suspended(&client->dev);
+ }
mutex_destroy(&sensor->lock);
}
diff --git a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
index 62ed92ff1d30..51d02649d7a7 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
@@ -218,7 +218,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
unsigned int max_send_queues, max_sram_blocks, max_devq_size;
struct ipu6_fw_syscom_queue_config *input_queue_cfg;
struct ipu6_fw_syscom_queue_config *output_queue_cfg;
- struct device *dev = &isys->adev->auxdev.dev;
int type_proxy = IPU6_FW_ISYS_QUEUE_TYPE_PROXY;
int type_dev = IPU6_FW_ISYS_QUEUE_TYPE_DEV;
int type_msg = IPU6_FW_ISYS_QUEUE_TYPE_MSG;
@@ -228,7 +227,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
struct ipu6_fw_isys_fw_config *isys_fw_cfg;
u32 num_in_message_queues;
unsigned int max_streams;
- unsigned int size;
unsigned int i;
max_streams = isys->pdata->ipdata->max_streams;
@@ -236,7 +234,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
max_sram_blocks = isys->pdata->ipdata->max_sram_blocks;
max_devq_size = isys->pdata->ipdata->max_devq_size;
num_in_message_queues = clamp(num_streams, 1U, max_streams);
- isys_fw_cfg = devm_kzalloc(dev, sizeof(*isys_fw_cfg), GFP_KERNEL);
+ isys_fw_cfg = kzalloc_obj(*isys_fw_cfg);
if (!isys_fw_cfg)
return -ENOMEM;
@@ -248,15 +246,14 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
isys_fw_cfg->num_recv_queues[type_dev] = 0;
isys_fw_cfg->num_recv_queues[type_msg] = 1;
- size = sizeof(*input_queue_cfg) * max_send_queues;
- input_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ input_queue_cfg = kzalloc_objs(*input_queue_cfg, max_send_queues);
if (!input_queue_cfg)
- return -ENOMEM;
+ goto err_free_fw_cfg;
- size = sizeof(*output_queue_cfg) * IPU6_N_MAX_RECV_QUEUES;
- output_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ output_queue_cfg = kzalloc_objs(*output_queue_cfg,
+ IPU6_N_MAX_RECV_QUEUES);
if (!output_queue_cfg)
- return -ENOMEM;
+ goto err_free_input_queue_cfg;
fwcom->input = input_queue_cfg;
fwcom->output = output_queue_cfg;
@@ -319,6 +316,13 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
fwcom->specific_size = sizeof(*isys_fw_cfg);
return 0;
+
+err_free_input_queue_cfg:
+ kfree(input_queue_cfg);
+err_free_fw_cfg:
+ kfree(isys_fw_cfg);
+
+ return -ENOMEM;
}
int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
@@ -332,10 +336,15 @@ int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
};
int ret;
- ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams);
+ ret = ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams);
+ if (ret)
+ return ret;
isys->fwcom = ipu6_fw_com_prepare(&fwcom, isys->adev,
isys->pdata->base);
+ kfree(fwcom.specific_addr);
+ kfree(fwcom.output);
+ kfree(fwcom.input);
if (!isys->fwcom) {
dev_err(dev, "isys fw com prepare failed\n");
return -EIO;
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.c b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
index 5ec14a7c70c7..d38be4761d4e 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
@@ -300,6 +300,13 @@ static int wave5_vpu_probe(struct platform_device *pdev)
dev_err(&pdev->dev, "failed to get irq resource, falling back to polling\n");
sema_init(&dev->irq_sem, 1);
dev->irq_thread = kthread_run(irq_thread, dev, "irq thread");
+ if (IS_ERR(dev->irq_thread)) {
+ ret = PTR_ERR(dev->irq_thread);
+ dev_err(&pdev->dev, "failed to create irq thread\n");
+ dev->irq_thread = NULL;
+ goto err_vdi_release;
+ }
+
hrtimer_init(&dev->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED);
dev->hrtimer.function = &wave5_vpu_timer_callback;
dev->worker = kthread_create_worker(0, "vpu_irq_thread");
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 3a7027459731..a69ca8fc5701 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1584,7 +1584,7 @@ static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
v4l2_m2m_buf_done(src_buf, buf_state);
v4l2_m2m_buf_done(dst_buf, buf_state);
v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
- pm_runtime_put(ctx->jpeg->dev);
+ pm_runtime_put(jpeg->dev);
return IRQ_HANDLED;
}
@@ -1871,7 +1871,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
v4l2_m2m_buf_done(src_buf, buf_state);
v4l2_m2m_buf_done(dst_buf, buf_state);
v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
- pm_runtime_put(ctx->jpeg->dev);
+ pm_runtime_put(jpeg->dev);
return IRQ_HANDLED;
}
diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c
index 97cd1f74ff2b..9c8d02722f92 100644
--- a/drivers/media/platform/microchip/microchip-isc-base.c
+++ b/drivers/media/platform/microchip/microchip-isc-base.c
@@ -8,6 +8,7 @@
* Author: Eugen Hristev <eugen.hristev@microchip.com>
*
*/
+#include <linux/bitfield.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/math64.h>
@@ -62,17 +63,17 @@ static inline void isc_update_awb_ctrls(struct isc_device *isc)
/* In here we set our actual hw pipeline config */
regmap_write(isc->regmap, ISC_WB_O_RGR,
- ((ctrls->offset[ISC_HIS_CFG_MODE_R])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GR]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_O_BGB,
- ((ctrls->offset[ISC_HIS_CFG_MODE_B])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GB]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GB]));
regmap_write(isc->regmap, ISC_WB_G_RGR,
- ctrls->gain[ISC_HIS_CFG_MODE_R] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GR] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_G_BGB,
- ctrls->gain[ISC_HIS_CFG_MODE_B] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GB] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GB]));
}
static inline void isc_reset_awb_ctrls(struct isc_device *isc)
@@ -291,7 +292,8 @@ static int isc_configure(struct isc_device *isc)
struct isc_subdev_entity *subdev = isc->current_subdev;
int ret;
- pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps;
+ pfe_cfg0 = FIELD_PREP(ISC_PFE_CFG0_BPS_MASK,
+ isc->config.sd_format->pfe_cfg0_bps);
pipeline = isc->config.bits_pipeline;
dcfg = isc->config.dcfg_imode | isc->dcfg;
diff --git a/drivers/media/platform/microchip/microchip-isc-regs.h b/drivers/media/platform/microchip/microchip-isc-regs.h
index e77e1d9a1db8..fe145b142b82 100644
--- a/drivers/media/platform/microchip/microchip-isc-regs.h
+++ b/drivers/media/platform/microchip/microchip-isc-regs.h
@@ -31,11 +31,11 @@
#define ISC_PFE_CFG0_MODE_PROGRESSIVE (0x0 << 4)
#define ISC_PFE_CFG0_MODE_MASK GENMASK(6, 4)
-#define ISC_PFE_CFG0_BPS_EIGHT (0x4 << 28)
-#define ISC_PFG_CFG0_BPS_NINE (0x3 << 28)
-#define ISC_PFG_CFG0_BPS_TEN (0x2 << 28)
-#define ISC_PFG_CFG0_BPS_ELEVEN (0x1 << 28)
-#define ISC_PFG_CFG0_BPS_TWELVE (0x0 << 28)
+#define ISC_PFE_CFG0_BPS_EIGHT 0x4
+#define ISC_PFE_CFG0_BPS_NINE 0x3
+#define ISC_PFE_CFG0_BPS_TEN 0x2
+#define ISC_PFE_CFG0_BPS_ELEVEN 0x1
+#define ISC_PFE_CFG0_BPS_TWELVE 0x0
#define ISC_PFE_CFG0_BPS_MASK GENMASK(30, 28)
#define ISC_PFE_CFG0_COLEN BIT(12)
@@ -149,6 +149,12 @@
/* ISC White Balance Gain for B, GB Register */
#define ISC_WB_G_BGB 0x0000006c
+/* Each WB offset/gain register packs two 13-bit fields, low and high */
+#define ISC_WB_O_LO GENMASK(12, 0) /* R or B offset [12:0] */
+#define ISC_WB_O_HI GENMASK(28, 16) /* GR or GB offset [28:16] */
+#define ISC_WB_G_LO GENMASK(12, 0) /* R or B gain [12:0] */
+#define ISC_WB_G_HI GENMASK(28, 16) /* GR or GB gain [28:16] */
+
/* ISC Color Filter Array Control Register */
#define ISC_CFA_CTRL 0x00000070
diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h
index f5e322c2e36b..d7bcd74efff9 100644
--- a/drivers/media/platform/microchip/microchip-isc.h
+++ b/drivers/media/platform/microchip/microchip-isc.h
@@ -62,7 +62,9 @@ struct isc_subdev_entity {
* @mbus_code: V4L2 media bus format code.
* @cfa_baycfg: If this format is RAW BAYER, indicate the type of bayer.
this is either BGBG, RGRG, etc.
- * @pfe_cfg0_bps: Number of hardware data lines connected to the ISC
+ * @pfe_cfg0_bps: ISC_PFE_CFG0 BPS field value (e.g. ISC_PFE_CFG0_BPS_EIGHT),
+ written into PFE_CFG0 with FIELD_PREP(ISC_PFE_CFG0_BPS_MASK)
+ at configure time.
* @raw: If the format is raw bayer.
*/
diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
index 583d3c4dca54..a5d9d37b4c40 100644
--- a/drivers/media/platform/microchip/microchip-sama5d2-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
@@ -147,49 +147,49 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -210,7 +210,7 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
index d2e1899869c4..ee4f66526bd6 100644
--- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
@@ -156,49 +156,49 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -224,7 +224,7 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/platform/qcom/camss/camss-vfe-17x.c b/drivers/media/platform/qcom/camss/camss-vfe-17x.c
index bf9aab765858..62c2405359a0 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe-17x.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe-17x.c
@@ -14,8 +14,6 @@
#include "camss.h"
#include "camss-vfe.h"
-#define VFE_HW_VERSION (0x000)
-
#define VFE_GLOBAL_RESET_CMD (0x018)
#define GLOBAL_RESET_CMD_CORE BIT(0)
#define GLOBAL_RESET_CMD_CAMIF BIT(1)
@@ -176,20 +174,6 @@
#define VFE_BUS_WM_FRAME_INC(n) (0x2258 + (n) * 0x100)
#define VFE_BUS_WM_BURST_LIMIT(n) (0x225c + (n) * 0x100)
-static u32 vfe_hw_version(struct vfe_device *vfe)
-{
- u32 hw_version = readl_relaxed(vfe->base + VFE_HW_VERSION);
-
- u32 gen = (hw_version >> 28) & 0xF;
- u32 rev = (hw_version >> 16) & 0xFFF;
- u32 step = hw_version & 0xFFFF;
-
- dev_dbg(vfe->camss->dev, "VFE HW Version = %u.%u.%u\n",
- gen, rev, step);
-
- return hw_version;
-}
-
static inline void vfe_reg_set(struct vfe_device *vfe, u32 reg, u32 set_bits)
{
u32 bits = readl_relaxed(vfe->base + reg);
@@ -398,101 +382,6 @@ static int vfe_halt(struct vfe_device *vfe)
return 0;
}
-static int vfe_get_output(struct vfe_line *line)
-{
- struct vfe_device *vfe = to_vfe(line);
- struct vfe_output *output;
- unsigned long flags;
- int wm_idx;
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- output = &line->output;
- if (output->state > VFE_OUTPUT_RESERVED) {
- dev_err(vfe->camss->dev, "Output is running\n");
- goto error;
- }
-
- output->wm_num = 1;
-
- wm_idx = vfe_reserve_wm(vfe, line->id);
- if (wm_idx < 0) {
- dev_err(vfe->camss->dev, "Can not reserve wm\n");
- goto error_get_wm;
- }
- output->wm_idx[0] = wm_idx;
-
- output->drop_update_idx = 0;
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return 0;
-
-error_get_wm:
- vfe_release_wm(vfe, output->wm_idx[0]);
- output->state = VFE_OUTPUT_OFF;
-error:
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return -EINVAL;
-}
-
-static int vfe_enable_output(struct vfe_line *line)
-{
- struct vfe_device *vfe = to_vfe(line);
- struct vfe_output *output = &line->output;
- const struct vfe_hw_ops *ops = vfe->res->hw_ops;
- struct media_entity *sensor;
- unsigned long flags;
- unsigned int frame_skip = 0;
- unsigned int i;
-
- sensor = camss_find_sensor(&line->subdev.entity);
- if (sensor) {
- struct v4l2_subdev *subdev = media_entity_to_v4l2_subdev(sensor);
-
- v4l2_subdev_call(subdev, sensor, g_skip_frames, &frame_skip);
- /* Max frame skip is 29 frames */
- if (frame_skip > VFE_FRAME_DROP_VAL - 1)
- frame_skip = VFE_FRAME_DROP_VAL - 1;
- }
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- ops->reg_update_clear(vfe, line->id);
-
- if (output->state > VFE_OUTPUT_RESERVED) {
- dev_err(vfe->camss->dev, "Output is not in reserved state %d\n",
- output->state);
- spin_unlock_irqrestore(&vfe->output_lock, flags);
- return -EINVAL;
- }
-
- WARN_ON(output->gen2.active_num);
-
- output->state = VFE_OUTPUT_ON;
-
- output->sequence = 0;
- output->wait_reg_update = 0;
- reinit_completion(&output->reg_update);
-
- vfe_wm_start(vfe, output->wm_idx[0], line);
-
- for (i = 0; i < 2; i++) {
- output->buf[i] = vfe_buf_get_pending(output);
- if (!output->buf[i])
- break;
- output->gen2.active_num++;
- vfe_wm_update(vfe, output->wm_idx[0], output->buf[i]->addr[0], line);
- }
-
- ops->reg_update(vfe, line->id);
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return 0;
-}
-
/*
* vfe_enable - Enable streaming on VFE line
* @line: VFE line
@@ -513,11 +402,11 @@ static int vfe_enable(struct vfe_line *line)
mutex_unlock(&vfe->stream_lock);
- ret = vfe_get_output(line);
+ ret = vfe_get_output_v2(line);
if (ret < 0)
goto error_get_output;
- ret = vfe_enable_output(line);
+ ret = vfe_enable_output_v2(line);
if (ret < 0)
goto error_enable_output;
@@ -626,40 +515,6 @@ static void vfe_isr_wm_done(struct vfe_device *vfe, u8 wm)
spin_unlock_irqrestore(&vfe->output_lock, flags);
}
-/*
- * vfe_queue_buffer - Add empty buffer
- * @vid: Video device structure
- * @buf: Buffer to be enqueued
- *
- * Add an empty buffer - depending on the current number of buffers it will be
- * put in pending buffer queue or directly given to the hardware to be filled.
- *
- * Return 0 on success or a negative error code otherwise
- */
-static int vfe_queue_buffer(struct camss_video *vid,
- struct camss_buffer *buf)
-{
- struct vfe_line *line = container_of(vid, struct vfe_line, video_out);
- struct vfe_device *vfe = to_vfe(line);
- struct vfe_output *output;
- unsigned long flags;
-
- output = &line->output;
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- if (output->state == VFE_OUTPUT_ON && output->gen2.active_num < 2) {
- output->buf[output->gen2.active_num++] = buf;
- vfe_wm_update(vfe, output->wm_idx[0], buf->addr[0], line);
- } else {
- vfe_buf_add_pending(output, buf);
- }
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return 0;
-}
-
static const struct vfe_isr_ops vfe_isr_ops_170 = {
.reset_ack = vfe_isr_reset_ack,
.halt_ack = vfe_isr_halt_ack,
@@ -670,7 +525,7 @@ static const struct vfe_isr_ops vfe_isr_ops_170 = {
};
static const struct camss_video_ops vfe_video_ops_170 = {
- .queue_buffer = vfe_queue_buffer,
+ .queue_buffer = vfe_queue_buffer_v2,
.flush_buffers = vfe_flush_buffers,
};
@@ -694,5 +549,7 @@ const struct vfe_hw_ops vfe_ops_170 = {
.vfe_enable = vfe_enable,
.vfe_halt = vfe_halt,
.violation_read = vfe_violation_read,
+ .vfe_wm_start = vfe_wm_start,
.vfe_wm_stop = vfe_wm_stop,
+ .vfe_wm_update = vfe_wm_update,
};
diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-1.c b/drivers/media/platform/qcom/camss/camss-vfe-4-1.c
index 1bd3a6ef1d04..6930799f77c2 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe-4-1.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe-4-1.c
@@ -210,15 +210,6 @@
#define MSM_VFE_VFE0_UB_SIZE 1023
#define MSM_VFE_VFE0_UB_SIZE_RDI (MSM_VFE_VFE0_UB_SIZE / 3)
-static u32 vfe_hw_version(struct vfe_device *vfe)
-{
- u32 hw_version = readl_relaxed(vfe->base + VFE_0_HW_VERSION);
-
- dev_dbg(vfe->camss->dev, "VFE HW Version = 0x%08x\n", hw_version);
-
- return hw_version;
-}
-
static u16 vfe_get_ub_size(u8 vfe_id)
{
if (vfe_id == 0)
diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-7.c b/drivers/media/platform/qcom/camss/camss-vfe-4-7.c
index ce0719106bd3..76729607db02 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe-4-7.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe-4-7.c
@@ -18,8 +18,6 @@
#include "camss-vfe-gen1.h"
-#define VFE_0_HW_VERSION 0x000
-
#define VFE_0_GLOBAL_RESET_CMD 0x018
#define VFE_0_GLOBAL_RESET_CMD_CORE BIT(0)
#define VFE_0_GLOBAL_RESET_CMD_CAMIF BIT(1)
@@ -254,15 +252,6 @@
#define MSM_VFE_VFE1_UB_SIZE 1535
#define MSM_VFE_VFE1_UB_SIZE_RDI (MSM_VFE_VFE1_UB_SIZE / 3)
-static u32 vfe_hw_version(struct vfe_device *vfe)
-{
- u32 hw_version = readl_relaxed(vfe->base + VFE_0_HW_VERSION);
-
- dev_dbg(vfe->camss->dev, "VFE HW Version = 0x%08x\n", hw_version);
-
- return hw_version;
-}
-
static u16 vfe_get_ub_size(u8 vfe_id)
{
if (vfe_id == 0)
diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-8.c b/drivers/media/platform/qcom/camss/camss-vfe-4-8.c
index 6b59c8107a3c..b2f7d855d8dd 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe-4-8.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe-4-8.c
@@ -17,8 +17,6 @@
#include "camss-vfe.h"
#include "camss-vfe-gen1.h"
-#define VFE_0_HW_VERSION 0x000
-
#define VFE_0_GLOBAL_RESET_CMD 0x018
#define VFE_0_GLOBAL_RESET_CMD_CORE BIT(0)
#define VFE_0_GLOBAL_RESET_CMD_CAMIF BIT(1)
@@ -247,15 +245,6 @@
#define MSM_VFE_VFE1_UB_SIZE 1535
#define MSM_VFE_VFE1_UB_SIZE_RDI (MSM_VFE_VFE1_UB_SIZE / 3)
-static u32 vfe_hw_version(struct vfe_device *vfe)
-{
- u32 hw_version = readl_relaxed(vfe->base + VFE_0_HW_VERSION);
-
- dev_dbg(vfe->camss->dev, "VFE HW Version = 0x%08x\n", hw_version);
-
- return hw_version;
-}
-
static inline void vfe_reg_clr(struct vfe_device *vfe, u32 reg, u32 clr_bits)
{
u32 bits = readl_relaxed(vfe->base + reg);
diff --git a/drivers/media/platform/qcom/camss/camss-vfe-480.c b/drivers/media/platform/qcom/camss/camss-vfe-480.c
index 003e1900d3c9..d73f733fde04 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe-480.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe-480.c
@@ -15,8 +15,6 @@
#include "camss.h"
#include "camss-vfe.h"
-#define VFE_HW_VERSION (0x00)
-
#define VFE_GLOBAL_RESET_CMD (vfe_is_lite(vfe) ? 0x0c : 0x1c)
#define GLOBAL_RESET_HW_AND_REG (vfe_is_lite(vfe) ? BIT(1) : BIT(0))
@@ -92,19 +90,6 @@ static inline int bus_irq_mask_0_comp_done(struct vfe_device *vfe, int n)
#define MAX_VFE_OUTPUT_LINES 4
-static u32 vfe_hw_version(struct vfe_device *vfe)
-{
- u32 hw_version = readl_relaxed(vfe->base + VFE_HW_VERSION);
-
- u32 gen = (hw_version >> 28) & 0xF;
- u32 rev = (hw_version >> 16) & 0xFFF;
- u32 step = hw_version & 0xFFFF;
-
- dev_dbg(vfe->camss->dev, "VFE HW Version = %u.%u.%u\n", gen, rev, step);
-
- return hw_version;
-}
-
static void vfe_global_reset(struct vfe_device *vfe)
{
writel_relaxed(IRQ_MASK_0_RESET_ACK, vfe->base + VFE_IRQ_MASK(0));
@@ -167,18 +152,16 @@ static inline void vfe_reg_update_clear(struct vfe_device *vfe,
vfe->reg_update &= ~REG_UPDATE_RDI(vfe, line_id);
}
-static void vfe_enable_irq_common(struct vfe_device *vfe)
-{
- /* enable reset ack IRQ and top BUS status IRQ */
- writel_relaxed(IRQ_MASK_0_RESET_ACK | IRQ_MASK_0_BUS_TOP_IRQ,
- vfe->base + VFE_IRQ_MASK(0));
-}
-
-static void vfe_enable_lines_irq(struct vfe_device *vfe)
+static void vfe_enable_irq(struct vfe_device *vfe)
{
int i;
u32 bus_irq_mask = 0;
+ if (!vfe->stream_count)
+ /* enable reset ack IRQ and top BUS status IRQ */
+ writel(IRQ_MASK_0_RESET_ACK | IRQ_MASK_0_BUS_TOP_IRQ,
+ vfe->base + VFE_IRQ_MASK(0));
+
for (i = 0; i < MAX_VFE_OUTPUT_LINES; i++) {
/* Enable IRQ for newly added lines, but also keep already running lines's IRQ */
if (vfe->line[i].output.state == VFE_OUTPUT_RESERVED ||
@@ -188,11 +171,10 @@ static void vfe_enable_lines_irq(struct vfe_device *vfe)
}
}
- writel_relaxed(bus_irq_mask, vfe->base + VFE_BUS_IRQ_MASK(0));
+ writel(bus_irq_mask, vfe->base + VFE_BUS_IRQ_MASK(0));
}
static void vfe_isr_reg_update(struct vfe_device *vfe, enum vfe_line_id line_id);
-static void vfe_isr_wm_done(struct vfe_device *vfe, u8 wm);
/*
* vfe_isr - VFE module interrupt handler
@@ -228,7 +210,7 @@ static irqreturn_t vfe_isr(int irq, void *dev)
/* Loop through all WMs IRQs */
for (i = 0; i < MSM_VFE_IMAGE_MASTERS_NUM; i++) {
if (status & BUS_IRQ_MASK_0_COMP_DONE(vfe, RDI_COMP_GROUP(i)))
- vfe_isr_wm_done(vfe, i);
+ vfe_buf_done(vfe, i);
}
}
@@ -247,132 +229,6 @@ static int vfe_halt(struct vfe_device *vfe)
return 0;
}
-static int vfe_get_output(struct vfe_line *line)
-{
- struct vfe_device *vfe = to_vfe(line);
- struct vfe_output *output;
- unsigned long flags;
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- output = &line->output;
- if (output->state > VFE_OUTPUT_RESERVED) {
- dev_err(vfe->camss->dev, "Output is running\n");
- goto error;
- }
-
- output->wm_num = 1;
-
- /* Correspondence between VFE line number and WM number.
- * line 0 -> RDI 0, line 1 -> RDI1, line 2 -> RDI2, line 3 -> PIX/RDI3
- * Note this 1:1 mapping will not work for PIX streams.
- */
- output->wm_idx[0] = line->id;
- vfe->wm_output_map[line->id] = line->id;
-
- output->drop_update_idx = 0;
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return 0;
-
-error:
- spin_unlock_irqrestore(&vfe->output_lock, flags);
- output->state = VFE_OUTPUT_OFF;
-
- return -EINVAL;
-}
-
-static int vfe_enable_output(struct vfe_line *line)
-{
- struct vfe_device *vfe = to_vfe(line);
- struct vfe_output *output = &line->output;
- unsigned long flags;
- unsigned int i;
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- vfe_reg_update_clear(vfe, line->id);
-
- if (output->state > VFE_OUTPUT_RESERVED) {
- dev_err(vfe->camss->dev, "Output is not in reserved state %d\n",
- output->state);
- spin_unlock_irqrestore(&vfe->output_lock, flags);
- return -EINVAL;
- }
-
- WARN_ON(output->gen2.active_num);
-
- output->state = VFE_OUTPUT_ON;
-
- output->sequence = 0;
- output->wait_reg_update = 0;
- reinit_completion(&output->reg_update);
-
- vfe_wm_start(vfe, output->wm_idx[0], line);
-
- for (i = 0; i < 2; i++) {
- output->buf[i] = vfe_buf_get_pending(output);
- if (!output->buf[i])
- break;
- output->gen2.active_num++;
- vfe_wm_update(vfe, output->wm_idx[0], output->buf[i]->addr[0], line);
- }
-
- vfe_reg_update(vfe, line->id);
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return 0;
-}
-
-/*
- * vfe_enable - Enable streaming on VFE line
- * @line: VFE line
- *
- * Return 0 on success or a negative error code otherwise
- */
-static int vfe_enable(struct vfe_line *line)
-{
- struct vfe_device *vfe = to_vfe(line);
- int ret;
-
- mutex_lock(&vfe->stream_lock);
-
- if (!vfe->stream_count)
- vfe_enable_irq_common(vfe);
-
- vfe->stream_count++;
-
- vfe_enable_lines_irq(vfe);
-
- mutex_unlock(&vfe->stream_lock);
-
- ret = vfe_get_output(line);
- if (ret < 0)
- goto error_get_output;
-
- ret = vfe_enable_output(line);
- if (ret < 0)
- goto error_enable_output;
-
- vfe->was_streaming = 1;
-
- return 0;
-
-error_enable_output:
- vfe_put_output(line);
-
-error_get_output:
- mutex_lock(&vfe->stream_lock);
-
- vfe->stream_count--;
-
- mutex_unlock(&vfe->stream_lock);
-
- return ret;
-}
-
/*
* vfe_isr_reg_update - Process reg update interrupt
* @vfe: VFE Device
@@ -396,114 +252,48 @@ static void vfe_isr_reg_update(struct vfe_device *vfe, enum vfe_line_id line_id)
spin_unlock_irqrestore(&vfe->output_lock, flags);
}
-/*
- * vfe_isr_wm_done - Process write master done interrupt
- * @vfe: VFE Device
- * @wm: Write master id
- */
-static void vfe_isr_wm_done(struct vfe_device *vfe, u8 wm)
-{
- struct vfe_line *line = &vfe->line[vfe->wm_output_map[wm]];
- struct camss_buffer *ready_buf;
- struct vfe_output *output;
- unsigned long flags;
- u32 index;
- u64 ts = ktime_get_ns();
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- if (vfe->wm_output_map[wm] == VFE_LINE_NONE) {
- dev_err_ratelimited(vfe->camss->dev,
- "Received wm done for unmapped index\n");
- goto out_unlock;
- }
- output = &vfe->line[vfe->wm_output_map[wm]].output;
-
- ready_buf = output->buf[0];
- if (!ready_buf) {
- dev_err_ratelimited(vfe->camss->dev,
- "Missing ready buf %d!\n", output->state);
- goto out_unlock;
- }
-
- ready_buf->vb.vb2_buf.timestamp = ts;
- ready_buf->vb.sequence = output->sequence++;
-
- index = 0;
- output->buf[0] = output->buf[1];
- if (output->buf[0])
- index = 1;
-
- output->buf[index] = vfe_buf_get_pending(output);
-
- if (output->buf[index])
- vfe_wm_update(vfe, output->wm_idx[0], output->buf[index]->addr[0], line);
- else
- output->gen2.active_num--;
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
-
- return;
+static const struct camss_video_ops vfe_video_ops_480 = {
+ .queue_buffer = vfe_queue_buffer_v2,
+ .flush_buffers = vfe_flush_buffers,
+};
-out_unlock:
- spin_unlock_irqrestore(&vfe->output_lock, flags);
+static void vfe_subdev_init(struct device *dev, struct vfe_device *vfe)
+{
+ vfe->video_ops = vfe_video_ops_480;
}
-/*
- * vfe_queue_buffer - Add empty buffer
- * @vid: Video device structure
- * @buf: Buffer to be enqueued
- *
- * Add an empty buffer - depending on the current number of buffers it will be
- * put in pending buffer queue or directly given to the hardware to be filled.
- *
- * Return 0 on success or a negative error code otherwise
- */
-static int vfe_queue_buffer(struct camss_video *vid,
- struct camss_buffer *buf)
+static void vfe_isr_read(struct vfe_device *vfe, u32 *value0, u32 *value1)
{
- struct vfe_line *line = container_of(vid, struct vfe_line, video_out);
- struct vfe_device *vfe = to_vfe(line);
- struct vfe_output *output;
- unsigned long flags;
-
- output = &line->output;
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- if (output->state == VFE_OUTPUT_ON && output->gen2.active_num < 2) {
- output->buf[output->gen2.active_num++] = buf;
- vfe_wm_update(vfe, output->wm_idx[0], buf->addr[0], line);
- } else {
- vfe_buf_add_pending(output, buf);
- }
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return 0;
+ /* nop */
}
-static const struct camss_video_ops vfe_video_ops_480 = {
- .queue_buffer = vfe_queue_buffer,
- .flush_buffers = vfe_flush_buffers,
-};
+static void vfe_violation_read(struct vfe_device *vfe)
+{
+ /* nop */
+}
-static void vfe_subdev_init(struct device *dev, struct vfe_device *vfe)
+static void vfe_buf_done_480(struct vfe_device *vfe, int port_id)
{
- vfe->video_ops = vfe_video_ops_480;
+ /* nop */
}
const struct vfe_hw_ops vfe_ops_480 = {
+ .enable_irq = vfe_enable_irq,
.global_reset = vfe_global_reset,
.hw_version = vfe_hw_version,
.isr = vfe_isr,
+ .isr_read = vfe_isr_read,
+ .reg_update = vfe_reg_update,
+ .reg_update_clear = vfe_reg_update_clear,
.pm_domain_off = vfe_pm_domain_off,
.pm_domain_on = vfe_pm_domain_on,
.subdev_init = vfe_subdev_init,
.vfe_disable = vfe_disable,
- .vfe_enable = vfe_enable,
+ .vfe_enable = vfe_enable_v2,
.vfe_halt = vfe_halt,
+ .violation_read = vfe_violation_read,
+ .vfe_wm_start = vfe_wm_start,
.vfe_wm_stop = vfe_wm_stop,
+ .vfe_buf_done = vfe_buf_done_480,
+ .vfe_wm_update = vfe_wm_update,
};
diff --git a/drivers/media/platform/qcom/camss/camss-vfe.c b/drivers/media/platform/qcom/camss/camss-vfe.c
index 0eeba59dd933..e86e648d8a0d 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe.c
@@ -32,6 +32,11 @@
#define SCALER_RATIO_MAX 16
+#define VFE_HW_VERSION 0x0
+#define HW_VERSION_STEPPING 0
+#define HW_VERSION_REVISION 16
+#define HW_VERSION_GENERATION 28
+
static const struct camss_format_info formats_rdi_8x16[] = {
{ MEDIA_BUS_FMT_UYVY8_1X16, 8, V4L2_PIX_FMT_UYVY, 1,
PER_PLANE_DATA(0, 1, 1, 1, 1, 16) },
@@ -406,6 +411,269 @@ static u32 vfe_src_pad_code(struct vfe_line *line, u32 sink_code,
return 0;
}
+/*
+ * vfe_hw_version - Process write master done interrupt
+ * @vfe: VFE Device
+ *
+ * Return vfe hw version
+ */
+u32 vfe_hw_version(struct vfe_device *vfe)
+{
+ u32 hw_version = readl_relaxed(vfe->base + VFE_HW_VERSION);
+
+ u32 gen = (hw_version >> HW_VERSION_GENERATION) & 0xF;
+ u32 rev = (hw_version >> HW_VERSION_REVISION) & 0xFFF;
+ u32 step = (hw_version >> HW_VERSION_STEPPING) & 0xFFFF;
+
+ dev_dbg(vfe->camss->dev, "VFE:%d HW Version = %u.%u.%u\n",
+ vfe->id, gen, rev, step);
+
+ return hw_version;
+}
+
+/*
+ * vfe_buf_done - Process write master done interrupt
+ * @vfe: VFE Device
+ * @wm: Write master id
+ */
+void vfe_buf_done(struct vfe_device *vfe, int wm)
+{
+ struct vfe_line *line = &vfe->line[vfe->wm_output_map[wm]];
+ const struct vfe_hw_ops *ops = vfe->res->hw_ops;
+ struct camss_buffer *ready_buf;
+ struct vfe_output *output;
+ unsigned long flags;
+ u32 index;
+ u64 ts = ktime_get_ns();
+
+ spin_lock_irqsave(&vfe->output_lock, flags);
+
+ if (vfe->wm_output_map[wm] == VFE_LINE_NONE) {
+ dev_err_ratelimited(vfe->camss->dev,
+ "Received wm done for unmapped index\n");
+ goto out_unlock;
+ }
+ output = &vfe->line[vfe->wm_output_map[wm]].output;
+
+ ready_buf = output->buf[0];
+ if (!ready_buf) {
+ dev_err_ratelimited(vfe->camss->dev,
+ "Missing ready buf %d!\n", output->state);
+ goto out_unlock;
+ }
+
+ ready_buf->vb.vb2_buf.timestamp = ts;
+ ready_buf->vb.sequence = output->sequence++;
+
+ index = 0;
+ output->buf[0] = output->buf[1];
+ if (output->buf[0])
+ index = 1;
+
+ output->buf[index] = vfe_buf_get_pending(output);
+
+ if (output->buf[index]) {
+ ops->vfe_wm_update(vfe, output->wm_idx[0],
+ output->buf[index]->addr[0],
+ line);
+ ops->reg_update(vfe, line->id);
+ } else {
+ output->gen2.active_num--;
+ }
+
+ spin_unlock_irqrestore(&vfe->output_lock, flags);
+
+ vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+
+ return;
+
+out_unlock:
+ spin_unlock_irqrestore(&vfe->output_lock, flags);
+}
+
+int vfe_enable_output_v2(struct vfe_line *line)
+{
+ struct vfe_device *vfe = to_vfe(line);
+ struct vfe_output *output = &line->output;
+ const struct vfe_hw_ops *ops = vfe->res->hw_ops;
+ struct media_entity *sensor;
+ unsigned long flags;
+ unsigned int frame_skip = 0;
+ unsigned int i;
+
+ sensor = camss_find_sensor(&line->subdev.entity);
+ if (sensor) {
+ struct v4l2_subdev *subdev = media_entity_to_v4l2_subdev(sensor);
+
+ v4l2_subdev_call(subdev, sensor, g_skip_frames, &frame_skip);
+ /* Max frame skip is 29 frames */
+ if (frame_skip > VFE_FRAME_DROP_VAL - 1)
+ frame_skip = VFE_FRAME_DROP_VAL - 1;
+ }
+
+ spin_lock_irqsave(&vfe->output_lock, flags);
+
+ ops->reg_update_clear(vfe, line->id);
+
+ if (output->state > VFE_OUTPUT_RESERVED) {
+ dev_err(vfe->camss->dev,
+ "Output is not in reserved state %d\n",
+ output->state);
+ spin_unlock_irqrestore(&vfe->output_lock, flags);
+ return -EINVAL;
+ }
+
+ WARN_ON(output->gen2.active_num);
+
+ output->state = VFE_OUTPUT_ON;
+
+ output->sequence = 0;
+ output->wait_reg_update = 0;
+ reinit_completion(&output->reg_update);
+
+ ops->vfe_wm_start(vfe, output->wm_idx[0], line);
+
+ for (i = 0; i < 2; i++) {
+ output->buf[i] = vfe_buf_get_pending(output);
+ if (!output->buf[i])
+ break;
+ output->gen2.active_num++;
+ ops->vfe_wm_update(vfe, output->wm_idx[0],
+ output->buf[i]->addr[0], line);
+ ops->reg_update(vfe, line->id);
+ }
+
+ spin_unlock_irqrestore(&vfe->output_lock, flags);
+
+ return 0;
+}
+
+/*
+ * vfe_queue_buffer_v2 - Add empty buffer
+ * @vid: Video device structure
+ * @buf: Buffer to be enqueued
+ *
+ * Add an empty buffer - depending on the current number of buffers it will be
+ * put in pending buffer queue or directly given to the hardware to be filled.
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+int vfe_queue_buffer_v2(struct camss_video *vid,
+ struct camss_buffer *buf)
+{
+ struct vfe_line *line = container_of(vid, struct vfe_line, video_out);
+ struct vfe_device *vfe = to_vfe(line);
+ const struct vfe_hw_ops *ops = vfe->res->hw_ops;
+ struct vfe_output *output;
+ unsigned long flags;
+
+ output = &line->output;
+
+ spin_lock_irqsave(&vfe->output_lock, flags);
+
+ if (output->state == VFE_OUTPUT_ON &&
+ output->gen2.active_num < 2) {
+ output->buf[output->gen2.active_num++] = buf;
+ ops->vfe_wm_update(vfe, output->wm_idx[0],
+ buf->addr[0], line);
+ ops->reg_update(vfe, line->id);
+ } else {
+ vfe_buf_add_pending(output, buf);
+ }
+
+ spin_unlock_irqrestore(&vfe->output_lock, flags);
+
+ return 0;
+}
+
+/*
+ * vfe_enable_v2 - Enable streaming on VFE line
+ * @line: VFE line
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+int vfe_enable_v2(struct vfe_line *line)
+{
+ struct vfe_device *vfe = to_vfe(line);
+ const struct vfe_hw_ops *ops = vfe->res->hw_ops;
+ int ret;
+
+ mutex_lock(&vfe->stream_lock);
+
+ if (vfe->res->hw_ops->enable_irq)
+ ops->enable_irq(vfe);
+
+ vfe->stream_count++;
+
+ mutex_unlock(&vfe->stream_lock);
+
+ ret = vfe_get_output_v2(line);
+ if (ret < 0)
+ goto error_get_output;
+
+ ret = vfe_enable_output_v2(line);
+ if (ret < 0)
+ goto error_enable_output;
+
+ vfe->was_streaming = 1;
+
+ return 0;
+
+error_enable_output:
+ vfe_put_output(line);
+
+error_get_output:
+ mutex_lock(&vfe->stream_lock);
+
+ vfe->stream_count--;
+
+ mutex_unlock(&vfe->stream_lock);
+
+ return ret;
+}
+
+/*
+ * vfe_get_output_v2 - Get vfe output port for corresponding VFE line
+ * @line: VFE line
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+int vfe_get_output_v2(struct vfe_line *line)
+{
+ struct vfe_device *vfe = to_vfe(line);
+ struct vfe_output *output;
+ unsigned long flags;
+
+ spin_lock_irqsave(&vfe->output_lock, flags);
+
+ output = &line->output;
+ if (output->state > VFE_OUTPUT_RESERVED) {
+ dev_err(vfe->camss->dev, "Output is running\n");
+ goto error;
+ }
+
+ output->wm_num = 1;
+
+ /* Correspondence between VFE line number and WM number.
+ * line 0 -> RDI 0, line 1 -> RDI1, line 2 -> RDI2, line 3 -> PIX/RDI3
+ * Note this 1:1 mapping will not work for PIX streams.
+ */
+ output->wm_idx[0] = line->id;
+ vfe->wm_output_map[line->id] = line->id;
+
+ output->drop_update_idx = 0;
+
+ spin_unlock_irqrestore(&vfe->output_lock, flags);
+
+ return 0;
+
+error:
+ spin_unlock_irqrestore(&vfe->output_lock, flags);
+ output->state = VFE_OUTPUT_OFF;
+
+ return -EINVAL;
+}
+
int vfe_reset(struct vfe_device *vfe)
{
unsigned long time;
diff --git a/drivers/media/platform/qcom/camss/camss-vfe.h b/drivers/media/platform/qcom/camss/camss-vfe.h
index 10e2cc3c0b83..fcbf4f609129 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe.h
+++ b/drivers/media/platform/qcom/camss/camss-vfe.h
@@ -99,7 +99,7 @@ struct vfe_line {
struct vfe_device;
struct vfe_hw_ops {
- void (*enable_irq_common)(struct vfe_device *vfe);
+ void (*enable_irq)(struct vfe_device *vfe);
void (*global_reset)(struct vfe_device *vfe);
u32 (*hw_version)(struct vfe_device *vfe);
irqreturn_t (*isr)(int irq, void *dev);
@@ -114,7 +114,12 @@ struct vfe_hw_ops {
int (*vfe_enable)(struct vfe_line *line);
int (*vfe_halt)(struct vfe_device *vfe);
void (*violation_read)(struct vfe_device *vfe);
+ void (*vfe_wm_start)(struct vfe_device *vfe, u8 wm,
+ struct vfe_line *line);
void (*vfe_wm_stop)(struct vfe_device *vfe, u8 wm);
+ void (*vfe_buf_done)(struct vfe_device *vfe, int port_id);
+ void (*vfe_wm_update)(struct vfe_device *vfe, u8 wm, u32 addr,
+ struct vfe_line *line);
};
struct vfe_isr_ops {
@@ -252,4 +257,55 @@ void vfe_put(struct vfe_device *vfe);
*/
bool vfe_is_lite(struct vfe_device *vfe);
+/*
+ * vfe_hw_version - Process write master done interrupt
+ * @vfe: VFE Device
+ *
+ * Return vfe hw version
+ */
+u32 vfe_hw_version(struct vfe_device *vfe);
+/*
+ * vfe_enable - Enable streaming on VFE line
+ * @line: VFE line
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+int vfe_enable_v2(struct vfe_line *line);
+
+/*
+ * vfe_buf_done - Process write master done interrupt
+ * @vfe: VFE Device
+ * @wm: Write master id
+ */
+void vfe_buf_done(struct vfe_device *vfe, int wm);
+
+/*
+ * vfe_get_output_v2 - Get vfe output line
+ * line: VFE line
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+int vfe_get_output_v2(struct vfe_line *line);
+
+/*
+ * vfe_enable_output_v2 - Enable vfe output line
+ * line: VFE line
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+int vfe_enable_output_v2(struct vfe_line *line);
+
+/*
+ * vfe_queue_buffer_v2 - Add empty buffer
+ * @vid: Video device structure
+ * @buf: Buffer to be enqueued
+ *
+ * Add an empty buffer - depending on the current number of buffers it will be
+ * put in pending buffer queue or directly given to the hardware to be filled.
+ *
+ * Return 0 on success or a negative error code otherwise
+ */
+int vfe_queue_buffer_v2(struct camss_video *vid,
+ struct camss_buffer *buf);
+
#endif /* QC_MSM_CAMSS_VFE_H */
diff --git a/drivers/media/platform/rockchip/rga/rga.c b/drivers/media/platform/rockchip/rga/rga.c
index de532b7ecd74..dd512785649a 100644
--- a/drivers/media/platform/rockchip/rga/rga.c
+++ b/drivers/media/platform/rockchip/rga/rga.c
@@ -704,48 +704,10 @@ static const struct video_device rga_videodev = {
.device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING,
};
-static int rga_enable_clocks(struct rockchip_rga *rga)
-{
- int ret;
-
- ret = clk_prepare_enable(rga->sclk);
- if (ret) {
- dev_err(rga->dev, "Cannot enable rga sclk: %d\n", ret);
- return ret;
- }
-
- ret = clk_prepare_enable(rga->aclk);
- if (ret) {
- dev_err(rga->dev, "Cannot enable rga aclk: %d\n", ret);
- goto err_disable_sclk;
- }
-
- ret = clk_prepare_enable(rga->hclk);
- if (ret) {
- dev_err(rga->dev, "Cannot enable rga hclk: %d\n", ret);
- goto err_disable_aclk;
- }
-
- return 0;
-
-err_disable_aclk:
- clk_disable_unprepare(rga->aclk);
-err_disable_sclk:
- clk_disable_unprepare(rga->sclk);
-
- return ret;
-}
-
-static void rga_disable_clocks(struct rockchip_rga *rga)
-{
- clk_disable_unprepare(rga->sclk);
- clk_disable_unprepare(rga->hclk);
- clk_disable_unprepare(rga->aclk);
-}
-
static int rga_parse_dt(struct rockchip_rga *rga)
{
struct reset_control *core_rst, *axi_rst, *ahb_rst;
+ int ret;
core_rst = devm_reset_control_get(rga->dev, "core");
if (IS_ERR(core_rst)) {
@@ -777,23 +739,12 @@ static int rga_parse_dt(struct rockchip_rga *rga)
udelay(1);
reset_control_deassert(ahb_rst);
- rga->sclk = devm_clk_get(rga->dev, "sclk");
- if (IS_ERR(rga->sclk)) {
- dev_err(rga->dev, "failed to get sclk clock\n");
- return PTR_ERR(rga->sclk);
- }
-
- rga->aclk = devm_clk_get(rga->dev, "aclk");
- if (IS_ERR(rga->aclk)) {
- dev_err(rga->dev, "failed to get aclk clock\n");
- return PTR_ERR(rga->aclk);
- }
-
- rga->hclk = devm_clk_get(rga->dev, "hclk");
- if (IS_ERR(rga->hclk)) {
- dev_err(rga->dev, "failed to get hclk clock\n");
- return PTR_ERR(rga->hclk);
+ ret = devm_clk_bulk_get_all(rga->dev, &rga->clks);
+ if (ret < 0) {
+ dev_err(rga->dev, "failed to get clocks\n");
+ return ret;
}
+ rga->num_clks = ret;
return 0;
}
@@ -833,6 +784,7 @@ static int rga_probe(struct platform_device *pdev)
ret = irq;
goto err_put_clk;
}
+ rga->irq = irq;
ret = devm_request_irq(rga->dev, irq, rga_isr, 0,
dev_name(rga->dev), rga);
@@ -930,8 +882,9 @@ static void rga_remove(struct platform_device *pdev)
v4l2_info(&rga->v4l2_dev, "Removing\n");
- v4l2_m2m_release(rga->m2m_dev);
video_unregister_device(rga->vfd);
+ devm_free_irq(rga->dev, rga->irq, rga);
+ v4l2_m2m_release(rga->m2m_dev);
v4l2_device_unregister(&rga->v4l2_dev);
pm_runtime_disable(rga->dev);
@@ -941,7 +894,7 @@ static int __maybe_unused rga_runtime_suspend(struct device *dev)
{
struct rockchip_rga *rga = dev_get_drvdata(dev);
- rga_disable_clocks(rga);
+ clk_bulk_disable_unprepare(rga->num_clks, rga->clks);
return 0;
}
@@ -950,7 +903,7 @@ static int __maybe_unused rga_runtime_resume(struct device *dev)
{
struct rockchip_rga *rga = dev_get_drvdata(dev);
- return rga_enable_clocks(rga);
+ return clk_bulk_prepare_enable(rga->num_clks, rga->clks);
}
static const struct dev_pm_ops rga_pm = {
diff --git a/drivers/media/platform/rockchip/rga/rga.h b/drivers/media/platform/rockchip/rga/rga.h
index 8105bb2efe57..8f0185d59210 100644
--- a/drivers/media/platform/rockchip/rga/rga.h
+++ b/drivers/media/platform/rockchip/rga/rga.h
@@ -6,6 +6,7 @@
#ifndef __RGA_H__
#define __RGA_H__
+#include <linux/clk.h>
#include <linux/platform_device.h>
#include <media/videobuf2-v4l2.h>
#include <media/v4l2-ctrls.h>
@@ -76,9 +77,9 @@ struct rockchip_rga {
struct device *dev;
struct regmap *grf;
void __iomem *regs;
- struct clk *sclk;
- struct clk *aclk;
- struct clk *hclk;
+ int irq;
+ struct clk_bulk_data *clks;
+ int num_clks;
struct rockchip_rga_version version;
/* vfd lock */
diff --git a/drivers/media/rc/ene_ir.c b/drivers/media/rc/ene_ir.c
index 3fd51a41c3b2..70e28bd4bdb9 100644
--- a/drivers/media/rc/ene_ir.c
+++ b/drivers/media/rc/ene_ir.c
@@ -1103,15 +1103,15 @@ static void ene_remove(struct pnp_dev *pnp_dev)
unsigned long flags;
rc_unregister_device(dev->rdev);
- del_timer_sync(&dev->tx_sim_timer);
spin_lock_irqsave(&dev->hw_lock, flags);
ene_rx_disable(dev);
ene_rx_restore_hw_buffer(dev);
spin_unlock_irqrestore(&dev->hw_lock, flags);
- rc_free_device(dev->rdev);
free_irq(dev->irq, dev);
+ timer_delete_sync(&dev->tx_sim_timer);
release_region(dev->hw_io, ENE_IO_SIZE);
+ rc_free_device(dev->rdev);
kfree(dev);
}
diff --git a/drivers/media/rc/redrat3.c b/drivers/media/rc/redrat3.c
index c9a5c030f772..080207aa888c 100644
--- a/drivers/media/rc/redrat3.c
+++ b/drivers/media/rc/redrat3.c
@@ -773,17 +773,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf,
u8 curlencheck = 0;
unsigned i, sendbuf_len;
- if (rr3->transmitting) {
- dev_warn(dev, "%s: transmitter already in use\n", __func__);
- return -EAGAIN;
- }
-
if (count > RR3_MAX_SIG_SIZE - RR3_TX_TRAILER_LEN)
return -EINVAL;
- /* rr3 will disable rc detector on transmit */
- rr3->transmitting = true;
-
sample_lens = kcalloc(RR3_DRIVER_MAXLENS,
sizeof(*sample_lens),
GFP_KERNEL);
@@ -796,6 +788,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf,
goto out;
}
+ /* rr3 will disable rc detector on transmit */
+ rr3->transmitting = true;
+
for (i = 0; i < count; i++) {
cur_sample_len = redrat3_us_to_len(txbuf[i]);
if (cur_sample_len > 0xffff) {
diff --git a/drivers/media/rc/serial_ir.c b/drivers/media/rc/serial_ir.c
index fc5fd3927177..f6702b3df9a0 100644
--- a/drivers/media/rc/serial_ir.c
+++ b/drivers/media/rc/serial_ir.c
@@ -798,7 +798,7 @@ static int __init serial_ir_init_module(void)
static void __exit serial_ir_exit_module(void)
{
- del_timer_sync(&serial_ir.timeout_timer);
+ timer_shutdown_sync(&serial_ir.timeout_timer);
serial_ir_exit();
}
diff --git a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
index 0fb3fa6100e4..675d9d1d4f47 100644
--- a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
+++ b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
@@ -141,6 +141,8 @@ int dvb_usb_get_hexline(const struct firmware *fw, struct hexline *hx,
if (hx->type == 0x04) {
/* b[4] and b[5] are the Extended linear address record data field */
+ if (hx->len != 2)
+ return -EINVAL;
hx->addr |= (b[4] << 24) | (b[5] << 16);
/* hx->len -= 2;
data_offs += 2; */
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index d3fa27bf6551..44ce06268065 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -2828,6 +2828,26 @@ static const struct usb_device_id uvc_ids[] = {
.bInterfaceSubClass = 1,
.bInterfaceProtocol = 0,
.driver_info = (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
+ /* Avermedia GC515 */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1515,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = UVC_PC_PROTOCOL_15,
+ .driver_info = (kernel_ulong_t)&(const struct uvc_device_info){
+ .uvc_version = 0x0110,
+ } },
+ /* AVerMedia Live Gamer HD 2 capture card */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1570,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = 0,
+ .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_DISABLE_AUTOSUSPEND) },
/* ViMicro Vega */
{ .match_flags = USB_DEVICE_ID_MATCH_DEVICE
| USB_DEVICE_ID_MATCH_INT_INFO,
diff --git a/drivers/mfd/ab8500-core.c b/drivers/mfd/ab8500-core.c
index 17c90850946b..53d2b117058c 100644
--- a/drivers/mfd/ab8500-core.c
+++ b/drivers/mfd/ab8500-core.c
@@ -1195,39 +1195,41 @@ static int ab8500_probe(struct platform_device *pdev)
return ret;
if (is_ab9540(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab9540_devs,
- ARRAY_SIZE(ab9540_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab9540_devs,
+ ARRAY_SIZE(ab9540_devs), NULL,
+ 0, ab8500->domain);
else if (is_ab8540(ab8500)) {
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_devs,
- ARRAY_SIZE(ab8540_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8540_devs,
+ ARRAY_SIZE(ab8540_devs), NULL,
+ 0, ab8500->domain);
if (ret)
return ret;
if (is_ab8540_1p2_or_earlier(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut1_devs,
- ARRAY_SIZE(ab8540_cut1_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0,
+ ab8540_cut1_devs,
+ ARRAY_SIZE(ab8540_cut1_devs),
+ NULL, 0, ab8500->domain);
else /* ab8540 >= cut2 */
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut2_devs,
- ARRAY_SIZE(ab8540_cut2_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0,
+ ab8540_cut2_devs,
+ ARRAY_SIZE(ab8540_cut2_devs),
+ NULL, 0, ab8500->domain);
} else if (is_ab8505(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab8505_devs,
- ARRAY_SIZE(ab8505_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8505_devs,
+ ARRAY_SIZE(ab8505_devs), NULL,
+ 0, ab8500->domain);
else
- ret = mfd_add_devices(ab8500->dev, 0, ab8500_devs,
- ARRAY_SIZE(ab8500_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_devs,
+ ARRAY_SIZE(ab8500_devs), NULL,
+ 0, ab8500->domain);
if (ret)
return ret;
/* Add battery management devices */
- ret = mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs,
- ARRAY_SIZE(ab8500_bm_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs,
+ ARRAY_SIZE(ab8500_bm_devs), NULL,
+ 0, ab8500->domain);
if (ret)
dev_err(ab8500->dev, "error adding bm devices\n");
diff --git a/drivers/mfd/twl-core.c b/drivers/mfd/twl-core.c
index c130ffef182f..8334066f8e22 100644
--- a/drivers/mfd/twl-core.c
+++ b/drivers/mfd/twl-core.c
@@ -715,6 +715,11 @@ static const struct mfd_cell twl6032_cells[] = {
{ .name = "twl6032-clk" },
};
+static void twl_unregister_device(void *data)
+{
+ platform_device_unregister(data);
+}
+
/* NOTE: This driver only handles a single twl4030/tps659x0 chip */
static int
twl_probe(struct i2c_client *client)
@@ -750,17 +755,22 @@ twl_probe(struct i2c_client *client)
return status;
}
- if (i2c_check_functionality(client->adapter, I2C_FUNC_I2C) == 0) {
+ status = devm_add_action_or_reset(&client->dev, twl_unregister_device,
+ pdev);
+ if (status)
+ return status;
+
+ if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C)) {
dev_dbg(&client->dev, "can't talk I2C?\n");
status = -EIO;
- goto free;
+ goto out;
}
twl_priv = devm_kzalloc(&client->dev, sizeof(struct twl_private),
GFP_KERNEL);
if (!twl_priv) {
status = -ENOMEM;
- goto free;
+ goto out;
}
if ((id->driver_data) & TWL6030_CLASS) {
@@ -780,7 +790,7 @@ twl_probe(struct i2c_client *client)
GFP_KERNEL);
if (!twl_priv->twl_modules) {
status = -ENOMEM;
- goto free;
+ goto out;
}
for (i = 0; i < num_slaves; i++) {
@@ -868,7 +878,7 @@ twl_probe(struct i2c_client *client)
ARRAY_SIZE(twl6032_cells),
NULL, 0, NULL);
if (status < 0)
- goto free;
+ goto out;
}
if (twl_class_is_6030()) {
@@ -886,10 +896,7 @@ twl_probe(struct i2c_client *client)
fail:
if (status < 0)
twl_remove(client);
-free:
- if (status < 0)
- platform_device_unregister(pdev);
-
+out:
return status;
}
diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c
index 576cd9e5b807..a4577ef5a394 100644
--- a/drivers/misc/fastrpc.c
+++ b/drivers/misc/fastrpc.c
@@ -109,7 +109,7 @@
static const char *domains[FASTRPC_DEV_MAX] = { "adsp", "mdsp",
"sdsp", "cdsp", "cdsp1" };
struct fastrpc_phy_page {
- u64 addr; /* physical address */
+ dma_addr_t addr; /* dma address */
u64 size; /* size of contiguous region */
};
@@ -174,7 +174,7 @@ struct fastrpc_msg {
u64 ctx; /* invoke caller context */
u32 handle; /* handle to invoke */
u32 sc; /* scalars structure describing the data */
- u64 addr; /* physical address */
+ dma_addr_t addr; /* dma address */
u64 size; /* size of contiguous region */
};
@@ -197,7 +197,7 @@ struct fastrpc_buf {
struct dma_buf *dmabuf;
struct device *dev;
void *virt;
- u64 phys;
+ dma_addr_t dma_addr;
u64 size;
/* Lock for dma buf attachments */
struct mutex lock;
@@ -220,7 +220,7 @@ struct fastrpc_map {
struct dma_buf *buf;
struct sg_table *table;
struct dma_buf_attachment *attach;
- u64 phys;
+ dma_addr_t dma_addr;
u64 size;
void *va;
u64 len;
@@ -325,11 +325,12 @@ static void fastrpc_free_map(struct kref *ref)
perm.vmid = QCOM_SCM_VMID_HLOS;
perm.perm = QCOM_SCM_PERM_RWX;
- err = qcom_scm_assign_mem(map->phys, map->len,
+ err = qcom_scm_assign_mem(map->dma_addr, map->len,
&src_perms, &perm, 1);
if (err) {
- dev_err(map->fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d\n",
- map->phys, map->len, err);
+ dev_err(map->fl->sctx->dev,
+ "Failed to assign memory dma_addr %pad size 0x%llx err %d\n",
+ &map->dma_addr, map->len, err);
return;
}
}
@@ -400,7 +401,7 @@ static int fastrpc_map_lookup(struct fastrpc_user *fl, int fd,
static void fastrpc_buf_free(struct fastrpc_buf *buf)
{
dma_free_coherent(buf->dev, buf->size, buf->virt,
- FASTRPC_PHYS(buf->phys));
+ FASTRPC_PHYS(buf->dma_addr));
kfree(buf);
}
@@ -419,12 +420,12 @@ static int __fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev,
buf->fl = fl;
buf->virt = NULL;
- buf->phys = 0;
+ buf->dma_addr = 0;
buf->size = size;
buf->dev = dev;
buf->raddr = 0;
- buf->virt = dma_alloc_coherent(dev, buf->size, (dma_addr_t *)&buf->phys,
+ buf->virt = dma_alloc_coherent(dev, buf->size, &buf->dma_addr,
GFP_KERNEL);
if (!buf->virt) {
mutex_destroy(&buf->lock);
@@ -450,7 +451,7 @@ static int fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev,
buf = *obuf;
if (fl->sctx && fl->sctx->sid)
- buf->phys += ((u64)fl->sctx->sid << 32);
+ buf->dma_addr += ((u64)fl->sctx->sid << 32);
return 0;
}
@@ -743,7 +744,7 @@ static int fastrpc_dma_buf_attach(struct dma_buf *dmabuf,
return -ENOMEM;
ret = dma_get_sgtable(buffer->dev, &a->sgt, buffer->virt,
- FASTRPC_PHYS(buffer->phys), buffer->size);
+ FASTRPC_PHYS(buffer->dma_addr), buffer->size);
if (ret < 0) {
dev_err(buffer->dev, "failed to get scatterlist from DMA API\n");
kfree(a);
@@ -792,7 +793,7 @@ static int fastrpc_mmap(struct dma_buf *dmabuf,
dma_resv_assert_held(dmabuf->resv);
return dma_mmap_coherent(buf->dev, vma, buf->virt,
- FASTRPC_PHYS(buf->phys), size);
+ FASTRPC_PHYS(buf->dma_addr), size);
}
static const struct dma_buf_ops fastrpc_dma_buf_ops = {
@@ -844,10 +845,10 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
map->table = table;
if (attr & FASTRPC_ATTR_SECUREMAP) {
- map->phys = sg_phys(map->table->sgl);
+ map->dma_addr = sg_phys(map->table->sgl);
} else {
- map->phys = sg_dma_address(map->table->sgl);
- map->phys += ((u64)fl->sctx->sid << 32);
+ map->dma_addr = sg_dma_address(map->table->sgl);
+ map->dma_addr += ((u64)fl->sctx->sid << 32);
}
for_each_sg(map->table->sgl, sgl, map->table->nents,
sgl_index)
@@ -856,7 +857,7 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
dev_dbg(sess->dev, "Bad size passed len 0x%llx map size 0x%llx\n",
len, map->size);
err = -EINVAL;
- goto map_err;
+ goto get_err;
}
map->va = sg_virt(map->table->sgl);
map->len = len;
@@ -874,11 +875,12 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
dst_perms[1].vmid = fl->cctx->vmperms[0].vmid;
dst_perms[1].perm = QCOM_SCM_PERM_RWX;
map->attr = attr;
- err = qcom_scm_assign_mem(map->phys, (u64)map->len, &src_perms, dst_perms, 2);
+ err = qcom_scm_assign_mem(map->dma_addr, (u64)map->len, &src_perms, dst_perms, 2);
if (err) {
- dev_err(sess->dev, "Failed to assign memory with phys 0x%llx size 0x%llx err %d\n",
- map->phys, map->len, err);
- goto map_err;
+ dev_err(sess->dev,
+ "Failed to assign memory with dma_addr %pad size 0x%llx err %d\n",
+ &map->dma_addr, map->len, err);
+ goto get_err;
}
}
spin_lock(&fl->lock);
@@ -1059,7 +1061,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx)
struct vm_area_struct *vma = NULL;
rpra[i].buf.pv = (u64) ctx->args[i].ptr;
- pages[i].addr = ctx->maps[i]->phys;
+ pages[i].addr = ctx->maps[i]->dma_addr;
mmap_read_lock(current->mm);
vma = vma_lookup(current->mm, ctx->args[i].ptr);
@@ -1086,7 +1088,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx)
goto bail;
rpra[i].buf.pv = args - ctx->olaps[oix].offset;
- pages[i].addr = ctx->buf->phys -
+ pages[i].addr = ctx->buf->dma_addr -
ctx->olaps[oix].offset +
(pkt_size - rlen);
pages[i].addr = pages[i].addr & PAGE_MASK;
@@ -1118,7 +1120,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx)
list[i].num = ctx->args[i].length ? 1 : 0;
list[i].pgidx = i;
if (ctx->maps[i]) {
- pages[i].addr = ctx->maps[i]->phys;
+ pages[i].addr = ctx->maps[i]->dma_addr;
pages[i].size = ctx->maps[i]->size;
}
rpra[i].dma.fd = ctx->args[i].fd;
@@ -1200,7 +1202,7 @@ static int fastrpc_invoke_send(struct fastrpc_session_ctx *sctx,
msg->ctx = ctx->ctxid | fl->pd;
msg->handle = handle;
msg->sc = ctx->sc;
- msg->addr = ctx->buf ? ctx->buf->phys : 0;
+ msg->addr = ctx->buf ? ctx->buf->dma_addr : 0;
msg->size = roundup(ctx->msg_sz, PAGE_SIZE);
fastrpc_context_get(ctx);
@@ -1358,13 +1360,15 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
if (fl->cctx->vmcount) {
u64 src_perms = BIT(QCOM_SCM_VMID_HLOS);
- err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys,
+ err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr,
(u64)fl->cctx->remote_heap->size,
&src_perms,
fl->cctx->vmperms, fl->cctx->vmcount);
if (err) {
- dev_err(fl->sctx->dev, "Failed to assign memory with phys 0x%llx size 0x%llx err %d\n",
- fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err);
+ dev_err(fl->sctx->dev,
+ "Failed to assign memory with dma_addr %pad size 0x%llx err %d\n",
+ &fl->cctx->remote_heap->dma_addr,
+ fl->cctx->remote_heap->size, err);
goto err_map;
}
scm_done = true;
@@ -1384,7 +1388,7 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
args[1].length = inbuf.namelen;
args[1].fd = -1;
- pages[0].addr = fl->cctx->remote_heap->phys;
+ pages[0].addr = fl->cctx->remote_heap->dma_addr;
pages[0].size = fl->cctx->remote_heap->size;
args[2].ptr = (u64)(uintptr_t) pages;
@@ -1413,12 +1417,12 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
dst_perms.vmid = QCOM_SCM_VMID_HLOS;
dst_perms.perm = QCOM_SCM_PERM_RWX;
- err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys,
+ err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr,
(u64)fl->cctx->remote_heap->size,
&src_perms, &dst_perms, 1);
if (err)
- dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d\n",
- fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err);
+ dev_err(fl->sctx->dev, "Failed to assign memory dma_addr %pad size 0x%llx err %d\n",
+ &fl->cctx->remote_heap->dma_addr, fl->cctx->remote_heap->size, err);
}
err_map:
fastrpc_buf_free(fl->cctx->remote_heap);
@@ -1508,7 +1512,7 @@ static int fastrpc_init_create_process(struct fastrpc_user *fl,
args[2].length = inbuf.filelen;
args[2].fd = init.filefd;
- pages[0].addr = imem->phys;
+ pages[0].addr = imem->dma_addr;
pages[0].size = imem->size;
args[3].ptr = (u64)(uintptr_t) pages;
@@ -1962,7 +1966,7 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
args[0].ptr = (u64) (uintptr_t) &req_msg;
args[0].length = sizeof(req_msg);
- pages.addr = buf->phys;
+ pages.addr = buf->dma_addr;
pages.size = buf->size;
args[1].ptr = (u64) (uintptr_t) &pages;
@@ -1990,11 +1994,12 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
if (req.flags == ADSP_MMAP_REMOTE_HEAP_ADDR && fl->cctx->vmcount) {
u64 src_perms = BIT(QCOM_SCM_VMID_HLOS);
- err = qcom_scm_assign_mem(buf->phys, (u64)buf->size,
+ err = qcom_scm_assign_mem(buf->dma_addr, (u64)buf->size,
&src_perms, fl->cctx->vmperms, fl->cctx->vmcount);
if (err) {
- dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d",
- buf->phys, buf->size, err);
+ dev_err(fl->sctx->dev,
+ "Failed to assign memory dma_addr %pad size 0x%llx err %d",
+ &buf->dma_addr, buf->size, err);
goto err_assign;
}
}
@@ -2108,7 +2113,7 @@ static int fastrpc_req_mem_map(struct fastrpc_user *fl, char __user *argp)
args[0].ptr = (u64) (uintptr_t) &req_msg;
args[0].length = sizeof(req_msg);
- pages.addr = map->phys;
+ pages.addr = map->dma_addr;
pages.size = map->len;
args[1].ptr = (u64) (uintptr_t) &pages;
diff --git a/drivers/mtd/nand/raw/Kconfig b/drivers/mtd/nand/raw/Kconfig
index d0aaccf72d78..91fae26a3d91 100644
--- a/drivers/mtd/nand/raw/Kconfig
+++ b/drivers/mtd/nand/raw/Kconfig
@@ -71,7 +71,8 @@ config MTD_NAND_AU1550
config MTD_NAND_NDFC
tristate "IBM/MCC 4xx NAND controller"
- depends on 4xx
+ depends on 44x || COMPILE_TEST
+ depends on OF
select MTD_NAND_ECC_SW_HAMMING
select MTD_NAND_ECC_SW_HAMMING_SMC
help
diff --git a/drivers/mtd/nand/raw/ndfc.c b/drivers/mtd/nand/raw/ndfc.c
index 3bb32a7c6d67..677d05d628a5 100644
--- a/drivers/mtd/nand/raw/ndfc.c
+++ b/drivers/mtd/nand/raw/ndfc.c
@@ -44,13 +44,13 @@ static void ndfc_select_chip(struct nand_chip *nchip, int chip)
uint32_t ccr;
struct ndfc_controller *ndfc = nand_get_controller_data(nchip);
- ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
+ ccr = ioread32be(ndfc->ndfcbase + NDFC_CCR);
if (chip >= 0) {
ccr &= ~NDFC_CCR_BS_MASK;
ccr |= NDFC_CCR_BS(chip + ndfc->chip_select);
} else
ccr |= NDFC_CCR_RESET_CE;
- out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
+ iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR);
}
static void ndfc_hwcontrol(struct nand_chip *chip, int cmd, unsigned int ctrl)
@@ -70,7 +70,7 @@ static int ndfc_ready(struct nand_chip *chip)
{
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
- return in_be32(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY;
+ return ioread32be(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY;
}
static void ndfc_enable_hwecc(struct nand_chip *chip, int mode)
@@ -78,9 +78,9 @@ static void ndfc_enable_hwecc(struct nand_chip *chip, int mode)
uint32_t ccr;
struct ndfc_controller *ndfc = nand_get_controller_data(chip);
- ccr = in_be32(ndfc->ndfcbase + NDFC_CCR);
+ ccr = ioread32be(ndfc->ndfcbase + NDFC_CCR);
ccr |= NDFC_CCR_RESET_ECC;
- out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
+ iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR);
wmb();
}
@@ -92,7 +92,7 @@ static int ndfc_calculate_ecc(struct nand_chip *chip,
uint8_t *p = (uint8_t *)&ecc;
wmb();
- ecc = in_be32(ndfc->ndfcbase + NDFC_ECC);
+ ecc = ioread32be(ndfc->ndfcbase + NDFC_ECC);
/* The NDFC uses Smart Media (SMC) bytes order */
ecc_code[0] = p[1];
ecc_code[1] = p[2];
@@ -114,7 +114,7 @@ static void ndfc_read_buf(struct nand_chip *chip, uint8_t *buf, int len)
uint32_t *p = (uint32_t *) buf;
for(;len > 0; len -= 4)
- *p++ = in_be32(ndfc->ndfcbase + NDFC_DATA);
+ *p++ = ioread32be(ndfc->ndfcbase + NDFC_DATA);
}
static void ndfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
@@ -123,7 +123,7 @@ static void ndfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len)
uint32_t *p = (uint32_t *) buf;
for(;len > 0; len -= 4)
- out_be32(ndfc->ndfcbase + NDFC_DATA, *p++);
+ iowrite32be(*p++, ndfc->ndfcbase + NDFC_DATA);
}
/*
@@ -185,10 +185,15 @@ static int ndfc_chip_init(struct ndfc_controller *ndfc,
static int ndfc_probe(struct platform_device *ofdev)
{
struct ndfc_controller *ndfc;
+ void __iomem *ndfcbase;
const __be32 *reg;
u32 ccr;
u32 cs;
- int err, len;
+ int len = 0;
+
+ ndfcbase = devm_platform_ioremap_resource(ofdev, 0);
+ if (IS_ERR(ndfcbase))
+ return PTR_ERR(ndfcbase);
/* Read the reg property to get the chip select */
reg = of_get_property(ofdev->dev.of_node, "reg", &len);
@@ -210,11 +215,7 @@ static int ndfc_probe(struct platform_device *ofdev)
ndfc->ofdev = ofdev;
dev_set_drvdata(&ofdev->dev, ndfc);
- ndfc->ndfcbase = of_iomap(ofdev->dev.of_node, 0);
- if (!ndfc->ndfcbase) {
- dev_err(&ofdev->dev, "failed to get memory\n");
- return -EIO;
- }
+ ndfc->ndfcbase = ndfcbase;
ccr = NDFC_CCR_BS(ndfc->chip_select);
@@ -223,22 +224,16 @@ static int ndfc_probe(struct platform_device *ofdev)
if (reg)
ccr |= be32_to_cpup(reg);
- out_be32(ndfc->ndfcbase + NDFC_CCR, ccr);
+ iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR);
/* Set the bank settings if given */
reg = of_get_property(ofdev->dev.of_node, "bank-settings", NULL);
if (reg) {
int offset = NDFC_BCFG0 + (ndfc->chip_select << 2);
- out_be32(ndfc->ndfcbase + offset, be32_to_cpup(reg));
+ iowrite32be(be32_to_cpup(reg), ndfc->ndfcbase + offset);
}
- err = ndfc_chip_init(ndfc, ofdev->dev.of_node);
- if (err) {
- iounmap(ndfc->ndfcbase);
- return err;
- }
-
- return 0;
+ return ndfc_chip_init(ndfc, ofdev->dev.of_node);
}
static void ndfc_remove(struct platform_device *ofdev)
diff --git a/drivers/net/can/dev/netlink.c b/drivers/net/can/dev/netlink.c
index 77d165ed0d53..884313a5cdd3 100644
--- a/drivers/net/can/dev/netlink.c
+++ b/drivers/net/can/dev/netlink.c
@@ -454,7 +454,7 @@ static size_t can_get_size(const struct net_device *dev)
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_STATE */
size += nla_total_size(sizeof(struct can_ctrlmode)); /* IFLA_CAN_CTRLMODE */
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_RESTART_MS */
- if (priv->do_get_berr_counter) /* IFLA_CAN_BERR_COUNTER */
+ if (priv->do_get_berr_counter && netif_running(dev)) /* IFLA_CAN_BERR_COUNTER */
size += nla_total_size(sizeof(struct can_berr_counter));
if (priv->data_bittiming.bitrate) /* IFLA_CAN_DATA_BITTIMING */
size += nla_total_size(sizeof(struct can_bittiming));
@@ -574,7 +574,7 @@ static int can_fill_info(struct sk_buff *skb, const struct net_device *dev)
nla_put(skb, IFLA_CAN_CTRLMODE, sizeof(cm), &cm) ||
nla_put_u32(skb, IFLA_CAN_RESTART_MS, priv->restart_ms) ||
- (priv->do_get_berr_counter &&
+ (priv->do_get_berr_counter && netif_running(dev) &&
!priv->do_get_berr_counter(dev, &bec) &&
nla_put(skb, IFLA_CAN_BERR_COUNTER, sizeof(bec), &bec)) ||
diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c
index 3ebd4f779b9b..6fb1f9b2a9e2 100644
--- a/drivers/net/can/dev/skb.c
+++ b/drivers/net/can/dev/skb.c
@@ -52,6 +52,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
if (idx >= priv->echo_skb_max) {
netdev_err(dev, "%s: BUG! Trying to access can_priv::echo_skb out of bounds (%u/max %u)\n",
__func__, idx, priv->echo_skb_max);
+ dev_kfree_skb_any(skb);
return -EINVAL;
}
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
index 6ebfbc0753d9..7954937e828b 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
@@ -754,7 +754,7 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
u16 rx_msg_type, rx_msg_size;
size_t rx_msg_min_size;
- if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ if (msg_end - msg_ptr < sizeof(rx_msg->size)) {
err = -EBADMSG;
break;
}
@@ -765,6 +765,11 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
break;
}
+ if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ err = -EBADMSG;
+ break;
+ }
+
rx_msg_size = le16_to_cpu(rx_msg->size);
rx_msg_type = le16_to_cpu(rx_msg->type);
diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
index 96152e5a920a..5af78b3de09d 100644
--- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
+++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
@@ -1934,6 +1934,39 @@ static int atl1c_clean_rx(struct napi_struct *napi, int budget)
break;
}
atl1c_clean_rfd(rfd_ring, rrs, rfd_num);
+
+ /* A link partner whose l2mtu is larger than ours can hand us a
+ * frame that does not fit the buffer the hardware was given:
+ * the descriptor reports the untruncated length while at most
+ * buffer_info->length bytes were written. skb_put() of such a
+ * length would pull in whatever the tail of the buffer holds
+ * and, past the end of the skb, trips skb_over_panic and takes
+ * the host down. Bound it by what the device actually wrote
+ * rather than by skb_tailroom(): the skb carries 64 bytes of
+ * slack for the DMA erratum, so the two are not the same.
+ *
+ * Frames shorter than an Ethernet header plus its FCS are
+ * rejected by the same test; before this, a length below
+ * ETH_FCS_LEN underflowed in skb_put().
+ */
+ if (unlikely(length < ETH_HLEN + ETH_FCS_LEN ||
+ length > buffer_info->length)) {
+ dev_core_stats_rx_dropped_inc(netdev);
+ if (netif_msg_rx_err(adapter))
+ dev_warn_ratelimited(&pdev->dev,
+ "dropped rx frame: %u bytes, buffer %u\n",
+ length, buffer_info->length);
+ kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
+ /* atl1c_clean_rfd() has already recycled the RFD, so
+ * count the drop as work: the refill at the end of the
+ * poll only runs when work_done is non-zero, and
+ * sustained oversized traffic would otherwise drain the
+ * ring and stop RX for good.
+ */
+ work_done++;
+ continue;
+ }
+
skb_put(skb, length - ETH_FCS_LEN);
skb->protocol = eth_type_trans(skb, netdev);
atl1c_rx_checksum(adapter, skb, rrs);
diff --git a/drivers/net/ethernet/broadcom/bgmac.c b/drivers/net/ethernet/broadcom/bgmac.c
index 6ffdc4229407..258b3b5a1cdb 100644
--- a/drivers/net/ethernet/broadcom/bgmac.c
+++ b/drivers/net/ethernet/broadcom/bgmac.c
@@ -1183,9 +1183,6 @@ static int bgmac_open(struct net_device *net_dev)
if (err)
return err;
- /* Specs say about reclaiming rings here, but we do that in DMA init */
- bgmac_chip_init(bgmac);
-
err = request_irq(bgmac->irq, bgmac_interrupt, IRQF_SHARED,
net_dev->name, net_dev);
if (err < 0) {
@@ -1195,6 +1192,9 @@ static int bgmac_open(struct net_device *net_dev)
}
napi_enable(&bgmac->napi);
+ /* Specs say about reclaiming rings here, but we do that in DMA init */
+ bgmac_chip_init(bgmac);
+
phy_start(net_dev->phydev);
netif_start_queue(net_dev);
@@ -1609,10 +1609,10 @@ int bgmac_enet_resume(struct bgmac *bgmac)
if (rc)
return rc;
- bgmac_chip_init(bgmac);
-
napi_enable(&bgmac->napi);
+ bgmac_chip_init(bgmac);
+
netif_tx_lock(bgmac->net_dev);
netif_device_attach(bgmac->net_dev);
netif_tx_unlock(bgmac->net_dev);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 2c16348d3b8c..fcdffdc3ffa8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -380,12 +380,34 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
return nxmit;
}
+static void bnxt_xdp_apply_cfg(struct bnxt *bp, int tx_xdp)
+{
+ struct net_device *dev = bp->dev;
+ int tc = bp->num_tc ? : 1;
+ int tx_cp;
+
+ if (bp->xdp_prog) {
+ bnxt_set_rx_skb_mode(bp, true);
+ xdp_features_set_redirect_target(dev, true);
+ } else {
+ xdp_features_clear_redirect_target(dev);
+ bnxt_set_rx_skb_mode(bp, false);
+ }
+ bp->tx_nr_rings_xdp = tx_xdp;
+ bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp;
+ tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
+ bp->cp_nr_rings = max_t(int, tx_cp, bp->rx_nr_rings);
+ bnxt_set_tpa_flags(bp);
+ bnxt_set_ring_params(bp);
+}
+
/* Under rtnl_lock */
static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
{
struct net_device *dev = bp->dev;
- int tx_xdp = 0, tx_cp, rc, tc;
+ int tx_xdp = 0, rc, tc;
struct bpf_prog *old;
+ int old_tx_xdp;
if (prog && !prog->aux->xdp_has_frags &&
bp->dev->mtu > BNXT_MAX_PAGE_MODE_MTU) {
@@ -412,26 +434,22 @@ static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
if (netif_running(dev))
bnxt_close_nic(bp, true, false);
+ old_tx_xdp = bp->tx_nr_rings_xdp;
old = xchg(&bp->xdp_prog, prog);
- if (old)
- bpf_prog_put(old);
-
- if (prog) {
- bnxt_set_rx_skb_mode(bp, true);
- xdp_features_set_redirect_target(dev, true);
- } else {
- xdp_features_clear_redirect_target(dev);
- bnxt_set_rx_skb_mode(bp, false);
+ bnxt_xdp_apply_cfg(bp, tx_xdp);
+
+ if (netif_running(dev)) {
+ rc = bnxt_open_nic(bp, true, false);
+ /* a detach cannot be undone, only an attach */
+ if (rc && prog) {
+ WRITE_ONCE(bp->xdp_prog, old);
+ bnxt_xdp_apply_cfg(bp, old_tx_xdp);
+ return rc;
+ }
}
- bp->tx_nr_rings_xdp = tx_xdp;
- bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp;
- tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
- bp->cp_nr_rings = max_t(int, tx_cp, bp->rx_nr_rings);
- bnxt_set_tpa_flags(bp);
- bnxt_set_ring_params(bp);
- if (netif_running(dev))
- return bnxt_open_nic(bp, true, false);
+ if (old)
+ bpf_prog_put(old);
return 0;
}
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 7f143e388d42..190e7f622e7f 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -18170,41 +18170,17 @@ static int tg3_resume(struct device *device)
static SIMPLE_DEV_PM_OPS(tg3_pm_ops, tg3_suspend, tg3_resume);
-/* Systems where ACPI _PTS (Prepare To Sleep) S5 will result in a fatal
- * PCIe AER event on the tg3 device if the tg3 device is not, or cannot
- * be, powered down.
+/* Systems where powering down a tg3 device on restarts triggers
+ * NMIs and fatal hangs during reboot if the network adapters were
+ * initialized under SNP.
*/
-static const struct dmi_system_id tg3_restart_aer_quirk_table[] = {
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R440"),
- },
- },
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R540"),
- },
- },
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R640"),
- },
- },
+static const struct dmi_system_id tg3_restart_snp_quirk_table[] = {
{
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R650"),
},
},
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R740"),
- },
- },
{
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
@@ -18228,14 +18204,18 @@ static void tg3_shutdown(struct pci_dev *pdev)
if (netif_running(dev))
dev_close(dev);
- if (system_state == SYSTEM_POWER_OFF)
+ if (!dmi_first_match(tg3_restart_snp_quirk_table) ||
+ system_state == SYSTEM_POWER_OFF) {
tg3_power_down(tp);
- else if (system_state == SYSTEM_RESTART &&
- dmi_first_match(tg3_restart_aer_quirk_table) &&
+ } else if (system_state == SYSTEM_RESTART &&
pdev->current_state != PCI_D3cold &&
pdev->current_state != PCI_UNKNOWN) {
/* Disable PCIe AER on the tg3 to avoid a fatal
* error during this system restart.
+ * Needed since on restarts for systems in
+ * tg3_restart_snp_quirk_table skipping on powering down the
+ * tg3 device causes a fatal AER error during ACPI _PTS
+ * (Prepare To Sleep).
*/
pcie_capability_clear_word(pdev, PCI_EXP_DEVCTL,
PCI_EXP_DEVCTL_CERE |
diff --git a/drivers/net/ethernet/jme.c b/drivers/net/ethernet/jme.c
index d8be0e4dcb07..6cfcfda05554 100644
--- a/drivers/net/ethernet/jme.c
+++ b/drivers/net/ethernet/jme.c
@@ -1502,7 +1502,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
* all other events are ignored
*/
jwrite32(jme, JME_IEVE, intrstat);
- schedule_work(&jme->linkch_task);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ schedule_work(&jme->linkch_task);
goto out_reenable;
}
@@ -1546,7 +1547,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
/*
* Re-enable interrupt
*/
- jwrite32f(jme, JME_IENS, INTR_ENABLE);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ jwrite32f(jme, JME_IENS, INTR_ENABLE);
}
static irqreturn_t
@@ -1892,6 +1894,7 @@ jme_powersave_phy(struct jme_adapter *jme)
jme_set_100m_half(jme);
if (jme->reg_pmcs & (PMCS_LFEN | PMCS_LREN))
jme_wait_link(jme);
+ jme_mac_rxclk_on(jme);
jme_clear_pm_enable_wol(jme);
} else {
jme_phy_off(jme);
@@ -3158,6 +3161,21 @@ jme_shutdown(struct pci_dev *pdev)
struct net_device *netdev = pci_get_drvdata(pdev);
struct jme_adapter *jme = netdev_priv(netdev);
+ if (netif_running(netdev)) {
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ netif_device_detach(netdev);
+ netif_stop_queue(netdev);
+ jme_stop_irq(jme);
+ synchronize_irq(pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ tasklet_disable(&jme->txclean_task);
+ tasklet_disable(&jme->rxclean_task);
+ tasklet_disable(&jme->rxempty_task);
+ }
+
jme_powersave_phy(jme);
pci_pme_active(pdev, true);
}
@@ -3172,11 +3190,16 @@ jme_suspend(struct device *dev)
if (!netif_running(netdev))
return 0;
- atomic_dec(&jme->link_changing);
-
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
netif_device_detach(netdev);
netif_stop_queue(netdev);
jme_stop_irq(jme);
+ synchronize_irq(jme->pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ atomic_dec(&jme->link_changing);
tasklet_disable(&jme->txclean_task);
tasklet_disable(&jme->rxclean_task);
@@ -3215,6 +3238,7 @@ jme_resume(struct device *dev)
return 0;
jme_clear_pm_disable_wol(jme);
+ jme_mac_rxclk_off(jme);
jme_phy_on(jme);
if (test_bit(JME_FLAG_SSET, &jme->flags))
jme_set_link_ksettings(netdev, &jme->old_cmd);
@@ -3226,8 +3250,11 @@ jme_resume(struct device *dev)
atomic_inc(&jme->link_changing);
- jme_reset_link(jme);
+ napi_enable(&jme->napi);
+ tasklet_enable(&jme->pcc_task);
+ clear_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ jme_reset_link(jme);
jme_start_irq(jme);
return 0;
diff --git a/drivers/net/ethernet/jme.h b/drivers/net/ethernet/jme.h
index 860494ff3714..ec5dfddf7322 100644
--- a/drivers/net/ethernet/jme.h
+++ b/drivers/net/ethernet/jme.h
@@ -450,6 +450,7 @@ enum jme_flags_bits {
JME_FLAG_SSET = 2,
JME_FLAG_POLL = 5,
JME_FLAG_SHUTDOWN = 6,
+ JME_FLAG_WOL_QUIESCE = 7,
};
#define TX_TIMEOUT (5 * HZ)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c
index 4a79c0d7e7ad..feaeb4118f0b 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core.c
@@ -2002,7 +2002,7 @@ mlxsw_core_health_fw_fatal_test(struct devlink_health_reporter *reporter,
struct netlink_ext_ack *extack)
{
struct mlxsw_core *mlxsw_core = devlink_health_reporter_priv(reporter);
- char mfgd_pl[MLXSW_REG_MFGD_LEN];
+ char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0};
int err;
/* Read the register first to make sure no other bits are changed. */
@@ -2023,7 +2023,7 @@ mlxsw_core_health_fw_fatal_ops = {
static int mlxsw_core_health_fw_fatal_config(struct mlxsw_core *mlxsw_core,
bool enable)
{
- char mfgd_pl[MLXSW_REG_MFGD_LEN];
+ char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0};
int err;
/* Read the register first to make sure no other bits are changed. */
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
index 9c12e1feb643..e39a8b0a5709 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
@@ -120,7 +120,7 @@ static ssize_t mlxsw_hwmon_temp_rst_store(struct device *dev,
container_of(attr, struct mlxsw_hwmon_attr, dev_attr);
struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev;
struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon;
- char mtmp_pl[MLXSW_REG_MTMP_LEN];
+ char mtmp_pl[MLXSW_REG_MTMP_LEN] = {0};
unsigned long val;
int index;
int err;
@@ -177,7 +177,7 @@ static ssize_t mlxsw_hwmon_fan_fault_show(struct device *dev,
container_of(attr, struct mlxsw_hwmon_attr, dev_attr);
struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev;
struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon;
- char fore_pl[MLXSW_REG_FORE_LEN];
+ char fore_pl[MLXSW_REG_FORE_LEN] = {0};
bool fault;
int err;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
index 32d2e61f2b82..91fb77c8e20d 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
@@ -1720,7 +1720,7 @@ static int mlxsw_sp2_span_init(struct mlxsw_sp *mlxsw_sp)
static int mlxsw_sp2_span_policer_id_base_set(struct mlxsw_sp *mlxsw_sp,
u16 policer_id_base)
{
- char mogcr_pl[MLXSW_REG_MOGCR_LEN];
+ char mogcr_pl[MLXSW_REG_MOGCR_LEN] = {0};
int err;
err = mlxsw_reg_query(mlxsw_sp->core, MLXSW_REG(mogcr), mogcr_pl);
diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
index e9d9d69e2ba9..12860cf2fb93 100644
--- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c
+++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
@@ -121,6 +121,7 @@
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 BIT(21)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 BIT(22)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO BIT(23)
+#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN BIT(24)
/* The PTP event messages - Sync, Delay_Req, Pdelay_Req, and Pdelay_Resp. */
#define AM65_CPSW_TS_EVENT_MSG_TYPE_BITS (BIT(0) | BIT(1) | BIT(2) | BIT(3))
@@ -1830,7 +1831,8 @@ static int am65_cpsw_nuss_hwtstamp_set(struct net_device *ndev,
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_132 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 |
- AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO;
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO |
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN;
ts_ctrl = AM65_CPSW_TS_EVENT_MSG_TYPE_BITS <<
AM65_CPSW_PN_TS_CTL_MSG_TYPE_EN_SHIFT;
diff --git a/drivers/net/hamradio/6pack.c b/drivers/net/hamradio/6pack.c
index 437e5e391a77..5168658ed2c9 100644
--- a/drivers/net/hamradio/6pack.c
+++ b/drivers/net/hamradio/6pack.c
@@ -256,6 +256,12 @@ static int sp_open_dev(struct net_device *dev)
if (sp->tty == NULL)
return -ENODEV;
+
+ /* sp_close() stopped the queue: restart it, or the interface
+ * never transmits again after it is set down and up.
+ */
+ netif_start_queue(dev);
+
return 0;
}
diff --git a/drivers/net/hamradio/bpqether.c b/drivers/net/hamradio/bpqether.c
index bac1bb69d63a..f027205347d3 100644
--- a/drivers/net/hamradio/bpqether.c
+++ b/drivers/net/hamradio/bpqether.c
@@ -287,11 +287,10 @@ static netdev_tx_t bpq_xmit(struct sk_buff *skb, struct net_device *dev)
skb->protocol = ax25_type_trans(skb, dev);
skb_reset_network_header(skb);
dev_hard_header(skb, dev, ETH_P_BPQ, bpq->dest_addr, NULL, 0);
- dev->stats.tx_packets++;
- dev->stats.tx_bytes+=skb->len;
-
+ orig_dev->stats.tx_packets++;
+ orig_dev->stats.tx_bytes += skb->len;
+
dev_queue_xmit(skb);
- netif_wake_queue(dev);
return NETDEV_TX_OK;
}
@@ -387,6 +386,7 @@ static void *bpq_seq_start(struct seq_file *seq, loff_t *pos)
list_for_each_entry_rcu(bpqdev, &bpq_devices, bpq_list) {
if (i == *pos)
return bpqdev;
+ i++;
}
return NULL;
}
diff --git a/drivers/net/hamradio/mkiss.c b/drivers/net/hamradio/mkiss.c
index 5f38a002bd9e..8b0e2ca5d0ea 100644
--- a/drivers/net/hamradio/mkiss.c
+++ b/drivers/net/hamradio/mkiss.c
@@ -30,6 +30,7 @@
#include <net/ax25.h>
#define AX_MTU 236
+#define AX_MAX_MTU 65535
/* some arch define END as assembly function ending, just undef it */
#undef END
@@ -355,13 +356,19 @@ static int ax_set_mac_address(struct net_device *dev, void *addr)
/*---------------------------------------------------------------------------*/
-static void ax_changedmtu(struct mkiss *ax)
-{
- struct net_device *dev = ax->dev;
- unsigned char *xbuff, *rbuff, *oxbuff, *orbuff;
- int len;
+/*
+ * Length of the frame buffers for the current MTU.
+ *
+ * ax_encaps() accepts frames of up to dev->mtu + 73 bytes and KISS escaping
+ * can double every one of them. The buffers are allocated AX_BUF_SLACK bytes
+ * larger than this, for the two END delimiters and the two CRC bytes, which
+ * can need escaping as well.
+ */
+#define AX_BUF_SLACK 6
- len = dev->mtu * 2;
+static int ax_buflen(const struct net_device *dev)
+{
+ int len = (dev->mtu + 73) * 2;
/*
* allow for arrival of larger UDP packets, even if we say not to
@@ -371,8 +378,19 @@ static void ax_changedmtu(struct mkiss *ax)
if (len < 576 * 2)
len = 576 * 2;
- xbuff = kmalloc(len + 4, GFP_ATOMIC);
- rbuff = kmalloc(len + 4, GFP_ATOMIC);
+ return len;
+}
+
+static void ax_changedmtu(struct mkiss *ax)
+{
+ struct net_device *dev = ax->dev;
+ unsigned char *xbuff, *rbuff, *oxbuff, *orbuff;
+ int len;
+
+ len = ax_buflen(dev);
+
+ xbuff = kmalloc(len + AX_BUF_SLACK, GFP_ATOMIC);
+ rbuff = kmalloc(len + AX_BUF_SLACK, GFP_ATOMIC);
if (xbuff == NULL || rbuff == NULL) {
printk(KERN_ERR "mkiss: %s: unable to grow ax25 buffers, "
@@ -508,6 +526,14 @@ static void ax_encaps(struct net_device *dev, unsigned char *icp, int len)
set_bit(TTY_DO_WRITE_WAKEUP, &ax->tty->flags);
actual = ax->tty->ops->write(ax->tty, ax->xbuff, count);
+ if (actual < 0) {
+ /* The tty refused the frame: drop it. */
+ clear_bit(TTY_DO_WRITE_WAKEUP, &ax->tty->flags);
+ dev->stats.tx_errors++;
+ ax->xleft = 0;
+ netif_start_queue(dev);
+ return;
+ }
dev->stats.tx_packets++;
dev->stats.tx_bytes += actual;
@@ -563,6 +589,11 @@ static int ax_open_dev(struct net_device *dev)
if (ax->tty == NULL)
return -ENODEV;
+ /* ax_close() stopped the queue: restart it, or the interface
+ * never transmits again after it is set down and up.
+ */
+ netif_start_queue(dev);
+
return 0;
}
@@ -581,20 +612,12 @@ static int ax_open(struct net_device *dev)
* rbuff Receive buffer.
* xbuff Transmit buffer.
*/
- len = dev->mtu * 2;
+ len = ax_buflen(dev);
- /*
- * allow for arrival of larger UDP packets, even if we say not to
- * also fixes a bug in which SunOS sends 512-byte packets even with
- * an MSS of 128
- */
- if (len < 576 * 2)
- len = 576 * 2;
-
- if ((ax->rbuff = kmalloc(len + 4, GFP_KERNEL)) == NULL)
+ if ((ax->rbuff = kmalloc(len + AX_BUF_SLACK, GFP_KERNEL)) == NULL)
goto norbuff;
- if ((ax->xbuff = kmalloc(len + 4, GFP_KERNEL)) == NULL)
+ if ((ax->xbuff = kmalloc(len + AX_BUF_SLACK, GFP_KERNEL)) == NULL)
goto noxbuff;
ax->mtu = dev->mtu + 73;
@@ -640,6 +663,7 @@ static void ax_setup(struct net_device *dev)
{
/* Finish setting up the DEVICE info. */
dev->mtu = AX_MTU;
+ dev->max_mtu = AX_MAX_MTU;
dev->hard_header_len = AX25_MAX_HEADER_LEN;
dev->addr_len = AX25_ADDR_LEN;
dev->type = ARPHRD_AX25;
@@ -928,6 +952,14 @@ static void mkiss_write_wakeup(struct tty_struct *tty)
}
actual = tty->ops->write(tty, ax->xhead, ax->xleft);
+ if (actual < 0) {
+ /* Give up on the rest of this frame. */
+ clear_bit(TTY_DO_WRITE_WAKEUP, &tty->flags);
+ ax->dev->stats.tx_errors++;
+ ax->xleft = 0;
+ netif_wake_queue(ax->dev);
+ goto out;
+ }
ax->xleft -= actual;
ax->xhead += actual;
diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c
index 8ff2024dde60..293b6fb1c1e9 100644
--- a/drivers/net/ppp/pppoe.c
+++ b/drivers/net/ppp/pppoe.c
@@ -194,18 +194,16 @@ static int __set_item(struct pppoe_net *pn, struct pppox_sock *po)
return 0;
}
-static void __delete_item(struct pppoe_net *pn, __be16 sid,
- char *addr, int ifindex)
+static void __delete_item(struct pppoe_net *pn, struct pppox_sock *po)
{
- int hash = hash_item(sid, addr);
+ int hash = hash_item(po->pppoe_pa.sid, po->pppoe_pa.remote);
struct pppox_sock *ret, **src;
ret = pn->hash_table[hash];
src = &pn->hash_table[hash];
while (ret) {
- if (cmp_addr(&ret->pppoe_pa, sid, addr) &&
- ret->pppoe_ifindex == ifindex) {
+ if (ret == po) {
*src = ret->next;
break;
}
@@ -255,11 +253,10 @@ static inline struct pppox_sock *get_item_by_addr(struct net *net,
return pppox_sock;
}
-static inline void delete_item(struct pppoe_net *pn, __be16 sid,
- char *addr, int ifindex)
+static void delete_item(struct pppoe_net *pn, struct pppox_sock *po)
{
write_lock_bh(&pn->hash_lock);
- __delete_item(pn, sid, addr, ifindex);
+ __delete_item(pn, po);
write_unlock_bh(&pn->hash_lock);
}
@@ -599,8 +596,7 @@ static int pppoe_release(struct socket *sock)
* protect "po" from concurrent updates
* on pppoe_flush_dev
*/
- delete_item(pn, po->pppoe_pa.sid, po->pppoe_pa.remote,
- po->pppoe_ifindex);
+ delete_item(pn, po);
sock_orphan(sk);
sock->sk = NULL;
@@ -651,8 +647,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr *uservaddr,
if (stage_session(po->pppoe_pa.sid)) {
pppox_unbind_sock(sk);
pn = pppoe_pernet(sock_net(sk));
- delete_item(pn, po->pppoe_pa.sid,
- po->pppoe_pa.remote, po->pppoe_ifindex);
+ delete_item(pn, po);
if (po->pppoe_dev) {
dev_put(po->pppoe_dev);
po->pppoe_dev = NULL;
@@ -703,8 +698,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr *uservaddr,
error = ppp_register_net_channel(dev_net(dev), &po->chan);
if (error) {
- delete_item(pn, po->pppoe_pa.sid,
- po->pppoe_pa.remote, po->pppoe_ifindex);
+ delete_item(pn, po);
goto err_put;
}
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 564bf57bc897..22387f66bf90 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1371,9 +1371,10 @@ static int veth_set_channels(struct net_device *dev,
* to identify the range we have to disable
*/
veth_disable_range_safe(dev, new_rx_count, old_rx_count);
- netif_carrier_on(dev);
- if (peer)
+ if (peer && (peer->flags & IFF_UP)) {
+ netif_carrier_on(dev);
netif_carrier_on(peer);
+ }
}
/* update XDP supported features */
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index e70a0b657570..caba78398d92 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -1006,12 +1006,6 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan,
return -EOPNOTSUPP;
}
- if (rcu_access_pointer(f->nh) &&
- !(state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic");
- return -EOPNOTSUPP;
- }
-
/* Do not allow an externally learned entry to take over an entry added
* by the user.
*/
@@ -1279,11 +1273,6 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[],
if (err)
return err;
- if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic");
- return -EINVAL;
- }
-
if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family)
return -EAFNOSUPPORT;
@@ -2896,6 +2885,9 @@ static void vxlan_cleanup(struct timer_list *t)
if (f->flags & NTF_EXT_LEARNED)
continue;
+ if (rcu_access_pointer(f->nh))
+ continue;
+
timeout = READ_ONCE(f->used) + vxlan->cfg.age_interval * HZ;
if (time_before_eq(timeout, jiffies)) {
netdev_dbg(vxlan->dev,
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
index c3a3a0817910..a2a0cc43bc72 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
@@ -6424,6 +6424,7 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
struct brcmf_cfg80211_profile *profile = &ifp->vif->profile;
struct brcmf_cfg80211_connect_info *conn_info = cfg_to_conn(cfg);
struct cfg80211_connect_resp_params conn_params;
+ bool authorized;
brcmf_dbg(TRACE, "Enter\n");
@@ -6448,7 +6449,13 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
conn_params.req_ie_len = conn_info->req_ie_len;
conn_params.resp_ie = conn_info->resp_ie;
conn_params.resp_ie_len = conn_info->resp_ie_len;
+ authorized = completed &&
+ (profile->use_fwsup == BRCMF_PROFILE_FWSUP_SAE ||
+ profile->use_fwsup == BRCMF_PROFILE_FWSUP_PSK);
cfg80211_connect_done(ndev, &conn_params, GFP_KERNEL);
+ if (authorized)
+ cfg80211_port_authorized(ndev, profile->bssid, NULL, 0,
+ GFP_KERNEL);
brcmf_dbg(CONN, "Report connect result - connection %s\n",
completed ? "succeeded" : "failed");
}
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index cd02dff7ab34..13e659494518 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -973,6 +973,20 @@ void mt76_update_survey(struct mt76_phy *phy)
}
EXPORT_SYMBOL_GPL(mt76_update_survey);
+void mt76_phy_chandef_set(struct mt76_phy *phy,
+ struct cfg80211_chan_def *chandef)
+{
+ if (!chandef->chan)
+ return;
+
+ mt76_update_survey(phy);
+
+ phy->chandef = *chandef;
+ phy->main_chan = chandef->chan;
+ phy->chan_state = mt76_channel_state(phy, chandef->chan);
+}
+EXPORT_SYMBOL_GPL(mt76_phy_chandef_set);
+
int mt76_set_channel(struct mt76_phy *phy, struct cfg80211_chan_def *chandef,
bool offchannel)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index 8addbffa97c5..d709b7415968 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -1400,6 +1400,8 @@ void mt76_release_buffered_frames(struct ieee80211_hw *hw,
bool mt76_has_tx_pending(struct mt76_phy *phy);
int mt76_update_channel(struct mt76_phy *phy);
void mt76_update_survey(struct mt76_phy *phy);
+void mt76_phy_chandef_set(struct mt76_phy *phy,
+ struct cfg80211_chan_def *chandef);
void mt76_update_survey_active_time(struct mt76_phy *phy, ktime_t time);
int mt76_get_survey(struct ieee80211_hw *hw, int idx,
struct survey_info *survey);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
index 99561094640f..18eb934da570 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
@@ -1394,6 +1394,7 @@ mt7921_change_chanctx(struct ieee80211_hw *hw,
mt7921_mcu_config_sniffer(mvif, ctx);
else
mt76_connac_mcu_uni_set_chctx(mvif->phy->mt76, &mvif->bss_conf.mt76, ctx);
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mt792x_mutex_release(phy->dev);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
index 9a52607dc5a8..dba0f4892207 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
@@ -1873,6 +1873,7 @@ mt7925_change_chanctx(struct ieee80211_hw *hw,
link_conf, ctx);
}
}
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mt792x_mutex_release(phy->dev);
}
@@ -2170,6 +2171,7 @@ static int mt7925_assign_vif_chanctx(struct ieee80211_hw *hw,
mconf->mt76.ctx = ctx;
mctx->bss_conf = mconf;
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mutex_unlock(&dev->mt76.mutex);
return 0;
diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_core.c b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
index c6afdf90fade..86b45807481f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt792x_core.c
+++ b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
@@ -308,6 +308,7 @@ int mt792x_assign_vif_chanctx(struct ieee80211_hw *hw,
mutex_lock(&dev->mt76.mutex);
mvif->bss_conf.mt76.ctx = ctx;
mctx->bss_conf = &mvif->bss_conf;
+ mt76_phy_chandef_set(&dev->mphy, &ctx->def);
mutex_unlock(&dev->mt76.mutex);
return 0;
@@ -658,7 +659,7 @@ int mt792x_init_wiphy(struct ieee80211_hw *hw)
}
EXPORT_SYMBOL_GPL(mt792x_init_wiphy);
-static u8
+static int
mt792x_get_offload_capability(struct device *dev, const char *fw_wm)
{
const struct mt76_connac2_fw_trailer *hdr;
@@ -720,13 +721,20 @@ mt792x_get_mac80211_ops(struct device *dev,
void *drv_data, u8 *fw_features)
{
struct ieee80211_ops *ops;
+ int ret;
ops = devm_kmemdup(dev, mac80211_ops, sizeof(struct ieee80211_ops),
GFP_KERNEL);
if (!ops)
return NULL;
- *fw_features = mt792x_get_offload_capability(dev, drv_data);
+ ret = mt792x_get_offload_capability(dev, drv_data);
+ if (ret < 0) {
+ dev_warn(dev, "failed to read firmware offload caps: %d\n", ret);
+ *fw_features = 0;
+ } else {
+ *fw_features = ret;
+ }
if (!(*fw_features & MT792x_FW_CAP_CNM)) {
ops->remain_on_channel = NULL;
ops->cancel_remain_on_channel = NULL;
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c
index 910431591420..bf21937f25b2 100644
--- a/drivers/net/wireless/realtek/rtw88/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw88/mac80211.c
@@ -466,6 +466,7 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_set(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_AP);
rtwdev->ap_active = true;
rtw_store_op_chan(rtwdev, true);
chip->ops->phy_calibration(rtwdev);
@@ -482,6 +483,7 @@ static void rtw_ops_stop_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_clr(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_DEFAULT);
rtwdev->ap_active = false;
if (!rtw_core_check_sta_active(rtwdev))
rtw_clear_op_chan(rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw88/reg.h b/drivers/net/wireless/realtek/rtw88/reg.h
index 2708ee2f12a4..6ff593f62066 100644
--- a/drivers/net/wireless/realtek/rtw88/reg.h
+++ b/drivers/net/wireless/realtek/rtw88/reg.h
@@ -431,6 +431,8 @@
#define REG_DRVERLYINT 0x0558
#define REG_BCNDMATIM 0x0559
#define REG_ATIMWND 0x055A
+#define ATIMWND_DEFAULT 0x2
+#define ATIMWND_AP 0xa
#define REG_USTIME_TSF 0x055C
#define REG_BCN_MAX_ERR 0x055D
#define REG_RXTSF_OFFSET_CCK 0x055E
diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c
index 0c8817bf46c7..959ae66776aa 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.c
+++ b/drivers/net/wireless/realtek/rtw88/tx.c
@@ -191,7 +191,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t)
void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
{
struct rtw_tx_report *tx_report = &rtwdev->tx_report;
- unsigned long timeout = RTW_TX_PROBE_TIMEOUT;
+ unsigned long timeout;
unsigned long flags;
u8 *drv_data;
@@ -203,9 +203,10 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
__skb_queue_tail(&tx_report->queue, skb);
spin_unlock_irqrestore(&tx_report->q_lock, flags);
- if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D &&
- rtwdev->hci.type == RTW_HCI_TYPE_USB)
- timeout = msecs_to_jiffies(2500);
+ if (rtwdev->hci.type == RTW_HCI_TYPE_PCIE)
+ timeout = RTW_TX_PROBE_TIMEOUT_SHORT;
+ else
+ timeout = RTW_TX_PROBE_TIMEOUT_LONG;
mod_timer(&tx_report->purge_timer, jiffies + timeout);
}
diff --git a/drivers/net/wireless/realtek/rtw88/tx.h b/drivers/net/wireless/realtek/rtw88/tx.h
index 3d544fd7f60f..f361713ece9f 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.h
+++ b/drivers/net/wireless/realtek/rtw88/tx.h
@@ -7,7 +7,8 @@
#define RTK_TX_MAX_AGG_NUM_MASK 0x1f
-#define RTW_TX_PROBE_TIMEOUT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_SHORT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_LONG msecs_to_jiffies(2500)
struct rtw_tx_desc {
__le32 w0;
diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c
index b3e1e9a0fda6..e1df0ad1852b 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.c
+++ b/drivers/net/wireless/realtek/rtw88/usb.c
@@ -2,9 +2,11 @@
/* Copyright(c) 2018-2019 Realtek Corporation
*/
+#include <linux/ip.h>
#include <linux/module.h>
-#include <linux/usb.h>
#include <linux/mutex.h>
+#include <linux/udp.h>
+#include <linux/usb.h>
#include "main.h"
#include "debug.h"
#include "reg.h"
@@ -555,9 +557,61 @@ static int rtw_usb_write_data_h2c(struct rtw_dev *rtwdev, u8 *buf, u32 size)
return rtw_usb_write_data(rtwdev, &pkt_info, buf);
}
-static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
+static bool rtw_usb_bmc_needs_dtim(struct sk_buff *skb)
+{
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+ struct udphdr *udphdr;
+
+ if (info->control.flags & IEEE80211_TX_CTRL_PORT_CTRL_PROTO)
+ return true;
+
+ if (skb->protocol == htons(ETH_P_ARP))
+ return true;
+
+ if (skb->protocol == htons(ETH_P_IP) &&
+ ip_hdr(skb)->protocol == IPPROTO_UDP) {
+ udphdr = udp_hdr(skb);
+
+ return udphdr->dest == htons(67) || udphdr->dest == htons(68);
+ }
+
+ return false;
+}
+
+#define RTW_USB_HIQ_REFILL_INTERVAL (HZ / 10) /* one unit of budget per 100 ms */
+#define RTW_USB_HIQ_BUDGET_MAX 16
+
+static bool rtw_usb_hiq_take_budget(struct rtw_usb *rtwusb)
+{
+ unsigned long flags, elapsed, add;
+ bool ok;
+
+ spin_lock_irqsave(&rtwusb->hiq_lock, flags);
+
+ elapsed = jiffies - rtwusb->hiq_refill;
+ add = elapsed / RTW_USB_HIQ_REFILL_INTERVAL;
+ if (add) {
+ rtwusb->hiq_budget = min_t(unsigned long, rtwusb->hiq_budget + add,
+ RTW_USB_HIQ_BUDGET_MAX);
+
+ /* keep the unfinished part of the interval for the next unit */
+ rtwusb->hiq_refill = jiffies - elapsed % RTW_USB_HIQ_REFILL_INTERVAL;
+ }
+
+ ok = rtwusb->hiq_budget > 0;
+ if (ok)
+ rtwusb->hiq_budget--;
+
+ spin_unlock_irqrestore(&rtwusb->hiq_lock, flags);
+
+ return ok;
+}
+
+static u8 rtw_usb_tx_queue_mapping_to_qsel(struct rtw_usb *rtwusb,
+ struct sk_buff *skb)
{
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
__le16 fc = hdr->frame_control;
u8 qsel;
@@ -565,7 +619,9 @@ static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
qsel = TX_DESC_QSEL_MGMT;
else if (is_broadcast_ether_addr(hdr->addr1) ||
is_multicast_ether_addr(hdr->addr1))
- qsel = TX_DESC_QSEL_HIGH;
+ qsel = (info->flags & IEEE80211_TX_CTL_SEND_AFTER_DTIM) &&
+ rtw_usb_bmc_needs_dtim(skb) && rtw_usb_hiq_take_budget(rtwusb) ?
+ TX_DESC_QSEL_HIGH : skb->priority;
else if (skb_get_queue_mapping(skb) <= IEEE80211_AC_BK)
qsel = skb->priority;
else
@@ -584,7 +640,7 @@ static int rtw_usb_tx_write(struct rtw_dev *rtwdev,
u8 *pkt_desc;
int ep;
- pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(skb);
+ pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(rtwusb, skb);
pkt_desc = skb_push(skb, chip->tx_pkt_desc_sz);
memset(pkt_desc, 0, chip->tx_pkt_desc_sz);
ep = qsel_to_ep(rtwusb, pkt_info->qsel);
@@ -935,6 +991,10 @@ static int rtw_usb_init_tx(struct rtw_dev *rtwdev)
struct rtw_usb *rtwusb = rtw_get_usb_priv(rtwdev);
int i;
+ spin_lock_init(&rtwusb->hiq_lock);
+ rtwusb->hiq_budget = RTW_USB_HIQ_BUDGET_MAX;
+ rtwusb->hiq_refill = jiffies;
+
rtwusb->txwq = create_singlethread_workqueue("rtw88_usb: tx wq");
if (!rtwusb->txwq) {
rtw_err(rtwdev, "failed to create TX work queue\n");
diff --git a/drivers/net/wireless/realtek/rtw88/usb.h b/drivers/net/wireless/realtek/rtw88/usb.h
index 86697a5c0103..36a57a0e0df6 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.h
+++ b/drivers/net/wireless/realtek/rtw88/usb.h
@@ -74,6 +74,11 @@ struct rtw_usb {
u8 out_ep[RTW_USB_EP_MAX];
int qsel_to_ep[TX_DESC_QSEL_MAX];
+ /* protects hiq_budget and hiq_refill */
+ spinlock_t hiq_lock;
+ u32 hiq_budget;
+ unsigned long hiq_refill;
+
struct workqueue_struct *txwq, *rxwq;
struct sk_buff_head tx_queue[RTW_USB_EP_MAX];
diff --git a/drivers/net/wireless/rsi/rsi_91x_core.c b/drivers/net/wireless/rsi/rsi_91x_core.c
index ead4d4e04328..3281fc94e107 100644
--- a/drivers/net/wireless/rsi/rsi_91x_core.c
+++ b/drivers/net/wireless/rsi/rsi_91x_core.c
@@ -458,7 +458,7 @@ void rsi_core_xmit(struct rsi_common *common, struct sk_buff *skb)
tx_params->sta_id = 0;
}
- if (rsta) {
+ if (rsta && tid < IEEE80211_NUM_TIDS) {
/* Start aggregation if not done for this tid */
if (!rsta->start_tx_aggr[tid]) {
rsta->start_tx_aggr[tid] = true;
diff --git a/drivers/nvme/common/keyring.c b/drivers/nvme/common/keyring.c
index ed5167f942d8..9f8c72c7bba7 100644
--- a/drivers/nvme/common/keyring.c
+++ b/drivers/nvme/common/keyring.c
@@ -36,6 +36,7 @@ struct key *nvme_tls_key_lookup(key_serial_t key_id)
}
if (nvme_tls_psk_revoked(key)) {
pr_err("key id %08x revoked\n", key_id);
+ key_put(key);
return ERR_PTR(-EKEYREVOKED);
}
return key;
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index cf79a6c45ccd..37dc942af0d2 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4375,9 +4375,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
mutex_lock(&ctrl->namespaces_lock);
- list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu);
+ list_splice_init_srcu(&ctrl->namespaces, &ns_list, &ctrl->srcu);
mutex_unlock(&ctrl->namespaces_lock);
- synchronize_srcu(&ctrl->srcu);
list_for_each_entry_safe(ns, next, &ns_list, list)
nvme_ns_remove(ns);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index df08ebe13ea0..8cc6023e478e 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -3683,6 +3683,8 @@ static const struct pci_device_id nvme_id_table[] = {
.driver_data = NVME_QUIRK_BOGUS_NID, },
{ PCI_DEVICE(0x1344, 0x6004), /* Micron 4100AT */
.driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
+ { PCI_DEVICE(0x1e0f, 0x0032), /* KIOXIA XG10d */
+ .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
{ PCI_DEVICE(0x1c5c, 0x1504), /* SK Hynix PC400 */
.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
{ PCI_DEVICE(0x1c5c, 0x174a), /* SK Hynix P31 SSD */
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 67d02cfb47ec..951506216714 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -6,6 +6,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/jiffies.h>
#include <linux/slab.h>
#include <linux/err.h>
#include <linux/key.h>
@@ -47,13 +48,34 @@ MODULE_PARM_DESC(wq_unbound, "Use unbound workqueue for nvme-tcp IO context (def
/*
* TLS handshake timeout
*/
-static int tls_handshake_timeout = 10;
+static unsigned int tls_handshake_timeout = 10;
#ifdef CONFIG_NVME_TCP_TLS
-module_param(tls_handshake_timeout, int, 0644);
+static int tls_handshake_timeout_set(const char *val,
+ const struct kernel_param *kp)
+{
+ return param_set_uint_minmax(val, kp, 0,
+ min_t(unsigned long, UINT_MAX / MSEC_PER_SEC,
+ MAX_JIFFY_OFFSET / HZ));
+}
+
+static const struct kernel_param_ops tls_handshake_timeout_ops = {
+ .set = tls_handshake_timeout_set,
+ .get = param_get_uint,
+};
+
+module_param_cb(tls_handshake_timeout, &tls_handshake_timeout_ops,
+ &tls_handshake_timeout, 0644);
+__MODULE_PARM_TYPE(tls_handshake_timeout, "uint");
MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
+/* read a TLS control record in chunks of this size */
+#define NVME_TCP_TLS_REC_CHUNK 256
+
+/* TLS records read while waiting for the ICResp, the ICResp included */
+#define NVME_TCP_ICRESP_MAX_RECORDS 8
+
static atomic_t nvme_tcp_cpu_queues[NR_CPUS];
#ifdef CONFIG_DEBUG_LOCK_ALLOC
@@ -1446,9 +1468,11 @@ static void nvme_tcp_io_work(struct work_struct *w)
do {
bool pending = false;
+ bool got_send_mutex = false;
int result;
if (mutex_trylock(&queue->send_mutex)) {
+ got_send_mutex = true;
result = nvme_tcp_try_send(queue);
mutex_unlock(&queue->send_mutex);
if (result > 0)
@@ -1463,8 +1487,13 @@ static void nvme_tcp_io_work(struct work_struct *w)
else if (unlikely(result < 0))
return;
- /* did we get some space after spending time in recv? */
- if (nvme_tcp_queue_has_pending(queue) &&
+ /*
+ * did we get some space after spending time in recv?
+ * Only check this if we could take send_mutex. Otherwise
+ * the task holding it sends the requests and queues io_work
+ * again after it releases the mutex.
+ */
+ if (got_send_mutex && nvme_tcp_queue_has_pending(queue) &&
sk_stream_is_writeable(queue->sock->sk))
pending = true;
@@ -1562,12 +1591,143 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *nctrl, int qid)
mutex_destroy(&queue->pf_cache_lock);
}
+static int nvme_tcp_recv_tls_alert(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, int len)
+{
+ struct sock *sk = queue->sock->sk;
+ u8 level, description;
+
+ if (len < 2) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: truncated TLS alert\n",
+ nvme_tcp_queue_id(queue));
+ return -ENOTCONN;
+ }
+
+ /* tls_alert_recv() reads through the iterator */
+ iov_iter_revert(&msg->msg_iter, len);
+ tls_alert_recv(sk, msg, &level, &description);
+
+ if (level == TLS_ALERT_LEVEL_FATAL) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: fatal TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+ return -ENOTCONN;
+ }
+
+ /* a peer shutting down normally is not worth a warning */
+ if (description == TLS_ALERT_DESC_CLOSE_NOTIFY)
+ dev_dbg(queue->ctrl->ctrl.device,
+ "queue %d: TLS close_notify\n",
+ nvme_tcp_queue_id(queue));
+ else
+ dev_warn(queue->ctrl->ctrl.device,
+ "queue %d: TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+
+ return 0;
+}
+
+/*
+ * Discard a TLS record that arrived in place of the ICResp. A peer may send
+ * a NewSessionTicket as soon as the handshake completes.
+ *
+ * Returns 0 when the record was discarded and the ICResp can be read, or a
+ * negative errno if the connection cannot continue.
+ */
+static int nvme_tcp_skip_tls_record(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, u8 ctype, int len)
+{
+ u8 buf[NVME_TCP_TLS_REC_CHUNK];
+ int ret;
+
+ switch (ctype) {
+ case TLS_RECORD_TYPE_HANDSHAKE:
+ break;
+ case TLS_RECORD_TYPE_ALERT:
+ ret = nvme_tcp_recv_tls_alert(queue, msg, len);
+ return ret ? : -ENOTCONN;
+ default:
+ pr_err("queue %d: unhandled TLS record %u\n",
+ nvme_tcp_queue_id(queue), ctype);
+ return -ENOTCONN;
+ }
+
+ /* the ICResp is in a later record, drain this one first */
+ while (!(msg->msg_flags & MSG_EOR)) {
+ struct kvec iov = {
+ .iov_base = buf,
+ .iov_len = sizeof(buf),
+ };
+ char cbuf[CMSG_LEN(sizeof(char))] = {};
+ /* kTLS fails a control record read without a cmsg buffer */
+ struct msghdr drain = {
+ .msg_control = cbuf,
+ .msg_controllen = sizeof(cbuf),
+ };
+
+ ret = kernel_recvmsg(queue->sock, &drain, &iov, 1, sizeof(buf),
+ 0);
+ if (ret <= 0) {
+ pr_warn("queue %d: failed to drain TLS record: %d\n",
+ nvme_tcp_queue_id(queue), ret);
+ return ret ? : -ECONNRESET;
+ }
+ msg->msg_flags = drain.msg_flags;
+ }
+
+ return 0;
+}
+
+/*
+ * Receive the ICResp. Over TLS, skip the records that a peer may send
+ * ahead of it.
+ *
+ * Returns the length received, or a negative errno.
+ */
+static int nvme_tcp_recv_icresp(struct nvme_tcp_queue *queue,
+ struct nvme_tcp_icresp_pdu *icresp)
+{
+ char cbuf[CMSG_LEN(sizeof(char))];
+ struct msghdr msg;
+ struct kvec iov;
+ u8 ctype;
+ int i, ret;
+
+ for (i = 0; i < NVME_TCP_ICRESP_MAX_RECORDS; i++) {
+ memset(&msg, 0, sizeof(msg));
+ memset(cbuf, 0, sizeof(cbuf));
+ iov.iov_base = icresp;
+ iov.iov_len = sizeof(*icresp);
+ if (nvme_tcp_queue_tls(queue)) {
+ msg.msg_control = cbuf;
+ msg.msg_controllen = sizeof(cbuf);
+ }
+ msg.msg_flags = MSG_WAITALL;
+ ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
+ iov.iov_len, msg.msg_flags);
+ if (ret <= 0 || !nvme_tcp_queue_tls(queue))
+ return ret;
+
+ ctype = tls_get_record_type(queue->sock->sk,
+ (struct cmsghdr *)cbuf);
+ if (ctype == TLS_RECORD_TYPE_DATA)
+ return ret;
+
+ ret = nvme_tcp_skip_tls_record(queue, &msg, ctype, ret);
+ if (ret < 0)
+ return ret;
+ }
+
+ pr_err("queue %d: too many TLS records before the ICResp\n",
+ nvme_tcp_queue_id(queue));
+ return -EPROTO;
+}
+
static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
{
struct nvme_tcp_icreq_pdu *icreq;
struct nvme_tcp_icresp_pdu *icresp;
- char cbuf[CMSG_LEN(sizeof(char))] = {};
- u8 ctype;
struct msghdr msg = {};
struct kvec iov;
bool ctrl_hdgst, ctrl_ddgst;
@@ -1605,16 +1765,7 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
goto free_icresp;
}
- memset(&msg, 0, sizeof(msg));
- iov.iov_base = icresp;
- iov.iov_len = sizeof(*icresp);
- if (nvme_tcp_queue_tls(queue)) {
- msg.msg_control = cbuf;
- msg.msg_controllen = sizeof(cbuf);
- }
- msg.msg_flags = MSG_WAITALL;
- ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
- iov.iov_len, msg.msg_flags);
+ ret = nvme_tcp_recv_icresp(queue, icresp);
if (ret >= 0 && ret < sizeof(*icresp))
ret = -ECONNRESET;
if (ret < 0) {
@@ -1622,16 +1773,6 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
nvme_tcp_queue_id(queue), ret);
goto free_icresp;
}
- ret = -ENOTCONN;
- if (nvme_tcp_queue_tls(queue)) {
- ctype = tls_get_record_type(queue->sock->sk,
- (struct cmsghdr *)cbuf);
- if (ctype != TLS_RECORD_TYPE_DATA) {
- pr_err("queue %d: unhandled TLS record %d\n",
- nvme_tcp_queue_id(queue), ctype);
- goto free_icresp;
- }
- }
ret = -EINVAL;
if (icresp->hdr.type != nvme_tcp_icresp) {
pr_err("queue %d: bad type returned %d\n",
@@ -1821,8 +1962,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
{
int qid = nvme_tcp_queue_id(queue);
int ret;
+ long wait_ret;
struct tls_handshake_args args;
- unsigned long tmo = tls_handshake_timeout * HZ;
+ unsigned int timeout = READ_ONCE(tls_handshake_timeout);
+ unsigned long tmo = (unsigned long)timeout * HZ;
key_serial_t keyring = nvme_keyring_id();
dev_dbg(nctrl->device, "queue %d: start TLS with key %x\n",
@@ -1836,7 +1979,7 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
if (nctrl->opts->keyring)
keyring = key_serial(nctrl->opts->keyring);
args.ta_keyring = keyring;
- args.ta_timeout_ms = tls_handshake_timeout * 1000;
+ args.ta_timeout_ms = timeout * MSEC_PER_SEC;
queue->tls_err = -EOPNOTSUPP;
init_completion(&queue->tls_complete);
ret = tls_client_hello_psk(&args, GFP_KERNEL);
@@ -1845,10 +1988,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
qid, ret);
return ret;
}
- ret = wait_for_completion_interruptible_timeout(&queue->tls_complete, tmo);
- if (ret <= 0) {
- if (ret == 0)
- ret = -ETIMEDOUT;
+ wait_ret = wait_for_completion_interruptible_timeout(&queue->tls_complete,
+ tmo);
+ if (wait_ret <= 0) {
+ ret = wait_ret < 0 ? -ERESTARTSYS : -ETIMEDOUT;
dev_err(nctrl->device,
"queue %d: TLS handshake failed, error %d\n",
diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index c8ff53faddbc..1cc2f57b53ff 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -282,6 +282,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req)
u16 ngrps = 0;
u16 status;
+ /*
+ * The header and one descriptor per enabled group are the only bytes
+ * written below; zero the whole transfer first so the unwritten tail
+ * does not leak stale kernel memory to the host.
+ */
+ status = nvmet_zero_sgl(req, 0, req->transfer_len);
+ if (status)
+ goto out;
+
status = NVME_SC_INTERNAL;
desc = kmalloc(struct_size(desc, nsids, NVMET_MAX_NAMESPACES),
GFP_KERNEL);
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 3b066601fd35..0e12f800beb3 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -148,7 +148,7 @@ static ssize_t nvmet_addr_traddr_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.traddr) != 1)
+ if (sscanf(page, "%255s", port->disc_addr.traddr) != 1)
return -EINVAL;
return count;
}
@@ -240,7 +240,7 @@ static ssize_t nvmet_addr_trsvcid_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.trsvcid) != 1)
+ if (sscanf(page, "%31s", port->disc_addr.trsvcid) != 1)
return -EINVAL;
return count;
}
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index 01b7722afb5e..290b51fef0b9 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -864,7 +864,8 @@ static u16 nvmet_rdma_map_sgl_inline(struct nvmet_rdma_rsp *rsp)
return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
}
- if (off + len > rsp->queue->dev->inline_data_size) {
+ if (off > (u64)rsp->queue->dev->inline_data_size ||
+ len > (u64)rsp->queue->dev->inline_data_size - off) {
pr_err("invalid inline data offset!\n");
return NVME_SC_SGL_INVALID_OFFSET | NVME_STATUS_DNR;
}
diff --git a/drivers/nvmem/rockchip-otp.c b/drivers/nvmem/rockchip-otp.c
index d88f12c53242..1475ce11a9c7 100644
--- a/drivers/nvmem/rockchip-otp.c
+++ b/drivers/nvmem/rockchip-otp.c
@@ -12,6 +12,7 @@
#include <linux/io.h>
#include <linux/iopoll.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/nvmem-provider.h>
#include <linux/reset.h>
#include <linux/slab.h>
@@ -77,9 +78,11 @@ struct rockchip_data {
struct rockchip_otp {
struct device *dev;
void __iomem *base;
- struct clk_bulk_data *clks;
struct reset_control *rst;
const struct rockchip_data *data;
+ /* Serializes access to the OTP controller state machine */
+ struct mutex mutex;
+ struct clk_bulk_data clks[];
};
static int rockchip_otp_reset(struct rockchip_otp *otp)
@@ -239,16 +242,21 @@ static int rockchip_otp_read(void *context, unsigned int offset,
if (!otp->data || !otp->data->reg_read)
return -EINVAL;
+ mutex_lock(&otp->mutex);
+
ret = clk_bulk_prepare_enable(otp->data->num_clks, otp->clks);
if (ret < 0) {
dev_err(otp->dev, "failed to prepare/enable clks\n");
- return ret;
+ goto unlock;
}
ret = otp->data->reg_read(context, offset, val, bytes);
clk_bulk_disable_unprepare(otp->data->num_clks, otp->clks);
+unlock:
+ mutex_unlock(&otp->mutex);
+
return ret;
}
@@ -327,23 +335,23 @@ static int rockchip_otp_probe(struct platform_device *pdev)
if (!data)
return dev_err_probe(dev, -EINVAL, "failed to get match data\n");
- otp = devm_kzalloc(&pdev->dev, sizeof(struct rockchip_otp),
+ otp = devm_kzalloc(&pdev->dev, struct_size(otp, clks, data->num_clks),
GFP_KERNEL);
if (!otp)
return -ENOMEM;
otp->data = data;
otp->dev = dev;
+
+ ret = devm_mutex_init(dev, &otp->mutex);
+ if (ret)
+ return ret;
+
otp->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(otp->base))
return dev_err_probe(dev, PTR_ERR(otp->base),
"failed to ioremap resource\n");
- otp->clks = devm_kcalloc(dev, data->num_clks, sizeof(*otp->clks),
- GFP_KERNEL);
- if (!otp->clks)
- return -ENOMEM;
-
for (i = 0; i < data->num_clks; ++i)
otp->clks[i].id = data->clks[i];
diff --git a/drivers/pci/controller/cadence/pcie-cadence-plat.c b/drivers/pci/controller/cadence/pcie-cadence-plat.c
index 0456845dabb9..c1ec619ab0a4 100644
--- a/drivers/pci/controller/cadence/pcie-cadence-plat.c
+++ b/drivers/pci/controller/cadence/pcie-cadence-plat.c
@@ -14,14 +14,6 @@
#define CDNS_PLAT_CPU_TO_BUS_ADDR 0x0FFFFFFF
-/**
- * struct cdns_plat_pcie - private data for this PCIe platform driver
- * @pcie: Cadence PCIe controller
- */
-struct cdns_plat_pcie {
- struct cdns_pcie *pcie;
-};
-
struct cdns_plat_pcie_of_data {
bool is_rc;
};
@@ -40,9 +32,9 @@ static const struct cdns_pcie_ops cdns_plat_ops = {
static int cdns_plat_pcie_probe(struct platform_device *pdev)
{
const struct cdns_plat_pcie_of_data *data;
- struct cdns_plat_pcie *cdns_plat_pcie;
struct device *dev = &pdev->dev;
struct pci_host_bridge *bridge;
+ struct cdns_pcie *pcie;
struct cdns_pcie_ep *ep;
struct cdns_pcie_rc *rc;
int phy_count;
@@ -56,11 +48,6 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
is_rc = data->is_rc;
pr_debug(" Started %s with is_rc: %d\n", __func__, is_rc);
- cdns_plat_pcie = devm_kzalloc(dev, sizeof(*cdns_plat_pcie), GFP_KERNEL);
- if (!cdns_plat_pcie)
- return -ENOMEM;
-
- platform_set_drvdata(pdev, cdns_plat_pcie);
if (is_rc) {
if (!IS_ENABLED(CONFIG_PCIE_CADENCE_PLAT_HOST))
return -ENODEV;
@@ -72,9 +59,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
rc = pci_host_bridge_priv(bridge);
rc->pcie.dev = dev;
rc->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &rc->pcie;
+ pcie = &rc->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -99,9 +86,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
ep->pcie.dev = dev;
ep->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &ep->pcie;
+ pcie = &ep->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -119,18 +106,20 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
goto err_init;
}
+ platform_set_drvdata(pdev, pcie);
+
return 0;
err_init:
err_get_sync:
pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
- cdns_pcie_disable_phy(cdns_plat_pcie->pcie);
- phy_count = cdns_plat_pcie->pcie->phy_count;
+ cdns_pcie_disable_phy(pcie);
+ phy_count = pcie->phy_count;
while (phy_count--)
- device_link_del(cdns_plat_pcie->pcie->link[phy_count]);
+ device_link_del(pcie->link[phy_count]);
- return 0;
+ return ret;
}
static void cdns_plat_pcie_shutdown(struct platform_device *pdev)
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index bc3d6269f33b..1dad7d69de91 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -514,6 +514,8 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
/* The data-less messages only for now */
val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code;
}
+ if (dw_pcie_ver_is_ge(pci, 460A))
+ val |= PCIE_ATU_DMA_BYPASS;
dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val);
/*
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index a5f58ac8ea94..728f15cebf9e 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -173,6 +173,7 @@
#define PCIE_ATU_REGION_CTRL2 0x004
#define PCIE_ATU_ENABLE BIT(31)
#define PCIE_ATU_BAR_MODE_ENABLE BIT(30)
+#define PCIE_ATU_DMA_BYPASS BIT(27)
#define PCIE_ATU_INHIBIT_PAYLOAD BIT(22)
#define PCIE_ATU_FUNC_NUM_MATCH_EN BIT(19)
#define PCIE_ATU_LOWER_BASE 0x008
diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
index 8dcb63f813be..a5f750709e2c 100644
--- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -305,6 +305,57 @@ static struct rcar_gen4_pcie *rcar_gen4_pcie_alloc(struct platform_device *pdev)
return rcar;
}
+static int rcar_gen4_pcie_enable_device(struct pci_host_bridge *bridge,
+ struct pci_dev *dev)
+{
+ /*
+ * R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
+ * Max_Payload_Size (MPS). PCIe specification indicates that the MPS
+ * must not exceed minimum MPS of any element along the packet path.
+ * The controller reports Max_Payload_Size_Supported (MPSS) 256 Bytes
+ * for header type 0 and 128 Bytes for header type 1. The PCIe core
+ * will not allow MPS to be set higher than MPSS; warn here in case
+ * something went very wrong in the core.
+ *
+ * For details, refer to chapter "104.1.1 Features" in either of:
+ * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+ * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+ * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+ */
+ WARN_ON(pcie_get_mps(dev) > 256);
+
+ /*
+ * R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
+ * MRRS (Max Read Request Size) states:
+ *
+ * Please set "Max Read Request Size" to 128 bytes or 256 bytes.
+ * If "Max Read Request Size" is set to anything other than the
+ * above, the transferred data will not match the expected value.
+ *
+ * This limitation also seems to apply to devices issuing MRd TLPs.
+ * This limitation can be triggered by using non-HMB NVMe SSD with
+ * Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
+ * Any write into the SSD (MRd TLP issued by the SSD) longer than
+ * 256 Bytes wraps around at 256 Byte boundary, and the same data
+ * are written into the SSD starting at offset 0 and at 256 Bytes.
+ *
+ * Limit Max_Read_Request_Size to at most 256 Bytes for each
+ * device connected to this PCIe controller to avoid this behavior.
+ *
+ * For details, refer to aforementioned chapter in either of:
+ * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+ * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+ * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+ */
+ bridge->no_inc_mrrs = 1;
+ if (pcie_get_readrq(dev) > 256) {
+ pci_info(dev, "Limiting MRRS to 256 bytes\n");
+ pcie_set_readrq(dev, 256);
+ }
+
+ return 0;
+}
+
/* Host mode */
static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
{
@@ -313,6 +364,9 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
int ret;
u32 val;
+ if (pp->bridge)
+ pp->bridge->enable_device = rcar_gen4_pcie_enable_device;
+
gpiod_set_value_cansleep(dw->pe_rst, 1);
ret = rcar_gen4_pcie_common_init(rcar);
diff --git a/drivers/pci/controller/pcie-rockchip.c b/drivers/pci/controller/pcie-rockchip.c
index c07d7129f1c7..6a970dc766d4 100644
--- a/drivers/pci/controller/pcie-rockchip.c
+++ b/drivers/pci/controller/pcie-rockchip.c
@@ -66,8 +66,10 @@ int rockchip_pcie_parse_dt(struct rockchip_pcie *rockchip)
}
rockchip->link_gen = of_pci_get_max_link_speed(node);
- if (rockchip->link_gen < 0 || rockchip->link_gen > 2)
- rockchip->link_gen = 2;
+ if (rockchip->link_gen < 0 || rockchip->link_gen >= 2) {
+ rockchip->link_gen = 1;
+ dev_warn(dev, "invalid max-link-speed, limited to 2.5 GT/s\n");
+ }
rockchip->core_rst = devm_reset_control_get_exclusive(dev, "core");
if (IS_ERR(rockchip->core_rst)) {
@@ -237,12 +239,8 @@ int rockchip_pcie_init_port(struct rockchip_pcie *rockchip)
goto err_exit_phy;
}
- if (rockchip->link_gen == 2)
- rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_2,
- PCIE_CLIENT_CONFIG);
- else
- rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1,
- PCIE_CLIENT_CONFIG);
+ rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1,
+ PCIE_CLIENT_CONFIG);
regs = PCIE_CLIENT_LINK_TRAIN_ENABLE | PCIE_CLIENT_ARI_ENABLE |
PCIE_CLIENT_CONF_LANE_NUM(rockchip->lanes);
diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h
index d916fcc8badb..5629ac4a7439 100644
--- a/drivers/pci/controller/pcie-rockchip.h
+++ b/drivers/pci/controller/pcie-rockchip.h
@@ -37,7 +37,6 @@
#define PCIE_CLIENT_MODE_RC HIWORD_UPDATE_BIT(0x0040)
#define PCIE_CLIENT_MODE_EP HIWORD_UPDATE(0x0040, 0)
#define PCIE_CLIENT_GEN_SEL_1 HIWORD_UPDATE(0x0080, 0)
-#define PCIE_CLIENT_GEN_SEL_2 HIWORD_UPDATE_BIT(0x0080)
#define PCIE_CLIENT_LEGACY_INT_CTRL (PCIE_CLIENT_BASE + 0x0c)
#define PCIE_CLIENT_INT_IN_ASSERT HIWORD_UPDATE_BIT(0x0002)
#define PCIE_CLIENT_INT_IN_DEASSERT HIWORD_UPDATE(0x0002, 0)
@@ -183,8 +182,6 @@
(((x) & PCIE_CLIENT_DEBUG_LTSSM_MASK) == PCIE_CLIENT_DEBUG_LTSSM_L2)
#define PCIE_LINK_UP(x) \
(((x) & PCIE_CLIENT_LINK_STATUS_MASK) == PCIE_CLIENT_LINK_STATUS_UP)
-#define PCIE_LINK_IS_GEN2(x) \
- (((x) & PCIE_CORE_PL_CONF_SPEED_MASK) == PCIE_CORE_PL_CONF_SPEED_5G)
#define RC_REGION_0_ADDR_TRANS_H 0x00000000
#define RC_REGION_0_ADDR_TRANS_L 0x00000000
diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
index 963a1a1f983d..710bddc30cdf 100644
--- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
@@ -116,7 +116,7 @@ struct epf_ntb_ctrl {
} __packed;
struct epf_ntb {
- struct ntb_dev ntb;
+ struct ntb_dev *ntb;
struct pci_epf *epf;
struct config_group group;
@@ -142,10 +142,15 @@ struct epf_ntb {
void __iomem *vpci_mw_addr[MAX_MW];
struct delayed_work cmd_handler;
+ struct pci_bus *vpci_bus;
};
#define to_epf_ntb(epf_group) container_of((epf_group), struct epf_ntb, group)
-#define ntb_ndev(__ntb) container_of(__ntb, struct epf_ntb, ntb)
+
+static struct epf_ntb *ntb_ndev(struct ntb_dev *ntb)
+{
+ return ntb->pdev->sysdata;
+}
static struct pci_epf_header epf_ntb_header = {
.vendorid = PCI_ANY_ID,
@@ -171,7 +176,7 @@ static int epf_ntb_link_up(struct epf_ntb *ntb, bool link_up)
else
ntb->reg->link_status &= ~LINK_STATUS_UP;
- ntb_link_event(&ntb->ntb);
+ ntb_link_event(ntb->ntb);
return 0;
}
@@ -259,7 +264,7 @@ static void epf_ntb_cmd_handler(struct work_struct *work)
for (i = 1; i < ntb->db_count; i++) {
if (ntb->epf_db[i]) {
ntb->db |= 1 << (i - 1);
- ntb_db_event(&ntb->ntb, i);
+ ntb_db_event(ntb->ntb, i);
ntb->epf_db[i] = 0;
}
}
@@ -760,7 +765,6 @@ static int epf_ntb_epc_init(struct epf_ntb *ntb)
}
INIT_DELAYED_WORK(&ntb->cmd_handler, epf_ntb_cmd_handler);
- queue_work(kpcintb_workqueue, &ntb->cmd_handler.work);
return 0;
@@ -1009,6 +1013,7 @@ static int vpci_scan_bus(void *sysdata)
pr_err("create pci bus failed\n");
return -EINVAL;
}
+ ndev->vpci_bus = vpci_bus;
pci_bus_add_devices(vpci_bus);
@@ -1053,7 +1058,7 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
int ret;
struct device *dev;
- dev = &ntb->ntb.dev;
+ dev = &ndev->dev;
barno = ntb->epf_ntb_bar[BAR_MW0 + idx];
epf_bar = &ntb->epf->bar[barno];
epf_bar->phys_addr = addr;
@@ -1153,7 +1158,7 @@ static int vntb_epf_peer_db_set(struct ntb_dev *ndev, u64 db_bits)
ret = pci_epc_raise_irq(ntb->epf->epc, func_no, vfunc_no,
PCI_IRQ_MSI, interrupt_num + 1);
if (ret)
- dev_err(&ntb->ntb.dev, "Failed to raise IRQ\n");
+ dev_err(&ntb->epf->dev, "Failed to raise IRQ\n");
return ret;
}
@@ -1236,13 +1241,18 @@ static const struct ntb_dev_ops vntb_epf_ops = {
static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
- int ret;
struct epf_ntb *ndev = (struct epf_ntb *)pdev->sysdata;
struct device *dev = &pdev->dev;
+ struct ntb_dev *ntb;
+ int ret;
- ndev->ntb.pdev = pdev;
- ndev->ntb.topo = NTB_TOPO_NONE;
- ndev->ntb.ops = &vntb_epf_ops;
+ ntb = devm_kzalloc(dev, sizeof(*ntb), GFP_KERNEL);
+ if (!ntb)
+ return -ENOMEM;
+
+ ntb->pdev = pdev;
+ ntb->topo = NTB_TOPO_NONE;
+ ntb->ops = &vntb_epf_ops;
ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
if (ret) {
@@ -1250,16 +1260,33 @@ static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
return ret;
}
- ret = ntb_register_device(&ndev->ntb);
+ ret = ntb_register_device(ntb);
if (ret) {
dev_err(dev, "Failed to register NTB device\n");
return ret;
}
+ /* Publish after ntb_register_device() succeeds */
+ smp_store_release(&ndev->ntb, ntb);
+ queue_delayed_work(kpcintb_workqueue, &ndev->cmd_handler, 0);
+
dev_dbg(dev, "PCI Virtual NTB driver loaded\n");
return 0;
}
+static void pci_vntb_remove(struct pci_dev *pdev)
+{
+ struct epf_ntb *ndev = pdev->sysdata;
+ struct ntb_dev *ntb;
+
+ /* Stop the work reader */
+ cancel_delayed_work_sync(&ndev->cmd_handler);
+
+ ntb = xchg(&ndev->ntb, NULL);
+
+ ntb_unregister_device(ntb);
+}
+
static struct pci_device_id pci_vntb_table[] = {
{
PCI_DEVICE(0xffff, 0xffff),
@@ -1271,6 +1298,7 @@ static struct pci_driver vntb_pci_driver = {
.name = "pci-vntb",
.id_table = pci_vntb_table,
.probe = pci_vntb_probe,
+ .remove = pci_vntb_remove,
};
/* ============ PCIe EPF Driver Bind ====================*/
@@ -1351,12 +1379,25 @@ static int epf_ntb_bind(struct pci_epf *epf)
*/
static void epf_ntb_unbind(struct pci_epf *epf)
{
+ struct pci_host_bridge *bridge;
struct epf_ntb *ntb = epf_get_drvdata(epf);
+ pci_unregister_driver(&vntb_pci_driver);
+
+ if (ntb->vpci_bus) {
+ bridge = to_pci_host_bridge(ntb->vpci_bus->bridge);
+
+ pci_lock_rescan_remove();
+ pci_stop_root_bus(ntb->vpci_bus);
+ pci_remove_root_bus(ntb->vpci_bus);
+ ntb->vpci_bus = NULL;
+ pci_unlock_rescan_remove();
+
+ pci_free_host_bridge(bridge);
+ }
+
epf_ntb_epc_cleanup(ntb);
epf_ntb_config_spad_bar_free(ntb);
-
- pci_unregister_driver(&vntb_pci_driver);
}
// EPF driver probe
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 2f575af94e09..b53ee0920bdc 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2090,6 +2090,28 @@ int __weak pcibios_enable_device(struct pci_dev *dev, int bars)
return pci_enable_resources(dev, bars);
}
+static int pci_host_bridge_enable_device(struct pci_dev *dev)
+{
+ struct pci_host_bridge *host_bridge = pci_find_host_bridge(dev->bus);
+ int err;
+
+ if (host_bridge && host_bridge->enable_device) {
+ err = host_bridge->enable_device(host_bridge, dev);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static void pci_host_bridge_disable_device(struct pci_dev *dev)
+{
+ struct pci_host_bridge *host_bridge = pci_find_host_bridge(dev->bus);
+
+ if (host_bridge && host_bridge->disable_device)
+ host_bridge->disable_device(host_bridge, dev);
+}
+
static int do_pci_enable_device(struct pci_dev *dev, int bars)
{
int err;
@@ -2105,9 +2127,13 @@ static int do_pci_enable_device(struct pci_dev *dev, int bars)
if (bridge)
pcie_aspm_powersave_config_link(bridge);
+ err = pci_host_bridge_enable_device(dev);
+ if (err)
+ return err;
+
err = pcibios_enable_device(dev, bars);
if (err < 0)
- return err;
+ goto err_enable;
pci_fixup_device(pci_fixup_enable, dev);
if (dev->msi_enabled || dev->msix_enabled)
@@ -2122,6 +2148,12 @@ static int do_pci_enable_device(struct pci_dev *dev, int bars)
}
return 0;
+
+err_enable:
+ pci_host_bridge_disable_device(dev);
+
+ return err;
+
}
/**
@@ -2305,6 +2337,8 @@ void pci_disable_device(struct pci_dev *dev)
if (atomic_dec_return(&dev->enable_cnt) != 0)
return;
+ pci_host_bridge_disable_device(dev);
+
do_pci_disable_device(dev);
dev->is_busmaster = 0;
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 8cfa7a3f55cb..5b266583acb5 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -93,8 +93,12 @@ int pcie_failed_link_retrain(struct pci_dev *dev)
pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &lnkctl2);
pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta);
oldlnkctl2 = lnkctl2;
- if ((lnksta & (PCI_EXP_LNKSTA_LBMS | PCI_EXP_LNKSTA_DLLLA)) ==
- PCI_EXP_LNKSTA_LBMS) {
+ if (lnksta & PCI_EXP_LNKSTA_DLLLA) {
+ ;
+ } else if ((lnkctl2 & PCI_EXP_LNKCTL2_TLS) ==
+ PCI_EXP_LNKCTL2_TLS_2_5GT) {
+ return ret;
+ } else if (lnksta & PCI_EXP_LNKSTA_LBMS) {
pci_info(dev, "broken device, retraining non-functional downstream link at 2.5GT/s\n");
lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS;
diff --git a/drivers/platform/x86/amd/pmf/core.c b/drivers/platform/x86/amd/pmf/core.c
index 496fea1389fc..2039c9d90f87 100644
--- a/drivers/platform/x86/amd/pmf/core.c
+++ b/drivers/platform/x86/amd/pmf/core.c
@@ -467,6 +467,7 @@ static const struct acpi_device_id amd_pmf_acpi_ids[] = {
{"AMDI0105", 0},
{"AMDI0107", 0},
{"AMDI0108", 0},
+ {"AMDI0112", 0},
{ }
};
MODULE_DEVICE_TABLE(acpi, amd_pmf_acpi_ids);
diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c
index 1465057b3e3d..e71d2be12e78 100644
--- a/drivers/platform/x86/asus-wmi.c
+++ b/drivers/platform/x86/asus-wmi.c
@@ -15,6 +15,7 @@
#include <linux/acpi.h>
#include <linux/backlight.h>
+#include <linux/cleanup.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
#include <linux/dmi.h>
@@ -27,6 +28,7 @@
#include <linux/leds.h>
#include <linux/minmax.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/pci.h>
#include <linux/pci_hotplug.h>
#include <linux/platform_data/x86/asus-wmi.h>
@@ -337,6 +339,20 @@ struct asus_wmi {
/* WMI ************************************************************************/
+/*
+ * Concurrent evaluations of ASUS WMI methods can re-enter firmware
+ * SMI/EC mailbox handling and corrupt the mailbox.
+ */
+static DEFINE_MUTEX(asus_wmi_eval_lock);
+
+static acpi_status asus_wmi_evaluate_method_locked(u32 method_id,
+ struct acpi_buffer *input,
+ struct acpi_buffer *output)
+{
+ guard(mutex)(&asus_wmi_eval_lock);
+ return wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id, input, output);
+}
+
static int asus_wmi_evaluate_method3(u32 method_id,
u32 arg0, u32 arg1, u32 arg2, u32 *retval)
{
@@ -351,8 +367,7 @@ static int asus_wmi_evaluate_method3(u32 method_id,
union acpi_object *obj;
u32 tmp = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1, arg2);
@@ -403,8 +418,7 @@ static int asus_wmi_evaluate_method5(u32 method_id,
union acpi_object *obj;
u32 tmp = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x, 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1, arg2, arg3, arg4);
@@ -451,8 +465,7 @@ static int asus_wmi_evaluate_method_buf(u32 method_id,
union acpi_object *obj;
int err = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1);
@@ -4632,9 +4645,8 @@ static int show_call(struct seq_file *m, void *data)
union acpi_object *obj;
acpi_status status;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID,
- 0, asus->debug.method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(asus->debug.method_id,
+ &input, &output);
if (ACPI_FAILURE(status))
return -EIO;
diff --git a/drivers/platform/x86/intel/int3472/discrete.c b/drivers/platform/x86/intel/int3472/discrete.c
index 9e69ac9cfb92..06191446e38b 100644
--- a/drivers/platform/x86/intel/int3472/discrete.c
+++ b/drivers/platform/x86/intel/int3472/discrete.c
@@ -237,7 +237,6 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
u8 active_value, pin, type;
union acpi_object *obj;
struct gpio_desc *gpio;
- const char *err_msg;
const char *func;
unsigned long gpio_flags;
int ret;
@@ -283,7 +282,8 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
case INT3472_GPIO_TYPE_POWERDOWN:
ret = skl_int3472_map_gpio_to_sensor(int3472, agpio, func, gpio_flags);
if (ret)
- err_msg = "Failed to map GPIO pin to sensor\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to map GPIO pin to sensor\n");
break;
case INT3472_GPIO_TYPE_CLK_ENABLE:
@@ -292,7 +292,7 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
gpio = skl_int3472_gpiod_get_from_temp_lookup(int3472, agpio, func, gpio_flags);
if (IS_ERR(gpio)) {
ret = PTR_ERR(gpio);
- err_msg = "Failed to get GPIO\n";
+ dev_err_probe(int3472->dev, ret, "Failed to get GPIO\n");
break;
}
@@ -300,19 +300,22 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
case INT3472_GPIO_TYPE_CLK_ENABLE:
ret = skl_int3472_register_gpio_clock(int3472, gpio);
if (ret)
- err_msg = "Failed to register clock\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to register clock\n");
break;
case INT3472_GPIO_TYPE_PRIVACY_LED:
ret = skl_int3472_register_pled(int3472, gpio);
if (ret)
- err_msg = "Failed to register LED\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to register LED\n");
break;
case INT3472_GPIO_TYPE_POWER_ENABLE:
ret = skl_int3472_register_regulator(int3472, gpio);
if (ret)
- err_msg = "Failed to map regulator to sensor\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to map regulator to sensor\n");
break;
default: /* Never reached */
@@ -331,9 +334,6 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
int3472->ngpios++;
ACPI_FREE(obj);
- if (ret < 0)
- return dev_err_probe(int3472->dev, ret, err_msg);
-
return ret;
}
diff --git a/drivers/power/supply/Makefile b/drivers/power/supply/Makefile
index 8dcb41545317..6673860ff26c 100644
--- a/drivers/power/supply/Makefile
+++ b/drivers/power/supply/Makefile
@@ -115,5 +115,5 @@ obj-$(CONFIG_BATTERY_ACER_A500) += acer_a500_battery.o
obj-$(CONFIG_BATTERY_SURFACE) += surface_battery.o
obj-$(CONFIG_CHARGER_SURFACE) += surface_charger.o
obj-$(CONFIG_BATTERY_UG3105) += ug3105_battery.o
-obj-$(CONFIG_CHARGER_QCOM_SMB2) += qcom_pmi8998_charger.o
+obj-$(CONFIG_CHARGER_QCOM_SMB2) += qcom_smbx.o
obj-$(CONFIG_FUEL_GAUGE_MM8013) += mm8013.o
diff --git a/drivers/power/supply/bq27xxx_battery.c b/drivers/power/supply/bq27xxx_battery.c
index 19c73aad769a..59d9e2d99c3f 100644
--- a/drivers/power/supply/bq27xxx_battery.c
+++ b/drivers/power/supply/bq27xxx_battery.c
@@ -43,6 +43,7 @@
*/
#include <linux/device.h>
+#include <linux/devm-helpers.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/param.h>
@@ -2150,26 +2151,17 @@ static void bq27xxx_external_power_changed(struct power_supply *psy)
mod_delayed_work(system_wq, &di->work, HZ / 2);
}
-static void bq27xxx_battery_mutex_destroy(void *data)
-{
- struct mutex *lock = data;
-
- mutex_destroy(lock);
-}
-
int bq27xxx_battery_setup(struct bq27xxx_device_info *di)
{
struct power_supply_desc *psy_desc;
struct power_supply_config psy_cfg = {
.of_node = di->dev->of_node,
.drv_data = di,
+ .no_wakeup_source = true,
};
int ret;
- INIT_DELAYED_WORK(&di->work, bq27xxx_battery_poll);
- mutex_init(&di->lock);
- ret = devm_add_action_or_reset(di->dev, bq27xxx_battery_mutex_destroy,
- &di->lock);
+ ret = devm_mutex_init(di->dev, &di->lock);
if (ret)
return ret;
@@ -2189,7 +2181,13 @@ int bq27xxx_battery_setup(struct bq27xxx_device_info *di)
psy_desc->get_property = bq27xxx_battery_get_property;
psy_desc->external_power_changed = bq27xxx_external_power_changed;
- di->bat = devm_power_supply_register_no_ws(di->dev, psy_desc, &psy_cfg);
+ /* Cancel the poll work after the power_supply is unregistered. */
+ ret = devm_delayed_work_autocancel(di->dev, &di->work,
+ bq27xxx_battery_poll);
+ if (ret)
+ return ret;
+
+ di->bat = devm_power_supply_register(di->dev, psy_desc, &psy_cfg);
if (IS_ERR(di->bat))
return dev_err_probe(di->dev, PTR_ERR(di->bat),
"failed to register battery\n");
@@ -2215,8 +2213,6 @@ void bq27xxx_battery_teardown(struct bq27xxx_device_info *di)
mutex_lock(&di->lock);
di->removed = true;
mutex_unlock(&di->lock);
-
- cancel_delayed_work_sync(&di->work);
}
EXPORT_SYMBOL_GPL(bq27xxx_battery_teardown);
diff --git a/drivers/power/supply/cros_charge-control.c b/drivers/power/supply/cros_charge-control.c
index 9b0a7500296b..2c16ce4cf6b7 100644
--- a/drivers/power/supply/cros_charge-control.c
+++ b/drivers/power/supply/cros_charge-control.c
@@ -14,6 +14,7 @@
#include <linux/platform_data/cros_ec_commands.h>
#include <linux/platform_data/cros_ec_proto.h>
#include <linux/platform_device.h>
+#include <linux/pm.h>
#include <linux/types.h>
#define EC_CHARGE_CONTROL_BEHAVIOURS (BIT(POWER_SUPPLY_CHARGE_BEHAVIOUR_AUTO) | \
@@ -322,6 +323,7 @@ static int cros_chctl_probe(struct platform_device *pdev)
dev_dbg(dev, "Command version: %u\n", (unsigned int)priv->cmd_version);
priv->cros_ec = cros_ec;
+ dev_set_drvdata(dev, priv);
priv->device_attrs[CROS_CHCTL_ATTR_START_THRESHOLD] =
(struct device_attribute)__ATTR_RW(charge_control_start_threshold);
priv->device_attrs[CROS_CHCTL_ATTR_END_THRESHOLD] =
@@ -352,13 +354,38 @@ static int cros_chctl_probe(struct platform_device *pdev)
return devm_battery_hook_register(dev, &priv->battery_hook);
}
+static int cros_chctl_resume(struct device *dev)
+{
+ struct cros_chctl_priv *priv = dev_get_drvdata(dev);
+ int ret;
+
+ /*
+ * The EC resets its charge control state when the AP powers off, so the
+ * configuration is lost across hibernation, while the values exposed to
+ * userspace still describe it. The cached state is the only remaining
+ * record of that configuration: reprogram the EC from it rather than
+ * re-reading the EC, which would adopt the reset defaults instead.
+ */
+ guard(mutex)(&priv->lock);
+ ret = cros_chctl_configure_ec(priv);
+ if (ret)
+ dev_err(dev, "failed to restore charge control state: %d\n", ret);
+
+ return ret;
+}
+
+static DEFINE_SIMPLE_DEV_PM_OPS(cros_chctl_pm_ops, NULL, cros_chctl_resume);
+
static const struct platform_device_id cros_chctl_id[] = {
{ "cros-charge-control", 0 },
{}
};
static struct platform_driver cros_chctl_driver = {
- .driver.name = "cros-charge-control",
+ .driver = {
+ .name = "cros-charge-control",
+ .pm = pm_sleep_ptr(&cros_chctl_pm_ops),
+ },
.probe = cros_chctl_probe,
.id_table = cros_chctl_id,
};
diff --git a/drivers/power/supply/ltc2941-battery-gauge.c b/drivers/power/supply/ltc2941-battery-gauge.c
index a1ddc4b060ce..bf0b268b10bf 100644
--- a/drivers/power/supply/ltc2941-battery-gauge.c
+++ b/drivers/power/supply/ltc2941-battery-gauge.c
@@ -464,9 +464,11 @@ static int ltc294x_i2c_probe(struct i2c_client *client)
/* r_sense can be negative, when sense+ is connected to the battery
* instead of the sense-. This results in reversed measurements. */
ret = of_property_read_u32(np, "lltc,resistor-sense", &r_sense);
- if (ret < 0)
+ if (ret < 0) {
+ of_node_put(np);
return dev_err_probe(&client->dev, ret,
"Could not find lltc,resistor-sense in devicetree\n");
+ }
info->r_sense = r_sense;
ret = of_property_read_u32(np, "lltc,prescaler-exponent",
@@ -477,6 +479,8 @@ static int ltc294x_i2c_probe(struct i2c_client *client)
prescaler_exp = LTC2941_MAX_PRESCALER_EXP;
}
+ of_node_put(np);
+
if (info->id == LTC2943_ID) {
if (prescaler_exp > LTC2943_MAX_PRESCALER_EXP)
prescaler_exp = LTC2943_MAX_PRESCALER_EXP;
diff --git a/drivers/power/supply/mt6370-charger.c b/drivers/power/supply/mt6370-charger.c
index ad8793bf997e..16ffa927147f 100644
--- a/drivers/power/supply/mt6370-charger.c
+++ b/drivers/power/supply/mt6370-charger.c
@@ -761,13 +761,6 @@ static int mt6370_chg_init_psy(struct mt6370_priv *priv)
return PTR_ERR_OR_ZERO(priv->psy);
}
-static void mt6370_chg_destroy_attach_lock(void *data)
-{
- struct mutex *attach_lock = data;
-
- mutex_destroy(attach_lock);
-}
-
static void mt6370_chg_destroy_wq(void *data)
{
struct workqueue_struct *wq = data;
@@ -895,9 +888,7 @@ static int mt6370_chg_probe(struct platform_device *pdev)
if (ret)
return dev_err_probe(dev, ret, "Failed to init psy\n");
- mutex_init(&priv->attach_lock);
- ret = devm_add_action_or_reset(dev, mt6370_chg_destroy_attach_lock,
- &priv->attach_lock);
+ ret = devm_mutex_init(dev, &priv->attach_lock);
if (ret)
return dev_err_probe(dev, ret, "Failed to init attach lock\n");
diff --git a/drivers/power/supply/power_supply_core.c b/drivers/power/supply/power_supply_core.c
index edc7b3bd9b5c..9610ea1c341d 100644
--- a/drivers/power/supply/power_supply_core.c
+++ b/drivers/power/supply/power_supply_core.c
@@ -66,10 +66,9 @@ static bool __power_supply_is_supplied_by(struct power_supply *supplier,
return false;
}
-static int __power_supply_changed_work(struct device *dev, void *data)
+static int __power_supply_changed_work(struct power_supply *pst, void *data)
{
struct power_supply *psy = data;
- struct power_supply *pst = dev_get_drvdata(dev);
if (__power_supply_is_supplied_by(psy, pst)) {
if (pst->desc->external_power_changed)
@@ -98,7 +97,7 @@ static void power_supply_changed_work(struct work_struct *work)
if (likely(psy->changed)) {
psy->changed = false;
spin_unlock_irqrestore(&psy->changed_lock, flags);
- power_supply_for_each_device(psy, __power_supply_changed_work);
+ power_supply_for_each_psy(psy, __power_supply_changed_work);
power_supply_update_leds(psy);
blocking_notifier_call_chain(&power_supply_notifier,
PSY_EVENT_PROP_CHANGED, psy);
@@ -122,6 +121,30 @@ int power_supply_for_each_device(void *data, int (*fn)(struct device *dev, void
}
EXPORT_SYMBOL_GPL(power_supply_for_each_device);
+struct psy_for_each_psy_cb_data {
+ int (*fn)(struct power_supply *psy, void *data);
+ void *data;
+};
+
+static int psy_for_each_psy_cb(struct device *dev, void *data)
+{
+ struct psy_for_each_psy_cb_data *cb_data = data;
+ struct power_supply *psy = dev_get_drvdata(dev);
+
+ return cb_data->fn(psy, cb_data->data);
+}
+
+int power_supply_for_each_psy(void *data, int (*fn)(struct power_supply *psy, void *data))
+{
+ struct psy_for_each_psy_cb_data cb_data = {
+ .fn = fn,
+ .data = data,
+ };
+
+ return power_supply_for_each_device(&cb_data, psy_for_each_psy_cb);
+}
+EXPORT_SYMBOL_GPL(power_supply_for_each_psy);
+
void power_supply_changed(struct power_supply *psy)
{
unsigned long flags;
@@ -165,21 +188,48 @@ static void power_supply_deferred_register_work(struct work_struct *work)
mutex_unlock(&psy->dev.parent->mutex);
}
-#ifdef CONFIG_OF
-static int __power_supply_populate_supplied_from(struct device *dev,
+static int power_supply_check_supplies_by_name(struct power_supply *psy)
+{
+ struct device *parent = psy->dev.parent;
+ int nval, ret;
+
+ if (!parent)
+ return 0;
+
+ nval = device_property_string_array_count(parent, "supplied-from");
+ if (nval <= 0)
+ return 0;
+
+ psy->supplied_from = devm_kmalloc_array(&psy->dev, nval,
+ sizeof(*psy->supplied_from),
+ GFP_KERNEL);
+ if (!psy->supplied_from)
+ return -ENOMEM;
+
+ ret = device_property_read_string_array(parent, "supplied-from",
+ (const char **)psy->supplied_from,
+ nval);
+ if (ret < 0)
+ return ret;
+
+ psy->num_supplies = nval;
+
+ return 0;
+}
+
+static int __power_supply_populate_supplied_from(struct power_supply *epsy,
void *data)
{
struct power_supply *psy = data;
- struct power_supply *epsy = dev_get_drvdata(dev);
struct device_node *np;
int i = 0;
do {
- np = of_parse_phandle(psy->of_node, "power-supplies", i++);
+ np = of_parse_phandle(psy->dev.of_node, "power-supplies", i++);
if (!np)
break;
- if (np == epsy->of_node) {
+ if (np == epsy->dev.of_node) {
dev_dbg(&psy->dev, "%s: Found supply : %s\n",
psy->desc->name, epsy->desc->name);
psy->supplied_from[i-1] = (char *)epsy->desc->name;
@@ -197,21 +247,20 @@ static int power_supply_populate_supplied_from(struct power_supply *psy)
{
int error;
- error = power_supply_for_each_device(psy, __power_supply_populate_supplied_from);
+ error = power_supply_for_each_psy(psy, __power_supply_populate_supplied_from);
dev_dbg(&psy->dev, "%s %d\n", __func__, error);
return error;
}
-static int __power_supply_find_supply_from_node(struct device *dev,
+static int __power_supply_find_supply_from_node(struct power_supply *epsy,
void *data)
{
struct device_node *np = data;
- struct power_supply *epsy = dev_get_drvdata(dev);
- /* returning non-zero breaks out of power_supply_for_each_device loop */
- if (epsy->of_node == np)
+ /* returning non-zero breaks out of power_supply_for_each_psy loop */
+ if (epsy->dev.of_node == np)
return 1;
return 0;
@@ -222,99 +271,70 @@ static int power_supply_find_supply_from_node(struct device_node *supply_node)
int error;
/*
- * power_supply_for_each_device() either returns its own errors or values
+ * power_supply_for_each_psy() either returns its own errors or values
* returned by __power_supply_find_supply_from_node().
*
* __power_supply_find_supply_from_node() will return 0 (no match)
* or 1 (match).
*
- * We return 0 if power_supply_for_each_device() returned 1, -EPROBE_DEFER if
+ * We return 0 if power_supply_for_each_psy() returned 1, -EPROBE_DEFER if
* it returned 0, or error as returned by it.
*/
- error = power_supply_for_each_device(supply_node, __power_supply_find_supply_from_node);
+ error = power_supply_for_each_psy(supply_node, __power_supply_find_supply_from_node);
return error ? (error == 1 ? 0 : error) : -EPROBE_DEFER;
}
static int power_supply_check_supplies(struct power_supply *psy)
{
- struct device_node *np;
- int cnt = 0;
+ struct of_phandle_args args;
+ int cnt, ret;
/* If there is already a list honor it */
if (psy->supplied_from && psy->num_supplies > 0)
return 0;
- /* No device node found, nothing to do */
- if (!psy->of_node)
- return 0;
-
- do {
- int ret;
+ for (cnt = 0; psy->dev.of_node; cnt++) {
+ ret = of_parse_phandle_with_fixed_args(psy->dev.of_node,
+ "power-supplies", 0, cnt, &args);
+ if (ret) {
+ if (ret != -ENOENT)
+ return ret;
- np = of_parse_phandle(psy->of_node, "power-supplies", cnt++);
- if (!np)
break;
+ }
- ret = power_supply_find_supply_from_node(np);
- of_node_put(np);
+ ret = power_supply_find_supply_from_node(args.np);
+ of_node_put(args.np);
if (ret) {
dev_dbg(&psy->dev, "Failed to find supply!\n");
return ret;
}
- } while (np);
+ }
- /* Missing valid "power-supplies" entries */
- if (cnt == 1)
- return 0;
+ /* Fall back to the name-based property if no references were specified. */
+ if (!cnt)
+ return power_supply_check_supplies_by_name(psy);
/* All supplies found, allocate char * array for filling */
psy->supplied_from = devm_kcalloc(&psy->dev,
- cnt - 1, sizeof(*psy->supplied_from),
+ cnt, sizeof(*psy->supplied_from),
GFP_KERNEL);
if (!psy->supplied_from)
return -ENOMEM;
return power_supply_populate_supplied_from(psy);
}
-#else
-static int power_supply_check_supplies(struct power_supply *psy)
-{
- int nval, ret;
-
- if (!psy->dev.parent)
- return 0;
-
- nval = device_property_string_array_count(psy->dev.parent, "supplied-from");
- if (nval <= 0)
- return 0;
-
- psy->supplied_from = devm_kmalloc_array(&psy->dev, nval,
- sizeof(char *), GFP_KERNEL);
- if (!psy->supplied_from)
- return -ENOMEM;
-
- ret = device_property_read_string_array(psy->dev.parent,
- "supplied-from", (const char **)psy->supplied_from, nval);
- if (ret < 0)
- return ret;
-
- psy->num_supplies = nval;
-
- return 0;
-}
-#endif
struct psy_am_i_supplied_data {
struct power_supply *psy;
unsigned int count;
};
-static int __power_supply_am_i_supplied(struct device *dev, void *_data)
+static int __power_supply_am_i_supplied(struct power_supply *epsy, void *_data)
{
union power_supply_propval ret = {0,};
- struct power_supply *epsy = dev_get_drvdata(dev);
struct psy_am_i_supplied_data *data = _data;
if (__power_supply_is_supplied_by(epsy, data->psy)) {
@@ -332,7 +352,7 @@ int power_supply_am_i_supplied(struct power_supply *psy)
struct psy_am_i_supplied_data data = { psy, 0 };
int error;
- error = power_supply_for_each_device(&data, __power_supply_am_i_supplied);
+ error = power_supply_for_each_psy(&data, __power_supply_am_i_supplied);
dev_dbg(&psy->dev, "%s count %u err %d\n", __func__, data.count, error);
@@ -343,10 +363,9 @@ int power_supply_am_i_supplied(struct power_supply *psy)
}
EXPORT_SYMBOL_GPL(power_supply_am_i_supplied);
-static int __power_supply_is_system_supplied(struct device *dev, void *data)
+static int __power_supply_is_system_supplied(struct power_supply *psy, void *data)
{
union power_supply_propval ret = {0,};
- struct power_supply *psy = dev_get_drvdata(dev);
unsigned int *count = data;
if (!psy->desc->get_property(psy, POWER_SUPPLY_PROP_SCOPE, &ret))
@@ -367,7 +386,7 @@ int power_supply_is_system_supplied(void)
int error;
unsigned int count = 0;
- error = power_supply_for_each_device(&count, __power_supply_is_system_supplied);
+ error = power_supply_for_each_psy(&count, __power_supply_is_system_supplied);
/*
* If no system scope power class device was found at all, most probably we
@@ -386,9 +405,8 @@ struct psy_get_supplier_prop_data {
union power_supply_propval *val;
};
-static int __power_supply_get_supplier_property(struct device *dev, void *_data)
+static int __power_supply_get_supplier_property(struct power_supply *epsy, void *_data)
{
- struct power_supply *epsy = dev_get_drvdata(dev);
struct psy_get_supplier_prop_data *data = _data;
if (__power_supply_is_supplied_by(epsy, data->psy))
@@ -413,7 +431,7 @@ int power_supply_get_property_from_supplier(struct power_supply *psy,
* This function is not intended for use with a supply with multiple
* suppliers, we simply pick the first supply to report the psp.
*/
- ret = power_supply_for_each_device(&data, __power_supply_get_supplier_property);
+ ret = power_supply_for_each_psy(&data, __power_supply_get_supplier_property);
if (ret < 0)
return ret;
if (ret == 0)
@@ -580,8 +598,8 @@ int power_supply_get_battery_info(struct power_supply *psy,
const __be32 *list;
u32 min_max[2];
- if (psy->of_node) {
- battery_np = of_parse_phandle(psy->of_node, "monitored-battery", 0);
+ if (psy->dev.of_node) {
+ battery_np = of_parse_phandle(psy->dev.of_node, "monitored-battery", 0);
if (!battery_np)
return -ENODEV;
@@ -1366,9 +1384,8 @@ __power_supply_register(struct device *parent,
if (cfg) {
dev->groups = cfg->attr_grp;
psy->drv_data = cfg->drv_data;
- psy->of_node =
+ dev->of_node =
cfg->fwnode ? to_of_node(cfg->fwnode) : cfg->of_node;
- dev->of_node = psy->of_node;
psy->supplied_to = cfg->supplied_to;
psy->num_supplicants = cfg->num_supplicants;
}
@@ -1403,6 +1420,9 @@ __power_supply_register(struct device *parent,
if (rc)
goto device_add_failed;
+ if (cfg && cfg->no_wakeup_source)
+ ws = false;
+
rc = device_init_wakeup(dev, ws);
if (rc)
goto wakeup_init_failed;
diff --git a/drivers/power/supply/qcom_battmgr.c b/drivers/power/supply/qcom_battmgr.c
index d55982553863..3674afb0d49d 100644
--- a/drivers/power/supply/qcom_battmgr.c
+++ b/drivers/power/supply/qcom_battmgr.c
@@ -998,6 +998,23 @@ static unsigned int qcom_battmgr_sc8280xp_convert_temp(unsigned int temperature)
return DIV_ROUND_CLOSEST(temperature, 10);
}
+/*
+ * The charger supplies' online state comes from the battery status response,
+ * and the firmware sends no notification of its own when it changes, so
+ * announce the change here. A supply that is not registered yet announces its
+ * state when it is.
+ */
+static void qcom_battmgr_sc8280xp_set_online(struct power_supply *psy,
+ bool *online, bool value)
+{
+ if (*online == value)
+ return;
+
+ *online = value;
+ if (psy)
+ power_supply_changed(psy);
+}
+
static void qcom_battmgr_sc8280xp_callback(struct qcom_battmgr *battmgr,
const struct qcom_battmgr_message *resp,
size_t len)
@@ -1067,9 +1084,12 @@ static void qcom_battmgr_sc8280xp_callback(struct qcom_battmgr *battmgr,
battmgr->status.temperature = qcom_battmgr_sc8280xp_convert_temp(le32_to_cpu(resp->status.temperature));
source = le32_to_cpu(resp->status.charging_source);
- battmgr->ac.online = source == BATTMGR_CHARGING_SOURCE_AC;
- battmgr->usb.online = source == BATTMGR_CHARGING_SOURCE_USB;
- battmgr->wireless.online = source == BATTMGR_CHARGING_SOURCE_WIRELESS;
+ qcom_battmgr_sc8280xp_set_online(battmgr->ac_psy, &battmgr->ac.online,
+ source == BATTMGR_CHARGING_SOURCE_AC);
+ qcom_battmgr_sc8280xp_set_online(battmgr->usb_psy, &battmgr->usb.online,
+ source == BATTMGR_CHARGING_SOURCE_USB);
+ qcom_battmgr_sc8280xp_set_online(battmgr->wls_psy, &battmgr->wireless.online,
+ source == BATTMGR_CHARGING_SOURCE_WIRELESS);
break;
case BATTMGR_BAT_DISCHARGE_TIME:
battmgr->status.discharge_time = le32_to_cpu(resp->time);
diff --git a/drivers/power/supply/qcom_pmi8998_charger.c b/drivers/power/supply/qcom_pmi8998_charger.c
deleted file mode 100644
index d4c9f85654da..000000000000
--- a/drivers/power/supply/qcom_pmi8998_charger.c
+++ /dev/null
@@ -1,1053 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (c) 2016-2019 The Linux Foundation. All rights reserved.
- * Copyright (c) 2023, Linaro Ltd.
- * Author: Caleb Connolly <caleb.connolly@linaro.org>
- *
- * This driver is for the switch-mode battery charger and boost
- * hardware found in pmi8998 and related PMICs.
- */
-
-#include <linux/bits.h>
-#include <linux/devm-helpers.h>
-#include <linux/iio/consumer.h>
-#include <linux/interrupt.h>
-#include <linux/kernel.h>
-#include <linux/minmax.h>
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/pm_wakeirq.h>
-#include <linux/of.h>
-#include <linux/power_supply.h>
-#include <linux/regmap.h>
-#include <linux/types.h>
-#include <linux/workqueue.h>
-
-/* clang-format off */
-#define BATTERY_CHARGER_STATUS_1 0x06
-#define BVR_INITIAL_RAMP_BIT BIT(7)
-#define CC_SOFT_TERMINATE_BIT BIT(6)
-#define STEP_CHARGING_STATUS_SHIFT 3
-#define STEP_CHARGING_STATUS_MASK GENMASK(5, 3)
-#define BATTERY_CHARGER_STATUS_MASK GENMASK(2, 0)
-
-#define BATTERY_CHARGER_STATUS_2 0x07
-#define INPUT_CURRENT_LIMITED_BIT BIT(7)
-#define CHARGER_ERROR_STATUS_SFT_EXPIRE_BIT BIT(6)
-#define CHARGER_ERROR_STATUS_BAT_OV_BIT BIT(5)
-#define CHARGER_ERROR_STATUS_BAT_TERM_MISSING_BIT BIT(4)
-#define BAT_TEMP_STATUS_MASK GENMASK(3, 0)
-#define BAT_TEMP_STATUS_SOFT_LIMIT_MASK GENMASK(3, 2)
-#define BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT BIT(3)
-#define BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT BIT(2)
-#define BAT_TEMP_STATUS_TOO_HOT_BIT BIT(1)
-#define BAT_TEMP_STATUS_TOO_COLD_BIT BIT(0)
-
-#define BATTERY_CHARGER_STATUS_4 0x0A
-#define CHARGE_CURRENT_POST_JEITA_MASK GENMASK(7, 0)
-
-#define BATTERY_CHARGER_STATUS_7 0x0D
-#define ENABLE_TRICKLE_BIT BIT(7)
-#define ENABLE_PRE_CHARGING_BIT BIT(6)
-#define ENABLE_FAST_CHARGING_BIT BIT(5)
-#define ENABLE_FULLON_MODE_BIT BIT(4)
-#define TOO_COLD_ADC_BIT BIT(3)
-#define TOO_HOT_ADC_BIT BIT(2)
-#define HOT_SL_ADC_BIT BIT(1)
-#define COLD_SL_ADC_BIT BIT(0)
-
-#define CHARGING_ENABLE_CMD 0x42
-#define CHARGING_ENABLE_CMD_BIT BIT(0)
-
-#define CHGR_CFG2 0x51
-#define CHG_EN_SRC_BIT BIT(7)
-#define CHG_EN_POLARITY_BIT BIT(6)
-#define PRETOFAST_TRANSITION_CFG_BIT BIT(5)
-#define BAT_OV_ECC_BIT BIT(4)
-#define I_TERM_BIT BIT(3)
-#define AUTO_RECHG_BIT BIT(2)
-#define EN_ANALOG_DROP_IN_VBATT_BIT BIT(1)
-#define CHARGER_INHIBIT_BIT BIT(0)
-
-#define PRE_CHARGE_CURRENT_CFG 0x60
-#define PRE_CHARGE_CURRENT_SETTING_MASK GENMASK(5, 0)
-
-#define FAST_CHARGE_CURRENT_CFG 0x61
-#define FAST_CHARGE_CURRENT_SETTING_MASK GENMASK(7, 0)
-
-#define FLOAT_VOLTAGE_CFG 0x70
-#define FLOAT_VOLTAGE_SETTING_MASK GENMASK(7, 0)
-
-#define FG_UPDATE_CFG_2_SEL 0x7D
-#define SOC_LT_OTG_THRESH_SEL_BIT BIT(3)
-#define SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(2)
-#define VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(1)
-#define IBT_LT_CHG_TERM_THRESH_SEL_BIT BIT(0)
-
-#define JEITA_EN_CFG 0x90
-#define JEITA_EN_HARDLIMIT_BIT BIT(4)
-#define JEITA_EN_HOT_SL_FCV_BIT BIT(3)
-#define JEITA_EN_COLD_SL_FCV_BIT BIT(2)
-#define JEITA_EN_HOT_SL_CCC_BIT BIT(1)
-#define JEITA_EN_COLD_SL_CCC_BIT BIT(0)
-
-#define INT_RT_STS 0x310
-#define TYPE_C_CHANGE_RT_STS_BIT BIT(7)
-#define USBIN_ICL_CHANGE_RT_STS_BIT BIT(6)
-#define USBIN_SOURCE_CHANGE_RT_STS_BIT BIT(5)
-#define USBIN_PLUGIN_RT_STS_BIT BIT(4)
-#define USBIN_OV_RT_STS_BIT BIT(3)
-#define USBIN_UV_RT_STS_BIT BIT(2)
-#define USBIN_LT_3P6V_RT_STS_BIT BIT(1)
-#define USBIN_COLLAPSE_RT_STS_BIT BIT(0)
-
-#define OTG_CFG 0x153
-#define OTG_RESERVED_MASK GENMASK(7, 6)
-#define DIS_OTG_ON_TLIM_BIT BIT(5)
-#define QUICKSTART_OTG_FASTROLESWAP_BIT BIT(4)
-#define INCREASE_DFP_TIME_BIT BIT(3)
-#define ENABLE_OTG_IN_DEBUG_MODE_BIT BIT(2)
-#define OTG_EN_SRC_CFG_BIT BIT(1)
-#define CONCURRENT_MODE_CFG_BIT BIT(0)
-
-#define OTG_ENG_OTG_CFG 0x1C0
-#define ENG_BUCKBOOST_HALT1_8_MODE_BIT BIT(0)
-
-#define APSD_STATUS 0x307
-#define APSD_STATUS_7_BIT BIT(7)
-#define HVDCP_CHECK_TIMEOUT_BIT BIT(6)
-#define SLOW_PLUGIN_TIMEOUT_BIT BIT(5)
-#define ENUMERATION_DONE_BIT BIT(4)
-#define VADP_CHANGE_DONE_AFTER_AUTH_BIT BIT(3)
-#define QC_AUTH_DONE_STATUS_BIT BIT(2)
-#define QC_CHARGER_BIT BIT(1)
-#define APSD_DTC_STATUS_DONE_BIT BIT(0)
-
-#define APSD_RESULT_STATUS 0x308
-#define ICL_OVERRIDE_LATCH_BIT BIT(7)
-#define APSD_RESULT_STATUS_MASK GENMASK(6, 0)
-#define QC_3P0_BIT BIT(6)
-#define QC_2P0_BIT BIT(5)
-#define FLOAT_CHARGER_BIT BIT(4)
-#define DCP_CHARGER_BIT BIT(3)
-#define CDP_CHARGER_BIT BIT(2)
-#define OCP_CHARGER_BIT BIT(1)
-#define SDP_CHARGER_BIT BIT(0)
-
-#define TYPE_C_STATUS_1 0x30B
-#define UFP_TYPEC_MASK GENMASK(7, 5)
-#define UFP_TYPEC_RDSTD_BIT BIT(7)
-#define UFP_TYPEC_RD1P5_BIT BIT(6)
-#define UFP_TYPEC_RD3P0_BIT BIT(5)
-#define UFP_TYPEC_FMB_255K_BIT BIT(4)
-#define UFP_TYPEC_FMB_301K_BIT BIT(3)
-#define UFP_TYPEC_FMB_523K_BIT BIT(2)
-#define UFP_TYPEC_FMB_619K_BIT BIT(1)
-#define UFP_TYPEC_OPEN_OPEN_BIT BIT(0)
-
-#define TYPE_C_STATUS_2 0x30C
-#define DFP_RA_OPEN_BIT BIT(7)
-#define TIMER_STAGE_BIT BIT(6)
-#define EXIT_UFP_MODE_BIT BIT(5)
-#define EXIT_DFP_MODE_BIT BIT(4)
-#define DFP_TYPEC_MASK GENMASK(3, 0)
-#define DFP_RD_OPEN_BIT BIT(3)
-#define DFP_RD_RA_VCONN_BIT BIT(2)
-#define DFP_RD_RD_BIT BIT(1)
-#define DFP_RA_RA_BIT BIT(0)
-
-#define TYPE_C_STATUS_3 0x30D
-#define ENABLE_BANDGAP_BIT BIT(7)
-#define U_USB_GND_NOVBUS_BIT BIT(6)
-#define U_USB_FLOAT_NOVBUS_BIT BIT(5)
-#define U_USB_GND_BIT BIT(4)
-#define U_USB_FMB1_BIT BIT(3)
-#define U_USB_FLOAT1_BIT BIT(2)
-#define U_USB_FMB2_BIT BIT(1)
-#define U_USB_FLOAT2_BIT BIT(0)
-
-#define TYPE_C_STATUS_4 0x30E
-#define UFP_DFP_MODE_STATUS_BIT BIT(7)
-#define TYPEC_VBUS_STATUS_BIT BIT(6)
-#define TYPEC_VBUS_ERROR_STATUS_BIT BIT(5)
-#define TYPEC_DEBOUNCE_DONE_STATUS_BIT BIT(4)
-#define TYPEC_UFP_AUDIO_ADAPT_STATUS_BIT BIT(3)
-#define TYPEC_VCONN_OVERCURR_STATUS_BIT BIT(2)
-#define CC_ORIENTATION_BIT BIT(1)
-#define CC_ATTACHED_BIT BIT(0)
-
-#define TYPE_C_STATUS_5 0x30F
-#define TRY_SOURCE_FAILED_BIT BIT(6)
-#define TRY_SINK_FAILED_BIT BIT(5)
-#define TIMER_STAGE_2_BIT BIT(4)
-#define TYPEC_LEGACY_CABLE_STATUS_BIT BIT(3)
-#define TYPEC_NONCOMP_LEGACY_CABLE_STATUS_BIT BIT(2)
-#define TYPEC_TRYSOURCE_DETECT_STATUS_BIT BIT(1)
-#define TYPEC_TRYSINK_DETECT_STATUS_BIT BIT(0)
-
-#define CMD_APSD 0x341
-#define ICL_OVERRIDE_BIT BIT(1)
-#define APSD_RERUN_BIT BIT(0)
-
-#define TYPE_C_CFG 0x358
-#define APSD_START_ON_CC_BIT BIT(7)
-#define WAIT_FOR_APSD_BIT BIT(6)
-#define FACTORY_MODE_DETECTION_EN_BIT BIT(5)
-#define FACTORY_MODE_ICL_3A_4A_BIT BIT(4)
-#define FACTORY_MODE_DIS_CHGING_CFG_BIT BIT(3)
-#define SUSPEND_NON_COMPLIANT_CFG_BIT BIT(2)
-#define VCONN_OC_CFG_BIT BIT(1)
-#define TYPE_C_OR_U_USB_BIT BIT(0)
-
-#define TYPE_C_CFG_2 0x359
-#define TYPE_C_DFP_CURRSRC_MODE_BIT BIT(7)
-#define DFP_CC_1P4V_OR_1P6V_BIT BIT(6)
-#define VCONN_SOFTSTART_CFG_MASK GENMASK(5, 4)
-#define EN_TRY_SOURCE_MODE_BIT BIT(3)
-#define USB_FACTORY_MODE_ENABLE_BIT BIT(2)
-#define TYPE_C_UFP_MODE_BIT BIT(1)
-#define EN_80UA_180UA_CUR_SOURCE_BIT BIT(0)
-
-#define TYPE_C_CFG_3 0x35A
-#define TVBUS_DEBOUNCE_BIT BIT(7)
-#define TYPEC_LEGACY_CABLE_INT_EN_BIT BIT(6)
-#define TYPEC_NONCOMPLIANT_LEGACY_CABLE_INT_EN_B BIT(5)
-#define TYPEC_TRYSOURCE_DETECT_INT_EN_BIT BIT(4)
-#define TYPEC_TRYSINK_DETECT_INT_EN_BIT BIT(3)
-#define EN_TRYSINK_MODE_BIT BIT(2)
-#define EN_LEGACY_CABLE_DETECTION_BIT BIT(1)
-#define ALLOW_PD_DRING_UFP_TCCDB_BIT BIT(0)
-
-#define USBIN_OPTIONS_1_CFG 0x362
-#define CABLE_R_SEL_BIT BIT(7)
-#define HVDCP_AUTH_ALG_EN_CFG_BIT BIT(6)
-#define HVDCP_AUTONOMOUS_MODE_EN_CFG_BIT BIT(5)
-#define INPUT_PRIORITY_BIT BIT(4)
-#define AUTO_SRC_DETECT_BIT BIT(3)
-#define HVDCP_EN_BIT BIT(2)
-#define VADP_INCREMENT_VOLTAGE_LIMIT_BIT BIT(1)
-#define VADP_TAPER_TIMER_EN_BIT BIT(0)
-
-#define USBIN_OPTIONS_2_CFG 0x363
-#define WIPWR_RST_EUD_CFG_BIT BIT(7)
-#define SWITCHER_START_CFG_BIT BIT(6)
-#define DCD_TIMEOUT_SEL_BIT BIT(5)
-#define OCD_CURRENT_SEL_BIT BIT(4)
-#define SLOW_PLUGIN_TIMER_EN_CFG_BIT BIT(3)
-#define FLOAT_OPTIONS_MASK GENMASK(2, 0)
-#define FLOAT_DIS_CHGING_CFG_BIT BIT(2)
-#define SUSPEND_FLOAT_CFG_BIT BIT(1)
-#define FORCE_FLOAT_SDP_CFG_BIT BIT(0)
-
-#define TAPER_TIMER_SEL_CFG 0x364
-#define TYPEC_SPARE_CFG_BIT BIT(7)
-#define TYPEC_DRP_DFP_TIME_CFG_BIT BIT(5)
-#define TAPER_TIMER_SEL_MASK GENMASK(1, 0)
-
-#define USBIN_LOAD_CFG 0x365
-#define USBIN_OV_CH_LOAD_OPTION_BIT BIT(7)
-#define ICL_OVERRIDE_AFTER_APSD_BIT BIT(4)
-
-#define USBIN_ICL_OPTIONS 0x366
-#define CFG_USB3P0_SEL_BIT BIT(2)
-#define USB51_MODE_BIT BIT(1)
-#define USBIN_MODE_CHG_BIT BIT(0)
-
-#define TYPE_C_INTRPT_ENB_SOFTWARE_CTRL 0x368
-#define EXIT_SNK_BASED_ON_CC_BIT BIT(7)
-#define VCONN_EN_ORIENTATION_BIT BIT(6)
-#define TYPEC_VCONN_OVERCURR_INT_EN_BIT BIT(5)
-#define VCONN_EN_SRC_BIT BIT(4)
-#define VCONN_EN_VALUE_BIT BIT(3)
-#define TYPEC_POWER_ROLE_CMD_MASK GENMASK(2, 0)
-#define UFP_EN_CMD_BIT BIT(2)
-#define DFP_EN_CMD_BIT BIT(1)
-#define TYPEC_DISABLE_CMD_BIT BIT(0)
-
-#define USBIN_CURRENT_LIMIT_CFG 0x370
-#define USBIN_CURRENT_LIMIT_MASK GENMASK(7, 0)
-
-#define USBIN_AICL_OPTIONS_CFG 0x380
-#define SUSPEND_ON_COLLAPSE_USBIN_BIT BIT(7)
-#define USBIN_AICL_HDC_EN_BIT BIT(6)
-#define USBIN_AICL_START_AT_MAX_BIT BIT(5)
-#define USBIN_AICL_RERUN_EN_BIT BIT(4)
-#define USBIN_AICL_ADC_EN_BIT BIT(3)
-#define USBIN_AICL_EN_BIT BIT(2)
-#define USBIN_HV_COLLAPSE_RESPONSE_BIT BIT(1)
-#define USBIN_LV_COLLAPSE_RESPONSE_BIT BIT(0)
-
-#define USBIN_5V_AICL_THRESHOLD_CFG 0x381
-#define USBIN_5V_AICL_THRESHOLD_CFG_MASK GENMASK(2, 0)
-
-#define USBIN_CONT_AICL_THRESHOLD_CFG 0x384
-#define USBIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0)
-
-#define DC_ENG_SSUPPLY_CFG2 0x4C1
-#define ENG_SSUPPLY_IVREF_OTG_SS_MASK GENMASK(2, 0)
-#define OTG_SS_SLOW 0x3
-
-#define DCIN_AICL_REF_SEL_CFG 0x481
-#define DCIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0)
-
-#define WI_PWR_OPTIONS 0x495
-#define CHG_OK_BIT BIT(7)
-#define WIPWR_UVLO_IRQ_OPT_BIT BIT(6)
-#define BUCK_HOLDOFF_ENABLE_BIT BIT(5)
-#define CHG_OK_HW_SW_SELECT_BIT BIT(4)
-#define WIPWR_RST_ENABLE_BIT BIT(3)
-#define DCIN_WIPWR_IRQ_SELECT_BIT BIT(2)
-#define AICL_SWITCH_ENABLE_BIT BIT(1)
-#define ZIN_ICL_ENABLE_BIT BIT(0)
-
-#define ICL_STATUS 0x607
-#define INPUT_CURRENT_LIMIT_MASK GENMASK(7, 0)
-
-#define POWER_PATH_STATUS 0x60B
-#define P_PATH_INPUT_SS_DONE_BIT BIT(7)
-#define P_PATH_USBIN_SUSPEND_STS_BIT BIT(6)
-#define P_PATH_DCIN_SUSPEND_STS_BIT BIT(5)
-#define P_PATH_USE_USBIN_BIT BIT(4)
-#define P_PATH_USE_DCIN_BIT BIT(3)
-#define P_PATH_POWER_PATH_MASK GENMASK(2, 1)
-#define P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT BIT(0)
-
-#define BARK_BITE_WDOG_PET 0x643
-#define BARK_BITE_WDOG_PET_BIT BIT(0)
-
-#define WD_CFG 0x651
-#define WATCHDOG_TRIGGER_AFP_EN_BIT BIT(7)
-#define BARK_WDOG_INT_EN_BIT BIT(6)
-#define BITE_WDOG_INT_EN_BIT BIT(5)
-#define SFT_AFTER_WDOG_IRQ_MASK GENMASK(4, 3)
-#define WDOG_IRQ_SFT_BIT BIT(2)
-#define WDOG_TIMER_EN_ON_PLUGIN_BIT BIT(1)
-#define WDOG_TIMER_EN_BIT BIT(0)
-
-#define SNARL_BARK_BITE_WD_CFG 0x653
-#define BITE_WDOG_DISABLE_CHARGING_CFG_BIT BIT(7)
-#define SNARL_WDOG_TIMEOUT_MASK GENMASK(6, 4)
-#define BARK_WDOG_TIMEOUT_MASK GENMASK(3, 2)
-#define BITE_WDOG_TIMEOUT_MASK GENMASK(1, 0)
-
-#define AICL_RERUN_TIME_CFG 0x661
-#define AICL_RERUN_TIME_MASK GENMASK(1, 0)
-
-#define STAT_CFG 0x690
-#define STAT_SW_OVERRIDE_VALUE_BIT BIT(7)
-#define STAT_SW_OVERRIDE_CFG_BIT BIT(6)
-#define STAT_PARALLEL_OFF_DG_CFG_MASK GENMASK(5, 4)
-#define STAT_POLARITY_CFG_BIT BIT(3)
-#define STAT_PARALLEL_CFG_BIT BIT(2)
-#define STAT_FUNCTION_CFG_BIT BIT(1)
-#define STAT_IRQ_PULSING_EN_BIT BIT(0)
-
-#define SDP_CURRENT_UA 500000
-#define CDP_CURRENT_UA 1500000
-#define DCP_CURRENT_UA 1500000
-#define CURRENT_MAX_UA DCP_CURRENT_UA
-
-/* pmi8998 registers represent current in increments of 1/40th of an amp */
-#define CURRENT_SCALE_FACTOR 25000
-#define SMB2_FLOAT_VOLTAGE_MIN_UV 3487500
-#define SMB2_FLOAT_VOLTAGE_STEP_UV 7500
-/* clang-format on */
-
-enum charger_status {
- TRICKLE_CHARGE = 0,
- PRE_CHARGE,
- FAST_CHARGE,
- FULLON_CHARGE,
- TAPER_CHARGE,
- TERMINATE_CHARGE,
- INHIBIT_CHARGE,
- DISABLE_CHARGE,
-};
-
-struct smb2_register {
- u16 addr;
- u8 mask;
- u8 val;
-};
-
-/**
- * struct smb2_chip - smb2 chip structure
- * @dev: Device reference for power_supply
- * @name: The platform device name
- * @base: Base address for smb2 registers
- * @regmap: Register map
- * @batt_info: Battery data from DT
- * @status_change_work: Worker to handle plug/unplug events
- * @cable_irq: USB plugin IRQ
- * @wakeup_enabled: If the cable IRQ will cause a wakeup
- * @usb_in_i_chan: USB_IN current measurement channel
- * @usb_in_v_chan: USB_IN voltage measurement channel
- * @chg_psy: Charger power supply instance
- */
-struct smb2_chip {
- struct device *dev;
- const char *name;
- unsigned int base;
- struct regmap *regmap;
- struct power_supply_battery_info *batt_info;
-
- struct delayed_work status_change_work;
- int cable_irq;
- bool wakeup_enabled;
-
- struct iio_channel *usb_in_i_chan;
- struct iio_channel *usb_in_v_chan;
-
- struct power_supply *chg_psy;
-};
-
-static enum power_supply_property smb2_properties[] = {
- POWER_SUPPLY_PROP_MANUFACTURER,
- POWER_SUPPLY_PROP_MODEL_NAME,
- POWER_SUPPLY_PROP_CURRENT_MAX,
- POWER_SUPPLY_PROP_CURRENT_NOW,
- POWER_SUPPLY_PROP_VOLTAGE_NOW,
- POWER_SUPPLY_PROP_STATUS,
- POWER_SUPPLY_PROP_HEALTH,
- POWER_SUPPLY_PROP_ONLINE,
- POWER_SUPPLY_PROP_USB_TYPE,
-};
-
-static int smb2_get_prop_usb_online(struct smb2_chip *chip, int *val)
-{
- unsigned int stat;
- int rc;
-
- rc = regmap_read(chip->regmap, chip->base + POWER_PATH_STATUS, &stat);
- if (rc < 0) {
- dev_err(chip->dev, "Couldn't read power path status: %d\n", rc);
- return rc;
- }
-
- *val = (stat & P_PATH_USE_USBIN_BIT) &&
- (stat & P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT);
- return 0;
-}
-
-/*
- * Qualcomm "automatic power source detection" aka APSD
- * tells us what type of charger we're connected to.
- */
-static int smb2_apsd_get_charger_type(struct smb2_chip *chip, int *val)
-{
- unsigned int apsd_stat, stat;
- int usb_online = 0;
- int rc;
-
- rc = smb2_get_prop_usb_online(chip, &usb_online);
- if (!usb_online) {
- *val = POWER_SUPPLY_USB_TYPE_UNKNOWN;
- return rc;
- }
-
- rc = regmap_read(chip->regmap, chip->base + APSD_STATUS, &apsd_stat);
- if (rc < 0) {
- dev_err(chip->dev, "Failed to read apsd status, rc = %d", rc);
- return rc;
- }
- if (!(apsd_stat & APSD_DTC_STATUS_DONE_BIT)) {
- dev_dbg(chip->dev, "Apsd not ready");
- return -EAGAIN;
- }
-
- rc = regmap_read(chip->regmap, chip->base + APSD_RESULT_STATUS, &stat);
- if (rc < 0) {
- dev_err(chip->dev, "Failed to read apsd result, rc = %d", rc);
- return rc;
- }
-
- stat &= APSD_RESULT_STATUS_MASK;
-
- if (stat & CDP_CHARGER_BIT)
- *val = POWER_SUPPLY_USB_TYPE_CDP;
- else if (stat & (DCP_CHARGER_BIT | OCP_CHARGER_BIT | FLOAT_CHARGER_BIT))
- *val = POWER_SUPPLY_USB_TYPE_DCP;
- else /* SDP_CHARGER_BIT (or others) */
- *val = POWER_SUPPLY_USB_TYPE_SDP;
-
- return 0;
-}
-
-static int smb2_get_prop_status(struct smb2_chip *chip, int *val)
-{
- unsigned char stat[2];
- int usb_online = 0;
- int rc;
-
- rc = smb2_get_prop_usb_online(chip, &usb_online);
- if (!usb_online) {
- *val = POWER_SUPPLY_STATUS_DISCHARGING;
- return rc;
- }
-
- rc = regmap_bulk_read(chip->regmap,
- chip->base + BATTERY_CHARGER_STATUS_1, &stat, 2);
- if (rc < 0) {
- dev_err(chip->dev, "Failed to read charging status ret=%d\n",
- rc);
- return rc;
- }
-
- if (stat[1] & CHARGER_ERROR_STATUS_BAT_OV_BIT) {
- *val = POWER_SUPPLY_STATUS_NOT_CHARGING;
- return 0;
- }
-
- stat[0] = stat[0] & BATTERY_CHARGER_STATUS_MASK;
-
- switch (stat[0]) {
- case TRICKLE_CHARGE:
- case PRE_CHARGE:
- case FAST_CHARGE:
- case FULLON_CHARGE:
- case TAPER_CHARGE:
- *val = POWER_SUPPLY_STATUS_CHARGING;
- return rc;
- case DISABLE_CHARGE:
- *val = POWER_SUPPLY_STATUS_NOT_CHARGING;
- return rc;
- case TERMINATE_CHARGE:
- case INHIBIT_CHARGE:
- *val = POWER_SUPPLY_STATUS_FULL;
- return rc;
- default:
- *val = POWER_SUPPLY_STATUS_UNKNOWN;
- return rc;
- }
-}
-
-static inline int smb2_get_current_limit(struct smb2_chip *chip,
- unsigned int *val)
-{
- int rc = regmap_read(chip->regmap, chip->base + ICL_STATUS, val);
-
- if (rc >= 0)
- *val *= CURRENT_SCALE_FACTOR;
- return rc;
-}
-
-static int smb2_set_current_limit(struct smb2_chip *chip, unsigned int val)
-{
- unsigned char val_raw;
-
- if (val > 4800000) {
- dev_err(chip->dev,
- "Can't set current limit higher than 4800000uA");
- return -EINVAL;
- }
- val_raw = val / CURRENT_SCALE_FACTOR;
-
- return regmap_write(chip->regmap, chip->base + USBIN_CURRENT_LIMIT_CFG,
- val_raw);
-}
-
-static void smb2_status_change_work(struct work_struct *work)
-{
- unsigned int charger_type, current_ua;
- int usb_online = 0;
- int count, rc;
- struct smb2_chip *chip;
-
- chip = container_of(work, struct smb2_chip, status_change_work.work);
-
- smb2_get_prop_usb_online(chip, &usb_online);
- if (!usb_online)
- return;
-
- for (count = 0; count < 3; count++) {
- dev_dbg(chip->dev, "get charger type retry %d\n", count);
- rc = smb2_apsd_get_charger_type(chip, &charger_type);
- if (rc != -EAGAIN)
- break;
- msleep(100);
- }
-
- if (rc < 0 && rc != -EAGAIN) {
- dev_err(chip->dev, "get charger type failed: %d\n", rc);
- return;
- }
-
- if (rc < 0) {
- rc = regmap_update_bits(chip->regmap, chip->base + CMD_APSD,
- APSD_RERUN_BIT, APSD_RERUN_BIT);
- schedule_delayed_work(&chip->status_change_work,
- msecs_to_jiffies(1000));
- dev_dbg(chip->dev, "get charger type failed, rerun apsd\n");
- return;
- }
-
- switch (charger_type) {
- case POWER_SUPPLY_USB_TYPE_CDP:
- current_ua = CDP_CURRENT_UA;
- break;
- case POWER_SUPPLY_USB_TYPE_DCP:
- current_ua = DCP_CURRENT_UA;
- break;
- case POWER_SUPPLY_USB_TYPE_SDP:
- default:
- current_ua = SDP_CURRENT_UA;
- break;
- }
-
- smb2_set_current_limit(chip, current_ua);
- power_supply_changed(chip->chg_psy);
-}
-
-static int smb2_get_iio_chan(struct smb2_chip *chip, struct iio_channel *chan,
- int *val)
-{
- int rc;
- union power_supply_propval status;
-
- rc = power_supply_get_property(chip->chg_psy, POWER_SUPPLY_PROP_STATUS,
- &status);
- if (rc < 0 || status.intval != POWER_SUPPLY_STATUS_CHARGING) {
- *val = 0;
- return 0;
- }
-
- if (IS_ERR(chan)) {
- dev_err(chip->dev, "Failed to chan, err = %li", PTR_ERR(chan));
- return PTR_ERR(chan);
- }
-
- return iio_read_channel_processed(chan, val);
-}
-
-static int smb2_get_prop_health(struct smb2_chip *chip, int *val)
-{
- int rc;
- unsigned int stat;
-
- rc = regmap_read(chip->regmap, chip->base + BATTERY_CHARGER_STATUS_2,
- &stat);
- if (rc < 0) {
- dev_err(chip->dev, "Couldn't read charger status rc=%d\n", rc);
- return rc;
- }
-
- if (stat & CHARGER_ERROR_STATUS_BAT_OV_BIT)
- *val = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
- else if (stat & BAT_TEMP_STATUS_TOO_COLD_BIT)
- *val = POWER_SUPPLY_HEALTH_COLD;
- else if (stat & BAT_TEMP_STATUS_TOO_HOT_BIT)
- *val = POWER_SUPPLY_HEALTH_OVERHEAT;
- else if (stat & BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT)
- *val = POWER_SUPPLY_HEALTH_COOL;
- else if (stat & BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT)
- *val = POWER_SUPPLY_HEALTH_WARM;
- else
- *val = POWER_SUPPLY_HEALTH_GOOD;
-
- return 0;
-}
-
-static int smb2_get_property(struct power_supply *psy,
- enum power_supply_property psp,
- union power_supply_propval *val)
-{
- struct smb2_chip *chip = power_supply_get_drvdata(psy);
-
- switch (psp) {
- case POWER_SUPPLY_PROP_MANUFACTURER:
- val->strval = "Qualcomm";
- return 0;
- case POWER_SUPPLY_PROP_MODEL_NAME:
- val->strval = chip->name;
- return 0;
- case POWER_SUPPLY_PROP_CURRENT_MAX:
- return smb2_get_current_limit(chip, &val->intval);
- case POWER_SUPPLY_PROP_CURRENT_NOW:
- return smb2_get_iio_chan(chip, chip->usb_in_i_chan,
- &val->intval);
- case POWER_SUPPLY_PROP_VOLTAGE_NOW:
- return smb2_get_iio_chan(chip, chip->usb_in_v_chan,
- &val->intval);
- case POWER_SUPPLY_PROP_ONLINE:
- return smb2_get_prop_usb_online(chip, &val->intval);
- case POWER_SUPPLY_PROP_STATUS:
- return smb2_get_prop_status(chip, &val->intval);
- case POWER_SUPPLY_PROP_HEALTH:
- return smb2_get_prop_health(chip, &val->intval);
- case POWER_SUPPLY_PROP_USB_TYPE:
- return smb2_apsd_get_charger_type(chip, &val->intval);
- default:
- dev_err(chip->dev, "invalid property: %d\n", psp);
- return -EINVAL;
- }
-}
-
-static int smb2_set_property(struct power_supply *psy,
- enum power_supply_property psp,
- const union power_supply_propval *val)
-{
- struct smb2_chip *chip = power_supply_get_drvdata(psy);
-
- switch (psp) {
- case POWER_SUPPLY_PROP_CURRENT_MAX:
- return smb2_set_current_limit(chip, val->intval);
- default:
- dev_err(chip->dev, "No setter for property: %d\n", psp);
- return -EINVAL;
- }
-}
-
-static int smb2_property_is_writable(struct power_supply *psy,
- enum power_supply_property psp)
-{
- switch (psp) {
- case POWER_SUPPLY_PROP_CURRENT_MAX:
- return 1;
- default:
- return 0;
- }
-}
-
-static irqreturn_t smb2_handle_batt_overvoltage(int irq, void *data)
-{
- struct smb2_chip *chip = data;
- unsigned int status;
- int rc;
-
- rc = regmap_read(chip->regmap,
- chip->base + BATTERY_CHARGER_STATUS_2, &status);
- if (rc < 0) {
- dev_err(chip->dev, "Couldn't read charger status: %d\n", rc);
- return IRQ_HANDLED;
- }
-
- if (status & CHARGER_ERROR_STATUS_BAT_OV_BIT) {
- /* The hardware stops charging automatically */
- dev_err(chip->dev, "battery overvoltage detected\n");
- }
-
- power_supply_changed(chip->chg_psy);
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t smb2_handle_usb_plugin(int irq, void *data)
-{
- struct smb2_chip *chip = data;
-
- power_supply_changed(chip->chg_psy);
-
- schedule_delayed_work(&chip->status_change_work,
- msecs_to_jiffies(1500));
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t smb2_handle_usb_icl_change(int irq, void *data)
-{
- struct smb2_chip *chip = data;
-
- power_supply_changed(chip->chg_psy);
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t smb2_handle_wdog_bark(int irq, void *data)
-{
- struct smb2_chip *chip = data;
- int rc;
-
- power_supply_changed(chip->chg_psy);
-
- rc = regmap_write(chip->regmap, chip->base + BARK_BITE_WDOG_PET,
- BARK_BITE_WDOG_PET_BIT);
- if (rc < 0)
- dev_err(chip->dev, "Couldn't pet the dog rc=%d\n", rc);
-
- return IRQ_HANDLED;
-}
-
-static const struct power_supply_desc smb2_psy_desc = {
- .name = "pmi8998_charger",
- .type = POWER_SUPPLY_TYPE_USB,
- .usb_types = BIT(POWER_SUPPLY_USB_TYPE_SDP) |
- BIT(POWER_SUPPLY_USB_TYPE_CDP) |
- BIT(POWER_SUPPLY_USB_TYPE_DCP) |
- BIT(POWER_SUPPLY_USB_TYPE_UNKNOWN),
- .properties = smb2_properties,
- .num_properties = ARRAY_SIZE(smb2_properties),
- .get_property = smb2_get_property,
- .set_property = smb2_set_property,
- .property_is_writeable = smb2_property_is_writable,
-};
-
-/* Init sequence derived from vendor downstream driver */
-static const struct smb2_register smb2_init_seq[] = {
- { .addr = AICL_RERUN_TIME_CFG, .mask = AICL_RERUN_TIME_MASK, .val = 0 },
- /*
- * By default configure us as an upstream facing port
- * FIXME: This will be handled by the type-c driver
- */
- { .addr = TYPE_C_INTRPT_ENB_SOFTWARE_CTRL,
- .mask = TYPEC_POWER_ROLE_CMD_MASK | VCONN_EN_SRC_BIT |
- VCONN_EN_VALUE_BIT,
- .val = VCONN_EN_SRC_BIT },
- /*
- * Disable Type-C factory mode and stay in Attached.SRC state when VCONN
- * over-current happens
- */
- { .addr = TYPE_C_CFG,
- .mask = FACTORY_MODE_DETECTION_EN_BIT | VCONN_OC_CFG_BIT,
- .val = 0 },
- /* Configure VBUS for software control */
- { .addr = OTG_CFG, .mask = OTG_EN_SRC_CFG_BIT, .val = 0 },
- /*
- * Use VBAT to determine the recharge threshold when battery is full
- * rather than the state of charge.
- */
- { .addr = FG_UPDATE_CFG_2_SEL,
- .mask = SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT |
- VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT,
- .val = VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT },
- /* Enable charging */
- { .addr = USBIN_OPTIONS_1_CFG, .mask = HVDCP_EN_BIT, .val = 0 },
- { .addr = CHARGING_ENABLE_CMD,
- .mask = CHARGING_ENABLE_CMD_BIT,
- .val = CHARGING_ENABLE_CMD_BIT },
- /*
- * Match downstream defaults
- * CHG_EN_SRC_BIT - charger enable is controlled by software
- * CHG_EN_POLARITY_BIT - polarity of charge enable pin when in HW control
- * pulled low on OnePlus 6 and SHIFT6mq
- * PRETOFAST_TRANSITION_CFG_BIT -
- * BAT_OV_ECC_BIT -
- * I_TERM_BIT - Current termination ?? 0 = enabled
- * AUTO_RECHG_BIT - Enable automatic recharge when battery is full
- * 0 = enabled
- * EN_ANALOG_DROP_IN_VBATT_BIT
- * CHARGER_INHIBIT_BIT - Inhibit charging based on battery voltage
- * instead of ??
- */
- { .addr = CHGR_CFG2,
- .mask = CHG_EN_SRC_BIT | CHG_EN_POLARITY_BIT |
- PRETOFAST_TRANSITION_CFG_BIT | BAT_OV_ECC_BIT | I_TERM_BIT |
- AUTO_RECHG_BIT | EN_ANALOG_DROP_IN_VBATT_BIT |
- CHARGER_INHIBIT_BIT,
- .val = CHARGER_INHIBIT_BIT },
- /* STAT pin software override, match downstream. Parallell charging? */
- { .addr = STAT_CFG,
- .mask = STAT_SW_OVERRIDE_CFG_BIT,
- .val = STAT_SW_OVERRIDE_CFG_BIT },
- /* Set the default SDP charger type to a 500ma USB 2.0 port */
- { .addr = USBIN_ICL_OPTIONS,
- .mask = USB51_MODE_BIT | USBIN_MODE_CHG_BIT,
- .val = USB51_MODE_BIT },
- /* Disable watchdog */
- { .addr = SNARL_BARK_BITE_WD_CFG, .mask = 0xff, .val = 0 },
- { .addr = WD_CFG,
- .mask = WATCHDOG_TRIGGER_AFP_EN_BIT | WDOG_TIMER_EN_ON_PLUGIN_BIT |
- BARK_WDOG_INT_EN_BIT,
- .val = 0 },
- /* These bits aren't documented anywhere */
- { .addr = USBIN_5V_AICL_THRESHOLD_CFG,
- .mask = USBIN_5V_AICL_THRESHOLD_CFG_MASK,
- .val = 0x3 },
- { .addr = USBIN_CONT_AICL_THRESHOLD_CFG,
- .mask = USBIN_CONT_AICL_THRESHOLD_CFG_MASK,
- .val = 0x3 },
- /*
- * Enable Automatic Input Current Limit, this will slowly ramp up the current
- * When connected to a wall charger, and automatically stop when it detects
- * the charger current limit (voltage drop?) or it reaches the programmed limit.
- */
- { .addr = USBIN_AICL_OPTIONS_CFG,
- .mask = USBIN_AICL_START_AT_MAX_BIT | USBIN_AICL_ADC_EN_BIT |
- USBIN_AICL_EN_BIT | SUSPEND_ON_COLLAPSE_USBIN_BIT |
- USBIN_HV_COLLAPSE_RESPONSE_BIT |
- USBIN_LV_COLLAPSE_RESPONSE_BIT,
- .val = USBIN_HV_COLLAPSE_RESPONSE_BIT |
- USBIN_LV_COLLAPSE_RESPONSE_BIT | USBIN_AICL_EN_BIT },
- /*
- * Set pre charge current to default, the OnePlus 6 bootloader
- * sets this very conservatively.
- */
- { .addr = PRE_CHARGE_CURRENT_CFG,
- .mask = PRE_CHARGE_CURRENT_SETTING_MASK,
- .val = 500000 / CURRENT_SCALE_FACTOR },
- /*
- * This overrides all of the current limit options exposed to userspace
- * and prevents the device from pulling more than ~1A. This is done
- * to minimise potential fire hazard risks.
- */
- { .addr = FAST_CHARGE_CURRENT_CFG,
- .mask = FAST_CHARGE_CURRENT_SETTING_MASK,
- .val = 1000000 / CURRENT_SCALE_FACTOR },
-};
-
-static int smb2_init_hw(struct smb2_chip *chip)
-{
- int rc, i;
-
- for (i = 0; i < ARRAY_SIZE(smb2_init_seq); i++) {
- dev_dbg(chip->dev, "%d: Writing 0x%02x to 0x%02x\n", i,
- smb2_init_seq[i].val, smb2_init_seq[i].addr);
- rc = regmap_update_bits(chip->regmap,
- chip->base + smb2_init_seq[i].addr,
- smb2_init_seq[i].mask,
- smb2_init_seq[i].val);
- if (rc < 0)
- return dev_err_probe(chip->dev, rc,
- "%s: init command %d failed\n",
- __func__, i);
- }
-
- return 0;
-}
-
-static int smb2_init_irq(struct smb2_chip *chip, int *irq, const char *name,
- irqreturn_t (*handler)(int irq, void *data))
-{
- int irqnum;
- int rc;
-
- irqnum = platform_get_irq_byname(to_platform_device(chip->dev), name);
- if (irqnum < 0)
- return irqnum;
-
- rc = devm_request_threaded_irq(chip->dev, irqnum, NULL, handler,
- IRQF_ONESHOT, name, chip);
- if (rc < 0)
- return dev_err_probe(chip->dev, rc, "Couldn't request irq %s\n",
- name);
-
- if (irq)
- *irq = irqnum;
-
- return 0;
-}
-
-static int smb2_probe(struct platform_device *pdev)
-{
- struct power_supply_config supply_config = {};
- struct power_supply_desc *desc;
- struct smb2_chip *chip;
- int rc, irq;
-
- chip = devm_kzalloc(&pdev->dev, sizeof(*chip), GFP_KERNEL);
- if (!chip)
- return -ENOMEM;
-
- chip->dev = &pdev->dev;
- chip->name = pdev->name;
-
- chip->regmap = dev_get_regmap(pdev->dev.parent, NULL);
- if (!chip->regmap)
- return dev_err_probe(chip->dev, -ENODEV,
- "failed to locate the regmap\n");
-
- rc = device_property_read_u32(chip->dev, "reg", &chip->base);
- if (rc < 0)
- return dev_err_probe(chip->dev, rc,
- "Couldn't read base address\n");
-
- chip->usb_in_v_chan = devm_iio_channel_get(chip->dev, "usbin_v");
- if (IS_ERR(chip->usb_in_v_chan))
- return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_v_chan),
- "Couldn't get usbin_v IIO channel\n");
-
- chip->usb_in_i_chan = devm_iio_channel_get(chip->dev, "usbin_i");
- if (IS_ERR(chip->usb_in_i_chan)) {
- return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_i_chan),
- "Couldn't get usbin_i IIO channel\n");
- }
-
- rc = smb2_init_hw(chip);
- if (rc < 0)
- return rc;
-
- supply_config.drv_data = chip;
- supply_config.of_node = pdev->dev.of_node;
-
- desc = devm_kzalloc(chip->dev, sizeof(smb2_psy_desc), GFP_KERNEL);
- if (!desc)
- return -ENOMEM;
- memcpy(desc, &smb2_psy_desc, sizeof(smb2_psy_desc));
- desc->name =
- devm_kasprintf(chip->dev, GFP_KERNEL, "%s-charger",
- (const char *)device_get_match_data(chip->dev));
- if (!desc->name)
- return -ENOMEM;
-
- chip->chg_psy =
- devm_power_supply_register(chip->dev, desc, &supply_config);
- if (IS_ERR(chip->chg_psy))
- return dev_err_probe(chip->dev, PTR_ERR(chip->chg_psy),
- "failed to register power supply\n");
-
- rc = power_supply_get_battery_info(chip->chg_psy, &chip->batt_info);
- if (rc)
- return dev_err_probe(chip->dev, rc,
- "Failed to get battery info\n");
-
- rc = devm_delayed_work_autocancel(chip->dev, &chip->status_change_work,
- smb2_status_change_work);
- if (rc)
- return dev_err_probe(chip->dev, rc,
- "Failed to init status change work\n");
-
- rc = (chip->batt_info->voltage_max_design_uv -
- SMB2_FLOAT_VOLTAGE_MIN_UV) / SMB2_FLOAT_VOLTAGE_STEP_UV;
- rc = regmap_update_bits(chip->regmap, chip->base + FLOAT_VOLTAGE_CFG,
- FLOAT_VOLTAGE_SETTING_MASK, rc);
- if (rc < 0)
- return dev_err_probe(chip->dev, rc, "Couldn't set vbat max\n");
-
- rc = smb2_init_irq(chip, &irq, "bat-ov", smb2_handle_batt_overvoltage);
- if (rc < 0)
- return rc;
-
- rc = smb2_init_irq(chip, &chip->cable_irq, "usb-plugin",
- smb2_handle_usb_plugin);
- if (rc < 0)
- return rc;
-
- rc = smb2_init_irq(chip, &irq, "usbin-icl-change",
- smb2_handle_usb_icl_change);
- if (rc < 0)
- return rc;
- rc = smb2_init_irq(chip, &irq, "wdog-bark", smb2_handle_wdog_bark);
- if (rc < 0)
- return rc;
-
- rc = dev_pm_set_wake_irq(chip->dev, chip->cable_irq);
- if (rc < 0)
- return dev_err_probe(chip->dev, rc, "Couldn't set wake irq\n");
-
- platform_set_drvdata(pdev, chip);
-
- /* Initialise charger state */
- schedule_delayed_work(&chip->status_change_work, 0);
-
- return 0;
-}
-
-static const struct of_device_id smb2_match_id_table[] = {
- { .compatible = "qcom,pmi8998-charger", .data = "pmi8998" },
- { .compatible = "qcom,pm660-charger", .data = "pm660" },
- { /* sentinal */ }
-};
-MODULE_DEVICE_TABLE(of, smb2_match_id_table);
-
-static struct platform_driver qcom_spmi_smb2 = {
- .probe = smb2_probe,
- .driver = {
- .name = "qcom-pmi8998/pm660-charger",
- .of_match_table = smb2_match_id_table,
- },
-};
-
-module_platform_driver(qcom_spmi_smb2);
-
-MODULE_AUTHOR("Caleb Connolly <caleb.connolly@linaro.org>");
-MODULE_DESCRIPTION("Qualcomm SMB2 Charger Driver");
-MODULE_LICENSE("GPL");
diff --git a/drivers/power/supply/qcom_smbx.c b/drivers/power/supply/qcom_smbx.c
new file mode 100644
index 000000000000..099d1385ea12
--- /dev/null
+++ b/drivers/power/supply/qcom_smbx.c
@@ -0,0 +1,1108 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2016-2019 The Linux Foundation. All rights reserved.
+ * Copyright (c) 2023, Linaro Ltd.
+ * Author: Caleb Connolly <caleb.connolly@linaro.org>
+ *
+ * This driver is for the switch-mode battery charger and boost
+ * hardware found in pmi8998 and related PMICs.
+ */
+
+#include <linux/bits.h>
+#include <linux/devm-helpers.h>
+#include <linux/iio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/minmax.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_wakeirq.h>
+#include <linux/of.h>
+#include <linux/power_supply.h>
+#include <linux/regmap.h>
+#include <linux/types.h>
+#include <linux/workqueue.h>
+
+/* clang-format off */
+#define BATTERY_CHARGER_STATUS_1 0x06
+#define BVR_INITIAL_RAMP_BIT BIT(7)
+#define CC_SOFT_TERMINATE_BIT BIT(6)
+#define STEP_CHARGING_STATUS_SHIFT 3
+#define STEP_CHARGING_STATUS_MASK GENMASK(5, 3)
+#define BATTERY_CHARGER_STATUS_MASK GENMASK(2, 0)
+
+#define BATTERY_CHARGER_STATUS_2 0x07
+#define INPUT_CURRENT_LIMITED_BIT BIT(7)
+#define CHARGER_ERROR_STATUS_SFT_EXPIRE_BIT BIT(6)
+#define CHARGER_ERROR_STATUS_BAT_OV_BIT BIT(5)
+#define CHARGER_ERROR_STATUS_BAT_TERM_MISSING_BIT BIT(4)
+#define BAT_TEMP_STATUS_MASK GENMASK(3, 0)
+#define BAT_TEMP_STATUS_SOFT_LIMIT_MASK GENMASK(3, 2)
+#define BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT BIT(3)
+#define BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT BIT(2)
+#define BAT_TEMP_STATUS_TOO_HOT_BIT BIT(1)
+#define BAT_TEMP_STATUS_TOO_COLD_BIT BIT(0)
+
+#define BATTERY_CHARGER_STATUS_4 0x0A
+#define CHARGE_CURRENT_POST_JEITA_MASK GENMASK(7, 0)
+
+#define BATTERY_CHARGER_STATUS_7 0x0D
+#define ENABLE_TRICKLE_BIT BIT(7)
+#define ENABLE_PRE_CHARGING_BIT BIT(6)
+#define ENABLE_FAST_CHARGING_BIT BIT(5)
+#define ENABLE_FULLON_MODE_BIT BIT(4)
+#define TOO_COLD_ADC_BIT BIT(3)
+#define TOO_HOT_ADC_BIT BIT(2)
+#define HOT_SL_ADC_BIT BIT(1)
+#define COLD_SL_ADC_BIT BIT(0)
+
+#define CHARGING_ENABLE_CMD 0x42
+#define CHARGING_ENABLE_CMD_BIT BIT(0)
+
+#define CHGR_CFG2 0x51
+#define CHG_EN_SRC_BIT BIT(7)
+#define CHG_EN_POLARITY_BIT BIT(6)
+#define PRETOFAST_TRANSITION_CFG_BIT BIT(5)
+#define BAT_OV_ECC_BIT BIT(4)
+#define I_TERM_BIT BIT(3)
+#define AUTO_RECHG_BIT BIT(2)
+#define EN_ANALOG_DROP_IN_VBATT_BIT BIT(1)
+#define CHARGER_INHIBIT_BIT BIT(0)
+
+#define PRE_CHARGE_CURRENT_CFG 0x60
+#define PRE_CHARGE_CURRENT_SETTING_MASK GENMASK(5, 0)
+
+#define FAST_CHARGE_CURRENT_CFG 0x61
+#define FAST_CHARGE_CURRENT_SETTING_MASK GENMASK(7, 0)
+
+#define FLOAT_VOLTAGE_CFG 0x70
+#define FLOAT_VOLTAGE_SETTING_MASK GENMASK(7, 0)
+
+#define FG_UPDATE_CFG_2_SEL 0x7D
+#define SOC_LT_OTG_THRESH_SEL_BIT BIT(3)
+#define SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(2)
+#define VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(1)
+#define IBT_LT_CHG_TERM_THRESH_SEL_BIT BIT(0)
+
+#define JEITA_EN_CFG 0x90
+#define JEITA_EN_HARDLIMIT_BIT BIT(4)
+#define JEITA_EN_HOT_SL_FCV_BIT BIT(3)
+#define JEITA_EN_COLD_SL_FCV_BIT BIT(2)
+#define JEITA_EN_HOT_SL_CCC_BIT BIT(1)
+#define JEITA_EN_COLD_SL_CCC_BIT BIT(0)
+
+#define INT_RT_STS 0x310
+#define TYPE_C_CHANGE_RT_STS_BIT BIT(7)
+#define USBIN_ICL_CHANGE_RT_STS_BIT BIT(6)
+#define USBIN_SOURCE_CHANGE_RT_STS_BIT BIT(5)
+#define USBIN_PLUGIN_RT_STS_BIT BIT(4)
+#define USBIN_OV_RT_STS_BIT BIT(3)
+#define USBIN_UV_RT_STS_BIT BIT(2)
+#define USBIN_LT_3P6V_RT_STS_BIT BIT(1)
+#define USBIN_COLLAPSE_RT_STS_BIT BIT(0)
+
+#define OTG_CFG 0x153
+#define OTG_RESERVED_MASK GENMASK(7, 6)
+#define DIS_OTG_ON_TLIM_BIT BIT(5)
+#define QUICKSTART_OTG_FASTROLESWAP_BIT BIT(4)
+#define INCREASE_DFP_TIME_BIT BIT(3)
+#define ENABLE_OTG_IN_DEBUG_MODE_BIT BIT(2)
+#define OTG_EN_SRC_CFG_BIT BIT(1)
+#define CONCURRENT_MODE_CFG_BIT BIT(0)
+
+#define OTG_ENG_OTG_CFG 0x1C0
+#define ENG_BUCKBOOST_HALT1_8_MODE_BIT BIT(0)
+
+#define APSD_STATUS 0x307
+#define APSD_STATUS_7_BIT BIT(7)
+#define HVDCP_CHECK_TIMEOUT_BIT BIT(6)
+#define SLOW_PLUGIN_TIMEOUT_BIT BIT(5)
+#define ENUMERATION_DONE_BIT BIT(4)
+#define VADP_CHANGE_DONE_AFTER_AUTH_BIT BIT(3)
+#define QC_AUTH_DONE_STATUS_BIT BIT(2)
+#define QC_CHARGER_BIT BIT(1)
+#define APSD_DTC_STATUS_DONE_BIT BIT(0)
+
+#define APSD_RESULT_STATUS 0x308
+#define ICL_OVERRIDE_LATCH_BIT BIT(7)
+#define APSD_RESULT_STATUS_MASK GENMASK(6, 0)
+#define QC_3P0_BIT BIT(6)
+#define QC_2P0_BIT BIT(5)
+#define FLOAT_CHARGER_BIT BIT(4)
+#define DCP_CHARGER_BIT BIT(3)
+#define CDP_CHARGER_BIT BIT(2)
+#define OCP_CHARGER_BIT BIT(1)
+#define SDP_CHARGER_BIT BIT(0)
+
+#define TYPE_C_STATUS_1 0x30B
+#define UFP_TYPEC_MASK GENMASK(7, 5)
+#define UFP_TYPEC_RDSTD_BIT BIT(7)
+#define UFP_TYPEC_RD1P5_BIT BIT(6)
+#define UFP_TYPEC_RD3P0_BIT BIT(5)
+#define UFP_TYPEC_FMB_255K_BIT BIT(4)
+#define UFP_TYPEC_FMB_301K_BIT BIT(3)
+#define UFP_TYPEC_FMB_523K_BIT BIT(2)
+#define UFP_TYPEC_FMB_619K_BIT BIT(1)
+#define UFP_TYPEC_OPEN_OPEN_BIT BIT(0)
+
+#define TYPE_C_STATUS_2 0x30C
+#define DFP_RA_OPEN_BIT BIT(7)
+#define TIMER_STAGE_BIT BIT(6)
+#define EXIT_UFP_MODE_BIT BIT(5)
+#define EXIT_DFP_MODE_BIT BIT(4)
+#define DFP_TYPEC_MASK GENMASK(3, 0)
+#define DFP_RD_OPEN_BIT BIT(3)
+#define DFP_RD_RA_VCONN_BIT BIT(2)
+#define DFP_RD_RD_BIT BIT(1)
+#define DFP_RA_RA_BIT BIT(0)
+
+#define TYPE_C_STATUS_3 0x30D
+#define ENABLE_BANDGAP_BIT BIT(7)
+#define U_USB_GND_NOVBUS_BIT BIT(6)
+#define U_USB_FLOAT_NOVBUS_BIT BIT(5)
+#define U_USB_GND_BIT BIT(4)
+#define U_USB_FMB1_BIT BIT(3)
+#define U_USB_FLOAT1_BIT BIT(2)
+#define U_USB_FMB2_BIT BIT(1)
+#define U_USB_FLOAT2_BIT BIT(0)
+
+#define TYPE_C_STATUS_4 0x30E
+#define UFP_DFP_MODE_STATUS_BIT BIT(7)
+#define TYPEC_VBUS_STATUS_BIT BIT(6)
+#define TYPEC_VBUS_ERROR_STATUS_BIT BIT(5)
+#define TYPEC_DEBOUNCE_DONE_STATUS_BIT BIT(4)
+#define TYPEC_UFP_AUDIO_ADAPT_STATUS_BIT BIT(3)
+#define TYPEC_VCONN_OVERCURR_STATUS_BIT BIT(2)
+#define CC_ORIENTATION_BIT BIT(1)
+#define CC_ATTACHED_BIT BIT(0)
+
+#define TYPE_C_STATUS_5 0x30F
+#define TRY_SOURCE_FAILED_BIT BIT(6)
+#define TRY_SINK_FAILED_BIT BIT(5)
+#define TIMER_STAGE_2_BIT BIT(4)
+#define TYPEC_LEGACY_CABLE_STATUS_BIT BIT(3)
+#define TYPEC_NONCOMP_LEGACY_CABLE_STATUS_BIT BIT(2)
+#define TYPEC_TRYSOURCE_DETECT_STATUS_BIT BIT(1)
+#define TYPEC_TRYSINK_DETECT_STATUS_BIT BIT(0)
+
+#define CMD_APSD 0x341
+#define ICL_OVERRIDE_BIT BIT(1)
+#define APSD_RERUN_BIT BIT(0)
+
+#define TYPE_C_CFG 0x358
+#define APSD_START_ON_CC_BIT BIT(7)
+#define WAIT_FOR_APSD_BIT BIT(6)
+#define FACTORY_MODE_DETECTION_EN_BIT BIT(5)
+#define FACTORY_MODE_ICL_3A_4A_BIT BIT(4)
+#define FACTORY_MODE_DIS_CHGING_CFG_BIT BIT(3)
+#define SUSPEND_NON_COMPLIANT_CFG_BIT BIT(2)
+#define VCONN_OC_CFG_BIT BIT(1)
+#define TYPE_C_OR_U_USB_BIT BIT(0)
+
+#define TYPE_C_CFG_2 0x359
+#define TYPE_C_DFP_CURRSRC_MODE_BIT BIT(7)
+#define DFP_CC_1P4V_OR_1P6V_BIT BIT(6)
+#define VCONN_SOFTSTART_CFG_MASK GENMASK(5, 4)
+#define EN_TRY_SOURCE_MODE_BIT BIT(3)
+#define USB_FACTORY_MODE_ENABLE_BIT BIT(2)
+#define TYPE_C_UFP_MODE_BIT BIT(1)
+#define EN_80UA_180UA_CUR_SOURCE_BIT BIT(0)
+
+#define TYPE_C_CFG_3 0x35A
+#define TVBUS_DEBOUNCE_BIT BIT(7)
+#define TYPEC_LEGACY_CABLE_INT_EN_BIT BIT(6)
+#define TYPEC_NONCOMPLIANT_LEGACY_CABLE_INT_EN_B BIT(5)
+#define TYPEC_TRYSOURCE_DETECT_INT_EN_BIT BIT(4)
+#define TYPEC_TRYSINK_DETECT_INT_EN_BIT BIT(3)
+#define EN_TRYSINK_MODE_BIT BIT(2)
+#define EN_LEGACY_CABLE_DETECTION_BIT BIT(1)
+#define ALLOW_PD_DRING_UFP_TCCDB_BIT BIT(0)
+
+#define USBIN_OPTIONS_1_CFG 0x362
+#define CABLE_R_SEL_BIT BIT(7)
+#define HVDCP_AUTH_ALG_EN_CFG_BIT BIT(6)
+#define HVDCP_AUTONOMOUS_MODE_EN_CFG_BIT BIT(5)
+#define INPUT_PRIORITY_BIT BIT(4)
+#define AUTO_SRC_DETECT_BIT BIT(3)
+#define HVDCP_EN_BIT BIT(2)
+#define VADP_INCREMENT_VOLTAGE_LIMIT_BIT BIT(1)
+#define VADP_TAPER_TIMER_EN_BIT BIT(0)
+
+#define USBIN_OPTIONS_2_CFG 0x363
+#define WIPWR_RST_EUD_CFG_BIT BIT(7)
+#define SWITCHER_START_CFG_BIT BIT(6)
+#define DCD_TIMEOUT_SEL_BIT BIT(5)
+#define OCD_CURRENT_SEL_BIT BIT(4)
+#define SLOW_PLUGIN_TIMER_EN_CFG_BIT BIT(3)
+#define FLOAT_OPTIONS_MASK GENMASK(2, 0)
+#define FLOAT_DIS_CHGING_CFG_BIT BIT(2)
+#define SUSPEND_FLOAT_CFG_BIT BIT(1)
+#define FORCE_FLOAT_SDP_CFG_BIT BIT(0)
+
+#define TAPER_TIMER_SEL_CFG 0x364
+#define TYPEC_SPARE_CFG_BIT BIT(7)
+#define TYPEC_DRP_DFP_TIME_CFG_BIT BIT(5)
+#define TAPER_TIMER_SEL_MASK GENMASK(1, 0)
+
+#define USBIN_LOAD_CFG 0x365
+#define USBIN_OV_CH_LOAD_OPTION_BIT BIT(7)
+#define ICL_OVERRIDE_AFTER_APSD_BIT BIT(4)
+
+#define USBIN_ICL_OPTIONS 0x366
+#define CFG_USB3P0_SEL_BIT BIT(2)
+#define USB51_MODE_BIT BIT(1)
+#define USBIN_MODE_CHG_BIT BIT(0)
+
+#define TYPE_C_INTRPT_ENB_SOFTWARE_CTRL 0x368
+#define EXIT_SNK_BASED_ON_CC_BIT BIT(7)
+#define VCONN_EN_ORIENTATION_BIT BIT(6)
+#define TYPEC_VCONN_OVERCURR_INT_EN_BIT BIT(5)
+#define VCONN_EN_SRC_BIT BIT(4)
+#define VCONN_EN_VALUE_BIT BIT(3)
+#define TYPEC_POWER_ROLE_CMD_MASK GENMASK(2, 0)
+#define UFP_EN_CMD_BIT BIT(2)
+#define DFP_EN_CMD_BIT BIT(1)
+#define TYPEC_DISABLE_CMD_BIT BIT(0)
+
+#define USBIN_CURRENT_LIMIT_CFG 0x370
+#define USBIN_CURRENT_LIMIT_MASK GENMASK(7, 0)
+
+#define USBIN_AICL_OPTIONS_CFG 0x380
+#define SUSPEND_ON_COLLAPSE_USBIN_BIT BIT(7)
+#define USBIN_AICL_HDC_EN_BIT BIT(6)
+#define USBIN_AICL_START_AT_MAX_BIT BIT(5)
+#define USBIN_AICL_RERUN_EN_BIT BIT(4)
+#define USBIN_AICL_ADC_EN_BIT BIT(3)
+#define USBIN_AICL_EN_BIT BIT(2)
+#define USBIN_HV_COLLAPSE_RESPONSE_BIT BIT(1)
+#define USBIN_LV_COLLAPSE_RESPONSE_BIT BIT(0)
+
+#define USBIN_5V_AICL_THRESHOLD_CFG 0x381
+#define USBIN_5V_AICL_THRESHOLD_CFG_MASK GENMASK(2, 0)
+
+#define USBIN_CONT_AICL_THRESHOLD_CFG 0x384
+#define USBIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0)
+
+#define DC_ENG_SSUPPLY_CFG2 0x4C1
+#define ENG_SSUPPLY_IVREF_OTG_SS_MASK GENMASK(2, 0)
+#define OTG_SS_SLOW 0x3
+
+#define DCIN_AICL_REF_SEL_CFG 0x481
+#define DCIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0)
+
+#define WI_PWR_OPTIONS 0x495
+#define CHG_OK_BIT BIT(7)
+#define WIPWR_UVLO_IRQ_OPT_BIT BIT(6)
+#define BUCK_HOLDOFF_ENABLE_BIT BIT(5)
+#define CHG_OK_HW_SW_SELECT_BIT BIT(4)
+#define WIPWR_RST_ENABLE_BIT BIT(3)
+#define DCIN_WIPWR_IRQ_SELECT_BIT BIT(2)
+#define AICL_SWITCH_ENABLE_BIT BIT(1)
+#define ZIN_ICL_ENABLE_BIT BIT(0)
+
+#define ICL_STATUS 0x607
+#define INPUT_CURRENT_LIMIT_MASK GENMASK(7, 0)
+
+#define POWER_PATH_STATUS 0x60B
+#define P_PATH_INPUT_SS_DONE_BIT BIT(7)
+#define P_PATH_USBIN_SUSPEND_STS_BIT BIT(6)
+#define P_PATH_DCIN_SUSPEND_STS_BIT BIT(5)
+#define P_PATH_USE_USBIN_BIT BIT(4)
+#define P_PATH_USE_DCIN_BIT BIT(3)
+#define P_PATH_POWER_PATH_MASK GENMASK(2, 1)
+#define P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT BIT(0)
+
+#define BARK_BITE_WDOG_PET 0x643
+#define BARK_BITE_WDOG_PET_BIT BIT(0)
+
+#define WD_CFG 0x651
+#define WATCHDOG_TRIGGER_AFP_EN_BIT BIT(7)
+#define BARK_WDOG_INT_EN_BIT BIT(6)
+#define BITE_WDOG_INT_EN_BIT BIT(5)
+#define SFT_AFTER_WDOG_IRQ_MASK GENMASK(4, 3)
+#define WDOG_IRQ_SFT_BIT BIT(2)
+#define WDOG_TIMER_EN_ON_PLUGIN_BIT BIT(1)
+#define WDOG_TIMER_EN_BIT BIT(0)
+
+#define SNARL_BARK_BITE_WD_CFG 0x653
+#define BITE_WDOG_DISABLE_CHARGING_CFG_BIT BIT(7)
+#define SNARL_WDOG_TIMEOUT_MASK GENMASK(6, 4)
+#define BARK_WDOG_TIMEOUT_MASK GENMASK(3, 2)
+#define BITE_WDOG_TIMEOUT_MASK GENMASK(1, 0)
+
+#define AICL_RERUN_TIME_CFG 0x661
+#define AICL_RERUN_TIME_MASK GENMASK(1, 0)
+
+#define STAT_CFG 0x690
+#define STAT_SW_OVERRIDE_VALUE_BIT BIT(7)
+#define STAT_SW_OVERRIDE_CFG_BIT BIT(6)
+#define STAT_PARALLEL_OFF_DG_CFG_MASK GENMASK(5, 4)
+#define STAT_POLARITY_CFG_BIT BIT(3)
+#define STAT_PARALLEL_CFG_BIT BIT(2)
+#define STAT_FUNCTION_CFG_BIT BIT(1)
+#define STAT_IRQ_PULSING_EN_BIT BIT(0)
+
+#define SDP_CURRENT_UA 500000
+#define CDP_CURRENT_UA 1500000
+#define DCP_CURRENT_UA 1500000
+#define CURRENT_MAX_UA DCP_CURRENT_UA
+
+/* pmi8998 registers represent current in increments of 1/40th of an amp */
+#define CURRENT_SCALE_FACTOR 25000
+#define SMB2_FLOAT_VOLTAGE_MIN_UV 3487500
+#define SMB2_FLOAT_VOLTAGE_MAX_UV 4920000
+#define SMB2_FLOAT_VOLTAGE_STEP_UV 7500
+/* clang-format on */
+
+enum charger_status {
+ TRICKLE_CHARGE = 0,
+ PRE_CHARGE,
+ FAST_CHARGE,
+ FULLON_CHARGE,
+ TAPER_CHARGE,
+ TERMINATE_CHARGE,
+ INHIBIT_CHARGE,
+ DISABLE_CHARGE,
+};
+
+struct smb_init_register {
+ u16 addr;
+ u8 mask;
+ u8 val;
+};
+
+/**
+ * struct smb_chip - smb chip structure
+ * @dev: Device reference for power_supply
+ * @name: The platform device name
+ * @base: Base address for smb registers
+ * @regmap: Register map
+ * @batt_info: Battery data from DT
+ * @initial_charge_enable: Charging enable state before hardware setup
+ * @status_change_work: Worker to handle plug/unplug events
+ * @cable_irq: USB plugin IRQ
+ * @wakeup_enabled: If the cable IRQ will cause a wakeup
+ * @usb_in_i_chan: USB_IN current measurement channel
+ * @usb_in_v_chan: USB_IN voltage measurement channel
+ * @chg_psy: Charger power supply instance
+ */
+struct smb_chip {
+ struct device *dev;
+ const char *name;
+ unsigned int base;
+ struct regmap *regmap;
+ struct power_supply_battery_info *batt_info;
+ u8 initial_charge_enable;
+
+ struct delayed_work status_change_work;
+ int cable_irq;
+ bool wakeup_enabled;
+
+ struct iio_channel *usb_in_i_chan;
+ struct iio_channel *usb_in_v_chan;
+
+ struct power_supply *chg_psy;
+};
+
+static enum power_supply_property smb_properties[] = {
+ POWER_SUPPLY_PROP_MANUFACTURER,
+ POWER_SUPPLY_PROP_MODEL_NAME,
+ POWER_SUPPLY_PROP_CURRENT_MAX,
+ POWER_SUPPLY_PROP_CURRENT_NOW,
+ POWER_SUPPLY_PROP_VOLTAGE_NOW,
+ POWER_SUPPLY_PROP_STATUS,
+ POWER_SUPPLY_PROP_HEALTH,
+ POWER_SUPPLY_PROP_ONLINE,
+ POWER_SUPPLY_PROP_USB_TYPE,
+};
+
+static int smb_get_prop_usb_online(struct smb_chip *chip, int *val)
+{
+ unsigned int stat;
+ int rc;
+
+ rc = regmap_read(chip->regmap, chip->base + POWER_PATH_STATUS, &stat);
+ if (rc < 0) {
+ dev_err(chip->dev, "Couldn't read power path status: %d\n", rc);
+ return rc;
+ }
+
+ *val = (stat & P_PATH_USE_USBIN_BIT) &&
+ (stat & P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT);
+ return 0;
+}
+
+/*
+ * Qualcomm "automatic power source detection" aka APSD
+ * tells us what type of charger we're connected to.
+ */
+static int smb_apsd_get_charger_type(struct smb_chip *chip, int *val)
+{
+ unsigned int apsd_stat, stat;
+ int usb_online = 0;
+ int rc;
+
+ rc = smb_get_prop_usb_online(chip, &usb_online);
+ if (!usb_online) {
+ *val = POWER_SUPPLY_USB_TYPE_UNKNOWN;
+ return rc;
+ }
+
+ rc = regmap_read(chip->regmap, chip->base + APSD_STATUS, &apsd_stat);
+ if (rc < 0) {
+ dev_err(chip->dev, "Failed to read apsd status, rc = %d", rc);
+ return rc;
+ }
+ if (!(apsd_stat & APSD_DTC_STATUS_DONE_BIT)) {
+ dev_dbg(chip->dev, "Apsd not ready");
+ return -EAGAIN;
+ }
+
+ rc = regmap_read(chip->regmap, chip->base + APSD_RESULT_STATUS, &stat);
+ if (rc < 0) {
+ dev_err(chip->dev, "Failed to read apsd result, rc = %d", rc);
+ return rc;
+ }
+
+ stat &= APSD_RESULT_STATUS_MASK;
+
+ if (stat & CDP_CHARGER_BIT)
+ *val = POWER_SUPPLY_USB_TYPE_CDP;
+ else if (stat & (DCP_CHARGER_BIT | OCP_CHARGER_BIT | FLOAT_CHARGER_BIT))
+ *val = POWER_SUPPLY_USB_TYPE_DCP;
+ else /* SDP_CHARGER_BIT (or others) */
+ *val = POWER_SUPPLY_USB_TYPE_SDP;
+
+ return 0;
+}
+
+static int smb_get_prop_status(struct smb_chip *chip, int *val)
+{
+ unsigned char stat[2];
+ int usb_online = 0;
+ int rc;
+
+ rc = smb_get_prop_usb_online(chip, &usb_online);
+ if (!usb_online) {
+ *val = POWER_SUPPLY_STATUS_DISCHARGING;
+ return rc;
+ }
+
+ rc = regmap_bulk_read(chip->regmap,
+ chip->base + BATTERY_CHARGER_STATUS_1, &stat, 2);
+ if (rc < 0) {
+ dev_err(chip->dev, "Failed to read charging status ret=%d\n",
+ rc);
+ return rc;
+ }
+
+ if (stat[1] & CHARGER_ERROR_STATUS_BAT_OV_BIT) {
+ *val = POWER_SUPPLY_STATUS_NOT_CHARGING;
+ return 0;
+ }
+
+ stat[0] = stat[0] & BATTERY_CHARGER_STATUS_MASK;
+
+ switch (stat[0]) {
+ case TRICKLE_CHARGE:
+ case PRE_CHARGE:
+ case FAST_CHARGE:
+ case FULLON_CHARGE:
+ case TAPER_CHARGE:
+ *val = POWER_SUPPLY_STATUS_CHARGING;
+ return rc;
+ case DISABLE_CHARGE:
+ *val = POWER_SUPPLY_STATUS_NOT_CHARGING;
+ return rc;
+ case TERMINATE_CHARGE:
+ case INHIBIT_CHARGE:
+ *val = POWER_SUPPLY_STATUS_FULL;
+ return rc;
+ default:
+ *val = POWER_SUPPLY_STATUS_UNKNOWN;
+ return rc;
+ }
+}
+
+static inline int smb_get_current_limit(struct smb_chip *chip,
+ unsigned int *val)
+{
+ int rc = regmap_read(chip->regmap, chip->base + ICL_STATUS, val);
+
+ if (rc >= 0)
+ *val *= CURRENT_SCALE_FACTOR;
+ return rc;
+}
+
+static int smb_set_current_limit(struct smb_chip *chip, unsigned int val)
+{
+ unsigned char val_raw;
+
+ if (val > 4800000) {
+ dev_err(chip->dev,
+ "Can't set current limit higher than 4800000uA");
+ return -EINVAL;
+ }
+ val_raw = val / CURRENT_SCALE_FACTOR;
+
+ return regmap_write(chip->regmap, chip->base + USBIN_CURRENT_LIMIT_CFG,
+ val_raw);
+}
+
+static void smb_status_change_work(struct work_struct *work)
+{
+ unsigned int charger_type, current_ua;
+ int usb_online = 0;
+ int count, rc;
+ struct smb_chip *chip;
+
+ chip = container_of(work, struct smb_chip, status_change_work.work);
+
+ smb_get_prop_usb_online(chip, &usb_online);
+ if (!usb_online)
+ return;
+
+ for (count = 0; count < 3; count++) {
+ dev_dbg(chip->dev, "get charger type retry %d\n", count);
+ rc = smb_apsd_get_charger_type(chip, &charger_type);
+ if (rc != -EAGAIN)
+ break;
+ msleep(100);
+ }
+
+ if (rc < 0 && rc != -EAGAIN) {
+ dev_err(chip->dev, "get charger type failed: %d\n", rc);
+ return;
+ }
+
+ if (rc < 0) {
+ rc = regmap_update_bits(chip->regmap, chip->base + CMD_APSD,
+ APSD_RERUN_BIT, APSD_RERUN_BIT);
+ schedule_delayed_work(&chip->status_change_work,
+ msecs_to_jiffies(1000));
+ dev_dbg(chip->dev, "get charger type failed, rerun apsd\n");
+ return;
+ }
+
+ switch (charger_type) {
+ case POWER_SUPPLY_USB_TYPE_CDP:
+ current_ua = CDP_CURRENT_UA;
+ break;
+ case POWER_SUPPLY_USB_TYPE_DCP:
+ current_ua = DCP_CURRENT_UA;
+ break;
+ case POWER_SUPPLY_USB_TYPE_SDP:
+ default:
+ current_ua = SDP_CURRENT_UA;
+ break;
+ }
+
+ smb_set_current_limit(chip, current_ua);
+ power_supply_changed(chip->chg_psy);
+}
+
+static int smb_get_iio_chan(struct smb_chip *chip, struct iio_channel *chan,
+ int *val)
+{
+ int rc;
+ union power_supply_propval status;
+
+ rc = power_supply_get_property(chip->chg_psy, POWER_SUPPLY_PROP_STATUS,
+ &status);
+ if (rc < 0 || status.intval != POWER_SUPPLY_STATUS_CHARGING) {
+ *val = 0;
+ return 0;
+ }
+
+ if (IS_ERR(chan)) {
+ dev_err(chip->dev, "Failed to chan, err = %li", PTR_ERR(chan));
+ return PTR_ERR(chan);
+ }
+
+ return iio_read_channel_processed(chan, val);
+}
+
+static int smb_get_prop_health(struct smb_chip *chip, int *val)
+{
+ int rc;
+ unsigned int stat;
+
+ rc = regmap_read(chip->regmap, chip->base + BATTERY_CHARGER_STATUS_2,
+ &stat);
+ if (rc < 0) {
+ dev_err(chip->dev, "Couldn't read charger status rc=%d\n", rc);
+ return rc;
+ }
+
+ if (stat & CHARGER_ERROR_STATUS_BAT_OV_BIT)
+ *val = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
+ else if (stat & BAT_TEMP_STATUS_TOO_COLD_BIT)
+ *val = POWER_SUPPLY_HEALTH_COLD;
+ else if (stat & BAT_TEMP_STATUS_TOO_HOT_BIT)
+ *val = POWER_SUPPLY_HEALTH_OVERHEAT;
+ else if (stat & BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT)
+ *val = POWER_SUPPLY_HEALTH_COOL;
+ else if (stat & BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT)
+ *val = POWER_SUPPLY_HEALTH_WARM;
+ else
+ *val = POWER_SUPPLY_HEALTH_GOOD;
+
+ return 0;
+}
+
+static int smb_get_property(struct power_supply *psy,
+ enum power_supply_property psp,
+ union power_supply_propval *val)
+{
+ struct smb_chip *chip = power_supply_get_drvdata(psy);
+
+ switch (psp) {
+ case POWER_SUPPLY_PROP_MANUFACTURER:
+ val->strval = "Qualcomm";
+ return 0;
+ case POWER_SUPPLY_PROP_MODEL_NAME:
+ val->strval = chip->name;
+ return 0;
+ case POWER_SUPPLY_PROP_CURRENT_MAX:
+ return smb_get_current_limit(chip, &val->intval);
+ case POWER_SUPPLY_PROP_CURRENT_NOW:
+ return smb_get_iio_chan(chip, chip->usb_in_i_chan,
+ &val->intval);
+ case POWER_SUPPLY_PROP_VOLTAGE_NOW:
+ return smb_get_iio_chan(chip, chip->usb_in_v_chan,
+ &val->intval);
+ case POWER_SUPPLY_PROP_ONLINE:
+ return smb_get_prop_usb_online(chip, &val->intval);
+ case POWER_SUPPLY_PROP_STATUS:
+ return smb_get_prop_status(chip, &val->intval);
+ case POWER_SUPPLY_PROP_HEALTH:
+ return smb_get_prop_health(chip, &val->intval);
+ case POWER_SUPPLY_PROP_USB_TYPE:
+ return smb_apsd_get_charger_type(chip, &val->intval);
+ default:
+ dev_err(chip->dev, "invalid property: %d\n", psp);
+ return -EINVAL;
+ }
+}
+
+static int smb_set_property(struct power_supply *psy,
+ enum power_supply_property psp,
+ const union power_supply_propval *val)
+{
+ struct smb_chip *chip = power_supply_get_drvdata(psy);
+
+ switch (psp) {
+ case POWER_SUPPLY_PROP_CURRENT_MAX:
+ return smb_set_current_limit(chip, val->intval);
+ default:
+ dev_err(chip->dev, "No setter for property: %d\n", psp);
+ return -EINVAL;
+ }
+}
+
+static int smb_property_is_writable(struct power_supply *psy,
+ enum power_supply_property psp)
+{
+ switch (psp) {
+ case POWER_SUPPLY_PROP_CURRENT_MAX:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+static irqreturn_t smb_handle_batt_overvoltage(int irq, void *data)
+{
+ struct smb_chip *chip = data;
+ unsigned int status;
+ int rc;
+
+ rc = regmap_read(chip->regmap,
+ chip->base + BATTERY_CHARGER_STATUS_2, &status);
+ if (rc < 0) {
+ dev_err(chip->dev, "Couldn't read charger status: %d\n", rc);
+ return IRQ_HANDLED;
+ }
+
+ if (status & CHARGER_ERROR_STATUS_BAT_OV_BIT) {
+ /* The hardware stops charging automatically */
+ dev_err(chip->dev, "battery overvoltage detected\n");
+ }
+
+ power_supply_changed(chip->chg_psy);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t smb_handle_usb_plugin(int irq, void *data)
+{
+ struct smb_chip *chip = data;
+
+ power_supply_changed(chip->chg_psy);
+
+ schedule_delayed_work(&chip->status_change_work,
+ msecs_to_jiffies(1500));
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t smb_handle_usb_icl_change(int irq, void *data)
+{
+ struct smb_chip *chip = data;
+
+ power_supply_changed(chip->chg_psy);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t smb_handle_wdog_bark(int irq, void *data)
+{
+ struct smb_chip *chip = data;
+ int rc;
+
+ power_supply_changed(chip->chg_psy);
+
+ rc = regmap_write(chip->regmap, chip->base + BARK_BITE_WDOG_PET,
+ BARK_BITE_WDOG_PET_BIT);
+ if (rc < 0)
+ dev_err(chip->dev, "Couldn't pet the dog rc=%d\n", rc);
+
+ return IRQ_HANDLED;
+}
+
+static const struct power_supply_desc smb_psy_desc = {
+ .name = "pmi8998_charger",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .usb_types = BIT(POWER_SUPPLY_USB_TYPE_SDP) |
+ BIT(POWER_SUPPLY_USB_TYPE_CDP) |
+ BIT(POWER_SUPPLY_USB_TYPE_DCP) |
+ BIT(POWER_SUPPLY_USB_TYPE_UNKNOWN),
+ .properties = smb_properties,
+ .num_properties = ARRAY_SIZE(smb_properties),
+ .get_property = smb_get_property,
+ .set_property = smb_set_property,
+ .property_is_writeable = smb_property_is_writable,
+};
+
+/* Init sequence derived from vendor downstream driver */
+static const struct smb_init_register smb_init_seq[] = {
+ { .addr = AICL_RERUN_TIME_CFG, .mask = AICL_RERUN_TIME_MASK, .val = 0 },
+ /*
+ * By default configure us as an upstream facing port
+ * FIXME: This will be handled by the type-c driver
+ */
+ { .addr = TYPE_C_INTRPT_ENB_SOFTWARE_CTRL,
+ .mask = TYPEC_POWER_ROLE_CMD_MASK | VCONN_EN_SRC_BIT |
+ VCONN_EN_VALUE_BIT,
+ .val = VCONN_EN_SRC_BIT },
+ /*
+ * Disable Type-C factory mode and stay in Attached.SRC state when VCONN
+ * over-current happens
+ */
+ { .addr = TYPE_C_CFG,
+ .mask = FACTORY_MODE_DETECTION_EN_BIT | VCONN_OC_CFG_BIT,
+ .val = 0 },
+ /* Configure VBUS for software control */
+ { .addr = OTG_CFG, .mask = OTG_EN_SRC_CFG_BIT, .val = 0 },
+ /*
+ * Use VBAT to determine the recharge threshold when battery is full
+ * rather than the state of charge.
+ */
+ { .addr = FG_UPDATE_CFG_2_SEL,
+ .mask = SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT |
+ VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT,
+ .val = VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT },
+ /* Enable charging */
+ { .addr = USBIN_OPTIONS_1_CFG, .mask = HVDCP_EN_BIT, .val = 0 },
+ { .addr = CHARGING_ENABLE_CMD,
+ .mask = CHARGING_ENABLE_CMD_BIT,
+ .val = CHARGING_ENABLE_CMD_BIT },
+ /*
+ * Match downstream defaults
+ * CHG_EN_SRC_BIT - charger enable is controlled by software
+ * CHG_EN_POLARITY_BIT - polarity of charge enable pin when in HW control
+ * pulled low on OnePlus 6 and SHIFT6mq
+ * PRETOFAST_TRANSITION_CFG_BIT -
+ * BAT_OV_ECC_BIT -
+ * I_TERM_BIT - Current termination ?? 0 = enabled
+ * AUTO_RECHG_BIT - Enable automatic recharge when battery is full
+ * 0 = enabled
+ * EN_ANALOG_DROP_IN_VBATT_BIT
+ * CHARGER_INHIBIT_BIT - Inhibit charging based on battery voltage
+ * instead of ??
+ */
+ { .addr = CHGR_CFG2,
+ .mask = CHG_EN_SRC_BIT | CHG_EN_POLARITY_BIT |
+ PRETOFAST_TRANSITION_CFG_BIT | BAT_OV_ECC_BIT | I_TERM_BIT |
+ AUTO_RECHG_BIT | EN_ANALOG_DROP_IN_VBATT_BIT |
+ CHARGER_INHIBIT_BIT,
+ .val = CHARGER_INHIBIT_BIT },
+ /* STAT pin software override, match downstream. Parallell charging? */
+ { .addr = STAT_CFG,
+ .mask = STAT_SW_OVERRIDE_CFG_BIT,
+ .val = STAT_SW_OVERRIDE_CFG_BIT },
+ /* Set the default SDP charger type to a 500ma USB 2.0 port */
+ { .addr = USBIN_ICL_OPTIONS,
+ .mask = USB51_MODE_BIT | USBIN_MODE_CHG_BIT,
+ .val = USB51_MODE_BIT },
+ /* Disable watchdog */
+ { .addr = SNARL_BARK_BITE_WD_CFG, .mask = 0xff, .val = 0 },
+ { .addr = WD_CFG,
+ .mask = WATCHDOG_TRIGGER_AFP_EN_BIT | WDOG_TIMER_EN_ON_PLUGIN_BIT |
+ BARK_WDOG_INT_EN_BIT,
+ .val = 0 },
+ /* These bits aren't documented anywhere */
+ { .addr = USBIN_5V_AICL_THRESHOLD_CFG,
+ .mask = USBIN_5V_AICL_THRESHOLD_CFG_MASK,
+ .val = 0x3 },
+ { .addr = USBIN_CONT_AICL_THRESHOLD_CFG,
+ .mask = USBIN_CONT_AICL_THRESHOLD_CFG_MASK,
+ .val = 0x3 },
+ /*
+ * Enable Automatic Input Current Limit, this will slowly ramp up the current
+ * When connected to a wall charger, and automatically stop when it detects
+ * the charger current limit (voltage drop?) or it reaches the programmed limit.
+ */
+ { .addr = USBIN_AICL_OPTIONS_CFG,
+ .mask = USBIN_AICL_START_AT_MAX_BIT | USBIN_AICL_ADC_EN_BIT |
+ USBIN_AICL_EN_BIT | SUSPEND_ON_COLLAPSE_USBIN_BIT |
+ USBIN_HV_COLLAPSE_RESPONSE_BIT |
+ USBIN_LV_COLLAPSE_RESPONSE_BIT,
+ .val = USBIN_HV_COLLAPSE_RESPONSE_BIT |
+ USBIN_LV_COLLAPSE_RESPONSE_BIT | USBIN_AICL_EN_BIT },
+ /*
+ * Set pre charge current to default, the OnePlus 6 bootloader
+ * sets this very conservatively.
+ */
+ { .addr = PRE_CHARGE_CURRENT_CFG,
+ .mask = PRE_CHARGE_CURRENT_SETTING_MASK,
+ .val = 500000 / CURRENT_SCALE_FACTOR },
+ /*
+ * This overrides all of the current limit options exposed to userspace
+ * and prevents the device from pulling more than ~1A. This is done
+ * to minimise potential fire hazard risks.
+ */
+ { .addr = FAST_CHARGE_CURRENT_CFG,
+ .mask = FAST_CHARGE_CURRENT_SETTING_MASK,
+ .val = 1000000 / CURRENT_SCALE_FACTOR },
+};
+
+static int smb_init_hw(struct smb_chip *chip)
+{
+ int rc, i;
+
+ for (i = 0; i < ARRAY_SIZE(smb_init_seq); i++) {
+ dev_dbg(chip->dev, "%d: Writing 0x%02x to 0x%02x\n", i,
+ smb_init_seq[i].val, smb_init_seq[i].addr);
+ rc = regmap_update_bits(chip->regmap,
+ chip->base + smb_init_seq[i].addr,
+ smb_init_seq[i].mask,
+ smb_init_seq[i].val);
+ if (rc < 0)
+ return dev_err_probe(chip->dev, rc,
+ "%s: init command %d failed\n",
+ __func__, i);
+ }
+
+ return 0;
+}
+
+static void smb_restore_charge_enable(void *data)
+{
+ struct smb_chip *chip = data;
+ int rc;
+
+ rc = regmap_update_bits(chip->regmap,
+ chip->base + CHARGING_ENABLE_CMD,
+ CHARGING_ENABLE_CMD_BIT,
+ chip->initial_charge_enable);
+ if (rc < 0)
+ dev_err(chip->dev, "Couldn't restore charging state: %d\n", rc);
+}
+
+static int smb_init_irq(struct smb_chip *chip, int *irq, const char *name,
+ irqreturn_t (*handler)(int irq, void *data))
+{
+ int irqnum;
+ int rc;
+
+ irqnum = platform_get_irq_byname(to_platform_device(chip->dev), name);
+ if (irqnum < 0)
+ return irqnum;
+
+ rc = devm_request_threaded_irq(chip->dev, irqnum, NULL, handler,
+ IRQF_ONESHOT, name, chip);
+ if (rc < 0)
+ return dev_err_probe(chip->dev, rc, "Couldn't request irq %s\n",
+ name);
+
+ if (irq)
+ *irq = irqnum;
+
+ return 0;
+}
+
+static int smb_probe(struct platform_device *pdev)
+{
+ struct power_supply_config supply_config = {};
+ struct power_supply_desc *desc;
+ struct smb_chip *chip;
+ unsigned int charge_enable;
+ unsigned int float_voltage_sel;
+ int float_voltage_uv;
+ int rc, irq;
+
+ chip = devm_kzalloc(&pdev->dev, sizeof(*chip), GFP_KERNEL);
+ if (!chip)
+ return -ENOMEM;
+
+ chip->dev = &pdev->dev;
+ chip->name = pdev->name;
+
+ chip->regmap = dev_get_regmap(pdev->dev.parent, NULL);
+ if (!chip->regmap)
+ return dev_err_probe(chip->dev, -ENODEV,
+ "failed to locate the regmap\n");
+
+ rc = device_property_read_u32(chip->dev, "reg", &chip->base);
+ if (rc < 0)
+ return dev_err_probe(chip->dev, rc,
+ "Couldn't read base address\n");
+
+ chip->usb_in_v_chan = devm_iio_channel_get(chip->dev, "usbin_v");
+ if (IS_ERR(chip->usb_in_v_chan))
+ return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_v_chan),
+ "Couldn't get usbin_v IIO channel\n");
+
+ chip->usb_in_i_chan = devm_iio_channel_get(chip->dev, "usbin_i");
+ if (IS_ERR(chip->usb_in_i_chan)) {
+ return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_i_chan),
+ "Couldn't get usbin_i IIO channel\n");
+ }
+
+ rc = regmap_read(chip->regmap, chip->base + CHARGING_ENABLE_CMD,
+ &charge_enable);
+ if (rc < 0)
+ return dev_err_probe(chip->dev, rc,
+ "Couldn't read charging state\n");
+
+ chip->initial_charge_enable = charge_enable & CHARGING_ENABLE_CMD_BIT;
+ rc = devm_add_action_or_reset(chip->dev, smb_restore_charge_enable, chip);
+ if (rc)
+ return dev_err_probe(chip->dev, rc,
+ "Couldn't register charging state rollback\n");
+
+ rc = smb_init_hw(chip);
+ if (rc < 0)
+ return rc;
+
+ supply_config.drv_data = chip;
+ supply_config.of_node = pdev->dev.of_node;
+
+ desc = devm_kzalloc(chip->dev, sizeof(smb_psy_desc), GFP_KERNEL);
+ if (!desc)
+ return -ENOMEM;
+ memcpy(desc, &smb_psy_desc, sizeof(smb_psy_desc));
+ desc->name =
+ devm_kasprintf(chip->dev, GFP_KERNEL, "%s-charger",
+ (const char *)device_get_match_data(chip->dev));
+ if (!desc->name)
+ return -ENOMEM;
+
+ chip->chg_psy =
+ devm_power_supply_register(chip->dev, desc, &supply_config);
+ if (IS_ERR(chip->chg_psy))
+ return dev_err_probe(chip->dev, PTR_ERR(chip->chg_psy),
+ "failed to register power supply\n");
+
+ rc = power_supply_get_battery_info(chip->chg_psy, &chip->batt_info);
+ if (rc)
+ return dev_err_probe(chip->dev, rc,
+ "Failed to get battery info\n");
+
+ rc = devm_delayed_work_autocancel(chip->dev, &chip->status_change_work,
+ smb_status_change_work);
+ if (rc)
+ return dev_err_probe(chip->dev, rc,
+ "Failed to init status change work\n");
+
+ if (power_supply_battery_info_has_prop(chip->batt_info,
+ POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX)) {
+ float_voltage_uv =
+ chip->batt_info->constant_charge_voltage_max_uv;
+ } else if (power_supply_battery_info_has_prop(chip->batt_info,
+ POWER_SUPPLY_PROP_VOLTAGE_MAX_DESIGN)) {
+ float_voltage_uv = chip->batt_info->voltage_max_design_uv;
+ } else {
+ dev_warn(chip->dev, "No battery float voltage; preserving hardware setting\n");
+ goto skip_float_voltage;
+ }
+
+ if (float_voltage_uv < SMB2_FLOAT_VOLTAGE_MIN_UV ||
+ float_voltage_uv > SMB2_FLOAT_VOLTAGE_MAX_UV)
+ return dev_err_probe(chip->dev, -EINVAL,
+ "float voltage %d uV outside %d-%d uV\n",
+ float_voltage_uv,
+ SMB2_FLOAT_VOLTAGE_MIN_UV,
+ SMB2_FLOAT_VOLTAGE_MAX_UV);
+
+ float_voltage_sel =
+ (float_voltage_uv - SMB2_FLOAT_VOLTAGE_MIN_UV) /
+ SMB2_FLOAT_VOLTAGE_STEP_UV;
+ rc = regmap_update_bits(chip->regmap, chip->base + FLOAT_VOLTAGE_CFG,
+ FLOAT_VOLTAGE_SETTING_MASK, float_voltage_sel);
+ if (rc < 0)
+ return dev_err_probe(chip->dev, rc, "Couldn't set vbat max\n");
+
+skip_float_voltage:
+ rc = smb_init_irq(chip, &irq, "bat-ov", smb_handle_batt_overvoltage);
+ if (rc < 0)
+ return rc;
+
+ rc = smb_init_irq(chip, &chip->cable_irq, "usb-plugin",
+ smb_handle_usb_plugin);
+ if (rc < 0)
+ return rc;
+
+ rc = smb_init_irq(chip, &irq, "usbin-icl-change",
+ smb_handle_usb_icl_change);
+ if (rc < 0)
+ return rc;
+ rc = smb_init_irq(chip, &irq, "wdog-bark", smb_handle_wdog_bark);
+ if (rc < 0)
+ return rc;
+
+ rc = dev_pm_set_wake_irq(chip->dev, chip->cable_irq);
+ if (rc < 0)
+ return dev_err_probe(chip->dev, rc, "Couldn't set wake irq\n");
+
+ devm_remove_action(chip->dev, smb_restore_charge_enable, chip);
+
+ platform_set_drvdata(pdev, chip);
+
+ /* Initialise charger state */
+ schedule_delayed_work(&chip->status_change_work, 0);
+
+ return 0;
+}
+
+static const struct of_device_id smb_match_id_table[] = {
+ { .compatible = "qcom,pmi8998-charger", .data = "pmi8998" },
+ { .compatible = "qcom,pm660-charger", .data = "pm660" },
+ { /* sentinal */ }
+};
+MODULE_DEVICE_TABLE(of, smb_match_id_table);
+
+static struct platform_driver qcom_spmi_smb = {
+ .probe = smb_probe,
+ .driver = {
+ .name = "qcom-smbx-charger",
+ .of_match_table = smb_match_id_table,
+ },
+};
+
+module_platform_driver(qcom_spmi_smb);
+
+MODULE_AUTHOR("Caleb Connolly <caleb.connolly@linaro.org>");
+MODULE_DESCRIPTION("Qualcomm SMB2 Charger Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/power/supply/rt9467-charger.c b/drivers/power/supply/rt9467-charger.c
index be65b0f51721..be1ca7f61c0b 100644
--- a/drivers/power/supply/rt9467-charger.c
+++ b/drivers/power/supply/rt9467-charger.c
@@ -1145,27 +1145,6 @@ static int rt9467_reset_chip(struct rt9467_chg_data *data)
return regmap_field_write(data->rm_field[F_RST], 1);
}
-static void rt9467_chg_destroy_adc_lock(void *data)
-{
- struct mutex *adc_lock = data;
-
- mutex_destroy(adc_lock);
-}
-
-static void rt9467_chg_destroy_attach_lock(void *data)
-{
- struct mutex *attach_lock = data;
-
- mutex_destroy(attach_lock);
-}
-
-static void rt9467_chg_destroy_ichg_ieoc_lock(void *data)
-{
- struct mutex *ichg_ieoc_lock = data;
-
- mutex_destroy(ichg_ieoc_lock);
-}
-
static void rt9467_chg_complete_aicl_done(void *data)
{
struct completion *aicl_done = data;
@@ -1218,21 +1197,15 @@ static int rt9467_charger_probe(struct i2c_client *i2c)
if (ret)
return dev_err_probe(dev, ret, "Failed to add irq chip\n");
- mutex_init(&data->adc_lock);
- ret = devm_add_action_or_reset(dev, rt9467_chg_destroy_adc_lock,
- &data->adc_lock);
+ ret = devm_mutex_init(dev, &data->adc_lock);
if (ret)
return dev_err_probe(dev, ret, "Failed to init ADC lock\n");
- mutex_init(&data->attach_lock);
- ret = devm_add_action_or_reset(dev, rt9467_chg_destroy_attach_lock,
- &data->attach_lock);
+ ret = devm_mutex_init(dev, &data->attach_lock);
if (ret)
return dev_err_probe(dev, ret, "Failed to init attach lock\n");
- mutex_init(&data->ichg_ieoc_lock);
- ret = devm_add_action_or_reset(dev, rt9467_chg_destroy_ichg_ieoc_lock,
- &data->ichg_ieoc_lock);
+ ret = devm_mutex_init(dev, &data->ichg_ieoc_lock);
if (ret)
return dev_err_probe(dev, ret, "Failed to init ICHG/IEOC lock\n");
diff --git a/drivers/ptp/ptp_sysfs.c b/drivers/ptp/ptp_sysfs.c
index 200eaf500696..0d4b081daf63 100644
--- a/drivers/ptp/ptp_sysfs.c
+++ b/drivers/ptp/ptp_sysfs.c
@@ -80,13 +80,17 @@ static ssize_t extts_fifo_show(struct device *dev,
size_t qcnt;
int cnt = 0;
+ spin_lock_irqsave(&ptp->tsevqs_lock, flags);
cnt = list_count_nodes(&ptp->tsevqs);
- if (cnt <= 0)
+ if (cnt <= 0) {
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
goto out;
+ }
/* The sysfs fifo will always draw from the fist queue */
queue = list_first_entry(&ptp->tsevqs, struct timestamp_event_queue,
qlist);
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
memset(&event, 0, sizeof(event));
spin_lock_irqsave(&queue->lock, flags);
diff --git a/drivers/s390/char/vmcp.c b/drivers/s390/char/vmcp.c
index c6d58335beb4..4fdc3f1dc8db 100644
--- a/drivers/s390/char/vmcp.c
+++ b/drivers/s390/char/vmcp.c
@@ -105,13 +105,11 @@ static int vmcp_open(struct inode *inode, struct file *file)
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- session = kmalloc(sizeof(*session), GFP_KERNEL);
+ session = kzalloc(sizeof(*session), GFP_KERNEL);
if (!session)
return -ENOMEM;
session->bufsize = PAGE_SIZE;
- session->response = NULL;
- session->resp_size = 0;
mutex_init(&session->mutex);
file->private_data = session;
return nonseekable_open(inode, file);
diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c
index 3e3100dbfda3..287df8723810 100644
--- a/drivers/scsi/aic7xxx/aic79xx_core.c
+++ b/drivers/scsi/aic7xxx/aic79xx_core.c
@@ -8942,8 +8942,10 @@ ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb)
}
#endif
- if (ahd_perform_autosense(scb) == 0)
+ if (ahd_perform_autosense(scb) == 0) {
+ ahd_done(ahd, scb);
break;
+ }
ahd_compile_devinfo(&devinfo, SCB_GET_OUR_ID(scb),
SCB_GET_TARGET(ahd, scb),
diff --git a/drivers/scsi/aic7xxx/aic79xx_osm.h b/drivers/scsi/aic7xxx/aic79xx_osm.h
index 793fe19993a9..3bcfcfe18f08 100644
--- a/drivers/scsi/aic7xxx/aic79xx_osm.h
+++ b/drivers/scsi/aic7xxx/aic79xx_osm.h
@@ -499,6 +499,8 @@ int ahd_linux_show_info(struct seq_file *,struct Scsi_Host *);
static inline
void ahd_cmd_set_transaction_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~(CAM_STATUS_MASK << 16);
cmd->result |= status << 16;
}
@@ -512,6 +514,8 @@ void ahd_set_transaction_status(struct scb *scb, uint32_t status)
static inline
void ahd_cmd_set_scsi_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~0xFFFF;
cmd->result |= status;
}
@@ -525,6 +529,8 @@ void ahd_set_scsi_status(struct scb *scb, uint32_t status)
static inline
uint32_t ahd_cmd_get_transaction_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return CAM_REQ_INPROG;
return ((cmd->result >> 16) & CAM_STATUS_MASK);
}
@@ -537,6 +543,8 @@ uint32_t ahd_get_transaction_status(struct scb *scb)
static inline
uint32_t ahd_cmd_get_scsi_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return SAM_STAT_GOOD;
return (cmd->result & 0xFFFF);
}
@@ -564,13 +572,16 @@ u_long ahd_get_transfer_length(struct scb *scb)
static inline
int ahd_get_transfer_dir(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return DMA_NONE;
return (scb->io_ctx->sc_data_direction);
}
static inline
void ahd_set_residual(struct scb *scb, u_long resid)
{
- scsi_set_resid(scb->io_ctx, resid);
+ if (scb->io_ctx)
+ scsi_set_resid(scb->io_ctx, resid);
}
static inline
@@ -582,6 +593,8 @@ void ahd_set_sense_residual(struct scb *scb, u_long resid)
static inline
u_long ahd_get_residual(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return 0;
return scsi_get_resid(scb->io_ctx);
}
@@ -595,11 +608,12 @@ static inline
int ahd_perform_autosense(struct scb *scb)
{
/*
- * We always perform autosense in Linux.
- * On other platforms this is set on a
- * per-transaction basis.
+ * We always perform autosense in Linux, except for the device
+ * reset SCB, which has no command to return sense data to.
*/
- return (1);
+ if (!scb->io_ctx)
+ return 0;
+ return 1;
}
static inline uint32_t
@@ -629,6 +643,8 @@ void ahd_platform_freeze_devq(struct ahd_softc *ahd, struct scb *scb);
static inline void
ahd_freeze_scb(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return;
if ((scb->io_ctx->result & (CAM_DEV_QFRZN << 16)) == 0) {
scb->io_ctx->result |= CAM_DEV_QFRZN << 16;
scb->platform_data->dev->qfrozen++;
diff --git a/drivers/scsi/aic7xxx/aic7xxx_core.c b/drivers/scsi/aic7xxx/aic7xxx_core.c
index a396f048a031..d747d0b72a13 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_core.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_core.c
@@ -5266,9 +5266,13 @@ ahc_init(struct ahc_softc *ahc)
/*
* Assume we have a board at this stage and it has been reset.
+ * PCI controllers keep the host ID ahc_pci_config() chose.
*/
- if ((ahc->flags & AHC_USEDEFAULTS) != 0)
- ahc->our_id = ahc->our_id_b = 7;
+ if ((ahc->flags & AHC_USEDEFAULTS) != 0) {
+ if ((ahc->chip & AHC_PCI) == 0)
+ ahc->our_id = 7;
+ ahc->our_id_b = 7;
+ }
/*
* Default to allowing initiator operations.
diff --git a/drivers/scsi/aic7xxx/aic7xxx_pci.c b/drivers/scsi/aic7xxx/aic7xxx_pci.c
index 2d4c85426dc3..5a04c7d55e4b 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_pci.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_pci.c
@@ -701,6 +701,21 @@ ahc_find_pci_device(ahc_dev_softc_t pci)
return (NULL);
}
+/*
+ * The SGI O2 firmware runs its onboard controllers, in slots 1 and 2 of the
+ * root bus, without the Adaptec BIOS and at host ID 0. It leaves no settings
+ * in scratch RAM, which may still hold power-on contents with bad parity.
+ */
+static bool
+ahc_pci_sgi_o2_onboard(struct ahc_softc *ahc)
+{
+ int slot = ahc_get_pci_slot(ahc->dev_softc);
+
+ return IS_ENABLED(CONFIG_SGI_IP32) &&
+ ahc_get_pci_bus(ahc->dev_softc) == 0 &&
+ (slot == 1 || slot == 2);
+}
+
int
ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
{
@@ -782,6 +797,9 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
ahc->bus_intr = ahc_pci_intr;
ahc->bus_chip_init = ahc_pci_chip_init;
+ if (ahc_pci_sgi_o2_onboard(ahc))
+ ahc->flags |= AHC_NO_BIOS_INIT;
+
/* Remember how the card was setup in case there is no SEEPROM */
if ((ahc_inb(ahc, HCNTRL) & POWRDN) == 0) {
ahc_pause(ahc);
@@ -898,7 +916,7 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
* Assume only one connector and always turn
* on termination.
*/
- our_id = 0x07;
+ our_id = ahc_pci_sgi_o2_onboard(ahc) ? 0 : 0x07;
sxfrctl1 = STPWEN;
}
ahc_outb(ahc, SCSICONF, our_id|ENSPCHK|RESET_SCSI);
diff --git a/drivers/scsi/libsas/sas_ata.c b/drivers/scsi/libsas/sas_ata.c
index 7b4e7a61965a..adb9e7a94785 100644
--- a/drivers/scsi/libsas/sas_ata.c
+++ b/drivers/scsi/libsas/sas_ata.c
@@ -559,8 +559,8 @@ static int sas_ata_prereset(struct ata_link *link, unsigned long deadline)
}
static struct ata_port_operations sas_sata_ops = {
- .prereset = sas_ata_prereset,
- .hardreset = sas_ata_hard_reset,
+ .reset.prereset = sas_ata_prereset,
+ .reset.hardreset = sas_ata_hard_reset,
.error_handler = ata_std_error_handler,
.post_internal_cmd = sas_ata_post_internal,
.qc_defer = ata_std_qc_defer,
diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c
index a9a9ec086a7e..e2e4b8bb559f 100644
--- a/drivers/scsi/scsi_bsg.c
+++ b/drivers/scsi/scsi_bsg.c
@@ -15,7 +15,7 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
struct scsi_cmnd *scmd;
struct request *rq;
struct bio *bio;
- int ret;
+ int ret, unmap_ret;
if (hdr->protocol != BSG_PROTOCOL_SCSI ||
hdr->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD)
@@ -89,7 +89,9 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
else
hdr->dout_resid = scmd->resid_len;
- blk_rq_unmap_user(bio);
+ unmap_ret = blk_rq_unmap_user(bio);
+ if (!ret)
+ ret = unmap_ret;
out_put_request:
blk_mq_free_request(rq);
diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c
index 6f6c5973c3ea..1e969f78f060 100644
--- a/drivers/scsi/scsi_ioctl.c
+++ b/drivers/scsi/scsi_ioctl.c
@@ -9,6 +9,7 @@
#include <linux/blkdev.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
+#include <linux/jiffies.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/mm.h>
@@ -180,10 +181,14 @@ static int sg_set_timeout(struct scsi_device *sdev, int __user *p)
{
int timeout, err = get_user(timeout, p);
- if (!err)
- sdev->sg_timeout = clock_t_to_jiffies(timeout);
+ if (err)
+ return err;
+ if (timeout < 0)
+ return -EIO;
- return err;
+ sdev->sg_timeout = min(clock_t_to_jiffies(timeout),
+ min(UINT_MAX, MAX_JIFFY_OFFSET));
+ return 0;
}
static int sg_get_reserved_size(struct scsi_device *sdev, int __user *p)
diff --git a/drivers/soc/qcom/pmic_glink_altmode.c b/drivers/soc/qcom/pmic_glink_altmode.c
index db25f406878b..f650c8de9639 100644
--- a/drivers/soc/qcom/pmic_glink_altmode.c
+++ b/drivers/soc/qcom/pmic_glink_altmode.c
@@ -304,7 +304,7 @@ static void pmic_glink_altmode_sc8180xp_notify(struct pmic_glink_altmode *altmod
alt_port->mode = mode;
alt_port->hpd_state = hpd_state;
alt_port->hpd_irq = hpd_irq;
- schedule_work(&alt_port->work);
+ queue_work(system_freezable_wq, &alt_port->work);
}
#define SC8280XP_DPAM_MASK 0x3f
@@ -347,7 +347,7 @@ static void pmic_glink_altmode_sc8280xp_notify(struct pmic_glink_altmode *altmod
alt_port->mode = mode;
alt_port->hpd_state = hpd_state;
alt_port->hpd_irq = hpd_irq;
- schedule_work(&alt_port->work);
+ queue_work(system_freezable_wq, &alt_port->work);
}
static void pmic_glink_altmode_callback(const void *data, size_t len, void *priv)
diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c
index c3676b21e816..ca6af39e977a 100644
--- a/drivers/spi/spi-cadence-xspi.c
+++ b/drivers/spi/spi-cadence-xspi.c
@@ -2,7 +2,6 @@
// Cadence XSPI flash controller driver
// Copyright (C) 2020-21 Cadence
-#include <linux/acpi.h>
#include <linux/completion.h>
#include <linux/delay.h>
#include <linux/err.h>
@@ -21,6 +20,7 @@
#include <linux/limits.h>
#include <linux/log2.h>
#include <linux/bitrev.h>
+#include <linux/util_macros.h>
#define CDNS_XSPI_MAGIC_NUM_VALUE 0x6522
#define CDNS_XSPI_MAX_BANKS 8
@@ -316,39 +316,10 @@ struct cdns_xspi_driver_data {
u32 rfile_phy_dll_slave_ctrl;
};
-static struct cdns_xspi_driver_data marvell_driver_data = {
- .mrvl_hw_overlay = true,
- .dll_phy_ctrl = MARVELL_REGS_DLL_PHY_CTRL,
- .ctb_rfile_phy_ctrl = MARVELL_CTB_RFILE_PHY_CTRL,
- .rfile_phy_tsel = MARVELL_RFILE_PHY_TSEL,
- .rfile_phy_dq_timing = MARVELL_RFILE_PHY_DQ_TIMING,
- .rfile_phy_dqs_timing = MARVELL_RFILE_PHY_DQS_TIMING,
- .rfile_phy_gate_lpbk_ctrl = MARVELL_RFILE_PHY_GATE_LPBK_CTRL,
- .rfile_phy_dll_master_ctrl = MARVELL_RFILE_PHY_DLL_MASTER_CTRL,
- .rfile_phy_dll_slave_ctrl = MARVELL_RFILE_PHY_DLL_SLAVE_CTRL,
-};
-
static struct cdns_xspi_driver_data cdns_driver_data = {
.mrvl_hw_overlay = false,
};
-static const int cdns_mrvl_xspi_clk_div_list[] = {
- 4, //0x0 = Divide by 4. SPI clock is 200 MHz.
- 6, //0x1 = Divide by 6. SPI clock is 133.33 MHz.
- 8, //0x2 = Divide by 8. SPI clock is 100 MHz.
- 10, //0x3 = Divide by 10. SPI clock is 80 MHz.
- 12, //0x4 = Divide by 12. SPI clock is 66.666 MHz.
- 16, //0x5 = Divide by 16. SPI clock is 50 MHz.
- 18, //0x6 = Divide by 18. SPI clock is 44.44 MHz.
- 20, //0x7 = Divide by 20. SPI clock is 40 MHz.
- 24, //0x8 = Divide by 24. SPI clock is 33.33 MHz.
- 32, //0x9 = Divide by 32. SPI clock is 25 MHz.
- 40, //0xA = Divide by 40. SPI clock is 20 MHz.
- 50, //0xB = Divide by 50. SPI clock is 16 MHz.
- 64, //0xC = Divide by 64. SPI clock is 12.5 MHz.
- 128 //0xD = Divide by 128. SPI clock is 6.25 MHz.
-};
-
struct cdns_xspi_dev {
struct platform_device *pdev;
struct device *dev;
@@ -361,6 +332,7 @@ struct cdns_xspi_dev {
int irq;
int cur_cs;
unsigned int sdmasize;
+ unsigned int sdma_xfer_len;
struct completion cmd_complete;
struct completion auto_cmd_complete;
@@ -373,100 +345,13 @@ struct cdns_xspi_dev {
u8 hw_num_banks;
const struct cdns_xspi_driver_data *driver_data;
- void (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi);
+ int (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi);
void (*set_interrupts_handler)(struct cdns_xspi_dev *cdns_xspi, bool enabled);
bool xfer_in_progress;
int current_xfer_qword;
};
-static void cdns_xspi_reset_dll(struct cdns_xspi_dev *cdns_xspi)
-{
- u32 dll_cntrl = readl(cdns_xspi->iobase +
- CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
-
- /* Reset DLL */
- dll_cntrl |= CDNS_XSPI_DLL_RST_N;
- writel(dll_cntrl, cdns_xspi->iobase +
- CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
-}
-
-static bool cdns_xspi_is_dll_locked(struct cdns_xspi_dev *cdns_xspi)
-{
- u32 dll_lock;
-
- return !readl_relaxed_poll_timeout(cdns_xspi->iobase +
- CDNS_XSPI_INTR_STATUS_REG,
- dll_lock, ((dll_lock & CDNS_XSPI_DLL_LOCK) == 1), 10, 10000);
-}
-
-/* Static configuration of PHY */
-static bool cdns_xspi_configure_phy(struct cdns_xspi_dev *cdns_xspi)
-{
- writel(cdns_xspi->driver_data->dll_phy_ctrl,
- cdns_xspi->iobase + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
- writel(cdns_xspi->driver_data->ctb_rfile_phy_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_CTRL);
- writel(cdns_xspi->driver_data->rfile_phy_tsel,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_TSEL);
- writel(cdns_xspi->driver_data->rfile_phy_dq_timing,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQ_TIMING);
- writel(cdns_xspi->driver_data->rfile_phy_dqs_timing,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQS_TIMING);
- writel(cdns_xspi->driver_data->rfile_phy_gate_lpbk_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_GATE_LPBK_CTRL);
- writel(cdns_xspi->driver_data->rfile_phy_dll_master_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_MASTER_CTRL);
- writel(cdns_xspi->driver_data->rfile_phy_dll_slave_ctrl,
- cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_SLAVE_CTRL);
-
- cdns_xspi_reset_dll(cdns_xspi);
-
- return cdns_xspi_is_dll_locked(cdns_xspi);
-}
-
-static bool cdns_mrvl_xspi_setup_clock(struct cdns_xspi_dev *cdns_xspi,
- int requested_clk)
-{
- int i = 0;
- int clk_val;
- u32 clk_reg;
- bool update_clk = false;
-
- while (i < (ARRAY_SIZE(cdns_mrvl_xspi_clk_div_list) - 1)) {
- clk_val = MRVL_XSPI_CLOCK_DIVIDED(
- cdns_mrvl_xspi_clk_div_list[i]);
- if (clk_val <= requested_clk)
- break;
- i++;
- }
-
- dev_dbg(cdns_xspi->dev, "Found clk div: %d, clk val: %d\n",
- cdns_mrvl_xspi_clk_div_list[i],
- MRVL_XSPI_CLOCK_DIVIDED(
- cdns_mrvl_xspi_clk_div_list[i]));
-
- clk_reg = readl(cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
-
- if (FIELD_GET(MRVL_XSPI_CLK_DIV, clk_reg) != i) {
- clk_reg &= ~MRVL_XSPI_CLK_ENABLE;
- writel(clk_reg,
- cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
- clk_reg = FIELD_PREP(MRVL_XSPI_CLK_DIV, i);
- clk_reg &= ~MRVL_XSPI_CLK_DIV;
- clk_reg |= FIELD_PREP(MRVL_XSPI_CLK_DIV, i);
- clk_reg |= MRVL_XSPI_CLK_ENABLE;
- clk_reg |= MRVL_XSPI_IRQ_ENABLE;
- update_clk = true;
- }
-
- if (update_clk)
- writel(clk_reg,
- cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
-
- return update_clk;
-}
-
static int cdns_xspi_wait_for_controller_idle(struct cdns_xspi_dev *cdns_xspi)
{
u32 ctrl_stat;
@@ -539,23 +424,6 @@ static void cdns_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
}
-static void marvell_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
- bool enabled)
-{
- u32 intr_enable;
- u32 irq_status;
-
- irq_status = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
- writel(irq_status, cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
-
- intr_enable = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
- if (enabled)
- intr_enable |= CDNS_XSPI_INTR_MASK;
- else
- intr_enable &= ~CDNS_XSPI_INTR_MASK;
- writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
-}
-
static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi)
{
u32 ctrl_ver;
@@ -578,7 +446,7 @@ static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi)
return 0;
}
-static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
+static int cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
{
u32 sdma_size, sdma_trd_info;
u8 sdma_dir;
@@ -587,6 +455,13 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
+ if (sdma_size > cdns_xspi->sdma_xfer_len) {
+ dev_err(cdns_xspi->dev,
+ "SDMA size %u exceeds the requested length %u\n",
+ sdma_size, cdns_xspi->sdma_xfer_len);
+ return -EINVAL;
+ }
+
switch (sdma_dir) {
case CDNS_XSPI_SDMA_DIR_READ:
ioread8_rep(cdns_xspi->sdmabase,
@@ -598,78 +473,8 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
cdns_xspi->out_buffer, sdma_size);
break;
}
-}
-
-static void m_ioreadq(void __iomem *addr, void *buf, int len)
-{
- if (IS_ALIGNED((long)buf, 8) && len >= 8) {
- u64 full_ops = len / 8;
- u64 *buffer = buf;
-
- len -= full_ops * 8;
- buf += full_ops * 8;
-
- do {
- u64 b = readq(addr);
- *buffer++ = b;
- } while (--full_ops);
- }
-
-
- while (len) {
- u64 tmp_buf;
-
- tmp_buf = readq(addr);
- memcpy(buf, &tmp_buf, min(len, 8));
- len = len > 8 ? len - 8 : 0;
- buf += 8;
- }
-}
-
-static void m_iowriteq(void __iomem *addr, const void *buf, int len)
-{
- if (IS_ALIGNED((long)buf, 8) && len >= 8) {
- u64 full_ops = len / 8;
- const u64 *buffer = buf;
-
- len -= full_ops * 8;
- buf += full_ops * 8;
-
- do {
- writeq(*buffer++, addr);
- } while (--full_ops);
- }
- while (len) {
- u64 tmp_buf;
-
- memcpy(&tmp_buf, buf, min(len, 8));
- writeq(tmp_buf, addr);
- len = len > 8 ? len - 8 : 0;
- buf += 8;
- }
-}
-
-static void marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
-{
- u32 sdma_size, sdma_trd_info;
- u8 sdma_dir;
-
- sdma_size = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_SIZE_REG);
- sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
- sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
-
- switch (sdma_dir) {
- case CDNS_XSPI_SDMA_DIR_READ:
- m_ioreadq(cdns_xspi->sdmabase,
- cdns_xspi->in_buffer, sdma_size);
- break;
-
- case CDNS_XSPI_SDMA_DIR_WRITE:
- m_iowriteq(cdns_xspi->sdmabase,
- cdns_xspi->out_buffer, sdma_size);
- break;
- }
+ return 0;
}
static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
@@ -715,6 +520,7 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
cdns_xspi->in_buffer = op->data.buf.in;
cdns_xspi->out_buffer = op->data.buf.out;
+ cdns_xspi->sdma_xfer_len = op->data.nbytes;
cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
@@ -723,7 +529,11 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
cdns_xspi->set_interrupts_handler(cdns_xspi, false);
return -EIO;
}
- cdns_xspi->sdma_handler(cdns_xspi);
+ ret = cdns_xspi->sdma_handler(cdns_xspi);
+ if (ret) {
+ cdns_xspi->set_interrupts_handler(cdns_xspi, false);
+ return ret;
+ }
}
wait_for_completion(&cdns_xspi->cmd_complete);
@@ -761,33 +571,15 @@ static int cdns_xspi_mem_op_execute(struct spi_mem *mem,
return ret;
}
-static int marvell_xspi_mem_op_execute(struct spi_mem *mem,
- const struct spi_mem_op *op)
-{
- struct cdns_xspi_dev *cdns_xspi =
- spi_controller_get_devdata(mem->spi->controller);
- int ret = 0;
-
- cdns_mrvl_xspi_setup_clock(cdns_xspi, mem->spi->max_speed_hz);
-
- ret = cdns_xspi_mem_op(cdns_xspi, mem, op);
-
- return ret;
-}
-
-#ifdef CONFIG_ACPI
static bool cdns_xspi_supports_op(struct spi_mem *mem,
const struct spi_mem_op *op)
{
struct spi_device *spi = mem->spi;
- const union acpi_object *obj;
- struct acpi_device *adev;
+ struct device *dev = &spi->dev;
+ u32 value;
- adev = ACPI_COMPANION(&spi->dev);
-
- if (!acpi_dev_get_property(adev, "spi-tx-bus-width", ACPI_TYPE_INTEGER,
- &obj)) {
- switch (obj->integer.value) {
+ if (!device_property_read_u32(dev, "spi-tx-bus-width", &value)) {
+ switch (value) {
case 1:
break;
case 2:
@@ -800,16 +592,13 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem,
spi->mode |= SPI_TX_OCTAL;
break;
default:
- dev_warn(&spi->dev,
- "spi-tx-bus-width %lld not supported\n",
- obj->integer.value);
+ dev_warn(dev, "spi-tx-bus-width %u not supported\n", value);
break;
}
}
- if (!acpi_dev_get_property(adev, "spi-rx-bus-width", ACPI_TYPE_INTEGER,
- &obj)) {
- switch (obj->integer.value) {
+ if (!device_property_read_u32(dev, "spi-rx-bus-width", &value)) {
+ switch (value) {
case 1:
break;
case 2:
@@ -822,9 +611,7 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem,
spi->mode |= SPI_RX_OCTAL;
break;
default:
- dev_warn(&spi->dev,
- "spi-rx-bus-width %lld not supported\n",
- obj->integer.value);
+ dev_warn(dev, "spi-rx-bus-width %u not supported\n", value);
break;
}
}
@@ -834,7 +621,6 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem,
return true;
}
-#endif
static int cdns_xspi_adjust_mem_op_size(struct spi_mem *mem, struct spi_mem_op *op)
{
@@ -847,21 +633,11 @@ static int cdns_xspi_adjust_mem_op_size(struct spi_mem *mem, struct spi_mem_op *
}
static const struct spi_controller_mem_ops cadence_xspi_mem_ops = {
-#ifdef CONFIG_ACPI
- .supports_op = cdns_xspi_supports_op,
-#endif
+ .supports_op = PTR_IF(IS_ENABLED(CONFIG_ACPI), cdns_xspi_supports_op),
.exec_op = cdns_xspi_mem_op_execute,
.adjust_op_size = cdns_xspi_adjust_mem_op_size,
};
-static const struct spi_controller_mem_ops marvell_xspi_mem_ops = {
-#ifdef CONFIG_ACPI
- .supports_op = cdns_xspi_supports_op,
-#endif
- .exec_op = marvell_xspi_mem_op_execute,
- .adjust_op_size = cdns_xspi_adjust_mem_op_size,
-};
-
static irqreturn_t cdns_xspi_irq_handler(int this_irq, void *dev)
{
struct cdns_xspi_dev *cdns_xspi = dev;
@@ -943,6 +719,239 @@ static void cdns_xspi_print_phy_config(struct cdns_xspi_dev *cdns_xspi)
readl(cdns_xspi->auxbase + CDNS_XSPI_CCP_PHY_DLL_SLAVE_CTRL));
}
+static struct cdns_xspi_driver_data marvell_driver_data = {
+ .mrvl_hw_overlay = true,
+ .dll_phy_ctrl = MARVELL_REGS_DLL_PHY_CTRL,
+ .ctb_rfile_phy_ctrl = MARVELL_CTB_RFILE_PHY_CTRL,
+ .rfile_phy_tsel = MARVELL_RFILE_PHY_TSEL,
+ .rfile_phy_dq_timing = MARVELL_RFILE_PHY_DQ_TIMING,
+ .rfile_phy_dqs_timing = MARVELL_RFILE_PHY_DQS_TIMING,
+ .rfile_phy_gate_lpbk_ctrl = MARVELL_RFILE_PHY_GATE_LPBK_CTRL,
+ .rfile_phy_dll_master_ctrl = MARVELL_RFILE_PHY_DLL_MASTER_CTRL,
+ .rfile_phy_dll_slave_ctrl = MARVELL_RFILE_PHY_DLL_SLAVE_CTRL,
+};
+
+static const int cdns_mrvl_xspi_clk_div_list[] = {
+ 4, //0x0 = Divide by 4. SPI clock is 200 MHz.
+ 6, //0x1 = Divide by 6. SPI clock is 133.33 MHz.
+ 8, //0x2 = Divide by 8. SPI clock is 100 MHz.
+ 10, //0x3 = Divide by 10. SPI clock is 80 MHz.
+ 12, //0x4 = Divide by 12. SPI clock is 66.666 MHz.
+ 16, //0x5 = Divide by 16. SPI clock is 50 MHz.
+ 18, //0x6 = Divide by 18. SPI clock is 44.44 MHz.
+ 20, //0x7 = Divide by 20. SPI clock is 40 MHz.
+ 24, //0x8 = Divide by 24. SPI clock is 33.33 MHz.
+ 32, //0x9 = Divide by 32. SPI clock is 25 MHz.
+ 40, //0xA = Divide by 40. SPI clock is 20 MHz.
+ 50, //0xB = Divide by 50. SPI clock is 16 MHz.
+ 64, //0xC = Divide by 64. SPI clock is 12.5 MHz.
+ 128 //0xD = Divide by 128. SPI clock is 6.25 MHz.
+};
+
+static void cdns_xspi_reset_dll(struct cdns_xspi_dev *cdns_xspi)
+{
+ u32 dll_cntrl = readl(cdns_xspi->iobase +
+ CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
+
+ /* Reset DLL */
+ dll_cntrl |= CDNS_XSPI_DLL_RST_N;
+ writel(dll_cntrl, cdns_xspi->iobase +
+ CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
+}
+
+static bool cdns_xspi_is_dll_locked(struct cdns_xspi_dev *cdns_xspi)
+{
+ u32 dll_lock;
+
+ return !readl_relaxed_poll_timeout(cdns_xspi->iobase +
+ CDNS_XSPI_INTR_STATUS_REG,
+ dll_lock, ((dll_lock & CDNS_XSPI_DLL_LOCK) == 1), 10, 10000);
+}
+
+/* Static configuration of PHY */
+static bool cdns_xspi_configure_phy(struct cdns_xspi_dev *cdns_xspi)
+{
+ writel(cdns_xspi->driver_data->dll_phy_ctrl,
+ cdns_xspi->iobase + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL);
+ writel(cdns_xspi->driver_data->ctb_rfile_phy_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_CTRL);
+ writel(cdns_xspi->driver_data->rfile_phy_tsel,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_TSEL);
+ writel(cdns_xspi->driver_data->rfile_phy_dq_timing,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQ_TIMING);
+ writel(cdns_xspi->driver_data->rfile_phy_dqs_timing,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQS_TIMING);
+ writel(cdns_xspi->driver_data->rfile_phy_gate_lpbk_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_GATE_LPBK_CTRL);
+ writel(cdns_xspi->driver_data->rfile_phy_dll_master_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_MASTER_CTRL);
+ writel(cdns_xspi->driver_data->rfile_phy_dll_slave_ctrl,
+ cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_SLAVE_CTRL);
+
+ cdns_xspi_reset_dll(cdns_xspi);
+
+ return cdns_xspi_is_dll_locked(cdns_xspi);
+}
+
+static bool cdns_mrvl_xspi_setup_clock(struct cdns_xspi_dev *cdns_xspi,
+ int requested_clk)
+{
+ int i = 0;
+ int clk_val;
+ u32 clk_reg;
+ bool update_clk = false;
+
+ while (i < (ARRAY_SIZE(cdns_mrvl_xspi_clk_div_list) - 1)) {
+ clk_val = MRVL_XSPI_CLOCK_DIVIDED(
+ cdns_mrvl_xspi_clk_div_list[i]);
+ if (clk_val <= requested_clk)
+ break;
+ i++;
+ }
+
+ dev_dbg(cdns_xspi->dev, "Found clk div: %d, clk val: %d\n",
+ cdns_mrvl_xspi_clk_div_list[i],
+ MRVL_XSPI_CLOCK_DIVIDED(
+ cdns_mrvl_xspi_clk_div_list[i]));
+
+ clk_reg = readl(cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
+
+ if (FIELD_GET(MRVL_XSPI_CLK_DIV, clk_reg) != i) {
+ clk_reg &= ~MRVL_XSPI_CLK_ENABLE;
+ writel(clk_reg,
+ cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
+ clk_reg = FIELD_PREP(MRVL_XSPI_CLK_DIV, i);
+ FIELD_MODIFY(MRVL_XSPI_CLK_DIV, &clk_reg, i);
+ clk_reg |= MRVL_XSPI_CLK_ENABLE;
+ clk_reg |= MRVL_XSPI_IRQ_ENABLE;
+ update_clk = true;
+ }
+
+ if (update_clk)
+ writel(clk_reg,
+ cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG);
+
+ return update_clk;
+}
+
+static void marvell_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
+ bool enabled)
+{
+ u32 intr_enable;
+ u32 irq_status;
+
+ irq_status = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
+ writel(irq_status, cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG);
+
+ intr_enable = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
+ if (enabled)
+ intr_enable |= CDNS_XSPI_INTR_MASK;
+ else
+ intr_enable &= ~CDNS_XSPI_INTR_MASK;
+ writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
+}
+
+static int marvell_xspi_mem_op_execute(struct spi_mem *mem,
+ const struct spi_mem_op *op)
+{
+ struct cdns_xspi_dev *cdns_xspi =
+ spi_controller_get_devdata(mem->spi->controller);
+ int ret = 0;
+
+ cdns_mrvl_xspi_setup_clock(cdns_xspi, mem->spi->max_speed_hz);
+
+ ret = cdns_xspi_mem_op(cdns_xspi, mem, op);
+
+ return ret;
+}
+
+static void m_ioreadq(void __iomem *addr, void *buf, int len)
+{
+ if (IS_ALIGNED((long)buf, 8) && len >= 8) {
+ u64 full_ops = len / 8;
+ u64 *buffer = buf;
+
+ len -= full_ops * 8;
+ buf += full_ops * 8;
+
+ do {
+ u64 b = readq(addr);
+ *buffer++ = b;
+ } while (--full_ops);
+ }
+
+
+ while (len) {
+ u64 tmp_buf;
+
+ tmp_buf = readq(addr);
+ memcpy(buf, &tmp_buf, min(len, 8));
+ len = len > 8 ? len - 8 : 0;
+ buf += 8;
+ }
+}
+
+static void m_iowriteq(void __iomem *addr, const void *buf, int len)
+{
+ if (IS_ALIGNED((long)buf, 8) && len >= 8) {
+ u64 full_ops = len / 8;
+ const u64 *buffer = buf;
+
+ len -= full_ops * 8;
+ buf += full_ops * 8;
+
+ do {
+ writeq(*buffer++, addr);
+ } while (--full_ops);
+ }
+
+ while (len) {
+ u64 tmp_buf;
+
+ memcpy(&tmp_buf, buf, min(len, 8));
+ writeq(tmp_buf, addr);
+ len = len > 8 ? len - 8 : 0;
+ buf += 8;
+ }
+}
+
+static int marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
+{
+ u32 sdma_size, sdma_trd_info;
+ u8 sdma_dir;
+
+ sdma_size = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_SIZE_REG);
+ sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
+ sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
+
+ if (sdma_size > cdns_xspi->sdma_xfer_len) {
+ dev_err(cdns_xspi->dev,
+ "SDMA size %u exceeds the requested length %u\n",
+ sdma_size, cdns_xspi->sdma_xfer_len);
+ return -EINVAL;
+ }
+
+ switch (sdma_dir) {
+ case CDNS_XSPI_SDMA_DIR_READ:
+ m_ioreadq(cdns_xspi->sdmabase,
+ cdns_xspi->in_buffer, sdma_size);
+ break;
+
+ case CDNS_XSPI_SDMA_DIR_WRITE:
+ m_iowriteq(cdns_xspi->sdmabase,
+ cdns_xspi->out_buffer, sdma_size);
+ break;
+ }
+
+ return 0;
+}
+
+static const struct spi_controller_mem_ops marvell_xspi_mem_ops = {
+ .supports_op = PTR_IF(IS_ENABLED(CONFIG_ACPI), cdns_xspi_supports_op),
+ .exec_op = marvell_xspi_mem_op_execute,
+ .adjust_op_size = cdns_xspi_adjust_mem_op_size,
+};
+
static int cdns_xspi_prepare_generic(int cs, const void *dout, int len, int glue, u32 *cmd_regs)
{
u8 *data = (u8 *)dout;
@@ -1099,9 +1108,11 @@ static int cdns_xspi_transfer_one_message_b0(struct spi_controller *controller,
cdns_xspi_prepare_transfer(cs, 1, current_transfer_len - 1,
cmd_regs);
cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
+ cdns_xspi->sdma_xfer_len = current_transfer_len - 1;
if (!cdns_xspi_is_sdma_ready(cdns_xspi, true))
return -EIO;
- cdns_xspi->sdma_handler(cdns_xspi);
+ if (cdns_xspi->sdma_handler(cdns_xspi))
+ return -EIO;
if (!cdns_xspi_is_stig_ready(cdns_xspi, true))
return -EIO;
@@ -1170,15 +1181,14 @@ static int cdns_xspi_probe(struct platform_device *pdev)
return -ENODEV;
}
+ host->mem_ops = &cadence_xspi_mem_ops;
+ cdns_xspi->sdma_handler = &cdns_xspi_sdma_handle;
+ cdns_xspi->set_interrupts_handler = &cdns_xspi_set_interrupts;
if (cdns_xspi->driver_data->mrvl_hw_overlay) {
host->mem_ops = &marvell_xspi_mem_ops;
host->transfer_one_message = cdns_xspi_transfer_one_message_b0;
cdns_xspi->sdma_handler = &marvell_xspi_sdma_handle;
cdns_xspi->set_interrupts_handler = &marvell_xspi_set_interrupts;
- } else {
- host->mem_ops = &cadence_xspi_mem_ops;
- cdns_xspi->sdma_handler = &cdns_xspi_sdma_handle;
- cdns_xspi->set_interrupts_handler = &cdns_xspi_set_interrupts;
}
host->dev.of_node = pdev->dev.of_node;
host->bus_num = -1;
diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c
index 3af9a26cdefc..a1ee106971cb 100644
--- a/drivers/target/target_core_fabric_lib.c
+++ b/drivers/target/target_core_fabric_lib.c
@@ -370,6 +370,16 @@ static bool iscsi_parse_pr_out_transport_id(
pr_err("Missing ISID in iSCSI Initiator port TransportID\n");
return false;
}
+ /*
+ * The registration code reads the ISID as a C string,
+ * and its first 8 bytes with get_unaligned_be64(), so
+ * require 8 bytes after the separator and a NUL inside
+ * the rest of the descriptor.
+ */
+ if (buf + tid_len - p < 8 || !memchr(p, '\0', buf + tid_len - p)) {
+ pr_err("Incomplete or unterminated ISID in iSCSI Initiator port TransportID\n");
+ return false;
+ }
*port_nexus_ptr = p;
/*
* Go ahead and do the lower case conversion of the received
diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c
index 717931267bda..6b9adab3d11e 100644
--- a/drivers/target/target_core_user.c
+++ b/drivers/target/target_core_user.c
@@ -2514,6 +2514,11 @@ static int tcmu_set_data_pages_per_blk(struct tcmu_dev *udev, substring_t *arg)
return ret;
}
+ if (val <= 0) {
+ pr_err("Invalid data_pages_per_blk %d.\n", val);
+ return -EINVAL;
+ }
+
if (val > TCMU_MBS_TO_PAGES(udev->data_area_mb)) {
pr_err("Invalid data_pages_per_blk %d: greater than max_data_area_mb %d -> %zd pages).\n",
val, udev->data_area_mb,
diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c
index 3e1932a3bfbb..a0e20bffefba 100644
--- a/drivers/tty/serial/amba-pl011.c
+++ b/drivers/tty/serial/amba-pl011.c
@@ -2224,8 +2224,21 @@ static int pl011_rs485_config(struct uart_port *port, struct ktermios *termios,
struct uart_amba_port *uap =
container_of(port, struct uart_amba_port, port);
- if (port->rs485.flags & SER_RS485_ENABLED)
+ if ((port->rs485.flags & SER_RS485_ENABLED) &&
+ !(rs485->flags & SER_RS485_ENABLED)) {
+ u32 cr = pl011_read(uap, REG_CR);
+
+ /*
+ * Leaving RS485 mode, which only enables the transmitter while
+ * sending. Don't stop it, as that stalls a DMA transfer still
+ * in flight. Keep the transmitter and receiver enabled.
+ */
+ uap->rs485_tx_started = false;
+ cr |= UART011_CR_TXE | UART011_CR_RXE;
+ pl011_write(cr, uap, REG_CR);
+ } else if (port->rs485.flags & SER_RS485_ENABLED) {
pl011_rs485_tx_stop(uap);
+ }
/* Make sure auto RTS is disabled */
if (rs485->flags & SER_RS485_ENABLED) {
diff --git a/drivers/tty/serial/sc16is7xx.c b/drivers/tty/serial/sc16is7xx.c
index 942d12f40a39..f104d6a7a3fd 100644
--- a/drivers/tty/serial/sc16is7xx.c
+++ b/drivers/tty/serial/sc16is7xx.c
@@ -535,10 +535,13 @@ EXPORT_SYMBOL_GPL(sc16is762_devtype);
static bool sc16is7xx_regmap_volatile(struct device *dev, unsigned int reg)
{
switch (reg) {
- case SC16IS7XX_RHR_REG:
- case SC16IS7XX_IIR_REG:
- case SC16IS7XX_LSR_REG:
- case SC16IS7XX_MSR_REG:
+ case SC16IS7XX_RHR_REG: /* Shared address space with THR & DLL */
+ case SC16IS7XX_IER_REG: /* Shared address space with DLH */
+ case SC16IS7XX_IIR_REG: /* Shared address space with FCR & EFR */
+ case SC16IS7XX_MCR_REG: /* Shared address space with XON1 */
+ case SC16IS7XX_LSR_REG: /* Shared address space with XON2 */
+ case SC16IS7XX_MSR_REG: /* Shared address space with TCR & XOFF1 */
+ case SC16IS7XX_SPR_REG: /* Shared address space with TLR & XOFF2 */
case SC16IS7XX_TXLVL_REG:
case SC16IS7XX_RXLVL_REG:
case SC16IS7XX_IOSTATE_REG:
@@ -781,6 +784,7 @@ static void sc16is7xx_update_mlines(struct sc16is7xx_one *one)
unsigned long flags;
unsigned int status, changed;
+ /* Lock required as MSR address is shared with TCR and XOFF1. */
lockdep_assert_held_once(&one->lock);
status = sc16is7xx_get_hwmctrl(port);
@@ -1615,7 +1619,7 @@ int sc16is7xx_probe(struct device *dev, const struct sc16is7xx_devtype *devtype,
s->p[i].port.irq = irq;
s->p[i].port.type = PORT_SC16IS7XX;
s->p[i].port.fifosize = SC16IS7XX_FIFO_SIZE;
- s->p[i].port.flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY;
+ s->p[i].port.flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY | UPF_SOFT_FLOW;
s->p[i].port.iobase = i;
/*
* Use all ones as membase to make sure uart_configure_port() in
diff --git a/drivers/ufs/core/ufs_bsg.c b/drivers/ufs/core/ufs_bsg.c
index 0e0b08a3821c..59a473261ae1 100644
--- a/drivers/ufs/core/ufs_bsg.c
+++ b/drivers/ufs/core/ufs_bsg.c
@@ -79,6 +79,9 @@ static int ufs_bsg_exec_advanced_rpmb_req(struct ufs_hba *hba, struct bsg_job *j
if (hba->ufs_version < ufshci_version(4, 0) || !hba->dev_info.b_advanced_rpmb_en)
return -EINVAL;
+ if (job->request_len < sizeof(*rpmb_request))
+ return -EINVAL;
+
if (rpmb_request->ehs_req.length != 2 || rpmb_request->ehs_req.ehs_type != 1)
return -EINVAL;
@@ -144,6 +147,11 @@ static int ufs_bsg_request(struct bsg_job *job)
memset(job->reply, 0, job->reply_len);
+ if (job->request_len < sizeof(*bsg_request)) {
+ ret = -EINVAL;
+ goto out_complete;
+ }
+
ufshcd_rpm_get_sync(hba);
msgcode = bsg_request->msgcode;
@@ -170,6 +178,10 @@ static int ufs_bsg_request(struct bsg_job *job)
break;
case UPIU_TRANSACTION_UIC_CMD:
memcpy(&uc, &bsg_request->upiu_req.uc, UIC_CMD_SIZE);
+ if (uc.argument2 & MASK_UIC_COMMAND_RESULT) {
+ ret = -EINVAL;
+ break;
+ }
ret = ufshcd_send_bsg_uic_cmd(hba, &uc);
if (ret)
dev_err(hba->dev, "send uic cmd: error code %d\n", ret);
@@ -193,6 +205,7 @@ static int ufs_bsg_request(struct bsg_job *job)
out:
ufshcd_rpm_put_sync(hba);
kfree(buff);
+out_complete:
bsg_reply->result = ret;
if (ret == 0)
job->reply_len = rpmb ? sizeof(struct ufs_rpmb_reply) :
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 4a78a5d9cd8b..798ad0338cd9 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -5653,6 +5653,37 @@ static void ufshcd_clear_polled(struct ufs_hba *hba,
}
}
+/*
+ * Return a bitmask of the transfer requests cleared from
+ * @hba->outstanding_reqs.
+ */
+static unsigned long __ufshcd_clear_outstanding_reqs(struct ufs_hba *hba,
+ unsigned long mask,
+ bool from_irq)
+ __must_hold(&hba->outstanding_lock)
+{
+ unsigned long completed_reqs = mask & hba->outstanding_reqs;
+
+ if (from_irq) {
+ /* Do not complete polled requests from interrupt context. */
+ ufshcd_clear_polled(hba, &completed_reqs);
+ }
+ hba->outstanding_reqs &= ~completed_reqs;
+
+ return completed_reqs;
+}
+
+/*
+ * Return a bitmask of the transfer requests cleared from
+ * @hba->outstanding_reqs.
+ */
+static unsigned long ufshcd_clear_outstanding_reqs(struct ufs_hba *hba,
+ unsigned long mask)
+{
+ guard(spinlock_irqsave)(&hba->outstanding_lock);
+ return __ufshcd_clear_outstanding_reqs(hba, mask, false);
+}
+
/*
* Return: > 0 if one or more commands have been completed or 0 if no
* requests have been completed.
@@ -5663,6 +5694,7 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)
unsigned long completed_reqs, flags;
u32 tr_doorbell;
struct ufs_hw_queue *hwq;
+ bool from_irq;
if (hba->mcq_enabled) {
hwq = &hba->uhq[queue_num];
@@ -5672,12 +5704,9 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)
spin_lock_irqsave(&hba->outstanding_lock, flags);
tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL);
- completed_reqs = ~tr_doorbell & hba->outstanding_reqs;
- if (queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT) {
- /* Do not complete polled requests from interrupt context. */
- ufshcd_clear_polled(hba, &completed_reqs);
- }
- hba->outstanding_reqs &= ~completed_reqs;
+ from_irq = queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT;
+ completed_reqs = __ufshcd_clear_outstanding_reqs(hba, ~tr_doorbell,
+ from_irq);
spin_unlock_irqrestore(&hba->outstanding_lock, flags);
if (completed_reqs)
@@ -7662,7 +7691,6 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
struct ufshcd_lrb *lrbp = &hba->lrb[tag];
unsigned long flags;
int err = FAILED;
- bool outstanding;
u32 reg;
ufshcd_hold(hba);
@@ -7743,11 +7771,7 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
* Clear the corresponding bit from outstanding_reqs since the command
* has been aborted successfully.
*/
- spin_lock_irqsave(&hba->outstanding_lock, flags);
- outstanding = __test_and_clear_bit(tag, &hba->outstanding_reqs);
- spin_unlock_irqrestore(&hba->outstanding_lock, flags);
-
- if (outstanding)
+ if (ufshcd_clear_outstanding_reqs(hba, BIT(tag)))
ufshcd_release_scsi_cmd(hba, lrbp);
err = SUCCESS;
diff --git a/drivers/usb/core/config.c b/drivers/usb/core/config.c
index 43f1419d6617..1dbff5398458 100644
--- a/drivers/usb/core/config.c
+++ b/drivers/usb/core/config.c
@@ -891,6 +891,18 @@ int usb_get_configuration(struct usb_device *dev)
unsigned char *bigbuffer;
struct usb_config_descriptor *desc;
int result;
+ size_t usb_config_req_size;
+
+ /*
+ * We usually start by grabbing the first 9-bytes descriptor so we know
+ * how long the whole configuration is. Some devices with quirky
+ * firmware will fail enumeration, so if the quirk is set, use 255 instead,
+ * mirroring the behavior of Windows.
+ */
+ if (dev->quirks & USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE)
+ usb_config_req_size = 255;
+ else
+ usb_config_req_size = USB_DT_CONFIG_SIZE;
if (ncfg > USB_MAXCONFIG) {
dev_notice(ddev, "too many configurations: %d, "
@@ -917,15 +929,13 @@ int usb_get_configuration(struct usb_device *dev)
if (!dev->rawdescriptors)
return -ENOMEM;
- desc = kmalloc(USB_DT_CONFIG_SIZE, GFP_KERNEL);
+ desc = kmalloc(usb_config_req_size, GFP_KERNEL);
if (!desc)
return -ENOMEM;
for (cfgno = 0; cfgno < ncfg; cfgno++) {
- /* We grab just the first descriptor so we know how long
- * the whole configuration is */
result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno,
- desc, USB_DT_CONFIG_SIZE);
+ desc, usb_config_req_size);
if (result < 0) {
dev_err(ddev, "unable to read config index %d "
"descriptor/%s: %d\n", cfgno, "start", result);
@@ -935,16 +945,14 @@ int usb_get_configuration(struct usb_device *dev)
dev->descriptor.bNumConfigurations = cfgno;
break;
} else if (result < 4) {
- dev_err(ddev, "config index %d descriptor too short "
- "(expected %i, got %i)\n", cfgno,
- USB_DT_CONFIG_SIZE, result);
+ dev_err(ddev, "config index %d descriptor too short (asked for %zu, got %i)\n",
+ cfgno, usb_config_req_size, result);
result = -EINVAL;
goto err;
}
length = max((int) le16_to_cpu(desc->wTotalLength),
USB_DT_CONFIG_SIZE);
- /* Now that we know the length, get the whole thing */
bigbuffer = kmalloc(length, GFP_KERNEL);
if (!bigbuffer) {
result = -ENOMEM;
@@ -954,6 +962,13 @@ int usb_get_configuration(struct usb_device *dev)
if (dev->quirks & USB_QUIRK_DELAY_INIT)
msleep(200);
+ /* Skip the second read if we already got everything */
+ if (result >= length) {
+ memcpy(bigbuffer, desc, length);
+ goto store_and_parse;
+ }
+
+ /* Get the whole thing */
result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno,
bigbuffer, length);
if (result < 0) {
@@ -968,6 +983,7 @@ int usb_get_configuration(struct usb_device *dev)
length = result;
}
+store_and_parse:
dev->rawdescriptors[cfgno] = bigbuffer;
result = usb_parse_configuration(dev, cfgno,
diff --git a/drivers/usb/core/quirks.c b/drivers/usb/core/quirks.c
index 6ed380599405..ee439a2ddf31 100644
--- a/drivers/usb/core/quirks.c
+++ b/drivers/usb/core/quirks.c
@@ -143,6 +143,10 @@ static int quirks_param_set(const char *value, const struct kernel_param *kp)
break;
case 'q':
flags |= USB_QUIRK_FORCE_ONE_CONFIG;
+ break;
+ case 'r':
+ flags |= USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE;
+ break;
/* Ignore unrecognized flag characters */
}
}
@@ -603,6 +607,9 @@ static const struct usb_device_id usb_quirk_list[] = {
/* ezcap401 - BOS descriptor fetch hangs at SuperSpeed Plus */
{ USB_DEVICE(0x32ed, 0x0401), .driver_info = USB_QUIRK_NO_BOS },
+ /* GameSir G7 SE */
+ { USB_DEVICE(0x3537, 0x10a0), .driver_info = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE },
+
/* DELL USB GEN2 */
{ USB_DEVICE(0x413c, 0xb062), .driver_info = USB_QUIRK_NO_LPM | USB_QUIRK_RESET_RESUME },
diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
index 597a9a8c16b9..2c35cb444f73 100644
--- a/drivers/usb/host/xhci-ring.c
+++ b/drivers/usb/host/xhci-ring.c
@@ -3583,15 +3583,13 @@ static u32 xhci_td_remainder(struct xhci_hcd *xhci, int transferred,
static int xhci_align_td(struct xhci_hcd *xhci, struct urb *urb, u32 enqd_len,
- u32 *trb_buff_len, struct xhci_segment *seg)
+ u32 *trb_buff_len, struct xhci_segment *seg, u32 max_pkt)
{
struct device *dev = xhci_to_hcd(xhci)->self.sysdev;
unsigned int unalign;
- unsigned int max_pkt;
u32 new_buff_len;
size_t len;
- max_pkt = usb_endpoint_maxp(&urb->ep->desc);
unalign = (enqd_len + *trb_buff_len) % max_pkt;
/* we got lucky, last normal TRB data on segment is packet aligned */
@@ -3736,9 +3734,8 @@ int xhci_queue_bulk_tx(struct xhci_hcd *xhci, gfp_t mem_flags,
if (enqd_len + trb_buff_len < full_len) {
field |= TRB_CHAIN;
if (trb_is_link(ring->enqueue + 1)) {
- if (xhci_align_td(xhci, urb, enqd_len,
- &trb_buff_len,
- ring->enq_seg)) {
+ if (xhci_align_td(xhci, urb, enqd_len, &trb_buff_len,
+ ring->enq_seg, ring->bounce_buf_len)) {
send_addr = ring->enq_seg->bounce_dma;
/* TD bounced at least, and last on this seg */
td->bounce_seg = ring->enq_seg;
diff --git a/drivers/usb/typec/ucsi/ucsi_ccg.c b/drivers/usb/typec/ucsi/ucsi_ccg.c
index 3a4e1b6f22ec..fbb65b734fc1 100644
--- a/drivers/usb/typec/ucsi/ucsi_ccg.c
+++ b/drivers/usb/typec/ucsi/ucsi_ccg.c
@@ -397,14 +397,28 @@ static int ucsi_ccg_init(struct ucsi_ccg *uc)
return -ETIMEDOUT;
}
-static void ucsi_ccg_update_get_current_cam_cmd(struct ucsi_ccg *uc, u8 *data)
+static int ucsi_ccg_update_get_current_cam_cmd(struct ucsi_ccg *uc, u8 *data)
{
u8 cam, new_cam;
cam = data[0];
+ if (cam >= UCSI_MAX_ALTMODES) {
+ dev_err(uc->dev, "PPM returned invalid alternate mode index %u\n",
+ cam);
+ return -EPROTO;
+ }
+
new_cam = uc->orig[cam].linked_idx;
+ if (new_cam >= UCSI_MAX_ALTMODES) {
+ dev_err(uc->dev, "invalid linked alternate mode index %u for %u\n",
+ new_cam, cam);
+ return -EPROTO;
+ }
+
uc->updated[new_cam].active_idx = cam;
data[0] = new_cam;
+
+ return 0;
}
static bool ucsi_ccg_update_altmodes(struct ucsi *ucsi,
@@ -593,6 +607,7 @@ static int ucsi_ccg_read_message_in(struct ucsi *ucsi, void *val, size_t val_len
struct ucsi_ccg *uc = ucsi_get_drvdata(ucsi);
struct ucsi_capability *cap;
struct ucsi_altmode *alt;
+ int ret = 0;
spin_lock(&uc->op_lock);
memcpy(val, uc->op_data.message_in, val_len);
@@ -601,7 +616,7 @@ static int ucsi_ccg_read_message_in(struct ucsi *ucsi, void *val, size_t val_len
switch (UCSI_COMMAND(uc->last_cmd_sent)) {
case UCSI_GET_CURRENT_CAM:
if (uc->has_multiple_dp)
- ucsi_ccg_update_get_current_cam_cmd(uc, (u8 *)val);
+ ret = ucsi_ccg_update_get_current_cam_cmd(uc, (u8 *)val);
break;
case UCSI_GET_ALTERNATE_MODES:
if (UCSI_ALTMODE_RECIPIENT(uc->last_cmd_sent) ==
@@ -622,7 +637,7 @@ static int ucsi_ccg_read_message_in(struct ucsi *ucsi, void *val, size_t val_len
}
uc->last_cmd_sent = 0;
- return 0;
+ return ret;
}
static int ucsi_ccg_async_control(struct ucsi *ucsi, u64 command)
diff --git a/drivers/vdpa/solidrun/snet_ctrl.c b/drivers/vdpa/solidrun/snet_ctrl.c
index 3cef2571d15d..1c3b5f2e4499 100644
--- a/drivers/vdpa/solidrun/snet_ctrl.c
+++ b/drivers/vdpa/solidrun/snet_ctrl.c
@@ -291,7 +291,6 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
/* Check for errors */
val = snet_read_ctrl(regs);
- ret = SNET_VAL_TO_ERR(val);
/* Clear the chunk ready bit */
val &= ~SNET_CTRL_CHUNK_RDY_MASK;
@@ -301,6 +300,8 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
if (ret)
SNET_WARN(pdev, "Timeout waiting for DPU to complete a control command, err %d\n",
ret);
+ else
+ ret = SNET_VAL_TO_ERR(val);
exit:
mutex_unlock(&snet->ctrl_lock);
diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c
index e8b837df328f..b8b2eb44410d 100644
--- a/drivers/vdpa/vdpa_user/iova_domain.c
+++ b/drivers/vdpa/vdpa_user/iova_domain.c
@@ -76,6 +76,10 @@ int vduse_domain_set_map(struct vduse_iova_domain *domain,
for (map = vhost_iotlb_itree_first(iotlb, start, last); map;
map = vhost_iotlb_itree_next(map, start, last)) {
map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file == domain->file) {
+ ret = -EINVAL;
+ goto err;
+ }
ret = vduse_iotlb_add_range(domain, map->start, map->last,
map->addr, map->perm,
map_file->file,
@@ -226,6 +230,7 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
{
u64 start = iova & PAGE_MASK;
u64 last = start + PAGE_SIZE - 1;
+ struct vdpa_map_file *map_file;
struct vhost_iotlb_map *map;
struct page *page = NULL;
@@ -234,6 +239,10 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
if (!map)
goto out;
+ map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file != domain->file)
+ goto out;
+
page = pfn_to_page((map->addr + iova - map->start) >> PAGE_SHIFT);
get_page(page);
out:
@@ -579,7 +588,7 @@ static int vduse_domain_mmap(struct file *file, struct vm_area_struct *vma)
{
struct vduse_iova_domain *domain = file->private_data;
- vm_flags_set(vma, VM_DONTDUMP | VM_DONTEXPAND);
+ vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND);
vma->vm_private_data = domain;
vma->vm_ops = &vduse_domain_mmap_ops;
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index a940718745de..33e3100b3836 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -1151,7 +1151,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
unsigned long list_size = PAGE_SIZE / sizeof(struct page *);
unsigned int gup_flags = FOLL_LONGTERM;
unsigned long npages, cur_base, map_pfn, last_pfn = 0;
- unsigned long lock_limit, sz2pin, nchunks, i;
+ unsigned long lock_limit, sz2pin, i;
unsigned long page_offset;
u64 start = iova;
long pinned;
@@ -1187,7 +1187,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
cur_base = uaddr & PAGE_MASK;
iova &= PAGE_MASK;
- nchunks = 0;
while (npages) {
sz2pin = min_t(unsigned long, npages, list_size);
@@ -1202,7 +1201,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
}
goto out;
}
- nchunks++;
if (!last_pfn)
map_pfn = page_to_pfn(page_list[0]);
@@ -1233,7 +1231,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
map_pfn = this_pfn;
iova += csize;
- nchunks = 0;
}
last_pfn = this_pfn;
@@ -1248,7 +1245,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
PFN_PHYS(map_pfn), perm, NULL);
out:
if (ret) {
- if (nchunks) {
+ if (last_pfn) {
unsigned long pfn;
/*
@@ -1260,7 +1257,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
* the corresponding unpinning will be handled by
* vdpa_unmap().
*/
- WARN_ON(!last_pfn);
for (pfn = map_pfn; pfn <= last_pfn; pfn++)
unpin_user_page(pfn_to_page(pfn));
}
diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c
index 781731eb95cf..db012d6b549e 100644
--- a/drivers/vhost/vringh.c
+++ b/drivers/vhost/vringh.c
@@ -197,8 +197,9 @@ static int move_to_indirect(const struct vringh *vrh,
}
len = vringh32_to_cpu(vrh, desc->len);
- if (unlikely(len % sizeof(struct vring_desc))) {
- vringh_bad("Strange indirect len %u", desc->len);
+ if (unlikely(len < sizeof(struct vring_desc) ||
+ len % sizeof(struct vring_desc))) {
+ vringh_bad("Invalid indirect len %u", desc->len);
return -EINVAL;
}
@@ -334,6 +335,17 @@ __vringh_iov(struct vringh *vrh, u16 i,
if (unlikely(err))
goto fail;
+ if (up_next == -1)
+ count++;
+ else
+ indirect_count++;
+
+ if (count > vrh->vring.num || indirect_count > desc_max) {
+ vringh_bad("Descriptor loop in %p", descs);
+ err = -ELOOP;
+ goto fail;
+ }
+
if (unlikely(desc.flags &
cpu_to_vringh16(vrh, VRING_DESC_F_INDIRECT))) {
u64 a = vringh64_to_cpu(vrh, desc.addr);
@@ -359,17 +371,6 @@ __vringh_iov(struct vringh *vrh, u16 i,
continue;
}
- if (up_next == -1)
- count++;
- else
- indirect_count++;
-
- if (count > vrh->vring.num || indirect_count > desc_max) {
- vringh_bad("Descriptor loop in %p", descs);
- err = -ELOOP;
- goto fail;
- }
-
if (desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_WRITE))
iov = wiov;
else {
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index 65297f245b27..f00182e9efec 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -331,7 +331,7 @@ static struct sk_buff *
vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
unsigned int out, unsigned int in)
{
- struct virtio_vsock_hdr *hdr;
+ struct virtio_vsock_hdr hdr;
struct iov_iter iov_iter;
struct sk_buff *skb;
size_t payload_len;
@@ -349,34 +349,32 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM)
return NULL;
- /* len contains both payload and hdr */
- skb = virtio_vsock_alloc_skb(len, GFP_KERNEL);
- if (!skb)
- return NULL;
-
iov_iter_init(&iov_iter, ITER_SOURCE, vq->iov, out, len);
- hdr = virtio_vsock_hdr(skb);
- nbytes = copy_from_iter(hdr, sizeof(*hdr), &iov_iter);
- if (nbytes != sizeof(*hdr)) {
+ nbytes = copy_from_iter(&hdr, sizeof(hdr), &iov_iter);
+ if (nbytes != sizeof(hdr)) {
vq_err(vq, "Expected %zu bytes for pkt->hdr, got %zu bytes\n",
- sizeof(*hdr), nbytes);
- kfree_skb(skb);
+ sizeof(hdr), nbytes);
return NULL;
}
- payload_len = le32_to_cpu(hdr->len);
+ payload_len = le32_to_cpu(hdr.len);
+
+ /* The pkt is too big or the length in the header is invalid */
+ if (payload_len > len - sizeof(hdr))
+ return NULL;
+
+ /* Allocate only for the payload declared in the header. */
+ skb = virtio_vsock_alloc_skb(payload_len + sizeof(hdr), GFP_KERNEL);
+ if (!skb)
+ return NULL;
+
+ memcpy(virtio_vsock_hdr(skb), &hdr, sizeof(hdr));
/* No payload */
if (!payload_len)
return skb;
- /* The pkt is too big or the length in the header is invalid */
- if (payload_len + sizeof(*hdr) > len) {
- kfree_skb(skb);
- return NULL;
- }
-
virtio_vsock_skb_put(skb, payload_len);
if (skb_copy_datagram_from_iter(skb, 0, &iov_iter, payload_len)) {
diff --git a/drivers/video/fbdev/core/fb_info.c b/drivers/video/fbdev/core/fb_info.c
index 4847ebe50d7d..9b36361bda6e 100644
--- a/drivers/video/fbdev/core/fb_info.c
+++ b/drivers/video/fbdev/core/fb_info.c
@@ -42,6 +42,7 @@ struct fb_info *framebuffer_alloc(size_t size, struct device *dev)
info->device = dev;
info->fbcon_rotate_hint = -1;
+ INIT_LIST_HEAD(&info->modelist);
#if IS_ENABLED(CONFIG_FB_BACKLIGHT)
mutex_init(&info->bl_curve_mutex);
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index a3f507825eed..3105682075c8 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -27,7 +27,8 @@ static int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
u8 *dst;
if (vc->vc_font.data == par->fontdata &&
- par->p->con_rotate == par->cur_rotate)
+ par->p->con_rotate == par->cur_rotate &&
+ par->fontbuffer)
goto finished;
src = par->fontdata = vc->vc_font.data;
diff --git a/drivers/video/fbdev/nvidia/nvidia.c b/drivers/video/fbdev/nvidia/nvidia.c
index 5f61d953e4c7..bd4a6dd573dd 100644
--- a/drivers/video/fbdev/nvidia/nvidia.c
+++ b/drivers/video/fbdev/nvidia/nvidia.c
@@ -1288,7 +1288,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
NVTRACE_ENTER();
assert(pd != NULL);
- if (pci_enable_device(pd)) {
+ if (pcim_enable_device(pd)) {
printk(KERN_ERR PFX "cannot enable PCI device\n");
return -ENODEV;
}
@@ -1301,7 +1301,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
nvidiafb_fix.mmio_start = pci_resource_start(pd, 0);
nvidiafb_fix.mmio_len = pci_resource_len(pd, 0);
- REGS = ioremap(nvidiafb_fix.mmio_start, nvidiafb_fix.mmio_len);
+ REGS = devm_ioremap(&pd->dev, nvidiafb_fix.mmio_start, nvidiafb_fix.mmio_len);
if (!REGS) {
printk(KERN_ERR PFX "cannot ioremap MMIO base\n");
return -ENODEV;
@@ -1329,7 +1329,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
if (info->pixmap.addr == NULL)
goto err_out_kfree;
- if (pci_request_regions(pd, "nvidiafb")) {
+ if (pcim_request_all_regions(pd, "nvidiafb")) {
printk(KERN_ERR PFX "cannot request PCI regions\n");
goto err_out_enable;
}
@@ -1354,7 +1354,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
sprintf(nvidiafb_fix.id, "NV%x", (pd->device & 0x0ff0) >> 4);
if (NVCommonSetup(info))
- goto err_out_free_base0;
+ goto err_out_enable;
par->FbAddress = nvidiafb_fix.smem_start;
par->FbMapSize = par->RamAmountKBytes * 1024;
@@ -1374,8 +1374,8 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
par->ScratchBufferStart = par->FbUsableSize - par->ScratchBufferSize;
par->CursorStart = par->FbUsableSize + (32 * 1024);
- info->screen_base = ioremap_wc(nvidiafb_fix.smem_start,
- par->FbMapSize);
+ info->screen_base = devm_ioremap_wc(&pd->dev, nvidiafb_fix.smem_start,
+ par->FbMapSize);
info->screen_size = par->FbUsableSize;
nvidiafb_fix.smem_len = par->RamAmountKBytes * 1024;
@@ -1419,19 +1419,16 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
return 0;
err_out_iounmap_fb:
- iounmap(info->screen_base);
+ arch_phys_wc_del(par->wc_cookie);
fb_destroy_modelist(&info->modelist);
err_out_free_base1:
fb_destroy_modedb(info->monspecs.modedb);
nvidia_delete_i2c_busses(par);
-err_out_free_base0:
- pci_release_regions(pd);
err_out_enable:
kfree(info->pixmap.addr);
err_out_kfree:
framebuffer_release(info);
err_out:
- iounmap(REGS);
return -ENODEV;
}
@@ -1446,11 +1443,8 @@ static void nvidiafb_remove(struct pci_dev *pd)
unregister_framebuffer(info);
arch_phys_wc_del(par->wc_cookie);
- iounmap(info->screen_base);
fb_destroy_modedb(info->monspecs.modedb);
nvidia_delete_i2c_busses(par);
- iounmap(par->REGS);
- pci_release_regions(pd);
kfree(info->pixmap.addr);
framebuffer_release(info);
NVTRACE_LEAVE();
diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c
index ae30e394d176..32e9ac24e0d5 100644
--- a/drivers/watchdog/hpwdt.c
+++ b/drivers/watchdog/hpwdt.c
@@ -22,6 +22,7 @@
#include <linux/watchdog.h>
#ifdef CONFIG_HPWDT_NMI_DECODING
#include <asm/nmi.h>
+#include <asm/uv/uv.h>
#endif
#include <linux/crash_dump.h>
@@ -158,24 +159,31 @@ static int hpwdt_set_pretimeout(struct watchdog_device *wdd, unsigned int req)
return 0;
}
-static int hpwdt_my_nmi(void)
-{
- return ioread8(hpwdt_nmistat) & 0x6;
-}
+#define NMISTAT_EASR BIT(0)
+#define NMISTAT_EWDOG BIT(1)
+#define NMISTAT_RTRAP BIT(2)
+#define NMISTAT_DBELL BIT(3)
+#define NMISTAT_EMSWDG BIT(4)
+#define NMISTAT_GPI BIT(5)
+#define NMISTAT_NMIOUT BIT(7)
/*
* NMI Handler
*/
static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs)
{
- unsigned int mynmi = hpwdt_my_nmi();
- static char panic_msg[] =
+ u8 nmistat = ioread8(hpwdt_nmistat);
+ bool mynmi = (nmistat & (NMISTAT_EWDOG | NMISTAT_RTRAP)) != 0;
+ static char panic_msg_default[] =
"00: An NMI occurred. Depending on your system the reason "
"for the NMI is logged in any one of the following resources:\n"
"1. Integrated Management Log (IML)\n"
"2. OA Syslog\n"
"3. OA Forward Progress Log\n"
"4. iLO Event Log";
+ static char panic_msg_uv[] =
+ "00: A watchdog NMI occurred.";
+ char *panic_msg = is_uv_system() ? panic_msg_uv : panic_msg_default;
if (ulReason == NMI_UNKNOWN && !mynmi)
return NMI_DONE;
@@ -189,8 +197,8 @@ static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs)
hpwdt_ping_ticks(SECS_TO_TICKS(val));
}
- hex_byte_pack(panic_msg, mynmi);
- nmi_panic(regs, panic_msg);
+ hex_byte_pack(panic_msg, nmistat);
+ nmi_panic(regs, "%s", panic_msg);
return NMI_HANDLED;
}
diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index 5ab973627d18..1072e781f12e 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -96,8 +96,10 @@ dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
struct gntdev_dmabuf_wait_obj *obj;
obj = kzalloc(sizeof(*obj), GFP_KERNEL);
- if (!obj)
+ if (!obj) {
+ kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release);
return ERR_PTR(-ENOMEM);
+ }
init_completion(&obj->completion);
obj->gntdev_dmabuf = gntdev_dmabuf;
diff --git a/fs/9p/v9fs.c b/fs/9p/v9fs.c
index bc879d32cfcf..66b072152745 100644
--- a/fs/9p/v9fs.c
+++ b/fs/9p/v9fs.c
@@ -16,6 +16,7 @@
#include <linux/parser.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
+#include <linux/sysfs.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <net/9p/transport.h>
@@ -555,21 +556,14 @@ static ssize_t caches_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *buf)
{
- ssize_t n = 0, count = 0, limit = PAGE_SIZE;
+ ssize_t count = 0;
struct v9fs_session_info *v9ses;
spin_lock(&v9fs_sessionlist_lock);
list_for_each_entry(v9ses, &v9fs_sessionlist, slist) {
- if (v9ses->cachetag) {
- n = snprintf(buf + count, limit, "%s\n", v9ses->cachetag);
- if (n < 0) {
- count = n;
- break;
- }
-
- count += n;
- limit -= n;
- }
+ if (v9ses->cachetag)
+ count += sysfs_emit_at(buf, count, "%s\n",
+ v9ses->cachetag);
}
spin_unlock(&v9fs_sessionlist_lock);
diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c
index 90fe3ea09595..7cb3f4b89dcc 100644
--- a/fs/9p/vfs_inode.c
+++ b/fs/9p/vfs_inode.c
@@ -493,6 +493,12 @@ static int v9fs_at_to_dotl_flags(int flags)
* nothing and races with concurrent metadata fetches that may already
* have observed the post-unlink value (nlink == 0).
*
+ * The same applies to a regular file whose cached nlink has already been
+ * driven to zero by the server: in the cached modes the attributes come
+ * from GETATTR and are applied verbatim, so decrementing a link count the
+ * server does not maintain only trips the drop_nlink() WARN_ON and
+ * underflows i_nlink.
+ *
* @inode: inode whose nlink is being dropped
*/
static void v9fs_dec_count(struct inode *inode)
@@ -501,6 +507,8 @@ static void v9fs_dec_count(struct inode *inode)
if (!(v9ses->cache & (CACHE_META | CACHE_LOOSE)))
return;
+ if (inode->i_nlink == 0)
+ return;
if (!S_ISDIR(inode->i_mode) || inode->i_nlink > 2)
drop_nlink(inode);
}
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index 55449c879142..934667e547dc 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -442,10 +442,8 @@ void afs_make_call(struct afs_call *call, gfp_t gfp)
return;
error_do_abort:
- if (ret != -ECONNABORTED)
- rxrpc_kernel_abort_call(call->net->socket, rxcall,
- RX_USER_ABORT, ret,
- afs_abort_send_data_error);
+ rxrpc_kernel_abort_call(call->net->socket, rxcall,
+ RX_USER_ABORT, ret, afs_abort_send_data_error);
if (call->async) {
afs_see_call(call, afs_call_trace_async_abort);
return;
@@ -853,6 +851,7 @@ void afs_send_empty_reply(struct afs_call *call)
{
struct afs_net *net = call->net;
struct msghdr msg;
+ int ret;
_enter("");
@@ -865,21 +864,13 @@ void afs_send_empty_reply(struct afs_call *call)
msg.msg_controllen = 0;
msg.msg_flags = 0;
- switch (rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, 0,
- afs_notify_end_reply_tx)) {
- case 0:
- _leave(" [replied]");
- return;
-
- case -ENOMEM:
- _debug("oom");
+ ret = rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, 0,
+ afs_notify_end_reply_tx);
+ if (ret < 0) {
rxrpc_kernel_abort_call(net->socket, call->rxcall,
- RXGEN_SS_MARSHAL, -ENOMEM,
- afs_abort_oom);
- fallthrough;
- default:
- _leave(" [error]");
- return;
+ RXGEN_SS_MARSHAL, ret,
+ afs_abort_send_error);
+ afs_set_call_complete(call, ret, RXGEN_SS_MARSHAL);
}
}
@@ -891,7 +882,7 @@ void afs_send_simple_reply(struct afs_call *call, const void *buf, size_t len)
struct afs_net *net = call->net;
struct msghdr msg;
struct kvec iov[1];
- int n;
+ int ret;
_enter("");
@@ -906,21 +897,15 @@ void afs_send_simple_reply(struct afs_call *call, const void *buf, size_t len)
msg.msg_controllen = 0;
msg.msg_flags = 0;
- n = rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, len,
- afs_notify_end_reply_tx);
- if (n >= 0) {
- /* Success */
- _leave(" [replied]");
- return;
- }
-
- if (n == -ENOMEM) {
- _debug("oom");
+ ret = rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, len,
+ afs_notify_end_reply_tx);
+ if (ret < 0) {
rxrpc_kernel_abort_call(net->socket, call->rxcall,
- RXGEN_SS_MARSHAL, -ENOMEM,
- afs_abort_oom);
+ RXGEN_SS_MARSHAL, ret,
+ afs_abort_send_error);
+ afs_set_call_complete(call, ret, RXGEN_SS_MARSHAL);
+ _leave(" [error]");
}
- _leave(" [error]");
}
/*
diff --git a/fs/backing-file.c b/fs/backing-file.c
index cad40411c63e..2ed8f17aa5f4 100644
--- a/fs/backing-file.c
+++ b/fs/backing-file.c
@@ -83,6 +83,7 @@ struct backing_aio {
refcount_t ref;
struct kiocb *orig_iocb;
/* used for aio completion */
+ bool is_write;
void (*end_write)(struct file *, loff_t, ssize_t);
struct work_struct work;
long res;
@@ -123,7 +124,8 @@ static void backing_aio_rw_complete(struct kiocb *iocb, long res)
struct backing_aio *aio = container_of(iocb, struct backing_aio, iocb);
struct kiocb *orig_iocb = aio->orig_iocb;
- if (iocb->ki_flags & IOCB_WRITE)
+ /* Pairs with kiocb_start_write() in vfs_iocb_iter_write() */
+ if (aio->is_write)
kiocb_end_write(iocb);
backing_aio_cleanup(aio, res);
@@ -256,6 +258,7 @@ ssize_t backing_file_write_iter(struct file *file, struct iov_iter *iter,
goto out;
aio->orig_iocb = iocb;
+ aio->is_write = true;
aio->end_write = ctx->end_write;
kiocb_clone(&aio->iocb, iocb, get_file(file));
aio->iocb.ki_flags = flags;
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 8f195e769ecf..70f5fd918cce 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -1750,6 +1750,7 @@ void btrfs_sysfs_add_block_group_type(struct btrfs_block_group *cache)
if (space_info->block_group_kobjs[index]) {
spin_unlock(&space_info->lock);
kobject_put(&rkobj->kobj);
+ memalloc_nofs_restore(nofs_flag);
return;
} else {
space_info->block_group_kobjs[index] = &rkobj->kobj;
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index aa551d94be1c..63ead85dff26 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -2173,6 +2173,12 @@ void ceph_fill_inline_data(struct inode *inode, struct page *locked_page,
doutc(cl, "%p %llx.%llx len %zu locked_page %p\n", inode,
ceph_vinop(inode), len, locked_page);
+ if (len > PAGE_SIZE) {
+ pr_warn_ratelimited_client(cl, "oversized inline data %zu\n",
+ len);
+ len = PAGE_SIZE;
+ }
+
if (len > 0) {
void *kaddr = kmap_atomic(page);
memcpy(kaddr, data, len);
diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c
index f955a37d6b17..97fa6b821d5e 100644
--- a/fs/ceph/crypto.c
+++ b/fs/ceph/crypto.c
@@ -389,6 +389,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
struct fscrypt_str iname;
char *name = fname->name;
int name_len = fname->name_len;
+ int dst_len;
int ret;
if (WARN_ON_ONCE(tname && is_vmalloc_addr(tname->name)))
@@ -464,6 +465,9 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
iname.len = fname->ctext_len;
}
+ /* oname->len is overwritten by the decryption below */
+ dst_len = oname->len;
+
_oname.name = unlikely(is_vmalloc_addr(oname->name)) ?
tname->name : oname->name;
_oname.len = oname->len;
@@ -481,6 +485,10 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,
name_len = snprintf(tmp_buf, sizeof(tmp_buf), "_%.*s_%ld",
oname->len, oname->name, dir->i_ino);
+ if (name_len > dst_len) {
+ ret = -ENAMETOOLONG;
+ goto out;
+ }
memcpy(oname->name, tmp_buf, name_len);
oname->len = name_len;
}
diff --git a/fs/dax.c b/fs/dax.c
index ae1e9497b416..e9f12b7de17a 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -2067,7 +2067,7 @@ int dax_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
compared = dax_range_compare_iter(&src_iter, &dst_iter,
min(src_iter.len, dst_iter.len), same);
if (compared < 0)
- return ret;
+ return compared;
src_iter.processed = dst_iter.processed = compared;
}
return ret;
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index 08720efbb1a1..10b9aa2c2854 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -1386,15 +1386,6 @@ static void z_erofs_decompressqueue_kthread_work(struct kthread_work *work)
}
#endif
-/* Use (kthread_)work in atomic contexts to minimize scheduling overhead */
-static inline bool z_erofs_in_atomic(void)
-{
- if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth())
- return true;
- if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
- return true;
- return !preemptible();
-}
static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io,
int bios)
@@ -1411,7 +1402,7 @@ static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io,
if (atomic_add_return(bios, &io->pending_bios))
return;
- if (z_erofs_in_atomic()) {
+ if (bio_in_atomic()) {
/* See `sync_decompress` in sysfs-fs-erofs for more details */
if (sbi->sync_decompress == EROFS_SYNC_DECOMPRESS_AUTO)
sbi->sync_decompress = EROFS_SYNC_DECOMPRESS_FORCE_ON;
diff --git a/fs/exfat/file.c b/fs/exfat/file.c
index 033852efe5dc..b624dc527f67 100644
--- a/fs/exfat/file.c
+++ b/fs/exfat/file.c
@@ -581,23 +581,18 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
struct file *file = iocb->ki_filp;
struct inode *inode = file_inode(file);
struct exfat_inode_info *ei = EXFAT_I(inode);
- loff_t pos = iocb->ki_pos;
- loff_t valid_size;
+ loff_t pos, valid_size;
if (unlikely(exfat_forced_shutdown(inode->i_sb)))
return -EIO;
inode_lock(inode);
- if (pos > i_size_read(inode))
- truncate_pagecache(inode, i_size_read(inode));
-
- valid_size = ei->valid_size;
-
ret = generic_write_checks(iocb, iter);
if (ret <= 0)
goto unlock;
+ pos = iocb->ki_pos;
if (iocb->ki_flags & IOCB_DIRECT) {
unsigned long align = pos | iov_iter_alignment(iter);
@@ -608,6 +603,10 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
}
}
+ if (pos > i_size_read(inode))
+ truncate_pagecache(inode, i_size_read(inode));
+
+ valid_size = ei->valid_size;
if (pos > valid_size) {
ret = exfat_extend_valid_size(file, pos);
if (ret < 0 && ret != -ENOSPC) {
@@ -617,6 +616,8 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
}
if (ret < 0)
goto unlock;
+
+ pos = valid_size;
}
ret = __generic_file_write_iter(iocb, iter);
@@ -625,9 +626,6 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
inode_unlock(inode);
- if (pos > valid_size)
- pos = valid_size;
-
if (iocb->ki_pos > pos) {
ssize_t err = generic_write_sync(iocb, iocb->ki_pos - pos);
if (err < 0)
diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index 908d4849874d..0f047c33e0fe 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -254,7 +254,8 @@ static ssize_t ext2_dio_write_iter(struct kiocb *iocb, struct iov_iter *from)
pos = iocb->ki_pos;
status = generic_perform_write(iocb, from);
if (unlikely(status < 0)) {
- ret = status;
+ if (!ret)
+ ret = status;
goto out_unlock;
}
diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c
index f3149022257a..e8a7b21eedb5 100644
--- a/fs/ext2/inode.c
+++ b/fs/ext2/inode.c
@@ -824,6 +824,12 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
(first_block << blkbits) < i_size_read(inode))
create = 0;
+ ret = ext2_get_blocks(inode, first_block, max_blocks,
+ &bno, &new, &boundary, create);
+ if (ret < 0)
+ return ret;
+
+ iomap->flags = 0;
/*
* Writes that span EOF might trigger an IO size update on completion,
* so consider them to be dirty for the purposes of O_DSYNC even if
@@ -831,13 +837,6 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
*/
if ((flags & IOMAP_WRITE) && offset + length > i_size_read(inode))
iomap->flags |= IOMAP_F_DIRTY;
-
- ret = ext2_get_blocks(inode, first_block, max_blocks,
- &bno, &new, &boundary, create);
- if (ret < 0)
- return ret;
-
- iomap->flags = 0;
iomap->offset = (u64)first_block << blkbits;
if (flags & IOMAP_DAX)
iomap->dax_dev = sbi->s_daxdev;
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index dc9540400caa..eb928d644542 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -423,7 +423,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
long nr_to_write, enum iostat_type io_type)
{
struct address_space *mapping = META_MAPPING(sbi);
- pgoff_t index = 0, prev = ULONG_MAX;
+ pgoff_t index = 0, next = ULONG_MAX;
struct folio_batch fbatch;
long nwritten = 0;
int nr_folios;
@@ -444,9 +444,8 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
for (i = 0; i < nr_folios; i++) {
struct folio *folio = fbatch.folios[i];
- if (nr_to_write != LONG_MAX && i != 0 &&
- folio->index != prev +
- folio_nr_pages(fbatch.folios[i-1])) {
+ if (nr_to_write != LONG_MAX && next != ULONG_MAX &&
+ folio->index != next) {
folio_batch_release(&fbatch);
goto stop;
}
@@ -475,7 +474,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
break;
}
nwritten += folio_nr_pages(folio);
- prev = folio->index;
+ next = folio->index + folio_nr_pages(folio);
if (unlikely(nwritten >= nr_to_write))
break;
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 47743a0bdede..b56e6a98e7f4 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -287,7 +287,7 @@ static void f2fs_read_end_io(struct bio *bio)
{
struct f2fs_sb_info *sbi = F2FS_P_SB(bio_first_page_all(bio));
struct bio_post_read_ctx *ctx;
- bool intask = in_task() && !irqs_disabled();
+ bool intask = !bio_in_atomic();
iostat_update_and_unbind_ctx(bio);
ctx = bio->bi_private;
@@ -460,6 +460,10 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
struct bio *bio;
bdev = f2fs_target_device(sbi, fio->new_blkaddr, §or);
+ /* avoid reversed IO if bio is split due to exceeding max_segments */
+ if (npages > 1 && !is_read_io(fio->op) &&
+ f2fs_is_sequential_zone_area(sbi, fio->new_blkaddr))
+ npages = min_t(int, npages, bdev_max_segments(bdev));
bio = bio_alloc_bioset(bdev, npages,
fio->op | fio->op_flags | f2fs_io_flags(fio),
GFP_NOIO, &f2fs_bioset);
@@ -738,7 +742,19 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
return false;
if (last_blkaddr + 1 != cur_blkaddr)
return false;
- return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
+ if (bio->bi_bdev != f2fs_target_device(sbi, cur_blkaddr, NULL))
+ return false;
+ /* avoid reversed IO if bio is split due to exceeding max_segments/sectors */
+ if (f2fs_is_sequential_zone_area(sbi, cur_blkaddr)) {
+ struct request_queue *q = bdev_get_queue(bio->bi_bdev);
+
+ if (bio->bi_vcnt >= queue_max_segments(q))
+ return false;
+ if (bio_sectors(bio) + (F2FS_BLKSIZE >> SECTOR_SHIFT) >
+ queue_max_sectors(q))
+ return false;
+ }
+ return true;
}
static bool io_type_is_mergeable(struct f2fs_bio_info *io,
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index cd6e69f09e18..d0d47fc2762c 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -2383,7 +2383,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
fi->original_i_size = isize;
if (truncate) {
set_inode_flag(inode, FI_ATOMIC_REPLACE);
- truncate_inode_pages_final(inode->i_mapping);
+ truncate_inode_pages(inode->i_mapping, 0);
f2fs_i_size_write(inode, 0);
isize = 0;
}
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 8a11623dbf1e..231a9281968e 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -785,8 +785,20 @@ int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
if (!npage[0]) {
npage[0] = f2fs_get_node_page(sbi, nids[0]);
- if (IS_ERR(npage[0]))
- return PTR_ERR(npage[0]);
+ if (IS_ERR(npage[0])) {
+ err = PTR_ERR(npage[0]);
+ /*
+ * cp_error drops dirty node caches, so the inode block
+ * of an inode that was never written reads as NEW_ADDR
+ * and returns -ENOENT. It is not a hole; the inode is
+ * gone. Callers treat -ENOENT as a hole and keep
+ * looking block by block (e.g. fiemap up to maxbytes).
+ */
+ if (err == -ENOENT && (f2fs_cp_error(sbi) ||
+ is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN)))
+ err = -EIO;
+ return err;
+ }
}
/* if inline_data is set, should not report any block indices */
@@ -2767,6 +2779,45 @@ int f2fs_recover_inline_xattr(struct inode *inode, struct page *page)
f2fs_wait_on_page_writeback(ipage, NODE, true, true);
memcpy(dst_addr, src_addr, inline_size);
update_inode:
+ /*
+ * The xattr node of the checkpoint was removed before fsync, e.g. the
+ * xattrs became small enough to fit inline. Release it as well, or
+ * the inode keeps pointing to the stale xattrs.
+ */
+ if (!ri->i_xattr_nid && F2FS_I(inode)->i_xattr_nid) {
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ struct node_info ni;
+ int err;
+
+ err = f2fs_get_node_info(sbi, F2FS_I(inode)->i_xattr_nid,
+ &ni, false);
+ if (err) {
+ f2fs_put_page(ipage, 1);
+ return err;
+ }
+
+ if (ni.blk_addr != NEW_ADDR &&
+ !f2fs_is_valid_blkaddr(sbi, ni.blk_addr,
+ DATA_GENERIC_ENHANCE)) {
+ f2fs_err_ratelimited(sbi,
+ "nat entry is corrupted, run fsck to fix it, ino:%u, nid:%u, blkaddr:%u",
+ ni.ino, ni.nid, ni.blk_addr);
+ set_sbi_flag(sbi, SBI_NEED_FSCK);
+ f2fs_handle_error(sbi, ERROR_INCONSISTENT_NAT);
+ f2fs_put_page(ipage, 1);
+ return -EFSCORRUPTED;
+ }
+
+ f2fs_invalidate_blocks(sbi, ni.blk_addr);
+ dec_valid_node_count(sbi, inode, false);
+ set_node_addr(sbi, &ni, NULL_ADDR, false);
+ /* drop the cached xattr node, which may be dirty */
+ truncate_inode_pages_range(NODE_MAPPING(sbi),
+ F2FS_BLK_TO_BYTES((loff_t)ni.nid),
+ F2FS_BLK_END_BYTES((loff_t)ni.nid));
+ f2fs_i_xnid_write(inode, 0);
+ }
+
f2fs_update_inode(inode, ipage);
f2fs_put_page(ipage, 1);
return 0;
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 49146476882b..7559a85bb9df 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -193,7 +193,7 @@ void f2fs_abort_atomic_write(struct inode *inode, bool clean)
return;
if (clean)
- truncate_inode_pages_final(inode->i_mapping);
+ truncate_inode_pages(inode->i_mapping, 0);
release_atomic_write_cnt(inode);
clear_inode_flag(inode, FI_ATOMIC_COMMITTED);
@@ -2970,6 +2970,7 @@ static int change_curseg(struct f2fs_sb_info *sbi, int type)
if (IS_ERR(sum_page)) {
/* GC won't be able to use stale summary pages by cp_error */
memset(curseg->sum_blk, 0, SUM_ENTRY_SIZE);
+ reset_curseg_fields(curseg);
return PTR_ERR(sum_page);
}
sum_node = (struct f2fs_summary_block *)page_address(sum_page);
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 31724ab224d4..ec3e20cd6c5e 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -3773,6 +3773,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
block_t avail_node_count, valid_node_count;
unsigned int nat_blocks, nat_bits_bytes, nat_bits_blocks;
unsigned int sit_blk_cnt;
+ unsigned long long ckpt_size, sit_end, nat_end;
int i, j;
total = le32_to_cpu(raw_super->segment_count);
@@ -3904,6 +3905,29 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
return 1;
}
+ /*
+ * Nothing above ties the bitmap lengths to sbi->ckpt, which is
+ * (1 + cp_payload) blocks. Bound both ends the way __bitmap_ptr()
+ * lays them out.
+ */
+ ckpt_size = (unsigned long long)(cp_payload + 1) * F2FS_BLKSIZE;
+ if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG)) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + sizeof(__le32) + nat_bitmap_size;
+ sit_end = nat_end + sit_bitmap_size;
+ } else if (cp_payload > 0) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + nat_bitmap_size;
+ sit_end = F2FS_BLKSIZE + sit_bitmap_size;
+ } else {
+ sit_end = CP_MIN_CHKSUM_OFFSET + sit_bitmap_size;
+ nat_end = sit_end + nat_bitmap_size;
+ }
+
+ if (sit_end > ckpt_size || nat_end > ckpt_size) {
+ f2fs_err(sbi, "Bitmaps do not fit the checkpoint pack: sit ends at %llu, nat at %llu, pack is %llu bytes",
+ sit_end, nat_end, ckpt_size);
+ return 1;
+ }
+
if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG) &&
le32_to_cpu(ckpt->checksum_offset) != CP_MIN_CHKSUM_OFFSET) {
f2fs_warn(sbi, "using deprecated layout of large_nat_bitmap, "
diff --git a/fs/fat/inode.c b/fs/fat/inode.c
index 3852bb66358c..1c9988efc8d2 100644
--- a/fs/fat/inode.c
+++ b/fs/fat/inode.c
@@ -215,7 +215,9 @@ static void fat_write_failed(struct address_space *mapping, loff_t to)
if (to > inode->i_size) {
truncate_pagecache(inode, inode->i_size);
+ down_write(&MSDOS_I(inode)->truncate_lock);
fat_truncate_blocks(inode, inode->i_size);
+ up_write(&MSDOS_I(inode)->truncate_lock);
}
}
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 435f14b7e2cc..3b67018191df 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -757,6 +757,13 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb)
new_wb = &wb->bdi->wb; /* wb_get() is noop for bdi's wb */
nr = 0;
+ /*
+ * The scan moves walked inodes to the newest end of b_dirty_time,
+ * past ones it didn't reach. If one of those was dirtied after a
+ * sync's cutoff, the sync's queue_io() stops there and misses the
+ * moved ones. Exclude syncs as process_inode_switch_wbs() does.
+ */
+ down_read(&wb->bdi->wb_switch_rwsem);
/*
* Paired with synchronize_rcu() in cgroup_writeback_umount().
* Holding rcu_read_lock across the SB_ACTIVE check, the inode grab
@@ -779,6 +786,7 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb)
restart = isw_prepare_wbs_switch(new_wb, isw, &wb->b_dirty_time,
&nr);
spin_unlock(&wb->list_lock);
+ up_read(&wb->bdi->wb_switch_rwsem);
/* no attached inodes? bail out */
if (nr == 0) {
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index d242613a1c8a..cada359fd03a 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -2538,6 +2538,10 @@ static int fuse_write_end(struct file *file, struct address_space *mapping,
static int fuse_launder_folio(struct folio *folio)
{
int err = 0;
+
+ if (fuse_is_bad(folio->mapping->host))
+ return -EIO;
+
if (folio_clear_dirty_for_io(folio)) {
struct inode *inode = folio->mapping->host;
@@ -3344,9 +3348,11 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in,
if (err)
goto out;
- truncate_inode_pages_range(inode_out->i_mapping,
- ALIGN_DOWN(pos_out, PAGE_SIZE),
- ALIGN(pos_out + outarg.size, PAGE_SIZE) - 1);
+ if (outarg.size)
+ truncate_inode_pages_range(inode_out->i_mapping,
+ ALIGN_DOWN(pos_out, PAGE_SIZE),
+ ALIGN(pos_out + outarg.size,
+ PAGE_SIZE) - 1);
file_update_time(file_out);
fuse_write_update_attr(inode_out, pos_out + outarg.size, outarg.size);
diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c
index 06e59b820ffa..16baca6943ff 100644
--- a/fs/fuse/virtio_fs.c
+++ b/fs/fuse/virtio_fs.c
@@ -720,6 +720,10 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req)
unsigned int num_out;
unsigned int i;
+ /* fuse_request_end() returns this error; there are no args to copy. */
+ if (req->out.h.error)
+ goto out;
+
remaining = req->out.h.len - sizeof(req->out.h);
num_in = args->in_numargs - args->in_pages;
num_out = args->out_numargs - args->out_pages;
@@ -745,10 +749,57 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req)
if (args->out_argvar)
args->out_args[args->out_numargs - 1].size = remaining;
+out:
kfree(req->argbuf);
req->argbuf = NULL;
}
+/* Verify that the server properly follows the FUSE protocol */
+static bool virtio_fs_verify_response(struct fuse_req *req, unsigned int len)
+{
+ struct fuse_args *args = req->args;
+ struct fuse_out_header *oh = &req->out.h;
+ unsigned int expected;
+
+ if (len < sizeof(*oh)) {
+ pr_warn("virtio-fs: response too short (%u)\n", len);
+ return false;
+ }
+ if (oh->len != len) {
+ pr_warn("virtio-fs: oh.len mismatch (%u != %u)\n", oh->len, len);
+ return false;
+ }
+ if (oh->unique != req->in.h.unique) {
+ pr_warn("virtio-fs: oh.unique mismatch (%llu != %llu)\n",
+ oh->unique, req->in.h.unique);
+ return false;
+ }
+ if (oh->error <= -ERESTARTSYS || oh->error > 0) {
+ pr_warn("virtio-fs: invalid error value (%d)\n", oh->error);
+ return false;
+ }
+
+ if (oh->error) {
+ if (len != sizeof(*oh)) {
+ pr_warn("virtio-fs: error response too long (%u)\n", len);
+ return false;
+ }
+ return true;
+ }
+
+ expected = sizeof(*oh) +
+ fuse_len_args(args->out_numargs, args->out_args);
+ if (len > expected ||
+ (len < expected &&
+ (!args->out_argvar ||
+ expected - len > args->out_args[args->out_numargs - 1].size))) {
+ pr_warn("virtio-fs: invalid response length (%u, expected %u)\n",
+ len, expected);
+ return false;
+ }
+ return true;
+}
+
/* Work function for request completion */
static void virtio_fs_request_complete(struct fuse_req *req,
struct virtio_fs_vq *fsvq)
@@ -759,10 +810,6 @@ static void virtio_fs_request_complete(struct fuse_req *req,
unsigned int len, i, thislen;
struct page *page;
- /*
- * TODO verify that server properly follows FUSE protocol
- * (oh.uniq, oh.len)
- */
args = req->args;
copy_args_from_argbuf(args, req);
@@ -818,6 +865,10 @@ static void virtio_fs_requests_done_work(struct work_struct *work)
virtqueue_disable_cb(vq);
while ((req = virtqueue_get_buf(vq, &len)) != NULL) {
+ if (!virtio_fs_verify_response(req, len)) {
+ req->out.h.error = -EIO;
+ req->out.h.len = sizeof(struct fuse_out_header);
+ }
spin_lock(&fpq->lock);
list_move_tail(&req->list, &reqs);
spin_unlock(&fpq->lock);
diff --git a/fs/gfs2/glops.c b/fs/gfs2/glops.c
index d13a050bcb9d..dc5160827201 100644
--- a/fs/gfs2/glops.c
+++ b/fs/gfs2/glops.c
@@ -395,6 +395,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
umode_t mode = be32_to_cpu(str->di_mode);
struct inode *inode = &ip->i_inode;
bool is_new = inode->i_state & I_NEW;
+ u64 size;
if (unlikely(ip->i_no_addr != be64_to_cpu(str->di_num.no_addr))) {
gfs2_consist_inode(ip);
@@ -420,7 +421,12 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
i_uid_write(inode, be32_to_cpu(str->di_uid));
i_gid_write(inode, be32_to_cpu(str->di_gid));
set_nlink(inode, be32_to_cpu(str->di_nlink));
- i_size_write(inode, be64_to_cpu(str->di_size));
+ size = be64_to_cpu(str->di_size);
+ if (unlikely(size > inode->i_sb->s_maxbytes)) {
+ gfs2_consist_inode(ip);
+ return -EIO;
+ }
+ i_size_write(inode, size);
gfs2_set_inode_blocks(inode, be64_to_cpu(str->di_blocks));
atime.tv_sec = be64_to_cpu(str->di_atime);
atime.tv_nsec = be32_to_cpu(str->di_atime_nsec);
@@ -459,7 +465,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
ip->i_depth = (u8)depth;
ip->i_entries = be32_to_cpu(str->di_entries);
- if (gfs2_is_stuffed(ip) && inode->i_size > gfs2_max_stuffed_size(ip)) {
+ if (gfs2_is_stuffed(ip) && size > gfs2_max_stuffed_size(ip)) {
gfs2_consist_inode(ip);
return -EIO;
}
diff --git a/fs/inode.c b/fs/inode.c
index 7942673f686f..87396234d1e2 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -502,12 +502,11 @@ static void inode_pin_lru_isolating(struct inode *inode)
static void inode_unpin_lru_isolating(struct inode *inode)
{
- spin_lock(&inode->i_lock);
+ lockdep_assert_held(&inode->i_lock);
WARN_ON(!(inode->i_state & I_LRU_ISOLATING));
inode->i_state &= ~I_LRU_ISOLATING;
/* Called with inode->i_lock which ensures memory ordering. */
inode_wake_up_bit(inode, __I_LRU_ISOLATING);
- spin_unlock(&inode->i_lock);
}
static void inode_wait_for_lru_isolating(struct inode *inode)
@@ -890,6 +889,12 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
if (!spin_trylock(&inode->i_lock))
return LRU_SKIP;
+ /* Another walker is dropping its page cache, leave it to that one */
+ if (inode->i_state & I_LRU_ISOLATING) {
+ spin_unlock(&inode->i_lock);
+ return LRU_SKIP;
+ }
+
/*
* Inodes can get referenced, redirtied, or repopulated while
* they're already on the LRU, and this can make them
@@ -930,7 +935,13 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
__count_vm_events(PGINODESTEAL, reap);
mm_account_reclaimed_pages(reap);
}
+
+ spin_lock(&inode->i_lock);
+ /* The walk restarts here: don't retry leftovers right away */
+ if (!mapping_empty(&inode->i_data))
+ inode->i_state |= I_REFERENCED;
inode_unpin_lru_isolating(inode);
+ spin_unlock(&inode->i_lock);
spin_lock(lru_lock);
return LRU_RETRY;
}
diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
index 02a5fc5105e7..23457bdfc20d 100644
--- a/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -760,6 +760,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
if (iomap_block_needs_zeroing(iter, block_start)) {
if (WARN_ON_ONCE(iter->flags & IOMAP_UNSHARE))
return -EIO;
+ /* uptodate once copied, see __iomap_write_end() */
folio_zero_segments(folio, poff, from, to, poff + plen);
} else {
int status;
@@ -771,8 +772,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
poff, plen, srcmap);
if (status)
return status;
+ iomap_set_range_uptodate(folio, poff, plen);
}
- iomap_set_range_uptodate(folio, poff, plen);
} while ((block_start += plen) < block_end);
return 0;
diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
index c81cfcbb1bb6..92eb494aa0f0 100644
--- a/fs/iomap/direct-io.c
+++ b/fs/iomap/direct-io.c
@@ -250,19 +250,19 @@ void iomap_dio_bio_end_io(struct bio *bio)
}
EXPORT_SYMBOL_GPL(iomap_dio_bio_end_io);
-static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
+static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
loff_t pos, unsigned len)
{
struct inode *inode = file_inode(dio->iocb->ki_filp);
struct bio *bio;
if (!len)
- return 0;
+ return;
/*
* Max block size supported is 64k
*/
if (WARN_ON_ONCE(len > IOMAP_ZERO_PAGE_SIZE))
- return -EINVAL;
+ return;
bio = iomap_dio_alloc_bio(iter, dio, 1, REQ_OP_WRITE | REQ_SYNC | REQ_IDLE);
fscrypt_set_bio_crypt_ctx(bio, inode, pos >> inode->i_blkbits,
@@ -273,7 +273,6 @@ static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
__bio_add_page(bio, zero_page, len, 0);
iomap_dio_submit_bio(iter, dio, bio, pos);
- return 0;
}
/*
@@ -382,10 +381,7 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,
if (need_zeroout) {
/* zero out from the start of the block to the write offset */
pad = pos & (fs_block_size - 1);
-
- ret = iomap_dio_zero(iter, dio, pos - pad, pad);
- if (ret)
- goto out;
+ iomap_dio_zero(iter, dio, pos - pad, pad);
}
/*
@@ -463,8 +459,7 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter,
/* zero out from the end of the write to the end of the block */
pad = pos & (fs_block_size - 1);
if (pad)
- ret = iomap_dio_zero(iter, dio, pos,
- fs_block_size - pad);
+ iomap_dio_zero(iter, dio, pos, fs_block_size - pad);
}
out:
/* Undo iter limitation to current extent */
diff --git a/fs/nfs/callback_xdr.c b/fs/nfs/callback_xdr.c
index 1aa6a6e14b05..34fa00f1a1c2 100644
--- a/fs/nfs/callback_xdr.c
+++ b/fs/nfs/callback_xdr.c
@@ -275,6 +275,13 @@ __be32 decode_devicenotify_args(struct svc_rqst *rqstp,
if (n == 0)
goto out;
+ /* sanity check the count against the remaining stream */
+ if (n > xdr_stream_remaining(xdr) /
+ ((4 * sizeof(uint32_t)) + NFS4_DEVICEID4_SIZE)) {
+ status = htonl(NFS4ERR_BADXDR);
+ goto out;
+ }
+
args->devs = kmalloc_array(n, sizeof(*args->devs), GFP_KERNEL);
if (!args->devs) {
status = htonl(NFS4ERR_DELAY);
diff --git a/fs/nfs/filelayout/filelayoutdev.c b/fs/nfs/filelayout/filelayoutdev.c
index b878ea466361..7142958b1265 100644
--- a/fs/nfs/filelayout/filelayoutdev.c
+++ b/fs/nfs/filelayout/filelayoutdev.c
@@ -177,27 +177,14 @@ nfs4_fl_alloc_deviceid_node(struct nfs_server *server, struct pnfs_device *pdev,
trace_fl_getdevinfo(server, &pdev->dev_id, dsaddr->ds_list[i]->ds_remotestr);
/* If DS was already in cache, free ds addrs */
- while (!list_empty(&dsaddrs)) {
- da = list_first_entry(&dsaddrs,
- struct nfs4_pnfs_ds_addr,
- da_node);
- list_del_init(&da->da_node);
- kfree(da->da_remotestr);
- kfree(da);
- }
+ nfs4_pnfs_ds_addr_list_free(&dsaddrs);
}
__free_page(scratch);
return dsaddr;
out_err_drain_dsaddrs:
- while (!list_empty(&dsaddrs)) {
- da = list_first_entry(&dsaddrs, struct nfs4_pnfs_ds_addr,
- da_node);
- list_del_init(&da->da_node);
- kfree(da->da_remotestr);
- kfree(da);
- }
+ nfs4_pnfs_ds_addr_list_free(&dsaddrs);
out_err_free_deviceid:
nfs4_fl_free_deviceid(dsaddr);
/* stripe_indicies was part of dsaddr */
diff --git a/fs/nfs/flexfilelayout/flexfilelayoutdev.c b/fs/nfs/flexfilelayout/flexfilelayoutdev.c
index 53655581f9cb..f33e4f820dec 100644
--- a/fs/nfs/flexfilelayout/flexfilelayoutdev.c
+++ b/fs/nfs/flexfilelayout/flexfilelayoutdev.c
@@ -155,26 +155,13 @@ nfs4_ff_alloc_deviceid_node(struct nfs_server *server, struct pnfs_device *pdev,
goto out_err_drain_dsaddrs;
/* If DS was already in cache, free ds addrs */
- while (!list_empty(&dsaddrs)) {
- da = list_first_entry(&dsaddrs,
- struct nfs4_pnfs_ds_addr,
- da_node);
- list_del_init(&da->da_node);
- kfree(da->da_remotestr);
- kfree(da);
- }
+ nfs4_pnfs_ds_addr_list_free(&dsaddrs);
__free_page(scratch);
return new_ds;
out_err_drain_dsaddrs:
- while (!list_empty(&dsaddrs)) {
- da = list_first_entry(&dsaddrs, struct nfs4_pnfs_ds_addr,
- da_node);
- list_del_init(&da->da_node);
- kfree(da->da_remotestr);
- kfree(da);
- }
+ nfs4_pnfs_ds_addr_list_free(&dsaddrs);
kfree(ds_versions);
out_scratch:
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 14d2a91250f2..9d612bed74e3 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -426,6 +426,7 @@ int nfs4_pnfs_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds,
struct nfs4_pnfs_ds_addr *nfs4_decode_mp_ds_addr(struct net *net,
struct xdr_stream *xdr,
gfp_t gfp_flags);
+void nfs4_pnfs_ds_addr_list_free(struct list_head *dsaddrs);
void pnfs_layout_mark_request_commit(struct nfs_page *req,
struct pnfs_layout_segment *lseg,
struct nfs_commit_info *cinfo,
diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c
index 634ee58ced12..1e457dd3b8bd 100644
--- a/fs/nfs/pnfs_nfs.c
+++ b/fs/nfs/pnfs_nfs.c
@@ -632,23 +632,28 @@ static void nfs4_pnfs_ds_addr_free(struct nfs4_pnfs_ds_addr *da)
kfree(da);
}
-static void destroy_ds(struct nfs4_pnfs_ds *ds)
+void nfs4_pnfs_ds_addr_list_free(struct list_head *dsaddrs)
{
struct nfs4_pnfs_ds_addr *da;
+ while (!list_empty(dsaddrs)) {
+ da = list_first_entry(dsaddrs, struct nfs4_pnfs_ds_addr,
+ da_node);
+ list_del_init(&da->da_node);
+ nfs4_pnfs_ds_addr_free(da);
+ }
+}
+EXPORT_SYMBOL_GPL(nfs4_pnfs_ds_addr_list_free);
+
+static void destroy_ds(struct nfs4_pnfs_ds *ds)
+{
dprintk("--> %s\n", __func__);
ifdebug(FACILITY)
print_ds(ds);
nfs_put_client(ds->ds_clp);
- while (!list_empty(&ds->ds_addrs)) {
- da = list_first_entry(&ds->ds_addrs,
- struct nfs4_pnfs_ds_addr,
- da_node);
- list_del_init(&da->da_node);
- nfs4_pnfs_ds_addr_free(da);
- }
+ nfs4_pnfs_ds_addr_list_free(&ds->ds_addrs);
kfree(ds->ds_remotestr);
kfree(ds);
diff --git a/fs/nfsd/blocklayout.c b/fs/nfsd/blocklayout.c
index bbd87cd16282..de789cefd30d 100644
--- a/fs/nfsd/blocklayout.c
+++ b/fs/nfsd/blocklayout.c
@@ -18,8 +18,8 @@
static __be32
-nfsd4_block_proc_layoutget(struct inode *inode, const struct svc_fh *fhp,
- struct nfsd4_layoutget *args)
+nfsd4_block_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode,
+ const struct svc_fh *fhp, struct nfsd4_layoutget *args)
{
struct nfsd4_layout_seg *seg = &args->lg_seg;
struct super_block *sb = inode->i_sb;
diff --git a/fs/nfsd/flexfilelayout.c b/fs/nfsd/flexfilelayout.c
index 3c4419da5e24..64b85b9fd9f4 100644
--- a/fs/nfsd/flexfilelayout.c
+++ b/fs/nfsd/flexfilelayout.c
@@ -14,19 +14,20 @@
#include <linux/sunrpc/addr.h>
#include "flexfilelayoutxdr.h"
+#include "auth.h"
#include "pnfs.h"
#include "vfs.h"
#define NFSDDBG_FACILITY NFSDDBG_PNFS
static __be32
-nfsd4_ff_proc_layoutget(struct inode *inode, const struct svc_fh *fhp,
- struct nfsd4_layoutget *args)
+nfsd4_ff_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode,
+ const struct svc_fh *fhp, struct nfsd4_layoutget *args)
{
+ struct user_namespace *userns = nfsd_user_namespace(rqstp);
struct nfsd4_layout_seg *seg = &args->lg_seg;
u32 device_generation = 0;
int error;
- uid_t u;
struct pnfs_ff_layout *fl;
@@ -49,13 +50,16 @@ nfsd4_ff_proc_layoutget(struct inode *inode, const struct svc_fh *fhp,
fl->flags = FF_FLAGS_NO_LAYOUTCOMMIT | FF_FLAGS_NO_IO_THRU_MDS |
FF_FLAGS_NO_READ_IO;
- /* Do not allow a IOMODE_READ segment to have write pemissions */
- if (seg->iomode == IOMODE_READ) {
- u = from_kuid(&init_user_ns, inode->i_uid) + 1;
- fl->uid = make_kuid(&init_user_ns, u);
- } else
- fl->uid = inode->i_uid;
- fl->gid = inode->i_gid;
+ fl->uid = from_kuid_munged(userns, inode->i_uid);
+ fl->gid = from_kgid_munged(userns, inode->i_gid);
+
+ /*
+ * Do not allow an IOMODE_READ segment to have write permissions.
+ * The group is left intact so group-readable files stay readable;
+ * nfsd_setuser() squashes an unmapped uid to the export's anon ID.
+ */
+ if (seg->iomode == IOMODE_READ)
+ fl->uid++;
error = nfsd4_set_deviceid(&fl->deviceid, fhp, device_generation);
if (error)
diff --git a/fs/nfsd/flexfilelayoutxdr.c b/fs/nfsd/flexfilelayoutxdr.c
index 374e52d3064a..b80352f41c0d 100644
--- a/fs/nfsd/flexfilelayoutxdr.c
+++ b/fs/nfsd/flexfilelayoutxdr.c
@@ -32,8 +32,8 @@ nfsd4_ff_encode_layoutget(struct xdr_stream *xdr,
fh_len = 4 + xdr_align_size(fl->fh.size);
- uid.len = sprintf(uid.buf, "%u", from_kuid(&init_user_ns, fl->uid));
- gid.len = sprintf(gid.buf, "%u", from_kgid(&init_user_ns, fl->gid));
+ uid.len = sprintf(uid.buf, "%u", fl->uid);
+ gid.len = sprintf(gid.buf, "%u", fl->gid);
/* data server entry: deviceid + efficiency + stateid + fh list +
* user + group + flags + stats_collect_hint
diff --git a/fs/nfsd/flexfilelayoutxdr.h b/fs/nfsd/flexfilelayoutxdr.h
index 6d5a1066a903..3e1876d49db2 100644
--- a/fs/nfsd/flexfilelayoutxdr.h
+++ b/fs/nfsd/flexfilelayoutxdr.h
@@ -35,8 +35,9 @@ struct pnfs_ff_device_addr {
struct pnfs_ff_layout {
u32 flags;
u32 stats_collect_hint;
- kuid_t uid;
- kgid_t gid;
+ /* Values to encode; nfsd4_ff_proc_layoutget() has mapped these */
+ u32 uid;
+ u32 gid;
struct nfsd4_deviceid deviceid;
stateid_t stateid;
struct nfs_fh fh;
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index 2998640f259d..cf92e1f8df7d 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -15,6 +15,20 @@
#define NFSDDBG_FACILITY NFSDDBG_PROC
+static __be32 nfsacld_map_status(__be32 status)
+{
+ switch (status) {
+ case nfserr_nofilehandle:
+ case nfserr_badhandle:
+ status = nfserr_stale;
+ break;
+ case nfserr_wrongsec:
+ status = nfserr_acces;
+ break;
+ }
+ return status;
+}
+
/*
* NULL call.
*/
@@ -79,6 +93,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst *rqstp)
/* resp->acl_{access,default} are released in nfssvc_release_getacl. */
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
fail:
@@ -137,6 +152,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp)
out:
/* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
out_drop_lock:
@@ -163,6 +179,7 @@ static __be32 nfsacld_proc_getattr(struct svc_rqst *rqstp)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
@@ -185,6 +202,7 @@ static __be32 nfsacld_proc_access(struct svc_rqst *rqstp)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 798fcc9c4092..47cfd4bd4ecc 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -1227,6 +1227,7 @@ static void nfsd41_destroy_cb(struct nfsd4_callback *cb)
trace_nfsd_cb_destroy(clp, cb);
nfsd41_cb_release_slot(cb);
+ clear_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags);
if (cb->cb_ops && cb->cb_ops->release)
cb->cb_ops->release(cb);
nfsd41_cb_inflight_end(clp);
@@ -1575,6 +1576,7 @@ void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp,
cb->cb_msg.rpc_proc = &nfs4_cb_procedures[op];
cb->cb_msg.rpc_argp = cb;
cb->cb_msg.rpc_resp = cb;
+ cb->cb_flags = 0;
cb->cb_ops = ops;
INIT_WORK(&cb->cb_work, nfsd4_run_cb_work);
cb->cb_status = 0;
diff --git a/fs/nfsd/nfs4layouts.c b/fs/nfsd/nfs4layouts.c
index 0ff899475263..1f37044db59f 100644
--- a/fs/nfsd/nfs4layouts.c
+++ b/fs/nfsd/nfs4layouts.c
@@ -125,6 +125,7 @@ nfsd4_set_deviceid(struct nfsd4_deviceid *id, const struct svc_fh *fhp,
void nfsd4_setup_layout_type(struct svc_export *exp)
{
+ struct nfsd_net *nn = net_generic(exp->cd->net, nfsd_net_id);
#if defined(CONFIG_NFSD_BLOCKLAYOUT) || defined(CONFIG_NFSD_SCSILAYOUT)
struct super_block *sb = exp->ex_path.mnt->mnt_sb;
#endif
@@ -132,9 +133,9 @@ void nfsd4_setup_layout_type(struct svc_export *exp)
if (!(exp->ex_flags & NFSEXP_PNFS))
return;
-#ifdef CONFIG_NFSD_FLEXFILELAYOUT
- exp->ex_layout_types |= 1 << LAYOUT_FLEX_FILES;
-#endif
+ if (IS_ENABLED(CONFIG_NFSD_FLEXFILELAYOUT) &&
+ nfsd_vers(nn, 3, NFSD_TEST))
+ exp->ex_layout_types |= 1 << LAYOUT_FLEX_FILES;
#ifdef CONFIG_NFSD_BLOCKLAYOUT
if (sb->s_export_op->get_uuid &&
sb->s_export_op->map_blocks &&
@@ -347,9 +348,10 @@ nfsd4_recall_file_layout(struct nfs4_layout_stateid *ls)
atomic_inc(&ls->ls_stid.sc_file->fi_lo_recalls);
trace_nfsd_layout_recall(&ls->ls_stid.sc_stateid);
- refcount_inc(&ls->ls_stid.sc_count);
- nfsd4_run_cb(&ls->ls_recall);
-
+ if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ls->ls_recall.cb_flags)) {
+ refcount_inc(&ls->ls_stid.sc_count);
+ nfsd4_run_cb(&ls->ls_recall);
+ }
out_unlock:
spin_unlock(&ls->ls_lock);
}
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 53ded5de593a..3d03ff6313d7 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1769,7 +1769,7 @@ static void nfsd4_send_cb_offload(struct nfsd4_copy *copy)
NFSPROC4_CLNT_CB_OFFLOAD);
trace_nfsd_cb_offload(copy->cp_clp, &cbo->co_res.cb_stateid,
&cbo->co_fh, copy->cp_count, copy->nfserr);
- nfsd4_run_cb(&cbo->co_cb);
+ nfsd4_try_run_cb(&cbo->co_cb);
}
/**
@@ -2359,7 +2359,7 @@ nfsd4_layoutget(struct svc_rqst *rqstp,
if (atomic_read(&ls->ls_stid.sc_file->fi_lo_recalls))
goto out_put_stid;
- nfserr = ops->proc_layoutget(d_inode(current_fh->fh_dentry),
+ nfserr = ops->proc_layoutget(rqstp, d_inode(current_fh->fh_dentry),
current_fh, lgp);
if (nfserr)
goto out_put_stid;
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index 104dafbe881b..ef0388d92aca 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -730,6 +730,7 @@ struct cld_upcall {
struct completion cu_done;
/* daemon has read the whole upcall; protected by cn_lock */
bool cu_inflight;
+ struct rpc_pipe_msg cu_pipe_msg;
union {
struct cld_msg_hdr cu_hdr;
struct cld_msg cu_msg;
@@ -741,22 +742,22 @@ static int
__cld_pipe_upcall(struct rpc_pipe *pipe, void *cmsg, struct nfsd_net *nn)
{
int ret;
- struct rpc_pipe_msg msg;
struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, cu_u);
+ struct rpc_pipe_msg *msg = &cup->cu_pipe_msg;
- memset(&msg, 0, sizeof(msg));
- msg.data = cmsg;
- msg.len = nn->client_tracking_ops->msglen;
+ memset(msg, 0, sizeof(*msg));
+ msg->data = cmsg;
+ msg->len = nn->client_tracking_ops->msglen;
- ret = rpc_queue_upcall(pipe, &msg);
+ ret = rpc_queue_upcall(pipe, msg);
if (ret < 0) {
goto out;
}
wait_for_completion(&cup->cu_done);
- if (msg.errno < 0)
- ret = msg.errno;
+ if (msg->errno < 0)
+ ret = msg->errno;
out:
return ret;
}
@@ -919,8 +920,11 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
if (status == -EINPROGRESS)
return __cld_pipe_inprogress_downcall(cmsg, nn);
- if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0)
+ if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) {
+ cup->cu_pipe_msg.errno = -EFAULT;
+ complete(&cup->cu_done);
return -EFAULT;
+ }
complete(&cup->cu_done);
return mlen;
@@ -929,9 +933,8 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
static void
cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
{
- struct cld_msg *cmsg = msg->data;
- struct cld_upcall *cup = container_of(cmsg, struct cld_upcall,
- cu_u.cu_msg);
+ struct cld_upcall *cup = container_of(msg, struct cld_upcall,
+ cu_pipe_msg);
struct cld_net *cn = cup->cu_net;
/*
@@ -948,10 +951,34 @@ cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
complete(&cup->cu_done);
}
+/*
+ * An upcall the daemon has consumed is off every pipe list, so the
+ * purge in rpc_pipe_release() and rpc_close_pipes() cannot reach it.
+ * Release its waiter here instead; no downcall can arrive now.
+ */
+static void
+cld_release_pipe(struct inode *inode)
+{
+ struct nfsd_net *nn = net_generic(inode->i_sb->s_fs_info, nfsd_net_id);
+ struct cld_net *cn = nn->cld_net;
+ struct cld_upcall *cup;
+
+ spin_lock(&cn->cn_lock);
+ list_for_each_entry(cup, &cn->cn_list, cu_list) {
+ if (!cup->cu_inflight)
+ continue;
+ cup->cu_inflight = false;
+ cup->cu_pipe_msg.errno = -EPIPE;
+ complete(&cup->cu_done);
+ }
+ spin_unlock(&cn->cn_lock);
+}
+
static const struct rpc_pipe_ops cld_upcall_ops = {
.upcall = rpc_pipe_generic_upcall,
.downcall = cld_pipe_downcall,
.destroy_msg = cld_pipe_destroy_msg,
+ .release_pipe = cld_release_pipe,
};
static int
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 723a1ae68836..061ce98eb414 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1420,6 +1420,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
dp->dl_stid.sc_type = SC_TYPE_DELEG;
list_add(&dp->dl_perfile, &fp->fi_delegations);
list_add(&dp->dl_perclnt, &clp->cl_delegations);
+ clp->cl_deleg_count++;
return 0;
}
@@ -1451,6 +1452,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask)
++dp->dl_time;
spin_lock(&fp->fi_lock);
list_del_init(&dp->dl_perclnt);
+ dp->dl_stid.sc_client->cl_deleg_count--;
list_del_init(&dp->dl_recall_lru);
list_del_init(&dp->dl_perfile);
spin_unlock(&fp->fi_lock);
@@ -3246,7 +3248,6 @@ nfsd4_cb_recall_any_release(struct nfsd4_callback *cb)
{
struct nfs4_client *clp = cb->cb_clp;
- clear_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags);
drop_client(clp);
}
@@ -3303,8 +3304,10 @@ static void nfs4_cb_getattr(struct nfs4_cb_fattr *ncf)
/* set to proper status when nfsd4_cb_getattr_done runs */
ncf->ncf_cb_status = NFS4ERR_IO;
- refcount_inc(&dp->dl_stid.sc_count);
- nfsd4_run_cb(&ncf->ncf_getattr);
+ if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ncf->ncf_getattr.cb_flags)) {
+ refcount_inc(&dp->dl_stid.sc_count);
+ nfsd4_run_cb(&ncf->ncf_getattr);
+ }
}
static struct nfs4_client *create_client(struct xdr_netobj name,
@@ -5477,6 +5480,10 @@ static const struct nfsd4_callback_ops nfsd4_cb_recall_ops = {
static void nfsd_break_one_deleg(struct nfs4_delegation *dp)
{
bool queued;
+
+ if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &dp->dl_recall.cb_flags))
+ return;
+
/*
* We're assuming the state code never drops its reference
* without first removing the lease. Since we're in this lease
@@ -5487,8 +5494,10 @@ static void nfsd_break_one_deleg(struct nfs4_delegation *dp)
refcount_inc(&dp->dl_stid.sc_count);
queued = nfsd4_run_cb(&dp->dl_recall);
WARN_ON_ONCE(!queued);
- if (!queued)
+ if (!queued) {
refcount_dec(&dp->dl_stid.sc_count);
+ clear_bit(NFSD4_CALLBACK_RUNNING, &dp->dl_recall.cb_flags);
+ }
}
/* Called from break_lease() with flc_lock held. */
@@ -6999,7 +7008,7 @@ deleg_reaper(struct nfsd_net *nn)
{
struct list_head *pos, *next;
struct nfs4_client *clp;
- LIST_HEAD(cblist);
+ unsigned int count;
spin_lock(&nn->client_lock);
list_for_each_safe(pos, next, &nn->client_lru) {
@@ -7009,35 +7018,39 @@ deleg_reaper(struct nfsd_net *nn)
continue;
if (clp->cl_state != NFSD4_ACTIVE)
continue;
- if (list_empty(&clp->cl_delegations))
- continue;
if (atomic_read(&clp->cl_delegs_in_recall))
continue;
- if (test_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags))
- continue;
if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5)
continue;
if (clp->cl_cb_state != NFSD4_CB_UP)
continue;
- list_add(&clp->cl_ra_cblist, &cblist);
+ /*
+ * This read races with hash_delegation_locked() and
+ * unhash_delegation_locked() on other CPUs. A stale
+ * count only skews the keep value; the next
+ * laundromat pass sees a more current one.
+ */
+ count = data_race(READ_ONCE(clp->cl_deleg_count));
+ if (!count)
+ continue;
+ if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags))
+ continue;
/* release in nfsd4_cb_recall_any_release */
kref_get(&clp->cl_nfsdfs.cl_ref);
- set_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags);
clp->cl_ra_time = ktime_get_boottime_seconds();
- }
- spin_unlock(&nn->client_lock);
-
- while (!list_empty(&cblist)) {
- clp = list_first_entry(&cblist, struct nfs4_client,
- cl_ra_cblist);
- list_del_init(&clp->cl_ra_cblist);
- clp->cl_ra->ra_keep = 0;
+ /*
+ * Ask for a single delegation. Recalling one before it
+ * is needed costs the client an OPEN when it next
+ * touches the file.
+ */
+ clp->cl_ra->ra_keep = count - 1;
clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) |
BIT(RCA4_TYPE_MASK_WDATA_DLG);
trace_nfsd_cb_recall_any(clp->cl_ra);
nfsd4_run_cb(&clp->cl_ra->ra_cb);
}
+ spin_unlock(&nn->client_lock);
}
static void
@@ -7977,7 +7990,7 @@ nfsd4_lm_notify(struct file_lock *fl)
if (queue) {
trace_nfsd_cb_notify_lock(lo, nbl);
- nfsd4_run_cb(&nbl->nbl_cb);
+ nfsd4_try_run_cb(&nbl->nbl_cb);
}
}
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 2a225756ef0d..f2e1079bd043 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -2107,8 +2107,13 @@ static int nfsd_nl_validate_listeners(struct genl_info *info)
int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
+ const struct nlattr *bad_attr = NULL;
struct svc_xprt *xprt, *tmp;
+ const char *bad_xprt = NULL;
+ unsigned int rpcb_failures;
const struct nlattr *attr;
+ bool skipped_rpcb = false;
+ bool bad_rpcb = false;
struct svc_serv *serv;
LIST_HEAD(permsocks);
struct nfsd_net *nn;
@@ -2198,13 +2203,16 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
if (delete)
svc_xprt_destroy_all(serv, net);
+ rpcb_failures = svc_rpcb_failure_count(serv);
+
/* walk list of addrs again, open any that still don't exist */
nlmsg_for_each_attr_type(attr, NFSD_A_SERVER_SOCK_ADDR, info->nlhdr,
GENL_HDRLEN, rem) {
struct nlattr *tb[NFSD_A_SOCK_MAX + 1];
const char *xcl_name;
struct sockaddr *sa;
- int ret;
+ bool hit_rpcb;
+ int flags, ret;
/* validated up front in nfsd_nl_validate_listeners() */
if (nla_parse_nested(tb, NFSD_A_SOCK_MAX, attr,
@@ -2223,11 +2231,53 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
continue;
}
- ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, 0,
+ flags = skipped_rpcb ? SVC_SOCK_ANONYMOUS : 0;
+ ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, flags,
current_cred());
+
+ hit_rpcb = false;
+ if (!skipped_rpcb &&
+ svc_rpcb_failure_count(serv) != rpcb_failures) {
+ skipped_rpcb = true;
+ hit_rpcb = true;
+ if (ret < 0)
+ ret = svc_xprt_create_from_sa(serv, xcl_name,
+ net, sa,
+ SVC_SOCK_ANONYMOUS,
+ current_cred());
+ }
+
/* always save the latest error */
- if (ret < 0)
+ if (ret < 0) {
+ bad_attr = attr;
+ bad_xprt = xcl_name;
+ bad_rpcb = hit_rpcb;
err = ret;
+ }
+ }
+
+ /*
+ * The ack carries the errno of the last entry that failed. Point at
+ * that entry as well, since several entries can share a transport
+ * name and the errno alone cannot tell them apart.
+ */
+ if (err) {
+ NL_SET_BAD_ATTR(info->extack, bad_attr);
+ if (bad_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer",
+ bad_xprt);
+ else if (skipped_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer earlier",
+ bad_xprt);
+ else
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener",
+ bad_xprt);
+ } else if (skipped_rpcb) {
+ NL_SET_ERR_MSG(info->extack,
+ "rpcbind did not answer, some listeners are not registered");
}
if (!serv->sv_nrthreads && list_empty(&nn->nfsd_serv->sv_permsocks))
diff --git a/fs/nfsd/pnfs.h b/fs/nfsd/pnfs.h
index dfd411d1f363..db9af780438b 100644
--- a/fs/nfsd/pnfs.h
+++ b/fs/nfsd/pnfs.h
@@ -29,8 +29,8 @@ struct nfsd4_layout_ops {
__be32 (*encode_getdeviceinfo)(struct xdr_stream *xdr,
const struct nfsd4_getdeviceinfo *gdevp);
- __be32 (*proc_layoutget)(struct inode *, const struct svc_fh *fhp,
- struct nfsd4_layoutget *lgp);
+ __be32 (*proc_layoutget)(struct svc_rqst *rqstp, struct inode *inode,
+ const struct svc_fh *fhp, struct nfsd4_layoutget *lgp);
__be32 (*encode_layoutget)(struct xdr_stream *xdr,
const struct nfsd4_layoutget *lgp);
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index f39a425639b0..1909d744d4ac 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -67,6 +67,8 @@ typedef struct {
struct nfsd4_callback {
struct nfs4_client *cb_clp;
struct rpc_message cb_msg;
+#define NFSD4_CALLBACK_RUNNING (0)
+ unsigned long cb_flags;
const struct nfsd4_callback_ops *cb_ops;
struct work_struct cb_work;
int cb_seq_status;
@@ -410,7 +412,6 @@ struct nfs4_client {
#define NFSD4_CLIENT_UPCALL_LOCK (5) /* upcall serialization */
#define NFSD4_CLIENT_CB_FLAG_MASK (1 << NFSD4_CLIENT_CB_UPDATE | \
1 << NFSD4_CLIENT_CB_KILL)
-#define NFSD4_CLIENT_CB_RECALL_ANY (6)
unsigned long cl_flags;
struct workqueue_struct *cl_callback_wq;
@@ -456,10 +457,11 @@ struct nfs4_client {
unsigned int cl_state;
atomic_t cl_delegs_in_recall;
+ /* Length of cl_delegations, updated under nn->deleg_lock */
+ unsigned int cl_deleg_count;
struct nfsd4_cb_recall_any *cl_ra;
time64_t cl_ra_time;
- struct list_head cl_ra_cblist;
};
/* struct nfs4_client_reset
@@ -748,6 +750,13 @@ extern void nfsd4_change_callback(struct nfs4_client *clp, struct nfs4_cb_conn *
extern void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp,
const struct nfsd4_callback_ops *ops, enum nfsd4_cb_op op);
extern bool nfsd4_run_cb(struct nfsd4_callback *cb);
+
+static inline void nfsd4_try_run_cb(struct nfsd4_callback *cb)
+{
+ if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags))
+ WARN_ON_ONCE(!nfsd4_run_cb(cb));
+}
+
extern void nfsd4_shutdown_callback(struct nfs4_client *);
extern void nfsd4_shutdown_copy(struct nfs4_client *clp);
extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(struct xdr_netobj name,
diff --git a/fs/nilfs2/dir.c b/fs/nilfs2/dir.c
index b0f182b1da8d..287e7c8e5dcc 100644
--- a/fs/nilfs2/dir.c
+++ b/fs/nilfs2/dir.c
@@ -95,7 +95,7 @@ static void nilfs_commit_chunk(struct folio *folio,
unsigned int nr_dirty;
int err;
- nr_dirty = nilfs_page_count_clean_buffers(&folio->page, from, to);
+ nr_dirty = nilfs_page_count_clean_buffers(folio, from, to);
copied = block_write_end(NULL, mapping, pos, len, len, folio, NULL);
if (pos + copied > dir->i_size)
i_size_write(dir, pos + copied);
diff --git a/fs/nilfs2/gcinode.c b/fs/nilfs2/gcinode.c
index 2dbb15767df1..561c220799c7 100644
--- a/fs/nilfs2/gcinode.c
+++ b/fs/nilfs2/gcinode.c
@@ -46,14 +46,11 @@
* specified by @pbn to the GC pagecache with the key @blkoff.
* This function sets @vbn (@pbn if @vbn is zero) in b_blocknr of the buffer.
*
- * Return Value: On success, 0 is returned. On Error, one of the following
- * negative error code is returned.
- *
- * %-EIO - I/O error.
- *
- * %-ENOMEM - Insufficient amount of memory available.
- *
- * %-ENOENT - The block specified with @pbn does not exist.
+ * Return: 0 on success, or one of the following negative error codes on
+ * failure:
+ * * %-EIO - I/O error (including metadata corruption).
+ * * %-ENOENT - The block specified with @pbn does not exist.
+ * * %-ENOMEM - Insufficient memory available.
*/
int nilfs_gccache_submit_read_data(struct inode *inode, sector_t blkoff,
sector_t pbn, __u64 vbn,
@@ -114,12 +111,11 @@ int nilfs_gccache_submit_read_data(struct inode *inode, sector_t blkoff,
* specified by @vbn to the GC pagecache. @pbn can be supplied by the
* caller to avoid translation of the disk block address.
*
- * Return Value: On success, 0 is returned. On Error, one of the following
- * negative error code is returned.
- *
- * %-EIO - I/O error.
- *
- * %-ENOMEM - Insufficient amount of memory available.
+ * Return: 0 on success, or one of the following negative error codes on
+ * failure:
+ * * %-EIO - I/O error (including metadata corruption).
+ * * %-ENOENT - Invalid virtual block address.
+ * * %-ENOMEM - Insufficient memory available.
*/
int nilfs_gccache_submit_read_node(struct inode *inode, sector_t pbn,
__u64 vbn, struct buffer_head **out_bh)
diff --git a/fs/nilfs2/inode.c b/fs/nilfs2/inode.c
index 6932f8daf7d9..7481912284fc 100644
--- a/fs/nilfs2/inode.c
+++ b/fs/nilfs2/inode.c
@@ -159,7 +159,7 @@ static int nilfs_writepages(struct address_space *mapping,
int err = 0;
if (sb_rdonly(inode->i_sb)) {
- nilfs_clear_dirty_pages(mapping);
+ nilfs_clear_dirty_pages(mapping, false);
return -EROFS;
}
@@ -170,37 +170,6 @@ static int nilfs_writepages(struct address_space *mapping,
return err;
}
-static int nilfs_writepage(struct page *page, struct writeback_control *wbc)
-{
- struct folio *folio = page_folio(page);
- struct inode *inode = folio->mapping->host;
- int err;
-
- if (sb_rdonly(inode->i_sb)) {
- /*
- * It means that filesystem was remounted in read-only
- * mode because of error or metadata corruption. But we
- * have dirty pages that try to be flushed in background.
- * So, here we simply discard this dirty page.
- */
- nilfs_clear_folio_dirty(folio);
- folio_unlock(folio);
- return -EROFS;
- }
-
- folio_redirty_for_writepage(wbc, folio);
- folio_unlock(folio);
-
- if (wbc->sync_mode == WB_SYNC_ALL) {
- err = nilfs_construct_segment(inode->i_sb);
- if (unlikely(err))
- return err;
- } else if (wbc->for_reclaim)
- nilfs_flush_segment(inode->i_sb, inode->i_ino);
-
- return 0;
-}
-
static bool nilfs_dirty_folio(struct address_space *mapping,
struct folio *folio)
{
@@ -273,7 +242,7 @@ static int nilfs_write_end(struct file *file, struct address_space *mapping,
unsigned int nr_dirty;
int err;
- nr_dirty = nilfs_page_count_clean_buffers(&folio->page, start,
+ nr_dirty = nilfs_page_count_clean_buffers(folio, start,
start + copied);
copied = generic_write_end(file, mapping, pos, len, copied, folio,
fsdata);
@@ -295,7 +264,6 @@ nilfs_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
}
const struct address_space_operations nilfs_aops = {
- .writepage = nilfs_writepage,
.read_folio = nilfs_read_folio,
.writepages = nilfs_writepages,
.dirty_folio = nilfs_dirty_folio,
@@ -304,6 +272,7 @@ const struct address_space_operations nilfs_aops = {
.write_end = nilfs_write_end,
.invalidate_folio = block_invalidate_folio,
.direct_IO = nilfs_direct_IO,
+ .migrate_folio = buffer_migrate_folio_norefs,
.is_partially_uptodate = block_is_partially_uptodate,
};
@@ -636,10 +605,7 @@ struct inode *nilfs_iget_for_gc(struct super_block *sb, unsigned long ino,
* or does nothing if the inode already has it. This function allocates
* an additional inode to maintain page cache of B-tree nodes one-on-one.
*
- * Return Value: On success, 0 is returned. On errors, one of the following
- * negative error code is returned.
- *
- * %-ENOMEM - Insufficient memory available.
+ * Return: 0 on success, or %-ENOMEM if memory is insufficient.
*/
int nilfs_attach_btree_node_cache(struct inode *inode)
{
@@ -698,11 +664,8 @@ void nilfs_detach_btree_node_cache(struct inode *inode)
* in one inode and the one for b-tree node pages is set up in the
* other inode, which is attached to the former inode.
*
- * Return Value: On success, a pointer to the inode for data pages is
- * returned. On errors, one of the following negative error code is returned
- * in a pointer type.
- *
- * %-ENOMEM - Insufficient memory available.
+ * Return: a pointer to the inode for data pages on success, or %-ENOMEM
+ * if memory is insufficient.
*/
struct inode *nilfs_iget_for_shadow(struct inode *inode)
{
diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c
index 03062b6bc323..11ccf18c3999 100644
--- a/fs/nilfs2/ioctl.c
+++ b/fs/nilfs2/ioctl.c
@@ -972,10 +972,10 @@ static int nilfs_ioctl_clean_segments(struct inode *inode, struct file *filp,
* Return: 0 on success, or one of the following negative error codes on
* failure:
* * %-EFAULT - Failure during execution of requested operation.
+ * * %-EINTR - Interrupted.
* * %-EIO - I/O error.
* * %-ENOMEM - Insufficient memory available.
* * %-ENOSPC - No space left on device (only in a panic state).
- * * %-ERESTARTSYS - Interrupted.
* * %-EROFS - Read only filesystem.
*/
static int nilfs_ioctl_sync(struct inode *inode, struct file *filp,
diff --git a/fs/nilfs2/mdt.c b/fs/nilfs2/mdt.c
index 2db6350b5ac2..8e66f2c13ad6 100644
--- a/fs/nilfs2/mdt.c
+++ b/fs/nilfs2/mdt.c
@@ -410,7 +410,7 @@ nilfs_mdt_write_page(struct page *page, struct writeback_control *wbc)
* have dirty folios that try to be flushed in background.
* So, here we simply discard this dirty folio.
*/
- nilfs_clear_folio_dirty(folio);
+ nilfs_clear_folio_dirty(folio, false);
folio_unlock(folio);
return -EROFS;
}
@@ -637,10 +637,10 @@ void nilfs_mdt_restore_from_shadow_map(struct inode *inode)
if (mi->mi_palloc_cache)
nilfs_palloc_clear_cache(inode);
- nilfs_clear_dirty_pages(inode->i_mapping);
+ nilfs_clear_dirty_pages(inode->i_mapping, true);
nilfs_copy_back_pages(inode->i_mapping, shadow->inode->i_mapping);
- nilfs_clear_dirty_pages(ii->i_assoc_inode->i_mapping);
+ nilfs_clear_dirty_pages(ii->i_assoc_inode->i_mapping, true);
nilfs_copy_back_pages(ii->i_assoc_inode->i_mapping,
NILFS_I(shadow->inode)->i_assoc_inode->i_mapping);
diff --git a/fs/nilfs2/page.c b/fs/nilfs2/page.c
index bc698bb66f79..2fc49a26d0b0 100644
--- a/fs/nilfs2/page.c
+++ b/fs/nilfs2/page.c
@@ -371,8 +371,9 @@ void nilfs_copy_back_pages(struct address_space *dmap,
/**
* nilfs_clear_dirty_pages - discard dirty pages in address space
* @mapping: address space with dirty pages for discarding
+ * @force: whether to clear the dirty state regardless of busy buffer heads
*/
-void nilfs_clear_dirty_pages(struct address_space *mapping)
+void nilfs_clear_dirty_pages(struct address_space *mapping, bool force)
{
struct folio_batch fbatch;
unsigned int i;
@@ -393,7 +394,7 @@ void nilfs_clear_dirty_pages(struct address_space *mapping)
* was acquired. Skip processing in that case.
*/
if (likely(folio->mapping == mapping))
- nilfs_clear_folio_dirty(folio);
+ nilfs_clear_folio_dirty(folio, force);
folio_unlock(folio);
}
@@ -405,13 +406,16 @@ void nilfs_clear_dirty_pages(struct address_space *mapping)
/**
* nilfs_clear_folio_dirty - discard dirty folio
* @folio: dirty folio that will be discarded
+ * @force: whether to clear the states regardless of busy buffer heads
*
* nilfs_clear_folio_dirty() clears working states including dirty state for
- * the folio and its buffers. If the folio has buffers, clear only if it is
- * confirmed that none of the buffer heads are busy (none have valid
- * references and none are locked).
+ * the folio and its buffers. If the folio has buffers and force is false,
+ * clear only if it is confirmed that none of the buffer heads are busy (none
+ * have valid references and none are locked). If force is true, the states
+ * are cleared unconditionally, the caller should guarantee that the folio is
+ * not being modified concurrently.
*/
-void nilfs_clear_folio_dirty(struct folio *folio)
+void nilfs_clear_folio_dirty(struct folio *folio, bool force)
{
struct buffer_head *bh, *head;
@@ -424,24 +428,28 @@ void nilfs_clear_folio_dirty(struct folio *folio)
BIT(BH_Async_Write) | BIT(BH_NILFS_Volatile) |
BIT(BH_NILFS_Checked) | BIT(BH_NILFS_Redirected) |
BIT(BH_Delay));
- bool busy, invalidated = false;
+
+ if (!force) {
+ bool busy, invalidated = false;
recheck_buffers:
- busy = false;
- bh = head;
- do {
- if (atomic_read(&bh->b_count) | buffer_locked(bh)) {
- busy = true;
- break;
+ busy = false;
+ bh = head;
+ do {
+ if (atomic_read(&bh->b_count) |
+ buffer_locked(bh)) {
+ busy = true;
+ break;
+ }
+ } while (bh = bh->b_this_page, bh != head);
+
+ if (busy) {
+ if (invalidated)
+ return;
+ invalidate_bh_lrus();
+ invalidated = true;
+ goto recheck_buffers;
}
- } while (bh = bh->b_this_page, bh != head);
-
- if (busy) {
- if (invalidated)
- return;
- invalidate_bh_lrus();
- invalidated = true;
- goto recheck_buffers;
}
bh = head;
@@ -458,14 +466,14 @@ void nilfs_clear_folio_dirty(struct folio *folio)
__nilfs_clear_folio_dirty(folio);
}
-unsigned int nilfs_page_count_clean_buffers(struct page *page,
+unsigned int nilfs_page_count_clean_buffers(struct folio *folio,
unsigned int from, unsigned int to)
{
unsigned int block_start, block_end;
struct buffer_head *bh, *head;
unsigned int nc = 0;
- for (bh = head = page_buffers(page), block_start = 0;
+ for (bh = head = folio_buffers(folio), block_start = 0;
bh != head || !block_start;
block_start = block_end, bh = bh->b_this_page) {
block_end = block_start + bh->b_size;
diff --git a/fs/nilfs2/page.h b/fs/nilfs2/page.h
index 64521a03a19e..c3ba3468af5c 100644
--- a/fs/nilfs2/page.h
+++ b/fs/nilfs2/page.h
@@ -41,10 +41,10 @@ void nilfs_folio_bug(struct folio *);
int nilfs_copy_dirty_pages(struct address_space *, struct address_space *);
void nilfs_copy_back_pages(struct address_space *, struct address_space *);
-void nilfs_clear_folio_dirty(struct folio *folio);
-void nilfs_clear_dirty_pages(struct address_space *mapping);
-unsigned int nilfs_page_count_clean_buffers(struct page *, unsigned int,
- unsigned int);
+void nilfs_clear_folio_dirty(struct folio *folio, bool force);
+void nilfs_clear_dirty_pages(struct address_space *mapping, bool force);
+unsigned int nilfs_page_count_clean_buffers(struct folio *folio,
+ unsigned int from, unsigned int to);
unsigned long nilfs_find_uncommitted_extent(struct inode *inode,
sector_t start_blk,
sector_t *blkoff);
diff --git a/fs/nilfs2/recovery.c b/fs/nilfs2/recovery.c
index 21d81097a89f..e6e044f8f7b8 100644
--- a/fs/nilfs2/recovery.c
+++ b/fs/nilfs2/recovery.c
@@ -757,18 +757,13 @@ static void nilfs_abort_roll_forward(struct the_nilfs *nilfs)
* @sb: super block instance
* @ri: pointer to a nilfs_recovery_info struct to store search results.
*
- * Return Value: On success, 0 is returned. On error, one of the following
- * negative error code is returned.
- *
- * %-EINVAL - Inconsistent filesystem state.
- *
- * %-EIO - I/O error
- *
- * %-ENOSPC - No space left on device (only in a panic state).
- *
- * %-ERESTARTSYS - Interrupted.
- *
- * %-ENOMEM - Insufficient memory available.
+ * Return: 0 on success, or one of the following negative error codes on
+ * failure:
+ * * %-EINTR - Interrupted.
+ * * %-EINVAL - Inconsistent filesystem state.
+ * * %-EIO - I/O error.
+ * * %-ENOMEM - Insufficient memory available.
+ * * %-ENOSPC - No space left on device (only in a panic state).
*/
int nilfs_salvage_orphan_logs(struct the_nilfs *nilfs,
struct super_block *sb,
@@ -833,14 +828,11 @@ int nilfs_salvage_orphan_logs(struct the_nilfs *nilfs,
* segment pointed by the superblock. It sets up struct the_nilfs through
* this search. It fills nilfs_recovery_info (ri) required for recovery.
*
- * Return Value: On success, 0 is returned. On error, one of the following
- * negative error code is returned.
- *
- * %-EINVAL - No valid segment found
- *
- * %-EIO - I/O error
- *
- * %-ENOMEM - Insufficient memory available.
+ * Return: 0 on success, or one of the following negative error codes on
+ * failure:
+ * * %-EINVAL - No valid segment found.
+ * * %-EIO - I/O error.
+ * * %-ENOMEM - Insufficient memory available.
*/
int nilfs_search_super_root(struct the_nilfs *nilfs,
struct nilfs_recovery_info *ri)
diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c
index dc431b4c34c9..8b62234d5dd5 100644
--- a/fs/nilfs2/segbuf.c
+++ b/fs/nilfs2/segbuf.c
@@ -403,12 +403,7 @@ static int nilfs_segbuf_submit_bh(struct nilfs_segment_buffer *segbuf,
* @segbuf: buffer storing a log to be written
* @nilfs: nilfs object
*
- * Return Value: On Success, 0 is returned. On Error, one of the following
- * negative error code is returned.
- *
- * %-EIO - I/O error
- *
- * %-ENOMEM - Insufficient memory available.
+ * Return: Always 0.
*/
static int nilfs_segbuf_write(struct nilfs_segment_buffer *segbuf,
struct the_nilfs *nilfs)
@@ -449,10 +444,7 @@ static int nilfs_segbuf_write(struct nilfs_segment_buffer *segbuf,
* nilfs_segbuf_wait - wait for completion of requested BIOs
* @segbuf: segment buffer
*
- * Return Value: On Success, 0 is returned. On Error, one of the following
- * negative error code is returned.
- *
- * %-EIO - I/O error
+ * Return: 0 on success, or %-EIO if I/O error is detected.
*/
static int nilfs_segbuf_wait(struct nilfs_segment_buffer *segbuf)
{
diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c
index 063400f8a73c..9da544759f10 100644
--- a/fs/nilfs2/segment.c
+++ b/fs/nilfs2/segment.c
@@ -191,12 +191,10 @@ static int nilfs_prepare_segment_lock(struct super_block *sb,
* When @vacancy_check flag is set, this function will check the amount of
* free space, and will wait for the GC to reclaim disk space if low capacity.
*
- * Return Value: On success, 0 is returned. On error, one of the following
- * negative error code is returned.
- *
- * %-ENOMEM - Insufficient memory available.
- *
- * %-ENOSPC - No space left on device
+ * Return: 0 on success, or one of the following negative error codes on
+ * failure:
+ * * %-ENOMEM - Insufficient memory available.
+ * * %-ENOSPC - No space left on device (if checking free space).
*/
int nilfs_transaction_begin(struct super_block *sb,
struct nilfs_transaction_info *ti,
@@ -2266,7 +2264,7 @@ static int nilfs_segctor_sync(struct nilfs_sc_info *sci)
wake_up(&sci->sc_wait_daemon);
for (;;) {
- set_current_state(TASK_INTERRUPTIBLE);
+ set_current_state(TASK_KILLABLE);
/*
* Synchronize only while the log writer thread is alive.
@@ -2280,11 +2278,11 @@ static int nilfs_segctor_sync(struct nilfs_sc_info *sci)
err = wait_req.err;
break;
}
- if (!signal_pending(current)) {
+ if (!fatal_signal_pending(current)) {
schedule();
continue;
}
- err = -ERESTARTSYS;
+ err = -EINTR;
break;
}
finish_wait(&sci->sc_wait_request, &wait_req.wq);
@@ -2316,18 +2314,13 @@ static void nilfs_segctor_wakeup(struct nilfs_sc_info *sci, int err, bool force)
* nilfs_construct_segment - construct a logical segment
* @sb: super block
*
- * Return Value: On success, 0 is returned. On errors, one of the following
- * negative error code is returned.
- *
- * %-EROFS - Read only filesystem.
- *
- * %-EIO - I/O error
- *
- * %-ENOSPC - No space left on device (only in a panic state).
- *
- * %-ERESTARTSYS - Interrupted.
- *
- * %-ENOMEM - Insufficient memory available.
+ * Return: 0 on success, or one of the following negative error codes on
+ * failure:
+ * * %-EINTR - Interrupted.
+ * * %-EIO - I/O error (including metadata corruption).
+ * * %-ENOMEM - Insufficient memory available.
+ * * %-ENOSPC - No space left on device (only in a panic state).
+ * * %-EROFS - Read only filesystem.
*/
int nilfs_construct_segment(struct super_block *sb)
{
@@ -2351,18 +2344,13 @@ int nilfs_construct_segment(struct super_block *sb)
* @start: start byte offset
* @end: end byte offset (inclusive)
*
- * Return Value: On success, 0 is returned. On errors, one of the following
- * negative error code is returned.
- *
- * %-EROFS - Read only filesystem.
- *
- * %-EIO - I/O error
- *
- * %-ENOSPC - No space left on device (only in a panic state).
- *
- * %-ERESTARTSYS - Interrupted.
- *
- * %-ENOMEM - Insufficient memory available.
+ * Return: 0 on success, or one of the following negative error codes on
+ * failure:
+ * * %-EINTR - Interrupted.
+ * * %-EIO - I/O error (including metadata corruption).
+ * * %-ENOMEM - Insufficient memory available.
+ * * %-ENOSPC - No space left on device (only in a panic state).
+ * * %-EROFS - Read only filesystem.
*/
int nilfs_construct_dsync_segment(struct super_block *sb, struct inode *inode,
loff_t start, loff_t end)
diff --git a/fs/nilfs2/the_nilfs.c b/fs/nilfs2/the_nilfs.c
index 575a543ee736..a0f14c1b3ef8 100644
--- a/fs/nilfs2/the_nilfs.c
+++ b/fs/nilfs2/the_nilfs.c
@@ -49,8 +49,8 @@ void nilfs_set_last_segment(struct the_nilfs *nilfs,
* alloc_nilfs - allocate a nilfs object
* @sb: super block instance
*
- * Return Value: On success, pointer to the_nilfs is returned.
- * On error, NULL is returned.
+ * Return: a pointer to the allocated nilfs object on success, or NULL on
+ * failure.
*/
struct the_nilfs *alloc_nilfs(struct super_block *sb)
{
@@ -200,8 +200,7 @@ static int nilfs_store_log_cursor(struct the_nilfs *nilfs,
* exponent information written in @sbp and stores it in @blocksize,
* or aborts with an error message if it's too large.
*
- * Return Value: On success, 0 is returned. If the block size is too
- * large, -EINVAL is returned.
+ * Return: 0 on success, or %-EINVAL if the block size is too large.
*/
static int nilfs_get_blocksize(struct super_block *sb,
struct nilfs_super_block *sbp, int *blocksize)
@@ -544,7 +543,7 @@ static int nilfs_valid_sb(struct nilfs_super_block *sbp)
* area, or if the parameters themselves are not normal, it is
* determined to be invalid.
*
- * Return Value: true if invalid, false if valid.
+ * Return: true if invalid, false if valid.
*/
static bool nilfs_sb2_bad_offset(struct nilfs_super_block *sbp, u64 offset)
{
@@ -690,8 +689,7 @@ static int nilfs_load_super_block(struct the_nilfs *nilfs,
* reading the super block, getting disk layout information, initializing
* shared fields in the_nilfs).
*
- * Return Value: On success, 0 is returned. On error, a negative error
- * code is returned.
+ * Return: 0 on success, or a negative error code on failure.
*/
int init_nilfs(struct the_nilfs *nilfs, struct super_block *sb)
{
diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
index 993375f0db67..ff0af29745a8 100644
--- a/fs/notify/inotify/inotify_fsnotify.c
+++ b/fs/notify/inotify/inotify_fsnotify.c
@@ -36,7 +36,7 @@ static bool event_compare(struct fsnotify_event *old_fsn,
old = INOTIFY_E(old_fsn);
new = INOTIFY_E(new_fsn);
- if (old->mask & FS_IN_IGNORED)
+ if (old->mask & (FS_IN_IGNORED | FS_MOVE))
return false;
if ((old->mask == new->mask) &&
(old->wd == new->wd) &&
diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c
index ee69f3447597..f929cef0f510 100644
--- a/fs/ntfs3/xattr.c
+++ b/fs/ntfs3/xattr.c
@@ -146,29 +146,32 @@ static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea,
for (off = 0; off < size; off += ea_size) {
const struct EA_FULL *ef = Add2Ptr(ea_p, off);
u32 bytes = size - off;
- size_t need;
-
- /* Check if we can use field ea->size. */
- if (bytes < sizeof(ef->size))
- goto out1;
+ u32 packed_size;
/* Check if we can use fields ef->name_len and ef->elength. */
if (bytes < offsetof(struct EA_FULL, name))
goto out1;
- /* Size needed to hold this record's name and value. */
- need = struct_size(ef, name,
- 1 + ef->name_len + le16_to_cpu(ef->elength));
+ /*
+ * The name and the value have to fit in the entry whether or
+ * not ef->size is set, because ntfs_get_ea() copies elength
+ * bytes out of this buffer and checks them only against the
+ * caller supplied buffer size.
+ */
+ packed_size =
+ ALIGN(struct_size(ef, name,
+ 1 + ef->name_len +
+ le16_to_cpu(ef->elength)),
+ 4);
if (ef->size) {
ea_size = le32_to_cpu(ef->size);
- /* ef->size must fit the list and cover the record. */
- if (ea_size > bytes || ea_size < need)
+ if (ea_size > bytes || packed_size > ea_size)
goto out1;
continue;
}
- ea_size = ALIGN(need, 4);
+ ea_size = packed_size;
if (ea_size > bytes)
goto out1;
}
diff --git a/fs/ocfs2/alloc.c b/fs/ocfs2/alloc.c
index eb6adb785e45..f33aa48a3bc5 100644
--- a/fs/ocfs2/alloc.c
+++ b/fs/ocfs2/alloc.c
@@ -1796,6 +1796,7 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci,
int i, ret = 0;
u32 range;
u64 blkno;
+ u32 prev_depth = OCFS2_MAX_PATH_DEPTH;
struct buffer_head *bh = NULL;
struct ocfs2_extent_block *eb;
struct ocfs2_extent_list *el;
@@ -1803,22 +1804,25 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci,
el = root_el;
while (el->l_tree_depth) {
- if (unlikely(le16_to_cpu(el->l_tree_depth) >= OCFS2_MAX_PATH_DEPTH)) {
+ if (unlikely(le16_to_cpu(el->l_tree_depth) >= prev_depth)) {
ocfs2_error(ocfs2_metadata_cache_get_super(ci),
- "Owner %llu has invalid tree depth %u in extent list\n",
+ "Owner %llu has invalid tree depth %u in extent list (max %u)\n",
(unsigned long long)ocfs2_metadata_cache_owner(ci),
- le16_to_cpu(el->l_tree_depth));
+ le16_to_cpu(el->l_tree_depth), prev_depth - 1);
ret = -EROFS;
goto out;
}
- if (le16_to_cpu(el->l_next_free_rec) == 0) {
+ prev_depth = le16_to_cpu(el->l_tree_depth);
+
+ if (!el->l_next_free_rec || !el->l_count) {
ocfs2_error(ocfs2_metadata_cache_get_super(ci),
- "Owner %llu has empty extent list at depth %u\n",
+ "Owner %llu has empty extent list at depth %u\n"
+ "(next free=%u count=%u)\n",
(unsigned long long)ocfs2_metadata_cache_owner(ci),
- le16_to_cpu(el->l_tree_depth));
+ le16_to_cpu(el->l_tree_depth),
+ le16_to_cpu(el->l_next_free_rec), le16_to_cpu(el->l_count));
ret = -EROFS;
goto out;
-
}
for(i = 0; i < le16_to_cpu(el->l_next_free_rec) - 1; i++) {
diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c
index dd4fb83d9cd8..a53b2c47f55d 100644
--- a/fs/ocfs2/cluster/tcp.c
+++ b/fs/ocfs2/cluster/tcp.c
@@ -391,6 +391,9 @@ static void sc_kref_release(struct kref *kref)
sclog(sc, "releasing\n");
+ /* Unlink before freeing anything the debugfs readers dereference. */
+ o2net_debug_del_sc(sc);
+
if (sc->sc_sock) {
sock_release(sc->sc_sock);
sc->sc_sock = NULL;
@@ -400,8 +403,6 @@ static void sc_kref_release(struct kref *kref)
o2nm_node_put(sc->sc_node);
sc->sc_node = NULL;
- o2net_debug_del_sc(sc);
-
if (sc->sc_page)
__free_page(sc->sc_page);
kfree(sc);
diff --git a/fs/ocfs2/dcache.h b/fs/ocfs2/dcache.h
index 7f246c5692d8..046ab9a80769 100644
--- a/fs/ocfs2/dcache.h
+++ b/fs/ocfs2/dcache.h
@@ -17,9 +17,8 @@ struct ocfs2_dentry_lock {
u64 dl_parent_blkno;
/*
- * The ocfs2_dentry_lock keeps an inode reference until
- * dl_lockres has been destroyed. This is usually done in
- * ->d_iput() anyway, so there should be minimal impact.
+ * The inode reference is dropped before dl_lockres is unlocked.
+ * Callbacks must not use it to find the mount context during teardown.
*/
struct inode *dl_inode;
struct ocfs2_lock_res dl_lockres;
diff --git a/fs/ocfs2/dlm/dlmmaster.c b/fs/ocfs2/dlm/dlmmaster.c
index d84e816c70c5..c9c5e5bd7d88 100644
--- a/fs/ocfs2/dlm/dlmmaster.c
+++ b/fs/ocfs2/dlm/dlmmaster.c
@@ -2537,6 +2537,52 @@ static int dlm_is_lockres_migratable(struct dlm_ctxt *dlm,
*/
+/*
+ * Drop the sender's references on a migration MLE after the attempt has
+ * failed. The finishing assert from the target may already have unlinked
+ * the MLE and consumed its base reference; decide under the locks the
+ * assert handler holds, and only drop the base reference while the MLE is
+ * still hashed. Returns true when the assert got there first: the target
+ * has become the master, so the caller must complete the migration.
+ */
+static bool dlm_put_migration_mle(struct dlm_ctxt *dlm,
+ struct dlm_master_list_entry *mle)
+{
+ bool asserted;
+
+ spin_lock(&dlm->spinlock);
+ spin_lock(&dlm->master_lock);
+ __dlm_mle_detach_hb_events(dlm, mle);
+ asserted = hlist_unhashed(&mle->master_hash_node);
+ if (!asserted) {
+ __dlm_unlink_mle(dlm, mle);
+ __dlm_put_mle(mle);
+ }
+ spin_unlock(&dlm->master_lock);
+ spin_unlock(&dlm->spinlock);
+
+ dlm_put_mle_inuse(mle);
+ return asserted;
+}
+
+/*
+ * The finishing assert got there first: it switched the owner to the
+ * target and cleared DLM_LOCK_RES_MIGRATING. Drop the other nodes' locks
+ * as a completed migration does, unless recovery has taken the lockres
+ * over since, and let the usage be recalculated.
+ */
+static void dlm_migrate_lockres_asserted(struct dlm_ctxt *dlm,
+ struct dlm_lock_resource *res,
+ u8 target)
+{
+ spin_lock(&res->spinlock);
+ if (res->owner == target)
+ dlm_remove_nonlocal_locks(dlm, res);
+ spin_unlock(&res->spinlock);
+
+ dlm_lockres_calc_usage(dlm, res);
+}
+
static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
struct dlm_lock_resource *res, u8 target)
{
@@ -2621,9 +2667,10 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
if (ret < 0) {
if (mle_added) {
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ }
} else if (mle) {
kmem_cache_free(dlm_mle_cache, mle);
mle = NULL;
@@ -2652,9 +2699,11 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
mlog(0, "migration to node %u failed with %d\n",
target, ret);
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
@@ -2698,9 +2747,12 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
res->lockname.name, target);
ret = -EINVAL;
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res,
+ target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index ec44ed33afdf..96d8f39a653a 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -56,7 +56,6 @@ struct ocfs2_mask_waiter {
#endif
};
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_inode_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_file_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_qinfo_osb(struct ocfs2_lock_res *lockres);
@@ -256,7 +255,6 @@ static const struct ocfs2_lock_res_ops ocfs2_orphan_scan_lops = {
};
static const struct ocfs2_lock_res_ops ocfs2_dentry_lops = {
- .get_osb = ocfs2_get_dentry_osb,
.post_unlock = ocfs2_dentry_post_unlock,
.downconvert_worker = ocfs2_dentry_convert_worker,
.flags = 0,
@@ -307,7 +305,7 @@ static inline struct ocfs2_dentry_lock *ocfs2_lock_res_dl(struct ocfs2_lock_res
{
BUG_ON(lockres->l_type != OCFS2_LOCK_TYPE_DENTRY);
- return (struct ocfs2_dentry_lock *)lockres->l_priv;
+ return container_of(lockres, struct ocfs2_dentry_lock, dl_lockres);
}
static inline struct ocfs2_mem_dqinfo *ocfs2_lock_res_qinfo(struct ocfs2_lock_res *lockres)
@@ -608,13 +606,6 @@ static __u64 ocfs2_get_dentry_lock_ino(struct ocfs2_lock_res *lockres)
return be64_to_cpu(inode_blkno_be);
}
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres)
-{
- struct ocfs2_dentry_lock *dl = lockres->l_priv;
-
- return OCFS2_SB(dl->dl_inode->i_sb);
-}
-
void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 parent, struct inode *inode)
{
@@ -622,6 +613,7 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 inode_blkno = OCFS2_I(inode)->ip_blkno;
__be64 inode_blkno_be = cpu_to_be64(inode_blkno);
struct ocfs2_lock_res *lockres = &dl->dl_lockres;
+ struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
ocfs2_lock_res_init_once(lockres);
@@ -650,9 +642,9 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
memcpy(&lockres->l_name[OCFS2_DENTRY_LOCK_INO_START], &inode_blkno_be,
sizeof(__be64));
- ocfs2_lock_res_init_common(OCFS2_SB(inode->i_sb), lockres,
+ ocfs2_lock_res_init_common(osb, lockres,
OCFS2_LOCK_TYPE_DENTRY, &ocfs2_dentry_lops,
- dl);
+ osb);
}
static void ocfs2_super_lock_res_init(struct ocfs2_lock_res *res,
@@ -3336,6 +3328,14 @@ static void ocfs2_dlm_shutdown_debug(struct ocfs2_super *osb)
{
struct ocfs2_dlm_debug *dlm_debug = osb->osb_dlm_debug;
+ /*
+ * Drain the readers before the cluster stack goes away: the
+ * locking_state reader resolves ocfs2_dlm_lvb() through the
+ * stack. Every caller removes the parent directory later.
+ */
+ debugfs_lookup_and_remove("locking_state", osb->osb_debug_root);
+ debugfs_lookup_and_remove("locking_filter", osb->osb_debug_root);
+
if (dlm_debug)
ocfs2_put_dlm_debug(dlm_debug);
}
@@ -3350,8 +3350,6 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
goto local;
}
- ocfs2_dlm_init_debug(osb);
-
/* launch downconvert thread */
osb->dc_task = kthread_run(ocfs2_downconvert_thread, osb, "ocfs2dc-%s",
osb->uuid_str);
@@ -3391,9 +3389,10 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
ocfs2_orphan_scan_lock_res_init(&osb->osb_orphan_scan.os_lockres, osb);
osb->cconn = conn;
+ if (conn)
+ ocfs2_dlm_init_debug(osb);
bail:
if (status < 0) {
- ocfs2_dlm_shutdown_debug(osb);
if (osb->dc_task)
kthread_stop(osb->dc_task);
}
@@ -3423,10 +3422,11 @@ void ocfs2_dlm_shutdown(struct ocfs2_super *osb,
ocfs2_lock_res_free(&osb->osb_orphan_scan.os_lockres);
if (osb->cconn) {
+ /* The disconnect below may release the selected stack. */
+ ocfs2_dlm_shutdown_debug(osb);
+
ocfs2_cluster_disconnect(osb->cconn, hangup_pending);
osb->cconn = NULL;
-
- ocfs2_dlm_shutdown_debug(osb);
}
}
diff --git a/fs/ocfs2/filecheck.c b/fs/ocfs2/filecheck.c
index c04b55bfaa6d..40b59f00ed57 100644
--- a/fs/ocfs2/filecheck.c
+++ b/fs/ocfs2/filecheck.c
@@ -189,6 +189,8 @@ int ocfs2_filecheck_create_sysfs(struct ocfs2_super *osb)
/* Mount failure still calls ocfs2_filecheck_remove_sysfs(). */
entry->fs_fcheck = NULL;
kobject_put(&entry->fs_kobj);
+ /* Keep ocfs2_super alive until the kobject is released. */
+ wait_for_completion(&entry->fs_kobj_unregister);
kfree(fcheck);
return ret;
}
diff --git a/fs/ocfs2/inode.c b/fs/ocfs2/inode.c
index ce78ef998217..51e487ea432e 100644
--- a/fs/ocfs2/inode.c
+++ b/fs/ocfs2/inode.c
@@ -575,14 +575,18 @@ static int ocfs2_read_locked_inode(struct inode *inode,
fe = (struct ocfs2_dinode *) bh->b_data;
/*
- * This is a code bug. Right now the caller needs to
- * understand whether it is asking for a system file inode or
- * not so the proper lock names can be built.
+ * The caller must know whether it is asking for a system file inode
+ * or not so the proper lock names can be built. Since i_flags comes
+ * from disk, a mismatch is filesystem corruption instead of a code
+ * bug, so handle it with ocfs2_error() rather than BUG_ON().
*/
- mlog_bug_on_msg(!!(fe->i_flags & cpu_to_le32(OCFS2_SYSTEM_FL)) !=
- !!(args->fi_flags & OCFS2_FI_FLAG_SYSFILE),
- "Inode %llu: system file state is ambiguous\n",
- (unsigned long long)args->fi_blkno);
+ if (!!(fe->i_flags & cpu_to_le32(OCFS2_SYSTEM_FL)) !=
+ !!(args->fi_flags & OCFS2_FI_FLAG_SYSFILE)) {
+ status = ocfs2_error(osb->sb,
+ "Inode %llu: system file state is ambiguous\n",
+ (unsigned long long)args->fi_blkno);
+ goto bail;
+ }
if (S_ISCHR(le16_to_cpu(fe->i_mode)) ||
S_ISBLK(le16_to_cpu(fe->i_mode)))
@@ -1599,8 +1603,23 @@ int ocfs2_validate_inode_block(struct super_block *sb,
}
}
- if (le16_to_cpu(di->i_suballoc_slot) != (u16)OCFS2_INVALID_SLOT &&
- (u32)le16_to_cpu(di->i_suballoc_slot) > OCFS2_SB(sb)->max_slots - 1) {
+ /*
+ * Only system inodes created by mkfs.ocfs2 are allocated from the
+ * global allocator and thus legitimately carry OCFS2_INVALID_SLOT.
+ * Regular inodes are always allocated from a per-slot suballocator.
+ * If a regular inode with OCFS2_INVALID_SLOT was accepted here,
+ * deleting it would pass the slot to get_local_system_inode() via
+ * ocfs2_remove_inode() and trigger BUG_ON(slot == OCFS2_INVALID_SLOT).
+ */
+ if (le16_to_cpu(di->i_suballoc_slot) == (u16)OCFS2_INVALID_SLOT) {
+ if (!(le32_to_cpu(di->i_flags) & OCFS2_SYSTEM_FL)) {
+ rc = ocfs2_error(sb,
+ "Invalid dinode %llu: suballoc slot %u for non-system inode\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(di->i_suballoc_slot));
+ goto bail;
+ }
+ } else if ((u32)le16_to_cpu(di->i_suballoc_slot) > OCFS2_SB(sb)->max_slots - 1) {
rc = ocfs2_error(sb, "Invalid dinode %llu: suballoc slot %u\n",
(unsigned long long)bh->b_blocknr,
le16_to_cpu(di->i_suballoc_slot));
diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c
index 66427185a5d0..e8f6e3a554a3 100644
--- a/fs/ocfs2/journal.c
+++ b/fs/ocfs2/journal.c
@@ -230,13 +230,22 @@ void ocfs2_recovery_disable_quota(struct ocfs2_super *osb)
ocfs2_recovery_disable(osb, OCFS2_REC_QUOTA_WANT_DISABLE);
}
+/* Stop producers and drain completion work without freeing recovery state. */
+void ocfs2_recovery_stop(struct ocfs2_super *osb)
+{
+ ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+}
+
void ocfs2_recovery_exit(struct ocfs2_super *osb)
{
struct ocfs2_recovery_map *rm;
- /* disable any new recovery threads and wait for any currently
- * running ones to exit. Do this before setting the vol_state. */
- ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+ /*
+ * Disable producers and drain work before freeing recovery state.
+ * Do this before setting vol_state to VOLUME_DISMOUNTED: completion
+ * work may still be waiting in __ocfs2_wait_on_mount().
+ */
+ ocfs2_recovery_stop(osb);
/*
* Now that recovery is shut down, and the osb is about to be
diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h
index 19fc920d26b1..4b52bf7b0938 100644
--- a/fs/ocfs2/journal.h
+++ b/fs/ocfs2/journal.h
@@ -147,6 +147,7 @@ void ocfs2_complete_recovery(struct work_struct *work);
void ocfs2_wait_for_recovery(struct ocfs2_super *osb);
int ocfs2_recovery_init(struct ocfs2_super *osb);
+void ocfs2_recovery_stop(struct ocfs2_super *osb);
void ocfs2_recovery_exit(struct ocfs2_super *osb);
void ocfs2_recovery_disable_quota(struct ocfs2_super *osb);
diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c
index 3535881de652..2a11202b5220 100644
--- a/fs/ocfs2/super.c
+++ b/fs/ocfs2/super.c
@@ -1884,6 +1884,9 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err)
/* Wait for worker to be done with the work structure in osb */
cancel_work_sync(&osb->dquot_drop_work);
+ /* Stop recovery producers before releasing their local allocators. */
+ ocfs2_recovery_stop(osb);
+
ocfs2_shutdown_local_alloc(osb);
ocfs2_truncate_log_shutdown(osb);
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index f73bfcfcde54..037ae2200c83 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -256,6 +256,7 @@ static int ocfs2_xattr_tree_list_index_block(struct inode *inode,
size_t buffer_size);
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt);
@@ -3228,7 +3229,7 @@ static int ocfs2_xattr_block_set(struct inode *inode,
else if ((ret != -ENOSPC) || ctxt->set_abort)
goto end;
else {
- ret = ocfs2_xattr_create_index_block(inode, xs, ctxt);
+ ret = ocfs2_xattr_create_index_block(inode, xi, xs, ctxt);
if (ret)
goto end;
}
@@ -4510,6 +4511,21 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
for (i = 0; i < blks; i++)
memset(bucket_block(bucket, i), 0, blocksize);
+ /*
+ * An empty block has no names or values to carry over, and the
+ * xh_entries[count - 1] below would be xh_entries[-1]. Leave the
+ * bucket empty, but carry over the hash ocfs2_xa_remove_entry() left
+ * in the block's xh_entries[0] so that the bucket still holds the key
+ * it is looked up by.
+ */
+ if (!count) {
+ xh->xh_num_buckets = cpu_to_le16(1);
+ xh->xh_free_start = cpu_to_le16(OCFS2_XATTR_BUCKET_SIZE);
+ xh->xh_entries[0].xe_name_hash =
+ xb_xh->xh_entries[0].xe_name_hash;
+ return;
+ }
+
/*
* Since the xe_name_offset is based on ocfs2_xattr_header,
* there is a offset change corresponding to the change of
@@ -4556,13 +4572,10 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
* real place of the xattr.
*/
static void ocfs2_xattr_update_xattr_search(struct inode *inode,
- struct ocfs2_xattr_search *xs,
- struct buffer_head *old_bh)
+ struct ocfs2_xattr_info *xi,
+ struct ocfs2_xattr_search *xs)
{
- char *buf = old_bh->b_data;
- struct ocfs2_xattr_block *old_xb = (struct ocfs2_xattr_block *)buf;
- struct ocfs2_xattr_header *old_xh = &old_xb->xb_attrs.xb_header;
- int i;
+ int i, count;
xs->header = bucket_xh(xs->bucket);
xs->base = bucket_block(xs->bucket, 0);
@@ -4571,11 +4584,47 @@ static void ocfs2_xattr_update_xattr_search(struct inode *inode,
if (xs->not_found)
return;
- i = xs->here - old_xh->xh_entries;
- xs->here = &xs->header->xh_entries[i];
+ /*
+ * A block holds its entries in insertion order while a bucket holds
+ * them by name hash, so ocfs2_cp_xattr_block_to_bucket() sorted the
+ * copies and the index this entry had in the block no longer says
+ * where it is. It copied every entry and every name unchanged, and
+ * (type, name) pairs are unique within a block, so find it again by
+ * name.
+ *
+ * Search for the name being set, not for the name xs->here carries.
+ * A failed set may have removed that entry, and ocfs2_xa_remove_entry()
+ * then memmoves its successor into the slot. Comparing every entry
+ * rather than narrowing on the hash also finds an entry stored under
+ * the legacy signed hash, which ocfs2_find_xe_in_bucket() would skip.
+ */
+ count = le16_to_cpu(xs->header->xh_count);
+ for (i = 0; i < count; i++) {
+ struct ocfs2_xattr_entry *xe = &xs->header->xh_entries[i];
+
+ if (ocfs2_xattr_get_type(xe) != xi->xi_name_index)
+ continue;
+ if (xe->xe_name_len != xi->xi_name_len)
+ continue;
+ if (!memcmp(xs->base + le16_to_cpu(xe->xe_name_offset),
+ xi->xi_name, xi->xi_name_len)) {
+ xs->here = xe;
+ return;
+ }
+ }
+
+ /*
+ * The entry did not survive the failed set, so this has become an
+ * insert. Point xs->here back inside the bucket: it still refers to
+ * the block being converted, which is about to be rewritten as a tree
+ * root.
+ */
+ xs->not_found = -ENODATA;
+ xs->here = xs->header->xh_entries;
}
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
@@ -4642,7 +4691,7 @@ static int ocfs2_xattr_create_index_block(struct inode *inode,
ocfs2_cp_xattr_block_to_bucket(inode, xb_bh, xs->bucket);
ocfs2_xattr_bucket_journal_dirty(handle, xs->bucket);
- ocfs2_xattr_update_xattr_search(inode, xs, xb_bh);
+ ocfs2_xattr_update_xattr_search(inode, xi, xs);
/* Change from ocfs2_xattr_header to ocfs2_xattr_tree_root */
memset(&xb->xb_attrs, 0, inode->i_sb->s_blocksize -
@@ -4995,7 +5044,15 @@ static int ocfs2_divide_xattr_bucket(struct inode *inode,
start = ocfs2_xattr_find_divide_pos(xh);
if (start == count) {
- xe = &xh->xh_entries[start-1];
+ /*
+ * start == count means every entry shares one hash, so the
+ * first of them carries it too. An empty bucket has no
+ * entries[start - 1], but its entries[0] still carries the
+ * bucket key: either the hash ocfs2_xa_remove_entry()
+ * deliberately leaves behind, or the sentinel a previous
+ * divide wrote.
+ */
+ xe = &xh->xh_entries[0];
/*
* initialized a new empty bucket here.
@@ -5825,11 +5882,20 @@ static int ocfs2_check_xattr_bucket_collision(struct ocfs2_xattr_bucket *bucket,
u32 name_hash)
{
struct ocfs2_xattr_header *xh = bucket_xh(bucket);
+ u16 count = le16_to_cpu(xh->xh_count);
+
+ /*
+ * An empty bucket keeps the hash of the entry that was removed in
+ * xh_entries[0] so that it can still be found, but it holds no
+ * entries to collide with.
+ */
+ if (!count)
+ return 0;
if (name_hash != le32_to_cpu(xh->xh_entries[0].xe_name_hash))
return 0;
- if (xh->xh_entries[le16_to_cpu(xh->xh_count) - 1].xe_name_hash ==
+ if (xh->xh_entries[count - 1].xe_name_hash ==
xh->xh_entries[0].xe_name_hash) {
mlog(ML_ERROR, "Too much hash collision in xattr bucket %llu, "
"hash = %u\n",
@@ -5850,13 +5916,18 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
- int ret;
+ int ret, found = 0;
+ u16 xe_index;
+ u32 name_hash;
struct ocfs2_xa_loc loc;
trace_ocfs2_xattr_set_entry_bucket(xi->xi_name);
ocfs2_init_xattr_bucket_xa_loc(&loc, xs->bucket,
xs->not_found ? NULL : xs->here);
+ /* Defragmenting below may move this entry, so remember its hash. */
+ name_hash = loc.xl_entry ? le32_to_cpu(loc.xl_entry->xe_name_hash) : 0;
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
@@ -5875,6 +5946,29 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
goto out;
}
+ /*
+ * ocfs2_defrag_xattr_bucket() sorts the entries by name hash and
+ * lib/sort.c is not stable, so entries sharing a hash may have traded
+ * slots. Look the entry up again before writing through it.
+ */
+ if (loc.xl_entry) {
+ ret = ocfs2_find_xe_in_bucket(inode, xs->bucket,
+ xi->xi_name_index, xi->xi_name,
+ name_hash, &xe_index, &found);
+ if (ret) {
+ mlog_errno(ret);
+ goto out;
+ }
+ if (!found) {
+ mlog(ML_ERROR, "xattr entry not found in bucket %llu after defrag\n",
+ (unsigned long long)bucket_blkno(xs->bucket));
+ ret = -EFSCORRUPTED;
+ goto out;
+ }
+ loc.xl_entry = &bucket_xh(xs->bucket)->xh_entries[xe_index];
+ xs->here = loc.xl_entry;
+ }
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 444aeeccb6da..576c89e440f7 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -720,9 +720,9 @@ static struct dentry *ovl_lower_fh_to_d(struct super_block *sb,
}
}
- /* Then lookup indexed upper/whiteout by origin fh */
+ /* Then lookup indexed upper/whiteout by origin */
if (ovl_indexdir(sb)) {
- index = ovl_get_index_fh(ofs, fh);
+ index = ovl_get_index_origin(ofs, origin.dentry);
err = PTR_ERR(index);
if (IS_ERR(index)) {
index = NULL;
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 8e8c416a06fd..9a44af6bf911 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -748,14 +748,21 @@ int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
return err;
}
-/* Lookup index by file handle for NFS export */
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh)
+/*
+ * Lookup index by decoded origin for NFS export.
+ *
+ * The name of an index is the hex of the file handle that copy up encoded.
+ * The bytes of the handle to decode cannot be used as the name, because a
+ * lower fs may decode many handles to the same inode. For example, it may
+ * ignore the fid type, extra fid bytes or a zero generation.
+ */
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin)
{
struct dentry *index;
struct qstr name;
int err;
- err = ovl_get_index_name_fh(fh, &name);
+ err = ovl_get_index_name(ofs, origin, &name);
if (err)
return ERR_PTR(err);
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 75eb9d379549..53d1f0e0c924 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -715,7 +715,7 @@ int ovl_verify_index(struct ovl_fs *ofs, struct dentry *index);
int ovl_get_index_name_fh(const struct ovl_fh *fh, struct qstr *name);
int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
struct qstr *name);
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh);
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin);
struct dentry *ovl_lookup_index(struct ovl_fs *ofs, struct dentry *upper,
struct dentry *origin, bool verify);
int ovl_path_next(int idx, struct dentry *dentry, struct path *path,
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 0f45320caf72..bc38ba6aa37e 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -2943,7 +2943,7 @@ static long do_pagemap_cmd(struct file *file, unsigned int cmd,
return do_pagemap_scan(mm, arg);
default:
- return -EINVAL;
+ return -ENOIOCTLCMD;
}
}
diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index f5ca2cd653e9..8ffc8b2ab6db 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -16,6 +16,7 @@
#include <linux/scatterlist.h>
#include <linux/mm.h>
#include <linux/mempool.h>
+#include <linux/refcount.h>
#include <linux/workqueue.h>
#include <linux/utsname.h>
#include <linux/sched/mm.h>
@@ -647,6 +648,7 @@ struct TCP_Server_Info {
spinlock_t srv_lock; /* protect anything here that is not protected */
__u64 conn_id; /* connection identifier (useful for debugging) */
int srv_count; /* reference counter */
+ refcount_t obj_refs; /* struct lifetime: srv_count users + cifsd */
/* 15 character server name + 0x20 16th byte indicating type = srv */
char server_RFC1001_name[RFC1001_NAME_LEN_WITH_NULL];
struct smb_version_operations *ops;
diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c
index 45c6eb11356b..c893b9fed8a0 100644
--- a/fs/smb/client/cifssmb.c
+++ b/fs/smb/client/cifssmb.c
@@ -2092,6 +2092,7 @@ CIFSSMBPosixLock(const unsigned int xid, struct cifs_tcon *tcon,
cifs_dbg(FYI, "Send error in Posix Lock = %d\n", rc);
} else if (pLockData) {
/* lock structure can be returned on get */
+ unsigned int rsp_len;
__u16 data_offset;
__u16 data_count;
rc = validate_t2((struct smb_t2_rsp *)pSMBr);
@@ -2102,10 +2103,16 @@ CIFSSMBPosixLock(const unsigned int xid, struct cifs_tcon *tcon,
}
data_offset = le16_to_cpu(pSMBr->t2.DataOffset);
data_count = le16_to_cpu(pSMBr->t2.DataCount);
+ rsp_len = smbCalcSize(&pSMBr->hdr) - 4;
if (data_count < sizeof(struct cifs_posix_lock)) {
rc = -EIO;
goto plk_err_exit;
}
+ if (data_offset > rsp_len ||
+ data_count > rsp_len - data_offset) {
+ rc = -EIO;
+ goto plk_err_exit;
+ }
parm_data = (struct cifs_posix_lock *)
((char *)&pSMBr->hdr.Protocol + data_offset);
if (parm_data->lock_type == cpu_to_le16(CIFS_UNLCK))
@@ -3998,8 +4005,9 @@ CIFSFindFirst(const unsigned int xid, struct cifs_tcon *tcon,
/* level 257 SMB_ */
TRANSACTION2_FFIRST_REQ *pSMB = NULL;
TRANSACTION2_FFIRST_RSP *pSMBr = NULL;
- T2_FFIRST_RSP_PARMS *parms;
+ unsigned int data_off, data_count;
struct nls_table *nls_codepage;
+ T2_FFIRST_RSP_PARMS *parms;
unsigned int lnoff;
__u16 params, byte_count;
int bytes_returned = 0;
@@ -4118,11 +4126,19 @@ CIFSFindFirst(const unsigned int xid, struct cifs_tcon *tcon,
return rc;
}
+ data_off = le16_to_cpu(pSMBr->t2.DataOffset);
+ data_count = le16_to_cpu(pSMBr->t2.DataCount);
+ if (data_off + data_count > (unsigned int)bytes_returned) {
+ cifs_dbg(VFS, "%s: data area offset %u count %u past response length %d\n",
+ __func__, data_off, data_count, bytes_returned);
+ cifs_buf_release(pSMB);
+ return -EINVAL;
+ }
+
psrch_inf->unicode = !!(pSMBr->hdr.Flags2 & SMBFLG2_UNICODE);
psrch_inf->ntwrk_buf_start = (char *)pSMBr;
psrch_inf->smallBuf = false;
- psrch_inf->srch_entries_start = (char *)&pSMBr->hdr.Protocol +
- le16_to_cpu(pSMBr->t2.DataOffset);
+ psrch_inf->srch_entries_start = (char *)&pSMBr->hdr.Protocol + data_off;
parms = (T2_FFIRST_RSP_PARMS *)((char *)&pSMBr->hdr.Protocol +
le16_to_cpu(pSMBr->t2.ParameterOffset));
@@ -4155,8 +4171,9 @@ int CIFSFindNext(const unsigned int xid, struct cifs_tcon *tcon,
{
TRANSACTION2_FNEXT_REQ *pSMB = NULL;
TRANSACTION2_FNEXT_RSP *pSMBr = NULL;
- T2_FNEXT_RSP_PARMS *parms;
+ unsigned int data_off, data_count;
unsigned int name_len;
+ T2_FNEXT_RSP_PARMS *parms;
unsigned int lnoff;
__u16 params, byte_count;
char *response_data;
@@ -4239,13 +4256,22 @@ int CIFSFindNext(const unsigned int xid, struct cifs_tcon *tcon,
cifs_buf_release(pSMB);
return rc;
}
+
+ data_off = le16_to_cpu(pSMBr->t2.DataOffset);
+ data_count = le16_to_cpu(pSMBr->t2.DataCount);
+ if (data_off + data_count > (unsigned int)bytes_returned) {
+ cifs_dbg(VFS, "%s: data area offset %u count %u past response length %d\n",
+ __func__, data_off, data_count, bytes_returned);
+ cifs_buf_release(pSMB);
+ return -EINVAL;
+ }
+
/* BB fixme add lock for file (srch_info) struct here */
psrch_inf->unicode = !!(pSMBr->hdr.Flags2 & SMBFLG2_UNICODE);
response_data = (char *)&pSMBr->hdr.Protocol +
le16_to_cpu(pSMBr->t2.ParameterOffset);
parms = (T2_FNEXT_RSP_PARMS *)response_data;
- response_data = (char *)&pSMBr->hdr.Protocol +
- le16_to_cpu(pSMBr->t2.DataOffset);
+ response_data = (char *)&pSMBr->hdr.Protocol + data_off;
if (psrch_inf->smallBuf)
cifs_small_buf_release(psrch_inf->ntwrk_buf_start);
@@ -5763,9 +5789,11 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon,
int rc = 0;
int bytes_returned;
int list_len;
+ u32 ea_list_len;
struct fealist *ea_response_data;
struct fea *temp_fea;
char *temp_ptr;
+ char *byte_area;
char *end_of_smb;
__u16 params, byte_count, data_offset;
unsigned int ea_name_len = ea_name ? strlen(ea_name) : 0;
@@ -5842,9 +5870,18 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon,
ea_response_data = (struct fealist *)
(((char *) &pSMBr->hdr.Protocol) + data_offset);
- list_len = le32_to_cpu(ea_response_data->list_len);
- cifs_dbg(FYI, "ea length %d\n", list_len);
- if (list_len <= 8) {
+ byte_area = pByteArea(&pSMBr->hdr);
+ end_of_smb = byte_area + get_bcc(&pSMBr->hdr);
+ if ((char *)ea_response_data < byte_area ||
+ (char *)ea_response_data >
+ end_of_smb - sizeof(ea_response_data->list_len)) {
+ rc = -EIO;
+ goto QAllEAsOut;
+ }
+
+ ea_list_len = le32_to_cpu(ea_response_data->list_len);
+ cifs_dbg(FYI, "%s: ea length: %u\n", __func__, ea_list_len);
+ if (ea_list_len <= 8) {
cifs_dbg(FYI, "empty EA list returned from server\n");
/* didn't find the named attribute */
if (ea_name)
@@ -5853,12 +5890,12 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon,
}
/* make sure list_len doesn't go past end of SMB */
- end_of_smb = (char *)pByteArea(&pSMBr->hdr) + get_bcc(&pSMBr->hdr);
- if ((char *)ea_response_data + list_len > end_of_smb) {
+ if (ea_list_len > end_of_smb - (char *)ea_response_data) {
cifs_dbg(FYI, "EA list appears to go beyond SMB\n");
rc = -EIO;
goto QAllEAsOut;
}
+ list_len = ea_list_len;
/* account for ea list len */
list_len -= 4;
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index eace4d1a3dce..82a70370a60f 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -977,6 +977,19 @@ cifs_enable_signing(struct TCP_Server_Info *server, bool mnt_sign_required)
return 0;
}
+static void cifs_tcp_server_put(struct TCP_Server_Info *server)
+{
+ if (!refcount_dec_and_test(&server->obj_refs))
+ return;
+
+ cifs_crypto_secmech_release(server);
+ kfree_sensitive(server->session_key.response);
+ put_net(cifs_net_ns(server));
+ kfree(server->leaf_fullpath);
+ kfree(server->hostname);
+ kfree_sensitive(server);
+}
+
static noinline_for_stack void
clean_demultiplex_info(struct TCP_Server_Info *server)
{
@@ -1062,14 +1075,11 @@ clean_demultiplex_info(struct TCP_Server_Info *server)
*/
}
- put_net(cifs_net_ns(server));
- kfree(server->leaf_fullpath);
- kfree(server->hostname);
- kfree(server);
-
length = atomic_dec_return(&tcpSesAllocCount);
if (length > 0)
mempool_resize(cifs_req_poolp, length + cifs_min_rcv);
+
+ cifs_tcp_server_put(server);
}
static int
@@ -1680,7 +1690,7 @@ cifs_put_tcp_session(struct TCP_Server_Info *server, int from_reconnect)
else
cancel_delayed_work_sync(&server->reconnect);
- /* For secondary channels, we pick up ref-count on the primary server */
+ /* For secondary channels, drop the reference on the primary server */
if (SERVER_IS_CHAN(server))
cifs_put_tcp_session(server->primary_server, from_reconnect);
@@ -1688,15 +1698,11 @@ cifs_put_tcp_session(struct TCP_Server_Info *server, int from_reconnect)
server->tcpStatus = CifsExiting;
spin_unlock(&server->srv_lock);
- cifs_crypto_secmech_release(server);
-
- kfree_sensitive(server->session_key.response);
- server->session_key.response = NULL;
- server->session_key.len = 0;
-
task = xchg(&server->tsk, NULL);
if (task)
send_sig(SIGKILL, task, 1);
+
+ cifs_tcp_server_put(server);
}
struct TCP_Server_Info *
@@ -1718,6 +1724,7 @@ cifs_get_tcp_session(struct smb3_fs_context *ctx,
rc = -ENOMEM;
goto out_err;
}
+ refcount_set(&tcp_ses->obj_refs, 1);
tcp_ses->hostname = kstrdup(ctx->server_hostname, GFP_KERNEL);
if (!tcp_ses->hostname) {
@@ -1858,6 +1865,8 @@ cifs_get_tcp_session(struct smb3_fs_context *ctx,
module_put(THIS_MODULE);
goto out_err_crypto_release;
}
+ /* Keep the server alive until the demultiplex thread has exited */
+ refcount_inc(&tcp_ses->obj_refs);
/* thread created, put it on the list */
spin_lock(&cifs_tcp_ses_lock);
list_add(&tcp_ses->tcp_ses_list, &cifs_tcp_ses_list);
diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c
index 9307a2ad9e44..5db59b429fc3 100644
--- a/fs/smb/client/inode.c
+++ b/fs/smb/client/inode.c
@@ -87,7 +87,9 @@ static void cifs_set_ops(struct inode *inode)
else
inode->i_data.a_ops = &cifs_addr_ops;
mapping_set_large_folios(inode->i_mapping);
- if (cifs_sb_master_tcon(cifs_sb)->ses->server->sign)
+ if (cifs_sb_master_tcon(cifs_sb)->ses->server->sign ||
+ (cifs_sb_master_tcon(cifs_sb)->ses->server->compression.enabled &&
+ (cifs_sb_master_tcon(cifs_sb)->share_flags & SMB2_SHAREFLAG_COMPRESS_DATA)))
mapping_set_stable_writes(inode->i_mapping);
break;
case S_IFDIR:
diff --git a/fs/smb/client/readdir.c b/fs/smb/client/readdir.c
index 9412c69e5aea..55b4171cec09 100644
--- a/fs/smb/client/readdir.c
+++ b/fs/smb/client/readdir.c
@@ -454,17 +454,20 @@ initiate_cifs_search(const unsigned int xid, struct file *file,
}
/* return length of unicode string in bytes */
-static int cifs_unicode_bytelen(const char *str)
+static size_t cifs_unicode_bytelen(const char *str, size_t maxlen)
{
- int len;
const __le16 *ustr = (const __le16 *)str;
+ size_t len;
- for (len = 0; len <= PATH_MAX; len++) {
+ maxlen = umin(PATH_MAX + 1, maxlen / sizeof(__le16));
+ for (len = 0; len < maxlen; len++) {
if (ustr[len] == 0)
- return len << 1;
+ return len * sizeof(__le16);
}
- cifs_dbg(FYI, "Unicode string longer than PATH_MAX found\n");
- return len << 1;
+ len *= sizeof(__le16);
+ cifs_dbg(FYI, "%s: Unicode string not terminated within %zu bytes\n",
+ __func__, len);
+ return len;
}
static char *nxt_dir_entry(char *old_entry, char *end_of_smb, int level)
@@ -530,13 +533,13 @@ static void cifs_fill_dirent_posix(struct cifs_dirent *de,
}
static void cifs_fill_dirent_unix(struct cifs_dirent *de,
- const FILE_UNIX_INFO *info, bool is_unicode)
+ const FILE_UNIX_INFO *info, bool is_unicode, size_t maxlen)
{
de->name = &info->FileName[0];
if (is_unicode)
- de->namelen = cifs_unicode_bytelen(de->name);
+ de->namelen = cifs_unicode_bytelen(de->name, maxlen);
else
- de->namelen = strnlen(de->name, PATH_MAX);
+ de->namelen = strnlen(de->name, min_t(size_t, maxlen, PATH_MAX));
de->resume_key = info->ResumeKey;
de->ino = le64_to_cpu(info->basic.UniqueId);
}
@@ -584,30 +587,49 @@ static void cifs_fill_dirent_std(struct cifs_dirent *de,
}
static int cifs_fill_dirent(struct cifs_dirent *de, const void *info,
- u16 level, bool is_unicode)
+ const char *end, u16 level, bool is_unicode)
{
+ size_t len = end > (const char *)info ? end - (const char *)info : 0;
+ size_t nul_len = 0;
+
memset(de, 0, sizeof(*de));
switch (level) {
case SMB_FIND_FILE_POSIX_INFO:
+ if (len < sizeof(struct smb2_posix_info))
+ goto too_short;
cifs_fill_dirent_posix(de, info);
break;
case SMB_FIND_FILE_UNIX:
- cifs_fill_dirent_unix(de, info, is_unicode);
+ if (len < offsetof(FILE_UNIX_INFO, FileName))
+ goto too_short;
+ cifs_fill_dirent_unix(de, info, is_unicode,
+ len - offsetof(FILE_UNIX_INFO, FileName));
+ nul_len = is_unicode ? sizeof(__le16) : sizeof(__u8);
break;
case SMB_FIND_FILE_DIRECTORY_INFO:
+ if (len < offsetof(FILE_DIRECTORY_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_dir(de, info);
break;
case SMB_FIND_FILE_FULL_DIRECTORY_INFO:
+ if (len < offsetof(FILE_FULL_DIRECTORY_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_full(de, info);
break;
case SMB_FIND_FILE_ID_FULL_DIR_INFO:
+ if (len < offsetof(SEARCH_ID_FULL_DIR_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_search(de, info);
break;
case SMB_FIND_FILE_BOTH_DIRECTORY_INFO:
+ if (len < offsetof(FILE_BOTH_DIRECTORY_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_both(de, info);
break;
case SMB_FIND_FILE_INFO_STANDARD:
+ if (len < offsetof(FIND_FILE_STANDARD_INFO, FileName))
+ goto too_short;
cifs_fill_dirent_std(de, info);
break;
default:
@@ -615,7 +637,20 @@ static int cifs_fill_dirent(struct cifs_dirent *de, const void *info,
return -EINVAL;
}
+ if (de->name &&
+ (end < de->name ||
+ (size_t)(end - de->name) < de->namelen + nul_len)) {
+ cifs_dbg(VFS, "%s: search entry name extends past end of SMB\n",
+ __func__);
+ return -EINVAL;
+ }
+
return 0;
+
+too_short:
+ cifs_dbg(VFS, "%s: search entry extends past end of SMB\n",
+ __func__);
+ return -EINVAL;
}
#define UNICODE_DOT cpu_to_le16(0x2e)
@@ -670,10 +705,14 @@ static int is_dir_changed(struct file *file)
static int cifs_save_resume_key(const char *current_entry,
struct cifsFileInfo *file_info)
{
+ struct TCP_Server_Info *server = tlink_tcon(file_info->tlink)->ses->server;
+ char *buf = file_info->srch_inf.ntwrk_buf_start;
+ const char *end = buf + server->ops->calc_smb_size(buf);
struct cifs_dirent de;
int rc;
- rc = cifs_fill_dirent(&de, current_entry, file_info->srch_inf.info_level,
+ rc = cifs_fill_dirent(&de, current_entry, end,
+ file_info->srch_inf.info_level,
file_info->srch_inf.unicode);
if (!rc) {
file_info->srch_inf.presume_name = de.name;
@@ -958,7 +997,8 @@ static int cifs_filldir(char *find_entry, struct file *file,
struct qstr name;
int rc = 0;
- rc = cifs_fill_dirent(&de, find_entry, file_info->srch_inf.info_level,
+ rc = cifs_fill_dirent(&de, find_entry, end_of_smb,
+ file_info->srch_inf.info_level,
file_info->srch_inf.unicode);
if (rc)
return rc;
diff --git a/fs/smb/client/smb2misc.c b/fs/smb/client/smb2misc.c
index 23320b794b17..7be338ad755b 100644
--- a/fs/smb/client/smb2misc.c
+++ b/fs/smb/client/smb2misc.c
@@ -193,6 +193,21 @@ smb2_check_message(char *buf, unsigned int len, struct TCP_Server_Info *server)
return 1;
}
+ /*
+ * A SESSION_SETUP response that asks for more processing carries a
+ * security blob and is parsed as a normal reply. Do not let the
+ * short-error path below accept a frame that cannot hold the fixed
+ * response fields.
+ */
+ if (command == SMB2_SESSION_SETUP_HE &&
+ shdr->Status == STATUS_MORE_PROCESSING_REQUIRED &&
+ len < sizeof(struct smb2_sess_setup_rsp)) {
+ cifs_server_dbg(VFS,
+ "%s: session setup response too short: %u < %zu\n",
+ __func__, len, sizeof(struct smb2_sess_setup_rsp));
+ return 1;
+ }
+
if (len < pdu_size) {
if ((len >= hdr_size)
&& (shdr->Status != 0)) {
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index cb164f87f908..8e72506a2409 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -1773,15 +1773,21 @@ SMB2_sess_auth_rawntlmssp_negotiate(struct SMB2_sess_data *sess_data)
if (rc)
goto out;
- if (offsetof(struct smb2_sess_setup_rsp, Buffer) !=
- le16_to_cpu(rsp->SecurityBufferOffset)) {
- cifs_dbg(VFS, "Invalid security buffer offset %d\n",
- le16_to_cpu(rsp->SecurityBufferOffset));
+ u16 boff = le16_to_cpu(rsp->SecurityBufferOffset);
+ u16 blen = le16_to_cpu(rsp->SecurityBufferLength);
+
+ if (offsetof(struct smb2_sess_setup_rsp, Buffer) != boff) {
+ cifs_dbg(VFS, "Invalid security buffer offset %d\n", boff);
rc = -EIO;
goto out;
}
- rc = decode_ntlmssp_challenge(rsp->Buffer,
- le16_to_cpu(rsp->SecurityBufferLength), ses);
+ if ((u32)boff + blen > sess_data->iov[0].iov_len) {
+ cifs_dbg(VFS, "%s: invalid security buffer length: %u\n",
+ __func__, blen);
+ rc = -EINVAL;
+ goto out;
+ }
+ rc = decode_ntlmssp_challenge(rsp->Buffer, blen, ses);
if (rc)
goto out;
@@ -4792,6 +4798,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
struct kvec iov[1];
struct kvec rsp_iov;
unsigned int total_len;
+ u16 data_offset;
int flags = CIFS_LOG_ERROR;
struct cifs_ses *ses = io_parms->tcon->ses;
@@ -4850,8 +4857,28 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
*nbytes = 0;
}
+ data_offset = rsp->DataOffset;
+ if (data_offset < io_parms->server->vals->read_rsp_size) {
+ /*
+ * win2k8 sometimes sends an offset of 0 when the read is beyond
+ * the EOF. Treat it as if the data starts just after the header,
+ * as handle_read_data() does.
+ */
+ cifs_dbg(FYI, "data offset %u inside read response header\n",
+ data_offset);
+ data_offset = io_parms->server->vals->read_rsp_size;
+ }
+
+ /* The data area must lie within the response actually received */
+ if (*nbytes && (size_t)data_offset + *nbytes > rsp_iov.iov_len) {
+ cifs_dbg(FYI, "bad data offset %u length %u for response of %zu\n",
+ data_offset, *nbytes, rsp_iov.iov_len);
+ rc = -EIO;
+ *nbytes = 0;
+ }
+
if (*buf) {
- memcpy(*buf, (char *)rsp + rsp->DataOffset, *nbytes);
+ memcpy(*buf, (char *)rsp + data_offset, *nbytes);
free_rsp_buf(resp_buftype, rsp_iov.iov_base);
} else if (resp_buftype != CIFS_NO_BUFFER) {
*buf = rsp_iov.iov_base;
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index 4e0f43051d9e..f7e3d7986c4c 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -3180,6 +3180,14 @@ int smb2_open(struct ksmbd_work *work)
req->CreateOptions = ~(FILE_NO_COMPRESSION_LE);
}
}
+
+ /* MS-FSA 2.1.5.1: delete-on-close requires DELETE access */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(smb_map_generic_desired_access(req->DesiredAccess) &
+ FILE_DELETE_LE)) {
+ rc = -EINVAL;
+ goto err_out2;
+ }
}
if (le32_to_cpu(req->CreateDisposition) >
@@ -3359,6 +3367,13 @@ int smb2_open(struct ksmbd_work *work)
maximal_access = daccess;
}
+ /* MAXIMUM_ALLOWED may not have granted the requested DELETE */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(daccess & FILE_DELETE_LE)) {
+ rc = -EACCES;
+ goto err_out;
+ }
+
open_flags = smb2_create_open_flags(file_present, daccess,
req->CreateDisposition,
&may_flags,
@@ -5275,17 +5290,8 @@ static int get_file_stream_info(struct ksmbd_work *work,
streamlen *= 2;
kfree(stream_buf);
file_info->StreamNameLength = cpu_to_le32(streamlen);
- /*
- * stream_name_len is the byte length of the xattr's *name*,
- * not its value -- same class of bug ksmbd_stream_eof()
- * (smb2pdu.c) already fixes for EndOfFile/AllocationSize on
- * a stream handle; this enumeration path needs the same
- * real xattr value length, not the name length reused as a
- * size.
- */
- slen = ksmbd_vfs_casexattr_len(file_mnt_idmap(fp->filp),
- path->dentry, stream_name,
- strlen(stream_name) + 1, NULL);
+ slen = ksmbd_vfs_xattr_len(file_mnt_idmap(fp->filp),
+ path->dentry, stream_name);
ssize = slen < 0 ? 0 : (loff_t)slen;
file_info->StreamSize = cpu_to_le64(ssize);
file_info->StreamAllocationSize = cpu_to_le64(ssize);
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index 33a607e709f1..63f7bccb4d9a 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -1606,7 +1606,10 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
else
continue;
- if (id == uid) {
+ /* A group entry grants only to the group's members */
+ if (id == uid &&
+ (pa_entry->e_tag == ACL_USER ||
+ in_group_p(make_kgid(&init_user_ns, id)))) {
mode_to_access_flags(pa_entry->e_perm,
0777,
&access_bits);
diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c
index dafb92aa39a6..9aee0214e8fd 100644
--- a/fs/smb/server/vfs.c
+++ b/fs/smb/server/vfs.c
@@ -943,8 +943,8 @@ ssize_t ksmbd_vfs_listxattr(struct dentry *dentry, char **list)
return size;
}
-static ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
- struct dentry *dentry, char *xattr_name)
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name)
{
return vfs_getxattr(idmap, dentry, xattr_name, NULL, 0);
}
diff --git a/fs/smb/server/vfs.h b/fs/smb/server/vfs.h
index 397dd2dd0988..1ca170dfb349 100644
--- a/fs/smb/server/vfs.h
+++ b/fs/smb/server/vfs.h
@@ -106,6 +106,8 @@ ssize_t ksmbd_vfs_getxattr(struct mnt_idmap *idmap,
struct dentry *dentry,
char *xattr_name,
char **xattr_buf);
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name);
ssize_t ksmbd_vfs_casexattr_len(struct mnt_idmap *idmap,
struct dentry *dentry, char *attr_name,
int attr_name_len, char *actual_name);
diff --git a/fs/udf/balloc.c b/fs/udf/balloc.c
index 30cec5600149..75124188e745 100644
--- a/fs/udf/balloc.c
+++ b/fs/udf/balloc.c
@@ -457,6 +457,7 @@ static void udf_table_free_blocks(struct super_block *sb,
int adsize;
+ eloc.partitionReferenceNum = bloc->partitionReferenceNum;
eloc.logicalBlockNum = start;
elen = EXT_RECORDED_ALLOCATED |
(count << sb->s_blocksize_bits);
@@ -532,7 +533,7 @@ static int udf_table_prealloc_blocks(struct super_block *sb,
eloc.logicalBlockNum, elen, first_block);
}
- if (first_block == eloc.logicalBlockNum) {
+ if (ret > 0 && first_block == eloc.logicalBlockNum) {
epos.offset -= adsize;
alloc_count = (elen >> sb->s_blocksize_bits);
@@ -572,7 +573,7 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
uint32_t elen, goal_elen = 0;
struct kernel_lb_addr eloc, goal_eloc;
struct extent_position epos, goal_epos;
- int8_t etype;
+ int8_t etype, goal_etype = 0;
struct udf_inode_info *iinfo = UDF_I(table);
int ret = 0;
@@ -623,7 +624,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_epos.block = epos.block;
goal_epos.offset = epos.offset - adsize;
goal_eloc = eloc;
- goal_elen = (etype << 30) | elen;
+ goal_elen = elen;
+ goal_etype = etype;
}
}
@@ -647,7 +649,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_elen -= sb->s_blocksize;
if (goal_elen)
- udf_write_aext(table, &goal_epos, &goal_eloc, goal_elen, 1);
+ udf_write_aext(table, &goal_epos, &goal_eloc,
+ (goal_etype << 30) | goal_elen, 1);
else
udf_delete_aext(table, goal_epos, &freed);
brelse(goal_epos.bh);
diff --git a/fs/udf/inode.c b/fs/udf/inode.c
index 96934d42d949..49d8a2b19641 100644
--- a/fs/udf/inode.c
+++ b/fs/udf/inode.c
@@ -1339,6 +1339,26 @@ int udf_setsize(struct inode *inode, loff_t newsize)
return err;
}
+/*
+ * Verify validity of struct deviceSpec on disk. udf_get_extendedattr() has
+ * already verified the generic header and made sure attribute fits in the
+ * inode so we just have to make sure attribute space is large enough for
+ * deviceSpec struct and required impUse information.
+ */
+static bool udf_device_spec_valid(struct deviceSpec *dsea)
+{
+ u32 attr_length, imp_use_length;
+
+ attr_length = le32_to_cpu(dsea->attrLength);
+ imp_use_length = le32_to_cpu(dsea->impUseLength);
+ if (attr_length < sizeof(struct deviceSpec) ||
+ imp_use_length < sizeof(struct regid) ||
+ imp_use_length > attr_length - sizeof(struct deviceSpec))
+ return false;
+
+ return true;
+}
+
/*
* Maximum length of linked list formed by ICB hierarchy. The chosen number is
* arbitrary - just that we hopefully don't limit any real use of rewritten
@@ -1657,13 +1677,12 @@ static int udf_read_inode(struct inode *inode, bool hidden_inode)
if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) {
struct deviceSpec *dsea =
(struct deviceSpec *)udf_get_extendedattr(inode, 12, 1);
- if (dsea) {
- init_special_inode(inode, inode->i_mode,
+
+ if (!dsea || !udf_device_spec_valid(dsea))
+ goto out;
+ init_special_inode(inode, inode->i_mode,
MKDEV(le32_to_cpu(dsea->majorDeviceIdent),
le32_to_cpu(dsea->minorDeviceIdent)));
- /* Developer ID ??? */
- } else
- goto out;
}
ret = 0;
out:
@@ -1811,6 +1830,10 @@ static int udf_update_inode(struct inode *inode, int do_sync)
udf_add_extendedattr(inode,
sizeof(struct deviceSpec) +
sizeof(struct regid), 12, 0x3);
+ if (!dsea) {
+ err = -ENOSPC;
+ goto out_unlock;
+ }
dsea->attrType = cpu_to_le32(12);
dsea->attrSubtype = 1;
dsea->attrLength = cpu_to_le32(
@@ -1818,6 +1841,10 @@ static int udf_update_inode(struct inode *inode, int do_sync)
sizeof(struct regid));
dsea->impUseLength = cpu_to_le32(sizeof(struct regid));
}
+ if (!udf_device_spec_valid(dsea)) {
+ err = -EFSCORRUPTED;
+ goto out_unlock;
+ }
eid = (struct regid *)dsea->impUse;
memset(eid, 0, sizeof(*eid));
strcpy(eid->ident, UDF_ID_DEVELOPER);
@@ -1959,6 +1986,11 @@ static int udf_update_inode(struct inode *inode, int do_sync)
brelse(bh);
return err;
+
+out_unlock:
+ unlock_buffer(bh);
+ brelse(bh);
+ return err;
}
struct inode *__udf_iget(struct super_block *sb, struct kernel_lb_addr *ino,
diff --git a/fs/udf/super.c b/fs/udf/super.c
index 556540f2797a..8d3d758714da 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -1105,6 +1105,73 @@ static int check_partition_desc(struct super_block *sb,
return 0;
}
+/*
+ * Check the Unallocated Space Table once when it is loaded: the allocator
+ * hands out blocks from the start of each extent and trusts that every
+ * extent covers at least one whole block inside the partition.
+ */
+static int udf_check_unalloc_table(struct super_block *sb,
+ struct inode *table, u32 partition_len)
+{
+ struct extent_position epos = {
+ .block = UDF_I(table)->i_location,
+ .offset = sizeof(struct unallocSpaceEntry),
+ };
+ struct kernel_lb_addr eloc;
+ uint32_t elen, blocks;
+ struct kernel_lb_addr seen_block = {};
+ uint32_t seen_offset = 0;
+ u64 steps = 0, period = 1;
+ int8_t etype;
+ int ret;
+
+ /* The walk below assumes the layout of an Unallocated Space Entry */
+ if (!UDF_I(table)->i_use) {
+ udf_err(sb, "unallocated space table is not an unallocated space entry\n");
+ return -EFSCORRUPTED;
+ }
+
+ while ((ret = udf_next_aext(table, &epos, &eloc, &elen, &etype, 1)) > 0) {
+ blocks = elen >> sb->s_blocksize_bits;
+ if (!blocks || (elen & (sb->s_blocksize - 1)) ||
+ eloc.logicalBlockNum >= partition_len ||
+ blocks > partition_len - eloc.logicalBlockNum) {
+ udf_err(sb, "invalid unallocated space table extent (block %u, length %u)\n",
+ eloc.logicalBlockNum, elen);
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ /*
+ * A chain of allocation extents can lead back to itself, and
+ * then the walk returns the same extents forever. Remember a
+ * position at doubling intervals; a walk that comes back to it
+ * is in a loop.
+ */
+ if (epos.block.logicalBlockNum == seen_block.logicalBlockNum &&
+ epos.block.partitionReferenceNum ==
+ seen_block.partitionReferenceNum &&
+ epos.offset == seen_offset) {
+ udf_err(sb, "unallocated space table loops back on itself\n");
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ if (++steps == period) {
+ seen_block = epos.block;
+ seen_offset = epos.offset;
+ steps = 0;
+ period <<= 1;
+ }
+ if (fatal_signal_pending(current)) {
+ udf_err(sb, "interrupted while checking the unallocated space table\n");
+ ret = -EINTR;
+ break;
+ }
+ cond_resched();
+ }
+ brelse(epos.bh);
+ return ret;
+}
+
static int udf_fill_partdesc_info(struct super_block *sb,
struct partitionDesc *p, int p_index)
{
@@ -1165,6 +1232,16 @@ static int udf_fill_partdesc_info(struct super_block *sb,
p_index);
return PTR_ERR(inode);
}
+ err = udf_check_unalloc_table(sb, inode, map->s_partition_len);
+ if (err) {
+ iput(inode);
+ if (err == -EINTR)
+ return err;
+ if (!sb_rdonly(sb))
+ return -EACCES;
+ UDF_SET_FLAG(sb, UDF_FLAG_RW_INCOMPAT);
+ return 0;
+ }
map->s_uspace.s_table = inode;
map->s_partition_flags |= UDF_PART_FLAG_UNALLOC_TABLE;
udf_debug("unallocSpaceTable (part %d) @ %lu\n",
@@ -2228,8 +2305,9 @@ static int udf_fill_super(struct super_block *sb, struct fs_context *fc)
/*
* EACCES is special - we want to propagate to
* upper layers that we cannot handle RW mount.
+ * EINTR means that a fatal signal is pending.
*/
- if (ret == -EACCES)
+ if (ret == -EACCES || ret == -EINTR)
break;
} else
break;
diff --git a/fs/udf/truncate.c b/fs/udf/truncate.c
index da5a61306390..85bbd6fa2a30 100644
--- a/fs/udf/truncate.c
+++ b/fs/udf/truncate.c
@@ -124,8 +124,8 @@ void udf_discard_prealloc(struct inode *inode)
{
struct extent_position epos = {};
struct extent_position prev_epos = {};
- struct kernel_lb_addr eloc;
- uint32_t elen;
+ struct kernel_lb_addr eloc, tmpeloc;
+ uint32_t elen, tmpelen;
uint64_t lbcount = 0;
int8_t etype = -1;
struct udf_inode_info *iinfo = UDF_I(inode);
@@ -141,7 +141,8 @@ void udf_discard_prealloc(struct inode *inode)
/* Find the last extent in the file */
while (1) {
- ret = udf_next_aext(inode, &epos, &eloc, &elen, &tmpetype, 0);
+ ret = udf_next_aext(inode, &epos, &tmpeloc, &tmpelen,
+ &tmpetype, 0);
if (ret < 0)
goto out;
if (ret == 0)
diff --git a/fs/verity/enable.c b/fs/verity/enable.c
index c284f46d1b53..197564ecd3ef 100644
--- a/fs/verity/enable.c
+++ b/fs/verity/enable.c
@@ -292,6 +292,14 @@ static int enable_verity(struct file *filp,
* after the filesystem has successfully enabled verity.
*/
fsverity_set_info(inode, vi);
+
+ /*
+ * Now that verity is enabled, report the signature (if any) to
+ * LSMs. We tolerate that a process racing with this ioctl can
+ * briefly see verity enabled with no signature reported yet.
+ */
+ fsverity_report_valid_sig(inode, desc->signature,
+ le32_to_cpu(desc->sig_size));
}
out:
kfree(params.hashstate);
diff --git a/fs/verity/fsverity_private.h b/fs/verity/fsverity_private.h
index b3506f56e180..760f2674c816 100644
--- a/fs/verity/fsverity_private.h
+++ b/fs/verity/fsverity_private.h
@@ -135,6 +135,8 @@ void __init fsverity_init_info_cache(void);
extern int fsverity_require_signatures;
int fsverity_verify_signature(const struct fsverity_info *vi,
const u8 *signature, size_t sig_size);
+void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature,
+ size_t sig_size);
void __init fsverity_init_signature(void);
#else /* !CONFIG_FS_VERITY_BUILTIN_SIGNATURES */
@@ -145,6 +147,12 @@ fsverity_verify_signature(const struct fsverity_info *vi,
return 0;
}
+static inline void fsverity_report_valid_sig(const struct inode *inode,
+ const u8 *signature,
+ size_t sig_size)
+{
+}
+
static inline void fsverity_init_signature(void)
{
}
diff --git a/fs/verity/measure.c b/fs/verity/measure.c
index 91ce39ced77b..b5af400c725e 100644
--- a/fs/verity/measure.c
+++ b/fs/verity/measure.c
@@ -30,6 +30,16 @@ int fsverity_ioctl_measure(struct file *filp, void __user *_uarg)
const struct fsverity_hash_alg *hash_alg;
struct fsverity_digest arg;
+ /*
+ * This ioctl returns a hash of the file contents, so it should require
+ * read permission. Disallow it on ioctl-only (i.e., nonstandard access
+ * mode 3) fds, which can in some cases be acquired without passing all
+ * LSM read-permission checks. Also disallow write-only fds, which can
+ * refer only to non-verity files which would fail with ENODATA anyway.
+ */
+ if (!(filp->f_mode & FMODE_READ))
+ return -EBADF;
+
vi = fsverity_get_info(inode);
if (!vi)
return -ENODATA; /* not a verity file */
diff --git a/fs/verity/open.c b/fs/verity/open.c
index 9c1b37a3b96a..dba70c0ab5b6 100644
--- a/fs/verity/open.c
+++ b/fs/verity/open.c
@@ -377,6 +377,15 @@ static int ensure_verity_info(struct inode *inode)
goto out_free_desc;
}
+ /*
+ * Now that the signature (if any) has been verified, report it to LSMs.
+ * Do this before setting the fsverity_info, so that anyone who
+ * concurrently retrieves the fsverity_info for the same inode is
+ * guaranteed to see the LSM's signature status populated.
+ */
+ fsverity_report_valid_sig(inode, desc->signature,
+ le32_to_cpu(desc->sig_size));
+
fsverity_set_info(inode, vi);
err = 0;
out_free_desc:
diff --git a/fs/verity/read_metadata.c b/fs/verity/read_metadata.c
index f58432772d9e..680792701ba2 100644
--- a/fs/verity/read_metadata.c
+++ b/fs/verity/read_metadata.c
@@ -156,13 +156,19 @@ int fsverity_ioctl_read_metadata(struct file *filp, const void __user *uarg)
int length;
void __user *buf;
+ /*
+ * This ioctl returns file contents block hashes, so it should require
+ * read permission. Disallow it on ioctl-only (i.e., nonstandard access
+ * mode 3) fds, which can in some cases be acquired without passing all
+ * LSM read-permission checks. Also disallow write-only fds, which can
+ * refer only to non-verity files which would fail with ENODATA anyway.
+ */
+ if (!(filp->f_mode & FMODE_READ))
+ return -EBADF;
+
vi = fsverity_get_info(inode);
if (!vi)
return -ENODATA; /* not a verity file */
- /*
- * Note that we don't have to explicitly check that the file is open for
- * reading, since verity files can only be opened for reading.
- */
if (copy_from_user(&arg, uarg, sizeof(arg)))
return -EFAULT;
diff --git a/fs/verity/signature.c b/fs/verity/signature.c
index 0302a4e506ec..212e873c8499 100644
--- a/fs/verity/signature.c
+++ b/fs/verity/signature.c
@@ -42,13 +42,16 @@ static struct key *fsverity_keyring;
* @sig_size: size of signature in bytes, or 0 if no signature
*
* If the file includes a signature of its fs-verity file digest, verify it
- * against the certificates in the fs-verity keyring. Note that signatures
- * are verified regardless of the state of the 'fsverity_require_signatures'
- * variable and the LSM subsystem relies on this behavior to help enforce
- * file integrity policies. Please discuss changes with the LSM list
- * (thank you!).
+ * against the certificates in the fs-verity keyring.
*
- * Return: 0 on success (signature valid or not required); -errno on failure
+ * Note that signatures are verified regardless of the state of the
+ * 'fsverity_require_signatures' variable, and the LSM subsystem relies on this
+ * behavior to help enforce file integrity policies. (See
+ * fsverity_report_valid_sig(), which assumes that the signature, if present,
+ * was verified here.) Please discuss changes with the LSM list (thank you!).
+ *
+ * Return: 0 if the signature is present and valid, or absent and not required;
+ * -errno on failure
*/
int fsverity_verify_signature(const struct fsverity_info *vi,
const u8 *signature, size_t sig_size)
@@ -111,18 +114,40 @@ int fsverity_verify_signature(const struct fsverity_info *vi,
return err;
}
- err = security_inode_setintegrity(inode,
- LSM_INT_FSVERITY_BUILTINSIG_VALID,
- signature,
- sig_size);
+ return 0;
+}
- if (err) {
+/*
+ * If the signature exists, this tells the LSM that it was verified.
+ *
+ * fsverity_verify_signature() must have already been called with the same
+ * signature, *and* the inode must have verity enabled with the signature
+ * matching it. If called from FS_IOC_ENABLE_VERITY, the latter means that we
+ * must be past the point of no return, i.e. verity won't fail to be enabled at
+ * a later point. This is critical, since LSMs must be told about the signature
+ * only when the verity status is permanent in the in-core inode.
+ */
+void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature,
+ size_t sig_size)
+{
+ int err;
+
+ if (WARN_ON_ONCE(!IS_VERITY(inode)))
+ return;
+
+ if (sig_size == 0)
+ return;
+
+ err = security_inode_setintegrity(
+ inode, LSM_INT_FSVERITY_BUILTINSIG_VALID, signature, sig_size);
+ if (err)
fsverity_err(inode, "Error %d exposing file signature to LSMs",
err);
- return err;
- }
-
- return 0;
+ /*
+ * Besides the logging above, ignore errors here. The LSM hook just
+ * records the signature and doesn't perform access control. And in the
+ * case of FS_IOC_ENABLE_VERITY verity was already enabled.
+ */
}
void __init fsverity_init_signature(void)
diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c
index aef8d66a016f..b3f1eaa1ec45 100644
--- a/fs/xfs/libxfs/xfs_attr_leaf.c
+++ b/fs/xfs/libxfs/xfs_attr_leaf.c
@@ -2961,7 +2961,14 @@ xfs_attr3_leaf_setflag(
#endif
entry = &xfs_attr3_leaf_entryp(leaf)[args->index];
- ASSERT((entry->flags & XFS_ATTR_INCOMPLETE) == 0);
+ if (entry->flags & XFS_ATTR_INCOMPLETE) {
+ if (!(args->op_flags & XFS_DA_OP_RECOVERY)) {
+ ASSERT(args->op_flags & XFS_DA_OP_RECOVERY);
+ return -EFSCORRUPTED;
+ }
+ return 0;
+ }
+
entry->flags |= XFS_ATTR_INCOMPLETE;
xfs_trans_log_buf(args->trans, bp,
XFS_DA_LOGRANGE(leaf, entry, sizeof(*entry)));
diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c
index f0ddaa136a0b..de09db78df1a 100644
--- a/fs/xfs/libxfs/xfs_btree.c
+++ b/fs/xfs/libxfs/xfs_btree.c
@@ -2568,7 +2568,7 @@ xfs_btree_lshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_decrement(tcur, level, &i);
@@ -2738,7 +2738,7 @@ xfs_btree_rshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_increment(tcur, level, &i);
diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c
index 7315d0cb1e2e..68d9666ab201 100644
--- a/fs/xfs/scrub/attr.c
+++ b/fs/xfs/scrub/attr.c
@@ -293,6 +293,78 @@ xchk_xattr_set_map(
return ret;
}
+static unsigned int
+xchk_xattr_local_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx)
+{
+ struct xfs_attr_leaf_name_local *lentry =
+ xfs_attr3_leaf_name_local(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
+ be16_to_cpu(lentry->valuelen));
+ name_end = (char *)lentry + namesize;
+ if (lentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
+static unsigned int
+xchk_xattr_remote_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx,
+ struct xfs_attr_leaf_entry *ent)
+{
+ struct xfs_attr_leaf_name_remote *rentry =
+ xfs_attr3_leaf_name_remote(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
+ name_end = (char *)rentry + namesize;
+ if (rentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (rentry->valueblk == 0 && !(ent->flags & XFS_ATTR_INCOMPLETE)) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
/*
* Check this leaf entry's relations to everything else.
* Returns the number of bytes used for the name/value data.
@@ -311,9 +383,6 @@ xchk_xattr_entry(
{
struct xfs_mount *mp = ds->state->mp;
struct xchk_xattr_buf *ab = ds->sc->buf;
- char *name_end;
- struct xfs_attr_leaf_name_local *lentry;
- struct xfs_attr_leaf_name_remote *rentry;
unsigned int nameidx;
unsigned int namesize;
@@ -333,25 +402,14 @@ xchk_xattr_entry(
}
/* Check the name information. */
- if (ent->flags & XFS_ATTR_LOCAL) {
- lentry = xfs_attr3_leaf_name_local(leaf, idx);
- namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
- be16_to_cpu(lentry->valuelen));
- name_end = (char *)lentry + namesize;
- if (lentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- } else {
- rentry = xfs_attr3_leaf_name_remote(leaf, idx);
- namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- name_end = (char *)rentry + namesize;
- if (rentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- if (rentry->valueblk == 0 &&
- !(ent->flags & XFS_ATTR_INCOMPLETE))
- xchk_da_set_corrupt(ds, level);
- }
- if (name_end > buf_end)
- xchk_da_set_corrupt(ds, level);
+ if (ent->flags & XFS_ATTR_LOCAL)
+ namesize = xchk_xattr_local_entry(ds, level, buf_end, leaf,
+ idx);
+ else
+ namesize = xchk_xattr_remote_entry(ds, level, buf_end, leaf,
+ idx, ent);
+ if (!namesize)
+ return;
if (!xchk_xattr_set_map(ds->sc, ab->usedmap, nameidx, namesize))
xchk_da_set_corrupt(ds, level);
diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c
index ab5bd7a4d4ab..2c573897c0c6 100644
--- a/fs/xfs/scrub/attr_repair.c
+++ b/fs/xfs/scrub/attr_repair.c
@@ -285,6 +285,7 @@ xrep_xattr_salvage_local_attr(
struct xfs_attr_leaf_name_local *lentry)
{
struct xchk_xattr_buf *ab = rx->sc->buf;
+ char *name_end;
unsigned char *value;
unsigned int valuelen;
unsigned int namesize;
@@ -293,10 +294,14 @@ xrep_xattr_salvage_local_attr(
* Decode the leaf local entry format. If something seems wrong, we
* junk the attribute.
*/
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end)
+ return 0;
value = &lentry->nameval[lentry->namelen];
valuelen = be16_to_cpu(lentry->valuelen);
namesize = xfs_attr_leaf_entsize_local(lentry->namelen, valuelen);
- if ((char *)lentry + namesize > buf_end)
+ name_end = (char *)lentry + namesize;
+ if (name_end > buf_end)
return 0;
if (!xrep_xattr_want_salvage(rx, ent->flags, lentry->nameval,
lentry->namelen, value, valuelen))
@@ -333,8 +338,8 @@ xrep_xattr_salvage_remote_attr(
.attr_filter = ent->flags & XFS_ATTR_NSP_ONDISK_MASK,
.namelen = rentry->namelen,
.name = rentry->name,
- .valuelen = be32_to_cpu(rentry->valuelen),
};
+ char *name_end;
unsigned int namesize;
int error;
@@ -343,9 +348,14 @@ xrep_xattr_salvage_remote_attr(
* junk the attribute. Note that we should never find a zero-length
* remote attribute value.
*/
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end)
+ return 0;
namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- if ((char *)rentry + namesize > buf_end)
+ name_end = (char *)rentry + namesize;
+ if (name_end > buf_end)
return 0;
+ args.valuelen = be32_to_cpu(rentry->valuelen);
if (args.valuelen == 0 ||
!xrep_xattr_want_salvage(rx, ent->flags, rentry->name,
rentry->namelen, NULL, args.valuelen))
diff --git a/fs/xfs/scrub/cow_repair.c b/fs/xfs/scrub/cow_repair.c
index e8f7cfd4dd22..853744729577 100644
--- a/fs/xfs/scrub/cow_repair.c
+++ b/fs/xfs/scrub/cow_repair.c
@@ -194,30 +194,37 @@ xrep_cow_mark_missing_staging_rmap(
void *priv)
{
struct xrep_cow *xc = priv;
- xfs_fsblock_t fsbno;
- xfs_agblock_t rec_bno;
- xfs_extlen_t rec_len;
- unsigned int adj;
+ const xfs_agblock_t rec_end =
+ rec->rm_startblock + rec->rm_blockcount;
+ int error = 0;
- if (rec->rm_owner == XFS_RMAP_OWN_COW)
- return 0;
-
- rec_bno = rec->rm_startblock;
- rec_len = rec->rm_blockcount;
- if (rec_bno < xc->irec_startbno) {
- adj = xc->irec_startbno - rec_bno;
- rec_len -= adj;
- rec_bno += adj;
+ /* Was there a gap since the last rmap? */
+ if (xc->next_bno < rec->rm_startblock) {
+ error = xrep_cow_mark_file_range(xc,
+ XFS_AGB_TO_FSB(xc->sc->mp, cur->bc_ag.pag->pag_agno,
+ xc->next_bno),
+ rec->rm_startblock - xc->next_bno);
+ if (error)
+ return error;
}
- if (rec_bno + rec_len > xc->irec_startbno + xc->irec.br_blockcount) {
- adj = (rec_bno + rec_len) -
- (xc->irec_startbno + xc->irec.br_blockcount);
- rec_len -= adj;
+ if (rec->rm_owner != XFS_RMAP_OWN_COW) {
+ const xfs_agblock_t bad_bno =
+ max(xc->irec_startbno, rec->rm_startblock);
+ const xfs_agblock_t irec_end =
+ xc->irec_startbno + xc->irec.br_blockcount;
+ const xfs_extlen_t bad_end = min(irec_end, rec_end);
+
+ error = xrep_cow_mark_file_range(xc,
+ XFS_AGB_TO_FSB(xc->sc->mp, cur->bc_ag.pag->pag_agno,
+ bad_bno),
+ bad_end - bad_bno);
+ if (error)
+ return error;
}
- fsbno = XFS_AGB_TO_FSB(xc->sc->mp, cur->bc_ag.pag->pag_agno, rec_bno);
- return xrep_cow_mark_file_range(xc, fsbno, rec_len);
+ xc->next_bno = max(xc->next_bno, rec_end);
+ return error;
}
/*
@@ -235,10 +242,12 @@ xrep_cow_find_bad(
struct xfs_perag *pag;
struct xfs_scrub *sc = xc->sc;
xfs_agnumber_t agno;
+ xfs_agblock_t irec_end;
int error;
agno = XFS_FSB_TO_AGNO(sc->mp, xc->irec.br_startblock);
xc->irec_startbno = XFS_FSB_TO_AGBNO(sc->mp, xc->irec.br_startblock);
+ irec_end = xc->irec_startbno + xc->irec.br_blockcount;
pag = xfs_perag_get(sc->mp, agno);
if (!pag)
@@ -250,7 +259,7 @@ xrep_cow_find_bad(
/* Mark any CoW fork extents that are shared. */
rc_low.rc_startblock = xc->irec_startbno;
- rc_high.rc_startblock = xc->irec_startbno + xc->irec.br_blockcount - 1;
+ rc_high.rc_startblock = irec_end - 1;
rc_low.rc_domain = rc_high.rc_domain = XFS_REFC_DOMAIN_SHARED;
error = xfs_refcount_query_range(sc->sa.refc_cur, &rc_low, &rc_high,
xrep_cow_mark_shared_staging, xc);
@@ -259,7 +268,7 @@ xrep_cow_find_bad(
/* Make sure there are CoW staging extents for the whole mapping. */
rc_low.rc_startblock = xc->irec_startbno;
- rc_high.rc_startblock = xc->irec_startbno + xc->irec.br_blockcount - 1;
+ rc_high.rc_startblock = irec_end - 1;
rc_low.rc_domain = rc_high.rc_domain = XFS_REFC_DOMAIN_COW;
xc->next_bno = xc->irec_startbno;
error = xfs_refcount_query_range(sc->sa.refc_cur, &rc_low, &rc_high,
@@ -267,12 +276,11 @@ xrep_cow_find_bad(
if (error)
goto out_sa;
- if (xc->next_bno < xc->irec_startbno + xc->irec.br_blockcount) {
+ if (xc->next_bno < irec_end) {
error = xrep_cow_mark_file_range(xc,
XFS_AGB_TO_FSB(sc->mp, pag->pag_agno,
xc->next_bno),
- xc->irec_startbno + xc->irec.br_blockcount -
- xc->next_bno);
+ irec_end - xc->next_bno);
if (error)
goto out_sa;
}
@@ -281,11 +289,21 @@ xrep_cow_find_bad(
rm_low.rm_startblock = xc->irec_startbno;
memset(&rm_high, 0xFF, sizeof(rm_high));
rm_high.rm_startblock = xc->irec_startbno + xc->irec.br_blockcount - 1;
+ xc->next_bno = xc->irec_startbno;
error = xfs_rmap_query_range(sc->sa.rmap_cur, &rm_low, &rm_high,
xrep_cow_mark_missing_staging_rmap, xc);
if (error)
goto out_sa;
+ if (xc->next_bno < irec_end) {
+ error = xrep_cow_mark_file_range(xc,
+ XFS_AGB_TO_FSB(sc->mp, pag->pag_agno,
+ xc->next_bno),
+ irec_end - xc->next_bno);
+ if (error)
+ goto out_sa;
+ }
+
/*
* If userspace is forcing us to rebuild the CoW fork or someone turned
* on the debugging knob, replace everything in the CoW fork.
diff --git a/fs/xfs/scrub/dir.c b/fs/xfs/scrub/dir.c
index 284f72ef2b58..2bb5ead7cffd 100644
--- a/fs/xfs/scrub/dir.c
+++ b/fs/xfs/scrub/dir.c
@@ -305,6 +305,74 @@ xchk_dir_actor(
return error;
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, zero to stop processing, or -1 if corruption
+ * was found.
+ */
+static int
+xchk_dir_check_unused(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ unsigned int advance;
+
+ if (iter_off + sizeof(*dup) > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ return advance;
+}
+
+/*
+ * Check a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or -1 if corruption was
+ * found.
+ */
+static int
+xchk_dir_check_entry(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ struct xfs_dir2_data_entry *dent,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ struct xfs_mount *mp = ds->state->mp;
+ unsigned int advance;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (iter_off + advance >= end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(mp, dep->namelen);
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ if (dep == dent)
+ return 0;
+
+ return advance;
+}
+
/* Scrub a directory btree record. */
STATIC int
xchk_dir_rec(
@@ -330,6 +398,7 @@ xchk_dir_rec(
xfs_dahash_t hash;
struct xfs_dir3_icleaf_hdr hdr;
unsigned int tag;
+ bool foundit = false;
int error;
ASSERT(blk->magic == XFS_DIR2_LEAF1_MAGIC ||
@@ -374,28 +443,42 @@ xchk_dir_rec(
dent = bp->b_addr + off;
/* Make sure we got a real directory entry. */
- iter_off = geo->data_entry_offset;
end = xfs_dir3_data_end_offset(geo, bp->b_addr);
if (!end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- for (;;) {
- struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ iter_off = geo->data_entry_offset;
+ while (iter_off < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ int advance;
- if (iter_off >= end) {
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (iter_off + advance >= end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- iter_off += be16_to_cpu(dup->length);
- continue;
- }
- if (dep == dent)
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xchk_dir_check_unused(ds, bp, iter_off, end,
+ rec_bno);
+ else
+ advance = xchk_dir_check_entry(ds, bp, iter_off, end,
+ dent, rec_bno);
+ if (advance < 0)
+ goto out_relse;
+ if (!advance) {
+ foundit = true;
break;
- iter_off += xfs_dir2_data_entsize(mp, dep->namelen);
+ }
+ iter_off += advance;
+ }
+
+ /* Hash tree must point to the exact dirent */
+ if (!foundit) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
}
/* Retrieve the entry, sanity check it, and compare hashes. */
@@ -409,6 +492,13 @@ xchk_dir_rec(
goto out_relse;
}
+ /* Name must not overflow end of block */
+ if ((char *)dent->name + dent->namelen >
+ (char *)bp->b_addr + BBTOB(bp->b_length)) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
+ }
+
/* Does the directory hash match? */
dname.name = dent->name;
dname.len = dent->namelen;
diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c
index 2bc2354cd85d..72fe0ae3e18d 100644
--- a/fs/xfs/scrub/dir_repair.c
+++ b/fs/xfs/scrub/dir_repair.c
@@ -456,6 +456,63 @@ xrep_dir_salvage_data_entry(
return xrep_dir_salvage_entry(rd, dep->name, dep->namelen, ino);
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, or zero to stop processing.
+ */
+static int
+xrep_dir_recover_unused(
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
+ unsigned int advance;
+
+ if (offset + sizeof(*dup) > end)
+ return 0;
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || offset + advance > end)
+ return 0;
+
+ return advance;
+}
+
+/*
+ * Salvage a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or a negative errno.
+ */
+static int
+xrep_dir_recover_entry(
+ struct xrep_dir *rd,
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
+ unsigned int advance;
+ int error;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (offset + advance >= end)
+ return 0;
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (!advance || offset + advance > end)
+ return 0;
+
+ /* Ok, let's save this entry. */
+ error = xrep_dir_salvage_data_entry(rd, dep);
+ if (error < 0)
+ return error;
+
+ return advance;
+}
+
+
/* Try to recover block/data format directory entries. */
STATIC int
xrep_dir_recover_data(
@@ -477,33 +534,25 @@ xrep_dir_recover_data(
while (offset < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
- struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
- unsigned int advance;
+ int advance;
if (xchk_should_terminate(rd->sc, &error))
return error;
- /* Skip unused entries. */
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- if (!dup->length)
- break;
- offset += be16_to_cpu(dup->length);
- continue;
- }
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (offset + advance >= end)
+ break;
- /* Don't walk off the end of the block. */
- advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xrep_dir_recover_unused(bp, offset, end);
+ else
+ advance = xrep_dir_recover_entry(rd, bp, offset, end);
+ if (advance < 0)
+ return advance;
if (!advance)
break;
offset += advance;
- if (offset > end)
- break;
-
- /* Ok, let's save this entry. */
- error = xrep_dir_salvage_data_entry(rd, dep);
- if (error)
- return error;
-
}
return 0;
@@ -526,6 +575,19 @@ xrep_dir_recover_sf(
hdr = ifp->if_data;
end = (unsigned char *)ifp->if_data + ifp->if_bytes;
+ /* sf header must be big enough for count/i8count */
+ if (!hdr || ifp->if_bytes < 2)
+ return 0;
+
+ /* and large enough for the parent inumber */
+ if (hdr->i8count) {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[8]))
+ return 0;
+ } else {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[4]))
+ return 0;
+ }
+
ino = xfs_dir2_sf_get_parent_ino(hdr);
trace_xrep_dir_salvaged_parent(rd->sc->ip, ino);
diff --git a/fs/xfs/scrub/dirtree.c b/fs/xfs/scrub/dirtree.c
index 8933506c4830..35848cbfae92 100644
--- a/fs/xfs/scrub/dirtree.c
+++ b/fs/xfs/scrub/dirtree.c
@@ -849,7 +849,8 @@ xchk_dirtree_find_paths_to_root(
* A parent pointer of @sc->ip is bad, don't
* bother continuing.
*/
- break;
+ xchk_set_incomplete(sc);
+ return error;
}
if (error == -ESTALE) {
/* This had better be an invalidation. */
@@ -970,6 +971,7 @@ xchk_dirtree(
* parent pointers are corrupt; this scan cannot be completed
* without full information.
*/
+ xchk_set_incomplete(sc);
xchk_ino_xref_set_corrupt(sc, sc->ip->i_ino);
error = 0;
goto out_scanlock;
diff --git a/fs/xfs/scrub/fscounters.c b/fs/xfs/scrub/fscounters.c
index 98634c668b56..ad23343c52f7 100644
--- a/fs/xfs/scrub/fscounters.c
+++ b/fs/xfs/scrub/fscounters.c
@@ -296,7 +296,7 @@ xchk_fscount_aggregate_agcounts(
{
struct xfs_mount *mp = sc->mp;
struct xfs_perag *pag;
- uint64_t delayed;
+ int64_t delayed;
xfs_agnumber_t agno;
int tries = 8;
int error = 0;
@@ -359,6 +359,11 @@ xchk_fscount_aggregate_agcounts(
* of the computation.
*/
delayed = percpu_counter_sum(&mp->m_delalloc_blks);
+ if (delayed < 0) {
+ ASSERT(delayed >= 0);
+ xchk_set_incomplete(sc);
+ return -EFSCORRUPTED;
+ }
fsc->fdblocks -= delayed;
trace_xchk_fscounters_calc(mp, fsc->icount, fsc->ifree, fsc->fdblocks,
@@ -416,6 +421,7 @@ xchk_fscount_count_frextents(
struct xchk_fscounters *fsc)
{
struct xfs_mount *mp = sc->mp;
+ int64_t delayed;
int error;
fsc->frextents = 0;
@@ -431,7 +437,14 @@ xchk_fscount_count_frextents(
goto out_unlock;
}
- fsc->frextents_delayed = percpu_counter_sum(&mp->m_delalloc_rtextents);
+ delayed = percpu_counter_sum(&mp->m_delalloc_rtextents);
+ if (delayed < 0) {
+ ASSERT(delayed >= 0);
+ xchk_set_incomplete(sc);
+ error = -EFSCORRUPTED;
+ goto out_unlock;
+ }
+ fsc->frextents_delayed = delayed;
out_unlock:
xfs_rtbitmap_unlock_shared(sc->mp, XFS_RBMLOCK_BITMAP);
@@ -606,6 +619,13 @@ xchk_fscounters(
try_again = true;
}
+ if (fsc->frextents < fsc->frextents_delayed) {
+ if (fsc->frozen)
+ xchk_set_incomplete(sc);
+ else
+ try_again = true;
+ }
+
if (!xchk_fscount_within_range(sc, frextents, &mp->m_frextents,
fsc->frextents - fsc->frextents_delayed)) {
if (fsc->frozen)
diff --git a/fs/xfs/scrub/inode_repair.c b/fs/xfs/scrub/inode_repair.c
index 6be4551bffbd..fa84a2efee43 100644
--- a/fs/xfs/scrub/inode_repair.c
+++ b/fs/xfs/scrub/inode_repair.c
@@ -38,6 +38,8 @@
#include "xfs_log_priv.h"
#include "xfs_health.h"
#include "xfs_symlink_remote.h"
+#include "xfs_dquot_item.h"
+#include "xfs_dquot.h"
#include "scrub/xfs_scrub.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
@@ -1570,6 +1572,20 @@ xrep_inode_ids(
trace_xrep_inode_ids(sc);
+ /*
+ * We changed one of the file IDs, so we must detach the dquots to
+ * prevent further changes from being accounted to them, and try to
+ * attach the dquot for the new id to absorb any other block usage
+ * changes. We can't do that right now because the existing dquots
+ * might be attached to the scrub transaction and we're in the middle
+ * of fixing the inode, so we'll switch the incore dquot pointers
+ * later.
+ *
+ * Schedule a quotacheck so the accounting discrepancies will get fixed
+ * eventually. We'll have to repeat this later when we do the actual
+ * switch.
+ */
+
if (!uid_valid(VFS_I(sc->ip)->i_uid)) {
i_uid_write(VFS_I(sc->ip), 0);
dirty = true;
@@ -1876,6 +1892,112 @@ xrep_inode_unlinked(
return 0;
}
+#ifdef CONFIG_XFS_QUOTA
+static inline bool
+xrep_inode_need_reset_dquots(
+ struct xfs_scrub *sc)
+{
+ struct xfs_mount *mp = sc->mp;
+ struct xfs_inode *ip = sc->ip;
+
+ if (!XFS_IS_QUOTA_ON(mp))
+ return false;
+ if (xfs_is_quota_inode(&mp->m_sb, I_INO(ip)))
+ return false;
+
+ if (XFS_IS_UQUOTA_ON(sc->mp) &&
+ (!ip->i_udquot || ip->i_udquot->q_id != i_uid_read(VFS_I(ip))))
+ return true;
+ if (XFS_IS_GQUOTA_ON(sc->mp) &&
+ (!ip->i_gdquot || ip->i_gdquot->q_id != i_gid_read(VFS_I(ip))))
+ return true;
+ if (XFS_IS_PQUOTA_ON(sc->mp) &&
+ (!ip->i_pdquot || ip->i_pdquot->q_id != ip->i_projid))
+ return true;
+
+ return false;
+}
+
+/* Switch dquot, return dqtype if we need quotacheck */
+static xfs_dqtype_t
+xrep_inode_switch_dquot(
+ struct xfs_scrub *sc,
+ xfs_dqtype_t type,
+ struct xfs_dquot **dqp)
+{
+ xfs_dqtype_t ret = 0;
+ xfs_dqid_t id = xfs_qm_id_for_quotatype(sc->ip, type);
+
+ if (*dqp != NULL && (*dqp)->q_id != id) {
+ xfs_qm_dqrele(*dqp);
+ *dqp = NULL;
+ }
+
+ if (!*dqp) {
+ if (!xfs_qm_dqget_inode(sc->ip, type, true, dqp))
+ xfs_dqunlock(*dqp);
+ ret = type;
+ }
+ return ret;
+}
+
+static int
+xrep_inode_reset_dquots(
+ struct xfs_scrub *sc)
+{
+ xfs_dqtype_t need = 0;
+ int error;
+
+ /*
+ * If we changed the file IDs, then we need to drop the transaction so
+ * that we can detach the dquots and attach new ones.
+ *
+ * Commit the scrub transaction so that there are no dangling
+ * references to any dquots that might get detached, and retain the
+ * ILOCK, as required by the dqattach code.
+ *
+ * Note that dqattach can cycle the ILOCK when creating a new dquot, so
+ * this must be the very last thing that the repair function does.
+ */
+ error = xrep_trans_commit(sc);
+ if (error)
+ return error;
+
+ if (XFS_IS_UQUOTA_ON(sc->mp))
+ need |= xrep_inode_switch_dquot(sc, XFS_DQTYPE_USER,
+ &sc->ip->i_udquot);
+ if (XFS_IS_GQUOTA_ON(sc->mp))
+ need |= xrep_inode_switch_dquot(sc, XFS_DQTYPE_GROUP,
+ &sc->ip->i_gdquot);
+ if (XFS_IS_PQUOTA_ON(sc->mp))
+ need |= xrep_inode_switch_dquot(sc, XFS_DQTYPE_PROJ,
+ &sc->ip->i_pdquot);
+
+ xchk_iunlock(sc, XFS_ILOCK_EXCL);
+
+ /*
+ * Now that we've updated the dquots, force quotacheck once again.
+ * In theory we could be racing against a concurrent quotacheck.
+ * Note that we can't hold the ILOCK when allocating a transaction.
+ */
+ error = xchk_trans_alloc(sc, 0);
+ if (error)
+ return error;
+
+ if (need & XFS_DQTYPE_USER)
+ xrep_force_quotacheck(sc, XFS_DQTYPE_USER);
+ if (need & XFS_DQTYPE_GROUP)
+ xrep_force_quotacheck(sc, XFS_DQTYPE_GROUP);
+ if (need & XFS_DQTYPE_PROJ)
+ xrep_force_quotacheck(sc, XFS_DQTYPE_PROJ);
+
+ return 0;
+}
+#else
+# define xrep_inode_need_reset_dquots(sc) (false)
+# define xrep_inode_reset_dquots(sc) (0)
+#endif
+
/* Repair an inode's fields. */
int
xrep_inode(
@@ -1930,5 +2052,13 @@ xrep_inode(
if (error)
return error;
- return xrep_defer_finish(sc);
+ error = xrep_defer_finish(sc);
+ if (error)
+ return error;
+
+ /* Must come last */
+ if (xrep_inode_need_reset_dquots(sc))
+ return xrep_inode_reset_dquots(sc);
+
+ return 0;
}
diff --git a/fs/xfs/scrub/newbt.c b/fs/xfs/scrub/newbt.c
index b818d9b21578..80f9bf7e399b 100644
--- a/fs/xfs/scrub/newbt.c
+++ b/fs/xfs/scrub/newbt.c
@@ -60,7 +60,7 @@ xrep_newbt_estimate_slack(
free = sc->sa.pag->pagf_freeblks;
sz = xfs_ag_block_count(sc->mp, sc->sa.pag->pag_agno);
} else {
- free = percpu_counter_sum(&sc->mp->m_fdblocks);
+ free = percpu_counter_sum_positive(&sc->mp->m_fdblocks);
sz = sc->mp->m_sb.sb_dblocks;
}
diff --git a/fs/xfs/scrub/quota_repair.c b/fs/xfs/scrub/quota_repair.c
index 95d5bb8c1003..0def7a1b366a 100644
--- a/fs/xfs/scrub/quota_repair.c
+++ b/fs/xfs/scrub/quota_repair.c
@@ -299,7 +299,6 @@ xrep_quota_block(
xfs_dqid_t id)
{
struct xfs_dqblk *dqblk;
- struct xfs_disk_dquot *ddq;
struct xfs_quotainfo *qi = sc->mp->m_quotainfo;
struct xfs_def_quota *defq = xfs_get_defquota(qi, dqtype);
struct xfs_buf *bp = NULL;
@@ -321,14 +320,21 @@ xrep_quota_block(
break;
case 0:
dqblk = bp->b_addr;
- ddq = &dqblk[0].dd_diskdq;
+ error = 0;
/*
* If there's nothing that would impede a dqiterate, we're
* done.
*/
- if ((ddq->d_type & XFS_DQTYPE_REC_MASK) == dqtype &&
- id == be32_to_cpu(ddq->d_id)) {
+ for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+
+ if ((ddq->d_type & XFS_DQTYPE_REC_MASK) != dqtype ||
+ id + i != be32_to_cpu(ddq->d_id))
+ error++;
+ }
+
+ if (!error) {
xfs_trans_brelse(sc->tp, bp);
return 0;
}
@@ -341,7 +347,9 @@ xrep_quota_block(
dqblk = bp->b_addr;
bp->b_ops = &xfs_dquot_buf_ops;
for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
- ddq = &dqblk->dd_diskdq;
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+ bool was_bigtime =
+ !!(ddq->d_type & XFS_DQTYPE_BIGTIME);
trace_xrep_disk_dquot(sc->mp, dqtype, id + i);
@@ -350,8 +358,21 @@ xrep_quota_block(
ddq->d_type = dqtype;
ddq->d_id = cpu_to_be32(id + i);
- if (xfs_has_bigtime(sc->mp) && ddq->d_id)
+ if (xfs_has_bigtime(sc->mp) && ddq->d_id) {
+ /*
+ * If something was broken with this dquot and it was a
+ * non-bigtime dquot, we'll grant everyone a fresh
+ * grace period by zeroing the timer field rather than
+ * try to interpret what might be garbage.
+ */
+ if (!was_bigtime) {
+ ddq->d_btimer = 0;
+ ddq->d_itimer = 0;
+ ddq->d_rtbtimer = 0;
+ }
+
ddq->d_type |= XFS_DQTYPE_BIGTIME;
+ }
xrep_quota_fix_timer(sc->mp, ddq, ddq->d_blk_softlimit,
ddq->d_bcount, &ddq->d_btimer,
diff --git a/fs/xfs/scrub/reap.c b/fs/xfs/scrub/reap.c
index 4ce345ae4565..0c827798ade6 100644
--- a/fs/xfs/scrub/reap.c
+++ b/fs/xfs/scrub/reap.c
@@ -905,12 +905,12 @@ xrep_reap_bmapi_iter(
/*
* Schedule removal of the mapping from the fork. We use
* deferred log intents in this function to control the exact
- * sequence of metadata updates.
+ * sequence of metadata updates. The BUI queues a RUI to
+ * remove the reverse mapping.
*/
xfs_bmap_unmap_extent(sc->tp, ip, whichfork, imap);
xfs_trans_mod_dquot_byino(sc->tp, ip, XFS_TRANS_DQ_BCOUNT,
-(int64_t)imap->br_blockcount);
- xfs_rmap_unmap_extent(sc->tp, ip, whichfork, imap);
return 0;
}
diff --git a/fs/xfs/scrub/rtsummary.c b/fs/xfs/scrub/rtsummary.c
index 746990ae0ec1..942f93b1e22e 100644
--- a/fs/xfs/scrub/rtsummary.c
+++ b/fs/xfs/scrub/rtsummary.c
@@ -201,15 +201,22 @@ xchk_rtsum_compute(
struct xfs_scrub *sc)
{
struct xfs_mount *mp = sc->mp;
+ struct xchk_rtsummary *rts = sc->buf;
unsigned long long rtbmp_blocks;
+ int error;
/* If the bitmap size doesn't match the computed size, bail. */
rtbmp_blocks = xfs_rtbitmap_blockcount(mp, mp->m_sb.sb_rextents);
if (XFS_FSB_TO_B(mp, rtbmp_blocks) != mp->m_rbmip->i_disk_size)
return -EFSCORRUPTED;
- return xfs_rtalloc_query_all(sc->mp, sc->tp, xchk_rtsum_record_free,
+ error = xfs_rtalloc_query_all(sc->mp, sc->tp, xchk_rtsum_record_free,
sc);
+ if (error)
+ return error;
+
+ rts->summary_computed = true;
+ return 0;
}
/* Compare the rtsummary file against the one we computed. */
diff --git a/fs/xfs/scrub/rtsummary.h b/fs/xfs/scrub/rtsummary.h
index e44b04cb6e2d..af1ab64c3f21 100644
--- a/fs/xfs/scrub/rtsummary.h
+++ b/fs/xfs/scrub/rtsummary.h
@@ -17,6 +17,7 @@ struct xchk_rtsummary {
xfs_filblks_t rsumblocks;
unsigned int rsumlevels;
unsigned int resblks;
+ bool summary_computed;
/* suminfo position of xfile as we write buffers to disk. */
xfs_rtsumoff_t prep_wordoff;
diff --git a/fs/xfs/scrub/rtsummary_repair.c b/fs/xfs/scrub/rtsummary_repair.c
index 7deeb948cb70..536f0b8e3bfd 100644
--- a/fs/xfs/scrub/rtsummary_repair.c
+++ b/fs/xfs/scrub/rtsummary_repair.c
@@ -110,7 +110,7 @@ xrep_rtsummary(
return -EOPNOTSUPP;
/* Walk away if we disagree on the size of the rt bitmap. */
- if (rts->rbmblocks != mp->m_sb.sb_rbmblocks)
+ if (rts->rbmblocks != mp->m_sb.sb_rbmblocks || !rts->summary_computed)
return 0;
/* Make sure any problems with the fork are fixed. */
diff --git a/fs/xfs/scrub/xfarray.c b/fs/xfs/scrub/xfarray.c
index ccf7cfb63bbc..3cf1ab618f2e 100644
--- a/fs/xfs/scrub/xfarray.c
+++ b/fs/xfs/scrub/xfarray.c
@@ -819,7 +819,7 @@ xfarray_sort_scan(
(idx < si->first_folio_idx || idx > si->last_folio_idx))
xfarray_sort_scan_done(si);
- /* Grab the first folio that backs this array element. */
+ /* Grab the folio that backs this array element. */
if (!si->folio) {
struct folio *folio;
loff_t next_pos;
@@ -830,6 +830,10 @@ xfarray_sort_scan(
return PTR_ERR(folio);
si->folio = folio;
+ /* No folio? Read into the scratchpad. */
+ if (!si->folio)
+ goto read_scratchpad;
+
si->first_folio_idx = xfarray_idx(si->array,
folio_pos(si->folio) + si->array->obj_size - 1);
@@ -838,14 +842,24 @@ xfarray_sort_scan(
if (xfarray_pos(si->array, si->last_folio_idx + 1) > next_pos)
si->last_folio_idx--;
+ /*
+ * If this folio still doesn't cover the desired element, it must
+ * cross a folio boundary. Read into the scratchpad.
+ */
+ if (idx < si->first_folio_idx || idx > si->last_folio_idx) {
+ xfarray_sort_scan_done(si);
+ goto read_scratchpad;
+ }
+
trace_xfarray_sort_scan(si, idx);
}
+read_scratchpad:
/*
- * If this folio still doesn't cover the desired element, it must cross
- * a folio boundary. Read into the scratchpad and we're done.
+ * If we don't have a folio mapping the entire array element, read into
+ * the scratchpad and we're done.
*/
- if (idx < si->first_folio_idx || idx > si->last_folio_idx) {
+ if (!si->folio) {
void *temp = xfarray_scratch(si->array);
error = xfile_load(si->array->xfile, temp, si->array->obj_size,
diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c
index 0b343776da8c..66c2f2fcec7b 100644
--- a/fs/xfs/xfs_discard.c
+++ b/fs/xfs/xfs_discard.c
@@ -160,6 +160,8 @@ struct xfs_trim_cur {
xfs_agblock_t end;
xfs_extlen_t minlen;
bool by_bno;
+ /* restart point (with count) for the by-length walk */
+ xfs_agblock_t restart_bno;
};
static int
@@ -205,14 +207,15 @@ xfs_trim_gather_extents(
error = xfs_alloc_lookup_le(cur, tcur->start, 0, &i);
if (!error && !i)
error = xfs_alloc_lookup_ge(cur, tcur->start, 0, &i);
- } else if (tcur->start == 0) {
- /* first time through a by-len starts with max length */
- cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_ge(cur, 0, tcur->count, &i);
} else {
- /* nth time through a by-len starts where we left off */
+ /*
+ * A by-len walk starts at the last record of the longest
+ * length (restart_bno is NULLAGBLOCK until the first batch
+ * ends) and then where the previous batch stopped.
+ */
cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_le(cur, tcur->start, tcur->count, &i);
+ error = xfs_alloc_lookup_le(cur, tcur->restart_bno,
+ tcur->count, &i);
}
if (error)
goto out_del_cursor;
@@ -242,9 +245,15 @@ xfs_trim_gather_extents(
if (--batch <= 0) {
/*
* Update the cursor to point at this extent so we
- * restart the next batch from this extent.
+ * restart the next batch from this extent. The
+ * by-length walk does not visit extents in block
+ * order, so it must keep the start of the range and
+ * restart from restart_bno instead.
*/
- tcur->start = fbno;
+ if (tcur->by_bno)
+ tcur->start = fbno;
+ else
+ tcur->restart_bno = fbno;
tcur->count = flen;
break;
}
@@ -347,6 +356,7 @@ xfs_trim_perag_extents(
.start = start,
.end = end,
.minlen = minlen,
+ .restart_bno = NULLAGBLOCK,
};
int error = 0;
diff --git a/fs/xfs/xfs_ioctl32.c b/fs/xfs/xfs_ioctl32.c
index b64785dc4354..c2e01fcb6015 100644
--- a/fs/xfs/xfs_ioctl32.c
+++ b/fs/xfs/xfs_ioctl32.c
@@ -123,8 +123,8 @@ xfs_ioctl32_bstat_copyin(
xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) ||
- get_user(bstat->bs_blocks, &bstat32->bs_size) ||
- get_user(bstat->bs_xflags, &bstat32->bs_size) ||
+ get_user(bstat->bs_blocks, &bstat32->bs_blocks) ||
+ get_user(bstat->bs_xflags, &bstat32->bs_xflags) ||
get_user(bstat->bs_extsize, &bstat32->bs_extsize) ||
get_user(bstat->bs_extents, &bstat32->bs_extents) ||
get_user(bstat->bs_gen, &bstat32->bs_gen) ||
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index 3483b3728b7b..afd655d8f2ae 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -258,6 +258,14 @@ xfs_iomap_write_direct(
ASSERT(count_fsb > 0);
+ /*
+ * xfs_bmapi_write() maps at most XFS_MAX_BMBT_EXTLEN blocks in one
+ * mapping, so don't reserve blocks for more. A pNFS layout range
+ * comes from a client, and its block count may not even fit the
+ * 32-bit reservation. The caller maps the rest with a later call.
+ */
+ count_fsb = min_t(xfs_filblks_t, count_fsb, XFS_MAX_BMBT_EXTLEN);
+
resaligned = xfs_aligned_fsb_count(offset_fsb, count_fsb,
xfs_get_extsz_hint(ip));
if (unlikely(XFS_IS_REALTIME_INODE(ip))) {
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index 23d16186e1a3..fa027574d6de 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -101,7 +101,15 @@ xfs_fs_map_update_inode(
ip->i_diflags |= XFS_DIFLAG_PREALLOC;
xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
- return xfs_trans_commit(tp);
+ error = xfs_trans_commit(tp);
+ if (error)
+ return error;
+
+ /*
+ * Force the log so that the blocks handed out to the client are
+ * guaranteed to be present even after a server crash.
+ */
+ return xfs_log_force_inode(ip);
}
/*
@@ -126,6 +134,7 @@ xfs_fs_map_blocks(
uint lock_flags;
int error = 0;
u64 seq;
+ bool update_inode = false;
if (xfs_is_shutdown(mp))
return -EIO;
@@ -159,10 +168,9 @@ xfs_fs_map_blocks(
if (!write)
limit = max(limit, round_up(i_size_read(inode),
inode->i_sb->s_blocksize));
- if (offset > limit)
+ if (offset < 0 || offset >= limit)
goto out_unlock;
- if (offset > limit - length)
- length = limit - offset;
+ length = min_t(u64, length, limit - offset);
error = filemap_write_and_wait(inode->i_mapping);
if (error)
@@ -194,20 +202,27 @@ xfs_fs_map_blocks(
end_fsb - offset_fsb, 0, &imap, &seq);
if (error)
goto out_unlock;
+ update_inode = true;
+ } else {
+ xfs_iunlock(ip, lock_flags);
/*
- * Ensure the next transaction is committed synchronously so
- * that the blocks allocated and handed out to the client are
- * guaranteed to be present even after a server crash.
+ * Blocks past EOF that were allocated for buffered writes, such
+ * as speculative preallocation, can be freed by
+ * xfs_free_eofblocks() without breaking the layout unless the
+ * inode is marked as preallocated. Mark it before handing them
+ * out for writing, as for blocks allocated above.
*/
+ if (write && !(ip->i_diflags & XFS_DIFLAG_PREALLOC) &&
+ imap.br_startoff + imap.br_blockcount >
+ XFS_B_TO_FSB(mp, XFS_ISIZE(ip)))
+ update_inode = true;
+ }
+
+ if (update_inode) {
error = xfs_fs_map_update_inode(ip);
- if (!error)
- error = xfs_log_force_inode(ip);
if (error)
goto out_unlock;
-
- } else {
- xfs_iunlock(ip, lock_flags);
}
xfs_iunlock(ip, XFS_IOLOCK_EXCL);
@@ -286,9 +301,7 @@ xfs_fs_commit_blocks(
if (start > size)
continue;
- end = start + maps[i].length;
- if (end > size)
- end = size;
+ end = start + min_t(u64, maps[i].length, size - start);
length = end - start;
if (!length)
diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h
index 31767c65be20..5fe0aa7a6c5a 100644
--- a/include/acpi/cppc_acpi.h
+++ b/include/acpi/cppc_acpi.h
@@ -160,6 +160,7 @@ extern int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val);
extern int cppc_get_epp_perf(int cpunum, u64 *epp_perf);
extern int cppc_set_epp_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls, bool enable);
extern int cppc_get_auto_sel(int cpu, bool *enable);
+bool cppc_auto_sel_is_immutable(int cpu);
extern int cppc_set_auto_sel(int cpu, bool enable);
extern int amd_get_highest_perf(unsigned int cpu, u32 *highest_perf);
extern int amd_get_boost_ratio_numerator(unsigned int cpu, u64 *numerator);
@@ -233,6 +234,11 @@ static inline int cppc_get_auto_sel(int cpu, bool *enable)
{
return -EOPNOTSUPP;
}
+static inline bool cppc_auto_sel_is_immutable(int cpu)
+{
+ return false;
+}
+
static inline int cppc_set_auto_sel(int cpu, bool enable)
{
return -EOPNOTSUPP;
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 253dee8077b1..c046a09b0b48 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -375,8 +375,7 @@ struct mmu_gather {
void tlb_flush_mmu(struct mmu_gather *tlb);
static inline void __tlb_adjust_range(struct mmu_gather *tlb,
- unsigned long address,
- unsigned int range_size)
+ unsigned long address, unsigned long range_size)
{
tlb->start = min(tlb->start, address);
tlb->end = max(tlb->end, address + range_size);
diff --git a/include/drm/drm_panel.h b/include/drm/drm_panel.h
index 10015891b056..daaa259ef898 100644
--- a/include/drm/drm_panel.h
+++ b/include/drm/drm_panel.h
@@ -269,6 +269,28 @@ struct drm_panel {
bool enabled;
};
+void *__devm_drm_panel_alloc(struct device *dev, size_t size, size_t offset,
+ const struct drm_panel_funcs *funcs,
+ int connector_type);
+
+/**
+ * devm_drm_panel_alloc - Allocate and initialize a refcounted panel.
+ *
+ * @dev: struct device of the panel device
+ * @type: the type of the struct which contains struct &drm_panel
+ * @member: the name of the &drm_panel within @type
+ * @funcs: callbacks for this panel
+ * @connector_type: the connector type (DRM_MODE_CONNECTOR_*) corresponding to
+ * the panel interface
+ *
+ * Returns:
+ * Pointer to container structure embedding the panel, ERR_PTR on failure.
+ */
+#define devm_drm_panel_alloc(dev, type, member, funcs, connector_type) \
+ ((type *)__devm_drm_panel_alloc(dev, sizeof(type), \
+ offsetof(type, member), funcs, \
+ connector_type))
+
void drm_panel_init(struct drm_panel *panel, struct device *dev,
const struct drm_panel_funcs *funcs,
int connector_type);
diff --git a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
index ba2d70f776a6..764d837d6162 100644
--- a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
+++ b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
@@ -248,6 +248,7 @@
#define ADC_GATE 233
#define EADC_DIV 234
#define EADC_GATE 235
-#define CLK_MAX_IDX 236
+#define PCLK3_DIV4096 236
+#define PCLK4_DIV4096 237
#endif /* __DT_BINDINGS_CLOCK_NUVOTON_MA35D1_CLK_H */
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index c819c5d16613..c468e3d177d7 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -152,4 +152,19 @@ static inline bool has_cntpoff(void)
return (has_vhe() && cpus_have_final_cap(ARM64_HAS_ECV_CNTPOFF));
}
+/*
+ * CVAL to program when the hardware won't apply the timer's offset, so that
+ * its compare against the raw counter matches the guest's at 'now'. An
+ * expired timer stays expired; one past the wrap never fires.
+ */
+static inline u64 timer_apply_offset(u64 cval, u64 offset, u64 now)
+{
+ u64 hw = cval + offset;
+
+ if (now - offset >= cval)
+ return 0;
+
+ return hw > now ? hw : U64_MAX;
+}
+
#endif
diff --git a/include/linux/assoc_array_priv.h b/include/linux/assoc_array_priv.h
index dca733ef6750..37a9f76e1505 100644
--- a/include/linux/assoc_array_priv.h
+++ b/include/linux/assoc_array_priv.h
@@ -80,7 +80,7 @@ struct assoc_array_edit {
struct assoc_array_ptr **leaf_p;
struct assoc_array_ptr *dead_leaf;
struct assoc_array_ptr *new_meta[3];
- struct assoc_array_ptr *excised_meta[1];
+ struct assoc_array_ptr *excised_meta[2];
struct assoc_array_ptr *excised_subtree;
struct assoc_array_ptr **set_backpointers[ASSOC_ARRAY_FAN_OUT];
struct assoc_array_ptr *set_backpointers_to;
diff --git a/include/linux/bio.h b/include/linux/bio.h
index d4f58a2d15b6..104a68749ab1 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -377,6 +377,21 @@ static inline struct bio *bio_alloc(struct block_device *bdev,
void submit_bio(struct bio *bio);
+/**
+ * bio_in_atomic - check if the current context is unsafe for bio completion
+ *
+ * Return: %true in atomic contexts (e.g. hard/soft IRQ, preempt-disabled);
+ * %false when a bio can be safely completed in the current context.
+ */
+static inline bool bio_in_atomic(void)
+{
+ if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth())
+ return true;
+ if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
+ return true;
+ return !preemptible();
+}
+
extern void bio_endio(struct bio *);
static inline void bio_io_error(struct bio *bio)
diff --git a/include/linux/bitfield.h b/include/linux/bitfield.h
index 20978c60b565..c60554acc16d 100644
--- a/include/linux/bitfield.h
+++ b/include/linux/bitfield.h
@@ -8,6 +8,7 @@
#define _LINUX_BITFIELD_H
#include <linux/build_bug.h>
+#include <linux/typecheck.h>
#include <asm/byteorder.h>
/*
@@ -38,8 +39,7 @@
* FIELD_PREP(REG_FIELD_D, 0x40);
*
* Modify:
- * reg &= ~REG_FIELD_C;
- * reg |= FIELD_PREP(REG_FIELD_C, c);
+ * FIELD_MODIFY(REG_FIELD_C, ®, c);
*/
#define __bf_shf __builtin_ctzll
@@ -156,6 +156,23 @@
(typeof(_mask))(((_reg) & (_mask)) >> __bf_shf(_mask)); \
})
+/**
+ * FIELD_MODIFY() - modify a bitfield element
+ * @_mask: shifted mask defining the field's length and position
+ * @_reg_p: pointer to the memory that should be updated
+ * @_val: value to store in the bitfield
+ *
+ * FIELD_MODIFY() modifies the set of bits in @_reg_p specified by @_mask,
+ * by replacing them with the bitfield value passed in as @_val.
+ */
+#define FIELD_MODIFY(_mask, _reg_p, _val) \
+ ({ \
+ typecheck_pointer(_reg_p); \
+ __BF_FIELD_CHECK(_mask, *(_reg_p), _val, "FIELD_MODIFY: "); \
+ *(_reg_p) &= ~(_mask); \
+ *(_reg_p) |= (((typeof(_mask))(_val) << __bf_shf(_mask)) & (_mask)); \
+ })
+
extern void __compiletime_error("value doesn't fit into mask")
__field_overflow(void);
extern void __compiletime_error("bad bitfield mask")
diff --git a/include/linux/can/can-ml.h b/include/linux/can/can-ml.h
index 8afa92d15a66..f10444c6115f 100644
--- a/include/linux/can/can-ml.h
+++ b/include/linux/can/can-ml.h
@@ -45,6 +45,7 @@
#include <linux/can.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/spinlock.h>
#define CAN_SFF_RCV_ARRAY_SZ (1 << CAN_SFF_ID_BITS)
#define CAN_EFF_RCV_HASH_BITS 10
@@ -53,6 +54,7 @@
enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX };
struct can_dev_rcv_lists {
+ spinlock_t lock; /* protects the receive lists and entries */
struct hlist_head rx[RX_MAX];
struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ];
struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ];
@@ -74,6 +76,7 @@ static inline struct can_ml_priv *can_get_ml_priv(struct net_device *dev)
static inline void can_set_ml_priv(struct net_device *dev,
struct can_ml_priv *ml_priv)
{
+ spin_lock_init(&ml_priv->dev_rcv_lists.lock);
netdev_set_ml_priv(dev, ml_priv, ML_PRIV_CAN);
}
diff --git a/include/linux/damon.h b/include/linux/damon.h
index 4688a10eca89..b05c626f6479 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -641,6 +641,12 @@ struct damon_ctx {
struct list_head adaptive_targets;
struct list_head schemes;
+/* private: */
+ /*
+ * number of regions after last merge. Read kdamond_split_regions()
+ * for more details.
+ */
+ unsigned long last_nr_regions;
};
static inline struct damon_region *damon_next_region(struct damon_region *r)
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 32cbaaaccfe5..96a201151a80 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -121,10 +121,9 @@ void hugepage_put_subpool(struct hugepage_subpool *spool);
void hugetlb_dup_vma_private(struct vm_area_struct *vma);
void clear_vma_resv_huge_pages(struct vm_area_struct *vma);
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len);
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len);
int copy_hugetlb_page_range(struct mm_struct *, struct mm_struct *,
struct vm_area_struct *, struct vm_area_struct *);
void unmap_hugepage_range(struct vm_area_struct *,
@@ -334,11 +333,9 @@ static inline int copy_hugetlb_page_range(struct mm_struct *dst,
return 0;
}
-static inline int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr,
- unsigned long new_addr,
- unsigned long len)
+static inline unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
BUG();
return 0;
@@ -690,6 +687,7 @@ struct huge_bootmem_page {
int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list);
void wait_for_freed_hugetlb_folios(void);
+int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long end_pfn);
struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
unsigned long addr, bool cow_from_owner);
struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid,
@@ -1081,6 +1079,12 @@ static inline void wait_for_freed_hugetlb_folios(void)
{
}
+static inline int replace_free_hugepage_folios(unsigned long start_pfn,
+ unsigned long end_pfn)
+{
+ return 0;
+}
+
static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
unsigned long addr,
bool cow_from_owner)
diff --git a/include/linux/libata.h b/include/linux/libata.h
index e5e098279803..c5ae2d98eb22 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -947,6 +947,13 @@ struct ata_port {
*/
#define ATA_OP_NULL (void *)(unsigned long)(-ENOENT)
+struct ata_reset_operations {
+ ata_prereset_fn_t prereset;
+ ata_reset_fn_t softreset;
+ ata_reset_fn_t hardreset;
+ ata_postreset_fn_t postreset;
+};
+
struct ata_port_operations {
/*
* Command execution
@@ -973,14 +980,8 @@ struct ata_port_operations {
void (*freeze)(struct ata_port *ap);
void (*thaw)(struct ata_port *ap);
- ata_prereset_fn_t prereset;
- ata_reset_fn_t softreset;
- ata_reset_fn_t hardreset;
- ata_postreset_fn_t postreset;
- ata_prereset_fn_t pmp_prereset;
- ata_reset_fn_t pmp_softreset;
- ata_reset_fn_t pmp_hardreset;
- ata_postreset_fn_t pmp_postreset;
+ struct ata_reset_operations reset;
+ struct ata_reset_operations pmp_reset;
void (*error_handler)(struct ata_port *ap);
void (*lost_interrupt)(struct ata_port *ap);
void (*post_internal_cmd)(struct ata_queued_cmd *qc);
@@ -1418,9 +1419,6 @@ extern void ata_eh_thaw_port(struct ata_port *ap);
extern void ata_eh_qc_complete(struct ata_queued_cmd *qc);
extern void ata_eh_qc_retry(struct ata_queued_cmd *qc);
-extern void ata_do_eh(struct ata_port *ap, ata_prereset_fn_t prereset,
- ata_reset_fn_t softreset, ata_reset_fn_t hardreset,
- ata_postreset_fn_t postreset);
extern void ata_std_error_handler(struct ata_port *ap);
extern void ata_std_sched_eh(struct ata_port *ap);
extern void ata_std_end_eh(struct ata_port *ap);
@@ -2153,6 +2151,12 @@ static inline u8 ata_wait_idle(struct ata_port *ap)
return status;
}
+#else /* CONFIG_ATA_SFF */
+static inline int sata_sff_hardreset(struct ata_link *link, unsigned int *class,
+ unsigned long deadline)
+{
+ return -EOPNOTSUPP;
+}
#endif /* CONFIG_ATA_SFF */
#endif /* __LINUX_LIBATA_H__ */
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 8dab6c755e10..2aba72d1de3a 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1884,6 +1884,16 @@ static inline void count_objcg_events(struct obj_cgroup *objcg,
#endif /* CONFIG_MEMCG */
+#if defined(CONFIG_MEMCG) && defined(CONFIG_LRU_GEN)
+void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg);
+#else
+static inline void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg)
+{
+}
+#endif
+
#if defined(CONFIG_MEMCG) && defined(CONFIG_ZSWAP)
bool obj_cgroup_may_zswap(struct obj_cgroup *objcg);
void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size);
diff --git a/include/linux/memfd.h b/include/linux/memfd.h
index 246daadbfde8..bb07ba299290 100644
--- a/include/linux/memfd.h
+++ b/include/linux/memfd.h
@@ -2,8 +2,17 @@
#ifndef __LINUX_MEMFD_H
#define __LINUX_MEMFD_H
+#include <linux/fcntl.h>
#include <linux/file.h>
+static inline int memfd_check_seals_mode(unsigned int seals, umode_t old_mode,
+ umode_t new_mode)
+{
+ if ((seals & F_SEAL_EXEC) && ((old_mode ^ new_mode) & 0111))
+ return -EPERM;
+ return 0;
+}
+
#ifdef CONFIG_MEMFD_CREATE
extern long memfd_fcntl(struct file *file, unsigned int cmd, unsigned int arg);
struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 1728dc335608..f05962963e28 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2058,11 +2058,13 @@ static inline long folio_nr_pages(const struct folio *folio)
/* Only hugetlbfs can allocate folios larger than MAX_ORDER */
#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE
-#define MAX_FOLIO_NR_PAGES (1UL << PUD_ORDER)
+#define MAX_FOLIO_ORDER PUD_ORDER
#else
-#define MAX_FOLIO_NR_PAGES MAX_ORDER_NR_PAGES
+#define MAX_FOLIO_ORDER MAX_PAGE_ORDER
#endif
+#define MAX_FOLIO_NR_PAGES (1UL << MAX_FOLIO_ORDER)
+
/*
* compound_nr() returns the number of pages in this potentially compound
* page. compound_nr() can be called on a tail page, and is defined to
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index d39ebb10caeb..fe486d52d48e 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -444,7 +444,8 @@ mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
/*
* This version of mmu_notifier_invalidate_range_start() avoids blocking, but it
* can return an error if a notifier can't proceed without blocking, in which
- * case you're not allowed to modify PTEs in the specified range.
+ * case you're not allowed to modify PTEs in the specified range or call
+ * mmu_notifier_invalidate_range_end().
*
* This is mainly intended for OOM handling.
*/
diff --git a/include/linux/panic.h b/include/linux/panic.h
index f0b779d2915e..99c53d2792b1 100644
--- a/include/linux/panic.h
+++ b/include/linux/panic.h
@@ -3,6 +3,7 @@
#define _LINUX_PANIC_H
#include <linux/compiler_attributes.h>
+#include <linux/stdarg.h>
#include <linux/types.h>
struct pt_regs;
@@ -10,7 +11,10 @@ struct pt_regs;
extern long (*panic_blink)(int state);
__printf(1, 2)
void panic(const char *fmt, ...) __noreturn __cold;
-void nmi_panic(struct pt_regs *regs, const char *msg);
+__printf(1, 0)
+void vpanic(const char *fmt, va_list args) __noreturn __cold;
+__printf(2, 3)
+void nmi_panic(struct pt_regs *regs, const char *fmt, ...);
void check_panic_on_warn(const char *origin);
extern void oops_enter(void);
extern void oops_exit(void);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index c65de9e8aa8a..63c24b5bdce5 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -602,6 +602,8 @@ struct pci_host_bridge {
u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
int (*map_irq)(const struct pci_dev *, u8, u8);
void (*release_fn)(struct pci_host_bridge *);
+ int (*enable_device)(struct pci_host_bridge *bridge, struct pci_dev *dev);
+ void (*disable_device)(struct pci_host_bridge *bridge, struct pci_dev *dev);
void *release_data;
unsigned int ignore_reset_delay:1; /* For entire hierarchy */
unsigned int no_ext_tags:1; /* No Extended Tags */
diff --git a/include/linux/power_supply.h b/include/linux/power_supply.h
index b03991a29a65..1e059ff7a971 100644
--- a/include/linux/power_supply.h
+++ b/include/linux/power_supply.h
@@ -236,6 +236,8 @@ struct power_supply_config {
char **supplied_to;
size_t num_supplicants;
+
+ bool no_wakeup_source;
};
/* Description of power supply */
@@ -287,7 +289,6 @@ struct power_supply {
char **supplied_from;
size_t num_supplies;
- struct device_node *of_node;
/* Driver private data */
void *drv_data;
@@ -890,6 +891,7 @@ extern int power_supply_powers(struct power_supply *psy, struct device *dev);
extern void *power_supply_get_drvdata(struct power_supply *psy);
extern int power_supply_for_each_device(void *data, int (*fn)(struct device *dev, void *data));
+extern int power_supply_for_each_psy(void *data, int (*fn)(struct power_supply *psy, void *data));
static inline bool power_supply_is_amp_property(enum power_supply_property psp)
{
diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index 835bbb2d1f88..4958b16d187e 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -18,6 +18,7 @@
#include <linux/mutex.h>
#include <linux/rcupdate.h>
+#include <linux/rculist.h>
#include <linux/workqueue.h>
#include <linux/rcu_segcblist.h>
@@ -58,6 +59,41 @@ int __srcu_read_lock(struct srcu_struct *ssp) __acquires(ssp);
void __srcu_read_unlock(struct srcu_struct *ssp, int idx) __releases(ssp);
void synchronize_srcu(struct srcu_struct *ssp);
+/**
+ * list_splice_init_srcu - splice an SRCU-protected list into an existing list
+ * @list: the SRCU-protected list to splice
+ * @head: the place in the existing list to splice the first list into
+ * @ssp: the srcu_struct that readers of @list use
+ *
+ * Like list_splice_init_rcu(), but wait for the SRCU readers of @list
+ * instead of RCU ones, so that none of them can follow a next pointer
+ * onto @head.
+ *
+ * Note that this function blocks, and that the caller must prevent any
+ * other updates to @list.
+ */
+static inline void list_splice_init_srcu(struct list_head *list,
+ struct list_head *head,
+ struct srcu_struct *ssp)
+{
+ struct list_head *first = list->next;
+ struct list_head *last = list->prev;
+ struct list_head *next = head->next;
+
+ if (list_empty(list))
+ return;
+
+ INIT_LIST_HEAD_RCU(list);
+ synchronize_srcu(ssp);
+ ASSERT_EXCLUSIVE_ACCESS(*first);
+ ASSERT_EXCLUSIVE_ACCESS(*last);
+
+ last->next = next;
+ rcu_assign_pointer(list_next_rcu(head), first);
+ first->prev = head;
+ next->prev = last;
+}
+
#define SRCU_GET_STATE_COMPLETED 0x1
/**
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index bd6ac060450c..acdb31039c35 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -195,7 +195,8 @@ struct rpc_xprt *rpc_task_get_xprt(struct rpc_clnt *clnt,
int rpcb_create_local(struct net *);
void rpcb_put_local(struct net *);
-int rpcb_register(struct net *, u32, u32, int, unsigned short);
+int rpcb_register(struct net *net, u32 prog, u32 vers, int prot,
+ unsigned short port);
int rpcb_v4_register(struct net *net, const u32 program,
const u32 version,
const struct sockaddr *address,
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 79f3c5c1d0e9..b7a61aafc2dd 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -79,6 +79,7 @@ struct svc_serv {
unsigned int sv_max_payload; /* datagram payload size */
unsigned int sv_max_mesg; /* max_payload + 1 page for overheads */
unsigned int sv_xdrsize; /* XDR buffer size */
+ atomic_t sv_rpcb_failures; /* unanswered rpcbind calls */
struct list_head sv_permsocks; /* all permanent sockets */
struct list_head sv_tempsocks; /* all temporary sockets */
int sv_tmpcnt; /* count of temporary "valid" sockets */
@@ -451,6 +452,7 @@ int sunrpc_set_pool_mode(const char *val);
int sunrpc_get_pool_mode(char *val, size_t size);
void svc_rpcb_cleanup(struct svc_serv *serv, struct net *net);
int svc_bind(struct svc_serv *serv, struct net *net);
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv);
struct svc_serv *svc_create(struct svc_program *, unsigned int,
int (*threadfn)(void *data));
bool svc_rqst_replace_page(struct svc_rqst *rqstp,
@@ -466,8 +468,9 @@ int svc_set_num_threads(struct svc_serv *, struct svc_pool *, int);
int svc_pool_stats_open(struct svc_info *si, struct file *file);
void svc_process(struct svc_rqst *rqstp);
void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp);
-int svc_register(const struct svc_serv *, struct net *, const int,
- const unsigned short, const unsigned short);
+int svc_register(struct svc_serv *serv, struct net *net,
+ const int family, const unsigned short proto,
+ const unsigned short port);
void svc_wake_up(struct svc_serv *);
void svc_reserve(struct svc_rqst *rqstp, int space);
diff --git a/include/linux/usb/quirks.h b/include/linux/usb/quirks.h
index b3cc7beab4a3..a4043b33c2c2 100644
--- a/include/linux/usb/quirks.h
+++ b/include/linux/usb/quirks.h
@@ -81,4 +81,7 @@
/* Device claims zero configurations, forcing to 1 */
#define USB_QUIRK_FORCE_ONE_CONFIG BIT(18)
+/* Use a 255 bytes config descriptor request mirroring windows behavior */
+#define USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE BIT(19)
+
#endif /* __LINUX_USB_QUIRKS_H */
diff --git a/include/net/espintcp.h b/include/net/espintcp.h
index c70efd704b6d..7cad3c6822df 100644
--- a/include/net/espintcp.h
+++ b/include/net/espintcp.h
@@ -34,7 +34,8 @@ static inline struct espintcp_ctx *espintcp_getctx(const struct sock *sk)
{
const struct inet_connection_sock *icsk = inet_csk(sk);
- /* RCU is only needed for diag */
- return (__force void *)icsk->icsk_ulp_data;
+ /* pairs with smp_wmb() in espintcp_init_sk() */
+ smp_rmb();
+ return (__force void *)READ_ONCE(icsk->icsk_ulp_data);
}
#endif
diff --git a/include/net/ip.h b/include/net/ip.h
index 97c60b300270..f00719e1863f 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -515,31 +515,6 @@ static inline unsigned int ip_dst_mtu_maybe_forward(const struct dst_entry *dst,
return res;
}
-/* Configured/administrative MTU of a route, for advertising the TCP MSS.
- *
- * Unlike ip_dst_mtu_maybe_forward(), this deliberately ignores the
- * ICMP-learned path MTU (rt->rt_pmtu). The advertised MSS bounds what the
- * peer may send to us and must reflect our receive capability (the device or
- * route-configured MTU), not a path MTU learned on the reverse (send)
- * direction, which may not apply to the peer->us path and outlives the fnhe
- * for the whole connection. See RFC 2923 section 2.3 and the comment above
- * tcp_advertise_mss().
- */
-static inline unsigned int ip_dst_mtu_configured(const struct dst_entry *dst)
-{
- unsigned int mtu, res;
-
- rcu_read_lock();
- mtu = dst_metric_raw(dst, RTAX_MTU);
- if (!mtu)
- mtu = READ_ONCE(dst_dev_rcu(dst)->mtu);
- mtu = min_t(unsigned int, mtu, IP_MAX_MTU);
- res = mtu - lwtunnel_headroom(dst->lwtstate, mtu);
- rcu_read_unlock();
-
- return res;
-}
-
static inline unsigned int ip_skb_dst_mtu(struct sock *sk,
const struct sk_buff *skb)
{
diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index 8be44dc5b5b7..ad35c7512f12 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -354,43 +354,6 @@ static inline unsigned int ip6_dst_mtu_maybe_forward(const struct dst_entry *dst
return mtu - lwtunnel_headroom(dst->lwtstate, mtu);
}
-/* Configured/administrative MTU of a route, for advertising the TCP MSS.
- *
- * Unlike ip6_dst_mtu_maybe_forward(), this ignores any ICMPv6-learned path
- * MTU (which is kept on the RTF_CACHE exception route) and returns the MTU of
- * the underlying route (fib6_pmtu) or the egress device. The advertised MSS
- * bounds what the peer may send to us and must reflect our receive
- * capability, not a path MTU learned on the reverse (send) direction. See
- * RFC 2923 section 2.3 and the comment above tcp_advertise_mss().
- */
-static inline unsigned int ip6_dst_mtu_configured(const struct dst_entry *dst)
-{
- const struct rt6_info *rt = dst_rt6_info(dst);
- const struct fib6_info *from;
- struct inet6_dev *idev;
- unsigned int mtu = 0;
-
- rcu_read_lock();
- /* IPv6 keeps the learned PMTU and the configured MTU in the same
- * RTAX_MTU slot: the learned value sits on this (possibly RTF_CACHE)
- * dst, the configured one on the underlying route. Reach the latter
- * via ->from (fib6_pmtu), populated by ip6_route_info_create().
- */
- from = rcu_dereference(rt->from);
- if (from)
- mtu = from->fib6_pmtu;
- if (!mtu) {
- mtu = IPV6_MIN_MTU;
- idev = __in6_dev_get(dst_dev_rcu(dst));
- if (idev)
- mtu = max_t(unsigned int, mtu, READ_ONCE(idev->cnf.mtu6));
- }
- rcu_read_unlock();
-
- mtu = min_t(unsigned int, mtu, IP6_MAX_MTU);
- return mtu - lwtunnel_headroom(dst->lwtstate, mtu);
-}
-
u32 ip6_mtu_from_fib6(const struct fib6_result *res,
const struct in6_addr *daddr,
const struct in6_addr *saddr);
diff --git a/include/net/netns/can.h b/include/net/netns/can.h
index 48b79f7e6236..54f23681e0d4 100644
--- a/include/net/netns/can.h
+++ b/include/net/netns/can.h
@@ -29,7 +29,7 @@ struct netns_can {
/* receive filters subscribed for 'all' CAN devices */
struct can_dev_rcv_lists *rx_alldev_list;
- spinlock_t rcvlists_lock;
+ spinlock_t rcvlists_lock; /* protects the rcv_entries(_max) stats */
struct timer_list stattimer; /* timer for statistics update */
struct can_pkg_stats *pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats;
diff --git a/include/net/netrom.h b/include/net/netrom.h
index 2b4685d52bd8..3f6fd2f89434 100644
--- a/include/net/netrom.h
+++ b/include/net/netrom.h
@@ -87,6 +87,19 @@ struct nr_sock {
#define nr_sk(sk) ((struct nr_sock *)(sk))
+/*
+ * Window negotiation, for a CONNREQ received or a CONNACK: use the smaller
+ * of our window and the peer's. A window of 0 would never let a frame
+ * out: take it as 1.
+ */
+static inline void nr_negotiate_window(struct nr_sock *nr, unsigned char window)
+{
+ if (window == 0)
+ window = 1;
+ if (window < nr->window)
+ nr->window = window;
+}
+
struct nr_neigh {
struct hlist_node neigh_node;
ax25_address callsign;
@@ -189,7 +202,7 @@ void nr_loopback_clear(void);
int nr_loopback_queue(struct sk_buff *);
/* nr_out.c */
-void nr_output(struct sock *, struct sk_buff *);
+int nr_output(struct sock *sk, struct sk_buff *skb);
void nr_send_nak_frame(struct sock *);
void nr_kick(struct sock *);
void nr_transmit_buffer(struct sock *, struct sk_buff *);
diff --git a/include/net/rose.h b/include/net/rose.h
index 2fa0331e36f0..30b0809dd0a6 100644
--- a/include/net/rose.h
+++ b/include/net/rose.h
@@ -231,6 +231,8 @@ int rose_rcv_frame(struct sock *sk, struct sk_buff *skb);
int rose_backlog_rcv(struct sock *sk, struct sk_buff *skb);
/* rose_link.c */
+void rose_neigh_init_timers(struct rose_neigh *neigh);
+bool rose_neigh_clear_ax25(struct rose_neigh *neigh, ax25_cb *ax25);
void rose_start_ftimer(struct rose_neigh *);
void rose_stop_ftimer(struct rose_neigh *);
void rose_stop_t0timer(struct rose_neigh *);
@@ -266,6 +268,7 @@ void rose_link_device_down(struct net_device *);
struct net_device *rose_dev_first(void);
struct net_device *rose_dev_get(rose_address *);
struct rose_route *rose_route_free_lci(unsigned int, struct rose_neigh *);
+unsigned int rose_assign_lci(struct sock *sk);
struct rose_neigh *rose_get_neigh(rose_address *, unsigned char *,
unsigned char *, int);
int rose_rt_ioctl(unsigned int, void __user *);
diff --git a/include/net/sctp/sm.h b/include/net/sctp/sm.h
index 64c42bd56bb2..84e71381e19b 100644
--- a/include/net/sctp/sm.h
+++ b/include/net/sctp/sm.h
@@ -253,6 +253,7 @@ struct sctp_chunk *sctp_make_fwdtsn(const struct sctp_association *asoc,
struct sctp_fwdtsn_skip *skiplist);
struct sctp_chunk *sctp_make_auth(const struct sctp_association *asoc,
__u16 key_id);
+void sctp_control_set_owner_w(struct sctp_chunk *chunk);
struct sctp_chunk *sctp_make_strreset_req(const struct sctp_association *asoc,
__u16 stream_num, __be16 *stream_list,
bool out, bool in);
diff --git a/include/sound/hwdep.h b/include/sound/hwdep.h
index b0da633184cd..7357727f0660 100644
--- a/include/sound/hwdep.h
+++ b/include/sound/hwdep.h
@@ -57,6 +57,7 @@ struct snd_hwdep {
struct mutex open_mutex;
int used; /* reference counter */
+ struct mutex dsp_mutex; /* serializes DSP load requests */
unsigned int dsp_loaded; /* bit fields of loaded dsp indices */
unsigned int exclusive:1; /* exclusive access mode */
};
diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 8c4d791507f1..5e29744e4503 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -20,10 +20,10 @@
/* AFS errors */ \
EM(afs_abort_general_error, "afs-error") \
EM(afs_abort_interrupted, "afs-intr") \
- EM(afs_abort_oom, "afs-oom") \
EM(afs_abort_op_not_supported, "afs-op-notsupp") \
EM(afs_abort_probeuuid_negative, "afs-probeuuid-neg") \
EM(afs_abort_send_data_error, "afs-send-data") \
+ EM(afs_abort_send_error, "afs-send-error") \
EM(afs_abort_unmarshal_error, "afs-unmarshal") \
/* rxperf errors */ \
EM(rxperf_abort_general_error, "rxperf-error") \
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index 41048271a066..e245fa295662 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -621,6 +621,12 @@ enum iommufd_hwpt_get_dirty_bitmap_flags {
* Walk the IOMMU pagetables for a given IOVA range to return a bitmap
* with the dirty IOVAs. In doing so it will also by default clear any
* dirty bit metadata set in the IOPTE.
+ *
+ * The IOVA range given must fully enclose any IOPTE. If the range starts or
+ * ends inside a dirty IOPTE then the ioctl will fail. This avoids partially
+ * reporting a dirty range.
+ *
+ * On failure the dirty state becomes unknown and the userspace must restart.
*/
struct iommu_hwpt_get_dirty_bitmap {
__u32 size;
diff --git a/io_uring/cancel.c b/io_uring/cancel.c
index 13d5d4f2924f..727166a2ba3e 100644
--- a/io_uring/cancel.c
+++ b/io_uring/cancel.c
@@ -189,7 +189,8 @@ static int __io_async_cancel(struct io_cancel_data *cd,
break;
if (!all)
return ret;
- nr++;
+ /* poll, futex and waitid return how many they cancelled */
+ nr += max(ret, 1);
} while (1);
/* slow path, try all io-wq's */
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 52759ee3bd6c..09551e3907f9 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -3282,7 +3282,16 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
}
if (loop) {
+ /*
+ * A request that runs in io-wq and doesn't end makes
+ * each round come here. Don't hold sqd->lock for that
+ * long either.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
cond_resched();
+ if (sqd)
+ mutex_lock(&sqd->lock);
continue;
}
@@ -3300,8 +3309,20 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
* avoids a race where a completion comes in before we did
* prepare_to_wait().
*/
- if (inflight == tctx_inflight(tctx, !cancel_all))
+ if (inflight == tctx_inflight(tctx, !cancel_all)) {
+ /*
+ * A zero-copy notification can't be canceled, and a
+ * peer that doesn't read keeps it pending. Don't hold
+ * sqd->lock while waiting for it, or whoever wants the
+ * lock sleeps as long. No ring is used across this,
+ * the next round walks ->ctx_list anew.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
schedule();
+ if (sqd)
+ mutex_lock(&sqd->lock);
+ }
end_wait:
finish_wait(&tctx->wait, &wait);
} while (1);
diff --git a/io_uring/poll.c b/io_uring/poll.c
index 31454c27072b..c1a9f697a023 100644
--- a/io_uring/poll.c
+++ b/io_uring/poll.c
@@ -169,12 +169,24 @@ static void io_poll_tw_hash_eject(struct io_kiocb *req, struct io_tw_state *ts)
}
}
+#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL)
+
+static __poll_t io_poll_unmask(__poll_t events)
+{
+ /*
+ * A peer that stopped sending is an event for a reader, but a write
+ * only poll must stay armed until the file is writable again.
+ */
+ if (events & (EPOLLIN | EPOLLRDNORM | EPOLLRDBAND))
+ return IO_POLL_UNMASK | EPOLLRDHUP;
+ return IO_POLL_UNMASK;
+}
+
static void io_init_poll_iocb(struct io_poll *poll, __poll_t events)
{
poll->head = NULL;
-#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL|EPOLLRDHUP)
/* mask in events that we always want/need */
- poll->events = events | IO_POLL_UNMASK;
+ poll->events = events | io_poll_unmask(events);
INIT_LIST_HEAD(&poll->wait.entry);
init_waitqueue_func_entry(&poll->wait, io_poll_wake);
}
@@ -745,7 +757,7 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
const struct io_issue_def *def = &io_issue_defs[req->opcode];
struct async_poll *apoll;
struct io_poll_table ipt;
- __poll_t mask = POLLPRI | POLLERR | EPOLLET;
+ __poll_t mask = POLLERR | EPOLLET;
int ret;
/*
@@ -762,7 +774,8 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
mask |= EPOLLONESHOT;
if (def->pollin) {
- mask |= EPOLLIN | EPOLLRDNORM;
+ /* urgent data is an event for a reader only */
+ mask |= EPOLLIN | EPOLLRDNORM | EPOLLPRI;
/* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */
if (req->flags & REQ_F_CLEAR_POLLIN)
@@ -892,14 +905,52 @@ static int io_poll_disarm(struct io_kiocb *req)
return 0;
}
+/*
+ * Cancel every request matching @cd in one pass. Cancelled requests stay
+ * hashed until their task work runs, so finding them one at a time would
+ * walk past all the earlier ones on each lookup.
+ */
+static int io_poll_cancel_all(struct io_ring_ctx *ctx, struct io_cancel_data *cd,
+ struct io_hash_table *table)
+{
+ unsigned int nr_buckets = 1U << table->hash_bits;
+ unsigned int i = 0;
+ struct hlist_node *tmp;
+ struct io_kiocb *req;
+ int nr = 0;
+
+ if (!(cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
+ IORING_ASYNC_CANCEL_ANY))) {
+ /* keyed by user_data, so only one bucket can match */
+ i = hash_long(cd->data, table->hash_bits);
+ nr_buckets = i + 1;
+ }
+
+ for (; i < nr_buckets; i++) {
+ struct io_hash_bucket *hb = &table->hbs[i];
+
+ spin_lock(&hb->lock);
+ hlist_for_each_entry_safe(req, tmp, &hb->list, hash_node) {
+ if (!io_cancel_req_match(req, cd))
+ continue;
+ io_poll_cancel_req(req);
+ nr++;
+ }
+ spin_unlock(&hb->lock);
+ }
+ return nr;
+}
+
static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd,
struct io_hash_table *table)
{
struct io_hash_bucket *bucket;
struct io_kiocb *req;
- if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
- IORING_ASYNC_CANCEL_ANY))
+ if (cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY))
+ return io_poll_cancel_all(ctx, cd, table) ?: -ENOENT;
+
+ if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP))
req = io_poll_file_find(ctx, cd, table, &bucket);
else
req = io_poll_find(ctx, false, cd, table, &bucket);
@@ -914,16 +965,22 @@ static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd,
int io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd,
unsigned issue_flags)
{
- int ret;
+ bool all = cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY);
+ int ret, nr;
ret = __io_poll_cancel(ctx, cd, &ctx->cancel_table);
- if (ret != -ENOENT)
+ if (ret != -ENOENT && !all)
return ret;
+ nr = max(ret, 0);
io_ring_submit_lock(ctx, issue_flags);
ret = __io_poll_cancel(ctx, cd, &ctx->cancel_table_locked);
io_ring_submit_unlock(ctx, issue_flags);
- return ret;
+ if (!all)
+ return ret;
+ /* with ALL or ANY, both tables return how many they cancelled */
+ nr += max(ret, 0);
+ return nr ?: -ENOENT;
}
static __poll_t io_poll_parse_events(const struct io_uring_sqe *sqe,
@@ -1068,7 +1125,7 @@ int io_poll_remove(struct io_kiocb *req, unsigned int issue_flags)
poll->events &= ~0xffff;
poll->events |= poll_update->events & 0xffff;
- poll->events |= IO_POLL_UNMASK;
+ poll->events |= io_poll_unmask(poll->events);
}
if (poll_update->update_user_data)
preq->cqe.user_data = poll_update->new_user_data;
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index ad7ad35545d6..168c6987435e 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -562,11 +562,12 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
{
struct io_rsrc_update *up = io_kiocb_to_cmd(req, struct io_rsrc_update);
__s32 __user *fds = u64_to_user_ptr(up->arg);
+ struct io_ring_ctx *ctx = req->ctx;
unsigned int done;
struct file *file;
int ret, fd;
- if (!req->ctx->file_data)
+ if (!ctx->file_data)
return -ENXIO;
for (done = 0; done < up->nr_args; done++) {
@@ -580,15 +581,25 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
ret = -EBADF;
break;
}
- ret = io_fixed_fd_install(req, issue_flags, file,
- IORING_FILE_INDEX_ALLOC);
- if (ret < 0)
+ /*
+ * Hold the lock across the copy_to_user(), if it's dropped the
+ * slot can be closed and reused before the error path runs.
+ */
+ io_ring_submit_lock(ctx, issue_flags);
+ ret = __io_fixed_fd_install(ctx, file,
+ IORING_FILE_INDEX_ALLOC);
+ if (ret < 0) {
+ io_ring_submit_unlock(ctx, issue_flags);
+ fput(file);
break;
+ }
if (copy_to_user(&fds[done], &ret, sizeof(ret))) {
- __io_close_fixed(req->ctx, issue_flags, ret);
+ io_fixed_fd_remove(ctx, ret);
+ io_ring_submit_unlock(ctx, issue_flags);
ret = -EFAULT;
break;
}
+ io_ring_submit_unlock(ctx, issue_flags);
}
if (done)
diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
index 696aba6bbf81..130da8f8f67b 100644
--- a/io_uring/sqpoll.c
+++ b/io_uring/sqpoll.c
@@ -26,6 +26,7 @@
enum {
IO_SQ_THREAD_SHOULD_STOP = 0,
IO_SQ_THREAD_SHOULD_PARK,
+ IO_SQ_THREAD_EXITING,
};
void io_sq_thread_unpark(struct io_sq_data *sqd)
@@ -354,6 +355,18 @@ static int io_sq_thread(void *data)
timeout = jiffies + sqd->sq_thread_idle;
}
+ /*
+ * The SQPOLL thread never returns to userspace, so do the
+ * return-to-userspace work, like memory.high reclaim, here.
+ * It can sleep for a long time, so drop sqd->lock.
+ */
+ if (test_thread_flag(TIF_NOTIFY_RESUME)) {
+ mutex_unlock(&sqd->lock);
+ resume_user_mode_work(NULL);
+ mutex_lock(&sqd->lock);
+ sqd->sq_cpu = raw_smp_processor_id();
+ }
+
cap_entries = !list_is_singular(&sqd->ctx_list);
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) {
int ret = __io_sq_thread(ctx, sqd, cap_entries, &ist);
@@ -435,6 +448,13 @@ static int io_sq_thread(void *data)
if (retry_list)
io_sq_tw(&retry_list, UINT_MAX);
+ /*
+ * Cancelation drops sqd->lock while it waits, and a ring could attach
+ * to a thread that is on its way out. A fatal signal doesn't set the
+ * stop bit, so mark the thread as exiting here for
+ * io_sq_offload_create() to see.
+ */
+ set_bit(IO_SQ_THREAD_EXITING, &sqd->state);
io_uring_cancel_generic(true, sqd);
rcu_assign_pointer(sqd->thread, NULL);
put_task_struct(current);
@@ -503,7 +523,11 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx,
list_add(&ctx->sqd_list, &sqd->ctx_list);
io_sqd_update_thread_idle(sqd);
/* don't attach to a dying SQPOLL thread, would be racy */
- ret = (attached && !sqd->thread) ? -ENXIO : 0;
+ ret = 0;
+ if (attached && (!sqd->thread ||
+ test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state) ||
+ test_bit(IO_SQ_THREAD_EXITING, &sqd->state)))
+ ret = -ENXIO;
io_sq_thread_unpark(sqd);
if (ret < 0)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index d14cf23a16a6..21a54cedd854 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -6916,6 +6916,15 @@ static void perf_sigtrap(struct perf_event *event)
if (event->ctx->task == TASK_TOMBSTONE)
return;
+ /*
+ * The event was removed after this signal was queued, e.g. by exec()
+ * (remove_on_exec), which may also have closed its fd (close-on-exec).
+ * The new program has default signal handlers, so a SIGTRAP would
+ * kill it.
+ */
+ if (READ_ONCE(event->state) <= PERF_EVENT_STATE_EXIT)
+ return;
+
/*
* We'd expect this to only occur if the irq_work is delayed and either
* ctx->task or current has changed in the meantime. This can be the
diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h
index 7062a99a1ddd..0b60f8aa6286 100644
--- a/kernel/irq/internals.h
+++ b/kernel/irq/internals.h
@@ -77,6 +77,7 @@ enum {
#include "settings.h"
extern int __irq_set_trigger(struct irq_desc *desc, unsigned long flags);
+extern int __irq_set_irqchip_state(struct irq_desc *desc, enum irqchip_irq_state which, bool val);
extern void __disable_irq(struct irq_desc *desc);
extern void __enable_irq(struct irq_desc *desc);
diff --git a/kernel/irq/kexec.c b/kernel/irq/kexec.c
index 0f9548c1708d..58d92a7f2af1 100644
--- a/kernel/irq/kexec.c
+++ b/kernel/irq/kexec.c
@@ -25,7 +25,7 @@ void machine_kexec_mask_interrupts(void)
* First try to remove the active state from an interrupt which is forwarded
* to a VM. If the interrupt is not forwarded, try to EOI the interrupt.
*/
- check_eoi = irq_set_irqchip_state(i, IRQCHIP_STATE_ACTIVE, false);
+ check_eoi = __irq_set_irqchip_state(desc, IRQCHIP_STATE_ACTIVE, false);
}
if (check_eoi && chip->irq_eoi && irqd_irq_inprogress(&desc->irq_data))
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 67b1c5a35201..927a6e52a58c 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -2867,6 +2867,28 @@ int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
}
EXPORT_SYMBOL_GPL(irq_get_irqchip_state);
+int __irq_set_irqchip_state(struct irq_desc *desc, enum irqchip_irq_state which, bool val)
+{
+ struct irq_data *data = irq_desc_get_irq_data(desc);
+ struct irq_chip *chip;
+
+ do {
+ chip = irq_data_get_irq_chip(data);
+
+ if (WARN_ON_ONCE(!chip))
+ return -ENODEV;
+
+ if (chip->irq_set_irqchip_state)
+ break;
+
+ data = irqd_get_parent_data(data);
+ } while (data);
+
+ if (data)
+ return chip->irq_set_irqchip_state(data, which, val);
+ return -EINVAL;
+}
+
/**
* irq_set_irqchip_state - set the state of a forwarded interrupt.
* @irq: Interrupt line that is forwarded to a VM
@@ -2883,36 +2905,15 @@ int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
bool val)
{
struct irq_desc *desc;
- struct irq_data *data;
- struct irq_chip *chip;
unsigned long flags;
- int err = -EINVAL;
+ int err;
desc = irq_get_desc_buslock(irq, &flags, 0);
if (!desc)
- return err;
-
- data = irq_desc_get_irq_data(desc);
-
- do {
- chip = irq_data_get_irq_chip(data);
- if (WARN_ON_ONCE(!chip)) {
- err = -ENODEV;
- goto out_unlock;
- }
- if (chip->irq_set_irqchip_state)
- break;
-#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
- data = data->parent_data;
-#else
- data = NULL;
-#endif
- } while (data);
+ return -EINVAL;
- if (data)
- err = chip->irq_set_irqchip_state(data, which, val);
+ err = __irq_set_irqchip_state(desc, which, val);
-out_unlock:
irq_put_desc_busunlock(desc, flags);
return err;
}
diff --git a/kernel/panic.c b/kernel/panic.c
index 97c472164393..0bc445155035 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -246,18 +246,19 @@ EXPORT_SYMBOL(panic_on_other_cpu);
* nmi_panic_self_stop() which can provide architecture dependent code such
* as saving register state for crash dump.
*/
-void nmi_panic(struct pt_regs *regs, const char *msg)
+void nmi_panic(struct pt_regs *regs, const char *fmt, ...)
{
- int old_cpu, this_cpu;
+ va_list args;
- old_cpu = PANIC_CPU_INVALID;
- this_cpu = raw_smp_processor_id();
+ va_start(args, fmt);
- /* atomic_try_cmpxchg updates old_cpu on failure */
- if (atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu))
- panic("%s", msg);
- else if (old_cpu != this_cpu)
+ if (panic_try_start())
+ vpanic(fmt, args);
+
+ if (panic_on_other_cpu())
nmi_panic_self_stop(regs);
+
+ va_end(args);
}
EXPORT_SYMBOL(nmi_panic);
@@ -330,17 +331,15 @@ static void panic_other_cpus_shutdown(bool crash_kexec)
}
/**
- * panic - halt the system
- * @fmt: The text string to print
+ * panic - halt the system
+ * @fmt: The text string to print
+ * @args: Arguments for the format string
*
- * Display a message, then perform cleanups.
- *
- * This function never returns.
+ * Display a message, then perform cleanups. This function never returns.
*/
-void panic(const char *fmt, ...)
+void vpanic(const char *fmt, va_list args)
{
static char buf[1024];
- va_list args;
long i, i_next = 0, len;
int state = 0;
int old_cpu, this_cpu;
@@ -391,9 +390,7 @@ void panic(const char *fmt, ...)
console_verbose();
bust_spinlocks(1);
- va_start(args, fmt);
len = vscnprintf(buf, sizeof(buf), fmt, args);
- va_end(args);
if (len && buf[len - 1] == '\n')
buf[len - 1] = '\0';
@@ -530,7 +527,17 @@ void panic(const char *fmt, ...)
mdelay(PANIC_TIMER_STEP);
}
}
+EXPORT_SYMBOL(vpanic);
+
+/* Identical to vpanic(), except it takes variadic arguments instead of va_list */
+void panic(const char *fmt, ...)
+{
+ va_list args;
+ va_start(args, fmt);
+ vpanic(fmt, args);
+ va_end(args);
+}
EXPORT_SYMBOL(panic);
#define TAINT_FLAG(taint, _c_true, _c_false, _module) \
diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index b18c3b4763b7..7a4b23e48cc6 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -2173,7 +2173,11 @@ static void ops_dequeue(struct task_struct *p, u64 deq_flags)
{
unsigned long opss;
- /* dequeue is always temporary, don't reset runnable_at */
+ /*
+ * Don't reset runnable_at. If @p is waiting, the wait continues across
+ * temporary dequeues, e.g. for a migration or an attribute change.
+ * scx_ops_disable_task() ends it if @p leaves its scheduler.
+ */
clr_task_runnable(p, false);
/* acquire ensures that we see the preceding updates on QUEUED */
@@ -3794,6 +3798,14 @@ static void scx_ops_disable_task(struct task_struct *p)
if (SCX_HAS_OP(disable))
SCX_CALL_OP_TASK(SCX_KF_REST, disable, p);
scx_set_task_state(p, SCX_TASK_READY);
+
+ /*
+ * If @p is runnable but not running, its wait started at runnable_at,
+ * which ops_dequeue() didn't reset. Leaving the BPF scheduler ends the
+ * wait. Reset runnable_at the next time @p is enqueued on SCX, so that
+ * the watchdog doesn't count the time @p spends elsewhere.
+ */
+ p->scx.flags |= SCX_TASK_RESET_RUNNABLE_AT;
}
static void scx_ops_exit_task(struct task_struct *p)
diff --git a/kernel/time/posix-stubs.c b/kernel/time/posix-stubs.c
index 9b6fcb8d85e7..5a27335a917e 100644
--- a/kernel/time/posix-stubs.c
+++ b/kernel/time/posix-stubs.c
@@ -83,7 +83,7 @@ SYSCALL_DEFINE2(clock_getres, const clockid_t, which_clock, struct __kernel_time
case CLOCK_REALTIME:
case CLOCK_MONOTONIC:
case CLOCK_BOOTTIME:
- if (put_timespec64(&rtn_tp, tp))
+ if (tp && put_timespec64(&rtn_tp, tp))
return -EFAULT;
return 0;
default:
@@ -166,7 +166,7 @@ SYSCALL_DEFINE2(clock_getres_time32, clockid_t, which_clock,
case CLOCK_REALTIME:
case CLOCK_MONOTONIC:
case CLOCK_BOOTTIME:
- if (put_old_timespec32(&rtn_tp, tp))
+ if (tp && put_old_timespec32(&rtn_tp, tp))
return -EFAULT;
return 0;
default:
diff --git a/kernel/time/posix-timers.c b/kernel/time/posix-timers.c
index c5d9bfbb082b..1e7f01c3daaa 100644
--- a/kernel/time/posix-timers.c
+++ b/kernel/time/posix-timers.c
@@ -656,16 +656,8 @@ void common_timer_get(struct k_itimer *timr, struct itimerspec64 *cur_setting)
if (iv) {
cur_setting->it_interval = ktime_to_timespec64(iv);
} else if (!timr->it_active) {
- /*
- * SIGEV_NONE oneshot timers are never queued and therefore
- * timr->it_active is always false. The check below
- * vs. remaining time will handle this case.
- *
- * For all other timers there is nothing to update here, so
- * return.
- */
- if (!sig_none)
- return;
+ /* Nothing to update for a disarmed timer */
+ return;
}
now = kc->clock_get_ktime(timr->it_clock);
@@ -916,7 +908,15 @@ int common_timer_set(struct k_itimer *timr, int flags,
sigev_none = timr->it_sigev_notify == SIGEV_NONE;
kc->timer_arm(timr, expires, flags & TIMER_ABSTIME, sigev_none);
- timr->it_active = !sigev_none;
+ /*
+ * SIGEV_NONE timers are not queued, but mark them armed
+ * so that common_timer_get() can tell them apart from
+ * disarmed timers. An expired SIGEV_NONE timer stays
+ * armed, which is harmless: common_timer_get() reports
+ * zero remaining time for it and it never reaches the
+ * signal delivery code.
+ */
+ timr->it_active = 1;
return 0;
}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 88fcd439e93d..2d4872092561 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -385,9 +385,21 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)
return true;
}
+static void tick_nohz_full_update_tick(struct tick_sched *ts);
+
static void nohz_full_kick_func(struct irq_work *work)
{
- /* Empty, the tick restart happens on tick_nohz_irq_exit() */
+ struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched);
+
+ if (WARN_ON_ONCE(!tick_nohz_full_cpu(smp_processor_id())))
+ return;
+
+ /* Task-context flushes may not be followed by an IRQ exit. */
+ if (!in_hardirq()) {
+ if (WARN_ON_ONCE(tick_sched_flag_test(ts, TS_FLAG_INIDLE)))
+ return;
+ tick_nohz_full_update_tick(ts);
+ }
}
static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index 171f3a13cd6b..aa9aca76e76e 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -1747,6 +1747,9 @@ static bool expr_field_str(struct hist_field *field, struct seq_buf *s)
seq_buf_printf(s, ".%s", flags_str);
}
+ if (field->buckets)
+ seq_buf_printf(s, "=%lu", field->buckets);
+
return !seq_buf_has_overflowed(s);
}
diff --git a/lib/assoc_array.c b/lib/assoc_array.c
index 01619e88a52c..d3ee08fa2ab5 100644
--- a/lib/assoc_array.c
+++ b/lib/assoc_array.c
@@ -470,33 +470,32 @@ static bool assoc_array_insert_in_empty_tree(struct assoc_array_edit *edit)
}
/*
- * Handle insertion into a terminal node.
+ * The pointer that leads to a node or shortcut with the given back pointer
+ * and parent slot: the root, a slot in a node, or a shortcut's next node.
*/
-static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
- const struct assoc_array_ops *ops,
- const void *index_key,
- struct assoc_array_walk_result *result)
+static struct assoc_array_ptr **assoc_array_parent_link(struct assoc_array *array,
+ struct assoc_array_ptr *back_pointer,
+ int parent_slot)
{
- struct assoc_array_shortcut *shortcut, *new_s0;
- struct assoc_array_node *node, *new_n0, *new_n1, *side;
- struct assoc_array_ptr *ptr;
- unsigned long dissimilarity, base_seg, blank;
- size_t keylen;
- bool have_meta;
- int level, diff;
- int slot, next_slot, free_slot, i, j;
-
- node = result->terminal_node.node;
- level = result->terminal_node.level;
- edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+ if (!back_pointer)
+ return &array->root;
+ if (assoc_array_ptr_is_node(back_pointer))
+ return &assoc_array_ptr_to_node(back_pointer)->slots[parent_slot];
+ return &assoc_array_ptr_to_shortcut(back_pointer)->next_node;
+}
- pr_devel("-->%s()\n", __func__);
+/*
+ * Insert into a terminal node without changing its shape, if that can be
+ * done: in place of a leaf with the same index key, or in a free slot.
+ */
+static bool assoc_array_insert_in_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node)
+{
+ struct assoc_array_ptr *ptr;
+ int free_slot, i;
- /* We arrived at a node which doesn't have an onward node or shortcut
- * pointer that we have to follow. This means that (a) the leaf we
- * want must go here (either by insertion or replacement) or (b) we
- * need to split this node and insert in one of the fragments.
- */
free_slot = -1;
/* Firstly, we have to check the leaves in this node to see if there's
@@ -514,7 +513,6 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("replace in slot %d\n", i);
edit->leaf_p = &node->slots[i];
edit->dead_leaf = node->slots[i];
- pr_devel("<--%s() = ok [replace]\n", __func__);
return true;
}
}
@@ -526,28 +524,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("insert in free slot %d\n", free_slot);
edit->leaf_p = &node->slots[free_slot];
edit->adjust_count_on = node;
- pr_devel("<--%s() = ok [insert]\n", __func__);
return true;
}
- /* The node has no spare slots - so we're either going to have to split
- * it or insert another node before it.
- *
- * Whatever, we're going to need at least two new nodes - so allocate
- * those now. We may also need a new shortcut, but we deal with that
- * when we need it.
- */
- new_n0 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
- if (!new_n0)
- return false;
- edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
- new_n1 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
- if (!new_n1)
- return false;
- edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+ return false;
+}
+
+/*
+ * Fill in the edit script's segment cache for a full node: the segment at
+ * this level of each leaf's index key, or 0xff for a metadata pointer.
+ * The new key's segment, in the last entry, is left to the caller. Returns
+ * true if there are any metadata pointers.
+ */
+static bool assoc_array_fill_segment_cache(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const struct assoc_array_node *node,
+ int level)
+{
+ struct assoc_array_ptr *ptr;
+ unsigned long base_seg;
+ bool have_meta;
+ int i;
- /* We need to find out how similar the leaves are. */
- pr_devel("no spare slots\n");
have_meta = false;
for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
ptr = node->slots[i];
@@ -561,65 +559,42 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
}
+ return have_meta;
+}
- if (have_meta) {
- pr_devel("have meta\n");
- goto split_node;
- }
-
- /* The node contains only leaves */
- dissimilarity = 0;
- base_seg = edit->segment_cache[0];
- for (i = 1; i < ASSOC_ARRAY_FAN_OUT; i++)
- dissimilarity |= edit->segment_cache[i] ^ base_seg;
-
- pr_devel("only leaves; dissimilarity=%lx\n", dissimilarity);
+/*
+ * Whether the leaves of a full node and the new key all share one segment,
+ * so that splitting the node cannot make room.
+ */
+static bool assoc_array_all_one_segment(const struct assoc_array_edit *edit)
+{
+ int i;
- if ((dissimilarity & ASSOC_ARRAY_FAN_MASK) == 0) {
- /* The old leaves all cluster in the same slot. We will need
- * to insert a shortcut if the new node wants to cluster with them.
- */
- if ((edit->segment_cache[ASSOC_ARRAY_FAN_OUT] ^ base_seg) == 0)
- goto all_leaves_cluster_together;
+ for (i = 1; i < ASSOC_ARRAY_FAN_OUT + 1; i++)
+ if (edit->segment_cache[i] != edit->segment_cache[0])
+ return false;
+ return true;
+}
- /* Otherwise all the old leaves cluster in the same slot, but
- * the new leaf wants to go into a different slot - so we
- * create a new node (n0) to hold the new leaf and a pointer to
- * a new node (n1) holding all the old leaves.
- *
- * This can be done by falling through to the node splitting
- * path.
- */
- pr_devel("present leaves cluster but not new leaf\n");
- }
+/*
+ * Split a full node into new_n0, which takes its place, and new_n1, below
+ * new_n0 in the slot of a segment that at least two of the leaves share,
+ * the new one included. The caller has filled in the segment cache and
+ * set new_n0's back pointer and parent slot.
+ */
+static void assoc_array_split_node(struct assoc_array_edit *edit,
+ struct assoc_array_node *node,
+ struct assoc_array_node *new_n0,
+ struct assoc_array_node *new_n1)
+{
+ struct assoc_array_shortcut *shortcut;
+ struct assoc_array_node *side;
+ struct assoc_array_ptr *ptr;
+ int slot, next_slot, free_slot, i, j;
-split_node:
pr_devel("split node\n");
- /* We need to split the current node. The node must contain anything
- * from a single leaf (in the one leaf case, this leaf will cluster
- * with the new leaf) and the rest meta-pointers, to all leaves, some
- * of which may cluster.
- *
- * It won't contain the case in which all the current leaves plus the
- * new leaves want to cluster in the same slot.
- *
- * We need to expel at least two leaves out of a set consisting of the
- * leaves in the node and the new leaf. The current meta pointers can
- * just be copied as they shouldn't cluster with any of the leaves.
- *
- * We need a new node (n0) to replace the current one and a new node to
- * take the expelled nodes (n1).
- */
- edit->set[0].to = assoc_array_node_to_ptr(new_n0);
- new_n0->back_pointer = node->back_pointer;
- new_n0->parent_slot = node->parent_slot;
new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
-
-do_split_node:
- pr_devel("do_split_node\n");
-
new_n0->nr_leaves_on_branch = node->nr_leaves_on_branch;
new_n1->nr_leaves_on_branch = 0;
@@ -699,26 +674,29 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
}
}
}
+}
- ptr = node->back_pointer;
- if (!ptr)
- edit->set[0].ptr = &edit->array->root;
- else if (assoc_array_ptr_is_node(ptr))
- edit->set[0].ptr = &assoc_array_ptr_to_node(ptr)->slots[node->parent_slot];
- else
- edit->set[0].ptr = &assoc_array_ptr_to_shortcut(ptr)->next_node;
- edit->excised_meta[0] = assoc_array_node_to_ptr(node);
- pr_devel("<--%s() = ok [split node]\n", __func__);
- return true;
+/*
+ * All the leaves, new and old, want to cluster together in this node in the
+ * same slot, so we have to replace this node with a shortcut to skip over
+ * the identical parts of the key, leading to new_n0. If the node is the
+ * target of a shortcut, the new one replaces that shortcut as well. The
+ * caller then splits the leaves between new_n0 and new_n1 at the shortcut's
+ * far end, for which this fills in the segment cache.
+ */
+static struct assoc_array_shortcut *
+assoc_array_insert_shortcut(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node, int level,
+ struct assoc_array_node *new_n0)
+{
+ struct assoc_array_shortcut *new_s0;
+ unsigned long base_seg, blank;
+ size_t keylen;
+ int diff, i;
-all_leaves_cluster_together:
- /* All the leaves, new and old, want to cluster together in this node
- * in the same slot, so we have to replace this node with a shortcut to
- * skip over the identical parts of the key and then place a pair of
- * nodes, one inside the other, at the end of the shortcut and
- * distribute the keys between them.
- *
- * Firstly we need to work out where the leaves start diverging as a
+ /* Firstly we need to work out where the leaves start diverging as a
* bit position into their keys so that we know how big the shortcut
* needs to be.
*
@@ -744,17 +722,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
new_s0 = kzalloc(struct_size(new_s0, index_key, keylen), GFP_KERNEL);
if (!new_s0)
- return false;
+ return NULL;
edit->new_meta[2] = assoc_array_shortcut_to_ptr(new_s0);
- edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
new_s0->back_pointer = node->back_pointer;
new_s0->parent_slot = node->parent_slot;
+
+ /* A shortcut has to lead to a node, so a node at the end of a shortcut
+ * is replaced together with that shortcut. The new one holds the index
+ * key from the root, so it covers what the old one did.
+ */
+ if (node->back_pointer &&
+ assoc_array_ptr_is_shortcut(node->back_pointer)) {
+ struct assoc_array_shortcut *above =
+ assoc_array_ptr_to_shortcut(node->back_pointer);
+
+ new_s0->back_pointer = above->back_pointer;
+ new_s0->parent_slot = above->parent_slot;
+ edit->excised_meta[1] = node->back_pointer;
+ }
new_s0->next_node = assoc_array_node_to_ptr(new_n0);
new_n0->back_pointer = assoc_array_shortcut_to_ptr(new_s0);
new_n0->parent_slot = 0;
- new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
new_s0->skip_to_level = level = diff & ~ASSOC_ARRAY_LEVEL_STEP_MASK;
pr_devel("skip_to_level = %d [diff %d]\n", level, diff);
@@ -773,18 +762,104 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
/* This now reduces to a node splitting exercise for which we'll need
* to regenerate the disparity table.
*/
- for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
- ptr = node->slots[i];
- base_seg = ops->get_object_key_chunk(assoc_array_ptr_to_leaf(ptr),
- level);
- base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
- edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
- }
-
+ assoc_array_fill_segment_cache(edit, ops, node, level);
base_seg = ops->get_key_chunk(index_key, level);
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = base_seg & ASSOC_ARRAY_FAN_MASK;
- goto do_split_node;
+ return new_s0;
+}
+
+/*
+ * Handle insertion into a terminal node.
+ */
+static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_walk_result *result)
+{
+ struct assoc_array_shortcut *new_s0;
+ struct assoc_array_node *node, *new_n0, *new_n1;
+ bool have_meta;
+ int level;
+
+ node = result->terminal_node.node;
+ level = result->terminal_node.level;
+
+ pr_devel("-->%s()\n", __func__);
+
+ /* We arrived at a node which doesn't have an onward node or shortcut
+ * pointer that we have to follow. This means that (a) the leaf we
+ * want must go here (either by insertion or replacement) or (b) we
+ * need to split this node and insert in one of the fragments.
+ */
+ if (assoc_array_insert_in_node(edit, ops, index_key, node)) {
+ pr_devel("<--%s() = ok [in node]\n", __func__);
+ return true;
+ }
+
+ /* The node has no spare slots - so we're either going to have to split
+ * it or insert another node before it.
+ *
+ * Whatever, we're going to need at least two new nodes - so allocate
+ * those now. We may also need a new shortcut, but we deal with that
+ * when we need it.
+ */
+ new_n0 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
+ if (!new_n0)
+ return false;
+ edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
+ new_n1 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
+ if (!new_n1)
+ return false;
+ edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+
+ /* We need to find out how similar the leaves are. */
+ pr_devel("no spare slots\n");
+ have_meta = assoc_array_fill_segment_cache(edit, ops, node, level);
+ edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+
+ if (!have_meta && assoc_array_all_one_segment(edit)) {
+ /* The node only contains leaves and all the leaves cluster in
+ * the same slot, so we need to insert a shortcut to skip the
+ * common portion of the key and split the node at the
+ * shortcut's far end. The shortcut goes wherever its back
+ * pointer says, which may be further up than the node.
+ */
+ new_s0 = assoc_array_insert_shortcut(edit, ops, index_key,
+ node, level, new_n0);
+ if (!new_s0)
+ return false;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_s0->back_pointer,
+ new_s0->parent_slot);
+ edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
+ } else {
+ /* We need to split the current node. The node must contain
+ * anything from a single leaf (in the one leaf case, this leaf
+ * will cluster with the new leaf) and the rest meta-pointers,
+ * to all leaves, some of which may cluster.
+ *
+ * We need to expel at least two leaves out of a set consisting
+ * of the leaves in the node and the new leaf. The current meta
+ * pointers can just be copied as they shouldn't cluster with
+ * any of the leaves. If all the old leaves cluster in one slot
+ * and the new leaf does not, they all go and the new leaf stays.
+ *
+ * We need a new node (n0) to replace the current one and a new
+ * node to take the expelled nodes (n1).
+ */
+ new_n0->back_pointer = node->back_pointer;
+ new_n0->parent_slot = node->parent_slot;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_n0->back_pointer,
+ new_n0->parent_slot);
+ edit->set[0].to = assoc_array_node_to_ptr(new_n0);
+ }
+
+ assoc_array_split_node(edit, node, new_n0, new_n1);
+ edit->excised_meta[0] = assoc_array_node_to_ptr(node);
+ pr_devel("<--%s() = ok [split node]\n", __func__);
+ return true;
}
/*
@@ -843,7 +918,8 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
* zero length - otherwise we just connect the new node directly to the
* parent.
*/
- level += ASSOC_ARRAY_LEVEL_STEP;
+ if (shortcut->back_pointer)
+ level += ASSOC_ARRAY_LEVEL_STEP;
if (diff > level) {
pr_devel("pre-shortcut %d...%d\n", level, diff);
keylen = round_up(diff, ASSOC_ARRAY_KEY_CHUNK_SIZE);
@@ -866,9 +942,11 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
memcpy(new_s0->index_key, shortcut->index_key,
flex_array_size(new_s0, index_key, keylen));
- blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
- pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
- new_s0->index_key[keylen - 1] &= ~blank;
+ if (diff & ASSOC_ARRAY_KEY_CHUNK_MASK) {
+ blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
+ pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
+ new_s0->index_key[keylen - 1] &= ~blank;
+ }
} else {
pr_devel("no pre-shortcut\n");
edit->set[0].to = assoc_array_node_to_ptr(new_n0);
@@ -1484,17 +1562,26 @@ int assoc_array_gc(struct assoc_array *array,
cursor = array->root;
descend:
- /* If this point is a shortcut, then we need to duplicate it and
- * advance the target cursor.
+ /* The node at this position is duplicated, and so is the shortcut in
+ * front of it if there is one. The node is allocated first: the copy
+ * of the shortcut starts out pointing at the old node, and must not be
+ * in the new tree if the node cannot be allocated, or the cleanup
+ * would follow it into the old tree.
*/
+ new_n = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
+ if (!new_n)
+ goto enomem;
+
if (assoc_array_ptr_is_shortcut(cursor)) {
shortcut = assoc_array_ptr_to_shortcut(cursor);
keylen = round_up(shortcut->skip_to_level, ASSOC_ARRAY_KEY_CHUNK_SIZE);
keylen >>= ASSOC_ARRAY_KEY_CHUNK_SHIFT;
new_s = kmalloc(struct_size(new_s, index_key, keylen),
GFP_KERNEL);
- if (!new_s)
+ if (!new_s) {
+ kfree(new_n);
goto enomem;
+ }
pr_devel("dup shortcut %p -> %p\n", shortcut, new_s);
memcpy(new_s, shortcut, struct_size(new_s, index_key, keylen));
new_s->back_pointer = new_parent;
@@ -1504,11 +1591,7 @@ int assoc_array_gc(struct assoc_array *array,
cursor = shortcut->next_node;
}
- /* Duplicate the node at this position */
node = assoc_array_ptr_to_node(cursor);
- new_n = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL);
- if (!new_n)
- goto enomem;
pr_devel("dup node %p -> %p\n", node, new_n);
new_n->back_pointer = new_parent;
new_n->parent_slot = node->parent_slot;
diff --git a/lib/cmdline.c b/lib/cmdline.c
index 90ed997d9570..f5dc59b3548e 100644
--- a/lib/cmdline.c
+++ b/lib/cmdline.c
@@ -11,6 +11,7 @@
#include <linux/export.h>
#include <linux/kernel.h>
+#include <linux/overflow.h>
#include <linux/string.h>
#include <linux/ctype.h>
@@ -26,7 +27,9 @@ static int get_range(char **str, int *pint, int n)
(*str)++;
upper_range = simple_strtol((*str), NULL, 0);
- inc_counter = upper_range - *pint;
+ /* Keep the sign of the result when the difference doesn't fit */
+ if (check_sub_overflow(upper_range, *pint, &inc_counter))
+ inc_counter = upper_range < *pint ? -1 : INT_MAX;
for (x = *pint; n && x < upper_range; x++, n--)
*pint++ = x;
return inc_counter;
@@ -122,6 +125,14 @@ char *get_options(const char *str, int nints, int *ints)
range_nums = get_range((char **)&str, pint, n);
if (range_nums < 0)
break;
+ /* The range didn't fit, so the array is full */
+ if (!validate && range_nums > n) {
+ i = nints;
+ break;
+ }
+ /* Leave room for the upper number of the range */
+ if (range_nums >= INT_MAX - i)
+ break;
/*
* Decrement the result by one to leave out the
* last number in the range. The next iteration
diff --git a/lib/cmdline_kunit.c b/lib/cmdline_kunit.c
index c1602f797637..74c4b5ea3225 100644
--- a/lib/cmdline_kunit.c
+++ b/lib/cmdline_kunit.c
@@ -139,11 +139,112 @@ static void cmdline_test_range(struct kunit *test)
} while (++i < ARRAY_SIZE(cmdline_test_range_strings));
}
+static const struct {
+ const char *in;
+ int parsed[4];
+ int validated;
+} cmdline_test_range_overflow_cases[] = {
+ { "1-100", { 3, 1, 2, 3, }, 100, },
+ { "1,5-100,7", { 3, 1, 5, 6, }, 98, },
+ { "1-2147483646", { 3, 1, 2, 3, }, 2147483646, },
+ { "0-2147483647", { 3, 0, 1, 2, }, 0, },
+ { "-5-2147483647", { 3, -5, -4, -3, }, 0, },
+ { "2147483647--5", { 0, 2147483647, }, 0, },
+ { "1,0-2147483647", { 3, 1, 0, 1, }, 1, },
+};
+
+static void cmdline_test_range_overflow(struct kunit *test)
+{
+ unsigned int i, j;
+
+ for (i = 0; i < ARRAY_SIZE(cmdline_test_range_overflow_cases); i++) {
+ const char *in = cmdline_test_range_overflow_cases[i].in;
+ const int *e = cmdline_test_range_overflow_cases[i].parsed;
+ /* Two guard elements past the array handed to get_options() */
+ int r[ARRAY_SIZE(cmdline_test_range_overflow_cases[0].parsed) + 2];
+ int n;
+
+ memset(r, 0, sizeof(r));
+ get_options(in, ARRAY_SIZE(r) - 2, r);
+ for (j = 0; j < ARRAY_SIZE(r) - 2; j++)
+ KUNIT_EXPECT_EQ_MSG(test, r[j], e[j], "Pattern: %s at %u", in, j);
+ for (; j < ARRAY_SIZE(r); j++)
+ KUNIT_EXPECT_EQ_MSG(test, r[j], 0, "Pattern: %s out of bound at %u", in, j);
+
+ get_options(in, 0, &n);
+ KUNIT_EXPECT_EQ_MSG(test, n, cmdline_test_range_overflow_cases[i].validated,
+ "Pattern: %s (validated)", in);
+ }
+}
+
+static void cmdline_test_next_arg_quoted_value(struct kunit *test)
+{
+ char in[] = "foo=\"bar baz\" qux=1";
+ char *next, *param, *val;
+
+ next = next_arg(in, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "foo");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "bar baz");
+ KUNIT_EXPECT_STREQ(test, next, "qux=1");
+
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "qux");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "1");
+ KUNIT_EXPECT_STREQ(test, next, "");
+}
+
+static void cmdline_test_next_arg_bare_quote_regression(struct kunit *test)
+{
+ char in[] = "foo=bar \"";
+ char *next, *param, *val;
+
+ next = next_arg(in, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "foo");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "bar");
+ KUNIT_EXPECT_STREQ(test, next, "\"");
+
+ /* This hits the i == 0 quoted-token case fixed by 9847f21225c4. */
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "");
+ KUNIT_EXPECT_PTR_EQ(test, val, NULL);
+ KUNIT_EXPECT_STREQ(test, next, "");
+}
+
+static void cmdline_test_next_arg_mixed_tokens(struct kunit *test)
+{
+ char in[] = "bbb= jjj kkk=\"a=b\"";
+ char *next, *param, *val;
+
+ next = next_arg(in, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "bbb");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "");
+ KUNIT_EXPECT_STREQ(test, next, "jjj kkk=\"a=b\"");
+
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "jjj");
+ KUNIT_EXPECT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, next, "kkk=\"a=b\"");
+
+ next = next_arg(next, ¶m, &val);
+ KUNIT_EXPECT_STREQ(test, param, "kkk");
+ KUNIT_ASSERT_NOT_NULL(test, val);
+ KUNIT_EXPECT_STREQ(test, val, "a=b");
+ KUNIT_EXPECT_STREQ(test, next, "");
+}
+
static struct kunit_case cmdline_test_cases[] = {
KUNIT_CASE(cmdline_test_noint),
KUNIT_CASE(cmdline_test_lead_int),
KUNIT_CASE(cmdline_test_tail_int),
KUNIT_CASE(cmdline_test_range),
+ KUNIT_CASE(cmdline_test_range_overflow),
+ KUNIT_CASE(cmdline_test_next_arg_quoted_value),
+ KUNIT_CASE(cmdline_test_next_arg_bare_quote_regression),
+ KUNIT_CASE(cmdline_test_next_arg_mixed_tokens),
{}
};
diff --git a/lib/raid6/avx2.c b/lib/raid6/avx2.c
index 059024234dce..2d03ce97f655 100644
--- a/lib/raid6/avx2.c
+++ b/lib/raid6/avx2.c
@@ -76,6 +76,7 @@ static void raid6_avx21_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -124,6 +125,7 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -186,6 +188,7 @@ static void raid6_avx22_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -254,6 +257,7 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -347,6 +351,7 @@ static void raid6_avx24_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -457,6 +462,7 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
asm volatile("vmovntdq %%ymm14,%0" : "=m" (q[d+96]));
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/raid6/avx512.c b/lib/raid6/avx512.c
index 9c3e822e1adf..30956989e7d7 100644
--- a/lib/raid6/avx512.c
+++ b/lib/raid6/avx512.c
@@ -95,6 +95,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -154,6 +155,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -227,6 +229,7 @@ static void raid6_avx5122_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -311,6 +314,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -417,6 +421,7 @@ static void raid6_avx5124_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -550,6 +555,7 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
"m" (q[d+128]), "m" (q[d+192]));
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
const struct raid6_calls raid6_avx512x4 = {
diff --git a/lib/raid6/recov_avx2.c b/lib/raid6/recov_avx2.c
index 4e8095403ee2..a7fd1ba177f1 100644
--- a/lib/raid6/recov_avx2.c
+++ b/lib/raid6/recov_avx2.c
@@ -180,6 +180,7 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -297,6 +298,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/raid6/recov_avx512.c b/lib/raid6/recov_avx512.c
index fd9e15bf3f30..c8a152d9ab6d 100644
--- a/lib/raid6/recov_avx512.c
+++ b/lib/raid6/recov_avx512.c
@@ -221,6 +221,7 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -363,6 +364,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/stackdepot.c b/lib/stackdepot.c
index 245d5b416699..1f1d63d21f26 100644
--- a/lib/stackdepot.c
+++ b/lib/stackdepot.c
@@ -273,7 +273,7 @@ static bool depot_init_pool(void **prealloc)
* NULL; do not reset to NULL if we have reached the maximum number of
* pools.
*/
- if (pools_num < DEPOT_MAX_POOLS)
+ if (pools_num + 1 < DEPOT_MAX_POOLS)
WRITE_ONCE(new_pool, NULL);
else
WRITE_ONCE(new_pool, STACK_DEPOT_POISON);
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 3142776e2e0e..e41d201d4b8f 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -324,6 +324,8 @@ struct damos_quota_goal *damos_new_quota_goal(
return NULL;
goal->metric = metric;
goal->target_value = target_value;
+ if (metric == DAMOS_QUOTA_SOME_MEM_PSI_US)
+ goal->last_psi_total = U64_MAX;
INIT_LIST_HEAD(&goal->list);
return goal;
}
@@ -707,7 +709,8 @@ static int damos_commit_quota_goal(
dst->target_value = src->target_value;
if (dst->metric == DAMOS_QUOTA_USER_INPUT)
dst->current_value = src->current_value;
- /* keep last_psi_total as is, since it will be updated in next cycle */
+ else if (dst->metric == DAMOS_QUOTA_SOME_MEM_PSI_US)
+ dst->last_psi_total = U64_MAX;
return 0;
}
@@ -1274,7 +1277,7 @@ static void kdamond_reset_aggregated(struct damon_ctx *c)
}
}
-static void damon_split_region_at(struct damon_target *t,
+static int damon_split_region_at(struct damon_target *t,
struct damon_region *r, unsigned long sz_r);
static bool __damos_valid_target(struct damon_region *r, struct damos *s)
@@ -1377,13 +1380,14 @@ static void damos_update_stat(struct damos *s,
}
static bool __damos_filter_out(struct damon_ctx *ctx, struct damon_target *t,
- struct damon_region *r, struct damos_filter *filter)
+ struct damon_region *r, struct damos_filter *filter, int *err)
{
bool matched = false;
struct damon_target *ti;
int target_idx = 0;
unsigned long start, end;
+ *err = 0;
switch (filter->type) {
case DAMOS_FILTER_TYPE_TARGET:
damon_for_each_target(ti, ctx) {
@@ -1409,12 +1413,13 @@ static bool __damos_filter_out(struct damon_ctx *ctx, struct damon_target *t,
}
/* start before the range and overlap */
if (r->ar.start < start) {
- damon_split_region_at(t, r, start - r->ar.start);
+ *err = damon_split_region_at(t, r,
+ start - r->ar.start);
matched = false;
break;
}
/* start inside the range */
- damon_split_region_at(t, r, end - r->ar.start);
+ *err = damon_split_region_at(t, r, end - r->ar.start);
matched = true;
break;
default:
@@ -1428,9 +1433,12 @@ static bool damos_filter_out(struct damon_ctx *ctx, struct damon_target *t,
struct damon_region *r, struct damos *s)
{
struct damos_filter *filter;
+ bool matching;
+ int err;
damos_for_each_filter(filter, s) {
- if (__damos_filter_out(ctx, t, r, filter))
+ matching = __damos_filter_out(ctx, t, r, filter, &err);
+ if (matching || err)
return true;
}
return false;
@@ -1478,7 +1486,8 @@ static void damos_apply_scheme(struct damon_ctx *c, struct damon_target *t,
DAMON_MIN_REGION);
if (!sz)
goto update_stat;
- damon_split_region_at(t, r, sz);
+ if (damon_split_region_at(t, r, sz))
+ goto update_stat;
}
if (damos_filter_out(c, t, r, s))
return;
@@ -1616,7 +1625,12 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
break;
case DAMOS_QUOTA_SOME_MEM_PSI_US:
now_psi_total = damos_get_some_mem_psi_total();
- goal->current_value = now_psi_total - goal->last_psi_total;
+ /* uninitialized last_psi_total; make no effect this round */
+ if (goal->last_psi_total == U64_MAX)
+ goal->current_value = goal->target_value;
+ else
+ goal->current_value = now_psi_total -
+ goal->last_psi_total;
goal->last_psi_total = now_psi_total;
break;
default:
@@ -1681,6 +1695,26 @@ static void damos_set_effective_quota(struct damos_quota *quota)
quota->esz = esz;
}
+static void damos_fix_stale_charge_pointers(struct damon_ctx *c,
+ struct damos *s)
+{
+ struct damos_quota *quota;
+ struct damon_target *t;
+
+ quota = &s->quota;
+ if (!quota->charge_target_from)
+ return;
+ damon_for_each_target(t, c) {
+ if (t == quota->charge_target_from) {
+ if (damon_nr_regions(t) > 0)
+ return;
+ break;
+ }
+ }
+ quota->charge_target_from = NULL;
+ quota->charge_addr_from = 0;
+}
+
static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
{
struct damos_quota *quota = &s->quota;
@@ -1709,6 +1743,8 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
damos_set_effective_quota(quota);
}
+ damos_fix_stale_charge_pointers(c, s);
+
if (!c->ops.get_scheme_score)
return;
@@ -1870,15 +1906,17 @@ static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold,
*
* r the region to be split
* sz_r size of the first sub-region that will be made
+ *
+ * Return: 0 on success, negative error code otherwise.
*/
-static void damon_split_region_at(struct damon_target *t,
+static int damon_split_region_at(struct damon_target *t,
struct damon_region *r, unsigned long sz_r)
{
struct damon_region *new;
new = damon_new_region(r->ar.start + sz_r, r->ar.end);
if (!new)
- return;
+ return -ENOMEM;
r->ar.end = new->ar.start;
@@ -1888,6 +1926,7 @@ static void damon_split_region_at(struct damon_target *t,
new->nr_accesses = r->nr_accesses;
damon_insert_region(new, r, damon_next_region(r), t);
+ return 0;
}
/* Split every region in the given target into 'nr_subs' regions */
@@ -1932,7 +1971,6 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
{
struct damon_target *t;
unsigned int nr_regions = 0;
- static unsigned int last_nr_regions;
int nr_subregions = 2;
damon_for_each_target(t, ctx)
@@ -1942,14 +1980,14 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
return;
/* Maybe the middle of the region has different access frequency */
- if (last_nr_regions == nr_regions &&
+ if (ctx->last_nr_regions == nr_regions &&
nr_regions < ctx->attrs.max_nr_regions / 3)
nr_subregions = 3;
damon_for_each_target(t, ctx)
damon_split_regions_of(t, nr_subregions);
- last_nr_regions = nr_regions;
+ ctx->last_nr_regions = nr_regions;
}
/*
diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c
index a6b1de97b1e8..6bac269cd679 100644
--- a/mm/damon/paddr.c
+++ b/mm/damon/paddr.c
@@ -273,7 +273,7 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s)
else
list_add(&folio->lru, &folio_list);
put_folio:
- addr += folio_size(folio);
+ addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio);
folio_put(folio);
}
if (install_young_filter)
@@ -306,7 +306,7 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate(
folio_deactivate(folio);
applied += folio_nr_pages(folio);
put_folio:
- addr += folio_size(folio);
+ addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio);
folio_put(folio);
}
return applied * PAGE_SIZE;
@@ -483,7 +483,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s)
folio_is_file_lru(folio));
list_add(&folio->lru, &folio_list);
put_folio:
- addr += folio_size(folio);
+ addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio);
folio_put(folio);
}
applied = damon_pa_migrate_pages(&folio_list, s->target_nid);
diff --git a/mm/damon/tests/core-kunit.h b/mm/damon/tests/core-kunit.h
index ff9f73579f7b..6b7cd8149a0e 100644
--- a/mm/damon/tests/core-kunit.h
+++ b/mm/damon/tests/core-kunit.h
@@ -527,6 +527,7 @@ static void damos_test_filter_out(struct kunit *test)
struct damon_target *t;
struct damon_region *r, *r2;
struct damos_filter *f;
+ int err;
f = damos_new_filter(DAMOS_FILTER_TYPE_ADDR, true);
f->addr_range = (struct damon_addr_range){
@@ -537,25 +538,25 @@ static void damos_test_filter_out(struct kunit *test)
damon_add_region(r, t);
/* region in the range */
- KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f));
+ KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f, &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region before the range */
r->ar.start = DAMON_MIN_REGION * 1;
r->ar.end = DAMON_MIN_REGION * 2;
- KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f));
+ KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f, &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region after the range */
r->ar.start = DAMON_MIN_REGION * 6;
r->ar.end = DAMON_MIN_REGION * 8;
- KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f));
+ KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f, &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region started before the range */
r->ar.start = DAMON_MIN_REGION * 1;
r->ar.end = DAMON_MIN_REGION * 4;
- KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f));
+ KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f, &err));
/* filter should have split the region */
KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 1);
KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 2);
@@ -570,7 +571,7 @@ static void damos_test_filter_out(struct kunit *test)
/* region started in the range */
r->ar.start = DAMON_MIN_REGION * 2;
r->ar.end = DAMON_MIN_REGION * 8;
- KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f));
+ KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f, &err));
/* filter should have split the region */
KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 2);
KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 6);
diff --git a/mm/filemap.c b/mm/filemap.c
index 64c2325b9238..602106b329bc 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1346,10 +1346,22 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr,
* return value based on that state without races.
*
* Also note that WQ_FLAG_WOKEN is sufficient for a non-exclusive
- * waiter, but an exclusive one requires WQ_FLAG_DONE.
+ * waiter, but an exclusive one must also acquire the bit.
*/
- if (behavior == EXCLUSIVE)
- return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR;
+ if (behavior == EXCLUSIVE) {
+ if (wait->flags & WQ_FLAG_DONE)
+ return 0;
+ /*
+ * We might have just been woken up and the waker will not
+ * wake up anybody else. If we ignore that now, the remaining
+ * waiters may not get woken up: take the bit now if it is
+ * still free.
+ */
+ if ((wait->flags & WQ_FLAG_WOKEN) &&
+ !test_and_set_bit(bit_nr, folio_flags(folio, 0)))
+ return 0;
+ return -EINTR;
+ }
return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR;
}
@@ -3096,11 +3108,11 @@ loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start,
* @folio - the folio to lock.
* @fpin - the pointer to the file we may pin (or is already pinned).
*
- * This works similar to lock_folio_or_retry in that it can drop the
- * mmap_lock. It differs in that it actually returns the folio locked
- * if it returns 1 and 0 if it couldn't lock the folio. If we did have
- * to drop the mmap_lock then fpin will point to the pinned file and
- * needs to be fput()'ed at a later point.
+ * Returns 1 with the folio locked, or 0 if locking failed.
+ *
+ * On both entry and return, a non-NULL *fpin means the fault lock has
+ * already been dropped and the caller must eventually fput() the file.
+ * Otherwise the fault lock is still held, including on failure.
*/
static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
struct file **fpin)
@@ -3117,21 +3129,9 @@ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
return 0;
*fpin = maybe_unlock_mmap_for_io(vmf, *fpin);
- if (vmf->flags & FAULT_FLAG_KILLABLE) {
- if (__folio_lock_killable(folio)) {
- /*
- * We didn't have the right flags to drop the
- * fault lock, but all fault_handlers only check
- * for fatal signals if we return VM_FAULT_RETRY,
- * so we need to drop the fault lock here and
- * return 0 if we don't have a fpin.
- */
- if (*fpin == NULL)
- release_fault_lock(vmf);
- return 0;
- }
- } else
- __folio_lock(folio);
+ if (vmf->flags & FAULT_FLAG_KILLABLE)
+ return !__folio_lock_killable(folio);
+ __folio_lock(folio);
return 1;
}
@@ -3293,7 +3293,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)
* vma->vm_mm->mmap_lock must be held on entry.
*
* If our return value has VM_FAULT_RETRY set, it's because the mmap_lock
- * may be dropped before doing I/O or by lock_folio_maybe_drop_mmap().
+ * may be dropped before doing I/O or when aborting an interrupted wait.
*
* If our return value does not have VM_FAULT_RETRY set, the mmap_lock
* has not been released.
@@ -3449,9 +3449,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
out_retry:
/*
- * We dropped the mmap_lock, we need to return to the fault handler to
- * re-find the vma and come back and find our hopefully still populated
- * page.
+ * With fpin set, the fault lock was already dropped. Otherwise it is
+ * still held. Release invalidate_lock before dropping either protection.
*/
if (!IS_ERR(folio))
folio_put(folio);
@@ -3459,6 +3458,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
filemap_invalidate_unlock_shared(mapping);
if (fpin)
fput(fpin);
+ else if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
+ release_fault_lock(vmf);
return ret | VM_FAULT_RETRY;
}
EXPORT_SYMBOL(filemap_fault);
diff --git a/mm/gup.c b/mm/gup.c
index 810a4c32ee87..d9107eb461a0 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -3448,6 +3448,9 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
FOLL_PCI_P2PDMA | FOLL_HONOR_NUMA_FAULT)))
return -EINVAL;
+ if (unlikely(!nr_pages))
+ return 0;
+
if (gup_flags & FOLL_PIN)
mm_set_has_pinned_flag(¤t->mm->flags);
@@ -3455,8 +3458,8 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
might_lock_read(¤t->mm->mmap_lock);
start = untagged_addr(start) & PAGE_MASK;
- len = nr_pages << PAGE_SHIFT;
- if (check_add_overflow(start, len, &end))
+ if (check_shl_overflow(nr_pages, PAGE_SHIFT, &len) ||
+ check_add_overflow(start, len, &end))
return -EOVERFLOW;
if (end > TASK_SIZE_MAX)
return -EFAULT;
@@ -3703,7 +3706,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
{
unsigned int flags, nr_folios, nr_found;
unsigned int i, pgshift = PAGE_SHIFT;
- pgoff_t start_idx, end_idx, next_idx;
+ pgoff_t start_idx, end_idx, next_idx, retry_idx;
struct folio *folio = NULL;
struct folio_batch fbatch;
struct hstate *h;
@@ -3744,6 +3747,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
* reason, we try to allocate them and add them to the
* page cache.
*/
+ retry_idx = start_idx;
nr_found = filemap_get_folios_contig(memfd->f_mapping,
&start_idx,
end_idx,
@@ -3768,12 +3772,38 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
folio = page_folio(&fbatch.folios[i]->page);
+ /*
+ * The folio might have been removed from the
+ * page cache (e.g., truncated or collapsed)
+ * after the lookup. Recheck under the folio
+ * lock.
+ */
+ folio_lock(folio);
+ if (unlikely(folio->mapping != memfd->f_mapping)) {
+ folio_unlock(folio);
+ start_idx = retry_idx;
+ break;
+ }
+
if (try_grab_folio(folio, 1, FOLL_PIN)) {
+ folio_unlock(folio);
folio_batch_release(&fbatch);
ret = -EINVAL;
goto err;
}
+ /*
+ * fallocate() can leave folios in the
+ * page cache that were never cleared.
+ */
+ if (!folio_test_uptodate(folio)) {
+ folio_zero_range(folio, 0, folio_size(folio));
+ flush_dcache_folio(folio);
+ folio_mark_uptodate(folio);
+ }
+ retry_idx = folio_next_index(folio);
+ folio_unlock(folio);
+
if (nr_folios == 0)
*offset = offset_in_folio(folio, start);
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index d175e93988a9..ce4d17871ff4 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2988,16 +2988,66 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address,
__split_huge_pmd(vma, pmd, address, freeze, folio);
}
-static inline void split_huge_pmd_if_needed(struct vm_area_struct *vma, unsigned long address)
+static bool pud_split_required(struct vm_area_struct *vma,
+ unsigned long address)
{
+ if (!IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD))
+ return false;
+ if (IS_ALIGNED(address, HPAGE_PUD_SIZE))
+ return false;
/*
- * If the new address isn't hpage aligned and it could previously
- * contain an hugepage: check if we need to split an huge pmd.
+ * Only selected VMAs support mapping PUDs for now. Lift the
+ * restriction once this changes.
*/
- if (!IS_ALIGNED(address, HPAGE_PMD_SIZE) &&
- range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
- ALIGN(address, HPAGE_PMD_SIZE)))
- split_huge_pmd_address(vma, address, false, NULL);
+ if (!vma_is_dax(vma) && !(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)))
+ return false;
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PUD_SIZE),
+ ALIGN(address, HPAGE_PUD_SIZE));
+}
+
+static bool pmd_split_required(struct vm_area_struct *vma,
+ unsigned long address)
+{
+ if (IS_ALIGNED(address, HPAGE_PMD_SIZE))
+ return false;
+
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
+ ALIGN(address, HPAGE_PMD_SIZE));
+}
+
+static void split_huge_leaf(struct vm_area_struct *vma,
+ unsigned long address)
+{
+ const bool split_pud = pud_split_required(vma, address);
+ const bool split_pmd = pmd_split_required(vma, address);
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
+ /*
+ * mmap, VMA and rmap write locks protect the walk; the split helpers
+ * take the page table locks.
+ */
+ if (!split_pud && !split_pmd)
+ return;
+
+ pgd = pgd_offset(vma->vm_mm, address);
+ if (!pgd_present(*pgd))
+ return;
+
+ p4d = p4d_offset(pgd, address);
+ if (!p4d_present(*p4d))
+ return;
+
+ pud = pud_offset(p4d, address);
+ if (pud_leaf(*pud))
+ __split_huge_pud(vma, pud, address);
+ if (!split_pmd || !pud_present(*pud))
+ return;
+
+ pmd = pmd_offset(pud, address);
+ __split_huge_pmd(vma, pmd, address, false, NULL);
}
void vma_adjust_trans_huge(struct vm_area_struct *vma,
@@ -3006,10 +3056,10 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
long adjust_next)
{
/* Check if we need to split start first. */
- split_huge_pmd_if_needed(vma, start);
+ split_huge_leaf(vma, start);
/* Check if we need to split end next. */
- split_huge_pmd_if_needed(vma, end);
+ split_huge_leaf(vma, end);
/*
* If we're also updating the next vma vm_start,
@@ -3019,7 +3069,7 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
struct vm_area_struct *next = find_vma(vma->vm_mm, vma->vm_end);
unsigned long nstart = next->vm_start;
nstart += adjust_next;
- split_huge_pmd_if_needed(next, nstart);
+ split_huge_leaf(next, nstart);
}
}
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 049726560c1a..1dd80a5b72b0 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -48,6 +48,7 @@
#include <linux/page_owner.h>
#include "internal.h"
#include "hugetlb_vmemmap.h"
+#include <linux/page-isolation.h>
int hugetlb_max_hstate __read_mostly;
unsigned int default_hstate_idx;
@@ -1350,6 +1351,9 @@ static struct folio *dequeue_hugetlb_folio_node_exact(struct hstate *h,
if (folio_test_hwpoison(folio))
continue;
+ if (is_migrate_isolate_page(&folio->page))
+ continue;
+
list_move(&folio->lru, &h->hugepage_activelist);
folio_ref_unfreeze(folio, 1);
folio_clear_hugetlb_freed(folio);
@@ -2862,20 +2866,24 @@ void restore_reserve_on_error(struct hstate *h, struct vm_area_struct *vma,
/*
* alloc_and_dissolve_hugetlb_folio - Allocate a new folio and dissolve
* the old one
- * @h: struct hstate old page belongs to
* @old_folio: Old folio to dissolve
* @list: List to isolate the page in case we need to
* Returns 0 on success, otherwise negated error.
*/
-static int alloc_and_dissolve_hugetlb_folio(struct hstate *h,
- struct folio *old_folio, struct list_head *list)
+static int alloc_and_dissolve_hugetlb_folio(struct folio *old_folio,
+ struct list_head *list)
{
- gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE;
+ gfp_t gfp_mask;
+ struct hstate *h;
int nid = folio_nid(old_folio);
struct folio *new_folio = NULL;
int ret = 0;
retry:
+ /*
+ * The old_folio might have been dissolved from under our feet, so make sure
+ * to carefully check the state under the lock.
+ */
spin_lock_irq(&hugetlb_lock);
if (!folio_test_hugetlb(old_folio)) {
/*
@@ -2904,8 +2912,10 @@ static int alloc_and_dissolve_hugetlb_folio(struct hstate *h,
cond_resched();
goto retry;
} else {
+ h = folio_hstate(old_folio);
if (!new_folio) {
spin_unlock_irq(&hugetlb_lock);
+ gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE;
new_folio = alloc_buddy_hugetlb_folio(h, gfp_mask, nid,
NULL, NULL);
if (!new_folio)
@@ -2949,36 +2959,25 @@ static int alloc_and_dissolve_hugetlb_folio(struct hstate *h,
int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list)
{
- struct hstate *h;
struct folio *folio = page_folio(page);
int ret = -EBUSY;
- /*
- * The page might have been dissolved from under our feet, so make sure
- * to carefully check the state under the lock.
- * Return success when racing as if we dissolved the page ourselves.
- */
- spin_lock_irq(&hugetlb_lock);
- if (folio_test_hugetlb(folio)) {
- h = folio_hstate(folio);
- } else {
- spin_unlock_irq(&hugetlb_lock);
+ /* Not to disrupt normal path by vainly holding hugetlb_lock */
+ if (!folio_test_hugetlb(folio))
return 0;
- }
- spin_unlock_irq(&hugetlb_lock);
/*
* Fence off gigantic pages as there is a cyclic dependency between
* alloc_contig_range and them. Return -ENOMEM as this has the effect
* of bailing out right away without further retrying.
*/
- if (hstate_is_gigantic(h))
+ if (folio_order(folio) > MAX_PAGE_ORDER)
return -ENOMEM;
if (folio_ref_count(folio) && isolate_hugetlb(folio, list))
ret = 0;
else if (!folio_ref_count(folio))
- ret = alloc_and_dissolve_hugetlb_folio(h, folio, list);
+ ret = alloc_and_dissolve_hugetlb_folio(folio, list);
return ret;
}
@@ -2991,6 +2990,37 @@ void wait_for_freed_hugetlb_folios(void)
flush_work(&free_hpage_work);
}
+/*
+ * replace_free_hugepage_folios - Replace free hugepage folios in a given pfn
+ * range with new folios.
+ * @start_pfn: start pfn of the given pfn range
+ * @end_pfn: end pfn of the given pfn range
+ * Returns 0 on success, otherwise negated error.
+ */
+int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long end_pfn)
+{
+ struct folio *folio;
+ int ret = 0;
+
+ LIST_HEAD(isolate_list);
+
+ while (start_pfn < end_pfn) {
+ folio = pfn_folio(start_pfn);
+
+ /* Not to disrupt normal path by vainly holding hugetlb_lock */
+ if (folio_test_hugetlb(folio) && !folio_ref_count(folio)) {
+ ret = alloc_and_dissolve_hugetlb_folio(folio, &isolate_list);
+ if (ret)
+ break;
+
+ putback_movable_pages(&isolate_list);
+ }
+ start_pfn++;
+ }
+
+ return ret;
+}
+
/*
* NOTE! "cow_from_owner" represents a very hacky usage only used in CoW
* faults of hugetlb private mappings on top of a non-page-cache folio (in
@@ -3013,7 +3043,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
bool deferred_reserve;
gfp_t gfp = htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL;
- memcg = get_mem_cgroup_from_current();
+ memcg = get_mem_cgroup_from_mm(vma->vm_mm);
memcg_charge_ret = mem_cgroup_hugetlb_try_charge(memcg, gfp, nr_pages);
if (memcg_charge_ret == -ENOMEM) {
mem_cgroup_put(memcg);
@@ -5275,10 +5305,11 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
raw_write_seqcount_begin(&src->write_protect_seq);
} else {
/*
- * For shared mappings the vma lock must be held before
- * calling hugetlb_walk() in the src vma. Otherwise, the
- * returned ptep could go away if part of a shared pmd and
- * another thread calls huge_pmd_unshare.
+ * Shared mappings only need copying when we could have
+ * information stored in page tables (e.g., userfaultfd PTE
+ * bits or markers), which is incompatible with PMD sharing.
+ * Still take the hugetlb VMA lock for now to satisfy
+ * hugetlb_walk()'s locking requirements.
*/
hugetlb_vma_lock_read(src_vma);
}
@@ -5286,6 +5317,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
last_addr_mask = hugetlb_mask_last_page(h);
for (addr = src_vma->vm_start; addr < src_vma->vm_end; addr += sz) {
spinlock_t *src_ptl, *dst_ptl;
+
+ if (WARN_ON_ONCE(want_pmd_share(src_vma, addr) ||
+ want_pmd_share(dst_vma, addr))) {
+ ret = -EINVAL;
+ break;
+ }
+
src_pte = hugetlb_walk(src_vma, addr, sz);
if (!src_pte) {
addr |= last_addr_mask;
@@ -5297,14 +5335,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
break;
}
-#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING
- /* If the pagetables are shared, there is nothing to do */
- if (ptdesc_pmd_is_shared(virt_to_ptdesc(dst_pte))) {
- addr |= last_addr_mask;
- continue;
- }
-#endif
-
dst_ptl = huge_pte_lock(h, dst, dst_pte);
src_ptl = huge_pte_lockptr(h, src, src_pte);
spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING);
@@ -5478,10 +5508,9 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
spin_unlock(dst_ptl);
}
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len)
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
struct hstate *h = hstate_vma(vma);
struct address_space *mapping = vma->vm_file->f_mapping;
@@ -5776,6 +5805,8 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
struct address_space *mapping;
pgoff_t pgoff;
+ VM_WARN_ON_ONCE(!folio_test_anon(page_folio(page)));
+
/*
* vm_pgoff is in PAGE_SIZE units, hence the different calculation
* from page cache lookup which is in HPAGE_SIZE units.
@@ -5811,9 +5842,17 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
* could insert a zeroed page instead of the data existing
* from the time of fork. This would look like data corruption
*/
- if (!is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
- unmap_hugepage_range(iter_vma, address,
- address + huge_page_size(h), page, 0);
+ if (is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
+ continue;
+
+ /*
+ * The address in other VMAs can differ. Even though our old
+ * folio is anon, we always operate on the underlying hugetlb
+ * file.
+ */
+ address = vma_address(iter_vma, pgoff, 1);
+ unmap_hugepage_range(iter_vma, address,
+ address + huge_page_size(h), page, 0);
}
i_mmap_unlock_write(mapping);
}
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 7a54ce0f126b..1250d878d724 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2018,6 +2018,11 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr,
index += folio_nr_pages(folio);
continue;
out_unlock:
+ /*
+ * The folio may have been unmapped with TTU_BATCH_FLUSH.
+ * Flush before releasing the lock and our last reference.
+ */
+ try_to_unmap_flush();
folio_unlock(folio);
folio_put(folio);
goto xa_unlocked;
@@ -2028,9 +2033,8 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr,
xa_unlocked:
/*
- * If collapse is successful, flush must be done now before copying.
- * If collapse is unsuccessful, does flush actually need to be done?
- * Do it anyway, to clear the state.
+ * Flush before copying the folios, or releasing them in rollback.
+ * This is a no-op if out_unlock already flushed the batch.
*/
try_to_unmap_flush();
diff --git a/mm/ksm.c b/mm/ksm.c
index 15f558f69093..8ce4cd5a382c 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -470,6 +470,7 @@ static void advisor_stop_scan(void)
#ifdef CONFIG_NUMA
/* Zeroed when merging across nodes is not allowed */
static unsigned int ksm_merge_across_nodes = 1;
+/* Number of allocated stable and unstable tree roots. */
static int ksm_nr_node_ids = 1;
#else
#define ksm_merge_across_nodes 1U
@@ -1407,6 +1408,10 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,
dec_mm_counter(mm, MM_ANONPAGES);
}
+ /* Preserve soft-dirty when replacing the page. */
+ if (pte_soft_dirty(orig_pte))
+ newpte = pte_mksoft_dirty(newpte);
+
flush_cache_page(vma, addr, pte_pfn(ptep_get(ptep)));
/*
* No need to notify as we are replacing a read only page with another
@@ -3528,14 +3533,13 @@ static ssize_t merge_across_nodes_store(struct kobject *kobj,
else {
root_stable_tree = buf;
root_unstable_tree = buf + nr_node_ids;
+ ksm_nr_node_ids = nr_node_ids;
/* Stable tree is empty but not the unstable */
root_unstable_tree[0] = one_unstable_tree[0];
}
}
- if (!err) {
+ if (!err)
ksm_merge_across_nodes = knob;
- ksm_nr_node_ids = knob ? 1 : nr_node_ids;
- }
}
mutex_unlock(&ksm_thread_mutex);
diff --git a/mm/madvise.c b/mm/madvise.c
index ea966d402f07..6ce97106d6d4 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -752,6 +752,15 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
if (folio_test_swapcache(folio) || folio_test_dirty(folio)) {
if (!folio_trylock(folio))
continue;
+ /*
+ * KSM folios can gain new mappings at any time, so we must
+ * not clear their dirty flag.
+ */
+ if (folio_test_ksm(folio)) {
+ folio_unlock(folio);
+ continue;
+ }
+
/*
* If we have a large folio at this point, we know it is
* fully mapped so if its mapcount is the same as its
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index f7dd9bea5196..2aae088c8608 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -591,7 +591,7 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val)
static void do_flush_stats(struct mem_cgroup *memcg)
{
if (mem_cgroup_is_root(memcg))
- WRITE_ONCE(flush_last_time, jiffies_64);
+ WRITE_ONCE(flush_last_time, get_jiffies_64());
cgroup_rstat_flush(memcg->css.cgroup);
}
@@ -620,7 +620,7 @@ void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg)
{
/* Only flush if the periodic flusher is one full cycle late */
- if (time_after64(jiffies_64, READ_ONCE(flush_last_time) + 2*FLUSH_TIME))
+ if (time_after64(get_jiffies_64(), READ_ONCE(flush_last_time) + 2 * FLUSH_TIME))
mem_cgroup_flush_stats(memcg);
}
@@ -1928,7 +1928,10 @@ static void high_work_func(struct work_struct *work)
struct mem_cgroup *memcg;
memcg = container_of(work, struct mem_cgroup, high_work);
+ if (!css_tryget_online(&memcg->css))
+ return;
reclaim_high(memcg, MEMCG_CHARGE_BATCH, GFP_KERNEL);
+ css_put(&memcg->css);
}
/*
@@ -2179,10 +2182,11 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
bool drained = false;
bool raised_max_event = false;
unsigned long pflags;
+ int ret = 0;
retry:
if (consume_stock(memcg, nr_pages))
- return 0;
+ return ret;
if (!do_memsw_account() ||
page_counter_try_charge(&memcg->memsw, batch, &counter)) {
@@ -2294,16 +2298,11 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
* put the burden of reclaim on regular allocation requests
* and let these go through as privileged allocations.
*/
- if (!(gfp_mask & (__GFP_NOFAIL | __GFP_HIGH)))
- return -ENOMEM;
+ if (!(gfp_mask & (__GFP_NOFAIL | __GFP_HIGH))) {
+ ret = -ENOMEM;
+ goto out;
+ }
force:
- /*
- * If the allocation has to be enforced, don't forget to raise
- * a MEMCG_MAX event.
- */
- if (!raised_max_event)
- memcg_memory_event(mem_over_limit, MEMCG_MAX);
-
/*
* The allocation either can't fail or will lead to more memory
* being freed very soon. Allow memory usage go over the limit
@@ -2313,7 +2312,15 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
if (do_memsw_account())
page_counter_charge(&memcg->memsw, nr_pages);
- return 0;
+out:
+ /*
+ * Don't forget to raise a MEMCG_MAX event for forced or rejected
+ * requests.
+ */
+ if (!raised_max_event)
+ memcg_memory_event(mem_over_limit, MEMCG_MAX);
+
+ return ret;
done_restock:
if (batch > nr_pages)
@@ -2372,7 +2379,7 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
!(current->flags & PF_MEMALLOC) &&
gfpflags_allow_blocking(gfp_mask))
mem_cgroup_handle_over_high(gfp_mask);
- return 0;
+ return ret;
}
/**
@@ -3311,7 +3318,7 @@ void mem_cgroup_flush_foreign(struct bdi_writeback *wb)
{
struct mem_cgroup *memcg = mem_cgroup_from_css(wb->memcg_css);
unsigned long intv = msecs_to_jiffies(dirty_expire_interval * 10);
- u64 now = jiffies_64;
+ u64 now = get_jiffies_64();
int i;
for (i = 0; i < MEMCG_CGWB_FRN_CNT; i++) {
@@ -3684,6 +3691,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
page_counter_set_min(&memcg->memory, 0);
page_counter_set_low(&memcg->memory, 0);
+ page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX);
+ page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX);
zswap_memcg_offline_cleanup(memcg);
@@ -4498,6 +4507,51 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection);
}
+#ifdef CONFIG_LRU_GEN
+/**
+ * mem_cgroup_calculate_protection_path - compute protection along a path
+ * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup)
+ * @memcg: the target memory cgroup
+ *
+ * Walk the ancestor path from @root down to @memcg and compute the effective
+ * protection at each level. This is safe for isolated queries because it
+ * ensures parents are computed before children.
+ */
+void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg)
+{
+ bool recursive_protection =
+ cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT;
+ struct cgroup *cg;
+ int root_level, i;
+
+ if (mem_cgroup_disabled())
+ return;
+
+ if (!root)
+ root = root_mem_cgroup;
+
+ if (memcg == root)
+ return;
+
+ root_level = root->css.cgroup->level;
+ cg = memcg->css.cgroup;
+
+ rcu_read_lock();
+ for (i = root_level + 1; i <= cg->level; i++) {
+ struct mem_cgroup *cur;
+
+ cur = mem_cgroup_from_css(rcu_dereference(
+ cg->ancestors[i]->subsys[memory_cgrp_id]));
+ if (cur)
+ page_counter_calculate_protection(&root->memory,
+ &cur->memory,
+ recursive_protection);
+ }
+ rcu_read_unlock();
+}
+#endif /* CONFIG_LRU_GEN */
+
static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg,
gfp_t gfp)
{
diff --git a/mm/memfd.c b/mm/memfd.c
index 3e5a014fdacc..e41dd806ac50 100644
--- a/mm/memfd.c
+++ b/mm/memfd.c
@@ -465,6 +465,8 @@ SYSCALL_DEFINE2(memfd_create,
goto err_fd;
}
file->f_mode |= FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE;
+ /* a regular file like any other: POSIX.1-2008/SUSv4 Section XSI 2.9.7 */
+ file->f_mode |= FMODE_ATOMIC_POS;
file->f_flags |= O_LARGEFILE;
if (flags & MFD_NOEXEC_SEAL) {
diff --git a/mm/memory.c b/mm/memory.c
index c493f18cfb3a..38019497d8d4 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3046,6 +3046,13 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src,
return 0;
}
+ /*
+ * __copy_from_user_inatomic() only works as expected for local faults.
+ * Reject remote faults.
+ */
+ if (unlikely(mm != current->mm))
+ return -EFAULT;
+
/*
* If the source page was a PFN mapping, we don't have
* a "struct page" for it. We do a best-effort copy by
@@ -3371,6 +3378,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
folio_put(old_folio);
delayacct_wpcopy_end();
+ if (err == -EFAULT)
+ return VM_FAULT_SIGBUS;
return err == -EHWPOISON ? VM_FAULT_HWPOISON : 0;
}
kmsan_copy_page_meta(&new_folio->page, vmf->page);
@@ -3858,8 +3867,8 @@ EXPORT_SYMBOL_GPL(unmap_mapping_pages);
void unmap_mapping_range(struct address_space *mapping,
loff_t const holebegin, loff_t const holelen, int even_cows)
{
- pgoff_t hba = (pgoff_t)(holebegin) >> PAGE_SHIFT;
- pgoff_t hlen = ((pgoff_t)(holelen) + PAGE_SIZE - 1) >> PAGE_SHIFT;
+ pgoff_t hba = (u64)holebegin >> PAGE_SHIFT;
+ pgoff_t hlen = ((u64)holelen + PAGE_SIZE - 1) >> PAGE_SHIFT;
/* Check for overflow. */
if (sizeof(holelen) > sizeof(hlen)) {
@@ -6112,8 +6121,12 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
lru_gen_exit_fault();
- /* If the mapping is droppable, then errors due to OOM aren't fatal. */
- if (is_droppable)
+ /*
+ * If the mapping is droppable, then errors due to OOM aren't fatal: the
+ * task returns to user mode, handles its signals and tries again. A
+ * fault from kernel mode has no such return, so it must see the error.
+ */
+ if (is_droppable && (flags & FAULT_FLAG_USER))
ret &= ~VM_FAULT_OOM;
if (flags & FAULT_FLAG_USER) {
@@ -6554,14 +6567,34 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
resource_size_t phys_addr;
unsigned long prot = 0;
void __iomem *maddr;
+ struct page *page;
int offset = offset_in_page(addr);
int ret = -EINVAL;
bool writable;
struct follow_pfnmap_args args = { .vma = vma, .address = addr };
+ /*
+ * Limit access to one page at a time, as that's what follow_pfnmap_start()
+ * guarantees; expect the caller to retry to read larger ranges.
+ */
+ len = min_t(int, len, PAGE_SIZE - offset);
+
retry:
if (follow_pfnmap_start(&args))
return -EINVAL;
+
+ /*
+ * Reject anything that looks like ordinary RAM, including anonymous
+ * folios in CoW mappings or allocated kernel memory. Note that we
+ * cannot distinguish memblock allocations from non-RAM reliably, so
+ * we cannot easily reject them here.
+ */
+ page = pfn_to_online_page(args.pfn);
+ if (page && !PageReserved(page)) {
+ follow_pfnmap_end(&args);
+ return -EINVAL;
+ }
+
prot = pgprot_val(args.pgprot);
phys_addr = (resource_size_t)args.pfn << PAGE_SHIFT;
writable = args.writable;
@@ -6570,7 +6603,7 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
if ((write & FOLL_WRITE) && !writable)
return -EINVAL;
- maddr = ioremap_prot(phys_addr, PAGE_ALIGN(len + offset), prot);
+ maddr = ioremap_prot(phys_addr, PAGE_SIZE, prot);
if (!maddr)
return -ENOMEM;
@@ -6578,7 +6611,7 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
goto out_unmap;
if ((prot != pgprot_val(args.pgprot)) ||
- (phys_addr != (args.pfn << PAGE_SHIFT)) ||
+ (phys_addr != ((resource_size_t)args.pfn << PAGE_SHIFT)) ||
(writable != args.writable)) {
follow_pfnmap_end(&args);
iounmap(maddr);
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 3853b6ff7117..193ebf34bf08 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -500,7 +500,8 @@ static inline bool queue_folio_required(struct folio *folio,
return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
}
-static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
+static void queue_folios_pmd(pmd_t *pmd, unsigned long addr,
+ struct mm_walk *walk)
{
struct folio *folio;
struct queue_pages *qp = walk->private;
@@ -509,11 +510,9 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
qp->nr_failed++;
return;
}
- folio = pmd_folio(*pmd);
- if (is_huge_zero_folio(folio)) {
- walk->action = ACTION_CONTINUE;
+ folio = vm_normal_folio_pmd(walk->vma, addr, *pmd);
+ if (!folio || folio_is_zone_device(folio))
return;
- }
if (!queue_folio_required(folio, qp))
return;
if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
@@ -545,7 +544,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
ptl = pmd_trans_huge_lock(pmd, vma);
if (ptl) {
- queue_folios_pmd(pmd, walk);
+ queue_folios_pmd(pmd, addr, walk);
spin_unlock(ptl);
goto out;
}
@@ -3099,11 +3098,11 @@ void __init numa_policy_init(void)
policy_cache = kmem_cache_create("numa_policy",
sizeof(struct mempolicy),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
sn_cache = kmem_cache_create("shared_policy_node",
sizeof(struct sp_node),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
for_each_node(nid) {
preferred_node_policy[nid] = (struct mempolicy) {
diff --git a/mm/migrate.c b/mm/migrate.c
index e07a098671ce..66af1dd6a941 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2411,7 +2411,7 @@ static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages,
mmap_read_lock(mm);
for (i = 0; i < nr_pages; i++) {
- unsigned long addr = (unsigned long)(*pages);
+ unsigned long addr = (unsigned long)untagged_addr_remote(mm, *pages);
struct vm_area_struct *vma;
struct folio_walk fw;
struct folio *folio;
@@ -2700,6 +2700,7 @@ int migrate_misplaced_folio(struct folio *folio, struct vm_area_struct *vma,
pg_data_t *pgdat = NODE_DATA(node);
int nr_remaining;
unsigned int nr_succeeded;
+ int src_node = folio_nid(folio);
LIST_HEAD(migratepages);
struct mem_cgroup *memcg = get_mem_cgroup_from_folio(folio);
struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat);
@@ -2714,7 +2715,7 @@ int migrate_misplaced_folio(struct folio *folio, struct vm_area_struct *vma,
count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded);
count_memcg_events(memcg, NUMA_PAGE_MIGRATE, nr_succeeded);
if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING)
- && !node_is_toptier(folio_nid(folio))
+ && !node_is_toptier(src_node)
&& node_is_toptier(node))
mod_lruvec_state(lruvec, PGPROMOTE_SUCCESS, nr_succeeded);
}
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 77f5f62628b7..f024ab7ae363 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -2481,8 +2481,6 @@ void __init memblock_free_pages(struct page *page, unsigned long pfn,
return;
}
- /* pages were reserved and not allocated */
- clear_page_tag_ref(page);
__free_pages_core(page, order, MEMINIT_EARLY);
}
diff --git a/mm/mmap.c b/mm/mmap.c
index 2d69552e6be3..c930e03bd799 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1415,14 +1415,17 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,
* Private writable mapping: check memory availability
*/
if (accountable_mapping(file, vm_flags)) {
- charged = pglen;
- charged -= vms.nr_accounted;
- if (charged) {
- error = security_vm_enough_memory_mm(mm, charged);
+ const unsigned long uncharged = pglen - vms.nr_accounted;
+
+ /* Charge only what's not already charged by mappings we replace. */
+ if (uncharged) {
+ error = security_vm_enough_memory_mm(mm, uncharged);
if (error)
goto abort_munmap;
}
+ /* Our new mapping owns the entire charge now. */
+ charged = pglen;
vms.nr_accounted = 0;
vm_flags |= VM_ACCOUNT;
vmg.flags = vm_flags;
diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c
index fc18fe274505..1c2707a8ed76 100644
--- a/mm/mmu_notifier.c
+++ b/mm/mmu_notifier.c
@@ -522,16 +522,25 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
{
struct mmu_notifier_subscriptions *subscriptions =
range->mm->notifier_subscriptions;
- int ret;
+ const bool has_itree = subscriptions->has_itree;
+ int ret = 0;
- if (subscriptions->has_itree) {
+ if (has_itree) {
ret = mn_itree_invalidate(subscriptions, range);
if (ret)
return ret;
}
if (!hlist_empty(&subscriptions->list))
- return mn_hlist_invalidate_range_start(subscriptions, range);
- return 0;
+ ret = mn_hlist_invalidate_range_start(subscriptions, range);
+
+ /*
+ * On error, the non-blocking caller is not allowed to call
+ * invalidate_range_end(), as in mn_itree_invalidate().
+ */
+ if (ret && has_itree && !mmu_notifier_range_blockable(range))
+ mn_itree_inv_end(subscriptions);
+
+ return ret;
}
static void
@@ -892,7 +901,6 @@ static int __mmu_interval_notifier_insert(
struct mmu_notifier_subscriptions *subscriptions, unsigned long start,
unsigned long length, const struct mmu_interval_notifier_ops *ops)
{
- interval_sub->mm = mm;
interval_sub->ops = ops;
RB_CLEAR_NODE(&interval_sub->interval_tree.rb);
interval_sub->interval_tree.start = start;
@@ -909,8 +917,12 @@ static int __mmu_interval_notifier_insert(
if (WARN_ON(atomic_read(&mm->mm_users) <= 0))
return -EINVAL;
- /* pairs with mmdrop in mmu_interval_notifier_remove() */
+ /*
+ * Pairs with mmdrop in mmu_interval_notifier_remove(). Drivers take a
+ * non-NULL ->mm for "registered", so set it only with the reference.
+ */
mmgrab(mm);
+ interval_sub->mm = mm;
/*
* If some invalidate_range_start/end region is going on in parallel
diff --git a/mm/mremap.c b/mm/mremap.c
index 8b507e228a27..d4666f1ab593 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -782,6 +782,21 @@ static unsigned long move_vma(struct vm_area_struct *vma,
*/
move_page_tables(new_vma, new_addr, vma, old_addr, moved_len,
true, false);
+
+ /*
+ * When the error path completes, the new VMA will be
+ * unmapped rather than the old one.
+ *
+ * When an accountable move is being performed, this unmapping
+ * will not uncharge any expansion, so do that now.
+ *
+ * A move is accountable if the VMA is accountable and
+ * MREMAP_DONTUNMAP is not specified, so only do so if the
+ * latter is true.
+ */
+ if (vm_flags & VM_ACCOUNT && !(flags & MREMAP_DONTUNMAP))
+ vm_unacct_memory(to_account >> PAGE_SHIFT);
+
vma = new_vma;
old_len = new_len;
old_addr = new_addr;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index bfb3f903bb6d..89b2e4a290b5 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -917,7 +917,9 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
unsigned long thresh)
{
struct wb_domain *dom = dtc_dom(dtc);
+ u64 threshold = thresh;
u64 wb_thresh;
+ u64 wb_max_thresh;
unsigned long numerator, denominator;
unsigned long wb_min_ratio, wb_max_ratio;
@@ -927,15 +929,17 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
fprop_fraction_percpu(&dom->completions, dtc->wb_completions,
&numerator, &denominator);
- wb_thresh = (thresh * (100 * BDI_RATIO_SCALE - bdi_min_ratio)) / (100 * BDI_RATIO_SCALE);
+ wb_thresh = div_u64(threshold * (100 * BDI_RATIO_SCALE - bdi_min_ratio),
+ 100 * BDI_RATIO_SCALE);
wb_thresh *= numerator;
wb_thresh = div64_ul(wb_thresh, denominator);
wb_min_max_ratio(dtc->wb, &wb_min_ratio, &wb_max_ratio);
- wb_thresh += (thresh * wb_min_ratio) / (100 * BDI_RATIO_SCALE);
- if (wb_thresh > (thresh * wb_max_ratio) / (100 * BDI_RATIO_SCALE))
- wb_thresh = thresh * wb_max_ratio / (100 * BDI_RATIO_SCALE);
+ wb_thresh += div_u64(threshold * wb_min_ratio, 100 * BDI_RATIO_SCALE);
+ wb_max_thresh = div_u64(threshold * wb_max_ratio, 100 * BDI_RATIO_SCALE);
+ if (wb_thresh > wb_max_thresh)
+ wb_thresh = wb_max_thresh;
return wb_thresh;
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a649f0245e3b..869296e43349 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1355,6 +1355,9 @@ void __meminit __free_pages_core(struct page *page, unsigned int order,
atomic_long_add(nr_pages, &page_zone(page)->managed_pages);
}
+ /* pages were never allocated */
+ clear_page_tag_ref(page);
+
if (page_contains_unaccepted(page, order)) {
if (order == MAX_PAGE_ORDER && __free_unaccepted(page))
return;
@@ -6682,6 +6685,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_mask, gfp_t *gfp_cc_mask)
int alloc_contig_range_noprof(unsigned long start, unsigned long end,
unsigned migratetype, gfp_t gfp_mask)
{
+ const unsigned int order = ilog2(end - start);
unsigned long outer_start, outer_end;
int ret = 0;
@@ -6696,6 +6700,14 @@ int alloc_contig_range_noprof(unsigned long start, unsigned long end,
};
INIT_LIST_HEAD(&cc.migratepages);
+ /*
+ * In contrast to the buddy, we allow for orders here that exceed
+ * MAX_PAGE_ORDER, so we must manually make sure that we are not
+ * exceeding the maximum folio order.
+ */
+ if (WARN_ON_ONCE((gfp_mask & __GFP_COMP) && order > MAX_FOLIO_ORDER))
+ return -EINVAL;
+
gfp_mask = current_gfp_context(gfp_mask);
if (__alloc_contig_verify_gfp_mask(gfp_mask, (gfp_t *)&cc.gfp_mask))
return -EINVAL;
@@ -6740,7 +6752,17 @@ int alloc_contig_range_noprof(unsigned long start, unsigned long end,
ret = __alloc_contig_migrate_range(&cc, start, end, migratetype);
if (ret && ret != -EBUSY)
goto done;
- ret = 0;
+
+ /*
+ * When in-use hugetlb pages are migrated, they may simply be released
+ * back into the free hugepage pool instead of being returned to the
+ * buddy system. After the migration of in-use huge pages is completed,
+ * we will invoke replace_free_hugepage_folios() to ensure that these
+ * hugepages are properly released to the buddy system.
+ */
+ ret = replace_free_hugepage_folios(start, end);
+ if (ret)
+ goto done;
/*
* Pages from [start, end) are within a pageblock_nr_pages
@@ -6783,7 +6805,6 @@ int alloc_contig_range_noprof(unsigned long start, unsigned long end,
free_contig_range(end, outer_end - end);
} else if (start == outer_start && end == outer_end && is_power_of_2(end - start)) {
struct page *head = pfn_to_page(start);
- int order = ilog2(end - start);
check_new_pages(head, order);
prep_new_page(head, order, gfp_mask, 0);
diff --git a/mm/page_counter.c b/mm/page_counter.c
index b249d15af9dd..9203b448e4f5 100644
--- a/mm/page_counter.c
+++ b/mm/page_counter.c
@@ -8,6 +8,7 @@
#include <linux/page_counter.h>
#include <linux/atomic.h>
#include <linux/kernel.h>
+#include <linux/math64.h>
#include <linux/string.h>
#include <linux/sched.h>
#include <linux/bug.h>
@@ -354,7 +355,8 @@ static unsigned long effective_protection(unsigned long usage,
* otherwise get a smaller chunk than what they claimed.
*/
if (siblings_protected > parent_effective)
- return protected * parent_effective / siblings_protected;
+ return mul_u64_u64_div_u64(protected, parent_effective,
+ siblings_protected);
/*
* Ok, utilized protection of all children is within what the
@@ -395,13 +397,20 @@ static unsigned long effective_protection(unsigned long usage,
if (parent_effective > siblings_protected &&
parent_usage > siblings_protected &&
usage > protected) {
- unsigned long unclaimed;
+ unsigned long parent_unclaimed, parent_unprotected, unprotected;
- unclaimed = parent_effective - siblings_protected;
- unclaimed *= usage - protected;
- unclaimed /= parent_usage - siblings_protected;
+ parent_unclaimed = parent_effective - siblings_protected;
+ parent_unprotected = parent_usage - siblings_protected;
- ep += unclaimed;
+ /*
+ * The usages aren't read atomically, so a child can transiently
+ * appear to use more than its parent, making the ratio exceed 1
+ * and the quotient overflow 64 bits (#DE on x86). Cap it.
+ */
+ unprotected = min(usage - protected, parent_unprotected);
+
+ ep += mul_u64_u64_div_u64(parent_unclaimed, unprotected,
+ parent_unprotected);
}
return ep;
diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index 6989c5ffd474..b5ceebc8e744 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -376,13 +376,19 @@ static int isolate_single_pageblock(unsigned long boundary_pfn, int flags,
}
if (PageBuddy(page)) {
- int order = buddy_order(page);
+ unsigned int order = buddy_order_unsafe(page);
- /* move_freepages_block_isolate() handled this */
- VM_WARN_ON_ONCE(pfn + (1 << order) > boundary_pfn);
-
- pfn += 1UL << order;
- continue;
+ /* buddy_order_unsafe() is racy. Validate the order before shifting. */
+ if (order <= MAX_PAGE_ORDER &&
+ /*
+ * move_freepages_block_isolate() splits
+ * cross-boundary PageBuddy, verify it.
+ */
+ pfn + (1UL << order) <= boundary_pfn) {
+ pfn += 1UL << order;
+ continue;
+ }
+ goto failed;
}
/*
@@ -401,10 +407,28 @@ static int isolate_single_pageblock(unsigned long boundary_pfn, int flags,
if (PageCompound(page)) {
struct page *head = compound_head(page);
unsigned long head_pfn = page_to_pfn(head);
- unsigned long nr_pages = compound_nr(head);
+ unsigned int order = compound_order(head);
+ unsigned long nr_pages;
+
+ /* compound_order() is racy. Cap it at MAX_FOLIO_ORDER. */
+ if (order > MAX_FOLIO_ORDER)
+ goto failed;
+
+ nr_pages = 1UL << order;
+
+ /*
+ * compound_head() is also racy, so the derived head_pfn
+ * needs additional checks to make sure it is valid.
+ * Otherwise, just fail the check. pfn comes from
+ * __first_valid_page() as a legitimate PFN, so use it to
+ * check head_pfn.
+ */
+ if (head_pfn > pfn || !IS_ALIGNED(head_pfn, nr_pages) ||
+ pfn - head_pfn >= nr_pages)
+ goto failed;
if (head_pfn + nr_pages <= boundary_pfn ||
- PageHuge(page)) {
+ PageHuge(head)) {
pfn = head_pfn + nr_pages;
continue;
}
diff --git a/mm/page_owner.c b/mm/page_owner.c
index 5d1c946c0008..d2783b7c0661 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -937,7 +937,7 @@ static const struct file_operations page_owner_stack_operations = {
.open = page_owner_stack_open,
.read = seq_read,
.llseek = seq_lseek,
- .release = seq_release_private,
+ .release = seq_release,
};
static int page_owner_threshold_get(void *data, u64 *val)
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 2efa910eef23..1076f8e055f3 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -67,7 +67,8 @@ static void page_table_check_clear(unsigned long pfn, unsigned long pgcnt)
unsigned long i;
bool anon;
- if (!pfn_valid(pfn))
+ if (!pfn_valid(pfn) || is_zero_pfn(pfn) ||
+ is_huge_zero_folio(pfn_folio(pfn)))
return;
page = pfn_to_page(pfn);
@@ -107,7 +108,8 @@ static void page_table_check_set(unsigned long pfn, unsigned long pgcnt,
unsigned long i;
bool anon;
- if (!pfn_valid(pfn))
+ if (!pfn_valid(pfn) || is_zero_pfn(pfn) ||
+ is_huge_zero_folio(pfn_folio(pfn)))
return;
page = pfn_to_page(pfn);
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index 1cc8fd382cd1..67e6a32644ba 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -255,18 +255,18 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
if (!thp_migration_supported() ||
!(pvmw->flags & PVMW_MIGRATION))
- return not_found(pvmw);
+ goto next_pmd;
entry = pmd_to_swp_entry(pmde);
if (!is_migration_entry(entry) ||
!check_pmd(swp_offset_pfn(entry), pvmw))
- return not_found(pvmw);
+ goto next_pmd;
return true;
}
if (likely(pmd_trans_huge(pmde) || pmd_devmap(pmde))) {
if (pvmw->flags & PVMW_MIGRATION)
- return not_found(pvmw);
+ goto next_pmd;
if (!check_pmd(pmd_pfn(pmde), pvmw))
- return not_found(pvmw);
+ goto next_pmd;
return true;
}
/* THP pmd was split under us: handle on pte level */
@@ -286,8 +286,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
spin_unlock(ptl);
}
- step_forward(pvmw, PMD_SIZE);
- continue;
+ goto next_pmd;
}
if (!map_pte(pvmw, &ptl)) {
if (!pvmw->pte)
@@ -320,6 +319,17 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
spin_lock(pvmw->ptl);
}
goto this_pte;
+next_pmd:
+ /*
+ * A large folio can be PTE-mapped across PMDs, so a
+ * nonmatching PMD does not end the walk.
+ */
+ if (pvmw->ptl) {
+ spin_unlock(pvmw->ptl);
+ pvmw->ptl = NULL;
+ }
+ pvmw->pmd = NULL;
+ step_forward(pvmw, PMD_SIZE);
} while (pvmw->address < end);
return false;
diff --git a/mm/shmem.c b/mm/shmem.c
index 4d01fc877e76..8de38d7a98f9 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -75,6 +75,7 @@ static struct vfsmount *shm_mnt __ro_after_init;
#include <linux/magic.h>
#include <linux/syscalls.h>
#include <linux/fcntl.h>
+#include <linux/memfd.h>
#include <uapi/linux/memfd.h>
#include <linux/rmap.h>
#include <linux/uuid.h>
@@ -985,12 +986,10 @@ void shmem_unlock_mapping(struct address_space *mapping)
static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
{
+ const gfp_t gfp = mapping_gfp_mask(inode->i_mapping);
struct folio *folio;
- /*
- * At first avoid shmem_get_folio(,,,SGP_READ): that fails
- * beyond i_size, and reports fallocated folios as holes.
- */
+retry:
folio = filemap_get_entry(inode->i_mapping, index);
if (!folio)
return folio;
@@ -1001,13 +1000,14 @@ static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
/* The folio has been swapped out */
folio_unlock(folio);
folio_put(folio);
+ goto retry;
}
/*
- * But read a folio back from swap if any of it is within i_size
- * (although in some cases this is just a waste of time).
+ * Truncation can look up a large swap entry at or beyond i_size.
+ * Swap it in directly to bypass the size check in shmem_get_folio().
*/
- folio = NULL;
- shmem_get_folio(inode, index, 0, &folio, SGP_READ);
+ if (shmem_swapin_folio(inode, index, &folio, SGP_READ, gfp, NULL, NULL) == -EEXIST)
+ goto retry;
return folio;
}
@@ -1126,14 +1126,16 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend,
order = shmem_confirm_swap(mapping, indices[i],
radix_to_swp_entry(folio));
/*
- * If found a large swap entry cross the end or start
- * border, skip it as the truncate_inode_partial_folio
- * above should have at least zerod its content once.
+ * Skip boundary entries when restarting
+ * the scan to avoid endless loops after
+ * swapin failure.
*/
if (order > 0) {
base = round_down(base, 1 << order);
- if (base < start || base + (1 << order) > end)
+ if (base < start || base + (1 << order) > end) {
+ start = max(start, base + (1 << order));
continue;
+ }
}
/* Swap was replaced by page or extended, retry */
index = base;
@@ -1155,7 +1157,16 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend,
VM_BUG_ON_FOLIO(folio_test_writeback(folio),
folio);
- if (!folio_test_large(folio)) {
+ /*
+ * A shared mapping can have dirty PTEs while
+ * the folio is still clean. Mark the folio
+ * dirty so a failed split cannot discard data
+ * outside the range.
+ */
+ if (!unfalloc && folio_test_large(folio))
+ folio_mark_dirty(folio);
+
+ if (unfalloc || !folio_test_large(folio)) {
truncate_inode_folio(mapping, folio);
} else if (truncate_inode_partial_folio(folio, lstart, lend)) {
/*
@@ -1171,6 +1182,15 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend,
index = start;
break;
}
+ } else if (folio->mapping) {
+ /*
+ * Splitting failed, but the part in the
+ * range is already zeroed. The folio
+ * was not dropped, so skip it when
+ * restarting the scan to avoid retrying
+ * it forever.
+ */
+ start = max(start, folio_next_index(folio));
}
}
folio_unlock(folio);
@@ -1236,10 +1256,11 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (error)
return error;
- if ((info->seals & F_SEAL_EXEC) && (attr->ia_valid & ATTR_MODE)) {
- if ((inode->i_mode ^ attr->ia_mode) & 0111) {
- return -EPERM;
- }
+ if (attr->ia_valid & ATTR_MODE) {
+ error = memfd_check_seals_mode(info->seals, inode->i_mode,
+ attr->ia_mode);
+ if (error)
+ return error;
}
if (S_ISREG(inode->i_mode) && (attr->ia_valid & ATTR_SIZE)) {
@@ -1266,7 +1287,8 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (oldsize > holebegin)
unmap_mapping_range(inode->i_mapping,
holebegin, 0, 1);
- if (info->alloced)
+ /* Swapin-error markers are not counted in alloced. */
+ if (info->alloced || !mapping_empty(inode->i_mapping))
shmem_truncate_range(inode,
newsize, (loff_t)-1);
/* unmap again to remove racily COWed private pages */
@@ -1435,6 +1457,10 @@ static int shmem_unuse_inode(struct inode *inode, unsigned int type)
if (ret < 0)
break;
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
start = indices[folio_batch_count(&fbatch) - 1];
} while (true);
@@ -4915,12 +4941,35 @@ static const struct file_operations shmem_file_operations = {
#endif
};
+#ifdef CONFIG_TMPFS_POSIX_ACL
+static int shmem_set_acl(struct mnt_idmap *idmap, struct dentry *dentry,
+ struct posix_acl *acl, int type)
+{
+ struct inode *inode = d_inode(dentry);
+ umode_t mode = inode->i_mode;
+ int error;
+
+ if (type == ACL_TYPE_ACCESS) {
+ error = posix_acl_equiv_mode(acl, &mode);
+ if (error < 0)
+ return error;
+ error = memfd_check_seals_mode(SHMEM_I(inode)->seals,
+ inode->i_mode, mode);
+ if (error)
+ return error;
+ }
+ return simple_set_acl(idmap, dentry, acl, type);
+}
+#else
+#define shmem_set_acl simple_set_acl
+#endif
+
static const struct inode_operations shmem_inode_operations = {
.getattr = shmem_getattr,
.setattr = shmem_setattr,
#ifdef CONFIG_TMPFS_XATTR
.listxattr = shmem_listxattr,
- .set_acl = simple_set_acl,
+ .set_acl = shmem_set_acl,
.fileattr_get = shmem_fileattr_get,
.fileattr_set = shmem_fileattr_set,
#endif
diff --git a/mm/shmem_quota.c b/mm/shmem_quota.c
index d1e32ac01407..868b3e05f1aa 100644
--- a/mm/shmem_quota.c
+++ b/mm/shmem_quota.c
@@ -234,7 +234,7 @@ static int shmem_acquire_dquot(struct dquot *dquot)
return ret;
}
-static bool shmem_is_empty_dquot(struct dquot *dquot)
+static bool shmem_is_default_dquot(struct dquot *dquot)
{
struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info;
qsize_t bhardlimit;
@@ -248,19 +248,21 @@ static bool shmem_is_empty_dquot(struct dquot *dquot)
ihardlimit = sbinfo->qlimits.grpquota_ihardlimit;
}
- if (test_bit(DQ_FAKE_B, &dquot->dq_flags) ||
- (dquot->dq_dqb.dqb_curspace == 0 &&
- dquot->dq_dqb.dqb_curinodes == 0 &&
- dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
- dquot->dq_dqb.dqb_ihardlimit == ihardlimit))
- return true;
-
- return false;
+ /*
+ * Defaults are zero usage and soft limits, and the mount's hard limits.
+ * DQ_FAKE_B only means that *all* limits are zero, which need not match
+ * the mount defaults.
+ */
+ return dquot->dq_dqb.dqb_curspace == 0 &&
+ dquot->dq_dqb.dqb_curinodes == 0 &&
+ dquot->dq_dqb.dqb_bsoftlimit == 0 &&
+ dquot->dq_dqb.dqb_isoftlimit == 0 &&
+ dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
+ dquot->dq_dqb.dqb_ihardlimit == ihardlimit;
}
/*
- * Store limits from dquot in the tree unless it's fake. If it is fake
- * remove the id from the tree since there is no useful information in
- * there.
+ * Remove the ID from the tree if it has no usage and its limits match
+ * the mount defaults. Otherwise, save its limits for the next acquire.
*/
static int shmem_release_dquot(struct dquot *dquot)
{
@@ -295,7 +297,7 @@ static int shmem_release_dquot(struct dquot *dquot)
return -ENOENT;
found:
- if (shmem_is_empty_dquot(dquot)) {
+ if (shmem_is_default_dquot(dquot)) {
/* Remove entry from the tree */
rb_erase(&entry->node, info->dqi_priv);
kfree(entry);
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 4ea0ebefe92d..99eb5878f5ac 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2262,6 +2262,8 @@ static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud,
pmd = pmd_offset(pud, addr);
do {
cond_resched();
+ if (signal_pending(current))
+ return -ERESTARTSYS;
next = pmd_addr_end(addr, end);
ret = unuse_pte_range(vma, pmd, addr, next, type);
if (ret)
@@ -2475,7 +2477,7 @@ static int try_to_unuse(unsigned int type)
if (READ_ONCE(si->inuse_pages)) {
if (!signal_pending(current))
goto retry;
- return -EINTR;
+ return -ERESTARTSYS;
}
success:
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 9951b4f42c65..29d2366d88fa 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -48,8 +48,7 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
vma = vma_lookup(mm, addr);
if (!vma)
vma = ERR_PTR(-ENOENT);
- else if (!(vma->vm_flags & VM_SHARED) &&
- unlikely(anon_vma_prepare(vma)))
+ else if (is_cow_mapping(vma->vm_flags) && unlikely(anon_vma_prepare(vma)))
vma = ERR_PTR(-ENOMEM);
return vma;
@@ -77,7 +76,7 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
* We know we're going to need to use anon_vma, so check
* that early.
*/
- if (!(vma->vm_flags & VM_SHARED) && unlikely(!vma->anon_vma))
+ if (is_cow_mapping(vma->vm_flags) && unlikely(!vma->anon_vma))
vma_end_read(vma);
else
return vma;
@@ -1323,8 +1322,12 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd,
pte_unmap(dst_pte);
src_pte = dst_pte = NULL;
err = split_folio(src_folio);
- if (err)
+ if (err) {
+ /* do not retry, extra refs may be held forever */
+ if (err == -EAGAIN)
+ err = -EBUSY;
goto out;
+ }
/* have to reacquire the folio after it got split */
folio_unlock(src_folio);
folio_put(src_folio);
diff --git a/mm/vma.c b/mm/vma.c
index 66b0472b7d8a..85660d67b794 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -1694,7 +1694,10 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
faulted_in_anon_vma = true;
}
- new_vma = vma_merge_new_range(&vmg);
+ /* If not registered for remap, uffd context will be cleared. */
+ new_vma = NULL;
+ if (!vma_has_uffd_without_event_remap(vma))
+ new_vma = vma_merge_new_range(&vmg);
if (new_vma) {
/*
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 512a31d7ae9e..b248dcb88369 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3600,7 +3600,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
walk->mm_stats[MM_NONLEAF_FOUND]++;
- if (!walk_pte_range(&val, addr, next, args))
+ if (!walk_pte_range(pmd + i, addr, next, args))
continue;
walk->mm_stats[MM_NONLEAF_ADDED]++;
@@ -3636,7 +3636,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long start, unsigned long end,
if (!pud_present(val) || WARN_ON_ONCE(pud_leaf(val)))
continue;
- walk_pmd_range(&val, addr, next, args);
+ walk_pmd_range(pud + i, addr, next, args);
if (need_resched() || walk->batched >= MAX_LRU_BATCH) {
end = (addr | ~PUD_MASK) + 1;
@@ -4752,10 +4752,6 @@ static bool should_abort_scan(struct lruvec *lruvec, struct scan_control *sc)
int i;
enum zone_watermarks mark;
- /* don't abort memcg reclaim to ensure fairness */
- if (!root_reclaim(sc))
- return false;
-
if (sc->nr_reclaimed >= max(sc->nr_to_reclaim, compact_gap(sc->order)))
return true;
@@ -4839,7 +4835,13 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc)
struct mem_cgroup *memcg = lruvec_memcg(lruvec);
struct pglist_data *pgdat = lruvec_pgdat(lruvec);
- /* lru_gen_age_node() called mem_cgroup_calculate_protection() */
+ /*
+ * For kswapd, mem_cgroup_calculate_protection() has already
+ * been called during the top-down cgroup traversal.
+ */
+ if (!current_is_kswapd())
+ mem_cgroup_calculate_protection_path(NULL, memcg);
+
if (mem_cgroup_below_min(NULL, memcg))
return MEMCG_LRU_YOUNG;
diff --git a/mm/zswap.c b/mm/zswap.c
index 6c50249d5398..61311c92bd6f 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -1778,7 +1778,7 @@ static int zswap_setup(void)
if (pool) {
pr_info("loaded using pool %s/%s\n", pool->tfm_name,
zpool_get_type(pool->zpool));
- list_add(&pool->list, &zswap_pools);
+ list_add_rcu(&pool->list, &zswap_pools);
zswap_has_pool = true;
} else {
pr_err("pool creation failed\n");
diff --git a/net/9p/trans_usbg.c b/net/9p/trans_usbg.c
index 468f7e8f0277..938e3130ab81 100644
--- a/net/9p/trans_usbg.c
+++ b/net/9p/trans_usbg.c
@@ -148,7 +148,8 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
- struct p9_req_t *p9_tx_req = req->context;
+ struct p9_client *client;
+ struct p9_req_t *p9_tx_req;
unsigned long flags;
/* reset zero packages */
@@ -164,18 +165,25 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
ep->name, req->status, req->actual, req->length);
spin_lock_irqsave(&usb9pfs->lock, flags);
- WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+ client = usb9pfs->client;
+ p9_tx_req = req->context;
+ req->context = NULL;
- p9_req_put(usb9pfs->client, p9_tx_req);
+ if (!client || !p9_tx_req)
+ goto unlock_complete;
- req->context = NULL;
+ WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+
+ p9_req_put(client, p9_tx_req);
+unlock_complete:
spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->send);
}
-static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
+static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs,
+ struct p9_client *client, void *buf)
{
struct p9_req_t *p9_rx_req;
struct p9_fcall rc;
@@ -201,7 +209,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
"mux %p pkt: size: %d bytes tag: %d\n",
usb9pfs, rc.size, rc.tag);
- p9_rx_req = p9_tag_lookup(usb9pfs->client, rc.tag);
+ p9_rx_req = p9_tag_lookup(client, rc.tag);
if (!p9_rx_req || p9_rx_req->status != REQ_STATUS_SENT) {
p9_debug(P9_DEBUG_ERROR, "Unexpected packet tag %d\n", rc.tag);
return NULL;
@@ -211,7 +219,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"requested packet size too big: %d for tag %d with capacity %zd\n",
rc.size, rc.tag, p9_rx_req->rc.capacity);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -219,7 +227,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"No recv fcall for tag %d (req %p), disconnecting!\n",
rc.tag, p9_rx_req);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -230,8 +238,10 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
+ struct p9_client *client;
struct p9_req_t *p9_rx_req;
unsigned int req_size = req->actual;
+ unsigned long flags;
int status = REQ_STATUS_RCVD;
if (req->status) {
@@ -240,9 +250,16 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
return;
}
- p9_rx_req = usb9pfs_rx_header(usb9pfs, req->buf);
- if (!p9_rx_req)
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ if (!client) {
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
return;
+ }
+
+ p9_rx_req = usb9pfs_rx_header(usb9pfs, client, req->buf);
+ if (!p9_rx_req)
+ goto out_unlock;
if (req_size > p9_rx_req->rc.capacity) {
dev_err(&cdev->gadget->dev,
@@ -256,8 +273,11 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
p9_rx_req->rc.size = req_size;
- p9_client_cb(usb9pfs->client, p9_rx_req, status);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_client_cb(client, p9_rx_req, status);
+ p9_req_put(client, p9_rx_req);
+
+out_unlock:
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->received);
}
@@ -277,18 +297,18 @@ static void disable_usb9pfs(struct f_usb9pfs *usb9pfs)
struct usb_composite_dev *cdev =
usb9pfs->function.config->cdev;
+ disable_ep(cdev, usb9pfs->in_ep);
+ disable_ep(cdev, usb9pfs->out_ep);
+
if (usb9pfs->in_req) {
usb_ep_free_request(usb9pfs->in_ep, usb9pfs->in_req);
usb9pfs->in_req = NULL;
}
if (usb9pfs->out_req) {
- usb_ep_free_request(usb9pfs->out_ep, usb9pfs->out_req);
+ free_ep_req(usb9pfs->out_ep, usb9pfs->out_req);
usb9pfs->out_req = NULL;
}
-
- disable_ep(cdev, usb9pfs->in_ep);
- disable_ep(cdev, usb9pfs->out_ep);
dev_dbg(&cdev->gadget->dev, "%s disabled\n",
usb9pfs->function.name);
}
@@ -313,10 +333,6 @@ static int alloc_requests(struct usb_composite_dev *cdev,
usb9pfs->in_req->complete = usb9pfs_tx_complete;
usb9pfs->out_req->complete = usb9pfs_rx_complete;
- /* length will be set in complete routine */
- usb9pfs->in_req->context = usb9pfs;
- usb9pfs->out_req->context = usb9pfs;
-
return 0;
fail_in:
@@ -389,7 +405,7 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a
guard(mutex)(&usb9pfs_lock);
list_for_each_entry(dev, &usbg_instance_list, usb9pfs_instance) {
- if (!strncmp(devname, dev->tag, strlen(devname))) {
+ if (!strcmp(devname, dev->tag)) {
if (!dev->inuse) {
dev->inuse = true;
found = 1;
@@ -414,7 +430,9 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a
client->status = Disconnected;
else
client->status = Connected;
+ spin_lock_irq(&usb9pfs->lock);
usb9pfs->client = client;
+ spin_unlock_irq(&usb9pfs->lock);
client->trans_mod->maxsize = usb9pfs->buflen;
@@ -425,18 +443,26 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a
static void usb9pfs_clear_tx(struct f_usb9pfs *usb9pfs)
{
+ struct p9_client *client;
struct p9_req_t *req;
+ unsigned long flags;
- guard(spinlock_irqsave)(&usb9pfs->lock);
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ usb9pfs->client = NULL;
+ req = usb9pfs->in_req ? usb9pfs->in_req->context : NULL;
+ if (usb9pfs->in_req)
+ usb9pfs->in_req->context = NULL;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
- req = usb9pfs->in_req->context;
- if (!req)
+ if (!req || !client)
return;
if (!req->t_err)
req->t_err = -ECONNRESET;
- p9_client_cb(usb9pfs->client, req, REQ_STATUS_ERROR);
+ p9_client_cb(client, req, REQ_STATUS_ERROR);
+ usb9pfs->in_req->context = NULL;
}
static void p9_usbg_close(struct p9_client *client)
@@ -464,6 +490,9 @@ static void p9_usbg_close(struct p9_client *client)
mutex_lock(&usb9pfs_lock);
dev->inuse = false;
mutex_unlock(&usb9pfs_lock);
+
+ disable_usb9pfs(usb9pfs);
+ reinit_completion(&usb9pfs->send);
}
static int p9_usbg_request(struct p9_client *client, struct p9_req_t *p9_req)
@@ -745,8 +774,18 @@ static int usb9pfs_set_alt(struct usb_function *f,
static void usb9pfs_disable(struct usb_function *f)
{
struct f_usb9pfs *usb9pfs = func_to_usb9pfs(f);
+ unsigned long flags;
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ if (usb9pfs->client)
+ usb9pfs->client->status = Disconnected;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
usb9pfs_clear_tx(usb9pfs);
+
+ if (usb9pfs->in_ep->enabled)
+ disable_usb9pfs(usb9pfs);
+
+ reinit_completion(&usb9pfs->send);
}
static struct usb_function *usb9pfs_alloc(struct usb_function_instance *fi)
diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
index 0b8086f58ad5..d1a680f69911 100644
--- a/net/9p/trans_virtio.c
+++ b/net/9p/trans_virtio.c
@@ -629,6 +629,11 @@ static int p9_virtio_probe(struct virtio_device *vdev)
err = -EINVAL;
goto out_free_vq;
}
+ if (tag_len > NAME_MAX) {
+ dev_err(&vdev->dev, "mount tag too long (%u bytes)\n", tag_len);
+ err = -EINVAL;
+ goto out_free_vq;
+ }
tag = kzalloc(tag_len + 1, GFP_KERNEL);
if (!tag) {
err = -ENOMEM;
diff --git a/net/9p/trans_xen.c b/net/9p/trans_xen.c
index 068d57515dd5..67f464acf617 100644
--- a/net/9p/trans_xen.c
+++ b/net/9p/trans_xen.c
@@ -198,9 +198,19 @@ static void p9_xen_response(struct work_struct *work)
masked_prod, &masked_cons,
XEN_9PFS_RING_SIZE(ring));
+ if (h.size < sizeof(h) ||
+ h.size > xen_9pfs_queued(prod, cons,
+ XEN_9PFS_RING_SIZE(ring))) {
+ dev_warn_ratelimited(&priv->dev->dev,
+ "bad response size %u from backend\n", h.size);
+ break;
+ }
+
req = p9_tag_lookup(priv->client, h.tag);
if (!req || req->status != REQ_STATUS_SENT) {
dev_warn(&priv->dev->dev, "Wrong req tag=%x\n", h.tag);
+ if (req)
+ p9_req_put(priv->client, req);
cons += h.size;
virt_mb();
ring->intf->in_cons = cons;
diff --git a/net/ax25/af_ax25.c b/net/ax25/af_ax25.c
index 95224600b5a1..d464f8ea4e55 100644
--- a/net/ax25/af_ax25.c
+++ b/net/ax25/af_ax25.c
@@ -78,6 +78,7 @@ static void ax25_kill_by_device(struct net_device *dev)
ax25_dev *ax25_dev;
ax25_cb *s;
struct sock *sk;
+ bool orphan;
if ((ax25_dev = ax25_dev_ax25dev(dev)) == NULL)
return;
@@ -109,6 +110,16 @@ static void ax25_kill_by_device(struct net_device *dev)
sock_hold(sk);
spin_unlock_bh(&ax25_list_lock);
lock_sock(sk);
+ /*
+ * A socket closed by its owner, or a connection left
+ * unaccepted by a closed listener, is freed by its
+ * heartbeat once it is down. ax25_disconnect() below
+ * stops that heartbeat for good, so free the socket
+ * here, or it is never freed and pins the module.
+ */
+ orphan = sock_flag(sk, SOCK_DESTROY) ||
+ (sk->sk_state == TCP_LISTEN &&
+ sock_flag(sk, SOCK_DEAD));
ax25_disconnect(s, ENETUNREACH);
WRITE_ONCE(s->ax25_dev, NULL);
if (sk->sk_socket) {
@@ -116,7 +127,10 @@ static void ax25_kill_by_device(struct net_device *dev)
&s->dev_tracker);
ax25_dev_put(ax25_dev);
}
- ax25_cb_del(s);
+ if (orphan)
+ ax25_destroy_socket(s);
+ else
+ ax25_cb_del(s);
release_sock(sk);
spin_lock_bh(&ax25_list_lock);
sock_put(sk);
@@ -694,6 +708,12 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname,
case AX25_EXTSEQ:
ax25->modulus = opt ? AX25_EMODULUS : AX25_MODULUS;
+ /*
+ * A window set for modulo 128 may not fit modulo 8:
+ * ax25_kick() would then find it full and send nothing.
+ */
+ if (ax25->modulus == AX25_MODULUS && ax25->window > 7)
+ ax25->window = 7;
break;
case AX25_PIDINCL:
@@ -1233,7 +1253,7 @@ static int __must_check ax25_connect(struct socket *sock,
struct sock *sk = sock->sk;
ax25_cb *ax25 = sk_to_ax25(sk), *ax25t;
struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)uaddr;
- ax25_digi *digi = NULL;
+ ax25_digi *digi = NULL, *old_digi;
int ct = 0, err = 0;
/*
@@ -1282,11 +1302,28 @@ static int __must_check ax25_connect(struct socket *sock,
goto out_release;
}
+ /*
+ * A connection that failed or ended through ax25_disconnect() left
+ * the socket SOCK_DEAD and shut down for sending. The socket is still
+ * its owner's: start afresh, or the state changes of the new attempt
+ * would wake no one and every send() would fail with -EPIPE.
+ */
+ sock_reset_flag(sk, SOCK_DEAD);
+ sk->sk_shutdown = 0;
+
sk->sk_state = TCP_CLOSE;
sock->state = SS_UNCONNECTED;
- kfree(ax25->digipeat);
+ /*
+ * A bound socket is on ax25_list, where the receive path and
+ * /proc/net/ax25 read its digipeater path under ax25_list_lock:
+ * take the old path off under that lock before freeing it.
+ */
+ spin_lock_bh(&ax25_list_lock);
+ old_digi = ax25->digipeat;
ax25->digipeat = NULL;
+ spin_unlock_bh(&ax25_list_lock);
+ kfree(old_digi);
/*
* Handle digi-peaters to be used.
@@ -1337,6 +1374,15 @@ static int __must_check ax25_connect(struct socket *sock,
goto out_release;
}
+ /*
+ * The socket is on ax25_list with the destination of its previous
+ * attempt, if any: forget it, or the lookup below finds the socket
+ * itself when the same station is called again.
+ */
+ spin_lock_bh(&ax25_list_lock);
+ ax25->dest_addr = null_ax25_address;
+ spin_unlock_bh(&ax25_list_lock);
+
if (sk->sk_type == SOCK_SEQPACKET &&
(ax25t=ax25_find_cb(&ax25->source_addr, &fsa->fsa_ax25.sax25_call, digi,
ax25->ax25_dev->dev))) {
@@ -1346,8 +1392,10 @@ static int __must_check ax25_connect(struct socket *sock,
goto out_release;
}
+ spin_lock_bh(&ax25_list_lock);
ax25->dest_addr = fsa->fsa_ax25.sax25_call;
ax25->digipeat = digi;
+ spin_unlock_bh(&ax25_list_lock);
/* First the easy one */
if (sk->sk_type != SOCK_SEQPACKET) {
@@ -1484,16 +1532,29 @@ static int ax25_accept(struct socket *sock, struct socket *newsock,
goto out;
newsk = skb->sk;
+
+ /*
+ * ax25_kill_by_device() detaches a connection from a device going
+ * down under the lock of its socket, clears its ax25_dev and drops
+ * the device references if the socket has been grafted by then.
+ * Graft the socket and take those references under the same lock,
+ * and take none if the device has already gone: the connection is
+ * closed then, and the socket is returned as such.
+ */
+ lock_sock_nested(newsk, SINGLE_DEPTH_NESTING);
sock_graft(newsk, newsock);
+ ax25 = sk_to_ax25(newsk);
+ ax25_dev = ax25->ax25_dev;
+ if (ax25_dev) {
+ netdev_hold(ax25_dev->dev, &ax25->dev_tracker, GFP_ATOMIC);
+ ax25_dev_hold(ax25_dev);
+ }
+ release_sock(newsk);
/* Now attach up the new socket */
kfree_skb(skb);
sk_acceptq_removed(sk);
newsock->state = SS_CONNECTED;
- ax25 = sk_to_ax25(newsk);
- ax25_dev = ax25->ax25_dev;
- netdev_hold(ax25_dev->dev, &ax25->dev_tracker, GFP_ATOMIC);
- ax25_dev_hold(ax25_dev);
out:
release_sock(sk);
@@ -1555,7 +1616,7 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
struct sock *sk = sock->sk;
struct sockaddr_ax25 sax;
struct sk_buff *skb;
- ax25_digi dtmp, *dp;
+ ax25_digi dtmp, *dp = NULL;
ax25_cb *ax25;
size_t size;
int lv, err, addr_len = msg->msg_namelen;
@@ -1630,6 +1691,7 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
}
dtmp.lastrepeat = 0;
+ dp = &dtmp;
}
sax = *usax;
@@ -1638,10 +1700,6 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
err = -EISCONN;
goto out;
}
- if (usax->sax25_ndigis == 0)
- dp = NULL;
- else
- dp = &dtmp;
} else {
/*
* FIXME: 1003.1g - if the socket is like this because
@@ -1779,25 +1837,36 @@ static int ax25_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
if (msg->msg_name) {
ax25_digi digi;
ax25_address src;
- const unsigned char *mac = skb_mac_header(skb);
+ const unsigned char *mac;
DECLARE_SOCKADDR(struct sockaddr_ax25 *, sax, msg->msg_name);
memset(sax, 0, sizeof(struct full_sockaddr_ax25));
- ax25_addr_parse(mac + 1, skb->data - mac - 1, &src, NULL,
- &digi, NULL, NULL);
sax->sax25_family = AF_AX25;
- /* We set this correctly, even though we may not let the
- application know the digi calls further down (because it
- did NOT ask to know them). This could get political... **/
- sax->sax25_ndigis = digi.ndigi;
- sax->sax25_call = src;
-
- if (sax->sax25_ndigis != 0) {
- int ct;
- struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)sax;
-
- for (ct = 0; ct < digi.ndigi; ct++)
- fsa->fsa_digipeater[ct] = digi.calls[ct];
+
+ /*
+ * A frame put together from fragments by ax25_rx_fragment()
+ * has no address header left, and no mac header: give no
+ * address then. ax25_addr_parse() fills neither src nor digi
+ * when it fails.
+ */
+ mac = skb_mac_header_was_set(skb) ? skb_mac_header(skb) : NULL;
+ if (mac && ax25_addr_parse(mac + 1, skb->data - mac - 1, &src,
+ NULL, &digi, NULL, NULL)) {
+ /*
+ * We set this correctly, even though we may not let the
+ * application know the digi calls further down (because
+ * it did NOT ask to know them). This could get political...
+ */
+ sax->sax25_ndigis = digi.ndigi;
+ sax->sax25_call = src;
+
+ if (sax->sax25_ndigis != 0) {
+ int ct;
+ struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)sax;
+
+ for (ct = 0; ct < digi.ndigi; ct++)
+ fsa->fsa_digipeater[ct] = digi.calls[ct];
+ }
}
msg->msg_namelen = sizeof(struct full_sockaddr_ax25);
}
diff --git a/net/ax25/ax25_ds_timer.c b/net/ax25/ax25_ds_timer.c
index c4f8adbf8144..5eb54dd1120a 100644
--- a/net/ax25/ax25_ds_timer.c
+++ b/net/ax25/ax25_ds_timer.c
@@ -211,7 +211,7 @@ void ax25_ds_t1_timeout(ax25_cb *ax25)
case AX25_STATE_2:
if (ax25->n2count == ax25->n2) {
ax25_send_control(ax25, AX25_DISC, AX25_POLLON, AX25_COMMAND);
- if (!sock_flag(ax25->sk, SOCK_DESTROY))
+ if (!ax25->sk || !sock_flag(ax25->sk, SOCK_DESTROY))
ax25_disconnect(ax25, ETIMEDOUT);
return;
} else {
diff --git a/net/ax25/ax25_in.c b/net/ax25/ax25_in.c
index 1bcb5e321467..fd8669e5b731 100644
--- a/net/ax25/ax25_in.c
+++ b/net/ax25/ax25_in.c
@@ -201,7 +201,7 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
int type = 0, mine = 0, dama;
struct sock *make, *sk;
ax25_digi dp, reverse_dp;
- ax25_cb *ax25;
+ ax25_cb *ax25, *existing;
ax25_dev *ax25_dev;
/*
@@ -401,11 +401,12 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
}
}
- skb_set_owner_r(skb, make);
- skb_queue_head(&sk->sk_receive_queue, skb);
-
- make->sk_state = TCP_ESTABLISHED;
-
+ /*
+ * Keep a place in the backlog, but put the skb on the
+ * listener's queue only once the connection is set up
+ * below: accept() takes it from there, and close() on the
+ * new socket frees the connection.
+ */
sk_acceptq_added(sk);
bh_unlock_sock(sk);
} else {
@@ -450,6 +451,24 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
ax25->window = ax25_dev->values[AX25_VALUES_WINDOW];
}
+ if (!sk) {
+ /*
+ * ax25_send_frame() may have created a connection for this
+ * link since the lookup above: add ours only if there is
+ * none, otherwise give the SABM to the existing one. Frames
+ * queued on ours before it reaches state 3 are sent by the
+ * ax25_kick() below.
+ */
+ existing = ax25_cb_add_unique(ax25);
+ if (existing) {
+ ax25_cb_put(ax25);
+ if (ax25_process_rx_frame(existing, skb, type, dama) == 0)
+ kfree_skb(skb);
+ ax25_cb_put(existing);
+ return 0;
+ }
+ }
+
ax25_send_control(ax25, AX25_UA, AX25_POLLON, AX25_RESPONSE);
#ifdef CONFIG_AX25_DAMA_SLAVE
@@ -459,15 +478,36 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev,
ax25->state = AX25_STATE_3;
- ax25_cb_add(ax25);
+ if (sk)
+ ax25_cb_add(ax25);
ax25_start_heartbeat(ax25);
ax25_start_t3timer(ax25);
ax25_start_idletimer(ax25);
+ if (!sk)
+ ax25_kick(ax25);
+
if (sk) {
- if (!sock_flag(sk, SOCK_DEAD))
+ bh_lock_sock(sk);
+ skb_set_owner_r(skb, make);
+ if (sk->sk_state == TCP_LISTEN && !sock_flag(sk, SOCK_DEAD)) {
+ skb_queue_head(&sk->sk_receive_queue, skb);
sk->sk_data_ready(sk);
+ } else {
+ /*
+ * The listener was closed meanwhile: leave the
+ * connection to its heartbeat, as
+ * ax25_destroy_socket() does with the connections
+ * that were still waiting on its queue.
+ */
+ sk_acceptq_removed(sk);
+ kfree_skb(skb);
+ sock_orphan(make);
+ make->sk_state = TCP_LISTEN;
+ ax25->state = AX25_STATE_0;
+ }
+ bh_unlock_sock(sk);
sock_put(sk);
} else {
free:
diff --git a/net/ax25/ax25_ip.c b/net/ax25/ax25_ip.c
index 215d4ccf12b9..b24cb88d75de 100644
--- a/net/ax25/ax25_ip.c
+++ b/net/ax25/ax25_ip.c
@@ -176,9 +176,14 @@ netdev_tx_t ax25_ip_xmit(struct sk_buff *skb)
ax25_dev->values[AX25_VALUES_PACLEN],
&src_c,
&dst_c, digipeat, dev);
- if (ax25) {
+ /*
+ * ax25_send_frame() leaves the skb to its caller
+ * when it gives up, as ROSE and NET/ROM expect.
+ */
+ if (ax25)
ax25_cb_put(ax25);
- }
+ else
+ kfree_skb(ourskb);
goto put;
}
}
diff --git a/net/ax25/ax25_out.c b/net/ax25/ax25_out.c
index 7c4ebef5ae6c..50b014f7b0df 100644
--- a/net/ax25/ax25_out.c
+++ b/net/ax25/ax25_out.c
@@ -137,7 +137,9 @@ EXPORT_SYMBOL(ax25_send_frame);
*/
void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb)
{
+ struct sk_buff_head frags;
struct sk_buff *skbn;
+ unsigned long flags;
unsigned char *p;
int frontlen, len, fragno, ka9qfrag, first = 1;
@@ -161,11 +163,20 @@ void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb)
frontlen = skb_headroom(skb); /* Address space + CTRL */
+ /*
+ * Build the fragments on a list of our own: a message that
+ * cannot be fragmented whole is not sent at all, and its
+ * fragments go on the write queue one after the other.
+ */
+ __skb_queue_head_init(&frags);
+
while (skb->len > 0) {
spin_lock_bh(&ax25_frag_lock);
if ((skbn = alloc_skb(paclen + 2 + frontlen, GFP_ATOMIC)) == NULL) {
spin_unlock_bh(&ax25_frag_lock);
printk(KERN_CRIT "AX.25: ax25_output - out of memory\n");
+ __skb_queue_purge(&frags);
+ kfree_skb(skb);
return;
}
@@ -200,10 +211,14 @@ void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb)
}
skb_pull(skb, len);
- skb_queue_tail(&ax25->write_queue, skbn); /* Throw it on the queue */
+ __skb_queue_tail(&frags, skbn);
}
kfree_skb(skb);
+
+ spin_lock_irqsave(&ax25->write_queue.lock, flags);
+ skb_queue_splice_tail_init(&frags, &ax25->write_queue);
+ spin_unlock_irqrestore(&ax25->write_queue.lock, flags);
} else {
skb_queue_tail(&ax25->write_queue, skb); /* Throw it on the queue */
}
diff --git a/net/ax25/ax25_std_timer.c b/net/ax25/ax25_std_timer.c
index b17da41210cb..05ff1032748f 100644
--- a/net/ax25/ax25_std_timer.c
+++ b/net/ax25/ax25_std_timer.c
@@ -143,7 +143,7 @@ void ax25_std_t1timer_expiry(ax25_cb *ax25)
case AX25_STATE_2:
if (ax25->n2count == ax25->n2) {
ax25_send_control(ax25, AX25_DISC, AX25_POLLON, AX25_COMMAND);
- if (!sock_flag(ax25->sk, SOCK_DESTROY))
+ if (!ax25->sk || !sock_flag(ax25->sk, SOCK_DESTROY))
ax25_disconnect(ax25, ETIMEDOUT);
return;
} else {
diff --git a/net/ax25/sysctl_net_ax25.c b/net/ax25/sysctl_net_ax25.c
index 68753aa30334..41a83e3a7e55 100644
--- a/net/ax25/sysctl_net_ax25.c
+++ b/net/ax25/sysctl_net_ax25.c
@@ -20,7 +20,7 @@ static int min_t2[] = {1}, max_t2[] = {20000};
static int min_t3[1], max_t3[] = {3600000};
static int min_idle[1], max_idle[] = {65535000};
static int min_n2[] = {1}, max_n2[] = {31};
-static int min_paclen[] = {1}, max_paclen[] = {512};
+static int min_paclen[] = {16}, max_paclen[] = {512};
static int min_proto[1], max_proto[] = { AX25_PROTO_MAX };
#ifdef CONFIG_AX25_DAMA_SLAVE
static int min_ds_timeout[1], max_ds_timeout[] = {65535000};
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index 5dd3e1f281ba..dd1f88af5b96 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -449,6 +449,7 @@ batadv_iv_ogm_can_aggregate(const struct batadv_ogm_packet *new_bat_ogm_packet,
struct batadv_ogm_packet *batadv_ogm_packet;
int aggregated_bytes = forw_packet->packet_len + packet_len;
struct batadv_hard_iface *primary_if = NULL;
+ u8 packet_num = forw_packet->num_packets + 1;
bool res = false;
unsigned long aggregation_end_time;
@@ -474,6 +475,9 @@ batadv_iv_ogm_can_aggregate(const struct batadv_ogm_packet *new_bat_ogm_packet,
if (aggregated_bytes > BATADV_MAX_AGGREGATION_BYTES)
return false;
+ if (packet_num >= BITS_PER_TYPE(forw_packet->direct_link_flags))
+ return false;
+
/* packet is not leaving on the same interface. */
if (forw_packet->if_outgoing != if_outgoing)
return false;
@@ -1789,6 +1793,11 @@ static void batadv_iv_send_outstanding_bat_ogm_packet(struct work_struct *work)
goto out;
}
+ /* Mark aggregate as full before forcing emit. */
+ spin_lock_bh(&bat_priv->forw_bat_list_lock);
+ forw_packet->num_packets = BITS_PER_TYPE(forw_packet->direct_link_flags);
+ spin_unlock_bh(&bat_priv->forw_bat_list_lock);
+
batadv_iv_ogm_emit(forw_packet);
/* we have to have at least one packet in the queue to determine the
diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index 45e58b348bd7..39d74b908d64 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -583,6 +583,7 @@ static void batadv_choose_next_candidate(struct batadv_priv *bat_priv,
* NOT_FOUND
*/
cands[select].type = BATADV_DAT_CANDIDATE_NOT_FOUND;
+ cands[select].orig_node = NULL;
/* iterate over the originator list and find the node with the closest
* dat_address which has not been selected yet
diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c
index 8091619981c0..ccfe1c7f33ed 100644
--- a/net/batman-adv/main.c
+++ b/net/batman-adv/main.c
@@ -263,6 +263,11 @@ void batadv_mesh_free(struct net_device *soft_iface)
atomic_set(&bat_priv->mesh_state, BATADV_MESH_DEACTIVATING);
+ /* the originator purge worker accesses (among others) the TT tables
+ * and must therefore be stopped before any of them are freed
+ */
+ disable_delayed_work_sync(&bat_priv->orig_work);
+
batadv_purge_outstanding_packets(bat_priv, NULL);
batadv_tp_stop_all(bat_priv);
@@ -449,8 +454,10 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
* shut down otherwise the packet may trigger de-reference errors
* further down in the receive path.
*/
- if (!kref_get_unless_zero(&hard_iface->refcount))
- goto err_out;
+ if (!kref_get_unless_zero(&hard_iface->refcount)) {
+ kfree_skb(skb);
+ return NET_RX_DROP;
+ }
skb = skb_share_check(skb, GFP_ATOMIC);
@@ -509,7 +516,6 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
kfree_skb(skb);
err_put:
batadv_hardif_put(hard_iface);
-err_out:
return NET_RX_DROP;
}
diff --git a/net/batman-adv/main.h b/net/batman-adv/main.h
index 3d4c36ae2e1a..9f3c1134e006 100644
--- a/net/batman-adv/main.h
+++ b/net/batman-adv/main.h
@@ -102,9 +102,6 @@
*/
#define BATADV_TQ_SIMILARITY_THRESHOLD 50
-/* should not be bigger than 512 bytes or change the size of
- * forw_packet->direct_link_flags
- */
#define BATADV_MAX_AGGREGATION_BYTES 512
#define BATADV_MAX_AGGREGATION_MS 100
diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c
index 893cb9ce9971..9752e3dd79ef 100644
--- a/net/batman-adv/multicast_forw.c
+++ b/net/batman-adv/multicast_forw.c
@@ -950,8 +950,11 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv,
dest = (u8 *)mcast_tracker + sizeof(*mcast_tracker);
- /* check if full tracker tvlv is within skb length */
+ /* check if full tracker tvlv, including padding, is within skb length */
tvlv_len = sizeof(*mcast_tracker) + ETH_ALEN * num_dests;
+ if (!(num_dests % 2))
+ tvlv_len += 2;
+
if (tvlv_len > skb_network_header_len(skb))
return -EINVAL;
diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c
index 50b83e0438a0..e28c4923f52f 100644
--- a/net/batman-adv/tp_meter.c
+++ b/net/batman-adv/tp_meter.c
@@ -226,9 +226,9 @@ static void batadv_tp_batctl_notify(enum batadv_tp_meter_reason reason,
unsigned long start_time, u64 total_sent,
u32 cookie)
{
+ u64 total_bytes;
u32 test_time;
u8 result;
- u32 total_bytes;
if (!batadv_tp_is_error(reason)) {
result = BATADV_TP_REASON_COMPLETE;
diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c
index 7df77cca52a0..c3331304cc39 100644
--- a/net/batman-adv/translation-table.c
+++ b/net/batman-adv/translation-table.c
@@ -455,6 +455,9 @@ batadv_tt_orig_list_entry_put(struct batadv_tt_orig_list_entry *orig_entry)
* @bat_priv: the bat priv with all the soft interface information
* @tt_local_entry: the TT entry involved in the event
* @event_flags: flags to store in the event structure
+ *
+ * Only a single change per client is queued. It describes the latest state of
+ * the client and a new event therefore replaces an already queued one.
*/
static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_local_entry *tt_local_entry,
@@ -463,14 +466,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_change_node *tt_change_node, *entry, *safe;
struct batadv_tt_common_entry *common = &tt_local_entry->common;
u8 flags = common->flags | event_flags;
- bool event_removed = false;
- bool del_op_requested, del_op_entry;
+ bool del_op_requested;
tt_change_node = kmem_cache_alloc(batadv_tt_change_cache, GFP_ATOMIC);
if (!tt_change_node)
return;
- tt_change_node->change.flags = flags;
memset(tt_change_node->change.reserved, 0,
sizeof(tt_change_node->change.reserved));
ether_addr_copy(tt_change_node->change.addr, common->addr);
@@ -478,6 +479,15 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
del_op_requested = flags & BATADV_TT_CLIENT_DEL;
+ /* the ROAM flag is only meaningful for the receiver in combination
+ * with a DEL. An ADD with ROAM would be interpreted as a roaming
+ * advertisement
+ */
+ if (!del_op_requested)
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
+ tt_change_node->change.flags = flags;
+
/* check for ADD+DEL or DEL+ADD events */
spin_lock_bh(&bat_priv->tt.changes_list_lock);
list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list,
@@ -488,44 +498,23 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
if (entry->change.vid != tt_change_node->change.vid)
continue;
- /* DEL+ADD in the same orig interval have no effect and can be
- * removed to avoid silly behaviour on the receiver side. The
- * other way around (ADD+DEL) can happen in case of roaming of
- * a client still in the NEW state. Roaming of NEW clients is
- * now possible due to automatically recognition of "temporary"
- * clients
- */
- del_op_entry = entry->change.flags & BATADV_TT_CLIENT_DEL;
- if (!del_op_requested && del_op_entry)
- goto del;
- if (del_op_requested && !del_op_entry)
- goto del;
-
- /* this is a second add in the same originator interval. It
- * means that flags have been changed: update them!
+ /* the other nodes may know the client with different
+ * flags. A DEL+ADD must therefore still announce the
+ * current flags and an ADD+DEL must still remove the
+ * client. ADD+ADD and DEL+DEL only update the flags
*/
- if (!del_op_requested && !del_op_entry)
- entry->change.flags = flags;
-
- continue;
-del:
- list_del(&entry->list);
- kmem_cache_free(batadv_tt_change_cache, entry);
+ entry->change.flags = flags;
kmem_cache_free(batadv_tt_change_cache, tt_change_node);
- event_removed = true;
+
goto unlock;
}
/* track the change in the OGMinterval list */
list_add_tail(&tt_change_node->list, &bat_priv->tt.changes_list);
+ atomic_inc(&bat_priv->tt.local_changes);
unlock:
spin_unlock_bh(&bat_priv->tt.changes_list_lock);
-
- if (event_removed)
- atomic_dec(&bat_priv->tt.local_changes);
- else
- atomic_inc(&bat_priv->tt.local_changes);
}
/**
@@ -650,6 +639,7 @@ bool batadv_tt_local_add(struct net_device *soft_iface, const u8 *addr,
int hash_added, table_size, packet_size_max;
bool ret = false;
bool roamed_back = false;
+ bool added = false;
u8 remote_flags;
u32 match_mark;
@@ -676,7 +666,7 @@ bool batadv_tt_local_add(struct net_device *soft_iface, const u8 *addr,
* flag can be reset like it was never enqueued
*/
tt_local->common.flags &= ~BATADV_TT_CLIENT_PENDING;
- goto add_event;
+ added = true;
}
if (tt_local->common.flags & BATADV_TT_CLIENT_ROAM) {
@@ -757,14 +747,17 @@ bool batadv_tt_local_add(struct net_device *soft_iface, const u8 *addr,
goto out;
}
-add_event:
- batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+ added = true;
check_roaming:
+ if (added)
+ batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+
/* Check whether it is a roaming, but don't do anything if the roaming
* process has already been handled
*/
- if (tt_global && !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM)) {
+ if (tt_global &&
+ (roamed_back || !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM))) {
/* These node are probably going to update their tt table */
head = &tt_global->orig_list;
rcu_read_lock();
@@ -1026,20 +1019,22 @@ batadv_tt_prepare_tvlv_local_data(struct batadv_priv *bat_priv,
* batadv_tt_tvlv_container_update() - update the translation table tvlv
* container after local tt changes have been committed
* @bat_priv: the bat priv with all the soft interface information
+ * @changes_list: list of committed changes which will be consumed
+ * @tt_diff_entries_num: number of entries in @changes_list
*/
-static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
+static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv,
+ struct list_head *changes_list,
+ int tt_diff_entries_num)
{
struct batadv_tt_change_node *entry, *safe;
struct batadv_tvlv_tt_data *tt_data;
struct batadv_tvlv_tt_change *tt_change;
int tt_diff_len, tt_change_len = 0;
- int tt_diff_entries_num = 0;
int tt_diff_entries_count = 0;
bool drop_changes = false;
size_t tt_extra_len = 0;
u16 tvlv_len;
- tt_diff_entries_num = atomic_read(&bat_priv->tt.local_changes);
tt_diff_len = batadv_tt_len(tt_diff_entries_num);
/* if we have too many changes for one packet don't send any
@@ -1057,19 +1052,39 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
tvlv_len = batadv_tt_prepare_tvlv_local_data(bat_priv, &tt_data,
&tt_change, &tt_diff_len);
- if (!tvlv_len)
+ if (!tvlv_len) {
+ /* the changes are lost and the other nodes have to request the
+ * full table. Don't answer TT requests for the new TTVN with
+ * the stale changeset of the previous TTVN. Without changes, the
+ * TTVN was not incremented and the last changeset is still valid
+ */
+ if (drop_changes || tt_diff_entries_num > 0) {
+ list_for_each_entry_safe(entry, safe, changes_list,
+ list) {
+ list_del(&entry->list);
+ kmem_cache_free(batadv_tt_change_cache, entry);
+ }
+
+ spin_lock_bh(&bat_priv->tt.last_changeset_lock);
+ kfree(bat_priv->tt.last_changeset);
+ bat_priv->tt.last_changeset = NULL;
+ bat_priv->tt.last_changeset_len = 0;
+ spin_unlock_bh(&bat_priv->tt.last_changeset_lock);
+ }
+
+ /* announce the already committed TTVN in the next OGM interval
+ * instead of waiting for BATADV_TT_OGM_APPEND_MAX idle rounds
+ */
+ atomic_set(&bat_priv->tt.ogm_append_cnt, 0);
return;
+ }
tt_data->flags = BATADV_TT_OGM_DIFF;
if (!drop_changes && tt_diff_len == 0)
goto container_register;
- spin_lock_bh(&bat_priv->tt.changes_list_lock);
- atomic_set(&bat_priv->tt.local_changes, 0);
-
- list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list,
- list) {
+ list_for_each_entry_safe(entry, safe, changes_list, list) {
if (tt_diff_entries_count < tt_diff_entries_num) {
memcpy(tt_change + tt_diff_entries_count,
&entry->change,
@@ -1079,7 +1094,6 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
list_del(&entry->list);
kmem_cache_free(batadv_tt_change_cache, entry);
}
- spin_unlock_bh(&bat_priv->tt.changes_list_lock);
tt_extra_len = batadv_tt_len(tt_diff_entries_num -
tt_diff_entries_count);
@@ -2757,8 +2771,12 @@ static bool batadv_tt_local_valid(const void *entry_ptr,
if (tt_common_entry->flags & BATADV_TT_CLIENT_NEW)
return false;
+ /* the entry is (still) announced. A ROAM flag of an entry which is
+ * pending to be removed must not be transmitted. Otherwise, the
+ * receiver would handle it like a roaming advertisement
+ */
if (flags)
- *flags = tt_common_entry->flags;
+ *flags = tt_common_entry->flags & ~BATADV_TT_CLIENT_ROAM;
return true;
}
@@ -3323,20 +3341,29 @@ static void _batadv_tt_update_changes(struct batadv_priv *bat_priv,
{
int i;
int roams;
+ u8 flags;
for (i = 0; i < tt_num_changes; i++) {
- if ((tt_change + i)->flags & BATADV_TT_CLIENT_DEL) {
- roams = (tt_change + i)->flags & BATADV_TT_CLIENT_ROAM;
+ flags = (tt_change + i)->flags;
+
+ if (flags & BATADV_TT_CLIENT_DEL) {
+ roams = flags & BATADV_TT_CLIENT_ROAM;
batadv_tt_global_del(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
"tt removed by changes",
roams);
} else {
+ /* an announced client cannot have roamed away from
+ * orig_node. Only a ROAMING_ADV is allowed to add a
+ * global entry with the ROAM flag
+ */
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
if (!batadv_tt_global_add(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
- (tt_change + i)->flags, ttvn))
+ flags, ttvn))
/* In case of problem while storing a
* global_entry, we stop the updating
* procedure without committing the
@@ -3755,11 +3782,25 @@ static void batadv_tt_local_purge_pending_clients(struct batadv_priv *bat_priv)
*/
static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
{
+ LIST_HEAD(changes_list);
+ int changes_num;
+
lockdep_assert_held(&bat_priv->tt.commit_lock);
- if (atomic_read(&bat_priv->tt.local_changes) < 1) {
+ /* atomically get the list of changes. The TTVN decision and the
+ * announced changeset must cover the same events. Events queued
+ * afterwards stay in the tt.changes_list for the next TT round
+ */
+ spin_lock_bh(&bat_priv->tt.changes_list_lock);
+ list_splice_init(&bat_priv->tt.changes_list, &changes_list);
+ changes_num = atomic_read(&bat_priv->tt.local_changes);
+ atomic_set(&bat_priv->tt.local_changes, 0);
+ spin_unlock_bh(&bat_priv->tt.changes_list_lock);
+
+ if (changes_num == 0) {
if (!batadv_atomic_dec_not_zero(&bat_priv->tt.ogm_append_cnt))
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list,
+ changes_num);
return;
}
@@ -3776,7 +3817,7 @@ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
/* reset the sending counter */
atomic_set(&bat_priv->tt.ogm_append_cnt, BATADV_TT_OGM_APPEND_MAX);
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list, changes_num);
}
/**
diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h
index 1b8ab18dcf2c..62fe0e29f669 100644
--- a/net/batman-adv/types.h
+++ b/net/batman-adv/types.h
@@ -2208,7 +2208,13 @@ struct batadv_forw_packet {
/** @direct_link_flags: direct link flags for aggregated OGM packets */
u32 direct_link_flags;
- /** @num_packets: counter for aggregated OGMv1 packets */
+ /**
+ * @num_packets: counter for aggregated OGMv1 packets. Set to
+ * BITS_PER_TYPE(forw_packet->direct_link_flags) (independent of the
+ * real number of aggregated packets) by the send worker to prevent
+ * further aggregation while the packet is emitted. It must therefore
+ * not be used to iterate over the aggregated packets
+ */
u8 num_packets;
/** @delayed_work: work queue callback item for packet sending */
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
index 654bb2454e20..e0bcb000107c 100644
--- a/net/bluetooth/hci_sync.c
+++ b/net/bluetooth/hci_sync.c
@@ -2732,7 +2732,7 @@ static int hci_pause_advertising_sync(struct hci_dev *hdev)
static int hci_resume_advertising_sync(struct hci_dev *hdev)
{
struct adv_info *adv, *tmp;
- int err;
+ int err = 0;
/* If advertising has not been paused there is nothing to do. */
if (!hdev->advertising_paused)
@@ -2765,13 +2765,19 @@ static int hci_resume_advertising_sync(struct hci_dev *hdev)
*/
if (hci_dev_test_and_clear_flag(hdev, HCI_LE_ADV_0))
err = hci_enable_ext_advertising_sync(hdev, 0x00);
- } else {
+ } else if (hdev->cur_adv_instance) {
/* Schedule for most recent instance to be restarted and begin
* the software rotation loop
*/
err = hci_schedule_adv_instance_sync(hdev,
hdev->cur_adv_instance,
true);
+ } else if (hci_dev_test_flag(hdev, HCI_ADVERTISING) &&
+ !hci_dev_test_flag(hdev, HCI_LE_ADV)) {
+ /* hci_schedule_adv_instance_sync() rejects instance 0x00 while
+ * HCI_ADVERTISING is set, so enable it directly.
+ */
+ err = hci_start_adv_sync(hdev, 0x00);
}
hdev->advertising_paused = false;
@@ -6148,6 +6154,7 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
u8 filter_policy = 0x00;
/* Default is to enable duplicates filter */
u8 filter_dup = LE_SCAN_FILTER_DUP_ENABLE;
+ bool paused;
int err;
bt_dev_dbg(hdev, "");
@@ -6171,6 +6178,12 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err)
goto failed;
+ /* LE Set Random Address is disallowed while advertising is enabled. */
+ paused = !hdev->advertising_paused;
+ err = hci_pause_advertising_sync(hdev);
+ if (err)
+ goto failed;
+
/* All active scans will be done with either a resolvable private
* address (when privacy feature has been enabled) or non-resolvable
* private address.
@@ -6180,6 +6193,9 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err < 0)
own_addr_type = ADDR_LE_DEV_PUBLIC;
+ if (paused)
+ hci_resume_advertising_sync(hdev);
+
if (hci_is_adv_monitoring(hdev) ||
(test_bit(HCI_QUIRK_STRICT_DUPLICATE_FILTER, &hdev->quirks) &&
hdev->discovery.result_filtering)) {
@@ -6203,9 +6219,8 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
return err;
failed:
- /* Resume advertising if it was paused */
- if (use_ll_privacy(hdev))
- hci_resume_advertising_sync(hdev);
+ /* No-op when advertising was not paused. */
+ hci_resume_advertising_sync(hdev);
/* Resume passive scanning */
hci_update_passive_scan_sync(hdev);
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index d8c39a430686..514abca6ed88 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -161,6 +161,28 @@ static struct l2cap_chan *__l2cap_get_chan_by_ident(struct l2cap_conn *conn,
return NULL;
}
+/* Our connection request has been sent and is still outstanding */
+static bool l2cap_chan_connect_pending(struct l2cap_chan *chan)
+{
+ return chan->state == BT_CONNECT &&
+ test_bit(CONF_CONNECT_PEND, &chan->conf_state);
+}
+
+/* Other channels may hold the same ident for a response they still have
+ * to send, so only look at channels waiting for a connection response.
+ */
+static struct l2cap_chan *__l2cap_get_connect_chan_by_ident(struct l2cap_conn *conn,
+ u8 ident)
+{
+ struct l2cap_chan *c;
+
+ list_for_each_entry(c, &conn->chan_l, list) {
+ if (c->ident == ident && l2cap_chan_connect_pending(c))
+ return c;
+ }
+ return NULL;
+}
+
static struct l2cap_chan *__l2cap_global_chan_by_addr(__le16 psm, bdaddr_t *src,
u8 src_type)
{
@@ -4265,7 +4287,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
if (!chan)
return -EBADSLT;
} else {
- chan = __l2cap_get_chan_by_ident(conn, cmd->ident);
+ chan = __l2cap_get_connect_chan_by_ident(conn, cmd->ident);
if (!chan)
return -EBADSLT;
}
@@ -4278,6 +4300,13 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
l2cap_chan_lock(chan);
+ /* Only accept a response matching a connection request we sent and
+ * that is still outstanding. A channel waiting in BT_CONNECT for
+ * security or the feature mask has not sent one yet.
+ */
+ if (!l2cap_chan_connect_pending(chan) || chan->ident != cmd->ident)
+ goto done;
+
switch (result) {
case L2CAP_CR_SUCCESS:
if (__l2cap_get_chan_by_dcid(conn, dcid)) {
@@ -4307,6 +4336,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
break;
}
+done:
l2cap_chan_unlock(chan);
l2cap_chan_put(chan);
@@ -4439,7 +4469,13 @@ static inline int l2cap_config_req(struct l2cap_conn *conn,
goto unlock;
}
- if (!test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
+ /* A channel in BT_CONNECT2 has not passed security or userspace
+ * accept yet. Answer the peer's request, but hold our own until
+ * the channel reaches BT_CONFIG: the CONNECT2 -> CONFIG sites send
+ * it. Connected channels may also need to reconfigure.
+ */
+ if ((chan->state == BT_CONFIG || chan->state == BT_CONNECTED) &&
+ !test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
u8 buf[64];
l2cap_send_cmd(conn, l2cap_get_ident(conn), L2CAP_CONF_REQ,
l2cap_build_conf_req(chan, buf, sizeof(buf)), buf);
@@ -4487,6 +4523,10 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (!chan)
return 0;
+ /* Connected channels may also receive reconfiguration responses. */
+ if (chan->state != BT_CONFIG && chan->state != BT_CONNECTED)
+ goto done;
+
switch (result) {
case L2CAP_CONF_SUCCESS:
l2cap_conf_rfc_get(chan, rsp->data, len);
@@ -4554,6 +4594,9 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (test_bit(CONF_OUTPUT_DONE, &chan->conf_state)) {
set_default_fcs(chan);
+ if (chan->state != BT_CONFIG)
+ goto done;
+
if (chan->mode == L2CAP_MODE_ERTM ||
chan->mode == L2CAP_MODE_STREAMING)
err = l2cap_ertm_init(chan);
diff --git a/net/bluetooth/selftest.c b/net/bluetooth/selftest.c
index f49604d44b87..6abf70c14f59 100644
--- a/net/bluetooth/selftest.c
+++ b/net/bluetooth/selftest.c
@@ -191,6 +191,7 @@ static ssize_t test_ecdh_read(struct file *file, char __user *user_buf,
}
static const struct file_operations test_ecdh_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = test_ecdh_read,
.llseek = default_llseek,
diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c
index c09379796226..be1bde5b8ed6 100644
--- a/net/bluetooth/smp.c
+++ b/net/bluetooth/smp.c
@@ -3735,6 +3735,7 @@ static ssize_t test_smp_read(struct file *file, char __user *user_buf,
}
static const struct file_operations test_smp_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = test_smp_read,
.llseek = default_llseek,
diff --git a/net/can/af_can.c b/net/can/af_can.c
index 1172073499d2..694a7741bb54 100644
--- a/net/can/af_can.c
+++ b/net/can/af_can.c
@@ -224,7 +224,7 @@ int can_send(struct sk_buff *skb, int loop)
goto inval_skb;
}
- if (unlikely(skb->dev->type != ARPHRD_CAN)) {
+ if (unlikely(!can_get_ml_priv(skb->dev))) {
err = -EPERM;
goto inval_skb;
}
@@ -450,7 +450,7 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
/* insert new receiver (dev,canid,mask) -> (func,data) */
- if (dev && (dev->type != ARPHRD_CAN || !can_get_ml_priv(dev)))
+ if (dev && !can_get_ml_priv(dev))
return -ENODEV;
if (dev && !net_eq(net, dev_net(dev)))
@@ -460,9 +460,10 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
if (!rcv)
return -ENOMEM;
- spin_lock_bh(&net->can.rcvlists_lock);
-
dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
+
+ spin_lock_bh(&dev_rcv_lists->lock);
+
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
rcv->can_id = can_id;
@@ -476,6 +477,9 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
hlist_add_head_rcu(&rcv->list, rcv_list);
dev_rcv_lists->entries++;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
rcv_lists_stats->rcv_entries++;
rcv_lists_stats->rcv_entries_max = max(rcv_lists_stats->rcv_entries_max,
rcv_lists_stats->rcv_entries);
@@ -507,6 +511,14 @@ static void can_rx_delete_receiver(struct rcu_head *rp)
*
* Description:
* Removes subscription entry depending on given (subscription) values.
+ * The receive lists of a CAN device are protected by a per-device lock,
+ * so the subscription is found even when the device has been moved to
+ * another network namespace after can_rx_register(). The caller has to
+ * ensure that @dev stays valid, e.g. by holding a reference to @dev or
+ * RTNL. @net should be the namespace that has been used for
+ * can_rx_register(). It selects the 'any' CAN device lists for
+ * @dev == NULL and is otherwise only used for the receive list
+ * statistics, which become imprecise when another namespace is passed.
*/
void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
canid_t mask, void (*func)(struct sk_buff *, void *),
@@ -517,15 +529,13 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
struct can_dev_rcv_lists *dev_rcv_lists;
- if (dev && dev->type != ARPHRD_CAN)
+ if (dev && !can_get_ml_priv(dev))
return;
- if (dev && !net_eq(net, dev_net(dev)))
- return;
+ dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
- spin_lock_bh(&net->can.rcvlists_lock);
+ spin_lock_bh(&dev_rcv_lists->lock);
- dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
/* Search the receiver list for the item to delete. This should
@@ -545,26 +555,26 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
* a warning here.
*/
if (!rcv) {
+ spin_unlock_bh(&dev_rcv_lists->lock);
pr_warn("can: receive list entry not found for dev %s, id %03X, mask %03X\n",
DNAME(dev), can_id, mask);
- goto out;
+ return;
}
hlist_del_rcu(&rcv->list);
dev_rcv_lists->entries--;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
if (rcv_lists_stats->rcv_entries > 0)
rcv_lists_stats->rcv_entries--;
-
- out:
spin_unlock_bh(&net->can.rcvlists_lock);
/* schedule the receiver item for deletion */
- if (rcv) {
- if (rcv->sk)
- sock_hold(rcv->sk);
- call_rcu(&rcv->rcu, can_rx_delete_receiver);
- }
+ if (rcv->sk)
+ sock_hold(rcv->sk);
+ call_rcu(&rcv->rcu, can_rx_delete_receiver);
}
EXPORT_SYMBOL(can_rx_unregister);
@@ -683,7 +693,7 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev)
static int can_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_can_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !can_is_can_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -698,7 +708,7 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev,
static int canfd_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_canfd_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !can_is_canfd_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -713,7 +723,7 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev,
static int canxl_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_canxl_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !can_is_canxl_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN XL skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -795,6 +805,7 @@ static int can_pernet_init(struct net *net)
kzalloc(sizeof(*net->can.rx_alldev_list), GFP_KERNEL);
if (!net->can.rx_alldev_list)
goto out;
+ spin_lock_init(&net->can.rx_alldev_list->lock);
net->can.pkg_stats = kzalloc(sizeof(*net->can.pkg_stats), GFP_KERNEL);
if (!net->can.pkg_stats)
goto out_free_rx_alldev_list;
diff --git a/net/can/bcm.c b/net/can/bcm.c
index 0c7e6203eb44..e8d3cb5ce0f1 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -54,6 +54,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/bcm.h>
@@ -956,7 +957,7 @@ static void bcm_remove_op(struct bcm_op *op)
static void bcm_rx_unreg(struct net_device *dev, struct bcm_op *op)
{
if (op->rx_reg_dev == dev) {
- can_rx_unregister(dev_net(dev), dev, op->can_id,
+ can_rx_unregister(sock_net(op->sk), dev, op->can_id,
REGMASK(op->can_id), bcm_rx_handler, op);
/* mark as removed subscription */
@@ -982,21 +983,10 @@ static int bcm_delete_rx_op(struct list_head *ops, struct bcm_msg_head *mh,
/* disable automatic timer on frame reception */
op->flags |= RX_NO_AUTOTIMER;
- /*
- * Don't care if we're bound or not (due to netdev
- * problems) can_rx_unregister() is always a save
- * thing to do here.
- */
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the dev.
+ /* Only remove subscriptions that had not been
+ * removed in bcm_notify(). op->rx_reg_dev keeps
+ * the device's receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
@@ -1710,7 +1700,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
goto out_release;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto out_release;
@@ -1858,7 +1848,7 @@ static int bcm_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -1926,6 +1916,7 @@ static int bcm_release(struct socket *sock)
list_del(&bo->notifier);
spin_unlock(&bcm_notifier_lock);
+ /* bcm_notify() can not access this socket anymore */
lock_sock(sk);
#if IS_ENABLED(CONFIG_PROC_FS)
@@ -1939,29 +1930,20 @@ static int bcm_release(struct socket *sock)
bcm_remove_op(op);
}
- list_for_each_entry_safe(op, next, &bo->rx_ops, list) {
- /*
- * Don't care if we're bound or not (due to netdev problems)
- * can_rx_unregister() is always a save thing to do here.
- */
+ list_for_each_entry(op, &bo->rx_ops, list) {
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the device.
+ /* Only remove subscriptions that had not been removed
+ * in bcm_notify(). op->rx_reg_dev keeps the device's
+ * receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
- } else
+ } else {
can_rx_unregister(net, NULL, op->can_id,
REGMASK(op->can_id),
bcm_rx_handler, op);
-
+ }
+ cond_resched();
}
synchronize_rcu();
@@ -2014,7 +1996,7 @@ static int bcm_connect(struct socket *sock, struct sockaddr *uaddr, int len,
ret = -ENODEV;
goto fail;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto fail;
diff --git a/net/can/gw.c b/net/can/gw.c
index 676af9a0b659..1fff84d8e4e1 100644
--- a/net/can/gw.c
+++ b/net/can/gw.c
@@ -52,6 +52,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/gw.h>
@@ -603,7 +604,7 @@ static int cgw_notifier(struct notifier_block *nb,
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
struct net *net = dev_net(dev);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg == NETDEV_UNREGISTER) {
@@ -1154,7 +1155,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->src.dev)
goto out;
- if (gwj->src.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->src.dev))
goto out;
gwj->dst.dev = __dev_get_by_index(net, gwj->ccgw.dst_idx);
@@ -1162,7 +1163,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->dst.dev)
goto out;
- if (gwj->dst.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->dst.dev))
goto out;
/* is sending the skb back to the incoming interface intended? */
diff --git a/net/can/isotp.c b/net/can/isotp.c
index 4c65fa4057a0..c3155d7473ab 100644
--- a/net/can/isotp.c
+++ b/net/can/isotp.c
@@ -65,6 +65,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/isotp.h>
@@ -747,6 +748,14 @@ static void isotp_rcv(struct sk_buff *skb, void *data)
if (skb->len != so->ll.mtu)
return;
+ /* check for correct CAN CC/FD frame content */
+ if (so->ll.mtu == CAN_MTU) {
+ if (!can_is_can_skb(skb))
+ return;
+ } else if (!can_is_canfd_skb(skb)) {
+ return;
+ }
+
cf = (struct canfd_frame *)skb->data;
/* if enabled: check reception of my configured extended address */
@@ -1104,7 +1113,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
int off;
int err;
- if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
+ if (!READ_ONCE(so->bound) || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
return -EADDRNOTAVAIL;
/* claim the socket under so->rx_lock: this serializes the claim
@@ -1149,7 +1158,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
/* so->bound is only checked once above - a wakeup may have
* unbound/rebound the socket meanwhile => recheck
*/
- if (!so->bound) {
+ if (!READ_ONCE(so->bound)) {
err = -EADDRNOTAVAIL;
goto err_out_drop;
}
@@ -1389,7 +1398,7 @@ static int isotp_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
if (flags & ~(MSG_DONTWAIT | MSG_TRUNC | MSG_PEEK | MSG_CMSG_COMPAT))
return -EINVAL;
- if (!so->bound)
+ if (!READ_ONCE(so->bound))
return -EADDRNOTAVAIL;
skb = skb_recv_datagram(sk, flags, &ret);
@@ -1463,11 +1472,11 @@ static int isotp_release(struct socket *sock)
list_del(&so->notifier);
spin_unlock(&isotp_notifier_lock);
- rtnl_lock();
lock_sock(sk);
/* remove current filters & unregister
- * tracked reference so->dev is taken at bind() time with rtnl_lock
+ * the tracked reference so->dev keeps the device's receive lists
+ * accessible, even after a namespace change
*/
if (so->bound && so->dev) {
if (isotp_register_rxid(so))
@@ -1482,11 +1491,9 @@ static int isotp_release(struct socket *sock)
}
so->ifindex = 0;
- so->bound = 0;
+ WRITE_ONCE(so->bound, 0);
so->dev = NULL;
- rtnl_unlock();
-
/* Always wait for a grace period before touching the timers below.
* A concurrent NETDEV_UNREGISTER may have already unregistered our
* filters and cleared so->bound in isotp_notify() without waiting
@@ -1515,7 +1522,6 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len)
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
struct net *net = sock_net(sk);
- int ifindex;
struct net_device *dev;
canid_t tx_id = addr->can_addr.tp.tx_id;
canid_t rx_id = addr->can_addr.tp.rx_id;
@@ -1553,7 +1559,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len)
if (!addr->can_ifindex)
return -ENODEV;
- rtnl_lock();
+ /* lock_sock() serializes the binding with isotp_notify() */
lock_sock(sk);
if (so->bound) {
@@ -1579,61 +1585,83 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len)
goto out;
}
- dev = dev_get_by_index(net, addr->can_ifindex);
- if (!dev) {
- err = -ENODEV;
- goto out;
- }
- if (dev->type != ARPHRD_CAN) {
+ /* Publish the device inside the RCU section of the lookup: When it
+ * is unlisted concurrently, either isotp_notify() sees so->dev after
+ * synchronize_net() or the lookup fails. Check the device before
+ * publishing it.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(net, addr->can_ifindex);
+ if (!dev || !can_get_ml_priv(dev))
err = -ENODEV;
- goto out_put_dev;
- }
- if (READ_ONCE(dev->mtu) < so->ll.mtu) {
+ else if (READ_ONCE(dev->mtu) < so->ll.mtu)
err = -EINVAL;
- goto out_put_dev;
+ if (!err) {
+ dev_hold(dev);
+ WRITE_ONCE(so->dev, dev);
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
- ifindex = dev->ifindex;
+ rcu_read_unlock();
+ if (err)
+ goto out;
- if (isotp_register_rxid(so))
- can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
- isotp_rcv, sk, "isotp", sk);
+ /* switch to new settings */
+ so->ifindex = dev->ifindex;
+ so->rxid = rx_id;
+ so->txid = tx_id;
/* no consecutive frame echo skb in flight */
WRITE_ONCE(so->cfecho, 0);
- /* register for echo skb's */
- can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
- isotp_rcv_echo, sk, "isotpe", sk);
+ /* register for echo skb's first (no echo traffic at this point) */
+ err = can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk, "isotpe", sk);
+ if (err)
+ goto out_put_dev;
- /* switch to new settings */
- so->ifindex = ifindex;
- so->rxid = rx_id;
- so->txid = tx_id;
- so->bound = 1;
+ if (isotp_register_rxid(so)) {
+ err = can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
+ isotp_rcv, sk, "isotp", sk);
+ if (err) {
+ /* unregister echo skb subscription */
+ can_rx_unregister(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk);
+ goto out_put_dev;
+ }
+ }
+
+ WRITE_ONCE(so->bound, 1);
/* bind() ok -> hold a reference for so->dev so that isotp_release()
* can safely reach the device later, even if a concurrent
* NETDEV_UNREGISTER has already unlisted it by ifindex.
*/
- so->dev = dev;
dev_hold(so->dev);
+ /* Either isotp_notify() sees so->dev on NETDEV_DOWN or we see the
+ * cleared IFF_UP, pairs with isotp_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* revert so->ifindex and so->dev on failed bind() */
+ if (err) {
+ so->ifindex = 0;
+ WRITE_ONCE(so->dev, NULL);
+ }
+
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -1655,8 +1683,8 @@ static int isotp_getname(struct socket *sock, struct sockaddr *uaddr, int peer)
return ISOTP_MIN_NAMELEN;
}
-static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
- sockptr_t optval, unsigned int optlen)
+static int isotp_setsockopt_locked(struct socket *sock, int optname,
+ const void *val)
{
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
@@ -1667,11 +1695,7 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_ISOTP_OPTS:
- if (optlen != sizeof(struct can_isotp_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->opt, optval, optlen))
- return -EFAULT;
+ memcpy(&so->opt, val, sizeof(so->opt));
/* no separate rx_ext_address is given => use ext_address */
if (!(so->opt.flags & CAN_ISOTP_RX_EXT_ADDR))
@@ -1696,55 +1720,37 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
break;
case CAN_ISOTP_RECV_FC:
- if (optlen != sizeof(struct can_isotp_fc_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->rxfc, optval, optlen))
- return -EFAULT;
+ memcpy(&so->rxfc, val, sizeof(so->rxfc));
break;
case CAN_ISOTP_TX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_tx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_tx_stmin = *(const u32 *)val;
break;
case CAN_ISOTP_RX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_rx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_rx_stmin = *(const u32 *)val;
break;
- case CAN_ISOTP_LL_OPTS:
- if (optlen == sizeof(struct can_isotp_ll_options)) {
- struct can_isotp_ll_options ll;
-
- if (copy_from_sockptr(&ll, optval, optlen))
- return -EFAULT;
+ case CAN_ISOTP_LL_OPTS: {
+ const struct can_isotp_ll_options *ll = val;
- /* check for correct ISO 11898-1 DLC data length */
- if (ll.tx_dl != padlen(ll.tx_dl))
- return -EINVAL;
+ /* check for correct ISO 11898-1 DLC data length */
+ if (ll->tx_dl != padlen(ll->tx_dl))
+ return -EINVAL;
- if (ll.mtu != CAN_MTU && ll.mtu != CANFD_MTU)
- return -EINVAL;
+ if (ll->mtu != CAN_MTU && ll->mtu != CANFD_MTU)
+ return -EINVAL;
- if (ll.mtu == CAN_MTU &&
- (ll.tx_dl > CAN_MAX_DLEN || ll.tx_flags != 0))
- return -EINVAL;
+ if (ll->mtu == CAN_MTU &&
+ (ll->tx_dl > CAN_MAX_DLEN || ll->tx_flags != 0))
+ return -EINVAL;
- memcpy(&so->ll, &ll, sizeof(ll));
+ memcpy(&so->ll, ll, sizeof(*ll));
- /* set ll_dl for tx path to similar place as for rx */
- so->tx.ll_dl = ll.tx_dl;
- } else {
- return -EINVAL;
- }
+ /* set ll_dl for tx path to similar place as for rx */
+ so->tx.ll_dl = ll->tx_dl;
break;
+ }
default:
ret = -ENOPROTOOPT;
@@ -1758,13 +1764,49 @@ static int isotp_setsockopt(struct socket *sock, int level, int optname,
{
struct sock *sk = sock->sk;
+ union {
+ struct can_isotp_options opt;
+ struct can_isotp_fc_options rxfc;
+ struct can_isotp_ll_options ll;
+ u32 stmin;
+ } val;
+ unsigned int size;
int ret;
if (level != SOL_CAN_ISOTP)
return -EINVAL;
+ /* copy the option values from user space before lock_sock() */
+ switch (optname) {
+ case CAN_ISOTP_OPTS:
+ size = sizeof(val.opt);
+ break;
+
+ case CAN_ISOTP_RECV_FC:
+ size = sizeof(val.rxfc);
+ break;
+
+ case CAN_ISOTP_TX_STMIN:
+ case CAN_ISOTP_RX_STMIN:
+ size = sizeof(val.stmin);
+ break;
+
+ case CAN_ISOTP_LL_OPTS:
+ size = sizeof(val.ll);
+ break;
+
+ default:
+ return -ENOPROTOOPT;
+ }
+
+ if (optlen != size)
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, size))
+ return -EFAULT;
+
lock_sock(sk);
- ret = isotp_setsockopt_locked(sock, level, optname, optval, optlen);
+ ret = isotp_setsockopt_locked(sock, optname, &val);
release_sock(sk);
return ret;
}
@@ -1829,12 +1871,21 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (so->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* pairs with isotp_bind(), see raw_notify() */
+
+ /* lockless fast path, see raw_notify() */
+ if (READ_ONCE(so->dev) != dev)
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (so->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (so->bound) {
if (isotp_register_rxid(so))
@@ -1849,16 +1900,19 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
}
so->ifindex = 0;
- so->bound = 0;
- so->dev = NULL;
- release_sock(sk);
+ WRITE_ONCE(so->bound, 0);
+ WRITE_ONCE(so->dev, NULL);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
+ if (!READ_ONCE(so->bound))
+ break;
+
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
@@ -1871,7 +1925,7 @@ static int isotp_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
diff --git a/net/can/proc.c b/net/can/proc.c
index 2f78ea8ac30b..42145a157335 100644
--- a/net/can/proc.c
+++ b/net/can/proc.c
@@ -76,16 +76,39 @@ static const char rx_list_name[][8] = {
* af_can statistics stuff
*/
+static void can_reset_pkg_stats(struct can_pkg_stats *pkg_stats)
+{
+ atomic_long_set(&pkg_stats->rx_frames, 0);
+ atomic_long_set(&pkg_stats->tx_frames, 0);
+ atomic_long_set(&pkg_stats->matches, 0);
+
+ pkg_stats->total_rx_rate = 0;
+ pkg_stats->total_tx_rate = 0;
+ pkg_stats->total_rx_match_ratio = 0;
+
+ pkg_stats->current_rx_rate = 0;
+ pkg_stats->current_tx_rate = 0;
+ pkg_stats->current_rx_match_ratio = 0;
+
+ pkg_stats->max_rx_rate = 0;
+ pkg_stats->max_tx_rate = 0;
+ pkg_stats->max_rx_match_ratio = 0;
+
+ atomic_long_set(&pkg_stats->rx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->tx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->matches_delta, 0);
+}
+
static void can_init_stats(struct net *net)
{
struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
/*
- * This memset function is called from a timer context (when
+ * This stats reset is called from a timer context (when
* can_stattimer is active which is the default) OR in a process
* context (reading the proc_fs when can_stattimer is disabled).
*/
- memset(pkg_stats, 0, sizeof(struct can_pkg_stats));
+ can_reset_pkg_stats(pkg_stats);
pkg_stats->jiffies_init = jiffies;
rcv_lists_stats->stats_reset++;
@@ -121,29 +144,23 @@ void can_stat_update(struct timer_list *t)
long rx_frames = atomic_long_read(&pkg_stats->rx_frames);
long tx_frames = atomic_long_read(&pkg_stats->tx_frames);
long matches = atomic_long_read(&pkg_stats->matches);
- long rx_frames_delta = atomic_long_read(&pkg_stats->rx_frames_delta);
- long tx_frames_delta = atomic_long_read(&pkg_stats->tx_frames_delta);
- long matches_delta = atomic_long_read(&pkg_stats->matches_delta);
+ /* fetch and clear values for 'current rate' calculation */
+ long rx_frames_delta = atomic_long_xchg(&pkg_stats->rx_frames_delta, 0);
+ long tx_frames_delta = atomic_long_xchg(&pkg_stats->tx_frames_delta, 0);
+ long matches_delta = atomic_long_xchg(&pkg_stats->matches_delta, 0);
- /* restart counting in timer context on user request */
- if (user_reset)
- can_init_stats(net);
-
- /* restart counting on jiffies overflow */
- if (j < pkg_stats->jiffies_init)
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (rx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (tx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* matches overflow - very improbable */
- if (matches > (LONG_MAX / 100))
+ /*
+ * Restart counting in timer context on user request, on jiffies
+ * overflow or to prevent overflows in calc_rate() and in the
+ * match ratio calculation. The local snapshots are stale after
+ * the reset, so skip the calculation for this interval.
+ */
+ if (user_reset || j < pkg_stats->jiffies_init ||
+ rx_frames > (LONG_MAX / HZ) || tx_frames > (LONG_MAX / HZ) ||
+ matches > (LONG_MAX / 100)) {
can_init_stats(net);
+ goto out;
+ }
/* calc total values */
if (rx_frames)
@@ -172,11 +189,7 @@ void can_stat_update(struct timer_list *t)
if (pkg_stats->max_rx_match_ratio < pkg_stats->current_rx_match_ratio)
pkg_stats->max_rx_match_ratio = pkg_stats->current_rx_match_ratio;
- /* clear values for 'current rate' calculation */
- atomic_long_set(&pkg_stats->tx_frames_delta, 0);
- atomic_long_set(&pkg_stats->rx_frames_delta, 0);
- atomic_long_set(&pkg_stats->matches_delta, 0);
-
+out:
/* restart timer (one second) */
mod_timer(&net->can.stattimer, round_jiffies(jiffies + HZ));
}
diff --git a/net/can/raw.c b/net/can/raw.c
index a5a0f69cd7ae..fd9f71ee32e9 100644
--- a/net/can/raw.c
+++ b/net/can/raw.c
@@ -49,6 +49,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/dev.h> /* for can_is_canxl_dev_mtu() */
#include <linux/can/skb.h>
@@ -86,6 +87,7 @@ struct raw_sock {
int ifindex;
struct net_device *dev;
netdevice_tracker dev_tracker;
+ struct net_device *bind_dev; /* hint for raw_notify(), no reference */
struct list_head notifier;
int loopback;
int recv_own_msgs;
@@ -217,6 +219,7 @@ static int raw_enable_filters(struct net *net, struct net_device *dev,
int i;
for (i = 0; i < count; i++) {
+ cond_resched();
err = can_rx_register(net, dev, filter[i].can_id,
filter[i].can_mask,
raw_rcv, sk, "raw", sk);
@@ -251,9 +254,11 @@ static void raw_disable_filters(struct net *net, struct net_device *dev,
{
int i;
- for (i = 0; i < count; i++)
+ for (i = 0; i < count; i++) {
can_rx_unregister(net, dev, filter[i].can_id,
filter[i].can_mask, raw_rcv, sk);
+ cond_resched();
+ }
}
static inline void raw_disable_errfilter(struct net *net,
@@ -298,16 +303,31 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
struct net_device *dev)
{
struct sock *sk = &ro->sk;
+ struct net_device *bind_dev;
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (ro->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* IFF_UP is cleared before, pairs with raw_bind() */
+
+ /* Lockless fast path, ro->bind_dev covers a concurrent bind().
+ * Read it before ro->dev, pairs with smp_store_release() in
+ * raw_bind(). A stale ro->bind_dev only costs a lock_sock().
+ */
+ bind_dev = smp_load_acquire(&ro->bind_dev);
+ if (READ_ONCE(ro->dev) != dev &&
+ (msg != NETDEV_UNREGISTER || bind_dev != dev))
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (ro->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (ro->bound) {
raw_disable_allfilters(dev_net(dev), dev, sk);
@@ -319,13 +339,13 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
ro->ifindex = 0;
ro->bound = 0;
- ro->dev = NULL;
+ WRITE_ONCE(ro->dev, NULL);
ro->count = 0;
- release_sock(sk);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
@@ -341,7 +361,7 @@ static int raw_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -374,6 +394,7 @@ static int raw_init(struct sock *sk)
ro->bound = 0;
ro->ifindex = 0;
ro->dev = NULL;
+ ro->bind_dev = NULL;
/* set default filter to single entry dfilter */
ro->dfilter.can_id = 0;
@@ -422,13 +443,15 @@ static int raw_release(struct socket *sock)
list_del(&ro->notifier);
spin_unlock(&raw_notifier_lock);
- rtnl_lock();
lock_sock(sk);
- /* remove current filters & unregister */
+ /* remove current filters & unregister
+ * the tracked reference ro->dev keeps the device's receive lists
+ * accessible, even after a namespace change
+ */
if (ro->bound) {
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev), ro->dev, sk);
+ raw_disable_allfilters(sock_net(sk), ro->dev, sk);
netdev_put(ro->dev, &ro->dev_tracker);
} else {
raw_disable_allfilters(sock_net(sk), NULL, sk);
@@ -447,7 +470,6 @@ static int raw_release(struct socket *sock)
sock->sk = NULL;
release_sock(sk);
- rtnl_unlock();
sock_put(sk);
@@ -469,26 +491,34 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len)
if (addr->can_family != AF_CAN)
return -EINVAL;
- rtnl_lock();
+ /* lock_sock() serializes the binding with raw_notify() */
lock_sock(sk);
if (ro->bound && addr->can_ifindex == ro->ifindex)
goto out;
if (addr->can_ifindex) {
- dev = dev_get_by_index(sock_net(sk), addr->can_ifindex);
+ /* Publish the device inside the RCU section of the lookup:
+ * When it is unlisted concurrently, either raw_notify() sees
+ * ro->bind_dev after synchronize_net() or the lookup fails.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(sock_net(sk), addr->can_ifindex);
+ if (dev) {
+ dev_hold(dev);
+ /* pairs with smp_load_acquire() in raw_notify() */
+ smp_store_release(&ro->bind_dev, dev);
+ }
+ rcu_read_unlock();
if (!dev) {
err = -ENODEV;
goto out;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
err = -ENODEV;
goto out_put_dev;
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
ifindex = dev->ifindex;
/* filters set by default/setsockopt */
@@ -507,7 +537,7 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len)
if (ro->bound) {
/* unregister old filters */
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev),
+ raw_disable_allfilters(sock_net(sk),
ro->dev, sk);
/* drop reference to old ro->dev */
netdev_put(ro->dev, &ro->dev_tracker);
@@ -518,24 +548,32 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len)
ro->ifindex = ifindex;
ro->bound = 1;
/* bind() ok -> hold a reference for new ro->dev */
- ro->dev = dev;
+ WRITE_ONCE(ro->dev, dev);
if (ro->dev)
netdev_hold(ro->dev, &ro->dev_tracker, GFP_KERNEL);
+
+ if (dev) {
+ /* Either raw_notify() sees ro->dev on NETDEV_DOWN or
+ * we see the cleared IFF_UP, pairs with raw_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+ }
}
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -592,17 +630,19 @@ static int raw_setsockopt(struct socket *sock, int level, int optname,
return -EFAULT;
}
- rtnl_lock();
lock_sock(sk);
+ /* The reg_state check is only a shortcut: On a concurrent
+ * unregistration or namespace change raw_notify() waits for
+ * lock_sock() and removes the filters set here.
+ */
dev = ro->dev;
- if (ro->bound && dev) {
- if (dev->reg_state != NETREG_REGISTERED) {
- if (count > 1)
- kfree(filter);
- err = -ENODEV;
- goto out_fil;
- }
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED) {
+ if (count > 1)
+ kfree(filter);
+ err = -ENODEV;
+ goto out_fil;
}
if (ro->bound) {
@@ -639,7 +679,6 @@ static int raw_setsockopt(struct socket *sock, int level, int optname,
out_fil:
release_sock(sk);
- rtnl_unlock();
break;
@@ -652,15 +691,14 @@ static int raw_setsockopt(struct socket *sock, int level, int optname,
err_mask &= CAN_ERR_MASK;
- rtnl_lock();
lock_sock(sk);
+ /* lockless shortcut, see CAN_RAW_FILTER */
dev = ro->dev;
- if (ro->bound && dev) {
- if (dev->reg_state != NETREG_REGISTERED) {
- err = -ENODEV;
- goto out_err;
- }
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED) {
+ err = -ENODEV;
+ goto out_err;
}
/* remove current error mask */
@@ -682,7 +720,6 @@ static int raw_setsockopt(struct socket *sock, int level, int optname,
out_err:
release_sock(sk);
- rtnl_unlock();
break;
@@ -775,29 +812,40 @@ static int raw_getsockopt(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_RAW_FILTER: {
+ struct can_filter *filter = NULL;
+ int fsize = 0;
int err = 0;
lock_sock(sk);
if (ro->count > 0) {
- int fsize = ro->count * sizeof(struct can_filter);
+ fsize = ro->count * sizeof(struct can_filter);
/* user space buffer to small for filter list? */
if (len < fsize) {
/* return -ERANGE and needed space in optlen */
err = -ERANGE;
- if (put_user(fsize, optlen))
- err = -EFAULT;
} else {
if (len > fsize)
len = fsize;
- if (copy_to_user(optval, ro->filter, len))
- err = -EFAULT;
+ /* copy to user space after release_sock() */
+ filter = kmemdup(ro->filter, len, GFP_KERNEL);
+ if (!filter)
+ err = -ENOMEM;
}
} else {
len = 0;
}
release_sock(sk);
+ if (err == -ERANGE) {
+ if (put_user(fsize, optlen))
+ err = -EFAULT;
+ } else if (filter) {
+ if (copy_to_user(optval, filter, len))
+ err = -EFAULT;
+ kfree(filter);
+ }
+
if (!err)
err = put_user(len, optlen);
return err;
diff --git a/net/core/dev.c b/net/core/dev.c
index 397fd0f9cc7a..f42ec2837143 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -3890,7 +3890,9 @@ static enum skb_drop_reason qdisc_pkt_len_init(struct sk_buff *skb)
if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) {
const struct tcphdr *th;
- if (!pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr)))
+ /* A header in front of skb->data gives a negative offset. */
+ if ((int)hdr_len < 0 ||
+ !pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr)))
return SKB_DROP_REASON_SKB_BAD_GSO;
th = (const struct tcphdr *)(skb->data + hdr_len);
@@ -5760,6 +5762,8 @@ static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc,
}
if (unlikely(skb_vlan_tag_present(skb)) && !netdev_uses_dsa(skb->dev)) {
+ unsigned int priority_tag_depth = 1;
+
check_vlan_id:
if (skb_vlan_tag_get_id(skb)) {
/* Vlan id is non 0 and vlan_do_receive() above couldn't
@@ -5771,6 +5775,10 @@ static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc,
* 802.1Q or 802.1AD and vlan_do_receive() above could
* not find vlan dev for vlan id 0.
*/
+ if (unlikely(priority_tag_depth >= VLAN_MAX_DEPTH))
+ goto drop;
+
+ priority_tag_depth++;
__vlan_hwaccel_clear_tag(skb);
skb = skb_vlan_untag(skb);
if (unlikely(!skb))
diff --git a/net/core/filter.c b/net/core/filter.c
index ce290de4b165..2d6e2f772a60 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2278,9 +2278,13 @@ static int bpf_out_neigh_v6(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v6(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct ipv6hdr *ip6h = ipv6_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct ipv6hdr *ip6h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip6h))))
+ goto out_drop;
+ ip6h = ipv6_hdr(skb);
if (!nh) {
struct dst_entry *dst;
@@ -2383,9 +2387,13 @@ static int bpf_out_neigh_v4(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct iphdr *ip4h = ip_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct iphdr *ip4h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip4h))))
+ goto out_drop;
+ ip4h = ip_hdr(skb);
if (!nh) {
struct flowi4 fl4 = {
@@ -2435,15 +2443,14 @@ static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
static int __bpf_redirect_neigh(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- struct ethhdr *ethh = eth_hdr(skb);
-
- if (unlikely(skb->mac_header >= skb->network_header))
+ if (unlikely(skb_network_offset(skb) < ETH_HLEN ||
+ !pskb_may_pull(skb, ETH_HLEN)))
goto out;
bpf_push_mac_rcsum(skb);
- if (is_multicast_ether_addr(ethh->h_dest))
+ if (is_multicast_ether_addr(eth_hdr(skb)->h_dest))
goto out;
- skb_pull(skb, sizeof(*ethh));
+ skb_pull(skb, ETH_HLEN);
skb_unset_mac_header(skb);
skb_reset_network_header(skb);
@@ -7618,7 +7625,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
if (sk_unhashed(sk))
return -EOPNOTSUPP;
if (sk_is_refcounted(sk) &&
- unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
return -ENOENT;
skb_orphan(skb);
diff --git a/net/core/sock.c b/net/core/sock.c
index 181a66488ec2..6a718b8e1dde 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -3131,7 +3131,7 @@ void __release_sock(struct sock *sk)
spin_unlock_bh(&sk->sk_lock.slock);
- while (1) {
+ do {
next = skb->next;
prefetch(next);
DEBUG_NET_WARN_ON_ONCE(skb_dst_is_noref(skb));
@@ -3139,12 +3139,10 @@ void __release_sock(struct sock *sk)
sk_backlog_rcv(sk, skb);
skb = next;
- if (!skb)
- break;
if (!(++nb & 15))
cond_resched();
- }
+ } while (skb);
spin_lock_bh(&sk->sk_lock.slock);
}
diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c
index 53fcb2089cde..019c16d6d695 100644
--- a/net/ipv4/cipso_ipv4.c
+++ b/net/ipv4/cipso_ipv4.c
@@ -2015,6 +2015,10 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
cipso_ptr = &opt->opt.__data[cipso_off];
cipso_len = cipso_ptr[1];
+ optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
+ opt->opt.optlen);
+ optlen_new -= cipso_len;
+
if (opt->opt.srr > opt->opt.cipso)
opt->opt.srr -= cipso_len;
if (opt->opt.rr > opt->opt.cipso)
@@ -2028,11 +2032,12 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
memmove(cipso_ptr, cipso_ptr + cipso_len,
opt->opt.optlen - cipso_off - cipso_len);
- optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
- opt->opt.optlen);
hdr_delta = opt->opt.optlen;
opt->opt.optlen = (optlen_new + 3) & ~3;
hdr_delta -= opt->opt.optlen;
+
+ memset(&opt->opt.__data[optlen_new], IPOPT_END,
+ opt->opt.optlen - optlen_new);
} else {
/* only the cipso option was present on the socket so we can
* remove the entire option struct */
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index 2cb125bbfb9d..61fbcaa913b3 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1105,8 +1105,8 @@ static void reqsk_timer_handler(struct timer_list *t)
* hold another count to prevent use-after-free and
* call reqsk_put() just before return.
*/
- refcount_set(&nreq->rsk_refcnt, 2 + 1);
timer_setup(&nreq->rsk_timer, reqsk_timer_handler, TIMER_PINNED);
+ refcount_set_release(&nreq->rsk_refcnt, 2 + 1);
reqsk_queue_migrated(&inet_csk(nsk)->icsk_accept_queue, req);
req = nreq;
@@ -1465,7 +1465,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,
goto child_put;
}
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
if (inet_csk_reqsk_queue_add(sk, nreq, child)) {
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMIGRATEREQSUCCESS);
reqsk_migrate_reset(req);
@@ -1519,7 +1519,7 @@ void inet_csk_listen_stop(struct sock *sk)
if (nsk) {
nreq = inet_reqsk_clone(req, nsk);
if (nreq) {
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
rcu_read_lock();
if (inet_csk_reqsk_queue_add(nsk, nreq, child)) {
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index e8adf8530596..e7b312f7cd98 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -510,7 +510,7 @@ struct sock *__inet_lookup_established(const struct net *net,
if (sk->sk_hash != hash)
continue;
if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) {
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
!inet_match(net, sk, acookie, ports, dif,
diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c
index 74b84ac418e9..e34deccdcb30 100644
--- a/net/ipv4/inet_timewait_sock.c
+++ b/net/ipv4/inet_timewait_sock.c
@@ -139,7 +139,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw,
* Also note that after this point, we lost our implicit reference
* so we are not allowed to use tw anymore.
*/
- refcount_set(&tw->tw_refcnt, 3);
+ refcount_set_release(&tw->tw_refcnt, 3);
/* Ensure tw_refcnt has been set before tw is published.
* smp_wmb() provides the necessary memory barrier to enforce this
@@ -324,7 +324,7 @@ void inet_twsk_purge(struct inet_hashinfo *hashinfo)
if (refcount_read(&sock_net(sk)->ns.count))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
continue;
if (refcount_read(&sock_net(sk)->ns.count)) {
diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index 5bf6c6777c34..f4a269938340 100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -1362,6 +1362,7 @@ static int erspan_tunnel_init(struct net_device *dev)
dev->features |= GRE_FEATURES;
dev->hw_features |= GRE_FEATURES;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+ dev->lltx = true;
netif_keep_dst(dev);
return ip_tunnel_init(dev);
diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c
index abd234f0861f..f9cec6b2bc1a 100644
--- a/net/ipv4/ipmr.c
+++ b/net/ipv4/ipmr.c
@@ -513,15 +513,22 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb, struct net_device *dev)
};
int err;
+ if (skb->protocol != htons(ETH_P_IP) || !pskb_inet_may_pull(skb)) {
+ DEV_STATS_INC(dev, tx_errors);
+ kfree_skb(skb);
+ return NETDEV_TX_OK;
+ }
+
+ rcu_read_lock();
err = ipmr_fib_lookup(net, &fl4, &mrt);
if (err < 0) {
+ rcu_read_unlock();
kfree_skb(skb);
return err;
}
DEV_STATS_ADD(dev, tx_bytes, skb->len);
DEV_STATS_INC(dev, tx_packets);
- rcu_read_lock();
/* Pairs with WRITE_ONCE() in vif_add() and vif_delete() */
ipmr_cache_report(mrt, skb, READ_ONCE(mrt->mroute_reg_vif_num),
@@ -1031,6 +1038,7 @@ static void ipmr_cache_resolve(struct net *net, struct mr_table *mrt,
static int ipmr_cache_report(const struct mr_table *mrt,
struct sk_buff *pkt, vifi_t vifi, int assert)
{
+ unsigned char pkt_cb[sizeof(pkt->cb)];
const int ihl = ip_hdrlen(pkt);
struct sock *mroute_sk;
struct igmphdr *igmp;
@@ -1086,17 +1094,25 @@ static int ipmr_cache_report(const struct mr_table *mrt,
msg = (struct igmpmsg *)skb_network_header(skb);
msg->im_vif = vifi;
msg->im_vif_hi = vifi >> 8;
- ipv4_pktinfo_prepare(mroute_sk, pkt, false);
- memcpy(skb->cb, pkt->cb, sizeof(skb->cb));
- /* Add our header */
- igmp = skb_put(skb, sizeof(struct igmphdr));
+ /* Add our header.
+ * Note that code, csum and group fields are cleared.
+ */
+ igmp = skb_put_zero(skb, sizeof(struct igmphdr));
igmp->type = assert;
msg->im_msgtype = assert;
- igmp->code = 0;
ip_hdr(skb)->tot_len = htons(skb->len); /* Fix the length */
skb->transport_header = skb->network_header;
}
+ memset(skb->cb, 0, sizeof(skb->cb));
+ /* Route queries from ipmr_get_route() carry NETLINK_CB, not IPCB. */
+ if (ip_hdr(pkt)->version) {
+ memcpy(pkt_cb, pkt->cb, sizeof(pkt_cb));
+ ipv4_pktinfo_prepare(mroute_sk, pkt, false);
+ *PKTINFO_SKB_CB(skb) = *PKTINFO_SKB_CB(pkt);
+ memcpy(pkt->cb, pkt_cb, sizeof(pkt_cb));
+ }
+
igmpmsg_netlink_event(mrt, skb);
/* Deliver to mrouted */
@@ -1991,7 +2007,8 @@ static void ip_mr_forward(struct net *net, struct mr_table *mrt,
}
/* Wrong interface: drop packet and (maybe) send PIM assert. */
- if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
+ if (true_vifi < 0 ||
+ rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
if (rt_is_output_route(skb_rtable(skb))) {
/* It is our own packet, looped back.
* Very complicated situation...
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 596c84fd5ab4..f1b0ab77b443 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -1358,8 +1358,8 @@ static unsigned int ipv4_default_advmss(const struct dst_entry *dst)
rcu_read_lock();
net = dst_dev_net_rcu(dst);
- advmss = max_t(unsigned int, ip_dst_mtu_configured(dst) - header_size,
- net->ipv4.ip_rt_min_advmss);
+ advmss = max_t(unsigned int, ipv4_mtu(dst) - header_size,
+ net->ipv4.ip_rt_min_advmss);
rcu_read_unlock();
return min(advmss, IPV4_MAX_PMTU - header_size);
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 16361ce3e806..43cf33f282c1 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -204,7 +204,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb,
child = icsk->icsk_af_ops->syn_recv_sock(sk, skb, req, dst,
NULL, &own_req, NULL);
if (child) {
- refcount_set(&req->rsk_refcnt, 1);
+ refcount_set_release(&req->rsk_refcnt, 1);
sock_rps_save_rxhash(child, skb);
if (rsk_drop_req(req)) {
diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c
index 5378ef3b6ab5..fbcd7d8c8d99 100644
--- a/net/ipv4/tcp_fastopen.c
+++ b/net/ipv4/tcp_fastopen.c
@@ -278,7 +278,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,
inet_csk_reset_xmit_timer(child, ICSK_TIME_RETRANS,
req->timeout, TCP_RTO_MAX);
- refcount_set(&req->rsk_refcnt, 2);
+ refcount_set_release(&req->rsk_refcnt, 2);
sk_mark_napi_id_set(child, skb);
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index eb32a0097844..60b2ed8b8fe7 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -1477,7 +1477,8 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block)
age = (now - tmp_tstamp + ADDRCONF_TIMER_FUZZ_MINUS) / HZ;
if (cfg.preferred_lft <= regen_advance + age) {
cfg.preferred_lft = regen_advance + age + 1;
- if (cfg.preferred_lft > cfg.valid_lft ||
+ if (cfg.preferred_lft <= regen_advance + age ||
+ cfg.preferred_lft > cfg.valid_lft ||
cfg.preferred_lft > if_public_preferred_lft) {
in6_ifa_put(ifp);
in6_dev_put(idev);
diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c
index 9bcec0828fe8..92b772ea224e 100644
--- a/net/ipv6/datagram.c
+++ b/net/ipv6/datagram.c
@@ -148,6 +148,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr,
__be32 fl6_flowlabel = 0;
__be32 old_fl6_flowlabel;
__be16 old_dport;
+ int old_bound_dev_if;
int addr_type;
int err;
@@ -218,6 +219,8 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr,
goto out;
}
+ old_bound_dev_if = sk->sk_bound_dev_if;
+
if (__ipv6_addr_needs_scope_id(addr_type)) {
if (addr_len >= sizeof(struct sockaddr_in6) &&
usin->sin6_scope_id) {
@@ -260,6 +263,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr,
sk->sk_v6_daddr = old_daddr;
np->flow_label = old_fl6_flowlabel;
inet->inet_dport = old_dport;
+ WRITE_ONCE(sk->sk_bound_dev_if, old_bound_dev_if);
goto out;
}
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index 70633fed6e6a..dee00ba290f6 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -71,7 +71,7 @@ struct sock *__inet6_lookup_established(const struct net *net,
continue;
if (!inet6_match(net, sk, saddr, daddr, ports, dif, sdif))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index 7cf429f11a1d..9cb3c1fea4c9 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -1902,6 +1902,7 @@ static int ip6erspan_tap_init(struct net_device *dev)
dev->mtu -= 8;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+ dev->lltx = true;
ip6erspan_tnl_link_config(tunnel, 1);
netdev_hold(dev, &tunnel->dev_tracker, GFP_KERNEL);
diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c
index 47afc8cef688..0ecb6af344fb 100644
--- a/net/ipv6/ip6mr.c
+++ b/net/ipv6/ip6mr.c
@@ -616,7 +616,7 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb,
.flowi6_mark = skb->mark,
};
- if (!pskb_inet_may_pull(skb))
+ if (skb->protocol != htons(ETH_P_IPV6) || !pskb_inet_may_pull(skb))
goto tx_err;
if (ip6mr_fib_lookup(net, &fl6, &mrt) < 0)
@@ -2146,7 +2146,8 @@ static void ip6_mr_forward(struct net *net, struct mr_table *mrt,
/*
* Wrong interface: drop packet and (maybe) send PIM assert.
*/
- if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
+ if (true_vifi < 0 ||
+ rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
atomic_long_inc(&c->_c.mfc_un.res.wrong_if);
if (true_vifi >= 0 && mrt->mroute_do_assert &&
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index deaaae63bda0..27bb9443c884 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -652,6 +652,9 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length,
if (unlikely(!skb))
return 0;
+ /* l3mdev_ip6_out() may have replaced the skb dst. */
+ rt = dst_rt6_info(skb_dst(skb));
+
/* Acquire rcu_read_lock() in case we need to use rt->rt6i_idev
* in the error path. Since skb has been freed, the dst could
* have been queued for deletion.
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 94fa1b34be5c..e6c2db89860c 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -3252,7 +3252,7 @@ EXPORT_SYMBOL_GPL(ip6_sk_redirect);
static unsigned int ip6_default_advmss(const struct dst_entry *dst)
{
struct net_device *dev = dst_dev(dst);
- unsigned int mtu = ip6_dst_mtu_configured(dst);
+ unsigned int mtu = dst6_mtu(dst);
struct net *net;
mtu -= sizeof(struct ipv6hdr) + sizeof(struct tcphdr);
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
index 7cdfab3a7809..6c83cb02d434 100644
--- a/net/l2tp/l2tp_core.c
+++ b/net/l2tp/l2tp_core.c
@@ -315,10 +315,19 @@ EXPORT_SYMBOL_GPL(l2tp_v2_session_get);
struct l2tp_session *l2tp_session_get(const struct net *net, struct sock *sk, int pver,
u32 tunnel_id, u32 session_id)
{
+ struct l2tp_session *session;
+
if (pver == L2TP_HDR_VER_2)
return l2tp_v2_session_get(net, tunnel_id, session_id);
- else
- return l2tp_v3_session_get(net, sk, session_id);
+
+ /* l2tp_v3_session_get() doesn't check sk if the ID hasn't collided */
+ session = l2tp_v3_session_get(net, sk, session_id);
+ if (session && session->tunnel->sock != sk) {
+ l2tp_session_put(session);
+ return NULL;
+ }
+
+ return session;
}
EXPORT_SYMBOL_GPL(l2tp_session_get);
diff --git a/net/mac80211/tdls.c b/net/mac80211/tdls.c
index b4bedf0b2b55..33657ca92ae5 100644
--- a/net/mac80211/tdls.c
+++ b/net/mac80211/tdls.c
@@ -2016,7 +2016,7 @@ void
ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
struct sk_buff *skb)
{
- struct ieee80211_tdls_data *tf = (void *)skb->data;
+ struct ieee80211_tdls_data *tf;
struct wiphy *wiphy = sdata->local->hw.wiphy;
lockdep_assert_wiphy(wiphy);
@@ -2032,6 +2032,7 @@ ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
* The packet/size was already validated by mac80211 Rx path, only look
* at the action type.
*/
+ tf = (void *)skb->data;
switch (tf->action_code) {
case WLAN_TDLS_CHANNEL_SWITCH_REQUEST:
ieee80211_process_tdls_channel_switch_req(sdata, skb);
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index 0fad278d8f24..fff9ea0dba07 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -276,7 +276,6 @@ ieee80211_tx_h_dynamic_ps(struct ieee80211_tx_data *tx)
static ieee80211_tx_result debug_noinline
ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx)
{
-
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data;
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb);
bool assoc = false;
@@ -284,23 +283,6 @@ ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx)
if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED))
return TX_CONTINUE;
- if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) &&
- test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) &&
- !ieee80211_is_probe_req(hdr->frame_control) &&
- !ieee80211_is_any_nullfunc(hdr->frame_control))
- /*
- * When software scanning only nullfunc frames (to notify
- * the sleep state to the AP) and probe requests (for the
- * active scan) are allowed, all other frames should not be
- * sent and we should not get here, but if we do
- * nonetheless, drop them to avoid sending them
- * off-channel. See the link below and
- * ieee80211_start_scan() for more.
- *
- * http://article.gmane.org/gmane.linux.kernel.wireless.general/30089
- */
- return TX_DROP;
-
if (tx->sdata->vif.type == NL80211_IFTYPE_OCB)
return TX_CONTINUE;
@@ -810,6 +792,33 @@ ieee80211_tx_h_rate_ctrl(struct ieee80211_tx_data *tx)
return TX_CONTINUE;
}
+static ieee80211_tx_result debug_noinline
+ieee80211_tx_h_check_offchannel(struct ieee80211_tx_data *tx)
+{
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data;
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb);
+
+ if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED))
+ return TX_CONTINUE;
+
+ if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) &&
+ test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) &&
+ !ieee80211_is_probe_req(hdr->frame_control) &&
+ !ieee80211_is_any_nullfunc(hdr->frame_control)) {
+ /*
+ * When software scanning only nullfunc frames (to notify
+ * the sleep state to the AP) and probe requests (for the
+ * active scan) are allowed, all other frames should not be
+ * sent and we should not get here, but if we do
+ * nonetheless, drop them to avoid sending them
+ * off-channel. See __ieee80211_start_scan() for more.
+ */
+ return TX_DROP;
+ }
+
+ return TX_CONTINUE;
+}
+
static __le16 ieee80211_tx_next_seq(struct sta_info *sta, int tid)
{
u16 *seq = &sta->tid_seq[tid];
@@ -1119,6 +1128,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
struct sk_buff *purge_skb = NULL;
if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else if (test_bit(HT_AGG_STATE_WANT_START, &tid_tx->state)) {
/*
@@ -1150,6 +1160,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
if (!tid_tx) {
/* do nothing, let packet pass through */
} else if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else {
queued = true;
@@ -1853,6 +1864,7 @@ static int invoke_tx_handlers_late(struct ieee80211_tx_data *tx)
goto txh_done;
}
+ CALL_TXH(ieee80211_tx_h_check_offchannel);
CALL_TXH(ieee80211_tx_h_michael_mic_add);
CALL_TXH(ieee80211_tx_h_sequence);
CALL_TXH(ieee80211_tx_h_fragment);
@@ -4667,10 +4679,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata,
queue = ieee80211_select_queue(sdata, sta, skb);
skb_set_queue_mapping(skb, queue);
- if (unlikely(test_bit(SCAN_SW_SCANNING, &local->scanning)) &&
- test_bit(SDATA_STATE_OFFCHANNEL, &sdata->state))
- goto out_free;
-
skb = skb_share_check(skb, GFP_ATOMIC);
if (unlikely(!skb))
return;
@@ -4748,11 +4756,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata,
ieee80211_tpt_led_trig_tx(local, len);
ieee80211_tx_8023(sdata, skb, sta, false);
-
- return;
-
-out_free:
- kfree_skb(skb);
}
static bool ieee80211_check_mcast_offload(struct ieee80211_sub_if_data *sdata,
diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
index 4de1375bf2a5..6e51b16f10d9 100644
--- a/net/mptcp/pm_kernel.c
+++ b/net/mptcp/pm_kernel.c
@@ -1270,6 +1270,7 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk,
* old status is known (TCP_CLOSE).
*/
inet_sk_state_store(newsk, TCP_LISTEN);
+ sock_set_flag(newsk, SOCK_RCU_FREE);
lock_sock(ssk);
WRITE_ONCE(mptcp_subflow_ctx(ssk)->pm_listener, true);
err = __inet_listen_sk(ssk, backlog);
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index ed41368392c9..c32ce7da5fd6 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -3485,6 +3485,24 @@ static void mptcp_copy_ip_options(struct sock *newsk, const struct sock *sk)
rcu_read_unlock();
}
+static void mptcp_sk_clone_destroy(struct sock *nsk)
+{
+ struct mptcp_sock *msk = mptcp_sk(nsk);
+
+ mptcp_release_sched(msk);
+ mptcp_set_state(nsk, TCP_CLOSE);
+ /* inet_csk_prepare_forced_close() clears TCP sock_ops state via
+ * tcp_sk(), but nsk is an MPTCP master socket; keep the inet-level
+ * destroy preparation here.
+ */
+ bh_unlock_sock(nsk);
+ sock_put(nsk);
+ sock_set_flag(nsk, SOCK_DEAD);
+ tcp_orphan_count_inc();
+ inet_sk(nsk)->inet_num = 0;
+ inet_csk_destroy_sock(nsk);
+}
+
struct sock *mptcp_sk_clone_init(const struct sock *sk,
const struct mptcp_options_received *mp_opt,
struct sock *ssk,
@@ -3544,11 +3562,6 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,
list_add(&subflow->node, &msk->conn_list);
sock_hold(ssk);
- /* new mpc subflow takes ownership of the newly
- * created mptcp socket
- */
- mptcp_token_accept(subflow_req, msk);
-
/* set msk addresses early to ensure mptcp_pm_get_local_id()
* uses the correct data
*/
@@ -3557,6 +3570,14 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,
mptcp_rcv_space_init(msk, ssk);
msk->rcvq_space.time = mptcp_stamp();
+ if (!mptcp_token_accept(subflow_req, msk)) {
+ list_del_init(&subflow->node);
+ WRITE_ONCE(msk->first, NULL);
+ sock_put(ssk);
+ mptcp_sk_clone_destroy(nsk);
+ return NULL;
+ }
+
if (mp_opt->suboptions & OPTION_MPTCP_MPC_ACK)
__mptcp_subflow_fully_established(msk, subflow, mp_opt);
bh_unlock_sock(nsk);
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index dbfe959d3a18..d9c20de69dc5 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -544,14 +544,16 @@ struct mptcp_subflow_context {
stale : 1, /* unable to snd/rcv data, do not use for xmit */
valid_csum_seen : 1, /* at least one csum validated */
is_mptfo : 1, /* subflow is doing TFO */
- close_event_done : 1, /* has done the post-closed part */
mpc_drop : 1, /* the MPC option has been dropped in a rtx */
resetting : 1, /* subflow is resetting */
- __unused : 7;
+ __unused : 8;
bool data_avail;
bool scheduled;
bool pm_listener; /* a listener managed by the kernel PM? */
bool fully_established; /* path validated */
+ bool close_event_done; /* has done the post-closed part,
+ * protected by msk socket lock
+ */
u32 remote_nonce;
u64 thmac;
u32 local_nonce;
@@ -992,9 +994,11 @@ static inline void mptcp_token_init_request(struct request_sock *req)
}
int mptcp_token_new_request(struct request_sock *req);
+void mptcp_token_move_request(struct request_sock *req,
+ struct request_sock *new_req);
void mptcp_token_destroy_request(struct request_sock *req);
int mptcp_token_new_connect(struct sock *ssk);
-void mptcp_token_accept(struct mptcp_subflow_request_sock *r,
+bool mptcp_token_accept(struct mptcp_subflow_request_sock *r,
struct mptcp_sock *msk);
bool mptcp_token_exists(u32 token);
struct mptcp_sock *mptcp_token_get_sock(struct net *net, u32 token);
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 17d392b042ba..8f8ffc88214b 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -78,6 +78,8 @@ void mptcp_subflow_reqsk_clone(struct request_sock *req,
}
new_subflow_req->msk = msk;
+
+ mptcp_token_move_request(req, new_req);
}
static void subflow_generate_hmac(u64 key1, u64 key2, u32 nonce1, u32 nonce2,
@@ -379,6 +381,10 @@ static void subflow_prep_synack(const struct sock *sk, struct request_sock *req,
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
struct inet_request_sock *ireq = inet_rsk(req);
+ /* MP_JOIN cannot use TFO, do not send a cookie in the SYN/ACK */
+ if (foc && mptcp_subflow_rsk(req)->mp_join)
+ foc->len = -1;
+
/* clear tstamp_ok, as needed depending on cookie */
if (foc && foc->len > -1)
ireq->tstamp_ok = 0;
@@ -893,6 +899,13 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
fallback = true;
} else if (subflow_req->mp_join) {
+ /* a SYN skb here comes from TFO, which MP_JOIN cannot use: just
+ * fall back to the regular path.
+ */
+ if ((TCP_SKB_CB(skb)->tcp_flags & (TCPHDR_SYN | TCPHDR_ACK)) ==
+ TCPHDR_SYN)
+ return NULL;
+
mptcp_get_options(skb, &mp_opt);
if (!(mp_opt.suboptions & OPTION_MPTCP_MPJ_ACK))
fallback = true;
@@ -924,8 +937,10 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
if (ctx->mp_capable) {
ctx->conn = mptcp_sk_clone_init(listener->conn, &mp_opt, child, req);
- if (!ctx->conn)
+ if (!ctx->conn) {
+ fallback = true;
goto fallback;
+ }
ctx->subflow_id = 1;
owner = mptcp_sk(ctx->conn);
diff --git a/net/mptcp/token.c b/net/mptcp/token.c
index 5bb924534387..d5735d3ac793 100644
--- a/net/mptcp/token.c
+++ b/net/mptcp/token.c
@@ -180,6 +180,43 @@ int mptcp_token_new_connect(struct sock *ssk)
return 0;
}
+/**
+ * mptcp_token_move_request - move request token ownership to a clone
+ * @req: original request socket
+ * @new_req: cloned request socket
+ *
+ * Move the token hash entry from the original request to its clone.
+ */
+void mptcp_token_move_request(struct request_sock *req,
+ struct request_sock *new_req)
+{
+ struct mptcp_subflow_request_sock *subflow_req = mptcp_subflow_rsk(req);
+ struct mptcp_subflow_request_sock *new_subflow_req;
+ struct mptcp_subflow_request_sock *pos;
+ struct token_bucket *bucket;
+
+ new_subflow_req = mptcp_subflow_rsk(new_req);
+
+ if (hlist_nulls_unhashed_lockless(&subflow_req->token_node)) {
+ mptcp_token_init_request(new_req);
+ return;
+ }
+
+ bucket = token_bucket(subflow_req->token);
+ spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&subflow_req->token_node)) {
+ mptcp_token_init_request(new_req);
+ } else {
+ pos = __token_lookup_req(bucket, subflow_req->token);
+ if (pos == subflow_req)
+ hlist_nulls_replace_init_rcu(&subflow_req->token_node,
+ &new_subflow_req->token_node);
+ else
+ mptcp_token_init_request(new_req);
+ }
+ spin_unlock_bh(&bucket->lock);
+}
+
/**
* mptcp_token_accept - replace a req sk with full sock in token hash
* @req: the request socket to be removed
@@ -187,24 +224,36 @@ int mptcp_token_new_connect(struct sock *ssk)
*
* Called when a SYN packet creates a new logical connection, i.e.
* is not a join request.
+ *
+ * Return: true on success.
*/
-void mptcp_token_accept(struct mptcp_subflow_request_sock *req,
+bool mptcp_token_accept(struct mptcp_subflow_request_sock *req,
struct mptcp_sock *msk)
{
struct mptcp_subflow_request_sock *pos;
struct sock *sk = (struct sock *)msk;
struct token_bucket *bucket;
+ bool ret = false;
- sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
bucket = token_bucket(req->token);
spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&req->token_node))
+ goto unlock;
- /* pedantic lookup check for the moved token */
pos = __token_lookup_req(bucket, req->token);
- if (!WARN_ON_ONCE(pos != req))
- hlist_nulls_del_init_rcu(&req->token_node);
+ if (pos != req)
+ goto unlock;
+
+ hlist_nulls_del_init_rcu(&req->token_node);
__sk_nulls_add_node_rcu((struct sock *)msk, &bucket->msk_chain);
+ ret = true;
+
+unlock:
spin_unlock_bh(&bucket->lock);
+ if (ret)
+ sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
+
+ return ret;
}
bool mptcp_token_exists(u32 token)
@@ -353,16 +402,21 @@ void mptcp_token_destroy_request(struct request_sock *req)
struct mptcp_subflow_request_sock *pos;
struct token_bucket *bucket;
- if (hlist_nulls_unhashed(&subflow_req->token_node))
+ if (hlist_nulls_unhashed_lockless(&subflow_req->token_node))
return;
bucket = token_bucket(subflow_req->token);
spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&subflow_req->token_node))
+ goto unlock;
+
pos = __token_lookup_req(bucket, subflow_req->token);
- if (!WARN_ON_ONCE(pos != subflow_req)) {
+ if (pos == subflow_req) {
hlist_nulls_del_init_rcu(&pos->token_node);
bucket->chain_len--;
}
+
+unlock:
spin_unlock_bh(&bucket->lock);
}
diff --git a/net/mptcp/token_test.c b/net/mptcp/token_test.c
index 4fc39fa2e262..be9acce8a567 100644
--- a/net/mptcp/token_test.c
+++ b/net/mptcp/token_test.c
@@ -99,7 +99,7 @@ static void mptcp_token_test_accept(struct kunit *test)
KUNIT_ASSERT_EQ(test, 0,
mptcp_token_new_request((struct request_sock *)req));
msk->token = req->token;
- mptcp_token_accept(req, msk);
+ KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk));
KUNIT_EXPECT_PTR_EQ(test, msk, mptcp_token_get_sock(&init_net, msk->token));
/* this is now a no-op */
@@ -122,7 +122,7 @@ static void mptcp_token_test_destroyed(struct kunit *test)
KUNIT_ASSERT_EQ(test, 0,
mptcp_token_new_request((struct request_sock *)req));
msk->token = req->token;
- mptcp_token_accept(req, msk);
+ KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk));
/* simulate race on removal */
refcount_set(&sk->sk_refcnt, 0);
diff --git a/net/netrom/af_netrom.c b/net/netrom/af_netrom.c
index fc7bd19ad0e3..99d9fe19d27f 100644
--- a/net/netrom/af_netrom.c
+++ b/net/netrom/af_netrom.c
@@ -96,16 +96,22 @@ static void nr_remove_socket(struct sock *sk)
}
/*
- * Kill all bound sockets on a dropped device.
+ * Kill all bound sockets on a dropped device. Each socket is disconnected
+ * under its lock, which its timers and nr_rx_frame() hold while they work
+ * on it; nr_info_show() takes the two locks in the same order.
*/
static void nr_kill_by_device(struct net_device *dev)
{
struct sock *s;
spin_lock_bh(&nr_list_lock);
- sk_for_each(s, &nr_list)
- if (nr_sk(s)->device == dev)
- nr_disconnect(s, ENETUNREACH);
+ sk_for_each(s, &nr_list) {
+ if (nr_sk(s)->device != dev)
+ continue;
+ bh_lock_sock(s);
+ nr_disconnect(s, ENETUNREACH);
+ bh_unlock_sock(s);
+ }
spin_unlock_bh(&nr_list_lock);
}
@@ -208,29 +214,40 @@ static struct sock *nr_find_peer(unsigned char index, unsigned char id,
}
/*
- * Find next free circuit ID.
+ * Give a socket the next free circuit ID. The search and the assignment
+ * are done under nr_list_lock, which the caller holds, with the socket
+ * already on nr_list: two sockets cannot pick the same ID.
*/
-static unsigned short nr_find_next_circuit(void)
+static void nr_assign_circuit(struct nr_sock *nr)
{
unsigned short id = circuit;
unsigned char i, j;
- struct sock *sk;
+ struct sock *s;
+ bool used;
+
+ lockdep_assert_held(&nr_list_lock);
for (;;) {
i = id / 256;
j = id % 256;
if (i != 0 && j != 0) {
- sk = nr_find_socket(i, j, NULL);
- if (!sk)
+ used = false;
+ sk_for_each(s, &nr_list)
+ if (nr_sk(s)->my_index == i && nr_sk(s)->my_id == j) {
+ used = true;
+ break;
+ }
+ if (!used)
break;
- sock_put(sk);
}
id++;
}
- return id;
+ nr->my_index = i;
+ nr->my_id = j;
+ circuit = id + 1;
}
/*
@@ -459,8 +476,7 @@ static int nr_create(struct net *net, struct socket *sock, int protocol,
msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_timeout));
nr->t2 =
msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_acknowledge_delay));
- nr->n2 =
- msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_maximum_tries));
+ nr->n2 = READ_ONCE(sysctl_netrom_transport_maximum_tries);
nr->t4 =
msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_busy_delay));
nr->idle =
@@ -710,14 +726,10 @@ static int nr_connect(struct socket *sock, struct sockaddr *uaddr,
nr->dest_addr = addr->sax25_call;
- release_sock(sk);
- circuit = nr_find_next_circuit();
- lock_sock(sk);
-
- nr->my_index = circuit / 256;
- nr->my_id = circuit % 256;
-
- circuit++;
+ /* The socket is on nr_list now, so others see the ID it gets */
+ spin_lock_bh(&nr_list_lock);
+ nr_assign_circuit(nr);
+ spin_unlock_bh(&nr_list_lock);
/* Move to connecting socket, start sending Connect Requests */
sock->state = SS_CONNECTING;
@@ -949,10 +961,16 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev)
bh_lock_sock(sk);
skb_reset_transport_header(skb);
- if (frametype == NR_CONNACK && skb->len == 22)
- nr_sk(sk)->bpqext = 1;
- else
- nr_sk(sk)->bpqext = 0;
+ /*
+ * Only the circuit set-up frames tell whether the peer uses
+ * the BPQ extension: a CONNACK of 22 bytes, or a CONNREQ of
+ * 37 bytes (repeated, for a circuit already accepted). The
+ * other frames say nothing about it.
+ */
+ if (frametype == NR_CONNACK)
+ nr_sk(sk)->bpqext = (skb->len == 22);
+ else if (frametype == NR_CONNREQ)
+ nr_sk(sk)->bpqext = (skb->len == 37);
ret = nr_process_rx_frame(sk, skb);
bh_unlock_sock(sk);
@@ -1014,23 +1032,29 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev)
nr_make->your_index = circuit_index;
nr_make->your_id = circuit_id;
+ /*
+ * Put the new socket on nr_list as it gets its circuit ID, under the
+ * same lock, so that no other socket can be given the same ID.
+ * nr_list_lock is not taken under a socket lock (nr_info_show() takes
+ * them the other way round).
+ */
bh_unlock_sock(sk);
- circuit = nr_find_next_circuit();
+ spin_lock_bh(&nr_list_lock);
+ nr_assign_circuit(nr_make);
+ sk_add_node(make, &nr_list);
+ spin_unlock_bh(&nr_list_lock);
bh_lock_sock(sk);
- nr_make->my_index = circuit / 256;
- nr_make->my_id = circuit % 256;
-
- circuit++;
-
/* Window negotiation */
- if (window < nr_make->window)
- nr_make->window = window;
+ nr_negotiate_window(nr_make, window);
- /* L4 timeout negotiation */
+ /*
+ * L4 timeout negotiation. Ignore a proposal shorter than the
+ * smallest transport timeout the sysctl accepts (5 s), such as 0.
+ */
if (skb->len == 37) {
timeout = skb->data[36] * 256 + skb->data[35];
- if (timeout * HZ < nr_make->t1)
+ if (timeout >= 5 && timeout * HZ < nr_make->t1)
nr_make->t1 = timeout * HZ;
nr_make->bpqext = 1;
} else {
@@ -1054,8 +1078,6 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev)
bh_unlock_sock(sk);
sock_put(sk);
- nr_insert_socket(make);
-
nr_start_heartbeat(make);
nr_start_idletimer(make);
@@ -1163,9 +1185,9 @@ static int nr_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
goto out;
}
- nr_output(sk, skb); /* Shove it onto the queue */
-
- err = len;
+ err = nr_output(sk, skb); /* Shove it onto the queue */
+ if (!err)
+ err = len;
out:
release_sock(sk);
return err;
@@ -1214,10 +1236,13 @@ static int nr_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
}
if (sax != NULL) {
+ /*
+ * The NET/ROM headers were pulled when the frame was queued:
+ * give the node at the other end of the circuit.
+ */
memset(sax, 0, sizeof(*sax));
sax->sax25_family = AF_NETROM;
- skb_copy_from_linear_data_offset(skb, 7, sax->sax25_call.ax25_call,
- AX25_ADDR_LEN);
+ sax->sax25_call = nr_sk(sk)->dest_addr;
msg->msg_namelen = sizeof(*sax);
}
diff --git a/net/netrom/nr_in.c b/net/netrom/nr_in.c
index b90a47dcf165..805881e5d6c0 100644
--- a/net/netrom/nr_in.c
+++ b/net/netrom/nr_in.c
@@ -67,26 +67,44 @@ static int nr_queue_rx_frame(struct sock *sk, struct sk_buff *skb, int more)
return 0;
}
- if (!more && nr->fraglen > 0) { /* End of fragment */
- nr->fraglen += skb->len;
- skb_queue_tail(&nr->frag_queue, skb);
-
- if ((skbn = alloc_skb(nr->fraglen, GFP_ATOMIC)) == NULL)
- return 1;
+ if (nr->fraglen > 0) { /* End of fragment */
+ skbn = alloc_skb(nr->fraglen + skb->len, GFP_ATOMIC);
+ if (!skbn)
+ goto busy;
skb_reset_transport_header(skbn);
- while ((skbo = skb_dequeue(&nr->frag_queue)) != NULL) {
+ skb_queue_walk(&nr->frag_queue, skbo)
skb_copy_from_linear_data(skbo,
skb_put(skbn, skbo->len),
skbo->len);
- kfree_skb(skbo);
- }
+ skb_copy_from_linear_data(skb, skb_put(skbn, skb->len),
+ skb->len);
+ }
+
+ if (sock_queue_rcv_skb(sk, skbn) != 0) {
+ if (skbn != skb)
+ kfree_skb(skbn);
+ goto busy;
+ }
+ if (skbn != skb) {
+ /* The whole message is delivered: drop its pieces */
+ skb_queue_purge(&nr->frag_queue);
nr->fraglen = 0;
+ kfree_skb(skb);
}
- return sock_queue_rcv_skb(sk, skbn);
+ return 0;
+
+busy:
+ /*
+ * Not delivered: give the frame back as it came, headers included,
+ * and leave the earlier fragments where they are, so that the
+ * caller can keep it on reseq_queue and deliver it later.
+ */
+ skb_push(skb, NR_NETWORK_LEN + NR_TRANSPORT_LEN);
+ return 1;
}
/*
@@ -111,7 +129,7 @@ static int nr_state1_machine(struct sock *sk, struct sk_buff *skb,
nr->vl = 0;
nr->state = NR_STATE_3;
nr->n2count = 0;
- nr->window = skb->data[20];
+ nr_negotiate_window(nr, skb->data[20]);
sk->sk_state = TCP_ESTABLISHED;
if (!sock_flag(sk, SOCK_DEAD))
sk->sk_state_change(sk);
@@ -176,7 +194,7 @@ static int nr_state3_machine(struct sock *sk, struct sk_buff *skb, int frametype
struct sk_buff *skbn;
unsigned short save_vr;
unsigned short nr, ns;
- int queued = 0;
+ int queued = 0, more;
nr = skb->data[18];
@@ -258,7 +276,9 @@ static int nr_state3_machine(struct sock *sk, struct sk_buff *skb, int frametype
while ((skbn = skb_dequeue(&nrom->reseq_queue)) != NULL) {
ns = skbn->data[17];
if (ns == nrom->vr) {
- if (nr_queue_rx_frame(sk, skbn, frametype & NR_MORE_FLAG) == 0) {
+ /* The flags of this frame, not of the one just received */
+ more = skbn->data[19] & NR_MORE_FLAG;
+ if (nr_queue_rx_frame(sk, skbn, more) == 0) {
nrom->vr = (nrom->vr + 1) % NR_MODULUS;
} else {
nrom->condition |= NR_COND_OWN_RX_BUSY;
diff --git a/net/netrom/nr_out.c b/net/netrom/nr_out.c
index 2b3cbceb0b52..150cb4263d31 100644
--- a/net/netrom/nr_out.c
+++ b/net/netrom/nr_out.c
@@ -28,11 +28,19 @@
/*
* This is where all NET/ROM frames pass, except for IP-over-NET/ROM which
* cannot be fragmented in this manner.
+ *
+ * A message too long for one frame is cut into fragments built on a list
+ * of our own, and only put on the write queue once all of them exist: a
+ * message that cannot be fragmented whole (no memory, or a signal while
+ * waiting for send buffer space) is not sent at all, and the error is
+ * returned. Consumes skb in all cases.
*/
-void nr_output(struct sock *sk, struct sk_buff *skb)
+int nr_output(struct sock *sk, struct sk_buff *skb)
{
+ struct sk_buff_head frags;
struct sk_buff *skbn;
unsigned char transport[NR_TRANSPORT_LEN];
+ unsigned long flags;
int err, frontlen, len;
if (skb->len - NR_TRANSPORT_LEN > NR_MAX_PACKET_SIZE) {
@@ -42,10 +50,13 @@ void nr_output(struct sock *sk, struct sk_buff *skb)
frontlen = skb_headroom(skb);
+ __skb_queue_head_init(&frags);
+
while (skb->len > 0) {
if ((skbn = sock_alloc_send_skb(sk, frontlen + NR_MAX_PACKET_SIZE, 0, &err)) == NULL) {
+ __skb_queue_purge(&frags);
kfree_skb(skb);
- return;
+ return err;
}
skb_reserve(skbn, frontlen);
@@ -63,15 +74,21 @@ void nr_output(struct sock *sk, struct sk_buff *skb)
if (skb->len > 0)
skbn->data[4] |= NR_MORE_FLAG;
- skb_queue_tail(&sk->sk_write_queue, skbn); /* Throw it on the queue */
+ __skb_queue_tail(&frags, skbn);
}
kfree_skb(skb);
+
+ spin_lock_irqsave(&sk->sk_write_queue.lock, flags);
+ skb_queue_splice_tail_init(&frags, &sk->sk_write_queue);
+ spin_unlock_irqrestore(&sk->sk_write_queue.lock, flags);
} else {
skb_queue_tail(&sk->sk_write_queue, skb); /* Throw it on the queue */
}
nr_kick(sk);
+
+ return 0;
}
/*
diff --git a/net/netrom/nr_route.c b/net/netrom/nr_route.c
index 288c4a91ea9b..7fbacc5fee84 100644
--- a/net/netrom/nr_route.c
+++ b/net/netrom/nr_route.c
@@ -344,6 +344,24 @@ static void nr_remove_neigh_locked(struct nr_neigh *nr_neigh)
nr_neigh_put(nr_neigh);
}
+/*
+ * Remove route i of a node: move the routes after it up, and keep which on
+ * the route it selected. If that route is the one removed, which now
+ * selects the next one, as after a link failure. The caller drops the
+ * route's use of its neighbour. Called with nr_route_lock held.
+ */
+static void nr_node_remove_route_locked(struct nr_node *nr_node, int i)
+{
+ lockdep_assert_held(&nr_route_lock);
+
+ if (i < nr_node->which)
+ nr_node->which--;
+
+ nr_node->count--;
+ for (; i < nr_node->count; i++)
+ nr_node->routes[i] = nr_node->routes[i + 1];
+}
+
/*
* "Delete" a node. Strictly speaking remove a route to a node. The node
* is only deleted if no routes are left to it.
@@ -373,22 +391,10 @@ static int nr_del_node(ax25_address *callsign, ax25_address *neighbour, struct n
nr_neigh_route_put_locked(nr_neigh);
nr_neigh_put(nr_neigh);
- nr_node->count--;
+ nr_node_remove_route_locked(nr_node, i);
- if (nr_node->count == 0) {
+ if (nr_node->count == 0)
nr_remove_node_locked(nr_node);
- } else {
- switch (i) {
- case 0:
- nr_node->routes[0] = nr_node->routes[1];
- fallthrough;
- case 1:
- nr_node->routes[1] = nr_node->routes[2];
- fallthrough;
- case 2:
- break;
- }
- }
/* Drop the reference taken by __nr_node_get() */
nr_node_put(nr_node);
ret = 0;
@@ -472,34 +478,24 @@ static int nr_dec_obs(void)
spin_lock_bh(&nr_route_lock);
nr_node_for_each_safe(s, nodet, &nr_node_list) {
- for (i = 0; i < s->count; i++) {
+ /* i only moves on when route i stays: the next one moves up */
+ for (i = 0; i < s->count; ) {
switch (s->routes[i].obs_count) {
case 0: /* A locked entry */
+ i++;
break;
case 1: /* From 1 -> 0 */
nr_neigh = s->routes[i].neighbour;
nr_neigh_route_put_locked(nr_neigh);
-
- s->count--;
-
- switch (i) {
- case 0:
- s->routes[0] = s->routes[1];
- fallthrough;
- case 1:
- s->routes[1] = s->routes[2];
- break;
- case 2:
- break;
- }
+ nr_node_remove_route_locked(s, i);
break;
default:
s->routes[i].obs_count--;
+ i++;
break;
-
}
}
@@ -525,23 +521,14 @@ void nr_rt_device_down(struct net_device *dev)
nr_neigh_for_each_safe(s, nodet, &nr_neigh_list) {
if (s->dev == dev) {
nr_node_for_each_safe(t, node2t, &nr_node_list) {
- for (i = 0; i < t->count; i++) {
+ for (i = 0; i < t->count; ) {
if (t->routes[i].neighbour == s) {
s->count--;
nr_neigh_put(s);
- t->count--;
-
- switch (i) {
- case 0:
- t->routes[0] = t->routes[1];
- fallthrough;
- case 1:
- t->routes[1] = t->routes[2];
- break;
- case 2:
- break;
- }
+ nr_node_remove_route_locked(t, i);
+ } else {
+ i++;
}
}
@@ -774,11 +761,16 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25)
nr_dest = (ax25_address *)(skb->data + 7);
if (ax25 != NULL) {
- ret = nr_add_node(nr_src, "", &ax25->dest_addr, ax25->digipeat,
- ax25->ax25_dev->dev, 0,
- READ_ONCE(sysctl_netrom_obsolescence_count_initialiser));
- if (ret)
- return ret;
+ /*
+ * A frame that cannot be noted here (it claims to come from
+ * one of our own callsigns, or memory is short) is dropped.
+ * Return 0, not the error: AX.25 takes any other value as
+ * "frame consumed" and would never free it.
+ */
+ if (nr_add_node(nr_src, "", &ax25->dest_addr, ax25->digipeat,
+ ax25->ax25_dev->dev, 0,
+ READ_ONCE(sysctl_netrom_obsolescence_count_initialiser)))
+ return 0;
}
if ((dev = nr_dev_get(nr_dest)) != NULL) { /* Its for me */
@@ -793,10 +785,12 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25)
if (!READ_ONCE(sysctl_netrom_routing_control) && ax25 != NULL)
return 0;
- /* Its Time-To-Live has expired */
- if (skb->data[14] == 1) {
+ /*
+ * Its Time-To-Live has expired. A frame received with a TTL of 0
+ * has expired too: decremented below, it would leave with 255.
+ */
+ if (skb->data[14] <= 1)
return 0;
- }
nr_node = nr_node_get(nr_dest);
if (nr_node == NULL)
@@ -848,8 +842,14 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25)
spin_unlock_bh(&nr_route_lock);
nr_node_put(nr_node);
+ /*
+ * ax25_send_frame() only returns NULL before taking the frame:
+ * free our copy then; the caller frees the original.
+ */
if (ret)
kfree_skb(oskb);
+ else
+ kfree_skb(skb);
return ret;
}
diff --git a/net/rds/recv.c b/net/rds/recv.c
index bd9d00326e94..110fb22c3e2e 100644
--- a/net/rds/recv.c
+++ b/net/rds/recv.c
@@ -343,7 +343,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
rds_stats_inc(s_recv_ping);
rds_send_pong(cp, inc->i_hdr.h_sport);
/* if this is a handshake ping, start multipath if necessary */
- if (RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport),
+ if (conn->c_trans->t_mp_capable &&
+ RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport),
be16_to_cpu(inc->i_hdr.h_dport))) {
rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn);
rds_start_mprds(cp->cp_conn);
@@ -351,7 +352,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
goto out;
}
- if (be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT &&
+ if (conn->c_trans->t_mp_capable &&
+ be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT &&
inc->i_hdr.h_sport == 0) {
rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn);
/* if this is a handshake pong, start multipath if necessary */
diff --git a/net/rose/af_rose.c b/net/rose/af_rose.c
index d8a86006c744..332d26771681 100644
--- a/net/rose/af_rose.c
+++ b/net/rose/af_rose.c
@@ -278,7 +278,7 @@ static void rose_insert_socket(struct sock *sk)
/*
* Find a socket that wants to accept the Call Request we just
- * received.
+ * received. A reference is taken on the socket found.
*/
static struct sock *rose_find_listener(rose_address *addr, ax25_address *call)
{
@@ -304,48 +304,73 @@ static struct sock *rose_find_listener(rose_address *addr, ax25_address *call)
}
s = NULL;
found:
+ if (s)
+ sock_hold(s);
spin_unlock_bh(&rose_list_lock);
return s;
}
/*
* Find a connected ROSE socket given my LCI and device.
+ * The caller holds rose_list_lock.
*/
-struct sock *rose_find_socket(unsigned int lci, struct rose_neigh *neigh)
+static struct sock *__rose_find_socket(unsigned int lci, struct rose_neigh *neigh)
{
struct sock *s;
- spin_lock_bh(&rose_list_lock);
sk_for_each(s, &rose_list) {
struct rose_sock *rose = rose_sk(s);
if (rose->lci == lci && rose->neighbour == neigh)
- goto found;
+ return s;
}
- s = NULL;
-found:
+
+ return NULL;
+}
+
+/*
+ * Find a connected ROSE socket given my LCI and device. A reference is
+ * taken on the socket found: it can be released and destroyed as soon
+ * as rose_list_lock is dropped.
+ */
+struct sock *rose_find_socket(unsigned int lci, struct rose_neigh *neigh)
+{
+ struct sock *s;
+
+ spin_lock_bh(&rose_list_lock);
+ s = __rose_find_socket(lci, neigh);
+ if (s)
+ sock_hold(s);
spin_unlock_bh(&rose_list_lock);
return s;
}
/*
* Find a unique LCI for a given device.
+ * The caller holds rose_route_list_lock.
*/
unsigned int rose_new_lci(struct rose_neigh *neigh)
{
+ unsigned int res = 0;
int lci;
+ spin_lock_bh(&rose_list_lock);
if (neigh->dce_mode) {
for (lci = 1; lci <= sysctl_rose_maximum_vcs; lci++)
- if (rose_find_socket(lci, neigh) == NULL && rose_route_free_lci(lci, neigh) == NULL)
- return lci;
+ if (!__rose_find_socket(lci, neigh) && !rose_route_free_lci(lci, neigh)) {
+ res = lci;
+ break;
+ }
} else {
for (lci = sysctl_rose_maximum_vcs; lci > 0; lci--)
- if (rose_find_socket(lci, neigh) == NULL && rose_route_free_lci(lci, neigh) == NULL)
- return lci;
+ if (!__rose_find_socket(lci, neigh) && !rose_route_free_lci(lci, neigh)) {
+ res = lci;
+ break;
+ }
}
+ spin_unlock_bh(&rose_list_lock);
- return 0;
+ return res;
}
/*
@@ -673,6 +698,7 @@ static struct sock *rose_make_new(struct sock *osk)
static int rose_release(struct socket *sock)
{
struct sock *sk = sock->sk;
+ struct rose_neigh *neigh;
struct rose_sock *rose;
if (sk == NULL) return 0;
@@ -691,10 +717,21 @@ static int rose_release(struct socket *sock)
break;
case ROSE_STATE_2:
- rose_neigh_put(rose->neighbour);
- release_sock(sk);
+ /*
+ * Disconnect before the socket lock is dropped: frames put on
+ * the backlog meanwhile, a Clear Confirmation for one, then
+ * find it in state 0 instead of dropping the neighbour
+ * reference a second time. rose_kill_by_neigh() does not take
+ * the socket lock: take the neighbour under rose_list_lock, as
+ * it does, so that only one of them drops the reference.
+ */
rose_disconnect(sk, 0, -1, -1);
- lock_sock(sk);
+ spin_lock_bh(&rose_list_lock);
+ neigh = rose->neighbour;
+ rose->neighbour = NULL;
+ spin_unlock_bh(&rose_list_lock);
+ if (neigh)
+ rose_neigh_put(neigh);
rose_destroy_socket(sk);
break;
@@ -854,20 +891,6 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le
sk->sk_state = TCP_CLOSE;
sock->state = SS_UNCONNECTED;
- rose->neighbour = rose_get_neigh(&addr->srose_addr, &cause,
- &diagnostic, 0);
- if (!rose->neighbour) {
- err = -ENETUNREACH;
- goto out_release;
- }
-
- rose->lci = rose_new_lci(rose->neighbour);
- if (!rose->lci) {
- err = -ENETUNREACH;
- rose_neigh_put(rose->neighbour);
- goto out_release;
- }
-
if (sock_flag(sk, SOCK_ZAPPED)) { /* Must bind first - autobinding in this may or may not work */
struct net_device *dev;
@@ -876,14 +899,12 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le
dev = rose_dev_first();
if (!dev) {
err = -ENETUNREACH;
- rose_neigh_put(rose->neighbour);
goto out_release;
}
user = ax25_findbyuid(current_euid());
if (!user) {
err = -EINVAL;
- rose_neigh_put(rose->neighbour);
dev_put(dev);
goto out_release;
}
@@ -897,6 +918,21 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le
rose_insert_socket(sk); /* Finish the bind */
}
+
+ rose->neighbour = rose_get_neigh(&addr->srose_addr, &cause,
+ &diagnostic, 0);
+ if (!rose->neighbour) {
+ err = -ENETUNREACH;
+ goto out_release;
+ }
+
+ /* The socket is on the list now, so others see the LCI it gets */
+ if (!rose_assign_lci(sk)) {
+ err = -ENETUNREACH;
+ rose_neigh_put(rose->neighbour);
+ goto out_release;
+ }
+
rose->dest_addr = addr->srose_addr;
rose->dest_call = addr->srose_call;
rose->rand = ((long)rose & 0xFFFF) + rose->lci;
@@ -1103,12 +1139,31 @@ int rose_rx_call_request(struct sk_buff *skb, struct net_device *dev, struct ros
}
sk = rose_find_listener(&facilities.source_addr, &facilities.source_call);
+ if (!sk) {
+ rose_transmit_clear_request(neigh, lci, ROSE_NETWORK_CONGESTION, 120);
+ return 0;
+ }
+
+ /*
+ * The listener may be closed by rose_release() at any time. It is
+ * marked dead before rose_destroy_socket() empties its queue, so
+ * once it is locked a listener that is not dead will see the new
+ * socket on its queue, either in rose_accept() or when it is
+ * destroyed.
+ */
+ bh_lock_sock(sk);
/*
* We can't accept the Call Request.
*/
- if (sk == NULL || sk_acceptq_is_full(sk) ||
- (make = rose_make_new(sk)) == NULL) {
+ if (sock_flag(sk, SOCK_DEAD) || sk->sk_state != TCP_LISTEN ||
+ sk_acceptq_is_full(sk))
+ make = NULL;
+ else
+ make = rose_make_new(sk);
+ if (!make) {
+ bh_unlock_sock(sk);
+ sock_put(sk);
rose_transmit_clear_request(neigh, lci, ROSE_NETWORK_CONGESTION, 120);
return 0;
}
@@ -1164,8 +1219,9 @@ int rose_rx_call_request(struct sk_buff *skb, struct net_device *dev, struct ros
rose_start_heartbeat(make);
- if (!sock_flag(sk, SOCK_DEAD))
- sk->sk_data_ready(sk);
+ sk->sk_data_ready(sk);
+ bh_unlock_sock(sk);
+ sock_put(sk);
return 1;
}
diff --git a/net/rose/rose_in.c b/net/rose/rose_in.c
index 7a556254ee79..84025b4b498f 100644
--- a/net/rose/rose_in.c
+++ b/net/rose/rose_in.c
@@ -75,17 +75,24 @@ static int rose_state1_machine(struct sock *sk, struct sk_buff *skb, int framety
static int rose_state2_machine(struct sock *sk, struct sk_buff *skb, int frametype)
{
struct rose_sock *rose = rose_sk(sk);
+ /*
+ * A call whose Call Request timed out (T1) is cleared from state 2
+ * while connect() is still waiting for it: report the timeout, or
+ * connect() and SO_ERROR would find no error on a call that never
+ * got through.
+ */
+ int reason = sk->sk_state == TCP_SYN_SENT ? ETIMEDOUT : 0;
switch (frametype) {
case ROSE_CLEAR_REQUEST:
rose_write_internal(sk, ROSE_CLEAR_CONFIRMATION);
- rose_disconnect(sk, 0, skb->data[3], skb->data[4]);
+ rose_disconnect(sk, reason, skb->data[3], skb->data[4]);
rose_neigh_put(rose->neighbour);
rose->neighbour = NULL;
break;
case ROSE_CLEAR_CONFIRMATION:
- rose_disconnect(sk, 0, -1, -1);
+ rose_disconnect(sk, reason, -1, -1);
rose_neigh_put(rose->neighbour);
rose->neighbour = NULL;
break;
diff --git a/net/rose/rose_link.c b/net/rose/rose_link.c
index 9cfd4aa174f8..2e2837e63242 100644
--- a/net/rose/rose_link.c
+++ b/net/rose/rose_link.c
@@ -27,29 +27,66 @@
static void rose_ftimer_expiry(struct timer_list *);
static void rose_t0timer_expiry(struct timer_list *);
-static void rose_transmit_restart_confirmation(struct rose_neigh *neigh);
-static void rose_transmit_restart_request(struct rose_neigh *neigh);
+/*
+ * neigh->ax25 is replaced when a frame is sent or the link is looked up,
+ * from process context, softirq and the neighbour timers, and cleared by
+ * rose_link_failed(). Each holder of the pointer owns a reference on the
+ * ax25_cb: swap the pointer under this lock, then drop the reference that
+ * was taken off, so that it is dropped once and none is lost.
+ */
+static DEFINE_SPINLOCK(rose_neigh_ax25_lock);
-void rose_start_ftimer(struct rose_neigh *neigh)
+static void rose_neigh_set_ax25(struct rose_neigh *neigh, ax25_cb *ax25)
{
- del_timer(&neigh->ftimer);
+ ax25_cb *old;
- neigh->ftimer.function = rose_ftimer_expiry;
- neigh->ftimer.expires =
- jiffies + msecs_to_jiffies(sysctl_rose_link_fail_timeout);
+ spin_lock_bh(&rose_neigh_ax25_lock);
+ old = neigh->ax25;
+ neigh->ax25 = ax25;
+ spin_unlock_bh(&rose_neigh_ax25_lock);
- add_timer(&neigh->ftimer);
+ if (old)
+ ax25_cb_put(old);
}
-static void rose_start_t0timer(struct rose_neigh *neigh)
+/*
+ * Clear neigh->ax25 if it still is @ax25, and drop its reference.
+ */
+bool rose_neigh_clear_ax25(struct rose_neigh *neigh, ax25_cb *ax25)
{
- del_timer(&neigh->t0timer);
+ bool found;
- neigh->t0timer.function = rose_t0timer_expiry;
- neigh->t0timer.expires =
- jiffies + msecs_to_jiffies(sysctl_rose_restart_request_timeout);
+ spin_lock_bh(&rose_neigh_ax25_lock);
+ found = neigh->ax25 == ax25;
+ if (found)
+ neigh->ax25 = NULL;
+ spin_unlock_bh(&rose_neigh_ax25_lock);
+
+ if (found)
+ ax25_cb_put(ax25);
+
+ return found;
+}
- add_timer(&neigh->t0timer);
+static void rose_transmit_restart_confirmation(struct rose_neigh *neigh);
+static void rose_transmit_restart_request(struct rose_neigh *neigh);
+
+void rose_neigh_init_timers(struct rose_neigh *neigh)
+{
+ timer_setup(&neigh->ftimer, rose_ftimer_expiry, 0);
+ timer_setup(&neigh->t0timer, rose_t0timer_expiry, 0);
+}
+
+void rose_start_ftimer(struct rose_neigh *neigh)
+{
+ mod_timer(&neigh->ftimer,
+ jiffies + msecs_to_jiffies(sysctl_rose_link_fail_timeout));
+}
+
+static void rose_start_t0timer(struct rose_neigh *neigh)
+{
+ mod_timer(&neigh->t0timer,
+ jiffies + msecs_to_jiffies(sysctl_rose_restart_request_timeout));
}
void rose_stop_ftimer(struct rose_neigh *neigh)
@@ -95,19 +132,17 @@ static void rose_t0timer_expiry(struct timer_list *t)
static int rose_send_frame(struct sk_buff *skb, struct rose_neigh *neigh)
{
const ax25_address *rose_call;
- ax25_cb *ax25s;
+ ax25_cb *ax25;
if (ax25cmp(&rose_callsign, &null_ax25_address) == 0)
rose_call = (const ax25_address *)neigh->dev->dev_addr;
else
rose_call = &rose_callsign;
- ax25s = neigh->ax25;
- neigh->ax25 = ax25_send_frame(skb, 260, rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
- if (ax25s)
- ax25_cb_put(ax25s);
+ ax25 = ax25_send_frame(skb, 260, rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
+ rose_neigh_set_ax25(neigh, ax25);
- return neigh->ax25 != NULL;
+ return !!ax25;
}
/*
@@ -118,19 +153,17 @@ static int rose_send_frame(struct sk_buff *skb, struct rose_neigh *neigh)
static int rose_link_up(struct rose_neigh *neigh)
{
const ax25_address *rose_call;
- ax25_cb *ax25s;
+ ax25_cb *ax25;
if (ax25cmp(&rose_callsign, &null_ax25_address) == 0)
rose_call = (const ax25_address *)neigh->dev->dev_addr;
else
rose_call = &rose_callsign;
- ax25s = neigh->ax25;
- neigh->ax25 = ax25_find_cb(rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
- if (ax25s)
- ax25_cb_put(ax25s);
+ ax25 = ax25_find_cb(rose_call, &neigh->callsign, neigh->digipeat, neigh->dev);
+ rose_neigh_set_ax25(neigh, ax25);
- return neigh->ax25 != NULL;
+ return !!ax25;
}
/*
diff --git a/net/rose/rose_loopback.c b/net/rose/rose_loopback.c
index 4e4a473910cc..6c5c35030e67 100644
--- a/net/rose/rose_loopback.c
+++ b/net/rose/rose_loopback.c
@@ -109,6 +109,7 @@ static void rose_loopback_timer(struct timer_list *unused)
if (sk) {
if (rose_rcv_frame(sk, skb) == 0)
kfree_skb(skb);
+ sock_put(sk);
continue;
}
diff --git a/net/rose/rose_route.c b/net/rose/rose_route.c
index 6a1de0055e28..00fbe47fffe0 100644
--- a/net/rose/rose_route.c
+++ b/net/rose/rose_route.c
@@ -102,8 +102,7 @@ static int __must_check rose_add_node(struct rose_route_struct *rose_route,
skb_queue_head_init(&rose_neigh->queue);
- timer_setup(&rose_neigh->ftimer, NULL, 0);
- timer_setup(&rose_neigh->t0timer, NULL, 0);
+ rose_neigh_init_timers(rose_neigh);
if (rose_route->ndigis != 0) {
rose_neigh->digipeat =
@@ -446,8 +445,7 @@ void rose_add_loopback_neigh(void)
skb_queue_head_init(&sn->queue);
- timer_setup(&sn->ftimer, NULL, 0);
- timer_setup(&sn->t0timer, NULL, 0);
+ rose_neigh_init_timers(sn);
spin_lock_bh(&rose_neigh_list_lock);
sn->next = rose_neigh_list;
@@ -726,6 +724,8 @@ struct rose_route *rose_route_free_lci(unsigned int lci, struct rose_neigh *neig
{
struct rose_route *rose_route;
+ lockdep_assert_held(&rose_route_list_lock);
+
for (rose_route = rose_route_list; rose_route != NULL; rose_route = rose_route->next)
if ((rose_route->neigh1 == neigh && rose_route->lci1 == lci) ||
(rose_route->neigh2 == neigh && rose_route->lci2 == lci))
@@ -734,6 +734,23 @@ struct rose_route *rose_route_free_lci(unsigned int lci, struct rose_neigh *neig
return NULL;
}
+/*
+ * Give a socket placing a call a free LCI on its neighbour. As for a
+ * through route in rose_route_frame(), the search and the assignment
+ * are done under rose_route_list_lock, so two calls cannot pick the
+ * same LCI. The socket must already be on the socket list.
+ */
+unsigned int rose_assign_lci(struct sock *sk)
+{
+ struct rose_sock *rose = rose_sk(sk);
+
+ spin_lock_bh(&rose_route_list_lock);
+ rose->lci = rose_new_lci(rose->neighbour);
+ spin_unlock_bh(&rose_route_list_lock);
+
+ return rose->lci;
+}
+
/*
* Find a neighbour or a route given a ROSE address.
*/
@@ -919,14 +936,13 @@ void rose_link_failed(ax25_cb *ax25, int reason)
spin_lock_bh(&rose_neigh_list_lock);
rose_neigh = rose_neigh_list;
while (rose_neigh != NULL) {
- if (rose_neigh->ax25 == ax25)
+ if (READ_ONCE(rose_neigh->ax25) == ax25)
break;
rose_neigh = rose_neigh->next;
}
if (rose_neigh != NULL) {
- rose_neigh->ax25 = NULL;
- ax25_cb_put(ax25);
+ rose_neigh_clear_ax25(rose_neigh, ax25);
rose_del_route_by_neigh(rose_neigh);
rose_kill_by_neigh(rose_neigh);
@@ -1037,25 +1053,34 @@ int rose_route_frame(struct sk_buff *skb, ax25_cb *ax25)
if (frametype == ROSE_CALL_REQUEST) {
struct rose_sock *rose = rose_sk(sk);
- /* Remove an existing unused socket */
- rose_clear_queues(sk);
- rose->cause = ROSE_NETWORK_CONGESTION;
- rose->diagnostic = 0;
- rose_neigh_put(rose->neighbour);
- rose->neighbour = NULL;
- rose->lci = 0;
- rose->state = ROSE_STATE_0;
- sk->sk_state = TCP_CLOSE;
- sk->sk_err = 0;
- sk->sk_shutdown |= SEND_SHUTDOWN;
- if (!sock_flag(sk, SOCK_DEAD)) {
- sk->sk_state_change(sk);
- sock_set_flag(sk, SOCK_DEAD);
+ /*
+ * Remove an existing unused socket, unless it was
+ * disconnected or reaped since it was found.
+ */
+ bh_lock_sock(sk);
+ if (rose->lci == lci && rose->neighbour == rose_neigh) {
+ rose_clear_queues(sk);
+ rose->cause = ROSE_NETWORK_CONGESTION;
+ rose->diagnostic = 0;
+ rose_neigh_put(rose->neighbour);
+ rose->neighbour = NULL;
+ rose->lci = 0;
+ rose->state = ROSE_STATE_0;
+ sk->sk_state = TCP_CLOSE;
+ sk->sk_err = 0;
+ sk->sk_shutdown |= SEND_SHUTDOWN;
+ if (!sock_flag(sk, SOCK_DEAD)) {
+ sk->sk_state_change(sk);
+ sock_set_flag(sk, SOCK_DEAD);
+ }
}
+ bh_unlock_sock(sk);
+ sock_put(sk);
}
else {
skb_reset_transport_header(skb);
res = rose_rcv_frame(sk, skb);
+ sock_put(sk);
goto out;
}
}
diff --git a/net/rose/sysctl_net_rose.c b/net/rose/sysctl_net_rose.c
index d801315b7083..a82735c34871 100644
--- a/net/rose/sysctl_net_rose.c
+++ b/net/rose/sysctl_net_rose.c
@@ -9,13 +9,14 @@
#include <net/ax25.h>
#include <net/rose.h>
-static int min_timer[] = {1 * HZ};
-static int max_timer[] = {300 * HZ};
-static int min_idle[] = {0 * HZ};
-static int max_idle[] = {65535 * HZ};
+/* The timers are given in milliseconds */
+static int min_timer[] = {1 * MSEC_PER_SEC};
+static int max_timer[] = {300 * MSEC_PER_SEC};
+static int min_idle[] = {0 * MSEC_PER_SEC};
+static int max_idle[] = {65535 * MSEC_PER_SEC};
static int min_route[1], max_route[] = {1};
-static int min_ftimer[] = {60 * HZ};
-static int max_ftimer[] = {600 * HZ};
+static int min_ftimer[] = {60 * MSEC_PER_SEC};
+static int max_ftimer[] = {600 * MSEC_PER_SEC};
static int min_maxvcs[] = {1}, max_maxvcs[] = {254};
static int min_window[] = {1}, max_window[] = {7};
diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index c9b85848e825..9d68e58dace9 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -279,10 +279,9 @@ static void rxrpc_queue_packet(struct rxrpc_sock *rx, struct rxrpc_call *call,
static int rxrpc_send_data(struct rxrpc_sock *rx,
struct rxrpc_call *call,
struct msghdr *msg, size_t len,
- rxrpc_notify_end_tx_t notify_end_tx,
- bool *_dropped_lock)
+ rxrpc_notify_end_tx_t notify_end_tx)
+ __releases(&call->user_mutex)
{
- struct rxrpc_txbuf *txb;
struct sock *sk = &rx->sk;
enum rxrpc_call_state state;
long timeo;
@@ -293,23 +292,18 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
ret = rxrpc_wait_to_be_connected(call, &timeo);
if (ret < 0)
- return ret;
+ goto out_unlock;
if (call->conn->state == RXRPC_CONN_CLIENT_UNSECURED) {
ret = rxrpc_init_client_conn_security(call->conn);
if (ret < 0)
- return ret;
+ goto out_unlock;
}
/* this should be in poll */
sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk);
reload:
- txb = call->tx_pending;
- call->tx_pending = NULL;
- if (txb)
- rxrpc_see_txbuf(txb, rxrpc_txbuf_see_send_more);
-
ret = -EPIPE;
if (sk->sk_shutdown & SEND_SHUTDOWN)
goto maybe_error;
@@ -337,6 +331,8 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
}
do {
+ struct rxrpc_txbuf *txb = call->tx_pending;
+
if (!txb) {
size_t remain;
@@ -355,6 +351,9 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
ret = -ENOMEM;
goto maybe_error;
}
+ call->tx_pending = txb;
+ } else {
+ rxrpc_see_txbuf(txb, rxrpc_txbuf_see_send_more);
}
_debug("append");
@@ -392,12 +391,12 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
ret = call->security->secure_packet(call, txb);
if (ret < 0)
- goto out;
+ goto out_unlock;
txb->kvec[0].iov_len += txb->len;
txb->len = txb->kvec[0].iov_len;
rxrpc_queue_packet(rx, call, txb, notify_end_tx);
- txb = NULL;
+ call->tx_pending = NULL;
}
} while (msg_data_left(msg) > 0);
@@ -406,45 +405,46 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
if (rxrpc_call_is_complete(call) &&
call->error < 0)
ret = call->error;
-out:
- call->tx_pending = txb;
+out_unlock:
+ mutex_unlock(&call->user_mutex);
_leave(" = %d", ret);
return ret;
call_terminated:
- rxrpc_put_txbuf(txb, rxrpc_txbuf_put_send_aborted);
- _leave(" = %d", call->error);
- return call->error;
+ ret = call->error;
+ goto out_unlock;
maybe_error:
if (copied)
goto success;
- goto out;
+ goto out_unlock;
efault:
ret = -EFAULT;
- goto out;
+ goto out_unlock;
wait_for_space:
ret = -EAGAIN;
if (msg->msg_flags & MSG_DONTWAIT)
goto maybe_error;
mutex_unlock(&call->user_mutex);
- *_dropped_lock = true;
+
ret = rxrpc_wait_for_tx_window(rx, call, &timeo,
msg->msg_flags & MSG_WAITALL);
if (ret < 0)
- goto maybe_error;
+ goto out_nolock;
if (call->interruptibility == RXRPC_INTERRUPTIBLE) {
if (mutex_lock_interruptible(&call->user_mutex) < 0) {
ret = sock_intr_errno(timeo);
- goto maybe_error;
+ goto out_nolock;
}
} else {
mutex_lock(&call->user_mutex);
}
- *_dropped_lock = false;
goto reload;
+out_nolock:
+ _leave(" = %d [intr]", ret);
+ return copied ?: ret;
}
/*
@@ -611,7 +611,6 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
__releases(&rx->sk.sk_lock.slock)
{
struct rxrpc_call *call;
- bool dropped_lock = false;
int ret;
struct rxrpc_send_params p = {
@@ -717,12 +716,12 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
} else if (p.command != RXRPC_CMD_SEND_DATA) {
ret = -EINVAL;
} else {
- ret = rxrpc_send_data(rx, call, msg, len, NULL, &dropped_lock);
+ ret = rxrpc_send_data(rx, call, msg, len, NULL);
+ goto error_put;
}
out_put_unlock:
- if (!dropped_lock)
- mutex_unlock(&call->user_mutex);
+ mutex_unlock(&call->user_mutex);
error_put:
rxrpc_put_call(call, rxrpc_call_put_sendmsg);
_leave(" = %d", ret);
@@ -752,7 +751,6 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call,
struct msghdr *msg, size_t len,
rxrpc_notify_end_tx_t notify_end_tx)
{
- bool dropped_lock = false;
int ret;
_enter("{%d},", call->debug_id);
@@ -763,12 +761,9 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call,
mutex_lock(&call->user_mutex);
ret = rxrpc_send_data(rxrpc_sk(sock->sk), call, msg, len,
- notify_end_tx, &dropped_lock);
+ notify_end_tx);
if (ret == -ESHUTDOWN)
ret = call->error;
-
- if (!dropped_lock)
- mutex_unlock(&call->user_mutex);
_leave(" = %d", ret);
return ret;
}
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index b8619b306fca..e5b65ea313c8 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -70,9 +70,34 @@ static void qdisc_maybe_clear_missed(struct Qdisc *q,
#define SKB_XOFF_MAGIC ((struct sk_buff *)1UL)
+/* Re-cap a stale queue_mapping; an ONETXQUEUE qdisc pins q->dev_queue. */
+static u16 qdisc_cap_skb_tx_queue(struct Qdisc *q, struct sk_buff *skb)
+{
+ u16 queue = skb_get_queue_mapping(skb);
+ struct net_device *dev = qdisc_dev(q);
+ u16 capped;
+
+ if (q->flags & TCQ_F_ONETXQUEUE)
+ capped = q->dev_queue - netdev_get_tx_queue(dev, 0);
+ else
+ capped = netdev_cap_txqueue(dev, queue);
+
+ if (unlikely(capped != queue))
+ skb_set_queue_mapping(skb, capped);
+
+ return capped;
+}
+
+static struct netdev_queue *skb_cap_tx_queue(struct Qdisc *q,
+ struct sk_buff *skb)
+{
+ return netdev_get_tx_queue(qdisc_dev(q),
+ qdisc_cap_skb_tx_queue(q, skb));
+}
+
static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
{
- const struct netdev_queue *txq = q->dev_queue;
+ const struct netdev_queue *txq;
spinlock_t *lock = NULL;
struct sk_buff *skb;
@@ -84,7 +109,8 @@ static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
skb = skb_peek(&q->skb_bad_txq);
if (skb) {
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ /* re-cap: real_num_tx_queues may have shrunk */
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->skb_bad_txq);
if (qdisc_is_percpu_stats(q)) {
@@ -198,6 +224,8 @@ static void try_bulk_dequeue_skb(struct Qdisc *q,
if (!nskb)
break;
+ /* Every element of an ONETXQUEUE qdisc is pinned to q->dev_queue. */
+ qdisc_cap_skb_tx_queue(q, nskb);
bytelimit -= nskb->len; /* covers GSO len */
skb->next = nskb;
skb = nskb;
@@ -215,7 +243,6 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
struct sk_buff *skb,
int *packets)
{
- int mapping = skb_get_queue_mapping(skb);
struct sk_buff *nskb;
int cnt = 0;
@@ -223,7 +250,9 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
nskb = q->dequeue(q);
if (!nskb)
break;
- if (unlikely(skb_get_queue_mapping(nskb) != mapping)) {
+ /* A tc BPF store may have changed the mapping; cap it. */
+ if (unlikely(qdisc_cap_skb_tx_queue(q, nskb) !=
+ skb_get_queue_mapping(skb))) {
qdisc_enqueue_skb_bad_txq(q, nskb);
break;
}
@@ -268,7 +297,7 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
if (xfrm_offload(skb))
*validate = true;
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->gso_skb);
if (qdisc_is_percpu_stats(q)) {
@@ -304,6 +333,8 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
skb = q->dequeue(q);
if (skb) {
bulk:
+ /* cap the head; the bulk helpers cap every appended element */
+ qdisc_cap_skb_tx_queue(q, skb);
if (qdisc_may_bulk(q))
try_bulk_dequeue_skb(q, skb, txq, packets, budget);
else
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 70a08f195010..b0a9966cc6ab 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -520,11 +520,14 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
sch_tree_lock(sch);
qfq_init_agg(q, new_agg, lmax, weight);
}
+ if (existing && new_agg == cl->agg)
+ goto unlock;
if (existing)
qfq_deact_rm_from_agg(q, cl);
else
qdisc_class_hash_insert(&q->clhash, &cl->common);
qfq_add_to_agg(q, new_agg, cl);
+unlock:
sch_tree_unlock(sch);
qdisc_class_hash_grow(sch, &q->clhash);
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 5091ffc3f475..54d0136dfe6c 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -631,6 +631,11 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
int queue;
queue = skb_get_queue_mapping(skb);
+ if (unlikely(queue >= qdisc_dev(sch)->real_num_tx_queues)) {
+ /* A tc BPF program may have stored an out-of-range mapping. */
+ queue = netdev_cap_txqueue(qdisc_dev(sch), queue);
+ skb_set_queue_mapping(skb, queue);
+ }
child = q->qdiscs[queue];
if (unlikely(child == &noop_qdisc))
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index 1bbc041c0644..6dae9831613a 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -95,7 +95,7 @@ static void sctp_control_release_owner(struct sk_buff *skb)
}
}
-static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
+void sctp_control_set_owner_w(struct sctp_chunk *chunk)
{
struct sctp_association *asoc = chunk->asoc;
struct sk_buff *skb = chunk->skb;
@@ -108,7 +108,8 @@ static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
* For now don't do anything for now.
*/
if (chunk->auth) {
- chunk->shkey = asoc->shkey;
+ if (!chunk->shkey)
+ chunk->shkey = asoc->shkey;
sctp_auth_shkey_hold(chunk->shkey);
}
skb->sk = asoc ? asoc->base.sk : NULL;
@@ -145,6 +146,7 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
size_t paylen)
{
struct sctp_errhdr err;
+ int chunklen, padlen;
__u16 len;
/* Cause code constants are now defined in network order. */
@@ -152,7 +154,13 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
len = sizeof(err) + paylen;
err.length = htons(len);
- if (skb_tailroom(chunk->skb) < len)
+ /* sctp_addto_chunk() pads the chunk up to a 4-byte boundary before
+ * appending the cause, so account for that padding here as well.
+ */
+ chunklen = ntohs(chunk->chunk_hdr->length);
+ padlen = SCTP_PAD4(chunklen) - chunklen;
+
+ if (skb_tailroom(chunk->skb) < padlen + len)
return -ENOSPC;
chunk->subh.err_hdr = sctp_addto_chunk(chunk, sizeof(err), &err);
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index e2885a994314..7734ca92335a 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -149,25 +149,47 @@ static inline void sctp_set_owner_w(struct sctp_chunk *chunk)
static void sctp_clear_owner_w(struct sctp_chunk *chunk)
{
+ /* Keep the shkey alive until the new owner takes its reference. */
+ if (chunk->shkey)
+ sctp_auth_shkey_hold(chunk->shkey);
skb_orphan(chunk->skb);
}
+static void sctp_set_owner_w_migrate(struct sctp_chunk *chunk)
+{
+ struct sctp_shared_key *shkey = chunk->shkey;
+
+ if (chunk->msg)
+ sctp_set_owner_w(chunk);
+ else
+ sctp_control_set_owner_w(chunk);
+
+ if (shkey)
+ sctp_auth_shkey_release(shkey);
+}
+
+static void sctp_process_tx_chunk(struct sctp_association *asoc,
+ struct sctp_chunk *chunk, bool clear,
+ void (*cb)(struct sctp_chunk *))
+{
+ if ((clear && asoc->base.sk == chunk->skb->sk) ||
+ (!clear && asoc->base.sk != chunk->skb->sk))
+ cb(chunk);
+}
+
#define traverse_and_process() \
do { \
msg = chunk->msg; \
if (msg == prev_msg) \
continue; \
- list_for_each_entry(c, &msg->chunks, frag_list) { \
- if ((clear && asoc->base.sk == c->skb->sk) || \
- (!clear && asoc->base.sk != c->skb->sk)) \
- cb(c); \
- } \
+ list_for_each_entry(c, &msg->chunks, frag_list) \
+ sctp_process_tx_chunk(asoc, c, clear, cb); \
prev_msg = msg; \
} while (0)
-static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
- bool clear,
- void (*cb)(struct sctp_chunk *))
+static void sctp_for_each_tx_chunk(struct sctp_association *asoc,
+ bool clear,
+ void (*cb)(struct sctp_chunk *))
{
struct sctp_datamsg *msg, *prev_msg = NULL;
@@ -190,6 +212,21 @@ static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
list_for_each_entry(chunk, &q->out_chunk_list, list)
traverse_and_process();
+
+ list_for_each_entry(chunk, &q->control_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->asconf_ack_list, transmitted_list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->addip_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ if (asoc->strreset_chunk)
+ sctp_process_tx_chunk(asoc, asoc->strreset_chunk, clear, cb);
+
+ if (asoc->addip_last_asconf)
+ sctp_process_tx_chunk(asoc, asoc->addip_last_asconf, clear, cb);
}
static void sctp_for_each_rx_skb(struct sctp_association *asoc, struct sock *sk,
@@ -9705,9 +9742,9 @@ static int sctp_sock_migrate(struct sock *oldsk, struct sock *newsk,
* paths won't try to lock it and then oldsk.
*/
lock_sock_nested(newsk, SINGLE_DEPTH_NESTING);
- sctp_for_each_tx_datachunk(assoc, true, sctp_clear_owner_w);
+ sctp_for_each_tx_chunk(assoc, true, sctp_clear_owner_w);
sctp_assoc_migrate(assoc, newsk);
- sctp_for_each_tx_datachunk(assoc, false, sctp_set_owner_w);
+ sctp_for_each_tx_chunk(assoc, false, sctp_set_owner_w_migrate);
/* If the association on the newsk is already closed before accept()
* is called, set RCV_SHUTDOWN flag.
diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c
index 56dc3175f047..947ac2e1b16b 100644
--- a/net/smc/smc_rx.c
+++ b/net/smc/smc_rx.c
@@ -286,6 +286,27 @@ int smc_rx_wait(struct smc_sock *smc, long *timeo, size_t peeked,
return rc;
}
+/* Wait for the pipe to release the spliced data. Unlike smc_rx_wait()
+ * it does not end on shutdown or sk_err, the data is already in the
+ * RMB; smc_conn_kill() wakes it and the caller checks conn->killed.
+ */
+static void smc_rx_wait_splice(struct smc_sock *smc, long *timeo)
+{
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
+ struct smc_connection *conn = &smc->conn;
+ struct sock *sk = &smc->sk;
+
+ sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+ add_wait_queue(sk_sleep(sk), &wait);
+ /* pairs with the smp_mb() in wq_has_sleeper(): the release drops
+ * splice_pending, then looks for a sleeper
+ */
+ smp_mb();
+ sk_wait_event(sk, timeo, !atomic_read(&conn->splice_pending), &wait);
+ remove_wait_queue(sk_sleep(sk), &wait);
+ sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+}
+
static int smc_rx_recv_urg(struct smc_sock *smc, struct msghdr *msg, int len,
int flags)
{
@@ -443,7 +464,23 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg,
/* could be just 1 byte, even after waiting on data above */
readable = smc_rx_data_available(conn, peeked_bytes);
splbytes = atomic_read(&conn->splice_pending);
- if (!readable || (msg && splbytes)) {
+ if (msg && splbytes) {
+ /* a pipe holds bytes ahead of these; the copy path
+ * skips the timeout and signal checks
+ */
+ if (!read_remaining)
+ break;
+ if (!timeo || signal_pending(current)) {
+ if (!read_done)
+ read_done = timeo ?
+ sock_intr_errno(timeo) :
+ -EAGAIN;
+ break;
+ }
+ smc_rx_wait_splice(smc, &timeo);
+ continue;
+ }
+ if (!readable) {
if (splbytes)
func = smc_rx_data_available_and_no_splice_pend;
else
diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c
index 2d74c660de66..9b90350820be 100644
--- a/net/sunrpc/rpcb_clnt.c
+++ b/net/sunrpc/rpcb_clnt.c
@@ -400,7 +400,8 @@ static struct rpc_clnt *rpcb_create(struct net *net, const char *nodename,
return rpc_create(&args);
}
-static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, struct rpc_message *msg, bool is_set)
+static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt,
+ struct rpc_message *msg, bool is_set)
{
int flags = RPC_TASK_NOCONNECT;
int error, result = 0;
@@ -410,8 +411,23 @@ static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, stru
msg->rpc_resp = &result;
error = rpc_call_sync(clnt, msg, flags);
- if (error < 0)
- return error;
+ if (error < 0) {
+ switch (error) {
+ /* rpcbind answered; the reply itself carries the error */
+ case -EPROTONOSUPPORT:
+ case -EPFNOSUPPORT:
+ case -EOPNOTSUPP:
+ case -EACCES:
+ /* the call never made it onto the wire */
+ case -ENOMEM:
+ case -EMSGSIZE:
+ /* the caller is going away; this says nothing about rpcbind */
+ case -ERESTARTSYS:
+ return error;
+ }
+ /* anything else, we assume that rpcbind isn't functional */
+ return -EIO;
+ }
if (!result)
return -EACCES;
diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index 4b4a2890a874..6d43f479342d 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -1204,10 +1204,40 @@ int svc_generic_rpcbind_set(struct net *net,
error = svc_rpcbind_set_version(net, progp, version,
family, proto, port);
+ /* -EIO means no answer, not a refusal, so vs_rpcb_optnl must keep it. */
+ if (error == -EIO)
+ return error;
+
return (vers->vs_rpcb_optnl) ? 0 : error;
}
EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
+/**
+ * svc_rpcb_failure_count - local rpcbind calls for @serv that got no answer
+ * @serv: RPC service to query
+ *
+ * svc_register() adds one for each of its calls that got no answer. A reply
+ * that refuses one entry does not count, because rpcbind answered and the
+ * next entry may still succeed.
+ *
+ * The count is kept per serv rather than per net. The local rpcbind client
+ * is per-net and lockd shares it, but a count that another service can move
+ * says nothing about this serv's own calls.
+ *
+ * This is for callers that cannot see the svc_register() return, because a
+ * transport class sits in between. Such a caller reads the count before it
+ * starts and compares as it goes, so there is no state to reset between
+ * operations. The count never resets, and callers must not attach meaning
+ * to the value itself.
+ *
+ * Return: the number of unanswered calls since this serv was created.
+ */
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv)
+{
+ return atomic_read(&serv->sv_rpcb_failures);
+}
+EXPORT_SYMBOL_GPL(svc_rpcb_failure_count);
+
/**
* svc_register - register an RPC service with the local portmapper
* @serv: svc_serv struct for the service to register
@@ -1218,10 +1248,11 @@ EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
*
* Service is registered for any address in the passed-in protocol family
*/
-int svc_register(const struct svc_serv *serv, struct net *net,
+int svc_register(struct svc_serv *serv, struct net *net,
const int family, const unsigned short proto,
const unsigned short port)
{
+ bool noanswer = false;
unsigned int p, i;
int error = 0;
@@ -1233,10 +1264,16 @@ int svc_register(const struct svc_serv *serv, struct net *net,
struct svc_program *progp = &serv->sv_programs[p];
for (i = 0; i < progp->pg_nvers; i++) {
+ const struct svc_version *vers = progp->pg_vers[i];
int ret;
ret = progp->pg_rpcbind_set(net, progp, i,
family, proto, port);
+ if (ret == -EIO) {
+ noanswer = true;
+ if (vers && vers->vs_rpcb_optnl)
+ ret = 0;
+ }
if (ret < 0) {
printk(KERN_WARNING "svc: failed to register "
"%sv%u RPC service (errno %d).\n",
@@ -1248,6 +1285,9 @@ int svc_register(const struct svc_serv *serv, struct net *net,
}
}
+ if (noanswer)
+ atomic_inc(&serv->sv_rpcb_failures);
+
return error;
}
diff --git a/net/wireless/reg.c b/net/wireless/reg.c
index daf51fb56648..04bb6d7f955c 100644
--- a/net/wireless/reg.c
+++ b/net/wireless/reg.c
@@ -691,6 +691,9 @@ static bool valid_rule(const u8 *data, unsigned int size, u16 rule_ptr)
/* mandatory fields */
if (rule->len < offsetofend(struct fwdb_rule, max_bw))
return false;
+ /* the rule must lie entirely within the database */
+ if ((u8 *)rule + rule->len > data + size)
+ return false;
if (rule->len >= offsetofend(struct fwdb_rule, wmm_ptr)) {
u32 wmm_ptr = be16_to_cpu(rule->wmm_ptr) << 2;
struct fwdb_wmm_rule *wmm;
diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c
index c4291ea86d1d..67254678180e 100644
--- a/net/xfrm/espintcp.c
+++ b/net/xfrm/espintcp.c
@@ -437,7 +437,9 @@ static void espintcp_destruct(struct sock *sk)
bool tcp_is_ulp_esp(struct sock *sk)
{
- return sk->sk_prot == &espintcp_prot || sk->sk_prot == &espintcp6_prot;
+ const struct proto *prot = READ_ONCE(sk->sk_prot);
+
+ return prot == &espintcp_prot || prot == &espintcp6_prot;
}
EXPORT_SYMBOL_GPL(tcp_is_ulp_esp);
@@ -469,34 +471,41 @@ static int espintcp_init_sk(struct sock *sk)
__sk_dst_reset(sk);
- strp_check_rcv(&ctx->strp);
skb_queue_head_init(&ctx->ike_queue);
skb_queue_head_init(&ctx->out_queue);
+ INIT_WORK(&ctx->work, espintcp_tx_work);
+
+ ctx->saved_data_ready = sk->sk_data_ready;
+ ctx->saved_write_space = sk->sk_write_space;
+ ctx->saved_destruct = sk->sk_destruct;
+
+ /* avoid using task_frag */
+ sk->sk_allocation = GFP_ATOMIC;
+ sk->sk_use_task_frag = false;
+
+ rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
+
+ /* pairs with smp_rmb() in espintcp_getctx() */
+ smp_wmb();
+
+ WRITE_ONCE(sk->sk_data_ready, espintcp_data_ready);
+ WRITE_ONCE(sk->sk_write_space, espintcp_write_space);
+ sk->sk_destruct = espintcp_destruct;
if (sk->sk_family == AF_INET) {
- sk->sk_prot = &espintcp_prot;
- sk->sk_socket->ops = &espintcp_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp_ops);
} else {
mutex_lock(&tcpv6_prot_mutex);
if (!espintcp6_prot.recvmsg)
build_protos(&espintcp6_prot, &espintcp6_ops, sk->sk_prot, sk->sk_socket->ops);
mutex_unlock(&tcpv6_prot_mutex);
- sk->sk_prot = &espintcp6_prot;
- sk->sk_socket->ops = &espintcp6_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp6_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp6_ops);
}
- ctx->saved_data_ready = sk->sk_data_ready;
- ctx->saved_write_space = sk->sk_write_space;
- ctx->saved_destruct = sk->sk_destruct;
- sk->sk_data_ready = espintcp_data_ready;
- sk->sk_write_space = espintcp_write_space;
- sk->sk_destruct = espintcp_destruct;
- rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
- INIT_WORK(&ctx->work, espintcp_tx_work);
- /* avoid using task_frag */
- sk->sk_allocation = GFP_ATOMIC;
- sk->sk_use_task_frag = false;
+ strp_check_rcv(&ctx->strp);
return 0;
diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c
index be0000448af2..2ff96577ff72 100644
--- a/net/xfrm/xfrm_interface_core.c
+++ b/net/xfrm/xfrm_interface_core.c
@@ -532,8 +532,13 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
memset(&fl, 0, sizeof(fl));
+ if (!pskb_inet_may_pull(skb))
+ goto tx_err;
+
switch (skb->protocol) {
case htons(ETH_P_IPV6):
+ if (ipv6_hdr(skb)->version != 6)
+ goto tx_err;
memset(IP6CB(skb), 0, sizeof(*IP6CB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET6);
if (!dst) {
@@ -549,6 +554,9 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
}
break;
case htons(ETH_P_IP):
+ if (ip_hdr(skb)->version != 4 || ip_hdr(skb)->ihl < 5 ||
+ !pskb_network_may_pull(skb, ip_hdr(skb)->ihl * 4))
+ goto tx_err;
memset(IPCB(skb), 0, sizeof(*IPCB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET);
if (!dst) {
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index e033390e7785..106e2f5cc586 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -2157,6 +2157,7 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
struct xfrm_pol_inexact_bin *bin;
struct xfrm_policy *pol, *ret;
struct hlist_head *chain;
+ unsigned int inexact_sequence;
unsigned int sequence;
int err;
@@ -2191,12 +2192,18 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
goto skip_inexact;
bin = xfrm_policy_inexact_lookup_rcu(net, type, family, dir, if_id);
- if (!bin || !xfrm_policy_find_inexact_candidates(&cand, bin, saddr,
- daddr))
+ if (!bin)
+ goto skip_inexact;
+
+ inexact_sequence = read_seqcount_begin(&bin->count);
+ if (!xfrm_policy_find_inexact_candidates(&cand, bin, saddr, daddr))
goto skip_inexact;
pol = xfrm_policy_eval_candidates(&cand, ret, fl, type,
family, if_id);
+ if (read_seqcount_retry(&bin->count, inexact_sequence))
+ goto retry;
+
if (pol) {
ret = pol;
if (IS_ERR(pol))
diff --git a/security/apparmor/policy_unpack.c b/security/apparmor/policy_unpack.c
index 46420587bcd5..61d7ef232f32 100644
--- a/security/apparmor/policy_unpack.c
+++ b/security/apparmor/policy_unpack.c
@@ -1202,13 +1202,22 @@ static int verify_header(struct aa_ext *e, int required, const char **ns)
* verify_dfa_accept_index - verify accept indexes are in range of perms table
* @dfa: the dfa to check accept indexes are in range
* @table_size: the permission table size the indexes should be within
+ * @pair: the accept indexes reference two slots, index and index + 1
*/
-static bool verify_dfa_accept_index(struct aa_dfa *dfa, int table_size)
+static bool verify_dfa_accept_index(struct aa_dfa *dfa, int table_size,
+ bool pair)
{
int i;
for (i = 0; i < dfa->tables[YYTD_ID_ACCEPT]->td_lolen; i++) {
if (ACCEPT_TABLE(dfa)[i] >= table_size)
return false;
+ /*
+ * Accept indexes for owner-conditional permissions come
+ * in pairs, so the non-owner entry at index + 1 must also
+ * be in bounds.
+ */
+ if (pair && ACCEPT_TABLE(dfa)[i] + 1 >= table_size)
+ return false;
}
return true;
}
@@ -1272,15 +1281,17 @@ static int verify_profile(struct aa_profile *profile)
if (!rules)
return 0;
- if (rules->file->dfa && !verify_dfa_accept_index(rules->file->dfa,
- rules->file->size)) {
+ if (rules->file->dfa &&
+ !verify_dfa_accept_index(rules->file->dfa, rules->file->size,
+ rules->file != rules->policy)) {
audit_iface(profile, NULL, NULL,
"Unpack: file Invalid named transition", NULL,
-EPROTO);
return -EPROTO;
}
if (rules->policy->dfa &&
- !verify_dfa_accept_index(rules->policy->dfa, rules->policy->size)) {
+ !verify_dfa_accept_index(rules->policy->dfa, rules->policy->size,
+ false)) {
audit_iface(profile, NULL, NULL,
"Unpack: policy Invalid named transition", NULL,
-EPROTO);
diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c
index ae8ab99f53f8..ec0d4f34404c 100644
--- a/sound/core/compress_offload.c
+++ b/sound/core/compress_offload.c
@@ -1014,17 +1014,17 @@ static long snd_compr_ioctl_compat(struct file *file, unsigned int cmd,
#endif
static const struct file_operations snd_compr_file_ops = {
- .owner = THIS_MODULE,
- .open = snd_compr_open,
- .release = snd_compr_free,
- .write = snd_compr_write,
- .read = snd_compr_read,
- .unlocked_ioctl = snd_compr_ioctl,
+ .owner = THIS_MODULE,
+ .open = snd_compr_open,
+ .release = snd_compr_free,
+ .write = snd_compr_write,
+ .read = snd_compr_read,
+ .unlocked_ioctl = snd_compr_ioctl,
#ifdef CONFIG_COMPAT
- .compat_ioctl = snd_compr_ioctl_compat,
+ .compat_ioctl = snd_compr_ioctl_compat,
#endif
- .mmap = snd_compr_mmap,
- .poll = snd_compr_poll,
+ .mmap = snd_compr_mmap,
+ .poll = snd_compr_poll,
};
static int snd_compress_dev_register(struct snd_device *device)
diff --git a/sound/core/control.c b/sound/core/control.c
index 460d8dd8303f..d8f071218c0e 100644
--- a/sound/core/control.c
+++ b/sound/core/control.c
@@ -2269,16 +2269,15 @@ EXPORT_SYMBOL_GPL(snd_ctl_disconnect_layer);
* INIT PART
*/
-static const struct file_operations snd_ctl_f_ops =
-{
- .owner = THIS_MODULE,
- .read = snd_ctl_read,
- .open = snd_ctl_open,
- .release = snd_ctl_release,
- .poll = snd_ctl_poll,
- .unlocked_ioctl = snd_ctl_ioctl,
- .compat_ioctl = snd_ctl_ioctl_compat,
- .fasync = snd_ctl_fasync,
+static const struct file_operations snd_ctl_f_ops = {
+ .owner = THIS_MODULE,
+ .read = snd_ctl_read,
+ .open = snd_ctl_open,
+ .release = snd_ctl_release,
+ .poll = snd_ctl_poll,
+ .unlocked_ioctl = snd_ctl_ioctl,
+ .compat_ioctl = snd_ctl_ioctl_compat,
+ .fasync = snd_ctl_fasync,
};
/* call lops under rwsems; called from snd_ctl_dev_*() below() */
diff --git a/sound/core/hwdep.c b/sound/core/hwdep.c
index 09200df2932c..4334162bc0b2 100644
--- a/sound/core/hwdep.c
+++ b/sound/core/hwdep.c
@@ -212,6 +212,7 @@ static int snd_hwdep_dsp_load(struct snd_hwdep *hw,
return -ENXIO;
if (info->index >= 32)
return -EINVAL;
+ guard(mutex)(&hw->dsp_mutex);
/* check whether the dsp was already loaded */
if (hw->dsp_loaded & (1u << info->index))
return -EBUSY;
@@ -323,18 +324,17 @@ static int snd_hwdep_control_ioctl(struct snd_card *card,
*/
-static const struct file_operations snd_hwdep_f_ops =
-{
- .owner = THIS_MODULE,
- .llseek = snd_hwdep_llseek,
- .read = snd_hwdep_read,
- .write = snd_hwdep_write,
- .open = snd_hwdep_open,
- .release = snd_hwdep_release,
- .poll = snd_hwdep_poll,
- .unlocked_ioctl = snd_hwdep_ioctl,
- .compat_ioctl = snd_hwdep_ioctl_compat,
- .mmap = snd_hwdep_mmap,
+static const struct file_operations snd_hwdep_f_ops = {
+ .owner = THIS_MODULE,
+ .llseek = snd_hwdep_llseek,
+ .read = snd_hwdep_read,
+ .write = snd_hwdep_write,
+ .open = snd_hwdep_open,
+ .release = snd_hwdep_release,
+ .poll = snd_hwdep_poll,
+ .unlocked_ioctl = snd_hwdep_ioctl,
+ .compat_ioctl = snd_hwdep_ioctl_compat,
+ .mmap = snd_hwdep_mmap,
};
static void snd_hwdep_free(struct snd_hwdep *hwdep)
@@ -381,6 +381,7 @@ int snd_hwdep_new(struct snd_card *card, char *id, int device,
init_waitqueue_head(&hwdep->open_wait);
mutex_init(&hwdep->open_mutex);
+ mutex_init(&hwdep->dsp_mutex);
hwdep->card = card;
hwdep->device = device;
if (id)
diff --git a/sound/core/init.c b/sound/core/init.c
index 92dff1736cfe..b37269b86be8 100644
--- a/sound/core/init.c
+++ b/sound/core/init.c
@@ -461,20 +461,19 @@ static int snd_disconnect_fasync(int fd, struct file *file, int on)
return -ENODEV;
}
-static const struct file_operations snd_shutdown_f_ops =
-{
- .owner = THIS_MODULE,
- .llseek = snd_disconnect_llseek,
- .read = snd_disconnect_read,
- .write = snd_disconnect_write,
- .release = snd_disconnect_release,
- .poll = snd_disconnect_poll,
- .unlocked_ioctl = snd_disconnect_ioctl,
+static const struct file_operations snd_shutdown_f_ops = {
+ .owner = THIS_MODULE,
+ .llseek = snd_disconnect_llseek,
+ .read = snd_disconnect_read,
+ .write = snd_disconnect_write,
+ .release = snd_disconnect_release,
+ .poll = snd_disconnect_poll,
+ .unlocked_ioctl = snd_disconnect_ioctl,
#ifdef CONFIG_COMPAT
- .compat_ioctl = snd_disconnect_ioctl,
+ .compat_ioctl = snd_disconnect_ioctl,
#endif
- .mmap = snd_disconnect_mmap,
- .fasync = snd_disconnect_fasync
+ .mmap = snd_disconnect_mmap,
+ .fasync = snd_disconnect_fasync
};
/**
diff --git a/sound/core/jack.c b/sound/core/jack.c
index e4bcecdf89b7..c21f041bf457 100644
--- a/sound/core/jack.c
+++ b/sound/core/jack.c
@@ -304,41 +304,41 @@ static ssize_t jack_type_read(struct file *file,
}
static const struct file_operations jack_type_fops = {
- .open = simple_open,
- .read = jack_type_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_type_read,
+ .llseek = default_llseek,
};
#endif
static const struct file_operations sw_inject_enable_fops = {
- .open = simple_open,
- .read = sw_inject_enable_read,
- .write = sw_inject_enable_write,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = sw_inject_enable_read,
+ .write = sw_inject_enable_write,
+ .llseek = default_llseek,
};
static const struct file_operations jackin_inject_fops = {
- .open = simple_open,
- .write = jackin_inject_write,
- .llseek = default_llseek,
+ .open = simple_open,
+ .write = jackin_inject_write,
+ .llseek = default_llseek,
};
static const struct file_operations jack_kctl_id_fops = {
- .open = simple_open,
- .read = jack_kctl_id_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_kctl_id_read,
+ .llseek = default_llseek,
};
static const struct file_operations jack_kctl_mask_bits_fops = {
- .open = simple_open,
- .read = jack_kctl_mask_bits_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_kctl_mask_bits_read,
+ .llseek = default_llseek,
};
static const struct file_operations jack_kctl_status_fops = {
- .open = simple_open,
- .read = jack_kctl_status_read,
- .llseek = default_llseek,
+ .open = simple_open,
+ .read = jack_kctl_status_read,
+ .llseek = default_llseek,
};
static int snd_jack_debugfs_add_inject_node(struct snd_jack *jack,
diff --git a/sound/core/oss/mixer_oss.c b/sound/core/oss/mixer_oss.c
index a6cc74bb25fd..34e4dc791119 100644
--- a/sound/core/oss/mixer_oss.c
+++ b/sound/core/oss/mixer_oss.c
@@ -407,13 +407,12 @@ static long snd_mixer_oss_ioctl_compat(struct file *file, unsigned int cmd,
* REGISTRATION PART
*/
-static const struct file_operations snd_mixer_oss_f_ops =
-{
- .owner = THIS_MODULE,
- .open = snd_mixer_oss_open,
- .release = snd_mixer_oss_release,
- .unlocked_ioctl = snd_mixer_oss_ioctl,
- .compat_ioctl = snd_mixer_oss_ioctl_compat,
+static const struct file_operations snd_mixer_oss_f_ops = {
+ .owner = THIS_MODULE,
+ .open = snd_mixer_oss_open,
+ .release = snd_mixer_oss_release,
+ .unlocked_ioctl = snd_mixer_oss_ioctl,
+ .compat_ioctl = snd_mixer_oss_ioctl_compat,
};
/*
diff --git a/sound/core/oss/pcm_oss.c b/sound/core/oss/pcm_oss.c
index a65a3b8d04b8..c49d71dd4280 100644
--- a/sound/core/oss/pcm_oss.c
+++ b/sound/core/oss/pcm_oss.c
@@ -3121,17 +3121,16 @@ static inline void snd_pcm_oss_proc_done(struct snd_pcm *pcm)
* ENTRY functions
*/
-static const struct file_operations snd_pcm_oss_f_reg =
-{
- .owner = THIS_MODULE,
- .read = snd_pcm_oss_read,
- .write = snd_pcm_oss_write,
- .open = snd_pcm_oss_open,
- .release = snd_pcm_oss_release,
- .poll = snd_pcm_oss_poll,
- .unlocked_ioctl = snd_pcm_oss_ioctl,
- .compat_ioctl = snd_pcm_oss_ioctl_compat,
- .mmap = snd_pcm_oss_mmap,
+static const struct file_operations snd_pcm_oss_f_reg = {
+ .owner = THIS_MODULE,
+ .read = snd_pcm_oss_read,
+ .write = snd_pcm_oss_write,
+ .open = snd_pcm_oss_open,
+ .release = snd_pcm_oss_release,
+ .poll = snd_pcm_oss_poll,
+ .unlocked_ioctl = snd_pcm_oss_ioctl,
+ .compat_ioctl = snd_pcm_oss_ioctl_compat,
+ .mmap = snd_pcm_oss_mmap,
};
static void register_oss_dsp(struct snd_pcm *pcm, int index)
diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c
index 45b34e34412c..dd135ed91c99 100644
--- a/sound/core/pcm_native.c
+++ b/sound/core/pcm_native.c
@@ -4182,29 +4182,29 @@ static unsigned long snd_pcm_get_unmapped_area(struct file *file,
const struct file_operations snd_pcm_f_ops[2] = {
{
- .owner = THIS_MODULE,
- .write = snd_pcm_write,
- .write_iter = snd_pcm_writev,
- .open = snd_pcm_playback_open,
- .release = snd_pcm_release,
- .poll = snd_pcm_poll,
- .unlocked_ioctl = snd_pcm_ioctl,
- .compat_ioctl = snd_pcm_ioctl_compat,
- .mmap = snd_pcm_mmap,
- .fasync = snd_pcm_fasync,
- .get_unmapped_area = snd_pcm_get_unmapped_area,
+ .owner = THIS_MODULE,
+ .write = snd_pcm_write,
+ .write_iter = snd_pcm_writev,
+ .open = snd_pcm_playback_open,
+ .release = snd_pcm_release,
+ .poll = snd_pcm_poll,
+ .unlocked_ioctl = snd_pcm_ioctl,
+ .compat_ioctl = snd_pcm_ioctl_compat,
+ .mmap = snd_pcm_mmap,
+ .fasync = snd_pcm_fasync,
+ .get_unmapped_area = snd_pcm_get_unmapped_area,
},
{
- .owner = THIS_MODULE,
- .read = snd_pcm_read,
- .read_iter = snd_pcm_readv,
- .open = snd_pcm_capture_open,
- .release = snd_pcm_release,
- .poll = snd_pcm_poll,
- .unlocked_ioctl = snd_pcm_ioctl,
- .compat_ioctl = snd_pcm_ioctl_compat,
- .mmap = snd_pcm_mmap,
- .fasync = snd_pcm_fasync,
- .get_unmapped_area = snd_pcm_get_unmapped_area,
+ .owner = THIS_MODULE,
+ .read = snd_pcm_read,
+ .read_iter = snd_pcm_readv,
+ .open = snd_pcm_capture_open,
+ .release = snd_pcm_release,
+ .poll = snd_pcm_poll,
+ .unlocked_ioctl = snd_pcm_ioctl,
+ .compat_ioctl = snd_pcm_ioctl_compat,
+ .mmap = snd_pcm_mmap,
+ .fasync = snd_pcm_fasync,
+ .get_unmapped_area = snd_pcm_get_unmapped_area,
}
};
diff --git a/sound/core/rawmidi.c b/sound/core/rawmidi.c
index c7b39e07e7c3..1eb234250c58 100644
--- a/sound/core/rawmidi.c
+++ b/sound/core/rawmidi.c
@@ -1784,14 +1784,14 @@ static void snd_rawmidi_proc_info_read(struct snd_info_entry *entry,
*/
static const struct file_operations snd_rawmidi_f_ops = {
- .owner = THIS_MODULE,
- .read = snd_rawmidi_read,
- .write = snd_rawmidi_write,
- .open = snd_rawmidi_open,
- .release = snd_rawmidi_release,
- .poll = snd_rawmidi_poll,
- .unlocked_ioctl = snd_rawmidi_ioctl,
- .compat_ioctl = snd_rawmidi_ioctl_compat,
+ .owner = THIS_MODULE,
+ .read = snd_rawmidi_read,
+ .write = snd_rawmidi_write,
+ .open = snd_rawmidi_open,
+ .release = snd_rawmidi_release,
+ .poll = snd_rawmidi_poll,
+ .unlocked_ioctl = snd_rawmidi_ioctl,
+ .compat_ioctl = snd_rawmidi_ioctl_compat,
};
static int snd_rawmidi_alloc_substreams(struct snd_rawmidi *rmidi,
diff --git a/sound/core/seq/oss/seq_oss.c b/sound/core/seq/oss/seq_oss.c
index 77c1214acd90..9de191f59c67 100644
--- a/sound/core/seq/oss/seq_oss.c
+++ b/sound/core/seq/oss/seq_oss.c
@@ -211,17 +211,16 @@ odev_poll(struct file *file, poll_table * wait)
* registration of sequencer minor device
*/
-static const struct file_operations seq_oss_f_ops =
-{
- .owner = THIS_MODULE,
- .read = odev_read,
- .write = odev_write,
- .open = odev_open,
- .release = odev_release,
- .poll = odev_poll,
- .unlocked_ioctl = odev_ioctl,
- .compat_ioctl = odev_ioctl_compat,
- .llseek = noop_llseek,
+static const struct file_operations seq_oss_f_ops = {
+ .owner = THIS_MODULE,
+ .read = odev_read,
+ .write = odev_write,
+ .open = odev_open,
+ .release = odev_release,
+ .poll = odev_poll,
+ .unlocked_ioctl = odev_ioctl,
+ .compat_ioctl = odev_ioctl_compat,
+ .llseek = noop_llseek,
};
static int __init
diff --git a/sound/core/seq/seq_clientmgr.c b/sound/core/seq/seq_clientmgr.c
index 86316cf027ae..4dc5b15029e4 100644
--- a/sound/core/seq/seq_clientmgr.c
+++ b/sound/core/seq/seq_clientmgr.c
@@ -2785,16 +2785,15 @@ void snd_seq_info_clients_read(struct snd_info_entry *entry,
* REGISTRATION PART
*/
-static const struct file_operations snd_seq_f_ops =
-{
- .owner = THIS_MODULE,
- .read = snd_seq_read,
- .write = snd_seq_write,
- .open = snd_seq_open,
- .release = snd_seq_release,
- .poll = snd_seq_poll,
- .unlocked_ioctl = snd_seq_ioctl,
- .compat_ioctl = snd_seq_ioctl_compat,
+static const struct file_operations snd_seq_f_ops = {
+ .owner = THIS_MODULE,
+ .read = snd_seq_read,
+ .write = snd_seq_write,
+ .open = snd_seq_open,
+ .release = snd_seq_release,
+ .poll = snd_seq_poll,
+ .unlocked_ioctl = snd_seq_ioctl,
+ .compat_ioctl = snd_seq_ioctl_compat,
};
static struct device *seq_dev;
diff --git a/sound/core/sound.c b/sound/core/sound.c
index 7980b60f4ba0..5ce042db4755 100644
--- a/sound/core/sound.c
+++ b/sound/core/sound.c
@@ -167,11 +167,10 @@ static int snd_open(struct inode *inode, struct file *file)
return err;
}
-static const struct file_operations snd_fops =
-{
- .owner = THIS_MODULE,
- .open = snd_open,
- .llseek = noop_llseek,
+static const struct file_operations snd_fops = {
+ .owner = THIS_MODULE,
+ .open = snd_open,
+ .llseek = noop_llseek,
};
#ifdef CONFIG_SND_DYNAMIC_MINORS
diff --git a/sound/core/timer.c b/sound/core/timer.c
index a548885e8178..8f84055d3e53 100644
--- a/sound/core/timer.c
+++ b/sound/core/timer.c
@@ -2101,9 +2101,10 @@ static long snd_utimer_ioctl(struct file *file, unsigned int ioctl, unsigned lon
}
static const struct file_operations snd_utimer_fops = {
- .llseek = noop_llseek,
- .release = snd_utimer_release,
- .unlocked_ioctl = snd_utimer_ioctl,
+ .llseek = noop_llseek,
+ .release = snd_utimer_release,
+ .unlocked_ioctl = snd_utimer_ioctl,
+ .compat_ioctl = snd_utimer_ioctl,
};
static int snd_utimer_start(struct snd_timer *t)
@@ -2455,16 +2456,15 @@ static __poll_t snd_timer_user_poll(struct file *file, poll_table * wait)
#define snd_timer_user_ioctl_compat NULL
#endif
-static const struct file_operations snd_timer_f_ops =
-{
- .owner = THIS_MODULE,
- .read = snd_timer_user_read,
- .open = snd_timer_user_open,
- .release = snd_timer_user_release,
- .poll = snd_timer_user_poll,
- .unlocked_ioctl = snd_timer_user_ioctl,
- .compat_ioctl = snd_timer_user_ioctl_compat,
- .fasync = snd_timer_user_fasync,
+static const struct file_operations snd_timer_f_ops = {
+ .owner = THIS_MODULE,
+ .read = snd_timer_user_read,
+ .open = snd_timer_user_open,
+ .release = snd_timer_user_release,
+ .poll = snd_timer_user_poll,
+ .unlocked_ioctl = snd_timer_user_ioctl,
+ .compat_ioctl = snd_timer_user_ioctl_compat,
+ .fasync = snd_timer_user_fasync,
};
/* unregister the system timer */
diff --git a/sound/core/timer_compat.c b/sound/core/timer_compat.c
index 4ae9eaeb5afb..7b648edcb9cb 100644
--- a/sound/core/timer_compat.c
+++ b/sound/core/timer_compat.c
@@ -98,6 +98,7 @@ static long __snd_timer_user_ioctl_compat(struct file *file, unsigned int cmd,
case SNDRV_TIMER_IOCTL_PAUSE:
case SNDRV_TIMER_IOCTL_PAUSE_OLD:
case SNDRV_TIMER_IOCTL_NEXT_DEVICE:
+ case SNDRV_TIMER_IOCTL_CREATE:
return __snd_timer_user_ioctl(file, cmd, (unsigned long)argp, true);
case SNDRV_TIMER_IOCTL_GPARAMS32:
return snd_timer_user_gparams_compat(file, argp);
diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c
index 0a7972d91ec5..ecbd894f4c83 100644
--- a/sound/core/ump_convert.c
+++ b/sound/core/ump_convert.c
@@ -69,7 +69,7 @@ static u32 upscale_14_to_32bit(u16 src)
static int cvt_ump_system_to_legacy(u32 data, unsigned char *buf)
{
buf[0] = ump_message_status_channel(data);
- switch (ump_message_status_code(data)) {
+ switch (buf[0]) {
case UMP_SYSTEM_STATUS_MIDI_TIME_CODE:
case UMP_SYSTEM_STATUS_SONG_SELECT:
buf[1] = (data >> 8) & 0x7f;
diff --git a/sound/drivers/aloop.c b/sound/drivers/aloop.c
index 50b8e7b3d817..94ea73b3869c 100644
--- a/sound/drivers/aloop.c
+++ b/sound/drivers/aloop.c
@@ -532,13 +532,12 @@ static int loopback_prepare(struct snd_pcm_substream *substream)
dpcm->pcm_salign = salign;
dpcm->pcm_period_size = frames_to_bytes(runtime, runtime->period_size);
- mutex_lock(&dpcm->loopback->cable_lock);
+ guard(mutex)(&dpcm->loopback->cable_lock);
if (!(cable->valid & ~(1 << substream->stream)) ||
(get_setup(dpcm)->notify &&
substream->stream == SNDRV_PCM_STREAM_PLAYBACK))
params_change(substream);
cable->valid |= 1 << substream->stream;
- mutex_unlock(&dpcm->loopback->cable_lock);
return 0;
}
@@ -979,9 +978,8 @@ static int loopback_hw_free(struct snd_pcm_substream *substream)
struct loopback_pcm *dpcm = runtime->private_data;
struct loopback_cable *cable = dpcm->cable;
- mutex_lock(&dpcm->loopback->cable_lock);
+ guard(mutex)(&dpcm->loopback->cable_lock);
cable->valid &= ~(1 << substream->stream);
- mutex_unlock(&dpcm->loopback->cable_lock);
return 0;
}
@@ -993,6 +991,18 @@ static unsigned int get_cable_index(struct snd_pcm_substream *substream)
return !substream->stream;
}
+/*
+ * In notify mode the playback side may change format, rate, channels and,
+ * with a sound timer, the period size. A running capture that no longer
+ * matches is stopped by loopback_check_format().
+ * call in loopback->cable_lock
+ */
+static bool loopback_playback_unconstrained(struct loopback_pcm *dpcm)
+{
+ return dpcm->substream->stream == SNDRV_PCM_STREAM_PLAYBACK &&
+ get_notify(dpcm);
+}
+
static int rule_format(struct snd_pcm_hw_params *params,
struct snd_pcm_hw_rule *rule)
{
@@ -1001,10 +1011,12 @@ static int rule_format(struct snd_pcm_hw_params *params,
struct snd_mask m;
snd_mask_none(&m);
- mutex_lock(&dpcm->loopback->cable_lock);
- m.bits[0] = (u_int32_t)cable->hw.formats;
- m.bits[1] = (u_int32_t)(cable->hw.formats >> 32);
- mutex_unlock(&dpcm->loopback->cable_lock);
+ scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
+ m.bits[0] = (u_int32_t)cable->hw.formats;
+ m.bits[1] = (u_int32_t)(cable->hw.formats >> 32);
+ }
return snd_mask_refine(hw_param_mask(params, rule->var), &m);
}
@@ -1015,10 +1027,12 @@ static int rule_rate(struct snd_pcm_hw_params *params,
struct loopback_cable *cable = dpcm->cable;
struct snd_interval t;
- mutex_lock(&dpcm->loopback->cable_lock);
- t.min = cable->hw.rate_min;
- t.max = cable->hw.rate_max;
- mutex_unlock(&dpcm->loopback->cable_lock);
+ scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
+ t.min = cable->hw.rate_min;
+ t.max = cable->hw.rate_max;
+ }
t.openmin = t.openmax = 0;
t.integer = 0;
return snd_interval_refine(hw_param_interval(params, rule->var), &t);
@@ -1031,10 +1045,12 @@ static int rule_channels(struct snd_pcm_hw_params *params,
struct loopback_cable *cable = dpcm->cable;
struct snd_interval t;
- mutex_lock(&dpcm->loopback->cable_lock);
- t.min = cable->hw.channels_min;
- t.max = cable->hw.channels_max;
- mutex_unlock(&dpcm->loopback->cable_lock);
+ scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
+ t.min = cable->hw.channels_min;
+ t.max = cable->hw.channels_max;
+ }
t.openmin = t.openmax = 0;
t.integer = 0;
return snd_interval_refine(hw_param_interval(params, rule->var), &t);
@@ -1047,10 +1063,12 @@ static int rule_period_bytes(struct snd_pcm_hw_params *params,
struct loopback_cable *cable = dpcm->cable;
struct snd_interval t;
- mutex_lock(&dpcm->loopback->cable_lock);
- t.min = cable->hw.period_bytes_min;
- t.max = cable->hw.period_bytes_max;
- mutex_unlock(&dpcm->loopback->cable_lock);
+ scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
+ t.min = cable->hw.period_bytes_min;
+ t.max = cable->hw.period_bytes_max;
+ }
t.openmin = 0;
t.openmax = 0;
t.integer = 0;
@@ -1264,12 +1282,10 @@ static int loopback_open(struct snd_pcm_substream *substream)
int err = 0;
int dev = get_cable_index(substream);
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
dpcm = kzalloc(sizeof(*dpcm), GFP_KERNEL);
- if (!dpcm) {
- err = -ENOMEM;
- goto unlock;
- }
+ if (!dpcm)
+ return -ENOMEM;
dpcm->loopback = loopback;
dpcm->substream = substream;
@@ -1354,7 +1370,6 @@ static int loopback_open(struct snd_pcm_substream *substream)
free_cable(substream);
kfree(dpcm);
}
- mutex_unlock(&loopback->cable_lock);
return err;
}
@@ -1366,9 +1381,8 @@ static int loopback_close(struct snd_pcm_substream *substream)
if (dpcm->cable->ops->close_substream)
err = dpcm->cable->ops->close_substream(dpcm);
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
free_cable(substream);
- mutex_unlock(&loopback->cable_lock);
return err;
}
@@ -1419,11 +1433,10 @@ static int loopback_rate_shift_get(struct snd_kcontrol *kcontrol,
{
struct loopback *loopback = snd_kcontrol_chip(kcontrol);
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
ucontrol->value.integer.value[0] =
loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].rate_shift;
- mutex_unlock(&loopback->cable_lock);
return 0;
}
@@ -1439,14 +1452,13 @@ static int loopback_rate_shift_put(struct snd_kcontrol *kcontrol,
val = 80000;
if (val > 120000)
val = 120000;
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
if (val != loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].rate_shift) {
loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].rate_shift = val;
change = 1;
}
- mutex_unlock(&loopback->cable_lock);
return change;
}
@@ -1455,11 +1467,10 @@ static int loopback_notify_get(struct snd_kcontrol *kcontrol,
{
struct loopback *loopback = snd_kcontrol_chip(kcontrol);
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
ucontrol->value.integer.value[0] =
loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].notify;
- mutex_unlock(&loopback->cable_lock);
return 0;
}
@@ -1471,14 +1482,13 @@ static int loopback_notify_put(struct snd_kcontrol *kcontrol,
int change = 0;
val = ucontrol->value.integer.value[0] ? 1 : 0;
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
if (val != loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].notify) {
loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].notify = val;
change = 1;
}
- mutex_unlock(&loopback->cable_lock);
return change;
}
@@ -1490,14 +1500,13 @@ static int loopback_active_get(struct snd_kcontrol *kcontrol,
unsigned int val = 0;
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
cable = loopback->cables[kcontrol->id.subdevice][kcontrol->id.device ^ 1];
if (cable != NULL) {
unsigned int running = cable->running ^ cable->pause;
val = (running & (1 << SNDRV_PCM_STREAM_PLAYBACK)) ? 1 : 0;
}
- mutex_unlock(&loopback->cable_lock);
ucontrol->value.integer.value[0] = val;
return 0;
}
@@ -1540,11 +1549,10 @@ static int loopback_rate_get(struct snd_kcontrol *kcontrol,
{
struct loopback *loopback = snd_kcontrol_chip(kcontrol);
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
ucontrol->value.integer.value[0] =
loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].rate;
- mutex_unlock(&loopback->cable_lock);
return 0;
}
@@ -1564,11 +1572,10 @@ static int loopback_channels_get(struct snd_kcontrol *kcontrol,
{
struct loopback *loopback = snd_kcontrol_chip(kcontrol);
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
ucontrol->value.integer.value[0] =
loopback->setup[kcontrol->id.subdevice]
[kcontrol->id.device].channels;
- mutex_unlock(&loopback->cable_lock);
return 0;
}
@@ -1586,12 +1593,11 @@ static int loopback_access_get(struct snd_kcontrol *kcontrol,
struct loopback *loopback = snd_kcontrol_chip(kcontrol);
snd_pcm_access_t access;
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
access = loopback->setup[kcontrol->id.subdevice][kcontrol->id.device].access;
ucontrol->value.enumerated.item[0] = !is_access_interleaved(access);
- mutex_unlock(&loopback->cable_lock);
return 0;
}
@@ -1759,12 +1765,11 @@ static void print_cable_info(struct snd_info_entry *entry,
struct loopback *loopback = entry->private_data;
int sub, num;
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
num = entry->name[strlen(entry->name)-1];
num = num == '0' ? 0 : 1;
for (sub = 0; sub < MAX_PCM_SUBSTREAMS; sub++)
print_substream_info(buffer, loopback, sub, num);
- mutex_unlock(&loopback->cable_lock);
}
static int loopback_cable_proc_new(struct loopback *loopback, int cidx)
@@ -1793,10 +1798,9 @@ static void print_timer_source_info(struct snd_info_entry *entry,
{
struct loopback *loopback = entry->private_data;
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
snd_iprintf(buffer, "%s\n",
loopback->timer_source ? loopback->timer_source : "");
- mutex_unlock(&loopback->cable_lock);
}
static void change_timer_source_info(struct snd_info_entry *entry,
@@ -1805,10 +1809,9 @@ static void change_timer_source_info(struct snd_info_entry *entry,
struct loopback *loopback = entry->private_data;
char line[64];
- mutex_lock(&loopback->cable_lock);
+ guard(mutex)(&loopback->cable_lock);
if (!snd_info_get_line(buffer, line, sizeof(line)))
loopback_set_timer_source(loopback, strim(line));
- mutex_unlock(&loopback->cable_lock);
}
static int loopback_timer_source_proc_new(struct loopback *loopback)
diff --git a/sound/drivers/mpu401/mpu401_uart.c b/sound/drivers/mpu401/mpu401_uart.c
index 8e3318e17717..86235aefab02 100644
--- a/sound/drivers/mpu401/mpu401_uart.c
+++ b/sound/drivers/mpu401/mpu401_uart.c
@@ -162,14 +162,15 @@ static void snd_mpu401_uart_timer(struct timer_list *t)
spin_lock_irqsave(&mpu->timer_lock, flags);
/*mpu->mode |= MPU401_MODE_TIMER;*/
- mod_timer(&mpu->timer, 1 + jiffies);
+ if (mpu->timer_invoked)
+ mod_timer(&mpu->timer, 1 + jiffies);
spin_unlock_irqrestore(&mpu->timer_lock, flags);
if (mpu->rmidi)
_snd_mpu401_uart_interrupt(mpu);
}
/*
- * initialize the timer callback if not programmed yet
+ * arm the timer if it is not already active
*/
static void snd_mpu401_uart_add_timer (struct snd_mpu401 *mpu, int input)
{
@@ -177,7 +178,6 @@ static void snd_mpu401_uart_add_timer (struct snd_mpu401 *mpu, int input)
spin_lock_irqsave (&mpu->timer_lock, flags);
if (mpu->timer_invoked == 0) {
- timer_setup(&mpu->timer, snd_mpu401_uart_timer, 0);
mod_timer(&mpu->timer, 1 + jiffies);
}
mpu->timer_invoked |= input ? MPU401_MODE_INPUT_TIMER :
@@ -493,6 +493,7 @@ static void snd_mpu401_uart_free(struct snd_rawmidi *rmidi)
struct snd_mpu401 *mpu = rmidi->private_data;
if (mpu->irq >= 0)
free_irq(mpu->irq, (void *) mpu);
+ timer_shutdown_sync(&mpu->timer);
release_and_free_resource(mpu->res);
kfree(mpu);
}
@@ -547,6 +548,7 @@ int snd_mpu401_uart_new(struct snd_card *card, int device,
spin_lock_init(&mpu->input_lock);
spin_lock_init(&mpu->output_lock);
spin_lock_init(&mpu->timer_lock);
+ timer_setup(&mpu->timer, snd_mpu401_uart_timer, 0);
mpu->hardware = hardware;
mpu->irq = -1;
mpu->rmidi = rmidi;
diff --git a/sound/pci/cs46xx/dsp_spos.c b/sound/pci/cs46xx/dsp_spos.c
index dd23b7d68c16..dd37c185bcf0 100644
--- a/sound/pci/cs46xx/dsp_spos.c
+++ b/sound/pci/cs46xx/dsp_spos.c
@@ -1110,6 +1110,8 @@ int cs46xx_dsp_scb_and_task_init (struct snd_cs46xx *chip)
null_scb.entry_point = null_algorithm->address;
ins->the_null_scb = cs46xx_dsp_create_scb(chip, "nullSCB", (u32 *)&null_scb, NULL_SCB_ADDR);
+ if (!ins->the_null_scb)
+ return -ENOMEM;
ins->the_null_scb->task_entry = null_algorithm;
ins->the_null_scb->sub_list_ptr = ins->the_null_scb;
ins->the_null_scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/pci/cs46xx/dsp_spos_scb_lib.c b/sound/pci/cs46xx/dsp_spos_scb_lib.c
index 1f90ca723f4d..af5c78e3fd20 100644
--- a/sound/pci/cs46xx/dsp_spos_scb_lib.c
+++ b/sound/pci/cs46xx/dsp_spos_scb_lib.c
@@ -285,7 +285,8 @@ _dsp_create_generic_scb (struct snd_cs46xx *chip, char * name, u32 * scb_data, u
dev_dbg(chip->card->dev, "dsp_spos: creating SCB <%s>\n", name);
scb = cs46xx_dsp_create_scb(chip,name,scb_data,dest);
-
+ if (!scb)
+ return NULL;
scb->sub_list_ptr = ins->the_null_scb;
scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/pci/hda/cs35l41_hda.c b/sound/pci/hda/cs35l41_hda.c
index 031978a62cb9..210cf48facec 100644
--- a/sound/pci/hda/cs35l41_hda.c
+++ b/sound/pci/hda/cs35l41_hda.c
@@ -1887,7 +1887,6 @@ int cs35l41_hda_parse_acpi(struct cs35l41_hda *cs35l41, struct device *physdev,
hw_cfg->valid = false;
hw_cfg->gpio1.valid = false;
hw_cfg->gpio2.valid = false;
- acpi_dev_put(cs35l41->dacpi);
return ret;
}
@@ -1895,7 +1894,6 @@ int cs35l41_hda_parse_acpi(struct cs35l41_hda *cs35l41, struct device *physdev,
static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, int id)
{
struct acpi_device *adev;
- struct device *physdev;
struct spi_device *spi;
const char *sub;
int ret;
@@ -1907,11 +1905,10 @@ static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, i
}
cs35l41->dacpi = adev;
- physdev = get_device(acpi_get_first_physical_node(adev));
- if (!physdev) {
- acpi_dev_put(adev);
+ struct device *physdev __free(put_device) =
+ get_device(acpi_get_first_physical_node(adev));
+ if (!physdev)
return -ENODEV;
- }
sub = acpi_get_subsystem_id(ACPI_HANDLE(physdev));
if (IS_ERR(sub))
@@ -1925,13 +1922,9 @@ static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, i
}
ret = cs35l41_hda_parse_acpi(cs35l41, physdev, id);
- if (ret) {
- put_device(physdev);
+ if (ret)
return ret;
- }
out:
- put_device(physdev);
-
cs35l41->bypass_fw = false;
if (cs35l41->control_bus == SPI) {
spi = to_spi_device(cs35l41->dev);
@@ -1970,8 +1963,10 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i
dev_set_drvdata(dev, cs35l41);
ret = cs35l41_hda_read_acpi(cs35l41, device_name, id);
- if (ret)
- return dev_err_probe(cs35l41->dev, ret, "Platform not supported\n");
+ if (ret) {
+ dev_err_probe(cs35l41->dev, ret, "Platform not supported\n");
+ goto err_acpi;
+ }
if (IS_ERR(cs35l41->reset_gpio)) {
ret = PTR_ERR(cs35l41->reset_gpio);
@@ -2059,6 +2054,7 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i
err:
if (cs35l41_safe_reset(cs35l41->regmap, cs35l41->hw_cfg.bst_type))
gpiod_set_value_cansleep(cs35l41->reset_gpio, 0);
+err_acpi:
gpiod_put(cs35l41->reset_gpio);
gpiod_put(cs35l41->cs_gpio);
acpi_dev_put(cs35l41->dacpi);
diff --git a/sound/pci/hda/hda_eld.c b/sound/pci/hda/hda_eld.c
index 301730432375..2980d0b949f6 100644
--- a/sound/pci/hda/hda_eld.c
+++ b/sound/pci/hda/hda_eld.c
@@ -525,9 +525,11 @@ void snd_hdmi_write_eld_info(struct hdmi_eld *eld,
e->aud_synch_delay = val;
else if (!strcmp(name, "speakers"))
e->spk_alloc = val;
- else if (!strcmp(name, "sad_count"))
+ else if (!strcmp(name, "sad_count")) {
+ if (val > ELD_MAX_SAD)
+ continue;
e->sad_count = val;
- else if (!strncmp(name, "sad", 3)) {
+ } else if (!strncmp(name, "sad", 3)) {
sname = name + 4;
n = name[3] - '0';
if (name[4] >= '0' && name[4] <= '9') {
diff --git a/sound/pci/hda/patch_hdmi.c b/sound/pci/hda/patch_hdmi.c
index 8b68386e1ac9..c097fc844b18 100644
--- a/sound/pci/hda/patch_hdmi.c
+++ b/sound/pci/hda/patch_hdmi.c
@@ -1355,7 +1355,8 @@ static int hdmi_read_pin_conn(struct hda_codec *codec, int pin_idx)
snd_hda_set_dev_select(codec, pin_nid, dev_id);
if (spec->intel_hsw_fixup) {
- conns = spec->num_cvts;
+ /* spec->cvt_nids[] only has ARRAY_SIZE(spec->cvt_nids) slots */
+ conns = min_t(int, spec->num_cvts, ARRAY_SIZE(spec->cvt_nids));
memcpy(per_pin->mux_nids, spec->cvt_nids,
sizeof(hda_nid_t) * conns);
} else {
diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c
index 0b370a9eb9f2..1a68e683801b 100644
--- a/sound/pci/hda/patch_realtek.c
+++ b/sound/pci/hda/patch_realtek.c
@@ -8212,6 +8212,7 @@ enum {
ALC287_FIXUP_LEGION_16ACHG6,
ALC287_FIXUP_CS35L41_I2C_2,
ALC287_FIXUP_CS35L41_I2C_2_HP_GPIO_LED,
+ ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS,
ALC287_FIXUP_CS35L41_I2C_4,
ALC245_FIXUP_CS35L41_SPI_1,
ALC245_FIXUP_CS35L41_SPI_2,
@@ -10414,6 +10415,12 @@ static const struct hda_fixup alc269_fixups[] = {
.chained = true,
.chain_id = ALC285_FIXUP_HP_MUTE_LED,
},
+ [ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = cs35l41_fixup_i2c_two,
+ .chained = true,
+ .chain_id = ALC245_FIXUP_HP_X360_MUTE_LEDS,
+ },
[ALC287_FIXUP_CS35L41_I2C_4] = {
.type = HDA_FIXUP_FUNC,
.v.func = cs35l41_fixup_i2c_four,
@@ -10835,6 +10842,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1028, 0x097e, "Dell Precision", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x098d, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x09bf, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1028, 0x09d1, "Alienware m17 R3", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x0a2e, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a30, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a38, "Dell Latitude 7520", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE_QUIET),
@@ -11218,7 +11226,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x103c, 0x8d9c, "HP 17 Turbine OmniBook 7 DIS", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8d9d, "HP 17 Turbine OmniBook X UMA", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8d9e, "HP 17 Turbine OmniBook X DIS", ALC287_FIXUP_CS35L41_I2C_2),
- SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8da0, "HP 16 Clipper OmniBook 7(X360)", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8da1, "HP 16 Clipper OmniBook X", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8da7, "HP 14 Enstrom OmniBook X", ALC287_FIXUP_CS35L41_I2C_2),
diff --git a/sound/soc/amd/acp-config.c b/sound/soc/amd/acp-config.c
index 653e8ce94608..7ed461c1d519 100644
--- a/sound/soc/amd/acp-config.c
+++ b/sound/soc/amd/acp-config.c
@@ -23,6 +23,44 @@
static int acp_quirk_data;
+static const struct dmi_system_id acp70_acpi_flag_override_table[] = {
+ {
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "HN7306EA"),
+ },
+ },
+ {
+ /* ASUS Zenbook S16 UM5606GA (Strix Point, ACP 7.0) */
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Zenbook S16 UM5606GA"),
+ },
+ },
+ {
+ /* Lenovo Yoga Pro 7 15ASH11 (Strix Halo, ACP 7.0) */
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "83W5"),
+ },
+ },
+ {
+ /* HP OmniBook X Flip 16-cc0xxx */
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "HP"),
+ DMI_MATCH(DMI_BOARD_NAME, "8EA2"),
+ },
+ },
+ {
+ /* Dell 14S DS14265 */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Dell 14S DS14265"),
+ },
+ },
+ {}
+};
+
static const struct config_entry config_table[] = {
{
.flags = FLAG_AMD_SOF,
@@ -200,8 +238,11 @@ int snd_amd_acp_find_config(struct pci_dev *pci)
*/
if (!pci->revision)
return 0;
- else if (pci->revision >= ACP_7_0_REV)
+ else if (pci->revision >= ACP_7_0_REV) {
+ if (dmi_check_system(acp70_acpi_flag_override_table))
+ return 0;
return snd_amd_acp_acpi_find_config(pci);
+ }
for (i = 0; i < ARRAY_SIZE(config_table); i++, table++) {
if (table->device != device)
diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c
index 1aedead0a927..4a9ff060ebbe 100644
--- a/sound/soc/amd/yc/acp6x-mach.c
+++ b/sound/soc/amd/yc/acp6x-mach.c
@@ -675,6 +675,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = {
DMI_MATCH(DMI_BOARD_NAME, "8BD6"),
}
},
+ {
+ .driver_data = &acp6x_card,
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "HP"),
+ DMI_MATCH(DMI_BOARD_NAME, "8EE3"),
+ }
+ },
{
.driver_data = &acp6x_card,
.matches = {
diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c
index 25a145cb8a51..fc5067edd530 100644
--- a/sound/soc/codecs/hdac_hda.c
+++ b/sound/soc/codecs/hdac_hda.c
@@ -635,12 +635,35 @@ static const struct snd_soc_component_driver hdac_hda_hdmi_codec = {
.endianness = 1,
};
+static void hdac_hda_restore_dev_type(void *data)
+{
+ struct hdac_device *hdev = data;
+
+ hdev->type = HDA_DEV_LEGACY;
+}
+
static int hdac_hda_dev_probe(struct hdac_device *hdev)
{
- struct hdac_hda_priv *hda_pvt = dev_get_drvdata(&hdev->dev);
+ struct hdac_hda_priv *hda_pvt;
struct hdac_ext_link *hlink;
int ret;
+ hda_pvt = devm_kzalloc(&hdev->dev, sizeof(*hda_pvt), GFP_KERNEL);
+ if (!hda_pvt)
+ return -ENOMEM;
+
+ hda_pvt->codec = dev_to_hda_codec(&hdev->dev);
+ hda_pvt->dev_index = hdev->addr;
+ /* Intel iDisp (HDMI/DP) codec */
+ if (((hdev->vendor_id >> 16) & 0xFFFF) == 0x8086)
+ hda_pvt->need_display_power = true;
+ dev_set_drvdata(&hdev->dev, hda_pvt);
+
+ /* Allow the legacy codec driver to match again after unbind */
+ ret = devm_add_action(&hdev->dev, hdac_hda_restore_dev_type, hdev);
+ if (ret)
+ return ret;
+
/* hold the ref while we probe */
hlink = snd_hdac_ext_bus_get_hlink_by_name(hdev->bus, dev_name(&hdev->dev));
if (!hlink) {
diff --git a/sound/soc/qcom/qdsp6/q6apm-dai.c b/sound/soc/qcom/qdsp6/q6apm-dai.c
index 3b00c6d5174a..fef69e431890 100644
--- a/sound/soc/qcom/qdsp6/q6apm-dai.c
+++ b/sound/soc/qcom/qdsp6/q6apm-dai.c
@@ -441,11 +441,15 @@ static snd_pcm_uframes_t q6apm_dai_pointer(struct snd_soc_component *component,
struct q6apm_dai_rtd *prtd = runtime->private_data;
snd_pcm_uframes_t ptr;
- ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) * runtime->period_size;
- if (ptr)
- return ptr - 1;
+ /*
+ * The hw pointer counts the period buffers the DSP has returned in
+ * this lap, 1..periods. A full lap is the start of the ring again:
+ * report 0 rather than one frame less, so that the position of a
+ * drained buffer reaches appl_ptr and ALSA sees the underrun.
+ */
+ ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) % runtime->periods;
- return 0;
+ return ptr * runtime->period_size;
}
static int q6apm_dai_hw_params(struct snd_soc_component *component,
diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c
index 426c1429d01b..1025ddde9e31 100644
--- a/sound/soc/qcom/qdsp6/q6apm.c
+++ b/sound/soc/qcom/qdsp6/q6apm.c
@@ -762,8 +762,6 @@ static int apm_probe(gpr_device_t *gdev)
if (!apm)
return -ENOMEM;
- dev_set_drvdata(dev, apm);
-
mutex_init(&apm->lock);
apm->dev = dev;
apm->gdev = gdev;
@@ -777,6 +775,7 @@ static int apm_probe(gpr_device_t *gdev)
idr_init(&apm->modules_idr);
+ dev_set_drvdata(dev, apm);
g_apm = apm;
q6apm_get_apm_state(apm);
@@ -817,6 +816,9 @@ static int apm_callback(struct gpr_resp_pkt *data, void *priv, int op)
struct gpr_ibasic_rsp_result_t *result;
struct gpr_hdr *hdr = &data->hdr;
+ if (!apm)
+ return 0;
+
result = data->payload;
switch (hdr->opcode) {
diff --git a/sound/soc/qcom/qdsp6/q6prm.c b/sound/soc/qcom/qdsp6/q6prm.c
index 81554d202658..22e2a174a387 100644
--- a/sound/soc/qcom/qdsp6/q6prm.c
+++ b/sound/soc/qcom/qdsp6/q6prm.c
@@ -197,6 +197,9 @@ static int prm_callback(struct gpr_resp_pkt *data, void *priv, int op)
struct gpr_ibasic_rsp_result_t *result;
struct gpr_hdr *hdr = &data->hdr;
+ if (!prm)
+ return 0;
+
switch (hdr->opcode) {
case PRM_CMD_RSP_REQUEST_HW_RSC:
case PRM_CMD_RSP_RELEASE_HW_RSC:
diff --git a/sound/soc/sof/intel/hda-codec.c b/sound/soc/sof/intel/hda-codec.c
index c0c58b429715..57998d3d8d15 100644
--- a/sound/soc/sof/intel/hda-codec.c
+++ b/sound/soc/sof/intel/hda-codec.c
@@ -15,8 +15,6 @@
#include "hda.h"
#if IS_ENABLED(CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC)
-#include "../../codecs/hdac_hda.h"
-
#define CODEC_PROBE_RETRIES 3
#define IDISP_VID_INTEL 0x80860000
@@ -148,7 +146,6 @@ static struct hda_codec *hda_codec_device_init(struct hdac_bus *bus, int addr, i
/* probe individual codec */
static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
{
- struct hdac_hda_priv *hda_priv;
struct hda_bus *hbus = sof_to_hbus(sdev);
struct hda_codec *codec;
u32 hda_cmd = (address << 28) | (AC_NODE_ROOT << 20) |
@@ -168,27 +165,15 @@ static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
dev_dbg(sdev->dev, "HDA codec #%d probed OK: response: %x\n",
address, resp);
- hda_priv = devm_kzalloc(sdev->dev, sizeof(*hda_priv), GFP_KERNEL);
- if (!hda_priv)
- return -ENOMEM;
-
codec = hda_codec_device_init(&hbus->core, address, HDA_DEV_LEGACY);
ret = PTR_ERR_OR_ZERO(codec);
if (ret < 0)
return ret;
- hda_priv->codec = codec;
- hda_priv->dev_index = address;
- dev_set_drvdata(&codec->core.dev, hda_priv);
-
- if ((resp & 0xFFFF0000) == IDISP_VID_INTEL) {
- if (!hbus->core.audio_component) {
- dev_dbg(sdev->dev,
- "iDisp hw present but no driver\n");
- ret = -ENOENT;
- goto out;
- }
- hda_priv->need_display_power = true;
+ if ((resp & 0xFFFF0000) == IDISP_VID_INTEL && !hbus->core.audio_component) {
+ dev_dbg(sdev->dev, "iDisp hw present but no driver\n");
+ ret = -ENOENT;
+ goto out;
}
if (is_generic_config(hbus))
diff --git a/sound/soc/stm/stm32_i2s.c b/sound/soc/stm/stm32_i2s.c
index faa00103ee7f..d62802f4f12e 100644
--- a/sound/soc/stm/stm32_i2s.c
+++ b/sound/soc/stm/stm32_i2s.c
@@ -1026,7 +1026,7 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev,
struct device_node *np = pdev->dev.of_node;
struct reset_control *rst;
struct resource *res;
- int irq, ret;
+ int ret;
if (!np)
return -ENODEV;
@@ -1069,18 +1069,6 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev,
return ret;
}
- /* Get irqs */
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
-
- ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0,
- dev_name(&pdev->dev), i2s);
- if (ret) {
- dev_err(&pdev->dev, "irq request returned %d\n", ret);
- return ret;
- }
-
/* Reset */
rst = devm_reset_control_get_optional_exclusive(&pdev->dev, NULL);
if (IS_ERR(rst))
@@ -1105,7 +1093,7 @@ static int stm32_i2s_probe(struct platform_device *pdev)
{
struct stm32_i2s_data *i2s;
u32 val;
- int ret;
+ int irq, ret;
i2s = devm_kzalloc(&pdev->dev, sizeof(*i2s), GFP_KERNEL);
if (!i2s)
@@ -1131,9 +1119,18 @@ static int stm32_i2s_probe(struct platform_device *pdev)
return dev_err_probe(&pdev->dev, PTR_ERR(i2s->regmap),
"Regmap init error\n");
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0,
+ dev_name(&pdev->dev), i2s);
+ if (ret)
+ return ret;
+
ret = snd_dmaengine_pcm_register(&pdev->dev, &stm32_i2s_pcm_config, 0);
if (ret)
- return dev_err_probe(&pdev->dev, ret, "PCM DMA register error\n");
+ return ret;
ret = snd_soc_register_component(&pdev->dev, &stm32_i2s_component,
i2s->dai_drv, 1);
diff --git a/sound/synth/emux/emux_effect.c b/sound/synth/emux/emux_effect.c
index 3c7314f5fb19..bfe383fa90ba 100644
--- a/sound/synth/emux/emux_effect.c
+++ b/sound/synth/emux/emux_effect.c
@@ -168,7 +168,6 @@ snd_emux_send_effect(struct snd_emux_port *port, struct snd_midi_channel *chan,
unsigned char *srcp, *origp;
struct snd_emux *emu;
struct snd_emux_effect_table *fx;
- unsigned long flags;
emu = port->emu;
fx = chan->private;
@@ -195,22 +194,22 @@ snd_emux_send_effect(struct snd_emux_port *port, struct snd_midi_channel *chan,
offset++;
#endif
/* modify the register values */
- spin_lock_irqsave(&emu->voice_lock, flags);
- for (i = 0; i < emu->max_voices; i++) {
- struct snd_emux_voice *vp = &emu->voices[i];
- if (!STATE_IS_PLAYING(vp->state) || vp->chan != chan)
- continue;
- srcp = (unsigned char*)&vp->reg.parm + offset;
- origp = (unsigned char*)&vp->zone->v.parm + offset;
- if (parm_defs[i].type & PARM_IS_BYTE) {
- *srcp = *origp;
- effect_set_byte(srcp, chan, type);
- } else {
- *(unsigned short*)srcp = *(unsigned short*)origp;
- effect_set_word((unsigned short*)srcp, chan, type);
+ scoped_guard(spinlock_irqsave, &emu->voice_lock) {
+ for (i = 0; i < emu->max_voices; i++) {
+ struct snd_emux_voice *vp = &emu->voices[i];
+ if (!STATE_IS_PLAYING(vp->state) || vp->chan != chan)
+ continue;
+ srcp = (unsigned char *)&vp->reg.parm + offset;
+ origp = (unsigned char *)&vp->zone->v.parm + offset;
+ if (parm_defs[i].type & PARM_IS_BYTE) {
+ *srcp = *origp;
+ effect_set_byte(srcp, chan, type);
+ } else {
+ *(unsigned short *)srcp = *(unsigned short *)origp;
+ effect_set_word((unsigned short *)srcp, chan, type);
+ }
}
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
/* activate them */
snd_emux_update_channel(port, chan, parm_defs[type].update);
diff --git a/sound/synth/emux/emux_synth.c b/sound/synth/emux/emux_synth.c
index 02e2c69d7f18..537a7c47ef47 100644
--- a/sound/synth/emux/emux_synth.c
+++ b/sound/synth/emux/emux_synth.c
@@ -49,7 +49,6 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan)
int i, key, nvoices;
struct snd_emux_voice *vp;
struct snd_sf_zone *table[SNDRV_EMUX_MAX_MULTI_VOICES];
- unsigned long flags;
struct snd_emux_port *port;
port = p;
@@ -61,6 +60,9 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan)
return;
key = note; /* remember the original note */
+
+ /* hold sflist->lock for protecting against concurrent free */
+ guard(spinlock_irqsave)(&emu->sflist->lock);
nvoices = get_zone(emu, port, ¬e, vel, chan, table);
if (! nvoices)
return;
@@ -77,7 +79,7 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan)
terminate_note1(emu, key, chan, 0);
#endif
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < nvoices; i++) {
/* set up each voice parameter */
@@ -124,7 +126,6 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan)
vp->ontime = jiffies; /* remember the trigger timing */
}
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
#ifdef SNDRV_EMUX_USE_RAW_EFFECT
if (port->port_mode == SNDRV_EMUX_PORT_MODE_OSS_SYNTH) {
@@ -147,7 +148,6 @@ snd_emux_note_off(void *p, int note, int vel, struct snd_midi_channel *chan)
int ch;
struct snd_emux *emu;
struct snd_emux_voice *vp;
- unsigned long flags;
struct snd_emux_port *port;
port = p;
@@ -158,7 +158,7 @@ snd_emux_note_off(void *p, int note, int vel, struct snd_midi_channel *chan)
if (snd_BUG_ON(!emu || !emu->ops.release))
return;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (ch = 0; ch < emu->max_voices; ch++) {
vp = &emu->voices[ch];
if (STATE_IS_PLAYING(vp->state) &&
@@ -180,7 +180,6 @@ snd_emux_note_off(void *p, int note, int vel, struct snd_midi_channel *chan)
emu->ops.release(vp);
}
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
/*
@@ -192,10 +191,9 @@ void snd_emux_timer_callback(struct timer_list *t)
{
struct snd_emux *emu = from_timer(emu, t, tlist);
struct snd_emux_voice *vp;
- unsigned long flags;
int ch, do_again = 0;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (ch = 0; ch < emu->max_voices; ch++) {
vp = &emu->voices[ch];
if (vp->state == SNDRV_EMUX_ST_PENDING) {
@@ -212,7 +210,6 @@ void snd_emux_timer_callback(struct timer_list *t)
emu->timer_active = 1;
} else
emu->timer_active = 0;
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
/*
@@ -224,7 +221,6 @@ snd_emux_key_press(void *p, int note, int vel, struct snd_midi_channel *chan)
int ch;
struct snd_emux *emu;
struct snd_emux_voice *vp;
- unsigned long flags;
struct snd_emux_port *port;
port = p;
@@ -235,7 +231,7 @@ snd_emux_key_press(void *p, int note, int vel, struct snd_midi_channel *chan)
if (snd_BUG_ON(!emu || !emu->ops.update))
return;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (ch = 0; ch < emu->max_voices; ch++) {
vp = &emu->voices[ch];
if (vp->state == SNDRV_EMUX_ST_ON &&
@@ -244,7 +240,6 @@ snd_emux_key_press(void *p, int note, int vel, struct snd_midi_channel *chan)
update_voice(emu, vp, SNDRV_EMUX_UPDATE_VOLUME);
}
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
@@ -257,7 +252,6 @@ snd_emux_update_channel(struct snd_emux_port *port, struct snd_midi_channel *cha
struct snd_emux *emu;
struct snd_emux_voice *vp;
int i;
- unsigned long flags;
if (! update)
return;
@@ -266,13 +260,12 @@ snd_emux_update_channel(struct snd_emux_port *port, struct snd_midi_channel *cha
if (snd_BUG_ON(!emu || !emu->ops.update))
return;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < emu->max_voices; i++) {
vp = &emu->voices[i];
if (vp->chan == chan)
update_voice(emu, vp, update);
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
/*
@@ -284,7 +277,6 @@ snd_emux_update_port(struct snd_emux_port *port, int update)
struct snd_emux *emu;
struct snd_emux_voice *vp;
int i;
- unsigned long flags;
if (! update)
return;
@@ -293,13 +285,12 @@ snd_emux_update_port(struct snd_emux_port *port, int update)
if (snd_BUG_ON(!emu || !emu->ops.update))
return;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < emu->max_voices; i++) {
vp = &emu->voices[i];
if (vp->port == port)
update_voice(emu, vp, update);
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
@@ -365,16 +356,14 @@ terminate_note1(struct snd_emux *emu, int note, struct snd_midi_channel *chan, i
{
int i;
struct snd_emux_voice *vp;
- unsigned long flags;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < emu->max_voices; i++) {
vp = &emu->voices[i];
if (STATE_IS_PLAYING(vp->state) && vp->chan == chan &&
vp->key == note)
terminate_voice(emu, vp, free);
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
@@ -407,9 +396,8 @@ snd_emux_terminate_all(struct snd_emux *emu)
{
int i;
struct snd_emux_voice *vp;
- unsigned long flags;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < emu->max_voices; i++) {
vp = &emu->voices[i];
if (STATE_IS_PLAYING(vp->state))
@@ -424,7 +412,6 @@ snd_emux_terminate_all(struct snd_emux *emu)
}
/* initialize allocation time */
emu->use_time = 0;
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
EXPORT_SYMBOL(snd_emux_terminate_all);
@@ -438,7 +425,6 @@ snd_emux_sounds_off_all(struct snd_emux_port *port)
int i;
struct snd_emux *emu;
struct snd_emux_voice *vp;
- unsigned long flags;
if (snd_BUG_ON(!port))
return;
@@ -446,7 +432,7 @@ snd_emux_sounds_off_all(struct snd_emux_port *port)
if (snd_BUG_ON(!emu || !emu->ops.terminate))
return;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < emu->max_voices; i++) {
vp = &emu->voices[i];
if (STATE_IS_PLAYING(vp->state) &&
@@ -459,7 +445,6 @@ snd_emux_sounds_off_all(struct snd_emux_port *port)
emu->ops.reset(emu, i);
}
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
@@ -472,9 +457,8 @@ exclusive_note_off(struct snd_emux *emu, struct snd_emux_port *port, int exclass
{
struct snd_emux_voice *vp;
int i;
- unsigned long flags;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < emu->max_voices; i++) {
vp = &emu->voices[i];
if (STATE_IS_PLAYING(vp->state) && vp->port == port &&
@@ -482,7 +466,6 @@ exclusive_note_off(struct snd_emux *emu, struct snd_emux_port *port, int exclass
terminate_voice(emu, vp, 0);
}
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
/*
@@ -916,9 +899,8 @@ snd_emux_init_voices(struct snd_emux *emu)
{
struct snd_emux_voice *vp;
int i;
- unsigned long flags;
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
for (i = 0; i < emu->max_voices; i++) {
vp = &emu->voices[i];
vp->ch = -1; /* not used */
@@ -929,23 +911,19 @@ snd_emux_init_voices(struct snd_emux *emu)
vp->emu = emu;
vp->hw = emu->hw;
}
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
/*
*/
void snd_emux_lock_voice(struct snd_emux *emu, int voice)
{
- unsigned long flags;
-
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
if (emu->voices[voice].state == SNDRV_EMUX_ST_OFF)
emu->voices[voice].state = SNDRV_EMUX_ST_LOCKED;
else
dev_warn(emu->card->dev,
"invalid voice for lock %d (state = %x)\n",
voice, emu->voices[voice].state);
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
EXPORT_SYMBOL(snd_emux_lock_voice);
@@ -954,16 +932,13 @@ EXPORT_SYMBOL(snd_emux_lock_voice);
*/
void snd_emux_unlock_voice(struct snd_emux *emu, int voice)
{
- unsigned long flags;
-
- spin_lock_irqsave(&emu->voice_lock, flags);
+ guard(spinlock_irqsave)(&emu->voice_lock);
if (emu->voices[voice].state == SNDRV_EMUX_ST_LOCKED)
emu->voices[voice].state = SNDRV_EMUX_ST_OFF;
else
dev_warn(emu->card->dev,
"invalid voice for unlock %d (state = %x)\n",
voice, emu->voices[voice].state);
- spin_unlock_irqrestore(&emu->voice_lock, flags);
}
EXPORT_SYMBOL(snd_emux_unlock_voice);
diff --git a/sound/synth/emux/soundfont.c b/sound/synth/emux/soundfont.c
index 47137cca50b8..ad4a0f262afa 100644
--- a/sound/synth/emux/soundfont.c
+++ b/sound/synth/emux/soundfont.c
@@ -66,11 +66,9 @@ static void snd_sf_clear(struct snd_sf_list *sflist);
static void
lock_preset(struct snd_sf_list *sflist)
{
- unsigned long flags;
mutex_lock(&sflist->presets_mutex);
- spin_lock_irqsave(&sflist->lock, flags);
+ guard(spinlock_irqsave)(&sflist->lock);
sflist->presets_locked = 1;
- spin_unlock_irqrestore(&sflist->lock, flags);
}
@@ -80,10 +78,8 @@ lock_preset(struct snd_sf_list *sflist)
static void
unlock_preset(struct snd_sf_list *sflist)
{
- unsigned long flags;
- spin_lock_irqsave(&sflist->lock, flags);
+ guard(spinlock_irqsave)(&sflist->lock);
sflist->presets_locked = 0;
- spin_unlock_irqrestore(&sflist->lock, flags);
mutex_unlock(&sflist->presets_mutex);
}
@@ -94,14 +90,11 @@ unlock_preset(struct snd_sf_list *sflist)
int
snd_soundfont_close_check(struct snd_sf_list *sflist, int client)
{
- unsigned long flags;
- spin_lock_irqsave(&sflist->lock, flags);
- if (sflist->open_client == client) {
- spin_unlock_irqrestore(&sflist->lock, flags);
- return close_patch(sflist);
+ scoped_guard(spinlock_irqsave, &sflist->lock) {
+ if (sflist->open_client != client)
+ return 0;
}
- spin_unlock_irqrestore(&sflist->lock, flags);
- return 0;
+ return close_patch(sflist);
}
@@ -119,7 +112,6 @@ snd_soundfont_load(struct snd_card *card,
long count, int client)
{
struct soundfont_patch_info patch;
- unsigned long flags;
int rc;
if (count < (long)sizeof(patch)) {
@@ -155,12 +147,10 @@ snd_soundfont_load(struct snd_card *card,
}
/* check if other client already opened patch */
- spin_lock_irqsave(&sflist->lock, flags);
- if (sflist->open_client != client) {
- spin_unlock_irqrestore(&sflist->lock, flags);
- return -EBUSY;
+ scoped_guard(spinlock_irqsave, &sflist->lock) {
+ if (sflist->open_client != client)
+ return -EBUSY;
}
- spin_unlock_irqrestore(&sflist->lock, flags);
lock_preset(sflist);
rc = -EINVAL;
@@ -223,14 +213,11 @@ open_patch(struct snd_sf_list *sflist, const char __user *data,
{
struct soundfont_open_parm parm;
struct snd_soundfont *sf;
- unsigned long flags;
- spin_lock_irqsave(&sflist->lock, flags);
- if (sflist->open_client >= 0 || sflist->currsf) {
- spin_unlock_irqrestore(&sflist->lock, flags);
- return -EBUSY;
+ scoped_guard(spinlock_irqsave, &sflist->lock) {
+ if (sflist->open_client >= 0 || sflist->currsf)
+ return -EBUSY;
}
- spin_unlock_irqrestore(&sflist->lock, flags);
if (copy_from_user(&parm, data, sizeof(parm)))
return -EFAULT;
@@ -244,10 +231,10 @@ open_patch(struct snd_sf_list *sflist, const char __user *data,
return -ENOMEM;
}
- spin_lock_irqsave(&sflist->lock, flags);
- sflist->open_client = client;
- sflist->currsf = sf;
- spin_unlock_irqrestore(&sflist->lock, flags);
+ scoped_guard(spinlock_irqsave, &sflist->lock) {
+ sflist->open_client = client;
+ sflist->currsf = sf;
+ }
return 0;
}
@@ -305,12 +292,10 @@ is_identical_font(struct snd_soundfont *sf, int type, unsigned char *name)
static int
close_patch(struct snd_sf_list *sflist)
{
- unsigned long flags;
-
- spin_lock_irqsave(&sflist->lock, flags);
- sflist->currsf = NULL;
- sflist->open_client = -1;
- spin_unlock_irqrestore(&sflist->lock, flags);
+ scoped_guard(spinlock_irqsave, &sflist->lock) {
+ sflist->currsf = NULL;
+ sflist->open_client = -1;
+ }
rebuild_presets(sflist);
@@ -1279,6 +1264,9 @@ delete_preset(struct snd_sf_list *sflist, struct snd_sf_zone *zp)
* The found zones are stored on 'table' array. max_layers defines
* the maximum number of elements in this array.
* This function returns the number of found zones. 0 if not found.
+ *
+ * The caller must hold sflist->lock, and keep holding it as long as
+ * the returned zones are accessed.
*/
int
snd_soundfont_search_zone(struct snd_sf_list *sflist, int *notep, int vel,
@@ -1287,17 +1275,14 @@ snd_soundfont_search_zone(struct snd_sf_list *sflist, int *notep, int vel,
struct snd_sf_zone **table, int max_layers)
{
int nvoices;
- unsigned long flags;
- /* this function is supposed to be called atomically,
- * so we check the lock. if it's busy, just returns 0 to
+ lockdep_assert_held(&sflist->lock);
+
+ /* if the presets are being modified, just returns 0 to
* tell the caller the busy state
*/
- spin_lock_irqsave(&sflist->lock, flags);
- if (sflist->presets_locked) {
- spin_unlock_irqrestore(&sflist->lock, flags);
+ if (sflist->presets_locked)
return 0;
- }
nvoices = search_zones(sflist, notep, vel, preset, bank,
table, max_layers, 0);
if (! nvoices) {
@@ -1306,7 +1291,6 @@ snd_soundfont_search_zone(struct snd_sf_list *sflist, int *notep, int vel,
def_preset, def_bank,
table, max_layers, 0);
}
- spin_unlock_irqrestore(&sflist->lock, flags);
return nvoices;
}
diff --git a/sound/usb/bcd2000/bcd2000.c b/sound/usb/bcd2000/bcd2000.c
index dce6d1f611f9..fc18a4348265 100644
--- a/sound/usb/bcd2000/bcd2000.c
+++ b/sound/usb/bcd2000/bcd2000.c
@@ -183,10 +183,24 @@ static int bcd2000_midi_output_open(struct snd_rawmidi_substream *substream)
static int bcd2000_midi_output_close(struct snd_rawmidi_substream *substream)
{
struct bcd2000 *bcd2k = substream->rmidi->private_data;
+ struct urb *urb = NULL;
- if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
- usb_kill_urb(bcd2k->midi_out_urb);
- bcd2k->midi_out_active = 0;
+ /*
+ * Reference the URB under the lock so that it stays valid until
+ * usb_kill_urb() returns, even if a disconnect frees it in the
+ * meantime. usb_kill_urb() cannot run under the lock, as the
+ * completion handler takes the same lock.
+ */
+ scoped_guard(spinlock_irqsave, &bcd2k->midi_lock) {
+ if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
+ urb = usb_get_urb(bcd2k->midi_out_urb);
+ bcd2k->midi_out_active = 0;
+ }
+ }
+
+ if (urb) {
+ usb_kill_urb(urb);
+ usb_put_urb(urb);
}
return 0;
diff --git a/sound/usb/caiaq/input.c b/sound/usb/caiaq/input.c
index c12eeb971000..6a5adb8d1e6a 100644
--- a/sound/usb/caiaq/input.c
+++ b/sound/usb/caiaq/input.c
@@ -830,6 +830,8 @@ int snd_usb_caiaq_input_init(struct snd_usb_caiaqdev *cdev)
return 0;
exit_free_idev:
+ usb_free_urb(cdev->ep4_in_urb);
+ cdev->ep4_in_urb = NULL;
input_free_device(input);
cdev->input_dev = NULL;
return ret;
diff --git a/sound/usb/line6/capture.c b/sound/usb/line6/capture.c
index 84a9b7b76f43..f59ec9c4a1a5 100644
--- a/sound/usb/line6/capture.c
+++ b/sound/usb/line6/capture.c
@@ -94,6 +94,18 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize)
if (runtime == NULL)
return;
+ /*
+ * snd_line6_capture_open() constrains the buffer to hold at least
+ * two packets, so this cannot happen for a conforming device. Drop
+ * the packet rather than overrun the single wrap below.
+ */
+ if (unlikely(frames > runtime->buffer_size)) {
+ dev_err_ratelimited(line6pcm->line6->ifcdev,
+ "packet of %d frames exceeds %lu-frame buffer, dropping\n",
+ frames, runtime->buffer_size);
+ return;
+ }
+
if (line6pcm->in.pos_done + frames > runtime->buffer_size) {
/*
The transferred area goes over buffer boundary,
@@ -125,28 +137,23 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize)
line6pcm->in.pos_done -= runtime->buffer_size;
}
-void line6_capture_check_period(struct snd_line6_pcm *line6pcm, int length)
+/* update the capture position and check whether we need period_elapsed call */
+bool line6_capture_check_period(struct snd_line6_pcm *line6pcm, int length)
{
- struct snd_pcm_substream *substream =
- get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE);
-
line6pcm->in.bytes += length;
if (line6pcm->in.bytes >= line6pcm->in.period) {
line6pcm->in.bytes %= line6pcm->in.period;
- spin_unlock(&line6pcm->in.lock);
- snd_pcm_period_elapsed(substream);
- spin_lock(&line6pcm->in.lock);
+ return true;
}
+ return false;
}
/*
* Callback for completed capture URB.
*/
-static void audio_in_callback(struct urb *urb)
+static bool handle_audio_in(struct urb *urb)
{
int i, index, length = 0, shutdown = 0;
- unsigned long flags;
-
struct snd_line6_pcm *line6pcm = (struct snd_line6_pcm *)urb->context;
line6pcm->in.last_frame = urb->start_frame;
@@ -156,7 +163,7 @@ static void audio_in_callback(struct urb *urb)
if (urb == line6pcm->in.urbs[index])
break;
- spin_lock_irqsave(&line6pcm->in.lock, flags);
+ guard(spinlock_irqsave)(&line6pcm->in.lock);
for (i = 0; i < LINE6_ISO_PACKETS; ++i) {
char *fbuf;
@@ -209,10 +216,18 @@ static void audio_in_callback(struct urb *urb)
if (!test_bit(LINE6_STREAM_IMPULSE, &line6pcm->in.running) &&
test_bit(LINE6_STREAM_PCM, &line6pcm->in.running))
- line6_capture_check_period(line6pcm, length);
+ return line6_capture_check_period(line6pcm, length);
}
- spin_unlock_irqrestore(&line6pcm->in.lock, flags);
+ return false;
+}
+
+static void audio_in_callback(struct urb *urb)
+{
+ struct snd_line6_pcm *line6pcm = (struct snd_line6_pcm *)urb->context;
+
+ if (handle_audio_in(urb))
+ snd_pcm_period_elapsed(get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE));
}
/* open capture callback */
@@ -228,6 +243,23 @@ static int snd_line6_capture_open(struct snd_pcm_substream *substream)
if (err < 0)
return err;
+ /*
+ * line6_capture_copy() wraps the ring buffer once per packet and
+ * advances pos_done by the packet, which only works when the buffer
+ * holds more than one packet: with exactly one, pos_done gains a
+ * buffer and loses it again on every packet. The packet size comes
+ * from the device, so bound the buffer by two packets here. The
+ * impulse-response path copies a playback URB into this ring, so
+ * take the larger of the two endpoints.
+ */
+ err = snd_pcm_hw_constraint_minmax(runtime,
+ SNDRV_PCM_HW_PARAM_BUFFER_BYTES,
+ 2 * max(line6pcm->max_packet_size_in,
+ line6pcm->max_packet_size_out),
+ UINT_MAX);
+ if (err < 0)
+ return err;
+
line6_pcm_acquire(line6pcm, LINE6_STREAM_CAPTURE_HELPER, false);
runtime->hw = line6pcm->properties->capture_hw;
diff --git a/sound/usb/line6/capture.h b/sound/usb/line6/capture.h
index 90572dae134e..628315389efc 100644
--- a/sound/usb/line6/capture.h
+++ b/sound/usb/line6/capture.h
@@ -17,7 +17,7 @@ extern const struct snd_pcm_ops snd_line6_capture_ops;
extern void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf,
int fsize);
-extern void line6_capture_check_period(struct snd_line6_pcm *line6pcm,
+extern bool line6_capture_check_period(struct snd_line6_pcm *line6pcm,
int length);
extern int line6_create_audio_in_urbs(struct snd_line6_pcm *line6pcm);
extern int line6_submit_audio_in_all_urbs(struct snd_line6_pcm *line6pcm);
diff --git a/sound/usb/line6/driver.c b/sound/usb/line6/driver.c
index e9eb5c74d6c7..caf9bc661686 100644
--- a/sound/usb/line6/driver.c
+++ b/sound/usb/line6/driver.c
@@ -286,31 +286,30 @@ static void line6_data_received(struct urb *urb)
{
struct usb_line6 *line6 = (struct usb_line6 *)urb->context;
struct midi_buffer *mb = &line6->line6midi->midibuf_in;
- unsigned long flags;
int done;
if (urb->status == -ESHUTDOWN)
return;
if (line6->properties->capabilities & LINE6_CAP_CONTROL_MIDI) {
- spin_lock_irqsave(&line6->line6midi->lock, flags);
- done =
- line6_midibuf_write(mb, urb->transfer_buffer, urb->actual_length);
-
- if (done < urb->actual_length) {
- line6_midibuf_ignore(mb, done);
- dev_dbg(line6->ifcdev, "%d %d buffer overflow - message skipped\n",
- done, urb->actual_length);
+ scoped_guard(spinlock_irqsave, &line6->line6midi->lock) {
+ done =
+ line6_midibuf_write(mb, urb->transfer_buffer, urb->actual_length);
+
+ if (done < urb->actual_length) {
+ line6_midibuf_ignore(mb, done);
+ dev_dbg(line6->ifcdev, "%d %d buffer overflow - message skipped\n",
+ done, urb->actual_length);
+ }
}
- spin_unlock_irqrestore(&line6->line6midi->lock, flags);
for (;;) {
- spin_lock_irqsave(&line6->line6midi->lock, flags);
- done =
- line6_midibuf_read(mb, line6->buffer_message,
- LINE6_MIDI_MESSAGE_MAXLEN,
- LINE6_MIDIBUF_READ_RX);
- spin_unlock_irqrestore(&line6->line6midi->lock, flags);
+ scoped_guard(spinlock_irqsave, &line6->line6midi->lock) {
+ done =
+ line6_midibuf_read(mb, line6->buffer_message,
+ LINE6_MIDI_MESSAGE_MAXLEN,
+ LINE6_MIDIBUF_READ_RX);
+ }
if (done <= 0)
break;
diff --git a/sound/usb/line6/midi.c b/sound/usb/line6/midi.c
index 9b5176086280..80d9a6d90342 100644
--- a/sound/usb/line6/midi.c
+++ b/sound/usb/line6/midi.c
@@ -72,7 +72,6 @@ static void line6_midi_transmit(struct snd_rawmidi_substream *substream)
*/
static void midi_sent(struct urb *urb)
{
- unsigned long flags;
int status;
int num;
struct usb_line6 *line6 = (struct usb_line6 *)urb->context;
@@ -84,7 +83,7 @@ static void midi_sent(struct urb *urb)
if (status == -ESHUTDOWN)
return;
- spin_lock_irqsave(&line6->line6midi->lock, flags);
+ guard(spinlock_irqsave)(&line6->line6midi->lock);
num = --line6->line6midi->num_active_send_urbs;
if (num == 0) {
@@ -94,8 +93,6 @@ static void midi_sent(struct urb *urb)
if (num == 0)
wake_up(&line6->line6midi->send_wait);
-
- spin_unlock_irqrestore(&line6->line6midi->lock, flags);
}
/*
@@ -158,17 +155,14 @@ static int line6_midi_output_close(struct snd_rawmidi_substream *substream)
static void line6_midi_output_trigger(struct snd_rawmidi_substream *substream,
int up)
{
- unsigned long flags;
struct usb_line6 *line6 =
line6_rawmidi_substream_midi(substream)->line6;
line6->line6midi->substream_transmit = substream;
- spin_lock_irqsave(&line6->line6midi->lock, flags);
+ guard(spinlock_irqsave)(&line6->line6midi->lock);
if (line6->line6midi->num_active_send_urbs == 0)
line6_midi_transmit(substream);
-
- spin_unlock_irqrestore(&line6->line6midi->lock, flags);
}
static void line6_midi_output_drain(struct snd_rawmidi_substream *substream)
diff --git a/sound/usb/line6/pcm.c b/sound/usb/line6/pcm.c
index 7f51a81d7ec9..77d5802872b1 100644
--- a/sound/usb/line6/pcm.c
+++ b/sound/usb/line6/pcm.c
@@ -190,11 +190,10 @@ static void line6_buffer_release(struct snd_line6_pcm *line6pcm,
static int line6_stream_start(struct snd_line6_pcm *line6pcm, int direction,
int type)
{
- unsigned long flags;
struct line6_pcm_stream *pstr = get_stream(line6pcm, direction);
int ret = 0;
- spin_lock_irqsave(&pstr->lock, flags);
+ guard(spinlock_irqsave)(&pstr->lock);
if (!test_and_set_bit(type, &pstr->running) &&
!(pstr->active_urbs || pstr->unlink_urbs)) {
pstr->count = 0;
@@ -207,7 +206,6 @@ static int line6_stream_start(struct snd_line6_pcm *line6pcm, int direction,
if (ret < 0)
clear_bit(type, &pstr->running);
- spin_unlock_irqrestore(&pstr->lock, flags);
return ret;
}
@@ -215,21 +213,20 @@ static int line6_stream_start(struct snd_line6_pcm *line6pcm, int direction,
static void line6_stream_stop(struct snd_line6_pcm *line6pcm, int direction,
int type)
{
- unsigned long flags;
struct line6_pcm_stream *pstr = get_stream(line6pcm, direction);
- spin_lock_irqsave(&pstr->lock, flags);
- clear_bit(type, &pstr->running);
- if (!pstr->running) {
- spin_unlock_irqrestore(&pstr->lock, flags);
- line6_unlink_audio_urbs(line6pcm, pstr);
- spin_lock_irqsave(&pstr->lock, flags);
- if (direction == SNDRV_PCM_STREAM_CAPTURE) {
- line6pcm->prev_fbuf = NULL;
- line6pcm->prev_fsize = 0;
- }
+ scoped_guard(spinlock_irqsave, &pstr->lock) {
+ clear_bit(type, &pstr->running);
+ if (pstr->running)
+ return;
+ }
+
+ line6_unlink_audio_urbs(line6pcm, pstr);
+ if (direction == SNDRV_PCM_STREAM_CAPTURE) {
+ guard(spinlock_irqsave)(&pstr->lock);
+ line6pcm->prev_fbuf = NULL;
+ line6pcm->prev_fsize = 0;
}
- spin_unlock_irqrestore(&pstr->lock, flags);
}
/* common PCM trigger callback */
diff --git a/sound/usb/line6/playback.c b/sound/usb/line6/playback.c
index 2e70ca5456c5..495a983a0c51 100644
--- a/sound/usb/line6/playback.c
+++ b/sound/usb/line6/playback.c
@@ -136,6 +136,8 @@ static void add_monitor_signal(struct urb *urb_out, unsigned char *signal,
/*
Find a free URB, prepare audio data, and submit URB.
must be called in line6pcm->out.lock context
+ Returns 1 if snd_pcm_period_elapsed() needs to be called for the
+ capture stream after releasing the lock, 0 if not, or a negative error
*/
static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
{
@@ -151,6 +153,7 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
line6pcm->properties->rates.rats[0].den *
(line6pcm->line6->intervals_per_second / LINE6_ISO_INTERVAL);
struct urb *urb_out;
+ bool capture_period_elapsed = false;
index = find_first_zero_bit(&line6pcm->out.active_urbs,
line6pcm->line6->iso_buffers);
@@ -260,8 +263,9 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
urb_out->transfer_buffer,
urb_out->
transfer_buffer_length);
- line6_capture_check_period(line6pcm,
- urb_out->transfer_buffer_length);
+ capture_period_elapsed =
+ line6_capture_check_period(line6pcm,
+ urb_out->transfer_buffer_length);
}
} else {
if (!(line6pcm->line6->properties->capabilities & LINE6_CAP_HWMON)
@@ -283,24 +287,26 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
dev_err(line6pcm->line6->ifcdev,
"URB out #%d submission failed (%d)\n", index, ret);
- return 0;
+ return capture_period_elapsed;
}
/*
Submit all currently available playback URBs.
must be called in line6pcm->out.lock context
+ The capture period notification is skipped here since we can't call
+ snd_pcm_period_elapsed() in the lock; it'll be caught up at the next one.
*/
int line6_submit_audio_out_all_urbs(struct snd_line6_pcm *line6pcm)
{
- int ret = 0, i;
+ int ret, i;
for (i = 0; i < line6pcm->line6->iso_buffers; ++i) {
ret = submit_audio_out_urb(line6pcm);
if (ret < 0)
- break;
+ return ret;
}
- return ret;
+ return 0;
}
/*
@@ -316,6 +322,8 @@ static void audio_out_callback(struct urb *urb)
const int bytes_per_frame =
line6pcm->properties->bytes_per_channel *
line6pcm->properties->playback_hw.channels_max;
+ bool capture_period_elapsed = false;
+ bool playback_period_elapsed = false;
#if USE_CLEAR_BUFFER_WORKAROUND
memset(urb->transfer_buffer, 0, urb->transfer_buffer_length);
@@ -358,19 +366,22 @@ static void audio_out_callback(struct urb *urb)
shutdown = 1;
if (!shutdown) {
- submit_audio_out_urb(line6pcm);
+ capture_period_elapsed = submit_audio_out_urb(line6pcm) > 0;
if (test_bit(LINE6_STREAM_PCM, &line6pcm->out.running)) {
line6pcm->out.bytes += length;
if (line6pcm->out.bytes >= line6pcm->out.period) {
line6pcm->out.bytes %= line6pcm->out.period;
- spin_unlock(&line6pcm->out.lock);
- snd_pcm_period_elapsed(substream);
- spin_lock(&line6pcm->out.lock);
+ playback_period_elapsed = true;
}
}
}
spin_unlock_irqrestore(&line6pcm->out.lock, flags);
+
+ if (capture_period_elapsed)
+ snd_pcm_period_elapsed(get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE));
+ if (playback_period_elapsed)
+ snd_pcm_period_elapsed(substream);
}
/* open playback callback */
diff --git a/sound/usb/midi.c b/sound/usb/midi.c
index cb9bcfa22afc..6ac38297c6c5 100644
--- a/sound/usb/midi.c
+++ b/sound/usb/midi.c
@@ -1152,7 +1152,9 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir,
int open)
{
struct snd_usb_midi *umidi = substream->rmidi->private_data;
- struct snd_kcontrol *ctl;
+ struct snd_ctl_elem_id ctl_id;
+ bool activate_ctl = false;
+ bool active;
down_read(&umidi->disc_rwsem);
if (umidi->disconnected) {
@@ -1164,11 +1166,9 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir,
if (open) {
if (!umidi->opened[0] && !umidi->opened[1]) {
if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access |=
- SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = false;
update_roland_altsetting(umidi);
}
}
@@ -1181,15 +1181,15 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir,
snd_usbmidi_input_stop(&umidi->list);
if (!umidi->opened[0] && !umidi->opened[1]) {
if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access &=
- ~SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = true;
}
}
}
mutex_unlock(&umidi->mutex);
+ if (activate_ctl)
+ snd_ctl_activate_id(umidi->card, &ctl_id, active);
up_read(&umidi->disc_rwsem);
return 0;
}
diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c
index 82c859626404..45145ef7db7c 100644
--- a/sound/usb/mixer.c
+++ b/sound/usb/mixer.c
@@ -1156,6 +1156,14 @@ static void volume_control_quirks(struct usb_mixer_elem_info *cval,
}
break;
+ case USB_ID(0x045e, 0x070f): /* MS LifeChat LX-3000 Headset */
+ if (!strcmp(kctl->id.name, "Speaker Playback Volume")) {
+ usb_audio_info(chip,
+ "set volume quirk for MS LifeChat LX-3000\n");
+ cval->res = 192;
+ }
+ break;
+
case USB_ID(0x0471, 0x0101):
case USB_ID(0x0471, 0x0104):
case USB_ID(0x0471, 0x0105):
diff --git a/sound/usb/mixer_quirks.c b/sound/usb/mixer_quirks.c
index 48f6b53fefd8..18e41ea17956 100644
--- a/sound/usb/mixer_quirks.c
+++ b/sound/usb/mixer_quirks.c
@@ -4501,9 +4501,12 @@ void snd_usb_mixer_fu_apply_quirk(struct usb_mixer_interface *mixer,
}
/* lowest playback value is muted on some devices */
- if (mixer->chip->quirk_flags & QUIRK_FLAG_MIXER_MIN_MUTE)
- if (strstr(kctl->id.name, "Playback"))
+ if (mixer->chip->quirk_flags & QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE)
+ if (strstr(kctl->id.name, "Playback")) {
+ usb_audio_info(mixer->chip,
+ "applying playback min mute quirk\n");
cval->min_mute = 1;
+ }
/* ALSA-ify some Plantronics headset control names */
if (USB_ID_VENDOR(mixer->chip->usb_id) == 0x047f &&
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 5aadd1221f72..dd7554cde865 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -1292,6 +1292,18 @@ static void retire_capture_urb(struct snd_usb_substream *subs,
bytes -= adj;
subs->stream_offset_adj -= adj;
}
+ /*
+ * A packet can be as large as the endpoint's maxpacksize, which
+ * the device chooses, while the ring buffer can be as small as
+ * two 64-byte periods. The hwptr_done wrap below handles one
+ * wrap only, so never take more than one buffer from a packet.
+ */
+ if (unlikely(bytes > subs->buffer_bytes)) {
+ dev_warn_ratelimited(&subs->dev->dev,
+ "%u-byte packet exceeds %u-byte buffer, truncating\n",
+ bytes, subs->buffer_bytes);
+ bytes = subs->buffer_bytes;
+ }
frames = bytes / stride;
if (!subs->txfr_quirk)
bytes = frames * stride;
@@ -1505,8 +1517,14 @@ static int prepare_playback_urb(struct snd_usb_substream *subs,
for (i = 0; i < ctx->packets; i++) {
counts = snd_usb_endpoint_next_packet_size(ep, ctx, i, avail);
+ /*
+ * Bound by the ring buffer as well as by the URB:
+ * copy_to_urb() wraps hwptr_done once, so one URB must
+ * never carry more than the whole buffer.
+ */
if (counts < 0 ||
- (frames + counts) * stride > ctx->buffer_size)
+ (frames + counts) * stride > ctx->buffer_size ||
+ (frames + counts) * stride > subs->buffer_bytes)
break;
/* set up descriptor */
urb->iso_frame_desc[i].offset = frames * stride;
diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c
index f224e41209e8..5a69bd87628a 100644
--- a/sound/usb/quirks.c
+++ b/sound/usb/quirks.c
@@ -2158,6 +2158,10 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x045e, 0x083c, /* MS USB Link headset */
QUIRK_FLAG_GET_SAMPLE_RATE | QUIRK_FLAG_CTL_MSG_DELAY |
QUIRK_FLAG_DISABLE_AUTOSUSPEND),
+ DEVICE_FLG(0x045e, 0x070f, /* MS LifeChat LX-3000 Headset */
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
+ DEVICE_FLG(0x046d, 0x0802, /* Logitech Webcam C200 */
+ QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIC_RES_384),
DEVICE_FLG(0x046d, 0x0807, /* Logitech Webcam C500 */
QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIC_RES_384),
DEVICE_FLG(0x046d, 0x0808, /* Logitech Webcam C600 */
@@ -2202,9 +2206,9 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x0556, 0x0014, /* Phoenix Audio TMX320VC */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x0572, 0x1b08, /* Conexant Systems (Rockwell), Inc. */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x0572, 0x1b09, /* Conexant Systems (Rockwell), Inc. */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x05a3, 0x9420, /* ELP HD USB Camera */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x05a7, 0x1020, /* Bose Companion 5 */
@@ -2254,15 +2258,15 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x0b0e, 0x0349, /* Jabra 550a */
QUIRK_FLAG_CTL_MSG_DELAY_1M),
DEVICE_FLG(0x0bda, 0x498a, /* Realtek Semiconductor Corp. */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x0c45, 0x6340, /* Sonix HD USB Camera */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x0c45, 0x636b, /* Microdia JP001 USB Camera */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x0d8c, 0x000c, /* C-Media */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x0d8c, 0x0014, /* C-Media */
- QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x0ecb, 0x205c, /* JBL Quantum610 Wireless */
QUIRK_FLAG_FIXED_RATE),
DEVICE_FLG(0x0ecb, 0x2125, /* JBL Quantum650 Wireless */
@@ -2274,7 +2278,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x1101, 0x0003, /* Audioengine D1 */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x12d1, 0x3a07, /* Huawei Technologies Co., Ltd. */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x1224, 0x2a25, /* Jieli Technology USB PHY 2.0 */
QUIRK_FLAG_GET_SAMPLE_RATE | QUIRK_FLAG_MIC_RES_16),
DEVICE_FLG(0x1395, 0x740a, /* Sennheiser DECT */
@@ -2320,7 +2324,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x1901, 0x0191, /* GE B850V3 CP2114 audio interface */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x19f7, 0x0003, /* RODE NT-USB */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x19f7, 0x0035, /* RODE NT-USB+ */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x1bcf, 0x2281, /* HD Webcam */
@@ -2378,7 +2382,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x2912, 0x30c8, /* Audioengine D1 */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x2a70, 0x1881, /* OnePlus Technology (Shenzhen) Co., Ltd. BE02T */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x2b53, 0x0023, /* Fiero SC-01 (firmware v1.0.0 @ 48 kHz) */
QUIRK_FLAG_GENERIC_IMPLICIT_FB),
DEVICE_FLG(0x2b53, 0x0024, /* Fiero SC-01 (firmware v1.0.0 @ 96 kHz) */
@@ -2390,7 +2394,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x2d95, 0x8021, /* VIVO USB-C-XE710 HEADSET */
QUIRK_FLAG_CTL_MSG_DELAY_1M),
DEVICE_FLG(0x2d99, 0x0026, /* HECATE G2 GAMING HEADSET */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x2fc6, 0xf0b7, /* iBasso DC07 Pro */
QUIRK_FLAG_CTL_MSG_DELAY_1M),
DEVICE_FLG(0x30be, 0x0101, /* Schiit Hel */
@@ -2398,7 +2402,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
DEVICE_FLG(0x3255, 0x0000, /* Luxman D-10X */
QUIRK_FLAG_ITF_USB_DSD_DAC | QUIRK_FLAG_CTL_MSG_DELAY),
DEVICE_FLG(0x339b, 0x3a07, /* Synaptics HONOR USB-C HEADSET */
- QUIRK_FLAG_MIXER_MIN_MUTE),
+ QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x413c, 0xa506, /* Dell AE515 sound bar */
QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x534d, 0x0021, /* MacroSilicon MS2100/MS2106 */
diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
index 1ef4d39978df..8a951394cabf 100644
--- a/sound/usb/usbaudio.h
+++ b/sound/usb/usbaudio.h
@@ -196,7 +196,7 @@ extern bool snd_usb_skip_validation;
* for the given endpoint.
* QUIRK_FLAG_MIC_RES_16 and QUIRK_FLAG_MIC_RES_384
* Set the fixed resolution for Mic Capture Volume (mostly for webcams)
- * QUIRK_FLAG_MIXER_MIN_MUTE
+ * QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE
* Set minimum volume control value as mute for devices where the lowest
* playback value represents muted state instead of minimum audible volume
*/
@@ -225,6 +225,6 @@ extern bool snd_usb_skip_validation;
#define QUIRK_FLAG_FIXED_RATE (1U << 21)
#define QUIRK_FLAG_MIC_RES_16 (1U << 22)
#define QUIRK_FLAG_MIC_RES_384 (1U << 23)
-#define QUIRK_FLAG_MIXER_MIN_MUTE (1U << 24)
+#define QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE (1U << 24)
#endif /* __USBAUDIO_H */
diff --git a/sound/usb/usx2y/us122l.c b/sound/usb/usx2y/us122l.c
index 2a107803af88..888005192704 100644
--- a/sound/usb/usx2y/us122l.c
+++ b/sound/usb/usx2y/us122l.c
@@ -148,19 +148,26 @@ static const struct vm_operations_struct usb_stream_hwdep_vm_ops = {
static int usb_stream_hwdep_open(struct snd_hwdep *hw, struct file *file)
{
struct us122l *us122l = hw->private_data;
- struct usb_interface *iface;
+ struct usb_interface *iface, *iface0 = NULL;
if (hw->used >= 2)
return -EBUSY;
- if (!us122l->first)
- us122l->first = file;
-
if (us122l->is_us144) {
- iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_get_interface(iface);
+ iface0 = usb_ifnum_to_if(us122l->dev, 0);
+ if (!iface0)
+ return -ENODEV;
}
+
iface = usb_ifnum_to_if(us122l->dev, 1);
+ if (!iface)
+ return -ENODEV;
+
+ if (!us122l->first)
+ us122l->first = file;
+
+ if (iface0)
+ usb_autopm_get_interface(iface0);
usb_autopm_get_interface(iface);
return 0;
}
@@ -172,10 +179,12 @@ static int usb_stream_hwdep_release(struct snd_hwdep *hw, struct file *file)
if (us122l->is_us144) {
iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
}
iface = usb_ifnum_to_if(us122l->dev, 1);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
if (us122l->first == file)
us122l->first = NULL;
mutex_lock(&us122l->mutex);
@@ -200,6 +209,8 @@ static int usb_stream_hwdep_mmap(struct snd_hwdep *hw,
offset = area->vm_pgoff << PAGE_SHIFT;
mutex_lock(&us122l->mutex);
s = us122l->sk.s;
+ if (!s)
+ return -ENODEV;
read = offset < s->read_size;
if (read) {
if (area->vm_flags & VM_WRITE) {
diff --git a/sound/usb/usx2y/usX2Yhwdep.c b/sound/usb/usx2y/usX2Yhwdep.c
index 2cc0f90a23be..be755a8afaf2 100644
--- a/sound/usb/usx2y/usX2Yhwdep.c
+++ b/sound/usb/usx2y/usX2Yhwdep.c
@@ -179,6 +179,9 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
int lret, err;
char *buf;
+ if (priv->chip_status & USX2Y_STAT_CHIP_HUP)
+ return -ENODEV;
+
buf = memdup_user(dsp->image, dsp->length);
if (IS_ERR(buf))
return PTR_ERR(buf);
@@ -201,11 +204,13 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
err = usx2y_in04_init(priv);
if (err) {
dev_err(&dev->dev, "usx2y_in04_init error\n");
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
err = usx2y_create_alsa_devices(hw->card);
if (err) {
dev_err(&dev->dev, "usx2y_create_alsa_devices error %i\n", err);
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
priv->chip_status |= USX2Y_STAT_CHIP_INIT;
diff --git a/sound/usb/usx2y/usb_stream.c b/sound/usb/usx2y/usb_stream.c
index 3122cf653273..7d621280fba3 100644
--- a/sound/usb/usx2y/usb_stream.c
+++ b/sound/usb/usx2y/usb_stream.c
@@ -682,6 +682,10 @@ int usb_stream_start(struct usb_stream_kernel *sk)
s->sync_packet = -1;
s->inpacket_head = -1;
sk->iso_frame_balance = 0;
+ sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
+ sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
+ sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
+ sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
++try;
for (u = 0; u < 2; u++) {
struct urb *inurb = sk->inurb[u];
@@ -739,11 +743,6 @@ int usb_stream_start(struct usb_stream_kernel *sk)
return -EFAULT;
}
- sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
- sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
- sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
- sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
-
/* wait, check */
{
int wait_ms = 3000;
diff --git a/sound/usb/usx2y/usbusx2yaudio.c b/sound/usb/usx2y/usbusx2yaudio.c
index 147dd0680528..bd803874412b 100644
--- a/sound/usb/usx2y/usbusx2yaudio.c
+++ b/sound/usb/usx2y/usbusx2yaudio.c
@@ -51,6 +51,18 @@ static int usx2y_urb_capt_retire(struct snd_usx2y_substream *subs)
dev_dbg(&usx2y->dev->dev, "%s: 0 == len ERROR!\n", __func__);
continue;
}
+ /*
+ * snd_usx2y_pcm_open() constrains the buffer to hold at least
+ * two URBs, so this cannot happen for a device that keeps its
+ * packets at 50 frames. Drop the packet rather than exceed the
+ * single wrap below.
+ */
+ if (unlikely(len > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i frames exceeds %lu-frame buffer, dropping\n",
+ __func__, len, runtime->buffer_size);
+ continue;
+ }
/* copy a data chunk */
if ((hwptr_done + len) > runtime->buffer_size) {
@@ -106,14 +118,31 @@ static int usx2y_urb_play_prepare(struct snd_usx2y_substream *subs,
__func__, counts);
return -EPIPE;
}
+ if (counts * usx2y->stride > subs->maxpacksize) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i bytes exceeds maxpacksize %u\n",
+ __func__, counts * usx2y->stride,
+ subs->maxpacksize);
+ return -EPIPE;
+ }
/* set up descriptor */
urb->iso_frame_desc[pack].offset = pack ?
urb->iso_frame_desc[pack - 1].offset +
urb->iso_frame_desc[pack - 1].length :
0;
- urb->iso_frame_desc[pack].length = cap_urb->iso_frame_desc[pack].actual_length;
+ urb->iso_frame_desc[pack].length = counts * usx2y->stride;
}
if (atomic_read(&subs->state) >= STATE_PRERUNNING) {
+ /*
+ * The copy below wraps once, which needs count <= buffer_size.
+ * snd_usx2y_pcm_open() guarantees it; guard anyway.
+ */
+ if (unlikely(count > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: URB of %i frames exceeds %lu-frame buffer, stopping\n",
+ __func__, count, runtime->buffer_size);
+ return -EPIPE;
+ }
if (subs->hwptr + count > runtime->buffer_size) {
/* err, the transferred area goes over buffer boundary.
* copy the data to the temp buffer.
@@ -897,6 +926,18 @@ static int snd_usx2y_pcm_open(struct snd_pcm_substream *substream)
runtime->private_data = subs;
subs->pcm_substream = substream;
snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_PERIOD_TIME, 1000, 200000);
+ /*
+ * Capture wraps the ring buffer once per packet and playback once
+ * per URB of nr_of_packs() packets of at most 50 frames (see
+ * usx2y_urb_play_prepare()). Playback needs the buffer to hold more
+ * than one URB and capture more than one packet; use the URB bound
+ * for both, and two URBs rather than one, because the position only
+ * advances when a URB is less than the whole buffer. The bound is in
+ * frames and does not depend on the altsetting, which is selected
+ * in prepare.
+ */
+ snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_BUFFER_SIZE,
+ 2 * nr_of_packs() * 50, UINT_MAX);
return 0;
}
diff --git a/sound/x86/intel_hdmi_audio.c b/sound/x86/intel_hdmi_audio.c
index 571a092b128a..3c8f1793522a 100644
--- a/sound/x86/intel_hdmi_audio.c
+++ b/sound/x86/intel_hdmi_audio.c
@@ -479,7 +479,8 @@ static void had_build_channel_allocation_map(struct snd_intelhad *intelhaddata)
kfree(intelhaddata->chmap->chmap);
intelhaddata->chmap->chmap = NULL;
- chmap = kzalloc(sizeof(*chmap), GFP_KERNEL);
+ /* info->chmap must be zero-terminated; allocate a spare terminator entry */
+ chmap = kzalloc_objs(*chmap, 2);
if (!chmap)
return;
@@ -582,6 +583,12 @@ static int had_register_chmap_ctls(struct snd_intelhad *intelhaddata,
intelhaddata->chmap->kctl->info = had_chmap_ctl_info;
intelhaddata->chmap->kctl->get = had_chmap_ctl_get;
intelhaddata->chmap->chmap = NULL;
+
+ /* drop the inherited TLV read: it walks chmap without our mutex */
+ intelhaddata->chmap->kctl->tlv.c = NULL;
+ intelhaddata->chmap->kctl->vd[0].access &=
+ ~(SNDRV_CTL_ELEM_ACCESS_TLV_READ | SNDRV_CTL_ELEM_ACCESS_TLV_CALLBACK);
+
return 0;
}
diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/include/asm/cpufeatures.h
index dd4682857c12..f66ac38c9f6f 100644
--- a/tools/arch/x86/include/asm/cpufeatures.h
+++ b/tools/arch/x86/include/asm/cpufeatures.h
@@ -473,6 +473,11 @@
#define X86_FEATURE_CLEAR_BHB_HW (21*32+ 3) /* BHI_DIS_S HW control enabled */
#define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */
#define X86_FEATURE_FAST_CPPC (21*32 + 5) /* AMD Fast CPPC */
+/*
+ * Upstream defines X86_FEATURE_ZEN6 as (3*32+6). That bit is still
+ * X86_FEATURE_P3 in this kernel, so use a free bit in this word.
+ */
+#define X86_FEATURE_ZEN6 (21*32+7) /* CPU based on Zen6 microarchitecture */
/*
* BUG word(s)
diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh
index 25a4ad02ac16..3a9d3d8419d4 100755
--- a/tools/testing/selftests/net/fib_nexthops.sh
+++ b/tools/testing/selftests/net/fib_nexthops.sh
@@ -421,6 +421,14 @@ stop_ip_monitor()
return $rc
}
+fdb_entry_aged()
+{
+ local mac=$1
+ local dev=$2
+
+ ! $BRIDGE fdb get to $mac dev $dev self &> /dev/null
+}
+
check_nexthop_fdb_support()
{
$IP nexthop help 2>&1 | grep -q fdb
@@ -505,14 +513,18 @@ ipv6_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 2001:db8:91::11 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66"
log_test $? 2 "Route add with fdb nexthop"
@@ -599,14 +611,18 @@ ipv4_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 10.0.0.3 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP ro add 172.16.0.0/22 nhid 16"
log_test $? 2 "Route add with fdb nexthop"
diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h
index 0a95dbb0346f..213e40afa8ee 100644
--- a/tools/testing/vma/vma_internal.h
+++ b/tools/testing/vma/vma_internal.h
@@ -605,6 +605,8 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(struct vm_area_struct *vma,
return true;
}
+#define vma_has_uffd_without_event_remap(vma) false
+
static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1,
struct anon_vma_name *anon_name2)
{
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 2e9b5129b5a2..4f69260cafbb 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -472,15 +472,13 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args)
return __kvm_gmem_create(kvm, size, flags);
}
-int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
- unsigned int fd, u64 offset)
+int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot,
+ unsigned int fd, u64 offset)
{
u64 size = slot->npages << PAGE_SHIFT;
- unsigned long start, end;
struct kvm_gmem *gmem;
struct inode *inode;
struct file *file;
- int r = -EINVAL;
BUILD_BUG_ON(sizeof(gpa_t) != sizeof(offset));
BUILD_BUG_ON(sizeof(gfn_t) != sizeof(slot->gmem.pgoff));
@@ -501,90 +499,91 @@ int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
if (!PAGE_ALIGNED(offset) || offset + size > i_size_read(inode))
goto err;
- filemap_invalidate_lock(inode->i_mapping);
-
- start = offset >> PAGE_SHIFT;
- end = start + slot->npages;
-
- if (!xa_empty(&gmem->bindings) &&
- xa_find(&gmem->bindings, &start, end - 1, XA_PRESENT)) {
- filemap_invalidate_unlock(inode->i_mapping);
- goto err;
- }
-
/*
* memslots of flag KVM_MEM_GUEST_MEMFD are immutable to change, so
* kvm_gmem_bind() must occur on a new memslot. Because the memslot
* is not visible yet, kvm_gmem_get_pfn() is guaranteed to see the file.
*/
WRITE_ONCE(slot->gmem.file, file);
- slot->gmem.pgoff = start;
-
- r = xa_err(xa_store_range(&gmem->bindings, start, end - 1, slot, GFP_KERNEL));
- if (r) {
- xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
- slot->gmem.file = NULL;
- slot->gmem.pgoff = 0;
- }
- filemap_invalidate_unlock(inode->i_mapping);
+ slot->gmem.pgoff = offset >> PAGE_SHIFT;
/*
- * Drop the reference to the file, even on success. The file pins KVM,
- * not the other way 'round. Active bindings are invalidated if the
- * file is closed before memslots are destroyed.
+ * Gift the caller a reference to the file. The reference will be
+ * dropped after bindings are established, or if installing the new
+ * memslot ultimately fails.
*/
+ return 0;
+
err:
fput(file);
- return r;
+ return -EINVAL;
}
-static void __kvm_gmem_unbind(struct kvm_memory_slot *slot, struct kvm_gmem *gmem)
+int kvm_gmem_commit_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot)
{
- unsigned long start = slot->gmem.pgoff;
- unsigned long end = start + slot->npages;
+ struct kvm_gmem *gmem = slot->gmem.file->private_data;
+ struct inode *inode = file_inode(slot->gmem.file);
+ unsigned long start, end;
+ int r;
- xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
+ if (WARN_ON_ONCE(slot->gmem.file->f_op != &kvm_gmem_fops))
+ return -EIO;
- /*
- * synchronize_srcu(&kvm->srcu) ensured that kvm_gmem_get_pfn()
- * cannot see this memslot.
- */
- WRITE_ONCE(slot->gmem.file, NULL);
+ filemap_invalidate_lock(inode->i_mapping);
+
+ start = slot->gmem.pgoff;
+ end = start + slot->npages;
+
+ if (!xa_empty(&gmem->bindings) &&
+ xa_find(&gmem->bindings, &start, end - 1, XA_PRESENT)) {
+ filemap_invalidate_unlock(inode->i_mapping);
+ return -EEXIST;
+ }
+
+ r = xa_err(xa_store_range(&gmem->bindings, start, end - 1, slot, GFP_KERNEL));
+ if (r)
+ xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
+
+ filemap_invalidate_unlock(inode->i_mapping);
+
+ return r;
}
void kvm_gmem_unbind(struct kvm_memory_slot *slot)
{
- struct file *file;
+ struct file *file = slot->gmem.file;
+ unsigned long start = slot->gmem.pgoff;
+ unsigned long end = start + slot->npages;
+ struct kvm_gmem *gmem;
/*
* Nothing to do if the underlying file was _already_ closed, as
* kvm_gmem_release() invalidates and nullifies all bindings.
*/
- if (!slot->gmem.file)
+ if (!file)
return;
- file = kvm_gmem_get_file(slot);
-
/*
* However, if the file is _being_ closed, then the bindings need to be
* removed as kvm_gmem_release() might not run until after the memslot
- * is freed. Note, modifying the bindings is safe even though the file
- * is dying as kvm_gmem_release() nullifies slot->gmem.file under
- * slots_lock, and only puts its reference to KVM after destroying all
- * bindings. I.e. reaching this point means kvm_gmem_release() hasn't
- * yet destroyed the bindings or freed the gmem_file, and can't do so
- * until the caller drops slots_lock.
+ * is freed. Modifying the bindings is safe even if the file is dying
+ * as kvm_gmem_release() nullifies slot->gmem.file under slots_lock,
+ * and only puts its reference to KVM after destroying all bindings.
+ * I.e. reaching this point means kvm_gmem_release() hasn't destroyed
+ * the bindings or freed the gmem_file and can't do so until the caller
+ * drops slots_lock, so there's no need to verify the file is live.
*/
- if (!file) {
- __kvm_gmem_unbind(slot, slot->gmem.file->private_data);
- return;
- }
+ gmem = file->private_data;
filemap_invalidate_lock(file->f_mapping);
- __kvm_gmem_unbind(slot, file->private_data);
- filemap_invalidate_unlock(file->f_mapping);
+ xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL);
- fput(file);
+ /*
+ * synchronize_srcu(&kvm->srcu) ensured that kvm_gmem_get_pfn()
+ * cannot see this memslot.
+ */
+ WRITE_ONCE(slot->gmem.file, NULL);
+ filemap_invalidate_unlock(file->f_mapping);
}
/* Returns a locked folio on success. */
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 7debedc1ddaa..3fac4fe389ce 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1725,6 +1725,12 @@ static void kvm_commit_memory_region(struct kvm *kvm,
kvm_free_memslot(kvm, old);
break;
case KVM_MR_MOVE:
+ /*
+ * Moving a guest_memfd memslot isn't supported, and will never
+ * be supported.
+ */
+ WARN_ON_ONCE(old->flags & KVM_MEM_GUEST_MEMFD);
+ fallthrough;
case KVM_MR_FLAGS_ONLY:
/*
* Free the dirty bitmap as needed; the below check encompasses
@@ -1733,6 +1739,15 @@ static void kvm_commit_memory_region(struct kvm *kvm,
if (old->dirty_bitmap && !new->dirty_bitmap)
kvm_destroy_dirty_bitmap(old);
+ /*
+ * TODO: Drop the WARN and do the unbind() call only for MOVE
+ * when dirty logging guest_memfd memslots is supported. Until
+ * then, flags-only changes on guest_memfd slots should also be
+ * impossible; unbind the old memslot for defense-in-depth.
+ */
+ if (WARN_ON_ONCE(old->flags & KVM_MEM_GUEST_MEMFD))
+ kvm_gmem_unbind(old);
+
/*
* The final quirk. Free the detached, old slot, but only its
* memory, not any metadata. Metadata, including arch specific
@@ -1917,6 +1932,15 @@ static int kvm_set_memslot(struct kvm *kvm,
if (r)
goto err;
+ if (change == KVM_MR_CREATE && (new->flags & KVM_MEM_GUEST_MEMFD)) {
+ r = kvm_gmem_commit_memory_region(kvm, new);
+ if (r) {
+ kvm_arch_free_memslot(kvm, new);
+ kvm_destroy_dirty_bitmap(new);
+ goto err;
+ }
+ }
+
/*
* For DELETE and MOVE, the working slot is now active as the INVALID
* version of the old slot. MOVE is particularly special as it reuses
@@ -2087,21 +2111,30 @@ int __kvm_set_memory_region(struct kvm *kvm,
new->npages = npages;
new->flags = mem->flags;
new->userspace_addr = mem->userspace_addr;
- if (mem->flags & KVM_MEM_GUEST_MEMFD) {
- r = kvm_gmem_bind(kvm, new, mem->guest_memfd, mem->guest_memfd_offset);
+ if (change == KVM_MR_CREATE && (mem->flags & KVM_MEM_GUEST_MEMFD)) {
+ r = kvm_gmem_prepare_memory_region(kvm, new, mem->guest_memfd,
+ mem->guest_memfd_offset);
if (r)
goto out;
}
r = kvm_set_memslot(kvm, old, new, change);
+
+ /*
+ * Drop the reference to the gmem file, even on success. The file pins
+ * KVM, not the other way 'round. Active bindings are invalidated if
+ * the file is closed before memslots are destroyed.
+ */
+#ifdef CONFIG_KVM_PRIVATE_MEM
+ if (change == KVM_MR_CREATE && (mem->flags & KVM_MEM_GUEST_MEMFD))
+ fput(new->gmem.file);
+#endif
+
if (r)
- goto out_unbind;
+ goto out;
return 0;
-out_unbind:
- if (mem->flags & KVM_MEM_GUEST_MEMFD)
- kvm_gmem_unbind(new);
out:
kfree(new);
return r;
diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h
index 15b2c728f7dd..92ec2bde6532 100644
--- a/virt/kvm/kvm_mm.h
+++ b/virt/kvm/kvm_mm.h
@@ -41,8 +41,9 @@ static inline void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm,
#ifdef CONFIG_KVM_PRIVATE_MEM
void kvm_gmem_init(struct module *module);
int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args);
-int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
- unsigned int fd, u64 offset);
+int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot,
+ unsigned int fd, u64 offset);
+int kvm_gmem_commit_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot);
void kvm_gmem_unbind(struct kvm_memory_slot *slot);
#else
static inline void kvm_gmem_init(struct module *module)
@@ -50,9 +51,16 @@ static inline void kvm_gmem_init(struct module *module)
}
-static inline int kvm_gmem_bind(struct kvm *kvm,
- struct kvm_memory_slot *slot,
- unsigned int fd, u64 offset)
+static inline int kvm_gmem_prepare_memory_region(struct kvm *kvm,
+ struct kvm_memory_slot *slot,
+ unsigned int fd, u64 offset)
+{
+ WARN_ON_ONCE(1);
+ return -EIO;
+}
+
+static inline int kvm_gmem_commit_memory_region(struct kvm *kvm,
+ struct kvm_memory_slot *slot)
{
WARN_ON_ONCE(1);
return -EIO;
prev parent reply other threads:[~2026-11-17 10:50 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 10:45 Linux 6.12.118 Greg Kroah-Hartman
2026-11-17 10:45 ` Greg Kroah-Hartman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2026111724-snowbound-swear-ef83@gregkh \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=lwn@lwn.net \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox