From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
torvalds@linux-foundation.org, stable@vger.kernel.org
Cc: lwn@lwn.net, jslaby@suse.cz,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: Re: Linux 6.12.97
Date: Fri, 24 Jul 2026 17:05:52 +0200 [thread overview]
Message-ID: <2026072452-june-static-438c@gregkh> (raw)
In-Reply-To: <2026072452-designing-partition-f98d@gregkh>
diff --git a/Documentation/admin-guide/pm/cpufreq.rst b/Documentation/admin-guide/pm/cpufreq.rst
index a21369eba034..0c3f7247e3a3 100644
--- a/Documentation/admin-guide/pm/cpufreq.rst
+++ b/Documentation/admin-guide/pm/cpufreq.rst
@@ -503,7 +503,7 @@ This governor exposes the following tunables:
of those tasks above 0 and set this attribute to 1.
``sampling_down_factor``
- Temporary multiplier, between 1 (default) and 100 inclusive, to apply to
+ Temporary multiplier, between 1 (default) and 100000 inclusive, to apply to
the ``sampling_rate`` value if the CPU load goes above ``up_threshold``.
This causes the next execution of the governor's worker routine (after
@@ -573,8 +573,8 @@ This governor exposes the following tunables:
100 (5 by default).
This is how much the frequency is allowed to change in one go. Setting
- it to 0 will cause the default frequency step (5 percent) to be used
- and setting it to 100 effectively causes the governor to periodically
+ it to 0 disables frequency changes by the governor entirely and setting
+ it to 100 effectively causes the governor to periodically
switch the frequency between the ``scaling_min_freq`` and
``scaling_max_freq`` policy limits.
diff --git a/Documentation/core-api/workqueue.rst b/Documentation/core-api/workqueue.rst
index 16f861c9791e..6597577f8c48 100644
--- a/Documentation/core-api/workqueue.rst
+++ b/Documentation/core-api/workqueue.rst
@@ -183,6 +183,12 @@ resources, scheduled and executed.
BH work items cannot sleep. All other features such as delayed queueing,
flushing and canceling are supported.
+``WQ_PERCPU``
+ Work items queued to a per-cpu wq are bound to a specific CPU.
+ This flag is the right choice when cpu locality is important.
+
+ This flag is the complement of ``WQ_UNBOUND``.
+
``WQ_UNBOUND``
Work items queued to an unbound wq are served by the special
worker-pools which host workers which are not bound to any
diff --git a/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml b/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml
index 541325f900a1..01f2afa023f0 100644
--- a/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml
+++ b/Documentation/devicetree/bindings/media/allwinner,sun4i-a10-video-engine.yaml
@@ -63,6 +63,16 @@ properties:
CMA pool to use for buffers allocation instead of the default
CMA pool.
+ # FIXME: This should be made required eventually once every SoC will
+ # have the MBUS declared.
+ interconnects:
+ maxItems: 1
+
+ # FIXME: This should be made required eventually once every SoC will
+ # have the MBUS declared.
+ interconnect-names:
+ const: dma-mem
+
required:
- compatible
- reg
diff --git a/Documentation/devicetree/bindings/net/renesas,ether.yaml b/Documentation/devicetree/bindings/net/renesas,ether.yaml
index 29355ab98569..fa1a9579f8c9 100644
--- a/Documentation/devicetree/bindings/net/renesas,ether.yaml
+++ b/Documentation/devicetree/bindings/net/renesas,ether.yaml
@@ -118,8 +118,7 @@ examples:
#size-cells = <0>;
phy1: ethernet-phy@1 {
- compatible = "ethernet-phy-id0022.1537",
- "ethernet-phy-ieee802.3-c22";
+ compatible = "ethernet-phy-id0022.1537";
reg = <1>;
interrupt-parent = <&irqc0>;
interrupts = <0 IRQ_TYPE_LEVEL_LOW>;
diff --git a/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux-aon.yaml b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux-aon.yaml
index db8224dfba2c..56fb9cf763ef 100644
--- a/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux-aon.yaml
+++ b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux-aon.yaml
@@ -58,6 +58,10 @@ patternProperties:
drive_soc_gpio27_pee6, drive_ao_retention_n_pee2,
drive_vcomp_alert_pee1, drive_hdmi_cec_pgg0 ]
+required:
+ - compatible
+ - reg
+
unevaluatedProperties: false
examples:
diff --git a/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux.yaml b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux.yaml
index f5a3a881dec4..bd305a34eee2 100644
--- a/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux.yaml
+++ b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra234-pinmux.yaml
@@ -115,6 +115,10 @@ patternProperties:
drive_sdmmc1_dat2_pj4, drive_sdmmc1_dat1_pj3,
drive_sdmmc1_dat0_pj2 ]
+required:
+ - compatible
+ - reg
+
unevaluatedProperties: false
examples:
diff --git a/Documentation/devicetree/bindings/regulator/mt6359-regulator.yaml b/Documentation/devicetree/bindings/regulator/mt6359-regulator.yaml
index d6b3b5a5c0b3..5e4b1e741795 100644
--- a/Documentation/devicetree/bindings/regulator/mt6359-regulator.yaml
+++ b/Documentation/devicetree/bindings/regulator/mt6359-regulator.yaml
@@ -18,84 +18,41 @@ patternProperties:
"^buck_v(s1|gpu11|modem|pu|core|s2|pa|proc2|proc1|core_sshub)$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^v(s1|gpu11|modem|pu|core|s2|pa|proc2|proc1|core_sshub)$"
-
unevaluatedProperties: false
"^ldo_v(ibr|rf12|usb|camio|efuse|xo22)$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^v(ibr|rf12|usb|camio|efuse|xo22)$"
-
unevaluatedProperties: false
"^ldo_v(rfck|emc|a12|a09|ufs|bbck)$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^v(rfck|emc|a12|a09|ufs|bbck)$"
-
unevaluatedProperties: false
"^ldo_vcn(18|13|33_1_bt|13_1_wifi|33_2_bt|33_2_wifi)$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^vcn(18|13|33_1_bt|13_1_wifi|33_2_bt|33_2_wifi)$"
-
unevaluatedProperties: false
"^ldo_vsram_(proc2|others|md|proc1|others_sshub)$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^vsram_(proc2|others|md|proc1|others_sshub)$"
-
unevaluatedProperties: false
"^ldo_v(fe|bif|io)28$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^v(fe|bif|io)28$"
-
unevaluatedProperties: false
"^ldo_v(aud|io|aux|rf|m)18$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^v(aud|io|aux|rf|m)18$"
-
unevaluatedProperties: false
"^ldo_vsim[12]$":
type: object
$ref: regulator.yaml#
-
- properties:
- regulator-name:
- pattern: "^vsim[12]$"
-
- required:
- - regulator-name
-
unevaluatedProperties: false
additionalProperties: false
diff --git a/Documentation/driver-api/dpll.rst b/Documentation/driver-api/dpll.rst
index e6855cd37e85..f8cbe7c48f7f 100644
--- a/Documentation/driver-api/dpll.rst
+++ b/Documentation/driver-api/dpll.rst
@@ -235,6 +235,31 @@ the pin.
``DPLL_A_PIN_ESYNC_PULSE`` pulse type of Embedded SYNC
========================================= =================================
+Reference SYNC
+==============
+
+The device may support the Reference SYNC feature, which allows the combination
+of two inputs into a input pair. In this configuration, clock signals
+from both inputs are used to synchronize the DPLL device. The higher frequency
+signal is utilized for the loop bandwidth of the DPLL, while the lower frequency
+signal is used to syntonize the output signal of the DPLL device. This feature
+enables the provision of a high-quality loop bandwidth signal from an external
+source.
+
+A capable input provides a list of inputs that can be bound with to create
+Reference SYNC. To control this feature, the user must request a desired
+state for a target pin: use ``DPLL_PIN_STATE_CONNECTED`` to enable or
+``DPLL_PIN_STATE_DISCONNECTED`` to disable the feature. An input pin can be
+bound to only one other pin at any given time.
+
+ ============================== ==========================================
+ ``DPLL_A_PIN_REFERENCE_SYNC`` nested attribute for providing info or
+ requesting configuration of the Reference
+ SYNC feature
+ ``DPLL_A_PIN_ID`` target pin id for Reference SYNC feature
+ ``DPLL_A_PIN_STATE`` state of Reference SYNC connection
+ ============================== ==========================================
+
Configuration commands group
============================
diff --git a/Documentation/driver-api/nvdimm/btt.rst b/Documentation/driver-api/nvdimm/btt.rst
index 107395c042ae..cb1afe9dc3cb 100644
--- a/Documentation/driver-api/nvdimm/btt.rst
+++ b/Documentation/driver-api/nvdimm/btt.rst
@@ -161,9 +161,8 @@ process::
nlanes = min(nfree, num_cpus)
A lane number is obtained at the start of any IO, and is used for indexing into
-all the on-disk and in-memory data structures for the duration of the IO. If
-there are more CPUs than the max number of available lanes, than lanes are
-protected by spinlocks.
+all the on-disk and in-memory data structures for the duration of the IO. Lanes
+are protected by mutexes.
d. In-memory data structure: Read Tracking Table (RTT)
diff --git a/Documentation/driver-api/pci/pci.rst b/Documentation/driver-api/pci/pci.rst
index aa40b1cc243b..31d5886388d8 100644
--- a/Documentation/driver-api/pci/pci.rst
+++ b/Documentation/driver-api/pci/pci.rst
@@ -37,6 +37,9 @@ PCI Support Library
.. kernel-doc:: drivers/pci/slot.c
:export:
+.. kernel-doc:: drivers/pci/rebar.c
+ :export:
+
.. kernel-doc:: drivers/pci/rom.c
:export:
diff --git a/Documentation/filesystems/proc.rst b/Documentation/filesystems/proc.rst
index e834779d9611..af6cfb45b905 100644
--- a/Documentation/filesystems/proc.rst
+++ b/Documentation/filesystems/proc.rst
@@ -23,13 +23,13 @@ fixes/update part 1.1 Stefani Seibold <stefani@seibold.net> June 9 2009
1 Collecting System Information
1.1 Process-Specific Subdirectories
1.2 Kernel data
- 1.3 IDE devices in /proc/ide
- 1.4 Networking info in /proc/net
- 1.5 SCSI info
- 1.6 Parallel port info in /proc/parport
- 1.7 TTY info in /proc/tty
- 1.8 Miscellaneous kernel statistics in /proc/stat
- 1.9 Ext4 file system parameters
+ 1.3 Networking info in /proc/net
+ 1.4 SCSI info
+ 1.5 Parallel port info in /proc/parport
+ 1.6 TTY info in /proc/tty
+ 1.7 Miscellaneous kernel statistics in /proc/stat
+ 1.8 Ext4 file system parameters
+ 1.9 /proc/consoles - Shows registered system consoles
2 Modifying System Parameters
diff --git a/Documentation/netlink/specs/dpll.yaml b/Documentation/netlink/specs/dpll.yaml
index 860350e61edb..cb5763f79ac2 100644
--- a/Documentation/netlink/specs/dpll.yaml
+++ b/Documentation/netlink/specs/dpll.yaml
@@ -365,6 +365,15 @@ attribute-sets:
doc: |
A ratio of high to low state of a SYNC signal pulse embedded
into base clock frequency. Value is in percents.
+ -
+ name: reference-sync
+ type: nest
+ multi-attr: true
+ nested-attributes: reference-sync
+ doc: |
+ Capable pin provides list of pins that can be bound to create a
+ reference-sync pin pair.
+
-
name: pin-parent-device
subset-of: pin
@@ -395,6 +404,14 @@ attribute-sets:
name: frequency-min
-
name: frequency-max
+ -
+ name: reference-sync
+ subset-of: pin
+ attributes:
+ -
+ name: id
+ -
+ name: state
operations:
enum-name: dpll_cmd
@@ -535,6 +552,7 @@ operations:
- esync-frequency
- esync-frequency-supported
- esync-pulse
+ - reference-sync
dump:
request:
@@ -562,6 +580,7 @@ operations:
- parent-pin
- phase-adjust
- esync-frequency
+ - reference-sync
-
name: pin-create-ntf
doc: Notification about pin appearing
diff --git a/Documentation/netlink/specs/handshake.yaml b/Documentation/netlink/specs/handshake.yaml
index 090fc11da460..4edaf9c71439 100644
--- a/Documentation/netlink/specs/handshake.yaml
+++ b/Documentation/netlink/specs/handshake.yaml
@@ -121,6 +121,7 @@ operations:
name: done
doc: Handler reports handshake completion
attribute-set: done
+ flags: [admin-perm]
do:
request:
attributes:
diff --git a/Documentation/process/deprecated.rst b/Documentation/process/deprecated.rst
index 1f7f3e6c9cda..fed56864d036 100644
--- a/Documentation/process/deprecated.rst
+++ b/Documentation/process/deprecated.rst
@@ -372,3 +372,34 @@ The helper must be used::
DECLARE_FLEX_ARRAY(struct type2, two);
};
};
+
+Open-coded kmalloc assignments for struct objects
+-------------------------------------------------
+Performing open-coded kmalloc()-family allocation assignments prevents
+the kernel (and compiler) from being able to examine the type of the
+variable being assigned, which limits any related introspection that
+may help with alignment, wrap-around, or additional hardening. The
+kmalloc_obj()-family of macros provide this introspection, which can be
+used for the common code patterns for single, array, and flexible object
+allocations. For example, these open coded assignments::
+
+ ptr = kmalloc(sizeof(*ptr), gfp);
+ ptr = kzalloc(sizeof(*ptr), gfp);
+ ptr = kmalloc_array(count, sizeof(*ptr), gfp);
+ ptr = kcalloc(count, sizeof(*ptr), gfp);
+ ptr = kmalloc(struct_size(ptr, flex_member, count), gfp);
+ ptr = kmalloc(sizeof(struct foo, gfp);
+
+become, respectively::
+
+ ptr = kmalloc_obj(*ptr, gfp);
+ ptr = kzalloc_obj(*ptr, gfp);
+ ptr = kmalloc_objs(*ptr, count, gfp);
+ ptr = kzalloc_objs(*ptr, count, gfp);
+ ptr = kmalloc_flex(*ptr, flex_member, count, gfp);
+ __auto_type ptr = kmalloc_obj(struct foo, gfp);
+
+If `ptr->flex_member` is annotated with __counted_by(), the allocation
+will automatically fail if `count` is larger than the maximum
+representable value that can be stored in the counter member associated
+with `flex_member`.
diff --git a/Makefile b/Makefile
index 95520f8d48d3..2eda2cddaace 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 6
PATCHLEVEL = 12
-SUBLEVEL = 96
+SUBLEVEL = 97
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/alpha/kernel/pci-sysfs.c b/arch/alpha/kernel/pci-sysfs.c
index 5808a66e2a81..22187531abb9 100644
--- a/arch/alpha/kernel/pci-sysfs.c
+++ b/arch/alpha/kernel/pci-sysfs.c
@@ -11,6 +11,7 @@
*/
#include <linux/sched.h>
+#include <linux/security.h>
#include <linux/stat.h>
#include <linux/slab.h>
#include <linux/pci.h>
@@ -36,12 +37,16 @@ static int hose_mmap_page_range(struct pci_controller *hose,
static int __pci_mmap_fits(struct pci_dev *pdev, int num,
struct vm_area_struct *vma, int sparse)
{
+ resource_size_t len = pci_resource_len(pdev, num);
unsigned long nr, start, size;
int shift = sparse ? 5 : 0;
+ if (!len)
+ return 0;
+
nr = vma_pages(vma);
start = vma->vm_pgoff;
- size = ((pci_resource_len(pdev, num) - 1) >> (PAGE_SHIFT - shift)) + 1;
+ size = ((len - 1) >> (PAGE_SHIFT - shift)) + 1;
if (start < size && size - start >= nr)
return 1;
@@ -71,7 +76,11 @@ static int pci_mmap_resource(struct kobject *kobj,
struct resource *res = attr->private;
enum pci_mmap_state mmap_type;
struct pci_bus_region bar;
- int i;
+ int i, ret;
+
+ ret = security_locked_down(LOCKDOWN_PCI_ACCESS);
+ if (ret)
+ return ret;
for (i = 0; i < PCI_STD_NUM_BARS; i++)
if (res == &pdev->resource[i])
diff --git a/arch/arm/boot/dts/st/stm32mp151c-mecio1r0.dts b/arch/arm/boot/dts/st/stm32mp151c-mecio1r0.dts
index a5ea1431c399..862782d20d10 100644
--- a/arch/arm/boot/dts/st/stm32mp151c-mecio1r0.dts
+++ b/arch/arm/boot/dts/st/stm32mp151c-mecio1r0.dts
@@ -36,6 +36,56 @@ led-1 {
};
};
+&adc1 {
+ channel@0 {
+ reg = <0>;
+ st,min-sample-time-ns = <20000>;
+ label = "ain_aux0";
+ };
+
+ channel@1 {
+ reg = <1>;
+ st,min-sample-time-ns = <20000>;
+ label = "ain_aux1";
+ };
+
+ channel@3 {
+ reg = <3>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpdcm1_i2";
+ };
+
+ channel@5 {
+ reg = <5>;
+ st,min-sample-time-ns = <20000>;
+ label = "pout1_i";
+ };
+
+ channel@9 {
+ reg = <9>;
+ st,min-sample-time-ns = <20000>;
+ label = "pout0_i";
+ };
+
+ channel@13 {
+ reg = <13>;
+ st,min-sample-time-ns = <200000>;
+ label = "phint1_ain";
+ };
+
+ channel@15 {
+ reg = <15>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpdcm0_i1";
+ };
+
+ channel@18 {
+ reg = <18>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpdcm0_i2";
+ };
+};
+
&clk_hse {
clock-frequency = <25000000>;
};
@@ -46,3 +96,81 @@ ðernet0 {
assigned-clock-rates = <125000000>; /* Clock PLL3 to 625Mhz in tf-a. */
st,eth-clk-sel;
};
+
+&gpio0 {
+ gpio-line-names = "HSIN0_BIAS", "HSIN1_BIAS", "HSIN2_BIAS", "HSIN3_BIAS",
+ "HSIN4_BIAS", "", "STP_VREF0_LVL", "HSIN_VREF0_LVL",
+ "STP0_FB_BIAS", "STP1_FB_BIAS", "STP2_FB_BIAS", "STP3_FB_BIAS",
+ "", "", "", "";
+};
+
+&gpio1 {
+ gpio-line-names = "HSIN5_BIAS", "HSIN6_BIAS", "HSIN7_BIAS", "HSIN8_BIAS",
+ "HSIN9_BIAS", "", "STP_VREF1_LVL", "HSIN_VREF1_LVL",
+ "STP4_FB_BIAS", "STP5_FB_BIAS", "STP6_FB_BIAS", "",
+ "", "", "LSIN8_BIAS", "LSIN9_BIAS";
+};
+
+&gpiod {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "",
+ "", "", "", "",
+ "STP_RESETN", "STP_ENABLEN", "HPOUT0", "HPOUT0_ALERTN";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_hog_d_mecsbc>;
+};
+
+&gpioe {
+ gpio-line-names = "HPOUT0_RESETN", "HPOUT1", "HPOUT1_ALERTN", "",
+ "", "", "HPOUT1_RESETN",
+ "LPOUT0", "LPOUT0_ALERTN", "LPOUT0_RESETN",
+ "LPOUT1", "LPOUT1_ALERTN", "LPOUT1_RESETN",
+ "LPOUT2", "LPOUT2_ALERTN", "LPOUT2_RESETN";
+};
+
+&gpiof {
+ gpio-line-names = "LPOUT3", "LPOUT3_ALERTN", "LPOUT3_RESETN",
+ "LPOUT4", "LPOUT4_ALERTN", "LPOUT4_RESETN",
+ "", "",
+ "", "", "", "",
+ "", "", "", "";
+};
+
+&gpiog {
+ gpio-line-names = "LPOUT5", "LPOUT5_ALERTN", "", "LPOUT5_RESETN",
+ "", "", "", "",
+ "", "", "", "",
+ "", "", "", "";
+};
+
+&gpioh {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "",
+ "GPIO0_RESETN", "", "", "",
+ "", "", "", "";
+};
+
+&gpioi {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "",
+ "HPDCM0_SLEEPN", "HPDCM1_SLEEPN", "GPIO1_RESETN", "",
+ "", "", "", "";
+};
+
+&gpioj {
+ gpio-line-names = "HSIN10", "HSIN11", "HSIN12", "HSIN13",
+ "HSIN14", "HSIN15", "", "",
+ "", "", "", "",
+ "", "RTD_RESETN", "", "";
+};
+
+&gpiok {
+ gpio-line-names = "", "", "HSIN0", "HSIN1",
+ "HSIN2", "HSIN3", "HSIN4", "HSIN5";
+};
+
+&gpioz {
+ gpio-line-names = "", "", "", "HSIN6",
+ "HSIN7", "HSIN8", "HSIN9", "";
+};
+
diff --git a/arch/arm/boot/dts/st/stm32mp153c-mecio1r1.dts b/arch/arm/boot/dts/st/stm32mp153c-mecio1r1.dts
index 16b814c19350..739cc18c3d3a 100644
--- a/arch/arm/boot/dts/st/stm32mp153c-mecio1r1.dts
+++ b/arch/arm/boot/dts/st/stm32mp153c-mecio1r1.dts
@@ -36,10 +36,154 @@ led-1 {
};
};
+&adc1 {
+ channel@0 {
+ reg = <0>;
+ st,min-sample-time-ns = <20000>;
+ label = "p24v_hpdcm";
+ };
+
+ channel@1 {
+ reg = <1>;
+ st,min-sample-time-ns = <20000>;
+ label = "p24v_stp";
+ };
+
+ channel@3 {
+ reg = <3>;
+ st,min-sample-time-ns = <200000>;
+ label = "phint1_ain";
+ };
+
+ channel@5 {
+ reg = <5>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpout1_i";
+ };
+
+ channel@9 {
+ reg = <9>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpout0_i";
+ };
+
+ channel@13 {
+ reg = <13>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpdcm0_i2";
+ };
+
+ channel@15 {
+ reg = <15>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpdcm1_i2";
+ };
+
+ channel@18 {
+ reg = <18>;
+ st,min-sample-time-ns = <20000>;
+ label = "hpdcm0_i1";
+ };
+};
+
&clk_hse {
clock-frequency = <24000000>;
};
+&gpio0 {
+ gpio-line-names = "HSIN0_BIAS", "HSIN1_BIAS", "HSIN2_BIAS", "HSIN3_BIAS",
+ "", "", "HSIN_VREF0_LVL", "HSIN_VREF1_LVL",
+ "HSIN4_BIAS", "HSIN5_BIAS", "HSIN6_BIAS", "HSIN7_BIAS",
+ "", "", "", "";
+};
+
+&gpio1 {
+ gpio-line-names = "HSIN8_BIAS", "HSIN9_BIAS", "HSIN10_BIAS", "HSIN11_BIAS",
+ "", "", "HSIN_VREF2_LVL", "HSIN_VREF3_LVL",
+ "HSIN12_BIAS", "HSIN13_BIAS", "HSIN14_BIAS", "HSIN15_BIAS",
+ "", "", "LSIN8_BIAS", "LSIN9_BIAS";
+};
+
+&gpioa {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "",
+ "", "", "GPIO1_RESETN", "",
+ "", "", "", "LPOUT5";
+};
+
+&gpiob {
+ gpio-line-names = "", "", "", "",
+ "LPOUT4_RESETN", "", "", "",
+ "", "LPOUT4_ALERTN", "", "",
+ "", "", "", "";
+};
+
+&gpioc {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "",
+ "", "LPOUT4", "", "",
+ "", "", "", "";
+};
+
+&gpiod {
+ gpio-line-names = "LPOUT2", "", "LPOUT3_RESETN", "",
+ "LPOUT2_ALERTN", "", "MECIO_ADDR0", "",
+ "HPOUT1_ALERTN", "HPOUT1_RESETN", "", "",
+ "", "", "HPOUT0", "HPOUT1";
+};
+
+&gpioe {
+ gpio-line-names = "LPOUT0_RESETN", "", "", "",
+ "", "LPOUT3", "LPOUT5_ALERTN", "",
+ "", "", "", "",
+ "", "", "", "HSIN_RESETN";
+};
+
+&gpiof {
+ gpio-line-names = "LPOUT5_RESETN", "", "", "HPOUT0_ALERTN",
+ "", "LPOUT1", "", "",
+ "", "", "", "",
+ "", "", "", "";
+};
+
+&gpiog {
+ gpio-line-names = "", "", "", "HPOUT0_RESETN",
+ "", "", "LPOUT3_ALERTN", "",
+ "", "", "GPIO0_RESETN", "",
+ "", "", "", "LPOUT2_RESETN";
+};
+
+&gpioh {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "",
+ "", "LPOUT0", "", "",
+ "", "LPOUT0_ALERTN", "STP_ENABLEN", "STP_RESETN";
+};
+
+&gpioi {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "",
+ "", "", "SPE_RESETN", "",
+ "HPDCM0_SLEEPN", "", "", "";
+};
+
+&gpioj {
+ gpio-line-names = "", "", "", "",
+ "", "", "", "MECIO_ADDR1",
+ "", "", "", "",
+ "", "", "", "LPOUT1_RESETN";
+};
+
+&gpiok {
+ gpio-line-names = "", "", "RTD_RESETN", "",
+ "", "LPOUT1_ALERTN", "", "";
+};
+
+&gpioz {
+ gpio-line-names = "", "", "", "",
+ "HPDCM1_SLEEPN", "", "", "";
+};
+
&m_can1 {
pinctrl-names = "default", "sleep";
pinctrl-0 = <&m_can1_pins_b>;
diff --git a/arch/arm/boot/dts/st/stm32mp15x-mecio1-io.dtsi b/arch/arm/boot/dts/st/stm32mp15x-mecio1-io.dtsi
index 915ba2526f45..1a4f5a523eb3 100644
--- a/arch/arm/boot/dts/st/stm32mp15x-mecio1-io.dtsi
+++ b/arch/arm/boot/dts/st/stm32mp15x-mecio1-io.dtsi
@@ -90,94 +90,45 @@ v5v: regulator-v5v {
};
&adc {
- /* ANA0, ANA1 are dedicated pins and don't need pinctrl: only in6. */
+ /* ANA0, ANA1 are dedicated pins and don't need pinctrl. */
pinctrl-0 = <&adc12_pins_mecsbc>;
pinctrl-names = "default";
vdd-supply = <&v3v3>;
vdda-supply = <&v3v3>;
- vref-supply = <&v3v3>;
+ vref-supply = <&vrefbuf>;
status = "okay";
};
&adc1 {
status = "okay";
- channel@0 {
- reg = <0>;
- /* 16.5 ck_cycles sampling time */
- st,min-sample-time-ns = <5000>;
- label = "p24v_stp";
- };
-
- channel@1 {
- reg = <1>;
- st,min-sample-time-ns = <5000>;
- label = "p24v_hpdcm";
- };
-
channel@2 {
reg = <2>;
- st,min-sample-time-ns = <5000>;
+ st,min-sample-time-ns = <20000>;
label = "ain0";
};
- channel@3 {
- reg = <3>;
- st,min-sample-time-ns = <5000>;
- label = "hpdcm1_i2";
- };
-
- channel@5 {
- reg = <5>;
- st,min-sample-time-ns = <5000>;
- label = "hpout1_i";
- };
-
channel@6 {
reg = <6>;
- st,min-sample-time-ns = <5000>;
+ st,min-sample-time-ns = <20000>;
label = "ain1";
};
- channel@9 {
- reg = <9>;
- st,min-sample-time-ns = <5000>;
- label = "hpout0_i";
- };
-
channel@10 {
reg = <10>;
- st,min-sample-time-ns = <5000>;
+ st,min-sample-time-ns = <200000>;
label = "phint0_ain";
};
- channel@13 {
- reg = <13>;
- st,min-sample-time-ns = <5000>;
- label = "phint1_ain";
- };
-
- channel@15 {
- reg = <15>;
- st,min-sample-time-ns = <5000>;
- label = "hpdcm0_i1";
- };
-
channel@16 {
reg = <16>;
- st,min-sample-time-ns = <5000>;
+ st,min-sample-time-ns = <20000>;
label = "lsin";
};
- channel@18 {
- reg = <18>;
- st,min-sample-time-ns = <5000>;
- label = "hpdcm0_i2";
- };
-
channel@19 {
reg = <19>;
- st,min-sample-time-ns = <5000>;
+ st,min-sample-time-ns = <20000>;
label = "hpdcm1_i1";
};
};
@@ -187,14 +138,13 @@ &adc2 {
channel@2 {
reg = <2>;
- /* 16.5 ck_cycles sampling time */
- st,min-sample-time-ns = <5000>;
+ st,min-sample-time-ns = <20000>;
label = "ain2";
};
channel@6 {
reg = <6>;
- st,min-sample-time-ns = <5000>;
+ st,min-sample-time-ns = <20000>;
label = "ain3";
};
};
@@ -223,69 +173,6 @@ phy0: ethernet-phy@8 {
};
};
-&gpiod {
- gpio-line-names = "", "", "", "",
- "", "", "", "",
- "", "", "", "",
- "STP_RESETN", "STP_ENABLEN", "HPOUT0", "HPOUT0_ALERTN";
- pinctrl-names = "default";
- pinctrl-0 = <&pinctrl_hog_d_mecsbc>;
-};
-
-&gpioe {
- gpio-line-names = "HPOUT0_RESETN", "HPOUT1", "HPOUT1_ALERTN", "",
- "", "", "HPOUT1_RESETN",
- "LPOUT0", "LPOUT0_ALERTN", "GPOUT0_RESETN",
- "LPOUT1", "LPOUT1_ALERTN", "GPOUT1_RESETN",
- "LPOUT2", "LPOUT2_ALERTN", "GPOUT2_RESETN";
-};
-
-&gpiof {
- gpio-line-names = "LPOUT3", "LPOUT3_ALERTN", "GPOUT3_RESETN",
- "LPOUT4", "LPOUT4_ALERTN", "GPOUT4_RESETN",
- "", "",
- "", "", "", "",
- "", "", "", "";
-};
-
-&gpiog {
- gpio-line-names = "LPOUT5", "LPOUT5_ALERTN", "", "LPOUT5_RESETN",
- "", "", "", "",
- "", "", "", "",
- "", "", "", "";
-};
-
-&gpioh {
- gpio-line-names = "", "", "", "",
- "", "", "", "",
- "GPIO0_RESETN", "", "", "",
- "", "", "", "";
-};
-
-&gpioi {
- gpio-line-names = "", "", "", "",
- "", "", "", "",
- "HPDCM0_SLEEPN", "HPDCM1_SLEEPN", "GPIO1_RESETN", "",
- "", "", "", "";
-};
-
-&gpioj {
- gpio-line-names = "HSIN10", "HSIN11", "HSIN12", "HSIN13",
- "HSIN14", "HSIN15", "", "",
- "", "", "", "",
- "", "RTD_RESETN", "", "";
-};
-
-&gpiok {
- gpio-line-names = "", "", "HSIN0", "HSIN1",
- "HSIN2", "HSIN3", "HSIN4", "HSIN5";
-};
-
-&gpioz {
- gpio-line-names = "", "", "", "HSIN6",
- "HSIN7", "HSIN8", "HSIN9", "";
-};
-
&i2c2 {
pinctrl-names = "default";
pinctrl-0 = <&i2c2_pins_a>;
@@ -297,10 +184,6 @@ gpio0: gpio@20 {
reg = <0x20>;
gpio-controller;
#gpio-cells = <2>;
- gpio-line-names = "HSIN0_BIAS", "HSIN1_BIAS", "HSIN2_BIAS", "HSIN3_BIAS",
- "", "", "HSIN_VREF0_LVL", "HSIN_VREF1_LVL",
- "HSIN4_BIAS", "HSIN5_BIAS", "HSIN6_BIAS", "HSIN9_BIAS",
- "", "", "", "";
};
gpio1: gpio@21 {
@@ -308,10 +191,6 @@ gpio1: gpio@21 {
reg = <0x21>;
gpio-controller;
#gpio-cells = <2>;
- gpio-line-names = "HSIN8_BIAS", "HSIN9_BIAS", "HSIN10_BIAS", "HSIN11_BIAS",
- "", "", "HSIN_VREF2_LVL", "HSIN_VREF3_LVL",
- "HSIN12_BIAS", "HSIN13_BIAS", "HSIN14_BIAS", "HSIN15_BIAS",
- "", "", "LSIN8_BIAS", "LSIN9_BIAS";
};
};
@@ -412,6 +291,13 @@ &usbphyc_port1 {
phy-supply = <&v3v3>;
};
+&vrefbuf {
+ regulator-min-microvolt = <2500000>;
+ regulator-max-microvolt = <2500000>;
+ vdda-supply = <&v3v3>;
+ status = "okay";
+};
+
&pinctrl {
adc12_pins_mecsbc: adc12-ain-mecsbc-0 {
pins {
diff --git a/arch/arm/boot/dts/ti/omap/am335x-sl50.dts b/arch/arm/boot/dts/ti/omap/am335x-sl50.dts
index 757ebd96b3f0..ccbde00d719c 100644
--- a/arch/arm/boot/dts/ti/omap/am335x-sl50.dts
+++ b/arch/arm/boot/dts/ti/omap/am335x-sl50.dts
@@ -558,8 +558,8 @@ cpu_endpoint: endpoint {
remote-endpoint = <&codec_endpoint>;
dai-format = "dsp_b";
- bitclock-master = <&codec_port>;
- frame-master = <&codec_port>;
+ bitclock-master = <&codec_endpoint>;
+ frame-master = <&codec_endpoint>;
bitclock-inversion;
clocks = <&audio_mclk>;
};
diff --git a/arch/arm/configs/multi_v7_defconfig b/arch/arm/configs/multi_v7_defconfig
index 9a5f5c439b87..61b652cb522e 100644
--- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -1112,7 +1112,7 @@ CONFIG_QCOM_COMMAND_DB=m
CONFIG_QCOM_GSBI=y
CONFIG_QCOM_OCMEM=m
CONFIG_QCOM_RMTFS_MEM=m
-CONFIG_QCOM_RPMH=y
+CONFIG_QCOM_RPMH=m
CONFIG_QCOM_SMEM=y
CONFIG_QCOM_SMD_RPM=y
CONFIG_QCOM_SMP2P=y
@@ -1149,7 +1149,7 @@ CONFIG_KEYSTONE_NAVIGATOR_QMSS=y
CONFIG_KEYSTONE_NAVIGATOR_DMA=y
CONFIG_RASPBERRYPI_POWER=y
CONFIG_QCOM_CPR=y
-CONFIG_QCOM_RPMHPD=y
+CONFIG_QCOM_RPMHPD=m
CONFIG_QCOM_RPMPD=y
CONFIG_ROCKCHIP_PM_DOMAINS=y
CONFIG_TI_SCI_PM_DOMAINS=y
diff --git a/arch/arm/configs/sunxi_defconfig b/arch/arm/configs/sunxi_defconfig
index a83d29fed175..f4b8d8f7dbef 100644
--- a/arch/arm/configs/sunxi_defconfig
+++ b/arch/arm/configs/sunxi_defconfig
@@ -170,7 +170,6 @@ CONFIG_ROOT_NFS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_CRYPTO_DEV_SUN4I_SS=y
-CONFIG_CRYPTO_DEV_SUN4I_SS_PRNG=y
CONFIG_CRYPTO_DEV_SUN8I_CE=y
CONFIG_CRYPTO_DEV_SUN8I_SS=y
CONFIG_DMA_CMA=y
diff --git a/arch/arm/mach-imx/cpu-imx31.c b/arch/arm/mach-imx/cpu-imx31.c
index 35c544924e50..e81ef9e36a1f 100644
--- a/arch/arm/mach-imx/cpu-imx31.c
+++ b/arch/arm/mach-imx/cpu-imx31.c
@@ -36,6 +36,7 @@ static int mx31_read_cpu_rev(void)
void __iomem *iim_base;
struct device_node *np;
u32 i, srev;
+ int rev = IMX_CHIP_REVISION_UNKNOWN;
np = of_find_compatible_node(NULL, NULL, "fsl,imx31-iim");
iim_base = of_iomap(np, 0);
@@ -48,13 +49,17 @@ static int mx31_read_cpu_rev(void)
for (i = 0; i < ARRAY_SIZE(mx31_cpu_type); i++)
if (srev == mx31_cpu_type[i].srev) {
+ rev = mx31_cpu_type[i].rev;
imx_print_silicon_rev(mx31_cpu_type[i].name,
mx31_cpu_type[i].rev);
- return mx31_cpu_type[i].rev;
+ goto out;
}
imx_print_silicon_rev("i.MX31", IMX_CHIP_REVISION_UNKNOWN);
- return IMX_CHIP_REVISION_UNKNOWN;
+
+out:
+ iounmap(iim_base);
+ return rev;
}
int mx31_revision(void)
diff --git a/arch/arm/mach-imx/mm-imx3.c b/arch/arm/mach-imx/mm-imx3.c
index 0788c5cc7f9e..9b0b014d7fe2 100644
--- a/arch/arm/mach-imx/mm-imx3.c
+++ b/arch/arm/mach-imx/mm-imx3.c
@@ -106,6 +106,7 @@ void __init imx31_init_early(void)
arm_pm_idle = imx31_idle;
np = of_find_compatible_node(NULL, NULL, "fsl,imx31-ccm");
mx3_ccm_base = of_iomap(np, 0);
+ of_node_put(np);
BUG_ON(!mx3_ccm_base);
}
#endif /* ifdef CONFIG_SOC_IMX31 */
@@ -143,6 +144,7 @@ void __init imx35_init_early(void)
arch_ioremap_caller = imx3_ioremap_caller;
np = of_find_compatible_node(NULL, NULL, "fsl,imx35-ccm");
mx3_ccm_base = of_iomap(np, 0);
+ of_node_put(np);
BUG_ON(!mx3_ccm_base);
}
#endif /* ifdef CONFIG_SOC_IMX35 */
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mpxl.dts b/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mpxl.dts
index 773282002e6c..2f52e3260886 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mpxl.dts
+++ b/arch/arm64/boot/dts/freescale/imx8mp-tqma8mpql-mba8mpxl.dts
@@ -518,6 +518,9 @@ tlv320aic3x04: audio-codec@18 {
reset-gpios = <&gpio4 29 GPIO_ACTIVE_LOW>;
iov-supply = <®_vcc_1v8>;
ldoin-supply = <®_vcc_3v3>;
+ assigned-clocks = <&clk IMX8MP_CLK_SAI3>;
+ assigned-clock-parents = <&clk IMX8MP_AUDIO_PLL1_OUT>;
+ assigned-clock-rates = <12288000>;
};
se97_1c: temperature-sensor@1c {
diff --git a/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts b/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
index e937e5f8fa8b..54f55fa083fb 100644
--- a/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
+++ b/arch/arm64/boot/dts/freescale/imx8ulp-evk.dts
@@ -130,7 +130,7 @@ &lpi2c7 {
ptn5150_1: typec@1d {
compatible = "nxp,ptn5150";
reg = <0x1d>;
- int-gpios = <&gpiof 3 IRQ_TYPE_EDGE_FALLING>;
+ int-gpios = <&gpiof 3 GPIO_ACTIVE_LOW>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_typec1>;
status = "disabled";
@@ -146,7 +146,7 @@ pcal6408: gpio@21 {
ptn5150_2: typec@3d {
compatible = "nxp,ptn5150";
reg = <0x3d>;
- int-gpios = <&gpiof 5 IRQ_TYPE_EDGE_FALLING>;
+ int-gpios = <&gpiof 5 GPIO_ACTIVE_LOW>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_typec2>;
status = "disabled";
diff --git a/arch/arm64/boot/dts/freescale/imx8x-colibri.dtsi b/arch/arm64/boot/dts/freescale/imx8x-colibri.dtsi
index edba5b582414..0151cb687134 100644
--- a/arch/arm64/boot/dts/freescale/imx8x-colibri.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8x-colibri.dtsi
@@ -577,12 +577,12 @@ pinctrl_adc0: adc0grp {
*/
pinctrl_atmel_adap: atmeladaptergrp {
fsl,pins = <IMX8QXP_UART1_RX_LSIO_GPIO0_IO22 0x21>, /* SODIMM 30 */
- <IMX8QXP_UART1_TX_LSIO_GPIO0_IO21 0x4000021>; /* SODIMM 28 */
+ <IMX8QXP_UART1_TX_LSIO_GPIO0_IO21 0x40000021>; /* SODIMM 28 */
};
/* Atmel MXT touchsceen + boards with built-in Capacitive Touch Connector */
pinctrl_atmel_conn: atmelconnectorgrp {
- fsl,pins = <IMX8QXP_QSPI0B_DATA2_LSIO_GPIO3_IO20 0x4000021>, /* SODIMM 107 */
+ fsl,pins = <IMX8QXP_QSPI0B_DATA2_LSIO_GPIO3_IO20 0x40000021>, /* SODIMM 107 */
<IMX8QXP_QSPI0B_SS1_B_LSIO_GPIO3_IO24 0x21>; /* SODIMM 106 */
};
diff --git a/arch/arm64/boot/dts/freescale/imx95.dtsi b/arch/arm64/boot/dts/freescale/imx95.dtsi
index ddbc94c375e0..227eae56125d 100644
--- a/arch/arm64/boot/dts/freescale/imx95.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx95.dtsi
@@ -1480,8 +1480,9 @@ pcie0: pcie@4c300000 {
<0 0x4c360000 0 0x10000>,
<0 0x4c340000 0 0x4000>;
reg-names = "dbi", "config", "atu", "app";
- ranges = <0x81000000 0x0 0x00000000 0x0 0x6ff00000 0 0x00100000>,
- <0x82000000 0x0 0x10000000 0x9 0x10000000 0 0x10000000>;
+ ranges = <0x43000000 0x9 0x00000000 0x9 0x00000000 0x0 0xe0000000>,
+ <0x82000000 0x0 0xe0000000 0x9 0xe0000000 0x0 0x10000000>,
+ <0x81000000 0x0 0x00000000 0x9 0xf0000000 0x0 0x00100000>;
#address-cells = <3>;
#size-cells = <2>;
device_type = "pci";
@@ -1547,8 +1548,9 @@ pcie1: pcie@4c380000 {
<0 0x4c3e0000 0 0x10000>,
<0 0x4c3c0000 0 0x4000>;
reg-names = "dbi", "config", "atu", "app";
- ranges = <0x81000000 0 0x00000000 0x8 0x8ff00000 0 0x00100000>,
- <0x82000000 0 0x10000000 0xa 0x10000000 0 0x10000000>;
+ ranges = <0x43000000 0xa 0x00000000 0xa 0x00000000 0x0 0xe0000000>,
+ <0x82000000 0x0 0xe0000000 0xa 0xe0000000 0x0 0x10000000>,
+ <0x81000000 0x0 0x00000000 0xa 0xf0000000 0x0 0x00100000>;
#address-cells = <3>;
#size-cells = <2>;
device_type = "pci";
diff --git a/arch/arm64/boot/dts/mediatek/mt8192-asurada.dtsi b/arch/arm64/boot/dts/mediatek/mt8192-asurada.dtsi
index ad52c1d6e4ee..aa12a14aabf6 100644
--- a/arch/arm64/boot/dts/mediatek/mt8192-asurada.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt8192-asurada.dtsi
@@ -515,6 +515,7 @@ flash@0 {
&pcie {
pinctrl-names = "default";
pinctrl-0 = <&pcie_pins>;
+ memory-region = <&wifi_restricted_dma_region>;
pcie0: pcie@0,0 {
device_type = "pci";
@@ -529,7 +530,6 @@ pcie0: pcie@0,0 {
wifi: wifi@0,0 {
reg = <0x10000 0 0 0 0x100000>,
<0x10000 0 0x100000 0 0x100000>;
- memory-region = <&wifi_restricted_dma_region>;
};
};
};
diff --git a/arch/arm64/boot/dts/nvidia/tegra234.dtsi b/arch/arm64/boot/dts/nvidia/tegra234.dtsi
index 2601b43b2d8c..75428e70a692 100644
--- a/arch/arm64/boot/dts/nvidia/tegra234.dtsi
+++ b/arch/arm64/boot/dts/nvidia/tegra234.dtsi
@@ -3529,7 +3529,7 @@ ethernet@6800000 {
<&bpmp TEGRA234_CLK_MGBE0_RX_PCS_M>,
<&bpmp TEGRA234_CLK_MGBE0_RX_PCS>,
<&bpmp TEGRA234_CLK_MGBE0_TX_PCS>;
- clock-names = "mgbe", "mac", "mac-divider", "ptp-ref", "rx-input-m",
+ clock-names = "mgbe", "mac", "mac-divider", "ptp_ref", "rx-input-m",
"rx-input", "tx", "eee-pcs", "rx-pcs-input", "rx-pcs-m",
"rx-pcs", "tx-pcs";
resets = <&bpmp TEGRA234_RESET_MGBE0_MAC>,
@@ -3571,7 +3571,7 @@ ethernet@6900000 {
<&bpmp TEGRA234_CLK_MGBE1_RX_PCS_M>,
<&bpmp TEGRA234_CLK_MGBE1_RX_PCS>,
<&bpmp TEGRA234_CLK_MGBE1_TX_PCS>;
- clock-names = "mgbe", "mac", "mac-divider", "ptp-ref", "rx-input-m",
+ clock-names = "mgbe", "mac", "mac-divider", "ptp_ref", "rx-input-m",
"rx-input", "tx", "eee-pcs", "rx-pcs-input", "rx-pcs-m",
"rx-pcs", "tx-pcs";
resets = <&bpmp TEGRA234_RESET_MGBE1_MAC>,
@@ -3613,7 +3613,7 @@ ethernet@6a00000 {
<&bpmp TEGRA234_CLK_MGBE2_RX_PCS_M>,
<&bpmp TEGRA234_CLK_MGBE2_RX_PCS>,
<&bpmp TEGRA234_CLK_MGBE2_TX_PCS>;
- clock-names = "mgbe", "mac", "mac-divider", "ptp-ref", "rx-input-m",
+ clock-names = "mgbe", "mac", "mac-divider", "ptp_ref", "rx-input-m",
"rx-input", "tx", "eee-pcs", "rx-pcs-input", "rx-pcs-m",
"rx-pcs", "tx-pcs";
resets = <&bpmp TEGRA234_RESET_MGBE2_MAC>,
@@ -3655,7 +3655,7 @@ ethernet@6b00000 {
<&bpmp TEGRA234_CLK_MGBE3_RX_PCS_M>,
<&bpmp TEGRA234_CLK_MGBE3_RX_PCS>,
<&bpmp TEGRA234_CLK_MGBE3_TX_PCS>;
- clock-names = "mgbe", "mac", "mac-divider", "ptp-ref", "rx-input-m",
+ clock-names = "mgbe", "mac", "mac-divider", "ptp_ref", "rx-input-m",
"rx-input", "tx", "eee-pcs", "rx-pcs-input", "rx-pcs-m",
"rx-pcs", "tx-pcs";
resets = <&bpmp TEGRA234_RESET_MGBE3_MAC>,
diff --git a/arch/arm64/boot/dts/qcom/sc7180.dtsi b/arch/arm64/boot/dts/qcom/sc7180.dtsi
index 6ae5ca00c718..e52d938df1be 100644
--- a/arch/arm64/boot/dts/qcom/sc7180.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180.dtsi
@@ -1601,7 +1601,11 @@ ice: crypto@1d90000 {
compatible = "qcom,sc7180-inline-crypto-engine",
"qcom,inline-crypto-engine";
reg = <0 0x01d90000 0 0x8000>;
- clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>;
+ clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>,
+ <&gcc GCC_UFS_PHY_AHB_CLK>;
+ clock-names = "core",
+ "iface";
+ power-domains = <&gcc UFS_PHY_GDSC>;
};
ipa: ipa@1e40000 {
diff --git a/arch/arm64/boot/dts/qcom/sc7280.dtsi b/arch/arm64/boot/dts/qcom/sc7280.dtsi
index 8fbc95cf63fe..b47ad94f7d1a 100644
--- a/arch/arm64/boot/dts/qcom/sc7280.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7280.dtsi
@@ -2394,8 +2394,12 @@ ufs_mem_phy: phy@1d87000 {
ice: crypto@1d88000 {
compatible = "qcom,sc7280-inline-crypto-engine",
"qcom,inline-crypto-engine";
- reg = <0 0x01d88000 0 0x8000>;
- clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>;
+ reg = <0 0x01d88000 0 0x18000>;
+ clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>,
+ <&gcc GCC_UFS_PHY_AHB_CLK>;
+ clock-names = "core",
+ "iface";
+ power-domains = <&gcc GCC_UFS_PHY_GDSC>;
};
cryptobam: dma-controller@1dc4000 {
diff --git a/arch/arm64/boot/dts/qcom/sc8180x.dtsi b/arch/arm64/boot/dts/qcom/sc8180x.dtsi
index 60f71b490261..6ca9fba74c08 100644
--- a/arch/arm64/boot/dts/qcom/sc8180x.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc8180x.dtsi
@@ -2637,7 +2637,7 @@ usb_mp_qmpphy1: phy@88ec000 {
status = "disabled";
};
- usb_sec_qmpphy: phy@88ee000 {
+ usb_sec_qmpphy: phy@88ed000 {
compatible = "qcom,sc8180x-qmp-usb3-dp-phy";
reg = <0 0x088ed000 0 0x3000>;
diff --git a/arch/arm64/boot/dts/qcom/sdm630.dtsi b/arch/arm64/boot/dts/qcom/sdm630.dtsi
index 37692d438d23..b78f1d7e7e4a 100644
--- a/arch/arm64/boot/dts/qcom/sdm630.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm630.dtsi
@@ -493,9 +493,12 @@ venus_region: venus@9f800000 {
no-map;
};
- adsp_mem: adsp-region@f6000000 {
- reg = <0x0 0xf6000000 0x0 0x800000>;
- no-map;
+ adsp_mem: adsp-region {
+ compatible = "shared-dma-pool";
+ alloc-ranges = <0x0 0x80000000 0x0 0x80000000>;
+ alignment = <0x0 0x400000>;
+ size = <0x0 0x800000>;
+ reusable;
};
qseecom_mem: qseecom-region@f6800000 {
diff --git a/arch/arm64/boot/dts/qcom/sdm845-db845c-navigation-mezzanine.dtso b/arch/arm64/boot/dts/qcom/sdm845-db845c-navigation-mezzanine.dtso
index 51f1a4883ab8..df1907560c52 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-db845c-navigation-mezzanine.dtso
+++ b/arch/arm64/boot/dts/qcom/sdm845-db845c-navigation-mezzanine.dtso
@@ -16,7 +16,12 @@ &camss {
status = "okay";
ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
port@0 {
+ reg = <0>;
+
csiphy0_ep: endpoint {
data-lanes = <0 1 2 3>;
remote-endpoint = <&ov8856_ep>;
diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi
index f5be69e3be99..4147d3164574 100644
--- a/arch/arm64/boot/dts/qcom/sm8450.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi
@@ -4543,8 +4543,12 @@ ufs_mem_phy: phy@1d87000 {
ice: crypto@1d88000 {
compatible = "qcom,sm8450-inline-crypto-engine",
"qcom,inline-crypto-engine";
- reg = <0 0x01d88000 0 0x8000>;
- clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>;
+ reg = <0 0x01d88000 0 0x18000>;
+ clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>,
+ <&gcc GCC_UFS_PHY_AHB_CLK>;
+ clock-names = "core",
+ "iface";
+ power-domains = <&gcc UFS_PHY_GDSC>;
};
cryptobam: dma-controller@1dc4000 {
diff --git a/arch/arm64/boot/dts/qcom/sm8650.dtsi b/arch/arm64/boot/dts/qcom/sm8650.dtsi
index 6098206350f9..b4d9900e1776 100644
--- a/arch/arm64/boot/dts/qcom/sm8650.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8650.dtsi
@@ -2608,7 +2608,11 @@ ice: crypto@1d88000 {
"qcom,inline-crypto-engine";
reg = <0 0x01d88000 0 0x8000>;
- clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>;
+ clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>,
+ <&gcc GCC_UFS_PHY_AHB_CLK>;
+ clock-names = "core",
+ "iface";
+ power-domains = <&gcc UFS_PHY_GDSC>;
};
tcsr_mutex: hwlock@1f40000 {
diff --git a/arch/arm64/boot/dts/rockchip/rk3566-roc-pc.dts b/arch/arm64/boot/dts/rockchip/rk3566-roc-pc.dts
index 67e7801bd489..43b0b202852c 100644
--- a/arch/arm64/boot/dts/rockchip/rk3566-roc-pc.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3566-roc-pc.dts
@@ -245,7 +245,7 @@ rk809: pmic@20 {
compatible = "rockchip,rk809";
reg = <0x20>;
interrupt-parent = <&gpio0>;
- interrupts = <RK_PA7 IRQ_TYPE_LEVEL_LOW>;
+ interrupts = <RK_PA3 IRQ_TYPE_LEVEL_LOW>;
clock-output-names = "rk808-clkout1", "rk808-clkout2";
assigned-clocks = <&cru I2S1_MCLKOUT_TX>;
assigned-clock-parents = <&cru CLK_I2S1_8CH_TX>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3568-nanopi-r5s.dts b/arch/arm64/boot/dts/rockchip/rk3568-nanopi-r5s.dts
index b6ad8328c7eb..b9f4b6e63d47 100644
--- a/arch/arm64/boot/dts/rockchip/rk3568-nanopi-r5s.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3568-nanopi-r5s.dts
@@ -77,7 +77,7 @@ &mdio0 {
rgmii_phy0: ethernet-phy@1 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <1>;
- pinctrl-0 = <ð_phy0_reset_pin>;
+ pinctrl-0 = <&gmac0_rstn_gpio0_c5_pin>;
pinctrl-names = "default";
};
};
@@ -111,8 +111,8 @@ &pcie3x2 {
&pinctrl {
gmac0 {
- eth_phy0_reset_pin: eth-phy0-reset-pin {
- rockchip,pins = <0 RK_PC4 RK_FUNC_GPIO &pcfg_pull_up>;
+ gmac0_rstn_gpio0_c5_pin: gmac0-rstn-gpio0-c5-pin {
+ rockchip,pins = <0 RK_PC5 RK_FUNC_GPIO &pcfg_pull_up>;
};
};
diff --git a/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts b/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts
index 0b6e371eea98..f94378379447 100644
--- a/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts
@@ -134,6 +134,7 @@ vddshv_sdio: regulator-5 {
gpios = <&main_gpio0 31 GPIO_ACTIVE_HIGH>;
states = <1800000 0x0>,
<3300000 0x1>;
+ bootph-all;
};
leds {
@@ -381,6 +382,7 @@ pmic_irq_pins_default: pmic-irq-default-pins {
pinctrl-single,pins = <
AM62AX_MCU_IOPAD(0x000, PIN_INPUT, 7) /* (E11) MCU_GPIO0_0 */
>;
+ bootph-all;
};
};
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index c838309e4ec4..19b8843374d0 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -144,5 +144,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val);
extern unsigned long kvm_nvhe_sym(__icache_flags);
extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits);
extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl);
+extern unsigned int kvm_nvhe_sym(hyp_gicv3_nr_lr);
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
index fb441702495f..28af4bcd97a5 100644
--- a/arch/arm64/kernel/entry-fpsimd.S
+++ b/arch/arm64/kernel/entry-fpsimd.S
@@ -38,10 +38,10 @@ SYM_FUNC_END(fpsimd_load_state)
*
* x0 - pointer to buffer for state
* x1 - pointer to storage for FPSR
- * x2 - Save FFR if non-zero
+ * w2 - Save FFR if non-zero
*/
SYM_FUNC_START(sve_save_state)
- sve_save 0, x1, x2, 3
+ sve_save 0, x1, w2, 3
ret
SYM_FUNC_END(sve_save_state)
@@ -50,10 +50,10 @@ SYM_FUNC_END(sve_save_state)
*
* x0 - pointer to buffer for state
* x1 - pointer to storage for FPSR
- * x2 - Restore FFR if non-zero
+ * w2 - Restore FFR if non-zero
*/
SYM_FUNC_START(sve_load_state)
- sve_load 0, x1, x2, 4
+ sve_load 0, x1, w2, 4
ret
SYM_FUNC_END(sve_load_state)
diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
index ab76b36dce82..73cce8ac8368 100644
--- a/arch/arm64/kernel/hw_breakpoint.c
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -559,6 +559,15 @@ int hw_breakpoint_arch_parse(struct perf_event *bp,
else
alignment_mask = 0x7;
offset = hw->address & alignment_mask;
+
+ /*
+ * BAS is an 8-bit field in WCR/BCR; the shift below would
+ * silently drop the high bits of ctrl.len when offset + len
+ * exceeds 8, programming hardware to watch fewer bytes than
+ * the user requested.
+ */
+ if (((u32)hw->ctrl.len << offset) > ARM_BREAKPOINT_LEN_8)
+ return -EINVAL;
}
hw->address &= ~alignment_mask;
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 3b3f6b56e733..4fdf102d8a97 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -543,23 +543,13 @@ void arch_unregister_cpu(int cpu)
{
acpi_handle acpi_handle = acpi_get_processor_handle(cpu);
struct cpu *c = &per_cpu(cpu_devices, cpu);
- acpi_status status;
unsigned long long sta;
-
- if (!acpi_handle) {
- pr_err_once("Removing a CPU without associated ACPI handle\n");
- return;
- }
+ acpi_status status;
status = acpi_evaluate_integer(acpi_handle, "_STA", NULL, &sta);
- if (ACPI_FAILURE(status))
- return;
-
- /* For now do not allow anything that looks like physical CPU HP */
- if (cpu_present(cpu) && !(sta & ACPI_STA_DEVICE_PRESENT)) {
+ if (!ACPI_FAILURE(status) &&
+ cpu_present(cpu) && !(sta & ACPI_STA_DEVICE_PRESENT))
pr_err_once("Changing CPU present bit is not supported\n");
- return;
- }
unregister_cpu(c);
}
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 3753ef782e98..a713f32a6545 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -2313,6 +2313,8 @@ static int __init init_subsystems(void)
switch (err) {
case 0:
vgic_present = true;
+ if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+ kvm_nvhe_sym(hyp_gicv3_nr_lr) = kvm_vgic_global_state.nr_lr;
break;
case -ENODEV:
case -ENXIO:
diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h
index cc9cb6395946..9b387b56189f 100644
--- a/arch/arm64/kvm/hyp/include/hyp/switch.h
+++ b/arch/arm64/kvm/hyp/include/hyp/switch.h
@@ -297,16 +297,19 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
static inline bool kvm_hyp_handle_mops(struct kvm_vcpu *vcpu, u64 *exit_code)
{
+ u64 spsr;
+
*vcpu_pc(vcpu) = read_sysreg_el2(SYS_ELR);
arm64_mops_reset_regs(vcpu_gp_regs(vcpu), vcpu->arch.fault.esr_el2);
write_sysreg_el2(*vcpu_pc(vcpu), SYS_ELR);
/*
* Finish potential single step before executing the prologue
- * instruction.
+ * instruction. Modify the hardware SPSR_EL2 directly, as vcpu_cpsr()
+ * may hold a synthetic (vEL2) value for a guest hypervisor.
*/
- *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;
- write_sysreg_el2(*vcpu_cpsr(vcpu), SYS_SPSR);
+ spsr = read_sysreg_el2(SYS_SPSR);
+ write_sysreg_el2(spsr & ~DBG_SPSR_SS, SYS_SPSR);
return true;
}
@@ -451,8 +454,6 @@ static inline bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code)
return false;
break;
case ESR_ELx_EC_SYS64:
- if (WARN_ON_ONCE(!is_hyp_ctxt(vcpu)))
- return false;
fallthrough;
case ESR_ELx_EC_SVE:
if (!sve_guest)
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
index 75f7e386de75..72384712377d 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
@@ -22,6 +22,9 @@
DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
+/* Number of implemented GICv3 LRs. Used by flush_hyp_vcpu(). */
+unsigned int hyp_gicv3_nr_lr;
+
void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt);
static void __hyp_sve_save_guest(struct kvm_vcpu *vcpu)
@@ -100,6 +103,9 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt;
+ /* __hyp_running_vcpu must be NULL in a guest context. */
+ hyp_vcpu->vcpu.arch.ctxt.__hyp_running_vcpu = NULL;
+
hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state);
/* Limit guest vector length to the maximum supported by the host. */
hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl);
@@ -116,6 +122,12 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
hyp_vcpu->vcpu.arch.vsesr_el2 = host_vcpu->arch.vsesr_el2;
hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3;
+
+ /* Bound used_lrs by the number of implemented list registers. */
+ hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs =
+ min_t(unsigned int,
+ hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs,
+ hyp_gicv3_nr_lr);
}
static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index deabc21caae3..dca66f7571be 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -1333,7 +1333,8 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr,
KVM_PGTABLE_WALK_HANDLE_FAULT |
KVM_PGTABLE_WALK_SHARED);
if (!ret || ret == -EAGAIN)
- kvm_call_hyp(__kvm_tlb_flush_vmid_ipa_nsh, pgt->mmu, addr, level);
+ kvm_call_hyp(__kvm_tlb_flush_vmid_ipa_nsh, pgt->mmu, addr,
+ (ret == -EAGAIN) ? TLBI_TTL_UNKNOWN : level);
return ret;
}
diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c
index dcd6b23ad2e1..7282305de347 100644
--- a/arch/arm64/kvm/vgic/vgic-its.c
+++ b/arch/arm64/kvm/vgic/vgic-its.c
@@ -510,6 +510,8 @@ static struct vgic_its *__vgic_doorbell_to_its(struct kvm *kvm, gpa_t db)
struct kvm_io_device *kvm_io_dev;
struct vgic_io_device *iodev;
+ guard(srcu)(&kvm->srcu);
+
kvm_io_dev = kvm_io_bus_get_dev(kvm, KVM_MMIO_BUS, db);
if (!kvm_io_dev)
return ERR_PTR(-EINVAL);
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index f50274fd5581..497b7af5caa4 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -684,15 +684,16 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)
raw_spin_lock(&irq->irq_lock);
/*
- * If the affinity has been preserved, move the
- * interrupt around. Otherwise, it means things have
- * changed while the interrupt was unlocked, and we
- * need to replay this.
+ * If the interrupt is still ours and its affinity has
+ * been preserved, move it around. Otherwise, it means
+ * things have changed while the interrupt was unlocked
+ * (it may even have been taken off the list with its
+ * affinity left untouched), and we need to replay this.
*
* In all cases, we cannot trust the list not to have
* changed, so we restart from the beginning.
*/
- if (target_vcpu == vgic_target_oracle(irq)) {
+ if (irq->vcpu == vcpu && target_vcpu == vgic_target_oracle(irq)) {
struct vgic_cpu *new_cpu = &target_vcpu->arch.vgic_cpu;
list_del(&irq->ap_list);
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 872227e4fada..2b5e0ccc0af5 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -900,7 +900,13 @@ static void unmap_hotplug_pmd_range(pud_t *pudp, unsigned long addr,
if (free_mapped) {
/* CONT blocks are not supported in the vmemmap */
WARN_ON(pmd_cont(pmd));
- flush_tlb_kernel_range(addr, addr + PMD_SIZE);
+ /*
+ * Invalidating a block entry requires just
+ * a single overlapping TLB invalidation,
+ * so limit the range of the flush to a single
+ * page.
+ */
+ flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
free_hotplug_page_range(pmd_page(pmd),
PMD_SIZE, altmap);
}
@@ -930,7 +936,8 @@ static void unmap_hotplug_pud_range(p4d_t *p4dp, unsigned long addr,
if (pud_sect(pud)) {
pud_clear(pudp);
if (free_mapped) {
- flush_tlb_kernel_range(addr, addr + PUD_SIZE);
+ /* See comment in unmap_hotplug_pmd_range(). */
+ flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
free_hotplug_page_range(pud_page(pud),
PUD_SIZE, altmap);
}
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index c852749405e0..27d27848db52 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -1558,15 +1558,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
/* speculation barrier */
case BPF_ST | BPF_NOSPEC:
- /*
- * Nothing required here.
- *
- * In case of arm64, we rely on the firmware mitigation of
- * Speculative Store Bypass as controlled via the ssbd kernel
- * parameter. Whenever the mitigation is enabled, it works
- * for all of the kernel code with no need to provide any
- * additional instructions.
- */
+ /* See bpf_jit_bypass_spec_v4() */
break;
/* ST: *(size *)(dst + off) = imm */
@@ -1881,7 +1873,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
image_size = extable_offset + extable_size;
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr,
sizeof(u64), &header, &image_ptr,
- jit_fill_hole);
+ jit_fill_hole, was_classic);
if (!ro_header) {
prog = orig_prog;
goto out_off;
@@ -2413,7 +2405,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, jit_fill_hole);
+ return bpf_prog_pack_alloc(size, jit_fill_hole, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
@@ -2730,6 +2722,17 @@ bool bpf_jit_supports_percpu_insn(void)
return true;
}
+bool bpf_jit_bypass_spec_v4(void)
+{
+ /* In case of arm64, we rely on the firmware mitigation of Speculative
+ * Store Bypass as controlled via the ssbd kernel parameter. Whenever
+ * the mitigation is enabled, it works for all of the kernel code with
+ * no need to provide any additional instructions. Therefore, skip
+ * inserting nospec insns against Spectre v4.
+ */
+ return true;
+}
+
bool bpf_jit_inlines_helper_call(s32 imm)
{
switch (imm) {
diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h
index 32899160086d..8f1df75686e2 100644
--- a/arch/loongarch/include/asm/pgtable.h
+++ b/arch/loongarch/include/asm/pgtable.h
@@ -390,6 +390,8 @@ static inline pte_t pte_mkwrite_novma(pte_t pte)
static inline pte_t pte_wrprotect(pte_t pte)
{
+ if (pte_val(pte) & _PAGE_DIRTY)
+ pte_val(pte) |= _PAGE_MODIFIED;
pte_val(pte) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
return pte;
}
@@ -505,6 +507,8 @@ static inline pmd_t pmd_mkwrite_novma(pmd_t pmd)
static inline pmd_t pmd_wrprotect(pmd_t pmd)
{
+ if (pmd_val(pmd) & _PAGE_DIRTY)
+ pmd_val(pmd) |= _PAGE_MODIFIED;
pmd_val(pmd) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
return pmd;
}
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index 5b1dee18d260..5108725a4507 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -103,7 +103,6 @@ static unsigned long kvm_emu_xchg_csr(struct kvm_vcpu *vcpu, int csrid,
old = kvm_read_sw_gcsr(csr, csrid);
val = (old & ~csr_mask) | (val & csr_mask);
kvm_write_sw_gcsr(csr, csrid, val);
- old = old & csr_mask;
} else
pr_warn_once("Unsupported csrxchg 0x%x with pc %lx\n", csrid, vcpu->arch.pc);
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index 38d3435d4fe0..c9f16c27e87d 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -1029,7 +1029,8 @@ static int kvm_loongarch_cpucfg_get_attr(struct kvm_vcpu *vcpu,
return -ENXIO;
}
- put_user(val, uaddr);
+ if (put_user(val, uaddr))
+ return -EFAULT;
return ret;
}
@@ -1233,7 +1234,7 @@ int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu)
fpu->fcc = vcpu->arch.fpu.fcc;
fpu->fcsr = vcpu->arch.fpu.fcsr;
for (i = 0; i < NUM_FPU_REGS; i++)
- memcpy(&fpu->fpr[i], &vcpu->arch.fpu.fpr[i], FPU_REG_WIDTH / 64);
+ memcpy(&fpu->fpr[i], &vcpu->arch.fpu.fpr[i], sizeof(union fpureg));
return 0;
}
@@ -1245,7 +1246,7 @@ int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu)
vcpu->arch.fpu.fcc = fpu->fcc;
vcpu->arch.fpu.fcsr = fpu->fcsr;
for (i = 0; i < NUM_FPU_REGS; i++)
- memcpy(&vcpu->arch.fpu.fpr[i], &fpu->fpr[i], FPU_REG_WIDTH / 64);
+ memcpy(&vcpu->arch.fpu.fpr[i], &fpu->fpr[i], sizeof(union fpureg));
return 0;
}
@@ -1429,6 +1430,10 @@ void kvm_lose_fpu(struct kvm_vcpu *vcpu)
int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu, struct kvm_interrupt *irq)
{
int intr = (int)irq->irq;
+ unsigned int vector = abs(intr);
+
+ if (vector >= EXCCODE_INT_NUM)
+ return -EINVAL;
if (intr > 0)
kvm_queue_irq(vcpu, intr);
diff --git a/arch/mips/dec/int-handler.S b/arch/mips/dec/int-handler.S
index 011d1d678840..a0b439c90488 100644
--- a/arch/mips/dec/int-handler.S
+++ b/arch/mips/dec/int-handler.S
@@ -277,7 +277,7 @@
srlv t3,t1,t2
handle_it:
- j dec_irq_dispatch
+ j do_IRQ
nop
#if defined(CONFIG_32BIT) && defined(CONFIG_MIPS_FP_SUPPORT)
diff --git a/arch/mips/dec/platform.c b/arch/mips/dec/platform.c
index 723ce16cbfc0..a005246a0ac5 100644
--- a/arch/mips/dec/platform.c
+++ b/arch/mips/dec/platform.c
@@ -38,6 +38,10 @@ static struct platform_device dec_rtc_device = {
.num_resources = ARRAY_SIZE(dec_rtc_resources),
};
+static struct platform_device *dec_rtc_devices[] __initdata = {
+ &dec_rtc_device,
+};
+
static struct resource dec_dz_resources[] = {
{ .name = "dz", .flags = IORESOURCE_MEM, },
{ .name = "dz", .flags = IORESOURCE_IRQ, },
@@ -137,7 +141,7 @@ static int __init dec_add_devices(void)
}
num_zs = i;
- ret1 = platform_device_register(&dec_rtc_device);
+ ret1 = platform_add_devices(dec_rtc_devices, 1);
ret2 = IS_ENABLED(CONFIG_32BIT) ?
platform_add_devices(dec_dz_devices, num_dz) : 0;
ret3 = platform_add_devices(dec_zs_devices, num_zs);
diff --git a/arch/mips/dec/prom/init.c b/arch/mips/dec/prom/init.c
index 8d74d7d6c05b..e46c7accefff 100644
--- a/arch/mips/dec/prom/init.c
+++ b/arch/mips/dec/prom/init.c
@@ -3,7 +3,7 @@
* init.c: PROM library initialisation code.
*
* Copyright (C) 1998 Harald Koerfgen
- * Copyright (C) 2002, 2004 Maciej W. Rozycki
+ * Copyright (C) 2002, 2004, 2026 Maciej W. Rozycki
*/
#include <linux/init.h>
#include <linux/kernel.h>
@@ -20,6 +20,10 @@
#include <asm/dec/prom.h>
+#ifdef CONFIG_64BIT
+unsigned long o32_stk[O32_STK_SIZE] __initdata = { 0 };
+#endif
+
int (*__rex_bootinit)(void);
int (*__rex_bootread)(void);
int (*__rex_getbitmap)(memmap *);
diff --git a/arch/mips/dec/setup.c b/arch/mips/dec/setup.c
index 87f0a1436bf9..abe42616498d 100644
--- a/arch/mips/dec/setup.c
+++ b/arch/mips/dec/setup.c
@@ -776,9 +776,3 @@ void __init arch_init_irq(void)
pr_err("Failed to register halt interrupt\n");
}
}
-
-asmlinkage unsigned int dec_irq_dispatch(unsigned int irq)
-{
- do_IRQ(irq);
- return 0;
-}
diff --git a/arch/mips/fw/lib/call_o32.S b/arch/mips/fw/lib/call_o32.S
index ee856709e0b6..77533cfbdfc1 100644
--- a/arch/mips/fw/lib/call_o32.S
+++ b/arch/mips/fw/lib/call_o32.S
@@ -74,7 +74,7 @@ NESTED(call_o32, O32_FRAMESZ, ra)
PTR_LA t1,6*O32_SZREG(fp)
li t2,O32_ARGC-6
1:
- lw t3,(t0)
+ ld t3,(t0)
REG_ADDU t0,SZREG
sw t3,(t1)
REG_SUBU t2,1
diff --git a/arch/mips/include/asm/dec/prom.h b/arch/mips/include/asm/dec/prom.h
index 8fcad6984389..af068e634f1e 100644
--- a/arch/mips/include/asm/dec/prom.h
+++ b/arch/mips/include/asm/dec/prom.h
@@ -4,7 +4,7 @@
*
* DECstation PROM interface.
*
- * Copyright (C) 2002 Maciej W. Rozycki
+ * Copyright (C) 2002, 2026 Maciej W. Rozycki
*
* Based on arch/mips/dec/prom/prom.h by the Anonymous.
*/
@@ -97,6 +97,17 @@ extern int (*__pmax_close)(int);
#ifdef CONFIG_64BIT
+#define O32_STK_SIZE 512
+extern unsigned long o32_stk[];
+
+/* Switch the stack if outside the 32-bit address space. */
+static inline unsigned long *o32_get_stk(void)
+{
+ long fp = (long)__builtin_frame_address(0);
+
+ return fp != (int)fp ? o32_stk + O32_STK_SIZE : NULL;
+}
+
/*
* On MIPS64 we have to call PROM functions via a helper
* dispatcher to accommodate ABI incompatibilities.
@@ -128,7 +139,7 @@ int __DEC_PROM_O32(_prom_printf, (int (*)(char *, ...), void *, char *, ...));
#define prom_getchar() _prom_getchar(__prom_getchar, NULL)
#define prom_getenv(x) _prom_getenv(__prom_getenv, NULL, x)
-#define prom_printf(x...) _prom_printf(__prom_printf, NULL, x)
+#define prom_printf(x...) _prom_printf(__prom_printf, o32_get_stk(), x)
#else /* !CONFIG_64BIT */
diff --git a/arch/mips/kernel/mips-mt-fpaff.c b/arch/mips/kernel/mips-mt-fpaff.c
index 10172fc4f627..4fead87d2f43 100644
--- a/arch/mips/kernel/mips-mt-fpaff.c
+++ b/arch/mips/kernel/mips-mt-fpaff.c
@@ -71,11 +71,16 @@ asmlinkage long mipsmt_sys_sched_setaffinity(pid_t pid, unsigned int len,
struct task_struct *p;
int retval;
- if (len < sizeof(new_mask))
- return -EINVAL;
-
- if (copy_from_user(&new_mask, user_mask_ptr, sizeof(new_mask)))
- return -EFAULT;
+ if (!alloc_cpumask_var(&new_mask, GFP_KERNEL))
+ return -ENOMEM;
+ if (len < cpumask_size())
+ cpumask_clear(new_mask);
+ else if (len > cpumask_size())
+ len = cpumask_size();
+ if (copy_from_user(new_mask, user_mask_ptr, len)) {
+ retval = -EFAULT;
+ goto out_free_new_mask;
+ }
cpus_read_lock();
rcu_read_lock();
@@ -84,7 +89,8 @@ asmlinkage long mipsmt_sys_sched_setaffinity(pid_t pid, unsigned int len,
if (!p) {
rcu_read_unlock();
cpus_read_unlock();
- return -ESRCH;
+ retval = -ESRCH;
+ goto out_free_new_mask;
}
/* Prevent p going away */
@@ -95,13 +101,9 @@ asmlinkage long mipsmt_sys_sched_setaffinity(pid_t pid, unsigned int len,
retval = -ENOMEM;
goto out_put_task;
}
- if (!alloc_cpumask_var(&new_mask, GFP_KERNEL)) {
- retval = -ENOMEM;
- goto out_free_cpus_allowed;
- }
if (!alloc_cpumask_var(&effective_mask, GFP_KERNEL)) {
retval = -ENOMEM;
- goto out_free_new_mask;
+ goto out_free_cpus_allowed;
}
if (!check_same_owner(p) && !capable(CAP_SYS_NICE)) {
retval = -EPERM;
@@ -142,13 +144,13 @@ asmlinkage long mipsmt_sys_sched_setaffinity(pid_t pid, unsigned int len,
}
out_unlock:
free_cpumask_var(effective_mask);
-out_free_new_mask:
- free_cpumask_var(new_mask);
out_free_cpus_allowed:
free_cpumask_var(cpus_allowed);
out_put_task:
put_task_struct(p);
cpus_read_unlock();
+out_free_new_mask:
+ free_cpumask_var(new_mask);
return retval;
}
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 4583d1a2a73e..045feacd5dcf 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -283,9 +283,15 @@ static int maar_res_walk(unsigned long start_pfn, unsigned long nr_pages,
void *data)
{
struct maar_walk_info *wi = data;
- struct maar_config *cfg = &wi->cfg[wi->num_cfg];
+ struct maar_config *cfg;
unsigned int maar_align;
+ /* Ensure we don't overflow the cfg array */
+ if (WARN_ON(wi->num_cfg >= ARRAY_SIZE(wi->cfg)))
+ return -1;
+
+ cfg = &wi->cfg[wi->num_cfg];
+
/* MAAR registers hold physical addresses right shifted by 4 bits */
maar_align = BIT(MIPS_MAAR_ADDR_SHIFT + 4);
@@ -294,9 +300,7 @@ static int maar_res_walk(unsigned long start_pfn, unsigned long nr_pages,
cfg->upper = ALIGN_DOWN(PFN_PHYS(start_pfn + nr_pages), maar_align) - 1;
cfg->attrs = MIPS_MAAR_S;
- /* Ensure we don't overflow the cfg array */
- if (!WARN_ON(wi->num_cfg >= ARRAY_SIZE(wi->cfg)))
- wi->num_cfg++;
+ wi->num_cfg++;
return 0;
}
diff --git a/arch/mips/n64/init.c b/arch/mips/n64/init.c
index dfbd864f4667..66ec28ab41f3 100644
--- a/arch/mips/n64/init.c
+++ b/arch/mips/n64/init.c
@@ -50,7 +50,7 @@ void __init prom_init(void)
#define W 320
#define H 240
-#define REG_BASE ((u32 *) CKSEG1ADDR(0x4400000))
+#define REG_BASE ((u32 __iomem *) CKSEG1ADDR(0x4400000))
static void __init n64rdp_write_reg(const u8 reg, const u32 value)
{
diff --git a/arch/mips/ralink/mt7621.c b/arch/mips/ralink/mt7621.c
index 5a9fd3fe41d7..e25444b149f7 100644
--- a/arch/mips/ralink/mt7621.c
+++ b/arch/mips/ralink/mt7621.c
@@ -63,7 +63,7 @@ phys_addr_t mips_cpc_default_phys_base(void)
static bool __init mt7621_addr_wraparound_test(phys_addr_t size)
{
- void *dm = (void *)KSEG1ADDR(&detect_magic);
+ void __iomem *dm = (void __iomem *)KSEG1ADDR(&detect_magic);
if (CPHYSADDR(dm + size) >= MT7621_LOWMEM_MAX_SIZE)
return true;
diff --git a/arch/mips/sgi-ip22/ip22-gio.c b/arch/mips/sgi-ip22/ip22-gio.c
index f6e66c858e69..6f0cffc5e6e4 100644
--- a/arch/mips/sgi-ip22/ip22-gio.c
+++ b/arch/mips/sgi-ip22/ip22-gio.c
@@ -30,7 +30,6 @@ static struct {
static void gio_bus_release(struct device *dev)
{
- kfree(dev);
}
static struct device gio_bus = {
@@ -101,6 +100,8 @@ int gio_device_register(struct gio_device *giodev)
{
giodev->dev.bus = &gio_bus_type;
giodev->dev.parent = &gio_bus;
+ giodev->dev.release = gio_release_dev;
+
return device_register(&giodev->dev);
}
EXPORT_SYMBOL_GPL(gio_device_register);
@@ -132,13 +133,9 @@ static int gio_device_probe(struct device *dev)
if (!drv->probe)
return error;
- gio_dev_get(gio_dev);
-
match = gio_match_device(drv->id_table, gio_dev);
if (match)
error = drv->probe(gio_dev, match);
- if (error)
- gio_dev_put(gio_dev);
return error;
}
diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c
index 825ab8a88f18..58c13a59b93b 100644
--- a/arch/powerpc/kexec/core_64.c
+++ b/arch/powerpc/kexec/core_64.c
@@ -169,7 +169,7 @@ static void kexec_prepare_cpus_wait(int wait_state)
int my_cpu, i, notified=-1;
hw_breakpoint_disable();
- my_cpu = get_cpu();
+ my_cpu = raw_smp_processor_id();
/* Make sure each CPU has at least made it to the state we need.
*
* FIXME: There is a (slim) chance of a problem if not all of the CPUs
@@ -267,8 +267,6 @@ static void kexec_prepare_cpus(void)
/* after we tell the others to go down */
if (ppc_md.kexec_cpu_down)
ppc_md.kexec_cpu_down(0, 0);
-
- put_cpu();
}
#else /* ! SMP */
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 55c3b64a5f3a..180175e072ad 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -174,7 +174,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
alloclen = proglen + FUNCTION_DESCR_SIZE + fixup_len + extable_len;
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
- bpf_jit_fill_ill_insns);
+ bpf_jit_fill_ill_insns, bpf_prog_was_classic(fp));
if (!fhdr) {
fp = org_fp;
goto out_addrs;
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
index af8b07cc5fcb..5a51476ec928 100644
--- a/arch/powerpc/net/bpf_jit_comp64.c
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -440,6 +440,23 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o
return 0;
}
+bool bpf_jit_bypass_spec_v1(void)
+{
+#if defined(CONFIG_PPC_E500) || defined(CONFIG_PPC_BOOK3S_64)
+ return !(security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) &&
+ security_ftr_enabled(SEC_FTR_BNDS_CHK_SPEC_BAR));
+#else
+ return true;
+#endif
+}
+
+bool bpf_jit_bypass_spec_v4(void)
+{
+ return !(security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) &&
+ security_ftr_enabled(SEC_FTR_STF_BARRIER) &&
+ stf_barrier_type_get() != STF_BARRIER_NONE);
+}
+
/*
* We spill into the redzone always, even if the bpf program has its own stackframe.
* Offsets hardcoded based on BPF_PPC_STACK_SAVE -- see bpf_jit_stack_local()
@@ -864,10 +881,6 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code
* BPF_ST NOSPEC (speculation barrier)
*/
case BPF_ST | BPF_NOSPEC:
- if (!security_ftr_enabled(SEC_FTR_FAVOUR_SECURITY) ||
- !security_ftr_enabled(SEC_FTR_STF_BARRIER))
- break;
-
switch (stf_barrier) {
case STF_BARRIER_EIEIO:
EMIT(PPC_RAW_EIEIO() | 0x02000000);
diff --git a/arch/powerpc/perf/core-fsl-emb.c b/arch/powerpc/perf/core-fsl-emb.c
index 1a53ab08447c..31037df4178c 100644
--- a/arch/powerpc/perf/core-fsl-emb.c
+++ b/arch/powerpc/perf/core-fsl-emb.c
@@ -366,9 +366,10 @@ static void fsl_emb_pmu_del(struct perf_event *event, int flags)
cpuhw->n_events--;
+ put_cpu_var(cpu_hw_events);
+
out:
perf_pmu_enable(event->pmu);
- put_cpu_var(cpu_hw_events);
}
static void fsl_emb_pmu_start(struct perf_event *event, int ef_flags)
diff --git a/arch/powerpc/platforms/cell/spufs/file.c b/arch/powerpc/platforms/cell/spufs/file.c
index d5a2c77bc908..efb8b3180347 100644
--- a/arch/powerpc/platforms/cell/spufs/file.c
+++ b/arch/powerpc/platforms/cell/spufs/file.c
@@ -268,10 +268,12 @@ static int spufs_mem_mmap_access(struct vm_area_struct *vma,
if (write && !(vma->vm_flags & VM_WRITE))
return -EACCES;
+ if (offset >= LS_SIZE)
+ return -EFAULT;
if (spu_acquire(ctx))
return -EINTR;
- if ((offset + len) > vma->vm_end)
- len = vma->vm_end - offset;
+ if ((offset + len) > LS_SIZE)
+ len = LS_SIZE - offset;
local_store = ctx->ops->get_ls(ctx);
if (write)
memcpy_toio(local_store + offset, buf, len);
diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
index 4dbb47ddbdcc..06ed5e2aa265 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -396,7 +396,8 @@ static void pnv_kexec_wait_secondaries_down(void)
{
int my_cpu, i, notified = -1;
- my_cpu = get_cpu();
+ /* Called with interrupts disabled, so the CPU is pinned. */
+ my_cpu = raw_smp_processor_id();
for_each_online_cpu(i) {
uint8_t status;
diff --git a/arch/powerpc/platforms/pseries/papr_platform_attributes.c b/arch/powerpc/platforms/pseries/papr_platform_attributes.c
index eea2041b270b..f63e2de9ccca 100644
--- a/arch/powerpc/platforms/pseries/papr_platform_attributes.c
+++ b/arch/powerpc/platforms/pseries/papr_platform_attributes.c
@@ -271,11 +271,9 @@ static int __init papr_init(void)
esi_buf_size = ESI_HDR_SIZE + (CURR_MAX_ESI_ATTRS * max_esi_attrs);
temp_esi_buf = krealloc(esi_buf, esi_buf_size, GFP_KERNEL);
- if (temp_esi_buf)
- esi_buf = temp_esi_buf;
- else
- return -ENOMEM;
-
+ if (!temp_esi_buf)
+ goto out_free_esi_buf;
+ esi_buf = temp_esi_buf;
goto retry;
}
diff --git a/arch/riscv/include/asm/cpu_ops.h b/arch/riscv/include/asm/cpu_ops.h
index 176b570ef982..065811fca594 100644
--- a/arch/riscv/include/asm/cpu_ops.h
+++ b/arch/riscv/include/asm/cpu_ops.h
@@ -24,7 +24,7 @@ struct cpu_operations {
struct task_struct *tidle);
#ifdef CONFIG_HOTPLUG_CPU
void (*cpu_stop)(void);
- int (*cpu_is_stopped)(unsigned int cpu);
+ bool (*cpu_is_stopped)(unsigned int cpu);
#endif
};
diff --git a/arch/riscv/kernel/cacheinfo.c b/arch/riscv/kernel/cacheinfo.c
index 26b085dbdd07..6c9a1ef2d45a 100644
--- a/arch/riscv/kernel/cacheinfo.c
+++ b/arch/riscv/kernel/cacheinfo.c
@@ -133,7 +133,7 @@ int populate_cache_leaves(unsigned int cpu)
ci_leaf_init(this_leaf++, CACHE_TYPE_DATA, level);
levels = level;
}
- of_node_put(np);
+ of_node_put(prev);
return 0;
}
diff --git a/arch/riscv/kernel/cpu-hotplug.c b/arch/riscv/kernel/cpu-hotplug.c
index 3f50d3dd76c6..4570ff4b4e8a 100644
--- a/arch/riscv/kernel/cpu-hotplug.c
+++ b/arch/riscv/kernel/cpu-hotplug.c
@@ -58,8 +58,8 @@ void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
/* Verify from the firmware if the cpu is really stopped*/
if (cpu_ops->cpu_is_stopped)
ret = cpu_ops->cpu_is_stopped(cpu);
- if (ret)
- pr_warn("CPU%u may not have stopped: %d\n", cpu, ret);
+ if (!ret)
+ pr_warn("CPU%u may not have stopped\n", cpu);
}
/*
diff --git a/arch/riscv/kernel/cpu_ops_sbi.c b/arch/riscv/kernel/cpu_ops_sbi.c
index 87d655944803..05b9fe3a35a4 100644
--- a/arch/riscv/kernel/cpu_ops_sbi.c
+++ b/arch/riscv/kernel/cpu_ops_sbi.c
@@ -88,16 +88,19 @@ static void sbi_cpu_stop(void)
pr_crit("Unable to stop the cpu %u (%d)\n", smp_processor_id(), ret);
}
-static int sbi_cpu_is_stopped(unsigned int cpuid)
+static bool sbi_cpu_is_stopped(unsigned int cpuid)
{
int rc;
unsigned long hartid = cpuid_to_hartid_map(cpuid);
rc = sbi_hsm_hart_get_status(hartid);
- if (rc == SBI_HSM_STATE_STOPPED)
- return 0;
- return rc;
+ if (rc != SBI_HSM_STATE_STOPPED) {
+ pr_warn("HART%lu isn't stopped; status %d\n", hartid, rc);
+ return false;
+ }
+
+ return true;
}
#endif
diff --git a/arch/riscv/kernel/machine_kexec.c b/arch/riscv/kernel/machine_kexec.c
index 2306ce3e5f22..738df176ff6f 100644
--- a/arch/riscv/kernel/machine_kexec.c
+++ b/arch/riscv/kernel/machine_kexec.c
@@ -41,6 +41,9 @@ machine_kexec_prepare(struct kimage *image)
if (image->segment[i].memsz <= sizeof(fdt))
continue;
+ if (!image->segment[i].buf)
+ continue;
+
if (image->file_mode)
memcpy(&fdt, image->segment[i].buf, sizeof(fdt));
else if (copy_from_user(&fdt, image->segment[i].buf, sizeof(fdt)))
diff --git a/arch/riscv/kernel/probes/rethook_trampoline.S b/arch/riscv/kernel/probes/rethook_trampoline.S
index f2cd83d9b0f0..c3aa8d8cf5af 100644
--- a/arch/riscv/kernel/probes/rethook_trampoline.S
+++ b/arch/riscv/kernel/probes/rethook_trampoline.S
@@ -41,6 +41,9 @@
REG_S x29, PT_T4(sp)
REG_S x30, PT_T5(sp)
REG_S x31, PT_T6(sp)
+ /* save original sp */
+ addi a0, sp, PT_SIZE_ON_STACK
+ REG_S a0, PT_SP(sp)
.endm
.macro restore_all_base_regs
diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c
index 5a0c0d5b449e..81d6d9d331e0 100644
--- a/arch/riscv/kernel/stacktrace.c
+++ b/arch/riscv/kernel/stacktrace.c
@@ -129,7 +129,7 @@ void notrace walk_stackframe(struct task_struct *task,
while (!kstack_end(ksp)) {
if (__kernel_text_address(pc) && unlikely(!fn(arg, pc)))
break;
- pc = READ_ONCE_NOCHECK(*ksp++) - 0x4;
+ pc = READ_ONCE_NOCHECK(*ksp++);
}
}
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index 5895c1b2be20..43a58e5e123f 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -1113,7 +1113,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, bpf_fill_ill_insns);
+ return bpf_prog_pack_alloc(size, bpf_fill_ill_insns, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 6de753c667f4..2f2af89ab4df 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -126,7 +126,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_jit_binary_pack_alloc(prog_size + extable_size,
&jit_data->ro_image, sizeof(u32),
&jit_data->header, &jit_data->image,
- bpf_fill_ill_insns);
+ bpf_fill_ill_insns,
+ bpf_prog_was_classic(prog));
if (!jit_data->ro_header) {
prog = orig_prog;
goto out_offset;
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 5e6aebf1b739..28e900e5212c 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -137,7 +137,6 @@ config S390
select ARCH_WANT_LD_ORPHAN_WARN
select BUILDTIME_TABLE_SORT
select CLONE_BACKWARDS2
- select DCACHE_WORD_ACCESS if !KMSAN
select DYNAMIC_FTRACE if FUNCTION_TRACER
select FUNCTION_ALIGNMENT_8B if CC_IS_GCC
select FUNCTION_ALIGNMENT_16B if !CC_IS_GCC
diff --git a/arch/s390/include/asm/asm-extable.h b/arch/s390/include/asm/asm-extable.h
index 00a67464c445..f87e06ec86ac 100644
--- a/arch/s390/include/asm/asm-extable.h
+++ b/arch/s390/include/asm/asm-extable.h
@@ -13,7 +13,6 @@
#define EX_TYPE_UA_LOAD_MEM 4
#define EX_TYPE_UA_LOAD_REG 5
#define EX_TYPE_UA_LOAD_REGPAIR 6
-#define EX_TYPE_ZEROPAD 7
#define EX_TYPE_FPC 8
#define EX_DATA_REG_ERR_SHIFT 0
@@ -82,9 +81,6 @@
#define EX_TABLE_UA_LOAD_REGPAIR(_fault, _target, _regerr, _regzero) \
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_UA_LOAD_REGPAIR, _regerr, _regzero, 0)
-#define EX_TABLE_ZEROPAD(_fault, _target, _regdata, _regaddr) \
- __EX_TABLE(__ex_table, _fault, _target, EX_TYPE_ZEROPAD, _regdata, _regaddr, 0)
-
#define EX_TABLE_FPC(_fault, _target) \
__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_FPC, __stringify(%%r0), __stringify(%%r0), 0)
diff --git a/arch/s390/include/asm/word-at-a-time.h b/arch/s390/include/asm/word-at-a-time.h
index 203acd6e431b..e9287036392d 100644
--- a/arch/s390/include/asm/word-at-a-time.h
+++ b/arch/s390/include/asm/word-at-a-time.h
@@ -4,7 +4,6 @@
#include <linux/bitops.h>
#include <linux/wordpart.h>
-#include <asm/asm-extable.h>
#include <asm/bitsperlong.h>
struct word_at_a_time {
@@ -41,25 +40,4 @@ static inline unsigned long zero_bytemask(unsigned long data)
return ~1UL << data;
}
-/*
- * Load an unaligned word from kernel space.
- *
- * In the (very unlikely) case of the word being a page-crosser
- * and the next page not being mapped, take the exception and
- * return zeroes in the non-existing part.
- */
-static inline unsigned long load_unaligned_zeropad(const void *addr)
-{
- unsigned long data;
-
- asm volatile(
- "0: lg %[data],0(%[addr])\n"
- "1: nopr %%r7\n"
- EX_TABLE_ZEROPAD(0b, 1b, %[data], %[addr])
- EX_TABLE_ZEROPAD(1b, 1b, %[data], %[addr])
- : [data] "=d" (data)
- : [addr] "a" (addr), "m" (*(unsigned long *)addr));
- return data;
-}
-
#endif /* _ASM_WORD_AT_A_TIME_H */
diff --git a/arch/s390/kernel/perf_cpum_cf.c b/arch/s390/kernel/perf_cpum_cf.c
index b017db3344cb..17d80e79452b 100644
--- a/arch/s390/kernel/perf_cpum_cf.c
+++ b/arch/s390/kernel/perf_cpum_cf.c
@@ -17,6 +17,7 @@
#include <linux/export.h>
#include <linux/miscdevice.h>
#include <linux/perf_event.h>
+#include <linux/nospec.h>
#include <asm/cpu_mf.h>
#include <asm/hwctrset.h>
@@ -770,6 +771,7 @@ static int __hw_perf_event_init(struct perf_event *event, unsigned int type)
if (!is_userspace_event(ev)) {
if (ev >= ARRAY_SIZE(cpumf_generic_events_user))
return -EOPNOTSUPP;
+ ev = array_index_nospec(ev, ARRAY_SIZE(cpumf_generic_events_user));
ev = cpumf_generic_events_user[ev];
}
} else if (!attr->exclude_kernel && attr->exclude_user) {
@@ -780,6 +782,7 @@ static int __hw_perf_event_init(struct perf_event *event, unsigned int type)
if (!is_userspace_event(ev)) {
if (ev >= ARRAY_SIZE(cpumf_generic_events_basic))
return -EOPNOTSUPP;
+ ev = array_index_nospec(ev, ARRAY_SIZE(cpumf_generic_events_basic));
ev = cpumf_generic_events_basic[ev];
}
}
diff --git a/arch/s390/kernel/process.c b/arch/s390/kernel/process.c
index 9637aee43c40..73af6d9d9bd1 100644
--- a/arch/s390/kernel/process.c
+++ b/arch/s390/kernel/process.c
@@ -52,7 +52,7 @@ void ret_from_fork(void) asm("ret_from_fork");
void __ret_from_fork(struct task_struct *prev, struct pt_regs *regs)
{
- void (*func)(void *arg);
+ int (*func)(void *arg);
schedule_tail(prev);
diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c
index e5d3c0793f61..6e6726b2a441 100644
--- a/arch/s390/kvm/pci.c
+++ b/arch/s390/kvm/pci.c
@@ -302,9 +302,14 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,
gaite->gisc = fib->fmt0.isc;
gaite->count++;
- gaite->aisbo = fib->fmt0.aisbo;
- gaite->aisb = virt_to_phys(page_address(aisb_page) + (fib->fmt0.aisb &
- ~PAGE_MASK));
+ if (fib->fmt0.sum == 1) {
+ gaite->aisbo = fib->fmt0.aisbo;
+ gaite->aisb = virt_to_phys(page_address(aisb_page) +
+ (fib->fmt0.aisb & ~PAGE_MASK));
+ } else {
+ gaite->aisbo = 0;
+ gaite->aisb = 0;
+ }
aift->kzdev[zdev->aisb] = zdev->kzdev;
spin_unlock_irq(&aift->gait_lock);
@@ -330,6 +335,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,
unpin1:
unpin_user_page(aibv_page);
out:
+ kvm_s390_gisc_unregister(kvm, fib->fmt0.isc);
return rc;
}
diff --git a/arch/s390/mm/extable.c b/arch/s390/mm/extable.c
index 812ec5be1291..30e4b96be207 100644
--- a/arch/s390/mm/extable.c
+++ b/arch/s390/mm/extable.c
@@ -61,22 +61,6 @@ static bool ex_handler_ua_load_reg(const struct exception_table_entry *ex,
return true;
}
-static bool ex_handler_zeropad(const struct exception_table_entry *ex, struct pt_regs *regs)
-{
- unsigned int reg_addr = FIELD_GET(EX_DATA_REG_ADDR, ex->data);
- unsigned int reg_data = FIELD_GET(EX_DATA_REG_ERR, ex->data);
- unsigned long data, addr, offset;
-
- addr = regs->gprs[reg_addr];
- offset = addr & (sizeof(unsigned long) - 1);
- addr &= ~(sizeof(unsigned long) - 1);
- data = *(unsigned long *)addr;
- data <<= BITS_PER_BYTE * offset;
- regs->gprs[reg_data] = data;
- regs->psw.addr = extable_fixup(ex);
- return true;
-}
-
static bool ex_handler_fpc(const struct exception_table_entry *ex, struct pt_regs *regs)
{
asm volatile("sfpc %[val]\n" : : [val] "d" (0));
@@ -104,8 +88,6 @@ bool fixup_exception(struct pt_regs *regs)
return ex_handler_ua_load_reg(ex, false, regs);
case EX_TYPE_UA_LOAD_REGPAIR:
return ex_handler_ua_load_reg(ex, true, regs);
- case EX_TYPE_ZEROPAD:
- return ex_handler_zeropad(ex, regs);
case EX_TYPE_FPC:
return ex_handler_fpc(ex, regs);
}
diff --git a/arch/s390/mm/mmap.c b/arch/s390/mm/mmap.c
index 96efa061ce01..0f76c68e3a9b 100644
--- a/arch/s390/mm/mmap.c
+++ b/arch/s390/mm/mmap.c
@@ -71,7 +71,7 @@ static inline unsigned long mmap_base(unsigned long rnd,
return PAGE_ALIGN(STACK_TOP - gap - rnd);
}
-static int get_align_mask(struct file *filp, unsigned long flags)
+static unsigned long get_align_mask(struct file *filp, unsigned long flags)
{
if (!(current->flags & PF_RANDOMIZE))
return 0;
diff --git a/arch/sparc/kernel/led.c b/arch/sparc/kernel/led.c
index ab657b359789..36d851c7454a 100644
--- a/arch/sparc/kernel/led.c
+++ b/arch/sparc/kernel/led.c
@@ -78,7 +78,7 @@ static ssize_t led_proc_write(struct file *file, const char __user *buffer,
return PTR_ERR(buf);
/* work around \n when echo'ing into proc */
- if (buf[count - 1] == '\n')
+ if (count > 0 && buf[count - 1] == '\n')
buf[count - 1] = '\0';
/* before we change anything we want to stop any running timers,
diff --git a/arch/x86/boot/compressed/acpi.c b/arch/x86/boot/compressed/acpi.c
index f196b1d1ddf8..aed27604c11f 100644
--- a/arch/x86/boot/compressed/acpi.c
+++ b/arch/x86/boot/compressed/acpi.c
@@ -184,10 +184,15 @@ static unsigned long get_cmdline_acpi_rsdp(void)
char val[MAX_ADDR_LEN] = { };
int ret;
- ret = cmdline_find_option("acpi_rsdp", val, MAX_ADDR_LEN);
+ ret = cmdline_find_option("acpi_rsdp", val, sizeof(val));
if (ret < 0)
return 0;
+ if (ret >= sizeof(val)) {
+ warn("acpi_rsdp= value too long; ignoring");
+ return 0;
+ }
+
if (boot_kstrtoul(val, 16, &addr))
return 0;
#endif
diff --git a/arch/x86/boot/early_serial_console.c b/arch/x86/boot/early_serial_console.c
index 023bf1c3de8b..5b83beab89e1 100644
--- a/arch/x86/boot/early_serial_console.c
+++ b/arch/x86/boot/early_serial_console.c
@@ -117,7 +117,7 @@ static unsigned int probe_baud(int port)
static void parse_console_uart8250(void)
{
char optstr[64], *options;
- int baud = DEFAULT_BAUD;
+ int baud;
int port = 0;
/*
@@ -136,10 +136,13 @@ static void parse_console_uart8250(void)
else
return;
- if (options && (options[0] == ','))
- baud = simple_strtoull(options + 1, &options, 0);
- else
+ if (options && (options[0] == ',')) {
+ baud = simple_strtoull(options + 1, NULL, 0);
+ if (!baud)
+ baud = DEFAULT_BAUD;
+ } else {
baud = probe_baud(port);
+ }
if (port)
early_serial_init(port, baud);
diff --git a/arch/x86/events/amd/brs.c b/arch/x86/events/amd/brs.c
index 780acd3dff22..4350279b7c1e 100644
--- a/arch/x86/events/amd/brs.c
+++ b/arch/x86/events/amd/brs.c
@@ -259,13 +259,13 @@ void amd_brs_disable_all(void)
amd_brs_disable();
}
-static bool amd_brs_match_plm(struct perf_event *event, u64 to)
+static bool amd_brs_match_plm(struct perf_event *event, u64 from, u64 to)
{
int type = event->attr.branch_sample_type;
int plm_k = PERF_SAMPLE_BRANCH_KERNEL | PERF_SAMPLE_BRANCH_HV;
int plm_u = PERF_SAMPLE_BRANCH_USER;
- if (!(type & plm_k) && kernel_ip(to))
+ if (!(type & plm_k) && (kernel_ip(to) || kernel_ip(from)))
return 0;
if (!(type & plm_u) && !kernel_ip(to))
@@ -338,11 +338,11 @@ void amd_brs_drain(void)
*/
to = (u64)(((s64)to << shift) >> shift);
- if (!amd_brs_match_plm(event, to))
- continue;
-
rdmsrl(brs_from(brs_idx), from);
+ if (!amd_brs_match_plm(event, from, to))
+ continue;
+
perf_clear_branch_entry_bitfields(br+nr);
br[nr].from = from;
diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index 36d28edf7a53..09b973deed34 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -750,13 +750,11 @@ static void amd_pmu_enable_event(struct perf_event *event)
x86_pmu_enable_event(event);
}
-static void amd_pmu_enable_all(int added)
+static void __amd_pmu_enable_all(void)
{
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
int idx;
- amd_brs_enable_all();
-
for_each_set_bit(idx, x86_pmu.cntr_mask, X86_PMC_IDX_MAX) {
/* only activate events which are marked as active */
if (!test_bit(idx, cpuc->active_mask))
@@ -771,6 +769,12 @@ static void amd_pmu_enable_all(int added)
}
}
+static void amd_pmu_enable_all(int added)
+{
+ amd_brs_enable_all();
+ __amd_pmu_enable_all();
+}
+
static void amd_pmu_v2_enable_event(struct perf_event *event)
{
struct hw_perf_event *hwc = &event->hw;
@@ -1412,12 +1416,12 @@ static int __init amd_core_pmu_init(void)
u64 even_ctr_mask = 0ULL;
int i;
- if (!boot_cpu_has(X86_FEATURE_PERFCTR_CORE))
- return 0;
-
/* Avoid calculating the value each time in the NMI handler */
perf_nmi_window = msecs_to_jiffies(100);
+ if (!boot_cpu_has(X86_FEATURE_PERFCTR_CORE))
+ return 0;
+
/*
* If core performance counter extensions exists, we must use
* MSR_F15H_PERF_CTL/MSR_F15H_PERF_CTR msrs. See also
@@ -1557,7 +1561,7 @@ static inline void amd_pmu_reload_virt(void)
* set global enable bits once again
*/
amd_pmu_v2_disable_all();
- amd_pmu_enable_all(0);
+ __amd_pmu_enable_all();
amd_pmu_v2_enable_all(0);
return;
}
diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
index 19c7b76e21bc..01d462766281 100644
--- a/arch/x86/events/amd/lbr.c
+++ b/arch/x86/events/amd/lbr.c
@@ -124,7 +124,8 @@ static void amd_pmu_lbr_filter(void)
}
/* If type does not correspond, then discard */
- if (type == X86_BR_NONE || (br_sel & type) != type) {
+ if (type == X86_BR_NONE || (br_sel & type) != type ||
+ (!(br_sel & X86_BR_KERNEL) && kernel_ip(cpuc->lbr_entries[i].from))) {
cpuc->lbr_entries[i].from = 0; /* mark invalid */
compress = true;
}
diff --git a/arch/x86/events/amd/uncore.c b/arch/x86/events/amd/uncore.c
index 0bb60fde1f4f..cad240e88b4f 100644
--- a/arch/x86/events/amd/uncore.c
+++ b/arch/x86/events/amd/uncore.c
@@ -639,7 +639,7 @@ void amd_uncore_df_ctx_scan(struct amd_uncore *uncore, unsigned int cpu)
info.split.aux_data = 0;
info.split.num_pmcs = NUM_COUNTERS_NB;
info.split.gid = 0;
- info.split.cid = topology_logical_package_id(cpu);
+ info.split.cid = topology_amd_node_id(cpu);
if (pmu_version >= 2) {
ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES);
@@ -932,8 +932,8 @@ void amd_uncore_umc_ctx_scan(struct amd_uncore *uncore, unsigned int cpu)
cpuid(EXT_PERFMON_DEBUG_FEATURES, &eax, &ebx.full, &ecx, &edx);
info.split.aux_data = ecx; /* stash active mask */
info.split.num_pmcs = ebx.split.num_umc_pmc;
- info.split.gid = topology_logical_package_id(cpu);
- info.split.cid = topology_logical_package_id(cpu);
+ info.split.gid = topology_amd_node_id(cpu);
+ info.split.cid = topology_amd_node_id(cpu);
*per_cpu_ptr(uncore->info, cpu) = info;
}
diff --git a/arch/x86/events/intel/uncore_discovery.c b/arch/x86/events/intel/uncore_discovery.c
index aad35190ccf6..618481ad9f57 100644
--- a/arch/x86/events/intel/uncore_discovery.c
+++ b/arch/x86/events/intel/uncore_discovery.c
@@ -432,8 +432,8 @@ static u64 intel_generic_uncore_box_ctl(struct intel_uncore_box *box)
struct intel_uncore_discovery_unit *unit;
unit = intel_uncore_find_discovery_unit(box->pmu->type->boxes,
- -1, box->pmu->pmu_idx);
- if (WARN_ON_ONCE(!unit))
+ box->dieid, box->pmu->pmu_idx);
+ if (!unit)
return 0;
return unit->addr;
diff --git a/arch/x86/events/intel/uncore_snb.c b/arch/x86/events/intel/uncore_snb.c
index 3934e1e4e3b1..c64e4f333563 100644
--- a/arch/x86/events/intel/uncore_snb.c
+++ b/arch/x86/events/intel/uncore_snb.c
@@ -538,12 +538,6 @@ void tgl_uncore_cpu_init(void)
skl_uncore_msr_ops.init_box = rkl_uncore_msr_init_box;
}
-static void adl_uncore_msr_init_box(struct intel_uncore_box *box)
-{
- if (box->pmu->pmu_idx == 0)
- wrmsrl(ADL_UNC_PERF_GLOBAL_CTL, SNB_UNC_GLOBAL_CTL_EN);
-}
-
static void adl_uncore_msr_enable_box(struct intel_uncore_box *box)
{
wrmsrl(ADL_UNC_PERF_GLOBAL_CTL, SNB_UNC_GLOBAL_CTL_EN);
@@ -562,7 +556,6 @@ static void adl_uncore_msr_exit_box(struct intel_uncore_box *box)
}
static struct intel_uncore_ops adl_uncore_msr_ops = {
- .init_box = adl_uncore_msr_init_box,
.enable_box = adl_uncore_msr_enable_box,
.disable_box = adl_uncore_msr_disable_box,
.exit_box = adl_uncore_msr_exit_box,
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index f2cc7754918c..2e0812a8ed77 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -405,6 +405,10 @@ extern void srso_alias_return_thunk(void);
extern void entry_untrain_ret(void);
extern void entry_ibpb(void);
+#ifdef CONFIG_BPF_JIT
+extern void bpf_arch_ibpb(void);
+#endif
+
#ifdef CONFIG_X86_64
extern void clear_bhb_loop(void);
#endif
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 939401b5d2ef..12544fdcadd6 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -16,6 +16,7 @@
#include <linux/sched/smt.h>
#include <linux/pgtable.h>
#include <linux/bpf.h>
+#include <linux/filter.h>
#include <asm/spec-ctrl.h>
#include <asm/cmdline.h>
@@ -1360,8 +1361,21 @@ static inline const char *spectre_v2_module_string(void)
{
return spectre_v2_bad_module ? " - vulnerable module loaded" : "";
}
+
+/*
+ * The "retpoline sequence" is the "call;mov;ret" sequence that
+ * replaces normal indirect branch instructions. Differentiate
+ * *the* retpoline sequence from the LFENCE-prefixed indirect
+ * branches that simply use the retpoline infrastructure.
+ */
+static inline bool retpoline_seq_enabled(void)
+{
+ return boot_cpu_has(X86_FEATURE_RETPOLINE) && !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE);
+}
+
#else
static inline const char *spectre_v2_module_string(void) { return ""; }
+static inline bool retpoline_seq_enabled(void) { return false; }
#endif
#define SPECTRE_V2_LFENCE_MSG "WARNING: LFENCE mitigation is not recommended for this CPU, data leaks possible!\n"
@@ -1843,8 +1857,7 @@ static void __init bhi_select_mitigation(void)
return;
/* Retpoline mitigates against BHI unless the CPU has RRSBA behavior */
- if (boot_cpu_has(X86_FEATURE_RETPOLINE) &&
- !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE)) {
+ if (retpoline_seq_enabled()) {
spec_ctrl_disable_kernel_rrsba();
if (rrsba_disabled)
return;
@@ -1868,6 +1881,27 @@ static void __init bhi_select_mitigation(void)
setup_force_cpu_cap(X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT);
}
+#ifdef CONFIG_BPF_JIT
+static void __bpf_arch_ibpb(void *unused)
+{
+ entry_ibpb();
+}
+
+void bpf_arch_ibpb(void)
+{
+ on_each_cpu(__bpf_arch_ibpb, NULL, 1);
+}
+
+static bool __init cpu_wants_ibpb_bpf(void)
+{
+ /* A genuine retpoline already neutralizes ring0 indirect predictions */
+ if (retpoline_seq_enabled())
+ return false;
+
+ return boot_cpu_has(X86_FEATURE_IBPB);
+}
+#endif
+
static void __init spectre_v2_select_mitigation(void)
{
enum spectre_v2_mitigation_cmd cmd = spectre_v2_parse_cmdline();
@@ -2051,6 +2085,14 @@ static void __init spectre_v2_select_mitigation(void)
pr_info("Enabling Restricted Speculation for firmware calls\n");
}
+#ifdef CONFIG_BPF_JIT
+ if (cpu_wants_ibpb_bpf()) {
+ static_call_update(bpf_arch_pred_flush, bpf_arch_ibpb);
+ static_branch_enable(&bpf_pred_flush_enabled);
+ pr_info("Enabling IBPB for BPF\n");
+ }
+#endif
+
/* Set up IBPB and STIBP depending on the general spectre V2 command */
spectre_v2_cmd = cmd;
}
@@ -3239,9 +3281,7 @@ static const char *spectre_bhi_state(void)
return "; BHI: BHI_DIS_S";
else if (boot_cpu_has(X86_FEATURE_CLEAR_BHB_LOOP))
return "; BHI: SW loop, KVM: SW loop";
- else if (boot_cpu_has(X86_FEATURE_RETPOLINE) &&
- !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE) &&
- rrsba_disabled)
+ else if (retpoline_seq_enabled() && rrsba_disabled)
return "; BHI: Retpoline";
else if (boot_cpu_has(X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT))
return "; BHI: Vulnerable, KVM: SW loop";
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 7bc38fed2072..ef9c8135311d 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -3169,6 +3169,12 @@ static void apic_sync_pv_eoi_from_guest(struct kvm_vcpu *vcpu,
struct kvm_lapic *apic)
{
int vector;
+
+ if (unlikely(!pv_eoi_enabled(vcpu))) {
+ __clear_bit(KVM_APIC_PV_EOI_PENDING, &vcpu->arch.apic_attention);
+ return;
+ }
+
/*
* PV EOI state is derived from KVM_APIC_PV_EOI_PENDING in host
* and KVM_PV_EOI_ENABLED in guest memory as follows:
@@ -3180,8 +3186,6 @@ static void apic_sync_pv_eoi_from_guest(struct kvm_vcpu *vcpu,
* KVM_APIC_PV_EOI_PENDING is set, KVM_PV_EOI_ENABLED is unset:
* -> host enabled PV EOI, guest executed EOI.
*/
- BUG_ON(!pv_eoi_enabled(vcpu));
-
if (pv_eoi_test_and_clr_pending(vcpu))
return;
vector = apic_set_eoi(apic);
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index ccfa4924196a..6cec696461da 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -2101,8 +2101,9 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
if (ret)
return ret;
+ /* Do not allow SNP VM migration until additional state transfer is implemented */
if (kvm->arch.vm_type != source_kvm->arch.vm_type ||
- sev_guest(kvm) || !sev_guest(source_kvm)) {
+ sev_guest(kvm) || !sev_guest(source_kvm) || sev_snp_guest(source_kvm)) {
ret = -EINVAL;
goto out_unlock;
}
@@ -2830,8 +2831,9 @@ int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd)
* disallow out-of-band SEV/SEV-ES init if the target is already an
* SEV guest, or if vCPUs have been created. KVM relies on vCPUs being
* created after SEV/SEV-ES initialization, e.g. to init intercepts.
+ * Also do not allow SNP VM mirroring until additional state transfer is implemented.
*/
- if (sev_guest(kvm) || !sev_guest(source_kvm) ||
+ if (sev_guest(kvm) || !sev_guest(source_kvm) || sev_snp_guest(source_kvm) ||
is_mirroring_enc_context(source_kvm) || kvm->created_vcpus) {
ret = -EINVAL;
goto e_unlock;
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 8629d90fdcd9..ed182831063c 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -606,6 +606,14 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next,
*/
cond_mitigation(tsk);
+ /*
+ * Indicate that CR3 is about to change. nmi_uaccess_okay()
+ * and others are sensitive to the window where mm_cpumask(),
+ * CR3 and cpu_tlbstate.loaded_mm are not all in sync.
+ */
+ this_cpu_write(cpu_tlbstate.loaded_mm, LOADED_MM_SWITCHING);
+ barrier();
+
/*
* Stop remote flushes for the previous mm.
* Skip kernel threads; we never send init_mm TLB flushing IPIs,
@@ -623,14 +631,6 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next,
next_tlb_gen = atomic64_read(&next->context.tlb_gen);
choose_new_asid(next, next_tlb_gen, &new_asid, &need_flush);
-
- /*
- * Indicate that CR3 is about to change. nmi_uaccess_okay()
- * and others are sensitive to the window where mm_cpumask(),
- * CR3 and cpu_tlbstate.loaded_mm are not all in sync.
- */
- this_cpu_write(cpu_tlbstate.loaded_mm, LOADED_MM_SWITCHING);
- barrier();
}
new_lam = mm_lam_cr3_mask(next);
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 8cbc26081bdb..4304596c8fdd 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3206,7 +3206,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, jit_fill_hole);
+ return bpf_prog_pack_alloc(size, jit_fill_hole, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
@@ -3491,7 +3491,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* allocate module memory for x86 insns and extable */
header = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,
&image, align, &rw_header, &rw_image,
- jit_fill_hole);
+ jit_fill_hole,
+ bpf_prog_was_classic(prog));
if (!header) {
prog = orig_prog;
goto out_addrs;
diff --git a/arch/x86/platform/olpc/olpc-xo15-sci.c b/arch/x86/platform/olpc/olpc-xo15-sci.c
index 68244a3422d1..683935da493d 100644
--- a/arch/x86/platform/olpc/olpc-xo15-sci.c
+++ b/arch/x86/platform/olpc/olpc-xo15-sci.c
@@ -185,6 +185,7 @@ static int xo15_sci_add(struct acpi_device *device)
static void xo15_sci_remove(struct acpi_device *device)
{
+ device_init_wakeup(&device->dev, false);
acpi_disable_gpe(NULL, xo15_sci_gpe);
acpi_remove_gpe_handler(NULL, xo15_sci_gpe, xo15_sci_gpe_handler);
cancel_work_sync(&sci_work);
diff --git a/block/blk.h b/block/blk.h
index 2a355ed506c5..b3a7e518d799 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -698,6 +698,7 @@ static inline int req_ref_read(struct request *req)
static inline u64 blk_time_get_ns(void)
{
struct blk_plug *plug = current->plug;
+ u64 now;
if (!plug || !in_task())
return ktime_get_ns();
@@ -706,12 +707,18 @@ static inline u64 blk_time_get_ns(void)
* 0 could very well be a valid time, but rather than flag "this is
* a valid timestamp" separately, just accept that we'll do an extra
* ktime_get_ns() if we just happen to get 0 as the current time.
+ *
+ * cur_ktime can be zeroed by pre-emption the moment PF_BLOCK_TS is set.
*/
- if (!plug->cur_ktime) {
- plug->cur_ktime = ktime_get_ns();
+ now = READ_ONCE(plug->cur_ktime);
+ if (!now) {
+ now = ktime_get_ns();
+ WRITE_ONCE(plug->cur_ktime, now);
+ /* Ensure PF_BLOCK_TS is set after cur_ktime. */
+ barrier();
current->flags |= PF_BLOCK_TS;
}
- return plug->cur_ktime;
+ return now;
}
static inline ktime_t blk_time_get(void)
diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c
index ba0a17fd95ac..35ebc3e0201b 100644
--- a/crypto/algif_skcipher.c
+++ b/crypto/algif_skcipher.c
@@ -79,20 +79,6 @@ static int algif_skcipher_export(struct sock *sk, struct skcipher_request *req)
return err;
}
-static void algif_skcipher_done(void *data, int err)
-{
- struct af_alg_async_req *areq = data;
- struct sock *sk = areq->sk;
-
- if (err)
- goto out;
-
- err = algif_skcipher_export(sk, &areq->cra_u.skcipher_req);
-
-out:
- af_alg_async_cb(data, err);
-}
-
static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg,
size_t ignored, int flags)
{
@@ -171,43 +157,30 @@ static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg,
cflags |= CRYPTO_SKCIPHER_REQ_CONT;
}
- if (msg->msg_iocb && !is_sync_kiocb(msg->msg_iocb)) {
- /* AIO operation */
- sock_hold(sk);
- areq->iocb = msg->msg_iocb;
-
- /* Remember output size that will be generated. */
- areq->outlen = len;
-
- skcipher_request_set_callback(&areq->cra_u.skcipher_req,
- cflags |
- CRYPTO_TFM_REQ_MAY_SLEEP,
- algif_skcipher_done, areq);
- err = ctx->enc ?
- crypto_skcipher_encrypt(&areq->cra_u.skcipher_req) :
- crypto_skcipher_decrypt(&areq->cra_u.skcipher_req);
-
- /* AIO operation in progress */
- if (err == -EINPROGRESS)
- return -EIOCBQUEUED;
-
- sock_put(sk);
- } else {
- /* Synchronous operation */
- skcipher_request_set_callback(&areq->cra_u.skcipher_req,
- cflags |
- CRYPTO_TFM_REQ_MAY_SLEEP |
- CRYPTO_TFM_REQ_MAY_BACKLOG,
- crypto_req_done, &ctx->wait);
- err = crypto_wait_req(ctx->enc ?
- crypto_skcipher_encrypt(&areq->cra_u.skcipher_req) :
- crypto_skcipher_decrypt(&areq->cra_u.skcipher_req),
- &ctx->wait);
-
- if (!err)
- err = algif_skcipher_export(
- sk, &areq->cra_u.skcipher_req);
- }
+ /*
+ * Force synchronous processing. The async (AIO) path passed the
+ * socket-wide ctx->iv into the request, which the worker
+ * dereferenced after the socket lock had been dropped, letting a
+ * concurrent sendmsg(ALG_SET_IV) inject an attacker IV. Mainline
+ * removed the AIO socket path in commit fcc77d33a34c ("net: Remove
+ * support for AIO on sockets"); the minimal stable fix is to always
+ * complete synchronously, so ctx->iv is only ever dereferenced under
+ * the socket lock. This also keeps the IV chaining intact: for
+ * ciphers with statesize == 0 (e.g. ctr, cbc) the chained IV is
+ * carried by the req->iv writeback into ctx->iv, which is only
+ * consistent on the synchronous path.
+ */
+ skcipher_request_set_callback(&areq->cra_u.skcipher_req,
+ cflags |
+ CRYPTO_TFM_REQ_MAY_SLEEP |
+ CRYPTO_TFM_REQ_MAY_BACKLOG,
+ crypto_req_done, &ctx->wait);
+ err = crypto_wait_req(ctx->enc ?
+ crypto_skcipher_encrypt(&areq->cra_u.skcipher_req) :
+ crypto_skcipher_decrypt(&areq->cra_u.skcipher_req),
+ &ctx->wait);
+ if (!err)
+ err = algif_skcipher_export(sk, &areq->cra_u.skcipher_req);
free:
af_alg_free_resources(areq);
diff --git a/crypto/asymmetric_keys/verify_pefile.c b/crypto/asymmetric_keys/verify_pefile.c
index 2863984b6700..95ba0a91dbbd 100644
--- a/crypto/asymmetric_keys/verify_pefile.c
+++ b/crypto/asymmetric_keys/verify_pefile.c
@@ -305,6 +305,8 @@ static int pefile_digest_pe_contents(const void *pebuf, unsigned int pelen,
if (pelen > hashed_bytes) {
tmp = hashed_bytes + ctx->certs_size;
+ if (tmp <= hashed_bytes || pelen < tmp)
+ return -ELIBBAD;
ret = crypto_shash_update(desc,
pebuf + hashed_bytes,
pelen - tmp);
diff --git a/crypto/ecrdsa.c b/crypto/ecrdsa.c
index 3811f3805b5d..38271526d1f8 100644
--- a/crypto/ecrdsa.c
+++ b/crypto/ecrdsa.c
@@ -153,7 +153,7 @@ int ecrdsa_param_curve(void *context, size_t hdrlen, unsigned char tag,
struct ecrdsa_ctx *ctx = context;
ctx->curve_oid = look_up_OID(value, vlen);
- if (!ctx->curve_oid)
+ if (ctx->curve_oid == OID__NR)
return -EINVAL;
ctx->curve = get_curve_by_oid(ctx->curve_oid);
return 0;
diff --git a/crypto/rng.c b/crypto/rng.c
index 72da96fdfb5d..b0bc13d14db5 100644
--- a/crypto/rng.c
+++ b/crypto/rng.c
@@ -225,5 +225,16 @@ void crypto_unregister_rngs(struct rng_alg *algs, int count)
}
EXPORT_SYMBOL_GPL(crypto_unregister_rngs);
+static void __exit rng_exit(void)
+{
+ int err;
+
+ err = crypto_del_default_rng();
+ if (err)
+ pr_err("Failed delete default RNG: %d\n", err);
+}
+
+module_exit(rng_exit);
+
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Random Number Generator");
diff --git a/drivers/accel/ivpu/ivpu_fw_log.c b/drivers/accel/ivpu/ivpu_fw_log.c
index 275baf844b56..716467aa3156 100644
--- a/drivers/accel/ivpu/ivpu_fw_log.c
+++ b/drivers/accel/ivpu/ivpu_fw_log.c
@@ -43,6 +43,10 @@ static int fw_log_from_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, u32 *off
ivpu_dbg(vdev, FW_BOOT, "Invalid header size 0x%x\n", log->header_size);
return -EINVAL;
}
+ if (log->size < log->header_size) {
+ ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", log->size);
+ return -EINVAL;
+ }
if ((char *)log + log->size > (char *)ivpu_bo_vaddr(bo) + ivpu_bo_size(bo)) {
ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", log->size);
return -EINVAL;
diff --git a/drivers/acpi/ac.c b/drivers/acpi/ac.c
index 7c5b040a83e8..71ccd0736622 100644
--- a/drivers/acpi/ac.c
+++ b/drivers/acpi/ac.c
@@ -203,11 +203,15 @@ static const struct dmi_system_id ac_dmi_table[] __initconst = {
static int acpi_ac_probe(struct platform_device *pdev)
{
- struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
struct power_supply_config psy_cfg = {};
+ struct acpi_device *adev;
struct acpi_ac *ac;
int result;
+ adev = ACPI_COMPANION(&pdev->dev);
+ if (!adev)
+ return -ENODEV;
+
ac = kzalloc(sizeof(struct acpi_ac), GFP_KERNEL);
if (!ac)
return -ENOMEM;
diff --git a/drivers/acpi/acpi_ipmi.c b/drivers/acpi/acpi_ipmi.c
index 5fba4dab5d08..cf0ee52d4126 100644
--- a/drivers/acpi/acpi_ipmi.c
+++ b/drivers/acpi/acpi_ipmi.c
@@ -490,7 +490,7 @@ static void ipmi_bmc_gone(int iface)
mutex_lock(&driver_data.ipmi_lock);
list_for_each_entry_safe(iter, temp,
&driver_data.ipmi_devices, head) {
- if (iter->ipmi_ifnum != iface) {
+ if (iter->ipmi_ifnum == iface) {
ipmi_device = iter;
__ipmi_dev_kill(iter);
break;
@@ -550,7 +550,6 @@ acpi_ipmi_space_handler(u32 function, acpi_physical_address address,
return AE_TYPE;
}
- acpi_ipmi_msg_get(tx_msg);
mutex_lock(&driver_data.ipmi_lock);
/* Do not add a tx_msg that can not be flushed. */
if (ipmi_device->dead) {
@@ -558,6 +557,7 @@ acpi_ipmi_space_handler(u32 function, acpi_physical_address address,
ipmi_msg_release(tx_msg);
return AE_NOT_EXIST;
}
+ acpi_ipmi_msg_get(tx_msg);
spin_lock_irqsave(&ipmi_device->tx_msg_lock, flags);
list_add_tail(&tx_msg->head, &ipmi_device->tx_msg_list);
spin_unlock_irqrestore(&ipmi_device->tx_msg_lock, flags);
diff --git a/drivers/acpi/acpi_pad.c b/drivers/acpi/acpi_pad.c
index 42b7220d4cfd..3207f3f536db 100644
--- a/drivers/acpi/acpi_pad.c
+++ b/drivers/acpi/acpi_pad.c
@@ -427,9 +427,13 @@ static void acpi_pad_notify(acpi_handle handle, u32 event,
static int acpi_pad_probe(struct platform_device *pdev)
{
- struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
+ struct acpi_device *adev;
acpi_status status;
+ adev = ACPI_COMPANION(&pdev->dev);
+ if (!adev)
+ return -ENODEV;
+
strscpy(acpi_device_name(adev), ACPI_PROCESSOR_AGGREGATOR_DEVICE_NAME);
strscpy(acpi_device_class(adev), ACPI_PROCESSOR_AGGREGATOR_CLASS);
diff --git a/drivers/acpi/acpi_tad.c b/drivers/acpi/acpi_tad.c
index b75ceaab716e..3ceec703548b 100644
--- a/drivers/acpi/acpi_tad.c
+++ b/drivers/acpi/acpi_tad.c
@@ -588,12 +588,16 @@ static void acpi_tad_remove(struct platform_device *pdev)
static int acpi_tad_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
- acpi_handle handle = ACPI_HANDLE(dev);
struct acpi_tad_driver_data *dd;
+ acpi_handle handle;
acpi_status status;
unsigned long long caps;
int ret;
+ handle = ACPI_HANDLE(dev);
+ if (!handle)
+ return -ENODEV;
+
ret = acpi_install_cmos_rtc_space_handler(handle);
if (ret < 0) {
dev_info(dev, "Unable to install space handler\n");
diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
index 6234055d2598..9deb98c0a6fe 100644
--- a/drivers/acpi/bus.c
+++ b/drivers/acpi/bus.c
@@ -622,6 +622,76 @@ void acpi_dev_remove_notify_handler(struct acpi_device *adev,
}
EXPORT_SYMBOL_GPL(acpi_dev_remove_notify_handler);
+struct acpi_notify_handler_devres {
+ struct acpi_device *adev;
+ acpi_notify_handler handler;
+ u32 handler_type;
+};
+
+static void devm_acpi_notify_handler_release(struct device *dev, void *res)
+{
+ struct acpi_notify_handler_devres *dr = res;
+
+ acpi_dev_remove_notify_handler(dr->adev, dr->handler_type,
+ dr->handler);
+}
+
+/**
+ * devm_acpi_install_notify_handler - Install an ACPI notify handler for a
+ * managed device
+ * @dev: Device to install a notify handler for
+ * @handler_type: Type of the notify handler
+ * @handler: Handler function to install
+ * @context: Data passed back to the handler function
+ *
+ * This function performs the same function as acpi_dev_install_notify_handler()
+ * called for the ACPI companion of @dev with the same @handler_type, @handler,
+ * and @context arguments, but the ACPI notify handler installed by it will be
+ * automatically removed on driver detach.
+ *
+ * Callers should ensure that all resources used by @handler have been allocated
+ * prior to invoking this function, in which case those resources should be
+ * devres-managed so that they won't be released before the notify handler
+ * removal. Otherwise, special synchronization between @handler and the
+ * management of those resources is required.
+ *
+ * When the request fails, an error message is printed. Don't add extra error
+ * messages at the call sites.
+ *
+ * Return: 0 on success or a negative error number.
+ */
+int devm_acpi_install_notify_handler(struct device *dev, u32 handler_type,
+ acpi_notify_handler handler, void *context)
+{
+ struct acpi_notify_handler_devres *dr;
+ struct acpi_device *adev;
+ int ret;
+
+ adev = ACPI_COMPANION(dev);
+ if (!adev && dev->bus == &acpi_bus_type)
+ adev = to_acpi_device(dev);
+ if (!adev)
+ return dev_err_probe(dev, -ENODEV, "No ACPI companion in %s()\n", __func__);
+
+ dr = devres_alloc(devm_acpi_notify_handler_release, sizeof(*dr), GFP_KERNEL);
+ if (!dr)
+ return -ENOMEM;
+
+ ret = acpi_dev_install_notify_handler(adev, handler_type, handler, context);
+ if (ret) {
+ devres_free(dr);
+ return dev_err_probe(dev, ret, "Failed to install an ACPI notify handler\n");
+ }
+
+ dr->adev = adev;
+ dr->handler = handler;
+ dr->handler_type = handler_type;
+ devres_add(dev, dr);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(devm_acpi_install_notify_handler);
+
/* Handle events targeting \_SB device (at present only graceful shutdown) */
#define ACPI_SB_NOTIFY_SHUTDOWN_REQUEST 0x81
diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c
index 2eb9989372ca..633eb625f037 100644
--- a/drivers/acpi/nfit/core.c
+++ b/drivers/acpi/nfit/core.c
@@ -55,6 +55,8 @@ MODULE_PARM_DESC(force_labels, "Opt-in to labels despite missing methods");
LIST_HEAD(acpi_descs);
DEFINE_MUTEX(acpi_desc_lock);
+DEFINE_MUTEX(acpi_notify_lock);
+
static struct workqueue_struct *nfit_wq;
struct nfit_table_prev {
@@ -1702,9 +1704,15 @@ static void acpi_nvdimm_notify(acpi_handle handle, u32 event, void *data)
struct acpi_device *adev = data;
struct device *dev = &adev->dev;
- device_lock(dev->parent);
- __acpi_nvdimm_notify(dev, event);
- device_unlock(dev->parent);
+ /*
+ * Locking is needed here for synchronization with driver probe and
+ * removal and the parent NFIT device's ACPI driver data pointer is
+ * NULL when teardown is in progress.
+ */
+ guard(mutex)(&acpi_notify_lock);
+
+ if (acpi_driver_data(to_acpi_device(dev->parent)))
+ __acpi_nvdimm_notify(dev, event);
}
static bool acpi_nvdimm_has_method(struct acpi_device *adev, char *method)
@@ -3061,6 +3069,8 @@ static void acpi_nfit_unregister(void *data)
struct acpi_nfit_desc *acpi_desc = data;
nvdimm_bus_unregister(acpi_desc->nvdimm_bus);
+ /* The nvdimm_bus object may have been freed, so clear the pointer. */
+ acpi_desc->nvdimm_bus = NULL;
}
int acpi_nfit_init(struct acpi_nfit_desc *acpi_desc, void *data, acpi_size sz)
@@ -3148,11 +3158,10 @@ EXPORT_SYMBOL_GPL(acpi_nfit_init);
static int acpi_nfit_flush_probe(struct nvdimm_bus_descriptor *nd_desc)
{
struct acpi_nfit_desc *acpi_desc = to_acpi_desc(nd_desc);
- struct device *dev = acpi_desc->dev;
- /* Bounce the device lock to flush acpi_nfit_add / acpi_nfit_notify */
- device_lock(dev);
- device_unlock(dev);
+ /* Bounce the notify lock to flush acpi_nfit_add / acpi_nfit_notify */
+ mutex_lock(&acpi_notify_lock);
+ mutex_unlock(&acpi_notify_lock);
/* Bounce the init_mutex to complete initial registration */
mutex_lock(&acpi_desc->init_mutex);
@@ -3283,25 +3292,27 @@ static void acpi_nfit_put_table(void *table)
static void acpi_nfit_notify(acpi_handle handle, u32 event, void *data)
{
- struct acpi_device *adev = data;
+ struct device *dev = data;
+ struct acpi_device *adev = to_acpi_device(dev);
- device_lock(&adev->dev);
- __acpi_nfit_notify(&adev->dev, handle, event);
- device_unlock(&adev->dev);
-}
-
-static void acpi_nfit_remove_notify_handler(void *data)
-{
- struct acpi_device *adev = data;
+ /*
+ * Locking is needed here for synchronization with driver probe and
+ * removal and the ACPI driver data pointer is NULL when teardown
+ * is in progress.
+ */
+ guard(mutex)(&acpi_notify_lock);
- acpi_dev_remove_notify_handler(adev, ACPI_DEVICE_NOTIFY,
- acpi_nfit_notify);
+ if (acpi_driver_data(adev))
+ __acpi_nfit_notify(dev, handle, event);
}
void acpi_nfit_shutdown(void *data)
{
struct acpi_nfit_desc *acpi_desc = data;
- struct device *bus_dev = to_nvdimm_bus_dev(acpi_desc->nvdimm_bus);
+ struct device *bus_dev;
+
+ if (!acpi_desc || !acpi_desc->nvdimm_bus)
+ return;
/*
* Destruct under acpi_desc_lock so that nfit_handle_mce does not
@@ -3316,6 +3327,7 @@ void acpi_nfit_shutdown(void *data)
mutex_unlock(&acpi_desc->init_mutex);
cancel_delayed_work_sync(&acpi_desc->dwork);
+ bus_dev = to_nvdimm_bus_dev(acpi_desc->nvdimm_bus);
/*
* Bounce the nvdimm bus lock to make sure any in-flight
* acpi_nfit_ars_rescan() submissions have had a chance to
@@ -3338,13 +3350,14 @@ static int acpi_nfit_add(struct acpi_device *adev)
acpi_size sz;
int rc = 0;
- rc = acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY,
- acpi_nfit_notify, adev);
- if (rc)
- return rc;
+ /*
+ * Prevent acpi_nfit_notify() from progressing until the probe is
+ * complete in case there is a concurrent event to process.
+ */
+ guard(mutex)(&acpi_notify_lock);
- rc = devm_add_action_or_reset(dev, acpi_nfit_remove_notify_handler,
- adev);
+ rc = devm_acpi_install_notify_handler(dev, ACPI_DEVICE_NOTIFY,
+ acpi_nfit_notify, dev);
if (rc)
return rc;
@@ -3358,6 +3371,11 @@ static int acpi_nfit_add(struct acpi_device *adev)
* data in the format of a series of NFIT Structures.
*/
dev_dbg(dev, "failed to find NFIT at startup\n");
+ /*
+ * Let acpi_nfit_update_notify() run in case it will need to
+ * allocate the acpi_desc object.
+ */
+ adev->driver_data = dev;
return 0;
}
@@ -3392,10 +3410,26 @@ static int acpi_nfit_add(struct acpi_device *adev)
+ sizeof(struct acpi_table_nfit),
sz - sizeof(struct acpi_table_nfit));
- if (rc)
+ if (rc) {
+ acpi_nfit_shutdown(acpi_desc);
return rc;
+ }
+
+ /*
+ * Let notify handlers operate (the actual value of the ACPI driver
+ * data pointer does not matter here so long as it is not NULL).
+ */
+ adev->driver_data = dev;
+ return 0;
+}
+
+static void acpi_nfit_remove(struct acpi_device *adev)
+{
+ guard(mutex)(&acpi_notify_lock);
- return devm_add_action_or_reset(dev, acpi_nfit_shutdown, acpi_desc);
+ /* Make notify handlers bail out early going forward. */
+ adev->driver_data = NULL;
+ acpi_nfit_shutdown(dev_get_drvdata(&adev->dev));
}
static void acpi_nfit_update_notify(struct device *dev, acpi_handle handle)
@@ -3482,6 +3516,7 @@ static struct acpi_driver acpi_nfit_driver = {
.ids = acpi_nfit_ids,
.ops = {
.add = acpi_nfit_add,
+ .remove = acpi_nfit_remove,
},
};
diff --git a/drivers/acpi/pfr_telemetry.c b/drivers/acpi/pfr_telemetry.c
index 998264a7333d..717361da4201 100644
--- a/drivers/acpi/pfr_telemetry.c
+++ b/drivers/acpi/pfr_telemetry.c
@@ -363,10 +363,14 @@ static void pfrt_log_put_idx(void *data)
static int acpi_pfrt_log_probe(struct platform_device *pdev)
{
- acpi_handle handle = ACPI_HANDLE(&pdev->dev);
struct pfrt_log_device *pfrt_log_dev;
+ acpi_handle handle;
int ret;
+ handle = ACPI_HANDLE(&pdev->dev);
+ if (!handle)
+ return -ENODEV;
+
if (!acpi_has_method(handle, "_DSM")) {
dev_dbg(&pdev->dev, "Missing _DSM\n");
return -ENODEV;
diff --git a/drivers/acpi/pfr_update.c b/drivers/acpi/pfr_update.c
index 35c7b04bc9d3..4aa179ca82c7 100644
--- a/drivers/acpi/pfr_update.c
+++ b/drivers/acpi/pfr_update.c
@@ -505,10 +505,14 @@ static void pfru_put_idx(void *data)
static int acpi_pfru_probe(struct platform_device *pdev)
{
- acpi_handle handle = ACPI_HANDLE(&pdev->dev);
struct pfru_device *pfru_dev;
+ acpi_handle handle;
int ret;
+ handle = ACPI_HANDLE(&pdev->dev);
+ if (!handle)
+ return -ENODEV;
+
if (!acpi_has_method(handle, "_DSM")) {
dev_dbg(&pdev->dev, "Missing _DSM\n");
return -ENODEV;
diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c
index b524cf27213d..b8a8cbf8fafd 100644
--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -1167,7 +1167,7 @@ static int acpi_processor_get_lpi_info(struct acpi_processor *pr)
return 0;
}
-int __weak acpi_processor_ffh_lpi_enter(struct acpi_lpi_state *lpi)
+int __weak __cpuidle acpi_processor_ffh_lpi_enter(struct acpi_lpi_state *lpi)
{
return -ENODEV;
}
@@ -1180,7 +1180,7 @@ int __weak acpi_processor_ffh_lpi_enter(struct acpi_lpi_state *lpi)
*
* Return: 0 for success or negative value for error
*/
-static int acpi_idle_lpi_enter(struct cpuidle_device *dev,
+static int __cpuidle acpi_idle_lpi_enter(struct cpuidle_device *dev,
struct cpuidle_driver *drv, int index)
{
struct acpi_processor *pr;
diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c
index 8a01a3718aa9..a15f390fa360 100644
--- a/drivers/acpi/resource.c
+++ b/drivers/acpi/resource.c
@@ -863,7 +863,7 @@ bool acpi_dev_resource_interrupt(struct acpi_resource *ares, int index,
EXPORT_SYMBOL_GPL(acpi_dev_resource_interrupt);
/**
- * acpi_dev_free_resource_list - Free resource from %acpi_dev_get_resources().
+ * acpi_dev_free_resource_list - Free resource from acpi_dev_get_resources().
* @list: The head of the resource list to free.
*/
void acpi_dev_free_resource_list(struct list_head *list)
@@ -983,7 +983,7 @@ static int __acpi_dev_get_resources(struct acpi_device *adev,
*
* The resultant struct resource objects are put on the list pointed to by
* @list, that must be empty initially, as members of struct resource_entry
- * objects. Callers of this routine should use %acpi_dev_free_resource_list() to
+ * objects. Callers of this routine should use acpi_dev_free_resource_list() to
* free that list.
*
* The number of resources in the output list is returned on success, an error
@@ -1024,7 +1024,7 @@ static int is_memory(struct acpi_resource *ares, void *not_used)
* The resultant struct resource objects are put on the list pointed to
* by @list, that must be empty initially, as members of struct
* resource_entry objects. Callers of this routine should use
- * %acpi_dev_free_resource_list() to free that list.
+ * acpi_dev_free_resource_list() to free that list.
*
* The number of resources in the output list is returned on success,
* an error code reflecting the error condition is returned otherwise.
diff --git a/drivers/acpi/riscv/cpuidle.c b/drivers/acpi/riscv/cpuidle.c
index 624f9bbdb58c..c76dbabff702 100644
--- a/drivers/acpi/riscv/cpuidle.c
+++ b/drivers/acpi/riscv/cpuidle.c
@@ -66,7 +66,7 @@ int acpi_processor_ffh_lpi_probe(unsigned int cpu)
return acpi_cpu_init_idle(cpu);
}
-int acpi_processor_ffh_lpi_enter(struct acpi_lpi_state *lpi)
+int __cpuidle acpi_processor_ffh_lpi_enter(struct acpi_lpi_state *lpi)
{
u32 state = lpi->address;
diff --git a/drivers/acpi/thermal.c b/drivers/acpi/thermal.c
index 125d7df8f30a..4f213486e6f9 100644
--- a/drivers/acpi/thermal.c
+++ b/drivers/acpi/thermal.c
@@ -789,7 +789,7 @@ static int acpi_thermal_add(struct acpi_device *device)
int i;
if (!device)
- return -EINVAL;
+ return -ENODEV;
tz = kzalloc(sizeof(struct acpi_thermal), GFP_KERNEL);
if (!tz)
diff --git a/drivers/amba/bus.c b/drivers/amba/bus.c
index 0230c43377c1..cc33d02af8fb 100644
--- a/drivers/amba/bus.c
+++ b/drivers/amba/bus.c
@@ -82,33 +82,6 @@ static void amba_put_disable_pclk(struct amba_device *pcdev)
}
-static ssize_t driver_override_show(struct device *_dev,
- struct device_attribute *attr, char *buf)
-{
- struct amba_device *dev = to_amba_device(_dev);
- ssize_t len;
-
- device_lock(_dev);
- len = sprintf(buf, "%s\n", dev->driver_override);
- device_unlock(_dev);
- return len;
-}
-
-static ssize_t driver_override_store(struct device *_dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct amba_device *dev = to_amba_device(_dev);
- int ret;
-
- ret = driver_set_override(_dev, &dev->driver_override, buf, count);
- if (ret)
- return ret;
-
- return count;
-}
-static DEVICE_ATTR_RW(driver_override);
-
#define amba_attr_func(name,fmt,arg...) \
static ssize_t name##_show(struct device *_dev, \
struct device_attribute *attr, char *buf) \
@@ -126,7 +99,6 @@ amba_attr_func(resource, "\t%016llx\t%016llx\t%016lx\n",
static struct attribute *amba_dev_attrs[] = {
&dev_attr_id.attr,
&dev_attr_resource.attr,
- &dev_attr_driver_override.attr,
NULL,
};
ATTRIBUTE_GROUPS(amba_dev);
@@ -209,10 +181,11 @@ static int amba_match(struct device *dev, const struct device_driver *drv)
{
struct amba_device *pcdev = to_amba_device(dev);
const struct amba_driver *pcdrv = to_amba_driver(drv);
+ int ret;
mutex_lock(&pcdev->periphid_lock);
if (!pcdev->periphid) {
- int ret = amba_read_periphid(pcdev);
+ ret = amba_read_periphid(pcdev);
/*
* Returning any error other than -EPROBE_DEFER from bus match
@@ -230,8 +203,9 @@ static int amba_match(struct device *dev, const struct device_driver *drv)
mutex_unlock(&pcdev->periphid_lock);
/* When driver_override is set, only bind to the matching driver */
- if (pcdev->driver_override)
- return !strcmp(pcdev->driver_override, drv->name);
+ ret = device_match_driver_override(dev, drv);
+ if (ret >= 0)
+ return ret;
return amba_lookup(pcdrv->id_table, pcdev) != NULL;
}
@@ -438,6 +412,7 @@ static const struct dev_pm_ops amba_pm = {
const struct bus_type amba_bustype = {
.name = "amba",
.dev_groups = amba_dev_groups,
+ .driver_override = true,
.match = amba_match,
.uevent = amba_uevent,
.probe = amba_probe,
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 434934927928..e216409f84cc 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -1324,7 +1324,7 @@ static int ata_hpa_resize(struct ata_device *dev)
/* do we need to do it? */
if ((dev->class != ATA_DEV_ATA && dev->class != ATA_DEV_ZAC) ||
!ata_id_has_lba(dev->id) || !ata_id_hpa_enabled(dev->id) ||
- (dev->quirks & ATA_QUIRK_BROKEN_HPA))
+ (dev->quirks & ATA_QUIRK_BROKEN_HPA) || ata_id_is_locked(dev->id))
return 0;
/* read native max address */
@@ -1526,6 +1526,7 @@ unsigned int ata_exec_internal(struct ata_device *dev, struct ata_taskfile *tf,
{
struct ata_link *link = dev->link;
struct ata_port *ap = link->ap;
+ const bool owns_eh_mutex = ap->host->eh_owner == current;
u8 command = tf->command;
struct ata_queued_cmd *qc;
struct scatterlist sgl;
@@ -1603,11 +1604,25 @@ unsigned int ata_exec_internal(struct ata_device *dev, struct ata_taskfile *tf,
}
}
- ata_eh_release(ap);
+ if (owns_eh_mutex) {
+ /*
+ * To prevent that the compiler complains about the
+ * ata_eh_release() call below.
+ */
+ __acquire(&ap->host->eh_mutex);
+ ata_eh_release(ap);
+ }
rc = wait_for_completion_timeout(&wait, msecs_to_jiffies(timeout));
- ata_eh_acquire(ap);
+ if (owns_eh_mutex) {
+ ata_eh_acquire(ap);
+ /*
+ * To prevent that the compiler complains about the above
+ * ata_eh_acquire() call.
+ */
+ __release(&ap->host->eh_mutex);
+ }
ata_sff_flush_pio_task(ap);
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 4cf537b6f492..4862d327c47a 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1942,8 +1942,13 @@ static void ata_scsi_rbuf_fill(struct ata_device *dev, struct scsi_cmnd *cmd,
memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE);
len = actor(dev, cmd, ata_scsi_rbuf);
if (len) {
+ if (WARN_ON(len > ATA_SCSI_RBUF_SIZE)) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0, 0);
+ spin_unlock_irqrestore(&ata_scsi_rbuf_lock, flags);
+ return;
+ }
sg_copy_from_buffer(scsi_sglist(cmd), scsi_sg_count(cmd),
- ata_scsi_rbuf, ATA_SCSI_RBUF_SIZE);
+ ata_scsi_rbuf, len);
cmd->result = SAM_STAT_GOOD;
if (scsi_bufflen(cmd) > len)
scsi_set_resid(cmd, scsi_bufflen(cmd) - len);
diff --git a/drivers/ata/pata_pxa.c b/drivers/ata/pata_pxa.c
index 1bdcd6ee741d..4e9fdcda8adb 100644
--- a/drivers/ata/pata_pxa.c
+++ b/drivers/ata/pata_pxa.c
@@ -286,6 +286,7 @@ static int pxa_ata_probe(struct platform_device *pdev)
ret = dmaengine_slave_config(data->dma_chan, &config);
if (ret < 0) {
dev_err(&pdev->dev, "dma configuration failed: %d\n", ret);
+ dma_release_channel(data->dma_chan);
return ret;
}
diff --git a/drivers/ata/sata_gemini.c b/drivers/ata/sata_gemini.c
index f574e3c3f5b4..b41a5ad1b06e 100644
--- a/drivers/ata/sata_gemini.c
+++ b/drivers/ata/sata_gemini.c
@@ -385,7 +385,7 @@ static int gemini_sata_probe(struct platform_device *pdev)
if (sg->ide_pins) {
ret = gemini_setup_ide_pins(dev);
if (ret)
- return ret;
+ goto out_unprep_clk;
}
dev_info(dev, "set up the Gemini IDE/SATA nexus\n");
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 7524555ff6cf..e2eda04bba0d 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -326,12 +326,11 @@ void deferred_probe_extend_timeout(void)
* If the work hasn't been queued yet or if the work expired, don't
* start a new one.
*/
- if (cancel_delayed_work(&deferred_probe_timeout_work)) {
- schedule_delayed_work(&deferred_probe_timeout_work,
- driver_deferred_probe_timeout * HZ);
+ if (delayed_work_pending(&deferred_probe_timeout_work) &&
+ mod_delayed_work(system_percpu_wq, &deferred_probe_timeout_work,
+ secs_to_jiffies(driver_deferred_probe_timeout)))
pr_debug("Extended deferred probe timeout by %d secs\n",
driver_deferred_probe_timeout);
- }
}
/**
diff --git a/drivers/base/firmware_loader/main.c b/drivers/base/firmware_loader/main.c
index c6664a787969..b4c4a5329570 100644
--- a/drivers/base/firmware_loader/main.c
+++ b/drivers/base/firmware_loader/main.c
@@ -1547,9 +1547,10 @@ static void device_cache_fw_images(void)
mutex_lock(&fw_lock);
fwc->state = FW_LOADER_START_CACHE;
- dpm_for_each_dev(NULL, dev_cache_fw_image);
mutex_unlock(&fw_lock);
+ dpm_for_each_dev(NULL, dev_cache_fw_image);
+
/* wait for completion of caching firmware for all devices */
async_synchronize_full_domain(&fw_cache_domain);
diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
index abb16a5bb796..6e5586d1e43c 100644
--- a/drivers/base/power/main.c
+++ b/drivers/base/power/main.c
@@ -100,7 +100,7 @@ void device_pm_sleep_init(struct device *dev)
dev->power.is_noirq_suspended = false;
dev->power.is_late_suspended = false;
init_completion(&dev->power.completion);
- complete_all(&dev->power.completion);
+ complete(&dev->power.completion);
dev->power.wakeup = NULL;
INIT_LIST_HEAD(&dev->power.entry);
}
@@ -237,6 +237,10 @@ static void dpm_wait(struct device *dev, bool async)
if (!dev)
return;
+ /* Devices with no PM support don't use the completion. */
+ if (dev->power.no_pm)
+ return;
+
if (async || (pm_async_enabled && dev->power.async_suspend))
wait_for_completion(&dev->power.completion);
}
diff --git a/drivers/base/property.c b/drivers/base/property.c
index 4217d00c76fd..ffe4e75908e4 100644
--- a/drivers/base/property.c
+++ b/drivers/base/property.c
@@ -1202,8 +1202,10 @@ fwnode_graph_get_endpoint_by_id(const struct fwnode_handle *fwnode,
if (fwnode_ep.port != port)
continue;
- if (fwnode_ep.id == endpoint)
+ if (fwnode_ep.id == endpoint) {
+ fwnode_handle_put(best_ep);
return ep;
+ }
if (!endpoint_next)
continue;
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index ac18d36b0ea8..f85ebdc3cff6 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -1998,6 +1998,11 @@ static int recv_dless_read(struct drbd_peer_device *peer_device, struct drbd_req
data_size -= digest_size;
}
+ if (data_size < 0) {
+ drbd_err(peer_device, "Invalid data reply size\n");
+ return -EIO;
+ }
+
/* optimistically update recv_cnt. if receiving fails below,
* we disconnect anyways, and counters will be reset. */
peer_device->device->recv_cnt += data_size>>9;
diff --git a/drivers/bluetooth/bpa10x.c b/drivers/bluetooth/bpa10x.c
index 1fa58c059cbf..8942ebd0d0b9 100644
--- a/drivers/bluetooth/bpa10x.c
+++ b/drivers/bluetooth/bpa10x.c
@@ -254,9 +254,13 @@ static int bpa10x_setup(struct hci_dev *hdev)
if (IS_ERR(skb))
return PTR_ERR(skb);
- bt_dev_info(hdev, "%s", (char *)(skb->data + 1));
+ /* Bounded print: the device controls skb->len. */
+ if (skb->len > 1) {
+ int len = skb->len - 1;
- hci_set_fw_info(hdev, "%s", skb->data + 1);
+ bt_dev_info(hdev, "%.*s", len, (char *)(skb->data + 1));
+ hci_set_fw_info(hdev, "%.*s", len, skb->data + 1);
+ }
kfree_skb(skb);
return 0;
diff --git a/drivers/bluetooth/btmtk.c b/drivers/bluetooth/btmtk.c
index 08a8c3a5d7b7..4d3f440565c1 100644
--- a/drivers/bluetooth/btmtk.c
+++ b/drivers/bluetooth/btmtk.c
@@ -1051,8 +1051,10 @@ struct urb *alloc_mtk_intr_urb(struct hci_dev *hdev, struct sk_buff *skb,
if (!urb)
return ERR_PTR(-ENOMEM);
- if (btmtk_isopkt_pad(hdev, skb))
+ if (btmtk_isopkt_pad(hdev, skb)) {
+ usb_free_urb(urb);
return ERR_PTR(-EINVAL);
+ }
pipe = usb_sndintpipe(btmtk_data->udev,
btmtk_data->isopkt_tx_ep->bEndpointAddress);
diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index 5e332b30696f..49b74827344c 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -786,8 +786,9 @@ static int rtlbt_parse_firmware(struct hci_dev *hdev,
}
BT_DBG("length=%x offset=%x index %d", patch_length, patch_offset, i);
- min_size = patch_offset + patch_length;
- if (btrtl_dev->fw_len < min_size)
+ if (patch_length < sizeof(epatch_info->fw_version) ||
+ patch_offset > btrtl_dev->fw_len ||
+ patch_length > btrtl_dev->fw_len - patch_offset)
return -EINVAL;
/* Copy the firmware into a new buffer and write the version at
diff --git a/drivers/bluetooth/hci_qca.c b/drivers/bluetooth/hci_qca.c
index 7897307a7fe2..de9fd0e77d39 100644
--- a/drivers/bluetooth/hci_qca.c
+++ b/drivers/bluetooth/hci_qca.c
@@ -1010,7 +1010,7 @@ static void qca_dmp_hdr(struct hci_dev *hdev, struct sk_buff *skb)
skb_put_data(skb, buf, strlen(buf));
snprintf(buf, sizeof(buf), "Driver: %s\n",
- hu->serdev->dev.driver->name);
+ hu->serdev ? hu->serdev->dev.driver->name : "hci_ldisc_qca");
skb_put_data(skb, buf, strlen(buf));
}
diff --git a/drivers/bluetooth/hci_vhci.c b/drivers/bluetooth/hci_vhci.c
index 59872e73c187..b03fe0d83632 100644
--- a/drivers/bluetooth/hci_vhci.c
+++ b/drivers/bluetooth/hci_vhci.c
@@ -337,7 +337,17 @@ static ssize_t force_devcd_write(struct file *file, const char __user *user_buf,
if (copy_from_user(&dump_data, user_buf, count))
return -EFAULT;
+ switch (dump_data.state) {
+ case HCI_DEVCOREDUMP_DONE:
+ case HCI_DEVCOREDUMP_ABORT:
+ case HCI_DEVCOREDUMP_TIMEOUT:
+ break;
+ default:
+ return -EINVAL;
+ }
+
data_size = count - offsetof(struct devcoredump_test_data, data);
+
skb = alloc_skb(data_size, GFP_ATOMIC);
if (!skb)
return -ENOMEM;
diff --git a/drivers/bus/imx-weim.c b/drivers/bus/imx-weim.c
index f735e0462c55..87070155b057 100644
--- a/drivers/bus/imx-weim.c
+++ b/drivers/bus/imx-weim.c
@@ -327,12 +327,6 @@ static int of_weim_notify(struct notifier_block *nb, unsigned long action,
"Failed to setup timing for '%pOF'\n", rd->dn);
if (!of_node_check_flag(rd->dn, OF_POPULATED)) {
- /*
- * Clear the flag before adding the device so that
- * fw_devlink doesn't skip adding consumers to this
- * device.
- */
- fwnode_clear_flag(&rd->dn->fwnode, FWNODE_FLAG_NOT_DEVICE);
if (!of_platform_device_create(rd->dn, NULL, &pdev->dev)) {
dev_err(&pdev->dev,
"Failed to create child device '%pOF'\n",
diff --git a/drivers/bus/mhi/ep/main.c b/drivers/bus/mhi/ep/main.c
index cdea24e92919..2f58ad0f14b6 100644
--- a/drivers/bus/mhi/ep/main.c
+++ b/drivers/bus/mhi/ep/main.c
@@ -232,7 +232,9 @@ static int mhi_ep_process_cmd_ring(struct mhi_ep_ring *ring, struct mhi_ring_ele
ret = mhi_ep_create_device(mhi_cntrl, ch_id);
if (ret) {
dev_err(dev, "Error creating device for channel (%u)\n", ch_id);
+ mutex_lock(&mhi_cntrl->state_lock);
mhi_ep_handle_syserr(mhi_cntrl);
+ mutex_unlock(&mhi_cntrl->state_lock);
return ret;
}
}
@@ -1087,11 +1089,12 @@ static void mhi_ep_reset_worker(struct work_struct *work)
mhi_ep_power_down(mhi_cntrl);
- mutex_lock(&mhi_cntrl->state_lock);
-
/* Reset MMIO to signal host that the MHI_RESET is completed in endpoint */
mhi_ep_mmio_reset(mhi_cntrl);
+
+ mutex_lock(&mhi_cntrl->state_lock);
cur_state = mhi_cntrl->mhi_state;
+ mutex_unlock(&mhi_cntrl->state_lock);
/*
* Only proceed further if the reset is due to SYS_ERR. The host will
@@ -1100,8 +1103,6 @@ static void mhi_ep_reset_worker(struct work_struct *work)
*/
if (cur_state == MHI_STATE_SYS_ERR)
mhi_ep_power_up(mhi_cntrl);
-
- mutex_unlock(&mhi_cntrl->state_lock);
}
/*
diff --git a/drivers/bus/sunxi-rsb.c b/drivers/bus/sunxi-rsb.c
index a89d78925637..7c9a113e133f 100644
--- a/drivers/bus/sunxi-rsb.c
+++ b/drivers/bus/sunxi-rsb.c
@@ -447,6 +447,9 @@ static int regmap_sunxi_rsb_reg_write(void *context, unsigned int reg,
struct sunxi_rsb_ctx *ctx = context;
struct sunxi_rsb_device *rdev = ctx->rdev;
+ if (reg > 0xff)
+ return -EINVAL;
+
return sunxi_rsb_write(rdev->rsb, rdev->rtaddr, reg, &val, ctx->size);
}
diff --git a/drivers/cdx/cdx.c b/drivers/cdx/cdx.c
index 4af1901c9d52..56fc47e48f4a 100644
--- a/drivers/cdx/cdx.c
+++ b/drivers/cdx/cdx.c
@@ -156,8 +156,6 @@ static int cdx_unregister_device(struct device *dev,
} else {
cdx_destroy_res_attr(cdx_dev, MAX_CDX_DEV_RESOURCES);
debugfs_remove_recursive(cdx_dev->debugfs_dir);
- kfree(cdx_dev->driver_override);
- cdx_dev->driver_override = NULL;
}
/*
@@ -268,6 +266,7 @@ static int cdx_bus_match(struct device *dev, const struct device_driver *drv)
const struct cdx_driver *cdx_drv = to_cdx_driver(drv);
const struct cdx_device_id *found_id = NULL;
const struct cdx_device_id *ids;
+ int ret;
if (cdx_dev->is_bus)
return false;
@@ -275,7 +274,8 @@ static int cdx_bus_match(struct device *dev, const struct device_driver *drv)
ids = cdx_drv->match_id_table;
/* When driver_override is set, only bind to the matching driver */
- if (cdx_dev->driver_override && strcmp(cdx_dev->driver_override, drv->name))
+ ret = device_match_driver_override(dev, drv);
+ if (ret == 0)
return false;
found_id = cdx_match_id(ids, cdx_dev);
@@ -289,7 +289,7 @@ static int cdx_bus_match(struct device *dev, const struct device_driver *drv)
*/
if (!found_id->override_only)
return true;
- if (cdx_dev->driver_override)
+ if (ret > 0)
return true;
ids = found_id + 1;
@@ -449,36 +449,6 @@ static ssize_t modalias_show(struct device *dev, struct device_attribute *attr,
}
static DEVICE_ATTR_RO(modalias);
-static ssize_t driver_override_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct cdx_device *cdx_dev = to_cdx_device(dev);
- int ret;
-
- if (WARN_ON(dev->bus != &cdx_bus_type))
- return -EINVAL;
-
- ret = driver_set_override(dev, &cdx_dev->driver_override, buf, count);
- if (ret)
- return ret;
-
- return count;
-}
-
-static ssize_t driver_override_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct cdx_device *cdx_dev = to_cdx_device(dev);
- ssize_t len;
-
- device_lock(dev);
- len = sysfs_emit(buf, "%s\n", cdx_dev->driver_override);
- device_unlock(dev);
- return len;
-}
-static DEVICE_ATTR_RW(driver_override);
-
static ssize_t enable_store(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
@@ -548,7 +518,6 @@ static struct attribute *cdx_dev_attrs[] = {
&dev_attr_class.attr,
&dev_attr_revision.attr,
&dev_attr_modalias.attr,
- &dev_attr_driver_override.attr,
NULL,
};
@@ -649,6 +618,7 @@ ATTRIBUTE_GROUPS(cdx_bus);
struct bus_type cdx_bus_type = {
.name = "cdx",
+ .driver_override = true,
.match = cdx_bus_match,
.probe = cdx_probe,
.remove = cdx_remove,
diff --git a/drivers/char/tlclk.c b/drivers/char/tlclk.c
index 377bebf6c925..103127c2aeab 100644
--- a/drivers/char/tlclk.c
+++ b/drivers/char/tlclk.c
@@ -264,6 +264,7 @@ static ssize_t tlclk_read(struct file *filp, char __user *buf, size_t count,
}
static const struct file_operations tlclk_fops = {
+ .owner = THIS_MODULE,
.read = tlclk_read,
.open = tlclk_open,
.release = tlclk_release,
@@ -853,6 +854,9 @@ static void __exit tlclk_cleanup(void)
misc_deregister(&tlclk_miscdev);
unregister_chrdev(tlclk_major, "telco_clock");
+ got_event = 1;
+ wake_up_all(&wq);
+
release_region(TLCLK_BASE, 8);
del_timer_sync(&switchover_timer);
kfree(alarm_events);
diff --git a/drivers/char/tpm/eventlog/tpm1.c b/drivers/char/tpm/eventlog/tpm1.c
index 12ee42a31c71..697b89af386b 100644
--- a/drivers/char/tpm/eventlog/tpm1.c
+++ b/drivers/char/tpm/eventlog/tpm1.c
@@ -236,12 +236,12 @@ static int tpm1_binary_bios_measurements_show(struct seq_file *m, void *v)
temp_ptr = (char *) &temp_event;
- for (i = 0; i < (sizeof(struct tcpa_event) - 1) ; i++)
+ for (i = 0; i < sizeof(struct tcpa_event); i++)
seq_putc(m, temp_ptr[i]);
temp_ptr = (char *) v;
- for (i = (sizeof(struct tcpa_event) - 1);
+ for (i = sizeof(struct tcpa_event);
i < (sizeof(struct tcpa_event) + temp_event.event_size); i++)
seq_putc(m, temp_ptr[i]);
diff --git a/drivers/char/tpm/tpm-dev.c b/drivers/char/tpm/tpm-dev.c
index 97c94b5e9340..9181f155912b 100644
--- a/drivers/char/tpm/tpm-dev.c
+++ b/drivers/char/tpm/tpm-dev.c
@@ -36,7 +36,7 @@ static int tpm_open(struct inode *inode, struct file *file)
tpm_common_open(file, chip, priv, NULL);
- return 0;
+ return nonseekable_open(inode, file);
out:
clear_bit(0, &chip->is_open);
diff --git a/drivers/char/tpm/tpm2-sessions.c b/drivers/char/tpm/tpm2-sessions.c
index cb944df7b3ca..5a28d7c38b33 100644
--- a/drivers/char/tpm/tpm2-sessions.c
+++ b/drivers/char/tpm/tpm2-sessions.c
@@ -534,15 +534,17 @@ static void tpm2_KDFe(u8 z[EC_PT_SZ], const char *str, u8 *pt_u, u8 *pt_v,
sha256_final(&sctx, out);
}
-static void tpm_buf_append_salt(struct tpm_buf *buf, struct tpm_chip *chip,
- struct tpm2_auth *auth)
+static int tpm_buf_append_salt(struct tpm_buf *buf, struct tpm_chip *chip,
+ struct tpm2_auth *auth)
{
struct crypto_kpp *kpp;
struct kpp_request *req;
+ DECLARE_CRYPTO_WAIT(wait);
struct scatterlist s[2], d[1];
struct ecdh p = {0};
u8 encoded_key[EC_PT_SZ], *x, *y;
unsigned int buf_len;
+ int rc;
/* secret is two sized points */
tpm_buf_append_u16(buf, (EC_PT_SZ + 2)*2);
@@ -565,14 +567,15 @@ static void tpm_buf_append_salt(struct tpm_buf *buf, struct tpm_chip *chip,
kpp = crypto_alloc_kpp("ecdh-nist-p256", CRYPTO_ALG_INTERNAL, 0);
if (IS_ERR(kpp)) {
dev_err(&chip->dev, "crypto ecdh allocation failed\n");
- return;
+ return PTR_ERR(kpp);
}
buf_len = crypto_ecdh_key_len(&p);
if (sizeof(encoded_key) < buf_len) {
dev_err(&chip->dev, "salt buffer too small needs %d\n",
buf_len);
- goto out;
+ rc = -EINVAL;
+ goto err_free_kpp;
}
crypto_ecdh_encode_key(encoded_key, buf_len, &p);
/* this generates a random private key */
@@ -580,11 +583,17 @@ static void tpm_buf_append_salt(struct tpm_buf *buf, struct tpm_chip *chip,
/* salt is now the public point of this private key */
req = kpp_request_alloc(kpp, GFP_KERNEL);
- if (!req)
- goto out;
+ if (!req) {
+ rc = -ENOMEM;
+ goto err_free_kpp;
+ }
+ kpp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG,
+ crypto_req_done, &wait);
kpp_request_set_input(req, NULL, 0);
kpp_request_set_output(req, s, EC_PT_SZ*2);
- crypto_kpp_generate_public_key(req);
+ rc = crypto_wait_req(crypto_kpp_generate_public_key(req), &wait);
+ if (rc)
+ goto err_free_req;
/*
* we're not done: now we have to compute the shared secret
* which is our private key multiplied by the tpm_key public
@@ -596,8 +605,9 @@ static void tpm_buf_append_salt(struct tpm_buf *buf, struct tpm_chip *chip,
kpp_request_set_input(req, s, EC_PT_SZ*2);
sg_init_one(d, auth->salt, EC_PT_SZ);
kpp_request_set_output(req, d, EC_PT_SZ);
- crypto_kpp_compute_shared_secret(req);
- kpp_request_free(req);
+ rc = crypto_wait_req(crypto_kpp_compute_shared_secret(req), &wait);
+ if (rc)
+ goto err_free_req;
/*
* pass the shared secret through KDFe for salt. Note salt
@@ -607,8 +617,16 @@ static void tpm_buf_append_salt(struct tpm_buf *buf, struct tpm_chip *chip,
*/
tpm2_KDFe(auth->salt, "SECRET", x, chip->null_ec_key_x, auth->salt);
- out:
+ kpp_request_free(req);
crypto_free_kpp(kpp);
+ return 0;
+
+err_free_req:
+ kpp_request_free(req);
+
+err_free_kpp:
+ crypto_free_kpp(kpp);
+ return rc;
}
/**
@@ -1061,7 +1079,12 @@ int tpm2_start_auth_session(struct tpm_chip *chip)
tpm_buf_append(&buf, auth->our_nonce, sizeof(auth->our_nonce));
/* append encrypted salt and squirrel away unencrypted in auth */
- tpm_buf_append_salt(&buf, chip, auth);
+ rc = tpm_buf_append_salt(&buf, chip, auth);
+ if (rc) {
+ tpm2_flush_context(chip, null_key);
+ tpm_buf_destroy(&buf);
+ goto out;
+ }
/* session type (HMAC, audit or policy) */
tpm_buf_append_u8(&buf, TPM2_SE_HMAC);
diff --git a/drivers/char/tpm/tpm_tis_core.c b/drivers/char/tpm/tpm_tis_core.c
index 78c6a21bc92b..359abae37096 100644
--- a/drivers/char/tpm/tpm_tis_core.c
+++ b/drivers/char/tpm/tpm_tis_core.c
@@ -66,8 +66,8 @@ static int wait_for_tpm_stat(struct tpm_chip *chip, u8 mask,
bool check_cancel)
{
struct tpm_tis_data *priv = dev_get_drvdata(&chip->dev);
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
unsigned long stop;
- long rc;
u8 status;
bool canceled = false;
u8 sts_mask;
@@ -87,23 +87,30 @@ static int wait_for_tpm_stat(struct tpm_chip *chip, u8 mask,
/* process status changes with irq support */
if (sts_mask) {
ret = -ETIME;
+ add_wait_queue(queue, &wait);
again:
+ if (wait_for_tpm_stat_cond(chip, sts_mask, check_cancel,
+ &canceled)) {
+ ret = canceled ? -ECANCELED : 0;
+ goto out;
+ }
+
timeout = stop - jiffies;
if ((long)timeout <= 0)
- return -ETIME;
- rc = wait_event_interruptible_timeout(*queue,
- wait_for_tpm_stat_cond(chip, sts_mask, check_cancel,
- &canceled),
- timeout);
- if (rc > 0) {
- if (canceled)
- return -ECANCELED;
- ret = 0;
- }
- if (rc == -ERESTARTSYS && freezing(current)) {
- clear_thread_flag(TIF_SIGPENDING);
- goto again;
+ goto out;
+
+ if (signal_pending(current)) {
+ if (freezing(current)) {
+ clear_thread_flag(TIF_SIGPENDING);
+ goto again;
+ }
+ goto out;
}
+
+ wait_woken(&wait, TASK_INTERRUPTIBLE, timeout);
+ goto again;
+out:
+ remove_wait_queue(queue, &wait);
}
if (ret)
diff --git a/drivers/char/tpm/tpmrm-dev.c b/drivers/char/tpm/tpmrm-dev.c
index c25df7ea064e..fd38de3449a5 100644
--- a/drivers/char/tpm/tpmrm-dev.c
+++ b/drivers/char/tpm/tpmrm-dev.c
@@ -29,7 +29,7 @@ static int tpmrm_open(struct inode *inode, struct file *file)
tpm_common_open(file, chip, &priv->priv, &priv->space);
- return 0;
+ return nonseekable_open(inode, file);
}
static int tpmrm_release(struct inode *inode, struct file *file)
diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c
index 8c4e2b1e3de0..8765e8fcb68a 100644
--- a/drivers/char/virtio_console.c
+++ b/drivers/char/virtio_console.c
@@ -1774,32 +1774,40 @@ static void config_intr(struct virtio_device *vdev)
schedule_work(&portdev->config_work);
}
-static void config_work_handler(struct work_struct *work)
+static void update_size_from_config(struct ports_device *portdev)
{
- struct ports_device *portdev;
+ struct virtio_device *vdev;
+ struct port *port;
+ u16 rows, cols;
- portdev = container_of(work, struct ports_device, config_work);
- if (!use_multiport(portdev)) {
- struct virtio_device *vdev;
- struct port *port;
- u16 rows, cols;
+ vdev = portdev->vdev;
- vdev = portdev->vdev;
- virtio_cread(vdev, struct virtio_console_config, cols, &cols);
- virtio_cread(vdev, struct virtio_console_config, rows, &rows);
+ /*
+ * We'll use this way of resizing only for legacy support.
+ * For multiport devices, use control messages to indicate
+ * console size changes so that it can be done per-port.
+ *
+ * Don't test F_SIZE at all if we're rproc: not a valid feature.
+ */
+ if (is_rproc_serial(vdev) ||
+ use_multiport(portdev) ||
+ !virtio_has_feature(vdev, VIRTIO_CONSOLE_F_SIZE))
+ return;
- port = find_port_by_id(portdev, 0);
- set_console_size(port, rows, cols);
+ virtio_cread(vdev, struct virtio_console_config, cols, &cols);
+ virtio_cread(vdev, struct virtio_console_config, rows, &rows);
- /*
- * We'll use this way of resizing only for legacy
- * support. For newer userspace
- * (VIRTIO_CONSOLE_F_MULTPORT+), use control messages
- * to indicate console size changes so that it can be
- * done per-port.
- */
- resize_console(port);
- }
+ port = find_port_by_id(portdev, 0);
+ set_console_size(port, rows, cols);
+ resize_console(port);
+}
+
+static void config_work_handler(struct work_struct *work)
+{
+ struct ports_device *portdev;
+
+ portdev = container_of(work, struct ports_device, config_work);
+ update_size_from_config(portdev);
}
static int init_vqs(struct ports_device *portdev)
@@ -2057,6 +2065,8 @@ static int virtcons_probe(struct virtio_device *vdev)
__send_control_msg(portdev, VIRTIO_CONSOLE_BAD_ID,
VIRTIO_CONSOLE_DEVICE_READY, 1);
+ update_size_from_config(portdev);
+
return 0;
free_chrdev:
diff --git a/drivers/clk/at91/pmc.c b/drivers/clk/at91/pmc.c
index 5aa9c1f1c886..40bdc1c80ab0 100644
--- a/drivers/clk/at91/pmc.c
+++ b/drivers/clk/at91/pmc.c
@@ -176,9 +176,9 @@ static int __init pmc_register_ops(void)
of_node_put(np);
return -ENODEV;
}
- of_node_put(np);
at91_pmc_backup_suspend = of_iomap(np, 0);
+ of_node_put(np);
if (!at91_pmc_backup_suspend) {
pr_warn("%s(): unable to map securam\n", __func__);
return -ENOMEM;
diff --git a/drivers/clk/at91/sam9x7.c b/drivers/clk/at91/sam9x7.c
index 740f52906f6b..7255bcf952a7 100644
--- a/drivers/clk/at91/sam9x7.c
+++ b/drivers/clk/at91/sam9x7.c
@@ -564,6 +564,15 @@ static const struct {
.pp_chg_id = INT_MIN,
},
+ {
+ .n = "gmac_gclk",
+ .id = 24,
+ .pp = { "audiopll_divpmcck", "plla_div2pmcck", },
+ .pp_mux_table = { 6, 8, },
+ .pp_count = 2,
+ .pp_chg_id = INT_MIN,
+ },
+
{
.n = "lcd_gclk",
.id = 25,
@@ -697,15 +706,6 @@ static const struct {
.pp_count = 1,
.pp_chg_id = INT_MIN,
},
-
- {
- .n = "gmac_gclk",
- .id = 67,
- .pp = { "audiopll_divpmcck", "plla_div2pmcck", },
- .pp_mux_table = { 6, 8, },
- .pp_count = 2,
- .pp_chg_id = INT_MIN,
- },
};
static void __init sam9x7_pmc_setup(struct device_node *np)
diff --git a/drivers/clk/clk-scmi.c b/drivers/clk/clk-scmi.c
index 7b7ea36333a6..c0e9db04f450 100644
--- a/drivers/clk/clk-scmi.c
+++ b/drivers/clk/clk-scmi.c
@@ -10,9 +10,9 @@
#include <linux/device.h>
#include <linux/err.h>
#include <linux/of.h>
+#include <linux/math64.h>
#include <linux/module.h>
#include <linux/scmi_protocol.h>
-#include <asm/div64.h>
#define NOT_ATOMIC false
#define ATOMIC true
@@ -78,7 +78,7 @@ static long scmi_clk_round_rate(struct clk_hw *hw, unsigned long rate,
ftmp = rate - fmin;
ftmp += clk->info->range.step_size - 1; /* to round up */
- do_div(ftmp, clk->info->range.step_size);
+ ftmp = div64_ul(ftmp, clk->info->range.step_size);
return ftmp * clk->info->range.step_size + fmin;
}
diff --git a/drivers/clk/clk-scpi.c b/drivers/clk/clk-scpi.c
index 19d530d52e64..50ac1cd25578 100644
--- a/drivers/clk/clk-scpi.c
+++ b/drivers/clk/clk-scpi.c
@@ -256,7 +256,7 @@ static void scpi_clocks_remove(struct platform_device *pdev)
}
for_each_available_child_of_node(np, child)
- of_clk_del_provider(np);
+ of_clk_del_provider(child);
}
static int scpi_clocks_probe(struct platform_device *pdev)
diff --git a/drivers/clk/qcom/a53-pll.c b/drivers/clk/qcom/a53-pll.c
index f43d455ab4b8..05581a10bea5 100644
--- a/drivers/clk/qcom/a53-pll.c
+++ b/drivers/clk/qcom/a53-pll.c
@@ -20,7 +20,7 @@
static const struct pll_freq_tbl a53pll_freq[] = {
{ 998400000, 52, 0x0, 0x1, 0 },
{ 1094400000, 57, 0x0, 0x1, 0 },
- { 1152000000, 62, 0x0, 0x1, 0 },
+ { 1152000000, 60, 0x0, 0x1, 0 },
{ 1209600000, 63, 0x0, 0x1, 0 },
{ 1248000000, 65, 0x0, 0x1, 0 },
{ 1363200000, 71, 0x0, 0x1, 0 },
diff --git a/drivers/clocksource/timer-sun5i.c b/drivers/clocksource/timer-sun5i.c
index 0d229a9058da..706a12a57598 100644
--- a/drivers/clocksource/timer-sun5i.c
+++ b/drivers/clocksource/timer-sun5i.c
@@ -284,6 +284,9 @@ static int sun5i_timer_probe(struct platform_device *pdev)
}
rstc = devm_reset_control_get_optional_exclusive(dev, NULL);
+ if (IS_ERR(rstc))
+ return dev_err_probe(dev, PTR_ERR(rstc),
+ "failed to get reset\n");
if (rstc)
reset_control_deassert(rstc);
diff --git a/drivers/cpufreq/cpufreq_conservative.c b/drivers/cpufreq/cpufreq_conservative.c
index c51b6d1ebcd5..3550c22e5d83 100644
--- a/drivers/cpufreq/cpufreq_conservative.c
+++ b/drivers/cpufreq/cpufreq_conservative.c
@@ -103,10 +103,6 @@ static unsigned int cs_dbs_update(struct cpufreq_policy *policy)
if (load > dbs_data->up_threshold) {
dbs_info->down_skip = 0;
- /* if we are already at full speed then break out early */
- if (requested_freq == policy->max)
- goto out;
-
requested_freq += freq_step;
if (requested_freq > policy->max)
requested_freq = policy->max;
@@ -124,13 +120,7 @@ static unsigned int cs_dbs_update(struct cpufreq_policy *policy)
/* Check for frequency decrease */
if (load < cs_tuners->down_threshold) {
- /*
- * if we cannot reduce the frequency anymore, break out early
- */
- if (requested_freq == policy->min)
- goto out;
-
- if (requested_freq > freq_step)
+ if (requested_freq > policy->min + freq_step)
requested_freq -= freq_step;
else
requested_freq = policy->min;
diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c
index 96a8359267ce..818507586cd9 100644
--- a/drivers/cpufreq/intel_pstate.c
+++ b/drivers/cpufreq/intel_pstate.c
@@ -964,12 +964,14 @@ static void hybrid_clear_cpu_capacity(unsigned int cpunum)
static void hybrid_get_capacity_perf(struct cpudata *cpu)
{
+ u64 hwp_cap = READ_ONCE(cpu->hwp_cap_cached);
+
if (READ_ONCE(global.no_turbo)) {
- cpu->capacity_perf = cpu->pstate.max_pstate_physical;
+ cpu->capacity_perf = HWP_GUARANTEED_PERF(hwp_cap);
return;
}
- cpu->capacity_perf = HWP_HIGHEST_PERF(READ_ONCE(cpu->hwp_cap_cached));
+ cpu->capacity_perf = HWP_HIGHEST_PERF(hwp_cap);
}
static void hybrid_set_capacity_of_cpus(void)
diff --git a/drivers/crypto/allwinner/Kconfig b/drivers/crypto/allwinner/Kconfig
index b8e75210a0e3..06ea0e9fe6f2 100644
--- a/drivers/crypto/allwinner/Kconfig
+++ b/drivers/crypto/allwinner/Kconfig
@@ -24,14 +24,6 @@ config CRYPTO_DEV_SUN4I_SS
To compile this driver as a module, choose M here: the module
will be called sun4i-ss.
-config CRYPTO_DEV_SUN4I_SS_PRNG
- bool "Support for Allwinner Security System PRNG"
- depends on CRYPTO_DEV_SUN4I_SS
- select CRYPTO_RNG
- help
- Select this option if you want to provide kernel-side support for
- the Pseudo-Random Number Generator found in the Security System.
-
config CRYPTO_DEV_SUN4I_SS_DEBUG
bool "Enable sun4i-ss stats"
depends on CRYPTO_DEV_SUN4I_SS
diff --git a/drivers/crypto/allwinner/sun4i-ss/Makefile b/drivers/crypto/allwinner/sun4i-ss/Makefile
index c0a2797d3168..06a9ae81f9f8 100644
--- a/drivers/crypto/allwinner/sun4i-ss/Makefile
+++ b/drivers/crypto/allwinner/sun4i-ss/Makefile
@@ -1,4 +1,3 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_CRYPTO_DEV_SUN4I_SS) += sun4i-ss.o
sun4i-ss-y += sun4i-ss-core.o sun4i-ss-hash.o sun4i-ss-cipher.o
-sun4i-ss-$(CONFIG_CRYPTO_DEV_SUN4I_SS_PRNG) += sun4i-ss-prng.o
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c
index 890664bd5f0f..cc34600f4a4e 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c
@@ -213,23 +213,6 @@ static struct sun4i_ss_alg_template ss_algs[] = {
}
}
},
-#ifdef CONFIG_CRYPTO_DEV_SUN4I_SS_PRNG
-{
- .type = CRYPTO_ALG_TYPE_RNG,
- .alg.rng = {
- .base = {
- .cra_name = "stdrng",
- .cra_driver_name = "sun4i_ss_rng",
- .cra_priority = 300,
- .cra_ctxsize = 0,
- .cra_module = THIS_MODULE,
- },
- .generate = sun4i_ss_prng_generate,
- .seed = sun4i_ss_prng_seed,
- .seedsize = SS_SEED_LEN / BITS_PER_BYTE,
- }
-},
-#endif
};
static int sun4i_ss_debugfs_show(struct seq_file *seq, void *v)
@@ -247,12 +230,6 @@ static int sun4i_ss_debugfs_show(struct seq_file *seq, void *v)
ss_algs[i].stat_req, ss_algs[i].stat_opti, ss_algs[i].stat_fb,
ss_algs[i].stat_bytes);
break;
- case CRYPTO_ALG_TYPE_RNG:
- seq_printf(seq, "%s %s reqs=%lu tsize=%lu\n",
- ss_algs[i].alg.rng.base.cra_driver_name,
- ss_algs[i].alg.rng.base.cra_name,
- ss_algs[i].stat_req, ss_algs[i].stat_bytes);
- break;
case CRYPTO_ALG_TYPE_AHASH:
seq_printf(seq, "%s %s reqs=%lu\n",
ss_algs[i].alg.hash.halg.base.cra_driver_name,
@@ -471,13 +448,6 @@ static int sun4i_ss_probe(struct platform_device *pdev)
goto error_alg;
}
break;
- case CRYPTO_ALG_TYPE_RNG:
- err = crypto_register_rng(&ss_algs[i].alg.rng);
- if (err) {
- dev_err(ss->dev, "Fail to register %s\n",
- ss_algs[i].alg.rng.base.cra_name);
- }
- break;
}
}
@@ -497,9 +467,6 @@ static int sun4i_ss_probe(struct platform_device *pdev)
case CRYPTO_ALG_TYPE_AHASH:
crypto_unregister_ahash(&ss_algs[i].alg.hash);
break;
- case CRYPTO_ALG_TYPE_RNG:
- crypto_unregister_rng(&ss_algs[i].alg.rng);
- break;
}
}
error_pm:
@@ -520,9 +487,6 @@ static void sun4i_ss_remove(struct platform_device *pdev)
case CRYPTO_ALG_TYPE_AHASH:
crypto_unregister_ahash(&ss_algs[i].alg.hash);
break;
- case CRYPTO_ALG_TYPE_RNG:
- crypto_unregister_rng(&ss_algs[i].alg.rng);
- break;
}
}
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c
deleted file mode 100644
index 491fcb7b81b4..000000000000
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c
+++ /dev/null
@@ -1,69 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-#include "sun4i-ss.h"
-
-int sun4i_ss_prng_seed(struct crypto_rng *tfm, const u8 *seed,
- unsigned int slen)
-{
- struct sun4i_ss_alg_template *algt;
- struct rng_alg *alg = crypto_rng_alg(tfm);
-
- algt = container_of(alg, struct sun4i_ss_alg_template, alg.rng);
- memcpy(algt->ss->seed, seed, slen);
-
- return 0;
-}
-
-int sun4i_ss_prng_generate(struct crypto_rng *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int dlen)
-{
- struct sun4i_ss_alg_template *algt;
- struct rng_alg *alg = crypto_rng_alg(tfm);
- int i, err;
- u32 v;
- u32 *data = (u32 *)dst;
- const u32 mode = SS_OP_PRNG | SS_PRNG_CONTINUE | SS_ENABLED;
- size_t len;
- struct sun4i_ss_ctx *ss;
- unsigned int todo = (dlen / 4) * 4;
-
- algt = container_of(alg, struct sun4i_ss_alg_template, alg.rng);
- ss = algt->ss;
-
- err = pm_runtime_resume_and_get(ss->dev);
- if (err < 0)
- return err;
-
- if (IS_ENABLED(CONFIG_CRYPTO_DEV_SUN4I_SS_DEBUG)) {
- algt->stat_req++;
- algt->stat_bytes += todo;
- }
-
- spin_lock_bh(&ss->slock);
-
- writel(mode, ss->base + SS_CTL);
-
- while (todo > 0) {
- /* write the seed */
- for (i = 0; i < SS_SEED_LEN / BITS_PER_LONG; i++)
- writel(ss->seed[i], ss->base + SS_KEY0 + i * 4);
-
- /* Read the random data */
- len = min_t(size_t, SS_DATA_LEN / BITS_PER_BYTE, todo);
- readsl(ss->base + SS_TXFIFO, data, len / 4);
- data += len / 4;
- todo -= len;
-
- /* Update the seed */
- for (i = 0; i < SS_SEED_LEN / BITS_PER_LONG; i++) {
- v = readl(ss->base + SS_KEY0 + i * 4);
- ss->seed[i] = v;
- }
- }
-
- writel(0, ss->base + SS_CTL);
- spin_unlock_bh(&ss->slock);
-
- pm_runtime_put(ss->dev);
-
- return 0;
-}
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h
index 6c5d4aa6453c..f7d1c79ac677 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss.h
@@ -31,8 +31,6 @@
#include <crypto/internal/skcipher.h>
#include <crypto/aes.h>
#include <crypto/internal/des.h>
-#include <crypto/internal/rng.h>
-#include <crypto/rng.h>
#define SS_CTL 0x00
#define SS_KEY0 0x04
@@ -62,10 +60,6 @@
/* SS_CTL configuration values */
-/* PRNG generator mode - bit 15 */
-#define SS_PRNG_ONESHOT (0 << 15)
-#define SS_PRNG_CONTINUE (1 << 15)
-
/* IV mode for hash */
#define SS_IV_ARBITRARY (1 << 14)
@@ -94,14 +88,10 @@
#define SS_OP_3DES (2 << 4)
#define SS_OP_SHA1 (3 << 4)
#define SS_OP_MD5 (4 << 4)
-#define SS_OP_PRNG (5 << 4)
/* Data end bit - bit 2 */
#define SS_DATA_END (1 << 2)
-/* PRNG start bit - bit 1 */
-#define SS_PRNG_START (1 << 1)
-
/* SS Enable bit - bit 0 */
#define SS_DISABLED (0 << 0)
#define SS_ENABLED (1 << 0)
@@ -128,9 +118,6 @@
#define SS_RXFIFO_EMP_INT_ENABLE (1 << 2)
#define SS_TXFIFO_AVA_INT_ENABLE (1 << 0)
-#define SS_SEED_LEN 192
-#define SS_DATA_LEN 160
-
/*
* struct ss_variant - Describe SS hardware variant
* @sha1_in_be: The SHA1 digest is given by SS in BE, and so need to be inverted.
@@ -151,9 +138,6 @@ struct sun4i_ss_ctx {
char buf[4 * SS_RX_MAX];/* buffer for linearize SG src */
char bufo[4 * SS_TX_MAX]; /* buffer for linearize SG dst */
spinlock_t slock; /* control the use of the device */
-#ifdef CONFIG_CRYPTO_DEV_SUN4I_SS_PRNG
- u32 seed[SS_SEED_LEN / BITS_PER_LONG];
-#endif
struct dentry *dbgfs_dir;
struct dentry *dbgfs_stats;
};
@@ -164,7 +148,6 @@ struct sun4i_ss_alg_template {
union {
struct skcipher_alg crypto;
struct ahash_alg hash;
- struct rng_alg rng;
} alg;
struct sun4i_ss_ctx *ss;
unsigned long stat_req;
@@ -231,6 +214,3 @@ int sun4i_ss_des_setkey(struct crypto_skcipher *tfm, const u8 *key,
unsigned int keylen);
int sun4i_ss_des3_setkey(struct crypto_skcipher *tfm, const u8 *key,
unsigned int keylen);
-int sun4i_ss_prng_generate(struct crypto_rng *tfm, const u8 *src,
- unsigned int slen, u8 *dst, unsigned int dlen);
-int sun4i_ss_prng_seed(struct crypto_rng *tfm, const u8 *seed, unsigned int slen);
diff --git a/drivers/crypto/amcc/crypto4xx_alg.c b/drivers/crypto/amcc/crypto4xx_alg.c
index e0af611a95d8..04a68622ec09 100644
--- a/drivers/crypto/amcc/crypto4xx_alg.c
+++ b/drivers/crypto/amcc/crypto4xx_alg.c
@@ -12,9 +12,6 @@
#include <linux/interrupt.h>
#include <linux/spinlock_types.h>
#include <linux/scatterlist.h>
-#include <linux/crypto.h>
-#include <linux/hash.h>
-#include <crypto/internal/hash.h>
#include <linux/dma-mapping.h>
#include <crypto/algapi.h>
#include <crypto/aead.h>
@@ -602,106 +599,3 @@ int crypto4xx_decrypt_aes_gcm(struct aead_request *req)
{
return crypto4xx_crypt_aes_gcm(req, true);
}
-
-/*
- * HASH SHA1 Functions
- */
-static int crypto4xx_hash_alg_init(struct crypto_tfm *tfm,
- unsigned int sa_len,
- unsigned char ha,
- unsigned char hm)
-{
- struct crypto_alg *alg = tfm->__crt_alg;
- struct crypto4xx_alg *my_alg;
- struct crypto4xx_ctx *ctx = crypto_tfm_ctx(tfm);
- struct dynamic_sa_hash160 *sa;
- int rc;
-
- my_alg = container_of(__crypto_ahash_alg(alg), struct crypto4xx_alg,
- alg.u.hash);
- ctx->dev = my_alg->dev;
-
- /* Create SA */
- if (ctx->sa_in || ctx->sa_out)
- crypto4xx_free_sa(ctx);
-
- rc = crypto4xx_alloc_sa(ctx, sa_len);
- if (rc)
- return rc;
-
- crypto_ahash_set_reqsize(__crypto_ahash_cast(tfm),
- sizeof(struct crypto4xx_ctx));
- sa = (struct dynamic_sa_hash160 *)ctx->sa_in;
- set_dynamic_sa_command_0(&sa->ctrl, SA_SAVE_HASH, SA_NOT_SAVE_IV,
- SA_NOT_LOAD_HASH, SA_LOAD_IV_FROM_SA,
- SA_NO_HEADER_PROC, ha, SA_CIPHER_ALG_NULL,
- SA_PAD_TYPE_ZERO, SA_OP_GROUP_BASIC,
- SA_OPCODE_HASH, DIR_INBOUND);
- set_dynamic_sa_command_1(&sa->ctrl, 0, SA_HASH_MODE_HASH,
- CRYPTO_FEEDBACK_MODE_NO_FB, SA_EXTENDED_SN_OFF,
- SA_SEQ_MASK_OFF, SA_MC_ENABLE,
- SA_NOT_COPY_PAD, SA_NOT_COPY_PAYLOAD,
- SA_NOT_COPY_HDR);
- /* Need to zero hash digest in SA */
- memset(sa->inner_digest, 0, sizeof(sa->inner_digest));
- memset(sa->outer_digest, 0, sizeof(sa->outer_digest));
-
- return 0;
-}
-
-int crypto4xx_hash_init(struct ahash_request *req)
-{
- struct crypto4xx_ctx *ctx = crypto_tfm_ctx(req->base.tfm);
- int ds;
- struct dynamic_sa_ctl *sa;
-
- sa = ctx->sa_in;
- ds = crypto_ahash_digestsize(
- __crypto_ahash_cast(req->base.tfm));
- sa->sa_command_0.bf.digest_len = ds >> 2;
- sa->sa_command_0.bf.load_hash_state = SA_LOAD_HASH_FROM_SA;
-
- return 0;
-}
-
-int crypto4xx_hash_update(struct ahash_request *req)
-{
- struct crypto_ahash *ahash = crypto_ahash_reqtfm(req);
- struct crypto4xx_ctx *ctx = crypto_tfm_ctx(req->base.tfm);
- struct scatterlist dst;
- unsigned int ds = crypto_ahash_digestsize(ahash);
-
- sg_init_one(&dst, req->result, ds);
-
- return crypto4xx_build_pd(&req->base, ctx, req->src, &dst,
- req->nbytes, NULL, 0, ctx->sa_in,
- ctx->sa_len, 0, NULL);
-}
-
-int crypto4xx_hash_final(struct ahash_request *req)
-{
- return 0;
-}
-
-int crypto4xx_hash_digest(struct ahash_request *req)
-{
- struct crypto_ahash *ahash = crypto_ahash_reqtfm(req);
- struct crypto4xx_ctx *ctx = crypto_tfm_ctx(req->base.tfm);
- struct scatterlist dst;
- unsigned int ds = crypto_ahash_digestsize(ahash);
-
- sg_init_one(&dst, req->result, ds);
-
- return crypto4xx_build_pd(&req->base, ctx, req->src, &dst,
- req->nbytes, NULL, 0, ctx->sa_in,
- ctx->sa_len, 0, NULL);
-}
-
-/*
- * SHA1 Algorithm
- */
-int crypto4xx_sha1_alg_init(struct crypto_tfm *tfm)
-{
- return crypto4xx_hash_alg_init(tfm, SA_HASH160_LEN, SA_HASH_ALG_SHA1,
- SA_HASH_MODE_HASH);
-}
diff --git a/drivers/crypto/amcc/crypto4xx_core.c b/drivers/crypto/amcc/crypto4xx_core.c
index 6006703fb6d7..ab3ab130eeee 100644
--- a/drivers/crypto/amcc/crypto4xx_core.c
+++ b/drivers/crypto/amcc/crypto4xx_core.c
@@ -31,11 +31,9 @@
#include <crypto/ctr.h>
#include <crypto/gcm.h>
#include <crypto/sha1.h>
-#include <crypto/rng.h>
#include <crypto/scatterwalk.h>
#include <crypto/skcipher.h>
#include <crypto/internal/aead.h>
-#include <crypto/internal/rng.h>
#include <crypto/internal/skcipher.h>
#include "crypto4xx_reg_def.h"
#include "crypto4xx_core.h"
@@ -485,18 +483,6 @@ static void crypto4xx_copy_pkt_to_dst(struct crypto4xx_device *dev,
}
}
-static void crypto4xx_copy_digest_to_dst(void *dst,
- struct pd_uinfo *pd_uinfo,
- struct crypto4xx_ctx *ctx)
-{
- struct dynamic_sa_ctl *sa = (struct dynamic_sa_ctl *) ctx->sa_in;
-
- if (sa->sa_command_0.bf.hash_alg == SA_HASH_ALG_SHA1) {
- memcpy(dst, pd_uinfo->sr_va->save_digest,
- SA_HASH_ALG_SHA1_DIGEST_SIZE);
- }
-}
-
static void crypto4xx_ret_sg_desc(struct crypto4xx_device *dev,
struct pd_uinfo *pd_uinfo)
{
@@ -549,23 +535,6 @@ static void crypto4xx_cipher_done(struct crypto4xx_device *dev,
skcipher_request_complete(req, 0);
}
-static void crypto4xx_ahash_done(struct crypto4xx_device *dev,
- struct pd_uinfo *pd_uinfo)
-{
- struct crypto4xx_ctx *ctx;
- struct ahash_request *ahash_req;
-
- ahash_req = ahash_request_cast(pd_uinfo->async_req);
- ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(ahash_req));
-
- crypto4xx_copy_digest_to_dst(ahash_req->result, pd_uinfo, ctx);
- crypto4xx_ret_sg_desc(dev, pd_uinfo);
-
- if (pd_uinfo->state & PD_ENTRY_BUSY)
- ahash_request_complete(ahash_req, -EINPROGRESS);
- ahash_request_complete(ahash_req, 0);
-}
-
static void crypto4xx_aead_done(struct crypto4xx_device *dev,
struct pd_uinfo *pd_uinfo,
struct ce_pd *pd)
@@ -642,9 +611,6 @@ static void crypto4xx_pd_done(struct crypto4xx_device *dev, u32 idx)
case CRYPTO_ALG_TYPE_AEAD:
crypto4xx_aead_done(dev, pd_uinfo, pd);
break;
- case CRYPTO_ALG_TYPE_AHASH:
- crypto4xx_ahash_done(dev, pd_uinfo);
- break;
}
}
@@ -915,8 +881,7 @@ int crypto4xx_build_pd(struct crypto_async_request *req,
}
pd->pd_ctl.w = PD_CTL_HOST_READY |
- ((crypto_tfm_alg_type(req->tfm) == CRYPTO_ALG_TYPE_AHASH) ||
- (crypto_tfm_alg_type(req->tfm) == CRYPTO_ALG_TYPE_AEAD) ?
+ ((crypto_tfm_alg_type(req->tfm) == CRYPTO_ALG_TYPE_AEAD) ?
PD_CTL_HASH_FINAL : 0);
pd->pd_ctl_len.w = 0x00400000 | (assoclen + datalen);
pd_uinfo->state = PD_ENTRY_INUSE | (is_busy ? PD_ENTRY_BUSY : 0);
@@ -1022,14 +987,6 @@ static int crypto4xx_register_alg(struct crypto4xx_device *sec_dev,
rc = crypto_register_aead(&alg->alg.u.aead);
break;
- case CRYPTO_ALG_TYPE_AHASH:
- rc = crypto_register_ahash(&alg->alg.u.hash);
- break;
-
- case CRYPTO_ALG_TYPE_RNG:
- rc = crypto_register_rng(&alg->alg.u.rng);
- break;
-
default:
rc = crypto_register_skcipher(&alg->alg.u.cipher);
break;
@@ -1051,18 +1008,10 @@ static void crypto4xx_unregister_alg(struct crypto4xx_device *sec_dev)
list_for_each_entry_safe(alg, tmp, &sec_dev->alg_list, entry) {
list_del(&alg->entry);
switch (alg->alg.type) {
- case CRYPTO_ALG_TYPE_AHASH:
- crypto_unregister_ahash(&alg->alg.u.hash);
- break;
-
case CRYPTO_ALG_TYPE_AEAD:
crypto_unregister_aead(&alg->alg.u.aead);
break;
- case CRYPTO_ALG_TYPE_RNG:
- crypto_unregister_rng(&alg->alg.u.rng);
- break;
-
default:
crypto_unregister_skcipher(&alg->alg.u.cipher);
}
@@ -1121,69 +1070,6 @@ static irqreturn_t crypto4xx_ce_interrupt_handler_revb(int irq, void *data)
PPC4XX_TMO_ERR_INT);
}
-static int ppc4xx_prng_data_read(struct crypto4xx_device *dev,
- u8 *data, unsigned int max)
-{
- unsigned int i, curr = 0;
- u32 val[2];
-
- do {
- /* trigger PRN generation */
- writel(PPC4XX_PRNG_CTRL_AUTO_EN,
- dev->ce_base + CRYPTO4XX_PRNG_CTRL);
-
- for (i = 0; i < 1024; i++) {
- /* usually 19 iterations are enough */
- if ((readl(dev->ce_base + CRYPTO4XX_PRNG_STAT) &
- CRYPTO4XX_PRNG_STAT_BUSY))
- continue;
-
- val[0] = readl_be(dev->ce_base + CRYPTO4XX_PRNG_RES_0);
- val[1] = readl_be(dev->ce_base + CRYPTO4XX_PRNG_RES_1);
- break;
- }
- if (i == 1024)
- return -ETIMEDOUT;
-
- if ((max - curr) >= 8) {
- memcpy(data, &val, 8);
- data += 8;
- curr += 8;
- } else {
- /* copy only remaining bytes */
- memcpy(data, &val, max - curr);
- break;
- }
- } while (curr < max);
-
- return curr;
-}
-
-static int crypto4xx_prng_generate(struct crypto_rng *tfm,
- const u8 *src, unsigned int slen,
- u8 *dstn, unsigned int dlen)
-{
- struct rng_alg *alg = crypto_rng_alg(tfm);
- struct crypto4xx_alg *amcc_alg;
- struct crypto4xx_device *dev;
- int ret;
-
- amcc_alg = container_of(alg, struct crypto4xx_alg, alg.u.rng);
- dev = amcc_alg->dev;
-
- mutex_lock(&dev->core_dev->rng_lock);
- ret = ppc4xx_prng_data_read(dev, dstn, dlen);
- mutex_unlock(&dev->core_dev->rng_lock);
- return ret;
-}
-
-
-static int crypto4xx_prng_seed(struct crypto_rng *tfm, const u8 *seed,
- unsigned int slen)
-{
- return 0;
-}
-
/*
* Supported Crypto Algorithms
*/
@@ -1313,18 +1199,6 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = {
.cra_module = THIS_MODULE,
},
} },
- { .type = CRYPTO_ALG_TYPE_RNG, .u.rng = {
- .base = {
- .cra_name = "stdrng",
- .cra_driver_name = "crypto4xx_rng",
- .cra_priority = 300,
- .cra_ctxsize = 0,
- .cra_module = THIS_MODULE,
- },
- .generate = crypto4xx_prng_generate,
- .seed = crypto4xx_prng_seed,
- .seedsize = 0,
- } },
};
/*
@@ -1402,7 +1276,6 @@ static int crypto4xx_probe(struct platform_device *ofdev)
core_dev->dev->core_dev = core_dev;
core_dev->dev->is_revb = is_revb;
core_dev->device = dev;
- mutex_init(&core_dev->rng_lock);
spin_lock_init(&core_dev->lock);
INIT_LIST_HEAD(&core_dev->dev->alg_list);
ratelimit_default_init(&core_dev->dev->aead_ratelimit);
@@ -1480,7 +1353,6 @@ static void crypto4xx_remove(struct platform_device *ofdev)
tasklet_kill(&core_dev->tasklet);
/* Un-register with Linux CryptoAPI */
crypto4xx_unregister_alg(core_dev->dev);
- mutex_destroy(&core_dev->rng_lock);
/* Free all allocated memory */
crypto4xx_stop_all(core_dev);
}
diff --git a/drivers/crypto/amcc/crypto4xx_core.h b/drivers/crypto/amcc/crypto4xx_core.h
index 3adcc5e65694..d6a6a5699b41 100644
--- a/drivers/crypto/amcc/crypto4xx_core.h
+++ b/drivers/crypto/amcc/crypto4xx_core.h
@@ -14,11 +14,8 @@
#define __CRYPTO4XX_CORE_H__
#include <linux/ratelimit.h>
-#include <linux/mutex.h>
#include <linux/scatterlist.h>
-#include <crypto/internal/hash.h>
#include <crypto/internal/aead.h>
-#include <crypto/internal/rng.h>
#include <crypto/internal/skcipher.h>
#include "crypto4xx_reg_def.h"
#include "crypto4xx_sa.h"
@@ -112,7 +109,6 @@ struct crypto4xx_core_device {
u32 irq;
struct tasklet_struct tasklet;
spinlock_t lock;
- struct mutex rng_lock;
};
struct crypto4xx_ctx {
@@ -135,9 +131,7 @@ struct crypto4xx_alg_common {
u32 type;
union {
struct skcipher_alg cipher;
- struct ahash_alg hash;
struct aead_alg aead;
- struct rng_alg rng;
} u;
};
@@ -177,11 +171,6 @@ int crypto4xx_encrypt_noiv_block(struct skcipher_request *req);
int crypto4xx_decrypt_noiv_block(struct skcipher_request *req);
int crypto4xx_rfc3686_encrypt(struct skcipher_request *req);
int crypto4xx_rfc3686_decrypt(struct skcipher_request *req);
-int crypto4xx_sha1_alg_init(struct crypto_tfm *tfm);
-int crypto4xx_hash_digest(struct ahash_request *req);
-int crypto4xx_hash_final(struct ahash_request *req);
-int crypto4xx_hash_update(struct ahash_request *req);
-int crypto4xx_hash_init(struct ahash_request *req);
/*
* Note: Only use this function to copy items that is word aligned.
diff --git a/drivers/crypto/amcc/crypto4xx_reg_def.h b/drivers/crypto/amcc/crypto4xx_reg_def.h
index 1038061224da..73d626308a84 100644
--- a/drivers/crypto/amcc/crypto4xx_reg_def.h
+++ b/drivers/crypto/amcc/crypto4xx_reg_def.h
@@ -90,20 +90,9 @@
#define CRYPTO4XX_BYTE_ORDER_CFG 0x000600d8
#define CRYPTO4XX_ENDIAN_CFG 0x000600d8
-#define CRYPTO4XX_PRNG_STAT 0x00070000
-#define CRYPTO4XX_PRNG_STAT_BUSY 0x1
#define CRYPTO4XX_PRNG_CTRL 0x00070004
#define CRYPTO4XX_PRNG_SEED_L 0x00070008
#define CRYPTO4XX_PRNG_SEED_H 0x0007000c
-
-#define CRYPTO4XX_PRNG_RES_0 0x00070020
-#define CRYPTO4XX_PRNG_RES_1 0x00070024
-#define CRYPTO4XX_PRNG_RES_2 0x00070028
-#define CRYPTO4XX_PRNG_RES_3 0x0007002C
-
-#define CRYPTO4XX_PRNG_LFSR_L 0x00070030
-#define CRYPTO4XX_PRNG_LFSR_H 0x00070034
-
/*
* Initialize CRYPTO ENGINE registers, and memory bases.
*/
diff --git a/drivers/crypto/atmel-ecc.c b/drivers/crypto/atmel-ecc.c
index 813d8517e1c3..9da5a0388080 100644
--- a/drivers/crypto/atmel-ecc.c
+++ b/drivers/crypto/atmel-ecc.c
@@ -380,7 +380,7 @@ MODULE_DEVICE_TABLE(of, atmel_ecc_dt_ids);
#endif
static const struct i2c_device_id atmel_ecc_id[] = {
- { "atecc508a", 0 },
+ { "atecc508a" },
{ }
};
MODULE_DEVICE_TABLE(i2c, atmel_ecc_id);
diff --git a/drivers/crypto/atmel-sha204a.c b/drivers/crypto/atmel-sha204a.c
index 1add8dcd306f..8a3520c9a0ff 100644
--- a/drivers/crypto/atmel-sha204a.c
+++ b/drivers/crypto/atmel-sha204a.c
@@ -19,6 +19,12 @@
#include <linux/workqueue.h>
#include "atmel-i2c.h"
+/*
+ * According to review by Bill Cox [1], the ATSHA204 has very low entropy.
+ * [1] https://www.metzdowd.com/pipermail/cryptography/2014-December/023858.html
+ */
+static const unsigned short atsha204_quality = 1;
+
static void atmel_sha204a_rng_done(struct atmel_i2c_work_data *work_data,
void *areq, int status)
{
@@ -48,8 +54,8 @@ static int atmel_sha204a_rng_read_nonblocking(struct hwrng *rng, void *data,
if (rng->priv) {
work_data = (struct atmel_i2c_work_data *)rng->priv;
- max = min(sizeof(work_data->cmd.data), max);
- memcpy(data, &work_data->cmd.data, max);
+ max = min(RANDOM_RSP_SIZE - CMD_OVERHEAD_SIZE, max);
+ memcpy(data, &work_data->cmd.data[RSP_DATA_IDX], max);
rng->priv = 0;
} else {
work_data = kmalloc(sizeof(*work_data), GFP_ATOMIC);
@@ -87,8 +93,8 @@ static int atmel_sha204a_rng_read(struct hwrng *rng, void *data, size_t max,
if (ret)
return ret;
- max = min(sizeof(cmd.data), max);
- memcpy(data, cmd.data, max);
+ max = min(RANDOM_RSP_SIZE - CMD_OVERHEAD_SIZE, max);
+ memcpy(data, &cmd.data[RSP_DATA_IDX], max);
return max;
}
@@ -158,6 +164,7 @@ static const struct attribute_group atmel_sha204a_groups = {
static int atmel_sha204a_probe(struct i2c_client *client)
{
struct atmel_i2c_client_priv *i2c_priv;
+ const unsigned short *quality;
int ret;
ret = atmel_i2c_probe(client);
@@ -171,15 +178,15 @@ static int atmel_sha204a_probe(struct i2c_client *client)
i2c_priv->hwrng.name = dev_name(&client->dev);
i2c_priv->hwrng.read = atmel_sha204a_rng_read;
- /*
- * According to review by Bill Cox [1], this HWRNG has very low entropy.
- * [1] https://www.metzdowd.com/pipermail/cryptography/2014-December/023858.html
- */
- i2c_priv->hwrng.quality = 1;
+ quality = i2c_get_match_data(client);
+ if (quality)
+ i2c_priv->hwrng.quality = *quality;
ret = devm_hwrng_register(&client->dev, &i2c_priv->hwrng);
- if (ret)
- dev_warn(&client->dev, "failed to register RNG (%d)\n", ret);
+ if (ret) {
+ dev_err(&client->dev, "failed to register RNG (%d)\n", ret);
+ return ret;
+ }
/* otp read out */
if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C))
@@ -187,7 +194,7 @@ static int atmel_sha204a_probe(struct i2c_client *client)
ret = sysfs_create_group(&client->dev.kobj, &atmel_sha204a_groups);
if (ret) {
- dev_err(&client->dev, "failed to register sysfs entry\n");
+ dev_err(&client->dev, "failed to create sysfs group (%d)\n", ret);
return ret;
}
@@ -207,15 +214,15 @@ static void atmel_sha204a_remove(struct i2c_client *client)
}
static const struct of_device_id atmel_sha204a_dt_ids[] __maybe_unused = {
- { .compatible = "atmel,atsha204", },
+ { .compatible = "atmel,atsha204", .data = &atsha204_quality },
{ .compatible = "atmel,atsha204a", },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, atmel_sha204a_dt_ids);
static const struct i2c_device_id atmel_sha204a_id[] = {
- { "atsha204", 0 },
- { "atsha204a", 0 },
+ { "atsha204", (kernel_ulong_t)&atsha204_quality },
+ { "atsha204a" },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(i2c, atmel_sha204a_id);
diff --git a/drivers/crypto/cavium/cpt/cptvf_reqmanager.c b/drivers/crypto/cavium/cpt/cptvf_reqmanager.c
index 153004bdfb5c..753bdb2ae33f 100644
--- a/drivers/crypto/cavium/cpt/cptvf_reqmanager.c
+++ b/drivers/crypto/cavium/cpt/cptvf_reqmanager.c
@@ -108,8 +108,8 @@ static int setup_sgio_components(struct cpt_vf *cptvf, struct buf_ptr *list,
sg_cleanup:
for (j = 0; j < i; j++) {
if (list[j].dma_addr) {
- dma_unmap_single(&pdev->dev, list[i].dma_addr,
- list[i].size, DMA_BIDIRECTIONAL);
+ dma_unmap_single(&pdev->dev, list[j].dma_addr,
+ list[j].size, DMA_BIDIRECTIONAL);
}
list[j].dma_addr = 0;
diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c
index 2258337f1ea4..d668f9318c28 100644
--- a/drivers/crypto/ccp/sev-dev.c
+++ b/drivers/crypto/ccp/sev-dev.c
@@ -105,6 +105,13 @@ static void *sev_init_ex_buffer;
static void __sev_firmware_shutdown(struct sev_device *sev, bool panic);
+static int snp_shutdown_on_panic(struct notifier_block *nb,
+ unsigned long reason, void *arg);
+
+static struct notifier_block snp_panic_notifier = {
+ .notifier_call = snp_shutdown_on_panic,
+};
+
static inline bool sev_version_greater_or_equal(u8 maj, u8 min)
{
struct sev_device *sev = psp_master->sev_data;
@@ -1066,10 +1073,11 @@ static int snp_filter_reserved_mem_regions(struct resource *rs, void *arg)
size_t size;
/*
- * Ensure the list of HV_FIXED pages that will be passed to firmware
- * do not exceed the page-sized argument buffer.
+ * Ensure the list of HV_FIXED pages passed to the firmware including
+ * the one about to be written to do not exceed the page-sized argument
+ * buffer.
*/
- if ((range_list->num_elements * sizeof(struct sev_data_range) +
+ if (((range_list->num_elements + 1) * sizeof(struct sev_data_range) +
sizeof(struct sev_data_range_list)) > PAGE_SIZE)
return -E2BIG;
@@ -1173,21 +1181,34 @@ static int __sev_snp_init_locked(int *error)
wbinvd_on_all_cpus();
rc = __sev_do_cmd_locked(cmd, arg, error);
- if (rc)
+ if (rc) {
+ dev_err(sev->dev, "SEV-SNP: %s failed rc %d, error %#x\n",
+ cmd == SEV_CMD_SNP_INIT_EX ? "SNP_INIT_EX" : "SNP_INIT",
+ rc, *error);
return rc;
+ }
/* Prepare for first SNP guest launch after INIT. */
wbinvd_on_all_cpus();
rc = __sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, error);
- if (rc)
+ if (rc) {
+ dev_err(sev->dev, "SEV-SNP: SNP_DF_FLUSH failed rc %d, error %#x\n",
+ rc, *error);
return rc;
+ }
sev->snp_initialized = true;
dev_dbg(sev->dev, "SEV-SNP firmware initialized\n");
+ dev_info(sev->dev, "SEV-SNP API:%d.%d build:%d\n", sev->api_major,
+ sev->api_minor, sev->build);
+
+ atomic_notifier_chain_register(&panic_notifier_list,
+ &snp_panic_notifier);
+
sev_es_tmr_size = SNP_TMR_SIZE;
- return rc;
+ return 0;
}
static void __sev_platform_init_handle_tmr(struct sev_device *sev)
@@ -1250,9 +1271,11 @@ static int __sev_platform_init_handle_init_ex_path(struct sev_device *sev)
static int __sev_platform_init_locked(int *error)
{
- int rc, psp_ret = SEV_RET_NO_FW_CALL;
+ int rc, psp_ret, dfflush_error;
struct sev_device *sev;
+ psp_ret = dfflush_error = SEV_RET_NO_FW_CALL;
+
if (!psp_master || !psp_master->sev_data)
return -ENODEV;
@@ -1284,16 +1307,22 @@ static int __sev_platform_init_locked(int *error)
if (error)
*error = psp_ret;
- if (rc)
+ if (rc) {
+ dev_err(sev->dev, "SEV: %s failed %#x, rc %d\n",
+ sev_init_ex_buffer ? "INIT_EX" : "INIT", psp_ret, rc);
return rc;
+ }
sev->state = SEV_STATE_INIT;
/* Prepare for first SEV guest launch after INIT */
wbinvd_on_all_cpus();
- rc = __sev_do_cmd_locked(SEV_CMD_DF_FLUSH, NULL, error);
- if (rc)
+ rc = __sev_do_cmd_locked(SEV_CMD_DF_FLUSH, NULL, &dfflush_error);
+ if (rc) {
+ dev_err(sev->dev, "SEV: DF_FLUSH failed %#x, rc %d\n",
+ dfflush_error, rc);
return rc;
+ }
dev_dbg(sev->dev, "SEV firmware initialized\n");
@@ -1325,10 +1354,6 @@ static int _sev_platform_init_locked(struct sev_platform_init_args *args)
if (sev->state == SEV_STATE_INIT)
return 0;
- /*
- * Legacy guests cannot be running while SNP_INIT(_EX) is executing,
- * so perform SEV-SNP initialization at probe time.
- */
rc = __sev_snp_init_locked(&args->error);
if (rc && rc != -ENODEV) {
/*
@@ -1373,8 +1398,11 @@ static int __sev_platform_shutdown_locked(int *error)
return 0;
ret = __sev_do_cmd_locked(SEV_CMD_SHUTDOWN, NULL, error);
- if (ret)
+ if (ret) {
+ dev_err(sev->dev, "SEV: failed to SHUTDOWN error %#x, rc %d\n",
+ *error, ret);
return ret;
+ }
sev->state = SEV_STATE_UNINIT;
dev_dbg(sev->dev, "SEV firmware shutdown\n");
@@ -1408,21 +1436,6 @@ static int sev_move_to_init_state(struct sev_issue_cmd *argp, bool *shutdown_req
return 0;
}
-static int snp_move_to_init_state(struct sev_issue_cmd *argp, bool *shutdown_required)
-{
- int error, rc;
-
- rc = __sev_snp_init_locked(&error);
- if (rc) {
- argp->error = SEV_RET_INVALID_PLATFORM_STATE;
- return rc;
- }
-
- *shutdown_required = true;
-
- return 0;
-}
-
static int sev_ioctl_do_reset(struct sev_issue_cmd *argp, bool writable)
{
int state, rc;
@@ -1732,9 +1745,12 @@ static int __sev_snp_shutdown_locked(int *error, bool panic)
ret = __sev_do_cmd_locked(SEV_CMD_SNP_SHUTDOWN_EX, &data, error);
/* SHUTDOWN may require DF_FLUSH */
if (*error == SEV_RET_DFFLUSH_REQUIRED) {
- ret = __sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, NULL);
+ int dfflush_error = SEV_RET_NO_FW_CALL;
+
+ ret = __sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, &dfflush_error);
if (ret) {
- dev_err(sev->dev, "SEV-SNP DF_FLUSH failed\n");
+ dev_err(sev->dev, "SEV-SNP DF_FLUSH failed, ret = %d, error = %#x\n",
+ ret, dfflush_error);
return ret;
}
/* reissue the shutdown command */
@@ -1742,7 +1758,8 @@ static int __sev_snp_shutdown_locked(int *error, bool panic)
error);
}
if (ret) {
- dev_err(sev->dev, "SEV-SNP firmware shutdown failed\n");
+ dev_err(sev->dev, "SEV-SNP firmware shutdown failed, rc %d, error %#x\n",
+ ret, *error);
return ret;
}
@@ -1768,6 +1785,18 @@ static int __sev_snp_shutdown_locked(int *error, bool panic)
sev->snp_initialized = false;
dev_dbg(sev->dev, "SEV-SNP firmware shutdown\n");
+ /*
+ * __sev_snp_shutdown_locked() deadlocks when it tries to unregister
+ * itself during panic as the panic notifier is called with RCU read
+ * lock held and notifier unregistration does RCU synchronization.
+ */
+ if (!panic)
+ atomic_notifier_chain_unregister(&panic_notifier_list,
+ &snp_panic_notifier);
+
+ /* Reset TMR size back to default */
+ sev_es_tmr_size = SEV_TMR_SIZE;
+
return ret;
}
@@ -1948,7 +1977,8 @@ static int sev_ioctl_do_pdh_export(struct sev_issue_cmd *argp, bool writable)
/* Userspace wants to query the certificate length. */
if (!input.pdh_cert_address ||
!input.pdh_cert_len ||
- !input.cert_chain_address)
+ !input.cert_chain_address ||
+ !input.cert_chain_len)
goto cmd;
input_pdh_cert_address = (void __user *)input.pdh_cert_address;
@@ -2034,11 +2064,10 @@ static int sev_ioctl_do_pdh_export(struct sev_issue_cmd *argp, bool writable)
static int sev_ioctl_do_snp_platform_status(struct sev_issue_cmd *argp)
{
struct sev_device *sev = psp_master->sev_data;
- bool shutdown_required = false;
struct sev_data_snp_addr buf;
struct page *status_page;
- int ret, error;
void *data;
+ int ret;
if (!argp->data)
return -EINVAL;
@@ -2049,31 +2078,33 @@ static int sev_ioctl_do_snp_platform_status(struct sev_issue_cmd *argp)
data = page_address(status_page);
- if (!sev->snp_initialized) {
- ret = snp_move_to_init_state(argp, &shutdown_required);
- if (ret)
- goto cleanup;
- }
-
/*
- * Firmware expects status page to be in firmware-owned state, otherwise
- * it will report firmware error code INVALID_PAGE_STATE (0x1A).
+ * SNP_PLATFORM_STATUS can be executed in any SNP state. But if executed
+ * when SNP has been initialized, the status page must be firmware-owned.
*/
- if (rmp_mark_pages_firmware(__pa(data), 1, true)) {
- ret = -EFAULT;
- goto cleanup;
+ if (sev->snp_initialized) {
+ /*
+ * Firmware expects the status page to be in Firmware state,
+ * otherwise it will report an error INVALID_PAGE_STATE.
+ */
+ if (rmp_mark_pages_firmware(__pa(data), 1, true)) {
+ ret = -EFAULT;
+ goto cleanup;
+ }
}
buf.address = __psp_pa(data);
ret = __sev_do_cmd_locked(SEV_CMD_SNP_PLATFORM_STATUS, &buf, &argp->error);
- /*
- * Status page will be transitioned to Reclaim state upon success, or
- * left in Firmware state in failure. Use snp_reclaim_pages() to
- * transition either case back to Hypervisor-owned state.
- */
- if (snp_reclaim_pages(__pa(data), 1, true))
- return -EFAULT;
+ if (sev->snp_initialized) {
+ /*
+ * The status page will be in Reclaim state on success, or left
+ * in Firmware state on failure. Use snp_reclaim_pages() to
+ * transition either case back to Hypervisor-owned state.
+ */
+ if (snp_reclaim_pages(__pa(data), 1, true))
+ return -EFAULT;
+ }
if (ret)
goto cleanup;
@@ -2083,9 +2114,6 @@ static int sev_ioctl_do_snp_platform_status(struct sev_issue_cmd *argp)
ret = -EFAULT;
cleanup:
- if (shutdown_required)
- __sev_snp_shutdown_locked(&error, false);
-
__free_pages(status_page, 0);
return ret;
}
@@ -2106,8 +2134,6 @@ static int sev_ioctl_do_snp_set_config(struct sev_issue_cmd *argp, bool writable
{
struct sev_device *sev = psp_master->sev_data;
struct sev_user_data_snp_config config;
- bool shutdown_required = false;
- int ret, error;
if (!argp->data)
return -EINVAL;
@@ -2115,21 +2141,13 @@ static int sev_ioctl_do_snp_set_config(struct sev_issue_cmd *argp, bool writable
if (!writable)
return -EPERM;
+ if (!sev->snp_initialized)
+ return -ENODEV;
+
if (copy_from_user(&config, (void __user *)argp->data, sizeof(config)))
return -EFAULT;
- if (!sev->snp_initialized) {
- ret = snp_move_to_init_state(argp, &shutdown_required);
- if (ret)
- return ret;
- }
-
- ret = __sev_do_cmd_locked(SEV_CMD_SNP_CONFIG, &config, &argp->error);
-
- if (shutdown_required)
- __sev_snp_shutdown_locked(&error, false);
-
- return ret;
+ return __sev_do_cmd_locked(SEV_CMD_SNP_CONFIG, &config, &argp->error);
}
static int sev_ioctl_do_snp_vlek_load(struct sev_issue_cmd *argp, bool writable)
@@ -2388,7 +2406,7 @@ static void __sev_firmware_shutdown(struct sev_device *sev, bool panic)
{
int error;
- __sev_platform_shutdown_locked(NULL);
+ __sev_platform_shutdown_locked(&error);
if (sev_es_tmr) {
/*
@@ -2460,10 +2478,6 @@ static int snp_shutdown_on_panic(struct notifier_block *nb,
return NOTIFY_DONE;
}
-static struct notifier_block snp_panic_notifier = {
- .notifier_call = snp_shutdown_on_panic,
-};
-
int sev_issue_cmd_external_user(struct file *filep, unsigned int cmd,
void *data, int *error)
{
@@ -2477,9 +2491,7 @@ EXPORT_SYMBOL_GPL(sev_issue_cmd_external_user);
void sev_pci_init(void)
{
struct sev_device *sev = psp_master->sev_data;
- struct sev_platform_init_args args = {0};
u8 api_major, api_minor, build;
- int rc;
if (!sev)
return;
@@ -2502,18 +2514,6 @@ void sev_pci_init(void)
api_major, api_minor, build,
sev->api_major, sev->api_minor, sev->build);
- /* Initialize the platform */
- args.probe = true;
- rc = sev_platform_init(&args);
- if (rc)
- dev_err(sev->dev, "SEV: failed to INIT error %#x, rc %d\n",
- args.error, rc);
-
- dev_info(sev->dev, "SEV%s API:%d.%d build:%d\n", sev->snp_initialized ?
- "-SNP" : "", sev->api_major, sev->api_minor, sev->build);
-
- atomic_notifier_chain_register(&panic_notifier_list,
- &snp_panic_notifier);
return;
err:
@@ -2530,7 +2530,4 @@ void sev_pci_exit(void)
return;
sev_firmware_shutdown(sev);
-
- atomic_notifier_chain_unregister(&panic_notifier_list,
- &snp_panic_notifier);
}
diff --git a/drivers/crypto/hisilicon/Kconfig b/drivers/crypto/hisilicon/Kconfig
index 4137a8bf131f..107bfffaead2 100644
--- a/drivers/crypto/hisilicon/Kconfig
+++ b/drivers/crypto/hisilicon/Kconfig
@@ -79,6 +79,5 @@ config CRYPTO_DEV_HISI_TRNG
tristate "Support for HISI TRNG Driver"
depends on ARM64 && ACPI
select HW_RANDOM
- select CRYPTO_RNG
help
Support for HiSilicon TRNG Driver.
diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c
index 2c64bf4cdf0d..38c9d633db5f 100644
--- a/drivers/crypto/hisilicon/qm.c
+++ b/drivers/crypto/hisilicon/qm.c
@@ -4505,8 +4505,6 @@ void hisi_qm_reset_prepare(struct pci_dev *pdev)
u32 delay = 0;
int ret;
- hisi_qm_dev_err_uninit(pf_qm);
-
/*
* Check whether there is an ECC mbit error, If it occurs, need to
* wait for soft reset to fix it.
@@ -4523,6 +4521,8 @@ void hisi_qm_reset_prepare(struct pci_dev *pdev)
return;
}
+ hisi_qm_dev_err_uninit(pf_qm);
+
/* PF obtains the information of VF by querying the register. */
if (qm->fun_type == QM_HW_PF)
qm_cmd_uninit(qm);
diff --git a/drivers/crypto/hisilicon/trng/trng.c b/drivers/crypto/hisilicon/trng/trng.c
index 85a4b99f9055..6584ed051e09 100644
--- a/drivers/crypto/hisilicon/trng/trng.c
+++ b/drivers/crypto/hisilicon/trng/trng.c
@@ -1,234 +1,27 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2019 HiSilicon Limited. */
-#include <crypto/internal/rng.h>
#include <linux/acpi.h>
-#include <linux/crypto.h>
#include <linux/err.h>
#include <linux/hw_random.h>
#include <linux/io.h>
#include <linux/iopoll.h>
#include <linux/kernel.h>
-#include <linux/list.h>
#include <linux/module.h>
-#include <linux/mutex.h>
#include <linux/platform_device.h>
#include <linux/random.h>
#define HISI_TRNG_REG 0x00F0
#define HISI_TRNG_BYTES 4
#define HISI_TRNG_QUALITY 512
-#define HISI_TRNG_VERSION 0x01B8
-#define HISI_TRNG_VER_V1 GENMASK(31, 0)
#define SLEEP_US 10
#define TIMEOUT_US 10000
-#define SW_DRBG_NUM_SHIFT 2
-#define SW_DRBG_KEY_BASE 0x082C
-#define SW_DRBG_SEED(n) (SW_DRBG_KEY_BASE - ((n) << SW_DRBG_NUM_SHIFT))
-#define SW_DRBG_SEED_REGS_NUM 12
-#define SW_DRBG_SEED_SIZE 48
-#define SW_DRBG_BLOCKS 0x0830
-#define SW_DRBG_INIT 0x0834
-#define SW_DRBG_GEN 0x083c
-#define SW_DRBG_STATUS 0x0840
-#define SW_DRBG_BLOCKS_NUM 4095
-#define SW_DRBG_DATA_BASE 0x0850
-#define SW_DRBG_DATA_NUM 4
-#define SW_DRBG_DATA(n) (SW_DRBG_DATA_BASE - ((n) << SW_DRBG_NUM_SHIFT))
-#define SW_DRBG_BYTES 16
-#define SW_DRBG_ENABLE_SHIFT 12
-#define SEED_SHIFT_24 24
-#define SEED_SHIFT_16 16
-#define SEED_SHIFT_8 8
-#define SW_MAX_RANDOM_BYTES 65520
-
-struct hisi_trng_list {
- struct mutex lock;
- struct list_head list;
- bool is_init;
-};
struct hisi_trng {
void __iomem *base;
- struct hisi_trng_list *trng_list;
- struct list_head list;
struct hwrng rng;
- u32 ver;
- u32 ctx_num;
- /* The bytes of the random number generated since the last seeding. */
- u32 random_bytes;
- struct mutex lock;
-};
-
-struct hisi_trng_ctx {
- struct hisi_trng *trng;
};
-static atomic_t trng_active_devs;
-static struct hisi_trng_list trng_devices;
-static int hisi_trng_read(struct hwrng *rng, void *buf, size_t max, bool wait);
-
-static int hisi_trng_set_seed(struct hisi_trng *trng, const u8 *seed)
-{
- u32 val, seed_reg, i;
- int ret;
-
- writel(0x0, trng->base + SW_DRBG_BLOCKS);
-
- for (i = 0; i < SW_DRBG_SEED_SIZE;
- i += SW_DRBG_SEED_SIZE / SW_DRBG_SEED_REGS_NUM) {
- val = seed[i] << SEED_SHIFT_24;
- val |= seed[i + 1UL] << SEED_SHIFT_16;
- val |= seed[i + 2UL] << SEED_SHIFT_8;
- val |= seed[i + 3UL];
-
- seed_reg = (i >> SW_DRBG_NUM_SHIFT) % SW_DRBG_SEED_REGS_NUM;
- writel(val, trng->base + SW_DRBG_SEED(seed_reg));
- }
-
- writel(SW_DRBG_BLOCKS_NUM | (0x1 << SW_DRBG_ENABLE_SHIFT),
- trng->base + SW_DRBG_BLOCKS);
- writel(0x1, trng->base + SW_DRBG_INIT);
- ret = readl_relaxed_poll_timeout(trng->base + SW_DRBG_STATUS,
- val, val & BIT(0), SLEEP_US, TIMEOUT_US);
- if (ret) {
- pr_err("failed to init trng(%d)\n", ret);
- return -EIO;
- }
-
- trng->random_bytes = 0;
-
- return 0;
-}
-
-static int hisi_trng_seed(struct crypto_rng *tfm, const u8 *seed,
- unsigned int slen)
-{
- struct hisi_trng_ctx *ctx = crypto_rng_ctx(tfm);
- struct hisi_trng *trng = ctx->trng;
- int ret;
-
- if (slen < SW_DRBG_SEED_SIZE) {
- pr_err("slen(%u) is not matched with trng(%d)\n", slen,
- SW_DRBG_SEED_SIZE);
- return -EINVAL;
- }
-
- mutex_lock(&trng->lock);
- ret = hisi_trng_set_seed(trng, seed);
- mutex_unlock(&trng->lock);
-
- return ret;
-}
-
-static int hisi_trng_reseed(struct hisi_trng *trng)
-{
- u8 seed[SW_DRBG_SEED_SIZE];
- int size;
-
- if (!trng->random_bytes)
- return 0;
-
- size = hisi_trng_read(&trng->rng, seed, SW_DRBG_SEED_SIZE, false);
- if (size != SW_DRBG_SEED_SIZE)
- return -EIO;
-
- return hisi_trng_set_seed(trng, seed);
-}
-
-static int hisi_trng_get_bytes(struct hisi_trng *trng, u8 *dstn, unsigned int dlen)
-{
- u32 data[SW_DRBG_DATA_NUM];
- u32 currsize = 0;
- u32 val = 0;
- int ret;
- u32 i;
-
- ret = hisi_trng_reseed(trng);
- if (ret)
- return ret;
-
- do {
- ret = readl_relaxed_poll_timeout(trng->base + SW_DRBG_STATUS,
- val, val & BIT(1), SLEEP_US, TIMEOUT_US);
- if (ret) {
- pr_err("failed to generate random number(%d)!\n", ret);
- break;
- }
-
- for (i = 0; i < SW_DRBG_DATA_NUM; i++)
- data[i] = readl(trng->base + SW_DRBG_DATA(i));
-
- if (dlen - currsize >= SW_DRBG_BYTES) {
- memcpy(dstn + currsize, data, SW_DRBG_BYTES);
- currsize += SW_DRBG_BYTES;
- } else {
- memcpy(dstn + currsize, data, dlen - currsize);
- currsize = dlen;
- }
-
- trng->random_bytes += SW_DRBG_BYTES;
- writel(0x1, trng->base + SW_DRBG_GEN);
- } while (currsize < dlen);
-
- return ret;
-}
-
-static int hisi_trng_generate(struct crypto_rng *tfm, const u8 *src,
- unsigned int slen, u8 *dstn, unsigned int dlen)
-{
- struct hisi_trng_ctx *ctx = crypto_rng_ctx(tfm);
- struct hisi_trng *trng = ctx->trng;
- unsigned int currsize = 0;
- unsigned int block_size;
- int ret;
-
- if (!dstn || !dlen) {
- pr_err("output is error, dlen %u!\n", dlen);
- return -EINVAL;
- }
-
- do {
- block_size = min_t(unsigned int, dlen - currsize, SW_MAX_RANDOM_BYTES);
- mutex_lock(&trng->lock);
- ret = hisi_trng_get_bytes(trng, dstn + currsize, block_size);
- mutex_unlock(&trng->lock);
- if (ret)
- return ret;
- currsize += block_size;
- } while (currsize < dlen);
-
- return 0;
-}
-
-static int hisi_trng_init(struct crypto_tfm *tfm)
-{
- struct hisi_trng_ctx *ctx = crypto_tfm_ctx(tfm);
- struct hisi_trng *trng;
- u32 ctx_num = ~0;
-
- mutex_lock(&trng_devices.lock);
- list_for_each_entry(trng, &trng_devices.list, list) {
- if (trng->ctx_num < ctx_num) {
- ctx_num = trng->ctx_num;
- ctx->trng = trng;
- }
- }
- ctx->trng->ctx_num++;
- mutex_unlock(&trng_devices.lock);
-
- return 0;
-}
-
-static void hisi_trng_exit(struct crypto_tfm *tfm)
-{
- struct hisi_trng_ctx *ctx = crypto_tfm_ctx(tfm);
-
- mutex_lock(&trng_devices.lock);
- ctx->trng->ctx_num--;
- mutex_unlock(&trng_devices.lock);
-}
-
static int hisi_trng_read(struct hwrng *rng, void *buf, size_t max, bool wait)
{
struct hisi_trng *trng;
@@ -260,42 +53,6 @@ static int hisi_trng_read(struct hwrng *rng, void *buf, size_t max, bool wait)
return currsize;
}
-static struct rng_alg hisi_trng_alg = {
- .generate = hisi_trng_generate,
- .seed = hisi_trng_seed,
- .seedsize = SW_DRBG_SEED_SIZE,
- .base = {
- .cra_name = "stdrng",
- .cra_driver_name = "hisi_stdrng",
- .cra_priority = 300,
- .cra_ctxsize = sizeof(struct hisi_trng_ctx),
- .cra_module = THIS_MODULE,
- .cra_init = hisi_trng_init,
- .cra_exit = hisi_trng_exit,
- },
-};
-
-static void hisi_trng_add_to_list(struct hisi_trng *trng)
-{
- mutex_lock(&trng_devices.lock);
- list_add_tail(&trng->list, &trng_devices.list);
- mutex_unlock(&trng_devices.lock);
-}
-
-static int hisi_trng_del_from_list(struct hisi_trng *trng)
-{
- int ret = -EBUSY;
-
- mutex_lock(&trng_devices.lock);
- if (!trng->ctx_num) {
- list_del(&trng->list);
- ret = 0;
- }
- mutex_unlock(&trng_devices.lock);
-
- return ret;
-}
-
static int hisi_trng_probe(struct platform_device *pdev)
{
struct hisi_trng *trng;
@@ -305,68 +62,20 @@ static int hisi_trng_probe(struct platform_device *pdev)
if (!trng)
return -ENOMEM;
- platform_set_drvdata(pdev, trng);
-
trng->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(trng->base))
return PTR_ERR(trng->base);
- trng->ctx_num = 0;
- trng->random_bytes = SW_MAX_RANDOM_BYTES;
- mutex_init(&trng->lock);
- trng->ver = readl(trng->base + HISI_TRNG_VERSION);
- if (!trng_devices.is_init) {
- INIT_LIST_HEAD(&trng_devices.list);
- mutex_init(&trng_devices.lock);
- trng_devices.is_init = true;
- }
-
- hisi_trng_add_to_list(trng);
- if (trng->ver != HISI_TRNG_VER_V1 &&
- atomic_inc_return(&trng_active_devs) == 1) {
- ret = crypto_register_rng(&hisi_trng_alg);
- if (ret) {
- dev_err(&pdev->dev,
- "failed to register crypto(%d)\n", ret);
- atomic_dec_return(&trng_active_devs);
- goto err_remove_from_list;
- }
- }
-
trng->rng.name = pdev->name;
trng->rng.read = hisi_trng_read;
trng->rng.quality = HISI_TRNG_QUALITY;
+
ret = devm_hwrng_register(&pdev->dev, &trng->rng);
- if (ret) {
+ if (ret)
dev_err(&pdev->dev, "failed to register hwrng: %d!\n", ret);
- goto err_crypto_unregister;
- }
-
- return ret;
-
-err_crypto_unregister:
- if (trng->ver != HISI_TRNG_VER_V1 &&
- atomic_dec_return(&trng_active_devs) == 0)
- crypto_unregister_rng(&hisi_trng_alg);
-
-err_remove_from_list:
- hisi_trng_del_from_list(trng);
return ret;
}
-static void hisi_trng_remove(struct platform_device *pdev)
-{
- struct hisi_trng *trng = platform_get_drvdata(pdev);
-
- /* Wait until the task is finished */
- while (hisi_trng_del_from_list(trng))
- ;
-
- if (trng->ver != HISI_TRNG_VER_V1 &&
- atomic_dec_return(&trng_active_devs) == 0)
- crypto_unregister_rng(&hisi_trng_alg);
-}
-
static const struct acpi_device_id hisi_trng_acpi_match[] = {
{ "HISI02B3", 0 },
{ }
@@ -375,7 +84,6 @@ MODULE_DEVICE_TABLE(acpi, hisi_trng_acpi_match);
static struct platform_driver hisi_trng_driver = {
.probe = hisi_trng_probe,
- .remove_new = hisi_trng_remove,
.driver = {
.name = "hisi-trng-v2",
.acpi_match_table = ACPI_PTR(hisi_trng_acpi_match),
diff --git a/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h b/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h
index cfe5bb9f5f7f..f5b4cf2b515c 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h
+++ b/drivers/crypto/intel/qat/qat_common/adf_accel_devices.h
@@ -485,6 +485,8 @@ struct adf_accel_dev {
struct {
/* protects VF2PF interrupts access */
spinlock_t vf2pf_ints_lock;
+ /* prevents VF2PF handling from racing with VF state teardown */
+ bool vf2pf_disabled;
/* vf_info is non-zero when SR-IOV is init'ed */
struct adf_accel_vf_info *vf_info;
} pf;
diff --git a/drivers/crypto/intel/qat/qat_common/adf_aer.c b/drivers/crypto/intel/qat/qat_common/adf_aer.c
index d2078cde8d86..bd81458bcf32 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_aer.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_aer.c
@@ -157,13 +157,14 @@ static int adf_dev_aer_schedule_reset(struct adf_accel_dev *accel_dev,
struct adf_reset_dev_data *reset_data;
if (!adf_dev_started(accel_dev) ||
- test_bit(ADF_STATUS_RESTARTING, &accel_dev->status))
+ test_and_set_bit(ADF_STATUS_RESTARTING, &accel_dev->status))
return 0;
- set_bit(ADF_STATUS_RESTARTING, &accel_dev->status);
reset_data = kzalloc(sizeof(*reset_data), GFP_KERNEL);
- if (!reset_data)
+ if (!reset_data) {
+ clear_bit(ADF_STATUS_RESTARTING, &accel_dev->status);
return -ENOMEM;
+ }
reset_data->accel_dev = accel_dev;
init_completion(&reset_data->compl);
reset_data->mode = mode;
diff --git a/drivers/crypto/intel/qat/qat_common/adf_common_drv.h b/drivers/crypto/intel/qat/qat_common/adf_common_drv.h
index fc2fea0d3ada..278f6ed9dfff 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_common_drv.h
+++ b/drivers/crypto/intel/qat/qat_common/adf_common_drv.h
@@ -91,7 +91,6 @@ void adf_exit_aer(void);
int adf_init_arb(struct adf_accel_dev *accel_dev);
void adf_exit_arb(struct adf_accel_dev *accel_dev);
void adf_update_ring_arb(struct adf_etr_ring_data *ring);
-int adf_disable_arb_thd(struct adf_accel_dev *accel_dev, u32 ae, u32 thr);
int adf_dev_get(struct adf_accel_dev *accel_dev);
void adf_dev_put(struct adf_accel_dev *accel_dev);
@@ -120,6 +119,7 @@ void qat_comp_alg_callback(void *resp);
int adf_isr_resource_alloc(struct adf_accel_dev *accel_dev);
void adf_isr_resource_free(struct adf_accel_dev *accel_dev);
+void adf_isr_sync_ae_cluster(struct adf_accel_dev *accel_dev);
int adf_vf_isr_resource_alloc(struct adf_accel_dev *accel_dev);
void adf_vf_isr_resource_free(struct adf_accel_dev *accel_dev);
@@ -193,6 +193,7 @@ int adf_sriov_configure(struct pci_dev *pdev, int numvfs);
void adf_disable_sriov(struct adf_accel_dev *accel_dev);
void adf_reenable_sriov(struct adf_accel_dev *accel_dev);
void adf_enable_vf2pf_interrupts(struct adf_accel_dev *accel_dev, u32 vf_mask);
+void adf_enable_all_vf2pf_interrupts(struct adf_accel_dev *accel_dev, u32 num_vfs);
void adf_disable_all_vf2pf_interrupts(struct adf_accel_dev *accel_dev);
bool adf_recv_and_handle_pf2vf_msg(struct adf_accel_dev *accel_dev);
bool adf_recv_and_handle_vf2pf_msg(struct adf_accel_dev *accel_dev, u32 vf_nr);
diff --git a/drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c b/drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c
index a3b474bdef6c..023c5f1e78b0 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c
@@ -64,10 +64,8 @@ int adf_heartbeat_inject_error(struct adf_accel_dev *accel_dev)
if (ret)
return ret;
- /* Configure worker threads to stop processing any packet */
- ret = adf_disable_arb_thd(accel_dev, rand_ae, rand_thr);
- if (ret)
- return ret;
+ /* Disable arbiter to stop processing any packet */
+ hw_device->exit_arb(accel_dev);
/* Change HB counters memory to simulate a hang */
adf_set_hb_counters_fail(accel_dev, rand_ae, rand_thr);
diff --git a/drivers/crypto/intel/qat/qat_common/adf_hw_arbiter.c b/drivers/crypto/intel/qat/qat_common/adf_hw_arbiter.c
index f93d9cca70ce..dd9a31c20bc9 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_hw_arbiter.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_hw_arbiter.c
@@ -99,28 +99,3 @@ void adf_exit_arb(struct adf_accel_dev *accel_dev)
csr_ops->write_csr_ring_srv_arb_en(csr, i, 0);
}
EXPORT_SYMBOL_GPL(adf_exit_arb);
-
-int adf_disable_arb_thd(struct adf_accel_dev *accel_dev, u32 ae, u32 thr)
-{
- void __iomem *csr = accel_dev->transport->banks[0].csr_addr;
- struct adf_hw_device_data *hw_data = accel_dev->hw_device;
- const u32 *thd_2_arb_cfg;
- struct arb_info info;
- u32 ae_thr_map;
-
- if (ADF_AE_STRAND0_THREAD == thr || ADF_AE_STRAND1_THREAD == thr)
- thr = ADF_AE_ADMIN_THREAD;
-
- hw_data->get_arb_info(&info);
- thd_2_arb_cfg = hw_data->get_arb_mapping(accel_dev);
- if (!thd_2_arb_cfg)
- return -EFAULT;
-
- /* Disable scheduling for this particular AE and thread */
- ae_thr_map = *(thd_2_arb_cfg + ae);
- ae_thr_map &= ~(GENMASK(3, 0) << (thr * BIT(2)));
-
- WRITE_CSR_ARB_WT2SAM(csr, info.arb_offset, info.wt2sam_offset, ae,
- ae_thr_map);
- return 0;
-}
diff --git a/drivers/crypto/intel/qat/qat_common/adf_isr.c b/drivers/crypto/intel/qat/qat_common/adf_isr.c
index 12e565613661..7683f04adc58 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_isr.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_isr.c
@@ -62,6 +62,23 @@ void adf_enable_vf2pf_interrupts(struct adf_accel_dev *accel_dev, u32 vf_mask)
unsigned long flags;
spin_lock_irqsave(&accel_dev->pf.vf2pf_ints_lock, flags);
+ if (!READ_ONCE(accel_dev->pf.vf2pf_disabled))
+ GET_PFVF_OPS(accel_dev)->enable_vf2pf_interrupts(pmisc_addr, vf_mask);
+ spin_unlock_irqrestore(&accel_dev->pf.vf2pf_ints_lock, flags);
+}
+
+void adf_enable_all_vf2pf_interrupts(struct adf_accel_dev *accel_dev, u32 num_vfs)
+{
+ void __iomem *pmisc_addr = adf_get_pmisc_base(accel_dev);
+ unsigned long flags;
+ u32 vf_mask;
+
+ vf_mask = BIT_ULL(num_vfs) - 1;
+ if (!vf_mask)
+ return;
+
+ spin_lock_irqsave(&accel_dev->pf.vf2pf_ints_lock, flags);
+ WRITE_ONCE(accel_dev->pf.vf2pf_disabled, false);
GET_PFVF_OPS(accel_dev)->enable_vf2pf_interrupts(pmisc_addr, vf_mask);
spin_unlock_irqrestore(&accel_dev->pf.vf2pf_ints_lock, flags);
}
@@ -72,6 +89,7 @@ void adf_disable_all_vf2pf_interrupts(struct adf_accel_dev *accel_dev)
unsigned long flags;
spin_lock_irqsave(&accel_dev->pf.vf2pf_ints_lock, flags);
+ WRITE_ONCE(accel_dev->pf.vf2pf_disabled, true);
GET_PFVF_OPS(accel_dev)->disable_all_vf2pf_interrupts(pmisc_addr);
spin_unlock_irqrestore(&accel_dev->pf.vf2pf_ints_lock, flags);
}
@@ -174,6 +192,27 @@ static irqreturn_t adf_msix_isr_ae(int irq, void *dev_ptr)
return IRQ_NONE;
}
+void adf_isr_sync_ae_cluster(struct adf_accel_dev *accel_dev)
+{
+ struct adf_accel_pci *pci_dev_info = &accel_dev->accel_pci_dev;
+ struct adf_hw_device_data *hw_data = GET_HW_DATA(accel_dev);
+ u32 num_entries = pci_dev_info->msix_entries.num_entries;
+ struct adf_irq *irqs = pci_dev_info->msix_entries.irqs;
+ u32 irq_idx;
+ int irq;
+
+ if (!test_bit(ADF_STATUS_IRQ_ALLOCATED, &accel_dev->status) || !irqs)
+ return;
+
+ irq_idx = num_entries > 1 ? hw_data->num_banks : 0;
+ if (irq_idx >= num_entries || !irqs[irq_idx].enabled)
+ return;
+
+ irq = pci_irq_vector(pci_dev_info->pci_dev, hw_data->num_banks);
+ if (irq > 0)
+ synchronize_irq(irq);
+}
+
static void adf_free_irqs(struct adf_accel_dev *accel_dev)
{
struct adf_accel_pci *pci_dev_info = &accel_dev->accel_pci_dev;
diff --git a/drivers/crypto/intel/qat/qat_common/adf_sriov.c b/drivers/crypto/intel/qat/qat_common/adf_sriov.c
index e9f825368b7e..ba1eb368fd65 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_sriov.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_sriov.c
@@ -26,6 +26,9 @@ static void adf_iov_send_resp(struct work_struct *work)
u32 vf_nr = vf_info->vf_nr;
bool ret;
+ if (READ_ONCE(accel_dev->pf.vf2pf_disabled))
+ goto out;
+
mutex_lock(&vf_info->pfvf_mig_lock);
ret = adf_recv_and_handle_vf2pf_msg(accel_dev, vf_nr);
if (ret)
@@ -33,13 +36,18 @@ static void adf_iov_send_resp(struct work_struct *work)
adf_enable_vf2pf_interrupts(accel_dev, 1 << vf_nr);
mutex_unlock(&vf_info->pfvf_mig_lock);
+out:
kfree(pf2vf_resp);
}
void adf_schedule_vf2pf_handler(struct adf_accel_vf_info *vf_info)
{
+ struct adf_accel_dev *accel_dev = vf_info->accel_dev;
struct adf_pf2vf_resp *pf2vf_resp;
+ if (READ_ONCE(accel_dev->pf.vf2pf_disabled))
+ return;
+
pf2vf_resp = kzalloc(sizeof(*pf2vf_resp), GFP_ATOMIC);
if (!pf2vf_resp)
return;
@@ -49,6 +57,12 @@ void adf_schedule_vf2pf_handler(struct adf_accel_vf_info *vf_info)
queue_work(pf2vf_resp_wq, &pf2vf_resp->pf2vf_resp_work);
}
+static void adf_flush_pf2vf_resp_wq(void)
+{
+ if (pf2vf_resp_wq)
+ flush_workqueue(pf2vf_resp_wq);
+}
+
static int adf_enable_sriov(struct adf_accel_dev *accel_dev)
{
struct pci_dev *pdev = accel_to_pci_dev(accel_dev);
@@ -75,7 +89,7 @@ static int adf_enable_sriov(struct adf_accel_dev *accel_dev)
hw_data->configure_iov_threads(accel_dev, true);
/* Enable VF to PF interrupts for all VFs */
- adf_enable_vf2pf_interrupts(accel_dev, BIT_ULL(totalvfs) - 1);
+ adf_enable_all_vf2pf_interrupts(accel_dev, totalvfs);
/* Do not enable SR-IOV if already enabled */
if (pci_num_vf(pdev))
@@ -259,8 +273,10 @@ void adf_disable_sriov(struct adf_accel_dev *accel_dev)
if (!test_bit(ADF_STATUS_RESTARTING, &accel_dev->status))
pci_disable_sriov(accel_to_pci_dev(accel_dev));
- /* Disable VF to PF interrupts */
+ /* Block VF2PF work and disable VF to PF interrupts */
adf_disable_all_vf2pf_interrupts(accel_dev);
+ adf_isr_sync_ae_cluster(accel_dev);
+ adf_flush_pf2vf_resp_wq();
/* Clear Valid bits in AE Thread to PCIe Function Mapping */
if (hw_data->configure_iov_threads)
diff --git a/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.c b/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.c
index c80baf1ad90b..89030e2711ce 100644
--- a/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.c
+++ b/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.c
@@ -157,8 +157,8 @@ static inline int setup_sgio_components(struct pci_dev *pdev,
sg_cleanup:
for (j = 0; j < i; j++) {
if (list[j].dma_addr) {
- dma_unmap_single(&pdev->dev, list[i].dma_addr,
- list[i].size, DMA_BIDIRECTIONAL);
+ dma_unmap_single(&pdev->dev, list[j].dma_addr,
+ list[j].size, DMA_BIDIRECTIONAL);
}
list[j].dma_addr = 0;
diff --git a/drivers/crypto/tegra/tegra-se-aes.c b/drivers/crypto/tegra/tegra-se-aes.c
index 30c78afe3dea..9094c03e991f 100644
--- a/drivers/crypto/tegra/tegra-se-aes.c
+++ b/drivers/crypto/tegra/tegra-se-aes.c
@@ -1201,6 +1201,7 @@ static int tegra_ccm_do_one_req(struct crypto_engine *engine, void *areq)
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct tegra_aead_ctx *ctx = crypto_aead_ctx(tfm);
struct tegra_se *se = ctx->se;
+ unsigned int bufsize;
int ret;
ret = tegra_ccm_crypt_init(req, se, rctx);
@@ -1210,19 +1211,19 @@ static int tegra_ccm_do_one_req(struct crypto_engine *engine, void *areq)
rctx->key_id = ctx->key_id;
/* Allocate buffers required */
- rctx->inbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen + 100;
- rctx->inbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->inbuf.size,
+ bufsize = rctx->assoclen + rctx->authsize + rctx->cryptlen + 100;
+ rctx->inbuf.size = bufsize;
+ rctx->inbuf.buf = dma_alloc_coherent(ctx->se->dev, bufsize,
&rctx->inbuf.addr, GFP_KERNEL);
+ ret = -ENOMEM;
if (!rctx->inbuf.buf)
goto out_finalize;
- rctx->outbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen + 100;
- rctx->outbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->outbuf.size,
+ rctx->outbuf.size = bufsize;
+ rctx->outbuf.buf = dma_alloc_coherent(ctx->se->dev, bufsize,
&rctx->outbuf.addr, GFP_KERNEL);
- if (!rctx->outbuf.buf) {
- ret = -ENOMEM;
+ if (!rctx->outbuf.buf)
goto out_free_inbuf;
- }
if (!ctx->key_id) {
ret = tegra_key_submit_reserved_aes(ctx->se, ctx->key,
@@ -1254,11 +1255,11 @@ static int tegra_ccm_do_one_req(struct crypto_engine *engine, void *areq)
}
out:
- dma_free_coherent(ctx->se->dev, rctx->inbuf.size,
+ dma_free_coherent(ctx->se->dev, bufsize,
rctx->outbuf.buf, rctx->outbuf.addr);
out_free_inbuf:
- dma_free_coherent(ctx->se->dev, rctx->outbuf.size,
+ dma_free_coherent(ctx->se->dev, bufsize,
rctx->inbuf.buf, rctx->inbuf.addr);
if (tegra_key_is_reserved(rctx->key_id))
@@ -1278,6 +1279,7 @@ static int tegra_gcm_do_one_req(struct crypto_engine *engine, void *areq)
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct tegra_aead_ctx *ctx = crypto_aead_ctx(tfm);
struct tegra_aead_reqctx *rctx = aead_request_ctx(req);
+ unsigned int bufsize;
int ret;
rctx->src_sg = req->src;
@@ -1296,16 +1298,17 @@ static int tegra_gcm_do_one_req(struct crypto_engine *engine, void *areq)
rctx->key_id = ctx->key_id;
/* Allocate buffers required */
- rctx->inbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen;
- rctx->inbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->inbuf.size,
+ bufsize = rctx->assoclen + rctx->authsize + rctx->cryptlen;
+ rctx->inbuf.size = bufsize;
+ rctx->inbuf.buf = dma_alloc_coherent(ctx->se->dev, bufsize,
&rctx->inbuf.addr, GFP_KERNEL);
if (!rctx->inbuf.buf) {
ret = -ENOMEM;
goto out_finalize;
}
- rctx->outbuf.size = rctx->assoclen + rctx->authsize + rctx->cryptlen;
- rctx->outbuf.buf = dma_alloc_coherent(ctx->se->dev, rctx->outbuf.size,
+ rctx->outbuf.size = bufsize;
+ rctx->outbuf.buf = dma_alloc_coherent(ctx->se->dev, bufsize,
&rctx->outbuf.addr, GFP_KERNEL);
if (!rctx->outbuf.buf) {
ret = -ENOMEM;
@@ -1342,11 +1345,11 @@ static int tegra_gcm_do_one_req(struct crypto_engine *engine, void *areq)
ret = tegra_gcm_do_verify(ctx->se, rctx);
out:
- dma_free_coherent(ctx->se->dev, rctx->outbuf.size,
+ dma_free_coherent(ctx->se->dev, bufsize,
rctx->outbuf.buf, rctx->outbuf.addr);
out_free_inbuf:
- dma_free_coherent(ctx->se->dev, rctx->inbuf.size,
+ dma_free_coherent(ctx->se->dev, bufsize,
rctx->inbuf.buf, rctx->inbuf.addr);
if (tegra_key_is_reserved(rctx->key_id))
diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
index e97d47f42ee2..a09bdc17e8f6 100644
--- a/drivers/dax/kmem.c
+++ b/drivers/dax/kmem.c
@@ -220,6 +220,12 @@ static void dev_dax_kmem_remove(struct dev_dax *dev_dax)
if (rc)
continue;
+ /* range was never added during probe */
+ if (!data->res[i]) {
+ success++;
+ continue;
+ }
+
rc = remove_memory(range.start, range_len(&range));
if (rc == 0) {
remove_resource(data->res[i]);
diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c
index 47d95d2d743b..06efffc1a3a5 100644
--- a/drivers/dma/dma-axi-dmac.c
+++ b/drivers/dma/dma-axi-dmac.c
@@ -11,6 +11,7 @@
#include <linux/device.h>
#include <linux/dma-mapping.h>
#include <linux/dmaengine.h>
+#include <linux/dmapool.h>
#include <linux/err.h>
#include <linux/interrupt.h>
#include <linux/io.h>
@@ -143,6 +144,7 @@ struct axi_dmac_chan {
struct virt_dma_chan vchan;
struct axi_dmac_desc *next_desc;
+ void *pool;
struct list_head active_descs;
enum dma_transfer_direction direction;
@@ -524,11 +526,17 @@ static void axi_dmac_issue_pending(struct dma_chan *c)
spin_unlock_irqrestore(&chan->vchan.lock, flags);
}
+static void axi_dmac_free_desc(struct axi_dmac_desc *desc)
+{
+ for (unsigned int i = 0; i < desc->num_sgs; i++)
+ dma_pool_free(desc->chan->pool, desc->sg[i].hw, desc->sg[i].hw_phys);
+
+ kfree(desc);
+}
+
static struct axi_dmac_desc *
axi_dmac_alloc_desc(struct axi_dmac_chan *chan, unsigned int num_sgs)
{
- struct axi_dmac *dmac = chan_to_axi_dmac(chan);
- struct device *dev = dmac->dma_dev.dev;
struct axi_dmac_hw_desc *hws;
struct axi_dmac_desc *desc;
dma_addr_t hw_phys;
@@ -540,22 +548,22 @@ axi_dmac_alloc_desc(struct axi_dmac_chan *chan, unsigned int num_sgs)
desc->num_sgs = num_sgs;
desc->chan = chan;
- hws = dma_alloc_coherent(dev, PAGE_ALIGN(num_sgs * sizeof(*hws)),
- &hw_phys, GFP_ATOMIC);
- if (!hws) {
- kfree(desc);
- return NULL;
- }
-
for (i = 0; i < num_sgs; i++) {
- desc->sg[i].hw = &hws[i];
- desc->sg[i].hw_phys = hw_phys + i * sizeof(*hws);
+ hws = dma_pool_zalloc(chan->pool, GFP_NOWAIT, &hw_phys);
+ if (!hws) {
+ desc->num_sgs = i;
+ axi_dmac_free_desc(desc);
+ return NULL;
+ }
+
+ desc->sg[i].hw = hws;
+ desc->sg[i].hw_phys = hw_phys;
- hws[i].id = AXI_DMAC_SG_UNUSED;
- hws[i].flags = 0;
+ hws->id = AXI_DMAC_SG_UNUSED;
/* Link hardware descriptors */
- hws[i].next_sg_addr = hw_phys + (i + 1) * sizeof(*hws);
+ if (i)
+ desc->sg[i - 1].hw->next_sg_addr = hw_phys;
}
/* The last hardware descriptor will trigger an interrupt */
@@ -564,18 +572,6 @@ axi_dmac_alloc_desc(struct axi_dmac_chan *chan, unsigned int num_sgs)
return desc;
}
-static void axi_dmac_free_desc(struct axi_dmac_desc *desc)
-{
- struct axi_dmac *dmac = chan_to_axi_dmac(desc->chan);
- struct device *dev = dmac->dma_dev.dev;
- struct axi_dmac_hw_desc *hw = desc->sg[0].hw;
- dma_addr_t hw_phys = desc->sg[0].hw_phys;
-
- dma_free_coherent(dev, PAGE_ALIGN(desc->num_sgs * sizeof(*hw)),
- hw, hw_phys);
- kfree(desc);
-}
-
static struct axi_dmac_sg *axi_dmac_fill_linear_sg(struct axi_dmac_chan *chan,
enum dma_transfer_direction direction, dma_addr_t addr,
unsigned int num_periods, unsigned int period_len,
@@ -643,7 +639,7 @@ axi_dmac_prep_peripheral_dma_vec(struct dma_chan *c, const struct dma_vec *vecs,
for (i = 0; i < nb; i++) {
if (!axi_dmac_check_addr(chan, vecs[i].addr) ||
!axi_dmac_check_len(chan, vecs[i].len)) {
- kfree(desc);
+ axi_dmac_free_desc(desc);
return NULL;
}
@@ -807,9 +803,26 @@ static struct dma_async_tx_descriptor *axi_dmac_prep_interleaved(
return vchan_tx_prep(&chan->vchan, &desc->vdesc, flags);
}
+static int axi_dmac_alloc_chan_resources(struct dma_chan *c)
+{
+ struct axi_dmac_chan *chan = to_axi_dmac_chan(c);
+ struct device *dev = c->device->dev;
+
+ chan->pool = dma_pool_create(dev_name(dev), dev,
+ sizeof(struct axi_dmac_hw_desc),
+ __alignof__(struct axi_dmac_hw_desc), 0);
+ if (!chan->pool)
+ return -ENOMEM;
+
+ return 0;
+}
+
static void axi_dmac_free_chan_resources(struct dma_chan *c)
{
+ struct axi_dmac_chan *chan = to_axi_dmac_chan(c);
+
vchan_free_chan_resources(to_virt_chan(c));
+ dma_pool_destroy(chan->pool);
}
static void axi_dmac_desc_free(struct virt_dma_desc *vdesc)
@@ -1093,6 +1106,7 @@ static int axi_dmac_probe(struct platform_device *pdev)
dma_cap_set(DMA_SLAVE, dma_dev->cap_mask);
dma_cap_set(DMA_CYCLIC, dma_dev->cap_mask);
dma_cap_set(DMA_INTERLEAVE, dma_dev->cap_mask);
+ dma_dev->device_alloc_chan_resources = axi_dmac_alloc_chan_resources;
dma_dev->device_free_chan_resources = axi_dmac_free_chan_resources;
dma_dev->device_tx_status = dma_cookie_status;
dma_dev->device_issue_pending = axi_dmac_issue_pending;
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index c1357d7f3dc6..e241b7b01233 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -905,11 +905,12 @@ void dma_release_channel(struct dma_chan *chan)
mutex_lock(&dma_list_mutex);
WARN_ONCE(chan->client_count != 1,
"chan reference count %d != 1\n", chan->client_count);
- dma_chan_put(chan);
/* drop PRIVATE cap enabled by __dma_request_channel() */
if (--chan->device->privatecnt == 0)
dma_cap_clear(DMA_PRIVATE, chan->device->cap_mask);
+ dma_chan_put(chan);
+
if (chan->slave) {
sysfs_remove_link(&chan->dev->device.kobj, DMA_SLAVE_NAME);
sysfs_remove_link(&chan->slave->kobj, chan->name);
diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-edma-core.h
index 71894b9e0b15..cc783e076671 100644
--- a/drivers/dma/dw-edma/dw-edma-core.h
+++ b/drivers/dma/dw-edma/dw-edma-core.h
@@ -108,7 +108,7 @@ struct dw_edma {
struct dw_edma_chan *chan;
- raw_spinlock_t lock; /* Only for legacy */
+ raw_spinlock_t lock; /* Protect v0 shared registers */
struct dw_edma_chip *chip;
diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw-edma-v0-core.c
index b75fdaffad9a..2850a9df80f5 100644
--- a/drivers/dma/dw-edma/dw-edma-v0-core.c
+++ b/drivers/dma/dw-edma/dw-edma-v0-core.c
@@ -364,6 +364,7 @@ static void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
{
struct dw_edma_chan *chan = chunk->chan;
struct dw_edma *dw = chan->dw;
+ unsigned long flags;
u32 tmp;
dw_edma_v0_core_write_chunk(chunk);
@@ -408,6 +409,8 @@ static void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
}
}
/* Interrupt unmask - done, abort */
+ raw_spin_lock_irqsave(&dw->lock, flags);
+
tmp = GET_RW_32(dw, chan->dir, int_mask);
tmp &= ~FIELD_PREP(EDMA_V0_DONE_INT_MASK, BIT(chan->id));
tmp &= ~FIELD_PREP(EDMA_V0_ABORT_INT_MASK, BIT(chan->id));
@@ -416,6 +419,9 @@ static void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
tmp = GET_RW_32(dw, chan->dir, linked_list_err_en);
tmp |= FIELD_PREP(EDMA_V0_LINKED_LIST_ERR_MASK, BIT(chan->id));
SET_RW_32(dw, chan->dir, linked_list_err_en, tmp);
+
+ raw_spin_unlock_irqrestore(&dw->lock, flags);
+
/* Channel control */
SET_CH_32(dw, chan->dir, chan->id, ch_control1,
(DW_EDMA_V0_CCS | DW_EDMA_V0_LLE));
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index 72299a08af44..3421712e397a 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -2377,7 +2377,9 @@ static int sdma_probe(struct platform_device *pdev)
goto err_register;
}
- spba_bus = of_find_compatible_node(NULL, NULL, "fsl,spba-bus");
+ struct device_node *sdma_parent_np __free(device_node) = of_get_parent(np);
+
+ spba_bus = of_get_compatible_child(sdma_parent_np, "fsl,spba-bus");
ret = of_address_to_resource(spba_bus, 0, &spba_res);
if (!ret) {
sdma->spba_start_addr = spba_res.start;
diff --git a/drivers/dma/qcom/gpi.c b/drivers/dma/qcom/gpi.c
index b0ed6260e75b..da0fa1020347 100644
--- a/drivers/dma/qcom/gpi.c
+++ b/drivers/dma/qcom/gpi.c
@@ -2245,6 +2245,7 @@ static int gpi_probe(struct platform_device *pdev)
/* clear and Set capabilities */
dma_cap_zero(gpi_dev->dma_device.cap_mask);
dma_cap_set(DMA_SLAVE, gpi_dev->dma_device.cap_mask);
+ dma_cap_set(DMA_PRIVATE, gpi_dev->dma_device.cap_mask);
/* configure dmaengine apis */
gpi_dev->dma_device.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
diff --git a/drivers/dma/tegra186-gpc-dma.c b/drivers/dma/tegra186-gpc-dma.c
index adca05ee98c9..1d17ba4af89f 100644
--- a/drivers/dma/tegra186-gpc-dma.c
+++ b/drivers/dma/tegra186-gpc-dma.c
@@ -825,6 +825,13 @@ static unsigned int get_burst_size(struct tegra_dma_channel *tdc,
* len to calculate the optimum burst size
*/
burst_byte = burst_size ? burst_size * slave_bw : len;
+
+ /*
+ * Find the largest burst size that evenly divides the transfer length.
+ * The hardware requires the transfer length to be a multiple of the
+ * burst size - partial bursts are not supported.
+ */
+ burst_byte = min(burst_byte, 1U << __ffs(len));
burst_mmio_width = burst_byte / 4;
if (burst_mmio_width < TEGRA_GPCDMA_MMIOSEQ_BURST_MIN)
diff --git a/drivers/dpll/dpll_core.c b/drivers/dpll/dpll_core.c
index cafb8832219d..6fa6a0d8e430 100644
--- a/drivers/dpll/dpll_core.c
+++ b/drivers/dpll/dpll_core.c
@@ -10,6 +10,8 @@
#include <linux/device.h>
#include <linux/err.h>
+#include <linux/idr.h>
+#include <linux/property.h>
#include <linux/slab.h>
#include <linux/string.h>
@@ -22,6 +24,9 @@ DEFINE_MUTEX(dpll_lock);
DEFINE_XARRAY_FLAGS(dpll_device_xa, XA_FLAGS_ALLOC);
DEFINE_XARRAY_FLAGS(dpll_pin_xa, XA_FLAGS_ALLOC);
+static RAW_NOTIFIER_HEAD(dpll_notifier_chain);
+static DEFINE_IDA(dpll_pin_idx_ida);
+
static u32 dpll_device_xa_id;
static u32 dpll_pin_xa_id;
@@ -45,6 +50,78 @@ struct dpll_pin_registration {
void *cookie;
};
+static int call_dpll_notifiers(unsigned long action, void *info)
+{
+ lockdep_assert_held(&dpll_lock);
+ return raw_notifier_call_chain(&dpll_notifier_chain, action, info);
+}
+
+void dpll_device_notify(struct dpll_device *dpll, unsigned long action)
+{
+ struct dpll_device_notifier_info info = {
+ .dpll = dpll,
+ .id = dpll->id,
+ .idx = dpll->device_idx,
+ .clock_id = dpll->clock_id,
+ .type = dpll->type,
+ };
+
+ call_dpll_notifiers(action, &info);
+}
+
+void dpll_pin_notify(struct dpll_pin *pin, unsigned long action)
+{
+ struct dpll_pin_notifier_info info = {
+ .pin = pin,
+ .id = pin->id,
+ .idx = pin->pin_idx,
+ .clock_id = pin->clock_id,
+ .fwnode = pin->fwnode,
+ .prop = &pin->prop,
+ };
+
+ call_dpll_notifiers(action, &info);
+}
+
+static void __dpll_device_hold(struct dpll_device *dpll)
+{
+ refcount_inc(&dpll->refcount);
+}
+
+static void __dpll_device_put(struct dpll_device *dpll)
+{
+ if (refcount_dec_and_test(&dpll->refcount)) {
+ ASSERT_DPLL_NOT_REGISTERED(dpll);
+ WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
+ xa_destroy(&dpll->pin_refs);
+ xa_erase(&dpll_device_xa, dpll->id);
+ WARN_ON(!list_empty(&dpll->registration_list));
+ kfree(dpll);
+ }
+}
+
+static void __dpll_pin_hold(struct dpll_pin *pin)
+{
+ refcount_inc(&pin->refcount);
+}
+
+static void dpll_pin_idx_free(u32 pin_idx);
+static void dpll_pin_prop_free(struct dpll_pin_properties *prop);
+
+static void __dpll_pin_put(struct dpll_pin *pin)
+{
+ if (refcount_dec_and_test(&pin->refcount)) {
+ xa_erase(&dpll_pin_xa, pin->id);
+ xa_destroy(&pin->dpll_refs);
+ xa_destroy(&pin->parent_refs);
+ xa_destroy(&pin->ref_sync_pins);
+ dpll_pin_prop_free(&pin->prop);
+ fwnode_handle_put(pin->fwnode);
+ dpll_pin_idx_free(pin->pin_idx);
+ kfree_rcu(pin, rcu);
+ }
+}
+
struct dpll_device *dpll_device_get_by_id(int id)
{
if (xa_get_mark(&dpll_device_xa, id, DPLL_REGISTERED))
@@ -114,6 +191,7 @@ dpll_xa_ref_pin_add(struct xarray *xa_pins, struct dpll_pin *pin,
reg->ops = ops;
reg->priv = priv;
reg->cookie = cookie;
+ __dpll_pin_hold(pin);
if (ref_exists)
refcount_inc(&ref->refcount);
list_add_tail(®->list, &ref->registration_list);
@@ -136,6 +214,7 @@ static int dpll_xa_ref_pin_del(struct xarray *xa_pins, struct dpll_pin *pin,
if (WARN_ON(!reg))
return -EINVAL;
list_del(®->list);
+ __dpll_pin_put(pin);
kfree(reg);
if (refcount_dec_and_test(&ref->refcount)) {
xa_erase(xa_pins, i);
@@ -193,6 +272,7 @@ dpll_xa_ref_dpll_add(struct xarray *xa_dplls, struct dpll_device *dpll,
reg->ops = ops;
reg->priv = priv;
reg->cookie = cookie;
+ __dpll_device_hold(dpll);
if (ref_exists)
refcount_inc(&ref->refcount);
list_add_tail(®->list, &ref->registration_list);
@@ -215,6 +295,7 @@ dpll_xa_ref_dpll_del(struct xarray *xa_dplls, struct dpll_device *dpll,
if (WARN_ON(!reg))
return;
list_del(®->list);
+ __dpll_device_put(dpll);
kfree(reg);
if (refcount_dec_and_test(&ref->refcount)) {
xa_erase(xa_dplls, i);
@@ -285,8 +366,8 @@ dpll_device_get(u64 clock_id, u32 device_idx, struct module *module)
if (dpll->clock_id == clock_id &&
dpll->device_idx == device_idx &&
dpll->module == module) {
+ __dpll_device_hold(dpll);
ret = dpll;
- refcount_inc(&ret->refcount);
break;
}
}
@@ -309,14 +390,7 @@ EXPORT_SYMBOL_GPL(dpll_device_get);
void dpll_device_put(struct dpll_device *dpll)
{
mutex_lock(&dpll_lock);
- if (refcount_dec_and_test(&dpll->refcount)) {
- ASSERT_DPLL_NOT_REGISTERED(dpll);
- WARN_ON_ONCE(!xa_empty(&dpll->pin_refs));
- xa_destroy(&dpll->pin_refs);
- xa_erase(&dpll_device_xa, dpll->id);
- WARN_ON(!list_empty(&dpll->registration_list));
- kfree(dpll);
- }
+ __dpll_device_put(dpll);
mutex_unlock(&dpll_lock);
}
EXPORT_SYMBOL_GPL(dpll_device_put);
@@ -378,6 +452,7 @@ int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
reg->ops = ops;
reg->priv = priv;
dpll->type = type;
+ __dpll_device_hold(dpll);
first_registration = list_empty(&dpll->registration_list);
list_add_tail(®->list, &dpll->registration_list);
if (!first_registration) {
@@ -417,6 +492,7 @@ void dpll_device_unregister(struct dpll_device *dpll,
return;
}
list_del(®->list);
+ __dpll_device_put(dpll);
kfree(reg);
if (!list_empty(&dpll->registration_list)) {
@@ -428,6 +504,36 @@ void dpll_device_unregister(struct dpll_device *dpll,
}
EXPORT_SYMBOL_GPL(dpll_device_unregister);
+static int dpll_pin_idx_alloc(u32 *pin_idx)
+{
+ int ret;
+
+ if (!pin_idx)
+ return -EINVAL;
+
+ /* Alloc unique number from IDA. Number belongs to <0, INT_MAX> range */
+ ret = ida_alloc(&dpll_pin_idx_ida, GFP_KERNEL);
+ if (ret < 0)
+ return ret;
+
+ /* Map the value to dynamic pin index range <INT_MAX+1, U32_MAX> */
+ *pin_idx = (u32)ret + INT_MAX + 1;
+
+ return 0;
+}
+
+static void dpll_pin_idx_free(u32 pin_idx)
+{
+ if (pin_idx <= INT_MAX)
+ return; /* Not a dynamic pin index */
+
+ /* Map the index value from dynamic pin index range to IDA range and
+ * free it.
+ */
+ pin_idx -= (u32)INT_MAX + 1;
+ ida_free(&dpll_pin_idx_ida, pin_idx);
+}
+
static void dpll_pin_prop_free(struct dpll_pin_properties *prop)
{
kfree(prop->package_label);
@@ -485,9 +591,18 @@ dpll_pin_alloc(u64 clock_id, u32 pin_idx, struct module *module,
struct dpll_pin *pin;
int ret;
+ if (pin_idx == DPLL_PIN_IDX_UNSPEC) {
+ ret = dpll_pin_idx_alloc(&pin_idx);
+ if (ret)
+ return ERR_PTR(ret);
+ } else if (pin_idx > INT_MAX) {
+ return ERR_PTR(-EINVAL);
+ }
pin = kzalloc(sizeof(*pin), GFP_KERNEL);
- if (!pin)
- return ERR_PTR(-ENOMEM);
+ if (!pin) {
+ ret = -ENOMEM;
+ goto err_pin_alloc;
+ }
pin->pin_idx = pin_idx;
pin->clock_id = clock_id;
pin->module = module;
@@ -502,6 +617,7 @@ dpll_pin_alloc(u64 clock_id, u32 pin_idx, struct module *module,
refcount_set(&pin->refcount, 1);
xa_init_flags(&pin->dpll_refs, XA_FLAGS_ALLOC);
xa_init_flags(&pin->parent_refs, XA_FLAGS_ALLOC);
+ xa_init_flags(&pin->ref_sync_pins, XA_FLAGS_ALLOC);
ret = xa_alloc_cyclic(&dpll_pin_xa, &pin->id, pin, xa_limit_32b,
&dpll_pin_xa_id, GFP_KERNEL);
if (ret < 0)
@@ -510,9 +626,12 @@ dpll_pin_alloc(u64 clock_id, u32 pin_idx, struct module *module,
err_xa_alloc:
xa_destroy(&pin->dpll_refs);
xa_destroy(&pin->parent_refs);
+ xa_destroy(&pin->ref_sync_pins);
dpll_pin_prop_free(&pin->prop);
err_pin_prop:
kfree(pin);
+err_pin_alloc:
+ dpll_pin_idx_free(pin_idx);
return ERR_PTR(ret);
}
@@ -536,6 +655,28 @@ void dpll_netdev_pin_clear(struct net_device *dev)
}
EXPORT_SYMBOL(dpll_netdev_pin_clear);
+int register_dpll_notifier(struct notifier_block *nb)
+{
+ int ret;
+
+ mutex_lock(&dpll_lock);
+ ret = raw_notifier_chain_register(&dpll_notifier_chain, nb);
+ mutex_unlock(&dpll_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(register_dpll_notifier);
+
+int unregister_dpll_notifier(struct notifier_block *nb)
+{
+ int ret;
+
+ mutex_lock(&dpll_lock);
+ ret = raw_notifier_chain_unregister(&dpll_notifier_chain, nb);
+ mutex_unlock(&dpll_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(unregister_dpll_notifier);
+
/**
* dpll_pin_get - find existing or create new dpll pin
* @clock_id: clock_id of creator
@@ -563,8 +704,8 @@ dpll_pin_get(u64 clock_id, u32 pin_idx, struct module *module,
if (pos->clock_id == clock_id &&
pos->pin_idx == pin_idx &&
pos->module == module) {
+ __dpll_pin_hold(pos);
ret = pos;
- refcount_inc(&ret->refcount);
break;
}
}
@@ -587,17 +728,58 @@ EXPORT_SYMBOL_GPL(dpll_pin_get);
void dpll_pin_put(struct dpll_pin *pin)
{
mutex_lock(&dpll_lock);
- if (refcount_dec_and_test(&pin->refcount)) {
- xa_erase(&dpll_pin_xa, pin->id);
- xa_destroy(&pin->dpll_refs);
- xa_destroy(&pin->parent_refs);
- dpll_pin_prop_free(&pin->prop);
- kfree_rcu(pin, rcu);
- }
+ __dpll_pin_put(pin);
mutex_unlock(&dpll_lock);
}
EXPORT_SYMBOL_GPL(dpll_pin_put);
+/**
+ * dpll_pin_fwnode_set - set dpll pin firmware node reference
+ * @pin: pointer to a dpll pin
+ * @fwnode: firmware node handle
+ *
+ * Set firmware node handle for the given dpll pin.
+ */
+void dpll_pin_fwnode_set(struct dpll_pin *pin, struct fwnode_handle *fwnode)
+{
+ mutex_lock(&dpll_lock);
+ fwnode_handle_put(pin->fwnode); /* Drop fwnode previously set */
+ pin->fwnode = fwnode_handle_get(fwnode);
+ mutex_unlock(&dpll_lock);
+}
+EXPORT_SYMBOL_GPL(dpll_pin_fwnode_set);
+
+/**
+ * fwnode_dpll_pin_find - find dpll pin by firmware node reference
+ * @fwnode: reference to firmware node
+ *
+ * Get existing object of a pin that is associated with given firmware node
+ * reference.
+ *
+ * Context: Acquires a lock (dpll_lock)
+ * Return:
+ * * valid dpll_pin pointer on success
+ * * NULL when no such pin exists
+ */
+struct dpll_pin *fwnode_dpll_pin_find(struct fwnode_handle *fwnode)
+{
+ struct dpll_pin *pin, *ret = NULL;
+ unsigned long index;
+
+ mutex_lock(&dpll_lock);
+ xa_for_each(&dpll_pin_xa, index, pin) {
+ if (pin->fwnode == fwnode) {
+ __dpll_pin_hold(pin);
+ ret = pin;
+ break;
+ }
+ }
+ mutex_unlock(&dpll_lock);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(fwnode_dpll_pin_find);
+
static int
__dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
const struct dpll_pin_ops *ops, void *priv, void *cookie)
@@ -655,15 +837,32 @@ dpll_pin_register(struct dpll_device *dpll, struct dpll_pin *pin,
}
EXPORT_SYMBOL_GPL(dpll_pin_register);
+static void dpll_pin_ref_sync_pair_del(u32 ref_sync_pin_id)
+{
+ struct dpll_pin *pin, *ref_sync_pin;
+ unsigned long i;
+
+ xa_for_each(&dpll_pin_xa, i, pin) {
+ ref_sync_pin = xa_load(&pin->ref_sync_pins, ref_sync_pin_id);
+ if (ref_sync_pin) {
+ xa_erase(&pin->ref_sync_pins, ref_sync_pin_id);
+ __dpll_pin_change_ntf(pin);
+ }
+ }
+}
+
static void
__dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin,
const struct dpll_pin_ops *ops, void *priv, void *cookie)
{
ASSERT_DPLL_PIN_REGISTERED(pin);
+ dpll_pin_delete_ntf(pin);
dpll_xa_ref_pin_del(&dpll->pin_refs, pin, ops, priv, cookie);
dpll_xa_ref_dpll_del(&pin->dpll_refs, dpll, ops, priv, cookie);
- if (xa_empty(&pin->dpll_refs))
+ if (xa_empty(&pin->dpll_refs)) {
+ dpll_pin_ref_sync_pair_del(pin->id);
xa_clear_mark(&dpll_pin_xa, pin->id, DPLL_REGISTERED);
+ }
}
/**
@@ -685,7 +884,6 @@ void dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin,
return;
mutex_lock(&dpll_lock);
- dpll_pin_delete_ntf(pin);
__dpll_pin_unregister(dpll, pin, ops, priv, NULL);
mutex_unlock(&dpll_lock);
}
@@ -725,7 +923,6 @@ int dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
ret = dpll_xa_ref_pin_add(&pin->parent_refs, parent, ops, priv, pin);
if (ret)
goto unlock;
- refcount_inc(&pin->refcount);
xa_for_each(&parent->dpll_refs, i, ref) {
ret = __dpll_pin_register(ref->dpll, pin, ops, priv, parent);
if (ret) {
@@ -741,11 +938,10 @@ int dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
dpll_unregister:
xa_for_each(&parent->dpll_refs, i, ref)
if (i < stop) {
+ dpll_pin_delete_ntf(pin);
__dpll_pin_unregister(ref->dpll, pin, ops, priv,
parent);
- dpll_pin_delete_ntf(pin);
}
- refcount_dec(&pin->refcount);
dpll_xa_ref_pin_del(&pin->parent_refs, parent, ops, priv, pin);
unlock:
mutex_unlock(&dpll_lock);
@@ -766,19 +962,50 @@ EXPORT_SYMBOL_GPL(dpll_pin_on_pin_register);
void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin,
const struct dpll_pin_ops *ops, void *priv)
{
+ struct dpll_pin_registration *reg;
struct dpll_pin_ref *ref;
unsigned long i;
mutex_lock(&dpll_lock);
- dpll_pin_delete_ntf(pin);
- dpll_xa_ref_pin_del(&pin->parent_refs, parent, ops, priv, pin);
- refcount_dec(&pin->refcount);
- xa_for_each(&pin->dpll_refs, i, ref)
+ xa_for_each(&pin->dpll_refs, i, ref) {
+ reg = dpll_pin_registration_find(ref, ops, priv, parent);
+ if (!reg)
+ continue;
+ dpll_pin_delete_ntf(pin);
__dpll_pin_unregister(ref->dpll, pin, ops, priv, parent);
+ }
+ dpll_xa_ref_pin_del(&pin->parent_refs, parent, ops, priv, pin);
mutex_unlock(&dpll_lock);
}
EXPORT_SYMBOL_GPL(dpll_pin_on_pin_unregister);
+/**
+ * dpll_pin_ref_sync_pair_add - create a reference sync signal pin pair
+ * @pin: pin which produces the base frequency
+ * @ref_sync_pin: pin which produces the sync signal
+ *
+ * Once pins are paired, the user-space configuration of reference sync pair
+ * is possible.
+ * Context: Acquires a lock (dpll_lock)
+ * Return:
+ * * 0 on success
+ * * negative - error value
+ */
+int dpll_pin_ref_sync_pair_add(struct dpll_pin *pin,
+ struct dpll_pin *ref_sync_pin)
+{
+ int ret;
+
+ mutex_lock(&dpll_lock);
+ ret = xa_insert(&pin->ref_sync_pins, ref_sync_pin->id,
+ ref_sync_pin, GFP_KERNEL);
+ __dpll_pin_change_ntf(pin);
+ mutex_unlock(&dpll_lock);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dpll_pin_ref_sync_pair_add);
+
static struct dpll_device_registration *
dpll_device_registration_first(struct dpll_device *dpll)
{
diff --git a/drivers/dpll/dpll_core.h b/drivers/dpll/dpll_core.h
index 2b6d8ef1cdf3..c75a3ba83731 100644
--- a/drivers/dpll/dpll_core.h
+++ b/drivers/dpll/dpll_core.h
@@ -42,8 +42,10 @@ struct dpll_device {
* @pin_idx: index of a pin given by dev driver
* @clock_id: clock_id of creator
* @module: module of creator
+ * @fwnode: optional reference to firmware node
* @dpll_refs: hold referencees to dplls pin was registered with
* @parent_refs: hold references to parent pins pin was registered with
+ * @ref_sync_pins: hold references to pins for Reference SYNC feature
* @prop: pin properties copied from the registerer
* @rclk_dev_name: holds name of device when pin can recover clock from it
* @refcount: refcount
@@ -54,8 +56,10 @@ struct dpll_pin {
u32 pin_idx;
u64 clock_id;
struct module *module;
+ struct fwnode_handle *fwnode;
struct xarray dpll_refs;
struct xarray parent_refs;
+ struct xarray ref_sync_pins;
struct dpll_pin_properties prop;
refcount_t refcount;
struct rcu_head rcu;
@@ -88,4 +92,8 @@ struct dpll_pin_ref *dpll_xa_ref_dpll_first(struct xarray *xa_refs);
extern struct xarray dpll_device_xa;
extern struct xarray dpll_pin_xa;
extern struct mutex dpll_lock;
+
+void dpll_device_notify(struct dpll_device *dpll, unsigned long action);
+void dpll_pin_notify(struct dpll_pin *pin, unsigned long action);
+
#endif
diff --git a/drivers/dpll/dpll_netlink.c b/drivers/dpll/dpll_netlink.c
index fc0280dcddd1..e3fb08e1578d 100644
--- a/drivers/dpll/dpll_netlink.c
+++ b/drivers/dpll/dpll_netlink.c
@@ -48,6 +48,24 @@ dpll_msg_add_dev_parent_handle(struct sk_buff *msg, u32 id)
return 0;
}
+static bool dpll_pin_available(struct dpll_pin *pin)
+{
+ struct dpll_pin_ref *par_ref;
+ unsigned long i;
+
+ if (!xa_get_mark(&dpll_pin_xa, pin->id, DPLL_REGISTERED))
+ return false;
+ xa_for_each(&pin->parent_refs, i, par_ref)
+ if (xa_get_mark(&dpll_pin_xa, par_ref->pin->id,
+ DPLL_REGISTERED))
+ return true;
+ xa_for_each(&pin->dpll_refs, i, par_ref)
+ if (xa_get_mark(&dpll_device_xa, par_ref->dpll->id,
+ DPLL_REGISTERED))
+ return true;
+ return false;
+}
+
/**
* dpll_msg_add_pin_handle - attach pin handle attribute to a given message
* @msg: pointer to sk_buff message to attach a pin handle
@@ -387,6 +405,50 @@ dpll_msg_add_pin_esync(struct sk_buff *msg, struct dpll_pin *pin,
return -EMSGSIZE;
}
+static int
+dpll_msg_add_pin_ref_sync(struct sk_buff *msg, struct dpll_pin *pin,
+ struct dpll_pin_ref *ref,
+ struct netlink_ext_ack *extack)
+{
+ const struct dpll_pin_ops *ops = dpll_pin_ops(ref);
+ struct dpll_device *dpll = ref->dpll;
+ void *pin_priv, *ref_sync_pin_priv;
+ struct dpll_pin *ref_sync_pin;
+ enum dpll_pin_state state;
+ struct nlattr *nest;
+ unsigned long index;
+ int ret;
+
+ pin_priv = dpll_pin_on_dpll_priv(dpll, pin);
+ xa_for_each(&pin->ref_sync_pins, index, ref_sync_pin) {
+ if (!dpll_pin_available(ref_sync_pin))
+ continue;
+ ref_sync_pin_priv = dpll_pin_on_dpll_priv(dpll, ref_sync_pin);
+ /* Pin may have been unregistered from this dpll already */
+ if (!ref_sync_pin_priv)
+ continue;
+ if (WARN_ON(!ops->ref_sync_get))
+ return -EOPNOTSUPP;
+ ret = ops->ref_sync_get(pin, pin_priv, ref_sync_pin,
+ ref_sync_pin_priv, &state, extack);
+ if (ret)
+ return ret;
+ nest = nla_nest_start(msg, DPLL_A_PIN_REFERENCE_SYNC);
+ if (!nest)
+ return -EMSGSIZE;
+ if (nla_put_s32(msg, DPLL_A_PIN_ID, ref_sync_pin->id))
+ goto nest_cancel;
+ if (nla_put_s32(msg, DPLL_A_PIN_STATE, state))
+ goto nest_cancel;
+ nla_nest_end(msg, nest);
+ }
+ return 0;
+
+nest_cancel:
+ nla_nest_cancel(msg, nest);
+ return -EMSGSIZE;
+}
+
static bool dpll_pin_is_freq_supported(struct dpll_pin *pin, u32 freq)
{
int fs;
@@ -529,6 +591,10 @@ dpll_cmd_pin_get_one(struct sk_buff *msg, struct dpll_pin *pin,
if (ret)
return ret;
ret = dpll_msg_add_pin_esync(msg, pin, ref, extack);
+ if (ret)
+ return ret;
+ if (!xa_empty(&pin->ref_sync_pins))
+ ret = dpll_msg_add_pin_ref_sync(msg, pin, ref, extack);
if (ret)
return ret;
if (xa_empty(&pin->parent_refs))
@@ -604,38 +670,23 @@ dpll_device_event_send(enum dpll_cmd event, struct dpll_device *dpll)
int dpll_device_create_ntf(struct dpll_device *dpll)
{
+ dpll_device_notify(dpll, DPLL_DEVICE_CREATED);
return dpll_device_event_send(DPLL_CMD_DEVICE_CREATE_NTF, dpll);
}
int dpll_device_delete_ntf(struct dpll_device *dpll)
{
+ dpll_device_notify(dpll, DPLL_DEVICE_DELETED);
return dpll_device_event_send(DPLL_CMD_DEVICE_DELETE_NTF, dpll);
}
static int
__dpll_device_change_ntf(struct dpll_device *dpll)
{
+ dpll_device_notify(dpll, DPLL_DEVICE_CHANGED);
return dpll_device_event_send(DPLL_CMD_DEVICE_CHANGE_NTF, dpll);
}
-static bool dpll_pin_available(struct dpll_pin *pin)
-{
- struct dpll_pin_ref *par_ref;
- unsigned long i;
-
- if (!xa_get_mark(&dpll_pin_xa, pin->id, DPLL_REGISTERED))
- return false;
- xa_for_each(&pin->parent_refs, i, par_ref)
- if (xa_get_mark(&dpll_pin_xa, par_ref->pin->id,
- DPLL_REGISTERED))
- return true;
- xa_for_each(&pin->dpll_refs, i, par_ref)
- if (xa_get_mark(&dpll_device_xa, par_ref->dpll->id,
- DPLL_REGISTERED))
- return true;
- return false;
-}
-
/**
* dpll_device_change_ntf - notify that the dpll device has been changed
* @dpll: registered dpll pointer
@@ -690,16 +741,19 @@ dpll_pin_event_send(enum dpll_cmd event, struct dpll_pin *pin)
int dpll_pin_create_ntf(struct dpll_pin *pin)
{
+ dpll_pin_notify(pin, DPLL_PIN_CREATED);
return dpll_pin_event_send(DPLL_CMD_PIN_CREATE_NTF, pin);
}
int dpll_pin_delete_ntf(struct dpll_pin *pin)
{
+ dpll_pin_notify(pin, DPLL_PIN_DELETED);
return dpll_pin_event_send(DPLL_CMD_PIN_DELETE_NTF, pin);
}
-static int __dpll_pin_change_ntf(struct dpll_pin *pin)
+int __dpll_pin_change_ntf(struct dpll_pin *pin)
{
+ dpll_pin_notify(pin, DPLL_PIN_CHANGED);
return dpll_pin_event_send(DPLL_CMD_PIN_CHANGE_NTF, pin);
}
@@ -863,6 +917,108 @@ dpll_pin_esync_set(struct dpll_pin *pin, struct nlattr *a,
return ret;
}
+static int
+dpll_pin_ref_sync_state_set(struct dpll_pin *pin,
+ unsigned long ref_sync_pin_idx,
+ const enum dpll_pin_state state,
+ struct netlink_ext_ack *extack)
+
+{
+ struct dpll_pin_ref *ref, *failed;
+ const struct dpll_pin_ops *ops;
+ enum dpll_pin_state old_state;
+ struct dpll_pin *ref_sync_pin;
+ struct dpll_device *dpll;
+ unsigned long i;
+ int ret;
+
+ ref_sync_pin = xa_find(&pin->ref_sync_pins, &ref_sync_pin_idx,
+ ULONG_MAX, XA_PRESENT);
+ if (!ref_sync_pin) {
+ NL_SET_ERR_MSG(extack, "reference sync pin not found");
+ return -EINVAL;
+ }
+ if (!dpll_pin_available(ref_sync_pin)) {
+ NL_SET_ERR_MSG(extack, "reference sync pin not available");
+ return -EINVAL;
+ }
+ ref = dpll_xa_ref_dpll_first(&pin->dpll_refs);
+ ASSERT_NOT_NULL(ref);
+ ops = dpll_pin_ops(ref);
+ if (!ops->ref_sync_set || !ops->ref_sync_get) {
+ NL_SET_ERR_MSG(extack, "reference sync not supported by this pin");
+ return -EOPNOTSUPP;
+ }
+ dpll = ref->dpll;
+ ret = ops->ref_sync_get(pin, dpll_pin_on_dpll_priv(dpll, pin),
+ ref_sync_pin,
+ dpll_pin_on_dpll_priv(dpll, ref_sync_pin),
+ &old_state, extack);
+ if (ret) {
+ NL_SET_ERR_MSG(extack, "unable to get old reference sync state");
+ return ret;
+ }
+ if (state == old_state)
+ return 0;
+ xa_for_each(&pin->dpll_refs, i, ref) {
+ ops = dpll_pin_ops(ref);
+ dpll = ref->dpll;
+ ret = ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin),
+ ref_sync_pin,
+ dpll_pin_on_dpll_priv(dpll,
+ ref_sync_pin),
+ state, extack);
+ if (ret) {
+ failed = ref;
+ NL_SET_ERR_MSG_FMT(extack, "reference sync set failed for dpll_id:%u",
+ dpll->id);
+ goto rollback;
+ }
+ }
+ __dpll_pin_change_ntf(pin);
+
+ return 0;
+
+rollback:
+ xa_for_each(&pin->dpll_refs, i, ref) {
+ if (ref == failed)
+ break;
+ ops = dpll_pin_ops(ref);
+ dpll = ref->dpll;
+ if (ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin),
+ ref_sync_pin,
+ dpll_pin_on_dpll_priv(dpll, ref_sync_pin),
+ old_state, extack))
+ NL_SET_ERR_MSG(extack, "set reference sync rollback failed");
+ }
+ return ret;
+}
+
+static int
+dpll_pin_ref_sync_set(struct dpll_pin *pin, struct nlattr *nest,
+ struct netlink_ext_ack *extack)
+{
+ struct nlattr *tb[DPLL_A_PIN_MAX + 1];
+ enum dpll_pin_state state;
+ u32 sync_pin_id;
+
+ nla_parse_nested(tb, DPLL_A_PIN_MAX, nest,
+ dpll_reference_sync_nl_policy, extack);
+ if (!tb[DPLL_A_PIN_ID]) {
+ NL_SET_ERR_MSG(extack, "sync pin id expected");
+ return -EINVAL;
+ }
+ sync_pin_id = nla_get_u32(tb[DPLL_A_PIN_ID]);
+
+ if (!tb[DPLL_A_PIN_STATE]) {
+ NL_SET_ERR_MSG(extack, "sync pin state expected");
+ return -EINVAL;
+ }
+ state = nla_get_u32(tb[DPLL_A_PIN_STATE]);
+
+ return dpll_pin_ref_sync_state_set(pin, sync_pin_id, state, extack);
+}
+
static int
dpll_pin_on_pin_state_set(struct dpll_pin *pin, u32 parent_idx,
enum dpll_pin_state state,
@@ -1169,6 +1325,11 @@ dpll_pin_set_from_nlattr(struct dpll_pin *pin, struct genl_info *info)
if (ret)
return ret;
break;
+ case DPLL_A_PIN_REFERENCE_SYNC:
+ ret = dpll_pin_ref_sync_set(pin, a, info->extack);
+ if (ret)
+ return ret;
+ break;
}
}
diff --git a/drivers/dpll/dpll_netlink.h b/drivers/dpll/dpll_netlink.h
index a9cfd55f57fc..dd28b56d27c5 100644
--- a/drivers/dpll/dpll_netlink.h
+++ b/drivers/dpll/dpll_netlink.h
@@ -11,3 +11,5 @@ int dpll_device_delete_ntf(struct dpll_device *dpll);
int dpll_pin_create_ntf(struct dpll_pin *pin);
int dpll_pin_delete_ntf(struct dpll_pin *pin);
+
+int __dpll_pin_change_ntf(struct dpll_pin *pin);
diff --git a/drivers/dpll/dpll_nl.c b/drivers/dpll/dpll_nl.c
index fe9b6893d261..d709a8dc304f 100644
--- a/drivers/dpll/dpll_nl.c
+++ b/drivers/dpll/dpll_nl.c
@@ -24,6 +24,11 @@ const struct nla_policy dpll_pin_parent_pin_nl_policy[DPLL_A_PIN_STATE + 1] = {
[DPLL_A_PIN_STATE] = NLA_POLICY_RANGE(NLA_U32, 1, 3),
};
+const struct nla_policy dpll_reference_sync_nl_policy[DPLL_A_PIN_STATE + 1] = {
+ [DPLL_A_PIN_ID] = { .type = NLA_U32, },
+ [DPLL_A_PIN_STATE] = NLA_POLICY_RANGE(NLA_U32, 1, 3),
+};
+
/* DPLL_CMD_DEVICE_ID_GET - do */
static const struct nla_policy dpll_device_id_get_nl_policy[DPLL_A_TYPE + 1] = {
[DPLL_A_MODULE_NAME] = { .type = NLA_NUL_STRING, },
@@ -62,7 +67,7 @@ static const struct nla_policy dpll_pin_get_dump_nl_policy[DPLL_A_PIN_ID + 1] =
};
/* DPLL_CMD_PIN_SET - do */
-static const struct nla_policy dpll_pin_set_nl_policy[DPLL_A_PIN_ESYNC_FREQUENCY + 1] = {
+static const struct nla_policy dpll_pin_set_nl_policy[DPLL_A_PIN_REFERENCE_SYNC + 1] = {
[DPLL_A_PIN_ID] = { .type = NLA_U32, },
[DPLL_A_PIN_FREQUENCY] = { .type = NLA_U64, },
[DPLL_A_PIN_DIRECTION] = NLA_POLICY_RANGE(NLA_U32, 1, 2),
@@ -72,6 +77,7 @@ static const struct nla_policy dpll_pin_set_nl_policy[DPLL_A_PIN_ESYNC_FREQUENCY
[DPLL_A_PIN_PARENT_PIN] = NLA_POLICY_NESTED(dpll_pin_parent_pin_nl_policy),
[DPLL_A_PIN_PHASE_ADJUST] = { .type = NLA_S32, },
[DPLL_A_PIN_ESYNC_FREQUENCY] = { .type = NLA_U64, },
+ [DPLL_A_PIN_REFERENCE_SYNC] = NLA_POLICY_NESTED(dpll_reference_sync_nl_policy),
};
/* Ops table for dpll */
@@ -139,7 +145,7 @@ static const struct genl_split_ops dpll_nl_ops[] = {
.doit = dpll_nl_pin_set_doit,
.post_doit = dpll_pin_post_doit,
.policy = dpll_pin_set_nl_policy,
- .maxattr = DPLL_A_PIN_ESYNC_FREQUENCY,
+ .maxattr = DPLL_A_PIN_REFERENCE_SYNC,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
};
diff --git a/drivers/dpll/dpll_nl.h b/drivers/dpll/dpll_nl.h
index f491262bee4f..3da10cfe9a6e 100644
--- a/drivers/dpll/dpll_nl.h
+++ b/drivers/dpll/dpll_nl.h
@@ -14,6 +14,7 @@
/* Common nested types */
extern const struct nla_policy dpll_pin_parent_device_nl_policy[DPLL_A_PIN_PHASE_OFFSET + 1];
extern const struct nla_policy dpll_pin_parent_pin_nl_policy[DPLL_A_PIN_STATE + 1];
+extern const struct nla_policy dpll_reference_sync_nl_policy[DPLL_A_PIN_STATE + 1];
int dpll_lock_doit(const struct genl_split_ops *ops, struct sk_buff *skb,
struct genl_info *info);
diff --git a/drivers/edac/skx_common.c b/drivers/edac/skx_common.c
index 88f5ff249f2e..f584165994cb 100644
--- a/drivers/edac/skx_common.c
+++ b/drivers/edac/skx_common.c
@@ -417,6 +417,9 @@ int skx_get_dimm_info(u32 mtr, u32 mcmtr, u32 amap, struct dimm_info *dimm,
rows = numrow(mtr);
cols = imc->hbm_mc ? 6 : numcol(mtr);
+ if (ranks < 0 || rows < 0 || cols < 0)
+ return 0;
+
if (imc->hbm_mc) {
banks = 32;
mtype = MEM_HBM2;
diff --git a/drivers/firmware/arm_scmi/power.c b/drivers/firmware/arm_scmi/power.c
index 59aa16444c64..7d3e5f3fc379 100644
--- a/drivers/firmware/arm_scmi/power.c
+++ b/drivers/firmware/arm_scmi/power.c
@@ -205,8 +205,12 @@ scmi_power_name_get(const struct scmi_protocol_handle *ph,
u32 domain)
{
struct scmi_power_info *pi = ph->get_priv(ph);
- struct power_dom_info *dom = pi->dom_info + domain;
+ struct power_dom_info *dom;
+
+ if (domain >= pi->num_domains)
+ return "unknown";
+ dom = pi->dom_info + domain;
return dom->name;
}
diff --git a/drivers/firmware/arm_scmi/sensors.c b/drivers/firmware/arm_scmi/sensors.c
index 791efd0f82d7..1be0f89fc2c4 100644
--- a/drivers/firmware/arm_scmi/sensors.c
+++ b/drivers/firmware/arm_scmi/sensors.c
@@ -795,7 +795,7 @@ static int scmi_sensor_config_get(const struct scmi_protocol_handle *ph,
if (!ret) {
struct scmi_sensor_info *s = si->sensors + sensor_id;
- *sensor_config = get_unaligned_le64(t->rx.buf);
+ *sensor_config = get_unaligned_le32(t->rx.buf);
s->sensor_config = *sensor_config;
}
diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c
index c6bad1d2d2ae..790e1f44425f 100644
--- a/drivers/fpga/dfl.c
+++ b/drivers/fpga/dfl.c
@@ -1117,6 +1117,8 @@ static int dfh_get_param_size(void __iomem *dfh_base, resource_size_t max)
return -EINVAL;
size += next * sizeof(u64);
+ if (size > max)
+ return -EINVAL;
if (FIELD_GET(DFHv1_PARAM_HDR_NEXT_EOP, v))
return size;
diff --git a/drivers/fpga/microchip-spi.c b/drivers/fpga/microchip-spi.c
index 6134cea86ac8..cc8f6d7bb978 100644
--- a/drivers/fpga/microchip-spi.c
+++ b/drivers/fpga/microchip-spi.c
@@ -116,6 +116,9 @@ static int mpf_ops_parse_header(struct fpga_manager *mgr,
}
header_size = *(buf + MPF_HEADER_SIZE_OFFSET);
+ if (!header_size)
+ return -EINVAL;
+
if (header_size > count) {
info->header_size = header_size;
return -EAGAIN;
diff --git a/drivers/gpio/gpio-davinci.c b/drivers/gpio/gpio-davinci.c
index 76b58c70b257..6d12a65b0752 100644
--- a/drivers/gpio/gpio-davinci.c
+++ b/drivers/gpio/gpio-davinci.c
@@ -552,8 +552,10 @@ static int davinci_gpio_irq_setup(struct platform_device *pdev)
sizeof(struct
davinci_gpio_irq_data),
GFP_KERNEL);
- if (!irqdata)
+ if (!irqdata) {
+ irq_domain_remove(chips->irq_domain);
return -ENOMEM;
+ }
irqdata->regs = g;
irqdata->bank_num = bank;
diff --git a/drivers/gpio/gpio-f7188x.c b/drivers/gpio/gpio-f7188x.c
index 3875fd940ccb..49323350e7d3 100644
--- a/drivers/gpio/gpio-f7188x.c
+++ b/drivers/gpio/gpio-f7188x.c
@@ -48,7 +48,8 @@
/*
* Nuvoton devices.
*/
-#define SIO_NCT6126D_ID 0xD283 /* NCT6126D chipset ID */
+#define SIO_NCT6126D_VER_A_ID 0xD283 /* NCT6126D version A chipset ID */
+#define SIO_NCT6126D_VER_B_ID 0xD284 /* NCT6126D version B chipset ID */
#define SIO_LD_GPIO_NUVOTON 0x07 /* GPIO logical device */
@@ -559,7 +560,8 @@ static int __init f7188x_find(int addr, struct f7188x_sio *sio)
case SIO_F81865_ID:
sio->type = f81865;
break;
- case SIO_NCT6126D_ID:
+ case SIO_NCT6126D_VER_A_ID:
+ case SIO_NCT6126D_VER_B_ID:
sio->device = SIO_LD_GPIO_NUVOTON;
sio->type = nct6126d;
break;
diff --git a/drivers/gpio/gpio-htc-egpio.c b/drivers/gpio/gpio-htc-egpio.c
index a40bd56673fe..5f12ce9ac28c 100644
--- a/drivers/gpio/gpio-htc-egpio.c
+++ b/drivers/gpio/gpio-htc-egpio.c
@@ -268,6 +268,7 @@ static int __init egpio_probe(struct platform_device *pdev)
struct gpio_chip *chip;
unsigned int irq, irq_end;
int i;
+ int ret;
/* Initialize ei data structure. */
ei = devm_kzalloc(&pdev->dev, sizeof(*ei), GFP_KERNEL);
@@ -331,7 +332,10 @@ static int __init egpio_probe(struct platform_device *pdev)
chip->base = pdata->chip[i].gpio_base;
chip->ngpio = pdata->chip[i].num_gpios;
- gpiochip_add_data(chip, &ei->chip[i]);
+ ret = devm_gpiochip_add_data(&pdev->dev, chip, &ei->chip[i]);
+ if (ret)
+ return dev_err_probe(&pdev->dev, ret,
+ "failed to register gpiochip %d\n", i);
}
/* Set initial pin values */
diff --git a/drivers/gpio/gpio-mlxbf3.c b/drivers/gpio/gpio-mlxbf3.c
index ed29b07d16c1..a662989ff21e 100644
--- a/drivers/gpio/gpio-mlxbf3.c
+++ b/drivers/gpio/gpio-mlxbf3.c
@@ -252,7 +252,8 @@ static int mlxbf3_gpio_probe(struct platform_device *pdev)
ret = devm_gpiochip_add_data(dev, &gs->gc, gs);
if (ret)
- dev_err_probe(dev, ret, "Failed adding memory mapped gpiochip\n");
+ return dev_err_probe(dev, ret,
+ "Failed adding memory mapped gpiochip\n");
return 0;
}
diff --git a/drivers/gpio/gpio-mvebu.c b/drivers/gpio/gpio-mvebu.c
index c85ab356bc72..e554bd0fb1a8 100644
--- a/drivers/gpio/gpio-mvebu.c
+++ b/drivers/gpio/gpio-mvebu.c
@@ -1116,6 +1116,7 @@ static void mvebu_gpio_remove_irq_domain(void *data)
{
struct irq_domain *domain = data;
+ irq_domain_remove_generic_chips(domain);
irq_domain_remove(domain);
}
@@ -1227,7 +1228,10 @@ static int mvebu_gpio_probe(struct platform_device *pdev)
BUG();
}
- devm_gpiochip_add_data(&pdev->dev, &mvchip->chip, mvchip);
+ err = devm_gpiochip_add_data(&pdev->dev, &mvchip->chip, mvchip);
+ if (err)
+ return dev_err_probe(&pdev->dev, err,
+ "failed to register gpiochip\n");
/* Some MVEBU SoCs have simple PWM support for GPIO lines */
if (IS_REACHABLE(CONFIG_PWM)) {
diff --git a/drivers/gpio/gpio-palmas.c b/drivers/gpio/gpio-palmas.c
index 28dba7048509..41a62353822d 100644
--- a/drivers/gpio/gpio-palmas.c
+++ b/drivers/gpio/gpio-palmas.c
@@ -117,6 +117,24 @@ static int palmas_gpio_input(struct gpio_chip *gc, unsigned offset)
return ret;
}
+static int palmas_gpio_get_direction(struct gpio_chip *gc, unsigned int offset)
+{
+ struct palmas_gpio *pg = gpiochip_get_data(gc);
+ struct palmas *palmas = pg->palmas;
+ unsigned int val;
+ unsigned int reg;
+ int ret;
+ int gpio16 = (offset/8);
+
+ offset %= 8;
+ reg = (gpio16) ? PALMAS_GPIO_DATA_DIR2 : PALMAS_GPIO_DATA_DIR;
+ ret = palmas_read(palmas, PALMAS_GPIO_BASE, reg, &val);
+ if (ret)
+ return ret;
+
+ return (val & BIT(offset)) ? GPIO_LINE_DIRECTION_OUT : GPIO_LINE_DIRECTION_IN;
+}
+
static int palmas_gpio_to_irq(struct gpio_chip *gc, unsigned offset)
{
struct palmas_gpio *pg = gpiochip_get_data(gc);
@@ -165,6 +183,7 @@ static int palmas_gpio_probe(struct platform_device *pdev)
palmas_gpio->gpio_chip.can_sleep = true;
palmas_gpio->gpio_chip.direction_input = palmas_gpio_input;
palmas_gpio->gpio_chip.direction_output = palmas_gpio_output;
+ palmas_gpio->gpio_chip.get_direction = palmas_gpio_get_direction;
palmas_gpio->gpio_chip.to_irq = palmas_gpio_to_irq;
palmas_gpio->gpio_chip.set = palmas_gpio_set;
palmas_gpio->gpio_chip.get = palmas_gpio_get;
diff --git a/drivers/gpio/gpio-sch.c b/drivers/gpio/gpio-sch.c
index ff0341b1222f..e45390e5857a 100644
--- a/drivers/gpio/gpio-sch.c
+++ b/drivers/gpio/gpio-sch.c
@@ -39,7 +39,7 @@
struct sch_gpio {
struct gpio_chip chip;
void __iomem *regs;
- spinlock_t lock;
+ raw_spinlock_t lock;
unsigned short resume_base;
/* GPE handling */
@@ -104,9 +104,9 @@ static int sch_gpio_direction_in(struct gpio_chip *gc, unsigned int gpio_num)
struct sch_gpio *sch = gpiochip_get_data(gc);
unsigned long flags;
- spin_lock_irqsave(&sch->lock, flags);
+ raw_spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GIO, 1);
- spin_unlock_irqrestore(&sch->lock, flags);
+ raw_spin_unlock_irqrestore(&sch->lock, flags);
return 0;
}
@@ -122,9 +122,9 @@ static void sch_gpio_set(struct gpio_chip *gc, unsigned int gpio_num, int val)
struct sch_gpio *sch = gpiochip_get_data(gc);
unsigned long flags;
- spin_lock_irqsave(&sch->lock, flags);
+ raw_spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GLV, val);
- spin_unlock_irqrestore(&sch->lock, flags);
+ raw_spin_unlock_irqrestore(&sch->lock, flags);
}
static int sch_gpio_direction_out(struct gpio_chip *gc, unsigned int gpio_num,
@@ -133,9 +133,9 @@ static int sch_gpio_direction_out(struct gpio_chip *gc, unsigned int gpio_num,
struct sch_gpio *sch = gpiochip_get_data(gc);
unsigned long flags;
- spin_lock_irqsave(&sch->lock, flags);
+ raw_spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GIO, 0);
- spin_unlock_irqrestore(&sch->lock, flags);
+ raw_spin_unlock_irqrestore(&sch->lock, flags);
/*
* according to the datasheet, writing to the level register has no
@@ -195,14 +195,14 @@ static int sch_irq_type(struct irq_data *d, unsigned int type)
return -EINVAL;
}
- spin_lock_irqsave(&sch->lock, flags);
+ raw_spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GTPE, rising);
sch_gpio_reg_set(sch, gpio_num, GTNE, falling);
irq_set_handler_locked(d, handle_edge_irq);
- spin_unlock_irqrestore(&sch->lock, flags);
+ raw_spin_unlock_irqrestore(&sch->lock, flags);
return 0;
}
@@ -214,9 +214,9 @@ static void sch_irq_ack(struct irq_data *d)
irq_hw_number_t gpio_num = irqd_to_hwirq(d);
unsigned long flags;
- spin_lock_irqsave(&sch->lock, flags);
+ raw_spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GTS, 1);
- spin_unlock_irqrestore(&sch->lock, flags);
+ raw_spin_unlock_irqrestore(&sch->lock, flags);
}
static void sch_irq_mask_unmask(struct gpio_chip *gc, irq_hw_number_t gpio_num, int val)
@@ -224,9 +224,9 @@ static void sch_irq_mask_unmask(struct gpio_chip *gc, irq_hw_number_t gpio_num,
struct sch_gpio *sch = gpiochip_get_data(gc);
unsigned long flags;
- spin_lock_irqsave(&sch->lock, flags);
+ raw_spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GGPE, val);
- spin_unlock_irqrestore(&sch->lock, flags);
+ raw_spin_unlock_irqrestore(&sch->lock, flags);
}
static void sch_irq_mask(struct irq_data *d)
@@ -267,12 +267,12 @@ static u32 sch_gpio_gpe_handler(acpi_handle gpe_device, u32 gpe, void *context)
int offset;
u32 ret;
- spin_lock_irqsave(&sch->lock, flags);
+ raw_spin_lock_irqsave(&sch->lock, flags);
core_status = ioread32(sch->regs + CORE_BANK_OFFSET + GTS);
resume_status = ioread32(sch->regs + RESUME_BANK_OFFSET + GTS);
- spin_unlock_irqrestore(&sch->lock, flags);
+ raw_spin_unlock_irqrestore(&sch->lock, flags);
pending = (resume_status << sch->resume_base) | core_status;
for_each_set_bit(offset, &pending, sch->chip.ngpio)
@@ -342,7 +342,7 @@ static int sch_gpio_probe(struct platform_device *pdev)
sch->regs = regs;
- spin_lock_init(&sch->lock);
+ raw_spin_lock_init(&sch->lock);
sch->chip = sch_gpio_chip;
sch->chip.label = dev_name(dev);
sch->chip.parent = dev;
diff --git a/drivers/gpio/gpio-tegra.c b/drivers/gpio/gpio-tegra.c
index cc7435ecada8..8a453b0798d0 100644
--- a/drivers/gpio/gpio-tegra.c
+++ b/drivers/gpio/gpio-tegra.c
@@ -170,18 +170,11 @@ static int tegra_gpio_direction_input(struct gpio_chip *chip,
unsigned int offset)
{
struct tegra_gpio_info *tgi = gpiochip_get_data(chip);
- int ret;
tegra_gpio_mask_write(tgi, GPIO_MSK_OE(tgi, offset), offset, 0);
tegra_gpio_enable(tgi, offset);
- ret = pinctrl_gpio_direction_input(chip, offset);
- if (ret < 0)
- dev_err(tgi->dev,
- "Failed to set pinctrl input direction of GPIO %d: %d",
- chip->base + offset, ret);
-
- return ret;
+ return 0;
}
static int tegra_gpio_direction_output(struct gpio_chip *chip,
@@ -189,19 +182,12 @@ static int tegra_gpio_direction_output(struct gpio_chip *chip,
int value)
{
struct tegra_gpio_info *tgi = gpiochip_get_data(chip);
- int ret;
tegra_gpio_set(chip, offset, value);
tegra_gpio_mask_write(tgi, GPIO_MSK_OE(tgi, offset), offset, 1);
tegra_gpio_enable(tgi, offset);
- ret = pinctrl_gpio_direction_output(chip, offset);
- if (ret < 0)
- dev_err(tgi->dev,
- "Failed to set pinctrl output direction of GPIO %d: %d",
- chip->base + offset, ret);
-
- return ret;
+ return 0;
}
static int tegra_gpio_get_direction(struct gpio_chip *chip,
diff --git a/drivers/gpio/gpio-timberdale.c b/drivers/gpio/gpio-timberdale.c
index fad979797486..c40fb5ec93e6 100644
--- a/drivers/gpio/gpio-timberdale.c
+++ b/drivers/gpio/gpio-timberdale.c
@@ -228,7 +228,7 @@ static int timbgpio_probe(struct platform_device *pdev)
tgpio = devm_kzalloc(dev, sizeof(*tgpio), GFP_KERNEL);
if (!tgpio)
- return -EINVAL;
+ return -ENOMEM;
tgpio->irq_base = pdata->irq_base;
diff --git a/drivers/gpio/gpiolib-acpi-core.c b/drivers/gpio/gpiolib-acpi-core.c
index f6bec9d2fd26..895b891f825d 100644
--- a/drivers/gpio/gpiolib-acpi-core.c
+++ b/drivers/gpio/gpiolib-acpi-core.c
@@ -233,12 +233,23 @@ static void acpi_gpiochip_request_irq(struct acpi_gpio_chip *acpi_gpio,
event->irq_requested = true;
- /* Make sure we trigger the initial state of edge-triggered IRQs */
+ /*
+ * Make sure we trigger the initial state of ActiveBoth IRQs.
+ *
+ * According to the Microsoft GPIO documentation, triggering GPIO
+ * interrupts marked as ActiveBoth during initialization is correct
+ * as long as the associated GPIO line is already "asserted"
+ * (logic level low). We should not trigger edge-based GPIO
+ * interrupts not marked as ActiveBoth.
+ *
+ * See: https://learn.microsoft.com/en-us/windows-hardware/drivers/bringup/general-purpose-i-o--gpio-
+ * Section: "GPIO controllers and ActiveBoth interrupts"
+ */
if (acpi_gpio_need_run_edge_events_on_boot() &&
- (event->irqflags & (IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING))) {
+ ((event->irqflags & (IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING)) ==
+ (IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING))) {
value = gpiod_get_raw_value_cansleep(event->desc);
- if (((event->irqflags & IRQF_TRIGGER_RISING) && value == 1) ||
- ((event->irqflags & IRQF_TRIGGER_FALLING) && value == 0))
+ if (value == 0)
event->handler(event->irq, event);
}
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index 3ae884b81aec..d81af7dcef44 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1916,13 +1916,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
mutex_lock(&mem->lock);
- /* Unpin MMIO/DOORBELL BO's that were pinned during allocation */
- if (mem->alloc_flags &
- (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL |
- KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) {
- amdgpu_amdkfd_gpuvm_unpin_bo(mem->bo);
- }
-
mapped_to_gpu_memory = mem->mapped_to_gpu_memory;
is_imported = mem->is_imported;
mutex_unlock(&mem->lock);
@@ -1936,6 +1929,15 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu(
return -EBUSY;
}
+ /* At this point the BO is guaranteed to be freed, so unpin the
+ * MMIO/DOORBELL BOs that were pinned during allocation.
+ */
+ if (mem->alloc_flags &
+ (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL |
+ KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) {
+ amdgpu_amdkfd_gpuvm_unpin_bo(mem->bo);
+ }
+
/* Make sure restore workers don't access the BO any more */
mutex_lock(&process_info->lock);
list_del(&mem->validate_list);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 13e8bc2426f8..d1daf5a79bbd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -1611,7 +1611,9 @@ int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev)
pci_release_resource(adev->pdev, 0);
- r = pci_resize_resource(adev->pdev, 0, rbar_size);
+ r = pci_resize_resource(adev->pdev, 0, rbar_size,
+ (adev->asic_type >= CHIP_BONAIRE) ? 1 << 5
+ : 1 << 2);
if (r == -ENOSPC)
DRM_INFO("Not enough PCI address space for a large BAR.");
else if (r && r != -ENOTSUPP)
@@ -4201,6 +4203,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
mutex_init(&adev->enforce_isolation_mutex);
mutex_init(&adev->gfx.kfd_sch_mutex);
+ spin_lock_init(&adev->irq.lock);
+
amdgpu_device_init_apu_flags(adev);
r = amdgpu_device_check_arguments(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 1a5df8b94661..51875f00084c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -27,6 +27,7 @@
*/
#include <linux/ktime.h>
#include <linux/module.h>
+#include <linux/overflow.h>
#include <linux/pagemap.h>
#include <linux/pci.h>
#include <linux/dma-buf.h>
@@ -913,13 +914,14 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
return r;
}
-static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
- int width,
- int cpp,
- bool tiled)
+static unsigned int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
+ unsigned int width,
+ unsigned int cpp,
+ bool tiled)
{
- int aligned = width;
- int pitch_mask = 0;
+ unsigned int aligned = width;
+ unsigned int pitch_mask = 0;
+ unsigned int pitch;
switch (cpp) {
case 1:
@@ -934,9 +936,12 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
break;
}
- aligned += pitch_mask;
+ if (check_add_overflow(aligned, pitch_mask, &aligned))
+ return 0;
aligned &= ~pitch_mask;
- return aligned * cpp;
+ if (check_mul_overflow(aligned, cpp, &pitch))
+ return 0;
+ return pitch;
}
int amdgpu_mode_dumb_create(struct drm_file *file_priv,
@@ -963,8 +968,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
args->pitch = amdgpu_gem_align_pitch(adev, args->width,
DIV_ROUND_UP(args->bpp, 8), 0);
+ if (!args->pitch)
+ return -EINVAL;
args->size = (u64)args->pitch * args->height;
args->size = ALIGN(args->size, PAGE_SIZE);
+ if (!args->size)
+ return -EINVAL;
domain = amdgpu_bo_get_preferred_domain(adev,
amdgpu_display_supported_domains(adev, flags));
r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index 19ce4da285e8..b718cc4fbfd4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -273,8 +273,6 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
unsigned int irq, flags;
int r;
- spin_lock_init(&adev->irq.lock);
-
/* Enable MSI if not disabled by module parameter */
adev->irq.msi_enabled = false;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mca.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mca.c
index 18ee60378727..731f44e1daf9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mca.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mca.c
@@ -99,6 +99,7 @@ int amdgpu_mca_mp0_ras_sw_init(struct amdgpu_device *adev)
strcpy(ras->ras_block.ras_comm.name, "mca.mp0");
ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__MCA;
+ ras->ras_block.ras_comm.sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP0;
ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
adev->mca.mp0.ras_if = &ras->ras_block.ras_comm;
@@ -123,6 +124,7 @@ int amdgpu_mca_mp1_ras_sw_init(struct amdgpu_device *adev)
strcpy(ras->ras_block.ras_comm.name, "mca.mp1");
ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__MCA;
+ ras->ras_block.ras_comm.sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP1;
ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
adev->mca.mp1.ras_if = &ras->ras_block.ras_comm;
@@ -147,6 +149,7 @@ int amdgpu_mca_mpio_ras_sw_init(struct amdgpu_device *adev)
strcpy(ras->ras_block.ras_comm.name, "mca.mpio");
ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__MCA;
+ ras->ras_block.ras_comm.sub_block_index = AMDGPU_RAS_MCA_BLOCK__MPIO;
ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE;
adev->mca.mpio.ras_if = &ras->ras_block.ras_comm;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 6bca87b99f7c..e75ca4dfc6a9 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -2322,6 +2322,9 @@ static int criu_restore_memory_of_gpu(struct kfd_process_device *pdd,
const bool criu_resume = true;
u64 offset;
+ if (bo_priv->idr_handle > INT_MAX)
+ return -EINVAL;
+
if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL) {
if (bo_bucket->size !=
kfd_doorbell_process_slice(pdd->dev->kfd))
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index e19f088c7dcf..9035eebe9cd6 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -2925,32 +2925,24 @@ int kfd_dqm_suspend_bad_queue_mes(struct kfd_node *knode, u32 pasid, u32 doorbel
list_for_each_entry(q, &qpd->queues_list, list) {
if (q->doorbell_id == doorbell_id && q->properties.is_active) {
- ret = suspend_all_queues_mes(dqm);
- if (ret) {
- dev_err(dev, "Suspending all queues failed");
- goto out;
- }
+ /* suspend all queues will save any good queues and mark the rest as bad */
+ suspend_all_queues_mes(dqm);
q->properties.is_evicted = true;
q->properties.is_active = false;
decrement_queue_count(dqm, qpd, q);
+ /* this will remove the bad queue and sched a GPU reset if needed */
ret = remove_queue_mes(dqm, q, qpd);
- if (ret) {
- dev_err(dev, "Removing bad queue failed");
- goto out;
- }
-
- ret = resume_all_queues_mes(dqm);
if (ret)
- dev_err(dev, "Resuming all queues failed");
-
+ dev_err(dev, "Removing bad queue failed");
+ /* resume the good queues */
+ resume_all_queues_mes(dqm);
break;
}
}
}
-out:
dqm_unlock(dqm);
return ret;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_events.c
index 4e5ae5656283..0b5f1e574afe 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_events.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_events.c
@@ -480,6 +480,11 @@ int kfd_criu_restore_event(struct file *devkfd,
}
*priv_data_offset += sizeof(*ev_priv);
+ if (ev_priv->event_id > INT_MAX) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
if (ev_priv->user_handle) {
ret = kfd_kmap_event_page(p, ev_priv->user_handle);
if (ret)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index 31b6903e6b74..08fcc94c0a2c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -4051,6 +4051,7 @@ int kfd_criu_resume_svm(struct kfd_process *p)
list_for_each_entry_safe(criu_svm_md, next, &svms->criu_svm_metadata_list, list) {
pr_debug("freeing criu_svm_md[]\n\tstart: 0x%llx\n",
criu_svm_md->data.start_addr);
+ list_del(&criu_svm_md->list);
kfree(criu_svm_md);
}
diff --git a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c
index b3d55cac3569..ae858a43c35f 100644
--- a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c
+++ b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c
@@ -447,6 +447,8 @@ void mod_build_vsc_infopacket(const struct dc_stream_state *stream,
*
* @stream: contains data we may need to construct VSIF (i.e. timing_3d_format, etc.)
* @info_packet: output structure where to store VSIF
+ * @ALLMEnabled: indicates whether ALLM HF-VSIF should be generated
+ * @ALLMValue: ALLM bit value to advertise in HF-VSIF
*/
void mod_build_hf_vsif_infopacket(const struct dc_stream_state *stream,
struct dc_info_packet *info_packet)
diff --git a/drivers/gpu/drm/amd/pm/amdgpu_pm.c b/drivers/gpu/drm/amd/pm/amdgpu_pm.c
index 57b5d90ca89b..9ab0a7fc3cd9 100644
--- a/drivers/gpu/drm/amd/pm/amdgpu_pm.c
+++ b/drivers/gpu/drm/amd/pm/amdgpu_pm.c
@@ -2363,12 +2363,12 @@ static ssize_t amdgpu_set_pm_policy_attr(struct device *dev,
.dev_attr = __ATTR(_name, 0644, amdgpu_get_pm_policy_attr, \
amdgpu_set_pm_policy_attr), \
.id = PP_PM_POLICY_##_id, \
- };
+ }
#define AMDGPU_PM_POLICY_ATTR_VAR(_name) pm_policy_attr_##_name.dev_attr.attr
-AMDGPU_PM_POLICY_ATTR(soc_pstate, SOC_PSTATE)
-AMDGPU_PM_POLICY_ATTR(xgmi_plpd, XGMI_PLPD)
+AMDGPU_PM_POLICY_ATTR(soc_pstate, SOC_PSTATE);
+AMDGPU_PM_POLICY_ATTR(xgmi_plpd, XGMI_PLPD);
static struct attribute *pm_policy_attrs[] = {
&AMDGPU_PM_POLICY_ATTR_VAR(soc_pstate),
diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c
index 9edb3247c767..b6f597af0d0a 100644
--- a/drivers/gpu/drm/drm_edid.c
+++ b/drivers/gpu/drm/drm_edid.c
@@ -7421,6 +7421,14 @@ static void drm_parse_tiled_block(struct drm_connector *connector,
u8 num_v_tile, num_h_tile;
struct drm_tile_group *tg;
+ /* tiled block payload per spec: cap 1 + topo 3 + size 4 + bezel 5 + id 9 = 22 */
+ if (block->num_bytes < 22) {
+ drm_dbg_kms(connector->dev,
+ "[CONNECTOR:%d:%s] Unexpected tiled block size %u\n",
+ connector->base.id, connector->name, block->num_bytes);
+ return;
+ }
+
w = tile->tile_size[0] | tile->tile_size[1] << 8;
h = tile->tile_size[2] | tile->tile_size[3] << 8;
diff --git a/drivers/gpu/drm/drm_gpuvm.c b/drivers/gpu/drm/drm_gpuvm.c
index f9eb56f24bef..2515cff33485 100644
--- a/drivers/gpu/drm/drm_gpuvm.c
+++ b/drivers/gpu/drm/drm_gpuvm.c
@@ -25,6 +25,7 @@
*
*/
+#include <drm/drm_drv.h>
#include <drm/drm_gpuvm.h>
#include <linux/interval_tree_generic.h>
@@ -1019,6 +1020,7 @@ drm_gpuvm_init(struct drm_gpuvm *gpuvm, const char *name,
gpuvm->drm = drm;
gpuvm->r_obj = r_obj;
+ drm_dev_get(drm);
drm_gem_object_get(r_obj);
drm_gpuvm_warn_check_overflow(gpuvm, start_offset, range);
@@ -1060,13 +1062,15 @@ static void
drm_gpuvm_free(struct kref *kref)
{
struct drm_gpuvm *gpuvm = container_of(kref, struct drm_gpuvm, kref);
+ struct drm_device *drm = gpuvm->drm;
drm_gpuvm_fini(gpuvm);
- if (drm_WARN_ON(gpuvm->drm, !gpuvm->ops->vm_free))
+ if (drm_WARN_ON(drm, !gpuvm->ops->vm_free))
return;
gpuvm->ops->vm_free(gpuvm);
+ drm_dev_put(drm);
}
/**
@@ -1219,6 +1223,9 @@ drm_gpuvm_prepare_range(struct drm_gpuvm *gpuvm, struct drm_exec *exec,
drm_gpuvm_for_each_va_range(va, gpuvm, addr, end) {
struct drm_gem_object *obj = va->gem.obj;
+ if (unlikely(!obj))
+ continue;
+
ret = exec_prepare_obj(exec, obj, num_fences);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c
index 4f2ab8a7b50f..effa2e31263b 100644
--- a/drivers/gpu/drm/drm_syncobj.c
+++ b/drivers/gpu/drm/drm_syncobj.c
@@ -441,13 +441,15 @@ int drm_syncobj_find_fence(struct drm_file *file_private,
u64 timeout = nsecs_to_jiffies64(DRM_SYNCOBJ_WAIT_FOR_SUBMIT_TIMEOUT);
int ret;
- if (flags & ~DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT)
- return -EINVAL;
-
if (!syncobj)
return -ENOENT;
- /* Waiting for userspace with locks help is illegal cause that can
+ if (flags & ~DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* Waiting for userspace with locks held is illegal cause that can
* trivial deadlock with page faults for example. Make lockdep complain
* about it early on.
*/
diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c
index 89bed78f1466..db7fce4e8cc3 100644
--- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c
+++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c
@@ -32,26 +32,43 @@ struct hibmc_display_panel_pll {
struct hibmc_dislay_pll_config {
u64 hdisplay;
u64 vdisplay;
+ int clock;
u32 pll1_config_value;
u32 pll2_config_value;
};
static const struct hibmc_dislay_pll_config hibmc_pll_table[] = {
- {640, 480, CRT_PLL1_HS_25MHZ, CRT_PLL2_HS_25MHZ},
- {800, 600, CRT_PLL1_HS_40MHZ, CRT_PLL2_HS_40MHZ},
- {1024, 768, CRT_PLL1_HS_65MHZ, CRT_PLL2_HS_65MHZ},
- {1152, 864, CRT_PLL1_HS_80MHZ_1152, CRT_PLL2_HS_80MHZ},
- {1280, 768, CRT_PLL1_HS_80MHZ, CRT_PLL2_HS_80MHZ},
- {1280, 720, CRT_PLL1_HS_74MHZ, CRT_PLL2_HS_74MHZ},
- {1280, 960, CRT_PLL1_HS_108MHZ, CRT_PLL2_HS_108MHZ},
- {1280, 1024, CRT_PLL1_HS_108MHZ, CRT_PLL2_HS_108MHZ},
- {1440, 900, CRT_PLL1_HS_106MHZ, CRT_PLL2_HS_106MHZ},
- {1600, 900, CRT_PLL1_HS_108MHZ, CRT_PLL2_HS_108MHZ},
- {1600, 1200, CRT_PLL1_HS_162MHZ, CRT_PLL2_HS_162MHZ},
- {1920, 1080, CRT_PLL1_HS_148MHZ, CRT_PLL2_HS_148MHZ},
- {1920, 1200, CRT_PLL1_HS_193MHZ, CRT_PLL2_HS_193MHZ},
+ {640, 480, 25000, CRT_PLL1_HS_25MHZ, CRT_PLL2_HS_25MHZ},
+ {800, 600, 40000, CRT_PLL1_HS_40MHZ, CRT_PLL2_HS_40MHZ},
+ {1024, 768, 65000, CRT_PLL1_HS_65MHZ, CRT_PLL2_HS_65MHZ},
+ {1152, 864, 78750, CRT_PLL1_HS_80MHZ_1152, CRT_PLL2_HS_80MHZ},
+ {1280, 768, 80000, CRT_PLL1_HS_80MHZ, CRT_PLL2_HS_80MHZ},
+ {1280, 720, 74375, CRT_PLL1_HS_74MHZ, CRT_PLL2_HS_74MHZ},
+ {1280, 960, 108000, CRT_PLL1_HS_108MHZ, CRT_PLL2_HS_108MHZ},
+ {1280, 1024, 108000, CRT_PLL1_HS_108MHZ, CRT_PLL2_HS_108MHZ},
+ {1440, 900, 105952, CRT_PLL1_HS_106MHZ, CRT_PLL2_HS_106MHZ},
+ {1600, 900, 108000, CRT_PLL1_HS_108MHZ, CRT_PLL2_HS_108MHZ},
+ {1600, 1200, 162500, CRT_PLL1_HS_162MHZ, CRT_PLL2_HS_162MHZ},
+ {1920, 1080, 148750, CRT_PLL1_HS_148MHZ, CRT_PLL2_HS_148MHZ},
+ {1920, 1200, 193750, CRT_PLL1_HS_193MHZ, CRT_PLL2_HS_193MHZ},
};
+static int hibmc_get_best_clock_idx(const struct drm_display_mode *mode)
+{
+ int i, diff;
+
+ for (i = 0; i < ARRAY_SIZE(hibmc_pll_table); i++) {
+ if (hibmc_pll_table[i].hdisplay == mode->hdisplay &&
+ hibmc_pll_table[i].vdisplay == mode->vdisplay) {
+ diff = abs(mode->clock - hibmc_pll_table[i].clock);
+ if (diff < mode->clock / 100) /* tolerance 1/100 */
+ return i;
+ }
+ }
+
+ return -MODE_CLOCK_RANGE;
+}
+
static int hibmc_plane_atomic_check(struct drm_plane *plane,
struct drm_atomic_state *state)
{
@@ -214,19 +231,15 @@ static enum drm_mode_status
hibmc_crtc_mode_valid(struct drm_crtc *crtc,
const struct drm_display_mode *mode)
{
- size_t i = 0;
int vrefresh = drm_mode_vrefresh(mode);
if (vrefresh < 59 || vrefresh > 61)
return MODE_NOCLOCK;
- for (i = 0; i < ARRAY_SIZE(hibmc_pll_table); i++) {
- if (hibmc_pll_table[i].hdisplay == mode->hdisplay &&
- hibmc_pll_table[i].vdisplay == mode->vdisplay)
- return MODE_OK;
- }
+ if (hibmc_get_best_clock_idx(mode) >= 0)
+ return MODE_OK;
- return MODE_BAD;
+ return MODE_CLOCK_RANGE;
}
static u32 format_pll_reg(void)
@@ -281,23 +294,20 @@ static void set_vclock_hisilicon(struct drm_device *dev, u64 pll)
writel(val, priv->mmio + CRT_PLL1_HS);
}
-static void get_pll_config(u64 x, u64 y, u32 *pll1, u32 *pll2)
+static void get_pll_config(struct drm_display_mode *mode, u32 *pll1, u32 *pll2)
{
- size_t i;
- size_t count = ARRAY_SIZE(hibmc_pll_table);
-
- for (i = 0; i < count; i++) {
- if (hibmc_pll_table[i].hdisplay == x &&
- hibmc_pll_table[i].vdisplay == y) {
- *pll1 = hibmc_pll_table[i].pll1_config_value;
- *pll2 = hibmc_pll_table[i].pll2_config_value;
- return;
- }
+ int idx;
+
+ idx = hibmc_get_best_clock_idx(mode);
+ if (idx < 0) {
+ /* if found none, we use default value */
+ *pll1 = CRT_PLL1_HS_25MHZ;
+ *pll2 = CRT_PLL2_HS_25MHZ;
+ return;
}
- /* if found none, we use default value */
- *pll1 = CRT_PLL1_HS_25MHZ;
- *pll2 = CRT_PLL2_HS_25MHZ;
+ *pll1 = hibmc_pll_table[idx].pll1_config_value;
+ *pll2 = hibmc_pll_table[idx].pll2_config_value;
}
/*
@@ -319,7 +329,7 @@ static u32 display_ctrl_adjust(struct drm_device *dev,
x = mode->hdisplay;
y = mode->vdisplay;
- get_pll_config(x, y, &pll1, &pll2);
+ get_pll_config(mode, &pll1, &pll2);
writel(pll2, priv->mmio + CRT_PLL2_HS);
set_vclock_hisilicon(dev, pll1);
diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c
index 1640609cdbc0..37748ef17360 100644
--- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c
+++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c
@@ -174,6 +174,15 @@ void hibmc_set_current_gate(struct hibmc_drm_private *priv, unsigned int gate)
writel(gate, mmio + gate_reg);
}
+static void hibmc_display_ctrl(struct hibmc_drm_private *priv)
+{
+ u32 reg;
+
+ reg = readl(priv->mmio + HIBMC_DISPLAY_CONTROL_HISILE);
+ reg |= HIBMC_DISPLAY_CONTROL_PANELDATE(1);
+ writel(reg, priv->mmio + HIBMC_DISPLAY_CONTROL_HISILE);
+}
+
static void hibmc_hw_config(struct hibmc_drm_private *priv)
{
u32 reg;
@@ -205,6 +214,8 @@ static void hibmc_hw_config(struct hibmc_drm_private *priv)
reg |= HIBMC_MSCCTL_LOCALMEM_RESET(1);
writel(reg, priv->mmio + HIBMC_MISC_CTRL);
+
+ hibmc_display_ctrl(priv);
}
static int hibmc_hw_map(struct hibmc_drm_private *priv)
diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c
index 9e29386700c8..1700bb2334f0 100644
--- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c
+++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_vdac.c
@@ -70,26 +70,6 @@ static const struct drm_connector_funcs hibmc_connector_funcs = {
.atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
};
-static void hibmc_encoder_mode_set(struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adj_mode)
-{
- u32 reg;
- struct drm_device *dev = encoder->dev;
- struct hibmc_drm_private *priv = to_hibmc_drm_private(dev);
-
- reg = readl(priv->mmio + HIBMC_DISPLAY_CONTROL_HISILE);
- reg |= HIBMC_DISPLAY_CONTROL_FPVDDEN(1);
- reg |= HIBMC_DISPLAY_CONTROL_PANELDATE(1);
- reg |= HIBMC_DISPLAY_CONTROL_FPEN(1);
- reg |= HIBMC_DISPLAY_CONTROL_VBIASEN(1);
- writel(reg, priv->mmio + HIBMC_DISPLAY_CONTROL_HISILE);
-}
-
-static const struct drm_encoder_helper_funcs hibmc_encoder_helper_funcs = {
- .mode_set = hibmc_encoder_mode_set,
-};
-
int hibmc_vdac_init(struct hibmc_drm_private *priv)
{
struct drm_device *dev = &priv->dev;
@@ -112,8 +92,6 @@ int hibmc_vdac_init(struct hibmc_drm_private *priv)
goto err;
}
- drm_encoder_helper_add(encoder, &hibmc_encoder_helper_funcs);
-
ret = drm_connector_init_with_ddc(dev, connector,
&hibmc_connector_funcs,
DRM_MODE_CONNECTOR_VGA,
diff --git a/drivers/gpu/drm/i915/display/intel_atomic.c b/drivers/gpu/drm/i915/display/intel_atomic.c
index 12d6ed940751..c1da487e8895 100644
--- a/drivers/gpu/drm/i915/display/intel_atomic.c
+++ b/drivers/gpu/drm/i915/display/intel_atomic.c
@@ -290,6 +290,12 @@ static void intel_crtc_put_color_blobs(struct intel_crtc_state *crtc_state)
drm_property_blob_put(crtc_state->pre_csc_lut);
drm_property_blob_put(crtc_state->post_csc_lut);
+
+ crtc_state->hw.degamma_lut = NULL;
+ crtc_state->hw.gamma_lut = NULL;
+ crtc_state->hw.ctm = NULL;
+ crtc_state->pre_csc_lut = NULL;
+ crtc_state->post_csc_lut = NULL;
}
void intel_crtc_free_hw_state(struct intel_crtc_state *crtc_state)
diff --git a/drivers/gpu/drm/i915/gt/intel_region_lmem.c b/drivers/gpu/drm/i915/gt/intel_region_lmem.c
index 51bb27e10a4f..7699e8fcf5ed 100644
--- a/drivers/gpu/drm/i915/gt/intel_region_lmem.c
+++ b/drivers/gpu/drm/i915/gt/intel_region_lmem.c
@@ -37,7 +37,7 @@ _resize_bar(struct drm_i915_private *i915, int resno, resource_size_t size)
_release_bars(pdev);
- ret = pci_resize_resource(pdev, resno, bar_size);
+ ret = pci_resize_resource(pdev, resno, bar_size, 0);
if (ret) {
drm_info(&i915->drm, "Failed to resize BAR%d to %dM (%pe)\n",
resno, 1 << bar_size, ERR_PTR(ret));
diff --git a/drivers/gpu/drm/msm/dp/dp_display.c b/drivers/gpu/drm/msm/dp/dp_display.c
index a5c1534eafdb..3bb8010f2efe 100644
--- a/drivers/gpu/drm/msm/dp/dp_display.c
+++ b/drivers/gpu/drm/msm/dp/dp_display.c
@@ -37,9 +37,9 @@ enum {
ISR_DISCONNECTED,
ISR_CONNECT_PENDING,
ISR_CONNECTED,
- ISR_HPD_REPLUG_COUNT,
+ ISR_HPD_IO_GLITCH_COUNT,
ISR_IRQ_HPD_PULSE_COUNT,
- ISR_HPD_LO_GLITH_COUNT,
+ ISR_HPD_REPLUG_COUNT,
};
/* event thread connection state */
diff --git a/drivers/gpu/drm/msm/dp/dp_reg.h b/drivers/gpu/drm/msm/dp/dp_reg.h
index 3835c7f5cb98..2b6c8ba2335f 100644
--- a/drivers/gpu/drm/msm/dp/dp_reg.h
+++ b/drivers/gpu/drm/msm/dp/dp_reg.h
@@ -49,8 +49,8 @@
#define DP_DP_IRQ_HPD_INT_ACK (0x00000002)
#define DP_DP_HPD_REPLUG_INT_ACK (0x00000004)
#define DP_DP_HPD_UNPLUG_INT_ACK (0x00000008)
-#define DP_DP_HPD_STATE_STATUS_BITS_MASK (0x0000000F)
-#define DP_DP_HPD_STATE_STATUS_BITS_SHIFT (0x1C)
+#define DP_DP_HPD_STATE_STATUS_BITS_MASK (0x00000007)
+#define DP_DP_HPD_STATE_STATUS_BITS_SHIFT (0x1D)
#define REG_DP_DP_HPD_INT_MASK (0x0000000C)
#define DP_DP_HPD_PLUG_INT_MASK (0x00000001)
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowramin.c b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowramin.c
index 023ddc7c5399..a295e94e4cb8 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowramin.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowramin.c
@@ -65,13 +65,14 @@ pramin_init(struct nvkm_bios *bios, const char *name)
/* we can't get the bios image pointer without PDISP */
if (device->card_type >= GA100)
- addr = device->chipset == 0x170; /*XXX: find the fuse reg for this */
+ addr = nvkm_rd32(device, 0x820c04);
else
if (device->card_type >= GM100)
addr = nvkm_rd32(device, 0x021c04);
else
if (device->card_type >= NV_C0)
addr = nvkm_rd32(device, 0x022500);
+
if (addr & 0x00000001) {
nvkm_debug(subdev, "... display disabled\n");
return ERR_PTR(-ENODEV);
diff --git a/drivers/gpu/drm/panthor/panthor_sched.c b/drivers/gpu/drm/panthor/panthor_sched.c
index 2d4dacece655..cedcde0f10bd 100644
--- a/drivers/gpu/drm/panthor/panthor_sched.c
+++ b/drivers/gpu/drm/panthor/panthor_sched.c
@@ -999,7 +999,8 @@ group_unbind_locked(struct panthor_group *group)
/* Tiler OOM events will be re-issued next time the group is scheduled. */
atomic_set(&group->tiler_oom, 0);
- cancel_work(&group->tiler_oom_work);
+ if (cancel_work(&group->tiler_oom_work))
+ group_put(group);
for (u32 i = 0; i < group->queue_count; i++)
group->queues[i]->doorbell_id = -1;
@@ -1439,7 +1440,10 @@ static int group_process_tiler_oom(struct panthor_group *group, u32 cs_id)
if (unlikely(csg_id < 0))
return 0;
- if (IS_ERR(heaps) || frag_end > vt_end || vt_end >= vt_start) {
+ if (IS_ERR(heaps)) {
+ ret = -EINVAL;
+ heaps = NULL;
+ } else if (frag_end > vt_end || vt_end >= vt_start) {
ret = -EINVAL;
} else {
/* We do the allocation without holding the scheduler lock to avoid
@@ -2232,7 +2236,13 @@ tick_ctx_apply(struct panthor_scheduler *sched, struct panthor_sched_tick_ctx *c
csg_iface = panthor_fw_get_csg_iface(ptdev, csg_id);
csg_slot = &sched->csg_slots[csg_id];
- group_bind_locked(group, csg_id);
+ ret = group_bind_locked(group, csg_id);
+ if (ret) {
+ panthor_device_schedule_reset(ptdev);
+ ctx->csg_upd_failed_mask |= BIT(csg_id);
+ return;
+ }
+
csg_slot_prog_locked(ptdev, csg_id, new_csg_prio--);
csgs_upd_ctx_queue_reqs(ptdev, &upd_ctx, csg_id,
group->state == PANTHOR_CS_GROUP_SUSPENDED ?
@@ -2532,7 +2542,14 @@ static void sched_resume_tick(struct panthor_device *ptdev)
else
delay_jiffies = 0;
- sched_queue_delayed_work(sched, tick, delay_jiffies);
+ /* We schedule immediate ticks when we need to process events on CSGs,
+ * but those don't change the resched_target because we want the other
+ * groups to stay scheduled for the remaining of the GPU timeslot they
+ * were given. Make sure those immediate ticks don't get overruled by
+ * a sched_queue_delayed_work() that would delay the tick execution.
+ */
+ if (!delayed_work_pending(&sched->tick_work))
+ sched_queue_delayed_work(sched, tick, delay_jiffies);
}
static void group_schedule_locked(struct panthor_group *group, u32 queue_mask)
diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index bf2d4b16dc2a..49a9ec358124 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -28,6 +28,7 @@
#include <linux/debugfs.h>
#include <linux/iosys-map.h>
+#include <linux/overflow.h>
#include <linux/pci.h>
#include <drm/drm_device.h>
@@ -807,6 +808,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
int aligned = width;
int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
int pitch_mask = 0;
+ int pitch;
switch (cpp) {
case 1:
@@ -821,9 +823,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
break;
}
- aligned += pitch_mask;
+ if (check_add_overflow(aligned, pitch_mask, &aligned))
+ return 0;
aligned &= ~pitch_mask;
- return aligned * cpp;
+ if (check_mul_overflow(aligned, cpp, &pitch))
+ return 0;
+ return pitch;
}
int radeon_mode_dumb_create(struct drm_file *file_priv,
@@ -837,8 +842,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
args->pitch = radeon_align_pitch(rdev, args->width,
DIV_ROUND_UP(args->bpp, 8), 0);
+ if (!args->pitch)
+ return -EINVAL;
args->size = (u64)args->pitch * args->height;
args->size = ALIGN(args->size, PAGE_SIZE);
+ if (!args->size)
+ return -EINVAL;
r = radeon_gem_object_create(rdev, args->size, 0,
RADEON_GEM_DOMAIN_VRAM, 0,
diff --git a/drivers/gpu/drm/radeon/radeon_ring.c b/drivers/gpu/drm/radeon/radeon_ring.c
index 581ae20c46e4..a5dff072c1ac 100644
--- a/drivers/gpu/drm/radeon/radeon_ring.c
+++ b/drivers/gpu/drm/radeon/radeon_ring.c
@@ -356,8 +356,10 @@ int radeon_ring_restore(struct radeon_device *rdev, struct radeon_ring *ring,
/* restore the saved ring content */
r = radeon_ring_lock(rdev, ring, size);
- if (r)
+ if (r) {
+ kvfree(data);
return r;
+ }
for (i = 0; i < size; ++i) {
radeon_ring_write(ring, data[i]);
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
index 93ed841f5dce..04879e23797f 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
@@ -332,7 +332,7 @@ void rockchip_gem_free_object(struct drm_gem_object *obj)
struct rockchip_drm_private *private = drm->dev_private;
struct rockchip_gem_object *rk_obj = to_rockchip_obj(obj);
- if (obj->import_attach) {
+ if (drm_gem_is_imported(obj)) {
if (private->domain) {
rockchip_gem_iommu_unmap(rk_obj);
} else {
diff --git a/drivers/gpu/drm/tegra/dc.c b/drivers/gpu/drm/tegra/dc.c
index 90d3d30a8415..c32e20e46169 100644
--- a/drivers/gpu/drm/tegra/dc.c
+++ b/drivers/gpu/drm/tegra/dc.c
@@ -100,8 +100,10 @@ bool tegra_dc_has_output(struct tegra_dc *dc, struct device *dev)
int err;
of_for_each_phandle(&it, err, np, "nvidia,outputs", NULL, 0)
- if (it.node == dev->of_node)
+ if (it.node == dev->of_node) {
+ of_node_put(it.node);
return true;
+ }
return false;
}
diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
index b4eb030ea961..59c574a87c6f 100644
--- a/drivers/gpu/drm/tegra/gem.c
+++ b/drivers/gpu/drm/tegra/gem.c
@@ -69,7 +69,7 @@ static struct host1x_bo_mapping *tegra_bo_pin(struct device *dev, struct host1x_
return ERR_PTR(-ENOMEM);
kref_init(&map->ref);
- map->bo = host1x_bo_get(bo);
+ map->bo = bo;
map->direction = direction;
map->dev = dev;
@@ -170,7 +170,6 @@ static void tegra_bo_unpin(struct host1x_bo_mapping *map)
kfree(map->sgt);
}
- host1x_bo_put(map->bo);
kfree(map);
}
@@ -235,6 +234,7 @@ static const struct host1x_bo_ops tegra_bo_ops = {
static int tegra_bo_iommu_map(struct tegra_drm *tegra, struct tegra_bo *bo)
{
int prot = IOMMU_READ | IOMMU_WRITE;
+ ssize_t size;
int err;
if (bo->mm)
@@ -256,13 +256,15 @@ static int tegra_bo_iommu_map(struct tegra_drm *tegra, struct tegra_bo *bo)
bo->iova = bo->mm->start;
- bo->size = iommu_map_sgtable(tegra->domain, bo->iova, bo->sgt, prot);
- if (!bo->size) {
+ size = iommu_map_sgtable(tegra->domain, bo->iova, bo->sgt, prot);
+ if (size < 0) {
dev_err(tegra->drm->dev, "failed to map buffer\n");
- err = -ENOMEM;
+ err = size;
goto remove;
}
+ bo->size = size;
+
mutex_unlock(&tegra->mm_lock);
return 0;
@@ -504,17 +506,9 @@ static struct tegra_bo *tegra_bo_import(struct drm_device *drm,
void tegra_bo_free_object(struct drm_gem_object *gem)
{
struct tegra_drm *tegra = gem->dev->dev_private;
- struct host1x_bo_mapping *mapping, *tmp;
struct tegra_bo *bo = to_tegra_bo(gem);
- /* remove all mappings of this buffer object from any caches */
- list_for_each_entry_safe(mapping, tmp, &bo->base.mappings, list) {
- if (mapping->cache)
- host1x_bo_unpin(mapping);
- else
- dev_err(gem->dev->dev, "mapping %p stale for device %s\n", mapping,
- dev_name(mapping->dev));
- }
+ host1x_bo_clear_cached_mappings(&bo->base);
if (tegra->domain)
tegra_bo_iommu_unmap(tegra, bo);
diff --git a/drivers/gpu/drm/tegra/gr2d.c b/drivers/gpu/drm/tegra/gr2d.c
index a160d01f26e1..e68847fb43d1 100644
--- a/drivers/gpu/drm/tegra/gr2d.c
+++ b/drivers/gpu/drm/tegra/gr2d.c
@@ -100,9 +100,6 @@ static int gr2d_exit(struct host1x_client *client)
if (err < 0)
return err;
- pm_runtime_dont_use_autosuspend(client->dev);
- pm_runtime_force_suspend(client->dev);
-
host1x_client_iommu_detach(client);
host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(gr2d->channel);
@@ -276,15 +273,21 @@ static int gr2d_probe(struct platform_device *pdev)
if (err)
return err;
+ /* initialize address register map */
+ for (i = 0; i < ARRAY_SIZE(gr2d_addr_regs); i++)
+ set_bit(gr2d_addr_regs[i], gr2d->addr_regs);
+
+ pm_runtime_enable(dev);
+
err = host1x_client_register(&gr2d->client.base);
if (err < 0) {
+ pm_runtime_disable(dev);
dev_err(dev, "failed to register host1x client: %d\n", err);
return err;
}
- /* initialize address register map */
- for (i = 0; i < ARRAY_SIZE(gr2d_addr_regs); i++)
- set_bit(gr2d_addr_regs[i], gr2d->addr_regs);
+ pm_runtime_use_autosuspend(dev);
+ pm_runtime_set_autosuspend_delay(dev, 500);
return 0;
}
@@ -367,10 +370,6 @@ static int __maybe_unused gr2d_runtime_resume(struct device *dev)
goto disable_clk;
}
- pm_runtime_enable(dev);
- pm_runtime_use_autosuspend(dev);
- pm_runtime_set_autosuspend_delay(dev, 500);
-
return 0;
disable_clk:
diff --git a/drivers/gpu/drm/tegra/gr3d.c b/drivers/gpu/drm/tegra/gr3d.c
index 00c8564520e7..3cde55b2ff28 100644
--- a/drivers/gpu/drm/tegra/gr3d.c
+++ b/drivers/gpu/drm/tegra/gr3d.c
@@ -108,9 +108,6 @@ static int gr3d_exit(struct host1x_client *client)
if (err < 0)
return err;
- pm_runtime_dont_use_autosuspend(client->dev);
- pm_runtime_force_suspend(client->dev);
-
host1x_client_iommu_detach(client);
host1x_syncpt_put(client->syncpts[0]);
host1x_channel_put(gr3d->channel);
@@ -529,16 +526,22 @@ static int gr3d_probe(struct platform_device *pdev)
if (err)
return err;
+ /* initialize address register map */
+ for (i = 0; i < ARRAY_SIZE(gr3d_addr_regs); i++)
+ set_bit(gr3d_addr_regs[i], gr3d->addr_regs);
+
+ pm_runtime_enable(&pdev->dev);
+
err = host1x_client_register(&gr3d->client.base);
if (err < 0) {
+ pm_runtime_disable(&pdev->dev);
dev_err(&pdev->dev, "failed to register host1x client: %d\n",
err);
return err;
}
- /* initialize address register map */
- for (i = 0; i < ARRAY_SIZE(gr3d_addr_regs); i++)
- set_bit(gr3d_addr_regs[i], gr3d->addr_regs);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_set_autosuspend_delay(&pdev->dev, 500);
return 0;
}
@@ -601,10 +604,6 @@ static int __maybe_unused gr3d_runtime_resume(struct device *dev)
goto disable_clk;
}
- pm_runtime_enable(dev);
- pm_runtime_use_autosuspend(dev);
- pm_runtime_set_autosuspend_delay(dev, 500);
-
return 0;
disable_clk:
diff --git a/drivers/gpu/drm/tegra/submit.c b/drivers/gpu/drm/tegra/submit.c
index 2430fcc97448..705382035f5f 100644
--- a/drivers/gpu/drm/tegra/submit.c
+++ b/drivers/gpu/drm/tegra/submit.c
@@ -76,7 +76,7 @@ gather_bo_pin(struct device *dev, struct host1x_bo *bo, enum dma_data_direction
return ERR_PTR(-ENOMEM);
kref_init(&map->ref);
- map->bo = host1x_bo_get(bo);
+ map->bo = bo;
map->direction = direction;
map->dev = dev;
@@ -117,7 +117,6 @@ static void gather_bo_unpin(struct host1x_bo_mapping *map)
dma_unmap_sgtable(map->dev, map->sgt, map->direction, 0);
sg_free_table(map->sgt);
kfree(map->sgt);
- host1x_bo_put(map->bo);
kfree(map);
}
diff --git a/drivers/gpu/drm/tidss/tidss_kms.c b/drivers/gpu/drm/tidss/tidss_kms.c
index f371518f8697..bc385d31ee6d 100644
--- a/drivers/gpu/drm/tidss/tidss_kms.c
+++ b/drivers/gpu/drm/tidss/tidss_kms.c
@@ -267,8 +267,6 @@ int tidss_modeset_init(struct tidss_device *tidss)
if (ret)
return ret;
- drm_mode_config_reset(ddev);
-
dev_dbg(tidss->dev, "%s done\n", __func__);
return 0;
diff --git a/drivers/gpu/drm/v3d/v3d_submit.c b/drivers/gpu/drm/v3d/v3d_submit.c
index 23472c7af41a..5ffa4c0f9e8a 100644
--- a/drivers/gpu/drm/v3d/v3d_submit.c
+++ b/drivers/gpu/drm/v3d/v3d_submit.c
@@ -461,6 +461,8 @@ v3d_get_cpu_indirect_csd_params(struct drm_file *file_priv,
sizeof(indirect_csd.wg_uniform_offsets));
info->indirect = drm_gem_object_lookup(file_priv, indirect_csd.indirect);
+ if (!info->indirect)
+ return -ENOENT;
return v3d_setup_csd_jobs_and_bos(file_priv, v3d, &indirect_csd.submit,
&info->job, &info->clean_job,
diff --git a/drivers/gpu/drm/xe/tests/xe_pci.c b/drivers/gpu/drm/xe/tests/xe_pci.c
index 67404863087e..69493e36de13 100644
--- a/drivers/gpu/drm/xe/tests/xe_pci.c
+++ b/drivers/gpu/drm/xe/tests/xe_pci.c
@@ -9,7 +9,6 @@
#include <kunit/test-bug.h>
#include <kunit/test.h>
-#include <kunit/test-bug.h>
#include <kunit/visibility.h>
/**
diff --git a/drivers/gpu/drm/xe/xe_guc_relay.c b/drivers/gpu/drm/xe/xe_guc_relay.c
index 0e5b43e1518e..decee13da311 100644
--- a/drivers/gpu/drm/xe/xe_guc_relay.c
+++ b/drivers/gpu/drm/xe/xe_guc_relay.c
@@ -675,12 +675,17 @@ static int relay_action_handler(struct xe_guc_relay *relay, u32 origin,
return relay_testloop_action_handler(relay, origin, msg, len, response, size);
type = FIELD_GET(GUC_HXG_MSG_0_TYPE, msg[0]);
+ relay_assert(relay, guc_hxg_type_is_action(type));
- if (IS_SRIOV_PF(relay_to_xe(relay)))
- ret = xe_gt_sriov_pf_service_process_request(gt, origin, msg, len, response, size);
- else
+ if (IS_SRIOV_PF(relay_to_xe(relay))) {
+ if (type == GUC_HXG_TYPE_REQUEST)
+ ret = xe_gt_sriov_pf_service_process_request(gt, origin, msg, len,
+ response, size);
+ else
+ ret = -EOPNOTSUPP;
+ } else {
ret = -EOPNOTSUPP;
-
+ }
if (type == GUC_HXG_TYPE_EVENT)
relay_assert(relay, ret <= 0);
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
index 0248597c6269..45d0ae074e65 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.c
+++ b/drivers/gpu/drm/xe/xe_hw_engine.c
@@ -561,7 +561,7 @@ static int hw_engine_init(struct xe_gt *gt, struct xe_hw_engine *hwe,
hwe->exl_port = xe_execlist_port_create(xe, hwe);
if (IS_ERR(hwe->exl_port)) {
err = PTR_ERR(hwe->exl_port);
- goto err_hwsp;
+ goto err_name;
}
} else {
/* GSCCS has a special interrupt for reset */
@@ -578,8 +578,6 @@ static int hw_engine_init(struct xe_gt *gt, struct xe_hw_engine *hwe,
return devm_add_action_or_reset(xe->drm.dev, hw_engine_fini, hwe);
-err_hwsp:
- xe_bo_unpin_map_no_vm(hwe->hwsp);
err_name:
hwe->name = NULL;
diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c
index 9b8903ad6f04..422dc351dc0e 100644
--- a/drivers/gpu/drm/xe/xe_pt.c
+++ b/drivers/gpu/drm/xe/xe_pt.c
@@ -786,12 +786,20 @@ static int xe_pt_zap_ptes_entry(struct xe_ptw *parent, pgoff_t offset,
{
struct xe_pt_zap_ptes_walk *xe_walk =
container_of(walk, typeof(*xe_walk), base);
- struct xe_pt *xe_child = container_of(*child, typeof(*xe_child), base);
+ struct xe_pt *xe_child;
pgoff_t end_offset;
- XE_WARN_ON(!*child);
XE_WARN_ON(!level);
+ /*
+ * Below would be unexpected behavior that needs to be root caused
+ * but better warn and bail than crash the driver.
+ */
+ if (XE_WARN_ON(!*child))
+ return 0;
+
+ xe_child = container_of(*child, typeof(*xe_child), base);
+
/*
* Note that we're called from an entry callback, and we're dealing
* with the child of that entry rather than the parent, so need to
diff --git a/drivers/gpu/drm/xe/xe_vram.c b/drivers/gpu/drm/xe/xe_vram.c
index 80ba2fc78837..db1304dc6c4a 100644
--- a/drivers/gpu/drm/xe/xe_vram.c
+++ b/drivers/gpu/drm/xe/xe_vram.c
@@ -33,7 +33,7 @@ _resize_bar(struct xe_device *xe, int resno, resource_size_t size)
if (pci_resource_len(pdev, resno))
pci_release_resource(pdev, resno);
- ret = pci_resize_resource(pdev, resno, bar_size);
+ ret = pci_resize_resource(pdev, resno, bar_size, 0);
if (ret) {
drm_info(&xe->drm, "Failed to resize BAR%d to %dM (%pe). Consider enabling 'Resizable BAR' support in your BIOS\n",
resno, 1 << bar_size, ERR_PTR(ret));
diff --git a/drivers/gpu/host1x/bus.c b/drivers/gpu/host1x/bus.c
index 8e09d6d328d2..ef42e9a883f4 100644
--- a/drivers/gpu/host1x/bus.c
+++ b/drivers/gpu/host1x/bus.c
@@ -457,7 +457,7 @@ static int host1x_device_add(struct host1x *host1x,
err = host1x_device_parse_dt(device, driver);
if (err < 0) {
- kfree(device);
+ put_device(&device->dev);
return err;
}
@@ -881,6 +881,20 @@ int host1x_client_resume(struct host1x_client *client)
}
EXPORT_SYMBOL(host1x_client_resume);
+/**
+ * host1x_bo_pin() - Create a DMA mapping for the buffer object
+ * @dev: Device onto which DMA map to
+ * @bo: Buffer object to map
+ * @dir: DMA direction
+ * @cache: Cache in which to store mapping, or NULL
+ *
+ * Creates a DMA mapping pointing to @bo for @dev. The refcount of @bo is incremented
+ * until host1x_bo_unpin is called.
+ *
+ * If @cache is specified, the mapping is also stored in the cache and not released
+ * until @bo is freed (refcount drops to zero). This improves performance when a buffer
+ * is pinned and unpinned frequently as in the case of display use.
+ */
struct host1x_bo_mapping *host1x_bo_pin(struct device *dev, struct host1x_bo *bo,
enum dma_data_direction dir,
struct host1x_bo_cache *cache)
@@ -893,6 +907,7 @@ struct host1x_bo_mapping *host1x_bo_pin(struct device *dev, struct host1x_bo *bo
list_for_each_entry(mapping, &cache->mappings, entry) {
if (mapping->bo == bo && mapping->direction == dir) {
kref_get(&mapping->ref);
+ host1x_bo_get(bo);
goto unlock;
}
}
@@ -902,6 +917,8 @@ struct host1x_bo_mapping *host1x_bo_pin(struct device *dev, struct host1x_bo *bo
if (IS_ERR(mapping))
goto unlock;
+ host1x_bo_get(bo);
+
spin_lock(&mapping->bo->lock);
list_add_tail(&mapping->list, &bo->mappings);
spin_unlock(&mapping->bo->lock);
@@ -912,7 +929,12 @@ struct host1x_bo_mapping *host1x_bo_pin(struct device *dev, struct host1x_bo *bo
list_add_tail(&mapping->entry, &cache->mappings);
- /* bump reference count to track the copy in the cache */
+ /*
+ * Bump the mapping reference count to track the mapping in the cache,
+ * but do not bump the BO's refcount. This allows the BO to still get freed,
+ * triggering the release of the cache mapping through
+ * host1x_bo_clear_cached_mappings.
+ */
kref_get(&mapping->ref);
}
@@ -942,9 +964,17 @@ static void __host1x_bo_unpin(struct kref *ref)
mapping->bo->ops->unpin(mapping);
}
+/**
+ * host1x_bo_unpin() - Release an established DMA mapping of a buffer object
+ * @mapping: Mapping to release
+ *
+ * Unmaps the given @mapping, unless it is cached. Decreases the refcount on
+ * the underlying buffer object.
+ */
void host1x_bo_unpin(struct host1x_bo_mapping *mapping)
{
struct host1x_bo_cache *cache = mapping->cache;
+ struct host1x_bo *bo = mapping->bo;
if (cache)
mutex_lock(&cache->lock);
@@ -953,5 +983,33 @@ void host1x_bo_unpin(struct host1x_bo_mapping *mapping)
if (cache)
mutex_unlock(&cache->lock);
+
+ host1x_bo_put(bo);
}
EXPORT_SYMBOL(host1x_bo_unpin);
+
+/**
+ * host1x_bo_clear_cached_mappings() - Remove all cached mappings pointing at a bo
+ * @bo: Buffer object to release mappings of
+ *
+ * Drops references to any mappings pointing to @bo left in any caches. This must
+ * be called by any host1x_bo implementers that may be pinned with caching enabled
+ * before freeing the bo.
+ */
+void host1x_bo_clear_cached_mappings(struct host1x_bo *bo)
+{
+ struct host1x_bo_mapping *mapping, *tmp;
+ struct host1x_bo_cache *cache;
+
+ list_for_each_entry_safe(mapping, tmp, &bo->mappings, list) {
+ cache = mapping->cache;
+ if (WARN_ON(!cache))
+ continue;
+
+ mutex_lock(&mapping->cache->lock);
+ WARN_ON(kref_read(&mapping->ref) != 1);
+ __host1x_bo_unpin(&mapping->ref);
+ mutex_unlock(&mapping->cache->lock);
+ }
+}
+EXPORT_SYMBOL(host1x_bo_clear_cached_mappings);
diff --git a/drivers/gpu/host1x/job.c b/drivers/gpu/host1x/job.c
index 3ed49e1fd933..70bda32f1ff4 100644
--- a/drivers/gpu/host1x/job.c
+++ b/drivers/gpu/host1x/job.c
@@ -235,6 +235,8 @@ static unsigned int pin_job(struct host1x *host, struct host1x_job *job)
}
if (host->domain) {
+ ssize_t map_err;
+
for_each_sgtable_sg(map->sgt, sg, j)
gather_size += sg->length;
@@ -248,11 +250,11 @@ static unsigned int pin_job(struct host1x *host, struct host1x_job *job)
goto put;
}
- err = iommu_map_sgtable(host->domain, iova_dma_addr(&host->iova, alloc),
- map->sgt, IOMMU_READ);
- if (err == 0) {
+ map_err = iommu_map_sgtable(host->domain, iova_dma_addr(&host->iova, alloc),
+ map->sgt, IOMMU_READ);
+ if (map_err < 0) {
__free_iova(&host->iova, alloc);
- err = -EINVAL;
+ err = map_err;
goto put;
}
diff --git a/drivers/hid/bpf/hid_bpf_dispatch.c b/drivers/hid/bpf/hid_bpf_dispatch.c
index b711d83dfde1..1de6a7e0af7b 100644
--- a/drivers/hid/bpf/hid_bpf_dispatch.c
+++ b/drivers/hid/bpf/hid_bpf_dispatch.c
@@ -17,6 +17,7 @@
#include <linux/kfifo.h>
#include <linux/minmax.h>
#include <linux/module.h>
+#include <linux/overflow.h>
#include "hid_bpf_dispatch.h"
struct hid_ops *hid_ops;
@@ -293,13 +294,15 @@ __bpf_kfunc __u8 *
hid_bpf_get_data(struct hid_bpf_ctx *ctx, unsigned int offset, const size_t rdwr_buf_size)
{
struct hid_bpf_ctx_kern *ctx_kern;
+ size_t end;
if (!ctx)
return NULL;
ctx_kern = container_of(ctx, struct hid_bpf_ctx_kern, ctx);
- if (rdwr_buf_size + offset > ctx->allocated_size)
+ if (check_add_overflow(rdwr_buf_size, offset, &end) ||
+ end > ctx->allocated_size)
return NULL;
return ctx_kern->data + offset;
diff --git a/drivers/hid/hid-appleir.c b/drivers/hid/hid-appleir.c
index c45e5aa569d2..7a3fcdf7cd14 100644
--- a/drivers/hid/hid-appleir.c
+++ b/drivers/hid/hid-appleir.c
@@ -109,9 +109,10 @@ struct appleir {
struct hid_device *hid;
unsigned short keymap[ARRAY_SIZE(appleir_key_table)];
struct timer_list key_up_timer; /* timer for key up */
- spinlock_t lock; /* protects .current_key */
+ spinlock_t lock; /* protects .current_key, .removing */
int current_key; /* the currently pressed key */
int prev_key_idx; /* key index in a 2 packets message */
+ bool removing; /* set during teardown; gates input_dev access */
};
static int get_key(int data)
@@ -172,7 +173,7 @@ static void key_up_tick(struct timer_list *t)
unsigned long flags;
spin_lock_irqsave(&appleir->lock, flags);
- if (appleir->current_key) {
+ if (!appleir->removing && appleir->current_key) {
key_up(hid, appleir, appleir->current_key);
appleir->current_key = 0;
}
@@ -195,6 +196,10 @@ static int appleir_raw_event(struct hid_device *hid, struct hid_report *report,
int index;
spin_lock_irqsave(&appleir->lock, flags);
+ if (appleir->removing) {
+ spin_unlock_irqrestore(&appleir->lock, flags);
+ goto out;
+ }
/*
* If we already have a key down, take it up before marking
* this one down
@@ -229,17 +234,25 @@ static int appleir_raw_event(struct hid_device *hid, struct hid_report *report,
appleir->prev_key_idx = 0;
if (!memcmp(data, keyrepeat, sizeof(keyrepeat))) {
- key_down(hid, appleir, appleir->current_key);
- /*
- * Remote doesn't do key up, either pull them up, in the test
- * above, or here set a timer which pulls them up after 1/8 s
- */
- mod_timer(&appleir->key_up_timer, jiffies + HZ / 8);
+ spin_lock_irqsave(&appleir->lock, flags);
+ if (!appleir->removing) {
+ key_down(hid, appleir, appleir->current_key);
+ /*
+ * Remote doesn't do key up, either pull them up, in
+ * the test above, or here set a timer which pulls them
+ * up after 1/8 s
+ */
+ mod_timer(&appleir->key_up_timer, jiffies + HZ / 8);
+ }
+ spin_unlock_irqrestore(&appleir->lock, flags);
goto out;
}
if (!memcmp(data, flatbattery, sizeof(flatbattery))) {
- battery_flat(appleir);
+ spin_lock_irqsave(&appleir->lock, flags);
+ if (!appleir->removing)
+ battery_flat(appleir);
+ spin_unlock_irqrestore(&appleir->lock, flags);
/* Fall through */
}
@@ -318,8 +331,20 @@ static int appleir_probe(struct hid_device *hid, const struct hid_device_id *id)
static void appleir_remove(struct hid_device *hid)
{
struct appleir *appleir = hid_get_drvdata(hid);
+ unsigned long flags;
+
+ /*
+ * Mark the driver as tearing down so that any concurrent raw_event
+ * (e.g. from a USB URB completion that hid_hw_stop() has not yet
+ * killed) and the key_up_timer softirq stop touching input_dev
+ * before hid_hw_stop() frees it via hidinput_disconnect().
+ */
+ spin_lock_irqsave(&appleir->lock, flags);
+ appleir->removing = true;
+ spin_unlock_irqrestore(&appleir->lock, flags);
+
+ timer_shutdown_sync(&appleir->key_up_timer);
hid_hw_stop(hid);
- del_timer_sync(&appleir->key_up_timer);
}
static const struct hid_device_id appleir_devices[] = {
diff --git a/drivers/hid/hid-core.c b/drivers/hid/hid-core.c
index 87d990ada868..9784258e48ea 100644
--- a/drivers/hid/hid-core.c
+++ b/drivers/hid/hid-core.c
@@ -2010,6 +2010,13 @@ int hid_report_raw_event(struct hid_device *hid, enum hid_report_type type, u8 *
u8 *cdata = data;
int ret = 0;
+ if (report_enum->numbered && (size < 1 || bufsize < 1)) {
+ hid_warn_ratelimited(hid,
+ "Event data for numbered report is too short (%d vs %zu)\n",
+ size, bufsize);
+ return -EINVAL;
+ }
+
report = hid_get_report(report_enum, data);
if (!report)
return 0;
diff --git a/drivers/hid/hid-logitech-hidpp.c b/drivers/hid/hid-logitech-hidpp.c
index 858ac2ab46bd..c4ea7f255256 100644
--- a/drivers/hid/hid-logitech-hidpp.c
+++ b/drivers/hid/hid-logitech-hidpp.c
@@ -164,7 +164,6 @@ struct hidpp_battery {
/**
* struct hidpp_scroll_counter - Utility class for processing high-resolution
* scroll events.
- * @dev: the input device for which events should be reported.
* @wheel_multiplier: the scalar multiplier to be applied to each wheel event
* @remainder: counts the number of high-resolution units moved since the last
* low-resolution event (REL_WHEEL or REL_HWHEEL) was sent. Should
diff --git a/drivers/hid/hid-multitouch.c b/drivers/hid/hid-multitouch.c
index 760f9db44c9e..b8e6cc67c6b6 100644
--- a/drivers/hid/hid-multitouch.c
+++ b/drivers/hid/hid-multitouch.c
@@ -31,6 +31,7 @@
* [1] https://gitlab.freedesktop.org/libevdev/hid-tools
*/
+#include <linux/bitmap.h>
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
@@ -84,8 +85,7 @@ enum latency_mode {
HID_LATENCY_HIGH = 1,
};
-#define MT_IO_SLOTS_MASK GENMASK(7, 0) /* reserve first 8 bits for slot tracking */
-#define MT_IO_FLAGS_RUNNING 32
+#define MT_IO_FLAGS_RUNNING 0
static const bool mtrue = true; /* default for true */
static const bool mfalse; /* default for false */
@@ -160,10 +160,9 @@ struct mt_device {
struct mt_class mtclass; /* our mt device class */
struct timer_list release_timer; /* to release sticky fingers */
struct hid_device *hdev; /* hid_device we're attached to */
- unsigned long mt_io_flags; /* mt flags (MT_IO_FLAGS_RUNNING)
- * first 8 bits are reserved for keeping the slot
- * states, this is fine because we only support up
- * to 250 slots (MT_MAX_MAXCONTACT)
+ unsigned long mt_io_flags; /* mt flags (MT_IO_FLAGS_RUNNING) */
+ unsigned long *active_slots; /* bitmap of slots with an active
+ * contact, sized for maxcontacts
*/
__u8 inputmode_value; /* InputMode HID feature value */
__u8 maxcontacts;
@@ -968,7 +967,7 @@ static void mt_release_pending_palms(struct mt_device *td,
for_each_set_bit(slotnum, app->pending_palm_slots, td->maxcontacts) {
clear_bit(slotnum, app->pending_palm_slots);
- clear_bit(slotnum, &td->mt_io_flags);
+ clear_bit(slotnum, td->active_slots);
input_mt_slot(input, slotnum);
input_mt_report_slot_inactive(input);
@@ -1174,9 +1173,9 @@ static int mt_process_slot(struct mt_device *td, struct input_dev *input,
input_event(input, EV_ABS, ABS_MT_TOUCH_MAJOR, major);
input_event(input, EV_ABS, ABS_MT_TOUCH_MINOR, minor);
- set_bit(slotnum, &td->mt_io_flags);
+ set_bit(slotnum, td->active_slots);
} else {
- clear_bit(slotnum, &td->mt_io_flags);
+ clear_bit(slotnum, td->active_slots);
}
return 0;
@@ -1311,7 +1310,7 @@ static void mt_touch_report(struct hid_device *hid,
* defect.
*/
if (app->quirks & MT_QUIRK_STICKY_FINGERS) {
- if (td->mt_io_flags & MT_IO_SLOTS_MASK)
+ if (!bitmap_empty(td->active_slots, td->maxcontacts))
mod_timer(&td->release_timer,
jiffies + msecs_to_jiffies(100));
else
@@ -1351,6 +1350,15 @@ static int mt_touch_input_configured(struct hid_device *hdev,
if (td->is_buttonpad)
__set_bit(INPUT_PROP_BUTTONPAD, input->propbit);
+ if (!td->active_slots) {
+ td->active_slots = devm_kcalloc(&td->hdev->dev,
+ BITS_TO_LONGS(td->maxcontacts),
+ sizeof(long),
+ GFP_KERNEL);
+ if (!td->active_slots)
+ return -ENOMEM;
+ }
+
app->pending_palm_slots = devm_kcalloc(&hi->input->dev,
BITS_TO_LONGS(td->maxcontacts),
sizeof(long),
@@ -1815,7 +1823,7 @@ static void mt_release_contacts(struct hid_device *hid)
for (i = 0; i < mt->num_slots; i++) {
input_mt_slot(input_dev, i);
input_mt_report_slot_inactive(input_dev);
- clear_bit(i, &td->mt_io_flags);
+ clear_bit(i, td->active_slots);
}
input_mt_sync_frame(input_dev);
input_sync(input_dev);
@@ -1838,7 +1846,7 @@ static void mt_expired_timeout(struct timer_list *t)
*/
if (test_and_set_bit_lock(MT_IO_FLAGS_RUNNING, &td->mt_io_flags))
return;
- if (td->mt_io_flags & MT_IO_SLOTS_MASK)
+ if (!bitmap_empty(td->active_slots, td->maxcontacts))
mt_release_contacts(hdev);
clear_bit_unlock(MT_IO_FLAGS_RUNNING, &td->mt_io_flags);
}
diff --git a/drivers/hid/hid-picolcd_core.c b/drivers/hid/hid-picolcd_core.c
index 297103be3381..282f11ec5df7 100644
--- a/drivers/hid/hid-picolcd_core.c
+++ b/drivers/hid/hid-picolcd_core.c
@@ -72,7 +72,8 @@ struct picolcd_pending *picolcd_send_and_wait(struct hid_device *hdev,
struct picolcd_pending *work;
struct hid_report *report = picolcd_out_report(report_id, hdev);
unsigned long flags;
- int i, j, k;
+ int i, j;
+ unsigned int k;
if (!report || !data)
return NULL;
diff --git a/drivers/hid/hid-playstation.c b/drivers/hid/hid-playstation.c
index 915b3ceb8d21..c40b0a0ed9e4 100644
--- a/drivers/hid/hid-playstation.c
+++ b/drivers/hid/hid-playstation.c
@@ -2247,6 +2247,12 @@ static int dualshock4_parse_report(struct ps_device *ps_dev, struct hid_report *
size == DS4_INPUT_REPORT_USB_SIZE) {
struct dualshock4_input_report_usb *usb = (struct dualshock4_input_report_usb *)data;
+ if (usb->num_touch_reports > ARRAY_SIZE(usb->touch_reports)) {
+ hid_err(hdev, "DualShock4 USB input report has invalid num_touch_reports=%d\n",
+ usb->num_touch_reports);
+ return -EINVAL;
+ }
+
ds4_report = &usb->common;
num_touch_reports = min_t(u8, usb->num_touch_reports,
ARRAY_SIZE(usb->touch_reports));
@@ -2262,6 +2268,12 @@ static int dualshock4_parse_report(struct ps_device *ps_dev, struct hid_report *
return -EILSEQ;
}
+ if (bt->num_touch_reports > ARRAY_SIZE(bt->touch_reports)) {
+ hid_err(hdev, "DualShock4 BT input report has invalid num_touch_reports=%d\n",
+ bt->num_touch_reports);
+ return -EINVAL;
+ }
+
ds4_report = &bt->common;
num_touch_reports = min_t(u8, bt->num_touch_reports,
ARRAY_SIZE(bt->touch_reports));
diff --git a/drivers/hid/hid-wiimote-modules.c b/drivers/hid/hid-wiimote-modules.c
index dbccdfa63916..dccb78bb3afd 100644
--- a/drivers/hid/hid-wiimote-modules.c
+++ b/drivers/hid/hid-wiimote-modules.c
@@ -2403,7 +2403,7 @@ static const struct wiimod_ops wiimod_guitar = {
.in_ext = wiimod_guitar_in_ext,
};
-/*
+/*
* Turntable
* DJ Hero came with a Turntable Controller that was plugged in
* as an extension.
@@ -2439,15 +2439,15 @@ static const __u16 wiimod_turntable_map[] = {
static void wiimod_turntable_in_ext(struct wiimote_data *wdata, const __u8 *ext)
{
__u8 be, cs, sx, sy, ed, rtt, rbg, rbr, rbb, ltt, lbg, lbr, lbb, bp, bm;
- /*
+ /*
* Byte | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
*------+------+-----+-----+-----+-----+------+------+--------+
- * 0 | RTT<4:3> | SX <5:0> |
- * 1 | RTT<2:1> | SY <5:0> |
+ * 0 | RTT<4:3> | SX <5:0> |
+ * 1 | RTT<2:1> | SY <5:0> |
*------+------+-----+-----+-----+-----+------+------+--------+
* 2 |RTT<0>| ED<4:3> | CS<3:0> | RTT<5> |
*------+------+-----+-----+-----+-----+------+------+--------+
- * 3 | ED<2:0> | LTT<4:0> |
+ * 3 | ED<2:0> | LTT<4:0> |
*------+------+-----+-----+-----+-----+------+------+--------+
* 4 | 0 | 0 | LBR | B- | 0 | B+ | RBR | LTT<5> |
*------+------+-----+-----+-----+-----+------+------+--------+
@@ -2458,20 +2458,20 @@ static void wiimod_turntable_in_ext(struct wiimote_data *wdata, const __u8 *ext)
* With Motion+ enabled, it will look like this:
* Byte | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
*------+------+-----+-----+-----+-----+------+------+--------+
- * 1 | RTT<4:3> | SX <5:1> | 0 |
- * 2 | RTT<2:1> | SY <5:1> | 0 |
+ * 1 | RTT<4:3> | SX <5:1> | 0 |
+ * 2 | RTT<2:1> | SY <5:1> | 0 |
*------+------+-----+-----+-----+-----+------+------+--------+
* 3 |RTT<0>| ED<4:3> | CS<3:0> | RTT<5> |
*------+------+-----+-----+-----+-----+------+------+--------+
- * 4 | ED<2:0> | LTT<4:0> |
+ * 4 | ED<2:0> | LTT<4:0> |
*------+------+-----+-----+-----+-----+------+------+--------+
* 5 | 0 | 0 | LBR | B- | 0 | B+ | RBR | XXXX |
*------+------+-----+-----+-----+-----+------+------+--------+
* 6 | LBB | 0 | RBG | BE | LBG | RBB | XXXX | XXXX |
*------+------+-----+-----+-----+-----+------+------+--------+
*/
-
- be = !(ext[5] & 0x10);
+
+ be = !(ext[5] & 0x10);
cs = ((ext[2] & 0x1e));
sx = ext[0] & 0x3f;
sy = ext[1] & 0x3f;
@@ -2499,32 +2499,32 @@ static void wiimod_turntable_in_ext(struct wiimote_data *wdata, const __u8 *ext)
input_report_abs(wdata->extension.input, ABS_HAT1X, ltt);
input_report_abs(wdata->extension.input, ABS_HAT2X, cs);
input_report_abs(wdata->extension.input, ABS_HAT3X, ed);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_G_RIGHT],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_G_RIGHT],
rbg);
input_report_key(wdata->extension.input,
wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_R_RIGHT],
rbr);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_B_RIGHT],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_B_RIGHT],
rbb);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_G_LEFT],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_G_LEFT],
lbg);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_R_LEFT],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_R_LEFT],
lbr);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_B_LEFT],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_B_LEFT],
lbb);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_EUPHORIA],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_EUPHORIA],
be);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_PLUS],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_PLUS],
bp);
- input_report_key(wdata->extension.input,
- wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_MINUS],
+ input_report_key(wdata->extension.input,
+ wiimod_turntable_map[WIIMOD_TURNTABLE_KEY_MINUS],
bm);
input_sync(wdata->extension.input);
@@ -2557,7 +2557,7 @@ static void wiimod_turntable_close(struct input_dev *dev)
static int wiimod_turntable_probe(const struct wiimod_ops *ops,
struct wiimote_data *wdata)
{
- int ret, i;
+ int ret, i;
wdata->extension.input = input_allocate_device();
if (!wdata->extension.input)
@@ -2594,9 +2594,9 @@ static int wiimod_turntable_probe(const struct wiimod_ops *ops,
input_set_abs_params(wdata->extension.input,
ABS_HAT1X, -8, 8, 0, 0);
input_set_abs_params(wdata->extension.input,
- ABS_HAT2X, 0, 31, 1, 1);
+ ABS_HAT2X, 0, 31, 1, 1);
input_set_abs_params(wdata->extension.input,
- ABS_HAT3X, 0, 7, 0, 0);
+ ABS_HAT3X, 0, 7, 0, 0);
ret = input_register_device(wdata->extension.input);
if (ret)
goto err_free;
diff --git a/drivers/hid/usbhid/hid-pidff.c b/drivers/hid/usbhid/hid-pidff.c
index 566eba0e045f..0e653b104b52 100644
--- a/drivers/hid/usbhid/hid-pidff.c
+++ b/drivers/hid/usbhid/hid-pidff.c
@@ -141,7 +141,7 @@ static const u8 pidff_effect_types[] = {
#define PID_BLOCK_LOAD_SUCCESS 0
#define PID_BLOCK_LOAD_FULL 1
#define PID_BLOCK_LOAD_ERROR 2
-static const u8 pidff_block_load_status[] = { 0x8c, 0x8d, 0x8e};
+static const u8 pidff_block_load_status[] = { 0x8c, 0x8d, 0x8e };
#define PID_EFFECT_START 0
#define PID_EFFECT_STOP 1
@@ -213,6 +213,62 @@ static int pidff_is_effect_conditional(struct ff_effect *effect)
effect->type == FF_FRICTION;
}
+/*
+ * Get PID effect index from FF effect type.
+ * Return 0 if invalid.
+ */
+static int pidff_effect_ff_to_pid(struct ff_effect *effect)
+{
+ switch (effect->type) {
+ case FF_CONSTANT:
+ return PID_CONSTANT;
+ case FF_RAMP:
+ return PID_RAMP;
+ case FF_SPRING:
+ return PID_SPRING;
+ case FF_DAMPER:
+ return PID_DAMPER;
+ case FF_INERTIA:
+ return PID_INERTIA;
+ case FF_FRICTION:
+ return PID_FRICTION;
+ case FF_PERIODIC:
+ switch (effect->u.periodic.waveform) {
+ case FF_SQUARE:
+ return PID_SQUARE;
+ case FF_TRIANGLE:
+ return PID_TRIANGLE;
+ case FF_SINE:
+ return PID_SINE;
+ case FF_SAW_UP:
+ return PID_SAW_UP;
+ case FF_SAW_DOWN:
+ return PID_SAW_DOWN;
+ }
+ }
+ pr_err("invalid effect type\n");
+ return -EINVAL;
+}
+
+/*
+ * Get effect id in the device descriptor.
+ * Return 0 if invalid.
+ */
+static int pidff_get_effect_type_id(struct pidff_device *pidff,
+ struct ff_effect *effect)
+{
+ int id = pidff_effect_ff_to_pid(effect);
+
+ if (id < 0)
+ return 0;
+
+ if (effect->type == FF_PERIODIC &&
+ pidff->quirks & HID_PIDFF_QUIRK_PERIODIC_SINE_ONLY)
+ id = PID_SINE;
+
+ return pidff->type_id[id];
+}
+
/*
* Clamp value for a given field
*/
@@ -249,11 +305,11 @@ static u32 pidff_rescale_time(u16 time, struct hid_field *field)
{
u32 scaled_time = time;
int exponent = field->unit_exponent;
- pr_debug("time field exponent: %d\n", exponent);
- for (;exponent < FF_TIME_EXPONENT; exponent++)
+ pr_debug("time field exponent: %d\n", exponent);
+ for (; exponent < FF_TIME_EXPONENT; exponent++)
scaled_time *= 10;
- for (;exponent > FF_TIME_EXPONENT; exponent--)
+ for (; exponent > FF_TIME_EXPONENT; exponent--)
scaled_time /= 10;
pr_debug("time calculated from %d to %d\n", time, scaled_time);
@@ -335,16 +391,16 @@ static void pidff_set_envelope_report(struct pidff_device *pidff,
pidff->set_envelope[PID_FADE_LEVEL].field);
pidff_set_time(&pidff->set_envelope[PID_ATTACK_TIME],
- envelope->attack_length);
+ envelope->attack_length);
pidff_set_time(&pidff->set_envelope[PID_FADE_TIME],
- envelope->fade_length);
+ envelope->fade_length);
hid_dbg(pidff->hid, "attack %u => %d\n",
envelope->attack_level,
pidff->set_envelope[PID_ATTACK_LEVEL].value[0]);
hid_hw_request(pidff->hid, pidff->reports[PID_SET_ENVELOPE],
- HID_REQ_SET_REPORT);
+ HID_REQ_SET_REPORT);
}
/*
@@ -353,15 +409,15 @@ static void pidff_set_envelope_report(struct pidff_device *pidff,
static int pidff_needs_set_envelope(struct ff_envelope *envelope,
struct ff_envelope *old)
{
- bool needs_new_envelope;
+ int needs_new_envelope;
+
needs_new_envelope = envelope->attack_level != 0 ||
envelope->fade_level != 0 ||
envelope->attack_length != 0 ||
envelope->fade_length != 0;
if (!needs_new_envelope)
- return false;
-
+ return 0;
if (!old)
return needs_new_envelope;
@@ -374,8 +430,8 @@ static int pidff_needs_set_envelope(struct ff_envelope *envelope,
/*
* Send constant force report to the device
*/
-static void pidff_set_constant_force_report(struct pidff_device *pidff,
- struct ff_effect *effect)
+static void pidff_set_constant_report(struct pidff_device *pidff,
+ struct ff_effect *effect)
{
pidff->set_constant[PID_EFFECT_BLOCK_INDEX].value[0] =
pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0];
@@ -383,7 +439,7 @@ static void pidff_set_constant_force_report(struct pidff_device *pidff,
effect->u.constant.level);
hid_hw_request(pidff->hid, pidff->reports[PID_SET_CONSTANT],
- HID_REQ_SET_REPORT);
+ HID_REQ_SET_REPORT);
}
/*
@@ -404,7 +460,7 @@ static void pidff_set_effect_report(struct pidff_device *pidff,
pidff->set_effect[PID_EFFECT_BLOCK_INDEX].value[0] =
pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0];
pidff->set_effect_type->value[0] =
- pidff->create_new_effect_type->value[0];
+ pidff_get_effect_type_id(pidff, effect);
pidff_set_duration(&pidff->set_effect[PID_DURATION],
effect->replay.length);
@@ -535,8 +591,8 @@ static int pidff_needs_set_condition(struct ff_effect *effect,
/*
* Send ramp force report to the device
*/
-static void pidff_set_ramp_force_report(struct pidff_device *pidff,
- struct ff_effect *effect)
+static void pidff_set_ramp_report(struct pidff_device *pidff,
+ struct ff_effect *effect)
{
pidff->set_ramp[PID_EFFECT_BLOCK_INDEX].value[0] =
pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0];
@@ -545,7 +601,7 @@ static void pidff_set_ramp_force_report(struct pidff_device *pidff,
pidff_set_signed(&pidff->set_ramp[PID_RAMP_END],
effect->u.ramp.end_level);
hid_hw_request(pidff->hid, pidff->reports[PID_SET_RAMP],
- HID_REQ_SET_REPORT);
+ HID_REQ_SET_REPORT);
}
/*
@@ -586,7 +642,7 @@ static void pidff_set_device_control(struct pidff_device *pidff, int field)
hid_dbg(pidff->hid, "DEVICE_CONTROL is a bitmask\n");
/* Clear current bitmask */
- for(i = 0; i < sizeof(pidff_device_control); i++) {
+ for (i = 0; i < sizeof(pidff_device_control); i++) {
index = pidff->control_id[i];
if (index < 1)
continue;
@@ -637,7 +693,7 @@ static void pidff_fetch_pool(struct pidff_device *pidff)
struct hid_device *hid = pidff->hid;
/* Repeat if PID_SIMULTANEOUS_MAX < 2 to make sure it's correct */
- for(i = 0; i < 20; i++) {
+ for (i = 0; i < 20; i++) {
hid_hw_request(hid, pidff->reports[PID_POOL], HID_REQ_GET_REPORT);
hid_hw_wait(hid);
@@ -663,9 +719,6 @@ static int pidff_request_effect_upload(struct pidff_device *pidff, int efnum)
{
int j;
- if (!pidff->effect_count)
- pidff_reset(pidff);
-
pidff->create_new_effect_type->value[0] = efnum;
hid_hw_request(pidff->hid, pidff->reports[PID_CREATE_NEW_EFFECT],
HID_REQ_SET_REPORT);
@@ -685,8 +738,6 @@ static int pidff_request_effect_upload(struct pidff_device *pidff, int efnum)
hid_dbg(pidff->hid, "device reported free memory: %d bytes\n",
pidff->block_load[PID_RAM_POOL_AVAILABLE].value ?
pidff->block_load[PID_RAM_POOL_AVAILABLE].value[0] : -1);
-
- pidff->effect_count++;
return 0;
}
if (pidff->block_load_status->value[0] ==
@@ -724,7 +775,7 @@ static void pidff_playback_pid(struct pidff_device *pidff, int pid_id, int n)
}
hid_hw_request(pidff->hid, pidff->reports[PID_EFFECT_OPERATION],
- HID_REQ_SET_REPORT);
+ HID_REQ_SET_REPORT);
}
/*
@@ -733,6 +784,7 @@ static void pidff_playback_pid(struct pidff_device *pidff, int pid_id, int n)
static int pidff_playback(struct input_dev *dev, int effect_id, int value)
{
struct pidff_device *pidff = dev->ff->private;
+
pidff_playback_pid(pidff, pidff->pid_id[effect_id], value);
return 0;
}
@@ -745,10 +797,7 @@ static void pidff_erase_pid(struct pidff_device *pidff, int pid_id)
{
pidff->block_free[PID_EFFECT_BLOCK_INDEX].value[0] = pid_id;
hid_hw_request(pidff->hid, pidff->reports[PID_BLOCK_FREE],
- HID_REQ_SET_REPORT);
-
- if (pidff->effect_count > 0)
- pidff->effect_count--;
+ HID_REQ_SET_REPORT);
}
/*
@@ -770,139 +819,79 @@ static int pidff_erase_effect(struct input_dev *dev, int effect_id)
pidff_playback_pid(pidff, pid_id, 0);
pidff_erase_pid(pidff, pid_id);
+ if (pidff->effect_count > 0)
+ pidff->effect_count--;
+
+ hid_dbg(pidff->hid, "current effect count: %d", pidff->effect_count);
return 0;
}
+#define PIDFF_SET_REPORT_IF_NEEDED(type, effect, old) \
+ ({ if (!old || pidff_needs_set_## type(effect, old)) \
+ pidff_set_ ##type## _report(pidff, effect); })
+
+#define PIDFF_SET_ENVELOPE_IF_NEEDED(type, effect, old) \
+ ({ if (pidff_needs_set_envelope(&effect->u.type.envelope, \
+ old ? &old->u.type.envelope : NULL)) \
+ pidff_set_envelope_report(pidff, &effect->u.type.envelope); })
+
/*
* Effect upload handler
*/
-static int pidff_upload_effect(struct input_dev *dev, struct ff_effect *effect,
+static int pidff_upload_effect(struct input_dev *dev, struct ff_effect *new,
struct ff_effect *old)
{
struct pidff_device *pidff = dev->ff->private;
- int type_id;
- int error;
+ const int type_id = pidff_get_effect_type_id(pidff, new);
- pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0] = 0;
- if (old) {
- pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0] =
- pidff->pid_id[effect->id];
+ if (!type_id) {
+ hid_err(pidff->hid, "effect type not supported\n");
+ return -EINVAL;
}
- switch (effect->type) {
+ if (!pidff->effect_count)
+ pidff_reset(pidff);
+
+ if (!old) {
+ int error = pidff_request_effect_upload(pidff, type_id);
+
+ if (error)
+ return error;
+
+ pidff->effect_count++;
+ hid_dbg(pidff->hid, "current effect count: %d", pidff->effect_count);
+ pidff->pid_id[new->id] =
+ pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0];
+ }
+
+ pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0] =
+ pidff->pid_id[new->id];
+
+ PIDFF_SET_REPORT_IF_NEEDED(effect, new, old);
+ switch (new->type) {
case FF_CONSTANT:
- if (!old) {
- error = pidff_request_effect_upload(pidff,
- pidff->type_id[PID_CONSTANT]);
- if (error)
- return error;
- }
- if (!old || pidff_needs_set_effect(effect, old))
- pidff_set_effect_report(pidff, effect);
- if (!old || pidff_needs_set_constant(effect, old))
- pidff_set_constant_force_report(pidff, effect);
- if (pidff_needs_set_envelope(&effect->u.constant.envelope,
- old ? &old->u.constant.envelope : NULL))
- pidff_set_envelope_report(pidff, &effect->u.constant.envelope);
+ PIDFF_SET_REPORT_IF_NEEDED(constant, new, old);
+ PIDFF_SET_ENVELOPE_IF_NEEDED(constant, new, old);
break;
case FF_PERIODIC:
- if (!old) {
- switch (effect->u.periodic.waveform) {
- case FF_SQUARE:
- type_id = PID_SQUARE;
- break;
- case FF_TRIANGLE:
- type_id = PID_TRIANGLE;
- break;
- case FF_SINE:
- type_id = PID_SINE;
- break;
- case FF_SAW_UP:
- type_id = PID_SAW_UP;
- break;
- case FF_SAW_DOWN:
- type_id = PID_SAW_DOWN;
- break;
- default:
- hid_err(pidff->hid, "invalid waveform\n");
- return -EINVAL;
- }
-
- if (pidff->quirks & HID_PIDFF_QUIRK_PERIODIC_SINE_ONLY)
- type_id = PID_SINE;
-
- error = pidff_request_effect_upload(pidff,
- pidff->type_id[type_id]);
- if (error)
- return error;
- }
- if (!old || pidff_needs_set_effect(effect, old))
- pidff_set_effect_report(pidff, effect);
- if (!old || pidff_needs_set_periodic(effect, old))
- pidff_set_periodic_report(pidff, effect);
- if (pidff_needs_set_envelope(&effect->u.periodic.envelope,
- old ? &old->u.periodic.envelope : NULL))
- pidff_set_envelope_report(pidff, &effect->u.periodic.envelope);
+ PIDFF_SET_REPORT_IF_NEEDED(periodic, new, old);
+ PIDFF_SET_ENVELOPE_IF_NEEDED(periodic, new, old);
break;
case FF_RAMP:
- if (!old) {
- error = pidff_request_effect_upload(pidff,
- pidff->type_id[PID_RAMP]);
- if (error)
- return error;
- }
- if (!old || pidff_needs_set_effect(effect, old))
- pidff_set_effect_report(pidff, effect);
- if (!old || pidff_needs_set_ramp(effect, old))
- pidff_set_ramp_force_report(pidff, effect);
- if (pidff_needs_set_envelope(&effect->u.ramp.envelope,
- old ? &old->u.ramp.envelope : NULL))
- pidff_set_envelope_report(pidff, &effect->u.ramp.envelope);
+ PIDFF_SET_REPORT_IF_NEEDED(ramp, new, old);
+ PIDFF_SET_ENVELOPE_IF_NEEDED(ramp, new, old);
break;
case FF_SPRING:
case FF_DAMPER:
case FF_INERTIA:
case FF_FRICTION:
- if (!old) {
- switch(effect->type) {
- case FF_SPRING:
- type_id = PID_SPRING;
- break;
- case FF_DAMPER:
- type_id = PID_DAMPER;
- break;
- case FF_INERTIA:
- type_id = PID_INERTIA;
- break;
- case FF_FRICTION:
- type_id = PID_FRICTION;
- break;
- }
- error = pidff_request_effect_upload(pidff,
- pidff->type_id[type_id]);
- if (error)
- return error;
- }
- if (!old || pidff_needs_set_effect(effect, old))
- pidff_set_effect_report(pidff, effect);
- if (!old || pidff_needs_set_condition(effect, old))
- pidff_set_condition_report(pidff, effect);
+ PIDFF_SET_REPORT_IF_NEEDED(condition, new, old);
break;
-
- default:
- hid_err(pidff->hid, "invalid type\n");
- return -EINVAL;
}
-
- if (!old)
- pidff->pid_id[effect->id] =
- pidff->block_load[PID_EFFECT_BLOCK_INDEX].value[0];
-
hid_dbg(pidff->hid, "uploaded\n");
-
return 0;
}
@@ -1233,6 +1222,7 @@ static int pidff_find_effects(struct pidff_device *pidff,
for (i = 0; i < sizeof(pidff_effect_types); i++) {
int pidff_type = pidff->type_id[i];
+
if (pidff->set_effect_type->usage[pidff_type].hid !=
pidff->create_new_effect_type->usage[pidff_type].hid) {
hid_err(pidff->hid,
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 0191562e606d..83f3215bc5bd 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -521,34 +521,6 @@ static ssize_t device_show(struct device *dev,
}
static DEVICE_ATTR_RO(device);
-static ssize_t driver_override_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct hv_device *hv_dev = device_to_hv_device(dev);
- int ret;
-
- ret = driver_set_override(dev, &hv_dev->driver_override, buf, count);
- if (ret)
- return ret;
-
- return count;
-}
-
-static ssize_t driver_override_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct hv_device *hv_dev = device_to_hv_device(dev);
- ssize_t len;
-
- device_lock(dev);
- len = sysfs_emit(buf, "%s\n", hv_dev->driver_override);
- device_unlock(dev);
-
- return len;
-}
-static DEVICE_ATTR_RW(driver_override);
-
/* Set up per device attributes in /sys/bus/vmbus/devices/<bus device> */
static struct attribute *vmbus_dev_attrs[] = {
&dev_attr_id.attr,
@@ -579,7 +551,6 @@ static struct attribute *vmbus_dev_attrs[] = {
&dev_attr_channel_vp_mapping.attr,
&dev_attr_vendor.attr,
&dev_attr_device.attr,
- &dev_attr_driver_override.attr,
NULL,
};
@@ -691,9 +662,11 @@ static const struct hv_vmbus_device_id *hv_vmbus_get_id(const struct hv_driver *
{
const guid_t *guid = &dev->dev_type;
const struct hv_vmbus_device_id *id;
+ int ret;
- /* When driver_override is set, only bind to the matching driver */
- if (dev->driver_override && strcmp(dev->driver_override, drv->name))
+ /* If a driver override is set, only bind to the matching driver */
+ ret = device_match_driver_override(&dev->device, &drv->driver);
+ if (ret == 0)
return NULL;
/* Look at the dynamic ids first, before the static ones */
@@ -701,8 +674,11 @@ static const struct hv_vmbus_device_id *hv_vmbus_get_id(const struct hv_driver *
if (!id)
id = hv_vmbus_dev_match(drv->id_table, guid);
- /* driver_override will always match, send a dummy id */
- if (!id && dev->driver_override)
+ /*
+ * If there's a matching driver override, this function should succeed,
+ * thus return a dummy device ID if no matching ID is found.
+ */
+ if (!id && ret > 0)
id = &vmbus_device_null;
return id;
@@ -981,6 +957,7 @@ static const struct dev_pm_ops vmbus_pm = {
/* The one and only one */
static const struct bus_type hv_bus = {
.name = "vmbus",
+ .driver_override = true,
.match = vmbus_match,
.shutdown = vmbus_shutdown,
.remove = vmbus_remove,
diff --git a/drivers/hwmon/Kconfig b/drivers/hwmon/Kconfig
index 19622dd6ec93..acd0f86cb9b3 100644
--- a/drivers/hwmon/Kconfig
+++ b/drivers/hwmon/Kconfig
@@ -1014,6 +1014,7 @@ config SENSORS_LTC2992
tristate "Linear Technology LTC2992"
depends on I2C
depends on GPIOLIB
+ select REGMAP_I2C
help
If you say yes here you get support for Linear Technology LTC2992
I2C System Monitor. The LTC2992 measures current, voltage, and
@@ -1140,6 +1141,7 @@ config SENSORS_MAX16065
config SENSORS_MAX1619
tristate "Maxim MAX1619 sensor chip"
depends on I2C
+ select REGMAP
help
If you say yes here you get support for MAX1619 sensor chip.
@@ -1258,6 +1260,7 @@ config SENSORS_MAX6650
config SENSORS_MAX6697
tristate "Maxim MAX6697 and compatibles"
depends on I2C
+ select REGMAP_I2C
help
If you say yes here you get support for MAX6581, MAX6602, MAX6622,
MAX6636, MAX6689, MAX6693, MAX6694, MAX6697, MAX6698, and MAX6699
diff --git a/drivers/hwmon/aspeed-g6-pwm-tach.c b/drivers/hwmon/aspeed-g6-pwm-tach.c
index d1f7f4397482..3be596544123 100644
--- a/drivers/hwmon/aspeed-g6-pwm-tach.c
+++ b/drivers/hwmon/aspeed-g6-pwm-tach.c
@@ -293,7 +293,10 @@ static int aspeed_tach_val_to_rpm(struct aspeed_pwm_tach_data *priv, u32 tach_va
priv->clk_rate, tach_val, tach_div);
rpm = (u64)priv->clk_rate * 60;
- do_div(rpm, tach_div);
+ if (tach_div)
+ do_div(rpm, tach_div);
+ else
+ rpm = 0;
return (int)rpm;
}
diff --git a/drivers/hwmon/asus_atk0110.c b/drivers/hwmon/asus_atk0110.c
index 1dc7e24fe4c5..4318a745f74c 100644
--- a/drivers/hwmon/asus_atk0110.c
+++ b/drivers/hwmon/asus_atk0110.c
@@ -1036,6 +1036,9 @@ static int atk_ec_present(struct atk_data *data)
if (obj->type != ACPI_TYPE_PACKAGE)
continue;
+ if (!obj->package.count)
+ continue;
+
id = &obj->package.elements[0];
if (id->type != ACPI_TYPE_INTEGER)
continue;
diff --git a/drivers/hwmon/it87.c b/drivers/hwmon/it87.c
index 5cfb98a0512f..c0c9826d44a2 100644
--- a/drivers/hwmon/it87.c
+++ b/drivers/hwmon/it87.c
@@ -1401,6 +1401,9 @@ static ssize_t set_fan(struct device *dev, struct device_attribute *attr,
if (kstrtol(buf, 10, &val) < 0)
return -EINVAL;
+ if (val < 0)
+ val = 0;
+
err = it87_lock(data);
if (err)
return err;
diff --git a/drivers/hwmon/occ/common.c b/drivers/hwmon/occ/common.c
index 42cc6068bb08..e18e80e832fd 100644
--- a/drivers/hwmon/occ/common.c
+++ b/drivers/hwmon/occ/common.c
@@ -214,6 +214,11 @@ int occ_update_response(struct occ *occ)
if (rc)
return rc;
+ if (!occ->active) {
+ rc = -ENODEV;
+ goto unlock;
+ }
+
/* limit the maximum rate of polling the OCC */
if (time_after(jiffies, occ->next_update)) {
rc = occ_poll(occ);
@@ -222,6 +227,7 @@ int occ_update_response(struct occ *occ)
rc = occ->last_error;
}
+unlock:
mutex_unlock(&occ->lock);
return rc;
}
@@ -1105,11 +1111,16 @@ static void occ_parse_poll_response(struct occ *occ)
int occ_active(struct occ *occ, bool active)
{
- int rc = mutex_lock_interruptible(&occ->lock);
+ struct device *hwmon = NULL;
+ int rc = mutex_lock_interruptible(&occ->hwmon_lock);
if (rc)
return rc;
+ rc = mutex_lock_interruptible(&occ->lock);
+ if (rc)
+ goto unlock_hwmon;
+
if (active) {
if (occ->active) {
rc = -EALREADY;
@@ -1154,14 +1165,17 @@ int occ_active(struct occ *occ, bool active)
goto unlock;
}
- if (occ->hwmon)
- hwmon_device_unregister(occ->hwmon);
+ hwmon = occ->hwmon;
occ->active = false;
occ->hwmon = NULL;
}
unlock:
mutex_unlock(&occ->lock);
+ if (hwmon)
+ hwmon_device_unregister(hwmon);
+unlock_hwmon:
+ mutex_unlock(&occ->hwmon_lock);
return rc;
}
@@ -1170,6 +1184,7 @@ int occ_setup(struct occ *occ)
int rc;
mutex_init(&occ->lock);
+ mutex_init(&occ->hwmon_lock);
occ->groups[0] = &occ->group;
rc = occ_setup_sysfs(occ);
@@ -1190,15 +1205,22 @@ EXPORT_SYMBOL_GPL(occ_setup);
void occ_shutdown(struct occ *occ)
{
- mutex_lock(&occ->lock);
+ struct device *hwmon;
occ_shutdown_sysfs(occ);
- if (occ->hwmon)
- hwmon_device_unregister(occ->hwmon);
+ mutex_lock(&occ->hwmon_lock);
+ mutex_lock(&occ->lock);
+
+ hwmon = occ->hwmon;
+ occ->active = false;
occ->hwmon = NULL;
mutex_unlock(&occ->lock);
+
+ if (hwmon)
+ hwmon_device_unregister(hwmon);
+ mutex_unlock(&occ->hwmon_lock);
}
EXPORT_SYMBOL_GPL(occ_shutdown);
diff --git a/drivers/hwmon/occ/common.h b/drivers/hwmon/occ/common.h
index 7ac4b2febce6..82f600093c7f 100644
--- a/drivers/hwmon/occ/common.h
+++ b/drivers/hwmon/occ/common.h
@@ -101,6 +101,7 @@ struct occ {
unsigned long next_update;
struct mutex lock; /* lock OCC access */
+ struct mutex hwmon_lock; /* serialize hwmon registration/removal */
struct device *hwmon;
struct occ_attribute *attrs;
diff --git a/drivers/hwmon/pmbus/adm1275.c b/drivers/hwmon/pmbus/adm1275.c
index 59ffc08289bd..622d0cdde001 100644
--- a/drivers/hwmon/pmbus/adm1275.c
+++ b/drivers/hwmon/pmbus/adm1275.c
@@ -509,7 +509,7 @@ static int adm1275_enable_vout_temp(struct adm1275_data *data,
static int adm1275_probe(struct i2c_client *client)
{
s32 (*config_read_fn)(const struct i2c_client *client, u8 reg);
- u8 block_buffer[I2C_SMBUS_BLOCK_MAX + 1];
+ u8 block_buffer[I2C_SMBUS_BLOCK_MAX + 1] = {0};
int config, device_config;
int ret;
struct pmbus_driver_info *info;
diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c
index e37fd206510a..a431cce33603 100644
--- a/drivers/hwmon/pmbus/pmbus_core.c
+++ b/drivers/hwmon/pmbus/pmbus_core.c
@@ -3315,18 +3315,23 @@ static void pmbus_regulator_notify_worker(struct work_struct *work)
int i, j;
for (i = 0; i < data->info->pages; i++) {
- int event;
+ unsigned int event;
event = atomic_xchg(&data->regulator_events[i], 0);
if (!event)
continue;
for (j = 0; j < data->info->num_regulators; j++) {
- if (i == rdev_get_id(data->rdevs[j])) {
+ if (i != rdev_get_id(data->rdevs[j]))
+ continue;
+ while (event) {
+ unsigned int _event = BIT(__ffs(event));
+
regulator_notifier_call_chain(data->rdevs[j],
- event, NULL);
- break;
+ _event, NULL);
+ event &= ~_event;
}
+ break;
}
}
}
diff --git a/drivers/hwmon/w83627hf.c b/drivers/hwmon/w83627hf.c
index 95115d7b863e..bb993bb09f40 100644
--- a/drivers/hwmon/w83627hf.c
+++ b/drivers/hwmon/w83627hf.c
@@ -1823,6 +1823,8 @@ static int w83627hf_probe(struct platform_device *pdev)
return 0;
error:
+ device_remove_file(dev, &dev_attr_vrm);
+ device_remove_file(dev, &dev_attr_cpu0_vid);
sysfs_remove_group(&dev->kobj, &w83627hf_group);
sysfs_remove_group(&dev->kobj, &w83627hf_group_opt);
return err;
@@ -1834,6 +1836,8 @@ static void w83627hf_remove(struct platform_device *pdev)
hwmon_device_unregister(data->hwmon_dev);
+ device_remove_file(&pdev->dev, &dev_attr_vrm);
+ device_remove_file(&pdev->dev, &dev_attr_cpu0_vid);
sysfs_remove_group(&pdev->dev.kobj, &w83627hf_group);
sysfs_remove_group(&pdev->dev.kobj, &w83627hf_group_opt);
}
diff --git a/drivers/hwmon/w83793.c b/drivers/hwmon/w83793.c
index 67728f60333f..ec03745328a6 100644
--- a/drivers/hwmon/w83793.c
+++ b/drivers/hwmon/w83793.c
@@ -1917,6 +1917,7 @@ static int w83793_probe(struct i2c_client *client)
for (i = 0; i < ARRAY_SIZE(w83793_vid); i++)
device_remove_file(dev, &w83793_vid[i].dev_attr);
+ device_remove_file(dev, &dev_attr_vrm);
for (i = 0; i < ARRAY_SIZE(w83793_left_fan); i++)
device_remove_file(dev, &w83793_left_fan[i].dev_attr);
diff --git a/drivers/hwspinlock/qcom_hwspinlock.c b/drivers/hwspinlock/qcom_hwspinlock.c
index 0390979fd765..712003a4640c 100644
--- a/drivers/hwspinlock/qcom_hwspinlock.c
+++ b/drivers/hwspinlock/qcom_hwspinlock.c
@@ -202,7 +202,6 @@ static struct regmap *qcom_hwspinlock_probe_mmio(struct platform_device *pdev,
static int qcom_hwspinlock_probe(struct platform_device *pdev)
{
struct hwspinlock_device *bank;
- struct reg_field field;
struct regmap *regmap;
size_t array_size;
u32 stride;
@@ -224,9 +223,7 @@ static int qcom_hwspinlock_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, bank);
for (i = 0; i < QCOM_MUTEX_NUM_LOCKS; i++) {
- field.reg = base + i * stride;
- field.lsb = 0;
- field.msb = 31;
+ struct reg_field field = REG_FIELD(base + i * stride, 0, 31);
bank->lock[i].priv = devm_regmap_field_alloc(&pdev->dev,
regmap, field);
diff --git a/drivers/hwtracing/coresight/coresight-cti-platform.c b/drivers/hwtracing/coresight/coresight-cti-platform.c
index d0ae10bf6128..0ed52a693c35 100644
--- a/drivers/hwtracing/coresight/coresight-cti-platform.c
+++ b/drivers/hwtracing/coresight/coresight-cti-platform.c
@@ -329,6 +329,7 @@ static int cti_plat_process_filter_sigs(struct cti_drvdata *drvdata,
if (!tg)
return -ENOMEM;
+ tg->nr_sigs = nr_filter_sigs;
err = cti_plat_read_trig_group(tg, fwnode, CTI_DT_FILTER_OUT_SIGS);
if (!err)
drvdata->config.trig_out_filter |= tg->used_mask;
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index 730ba893bf4c..05958827b3bb 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -1876,7 +1876,7 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata)
state->trcvmidcctlr0 = etm4x_read32(csa, TRCVMIDCCTLR0);
if (drvdata->numvmidc > 4)
- state->trcvmidcctlr0 = etm4x_read32(csa, TRCVMIDCCTLR1);
+ state->trcvmidcctlr1 = etm4x_read32(csa, TRCVMIDCCTLR1);
state->trcclaimset = etm4x_read32(csa, TRCCLAIMCLR);
@@ -2002,7 +2002,7 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata)
etm4x_relaxed_write32(csa, state->trcvmidcctlr0, TRCVMIDCCTLR0);
if (drvdata->numvmidc > 4)
- etm4x_relaxed_write32(csa, state->trcvmidcctlr0, TRCVMIDCCTLR1);
+ etm4x_relaxed_write32(csa, state->trcvmidcctlr1, TRCVMIDCCTLR1);
etm4x_relaxed_write32(csa, state->trcclaimset, TRCCLAIMSET);
diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c
index a01c9e54e2ed..f26dcb7a42a5 100644
--- a/drivers/hwtracing/coresight/coresight-sysfs.c
+++ b/drivers/hwtracing/coresight/coresight-sysfs.c
@@ -238,8 +238,10 @@ int coresight_enable_sysfs(struct coresight_device *csdev)
*/
hash = hashlen_hash(hashlen_string(NULL, dev_name(&csdev->dev)));
ret = idr_alloc_u32(&path_idr, path, &hash, hash, GFP_KERNEL);
- if (ret)
+ if (ret) {
+ coresight_disable_source_sysfs(csdev, NULL);
goto err_source;
+ }
break;
default:
/* We can't be here */
diff --git a/drivers/i2c/busses/i2c-mlxbf.c b/drivers/i2c/busses/i2c-mlxbf.c
index b3a73921ab69..d3f4885c49f6 100644
--- a/drivers/i2c/busses/i2c-mlxbf.c
+++ b/drivers/i2c/busses/i2c-mlxbf.c
@@ -1082,8 +1082,10 @@ static int mlxbf_i2c_init_resource(struct platform_device *pdev,
tmp_res->io = devm_platform_get_and_ioremap_resource(pdev, type, &tmp_res->params);
if (IS_ERR(tmp_res->io)) {
+ int ret = PTR_ERR(tmp_res->io);
+
devm_kfree(dev, tmp_res);
- return PTR_ERR(tmp_res->io);
+ return ret;
}
tmp_res->type = type;
diff --git a/drivers/i2c/busses/i2c-mt65xx.c b/drivers/i2c/busses/i2c-mt65xx.c
index 8beef73960c7..adcc43a24b5d 100644
--- a/drivers/i2c/busses/i2c-mt65xx.c
+++ b/drivers/i2c/busses/i2c-mt65xx.c
@@ -1253,7 +1253,7 @@ static int mtk_i2c_transfer(struct i2c_adapter *adap,
i2c->auto_restart = i2c->dev_comp->auto_restart;
/* checking if we can skip restart and optimize using WRRD mode */
- if (i2c->auto_restart && num == 2) {
+ if (num == 2) {
if (!(msgs[0].flags & I2C_M_RD) && (msgs[1].flags & I2C_M_RD) &&
msgs[0].addr == msgs[1].addr) {
i2c->auto_restart = 0;
diff --git a/drivers/i2c/i2c-core-of.c b/drivers/i2c/i2c-core-of.c
index 50e97e2ed2cf..a250921bbce0 100644
--- a/drivers/i2c/i2c-core-of.c
+++ b/drivers/i2c/i2c-core-of.c
@@ -178,11 +178,6 @@ static int of_i2c_notify(struct notifier_block *nb, unsigned long action,
return NOTIFY_OK;
}
- /*
- * Clear the flag before adding the device so that fw_devlink
- * doesn't skip adding consumers to this device.
- */
- fwnode_clear_flag(&rd->dn->fwnode, FWNODE_FLAG_NOT_DEVICE);
client = of_i2c_register_device(adap, rd->dn);
if (IS_ERR(client)) {
dev_err(&adap->dev, "failed to create client for '%pOF'\n",
diff --git a/drivers/i3c/master.c b/drivers/i3c/master.c
index fe6f956cc311..dd1508455da2 100644
--- a/drivers/i3c/master.c
+++ b/drivers/i3c/master.c
@@ -2055,12 +2055,11 @@ int i3c_master_add_i3c_dev_locked(struct i3c_master_controller *master,
bool enable_ibi = false;
int ret;
- if (!master)
- return -EINVAL;
-
newdev = i3c_master_alloc_i3c_dev(master, &info);
- if (IS_ERR(newdev))
- return PTR_ERR(newdev);
+ if (IS_ERR(newdev)) {
+ ret = PTR_ERR(newdev);
+ goto err_prevent_addr_reuse;
+ }
ret = i3c_master_attach_i3c_dev(master, newdev);
if (ret)
@@ -2182,6 +2181,16 @@ int i3c_master_add_i3c_dev_locked(struct i3c_master_controller *master,
err_free_dev:
i3c_master_free_i3c_dev(newdev);
+err_prevent_addr_reuse:
+ /*
+ * Although the device has not been added, the address has been
+ * assigned. Prevent the address from being used again.
+ */
+ if (i3c_bus_get_addr_slot_status(&master->bus, addr) == I3C_ADDR_SLOT_FREE)
+ i3c_bus_set_addr_slot_status(&master->bus, addr, I3C_ADDR_SLOT_I3C_DEV);
+
+ dev_err(&master->dev, "Failed to add I3C device at address %u, error %d\n", addr, ret);
+
return ret;
}
EXPORT_SYMBOL_GPL(i3c_master_add_i3c_dev_locked);
@@ -2853,7 +2862,7 @@ int i3c_master_register(struct i3c_master_controller *master,
if (ret)
goto err_put_dev;
- master->wq = alloc_workqueue("%s", 0, 0, dev_name(parent));
+ master->wq = alloc_workqueue("%s", WQ_PERCPU | WQ_FREEZABLE, 0, dev_name(parent));
if (!master->wq) {
ret = -ENOMEM;
goto err_put_dev;
diff --git a/drivers/iio/accel/mma8452.c b/drivers/iio/accel/mma8452.c
index de207526babe..c863359c02dd 100644
--- a/drivers/iio/accel/mma8452.c
+++ b/drivers/iio/accel/mma8452.c
@@ -254,6 +254,8 @@ static int mma8452_read(struct mma8452_data *data, __be16 buf[3])
ret = i2c_smbus_read_i2c_block_data(data->client, MMA8452_OUT_X,
3 * sizeof(__be16), (u8 *)buf);
+ if (ret < 0)
+ return ret;
ret = mma8452_set_runtime_pm_state(data->client, false);
diff --git a/drivers/iio/adc/Kconfig b/drivers/iio/adc/Kconfig
index 216f3c9ce183..57fc8f0de7b0 100644
--- a/drivers/iio/adc/Kconfig
+++ b/drivers/iio/adc/Kconfig
@@ -188,6 +188,7 @@ config AD7298
config AD7380
tristate "Analog Devices AD7380 ADC driver"
depends on SPI_MASTER
+ select REGMAP
select IIO_BUFFER
select IIO_TRIGGER
select IIO_TRIGGERED_BUFFER
diff --git a/drivers/iio/adc/xilinx-ams.c b/drivers/iio/adc/xilinx-ams.c
index e12eb7137b06..7d6820c6ea7f 100644
--- a/drivers/iio/adc/xilinx-ams.c
+++ b/drivers/iio/adc/xilinx-ams.c
@@ -871,6 +871,9 @@ static const struct iio_chan_spec *ams_event_to_channel(struct iio_dev *dev,
if (dev->channels[i].scan_index == scan_index)
break;
+ if (i == dev->num_channels)
+ return NULL;
+
return &dev->channels[i];
}
@@ -1012,6 +1015,8 @@ static void ams_handle_event(struct iio_dev *indio_dev, u32 event)
const struct iio_chan_spec *chan;
chan = ams_event_to_channel(indio_dev, event);
+ if (!chan)
+ return;
if (chan->type == IIO_TEMP) {
/*
diff --git a/drivers/iio/light/si1133.c b/drivers/iio/light/si1133.c
index 44fa152dbd24..bf7bf0f1631d 100644
--- a/drivers/iio/light/si1133.c
+++ b/drivers/iio/light/si1133.c
@@ -395,8 +395,14 @@ static int si1133_command(struct si1133_data *data, u8 cmd)
expected_seq = (data->rsp_seq + 1) & SI1133_MAX_CMD_CTR;
- if (cmd == SI1133_CMD_FORCE)
+ if (cmd == SI1133_CMD_FORCE) {
+ /* Flush pending IRQs from a previous timeout. */
+ regmap_read(data->regmap, SI1133_REG_IRQ_STATUS, &resp);
+ regmap_write(data->regmap, SI1133_REG_IRQ_ENABLE,
+ SI1133_IRQ_CHANNEL_ENABLE);
+
reinit_completion(&data->completion);
+ }
err = regmap_write(data->regmap, SI1133_REG_COMMAND, cmd);
if (err) {
@@ -409,6 +415,7 @@ static int si1133_command(struct si1133_data *data, u8 cmd)
/* wait for irq */
if (!wait_for_completion_timeout(&data->completion,
msecs_to_jiffies(SI1133_COMPLETION_TIMEOUT_MS))) {
+ regmap_write(data->regmap, SI1133_REG_IRQ_ENABLE, 0);
err = -ETIMEDOUT;
goto out;
}
@@ -427,6 +434,11 @@ static int si1133_command(struct si1133_data *data, u8 cmd)
dev_warn(dev,
"Failed to read command 0x%02x, ret=%d\n",
cmd, err);
+ /*
+ * Reset counter on err to prevent software and hardware
+ * counters being out of sync.
+ */
+ si1133_cmd_reset_counter(data);
goto out;
}
}
diff --git a/drivers/iio/light/tcs3472.c b/drivers/iio/light/tcs3472.c
index afc90b5bb0ec..7ed68daea690 100644
--- a/drivers/iio/light/tcs3472.c
+++ b/drivers/iio/light/tcs3472.c
@@ -441,6 +441,23 @@ static const struct iio_info tcs3472_info = {
.attrs = &tcs3472_attribute_group,
};
+static int tcs3472_powerdown(struct tcs3472_data *data)
+{
+ int ret;
+ u8 enable_mask = TCS3472_ENABLE_AEN | TCS3472_ENABLE_PON;
+
+ mutex_lock(&data->lock);
+
+ ret = i2c_smbus_write_byte_data(data->client, TCS3472_ENABLE,
+ data->enable & ~enable_mask);
+ if (!ret)
+ data->enable &= ~enable_mask;
+
+ mutex_unlock(&data->lock);
+
+ return ret;
+}
+
static int tcs3472_probe(struct i2c_client *client)
{
struct tcs3472_data *data;
@@ -514,7 +531,7 @@ static int tcs3472_probe(struct i2c_client *client)
ret = iio_triggered_buffer_setup(indio_dev, NULL,
tcs3472_trigger_handler, NULL);
if (ret < 0)
- return ret;
+ goto error_powerdown;
if (client->irq) {
ret = request_threaded_irq(client->irq, NULL,
@@ -537,23 +554,8 @@ static int tcs3472_probe(struct i2c_client *client)
free_irq(client->irq, indio_dev);
buffer_cleanup:
iio_triggered_buffer_cleanup(indio_dev);
- return ret;
-}
-
-static int tcs3472_powerdown(struct tcs3472_data *data)
-{
- int ret;
- u8 enable_mask = TCS3472_ENABLE_AEN | TCS3472_ENABLE_PON;
-
- mutex_lock(&data->lock);
-
- ret = i2c_smbus_write_byte_data(data->client, TCS3472_ENABLE,
- data->enable & ~enable_mask);
- if (!ret)
- data->enable &= ~enable_mask;
-
- mutex_unlock(&data->lock);
-
+error_powerdown:
+ tcs3472_powerdown(data);
return ret;
}
diff --git a/drivers/iio/magnetometer/ak8975.c b/drivers/iio/magnetometer/ak8975.c
index 783a3defa8e6..4beb4fe0ea7d 100644
--- a/drivers/iio/magnetometer/ak8975.c
+++ b/drivers/iio/magnetometer/ak8975.c
@@ -495,6 +495,10 @@ static int ak8975_who_i_am(struct i2c_client *client,
dev_err(&client->dev, "Error reading WIA\n");
return ret;
}
+ if (ret != sizeof(wia_val)) {
+ dev_err(&client->dev, "Error reading WIA\n");
+ return -EIO;
+ }
if (wia_val[0] != AK8975_DEVICE_ID)
return -ENODEV;
@@ -619,6 +623,10 @@ static int ak8975_setup(struct i2c_client *client)
dev_err(&client->dev, "Not able to read asa data\n");
return ret;
}
+ if (ret != sizeof(data->asa)) {
+ dev_err(&client->dev, "Error reading asa data\n");
+ return -EIO;
+ }
/* After reading fuse ROM data set power-down mode */
ret = ak8975_set_mode(data, POWER_DOWN);
@@ -758,6 +766,10 @@ static int ak8975_read_axis(struct iio_dev *indio_dev, int index, int *val)
sizeof(rval), (u8*)&rval);
if (ret < 0)
goto exit;
+ if (ret != sizeof(rval)) {
+ ret = -EIO;
+ goto exit;
+ }
/* Read out ST2 for release lock on measurment data. */
ret = i2c_smbus_read_byte_data(client, data->def->ctrl_regs[ST2]);
@@ -875,6 +887,8 @@ static void ak8975_fill_buffer(struct iio_dev *indio_dev)
(u8 *)fval);
if (ret < 0)
goto unlock;
+ if (ret != sizeof(fval))
+ goto unlock;
mutex_unlock(&data->lock);
diff --git a/drivers/iio/orientation/hid-sensor-rotation.c b/drivers/iio/orientation/hid-sensor-rotation.c
index 5e8cadd5177a..a0cf5d2a87fb 100644
--- a/drivers/iio/orientation/hid-sensor-rotation.c
+++ b/drivers/iio/orientation/hid-sensor-rotation.c
@@ -69,6 +69,13 @@ static int dev_rot_read_raw(struct iio_dev *indio_dev,
long mask)
{
struct dev_rot_state *rot_state = iio_priv(indio_dev);
+ struct hid_sensor_hub_device *hsdev = rot_state->common_attributes.hsdev;
+ struct hid_sensor_hub_attribute_info *info = &rot_state->quaternion;
+ u32 usage_id = HID_USAGE_SENSOR_ORIENT_QUATERNION;
+ union {
+ s16 val16[4];
+ s32 val32[4];
+ } raw_buf;
int ret_type;
int i;
@@ -78,8 +85,37 @@ static int dev_rot_read_raw(struct iio_dev *indio_dev,
switch (mask) {
case IIO_CHAN_INFO_RAW:
if (size >= 4) {
- for (i = 0; i < 4; ++i)
- vals[i] = rot_state->scan.sampled_vals[i];
+ if (info->size <= 0 || info->size > sizeof(raw_buf))
+ return -EINVAL;
+
+ hid_sensor_power_state(&rot_state->common_attributes, true);
+
+ ret_type = sensor_hub_input_attr_read_values(hsdev,
+ hsdev->usage,
+ usage_id,
+ info->report_id,
+ SENSOR_HUB_SYNC,
+ info->size,
+ (u8 *)&raw_buf);
+
+ hid_sensor_power_state(&rot_state->common_attributes, false);
+
+ if (ret_type < 0)
+ return ret_type;
+
+ switch (info->size) {
+ case sizeof(raw_buf.val16):
+ for (i = 0; i < ARRAY_SIZE(raw_buf.val16); i++)
+ vals[i] = raw_buf.val16[i];
+ break;
+ case sizeof(raw_buf.val32):
+ for (i = 0; i < ARRAY_SIZE(raw_buf.val32); i++)
+ vals[i] = raw_buf.val32[i];
+ break;
+ default:
+ return -EINVAL;
+ }
+
ret_type = IIO_VAL_INT_MULTIPLE;
*val_len = 4;
} else
diff --git a/drivers/iio/pressure/bmp280-core.c b/drivers/iio/pressure/bmp280-core.c
index 9e46aa65acef..088b13ac2d9d 100644
--- a/drivers/iio/pressure/bmp280-core.c
+++ b/drivers/iio/pressure/bmp280-core.c
@@ -747,7 +747,6 @@ static int bmp280_read_raw(struct iio_dev *indio_dev,
pm_runtime_get_sync(data->dev);
ret = bmp280_read_raw_impl(indio_dev, chan, val, val2, mask);
- pm_runtime_mark_last_busy(data->dev);
pm_runtime_put_autosuspend(data->dev);
return ret;
@@ -922,7 +921,6 @@ static int bmp280_write_raw(struct iio_dev *indio_dev,
pm_runtime_get_sync(data->dev);
ret = bmp280_write_raw_impl(indio_dev, chan, val, val2, mask);
- pm_runtime_mark_last_busy(data->dev);
pm_runtime_put_autosuspend(data->dev);
return ret;
@@ -1951,7 +1949,6 @@ static int bmp580_nvmem_read(void *priv, unsigned int offset, void *val,
pm_runtime_get_sync(data->dev);
ret = bmp580_nvmem_read_impl(priv, offset, val, bytes);
- pm_runtime_mark_last_busy(data->dev);
pm_runtime_put_autosuspend(data->dev);
return ret;
@@ -2026,7 +2023,6 @@ static int bmp580_nvmem_write(void *priv, unsigned int offset, void *val,
pm_runtime_get_sync(data->dev);
ret = bmp580_nvmem_write_impl(priv, offset, val, bytes);
- pm_runtime_mark_last_busy(data->dev);
pm_runtime_put_autosuspend(data->dev);
return ret;
@@ -2634,7 +2630,6 @@ static int bmp280_buffer_postdisable(struct iio_dev *indio_dev)
{
struct bmp280_data *data = iio_priv(indio_dev);
- pm_runtime_mark_last_busy(data->dev);
pm_runtime_put_autosuspend(data->dev);
return 0;
diff --git a/drivers/iio/pressure/icp10100.c b/drivers/iio/pressure/icp10100.c
index 3e0bf5d31ad7..ff6b36b5927c 100644
--- a/drivers/iio/pressure/icp10100.c
+++ b/drivers/iio/pressure/icp10100.c
@@ -265,7 +265,6 @@ static int icp10100_get_measures(struct icp10100_state *st,
(be16_to_cpu(measures[1]) >> 8);
*temperature = be16_to_cpu(measures[2]);
- pm_runtime_mark_last_busy(&st->client->dev);
error_measure:
pm_runtime_put_autosuspend(&st->client->dev);
return ret;
diff --git a/drivers/iio/pressure/mpl115.c b/drivers/iio/pressure/mpl115.c
index 02ea38c8a3e4..11ee879f4da4 100644
--- a/drivers/iio/pressure/mpl115.c
+++ b/drivers/iio/pressure/mpl115.c
@@ -106,20 +106,18 @@ static int mpl115_read_raw(struct iio_dev *indio_dev,
case IIO_CHAN_INFO_PROCESSED:
pm_runtime_get_sync(data->dev);
ret = mpl115_comp_pressure(data, val, val2);
+ pm_runtime_put_autosuspend(data->dev);
if (ret < 0)
return ret;
- pm_runtime_mark_last_busy(data->dev);
- pm_runtime_put_autosuspend(data->dev);
return IIO_VAL_INT_PLUS_MICRO;
case IIO_CHAN_INFO_RAW:
pm_runtime_get_sync(data->dev);
/* temperature -5.35 C / LSB, 472 LSB is 25 C */
ret = mpl115_read_temp(data);
+ pm_runtime_put_autosuspend(data->dev);
if (ret < 0)
return ret;
- pm_runtime_mark_last_busy(data->dev);
- pm_runtime_put_autosuspend(data->dev);
*val = ret >> 6;
return IIO_VAL_INT;
diff --git a/drivers/iio/pressure/zpa2326.c b/drivers/iio/pressure/zpa2326.c
index 8fae58db1d63..4146396755b6 100644
--- a/drivers/iio/pressure/zpa2326.c
+++ b/drivers/iio/pressure/zpa2326.c
@@ -699,7 +699,6 @@ static void zpa2326_suspend(struct iio_dev *indio_dev)
zpa2326_sleep(indio_dev);
- pm_runtime_mark_last_busy(parent);
pm_runtime_put_autosuspend(parent);
}
@@ -710,7 +709,6 @@ static void zpa2326_init_runtime(struct device *parent)
pm_runtime_enable(parent);
pm_runtime_set_autosuspend_delay(parent, 1000);
pm_runtime_use_autosuspend(parent);
- pm_runtime_mark_last_busy(parent);
pm_runtime_put_autosuspend(parent);
}
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index 50bb3c43f40b..8d0d77628d9c 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -514,6 +514,7 @@ static int cm_init_av_by_path(struct sa_path_rec *path,
struct rdma_ah_attr new_ah_attr;
struct cm_device *cm_dev;
struct cm_port *port;
+ u16 pkey_index;
int ret;
port = get_cm_port_from_path(path, sgid_attr);
@@ -522,12 +523,10 @@ static int cm_init_av_by_path(struct sa_path_rec *path,
cm_dev = port->cm_dev;
ret = ib_find_cached_pkey(cm_dev->ib_device, port->port_num,
- be16_to_cpu(path->pkey), &av->pkey_index);
+ be16_to_cpu(path->pkey), &pkey_index);
if (ret)
return ret;
- cm_set_av_port(av, port);
-
/*
* av->ah_attr might be initialized based on wc or during
* request processing time which might have reference to sgid_attr.
@@ -542,6 +541,8 @@ static int cm_init_av_by_path(struct sa_path_rec *path,
if (ret)
return ret;
+ av->pkey_index = pkey_index;
+ cm_set_av_port(av, port);
av->timeout = path->packet_life_time + 1;
rdma_move_ah_attr(&av->ah_attr, &new_ah_attr);
return 0;
@@ -2169,8 +2170,10 @@ static int cm_req_handler(struct cm_work *work)
cm_id_priv->av.ah_attr.roce.dmac);
work->path[0].hop_limit = grh->hop_limit;
- /* This destroy call is needed to pair with cm_init_av_for_response */
- cm_destroy_av(&cm_id_priv->av);
+ /*
+ * cm_init_av_by_path() will internally pair with the above
+ * cm_init_av_for_response() if it succeeds.
+ */
ret = cm_init_av_by_path(&work->path[0], gid_attr, &cm_id_priv->av);
if (ret) {
int err;
diff --git a/drivers/infiniband/core/counters.c b/drivers/infiniband/core/counters.c
index af59486fe418..d41af5ab6403 100644
--- a/drivers/infiniband/core/counters.c
+++ b/drivers/infiniband/core/counters.c
@@ -649,7 +649,7 @@ void rdma_counter_init(struct ib_device *dev)
fail:
for (i = port; i >= rdma_start_port(dev); i--) {
- port_counter = &dev->port_data[port].port_counter;
+ port_counter = &dev->port_data[i].port_counter;
rdma_free_hw_stats_struct(port_counter->hstats);
port_counter->hstats = NULL;
mutex_destroy(&port_counter->lock);
diff --git a/drivers/infiniband/hw/hns/hns_roce_cq.c b/drivers/infiniband/hw/hns/hns_roce_cq.c
index 6aa82fe9dd3d..150f5052d0b0 100644
--- a/drivers/infiniband/hw/hns/hns_roce_cq.c
+++ b/drivers/infiniband/hw/hns/hns_roce_cq.c
@@ -171,9 +171,9 @@ static int hns_roce_create_cqc(struct hns_roce_dev *hr_dev,
ret = hns_roce_create_hw_ctx(hr_dev, mailbox, HNS_ROCE_CMD_CREATE_CQC,
hr_cq->cqn);
if (ret)
- ibdev_err(ibdev,
- "failed to send create cmd for CQ(0x%lx), ret = %d.\n",
- hr_cq->cqn, ret);
+ ibdev_err_ratelimited(ibdev,
+ "failed to send create cmd for CQ(0x%lx), ret = %d.\n",
+ hr_cq->cqn, ret);
hns_roce_free_cmd_mailbox(hr_dev, mailbox);
diff --git a/drivers/infiniband/hw/hns/hns_roce_debugfs.c b/drivers/infiniband/hw/hns/hns_roce_debugfs.c
index b869cdc54118..db32c5897640 100644
--- a/drivers/infiniband/hw/hns/hns_roce_debugfs.c
+++ b/drivers/infiniband/hw/hns/hns_roce_debugfs.c
@@ -26,17 +26,6 @@ static const struct file_operations hns_debugfs_seqfile_fops = {
.llseek = seq_lseek
};
-static void init_debugfs_seqfile(struct hns_debugfs_seqfile *seq,
- const char *name, struct dentry *parent,
- int (*read_fn)(struct seq_file *, void *),
- void *data)
-{
- debugfs_create_file(name, 0400, parent, seq, &hns_debugfs_seqfile_fops);
-
- seq->read = read_fn;
- seq->data = data;
-}
-
static const char * const sw_stat_info[] = {
[HNS_ROCE_DFX_AEQE_CNT] = "aeqe",
[HNS_ROCE_DFX_CEQE_CNT] = "ceqe",
@@ -76,10 +65,12 @@ static void create_sw_stat_debugfs(struct hns_roce_dev *hr_dev,
{
struct hns_sw_stat_debugfs *dbgfs = &hr_dev->dbgfs.sw_stat_root;
- dbgfs->root = debugfs_create_dir("sw_stat", parent);
+ dbgfs->sw_stat.read = sw_stat_debugfs_show;
+ dbgfs->sw_stat.data = hr_dev;
- init_debugfs_seqfile(&dbgfs->sw_stat, "sw_stat", dbgfs->root,
- sw_stat_debugfs_show, hr_dev);
+ dbgfs->root = debugfs_create_dir("sw_stat", parent);
+ debugfs_create_file("sw_stat", 0400, dbgfs->root, &dbgfs->sw_stat,
+ &hns_debugfs_seqfile_fops);
}
/* debugfs for device */
diff --git a/drivers/infiniband/hw/hns/hns_roce_hem.c b/drivers/infiniband/hw/hns/hns_roce_hem.c
index 3d479c63b117..1680d0ac071e 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hem.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hem.c
@@ -314,14 +314,14 @@ static int calc_hem_config(struct hns_roce_dev *hr_dev,
bt_num = hns_roce_get_bt_num(table->type, mhop->hop_num);
switch (bt_num) {
case 3:
- index->l1 = l0_idx * chunk_ba_num + l1_idx;
+ index->l1 = (u64)l0_idx * chunk_ba_num + l1_idx;
index->l0 = l0_idx;
- index->buf = l0_idx * chunk_ba_num * chunk_ba_num +
- l1_idx * chunk_ba_num + l2_idx;
+ index->buf = (u64)l0_idx * chunk_ba_num * chunk_ba_num +
+ (u64)l1_idx * chunk_ba_num + l2_idx;
break;
case 2:
index->l0 = l0_idx;
- index->buf = l0_idx * chunk_ba_num + l1_idx;
+ index->buf = (u64)l0_idx * chunk_ba_num + l1_idx;
break;
case 1:
index->buf = l0_idx;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 5e1ea6335c11..a54f86468b05 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -5972,9 +5972,9 @@ static int hns_roce_v2_modify_srq(struct ib_srq *ibsrq,
HNS_ROCE_CMD_MODIFY_SRQC, srq->srqn);
hns_roce_free_cmd_mailbox(hr_dev, mailbox);
if (ret)
- ibdev_err(&hr_dev->ib_dev,
- "failed to handle cmd of modifying SRQ, ret = %d.\n",
- ret);
+ ibdev_err_ratelimited(&hr_dev->ib_dev,
+ "failed to handle cmd of modifying SRQ, ret = %d.\n",
+ ret);
}
out:
@@ -6000,9 +6000,9 @@ static int hns_roce_v2_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
ret = hns_roce_cmd_mbox(hr_dev, 0, mailbox->dma,
HNS_ROCE_CMD_QUERY_SRQC, srq->srqn);
if (ret) {
- ibdev_err(&hr_dev->ib_dev,
- "failed to process cmd of querying SRQ, ret = %d.\n",
- ret);
+ ibdev_err_ratelimited(&hr_dev->ib_dev,
+ "failed to process cmd of querying SRQ, ret = %d.\n",
+ ret);
goto out;
}
@@ -6108,9 +6108,9 @@ static int hns_roce_v2_query_mpt(struct hns_roce_dev *hr_dev, u32 key,
ret = hns_roce_cmd_mbox(hr_dev, 0, mailbox->dma, HNS_ROCE_CMD_QUERY_MPT,
key_to_hw_index(key));
if (ret) {
- ibdev_err(&hr_dev->ib_dev,
- "failed to process cmd when querying MPT, ret = %d.\n",
- ret);
+ ibdev_err_ratelimited(&hr_dev->ib_dev,
+ "failed to process cmd when querying MPT, ret = %d.\n",
+ ret);
goto err_mailbox;
}
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index ed9b5e5778ed..63fed96379ba 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -1013,7 +1013,7 @@ static void check_and_get_armed_cq(struct list_head *cq_list, struct ib_cq *cq)
unsigned long flags;
spin_lock_irqsave(&hr_cq->lock, flags);
- if (cq->comp_handler) {
+ if (cq->comp_handler && hr_cq->ib_cq.poll_ctx != IB_POLL_DIRECT) {
if (!hr_cq->is_armed) {
hr_cq->is_armed = 1;
list_add_tail(&hr_cq->node, cq_list);
diff --git a/drivers/infiniband/hw/hns/hns_roce_mr.c b/drivers/infiniband/hw/hns/hns_roce_mr.c
index f86859ffb413..4ce9b46eb6bd 100644
--- a/drivers/infiniband/hw/hns/hns_roce_mr.c
+++ b/drivers/infiniband/hw/hns/hns_roce_mr.c
@@ -171,7 +171,7 @@ static int hns_roce_mr_enable(struct hns_roce_dev *hr_dev,
ret = hns_roce_create_hw_ctx(hr_dev, mailbox, HNS_ROCE_CMD_CREATE_MPT,
mtpt_idx & (hr_dev->caps.num_mtpts - 1));
if (ret) {
- dev_err(dev, "failed to create mpt, ret = %d.\n", ret);
+ dev_err_ratelimited(dev, "failed to create mpt, ret = %d.\n", ret);
goto err_page;
}
@@ -311,7 +311,7 @@ struct ib_mr *hns_roce_rereg_user_mr(struct ib_mr *ibmr, int flags, u64 start,
ret = hns_roce_destroy_hw_ctx(hr_dev, HNS_ROCE_CMD_DESTROY_MPT,
mtpt_idx);
if (ret)
- ibdev_warn(ib_dev, "failed to destroy MPT, ret = %d.\n", ret);
+ ibdev_warn_ratelimited(ib_dev, "failed to destroy MPT, ret = %d.\n", ret);
mr->enabled = 0;
mr->iova = virt_addr;
@@ -342,7 +342,7 @@ struct ib_mr *hns_roce_rereg_user_mr(struct ib_mr *ibmr, int flags, u64 start,
ret = hns_roce_create_hw_ctx(hr_dev, mailbox, HNS_ROCE_CMD_CREATE_MPT,
mtpt_idx);
if (ret) {
- ibdev_err(ib_dev, "failed to create MPT, ret = %d.\n", ret);
+ ibdev_err_ratelimited(ib_dev, "failed to create MPT, ret = %d.\n", ret);
goto free_cmd_mbox;
}
diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c
index 70c06ef65603..314a879f1c5e 100644
--- a/drivers/infiniband/hw/hns/hns_roce_srq.c
+++ b/drivers/infiniband/hw/hns/hns_roce_srq.c
@@ -104,7 +104,7 @@ static int hns_roce_create_srqc(struct hns_roce_dev *hr_dev,
ret = hns_roce_create_hw_ctx(hr_dev, mailbox, HNS_ROCE_CMD_CREATE_SRQ,
srq->srqn);
if (ret)
- ibdev_err(ibdev, "failed to config SRQC, ret = %d.\n", ret);
+ ibdev_err_ratelimited(ibdev, "failed to config SRQC, ret = %d.\n", ret);
err_mbox:
hns_roce_free_cmd_mailbox(hr_dev, mailbox);
diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c
index 04015eb3663d..92b56338819e 100644
--- a/drivers/infiniband/hw/irdma/verbs.c
+++ b/drivers/infiniband/hw/irdma/verbs.c
@@ -2134,7 +2134,6 @@ static int irdma_create_cq(struct ib_cq *ibcq,
}
cqmr_shadow = &iwpbl_shadow->cq_mr;
info.shadow_area_pa = cqmr_shadow->cq_pbl.addr;
- cqmr->split = true;
} else {
info.shadow_area_pa = cqmr->shadow;
}
@@ -2327,10 +2326,11 @@ static inline u64 *irdma_next_pbl_addr(u64 *pbl, struct irdma_pble_info **pinfo,
* irdma_copy_user_pgaddrs - copy user page address to pble's os locally
* @iwmr: iwmr for IB's user page addresses
* @pbl: ple pointer to save 1 level or 0 level pble
+ * @pbl_len: Max number of PBL entries to populate
* @level: indicated level 0, 1 or 2
*/
static void irdma_copy_user_pgaddrs(struct irdma_mr *iwmr, u64 *pbl,
- enum irdma_pble_level level)
+ u32 pbl_len, enum irdma_pble_level level)
{
struct ib_umem *region = iwmr->region;
struct irdma_pbl *iwpbl = &iwmr->iwpbl;
@@ -2338,7 +2338,9 @@ static void irdma_copy_user_pgaddrs(struct irdma_mr *iwmr, u64 *pbl,
struct irdma_pble_info *pinfo;
struct ib_block_iter biter;
u32 idx = 0;
- u32 pbl_cnt = 0;
+
+ if (!pbl_len)
+ return;
pinfo = (level == PBLE_LEVEL_1) ? NULL : palloc->level2.leaf;
@@ -2347,7 +2349,7 @@ static void irdma_copy_user_pgaddrs(struct irdma_mr *iwmr, u64 *pbl,
rdma_umem_for_each_dma_block(region, &biter, iwmr->page_size) {
*pbl = rdma_block_iter_dma_address(&biter);
- if (++pbl_cnt == palloc->total_cnt)
+ if (!--pbl_len)
break;
pbl = irdma_next_pbl_addr(pbl, &pinfo, &idx);
}
@@ -2423,6 +2425,7 @@ static int irdma_setup_pbles(struct irdma_pci_f *rf, struct irdma_mr *iwmr,
u64 *pbl;
int status;
enum irdma_pble_level level = PBLE_LEVEL_1;
+ u32 pbl_len;
if (lvl) {
status = irdma_get_pble(rf->pble_rsrc, palloc, iwmr->page_cnt,
@@ -2430,16 +2433,18 @@ static int irdma_setup_pbles(struct irdma_pci_f *rf, struct irdma_mr *iwmr,
if (status)
return status;
+ pbl_len = palloc->total_cnt;
iwpbl->pbl_allocated = true;
level = palloc->level;
pinfo = (level == PBLE_LEVEL_1) ? &palloc->level1 :
palloc->level2.leaf;
pbl = pinfo->addr;
} else {
+ pbl_len = IRDMA_MAX_SAVED_PHY_PGADDR;
pbl = iwmr->pgaddrmem;
}
- irdma_copy_user_pgaddrs(iwmr, pbl, level);
+ irdma_copy_user_pgaddrs(iwmr, pbl, pbl_len, level);
if (lvl)
iwmr->pgaddrmem[0] = *pbl;
@@ -2504,7 +2509,8 @@ static int irdma_handle_q_mem(struct irdma_device *iwdev,
case IRDMA_MEMREG_TYPE_CQ:
hmc_p = &cqmr->cq_pbl;
- if (!cqmr->split)
+ if (!(iwdev->rf->sc_dev.hw_attrs.uk_attrs.feature_flags &
+ IRDMA_FEATURE_CQ_RESIZE))
cqmr->shadow = (dma_addr_t)arr[req->cq_pages];
if (lvl)
@@ -2848,6 +2854,7 @@ static int irdma_reg_user_mr_type_mem(struct irdma_mr *iwmr, int access,
int err;
lvl = iwmr->page_cnt != 1 ? PBLE_LEVEL_1 | PBLE_LEVEL_2 : PBLE_LEVEL_0;
+ iwmr->access = access;
err = irdma_setup_pbles(iwdev->rf, iwmr, lvl);
if (err)
diff --git a/drivers/infiniband/hw/irdma/verbs.h b/drivers/infiniband/hw/irdma/verbs.h
index cbd8bef68ae4..f27ca72695e5 100644
--- a/drivers/infiniband/hw/irdma/verbs.h
+++ b/drivers/infiniband/hw/irdma/verbs.h
@@ -62,7 +62,6 @@ struct irdma_hmc_pble {
struct irdma_cq_mr {
struct irdma_hmc_pble cq_pbl;
dma_addr_t shadow;
- bool split;
};
struct irdma_qp_mr {
diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index f6bf289041bf..ebc9648ca0ac 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -592,8 +592,7 @@ int mana_ib_query_port(struct ib_device *ibdev, u32 port,
props->phys_state = IB_PORT_PHYS_STATE_DISABLED;
}
- props->active_width = IB_WIDTH_4X;
- props->active_speed = IB_SPEED_EDR;
+ ib_get_eth_speed(ibdev, port, &props->active_speed, &props->active_width);
props->pkey_tbl_len = 1;
if (port == 1)
props->gid_tbl_len = 16;
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index b52bceff7d97..09c4066fa160 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -137,6 +137,7 @@ struct mlx4_ib_mr {
dma_addr_t page_map;
u32 npages;
u32 max_pages;
+ int access_flags;
struct mlx4_mr mmr;
struct ib_umem *umem;
size_t page_map_size;
diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
index e331c04967df..1f2f924b5c30 100644
--- a/drivers/infiniband/hw/mlx4/mr.c
+++ b/drivers/infiniband/hw/mlx4/mr.c
@@ -439,6 +439,7 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
if (err)
goto err_mr;
+ mr->access_flags = access_flags;
mr->ibmr.rkey = mr->ibmr.lkey = mr->mmr.key;
mr->ibmr.page_size = 1U << shift;
@@ -499,6 +500,8 @@ struct ib_mr *mlx4_ib_rereg_user_mr(struct ib_mr *mr, int flags, u64 start,
if (err)
goto release_mpt_entry;
+ } else {
+ mr_access_flags = mmr->access_flags;
}
if (flags & IB_MR_REREG_TRANS) {
@@ -540,8 +543,10 @@ struct ib_mr *mlx4_ib_rereg_user_mr(struct ib_mr *mr, int flags, u64 start,
* return a failure. But dereg_mr will free the resources.
*/
err = mlx4_mr_hw_write_mpt(dev->dev, &mmr->mmr, pmpt_entry);
- if (!err && flags & IB_MR_REREG_ACCESS)
- mmr->mmr.access = mr_access_flags;
+ if (!err && flags & IB_MR_REREG_ACCESS) {
+ mmr->access_flags = mr_access_flags;
+ mmr->mmr.access = convert_access(mr_access_flags);
+ }
release_mpt_entry:
mlx4_mr_hw_put_mpt(dev->dev, pmpt_entry);
diff --git a/drivers/infiniband/hw/mlx4/sysfs.c b/drivers/infiniband/hw/mlx4/sysfs.c
index 88f534cf690e..1676f819c2f7 100644
--- a/drivers/infiniband/hw/mlx4/sysfs.c
+++ b/drivers/infiniband/hw/mlx4/sysfs.c
@@ -638,12 +638,6 @@ static int add_port(struct mlx4_ib_dev *dev, int port_num, int slave)
p->port_num = port_num;
p->slave = slave;
- ret = kobject_init_and_add(&p->kobj, &port_type,
- kobject_get(dev->dev_ports_parent[slave]),
- "%d", port_num);
- if (ret)
- goto err_alloc;
-
p->pkey_group.name = "pkey_idx";
p->pkey_group.attrs =
alloc_group_attrs(show_port_pkey,
@@ -651,13 +645,9 @@ static int add_port(struct mlx4_ib_dev *dev, int port_num, int slave)
dev->dev->caps.pkey_table_len[port_num]);
if (!p->pkey_group.attrs) {
ret = -ENOMEM;
- goto err_alloc;
+ goto err_free_port;
}
- ret = sysfs_create_group(&p->kobj, &p->pkey_group);
- if (ret)
- goto err_free_pkey;
-
p->gid_group.name = "gid_idx";
p->gid_group.attrs = alloc_group_attrs(show_port_gid_idx, NULL, 1);
if (!p->gid_group.attrs) {
@@ -665,28 +655,47 @@ static int add_port(struct mlx4_ib_dev *dev, int port_num, int slave)
goto err_free_pkey;
}
+ ret = kobject_init_and_add(&p->kobj, &port_type,
+ kobject_get(dev->dev_ports_parent[slave]),
+ "%d", port_num);
+ if (ret)
+ goto err_put;
+
+ ret = sysfs_create_group(&p->kobj, &p->pkey_group);
+ if (ret)
+ goto err_del;
+
ret = sysfs_create_group(&p->kobj, &p->gid_group);
if (ret)
- goto err_free_gid;
+ goto err_remove_pkey;
ret = add_vf_smi_entries(p);
if (ret)
- goto err_free_gid;
+ goto err_remove_gid;
list_add_tail(&p->kobj.entry, &dev->pkeys.pkey_port_list[slave]);
return 0;
-err_free_gid:
- kfree(p->gid_group.attrs[0]);
- kfree(p->gid_group.attrs);
+err_remove_gid:
+ sysfs_remove_group(&p->kobj, &p->gid_group);
+
+err_remove_pkey:
+ sysfs_remove_group(&p->kobj, &p->pkey_group);
+
+err_del:
+ kobject_del(&p->kobj);
+
+err_put:
+ kobject_put(dev->dev_ports_parent[slave]);
+ kobject_put(&p->kobj);
+ return ret;
err_free_pkey:
for (i = 0; i < dev->dev->caps.pkey_table_len[port_num]; ++i)
kfree(p->pkey_group.attrs[i]);
kfree(p->pkey_group.attrs);
-err_alloc:
- kobject_put(dev->dev_ports_parent[slave]);
+err_free_port:
kfree(p);
return ret;
}
diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index e74273bc078b..cdf1e2eb6ca9 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -1851,6 +1851,17 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_DEVX_OBJ_ASYNC_QUERY)(
return err;
}
+static bool devx_key_in_sub_list(struct list_head *list, u32 key_level1)
+{
+ struct devx_event_subscription *s;
+
+ list_for_each_entry(s, list, event_list)
+ if (s->xa_key_level1 == key_level1)
+ return true;
+
+ return false;
+}
+
static void
subscribe_event_xa_dealloc(struct mlx5_devx_event_table *devx_event_table,
u32 key_level1,
@@ -2098,10 +2109,17 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_DEVX_SUBSCRIBE_EVENT)(
event_sub = kzalloc(sizeof(*event_sub), GFP_KERNEL);
if (!event_sub) {
+ if (!devx_key_in_sub_list(&sub_list, key_level1))
+ subscribe_event_xa_dealloc(devx_event_table,
+ key_level1,
+ obj,
+ obj_id);
err = -ENOMEM;
goto err;
}
+ event_sub->ev_file = ev_file;
+ event_sub->xa_key_level1 = key_level1;
list_add_tail(&event_sub->event_list, &sub_list);
uverbs_uobject_get(&ev_file->uobj);
if (use_eventfd) {
@@ -2116,9 +2134,6 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_DEVX_SUBSCRIBE_EVENT)(
}
event_sub->cookie = cookie;
- event_sub->ev_file = ev_file;
- /* May be needed upon cleanup the devx object/subscription */
- event_sub->xa_key_level1 = key_level1;
event_sub->xa_key_level2 = obj_id;
INIT_LIST_HEAD(&event_sub->obj_list);
}
@@ -2163,10 +2178,11 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_DEVX_SUBSCRIBE_EVENT)(
list_for_each_entry_safe(event_sub, tmp_sub, &sub_list, event_list) {
list_del(&event_sub->event_list);
- subscribe_event_xa_dealloc(devx_event_table,
- event_sub->xa_key_level1,
- obj,
- obj_id);
+ if (!devx_key_in_sub_list(&sub_list, event_sub->xa_key_level1))
+ subscribe_event_xa_dealloc(devx_event_table,
+ event_sub->xa_key_level1,
+ obj,
+ obj_id);
if (event_sub->eventfd)
eventfd_ctx_put(event_sub->eventfd);
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index b0a0e11ef721..edd38510c025 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -1915,9 +1915,13 @@ static int mlx5_ib_alloc_transport_domain(struct mlx5_ib_dev *dev, u32 *tdn,
if ((MLX5_CAP_GEN(dev->mdev, port_type) != MLX5_CAP_PORT_TYPE_ETH) ||
(!MLX5_CAP_GEN(dev->mdev, disable_local_lb_uc) &&
!MLX5_CAP_GEN(dev->mdev, disable_local_lb_mc)))
- return err;
+ return 0;
+
+ err = mlx5_ib_enable_lb(dev, true, false);
+ if (err)
+ mlx5_cmd_dealloc_transport_domain(dev->mdev, *tdn, uid);
- return mlx5_ib_enable_lb(dev, true, false);
+ return err;
}
static void mlx5_ib_dealloc_transport_domain(struct mlx5_ib_dev *dev, u32 tdn,
@@ -2408,7 +2412,7 @@ static int uar_mmap(struct mlx5_ib_dev *dev, enum mlx5_ib_mmap_cmd cmd,
if (!dyn_uar)
return err;
- mlx5_cmd_uar_dealloc(dev->mdev, idx, context->devx_uid);
+ mlx5_cmd_uar_dealloc(dev->mdev, uar_index, context->devx_uid);
free_bfreg:
mlx5_ib_free_bfreg(dev, bfregi, bfreg_dyn_idx);
@@ -4164,6 +4168,8 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev)
dev->port[i].roce.last_port_state = IB_PORT_DOWN;
}
+ mutex_init(&dev->lb.mutex);
+
err = mlx5r_cmd_query_special_mkeys(dev);
if (err)
return err;
@@ -4378,11 +4384,6 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev)
if (err)
return err;
- if ((MLX5_CAP_GEN(dev->mdev, port_type) == MLX5_CAP_PORT_TYPE_ETH) &&
- (MLX5_CAP_GEN(dev->mdev, disable_local_lb_uc) ||
- MLX5_CAP_GEN(dev->mdev, disable_local_lb_mc)))
- mutex_init(&dev->lb.mutex);
-
if (MLX5_CAP_GEN_64(dev->mdev, general_obj_types) &
MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q) {
err = mlx5_ib_init_var_table(dev);
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index c8238db895bf..76d2238ea5e7 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1838,6 +1838,21 @@ struct ib_mr *mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start,
if (!(flags & IB_MR_REREG_PD))
new_pd = ib_mr->pd;
+ if (mr->is_odp_implicit && !(flags & IB_MR_REREG_TRANS)) {
+ if (!(new_access_flags & IB_ACCESS_ON_DEMAND))
+ return ERR_PTR(-EOPNOTSUPP);
+
+ /*
+ * Due to all the child mkeys we cannot actually change an
+ * implicit MR in place. If the user did not specify a new
+ * translation then force the fixed implicit MR values.
+ */
+ start = 0;
+ iova = 0;
+ length = U64_MAX;
+ flags |= IB_MR_REREG_TRANS;
+ }
+
if (!(flags & IB_MR_REREG_TRANS)) {
struct ib_umem *umem;
@@ -1852,7 +1867,7 @@ struct ib_mr *mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start,
}
/* DM or ODP MR's don't have a normal umem so we can't re-use it */
if (!mr->umem || is_odp_mr(mr) || is_dmabuf_mr(mr))
- goto recreate;
+ return ERR_PTR(-EOPNOTSUPP);
/*
* Only one active MR can refer to a umem at one time, revoke
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index ded139b4e87a..badec2e33972 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -451,16 +451,13 @@ static int set_rq_size(struct mlx5_ib_dev *dev, struct ib_qp_cap *cap,
if (ucmd) {
qp->rq.wqe_cnt = ucmd->rq_wqe_count;
- if (ucmd->rq_wqe_shift > BITS_PER_BYTE * sizeof(ucmd->rq_wqe_shift))
- return -EINVAL;
qp->rq.wqe_shift = ucmd->rq_wqe_shift;
- if ((1 << qp->rq.wqe_shift) /
- sizeof(struct mlx5_wqe_data_seg) <
- wq_sig)
+ if (check_shl_overflow(1, qp->rq.wqe_shift, &wqe_size))
+ return -EINVAL;
+ if (wqe_size / sizeof(struct mlx5_wqe_data_seg) < wq_sig)
return -EINVAL;
qp->rq.max_gs =
- (1 << qp->rq.wqe_shift) /
- sizeof(struct mlx5_wqe_data_seg) -
+ wqe_size / sizeof(struct mlx5_wqe_data_seg) -
wq_sig;
qp->rq.max_post = qp->rq.wqe_cnt;
} else {
@@ -3078,12 +3075,14 @@ static int create_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd,
int err;
if (params->is_rss_raw) {
+ rdma_restrack_no_track(&qp->ibqp.res);
err = create_rss_raw_qp_tir(dev, pd, qp, params);
goto out;
}
switch (qp->type) {
case MLX5_IB_QPT_DCT:
+ rdma_restrack_no_track(&qp->ibqp.res);
err = create_dct(dev, pd, qp, params);
break;
case MLX5_IB_QPT_DCI:
diff --git a/drivers/infiniband/hw/mlx5/restrack.c b/drivers/infiniband/hw/mlx5/restrack.c
index affcf8fe943c..76077fdfc8a9 100644
--- a/drivers/infiniband/hw/mlx5/restrack.c
+++ b/drivers/infiniband/hw/mlx5/restrack.c
@@ -169,9 +169,6 @@ static int fill_res_qp_entry(struct sk_buff *msg, struct ib_qp *ibqp)
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_RES_SUBTYPE,
"REG_UMR");
break;
- case MLX5_IB_QPT_DCT:
- ret = nla_put_string(msg, RDMA_NLDEV_ATTR_RES_SUBTYPE, "DCT");
- break;
case MLX5_IB_QPT_DCI:
ret = nla_put_string(msg, RDMA_NLDEV_ATTR_RES_SUBTYPE, "DCI");
break;
diff --git a/drivers/infiniband/sw/rxe/rxe_mmap.c b/drivers/infiniband/sw/rxe/rxe_mmap.c
index 6b7f2bd69879..466ab6dbe9af 100644
--- a/drivers/infiniband/sw/rxe/rxe_mmap.c
+++ b/drivers/infiniband/sw/rxe/rxe_mmap.c
@@ -93,18 +93,31 @@ int rxe_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
goto done;
found_it:
+ /*
+ * Increment refcount and check whether it is being freed atm while
+ * holding lock to prevent UAF
+ */
+ if (!kref_get_unless_zero(&ip->ref)) {
+ spin_unlock_bh(&rxe->pending_lock);
+ ret = -ENXIO;
+ goto done;
+ }
+
list_del_init(&ip->pending_mmaps);
spin_unlock_bh(&rxe->pending_lock);
+ vma->vm_ops = &rxe_vm_ops;
+ vma->vm_private_data = ip;
+
ret = remap_vmalloc_range(vma, ip->obj, 0);
if (ret) {
+ vma->vm_private_data = NULL;
+ vma->vm_ops = NULL;
+ kref_put(&ip->ref, rxe_mmap_release);
rxe_dbg_dev(rxe, "err %d from remap_vmalloc_range\n", ret);
goto done;
}
- vma->vm_ops = &rxe_vm_ops;
- vma->vm_private_data = ip;
- rxe_vma_open(vma);
done:
return ret;
}
diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c
index 29e32b91d74b..5611f63a1115 100644
--- a/drivers/infiniband/sw/rxe/rxe_resp.c
+++ b/drivers/infiniband/sw/rxe/rxe_resp.c
@@ -263,6 +263,7 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
struct rxe_recv_wqe *wqe;
struct ib_event ev;
unsigned int count;
+ unsigned int num_sge;
size_t size;
unsigned long flags;
@@ -278,12 +279,13 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
}
/* don't trust user space data */
- if (unlikely(wqe->dma.num_sge > srq->rq.max_sge)) {
+ num_sge = wqe->dma.num_sge;
+ if (unlikely(num_sge > srq->rq.max_sge)) {
spin_unlock_irqrestore(&srq->rq.consumer_lock, flags);
rxe_dbg_qp(qp, "invalid num_sge in SRQ entry\n");
return RESPST_ERR_MALFORMED_WQE;
}
- size = sizeof(*wqe) + wqe->dma.num_sge*sizeof(struct rxe_sge);
+ size = sizeof(*wqe) + num_sge * sizeof(struct rxe_sge);
memcpy(&qp->resp.srq_wqe, wqe, size);
qp->resp.wqe = &qp->resp.srq_wqe.wqe;
@@ -307,6 +309,29 @@ static enum resp_states get_srq_wqe(struct rxe_qp *qp)
return RESPST_CHK_LENGTH;
}
+static enum resp_states rxe_get_recv_wqe(struct rxe_qp *qp)
+{
+ struct rxe_queue *q = qp->rq.queue;
+ struct rxe_recv_wqe *wqe;
+ unsigned int num_sge;
+ size_t size;
+
+ wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT);
+ if (!wqe)
+ return RESPST_ERR_RNR;
+
+ num_sge = wqe->dma.num_sge;
+ if (unlikely(num_sge > qp->rq.max_sge)) {
+ rxe_dbg_qp(qp, "invalid num_sge in recv WQE\n");
+ return RESPST_ERR_MALFORMED_WQE;
+ }
+ size = sizeof(*wqe) + num_sge * sizeof(struct rxe_sge);
+ memcpy(&qp->resp.srq_wqe, wqe, size);
+
+ qp->resp.wqe = &qp->resp.srq_wqe.wqe;
+ return RESPST_CHK_LENGTH;
+}
+
static enum resp_states check_resource(struct rxe_qp *qp,
struct rxe_pkt_info *pkt)
{
@@ -327,9 +352,7 @@ static enum resp_states check_resource(struct rxe_qp *qp,
if (srq)
return get_srq_wqe(qp);
- qp->resp.wqe = queue_head(qp->rq.queue,
- QUEUE_TYPE_FROM_CLIENT);
- return (qp->resp.wqe) ? RESPST_CHK_LENGTH : RESPST_ERR_RNR;
+ return rxe_get_recv_wqe(qp);
}
return RESPST_CHK_LENGTH;
diff --git a/drivers/infiniband/sw/siw/siw_cm.c b/drivers/infiniband/sw/siw/siw_cm.c
index 708b13993fdf..49ff121f77fe 100644
--- a/drivers/infiniband/sw/siw/siw_cm.c
+++ b/drivers/infiniband/sw/siw/siw_cm.c
@@ -89,6 +89,7 @@ static void siw_socket_disassoc(struct socket *s)
cep = sk_to_cep(sk);
if (cep) {
siw_sk_restore_upcalls(sk, cep);
+ cep->sock = NULL;
siw_cep_put(cep);
} else {
pr_warn("siw: cannot restore sk callbacks: no ep\n");
@@ -369,10 +370,11 @@ static void siw_free_cm_id(struct siw_cep *cep)
static void siw_destroy_cep_sock(struct siw_cep *cep)
{
- if (cep->sock) {
- siw_socket_disassoc(cep->sock);
- sock_release(cep->sock);
- cep->sock = NULL;
+ struct socket *s = cep->sock;
+
+ if (s) {
+ siw_socket_disassoc(s);
+ sock_release(s);
}
}
@@ -1001,7 +1003,6 @@ static void siw_accept_newconn(struct siw_cep *cep)
if (new_s) {
siw_socket_disassoc(new_s);
sock_release(new_s);
- new_cep->sock = NULL;
}
siw_dbg_cep(cep, "error %d\n", rv);
}
@@ -1153,6 +1154,8 @@ static void siw_cm_work_handler(struct work_struct *w)
WARN(1, "Undefined CM work type: %d\n", work->type);
}
if (release_cep) {
+ struct socket *s = cep->sock;
+
siw_dbg_cep(cep,
"release: timer=%s, QP[%u]\n",
cep->mpa_timer ? "y" : "n",
@@ -1178,10 +1181,9 @@ static void siw_cm_work_handler(struct work_struct *w)
cep->qp = NULL;
siw_qp_put(qp);
}
- if (cep->sock) {
- siw_socket_disassoc(cep->sock);
- sock_release(cep->sock);
- cep->sock = NULL;
+ if (s) {
+ siw_socket_disassoc(s);
+ sock_release(s);
}
if (cep->cm_id) {
siw_free_cm_id(cep);
@@ -1511,7 +1513,6 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
if (cep) {
siw_socket_disassoc(s);
sock_release(s);
- cep->sock = NULL;
cep->qp = NULL;
@@ -1886,7 +1887,6 @@ int siw_create_listen(struct iw_cm_id *id, int backlog)
siw_cep_set_inuse(cep);
siw_free_cm_id(cep);
- cep->sock = NULL;
siw_socket_disassoc(s);
cep->state = SIW_EPSTATE_CLOSED;
@@ -1908,6 +1908,7 @@ static void siw_drop_listeners(struct iw_cm_id *id)
*/
list_for_each_safe(p, tmp, (struct list_head *)id->provider_data) {
struct siw_cep *cep = list_entry(p, struct siw_cep, listenq);
+ struct socket *s = cep->sock;
list_del(p);
@@ -1916,10 +1917,9 @@ static void siw_drop_listeners(struct iw_cm_id *id)
siw_cep_set_inuse(cep);
siw_free_cm_id(cep);
- if (cep->sock) {
- siw_socket_disassoc(cep->sock);
- sock_release(cep->sock);
- cep->sock = NULL;
+ if (s) {
+ siw_socket_disassoc(s);
+ sock_release(s);
}
cep->state = SIW_EPSTATE_CLOSED;
siw_cep_set_free_and_put(cep);
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 5dfb4644446b..128e4e836470 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -1129,9 +1129,10 @@ static int srpt_get_desc_tbl(struct srpt_recv_ioctx *recv_ioctx,
struct srp_imm_buf *imm_buf = srpt_get_desc_buf(srp_cmd);
void *data = (void *)srp_cmd + imm_data_offset;
uint32_t len = be32_to_cpu(imm_buf->len);
- uint32_t req_size = imm_data_offset + len;
+ uint32_t req_size;
- if (req_size > srp_max_req_size) {
+ if (check_add_overflow((uint32_t)imm_data_offset, len, &req_size) ||
+ req_size > srp_max_req_size) {
pr_err("Immediate data (length %d + %d) exceeds request size %d\n",
imm_data_offset, len, srp_max_req_size);
return -EINVAL;
diff --git a/drivers/input/misc/ims-pcu.c b/drivers/input/misc/ims-pcu.c
index bcbaefe03e47..4073e66e4d62 100644
--- a/drivers/input/misc/ims-pcu.c
+++ b/drivers/input/misc/ims-pcu.c
@@ -14,6 +14,7 @@
#include <linux/leds.h>
#include <linux/module.h>
#include <linux/slab.h>
+#include <linux/sysfs.h>
#include <linux/types.h>
#include <linux/usb/input.h>
#include <linux/usb/cdc.h>
@@ -406,7 +407,16 @@ static void ims_pcu_destroy_gamepad(struct ims_pcu *pcu)
static void ims_pcu_report_events(struct ims_pcu *pcu)
{
- u32 data = get_unaligned_be32(&pcu->read_buf[3]);
+ u32 data;
+
+ /* 6-axis setting (1 byte) + button data + checksum */
+ if (pcu->read_pos < IMS_PCU_DATA_OFFSET + 1 + sizeof(data) + 1) {
+ dev_warn(pcu->dev, "Short buttons report: %d bytes\n",
+ pcu->read_pos);
+ return;
+ }
+
+ data = get_unaligned_be32(&pcu->read_buf[IMS_PCU_DATA_OFFSET + 1]);
ims_pcu_buttons_report(pcu, data & ~IMS_PCU_GAMEPAD_MASK);
if (pcu->gamepad)
@@ -666,11 +676,19 @@ static int __ims_pcu_execute_bl_command(struct ims_pcu *pcu,
return error;
}
- if (expected_response && pcu->cmd_buf[2] != expected_response) {
- dev_err(pcu->dev,
- "Unexpected response from bootloader: 0x%02x, wanted 0x%02x\n",
- pcu->cmd_buf[2], expected_response);
- return -EINVAL;
+ if (expected_response) {
+ if (pcu->cmd_buf_len < 3) {
+ dev_err(pcu->dev, "Short response from bootloader: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
+ if (pcu->cmd_buf[2] != expected_response) {
+ dev_err(pcu->dev,
+ "Unexpected response from bootloader: 0x%02x, wanted 0x%02x\n",
+ pcu->cmd_buf[2], expected_response);
+ return -EINVAL;
+ }
}
return 0;
@@ -698,6 +716,12 @@ static int ims_pcu_get_info(struct ims_pcu *pcu)
return error;
}
+ if (pcu->cmd_buf_len < IMS_PCU_DATA_OFFSET + IMS_PCU_SET_INFO_SIZE + 1) {
+ dev_err(pcu->dev, "Short GET_INFO response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
memcpy(pcu->part_number,
&pcu->cmd_buf[IMS_PCU_INFO_PART_OFFSET],
sizeof(pcu->part_number));
@@ -795,6 +819,12 @@ static int ims_pcu_verify_block(struct ims_pcu *pcu,
return error;
}
+ if (pcu->cmd_buf_len < IMS_PCU_BL_DATA_OFFSET + sizeof(*fragment) + len + 1) {
+ dev_err(pcu->dev, "Short READ_APP response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
fragment = (void *)&pcu->cmd_buf[IMS_PCU_BL_DATA_OFFSET];
if (get_unaligned_le32(&fragment->addr) != addr ||
fragment->len != len) {
@@ -914,9 +944,10 @@ static int ims_pcu_handle_firmware_update(struct ims_pcu *pcu,
return retval;
}
-static void ims_pcu_process_async_firmware(const struct firmware *fw,
+static void ims_pcu_process_async_firmware(const struct firmware *_fw,
void *context)
{
+ const struct firmware *fw __free(firmware) = _fw;
struct ims_pcu *pcu = context;
int error;
@@ -936,8 +967,6 @@ static void ims_pcu_process_async_firmware(const struct firmware *fw,
scoped_guard(mutex, &pcu->cmd_mutex)
ims_pcu_handle_firmware_update(pcu, fw);
- release_firmware(fw);
-
out:
complete(&pcu->async_firmware_done);
}
@@ -989,6 +1018,10 @@ ims_pcu_backlight_get_brightness(struct led_classdev *cdev)
error);
/* Assume the LED is OFF */
brightness = LED_OFF;
+ } else if (pcu->cmd_buf_len < IMS_PCU_DATA_OFFSET + 2 + 1) {
+ dev_err(pcu->dev, "Short GET_BRIGHTNESS response: %d bytes\n",
+ pcu->cmd_buf_len);
+ brightness = LED_OFF;
} else {
brightness =
get_unaligned_le16(&pcu->cmd_buf[IMS_PCU_DATA_OFFSET]);
@@ -1133,6 +1166,8 @@ static ssize_t ims_pcu_reset_device(struct device *dev,
dev_info(pcu->dev, "Attempting to reset device\n");
+ guard(mutex)(&pcu->cmd_mutex);
+
error = ims_pcu_execute_command(pcu, PCU_RESET, &reset_byte, 1);
if (error) {
dev_info(pcu->dev,
@@ -1229,6 +1264,9 @@ static umode_t ims_pcu_is_attr_visible(struct kobject *kobj,
struct ims_pcu *pcu = usb_get_intfdata(intf);
umode_t mode = attr->mode;
+ if (intf != pcu->ctrl_intf)
+ return 0;
+
if (pcu->bootloader_mode) {
if (attr != &dev_attr_update_firmware_status.attr &&
attr != &dev_attr_update_firmware.attr &&
@@ -1262,6 +1300,12 @@ static int ims_pcu_read_ofn_config(struct ims_pcu *pcu, u8 addr, u8 *data)
if (error)
return error;
+ if (pcu->cmd_buf_len < OFN_REG_RESULT_OFFSET + 2 + 1) {
+ dev_err(pcu->dev, "Short OFN_GET_CONFIG response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
result = (s16)get_unaligned_le16(pcu->cmd_buf + OFN_REG_RESULT_OFFSET);
if (result < 0)
return -EIO;
@@ -1282,6 +1326,12 @@ static int ims_pcu_write_ofn_config(struct ims_pcu *pcu, u8 addr, u8 data)
if (error)
return error;
+ if (pcu->cmd_buf_len < OFN_REG_RESULT_OFFSET + 2 + 1) {
+ dev_err(pcu->dev, "Short OFN_SET_CONFIG response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
result = (s16)get_unaligned_le16(pcu->cmd_buf + OFN_REG_RESULT_OFFSET);
if (result < 0)
return -EIO;
@@ -1468,6 +1518,9 @@ static umode_t ims_pcu_ofn_is_attr_visible(struct kobject *kobj,
struct ims_pcu *pcu = usb_get_intfdata(intf);
umode_t mode = attr->mode;
+ if (intf != pcu->ctrl_intf)
+ return SYSFS_GROUP_INVISIBLE;
+
/*
* PCU-B devices, both GEN_1 and GEN_2 do not have OFN sensor.
*/
@@ -1508,7 +1561,7 @@ static void ims_pcu_irq(struct urb *urb)
}
dev_dbg(pcu->dev, "%s: received %d: %*ph\n", __func__,
- urb->actual_length, urb->actual_length, pcu->urb_in_buf);
+ urb->actual_length, urb->actual_length, urb->transfer_buffer);
if (urb == pcu->urb_in)
ims_pcu_process_data(pcu, urb);
@@ -1636,8 +1689,9 @@ ims_pcu_get_cdc_union_desc(struct usb_interface *intf)
while (buflen >= sizeof(*union_desc)) {
union_desc = (struct usb_cdc_union_desc *)buf;
- if (union_desc->bLength > buflen) {
- dev_err(&intf->dev, "Too large descriptor\n");
+ if (union_desc->bLength < 2 || union_desc->bLength > buflen) {
+ dev_err(&intf->dev, "Invalid descriptor length: %d\n",
+ union_desc->bLength);
return NULL;
}
@@ -1673,7 +1727,7 @@ static int ims_pcu_parse_cdc_data(struct usb_interface *intf, struct ims_pcu *pc
pcu->ctrl_intf = usb_ifnum_to_if(pcu->udev,
union_desc->bMasterInterface0);
- if (!pcu->ctrl_intf)
+ if (pcu->ctrl_intf != intf)
return -EINVAL;
alt = pcu->ctrl_intf->cur_altsetting;
@@ -1682,6 +1736,12 @@ static int ims_pcu_parse_cdc_data(struct usb_interface *intf, struct ims_pcu *pc
return -ENODEV;
pcu->ep_ctrl = &alt->endpoint[0].desc;
+ if (!usb_endpoint_is_int_in(pcu->ep_ctrl)) {
+ dev_err(pcu->dev,
+ "Control endpoint is not INTERRUPT IN\n");
+ return -EINVAL;
+ }
+
pcu->max_ctrl_size = usb_endpoint_maxp(pcu->ep_ctrl);
pcu->data_intf = usb_ifnum_to_if(pcu->udev,
@@ -1763,11 +1823,16 @@ static void ims_pcu_stop_io(struct ims_pcu *pcu)
static int ims_pcu_line_setup(struct ims_pcu *pcu)
{
struct usb_host_interface *interface = pcu->ctrl_intf->cur_altsetting;
- struct usb_cdc_line_coding *line = (void *)pcu->cmd_buf;
+ struct usb_cdc_line_coding *line __free(kfree) =
+ kmalloc(sizeof(*line), GFP_KERNEL);
int error;
- memset(line, 0, sizeof(*line));
+ if (!line)
+ return -ENOMEM;
+
line->dwDTERate = cpu_to_le32(57600);
+ line->bCharFormat = USB_CDC_1_STOP_BITS;
+ line->bParityType = USB_CDC_NO_PARITY;
line->bDataBits = 8;
error = usb_control_msg(pcu->udev, usb_sndctrlpipe(pcu->udev, 0),
@@ -1811,6 +1876,12 @@ static int ims_pcu_get_device_info(struct ims_pcu *pcu)
return error;
}
+ if (pcu->cmd_buf_len < IMS_PCU_DATA_OFFSET + 6 + 1) {
+ dev_err(pcu->dev, "Short GET_FW_VERSION response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
snprintf(pcu->fw_version, sizeof(pcu->fw_version),
"%02d%02d%02d%02d.%c%c",
pcu->cmd_buf[2], pcu->cmd_buf[3], pcu->cmd_buf[4], pcu->cmd_buf[5],
@@ -1823,6 +1894,12 @@ static int ims_pcu_get_device_info(struct ims_pcu *pcu)
return error;
}
+ if (pcu->cmd_buf_len < IMS_PCU_DATA_OFFSET + 6 + 1) {
+ dev_err(pcu->dev, "Short GET_BL_VERSION response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
snprintf(pcu->bl_version, sizeof(pcu->bl_version),
"%02d%02d%02d%02d.%c%c",
pcu->cmd_buf[2], pcu->cmd_buf[3], pcu->cmd_buf[4], pcu->cmd_buf[5],
@@ -1835,6 +1912,12 @@ static int ims_pcu_get_device_info(struct ims_pcu *pcu)
return error;
}
+ if (pcu->cmd_buf_len < IMS_PCU_DATA_OFFSET + 1 + 1) {
+ dev_err(pcu->dev, "Short RESET_REASON response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
snprintf(pcu->reset_reason, sizeof(pcu->reset_reason),
"%02x", pcu->cmd_buf[IMS_PCU_DATA_OFFSET]);
@@ -1861,6 +1944,12 @@ static int ims_pcu_identify_type(struct ims_pcu *pcu, u8 *device_id)
return error;
}
+ if (pcu->cmd_buf_len < IMS_PCU_DATA_OFFSET + 1 + 1) {
+ dev_err(pcu->dev, "Short GET_DEVICE_ID response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
*device_id = pcu->cmd_buf[IMS_PCU_DATA_OFFSET];
dev_dbg(pcu->dev, "Detected device ID: %d\n", *device_id);
@@ -1952,6 +2041,12 @@ static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
return error;
}
+ if (pcu->cmd_buf_len < IMS_PCU_DATA_OFFSET + 15 + 4 + 1) {
+ dev_err(pcu->dev, "Short QUERY_DEVICE response: %d bytes\n",
+ pcu->cmd_buf_len);
+ return -EIO;
+ }
+
pcu->fw_start_addr =
get_unaligned_le32(&pcu->cmd_buf[IMS_PCU_DATA_OFFSET + 11]);
pcu->fw_end_addr =
@@ -2051,7 +2146,6 @@ static int ims_pcu_probe(struct usb_interface *intf,
static void ims_pcu_disconnect(struct usb_interface *intf)
{
struct ims_pcu *pcu = usb_get_intfdata(intf);
- struct usb_host_interface *alt = intf->cur_altsetting;
usb_set_intfdata(intf, NULL);
@@ -2059,7 +2153,7 @@ static void ims_pcu_disconnect(struct usb_interface *intf)
* See if we are dealing with control or data interface. The cleanup
* happens when we unbind primary (control) interface.
*/
- if (alt->desc.bInterfaceClass != USB_CLASS_COMM)
+ if (intf != pcu->ctrl_intf)
return;
ims_pcu_stop_io(pcu);
@@ -2070,6 +2164,7 @@ static void ims_pcu_disconnect(struct usb_interface *intf)
ims_pcu_destroy_application_mode(pcu);
ims_pcu_buffers_free(pcu);
+ usb_driver_release_interface(&ims_pcu_driver, pcu->data_intf);
kfree(pcu);
}
diff --git a/drivers/iommu/amd/amd_iommu.h b/drivers/iommu/amd/amd_iommu.h
index 6fac9ee8dd3e..8be6b4514dd5 100644
--- a/drivers/iommu/amd/amd_iommu.h
+++ b/drivers/iommu/amd/amd_iommu.h
@@ -11,6 +11,9 @@
#include "amd_iommu_types.h"
+extern int amd_iommu_evtlog_size;
+extern int amd_iommu_pprlog_size;
+
irqreturn_t amd_iommu_int_thread(int irq, void *data);
irqreturn_t amd_iommu_int_thread_evtlog(int irq, void *data);
irqreturn_t amd_iommu_int_thread_pprlog(int irq, void *data);
@@ -52,7 +55,8 @@ struct iommu_domain *amd_iommu_domain_alloc_sva(struct device *dev,
struct mm_struct *mm);
void amd_iommu_domain_free(struct iommu_domain *dom);
int iommu_sva_set_dev_pasid(struct iommu_domain *domain,
- struct device *dev, ioasid_t pasid);
+ struct device *dev, ioasid_t pasid,
+ struct iommu_domain *old);
void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid,
struct iommu_domain *domain);
diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h
index 7f13b314abbc..b3af429f2314 100644
--- a/drivers/iommu/amd/amd_iommu_types.h
+++ b/drivers/iommu/amd/amd_iommu_types.h
@@ -15,6 +15,7 @@
#include <linux/mutex.h>
#include <linux/msi.h>
#include <linux/list.h>
+#include <linux/sizes.h>
#include <linux/spinlock.h>
#include <linux/pci.h>
#include <linux/irqreturn.h>
@@ -132,7 +133,6 @@
#define MMIO_STATUS_GALOG_INT_MASK BIT(10)
/* event logging constants */
-#define EVENT_ENTRY_SIZE 0x10
#define EVENT_TYPE_SHIFT 28
#define EVENT_TYPE_MASK 0xf
#define EVENT_TYPE_ILL_DEV 0x1
@@ -240,15 +240,20 @@
#define MMIO_CMD_SIZE_512 (0x9ULL << MMIO_CMD_SIZE_SHIFT)
/* constants for event buffer handling */
-#define EVT_BUFFER_SIZE 8192 /* 512 entries */
-#define EVT_LEN_MASK (0x9ULL << 56)
+#define EVTLOG_ENTRY_SIZE 0x10
+#define EVTLOG_SIZE_SHIFT 56
+#define EVTLOG_SIZE_DEF SZ_8K /* 512 entries */
+#define EVTLOG_LEN_MASK_DEF (0x9ULL << EVTLOG_SIZE_SHIFT)
+#define EVTLOG_SIZE_MAX SZ_512K /* 32K entries */
+#define EVTLOG_LEN_MASK_MAX (0xFULL << EVTLOG_SIZE_SHIFT)
/* Constants for PPR Log handling */
-#define PPR_LOG_ENTRIES 512
-#define PPR_LOG_SIZE_SHIFT 56
-#define PPR_LOG_SIZE_512 (0x9ULL << PPR_LOG_SIZE_SHIFT)
-#define PPR_ENTRY_SIZE 16
-#define PPR_LOG_SIZE (PPR_ENTRY_SIZE * PPR_LOG_ENTRIES)
+#define PPRLOG_ENTRY_SIZE 0x10
+#define PPRLOG_SIZE_SHIFT 56
+#define PPRLOG_SIZE_DEF SZ_8K /* 512 entries */
+#define PPRLOG_LEN_MASK_DEF (0x9ULL << PPRLOG_SIZE_SHIFT)
+#define PPRLOG_SIZE_MAX SZ_512K /* 32K entries */
+#define PPRLOG_LEN_MASK_MAX (0xFULL << PPRLOG_SIZE_SHIFT)
/* PAGE_SERVICE_REQUEST PPR Log Buffer Entry flags */
#define PPR_FLAG_EXEC 0x002 /* Execute permission requested */
@@ -964,12 +969,13 @@ static inline int get_hpet_devid(int id)
}
enum amd_iommu_intr_mode_type {
- AMD_IOMMU_GUEST_IR_LEGACY,
-
- /* This mode is not visible to users. It is used when
- * we cannot fully enable vAPIC and fallback to only support
- * legacy interrupt remapping via 128-bit IRTE.
+ /*
+ * The legacy format mode is not visible to users to prevent the user
+ * from crashing x2APIC systems, which for all intents and purposes
+ * require 128-bit IRTEs. The legacy format will be forced as needed
+ * when hardware doesn't support 128-bit IRTEs.
*/
+ AMD_IOMMU_GUEST_IR_LEGACY,
AMD_IOMMU_GUEST_IR_LEGACY_GA,
AMD_IOMMU_GUEST_IR_VAPIC,
};
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 78e9ceda2338..7fa340281b9f 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -133,6 +133,9 @@ struct ivhd_entry {
u8 uid;
} __attribute__((packed));
+int amd_iommu_evtlog_size = EVTLOG_SIZE_DEF;
+int amd_iommu_pprlog_size = PPRLOG_SIZE_DEF;
+
/*
* An AMD IOMMU memory definition structure. It defines things like exclusion
* ranges for devices and regions that should be unity mapped.
@@ -855,30 +858,41 @@ void *__init iommu_alloc_4k_pages(struct amd_iommu *iommu, gfp_t gfp,
}
/* allocates the memory where the IOMMU will log its events to */
-static int __init alloc_event_buffer(struct amd_iommu *iommu)
+static int __init alloc_event_buffer(void)
{
- iommu->evt_buf = iommu_alloc_4k_pages(iommu, GFP_KERNEL,
- EVT_BUFFER_SIZE);
+ struct amd_iommu *iommu;
+
+ for_each_iommu(iommu) {
+ iommu->evt_buf = iommu_alloc_4k_pages(iommu, GFP_KERNEL,
+ amd_iommu_evtlog_size);
+ if (!iommu->evt_buf)
+ return -ENOMEM;
+ }
- return iommu->evt_buf ? 0 : -ENOMEM;
+ return 0;
}
-static void iommu_enable_event_buffer(struct amd_iommu *iommu)
+static void iommu_enable_event_buffer(void)
{
+ struct amd_iommu *iommu;
u64 entry;
- BUG_ON(iommu->evt_buf == NULL);
+ for_each_iommu(iommu) {
+ BUG_ON(iommu->evt_buf == NULL);
- entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
+ entry = iommu_virt_to_phys(iommu->evt_buf);
+ entry |= (amd_iommu_evtlog_size == EVTLOG_SIZE_DEF) ?
+ EVTLOG_LEN_MASK_DEF : EVTLOG_LEN_MASK_MAX;
- memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
- &entry, sizeof(entry));
+ memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
+ &entry, sizeof(entry));
- /* set head and tail to zero manually */
- writel(0x00, iommu->mmio_base + MMIO_EVT_HEAD_OFFSET);
- writel(0x00, iommu->mmio_base + MMIO_EVT_TAIL_OFFSET);
+ /* set head and tail to zero manually */
+ writel(0x00, iommu->mmio_base + MMIO_EVT_HEAD_OFFSET);
+ writel(0x00, iommu->mmio_base + MMIO_EVT_TAIL_OFFSET);
- iommu_feature_enable(iommu, CONTROL_EVT_LOG_EN);
+ iommu_feature_enable(iommu, CONTROL_EVT_LOG_EN);
+ }
}
/*
@@ -891,7 +905,7 @@ static void iommu_disable_event_buffer(struct amd_iommu *iommu)
static void __init free_event_buffer(struct amd_iommu *iommu)
{
- iommu_free_pages(iommu->evt_buf, get_order(EVT_BUFFER_SIZE));
+ iommu_free_pages(iommu->evt_buf, get_order(amd_iommu_evtlog_size));
}
static void free_ga_log(struct amd_iommu *iommu)
@@ -1828,9 +1842,6 @@ static int __init init_iommu_one_late(struct amd_iommu *iommu)
if (alloc_command_buffer(iommu))
return -ENOMEM;
- if (alloc_event_buffer(iommu))
- return -ENOMEM;
-
iommu->int_enabled = false;
init_translation_status(iommu);
@@ -2754,7 +2765,6 @@ static void early_enable_iommu(struct amd_iommu *iommu)
iommu_init_flags(iommu);
iommu_set_device_table(iommu);
iommu_enable_command_buffer(iommu);
- iommu_enable_event_buffer(iommu);
iommu_set_exclusion_range(iommu);
iommu_enable_gt(iommu);
iommu_enable_ga(iommu);
@@ -2812,7 +2822,6 @@ static void early_enable_iommus(void)
iommu_disable_event_buffer(iommu);
iommu_disable_irtcachedis(iommu);
iommu_enable_command_buffer(iommu);
- iommu_enable_event_buffer(iommu);
iommu_enable_ga(iommu);
iommu_enable_xt(iommu);
iommu_enable_irtcachedis(iommu);
@@ -2928,6 +2937,7 @@ static void amd_iommu_resume(void)
/* re-load the hardware */
enable_iommus();
+ iommu_enable_event_buffer();
amd_iommu_enable_interrupts();
}
@@ -3242,6 +3252,33 @@ static void iommu_snp_enable(void)
#endif
}
+static void amd_iommu_apply_erratum_snp(void)
+{
+#ifdef CONFIG_KVM_AMD_SEV
+ if (!amd_iommu_snp_en)
+ return;
+
+ /* Errata fix for Family 0x19 */
+ if (boot_cpu_data.x86 != 0x19)
+ return;
+
+ /* Set event log buffer size to max */
+ amd_iommu_evtlog_size = EVTLOG_SIZE_MAX;
+ pr_info("Applying erratum: Increase Event log size to 0x%x\n",
+ amd_iommu_evtlog_size);
+
+ /*
+ * Set PPR log buffer size to max.
+ * (Family 0x19, model < 0x10 doesn't support PPR when SNP is enabled).
+ */
+ if (boot_cpu_data.x86_model >= 0x10) {
+ amd_iommu_pprlog_size = PPRLOG_SIZE_MAX;
+ pr_info("Applying erratum: Increase PPR log size to 0x%x\n",
+ amd_iommu_pprlog_size);
+ }
+#endif
+}
+
/****************************************************************************
*
* AMD IOMMU Initialization State Machine
@@ -3278,6 +3315,17 @@ static int __init state_next(void)
case IOMMU_ENABLED:
register_syscore_ops(&amd_iommu_syscore_ops);
iommu_snp_enable();
+
+ amd_iommu_apply_erratum_snp();
+
+ /* Allocate/enable event log buffer */
+ ret = alloc_event_buffer();
+ if (ret) {
+ init_state = IOMMU_INIT_ERROR;
+ break;
+ }
+ iommu_enable_event_buffer();
+
ret = amd_iommu_init_pci();
init_state = ret ? IOMMU_INIT_ERROR : IOMMU_PCI_INIT;
break;
@@ -3865,11 +3913,11 @@ int amd_iommu_snp_disable(void)
return 0;
for_each_iommu(iommu) {
- ret = iommu_make_shared(iommu->evt_buf, EVT_BUFFER_SIZE);
+ ret = iommu_make_shared(iommu->evt_buf, amd_iommu_evtlog_size);
if (ret)
return ret;
- ret = iommu_make_shared(iommu->ppr_log, PPR_LOG_SIZE);
+ ret = iommu_make_shared(iommu->ppr_log, amd_iommu_pprlog_size);
if (ret)
return ret;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 5ea0d9eb0ccc..b4052fa9cc76 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1003,7 +1003,7 @@ static void iommu_poll_events(struct amd_iommu *iommu)
iommu_print_event(iommu, iommu->evt_buf + head);
/* Update head pointer of hardware ring-buffer */
- head = (head + EVENT_ENTRY_SIZE) % EVT_BUFFER_SIZE;
+ head = (head + EVTLOG_ENTRY_SIZE) % amd_iommu_evtlog_size;
writel(head, iommu->mmio_base + MMIO_EVT_HEAD_OFFSET);
}
diff --git a/drivers/iommu/amd/pasid.c b/drivers/iommu/amd/pasid.c
index 0657b9373be5..d1dfc745f55e 100644
--- a/drivers/iommu/amd/pasid.c
+++ b/drivers/iommu/amd/pasid.c
@@ -100,7 +100,8 @@ static const struct mmu_notifier_ops sva_mn = {
};
int iommu_sva_set_dev_pasid(struct iommu_domain *domain,
- struct device *dev, ioasid_t pasid)
+ struct device *dev, ioasid_t pasid,
+ struct iommu_domain *old)
{
struct pdom_dev_data *pdom_dev_data;
struct protection_domain *sva_pdom = to_pdomain(domain);
diff --git a/drivers/iommu/amd/ppr.c b/drivers/iommu/amd/ppr.c
index 7c67d69f0b8c..42d895bd1c44 100644
--- a/drivers/iommu/amd/ppr.c
+++ b/drivers/iommu/amd/ppr.c
@@ -20,7 +20,7 @@
int __init amd_iommu_alloc_ppr_log(struct amd_iommu *iommu)
{
iommu->ppr_log = iommu_alloc_4k_pages(iommu, GFP_KERNEL | __GFP_ZERO,
- PPR_LOG_SIZE);
+ amd_iommu_pprlog_size);
return iommu->ppr_log ? 0 : -ENOMEM;
}
@@ -33,7 +33,9 @@ void amd_iommu_enable_ppr_log(struct amd_iommu *iommu)
iommu_feature_enable(iommu, CONTROL_PPR_EN);
- entry = iommu_virt_to_phys(iommu->ppr_log) | PPR_LOG_SIZE_512;
+ entry = iommu_virt_to_phys(iommu->ppr_log);
+ entry |= (amd_iommu_pprlog_size == PPRLOG_SIZE_DEF) ?
+ PPRLOG_LEN_MASK_DEF : PPRLOG_LEN_MASK_MAX;
memcpy_toio(iommu->mmio_base + MMIO_PPR_LOG_OFFSET,
&entry, sizeof(entry));
@@ -48,7 +50,7 @@ void amd_iommu_enable_ppr_log(struct amd_iommu *iommu)
void __init amd_iommu_free_ppr_log(struct amd_iommu *iommu)
{
- iommu_free_pages(iommu->ppr_log, get_order(PPR_LOG_SIZE));
+ iommu_free_pages(iommu->ppr_log, get_order(amd_iommu_pprlog_size));
}
/*
@@ -201,7 +203,7 @@ void amd_iommu_poll_ppr_log(struct amd_iommu *iommu)
raw[0] = raw[1] = 0UL;
/* Update head pointer of hardware ring-buffer */
- head = (head + PPR_ENTRY_SIZE) % PPR_LOG_SIZE;
+ head = (head + PPRLOG_ENTRY_SIZE) % amd_iommu_pprlog_size;
writel(head, iommu->mmio_base + MMIO_PPR_HEAD_OFFSET);
/* Handle PPR entry */
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index 32f3e91a7d7f..6cb2fbf43037 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -332,7 +332,8 @@ void arm_smmu_sva_notifier_synchronize(void)
}
static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain,
- struct device *dev, ioasid_t id)
+ struct device *dev, ioasid_t id,
+ struct iommu_domain *old)
{
struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
struct arm_smmu_master *master = dev_iommu_priv_get(dev);
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 04e0058daab8..ebc25e5a8f55 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2879,7 +2879,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
}
static int arm_smmu_s1_set_dev_pasid(struct iommu_domain *domain,
- struct device *dev, ioasid_t id)
+ struct device *dev, ioasid_t id,
+ struct iommu_domain *old)
{
struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
struct arm_smmu_master *master = dev_iommu_priv_get(dev);
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index cce5a19b5d33..a1ec11c90a8b 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1914,10 +1914,12 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8
}
did = context_domain_id(context);
- context_clear_entry(context);
+ context_clear_present(context);
__iommu_flush_cache(iommu, context, sizeof(*context));
spin_unlock(&iommu->lock);
- intel_context_flush_present(info, context, did, true);
+ intel_context_flush_no_pasid(info, context, did);
+ context_clear_entry(context);
+ __iommu_flush_cache(iommu, context, sizeof(*context));
}
static int domain_setup_first_level(struct intel_iommu *iommu,
@@ -4151,7 +4153,7 @@ static int context_flip_pri(struct device_domain_info *info, bool enable)
if (!ecap_coherent(iommu->ecap))
clflush_cache_range(context, sizeof(*context));
- intel_context_flush_present(info, context, did, true);
+ intel_context_flush_no_pasid(info, context, did);
spin_unlock(&iommu->lock);
return 0;
@@ -4336,7 +4338,8 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid,
}
static int intel_iommu_set_dev_pasid(struct iommu_domain *domain,
- struct device *dev, ioasid_t pasid)
+ struct device *dev, ioasid_t pasid,
+ struct iommu_domain *old)
{
struct device_domain_info *info = dev_iommu_priv_get(dev);
struct dmar_domain *dmar_domain = to_dmar_domain(domain);
@@ -4622,7 +4625,8 @@ static int identity_domain_attach_dev(struct iommu_domain *domain, struct device
}
static int identity_domain_set_dev_pasid(struct iommu_domain *domain,
- struct device *dev, ioasid_t pasid)
+ struct device *dev, ioasid_t pasid,
+ struct iommu_domain *old)
{
struct device_domain_info *info = dev_iommu_priv_get(dev);
struct intel_iommu *iommu = info->iommu;
diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index b33d8888d7eb..8a91a8d56614 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -958,7 +958,26 @@ static inline unsigned long virt_to_dma_pfn(void *p)
static inline void context_set_present(struct context_entry *context)
{
- context->lo |= 1;
+ u64 val;
+
+ dma_wmb();
+ val = READ_ONCE(context->lo) | 1;
+ WRITE_ONCE(context->lo, val);
+}
+
+/*
+ * Clear the Present (P) bit (bit 0) of a context table entry. This initiates
+ * the transition of the entry's ownership from hardware to software. The
+ * caller is responsible for fulfilling the invalidation handshake recommended
+ * by the VT-d spec, Section 6.5.3.3 (Guidance to Software for Invalidations).
+ */
+static inline void context_clear_present(struct context_entry *context)
+{
+ u64 val;
+
+ val = READ_ONCE(context->lo) & GENMASK_ULL(63, 1);
+ WRITE_ONCE(context->lo, val);
+ dma_wmb();
}
static inline void context_set_fault_enable(struct context_entry *context)
@@ -1277,9 +1296,8 @@ void cache_tag_flush_all(struct dmar_domain *domain);
void cache_tag_flush_range_np(struct dmar_domain *domain, unsigned long start,
unsigned long end);
-void intel_context_flush_present(struct device_domain_info *info,
- struct context_entry *context,
- u16 did, bool affect_domains);
+void intel_context_flush_no_pasid(struct device_domain_info *info,
+ struct context_entry *context, u16 did);
int intel_iommu_enable_prq(struct intel_iommu *iommu);
int intel_iommu_finish_prq(struct intel_iommu *iommu);
diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c
index 74be6b547fc0..93e9b9243d5f 100644
--- a/drivers/iommu/intel/pasid.c
+++ b/drivers/iommu/intel/pasid.c
@@ -714,10 +714,12 @@ static void device_pasid_table_teardown(struct device *dev, u8 bus, u8 devfn)
}
did = context_domain_id(context);
- context_clear_entry(context);
+ context_clear_present(context);
__iommu_flush_cache(iommu, context, sizeof(*context));
spin_unlock(&iommu->lock);
- intel_context_flush_present(info, context, did, false);
+ intel_context_flush_no_pasid(info, context, did);
+ context_clear_entry(context);
+ __iommu_flush_cache(iommu, context, sizeof(*context));
}
static int pci_pasid_table_teardown(struct pci_dev *pdev, u16 alias, void *data)
@@ -804,7 +806,7 @@ static int device_pasid_table_setup(struct device *dev, u8 bus, u8 devfn)
}
if (context_copied(iommu, bus, devfn)) {
- context_clear_entry(context);
+ context_clear_present(context);
__iommu_flush_cache(iommu, context, sizeof(*context));
/*
@@ -824,6 +826,9 @@ static int device_pasid_table_setup(struct device *dev, u8 bus, u8 devfn)
iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
devtlb_invalidation_with_pasid(iommu, dev, IOMMU_NO_PASID);
+ context_clear_entry(context);
+ __iommu_flush_cache(iommu, context, sizeof(*context));
+
/*
* At this point, the device is supposed to finish reset at
* its driver probe stage, so no in-flight DMA will exist,
@@ -910,17 +915,15 @@ static void __context_flush_dev_iotlb(struct device_domain_info *info)
/*
* Cache invalidations after change in a context table entry that was present
- * according to the Spec 6.5.3.3 (Guidance to Software for Invalidations). If
- * IOMMU is in scalable mode and all PASID table entries of the device were
- * non-present, set flush_domains to false. Otherwise, true.
+ * according to the Spec 6.5.3.3 (Guidance to Software for Invalidations).
+ * This helper can only be used when IOMMU is working in the legacy mode or
+ * IOMMU is in scalable mode but all PASID table entries of the device are
+ * non-present.
*/
-void intel_context_flush_present(struct device_domain_info *info,
- struct context_entry *context,
- u16 did, bool flush_domains)
+void intel_context_flush_no_pasid(struct device_domain_info *info,
+ struct context_entry *context, u16 did)
{
struct intel_iommu *iommu = info->iommu;
- struct pasid_entry *pte;
- int i;
/*
* Device-selective context-cache invalidation. The Domain-ID field
@@ -943,30 +946,5 @@ void intel_context_flush_present(struct device_domain_info *info,
return;
}
- /*
- * For scalable mode:
- * - Domain-selective PASID-cache invalidation to affected domains
- * - Domain-selective IOTLB invalidation to affected domains
- * - Global Device-TLB invalidation to affected functions
- */
- if (flush_domains) {
- /*
- * If the IOMMU is running in scalable mode and there might
- * be potential PASID translations, the caller should hold
- * the lock to ensure that context changes and cache flushes
- * are atomic.
- */
- assert_spin_locked(&iommu->lock);
- for (i = 0; i < info->pasid_table->max_pasid; i++) {
- pte = intel_pasid_get_entry(info->dev, i);
- if (!pte || !pasid_pte_is_present(pte))
- continue;
-
- did = pasid_get_domain_id(pte);
- qi_flush_pasid_cache(iommu, did, QI_PC_ALL_PASIDS, 0);
- iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH);
- }
- }
-
__context_flush_dev_iotlb(info);
}
diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c
index 3cc43a958b4d..4a2bd65614ad 100644
--- a/drivers/iommu/intel/svm.c
+++ b/drivers/iommu/intel/svm.c
@@ -111,7 +111,8 @@ static const struct mmu_notifier_ops intel_mmuops = {
};
static int intel_svm_set_dev_pasid(struct iommu_domain *domain,
- struct device *dev, ioasid_t pasid)
+ struct device *dev, ioasid_t pasid,
+ struct iommu_domain *old)
{
struct device_domain_info *info = dev_iommu_priv_get(dev);
struct dmar_domain *dmar_domain = to_dmar_domain(domain);
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 62e1d6372503..fdebfe5380ba 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -3342,7 +3342,8 @@ static int __iommu_set_group_pasid(struct iommu_domain *domain,
for_each_group_device(group, device) {
if (device->dev->iommu->max_pasids > 0) {
- ret = domain->ops->set_dev_pasid(domain, device->dev, pasid);
+ ret = domain->ops->set_dev_pasid(domain, device->dev,
+ pasid, NULL);
if (ret)
goto err_revert;
}
diff --git a/drivers/irqchip/irq-crossbar.c b/drivers/irqchip/irq-crossbar.c
index a05a7501e107..23b1960024b6 100644
--- a/drivers/irqchip/irq-crossbar.c
+++ b/drivers/irqchip/irq-crossbar.c
@@ -158,11 +158,17 @@ static void crossbar_domain_free(struct irq_domain *domain, unsigned int virq,
for (i = 0; i < nr_irqs; i++) {
struct irq_data *d = irq_domain_get_irq_data(domain, virq + i);
+ /*
+ * irq_map[] is indexed by GIC SPI number. The parent domain's
+ * hwirq contains the GIC interrupt number (GIC SPI +
+ * GIC_IRQ_START).
+ */
+ cb->irq_map[d->parent_data->hwirq - GIC_IRQ_START] = IRQ_FREE;
+ cb->write(d->parent_data->hwirq - GIC_IRQ_START, cb->safe_map);
irq_domain_reset_irq_data(d);
- cb->irq_map[d->hwirq] = IRQ_FREE;
- cb->write(d->hwirq, cb->safe_map);
}
raw_spin_unlock(&cb->lock);
+ irq_domain_free_irqs_parent(domain, virq, nr_irqs);
}
static int crossbar_domain_translate(struct irq_domain *d,
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index b0eb6646a998..95d337978c75 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -3209,11 +3209,9 @@ static void its_cpu_init_collection(struct its_node *its)
/* avoid cross node collections and its mapping */
if (its->flags & ITS_FLAGS_WORKAROUND_CAVIUM_23144) {
- struct device_node *cpu_node;
+ struct device_node *cpu_node __free(device_node) = of_get_cpu_node(cpu, NULL);
- cpu_node = of_get_cpu_node(cpu, NULL);
- if (its->numa_node != NUMA_NO_NODE &&
- its->numa_node != of_node_to_nid(cpu_node))
+ if (its->numa_node != NUMA_NO_NODE && its->numa_node != of_node_to_nid(cpu_node))
return;
}
diff --git a/drivers/irqchip/irq-riscv-imsic-early.c b/drivers/irqchip/irq-riscv-imsic-early.c
index dbd02dfe0397..ab5e49aa94f0 100644
--- a/drivers/irqchip/irq-riscv-imsic-early.c
+++ b/drivers/irqchip/irq-riscv-imsic-early.c
@@ -236,16 +236,13 @@ static int __init imsic_early_acpi_init(union acpi_subtable_headers *header,
rc = imsic_setup_state(imsic_acpi_fwnode, imsic);
if (rc) {
pr_err("%pfwP: failed to setup state (error %d)\n", imsic_acpi_fwnode, rc);
- return rc;
+ goto cleanup;
}
/* Do early setup of IMSIC state and IPIs */
rc = imsic_early_probe(imsic_acpi_fwnode);
- if (rc) {
- irq_domain_free_fwnode(imsic_acpi_fwnode);
- imsic_acpi_fwnode = NULL;
- return rc;
- }
+ if (rc)
+ goto cleanup;
rc = imsic_platform_acpi_probe(imsic_acpi_fwnode);
@@ -264,8 +261,12 @@ static int __init imsic_early_acpi_init(union acpi_subtable_headers *header,
* DT where IPI works but MSI probe fails for some reason.
*/
return 0;
-}
+cleanup:
+ irq_domain_free_fwnode(imsic_acpi_fwnode);
+ imsic_acpi_fwnode = NULL;
+ return rc;
+}
IRQCHIP_ACPI_DECLARE(riscv_imsic, ACPI_MADT_TYPE_IMSIC, NULL,
1, imsic_early_acpi_init);
#endif
diff --git a/drivers/irqchip/irq-ts4800.c b/drivers/irqchip/irq-ts4800.c
index b5dddb3c1568..a978db71ac32 100644
--- a/drivers/irqchip/irq-ts4800.c
+++ b/drivers/irqchip/irq-ts4800.c
@@ -28,6 +28,7 @@ struct ts4800_irq_data {
void __iomem *base;
struct platform_device *pdev;
struct irq_domain *domain;
+ unsigned int parent_irq;
};
static void ts4800_irq_mask(struct irq_data *d)
@@ -134,6 +135,7 @@ static int ts4800_ic_probe(struct platform_device *pdev)
irq_set_chained_handler_and_data(parent_irq,
ts4800_ic_chained_handle_irq, data);
+ data->parent_irq = parent_irq;
platform_set_drvdata(pdev, data);
return 0;
@@ -142,6 +144,14 @@ static int ts4800_ic_probe(struct platform_device *pdev)
static void ts4800_ic_remove(struct platform_device *pdev)
{
struct ts4800_irq_data *data = platform_get_drvdata(pdev);
+ unsigned int hwirq;
+
+ irq_set_chained_handler_and_data(data->parent_irq, NULL, NULL);
+
+ for (hwirq = 0; hwirq < 8; hwirq++)
+ irq_dispose_mapping(irq_find_mapping(data->domain, hwirq));
+
+ irq_dispose_mapping(data->parent_irq);
irq_domain_remove(data->domain);
}
diff --git a/drivers/leds/uleds.c b/drivers/leds/uleds.c
index 374a841f18c3..48ca73c106bc 100644
--- a/drivers/leds/uleds.c
+++ b/drivers/leds/uleds.c
@@ -102,7 +102,8 @@ static ssize_t uleds_write(struct file *file, const char __user *buffer,
name = udev->user_dev.name;
if (!name[0] || !strcmp(name, ".") || !strcmp(name, "..") ||
- strchr(name, '/')) {
+ strnchr(name, sizeof(udev->user_dev.name), '/') ||
+ !strnchr(name, sizeof(udev->user_dev.name), '\0')) {
ret = -EINVAL;
goto out;
}
diff --git a/drivers/mailbox/imx-mailbox.c b/drivers/mailbox/imx-mailbox.c
index aa9a299859d6..056c71ec97d1 100644
--- a/drivers/mailbox/imx-mailbox.c
+++ b/drivers/mailbox/imx-mailbox.c
@@ -229,6 +229,7 @@ static int imx_mu_generic_tx(struct imx_mu_priv *priv,
u32 val;
int ret, count;
+ ret = 0;
switch (cp->type) {
case IMX_MU_TYPE_TX:
imx_mu_write(priv, *arg, priv->dcfg->xTR + cp->idx * 4);
@@ -261,7 +262,7 @@ static int imx_mu_generic_tx(struct imx_mu_priv *priv,
return -EINVAL;
}
- return 0;
+ return ret;
}
static int imx_mu_generic_rx(struct imx_mu_priv *priv,
diff --git a/drivers/mailbox/mtk-adsp-mailbox.c b/drivers/mailbox/mtk-adsp-mailbox.c
index 91487aa4d7da..8bcecddee0eb 100644
--- a/drivers/mailbox/mtk-adsp-mailbox.c
+++ b/drivers/mailbox/mtk-adsp-mailbox.c
@@ -19,6 +19,7 @@ struct mtk_adsp_mbox_priv {
struct mbox_controller mbox;
void __iomem *va_mboxreg;
const struct mtk_adsp_mbox_cfg *cfg;
+ int irq;
};
struct mtk_adsp_mbox_cfg {
@@ -67,6 +68,8 @@ static int mtk_adsp_mbox_startup(struct mbox_chan *chan)
writel(0xFFFFFFFF, priv->va_mboxreg + priv->cfg->clr_in);
writel(0xFFFFFFFF, priv->va_mboxreg + priv->cfg->clr_out);
+ enable_irq(priv->irq);
+
return 0;
}
@@ -74,6 +77,8 @@ static void mtk_adsp_mbox_shutdown(struct mbox_chan *chan)
{
struct mtk_adsp_mbox_priv *priv = get_mtk_adsp_mbox_priv(chan->mbox);
+ disable_irq(priv->irq);
+
/* Clear ADSP mbox command */
writel(0xFFFFFFFF, priv->va_mboxreg + priv->cfg->clr_in);
writel(0xFFFFFFFF, priv->va_mboxreg + priv->cfg->clr_out);
@@ -139,8 +144,10 @@ static int mtk_adsp_mbox_probe(struct platform_device *pdev)
if (irq < 0)
return irq;
+ priv->irq = irq;
ret = devm_request_threaded_irq(dev, irq, mtk_adsp_mbox_irq,
- mtk_adsp_mbox_isr, IRQF_TRIGGER_NONE,
+ mtk_adsp_mbox_isr,
+ IRQF_TRIGGER_NONE | IRQF_NO_AUTOEN,
dev_name(dev), mbox->chans);
if (ret < 0)
return ret;
diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
index c3252fdc75fd..be83be508152 100644
--- a/drivers/md/dm-bufio.c
+++ b/drivers/md/dm-bufio.c
@@ -2216,7 +2216,9 @@ int dm_bufio_issue_discard(struct dm_bufio_client *c, sector_t block, sector_t c
struct dm_io_region io_reg = {
.bdev = c->bdev,
.sector = block_to_sector(c, block),
- .count = block_to_sector(c, count),
+ .count = likely(c->sectors_per_block_bits >= 0) ?
+ count << c->sectors_per_block_bits :
+ count * (c->block_size >> SECTOR_SHIFT),
};
if (WARN_ON_ONCE(dm_bufio_in_request()))
diff --git a/drivers/md/dm-era-target.c b/drivers/md/dm-era-target.c
index 9c84e9d13eca..883d1549cfc0 100644
--- a/drivers/md/dm-era-target.c
+++ b/drivers/md/dm-era-target.c
@@ -810,8 +810,10 @@ static struct era_metadata *metadata_open(struct block_device *bdev,
int r;
struct era_metadata *md = kzalloc(sizeof(*md), GFP_KERNEL);
- if (!md)
- return NULL;
+ if (!md) {
+ DMERR("could not allocate metadata struct");
+ return ERR_PTR(-ENOMEM);
+ }
md->bdev = bdev;
md->block_size = block_size;
@@ -1229,6 +1231,7 @@ static dm_block_t get_block(struct era *era, struct bio *bio)
static void remap_to_origin(struct era *era, struct bio *bio)
{
bio_set_dev(bio, era->origin_dev->bdev);
+ bio->bi_iter.bi_sector = dm_target_offset(era->ti, bio->bi_iter.bi_sector);
}
/*
@@ -1560,7 +1563,7 @@ static void era_dtr(struct dm_target *ti)
static int era_map(struct dm_target *ti, struct bio *bio)
{
struct era *era = ti->private;
- dm_block_t block = get_block(era, bio);
+ dm_block_t block;
/*
* All bios get remapped to the origin device. We do this now, but
@@ -1568,6 +1571,7 @@ static int era_map(struct dm_target *ti, struct bio *bio)
* block is marked in this era.
*/
remap_to_origin(era, bio);
+ block = get_block(era, bio);
/*
* REQ_PREFLUSH bios carry no data, so we're not interested in them.
diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c
index e1dc373ad209..b661be629da2 100644
--- a/drivers/md/dm-integrity.c
+++ b/drivers/md/dm-integrity.c
@@ -1470,9 +1470,6 @@ static int dm_integrity_rw_tag(struct dm_integrity_c *ic, unsigned char *tag, se
*metadata_offset = 0;
}
- if (unlikely(!is_power_of_2(ic->tag_size)))
- hash_offset = (hash_offset + to_copy) % ic->tag_size;
-
total_size -= to_copy;
} while (unlikely(total_size));
@@ -2423,6 +2420,9 @@ static int dm_integrity_map_inline(struct dm_integrity_io *dio, bool from_map)
if (unlikely((bio->bi_opf & REQ_PREFLUSH) != 0))
return DM_MAPIO_REMAPPED;
+ if (unlikely(!dm_integrity_check_limits(ic, bio->bi_iter.bi_sector, bio)))
+ return DM_MAPIO_KILL;
+
retry:
if (!dio->integrity_payload) {
unsigned digest_size, extra_size;
@@ -2487,10 +2487,6 @@ static int dm_integrity_map_inline(struct dm_integrity_io *dio, bool from_map)
dio->bio_details.bi_iter = bio->bi_iter;
- if (unlikely(!dm_integrity_check_limits(ic, bio->bi_iter.bi_sector, bio))) {
- return DM_MAPIO_KILL;
- }
-
bio->bi_iter.bi_sector += ic->start + SB_SECTORS;
bip = bio_integrity_alloc(bio, GFP_NOIO, 1);
diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c
index 1bd10e53d65f..abd6b7395f6d 100644
--- a/drivers/md/dm-ioctl.c
+++ b/drivers/md/dm-ioctl.c
@@ -683,7 +683,7 @@ static void list_version_get_info(struct target_type *tt, void *param)
struct vers_iter *info = param;
/* Check space - it might have changed since the first iteration */
- if ((char *)info->vers + sizeof(tt->version) + strlen(tt->name) + 1 > info->end) {
+ if ((char *)info->vers + sizeof(struct dm_target_versions) + strlen(tt->name) + 1 > info->end) {
info->flags = DM_BUFFER_FULL_FLAG;
return;
}
@@ -2340,7 +2340,7 @@ int __init dm_early_create(struct dm_ioctl *dmi,
/* resume device */
r = dm_resume(md);
if (r)
- goto err_destroy_table;
+ goto err_hash_remove;
DMINFO("%s (%s) is ready", md->disk->disk_name, dmi->name);
dm_put(md);
diff --git a/drivers/md/dm-log.c b/drivers/md/dm-log.c
index 4a1369b8f44a..c8a1f9b6385f 100644
--- a/drivers/md/dm-log.c
+++ b/drivers/md/dm-log.c
@@ -425,6 +425,9 @@ static int create_log_context(struct dm_dirty_log *log, struct dm_target *ti,
*/
bitset_size = dm_round_up(region_count, BITS_PER_LONG);
bitset_size >>= BYTE_SHIFT;
+ /* Handle dm_round_up rollover on 32-bit systems */
+ if (!bitset_size)
+ bitset_size = 1UL << (BITS_PER_LONG - BYTE_SHIFT);
lc->bitset_uint32_count = bitset_size / sizeof(*lc->clean_bits);
diff --git a/drivers/md/dm-stats.c b/drivers/md/dm-stats.c
index 1e5d988f44da..dbff4922e068 100644
--- a/drivers/md/dm-stats.c
+++ b/drivers/md/dm-stats.c
@@ -692,10 +692,8 @@ void dm_stats_account_io(struct dm_stats *stats, unsigned long bi_rw,
*/
last = raw_cpu_ptr(stats->last);
stats_aux->merged =
- (bi_sector == (READ_ONCE(last->last_sector) &&
- ((bi_rw == WRITE) ==
- (READ_ONCE(last->last_rw) == WRITE))
- ));
+ bi_sector == READ_ONCE(last->last_sector) &&
+ (bi_rw == WRITE) == (READ_ONCE(last->last_rw) == WRITE);
WRITE_ONCE(last->last_sector, end_sector);
WRITE_ONCE(last->last_rw, bi_rw);
} else
@@ -842,10 +840,10 @@ static unsigned long long dm_jiffies_to_msec64(struct dm_stat *s, unsigned long
result = jiffies_to_msecs(j & 0x3fffff);
if (j >= 1 << 22) {
mult = jiffies_to_msecs(1 << 22);
- result += (unsigned long long)mult * (unsigned long long)jiffies_to_msecs((j >> 22) & 0x3fffff);
+ result += (unsigned long long)mult * ((j >> 22) & 0x3fffff);
}
if (j >= 1ULL << 44)
- result += (unsigned long long)mult * (unsigned long long)mult * (unsigned long long)jiffies_to_msecs(j >> 44);
+ result += (unsigned long long)mult * (unsigned long long)(1 << 22) * (j >> 44);
return result;
}
diff --git a/drivers/md/dm-thin-metadata.c b/drivers/md/dm-thin-metadata.c
index f90679cfec5b..9224d91fca71 100644
--- a/drivers/md/dm-thin-metadata.c
+++ b/drivers/md/dm-thin-metadata.c
@@ -186,6 +186,7 @@ struct dm_pool_metadata {
uint32_t time;
dm_block_t root;
dm_block_t details_root;
+ dm_block_t held_root;
struct list_head thin_devices;
uint64_t trans_id;
unsigned long flags;
@@ -748,6 +749,7 @@ static int __open_metadata(struct dm_pool_metadata *pmd)
*/
pmd->root = le64_to_cpu(disk_super->data_mapping_root);
pmd->details_root = le64_to_cpu(disk_super->device_details_root);
+ pmd->held_root = le64_to_cpu(disk_super->held_root);
__setup_btree_details(pmd);
dm_bm_unlock(sblock);
@@ -838,6 +840,7 @@ static int __begin_transaction(struct dm_pool_metadata *pmd)
pmd->time = le32_to_cpu(disk_super->time);
pmd->root = le64_to_cpu(disk_super->data_mapping_root);
pmd->details_root = le64_to_cpu(disk_super->device_details_root);
+ pmd->held_root = le64_to_cpu(disk_super->held_root);
pmd->trans_id = le64_to_cpu(disk_super->trans_id);
pmd->flags = le32_to_cpu(disk_super->flags);
pmd->data_block_size = le32_to_cpu(disk_super->data_block_size);
@@ -928,6 +931,7 @@ static int __commit_transaction(struct dm_pool_metadata *pmd)
disk_super->time = cpu_to_le32(pmd->time);
disk_super->data_mapping_root = cpu_to_le64(pmd->root);
disk_super->device_details_root = cpu_to_le64(pmd->details_root);
+ disk_super->held_root = cpu_to_le64(pmd->held_root);
disk_super->trans_id = cpu_to_le64(pmd->trans_id);
disk_super->flags = cpu_to_le32(pmd->flags);
@@ -1333,9 +1337,14 @@ static int __reserve_metadata_snap(struct dm_pool_metadata *pmd)
{
int r, inc;
struct thin_disk_superblock *disk_super;
- struct dm_block *copy, *sblock;
+ struct dm_block *copy;
dm_block_t held_root;
+ if (pmd->held_root) {
+ DMWARN("Pool metadata snapshot already exists: release this before taking another.");
+ return -EBUSY;
+ }
+
/*
* We commit to ensure the btree roots which we increment in a
* moment are up to date.
@@ -1353,22 +1362,16 @@ static int __reserve_metadata_snap(struct dm_pool_metadata *pmd)
dm_sm_inc_block(pmd->metadata_sm, THIN_SUPERBLOCK_LOCATION);
r = dm_tm_shadow_block(pmd->tm, THIN_SUPERBLOCK_LOCATION,
&sb_validator, ©, &inc);
- if (r)
+ if (r) {
+ dm_sm_dec_block(pmd->metadata_sm, THIN_SUPERBLOCK_LOCATION);
return r;
+ }
BUG_ON(!inc);
held_root = dm_block_location(copy);
disk_super = dm_block_data(copy);
- if (le64_to_cpu(disk_super->held_root)) {
- DMWARN("Pool metadata snapshot already exists: release this before taking another.");
-
- dm_tm_dec(pmd->tm, held_root);
- dm_tm_unlock(pmd->tm, copy);
- return -EBUSY;
- }
-
/*
* Wipe the spacemap since we're not publishing this.
*/
@@ -1384,18 +1387,8 @@ static int __reserve_metadata_snap(struct dm_pool_metadata *pmd)
dm_tm_inc(pmd->tm, le64_to_cpu(disk_super->device_details_root));
dm_tm_unlock(pmd->tm, copy);
- /*
- * Write the held root into the superblock.
- */
- r = superblock_lock(pmd, &sblock);
- if (r) {
- dm_tm_dec(pmd->tm, held_root);
- return r;
- }
+ pmd->held_root = held_root;
- disk_super = dm_block_data(sblock);
- disk_super->held_root = cpu_to_le64(held_root);
- dm_bm_unlock(sblock);
return 0;
}
@@ -1415,18 +1408,10 @@ static int __release_metadata_snap(struct dm_pool_metadata *pmd)
{
int r;
struct thin_disk_superblock *disk_super;
- struct dm_block *sblock, *copy;
+ struct dm_block *copy;
dm_block_t held_root;
- r = superblock_lock(pmd, &sblock);
- if (r)
- return r;
-
- disk_super = dm_block_data(sblock);
- held_root = le64_to_cpu(disk_super->held_root);
- disk_super->held_root = cpu_to_le64(0);
-
- dm_bm_unlock(sblock);
+ held_root = pmd->held_root;
if (!held_root) {
DMWARN("No pool metadata snapshot found: nothing to release.");
@@ -1437,13 +1422,15 @@ static int __release_metadata_snap(struct dm_pool_metadata *pmd)
if (r)
return r;
+ pmd->held_root = 0;
+
disk_super = dm_block_data(copy);
dm_btree_del(&pmd->info, le64_to_cpu(disk_super->data_mapping_root));
dm_btree_del(&pmd->details_info, le64_to_cpu(disk_super->device_details_root));
- dm_sm_dec_block(pmd->metadata_sm, held_root);
-
dm_tm_unlock(pmd->tm, copy);
+ dm_sm_dec_block(pmd->metadata_sm, held_root);
+
return 0;
}
@@ -1462,19 +1449,7 @@ int dm_pool_release_metadata_snap(struct dm_pool_metadata *pmd)
static int __get_metadata_snap(struct dm_pool_metadata *pmd,
dm_block_t *result)
{
- int r;
- struct thin_disk_superblock *disk_super;
- struct dm_block *sblock;
-
- r = dm_bm_read_lock(pmd->bm, THIN_SUPERBLOCK_LOCATION,
- &sb_validator, &sblock);
- if (r)
- return r;
-
- disk_super = dm_block_data(sblock);
- *result = le64_to_cpu(disk_super->held_root);
-
- dm_bm_unlock(sblock);
+ *result = pmd->held_root;
return 0;
}
diff --git a/drivers/md/dm-verity-loadpin.c b/drivers/md/dm-verity-loadpin.c
index 0666699b6858..9a64f575ae5f 100644
--- a/drivers/md/dm-verity-loadpin.c
+++ b/drivers/md/dm-verity-loadpin.c
@@ -70,7 +70,7 @@ bool dm_verity_loadpin_is_bdev_trusted(struct block_device *bdev)
table = dm_get_live_table(md, &srcu_idx);
- if (table->num_targets != 1)
+ if (!table || table->num_targets != 1)
goto out;
ti = dm_table_get_target(table, 0);
diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c
index ce0462e751a6..6f2ea05bea8d 100644
--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -26,7 +26,7 @@
#define DM_MSG_PREFIX "verity"
-#define DM_VERITY_ENV_LENGTH 42
+#define DM_VERITY_ENV_LENGTH 46
#define DM_VERITY_ENV_VAR_NAME "DM_VERITY_ERR_BLOCK_NR"
#define DM_VERITY_DEFAULT_PREFETCH_SIZE 262144
@@ -238,14 +238,16 @@ static int verity_handle_err(struct dm_verity *v, enum verity_block_type type,
char *envp[] = { verity_env, NULL };
const char *type_str = "";
struct mapped_device *md = dm_table_get_md(v->ti->table);
+ int ce;
/* Corruption should be visible in device status in all modes */
v->hash_failed = true;
- if (v->corrupted_errs >= DM_VERITY_MAX_CORRUPTED_ERRS)
- goto out;
-
- v->corrupted_errs++;
+ ce = atomic_read(&v->corrupted_errs);
+ do {
+ if (ce >= DM_VERITY_MAX_CORRUPTED_ERRS)
+ goto out;
+ } while (!atomic_try_cmpxchg(&v->corrupted_errs, &ce, ce + 1));
switch (type) {
case DM_VERITY_BLOCK_TYPE_DATA:
@@ -261,7 +263,7 @@ static int verity_handle_err(struct dm_verity *v, enum verity_block_type type,
DMERR_LIMIT("%s: %s block %llu is corrupted", v->data_dev->name,
type_str, block);
- if (v->corrupted_errs == DM_VERITY_MAX_CORRUPTED_ERRS) {
+ if (ce + 1 == DM_VERITY_MAX_CORRUPTED_ERRS) {
DMERR("%s: reached maximum errors", v->data_dev->name);
dm_audit_log_target(DM_MSG_PREFIX, "max-corrupted-errors", v->ti, 0);
}
@@ -1238,6 +1240,8 @@ static int verity_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
continue;
} else if (!strcasecmp(arg_name, DM_VERITY_OPT_TASKLET_VERIFY)) {
+ if (v->use_bh_wq)
+ continue;
v->use_bh_wq = true;
static_branch_inc(&use_bh_wq_enabled);
continue;
diff --git a/drivers/md/dm-verity.h b/drivers/md/dm-verity.h
index c996140bda94..47648f185afb 100644
--- a/drivers/md/dm-verity.h
+++ b/drivers/md/dm-verity.h
@@ -65,7 +65,7 @@ struct dm_verity {
unsigned int hash_reqsize; /* the size of temporary space for crypto */
enum verity_mode mode; /* mode for handling verification errors */
enum verity_mode error_mode;/* mode for handling I/O errors */
- unsigned int corrupted_errs;/* Number of errors for corrupted blocks */
+ atomic_t corrupted_errs;/* Number of errors for corrupted blocks */
struct workqueue_struct *verify_wq;
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index eb583df45ecb..4b18151cd813 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -1335,6 +1335,12 @@ static void raid1_read_request(struct mddev *mddev, struct bio *bio,
if (!wait_read_barrier(conf, bio->bi_iter.bi_sector,
bio->bi_opf & REQ_NOWAIT)) {
bio_wouldblock_error(bio);
+
+ if (r1bio_existed) {
+ set_bit(R1BIO_Returned, &r1_bio->state);
+ raid_end_bio_io(r1_bio);
+ }
+
return;
}
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 5cc827cff918..de441867d3bd 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -1730,6 +1730,7 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio)
r10_bio->mddev = mddev;
r10_bio->state = 0;
r10_bio->sectors = 0;
+ r10_bio->read_slot = -1;
memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * geo->raid_disks);
wait_blocked_dev(mddev, r10_bio);
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index f71ed0f53825..b315d36b7f50 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4835,55 +4835,62 @@ static void break_stripe_batch_list(struct stripe_head *head_sh,
{
struct stripe_head *sh, *next;
int i;
+ unsigned long state;
list_for_each_entry_safe(sh, next, &head_sh->batch_list, batch_list) {
list_del_init(&sh->batch_list);
- WARN_ONCE(sh->state & ((1 << STRIPE_ACTIVE) |
- (1 << STRIPE_SYNCING) |
- (1 << STRIPE_REPLACED) |
- (1 << STRIPE_DELAYED) |
- (1 << STRIPE_BIT_DELAY) |
- (1 << STRIPE_FULL_WRITE) |
- (1 << STRIPE_BIOFILL_RUN) |
- (1 << STRIPE_COMPUTE_RUN) |
- (1 << STRIPE_DISCARD) |
- (1 << STRIPE_BATCH_READY) |
- (1 << STRIPE_BATCH_ERR)),
- "stripe state: %lx\n", sh->state);
- WARN_ONCE(head_sh->state & ((1 << STRIPE_DISCARD) |
- (1 << STRIPE_REPLACED)),
- "head stripe state: %lx\n", head_sh->state);
+ state = READ_ONCE(sh->state);
+ WARN_ONCE(state & ((1 << STRIPE_ACTIVE) |
+ (1 << STRIPE_SYNCING) |
+ (1 << STRIPE_REPLACED) |
+ (1 << STRIPE_DELAYED) |
+ (1 << STRIPE_BIT_DELAY) |
+ (1 << STRIPE_FULL_WRITE) |
+ (1 << STRIPE_BIOFILL_RUN) |
+ (1 << STRIPE_COMPUTE_RUN) |
+ (1 << STRIPE_DISCARD) |
+ (1 << STRIPE_BATCH_READY) |
+ (1 << STRIPE_BATCH_ERR)),
+ "stripe state: %lx\n", state);
+
+ state = READ_ONCE(head_sh->state);
+ WARN_ONCE(state & ((1 << STRIPE_DISCARD) |
+ (1 << STRIPE_REPLACED)),
+ "head stripe state: %lx\n", state);
set_mask_bits(&sh->state, ~(STRIPE_EXPAND_SYNC_FLAGS |
(1 << STRIPE_PREREAD_ACTIVE) |
(1 << STRIPE_ON_UNPLUG_LIST)),
- head_sh->state & (1 << STRIPE_INSYNC));
+ state & (1 << STRIPE_INSYNC));
sh->check_state = head_sh->check_state;
sh->reconstruct_state = head_sh->reconstruct_state;
spin_lock_irq(&sh->stripe_lock);
- sh->batch_head = NULL;
- spin_unlock_irq(&sh->stripe_lock);
for (i = 0; i < sh->disks; i++) {
if (test_and_clear_bit(R5_Overlap, &sh->dev[i].flags))
wake_up_bit(&sh->dev[i].flags, R5_Overlap);
- sh->dev[i].flags = head_sh->dev[i].flags &
+ sh->dev[i].flags = READ_ONCE(head_sh->dev[i].flags) &
(~((1 << R5_WriteError) | (1 << R5_Overlap)));
}
- if (handle_flags == 0 ||
- sh->state & handle_flags)
+ sh->batch_head = NULL;
+ spin_unlock_irq(&sh->stripe_lock);
+
+ state = READ_ONCE(sh->state);
+ if (handle_flags == 0 || (state & handle_flags))
set_bit(STRIPE_HANDLE, &sh->state);
raid5_release_stripe(sh);
}
spin_lock_irq(&head_sh->stripe_lock);
- head_sh->batch_head = NULL;
- spin_unlock_irq(&head_sh->stripe_lock);
for (i = 0; i < head_sh->disks; i++)
if (test_and_clear_bit(R5_Overlap, &head_sh->dev[i].flags))
wake_up_bit(&head_sh->dev[i].flags, R5_Overlap);
- if (head_sh->state & handle_flags)
+ head_sh->batch_head = NULL;
+ spin_unlock_irq(&head_sh->stripe_lock);
+
+ state = READ_ONCE(head_sh->state);
+ if (state & handle_flags)
set_bit(STRIPE_HANDLE, &head_sh->state);
}
diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c
index c2013995049c..a161dc326895 100644
--- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c
+++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c
@@ -472,13 +472,14 @@ static int mxc_isi_probe(struct platform_device *pdev)
return ret;
}
- pm_runtime_enable(dev);
+ ret = devm_pm_runtime_enable(dev);
+ if (ret)
+ return ret;
ret = mxc_isi_crossbar_init(isi);
- if (ret) {
- dev_err(dev, "Failed to initialize crossbar: %d\n", ret);
- goto err_pm;
- }
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to initialize crossbar\n");
for (i = 0; i < isi->pdata->num_channels; ++i) {
ret = mxc_isi_pipe_init(isi, i);
@@ -501,8 +502,7 @@ static int mxc_isi_probe(struct platform_device *pdev)
err_xbar:
mxc_isi_crossbar_cleanup(&isi->crossbar);
-err_pm:
- pm_runtime_disable(isi->dev);
+
return ret;
}
@@ -519,10 +519,8 @@ static void mxc_isi_remove(struct platform_device *pdev)
mxc_isi_pipe_cleanup(pipe);
}
- mxc_isi_crossbar_cleanup(&isi->crossbar);
mxc_isi_v4l2_cleanup(isi);
-
- pm_runtime_disable(isi->dev);
+ mxc_isi_crossbar_cleanup(&isi->crossbar);
}
static const struct of_device_id mxc_isi_of_match[] = {
diff --git a/drivers/media/platform/qcom/camss/camss-vfe.c b/drivers/media/platform/qcom/camss/camss-vfe.c
index 8f6b0eccefb4..76839898ace9 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe.c
@@ -1740,7 +1740,7 @@ int msm_vfe_register_entities(struct vfe_device *vfe,
v4l2_subdev_init(sd, &vfe_v4l2_ops);
sd->internal_ops = &vfe_v4l2_internal_ops;
sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
- if (i == VFE_LINE_PIX)
+ if (i == VFE_LINE_PIX && vfe->res->is_lite == false)
snprintf(sd->name, ARRAY_SIZE(sd->name), "%s%d_%s",
MSM_VFE_NAME, vfe->id, "pix");
else
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 8295542e1a7c..d6984129c2f3 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -954,8 +954,8 @@ static u32 get_framesize_raw_nv12(u32 width, u32 height)
uv_sclines = ALIGN(((height + 1) >> 1), 16);
y_plane = y_stride * y_sclines;
- uv_plane = uv_stride * uv_sclines + SZ_4K;
- size = y_plane + uv_plane + SZ_8K;
+ uv_plane = uv_stride * uv_sclines;
+ size = y_plane + uv_plane;
return ALIGN(size, SZ_4K);
}
diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c b/drivers/media/platform/qcom/venus/hfi_platform_v4.c
index e3f0a90a567b..277322d1fa27 100644
--- a/drivers/media/platform/qcom/venus/hfi_platform_v4.c
+++ b/drivers/media/platform/qcom/venus/hfi_platform_v4.c
@@ -135,8 +135,8 @@ static const struct hfi_plat_caps caps[] = {
.codec = HFI_VIDEO_CODEC_H264,
.domain = VIDC_SESSION_TYPE_ENC,
.cap_bufs_mode_dynamic = true,
- .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 96, 4096, 16},
- .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 96, 4096, 16},
+ .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 96, 4096, 1},
+ .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 96, 4096, 1},
.caps[2] = {HFI_CAPABILITY_MBS_PER_FRAME, 1, 36864, 1},
.caps[3] = {HFI_CAPABILITY_BITRATE, 1, 120000000, 1},
.caps[4] = {HFI_CAPABILITY_SCALE_X, 8192, 65536, 1},
@@ -172,8 +172,8 @@ static const struct hfi_plat_caps caps[] = {
.codec = HFI_VIDEO_CODEC_HEVC,
.domain = VIDC_SESSION_TYPE_ENC,
.cap_bufs_mode_dynamic = true,
- .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 96, 4096, 16},
- .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 96, 4096, 16},
+ .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 96, 4096, 1},
+ .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 96, 4096, 1},
.caps[2] = {HFI_CAPABILITY_MBS_PER_FRAME, 1, 36864, 1},
.caps[3] = {HFI_CAPABILITY_BITRATE, 1, 120000000, 1},
.caps[4] = {HFI_CAPABILITY_SCALE_X, 8192, 65536, 1},
@@ -194,8 +194,8 @@ static const struct hfi_plat_caps caps[] = {
.caps[19] = {HFI_CAPABILITY_RATE_CONTROL_MODES, 0x1000001, 0x1000005, 1},
.caps[20] = {HFI_CAPABILITY_COLOR_SPACE_CONVERSION, 0, 2, 1},
.caps[21] = {HFI_CAPABILITY_ROTATION, 1, 4, 90},
- .caps[22] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 16},
- .caps[23] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 16},
+ .caps[22] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 1},
+ .caps[23] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 1},
.num_caps = 24,
.pl[0] = {HFI_HEVC_PROFILE_MAIN, HFI_HEVC_LEVEL_6 | HFI_HEVC_TIER_HIGH0},
.pl[1] = {HFI_HEVC_PROFILE_MAIN10, HFI_HEVC_LEVEL_6 | HFI_HEVC_TIER_HIGH0},
@@ -209,8 +209,8 @@ static const struct hfi_plat_caps caps[] = {
.codec = HFI_VIDEO_CODEC_VP8,
.domain = VIDC_SESSION_TYPE_ENC,
.cap_bufs_mode_dynamic = true,
- .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 96, 4096, 16},
- .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 96, 4096, 16},
+ .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 96, 4096, 1},
+ .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 96, 4096, 1},
.caps[2] = {HFI_CAPABILITY_MBS_PER_FRAME, 1, 36864, 1},
.caps[3] = {HFI_CAPABILITY_BITRATE, 1, 120000000, 1},
.caps[4] = {HFI_CAPABILITY_SCALE_X, 8192, 65536, 1},
@@ -228,8 +228,8 @@ static const struct hfi_plat_caps caps[] = {
.caps[16] = {HFI_CAPABILITY_P_FRAME_QP, 0, 127, 1},
.caps[17] = {HFI_CAPABILITY_MAX_WORKMODES, 1, 2, 1},
.caps[18] = {HFI_CAPABILITY_RATE_CONTROL_MODES, 0x1000001, 0x1000005, 1},
- .caps[19] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 16},
- .caps[20] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 16},
+ .caps[19] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 1},
+ .caps[20] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 1},
.caps[21] = {HFI_CAPABILITY_COLOR_SPACE_CONVERSION, 0, 2, 1},
.caps[22] = {HFI_CAPABILITY_ROTATION, 1, 4, 90},
.num_caps = 23,
diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c b/drivers/media/platform/qcom/venus/hfi_platform_v6.c
index 4e8af645f8b9..b5b769d735c1 100644
--- a/drivers/media/platform/qcom/venus/hfi_platform_v6.c
+++ b/drivers/media/platform/qcom/venus/hfi_platform_v6.c
@@ -172,8 +172,8 @@ static const struct hfi_plat_caps caps[] = {
.codec = HFI_VIDEO_CODEC_HEVC,
.domain = VIDC_SESSION_TYPE_ENC,
.cap_bufs_mode_dynamic = true,
- .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 128, 8192, 16},
- .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 128, 8192, 16},
+ .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 128, 8192, 1},
+ .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 128, 8192, 1},
.caps[2] = {HFI_CAPABILITY_MBS_PER_FRAME, 64, 138240, 1},
.caps[3] = {HFI_CAPABILITY_BITRATE, 1, 160000000, 1},
.caps[4] = {HFI_CAPABILITY_SCALE_X, 8192, 65536, 1},
@@ -194,8 +194,8 @@ static const struct hfi_plat_caps caps[] = {
.caps[19] = {HFI_CAPABILITY_RATE_CONTROL_MODES, 0x1000001, 0x1000005, 1},
.caps[20] = {HFI_CAPABILITY_COLOR_SPACE_CONVERSION, 0, 2, 1},
.caps[21] = {HFI_CAPABILITY_ROTATION, 1, 4, 90},
- .caps[22] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 16},
- .caps[23] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 16},
+ .caps[22] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 1},
+ .caps[23] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 1},
.num_caps = 24,
.pl[0] = {HFI_HEVC_PROFILE_MAIN, HFI_HEVC_LEVEL_6 | HFI_HEVC_TIER_HIGH0},
.pl[1] = {HFI_HEVC_PROFILE_MAIN10, HFI_HEVC_LEVEL_6 | HFI_HEVC_TIER_HIGH0},
@@ -209,8 +209,8 @@ static const struct hfi_plat_caps caps[] = {
.codec = HFI_VIDEO_CODEC_VP8,
.domain = VIDC_SESSION_TYPE_ENC,
.cap_bufs_mode_dynamic = true,
- .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 128, 4096, 16},
- .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 128, 4096, 16},
+ .caps[0] = {HFI_CAPABILITY_FRAME_WIDTH, 128, 4096, 1},
+ .caps[1] = {HFI_CAPABILITY_FRAME_HEIGHT, 128, 4096, 1},
.caps[2] = {HFI_CAPABILITY_MBS_PER_FRAME, 64, 36864, 1},
.caps[3] = {HFI_CAPABILITY_BITRATE, 1, 74000000, 1},
.caps[4] = {HFI_CAPABILITY_SCALE_X, 8192, 65536, 1},
@@ -228,8 +228,8 @@ static const struct hfi_plat_caps caps[] = {
.caps[16] = {HFI_CAPABILITY_P_FRAME_QP, 0, 127, 1},
.caps[17] = {HFI_CAPABILITY_MAX_WORKMODES, 1, 2, 1},
.caps[18] = {HFI_CAPABILITY_RATE_CONTROL_MODES, 0x1000001, 0x1000005, 1},
- .caps[19] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 16},
- .caps[20] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 16},
+ .caps[19] = {HFI_CAPABILITY_BLUR_WIDTH, 96, 4096, 1},
+ .caps[20] = {HFI_CAPABILITY_BLUR_HEIGHT, 96, 4096, 1},
.caps[21] = {HFI_CAPABILITY_COLOR_SPACE_CONVERSION, 0, 2, 1},
.caps[22] = {HFI_CAPABILITY_ROTATION, 1, 4, 90},
.num_caps = 23,
diff --git a/drivers/media/platform/rockchip/rga/rga-hw.c b/drivers/media/platform/rockchip/rga/rga-hw.c
index b2ef3beec525..7de6f80fdce7 100644
--- a/drivers/media/platform/rockchip/rga/rga-hw.c
+++ b/drivers/media/platform/rockchip/rga/rga-hw.c
@@ -414,7 +414,7 @@ static void rga_cmd_set(struct rga_ctx *ctx,
{
struct rockchip_rga *rga = ctx->rga;
- memset(rga->cmdbuf_virt, 0, RGA_CMDBUF_SIZE * 4);
+ memset(rga->cmdbuf_virt, 0, RGA_CMDBUF_SIZE);
rga_cmd_set_src_addr(ctx, src->dma_desc_pa);
/*
diff --git a/drivers/media/platform/rockchip/rga/rga-hw.h b/drivers/media/platform/rockchip/rga/rga-hw.h
index e8917e5630a4..03990fd30f15 100644
--- a/drivers/media/platform/rockchip/rga/rga-hw.h
+++ b/drivers/media/platform/rockchip/rga/rga-hw.h
@@ -6,7 +6,7 @@
#ifndef __RGA_HW_H__
#define __RGA_HW_H__
-#define RGA_CMDBUF_SIZE 0x20
+#define RGA_CMDBUF_SIZE 0x80
/* Hardware limits */
#define MAX_WIDTH 8192
diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 57bccbf17f6d..26a4dccc193c 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -489,6 +489,13 @@ static int uvc_commit_video(struct uvc_streaming *stream,
* Clocks and timestamps
*/
+/*
+ * The accuracy of the hardware timestamping depends on having enough data to
+ * interpolate between the different clock domains. This value is sof cycles,
+ * this is, milliseconds.
+ */
+#define UVC_MIN_HW_TIMESTAMP_DIFF 100
+
static inline ktime_t uvc_video_get_time(void)
{
if (uvc_clock_param == CLOCK_MONOTONIC)
@@ -532,6 +539,15 @@ static void uvc_video_clock_add_sample(struct uvc_clock *clock,
spin_unlock_irqrestore(&clock->lock, flags);
}
+static inline u16 sof_diff(u16 a, u16 b)
+{
+ /*
+ * Because the result is modulo 2048 (via & 2047), we do not need a
+ * special case for a < b.
+ */
+ return (a - b) & 2047;
+}
+
static void
uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf,
const u8 *data, int len)
@@ -578,16 +594,7 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf,
if (!has_scr)
return;
- /*
- * To limit the amount of data, drop SCRs with an SOF identical to the
- * previous one. This filtering is also needed to support UVC 1.5, where
- * all the data packets of the same frame contains the same SOF. In that
- * case only the first one will match the host_sof.
- */
sample.dev_sof = get_unaligned_le16(&data[header_size - 2]);
- if (sample.dev_sof == stream->clock.last_sof)
- return;
-
sample.dev_stc = get_unaligned_le32(&data[header_size - 6]);
/*
@@ -659,6 +666,17 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf,
}
sample.dev_sof = (sample.dev_sof + stream->clock.sof_offset) & 2047;
+
+ /*
+ * To limit the amount of data, drop SCRs with an SOF similar to the
+ * previous one. This filtering is also needed to support UVC 1.5, where
+ * all the data packets of the same frame contains the same SOF. In that
+ * case only the first one will match the host_sof.
+ */
+ if (sof_diff(sample.dev_sof, stream->clock.last_sof) <=
+ (UVC_MIN_HW_TIMESTAMP_DIFF / stream->clock.size))
+ return;
+
uvc_video_clock_add_sample(&stream->clock, &sample);
stream->clock.last_sof = sample.dev_sof;
}
@@ -829,15 +847,22 @@ void uvc_video_clock_update(struct uvc_streaming *stream,
y2 += 2048 << 16;
/*
- * Have at least 1/4 of a second of timestamps before we
- * try to do any calculation. Otherwise we do not have enough
- * precision. This value was determined by running Android CTS
- * on different devices.
+ * If the buffer is not full, we want to gather at least 1/4th of
+ * timestamps before using HW timestamping. We do this to avoid jitter
+ * on the initial frames.
+ *
+ * If the buffer is full we would use it regardless of how much data
+ * it represents. This could be solved with an infinite big circular
+ * buffer, but RAM is expensive these days, specially the infinitely
+ * big.
+ *
+ * The value of UVC_MIN_HW_TIMESTAMP_DIFF was determined by running
+ * Android's CTS on different devices.
*
- * dev_sof runs at 1KHz, and we have a fixed point precision of
- * 16 bits.
+ * y1 and y2 are dev_sof with a fixed point precision of 16 bits.
*/
- if ((y2 - y1) < ((1000 / 4) << 16))
+ if (clock->size != clock->count &&
+ (y2 - y1) < (UVC_MIN_HW_TIMESTAMP_DIFF << 16))
goto done;
y = (u64)(y2 - y1) * (1ULL << 31) + (u64)y1 * (u64)x2
@@ -1145,7 +1170,9 @@ static void uvc_video_stats_stop(struct uvc_streaming *stream)
* uvc_video_decode_end will never be called with a NULL buffer.
*/
static int uvc_video_decode_start(struct uvc_streaming *stream,
- struct uvc_buffer *buf, const u8 *data, int len)
+ struct uvc_buffer *buf,
+ struct uvc_buffer *meta_buf,
+ const u8 *data, int len)
{
u8 header_len;
u8 fid;
@@ -1164,6 +1191,53 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
header_len = data[0];
fid = data[1] & UVC_STREAM_FID;
+ /*
+ * Mark the buffer as done if we're at the beginning of a new frame.
+ * End of frame detection is better implemented by checking the EOF
+ * bit (FID bit toggling is delayed by one frame compared to the EOF
+ * bit), but some devices don't set the bit at end of frame (and the
+ * last payload can be lost anyway). We thus must check if the FID has
+ * been toggled.
+ *
+ * stream->last_fid is initialized to -1, and buf->bytesused to 0,
+ * so the first isochronous frame will never trigger an end of frame
+ * detection.
+ *
+ * Empty buffers (bytesused == 0) don't trigger end of frame detection
+ * as it doesn't make sense to return an empty buffer. This also
+ * avoids detecting end of frame conditions at FID toggling if the
+ * previous payload had the EOF bit set.
+ */
+ if (fid != stream->last_fid && buf && buf->bytesused != 0) {
+ uvc_dbg(stream->dev, FRAME,
+ "Frame complete (FID bit toggled)\n");
+ buf->state = UVC_BUF_STATE_READY;
+
+ return -EAGAIN;
+ }
+
+ /*
+ * Some cameras, when running two parallel streams (one MJPEG alongside
+ * another non-MJPEG stream), are known to lose the EOF packet for a frame.
+ * We can detect the end of a frame by checking for a new SOI marker, as
+ * the SOI always lies on the packet boundary between two frames for
+ * these devices.
+ */
+ if (stream->dev->quirks & UVC_QUIRK_MJPEG_NO_EOF &&
+ (stream->cur_format->fcc == V4L2_PIX_FMT_MJPEG ||
+ stream->cur_format->fcc == V4L2_PIX_FMT_JPEG) &&
+ buf && buf->bytesused != 0) {
+ const u8 *packet = data + header_len;
+
+ if (len >= header_len + 2 &&
+ packet[0] == 0xff && packet[1] == JPEG_MARKER_SOI) {
+ buf->state = UVC_BUF_STATE_READY;
+ buf->error = 1;
+ stream->last_fid ^= UVC_STREAM_FID;
+ return -EAGAIN;
+ }
+ }
+
/*
* Increase the sequence number regardless of any buffer states, so
* that discontinuous sequence numbers always indicate lost frames.
@@ -1172,6 +1246,19 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
stream->sequence++;
if (stream->sequence)
uvc_video_stats_update(stream);
+
+ /*
+ * On a FID flip initialize sequence number and timestamp.
+ *
+ * The driver already takes care of injecting FID flips for
+ * UVC_QUIRK_STREAM_NO_FID and UVC_QUIRK_MJPEG_NO_EOF.
+ */
+ if (buf) {
+ buf->buf.field = V4L2_FIELD_NONE;
+ buf->buf.sequence = stream->sequence;
+ buf->buf.vb2_buf.timestamp =
+ ktime_to_ns(uvc_video_get_time());
+ }
}
uvc_video_clock_decode(stream, buf, data, len);
@@ -1212,57 +1299,10 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
return -ENODATA;
}
- buf->buf.field = V4L2_FIELD_NONE;
- buf->buf.sequence = stream->sequence;
- buf->buf.vb2_buf.timestamp = ktime_to_ns(uvc_video_get_time());
-
/* TODO: Handle PTS and SCR. */
buf->state = UVC_BUF_STATE_ACTIVE;
- }
-
- /*
- * Mark the buffer as done if we're at the beginning of a new frame.
- * End of frame detection is better implemented by checking the EOF
- * bit (FID bit toggling is delayed by one frame compared to the EOF
- * bit), but some devices don't set the bit at end of frame (and the
- * last payload can be lost anyway). We thus must check if the FID has
- * been toggled.
- *
- * stream->last_fid is initialized to -1, so the first isochronous
- * frame will never trigger an end of frame detection.
- *
- * Empty buffers (bytesused == 0) don't trigger end of frame detection
- * as it doesn't make sense to return an empty buffer. This also
- * avoids detecting end of frame conditions at FID toggling if the
- * previous payload had the EOF bit set.
- */
- if (fid != stream->last_fid && buf->bytesused != 0) {
- uvc_dbg(stream->dev, FRAME,
- "Frame complete (FID bit toggled)\n");
- buf->state = UVC_BUF_STATE_READY;
- return -EAGAIN;
- }
-
- /*
- * Some cameras, when running two parallel streams (one MJPEG alongside
- * another non-MJPEG stream), are known to lose the EOF packet for a frame.
- * We can detect the end of a frame by checking for a new SOI marker, as
- * the SOI always lies on the packet boundary between two frames for
- * these devices.
- */
- if (stream->dev->quirks & UVC_QUIRK_MJPEG_NO_EOF &&
- (stream->cur_format->fcc == V4L2_PIX_FMT_MJPEG ||
- stream->cur_format->fcc == V4L2_PIX_FMT_JPEG)) {
- const u8 *packet = data + header_len;
-
- if (len >= header_len + 2 &&
- packet[0] == 0xff && packet[1] == JPEG_MARKER_SOI &&
- buf->bytesused != 0) {
- buf->state = UVC_BUF_STATE_READY;
- buf->error = 1;
- stream->last_fid ^= UVC_STREAM_FID;
- return -EAGAIN;
- }
+ if (meta_buf)
+ meta_buf->state = UVC_BUF_STATE_ACTIVE;
}
stream->last_fid = fid;
@@ -1434,7 +1474,7 @@ static void uvc_video_decode_meta(struct uvc_streaming *stream,
ktime_t time;
const u8 *scr;
- if (!meta_buf || length == 2)
+ if (length <= 2 || !meta_buf || meta_buf->state != UVC_BUF_STATE_ACTIVE)
return;
has_pts = mem[1] & UVC_STREAM_PTS;
@@ -1551,7 +1591,7 @@ static void uvc_video_decode_isoc(struct uvc_urb *uvc_urb,
/* Decode the payload header. */
mem = urb->transfer_buffer + urb->iso_frame_desc[i].offset;
do {
- ret = uvc_video_decode_start(stream, buf, mem,
+ ret = uvc_video_decode_start(stream, buf, meta_buf, mem,
urb->iso_frame_desc[i].actual_length);
if (ret == -EAGAIN)
uvc_video_next_buffers(stream, &buf, &meta_buf);
@@ -1600,7 +1640,8 @@ static void uvc_video_decode_bulk(struct uvc_urb *uvc_urb,
*/
if (stream->bulk.header_size == 0 && !stream->bulk.skip_payload) {
do {
- ret = uvc_video_decode_start(stream, buf, mem, len);
+ ret = uvc_video_decode_start(stream, buf, meta_buf, mem,
+ len);
if (ret == -EAGAIN)
uvc_video_next_buffers(stream, &buf, &meta_buf);
} while (ret == -EAGAIN);
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index ad0b9d5fbee7..143ca5284b79 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -269,6 +269,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
{ .format = V4L2_PIX_FMT_Y212, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_Y216, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_YUV48_12, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_YUV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 3, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_MT2110T, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2,
.block_w = { 16, 8, 0, 0 }, .block_h = { 32, 16, 0, 0 }},
{ .format = V4L2_PIX_FMT_MT2110R, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2,
diff --git a/drivers/memory/tegra/mc.c b/drivers/memory/tegra/mc.c
index bd5b58f1fd42..7a18d6e3b368 100644
--- a/drivers/memory/tegra/mc.c
+++ b/drivers/memory/tegra/mc.c
@@ -13,6 +13,7 @@
#include <linux/of.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
+#include <linux/pm.h>
#include <linux/slab.h>
#include <linux/sort.h>
#include <linux/tegra-icc.h>
@@ -986,10 +987,23 @@ static void tegra_mc_sync_state(struct device *dev)
icc_sync_state(dev);
}
+static int tegra_mc_resume(struct device *dev)
+{
+ struct tegra_mc *mc = dev_get_drvdata(dev);
+
+ if (mc->soc->ops && mc->soc->ops->resume)
+ mc->soc->ops->resume(mc);
+
+ return 0;
+}
+
+static DEFINE_SIMPLE_DEV_PM_OPS(tegra_mc_pm_ops, NULL, tegra_mc_resume);
+
static struct platform_driver tegra_mc_driver = {
.driver = {
.name = "tegra-mc",
.of_match_table = tegra_mc_of_match,
+ .pm = pm_sleep_ptr(&tegra_mc_pm_ops),
.suppress_bind_attrs = true,
.sync_state = tegra_mc_sync_state,
},
diff --git a/drivers/memstick/core/ms_block.c b/drivers/memstick/core/ms_block.c
index 47a314a4eb6f..a1a841c3bbf8 100644
--- a/drivers/memstick/core/ms_block.c
+++ b/drivers/memstick/core/ms_block.c
@@ -1339,6 +1339,10 @@ static int msb_ftl_initialize(struct msb_data *msb)
return 0;
msb->zone_count = msb->block_count / MS_BLOCKS_IN_ZONE;
+ if (msb->block_count > MS_MAX_ZONES * MS_BLOCKS_IN_ZONE) {
+ pr_err("Too many blocks: %d\n", msb->block_count);
+ return -EINVAL;
+ }
msb->logical_block_count = msb->zone_count * 496 - 2;
msb->used_blocks_bitmap = bitmap_zalloc(msb->block_count, GFP_KERNEL);
diff --git a/drivers/mfd/cs42l43.c b/drivers/mfd/cs42l43.c
index ae8fd37afb75..a3e4a61cf2c1 100644
--- a/drivers/mfd/cs42l43.c
+++ b/drivers/mfd/cs42l43.c
@@ -675,7 +675,7 @@ static void cs42l43_mcu_load_firmware(const struct firmware *firmware, void *con
unsigned int loadaddr, val;
int ret;
- if (!firmware) {
+ if (!firmware || firmware->size < sizeof(*hdr)) {
dev_err(cs42l43->dev, "Failed to load firmware\n");
cs42l43->firmware_error = -ENODEV;
goto err;
diff --git a/drivers/mfd/rsmu_i2c.c b/drivers/mfd/rsmu_i2c.c
index cba64f107a2f..9e5fc8259eec 100644
--- a/drivers/mfd/rsmu_i2c.c
+++ b/drivers/mfd/rsmu_i2c.c
@@ -134,14 +134,10 @@ static int rsmu_i2c_write_device(struct rsmu_ddata *rsmu, u8 reg, u8 *buf, u8 by
static int rsmu_write_page_register(struct rsmu_ddata *rsmu, u32 reg,
rsmu_rw_device rsmu_write_device)
{
- u32 page = reg & RSMU_CM_PAGE_MASK;
+ u32 page = (reg | RSMU_CM_SCSR_BASE) & RSMU_CM_PAGE_MASK;
u8 buf[4];
int err;
- /* Do not modify offset register for none-scsr registers */
- if (reg < RSMU_CM_SCSR_BASE)
- return 0;
-
/* Simply return if we are on the same page */
if (rsmu->page == page)
return 0;
diff --git a/drivers/mfd/rsmu_spi.c b/drivers/mfd/rsmu_spi.c
index 39d9be1e141f..c931d8cea0a1 100644
--- a/drivers/mfd/rsmu_spi.c
+++ b/drivers/mfd/rsmu_spi.c
@@ -101,11 +101,8 @@ static int rsmu_write_page_register(struct rsmu_ddata *rsmu, u32 reg)
switch (rsmu->type) {
case RSMU_CM:
- /* Do not modify page register for none-scsr registers */
- if (reg < RSMU_CM_SCSR_BASE)
- return 0;
page_reg = RSMU_CM_PAGE_ADDR;
- page = reg & RSMU_PAGE_MASK;
+ page = (reg | RSMU_CM_SCSR_BASE) & RSMU_PAGE_MASK;
buf[0] = (u8)(page & 0xFF);
buf[1] = (u8)((page >> 8) & 0xFF);
buf[2] = (u8)((page >> 16) & 0xFF);
diff --git a/drivers/mfd/sm501.c b/drivers/mfd/sm501.c
index 5b6dc1cb9bfc..06e796883189 100644
--- a/drivers/mfd/sm501.c
+++ b/drivers/mfd/sm501.c
@@ -747,9 +747,11 @@ static int sm501_register_device(struct sm501_devdata *sm,
if (ret >= 0) {
dev_dbg(sm->dev, "registered %s\n", pdev->name);
list_add_tail(&smdev->list, &sm->devices);
- } else
+ } else {
dev_err(sm->dev, "error registering %s (%d)\n",
pdev->name, ret);
+ platform_device_put(pdev);
+ }
return ret;
}
diff --git a/drivers/mfd/tps6586x.c b/drivers/mfd/tps6586x.c
index 82714899efb2..85591e487dab 100644
--- a/drivers/mfd/tps6586x.c
+++ b/drivers/mfd/tps6586x.c
@@ -397,7 +397,7 @@ static int tps6586x_add_subdevs(struct tps6586x *tps6586x,
pdev->dev.parent = tps6586x->dev;
pdev->dev.platform_data = subdev->platform_data;
- pdev->dev.of_node = subdev->of_node;
+ pdev->dev.of_node = of_node_get(subdev->of_node);
ret = platform_device_add(pdev);
if (ret) {
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index d91b3a9cf376..3aefe2cbc085 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -2768,7 +2768,6 @@ static void mmc_blk_rpmb_device_release(struct device *dev)
{
struct mmc_rpmb_data *rpmb = dev_get_drvdata(dev);
- rpmb_dev_unregister(rpmb->rdev);
mmc_blk_put(rpmb->md);
ida_free(&mmc_rpmb_ida, rpmb->id);
kfree(rpmb);
@@ -2989,8 +2988,8 @@ static int mmc_blk_alloc_rpmb_part(struct mmc_card *card,
}
static void mmc_blk_remove_rpmb_part(struct mmc_rpmb_data *rpmb)
-
{
+ rpmb_dev_unregister(rpmb->rdev);
cdev_device_del(&rpmb->chrdev, &rpmb->dev);
put_device(&rpmb->dev);
}
diff --git a/drivers/mmc/host/sdhci-of-dwcmshc.c b/drivers/mmc/host/sdhci-of-dwcmshc.c
index da42f27fc178..4a40a43c3c6d 100644
--- a/drivers/mmc/host/sdhci-of-dwcmshc.c
+++ b/drivers/mmc/host/sdhci-of-dwcmshc.c
@@ -1464,13 +1464,16 @@ static int dwcmshc_probe(struct platform_device *pdev)
goto free_pltfm;
priv->bus_clk = devm_clk_get(dev, "bus");
- if (!IS_ERR(priv->bus_clk))
- clk_prepare_enable(priv->bus_clk);
+ if (!IS_ERR(priv->bus_clk)) {
+ err = clk_prepare_enable(priv->bus_clk);
+ if (err)
+ goto err_clk;
+ }
}
err = mmc_of_parse(host->mmc);
if (err)
- goto err_clk;
+ goto err_bus_clk;
sdhci_get_of_property(pdev);
@@ -1484,7 +1487,7 @@ static int dwcmshc_probe(struct platform_device *pdev)
if (pltfm_data->init) {
err = pltfm_data->init(&pdev->dev, host, priv);
if (err)
- goto err_clk;
+ goto err_bus_clk;
}
#ifdef CONFIG_ACPI
@@ -1530,9 +1533,10 @@ static int dwcmshc_probe(struct platform_device *pdev)
err_rpm:
pm_runtime_disable(dev);
pm_runtime_put_noidle(dev);
+err_bus_clk:
+ clk_disable_unprepare(priv->bus_clk);
err_clk:
clk_disable_unprepare(pltfm_host->clk);
- clk_disable_unprepare(priv->bus_clk);
clk_bulk_disable_unprepare(priv->num_other_clks, priv->other_clks);
free_pltfm:
sdhci_pltfm_free(pdev);
diff --git a/drivers/mmc/host/vub300.c b/drivers/mmc/host/vub300.c
index 68a5f213c5fb..35393f534de2 100644
--- a/drivers/mmc/host/vub300.c
+++ b/drivers/mmc/host/vub300.c
@@ -1585,7 +1585,7 @@ static int __command_write_data(struct vub300_mmc_host *vub300,
return linear_length;
}
-static void __vub300_command_response(struct vub300_mmc_host *vub300,
+static bool __vub300_command_response(struct vub300_mmc_host *vub300,
struct mmc_command *cmd,
struct mmc_data *data, int data_length)
{
@@ -1597,17 +1597,11 @@ static void __vub300_command_response(struct vub300_mmc_host *vub300,
msecs_to_jiffies(msec_timeout));
if (respretval == 0) { /* TIMED OUT */
/* we don't know which of "out" and "res" if any failed */
- int result;
vub300->usb_timed_out = 1;
usb_kill_urb(vub300->command_out_urb);
usb_kill_urb(vub300->command_res_urb);
cmd->error = -ETIMEDOUT;
- result = usb_lock_device_for_reset(vub300->udev,
- vub300->interface);
- if (result == 0) {
- result = usb_reset_device(vub300->udev);
- usb_unlock_device(vub300->udev);
- }
+ return true;
} else if (respretval < 0) {
/* we don't know which of "out" and "res" if any failed */
usb_kill_urb(vub300->command_out_urb);
@@ -1703,6 +1697,8 @@ static void __vub300_command_response(struct vub300_mmc_host *vub300,
} else {
cmd->error = -EINVAL;
}
+
+ return false;
}
static void construct_request_response(struct vub300_mmc_host *vub300,
@@ -1748,6 +1744,7 @@ static void vub300_cmndwork_thread(struct work_struct *work)
struct mmc_request *req = vub300->req;
struct mmc_command *cmd = vub300->cmd;
struct mmc_data *data = vub300->data;
+ bool reset_device;
int data_length;
mutex_lock(&vub300->cmd_mutex);
init_completion(&vub300->command_complete);
@@ -1770,7 +1767,8 @@ static void vub300_cmndwork_thread(struct work_struct *work)
data_length = __command_read_data(vub300, cmd, data);
else
data_length = __command_write_data(vub300, cmd, data);
- __vub300_command_response(vub300, cmd, data, data_length);
+ reset_device = __vub300_command_response(vub300, cmd,
+ data, data_length);
vub300->req = NULL;
vub300->cmd = NULL;
vub300->data = NULL;
@@ -1778,6 +1776,16 @@ static void vub300_cmndwork_thread(struct work_struct *work)
if (cmd->error == -ENOMEDIUM)
check_vub300_port_status(vub300);
mutex_unlock(&vub300->cmd_mutex);
+ if (reset_device) {
+ int result;
+
+ result = usb_lock_device_for_reset(vub300->udev,
+ vub300->interface);
+ if (result == 0) {
+ result = usb_reset_device(vub300->udev);
+ usb_unlock_device(vub300->udev);
+ }
+ }
mmc_request_done(vub300->mmc, req);
kref_put(&vub300->kref, vub300_delete);
return;
diff --git a/drivers/mtd/devices/mchp23k256.c b/drivers/mtd/devices/mchp23k256.c
index cef5f9677d39..66e058df4c32 100644
--- a/drivers/mtd/devices/mchp23k256.c
+++ b/drivers/mtd/devices/mchp23k256.c
@@ -188,7 +188,7 @@ static int mchp23k256_probe(struct spi_device *spi)
data = dev_get_platdata(&spi->dev);
- flash->caps = of_device_get_match_data(&spi->dev);
+ flash->caps = spi_get_device_match_data(spi);
if (!flash->caps)
flash->caps = &mchp23k256_caps;
diff --git a/drivers/mtd/devices/slram.c b/drivers/mtd/devices/slram.c
index 8297b366a066..a476e53562a4 100644
--- a/drivers/mtd/devices/slram.c
+++ b/drivers/mtd/devices/slram.c
@@ -129,6 +129,7 @@ static int slram_write(struct mtd_info *mtd, loff_t to, size_t len,
static int register_device(char *name, unsigned long start, unsigned long length)
{
slram_mtd_list_t **curmtd;
+ int ret = -ENOMEM;
curmtd = &slram_mtdlist;
while (*curmtd) {
@@ -155,14 +156,15 @@ static int register_device(char *name, unsigned long start, unsigned long length
if (!(*curmtd)->mtdinfo) {
E("slram: Cannot allocate new MTD device.\n");
- return(-ENOMEM);
+ goto err_free_list;
}
if (!(((slram_priv_t *)(*curmtd)->mtdinfo->priv)->start =
memremap(start, length,
MEMREMAP_WB | MEMREMAP_WT | MEMREMAP_WC))) {
E("slram: memremap failed\n");
- return -EIO;
+ ret = -EIO;
+ goto err_free_priv;
}
((slram_priv_t *)(*curmtd)->mtdinfo->priv)->end =
((slram_priv_t *)(*curmtd)->mtdinfo->priv)->start + length;
@@ -183,10 +185,8 @@ static int register_device(char *name, unsigned long start, unsigned long length
if (mtd_device_register((*curmtd)->mtdinfo, NULL, 0)) {
E("slram: Failed to register new device\n");
- memunmap(((slram_priv_t *)(*curmtd)->mtdinfo->priv)->start);
- kfree((*curmtd)->mtdinfo->priv);
- kfree((*curmtd)->mtdinfo);
- return(-EAGAIN);
+ ret = -EAGAIN;
+ goto err_unmap;
}
T("slram: Registered device %s from %luKiB to %luKiB\n", name,
(start / 1024), ((start + length) / 1024));
@@ -194,6 +194,16 @@ static int register_device(char *name, unsigned long start, unsigned long length
((slram_priv_t *)(*curmtd)->mtdinfo->priv)->start,
((slram_priv_t *)(*curmtd)->mtdinfo->priv)->end);
return(0);
+
+err_unmap:
+ memunmap(((slram_priv_t *)(*curmtd)->mtdinfo->priv)->start);
+err_free_priv:
+ kfree((*curmtd)->mtdinfo->priv);
+err_free_list:
+ kfree((*curmtd)->mtdinfo);
+ kfree(*curmtd);
+ *curmtd = NULL;
+ return ret;
}
static void unregister_devices(void)
diff --git a/drivers/mtd/maps/vmu-flash.c b/drivers/mtd/maps/vmu-flash.c
index 53019d313db7..3667faefe6c5 100644
--- a/drivers/mtd/maps/vmu-flash.c
+++ b/drivers/mtd/maps/vmu-flash.c
@@ -547,6 +547,7 @@ static void vmu_queryblocks(struct mapleq *mq)
mpart->partition = card->partition;
mtd_cur->priv = mpart;
mtd_cur->owner = THIS_MODULE;
+ mtd_cur->dev.parent = &mdev->dev;
pcache = kzalloc(sizeof(struct vmu_cache), GFP_KERNEL);
if (!pcache)
diff --git a/drivers/mtd/nand/onenand/onenand_samsung.c b/drivers/mtd/nand/onenand/onenand_samsung.c
index 0e21d443078e..5a7b8508e066 100644
--- a/drivers/mtd/nand/onenand/onenand_samsung.c
+++ b/drivers/mtd/nand/onenand/onenand_samsung.c
@@ -554,6 +554,9 @@ static int s5pc110_dma_poll(dma_addr_t dst, dma_addr_t src, size_t count, int di
} while (!(status & S5PC110_DMA_TRANS_STATUS_TD) &&
time_before(jiffies, timeout));
+ if (!(status & S5PC110_DMA_TRANS_STATUS_TD))
+ return -ETIMEDOUT;
+
writel(S5PC110_DMA_TRANS_CMD_TDC, base + S5PC110_DMA_TRANS_CMD);
return 0;
@@ -608,7 +611,9 @@ static int s5pc110_dma_irq(dma_addr_t dst, dma_addr_t src, size_t count, int dir
writel(S5PC110_DMA_TRANS_CMD_TR, base + S5PC110_DMA_TRANS_CMD);
- wait_for_completion_timeout(&onenand->complete, msecs_to_jiffies(20));
+ if (!wait_for_completion_timeout(&onenand->complete,
+ msecs_to_jiffies(20)))
+ return -ETIMEDOUT;
return 0;
}
diff --git a/drivers/mtd/nand/raw/fsl_ifc_nand.c b/drivers/mtd/nand/raw/fsl_ifc_nand.c
index f0e2318ce088..d3c58c3aab1a 100644
--- a/drivers/mtd/nand/raw/fsl_ifc_nand.c
+++ b/drivers/mtd/nand/raw/fsl_ifc_nand.c
@@ -683,8 +683,15 @@ static int fsl_ifc_read_page(struct nand_chip *chip, uint8_t *buf,
return check_erased_page(chip, buf);
}
- if (ctrl->nand_stat != IFC_NAND_EVTER_STAT_OPC)
+ if (!ctrl->nand_stat) {
mtd->ecc_stats.failed++;
+ return -ETIMEDOUT;
+ }
+
+ if (ctrl->nand_stat != IFC_NAND_EVTER_STAT_OPC) {
+ mtd->ecc_stats.failed++;
+ return -EIO;
+ }
return nctrl->max_bitflips;
}
diff --git a/drivers/mtd/nand/raw/lpc32xx_mlc.c b/drivers/mtd/nand/raw/lpc32xx_mlc.c
index b9c3adc54c01..6fa72e6ed8e1 100644
--- a/drivers/mtd/nand/raw/lpc32xx_mlc.c
+++ b/drivers/mtd/nand/raw/lpc32xx_mlc.c
@@ -396,6 +396,7 @@ static int lpc32xx_xmit_dma(struct mtd_info *mtd, void *mem, int len,
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
struct dma_async_tx_descriptor *desc;
int flags = DMA_CTRL_ACK | DMA_PREP_INTERRUPT;
+ unsigned long time_left;
int res;
sg_init_one(&host->sgl, mem, len);
@@ -410,6 +411,7 @@ static int lpc32xx_xmit_dma(struct mtd_info *mtd, void *mem, int len,
flags);
if (!desc) {
dev_err(mtd->dev.parent, "Failed to prepare slave sg\n");
+ res = -ENXIO;
goto out1;
}
@@ -420,7 +422,13 @@ static int lpc32xx_xmit_dma(struct mtd_info *mtd, void *mem, int len,
dmaengine_submit(desc);
dma_async_issue_pending(host->dma_chan);
- wait_for_completion_timeout(&host->comp_dma, msecs_to_jiffies(1000));
+ time_left = wait_for_completion_timeout(&host->comp_dma,
+ msecs_to_jiffies(1000));
+ if (!time_left) {
+ dmaengine_terminate_sync(host->dma_chan);
+ res = -ETIMEDOUT;
+ goto out1;
+ }
dma_unmap_sg(host->dma_chan->device->dev, &host->sgl, 1,
DMA_BIDIRECTIONAL);
@@ -428,7 +436,7 @@ static int lpc32xx_xmit_dma(struct mtd_info *mtd, void *mem, int len,
out1:
dma_unmap_sg(host->dma_chan->device->dev, &host->sgl, 1,
DMA_BIDIRECTIONAL);
- return -ENXIO;
+ return res;
}
static int lpc32xx_read_page(struct nand_chip *chip, uint8_t *buf,
diff --git a/drivers/mtd/nand/raw/lpc32xx_slc.c b/drivers/mtd/nand/raw/lpc32xx_slc.c
index 09d6c4f90d85..f057ac132b85 100644
--- a/drivers/mtd/nand/raw/lpc32xx_slc.c
+++ b/drivers/mtd/nand/raw/lpc32xx_slc.c
@@ -430,6 +430,7 @@ static int lpc32xx_xmit_dma(struct mtd_info *mtd, dma_addr_t dma,
struct dma_async_tx_descriptor *desc;
int flags = DMA_CTRL_ACK | DMA_PREP_INTERRUPT;
int res;
+ unsigned long time_left;
host->dma_slave_config.direction = dir;
host->dma_slave_config.src_addr = dma;
@@ -467,12 +468,19 @@ static int lpc32xx_xmit_dma(struct mtd_info *mtd, dma_addr_t dma,
dmaengine_submit(desc);
dma_async_issue_pending(host->dma_chan);
- wait_for_completion_timeout(&host->comp, msecs_to_jiffies(1000));
+ time_left = wait_for_completion_timeout(&host->comp,
+ msecs_to_jiffies(1000));
+ if (!time_left) {
+ dmaengine_terminate_sync(host->dma_chan);
+ res = -ETIMEDOUT;
+ } else {
+ res = 0;
+ }
dma_unmap_sg(host->dma_chan->device->dev, &host->sgl, 1,
DMA_BIDIRECTIONAL);
- return 0;
+ return res;
out1:
dma_unmap_sg(host->dma_chan->device->dev, &host->sgl, 1,
DMA_BIDIRECTIONAL);
diff --git a/drivers/mtd/nand/raw/nand_base.c b/drivers/mtd/nand/raw/nand_base.c
index d654e5f52ee0..9958b14d0437 100644
--- a/drivers/mtd/nand/raw/nand_base.c
+++ b/drivers/mtd/nand/raw/nand_base.c
@@ -174,7 +174,7 @@ void nand_select_target(struct nand_chip *chip, unsigned int cs)
* cs should always lie between 0 and nanddev_ntargets(), when that's
* not the case it's a bug and the caller should be fixed.
*/
- if (WARN_ON(cs > nanddev_ntargets(&chip->base)))
+ if (WARN_ON(cs >= nanddev_ntargets(&chip->base)))
return;
chip->cur_cs = cs;
@@ -1215,32 +1215,32 @@ static int nand_lp_exec_read_page_op(struct nand_chip *chip, unsigned int page,
return nand_exec_op(chip, &op);
}
-static unsigned int rawnand_last_page_of_lun(unsigned int pages_per_lun, unsigned int lun)
+static unsigned int rawnand_last_page_of_block(unsigned int ppb, unsigned int block)
{
- /* lun is expected to be very small */
- return (lun * pages_per_lun) + pages_per_lun - 1;
+ /* block is expected to be very small */
+ return (block * ppb) + ppb - 1;
}
static void rawnand_cap_cont_reads(struct nand_chip *chip)
{
struct nand_memory_organization *memorg;
- unsigned int ppl, first_lun, last_lun;
+ unsigned int ppb, first_block, last_block;
memorg = nanddev_get_memorg(&chip->base);
- ppl = memorg->pages_per_eraseblock * memorg->eraseblocks_per_lun;
- first_lun = chip->cont_read.first_page / ppl;
- last_lun = chip->cont_read.last_page / ppl;
+ ppb = memorg->pages_per_eraseblock;
+ first_block = chip->cont_read.first_page / ppb;
+ last_block = chip->cont_read.last_page / ppb;
- /* Prevent sequential cache reads across LUN boundaries */
- if (first_lun != last_lun)
- chip->cont_read.pause_page = rawnand_last_page_of_lun(ppl, first_lun);
+ /* Prevent sequential cache reads across block boundaries */
+ if (first_block != last_block)
+ chip->cont_read.pause_page = rawnand_last_page_of_block(ppb, first_block);
else
chip->cont_read.pause_page = chip->cont_read.last_page;
if (chip->cont_read.first_page == chip->cont_read.pause_page) {
chip->cont_read.first_page++;
chip->cont_read.pause_page = min(chip->cont_read.last_page,
- rawnand_last_page_of_lun(ppl, first_lun + 1));
+ rawnand_last_page_of_block(ppb, first_block + 1));
}
if (chip->cont_read.first_page >= chip->cont_read.last_page)
diff --git a/drivers/mtd/nand/raw/pl35x-nand-controller.c b/drivers/mtd/nand/raw/pl35x-nand-controller.c
index ffbf769e4fc0..5691d78cab98 100644
--- a/drivers/mtd/nand/raw/pl35x-nand-controller.c
+++ b/drivers/mtd/nand/raw/pl35x-nand-controller.c
@@ -1152,7 +1152,7 @@ static int pl35x_nand_probe(struct platform_device *pdev)
nfc->controller.ops = &pl35x_nandc_ops;
INIT_LIST_HEAD(&nfc->chips);
- nfc->conf_regs = devm_ioremap_resource(&smc_amba->dev, &smc_amba->res);
+ nfc->conf_regs = devm_ioremap_resource(nfc->dev, &smc_amba->res);
if (IS_ERR(nfc->conf_regs))
return PTR_ERR(nfc->conf_regs);
diff --git a/drivers/mtd/spi-nor/Kconfig b/drivers/mtd/spi-nor/Kconfig
index 24cd25de2b8b..fd05a24d64a9 100644
--- a/drivers/mtd/spi-nor/Kconfig
+++ b/drivers/mtd/spi-nor/Kconfig
@@ -1,7 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
menuconfig MTD_SPI_NOR
tristate "SPI NOR device support"
- depends on MTD
depends on MTD && SPI_MASTER
select SPI_MEM
help
diff --git a/drivers/mtd/spi-nor/spansion.c b/drivers/mtd/spi-nor/spansion.c
index fcd081b75784..36407ea90f92 100644
--- a/drivers/mtd/spi-nor/spansion.c
+++ b/drivers/mtd/spi-nor/spansion.c
@@ -674,7 +674,9 @@ static int s25hx_t_late_init(struct spi_nor *nor)
params->ready = cypress_nor_sr_ready_and_clear;
cypress_nor_ecc_init(nor);
- params->die_erase_opcode = SPINOR_OP_CYPRESS_DIE_ERASE;
+ if (params->n_dice > 1)
+ params->die_erase_opcode = SPINOR_OP_CYPRESS_DIE_ERASE;
+
return 0;
}
diff --git a/drivers/mtd/spi-nor/swp.c b/drivers/mtd/spi-nor/swp.c
index 7208be75d850..686ca5728025 100644
--- a/drivers/mtd/spi-nor/swp.c
+++ b/drivers/mtd/spi-nor/swp.c
@@ -281,8 +281,15 @@ static int spi_nor_sr_unlock(struct spi_nor *nor, loff_t ofs, u64 len)
/* Prefer top, if both are valid */
use_top = can_be_top;
- /* lock_len: length of region that should remain locked */
- if (use_top)
+ /*
+ * lock_len: length of region that should remain locked.
+ *
+ * When can_be_top and can_be_bottom booleans are true, both adjacent
+ * regions are unlocked, thus the entire flash can be unlocked.
+ */
+ if (can_be_top && can_be_bottom)
+ lock_len = 0;
+ else if (use_top)
lock_len = nor->params->size - (ofs + len);
else
lock_len = ofs;
diff --git a/drivers/net/amt.c b/drivers/net/amt.c
index 0433a0f36d1b..ef6848c2e608 100644
--- a/drivers/net/amt.c
+++ b/drivers/net/amt.c
@@ -3297,9 +3297,9 @@ static size_t amt_get_size(const struct net_device *dev)
nla_total_size(sizeof(__u16)) + /* IFLA_AMT_GATEWAY_PORT */
nla_total_size(sizeof(__u32)) + /* IFLA_AMT_LINK */
nla_total_size(sizeof(__u32)) + /* IFLA_MAX_TUNNELS */
- nla_total_size(sizeof(struct iphdr)) + /* IFLA_AMT_DISCOVERY_IP */
- nla_total_size(sizeof(struct iphdr)) + /* IFLA_AMT_REMOTE_IP */
- nla_total_size(sizeof(struct iphdr)); /* IFLA_AMT_LOCAL_IP */
+ nla_total_size(sizeof(__be32)) + /* IFLA_AMT_DISCOVERY_IP */
+ nla_total_size(sizeof(__be32)) + /* IFLA_AMT_REMOTE_IP */
+ nla_total_size(sizeof(__be32)); /* IFLA_AMT_LOCAL_IP */
}
static int amt_fill_info(struct sk_buff *skb, const struct net_device *dev)
diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c
index b777902ed850..c3332d3d3802 100644
--- a/drivers/net/bonding/bond_3ad.c
+++ b/drivers/net/bonding/bond_3ad.c
@@ -1284,6 +1284,7 @@ static void ad_rx_machine(struct lacpdu *lacpdu, struct port *port)
fallthrough;
case AD_RX_PORT_DISABLED:
port->sm_vars &= ~AD_PORT_MATCHED;
+ port->partner_oper.port_state &= ~LACP_STATE_SYNCHRONIZATION;
break;
case AD_RX_LACP_DISABLED:
port->sm_vars &= ~AD_PORT_SELECTED;
diff --git a/drivers/net/can/usb/esd_usb.c b/drivers/net/can/usb/esd_usb.c
index fdf2136824b3..75494260dd01 100644
--- a/drivers/net/can/usb/esd_usb.c
+++ b/drivers/net/can/usb/esd_usb.c
@@ -1365,10 +1365,13 @@ static void esd_usb_disconnect(struct usb_interface *intf)
if (dev->nets[i]) {
netdev = dev->nets[i]->netdev;
unregister_netdev(netdev);
- free_candev(netdev);
}
}
unlink_all_urbs(dev);
+ for (i = 0; i < dev->net_count; i++) {
+ if (dev->nets[i])
+ free_candev(dev->nets[i]->netdev);
+ }
kfree(dev);
}
}
diff --git a/drivers/net/dsa/qca/qca8k-leds.c b/drivers/net/dsa/qca/qca8k-leds.c
index 43ac68052baf..ef496e345a4e 100644
--- a/drivers/net/dsa/qca/qca8k-leds.c
+++ b/drivers/net/dsa/qca/qca8k-leds.c
@@ -429,7 +429,8 @@ qca8k_parse_port_leds(struct qca8k_priv *priv, struct fwnode_handle *port, int p
init_data.fwnode = led;
init_data.devname_mandatory = true;
init_data.devicename = kasprintf(GFP_KERNEL, "%s:0%d",
- priv->internal_mdio_bus->id,
+ priv->internal_mdio_bus ?
+ priv->internal_mdio_bus->id : priv->bus->id,
port_num);
if (!init_data.devicename) {
fwnode_handle_put(led);
diff --git a/drivers/net/dsa/realtek/rtl8366rb-leds.c b/drivers/net/dsa/realtek/rtl8366rb-leds.c
index 509ffd3f8db5..ba50d311cb15 100644
--- a/drivers/net/dsa/realtek/rtl8366rb-leds.c
+++ b/drivers/net/dsa/realtek/rtl8366rb-leds.c
@@ -89,6 +89,7 @@ static int rtl8366rb_setup_led(struct realtek_priv *priv, struct dsa_port *dp,
struct led_init_data init_data = { };
enum led_default_state state;
struct rtl8366rb_led *led;
+ char name[64];
u32 led_group;
int ret;
@@ -129,10 +130,9 @@ static int rtl8366rb_setup_led(struct realtek_priv *priv, struct dsa_port *dp,
init_data.fwnode = led_fwnode;
init_data.devname_mandatory = true;
- init_data.devicename = kasprintf(GFP_KERNEL, "Realtek-%d:0%d:%d",
- dp->ds->index, dp->index, led_group);
- if (!init_data.devicename)
- return -ENOMEM;
+ snprintf(name, sizeof(name), "Realtek-%d:0%d:%d",
+ dp->ds->index, dp->index, led_group);
+ init_data.devicename = name;
ret = devm_led_classdev_register_ext(priv->dev, &led->cdev, &init_data);
if (ret) {
diff --git a/drivers/net/dsa/sja1105/sja1105_ptp.c b/drivers/net/dsa/sja1105/sja1105_ptp.c
index a1f4ca6ad888..cc47e9b5623c 100644
--- a/drivers/net/dsa/sja1105/sja1105_ptp.c
+++ b/drivers/net/dsa/sja1105/sja1105_ptp.c
@@ -755,7 +755,7 @@ static int sja1105_per_out_enable(struct sja1105_private *priv,
* 2 edges on PTP_CLK. So check for truncation which happens
* at periods larger than around 68.7 seconds.
*/
- pin_duration = ns_to_sja1105_ticks(pin_duration / 2);
+ pin_duration = max_t(u64, ns_to_sja1105_ticks(pin_duration / 2), 1);
if (pin_duration > U32_MAX) {
rc = -ERANGE;
goto out;
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
index c5b50cfa935a..9b77dc4a146a 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
@@ -775,6 +775,18 @@ static void ena_destroy_all_tx_queues(struct ena_adapter *adapter)
}
}
+static void ena_destroy_xdp_tx_queues(struct ena_adapter *adapter)
+{
+ u16 ena_qid;
+ int i;
+
+ for (i = adapter->xdp_first_ring;
+ i < adapter->xdp_first_ring + adapter->xdp_num_queues; i++) {
+ ena_qid = ENA_IO_TXQ_IDX(i);
+ ena_com_destroy_io_queue(adapter->ena_dev, ena_qid);
+ }
+}
+
static void ena_destroy_all_rx_queues(struct ena_adapter *adapter)
{
u16 ena_qid;
@@ -2074,14 +2086,21 @@ static int create_queues_with_size_backoff(struct ena_adapter *adapter)
rc = ena_setup_tx_resources_in_range(adapter,
0,
adapter->num_io_queues);
- if (rc)
+ if (rc) {
+ ena_destroy_xdp_tx_queues(adapter);
+ ena_free_all_io_tx_resources_in_range(adapter,
+ adapter->xdp_first_ring,
+ adapter->xdp_num_queues);
goto err_setup_tx;
+ }
rc = ena_create_io_tx_queues_in_range(adapter,
0,
adapter->num_io_queues);
- if (rc)
+ if (rc) {
+ ena_destroy_xdp_tx_queues(adapter);
goto err_create_tx_queues;
+ }
rc = ena_setup_all_rx_resources(adapter);
if (rc)
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
index a8e07e51418f..fbb34d16ff02 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
@@ -4750,6 +4750,7 @@ int bnx2x_alloc_mem_bp(struct bnx2x *bp)
fp = kcalloc(bp->fp_array_size, sizeof(*fp), GFP_KERNEL);
if (!fp)
goto alloc_err;
+ bp->fp = fp;
for (i = 0; i < bp->fp_array_size; i++) {
fp[i].tpa_info =
kcalloc(ETH_MAX_AGGREGATION_QUEUES_E1H_E2,
@@ -4758,8 +4759,6 @@ int bnx2x_alloc_mem_bp(struct bnx2x *bp)
goto alloc_err;
}
- bp->fp = fp;
-
/* allocate sp objs */
bp->sp_objs = kcalloc(bp->fp_array_size, sizeof(struct bnx2x_sp_objs),
GFP_KERNEL);
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
index 678829646cec..7346acffe14a 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
@@ -10219,8 +10219,7 @@ static int bnx2x_udp_tunnel_sync(struct net_device *netdev, unsigned int table)
static const struct udp_tunnel_nic_info bnx2x_udp_tunnels = {
.sync_table = bnx2x_udp_tunnel_sync,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP |
- UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
+ .flags = UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_GENEVE, },
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index cff92020b381..4970e20b2cb1 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -1733,7 +1733,8 @@ static inline struct sk_buff *bnxt_gro_skb(struct bnxt *bp,
struct bnxt_tpa_info *tpa_info,
struct rx_tpa_end_cmp *tpa_end,
struct rx_tpa_end_cmp_ext *tpa_end1,
- struct sk_buff *skb)
+ struct sk_buff *skb,
+ struct bnxt_rx_sw_stats *rx_stats)
{
#ifdef CONFIG_INET
int payload_off;
@@ -1743,6 +1744,9 @@ static inline struct sk_buff *bnxt_gro_skb(struct bnxt *bp,
if (segs == 1)
return skb;
+ rx_stats->rx_hw_gro_packets++;
+ rx_stats->rx_hw_gro_wire_packets += segs;
+
NAPI_GRO_CB(skb)->count = segs;
skb_shinfo(skb)->gso_size =
le32_to_cpu(tpa_end1->rx_tpa_end_cmp_seg_len);
@@ -1915,7 +1919,8 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
}
if (gro)
- skb = bnxt_gro_skb(bp, tpa_info, tpa_end, tpa_end1, skb);
+ skb = bnxt_gro_skb(bp, tpa_info, tpa_end, tpa_end1, skb,
+ &cpr->sw_stats->rx);
return skb;
}
@@ -9958,7 +9963,7 @@ static void bnxt_accumulate_stats(struct bnxt_stats_mem *stats)
stats->hw_masks, stats->len / 8, false);
}
-static void bnxt_accumulate_all_stats(struct bnxt *bp)
+static void bnxt_accumulate_ring_stats(struct bnxt *bp)
{
struct bnxt_stats_mem *ring0_stats;
bool ignore_zero = false;
@@ -9981,6 +9986,10 @@ static void bnxt_accumulate_all_stats(struct bnxt *bp)
ring0_stats->hw_masks,
ring0_stats->len / 8, ignore_zero);
}
+}
+
+static void bnxt_accumulate_port_stats(struct bnxt *bp)
+{
if (bp->flags & BNXT_FLAG_PORT_STATS) {
struct bnxt_stats_mem *stats = &bp->port_stats;
__le64 *hw_stats = stats->hw_stats;
@@ -10003,6 +10012,41 @@ static void bnxt_accumulate_all_stats(struct bnxt *bp)
}
}
+static void bnxt_accumulate_all_stats(struct bnxt *bp)
+{
+ bnxt_accumulate_ring_stats(bp);
+ bnxt_accumulate_port_stats(bp);
+}
+
+/* Re-accumulate ring stats from DMA buffers if stale.
+ * uAPIs for reading sw_stats should call this first.
+ *
+ * We promise user space update frequency of bp->stats_coal_ticks but
+ * the update is a two step process - first device updates the DMA buffer,
+ * then we have to update from that buffer to driver stats in the service work.
+ * Worst case we would be 2x off from the desired frequency.
+ * Sync the stats sooner, if stale. The 20% threshold was chosen arbitrarily.
+ *
+ * Ideally we would split the user-configured time into two portions,
+ * i.e. also lower the DMA period by the 20%. But the DMA timer seems to have
+ * too coarse granularity to play such tricks.
+ */
+void bnxt_sync_ring_stats(struct bnxt *bp)
+{
+ unsigned long stale;
+
+ if (!netif_running(bp->dev) || !bp->stats_coal_ticks)
+ return;
+
+ spin_lock(&bp->stats_lock);
+ stale = usecs_to_jiffies(bp->stats_coal_ticks / 5);
+ if (time_after_eq(jiffies, bp->stats_updated_jiffies + stale)) {
+ bnxt_accumulate_ring_stats(bp);
+ bp->stats_updated_jiffies = jiffies;
+ }
+ spin_unlock(&bp->stats_lock);
+}
+
static int bnxt_hwrm_port_qstats(struct bnxt *bp, u8 flags)
{
struct hwrm_port_qstats_input *req;
@@ -12552,7 +12596,7 @@ static void __bnxt_close_nic(struct bnxt *bp, bool irq_re_init,
/* Save ring stats before shutdown */
if (bp->bnapi && irq_re_init) {
bnxt_get_ring_stats(bp, &bp->net_stats_prev);
- bnxt_get_ring_err_stats(bp, &bp->ring_err_stats_prev);
+ bnxt_get_ring_drv_stats(bp, &bp->ring_drv_stats_prev);
}
if (irq_re_init) {
bnxt_free_irq(bp);
@@ -12774,6 +12818,7 @@ bnxt_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
return;
}
+ bnxt_sync_ring_stats(bp);
bnxt_get_ring_stats(bp, stats);
bnxt_add_prev_stats(bp, stats);
@@ -12802,8 +12847,8 @@ bnxt_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
clear_bit(BNXT_STATE_READ_STATS, &bp->state);
}
-static void bnxt_get_one_ring_err_stats(struct bnxt *bp,
- struct bnxt_total_ring_err_stats *stats,
+static void bnxt_get_one_ring_drv_stats(struct bnxt *bp,
+ struct bnxt_total_ring_drv_stats *stats,
struct bnxt_cp_ring_info *cpr)
{
struct bnxt_sw_stats *sw_stats = cpr->sw_stats;
@@ -12816,19 +12861,21 @@ static void bnxt_get_one_ring_err_stats(struct bnxt *bp,
stats->rx_total_netpoll_discards += sw_stats->rx.rx_netpoll_discards;
stats->rx_total_ring_discards +=
BNXT_GET_RING_STATS64(hw_stats, rx_discard_pkts);
+ stats->rx_total_hw_gro_packets += sw_stats->rx.rx_hw_gro_packets;
+ stats->rx_total_hw_gro_wire_packets += sw_stats->rx.rx_hw_gro_wire_packets;
stats->tx_total_resets += sw_stats->tx.tx_resets;
stats->tx_total_ring_discards +=
BNXT_GET_RING_STATS64(hw_stats, tx_discard_pkts);
stats->total_missed_irqs += sw_stats->cmn.missed_irqs;
}
-void bnxt_get_ring_err_stats(struct bnxt *bp,
- struct bnxt_total_ring_err_stats *stats)
+void bnxt_get_ring_drv_stats(struct bnxt *bp,
+ struct bnxt_total_ring_drv_stats *stats)
{
int i;
for (i = 0; i < bp->cp_nr_rings; i++)
- bnxt_get_one_ring_err_stats(bp, stats, &bp->bnapi[i]->cp_ring);
+ bnxt_get_one_ring_drv_stats(bp, stats, &bp->bnapi[i]->cp_ring);
}
static bool bnxt_mc_list_updated(struct bnxt *bp, u32 *rx_mask)
@@ -13941,7 +13988,10 @@ static void bnxt_sp_task(struct work_struct *work)
if (test_and_clear_bit(BNXT_PERIODIC_STATS_SP_EVENT, &bp->sp_event)) {
bnxt_hwrm_port_qstats(bp, 0);
bnxt_hwrm_port_qstats_ext(bp, 0);
+ spin_lock(&bp->stats_lock);
bnxt_accumulate_all_stats(bp);
+ bp->stats_updated_jiffies = jiffies;
+ spin_unlock(&bp->stats_lock);
}
if (test_and_clear_bit(BNXT_LINK_CHNG_SP_EVENT, &bp->sp_event)) {
@@ -14654,6 +14704,7 @@ static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev)
INIT_DELAYED_WORK(&bp->fw_reset_task, bnxt_fw_reset_task);
spin_lock_init(&bp->ntp_fltr_lock);
+ spin_lock_init(&bp->stats_lock);
#if BITS_PER_LONG == 32
spin_lock_init(&bp->db_lock);
#endif
@@ -15092,8 +15143,7 @@ static int bnxt_udp_tunnel_unset_port(struct net_device *netdev, unsigned int ta
static const struct udp_tunnel_nic_info bnxt_udp_tunnels = {
.set_port = bnxt_udp_tunnel_set_port,
.unset_port = bnxt_udp_tunnel_unset_port,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP |
- UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
+ .flags = UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_GENEVE, },
@@ -15101,8 +15151,7 @@ static const struct udp_tunnel_nic_info bnxt_udp_tunnels = {
}, bnxt_udp_tunnels_p7 = {
.set_port = bnxt_udp_tunnel_set_port,
.unset_port = bnxt_udp_tunnel_unset_port,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP |
- UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
+ .flags = UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_GENEVE, },
@@ -15210,6 +15259,7 @@ static void bnxt_get_queue_stats_rx(struct net_device *dev, int i,
if (!bp->bnapi)
return;
+ bnxt_sync_ring_stats(bp);
cpr = &bp->bnapi[i]->cp_ring;
sw = cpr->stats.sw_stats;
@@ -15224,6 +15274,8 @@ static void bnxt_get_queue_stats_rx(struct net_device *dev, int i,
stats->bytes += BNXT_GET_RING_STATS64(sw, rx_bcast_bytes);
stats->alloc_fail = cpr->sw_stats->rx.rx_oom_discards;
+ stats->hw_gro_packets = cpr->sw_stats->rx.rx_hw_gro_packets;
+ stats->hw_gro_wire_packets = cpr->sw_stats->rx.rx_hw_gro_wire_packets;
}
static void bnxt_get_queue_stats_tx(struct net_device *dev, int i,
@@ -15236,6 +15288,7 @@ static void bnxt_get_queue_stats_tx(struct net_device *dev, int i,
if (!bp->tx_ring)
return;
+ bnxt_sync_ring_stats(bp);
bnapi = bp->tx_ring[bp->tx_ring_map[i]].bnapi;
sw = bnapi->cp_ring.stats.sw_stats;
@@ -15258,7 +15311,9 @@ static void bnxt_get_base_stats(struct net_device *dev,
rx->packets = bp->net_stats_prev.rx_packets;
rx->bytes = bp->net_stats_prev.rx_bytes;
- rx->alloc_fail = bp->ring_err_stats_prev.rx_total_oom_discards;
+ rx->alloc_fail = bp->ring_drv_stats_prev.rx_total_oom_discards;
+ rx->hw_gro_packets = bp->ring_drv_stats_prev.rx_total_hw_gro_packets;
+ rx->hw_gro_wire_packets = bp->ring_drv_stats_prev.rx_total_hw_gro_wire_packets;
tx->packets = bp->net_stats_prev.tx_packets;
tx->bytes = bp->net_stats_prev.tx_bytes;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 38690fdc3c46..37fb74f18eee 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -1113,8 +1113,11 @@ struct bnxt_rx_sw_stats {
u64 rx_l4_csum_errors;
u64 rx_resets;
u64 rx_buf_errors;
+ /* end of ethtool -S stats */
u64 rx_oom_discards;
u64 rx_netpoll_discards;
+ u64 rx_hw_gro_packets;
+ u64 rx_hw_gro_wire_packets;
};
struct bnxt_tx_sw_stats {
@@ -1131,7 +1134,7 @@ struct bnxt_sw_stats {
struct bnxt_cmn_sw_stats cmn;
};
-struct bnxt_total_ring_err_stats {
+struct bnxt_total_ring_drv_stats {
u64 rx_total_l4_csum_errors;
u64 rx_total_resets;
u64 rx_total_buf_errors;
@@ -1141,6 +1144,9 @@ struct bnxt_total_ring_err_stats {
u64 tx_total_resets;
u64 tx_total_ring_discards;
u64 total_missed_irqs;
+ /* end of ethtool -S stats */
+ u64 rx_total_hw_gro_packets;
+ u64 rx_total_hw_gro_wire_packets;
};
struct bnxt_stats_mem {
@@ -2473,7 +2479,7 @@ struct bnxt {
u8 pri2cos_idx[8];
u8 pri2cos_valid;
- struct bnxt_total_ring_err_stats ring_err_stats_prev;
+ struct bnxt_total_ring_drv_stats ring_drv_stats_prev;
u16 hwrm_max_req_len;
u16 hwrm_max_ext_req_len;
@@ -2512,6 +2518,10 @@ struct bnxt {
#define BNXT_MIN_STATS_COAL_TICKS 250000
#define BNXT_MAX_STATS_COAL_TICKS 1000000
+ /* Protects stats_updated_jiffies and writes to sw_stats */
+ spinlock_t stats_lock;
+ unsigned long stats_updated_jiffies;
+
struct work_struct sp_task;
unsigned long sp_event;
#define BNXT_RX_MASK_SP_EVENT 0
@@ -2858,8 +2868,9 @@ int bnxt_half_open_nic(struct bnxt *bp);
void bnxt_half_close_nic(struct bnxt *bp);
void bnxt_reenable_sriov(struct bnxt *bp);
void bnxt_close_nic(struct bnxt *, bool, bool);
-void bnxt_get_ring_err_stats(struct bnxt *bp,
- struct bnxt_total_ring_err_stats *stats);
+void bnxt_get_ring_drv_stats(struct bnxt *bp,
+ struct bnxt_total_ring_drv_stats *stats);
+void bnxt_sync_ring_stats(struct bnxt *bp);
bool bnxt_rfs_capable(struct bnxt *bp, bool new_rss_ctx);
int bnxt_dbg_hwrm_rd_reg(struct bnxt *bp, u32 reg_off, u16 num_words,
u32 *reg_buf);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 0be9c64ae2fa..d883e2e57da0 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -339,7 +339,7 @@ enum {
RX_NETPOLL_DISCARDS,
};
-static const char *const bnxt_ring_err_stats_arr[] = {
+static const char *const bnxt_ring_drv_stats_arr[] = {
"rx_total_l4_csum_errors",
"rx_total_resets",
"rx_total_buf_errors",
@@ -499,7 +499,7 @@ static const struct {
BNXT_TX_STATS_PRI_ENTRIES(tx_packets),
};
-#define BNXT_NUM_RING_ERR_STATS ARRAY_SIZE(bnxt_ring_err_stats_arr)
+#define BNXT_NUM_RING_DRV_STATS ARRAY_SIZE(bnxt_ring_drv_stats_arr)
#define BNXT_NUM_PORT_STATS ARRAY_SIZE(bnxt_port_stats_arr)
#define BNXT_NUM_STATS_PRI \
(ARRAY_SIZE(bnxt_rx_bytes_pri_arr) + \
@@ -538,7 +538,7 @@ static int bnxt_get_num_stats(struct bnxt *bp)
int num_stats = bnxt_get_num_ring_stats(bp);
int len;
- num_stats += BNXT_NUM_RING_ERR_STATS;
+ num_stats += BNXT_NUM_RING_DRV_STATS;
if (bp->flags & BNXT_FLAG_PORT_STATS)
num_stats += BNXT_NUM_PORT_STATS;
@@ -593,7 +593,7 @@ static bool is_tx_ring(struct bnxt *bp, int ring_num)
static void bnxt_get_ethtool_stats(struct net_device *dev,
struct ethtool_stats *stats, u64 *buf)
{
- struct bnxt_total_ring_err_stats ring_err_stats = {0};
+ struct bnxt_total_ring_drv_stats ring_drv_stats = {0};
struct bnxt *bp = netdev_priv(dev);
u64 *curr, *prev;
u32 tpa_stats;
@@ -604,6 +604,7 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,
goto skip_ring_stats;
}
+ bnxt_sync_ring_stats(bp);
tpa_stats = bnxt_get_num_tpa_ring_stats(bp);
for (i = 0; i < bp->cp_nr_rings; i++) {
struct bnxt_napi *bnapi = bp->bnapi[i];
@@ -642,12 +643,12 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,
buf[j] = sw[k];
}
- bnxt_get_ring_err_stats(bp, &ring_err_stats);
+ bnxt_get_ring_drv_stats(bp, &ring_drv_stats);
skip_ring_stats:
- curr = &ring_err_stats.rx_total_l4_csum_errors;
- prev = &bp->ring_err_stats_prev.rx_total_l4_csum_errors;
- for (i = 0; i < BNXT_NUM_RING_ERR_STATS; i++, j++, curr++, prev++)
+ curr = &ring_drv_stats.rx_total_l4_csum_errors;
+ prev = &bp->ring_drv_stats_prev.rx_total_l4_csum_errors;
+ for (i = 0; i < BNXT_NUM_RING_DRV_STATS; i++, j++, curr++, prev++)
buf[j] = *curr + *prev;
if (bp->flags & BNXT_FLAG_PORT_STATS) {
@@ -705,112 +706,105 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,
static void bnxt_get_strings(struct net_device *dev, u32 stringset, u8 *buf)
{
struct bnxt *bp = netdev_priv(dev);
- static const char * const *str;
u32 i, j, num_str;
+ const char *str;
switch (stringset) {
case ETH_SS_STATS:
for (i = 0; i < bp->cp_nr_rings; i++) {
- if (is_rx_ring(bp, i)) {
- num_str = NUM_RING_RX_HW_STATS;
- for (j = 0; j < num_str; j++) {
- sprintf(buf, "[%d]: %s", i,
- bnxt_ring_rx_stats_str[j]);
- buf += ETH_GSTRING_LEN;
+ if (is_rx_ring(bp, i))
+ for (j = 0; j < NUM_RING_RX_HW_STATS; j++) {
+ str = bnxt_ring_rx_stats_str[j];
+ ethtool_sprintf(&buf, "[%d]: %s", i,
+ str);
}
- }
- if (is_tx_ring(bp, i)) {
- num_str = NUM_RING_TX_HW_STATS;
- for (j = 0; j < num_str; j++) {
- sprintf(buf, "[%d]: %s", i,
- bnxt_ring_tx_stats_str[j]);
- buf += ETH_GSTRING_LEN;
+ if (is_tx_ring(bp, i))
+ for (j = 0; j < NUM_RING_TX_HW_STATS; j++) {
+ str = bnxt_ring_tx_stats_str[j];
+ ethtool_sprintf(&buf, "[%d]: %s", i,
+ str);
}
- }
num_str = bnxt_get_num_tpa_ring_stats(bp);
if (!num_str || !is_rx_ring(bp, i))
goto skip_tpa_stats;
if (bp->max_tpa_v2)
- str = bnxt_ring_tpa2_stats_str;
+ for (j = 0; j < num_str; j++) {
+ str = bnxt_ring_tpa2_stats_str[j];
+ ethtool_sprintf(&buf, "[%d]: %s", i,
+ str);
+ }
else
- str = bnxt_ring_tpa_stats_str;
-
- for (j = 0; j < num_str; j++) {
- sprintf(buf, "[%d]: %s", i, str[j]);
- buf += ETH_GSTRING_LEN;
- }
-skip_tpa_stats:
- if (is_rx_ring(bp, i)) {
- num_str = NUM_RING_RX_SW_STATS;
for (j = 0; j < num_str; j++) {
- sprintf(buf, "[%d]: %s", i,
- bnxt_rx_sw_stats_str[j]);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_ring_tpa_stats_str[j];
+ ethtool_sprintf(&buf, "[%d]: %s", i,
+ str);
}
+skip_tpa_stats:
+ if (is_rx_ring(bp, i))
+ for (j = 0; j < NUM_RING_RX_SW_STATS; j++) {
+ str = bnxt_rx_sw_stats_str[j];
+ ethtool_sprintf(&buf, "[%d]: %s", i,
+ str);
+ }
+ for (j = 0; j < NUM_RING_CMN_SW_STATS; j++) {
+ str = bnxt_cmn_sw_stats_str[j];
+ ethtool_sprintf(&buf, "[%d]: %s", i, str);
}
- num_str = NUM_RING_CMN_SW_STATS;
- for (j = 0; j < num_str; j++) {
- sprintf(buf, "[%d]: %s", i,
- bnxt_cmn_sw_stats_str[j]);
- buf += ETH_GSTRING_LEN;
- }
- }
- for (i = 0; i < BNXT_NUM_RING_ERR_STATS; i++) {
- strscpy(buf, bnxt_ring_err_stats_arr[i], ETH_GSTRING_LEN);
- buf += ETH_GSTRING_LEN;
}
+ for (i = 0; i < BNXT_NUM_RING_DRV_STATS; i++)
+ ethtool_puts(&buf, bnxt_ring_drv_stats_arr[i]);
- if (bp->flags & BNXT_FLAG_PORT_STATS) {
+ if (bp->flags & BNXT_FLAG_PORT_STATS)
for (i = 0; i < BNXT_NUM_PORT_STATS; i++) {
- strcpy(buf, bnxt_port_stats_arr[i].string);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_port_stats_arr[i].string;
+ ethtool_puts(&buf, str);
}
- }
+
if (bp->flags & BNXT_FLAG_PORT_STATS_EXT) {
u32 len;
len = min_t(u32, bp->fw_rx_stats_ext_size,
ARRAY_SIZE(bnxt_port_stats_ext_arr));
for (i = 0; i < len; i++) {
- strcpy(buf, bnxt_port_stats_ext_arr[i].string);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_port_stats_ext_arr[i].string;
+ ethtool_puts(&buf, str);
}
+
len = min_t(u32, bp->fw_tx_stats_ext_size,
ARRAY_SIZE(bnxt_tx_port_stats_ext_arr));
for (i = 0; i < len; i++) {
- strcpy(buf,
- bnxt_tx_port_stats_ext_arr[i].string);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_tx_port_stats_ext_arr[i].string;
+ ethtool_puts(&buf, str);
}
+
if (bp->pri2cos_valid) {
for (i = 0; i < 8; i++) {
- strcpy(buf,
- bnxt_rx_bytes_pri_arr[i].string);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_rx_bytes_pri_arr[i].string;
+ ethtool_puts(&buf, str);
}
+
for (i = 0; i < 8; i++) {
- strcpy(buf,
- bnxt_rx_pkts_pri_arr[i].string);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_rx_pkts_pri_arr[i].string;
+ ethtool_puts(&buf, str);
}
+
for (i = 0; i < 8; i++) {
- strcpy(buf,
- bnxt_tx_bytes_pri_arr[i].string);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_tx_bytes_pri_arr[i].string;
+ ethtool_puts(&buf, str);
}
+
for (i = 0; i < 8; i++) {
- strcpy(buf,
- bnxt_tx_pkts_pri_arr[i].string);
- buf += ETH_GSTRING_LEN;
+ str = bnxt_tx_pkts_pri_arr[i].string;
+ ethtool_puts(&buf, str);
}
}
}
break;
case ETH_SS_TEST:
if (bp->num_tests)
- memcpy(buf, bp->test_info->string,
- bp->num_tests * ETH_GSTRING_LEN);
+ for (i = 0; i < bp->num_tests; i++)
+ ethtool_puts(&buf, bp->test_info->string[i]);
break;
default:
netdev_err(bp->dev, "bnxt_get_strings invalid request %x\n",
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 1db5181179b8..eb6ef5ad8c2e 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -41,9 +41,8 @@
#include "bcmgenet.h"
-/* Default highest priority queue for multi queue support */
-#define GENET_Q1_PRIORITY 0
-#define GENET_Q0_PRIORITY 1
+#define GENET_Q0_WEIGHT 1
+#define GENET_Q1_WEIGHT 4
#define GENET_Q0_RX_BD_CNT \
(TOTAL_DESC - priv->hw_params->rx_queues * priv->hw_params->rx_bds_per_q)
@@ -2077,13 +2076,6 @@ static netdev_tx_t bcmgenet_xmit(struct sk_buff *skb, struct net_device *dev)
int i;
index = skb_get_queue_mapping(skb);
- /* Mapping strategy:
- * queue_mapping = 0, unclassified, packet xmited through ring 0
- * queue_mapping = 1, goes to ring 1. (highest priority queue)
- * queue_mapping = 2, goes to ring 2.
- * queue_mapping = 3, goes to ring 3.
- * queue_mapping = 4, goes to ring 4.
- */
ring = &priv->tx_rings[index];
txq = netdev_get_tx_queue(dev, index);
@@ -2783,8 +2775,9 @@ static void bcmgenet_fini_tx_napi(struct bcmgenet_priv *priv)
/* Initialize Tx queues
*
- * Queues 1-4 are priority-based, each one has 32 descriptors,
- * with queue 1 being the highest priority queue.
+ * Queues 1-4 are the priority queues, each one has 32 descriptors.
+ * The weighted round-robin arbiter gives them a larger share of TX
+ * bandwidth than the default queue 0.
*
* Queue 0 is the default Tx queue with
* GENET_Q0_TX_BD_CNT = 256 - 4 * 32 = 128 descriptors.
@@ -2812,8 +2805,8 @@ static void bcmgenet_init_tx_queues(struct net_device *dev)
dma_ctrl = 0;
ring_cfg = 0;
- /* Enable strict priority arbiter mode */
- bcmgenet_tdma_writel(priv, DMA_ARBITER_SP, DMA_ARB_CTRL);
+ /* Enable Weighted Round-Robin arbiter mode */
+ bcmgenet_tdma_writel(priv, DMA_ARBITER_WRR, DMA_ARB_CTRL);
/* Initialize Tx priority queues */
for (i = 0; i <= priv->hw_params->tx_queues; i++) {
@@ -2823,7 +2816,7 @@ static void bcmgenet_init_tx_queues(struct net_device *dev)
ring_cfg |= (1 << i);
dma_ctrl |= (1 << (i + DMA_RING_BUF_EN_SHIFT));
dma_priority[DMA_PRIO_REG_INDEX(i)] |=
- (i ? GENET_Q1_PRIORITY : GENET_Q0_PRIORITY)
+ (i ? GENET_Q1_WEIGHT : GENET_Q0_WEIGHT)
<< DMA_PRIO_REG_SHIFT(i);
}
diff --git a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c
index b3c81a2e9d46..29cce4cbb5cf 100644
--- a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c
+++ b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c
@@ -1332,18 +1332,14 @@ int setup_cn23xx_octeon_pf_device(struct octeon_device *oct)
if (octeon_map_pci_barx(oct, 1, MAX_BAR1_IOREMAP_SIZE)) {
dev_err(&oct->pci_dev->dev, "%s CN23XX BAR1 map failed\n",
__func__);
- octeon_unmap_pci_barx(oct, 0);
- return 1;
+ goto err_unmap_bar0;
}
if (cn23xx_get_pf_num(oct) != 0)
- return 1;
+ goto err_unmap_bar1;
- if (cn23xx_sriov_config(oct)) {
- octeon_unmap_pci_barx(oct, 0);
- octeon_unmap_pci_barx(oct, 1);
- return 1;
- }
+ if (cn23xx_sriov_config(oct))
+ goto err_unmap_bar1;
octeon_write_csr64(oct, CN23XX_SLI_MAC_CREDIT_CNT, 0x3F802080802080ULL);
@@ -1374,6 +1370,12 @@ int setup_cn23xx_octeon_pf_device(struct octeon_device *oct)
oct->coproc_clock_rate = 1000000ULL * cn23xx_coprocessor_clock(oct);
return 0;
+
+err_unmap_bar1:
+ octeon_unmap_pci_barx(oct, 1);
+err_unmap_bar0:
+ octeon_unmap_pci_barx(oct, 0);
+ return 1;
}
EXPORT_SYMBOL_GPL(setup_cn23xx_octeon_pf_device);
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
index 175bf9b13058..1b8247bd9e9b 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
@@ -6737,14 +6737,6 @@ void t4_sge_decode_idma_state(struct adapter *adapter, int state)
return;
}
- if (is_t4(adapter->params.chip)) {
- sge_idma_decode = (const char **)t4_decode;
- sge_idma_decode_nstates = ARRAY_SIZE(t4_decode);
- } else {
- sge_idma_decode = (const char **)t5_decode;
- sge_idma_decode_nstates = ARRAY_SIZE(t5_decode);
- }
-
if (state < sge_idma_decode_nstates)
CH_WARN(adapter, "idma state %s\n", sge_idma_decode[state]);
else
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index 71565b27893e..995c159003d7 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -4034,8 +4034,7 @@ static int be_vxlan_unset_port(struct net_device *netdev, unsigned int table,
static const struct udp_tunnel_nic_info be_udp_tunnels = {
.set_port = be_vxlan_set_port,
.unset_port = be_vxlan_unset_port,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP |
- UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
+ .flags = UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
},
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
index a7c8ec0bdfe5..4520c2788b1c 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c
@@ -2176,7 +2176,7 @@ dpaa2_switch_prechangeupper_sanity_checks(struct net_device *netdev,
if (err) {
NL_SET_ERR_MSG_MOD(extack,
"Cannot join a bridge while VLAN uppers are present");
- return 0;
+ return err;
}
netdev_for_each_lower_dev(upper_dev, other_dev, iter) {
@@ -2197,6 +2197,7 @@ dpaa2_switch_prechangeupper_sanity_checks(struct net_device *netdev,
static int dpaa2_switch_port_prechangeupper(struct net_device *netdev,
struct netdev_notifier_changeupper_info *info)
{
+ struct ethsw_port_priv *port_priv;
struct netlink_ext_ack *extack;
struct net_device *upper_dev;
int err;
@@ -2215,6 +2216,13 @@ static int dpaa2_switch_port_prechangeupper(struct net_device *netdev,
if (!info->linking)
dpaa2_switch_port_pre_bridge_leave(netdev);
+ } else if (is_vlan_dev(upper_dev)) {
+ port_priv = netdev_priv(netdev);
+ if (port_priv->fdb->bridge_dev) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Cannot accept VLAN uppers while bridged");
+ return -EOPNOTSUPP;
+ }
}
return 0;
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index c58e44144c2f..e999671af46d 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c
@@ -1425,6 +1425,7 @@ int enetc_xdp_xmit(struct net_device *ndev, int num_frames,
{
struct enetc_tx_swbd xdp_redirect_arr[ENETC_MAX_SKB_FRAGS] = {0};
struct enetc_ndev_priv *priv = netdev_priv(ndev);
+ struct skb_shared_info *shinfo;
struct enetc_bdr *tx_ring;
int xdp_tx_bd_cnt, i, k;
int xdp_tx_frm_cnt = 0;
@@ -1440,6 +1441,12 @@ int enetc_xdp_xmit(struct net_device *ndev, int num_frames,
prefetchw(ENETC_TXBD(*tx_ring, tx_ring->next_to_use));
for (k = 0; k < num_frames; k++) {
+ if (xdp_frame_has_frags(frames[k])) {
+ shinfo = xdp_get_shared_info_from_frame(frames[k]);
+ if (unlikely((shinfo->nr_frags + 1) > ENETC_MAX_SKB_FRAGS))
+ break;
+ }
+
xdp_tx_bd_cnt = enetc_xdp_frame_to_xdp_tx_swbd(tx_ring,
xdp_redirect_arr,
frames[k]);
diff --git a/drivers/net/ethernet/freescale/fman/fman.c b/drivers/net/ethernet/freescale/fman/fman.c
index fb416d60dcd7..bfc784b7f130 100644
--- a/drivers/net/ethernet/freescale/fman/fman.c
+++ b/drivers/net/ethernet/freescale/fman/fman.c
@@ -1995,8 +1995,10 @@ static int fman_init(struct fman *fman)
/* Init KeyGen */
fman->keygen = keygen_init(fman->kg_regs);
- if (!fman->keygen)
+ if (!fman->keygen) {
+ free_init_resources(fman);
return -EINVAL;
+ }
err = enable(fman, cfg);
if (err != 0)
diff --git a/drivers/net/ethernet/freescale/gianfar.c b/drivers/net/ethernet/freescale/gianfar.c
index ecb1703ea150..54efa109a18f 100644
--- a/drivers/net/ethernet/freescale/gianfar.c
+++ b/drivers/net/ethernet/freescale/gianfar.c
@@ -470,10 +470,13 @@ static void free_gfar_dev(struct gfar_private *priv)
{
int i, j;
- for (i = 0; i < priv->num_grps; i++)
+ for (i = 0; i < MAXGROUPS; i++)
for (j = 0; j < GFAR_NUM_IRQS; j++) {
- kfree(priv->gfargrp[i].irqinfo[j]);
- priv->gfargrp[i].irqinfo[j] = NULL;
+ if (priv->gfargrp[i].irqinfo[j]) {
+ irq_dispose_mapping(priv->gfargrp[i].irqinfo[j]->irq);
+ kfree(priv->gfargrp[i].irqinfo[j]);
+ priv->gfargrp[i].irqinfo[j] = NULL;
+ }
}
free_netdev(priv->ndev);
@@ -630,7 +633,7 @@ static phy_interface_t gfar_get_interface(struct net_device *dev)
static int gfar_of_init(struct platform_device *ofdev, struct net_device **pdev)
{
const char *model;
- int err = 0, i;
+ int err = 0, i, j;
phy_interface_t interface;
struct net_device *dev = NULL;
struct gfar_private *priv = NULL;
@@ -714,8 +717,11 @@ static int gfar_of_init(struct platform_device *ofdev, struct net_device **pdev)
priv->rx_list.count = 0;
mutex_init(&priv->rx_queue_access);
- for (i = 0; i < MAXGROUPS; i++)
+ for (i = 0; i < MAXGROUPS; i++) {
priv->gfargrp[i].regs = NULL;
+ for (j = 0; j < GFAR_NUM_IRQS; j++)
+ priv->gfargrp[i].irqinfo[j] = NULL;
+ }
/* Parse and initialize group specific information */
if (priv->mode == MQ_MG_MODE) {
diff --git a/drivers/net/ethernet/hisilicon/hns3/hnae3.h b/drivers/net/ethernet/hisilicon/hns3/hnae3.h
index 388c70331a55..4e44f28288f9 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hnae3.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hnae3.h
@@ -677,7 +677,7 @@ struct hnae3_ae_ops {
void (*get_mac_stats)(struct hnae3_handle *handle,
struct hns3_mac_stats *mac_stats);
void (*get_strings)(struct hnae3_handle *handle,
- u32 stringset, u8 *data);
+ u32 stringset, u8 **data);
int (*get_sset_count)(struct hnae3_handle *handle, int stringset);
void (*get_regs)(struct hnae3_handle *handle, u32 *version,
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c
index 4ad4e8ab2f1f..37396ca4ecfc 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c
@@ -348,7 +348,7 @@ static int hclge_comm_cmd_csq_clean(struct hclge_comm_hw *hw)
static int hclge_comm_cmd_csq_done(struct hclge_comm_hw *hw)
{
u32 head = hclge_comm_read_dev(hw, HCLGE_COMM_NIC_CSQ_HEAD_REG);
- return head == hw->cmq.csq.next_to_use;
+ return head == (u32)hw->cmq.csq.next_to_use;
}
static u32 hclge_get_cmdq_tx_timeout(u16 opcode, u32 tx_timeout)
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c
index 2b31188ff555..f9a3d6fc4416 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c
@@ -36,27 +36,22 @@ int hclge_comm_tqps_get_sset_count(struct hnae3_handle *handle)
}
EXPORT_SYMBOL_GPL(hclge_comm_tqps_get_sset_count);
-u8 *hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 *data)
+void hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 **data)
{
struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- u8 *buff = data;
u16 i;
for (i = 0; i < kinfo->num_tqps; i++) {
struct hclge_comm_tqp *tqp =
container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
- snprintf(buff, ETH_GSTRING_LEN, "txq%u_pktnum_rcd", tqp->index);
- buff += ETH_GSTRING_LEN;
+ ethtool_sprintf(data, "txq%u_pktnum_rcd", tqp->index);
}
for (i = 0; i < kinfo->num_tqps; i++) {
struct hclge_comm_tqp *tqp =
container_of(kinfo->tqp[i], struct hclge_comm_tqp, q);
- snprintf(buff, ETH_GSTRING_LEN, "rxq%u_pktnum_rcd", tqp->index);
- buff += ETH_GSTRING_LEN;
+ ethtool_sprintf(data, "rxq%u_pktnum_rcd", tqp->index);
}
-
- return buff;
}
EXPORT_SYMBOL_GPL(hclge_comm_tqps_get_strings);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.h b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.h
index a46350162ee8..b9ff424c0bc2 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.h
@@ -32,7 +32,7 @@ struct hclge_comm_tqp {
u64 *hclge_comm_tqps_get_stats(struct hnae3_handle *handle, u64 *data);
int hclge_comm_tqps_get_sset_count(struct hnae3_handle *handle);
-u8 *hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 *data);
+void hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 **data);
void hclge_comm_reset_tqp_stats(struct hnae3_handle *handle);
int hclge_comm_tqps_update_stats(struct hnae3_handle *handle,
struct hclge_comm_hw *hw);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
index 3d70c97a0bed..35e57eebcf57 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
@@ -3,6 +3,7 @@
#include <linux/debugfs.h>
#include <linux/device.h>
+#include <linux/string_choices.h>
#include "hnae3.h"
#include "hns3_debugfs.h"
@@ -661,12 +662,14 @@ static void hns3_dump_rx_queue_info(struct hns3_enet_ring *ring,
HNS3_RING_RX_RING_PKTNUM_RECORD_REG));
sprintf(result[j++], "%u", ring->rx_copybreak);
- sprintf(result[j++], "%s", readl_relaxed(ring->tqp->io_base +
- HNS3_RING_EN_REG) ? "on" : "off");
+ sprintf(result[j++], "%s",
+ str_on_off(readl_relaxed(ring->tqp->io_base +
+ HNS3_RING_EN_REG)));
if (hnae3_ae_dev_tqp_txrx_indep_supported(ae_dev))
- sprintf(result[j++], "%s", readl_relaxed(ring->tqp->io_base +
- HNS3_RING_RX_EN_REG) ? "on" : "off");
+ sprintf(result[j++], "%s",
+ str_on_off(readl_relaxed(ring->tqp->io_base +
+ HNS3_RING_RX_EN_REG)));
else
sprintf(result[j++], "%s", "NA");
@@ -681,7 +684,7 @@ static int hns3_dbg_rx_queue_info(struct hnae3_handle *h,
char *buf, int len)
{
char data_str[ARRAY_SIZE(rx_queue_info_items)][HNS3_DBG_DATA_STR_LEN];
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
char *result[ARRAY_SIZE(rx_queue_info_items)];
struct hns3_nic_priv *priv = h->priv;
char content[HNS3_DBG_INFO_LEN];
@@ -764,12 +767,14 @@ static void hns3_dump_tx_queue_info(struct hns3_enet_ring *ring,
sprintf(result[j++], "%u", readl_relaxed(ring->tqp->io_base +
HNS3_RING_TX_RING_PKTNUM_RECORD_REG));
- sprintf(result[j++], "%s", readl_relaxed(ring->tqp->io_base +
- HNS3_RING_EN_REG) ? "on" : "off");
+ sprintf(result[j++], "%s",
+ str_on_off(readl_relaxed(ring->tqp->io_base +
+ HNS3_RING_EN_REG)));
if (hnae3_ae_dev_tqp_txrx_indep_supported(ae_dev))
- sprintf(result[j++], "%s", readl_relaxed(ring->tqp->io_base +
- HNS3_RING_TX_EN_REG) ? "on" : "off");
+ sprintf(result[j++], "%s",
+ str_on_off(readl_relaxed(ring->tqp->io_base +
+ HNS3_RING_TX_EN_REG)));
else
sprintf(result[j++], "%s", "NA");
@@ -784,7 +789,7 @@ static int hns3_dbg_tx_queue_info(struct hnae3_handle *h,
char *buf, int len)
{
char data_str[ARRAY_SIZE(tx_queue_info_items)][HNS3_DBG_DATA_STR_LEN];
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
char *result[ARRAY_SIZE(tx_queue_info_items)];
struct hns3_nic_priv *priv = h->priv;
char content[HNS3_DBG_INFO_LEN];
@@ -1029,8 +1034,7 @@ static int hns3_dbg_tx_bd_info(struct hns3_dbg_data *d, char *buf, int len)
static void
hns3_dbg_dev_caps(struct hnae3_handle *h, char *buf, int len, int *pos)
{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
- const char * const str[] = {"no", "yes"};
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
unsigned long *caps = ae_dev->caps;
u32 i, state;
@@ -1039,7 +1043,7 @@ hns3_dbg_dev_caps(struct hnae3_handle *h, char *buf, int len, int *pos)
for (i = 0; i < ARRAY_SIZE(hns3_dbg_cap); i++) {
state = test_bit(hns3_dbg_cap[i].cap_bit, caps);
*pos += scnprintf(buf + *pos, len - *pos, "%s: %s\n",
- hns3_dbg_cap[i].name, str[state]);
+ hns3_dbg_cap[i].name, str_yes_no(state));
}
*pos += scnprintf(buf + *pos, len - *pos, "\n");
@@ -1235,7 +1239,7 @@ static const struct hns3_dbg_func hns3_dbg_cmd_func[] = {
static int hns3_dbg_read_cmd(struct hns3_dbg_data *dbg_data,
enum hnae3_dbg_cmd cmd, char *buf, int len)
{
- const struct hnae3_ae_ops *ops = dbg_data->handle->ae_algo->ops;
+ const struct hnae3_ae_ops *ops = hns3_get_ops(dbg_data->handle);
const struct hns3_dbg_func *cmd_func;
u32 i;
@@ -1360,7 +1364,7 @@ hns3_dbg_common_file_init(struct hnae3_handle *handle, u32 cmd)
int hns3_dbg_init(struct hnae3_handle *handle)
{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
const char *name = pci_name(handle->pdev);
int ret;
u32 i;
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
index 803da392c8ef..9fc6d588b02f 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
@@ -548,9 +548,9 @@ void hns3_set_vector_coalesce_rx_ql(struct hns3_enet_tqp_vector *tqp_vector,
static void hns3_vector_coalesce_init(struct hns3_enet_tqp_vector *tqp_vector,
struct hns3_nic_priv *priv)
{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(priv->ae_handle->pdev);
struct hns3_enet_coalesce *tx_coal = &tqp_vector->tx_group.coal;
struct hns3_enet_coalesce *rx_coal = &tqp_vector->rx_group.coal;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(priv->ae_handle);
struct hns3_enet_coalesce *ptx_coal = &priv->tx_coal;
struct hns3_enet_coalesce *prx_coal = &priv->rx_coal;
@@ -961,7 +961,7 @@ static void hns3_nic_set_rx_mode(struct net_device *netdev)
void hns3_request_update_promisc_mode(struct hnae3_handle *handle)
{
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
if (ops->request_update_promisc_mode)
ops->request_update_promisc_mode(handle);
@@ -1315,7 +1315,7 @@ static int hns3_get_l4_protocol(struct sk_buff *skb, u8 *ol4_proto,
static bool hns3_tunnel_csum_bug(struct sk_buff *skb)
{
struct hns3_nic_priv *priv = netdev_priv(skb->dev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(priv->ae_handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(priv->ae_handle);
union l4_hdr_info l4;
/* device version above V3(include V3), the hardware can
@@ -1515,7 +1515,7 @@ static int hns3_handle_vtags(struct hns3_enet_ring *tx_ring,
* VLAN enabled, only one VLAN header is allowed in skb, otherwise it
* will cause RAS error.
*/
- ae_dev = pci_get_drvdata(handle->pdev);
+ ae_dev = hns3_get_ae_dev(handle);
if (unlikely(skb_vlan_tagged_multi(skb) &&
ae_dev->dev_version <= HNAE3_DEVICE_VERSION_V2 &&
handle->port_base_vlan_state ==
@@ -1701,8 +1701,8 @@ static int hns3_fill_desc(struct hns3_enet_ring *ring, dma_addr_t dma,
#define HNS3_LIKELY_BD_NUM 1
struct hns3_desc *desc = &ring->desc[ring->next_to_use];
- unsigned int frag_buf_num;
- int k, sizeoflast;
+ unsigned int frag_buf_num, k;
+ int sizeoflast;
if (likely(size <= HNS3_MAX_BD_SIZE)) {
desc->addr = cpu_to_le64(dma);
@@ -1874,7 +1874,7 @@ static bool hns3_skb_need_linearized(struct sk_buff *skb, unsigned int *bd_size,
unsigned int bd_num, u8 max_non_tso_bd_num)
{
unsigned int tot_len = 0;
- int i;
+ unsigned int i;
for (i = 0; i < max_non_tso_bd_num - 1U; i++)
tot_len += bd_size[i];
@@ -1902,7 +1902,7 @@ static bool hns3_skb_need_linearized(struct sk_buff *skb, unsigned int *bd_size,
void hns3_shinfo_pack(struct skb_shared_info *shinfo, __u32 *size)
{
- int i;
+ u32 i;
for (i = 0; i < MAX_SKB_FRAGS; i++)
size[i] = skb_frag_size(&shinfo->frags[i]);
@@ -2218,9 +2218,9 @@ static int hns3_handle_tx_sgl(struct hns3_enet_ring *ring,
struct hns3_desc_cb *desc_cb = &ring->desc_cb[ring->next_to_use];
u32 nfrag = skb_shinfo(skb)->nr_frags + 1;
struct sg_table *sgt;
- int i, bd_num = 0;
+ int bd_num = 0;
dma_addr_t dma;
- u32 cb_len;
+ u32 cb_len, i;
int nents;
if (skb_has_frag_list(skb))
@@ -2558,7 +2558,7 @@ static void hns3_nic_get_stats64(struct net_device *netdev,
struct hnae3_handle *handle = priv->ae_handle;
struct rtnl_link_stats64 ring_total_stats;
struct hns3_enet_ring *ring;
- unsigned int idx;
+ int idx;
if (test_bit(HNS3_NIC_STATE_DOWN, &priv->state))
return;
@@ -2784,7 +2784,7 @@ static int hns3_nic_change_mtu(struct net_device *netdev, int new_mtu)
static int hns3_get_timeout_queue(struct net_device *ndev)
{
- int i;
+ unsigned int i;
/* Find the stopped queue the same way the stack does */
for (i = 0; i < ndev->num_tx_queues; i++) {
@@ -2865,7 +2865,7 @@ static bool hns3_get_tx_timeo_queue_info(struct net_device *ndev)
struct hns3_nic_priv *priv = netdev_priv(ndev);
struct hnae3_handle *h = hns3_get_handle(ndev);
struct hns3_enet_ring *tx_ring;
- int timeout_queue;
+ u32 timeout_queue;
timeout_queue = hns3_get_timeout_queue(ndev);
if (timeout_queue >= ndev->num_tx_queues) {
@@ -3835,7 +3835,7 @@ static int hns3_gro_complete(struct sk_buff *skb, u32 l234info)
{
__be16 type = skb->protocol;
struct tcphdr *th;
- int depth = 0;
+ u32 depth = 0;
while (eth_type_vlan(type)) {
struct vlan_hdr *vh;
@@ -4761,7 +4761,7 @@ static int hns3_nic_init_vector_data(struct hns3_nic_priv *priv)
static void hns3_nic_init_coal_cfg(struct hns3_nic_priv *priv)
{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(priv->ae_handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(priv->ae_handle);
struct hns3_enet_coalesce *tx_coal = &priv->tx_coal;
struct hns3_enet_coalesce *rx_coal = &priv->rx_coal;
@@ -5265,7 +5265,7 @@ static void hns3_info_show(struct hns3_nic_priv *priv)
static void hns3_set_cq_period_mode(struct hns3_nic_priv *priv,
enum dim_cq_period_mode mode, bool is_tx)
{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(priv->ae_handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(priv->ae_handle);
struct hnae3_handle *handle = priv->ae_handle;
int i;
@@ -5303,7 +5303,7 @@ void hns3_cq_period_mode_init(struct hns3_nic_priv *priv,
static void hns3_state_init(struct hnae3_handle *handle)
{
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
struct net_device *netdev = handle->kinfo.netdev;
struct hns3_nic_priv *priv = netdev_priv(netdev);
@@ -5975,7 +5975,7 @@ static const struct hns3_hw_error_info hns3_hw_err[] = {
static void hns3_process_hw_error(struct hnae3_handle *handle,
enum hnae3_hw_error_type type)
{
- int i;
+ u32 i;
for (i = 0; i < ARRAY_SIZE(hns3_hw_err); i++) {
if (hns3_hw_err[i].type == type) {
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
index caf7a4df8585..57beda1f9168 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
@@ -623,7 +623,7 @@ struct hns3_reset_type_map {
enum hnae3_reset_type rst_type;
};
-static inline int ring_space(struct hns3_enet_ring *ring)
+static inline u32 ring_space(struct hns3_enet_ring *ring)
{
/* This smp_load_acquire() pairs with smp_store_release() in
* hns3_nic_reclaim_one_desc called by hns3_clean_tx_ring.
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
index b1e988347347..dba96e2f9e99 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
@@ -3,6 +3,7 @@
#include <linux/etherdevice.h>
#include <linux/string.h>
+#include <linux/string_choices.h>
#include <linux/phy.h>
#include <linux/sfp.h>
@@ -85,7 +86,7 @@ static int hns3_get_sset_count(struct net_device *netdev, int stringset);
static int hns3_lp_setup(struct net_device *ndev, enum hnae3_loop loop, bool en)
{
struct hnae3_handle *h = hns3_get_handle(ndev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
int ret;
if (!h->ae_algo->ops->set_loopback ||
@@ -170,7 +171,7 @@ static void hns3_lp_setup_skb(struct sk_buff *skb)
* the purpose of mac or serdes selftest.
*/
handle = hns3_get_handle(ndev);
- ae_dev = pci_get_drvdata(handle->pdev);
+ ae_dev = hns3_get_ae_dev(handle);
if (ae_dev->dev_version < HNAE3_DEVICE_VERSION_V2)
ethh->h_dest[5] += HNS3_NIC_LB_DST_MAC_ADDR;
eth_zero_addr(ethh->h_source);
@@ -488,7 +489,7 @@ static const struct hns3_pflag_desc hns3_priv_flags[HNAE3_PFLAG_MAX] = {
static int hns3_get_sset_count(struct net_device *netdev, int stringset)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
- const struct hnae3_ae_ops *ops = h->ae_algo->ops;
+ const struct hnae3_ae_ops *ops = hns3_get_ops(h);
if (!ops->get_sset_count)
return -EOPNOTSUPP;
@@ -509,73 +510,53 @@ static int hns3_get_sset_count(struct net_device *netdev, int stringset)
}
}
-static void *hns3_update_strings(u8 *data, const struct hns3_stats *stats,
- u32 stat_count, u32 num_tqps, const char *prefix)
+static void hns3_update_strings(u8 **data, const struct hns3_stats *stats,
+ u32 stat_count, u32 num_tqps,
+ const char *prefix)
{
-#define MAX_PREFIX_SIZE (6 + 4)
- u32 size_left;
u32 i, j;
- u32 n1;
- for (i = 0; i < num_tqps; i++) {
- for (j = 0; j < stat_count; j++) {
- data[ETH_GSTRING_LEN - 1] = '\0';
-
- /* first, prepend the prefix string */
- n1 = scnprintf(data, MAX_PREFIX_SIZE, "%s%u_",
- prefix, i);
- size_left = (ETH_GSTRING_LEN - 1) - n1;
-
- /* now, concatenate the stats string to it */
- strncat(data, stats[j].stats_string, size_left);
- data += ETH_GSTRING_LEN;
- }
- }
-
- return data;
+ for (i = 0; i < num_tqps; i++)
+ for (j = 0; j < stat_count; j++)
+ ethtool_sprintf(data, "%s%u_%s", prefix, i,
+ stats[j].stats_string);
}
-static u8 *hns3_get_strings_tqps(struct hnae3_handle *handle, u8 *data)
+static void hns3_get_strings_tqps(struct hnae3_handle *handle, u8 **data)
{
struct hnae3_knic_private_info *kinfo = &handle->kinfo;
const char tx_prefix[] = "txq";
const char rx_prefix[] = "rxq";
/* get strings for Tx */
- data = hns3_update_strings(data, hns3_txq_stats, HNS3_TXQ_STATS_COUNT,
- kinfo->num_tqps, tx_prefix);
+ hns3_update_strings(data, hns3_txq_stats, HNS3_TXQ_STATS_COUNT,
+ kinfo->num_tqps, tx_prefix);
/* get strings for Rx */
- data = hns3_update_strings(data, hns3_rxq_stats, HNS3_RXQ_STATS_COUNT,
- kinfo->num_tqps, rx_prefix);
-
- return data;
+ hns3_update_strings(data, hns3_rxq_stats, HNS3_RXQ_STATS_COUNT,
+ kinfo->num_tqps, rx_prefix);
}
static void hns3_get_strings(struct net_device *netdev, u32 stringset, u8 *data)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
- const struct hnae3_ae_ops *ops = h->ae_algo->ops;
- char *buff = (char *)data;
- int i;
+ const struct hnae3_ae_ops *ops = hns3_get_ops(h);
+ u32 i;
if (!ops->get_strings)
return;
switch (stringset) {
case ETH_SS_STATS:
- buff = hns3_get_strings_tqps(h, buff);
- ops->get_strings(h, stringset, (u8 *)buff);
+ hns3_get_strings_tqps(h, &data);
+ ops->get_strings(h, stringset, &data);
break;
case ETH_SS_TEST:
- ops->get_strings(h, stringset, data);
+ ops->get_strings(h, stringset, &data);
break;
case ETH_SS_PRIV_FLAGS:
- for (i = 0; i < HNS3_PRIV_FLAGS_LEN; i++) {
- snprintf(buff, ETH_GSTRING_LEN, "%s",
- hns3_priv_flags[i].name);
- buff += ETH_GSTRING_LEN;
- }
+ for (i = 0; i < HNS3_PRIV_FLAGS_LEN; i++)
+ ethtool_puts(&data, hns3_priv_flags[i].name);
break;
default:
break;
@@ -588,7 +569,7 @@ static u64 *hns3_get_stats_tqps(struct hnae3_handle *handle, u64 *data)
struct hns3_nic_priv *nic_priv = handle->priv;
struct hns3_enet_ring *ring;
u8 *stat;
- int i, j;
+ u32 i, j;
/* get stats for Tx */
for (i = 0; i < kinfo->num_tqps; i++) {
@@ -711,7 +692,7 @@ static void hns3_get_pauseparam(struct net_device *netdev,
struct ethtool_pauseparam *param)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
if (!test_bit(HNAE3_DEV_SUPPORT_PAUSE_B, ae_dev->caps))
return;
@@ -725,7 +706,7 @@ static int hns3_set_pauseparam(struct net_device *netdev,
struct ethtool_pauseparam *param)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
if (!test_bit(HNAE3_DEV_SUPPORT_PAUSE_B, ae_dev->caps))
return -EOPNOTSUPP;
@@ -744,7 +725,7 @@ static int hns3_set_pauseparam(struct net_device *netdev,
static void hns3_get_ksettings(struct hnae3_handle *h,
struct ethtool_link_ksettings *cmd)
{
- const struct hnae3_ae_ops *ops = h->ae_algo->ops;
+ const struct hnae3_ae_ops *ops = hns3_get_ops(h);
/* 1.auto_neg & speed & duplex from cmd */
if (ops->get_ksettings_an_result)
@@ -770,7 +751,7 @@ static int hns3_get_link_ksettings(struct net_device *netdev,
struct ethtool_link_ksettings *cmd)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
const struct hnae3_ae_ops *ops;
u8 module_type;
u8 media_type;
@@ -830,12 +811,11 @@ static int hns3_get_link_ksettings(struct net_device *netdev,
}
static int hns3_check_ksettings_param(const struct net_device *netdev,
- const struct ethtool_link_ksettings *cmd)
+ const struct ethtool_link_ksettings *cmd,
+ u8 media_type)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
- u8 module_type = HNAE3_MODULE_TYPE_UNKNOWN;
- u8 media_type = HNAE3_MEDIA_TYPE_UNKNOWN;
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
u32 lane_num;
u8 autoneg;
u32 speed;
@@ -855,9 +835,6 @@ static int hns3_check_ksettings_param(const struct net_device *netdev,
return 0;
}
- if (ops->get_media_type)
- ops->get_media_type(handle, &media_type, &module_type);
-
if (cmd->base.duplex == DUPLEX_HALF &&
media_type != HNAE3_MEDIA_TYPE_COPPER) {
netdev_err(netdev,
@@ -880,8 +857,10 @@ static int hns3_set_link_ksettings(struct net_device *netdev,
const struct ethtool_link_ksettings *cmd)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
+ u8 module_type = HNAE3_MODULE_TYPE_UNKNOWN;
+ u8 media_type = HNAE3_MEDIA_TYPE_UNKNOWN;
int ret;
/* Chip don't support this mode. */
@@ -897,22 +876,23 @@ static int hns3_set_link_ksettings(struct net_device *netdev,
cmd->base.autoneg, cmd->base.speed, cmd->base.duplex,
cmd->lanes);
- /* Only support ksettings_set for netdev with phy attached for now */
- if (netdev->phydev) {
- if (cmd->base.speed == SPEED_1000 &&
- cmd->base.autoneg == AUTONEG_DISABLE)
- return -EINVAL;
+ if (!ops->get_media_type)
+ return -EOPNOTSUPP;
+ ops->get_media_type(handle, &media_type, &module_type);
- return phy_ethtool_ksettings_set(netdev->phydev, cmd);
- } else if (test_bit(HNAE3_DEV_SUPPORT_PHY_IMP_B, ae_dev->caps) &&
- ops->set_phy_link_ksettings) {
- return ops->set_phy_link_ksettings(handle, cmd);
+ if (media_type == HNAE3_MEDIA_TYPE_COPPER) {
+ if (!ops->set_phy_link_ksettings)
+ return -EOPNOTSUPP;
+ ret = ops->set_phy_link_ksettings(handle, cmd);
+ if (ret != -ENODEV)
+ return ret;
+ /* PHY_INEXISTENT, use MAC-level configuration */
}
if (ae_dev->dev_version < HNAE3_DEVICE_VERSION_V2)
return -EOPNOTSUPP;
- ret = hns3_check_ksettings_param(netdev, cmd);
+ ret = hns3_check_ksettings_param(netdev, cmd, media_type);
if (ret)
return ret;
@@ -951,7 +931,7 @@ static u32 hns3_get_rss_key_size(struct net_device *netdev)
static u32 hns3_get_rss_indir_size(struct net_device *netdev)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
return ae_dev->dev_specs.rss_ind_tbl_size;
}
@@ -973,7 +953,7 @@ static int hns3_set_rss(struct net_device *netdev,
struct netlink_ext_ack *extack)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
if (!h->ae_algo->ops->set_rss)
return -EOPNOTSUPP;
@@ -1043,8 +1023,8 @@ static int hns3_set_reset(struct net_device *netdev, u32 *flags)
{
enum hnae3_reset_type rst_type = HNAE3_NONE_RESET;
struct hnae3_handle *h = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
- const struct hnae3_ae_ops *ops = h->ae_algo->ops;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
+ const struct hnae3_ae_ops *ops = hns3_get_ops(h);
const struct hns3_reset_type_map *rst_type_map;
enum ethtool_reset_flags rst_flags;
u32 i, size;
@@ -1208,7 +1188,7 @@ static int hns3_set_tx_push(struct net_device *netdev, u32 tx_push)
{
struct hns3_nic_priv *priv = netdev_priv(netdev);
struct hnae3_handle *h = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(h->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(h);
u32 old_state = test_bit(HNS3_NIC_STATE_TX_PUSH_ENABLE, &priv->state);
if (!test_bit(HNAE3_DEV_SUPPORT_TX_PUSH_B, ae_dev->caps) && tx_push)
@@ -1218,7 +1198,7 @@ static int hns3_set_tx_push(struct net_device *netdev, u32 tx_push)
return 0;
netdev_dbg(netdev, "Changing tx push from %s to %s\n",
- old_state ? "on" : "off", tx_push ? "on" : "off");
+ str_on_off(old_state), str_on_off(tx_push));
if (tx_push)
set_bit(HNS3_NIC_STATE_TX_PUSH_ENABLE, &priv->state);
@@ -1319,7 +1299,7 @@ static int hns3_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd)
static int hns3_nway_reset(struct net_device *netdev)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
struct phy_device *phy = netdev->phydev;
int autoneg;
@@ -1396,7 +1376,7 @@ static int hns3_check_gl_coalesce_para(struct net_device *netdev,
struct ethtool_coalesce *cmd)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
u32 rx_gl, tx_gl;
if (cmd->rx_coalesce_usecs > ae_dev->dev_specs.max_int_gl) {
@@ -1468,7 +1448,7 @@ static int hns3_check_ql_coalesce_param(struct net_device *netdev,
struct ethtool_coalesce *cmd)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
if ((cmd->tx_max_coalesced_frames || cmd->rx_max_coalesced_frames) &&
!ae_dev->dev_specs.int_ql_max) {
@@ -1492,7 +1472,7 @@ hns3_check_cqe_coalesce_param(struct net_device *netdev,
struct kernel_ethtool_coalesce *kernel_coal)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
if ((kernel_coal->use_cqe_mode_tx || kernel_coal->use_cqe_mode_rx) &&
!hnae3_ae_dev_cq_supported(ae_dev)) {
@@ -1668,8 +1648,8 @@ static void hns3_get_fec_stats(struct net_device *netdev,
struct ethtool_fec_stats *fec_stats)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
if (!hnae3_ae_dev_fec_stats_supported(ae_dev) || !ops->get_fec_stats)
return;
@@ -1719,8 +1699,8 @@ static int hns3_get_fecparam(struct net_device *netdev,
struct ethtool_fecparam *fec)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
u8 fec_ability;
u8 fec_mode;
@@ -1744,8 +1724,8 @@ static int hns3_set_fecparam(struct net_device *netdev,
struct ethtool_fecparam *fec)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
u32 fec_mode;
if (!test_bit(HNAE3_DEV_SUPPORT_FEC_B, ae_dev->caps))
@@ -1766,8 +1746,8 @@ static int hns3_get_module_info(struct net_device *netdev,
#define HNS3_SFF_8636_V1_3 0x03
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
struct hns3_sfp_type sfp_type;
int ret;
@@ -1816,8 +1796,8 @@ static int hns3_get_module_eeprom(struct net_device *netdev,
struct ethtool_eeprom *ee, u8 *data)
{
struct hnae3_handle *handle = hns3_get_handle(netdev);
- struct hnae3_ae_dev *ae_dev = pci_get_drvdata(handle->pdev);
- const struct hnae3_ae_ops *ops = handle->ae_algo->ops;
+ struct hnae3_ae_dev *ae_dev = hns3_get_ae_dev(handle);
+ const struct hnae3_ae_ops *ops = hns3_get_ops(handle);
if (ae_dev->dev_version < HNAE3_DEVICE_VERSION_V2 ||
!ops->get_module_eeprom)
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
index debf143e9940..d1872b9b6469 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
@@ -3,6 +3,7 @@
#include <linux/device.h>
#include <linux/sched/clock.h>
+#include <linux/string_choices.h>
#include "hclge_debugfs.h"
#include "hclge_err.h"
@@ -11,7 +12,6 @@
#include "hclge_tm.h"
#include "hnae3.h"
-static const char * const state_str[] = { "off", "on" };
static const char * const hclge_mac_state_str[] = {
"TO_ADD", "TO_DEL", "ACTIVE"
};
@@ -830,10 +830,10 @@ hclge_dbg_dump_reg_tqp(struct hclge_dev *hdev,
{
const struct hclge_dbg_dfx_message *dfx_message = reg_info->dfx_msg;
const struct hclge_dbg_reg_common_msg *reg_msg = ®_info->reg_msg;
+ u32 index, entry, i, cnt, min_num;
struct hclge_desc *desc_src;
- u32 index, entry, i, cnt;
- int bd_num, min_num, ret;
struct hclge_desc *desc;
+ int bd_num, ret;
ret = hclge_dbg_get_dfx_bd_num(hdev, reg_msg->offset, &bd_num);
if (ret)
@@ -885,9 +885,9 @@ hclge_dbg_dump_reg_common(struct hclge_dev *hdev,
const struct hclge_dbg_reg_common_msg *reg_msg = ®_info->reg_msg;
const struct hclge_dbg_dfx_message *dfx_message = reg_info->dfx_msg;
struct hclge_desc *desc_src;
- int bd_num, min_num, ret;
+ int bd_num, min_num, ret, i;
struct hclge_desc *desc;
- u32 entry, i;
+ u32 entry;
ret = hclge_dbg_get_dfx_bd_num(hdev, reg_msg->offset, &bd_num);
if (ret)
@@ -1279,7 +1279,7 @@ static int hclge_dbg_dump_reg_cmd(struct hclge_dev *hdev,
{
const struct hclge_dbg_reg_type_info *reg_info;
int pos = 0, ret = 0;
- int i;
+ u32 i;
for (i = 0; i < ARRAY_SIZE(hclge_dbg_reg_info); i++) {
reg_info = &hclge_dbg_reg_info[i];
@@ -2573,7 +2573,7 @@ static int hclge_dbg_dump_loopback(struct hclge_dev *hdev, char *buf, int len)
loopback_en = hnae3_get_bit(le32_to_cpu(req_app->txrx_pad_fcs_loop_en),
HCLGE_MAC_APP_LP_B);
pos += scnprintf(buf + pos, len - pos, "app loopback: %s\n",
- state_str[loopback_en]);
+ str_on_off(loopback_en));
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_COMMON_LOOPBACK, true);
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
@@ -2586,22 +2586,22 @@ static int hclge_dbg_dump_loopback(struct hclge_dev *hdev, char *buf, int len)
loopback_en = req_common->enable & HCLGE_CMD_SERDES_SERIAL_INNER_LOOP_B;
pos += scnprintf(buf + pos, len - pos, "serdes serial loopback: %s\n",
- state_str[loopback_en]);
+ str_on_off(loopback_en));
loopback_en = req_common->enable &
HCLGE_CMD_SERDES_PARALLEL_INNER_LOOP_B ? 1 : 0;
pos += scnprintf(buf + pos, len - pos, "serdes parallel loopback: %s\n",
- state_str[loopback_en]);
+ str_on_off(loopback_en));
if (phydev) {
loopback_en = phydev->loopback_enabled;
pos += scnprintf(buf + pos, len - pos, "phy loopback: %s\n",
- state_str[loopback_en]);
+ str_on_off(loopback_en));
} else if (hnae3_dev_phy_imp_supported(hdev)) {
loopback_en = req_common->enable &
HCLGE_CMD_GE_PHY_INNER_LOOP_B;
pos += scnprintf(buf + pos, len - pos, "phy loopback: %s\n",
- state_str[loopback_en]);
+ str_on_off(loopback_en));
}
return 0;
@@ -2648,9 +2648,8 @@ static void hclge_dbg_dump_mac_list(struct hclge_dev *hdev, char *buf, int len,
struct hclge_mac_node *mac_node, *tmp;
struct hclge_vport *vport;
struct list_head *list;
- u32 func_id;
+ u32 func_id, i;
int pos = 0;
- int i;
for (i = 0; i < ARRAY_SIZE(mac_list_items); i++)
result[i] = &data_str[i][0];
@@ -2894,9 +2893,9 @@ static int hclge_dbg_dump_vlan_filter_config(struct hclge_dev *hdev, char *buf,
egress = vlan_fe & HCLGE_FILTER_FE_NIC_EGRESS_B ? 1 : 0;
*pos += scnprintf(buf, len, "I_PORT_VLAN_FILTER: %s\n",
- state_str[ingress]);
+ str_on_off(ingress));
*pos += scnprintf(buf + *pos, len - *pos, "E_PORT_VLAN_FILTER: %s\n",
- state_str[egress]);
+ str_on_off(egress));
hclge_dbg_fill_content(content, sizeof(content), vlan_filter_items,
NULL, ARRAY_SIZE(vlan_filter_items));
@@ -2915,11 +2914,11 @@ static int hclge_dbg_dump_vlan_filter_config(struct hclge_dev *hdev, char *buf,
return ret;
j = 0;
result[j++] = hclge_dbg_get_func_id_str(str_id, i);
- result[j++] = state_str[ingress];
- result[j++] = state_str[egress];
- result[j++] =
- test_bit(HNAE3_DEV_SUPPORT_PORT_VLAN_BYPASS_B,
- hdev->ae_dev->caps) ? state_str[bypass] : "NA";
+ result[j++] = str_on_off(ingress);
+ result[j++] = str_on_off(egress);
+ result[j++] = test_bit(HNAE3_DEV_SUPPORT_PORT_VLAN_BYPASS_B,
+ hdev->ae_dev->caps) ?
+ str_on_off(bypass) : "NA";
hclge_dbg_fill_content(content, sizeof(content),
vlan_filter_items, result,
ARRAY_SIZE(vlan_filter_items));
@@ -2958,19 +2957,19 @@ static int hclge_dbg_dump_vlan_offload_config(struct hclge_dev *hdev, char *buf,
j = 0;
result[j++] = hclge_dbg_get_func_id_str(str_id, i);
result[j++] = str_pvid;
- result[j++] = state_str[vlan_cfg.accept_tag1];
- result[j++] = state_str[vlan_cfg.accept_tag2];
- result[j++] = state_str[vlan_cfg.accept_untag1];
- result[j++] = state_str[vlan_cfg.accept_untag2];
- result[j++] = state_str[vlan_cfg.insert_tag1];
- result[j++] = state_str[vlan_cfg.insert_tag2];
- result[j++] = state_str[vlan_cfg.shift_tag];
- result[j++] = state_str[vlan_cfg.strip_tag1];
- result[j++] = state_str[vlan_cfg.strip_tag2];
- result[j++] = state_str[vlan_cfg.drop_tag1];
- result[j++] = state_str[vlan_cfg.drop_tag2];
- result[j++] = state_str[vlan_cfg.pri_only1];
- result[j++] = state_str[vlan_cfg.pri_only2];
+ result[j++] = str_on_off(vlan_cfg.accept_tag1);
+ result[j++] = str_on_off(vlan_cfg.accept_tag2);
+ result[j++] = str_on_off(vlan_cfg.accept_untag1);
+ result[j++] = str_on_off(vlan_cfg.accept_untag2);
+ result[j++] = str_on_off(vlan_cfg.insert_tag1);
+ result[j++] = str_on_off(vlan_cfg.insert_tag2);
+ result[j++] = str_on_off(vlan_cfg.shift_tag);
+ result[j++] = str_on_off(vlan_cfg.strip_tag1);
+ result[j++] = str_on_off(vlan_cfg.strip_tag2);
+ result[j++] = str_on_off(vlan_cfg.drop_tag1);
+ result[j++] = str_on_off(vlan_cfg.drop_tag2);
+ result[j++] = str_on_off(vlan_cfg.pri_only1);
+ result[j++] = str_on_off(vlan_cfg.pri_only2);
hclge_dbg_fill_content(content, sizeof(content),
vlan_offload_items, result,
@@ -3007,14 +3006,13 @@ static int hclge_dbg_dump_ptp_info(struct hclge_dev *hdev, char *buf, int len)
pos += scnprintf(buf + pos, len - pos, "phc %s's debug info:\n",
ptp->info.name);
pos += scnprintf(buf + pos, len - pos, "ptp enable: %s\n",
- test_bit(HCLGE_PTP_FLAG_EN, &ptp->flags) ?
- "yes" : "no");
+ str_yes_no(test_bit(HCLGE_PTP_FLAG_EN, &ptp->flags)));
pos += scnprintf(buf + pos, len - pos, "ptp tx enable: %s\n",
- test_bit(HCLGE_PTP_FLAG_TX_EN, &ptp->flags) ?
- "yes" : "no");
+ str_yes_no(test_bit(HCLGE_PTP_FLAG_TX_EN,
+ &ptp->flags)));
pos += scnprintf(buf + pos, len - pos, "ptp rx enable: %s\n",
- test_bit(HCLGE_PTP_FLAG_RX_EN, &ptp->flags) ?
- "yes" : "no");
+ str_yes_no(test_bit(HCLGE_PTP_FLAG_RX_EN,
+ &ptp->flags)));
last_rx = jiffies_to_msecs(ptp->last_rx);
pos += scnprintf(buf + pos, len - pos, "last rx time: %lu.%lu\n",
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
index 434f40ce3062..79858b06c97b 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
@@ -582,7 +582,7 @@ static u64 *hclge_comm_get_stats(struct hclge_dev *hdev,
int size, u64 *data)
{
u64 *buf = data;
- u32 i;
+ int i;
for (i = 0; i < size; i++) {
if (strs[i].stats_num > hdev->ae_dev->dev_specs.mac_stats_num)
@@ -595,25 +595,21 @@ static u64 *hclge_comm_get_stats(struct hclge_dev *hdev,
return buf;
}
-static u8 *hclge_comm_get_strings(struct hclge_dev *hdev, u32 stringset,
- const struct hclge_comm_stats_str strs[],
- int size, u8 *data)
+static void hclge_comm_get_strings(struct hclge_dev *hdev, u32 stringset,
+ const struct hclge_comm_stats_str strs[],
+ int size, u8 **data)
{
- char *buff = (char *)data;
- u32 i;
+ int i;
if (stringset != ETH_SS_STATS)
- return buff;
+ return;
for (i = 0; i < size; i++) {
if (strs[i].stats_num > hdev->ae_dev->dev_specs.mac_stats_num)
continue;
- snprintf(buff, ETH_GSTRING_LEN, "%s", strs[i].desc);
- buff = buff + ETH_GSTRING_LEN;
+ ethtool_puts(data, strs[i].desc);
}
-
- return (u8 *)buff;
}
static void hclge_update_stats_for_all(struct hclge_dev *hdev)
@@ -718,44 +714,38 @@ static int hclge_get_sset_count(struct hnae3_handle *handle, int stringset)
}
static void hclge_get_strings(struct hnae3_handle *handle, u32 stringset,
- u8 *data)
+ u8 **data)
{
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_dev *hdev = vport->back;
- u8 *p = (char *)data;
+ const char *str;
int size;
if (stringset == ETH_SS_STATS) {
size = ARRAY_SIZE(g_mac_stats_string);
- p = hclge_comm_get_strings(hdev, stringset, g_mac_stats_string,
- size, p);
- p = hclge_comm_tqps_get_strings(handle, p);
+ hclge_comm_get_strings(hdev, stringset, g_mac_stats_string,
+ size, data);
+ hclge_comm_tqps_get_strings(handle, data);
} else if (stringset == ETH_SS_TEST) {
if (handle->flags & HNAE3_SUPPORT_EXTERNAL_LOOPBACK) {
- memcpy(p, hns3_nic_test_strs[HNAE3_LOOP_EXTERNAL],
- ETH_GSTRING_LEN);
- p += ETH_GSTRING_LEN;
+ str = hns3_nic_test_strs[HNAE3_LOOP_EXTERNAL];
+ ethtool_puts(data, str);
}
if (handle->flags & HNAE3_SUPPORT_APP_LOOPBACK) {
- memcpy(p, hns3_nic_test_strs[HNAE3_LOOP_APP],
- ETH_GSTRING_LEN);
- p += ETH_GSTRING_LEN;
+ str = hns3_nic_test_strs[HNAE3_LOOP_APP];
+ ethtool_puts(data, str);
}
if (handle->flags & HNAE3_SUPPORT_SERDES_SERIAL_LOOPBACK) {
- memcpy(p, hns3_nic_test_strs[HNAE3_LOOP_SERIAL_SERDES],
- ETH_GSTRING_LEN);
- p += ETH_GSTRING_LEN;
+ str = hns3_nic_test_strs[HNAE3_LOOP_SERIAL_SERDES];
+ ethtool_puts(data, str);
}
if (handle->flags & HNAE3_SUPPORT_SERDES_PARALLEL_LOOPBACK) {
- memcpy(p,
- hns3_nic_test_strs[HNAE3_LOOP_PARALLEL_SERDES],
- ETH_GSTRING_LEN);
- p += ETH_GSTRING_LEN;
+ str = hns3_nic_test_strs[HNAE3_LOOP_PARALLEL_SERDES];
+ ethtool_puts(data, str);
}
if (handle->flags & HNAE3_SUPPORT_PHY_LOOPBACK) {
- memcpy(p, hns3_nic_test_strs[HNAE3_LOOP_PHY],
- ETH_GSTRING_LEN);
- p += ETH_GSTRING_LEN;
+ str = hns3_nic_test_strs[HNAE3_LOOP_PHY];
+ ethtool_puts(data, str);
}
}
}
@@ -1587,6 +1577,11 @@ static int hclge_configure(struct hclge_dev *hdev)
hdev->hw.mac.req_autoneg = AUTONEG_ENABLE;
hdev->hw.mac.req_duplex = DUPLEX_FULL;
+ /* When lane_num is 0, the firmware will automatically
+ * select the appropriate lane_num based on the speed.
+ */
+ hdev->hw.mac.req_lane_num = 0;
+
hclge_parse_link_mode(hdev, cfg.speed_ability);
hdev->hw.mac.max_speed = hclge_get_max_speed(cfg.speed_ability);
@@ -2634,7 +2629,7 @@ int hclge_cfg_mac_speed_dup(struct hclge_dev *hdev, int speed, u8 duplex, u8 lan
int ret;
duplex = hclge_check_speed_dup(duplex, speed);
- if (!mac->support_autoneg && mac->speed == speed &&
+ if (!mac->support_autoneg && mac->speed == (u32)speed &&
mac->duplex == duplex && (mac->lane_num == lane_num || lane_num == 0))
return 0;
@@ -2662,8 +2657,11 @@ static int hclge_cfg_mac_speed_dup_h(struct hnae3_handle *handle, int speed,
if (ret)
return ret;
- hdev->hw.mac.req_speed = speed;
- hdev->hw.mac.req_duplex = duplex;
+ hdev->hw.mac.req_lane_num = lane_num;
+ if (speed != SPEED_UNKNOWN)
+ hdev->hw.mac.req_speed = (u32)speed;
+ if (duplex != DUPLEX_UNKNOWN)
+ hdev->hw.mac.req_duplex = duplex;
return 0;
}
@@ -2694,6 +2692,7 @@ static int hclge_set_autoneg(struct hnae3_handle *handle, bool enable)
{
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_dev *hdev = vport->back;
+ int ret;
if (!hdev->hw.mac.support_autoneg) {
if (enable) {
@@ -2705,7 +2704,10 @@ static int hclge_set_autoneg(struct hnae3_handle *handle, bool enable)
}
}
- return hclge_set_autoneg_en(hdev, enable);
+ ret = hclge_set_autoneg_en(hdev, enable);
+ if (!ret)
+ hdev->hw.mac.req_autoneg = enable;
+ return ret;
}
static int hclge_get_autoneg(struct hnae3_handle *handle)
@@ -2967,20 +2969,6 @@ static int hclge_mac_init(struct hclge_dev *hdev)
if (!test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state))
hdev->hw.mac.duplex = HCLGE_MAC_FULL;
- if (hdev->hw.mac.support_autoneg) {
- ret = hclge_set_autoneg_en(hdev, hdev->hw.mac.autoneg);
- if (ret)
- return ret;
- }
-
- if (!hdev->hw.mac.autoneg) {
- ret = hclge_cfg_mac_speed_dup_hw(hdev, hdev->hw.mac.req_speed,
- hdev->hw.mac.req_duplex,
- hdev->hw.mac.lane_num);
- if (ret)
- return ret;
- }
-
mac->link = 0;
if (mac->user_fec_mode & BIT(HNAE3_FEC_USER_DEF)) {
@@ -3368,8 +3356,8 @@ static int hclge_get_phy_link_ksettings(struct hnae3_handle *handle,
}
static int
-hclge_set_phy_link_ksettings(struct hnae3_handle *handle,
- const struct ethtool_link_ksettings *cmd)
+hclge_ethtool_ksettings_set(struct hnae3_handle *handle,
+ const struct ethtool_link_ksettings *cmd)
{
struct hclge_desc desc[HCLGE_PHY_LINK_SETTING_BD_NUM];
struct hclge_vport *vport = hclge_get_vport(handle);
@@ -3410,10 +3398,34 @@ hclge_set_phy_link_ksettings(struct hnae3_handle *handle,
return ret;
}
- hdev->hw.mac.req_autoneg = cmd->base.autoneg;
- hdev->hw.mac.req_speed = cmd->base.speed;
- hdev->hw.mac.req_duplex = cmd->base.duplex;
linkmode_copy(hdev->hw.mac.advertising, cmd->link_modes.advertising);
+ return 0;
+}
+
+static int
+hclge_set_phy_link_ksettings(struct hnae3_handle *handle,
+ const struct ethtool_link_ksettings *cmd)
+{
+ struct hclge_vport *vport = hclge_get_vport(handle);
+ struct hclge_dev *hdev = vport->back;
+ int ret = -ENODEV;
+
+ if (hnae3_dev_phy_imp_supported(hdev)) {
+ ret = hclge_ethtool_ksettings_set(handle, cmd);
+ } else if (handle->netdev->phydev) {
+ if (cmd->base.speed == SPEED_1000 &&
+ cmd->base.autoneg == AUTONEG_DISABLE)
+ return -EINVAL;
+ ret = phy_ethtool_ksettings_set(handle->netdev->phydev, cmd);
+ }
+ if (ret)
+ return ret;
+
+ hdev->hw.mac.req_autoneg = cmd->base.autoneg;
+ if (cmd->base.speed != SPEED_UNKNOWN)
+ hdev->hw.mac.req_speed = cmd->base.speed;
+ if (cmd->base.duplex != DUPLEX_UNKNOWN)
+ hdev->hw.mac.req_duplex = cmd->base.duplex;
return 0;
}
@@ -3456,7 +3468,7 @@ static int hclge_tp_port_init(struct hclge_dev *hdev)
static int hclge_update_port_info(struct hclge_dev *hdev)
{
struct hclge_mac *mac = &hdev->hw.mac;
- int speed;
+ u32 speed;
int ret;
/* get the port info from SFP cmd if not copper port */
@@ -7001,7 +7013,7 @@ static int hclge_get_all_rules(struct hnae3_handle *handle,
struct hclge_dev *hdev = vport->back;
struct hclge_fd_rule *rule;
struct hlist_node *node2;
- int cnt = 0;
+ u32 cnt = 0;
if (!hnae3_ae_dev_fd_supported(hdev->ae_dev))
return -EOPNOTSUPP;
@@ -8012,7 +8024,7 @@ static int hclge_set_loopback(struct hnae3_handle *handle,
ret = hclge_tqp_enable(handle, en);
if (ret)
dev_err(&hdev->pdev->dev, "failed to %s tqp in loopback, ret = %d\n",
- en ? "enable" : "disable", ret);
+ str_enable_disable(en), ret);
return ret;
}
@@ -8235,14 +8247,14 @@ static int hclge_update_desc_vfid(struct hclge_desc *desc, int vfid, bool clr)
word_num = vfid / 32;
bit_num = vfid % 32;
if (clr)
- desc[1].data[word_num] &= cpu_to_le32(~(1 << bit_num));
+ desc[1].data[word_num] &= cpu_to_le32(~(1U << bit_num));
else
desc[1].data[word_num] |= cpu_to_le32(1 << bit_num);
} else {
word_num = (vfid - HCLGE_VF_NUM_IN_FIRST_DESC) / 32;
bit_num = vfid % 32;
if (clr)
- desc[2].data[word_num] &= cpu_to_le32(~(1 << bit_num));
+ desc[2].data[word_num] &= cpu_to_le32(~(1U << bit_num));
else
desc[2].data[word_num] |= cpu_to_le32(1 << bit_num);
}
@@ -9304,7 +9316,7 @@ static int hclge_add_mgr_tbl(struct hclge_dev *hdev,
static int init_mgr_tbl(struct hclge_dev *hdev)
{
int ret;
- int i;
+ u32 i;
for (i = 0; i < ARRAY_SIZE(hclge_mgr_table); i++) {
ret = hclge_add_mgr_tbl(hdev, &hclge_mgr_table[i]);
@@ -10733,7 +10745,7 @@ int hclge_set_vport_mtu(struct hclge_vport *vport, int new_mtu)
max_frm_size = max(max_frm_size, HCLGE_MAC_DEFAULT_FRAME);
mutex_lock(&hdev->vport_lock);
/* VF's mps must fit within hdev->mps */
- if (vport->vport_id && max_frm_size > hdev->mps) {
+ if (vport->vport_id && (u32)max_frm_size > hdev->mps) {
mutex_unlock(&hdev->vport_lock);
return -EINVAL;
} else if (vport->vport_id) {
@@ -10744,7 +10756,7 @@ int hclge_set_vport_mtu(struct hclge_vport *vport, int new_mtu)
/* PF's mps must be greater then VF's mps */
for (i = 1; i < hdev->num_alloc_vport; i++)
- if (max_frm_size < hdev->vport[i].mps) {
+ if ((u32)max_frm_size < hdev->vport[i].mps) {
dev_err(&hdev->pdev->dev,
"failed to set pf mtu for less than vport %d, mps = %u.\n",
i, hdev->vport[i].mps);
@@ -11220,9 +11232,9 @@ static void hclge_info_show(struct hclge_dev *hdev)
dev_info(dev, "This is %s PF\n",
hdev->flag & HCLGE_FLAG_MAIN ? "main" : "not main");
dev_info(dev, "DCB %s\n",
- handle->kinfo.tc_info.dcb_ets_active ? "enable" : "disable");
+ str_enable_disable(handle->kinfo.tc_info.dcb_ets_active));
dev_info(dev, "MQPRIO %s\n",
- handle->kinfo.tc_info.mqprio_active ? "enable" : "disable");
+ str_enable_disable(handle->kinfo.tc_info.mqprio_active));
dev_info(dev, "Default tx spare buffer size: %u\n",
hdev->tx_spare_buf_size);
@@ -11234,7 +11246,7 @@ static int hclge_init_nic_client_instance(struct hnae3_ae_dev *ae_dev,
{
struct hnae3_client *client = vport->nic.client;
struct hclge_dev *hdev = ae_dev->priv;
- int rst_cnt = hdev->rst_stats.reset_cnt;
+ u32 rst_cnt = hdev->rst_stats.reset_cnt;
int ret;
ret = client->ops->init_instance(&vport->nic);
@@ -11278,7 +11290,7 @@ static int hclge_init_roce_client_instance(struct hnae3_ae_dev *ae_dev,
{
struct hclge_dev *hdev = ae_dev->priv;
struct hnae3_client *client;
- int rst_cnt;
+ u32 rst_cnt;
int ret;
if (!hnae3_dev_roce_supported(hdev) || !hdev->roce_client ||
@@ -11736,6 +11748,27 @@ static int hclge_set_wol(struct hnae3_handle *handle,
return ret;
}
+static int hclge_set_autoneg_speed_dup(struct hclge_dev *hdev)
+{
+ int ret;
+
+ if (hdev->hw.mac.support_autoneg) {
+ ret = hclge_set_autoneg_en(hdev, hdev->hw.mac.req_autoneg);
+ if (ret)
+ return ret;
+ }
+
+ if (!hdev->hw.mac.req_autoneg) {
+ ret = hclge_cfg_mac_speed_dup_hw(hdev, hdev->hw.mac.req_speed,
+ hdev->hw.mac.req_duplex,
+ hdev->hw.mac.req_lane_num);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev)
{
struct pci_dev *pdev = ae_dev->pdev;
@@ -11897,6 +11930,20 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev)
if (ret)
goto err_ptp_uninit;
+ if (hdev->hw.mac.media_type != HNAE3_MEDIA_TYPE_COPPER) {
+ hdev->hw.mac.req_autoneg = hdev->hw.mac.autoneg;
+ if (hdev->hw.mac.autoneg == AUTONEG_DISABLE &&
+ hdev->hw.mac.speed != SPEED_UNKNOWN)
+ hdev->hw.mac.req_speed = hdev->hw.mac.speed;
+ }
+
+ ret = hclge_set_autoneg_speed_dup(hdev);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to set autoneg speed duplex, ret = %d\n", ret);
+ goto err_ptp_uninit;
+ }
+
INIT_KFIFO(hdev->mac_tnl_log);
hclge_dcb_ops_set(hdev);
@@ -11996,7 +12043,7 @@ static int hclge_set_vf_spoofchk_hw(struct hclge_dev *hdev, int vf, bool enable)
if (ret) {
dev_err(&hdev->pdev->dev,
"Set vf %d mac spoof check %s failed, ret=%d\n",
- vf, enable ? "on" : "off", ret);
+ vf, str_on_off(enable), ret);
return ret;
}
@@ -12004,7 +12051,7 @@ static int hclge_set_vf_spoofchk_hw(struct hclge_dev *hdev, int vf, bool enable)
if (ret)
dev_err(&hdev->pdev->dev,
"Set vf %d vlan spoof check %s failed, ret=%d\n",
- vf, enable ? "on" : "off", ret);
+ vf, str_on_off(enable), ret);
return ret;
}
@@ -12108,7 +12155,7 @@ static int hclge_vf_rate_param_check(struct hclge_dev *hdev,
int min_tx_rate, int max_tx_rate)
{
if (min_tx_rate != 0 ||
- max_tx_rate < 0 || max_tx_rate > hdev->hw.mac.max_speed) {
+ max_tx_rate < 0 || (u32)max_tx_rate > hdev->hw.mac.max_speed) {
dev_err(&hdev->pdev->dev,
"min_tx_rate:%d [0], max_tx_rate:%d [0, %u]\n",
min_tx_rate, max_tx_rate, hdev->hw.mac.max_speed);
@@ -12133,7 +12180,7 @@ static int hclge_set_vf_rate(struct hnae3_handle *handle, int vf,
if (!vport)
return -EINVAL;
- if (!force && max_tx_rate == vport->vf_info.max_tx_rate)
+ if (!force && (u32)max_tx_rate == vport->vf_info.max_tx_rate)
return 0;
ret = hclge_tm_qs_shaper_cfg(vport, max_tx_rate);
@@ -12227,6 +12274,13 @@ static int hclge_reset_ae_dev(struct hnae3_ae_dev *ae_dev)
return ret;
}
+ ret = hclge_set_autoneg_speed_dup(hdev);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to set autoneg speed duplex, ret = %d\n", ret);
+ return ret;
+ }
+
ret = hclge_tp_port_init(hdev);
if (ret) {
dev_err(&pdev->dev, "failed to init tp port, ret = %d\n",
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
index b9fc719880bb..af50ad906fcf 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
@@ -287,6 +287,7 @@ struct hclge_mac {
u8 support_autoneg;
u8 speed_type; /* 0: sfp speed, 1: active speed */
u8 lane_num;
+ u8 req_lane_num;
u32 speed;
u32 req_speed;
u32 max_speed;
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
index 9eab095d784b..b7d4e06a55d4 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
@@ -749,16 +749,17 @@ static int hclge_get_rss_key(struct hclge_vport *vport,
#define HCLGE_RSS_MBX_RESP_LEN 8
struct hclge_dev *hdev = vport->back;
struct hclge_comm_rss_cfg *rss_cfg;
+ int rss_hash_key_size;
u8 index;
index = mbx_req->msg.data[0];
rss_cfg = &hdev->rss_cfg;
+ rss_hash_key_size = sizeof(rss_cfg->rss_hash_key);
/* Check the query index of rss_hash_key from VF, make sure no
* more than the size of rss_hash_key.
*/
- if (((index + 1) * HCLGE_RSS_MBX_RESP_LEN) >
- sizeof(rss_cfg->rss_hash_key)) {
+ if (((index + 1) * HCLGE_RSS_MBX_RESP_LEN) > rss_hash_key_size) {
dev_warn(&hdev->pdev->dev,
"failed to get the rss hash key, the index(%u) invalid !\n",
index);
@@ -800,7 +801,7 @@ static void hclge_handle_link_change_event(struct hclge_dev *hdev,
static bool hclge_cmd_crq_empty(struct hclge_hw *hw)
{
- u32 tail = hclge_read_dev(hw, HCLGE_COMM_NIC_CRQ_TAIL_REG);
+ int tail = hclge_read_dev(hw, HCLGE_COMM_NIC_CRQ_TAIL_REG);
return tail == hw->hw.cmq.crq.next_to_use;
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c
index b8dbf932caf9..17aa7fcd0706 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c
@@ -151,7 +151,7 @@ int hclge_mac_mdio_config(struct hclge_dev *hdev)
mdio_bus->parent = &hdev->pdev->dev;
mdio_bus->priv = hdev;
- mdio_bus->phy_mask = ~(1 << mac->phy_addr);
+ mdio_bus->phy_mask = ~(1U << mac->phy_addr);
ret = mdiobus_register(mdio_bus);
if (ret) {
dev_err(mdio_bus->parent,
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c
index 16ef5a584af3..4bd52eab3914 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.c
@@ -2,6 +2,7 @@
// Copyright (c) 2021 Hisilicon Limited.
#include <linux/skbuff.h>
+#include <linux/string_choices.h>
#include "hclge_main.h"
#include "hnae3.h"
@@ -226,7 +227,7 @@ static int hclge_ptp_int_en(struct hclge_dev *hdev, bool en)
if (ret)
dev_err(&hdev->pdev->dev,
"failed to %s ptp interrupt, ret = %d\n",
- en ? "enable" : "disable", ret);
+ str_enable_disable(en), ret);
return ret;
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h
index 63483636c074..61faddcc3dd0 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_ptp.h
@@ -25,7 +25,7 @@ struct ifreq;
#define HCLGE_PTP_TIME_SEC_H_MASK GENMASK(15, 0)
#define HCLGE_PTP_TIME_SEC_L_REG 0x54
#define HCLGE_PTP_TIME_NSEC_REG 0x58
-#define HCLGE_PTP_TIME_NSEC_MASK GENMASK(29, 0)
+#define HCLGE_PTP_TIME_NSEC_MASK 0x3fffffffLL
#define HCLGE_PTP_TIME_NSEC_NEG BIT(31)
#define HCLGE_PTP_TIME_SYNC_REG 0x5C
#define HCLGE_PTP_TIME_SYNC_EN BIT(0)
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
index 0bf8fc7e6b3a..b868dc857810 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
@@ -130,12 +130,10 @@ static int hclgevf_get_sset_count(struct hnae3_handle *handle, int strset)
}
static void hclgevf_get_strings(struct hnae3_handle *handle, u32 strset,
- u8 *data)
+ u8 **data)
{
- u8 *p = (char *)data;
-
if (strset == ETH_SS_STATS)
- p = hclge_comm_tqps_get_strings(handle, p);
+ hclge_comm_tqps_get_strings(handle, data);
}
static void hclgevf_get_stats(struct hnae3_handle *handle, u64 *data)
@@ -2467,7 +2465,7 @@ static int hclgevf_init_nic_client_instance(struct hnae3_ae_dev *ae_dev,
struct hnae3_client *client)
{
struct hclgevf_dev *hdev = ae_dev->priv;
- int rst_cnt = hdev->rst_stats.rst_cnt;
+ u32 rst_cnt = hdev->rst_stats.rst_cnt;
int ret;
ret = client->ops->init_instance(&hdev->nic);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
index 85c2a634c8f9..f5c99ca54369 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
@@ -159,7 +159,7 @@ static bool hclgevf_cmd_crq_empty(struct hclgevf_hw *hw)
{
u32 tail = hclgevf_read_dev(hw, HCLGE_COMM_NIC_CRQ_TAIL_REG);
- return tail == hw->hw.cmq.crq.next_to_use;
+ return tail == (u32)hw->hw.cmq.crq.next_to_use;
}
static void hclgevf_handle_mbx_response(struct hclgevf_dev *hdev,
diff --git a/drivers/net/ethernet/intel/i40e/i40e_debug.h b/drivers/net/ethernet/intel/i40e/i40e_debug.h
index e9871dfb32bd..01fd70db9086 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_debug.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_debug.h
@@ -42,7 +42,7 @@ struct device *i40e_hw_to_dev(struct i40e_hw *hw);
#define i40e_debug(h, m, s, ...) \
do { \
if (((m) & (h)->debug_mask)) \
- dev_info(i40e_hw_to_dev(hw), s, ##__VA_ARGS__); \
+ dev_info(i40e_hw_to_dev(h), s, ##__VA_ARGS__); \
} while (0)
#endif /* _I40E_DEBUG_H_ */
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 059cf9376ffc..0b29c750c441 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -9110,7 +9110,6 @@ int i40e_open(struct net_device *netdev)
TCP_FLAG_FIN |
TCP_FLAG_CWR) >> 16);
wr32(&pf->hw, I40E_GLLAN_TSOMSK_L, be32_to_cpu(TCP_FLAG_CWR) >> 16);
- udp_tunnel_get_rx_info(netdev);
return 0;
}
@@ -15963,7 +15962,6 @@ static int i40e_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
pf->udp_tunnel_nic.set_port = i40e_udp_tunnel_set_port;
pf->udp_tunnel_nic.unset_port = i40e_udp_tunnel_unset_port;
- pf->udp_tunnel_nic.flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP;
pf->udp_tunnel_nic.shared = &pf->udp_tunnel_shared;
pf->udp_tunnel_nic.tables[0].n_entries = I40E_MAX_PF_UDP_OFFLOAD_PORTS;
pf->udp_tunnel_nic.tables[0].tunnel_types = UDP_TUNNEL_TYPE_VXLAN |
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 484e48135e17..b01ad401fb60 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -3623,7 +3623,6 @@ ice_set_fc(struct ice_port_info *pi, u8 *aq_failures, bool ena_auto_link_update)
if (!pi || !aq_failures)
return -EINVAL;
- *aq_failures = 0;
hw = pi->hw;
pcaps = kzalloc(sizeof(*pcaps), GFP_KERNEL);
diff --git a/drivers/net/ethernet/intel/ice/ice_dpll.c b/drivers/net/ethernet/intel/ice/ice_dpll.c
index 38e151c7ea23..fffcac5ef9df 100644
--- a/drivers/net/ethernet/intel/ice/ice_dpll.c
+++ b/drivers/net/ethernet/intel/ice/ice_dpll.c
@@ -2279,9 +2279,13 @@ ice_dpll_init_pins_info(struct ice_pf *pf, enum ice_dpll_pin_type pin_type)
static void ice_dpll_deinit_info(struct ice_pf *pf)
{
kfree(pf->dplls.inputs);
+ pf->dplls.inputs = NULL;
kfree(pf->dplls.outputs);
+ pf->dplls.outputs = NULL;
kfree(pf->dplls.eec.input_prio);
+ pf->dplls.eec.input_prio = NULL;
kfree(pf->dplls.pps.input_prio);
+ pf->dplls.pps.input_prio = NULL;
}
/**
@@ -2329,12 +2333,16 @@ static int ice_dpll_init_info(struct ice_pf *pf, bool cgu)
alloc_size = sizeof(*de->input_prio) * d->num_inputs;
de->input_prio = kzalloc(alloc_size, GFP_KERNEL);
- if (!de->input_prio)
- return -ENOMEM;
+ if (!de->input_prio) {
+ ret = -ENOMEM;
+ goto deinit_info;
+ }
dp->input_prio = kzalloc(alloc_size, GFP_KERNEL);
- if (!dp->input_prio)
- return -ENOMEM;
+ if (!dp->input_prio) {
+ ret = -ENOMEM;
+ goto deinit_info;
+ }
ret = ice_dpll_init_pins_info(pf, ICE_DPLL_PIN_TYPE_INPUT);
if (ret)
@@ -2356,12 +2364,12 @@ static int ice_dpll_init_info(struct ice_pf *pf, bool cgu)
ret = ice_get_cgu_rclk_pin_info(&pf->hw, &d->base_rclk_idx,
&pf->dplls.rclk.num_parents);
if (ret)
- return ret;
+ goto deinit_info;
for (i = 0; i < pf->dplls.rclk.num_parents; i++)
pf->dplls.rclk.parent_idx[i] = d->base_rclk_idx + i;
ret = ice_dpll_init_pins_info(pf, ICE_DPLL_PIN_TYPE_RCLK_INPUT);
if (ret)
- return ret;
+ goto deinit_info;
de->mode = DPLL_MODE_AUTOMATIC;
dp->mode = DPLL_MODE_AUTOMATIC;
diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.c b/drivers/net/ethernet/intel/ice/ice_eswitch.c
index 5b9a7ee278f1..32b52ecff04f 100644
--- a/drivers/net/ethernet/intel/ice/ice_eswitch.c
+++ b/drivers/net/ethernet/intel/ice/ice_eswitch.c
@@ -116,8 +116,6 @@ static int ice_eswitch_setup_repr(struct ice_pf *pf, struct ice_repr *repr)
if (!repr->dst)
return -ENOMEM;
- netif_keep_dst(uplink_vsi->netdev);
-
dst = repr->dst;
dst->u.port_info.port_id = vsi->vsi_num;
dst->u.port_info.lower_dev = uplink_vsi->netdev;
@@ -306,6 +304,8 @@ static int ice_eswitch_enable_switchdev(struct ice_pf *pf)
if (ice_eswitch_br_offloads_init(pf))
goto err_br_offloads;
+ netif_keep_dst(uplink_vsi->netdev);
+
pf->eswitch.is_running = true;
return 0;
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index c0bf93b38cbd..e6e635ad971f 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -3499,7 +3499,7 @@ ice_set_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
struct ice_vsi *vsi = np->vsi;
struct ice_hw *hw = &pf->hw;
struct ice_port_info *pi;
- u8 aq_failures;
+ u8 aq_failures = 0;
bool link_up;
u32 is_an;
int err;
@@ -3570,18 +3570,22 @@ ice_set_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
/* Set the FC mode and only restart AN if link is up */
err = ice_set_fc(pi, &aq_failures, link_up);
- if (aq_failures & ICE_SET_FC_AQ_FAIL_GET) {
+ switch (aq_failures) {
+ case ICE_SET_FC_AQ_FAIL_GET:
netdev_info(netdev, "Set fc failed on the get_phy_capabilities call with err %d aq_err %s\n",
err, ice_aq_str(hw->adminq.sq_last_status));
err = -EAGAIN;
- } else if (aq_failures & ICE_SET_FC_AQ_FAIL_SET) {
+ break;
+ case ICE_SET_FC_AQ_FAIL_SET:
netdev_info(netdev, "Set fc failed on the set_phy_config call with err %d aq_err %s\n",
err, ice_aq_str(hw->adminq.sq_last_status));
err = -EAGAIN;
- } else if (aq_failures & ICE_SET_FC_AQ_FAIL_UPDATE) {
+ break;
+ case ICE_SET_FC_AQ_FAIL_UPDATE:
netdev_info(netdev, "Set fc failed on the get_link_info call with err %d aq_err %s\n",
err, ice_aq_str(hw->adminq.sq_last_status));
err = -EAGAIN;
+ break;
}
return err;
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 664bedfbd805..f1c50910b433 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -4817,7 +4817,6 @@ int ice_init_dev(struct ice_pf *pf)
pf->hw.udp_tunnel_nic.set_port = ice_udp_tunnel_set_port;
pf->hw.udp_tunnel_nic.unset_port = ice_udp_tunnel_unset_port;
- pf->hw.udp_tunnel_nic.flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP;
pf->hw.udp_tunnel_nic.shared = &pf->hw.udp_tunnel_shared;
if (pf->hw.tnl.valid_count[TNL_VXLAN]) {
pf->hw.udp_tunnel_nic.tables[0].n_entries =
@@ -4944,16 +4943,14 @@ static void ice_init_wakeup(struct ice_pf *pf)
device_set_wakeup_enable(ice_pf_to_dev(pf), false);
}
-static int ice_init_link(struct ice_pf *pf)
+static void ice_init_link(struct ice_pf *pf)
{
struct device *dev = ice_pf_to_dev(pf);
int err;
err = ice_init_link_events(pf->hw.port_info);
- if (err) {
+ if (err)
dev_err(dev, "ice_init_link_events failed: %d\n", err);
- return err;
- }
/* not a fatal error if this fails */
err = ice_init_nvm_phy_type(pf->hw.port_info);
@@ -4987,8 +4984,6 @@ static int ice_init_link(struct ice_pf *pf)
} else {
set_bit(ICE_FLAG_NO_MEDIA, pf->flags);
}
-
- return err;
}
static int ice_init_pf_sw(struct ice_pf *pf)
@@ -5120,13 +5115,11 @@ static int ice_init(struct ice_pf *pf)
ice_init_wakeup(pf);
- err = ice_init_link(pf);
- if (err)
- goto err_init_link;
+ ice_init_link(pf);
err = ice_send_version(pf);
if (err)
- goto err_init_link;
+ goto err_deinit_pf_sw;
ice_verify_cacheline_size(pf);
@@ -5145,7 +5138,7 @@ static int ice_init(struct ice_pf *pf)
return 0;
-err_init_link:
+err_deinit_pf_sw:
ice_deinit_pf_sw(pf);
err_init_pf_sw:
ice_dealloc_vsis(pf);
@@ -9581,9 +9574,6 @@ int ice_open_internal(struct net_device *netdev)
netdev_err(netdev, "Failed to open VSI 0x%04X on switch 0x%04X\n",
vsi->vsi_num, vsi->vsw->sw_id);
- /* Update existing tunnels information */
- udp_tunnel_get_rx_info(netdev);
-
return err;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
index 5267b08011fc..afa9e7ce4637 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
@@ -800,7 +800,7 @@ void ice_reset_all_vfs(struct ice_pf *pf)
* setup only when VF creates its first FDIR rule.
*/
if (vf->ctrl_vsi_idx != ICE_NO_VSI)
- ice_vf_ctrl_invalidate_vsi(vf);
+ ice_vf_ctrl_vsi_release(vf);
ice_vf_pre_vsi_rebuild(vf);
if (ice_vf_rebuild_vsi(vf)) {
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index d72b2d5f96db..a20a02b7926b 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -5866,6 +5866,9 @@ static int mvneta_resume(struct device *device)
rtnl_unlock();
mvneta_set_rx_mode(dev);
+ if (!pp->neta_armada3700)
+ on_each_cpu(mvneta_percpu_enable, pp, true);
+
return 0;
}
#endif
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs.c b/drivers/net/ethernet/marvell/octeontx2/af/mcs.c
index c1775bd01c2b..a07e0b3d8d00 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/mcs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs.c
@@ -120,13 +120,13 @@ void mcs_get_rx_secy_stats(struct mcs *mcs, struct mcs_secy_stats *stats, int id
reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYUNTAGGEDX(id);
stats->pkt_untaged_cnt = mcs_reg_read(mcs, reg);
- reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYCTLX(id);
- stats->pkt_ctl_cnt = mcs_reg_read(mcs, reg);
-
if (mcs->hw->mcs_blks > 1) {
reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYNOTAGX(id);
stats->pkt_notag_cnt = mcs_reg_read(mcs, reg);
+ return;
}
+ reg = MCSX_CSE_RX_MEM_SLAVE_INPKTSSECYCTLX(id);
+ stats->pkt_ctl_cnt = mcs_reg_read(mcs, reg);
}
void mcs_get_flowid_stats(struct mcs *mcs, struct mcs_flowid_stats *stats,
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
index e24accfecb3f..53c2a9d6250a 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
@@ -505,10 +505,11 @@ static int rvu_dbg_mcs_rx_secy_stats_display(struct seq_file *filp, void *unused
seq_printf(filp, "secy%d: Tagged ctrl pkts: %lld\n", secy_id,
stats.pkt_tagged_ctl_cnt);
seq_printf(filp, "secy%d: Untaged pkts: %lld\n", secy_id, stats.pkt_untaged_cnt);
- seq_printf(filp, "secy%d: Ctrl pkts: %lld\n", secy_id, stats.pkt_ctl_cnt);
if (mcs->hw->mcs_blks > 1)
seq_printf(filp, "secy%d: pkts notag: %lld\n", secy_id,
stats.pkt_notag_cnt);
+ else
+ seq_printf(filp, "secy%d: Ctrl pkts: %lld\n", secy_id, stats.pkt_ctl_cnt);
}
mutex_unlock(&mcs->stats_lock);
return 0;
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c
index 6f8914431de4..dac421cbaf0c 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c
@@ -1399,7 +1399,9 @@ static int rvu_af_dl_nix_maxlf_validate(struct devlink *devlink, u32 id,
struct rvu_devlink *rvu_dl = devlink_priv(devlink);
struct rvu *rvu = rvu_dl->rvu;
u16 max_nix0_lf, max_nix1_lf;
- struct npc_mcam *mcam;
+ struct rvu_block *block;
+ int blkaddr = 0;
+ int free_lfs;
u64 cfg;
cfg = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_CONST2);
@@ -1407,14 +1409,23 @@ static int rvu_af_dl_nix_maxlf_validate(struct devlink *devlink, u32 id,
cfg = rvu_read64(rvu, BLKADDR_NIX1, NIX_AF_CONST2);
max_nix1_lf = cfg & 0xFFF;
- /* Do not allow user to modify maximum NIX LFs while mcam entries
- * have already been assigned.
+ /* Do not allow user to modify maximum NIX LFs while NIX LFs
+ * have already been assigned. Note that modifying NIX LFs count
+ * can be done only before any LF attach requests from PFs and VFs
+ * and not later or concurrently.
*/
- mcam = &rvu->hw->mcam;
- if (mcam->bmap_fcnt < mcam->bmap_entries) {
- NL_SET_ERR_MSG_MOD(extack,
- "mcam entries have already been assigned, can't resize");
- return -EPERM;
+ blkaddr = rvu_get_next_nix_blkaddr(rvu, blkaddr);
+ while (blkaddr) {
+ block = &rvu->hw->block[blkaddr];
+
+ free_lfs = rvu_rsrc_free_count(&block->lf);
+ if (free_lfs != block->lf.max) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "NIX LFs already assigned, can't resize");
+ return -EPERM;
+ }
+
+ blkaddr = rvu_get_next_nix_blkaddr(rvu, blkaddr);
}
if (max_nix0_lf && val.vu16 > max_nix0_lf) {
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c
index 0a23e0a91828..f9b7263eeb37 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c
@@ -524,19 +524,24 @@ static int nix_setup_bpids(struct rvu *rvu, struct nix_hw *hw, int blkaddr)
bp->fn_map = devm_kcalloc(rvu->dev, bp->bpids.max,
sizeof(u16), GFP_KERNEL);
if (!bp->fn_map)
- return -ENOMEM;
+ goto free_bpids;
bp->intf_map = devm_kcalloc(rvu->dev, bp->bpids.max,
sizeof(u8), GFP_KERNEL);
if (!bp->intf_map)
- return -ENOMEM;
+ goto free_bpids;
bp->ref_cnt = devm_kcalloc(rvu->dev, bp->bpids.max,
sizeof(u8), GFP_KERNEL);
if (!bp->ref_cnt)
- return -ENOMEM;
+ goto free_bpids;
return 0;
+
+free_bpids:
+ rvu_free_bitmap(&bp->bpids);
+ bp->bpids.bmap = NULL;
+ return -ENOMEM;
}
void rvu_nix_flr_free_bpids(struct rvu *rvu, u16 pcifunc)
@@ -4473,7 +4478,7 @@ int rvu_mbox_handler_nix_set_rx_mode(struct rvu *rvu, struct nix_rx_mode *req,
rvu_npc_install_allmulti_entry(rvu, pcifunc, nixlf,
pfvf->rx_chan_base);
} else {
- if (!nix_rx_multicast)
+ if (!nix_rx_multicast && !is_vf(pcifunc))
rvu_npc_enable_allmulti_entry(rvu, pcifunc, nixlf, false);
}
@@ -4483,7 +4488,7 @@ int rvu_mbox_handler_nix_set_rx_mode(struct rvu *rvu, struct nix_rx_mode *req,
pfvf->rx_chan_base,
pfvf->rx_chan_cnt);
else
- if (!nix_rx_multicast)
+ if (!nix_rx_multicast && !is_vf(pcifunc))
rvu_npc_enable_promisc_entry(rvu, pcifunc, nixlf, false);
return 0;
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c
index 0163fbb758d3..5c5376eace10 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c
@@ -1933,7 +1933,7 @@ int npc_mcam_rsrcs_init(struct rvu *rvu, int blkaddr)
/* Alloc memory for MCAM entry to counter mapping and for tracking
* counter's reference count.
*/
- mcam->entry2cntr_map = kcalloc(mcam->bmap_entries, sizeof(u16),
+ mcam->entry2cntr_map = kcalloc(mcam->total_entries, sizeof(u16),
GFP_KERNEL);
if (!mcam->entry2cntr_map)
goto free_cntr_map;
@@ -1949,10 +1949,11 @@ int npc_mcam_rsrcs_init(struct rvu *rvu, int blkaddr)
if (!mcam->entry2target_pffunc)
goto free_cntr_refcnt;
- for (index = 0; index < mcam->bmap_entries; index++) {
+ for (index = 0; index < mcam->bmap_entries; index++)
mcam->entry2pfvf_map[index] = NPC_MCAM_INVALID_MAP;
+
+ for (index = 0; index < mcam->total_entries; index++)
mcam->entry2cntr_map[index] = NPC_MCAM_INVALID_MAP;
- }
for (cntr = 0; cntr < mcam->counters.max; cntr++)
mcam->cntr2pfvf_map[cntr] = NPC_MCAM_INVALID_MAP;
@@ -3046,7 +3047,7 @@ int rvu_mbox_handler_npc_mcam_free_counter(struct rvu *rvu,
struct npc_mcam_oper_counter_req *req, struct msg_rsp *rsp)
{
struct npc_mcam *mcam = &rvu->hw->mcam;
- u16 index, entry = 0;
+ u16 index;
int blkaddr, err;
blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
@@ -3064,20 +3065,16 @@ int rvu_mbox_handler_npc_mcam_free_counter(struct rvu *rvu,
mcam->cntr2pfvf_map[req->cntr] = NPC_MCAM_INVALID_MAP;
rvu_free_rsrc(&mcam->counters, req->cntr);
- /* Disable all MCAM entry's stats which are using this counter */
- while (entry < mcam->bmap_entries) {
+ /* Disable all MCAM entry's stats which are using this counter.
+ * Scan the full MCAM index range: AF-reserved rules (e.g. CPT pass-2)
+ */
+ for (index = 0; index < mcam->total_entries; index++) {
if (!mcam->cntr_refcnt[req->cntr])
break;
-
- index = find_next_bit(mcam->bmap, mcam->bmap_entries, entry);
- if (index >= mcam->bmap_entries)
- break;
- entry = index + 1;
if (mcam->entry2cntr_map[index] != req->cntr)
continue;
-
- npc_unmap_mcam_entry_and_cntr(rvu, mcam, blkaddr,
- index, req->cntr);
+ npc_unmap_mcam_entry_and_cntr(rvu, mcam, blkaddr, index,
+ req->cntr);
}
mutex_unlock(&mcam->lock);
@@ -3088,7 +3085,7 @@ int rvu_mbox_handler_npc_mcam_unmap_counter(struct rvu *rvu,
struct npc_mcam_unmap_counter_req *req, struct msg_rsp *rsp)
{
struct npc_mcam *mcam = &rvu->hw->mcam;
- u16 index, entry = 0;
+ u16 index;
int blkaddr, rc;
blkaddr = rvu_get_blkaddr(rvu, BLKTYPE_NPC, 0);
@@ -3111,20 +3108,13 @@ int rvu_mbox_handler_npc_mcam_unmap_counter(struct rvu *rvu,
}
/* Disable all MCAM entry's stats which are using this counter */
- while (entry < mcam->bmap_entries) {
+ for (index = 0; index < mcam->total_entries; index++) {
if (!mcam->cntr_refcnt[req->cntr])
break;
-
- index = find_next_bit(mcam->bmap, mcam->bmap_entries, entry);
- if (index >= mcam->bmap_entries)
- break;
- entry = index + 1;
-
if (mcam->entry2cntr_map[index] != req->cntr)
continue;
-
- npc_unmap_mcam_entry_and_cntr(rvu, mcam, blkaddr,
- index, req->cntr);
+ npc_unmap_mcam_entry_and_cntr(rvu, mcam, blkaddr, index,
+ req->cntr);
}
exit:
mutex_unlock(&mcam->lock);
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c
index 3af58bc9f533..8f07d19819ce 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k_macsec.c
@@ -213,6 +213,7 @@ static void cn10k_mcs_free_rsrc(struct otx2_nic *pfvf, enum mcs_direction dir,
clear_req->id = hw_rsrc_id;
clear_req->type = type;
clear_req->dir = dir;
+ clear_req->all = all;
req = otx2_mbox_alloc_msg_mcs_free_resources(mbox);
if (!req)
@@ -1807,11 +1808,16 @@ int cn10k_mcs_init(struct otx2_nic *pfvf)
void cn10k_mcs_free(struct otx2_nic *pfvf)
{
+ struct cn10k_mcs_cfg *cfg = pfvf->macsec_cfg;
+
if (!test_bit(CN10K_HW_MACSEC, &pfvf->hw.cap_flag))
return;
- cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SECY, 0, true);
- cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SECY, 0, true);
+ if (!list_empty(&cfg->txsc_list)) {
+ cn10k_mcs_free_rsrc(pfvf, MCS_TX, MCS_RSRC_TYPE_SECY, 0, true);
+ cn10k_mcs_free_rsrc(pfvf, MCS_RX, MCS_RSRC_TYPE_SECY, 0, true);
+ }
+
kfree(pfvf->macsec_cfg);
pfvf->macsec_cfg = NULL;
}
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
index ce01fab28624..b0b0b4929e8c 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
@@ -1444,6 +1444,7 @@ static void otx2_free_sq_res(struct otx2_nic *pf)
continue;
qmem_free(pf->dev, sq->sqe);
qmem_free(pf->dev, sq->tso_hdrs);
+ qmem_free(pf->dev, sq->timestamps);
kfree(sq->sg);
kfree(sq->sqb_ptrs);
}
@@ -2364,10 +2365,42 @@ EXPORT_SYMBOL(otx2_ioctl);
static int otx2_do_set_vf_mac(struct otx2_nic *pf, int vf, const u8 *mac)
{
+ struct npc_get_field_status_req *freq;
+ struct npc_get_field_status_rsp *frsp;
struct npc_install_flow_req *req;
int err;
mutex_lock(&pf->mbox.lock);
+
+ /* Skip installing the DMAC filter if the hardware parser profile
+ * does not support DMAC extraction.
+ */
+ freq = otx2_mbox_alloc_msg_npc_get_field_status(&pf->mbox);
+ if (!freq) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ freq->field = NPC_DMAC;
+ err = otx2_sync_mbox_msg(&pf->mbox);
+ if (err)
+ goto out;
+
+ frsp = (struct npc_get_field_status_rsp *)otx2_mbox_get_rsp
+ (&pf->mbox.mbox, 0, &freq->hdr);
+ if (IS_ERR(frsp)) {
+ err = PTR_ERR(frsp);
+ goto out;
+ }
+
+ if (!frsp->enable) {
+ netdev_warn(pf->netdev,
+ "VF %d MAC filter not installed: DMAC extraction not supported by parser profile\n",
+ vf);
+ err = -EOPNOTSUPP;
+ goto out;
+ }
+
req = otx2_mbox_alloc_msg_npc_install_flow(&pf->mbox);
if (!req) {
err = -ENOMEM;
@@ -2406,13 +2439,12 @@ static int otx2_set_vf_mac(struct net_device *netdev, int vf, u8 *mac)
if (!is_valid_ether_addr(mac))
return -EINVAL;
- config = &pf->vf_configs[vf];
- ether_addr_copy(config->mac, mac);
-
ret = otx2_do_set_vf_mac(pf, vf, mac);
- if (ret == 0)
- dev_info(&pdev->dev,
- "Load/Reload VF driver\n");
+ if (ret == 0) {
+ config = &pf->vf_configs[vf];
+ ether_addr_copy(config->mac, mac);
+ dev_info(&pdev->dev, "Load/Reload VF driver\n");
+ }
return ret;
}
diff --git a/drivers/net/ethernet/marvell/prestera/prestera_main.c b/drivers/net/ethernet/marvell/prestera/prestera_main.c
index 22ca6ee9665e..3f7cc04a873b 100644
--- a/drivers/net/ethernet/marvell/prestera/prestera_main.c
+++ b/drivers/net/ethernet/marvell/prestera/prestera_main.c
@@ -372,7 +372,7 @@ static int prestera_port_sfp_bind(struct prestera_port *port)
struct device_node *ports, *node;
struct fwnode_handle *fwnode;
struct phylink *phy_link;
- int err;
+ int err = 0;
if (!sw->np)
return 0;
diff --git a/drivers/net/ethernet/mediatek/airoha_eth.c b/drivers/net/ethernet/mediatek/airoha_eth.c
index 7b929b20fe64..07f89e08b5fa 100644
--- a/drivers/net/ethernet/mediatek/airoha_eth.c
+++ b/drivers/net/ethernet/mediatek/airoha_eth.c
@@ -15,6 +15,7 @@
#include <linux/u64_stats_sync.h>
#include <net/dsa.h>
#include <net/page_pool/helpers.h>
+#include <net/pkt_cls.h>
#include <uapi/linux/ppp_defs.h>
#define AIROHA_MAX_NUM_GDM_PORTS 1
@@ -23,6 +24,8 @@
#define AIROHA_MAX_NUM_XSI_RSTS 5
#define AIROHA_MAX_MTU 2000
#define AIROHA_MAX_PACKET_SIZE 2048
+#define AIROHA_NUM_QOS_CHANNELS 4
+#define AIROHA_NUM_QOS_QUEUES 8
#define AIROHA_NUM_TX_RING 32
#define AIROHA_NUM_RX_RING 32
#define AIROHA_FE_MC_MAX_VLAN_TABLE 64
@@ -541,9 +544,24 @@
#define INGRESS_SLOW_TICK_RATIO_MASK GENMASK(29, 16)
#define INGRESS_FAST_TICK_MASK GENMASK(15, 0)
+#define REG_QUEUE_CLOSE_CFG(_n) (0x00a0 + ((_n) & 0xfc))
+#define TXQ_DISABLE_CHAN_QUEUE_MASK(_n, _m) BIT((_m) + (((_n) & 0x3) << 3))
+
#define REG_TXQ_DIS_CFG_BASE(_n) ((_n) ? 0x20a0 : 0x00a0)
#define REG_TXQ_DIS_CFG(_n, _m) (REG_TXQ_DIS_CFG_BASE((_n)) + (_m) << 2)
+#define REG_CNTR_CFG(_n) (0x0400 + ((_n) << 3))
+#define CNTR_EN_MASK BIT(31)
+#define CNTR_ALL_CHAN_EN_MASK BIT(30)
+#define CNTR_ALL_QUEUE_EN_MASK BIT(29)
+#define CNTR_ALL_DSCP_RING_EN_MASK BIT(28)
+#define CNTR_SRC_MASK GENMASK(27, 24)
+#define CNTR_DSCP_RING_MASK GENMASK(20, 16)
+#define CNTR_CHAN_MASK GENMASK(7, 3)
+#define CNTR_QUEUE_MASK GENMASK(2, 0)
+
+#define REG_CNTR_VAL(_n) (0x0404 + ((_n) << 3))
+
#define REG_LMGR_INIT_CFG 0x1000
#define LMGR_INIT_START BIT(31)
#define LMGR_SRAM_MODE_MASK BIT(30)
@@ -569,9 +587,19 @@
#define TWRR_WEIGHT_SCALE_MASK BIT(31)
#define TWRR_WEIGHT_BASE_MASK BIT(3)
+#define REG_TXWRR_WEIGHT_CFG 0x1024
+#define TWRR_RW_CMD_MASK BIT(31)
+#define TWRR_RW_CMD_DONE BIT(30)
+#define TWRR_CHAN_IDX_MASK GENMASK(23, 19)
+#define TWRR_QUEUE_IDX_MASK GENMASK(18, 16)
+#define TWRR_VALUE_MASK GENMASK(15, 0)
+
#define REG_PSE_BUF_USAGE_CFG 0x1028
#define PSE_BUF_ESTIMATE_EN_MASK BIT(29)
+#define REG_CHAN_QOS_MODE(_n) (0x1040 + ((_n) << 2))
+#define CHAN_QOS_MODE_MASK(_n) GENMASK(2 + ((_n) << 2), (_n) << 2)
+
#define REG_GLB_TRTCM_CFG 0x1080
#define GLB_TRTCM_EN_MASK BIT(31)
#define GLB_TRTCM_MODE_MASK BIT(30)
@@ -720,6 +748,17 @@ enum {
FE_PSE_PORT_DROP = 0xf,
};
+enum tx_sched_mode {
+ TC_SCH_WRR8,
+ TC_SCH_SP,
+ TC_SCH_WRR7,
+ TC_SCH_WRR6,
+ TC_SCH_WRR5,
+ TC_SCH_WRR4,
+ TC_SCH_WRR3,
+ TC_SCH_WRR2,
+};
+
struct airoha_queue_entry {
union {
void *buf;
@@ -810,6 +849,10 @@ struct airoha_gdm_port {
int id;
struct airoha_hw_stats stats;
+
+ /* qos stats counters */
+ u64 cpu_tx_packets;
+ u64 fwd_tx_packets;
};
struct airoha_eth {
@@ -1195,7 +1238,7 @@ static void airoha_fe_pse_ports_init(struct airoha_eth *eth)
PSE_QUEUE_RSV_PAGES);
/* PPE1 */
for (q = 0; q < pse_port_num_queues[FE_PSE_PORT_PPE1]; q++) {
- if (q < pse_port_num_queues[FE_PSE_PORT_PPE1])
+ if (q < pse_port_num_queues[FE_PSE_PORT_PPE1] / 2)
airoha_fe_set_pse_oq_rsv(eth, FE_PSE_PORT_PPE1, q,
PSE_QUEUE_RSV_PAGES);
else
@@ -1966,6 +2009,27 @@ static void airoha_qdma_init_qos(struct airoha_qdma *qdma)
FIELD_PREP(SLA_SLOW_TICK_RATIO_MASK, 40));
}
+static void airoha_qdma_init_qos_stats(struct airoha_qdma *qdma)
+{
+ int i;
+
+ for (i = 0; i < AIROHA_NUM_QOS_CHANNELS; i++) {
+ /* Tx-cpu transferred count */
+ airoha_qdma_wr(qdma, REG_CNTR_VAL(i << 1), 0);
+ airoha_qdma_wr(qdma, REG_CNTR_CFG(i << 1),
+ CNTR_EN_MASK | CNTR_ALL_QUEUE_EN_MASK |
+ CNTR_ALL_DSCP_RING_EN_MASK |
+ FIELD_PREP(CNTR_CHAN_MASK, i));
+ /* Tx-fwd transferred count */
+ airoha_qdma_wr(qdma, REG_CNTR_VAL((i << 1) + 1), 0);
+ airoha_qdma_wr(qdma, REG_CNTR_CFG(i << 1),
+ CNTR_EN_MASK | CNTR_ALL_QUEUE_EN_MASK |
+ CNTR_ALL_DSCP_RING_EN_MASK |
+ FIELD_PREP(CNTR_SRC_MASK, 1) |
+ FIELD_PREP(CNTR_CHAN_MASK, i));
+ }
+}
+
static int airoha_qdma_hw_init(struct airoha_qdma *qdma)
{
int i;
@@ -2016,6 +2080,7 @@ static int airoha_qdma_hw_init(struct airoha_qdma *qdma)
airoha_qdma_set(qdma, REG_TXQ_CNGST_CFG,
TXQ_CNGST_DROP_EN | TXQ_CNGST_DEI_DROP_EN);
+ airoha_qdma_init_qos_stats(qdma);
return 0;
}
@@ -2428,21 +2493,44 @@ static void airoha_dev_get_stats64(struct net_device *dev,
} while (u64_stats_fetch_retry(&port->stats.syncp, start));
}
+static u16 airoha_dev_select_queue(struct net_device *dev, struct sk_buff *skb,
+ struct net_device *sb_dev)
+{
+ struct airoha_gdm_port *port = netdev_priv(dev);
+ int queue, channel;
+
+ /* For dsa device select QoS channel according to the dsa user port
+ * index, rely on port id otherwise. Select QoS queue based on the
+ * skb priority.
+ */
+ channel = netdev_uses_dsa(dev) ? skb_get_queue_mapping(skb) : port->id;
+ channel = channel % AIROHA_NUM_QOS_CHANNELS;
+ queue = (skb->priority - 1) % AIROHA_NUM_QOS_QUEUES; /* QoS queue */
+ queue = channel * AIROHA_NUM_QOS_QUEUES + queue;
+
+ return queue < dev->num_tx_queues ? queue : 0;
+}
+
static netdev_tx_t airoha_dev_xmit(struct sk_buff *skb,
struct net_device *dev)
{
struct skb_shared_info *sinfo = skb_shinfo(skb);
struct airoha_gdm_port *port = netdev_priv(dev);
- u32 msg0 = 0, msg1, len = skb_headlen(skb);
- int i, qid = skb_get_queue_mapping(skb);
+ u32 msg0, msg1, len = skb_headlen(skb);
struct airoha_qdma *qdma = port->qdma;
u32 nr_frags = 1 + sinfo->nr_frags;
struct netdev_queue *txq;
struct airoha_queue *q;
void *data = skb->data;
+ int i, qid;
u16 index;
u8 fport;
+ qid = skb_get_queue_mapping(skb) % ARRAY_SIZE(qdma->q_tx);
+ msg0 = FIELD_PREP(QDMA_ETH_TXMSG_CHAN_MASK,
+ qid / AIROHA_NUM_QOS_QUEUES) |
+ FIELD_PREP(QDMA_ETH_TXMSG_QUEUE_MASK,
+ qid % AIROHA_NUM_QOS_QUEUES);
if (skb->ip_summed == CHECKSUM_PARTIAL)
msg0 |= FIELD_PREP(QDMA_ETH_TXMSG_TCO_MASK, 1) |
FIELD_PREP(QDMA_ETH_TXMSG_UCO_MASK, 1) |
@@ -2612,13 +2700,147 @@ airoha_ethtool_get_rmon_stats(struct net_device *dev,
} while (u64_stats_fetch_retry(&port->stats.syncp, start));
}
+static int airoha_qdma_set_chan_tx_sched(struct airoha_gdm_port *port,
+ int channel, enum tx_sched_mode mode,
+ const u16 *weights, u8 n_weights)
+{
+ int i;
+
+ for (i = 0; i < AIROHA_NUM_TX_RING; i++)
+ airoha_qdma_clear(port->qdma, REG_QUEUE_CLOSE_CFG(channel),
+ TXQ_DISABLE_CHAN_QUEUE_MASK(channel, i));
+
+ for (i = 0; i < n_weights; i++) {
+ u32 status;
+ int err;
+
+ airoha_qdma_wr(port->qdma, REG_TXWRR_WEIGHT_CFG,
+ TWRR_RW_CMD_MASK |
+ FIELD_PREP(TWRR_CHAN_IDX_MASK, channel) |
+ FIELD_PREP(TWRR_QUEUE_IDX_MASK, i) |
+ FIELD_PREP(TWRR_VALUE_MASK, weights[i]));
+ err = read_poll_timeout(airoha_qdma_rr, status,
+ status & TWRR_RW_CMD_DONE,
+ USEC_PER_MSEC, 10 * USEC_PER_MSEC,
+ true, port->qdma,
+ REG_TXWRR_WEIGHT_CFG);
+ if (err)
+ return err;
+ }
+
+ airoha_qdma_rmw(port->qdma, REG_CHAN_QOS_MODE(channel >> 3),
+ CHAN_QOS_MODE_MASK(channel),
+ mode << __ffs(CHAN_QOS_MODE_MASK(channel)));
+
+ return 0;
+}
+
+static int airoha_qdma_set_tx_prio_sched(struct airoha_gdm_port *port,
+ int channel)
+{
+ static const u16 w[AIROHA_NUM_QOS_QUEUES] = {};
+
+ return airoha_qdma_set_chan_tx_sched(port, channel, TC_SCH_SP, w,
+ ARRAY_SIZE(w));
+}
+
+static int airoha_qdma_set_tx_ets_sched(struct airoha_gdm_port *port,
+ int channel,
+ struct tc_ets_qopt_offload *opt)
+{
+ struct tc_ets_qopt_offload_replace_params *p = &opt->replace_params;
+ enum tx_sched_mode mode = TC_SCH_SP;
+ u16 w[AIROHA_NUM_QOS_QUEUES] = {};
+ int i, nstrict = 0;
+
+ if (p->bands > AIROHA_NUM_QOS_QUEUES)
+ return -EINVAL;
+
+ for (i = 0; i < p->bands; i++) {
+ if (!p->quanta[i])
+ nstrict++;
+ }
+
+ /* this configuration is not supported by the hw */
+ if (nstrict == AIROHA_NUM_QOS_QUEUES - 1)
+ return -EINVAL;
+
+ for (i = 0; i < p->bands - nstrict; i++)
+ w[i] = p->weights[nstrict + i];
+
+ if (!nstrict)
+ mode = TC_SCH_WRR8;
+ else if (nstrict < AIROHA_NUM_QOS_QUEUES - 1)
+ mode = nstrict + 1;
+
+ return airoha_qdma_set_chan_tx_sched(port, channel, mode, w,
+ ARRAY_SIZE(w));
+}
+
+static int airoha_qdma_get_tx_ets_stats(struct airoha_gdm_port *port,
+ int channel,
+ struct tc_ets_qopt_offload *opt)
+{
+ u64 cpu_tx_packets = airoha_qdma_rr(port->qdma,
+ REG_CNTR_VAL(channel << 1));
+ u64 fwd_tx_packets = airoha_qdma_rr(port->qdma,
+ REG_CNTR_VAL((channel << 1) + 1));
+ u64 tx_packets = (cpu_tx_packets - port->cpu_tx_packets) +
+ (fwd_tx_packets - port->fwd_tx_packets);
+ _bstats_update(opt->stats.bstats, 0, tx_packets);
+
+ port->cpu_tx_packets = cpu_tx_packets;
+ port->fwd_tx_packets = fwd_tx_packets;
+
+ return 0;
+}
+
+static int airoha_tc_setup_qdisc_ets(struct airoha_gdm_port *port,
+ struct tc_ets_qopt_offload *opt)
+{
+ int channel;
+
+ if (opt->parent == TC_H_ROOT)
+ return -EINVAL;
+
+ channel = TC_H_MAJ(opt->handle) >> 16;
+ channel = channel % AIROHA_NUM_QOS_CHANNELS;
+
+ switch (opt->command) {
+ case TC_ETS_REPLACE:
+ return airoha_qdma_set_tx_ets_sched(port, channel, opt);
+ case TC_ETS_DESTROY:
+ /* PRIO is default qdisc scheduler */
+ return airoha_qdma_set_tx_prio_sched(port, channel);
+ case TC_ETS_STATS:
+ return airoha_qdma_get_tx_ets_stats(port, channel, opt);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int airoha_dev_tc_setup(struct net_device *dev, enum tc_setup_type type,
+ void *type_data)
+{
+ struct airoha_gdm_port *port = netdev_priv(dev);
+
+ switch (type) {
+ case TC_SETUP_QDISC_ETS:
+ return airoha_tc_setup_qdisc_ets(port, type_data);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
static const struct net_device_ops airoha_netdev_ops = {
.ndo_init = airoha_dev_init,
.ndo_open = airoha_dev_open,
.ndo_stop = airoha_dev_stop,
+ .ndo_select_queue = airoha_dev_select_queue,
.ndo_start_xmit = airoha_dev_xmit,
.ndo_get_stats64 = airoha_dev_get_stats64,
.ndo_set_mac_address = airoha_dev_set_macaddr,
+ .ndo_setup_tc = airoha_dev_tc_setup,
};
static const struct ethtool_ops airoha_ethtool_ops = {
@@ -2668,7 +2890,8 @@ static int airoha_alloc_gdm_port(struct airoha_eth *eth, struct device_node *np)
dev->watchdog_timeo = 5 * HZ;
dev->hw_features = NETIF_F_IP_CSUM | NETIF_F_RXCSUM |
NETIF_F_TSO6 | NETIF_F_IPV6_CSUM |
- NETIF_F_SG | NETIF_F_TSO;
+ NETIF_F_SG | NETIF_F_TSO |
+ NETIF_F_HW_TC;
dev->features |= dev->hw_features;
dev->dev.of_node = np;
dev->irq = qdma->irq;
diff --git a/drivers/net/ethernet/mediatek/mtk_ppe.c b/drivers/net/ethernet/mediatek/mtk_ppe.c
index fa688a42a22f..bb01a7355932 100644
--- a/drivers/net/ethernet/mediatek/mtk_ppe.c
+++ b/drivers/net/ethernet/mediatek/mtk_ppe.c
@@ -918,7 +918,7 @@ struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base, int index)
mib = dmam_alloc_coherent(ppe->dev, MTK_PPE_ENTRIES * sizeof(*mib),
&ppe->mib_phys, GFP_KERNEL);
if (!mib)
- return NULL;
+ goto err_free_l2_flows;
ppe->mib_table = mib;
@@ -926,7 +926,7 @@ struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base, int index)
GFP_KERNEL);
if (!acct)
- return NULL;
+ goto err_free_l2_flows;
ppe->acct_table = acct;
}
diff --git a/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c b/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c
index 781c691473e1..519c364e87d1 100644
--- a/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c
+++ b/drivers/net/ethernet/mediatek/mtk_wed_debugfs.c
@@ -310,9 +310,9 @@ wed_amsdu_show(struct seq_file *s, void *data)
WED_AMSDU_ENG_MAX_QGPP_CNT),
DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(1),
WED_AMSDU_ENG_CUR_ENTRY),
- DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(2),
+ DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(1),
WED_AMSDU_ENG_MAX_BUF_MERGED),
- DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(2),
+ DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(1),
WED_AMSDU_ENG_MAX_MSDU_MERGED),
DUMP_STR("WED AMDSU ENG2 INFO"),
@@ -414,7 +414,7 @@ wed_amsdu_show(struct seq_file *s, void *data)
WED_AMSDU_ENG_CUR_ENTRY),
DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(7),
WED_AMSDU_ENG_MAX_BUF_MERGED),
- DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(4),
+ DUMP_WED_MASK(WED_MON_AMSDU_ENG_CNT9(7),
WED_AMSDU_ENG_MAX_MSDU_MERGED),
DUMP_STR("WED AMDSU ENG8 INFO"),
diff --git a/drivers/net/ethernet/mediatek/mtk_wed_mcu.c b/drivers/net/ethernet/mediatek/mtk_wed_mcu.c
index c06e5ad18b01..76eff9e3f473 100644
--- a/drivers/net/ethernet/mediatek/mtk_wed_mcu.c
+++ b/drivers/net/ethernet/mediatek/mtk_wed_mcu.c
@@ -377,8 +377,12 @@ mtk_wed_mcu_load_firmware(struct mtk_wed_wo *wo)
/* wo firmware reset */
wo_w32(MTK_WO_MCU_CFG_LS_WF_MCCR_CLR_ADDR, 0xc00);
- val = wo_r32(MTK_WO_MCU_CFG_LS_WF_MCU_CFG_WM_WA_ADDR) |
- MTK_WO_MCU_CFG_LS_WF_WM_WA_WM_CPU_RSTB_MASK;
+ val = wo_r32(MTK_WO_MCU_CFG_LS_WF_MCU_CFG_WM_WA_ADDR);
+
+ if (!mtk_wed_is_v3_or_greater(wo->hw) && wo->hw->index)
+ val |= MTK_WO_MCU_CFG_LS_WF_WM_WA_WA_CPU_RSTB_MASK;
+ else
+ val |= MTK_WO_MCU_CFG_LS_WF_WM_WA_WM_CPU_RSTB_MASK;
wo_w32(MTK_WO_MCU_CFG_LS_WF_MCU_CFG_WM_WA_ADDR, val);
out:
release_firmware(fw);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index 3160a1a2d2ab..308b4458e0d4 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -2670,8 +2670,7 @@ static int mlx4_udp_tunnel_sync(struct net_device *dev, unsigned int table)
static const struct udp_tunnel_nic_info mlx4_udp_tunnels = {
.sync_table = mlx4_udp_tunnel_sync,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP |
- UDP_TUNNEL_NIC_INFO_IPV4_ONLY,
+ .flags = UDP_TUNNEL_NIC_INFO_IPV4_ONLY,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
},
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
index b4f3bd7d346e..1856d211339f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
@@ -54,6 +54,12 @@ static int mlx5e_hv_vhca_stats_buf_size(struct mlx5e_priv *priv)
priv->stats_nch);
}
+static int mlx5e_hv_vhca_stats_buf_max_size(struct mlx5e_priv *priv)
+{
+ return (sizeof(struct mlx5e_hv_vhca_per_ring_stats) *
+ max(priv->max_nch, priv->stats_nch));
+}
+
static void mlx5e_hv_vhca_stats_work(struct work_struct *work)
{
struct mlx5e_hv_vhca_stats_agent *sagent;
@@ -67,7 +73,7 @@ static void mlx5e_hv_vhca_stats_work(struct work_struct *work)
sagent = container_of(dwork, struct mlx5e_hv_vhca_stats_agent, work);
priv = container_of(sagent, struct mlx5e_priv, stats_agent);
buf_len = mlx5e_hv_vhca_stats_buf_size(priv);
- agent = sagent->agent;
+ agent = READ_ONCE(sagent->agent);
buf = sagent->buf;
memset(buf, 0, buf_len);
@@ -122,18 +128,21 @@ static void mlx5e_hv_vhca_stats_cleanup(struct mlx5_hv_vhca_agent *agent)
void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
{
- int buf_len = mlx5e_hv_vhca_stats_buf_size(priv);
+ int buf_len = mlx5e_hv_vhca_stats_buf_max_size(priv);
struct mlx5_hv_vhca_agent *agent;
priv->stats_agent.buf = kvzalloc(buf_len, GFP_KERNEL);
if (!priv->stats_agent.buf)
return;
+ INIT_DELAYED_WORK(&priv->stats_agent.work, mlx5e_hv_vhca_stats_work);
+
agent = mlx5_hv_vhca_agent_create(priv->mdev->hv_vhca,
MLX5_HV_VHCA_AGENT_STATS,
mlx5e_hv_vhca_stats_control, NULL,
mlx5e_hv_vhca_stats_cleanup,
- priv);
+ priv,
+ &priv->stats_agent.agent);
if (IS_ERR_OR_NULL(agent)) {
if (IS_ERR(agent))
@@ -142,18 +151,20 @@ void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
PTR_ERR(agent));
kvfree(priv->stats_agent.buf);
- return;
+ priv->stats_agent.buf = NULL;
}
-
- priv->stats_agent.agent = agent;
- INIT_DELAYED_WORK(&priv->stats_agent.work, mlx5e_hv_vhca_stats_work);
}
void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv)
{
- if (IS_ERR_OR_NULL(priv->stats_agent.agent))
+ struct mlx5_hv_vhca_agent *agent;
+
+ agent = READ_ONCE(priv->stats_agent.agent);
+ if (IS_ERR_OR_NULL(agent))
return;
- mlx5_hv_vhca_agent_destroy(priv->stats_agent.agent);
+ mlx5_hv_vhca_agent_destroy(agent);
+ WRITE_ONCE(priv->stats_agent.agent, NULL);
kvfree(priv->stats_agent.buf);
+ priv->stats_agent.buf = NULL;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
index 6ab02f3fc291..70d8cce7c173 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
@@ -713,34 +713,43 @@ static int mlx5e_macsec_add_rxsc(struct macsec_context *ctx)
}
sc_xarray_element->rx_sc = rx_sc;
- err = xa_alloc(&macsec->sc_xarray, &sc_xarray_element->fs_id, sc_xarray_element,
- XA_LIMIT(1, MLX5_MACEC_RX_FS_ID_MAX), GFP_KERNEL);
- if (err) {
- if (err == -EBUSY)
- netdev_err(ctx->netdev,
- "MACsec offload: unable to create entry for RX SC (%d Rx SCs already allocated)\n",
- MLX5_MACEC_RX_FS_ID_MAX);
- goto destroy_sc_xarray_elemenet;
- }
rx_sc->md_dst = metadata_dst_alloc(0, METADATA_MACSEC, GFP_KERNEL);
if (!rx_sc->md_dst) {
err = -ENOMEM;
- goto erase_xa_alloc;
+ goto destroy_sc_xarray_elemenet;
}
rx_sc->sci = ctx_rx_sc->sci;
rx_sc->active = ctx_rx_sc->active;
- list_add_rcu(&rx_sc->rx_sc_list_element, rx_sc_list);
-
rx_sc->sc_xarray_element = sc_xarray_element;
rx_sc->md_dst->u.macsec_info.sci = rx_sc->sci;
+
+ /*
+ * Publish the fully-initialised SC last: xa_alloc() makes
+ * sc_xarray_element->rx_sc (and rx_sc->md_dst) reachable from the RX
+ * datapath via xa_load(). Doing it only after md_dst is allocated and
+ * initialised pairs with the rcu_read_lock()/xa_load() in
+ * mlx5e_macsec_offload_handle_rx_skb(), so a reader can never observe
+ * a non-NULL md_dst with uninitialised contents.
+ */
+ err = xa_alloc(&macsec->sc_xarray, &sc_xarray_element->fs_id, sc_xarray_element,
+ XA_LIMIT(1, MLX5_MACEC_RX_FS_ID_MAX), GFP_KERNEL);
+ if (err) {
+ if (err == -EBUSY)
+ netdev_err(ctx->netdev,
+ "MACsec offload: unable to create entry for RX SC (%d Rx SCs already allocated)\n",
+ MLX5_MACEC_RX_FS_ID_MAX);
+ goto destroy_md_dst;
+ }
+
+ list_add_rcu(&rx_sc->rx_sc_list_element, rx_sc_list);
mutex_unlock(&macsec->lock);
return 0;
-erase_xa_alloc:
- xa_erase(&macsec->sc_xarray, sc_xarray_element->fs_id);
+destroy_md_dst:
+ dst_release(&rx_sc->md_dst->dst);
destroy_sc_xarray_elemenet:
kfree(sc_xarray_element);
destroy_rx_sc:
@@ -828,7 +837,7 @@ static void macsec_del_rxsc_ctx(struct mlx5e_macsec *macsec, struct mlx5e_macsec
*/
list_del_rcu(&rx_sc->rx_sc_list_element);
xa_erase(&macsec->sc_xarray, rx_sc->sc_xarray_element->fs_id);
- metadata_dst_free(rx_sc->md_dst);
+ dst_release(&rx_sc->md_dst->dst);
kfree(rx_sc->sc_xarray_element);
kfree_rcu_mightsleep(rx_sc);
}
@@ -1698,10 +1707,10 @@ void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev,
rcu_read_lock();
sc_xarray_element = xa_load(&macsec->sc_xarray, fs_id);
- rx_sc = sc_xarray_element->rx_sc;
- if (rx_sc) {
- dst_hold(&rx_sc->md_dst->dst);
- skb_dst_set(skb, &rx_sc->md_dst->dst);
+ rx_sc = sc_xarray_element ? sc_xarray_element->rx_sc : NULL;
+ if (rx_sc && rx_sc->md_dst) {
+ if (dst_hold_safe(&rx_sc->md_dst->dst))
+ skb_dst_set(skb, &rx_sc->md_dst->dst);
}
rcu_read_unlock();
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index dbfb900980de..17f149cdcb0f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -5258,8 +5258,7 @@ void mlx5e_vxlan_set_netdev_info(struct mlx5e_priv *priv)
priv->nic_info.set_port = mlx5e_vxlan_set_port;
priv->nic_info.unset_port = mlx5e_vxlan_unset_port;
- priv->nic_info.flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP |
- UDP_TUNNEL_NIC_INFO_STATIC_IANA_VXLAN;
+ priv->nic_info.flags = UDP_TUNNEL_NIC_INFO_STATIC_IANA_VXLAN;
priv->nic_info.tables[0].tunnel_types = UDP_TUNNEL_TYPE_VXLAN;
/* Don't count the space hard-coded to the IANA port */
priv->nic_info.tables[0].n_entries =
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
index 2381a0eec190..c433d29a62f2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
@@ -134,8 +134,10 @@ static void mlx5_mpesw_work(struct work_struct *work)
struct mlx5_lag *ldev = mpesww->lag;
devcom = mlx5_lag_get_devcom_comp(ldev);
- if (!devcom)
- return;
+ if (!devcom) {
+ mpesww->result = -ENODEV;
+ goto complete;
+ }
mlx5_devcom_comp_lock(devcom);
mutex_lock(&ldev->lock);
@@ -151,6 +153,7 @@ static void mlx5_mpesw_work(struct work_struct *work)
unlock:
mutex_unlock(&ldev->lock);
mlx5_devcom_comp_unlock(devcom);
+complete:
complete(&mpesww->comp);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.c
index 30564d9b00e9..779342ebc843 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.c
@@ -190,7 +190,7 @@ mlx5_hv_vhca_control_agent_create(struct mlx5_hv_vhca *hv_vhca)
return mlx5_hv_vhca_agent_create(hv_vhca, MLX5_HV_VHCA_AGENT_CONTROL,
NULL,
mlx5_hv_vhca_control_agent_invalidate,
- NULL, NULL);
+ NULL, NULL, NULL);
}
static void mlx5_hv_vhca_control_agent_destroy(struct mlx5_hv_vhca_agent *agent)
@@ -256,7 +256,8 @@ mlx5_hv_vhca_agent_create(struct mlx5_hv_vhca *hv_vhca,
void (*invalidate)(struct mlx5_hv_vhca_agent*,
u64 block_mask),
void (*cleaup)(struct mlx5_hv_vhca_agent *agent),
- void *priv)
+ void *priv,
+ struct mlx5_hv_vhca_agent **ctx_update)
{
struct mlx5_hv_vhca_agent *agent;
@@ -284,6 +285,9 @@ mlx5_hv_vhca_agent_create(struct mlx5_hv_vhca *hv_vhca,
agent->invalidate = invalidate;
agent->cleanup = cleaup;
+ if (ctx_update)
+ WRITE_ONCE(*ctx_update, agent);
+
mutex_lock(&hv_vhca->agents_lock);
hv_vhca->agents[type] = agent;
mutex_unlock(&hv_vhca->agents_lock);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.h
index f240ffe5116c..8b3974cf0ee4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/hv_vhca.h
@@ -43,7 +43,8 @@ mlx5_hv_vhca_agent_create(struct mlx5_hv_vhca *hv_vhca,
void (*invalidate)(struct mlx5_hv_vhca_agent*,
u64 block_mask),
void (*cleanup)(struct mlx5_hv_vhca_agent *agent),
- void *context);
+ void *context,
+ struct mlx5_hv_vhca_agent **ctx_update);
void mlx5_hv_vhca_agent_destroy(struct mlx5_hv_vhca_agent *agent);
int mlx5_hv_vhca_agent_write(struct mlx5_hv_vhca_agent *agent,
@@ -84,7 +85,8 @@ mlx5_hv_vhca_agent_create(struct mlx5_hv_vhca *hv_vhca,
void (*invalidate)(struct mlx5_hv_vhca_agent*,
u64 block_mask),
void (*cleanup)(struct mlx5_hv_vhca_agent *agent),
- void *context)
+ void *context,
+ struct mlx5_hv_vhca_agent **ctx_update)
{
return NULL;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/port_tun.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/port_tun.c
index 4571c56ec3c9..97f6097d4c70 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/port_tun.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/port_tun.c
@@ -176,7 +176,8 @@ void mlx5_tun_entropy_refcount_dec(struct mlx5_tun_entropy *tun_entropy,
int reformat_type)
{
mutex_lock(&tun_entropy->lock);
- if (reformat_type == MLX5_REFORMAT_TYPE_L2_TO_VXLAN)
+ if (reformat_type == MLX5_REFORMAT_TYPE_L2_TO_VXLAN ||
+ reformat_type == MLX5_REFORMAT_TYPE_L2_TO_L3_TUNNEL)
tun_entropy->num_enabling_entries--;
else if (reformat_type == MLX5_REFORMAT_TYPE_L2_TO_NVGRE &&
--tun_entropy->num_disabling_entries == 0)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 4d8295249c42..f77b6362fd9f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -576,7 +576,6 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
{
struct mlx5_profile *prof = &dev->profile;
void *set_hca_cap;
- int max_uc_list;
int err;
err = mlx5_core_get_caps(dev, MLX5_CAP_GENERAL);
@@ -659,10 +658,13 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
MLX5_SET(cmd_hca_cap, set_hca_cap, roce,
mlx5_is_roce_on(dev));
- max_uc_list = max_uc_list_get_devlink_param(dev);
- if (max_uc_list > 0)
- MLX5_SET(cmd_hca_cap, set_hca_cap, log_max_current_uc_list,
- ilog2(max_uc_list));
+ if (MLX5_CAP_GEN_MAX(dev, log_max_current_uc_list)) {
+ int max_uc_list = max_uc_list_get_devlink_param(dev);
+
+ if (max_uc_list > 0)
+ MLX5_SET(cmd_hca_cap, set_hca_cap,
+ log_max_current_uc_list, ilog2(max_uc_list));
+ }
return set_caps(dev, set_ctx, MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE);
}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
index 26401bb57572..81164f12cad3 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
@@ -4507,7 +4507,7 @@ static int mlxsw_sp_port_lag_join(struct mlxsw_sp_port *mlxsw_sp_port,
lag_id = lag->lag_id;
err = mlxsw_sp_port_lag_index_get(mlxsw_sp, lag_id, &port_index);
if (err)
- return err;
+ goto err_lag_uppers_bridge_join;
err = mlxsw_sp_lag_uppers_bridge_join(mlxsw_sp_port, lag_dev,
extack);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c
index 7066bc5612c6..06b35aa35bb4 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c
@@ -1019,6 +1019,7 @@ static int mlxsw_sp_vrs_lpm_tree_replace(struct mlxsw_sp *mlxsw_sp,
err_tree_replace:
for (i--; i >= 0; i--) {
+ vr = &mlxsw_sp->router->vrs[i];
if (!mlxsw_sp_vr_lpm_tree_should_replace(vr, proto, new_id))
continue;
mlxsw_sp_vr_lpm_tree_replace(mlxsw_sp,
diff --git a/drivers/net/ethernet/microchip/lan743x_main.c b/drivers/net/ethernet/microchip/lan743x_main.c
index dff5767671b1..d4622efc5b2b 100644
--- a/drivers/net/ethernet/microchip/lan743x_main.c
+++ b/drivers/net/ethernet/microchip/lan743x_main.c
@@ -3506,8 +3506,8 @@ static int lan743x_hardware_init(struct lan743x_adapter *adapter,
adapter->max_tx_channels = PCI11X1X_MAX_TX_CHANNELS;
adapter->used_tx_channels = PCI11X1X_USED_TX_CHANNELS;
adapter->max_vector_count = PCI11X1X_MAX_VECTOR_COUNT;
- pci11x1x_strap_get_status(adapter);
spin_lock_init(&adapter->eth_syslock_spinlock);
+ pci11x1x_strap_get_status(adapter);
mutex_init(&adapter->sgmii_rw_lock);
pci11x1x_set_rfe_rd_fifo_threshold(adapter);
sgmii_ctl = lan743x_csr_read(adapter, SGMII_CTL);
diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_vcap_impl.c b/drivers/net/ethernet/microchip/lan966x/lan966x_vcap_impl.c
index 2a37fc1ba4bc..dca60c818d4b 100644
--- a/drivers/net/ethernet/microchip/lan966x/lan966x_vcap_impl.c
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_vcap_impl.c
@@ -601,7 +601,6 @@ static void lan966x_vcap_admin_free(struct vcap_admin *admin)
kfree(admin->cache.keystream);
kfree(admin->cache.maskstream);
kfree(admin->cache.actionstream);
- mutex_destroy(&admin->lock);
kfree(admin);
}
@@ -615,7 +614,7 @@ lan966x_vcap_admin_alloc(struct lan966x *lan966x, struct vcap_control *ctrl,
if (!admin)
return ERR_PTR(-ENOMEM);
- mutex_init(&admin->lock);
+ admin->vctrl = ctrl;
INIT_LIST_HEAD(&admin->list);
INIT_LIST_HEAD(&admin->rules);
INIT_LIST_HEAD(&admin->enabled);
@@ -721,6 +720,7 @@ int lan966x_vcap_init(struct lan966x *lan966x)
ctrl->ops = &lan966x_vcap_ops;
INIT_LIST_HEAD(&ctrl->list);
+ mutex_init(&ctrl->lock);
for (int i = 0; i < ARRAY_SIZE(lan966x_vcap_inst_cfg); ++i) {
cfg = &lan966x_vcap_inst_cfg[i];
@@ -780,5 +780,6 @@ void lan966x_vcap_deinit(struct lan966x *lan966x)
lan966x_vcap_admin_free(admin);
}
+ mutex_destroy(&ctrl->lock);
kfree(ctrl);
}
diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c b/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c
index 0b4abc3eb53d..915efa62f7ee 100644
--- a/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c
+++ b/drivers/net/ethernet/microchip/sparx5/sparx5_switchdev.c
@@ -740,11 +740,13 @@ int sparx5_register_notifier_blocks(struct sparx5 *s5)
sparx5_owq = alloc_ordered_workqueue("sparx5_order", 0);
if (!sparx5_owq) {
err = -ENOMEM;
- goto err_switchdev_blocking_nb;
+ goto err_alloc_workqueue;
}
return 0;
+err_alloc_workqueue:
+ unregister_switchdev_blocking_notifier(&s5->switchdev_blocking_nb);
err_switchdev_blocking_nb:
unregister_switchdev_notifier(&s5->switchdev_nb);
err_switchdev_nb:
diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_vcap_impl.c b/drivers/net/ethernet/microchip/sparx5/sparx5_vcap_impl.c
index 967c8621c250..fd03b56432a7 100644
--- a/drivers/net/ethernet/microchip/sparx5/sparx5_vcap_impl.c
+++ b/drivers/net/ethernet/microchip/sparx5/sparx5_vcap_impl.c
@@ -1941,7 +1941,6 @@ static void sparx5_vcap_admin_free(struct vcap_admin *admin)
{
if (!admin)
return;
- mutex_destroy(&admin->lock);
kfree(admin->cache.keystream);
kfree(admin->cache.maskstream);
kfree(admin->cache.actionstream);
@@ -1961,7 +1960,7 @@ sparx5_vcap_admin_alloc(struct sparx5 *sparx5, struct vcap_control *ctrl,
INIT_LIST_HEAD(&admin->list);
INIT_LIST_HEAD(&admin->rules);
INIT_LIST_HEAD(&admin->enabled);
- mutex_init(&admin->lock);
+ admin->vctrl = ctrl;
admin->vtype = cfg->vtype;
admin->vinst = cfg->vinst;
admin->ingress = cfg->ingress;
@@ -2069,6 +2068,7 @@ int sparx5_vcap_init(struct sparx5 *sparx5)
ctrl->ops = &sparx5_vcap_ops;
INIT_LIST_HEAD(&ctrl->list);
+ mutex_init(&ctrl->lock);
for (idx = 0; idx < ARRAY_SIZE(sparx5_vcap_inst_cfg); ++idx) {
cfg = &sparx5_vcap_inst_cfg[idx];
admin = sparx5_vcap_admin_alloc(sparx5, ctrl, cfg);
@@ -2107,5 +2107,6 @@ void sparx5_vcap_destroy(struct sparx5 *sparx5)
list_del(&admin->list);
sparx5_vcap_admin_free(admin);
}
+ mutex_destroy(&ctrl->lock);
kfree(ctrl);
}
diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api.c b/drivers/net/ethernet/microchip/vcap/vcap_api.c
index 2687765abe52..49eef025d290 100644
--- a/drivers/net/ethernet/microchip/vcap/vcap_api.c
+++ b/drivers/net/ethernet/microchip/vcap/vcap_api.c
@@ -934,6 +934,16 @@ static bool vcap_rule_exists(struct vcap_control *vctrl, u32 id)
return false;
}
+void vcap_lock(struct vcap_admin *admin)
+{
+ mutex_lock(&admin->vctrl->lock);
+}
+
+void vcap_unlock(struct vcap_admin *admin)
+{
+ mutex_unlock(&admin->vctrl->lock);
+}
+
/* Find a rule with a provided rule id return a locked vcap */
static struct vcap_rule_internal *
vcap_get_locked_rule(struct vcap_control *vctrl, u32 id)
@@ -943,11 +953,11 @@ vcap_get_locked_rule(struct vcap_control *vctrl, u32 id)
/* Look for the rule id in all vcaps */
list_for_each_entry(admin, &vctrl->list, list) {
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
list_for_each_entry(ri, &admin->rules, list)
if (ri->data.id == id)
return ri;
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
}
return NULL;
}
@@ -961,14 +971,14 @@ int vcap_lookup_rule_by_cookie(struct vcap_control *vctrl, u64 cookie)
/* Look for the rule id in all vcaps */
list_for_each_entry(admin, &vctrl->list, list) {
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
list_for_each_entry(ri, &admin->rules, list) {
if (ri->data.cookie == cookie) {
id = ri->data.id;
break;
}
}
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
if (id)
return id;
}
@@ -985,11 +995,11 @@ int vcap_admin_rule_count(struct vcap_admin *admin, int cid)
int count = 0;
list_for_each_entry(elem, &admin->rules, list) {
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
if (elem->data.vcap_chain_id >= min_cid &&
elem->data.vcap_chain_id < max_cid)
++count;
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
}
return count;
}
@@ -2266,7 +2276,7 @@ int vcap_add_rule(struct vcap_rule *rule)
if (ret)
return ret;
/* Insert the new rule in the list of vcap rules */
- mutex_lock(&ri->admin->lock);
+ vcap_lock(ri->admin);
vcap_rule_set_state(ri);
ret = vcap_insert_rule(ri, &move);
@@ -2302,7 +2312,7 @@ int vcap_add_rule(struct vcap_rule *rule)
goto out;
}
out:
- mutex_unlock(&ri->admin->lock);
+ vcap_unlock(ri->admin);
return ret;
}
EXPORT_SYMBOL_GPL(vcap_add_rule);
@@ -2330,7 +2340,7 @@ struct vcap_rule *vcap_alloc_rule(struct vcap_control *vctrl,
if (vctrl->vcaps[admin->vtype].rows == 0)
return ERR_PTR(-EINVAL);
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
/* Check if a rule with this id already exists */
if (vcap_rule_exists(vctrl, id)) {
err = -EINVAL;
@@ -2369,13 +2379,13 @@ struct vcap_rule *vcap_alloc_rule(struct vcap_control *vctrl,
goto out_free;
}
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
return (struct vcap_rule *)ri;
out_free:
kfree(ri);
out_unlock:
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
return ERR_PTR(err);
}
@@ -2446,7 +2456,7 @@ struct vcap_rule *vcap_get_rule(struct vcap_control *vctrl, u32 id)
return ERR_PTR(-ENOENT);
rule = vcap_decode_rule(elem);
- mutex_unlock(&elem->admin->lock);
+ vcap_unlock(elem->admin);
return rule;
}
EXPORT_SYMBOL_GPL(vcap_get_rule);
@@ -2483,7 +2493,7 @@ int vcap_mod_rule(struct vcap_rule *rule)
err = vcap_write_counter(ri, &ctr);
out:
- mutex_unlock(&ri->admin->lock);
+ vcap_unlock(ri->admin);
return err;
}
EXPORT_SYMBOL_GPL(vcap_mod_rule);
@@ -2570,7 +2580,7 @@ int vcap_del_rule(struct vcap_control *vctrl, struct net_device *ndev, u32 id)
admin->last_used_addr = elem->addr;
}
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
return err;
}
EXPORT_SYMBOL_GPL(vcap_del_rule);
@@ -2585,7 +2595,7 @@ int vcap_del_rules(struct vcap_control *vctrl, struct vcap_admin *admin)
if (ret)
return ret;
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
list_for_each_entry_safe(ri, next_ri, &admin->rules, list) {
vctrl->ops->init(ri->ndev, admin, ri->addr, ri->size);
list_del(&ri->list);
@@ -2598,7 +2608,7 @@ int vcap_del_rules(struct vcap_control *vctrl, struct vcap_admin *admin)
list_del(&eport->list);
kfree(eport);
}
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
return 0;
}
@@ -3016,7 +3026,7 @@ static int vcap_enable_rules(struct vcap_control *vctrl,
continue;
/* Found the admin, now find the offloadable rules */
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
list_for_each_entry(ri, &admin->rules, list) {
/* Is the rule in the lookup defined by the chain */
if (!(ri->data.vcap_chain_id >= chain &&
@@ -3034,7 +3044,7 @@ static int vcap_enable_rules(struct vcap_control *vctrl,
if (err)
break;
}
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
if (err)
break;
}
@@ -3074,7 +3084,7 @@ static int vcap_disable_rules(struct vcap_control *vctrl,
continue;
/* Found the admin, now find the rules on the chain */
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
list_for_each_entry(ri, &admin->rules, list) {
if (ri->data.vcap_chain_id != chain)
continue;
@@ -3089,7 +3099,7 @@ static int vcap_disable_rules(struct vcap_control *vctrl,
if (err)
break;
}
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
if (err)
break;
}
@@ -3133,9 +3143,9 @@ static int vcap_enable(struct vcap_control *vctrl, struct net_device *ndev,
eport->cookie = cookie;
eport->src_cid = src_cid;
eport->dst_cid = dst_cid;
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
list_add_tail(&eport->list, &admin->enabled);
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
if (vcap_path_exist(vctrl, ndev, src_cid)) {
/* Enable chained lookups */
@@ -3185,9 +3195,9 @@ static int vcap_disable(struct vcap_control *vctrl, struct net_device *ndev,
dst_cid = vcap_get_next_chain(vctrl, ndev, dst_cid);
}
- mutex_lock(&found->lock);
+ vcap_lock(found);
list_del(&eport->list);
- mutex_unlock(&found->lock);
+ vcap_unlock(found);
kfree(eport);
return 0;
}
@@ -3270,9 +3280,9 @@ int vcap_rule_set_counter(struct vcap_rule *rule, struct vcap_counter *ctr)
return -EINVAL;
}
- mutex_lock(&ri->admin->lock);
+ vcap_lock(ri->admin);
err = vcap_write_counter(ri, ctr);
- mutex_unlock(&ri->admin->lock);
+ vcap_unlock(ri->admin);
return err;
}
@@ -3291,9 +3301,9 @@ int vcap_rule_get_counter(struct vcap_rule *rule, struct vcap_counter *ctr)
return -EINVAL;
}
- mutex_lock(&ri->admin->lock);
+ vcap_lock(ri->admin);
err = vcap_read_counter(ri, ctr);
- mutex_unlock(&ri->admin->lock);
+ vcap_unlock(ri->admin);
return err;
}
@@ -3395,7 +3405,7 @@ int vcap_get_rule_count_by_cookie(struct vcap_control *vctrl,
/* Iterate all rules in each VCAP instance */
list_for_each_entry(admin, &vctrl->list, list) {
- mutex_lock(&admin->lock);
+ vcap_lock(admin);
list_for_each_entry(ri, &admin->rules, list) {
if (ri->data.cookie != cookie)
continue;
@@ -3412,12 +3422,12 @@ int vcap_get_rule_count_by_cookie(struct vcap_control *vctrl,
if (err)
goto unlock;
}
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
}
return err;
unlock:
- mutex_unlock(&admin->lock);
+ vcap_unlock(admin);
return err;
}
EXPORT_SYMBOL_GPL(vcap_get_rule_count_by_cookie);
diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api.h b/drivers/net/ethernet/microchip/vcap/vcap_api.h
index 6069ad95c27e..05b4b02e59ef 100644
--- a/drivers/net/ethernet/microchip/vcap/vcap_api.h
+++ b/drivers/net/ethernet/microchip/vcap/vcap_api.h
@@ -164,7 +164,7 @@ struct vcap_admin {
struct list_head list; /* for insertion in vcap_control */
struct list_head rules; /* list of rules */
struct list_head enabled; /* list of enabled ports */
- struct mutex lock; /* control access to rules */
+ struct vcap_control *vctrl; /* the control instance owning this vcap */
enum vcap_type vtype; /* type of vcap */
int vinst; /* instance number within the same type */
int first_cid; /* first chain id in this vcap */
@@ -275,6 +275,7 @@ struct vcap_control {
const struct vcap_info *vcaps; /* client supplied vcap models */
const struct vcap_statistics *stats; /* client supplied vcap stats */
struct list_head list; /* list of vcap instances */
+ struct mutex lock; /* serialize access to all vcap instances */
};
#endif /* __VCAP_API__ */
diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs.c b/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs.c
index 59bfbda29bb3..e0c65c7ab23e 100644
--- a/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs.c
+++ b/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs.c
@@ -410,9 +410,9 @@ static int vcap_debugfs_show(struct seq_file *m, void *unused)
};
int ret;
- mutex_lock(&info->admin->lock);
+ vcap_lock(info->admin);
ret = vcap_show_admin(info->vctrl, info->admin, &out);
- mutex_unlock(&info->admin->lock);
+ vcap_unlock(info->admin);
return ret;
}
DEFINE_SHOW_ATTRIBUTE(vcap_debugfs);
@@ -427,9 +427,9 @@ static int vcap_raw_debugfs_show(struct seq_file *m, void *unused)
};
int ret;
- mutex_lock(&info->admin->lock);
+ vcap_lock(info->admin);
ret = vcap_show_admin_raw(info->vctrl, info->admin, &out);
- mutex_unlock(&info->admin->lock);
+ vcap_unlock(info->admin);
return ret;
}
DEFINE_SHOW_ATTRIBUTE(vcap_raw_debugfs);
diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs_kunit.c b/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs_kunit.c
index 9c9d38042125..ac2a3b8c4f32 100644
--- a/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs_kunit.c
+++ b/drivers/net/ethernet/microchip/vcap/vcap_api_debugfs_kunit.c
@@ -243,10 +243,11 @@ static void vcap_test_api_init(struct vcap_admin *admin)
{
/* Initialize the shared objects */
INIT_LIST_HEAD(&test_vctrl.list);
+ mutex_init(&test_vctrl.lock);
INIT_LIST_HEAD(&admin->list);
INIT_LIST_HEAD(&admin->rules);
INIT_LIST_HEAD(&admin->enabled);
- mutex_init(&admin->lock);
+ admin->vctrl = &test_vctrl;
list_add_tail(&admin->list, &test_vctrl.list);
memset(test_updateaddr, 0, sizeof(test_updateaddr));
test_updateaddridx = 0;
diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c b/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c
index 16eb3de60eb6..70683855cbfc 100644
--- a/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c
+++ b/drivers/net/ethernet/microchip/vcap/vcap_api_kunit.c
@@ -233,10 +233,11 @@ static void vcap_test_api_init(struct vcap_admin *admin)
{
/* Initialize the shared objects */
INIT_LIST_HEAD(&test_vctrl.list);
+ mutex_init(&test_vctrl.lock);
INIT_LIST_HEAD(&admin->list);
INIT_LIST_HEAD(&admin->rules);
INIT_LIST_HEAD(&admin->enabled);
- mutex_init(&admin->lock);
+ admin->vctrl = &test_vctrl;
list_add_tail(&admin->list, &test_vctrl.list);
memset(test_updateaddr, 0, sizeof(test_updateaddr));
test_updateaddridx = 0;
diff --git a/drivers/net/ethernet/microchip/vcap/vcap_api_private.h b/drivers/net/ethernet/microchip/vcap/vcap_api_private.h
index 844bdf6b5f45..b4057fbe3d18 100644
--- a/drivers/net/ethernet/microchip/vcap/vcap_api_private.h
+++ b/drivers/net/ethernet/microchip/vcap/vcap_api_private.h
@@ -50,6 +50,9 @@ struct vcap_stream_iter {
/* Check that the control has a valid set of callbacks */
int vcap_api_check(struct vcap_control *ctrl);
+/* Serialize access to the vcap instances of a control */
+void vcap_lock(struct vcap_admin *admin);
+void vcap_unlock(struct vcap_admin *admin);
/* Erase the VCAP cache area used or encoding and decoding */
void vcap_erase_cache(struct vcap_rule_internal *ri);
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
index 876de6db63c4..ab5c910e43fb 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
@@ -845,6 +845,8 @@ int mana_gd_create_mana_wq_cq(struct gdma_dev *gd,
if (!queue)
return -ENOMEM;
+ queue->id = INVALID_QUEUE_ID;
+
gmi = &queue->mem_info;
err = mana_gd_alloc_memory(gc, spec->queue_size, gmi);
if (err)
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 0e4b0ac4acf8..95c4a254953a 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1865,7 +1865,8 @@ static void mana_destroy_txq(struct mana_port_context *apc)
netif_napi_del(napi);
apc->tx_qp[i].txq.napi_initialized = false;
}
- mana_destroy_wq_obj(apc, GDMA_SQ, apc->tx_qp[i].tx_object);
+ if (apc->tx_qp[i].tx_object != INVALID_MANA_HANDLE)
+ mana_destroy_wq_obj(apc, GDMA_SQ, apc->tx_qp[i].tx_object);
mana_deinit_cq(apc, &apc->tx_qp[i].tx_cq);
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
index 6e0929af0f72..5b7624332c45 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
@@ -2394,8 +2394,7 @@ static int nfp_udp_tunnel_sync(struct net_device *netdev, unsigned int table)
static const struct udp_tunnel_nic_info nfp_udp_tunnels = {
.sync_table = nfp_udp_tunnel_sync,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP |
- UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
+ .flags = UDP_TUNNEL_NIC_INFO_OPEN_ONLY,
.tables = {
{
.n_entries = NFP_NET_N_VXLAN_PORTS,
diff --git a/drivers/net/ethernet/oa_tc6.c b/drivers/net/ethernet/oa_tc6.c
index 91a906a7918a..cfe1f6e3cd12 100644
--- a/drivers/net/ethernet/oa_tc6.c
+++ b/drivers/net/ethernet/oa_tc6.c
@@ -752,6 +752,17 @@ static int oa_tc6_process_rx_chunk_footer(struct oa_tc6 *tc6, u32 footer)
static void oa_tc6_submit_rx_skb(struct oa_tc6 *tc6)
{
+ /* MAC-PHY delivers each frame with its Ethernet FCS attached.
+ * Strip it before handing over to the stack, unless the user
+ * has asked to keep it via NETIF_F_RXFCS. Keeping the FCS
+ * in the frame is harmless for IP traffic, but is parsed as
+ * a (malformed) suffix TLV by PTP, which makes ptp4l reject
+ * every message with "bad message" error.
+ */
+ if (!(tc6->netdev->features & NETIF_F_RXFCS) &&
+ tc6->rx_skb->len > ETH_FCS_LEN)
+ skb_trim(tc6->rx_skb, tc6->rx_skb->len - ETH_FCS_LEN);
+
tc6->rx_skb->protocol = eth_type_trans(tc6->rx_skb, tc6->netdev);
tc6->netdev->stats.rx_packets++;
tc6->netdev->stats.rx_bytes += tc6->rx_skb->len;
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
index a632536bd7f2..8ce45aaa74a7 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
@@ -116,8 +116,15 @@ static void ionic_get_link_ext_stats(struct net_device *netdev,
{
struct ionic_lif *lif = netdev_priv(netdev);
- if (lif->ionic->pdev->is_physfn)
- stats->link_down_events = lif->link_down_count;
+ if (lif->ionic->pdev->is_virtfn)
+ return;
+
+ if (!lif->ionic->idev.port_info) {
+ netdev_err_once(netdev, "port_info not initialized\n");
+ return;
+ }
+
+ stats->link_down_events = lif->link_down_count;
}
static int ionic_get_link_ksettings(struct net_device *netdev,
diff --git a/drivers/net/ethernet/qlogic/qede/qede_filter.c b/drivers/net/ethernet/qlogic/qede/qede_filter.c
index 985026dd816f..7e341e026489 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_filter.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_filter.c
@@ -987,20 +987,17 @@ static int qede_udp_tunnel_sync(struct net_device *dev, unsigned int table)
static const struct udp_tunnel_nic_info qede_udp_tunnels_both = {
.sync_table = qede_udp_tunnel_sync,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_GENEVE, },
},
}, qede_udp_tunnels_vxlan = {
.sync_table = qede_udp_tunnel_sync,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
},
}, qede_udp_tunnels_geneve = {
.sync_table = qede_udp_tunnel_sync,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_GENEVE, },
},
diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c
index e338bfc8b7b2..c11e0d8f98aa 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_fp.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c
@@ -765,6 +765,9 @@ qede_tpa_rx_build_skb(struct qede_dev *edev,
struct sk_buff *skb;
skb = qede_build_skb(rxq, bd, len, pad);
+ if (unlikely(!skb))
+ return NULL;
+
bd->page_offset += rxq->rx_buf_seg_size;
if (bd->page_offset == PAGE_SIZE) {
@@ -812,6 +815,8 @@ qede_rx_build_skb(struct qede_dev *edev,
}
skb = qede_build_skb(rxq, bd, len, pad);
+ if (unlikely(!skb))
+ return NULL;
if (unlikely(qede_realloc_rx_buffer(rxq, bd))) {
/* Incr page ref count to reuse on allocation failure so
@@ -961,7 +966,7 @@ static inline void qede_tpa_cont(struct qede_dev *edev,
{
int i;
- for (i = 0; cqe->len_list[i] && i < ARRAY_SIZE(cqe->len_list); i++)
+ for (i = 0; i < ARRAY_SIZE(cqe->len_list) && cqe->len_list[i]; i++)
qede_fill_frag_skb(edev, rxq, cqe->tpa_agg_index,
le16_to_cpu(cqe->len_list[i]));
@@ -986,7 +991,7 @@ static int qede_tpa_end(struct qede_dev *edev,
dma_unmap_page(rxq->dev, tpa_info->buffer.mapping,
PAGE_SIZE, rxq->data_direction);
- for (i = 0; cqe->len_list[i] && i < ARRAY_SIZE(cqe->len_list); i++)
+ for (i = 0; i < ARRAY_SIZE(cqe->len_list) && cqe->len_list[i]; i++)
qede_fill_frag_skb(edev, rxq, cqe->tpa_agg_index,
le16_to_cpu(cqe->len_list[i]));
if (unlikely(i > 1))
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
index 5e170400c61b..1187bfb5b880 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
@@ -486,7 +486,6 @@ static int qlcnic_udp_tunnel_sync(struct net_device *dev, unsigned int table)
static const struct udp_tunnel_nic_info qlcnic_udp_tunnels = {
.sync_table = qlcnic_udp_tunnel_sync,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP,
.tables = {
{ .n_entries = 1, .tunnel_types = UDP_TUNNEL_TYPE_VXLAN, },
},
diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_handlers.c b/drivers/net/ethernet/qualcomm/rmnet/rmnet_handlers.c
index 9f3479500f85..d055a2628d8c 100644
--- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_handlers.c
+++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_handlers.c
@@ -126,7 +126,10 @@ rmnet_map_ingress_handler(struct sk_buff *skb,
consume_skb(skb);
} else {
- __rmnet_map_ingress_handler(skb, port);
+ if (rmnet_map_validate_packet_len(skb, port))
+ __rmnet_map_ingress_handler(skb, port);
+ else
+ kfree_skb(skb);
}
}
diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h
index b70284095568..60ca8b780c88 100644
--- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h
+++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h
@@ -59,5 +59,6 @@ void rmnet_map_tx_aggregate_init(struct rmnet_port *port);
void rmnet_map_tx_aggregate_exit(struct rmnet_port *port);
void rmnet_map_update_ul_agg_config(struct rmnet_port *port, u32 size,
u32 count, u32 time);
+u32 rmnet_map_validate_packet_len(struct sk_buff *skb, struct rmnet_port *port);
#endif /* _RMNET_MAP_H_ */
diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c
index a5e3d1a88305..8471157a431b 100644
--- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c
+++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c
@@ -333,54 +333,62 @@ struct rmnet_map_header *rmnet_map_add_map_header(struct sk_buff *skb,
return map_header;
}
-/* Deaggregates a single packet
- * A whole new buffer is allocated for each portion of an aggregated frame.
- * Caller should keep calling deaggregate() on the source skb until 0 is
- * returned, indicating that there are no more packets to deaggregate. Caller
- * is responsible for freeing the original skb.
- */
-struct sk_buff *rmnet_map_deaggregate(struct sk_buff *skb,
- struct rmnet_port *port)
+u32 rmnet_map_validate_packet_len(struct sk_buff *skb, struct rmnet_port *port)
{
struct rmnet_map_v5_csum_header *next_hdr = NULL;
struct rmnet_map_header *maph;
void *data = skb->data;
- struct sk_buff *skbn;
- u8 nexthdr_type;
u32 packet_len;
- if (skb->len == 0)
- return NULL;
+ if (skb->len < sizeof(*maph))
+ return 0;
maph = (struct rmnet_map_header *)skb->data;
+
+ /* Some hardware can send us empty frames. Catch them */
+ if (!maph->pkt_len)
+ return 0;
+
packet_len = ntohs(maph->pkt_len) + sizeof(*maph);
if (port->data_format & RMNET_FLAGS_INGRESS_MAP_CKSUMV4) {
packet_len += sizeof(struct rmnet_map_dl_csum_trailer);
- } else if (port->data_format & RMNET_FLAGS_INGRESS_MAP_CKSUMV5) {
- if (!(maph->flags & MAP_CMD_FLAG)) {
- packet_len += sizeof(*next_hdr);
- if (maph->flags & MAP_NEXT_HEADER_FLAG)
- next_hdr = data + sizeof(*maph);
- else
- /* Mapv5 data pkt without csum hdr is invalid */
- return NULL;
- }
+ } else if ((port->data_format & RMNET_FLAGS_INGRESS_MAP_CKSUMV5) &&
+ !(maph->flags & MAP_CMD_FLAG)) {
+ /* Mapv5 data pkt without csum hdr is invalid */
+ if (!(maph->flags & MAP_NEXT_HEADER_FLAG))
+ return 0;
+
+ packet_len += sizeof(*next_hdr);
+ next_hdr = data + sizeof(*maph);
}
- if (((int)skb->len - (int)packet_len) < 0)
- return NULL;
+ if (skb->len < packet_len)
+ return 0;
- /* Some hardware can send us empty frames. Catch them */
- if (!maph->pkt_len)
- return NULL;
+ if (next_hdr &&
+ u8_get_bits(next_hdr->header_info, MAPV5_HDRINFO_HDR_TYPE_FMASK) !=
+ RMNET_MAP_HEADER_TYPE_CSUM_OFFLOAD)
+ return 0;
- if (next_hdr) {
- nexthdr_type = u8_get_bits(next_hdr->header_info,
- MAPV5_HDRINFO_HDR_TYPE_FMASK);
- if (nexthdr_type != RMNET_MAP_HEADER_TYPE_CSUM_OFFLOAD)
- return NULL;
- }
+ return packet_len;
+}
+
+/* Deaggregates a single packet
+ * A whole new buffer is allocated for each portion of an aggregated frame.
+ * Caller should keep calling deaggregate() on the source skb until 0 is
+ * returned, indicating that there are no more packets to deaggregate. Caller
+ * is responsible for freeing the original skb.
+ */
+struct sk_buff *rmnet_map_deaggregate(struct sk_buff *skb,
+ struct rmnet_port *port)
+{
+ struct sk_buff *skbn;
+ u32 packet_len;
+
+ packet_len = rmnet_map_validate_packet_len(skb, port);
+ if (!packet_len)
+ return NULL;
skbn = alloc_skb(packet_len + RMNET_MAP_DEAGGR_SPACING, GFP_ATOMIC);
if (!skbn)
diff --git a/drivers/net/ethernet/sfc/ef10.c b/drivers/net/ethernet/sfc/ef10.c
index de131fc5fa0b..9b83c9719d18 100644
--- a/drivers/net/ethernet/sfc/ef10.c
+++ b/drivers/net/ethernet/sfc/ef10.c
@@ -3982,7 +3982,6 @@ static int efx_ef10_udp_tnl_unset_port(struct net_device *dev,
static const struct udp_tunnel_nic_info efx_ef10_udp_tunnels = {
.set_port = efx_ef10_udp_tnl_set_port,
.unset_port = efx_ef10_udp_tnl_unset_port,
- .flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP,
.tables = {
{
.n_entries = 16,
diff --git a/drivers/net/ethernet/sun/sungem.c b/drivers/net/ethernet/sun/sungem.c
index 3e5f9b17c777..e491c2d045ef 100644
--- a/drivers/net/ethernet/sun/sungem.c
+++ b/drivers/net/ethernet/sun/sungem.c
@@ -2986,10 +2986,10 @@ static int gem_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
dev->max_mtu = GEM_MAX_MTU;
/* Register with kernel */
- if (register_netdev(dev)) {
+ err = register_netdev(dev);
+ if (err) {
pr_err("Cannot register net device, aborting\n");
- err = -ENOMEM;
- goto err_out_free_consistent;
+ goto err_out_clear_drvdata;
}
/* Undo the get_cell with appropriate locking (we could use
@@ -3003,8 +3003,13 @@ static int gem_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
dev->dev_addr);
return 0;
+err_out_clear_drvdata:
+ pci_set_drvdata(pdev, NULL);
+ netif_napi_del(&gp->napi);
+
err_out_free_consistent:
- gem_remove_one(pdev);
+ dma_free_coherent(&pdev->dev, sizeof(struct gem_init_block),
+ gp->init_block, gp->gblock_dvma);
err_out_iounmap:
gem_put_cell(gp);
iounmap(gp->regs);
diff --git a/drivers/net/ethernet/sunplus/spl2sw_phy.c b/drivers/net/ethernet/sunplus/spl2sw_phy.c
index 6f899e48f51d..a4889c52e00e 100644
--- a/drivers/net/ethernet/sunplus/spl2sw_phy.c
+++ b/drivers/net/ethernet/sunplus/spl2sw_phy.c
@@ -79,12 +79,14 @@ int spl2sw_phy_connect(struct spl2sw_common *comm)
void spl2sw_phy_remove(struct spl2sw_common *comm)
{
struct net_device *ndev;
+ struct spl2sw_mac *mac;
int i;
for (i = 0; i < MAX_NETDEV_NUM; i++)
if (comm->ndev[i]) {
ndev = comm->ndev[i];
- if (ndev)
- phy_disconnect(ndev->phydev);
+ mac = netdev_priv(ndev);
+ phy_disconnect(ndev->phydev);
+ of_node_put(mac->phy_node);
}
}
diff --git a/drivers/net/ieee802154/ca8210.c b/drivers/net/ieee802154/ca8210.c
index a036910f6082..bb961d686fb4 100644
--- a/drivers/net/ieee802154/ca8210.c
+++ b/drivers/net/ieee802154/ca8210.c
@@ -597,7 +597,7 @@ static int ca8210_test_int_driver_write(
fifo_buffer = kmemdup(buf, len, GFP_KERNEL);
if (!fifo_buffer)
return -ENOMEM;
- kfifo_in(&test->up_fifo, &fifo_buffer, 4);
+ kfifo_in(&test->up_fifo, &fifo_buffer, sizeof(fifo_buffer));
wake_up_interruptible(&priv->test.readq);
return 0;
@@ -921,9 +921,10 @@ static int ca8210_spi_transfer(
if (status < 0) {
dev_crit(
&spi->dev,
- "status %d from spi_sync in write\n",
+ "status %d from spi_async in write\n",
status
);
+ kfree(cas_ctl);
}
return status;
@@ -2527,6 +2528,7 @@ static ssize_t ca8210_test_int_user_read(
struct ca8210_priv *priv = filp->private_data;
unsigned char *fifo_buffer;
unsigned long bytes_not_copied;
+ unsigned int copied;
if (filp->f_flags & O_NONBLOCK) {
/* Non-blocking mode */
@@ -2540,7 +2542,8 @@ static ssize_t ca8210_test_int_user_read(
);
}
- if (kfifo_out(&priv->test.up_fifo, &fifo_buffer, 4) != 4) {
+ copied = kfifo_out(&priv->test.up_fifo, &fifo_buffer, sizeof(fifo_buffer));
+ if (copied != sizeof(fifo_buffer)) {
dev_err(
&priv->spi->dev,
"test_interface: Wrong number of elements popped from upstream fifo\n"
diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c
index 95c655341a61..66f7407a2b17 100644
--- a/drivers/net/macsec.c
+++ b/drivers/net/macsec.c
@@ -641,7 +641,7 @@ static struct sk_buff *macsec_encrypt(struct sk_buff *skb,
}
unprotected_len = skb->len;
- eth = eth_hdr(skb);
+ eth = skb_eth_hdr(skb);
sci_present = macsec_send_sci(secy);
hh = skb_push(skb, macsec_extra_len(sci_present));
memmove(hh, eth, 2 * ETH_ALEN);
@@ -3627,19 +3627,22 @@ static int macsec_dev_open(struct net_device *dev)
ops = macsec_get_ops(netdev_priv(dev), &ctx);
if (!ops) {
err = -EOPNOTSUPP;
- goto clear_allmulti;
+ goto clear_promisc;
}
ctx.secy = &macsec->secy;
err = macsec_offload(ops->mdo_dev_open, &ctx);
if (err)
- goto clear_allmulti;
+ goto clear_promisc;
}
if (netif_carrier_ok(real_dev))
netif_carrier_on(dev);
return 0;
+clear_promisc:
+ if (dev->flags & IFF_PROMISC)
+ dev_set_promiscuity(real_dev, -1);
clear_allmulti:
if (dev->flags & IFF_ALLMULTI)
dev_set_allmulti(real_dev, -1);
diff --git a/drivers/net/netdevsim/udp_tunnels.c b/drivers/net/netdevsim/udp_tunnels.c
index 640b4983a9a0..10cbbf1c584b 100644
--- a/drivers/net/netdevsim/udp_tunnels.c
+++ b/drivers/net/netdevsim/udp_tunnels.c
@@ -112,12 +112,10 @@ nsim_udp_tunnels_info_reset_write(struct file *file, const char __user *data,
struct net_device *dev = file->private_data;
struct netdevsim *ns = netdev_priv(dev);
- rtnl_lock();
if (dev->reg_state == NETREG_REGISTERED) {
memset(ns->udp_ports.ports, 0, sizeof(ns->udp_ports.__ports));
udp_tunnel_nic_reset_ntf(dev);
}
- rtnl_unlock();
return count;
}
@@ -181,8 +179,6 @@ int nsim_udp_tunnels_info_create(struct nsim_dev *nsim_dev,
info->sync_table = NULL;
}
- if (ns->udp_ports.sleep)
- info->flags |= UDP_TUNNEL_NIC_INFO_MAY_SLEEP;
if (nsim_dev->udp_ports.open_only)
info->flags |= UDP_TUNNEL_NIC_INFO_OPEN_ONLY;
if (nsim_dev->udp_ports.ipv4_only)
diff --git a/drivers/net/pfcp.c b/drivers/net/pfcp.c
index 68d0d9e92a22..b07517d4fe2d 100644
--- a/drivers/net/pfcp.c
+++ b/drivers/net/pfcp.c
@@ -148,6 +148,7 @@ static void pfcp_link_setup(struct net_device *dev)
dev->flags = IFF_POINTOPOINT | IFF_NOARP | IFF_MULTICAST;
dev->priv_flags |= IFF_NO_QUEUE;
+ dev->pcpu_stat_type = NETDEV_PCPU_STAT_TSTATS;
netif_keep_dst(dev);
}
diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index 90ffba8f7952..6eefe98316f5 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -787,6 +787,7 @@ static int sfp_i2c_mdiobus_create(struct sfp *sfp)
static void sfp_i2c_mdiobus_destroy(struct sfp *sfp)
{
mdiobus_unregister(sfp->i2c_mii);
+ mdiobus_free(sfp->i2c_mii);
sfp->i2c_mii = NULL;
}
diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
index dcaa62377808..5d7d11d75b0b 100644
--- a/drivers/net/thunderbolt/main.c
+++ b/drivers/net/thunderbolt/main.c
@@ -786,8 +786,12 @@ static bool tbnet_check_frame(struct tbnet *net, const struct tbnet_frame *tf,
return true;
}
- /* Start of packet, validate the frame header */
- if (frame_count == 0 || frame_count > TBNET_RING_SIZE / 4) {
+ /* Start of packet, validate the frame header. tbnet_poll() puts the
+ * first frame in the skb linear area and every further frame in a page
+ * fragment, so a packet may not span more than MAX_SKB_FRAGS + 1 frames
+ * without overflowing skb_shinfo()->frags[].
+ */
+ if (frame_count == 0 || frame_count > MAX_SKB_FRAGS + 1) {
net->stats.rx_length_errors++;
return false;
}
diff --git a/drivers/net/usb/gl620a.c b/drivers/net/usb/gl620a.c
index 0bfa37c14059..09afd137b64e 100644
--- a/drivers/net/usb/gl620a.c
+++ b/drivers/net/usb/gl620a.c
@@ -104,6 +104,9 @@ static int genelink_rx_fixup(struct usbnet *dev, struct sk_buff *skb)
return 0;
}
+ if (!skb_pull(skb, size + 4))
+ return 0;
+
// allocate the skb for the individual packet
gl_skb = alloc_skb(size, GFP_ATOMIC);
if (gl_skb) {
@@ -116,9 +119,6 @@ static int genelink_rx_fixup(struct usbnet *dev, struct sk_buff *skb)
// advance to the next packet
packet = (struct gl_packet *)&packet->packet_data[size];
count--;
-
- // shift the data pointer to the next gl_packet
- skb_pull(skb, size + 4);
}
// skip the packet length field 4 bytes
diff --git a/drivers/net/usb/lan78xx.c b/drivers/net/usb/lan78xx.c
index 8647980df005..f6373d93e042 100644
--- a/drivers/net/usb/lan78xx.c
+++ b/drivers/net/usb/lan78xx.c
@@ -812,6 +812,156 @@ static void lan78xx_update_stats(struct lan78xx_net *dev)
usb_autopm_put_interface(dev->intf);
}
+static int lan78xx_start_hw(struct lan78xx_net *dev, u32 reg, u32 hw_enable)
+{
+ return lan78xx_update_reg(dev, reg, hw_enable, hw_enable);
+}
+
+static int lan78xx_stop_hw(struct lan78xx_net *dev, u32 reg, u32 hw_enabled,
+ u32 hw_disabled)
+{
+ unsigned long timeout;
+ bool stopped = true;
+ int ret;
+ u32 buf;
+
+ /* Stop the h/w block (if not already stopped) */
+
+ ret = lan78xx_read_reg(dev, reg, &buf);
+ if (ret < 0)
+ return ret;
+
+ if (buf & hw_enabled) {
+ buf &= ~hw_enabled;
+
+ ret = lan78xx_write_reg(dev, reg, buf);
+ if (ret < 0)
+ return ret;
+
+ stopped = false;
+ timeout = jiffies + HW_DISABLE_TIMEOUT;
+ do {
+ ret = lan78xx_read_reg(dev, reg, &buf);
+ if (ret < 0)
+ return ret;
+
+ if (buf & hw_disabled)
+ stopped = true;
+ else
+ msleep(HW_DISABLE_DELAY_MS);
+ } while (!stopped && !time_after(jiffies, timeout));
+ }
+
+ ret = stopped ? 0 : -ETIME;
+
+ return ret;
+}
+
+static int lan78xx_flush_fifo(struct lan78xx_net *dev, u32 reg, u32 fifo_flush)
+{
+ return lan78xx_update_reg(dev, reg, fifo_flush, fifo_flush);
+}
+
+static int lan78xx_start_tx_path(struct lan78xx_net *dev)
+{
+ int ret;
+
+ netif_dbg(dev, drv, dev->net, "start tx path");
+
+ /* Start the MAC transmitter */
+
+ ret = lan78xx_start_hw(dev, MAC_TX, MAC_TX_TXEN_);
+ if (ret < 0)
+ return ret;
+
+ /* Start the Tx FIFO */
+
+ ret = lan78xx_start_hw(dev, FCT_TX_CTL, FCT_TX_CTL_EN_);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int lan78xx_stop_tx_path(struct lan78xx_net *dev)
+{
+ int ret;
+
+ netif_dbg(dev, drv, dev->net, "stop tx path");
+
+ /* Stop the Tx FIFO */
+
+ ret = lan78xx_stop_hw(dev, FCT_TX_CTL, FCT_TX_CTL_EN_, FCT_TX_CTL_DIS_);
+ if (ret < 0)
+ return ret;
+
+ /* Stop the MAC transmitter */
+
+ ret = lan78xx_stop_hw(dev, MAC_TX, MAC_TX_TXEN_, MAC_TX_TXD_);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+/* The caller must ensure the Tx path is stopped before calling
+ * lan78xx_flush_tx_fifo().
+ */
+static int lan78xx_flush_tx_fifo(struct lan78xx_net *dev)
+{
+ return lan78xx_flush_fifo(dev, FCT_TX_CTL, FCT_TX_CTL_RST_);
+}
+
+static int lan78xx_start_rx_path(struct lan78xx_net *dev)
+{
+ int ret;
+
+ netif_dbg(dev, drv, dev->net, "start rx path");
+
+ /* Start the Rx FIFO */
+
+ ret = lan78xx_start_hw(dev, FCT_RX_CTL, FCT_RX_CTL_EN_);
+ if (ret < 0)
+ return ret;
+
+ /* Start the MAC receiver*/
+
+ ret = lan78xx_start_hw(dev, MAC_RX, MAC_RX_RXEN_);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int lan78xx_stop_rx_path(struct lan78xx_net *dev)
+{
+ int ret;
+
+ netif_dbg(dev, drv, dev->net, "stop rx path");
+
+ /* Stop the MAC receiver */
+
+ ret = lan78xx_stop_hw(dev, MAC_RX, MAC_RX_RXEN_, MAC_RX_RXD_);
+ if (ret < 0)
+ return ret;
+
+ /* Stop the Rx FIFO */
+
+ ret = lan78xx_stop_hw(dev, FCT_RX_CTL, FCT_RX_CTL_EN_, FCT_RX_CTL_DIS_);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+/* The caller must ensure the Rx path is stopped before calling
+ * lan78xx_flush_rx_fifo().
+ */
+static int lan78xx_flush_rx_fifo(struct lan78xx_net *dev)
+{
+ return lan78xx_flush_fifo(dev, FCT_RX_CTL, FCT_RX_CTL_RST_);
+}
+
/* Loop until the read is completed with timeout called with phy_mutex held */
static int lan78xx_phy_wait_not_busy(struct lan78xx_net *dev)
{
@@ -1263,6 +1413,17 @@ static void lan78xx_deferred_multicast_write(struct work_struct *param)
lan78xx_write_reg(dev, RFE_CTL, pdata->rfe_ctl);
}
+static void lan78xx_update_vlan_filter(struct lan78xx_priv *pdata,
+ struct net_device *netdev,
+ netdev_features_t features)
+{
+ if ((features & NETIF_F_HW_VLAN_CTAG_FILTER) &&
+ !(netdev->flags & IFF_PROMISC))
+ pdata->rfe_ctl |= RFE_CTL_VLAN_FILTER_;
+ else
+ pdata->rfe_ctl &= ~RFE_CTL_VLAN_FILTER_;
+}
+
static void lan78xx_set_multicast(struct net_device *netdev)
{
struct lan78xx_net *dev = netdev_priv(netdev);
@@ -1297,6 +1458,8 @@ static void lan78xx_set_multicast(struct net_device *netdev)
}
}
+ lan78xx_update_vlan_filter(pdata, dev->net, dev->net->features);
+
if (netdev_mc_count(dev->net)) {
struct netdev_hw_addr *ha;
int i;
@@ -2601,10 +2764,7 @@ static int lan78xx_set_features(struct net_device *netdev,
else
pdata->rfe_ctl &= ~RFE_CTL_VLAN_STRIP_;
- if (features & NETIF_F_HW_VLAN_CTAG_FILTER)
- pdata->rfe_ctl |= RFE_CTL_VLAN_FILTER_;
- else
- pdata->rfe_ctl &= ~RFE_CTL_VLAN_FILTER_;
+ lan78xx_update_vlan_filter(pdata, netdev, features);
spin_unlock_irqrestore(&pdata->rfe_ctl_lock, flags);
@@ -2742,156 +2902,6 @@ static int lan78xx_urb_config_init(struct lan78xx_net *dev)
return result;
}
-static int lan78xx_start_hw(struct lan78xx_net *dev, u32 reg, u32 hw_enable)
-{
- return lan78xx_update_reg(dev, reg, hw_enable, hw_enable);
-}
-
-static int lan78xx_stop_hw(struct lan78xx_net *dev, u32 reg, u32 hw_enabled,
- u32 hw_disabled)
-{
- unsigned long timeout;
- bool stopped = true;
- int ret;
- u32 buf;
-
- /* Stop the h/w block (if not already stopped) */
-
- ret = lan78xx_read_reg(dev, reg, &buf);
- if (ret < 0)
- return ret;
-
- if (buf & hw_enabled) {
- buf &= ~hw_enabled;
-
- ret = lan78xx_write_reg(dev, reg, buf);
- if (ret < 0)
- return ret;
-
- stopped = false;
- timeout = jiffies + HW_DISABLE_TIMEOUT;
- do {
- ret = lan78xx_read_reg(dev, reg, &buf);
- if (ret < 0)
- return ret;
-
- if (buf & hw_disabled)
- stopped = true;
- else
- msleep(HW_DISABLE_DELAY_MS);
- } while (!stopped && !time_after(jiffies, timeout));
- }
-
- ret = stopped ? 0 : -ETIME;
-
- return ret;
-}
-
-static int lan78xx_flush_fifo(struct lan78xx_net *dev, u32 reg, u32 fifo_flush)
-{
- return lan78xx_update_reg(dev, reg, fifo_flush, fifo_flush);
-}
-
-static int lan78xx_start_tx_path(struct lan78xx_net *dev)
-{
- int ret;
-
- netif_dbg(dev, drv, dev->net, "start tx path");
-
- /* Start the MAC transmitter */
-
- ret = lan78xx_start_hw(dev, MAC_TX, MAC_TX_TXEN_);
- if (ret < 0)
- return ret;
-
- /* Start the Tx FIFO */
-
- ret = lan78xx_start_hw(dev, FCT_TX_CTL, FCT_TX_CTL_EN_);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-static int lan78xx_stop_tx_path(struct lan78xx_net *dev)
-{
- int ret;
-
- netif_dbg(dev, drv, dev->net, "stop tx path");
-
- /* Stop the Tx FIFO */
-
- ret = lan78xx_stop_hw(dev, FCT_TX_CTL, FCT_TX_CTL_EN_, FCT_TX_CTL_DIS_);
- if (ret < 0)
- return ret;
-
- /* Stop the MAC transmitter */
-
- ret = lan78xx_stop_hw(dev, MAC_TX, MAC_TX_TXEN_, MAC_TX_TXD_);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-/* The caller must ensure the Tx path is stopped before calling
- * lan78xx_flush_tx_fifo().
- */
-static int lan78xx_flush_tx_fifo(struct lan78xx_net *dev)
-{
- return lan78xx_flush_fifo(dev, FCT_TX_CTL, FCT_TX_CTL_RST_);
-}
-
-static int lan78xx_start_rx_path(struct lan78xx_net *dev)
-{
- int ret;
-
- netif_dbg(dev, drv, dev->net, "start rx path");
-
- /* Start the Rx FIFO */
-
- ret = lan78xx_start_hw(dev, FCT_RX_CTL, FCT_RX_CTL_EN_);
- if (ret < 0)
- return ret;
-
- /* Start the MAC receiver*/
-
- ret = lan78xx_start_hw(dev, MAC_RX, MAC_RX_RXEN_);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-static int lan78xx_stop_rx_path(struct lan78xx_net *dev)
-{
- int ret;
-
- netif_dbg(dev, drv, dev->net, "stop rx path");
-
- /* Stop the MAC receiver */
-
- ret = lan78xx_stop_hw(dev, MAC_RX, MAC_RX_RXEN_, MAC_RX_RXD_);
- if (ret < 0)
- return ret;
-
- /* Stop the Rx FIFO */
-
- ret = lan78xx_stop_hw(dev, FCT_RX_CTL, FCT_RX_CTL_EN_, FCT_RX_CTL_DIS_);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-/* The caller must ensure the Rx path is stopped before calling
- * lan78xx_flush_rx_fifo().
- */
-static int lan78xx_flush_rx_fifo(struct lan78xx_net *dev)
-{
- return lan78xx_flush_fifo(dev, FCT_RX_CTL, FCT_RX_CTL_RST_);
-}
-
static int lan78xx_reset(struct lan78xx_net *dev)
{
struct lan78xx_priv *pdata = (struct lan78xx_priv *)(dev->data[0]);
diff --git a/drivers/net/usb/net1080.c b/drivers/net/usb/net1080.c
index 5d4a1fd2b524..19f6e1222d93 100644
--- a/drivers/net/usb/net1080.c
+++ b/drivers/net/usb/net1080.c
@@ -381,7 +381,7 @@ static int net1080_rx_fixup(struct usbnet *dev, struct sk_buff *skb)
skb_trim(skb, skb->len - sizeof *trailer);
if ((packet_len & 0x01) == 0) {
- if (skb->data [packet_len] != PAD_BYTE) {
+ if (packet_len >= skb->len || skb->data[packet_len] != PAD_BYTE) {
dev->net->stats.rx_frame_errors++;
netdev_dbg(dev->net, "bad pad\n");
return 0;
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 77e4b0d1ca55..eb7ce1211784 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1136,6 +1136,8 @@ static int veth_enable_xdp_range(struct net_device *dev, int start, int end,
err_reg_mem:
xdp_rxq_info_unreg(&priv->rq[i].xdp_rxq);
err_rxq_reg:
+ if (!napi_already_on)
+ netif_napi_del(&priv->rq[i].xdp_napi);
for (i--; i >= start; i--) {
struct veth_rq *rq = &priv->rq[i];
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 324802cef40b..0e8fa0f81e84 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -1895,15 +1895,18 @@ static struct sk_buff *receive_big(struct net_device *dev,
struct virtnet_rq_stats *stats)
{
struct page *page = buf;
+ unsigned long max_len;
struct sk_buff *skb;
+ max_len = (vi->big_packets_num_skbfrags + 1) * PAGE_SIZE -
+ sizeof(struct padded_vnet_hdr) + vi->hdr_len;
+
/* Make sure that len does not exceed the size allocated in
* add_recvbuf_big.
*/
- if (unlikely(len > (vi->big_packets_num_skbfrags + 1) * PAGE_SIZE)) {
+ if (unlikely(len > max_len)) {
pr_debug("%s: rx error: len %u exceeds allocated size %lu\n",
- dev->name, len,
- (vi->big_packets_num_skbfrags + 1) * PAGE_SIZE);
+ dev->name, len, max_len);
goto err;
}
@@ -2883,6 +2886,9 @@ static int virtnet_poll(struct napi_struct *napi, int budget)
unsigned int xdp_xmit = 0;
bool napi_complete;
+ if (budget)
+ virtqueue_disable_cb(rq->vq);
+
virtnet_poll_cleantx(rq, budget);
received = virtnet_receive(rq, budget, &xdp_xmit);
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index 765d25eee2fe..596efc51cac7 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -679,14 +679,18 @@ static struct vxlanhdr *vxlan_gro_prepare_receive(struct sock *sk,
struct sk_buff *skb,
struct gro_remcsum *grc)
{
- struct sk_buff *p;
struct vxlanhdr *vh, *vh2;
unsigned int hlen, off_vx;
- struct vxlan_sock *vs = rcu_dereference_sk_user_data(sk);
+ struct vxlan_sock *vs;
+ struct sk_buff *p;
__be32 flags;
skb_gro_remcsum_init(grc);
+ vs = rcu_dereference_sk_user_data(sk);
+ if (!vs)
+ return NULL;
+
off_vx = skb_gro_offset(skb);
hlen = off_vx + sizeof(*vh);
vh = skb_gro_header(skb, hlen, off_vx);
diff --git a/drivers/net/wan/ixp4xx_hss.c b/drivers/net/wan/ixp4xx_hss.c
index 931c5ca79ea5..18e8ececa96b 100644
--- a/drivers/net/wan/ixp4xx_hss.c
+++ b/drivers/net/wan/ixp4xx_hss.c
@@ -1487,11 +1487,11 @@ static int ixp4xx_hss_probe(struct platform_device *pdev)
"unable to get CLK internal GPIO\n");
ndev = alloc_hdlcdev(port);
- port->netdev = alloc_hdlcdev(port);
- if (!port->netdev) {
+ if (!ndev) {
err = -ENOMEM;
goto err_plat;
}
+ port->netdev = ndev;
SET_NETDEV_DEV(ndev, &pdev->dev);
hdlc = dev_to_hdlc(ndev);
diff --git a/drivers/net/wireless/ath/ath9k/xmit.c b/drivers/net/wireless/ath/ath9k/xmit.c
index 35aa47a9db90..26faccc98dd8 100644
--- a/drivers/net/wireless/ath/ath9k/xmit.c
+++ b/drivers/net/wireless/ath/ath9k/xmit.c
@@ -2759,6 +2759,11 @@ void ath_tx_edma_tasklet(struct ath_softc *sc)
continue;
}
+ if (ts.qid >= ATH9K_NUM_TX_QUEUES) {
+ ath_dbg(common, XMIT, "invalid qid %d\n", ts.qid);
+ continue;
+ }
+
txq = &sc->tx.txq[ts.qid];
ath_txq_lock(sc, txq);
diff --git a/drivers/net/wireless/ath/wcn36xx/smd.c b/drivers/net/wireless/ath/wcn36xx/smd.c
index 2cf86fc3f8fe..f7457dcd8980 100644
--- a/drivers/net/wireless/ath/wcn36xx/smd.c
+++ b/drivers/net/wireless/ath/wcn36xx/smd.c
@@ -2659,6 +2659,9 @@ static int wcn36xx_smd_trigger_ba_rsp(void *buf, int len, struct add_ba_info *ba
if (rsp->candidate_cnt < 1)
return rsp->status ? rsp->status : -EINVAL;
+ if (len < sizeof(*rsp) + sizeof(*candidate))
+ return -EINVAL;
+
candidate = (struct wcn36xx_hal_trigger_ba_rsp_candidate *)(buf + sizeof(*rsp));
for (i = 0; i < STACFG_MAX_TC; i++) {
@@ -2865,6 +2868,12 @@ static int wcn36xx_smd_print_reg_info_ind(struct wcn36xx *wcn,
return -EIO;
}
+ if (rsp->count > (len - sizeof(*rsp)) / sizeof(rsp->regs[0])) {
+ wcn36xx_warn("Truncated print reg info indication: count %u, len %zu\n",
+ rsp->count, len);
+ return -EIO;
+ }
+
wcn36xx_dbg(WCN36XX_DBG_HAL,
"reginfo indication, scenario: 0x%x reason: 0x%x\n",
rsp->scenario, rsp->reason);
@@ -3353,6 +3362,10 @@ int wcn36xx_smd_rsp_process(struct rpmsg_device *rpdev,
case WCN36XX_HAL_EXIT_IMPS_RSP:
case WCN36XX_HAL_UPDATE_CHANNEL_LIST_RSP:
case WCN36XX_HAL_ADD_BCN_FILTER_RSP:
+ if (len > WCN36XX_HAL_BUF_SIZE) {
+ wcn36xx_warn("HAL response too large: %d\n", len);
+ break;
+ }
memcpy(wcn->hal_buf, buf, len);
wcn->hal_rsp_len = len;
complete(&wcn->hal_rsp_compl);
diff --git a/drivers/net/wireless/marvell/mwifiex/cfg80211.c b/drivers/net/wireless/marvell/mwifiex/cfg80211.c
index 8801b93eacd4..dceee429bf71 100644
--- a/drivers/net/wireless/marvell/mwifiex/cfg80211.c
+++ b/drivers/net/wireless/marvell/mwifiex/cfg80211.c
@@ -4333,7 +4333,7 @@ mwifiex_cfg80211_authenticate(struct wiphy *wiphy,
return -EOPNOTSUPP;
}
- if (!priv->auth_flag) {
+ if (!(priv->auth_flag & HOST_MLME_AUTH_PENDING)) {
ret = mwifiex_remain_on_chan_cfg(priv, HostCmd_ACT_GEN_SET,
req->bss->channel,
AUTH_TX_DEFAULT_WAIT_TIME);
diff --git a/drivers/net/wireless/marvell/mwifiex/join.c b/drivers/net/wireless/marvell/mwifiex/join.c
index 6d8f1d1d7ca4..6500adae252c 100644
--- a/drivers/net/wireless/marvell/mwifiex/join.c
+++ b/drivers/net/wireless/marvell/mwifiex/join.c
@@ -737,7 +737,6 @@ int mwifiex_ret_802_11_associate(struct mwifiex_private *priv,
/* Send a Media Connected event, according to the Spec */
priv->media_connected = true;
- priv->adapter->ps_state = PS_STATE_AWAKE;
priv->adapter->pps_uapsd_mode = false;
priv->adapter->tx_lock_flag = false;
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index 5373f8c419b0..e538bb36a730 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -1188,7 +1188,7 @@ mt76_check_ccmp_pn(struct sk_buff *skb)
* All further fragments will be validated by mac80211 only.
*/
if (ieee80211_is_frag(hdr) &&
- !ieee80211_is_first_frag(hdr->frame_control))
+ !ieee80211_is_first_frag(hdr->seq_ctrl))
return;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
index ab31a350ca73..b058a38a88cf 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
@@ -923,16 +923,16 @@ mt7915_mac_tx_free(struct mt7915_dev *dev, void *data, int len)
}
if (!mtk_wed_device_active(&mdev->mmio.wed) && wcid) {
- u32 tx_retries = 0, tx_failed = 0;
+ u32 tx_retries = 0, tx_failed = 0, count;
if (v3 && (info & MT_TX_FREE_MPDU_HEADER_V3)) {
- tx_retries =
- FIELD_GET(MT_TX_FREE_COUNT_V3, info) - 1;
+ count = FIELD_GET(MT_TX_FREE_COUNT_V3, info);
+ tx_retries = count ? count - 1 : 0;
tx_failed = tx_retries +
!!FIELD_GET(MT_TX_FREE_STAT_V3, info);
} else if (!v3 && (info & MT_TX_FREE_MPDU_HEADER)) {
- tx_retries =
- FIELD_GET(MT_TX_FREE_COUNT, info) - 1;
+ count = FIELD_GET(MT_TX_FREE_COUNT, info);
+ tx_retries = count ? count - 1 : 0;
tx_failed = tx_retries +
!!FIELD_GET(MT_TX_FREE_STAT, info);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
index bd1455698ebe..0c6b6d6c246e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
@@ -541,8 +541,9 @@ static void mt7921_mac_tx_free(struct mt792x_dev *dev, void *data, int len)
stat = FIELD_GET(MT_TX_FREE_STATUS, info);
if (wcid) {
- wcid->stats.tx_retries +=
- FIELD_GET(MT_TX_FREE_COUNT, info) - 1;
+ u32 count = FIELD_GET(MT_TX_FREE_COUNT, info);
+
+ wcid->stats.tx_retries += count ? count - 1 : 0;
wcid->stats.tx_failed += !!stat;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
index e5de05a91aee..f01dbca6e9b9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
@@ -844,7 +844,6 @@ static void mt7925_tx_check_aggr(struct ieee80211_sta *sta, struct sk_buff *skb,
{
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
struct ieee80211_link_sta *link_sta;
- struct mt792x_link_sta *mlink;
struct mt792x_sta *msta;
bool is_8023;
u16 fc, tid;
@@ -883,14 +882,14 @@ static void mt7925_tx_check_aggr(struct ieee80211_sta *sta, struct sk_buff *skb,
msta = (struct mt792x_sta *)sta->drv_priv;
- if (sta->mlo && msta->deflink_id != IEEE80211_LINK_UNSPECIFIED)
- mlink = rcu_dereference(msta->link[msta->deflink_id]);
- else
- mlink = &msta->deflink;
-
- if (!test_and_set_bit(tid, &mlink->wcid.ampdu_state)) {
+ /* Packets belonging to the same TID can be transmitted over multiple
+ * links. Keep the TX BA session state in the primary link so all links
+ * share the same AMPDU bookkeeping.
+ */
+ if (!test_and_set_bit(tid, &msta->deflink.wcid.ampdu_state)) {
if (ieee80211_start_tx_ba_session(sta, tid, 0))
- clear_bit(tid, &mlink->wcid.ampdu_state);
+ clear_bit(tid, &msta->deflink.wcid.ampdu_state);
+
}
}
@@ -1151,8 +1150,9 @@ mt7925_mac_tx_free(struct mt792x_dev *dev, void *data, int len)
if (info & MT_TXFREE_INFO_HEADER) {
if (wcid) {
- wcid->stats.tx_retries +=
- FIELD_GET(MT_TXFREE_INFO_COUNT, info) - 1;
+ u32 count = FIELD_GET(MT_TXFREE_INFO_COUNT, info);
+
+ wcid->stats.tx_retries += count ? count - 1 : 0;
wcid->stats.tx_failed +=
!!FIELD_GET(MT_TXFREE_INFO_STAT, info);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
index b1f54cb54e17..98ec76cf2fe8 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
@@ -2063,9 +2063,9 @@ mt7925_change_vif_links(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
rcu_assign_pointer(mvif->link_conf[link_id], NULL);
rcu_assign_pointer(mvif->sta.link[link_id], NULL);
- if (mconf != &mvif->bss_conf)
+ if (mconfs[link_id] != &mvif->bss_conf)
devm_kfree(dev->mt76.dev, mconfs[link_id]);
- if (mlink != &mvif->sta.deflink)
+ if (mlinks[link_id] != &mvif->sta.deflink)
devm_kfree(dev->mt76.dev, mlinks[link_id]);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/pci.c b/drivers/net/wireless/mediatek/mt76/mt7925/pci.c
index a90e90131276..eb4684f575af 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/pci.c
@@ -410,10 +410,12 @@ static int mt7925_pci_probe(struct pci_dev *pdev,
ret = mt7925_register_device(dev);
if (ret)
- goto err_free_irq;
+ goto err_free_dma;
return 0;
+err_free_dma:
+ mt792x_dma_cleanup(dev);
err_free_irq:
devm_free_irq(&pdev->dev, pdev->irq, dev);
err_free_dev:
diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_core.c b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
index b87eed4d168d..c6afdf90fade 100644
--- a/drivers/net/wireless/mediatek/mt76/mt792x_core.c
+++ b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
@@ -79,7 +79,8 @@ void mt792x_tx(struct ieee80211_hw *hw, struct ieee80211_tx_control *control,
wcid = &mvif->sta.deflink.wcid;
}
- if (vif && control->sta && ieee80211_vif_is_mld(vif)) {
+ if (vif && control->sta && ieee80211_vif_is_mld(vif) &&
+ !(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP)) {
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
struct ieee80211_link_sta *link_sta;
struct ieee80211_bss_conf *conf;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
index 6f8167bb8613..54bbc871c0bd 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
@@ -1155,13 +1155,13 @@ mt7996_mac_tx_free(struct mt7996_dev *dev, void *data, int len)
spin_unlock_bh(&mdev->sta_poll_lock);
continue;
} else if (info & MT_TXFREE_INFO_HEADER) {
- u32 tx_retries = 0, tx_failed = 0;
+ u32 tx_retries = 0, tx_failed = 0, count;
if (!wcid)
continue;
- tx_retries =
- FIELD_GET(MT_TXFREE_INFO_COUNT, info) - 1;
+ count = FIELD_GET(MT_TXFREE_INFO_COUNT, info);
+ tx_retries = count ? count - 1 : 0;
tx_failed = tx_retries +
!!FIELD_GET(MT_TXFREE_INFO_STAT, info);
diff --git a/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c b/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c
index 9e7d9dbe954c..5f74f5fec3e4 100644
--- a/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c
+++ b/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c
@@ -1382,7 +1382,7 @@ int rt2x00lib_probe_dev(struct rt2x00_dev *rt2x00dev)
GFP_KERNEL);
if (!rt2x00dev->drv_data) {
retval = -ENOMEM;
- goto exit;
+ return retval;
}
}
@@ -1416,7 +1416,7 @@ int rt2x00lib_probe_dev(struct rt2x00_dev *rt2x00dev)
alloc_ordered_workqueue("%s", 0, wiphy_name(rt2x00dev->hw->wiphy));
if (!rt2x00dev->workqueue) {
retval = -ENOMEM;
- goto exit;
+ goto exit_free_drv_data;
}
INIT_WORK(&rt2x00dev->intf_work, rt2x00lib_intf_scheduled);
@@ -1488,6 +1488,14 @@ int rt2x00lib_probe_dev(struct rt2x00_dev *rt2x00dev)
exit:
rt2x00lib_remove_dev(rt2x00dev);
+ return retval;
+
+exit_free_drv_data:
+ clear_bit(DEVICE_STATE_PRESENT, &rt2x00dev->flags);
+
+ kfree(rt2x00dev->drv_data);
+ rt2x00dev->drv_data = NULL;
+
return retval;
}
EXPORT_SYMBOL_GPL(rt2x00lib_probe_dev);
diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c
index 7e33518ce311..f58a5a1a083c 100644
--- a/drivers/net/wireless/realtek/rtw88/pci.c
+++ b/drivers/net/wireless/realtek/rtw88/pci.c
@@ -1076,6 +1076,11 @@ static u32 rtw_pci_rx_napi(struct rtw_dev *rtwdev, struct rtw_pci *rtwpci,
* discard the frame if none available
*/
new_len = pkt_stat.pkt_len + pkt_offset;
+ if (unlikely(new_len > RTK_PCI_RX_BUF_SIZE)) {
+ rtw_dbg(rtwdev, RTW_DBG_RX,
+ "oversized RX packet: %u\n", new_len);
+ goto next_rp;
+ }
new = dev_alloc_skb(new_len);
if (WARN_ONCE(!new, "rx routine starvation\n"))
goto next_rp;
diff --git a/drivers/net/wireless/realtek/rtw89/fw.c b/drivers/net/wireless/realtek/rtw89/fw.c
index b62b59bf528e..832638fb100a 100644
--- a/drivers/net/wireless/realtek/rtw89/fw.c
+++ b/drivers/net/wireless/realtek/rtw89/fw.c
@@ -6144,7 +6144,7 @@ static int rtw89_update_6ghz_rnr_chan(struct rtw89_dev *rtwdev,
struct sk_buff *skb;
bool found;
int ret = 0;
- u8 i;
+ u32 i;
if (!req->n_6ghz_params)
return 0;
diff --git a/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c b/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c
index bff46f7ca59f..0bbd41263cc2 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_mux_codec.c
@@ -553,19 +553,21 @@ static int mux_dl_process_dg(struct iosm_mux *ipc_mux, struct mux_adbh *adbh,
u32 packet_offset, i, rc, dg_len;
for (i = 0; i < nr_of_dg; i++, dg++) {
- if (le32_to_cpu(dg->datagram_index)
- < sizeof(struct mux_adbh))
+ u32 dg_index = le32_to_cpu(dg->datagram_index);
+
+ dg_len = le16_to_cpu(dg->datagram_length);
+
+ if (dg_index < sizeof(struct mux_adbh))
goto dg_error;
- /* Is the packet inside of the ADB */
- if (le32_to_cpu(dg->datagram_index) >=
- le32_to_cpu(adbh->block_length)) {
+ /* Is the packet inside of the ADB and the received skb ? */
+ if (dg_index >= le32_to_cpu(adbh->block_length) ||
+ dg_index >= skb->len ||
+ dg_len > skb->len - dg_index ||
+ dl_head_pad_len >= dg_len) {
goto dg_error;
} else {
- packet_offset =
- le32_to_cpu(dg->datagram_index) +
- dl_head_pad_len;
- dg_len = le16_to_cpu(dg->datagram_length);
+ packet_offset = dg_index + dl_head_pad_len;
/* Pass the packet to the netif layer. */
rc = ipc_mux_net_receive(ipc_mux, if_id, ipc_mux->wwan,
packet_offset,
@@ -589,12 +591,16 @@ static void mux_dl_adb_decode(struct iosm_mux *ipc_mux,
struct mux_adbh *adbh;
struct mux_adth *adth;
int nr_of_dg, if_id;
- u32 adth_index;
+ u32 adth_index, prev_index = 0;
u8 *block;
block = skb->data;
adbh = (struct mux_adbh *)block;
+ /* The block header itself must fit in the received skb. */
+ if (skb->len < sizeof(struct mux_adbh))
+ goto adb_decode_err;
+
/* Process the aggregated datagram tables. */
adth_index = le32_to_cpu(adbh->first_table_index);
@@ -606,6 +612,16 @@ static void mux_dl_adb_decode(struct iosm_mux *ipc_mux,
/* Loop through mixed session tables. */
while (adth_index) {
+ /* The table header must lie within the received skb, and the
+ * chain must move forward so a modem cannot make the loop
+ * cycle between two tables.
+ */
+ if (adth_index <= prev_index ||
+ adth_index < sizeof(struct mux_adbh) ||
+ adth_index > skb->len - sizeof(struct mux_adth))
+ goto adb_decode_err;
+ prev_index = adth_index;
+
/* Get the reference to the table header. */
adth = (struct mux_adth *)(block + adth_index);
@@ -629,6 +645,10 @@ static void mux_dl_adb_decode(struct iosm_mux *ipc_mux,
if (le16_to_cpu(adth->table_length) < sizeof(struct mux_adth))
goto adb_decode_err;
+ /* The whole datagram table must fit in the received skb. */
+ if (le16_to_cpu(adth->table_length) > skb->len - adth_index)
+ goto adb_decode_err;
+
/* Calculate the number of datagrams. */
nr_of_dg = (le16_to_cpu(adth->table_length) -
sizeof(struct mux_adth)) /
diff --git a/drivers/net/wwan/t7xx/t7xx_hif_cldma.c b/drivers/net/wwan/t7xx/t7xx_hif_cldma.c
index 97163e1e5783..f7aa9398f5cd 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_cldma.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_cldma.c
@@ -1066,6 +1066,9 @@ static int t7xx_cldma_late_init(struct cldma_ctrl *md_ctrl)
while (i--)
t7xx_cldma_ring_free(md_ctrl, &md_ctrl->tx_ring[i], DMA_TO_DEVICE);
+ dma_pool_destroy(md_ctrl->gpd_dmapool);
+ md_ctrl->gpd_dmapool = NULL;
+
return ret;
}
diff --git a/drivers/net/wwan/t7xx/t7xx_port_wwan.c b/drivers/net/wwan/t7xx/t7xx_port_wwan.c
index 4b23ba693f3f..5de2b5fb6476 100644
--- a/drivers/net/wwan/t7xx/t7xx_port_wwan.c
+++ b/drivers/net/wwan/t7xx/t7xx_port_wwan.c
@@ -106,6 +106,8 @@ static int t7xx_port_ctrl_tx(struct t7xx_port *port, struct sk_buff *skb)
while (cur) {
cloned = skb_clone(cur, GFP_KERNEL);
+ if (!cloned)
+ return cnt ? cnt : -ENOMEM;
cloned->len = skb_headlen(cur);
ret = t7xx_port_send_skb(port, cloned, 0, 0);
if (ret) {
diff --git a/drivers/ntb/hw/epf/ntb_hw_epf.c b/drivers/ntb/hw/epf/ntb_hw_epf.c
index dac1520906a4..b14c5db32916 100644
--- a/drivers/ntb/hw/epf/ntb_hw_epf.c
+++ b/drivers/ntb/hw/epf/ntb_hw_epf.c
@@ -6,6 +6,7 @@
* Author: Kishon Vijay Abraham I <kishon@ti.com>
*/
+#include <linux/atomic.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/pci.h>
@@ -69,8 +70,21 @@ enum epf_ntb_bar {
NTB_BAR_NUM,
};
+enum epf_irq_slot {
+ EPF_IRQ_LINK = 0,
+ EPF_IRQ_RESERVED_DB, /* Historically skipped slot */
+ EPF_IRQ_DB_START,
+};
+
#define NTB_EPF_MAX_MW_COUNT (NTB_BAR_NUM - BAR_MW1)
+struct ntb_epf_dev;
+
+struct ntb_epf_irq_ctx {
+ struct ntb_epf_dev *ndev;
+ unsigned int irq_no;
+};
+
struct ntb_epf_dev {
struct ntb_dev ntb;
struct device *dev;
@@ -90,9 +104,9 @@ struct ntb_epf_dev {
unsigned int self_spad;
unsigned int peer_spad;
- int db_val;
+ atomic64_t db_val;
u64 db_valid_mask;
- int irq_base;
+ struct ntb_epf_irq_ctx irq_ctx[NTB_EPF_MAX_DB_COUNT + 1];
};
#define ntb_ndev(__ntb) container_of(__ntb, struct ntb_epf_dev, ntb)
@@ -316,16 +330,29 @@ static int ntb_epf_link_disable(struct ntb_dev *ntb)
static irqreturn_t ntb_epf_vec_isr(int irq, void *dev)
{
- struct ntb_epf_dev *ndev = dev;
- int irq_no;
-
- irq_no = irq - ndev->irq_base;
- ndev->db_val = irq_no + 1;
+ struct ntb_epf_irq_ctx *ctx = dev;
+ struct ntb_epf_dev *ndev = ctx->ndev;
+ unsigned int db_vector;
+ unsigned int irq_no = ctx->irq_no;
- if (irq_no == 0)
+ if (irq_no == EPF_IRQ_LINK) {
ntb_link_event(&ndev->ntb);
- else
- ntb_db_event(&ndev->ntb, irq_no);
+ } else if (irq_no == EPF_IRQ_RESERVED_DB) {
+ dev_warn_ratelimited(ndev->dev,
+ "Unexpected reserved doorbell slot IRQ received\n");
+ } else {
+ db_vector = irq_no - EPF_IRQ_DB_START;
+ if (ndev->db_count < NTB_EPF_MIN_DB_COUNT ||
+ db_vector >= ndev->db_count - 1) {
+ dev_warn_ratelimited(ndev->dev,
+ "Unexpected doorbell vector %u (db_count %u)\n",
+ db_vector, ndev->db_count);
+ return IRQ_HANDLED;
+ }
+
+ atomic64_or(BIT_ULL(db_vector), &ndev->db_val);
+ ntb_db_event(&ndev->ntb, db_vector);
+ }
return IRQ_HANDLED;
}
@@ -351,18 +378,18 @@ static int ntb_epf_init_isr(struct ntb_epf_dev *ndev, int msi_min, int msi_max)
argument &= ~MSIX_ENABLE;
}
- ndev->irq_base = pci_irq_vector(pdev, 0);
+ ndev->db_count = irq - 1;
for (i = 0; i < irq; i++) {
+ ndev->irq_ctx[i].ndev = ndev;
+ ndev->irq_ctx[i].irq_no = i;
ret = request_irq(pci_irq_vector(pdev, i), ntb_epf_vec_isr,
- 0, "ntb_epf", ndev);
+ 0, "ntb_epf", &ndev->irq_ctx[i]);
if (ret) {
dev_err(dev, "Failed to request irq\n");
goto err_free_irq;
}
}
- ndev->db_count = irq - 1;
-
ret = ntb_epf_send_command(ndev, CMD_CONFIGURE_DOORBELL,
argument | irq);
if (ret) {
@@ -374,7 +401,7 @@ static int ntb_epf_init_isr(struct ntb_epf_dev *ndev, int msi_min, int msi_max)
err_free_irq:
while (i--)
- free_irq(pci_irq_vector(pdev, i), ndev);
+ free_irq(pci_irq_vector(pdev, i), &ndev->irq_ctx[i]);
pci_free_irq_vectors(pdev);
return ret;
@@ -499,7 +526,7 @@ static u64 ntb_epf_db_read(struct ntb_dev *ntb)
{
struct ntb_epf_dev *ndev = ntb_ndev(ntb);
- return ndev->db_val;
+ return atomic64_read(&ndev->db_val);
}
static int ntb_epf_db_clear_mask(struct ntb_dev *ntb, u64 db_bits)
@@ -511,7 +538,7 @@ static int ntb_epf_db_clear(struct ntb_dev *ntb, u64 db_bits)
{
struct ntb_epf_dev *ndev = ntb_ndev(ntb);
- ndev->db_val = 0;
+ atomic64_and(~db_bits, &ndev->db_val);
return 0;
}
@@ -552,6 +579,12 @@ static int ntb_epf_init_dev(struct ntb_epf_dev *ndev)
struct device *dev = ndev->dev;
int ret;
+ ndev->mw_count = readl(ndev->ctrl_reg + NTB_EPF_MW_COUNT);
+ if (ndev->mw_count > NTB_EPF_MAX_MW_COUNT) {
+ dev_err(dev, "Unsupported MW count: %u\n", ndev->mw_count);
+ return -EINVAL;
+ }
+
/* One Link interrupt and rest doorbell interrupt */
ret = ntb_epf_init_isr(ndev, NTB_EPF_MIN_DB_COUNT + 1,
NTB_EPF_MAX_DB_COUNT + 1);
@@ -560,15 +593,13 @@ static int ntb_epf_init_dev(struct ntb_epf_dev *ndev)
return ret;
}
- ndev->db_valid_mask = BIT_ULL(ndev->db_count) - 1;
- ndev->mw_count = readl(ndev->ctrl_reg + NTB_EPF_MW_COUNT);
+ /*
+ * ndev->db_count includes an extra skipped slot due to the legacy
+ * doorbell layout, hence -1.
+ */
+ ndev->db_valid_mask = BIT_ULL(ndev->db_count - 1) - 1;
ndev->spad_count = readl(ndev->ctrl_reg + NTB_EPF_SPAD_COUNT);
- if (ndev->mw_count > NTB_EPF_MAX_MW_COUNT) {
- dev_err(dev, "Unsupported MW count: %u\n", ndev->mw_count);
- return -EINVAL;
- }
-
return 0;
}
@@ -663,7 +694,7 @@ static void ntb_epf_cleanup_isr(struct ntb_epf_dev *ndev)
ntb_epf_send_command(ndev, CMD_TEARDOWN_DOORBELL, ndev->db_count + 1);
for (i = 0; i < ndev->db_count + 1; i++)
- free_irq(pci_irq_vector(pdev, i), ndev);
+ free_irq(pci_irq_vector(pdev, i), &ndev->irq_ctx[i]);
pci_free_irq_vectors(pdev);
}
diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index 423dcd190906..80ecbfb3c6e9 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -921,6 +921,9 @@ static int discover_arenas(struct btt *btt)
return ret;
out:
+ kfree(arena->freelist);
+ kfree(arena->rtt);
+ kfree(arena->map_locks);
kfree(arena);
free_arenas(btt);
return ret;
@@ -1589,7 +1592,7 @@ static struct btt *btt_init(struct nd_btt *nd_btt, unsigned long long rawsize,
if (btt->init_state != INIT_READY && nd_region->ro) {
dev_warn(dev, "%s is read-only, unable to init btt metadata\n",
dev_name(&nd_region->dev));
- return NULL;
+ goto err;
} else if (btt->init_state != INIT_READY) {
btt->num_arenas = (rawsize / ARENA_MAX_SIZE) +
((rawsize % ARENA_MAX_SIZE) ? 1 : 0);
@@ -1599,25 +1602,28 @@ static struct btt *btt_init(struct nd_btt *nd_btt, unsigned long long rawsize,
ret = create_arenas(btt);
if (ret) {
dev_info(dev, "init: create_arenas: %d\n", ret);
- return NULL;
+ goto err;
}
ret = btt_meta_init(btt);
if (ret) {
dev_err(dev, "init: error in meta_init: %d\n", ret);
- return NULL;
+ goto err;
}
}
ret = btt_blk_init(btt);
if (ret) {
dev_err(dev, "init: error in blk_init: %d\n", ret);
- return NULL;
+ goto err;
}
btt_debugfs_init(btt);
return btt;
+err:
+ free_arenas(btt);
+ return NULL;
}
/**
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index 5ca06e9a2d29..28ea8b94bae0 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -365,11 +365,6 @@ unsigned sizeof_namespace_label(struct nvdimm_drvdata *ndd);
for (res = (ndd)->dpa.child, next = res ? res->sibling : NULL; \
res; res = next, next = next ? next->sibling : NULL)
-struct nd_percpu_lane {
- int count;
- spinlock_t lock;
-};
-
enum nd_label_flags {
ND_LABEL_REAP,
};
@@ -400,6 +395,10 @@ struct nd_mapping {
struct nvdimm_drvdata *ndd;
};
+struct nd_lane {
+ struct mutex lock; /* serialize lane access */
+} ____cacheline_aligned_in_smp;
+
struct nd_region {
struct device dev;
struct ida ns_ida;
@@ -420,7 +419,7 @@ struct nd_region {
struct kernfs_node *bb_state;
struct badblocks bb;
struct nd_interleave_set *nd_set;
- struct nd_percpu_lane __percpu *lane;
+ struct nd_lane *lane;
int (*flush)(struct nd_region *nd_region, struct bio *bio);
struct nd_mapping mapping[] __counted_by(ndr_mappings);
};
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index 9e4f7ff024a0..849c131f39c7 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -179,7 +179,9 @@ static void nd_region_release(struct device *dev)
put_device(&nvdimm->dev);
}
- free_percpu(nd_region->lane);
+ for (i = 0; i < nd_region->num_lanes; i++)
+ mutex_destroy(&nd_region->lane[i].lock);
+ kfree(nd_region->lane);
if (!test_bit(ND_REGION_CXL, &nd_region->flags))
memregion_free(nd_region->id);
kfree(nd_region);
@@ -922,52 +924,30 @@ void nd_region_advance_seeds(struct nd_region *nd_region, struct device *dev)
* nd_region_acquire_lane - allocate and lock a lane
* @nd_region: region id and number of lanes possible
*
- * A lane correlates to a BLK-data-window and/or a log slot in the BTT.
- * We optimize for the common case where there are 256 lanes, one
- * per-cpu. For larger systems we need to lock to share lanes. For now
- * this implementation assumes the cost of maintaining an allocator for
- * free lanes is on the order of the lock hold time, so it implements a
- * static lane = cpu % num_lanes mapping.
+ * A lane correlates to a log slot in the BTT. Lanes are shared across
+ * CPUs using a static lane = cpu % num_lanes mapping, with a per-lane
+ * mutex to serialize access.
*
- * In the case of a BTT instance on top of a BLK namespace a lane may be
- * acquired recursively. We lock on the first instance.
- *
- * In the case of a BTT instance on top of PMEM, we only acquire a lane
- * for the BTT metadata updates.
+ * Callers must be in sleepable context. The only in-tree caller is
+ * BTT's ->submit_bio handler (btt_read_pg / btt_write_pg).
*/
unsigned int nd_region_acquire_lane(struct nd_region *nd_region)
+ __acquires(&nd_region->lane[lane].lock)
{
- unsigned int cpu, lane;
-
- migrate_disable();
- cpu = smp_processor_id();
- if (nd_region->num_lanes < nr_cpu_ids) {
- struct nd_percpu_lane *ndl_lock, *ndl_count;
+ unsigned int lane;
- lane = cpu % nd_region->num_lanes;
- ndl_count = per_cpu_ptr(nd_region->lane, cpu);
- ndl_lock = per_cpu_ptr(nd_region->lane, lane);
- if (ndl_count->count++ == 0)
- spin_lock(&ndl_lock->lock);
- } else
- lane = cpu;
+ might_sleep();
+ lane = raw_smp_processor_id() % nd_region->num_lanes;
+ mutex_lock(&nd_region->lane[lane].lock);
return lane;
}
EXPORT_SYMBOL(nd_region_acquire_lane);
void nd_region_release_lane(struct nd_region *nd_region, unsigned int lane)
+ __releases(&nd_region->lane[lane].lock)
{
- if (nd_region->num_lanes < nr_cpu_ids) {
- unsigned int cpu = smp_processor_id();
- struct nd_percpu_lane *ndl_lock, *ndl_count;
-
- ndl_count = per_cpu_ptr(nd_region->lane, cpu);
- ndl_lock = per_cpu_ptr(nd_region->lane, lane);
- if (--ndl_count->count == 0)
- spin_unlock(&ndl_lock->lock);
- }
- migrate_enable();
+ mutex_unlock(&nd_region->lane[lane].lock);
}
EXPORT_SYMBOL(nd_region_release_lane);
@@ -1038,17 +1018,16 @@ static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus,
goto err_id;
}
- nd_region->lane = alloc_percpu(struct nd_percpu_lane);
+ nd_region->num_lanes = ndr_desc->num_lanes;
+ if (!nd_region->num_lanes)
+ goto err_percpu;
+ nd_region->lane = kcalloc(nd_region->num_lanes,
+ sizeof(*nd_region->lane), GFP_KERNEL);
if (!nd_region->lane)
goto err_percpu;
- for (i = 0; i < nr_cpu_ids; i++) {
- struct nd_percpu_lane *ndl;
-
- ndl = per_cpu_ptr(nd_region->lane, i);
- spin_lock_init(&ndl->lock);
- ndl->count = 0;
- }
+ for (i = 0; i < nd_region->num_lanes; i++)
+ mutex_init(&nd_region->lane[i].lock);
for (i = 0; i < ndr_desc->num_mappings; i++) {
struct nd_mapping_desc *mapping = &ndr_desc->mapping[i];
@@ -1065,7 +1044,6 @@ static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus,
}
nd_region->provider_data = ndr_desc->provider_data;
nd_region->nd_set = ndr_desc->nd_set;
- nd_region->num_lanes = ndr_desc->num_lanes;
nd_region->flags = ndr_desc->flags;
nd_region->ro = ro;
nd_region->numa_node = ndr_desc->numa_node;
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 122de4e35d45..ed82d3c57724 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3656,7 +3656,7 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
int ret = -ENOMEM;
#ifdef CONFIG_NVME_MULTIPATH
- size += num_possible_nodes() * sizeof(struct nvme_ns *);
+ size += nr_node_ids * sizeof(struct nvme_ns *);
#endif
head = kzalloc(size, GFP_KERNEL);
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index fed2af425330..eb971feece0f 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -667,7 +667,8 @@ static void nvmet_rdma_release_rsp(struct nvmet_rdma_rsp *rsp)
if (rsp->n_rdma)
nvmet_rdma_rw_ctx_destroy(rsp);
- if (rsp->req.sg != rsp->cmd->inline_sg)
+ if (rsp->req.sg < rsp->cmd->inline_sg ||
+ rsp->req.sg >= rsp->cmd->inline_sg + queue->dev->inline_page_count)
nvmet_req_free_sgls(&rsp->req);
if (unlikely(!list_empty_careful(&queue->rsp_wr_wait_list)))
@@ -822,24 +823,25 @@ static void nvmet_rdma_write_data_done(struct ib_cq *cq, struct ib_wc *wc)
static void nvmet_rdma_use_inline_sg(struct nvmet_rdma_rsp *rsp, u32 len,
u64 off)
{
- int sg_count = num_pages(len);
+ u64 page_off = off % PAGE_SIZE;
+ u64 page_idx = off / PAGE_SIZE;
+ int sg_count = num_pages(page_off + len);
struct scatterlist *sg;
int i;
- sg = rsp->cmd->inline_sg;
+ sg = &rsp->cmd->inline_sg[page_idx];
for (i = 0; i < sg_count; i++, sg++) {
if (i < sg_count - 1)
sg_unmark_end(sg);
else
sg_mark_end(sg);
- sg->offset = off;
- sg->length = min_t(int, len, PAGE_SIZE - off);
+ sg->offset = page_off;
+ sg->length = min_t(u64, len, PAGE_SIZE - page_off);
len -= sg->length;
- if (!i)
- off = 0;
+ page_off = 0;
}
- rsp->req.sg = rsp->cmd->inline_sg;
+ rsp->req.sg = &rsp->cmd->inline_sg[page_idx];
rsp->req.sg_cnt = sg_count;
}
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index 6c64af6a7c07..dbb6230b4b7f 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1383,7 +1383,10 @@ static int nvmet_tcp_try_recv_ddgst(struct nvmet_tcp_queue *queue)
queue->idx, cmd->req.cmd->common.command_id,
queue->pdu.cmd.hdr.type, le32_to_cpu(cmd->recv_ddgst),
le32_to_cpu(cmd->exp_ddgst));
- nvmet_req_uninit(&cmd->req);
+ if (!(cmd->flags & NVMET_TCP_F_INIT_FAILED)) {
+ cmd->req.cqe->status = NVME_SC_CMD_SEQ_ERROR;
+ nvmet_req_uninit(&cmd->req);
+ }
nvmet_tcp_free_cmd_buffers(cmd);
nvmet_tcp_fatal_error(queue);
ret = -EPROTO;
@@ -2032,6 +2035,12 @@ static void nvmet_tcp_alloc_queue(struct nvmet_tcp_port *port,
nvmet_tcp_free_cmd(&queue->connect);
out_ida_remove:
ida_free(&nvmet_tcp_queue_ida, queue->idx);
+ /*
+ * Drain the page fragment cache if any allocations were done.
+ * The first allocation using pf_cache is nvmet_tcp_alloc_cmd()
+ * for queue->connect after ida_alloc().
+ */
+ page_frag_cache_drain(&queue->pf_cache);
out_sock:
fput(queue->sock->file);
out_free_queue:
diff --git a/drivers/of/cpu.c b/drivers/of/cpu.c
index d17b2f851082..8b618e8c5ddf 100644
--- a/drivers/of/cpu.c
+++ b/drivers/of/cpu.c
@@ -60,7 +60,7 @@ static bool __of_find_n_match_cpu_property(struct device_node *cpun,
cell = of_get_property(cpun, prop_name, &prop_len);
if (!cell && !ac && arch_match_cpu_phys_id(cpu, 0))
return true;
- if (!cell || !ac)
+ if (!cell || !ac || ac > 2)
return false;
prop_len /= sizeof(*cell) * ac;
for (tid = 0; tid < prop_len; tid++) {
diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
index f5f624fc327f..77818436fd60 100644
--- a/drivers/of/dynamic.c
+++ b/drivers/of/dynamic.c
@@ -225,7 +225,6 @@ static void __of_attach_node(struct device_node *np)
np->sibling = np->parent->child;
np->parent->child = np;
of_node_clear_flag(np, OF_DETACHED);
- fwnode_set_flag(&np->fwnode, FWNODE_FLAG_NOT_DEVICE);
raw_spin_unlock_irqrestore(&devtree_lock, flags);
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 7fe456dddce3..052d7aceec97 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -740,11 +740,6 @@ static int of_platform_notify(struct notifier_block *nb,
if (of_node_check_flag(rd->dn, OF_POPULATED))
return NOTIFY_OK;
- /*
- * Clear the flag before adding the device so that fw_devlink
- * doesn't skip adding consumers to this device.
- */
- fwnode_clear_flag(&rd->dn->fwnode, FWNODE_FLAG_NOT_DEVICE);
/* pdev_parent may be NULL when no bus platform device */
pdev_parent = of_find_device_by_node(parent);
pdev = of_platform_device_create(rd->dn, NULL,
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 374c5c06d92f..452f8f5a6d6f 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -4,7 +4,7 @@
obj-$(CONFIG_PCI) += access.o bus.o probe.o host-bridge.o \
remove.o pci.o pci-driver.o search.o \
- rom.o setup-res.o irq.o vpd.o \
+ rebar.o rom.o setup-res.o irq.o vpd.o \
setup-bus.o vc.o mmap.o devres.o
obj-$(CONFIG_PCI) += msi/
diff --git a/drivers/pci/controller/Kconfig b/drivers/pci/controller/Kconfig
index 481acb03af80..a44f0d46b26a 100644
--- a/drivers/pci/controller/Kconfig
+++ b/drivers/pci/controller/Kconfig
@@ -190,6 +190,7 @@ config PCIE_MEDIATEK
depends on ARCH_AIROHA || ARCH_MEDIATEK || COMPILE_TEST
depends on OF
depends on PCI_MSI
+ select IRQ_MSI_LIB
help
Say Y here if you want to enable PCIe controller support on
MediaTek SoCs.
diff --git a/drivers/pci/controller/dwc/pci-meson.c b/drivers/pci/controller/dwc/pci-meson.c
index db9482a113e9..3fff8ed93ef3 100644
--- a/drivers/pci/controller/dwc/pci-meson.c
+++ b/drivers/pci/controller/dwc/pci-meson.c
@@ -193,7 +193,9 @@ static inline struct clk *meson_pcie_probe_clock(struct device *dev,
return ERR_PTR(ret);
}
- devm_add_action_or_reset(dev, meson_pcie_disable_clock, clk);
+ ret = devm_add_action_or_reset(dev, meson_pcie_disable_clock, clk);
+ if (ret)
+ return ERR_PTR(ret);
return clk;
}
@@ -470,6 +472,14 @@ static int meson_pcie_probe(struct platform_device *pdev)
return ret;
}
+static void meson_pcie_remove(struct platform_device *pdev)
+{
+ struct meson_pcie *mp = platform_get_drvdata(pdev);
+
+ dw_pcie_host_deinit(&mp->pci.pp);
+ meson_pcie_power_off(mp);
+}
+
static const struct of_device_id meson_pcie_of_match[] = {
{
.compatible = "amlogic,axg-pcie",
@@ -483,6 +493,7 @@ MODULE_DEVICE_TABLE(of, meson_pcie_of_match);
static struct platform_driver meson_pcie_driver = {
.probe = meson_pcie_probe,
+ .remove = meson_pcie_remove,
.driver = {
.name = "meson-pcie",
.of_match_table = meson_pcie_of_match,
diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index deda5b040d7a..79c9bb64abd4 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -227,7 +227,7 @@ static const struct irq_domain_ops dw_pcie_msi_domain_ops = {
int dw_pcie_allocate_domains(struct dw_pcie_rp *pp)
{
struct dw_pcie *pci = to_dw_pcie_from_pp(pp);
- struct fwnode_handle *fwnode = of_node_to_fwnode(pci->dev->of_node);
+ struct fwnode_handle *fwnode = dev_fwnode(pci->dev);
pp->irq_domain = irq_domain_create_linear(fwnode, pp->num_vectors,
&dw_pcie_msi_domain_ops, pp);
diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
index 5d27cd149f51..81ba04cedc72 100644
--- a/drivers/pci/controller/dwc/pcie-qcom.c
+++ b/drivers/pci/controller/dwc/pcie-qcom.c
@@ -1385,6 +1385,7 @@ static const struct qcom_pcie_cfg cfg_1_9_0 = {
static const struct qcom_pcie_cfg cfg_1_34_0 = {
.ops = &ops_1_9_0,
.override_no_snoop = true,
+ .no_l0s = true,
};
static const struct qcom_pcie_cfg cfg_2_1_0 = {
@@ -1508,6 +1509,22 @@ static void qcom_pcie_icc_opp_update(struct qcom_pcie *pcie)
}
}
+static int qcom_pcie_set_max_opp(struct device *dev)
+{
+ unsigned long max_freq = ULONG_MAX;
+ struct dev_pm_opp *opp;
+ int ret;
+
+ opp = dev_pm_opp_find_freq_floor(dev, &max_freq);
+ if (IS_ERR(opp))
+ return PTR_ERR(opp);
+
+ ret = dev_pm_opp_set_opp(dev, opp);
+ dev_pm_opp_put(opp);
+
+ return ret;
+}
+
static int qcom_pcie_link_transition_count(struct seq_file *s, void *data)
{
struct qcom_pcie *pcie = (struct qcom_pcie *)dev_get_drvdata(s->private);
@@ -1573,9 +1590,7 @@ static irqreturn_t qcom_pcie_global_irq_thread(int irq, void *data)
static int qcom_pcie_probe(struct platform_device *pdev)
{
const struct qcom_pcie_cfg *pcie_cfg;
- unsigned long max_freq = ULONG_MAX;
struct device *dev = &pdev->dev;
- struct dev_pm_opp *opp;
struct qcom_pcie *pcie;
struct dw_pcie_rp *pp;
struct resource *res;
@@ -1658,21 +1673,9 @@ static int qcom_pcie_probe(struct platform_device *pdev)
* probe(), OPP will be updated using qcom_pcie_icc_opp_update().
*/
if (!ret) {
- opp = dev_pm_opp_find_freq_floor(dev, &max_freq);
- if (IS_ERR(opp)) {
- ret = PTR_ERR(opp);
- dev_err_probe(pci->dev, ret,
- "Unable to find max freq OPP\n");
- goto err_pm_runtime_put;
- } else {
- ret = dev_pm_opp_set_opp(dev, opp);
- }
-
- dev_pm_opp_put(opp);
+ ret = qcom_pcie_set_max_opp(dev);
if (ret) {
- dev_err_probe(pci->dev, ret,
- "Failed to set OPP for freq %lu\n",
- max_freq);
+ dev_err_probe(dev, ret, "Failed to set max OPP\n");
goto err_pm_runtime_put;
}
@@ -1803,6 +1806,14 @@ static int qcom_pcie_resume_noirq(struct device *dev)
int ret;
if (pm_suspend_target_state != PM_SUSPEND_MEM) {
+ if (pcie->use_pm_opp) {
+ ret = qcom_pcie_set_max_opp(dev);
+ if (ret) {
+ dev_err(dev, "Failed to set max OPP: %d\n", ret);
+ return ret;
+ }
+ }
+
ret = icc_enable(pcie->icc_cpu);
if (ret) {
dev_err(dev, "Failed to enable CPU-PCIe interconnect path: %d\n", ret);
diff --git a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
index 0e088e74155d..910b589cb096 100644
--- a/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
+++ b/drivers/pci/controller/mobiveil/pcie-mobiveil-host.c
@@ -435,7 +435,7 @@ static const struct irq_domain_ops msi_domain_ops = {
static int mobiveil_allocate_msi_domains(struct mobiveil_pcie *pcie)
{
struct device *dev = &pcie->pdev->dev;
- struct fwnode_handle *fwnode = of_node_to_fwnode(dev->of_node);
+ struct fwnode_handle *fwnode = dev_fwnode(dev);
struct mobiveil_msi *msi = &pcie->rp.msi;
mutex_init(&msi->lock);
@@ -461,13 +461,11 @@ static int mobiveil_allocate_msi_domains(struct mobiveil_pcie *pcie)
static int mobiveil_pcie_init_irq_domain(struct mobiveil_pcie *pcie)
{
struct device *dev = &pcie->pdev->dev;
- struct device_node *node = dev->of_node;
struct mobiveil_root_port *rp = &pcie->rp;
/* setup INTx */
- rp->intx_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
- &intx_domain_ops, pcie);
-
+ rp->intx_domain = irq_domain_create_linear(dev_fwnode(dev), PCI_NUM_INTX, &intx_domain_ops,
+ pcie);
if (!rp->intx_domain) {
dev_err(dev, "Failed to get a INTx IRQ domain\n");
return -ENOMEM;
diff --git a/drivers/pci/controller/pci-loongson.c b/drivers/pci/controller/pci-loongson.c
index 9609e6f50b98..d0c643996476 100644
--- a/drivers/pci/controller/pci-loongson.c
+++ b/drivers/pci/controller/pci-loongson.c
@@ -80,6 +80,18 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON,
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON,
DEV_LS7A_LPC, system_bus_quirk);
+static const struct pci_device_id loongson_internal_bridge_devids[] = {
+ { PCI_VDEVICE(LOONGSON, DEV_LS2K_PCIE_PORT0) },
+ { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT0) },
+ { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT1) },
+ { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT2) },
+ { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT3) },
+ { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT4) },
+ { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT5) },
+ { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT6) },
+ { 0, },
+};
+
/*
* Some Loongson PCIe ports have hardware limitations on their Maximum Read
* Request Size. They can't handle anything larger than this. Sane
@@ -92,24 +104,13 @@ static void loongson_set_min_mrrs_quirk(struct pci_dev *pdev)
{
struct pci_bus *bus = pdev->bus;
struct pci_dev *bridge;
- static const struct pci_device_id bridge_devids[] = {
- { PCI_VDEVICE(LOONGSON, DEV_LS2K_PCIE_PORT0) },
- { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT0) },
- { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT1) },
- { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT2) },
- { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT3) },
- { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT4) },
- { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT5) },
- { PCI_VDEVICE(LOONGSON, DEV_LS7A_PCIE_PORT6) },
- { 0, },
- };
/* look for the matching bridge */
while (!pci_is_root_bus(bus)) {
bridge = bus->self;
bus = bus->parent;
- if (pci_match_id(bridge_devids, bridge)) {
+ if (pci_match_id(loongson_internal_bridge_devids, bridge)) {
if (pcie_get_readrq(pdev) > 256) {
pci_info(pdev, "limiting MRRS to 256\n");
pcie_set_readrq(pdev, 256);
@@ -266,11 +267,11 @@ static void __iomem *pci_loongson_map_bus(struct pci_bus *bus,
struct loongson_pci *priv = pci_bus_to_loongson_pci(bus);
/*
- * Do not read more than one device on the bus other than
- * the host bus.
+ * Do not read more than one device on the internal bridges.
*/
if ((priv->data->flags & FLAG_DEV_FIX) && bus->self) {
- if (!pci_is_root_bus(bus) && (device > 0))
+ if (!pci_is_root_bus(bus) && (device > 0) &&
+ pci_match_id(loongson_internal_bridge_devids, bus->self))
return NULL;
}
diff --git a/drivers/pci/controller/pcie-altera-msi.c b/drivers/pci/controller/pcie-altera-msi.c
index e36a6e158d23..ae5a7e708821 100644
--- a/drivers/pci/controller/pcie-altera-msi.c
+++ b/drivers/pci/controller/pcie-altera-msi.c
@@ -164,7 +164,7 @@ static const struct irq_domain_ops msi_domain_ops = {
static int altera_allocate_domains(struct altera_msi *msi)
{
- struct fwnode_handle *fwnode = of_node_to_fwnode(msi->pdev->dev.of_node);
+ struct fwnode_handle *fwnode = dev_fwnode(&msi->pdev->dev);
msi->inner_domain = irq_domain_add_linear(NULL, msi->num_of_vectors,
&msi_domain_ops, msi);
diff --git a/drivers/pci/controller/pcie-altera.c b/drivers/pci/controller/pcie-altera.c
index ba1640641753..8769caa96392 100644
--- a/drivers/pci/controller/pcie-altera.c
+++ b/drivers/pci/controller/pcie-altera.c
@@ -666,10 +666,9 @@ static void altera_pcie_isr(struct irq_desc *desc)
static int altera_pcie_init_irq_domain(struct altera_pcie *pcie)
{
struct device *dev = &pcie->pdev->dev;
- struct device_node *node = dev->of_node;
/* Setup INTx */
- pcie->irq_domain = irq_domain_add_linear(node, PCI_NUM_INTX,
+ pcie->irq_domain = irq_domain_create_linear(dev_fwnode(dev), PCI_NUM_INTX,
&intx_domain_ops, pcie);
if (!pcie->irq_domain) {
dev_err(dev, "Failed to get a INTx IRQ domain\n");
@@ -679,8 +678,18 @@ static int altera_pcie_init_irq_domain(struct altera_pcie *pcie)
return 0;
}
+static void altera_pcie_disable_irq(struct altera_pcie *pcie)
+{
+ if (pcie->pcie_data->version == ALTERA_PCIE_V1 ||
+ pcie->pcie_data->version == ALTERA_PCIE_V2) {
+ /* Disable all P2A interrupts */
+ cra_writel(pcie, 0, P2A_INT_ENABLE);
+ }
+}
+
static void altera_pcie_irq_teardown(struct altera_pcie *pcie)
{
+ altera_pcie_disable_irq(pcie);
irq_set_chained_handler_and_data(pcie->irq, NULL, NULL);
irq_domain_remove(pcie->irq_domain);
}
@@ -705,7 +714,6 @@ static int altera_pcie_parse_dt(struct altera_pcie *pcie)
if (pcie->irq < 0)
return pcie->irq;
- irq_set_chained_handler_and_data(pcie->irq, altera_pcie_isr, pcie);
return 0;
}
@@ -790,6 +798,12 @@ static int altera_pcie_probe(struct platform_device *pdev)
return ret;
}
+ /*
+ * The chained handler uses pcie->irq_domain, so set it only after the
+ * INTx domain has been created.
+ */
+ irq_set_chained_handler_and_data(pcie->irq, altera_pcie_isr, pcie);
+
/* clear all interrupts */
cra_writel(pcie, P2A_INT_STS_ALL, P2A_INT_STATUS);
/* enable all interrupts */
@@ -800,7 +814,16 @@ static int altera_pcie_probe(struct platform_device *pdev)
bridge->busnr = pcie->root_bus_nr;
bridge->ops = &altera_pcie_ops;
- return pci_host_probe(bridge);
+ ret = pci_host_probe(bridge);
+ if (ret)
+ goto err_teardown_irq;
+
+ return 0;
+
+err_teardown_irq:
+ altera_pcie_irq_teardown(pcie);
+
+ return ret;
}
static void altera_pcie_remove(struct platform_device *pdev)
diff --git a/drivers/pci/controller/pcie-iproc-bcma.c b/drivers/pci/controller/pcie-iproc-bcma.c
index 99a99900444d..593418c2bc3a 100644
--- a/drivers/pci/controller/pcie-iproc-bcma.c
+++ b/drivers/pci/controller/pcie-iproc-bcma.c
@@ -64,7 +64,7 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev)
if (ret)
return ret;
- pcie->map_irq = iproc_bcma_pcie_map_irq;
+ bridge->map_irq = iproc_bcma_pcie_map_irq;
bcma_set_drvdata(bdev, pcie);
diff --git a/drivers/pci/controller/pcie-iproc-platform.c b/drivers/pci/controller/pcie-iproc-platform.c
index 4e6aa882a567..de0e812cecf1 100644
--- a/drivers/pci/controller/pcie-iproc-platform.c
+++ b/drivers/pci/controller/pcie-iproc-platform.c
@@ -98,7 +98,7 @@ static int iproc_pltfm_pcie_probe(struct platform_device *pdev)
switch (pcie->type) {
case IPROC_PCIE_PAXC:
case IPROC_PCIE_PAXC_V2:
- pcie->map_irq = NULL;
+ bridge->map_irq = NULL;
break;
default:
break;
diff --git a/drivers/pci/controller/pcie-iproc.c b/drivers/pci/controller/pcie-iproc.c
index 22134e95574b..5aa677f81e4f 100644
--- a/drivers/pci/controller/pcie-iproc.c
+++ b/drivers/pci/controller/pcie-iproc.c
@@ -1514,7 +1514,6 @@ int iproc_pcie_setup(struct iproc_pcie *pcie, struct list_head *res)
host->ops = &iproc_pcie_ops;
host->sysdata = pcie;
- host->map_irq = pcie->map_irq;
ret = pci_host_probe(host);
if (ret < 0) {
diff --git a/drivers/pci/controller/pcie-iproc.h b/drivers/pci/controller/pcie-iproc.h
index 969ded03b8c2..c4443f236ca3 100644
--- a/drivers/pci/controller/pcie-iproc.h
+++ b/drivers/pci/controller/pcie-iproc.h
@@ -61,7 +61,6 @@ struct iproc_msi;
* @base_addr: PCIe host controller register base physical address
* @mem: host bridge memory window resource
* @phy: optional PHY device that controls the Serdes
- * @map_irq: function callback to map interrupts
* @ep_is_internal: indicates an internal emulated endpoint device is connected
* @iproc_cfg_read: indicates the iProc config read function should be used
* @rej_unconfig_pf: indicates the root complex needs to detect and reject
@@ -91,7 +90,6 @@ struct iproc_pcie {
phys_addr_t base_addr;
struct resource mem;
struct phy *phy;
- int (*map_irq)(const struct pci_dev *, u8, u8);
bool ep_is_internal;
bool iproc_cfg_read;
bool rej_unconfig_pf;
diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c
index b373ece9542c..86da2046533a 100644
--- a/drivers/pci/controller/pcie-mediatek-gen3.c
+++ b/drivers/pci/controller/pcie-mediatek-gen3.c
@@ -695,8 +695,8 @@ static int mtk_pcie_init_irq_domains(struct mtk_gen3_pcie *pcie)
/* Setup MSI */
mutex_init(&pcie->lock);
- pcie->msi_bottom_domain = irq_domain_add_linear(node, PCIE_MSI_IRQS_NUM,
- &mtk_msi_bottom_domain_ops, pcie);
+ pcie->msi_bottom_domain = irq_domain_create_linear(dev_fwnode(dev), PCIE_MSI_IRQS_NUM,
+ &mtk_msi_bottom_domain_ops, pcie);
if (!pcie->msi_bottom_domain) {
dev_err(dev, "failed to create MSI bottom domain\n");
ret = -ENODEV;
diff --git a/drivers/pci/controller/pcie-mediatek.c b/drivers/pci/controller/pcie-mediatek.c
index c4cc9d76b42a..18d4a0615bc8 100644
--- a/drivers/pci/controller/pcie-mediatek.c
+++ b/drivers/pci/controller/pcie-mediatek.c
@@ -7,11 +7,13 @@
* Honghui Zhang <honghui.zhang@mediatek.com>
*/
+#include <linux/bitfield.h>
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/iopoll.h>
#include <linux/irq.h>
#include <linux/irqchip/chained_irq.h>
+#include "../../irqchip/irq-msi-lib.h"
#include <linux/irqdomain.h>
#include <linux/kernel.h>
#include <linux/mfd/syscon.h>
@@ -60,7 +62,7 @@
/* MediaTek specific configuration registers */
#define PCIE_FTS_NUM 0x70c
#define PCIE_FTS_NUM_MASK GENMASK(15, 8)
-#define PCIE_FTS_NUM_L0(x) ((x) & 0xff << 8)
+#define PCIE_FTS_NUM_L0(x) FIELD_PREP(PCIE_FTS_NUM_MASK, x)
#define PCIE_FC_CREDIT 0x73c
#define PCIE_FC_CREDIT_MASK (GENMASK(31, 31) | GENMASK(28, 16))
@@ -141,29 +143,38 @@
struct mtk_pcie_port;
+/**
+ * enum mtk_pcie_quirks - MTK PCIe quirks
+ * @MTK_PCIE_FIX_CLASS_ID: host's class ID needed to be fixed
+ * @MTK_PCIE_FIX_DEVICE_ID: host's device ID needed to be fixed
+ * @MTK_PCIE_NO_MSI: Bridge has no MSI support, and relies on an external block
+ */
+enum mtk_pcie_quirks {
+ MTK_PCIE_FIX_CLASS_ID = BIT(0),
+ MTK_PCIE_FIX_DEVICE_ID = BIT(1),
+ MTK_PCIE_NO_MSI = BIT(2),
+};
+
/**
* struct mtk_pcie_soc - differentiate between host generations
- * @need_fix_class_id: whether this host's class ID needed to be fixed or not
- * @need_fix_device_id: whether this host's device ID needed to be fixed or not
- * @no_msi: Bridge has no MSI support, and relies on an external block
* @device_id: device ID which this host need to be fixed
* @ops: pointer to configuration access functions
* @startup: pointer to controller setting functions
* @setup_irq: pointer to initialize IRQ functions
+ * @quirks: PCIe device quirks.
*/
struct mtk_pcie_soc {
- bool need_fix_class_id;
- bool need_fix_device_id;
- bool no_msi;
unsigned int device_id;
struct pci_ops *ops;
int (*startup)(struct mtk_pcie_port *port);
int (*setup_irq)(struct mtk_pcie_port *port, struct device_node *node);
+ enum mtk_pcie_quirks quirks;
};
/**
* struct mtk_pcie_port - PCIe port information
* @base: IO mapped register base
+ * @phys_base: Physical address of the I/O register base region
* @list: port list
* @pcie: pointer to PCIe host info
* @reset: pointer to port reset control
@@ -180,12 +191,12 @@ struct mtk_pcie_soc {
* @irq: GIC irq
* @irq_domain: legacy INTx IRQ domain
* @inner_domain: inner IRQ domain
- * @msi_domain: MSI IRQ domain
* @lock: protect the msi_irq_in_use bitmap
* @msi_irq_in_use: bit map for assigned MSI IRQ
*/
struct mtk_pcie_port {
void __iomem *base;
+ phys_addr_t phys_base;
struct list_head list;
struct mtk_pcie *pcie;
struct reset_control *reset;
@@ -200,7 +211,6 @@ struct mtk_pcie_port {
int irq;
struct irq_domain *irq_domain;
struct irq_domain *inner_domain;
- struct irq_domain *msi_domain;
struct mutex lock;
DECLARE_BITMAP(msi_irq_in_use, MTK_MSI_IRQS_NUM);
};
@@ -396,7 +406,7 @@ static void mtk_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
phys_addr_t addr;
/* MT2712/MT7622 only support 32-bit MSI addresses */
- addr = virt_to_phys(port->base + PCIE_MSI_VECTOR);
+ addr = port->phys_base + PCIE_MSI_VECTOR;
msg->address_hi = 0;
msg->address_lo = lower_32_bits(addr);
@@ -470,39 +480,42 @@ static const struct irq_domain_ops msi_domain_ops = {
.free = mtk_pcie_irq_domain_free,
};
-static struct irq_chip mtk_msi_irq_chip = {
- .name = "MTK PCIe MSI",
- .irq_ack = irq_chip_ack_parent,
- .irq_mask = pci_msi_mask_irq,
- .irq_unmask = pci_msi_unmask_irq,
-};
+#define MTK_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS | \
+ MSI_FLAG_USE_DEF_CHIP_OPS | \
+ MSI_FLAG_NO_AFFINITY)
+
+#define MTK_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK | \
+ MSI_FLAG_PCI_MSIX)
-static struct msi_domain_info mtk_msi_domain_info = {
- .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
- MSI_FLAG_NO_AFFINITY | MSI_FLAG_PCI_MSIX,
- .chip = &mtk_msi_irq_chip,
+static const struct msi_parent_ops mtk_msi_parent_ops = {
+ .required_flags = MTK_MSI_FLAGS_REQUIRED,
+ .supported_flags = MTK_MSI_FLAGS_SUPPORTED,
+ .bus_select_token = DOMAIN_BUS_PCI_MSI,
+ .prefix = "MTK-",
+ .init_dev_msi_info = msi_lib_init_dev_msi_info,
};
static int mtk_pcie_allocate_msi_domains(struct mtk_pcie_port *port)
{
- struct fwnode_handle *fwnode = of_node_to_fwnode(port->pcie->dev->of_node);
-
mutex_init(&port->lock);
- port->inner_domain = irq_domain_create_linear(fwnode, MTK_MSI_IRQS_NUM,
- &msi_domain_ops, port);
- if (!port->inner_domain) {
+ struct irq_domain_info info = {
+ .fwnode = dev_fwnode(port->pcie->dev),
+ .ops = &msi_domain_ops,
+ .host_data = port,
+ .size = MTK_MSI_IRQS_NUM,
+ .hwirq_max = MTK_MSI_IRQS_NUM,
+ .bus_token = DOMAIN_BUS_PCI_MSI,
+ .domain_flags = IRQ_DOMAIN_FLAG_MSI_PARENT,
+ };
+
+ port->inner_domain = irq_domain_instantiate(&info);
+ if (IS_ERR(port->inner_domain)) {
dev_err(port->pcie->dev, "failed to create IRQ domain\n");
+ port->inner_domain = NULL;
return -ENOMEM;
}
-
- port->msi_domain = pci_msi_create_irq_domain(fwnode, &mtk_msi_domain_info,
- port->inner_domain);
- if (!port->msi_domain) {
- dev_err(port->pcie->dev, "failed to create MSI domain\n");
- irq_domain_remove(port->inner_domain);
- return -ENOMEM;
- }
+ port->inner_domain->msi_parent_ops = &mtk_msi_parent_ops;
return 0;
}
@@ -512,7 +525,7 @@ static void mtk_pcie_enable_msi(struct mtk_pcie_port *port)
u32 val;
phys_addr_t msg_addr;
- msg_addr = virt_to_phys(port->base + PCIE_MSI_VECTOR);
+ msg_addr = port->phys_base + PCIE_MSI_VECTOR;
val = lower_32_bits(msg_addr);
writel(val, port->base + PCIE_IMSI_ADDR);
@@ -521,25 +534,27 @@ static void mtk_pcie_enable_msi(struct mtk_pcie_port *port)
writel(val, port->base + PCIE_INT_MASK);
}
-static void mtk_pcie_irq_teardown(struct mtk_pcie *pcie)
+static void mtk_pcie_irq_teardown_port(struct mtk_pcie_port *port)
{
- struct mtk_pcie_port *port, *tmp;
+ irq_set_chained_handler_and_data(port->irq, NULL, NULL);
- list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
- irq_set_chained_handler_and_data(port->irq, NULL, NULL);
+ if (port->irq_domain)
+ irq_domain_remove(port->irq_domain);
- if (port->irq_domain)
- irq_domain_remove(port->irq_domain);
+ if (IS_ENABLED(CONFIG_PCI_MSI)) {
+ if (port->inner_domain)
+ irq_domain_remove(port->inner_domain);
+ }
- if (IS_ENABLED(CONFIG_PCI_MSI)) {
- if (port->msi_domain)
- irq_domain_remove(port->msi_domain);
- if (port->inner_domain)
- irq_domain_remove(port->inner_domain);
- }
+ irq_dispose_mapping(port->irq);
+}
- irq_dispose_mapping(port->irq);
- }
+static void mtk_pcie_irq_teardown(struct mtk_pcie *pcie)
+{
+ struct mtk_pcie_port *port, *tmp;
+
+ list_for_each_entry_safe(port, tmp, &pcie->ports, list)
+ mtk_pcie_irq_teardown_port(port);
}
static int mtk_pcie_intx_map(struct irq_domain *domain, unsigned int irq,
@@ -695,12 +710,7 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
*/
writel(PCIE_LINKDOWN_RST_EN, port->base + PCIE_RST_CTRL);
- /*
- * Described in PCIe CEM specification sections 2.2 (PERST# Signal) and
- * 2.2.1 (Initial Power-Up (G3 to S0)). The deassertion of PERST# should
- * be delayed 100ms (TPVPERL) for the power and clock to become stable.
- */
- msleep(100);
+ msleep(PCIE_T_PVPERL_MS);
/* De-assert PHY, PE, PIPE, MAC and configuration reset */
val = readl(port->base + PCIE_RST_CTRL);
@@ -709,7 +719,7 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
writel(val, port->base + PCIE_RST_CTRL);
/* Set up vendor ID and class code */
- if (soc->need_fix_class_id) {
+ if (soc->quirks & MTK_PCIE_FIX_CLASS_ID) {
val = PCI_VENDOR_ID_MEDIATEK;
writew(val, port->base + PCIE_CONF_VEND_ID);
@@ -717,7 +727,7 @@ static int mtk_pcie_startup_port_v2(struct mtk_pcie_port *port)
writew(val, port->base + PCIE_CONF_CLASS_ID);
}
- if (soc->need_fix_device_id)
+ if (soc->quirks & MTK_PCIE_FIX_DEVICE_ID)
writew(soc->device_id, port->base + PCIE_CONF_DEVICE_ID);
/* 100ms timeout value should be enough for Gen1/2 training */
@@ -827,7 +837,7 @@ static int mtk_pcie_startup_port(struct mtk_pcie_port *port)
return 0;
}
-static void mtk_pcie_enable_port(struct mtk_pcie_port *port)
+static int mtk_pcie_enable_port(struct mtk_pcie_port *port)
{
struct mtk_pcie *pcie = port->pcie;
struct device *dev = pcie->dev;
@@ -836,7 +846,7 @@ static void mtk_pcie_enable_port(struct mtk_pcie_port *port)
err = clk_prepare_enable(port->sys_ck);
if (err) {
dev_err(dev, "failed to enable sys_ck%d clock\n", port->slot);
- goto err_sys_clk;
+ return err;
}
err = clk_prepare_enable(port->ahb_ck);
@@ -884,11 +894,15 @@ static void mtk_pcie_enable_port(struct mtk_pcie_port *port)
goto err_phy_on;
}
- if (!pcie->soc->startup(port))
- return;
+ err = pcie->soc->startup(port);
+ if (err) {
+ dev_info(dev, "Port%d link down\n", port->slot);
+ goto err_soc_startup;
+ }
- dev_info(dev, "Port%d link down\n", port->slot);
+ return 0;
+err_soc_startup:
phy_power_off(port->phy);
err_phy_on:
phy_exit(port->phy);
@@ -904,8 +918,8 @@ static void mtk_pcie_enable_port(struct mtk_pcie_port *port)
clk_disable_unprepare(port->ahb_ck);
err_ahb_clk:
clk_disable_unprepare(port->sys_ck);
-err_sys_clk:
- mtk_pcie_port_free(port);
+
+ return err;
}
static int mtk_pcie_parse_port(struct mtk_pcie *pcie,
@@ -915,7 +929,8 @@ static int mtk_pcie_parse_port(struct mtk_pcie *pcie,
struct mtk_pcie_port *port;
struct device *dev = pcie->dev;
struct platform_device *pdev = to_platform_device(dev);
- char name[10];
+ struct resource *res;
+ char name[20];
int err;
port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL);
@@ -923,7 +938,14 @@ static int mtk_pcie_parse_port(struct mtk_pcie *pcie,
return -ENOMEM;
snprintf(name, sizeof(name), "port%d", slot);
- port->base = devm_platform_ioremap_resource_byname(pdev, name);
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, name);
+ if (!res) {
+ dev_err(dev, "failed to get port%d base\n", slot);
+ return -EINVAL;
+ }
+
+ port->phys_base = res->start;
+ port->base = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(port->base)) {
dev_err(dev, "failed to map port%d base\n", slot);
return PTR_ERR(port->base);
@@ -1073,8 +1095,13 @@ static int mtk_pcie_setup(struct mtk_pcie *pcie)
return err;
/* enable each port, and then check link status */
- list_for_each_entry_safe(port, tmp, &pcie->ports, list)
- mtk_pcie_enable_port(port);
+ list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
+ err = mtk_pcie_enable_port(port);
+ if (err) {
+ mtk_pcie_irq_teardown_port(port);
+ mtk_pcie_port_free(port);
+ }
+ }
/* power down PCIe subsys if slots are all empty (link down) */
if (list_empty(&pcie->ports))
@@ -1110,7 +1137,7 @@ static int mtk_pcie_probe(struct platform_device *pdev)
host->ops = pcie->soc->ops;
host->sysdata = pcie;
- host->msi_domain = pcie->soc->no_msi;
+ host->msi_domain = !!(pcie->soc->quirks & MTK_PCIE_NO_MSI);
err = pci_host_probe(host);
if (err)
@@ -1176,14 +1203,18 @@ static int mtk_pcie_resume_noirq(struct device *dev)
{
struct mtk_pcie *pcie = dev_get_drvdata(dev);
struct mtk_pcie_port *port, *tmp;
+ int err;
if (list_empty(&pcie->ports))
return 0;
clk_prepare_enable(pcie->free_ck);
- list_for_each_entry_safe(port, tmp, &pcie->ports, list)
- mtk_pcie_enable_port(port);
+ list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
+ err = mtk_pcie_enable_port(port);
+ if (err)
+ mtk_pcie_port_free(port);
+ }
/* In case of EP was removed while system suspend. */
if (list_empty(&pcie->ports))
@@ -1198,9 +1229,9 @@ static const struct dev_pm_ops mtk_pcie_pm_ops = {
};
static const struct mtk_pcie_soc mtk_pcie_soc_v1 = {
- .no_msi = true,
.ops = &mtk_pcie_ops,
.startup = mtk_pcie_startup_port,
+ .quirks = MTK_PCIE_NO_MSI,
};
static const struct mtk_pcie_soc mtk_pcie_soc_mt2712 = {
@@ -1210,19 +1241,18 @@ static const struct mtk_pcie_soc mtk_pcie_soc_mt2712 = {
};
static const struct mtk_pcie_soc mtk_pcie_soc_mt7622 = {
- .need_fix_class_id = true,
.ops = &mtk_pcie_ops_v2,
.startup = mtk_pcie_startup_port_v2,
.setup_irq = mtk_pcie_setup_irq,
+ .quirks = MTK_PCIE_FIX_CLASS_ID,
};
static const struct mtk_pcie_soc mtk_pcie_soc_mt7629 = {
- .need_fix_class_id = true,
- .need_fix_device_id = true,
.device_id = PCI_DEVICE_ID_MEDIATEK_7629,
.ops = &mtk_pcie_ops_v2,
.startup = mtk_pcie_startup_port_v2,
.setup_irq = mtk_pcie_setup_irq,
+ .quirks = MTK_PCIE_FIX_CLASS_ID | MTK_PCIE_FIX_DEVICE_ID,
};
static const struct of_device_id mtk_pcie_ids[] = {
diff --git a/drivers/pci/controller/pcie-rcar-host.c b/drivers/pci/controller/pcie-rcar-host.c
index a90beedd8e24..4ecb41a0e3b3 100644
--- a/drivers/pci/controller/pcie-rcar-host.c
+++ b/drivers/pci/controller/pcie-rcar-host.c
@@ -345,7 +345,6 @@ static void rcar_pcie_hw_enable(struct rcar_pcie_host *host)
struct rcar_pcie *pcie = &host->pcie;
struct pci_host_bridge *bridge = pci_host_bridge_from_priv(host);
struct resource_entry *win;
- LIST_HEAD(res);
int i = 0;
/* Try setting 5 GT/s link speed */
diff --git a/drivers/pci/controller/pcie-xilinx-dma-pl.c b/drivers/pci/controller/pcie-xilinx-dma-pl.c
index dd117f07fc95..116c6e0e7145 100644
--- a/drivers/pci/controller/pcie-xilinx-dma-pl.c
+++ b/drivers/pci/controller/pcie-xilinx-dma-pl.c
@@ -470,7 +470,7 @@ static int xilinx_pl_dma_pcie_init_msi_irq_domain(struct pl_dma_pcie *port)
struct device *dev = port->dev;
struct xilinx_msi *msi = &port->msi;
int size = BITS_TO_LONGS(XILINX_NUM_MSI_IRQS) * sizeof(long);
- struct fwnode_handle *fwnode = of_node_to_fwnode(port->dev->of_node);
+ struct fwnode_handle *fwnode = dev_fwnode(port->dev);
msi->dev_domain = irq_domain_add_linear(NULL, XILINX_NUM_MSI_IRQS,
&dev_msi_domain_ops, port);
diff --git a/drivers/pci/controller/pcie-xilinx-nwl.c b/drivers/pci/controller/pcie-xilinx-nwl.c
index 2e7356c28b5e..4f01602ad9f1 100644
--- a/drivers/pci/controller/pcie-xilinx-nwl.c
+++ b/drivers/pci/controller/pcie-xilinx-nwl.c
@@ -495,7 +495,7 @@ static int nwl_pcie_init_msi_irq_domain(struct nwl_pcie *pcie)
{
#ifdef CONFIG_PCI_MSI
struct device *dev = pcie->dev;
- struct fwnode_handle *fwnode = of_node_to_fwnode(dev->of_node);
+ struct fwnode_handle *fwnode = dev_fwnode(dev);
struct nwl_msi *msi = &pcie->msi;
msi->dev_domain = irq_domain_add_linear(NULL, INT_PCI_MSI_NR,
diff --git a/drivers/pci/controller/plda/pcie-plda-host.c b/drivers/pci/controller/plda/pcie-plda-host.c
index 4153214ca410..d2bdc774f0c3 100644
--- a/drivers/pci/controller/plda/pcie-plda-host.c
+++ b/drivers/pci/controller/plda/pcie-plda-host.c
@@ -150,7 +150,7 @@ static struct msi_domain_info plda_msi_domain_info = {
static int plda_allocate_msi_domains(struct plda_pcie_rp *port)
{
struct device *dev = port->dev;
- struct fwnode_handle *fwnode = of_node_to_fwnode(dev->of_node);
+ struct fwnode_handle *fwnode = dev_fwnode(dev);
struct plda_msi *msi = &port->msi;
mutex_init(&port->msi.lock);
diff --git a/drivers/pci/endpoint/functions/pci-epf-ntb.c b/drivers/pci/endpoint/functions/pci-epf-ntb.c
index 7702ebb81d99..d12cf226f328 100644
--- a/drivers/pci/endpoint/functions/pci-epf-ntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-ntb.c
@@ -559,12 +559,15 @@ static int epf_ntb_configure_db(struct epf_ntb *ntb,
struct pci_epc *epc;
int ret;
- if (db_count > MAX_DB_COUNT)
- return -EINVAL;
-
ntb_epc = ntb->epc[type];
epc = ntb_epc->epc;
+ if (!db_count || db_count > MAX_DB_COUNT) {
+ dev_err(&epc->dev, "DB count %d out of range (1 - %d)\n",
+ db_count, MAX_DB_COUNT);
+ return -EINVAL;
+ }
+
if (msix)
ret = epf_ntb_configure_msix(ntb, type, db_count);
else
@@ -1278,7 +1281,6 @@ static int epf_ntb_configure_interrupt(struct epf_ntb *ntb,
u8 func_no, vfunc_no;
struct pci_epc *epc;
struct device *dev;
- u32 db_count;
int ret;
ntb_epc = ntb->epc[type];
@@ -1296,17 +1298,16 @@ static int epf_ntb_configure_interrupt(struct epf_ntb *ntb,
func_no = ntb_epc->func_no;
vfunc_no = ntb_epc->vfunc_no;
- db_count = ntb->db_count;
- if (db_count > MAX_DB_COUNT) {
- dev_err(dev, "DB count cannot be more than %d\n", MAX_DB_COUNT);
+ if (!ntb->db_count || ntb->db_count > MAX_DB_COUNT) {
+ dev_err(dev, "DB count %d out of range (1 - %d)\n",
+ ntb->db_count, MAX_DB_COUNT);
return -EINVAL;
}
- ntb->db_count = db_count;
epc = ntb_epc->epc;
if (msi_capable) {
- ret = pci_epc_set_msi(epc, func_no, vfunc_no, db_count);
+ ret = pci_epc_set_msi(epc, func_no, vfunc_no, ntb->db_count);
if (ret) {
dev_err(dev, "%s intf: MSI configuration failed\n",
pci_epc_interface_string(type));
@@ -1315,7 +1316,7 @@ static int epf_ntb_configure_interrupt(struct epf_ntb *ntb,
}
if (msix_capable) {
- ret = pci_epc_set_msix(epc, func_no, vfunc_no, db_count,
+ ret = pci_epc_set_msix(epc, func_no, vfunc_no, ntb->db_count,
ntb_epc->msix_bar,
ntb_epc->msix_table_offset);
if (ret) {
diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
index 6875af691b2d..f9786637d21c 100644
--- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
@@ -484,7 +484,6 @@ static int epf_ntb_configure_interrupt(struct epf_ntb *ntb)
{
const struct pci_epc_features *epc_features;
struct device *dev;
- u32 db_count;
int ret;
dev = &ntb->epf->dev;
@@ -496,14 +495,12 @@ static int epf_ntb_configure_interrupt(struct epf_ntb *ntb)
return -EINVAL;
}
- db_count = ntb->db_count;
- if (db_count > MAX_DB_COUNT) {
- dev_err(dev, "DB count cannot be more than %d\n", MAX_DB_COUNT);
+ if (!ntb->db_count || ntb->db_count > MAX_DB_COUNT) {
+ dev_err(dev, "DB count %d out of range (1 - %d)\n",
+ ntb->db_count, MAX_DB_COUNT);
return -EINVAL;
}
- ntb->db_count = db_count;
-
if (epc_features->msi_capable) {
ret = pci_epc_set_msi(ntb->epf->epc,
ntb->epf->func_no,
diff --git a/drivers/pci/msi/msi.c b/drivers/pci/msi/msi.c
index 54627c3a2dc8..8b8848788618 100644
--- a/drivers/pci/msi/msi.c
+++ b/drivers/pci/msi/msi.c
@@ -739,7 +739,7 @@ static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries,
ret = msix_setup_interrupts(dev, entries, nvec, affd);
if (ret)
- goto out_unmap;
+ goto out_disable;
/* Disable INTX */
pci_intx_for_msi(dev, 0);
@@ -760,8 +760,6 @@ static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries,
pcibios_free_irq(dev);
return 0;
-out_unmap:
- iounmap(dev->msix_base);
out_disable:
dev->msix_enabled = 0;
pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_MASKALL | PCI_MSIX_FLAGS_ENABLE, 0);
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 122c182229b3..db6407ccd144 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -1426,8 +1426,9 @@ static ssize_t __resource_resize_store(struct device *dev, int n,
const char *buf, size_t count)
{
struct pci_dev *pdev = to_pci_dev(dev);
- unsigned long size, flags;
- int ret, i;
+ struct pci_bus *bus = pdev->bus;
+ unsigned long size;
+ int ret;
u16 cmd;
if (kstrtoul(buf, 0, &size) < 0)
@@ -1452,19 +1453,11 @@ static ssize_t __resource_resize_store(struct device *dev, int n,
pci_write_config_word(pdev, PCI_COMMAND,
cmd & ~PCI_COMMAND_MEMORY);
- flags = pci_resource_flags(pdev, n);
-
pci_remove_resource_files(pdev);
- for (i = 0; i < PCI_BRIDGE_RESOURCES; i++) {
- if (pci_resource_len(pdev, i) &&
- pci_resource_flags(pdev, i) == flags)
- pci_release_resource(pdev, i);
- }
-
- ret = pci_resize_resource(pdev, n, size);
+ ret = pci_resize_resource(pdev, n, size, 0);
- pci_assign_unassigned_bus_resources(pdev->bus);
+ pci_assign_unassigned_bus_resources(bus);
if (pci_create_resource_files(pdev))
pci_warn(pdev, "Failed to recreate resource files after BAR resizing\n");
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index d44c07bb0a22..451ee220f954 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1604,9 +1604,6 @@ static int pci_set_low_power_state(struct pci_dev *dev, pci_power_t state, bool
pci_power_name(dev->current_state),
pci_power_name(state));
- if (dev->bus->self)
- pcie_aspm_pm_state_change(dev->bus->self, locked);
-
return 0;
}
@@ -1910,32 +1907,6 @@ static void pci_restore_config_space(struct pci_dev *pdev)
}
}
-static void pci_restore_rebar_state(struct pci_dev *pdev)
-{
- unsigned int pos, nbars, i;
- u32 ctrl;
-
- pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
- if (!pos)
- return;
-
- pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
- nbars = FIELD_GET(PCI_REBAR_CTRL_NBAR_MASK, ctrl);
-
- for (i = 0; i < nbars; i++, pos += 8) {
- struct resource *res;
- int bar_idx, size;
-
- pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
- bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
- res = pdev->resource + bar_idx;
- size = pci_rebar_bytes_to_size(resource_size(res));
- ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
- ctrl |= FIELD_PREP(PCI_REBAR_CTRL_BAR_SIZE, size);
- pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
- }
-}
-
/**
* pci_restore_state - Restore the saved state of a PCI device
* @dev: PCI device that we're dealing with
@@ -3727,114 +3698,6 @@ void pci_acs_init(struct pci_dev *dev)
pci_enable_acs(dev);
}
-/**
- * pci_rebar_find_pos - find position of resize ctrl reg for BAR
- * @pdev: PCI device
- * @bar: BAR to find
- *
- * Helper to find the position of the ctrl register for a BAR.
- * Returns -ENOTSUPP if resizable BARs are not supported at all.
- * Returns -ENOENT if no ctrl register for the BAR could be found.
- */
-static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
-{
- unsigned int pos, nbars, i;
- u32 ctrl;
-
- pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
- if (!pos)
- return -ENOTSUPP;
-
- pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
- nbars = FIELD_GET(PCI_REBAR_CTRL_NBAR_MASK, ctrl);
-
- for (i = 0; i < nbars; i++, pos += 8) {
- int bar_idx;
-
- pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
- bar_idx = FIELD_GET(PCI_REBAR_CTRL_BAR_IDX, ctrl);
- if (bar_idx == bar)
- return pos;
- }
-
- return -ENOENT;
-}
-
-/**
- * pci_rebar_get_possible_sizes - get possible sizes for BAR
- * @pdev: PCI device
- * @bar: BAR to query
- *
- * Get the possible sizes of a resizable BAR as bitmask defined in the spec
- * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
- */
-u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
-{
- int pos;
- u32 cap;
-
- pos = pci_rebar_find_pos(pdev, bar);
- if (pos < 0)
- return 0;
-
- pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap);
- cap = FIELD_GET(PCI_REBAR_CAP_SIZES, cap);
-
- /* Sapphire RX 5600 XT Pulse has an invalid cap dword for BAR 0 */
- if (pdev->vendor == PCI_VENDOR_ID_ATI && pdev->device == 0x731f &&
- bar == 0 && cap == 0x700)
- return 0x3f00;
-
- return cap;
-}
-EXPORT_SYMBOL(pci_rebar_get_possible_sizes);
-
-/**
- * pci_rebar_get_current_size - get the current size of a BAR
- * @pdev: PCI device
- * @bar: BAR to set size to
- *
- * Read the size of a BAR from the resizable BAR config.
- * Returns size if found or negative error code.
- */
-int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
-{
- int pos;
- u32 ctrl;
-
- pos = pci_rebar_find_pos(pdev, bar);
- if (pos < 0)
- return pos;
-
- pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
- return FIELD_GET(PCI_REBAR_CTRL_BAR_SIZE, ctrl);
-}
-
-/**
- * pci_rebar_set_size - set a new size for a BAR
- * @pdev: PCI device
- * @bar: BAR to set size to
- * @size: new size as defined in the spec (0=1MB, 19=512GB)
- *
- * Set the new size of a BAR as defined in the spec.
- * Returns zero if resizing was successful, error code otherwise.
- */
-int pci_rebar_set_size(struct pci_dev *pdev, int bar, int size)
-{
- int pos;
- u32 ctrl;
-
- pos = pci_rebar_find_pos(pdev, bar);
- if (pos < 0)
- return pos;
-
- pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
- ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
- ctrl |= FIELD_PREP(PCI_REBAR_CTRL_BAR_SIZE, size);
- pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
- return 0;
-}
-
/**
* pci_enable_atomic_ops_to_root - enable AtomicOp requests to root port
* @dev: the PCI device
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index b1f393a42a87..cdc0c78b36f5 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -330,6 +330,9 @@ enum pci_bar_type {
struct device *pci_get_host_bridge_device(struct pci_dev *dev);
void pci_put_host_bridge_device(struct device *dev);
+int pci_do_resource_release_and_resize(struct pci_dev *dev, int resno, int size,
+ int exclude_bars);
+
int pci_configure_extended_tags(struct pci_dev *dev, void *ign);
bool pci_bus_read_dev_vendor_id(struct pci_bus *bus, int devfn, u32 *pl,
int rrs_timeout);
@@ -748,6 +751,7 @@ static inline int acpi_get_rc_resources(struct device *dev, const char *hid,
}
#endif
+void pci_restore_rebar_state(struct pci_dev *pdev);
int pci_rebar_get_current_size(struct pci_dev *pdev, int bar);
int pci_rebar_set_size(struct pci_dev *pdev, int bar, int size);
static inline u64 pci_rebar_size_to_bytes(int size)
diff --git a/drivers/pci/rebar.c b/drivers/pci/rebar.c
new file mode 100644
index 000000000000..548894a3fad5
--- /dev/null
+++ b/drivers/pci/rebar.c
@@ -0,0 +1,197 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCI Resizable BAR Extended Capability handling.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/errno.h>
+#include <linux/export.h>
+#include <linux/ioport.h>
+#include <linux/pci.h>
+#include <linux/types.h>
+
+#include "pci.h"
+
+/**
+ * pci_rebar_find_pos - find position of resize ctrl reg for BAR
+ * @pdev: PCI device
+ * @bar: BAR to find
+ *
+ * Helper to find the position of the ctrl register for a BAR.
+ * Returns -ENOTSUPP if resizable BARs are not supported at all.
+ * Returns -ENOENT if no ctrl register for the BAR could be found.
+ */
+static int pci_rebar_find_pos(struct pci_dev *pdev, int bar)
+{
+ unsigned int pos, nbars, i;
+ u32 ctrl;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return -ENOTSUPP;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ nbars = FIELD_GET(PCI_REBAR_CTRL_NBAR_MASK, ctrl);
+
+ for (i = 0; i < nbars; i++, pos += 8) {
+ int bar_idx;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ bar_idx = FIELD_GET(PCI_REBAR_CTRL_BAR_IDX, ctrl);
+ if (bar_idx == bar)
+ return pos;
+ }
+
+ return -ENOENT;
+}
+
+/**
+ * pci_rebar_get_possible_sizes - get possible sizes for BAR
+ * @pdev: PCI device
+ * @bar: BAR to query
+ *
+ * Get the possible sizes of a resizable BAR as bitmask defined in the spec
+ * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizable.
+ */
+u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar)
+{
+ int pos;
+ u32 cap;
+
+ pos = pci_rebar_find_pos(pdev, bar);
+ if (pos < 0)
+ return 0;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap);
+ cap = FIELD_GET(PCI_REBAR_CAP_SIZES, cap);
+
+ /* Sapphire RX 5600 XT Pulse has an invalid cap dword for BAR 0 */
+ if (pdev->vendor == PCI_VENDOR_ID_ATI && pdev->device == 0x731f &&
+ bar == 0 && cap == 0x700)
+ return 0x3f00;
+
+ return cap;
+}
+EXPORT_SYMBOL(pci_rebar_get_possible_sizes);
+
+/**
+ * pci_rebar_get_current_size - get the current size of a BAR
+ * @pdev: PCI device
+ * @bar: BAR to set size to
+ *
+ * Read the size of a BAR from the resizable BAR config.
+ * Returns size if found or negative error code.
+ */
+int pci_rebar_get_current_size(struct pci_dev *pdev, int bar)
+{
+ int pos;
+ u32 ctrl;
+
+ pos = pci_rebar_find_pos(pdev, bar);
+ if (pos < 0)
+ return pos;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ return FIELD_GET(PCI_REBAR_CTRL_BAR_SIZE, ctrl);
+}
+
+/**
+ * pci_rebar_set_size - set a new size for a BAR
+ * @pdev: PCI device
+ * @bar: BAR to set size to
+ * @size: new size as defined in the spec (0=1MB, 19=512GB)
+ *
+ * Set the new size of a BAR as defined in the spec.
+ * Returns zero if resizing was successful, error code otherwise.
+ */
+int pci_rebar_set_size(struct pci_dev *pdev, int bar, int size)
+{
+ int pos;
+ u32 ctrl;
+
+ pos = pci_rebar_find_pos(pdev, bar);
+ if (pos < 0)
+ return pos;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
+ ctrl |= FIELD_PREP(PCI_REBAR_CTRL_BAR_SIZE, size);
+ pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
+ return 0;
+}
+
+void pci_restore_rebar_state(struct pci_dev *pdev)
+{
+ unsigned int pos, nbars, i;
+ u32 ctrl;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ if (PCI_POSSIBLE_ERROR(ctrl))
+ return;
+
+ nbars = FIELD_GET(PCI_REBAR_CTRL_NBAR_MASK, ctrl);
+
+ for (i = 0; i < nbars; i++, pos += 8) {
+ struct resource *res;
+ int bar_idx, size;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ if (PCI_POSSIBLE_ERROR(ctrl))
+ return;
+
+ bar_idx = ctrl & PCI_REBAR_CTRL_BAR_IDX;
+ res = pci_resource_n(pdev, bar_idx);
+ size = pci_rebar_bytes_to_size(resource_size(res));
+ ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE;
+ ctrl |= FIELD_PREP(PCI_REBAR_CTRL_BAR_SIZE, size);
+ pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
+ }
+}
+
+int pci_resize_resource(struct pci_dev *dev, int resno, int size,
+ int exclude_bars)
+{
+ struct pci_host_bridge *host;
+ int old, ret;
+ u32 sizes;
+ u16 cmd;
+
+ /* Check if we must preserve the firmware's resource assignment */
+ host = pci_find_host_bridge(dev->bus);
+ if (host->preserve_config)
+ return -ENOTSUPP;
+
+ pci_read_config_word(dev, PCI_COMMAND, &cmd);
+ if (cmd & PCI_COMMAND_MEMORY)
+ return -EBUSY;
+
+ sizes = pci_rebar_get_possible_sizes(dev, resno);
+ if (!sizes)
+ return -ENOTSUPP;
+
+ if (!(sizes & BIT(size)))
+ return -EINVAL;
+
+ old = pci_rebar_get_current_size(dev, resno);
+ if (old < 0)
+ return old;
+
+ ret = pci_rebar_set_size(dev, resno, size);
+ if (ret)
+ return ret;
+
+ ret = pci_do_resource_release_and_resize(dev, resno, size, exclude_bars);
+ if (ret)
+ goto error_resize;
+
+ return 0;
+
+error_resize:
+ pci_rebar_set_size(dev, resno, old);
+ return ret;
+}
+EXPORT_SYMBOL(pci_resize_resource);
diff --git a/drivers/pci/rom.c b/drivers/pci/rom.c
index e18d3a4383ba..f2105c6ceef5 100644
--- a/drivers/pci/rom.c
+++ b/drivers/pci/rom.c
@@ -5,13 +5,40 @@
* (C) Copyright 2004 Jon Smirl <jonsmirl@yahoo.com>
* (C) Copyright 2004 Silicon Graphics, Inc. Jesse Barnes <jbarnes@sgi.com>
*/
+
+#include <linux/align.h>
+#include <linux/bits.h>
#include <linux/kernel.h>
#include <linux/export.h>
+#include <linux/io.h>
+#include <linux/overflow.h>
#include <linux/pci.h>
+#include <linux/sizes.h>
#include <linux/slab.h>
#include "pci.h"
+#define PCI_ROM_HEADER_SIZE 0x1A
+#define PCI_ROM_POINTER_TO_DATA_STRUCT 0x18
+#define PCI_ROM_LAST_IMAGE_INDICATOR 0x15
+#define PCI_ROM_LAST_IMAGE_INDICATOR_BIT BIT(7)
+#define PCI_ROM_IMAGE_LEN 0x10
+#define PCI_ROM_IMAGE_SECTOR_SIZE SZ_512
+#define PCI_ROM_IMAGE_SIGNATURE 0xAA55
+
+/* Data structure signature is "PCIR" in ASCII representation */
+#define PCI_ROM_DATA_STRUCT_SIGNATURE 0x52494350
+#define PCI_ROM_DATA_STRUCT_LEN 0x0A
+
+/*
+ * Per PCI Firmware r3.3, sec 5.1.3, a conformant PCI Data Structure is at
+ * least 24 bytes (0x18), large enough to cover every fixed field this
+ * driver reads (up to the Indicator byte at offset 0x15). Reject smaller
+ * device-claimed lengths so the follow-up readers in pci_get_rom_size()
+ * cannot escape the mapped ROM window.
+ */
+#define PCI_ROM_DATA_STRUCT_MIN_LEN 0x18
+
/**
* pci_enable_rom - enable ROM decoding for a PCI device
* @pdev: PCI device to enable
@@ -69,6 +96,91 @@ void pci_disable_rom(struct pci_dev *pdev)
}
EXPORT_SYMBOL_GPL(pci_disable_rom);
+static bool pci_rom_header_valid(struct pci_dev *pdev, void __iomem *image,
+ void __iomem *rom, size_t size,
+ bool expect_valid)
+{
+ unsigned long rom_end = (unsigned long)rom + size - 1;
+ unsigned long header_end;
+ u16 signature;
+
+ /*
+ * Per PCI Firmware r3.3, sec 5.1, each image must start on a
+ * 512-byte boundary and must contain the PCI Expansion ROM header.
+ * Because @rom is page-aligned (returned by ioremap()), checking
+ * 512-byte alignment of @image is equivalent to enforcing the
+ * spec's sector-aligned layout within the ROM. This also
+ * satisfies the natural-alignment requirement of readw() on archs
+ * such as arm64 that disallow unaligned IOMEM access.
+ */
+ if (!IS_ALIGNED((unsigned long)image, PCI_ROM_IMAGE_SECTOR_SIZE))
+ return false;
+
+ if (check_add_overflow((unsigned long)image, PCI_ROM_HEADER_SIZE - 1,
+ &header_end))
+ return false;
+
+ if (image < rom || header_end > rom_end)
+ return false;
+
+ /* Standard PCI ROMs start out with these bytes 55 AA */
+ signature = readw(image);
+ if (signature != PCI_ROM_IMAGE_SIGNATURE) {
+ if (expect_valid) {
+ pci_info(pdev, "Invalid PCI ROM header signature: expecting %#06x, got %#06x\n",
+ PCI_ROM_IMAGE_SIGNATURE, signature);
+ } else {
+ pci_info(pdev, "No more images in PCI ROM\n");
+ }
+ return false;
+ }
+
+ return true;
+}
+
+static bool pci_rom_data_struct_valid(struct pci_dev *pdev, void __iomem *pds,
+ void __iomem *rom, size_t size)
+{
+ unsigned long rom_end = (unsigned long)rom + size - 1;
+ unsigned long end;
+ u32 signature;
+ u16 data_len;
+
+ /*
+ * Some CPU architectures require IOMEM access addresses to be
+ * aligned, for example arm64, so since we're about to call
+ * readl(), check here for 4-byte alignment.
+ */
+ if (!IS_ALIGNED((unsigned long)pds, 4))
+ return false;
+
+ if (check_add_overflow((unsigned long)pds, PCI_ROM_DATA_STRUCT_LEN + 1,
+ &end))
+ return false;
+
+ if (pds < rom || end > rom_end)
+ return false;
+
+ signature = readl(pds);
+ if (signature != PCI_ROM_DATA_STRUCT_SIGNATURE) {
+ pci_info(pdev, "Invalid PCI ROM data signature: expecting %#010x, got %#010x\n",
+ PCI_ROM_DATA_STRUCT_SIGNATURE, signature);
+ return false;
+ }
+
+ data_len = readw(pds + PCI_ROM_DATA_STRUCT_LEN);
+ if (data_len < PCI_ROM_DATA_STRUCT_MIN_LEN || data_len == U16_MAX)
+ return false;
+
+ if (check_add_overflow((unsigned long)pds, data_len - 1, &end))
+ return false;
+
+ if (end > rom_end)
+ return false;
+
+ return true;
+}
+
/**
* pci_get_rom_size - obtain the actual size of the ROM image
* @pdev: target PCI device
@@ -84,37 +196,28 @@ static size_t pci_get_rom_size(struct pci_dev *pdev, void __iomem *rom,
size_t size)
{
void __iomem *image;
- int last_image;
unsigned int length;
+ bool last_image;
image = rom;
do {
void __iomem *pds;
- /* Standard PCI ROMs start out with these bytes 55 AA */
- if (readw(image) != 0xAA55) {
- pci_info(pdev, "Invalid PCI ROM header signature: expecting 0xaa55, got %#06x\n",
- readw(image));
+ if (!pci_rom_header_valid(pdev, image, rom, size, true))
break;
- }
- /* get the PCI data structure and check its "PCIR" signature */
- pds = image + readw(image + 24);
- if (readl(pds) != 0x52494350) {
- pci_info(pdev, "Invalid PCI ROM data signature: expecting 0x52494350, got %#010x\n",
- readl(pds));
+
+ /* Get the PCI data structure and check its "PCIR" signature */
+ pds = image + readw(image + PCI_ROM_POINTER_TO_DATA_STRUCT);
+ if (!pci_rom_data_struct_valid(pdev, pds, rom, size))
break;
- }
- last_image = readb(pds + 21) & 0x80;
- length = readw(pds + 16);
- image += length * 512;
- /* Avoid iterating through memory outside the resource window */
- if (image >= rom + size)
+
+ last_image = readb(pds + PCI_ROM_LAST_IMAGE_INDICATOR) &
+ PCI_ROM_LAST_IMAGE_INDICATOR_BIT;
+ length = readw(pds + PCI_ROM_IMAGE_LEN);
+ image += length * PCI_ROM_IMAGE_SECTOR_SIZE;
+
+ if (!last_image &&
+ !pci_rom_header_valid(pdev, image, rom, size, false))
break;
- if (!last_image) {
- if (readw(image) != 0xAA55) {
- pci_info(pdev, "No more image in the PCI ROM\n");
- break;
- }
- }
} while (length && !last_image);
/* never return a size larger than the PCI resource window */
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 1eceabef9e84..576c6112790c 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2297,18 +2297,16 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
}
EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
-int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
+static int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type,
+ struct list_head *saved)
{
struct pci_dev_resource *dev_res;
struct pci_dev *next;
- LIST_HEAD(saved);
LIST_HEAD(added);
LIST_HEAD(failed);
unsigned int i;
int ret;
- down_read(&pci_bus_sem);
-
/* Walk to the root hub, releasing bridge BARs when possible */
next = bridge;
do {
@@ -2325,9 +2323,9 @@ int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
if (res->child)
continue;
- ret = add_to_list(&saved, bridge, res, 0, 0);
+ ret = add_to_list(saved, bridge, res, 0, 0);
if (ret)
- goto cleanup;
+ return ret;
pci_info(bridge, "%s %pR: releasing\n", res_name, res);
@@ -2343,62 +2341,108 @@ int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
next = bridge->bus ? bridge->bus->self : NULL;
} while (next);
- if (list_empty(&saved)) {
- up_read(&pci_bus_sem);
+ if (list_empty(saved))
return -ENOENT;
- }
__pci_bus_size_bridges(bridge->subordinate, &added);
__pci_bridge_assign_resources(bridge, &added, &failed);
BUG_ON(!list_empty(&added));
if (!list_empty(&failed)) {
- ret = -ENOSPC;
- goto cleanup;
+ free_list(&failed);
+ return -ENOSPC;
}
- list_for_each_entry(dev_res, &saved, list) {
+ list_for_each_entry(dev_res, saved, list) {
/* Skip the bridge we just assigned resources for */
if (bridge == dev_res->dev)
continue;
+ if (!dev_res->dev->subordinate)
+ continue;
+
bridge = dev_res->dev;
pci_setup_bridge(bridge->subordinate);
}
- free_list(&saved);
- up_read(&pci_bus_sem);
return 0;
+}
-cleanup:
- /* Restore size and flags */
- list_for_each_entry(dev_res, &failed, list) {
- struct resource *res = dev_res->res;
+int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size,
+ int exclude_bars)
+{
+ struct resource *res = pdev->resource + resno;
+ unsigned long flags = res->flags;
+ struct pci_dev_resource *dev_res;
+ struct pci_bus *bus = pdev->bus;
+ struct resource *r;
+ LIST_HEAD(saved);
+ unsigned int i;
+ int ret = 0;
- res->start = dev_res->start;
- res->end = dev_res->end;
- res->flags = dev_res->flags;
+ down_read(&pci_bus_sem);
+
+ pci_dev_for_each_resource(pdev, r, i) {
+ if (i >= PCI_BRIDGE_RESOURCES)
+ break;
+
+ if (exclude_bars & BIT(i))
+ continue;
+
+ if (!pci_resource_len(pdev, i) || r->flags != flags)
+ continue;
+
+ ret = add_to_list(&saved, pdev, r, 0, 0);
+ if (ret)
+ goto restore;
+ pci_release_resource(pdev, i);
}
- free_list(&failed);
+ res->end = res->start + pci_rebar_size_to_bytes(size) - 1;
+
+ if (!bus->self)
+ goto out;
+
+ ret = pci_reassign_bridge_resources(bus->self, res->flags, &saved);
+ if (ret)
+ goto restore;
+
+out:
+ up_read(&pci_bus_sem);
+ free_list(&saved);
+ return ret;
+
+restore:
/* Revert to the old configuration */
list_for_each_entry(dev_res, &saved, list) {
struct resource *res = dev_res->res;
+ struct pci_dev *dev = dev_res->dev;
- bridge = dev_res->dev;
- i = res - bridge->resource;
+ i = res - dev->resource;
+
+ if (res->parent) {
+ release_child_resources(res);
+ pci_release_resource(dev, i);
+ }
res->start = dev_res->start;
res->end = dev_res->end;
res->flags = dev_res->flags;
- pci_claim_resource(bridge, i);
- pci_setup_bridge(bridge->subordinate);
- }
- free_list(&saved);
- up_read(&pci_bus_sem);
+ if (pci_claim_resource(dev, i))
+ continue;
- return ret;
+ if (i < PCI_BRIDGE_RESOURCES) {
+ const char *res_name = pci_resource_name(dev, i);
+
+ pci_update_resource(dev, i);
+ pci_info(dev, "%s %pR: old value restored\n",
+ res_name, res);
+ }
+ if (dev->subordinate)
+ pci_setup_bridge(dev->subordinate);
+ }
+ goto out;
}
void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 86359552368a..f2156ca40589 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -429,59 +429,6 @@ void pci_release_resource(struct pci_dev *dev, int resno)
}
EXPORT_SYMBOL(pci_release_resource);
-int pci_resize_resource(struct pci_dev *dev, int resno, int size)
-{
- struct resource *res = dev->resource + resno;
- struct pci_host_bridge *host;
- int old, ret;
- u32 sizes;
- u16 cmd;
-
- /* Check if we must preserve the firmware's resource assignment */
- host = pci_find_host_bridge(dev->bus);
- if (host->preserve_config)
- return -ENOTSUPP;
-
- /* Make sure the resource isn't assigned before resizing it. */
- if (!(res->flags & IORESOURCE_UNSET))
- return -EBUSY;
-
- pci_read_config_word(dev, PCI_COMMAND, &cmd);
- if (cmd & PCI_COMMAND_MEMORY)
- return -EBUSY;
-
- sizes = pci_rebar_get_possible_sizes(dev, resno);
- if (!sizes)
- return -ENOTSUPP;
-
- if (!(sizes & BIT(size)))
- return -EINVAL;
-
- old = pci_rebar_get_current_size(dev, resno);
- if (old < 0)
- return old;
-
- ret = pci_rebar_set_size(dev, resno, size);
- if (ret)
- return ret;
-
- res->end = res->start + pci_rebar_size_to_bytes(size) - 1;
-
- /* Check if the new config works by trying to assign everything. */
- if (dev->bus->self) {
- ret = pci_reassign_bridge_resources(dev->bus->self, res->flags);
- if (ret)
- goto error_resize;
- }
- return 0;
-
-error_resize:
- pci_rebar_set_size(dev, resno, old);
- res->end = res->start + pci_rebar_size_to_bytes(old) - 1;
- return ret;
-}
-EXPORT_SYMBOL(pci_resize_resource);
-
int pci_enable_resources(struct pci_dev *dev, int mask)
{
u16 cmd, old_cmd;
diff --git a/drivers/phy/phy-can-transceiver.c b/drivers/phy/phy-can-transceiver.c
index ee4ce4249698..95a02705bf2a 100644
--- a/drivers/phy/phy-can-transceiver.c
+++ b/drivers/phy/phy-can-transceiver.c
@@ -111,6 +111,9 @@ static int can_transceiver_phy_probe(struct platform_device *pdev)
return -ENOMEM;
match = of_match_node(can_transceiver_phy_ids, pdev->dev.of_node);
+ if (!match || !match->data)
+ return -ENODEV;
+
drvdata = match->data;
if (of_property_read_bool(dev->of_node, "mux-states")) {
diff --git a/drivers/pinctrl/cirrus/pinctrl-cs42l43.c b/drivers/pinctrl/cirrus/pinctrl-cs42l43.c
index 7734dae06a4a..dc808ce6d353 100644
--- a/drivers/pinctrl/cirrus/pinctrl-cs42l43.c
+++ b/drivers/pinctrl/cirrus/pinctrl-cs42l43.c
@@ -343,7 +343,7 @@ static int cs42l43_pin_set_db(struct cs42l43_pin *priv, unsigned int pin,
return regmap_update_bits(priv->regmap, CS42L43_GPIO_CTRL2,
CS42L43_GPIO1_DEGLITCH_BYP_MASK << pin,
- !!us << pin);
+ !us << pin);
}
static int cs42l43_pin_config_get(struct pinctrl_dev *pctldev,
diff --git a/drivers/pinctrl/mediatek/pinctrl-mt8167.c b/drivers/pinctrl/mediatek/pinctrl-mt8167.c
index 143c26622272..c812d614e9d4 100644
--- a/drivers/pinctrl/mediatek/pinctrl-mt8167.c
+++ b/drivers/pinctrl/mediatek/pinctrl-mt8167.c
@@ -244,7 +244,7 @@ static const struct mtk_pin_ies_smt_set mt8167_smt_set[] = {
MTK_PIN_IES_SMT_SPEC(24, 25, 0xA00, 12),
MTK_PIN_IES_SMT_SPEC(26, 30, 0xA00, 0),
MTK_PIN_IES_SMT_SPEC(31, 33, 0xA00, 1),
- MTK_PIN_IES_SMT_SPEC(34, 39, 0xA900, 2),
+ MTK_PIN_IES_SMT_SPEC(34, 39, 0xA00, 2),
MTK_PIN_IES_SMT_SPEC(40, 40, 0xA10, 11),
MTK_PIN_IES_SMT_SPEC(41, 43, 0xA00, 10),
MTK_PIN_IES_SMT_SPEC(44, 47, 0xA00, 11),
diff --git a/drivers/pinctrl/mediatek/pinctrl-mt8516.c b/drivers/pinctrl/mediatek/pinctrl-mt8516.c
index abda75d4354e..68d6638e7f4b 100644
--- a/drivers/pinctrl/mediatek/pinctrl-mt8516.c
+++ b/drivers/pinctrl/mediatek/pinctrl-mt8516.c
@@ -244,7 +244,7 @@ static const struct mtk_pin_ies_smt_set mt8516_smt_set[] = {
MTK_PIN_IES_SMT_SPEC(24, 25, 0xA00, 12),
MTK_PIN_IES_SMT_SPEC(26, 30, 0xA00, 0),
MTK_PIN_IES_SMT_SPEC(31, 33, 0xA00, 1),
- MTK_PIN_IES_SMT_SPEC(34, 39, 0xA900, 2),
+ MTK_PIN_IES_SMT_SPEC(34, 39, 0xA00, 2),
MTK_PIN_IES_SMT_SPEC(40, 40, 0xA10, 11),
MTK_PIN_IES_SMT_SPEC(41, 43, 0xA00, 10),
MTK_PIN_IES_SMT_SPEC(44, 47, 0xA00, 11),
diff --git a/drivers/pinctrl/meson/pinctrl-meson.c b/drivers/pinctrl/meson/pinctrl-meson.c
index b4b4638df0f2..e5a32a0532ee 100644
--- a/drivers/pinctrl/meson/pinctrl-meson.c
+++ b/drivers/pinctrl/meson/pinctrl-meson.c
@@ -619,7 +619,7 @@ static int meson_gpiolib_register(struct meson_pinctrl *pc)
pc->chip.set = meson_gpio_set;
pc->chip.base = -1;
pc->chip.ngpio = pc->data->num_pins;
- pc->chip.can_sleep = true;
+ pc->chip.can_sleep = false;
ret = gpiochip_add_data(&pc->chip, pc);
if (ret) {
diff --git a/drivers/pinctrl/nuvoton/pinctrl-ma35.c b/drivers/pinctrl/nuvoton/pinctrl-ma35.c
index 59c4e7c6cdde..3330e444ede1 100644
--- a/drivers/pinctrl/nuvoton/pinctrl-ma35.c
+++ b/drivers/pinctrl/nuvoton/pinctrl-ma35.c
@@ -1026,7 +1026,8 @@ static int ma35_pinctrl_parse_groups(struct device_node *np, struct ma35_pin_gro
pin = grp->settings;
for (i = 0, j = 0; i < count; i += 3, j++) {
- pin->offset = elems[i] * MA35_MFP_REG_SZ_PER_BANK + MA35_MFP_REG_BASE;
+ pin->offset = elems[i] * MA35_MFP_REG_SZ_PER_BANK + MA35_MFP_REG_BASE +
+ (elems[i + 1] >= 8 ? 4 : 0);
pin->shift = (elems[i + 1] * MA35_MFP_BITS_PER_PORT) % 32;
pin->muxval = elems[i + 2];
pin->configs = configs;
diff --git a/drivers/pinctrl/nuvoton/pinctrl-ma35d1.c b/drivers/pinctrl/nuvoton/pinctrl-ma35d1.c
index 8bb9a5a35954..079b73a36ca4 100644
--- a/drivers/pinctrl/nuvoton/pinctrl-ma35d1.c
+++ b/drivers/pinctrl/nuvoton/pinctrl-ma35d1.c
@@ -114,6 +114,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x0, "GPA14"),
MA35_MUX(0x2, "UART7_RXD"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL3_DM"),
MA35_MUX(0x6, "NAND_nWP"),
MA35_MUX(0x7, "EBI_AD14"),
MA35_MUX(0x9, "EBI_ADR14")),
@@ -124,6 +125,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x3, "UART6_RXD"),
MA35_MUX(0x4, "I2C4_SDA"),
MA35_MUX(0x5, "CAN2_RXD"),
+ MA35_MUX(0x6, "USBHL0_DM"),
MA35_MUX(0x7, "EBI_ALE"),
MA35_MUX(0x9, "QEI0_A"),
MA35_MUX(0xb, "TM1"),
@@ -188,6 +190,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x1, "EPWM2_CH5"),
MA35_MUX(0x2, "UART2_RXD"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL2_DM"),
MA35_MUX(0x5, "SPI0_MOSI"),
MA35_MUX(0x6, "EBI_MCLK"),
MA35_MUX(0x7, "CCAP1_VSYNC"),
@@ -203,6 +206,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x1, "EPWM2_BRAKE1"),
MA35_MUX(0x2, "UART2_TXD"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL2_DP"),
MA35_MUX(0x5, "SPI0_MISO"),
MA35_MUX(0x6, "I2S1_MCLK"),
MA35_MUX(0x7, "CCAP1_SFIELD"),
@@ -221,6 +225,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x4, "I2C3_SDA"),
MA35_MUX(0x5, "CAN2_RXD"),
MA35_MUX(0x6, "I2S1_LRCK"),
+ MA35_MUX(0x7, "USBHL1_DM"),
MA35_MUX(0x8, "ADC0_CH4"),
MA35_MUX(0x9, "EBI_ADR16"),
MA35_MUX(0xe, "ECAP2_IC0")),
@@ -232,6 +237,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x4, "I2C3_SCL"),
MA35_MUX(0x5, "CAN2_TXD"),
MA35_MUX(0x6, "I2S1_BCLK"),
+ MA35_MUX(0x7, "USBHL1_DP"),
MA35_MUX(0x8, "ADC0_CH5"),
MA35_MUX(0x9, "EBI_ADR17"),
MA35_MUX(0xe, "ECAP2_IC1")),
@@ -240,6 +246,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x1, "EPWM2_CH2"),
MA35_MUX(0x2, "UART4_RXD"),
MA35_MUX(0x3, "CAN1_RXD"),
+ MA35_MUX(0x4, "USBHL3_DM"),
MA35_MUX(0x5, "I2C4_SDA"),
MA35_MUX(0x6, "I2S1_DI"),
MA35_MUX(0x8, "ADC0_CH6"),
@@ -250,6 +257,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x1, "EPWM2_CH3"),
MA35_MUX(0x2, "UART4_TXD"),
MA35_MUX(0x3, "CAN1_TXD"),
+ MA35_MUX(0x4, "USBHL3_DP"),
MA35_MUX(0x5, "I2C4_SCL"),
MA35_MUX(0x6, "I2S1_DO"),
MA35_MUX(0x8, "ADC0_CH7"),
@@ -265,10 +273,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_PIN(34, PC2, 0x90, 0x8,
MA35_MUX(0x0, "GPC2"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL4_DM"),
MA35_MUX(0x6, "SD0_DAT0/eMMC0_DAT0")),
MA35_PIN(35, PC3, 0x90, 0xc,
MA35_MUX(0x0, "GPC3"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL4_DP"),
MA35_MUX(0x6, "SD0_DAT1/eMMC0_DAT1")),
MA35_PIN(36, PC4, 0x90, 0x10,
MA35_MUX(0x0, "GPC4"),
@@ -281,65 +291,100 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_PIN(38, PC6, 0x90, 0x18,
MA35_MUX(0x0, "GPC6"),
MA35_MUX(0x3, "CAN1_RXD"),
+ MA35_MUX(0x4, "USBHL5_DM"),
MA35_MUX(0x6, "SD0_nCD")),
MA35_PIN(39, PC7, 0x90, 0x1c,
MA35_MUX(0x0, "GPC7"),
MA35_MUX(0x3, "CAN1_TXD"),
+ MA35_MUX(0x4, "USBHL5_DP"),
MA35_MUX(0x6, "SD0_WP")),
- MA35_PIN(40, PC12, 0x94, 0x10,
+ MA35_PIN(40, PC8, 0x94, 0x0,
+ MA35_MUX(0x0, "GPC8"),
+ MA35_MUX(0x1, "EPWM2_CH0"),
+ MA35_MUX(0x2, "UART10_nCTS"),
+ MA35_MUX(0x3, "UART9_RXD"),
+ MA35_MUX(0x4, "I2C0_SDA"),
+ MA35_MUX(0x5, "SPI1_SS0"),
+ MA35_MUX(0x6, "SD0_DAT4/eMMC0_DAT4")),
+ MA35_PIN(41, PC9, 0x94, 0x4,
+ MA35_MUX(0x0, "GPC9"),
+ MA35_MUX(0x1, "EPWM2_CH1"),
+ MA35_MUX(0x2, "UART10_nRTS"),
+ MA35_MUX(0x3, "UART9_TXD"),
+ MA35_MUX(0x4, "I2C0_SCL"),
+ MA35_MUX(0x5, "SPI1_CLK"),
+ MA35_MUX(0x6, "SD0_DAT5/eMMC0_DAT5")),
+ MA35_PIN(42, PC10, 0x94, 0x8,
+ MA35_MUX(0x0, "GPC10"),
+ MA35_MUX(0x1, "EPWM2_CH2"),
+ MA35_MUX(0x2, "UART10_RXD"),
+ MA35_MUX(0x3, "CAN2_RXD"),
+ MA35_MUX(0x4, "USBHL0_DM"),
+ MA35_MUX(0x5, "SPI1_MOSI"),
+ MA35_MUX(0x6, "SD0_DAT6/eMMC0_DAT6")),
+ MA35_PIN(43, PC11, 0x94, 0xc,
+ MA35_MUX(0x0, "GPC11"),
+ MA35_MUX(0x1, "EPWM2_CH3"),
+ MA35_MUX(0x2, "UART10_TXD"),
+ MA35_MUX(0x3, "CAN2_TXD"),
+ MA35_MUX(0x4, "USBHL0_DP"),
+ MA35_MUX(0x5, "SPI1_MISO"),
+ MA35_MUX(0x6, "SD0_DAT7/eMMC0_DAT7")),
+ MA35_PIN(44, PC12, 0x94, 0x10,
MA35_MUX(0x0, "GPC12"),
MA35_MUX(0x2, "UART12_nCTS"),
MA35_MUX(0x3, "UART11_RXD"),
MA35_MUX(0x6, "LCM_DATA16")),
- MA35_PIN(41, PC13, 0x94, 0x14,
+ MA35_PIN(45, PC13, 0x94, 0x14,
MA35_MUX(0x0, "GPC13"),
MA35_MUX(0x2, "UART12_nRTS"),
MA35_MUX(0x3, "UART11_TXD"),
MA35_MUX(0x6, "LCM_DATA17")),
- MA35_PIN(42, PC14, 0x94, 0x18,
+ MA35_PIN(46, PC14, 0x94, 0x18,
MA35_MUX(0x0, "GPC14"),
MA35_MUX(0x2, "UART12_RXD"),
MA35_MUX(0x6, "LCM_DATA18")),
- MA35_PIN(43, PC15, 0x94, 0x1c,
+ MA35_PIN(47, PC15, 0x94, 0x1c,
MA35_MUX(0x0, "GPC15"),
MA35_MUX(0x2, "UART12_TXD"),
MA35_MUX(0x6, "LCM_DATA19"),
MA35_MUX(0x7, "LCM_MPU_TE"),
MA35_MUX(0x8, "LCM_MPU_VSYNC")),
- MA35_PIN(44, PD0, 0x98, 0x0,
+ MA35_PIN(48, PD0, 0x98, 0x0,
MA35_MUX(0x0, "GPD0"),
MA35_MUX(0x2, "UART3_nCTS"),
MA35_MUX(0x3, "UART4_RXD"),
MA35_MUX(0x5, "QSPI0_SS0")),
- MA35_PIN(45, PD1, 0x98, 0x4,
+ MA35_PIN(49, PD1, 0x98, 0x4,
MA35_MUX(0x0, "GPD1"),
MA35_MUX(0x2, "UART3_nRTS"),
MA35_MUX(0x3, "UART4_TXD"),
MA35_MUX(0x5, "QSPI0_CLK")),
- MA35_PIN(46, PD2, 0x98, 0x8,
+ MA35_PIN(50, PD2, 0x98, 0x8,
MA35_MUX(0x0, "GPD2"),
MA35_MUX(0x2, "UART3_RXD"),
MA35_MUX(0x5, "QSPI0_MOSI0")),
- MA35_PIN(47, PD3, 0x98, 0xc,
+ MA35_PIN(51, PD3, 0x98, 0xc,
MA35_MUX(0x0, "GPD3"),
MA35_MUX(0x2, "UART3_TXD"),
MA35_MUX(0x5, "QSPI0_MISO0")),
- MA35_PIN(48, PD4, 0x98, 0x10,
+ MA35_PIN(52, PD4, 0x98, 0x10,
MA35_MUX(0x0, "GPD4"),
MA35_MUX(0x2, "UART1_nCTS"),
MA35_MUX(0x3, "UART2_RXD"),
MA35_MUX(0x4, "I2C2_SDA"),
MA35_MUX(0x5, "QSPI0_MOSI1")),
- MA35_PIN(49, PD5, 0x98, 0x14,
+ MA35_PIN(53, PD5, 0x98, 0x14,
MA35_MUX(0x0, "GPD5"),
MA35_MUX(0x2, "UART1_nRTS"),
MA35_MUX(0x3, "UART2_TXD"),
MA35_MUX(0x4, "I2C2_SCL"),
MA35_MUX(0x5, "QSPI0_MISO1")),
- MA35_PIN(50, PD6, 0x98, 0x18,
+ MA35_PIN(54, PD6, 0x98, 0x18,
MA35_MUX(0x0, "GPD6"),
MA35_MUX(0x1, "EPWM0_SYNC_IN"),
MA35_MUX(0x2, "UART1_RXD"),
+ MA35_MUX(0x4, "USBHL3_DM"),
MA35_MUX(0x5, "QSPI1_MOSI1"),
MA35_MUX(0x6, "I2C0_SDA"),
MA35_MUX(0x7, "I2S0_MCLK"),
@@ -347,10 +392,11 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "EBI_AD5"),
MA35_MUX(0xa, "SPI3_SS1"),
MA35_MUX(0xb, "TRACE_CLK")),
- MA35_PIN(51, PD7, 0x98, 0x1c,
+ MA35_PIN(55, PD7, 0x98, 0x1c,
MA35_MUX(0x0, "GPD7"),
MA35_MUX(0x1, "EPWM0_SYNC_OUT"),
MA35_MUX(0x2, "UART1_TXD"),
+ MA35_MUX(0x4, "USBHL3_DP"),
MA35_MUX(0x5, "QSPI1_MISO1"),
MA35_MUX(0x6, "I2C0_SCL"),
MA35_MUX(0x7, "I2S1_MCLK"),
@@ -358,7 +404,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "EBI_AD6"),
MA35_MUX(0xa, "SC1_nCD"),
MA35_MUX(0xb, "EADC0_ST")),
- MA35_PIN(52, PD8, 0x9c, 0x0,
+ MA35_PIN(56, PD8, 0x9c, 0x0,
MA35_MUX(0x0, "GPD8"),
MA35_MUX(0x1, "EPWM0_BRAKE0"),
MA35_MUX(0x2, "UART16_nCTS"),
@@ -369,7 +415,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "EBI_AD7"),
MA35_MUX(0xa, "SC1_CLK"),
MA35_MUX(0xb, "TM0")),
- MA35_PIN(53, PD9, 0x9c, 0x4,
+ MA35_PIN(57, PD9, 0x9c, 0x4,
MA35_MUX(0x0, "GPD9"),
MA35_MUX(0x1, "EPWM0_BRAKE1"),
MA35_MUX(0x2, "UART16_nRTS"),
@@ -380,7 +426,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "EBI_AD8"),
MA35_MUX(0xa, "SC1_DAT"),
MA35_MUX(0xb, "TM0_EXT")),
- MA35_PIN(54, PD10, 0x9c, 0x8,
+ MA35_PIN(58, PD10, 0x9c, 0x8,
MA35_MUX(0x0, "GPD10"),
MA35_MUX(0x1, "EPWM1_BRAKE0"),
MA35_MUX(0x2, "UART16_RXD"),
@@ -390,7 +436,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "EBI_AD9"),
MA35_MUX(0xa, "SC1_RST"),
MA35_MUX(0xb, "TM2")),
- MA35_PIN(55, PD11, 0x9c, 0xc,
+ MA35_PIN(59, PD11, 0x9c, 0xc,
MA35_MUX(0x0, "GPD11"),
MA35_MUX(0x1, "EPWM1_BRAKE1"),
MA35_MUX(0x2, "UART16_TXD"),
@@ -400,7 +446,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "EBI_AD10"),
MA35_MUX(0xa, "SC1_PWR"),
MA35_MUX(0xb, "TM2_EXT")),
- MA35_PIN(56, PD12, 0x9c, 0x10,
+ MA35_PIN(60, PD12, 0x9c, 0x10,
MA35_MUX(0x0, "GPD12"),
MA35_MUX(0x1, "EPWM0_BRAKE0"),
MA35_MUX(0x2, "UART11_TXD"),
@@ -413,7 +459,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xb, "TM5"),
MA35_MUX(0xc, "I2S1_LRCK"),
MA35_MUX(0xd, "INT1")),
- MA35_PIN(57, PD13, 0x9c, 0x14,
+ MA35_PIN(61, PD13, 0x9c, 0x14,
MA35_MUX(0x0, "GPD13"),
MA35_MUX(0x1, "EPWM0_BRAKE1"),
MA35_MUX(0x2, "UART11_RXD"),
@@ -425,11 +471,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "ECAP0_IC0"),
MA35_MUX(0xb, "TM5_EXT"),
MA35_MUX(0xc, "I2S1_BCLK")),
- MA35_PIN(58, PD14, 0x9c, 0x18,
+ MA35_PIN(62, PD14, 0x9c, 0x18,
MA35_MUX(0x0, "GPD14"),
MA35_MUX(0x1, "EPWM0_SYNC_IN"),
MA35_MUX(0x2, "UART11_nCTS"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL5_DM"),
MA35_MUX(0x6, "TRACE_DATA2"),
MA35_MUX(0x7, "EBI_MCLK"),
MA35_MUX(0x8, "EBI_AD6"),
@@ -437,116 +484,117 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xb, "TM6"),
MA35_MUX(0xc, "I2S1_DI"),
MA35_MUX(0xd, "INT3")),
- MA35_PIN(59, PD15, 0x9c, 0x1c,
+ MA35_PIN(63, PD15, 0x9c, 0x1c,
MA35_MUX(0x0, "GPD15"),
MA35_MUX(0x1, "EPWM0_SYNC_OUT"),
MA35_MUX(0x2, "UART11_nRTS"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL5_DP"),
MA35_MUX(0x6, "TRACE_DATA3"),
MA35_MUX(0x7, "EBI_ALE"),
MA35_MUX(0x8, "EBI_AD7"),
MA35_MUX(0x9, "ECAP0_IC2"),
MA35_MUX(0xb, "TM6_EXT"),
MA35_MUX(0xc, "I2S1_DO")),
- MA35_PIN(60, PE0, 0xa0, 0x0,
+ MA35_PIN(64, PE0, 0xa0, 0x0,
MA35_MUX(0x0, "GPE0"),
MA35_MUX(0x2, "UART9_nCTS"),
MA35_MUX(0x3, "UART8_RXD"),
MA35_MUX(0x7, "CCAP1_DATA0"),
MA35_MUX(0x8, "RGMII0_MDC"),
MA35_MUX(0x9, "RMII0_MDC")),
- MA35_PIN(61, PE1, 0xa0, 0x4,
+ MA35_PIN(65, PE1, 0xa0, 0x4,
MA35_MUX(0x0, "GPE1"),
MA35_MUX(0x2, "UART9_nRTS"),
MA35_MUX(0x3, "UART8_TXD"),
MA35_MUX(0x7, "CCAP1_DATA1"),
MA35_MUX(0x8, "RGMII0_MDIO"),
MA35_MUX(0x9, "RMII0_MDIO")),
- MA35_PIN(62, PE2, 0xa0, 0x8,
+ MA35_PIN(66, PE2, 0xa0, 0x8,
MA35_MUX(0x0, "GPE2"),
MA35_MUX(0x2, "UART9_RXD"),
MA35_MUX(0x7, "CCAP1_DATA2"),
MA35_MUX(0x8, "RGMII0_TXCTL"),
MA35_MUX(0x9, "RMII0_TXEN")),
- MA35_PIN(63, PE3, 0xa0, 0xc,
+ MA35_PIN(67, PE3, 0xa0, 0xc,
MA35_MUX(0x0, "GPE3"),
MA35_MUX(0x2, "UART9_TXD"),
MA35_MUX(0x7, "CCAP1_DATA3"),
MA35_MUX(0x8, "RGMII0_TXD0"),
MA35_MUX(0x9, "RMII0_TXD0")),
- MA35_PIN(64, PE4, 0xa0, 0x10,
+ MA35_PIN(68, PE4, 0xa0, 0x10,
MA35_MUX(0x0, "GPE4"),
MA35_MUX(0x2, "UART4_nCTS"),
MA35_MUX(0x3, "UART3_RXD"),
MA35_MUX(0x7, "CCAP1_DATA4"),
MA35_MUX(0x8, "RGMII0_TXD1"),
MA35_MUX(0x9, "RMII0_TXD1")),
- MA35_PIN(65, PE5, 0xa0, 0x14,
+ MA35_PIN(69, PE5, 0xa0, 0x14,
MA35_MUX(0x0, "GPE5"),
MA35_MUX(0x2, "UART4_nRTS"),
MA35_MUX(0x3, "UART3_TXD"),
MA35_MUX(0x7, "CCAP1_DATA5"),
MA35_MUX(0x8, "RGMII0_RXCLK"),
MA35_MUX(0x9, "RMII0_REFCLK")),
- MA35_PIN(66, PE6, 0xa0, 0x18,
+ MA35_PIN(70, PE6, 0xa0, 0x18,
MA35_MUX(0x0, "GPE6"),
MA35_MUX(0x2, "UART4_RXD"),
MA35_MUX(0x7, "CCAP1_DATA6"),
MA35_MUX(0x8, "RGMII0_RXCTL"),
MA35_MUX(0x9, "RMII0_CRSDV")),
- MA35_PIN(67, PE7, 0xa0, 0x1c,
+ MA35_PIN(71, PE7, 0xa0, 0x1c,
MA35_MUX(0x0, "GPE7"),
MA35_MUX(0x2, "UART4_TXD"),
MA35_MUX(0x7, "CCAP1_DATA7"),
MA35_MUX(0x8, "RGMII0_RXD0"),
MA35_MUX(0x9, "RMII0_RXD0")),
- MA35_PIN(68, PE8, 0xa4, 0x0,
+ MA35_PIN(72, PE8, 0xa4, 0x0,
MA35_MUX(0x0, "GPE8"),
MA35_MUX(0x2, "UART13_nCTS"),
MA35_MUX(0x3, "UART12_RXD"),
MA35_MUX(0x7, "CCAP1_SCLK"),
MA35_MUX(0x8, "RGMII0_RXD1"),
MA35_MUX(0x9, "RMII0_RXD1")),
- MA35_PIN(69, PE9, 0xa4, 0x4,
+ MA35_PIN(73, PE9, 0xa4, 0x4,
MA35_MUX(0x0, "GPE9"),
MA35_MUX(0x2, "UART13_nRTS"),
MA35_MUX(0x3, "UART12_TXD"),
MA35_MUX(0x7, "CCAP1_PIXCLK"),
MA35_MUX(0x8, "RGMII0_RXD2"),
MA35_MUX(0x9, "RMII0_RXERR")),
- MA35_PIN(70, PE10, 0xa4, 0x8,
+ MA35_PIN(74, PE10, 0xa4, 0x8,
MA35_MUX(0x0, "GPE10"),
MA35_MUX(0x2, "UART15_nCTS"),
MA35_MUX(0x3, "UART14_RXD"),
MA35_MUX(0x5, "SPI1_SS0"),
MA35_MUX(0x7, "CCAP1_HSYNC"),
MA35_MUX(0x8, "RGMII0_RXD3")),
- MA35_PIN(71, PE11, 0xa4, 0xc,
+ MA35_PIN(75, PE11, 0xa4, 0xc,
MA35_MUX(0x0, "GPE11"),
MA35_MUX(0x2, "UART15_nRTS"),
MA35_MUX(0x3, "UART14_TXD"),
MA35_MUX(0x5, "SPI1_CLK"),
MA35_MUX(0x7, "CCAP1_VSYNC"),
MA35_MUX(0x8, "RGMII0_TXCLK")),
- MA35_PIN(72, PE12, 0xa4, 0x10,
+ MA35_PIN(76, PE12, 0xa4, 0x10,
MA35_MUX(0x0, "GPE12"),
MA35_MUX(0x2, "UART15_RXD"),
MA35_MUX(0x5, "SPI1_MOSI"),
MA35_MUX(0x7, "CCAP1_DATA8"),
MA35_MUX(0x8, "RGMII0_TXD2")),
- MA35_PIN(73, PE13, 0xa4, 0x14,
+ MA35_PIN(77, PE13, 0xa4, 0x14,
MA35_MUX(0x0, "GPE13"),
MA35_MUX(0x2, "UART15_TXD"),
MA35_MUX(0x5, "SPI1_MISO"),
MA35_MUX(0x7, "CCAP1_DATA9"),
MA35_MUX(0x8, "RGMII0_TXD3")),
- MA35_PIN(74, PE14, 0xa4, 0x18,
+ MA35_PIN(78, PE14, 0xa4, 0x18,
MA35_MUX(0x0, "GPE14"),
MA35_MUX(0x1, "UART0_TXD")),
- MA35_PIN(75, PE15, 0xa4, 0x1c,
+ MA35_PIN(79, PE15, 0xa4, 0x1c,
MA35_MUX(0x0, "GPE15"),
MA35_MUX(0x1, "UART0_RXD")),
- MA35_PIN(76, PF0, 0xa8, 0x0,
+ MA35_PIN(80, PF0, 0xa8, 0x0,
MA35_MUX(0x0, "GPF0"),
MA35_MUX(0x2, "UART2_nCTS"),
MA35_MUX(0x3, "UART1_RXD"),
@@ -554,7 +602,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "RGMII1_MDC"),
MA35_MUX(0x9, "RMII1_MDC"),
MA35_MUX(0xe, "KPI_COL0")),
- MA35_PIN(77, PF1, 0xa8, 0x4,
+ MA35_PIN(81, PF1, 0xa8, 0x4,
MA35_MUX(0x0, "GPF1"),
MA35_MUX(0x2, "UART2_nRTS"),
MA35_MUX(0x3, "UART1_TXD"),
@@ -562,21 +610,21 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "RGMII1_MDIO"),
MA35_MUX(0x9, "RMII1_MDIO"),
MA35_MUX(0xe, "KPI_COL1")),
- MA35_PIN(78, PF2, 0xa8, 0x8,
+ MA35_PIN(82, PF2, 0xa8, 0x8,
MA35_MUX(0x0, "GPF2"),
MA35_MUX(0x2, "UART2_RXD"),
MA35_MUX(0x6, "RGMII0_TXD2"),
MA35_MUX(0x8, "RGMII1_TXCTL"),
MA35_MUX(0x9, "RMII1_TXEN"),
MA35_MUX(0xe, "KPI_COL2")),
- MA35_PIN(79, PF3, 0xa8, 0xc,
+ MA35_PIN(83, PF3, 0xa8, 0xc,
MA35_MUX(0x0, "GPF3"),
MA35_MUX(0x2, "UART2_TXD"),
MA35_MUX(0x6, "RGMII0_TXD3"),
MA35_MUX(0x8, "RGMII1_TXD0"),
MA35_MUX(0x9, "RMII1_TXD0"),
MA35_MUX(0xe, "KPI_COL3")),
- MA35_PIN(80, PF4, 0xa8, 0x10,
+ MA35_PIN(84, PF4, 0xa8, 0x10,
MA35_MUX(0x0, "GPF4"),
MA35_MUX(0x2, "UART11_nCTS"),
MA35_MUX(0x3, "UART10_RXD"),
@@ -584,9 +632,10 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x5, "SPI1_SS0"),
MA35_MUX(0x8, "RGMII1_TXD1"),
MA35_MUX(0x9, "RMII1_TXD1"),
+ MA35_MUX(0xc, "USBHL0_DM"),
MA35_MUX(0xd, "CAN2_RXD"),
MA35_MUX(0xe, "KPI_ROW0")),
- MA35_PIN(81, PF5, 0xa8, 0x14,
+ MA35_PIN(85, PF5, 0xa8, 0x14,
MA35_MUX(0x0, "GPF5"),
MA35_MUX(0x2, "UART11_nRTS"),
MA35_MUX(0x3, "UART10_TXD"),
@@ -594,9 +643,10 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x5, "SPI1_CLK"),
MA35_MUX(0x8, "RGMII1_RXCLK"),
MA35_MUX(0x9, "RMII1_REFCLK"),
+ MA35_MUX(0xc, "USBHL0_DP"),
MA35_MUX(0xd, "CAN2_TXD"),
MA35_MUX(0xe, "KPI_ROW1")),
- MA35_PIN(82, PF6, 0xa8, 0x18,
+ MA35_PIN(86, PF6, 0xa8, 0x18,
MA35_MUX(0x0, "GPF6"),
MA35_MUX(0x2, "UART11_RXD"),
MA35_MUX(0x4, "I2S0_DI"),
@@ -606,7 +656,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "I2C4_SDA"),
MA35_MUX(0xd, "SC0_CLK"),
MA35_MUX(0xe, "KPI_ROW2")),
- MA35_PIN(83, PF7, 0xa8, 0x1c,
+ MA35_PIN(87, PF7, 0xa8, 0x1c,
MA35_MUX(0x0, "GPF7"),
MA35_MUX(0x2, "UART11_TXD"),
MA35_MUX(0x4, "I2S0_DO"),
@@ -616,7 +666,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "I2C4_SCL"),
MA35_MUX(0xd, "SC0_DAT"),
MA35_MUX(0xe, "KPI_ROW3")),
- MA35_PIN(84, PF8, 0xac, 0x0,
+ MA35_PIN(88, PF8, 0xac, 0x0,
MA35_MUX(0x0, "GPF8"),
MA35_MUX(0x2, "UART13_RXD"),
MA35_MUX(0x4, "I2C5_SDA"),
@@ -625,7 +675,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "RMII1_RXD1"),
MA35_MUX(0xd, "SC0_RST"),
MA35_MUX(0xe, "KPI_COL4")),
- MA35_PIN(85, PF9, 0xac, 0x4,
+ MA35_PIN(89, PF9, 0xac, 0x4,
MA35_MUX(0x0, "GPF9"),
MA35_MUX(0x2, "UART13_TXD"),
MA35_MUX(0x4, "I2C5_SCL"),
@@ -634,7 +684,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "RMII1_RXERR"),
MA35_MUX(0xd, "SC0_PWR"),
MA35_MUX(0xe, "KPI_COL5")),
- MA35_PIN(86, PF10, 0xac, 0x8,
+ MA35_PIN(90, PF10, 0xac, 0x8,
MA35_MUX(0x0, "GPF10"),
MA35_MUX(0x2, "UART13_nCTS"),
MA35_MUX(0x5, "I2S0_LRCK"),
@@ -642,7 +692,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "RGMII1_RXD3"),
MA35_MUX(0x9, "SC0_CLK"),
MA35_MUX(0xe, "KPI_COL6")),
- MA35_PIN(87, PF11, 0xac, 0xc,
+ MA35_PIN(91, PF11, 0xac, 0xc,
MA35_MUX(0x0, "GPF11"),
MA35_MUX(0x2, "UART13_nRTS"),
MA35_MUX(0x5, "I2S0_BCLK"),
@@ -650,21 +700,21 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "RGMII1_TXCLK"),
MA35_MUX(0x9, "SC0_DAT"),
MA35_MUX(0xe, "KPI_COL7")),
- MA35_PIN(88, PF12, 0xac, 0x10,
+ MA35_PIN(92, PF12, 0xac, 0x10,
MA35_MUX(0x0, "GPF12"),
MA35_MUX(0x5, "I2S0_DI"),
MA35_MUX(0x6, "SPI1_MOSI"),
MA35_MUX(0x8, "RGMII1_TXD2"),
MA35_MUX(0x9, "SC0_RST"),
MA35_MUX(0xe, "KPI_ROW4")),
- MA35_PIN(89, PF13, 0xac, 0x14,
+ MA35_PIN(93, PF13, 0xac, 0x14,
MA35_MUX(0x0, "GPF13"),
MA35_MUX(0x5, "I2S0_DO"),
MA35_MUX(0x6, "SPI1_MISO"),
MA35_MUX(0x8, "RGMII1_TXD3"),
MA35_MUX(0x9, "SC0_PWR"),
MA35_MUX(0xe, "KPI_ROW5")),
- MA35_PIN(90, PF14, 0xac, 0x18,
+ MA35_PIN(94, PF14, 0xac, 0x18,
MA35_MUX(0x0, "GPF14"),
MA35_MUX(0x1, "EPWM2_BRAKE0"),
MA35_MUX(0x2, "EADC0_ST"),
@@ -680,10 +730,10 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xd, "SPI1_SS1"),
MA35_MUX(0xe, "QEI2_INDEX"),
MA35_MUX(0xf, "I2S0_MCLK")),
- MA35_PIN(91, PF15, 0xac, 0x1c,
+ MA35_PIN(95, PF15, 0xac, 0x1c,
MA35_MUX(0x0, "GPF15"),
MA35_MUX(0x1, "HSUSB0_VBUSVLD")),
- MA35_PIN(92, PG0, 0xb0, 0x0,
+ MA35_PIN(96, PG0, 0xb0, 0x0,
MA35_MUX(0x0, "GPG0"),
MA35_MUX(0x1, "EPWM0_CH0"),
MA35_MUX(0x2, "UART7_TXD"),
@@ -697,19 +747,20 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xc, "CLKO"),
MA35_MUX(0xd, "INT0"),
MA35_MUX(0xf, "EBI_ADR15")),
- MA35_PIN(93, PG1, 0xb0, 0x4,
+ MA35_PIN(97, PG1, 0xb0, 0x4,
MA35_MUX(0x0, "GPG1"),
MA35_MUX(0x1, "EPWM0_CH3"),
MA35_MUX(0x2, "UART9_nRTS"),
MA35_MUX(0x3, "UART6_TXD"),
MA35_MUX(0x4, "I2C4_SCL"),
MA35_MUX(0x5, "CAN2_TXD"),
+ MA35_MUX(0x6, "USBHL0_DP"),
MA35_MUX(0x7, "EBI_nCS0"),
MA35_MUX(0x9, "QEI0_B"),
MA35_MUX(0xb, "TM1_EXT"),
MA35_MUX(0xe, "RGMII1_PPS"),
MA35_MUX(0xf, "RMII1_PPS")),
- MA35_PIN(94, PG2, 0xb0, 0x8,
+ MA35_PIN(98, PG2, 0xb0, 0x8,
MA35_MUX(0x0, "GPG2"),
MA35_MUX(0x1, "EPWM0_CH4"),
MA35_MUX(0x2, "UART9_RXD"),
@@ -720,7 +771,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "QEI0_A"),
MA35_MUX(0xb, "TM3"),
MA35_MUX(0xd, "INT1")),
- MA35_PIN(95, PG3, 0xb0, 0xc,
+ MA35_PIN(99, PG3, 0xb0, 0xc,
MA35_MUX(0x0, "GPG3"),
MA35_MUX(0x1, "EPWM0_CH5"),
MA35_MUX(0x2, "UART9_TXD"),
@@ -732,7 +783,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "QEI0_B"),
MA35_MUX(0xb, "TM3_EXT"),
MA35_MUX(0xc, "I2S1_MCLK")),
- MA35_PIN(96, PG4, 0xb0, 0x10,
+ MA35_PIN(100, PG4, 0xb0, 0x10,
MA35_MUX(0x0, "GPG4"),
MA35_MUX(0x1, "EPWM1_CH0"),
MA35_MUX(0x2, "UART5_nCTS"),
@@ -746,7 +797,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xb, "TM4"),
MA35_MUX(0xd, "INT2"),
MA35_MUX(0xe, "ECAP1_IC2")),
- MA35_PIN(97, PG5, 0xb0, 0x14,
+ MA35_PIN(101, PG5, 0xb0, 0x14,
MA35_MUX(0x0, "GPG5"),
MA35_MUX(0x1, "EPWM1_CH1"),
MA35_MUX(0x2, "UART5_nRTS"),
@@ -758,7 +809,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "I2S1_DI"),
MA35_MUX(0xa, "SC1_DAT"),
MA35_MUX(0xb, "TM4_EXT")),
- MA35_PIN(98, PG6, 0xb0, 0x18,
+ MA35_PIN(102, PG6, 0xb0, 0x18,
MA35_MUX(0x0, "GPG6"),
MA35_MUX(0x1, "EPWM1_CH2"),
MA35_MUX(0x2, "UART5_RXD"),
@@ -770,7 +821,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "SC1_RST"),
MA35_MUX(0xb, "TM7"),
MA35_MUX(0xd, "INT3")),
- MA35_PIN(99, PG7, 0xb0, 0x1c,
+ MA35_PIN(103, PG7, 0xb0, 0x1c,
MA35_MUX(0x0, "GPG7"),
MA35_MUX(0x1, "EPWM1_CH3"),
MA35_MUX(0x2, "UART5_TXD"),
@@ -781,27 +832,29 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "I2S1_LRCK"),
MA35_MUX(0xa, "SC1_PWR"),
MA35_MUX(0xb, "TM7_EXT")),
- MA35_PIN(100, PG8, 0xb4, 0x0,
+ MA35_PIN(104, PG8, 0xb4, 0x0,
MA35_MUX(0x0, "GPG8"),
MA35_MUX(0x1, "EPWM1_CH4"),
MA35_MUX(0x2, "UART12_RXD"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL4_DM"),
MA35_MUX(0x5, "SPI2_SS0"),
MA35_MUX(0x6, "LCM_VSYNC"),
MA35_MUX(0x7, "I2C3_SDA"),
MA35_MUX(0xc, "EBI_AD7"),
MA35_MUX(0xd, "EBI_nCS0")),
- MA35_PIN(101, PG9, 0xb4, 0x4,
+ MA35_PIN(105, PG9, 0xb4, 0x4,
MA35_MUX(0x0, "GPG9"),
MA35_MUX(0x1, "EPWM1_CH5"),
MA35_MUX(0x2, "UART12_TXD"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL4_DP"),
MA35_MUX(0x5, "SPI2_CLK"),
MA35_MUX(0x6, "LCM_HSYNC"),
MA35_MUX(0x7, "I2C3_SCL"),
MA35_MUX(0xc, "EBI_AD8"),
MA35_MUX(0xd, "EBI_nCS1")),
- MA35_PIN(102, PG10, 0xb4, 0x8,
+ MA35_PIN(106, PG10, 0xb4, 0x8,
MA35_MUX(0x0, "GPG10"),
MA35_MUX(0x2, "UART12_nRTS"),
MA35_MUX(0x3, "UART13_TXD"),
@@ -809,7 +862,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x6, "LCM_CLK"),
MA35_MUX(0xc, "EBI_AD9"),
MA35_MUX(0xd, "EBI_nWRH")),
- MA35_PIN(103, PG11, 0xb4, 0xc,
+ MA35_PIN(107, PG11, 0xb4, 0xc,
MA35_MUX(0x0, "GPG11"),
MA35_MUX(0x3, "JTAG_TDO"),
MA35_MUX(0x5, "I2S0_MCLK"),
@@ -817,93 +870,93 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x7, "EBI_nWRH"),
MA35_MUX(0x8, "EBI_nCS1"),
MA35_MUX(0xa, "EBI_AD0")),
- MA35_PIN(104, PG12, 0xb4, 0x10,
+ MA35_PIN(108, PG12, 0xb4, 0x10,
MA35_MUX(0x0, "GPG12"),
MA35_MUX(0x3, "JTAG_TCK/SW_CLK"),
MA35_MUX(0x5, "I2S0_LRCK"),
MA35_MUX(0x7, "EBI_nWRL"),
MA35_MUX(0xa, "EBI_AD1")),
- MA35_PIN(105, PG13, 0xb4, 0x14,
+ MA35_PIN(109, PG13, 0xb4, 0x14,
MA35_MUX(0x0, "GPG13"),
MA35_MUX(0x3, "JTAG_TMS/SW_DIO"),
MA35_MUX(0x5, "I2S0_BCLK"),
MA35_MUX(0x7, "EBI_MCLK"),
MA35_MUX(0xa, "EBI_AD2")),
- MA35_PIN(106, PG14, 0xb4, 0x18,
+ MA35_PIN(110, PG14, 0xb4, 0x18,
MA35_MUX(0x0, "GPG14"),
MA35_MUX(0x3, "JTAG_TDI"),
MA35_MUX(0x5, "I2S0_DI"),
MA35_MUX(0x6, "NAND_nCS1"),
MA35_MUX(0x7, "EBI_ALE"),
MA35_MUX(0xa, "EBI_AD3")),
- MA35_PIN(107, PG15, 0xb4, 0x1c,
+ MA35_PIN(111, PG15, 0xb4, 0x1c,
MA35_MUX(0x0, "GPG15"),
MA35_MUX(0x3, "JTAG_nTRST"),
MA35_MUX(0x5, "I2S0_DO"),
MA35_MUX(0x7, "EBI_nCS0"),
MA35_MUX(0xa, "EBI_AD4")),
- MA35_PIN(108, PH0, 0xb8, 0x0,
+ MA35_PIN(112, PH0, 0xb8, 0x0,
MA35_MUX(0x0, "GPH0"),
MA35_MUX(0x2, "UART8_nCTS"),
MA35_MUX(0x3, "UART7_RXD"),
MA35_MUX(0x6, "LCM_DATA8")),
- MA35_PIN(109, PH1, 0xb8, 0x4,
+ MA35_PIN(113, PH1, 0xb8, 0x4,
MA35_MUX(0x0, "GPH1"),
MA35_MUX(0x2, "UART8_nRTS"),
MA35_MUX(0x3, "UART7_TXD"),
MA35_MUX(0x6, "LCM_DATA9")),
- MA35_PIN(110, PH2, 0xb8, 0x8,
+ MA35_PIN(114, PH2, 0xb8, 0x8,
MA35_MUX(0x0, "GPH2"),
MA35_MUX(0x2, "UART8_RXD"),
MA35_MUX(0x6, "LCM_DATA10")),
- MA35_PIN(111, PH3, 0xb8, 0xc,
+ MA35_PIN(115, PH3, 0xb8, 0xc,
MA35_MUX(0x0, "GPH3"),
MA35_MUX(0x2, "UART8_TXD"),
MA35_MUX(0x6, "LCM_DATA11")),
- MA35_PIN(112, PH4, 0xb8, 0x10,
+ MA35_PIN(116, PH4, 0xb8, 0x10,
MA35_MUX(0x0, "GPH4"),
MA35_MUX(0x2, "UART10_nCTS"),
MA35_MUX(0x3, "UART9_RXD"),
MA35_MUX(0x6, "LCM_DATA12")),
- MA35_PIN(113, PH5, 0xb8, 0x14,
+ MA35_PIN(117, PH5, 0xb8, 0x14,
MA35_MUX(0x0, "GPH5"),
MA35_MUX(0x2, "UART10_nRTS"),
MA35_MUX(0x3, "UART9_TXD"),
MA35_MUX(0x6, "LCM_DATA13")),
- MA35_PIN(114, PH6, 0xb8, 0x18,
+ MA35_PIN(118, PH6, 0xb8, 0x18,
MA35_MUX(0x0, "GPH6"),
MA35_MUX(0x2, "UART10_RXD"),
MA35_MUX(0x6, "LCM_DATA14")),
- MA35_PIN(115, PH7, 0xb8, 0x1c,
+ MA35_PIN(119, PH7, 0xb8, 0x1c,
MA35_MUX(0x0, "GPH7"),
MA35_MUX(0x2, "UART10_TXD"),
MA35_MUX(0x6, "LCM_DATA15")),
- MA35_PIN(116, PH8, 0xbc, 0x0,
+ MA35_PIN(120, PH8, 0xbc, 0x0,
MA35_MUX(0x0, "GPH8"),
MA35_MUX(0x6, "TAMPER0")),
- MA35_PIN(117, PH9, 0xbc, 0x4,
+ MA35_PIN(121, PH9, 0xbc, 0x4,
MA35_MUX(0x0, "GPH9"),
MA35_MUX(0x4, "CLK_32KOUT"),
MA35_MUX(0x6, "TAMPER1")),
- MA35_PIN(118, PH12, 0xbc, 0x10,
+ MA35_PIN(124, PH12, 0xbc, 0x10,
MA35_MUX(0x0, "GPH12"),
MA35_MUX(0x2, "UART14_nCTS"),
MA35_MUX(0x3, "UART13_RXD"),
MA35_MUX(0x6, "LCM_DATA20")),
- MA35_PIN(119, PH13, 0xbc, 0x14,
+ MA35_PIN(125, PH13, 0xbc, 0x14,
MA35_MUX(0x0, "GPH13"),
MA35_MUX(0x2, "UART14_nRTS"),
MA35_MUX(0x3, "UART13_TXD"),
MA35_MUX(0x6, "LCM_DATA21")),
- MA35_PIN(120, PH14, 0xbc, 0x18,
+ MA35_PIN(126, PH14, 0xbc, 0x18,
MA35_MUX(0x0, "GPH14"),
MA35_MUX(0x2, "UART14_RXD"),
MA35_MUX(0x6, "LCM_DATA22")),
- MA35_PIN(121, PH15, 0xbc, 0x1c,
+ MA35_PIN(127, PH15, 0xbc, 0x1c,
MA35_MUX(0x0, "GPH15"),
MA35_MUX(0x2, "UART14_TXD"),
MA35_MUX(0x6, "LCM_DATA23")),
- MA35_PIN(122, PI0, 0xc0, 0x0,
+ MA35_PIN(128, PI0, 0xc0, 0x0,
MA35_MUX(0x0, "GPI0"),
MA35_MUX(0x1, "EPWM0_CH0"),
MA35_MUX(0x2, "UART12_nCTS"),
@@ -914,7 +967,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "EBI_ADR0"),
MA35_MUX(0xb, "TM0"),
MA35_MUX(0xc, "ECAP1_IC0")),
- MA35_PIN(123, PI1, 0xc0, 0x4,
+ MA35_PIN(129, PI1, 0xc0, 0x4,
MA35_MUX(0x0, "GPI1"),
MA35_MUX(0x1, "EPWM0_CH1"),
MA35_MUX(0x2, "UART12_nRTS"),
@@ -925,26 +978,28 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "EBI_ADR1"),
MA35_MUX(0xb, "TM0_EXT"),
MA35_MUX(0xc, "ECAP1_IC1")),
- MA35_PIN(124, PI2, 0xc0, 0x8,
+ MA35_PIN(130, PI2, 0xc0, 0x8,
MA35_MUX(0x0, "GPI2"),
MA35_MUX(0x1, "EPWM0_CH2"),
MA35_MUX(0x2, "UART12_RXD"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL2_DM"),
MA35_MUX(0x5, "SPI3_MOSI"),
MA35_MUX(0x7, "SC0_DAT"),
MA35_MUX(0x8, "EBI_ADR2"),
MA35_MUX(0xb, "TM1"),
MA35_MUX(0xc, "ECAP1_IC2")),
- MA35_PIN(125, PI3, 0xc0, 0xc,
+ MA35_PIN(131, PI3, 0xc0, 0xc,
MA35_MUX(0x0, "GPI3"),
MA35_MUX(0x1, "EPWM0_CH3"),
MA35_MUX(0x2, "UART12_TXD"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL2_DP"),
MA35_MUX(0x5, "SPI3_MISO"),
MA35_MUX(0x7, "SC0_RST"),
MA35_MUX(0x8, "EBI_ADR3"),
MA35_MUX(0xb, "TM1_EXT")),
- MA35_PIN(126, PI4, 0xc0, 0x10,
+ MA35_PIN(132, PI4, 0xc0, 0x10,
MA35_MUX(0x0, "GPI4"),
MA35_MUX(0x1, "EPWM0_CH4"),
MA35_MUX(0x2, "UART14_nCTS"),
@@ -954,7 +1009,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x6, "I2S1_LRCK"),
MA35_MUX(0x8, "EBI_ADR4"),
MA35_MUX(0xd, "INT0")),
- MA35_PIN(127, PI5, 0xc0, 0x14,
+ MA35_PIN(133, PI5, 0xc0, 0x14,
MA35_MUX(0x0, "GPI5"),
MA35_MUX(0x1, "EPWM0_CH5"),
MA35_MUX(0x2, "UART14_nRTS"),
@@ -963,65 +1018,67 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x6, "I2S1_BCLK"),
MA35_MUX(0x8, "EBI_ADR5"),
MA35_MUX(0xd, "INT1")),
- MA35_PIN(128, PI6, 0xc0, 0x18,
+ MA35_PIN(134, PI6, 0xc0, 0x18,
MA35_MUX(0x0, "GPI6"),
MA35_MUX(0x1, "EPWM0_BRAKE0"),
MA35_MUX(0x2, "UART14_RXD"),
MA35_MUX(0x3, "CAN1_RXD"),
+ MA35_MUX(0x4, "USBHL3_DM"),
MA35_MUX(0x6, "I2S1_DI"),
MA35_MUX(0x8, "EBI_ADR6"),
MA35_MUX(0xc, "QEI1_INDEX"),
MA35_MUX(0xd, "INT2")),
- MA35_PIN(129, PI7, 0xc0, 0x1c,
+ MA35_PIN(135, PI7, 0xc0, 0x1c,
MA35_MUX(0x0, "GPI7"),
MA35_MUX(0x1, "EPWM0_BRAKE1"),
MA35_MUX(0x2, "UART14_TXD"),
MA35_MUX(0x3, "CAN1_TXD"),
+ MA35_MUX(0x4, "USBHL3_DP"),
MA35_MUX(0x6, "I2S1_DO"),
MA35_MUX(0x8, "EBI_ADR7"),
MA35_MUX(0xc, "ECAP0_IC0"),
MA35_MUX(0xd, "INT3")),
- MA35_PIN(130, PI8, 0xc4, 0x0,
+ MA35_PIN(136, PI8, 0xc4, 0x0,
MA35_MUX(0x0, "GPI8"),
MA35_MUX(0x2, "UART4_nCTS"),
MA35_MUX(0x3, "UART3_RXD"),
MA35_MUX(0x6, "LCM_DATA0"),
MA35_MUX(0xc, "EBI_AD11")),
- MA35_PIN(131, PI9, 0xc4, 0x4,
+ MA35_PIN(137, PI9, 0xc4, 0x4,
MA35_MUX(0x0, "GPI9"),
MA35_MUX(0x2, "UART4_nRTS"),
MA35_MUX(0x3, "UART3_TXD"),
MA35_MUX(0x6, "LCM_DATA1"),
MA35_MUX(0xc, "EBI_AD12")),
- MA35_PIN(132, PI10, 0xc4, 0x8,
+ MA35_PIN(138, PI10, 0xc4, 0x8,
MA35_MUX(0x0, "GPI10"),
MA35_MUX(0x2, "UART4_RXD"),
MA35_MUX(0x6, "LCM_DATA2"),
MA35_MUX(0xc, "EBI_AD13")),
- MA35_PIN(133, PI11, 0xC4, 0xc,
+ MA35_PIN(139, PI11, 0xC4, 0xc,
MA35_MUX(0x0, "GPI11"),
MA35_MUX(0x2, "UART4_TXD"),
MA35_MUX(0x6, "LCM_DATA3"),
MA35_MUX(0xc, "EBI_AD14")),
- MA35_PIN(134, PI12, 0xc4, 0x10,
+ MA35_PIN(140, PI12, 0xc4, 0x10,
MA35_MUX(0x0, "GPI12"),
MA35_MUX(0x2, "UART6_nCTS"),
MA35_MUX(0x3, "UART5_RXD"),
MA35_MUX(0x6, "LCM_DATA4")),
- MA35_PIN(135, PI13, 0xc4, 0x14,
+ MA35_PIN(141, PI13, 0xc4, 0x14,
MA35_MUX(0x0, "GPI13"),
MA35_MUX(0x2, "UART6_nRTS"),
MA35_MUX(0x3, "UART5_TXD"),
MA35_MUX(0x6, "LCM_DATA5")),
- MA35_PIN(136, PI14, 0xc4, 0x18,
+ MA35_PIN(142, PI14, 0xc4, 0x18,
MA35_MUX(0x0, "GPI14"),
MA35_MUX(0x2, "UART6_RXD"),
MA35_MUX(0x6, "LCM_DATA6")),
- MA35_PIN(137, PI15, 0xc4, 0x1c,
+ MA35_PIN(143, PI15, 0xc4, 0x1c,
MA35_MUX(0x0, "GPI15"),
MA35_MUX(0x2, "UART6_TXD"),
MA35_MUX(0x6, "LCM_DATA7")),
- MA35_PIN(138, PJ0, 0xc8, 0x0,
+ MA35_PIN(144, PJ0, 0xc8, 0x0,
MA35_MUX(0x0, "GPJ0"),
MA35_MUX(0x1, "EPWM1_BRAKE0"),
MA35_MUX(0x2, "UART8_nCTS"),
@@ -1035,7 +1092,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "EBI_ADR16"),
MA35_MUX(0xb, "EBI_nCS0"),
MA35_MUX(0xc, "EBI_AD7")),
- MA35_PIN(139, PJ1, 0xc8, 0x4,
+ MA35_PIN(145, PJ1, 0xc8, 0x4,
MA35_MUX(0x0, "GPJ1"),
MA35_MUX(0x1, "EPWM1_BRAKE1"),
MA35_MUX(0x2, "UART8_nRTS"),
@@ -1049,11 +1106,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "EBI_ADR17"),
MA35_MUX(0xb, "EBI_nCS1"),
MA35_MUX(0xc, "EBI_AD8")),
- MA35_PIN(140, PJ2, 0xc8, 0x8,
+ MA35_PIN(146, PJ2, 0xc8, 0x8,
MA35_MUX(0x0, "GPJ2"),
MA35_MUX(0x1, "EPWM1_CH4"),
MA35_MUX(0x2, "UART8_RXD"),
MA35_MUX(0x3, "CAN1_RXD"),
+ MA35_MUX(0x4, "USBHL5_DM"),
MA35_MUX(0x5, "SPI2_MOSI"),
MA35_MUX(0x6, "eMMC1_DAT6"),
MA35_MUX(0x7, "I2S0_DI"),
@@ -1062,11 +1120,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "EBI_ADR18"),
MA35_MUX(0xb, "EBI_nWRH"),
MA35_MUX(0xc, "EBI_AD9")),
- MA35_PIN(141, PJ3, 0xc8, 0xc,
+ MA35_PIN(147, PJ3, 0xc8, 0xc,
MA35_MUX(0x0, "GPJ3"),
MA35_MUX(0x1, "EPWM1_CH5"),
MA35_MUX(0x2, "UART8_TXD"),
MA35_MUX(0x3, "CAN1_TXD"),
+ MA35_MUX(0x4, "USBHL5_DP"),
MA35_MUX(0x5, "SPI2_MISO"),
MA35_MUX(0x6, "eMMC1_DAT7"),
MA35_MUX(0x7, "I2S0_DO"),
@@ -1075,39 +1134,43 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "EBI_ADR19"),
MA35_MUX(0xb, "EBI_nWRL"),
MA35_MUX(0xc, "EBI_AD10")),
- MA35_PIN(142, PJ4, 0xc8, 0x10,
+ MA35_PIN(148, PJ4, 0xc8, 0x10,
MA35_MUX(0x0, "GPJ4"),
MA35_MUX(0x4, "I2C3_SDA"),
MA35_MUX(0x6, "SD1_WP")),
- MA35_PIN(143, PJ5, 0xc8, 0x14,
+ MA35_PIN(149, PJ5, 0xc8, 0x14,
MA35_MUX(0x0, "GPJ5"),
MA35_MUX(0x4, "I2C3_SCL"),
MA35_MUX(0x6, "SD1_nCD")),
- MA35_PIN(144, PJ6, 0xc8, 0x18,
+ MA35_PIN(150, PJ6, 0xc8, 0x18,
MA35_MUX(0x0, "GPJ6"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL0_DM"),
MA35_MUX(0x6, "SD1_CMD/eMMC1_CMD")),
- MA35_PIN(145, PJ7, 0xc8, 0x1c,
+ MA35_PIN(151, PJ7, 0xc8, 0x1c,
MA35_MUX(0x0, "GPJ7"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL0_DP"),
MA35_MUX(0x6, "SD1_CLK/eMMC1_CLK")),
- MA35_PIN(146, PJ8, 0xcc, 0x0,
+ MA35_PIN(152, PJ8, 0xcc, 0x0,
MA35_MUX(0x0, "GPJ8"),
MA35_MUX(0x4, "I2C4_SDA"),
MA35_MUX(0x6, "SD1_DAT0/eMMC1_DAT0")),
- MA35_PIN(147, PJ9, 0xcc, 0x4,
+ MA35_PIN(153, PJ9, 0xcc, 0x4,
MA35_MUX(0x0, "GPJ9"),
MA35_MUX(0x4, "I2C4_SCL"),
MA35_MUX(0x6, "SD1_DAT1/eMMC1_DAT1")),
- MA35_PIN(148, PJ10, 0xcc, 0x8,
+ MA35_PIN(154, PJ10, 0xcc, 0x8,
MA35_MUX(0x0, "GPJ10"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL1_DM"),
MA35_MUX(0x6, "SD1_DAT2/eMMC1_DAT2")),
- MA35_PIN(149, PJ11, 0xcc, 0xc,
+ MA35_PIN(155, PJ11, 0xcc, 0xc,
MA35_MUX(0x0, "GPJ11"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL1_DP"),
MA35_MUX(0x6, "SD1_DAT3/eMMC1_DAT3")),
- MA35_PIN(150, PJ12, 0xcc, 0x10,
+ MA35_PIN(156, PJ12, 0xcc, 0x10,
MA35_MUX(0x0, "GPJ12"),
MA35_MUX(0x1, "EPWM1_CH2"),
MA35_MUX(0x2, "UART2_nCTS"),
@@ -1118,7 +1181,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "EBI_ADR12"),
MA35_MUX(0xb, "TM2"),
MA35_MUX(0xc, "QEI0_INDEX")),
- MA35_PIN(151, PJ13, 0xcc, 0x14,
+ MA35_PIN(157, PJ13, 0xcc, 0x14,
MA35_MUX(0x0, "GPJ13"),
MA35_MUX(0x1, "EPWM1_CH3"),
MA35_MUX(0x2, "UART2_nRTS"),
@@ -1128,27 +1191,29 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x7, "SC1_DAT"),
MA35_MUX(0x8, "EBI_ADR13"),
MA35_MUX(0xb, "TM2_EXT")),
- MA35_PIN(152, PJ14, 0xcc, 0x18,
+ MA35_PIN(158, PJ14, 0xcc, 0x18,
MA35_MUX(0x0, "GPJ14"),
MA35_MUX(0x1, "EPWM1_CH4"),
MA35_MUX(0x2, "UART2_RXD"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL5_DM"),
MA35_MUX(0x5, "SPI3_MISO"),
MA35_MUX(0x7, "SC1_RST"),
MA35_MUX(0x8, "EBI_ADR14"),
MA35_MUX(0xb, "TM3")),
- MA35_PIN(153, PJ15, 0xcc, 0x1c,
+ MA35_PIN(159, PJ15, 0xcc, 0x1c,
MA35_MUX(0x0, "GPJ15"),
MA35_MUX(0x1, "EPWM1_CH5"),
MA35_MUX(0x2, "UART2_TXD"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL5_DP"),
MA35_MUX(0x5, "SPI3_CLK"),
MA35_MUX(0x6, "EADC0_ST"),
MA35_MUX(0x7, "SC1_PWR"),
MA35_MUX(0x8, "EBI_ADR15"),
MA35_MUX(0xb, "TM3_EXT"),
MA35_MUX(0xd, "INT1")),
- MA35_PIN(154, PK0, 0xd0, 0x0,
+ MA35_PIN(160, PK0, 0xd0, 0x0,
MA35_MUX(0x0, "GPK0"),
MA35_MUX(0x1, "EPWM0_SYNC_IN"),
MA35_MUX(0x2, "UART16_nCTS"),
@@ -1158,7 +1223,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "EBI_ADR8"),
MA35_MUX(0xb, "TM7"),
MA35_MUX(0xc, "ECAP0_IC1")),
- MA35_PIN(155, PK1, 0xd0, 0x4,
+ MA35_PIN(161, PK1, 0xd0, 0x4,
MA35_MUX(0x0, "GPK1"),
MA35_MUX(0x1, "EPWM0_SYNC_OUT"),
MA35_MUX(0x2, "UART16_nRTS"),
@@ -1168,25 +1233,27 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "EBI_ADR9"),
MA35_MUX(0xb, "TM7_EXT"),
MA35_MUX(0xc, "ECAP0_IC2")),
- MA35_PIN(156, PK2, 0xd0, 0x8,
+ MA35_PIN(162, PK2, 0xd0, 0x8,
MA35_MUX(0x0, "GPK2"),
MA35_MUX(0x1, "EPWM1_CH0"),
MA35_MUX(0x2, "UART16_RXD"),
MA35_MUX(0x3, "CAN2_RXD"),
+ MA35_MUX(0x4, "USBHL4_DM"),
MA35_MUX(0x5, "SPI3_I2SMCLK"),
MA35_MUX(0x7, "SC0_PWR"),
MA35_MUX(0x8, "EBI_ADR10"),
MA35_MUX(0xc, "QEI0_A")),
- MA35_PIN(157, PK3, 0xd0, 0xc,
+ MA35_PIN(163, PK3, 0xd0, 0xc,
MA35_MUX(0x0, "GPK3"),
MA35_MUX(0x1, "EPWM1_CH1"),
MA35_MUX(0x2, "UART16_TXD"),
MA35_MUX(0x3, "CAN2_TXD"),
+ MA35_MUX(0x4, "USBHL4_DP"),
MA35_MUX(0x5, "SPI3_SS1"),
MA35_MUX(0x7, "SC1_nCD"),
MA35_MUX(0x8, "EBI_ADR11"),
MA35_MUX(0xc, "QEI0_B")),
- MA35_PIN(158, PK4, 0xd0, 0x10,
+ MA35_PIN(164, PK4, 0xd0, 0x10,
MA35_MUX(0x0, "GPK4"),
MA35_MUX(0x2, "UART12_nCTS"),
MA35_MUX(0x3, "UART13_RXD"),
@@ -1194,7 +1261,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x6, "LCM_DEN"),
MA35_MUX(0xc, "EBI_AD10"),
MA35_MUX(0xd, "EBI_nWRL")),
- MA35_PIN(159, PK5, 0xd0, 0x14,
+ MA35_PIN(165, PK5, 0xd0, 0x14,
MA35_MUX(0x0, "GPK5"),
MA35_MUX(0x1, "EPWM1_CH1"),
MA35_MUX(0x2, "UART12_nRTS"),
@@ -1206,28 +1273,30 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "EADC0_ST"),
MA35_MUX(0xb, "TM8_EXT"),
MA35_MUX(0xd, "INT1")),
- MA35_PIN(160, PK6, 0xd0, 0x18,
+ MA35_PIN(166, PK6, 0xd0, 0x18,
MA35_MUX(0x0, "GPK6"),
MA35_MUX(0x1, "EPWM1_CH2"),
MA35_MUX(0x2, "UART12_RXD"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL4_DM"),
MA35_MUX(0x5, "SPI2_MOSI"),
MA35_MUX(0x7, "I2S1_BCLK"),
MA35_MUX(0x8, "SC0_RST"),
MA35_MUX(0xb, "TM6"),
MA35_MUX(0xd, "INT2")),
- MA35_PIN(161, PK7, 0xd0, 0x1c,
+ MA35_PIN(167, PK7, 0xd0, 0x1c,
MA35_MUX(0x0, "GPK7"),
MA35_MUX(0x1, "EPWM1_CH3"),
MA35_MUX(0x2, "UART12_TXD"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL4_DP"),
MA35_MUX(0x5, "SPI2_MISO"),
MA35_MUX(0x7, "I2S1_LRCK"),
MA35_MUX(0x8, "SC0_PWR"),
MA35_MUX(0x9, "CLKO"),
MA35_MUX(0xb, "TM6_EXT"),
MA35_MUX(0xd, "INT3")),
- MA35_PIN(162, PK8, 0xd4, 0x0,
+ MA35_PIN(168, PK8, 0xd4, 0x0,
MA35_MUX(0x0, "GPK8"),
MA35_MUX(0x1, "EPWM1_CH0"),
MA35_MUX(0x4, "I2C3_SDA"),
@@ -1238,25 +1307,27 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xa, "EBI_ADR15"),
MA35_MUX(0xb, "TM8"),
MA35_MUX(0xc, "QEI1_INDEX")),
- MA35_PIN(163, PK9, 0xd4, 0x4,
+ MA35_PIN(169, PK9, 0xd4, 0x4,
MA35_MUX(0x0, "GPK9"),
MA35_MUX(0x4, "I2C3_SCL"),
MA35_MUX(0x6, "CCAP0_SCLK"),
MA35_MUX(0x8, "EBI_AD0"),
MA35_MUX(0xa, "EBI_ADR0")),
- MA35_PIN(164, PK10, 0xd4, 0x8,
+ MA35_PIN(170, PK10, 0xd4, 0x8,
MA35_MUX(0x0, "GPK10"),
MA35_MUX(0x3, "CAN1_RXD"),
+ MA35_MUX(0x4, "USBHL3_DM"),
MA35_MUX(0x6, "CCAP0_PIXCLK"),
MA35_MUX(0x8, "EBI_AD1"),
MA35_MUX(0xa, "EBI_ADR1")),
- MA35_PIN(165, PK11, 0xd4, 0xc,
+ MA35_PIN(171, PK11, 0xd4, 0xc,
MA35_MUX(0x0, "GPK11"),
MA35_MUX(0x3, "CAN1_TXD"),
+ MA35_MUX(0x4, "USBHL3_DP"),
MA35_MUX(0x6, "CCAP0_HSYNC"),
MA35_MUX(0x8, "EBI_AD2"),
MA35_MUX(0xa, "EBI_ADR2")),
- MA35_PIN(166, PK12, 0xd4, 0x10,
+ MA35_PIN(172, PK12, 0xd4, 0x10,
MA35_MUX(0x0, "GPK12"),
MA35_MUX(0x1, "EPWM2_CH0"),
MA35_MUX(0x2, "UART1_nCTS"),
@@ -1267,7 +1338,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "SC0_CLK"),
MA35_MUX(0xb, "TM10"),
MA35_MUX(0xd, "INT2")),
- MA35_PIN(167, PK13, 0xd4, 0x14,
+ MA35_PIN(173, PK13, 0xd4, 0x14,
MA35_MUX(0x0, "GPK13"),
MA35_MUX(0x1, "EPWM2_CH1"),
MA35_MUX(0x2, "UART1_nRTS"),
@@ -1277,28 +1348,30 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x6, "SPI1_CLK"),
MA35_MUX(0x8, "SC0_DAT"),
MA35_MUX(0xb, "TM10_EXT")),
- MA35_PIN(168, PK14, 0xd4, 0x18,
+ MA35_PIN(174, PK14, 0xd4, 0x18,
MA35_MUX(0x0, "GPK14"),
MA35_MUX(0x1, "EPWM2_CH2"),
MA35_MUX(0x2, "UART1_RXD"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL4_DM"),
MA35_MUX(0x5, "I2S0_DI"),
MA35_MUX(0x6, "SPI1_MOSI"),
MA35_MUX(0x8, "SC0_RST"),
MA35_MUX(0xa, "I2C5_SDA"),
MA35_MUX(0xb, "TM11"),
MA35_MUX(0xd, "INT3")),
- MA35_PIN(169, PK15, 0xd4, 0x1c,
+ MA35_PIN(175, PK15, 0xd4, 0x1c,
MA35_MUX(0x0, "GPK15"),
MA35_MUX(0x1, "EPWM2_CH3"),
MA35_MUX(0x2, "UART1_TXD"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL4_DP"),
MA35_MUX(0x5, "I2S0_DO"),
MA35_MUX(0x6, "SPI1_MISO"),
MA35_MUX(0x8, "SC0_PWR"),
MA35_MUX(0xa, "I2C5_SCL"),
MA35_MUX(0xb, "TM11_EXT")),
- MA35_PIN(170, PL0, 0xd8, 0x0,
+ MA35_PIN(176, PL0, 0xd8, 0x0,
MA35_MUX(0x0, "GPL0"),
MA35_MUX(0x1, "EPWM1_CH0"),
MA35_MUX(0x2, "UART11_nCTS"),
@@ -1311,7 +1384,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC1_CLK"),
MA35_MUX(0xb, "TM5"),
MA35_MUX(0xc, "QEI1_A")),
- MA35_PIN(171, PL1, 0xd8, 0x4,
+ MA35_PIN(177, PL1, 0xd8, 0x4,
MA35_MUX(0x0, "GPL1"),
MA35_MUX(0x1, "EPWM1_CH1"),
MA35_MUX(0x2, "UART11_nRTS"),
@@ -1324,11 +1397,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC1_DAT"),
MA35_MUX(0xb, "TM5_EXT"),
MA35_MUX(0xc, "QEI1_B")),
- MA35_PIN(172, PL2, 0xd8, 0x8,
+ MA35_PIN(178, PL2, 0xd8, 0x8,
MA35_MUX(0x0, "GPL2"),
MA35_MUX(0x1, "EPWM1_CH2"),
MA35_MUX(0x2, "UART11_RXD"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL4_DM"),
MA35_MUX(0x5, "SPI2_SS0"),
MA35_MUX(0x6, "QSPI1_SS1"),
MA35_MUX(0x7, "I2S0_DI"),
@@ -1336,11 +1410,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC1_RST"),
MA35_MUX(0xb, "TM7"),
MA35_MUX(0xc, "QEI1_INDEX")),
- MA35_PIN(173, PL3, 0xd8, 0xc,
+ MA35_PIN(179, PL3, 0xd8, 0xc,
MA35_MUX(0x0, "GPL3"),
MA35_MUX(0x1, "EPWM1_CH3"),
MA35_MUX(0x2, "UART11_TXD"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL4_DP"),
MA35_MUX(0x5, "SPI2_CLK"),
MA35_MUX(0x6, "QSPI1_CLK"),
MA35_MUX(0x7, "I2S0_DO"),
@@ -1348,7 +1423,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC1_PWR"),
MA35_MUX(0xb, "TM7_EXT"),
MA35_MUX(0xc, "ECAP0_IC0")),
- MA35_PIN(174, PL4, 0xd8, 0x10,
+ MA35_PIN(180, PL4, 0xd8, 0x10,
MA35_MUX(0x0, "GPL4"),
MA35_MUX(0x1, "EPWM1_CH4"),
MA35_MUX(0x2, "UART2_nCTS"),
@@ -1361,7 +1436,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC1_nCD"),
MA35_MUX(0xb, "TM9"),
MA35_MUX(0xc, "ECAP0_IC1")),
- MA35_PIN(175, PL5, 0xd8, 0x14,
+ MA35_PIN(181, PL5, 0xd8, 0x14,
MA35_MUX(0x0, "GPL5"),
MA35_MUX(0x1, "EPWM1_CH5"),
MA35_MUX(0x2, "UART2_nRTS"),
@@ -1374,28 +1449,30 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC0_nCD"),
MA35_MUX(0xb, "TM9_EXT"),
MA35_MUX(0xc, "ECAP0_IC2")),
- MA35_PIN(176, PL6, 0xd8, 0x18,
+ MA35_PIN(182, PL6, 0xd8, 0x18,
MA35_MUX(0x0, "GPL6"),
MA35_MUX(0x1, "EPWM0_CH0"),
MA35_MUX(0x2, "UART2_RXD"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL5_DM"),
MA35_MUX(0x6, "QSPI1_MOSI1"),
MA35_MUX(0x7, "TRACE_CLK"),
MA35_MUX(0x8, "EBI_AD5"),
MA35_MUX(0xb, "TM3"),
MA35_MUX(0xc, "ECAP1_IC0"),
MA35_MUX(0xd, "INT0")),
- MA35_PIN(177, PL7, 0xd8, 0x1c,
+ MA35_PIN(183, PL7, 0xd8, 0x1c,
MA35_MUX(0x0, "GPL7"),
MA35_MUX(0x1, "EPWM0_CH1"),
MA35_MUX(0x2, "UART2_TXD"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL5_DP"),
MA35_MUX(0x6, "QSPI1_MISO1"),
MA35_MUX(0x8, "EBI_AD6"),
MA35_MUX(0xb, "TM3_EXT"),
MA35_MUX(0xc, "ECAP1_IC1"),
MA35_MUX(0xd, "INT1")),
- MA35_PIN(178, PL8, 0xdc, 0x0,
+ MA35_PIN(184, PL8, 0xdc, 0x0,
MA35_MUX(0x0, "GPL8"),
MA35_MUX(0x1, "EPWM0_CH2"),
MA35_MUX(0x2, "UART14_nCTS"),
@@ -1409,7 +1486,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xb, "TM4"),
MA35_MUX(0xc, "ECAP1_IC2"),
MA35_MUX(0xd, "INT2")),
- MA35_PIN(179, PL9, 0xdc, 0x4,
+ MA35_PIN(185, PL9, 0xdc, 0x4,
MA35_MUX(0x0, "GPL9"),
MA35_MUX(0x1, "EPWM0_CH3"),
MA35_MUX(0x2, "UART14_nRTS"),
@@ -1423,11 +1500,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xb, "TM4_EXT"),
MA35_MUX(0xc, "QEI0_A"),
MA35_MUX(0xd, "INT3")),
- MA35_PIN(180, PL10, 0xdc, 0x8,
+ MA35_PIN(186, PL10, 0xdc, 0x8,
MA35_MUX(0x0, "GPL10"),
MA35_MUX(0x1, "EPWM0_CH4"),
MA35_MUX(0x2, "UART14_RXD"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL2_DM"),
MA35_MUX(0x5, "SPI3_MOSI"),
MA35_MUX(0x6, "EPWM0_CH5"),
MA35_MUX(0x7, "I2S1_DI"),
@@ -1435,11 +1513,12 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC0_RST"),
MA35_MUX(0xb, "EBI_nWRH"),
MA35_MUX(0xc, "QEI0_B")),
- MA35_PIN(181, PL11, 0xdc, 0xc,
+ MA35_PIN(187, PL11, 0xdc, 0xc,
MA35_MUX(0x0, "GPL11"),
MA35_MUX(0x1, "EPWM0_CH5"),
MA35_MUX(0x2, "UART14_TXD"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL2_DP"),
MA35_MUX(0x5, "SPI3_MISO"),
MA35_MUX(0x6, "EPWM1_CH5"),
MA35_MUX(0x7, "I2S1_DO"),
@@ -1447,7 +1526,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x9, "SC0_PWR"),
MA35_MUX(0xb, "EBI_nWRL"),
MA35_MUX(0xc, "QEI0_INDEX")),
- MA35_PIN(182, PL12, 0xdc, 0x10,
+ MA35_PIN(188, PL12, 0xdc, 0x10,
MA35_MUX(0x0, "GPL12"),
MA35_MUX(0x1, "EPWM0_SYNC_IN"),
MA35_MUX(0x2, "UART7_nCTS"),
@@ -1464,7 +1543,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xd, "EBI_AD11"),
MA35_MUX(0xe, "RGMII0_PPS"),
MA35_MUX(0xf, "RMII0_PPS")),
- MA35_PIN(183, PL13, 0xdc, 0x14,
+ MA35_PIN(189, PL13, 0xdc, 0x14,
MA35_MUX(0x0, "GPL13"),
MA35_MUX(0x1, "EPWM0_SYNC_OUT"),
MA35_MUX(0x2, "UART7_nRTS"),
@@ -1481,7 +1560,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xd, "EBI_AD12"),
MA35_MUX(0xe, "RGMII1_PPS"),
MA35_MUX(0xf, "RMII1_PPS")),
- MA35_PIN(184, PL14, 0xdc, 0x18,
+ MA35_PIN(190, PL14, 0xdc, 0x18,
MA35_MUX(0x0, "GPL14"),
MA35_MUX(0x1, "EPWM0_CH2"),
MA35_MUX(0x2, "UART7_RXD"),
@@ -1493,7 +1572,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xb, "TM2"),
MA35_MUX(0xc, "INT0"),
MA35_MUX(0xd, "EBI_AD13")),
- MA35_PIN(185, PL15, 0xdc, 0x1c,
+ MA35_PIN(191, PL15, 0xdc, 0x1c,
MA35_MUX(0x0, "GPL15"),
MA35_MUX(0x1, "EPWM0_CH1"),
MA35_MUX(0x2, "UART7_TXD"),
@@ -1506,86 +1585,92 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0xb, "TM2_EXT"),
MA35_MUX(0xc, "INT2"),
MA35_MUX(0xd, "EBI_AD14")),
- MA35_PIN(186, PM0, 0xe0, 0x0,
+ MA35_PIN(192, PM0, 0xe0, 0x0,
MA35_MUX(0x0, "GPM0"),
MA35_MUX(0x4, "I2C4_SDA"),
MA35_MUX(0x6, "CCAP0_VSYNC"),
MA35_MUX(0x8, "EBI_AD3"),
MA35_MUX(0xa, "EBI_ADR3")),
- MA35_PIN(187, PM1, 0xe0, 0x4,
+ MA35_PIN(193, PM1, 0xe0, 0x4,
MA35_MUX(0x0, "GPM1"),
MA35_MUX(0x4, "I2C4_SCL"),
MA35_MUX(0x5, "SPI3_I2SMCLK"),
MA35_MUX(0x6, "CCAP0_SFIELD"),
MA35_MUX(0x8, "EBI_AD4"),
MA35_MUX(0xa, "EBI_ADR4")),
- MA35_PIN(188, PM2, 0xe0, 0x8,
+ MA35_PIN(194, PM2, 0xe0, 0x8,
MA35_MUX(0x0, "GPM2"),
MA35_MUX(0x3, "CAN3_RXD"),
+ MA35_MUX(0x4, "USBHL0_DM"),
MA35_MUX(0x6, "CCAP0_DATA0"),
MA35_MUX(0x8, "EBI_AD5"),
MA35_MUX(0xa, "EBI_ADR5")),
- MA35_PIN(189, PM3, 0xe0, 0xc,
+ MA35_PIN(195, PM3, 0xe0, 0xc,
MA35_MUX(0x0, "GPM3"),
MA35_MUX(0x3, "CAN3_TXD"),
+ MA35_MUX(0x4, "USBHL0_DP"),
MA35_MUX(0x6, "CCAP0_DATA1"),
MA35_MUX(0x8, "EBI_AD6"),
MA35_MUX(0xa, "EBI_ADR6")),
- MA35_PIN(190, PM4, 0xe0, 0x10,
+ MA35_PIN(196, PM4, 0xe0, 0x10,
MA35_MUX(0x0, "GPM4"),
MA35_MUX(0x4, "I2C5_SDA"),
MA35_MUX(0x6, "CCAP0_DATA2"),
MA35_MUX(0x8, "EBI_AD7"),
MA35_MUX(0xa, "EBI_ADR7")),
- MA35_PIN(191, PM5, 0xe0, 0x14,
+ MA35_PIN(197, PM5, 0xe0, 0x14,
MA35_MUX(0x0, "GPM5"),
MA35_MUX(0x4, "I2C5_SCL"),
MA35_MUX(0x6, "CCAP0_DATA3"),
MA35_MUX(0x8, "EBI_AD8"),
MA35_MUX(0xa, "EBI_ADR8")),
- MA35_PIN(192, PM6, 0xe0, 0x18,
+ MA35_PIN(198, PM6, 0xe0, 0x18,
MA35_MUX(0x0, "GPM6"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL1_DM"),
MA35_MUX(0x6, "CCAP0_DATA4"),
MA35_MUX(0x8, "EBI_AD9"),
MA35_MUX(0xa, "EBI_ADR9")),
- MA35_PIN(193, PM7, 0xe0, 0x1c,
+ MA35_PIN(199, PM7, 0xe0, 0x1c,
MA35_MUX(0x0, "GPM7"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL1_DP"),
MA35_MUX(0x6, "CCAP0_DATA5"),
MA35_MUX(0x8, "EBI_AD10"),
MA35_MUX(0xa, "EBI_ADR10")),
- MA35_PIN(194, PM8, 0xe4, 0x0,
+ MA35_PIN(200, PM8, 0xe4, 0x0,
MA35_MUX(0x0, "GPM8"),
MA35_MUX(0x4, "I2C0_SDA"),
MA35_MUX(0x6, "CCAP0_DATA6"),
MA35_MUX(0x8, "EBI_AD11"),
MA35_MUX(0xa, "EBI_ADR11")),
- MA35_PIN(195, PM9, 0xe4, 0x4,
+ MA35_PIN(201, PM9, 0xe4, 0x4,
MA35_MUX(0x0, "GPM9"),
MA35_MUX(0x4, "I2C0_SCL"),
MA35_MUX(0x6, "CCAP0_DATA7"),
MA35_MUX(0x8, "EBI_AD12"),
MA35_MUX(0xa, "EBI_ADR12")),
- MA35_PIN(196, PM10, 0xe4, 0x8,
+ MA35_PIN(202, PM10, 0xe4, 0x8,
MA35_MUX(0x0, "GPM10"),
MA35_MUX(0x1, "EPWM1_CH2"),
MA35_MUX(0x3, "CAN2_RXD"),
+ MA35_MUX(0x4, "USBHL4_DM"),
MA35_MUX(0x5, "SPI3_SS0"),
MA35_MUX(0x6, "CCAP0_DATA8"),
MA35_MUX(0x7, "SPI2_I2SMCLK"),
MA35_MUX(0x8, "EBI_AD13"),
MA35_MUX(0xa, "EBI_ADR13")),
- MA35_PIN(197, PM11, 0xe4, 0xc,
+ MA35_PIN(203, PM11, 0xe4, 0xc,
MA35_MUX(0x0, "GPM11"),
MA35_MUX(0x1, "EPWM1_CH3"),
MA35_MUX(0x3, "CAN2_TXD"),
+ MA35_MUX(0x4, "USBHL4_DP"),
MA35_MUX(0x5, "SPI3_SS1"),
MA35_MUX(0x6, "CCAP0_DATA9"),
MA35_MUX(0x7, "SPI2_SS1"),
MA35_MUX(0x8, "EBI_AD14"),
MA35_MUX(0xa, "EBI_ADR14")),
- MA35_PIN(198, PM12, 0xe4, 0x10,
+ MA35_PIN(204, PM12, 0xe4, 0x10,
MA35_MUX(0x0, "GPM12"),
MA35_MUX(0x1, "EPWM1_CH4"),
MA35_MUX(0x2, "UART10_nCTS"),
@@ -1596,7 +1681,7 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "EBI_AD8"),
MA35_MUX(0x9, "I2S1_MCLK"),
MA35_MUX(0xb, "TM8")),
- MA35_PIN(199, PM13, 0xe4, 0x14,
+ MA35_PIN(205, PM13, 0xe4, 0x14,
MA35_MUX(0x0, "GPM13"),
MA35_MUX(0x1, "EPWM1_CH5"),
MA35_MUX(0x2, "UART10_nRTS"),
@@ -1606,99 +1691,66 @@ static const struct pinctrl_pin_desc ma35d1_pins[] = {
MA35_MUX(0x8, "EBI_AD9"),
MA35_MUX(0x9, "ECAP1_IC0"),
MA35_MUX(0xb, "TM8_EXT")),
- MA35_PIN(200, PM14, 0xe4, 0x18,
+ MA35_PIN(206, PM14, 0xe4, 0x18,
MA35_MUX(0x0, "GPM14"),
MA35_MUX(0x1, "EPWM1_BRAKE0"),
MA35_MUX(0x2, "UART10_RXD"),
MA35_MUX(0x3, "TRACE_DATA2"),
MA35_MUX(0x4, "CAN2_RXD"),
+ MA35_MUX(0x5, "USBHL3_DM"),
MA35_MUX(0x6, "I2C3_SDA"),
MA35_MUX(0x8, "EBI_AD10"),
MA35_MUX(0x9, "ECAP1_IC1"),
MA35_MUX(0xb, "TM10"),
MA35_MUX(0xd, "INT1")),
- MA35_PIN(201, PM15, 0xe4, 0x1c,
+ MA35_PIN(207, PM15, 0xe4, 0x1c,
MA35_MUX(0x0, "GPM15"),
MA35_MUX(0x1, "EPWM1_BRAKE1"),
MA35_MUX(0x2, "UART10_TXD"),
MA35_MUX(0x3, "TRACE_DATA3"),
MA35_MUX(0x4, "CAN2_TXD"),
+ MA35_MUX(0x5, "USBHL3_DP"),
MA35_MUX(0x6, "I2C3_SCL"),
MA35_MUX(0x8, "EBI_AD11"),
MA35_MUX(0x9, "ECAP1_IC2"),
MA35_MUX(0xb, "TM10_EXT"),
MA35_MUX(0xd, "INT2")),
- MA35_PIN(202, PN0, 0xe8, 0x0,
+ MA35_PIN(208, PN0, 0xe8, 0x0,
MA35_MUX(0x0, "GPN0"),
MA35_MUX(0x4, "I2C2_SDA"),
MA35_MUX(0x6, "CCAP1_DATA0")),
- MA35_PIN(203, PN1, 0xe8, 0x4,
+ MA35_PIN(209, PN1, 0xe8, 0x4,
MA35_MUX(0x0, "GPN1"),
MA35_MUX(0x4, "I2C2_SCL"),
MA35_MUX(0x6, "CCAP1_DATA1")),
- MA35_PIN(204, PN2, 0xe8, 0x8,
+ MA35_PIN(210, PN2, 0xe8, 0x8,
MA35_MUX(0x0, "GPN2"),
MA35_MUX(0x3, "CAN0_RXD"),
+ MA35_MUX(0x4, "USBHL0_DM"),
MA35_MUX(0x6, "CCAP1_DATA2")),
- MA35_PIN(205, PN3, 0xe8, 0xc,
+ MA35_PIN(211, PN3, 0xe8, 0xc,
MA35_MUX(0x0, "GPN3"),
MA35_MUX(0x3, "CAN0_TXD"),
+ MA35_MUX(0x4, "USBHL0_DP"),
MA35_MUX(0x6, "CCAP1_DATA3")),
- MA35_PIN(206, PN4, 0xe8, 0x10,
+ MA35_PIN(212, PN4, 0xe8, 0x10,
MA35_MUX(0x0, "GPN4"),
MA35_MUX(0x4, "I2C1_SDA"),
MA35_MUX(0x6, "CCAP1_DATA4")),
- MA35_PIN(207, PN5, 0xe8, 0x14,
+ MA35_PIN(213, PN5, 0xe8, 0x14,
MA35_MUX(0x0, "GPN5"),
MA35_MUX(0x4, "I2C1_SCL"),
MA35_MUX(0x6, "CCAP1_DATA5")),
- MA35_PIN(208, PN6, 0xe8, 0x18,
+ MA35_PIN(214, PN6, 0xe8, 0x18,
MA35_MUX(0x0, "GPN6"),
MA35_MUX(0x3, "CAN1_RXD"),
+ MA35_MUX(0x4, "USBHL1_DM"),
MA35_MUX(0x6, "CCAP1_DATA6")),
- MA35_PIN(209, PN7, 0xe8, 0x1c,
+ MA35_PIN(215, PN7, 0xe8, 0x1c,
MA35_MUX(0x0, "GPN7"),
MA35_MUX(0x3, "CAN1_TXD"),
+ MA35_MUX(0x4, "USBHL1_DP"),
MA35_MUX(0x6, "CCAP1_DATA7")),
- MA35_PIN(210, PN10, 0xec, 0x8,
- MA35_MUX(0x0, "GPN10"),
- MA35_MUX(0x3, "CAN2_RXD"),
- MA35_MUX(0x6, "CCAP1_SCLK")),
- MA35_PIN(211, PN11, 0xec, 0xc,
- MA35_MUX(0x0, "GPN11"),
- MA35_MUX(0x3, "CAN2_TXD"),
- MA35_MUX(0x6, "CCAP1_PIXCLK")),
- MA35_PIN(212, PN12, 0xec, 0x10,
- MA35_MUX(0x0, "GPN12"),
- MA35_MUX(0x2, "UART6_nCTS"),
- MA35_MUX(0x3, "UART12_RXD"),
- MA35_MUX(0x4, "I2C5_SDA"),
- MA35_MUX(0x6, "CCAP1_HSYNC")),
- MA35_PIN(213, PN13, 0xec, 0x14,
- MA35_MUX(0x0, "GPN13"),
- MA35_MUX(0x2, "UART6_nRTS"),
- MA35_MUX(0x3, "UART12_TXD"),
- MA35_MUX(0x4, "I2C5_SCL"),
- MA35_MUX(0x6, "CCAP1_VSYNC")),
- MA35_PIN(214, PN14, 0xec, 0x18,
- MA35_MUX(0x0, "GPN14"),
- MA35_MUX(0x2, "UART6_RXD"),
- MA35_MUX(0x3, "CAN3_RXD"),
- MA35_MUX(0x5, "SPI1_SS1"),
- MA35_MUX(0x6, "CCAP1_SFIELD"),
- MA35_MUX(0x7, "SPI1_I2SMCLK")),
- MA35_PIN(215, PN15, 0xec, 0x1c,
- MA35_MUX(0x0, "GPN15"),
- MA35_MUX(0x1, "EPWM2_CH4"),
- MA35_MUX(0x2, "UART6_TXD"),
- MA35_MUX(0x3, "CAN3_TXD"),
- MA35_MUX(0x5, "I2S0_MCLK"),
- MA35_MUX(0x6, "SPI1_SS1"),
- MA35_MUX(0x7, "SPI1_I2SMCLK"),
- MA35_MUX(0x8, "SC0_nCD"),
- MA35_MUX(0x9, "EADC0_ST"),
- MA35_MUX(0xa, "CLKO"),
- MA35_MUX(0xb, "TM6")),
MA35_PIN(216, PN8, 0xec, 0x0,
MA35_MUX(0x0, "GPN8"),
MA35_MUX(0x1, "EPWM2_CH4"),
diff --git a/drivers/pinctrl/sunxi/pinctrl-sunxi.c b/drivers/pinctrl/sunxi/pinctrl-sunxi.c
index 8fbbdcc52deb..ce811cb43651 100644
--- a/drivers/pinctrl/sunxi/pinctrl-sunxi.c
+++ b/drivers/pinctrl/sunxi/pinctrl-sunxi.c
@@ -880,7 +880,7 @@ static int sunxi_pmx_request(struct pinctrl_dev *pctldev, unsigned offset)
return 0;
out:
- regulator_put(s_reg->regulator);
+ regulator_put(reg);
return ret;
}
diff --git a/drivers/platform/x86/amd/pmc/pmc-quirks.c b/drivers/platform/x86/amd/pmc/pmc-quirks.c
index a3921f8106c1..a3e5f65f9d72 100644
--- a/drivers/platform/x86/amd/pmc/pmc-quirks.c
+++ b/drivers/platform/x86/amd/pmc/pmc-quirks.c
@@ -17,6 +17,7 @@
struct quirk_entry {
u32 s2idle_bug_mmio;
bool spurious_8042;
+ bool need_suspend_delay;
};
static struct quirk_entry quirk_s2idle_bug = {
@@ -27,6 +28,10 @@ static struct quirk_entry quirk_spurious_8042 = {
.spurious_8042 = true,
};
+static struct quirk_entry quirk_s2idle_need_suspend_delay = {
+ .need_suspend_delay = true,
+};
+
static const struct dmi_system_id fwbug_list[] = {
{
.ident = "L14 Gen2 AMD",
@@ -197,6 +202,35 @@ static const struct dmi_system_id fwbug_list[] = {
DMI_MATCH(DMI_PRODUCT_NAME, "82XQ"),
}
},
+ /* https://bugzilla.kernel.org/show_bug.cgi?id=221383 */
+ {
+ .ident = "Zen3-based IdeaPad Slim and similar",
+ .driver_data = &quirk_s2idle_need_suspend_delay,
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"),
+ /*
+ * Note: there are also some Zen2-based 82X* devices that
+ * need different quirks, they're already handled above
+ */
+ DMI_MATCH(DMI_PRODUCT_NAME, "82X"),
+ }
+ },
+ {
+ .ident = "Zen3+-based IdeaPad Slim and similar",
+ .driver_data = &quirk_s2idle_need_suspend_delay,
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "83K"),
+ }
+ },
+ {
+ .ident = "IdeaPad Slim 3 15ARP10 (83MM)",
+ .driver_data = &quirk_s2idle_need_suspend_delay,
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "83MM"),
+ }
+ },
/* https://bugzilla.kernel.org/show_bug.cgi?id=221273 */
{
.ident = "Thinkpad L14 Gen3",
@@ -350,6 +384,11 @@ void amd_pmc_process_restore_quirks(struct amd_pmc_dev *dev)
amd_pmc_skip_nvme_smi_handler(dev->quirks->s2idle_bug_mmio);
}
+bool amd_pmc_quirk_need_suspend_delay(struct amd_pmc_dev *dev)
+{
+ return dev->quirks && dev->quirks->need_suspend_delay;
+}
+
void amd_pmc_quirks_init(struct amd_pmc_dev *dev)
{
const struct dmi_system_id *dmi_id;
diff --git a/drivers/platform/x86/amd/pmc/pmc.c b/drivers/platform/x86/amd/pmc/pmc.c
index 6e8eedb8d521..343f75eeb7f2 100644
--- a/drivers/platform/x86/amd/pmc/pmc.c
+++ b/drivers/platform/x86/amd/pmc/pmc.c
@@ -16,6 +16,7 @@
#include <linux/bits.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
+#include <linux/dmi.h>
#include <linux/io.h>
#include <linux/iopoll.h>
#include <linux/limits.h>
@@ -161,6 +162,11 @@ static bool dump_custom_stb;
module_param(dump_custom_stb, bool, 0644);
MODULE_PARM_DESC(dump_custom_stb, "Enable to dump full STB buffer");
+static int delay_suspend = -1;
+module_param(delay_suspend, int, 0644);
+MODULE_PARM_DESC(delay_suspend,
+ "Delays s2idle by 2.5 seconds to work around buggy ECs, often causing keyboard issues after suspend. 0: don't delay, 1: do delay, -1 (default): let amd_pmc decide. If you need this please report this to: platform-driver-x86@vger.kernel.org");
+
static struct amd_pmc_dev pmc;
static int amd_pmc_send_cmd(struct amd_pmc_dev *dev, u32 arg, u32 *data, u8 msg, bool ret);
static int amd_pmc_read_stb(struct amd_pmc_dev *dev, u32 *buf);
@@ -859,6 +865,72 @@ static int amd_pmc_verify_czn_rtc(struct amd_pmc_dev *pdev, u32 *arg)
return rc;
}
+static bool amd_pmc_intermediate_wakeup_need_delay(struct amd_pmc_dev *pdev)
+{
+ /*
+ * Starting a new HW sleep cycle right after waking from one
+ * can cause electrical problems triggering the over voltage protection.
+ * That is avoided by delaying the next suspend a bit, see also
+ * https://lore.kernel.org/all/20250414162446.3853194-1-superm1@kernel.org/
+ */
+ struct smu_metrics table;
+
+ return get_metrics_table(pdev, &table) == 0 && table.s0i3_last_entry_status;
+}
+
+static bool amd_pmc_want_suspend_delay(struct amd_pmc_dev *pdev)
+{
+ /*
+ * intermediate_wakeup implies that the machine didn't get to deepest sleep
+ * state before - otherwise this function isn't called in amd_pmc_s2idle_check()
+ * because amd_pmc_intermediate_wakeup_need_delay() returns true first.
+ * On some IdeaPads that happens when charging, because the EC seems
+ * to send lots of messages then that wake the machine.
+ *
+ * But even in that case, the sleep here is necessary (on those IdeaPads),
+ * otherwise they wake up completely (resume) after a few seconds.
+ * So this variable is only used to avoid spamming dmesg on each
+ * intermediate wakeup.
+ */
+ bool intermediate_wakeup = !pdev->is_first_check_after_suspend;
+
+ /*
+ * Some Lenovo Laptops (like different IdeaPad 3 Slims) need some
+ * me-time before sleeping or they get uncooperative after waking
+ * up and don't send events for keyboard and lid switch anymore.
+ *
+ * Unfortunately this doesn't entirely fix the problem: It can still
+ * happen when resuming with a timer (wakealarm), but at least the
+ * more common usecases (wakeup by opening lid or pressing a key)
+ * work fine with this workaround.
+ *
+ * See https://bugzilla.kernel.org/show_bug.cgi?id=221383
+ */
+ if (amd_pmc_quirk_need_suspend_delay(pdev)) {
+ /*
+ * delay_suspend=1 force-enables this, otherwise it can be
+ * disabled with disable_workarounds or delay_suspend=0
+ */
+ if (delay_suspend == 1 || (delay_suspend == -1 && !disable_workarounds)) {
+ if (!intermediate_wakeup)
+ dev_info(pdev->dev, "Delaying suspend by 2.5s to avoid platform bug\n");
+ return true;
+ }
+ if (!intermediate_wakeup)
+ dev_info(pdev->dev, "Not delaying suspend because of module parameter, even though your device is assumed to need it!\n");
+ } else if (delay_suspend == 1) {
+ if (!intermediate_wakeup)
+ dev_info(pdev->dev, "Delaying suspend by 2.5s because delay_suspend=1. If this solves problems on your machine, please report this whole line to: platform-driver-x86@vger.kernel.org so it can be automatically detected as affected in the future. System Vendor: \"%s\" Product Name: \"%s\" Product Family: \"%s\" Board Vendor: \"%s\" Board Name: \"%s\"\n",
+ dmi_get_system_info(DMI_SYS_VENDOR) ?: "(Unknown)",
+ dmi_get_system_info(DMI_PRODUCT_NAME) ?: "(Unknown)",
+ dmi_get_system_info(DMI_PRODUCT_FAMILY) ?: "(Unknown)",
+ dmi_get_system_info(DMI_BOARD_VENDOR) ?: "(Unknown)",
+ dmi_get_system_info(DMI_BOARD_NAME) ?: "(Unknown)");
+ return true;
+ }
+ return false;
+}
+
static void amd_pmc_s2idle_prepare(void)
{
struct amd_pmc_dev *pdev = &pmc;
@@ -866,6 +938,9 @@ static void amd_pmc_s2idle_prepare(void)
u8 msg;
u32 arg = 1;
+ /* Reset this variable because this is a fresh suspend */
+ pdev->is_first_check_after_suspend = true;
+
/* Reset and Start SMU logging - to monitor the s0i3 stats */
amd_pmc_setup_smu_logging(pdev);
@@ -893,11 +968,10 @@ static void amd_pmc_s2idle_prepare(void)
static void amd_pmc_s2idle_check(void)
{
struct amd_pmc_dev *pdev = &pmc;
- struct smu_metrics table;
int rc;
- /* Avoid triggering OVP */
- if (!get_metrics_table(pdev, &table) && table.s0i3_last_entry_status)
+ if (amd_pmc_intermediate_wakeup_need_delay(pdev) ||
+ amd_pmc_want_suspend_delay(pdev))
msleep(2500);
/* Dump the IdleMask before we add to the STB */
@@ -906,6 +980,9 @@ static void amd_pmc_s2idle_check(void)
rc = amd_pmc_write_stb(pdev, AMD_PMC_STB_S2IDLE_CHECK);
if (rc)
dev_err(pdev->dev, "error writing to STB: %d\n", rc);
+
+ /* remember that first check after suspend is done (until next prepare) */
+ pdev->is_first_check_after_suspend = false;
}
static int amd_pmc_dump_data(struct amd_pmc_dev *pdev)
diff --git a/drivers/platform/x86/amd/pmc/pmc.h b/drivers/platform/x86/amd/pmc/pmc.h
index 17bc0994f376..89f8891d3e38 100644
--- a/drivers/platform/x86/amd/pmc/pmc.h
+++ b/drivers/platform/x86/amd/pmc/pmc.h
@@ -49,10 +49,12 @@ struct amd_pmc_dev {
struct dentry *dbgfs_dir;
struct quirk_entry *quirks;
bool disable_8042_wakeup;
+ bool is_first_check_after_suspend;
struct amd_mp2_dev *mp2;
};
void amd_pmc_process_restore_quirks(struct amd_pmc_dev *dev);
+bool amd_pmc_quirk_need_suspend_delay(struct amd_pmc_dev *dev);
void amd_pmc_quirks_init(struct amd_pmc_dev *dev);
void amd_mp2_stb_init(struct amd_pmc_dev *dev);
void amd_mp2_stb_deinit(struct amd_pmc_dev *dev);
diff --git a/drivers/platform/x86/dell/dell-laptop.c b/drivers/platform/x86/dell/dell-laptop.c
index 5671bd0deee7..5a65e8b3ca1d 100644
--- a/drivers/platform/x86/dell/dell-laptop.c
+++ b/drivers/platform/x86/dell/dell-laptop.c
@@ -2555,7 +2555,12 @@ static int __init dell_init(void)
if (mute_led_registered)
led_classdev_unregister(&mute_led_cdev);
fail_led:
+ dell_laptop_unregister_notifier(&dell_laptop_notifier);
+ debugfs_remove_recursive(dell_laptop_dir);
dell_battery_exit();
+ kbd_led_exit();
+ if (quirks && quirks->touchpad_led)
+ touchpad_led_exit();
dell_cleanup_rfkill();
fail_rfkill:
platform_device_del(platform_device);
diff --git a/drivers/platform/x86/xo15-ebook.c b/drivers/platform/x86/xo15-ebook.c
index df2bf1c58523..baeee33d6352 100644
--- a/drivers/platform/x86/xo15-ebook.c
+++ b/drivers/platform/x86/xo15-ebook.c
@@ -38,6 +38,7 @@ MODULE_DEVICE_TABLE(acpi, ebook_device_ids);
struct ebook_switch {
struct input_dev *input;
char phys[32]; /* for input device */
+ bool gpe_enabled;
};
static int ebook_send_state(struct acpi_device *device)
@@ -132,7 +133,7 @@ static int ebook_switch_add(struct acpi_device *device)
/* Button's GPE is run-wake GPE */
acpi_enable_gpe(device->wakeup.gpe_device,
device->wakeup.gpe_number);
- device_set_wakeup_enable(&device->dev, true);
+ button->gpe_enabled = true;
}
return 0;
@@ -148,6 +149,10 @@ static void ebook_switch_remove(struct acpi_device *device)
{
struct ebook_switch *button = acpi_driver_data(device);
+ if (button->gpe_enabled)
+ acpi_disable_gpe(device->wakeup.gpe_device,
+ device->wakeup.gpe_number);
+
input_unregister_device(button->input);
kfree(button);
}
diff --git a/drivers/pmdomain/imx/imx8m-blk-ctrl.c b/drivers/pmdomain/imx/imx8m-blk-ctrl.c
index f88e2cb8ab2d..fc604b39cafa 100644
--- a/drivers/pmdomain/imx/imx8m-blk-ctrl.c
+++ b/drivers/pmdomain/imx/imx8m-blk-ctrl.c
@@ -54,6 +54,15 @@ struct imx8m_blk_ctrl_domain_data {
* register.
*/
u32 mipi_phy_rst_mask;
+
+ /*
+ * VC8000E reset de-assertion edge and AXI clock may have a timing issue.
+ * Workaround: Set bit2 (vc8000e_clk_en) of BLK_CLK_EN_CSR to 0 to gate off
+ * both AXI clock and VC8000E clock sent to VC8000E and AXI clock sent to
+ * VPU_NOC m_v_2 interface during VC8000E power up(VC8000E reset is
+ * de-asserted by HW)
+ */
+ bool is_errata_err050531;
};
#define DOMAIN_MAX_CLKS 4
@@ -108,7 +117,11 @@ static int imx8m_blk_ctrl_power_on(struct generic_pm_domain *genpd)
dev_err(bc->dev, "failed to enable clocks\n");
goto bus_put;
}
- regmap_set_bits(bc->regmap, BLK_CLK_EN, data->clk_mask);
+
+ if (data->is_errata_err050531)
+ regmap_clear_bits(bc->regmap, BLK_CLK_EN, data->clk_mask);
+ else
+ regmap_set_bits(bc->regmap, BLK_CLK_EN, data->clk_mask);
/* power up upstream GPC domain */
ret = pm_runtime_get_sync(domain->power_dev);
@@ -117,6 +130,9 @@ static int imx8m_blk_ctrl_power_on(struct generic_pm_domain *genpd)
goto clk_disable;
}
+ if (data->is_errata_err050531)
+ regmap_set_bits(bc->regmap, BLK_CLK_EN, data->clk_mask);
+
/* wait for reset to propagate */
udelay(5);
@@ -511,12 +527,38 @@ static const struct imx8m_blk_ctrl_domain_data imx8mp_vpu_blk_ctl_domain_data[]
.clk_mask = BIT(2),
.path_names = (const char *[]){"vc8000e"},
.num_paths = 1,
+ .is_errata_err050531 = true,
},
};
+static int imx8mp_vpu_power_notifier(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl,
+ power_nb);
+
+ if (action == GENPD_NOTIFY_ON) {
+ /*
+ * On power up we have no software backchannel to the GPC to
+ * wait for the ADB handshake to happen, so we just delay for a
+ * bit. On power down the GPC driver waits for the handshake.
+ */
+
+ udelay(5);
+
+ /* set "fuse" bits to enable the VPUs */
+ regmap_set_bits(bc->regmap, 0x8, 0xffffffff);
+ regmap_set_bits(bc->regmap, 0xc, 0xffffffff);
+ regmap_set_bits(bc->regmap, 0x10, 0xffffffff);
+ regmap_set_bits(bc->regmap, 0x14, 0xffffffff);
+ }
+
+ return NOTIFY_OK;
+}
+
static const struct imx8m_blk_ctrl_data imx8mp_vpu_blk_ctl_dev_data = {
.max_reg = 0x18,
- .power_notifier_fn = imx8mm_vpu_power_notifier,
+ .power_notifier_fn = imx8mp_vpu_power_notifier,
.domains = imx8mp_vpu_blk_ctl_domain_data,
.num_domains = ARRAY_SIZE(imx8mp_vpu_blk_ctl_domain_data),
};
diff --git a/drivers/power/sequencing/core.c b/drivers/power/sequencing/core.c
index 50aed0a5e8b2..7e94abd913a0 100644
--- a/drivers/power/sequencing/core.c
+++ b/drivers/power/sequencing/core.c
@@ -543,15 +543,18 @@ void pwrseq_device_unregister(struct pwrseq_device *pwrseq)
struct device *dev = &pwrseq->dev;
struct pwrseq_target *target;
- scoped_guard(mutex, &pwrseq->state_lock) {
+ scoped_guard(rwsem_write, &pwrseq_sem) {
guard(rwsem_write)(&pwrseq->rw_lock);
+ /*
+ * Holding rw_lock for write excludes all power on/off callers
+ * (they hold it for read), so it's safe to read enable_count
+ * here without taking the state_lock.
+ */
list_for_each_entry(target, &pwrseq->targets, list)
WARN(target->unit->enable_count,
"REMOVING POWER SEQUENCER WITH ACTIVE USERS\n");
- guard(rwsem_write)(&pwrseq_sem);
-
device_del(dev);
}
diff --git a/drivers/power/supply/charger-manager.c b/drivers/power/supply/charger-manager.c
index 96f0a7fbf105..8074c828080a 100644
--- a/drivers/power/supply/charger-manager.c
+++ b/drivers/power/supply/charger-manager.c
@@ -302,8 +302,10 @@ static bool is_full_charged(struct charger_manager *cm)
if (cm->battery_status == POWER_SUPPLY_STATUS_FULL
&& desc->fullbatt_vchkdrop_uV)
uV += desc->fullbatt_vchkdrop_uV;
- if (uV >= desc->fullbatt_uV)
- return true;
+ if (uV >= desc->fullbatt_uV) {
+ is_full = true;
+ goto out;
+ }
}
}
diff --git a/drivers/power/supply/cpcap-battery.c b/drivers/power/supply/cpcap-battery.c
index eaad9f53bb5c..8e600b0dbfdc 100644
--- a/drivers/power/supply/cpcap-battery.c
+++ b/drivers/power/supply/cpcap-battery.c
@@ -415,10 +415,13 @@ static void cpcap_battery_detect_battery_type(struct cpcap_battery_ddata *ddata)
if (IS_ERR_OR_NULL(nvmem)) {
ddata->check_nvmem = true;
dev_info_once(ddata->dev, "Can not find battery nvmem device. Assuming generic lipo battery\n");
- } else if (nvmem_device_read(nvmem, 2, 1, &battery_id) < 0) {
- battery_id = 0;
- ddata->check_nvmem = true;
- dev_warn(ddata->dev, "Can not read battery nvmem device. Assuming generic lipo battery\n");
+ } else {
+ if (nvmem_device_read(nvmem, 2, 1, &battery_id) < 0) {
+ battery_id = 0;
+ ddata->check_nvmem = true;
+ dev_warn(ddata->dev, "Can not read battery nvmem device. Assuming generic lipo battery\n");
+ }
+ nvmem_device_put(nvmem);
}
switch (battery_id) {
diff --git a/drivers/power/supply/power_supply_core.c b/drivers/power/supply/power_supply_core.c
index 73cc9c236e83..edc7b3bd9b5c 100644
--- a/drivers/power/supply/power_supply_core.c
+++ b/drivers/power/supply/power_supply_core.c
@@ -269,18 +269,13 @@ static int power_supply_check_supplies(struct power_supply *psy)
if (cnt == 1)
return 0;
- /* All supplies found, allocate char ** array for filling */
- psy->supplied_from = devm_kzalloc(&psy->dev, sizeof(*psy->supplied_from),
+ /* All supplies found, allocate char * array for filling */
+ psy->supplied_from = devm_kcalloc(&psy->dev,
+ cnt - 1, sizeof(*psy->supplied_from),
GFP_KERNEL);
if (!psy->supplied_from)
return -ENOMEM;
- *psy->supplied_from = devm_kcalloc(&psy->dev,
- cnt - 1, sizeof(**psy->supplied_from),
- GFP_KERNEL);
- if (!*psy->supplied_from)
- return -ENOMEM;
-
return power_supply_populate_supplied_from(psy);
}
#else
diff --git a/drivers/pwm/pwm-imx27.c b/drivers/pwm/pwm-imx27.c
index 037598719431..32613165f2b3 100644
--- a/drivers/pwm/pwm-imx27.c
+++ b/drivers/pwm/pwm-imx27.c
@@ -220,7 +220,7 @@ static void pwm_imx27_wait_fifo_slot(struct pwm_chip *chip,
static int pwm_imx27_apply(struct pwm_chip *chip, struct pwm_device *pwm,
const struct pwm_state *state)
{
- unsigned long period_cycles, duty_cycles, prescale, period_us, tmp;
+ unsigned long period_cycles, duty_cycles, prescale, period_us;
struct pwm_imx27_chip *imx = to_pwm_imx27_chip(chip);
unsigned long long c;
unsigned long long clkrate;
@@ -228,6 +228,7 @@ static int pwm_imx27_apply(struct pwm_chip *chip, struct pwm_device *pwm,
int val;
int ret;
u32 cr;
+ u64 tmp;
clkrate = clk_get_rate(imx->clk_per);
c = clkrate * state->period;
@@ -269,6 +270,11 @@ static int pwm_imx27_apply(struct pwm_chip *chip, struct pwm_device *pwm,
val = readl(imx->mmio_base + MX3_PWMPR);
val = val >= MX3_PWMPR_MAX ? MX3_PWMPR_MAX : val;
cr = readl(imx->mmio_base + MX3_PWMCR);
+
+ /*
+ * tmp stores period in nanoseconds. Result fits in u64 since
+ * val <= 0xfffe and prescaler in [1, 0x1000].
+ */
tmp = NSEC_PER_SEC * (u64)(val + 2) * MX3_PWMCR_PRESCALER_GET(cr);
tmp = DIV_ROUND_UP_ULL(tmp, clkrate);
period_us = DIV_ROUND_UP_ULL(tmp, 1000);
diff --git a/drivers/rapidio/devices/tsi721.c b/drivers/rapidio/devices/tsi721.c
index 4b84270a8906..ab64c991468d 100644
--- a/drivers/rapidio/devices/tsi721.c
+++ b/drivers/rapidio/devices/tsi721.c
@@ -394,7 +394,6 @@ static void tsi721_db_dpc(struct work_struct *work)
idb_work);
struct rio_mport *mport;
struct rio_dbell *dbell;
- int found = 0;
u32 wr_ptr, rd_ptr;
u64 *idb_entry;
u32 regval;
@@ -412,6 +411,8 @@ static void tsi721_db_dpc(struct work_struct *work)
rd_ptr = ioread32(priv->regs + TSI721_IDQ_RP(IDB_QUEUE)) % IDB_QSIZE;
while (wr_ptr != rd_ptr) {
+ int found = 0;
+
idb_entry = (u64 *)(priv->idb_base +
(TSI721_IDB_ENTRY_SIZE * rd_ptr));
rd_ptr++;
diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index 54fd06891bdc..264c8ab434f0 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -232,7 +232,7 @@ static void regulator_lock_two(struct regulator_dev *rdev1,
ret = regulator_lock_nested(rdev1, ww_ctx);
WARN_ON(ret);
ret = regulator_lock_nested(rdev2, ww_ctx);
- if (ret != -EDEADLOCK) {
+ if (ret != -EDEADLK) {
WARN_ON(ret);
goto exit;
}
@@ -248,7 +248,7 @@ static void regulator_lock_two(struct regulator_dev *rdev1,
swap(held, contended);
ret = regulator_lock_nested(contended, ww_ctx);
- if (ret != -EDEADLOCK) {
+ if (ret != -EDEADLK) {
WARN_ON(ret);
break;
}
diff --git a/drivers/regulator/ltc3676.c b/drivers/regulator/ltc3676.c
index 73d511eb1c1d..768cdce0d4ec 100644
--- a/drivers/regulator/ltc3676.c
+++ b/drivers/regulator/ltc3676.c
@@ -45,11 +45,11 @@
#define LTC3676_DVBxA_REF_SELECT BIT(5)
#define LTC3676_DVBxB_PGOOD_MASK BIT(5)
-#define LTC3676_IRQSTAT_PGOOD_TIMEOUT BIT(3)
-#define LTC3676_IRQSTAT_UNDERVOLT_WARN BIT(4)
-#define LTC3676_IRQSTAT_UNDERVOLT_FAULT BIT(5)
-#define LTC3676_IRQSTAT_THERMAL_WARN BIT(6)
-#define LTC3676_IRQSTAT_THERMAL_FAULT BIT(7)
+#define LTC3676_IRQSTAT_PGOOD_TIMEOUT BIT(2)
+#define LTC3676_IRQSTAT_UNDERVOLT_WARN BIT(3)
+#define LTC3676_IRQSTAT_UNDERVOLT_FAULT BIT(4)
+#define LTC3676_IRQSTAT_THERMAL_WARN BIT(5)
+#define LTC3676_IRQSTAT_THERMAL_FAULT BIT(6)
enum ltc3676_reg {
LTC3676_SW1,
diff --git a/drivers/remoteproc/qcom_common.c b/drivers/remoteproc/qcom_common.c
index 8c8688f99f0a..5bbbe36d0964 100644
--- a/drivers/remoteproc/qcom_common.c
+++ b/drivers/remoteproc/qcom_common.c
@@ -110,6 +110,7 @@ static int qcom_add_minidump_segments(struct rproc *rproc, struct minidump_subsy
struct minidump_region __iomem *ptr;
struct minidump_region region;
int seg_cnt, i;
+ int ret = 0;
dma_addr_t da;
size_t size;
char *name;
@@ -130,17 +131,22 @@ static int qcom_add_minidump_segments(struct rproc *rproc, struct minidump_subsy
if (le32_to_cpu(region.valid) == MINIDUMP_REGION_VALID) {
name = kstrndup(region.name, MAX_REGION_NAME_LENGTH - 1, GFP_KERNEL);
if (!name) {
- iounmap(ptr);
- return -ENOMEM;
+ ret = -ENOMEM;
+ break;
}
da = le64_to_cpu(region.address);
size = le64_to_cpu(region.size);
- rproc_coredump_add_custom_segment(rproc, da, size, rproc_dumpfn_t, name);
+ ret = rproc_coredump_add_custom_segment(rproc, da, size, rproc_dumpfn_t,
+ name);
+ if (ret) {
+ kfree(name);
+ break;
+ }
}
}
iounmap(ptr);
- return 0;
+ return ret;
}
void qcom_minidump(struct rproc *rproc, unsigned int minidump_id,
diff --git a/drivers/reset/reset-sunxi.c b/drivers/reset/reset-sunxi.c
index e752594b6971..8e01589ab7af 100644
--- a/drivers/reset/reset-sunxi.c
+++ b/drivers/reset/reset-sunxi.c
@@ -44,7 +44,7 @@ static int sunxi_reset_init(struct device_node *np)
data->membase = ioremap(res.start, size);
if (!data->membase) {
ret = -ENOMEM;
- goto err_alloc;
+ goto err_mem_region;
}
spin_lock_init(&data->lock);
@@ -57,6 +57,8 @@ static int sunxi_reset_init(struct device_node *np)
return reset_controller_register(&data->rcdev);
+err_mem_region:
+ release_mem_region(res.start, size);
err_alloc:
kfree(data);
return ret;
diff --git a/drivers/rpmsg/qcom_glink_native.c b/drivers/rpmsg/qcom_glink_native.c
index 58c4e2aac951..c2d39a3bcea7 100644
--- a/drivers/rpmsg/qcom_glink_native.c
+++ b/drivers/rpmsg/qcom_glink_native.c
@@ -1618,7 +1618,6 @@ static void qcom_glink_rpdev_release(struct device *dev)
{
struct rpmsg_device *rpdev = to_rpmsg_device(dev);
- kfree(rpdev->driver_override);
kfree(rpdev);
}
@@ -1870,7 +1869,6 @@ static void qcom_glink_device_release(struct device *dev)
/* Release qcom_glink_alloc_channel() reference */
kref_put(&channel->refcount, qcom_glink_channel_release);
- kfree(rpdev->driver_override);
kfree(rpdev);
}
diff --git a/drivers/rpmsg/rpmsg_core.c b/drivers/rpmsg/rpmsg_core.c
index a8ae840d1707..23b33529e402 100644
--- a/drivers/rpmsg/rpmsg_core.c
+++ b/drivers/rpmsg/rpmsg_core.c
@@ -421,33 +421,6 @@ rpmsg_show_attr(src, src, "0x%x\n");
rpmsg_show_attr(dst, dst, "0x%x\n");
rpmsg_show_attr(announce, announce ? "true" : "false", "%s\n");
-static ssize_t driver_override_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct rpmsg_device *rpdev = to_rpmsg_device(dev);
- int ret;
-
- ret = driver_set_override(dev, &rpdev->driver_override, buf, count);
- if (ret)
- return ret;
-
- return count;
-}
-
-static ssize_t driver_override_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct rpmsg_device *rpdev = to_rpmsg_device(dev);
- ssize_t len;
-
- device_lock(dev);
- len = sysfs_emit(buf, "%s\n", rpdev->driver_override);
- device_unlock(dev);
- return len;
-}
-static DEVICE_ATTR_RW(driver_override);
-
static ssize_t modalias_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -468,7 +441,6 @@ static struct attribute *rpmsg_dev_attrs[] = {
&dev_attr_dst.attr,
&dev_attr_src.attr,
&dev_attr_announce.attr,
- &dev_attr_driver_override.attr,
NULL,
};
ATTRIBUTE_GROUPS(rpmsg_dev);
@@ -487,9 +459,11 @@ static int rpmsg_dev_match(struct device *dev, const struct device_driver *drv)
const struct rpmsg_driver *rpdrv = to_rpmsg_driver(drv);
const struct rpmsg_device_id *ids = rpdrv->id_table;
unsigned int i;
+ int ret;
- if (rpdev->driver_override)
- return !strcmp(rpdev->driver_override, drv->name);
+ ret = device_match_driver_override(dev, drv);
+ if (ret >= 0)
+ return ret;
if (ids)
for (i = 0; ids[i].name[0]; i++)
@@ -599,6 +573,7 @@ static const struct bus_type rpmsg_bus = {
.name = "rpmsg",
.match = rpmsg_dev_match,
.dev_groups = rpmsg_dev_groups,
+ .driver_override = true,
.uevent = rpmsg_uevent,
.probe = rpmsg_dev_probe,
.remove = rpmsg_dev_remove,
@@ -624,11 +599,9 @@ int rpmsg_register_device_override(struct rpmsg_device *rpdev,
device_initialize(dev);
if (driver_override) {
- ret = driver_set_override(dev, &rpdev->driver_override,
- driver_override,
- strlen(driver_override));
+ ret = device_set_driver_override(dev, driver_override);
if (ret) {
- dev_err(dev, "device_set_override failed: %d\n", ret);
+ dev_err(dev, "device_set_driver_override() failed: %d\n", ret);
put_device(dev);
return ret;
}
@@ -637,8 +610,6 @@ int rpmsg_register_device_override(struct rpmsg_device *rpdev,
ret = device_add(dev);
if (ret) {
dev_err(dev, "device_add failed: %d\n", ret);
- kfree(rpdev->driver_override);
- rpdev->driver_override = NULL;
put_device(dev);
}
diff --git a/drivers/rpmsg/virtio_rpmsg_bus.c b/drivers/rpmsg/virtio_rpmsg_bus.c
index 89d7a3b8c48b..ee60e4580c81 100644
--- a/drivers/rpmsg/virtio_rpmsg_bus.c
+++ b/drivers/rpmsg/virtio_rpmsg_bus.c
@@ -378,7 +378,6 @@ static void virtio_rpmsg_release_device(struct device *dev)
struct rpmsg_device *rpdev = to_rpmsg_device(dev);
struct virtio_rpmsg_channel *vch = to_virtio_rpmsg_channel(rpdev);
- kfree(rpdev->driver_override);
kfree(vch);
}
diff --git a/drivers/rtc/rtc-abx80x.c b/drivers/rtc/rtc-abx80x.c
index 5f3a3e60a19d..862833519d7f 100644
--- a/drivers/rtc/rtc-abx80x.c
+++ b/drivers/rtc/rtc-abx80x.c
@@ -545,7 +545,8 @@ static int abx80x_ioctl(struct device *dev, unsigned int cmd, unsigned long arg)
status &= ~ABX8XX_STATUS_BLF;
- tmp = i2c_smbus_write_byte_data(client, ABX8XX_REG_STATUS, 0);
+ tmp = i2c_smbus_write_byte_data(client, ABX8XX_REG_STATUS,
+ status);
if (tmp < 0)
return tmp;
diff --git a/drivers/rtc/rtc-cmos.c b/drivers/rtc/rtc-cmos.c
index 095de4e0e4f3..e06f3b9cab2c 100644
--- a/drivers/rtc/rtc-cmos.c
+++ b/drivers/rtc/rtc-cmos.c
@@ -932,6 +932,7 @@ cmos_do_probe(struct device *dev, struct resource *ports, int rtc_irq)
unsigned char rtc_control;
unsigned address_space;
u32 flags = 0;
+ bool hpet_registered = false;
struct nvmem_config nvmem_cfg = {
.name = "cmos_nvram",
.word_size = 1,
@@ -1089,6 +1090,7 @@ cmos_do_probe(struct device *dev, struct resource *ports, int rtc_irq)
" failed in rtc_init().");
goto cleanup1;
}
+ hpet_registered = true;
} else
rtc_cmos_int_handler = cmos_interrupt;
@@ -1138,6 +1140,10 @@ cmos_do_probe(struct device *dev, struct resource *ports, int rtc_irq)
if (is_valid_irq(rtc_irq))
free_irq(rtc_irq, cmos_rtc.rtc);
cleanup1:
+ if (hpet_registered) {
+ hpet_mask_rtc_irq_bit(RTC_IRQMASK);
+ hpet_unregister_irq_handler(cmos_interrupt);
+ }
cmos_rtc.dev = NULL;
cleanup0:
if (RTC_IOMAPPED)
diff --git a/drivers/rtc/rtc-ds1307.c b/drivers/rtc/rtc-ds1307.c
index 1960d1bd851c..858c29494f5f 100644
--- a/drivers/rtc/rtc-ds1307.c
+++ b/drivers/rtc/rtc-ds1307.c
@@ -269,6 +269,16 @@ static int ds1307_get_time(struct device *dev, struct rtc_time *t)
if (tmp & DS1338_BIT_OSF)
return -EINVAL;
break;
+ case ds_1337:
+ case ds_1339:
+ case ds_1341:
+ case ds_3231:
+ ret = regmap_read(ds1307->regmap, DS1337_REG_STATUS, &tmp);
+ if (ret)
+ return ret;
+ if (tmp & DS1337_BIT_OSF)
+ return -EINVAL;
+ break;
case ds_1340:
if (tmp & DS1340_BIT_nEOSC)
return -EINVAL;
@@ -279,13 +289,6 @@ static int ds1307_get_time(struct device *dev, struct rtc_time *t)
if (tmp & DS1340_BIT_OSF)
return -EINVAL;
break;
- case ds_1341:
- ret = regmap_read(ds1307->regmap, DS1337_REG_STATUS, &tmp);
- if (ret)
- return ret;
- if (tmp & DS1337_BIT_OSF)
- return -EINVAL;
- break;
case ds_1388:
ret = regmap_read(ds1307->regmap, DS1388_REG_FLAG, &tmp);
if (ret)
@@ -308,7 +311,7 @@ static int ds1307_get_time(struct device *dev, struct rtc_time *t)
t->tm_hour = bcd2bin(tmp);
/* rx8130 is bit position, not BCD */
if (ds1307->type == rx_8130)
- t->tm_wday = fls(regs[DS1307_REG_WDAY] & 0x7f);
+ t->tm_wday = fls(regs[DS1307_REG_WDAY] & 0x7f) - 1;
else
t->tm_wday = bcd2bin(regs[DS1307_REG_WDAY] & 0x07) - 1;
t->tm_mday = bcd2bin(regs[DS1307_REG_MDAY] & 0x3f);
@@ -380,14 +383,17 @@ static int ds1307_set_time(struct device *dev, struct rtc_time *t)
regmap_update_bits(ds1307->regmap, DS1307_REG_CONTROL,
DS1338_BIT_OSF, 0);
break;
- case ds_1340:
- regmap_update_bits(ds1307->regmap, DS1340_REG_FLAG,
- DS1340_BIT_OSF, 0);
- break;
+ case ds_1337:
+ case ds_1339:
case ds_1341:
+ case ds_3231:
regmap_update_bits(ds1307->regmap, DS1337_REG_STATUS,
DS1337_BIT_OSF, 0);
break;
+ case ds_1340:
+ regmap_update_bits(ds1307->regmap, DS1340_REG_FLAG,
+ DS1340_BIT_OSF, 0);
+ break;
case ds_1388:
regmap_update_bits(ds1307->regmap, DS1388_REG_FLAG,
DS1388_BIT_OSF, 0);
diff --git a/drivers/rtc/rtc-mpfs.c b/drivers/rtc/rtc-mpfs.c
index 5b96a6d39210..aa724a945c08 100644
--- a/drivers/rtc/rtc-mpfs.c
+++ b/drivers/rtc/rtc-mpfs.c
@@ -112,7 +112,7 @@ static int mpfs_rtc_settime(struct device *dev, struct rtc_time *tm)
ctrl |= CONTROL_UPLOAD_BIT;
writel(ctrl, rtcdev->base + CONTROL_REG);
- ret = read_poll_timeout(readl, prog, prog & CONTROL_UPLOAD_BIT, 0, UPLOAD_TIMEOUT_US,
+ ret = read_poll_timeout(readl, prog, !(prog & CONTROL_UPLOAD_BIT), 0, UPLOAD_TIMEOUT_US,
false, rtcdev->base + CONTROL_REG);
if (ret) {
dev_err(dev, "timed out uploading time to rtc");
diff --git a/drivers/rtc/rtc-msc313.c b/drivers/rtc/rtc-msc313.c
index 8d7737e0e2e0..6ef9c4efd7c9 100644
--- a/drivers/rtc/rtc-msc313.c
+++ b/drivers/rtc/rtc-msc313.c
@@ -160,7 +160,7 @@ static const struct rtc_class_ops msc313_rtc_ops = {
static irqreturn_t msc313_rtc_interrupt(s32 irq, void *dev_id)
{
- struct msc313_rtc *priv = dev_get_drvdata(dev_id);
+ struct msc313_rtc *priv = dev_id;
u16 reg;
reg = readw(priv->rtc_base + REG_RTC_STATUS_INT);
@@ -206,7 +206,7 @@ static int msc313_rtc_probe(struct platform_device *pdev)
priv->rtc_dev->range_max = U32_MAX;
ret = devm_request_irq(dev, irq, msc313_rtc_interrupt, IRQF_SHARED,
- dev_name(&pdev->dev), &pdev->dev);
+ dev_name(&pdev->dev), priv);
if (ret) {
dev_err(dev, "Could not request IRQ\n");
return ret;
diff --git a/drivers/s390/char/monwriter.c b/drivers/s390/char/monwriter.c
index bc5193d81f9c..54debecd3cb7 100644
--- a/drivers/s390/char/monwriter.c
+++ b/drivers/s390/char/monwriter.c
@@ -122,6 +122,9 @@ static int monwrite_new_hdr(struct mon_private *monpriv)
kfree(monbuf->data);
kfree(monbuf);
monbuf = NULL;
+ } else if (monbuf->hdr.datalen != monhdr->datalen) {
+ /* Data with buffer reuse must not change its length */
+ return -EINVAL;
}
} else if (monhdr->mon_function != MONWRITE_STOP_INTERVAL) {
if (mon_buf_count >= mon_max_bufs)
diff --git a/drivers/s390/crypto/pkey_api.c b/drivers/s390/crypto/pkey_api.c
index d62fea0fbdfc..08989af9ed8f 100644
--- a/drivers/s390/crypto/pkey_api.c
+++ b/drivers/s390/crypto/pkey_api.c
@@ -325,16 +325,14 @@ static int pkey_ioctl_verifyprotk(struct pkey_verifyprotk __user *uvp)
{
struct pkey_verifyprotk kvp;
struct protaeskeytoken *t;
- u32 keytype;
u8 *tmpbuf;
int rc;
if (copy_from_user(&kvp, uvp, sizeof(kvp)))
return -EFAULT;
- keytype = pkey_aes_bitsize_to_keytype(8 * kvp.protkey.len);
- if (!keytype) {
- PKEY_DBF_ERR("%s unknown/unsupported protkey length %u\n",
+ if (kvp.protkey.len > sizeof(kvp.protkey.protkey)) {
+ PKEY_DBF_ERR("%s protkey length %u exceeds protkey buffer size\n",
__func__, kvp.protkey.len);
memzero_explicit(&kvp, sizeof(kvp));
return -EINVAL;
@@ -349,7 +347,7 @@ static int pkey_ioctl_verifyprotk(struct pkey_verifyprotk __user *uvp)
t = (struct protaeskeytoken *)tmpbuf;
t->type = TOKTYPE_NON_CCA;
t->version = TOKVER_PROTECTED_KEY;
- t->keytype = keytype;
+ t->keytype = kvp.protkey.type;
t->len = kvp.protkey.len;
memcpy(t->protkey, kvp.protkey.protkey, kvp.protkey.len);
diff --git a/drivers/s390/crypto/pkey_pckmo.c b/drivers/s390/crypto/pkey_pckmo.c
index beeca8827c46..9aa04e6fbf94 100644
--- a/drivers/s390/crypto/pkey_pckmo.c
+++ b/drivers/s390/crypto/pkey_pckmo.c
@@ -321,6 +321,10 @@ static int pckmo_key2protkey(const u8 *key, u32 keylen,
__func__, t->keytype);
goto out;
}
+ if (t->len > *protkeylen) {
+ rc = -EINVAL;
+ goto out;
+ }
memcpy(protkey, t->protkey, t->len);
*protkeylen = t->len;
*protkeytype = t->keytype;
diff --git a/drivers/s390/crypto/zcrypt_cex2a.c b/drivers/s390/crypto/zcrypt_cex2a.c
deleted file mode 100644
index e69de29bb2d1..000000000000
diff --git a/drivers/s390/crypto/zcrypt_cex2a.h b/drivers/s390/crypto/zcrypt_cex2a.h
deleted file mode 100644
index e69de29bb2d1..000000000000
diff --git a/drivers/s390/crypto/zcrypt_cex2c.c b/drivers/s390/crypto/zcrypt_cex2c.c
deleted file mode 100644
index e69de29bb2d1..000000000000
diff --git a/drivers/s390/crypto/zcrypt_cex2c.h b/drivers/s390/crypto/zcrypt_cex2c.h
deleted file mode 100644
index e69de29bb2d1..000000000000
diff --git a/drivers/scsi/elx/efct/efct_hw.c b/drivers/scsi/elx/efct/efct_hw.c
index 5b079b8b7a08..f41b7f1e4416 100644
--- a/drivers/scsi/elx/efct/efct_hw.c
+++ b/drivers/scsi/elx/efct/efct_hw.c
@@ -1998,6 +1998,8 @@ efct_hw_io_abort(struct efct_hw *hw, struct efct_hw_io *io_to_abort,
wqcb = efct_hw_reqtag_alloc(hw, efct_hw_wq_process_abort, io_to_abort);
if (!wqcb) {
efc_log_err(hw->os, "can't allocate request tag\n");
+ io_to_abort->abort_in_progress = false;
+ kref_put(&io_to_abort->ref, io_to_abort->release);
return -ENOSPC;
}
diff --git a/drivers/scsi/elx/efct/efct_unsol.c b/drivers/scsi/elx/efct/efct_unsol.c
index e6addab66a60..6a871a59c909 100644
--- a/drivers/scsi/elx/efct/efct_unsol.c
+++ b/drivers/scsi/elx/efct/efct_unsol.c
@@ -385,6 +385,7 @@ efct_dispatch_fcp_cmd(struct efct_node *node, struct efc_hw_sequence *seq)
if (cmnd->fc_flags & FCP_CFL_LEN_MASK) {
efc_log_err(efct, "Additional CDB not supported\n");
+ efct_scsi_io_free(io);
return -EIO;
}
/*
diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 6cb6586f4790..c196eede458a 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -5031,6 +5031,10 @@ static int hpsa_scsi_ioaccel2_queue_command(struct ctlr_info *h,
if (phys_disk->in_reset) {
cmd->result = DID_RESET << 16;
+ atomic_dec(&phys_disk->ioaccel_cmds_out);
+ scsi_dma_unmap(cmd);
+ if (use_sg > h->ioaccel_maxsg)
+ hpsa_unmap_ioaccel2_sg_chain_block(h, cp);
return -1;
}
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 5b9830a28c8d..d1c95076a25a 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -8192,6 +8192,7 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
goto out_free_bsmbx;
}
}
+ mempool_free(mboxq, phba->mbox_mem_pool);
/*
* 1 for cmd, 1 for rsp, NVME adds an extra one
@@ -8317,8 +8318,6 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
goto out_free_sg_dma_buf;
}
- mempool_free(mboxq, phba->mbox_mem_pool);
-
/* Verify OAS is supported */
lpfc_sli4_oas_verify(phba);
diff --git a/drivers/scsi/pm8001/pm8001_ctl.c b/drivers/scsi/pm8001/pm8001_ctl.c
index 34cddfdbc22f..4362bd6cc29e 100644
--- a/drivers/scsi/pm8001/pm8001_ctl.c
+++ b/drivers/scsi/pm8001/pm8001_ctl.c
@@ -588,10 +588,7 @@ static DEVICE_ATTR(fatal_log, S_IRUGO, pm8001_ctl_fatal_log_show, NULL);
static ssize_t non_fatal_log_show(struct device *cdev,
struct device_attribute *attr, char *buf)
{
- u32 count;
-
- count = pm80xx_get_non_fatal_dump(cdev, attr, buf);
- return count;
+ return pm80xx_get_non_fatal_dump(cdev, attr, buf);
}
static DEVICE_ATTR_RO(non_fatal_log);
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index f57260e28c7a..e9599c26250f 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -1878,7 +1878,7 @@ int scsi_scan_host_selected(struct Scsi_Host *shost, unsigned int channel,
return 0;
}
-EXPORT_SYMBOL(scsi_scan_host_selected);
+
static void scsi_sysfs_add_devices(struct Scsi_Host *shost)
{
struct scsi_device *sdev;
diff --git a/drivers/scsi/scsi_transport_sas.c b/drivers/scsi/scsi_transport_sas.c
index f21d2be7ef1b..01358879c0ad 100644
--- a/drivers/scsi/scsi_transport_sas.c
+++ b/drivers/scsi/scsi_transport_sas.c
@@ -27,6 +27,7 @@
#include <linux/module.h>
#include <linux/jiffies.h>
#include <linux/err.h>
+#include <linux/log2.h>
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/blkdev.h>
@@ -40,8 +41,6 @@
#include <scsi/scsi_transport_sas.h>
#include "scsi_sas_internal.h"
-#include "scsi_priv.h"
-
struct sas_host_attrs {
struct list_head rphy_list;
struct mutex lock;
@@ -222,12 +221,45 @@ static int sas_bsg_initialize(struct Scsi_Host *shost, struct sas_rphy *rphy)
* SAS host attributes
*/
+/*
+ * Set shost->opt_sectors from the DMA optimal mapping size, but only
+ * when dma_opt_mapping_size() is strictly less than dma_max_mapping_size(),
+ * indicating a genuine optimization hint from an IOMMU or DMA backend.
+ * When the two are equal (e.g. IOMMU disabled / passthrough), no real
+ * hint exists, so leave opt_sectors at 0 to avoid bogus optimal_io_size
+ * values that break filesystem geometry (e.g. mkfs.xfs stripe alignment).
+ */
+static void sas_dma_setup_opt_sectors(struct Scsi_Host *shost)
+{
+ struct device *dma_dev = shost->dma_dev;
+ size_t opt = dma_opt_mapping_size(dma_dev);
+ size_t max = dma_max_mapping_size(dma_dev);
+ unsigned int opt_sectors;
+
+ /* opt >= max means no real hint was provided by the DMA layer */
+ if (opt >= max)
+ return;
+
+ /* Clamp to max_sectors to avoid overflow in sector arithmetic */
+ opt_sectors = min_t(unsigned int, opt >> SECTOR_SHIFT,
+ shost->max_sectors);
+
+ /* Guard against zero before rounddown_pow_of_two() */
+ if (!opt_sectors)
+ return;
+
+ /*
+ * Round down to power-of-two so filesystem geometry calculations
+ * (e.g. XFS stripe width/unit) always produce clean divisors.
+ */
+ shost->opt_sectors = rounddown_pow_of_two(opt_sectors);
+}
+
static int sas_host_setup(struct transport_container *tc, struct device *dev,
struct device *cdev)
{
struct Scsi_Host *shost = dev_to_shost(dev);
struct sas_host_attrs *sas_host = to_sas_host_attrs(shost);
- struct device *dma_dev = shost->dma_dev;
INIT_LIST_HEAD(&sas_host->rphy_list);
mutex_init(&sas_host->lock);
@@ -239,10 +271,7 @@ static int sas_host_setup(struct transport_container *tc, struct device *dev,
dev_printk(KERN_ERR, dev, "fail to a bsg device %d\n",
shost->host_no);
- if (dma_dev->dma_mask) {
- shost->opt_sectors = min_t(unsigned int, shost->max_sectors,
- dma_opt_mapping_size(dma_dev) >> SECTOR_SHIFT);
- }
+ sas_dma_setup_opt_sectors(shost);
return 0;
}
@@ -1683,22 +1712,6 @@ int scsi_is_sas_rphy(const struct device *dev)
}
EXPORT_SYMBOL(scsi_is_sas_rphy);
-static void scan_channel_zero(struct Scsi_Host *shost, uint id, u64 lun)
-{
- struct sas_host_attrs *sas_host = to_sas_host_attrs(shost);
- struct sas_rphy *rphy;
-
- list_for_each_entry(rphy, &sas_host->rphy_list, list) {
- if (rphy->identify.device_type != SAS_END_DEVICE ||
- rphy->scsi_target_id == -1)
- continue;
-
- if (id == SCAN_WILD_CARD || id == rphy->scsi_target_id) {
- scsi_scan_target(&rphy->dev, 0, rphy->scsi_target_id,
- lun, SCSI_SCAN_MANUAL);
- }
- }
-}
/*
* SCSI scan helper
@@ -1708,41 +1721,23 @@ static int sas_user_scan(struct Scsi_Host *shost, uint channel,
uint id, u64 lun)
{
struct sas_host_attrs *sas_host = to_sas_host_attrs(shost);
- int res = 0;
- int i;
-
- switch (channel) {
- case 0:
- mutex_lock(&sas_host->lock);
- scan_channel_zero(shost, id, lun);
- mutex_unlock(&sas_host->lock);
- break;
-
- case SCAN_WILD_CARD:
- mutex_lock(&sas_host->lock);
- scan_channel_zero(shost, id, lun);
- mutex_unlock(&sas_host->lock);
+ struct sas_rphy *rphy;
- for (i = 1; i <= shost->max_channel; i++) {
- res = scsi_scan_host_selected(shost, i, id, lun,
- SCSI_SCAN_MANUAL);
- if (res)
- goto exit_scan;
- }
- break;
+ mutex_lock(&sas_host->lock);
+ list_for_each_entry(rphy, &sas_host->rphy_list, list) {
+ if (rphy->identify.device_type != SAS_END_DEVICE ||
+ rphy->scsi_target_id == -1)
+ continue;
- default:
- if (channel <= shost->max_channel) {
- res = scsi_scan_host_selected(shost, channel, id, lun,
- SCSI_SCAN_MANUAL);
- } else {
- res = -EINVAL;
+ if ((channel == SCAN_WILD_CARD || channel == 0) &&
+ (id == SCAN_WILD_CARD || id == rphy->scsi_target_id)) {
+ scsi_scan_target(&rphy->dev, 0, rphy->scsi_target_id,
+ lun, SCSI_SCAN_MANUAL);
}
- break;
}
+ mutex_unlock(&sas_host->lock);
-exit_scan:
- return res;
+ return 0;
}
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index f85a52e9a7a7..da1060e81cc1 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -862,10 +862,9 @@ sg_fill_request_table(Sg_fd *sfp, sg_req_info_t *rinfo)
if (val >= SG_MAX_QUEUE)
break;
rinfo[val].req_state = srp->done + 1;
- rinfo[val].problem =
- srp->header.masked_status &
- srp->header.host_status &
- srp->header.driver_status;
+ rinfo[val].problem = srp->header.masked_status ||
+ srp->header.host_status ||
+ srp->header.driver_status;
if (srp->done)
rinfo[val].duration =
srp->header.duration;
diff --git a/drivers/scsi/smartpqi/smartpqi_init.c b/drivers/scsi/smartpqi/smartpqi_init.c
index e7836f66c89a..4d751a237672 100644
--- a/drivers/scsi/smartpqi/smartpqi_init.c
+++ b/drivers/scsi/smartpqi/smartpqi_init.c
@@ -2629,7 +2629,7 @@ static int pqi_scan_finished(struct Scsi_Host *shost,
{
struct pqi_ctrl_info *ctrl_info;
- ctrl_info = shost_priv(shost);
+ ctrl_info = shost_to_hba(shost);
return !mutex_is_locked(&ctrl_info->scan_mutex);
}
diff --git a/drivers/soc/fsl/qe/qe.c b/drivers/soc/fsl/qe/qe.c
index 70b6eddb867b..3ecc4ce9cfa2 100644
--- a/drivers/soc/fsl/qe/qe.c
+++ b/drivers/soc/fsl/qe/qe.c
@@ -89,6 +89,9 @@ void qe_reset(void)
if (qe_immr == NULL)
qe_immr = ioremap(get_qe_base(), QE_IMMAP_SIZE);
+ if (!qe_immr)
+ panic("QE:ioremap failed!");
+
qe_snums_init();
qe_issue_cmd(QE_RESET, QE_CR_SUBBLOCK_INVALID,
diff --git a/drivers/soc/ti/k3-ringacc.c b/drivers/soc/ti/k3-ringacc.c
index 82a15cad1c6c..edc2ea8a1f42 100644
--- a/drivers/soc/ti/k3-ringacc.c
+++ b/drivers/soc/ti/k3-ringacc.c
@@ -1012,7 +1012,7 @@ static int k3_ringacc_ring_pop_head_proxy(struct k3_ring *ring, void *elem)
static int k3_ringacc_ring_pop_tail_proxy(struct k3_ring *ring, void *elem)
{
return k3_ringacc_ring_access_proxy(ring, elem,
- K3_RINGACC_ACCESS_MODE_POP_HEAD);
+ K3_RINGACC_ACCESS_MODE_POP_TAIL);
}
static int k3_ringacc_ring_access_io(struct k3_ring *ring, void *elem,
@@ -1083,7 +1083,7 @@ static int k3_ringacc_ring_pop_io(struct k3_ring *ring, void *elem)
static int k3_ringacc_ring_pop_tail_io(struct k3_ring *ring, void *elem)
{
return k3_ringacc_ring_access_io(ring, elem,
- K3_RINGACC_ACCESS_MODE_POP_HEAD);
+ K3_RINGACC_ACCESS_MODE_POP_TAIL);
}
/*
diff --git a/drivers/soc/xilinx/zynqmp_power.c b/drivers/soc/xilinx/zynqmp_power.c
index ae59bf16659a..84ef7c87dfaa 100644
--- a/drivers/soc/xilinx/zynqmp_power.c
+++ b/drivers/soc/xilinx/zynqmp_power.c
@@ -396,8 +396,10 @@ static void zynqmp_pm_remove(struct platform_device *pdev)
{
sysfs_remove_file(&pdev->dev.kobj, &dev_attr_suspend_mode.attr);
- if (!rx_chan)
+ if (rx_chan) {
mbox_free_channel(rx_chan);
+ rx_chan = NULL;
+ }
}
static const struct of_device_id pm_of_match[] = {
diff --git a/drivers/spi/spi-dw.h b/drivers/spi/spi-dw.h
index fc267c6437ae..8a3c085c85e0 100644
--- a/drivers/spi/spi-dw.h
+++ b/drivers/spi/spi-dw.h
@@ -282,6 +282,7 @@ static inline void dw_spi_shutdown_chip(struct dw_spi *dws)
{
dw_spi_enable_chip(dws, 0);
dw_spi_set_clk(dws, 0);
+ dws->current_freq = 0;
}
extern void dw_spi_set_cs(struct spi_device *spi, bool enable);
diff --git a/drivers/spi/spi-ep93xx.c b/drivers/spi/spi-ep93xx.c
index 0b42eeccd42e..fdf72636b66a 100644
--- a/drivers/spi/spi-ep93xx.c
+++ b/drivers/spi/spi-ep93xx.c
@@ -600,6 +600,7 @@ static int ep93xx_spi_setup_dma(struct device *dev, struct ep93xx_spi *espi)
espi->dma_rx = NULL;
fail_free_page:
free_page((unsigned long)espi->zeropage);
+ espi->zeropage = NULL;
return ret;
}
diff --git a/drivers/spi/spi-hisi-kunpeng.c b/drivers/spi/spi-hisi-kunpeng.c
index 77526f794068..6afa37d10c7c 100644
--- a/drivers/spi/spi-hisi-kunpeng.c
+++ b/drivers/spi/spi-hisi-kunpeng.c
@@ -518,10 +518,8 @@ static int hisi_spi_probe(struct platform_device *pdev)
}
ret = spi_register_controller(host);
- if (ret) {
- dev_err(dev, "failed to register spi host, ret=%d\n", ret);
- return ret;
- }
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to register spi host\n");
if (hisi_spi_debugfs_init(hs))
dev_info(dev, "failed to create debugfs dir\n");
diff --git a/drivers/spi/spi-meson-spifc.c b/drivers/spi/spi-meson-spifc.c
index fd8b26dd4a79..8d155f753fd7 100644
--- a/drivers/spi/spi-meson-spifc.c
+++ b/drivers/spi/spi-meson-spifc.c
@@ -352,6 +352,7 @@ static void meson_spifc_remove(struct platform_device *pdev)
{
pm_runtime_get_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ pm_runtime_put_noidle(&pdev->dev);
}
#ifdef CONFIG_PM_SLEEP
diff --git a/drivers/spi/spi-rpc-if.c b/drivers/spi/spi-rpc-if.c
index 43e829251af5..e49a04ae32cf 100644
--- a/drivers/spi/spi-rpc-if.c
+++ b/drivers/spi/spi-rpc-if.c
@@ -192,8 +192,12 @@ static int __maybe_unused rpcif_spi_suspend(struct device *dev)
static int __maybe_unused rpcif_spi_resume(struct device *dev)
{
struct spi_controller *ctlr = dev_get_drvdata(dev);
+ struct rpcif *rpc = spi_controller_get_devdata(ctlr);
+ int ret;
- rpcif_hw_init(dev, false);
+ ret = rpcif_hw_init(rpc->dev, false);
+ if (ret)
+ return ret;
return spi_controller_resume(ctlr);
}
diff --git a/drivers/spi/spi-sh-msiof.c b/drivers/spi/spi-sh-msiof.c
index 1870f8c85213..3367c3e18de8 100644
--- a/drivers/spi/spi-sh-msiof.c
+++ b/drivers/spi/spi-sh-msiof.c
@@ -242,7 +242,7 @@ static irqreturn_t sh_msiof_spi_irq(int irq, void *data)
return IRQ_HANDLED;
}
-static void sh_msiof_spi_reset_regs(struct sh_msiof_spi_priv *p)
+static int sh_msiof_spi_reset_regs(struct sh_msiof_spi_priv *p)
{
u32 mask = SICTR_TXRST | SICTR_RXRST;
u32 data;
@@ -251,8 +251,8 @@ static void sh_msiof_spi_reset_regs(struct sh_msiof_spi_priv *p)
data |= mask;
sh_msiof_write(p, SICTR, data);
- readl_poll_timeout_atomic(p->mapbase + SICTR, data, !(data & mask), 1,
- 100);
+ return readl_poll_timeout_atomic(p->mapbase + SICTR, data,
+ !(data & mask), 1, 100);
}
static const u32 sh_msiof_spi_div_array[] = {
@@ -926,7 +926,9 @@ static int sh_msiof_transfer_one(struct spi_controller *ctlr,
int ret;
/* reset registers */
- sh_msiof_spi_reset_regs(p);
+ ret = sh_msiof_spi_reset_regs(p);
+ if (ret)
+ return ret;
/* setup clocks (clock already enabled in chipselect()) */
if (!spi_controller_is_target(p->ctlr))
diff --git a/drivers/spi/spi-uniphier.c b/drivers/spi/spi-uniphier.c
index 8eb1821689a0..5f50a8ed03d7 100644
--- a/drivers/spi/spi-uniphier.c
+++ b/drivers/spi/spi-uniphier.c
@@ -659,6 +659,8 @@ static int uniphier_spi_probe(struct platform_device *pdev)
priv->host = host;
priv->is_save_param = false;
+ init_completion(&priv->xfer_done);
+
priv->base = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
if (IS_ERR(priv->base)) {
ret = PTR_ERR(priv->base);
@@ -686,8 +688,6 @@ static int uniphier_spi_probe(struct platform_device *pdev)
goto out_host_put;
}
- init_completion(&priv->xfer_done);
-
clk_rate = clk_get_rate(priv->clk);
host->max_speed_hz = DIV_ROUND_UP(clk_rate, SSI_MIN_CLK_DIVIDER);
diff --git a/drivers/spi/spi-xilinx.c b/drivers/spi/spi-xilinx.c
index f5e41813b958..9eef5688dff2 100644
--- a/drivers/spi/spi-xilinx.c
+++ b/drivers/spi/spi-xilinx.c
@@ -370,11 +370,18 @@ static int xilinx_spi_find_buffer_size(struct xilinx_spi *xspi)
xspi->regs + XIPIF_V123B_RESETR_OFFSET);
/* Fill the Tx FIFO with as many words as possible */
- do {
+ while (1) {
xspi->write_fn(0, xspi->regs + XSPI_TXD_OFFSET);
sr = xspi->read_fn(xspi->regs + XSPI_SR_OFFSET);
+ if (sr & XSPI_SR_TX_FULL_MASK)
+ break;
+
n_words++;
- } while (!(sr & XSPI_SR_TX_FULL_MASK));
+ }
+
+ /* Handle the NO FIFO case separately */
+ if (!n_words)
+ return 1;
return n_words;
}
diff --git a/drivers/spi/spi.c b/drivers/spi/spi.c
index 5a6c2f5672ae..319d3834476e 100644
--- a/drivers/spi/spi.c
+++ b/drivers/spi/spi.c
@@ -4820,11 +4820,6 @@ static int of_spi_notify(struct notifier_block *nb, unsigned long action,
return NOTIFY_OK;
}
- /*
- * Clear the flag before adding the device so that fw_devlink
- * doesn't skip adding consumers to this device.
- */
- fwnode_clear_flag(&rd->dn->fwnode, FWNODE_FLAG_NOT_DEVICE);
spi = of_register_spi_device(ctlr, rd->dn);
put_device(&ctlr->dev);
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c b/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
index 994b8bceb4f5..9abe12e70ed4 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
@@ -821,7 +821,7 @@ static int gc2235_probe(struct i2c_client *client)
ret = gc2235_s_config(&dev->sd, client->irq, gcpdev);
if (ret)
- goto out_free;
+ goto err_unregister_subdev;
dev->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
dev->pad.flags = MEDIA_PAD_FL_SOURCE;
@@ -830,18 +830,16 @@ static int gc2235_probe(struct i2c_client *client)
ret =
v4l2_ctrl_handler_init(&dev->ctrl_handler,
ARRAY_SIZE(gc2235_controls));
- if (ret) {
- gc2235_remove(client);
- return ret;
- }
+ if (ret)
+ goto err_csi_cfg;
for (i = 0; i < ARRAY_SIZE(gc2235_controls); i++)
v4l2_ctrl_new_custom(&dev->ctrl_handler, &gc2235_controls[i],
NULL);
if (dev->ctrl_handler.error) {
- gc2235_remove(client);
- return dev->ctrl_handler.error;
+ ret = dev->ctrl_handler.error;
+ goto err_ctrl_handler;
}
/* Use same lock for controls as for everything else. */
@@ -850,14 +848,23 @@ static int gc2235_probe(struct i2c_client *client)
ret = media_entity_pads_init(&dev->sd.entity, 1, &dev->pad);
if (ret)
- gc2235_remove(client);
+ goto err_ctrl_handler;
+
+ ret = atomisp_register_i2c_module(&dev->sd, gcpdev);
+ if (ret)
+ goto err_media_cleanup;
- return atomisp_register_i2c_module(&dev->sd, gcpdev);
+ return 0;
-out_free:
+err_media_cleanup:
+ media_entity_cleanup(&dev->sd.entity);
+err_ctrl_handler:
+ v4l2_ctrl_handler_free(&dev->ctrl_handler);
+err_csi_cfg:
+ dev->platform_data->csi_cfg(&dev->sd, 0);
+err_unregister_subdev:
v4l2_device_unregister_subdev(&dev->sd);
kfree(dev);
-
return ret;
}
diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.c b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
index 102d39a45c9c..fcfccd3b3eb7 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_cmd.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
@@ -3376,10 +3376,8 @@ int atomisp_fixed_pattern_table(struct atomisp_sub_device *asd,
if (ret)
return ret;
- if (sh_css_set_black_frame(asd->stream_env[ATOMISP_INPUT_STREAM_GENERAL].stream,
- raw_black_frame) != 0)
- return -ENOMEM;
-
+ ret = sh_css_set_black_frame(asd->stream_env[ATOMISP_INPUT_STREAM_GENERAL].stream,
+ raw_black_frame);
ia_css_frame_free(raw_black_frame);
return ret;
}
diff --git a/drivers/staging/media/atomisp/pci/sh_css.c b/drivers/staging/media/atomisp/pci/sh_css.c
index 499afd739392..e6ffef36a7ae 100644
--- a/drivers/staging/media/atomisp/pci/sh_css.c
+++ b/drivers/staging/media/atomisp/pci/sh_css.c
@@ -8237,7 +8237,7 @@ ia_css_stream_create(const struct ia_css_stream_config *stream_config,
int
ia_css_stream_destroy(struct ia_css_stream *stream)
{
- int i;
+ int i, j;
int err = 0;
IA_CSS_ENTER_PRIVATE("stream = %p", stream);
@@ -8268,10 +8268,10 @@ ia_css_stream_destroy(struct ia_css_stream *stream)
sp_pipeline_input_terminal =
&sh_css_sp_group.pipe_io[sp_thread_id].input;
- for (i = 0; i < IA_CSS_STREAM_MAX_ISYS_STREAM_PER_CH; i++) {
+ for (j = 0; j < IA_CSS_STREAM_MAX_ISYS_STREAM_PER_CH; j++) {
ia_css_isys_stream_h isys_stream =
- &sp_pipeline_input_terminal->context.virtual_input_system_stream[i];
- if (stream->config.isys_config[i].valid && isys_stream->valid)
+ &sp_pipeline_input_terminal->context.virtual_input_system_stream[j];
+ if (stream->config.isys_config[j].valid && isys_stream->valid)
ia_css_isys_stream_destroy(isys_stream);
}
}
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index f52df6836045..992b30e5ccac 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -478,7 +478,7 @@ static int cedrus_probe(struct platform_device *pdev)
ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
if (ret) {
dev_err(&pdev->dev, "Failed to register V4L2 device\n");
- return ret;
+ goto err_hw;
}
vfd = &dev->vfd;
@@ -539,6 +539,8 @@ static int cedrus_probe(struct platform_device *pdev)
v4l2_m2m_release(dev->m2m_dev);
err_v4l2:
v4l2_device_unregister(&dev->v4l2_dev);
+err_hw:
+ cedrus_hw_remove(dev);
return ret;
}
diff --git a/drivers/staging/most/video/video.c b/drivers/staging/most/video/video.c
index 2b3cdb1ce140..60d554e74b26 100644
--- a/drivers/staging/most/video/video.c
+++ b/drivers/staging/most/video/video.c
@@ -415,6 +415,7 @@ static int comp_register_videodev(struct most_video_dev *mdev)
/* Fill the video capture device struct */
*mdev->vdev = comp_videodev_template;
+ mdev->vdev->release = video_device_release_empty;
mdev->vdev->v4l2_dev = &mdev->v4l2_dev;
mdev->vdev->lock = &mdev->lock;
snprintf(mdev->vdev->name, sizeof(mdev->vdev->name), "MOST: %s",
@@ -427,9 +428,13 @@ static int comp_register_videodev(struct most_video_dev *mdev)
v4l2_err(&mdev->v4l2_dev, "video_register_device failed (%d)\n",
ret);
video_device_release(mdev->vdev);
+ return ret;
}
- return ret;
+ mdev->vdev->release = video_device_release;
+
+ return 0;
+
}
static void comp_unregister_videodev(struct most_video_dev *mdev)
diff --git a/drivers/staging/nvec/nvec.c b/drivers/staging/nvec/nvec.c
index 977f8fc29e63..9d15dbdd31aa 100644
--- a/drivers/staging/nvec/nvec.c
+++ b/drivers/staging/nvec/nvec.c
@@ -494,6 +494,8 @@ static void nvec_tx_completed(struct nvec_chip *nvec)
static void nvec_rx_completed(struct nvec_chip *nvec)
{
if (nvec->rx->pos != nvec_msg_size(nvec->rx)) {
+ unsigned char msg_type = nvec->rx->data[0];
+
dev_err(nvec->dev, "RX incomplete: Expected %u bytes, got %u\n",
(uint)nvec_msg_size(nvec->rx),
(uint)nvec->rx->pos);
@@ -502,7 +504,7 @@ static void nvec_rx_completed(struct nvec_chip *nvec)
nvec->state = 0;
/* Battery quirk - Often incomplete, and likes to crash */
- if (nvec->rx->data[0] == NVEC_BAT)
+ if (msg_type == NVEC_BAT)
complete(&nvec->ec_transfer);
return;
diff --git a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c
index 83b770571bee..261b3c91da6a 100644
--- a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c
+++ b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c
@@ -456,10 +456,10 @@ int rtw_parse_wpa_ie(u8 *wpa_ie, int wpa_ie_len, int *group_cipher, int *pairwis
return _FAIL;
}
- if ((*wpa_ie != WLAN_EID_VENDOR_SPECIFIC) || (*(wpa_ie+1) != (u8)(wpa_ie_len - 2)) ||
- (memcmp(wpa_ie+2, RTW_WPA_OUI_TYPE, WPA_SELECTOR_LEN))) {
+ if ((*wpa_ie != WLAN_EID_VENDOR_SPECIFIC) ||
+ (*(wpa_ie + 1) != (u8)(wpa_ie_len - 2)) ||
+ (memcmp(wpa_ie + 2, RTW_WPA_OUI_TYPE, WPA_SELECTOR_LEN)))
return _FAIL;
- }
pos = wpa_ie;
@@ -519,7 +519,7 @@ int rtw_parse_wpa2_ie(u8 *rsn_ie, int rsn_ie_len, int *group_cipher, int *pairwi
return _FAIL;
}
- if ((*rsn_ie != WLAN_EID_RSN) || (*(rsn_ie+1) != (u8)(rsn_ie_len - 2)))
+ if ((*rsn_ie != WLAN_EID_RSN) || (*(rsn_ie + 1) != (u8)(rsn_ie_len - 2)))
return _FAIL;
pos = rsn_ie;
@@ -585,20 +585,25 @@ int rtw_get_wapi_ie(u8 *in_ie, uint in_len, u8 *wapi_ie, u16 *wapi_len)
cnt = (_TIMESTAMP_ + _BEACON_ITERVAL_ + _CAPABILITY_);
while (cnt < in_len) {
+ if (cnt + 2 > in_len)
+ break;
+ if (cnt + 2 + in_ie[cnt + 1] > in_len)
+ break;
authmode = in_ie[cnt];
- /* if (authmode == WLAN_EID_BSS_AC_ACCESS_DELAY) */
- if (authmode == WLAN_EID_BSS_AC_ACCESS_DELAY && (!memcmp(&in_ie[cnt+6], wapi_oui1, 4) ||
- !memcmp(&in_ie[cnt+6], wapi_oui2, 4))) {
+ if (authmode == WLAN_EID_BSS_AC_ACCESS_DELAY &&
+ in_ie[cnt + 1] >= 8 &&
+ (!memcmp(&in_ie[cnt + 6], wapi_oui1, 4) ||
+ !memcmp(&in_ie[cnt + 6], wapi_oui2, 4))) {
if (wapi_ie)
- memcpy(wapi_ie, &in_ie[cnt], in_ie[cnt+1]+2);
+ memcpy(wapi_ie, &in_ie[cnt], in_ie[cnt + 1] + 2);
if (wapi_len)
- *wapi_len = in_ie[cnt+1]+2;
+ *wapi_len = in_ie[cnt + 1] + 2;
- cnt += in_ie[cnt+1]+2; /* get next */
+ cnt += in_ie[cnt + 1] + 2; /* get next */
} else {
- cnt += in_ie[cnt+1]+2; /* get next */
+ cnt += in_ie[cnt + 1] + 2; /* get next */
}
}
@@ -620,11 +625,17 @@ void rtw_get_sec_ie(u8 *in_ie, uint in_len, u8 *rsn_ie, u16 *rsn_len, u8 *wpa_ie
cnt = (_TIMESTAMP_ + _BEACON_ITERVAL_ + _CAPABILITY_);
while (cnt < in_len) {
+ if (cnt + 2 > in_len)
+ break;
+ if (cnt + 2 + in_ie[cnt + 1] > in_len)
+ break;
authmode = in_ie[cnt];
- if ((authmode == WLAN_EID_VENDOR_SPECIFIC) && (!memcmp(&in_ie[cnt+2], &wpa_oui[0], 4))) {
+ if ((authmode == WLAN_EID_VENDOR_SPECIFIC) &&
+ in_ie[cnt + 1] >= 4 &&
+ (!memcmp(&in_ie[cnt + 2], &wpa_oui[0], 4))) {
if (wpa_ie)
- memcpy(wpa_ie, &in_ie[cnt], in_ie[cnt+1]+2);
+ memcpy(wpa_ie, &in_ie[cnt], in_ie[cnt + 1] + 2);
*wpa_len = in_ie[cnt + 1] + 2;
cnt += in_ie[cnt + 1] + 2; /* get next */
@@ -633,10 +644,10 @@ void rtw_get_sec_ie(u8 *in_ie, uint in_len, u8 *rsn_ie, u16 *rsn_len, u8 *wpa_ie
if (rsn_ie)
memcpy(rsn_ie, &in_ie[cnt], in_ie[cnt + 1] + 2);
- *rsn_len = in_ie[cnt+1]+2;
- cnt += in_ie[cnt+1]+2; /* get next */
+ *rsn_len = in_ie[cnt + 1] + 2;
+ cnt += in_ie[cnt + 1] + 2; /* get next */
} else {
- cnt += in_ie[cnt+1]+2; /* get next */
+ cnt += in_ie[cnt + 1] + 2; /* get next */
}
}
}
@@ -668,20 +679,20 @@ u8 *rtw_get_wps_ie(u8 *in_ie, uint in_len, u8 *wps_ie, uint *wps_ielen)
while (cnt < in_len) {
eid = in_ie[cnt];
- if ((eid == WLAN_EID_VENDOR_SPECIFIC) && (!memcmp(&in_ie[cnt+2], wps_oui, 4))) {
+ if ((eid == WLAN_EID_VENDOR_SPECIFIC) && (!memcmp(&in_ie[cnt + 2], wps_oui, 4))) {
wpsie_ptr = &in_ie[cnt];
if (wps_ie)
- memcpy(wps_ie, &in_ie[cnt], in_ie[cnt+1]+2);
+ memcpy(wps_ie, &in_ie[cnt], in_ie[cnt + 1] + 2);
if (wps_ielen)
- *wps_ielen = in_ie[cnt+1]+2;
+ *wps_ielen = in_ie[cnt + 1] + 2;
- cnt += in_ie[cnt+1]+2;
+ cnt += in_ie[cnt + 1] + 2;
break;
}
- cnt += in_ie[cnt+1]+2; /* goto next */
+ cnt += in_ie[cnt + 1] + 2; /* goto next */
}
return wpsie_ptr;
@@ -706,6 +717,9 @@ u8 *rtw_get_wps_attr(u8 *wps_ie, uint wps_ielen, u16 target_attr_id, u8 *buf_att
if (len_attr)
*len_attr = 0;
+ if (wps_ielen < 6)
+ return attr_ptr;
+
if ((wps_ie[0] != WLAN_EID_VENDOR_SPECIFIC) ||
(memcmp(wps_ie + 2, wps_oui, 4))) {
return attr_ptr;
@@ -716,6 +730,8 @@ u8 *rtw_get_wps_attr(u8 *wps_ie, uint wps_ielen, u16 target_attr_id, u8 *buf_att
while (attr_ptr - wps_ie < wps_ielen) {
/* 4 = 2(Attribute ID) + 2(Length) */
+ if (attr_ptr + 4 > wps_ie + wps_ielen)
+ break;
u16 attr_id = get_unaligned_be16(attr_ptr);
u16 attr_data_len = get_unaligned_be16(attr_ptr + 2);
u16 attr_len = attr_data_len + 4;
@@ -759,12 +775,12 @@ u8 *rtw_get_wps_attr_content(u8 *wps_ie, uint wps_ielen, u16 target_attr_id, u8
if (attr_ptr && attr_len) {
if (buf_content)
- memcpy(buf_content, attr_ptr+4, attr_len-4);
+ memcpy(buf_content, attr_ptr + 4, attr_len - 4);
if (len_content)
- *len_content = attr_len-4;
+ *len_content = attr_len - 4;
- return attr_ptr+4;
+ return attr_ptr + 4;
}
return NULL;
@@ -1010,20 +1026,25 @@ static int rtw_get_cipher_info(struct wlan_network *pnetwork)
int group_cipher = 0, pairwise_cipher = 0, is8021x = 0;
int ret = _FAIL;
- pbuf = rtw_get_wpa_ie(&pnetwork->network.ies[12], &wpa_ielen, pnetwork->network.ie_length-12);
+ pbuf = rtw_get_wpa_ie(&pnetwork->network.ies[12],
+ &wpa_ielen,
+ pnetwork->network.ie_length - 12);
if (pbuf && (wpa_ielen > 0)) {
- if (_SUCCESS == rtw_parse_wpa_ie(pbuf, wpa_ielen+2, &group_cipher, &pairwise_cipher, &is8021x)) {
+ if (rtw_parse_wpa_ie(pbuf, wpa_ielen + 2, &group_cipher,
+ &pairwise_cipher, &is8021x) == _SUCCESS) {
pnetwork->bcn_info.pairwise_cipher = pairwise_cipher;
pnetwork->bcn_info.group_cipher = group_cipher;
pnetwork->bcn_info.is_8021x = is8021x;
ret = _SUCCESS;
}
} else {
- pbuf = rtw_get_wpa2_ie(&pnetwork->network.ies[12], &wpa_ielen, pnetwork->network.ie_length-12);
+ pbuf = rtw_get_wpa2_ie(&pnetwork->network.ies[12], &wpa_ielen,
+ pnetwork->network.ie_length - 12);
if (pbuf && (wpa_ielen > 0)) {
- if (_SUCCESS == rtw_parse_wpa2_ie(pbuf, wpa_ielen+2, &group_cipher, &pairwise_cipher, &is8021x)) {
+ if (rtw_parse_wpa2_ie(pbuf, wpa_ielen + 2, &group_cipher,
+ &pairwise_cipher, &is8021x) == _SUCCESS) {
pnetwork->bcn_info.pairwise_cipher = pairwise_cipher;
pnetwork->bcn_info.group_cipher = group_cipher;
pnetwork->bcn_info.is_8021x = is8021x;
@@ -1092,21 +1113,21 @@ u16 rtw_mcs_rate(u8 bw_40MHz, u8 short_GI, unsigned char *MCS_rate)
u16 max_rate = 0;
if (MCS_rate[0] & BIT(7))
- max_rate = (bw_40MHz) ? ((short_GI)?1500:1350):((short_GI)?722:650);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 1500 : 1350) : ((short_GI) ? 722 : 650);
else if (MCS_rate[0] & BIT(6))
- max_rate = (bw_40MHz) ? ((short_GI)?1350:1215):((short_GI)?650:585);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 1350 : 1215) : ((short_GI) ? 650 : 585);
else if (MCS_rate[0] & BIT(5))
- max_rate = (bw_40MHz) ? ((short_GI)?1200:1080):((short_GI)?578:520);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 1200 : 1080) : ((short_GI) ? 578 : 520);
else if (MCS_rate[0] & BIT(4))
- max_rate = (bw_40MHz) ? ((short_GI)?900:810):((short_GI)?433:390);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 900 : 810) : ((short_GI) ? 433 : 390);
else if (MCS_rate[0] & BIT(3))
- max_rate = (bw_40MHz) ? ((short_GI)?600:540):((short_GI)?289:260);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 600 : 540) : ((short_GI) ? 289 : 260);
else if (MCS_rate[0] & BIT(2))
- max_rate = (bw_40MHz) ? ((short_GI)?450:405):((short_GI)?217:195);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 450 : 405) : ((short_GI) ? 217 : 195);
else if (MCS_rate[0] & BIT(1))
- max_rate = (bw_40MHz) ? ((short_GI)?300:270):((short_GI)?144:130);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 300 : 270) : ((short_GI) ? 144 : 130);
else if (MCS_rate[0] & BIT(0))
- max_rate = (bw_40MHz) ? ((short_GI)?150:135):((short_GI)?72:65);
+ max_rate = (bw_40MHz) ? ((short_GI) ? 150 : 135) : ((short_GI) ? 72 : 65);
return max_rate;
}
diff --git a/drivers/staging/rtl8723bs/core/rtw_security.c b/drivers/staging/rtl8723bs/core/rtw_security.c
index 7bbb0abf1d87..38ea6bf68295 100644
--- a/drivers/staging/rtl8723bs/core/rtw_security.c
+++ b/drivers/staging/rtl8723bs/core/rtw_security.c
@@ -1537,7 +1537,7 @@ void rtw_sec_restore_wep_key(struct adapter *adapter)
struct security_priv *securitypriv = &(adapter->securitypriv);
signed int keyid;
- if ((_WEP40_ == securitypriv->dot11PrivacyAlgrthm) || (_WEP104_ == securitypriv->dot11PrivacyAlgrthm)) {
+ if ((securitypriv->dot11PrivacyAlgrthm == _WEP40_) || (securitypriv->dot11PrivacyAlgrthm == _WEP104_)) {
for (keyid = 0; keyid < 4; keyid++) {
if (securitypriv->key_mask & BIT(keyid)) {
if (keyid == securitypriv->dot11PrivacyKeyIndex)
diff --git a/drivers/target/iscsi/iscsi_target_auth.c b/drivers/target/iscsi/iscsi_target_auth.c
index 02a4c9aff98d..41e4def46ede 100644
--- a/drivers/target/iscsi/iscsi_target_auth.c
+++ b/drivers/target/iscsi/iscsi_target_auth.c
@@ -437,9 +437,11 @@ static int chap_server_compute_hash(
}
if (type == HEX)
- ret = kstrtoul(&identifier[2], 0, &id);
+ ret = kstrtoul(identifier, 16, &id);
+ else if (type == DECIMAL)
+ ret = kstrtoul(identifier, 10, &id);
else
- ret = kstrtoul(identifier, 0, &id);
+ ret = -EINVAL;
if (ret < 0) {
pr_err("kstrtoul() failed for CHAP identifier: %d\n", ret);
diff --git a/drivers/target/loopback/tcm_loop.c b/drivers/target/loopback/tcm_loop.c
index 1d1043cc649c..31b2563fee5e 100644
--- a/drivers/target/loopback/tcm_loop.c
+++ b/drivers/target/loopback/tcm_loop.c
@@ -268,69 +268,6 @@ static int tcm_loop_device_reset(struct scsi_cmnd *sc)
return (ret == TMR_FUNCTION_COMPLETE) ? SUCCESS : FAILED;
}
-static bool tcm_loop_flush_work_iter(struct request *rq, void *data)
-{
- struct scsi_cmnd *sc = blk_mq_rq_to_pdu(rq);
- struct tcm_loop_cmd *tl_cmd = scsi_cmd_priv(sc);
- struct se_cmd *se_cmd = &tl_cmd->tl_se_cmd;
-
- flush_work(&se_cmd->work);
- return true;
-}
-
-static int tcm_loop_target_reset(struct scsi_cmnd *sc)
-{
- struct tcm_loop_hba *tl_hba;
- struct tcm_loop_tpg *tl_tpg;
- struct Scsi_Host *sh = sc->device->host;
- int ret;
-
- /*
- * Locate the tcm_loop_hba_t pointer
- */
- tl_hba = *(struct tcm_loop_hba **)shost_priv(sh);
- if (!tl_hba) {
- pr_err("Unable to perform device reset without active I_T Nexus\n");
- return FAILED;
- }
- /*
- * Locate the tl_tpg pointer from TargetID in sc->device->id
- */
- tl_tpg = &tl_hba->tl_hba_tpgs[sc->device->id];
- if (!tl_tpg)
- return FAILED;
-
- /*
- * Issue a LUN_RESET to drain all commands that the target core
- * knows about. This handles commands not yet marked CMD_T_COMPLETE.
- */
- ret = tcm_loop_issue_tmr(tl_tpg, sc->device->lun, 0, TMR_LUN_RESET);
- if (ret != TMR_FUNCTION_COMPLETE)
- return FAILED;
-
- /*
- * Flush any deferred target core completion work that may still be
- * queued. Commands that already had CMD_T_COMPLETE set before the TMR
- * are skipped by the TMR drain, but their async completion work
- * (transport_lun_remove_cmd → percpu_ref_put, release_cmd → scsi_done)
- * may still be pending in target_completion_wq.
- *
- * The SCSI EH will reuse in-flight scsi_cmnd structures for recovery
- * commands (e.g. TUR) immediately after this handler returns SUCCESS —
- * if deferred work is still pending, the memset in queuecommand would
- * zero the se_cmd while the work accesses it, leaking the LUN
- * percpu_ref and hanging configfs unlink forever.
- *
- * Use blk_mq_tagset_busy_iter() to find all started requests and
- * flush_work() on each — the same pattern used by mpi3mr, scsi_debug,
- * and other SCSI drivers to drain outstanding commands during reset.
- */
- blk_mq_tagset_busy_iter(&sh->tag_set, tcm_loop_flush_work_iter, NULL);
-
- tl_tpg->tl_transport_status = TCM_TRANSPORT_ONLINE;
- return SUCCESS;
-}
-
static const struct scsi_host_template tcm_loop_driver_template = {
.show_info = tcm_loop_show_info,
.proc_name = "tcm_loopback",
@@ -339,7 +276,6 @@ static const struct scsi_host_template tcm_loop_driver_template = {
.change_queue_depth = scsi_change_queue_depth,
.eh_abort_handler = tcm_loop_abort_task,
.eh_device_reset_handler = tcm_loop_device_reset,
- .eh_target_reset_handler = tcm_loop_target_reset,
.this_id = -1,
.sg_tablesize = 256,
.max_sectors = 0xFFFF,
diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c
index ec7bc6e30228..3af9a26cdefc 100644
--- a/drivers/target/target_core_fabric_lib.c
+++ b/drivers/target/target_core_fabric_lib.c
@@ -289,13 +289,24 @@ static void sbp_parse_pr_out_transport_id(char *buf, char *i_str)
static bool iscsi_parse_pr_out_transport_id(
struct se_portal_group *se_tpg,
char *buf,
+ u32 buf_len,
u32 *out_tid_len,
char **port_nexus_ptr,
char *i_str)
{
char *p;
+ u32 tid_len;
int i;
- u8 format_code = (buf[0] & 0xc0);
+ u8 format_code;
+
+ /*
+ * The 4-byte iSCSI TransportID header (FORMAT CODE + 2-byte ADDITIONAL
+ * LENGTH) must be present before any of it can be parsed.
+ */
+ if (buf_len < 4)
+ return false;
+
+ format_code = buf[0] & 0xc0;
/*
* Check for FORMAT CODE 00b or 01b from spc4r17, section 7.5.4.6:
*
@@ -315,15 +326,17 @@ static bool iscsi_parse_pr_out_transport_id(
return false;
}
/*
- * If the caller wants the TransportID Length, we set that value for the
- * entire iSCSI Tarnsport ID now.
+ * Reconstruct the self-described TransportID length from the ADDITIONAL
+ * LENGTH field plus the 4-byte header. Reject it if it is below the
+ * spc4r17 section 7.5.4.6 minimum (ADDITIONAL LENGTH shall be at least
+ * 20) or if it runs past the bytes actually received, so that every
+ * access below stays inside the TransportID.
*/
- if (out_tid_len) {
- /* The shift works thanks to integer promotion rules */
- *out_tid_len = get_unaligned_be16(&buf[2]);
- /* Add four bytes for iSCSI Transport ID header */
- *out_tid_len += 4;
- }
+ tid_len = get_unaligned_be16(&buf[2]) + 4;
+ if (tid_len < 24 || tid_len > buf_len)
+ return false;
+ if (out_tid_len)
+ *out_tid_len = tid_len;
/*
* Check for ',i,0x' separator between iSCSI Name and iSCSI Initiator
@@ -331,16 +344,32 @@ static bool iscsi_parse_pr_out_transport_id(
* format.
*/
if (format_code == 0x40) {
- p = strstr(&buf[4], ",i,0x");
+ p = strnstr(&buf[4], ",i,0x", tid_len - 4);
if (!p) {
- pr_err("Unable to locate \",i,0x\" separator"
- " for Initiator port identifier: %s\n",
- &buf[4]);
+ pr_err("Unable to locate \",i,0x\" separator in iSCSI TransportID\n");
+ return false;
+ }
+ /*
+ * The iSCSI name runs from &buf[4] up to the separator; reject it
+ * if it cannot fit in i_str[TRANSPORT_IQN_LEN].
+ */
+ if (p - &buf[4] >= TRANSPORT_IQN_LEN) {
+ pr_err("iSCSI Initiator port name too long in TransportID\n");
return false;
}
*p = '\0'; /* Terminate iSCSI Name */
p += 5; /* Skip over ",i,0x" separator */
+ /*
+ * The ISID must follow the separator. A ",i,0x" sitting at the
+ * very end of the TransportID leaves no ISID and would point the
+ * port nexus at buf + tid_len, i.e. past the descriptor, which
+ * the registration code then reads as the ISID string.
+ */
+ if (p >= buf + tid_len) {
+ pr_err("Missing ISID in iSCSI Initiator port TransportID\n");
+ return false;
+ }
*port_nexus_ptr = p;
/*
* Go ahead and do the lower case conversion of the received
@@ -348,7 +377,7 @@ static bool iscsi_parse_pr_out_transport_id(
* for comparison against the running iSCSI session's ISID from
* iscsi_target.c:lio_sess_get_initiator_sid()
*/
- for (i = 0; i < 12; i++) {
+ for (i = 0; i < 12 && p < buf + tid_len; i++) {
/*
* The first ISCSI INITIATOR SESSION ID field byte
* containing an ASCII null character terminates the
@@ -366,10 +395,22 @@ static bool iscsi_parse_pr_out_transport_id(
*p = tolower(*p);
p++;
}
- } else
+ strscpy(i_str, &buf[4], TRANSPORT_IQN_LEN);
+ } else {
*port_nexus_ptr = NULL;
-
- strscpy(i_str, &buf[4], TRANSPORT_IQN_LEN);
+ /*
+ * FORMAT CODE 00b: the name occupies buf[4..tid_len-1]. The
+ * declared length tid_len - 4 must fit in i_str[TRANSPORT_IQN_LEN].
+ * (For 01b the same tid_len bound would be over-restrictive: the
+ * descriptor also carries the separator and ISID, so a legal
+ * <=223-byte name gives tid_len up to 244.)
+ */
+ if (tid_len - 4 >= TRANSPORT_IQN_LEN) {
+ pr_err("iSCSI Initiator port name too long in TransportID\n");
+ return false;
+ }
+ strscpy(i_str, &buf[4], tid_len - 4);
+ }
return true;
}
@@ -419,8 +460,16 @@ int target_get_pr_transport_id(struct se_node_acl *nacl,
}
bool target_parse_pr_out_transport_id(struct se_portal_group *tpg,
- char *buf, u32 *out_tid_len, char **port_nexus_ptr, char *i_str)
+ char *buf, u32 buf_len, u32 *out_tid_len,
+ char **port_nexus_ptr, char *i_str)
{
+ /*
+ * The fixed-length SAS/SRP/FCP/SBP TransportIDs are 24 bytes; the iSCSI
+ * format is variable and bounds itself against buf_len below.
+ */
+ if (tpg->proto_id != SCSI_PROTOCOL_ISCSI && buf_len < 24)
+ return false;
+
switch (tpg->proto_id) {
case SCSI_PROTOCOL_SAS:
/*
@@ -439,8 +488,8 @@ bool target_parse_pr_out_transport_id(struct se_portal_group *tpg,
sbp_parse_pr_out_transport_id(buf, i_str);
break;
case SCSI_PROTOCOL_ISCSI:
- return iscsi_parse_pr_out_transport_id(tpg, buf, out_tid_len,
- port_nexus_ptr, i_str);
+ return iscsi_parse_pr_out_transport_id(tpg, buf, buf_len,
+ out_tid_len, port_nexus_ptr, i_str);
default:
pr_err("Unknown proto_id: 0x%02x\n", tpg->proto_id);
return false;
diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h
index 20aab1f50565..32d0da505baf 100644
--- a/drivers/target/target_core_internal.h
+++ b/drivers/target/target_core_internal.h
@@ -104,7 +104,8 @@ int target_get_pr_transport_id(struct se_node_acl *nacl,
struct t10_pr_registration *pr_reg, int *format_code,
unsigned char *buf);
bool target_parse_pr_out_transport_id(struct se_portal_group *tpg,
- char *buf, u32 *out_tid_len, char **port_nexus_ptr, char *i_str);
+ char *buf, u32 buf_len, u32 *out_tid_len,
+ char **port_nexus_ptr, char *i_str);
/* target_core_hba.c */
struct se_hba *core_alloc_hba(const char *, u32, u32);
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index 82061cbe6781..c7f16c2ccf1e 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -1573,7 +1573,7 @@ core_scsi3_decode_spec_i_port(
iport_ptr = NULL;
tid_found = target_parse_pr_out_transport_id(tmp_tpg,
- ptr, &tid_len, &iport_ptr, i_str);
+ ptr, tpdl, &tid_len, &iport_ptr, i_str);
if (!tid_found)
continue;
/*
@@ -3281,7 +3281,7 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key,
goto out;
}
tid_found = target_parse_pr_out_transport_id(dest_se_tpg,
- &buf[24], &tmp_tid_len, &iport_ptr, initiator_str);
+ &buf[24], tid_len, &tmp_tid_len, &iport_ptr, initiator_str);
if (!tid_found) {
pr_err("SPC-3 PR REGISTER_AND_MOVE: Unable to locate"
" initiator_str from Transport ID\n");
@@ -3289,9 +3289,6 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key,
goto out;
}
- transport_kunmap_data_sg(cmd);
- buf = NULL;
-
pr_debug("SPC-3 PR [%s] Extracted initiator %s identifier: %s"
" %s\n", dest_tf_ops->fabric_name, (iport_ptr != NULL) ?
"port" : "device", initiator_str, (iport_ptr != NULL) ?
@@ -3528,6 +3525,11 @@ core_scsi3_emulate_pro_register_and_move(struct se_cmd *cmd, u64 res_key,
core_scsi3_update_and_write_aptpl(cmd->se_dev, aptpl);
core_scsi3_put_pr_reg(dest_pr_reg);
+ /*
+ * iport_ptr aliases the PR-OUT parameter list mapped above, so the
+ * buffer is unmapped only here on success (and at out: on error).
+ */
+ transport_kunmap_data_sg(cmd);
return 0;
out:
if (buf)
diff --git a/drivers/thermal/intel/therm_throt.c b/drivers/thermal/intel/therm_throt.c
index e69868e868eb..dfdc8c1b0eb5 100644
--- a/drivers/thermal/intel/therm_throt.c
+++ b/drivers/thermal/intel/therm_throt.c
@@ -528,8 +528,13 @@ static int thermal_throttle_online(unsigned int cpu)
{
struct thermal_state *state = &per_cpu(thermal_state, cpu);
struct device *dev = get_cpu_device(cpu);
+ int err;
u32 l;
+ err = thermal_throttle_add_dev(dev, cpu);
+ if (err)
+ return err;
+
state->package_throttle.level = PACKAGE_LEVEL;
state->core_throttle.level = CORE_LEVEL;
@@ -547,7 +552,7 @@ static int thermal_throttle_online(unsigned int cpu)
l = apic_read(APIC_LVTTHMR);
apic_write(APIC_LVTTHMR, l & ~APIC_LVT_MASKED);
- return thermal_throttle_add_dev(dev, cpu);
+ return err;
}
static int thermal_throttle_offline(unsigned int cpu)
diff --git a/drivers/thermal/testing/command.c b/drivers/thermal/testing/command.c
index ba11d70e8021..a0eb43e8fdce 100644
--- a/drivers/thermal/testing/command.c
+++ b/drivers/thermal/testing/command.c
@@ -86,7 +86,10 @@
#include "thermal_testing.h"
-struct dentry *d_testing;
+struct workqueue_struct *tt_wq __ro_after_init;
+
+struct dentry *d_testing __ro_after_init;
+static struct dentry *d_command __ro_after_init;
#define TT_COMMAND_SIZE 16
@@ -116,18 +119,30 @@ static int tt_command_exec(int index, const char *arg)
break;
case TT_CMD_DELTZ:
+ if (!arg || !*arg)
+ return -EINVAL;
+
ret = tt_del_tz(arg);
break;
case TT_CMD_TZADDTRIP:
+ if (!arg || !*arg)
+ return -EINVAL;
+
ret = tt_zone_add_trip(arg);
break;
case TT_CMD_TZREG:
+ if (!arg || !*arg)
+ return -EINVAL;
+
ret = tt_zone_reg(arg);
break;
case TT_CMD_TZUNREG:
+ if (!arg || !*arg)
+ return -EINVAL;
+
ret = tt_zone_unreg(arg);
break;
@@ -201,17 +216,42 @@ static const struct file_operations tt_command_fops = {
static int __init thermal_testing_init(void)
{
+ int error;
+
+ tt_wq = alloc_workqueue("thermal_testing", WQ_UNBOUND, 0);
+ if (!tt_wq)
+ return -ENOMEM;
+
d_testing = debugfs_create_dir("thermal-testing", NULL);
- if (!IS_ERR(d_testing))
- debugfs_create_file("command", 0200, d_testing, NULL,
- &tt_command_fops);
+ if (IS_ERR(d_testing)) {
+ error = PTR_ERR(d_testing);
+ goto destroy_wq;
+ }
+
+ d_command = debugfs_create_file("command", 0200, d_testing, NULL, &tt_command_fops);
+ if (IS_ERR(d_command)) {
+ error = PTR_ERR(d_command);
+ goto remove_d_testing;
+ }
return 0;
+
+remove_d_testing:
+ debugfs_remove(d_testing);
+destroy_wq:
+ destroy_workqueue(tt_wq);
+ return error;
}
module_init(thermal_testing_init);
static void __exit thermal_testing_exit(void)
{
+ /* First, prevent new commands from being entered. */
+ debugfs_remove(d_command);
+ /* Flush commands in progress (if any). */
+ flush_workqueue(tt_wq);
+ destroy_workqueue(tt_wq);
+ /* Remove the directory structure and clean up. */
debugfs_remove(d_testing);
tt_zone_cleanup();
}
diff --git a/drivers/thermal/testing/thermal_testing.h b/drivers/thermal/testing/thermal_testing.h
index c790a32aae4e..5880c9a63dba 100644
--- a/drivers/thermal/testing/thermal_testing.h
+++ b/drivers/thermal/testing/thermal_testing.h
@@ -1,4 +1,12 @@
/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/workqueue.h>
+
+extern struct workqueue_struct *tt_wq;
+
+static inline void tt_queue_work(struct work_struct *work)
+{
+ queue_work(tt_wq, work);
+}
extern struct dentry *d_testing;
diff --git a/drivers/thermal/testing/zone.c b/drivers/thermal/testing/zone.c
index 1f01f4952703..3d5c6804106a 100644
--- a/drivers/thermal/testing/zone.c
+++ b/drivers/thermal/testing/zone.c
@@ -13,7 +13,6 @@
#include <linux/idr.h>
#include <linux/list.h>
#include <linux/thermal.h>
-#include <linux/workqueue.h>
#include "thermal_testing.h"
@@ -209,7 +208,7 @@ int tt_add_tz(void)
INIT_WORK(&tt_work->work, tt_add_tz_work_fn);
tt_work->tt_zone = no_free_ptr(tt_zone);
- schedule_work(&(no_free_ptr(tt_work)->work));
+ tt_queue_work(&(no_free_ptr(tt_work)->work));
return 0;
}
@@ -272,7 +271,7 @@ int tt_del_tz(const char *arg)
INIT_WORK(&tt_work->work, tt_del_tz_work_fn);
tt_work->tt_zone = tt_zone;
- schedule_work(&(no_free_ptr(tt_work)->work));
+ tt_queue_work(&(no_free_ptr(tt_work)->work));
return 0;
}
@@ -369,7 +368,7 @@ int tt_zone_add_trip(const char *arg)
INIT_WORK(&tt_work->work, tt_zone_add_trip_work_fn);
tt_work->tt_zone = no_free_ptr(tt_zone);
tt_work->tt_trip = no_free_ptr(tt_trip);
- schedule_work(&(no_free_ptr(tt_work)->work));
+ tt_queue_work(&(no_free_ptr(tt_work)->work));
return 0;
}
diff --git a/drivers/thermal/thermal_hwmon.c b/drivers/thermal/thermal_hwmon.c
index f0e504fd866a..112188c15c30 100644
--- a/drivers/thermal/thermal_hwmon.c
+++ b/drivers/thermal/thermal_hwmon.c
@@ -40,6 +40,7 @@ struct thermal_hwmon_temp {
struct thermal_zone_device *tz;
struct thermal_hwmon_attr temp_input; /* hwmon sys attr */
struct thermal_hwmon_attr temp_crit; /* hwmon sys attr */
+ bool temp_crit_present;
};
static LIST_HEAD(thermal_hwmon_list);
@@ -194,6 +195,8 @@ int thermal_add_hwmon_sysfs(struct thermal_zone_device *tz)
&temp->temp_crit.attr);
if (result)
goto unregister_input;
+
+ temp->temp_crit_present = true;
}
mutex_lock(&thermal_hwmon_list_lock);
@@ -238,7 +241,7 @@ void thermal_remove_hwmon_sysfs(struct thermal_zone_device *tz)
}
device_remove_file(hwmon->device, &temp->temp_input.attr);
- if (thermal_zone_crit_temp_valid(tz))
+ if (temp->temp_crit_present)
device_remove_file(hwmon->device, &temp->temp_crit.attr);
mutex_lock(&thermal_hwmon_list_lock);
diff --git a/drivers/thunderbolt/debugfs.c b/drivers/thunderbolt/debugfs.c
index 350310bd0fee..e2178fe368d4 100644
--- a/drivers/thunderbolt/debugfs.c
+++ b/drivers/thunderbolt/debugfs.c
@@ -797,7 +797,9 @@ margining_error_counter_write(struct file *file, const char __user *user_buf,
else if (!strcmp(buf, "stop"))
error_counter = USB4_MARGIN_SW_ERROR_COUNTER_STOP;
else
- return -EINVAL;
+ goto err_free;
+
+ free_page((unsigned long)buf);
scoped_cond_guard(mutex_intr, return -ERESTARTSYS, &tb->lock) {
if (!margining->software)
@@ -807,6 +809,10 @@ margining_error_counter_write(struct file *file, const char __user *user_buf,
}
return count;
+
+err_free:
+ free_page((unsigned long)buf);
+ return -EINVAL;
}
static int margining_error_counter_show(struct seq_file *s, void *not_used)
diff --git a/drivers/tty/serial/8250/8250_omap.c b/drivers/tty/serial/8250/8250_omap.c
index 0f4ce0c69114..07b1918e35ec 100644
--- a/drivers/tty/serial/8250/8250_omap.c
+++ b/drivers/tty/serial/8250/8250_omap.c
@@ -965,11 +965,12 @@ static void __dma_rx_do_complete(struct uart_8250_port *p)
dev_err(p->port.dev, "teardown incomplete\n");
}
}
+
+ dma->rx_running = 0;
if (!count)
goto out;
ret = tty_insert_flip_string(tty_port, dma->rx_buf, count);
- dma->rx_running = 0;
p->port.icount.rx += ret;
p->port.icount.buf_overrun += count - ret;
out:
diff --git a/drivers/tty/serial/max310x.c b/drivers/tty/serial/max310x.c
index 9850446ae703..163088b4b543 100644
--- a/drivers/tty/serial/max310x.c
+++ b/drivers/tty/serial/max310x.c
@@ -1198,6 +1198,17 @@ static void max310x_gpio_set(struct gpio_chip *chip, unsigned int offset, int va
value ? 1 << (offset % 4) : 0);
}
+static int max310x_gpio_get_direction(struct gpio_chip *chip, unsigned int offset)
+{
+ struct max310x_port *s = gpiochip_get_data(chip);
+ struct uart_port *port = &s->p[offset / 4].port;
+ unsigned int val;
+
+ val = max310x_port_read(port, MAX310X_GPIOCFG_REG);
+
+ return val & BIT(offset % 4) ? GPIO_LINE_DIRECTION_OUT : GPIO_LINE_DIRECTION_IN;
+}
+
static int max310x_gpio_direction_input(struct gpio_chip *chip, unsigned int offset)
{
struct max310x_port *s = gpiochip_get_data(chip);
@@ -1407,6 +1418,7 @@ static int max310x_probe(struct device *dev, const struct max310x_devtype *devty
s->gpio.owner = THIS_MODULE;
s->gpio.parent = dev;
s->gpio.label = devtype->name;
+ s->gpio.get_direction = max310x_gpio_get_direction;
s->gpio.direction_input = max310x_gpio_direction_input;
s->gpio.get = max310x_gpio_get;
s->gpio.direction_output= max310x_gpio_direction_output;
diff --git a/drivers/tty/serial/msm_serial.c b/drivers/tty/serial/msm_serial.c
index 9eeabedfb4d0..138abadf2d61 100644
--- a/drivers/tty/serial/msm_serial.c
+++ b/drivers/tty/serial/msm_serial.c
@@ -1228,7 +1228,8 @@ static int msm_startup(struct uart_port *port)
data |= MSM_UART_MR1_AUTO_RFR_LEVEL0 & rfr_level;
msm_write(port, data, MSM_UART_MR1);
- if (msm_port->is_uartdm) {
+ /* Disable DMA for console to prevent PIO/DMA collisions */
+ if (msm_port->is_uartdm && !uart_console(port)) {
msm_request_tx_dma(msm_port, msm_port->uart.mapbase);
msm_request_rx_dma(msm_port, msm_port->uart.mapbase);
}
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 22d3cb0ddbca..205427f6c465 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -5456,7 +5456,7 @@ ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp,
default:
dev_err(hba->dev,
"Unexpected request response code = %x\n",
- result);
+ ufshcd_get_req_rsp(lrbp->ucd_rsp_ptr));
result = DID_ERROR << 16;
break;
}
diff --git a/drivers/usb/atm/ueagle-atm.c b/drivers/usb/atm/ueagle-atm.c
index e8e43c38aa1b..cd53d36aa307 100644
--- a/drivers/usb/atm/ueagle-atm.c
+++ b/drivers/usb/atm/ueagle-atm.c
@@ -599,7 +599,9 @@ static int uea_send_modem_cmd(struct usb_device *usb,
static void uea_upload_pre_firmware(const struct firmware *fw_entry,
void *context)
{
- struct usb_device *usb = context;
+ struct usb_interface *intf = context;
+ struct usb_device *usb = interface_to_usbdev(intf);
+ struct completion *fw_done = usb_get_intfdata(intf);
const u8 *pfw;
u8 value;
u32 crc = 0;
@@ -670,15 +672,17 @@ static void uea_upload_pre_firmware(const struct firmware *fw_entry,
err:
release_firmware(fw_entry);
uea_leaves(usb);
+ complete(fw_done);
}
/*
* uea_load_firmware - Load usb firmware for pre-firmware devices.
*/
-static int uea_load_firmware(struct usb_device *usb, unsigned int ver)
+static int uea_load_firmware(struct usb_interface *intf, unsigned int ver)
{
int ret;
char *fw_name = EAGLE_FIRMWARE;
+ struct usb_device *usb = interface_to_usbdev(intf);
uea_enters(usb);
uea_info(usb, "pre-firmware device, uploading firmware\n");
@@ -702,7 +706,7 @@ static int uea_load_firmware(struct usb_device *usb, unsigned int ver)
}
ret = request_firmware_nowait(THIS_MODULE, 1, fw_name, &usb->dev,
- GFP_KERNEL, usb,
+ GFP_KERNEL, intf,
uea_upload_pre_firmware);
if (ret)
uea_err(usb, "firmware %s is not available\n", fw_name);
@@ -2597,8 +2601,23 @@ static int uea_probe(struct usb_interface *intf, const struct usb_device_id *id)
usb_reset_device(usb);
- if (UEA_IS_PREFIRM(id))
- return uea_load_firmware(usb, UEA_CHIP_VERSION(id));
+ if (UEA_IS_PREFIRM(id)) {
+ struct completion *fw_done;
+
+ /* Wait for the firmware load to be done, in .disconnect() */
+ fw_done = kzalloc(sizeof(*fw_done), GFP_KERNEL);
+ if (!fw_done)
+ return -ENOMEM;
+
+ init_completion(fw_done);
+ usb_set_intfdata(intf, fw_done);
+
+ ret = uea_load_firmware(intf, UEA_CHIP_VERSION(id));
+ if (ret)
+ kfree(fw_done);
+
+ return ret;
+ }
ret = usbatm_usb_probe(intf, id, &uea_usbatm_driver);
if (ret == 0) {
@@ -2629,6 +2648,13 @@ static void uea_disconnect(struct usb_interface *intf)
usbatm_usb_disconnect(intf);
mutex_unlock(&uea_mutex);
uea_info(usb, "ADSL device removed\n");
+ } else if (usb->config->desc.bNumInterfaces == 1) {
+ struct completion *fw_done = usb_get_intfdata(intf);
+
+ uea_dbg(usb, "pre-firmware device, waiting firmware upload\n");
+ wait_for_completion(fw_done);
+ uea_dbg(usb, "pre-firmware device, finished waiting\n");
+ kfree(fw_done);
}
uea_leaves(usb);
diff --git a/drivers/usb/gadget/function/f_fs.c b/drivers/usb/gadget/function/f_fs.c
index abc1ec50933f..ae34a12080ed 100644
--- a/drivers/usb/gadget/function/f_fs.c
+++ b/drivers/usb/gadget/function/f_fs.c
@@ -291,6 +291,7 @@ static int ffs_acquire_dev(const char *dev_name, struct ffs_data *ffs_data);
static void ffs_release_dev(struct ffs_dev *ffs_dev);
static int ffs_ready(struct ffs_data *ffs);
static void ffs_closed(struct ffs_data *ffs);
+static void ffs_reset_work(struct work_struct *work);
/* Misc helper functions ****************************************************/
@@ -1364,7 +1365,6 @@ ffs_epfile_release(struct inode *inode, struct file *file)
mutex_unlock(&epfile->dmabufs_mutex);
- __ffs_epfile_read_buffer_free(epfile);
ffs_data_closed(epfile->ffs);
return 0;
@@ -2224,6 +2224,7 @@ static struct ffs_data *ffs_data_new(const char *dev_name)
init_waitqueue_head(&ffs->ev.waitq);
init_waitqueue_head(&ffs->wait);
init_completion(&ffs->ep0req_completion);
+ INIT_WORK(&ffs->reset_work, ffs_reset_work);
/* XXX REVISIT need to update it in some places, or do we? */
ffs->ev.can_stall = 1;
@@ -2384,6 +2385,7 @@ static void ffs_epfiles_destroy(struct ffs_epfile *epfiles, unsigned count)
for (; count; --count, ++epfile) {
BUG_ON(mutex_is_locked(&epfile->mutex));
+ __ffs_epfile_read_buffer_free(epfile);
if (epfile->dentry) {
d_delete(epfile->dentry);
dput(epfile->dentry);
@@ -3770,7 +3772,6 @@ static int ffs_func_set_alt(struct usb_function *f,
if (ffs->state == FFS_DEACTIVATED) {
ffs->state = FFS_CLOSING;
- INIT_WORK(&ffs->reset_work, ffs_reset_work);
schedule_work(&ffs->reset_work);
return -ENODEV;
}
@@ -3797,7 +3798,6 @@ static void ffs_func_disable(struct usb_function *f)
if (ffs->state == FFS_DEACTIVATED) {
ffs->state = FFS_CLOSING;
- INIT_WORK(&ffs->reset_work, ffs_reset_work);
schedule_work(&ffs->reset_work);
return;
}
diff --git a/drivers/usb/host/max3421-hcd.c b/drivers/usb/host/max3421-hcd.c
index 4b5f03f683f7..8fbbf9e195b0 100644
--- a/drivers/usb/host/max3421-hcd.c
+++ b/drivers/usb/host/max3421-hcd.c
@@ -1685,6 +1685,8 @@ max3421_hub_control(struct usb_hcd *hcd, u16 type_req, u16 value, u16 index,
case ClearHubFeature:
break;
case ClearPortFeature:
+ if (index != 1)
+ goto error;
switch (value) {
case USB_PORT_FEAT_SUSPEND:
break;
@@ -1694,6 +1696,8 @@ max3421_hub_control(struct usb_hcd *hcd, u16 type_req, u16 value, u16 index,
!pdata->vbus_active_level);
fallthrough;
default:
+ if (value >= 32)
+ goto error;
max3421_hcd->port_status &= ~(1 << value);
}
break;
@@ -1726,6 +1730,8 @@ max3421_hub_control(struct usb_hcd *hcd, u16 type_req, u16 value, u16 index,
break;
case SetPortFeature:
+ if (index != 1)
+ goto error;
switch (value) {
case USB_PORT_FEAT_LINK_STATE:
case USB_PORT_FEAT_U1_TIMEOUT:
@@ -1747,6 +1753,8 @@ max3421_hub_control(struct usb_hcd *hcd, u16 type_req, u16 value, u16 index,
max3421_reset_port(hcd);
fallthrough;
default:
+ if (value >= 32)
+ goto error;
if ((max3421_hcd->port_status & USB_PORT_STAT_POWER)
!= 0)
max3421_hcd->port_status |= (1 << value);
diff --git a/drivers/usb/misc/iowarrior.c b/drivers/usb/misc/iowarrior.c
index 38f6dcdfd7fb..827b364c9f5d 100644
--- a/drivers/usb/misc/iowarrior.c
+++ b/drivers/usb/misc/iowarrior.c
@@ -71,10 +71,10 @@ static struct usb_driver iowarrior_driver;
/* Structure to hold all of our device specific stuff */
struct iowarrior {
+ struct kref kref;
struct mutex mutex; /* locks this structure */
struct usb_device *udev; /* save off the usb device pointer */
struct usb_interface *interface; /* the interface for this device */
- unsigned char minor; /* the starting minor number for this device */
struct usb_endpoint_descriptor *int_out_endpoint; /* endpoint for reading (needed for IOW56 only) */
struct usb_endpoint_descriptor *int_in_endpoint; /* endpoint for reading */
struct urb *int_in_urb; /* the urb for reading data */
@@ -244,9 +244,10 @@ static void iowarrior_write_callback(struct urb *urb)
/*
* iowarrior_delete
*/
-static inline void iowarrior_delete(struct iowarrior *dev)
+static inline void iowarrior_delete(struct kref *kref)
{
- dev_dbg(&dev->interface->dev, "minor %d\n", dev->minor);
+ struct iowarrior *dev = container_of(kref, struct iowarrior, kref);
+
kfree(dev->int_in_buffer);
usb_free_urb(dev->int_in_urb);
kfree(dev->read_queue);
@@ -297,9 +298,6 @@ static ssize_t iowarrior_read(struct file *file, char __user *buffer,
goto exit;
}
- dev_dbg(&dev->interface->dev, "minor %d, count = %zd\n",
- dev->minor, count);
-
/* read count must be packet size (+ time stamp) */
if ((count != dev->report_size)
&& (count != (dev->report_size + 1))) {
@@ -379,8 +377,6 @@ static ssize_t iowarrior_write(struct file *file,
retval = -ENODEV;
goto exit;
}
- dev_dbg(&dev->interface->dev, "minor %d, count = %zd\n",
- dev->minor, count);
/* if count is 0 we're already done */
if (count == 0) {
retval = 0;
@@ -523,9 +519,6 @@ static long iowarrior_ioctl(struct file *file, unsigned int cmd,
goto error_out;
}
- dev_dbg(&dev->interface->dev, "minor %d, cmd 0x%.4x, arg %ld\n",
- dev->minor, cmd, arg);
-
retval = 0;
switch (cmd) {
case IOW_WRITE:
@@ -650,6 +643,9 @@ static int iowarrior_open(struct inode *inode, struct file *file)
}
/* increment our usage count for the driver */
++dev->opened;
+
+ kref_get(&dev->kref);
+
/* save our object in the file's private structure */
file->private_data = dev;
retval = 0;
@@ -665,39 +661,25 @@ static int iowarrior_open(struct inode *inode, struct file *file)
static int iowarrior_release(struct inode *inode, struct file *file)
{
struct iowarrior *dev;
- int retval = 0;
dev = file->private_data;
if (!dev)
return -ENODEV;
- dev_dbg(&dev->interface->dev, "minor %d\n", dev->minor);
-
/* lock our device */
mutex_lock(&dev->mutex);
+ dev->opened = 0; /* we're closing now */
- if (dev->opened <= 0) {
- retval = -ENODEV; /* close called more than once */
- mutex_unlock(&dev->mutex);
- } else {
- dev->opened = 0; /* we're closing now */
- retval = 0;
- if (dev->present) {
- /*
- The device is still connected so we only shutdown
- pending read-/write-ops.
- */
- usb_kill_urb(dev->int_in_urb);
- wake_up_interruptible(&dev->read_wait);
- wake_up_interruptible(&dev->write_wait);
- mutex_unlock(&dev->mutex);
- } else {
- /* The device was unplugged, cleanup resources */
- mutex_unlock(&dev->mutex);
- iowarrior_delete(dev);
- }
+ if (dev->present) {
+ usb_kill_urb(dev->int_in_urb);
+ wake_up_interruptible(&dev->read_wait);
+ wake_up_interruptible(&dev->write_wait);
}
- return retval;
+ mutex_unlock(&dev->mutex);
+
+ kref_put(&dev->kref, iowarrior_delete);
+
+ return 0;
}
static __poll_t iowarrior_poll(struct file *file, poll_table * wait)
@@ -775,12 +757,14 @@ static int iowarrior_probe(struct usb_interface *interface,
struct usb_host_interface *iface_desc;
int retval = -ENOMEM;
int res;
+ int minor;
/* allocate memory for our device state and initialize it */
dev = kzalloc(sizeof(struct iowarrior), GFP_KERNEL);
if (!dev)
return retval;
+ kref_init(&dev->kref);
mutex_init(&dev->mutex);
atomic_set(&dev->intr_idx, 0);
@@ -890,16 +874,17 @@ static int iowarrior_probe(struct usb_interface *interface,
goto error;
}
- dev->minor = interface->minor;
+ minor = interface->minor;
/* let the user know what node this device is now attached to */
dev_info(&interface->dev, "IOWarrior product=0x%x, serial=%s interface=%d "
"now attached to iowarrior%d\n", dev->product_id, dev->chip_serial,
- iface_desc->desc.bInterfaceNumber, dev->minor - IOWARRIOR_MINOR_BASE);
+ iface_desc->desc.bInterfaceNumber, minor - IOWARRIOR_MINOR_BASE);
return retval;
error:
- iowarrior_delete(dev);
+ kref_put(&dev->kref, iowarrior_delete);
+
return retval;
}
@@ -923,19 +908,14 @@ static void iowarrior_disconnect(struct usb_interface *interface)
usb_kill_anchored_urbs(&dev->submitted);
if (dev->opened) {
- /* There is a process that holds a filedescriptor to the device ,
- so we only shutdown read-/write-ops going on.
- Deleting the device is postponed until close() was called.
- */
usb_kill_urb(dev->int_in_urb);
wake_up_interruptible(&dev->read_wait);
wake_up_interruptible(&dev->write_wait);
- mutex_unlock(&dev->mutex);
- } else {
- /* no process is using the device, cleanup now */
- mutex_unlock(&dev->mutex);
- iowarrior_delete(dev);
}
+
+ mutex_unlock(&dev->mutex);
+
+ kref_put(&dev->kref, iowarrior_delete);
}
/* usb specific object needed to register this driver with the usb subsystem */
diff --git a/drivers/vdpa/octeon_ep/octep_vdpa_main.c b/drivers/vdpa/octeon_ep/octep_vdpa_main.c
index 70c932df42fa..e65565ea5b83 100644
--- a/drivers/vdpa/octeon_ep/octep_vdpa_main.c
+++ b/drivers/vdpa/octeon_ep/octep_vdpa_main.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright (C) 2024 Marvell. */
+#include <linux/bitfield.h>
#include <linux/interrupt.h>
#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/module.h>
@@ -678,6 +679,8 @@ static int octep_sriov_enable(struct pci_dev *pdev, int num_vfs)
bool done = false;
int index = 0;
int ret, i;
+ u8 rpvf;
+ u64 val;
ret = pci_enable_sriov(pdev, num_vfs);
if (ret)
@@ -697,9 +700,11 @@ static int octep_sriov_enable(struct pci_dev *pdev, int num_vfs)
}
}
+ val = readq(addr + OCTEP_EPF_RINFO(0));
+ rpvf = FIELD_GET(GENMASK_ULL(35, 32), val);
if (done) {
for (i = 0; i < pf->enabled_vfs; i++)
- writeq(OCTEP_DEV_READY_SIGNATURE, addr + OCTEP_PF_MBOX_DATA(i));
+ writeq(OCTEP_DEV_READY_SIGNATURE, addr + OCTEP_PF_MBOX_DATA(i * rpvf));
}
return num_vfs;
diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c
index 7f569ce8fc7b..6f0b2b04958a 100644
--- a/drivers/vdpa/vdpa_user/vduse_dev.c
+++ b/drivers/vdpa/vdpa_user/vduse_dev.c
@@ -193,6 +193,12 @@ static void vduse_enqueue_msg(struct list_head *head,
list_add_tail(&msg->list, head);
}
+static void vduse_enqueue_msg_head(struct list_head *head,
+ struct vduse_dev_msg *msg)
+{
+ list_add(&msg->list, head);
+}
+
static void vduse_dev_broken(struct vduse_dev *dev)
{
struct vduse_dev_msg *msg, *tmp;
@@ -324,6 +330,7 @@ static ssize_t vduse_dev_read_iter(struct kiocb *iocb, struct iov_iter *to)
struct file *file = iocb->ki_filp;
struct vduse_dev *dev = file->private_data;
struct vduse_dev_msg *msg;
+ struct vduse_dev_request req;
int size = sizeof(struct vduse_dev_request);
ssize_t ret;
@@ -335,12 +342,11 @@ static ssize_t vduse_dev_read_iter(struct kiocb *iocb, struct iov_iter *to)
msg = vduse_dequeue_msg(&dev->send_list);
if (msg)
break;
+ spin_unlock(&dev->msg_lock);
- ret = -EAGAIN;
if (file->f_flags & O_NONBLOCK)
- goto unlock;
+ return -EAGAIN;
- spin_unlock(&dev->msg_lock);
ret = wait_event_interruptible_exclusive(dev->waitq,
!list_empty(&dev->send_list));
if (ret)
@@ -348,17 +354,34 @@ static ssize_t vduse_dev_read_iter(struct kiocb *iocb, struct iov_iter *to)
spin_lock(&dev->msg_lock);
}
+
+ memcpy(&req, &msg->req, sizeof(req));
+ /*
+ * We must ensure vduse_msg is on send_list or recv_list before unlock
+ * dev->msg_lock. Because vduse_dev_msg_sync() may be timeout when we
+ * copy data to userspace, and will call list_del() for this msg.
+ */
+ vduse_enqueue_msg(&dev->recv_list, msg);
spin_unlock(&dev->msg_lock);
- ret = copy_to_iter(&msg->req, size, to);
- spin_lock(&dev->msg_lock);
+
+ ret = copy_to_iter(&req, size, to);
if (ret != size) {
+ /*
+ * Roll back: move msg back to send_list if still pending.
+ *
+ * NOTE:
+ * vduse_find_msg() must use req.request_id instead of `msg`.
+ * A malicious userspace may reply to this request, and wake up
+ * the caller, after which `msg` will have already been freed.
+ * And here vduse_find_msg() will return NULL then do nothing.
+ */
+ spin_lock(&dev->msg_lock);
+ msg = vduse_find_msg(&dev->recv_list, req.request_id);
+ if (msg)
+ vduse_enqueue_msg_head(&dev->send_list, msg);
+ spin_unlock(&dev->msg_lock);
ret = -EFAULT;
- vduse_enqueue_msg(&dev->send_list, msg);
- goto unlock;
}
- vduse_enqueue_msg(&dev->recv_list, msg);
-unlock:
- spin_unlock(&dev->msg_lock);
return ret;
}
@@ -1429,26 +1452,18 @@ static int vduse_dev_release(struct inode *inode, struct file *file)
return 0;
}
-static struct vduse_dev *vduse_dev_get_from_minor(int minor)
+static int vduse_dev_open(struct inode *inode, struct file *file)
{
+ int ret = -EBUSY;
struct vduse_dev *dev;
mutex_lock(&vduse_lock);
- dev = idr_find(&vduse_idr, minor);
- mutex_unlock(&vduse_lock);
-
- return dev;
-}
-
-static int vduse_dev_open(struct inode *inode, struct file *file)
-{
- int ret;
- struct vduse_dev *dev = vduse_dev_get_from_minor(iminor(inode));
-
- if (!dev)
+ dev = idr_find(&vduse_idr, iminor(inode));
+ if (!dev) {
+ mutex_unlock(&vduse_lock);
return -ENODEV;
+ }
- ret = -EBUSY;
mutex_lock(&dev->lock);
if (dev->connected)
goto unlock;
@@ -1458,6 +1473,7 @@ static int vduse_dev_open(struct inode *inode, struct file *file)
file->private_data = dev;
unlock:
mutex_unlock(&dev->lock);
+ mutex_unlock(&vduse_lock);
return ret;
}
diff --git a/drivers/vfio/pci/mlx5/cmd.h b/drivers/vfio/pci/mlx5/cmd.h
index df421dc6de04..5e8ebb685768 100644
--- a/drivers/vfio/pci/mlx5/cmd.h
+++ b/drivers/vfio/pci/mlx5/cmd.h
@@ -157,25 +157,28 @@ struct mlx5_vhca_qp {
struct mlx5_vhca_page_tracker {
u32 id;
u32 pdn;
- u8 is_err:1;
- u8 object_changed:1;
+ /* Flags modified at runtime - dedicated storage unit */
+ u8 is_err;
+ u8 object_changed;
+ int status;
struct mlx5_uars_page *uar;
struct mlx5_vhca_cq cq;
struct mlx5_vhca_qp *host_qp;
struct mlx5_vhca_qp *fw_qp;
struct mlx5_nb nb;
- int status;
};
struct mlx5vf_pci_core_device {
struct vfio_pci_core_device core_device;
int vf_id;
u16 vhca_id;
+ /* Flags only modified on setup/release - bitfield ok */
u8 migrate_cap:1;
- u8 deferred_reset:1;
- u8 mdev_detach:1;
- u8 log_active:1;
u8 chunk_mode:1;
+ /* Flags modified at runtime - dedicated storage unit */
+ u8 mdev_detach;
+ u8 log_active;
+ u8 deferred_reset;
struct completion tracker_comp;
/* protect migration state */
struct mutex state_mutex;
diff --git a/drivers/vfio/pci/qat/main.c b/drivers/vfio/pci/qat/main.c
index c78cb6de9390..eaf996c85eb2 100644
--- a/drivers/vfio/pci/qat/main.c
+++ b/drivers/vfio/pci/qat/main.c
@@ -303,14 +303,18 @@ static ssize_t qat_vf_resume_write(struct file *filp, const char __user *buf,
return -ESPIPE;
offs = &filp->f_pos;
- if (*offs < 0 ||
- check_add_overflow(len, *offs, &end))
- return -EOVERFLOW;
+ mutex_lock(&migf->lock);
- if (end > mig_dev->state_size)
- return -ENOMEM;
+ if (*offs < 0 || check_add_overflow(len, *offs, &end)) {
+ done = -EOVERFLOW;
+ goto out_unlock;
+ }
+
+ if (end > mig_dev->state_size) {
+ done = -ENOMEM;
+ goto out_unlock;
+ }
- mutex_lock(&migf->lock);
if (migf->disabled) {
done = -ENODEV;
goto out_unlock;
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 3ad1a6f4ef96..c4c013960547 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -388,13 +388,20 @@ static void vhost_zerocopy_signal_used(struct vhost_net *net,
static void vhost_zerocopy_complete(struct sk_buff *skb,
struct ubuf_info *ubuf_base, bool success)
{
- struct ubuf_info_msgzc *ubuf = uarg_to_msgzc(ubuf_base);
- struct vhost_net_ubuf_ref *ubufs = ubuf->ctx;
- struct vhost_virtqueue *vq = ubufs->vq;
+ struct ubuf_info_msgzc *ubuf;
+ struct vhost_net_ubuf_ref *ubufs;
+ struct vhost_virtqueue *vq;
int cnt;
- rcu_read_lock_bh();
+ /* Only the final cloned skb reference completes the vhost descriptor. */
+ if (!refcount_dec_and_test(&ubuf_base->refcnt))
+ return;
+
+ ubuf = uarg_to_msgzc(ubuf_base);
+ ubufs = ubuf->ctx;
+ vq = ubufs->vq;
+ rcu_read_lock_bh();
/* set len to mark this desc buffers done DMA */
vq->heads[ubuf->desc].len = success ?
VHOST_DMA_DONE_LEN : VHOST_DMA_FAILED_LEN;
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 0ba5237980bb..2464c9de6771 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -1480,16 +1480,32 @@ static int vhost_vdpa_release(struct inode *inode, struct file *filep)
}
#ifdef CONFIG_MMU
-static vm_fault_t vhost_vdpa_fault(struct vm_fault *vmf)
+static int
+vhost_vdpa_get_vq_notification(struct vhost_vdpa *v, unsigned long index,
+ struct vdpa_notification_area *notify)
{
- struct vhost_vdpa *v = vmf->vma->vm_file->private_data;
struct vdpa_device *vdpa = v->vdpa;
const struct vdpa_config_ops *ops = vdpa->config;
+
+ if (index > 65535 || index >= v->nvqs)
+ return -EINVAL;
+
+ index = array_index_nospec(index, v->nvqs);
+
+ *notify = ops->get_vq_notification(vdpa, index);
+
+ return 0;
+}
+
+static vm_fault_t vhost_vdpa_fault(struct vm_fault *vmf)
+{
+ struct vhost_vdpa *v = vmf->vma->vm_file->private_data;
struct vdpa_notification_area notify;
struct vm_area_struct *vma = vmf->vma;
- u16 index = vma->vm_pgoff;
+ unsigned long index = vma->vm_pgoff;
- notify = ops->get_vq_notification(vdpa, index);
+ if (vhost_vdpa_get_vq_notification(v, index, ¬ify))
+ return VM_FAULT_SIGBUS;
return vmf_insert_pfn(vma, vmf->address & PAGE_MASK, PFN_DOWN(notify.addr));
}
@@ -1512,8 +1528,6 @@ static int vhost_vdpa_mmap(struct file *file, struct vm_area_struct *vma)
return -EINVAL;
if (vma->vm_flags & VM_READ)
return -EINVAL;
- if (index > 65535)
- return -EINVAL;
if (!ops->get_vq_notification)
return -ENOTSUPP;
@@ -1521,7 +1535,8 @@ static int vhost_vdpa_mmap(struct file *file, struct vm_area_struct *vma)
* support the doorbell which sits on the page boundary and
* does not share the page with other registers.
*/
- notify = ops->get_vq_notification(vdpa, index);
+ if (vhost_vdpa_get_vq_notification(v, index, ¬ify))
+ return -EINVAL;
if (notify.addr & (PAGE_SIZE - 1))
return -EINVAL;
if (vma->vm_end - vma->vm_start != notify.size)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 276dded52212..68027fd993e3 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -1520,6 +1520,7 @@ static void vhost_dev_unlock_vqs(struct vhost_dev *d)
static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq)
{
__virtio16 idx;
+ u16 avail_idx;
int r;
r = vhost_get_avail(vq, idx, &vq->avail->idx);
@@ -1530,17 +1531,19 @@ static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq)
}
/* Check it isn't doing very strange thing with available indexes */
- vq->avail_idx = vhost16_to_cpu(vq, idx);
- if (unlikely((u16)(vq->avail_idx - vq->last_avail_idx) > vq->num)) {
+ avail_idx = vhost16_to_cpu(vq, idx);
+ if (unlikely((u16)(avail_idx - vq->last_avail_idx) > vq->num)) {
vq_err(vq, "Invalid available index change from %u to %u",
- vq->last_avail_idx, vq->avail_idx);
+ vq->last_avail_idx, avail_idx);
return -EINVAL;
}
/* We're done if there is nothing new */
- if (vq->avail_idx == vq->last_avail_idx)
+ if (avail_idx == vq->avail_idx)
return 0;
+ vq->avail_idx = avail_idx;
+
/*
* We updated vq->avail_idx so we need a memory barrier between
* the index read above and the caller reading avail ring entries.
diff --git a/drivers/video/fbdev/aty/radeon_base.c b/drivers/video/fbdev/aty/radeon_base.c
index 36bfb6deb8ab..eb5dc8bbed4d 100644
--- a/drivers/video/fbdev/aty/radeon_base.c
+++ b/drivers/video/fbdev/aty/radeon_base.c
@@ -2476,6 +2476,7 @@ static int radeonfb_pci_register(struct pci_dev *pdev,
return 0;
err_unmap_fb:
iounmap(rinfo->fb_base);
+ fb_destroy_modelist(&info->modelist);
err_unmap_rom:
kfree(rinfo->mon1_EDID);
kfree(rinfo->mon2_EDID);
diff --git a/drivers/video/fbdev/broadsheetfb.c b/drivers/video/fbdev/broadsheetfb.c
index c8ba098a8c42..582f1ee4c9b6 100644
--- a/drivers/video/fbdev/broadsheetfb.c
+++ b/drivers/video/fbdev/broadsheetfb.c
@@ -1072,12 +1072,14 @@ static int broadsheetfb_probe(struct platform_device *dev)
info->flags = FBINFO_VIRTFB;
info->fbdefio = &broadsheetfb_defio;
- fb_deferred_io_init(info);
+ retval = fb_deferred_io_init(info);
+ if (retval)
+ goto err_vfree;
retval = fb_alloc_cmap(&info->cmap, 16, 0);
if (retval < 0) {
dev_err(&dev->dev, "Failed to allocate colormap\n");
- goto err_vfree;
+ goto err_fbdefio;
}
/* set cmap */
@@ -1121,6 +1123,8 @@ static int broadsheetfb_probe(struct platform_device *dev)
board->cleanup(par);
err_cmap:
fb_dealloc_cmap(&info->cmap);
+err_fbdefio:
+ fb_deferred_io_cleanup(info);
err_vfree:
vfree(videomemory);
err_fb_rel:
diff --git a/drivers/video/fbdev/carminefb.c b/drivers/video/fbdev/carminefb.c
index e56065cdba97..49b6f1f977b6 100644
--- a/drivers/video/fbdev/carminefb.c
+++ b/drivers/video/fbdev/carminefb.c
@@ -589,6 +589,7 @@ static int alloc_carmine_fb(void __iomem *regs, void __iomem *smem_base,
return 0;
err_dealloc_cmap:
+ fb_destroy_modelist(&info->modelist);
fb_dealloc_cmap(&info->cmap);
err_free_fb:
framebuffer_release(info);
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index d6ed7ae65462..fb60d2aff5b0 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -1201,6 +1201,7 @@ static void fbcon_deinit(struct vc_data *vc)
int idx;
fbcon_free_font(p);
+ p->mode = NULL;
idx = con2fb_map[vc->vc_num];
if (idx == -1)
@@ -1371,14 +1372,14 @@ static void fbcon_set_disp(struct fb_info *info, struct fb_var_screeninfo *var,
p = &fb_display[unit];
- if (var_to_display(p, var, info))
- return;
-
vc = vc_cons[unit].d;
if (!vc)
return;
+ if (var_to_display(p, var, info))
+ return;
+
default_mode = vc->vc_display_fg;
svc = *default_mode;
t = &fb_display[svc->vc_num];
diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c
index 20517448487e..208f31360943 100644
--- a/drivers/video/fbdev/efifb.c
+++ b/drivers/video/fbdev/efifb.c
@@ -377,6 +377,7 @@ static int efifb_probe(struct platform_device *dev)
if (fb_get_options("efifb", &option))
return -ENODEV;
efifb_setup(si, option);
+ kfree(option);
/* We don't get linelength from UGA Draw Protocol, only from
* EFI Graphics Protocol. So if it's not in DMI, and it's not
diff --git a/drivers/video/fbdev/hecubafb.c b/drivers/video/fbdev/hecubafb.c
index 3547d58a29cf..dd2af980f3d8 100644
--- a/drivers/video/fbdev/hecubafb.c
+++ b/drivers/video/fbdev/hecubafb.c
@@ -192,7 +192,9 @@ static int hecubafb_probe(struct platform_device *dev)
info->flags = FBINFO_VIRTFB;
info->fbdefio = &hecubafb_defio;
- fb_deferred_io_init(info);
+ retval = fb_deferred_io_init(info);
+ if (retval)
+ goto err_fbdefio;
retval = register_framebuffer(info);
if (retval < 0)
@@ -209,6 +211,8 @@ static int hecubafb_probe(struct platform_device *dev)
return 0;
err_fbreg:
+ fb_deferred_io_cleanup(info);
+err_fbdefio:
framebuffer_release(info);
err_fballoc:
vfree(videomemory);
diff --git a/drivers/video/fbdev/i740fb.c b/drivers/video/fbdev/i740fb.c
index 9b74dae71472..c14a19382769 100644
--- a/drivers/video/fbdev/i740fb.c
+++ b/drivers/video/fbdev/i740fb.c
@@ -1152,6 +1152,7 @@ static int i740fb_probe(struct pci_dev *dev, const struct pci_device_id *ent)
fb_dealloc_cmap(&info->cmap);
err_alloc_cmap:
err_find_mode:
+ fb_destroy_modelist(&info->modelist);
if (par->ddc_registered)
i2c_del_adapter(&par->ddc_adapter);
pci_iounmap(dev, par->regs);
diff --git a/drivers/video/fbdev/metronomefb.c b/drivers/video/fbdev/metronomefb.c
index 6f0942c6e5f1..83c614963a0a 100644
--- a/drivers/video/fbdev/metronomefb.c
+++ b/drivers/video/fbdev/metronomefb.c
@@ -645,12 +645,14 @@ static int metronomefb_probe(struct platform_device *dev)
info->flags = FBINFO_VIRTFB;
info->fbdefio = &metronomefb_defio;
- fb_deferred_io_init(info);
+ retval = fb_deferred_io_init(info);
+ if (retval)
+ goto err_free_irq;
retval = fb_alloc_cmap(&info->cmap, 8, 0);
if (retval < 0) {
dev_err(&dev->dev, "Failed to allocate colormap\n");
- goto err_free_irq;
+ goto err_fbdefio;
}
/* set cmap */
@@ -673,6 +675,8 @@ static int metronomefb_probe(struct platform_device *dev)
err_cmap:
fb_dealloc_cmap(&info->cmap);
+err_fbdefio:
+ fb_deferred_io_cleanup(info);
err_free_irq:
board->cleanup(par);
err_csum_table:
diff --git a/drivers/video/fbdev/nvidia/nvidia.c b/drivers/video/fbdev/nvidia/nvidia.c
index 8900f181f195..5f61d953e4c7 100644
--- a/drivers/video/fbdev/nvidia/nvidia.c
+++ b/drivers/video/fbdev/nvidia/nvidia.c
@@ -1420,6 +1420,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
err_out_iounmap_fb:
iounmap(info->screen_base);
+ fb_destroy_modelist(&info->modelist);
err_out_free_base1:
fb_destroy_modedb(info->monspecs.modedb);
nvidia_delete_i2c_busses(par);
diff --git a/drivers/video/fbdev/s3fb.c b/drivers/video/fbdev/s3fb.c
index ff84106ecf1c..0f1f9eb0f7d9 100644
--- a/drivers/video/fbdev/s3fb.c
+++ b/drivers/video/fbdev/s3fb.c
@@ -1382,6 +1382,7 @@ static int s3_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
err_alloc_cmap:
err_find_mode:
#ifdef CONFIG_FB_S3_DDC
+ fb_destroy_modelist(&info->modelist);
if (par->ddc_registered)
i2c_del_adapter(&par->ddc_adapter);
if (par->mmio)
diff --git a/drivers/video/fbdev/sm501fb.c b/drivers/video/fbdev/sm501fb.c
index 2eb27ebf822e..571530bacbd9 100644
--- a/drivers/video/fbdev/sm501fb.c
+++ b/drivers/video/fbdev/sm501fb.c
@@ -95,6 +95,7 @@ struct sm501fb_info {
void __iomem *fbmem; /* remapped framebuffer */
size_t fbmem_len; /* length of remapped region */
u8 *edid_data;
+ char *fb_mode;
};
/* per-framebuffer private data */
@@ -1792,12 +1793,11 @@ static int sm501fb_init_fb(struct fb_info *fb, enum sm501_controller head,
fb->var.yres_virtual = fb->var.yres;
} else {
if (info->edid_data) {
- ret = fb_find_mode(&fb->var, fb, fb_mode,
+ ret = fb_find_mode(&fb->var, fb,
+ info->fb_mode ?: fb_mode,
fb->monspecs.modedb,
fb->monspecs.modedb_len,
&sm501_default_mode, default_bpp);
- /* edid_data is no longer needed, free it */
- kfree(info->edid_data);
} else {
ret = fb_find_mode(&fb->var, fb,
NULL, NULL, 0, NULL, 8);
@@ -1973,7 +1973,7 @@ static int sm501fb_probe(struct platform_device *pdev)
/* Get EDID */
cp = of_get_property(np, "mode", &len);
if (cp)
- strcpy(fb_mode, cp);
+ info->fb_mode = kstrdup(cp, GFP_KERNEL);
prop = of_get_property(np, "edid", &len);
if (prop && len == EDID_LENGTH) {
info->edid_data = kmemdup(prop, EDID_LENGTH,
@@ -2030,6 +2030,12 @@ static int sm501fb_probe(struct platform_device *pdev)
goto err_started_crt;
}
+ /* These aren't needed any more */
+ kfree(info->edid_data);
+ kfree(info->fb_mode);
+ info->edid_data = NULL;
+ info->fb_mode = NULL;
+
/* we registered, return ok */
return 0;
@@ -2047,6 +2053,8 @@ static int sm501fb_probe(struct platform_device *pdev)
framebuffer_release(info->fb[HEAD_CRT]);
err_alloc:
+ kfree(info->edid_data);
+ kfree(info->fb_mode);
kfree(info);
return ret;
diff --git a/drivers/video/fbdev/sm712.h b/drivers/video/fbdev/sm712.h
index c7ebf03b8d53..83fe25fc61f2 100644
--- a/drivers/video/fbdev/sm712.h
+++ b/drivers/video/fbdev/sm712.h
@@ -101,7 +101,7 @@ struct modeinit {
#define mmio_addr 0x00800000
#define seqw17() smtc_seqw(0x17, 0x30)
#define big_pixel_depth(p, d) {if (p == 24) {p = 32; d = 32; } }
-#define big_swap(p) ((p & 0xff00ff00 >> 8) | (p & 0x00ff00ff << 8))
+#define big_swap(p) (((p & 0xff00ff00) >> 8) | ((p & 0x00ff00ff) << 8))
#else
#define pal_rgb(r, g, b, val) val
#define big_addr 0
diff --git a/drivers/video/fbdev/tdfxfb.c b/drivers/video/fbdev/tdfxfb.c
index 531fb8478e20..c5095358209e 100644
--- a/drivers/video/fbdev/tdfxfb.c
+++ b/drivers/video/fbdev/tdfxfb.c
@@ -1550,6 +1550,7 @@ static int tdfxfb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
out_err_iobase:
#ifdef CONFIG_FB_3DFX_I2C
+ fb_destroy_modelist(&info->modelist);
tdfxfb_delete_i2c_busses(default_par);
#endif
arch_phys_wc_del(default_par->wc_cookie);
diff --git a/drivers/video/fbdev/tridentfb.c b/drivers/video/fbdev/tridentfb.c
index 516cf2a18757..2f7338de0912 100644
--- a/drivers/video/fbdev/tridentfb.c
+++ b/drivers/video/fbdev/tridentfb.c
@@ -1706,6 +1706,7 @@ static int trident_pci_probe(struct pci_dev *dev,
return 0;
out_unmap2:
+ fb_destroy_modelist(&info->modelist);
if (default_par->ddc_registered)
i2c_del_adapter(&default_par->ddc_adapter);
kfree(info->pixmap.addr);
diff --git a/drivers/video/fbdev/uvesafb.c b/drivers/video/fbdev/uvesafb.c
index 5d52fd00806e..b806da0538bc 100644
--- a/drivers/video/fbdev/uvesafb.c
+++ b/drivers/video/fbdev/uvesafb.c
@@ -1697,14 +1697,14 @@ static int uvesafb_probe(struct platform_device *dev)
i = uvesafb_vbe_init_mode(info);
if (i < 0) {
err = -EINVAL;
- goto out;
+ goto out_mode;
} else {
mode = &par->vbe_modes[i];
}
if (fb_alloc_cmap(&info->cmap, 256, 0) < 0) {
err = -ENXIO;
- goto out;
+ goto out_mode;
}
uvesafb_init_info(info, mode);
diff --git a/drivers/video/fbdev/vesafb.c b/drivers/video/fbdev/vesafb.c
index a81df8865143..634d4c9898d1 100644
--- a/drivers/video/fbdev/vesafb.c
+++ b/drivers/video/fbdev/vesafb.c
@@ -267,6 +267,7 @@ static int vesafb_probe(struct platform_device *dev)
/* ignore error return of fb_get_options */
fb_get_options("vesafb", &option);
vesafb_setup(option);
+ kfree(option);
if (si->orig_video_isVGA != VIDEO_TYPE_VLFB)
return -ENODEV;
diff --git a/drivers/watchdog/sama5d4_wdt.c b/drivers/watchdog/sama5d4_wdt.c
index 13e72918338a..704b786cc2ec 100644
--- a/drivers/watchdog/sama5d4_wdt.c
+++ b/drivers/watchdog/sama5d4_wdt.c
@@ -30,6 +30,7 @@ struct sama5d4_wdt {
void __iomem *reg_base;
u32 mr;
u32 ir;
+ u32 wddis_mask;
unsigned long last_ping;
bool need_irq;
bool sam9x60_support;
@@ -48,7 +49,10 @@ MODULE_PARM_DESC(nowayout,
"Watchdog cannot be stopped once started (default="
__MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
-#define wdt_enabled (!(wdt->mr & AT91_WDT_WDDIS))
+static inline bool wdt_enabled(struct sama5d4_wdt *wdt)
+{
+ return !(wdt->mr & wdt->wddis_mask);
+}
#define wdt_read(wdt, field) \
readl_relaxed((wdt)->reg_base + (field))
@@ -81,12 +85,9 @@ static int sama5d4_wdt_start(struct watchdog_device *wdd)
{
struct sama5d4_wdt *wdt = watchdog_get_drvdata(wdd);
- if (wdt->sam9x60_support) {
+ if (wdt->sam9x60_support)
writel_relaxed(wdt->ir, wdt->reg_base + AT91_SAM9X60_IER);
- wdt->mr &= ~AT91_SAM9X60_WDDIS;
- } else {
- wdt->mr &= ~AT91_WDT_WDDIS;
- }
+ wdt->mr &= ~wdt->wddis_mask;
wdt_write(wdt, AT91_WDT_MR, wdt->mr);
return 0;
@@ -96,12 +97,9 @@ static int sama5d4_wdt_stop(struct watchdog_device *wdd)
{
struct sama5d4_wdt *wdt = watchdog_get_drvdata(wdd);
- if (wdt->sam9x60_support) {
+ if (wdt->sam9x60_support)
writel_relaxed(wdt->ir, wdt->reg_base + AT91_SAM9X60_IDR);
- wdt->mr |= AT91_SAM9X60_WDDIS;
- } else {
- wdt->mr |= AT91_WDT_WDDIS;
- }
+ wdt->mr |= wdt->wddis_mask;
wdt_write(wdt, AT91_WDT_MR, wdt->mr);
return 0;
@@ -117,7 +115,7 @@ static int sama5d4_wdt_ping(struct watchdog_device *wdd)
}
static int sama5d4_wdt_set_timeout(struct watchdog_device *wdd,
- unsigned int timeout)
+ unsigned int timeout)
{
struct sama5d4_wdt *wdt = watchdog_get_drvdata(wdd);
u32 value = WDT_SEC2TICKS(timeout);
@@ -140,8 +138,8 @@ static int sama5d4_wdt_set_timeout(struct watchdog_device *wdd,
* If the watchdog is enabled, then the timeout can be updated. Else,
* wait that the user enables it.
*/
- if (wdt_enabled)
- wdt_write(wdt, AT91_WDT_MR, wdt->mr & ~AT91_WDT_WDDIS);
+ if (wdt_enabled(wdt))
+ wdt_write(wdt, AT91_WDT_MR, wdt->mr & ~wdt->wddis_mask);
wdd->timeout = timeout;
@@ -184,10 +182,7 @@ static int of_sama5d4_wdt_init(struct device_node *np, struct sama5d4_wdt *wdt)
{
const char *tmp;
- if (wdt->sam9x60_support)
- wdt->mr = AT91_SAM9X60_WDDIS;
- else
- wdt->mr = AT91_WDT_WDDIS;
+ wdt->mr = wdt->wddis_mask;
if (!of_property_read_string(np, "atmel,watchdog-type", &tmp) &&
!strcmp(tmp, "software"))
@@ -213,15 +208,11 @@ static int sama5d4_wdt_init(struct sama5d4_wdt *wdt)
* If the watchdog is already running, we can safely update it.
* Else, we have to disable it properly.
*/
- if (!wdt_enabled) {
+ if (!wdt_enabled(wdt)) {
reg = wdt_read(wdt, AT91_WDT_MR);
- if (wdt->sam9x60_support && (!(reg & AT91_SAM9X60_WDDIS)))
- wdt_write_nosleep(wdt, AT91_WDT_MR,
- reg | AT91_SAM9X60_WDDIS);
- else if (!wdt->sam9x60_support &&
- (!(reg & AT91_WDT_WDDIS)))
+ if (!(reg & wdt->wddis_mask))
wdt_write_nosleep(wdt, AT91_WDT_MR,
- reg | AT91_WDT_WDDIS);
+ reg | wdt->wddis_mask);
}
if (wdt->sam9x60_support) {
@@ -273,6 +264,9 @@ static int sama5d4_wdt_probe(struct platform_device *pdev)
of_device_is_compatible(dev->of_node, "microchip,sama7g5-wdt"))
wdt->sam9x60_support = true;
+ wdt->wddis_mask = wdt->sam9x60_support ? AT91_SAM9X60_WDDIS
+ : AT91_WDT_WDDIS;
+
watchdog_set_drvdata(wdd, wdt);
regs = devm_platform_ioremap_resource(pdev, 0);
@@ -306,8 +300,8 @@ static int sama5d4_wdt_probe(struct platform_device *pdev)
watchdog_init_timeout(wdd, wdt_timeout, dev);
reg = wdt_read(wdt, AT91_WDT_MR);
- if (!(reg & AT91_WDT_WDDIS)) {
- wdt->mr &= ~AT91_WDT_WDDIS;
+ if (!(reg & wdt->wddis_mask)) {
+ wdt->mr &= ~wdt->wddis_mask;
set_bit(WDOG_HW_RUNNING, &wdd->status);
}
diff --git a/drivers/watchdog/sp5100_tco.c b/drivers/watchdog/sp5100_tco.c
index 2bd3dc25cb03..7e99c3b1f367 100644
--- a/drivers/watchdog/sp5100_tco.c
+++ b/drivers/watchdog/sp5100_tco.c
@@ -92,7 +92,8 @@ static enum tco_reg_layout tco_reg_layout(struct pci_dev *dev)
dev->device == PCI_DEVICE_ID_ATI_SBX00_SMBUS &&
dev->revision < 0x40) {
return sp5100;
- } else if (dev->vendor == PCI_VENDOR_ID_AMD &&
+ } else if ((dev->vendor == PCI_VENDOR_ID_AMD ||
+ dev->vendor == PCI_VENDOR_ID_HYGON) &&
sp5100_tco_pci->device == PCI_DEVICE_ID_AMD_KERNCZ_SMBUS &&
sp5100_tco_pci->revision >= AMD_ZEN_SMBUS_PCI_REV) {
return efch_mmio;
diff --git a/drivers/watchdog/sprd_wdt.c b/drivers/watchdog/sprd_wdt.c
index 4e689b6ff141..aacf04616fef 100644
--- a/drivers/watchdog/sprd_wdt.c
+++ b/drivers/watchdog/sprd_wdt.c
@@ -320,10 +320,9 @@ static int sprd_wdt_probe(struct platform_device *pdev)
watchdog_init_timeout(&wdt->wdd, 0, dev);
ret = devm_watchdog_register_device(dev, &wdt->wdd);
- if (ret) {
- sprd_wdt_disable(wdt);
+ if (ret)
return ret;
- }
+
platform_set_drvdata(pdev, wdt);
return 0;
diff --git a/drivers/watchdog/watchdog_core.c b/drivers/watchdog/watchdog_core.c
index aff2c3912ead..fd7161a5da33 100644
--- a/drivers/watchdog/watchdog_core.c
+++ b/drivers/watchdog/watchdog_core.c
@@ -382,6 +382,9 @@ static void __watchdog_unregister_device(struct watchdog_device *wdd)
if (test_bit(WDOG_STOP_ON_REBOOT, &wdd->status))
unregister_reboot_notifier(&wdd->reboot_nb);
+ if (test_bit(WDOG_NO_PING_ON_SUSPEND, &wdd->status))
+ unregister_pm_notifier(&wdd->pm_nb);
+
watchdog_dev_unregister(wdd);
ida_free(&watchdog_ida, wdd->id);
}
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index 1f2160765618..16af955c5ea5 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -678,11 +678,15 @@ static long gntdev_ioctl_map_grant_ref(struct gntdev_priv *priv,
mutex_lock(&priv->lock);
gntdev_add_map(priv, map);
op.index = map->index << PAGE_SHIFT;
- mutex_unlock(&priv->lock);
- if (copy_to_user(u, &op, sizeof(op)) != 0)
+ if (copy_to_user(u, &op, sizeof(op)) != 0) {
+ list_del(&map->next);
+ mutex_unlock(&priv->lock);
+ gntdev_put_map(priv, map);
return -EFAULT;
+ }
+ mutex_unlock(&priv->lock);
return 0;
}
diff --git a/drivers/xen/xen-scsiback.c b/drivers/xen/xen-scsiback.c
index 7d5117e5efe0..82a757d7fe26 100644
--- a/drivers/xen/xen-scsiback.c
+++ b/drivers/xen/xen-scsiback.c
@@ -612,6 +612,25 @@ static void scsiback_disconnect(struct vscsibk_info *info)
xenbus_unmap_ring_vfree(info->dev, info->ring.sring);
}
+/*
+ * Send the error response for a request that did not reach the target core
+ * and return its tag. Free the tag before the response drops the v2p
+ * reference that keeps the session alive, and snapshot what the response
+ * needs since returning the tag can let the slot be reused.
+ */
+static void scsiback_resp_and_free(struct vscsibk_pend *pending_req,
+ int32_t result)
+{
+ struct vscsibk_info *info = pending_req->info;
+ struct v2p_entry *v2p = pending_req->v2p;
+ struct se_session *se_sess = v2p->tpg->tpg_nexus->tvn_se_sess;
+ u16 rqid = pending_req->rqid;
+
+ target_free_tag(se_sess, &pending_req->se_cmd);
+ scsiback_send_response(info, NULL, result, 0, rqid);
+ kref_put(&v2p->kref, scsiback_free_translation_entry);
+}
+
static void scsiback_device_action(struct vscsibk_pend *pending_req,
enum tcm_tmreq_table act, int tag)
{
@@ -640,7 +659,7 @@ static void scsiback_device_action(struct vscsibk_pend *pending_req,
return;
err:
- scsiback_do_resp_with_sense(NULL, err, 0, pending_req);
+ scsiback_resp_and_free(pending_req, err);
}
/*
@@ -793,9 +812,8 @@ static int scsiback_do_cmd_fn(struct vscsibk_info *info,
case VSCSIIF_ACT_SCSI_CDB:
if (scsiback_gnttab_data_map(&ring_req, pending_req)) {
scsiback_fast_flush_area(pending_req);
- scsiback_do_resp_with_sense(NULL,
- DID_ERROR << 16, 0, pending_req);
- transport_generic_free_cmd(&pending_req->se_cmd, 0);
+ scsiback_resp_and_free(pending_req,
+ DID_ERROR << 16);
} else {
scsiback_cmd_exec(pending_req);
}
@@ -809,9 +827,7 @@ static int scsiback_do_cmd_fn(struct vscsibk_info *info,
break;
default:
pr_err_ratelimited("invalid request\n");
- scsiback_do_resp_with_sense(NULL, DID_ERROR << 16, 0,
- pending_req);
- transport_generic_free_cmd(&pending_req->se_cmd, 0);
+ scsiback_resp_and_free(pending_req, DID_ERROR << 16);
break;
}
diff --git a/fs/9p/vfs_addr.c b/fs/9p/vfs_addr.c
index db78c06ba0cc..d6107ac50e88 100644
--- a/fs/9p/vfs_addr.c
+++ b/fs/9p/vfs_addr.c
@@ -83,7 +83,8 @@ static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
if (!err)
subreq->transferred += total;
- netfs_read_subreq_terminated(subreq, err, false);
+ subreq->error = err;
+ netfs_read_subreq_terminated(subreq, false);
}
/**
diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c
index 1723a37d1846..90fe3ea09595 100644
--- a/fs/9p/vfs_inode.c
+++ b/fs/9p/vfs_inode.c
@@ -488,10 +488,19 @@ static int v9fs_at_to_dotl_flags(int flags)
* - ext4 (with dir_nlink feature enabled) sets nlink to 1 if a dir has more
* than EXT4_LINK_MAX (65000) links.
*
+ * In cacheless mode the server is the source of truth for nlink and the
+ * inode is going away immediately, so locally adjusting i_nlink buys
+ * nothing and races with concurrent metadata fetches that may already
+ * have observed the post-unlink value (nlink == 0).
+ *
* @inode: inode whose nlink is being dropped
*/
static void v9fs_dec_count(struct inode *inode)
{
+ struct v9fs_session_info *v9ses = v9fs_inode2v9ses(inode);
+
+ if (!(v9ses->cache & (CACHE_META | CACHE_LOOSE)))
+ return;
if (!S_ISDIR(inode->i_mode) || inode->i_nlink > 2)
drop_nlink(inode);
}
diff --git a/fs/afs/callback.c b/fs/afs/callback.c
index 99b2c8172021..4ddc2d666dfb 100644
--- a/fs/afs/callback.c
+++ b/fs/afs/callback.c
@@ -140,7 +140,6 @@ static struct afs_volume *afs_lookup_volume_rcu(struct afs_cell *cell,
break;
if (!need_seqretry(&cell->volume_lock, seq))
break;
- seq |= 1; /* Want a lock next time */
}
done_seqretry(&cell->volume_lock, seq);
@@ -221,7 +220,11 @@ static void afs_break_some_callbacks(struct afs_server *server,
rcu_read_lock();
volume = afs_lookup_volume_rcu(server->cell, vid);
- if (cbb->fid.vnode == 0 && cbb->fid.unique == 0) {
+ if (!volume) {
+ /* Ignore breaks on unknown volumes. */
+ rcu_read_unlock();
+ *_count = 0;
+ } else if (cbb->fid.vnode == 0 && cbb->fid.unique == 0) {
afs_break_volume_callback(server, volume);
*_count -= 1;
if (*_count)
diff --git a/fs/afs/cell.c b/fs/afs/cell.c
index caa09875f520..ce74438b9fb8 100644
--- a/fs/afs/cell.c
+++ b/fs/afs/cell.c
@@ -64,7 +64,8 @@ static struct afs_cell *afs_find_cell_locked(struct afs_net *net,
return ERR_PTR(-ENAMETOOLONG);
if (!name) {
- cell = net->ws_cell;
+ cell = rcu_dereference_protected(net->ws_cell,
+ lockdep_is_held(&net->cells_lock));
if (!cell)
return ERR_PTR(-EDESTADDRREQ);
goto found;
@@ -146,18 +147,20 @@ static struct afs_cell *afs_alloc_cell(struct afs_net *net,
return ERR_PTR(-ENOMEM);
}
- cell->name = kmalloc(namelen + 1, GFP_KERNEL);
+ cell->name = kmalloc(1 + namelen + 1, GFP_KERNEL);
if (!cell->name) {
kfree(cell);
return ERR_PTR(-ENOMEM);
}
- cell->net = net;
+ cell->name[0] = '.';
+ cell->name++;
cell->name_len = namelen;
for (i = 0; i < namelen; i++)
cell->name[i] = tolower(name[i]);
cell->name[i] = 0;
+ cell->net = net;
refcount_set(&cell->ref, 1);
atomic_set(&cell->active, 0);
INIT_WORK(&cell->manager, afs_manage_cell_work);
@@ -200,8 +203,14 @@ static struct afs_cell *afs_alloc_cell(struct afs_net *net,
cell->dns_source = vllist->source;
cell->dns_status = vllist->status;
smp_store_release(&cell->dns_lookup_count, 1); /* vs source/status */
+ ret = idr_alloc_cyclic(&net->cells_dyn_ino, cell,
+ 2, INT_MAX / 2, GFP_KERNEL);
+ if (ret < 0)
+ goto error;
atomic_inc(&net->cells_outstanding);
+ cell->dynroot_ino = ret;
cell->debug_id = atomic_inc_return(&cell_debug_id);
+
trace_afs_cell(cell->debug_id, 1, 0, afs_cell_trace_alloc);
_leave(" = %p", cell);
@@ -211,7 +220,7 @@ static struct afs_cell *afs_alloc_cell(struct afs_net *net,
if (ret == -EINVAL)
printk(KERN_ERR "kAFS: bad VL server IP address\n");
error:
- kfree(cell->name);
+ kfree(cell->name - 1);
kfree(cell);
_leave(" = %d", ret);
return ERR_PTR(ret);
@@ -365,6 +374,14 @@ int afs_cell_init(struct afs_net *net, const char *rootcell)
len = cp - rootcell;
}
+ if (len == 0 || !rootcell[0] || rootcell[0] == '.' || rootcell[len - 1] == '.')
+ return -EINVAL;
+ if (memchr(rootcell, '/', len))
+ return -EINVAL;
+ cp = strstr(rootcell, "..");
+ if (cp && cp < rootcell + len)
+ return -EINVAL;
+
/* allocate a cell record for the root cell */
new_root = afs_lookup_cell(net, rootcell, len, vllist, false);
if (IS_ERR(new_root)) {
@@ -378,8 +395,8 @@ int afs_cell_init(struct afs_net *net, const char *rootcell)
/* install the new cell */
down_write(&net->cells_lock);
afs_see_cell(new_root, afs_cell_trace_see_ws);
- old_root = net->ws_cell;
- net->ws_cell = new_root;
+ old_root = rcu_replace_pointer(net->ws_cell, new_root,
+ lockdep_is_held(&net->cells_lock));
up_write(&net->cells_lock);
afs_unuse_cell(net, old_root, afs_cell_trace_unuse_ws);
@@ -472,6 +489,8 @@ static int afs_update_cell(struct afs_cell *cell)
rcu_assign_pointer(cell->vl_servers, vllist);
cell->dns_source = vllist->source;
old = p;
+ } else {
+ old = vllist;
}
write_unlock(&cell->vl_servers_lock);
afs_put_vlserverlist(cell->net, old);
@@ -502,7 +521,8 @@ static void afs_cell_destroy(struct rcu_head *rcu)
afs_put_vlserverlist(net, rcu_access_pointer(cell->vl_servers));
afs_unuse_cell(net, cell->alias_of, afs_cell_trace_unuse_alias);
key_put(cell->anonymous_key);
- kfree(cell->name);
+ idr_remove(&net->cells_dyn_ino, cell->dynroot_ino);
+ kfree(cell->name - 1);
kfree(cell);
afs_dec_cells_outstanding(net);
@@ -695,7 +715,6 @@ static int afs_activate_cell(struct afs_net *net, struct afs_cell *cell)
if (cell->proc_link.next)
cell->proc_link.next->pprev = &cell->proc_link.next;
- afs_dynroot_mkdir(net, cell);
mutex_unlock(&net->proc_cells_lock);
return 0;
}
@@ -710,8 +729,8 @@ static void afs_deactivate_cell(struct afs_net *net, struct afs_cell *cell)
afs_proc_cell_remove(cell);
mutex_lock(&net->proc_cells_lock);
- hlist_del_rcu(&cell->proc_link);
- afs_dynroot_rmdir(net, cell);
+ if (!hlist_unhashed(&cell->proc_link))
+ hlist_del_rcu(&cell->proc_link);
mutex_unlock(&net->proc_cells_lock);
_leave("");
@@ -934,8 +953,8 @@ void afs_cell_purge(struct afs_net *net)
_enter("");
down_write(&net->cells_lock);
- ws = net->ws_cell;
- net->ws_cell = NULL;
+ ws = rcu_replace_pointer(net->ws_cell, NULL,
+ lockdep_is_held(&net->cells_lock));
up_write(&net->cells_lock);
afs_unuse_cell(net, ws, afs_cell_trace_unuse_ws);
diff --git a/fs/afs/cmservice.c b/fs/afs/cmservice.c
index 99a3f20bc786..0638cab00c61 100644
--- a/fs/afs/cmservice.c
+++ b/fs/afs/cmservice.c
@@ -391,7 +391,6 @@ static int afs_deliver_cb_init_call_back_state3(struct afs_call *call)
ret = afs_extract_data(call, false);
switch (ret) {
case 0: break;
- case -EAGAIN: return 0;
default: return ret;
}
@@ -504,7 +503,6 @@ static int afs_deliver_cb_probe_uuid(struct afs_call *call)
ret = afs_extract_data(call, false);
switch (ret) {
case 0: break;
- case -EAGAIN: return 0;
default: return ret;
}
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 50edd1cae28a..037c89e05f4c 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -1031,9 +1031,8 @@ static struct dentry *afs_lookup(struct inode *dir, struct dentry *dentry,
inode = afs_do_lookup(dir, dentry, key);
key_put(key);
if (inode == ERR_PTR(-ENOENT))
- inode = afs_try_auto_mntpt(dentry, dir);
-
- if (!IS_ERR_OR_NULL(inode))
+ inode = NULL;
+ else if (!IS_ERR_OR_NULL(inode))
fid = AFS_FS_I(inode)->fid;
_debug("splice %p", dentry->d_inode);
diff --git a/fs/afs/dynroot.c b/fs/afs/dynroot.c
index c4d2711e20ad..17dffa3b27e5 100644
--- a/fs/afs/dynroot.c
+++ b/fs/afs/dynroot.c
@@ -10,16 +10,19 @@
#include <linux/dns_resolver.h>
#include "internal.h"
-static atomic_t afs_autocell_ino;
+#define AFS_MIN_DYNROOT_CELL_INO 4 /* Allow for ., .., @cell, .@cell */
+#define AFS_MAX_DYNROOT_CELL_INO ((unsigned int)INT_MAX)
+
+static struct dentry *afs_lookup_atcell(struct inode *dir, struct dentry *dentry, ino_t ino);
/*
* iget5() comparator for inode created by autocell operations
- *
- * These pseudo inodes don't match anything.
*/
static int afs_iget5_pseudo_test(struct inode *inode, void *opaque)
{
- return 0;
+ struct afs_fid *fid = opaque;
+
+ return inode->i_ino == fid->vnode;
}
/*
@@ -39,28 +42,16 @@ static int afs_iget5_pseudo_set(struct inode *inode, void *opaque)
}
/*
- * Create an inode for a dynamic root directory or an autocell dynamic
- * automount dir.
+ * Create an inode for an autocell dynamic automount dir.
*/
-struct inode *afs_iget_pseudo_dir(struct super_block *sb, bool root)
+static struct inode *afs_iget_pseudo_dir(struct super_block *sb, ino_t ino)
{
- struct afs_super_info *as = AFS_FS_S(sb);
struct afs_vnode *vnode;
struct inode *inode;
- struct afs_fid fid = {};
+ struct afs_fid fid = { .vnode = ino, .unique = 1, };
_enter("");
- if (as->volume)
- fid.vid = as->volume->vid;
- if (root) {
- fid.vnode = 1;
- fid.unique = 1;
- } else {
- fid.vnode = atomic_inc_return(&afs_autocell_ino);
- fid.unique = 0;
- }
-
inode = iget5_locked(sb, fid.vnode,
afs_iget5_pseudo_test, afs_iget5_pseudo_set, &fid);
if (!inode) {
@@ -73,162 +64,73 @@ struct inode *afs_iget_pseudo_dir(struct super_block *sb, bool root)
vnode = AFS_FS_I(inode);
- /* there shouldn't be an existing inode */
- BUG_ON(!(inode->i_state & I_NEW));
-
- netfs_inode_init(&vnode->netfs, NULL, false);
- inode->i_size = 0;
- inode->i_mode = S_IFDIR | S_IRUGO | S_IXUGO;
- if (root) {
- inode->i_op = &afs_dynroot_inode_operations;
- inode->i_fop = &simple_dir_operations;
- } else {
- inode->i_op = &afs_autocell_inode_operations;
- }
- set_nlink(inode, 2);
- inode->i_uid = GLOBAL_ROOT_UID;
- inode->i_gid = GLOBAL_ROOT_GID;
- simple_inode_init_ts(inode);
- inode->i_blocks = 0;
- inode->i_generation = 0;
-
- set_bit(AFS_VNODE_PSEUDODIR, &vnode->flags);
- if (!root) {
+ if (inode->i_state & I_NEW) {
+ netfs_inode_init(&vnode->netfs, NULL, false);
+ simple_inode_init_ts(inode);
+ set_nlink(inode, 2);
+ inode->i_size = 0;
+ inode->i_mode = S_IFDIR | 0555;
+ inode->i_op = &afs_autocell_inode_operations;
+ inode->i_uid = GLOBAL_ROOT_UID;
+ inode->i_gid = GLOBAL_ROOT_GID;
+ inode->i_blocks = 0;
+ inode->i_generation = 0;
+ inode->i_flags |= S_AUTOMOUNT | S_NOATIME;
+
+ set_bit(AFS_VNODE_PSEUDODIR, &vnode->flags);
set_bit(AFS_VNODE_MOUNTPOINT, &vnode->flags);
- inode->i_flags |= S_AUTOMOUNT;
- }
- inode->i_flags |= S_NOATIME;
- unlock_new_inode(inode);
+ unlock_new_inode(inode);
+ }
_leave(" = %p", inode);
return inode;
}
/*
- * Probe to see if a cell may exist. This prevents positive dentries from
- * being created unnecessarily.
+ * Try to automount the mountpoint with pseudo directory, if the autocell
+ * option is set.
*/
-static int afs_probe_cell_name(struct dentry *dentry)
+static struct dentry *afs_dynroot_lookup_cell(struct inode *dir, struct dentry *dentry,
+ unsigned int flags)
{
- struct afs_cell *cell;
+ struct afs_cell *cell = NULL;
struct afs_net *net = afs_d2net(dentry);
+ struct inode *inode = NULL;
const char *name = dentry->d_name.name;
size_t len = dentry->d_name.len;
- char *result = NULL;
- int ret;
+ bool dotted = false;
+ int ret = -ENOENT;
/* Names prefixed with a dot are R/W mounts. */
if (name[0] == '.') {
- if (len == 1)
- return -EINVAL;
name++;
len--;
+ dotted = true;
}
- cell = afs_find_cell(net, name, len, afs_cell_trace_use_probe);
- if (!IS_ERR(cell)) {
- afs_unuse_cell(net, cell, afs_cell_trace_unuse_probe);
- return 0;
- }
-
- ret = dns_query(net->net, "afsdb", name, len, "srv=1",
- &result, NULL, false);
- if (ret == -ENODATA || ret == -ENOKEY || ret == 0)
- ret = -ENOENT;
- if (ret > 0 && ret >= sizeof(struct dns_server_list_v1_header)) {
- struct dns_server_list_v1_header *v1 = (void *)result;
-
- if (v1->hdr.zero == 0 &&
- v1->hdr.content == DNS_PAYLOAD_IS_SERVER_LIST &&
- v1->hdr.version == 1 &&
- (v1->status != DNS_LOOKUP_GOOD &&
- v1->status != DNS_LOOKUP_GOOD_WITH_BAD))
- return -ENOENT;
-
+ cell = afs_lookup_cell(net, name, len, NULL, false);
+ if (IS_ERR(cell)) {
+ ret = PTR_ERR(cell);
+ goto out_no_cell;
}
- kfree(result);
- return ret;
-}
-
-/*
- * Try to auto mount the mountpoint with pseudo directory, if the autocell
- * operation is setted.
- */
-struct inode *afs_try_auto_mntpt(struct dentry *dentry, struct inode *dir)
-{
- struct afs_vnode *vnode = AFS_FS_I(dir);
- struct inode *inode;
- int ret = -ENOENT;
-
- _enter("%p{%pd}, {%llx:%llu}",
- dentry, dentry, vnode->fid.vid, vnode->fid.vnode);
-
- if (!test_bit(AFS_VNODE_AUTOCELL, &vnode->flags))
- goto out;
-
- ret = afs_probe_cell_name(dentry);
- if (ret < 0)
- goto out;
-
- inode = afs_iget_pseudo_dir(dir->i_sb, false);
+ inode = afs_iget_pseudo_dir(dir->i_sb, cell->dynroot_ino * 2 + dotted);
if (IS_ERR(inode)) {
ret = PTR_ERR(inode);
goto out;
}
- _leave("= %p", inode);
- return inode;
+ dentry->d_fsdata = cell;
+ return d_splice_alias(inode, dentry);
out:
- _leave("= %d", ret);
+ afs_unuse_cell(cell->net, cell, afs_cell_trace_unuse_lookup_dynroot);
+out_no_cell:
+ if (!inode)
+ return d_splice_alias(inode, dentry);
return ret == -ENOENT ? NULL : ERR_PTR(ret);
}
-/*
- * Look up @cell in a dynroot directory. This is a substitution for the
- * local cell name for the net namespace.
- */
-static struct dentry *afs_lookup_atcell(struct dentry *dentry)
-{
- struct afs_cell *cell;
- struct afs_net *net = afs_d2net(dentry);
- struct dentry *ret;
- char *name;
- int len;
-
- if (!net->ws_cell)
- return ERR_PTR(-ENOENT);
-
- ret = ERR_PTR(-ENOMEM);
- name = kmalloc(AFS_MAXCELLNAME + 1, GFP_KERNEL);
- if (!name)
- goto out_p;
-
- down_read(&net->cells_lock);
- cell = net->ws_cell;
- if (cell) {
- len = cell->name_len;
- memcpy(name, cell->name, len + 1);
- }
- up_read(&net->cells_lock);
-
- ret = ERR_PTR(-ENOENT);
- if (!cell)
- goto out_n;
-
- ret = lookup_one_len(name, dentry->d_parent, len);
-
- /* We don't want to d_add() the @cell dentry here as we don't want to
- * the cached dentry to hide changes to the local cell name.
- */
-
-out_n:
- kfree(name);
-out_p:
- return ret;
-}
-
/*
* Look up an entry in a dynroot directory.
*/
@@ -237,8 +139,6 @@ static struct dentry *afs_dynroot_lookup(struct inode *dir, struct dentry *dentr
{
_enter("%pd", dentry);
- ASSERTCMP(d_inode(dentry), ==, NULL);
-
if (flags & LOOKUP_CREATE)
return ERR_PTR(-EOPNOTSUPP);
@@ -249,141 +149,256 @@ static struct dentry *afs_dynroot_lookup(struct inode *dir, struct dentry *dentr
if (dentry->d_name.len == 5 &&
memcmp(dentry->d_name.name, "@cell", 5) == 0)
- return afs_lookup_atcell(dentry);
+ return afs_lookup_atcell(dir, dentry, 2);
+
+ if (dentry->d_name.len == 6 &&
+ memcmp(dentry->d_name.name, ".@cell", 6) == 0)
+ return afs_lookup_atcell(dir, dentry, 3);
- return d_splice_alias(afs_try_auto_mntpt(dentry, dir), dentry);
+ return afs_dynroot_lookup_cell(dir, dentry, flags);
}
const struct inode_operations afs_dynroot_inode_operations = {
.lookup = afs_dynroot_lookup,
};
+static void afs_dynroot_d_release(struct dentry *dentry)
+{
+ struct afs_cell *cell = dentry->d_fsdata;
+
+ afs_unuse_cell(cell->net, cell, afs_cell_trace_unuse_dynroot_mntpt);
+}
+
+/*
+ * Keep @cell symlink dentries around, but only keep cell autodirs when they're
+ * being used.
+ */
+static int afs_dynroot_delete_dentry(const struct dentry *dentry)
+{
+ const struct qstr *name = &dentry->d_name;
+
+ if (name->len == 5 && memcmp(name->name, "@cell", 5) == 0)
+ return 0;
+ if (name->len == 6 && memcmp(name->name, ".@cell", 6) == 0)
+ return 0;
+ return 1;
+}
+
const struct dentry_operations afs_dynroot_dentry_operations = {
- .d_delete = always_delete_dentry,
- .d_release = afs_d_release,
+ .d_delete = afs_dynroot_delete_dentry,
+ .d_release = afs_dynroot_d_release,
.d_automount = afs_d_automount,
};
+static void afs_atcell_delayed_put_cell(void *arg)
+{
+ struct afs_cell *cell = arg;
+
+ afs_put_cell(cell, afs_cell_trace_put_atcell);
+}
+
/*
- * Create a manually added cell mount directory.
- * - The caller must hold net->proc_cells_lock
+ * Read @cell or .@cell symlinks.
*/
-int afs_dynroot_mkdir(struct afs_net *net, struct afs_cell *cell)
+static const char *afs_atcell_get_link(struct dentry *dentry, struct inode *inode,
+ struct delayed_call *done)
{
- struct super_block *sb = net->dynroot_sb;
- struct dentry *root, *subdir;
- int ret;
+ struct afs_vnode *vnode = AFS_FS_I(inode);
+ struct afs_cell *cell;
+ struct afs_net *net = afs_i2net(inode);
+ const char *name;
+ bool dotted = vnode->fid.vnode == 3;
- if (!sb || atomic_read(&sb->s_active) == 0)
- return 0;
+ if (!rcu_access_pointer(net->ws_cell))
+ return ERR_PTR(-ENOENT);
- /* Let the ->lookup op do the creation */
- root = sb->s_root;
- inode_lock(root->d_inode);
- subdir = lookup_one_len(cell->name, root, cell->name_len);
- if (IS_ERR(subdir)) {
- ret = PTR_ERR(subdir);
- goto unlock;
+ if (!dentry) {
+ /* We're in RCU-pathwalk. */
+ cell = rcu_dereference(net->ws_cell);
+ if (dotted)
+ name = cell->name - 1;
+ else
+ name = cell->name;
+ /* Shouldn't need to set a delayed call. */
+ return name;
}
- /* Note that we're retaining an extra ref on the dentry */
- subdir->d_fsdata = (void *)1UL;
- ret = 0;
-unlock:
- inode_unlock(root->d_inode);
- return ret;
+ down_read(&net->cells_lock);
+
+ cell = rcu_dereference_protected(net->ws_cell, lockdep_is_held(&net->cells_lock));
+ if (dotted)
+ name = cell->name - 1;
+ else
+ name = cell->name;
+ afs_get_cell(cell, afs_cell_trace_get_atcell);
+ set_delayed_call(done, afs_atcell_delayed_put_cell, cell);
+
+ up_read(&net->cells_lock);
+ return name;
}
+static const struct inode_operations afs_atcell_inode_operations = {
+ .get_link = afs_atcell_get_link,
+};
+
/*
- * Remove a manually added cell mount directory.
- * - The caller must hold net->proc_cells_lock
+ * Create an inode for the @cell or .@cell symlinks.
*/
-void afs_dynroot_rmdir(struct afs_net *net, struct afs_cell *cell)
+static struct dentry *afs_lookup_atcell(struct inode *dir, struct dentry *dentry, ino_t ino)
{
- struct super_block *sb = net->dynroot_sb;
- struct dentry *root, *subdir;
+ struct afs_vnode *vnode;
+ struct inode *inode;
+ struct afs_fid fid = { .vnode = ino, .unique = 1, };
- if (!sb || atomic_read(&sb->s_active) == 0)
- return;
+ inode = iget5_locked(dir->i_sb, fid.vnode,
+ afs_iget5_pseudo_test, afs_iget5_pseudo_set, &fid);
+ if (!inode)
+ return ERR_PTR(-ENOMEM);
- root = sb->s_root;
- inode_lock(root->d_inode);
+ vnode = AFS_FS_I(inode);
- /* Don't want to trigger a lookup call, which will re-add the cell */
- subdir = try_lookup_one_len(cell->name, root, cell->name_len);
- if (IS_ERR_OR_NULL(subdir)) {
- _debug("lookup %ld", PTR_ERR(subdir));
- goto no_dentry;
+ if (inode->i_state & I_NEW) {
+ netfs_inode_init(&vnode->netfs, NULL, false);
+ simple_inode_init_ts(inode);
+ set_nlink(inode, 1);
+ inode->i_size = 0;
+ inode->i_mode = S_IFLNK | 0555;
+ inode->i_op = &afs_atcell_inode_operations;
+ inode->i_uid = GLOBAL_ROOT_UID;
+ inode->i_gid = GLOBAL_ROOT_GID;
+ inode->i_blocks = 0;
+ inode->i_generation = 0;
+ inode->i_flags |= S_NOATIME;
+
+ unlock_new_inode(inode);
}
+ return d_splice_alias(inode, dentry);
+}
- _debug("rmdir %pd %u", subdir, d_count(subdir));
+/*
+ * Transcribe the cell database into readdir content under the RCU read lock.
+ * Each cell produces two entries, one prefixed with a dot and one not.
+ */
+static int afs_dynroot_readdir_cells(struct afs_net *net, struct dir_context *ctx)
+{
+ const struct afs_cell *cell;
+ loff_t newpos;
+
+ _enter("%llu", ctx->pos);
+
+ for (;;) {
+ unsigned int ix = ctx->pos >> 1;
+
+ cell = idr_get_next(&net->cells_dyn_ino, &ix);
+ if (!cell)
+ return 0;
+ if (READ_ONCE(cell->state) == AFS_CELL_FAILED ||
+ READ_ONCE(cell->state) == AFS_CELL_REMOVED) {
+ ctx->pos += 2;
+ ctx->pos &= ~1;
+ continue;
+ }
+
+ newpos = ix << 1;
+ if (newpos > ctx->pos)
+ ctx->pos = newpos;
- if (subdir->d_fsdata) {
- _debug("unpin %u", d_count(subdir));
- subdir->d_fsdata = NULL;
- dput(subdir);
+ _debug("pos %llu -> cell %u", ctx->pos, cell->dynroot_ino);
+
+ if ((ctx->pos & 1) == 0) {
+ if (!dir_emit(ctx, cell->name, cell->name_len,
+ cell->dynroot_ino, DT_DIR))
+ return 0;
+ ctx->pos++;
+ }
+ if ((ctx->pos & 1) == 1) {
+ if (!dir_emit(ctx, cell->name - 1, cell->name_len + 1,
+ cell->dynroot_ino + 1, DT_DIR))
+ return 0;
+ ctx->pos++;
+ }
}
- dput(subdir);
-no_dentry:
- inode_unlock(root->d_inode);
- _leave("");
+ return 0;
}
/*
- * Populate a newly created dynamic root with cell names.
+ * Read the AFS dynamic root directory. This produces a list of cellnames,
+ * dotted and undotted, along with @cell and .@cell links if configured.
*/
-int afs_dynroot_populate(struct super_block *sb)
+static int afs_dynroot_readdir(struct file *file, struct dir_context *ctx)
{
- struct afs_cell *cell;
- struct afs_net *net = afs_sb2net(sb);
- int ret;
+ struct afs_net *net = afs_d2net(file->f_path.dentry);
+ int ret = 0;
- mutex_lock(&net->proc_cells_lock);
+ if (!dir_emit_dots(file, ctx))
+ return 0;
- net->dynroot_sb = sb;
- hlist_for_each_entry(cell, &net->proc_cells, proc_link) {
- ret = afs_dynroot_mkdir(net, cell);
- if (ret < 0)
- goto error;
+ if (ctx->pos == 2) {
+ if (rcu_access_pointer(net->ws_cell) &&
+ !dir_emit(ctx, "@cell", 5, 2, DT_LNK))
+ return 0;
+ ctx->pos = 3;
+ }
+ if (ctx->pos == 3) {
+ if (rcu_access_pointer(net->ws_cell) &&
+ !dir_emit(ctx, ".@cell", 6, 3, DT_LNK))
+ return 0;
+ ctx->pos = 4;
}
- ret = 0;
-out:
- mutex_unlock(&net->proc_cells_lock);
+ if ((unsigned long long)ctx->pos <= AFS_MAX_DYNROOT_CELL_INO) {
+ down_read(&net->cells_lock);
+ ret = afs_dynroot_readdir_cells(net, ctx);
+ up_read(&net->cells_lock);
+ }
return ret;
-
-error:
- net->dynroot_sb = NULL;
- goto out;
}
+static const struct file_operations afs_dynroot_file_operations = {
+ .llseek = generic_file_llseek,
+ .read = generic_read_dir,
+ .iterate_shared = afs_dynroot_readdir,
+ .fsync = noop_fsync,
+};
+
/*
- * When a dynamic root that's in the process of being destroyed, depopulate it
- * of pinned directories.
+ * Create an inode for a dynamic root directory.
*/
-void afs_dynroot_depopulate(struct super_block *sb)
+struct inode *afs_dynroot_iget_root(struct super_block *sb)
{
- struct afs_net *net = afs_sb2net(sb);
- struct dentry *root = sb->s_root, *subdir;
-
- /* Prevent more subdirs from being created */
- mutex_lock(&net->proc_cells_lock);
- if (net->dynroot_sb == sb)
- net->dynroot_sb = NULL;
- mutex_unlock(&net->proc_cells_lock);
-
- if (root) {
- struct hlist_node *n;
- inode_lock(root->d_inode);
-
- /* Remove all the pins for dirs created for manually added cells */
- hlist_for_each_entry_safe(subdir, n, &root->d_children, d_sib) {
- if (subdir->d_fsdata) {
- subdir->d_fsdata = NULL;
- dput(subdir);
- }
- }
+ struct afs_super_info *as = AFS_FS_S(sb);
+ struct afs_vnode *vnode;
+ struct inode *inode;
+ struct afs_fid fid = { .vid = 0, .vnode = 1, .unique = 1,};
+
+ if (as->volume)
+ fid.vid = as->volume->vid;
- inode_unlock(root->d_inode);
+ inode = iget5_locked(sb, fid.vnode,
+ afs_iget5_pseudo_test, afs_iget5_pseudo_set, &fid);
+ if (!inode)
+ return ERR_PTR(-ENOMEM);
+
+ vnode = AFS_FS_I(inode);
+
+ /* there shouldn't be an existing inode */
+ if (inode->i_state & I_NEW) {
+ netfs_inode_init(&vnode->netfs, NULL, false);
+ simple_inode_init_ts(inode);
+ set_nlink(inode, 2);
+ inode->i_size = 0;
+ inode->i_mode = S_IFDIR | 0555;
+ inode->i_op = &afs_dynroot_inode_operations;
+ inode->i_fop = &afs_dynroot_file_operations;
+ inode->i_uid = GLOBAL_ROOT_UID;
+ inode->i_gid = GLOBAL_ROOT_GID;
+ inode->i_blocks = 0;
+ inode->i_generation = 0;
+ inode->i_flags |= S_NOATIME;
+
+ set_bit(AFS_VNODE_PSEUDODIR, &vnode->flags);
+ unlock_new_inode(inode);
}
+ _leave(" = %p", inode);
+ return inode;
}
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 6762eff97517..56248a078bca 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -246,7 +246,8 @@ static void afs_fetch_data_notify(struct afs_operation *op)
subreq->rreq->i_size = req->file_size;
if (req->pos + req->actual_len >= req->file_size)
__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
- netfs_read_subreq_terminated(subreq, error, false);
+ subreq->error = error;
+ netfs_read_subreq_terminated(subreq, false);
req->subreq = NULL;
} else if (req->done) {
req->done(req);
@@ -301,8 +302,10 @@ int afs_fetch_data(struct afs_vnode *vnode, struct afs_read *req)
op = afs_alloc_operation(req->key, vnode->volume);
if (IS_ERR(op)) {
- if (req->subreq)
- netfs_read_subreq_terminated(req->subreq, PTR_ERR(op), false);
+ if (req->subreq) {
+ req->subreq->error = PTR_ERR(op);
+ netfs_read_subreq_terminated(req->subreq, false);
+ }
return PTR_ERR(op);
}
@@ -320,8 +323,10 @@ static void afs_read_worker(struct work_struct *work)
struct afs_read *fsreq;
fsreq = afs_alloc_read(GFP_NOFS);
- if (!fsreq)
- return netfs_read_subreq_terminated(subreq, -ENOMEM, false);
+ if (!fsreq) {
+ subreq->error = -ENOMEM;
+ return netfs_read_subreq_terminated(subreq, false);
+ }
fsreq->subreq = subreq;
fsreq->pos = subreq->start + subreq->transferred;
diff --git a/fs/afs/fs_operation.c b/fs/afs/fs_operation.c
index 428721bbe4f6..384cd893d878 100644
--- a/fs/afs/fs_operation.c
+++ b/fs/afs/fs_operation.c
@@ -249,7 +249,7 @@ int afs_put_operation(struct afs_operation *op)
for (i = 0; i < op->nr_files - 2; i++)
if (op->more_files[i].put_vnode)
iput(&op->more_files[i].vnode->netfs.inode);
- kfree(op->more_files);
+ kvfree(op->more_files);
}
if (op->estate) {
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index a95e77670b49..c4f92aaa2527 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -546,7 +546,6 @@ struct inode *afs_root_iget(struct super_block *sb, struct key *key)
vnode = AFS_FS_I(inode);
vnode->cb_v_check = atomic_read(&as->volume->cb_v_break);
- afs_set_netfs_context(vnode);
op = afs_alloc_operation(key, as->volume);
if (IS_ERR(op)) {
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index d9760b2a8d8d..f9b90e596593 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -302,7 +302,8 @@ struct afs_net {
/* Cell database */
struct rb_root cells;
- struct afs_cell *ws_cell;
+ struct idr cells_dyn_ino; /* cell->dynroot_ino mapping */
+ struct afs_cell __rcu *ws_cell;
struct work_struct cells_manager;
struct timer_list cells_timer;
atomic_t cells_outstanding;
@@ -413,6 +414,7 @@ struct afs_cell {
enum dns_lookup_status dns_status:8; /* Latest status of data from lookup */
unsigned int dns_lookup_count; /* Counter of DNS lookups */
unsigned int debug_id;
+ unsigned int dynroot_ino; /* Inode numbers for dynroot (a pair) */
/* The volumes belonging to this cell */
struct rw_semaphore vs_lock; /* Lock for server->volumes */
@@ -714,7 +716,6 @@ struct afs_vnode {
#define AFS_VNODE_ZAP_DATA 3 /* set if vnode's data should be invalidated */
#define AFS_VNODE_DELETED 4 /* set if vnode deleted on server */
#define AFS_VNODE_MOUNTPOINT 5 /* set if vnode is a mountpoint symlink */
-#define AFS_VNODE_AUTOCELL 6 /* set if Vnode is an auto mount point */
#define AFS_VNODE_PSEUDODIR 7 /* set if Vnode is a pseudo directory */
#define AFS_VNODE_NEW_CONTENT 8 /* Set if file has new content (create/trunc-0) */
#define AFS_VNODE_SILLY_DELETED 9 /* Set if file has been silly-deleted */
@@ -1089,11 +1090,7 @@ extern int afs_silly_iput(struct dentry *, struct inode *);
extern const struct inode_operations afs_dynroot_inode_operations;
extern const struct dentry_operations afs_dynroot_dentry_operations;
-extern struct inode *afs_try_auto_mntpt(struct dentry *, struct inode *);
-extern int afs_dynroot_mkdir(struct afs_net *, struct afs_cell *);
-extern void afs_dynroot_rmdir(struct afs_net *, struct afs_cell *);
-extern int afs_dynroot_populate(struct super_block *);
-extern void afs_dynroot_depopulate(struct super_block *);
+struct inode *afs_dynroot_iget_root(struct super_block *sb);
/*
* file.c
@@ -1208,7 +1205,6 @@ extern const struct afs_operation_ops afs_fetch_status_operation;
extern void afs_vnode_commit_status(struct afs_operation *, struct afs_vnode_param *);
extern int afs_fetch_status(struct afs_vnode *, struct key *, bool, afs_access_t *);
extern int afs_ilookup5_test_by_fid(struct inode *, void *);
-extern struct inode *afs_iget_pseudo_dir(struct super_block *, bool);
extern struct inode *afs_iget(struct afs_operation *, struct afs_vnode_param *);
extern struct inode *afs_root_iget(struct super_block *, struct key *);
extern int afs_getattr(struct mnt_idmap *idmap, const struct path *,
diff --git a/fs/afs/main.c b/fs/afs/main.c
index a14f6013e316..5fffbc4d47af 100644
--- a/fs/afs/main.c
+++ b/fs/afs/main.c
@@ -76,6 +76,7 @@ static int __net_init afs_net_init(struct net *net_ns)
mutex_init(&net->socket_mutex);
net->cells = RB_ROOT;
+ idr_init(&net->cells_dyn_ino);
init_rwsem(&net->cells_lock);
INIT_WORK(&net->cells_manager, afs_manage_cells);
timer_setup(&net->cells_timer, afs_cells_timer, 0);
@@ -137,6 +138,7 @@ static int __net_init afs_net_init(struct net *net_ns)
error_proc:
afs_put_sysnames(net->sysnames);
error_sysnames:
+ idr_destroy(&net->cells_dyn_ino);
net->live = false;
return ret;
}
@@ -155,6 +157,7 @@ static void __net_exit afs_net_exit(struct net *net_ns)
afs_close_socket(net);
afs_proc_cleanup(net);
afs_put_sysnames(net->sysnames);
+ idr_destroy(&net->cells_dyn_ino);
kfree_rcu(rcu_access_pointer(net->address_prefs), rcu);
}
diff --git a/fs/afs/proc.c b/fs/afs/proc.c
index 15eab053af6d..12c88d8be3fe 100644
--- a/fs/afs/proc.c
+++ b/fs/afs/proc.c
@@ -206,7 +206,7 @@ static int afs_proc_rootcell_show(struct seq_file *m, void *v)
net = afs_seq2net_single(m);
down_read(&net->cells_lock);
- cell = net->ws_cell;
+ cell = rcu_dereference_protected(net->ws_cell, lockdep_is_held(&net->cells_lock));
if (cell)
seq_printf(m, "%s\n", cell->name);
up_read(&net->cells_lock);
@@ -240,7 +240,13 @@ static int afs_proc_rootcell_write(struct file *file, char *buf, size_t size)
/* determine command to perform */
_debug("rootcell=%s", buf);
- ret = afs_cell_init(net, buf);
+ ret = -EEXIST;
+ inode_lock(file_inode(file));
+ if (!rcu_access_pointer(net->ws_cell))
+ ret = afs_cell_init(net, buf);
+ else
+ printk("busy\n");
+ inode_unlock(file_inode(file));
out:
_leave(" = %d", ret);
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index a122c6366ce1..e9707624c565 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -110,8 +110,14 @@ void afs_close_socket(struct afs_net *net)
{
_enter("");
+ cancel_work_sync(&net->charge_preallocation_work);
+ /* Future work items should now see ->live is false. */
+
kernel_listen(net->socket, 0);
+
+ /* Make sure work items are no longer running. */
flush_workqueue(afs_async_calls);
+ cancel_work_sync(&net->charge_preallocation_work);
if (net->spare_incoming_call) {
afs_put_call(net->spare_incoming_call);
@@ -737,7 +743,7 @@ void afs_charge_preallocation(struct work_struct *work)
container_of(work, struct afs_net, charge_preallocation_work);
struct afs_call *call = net->spare_incoming_call;
- for (;;) {
+ while (READ_ONCE(net->live)) {
if (!call) {
call = afs_alloc_call(net, &afs_RXCMxxxx, GFP_KERNEL);
if (!call)
@@ -782,7 +788,8 @@ static void afs_rx_new_call(struct sock *sk, struct rxrpc_call *rxcall,
{
struct afs_net *net = afs_sock2net(sk);
- queue_work(afs_wq, &net->charge_preallocation_work);
+ if (net->live)
+ queue_work(afs_wq, &net->charge_preallocation_work);
}
/*
diff --git a/fs/afs/super.c b/fs/afs/super.c
index f3ba1c3e72f5..c6378d94fb64 100644
--- a/fs/afs/super.c
+++ b/fs/afs/super.c
@@ -194,8 +194,6 @@ static int afs_show_options(struct seq_file *m, struct dentry *root)
if (as->dyn_root)
seq_puts(m, ",dyn");
- if (test_bit(AFS_VNODE_AUTOCELL, &AFS_FS_I(d_inode(root))->flags))
- seq_puts(m, ",autocell");
switch (as->flock_mode) {
case afs_flock_mode_unset: break;
case afs_flock_mode_local: p = "local"; break;
@@ -468,7 +466,7 @@ static int afs_fill_super(struct super_block *sb, struct afs_fs_context *ctx)
/* allocate the root inode and dentry */
if (as->dyn_root) {
- inode = afs_iget_pseudo_dir(sb, true);
+ inode = afs_dynroot_iget_root(sb);
} else {
sprintf(sb->s_id, "%llu", as->volume->vid);
afs_activate_volume(as->volume);
@@ -478,9 +476,6 @@ static int afs_fill_super(struct super_block *sb, struct afs_fs_context *ctx)
if (IS_ERR(inode))
return PTR_ERR(inode);
- if (ctx->autocell || as->dyn_root)
- set_bit(AFS_VNODE_AUTOCELL, &AFS_FS_I(inode)->flags);
-
ret = -ENOMEM;
sb->s_root = d_make_root(inode);
if (!sb->s_root)
@@ -488,9 +483,6 @@ static int afs_fill_super(struct super_block *sb, struct afs_fs_context *ctx)
if (as->dyn_root) {
sb->s_d_op = &afs_dynroot_dentry_operations;
- ret = afs_dynroot_populate(sb);
- if (ret < 0)
- goto error;
} else {
sb->s_d_op = &afs_fs_dentry_operations;
rcu_assign_pointer(as->volume->sb, sb);
@@ -539,9 +531,6 @@ static void afs_kill_super(struct super_block *sb)
{
struct afs_super_info *as = AFS_FS_S(sb);
- if (as->dyn_root)
- afs_dynroot_depopulate(sb);
-
/* Clear the callback interests (which will do ilookup5) before
* deactivating the superblock.
*/
@@ -598,7 +587,8 @@ static int afs_get_tree(struct fs_context *fc)
}
fc->root = dget(sb->s_root);
- trace_afs_get_tree(as->cell, as->volume);
+ if (!ctx->dyn_root)
+ trace_afs_get_tree(as->cell, as->volume);
_leave(" = 0 [%p]", sb);
return 0;
diff --git a/fs/afs/vl_list.c b/fs/afs/vl_list.c
index 9b1c20daac53..429fde817e1d 100644
--- a/fs/afs/vl_list.c
+++ b/fs/afs/vl_list.c
@@ -93,7 +93,7 @@ static struct afs_addr_list *afs_extract_vl_addrs(struct afs_net *net,
{
struct afs_addr_list *alist;
const u8 *b = *_b;
- int ret = -EINVAL;
+ int ret;
alist = afs_alloc_addrlist(nr_addrs);
if (!alist)
@@ -111,6 +111,7 @@ static struct afs_addr_list *afs_extract_vl_addrs(struct afs_net *net,
case DNS_ADDRESS_IS_IPV4:
if (end - b < 4) {
_leave(" = -EINVAL [short inet]");
+ ret = -EINVAL;
goto error;
}
memcpy(x, b, 4);
@@ -123,6 +124,7 @@ static struct afs_addr_list *afs_extract_vl_addrs(struct afs_net *net,
case DNS_ADDRESS_IS_IPV6:
if (end - b < 16) {
_leave(" = -EINVAL [short inet6]");
+ ret = -EINVAL;
goto error;
}
memcpy(x, b, 16);
@@ -199,6 +201,8 @@ struct afs_vlserver_list *afs_extract_vlserver_list(struct afs_cell *cell,
b += sizeof(*hdr);
while (end - b >= sizeof(bs)) {
+ int nlen;
+
bs.name_len = afs_extract_le16(&b);
bs.priority = afs_extract_le16(&b);
bs.weight = afs_extract_le16(&b);
@@ -208,10 +212,12 @@ struct afs_vlserver_list *afs_extract_vlserver_list(struct afs_cell *cell,
bs.protocol = *b++;
bs.nr_addrs = *b++;
+ nlen = min3(bs.name_len, end - b, 255);
+
_debug("extract %u %u %u %u %u %u %*.*s",
bs.name_len, bs.priority, bs.weight,
bs.port, bs.protocol, bs.nr_addrs,
- bs.name_len, bs.name_len, b);
+ bs.name_len, nlen, b);
if (end - b < bs.name_len)
break;
diff --git a/fs/afs/volume.c b/fs/afs/volume.c
index af3a3f57c1b3..028dc954d0fe 100644
--- a/fs/afs/volume.c
+++ b/fs/afs/volume.c
@@ -39,7 +39,7 @@ static struct afs_volume *afs_insert_volume_into_cell(struct afs_cell *cell,
goto found;
}
- set_bit(AFS_VOLUME_RM_TREE, &volume->flags);
+ set_bit(AFS_VOLUME_RM_TREE, &p->flags);
rb_replace_node_rcu(&p->cell_node, &volume->cell_node, &cell->volumes);
}
}
diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h
index 89f582612fb9..877f9fc54d97 100644
--- a/fs/btrfs/btrfs_inode.h
+++ b/fs/btrfs/btrfs_inode.h
@@ -547,6 +547,8 @@ int btrfs_start_delalloc_roots(struct btrfs_fs_info *fs_info, long nr,
int btrfs_set_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
unsigned int extra_bits,
struct extent_state **cached_state);
+int btrfs_reset_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
+ unsigned int extra_bits, struct extent_state **cached_state);
struct btrfs_new_inode_args {
/* Input */
@@ -609,7 +611,6 @@ int btrfs_prealloc_file_range_trans(struct inode *inode,
loff_t actual_len, u64 *alloc_hint);
int btrfs_run_delalloc_range(struct btrfs_inode *inode, struct folio *locked_folio,
u64 start, u64 end, struct writeback_control *wbc);
-int btrfs_writepage_cow_fixup(struct folio *folio);
int btrfs_encoded_io_compression_from_extent(struct btrfs_fs_info *fs_info,
int compress_type);
int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index 65d883da86c6..7bd5e7ff97f2 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -316,7 +316,7 @@ static void btrfs_finish_compressed_write_work(struct work_struct *work)
struct compressed_bio *cb =
container_of(work, struct compressed_bio, write_end_work);
- btrfs_finish_ordered_extent(cb->bbio.ordered, NULL, cb->start, cb->len,
+ btrfs_finish_ordered_extent(cb->bbio.ordered, cb->start, cb->len,
cb->bbio.bio.bi_status == BLK_STS_OK);
if (cb->writeback)
diff --git a/fs/btrfs/direct-io.c b/fs/btrfs/direct-io.c
index 8ce58aa4189f..3e286f5fcdbd 100644
--- a/fs/btrfs/direct-io.c
+++ b/fs/btrfs/direct-io.c
@@ -13,10 +13,12 @@
struct btrfs_dio_data {
ssize_t submitted;
+ loff_t old_isize;
struct extent_changeset *data_reserved;
struct btrfs_ordered_extent *ordered;
bool data_space_reserved;
bool nocow_done;
+ bool updated_isize;
};
struct btrfs_dio_private {
@@ -227,6 +229,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
bool space_reserved = false;
u64 len = *lenp;
u64 prev_len;
+ loff_t old_isize;
int ret = 0;
/*
@@ -340,8 +343,14 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
* Need to update the i_size under the extent lock so buffered
* readers will get the updated i_size when we unlock.
*/
- if (start + len > i_size_read(inode))
+ old_isize = i_size_read(inode);
+ if (start + len > old_isize) {
+ if (!dio_data->updated_isize) {
+ dio_data->old_isize = old_isize;
+ dio_data->updated_isize = true;
+ }
i_size_write(inode, start + len);
+ }
out:
if (ret && space_reserved) {
btrfs_delalloc_release_extents(BTRFS_I(inode), len);
@@ -623,12 +632,55 @@ static int btrfs_dio_iomap_end(struct inode *inode, loff_t pos, loff_t length,
if (submitted < length) {
pos += submitted;
length -= submitted;
- if (write)
- btrfs_finish_ordered_extent(dio_data->ordered, NULL,
- pos, length, false);
- else
+ if (write) {
+ /*
+ * Got a short write and have updated the isize, need to
+ * revert the isize change.
+ *
+ * Normally we need to update isize with extent lock hold,
+ * but we're safe due to the following factors:
+ *
+ * - Only a single writer can be enlarging isize
+ * Enlarging isize will take the exclusive inode lock.
+ *
+ * - Buffered readers need to wait for the OE we're holding
+ * Buffered readers will lock extent and wait for OE
+ * of the folio range, and since page cache is invalidated
+ * the OE wait can not be skipped.
+ *
+ * So here we are safe to revert the isize before
+ * finishing the OE, and no reader of the remaining range
+ * can see the enlarged size.
+ *
+ * TODO: Extend the DIO_LOCKED lifespan for direct writes,
+ * and only enlarge isize after a successful write.
+ */
+ if (dio_data->updated_isize) {
+ u64 new_isize;
+
+ if (submitted == 0)
+ new_isize = dio_data->old_isize;
+ else
+ new_isize = max(dio_data->old_isize, pos);
+ i_size_write(inode, new_isize);
+ dio_data->updated_isize = false;
+ }
+ /*
+ * We have a short write, if there is any range
+ * that is submitted properly, that part will have
+ * its own OE split from the original one.
+ *
+ * So for the OE at dio_data->ordered, it's the part
+ * that is not submitted, and should be marked
+ * as fully truncated.
+ */
+ btrfs_mark_ordered_extent_truncated(dio_data->ordered, 0);
+ btrfs_finish_ordered_extent(dio_data->ordered,
+ pos, length, true);
+ } else {
unlock_dio_extent(&BTRFS_I(inode)->io_tree, pos,
pos + length - 1, NULL);
+ }
ret = -ENOTBLK;
}
if (write) {
@@ -656,9 +708,8 @@ static void btrfs_dio_end_io(struct btrfs_bio *bbio)
}
if (btrfs_op(bio) == BTRFS_MAP_WRITE) {
- btrfs_finish_ordered_extent(bbio->ordered, NULL,
- dip->file_offset, dip->bytes,
- !bio->bi_status);
+ btrfs_finish_ordered_extent(bbio->ordered, dip->file_offset,
+ dip->bytes, !bio->bi_status);
} else {
unlock_dio_extent(&inode->io_tree, dip->file_offset,
dip->file_offset + dip->bytes - 1, NULL);
@@ -736,7 +787,7 @@ static void btrfs_dio_submit_io(const struct iomap_iter *iter, struct bio *bio,
ret = btrfs_extract_ordered_extent(bbio, dio_data->ordered);
if (ret) {
- btrfs_finish_ordered_extent(dio_data->ordered, NULL,
+ btrfs_finish_ordered_extent(dio_data->ordered,
file_offset, dip->bytes,
!ret);
bio->bi_status = errno_to_blk_status(ret);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 2dab2ce94cc4..8befa96374a9 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1747,7 +1747,6 @@ static int read_backup_root(struct btrfs_fs_info *fs_info, u8 priority)
/* helper to cleanup workers */
static void btrfs_stop_all_workers(struct btrfs_fs_info *fs_info)
{
- btrfs_destroy_workqueue(fs_info->fixup_workers);
btrfs_destroy_workqueue(fs_info->delalloc_workers);
btrfs_destroy_workqueue(fs_info->workers);
if (fs_info->endio_workers)
@@ -1954,9 +1953,6 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info)
fs_info->caching_workers =
btrfs_alloc_workqueue(fs_info, "cache", flags, max_active, 0);
- fs_info->fixup_workers =
- btrfs_alloc_ordered_workqueue(fs_info, "fixup", ordered_flags);
-
fs_info->endio_workers =
alloc_workqueue("btrfs-endio", flags, max_active);
fs_info->endio_meta_workers =
@@ -1985,7 +1981,7 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info)
fs_info->compressed_write_workers &&
fs_info->endio_write_workers &&
fs_info->endio_freespace_worker && fs_info->rmw_workers &&
- fs_info->caching_workers && fs_info->fixup_workers &&
+ fs_info->caching_workers &&
fs_info->delayed_workers && fs_info->qgroup_rescan_workers &&
fs_info->discard_ctl.discard_workers)) {
return -ENOMEM;
@@ -4245,16 +4241,6 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info)
if (unlikely(BTRFS_FS_ERROR(fs_info)))
btrfs_error_commit_super(fs_info);
- /*
- * Wait for any fixup workers to complete.
- * If we don't wait for them here and they are still running by the time
- * we call kthread_stop() against the cleaner kthread further below, we
- * get an use-after-free on the cleaner because the fixup worker adds an
- * inode to the list of delayed iputs and then attempts to wakeup the
- * cleaner kthread, which was already stopped and destroyed. We parked
- * already the cleaner, but below we run all pending delayed iputs.
- */
- btrfs_flush_workqueue(fs_info->fixup_workers);
/*
* Similar case here, we have to wait for delalloc workers before we
* proceed below and stop the cleaner kthread, otherwise we trigger a
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index e2defa21b6c7..28dca13fb9b0 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -2075,7 +2075,8 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,
locked_ref = btrfs_obtain_ref_head(trans);
if (IS_ERR_OR_NULL(locked_ref)) {
if (PTR_ERR(locked_ref) == -EAGAIN) {
- continue;
+ count++;
+ goto again;
} else {
break;
}
@@ -2123,7 +2124,7 @@ static noinline int __btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,
* Either success case or btrfs_run_delayed_refs_for_head
* returned -EAGAIN, meaning we need to select another head
*/
-
+again:
locked_ref = NULL;
cond_resched();
} while ((min_bytes != U64_MAX && bytes_processed < min_bytes) ||
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 0d50f3063d34..29982abb4f9d 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -485,6 +485,7 @@ static void end_bbio_data_write(struct btrfs_bio *bbio)
int error = blk_status_to_errno(bio->bi_status);
struct folio_iter fi;
const u32 sectorsize = fs_info->sectorsize;
+ u32 bio_size = 0;
ASSERT(!bio_flagged(bio, BIO_CLONED));
bio_for_each_folio_all(fi, bio) {
@@ -495,6 +496,7 @@ static void end_bbio_data_write(struct btrfs_bio *bbio)
/* Only order 0 (single page) folios are allowed for data. */
ASSERT(folio_order(folio) == 0);
+ bio_size += len;
/* Our read/write should always be sector aligned. */
if (!IS_ALIGNED(fi.offset, sectorsize))
btrfs_err(fs_info,
@@ -505,13 +507,15 @@ static void end_bbio_data_write(struct btrfs_bio *bbio)
"incomplete page write with offset %zu and length %zu",
fi.offset, fi.length);
- btrfs_finish_ordered_extent(bbio->ordered, folio, start, len,
- !error);
if (error)
mapping_set_error(folio->mapping, error);
+
+ ASSERT(btrfs_folio_test_ordered(fs_info, folio, start, len));
+ btrfs_folio_clear_ordered(fs_info, folio, start, len);
btrfs_folio_clear_writeback(fs_info, folio, start, len);
}
+ btrfs_finish_ordered_extent(bbio->ordered, bbio->file_offset, bio_size, !error);
bio_put(bio);
}
@@ -1245,8 +1249,9 @@ static noinline_for_stack int writepage_delalloc(struct btrfs_inode *inode,
u64 delalloc_start = page_start;
u64 delalloc_end = page_end;
u64 delalloc_to_write = 0;
+ unsigned int start_bit;
+ unsigned int end_bit;
int ret = 0;
- int bit;
/* Save the dirty bitmap as our submission bitmap will be a subset of it. */
if (btrfs_is_subpage(fs_info, inode->vfs_inode.i_mapping)) {
@@ -1256,10 +1261,12 @@ static noinline_for_stack int writepage_delalloc(struct btrfs_inode *inode,
bio_ctrl->submit_bitmap = 1;
}
- for_each_set_bit(bit, &bio_ctrl->submit_bitmap, blocks_per_folio) {
- u64 start = page_start + (bit << fs_info->sectorsize_bits);
+ for_each_set_bitrange(start_bit, end_bit, &bio_ctrl->submit_bitmap,
+ blocks_per_folio) {
+ u64 start = page_start + (start_bit << fs_info->sectorsize_bits);
+ u32 len = (end_bit - start_bit) << fs_info->sectorsize_bits;
- btrfs_folio_set_lock(fs_info, folio, start, fs_info->sectorsize);
+ btrfs_folio_set_lock(fs_info, folio, start, len);
}
/* Lock all (subpage) delalloc ranges inside the folio first. */
@@ -1376,10 +1383,14 @@ static noinline_for_stack int writepage_delalloc(struct btrfs_inode *inode,
fs_info->sectorsize_bits,
blocks_per_folio);
- for_each_set_bit(bit, &bio_ctrl->submit_bitmap, bitmap_size)
- btrfs_mark_ordered_io_finished(inode, folio,
- page_start + (bit << fs_info->sectorsize_bits),
- fs_info->sectorsize, false);
+ for_each_set_bitrange(start_bit, end_bit, &bio_ctrl->submit_bitmap,
+ bitmap_size) {
+ u64 start = page_start + (start_bit << fs_info->sectorsize_bits);
+ u32 len = (end_bit - start_bit) << fs_info->sectorsize_bits;
+
+ btrfs_folio_clear_ordered(fs_info, folio, start, len);
+ btrfs_mark_ordered_io_finished(inode, start, len, false);
+ }
return ret;
}
out:
@@ -1417,7 +1428,7 @@ static noinline_for_stack int writepage_delalloc(struct btrfs_inode *inode,
/*
* Return 0 if we have submitted or queued the sector for submission.
- * Return <0 for critical errors.
+ * Return <0 for critical errors, and the involved sector will be cleaned up.
*
* Caller should make sure filepos < i_size and handle filepos >= i_size case.
*/
@@ -1440,8 +1451,32 @@ static int submit_one_sector(struct btrfs_inode *inode,
ASSERT(filepos < i_size);
em = btrfs_get_extent(inode, NULL, filepos, sectorsize);
- if (IS_ERR(em))
- return PTR_ERR_OR_ZERO(em);
+ if (IS_ERR(em)) {
+ /*
+ * bio_ctrl may contain a bio crossing several folios.
+ * Submit it immediately so that the bio has a chance
+ * to finish normally, other than marked as error.
+ */
+ submit_one_bio(bio_ctrl);
+
+ /*
+ * When submission failed, we should still clear the folio dirty.
+ * Or the folio will be written back again but without any
+ * ordered extent.
+ */
+ btrfs_folio_clear_dirty(fs_info, folio, filepos, sectorsize);
+ btrfs_folio_clear_ordered(fs_info, folio, filepos, sectorsize);
+ btrfs_folio_set_writeback(fs_info, folio, filepos, sectorsize);
+ btrfs_folio_clear_writeback(fs_info, folio, filepos, sectorsize);
+
+ /*
+ * Since there is no bio submitted to finish the ordered
+ * extent, we have to manually finish this sector.
+ */
+ btrfs_mark_ordered_io_finished(inode, filepos, fs_info->sectorsize,
+ false);
+ return PTR_ERR(em);
+ }
extent_offset = filepos - em->start;
em_end = extent_map_end(em);
@@ -1508,16 +1543,21 @@ static noinline_for_stack int extent_writepage_io(struct btrfs_inode *inode,
ASSERT(start >= folio_start && end <= folio_end);
- ret = btrfs_writepage_cow_fixup(folio);
- if (ret) {
- /* Fixup worker will requeue */
- folio_redirty_for_writepage(bio_ctrl->wbc, folio);
- folio_unlock(folio);
- return 1;
+ if (unlikely(!folio_test_ordered(folio))) {
+ WARN_ON(IS_ENABLED(CONFIG_BTRFS_DEBUG));
+ btrfs_err_rl(fs_info,
+ "root %lld ino %llu folio %llu is marked dirty without notifying the fs",
+ btrfs_root_id(inode->root),
+ btrfs_ino(inode),
+ folio_pos(folio));
+ btrfs_folio_clear_dirty(fs_info, folio, start, len);
+ btrfs_folio_set_writeback(fs_info, folio, start, len);
+ btrfs_folio_clear_writeback(fs_info, folio, start, len);
+ return -EUCLEAN;
}
- for (cur = start; cur < end; cur += fs_info->sectorsize)
- set_bit((cur - folio_start) >> fs_info->sectorsize_bits, &range_bitmap);
+ bitmap_set(&range_bitmap, (start - folio_pos(folio)) >> fs_info->sectorsize_bits,
+ len >> fs_info->sectorsize_bits);
bitmap_and(&bio_ctrl->submit_bitmap, &bio_ctrl->submit_bitmap, &range_bitmap,
blocks_per_folio);
@@ -1544,8 +1584,8 @@ static noinline_for_stack int extent_writepage_io(struct btrfs_inode *inode,
spin_unlock_irqrestore(&inode->ordered_tree_lock, flags);
btrfs_put_ordered_extent(ordered);
- btrfs_mark_ordered_io_finished(inode, folio, cur,
- fs_info->sectorsize, true);
+ btrfs_folio_clear_ordered(fs_info, folio, cur, fs_info->sectorsize);
+ btrfs_mark_ordered_io_finished(inode, cur, fs_info->sectorsize, true);
/*
* This range is beyond i_size, thus we don't need to
* bother writing back.
@@ -1559,19 +1599,6 @@ static noinline_for_stack int extent_writepage_io(struct btrfs_inode *inode,
}
ret = submit_one_sector(inode, folio, cur, bio_ctrl, i_size);
if (unlikely(ret < 0)) {
- /*
- * bio_ctrl may contain a bio crossing several folios.
- * Submit it immediately so that the bio has a chance
- * to finish normally, other than marked as error.
- */
- submit_one_bio(bio_ctrl);
- /*
- * Failed to grab the extent map which should be very rare.
- * Since there is no bio submitted to finish the ordered
- * extent, we have to manually finish this sector.
- */
- btrfs_mark_ordered_io_finished(inode, folio, cur,
- fs_info->sectorsize, false);
if (!found_error)
found_error = ret;
continue;
@@ -1587,8 +1614,8 @@ static noinline_for_stack int extent_writepage_io(struct btrfs_inode *inode,
* Here we set writeback and clear for the range. If the full folio
* is no longer dirty then we clear the PAGECACHE_TAG_DIRTY tag.
*
- * If we hit any error, the corresponding sector will still be dirty
- * thus no need to clear PAGECACHE_TAG_DIRTY.
+ * If we hit any error, the corresponding sector will have its dirty
+ * flag cleared and writeback finished, thus no need to handle the error case.
*/
if (!submitted_io && !found_error) {
btrfs_folio_set_writeback(fs_info, folio, start, len);
@@ -2399,8 +2426,7 @@ void extent_write_locked_range(struct inode *inode, const struct folio *locked_f
* code is just in case, but shouldn't actually be run.
*/
if (IS_ERR(folio)) {
- btrfs_mark_ordered_io_finished(BTRFS_I(inode), NULL,
- cur, cur_len, false);
+ btrfs_mark_ordered_io_finished(BTRFS_I(inode), cur, cur_len, false);
mapping_set_error(mapping, PTR_ERR(folio));
cur = cur_end + 1;
continue;
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index 9ed2771f303c..fe60f2d7e8ae 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -150,16 +150,8 @@ int btrfs_dirty_pages(struct btrfs_inode *inode, struct page **pages,
end_of_last_block = start_pos + num_bytes - 1;
- /*
- * The pages may have already been dirty, clear out old accounting so
- * we can set things up properly
- */
- clear_extent_bit(&inode->io_tree, start_pos, end_of_last_block,
- EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING | EXTENT_DEFRAG,
- cached);
-
- ret = btrfs_set_extent_delalloc(inode, start_pos, end_of_last_block,
- extra_bits, cached);
+ ret = btrfs_reset_extent_delalloc(inode, start_pos, end_of_last_block,
+ extra_bits, cached);
if (ret)
return ret;
@@ -1997,19 +1989,8 @@ static vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)
}
}
- /*
- * page_mkwrite gets called when the page is firstly dirtied after it's
- * faulted in, but write(2) could also dirty a page and set delalloc
- * bits, thus in this case for space account reason, we still need to
- * clear any delalloc bits within this page range since we have to
- * reserve data&meta space before lock_page() (see above comments).
- */
- clear_extent_bit(&BTRFS_I(inode)->io_tree, page_start, end,
- EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING |
- EXTENT_DEFRAG, &cached_state);
-
- ret2 = btrfs_set_extent_delalloc(BTRFS_I(inode), page_start, end, 0,
- &cached_state);
+ ret2 = btrfs_reset_extent_delalloc(BTRFS_I(inode), page_start, end, 0,
+ &cached_state);
if (ret2) {
unlock_extent(io_tree, page_start, page_end, &cached_state);
ret = VM_FAULT_SIGBUS;
diff --git a/fs/btrfs/fs.h b/fs/btrfs/fs.h
index c49ffaeee769..219345ff17f5 100644
--- a/fs/btrfs/fs.h
+++ b/fs/btrfs/fs.h
@@ -620,13 +620,6 @@ struct btrfs_fs_info {
struct btrfs_workqueue *endio_write_workers;
struct btrfs_workqueue *endio_freespace_worker;
struct btrfs_workqueue *caching_workers;
-
- /*
- * Fixup workers take dirty pages that didn't properly go through the
- * cow mechanism and make them safe to write. It happens for the
- * sys_munmap function call path.
- */
- struct btrfs_workqueue *fixup_workers;
struct btrfs_workqueue *delayed_workers;
struct task_struct *transaction_kthread;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index c79dd6ff03dc..f60f8bf38dc5 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -456,7 +456,7 @@ static inline void btrfs_cleanup_ordered_extents(struct btrfs_inode *inode,
}
}
- return btrfs_mark_ordered_io_finished(inode, NULL, offset, bytes, false);
+ return btrfs_mark_ordered_io_finished(inode, offset, bytes, false);
}
static int btrfs_dirty_inode(struct btrfs_inode *inode);
@@ -2799,7 +2799,11 @@ int btrfs_set_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
unsigned int extra_bits,
struct extent_state **cached_state)
{
- WARN_ON(PAGE_ALIGNED(end));
+ const u32 blocksize = inode->root->fs_info->sectorsize;
+
+ /* Basic alignment check. */
+ ASSERT(IS_ALIGNED(start, blocksize));
+ ASSERT(IS_ALIGNED(end + 1, blocksize));
if (start >= i_size_read(&inode->vfs_inode) &&
!(inode->flags & BTRFS_INODE_PREALLOC)) {
@@ -2822,189 +2826,48 @@ int btrfs_set_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
EXTENT_DELALLOC | extra_bits, cached_state);
}
-/* see btrfs_writepage_start_hook for details on why this is required */
-struct btrfs_writepage_fixup {
- struct folio *folio;
- struct btrfs_inode *inode;
- struct btrfs_work work;
-};
-
-static void btrfs_writepage_fixup_worker(struct btrfs_work *work)
-{
- struct btrfs_writepage_fixup *fixup =
- container_of(work, struct btrfs_writepage_fixup, work);
- struct btrfs_ordered_extent *ordered;
- struct extent_state *cached_state = NULL;
- struct extent_changeset *data_reserved = NULL;
- struct folio *folio = fixup->folio;
- struct btrfs_inode *inode = fixup->inode;
- struct btrfs_fs_info *fs_info = inode->root->fs_info;
- u64 page_start = folio_pos(folio);
- u64 page_end = folio_pos(folio) + folio_size(folio) - 1;
- int ret = 0;
- bool free_delalloc_space = true;
-
- /*
- * This is similar to page_mkwrite, we need to reserve the space before
- * we take the folio lock.
- */
- ret = btrfs_delalloc_reserve_space(inode, &data_reserved, page_start,
- folio_size(folio));
-again:
- folio_lock(folio);
-
- /*
- * Before we queued this fixup, we took a reference on the folio.
- * folio->mapping may go NULL, but it shouldn't be moved to a different
- * address space.
- */
- if (!folio->mapping || !folio_test_dirty(folio) ||
- !folio_test_checked(folio)) {
- /*
- * Unfortunately this is a little tricky, either
- *
- * 1) We got here and our folio had already been dealt with and
- * we reserved our space, thus ret == 0, so we need to just
- * drop our space reservation and bail. This can happen the
- * first time we come into the fixup worker, or could happen
- * while waiting for the ordered extent.
- * 2) Our folio was already dealt with, but we happened to get an
- * ENOSPC above from the btrfs_delalloc_reserve_space. In
- * this case we obviously don't have anything to release, but
- * because the folio was already dealt with we don't want to
- * mark the folio with an error, so make sure we're resetting
- * ret to 0. This is why we have this check _before_ the ret
- * check, because we do not want to have a surprise ENOSPC
- * when the folio was already properly dealt with.
- */
- if (!ret) {
- btrfs_delalloc_release_extents(inode, folio_size(folio));
- btrfs_delalloc_release_space(inode, data_reserved,
- page_start, folio_size(folio),
- true);
- }
- ret = 0;
- goto out_page;
- }
-
- /*
- * We can't mess with the folio state unless it is locked, so now that
- * it is locked bail if we failed to make our space reservation.
- */
- if (ret)
- goto out_page;
-
- lock_extent(&inode->io_tree, page_start, page_end, &cached_state);
-
- /* already ordered? We're done */
- if (folio_test_ordered(folio))
- goto out_reserved;
-
- ordered = btrfs_lookup_ordered_range(inode, page_start, PAGE_SIZE);
- if (ordered) {
- unlock_extent(&inode->io_tree, page_start, page_end,
- &cached_state);
- folio_unlock(folio);
- btrfs_start_ordered_extent(ordered);
- btrfs_put_ordered_extent(ordered);
- goto again;
- }
-
- ret = btrfs_set_extent_delalloc(inode, page_start, page_end, 0,
- &cached_state);
- if (ret)
- goto out_reserved;
-
- /*
- * Everything went as planned, we're now the owner of a dirty page with
- * delayed allocation bits set and space reserved for our COW
- * destination.
- *
- * The page was dirty when we started, nothing should have cleaned it.
- */
- BUG_ON(!folio_test_dirty(folio));
- free_delalloc_space = false;
-out_reserved:
- btrfs_delalloc_release_extents(inode, PAGE_SIZE);
- if (free_delalloc_space)
- btrfs_delalloc_release_space(inode, data_reserved, page_start,
- PAGE_SIZE, true);
- unlock_extent(&inode->io_tree, page_start, page_end, &cached_state);
-out_page:
- if (ret) {
- /*
- * We hit ENOSPC or other errors. Update the mapping and page
- * to reflect the errors and clean the page.
- */
- mapping_set_error(folio->mapping, ret);
- btrfs_mark_ordered_io_finished(inode, folio, page_start,
- folio_size(folio), !ret);
- folio_clear_dirty_for_io(folio);
- }
- btrfs_folio_clear_checked(fs_info, folio, page_start, PAGE_SIZE);
- folio_unlock(folio);
- folio_put(folio);
- kfree(fixup);
- extent_changeset_free(data_reserved);
- /*
- * As a precaution, do a delayed iput in case it would be the last iput
- * that could need flushing space. Recursing back to fixup worker would
- * deadlock.
- */
- btrfs_add_delayed_iput(inode);
-}
-
/*
- * There are a few paths in the higher layers of the kernel that directly
- * set the folio dirty bit without asking the filesystem if it is a
- * good idea. This causes problems because we want to make sure COW
- * properly happens and the data=ordered rules are followed.
+ * Clear the old accounting flags and set EXTENT_DELALLOC for the range.
*
- * In our case any range that doesn't have the ORDERED bit set
- * hasn't been properly setup for IO. We kick off an async process
- * to fix it up. The async helper will wait for ordered extents, set
- * the delalloc bit and make it safe to write the folio.
+ * Return <0 for error, in that case no range has EXTENT_DELALLOC bit cleared or set.
*/
-int btrfs_writepage_cow_fixup(struct folio *folio)
+int btrfs_reset_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
+ unsigned int extra_bits, struct extent_state **cached_state)
{
- struct inode *inode = folio->mapping->host;
- struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
- struct btrfs_writepage_fixup *fixup;
+ const u32 blocksize = inode->root->fs_info->sectorsize;
- /* This folio has ordered extent covering it already */
- if (folio_test_ordered(folio))
- return 0;
+ /* The @extra_bits can only be EXTENT_NORESERVE for now. */
+ ASSERT(!(extra_bits & ~EXTENT_NORESERVE));
- /*
- * folio_checked is set below when we create a fixup worker for this
- * folio, don't try to create another one if we're already
- * folio_test_checked.
- *
- * The extent_io writepage code will redirty the foio if we send back
- * EAGAIN.
- */
- if (folio_test_checked(folio))
- return -EAGAIN;
-
- fixup = kzalloc(sizeof(*fixup), GFP_NOFS);
- if (!fixup)
- return -EAGAIN;
+ /* Basic alignment check. */
+ ASSERT(IS_ALIGNED(start, blocksize));
+ ASSERT(IS_ALIGNED(end + 1, blocksize));
/*
- * We are already holding a reference to this inode from
- * write_cache_pages. We need to hold it because the space reservation
- * takes place outside of the folio lock, and we can't trust
- * page->mapping outside of the folio lock.
+ * Check and set DELALLOC_NEW flag, this needs to search tree thus can
+ * fail early. Thus we want to do this before clearing EXTENT_DELALLOC.
*/
- ihold(inode);
- btrfs_folio_set_checked(fs_info, folio, folio_pos(folio), folio_size(folio));
- folio_get(folio);
- btrfs_init_work(&fixup->work, btrfs_writepage_fixup_worker, NULL);
- fixup->folio = folio;
- fixup->inode = BTRFS_I(inode);
- btrfs_queue_work(fs_info->fixup_workers, &fixup->work);
+ if (start >= i_size_read(&inode->vfs_inode) &&
+ !(inode->flags & BTRFS_INODE_PREALLOC)) {
+ /*
+ * There can't be any extents following EOF in this case so just
+ * set the delalloc new bit for the range directly.
+ */
+ extra_bits |= EXTENT_DELALLOC_NEW;
+ } else {
+ int ret;
- return -EAGAIN;
+ ret = btrfs_find_new_delalloc_bytes(inode, start, end + 1 - start,
+ NULL);
+ if (unlikely(ret))
+ return ret;
+ }
+ /* Clear the old accounting as the range may already be dirty. */
+ clear_extent_bit(&inode->io_tree, start, end,
+ EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING |
+ EXTENT_DEFRAG, cached_state);
+ return set_extent_bit(&inode->io_tree, start, end,
+ EXTENT_DELALLOC | extra_bits, cached_state);
}
static int insert_reserved_file_extent(struct btrfs_trans_handle *trans,
@@ -4930,12 +4793,7 @@ int btrfs_truncate_block(struct btrfs_inode *inode, loff_t from, loff_t len,
goto again;
}
- clear_extent_bit(&inode->io_tree, block_start, block_end,
- EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING | EXTENT_DEFRAG,
- &cached_state);
-
- ret = btrfs_set_extent_delalloc(inode, block_start, block_end, 0,
- &cached_state);
+ ret = btrfs_reset_extent_delalloc(inode, block_start, block_end, 0, &cached_state);
if (ret) {
unlock_extent(io_tree, block_start, block_end, &cached_state);
goto out_unlock;
@@ -7530,11 +7388,7 @@ static void btrfs_invalidate_folio(struct folio *folio, size_t offset,
EXTENT_LOCKED | EXTENT_DO_ACCOUNTING |
EXTENT_DEFRAG, &cached_state);
- spin_lock_irq(&inode->ordered_tree_lock);
- set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
- ordered->truncated_len = min(ordered->truncated_len,
- cur - ordered->file_offset);
- spin_unlock_irq(&inode->ordered_tree_lock);
+ btrfs_mark_ordered_extent_truncated(ordered, cur - ordered->file_offset);
/*
* If the ordered extent has finished, we're safe to delete all
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index 6ac254a52907..692f03803da9 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -329,6 +329,18 @@ void btrfs_mark_ordered_extent_error(struct btrfs_ordered_extent *ordered)
mapping_set_error(ordered->inode->vfs_inode.i_mapping, -EIO);
}
+void btrfs_mark_ordered_extent_truncated(struct btrfs_ordered_extent *ordered,
+ u64 truncate_len)
+{
+ struct btrfs_inode *inode = ordered->inode;
+
+ ASSERT(truncate_len <= ordered->num_bytes);
+ spin_lock_irq(&inode->ordered_tree_lock);
+ set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
+ ordered->truncated_len = min(ordered->truncated_len, truncate_len);
+ spin_unlock_irq(&inode->ordered_tree_lock);
+}
+
static void finish_ordered_fn(struct btrfs_work *work)
{
struct btrfs_ordered_extent *ordered_extent;
@@ -338,30 +350,13 @@ static void finish_ordered_fn(struct btrfs_work *work)
}
static bool can_finish_ordered_extent(struct btrfs_ordered_extent *ordered,
- struct folio *folio, u64 file_offset,
- u64 len, bool uptodate)
+ u64 file_offset, u64 len, bool uptodate)
{
struct btrfs_inode *inode = ordered->inode;
struct btrfs_fs_info *fs_info = inode->root->fs_info;
lockdep_assert_held(&inode->ordered_tree_lock);
- if (folio) {
- ASSERT(folio->mapping);
- ASSERT(folio_pos(folio) <= file_offset);
- ASSERT(file_offset + len <= folio_pos(folio) + folio_size(folio));
-
- /*
- * Ordered (Private2) bit indicates whether we still have
- * pending io unfinished for the ordered extent.
- *
- * If there's no such bit, we need to skip to next range.
- */
- if (!btrfs_folio_test_ordered(fs_info, folio, file_offset, len))
- return false;
- btrfs_folio_clear_ordered(fs_info, folio, file_offset, len);
- }
-
/* Now we're fine to update the accounting. */
if (WARN_ON_ONCE(len > ordered->bytes_left)) {
btrfs_crit(fs_info,
@@ -403,8 +398,7 @@ static void btrfs_queue_ordered_fn(struct btrfs_ordered_extent *ordered)
}
void btrfs_finish_ordered_extent(struct btrfs_ordered_extent *ordered,
- struct folio *folio, u64 file_offset, u64 len,
- bool uptodate)
+ u64 file_offset, u64 len, bool uptodate)
{
struct btrfs_inode *inode = ordered->inode;
unsigned long flags;
@@ -413,7 +407,7 @@ void btrfs_finish_ordered_extent(struct btrfs_ordered_extent *ordered,
trace_btrfs_finish_ordered_extent(inode, file_offset, len, uptodate);
spin_lock_irqsave(&inode->ordered_tree_lock, flags);
- ret = can_finish_ordered_extent(ordered, folio, file_offset, len,
+ ret = can_finish_ordered_extent(ordered, file_offset, len,
uptodate);
spin_unlock_irqrestore(&inode->ordered_tree_lock, flags);
@@ -466,8 +460,7 @@ void btrfs_finish_ordered_extent(struct btrfs_ordered_extent *ordered,
* extent(s) covering it.
*/
void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,
- struct folio *folio, u64 file_offset,
- u64 num_bytes, bool uptodate)
+ u64 file_offset, u64 num_bytes, bool uptodate)
{
struct rb_node *node;
struct btrfs_ordered_extent *entry = NULL;
@@ -530,7 +523,7 @@ void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,
ASSERT(end + 1 - cur < U32_MAX);
len = end + 1 - cur;
- if (can_finish_ordered_extent(entry, folio, cur, len, uptodate)) {
+ if (can_finish_ordered_extent(entry, cur, len, uptodate)) {
spin_unlock_irqrestore(&inode->ordered_tree_lock, flags);
btrfs_queue_ordered_fn(entry);
spin_lock_irqsave(&inode->ordered_tree_lock, flags);
diff --git a/fs/btrfs/ordered-data.h b/fs/btrfs/ordered-data.h
index 4e152736d06c..a1a9f8d46e10 100644
--- a/fs/btrfs/ordered-data.h
+++ b/fs/btrfs/ordered-data.h
@@ -163,11 +163,9 @@ void btrfs_put_ordered_extent(struct btrfs_ordered_extent *entry);
void btrfs_remove_ordered_extent(struct btrfs_inode *btrfs_inode,
struct btrfs_ordered_extent *entry);
void btrfs_finish_ordered_extent(struct btrfs_ordered_extent *ordered,
- struct folio *folio, u64 file_offset, u64 len,
- bool uptodate);
+ u64 file_offset, u64 len, bool uptodate);
void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,
- struct folio *folio, u64 file_offset,
- u64 num_bytes, bool uptodate);
+ u64 file_offset, u64 num_bytes, bool uptodate);
bool btrfs_dec_test_ordered_pending(struct btrfs_inode *inode,
struct btrfs_ordered_extent **cached,
u64 file_offset, u64 io_size);
@@ -215,6 +213,8 @@ bool btrfs_try_lock_ordered_range(struct btrfs_inode *inode, u64 start, u64 end,
struct btrfs_ordered_extent *btrfs_split_ordered_extent(
struct btrfs_ordered_extent *ordered, u64 len);
void btrfs_mark_ordered_extent_error(struct btrfs_ordered_extent *ordered);
+void btrfs_mark_ordered_extent_truncated(struct btrfs_ordered_extent *ordered,
+ u64 truncate_len);
int __init ordered_data_init(void);
void __cold ordered_data_exit(void);
diff --git a/fs/btrfs/reflink.c b/fs/btrfs/reflink.c
index e86df9c83aba..08e84c1eb47f 100644
--- a/fs/btrfs/reflink.c
+++ b/fs/btrfs/reflink.c
@@ -95,10 +95,7 @@ static int copy_inline_to_page(struct btrfs_inode *inode,
if (ret < 0)
goto out_unlock;
- clear_extent_bit(&inode->io_tree, file_offset, range_end,
- EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING | EXTENT_DEFRAG,
- NULL);
- ret = btrfs_set_extent_delalloc(inode, file_offset, range_end, 0, NULL);
+ ret = btrfs_reset_extent_delalloc(inode, file_offset, range_end, 0, NULL);
if (ret)
goto out_unlock;
@@ -183,10 +180,12 @@ static int clone_copy_inline_extent(struct btrfs_inode *inode,
struct btrfs_drop_extents_args drop_args = { 0 };
int ret;
struct btrfs_key key;
+ bool copied_inline_to_page = false;
if (new_key->offset > 0) {
ret = copy_inline_to_page(inode, new_key->offset,
inline_data, size, datal, comp_type);
+ copied_inline_to_page = (ret == 0);
goto out;
}
@@ -286,6 +285,60 @@ static int clone_copy_inline_extent(struct btrfs_inode *inode,
ret = btrfs_inode_set_file_extent_range(inode, 0, aligned_end);
out:
if (!ret && !trans) {
+ if (copied_inline_to_page &&
+ new_key->offset + datal > i_size_read(&inode->vfs_inode)) {
+ /*
+ * If we copied the inline extent data to a page/folio
+ * beyond the i_size of the destination inode, then we
+ * need to increase the i_size before we start a
+ * transaction to update the inode item. This is to
+ * prevent a deadlock when the flushoncommit mount
+ * option is used, which happens like this:
+ *
+ * 1) Task A clones an inline extent from inode X to an
+ * offset of inode Y that is beyond Y's current
+ * i_size. This means we copied the inline extent's
+ * data to a folio of inode Y that is beyond its EOF,
+ * using the call above to copy_inline_to_page();
+ *
+ * 2) Task B starts a transaction commit and calls
+ * btrfs_start_delalloc_flush() to flush delalloc;
+ *
+ * 3) The delalloc flushing sees the new dirty folio of
+ * inode Y and when it attempts to flush it, it ends
+ * up at extent_writepage() and sees that the offset
+ * of the folio is beyond the i_size of inode Y, so
+ * it attempts to invalidate the folio by calling
+ * folio_invalidate(), which ends up at btrfs' folio
+ * invalidate callback - btrfs_invalidate_folio().
+ * There it tries to lock the folio's range in inode
+ * Y's extent io tree, but it blocks since it's
+ * currently locked by task A - during reflink we
+ * lock the inodes and the source and destination
+ * ranges after flushing all delalloc and waiting for
+ * ordered extent completion - after that we don't
+ * expect to have dirty folios in the ranges, the
+ * exception is if we have to copy an inline extent's
+ * data (because the destination offset is not zero);
+ *
+ * 4) Task A then does the 'goto out' below and attempts
+ * to start a transaction to update the inode item,
+ * and then it's blocked since the current
+ * transaction is in the TRANS_STATE_COMMIT_START
+ * state. Therefore task A has to wait for the
+ * current transaction to become unblocked (its
+ * state >= TRANS_STATE_UNBLOCKED).
+ *
+ * This leads to a deadlock - the task committing the
+ * transaction waiting for the delalloc flushing which
+ * is blocked during folio invalidation on the inode's
+ * extent lock and the reflink task waiting for the
+ * current transaction to be unblocked so that it can
+ * start a new one to update the inode item (while
+ * holding the extent lock).
+ */
+ i_size_write(&inode->vfs_inode, new_key->offset + datal);
+ }
/*
* No transaction here means we copied the inline extent into a
* page of the destination inode.
@@ -320,50 +373,7 @@ static int clone_copy_inline_extent(struct btrfs_inode *inode,
ret = copy_inline_to_page(inode, new_key->offset,
inline_data, size, datal, comp_type);
-
- /*
- * If we copied the inline extent data to a page/folio beyond the i_size
- * of the destination inode, then we need to increase the i_size before
- * we start a transaction to update the inode item. This is to prevent a
- * deadlock when the flushoncommit mount option is used, which happens
- * like this:
- *
- * 1) Task A clones an inline extent from inode X to an offset of inode
- * Y that is beyond Y's current i_size. This means we copied the
- * inline extent's data to a folio of inode Y that is beyond its EOF,
- * using the call above to copy_inline_to_page();
- *
- * 2) Task B starts a transaction commit and calls
- * btrfs_start_delalloc_flush() to flush delalloc;
- *
- * 3) The delalloc flushing sees the new dirty folio of inode Y and when
- * it attempts to flush it, it ends up at extent_writepage() and sees
- * that the offset of the folio is beyond the i_size of inode Y, so
- * it attempts to invalidate the folio by calling folio_invalidate(),
- * which ends up at btrfs' folio invalidate callback -
- * btrfs_invalidate_folio(). There it tries to lock the folio's range
- * in inode Y's extent io tree, but it blocks since it's currently
- * locked by task A - during reflink we lock the inodes and the
- * source and destination ranges after flushing all delalloc and
- * waiting for ordered extent completion - after that we don't expect
- * to have dirty folios in the ranges, the exception is if we have to
- * copy an inline extent's data (because the destination offset is
- * not zero);
- *
- * 4) Task A then does the 'goto out' below and attempts to start a
- * transaction to update the inode item, and then it's blocked since
- * the current transaction is in the TRANS_STATE_COMMIT_START state.
- * Therefore task A has to wait for the current transaction to become
- * unblocked (its state >= TRANS_STATE_UNBLOCKED).
- *
- * This leads to a deadlock - the task committing the transaction
- * waiting for the delalloc flushing which is blocked during folio
- * invalidation on the inode's extent lock and the reflink task waiting
- * for the current transaction to be unblocked so that it can start a
- * a new one to update the inode item (while holding the extent lock).
- */
- if (ret == 0 && new_key->offset + datal > i_size_read(&inode->vfs_inode))
- i_size_write(&inode->vfs_inode, new_key->offset + datal);
+ copied_inline_to_page = (ret == 0);
goto out;
}
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 833602511f62..21419bc33863 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1732,7 +1732,8 @@ static int btrfs_statfs(struct dentry *dentry, struct kstatfs *buf)
int mixed = 0;
list_for_each_entry(found, &fs_info->space_info, list) {
- if (found->flags & BTRFS_BLOCK_GROUP_DATA) {
+ if (found->flags & BTRFS_BLOCK_GROUP_DATA &&
+ found->subgroup_id != BTRFS_SUB_GROUP_DATA_RELOC) {
int i;
total_free_data += found->disk_total - found->disk_used;
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index 5addf09cb6e6..8d1a646e229a 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -253,8 +253,9 @@ static void finish_netfs_read(struct ceph_osd_request *req)
subreq->transferred = err;
err = 0;
}
+ subreq->error = err;
trace_netfs_sreq(subreq, netfs_sreq_trace_io_progress);
- netfs_read_subreq_terminated(subreq, err, false);
+ netfs_read_subreq_terminated(subreq, false);
iput(req->r_inode);
ceph_dec_osd_stopping_blocker(fsc->mdsc);
}
@@ -314,7 +315,9 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
ceph_mdsc_put_request(req);
out:
- netfs_read_subreq_terminated(subreq, err, false);
+ subreq->error = err;
+ trace_netfs_sreq(subreq, netfs_sreq_trace_io_progress);
+ netfs_read_subreq_terminated(subreq, false);
return true;
}
@@ -435,8 +438,10 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
ceph_osdc_start_request(req->r_osdc, req);
out:
ceph_osdc_put_request(req);
- if (err)
- netfs_read_subreq_terminated(subreq, err, false);
+ if (err) {
+ subreq->error = err;
+ netfs_read_subreq_terminated(subreq, false);
+ }
doutc(cl, "%llx.%llx result %d\n", ceph_vinop(inode), err);
}
diff --git a/fs/configfs/dir.c b/fs/configfs/dir.c
index e5b6a427f31c..248d70b26f3d 100644
--- a/fs/configfs/dir.c
+++ b/fs/configfs/dir.c
@@ -225,15 +225,16 @@ static int configfs_dirent_exists(struct configfs_dirent *parent_sd,
{
struct configfs_dirent * sd;
+ spin_lock(&configfs_dirent_lock);
list_for_each_entry(sd, &parent_sd->s_children, s_sibling) {
if (sd->s_element) {
- const unsigned char *existing = configfs_get_name(sd);
- if (strcmp(existing, new))
- continue;
- else
+ if (strcmp(configfs_get_name(sd), new) == 0) {
+ spin_unlock(&configfs_dirent_lock);
return -EEXIST;
+ }
}
}
+ spin_unlock(&configfs_dirent_lock);
return 0;
}
@@ -462,6 +463,9 @@ static struct dentry * configfs_lookup(struct inode *dir,
inode = configfs_create(dentry, mode);
if (IS_ERR(inode)) {
+ spin_lock(&configfs_dirent_lock);
+ sd->s_dentry = NULL;
+ spin_unlock(&configfs_dirent_lock);
configfs_put(sd);
return ERR_CAST(inode);
}
@@ -548,11 +552,28 @@ static void configfs_detach_rollback(struct dentry *dentry)
configfs_detach_rollback(sd->s_dentry);
}
+/*
+ * Find the next non-cursor. configfs_dirent_lock held by caller.
+ */
+static struct configfs_dirent *next_dirent(struct configfs_dirent *parent,
+ struct configfs_dirent *last)
+{
+ struct configfs_dirent *s;
+
+ s = list_prepare_entry(last, &parent->s_children, s_sibling);
+
+ list_for_each_entry_continue(s, &parent->s_children, s_sibling) {
+ if (s->s_element)
+ return s;
+ }
+ return NULL;
+}
+
static void detach_attrs(struct config_item * item)
{
struct dentry * dentry = dget(item->ci_dentry);
- struct configfs_dirent * parent_sd;
- struct configfs_dirent * sd, * tmp;
+ struct configfs_dirent *parent_sd;
+ struct configfs_dirent *sd, *next;
if (!dentry)
return;
@@ -561,15 +582,19 @@ static void detach_attrs(struct config_item * item)
dentry->d_name.name);
parent_sd = dentry->d_fsdata;
- list_for_each_entry_safe(sd, tmp, &parent_sd->s_children, s_sibling) {
- if (!sd->s_element || !(sd->s_type & CONFIGFS_NOT_PINNED))
+
+ spin_lock(&configfs_dirent_lock);
+ for (sd = next_dirent(parent_sd, NULL); sd; sd = next) {
+ next = next_dirent(parent_sd, sd);
+ if (!(sd->s_type & CONFIGFS_NOT_PINNED))
continue;
- spin_lock(&configfs_dirent_lock);
list_del_init(&sd->s_sibling);
spin_unlock(&configfs_dirent_lock);
configfs_drop_dentry(sd, dentry);
configfs_put(sd);
+ spin_lock(&configfs_dirent_lock);
}
+ spin_unlock(&configfs_dirent_lock);
/**
* Drop reference from dget() on entrance.
@@ -628,18 +653,20 @@ static void detach_groups(struct config_group *group)
struct dentry * dentry = dget(group->cg_item.ci_dentry);
struct dentry *child;
struct configfs_dirent *parent_sd;
- struct configfs_dirent *sd, *tmp;
+ struct configfs_dirent *sd, *next;
if (!dentry)
return;
parent_sd = dentry->d_fsdata;
- list_for_each_entry_safe(sd, tmp, &parent_sd->s_children, s_sibling) {
- if (!sd->s_element ||
- !(sd->s_type & CONFIGFS_USET_DEFAULT))
+ spin_lock(&configfs_dirent_lock);
+ for (sd = next_dirent(parent_sd, NULL); sd; sd = next) {
+ next = next_dirent(parent_sd, sd);
+ if (!(sd->s_type & CONFIGFS_USET_DEFAULT))
continue;
child = sd->s_dentry;
+ spin_unlock(&configfs_dirent_lock);
inode_lock(d_inode(child));
@@ -651,7 +678,9 @@ static void detach_groups(struct config_group *group)
d_delete(child);
dput(child);
+ spin_lock(&configfs_dirent_lock);
}
+ spin_unlock(&configfs_dirent_lock);
/**
* Drop reference from dget() on entrance.
@@ -1103,6 +1132,7 @@ configfs_find_subsys_dentry(struct configfs_dirent *root_sd,
struct configfs_dirent *p;
struct configfs_dirent *ret = NULL;
+ spin_lock(&configfs_dirent_lock);
list_for_each_entry(p, &root_sd->s_children, s_sibling) {
if (p->s_type & CONFIGFS_DIR &&
p->s_element == subsys_item) {
@@ -1110,6 +1140,7 @@ configfs_find_subsys_dentry(struct configfs_dirent *root_sd,
break;
}
}
+ spin_unlock(&configfs_dirent_lock);
return ret;
}
diff --git a/fs/crypto/fscrypt_private.h b/fs/crypto/fscrypt_private.h
index 25bcfcc2d706..eac9f657caea 100644
--- a/fs/crypto/fscrypt_private.h
+++ b/fs/crypto/fscrypt_private.h
@@ -27,6 +27,9 @@
*/
#define FSCRYPT_MIN_KEY_SIZE 16
+/* Maximum size of a raw fscrypt master key */
+#define FSCRYPT_MAX_RAW_KEY_SIZE 64
+
/*
* This mask is passed as the third argument to the crypto_alloc_*() functions
* to prevent fscrypt from using the Crypto API drivers for non-inline crypto
@@ -217,7 +220,7 @@ struct fscrypt_symlink_data {
* @tfm: crypto API transform object
* @blk_key: key for blk-crypto
*
- * Normally only one of the fields will be non-NULL.
+ * Only one of the fields is non-NULL.
*/
struct fscrypt_prepared_key {
struct crypto_skcipher *tfm;
@@ -226,6 +229,15 @@ struct fscrypt_prepared_key {
#endif
};
+/* An entry in the linked list ->mk_mode_keys */
+struct fscrypt_mode_key {
+ struct fscrypt_prepared_key key;
+ struct list_head link;
+ u8 hkdf_context;
+ u8 mode_num;
+ u8 data_unit_bits;
+};
+
/*
* fscrypt_inode_info - the "encryption key" for an inode
*
@@ -413,20 +425,12 @@ void fscrypt_destroy_inline_crypt_key(struct super_block *sb,
* @prep_key, depending on which encryption implementation the file will use.
*/
static inline bool
-fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key,
+fscrypt_is_key_prepared(const struct fscrypt_prepared_key *prep_key,
const struct fscrypt_inode_info *ci)
{
- /*
- * The two smp_load_acquire()'s here pair with the smp_store_release()'s
- * in fscrypt_prepare_inline_crypt_key() and fscrypt_prepare_key().
- * I.e., in some cases (namely, if this prep_key is a per-mode
- * encryption key) another task can publish blk_key or tfm concurrently,
- * executing a RELEASE barrier. We need to use smp_load_acquire() here
- * to safely ACQUIRE the memory the other task published.
- */
if (fscrypt_using_inline_encryption(ci))
- return smp_load_acquire(&prep_key->blk_key) != NULL;
- return smp_load_acquire(&prep_key->tfm) != NULL;
+ return prep_key->blk_key != NULL;
+ return prep_key->tfm != NULL;
}
#else /* CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
@@ -458,15 +462,28 @@ fscrypt_destroy_inline_crypt_key(struct super_block *sb,
}
static inline bool
-fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key,
+fscrypt_is_key_prepared(const struct fscrypt_prepared_key *prep_key,
const struct fscrypt_inode_info *ci)
{
- return smp_load_acquire(&prep_key->tfm) != NULL;
+ return prep_key->tfm != NULL;
}
#endif /* !CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
/* keyring.c */
+/*
+ * fscrypt_master_key_user - a user's claim to a master key
+ */
+struct fscrypt_master_key_user {
+ struct list_head link;
+ kuid_t uid;
+ /*
+ * This 'struct key' contains no secret. It exists solely to charge the
+ * appropriate user's key quota.
+ */
+ struct key *quota_key;
+};
+
/*
* fscrypt_master_key_secret - secret key material of an in-use master key
*/
@@ -531,8 +548,8 @@ struct fscrypt_master_key {
/*
* Active and structural reference counts. An active ref guarantees
* that the struct continues to exist, continues to be in the keyring
- * ->s_master_keys, and that any embedded subkeys (e.g.
- * ->mk_direct_keys) that have been prepared continue to exist.
+ * ->s_master_keys, and that any non-file-scoped subkeys (e.g.
+ * ->mk_mode_keys) that have been prepared continue to exist.
* A structural ref only guarantees that the struct continues to exist.
*
* There is one active ref associated with ->mk_present being true, and
@@ -564,19 +581,18 @@ struct fscrypt_master_key {
struct fscrypt_key_specifier mk_spec;
/*
- * Keyring which contains a key of type 'key_type_fscrypt_user' for each
- * user who has added this key. Normally each key will be added by just
- * one user, but it's possible that multiple users share a key, and in
- * that case we need to keep track of those users so that one user can't
- * remove the key before the others want it removed too.
+ * List of user claims to this key (struct fscrypt_master_key_user).
+ * Normally each key will be added by just one user, but it's possible
+ * that multiple users share a key, and in that case we need to keep
+ * track of those users so that one user can't remove the key before the
+ * others want it removed too.
*
- * This is NULL for v1 policy keys; those can only be added by root.
+ * Used only for v2 policy keys. v1 policy keys can be added only by
+ * root, so user tracking doesn't apply to them.
*
- * Locking: protected by ->mk_sem. (We don't just rely on the keyrings
- * subsystem semaphore ->mk_users->sem, as we need support for atomic
- * search+insert along with proper synchronization with other fields.)
+ * Locking: protected by ->mk_sem.
*/
- struct key *mk_users;
+ struct list_head mk_users;
/*
* List of inodes that were unlocked using this key. This allows the
@@ -586,12 +602,21 @@ struct fscrypt_master_key {
spinlock_t mk_decrypted_inodes_lock;
/*
- * Per-mode encryption keys for the various types of encryption policies
- * that use them. Allocated and derived on-demand.
+ * A list of 'struct fscrypt_mode_key' for the (hkdf_context, mode_num,
+ * data_unit_bits, inlinecrypt) combinations that are in use for this
+ * master key, for hkdf_context in [HKDF_CONTEXT_DIRECT_KEY,
+ * HKDF_CONTEXT_IV_INO_LBLK_32_KEY, HKDF_CONTEXT_IV_INO_LBLK_64_KEY].
+ *
+ * This is a linked list and not a hash table because in practice
+ * there's just a single encryption policy per master key, using
+ * _at most_ 2 nodes in this list. Per-file keys don't use this at all.
+ *
+ * This list is append-only until the master key is fully removed, at
+ * which time the list is cleared. Before then,
+ * fscrypt_mode_key_setup_mutex synchronizes appends, and searches use
+ * the RCU read lock together with ->mk_sem held for read.
*/
- struct fscrypt_prepared_key mk_direct_keys[FSCRYPT_MODE_MAX + 1];
- struct fscrypt_prepared_key mk_iv_ino_lblk_64_keys[FSCRYPT_MODE_MAX + 1];
- struct fscrypt_prepared_key mk_iv_ino_lblk_32_keys[FSCRYPT_MODE_MAX + 1];
+ struct list_head mk_mode_keys;
/* Hash key for inode numbers. Initialized only when needed. */
siphash_key_t mk_ino_hash_key;
diff --git a/fs/crypto/inline_crypt.c b/fs/crypto/inline_crypt.c
index 40de69860dcf..4b1f10f1cbd8 100644
--- a/fs/crypto/inline_crypt.c
+++ b/fs/crypto/inline_crypt.c
@@ -191,13 +191,7 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key,
goto fail;
}
- /*
- * Pairs with the smp_load_acquire() in fscrypt_is_key_prepared().
- * I.e., here we publish ->blk_key with a RELEASE barrier so that
- * concurrent tasks can ACQUIRE it. Note that this concurrency is only
- * possible for per-mode keys, not for per-file keys.
- */
- smp_store_release(&prep_key->blk_key, blk_key);
+ prep_key->blk_key = blk_key;
return 0;
fail:
diff --git a/fs/crypto/keyring.c b/fs/crypto/keyring.c
index 206835e31efa..46e2bf043e73 100644
--- a/fs/crypto/keyring.c
+++ b/fs/crypto/keyring.c
@@ -64,36 +64,33 @@ static void fscrypt_free_master_key(struct rcu_head *head)
kfree_sensitive(mk);
}
+static void clear_mk_users(struct fscrypt_master_key *mk);
+
void fscrypt_put_master_key(struct fscrypt_master_key *mk)
{
if (!refcount_dec_and_test(&mk->mk_struct_refs))
return;
/*
- * No structural references left, so free ->mk_users, and also free the
+ * No structural references left, so clear ->mk_users, and also free the
* fscrypt_master_key struct itself after an RCU grace period ensures
* that concurrent keyring lookups can no longer find it.
*/
WARN_ON_ONCE(refcount_read(&mk->mk_active_refs) != 0);
- if (mk->mk_users) {
- /* Clear the keyring so the quota gets released right away. */
- keyring_clear(mk->mk_users);
- key_put(mk->mk_users);
- mk->mk_users = NULL;
- }
+ clear_mk_users(mk);
call_rcu(&mk->mk_rcu_head, fscrypt_free_master_key);
}
void fscrypt_put_master_key_activeref(struct super_block *sb,
struct fscrypt_master_key *mk)
{
- size_t i;
+ struct fscrypt_mode_key *node, *tmp;
if (!refcount_dec_and_test(&mk->mk_active_refs))
return;
/*
* No active references left, so complete the full removal of this
* fscrypt_master_key struct by removing it from the keyring and
- * destroying any subkeys embedded in it.
+ * destroying any non-file-scoped subkeys.
*/
if (WARN_ON_ONCE(!sb->s_master_keys))
@@ -109,13 +106,16 @@ void fscrypt_put_master_key_activeref(struct super_block *sb,
WARN_ON_ONCE(mk->mk_present);
WARN_ON_ONCE(!list_empty(&mk->mk_decrypted_inodes));
- for (i = 0; i <= FSCRYPT_MODE_MAX; i++) {
- fscrypt_destroy_prepared_key(
- sb, &mk->mk_direct_keys[i]);
- fscrypt_destroy_prepared_key(
- sb, &mk->mk_iv_ino_lblk_64_keys[i]);
- fscrypt_destroy_prepared_key(
- sb, &mk->mk_iv_ino_lblk_32_keys[i]);
+ /*
+ * Destroy any non-file-scoped subkeys. Since ->mk_active_refs == 0,
+ * they're no longer referenced by any inodes. Nor can key setup run
+ * and use them again. So they're no longer needed. (This implies no
+ * concurrent readers, so we don't need list_del_rcu() for example.)
+ */
+ list_for_each_entry_safe(node, tmp, &mk->mk_mode_keys, link) {
+ fscrypt_destroy_prepared_key(sb, &node->key);
+ list_del(&node->link);
+ kfree(node);
}
memzero_explicit(&mk->mk_ino_hash_key,
sizeof(mk->mk_ino_hash_key));
@@ -161,8 +161,8 @@ static void fscrypt_user_key_describe(const struct key *key, struct seq_file *m)
}
/*
- * Type of key in ->mk_users. Each key of this type represents a particular
- * user who has added a particular master key.
+ * Type of fscrypt_master_key_user::quota_key. This contains no secret; it
+ * exists solely to charge a user's key quota.
*
* Note that the name of this key type really should be something like
* ".fscrypt-user" instead of simply ".fscrypt". But the shorter name is chosen
@@ -176,30 +176,9 @@ static struct key_type key_type_fscrypt_user = {
.describe = fscrypt_user_key_describe,
};
-#define FSCRYPT_MK_USERS_DESCRIPTION_SIZE \
- (CONST_STRLEN("fscrypt-") + 2 * FSCRYPT_KEY_IDENTIFIER_SIZE + \
- CONST_STRLEN("-users") + 1)
-
#define FSCRYPT_MK_USER_DESCRIPTION_SIZE \
(2 * FSCRYPT_KEY_IDENTIFIER_SIZE + CONST_STRLEN(".uid.") + 10 + 1)
-static void format_mk_users_keyring_description(
- char description[FSCRYPT_MK_USERS_DESCRIPTION_SIZE],
- const u8 mk_identifier[FSCRYPT_KEY_IDENTIFIER_SIZE])
-{
- sprintf(description, "fscrypt-%*phN-users",
- FSCRYPT_KEY_IDENTIFIER_SIZE, mk_identifier);
-}
-
-static void format_mk_user_description(
- char description[FSCRYPT_MK_USER_DESCRIPTION_SIZE],
- const u8 mk_identifier[FSCRYPT_KEY_IDENTIFIER_SIZE])
-{
-
- sprintf(description, "%*phN.uid.%u", FSCRYPT_KEY_IDENTIFIER_SIZE,
- mk_identifier, __kuid_val(current_fsuid()));
-}
-
/* Create ->s_master_keys if needed. Synchronized by fscrypt_add_key_mutex. */
static int allocate_filesystem_keyring(struct super_block *sb)
{
@@ -334,91 +313,94 @@ fscrypt_find_master_key(struct super_block *sb,
return mk;
}
-static int allocate_master_key_users_keyring(struct fscrypt_master_key *mk)
-{
- char description[FSCRYPT_MK_USERS_DESCRIPTION_SIZE];
- struct key *keyring;
-
- format_mk_users_keyring_description(description,
- mk->mk_spec.u.identifier);
- keyring = keyring_alloc(description, GLOBAL_ROOT_UID, GLOBAL_ROOT_GID,
- current_cred(), KEY_POS_SEARCH |
- KEY_USR_SEARCH | KEY_USR_READ | KEY_USR_VIEW,
- KEY_ALLOC_NOT_IN_QUOTA, NULL, NULL);
- if (IS_ERR(keyring))
- return PTR_ERR(keyring);
-
- mk->mk_users = keyring;
- return 0;
-}
-
-/*
- * Find the current user's "key" in the master key's ->mk_users.
- * Returns ERR_PTR(-ENOKEY) if not found.
- */
-static struct key *find_master_key_user(struct fscrypt_master_key *mk)
+/* Find the current user's claim in ->mk_users. ->mk_sem must be held. */
+static struct fscrypt_master_key_user *
+find_master_key_user(struct fscrypt_master_key *mk)
{
- char description[FSCRYPT_MK_USER_DESCRIPTION_SIZE];
- key_ref_t keyref;
-
- format_mk_user_description(description, mk->mk_spec.u.identifier);
+ struct fscrypt_master_key_user *mk_user;
+ kuid_t uid = current_fsuid();
- /*
- * We need to mark the keyring reference as "possessed" so that we
- * acquire permission to search it, via the KEY_POS_SEARCH permission.
- */
- keyref = keyring_search(make_key_ref(mk->mk_users, true /*possessed*/),
- &key_type_fscrypt_user, description, false);
- if (IS_ERR(keyref)) {
- if (PTR_ERR(keyref) == -EAGAIN || /* not found */
- PTR_ERR(keyref) == -EKEYREVOKED) /* recently invalidated */
- keyref = ERR_PTR(-ENOKEY);
- return ERR_CAST(keyref);
+ list_for_each_entry(mk_user, &mk->mk_users, link) {
+ if (uid_eq(mk_user->uid, uid))
+ return mk_user;
}
- return key_ref_to_ptr(keyref);
+ return NULL;
}
/*
- * Give the current user a "key" in ->mk_users. This charges the user's quota
+ * Give the current user a claim in ->mk_users. This charges the user's quota
* and marks the master key as added by the current user, so that it cannot be
* removed by another user with the key. Either ->mk_sem must be held for
* write, or the master key must be still undergoing initialization.
*/
static int add_master_key_user(struct fscrypt_master_key *mk)
{
+ kuid_t uid = current_fsuid();
char description[FSCRYPT_MK_USER_DESCRIPTION_SIZE];
- struct key *mk_user;
+ struct key *quota_key;
+ struct fscrypt_master_key_user *mk_user;
int err;
- format_mk_user_description(description, mk->mk_spec.u.identifier);
- mk_user = key_alloc(&key_type_fscrypt_user, description,
- current_fsuid(), current_gid(), current_cred(),
- KEY_POS_SEARCH | KEY_USR_VIEW, 0, NULL);
- if (IS_ERR(mk_user))
- return PTR_ERR(mk_user);
+ snprintf(description, sizeof(description), "%*phN.uid.%u",
+ FSCRYPT_KEY_IDENTIFIER_SIZE, mk->mk_spec.u.identifier,
+ __kuid_val(uid));
+ quota_key = key_alloc(&key_type_fscrypt_user, description, uid,
+ current_gid(), current_cred(),
+ KEY_POS_SEARCH | KEY_USR_VIEW, 0, NULL);
+ if (IS_ERR(quota_key))
+ return PTR_ERR(quota_key);
+
+ err = key_instantiate_and_link(quota_key, NULL, 0, NULL, NULL);
+ if (err) {
+ key_put(quota_key);
+ return err;
+ }
- err = key_instantiate_and_link(mk_user, NULL, 0, mk->mk_users, NULL);
- key_put(mk_user);
- return err;
+ mk_user = kzalloc_obj(*mk_user);
+ if (!mk_user) {
+ key_put(quota_key);
+ return -ENOMEM;
+ }
+ mk_user->uid = uid;
+ mk_user->quota_key = quota_key;
+ list_add(&mk_user->link, &mk->mk_users);
+ return 0;
+}
+
+static void unlink_and_free_mk_user(struct fscrypt_master_key_user *mk_user)
+{
+ list_del(&mk_user->link);
+ key_put(mk_user->quota_key);
+ kfree(mk_user);
}
/*
- * Remove the current user's "key" from ->mk_users.
+ * Remove the current user's claim from ->mk_users.
* ->mk_sem must be held for write.
*
- * Returns 0 if removed, -ENOKEY if not found, or another -errno code.
+ * Returns 0 if removed or -ENOKEY if not found.
*/
static int remove_master_key_user(struct fscrypt_master_key *mk)
{
- struct key *mk_user;
- int err;
+ struct fscrypt_master_key_user *mk_user;
mk_user = find_master_key_user(mk);
- if (IS_ERR(mk_user))
- return PTR_ERR(mk_user);
- err = key_unlink(mk->mk_users, mk_user);
- key_put(mk_user);
- return err;
+ if (!mk_user)
+ return -ENOKEY;
+ unlink_and_free_mk_user(mk_user);
+ return 0;
+}
+
+/*
+ * Clear ->mk_users. Either ->mk_sem must be held for write, or 'mk' must have
+ * no structural references left.
+ */
+static void clear_mk_users(struct fscrypt_master_key *mk)
+{
+ struct fscrypt_master_key_user *mk_user, *tmp;
+
+ list_for_each_entry_safe(mk_user, tmp, &mk->mk_users, link)
+ unlink_and_free_mk_user(mk_user);
}
/*
@@ -441,13 +423,14 @@ static int add_new_master_key(struct super_block *sb,
refcount_set(&mk->mk_struct_refs, 1);
mk->mk_spec = *mk_spec;
+ INIT_LIST_HEAD(&mk->mk_users);
+
INIT_LIST_HEAD(&mk->mk_decrypted_inodes);
spin_lock_init(&mk->mk_decrypted_inodes_lock);
+ INIT_LIST_HEAD(&mk->mk_mode_keys);
+
if (mk_spec->type == FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER) {
- err = allocate_master_key_users_keyring(mk);
- if (err)
- goto out_put;
err = add_master_key_user(mk);
if (err)
goto out_put;
@@ -476,19 +459,13 @@ static int add_existing_master_key(struct fscrypt_master_key *mk,
int err;
/*
- * If the current user is already in ->mk_users, then there's nothing to
- * do. Otherwise, we need to add the user to ->mk_users. (Neither is
- * applicable for v1 policy keys, which have NULL ->mk_users.)
+ * For v2 policy keys (FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER): If the current
+ * user is already in ->mk_users, then there's nothing to do.
+ * Otherwise, add the user to ->mk_users.
*/
- if (mk->mk_users) {
- struct key *mk_user = find_master_key_user(mk);
-
- if (mk_user != ERR_PTR(-ENOKEY)) {
- if (IS_ERR(mk_user))
- return PTR_ERR(mk_user);
- key_put(mk_user);
+ if (mk->mk_spec.type == FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER) {
+ if (find_master_key_user(mk) != NULL)
return 0;
- }
err = add_master_key_user(mk);
if (err)
return err;
@@ -842,7 +819,6 @@ int fscrypt_verify_key_added(struct super_block *sb,
{
struct fscrypt_key_specifier mk_spec;
struct fscrypt_master_key *mk;
- struct key *mk_user;
int err;
mk_spec.type = FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER;
@@ -854,13 +830,10 @@ int fscrypt_verify_key_added(struct super_block *sb,
goto out;
}
down_read(&mk->mk_sem);
- mk_user = find_master_key_user(mk);
- if (IS_ERR(mk_user)) {
- err = PTR_ERR(mk_user);
- } else {
- key_put(mk_user);
+ if (find_master_key_user(mk) != NULL)
err = 0;
- }
+ else
+ err = -ENOKEY;
up_read(&mk->mk_sem);
fscrypt_put_master_key(mk);
out:
@@ -1052,16 +1025,18 @@ static int do_remove_key(struct file *filp, void __user *_uarg, bool all_users)
down_write(&mk->mk_sem);
/* If relevant, remove current user's (or all users) claim to the key */
- if (mk->mk_users && mk->mk_users->keys.nr_leaves_on_tree != 0) {
- if (all_users)
- err = keyring_clear(mk->mk_users);
- else
+ if (!list_empty(&mk->mk_users)) {
+ if (all_users) {
+ clear_mk_users(mk);
+ err = 0;
+ } else {
err = remove_master_key_user(mk);
+ }
if (err) {
up_write(&mk->mk_sem);
goto out_put_key;
}
- if (mk->mk_users->keys.nr_leaves_on_tree != 0) {
+ if (!list_empty(&mk->mk_users)) {
/*
* Other users have still added the key too. We removed
* the current user's claim to the key, but we still
@@ -1147,6 +1122,8 @@ int fscrypt_ioctl_get_key_status(struct file *filp, void __user *uarg)
struct super_block *sb = file_inode(filp)->i_sb;
struct fscrypt_get_key_status_arg arg;
struct fscrypt_master_key *mk;
+ kuid_t uid;
+ const struct fscrypt_master_key_user *mk_user;
int err;
if (copy_from_user(&arg, uarg, sizeof(arg)))
@@ -1179,19 +1156,13 @@ int fscrypt_ioctl_get_key_status(struct file *filp, void __user *uarg)
}
arg.status = FSCRYPT_KEY_STATUS_PRESENT;
- if (mk->mk_users) {
- struct key *mk_user;
- arg.user_count = mk->mk_users->keys.nr_leaves_on_tree;
- mk_user = find_master_key_user(mk);
- if (!IS_ERR(mk_user)) {
+ uid = current_fsuid();
+ list_for_each_entry(mk_user, &mk->mk_users, link) {
+ arg.user_count++;
+ if (uid_eq(mk_user->uid, uid))
arg.status_flags |=
FSCRYPT_KEY_STATUS_FLAG_ADDED_BY_SELF;
- key_put(mk_user);
- } else if (mk_user != ERR_PTR(-ENOKEY)) {
- err = PTR_ERR(mk_user);
- goto out_release_key;
- }
}
err = 0;
out_release_key:
diff --git a/fs/crypto/keysetup.c b/fs/crypto/keysetup.c
index 2896046a4977..71586440d043 100644
--- a/fs/crypto/keysetup.c
+++ b/fs/crypto/keysetup.c
@@ -159,13 +159,7 @@ int fscrypt_prepare_key(struct fscrypt_prepared_key *prep_key,
tfm = fscrypt_allocate_skcipher(ci->ci_mode, raw_key, ci->ci_inode);
if (IS_ERR(tfm))
return PTR_ERR(tfm);
- /*
- * Pairs with the smp_load_acquire() in fscrypt_is_key_prepared().
- * I.e., here we publish ->tfm with a RELEASE barrier so that
- * concurrent tasks can ACQUIRE it. Note that this concurrency is only
- * possible for per-mode keys, not for per-file keys.
- */
- smp_store_release(&prep_key->tfm, tfm);
+ prep_key->tfm = tfm;
return 0;
}
@@ -186,9 +180,37 @@ int fscrypt_set_per_file_enc_key(struct fscrypt_inode_info *ci,
return fscrypt_prepare_key(&ci->ci_enc_key, raw_key, ci);
}
+/*
+ * Find the fscrypt_prepared_key (if any) for a particular (mk, hkdf_context,
+ * mode_num, data_unit_bits, inlinecrypt) combination.
+ *
+ * The caller must hold ->mk_sem for reading and ->mk_present must be true,
+ * ensuring that ->mk_mode_keys is still append-only.
+ */
+static struct fscrypt_prepared_key *
+fscrypt_find_mode_key(struct fscrypt_master_key *mk, u8 hkdf_context,
+ u8 mode_num, const struct fscrypt_inode_info *ci)
+{
+ struct fscrypt_mode_key *node;
+
+ /*
+ * The RCU read lock here is used only to synchronize with concurrent
+ * list_add_tail_rcu(). Concurrent deletions are impossible here, so
+ * returning a pointer to a node without taking any refcount is safe.
+ */
+ guard(rcu)();
+ list_for_each_entry_rcu(node, &mk->mk_mode_keys, link) {
+ if (node->hkdf_context == hkdf_context &&
+ node->mode_num == mode_num &&
+ node->data_unit_bits == ci->ci_data_unit_bits &&
+ fscrypt_is_key_prepared(&node->key, ci))
+ return &node->key;
+ }
+ return NULL;
+}
+
static int setup_per_mode_enc_key(struct fscrypt_inode_info *ci,
struct fscrypt_master_key *mk,
- struct fscrypt_prepared_key *keys,
u8 hkdf_context, bool include_fs_uuid)
{
const struct inode *inode = ci->ci_inode;
@@ -196,7 +218,8 @@ static int setup_per_mode_enc_key(struct fscrypt_inode_info *ci,
struct fscrypt_mode *mode = ci->ci_mode;
const u8 mode_num = mode - fscrypt_modes;
struct fscrypt_prepared_key *prep_key;
- u8 mode_key[FSCRYPT_MAX_KEY_SIZE];
+ struct fscrypt_mode_key *new_node;
+ u8 raw_mode_key[FSCRYPT_MAX_RAW_KEY_SIZE];
u8 hkdf_info[sizeof(mode_num) + sizeof(sb->s_uuid)];
unsigned int hkdf_infolen = 0;
int err;
@@ -204,41 +227,52 @@ static int setup_per_mode_enc_key(struct fscrypt_inode_info *ci,
if (WARN_ON_ONCE(mode_num > FSCRYPT_MODE_MAX))
return -EINVAL;
- prep_key = &keys[mode_num];
- if (fscrypt_is_key_prepared(prep_key, ci)) {
+ prep_key = fscrypt_find_mode_key(mk, hkdf_context, mode_num, ci);
+ if (prep_key) {
ci->ci_enc_key = *prep_key;
return 0;
}
- mutex_lock(&fscrypt_mode_key_setup_mutex);
+ guard(mutex)(&fscrypt_mode_key_setup_mutex);
- if (fscrypt_is_key_prepared(prep_key, ci))
- goto done_unlock;
+ prep_key = fscrypt_find_mode_key(mk, hkdf_context, mode_num, ci);
+ if (prep_key) {
+ ci->ci_enc_key = *prep_key;
+ return 0;
+ }
- BUILD_BUG_ON(sizeof(mode_num) != 1);
- BUILD_BUG_ON(sizeof(sb->s_uuid) != 16);
- BUILD_BUG_ON(sizeof(hkdf_info) != 17);
- hkdf_info[hkdf_infolen++] = mode_num;
- if (include_fs_uuid) {
- memcpy(&hkdf_info[hkdf_infolen], &sb->s_uuid,
- sizeof(sb->s_uuid));
- hkdf_infolen += sizeof(sb->s_uuid);
+ new_node = kzalloc_obj(*new_node);
+ if (!new_node)
+ return -ENOMEM;
+ new_node->hkdf_context = hkdf_context;
+ new_node->mode_num = mode_num;
+ new_node->data_unit_bits = ci->ci_data_unit_bits;
+ prep_key = &new_node->key;
+
+ {
+ static_assert(sizeof(mode_num) == 1);
+ static_assert(sizeof(sb->s_uuid) == 16);
+ static_assert(sizeof(hkdf_info) == 17);
+ hkdf_info[hkdf_infolen++] = mode_num;
+ if (include_fs_uuid) {
+ memcpy(&hkdf_info[hkdf_infolen], &sb->s_uuid,
+ sizeof(sb->s_uuid));
+ hkdf_infolen += sizeof(sb->s_uuid);
+ }
+ err = fscrypt_hkdf_expand(&mk->mk_secret.hkdf, hkdf_context,
+ hkdf_info, hkdf_infolen, raw_mode_key,
+ mode->keysize);
+ if (!err)
+ err = fscrypt_prepare_key(prep_key, raw_mode_key, ci);
+ memzero_explicit(raw_mode_key, mode->keysize);
}
- err = fscrypt_hkdf_expand(&mk->mk_secret.hkdf,
- hkdf_context, hkdf_info, hkdf_infolen,
- mode_key, mode->keysize);
- if (err)
- goto out_unlock;
- err = fscrypt_prepare_key(prep_key, mode_key, ci);
- memzero_explicit(mode_key, mode->keysize);
- if (err)
- goto out_unlock;
-done_unlock:
+ if (err) {
+ kfree(new_node);
+ return err;
+ }
+ list_add_tail_rcu(&new_node->link, &mk->mk_mode_keys);
ci->ci_enc_key = *prep_key;
- err = 0;
-out_unlock:
- mutex_unlock(&fscrypt_mode_key_setup_mutex);
- return err;
+ return 0;
}
/*
@@ -296,8 +330,8 @@ static int fscrypt_setup_iv_ino_lblk_32_key(struct fscrypt_inode_info *ci,
{
int err;
- err = setup_per_mode_enc_key(ci, mk, mk->mk_iv_ino_lblk_32_keys,
- HKDF_CONTEXT_IV_INO_LBLK_32_KEY, true);
+ err = setup_per_mode_enc_key(ci, mk, HKDF_CONTEXT_IV_INO_LBLK_32_KEY,
+ true);
if (err)
return err;
@@ -346,8 +380,8 @@ static int fscrypt_setup_v2_file_key(struct fscrypt_inode_info *ci,
* encryption key. This ensures that the master key is
* consistently used only for HKDF, avoiding key reuse issues.
*/
- err = setup_per_mode_enc_key(ci, mk, mk->mk_direct_keys,
- HKDF_CONTEXT_DIRECT_KEY, false);
+ err = setup_per_mode_enc_key(ci, mk, HKDF_CONTEXT_DIRECT_KEY,
+ false);
} else if (ci->ci_policy.v2.flags &
FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64) {
/*
@@ -356,9 +390,8 @@ static int fscrypt_setup_v2_file_key(struct fscrypt_inode_info *ci,
* the IVs. This format is optimized for use with inline
* encryption hardware compliant with the UFS standard.
*/
- err = setup_per_mode_enc_key(ci, mk, mk->mk_iv_ino_lblk_64_keys,
- HKDF_CONTEXT_IV_INO_LBLK_64_KEY,
- true);
+ err = setup_per_mode_enc_key(
+ ci, mk, HKDF_CONTEXT_IV_INO_LBLK_64_KEY, true);
} else if (ci->ci_policy.v2.flags &
FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32) {
err = fscrypt_setup_iv_ino_lblk_32_key(ci, mk);
diff --git a/fs/dlm/midcomms.c b/fs/dlm/midcomms.c
index 2c101bbe261a..2015a5f40a14 100644
--- a/fs/dlm/midcomms.c
+++ b/fs/dlm/midcomms.c
@@ -968,10 +968,10 @@ static void midcomms_new_msg_cb(void *data)
atomic_inc(&mh->node->send_queue_cnt);
spin_lock_bh(&mh->node->send_queue_lock);
+ /* need to be locked with list_add_tail_rcu() because list is ordered */
+ mh->seq = atomic_fetch_inc(&mh->node->seq_send);
list_add_tail_rcu(&mh->list, &mh->node->send_queue);
spin_unlock_bh(&mh->node->send_queue_lock);
-
- mh->seq = atomic_fetch_inc(&mh->node->seq_send);
}
static struct dlm_msg *dlm_midcomms_get_msg_3_2(struct dlm_mhandle *mh, int nodeid,
diff --git a/fs/efs/file.c b/fs/efs/file.c
index 9e641da6fab2..9153dfe79bbc 100644
--- a/fs/efs/file.c
+++ b/fs/efs/file.c
@@ -18,16 +18,9 @@ int efs_get_block(struct inode *inode, sector_t iblock,
if (create)
return error;
- if (iblock >= inode->i_blocks) {
-#ifdef DEBUG
- /*
- * i have no idea why this happens as often as it does
- */
- pr_warn("%s(): block %d >= %ld (filesize %ld)\n",
- __func__, block, inode->i_blocks, inode->i_size);
-#endif
+ if (iblock >= inode->i_blocks)
return 0;
- }
+
phys = efs_map_block(inode, iblock);
if (phys)
map_bh(bh_result, inode->i_sb, phys);
@@ -42,16 +35,8 @@ int efs_bmap(struct inode *inode, efs_block_t block) {
}
/* are we about to read past the end of a file ? */
- if (!(block < inode->i_blocks)) {
-#ifdef DEBUG
- /*
- * i have no idea why this happens as often as it does
- */
- pr_warn("%s(): block %d >= %ld (filesize %ld)\n",
- __func__, block, inode->i_blocks, inode->i_size);
-#endif
+ if (!(block < inode->i_blocks))
return 0;
- }
return efs_map_block(inode, block);
}
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 22605fbc12de..5a19edda2b77 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -38,48 +38,174 @@
#include <linux/compat.h>
#include <linux/rculist.h>
#include <linux/capability.h>
+#include <linux/seqlock.h>
#include <net/busy_poll.h>
/*
- * LOCKING:
- * There are three level of locking required by epoll :
+ * fs/eventpoll.c - Efficient event polling ("epoll") kernel implementation.
*
- * 1) epnested_mutex (mutex)
- * 2) ep->mtx (mutex)
- * 3) ep->lock (spinlock)
*
- * The acquire order is the one listed above, from 1 to 3.
- * We need a spinlock (ep->lock) because we manipulate objects
- * from inside the poll callback, that might be triggered from
- * a wake_up() that in turn might be called from IRQ context.
- * So we can't sleep inside the poll callback and hence we need
- * a spinlock. During the event transfer loop (from kernel to
- * user space) we could end up sleeping due a copy_to_user(), so
- * we need a lock that will allow us to sleep. This lock is a
- * mutex (ep->mtx). It is acquired during the event transfer loop,
- * during epoll_ctl(EPOLL_CTL_DEL) and during eventpoll_release_file().
- * The epnested_mutex is acquired when inserting an epoll fd onto another
- * epoll fd. We do this so that we walk the epoll tree and ensure that this
- * insertion does not create a cycle of epoll file descriptors, which
- * could lead to deadlock. We need a global mutex to prevent two
- * simultaneous inserts (A into B and B into A) from racing and
- * constructing a cycle without either insert observing that it is
- * going to.
- * It is necessary to acquire multiple "ep->mtx"es at once in the
- * case when one epoll fd is added to another. In this case, we
- * always acquire the locks in the order of nesting (i.e. after
- * epoll_ctl(e1, EPOLL_CTL_ADD, e2), e1->mtx will always be acquired
- * before e2->mtx). Since we disallow cycles of epoll file
- * descriptors, this ensures that the mutexes are well-ordered. In
- * order to communicate this nesting to lockdep, when walking a tree
- * of epoll file descriptors, we use the current recursion depth as
- * the lockdep subkey.
- * It is possible to drop the "ep->mtx" and to use the global
- * mutex "epnested_mutex" (together with "ep->lock") to have it working,
- * but having "ep->mtx" will make the interface more scalable.
- * Events that require holding "epnested_mutex" are very rare, while for
- * normal operations the epoll private "ep->mtx" will guarantee
- * a better scalability.
+ * Overview
+ * --------
+ *
+ * Each epoll_create(2) returns an anonymous [eventpoll] file whose
+ * ->private_data is a struct eventpoll. Each EPOLL_CTL_ADD installs
+ * a struct epitem linking one (watched file, fd) pair back to that
+ * eventpoll via the watched file's f_op->poll() wait queue(s). When
+ * the watched file signals readiness, ep_poll_callback() fires and
+ * marks the epitem ready. epoll_wait(2) drains the ready list under
+ * ep->mtx, re-queueing items in level-triggered mode.
+ *
+ * epoll instances can watch other epoll instances up to EP_MAX_NESTS
+ * deep; cycles are forbidden and detected at EPOLL_CTL_ADD time.
+ *
+ *
+ * Locking
+ * -------
+ *
+ * Three levels, acquired from outer to inner:
+ *
+ * epnested_mutex (global; rare; taken only for EPOLL_CTL_ADD
+ * loop / path checks)
+ * > ep->mtx (per-eventpoll; sleepable; serializes most ops)
+ * > ep->lock (per-eventpoll; IRQ-safe spinlock)
+ *
+ * file->f_lock (per-file; NOT IRQ-safe; guards f_ep hlist ops;
+ * nested inside ep->mtx, outside ep->lock)
+ *
+ * Rationale:
+ * - ep->lock is a spinlock because ep_poll_callback() is called from
+ * wake_up() which may run in hard-IRQ context. All ep->lock
+ * critical sections use spin_lock_irqsave().
+ * - ep->mtx is a sleepable mutex because the event delivery loop
+ * calls copy_to_user(), and ep_insert() may sleep in
+ * kmem_cache_alloc() and f_op->poll().
+ * - epnested_mutex is global because cycle detection needs a global
+ * view of the epoll topology; a per-object scheme would let two
+ * concurrent inserts (A into B, B into A) construct a cycle
+ * without either observer seeing it.
+ * - Per-ep ep->mtx is preferred for scalability elsewhere. Events
+ * that require epnested_mutex are rare.
+ *
+ * When EPOLL_CTL_ADD nests one eventpoll inside another we acquire
+ * ep->mtx on both: outer first, target second. Since cycles are
+ * forbidden the set of live ep->mtx holds is always a strict chain,
+ * communicated to lockdep via mutex_lock_nested() subclasses derived
+ * from the current recursion depth.
+ *
+ *
+ * Field protection
+ * ----------------
+ *
+ * struct eventpoll:
+ * mtx - self
+ * rbr - ep->mtx
+ * ovflist, rdllist - ep->lock (IRQ-safe)
+ * wq - ep->lock for queue mutation
+ * poll_wait - internal waitqueue spinlock
+ * refs - file->f_lock for adds; ep->mtx for removes;
+ * RCU for readers (hlist_del_rcu + kfree_rcu(ep))
+ * ws - ep->mtx
+ * gen, loop_check_depth - epnested_mutex
+ * file, user - immutable after setup
+ * refcount - atomic (refcount_t)
+ * napi_* - READ_ONCE / WRITE_ONCE
+ *
+ * struct epitem:
+ * rbn / rcu union - rbn: ep->mtx (while epi is linked in ep->rbr).
+ * rcu: written only by kfree_rcu(epi) on the free
+ * path; otherwise untouched by epoll code.
+ * rdllink, next - ep->lock
+ * ffd, ep - immutable after ep_insert()
+ * pwqlist - ep->mtx for writes; POLLFREE clears pwq->whead
+ * via smp_store_release(), see below
+ * fllink - file->f_lock for mutation; hlist_del_rcu +
+ * kfree_rcu(epi) for safe RCU readers
+ * ws - RCU (rcu_assign_pointer /
+ * rcu_dereference_check(mtx))
+ * event - ep->mtx for writes; lockless read in
+ * ep_poll_callback pairs with smp_mb() in
+ * ep_modify()
+ *
+ *
+ * Ready-list state machine
+ * ------------------------
+ *
+ * Readiness is tracked in two lists under ep->lock:
+ *
+ * rdllist - doubly-linked FIFO; the "current" ready list.
+ * ovflist - singly-linked LIFO; used during a scan to catch
+ * events that arrive while rdllist is being iterated
+ * without ep->lock.
+ *
+ * Encoded in ep->ovflist:
+ * EP_UNACTIVE_PTR - no scan active; callback appends to rdllist.
+ * NULL - scan active, no spill yet.
+ * pointer to epi - scan active with spilled items (LIFO).
+ *
+ * Encoded in epi->ovflist_next:
+ * EP_UNACTIVE_PTR - epi is not on ovflist.
+ * otherwise - next epi on ovflist (NULL at tail).
+ *
+ * ep_start_scan() flips "not scanning" to "scanning" and splices
+ * rdllist into a caller-local scan_batch. ep_done_scan() drains ovflist
+ * back to rdllist (list_add head-insert reverses LIFO to FIFO),
+ * flips back to "not scanning", and re-splices any items the caller
+ * left in scan_batch (e.g., level-triggered re-queues).
+ *
+ *
+ * Removal paths
+ * -------------
+ *
+ * Three paths dispose of epitems and/or eventpolls:
+ *
+ * A. ep_remove() - EPOLL_CTL_DEL and ep_insert()
+ * rollback. Caller holds ep->mtx.
+ * B. ep_clear_and_put() - close of the epoll fd itself
+ * (ep_eventpoll_release).
+ * C. eventpoll_release_file() - close of a watched file, invoked
+ * from __fput().
+ *
+ * Coordination:
+ * A and C exclude each other via the watched file's refcount.
+ * A pins the file with epi_fget() before touching file->f_ep or
+ * file->f_lock; if the pin fails, __fput() is in flight and C
+ * will clean this epi up. See the epi_fget() block comment.
+ * A and B both hold ep->mtx serially. B walks the rbtree with
+ * rb_next() captured before ep_remove() erases the current node.
+ * B and C both take ep->mtx; the loser sees fewer entries or an
+ * empty file->f_ep.
+ *
+ * Within every path the internal order is strict:
+ * ep_unregister_pollwait() - drain pwqlist; synchronizes with any
+ * in-flight ep_poll_callback via the
+ * watched wait-queue head's lock.
+ * ep_remove_file() - hlist_del_rcu of epi->fllink and,
+ * if last watcher, clear file->f_ep,
+ * under file->f_lock.
+ * ep_remove_epi() - rb_erase, rdllist unlink (ep->lock),
+ * wakeup_source_unregister,
+ * kfree_rcu(epi).
+ *
+ * kfree_rcu(epi) defers the free past RCU readers in
+ * reverse_path_check_proc(); kfree_rcu(ep) defers past readers in
+ * ep_get_upwards_depth_proc().
+ *
+ *
+ * POLLFREE handshake
+ * ------------------
+ *
+ * When a subsystem tears down a wait-queue head that an epitem is
+ * registered on (binder, signalfd, ...), it wakes the callback with
+ * POLLFREE and must RCU-defer the head's free. The store/load pair:
+ *
+ * ep_poll_callback() POLLFREE branch:
+ * smp_store_release(&pwq->whead, NULL)
+ *
+ * ep_remove_wait_queue():
+ * smp_load_acquire(&pwq->whead)
+ *
+ * See those sites for the full argument.
*/
/* Epoll private bits inside the event mask */
@@ -136,14 +262,16 @@ struct epitem {
struct rcu_head rcu;
};
- /* List header used to link this structure to the eventpoll ready list */
+ /* Link on the owning eventpoll's ready list (ep->rdllist). */
struct list_head rdllink;
/*
- * Works together "struct eventpoll"->ovflist in keeping the
- * single linked chain of items.
+ * Link on the owning eventpoll's scan-overflow list (ep->ovflist),
+ * EP_UNACTIVE_PTR when not linked. See epi_on_ovflist() /
+ * epi_clear_ovflist() and the "Ready-list state machine" section
+ * in the top-of-file banner.
*/
- struct epitem *next;
+ struct epitem *ovflist_next;
/* The file descriptor information this item refers to */
struct epoll_filefd ffd;
@@ -197,6 +325,9 @@ struct eventpoll {
/* Lock which protects rdllist and ovflist */
spinlock_t lock;
+ /* Protect switching between rdllist and ovflist */
+ seqcount_spinlock_t seq;
+
/* RB tree root used to store monitored fd structs */
struct rb_root_cached rbr;
@@ -382,6 +513,43 @@ static inline struct epitem *ep_item_from_wait(wait_queue_entry_t *p)
return container_of(p, struct eppoll_entry, wait)->base;
}
+/*
+ * Ready-list / ovflist state (see "Ready-list state machine" in the
+ * top-of-file banner for the full state machine). EP_UNACTIVE_PTR is
+ * the sentinel; these wrappers name each transition and each test so
+ * call sites do not need to know the sentinel's value.
+ */
+
+/* True iff @ep is between ep_enter_scan() and ep_exit_scan(). */
+static inline bool ep_is_scanning(struct eventpoll *ep)
+{
+ return READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR;
+}
+
+/* Called by ep_start_scan(): divert ep_poll_callback() to ovflist. */
+static inline void ep_enter_scan(struct eventpoll *ep)
+{
+ WRITE_ONCE(ep->ovflist, NULL);
+}
+
+/* Called by ep_done_scan(): redirect ep_poll_callback() back to rdllist. */
+static inline void ep_exit_scan(struct eventpoll *ep)
+{
+ WRITE_ONCE(ep->ovflist, EP_UNACTIVE_PTR);
+}
+
+/* True iff @epi is currently linked on its ep's ovflist. */
+static inline bool epi_on_ovflist(const struct epitem *epi)
+{
+ return epi->ovflist_next != EP_UNACTIVE_PTR;
+}
+
+/* Mark @epi as not on any ovflist (init and post-drain). */
+static inline void epi_clear_ovflist(struct epitem *epi)
+{
+ epi->ovflist_next = EP_UNACTIVE_PTR;
+}
+
/**
* ep_events_available - Checks if ready events might be available.
*
@@ -392,8 +560,10 @@ static inline struct epitem *ep_item_from_wait(wait_queue_entry_t *p)
*/
static inline int ep_events_available(struct eventpoll *ep)
{
- return !list_empty_careful(&ep->rdllist) ||
- READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR;
+ unsigned int seq = read_seqcount_begin(&ep->seq);
+
+ return !list_empty_careful(&ep->rdllist) || ep_is_scanning(ep) ||
+ read_seqcount_retry(&ep->seq, seq);
}
#ifdef CONFIG_NET_RX_BUSY_POLL
@@ -705,7 +875,7 @@ static inline void ep_pm_stay_awake_rcu(struct epitem *epi)
* ep->mutex needs to be held because we could be hit by
* eventpoll_release_file() and epoll_ctl().
*/
-static void ep_start_scan(struct eventpoll *ep, struct list_head *txlist)
+static void ep_start_scan(struct eventpoll *ep, struct list_head *scan_batch)
{
/*
* Steal the ready list, and re-init the original one to the
@@ -717,13 +887,17 @@ static void ep_start_scan(struct eventpoll *ep, struct list_head *txlist)
*/
lockdep_assert_irqs_enabled();
spin_lock_irq(&ep->lock);
- list_splice_init(&ep->rdllist, txlist);
- WRITE_ONCE(ep->ovflist, NULL);
+ write_seqcount_begin(&ep->seq);
+
+ list_splice_init(&ep->rdllist, scan_batch);
+ ep_enter_scan(ep);
+
+ write_seqcount_end(&ep->seq);
spin_unlock_irq(&ep->lock);
}
static void ep_done_scan(struct eventpoll *ep,
- struct list_head *txlist)
+ struct list_head *scan_batch)
{
struct epitem *epi, *nepi;
@@ -733,34 +907,35 @@ static void ep_done_scan(struct eventpoll *ep,
* other events might have been queued by the poll callback.
* We re-insert them inside the main ready-list here.
*/
- for (nepi = READ_ONCE(ep->ovflist); (epi = nepi) != NULL;
- nepi = epi->next, epi->next = EP_UNACTIVE_PTR) {
+ for (nepi = READ_ONCE(ep->ovflist); (epi = nepi) != NULL; ) {
+ nepi = epi->ovflist_next;
+ epi_clear_ovflist(epi);
/*
- * We need to check if the item is already in the list.
- * During the "sproc" callback execution time, items are
- * queued into ->ovflist but the "txlist" might already
- * contain them, and the list_splice() below takes care of them.
+ * Skip items that the caller already returned via @scan_batch
+ * -- the list_splice() below takes care of those.
*/
if (!ep_is_linked(epi)) {
/*
- * ->ovflist is LIFO, so we have to reverse it in order
- * to keep in FIFO.
+ * ovflist is LIFO; list_add() head-insert here
+ * reverses the iteration order into FIFO.
*/
list_add(&epi->rdllink, &ep->rdllist);
ep_pm_stay_awake(epi);
}
}
- /*
- * We need to set back ep->ovflist to EP_UNACTIVE_PTR, so that after
- * releasing the lock, events will be queued in the normal way inside
- * ep->rdllist.
- */
- WRITE_ONCE(ep->ovflist, EP_UNACTIVE_PTR);
+
+ write_seqcount_begin(&ep->seq);
+
+ /* Back out of scan mode; callbacks target ep->rdllist again. */
+ ep_exit_scan(ep);
/*
- * Quickly re-inject items left on "txlist".
+ * Quickly re-inject items left on "scan_batch".
*/
- list_splice(txlist, &ep->rdllist);
+ list_splice(scan_batch, &ep->rdllist);
+
+ write_seqcount_end(&ep->seq);
+
__pm_relax(ep->ws);
if (!list_empty(&ep->rdllist)) {
@@ -986,7 +1161,7 @@ static __poll_t ep_item_poll(const struct epitem *epi, poll_table *pt, int depth
static __poll_t __ep_eventpoll_poll(struct file *file, poll_table *wait, int depth)
{
struct eventpoll *ep = file->private_data;
- LIST_HEAD(txlist);
+ LIST_HEAD(scan_batch);
struct epitem *epi, *tmp;
poll_table pt;
__poll_t res = 0;
@@ -1001,8 +1176,8 @@ static __poll_t __ep_eventpoll_poll(struct file *file, poll_table *wait, int dep
* the ready list.
*/
mutex_lock_nested(&ep->mtx, depth);
- ep_start_scan(ep, &txlist);
- list_for_each_entry_safe(epi, tmp, &txlist, rdllink) {
+ ep_start_scan(ep, &scan_batch);
+ list_for_each_entry_safe(epi, tmp, &scan_batch, rdllink) {
if (ep_item_poll(epi, &pt, depth + 1)) {
res = EPOLLIN | EPOLLRDNORM;
break;
@@ -1016,7 +1191,7 @@ static __poll_t __ep_eventpoll_poll(struct file *file, poll_table *wait, int dep
list_del_init(&epi->rdllink);
}
}
- ep_done_scan(ep, &txlist);
+ ep_done_scan(ep, &scan_batch);
mutex_unlock(&ep->mtx);
return res;
}
@@ -1144,11 +1319,12 @@ static int ep_alloc(struct eventpoll **pep)
mutex_init(&ep->mtx);
spin_lock_init(&ep->lock);
+ seqcount_spinlock_init(&ep->seq, &ep->lock);
init_waitqueue_head(&ep->wq);
init_waitqueue_head(&ep->poll_wait);
INIT_LIST_HEAD(&ep->rdllist);
ep->rbr = RB_ROOT_CACHED;
- ep->ovflist = EP_UNACTIVE_PTR;
+ ep->ovflist = EP_UNACTIVE_PTR; /* not scanning */
ep->user = get_current_user();
refcount_set(&ep->refcount, 1);
@@ -1272,9 +1448,9 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v
* semantics). All the events that happen during that period of time are
* chained in ep->ovflist and requeued later on.
*/
- if (READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR) {
- if (epi->next == EP_UNACTIVE_PTR) {
- epi->next = READ_ONCE(ep->ovflist);
+ if (ep_is_scanning(ep)) {
+ if (!epi_on_ovflist(epi)) {
+ epi->ovflist_next = READ_ONCE(ep->ovflist);
WRITE_ONCE(ep->ovflist, epi);
ep_pm_stay_awake_rcu(epi);
}
@@ -1515,7 +1691,7 @@ static noinline void ep_destroy_wakeup_source(struct epitem *epi)
wakeup_source_unregister(ws);
}
-static int attach_epitem(struct file *file, struct epitem *epi)
+static int ep_attach_file(struct file *file, struct epitem *epi)
{
struct epitems_head *to_free = NULL;
struct hlist_head *head = NULL;
@@ -1550,68 +1726,112 @@ static int attach_epitem(struct file *file, struct epitem *epi)
}
/*
- * Must be called with "mtx" held.
+ * Charge the user's epoll_watches quota, allocate a fresh epitem for
+ * @tfile/@fd, and initialize its fields. The returned item is not yet
+ * linked into any data structure; the caller must install it via
+ * ep_register_epitem() (which takes over on success) or kmem_cache_free()
+ * it and decrement epoll_watches on its own.
+ *
+ * Returns ERR_PTR(-ENOSPC) if the quota is exceeded, ERR_PTR(-ENOMEM)
+ * if the slab allocation fails.
*/
-static int ep_insert(struct eventpoll *ep, const struct epoll_event *event,
- struct file *tfile, int fd, int full_check)
+static struct epitem *ep_alloc_epitem(struct eventpoll *ep,
+ const struct epoll_event *event,
+ struct file *tfile, int fd)
{
- int error, pwake = 0;
- __poll_t revents;
struct epitem *epi;
- struct ep_pqueue epq;
- struct eventpoll *tep = NULL;
-
- if (is_file_epoll(tfile))
- tep = tfile->private_data;
-
- lockdep_assert_irqs_enabled();
if (unlikely(percpu_counter_compare(&ep->user->epoll_watches,
max_user_watches) >= 0))
- return -ENOSPC;
+ return ERR_PTR(-ENOSPC);
percpu_counter_inc(&ep->user->epoll_watches);
- if (!(epi = kmem_cache_zalloc(epi_cache, GFP_KERNEL))) {
+ epi = kmem_cache_zalloc(epi_cache, GFP_KERNEL);
+ if (unlikely(!epi)) {
percpu_counter_dec(&ep->user->epoll_watches);
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
- /* Item initialization follow here ... */
INIT_LIST_HEAD(&epi->rdllink);
epi->ep = ep;
ep_set_ffd(&epi->ffd, tfile, fd);
epi->event = *event;
- epi->next = EP_UNACTIVE_PTR;
+ epi_clear_ovflist(epi);
+
+ return epi;
+}
+
+/*
+ * Install @epi into its target file's f_ep hlist and into @ep's rbtree,
+ * taking one additional reference on @ep for the lifetime of the item.
+ *
+ * If @tep is non-NULL, the target file is itself an eventpoll; we hold
+ * tep->mtx at subclass 1 across the attach + rbtree insert to serialize
+ * with the target side. RB tree ops are protected by @ep->mtx, which
+ * the caller already holds.
+ *
+ * On failure the epi is freed and the epoll_watches counter decremented,
+ * matching ep_alloc_epitem()'s allocation. After this returns
+ * successfully, ep_insert()'s later error paths use ep_remove() for
+ * unwind; that cannot drop @ep's refcount to zero because the ep file
+ * itself still holds the original reference.
+ */
+static int ep_register_epitem(struct eventpoll *ep, struct epitem *epi,
+ struct eventpoll *tep, int full_check)
+{
+ struct file *tfile = epi->ffd.file;
+ int error;
if (tep)
mutex_lock_nested(&tep->mtx, 1);
- /* Add the current item to the list of active epoll hook for this file */
- if (unlikely(attach_epitem(tfile, epi) < 0)) {
+
+ error = ep_attach_file(tfile, epi);
+ if (unlikely(error)) {
if (tep)
mutex_unlock(&tep->mtx);
kmem_cache_free(epi_cache, epi);
percpu_counter_dec(&ep->user->epoll_watches);
- return -ENOMEM;
+ return error;
}
if (full_check && !tep)
list_file(tfile);
- /*
- * Add the current item to the RB tree. All RB tree operations are
- * protected by "mtx", and ep_insert() is called with "mtx" held.
- */
ep_rbtree_insert(ep, epi);
+
if (tep)
mutex_unlock(&tep->mtx);
- /*
- * ep_remove() calls in the later error paths can't lead to
- * ep_free() as the ep file itself still holds an ep reference.
- */
ep_get(ep);
+ return 0;
+}
+
+/*
+ * Must be called with "mtx" held.
+ */
+static int ep_insert(struct eventpoll *ep, const struct epoll_event *event,
+ struct file *tfile, int fd, int full_check)
+{
+ int error, pwake = 0;
+ __poll_t revents;
+ struct epitem *epi;
+ struct ep_pqueue epq;
+ struct eventpoll *tep = NULL;
- /* now check if we've created too many backpaths */
+ if (is_file_epoll(tfile))
+ tep = tfile->private_data;
+
+ lockdep_assert_irqs_enabled();
+
+ epi = ep_alloc_epitem(ep, event, tfile, fd);
+ if (IS_ERR(epi))
+ return PTR_ERR(epi);
+
+ error = ep_register_epitem(ep, epi, tep, full_check);
+ if (error)
+ return error;
+
+ /* Reject the insert if the new link would create too many back-paths. */
if (unlikely(full_check && reverse_path_check())) {
ep_remove(ep, epi);
return -EINVAL;
@@ -1638,28 +1858,21 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event,
*/
revents = ep_item_poll(epi, &epq.pt, 1);
- /*
- * We have to check if something went wrong during the poll wait queue
- * install process. Namely an allocation for a wait queue failed due
- * high memory pressure.
- */
+ /* ep_ptable_queue_proc() signals allocation failure by clearing epq.epi. */
if (unlikely(!epq.epi)) {
ep_remove(ep, epi);
return -ENOMEM;
}
- /* We have to drop the new item inside our item list to keep track of it */
+ /* Drop the new item onto the ready list if it is already ready. */
spin_lock_irq(&ep->lock);
- /* record NAPI ID of new item if present */
ep_set_busy_poll_napi_id(epi);
- /* If the file is already "ready" we drop it inside the ready list */
if (revents && !ep_is_linked(epi)) {
list_add_tail(&epi->rdllink, &ep->rdllist);
ep_pm_stay_awake(epi);
- /* Notify waiting tasks that events are available */
if (waitqueue_active(&ep->wq))
wake_up(&ep->wq);
if (waitqueue_active(&ep->poll_wait))
@@ -1751,11 +1964,87 @@ static int ep_modify(struct eventpoll *ep, struct epitem *epi,
return 0;
}
+/*
+ * Attempt to deliver one event for @epi into @*uevents.
+ *
+ * Returns 1 if an event was delivered (with *uevents advanced to the
+ * next slot), 0 if the re-poll reported no caller-requested events
+ * (@epi drops out of the ready list; a future callback will re-add
+ * it), or -EFAULT if copy_to_user() faulted (in which case @epi is
+ * re-inserted at the head of @scan_batch so ep_done_scan() merges it
+ * back to rdllist for the next attempt).
+ *
+ * PM bookkeeping and level-triggered re-queue are handled here.
+ * Caller holds ep->mtx and the scan is active.
+ */
+static int ep_deliver_event(struct eventpoll *ep, struct epitem *epi,
+ poll_table *pt,
+ struct epoll_event __user **uevents,
+ struct list_head *scan_batch)
+{
+ struct epoll_event __user *next;
+ struct wakeup_source *ws;
+ __poll_t revents;
+
+ /*
+ * Activate ep->ws before deactivating epi->ws to prevent
+ * triggering auto-suspend here (in case we reactivate epi->ws
+ * below). Rearranging to delay the deactivation would let
+ * epi->ws drift out of sync with ep_is_linked().
+ */
+ ws = ep_wakeup_source(epi);
+ if (ws) {
+ if (ws->active)
+ __pm_stay_awake(ep->ws);
+ __pm_relax(ws);
+ }
+
+ list_del_init(&epi->rdllink);
+
+ /*
+ * Re-poll under ep->mtx so userspace cannot change the item
+ * out from under us. If no caller-requested events remain,
+ * @epi stays off the ready list; the poll callback will
+ * re-queue it when events next appear.
+ */
+ revents = ep_item_poll(epi, pt, 1);
+ if (!revents)
+ return 0;
+
+ next = epoll_put_uevent(revents, epi->event.data, *uevents);
+ if (!next) {
+ /*
+ * copy_to_user() faulted: put the item back so
+ * ep_done_scan() splices it onto rdllist for the next
+ * attempt.
+ */
+ list_add(&epi->rdllink, scan_batch);
+ ep_pm_stay_awake(epi);
+ return -EFAULT;
+ }
+ *uevents = next;
+
+ if (epi->event.events & EPOLLONESHOT) {
+ epi->event.events &= EP_PRIVATE_BITS;
+ } else if (!(epi->event.events & EPOLLET)) {
+ /*
+ * Level-triggered: re-queue so the next epoll_wait()
+ * rechecks availability. We are the sole writer to
+ * rdllist here -- epoll_ctl() callers are locked out
+ * by ep->mtx, and the poll callback queues to ovflist
+ * during scans.
+ */
+ list_add_tail(&epi->rdllink, &ep->rdllist);
+ ep_pm_stay_awake(epi);
+ }
+ return 1;
+}
+
static int ep_send_events(struct eventpoll *ep,
struct epoll_event __user *events, int maxevents)
{
struct epitem *epi, *tmp;
- LIST_HEAD(txlist);
+ LIST_HEAD(scan_batch);
poll_table pt;
int res = 0;
@@ -1770,74 +2059,28 @@ static int ep_send_events(struct eventpoll *ep,
init_poll_funcptr(&pt, NULL);
mutex_lock(&ep->mtx);
- ep_start_scan(ep, &txlist);
+ ep_start_scan(ep, &scan_batch);
/*
- * We can loop without lock because we are passed a task private list.
- * Items cannot vanish during the loop we are holding ep->mtx.
+ * We can loop without lock because we are passed a task-private
+ * scan_batch; items cannot vanish while we hold ep->mtx.
*/
- list_for_each_entry_safe(epi, tmp, &txlist, rdllink) {
- struct wakeup_source *ws;
- __poll_t revents;
+ list_for_each_entry_safe(epi, tmp, &scan_batch, rdllink) {
+ int delivered;
if (res >= maxevents)
break;
- /*
- * Activate ep->ws before deactivating epi->ws to prevent
- * triggering auto-suspend here (in case we reactive epi->ws
- * below).
- *
- * This could be rearranged to delay the deactivation of epi->ws
- * instead, but then epi->ws would temporarily be out of sync
- * with ep_is_linked().
- */
- ws = ep_wakeup_source(epi);
- if (ws) {
- if (ws->active)
- __pm_stay_awake(ep->ws);
- __pm_relax(ws);
- }
-
- list_del_init(&epi->rdllink);
-
- /*
- * If the event mask intersect the caller-requested one,
- * deliver the event to userspace. Again, we are holding ep->mtx,
- * so no operations coming from userspace can change the item.
- */
- revents = ep_item_poll(epi, &pt, 1);
- if (!revents)
- continue;
-
- events = epoll_put_uevent(revents, epi->event.data, events);
- if (!events) {
- list_add(&epi->rdllink, &txlist);
- ep_pm_stay_awake(epi);
+ delivered = ep_deliver_event(ep, epi, &pt, &events, &scan_batch);
+ if (delivered < 0) {
if (!res)
- res = -EFAULT;
+ res = delivered;
break;
}
- res++;
- if (epi->event.events & EPOLLONESHOT)
- epi->event.events &= EP_PRIVATE_BITS;
- else if (!(epi->event.events & EPOLLET)) {
- /*
- * If this file has been added with Level
- * Trigger mode, we need to insert back inside
- * the ready list, so that the next call to
- * epoll_wait() will check again the events
- * availability. At this point, no one can insert
- * into ep->rdllist besides us. The epoll_ctl()
- * callers are locked out by
- * ep_send_events() holding "mtx" and the
- * poll callback will queue them in ep->ovflist.
- */
- list_add_tail(&epi->rdllink, &ep->rdllist);
- ep_pm_stay_awake(epi);
- }
+ res += delivered;
}
- ep_done_scan(ep, &txlist);
+
+ ep_done_scan(ep, &scan_batch);
mutex_unlock(&ep->mtx);
return res;
diff --git a/fs/exfat/dir.c b/fs/exfat/dir.c
index 39310bb9eccc..e624fc7c2788 100644
--- a/fs/exfat/dir.c
+++ b/fs/exfat/dir.c
@@ -500,31 +500,70 @@ static void exfat_free_benign_secondary_clusters(struct inode *inode,
exfat_free_cluster(inode, &dir);
}
+/*
+ * exfat_init_ext_entry - initialize extension entries in a directory entry set
+ * @es: target entry set
+ * @num_entries: number of entries excluding benign secondary entries
+ * @p_uniname: filename to store
+ * @old_es: optional source entry set with benign secondary entries, or NULL
+ * @num_extra: number of benign secondary entries to copy from @old_es
+ *
+ * Set up the file, stream extension, and filename entries in @es, optionally
+ * preserving @num_extra benign secondary entries from @old_es. @es and @old_es
+ * may refer to the same entry set; excess entries are marked as deleted.
+ */
void exfat_init_ext_entry(struct exfat_entry_set_cache *es, int num_entries,
- struct exfat_uni_name *p_uniname)
+ struct exfat_uni_name *p_uniname,
+ struct exfat_entry_set_cache *old_es, int num_extra)
{
- int i;
+ int i, src_start = 0, old_num;
unsigned short *uniname = p_uniname->name;
struct exfat_dentry *ep;
+ if (WARN_ON(num_extra < 0 || (num_extra && (!old_es ||
+ old_es->num_entries < ES_IDX_FIRST_FILENAME + num_extra))))
+ num_extra = 0;
+
+ /*
+ * Save old entry count and source position before modifying
+ * es->num_entries, since old_es and es may point to the same
+ * entry set.
+ */
+ old_num = es->num_entries;
+ if (old_es && num_extra > 0)
+ src_start = old_es->num_entries - num_extra;
+
+ es->num_entries = num_entries + num_extra;
ep = exfat_get_dentry_cached(es, ES_IDX_FILE);
- ep->dentry.file.num_ext = (unsigned char)(num_entries - 1);
+ ep->dentry.file.num_ext = (unsigned char)(num_entries - 1 + num_extra);
ep = exfat_get_dentry_cached(es, ES_IDX_STREAM);
ep->dentry.stream.name_len = p_uniname->name_len;
ep->dentry.stream.name_hash = cpu_to_le16(p_uniname->name_hash);
+ if (old_es && num_extra > 0) {
+ for (i = 0; i < num_extra; i++)
+ *exfat_get_dentry_cached(es, num_entries + i) =
+ *exfat_get_dentry_cached(old_es, src_start + i);
+ }
+
for (i = ES_IDX_FIRST_FILENAME; i < num_entries; i++) {
ep = exfat_get_dentry_cached(es, i);
exfat_init_name_entry(ep, uniname);
uniname += EXFAT_FILE_NAME_LEN;
}
+ /* Mark excess old entries as deleted (in-place shrink) */
+ for (i = num_entries + num_extra; i < old_num; i++) {
+ ep = exfat_get_dentry_cached(es, i);
+ exfat_set_entry_type(ep, TYPE_DELETED);
+ }
+
exfat_update_dir_chksum(es);
}
void exfat_remove_entries(struct inode *inode, struct exfat_entry_set_cache *es,
- int order)
+ int order, bool free_benign)
{
int i;
struct exfat_dentry *ep;
@@ -532,7 +571,7 @@ void exfat_remove_entries(struct inode *inode, struct exfat_entry_set_cache *es,
for (i = order; i < es->num_entries; i++) {
ep = exfat_get_dentry_cached(es, i);
- if (exfat_get_entry_type(ep) & TYPE_BENIGN_SEC)
+ if (free_benign && (exfat_get_entry_type(ep) & TYPE_BENIGN_SEC))
exfat_free_benign_secondary_clusters(inode, ep);
exfat_set_entry_type(ep, TYPE_DELETED);
diff --git a/fs/exfat/exfat_fs.h b/fs/exfat/exfat_fs.h
index d2ba8e2d0c39..13cd686e1dbc 100644
--- a/fs/exfat/exfat_fs.h
+++ b/fs/exfat/exfat_fs.h
@@ -492,9 +492,10 @@ void exfat_init_dir_entry(struct exfat_entry_set_cache *es,
unsigned int type, unsigned int start_clu,
unsigned long long size, struct timespec64 *ts);
void exfat_init_ext_entry(struct exfat_entry_set_cache *es, int num_entries,
- struct exfat_uni_name *p_uniname);
+ struct exfat_uni_name *p_uniname,
+ struct exfat_entry_set_cache *old_es, int num_extra);
void exfat_remove_entries(struct inode *inode, struct exfat_entry_set_cache *es,
- int order);
+ int order, bool free_benign);
void exfat_update_dir_chksum(struct exfat_entry_set_cache *es);
int exfat_calc_num_entries(struct exfat_uni_name *p_uniname);
int exfat_find_dir_entry(struct super_block *sb, struct exfat_inode_info *ei,
@@ -508,6 +509,8 @@ struct exfat_dentry *exfat_get_dentry_cached(struct exfat_entry_set_cache *es,
int exfat_get_dentry_set(struct exfat_entry_set_cache *es,
struct super_block *sb, struct exfat_chain *p_dir, int entry,
unsigned int num_entries);
+#define exfat_get_dentry_set_by_ei(es, sb, ei) \
+ exfat_get_dentry_set(es, sb, &(ei)->dir, (ei)->entry, ES_ALL_ENTRIES)
int exfat_get_empty_dentry_set(struct exfat_entry_set_cache *es,
struct super_block *sb, struct exfat_chain *p_dir, int entry,
unsigned int num_entries);
diff --git a/fs/exfat/inode.c b/fs/exfat/inode.c
index cc4dcb7a32b5..b22c02d6000f 100644
--- a/fs/exfat/inode.c
+++ b/fs/exfat/inode.c
@@ -43,7 +43,7 @@ int __exfat_write_inode(struct inode *inode, int sync)
exfat_set_volume_dirty(sb);
/* get the directory entry of given file or directory */
- if (exfat_get_dentry_set(&es, sb, &(ei->dir), ei->entry, ES_ALL_ENTRIES))
+ if (exfat_get_dentry_set_by_ei(&es, sb, ei))
return -EIO;
ep = exfat_get_dentry_cached(&es, ES_IDX_FILE);
ep2 = exfat_get_dentry_cached(&es, ES_IDX_STREAM);
diff --git a/fs/exfat/namei.c b/fs/exfat/namei.c
index 44cce2544a74..22eca587cb95 100644
--- a/fs/exfat/namei.c
+++ b/fs/exfat/namei.c
@@ -311,6 +311,9 @@ static int exfat_find_empty_entry(struct inode *inode,
ei->hint_femp.eidx = EXFAT_HINT_NONE;
}
+ exfat_chain_set(p_dir, ei->start_clu,
+ EXFAT_B_TO_CLU(i_size_read(inode), sbi), ei->flags);
+
while ((dentry = exfat_search_empty_slot(sb, &hint_femp, p_dir,
num_entries, es)) < 0) {
if (dentry == -EIO)
@@ -386,14 +389,11 @@ static int exfat_find_empty_entry(struct inode *inode,
* Zero if it was successful; otherwise nonzero.
*/
static int __exfat_resolve_path(struct inode *inode, const unsigned char *path,
- struct exfat_chain *p_dir, struct exfat_uni_name *p_uniname,
- int lookup)
+ struct exfat_uni_name *p_uniname, int lookup)
{
int namelen;
int lossy = NLS_NAME_NO_LOSSY;
struct super_block *sb = inode->i_sb;
- struct exfat_sb_info *sbi = EXFAT_SB(sb);
- struct exfat_inode_info *ei = EXFAT_I(inode);
int pathlen = strlen(path);
/*
@@ -432,24 +432,19 @@ static int __exfat_resolve_path(struct inode *inode, const unsigned char *path,
if ((lossy && !lookup) || !namelen)
return (lossy & NLS_NAME_OVERLEN) ? -ENAMETOOLONG : -EINVAL;
- exfat_chain_set(p_dir, ei->start_clu,
- EXFAT_B_TO_CLU(i_size_read(inode), sbi), ei->flags);
-
return 0;
}
static inline int exfat_resolve_path(struct inode *inode,
- const unsigned char *path, struct exfat_chain *dir,
- struct exfat_uni_name *uni)
+ const unsigned char *path, struct exfat_uni_name *uni)
{
- return __exfat_resolve_path(inode, path, dir, uni, 0);
+ return __exfat_resolve_path(inode, path, uni, 0);
}
static inline int exfat_resolve_path_for_lookup(struct inode *inode,
- const unsigned char *path, struct exfat_chain *dir,
- struct exfat_uni_name *uni)
+ const unsigned char *path, struct exfat_uni_name *uni)
{
- return __exfat_resolve_path(inode, path, dir, uni, 1);
+ return __exfat_resolve_path(inode, path, uni, 1);
}
static inline loff_t exfat_make_i_pos(struct exfat_dir_entry *info)
@@ -471,7 +466,7 @@ static int exfat_add_entry(struct inode *inode, const char *path,
int clu_size = 0;
unsigned int start_clu = EXFAT_FREE_CLUSTER;
- ret = exfat_resolve_path(inode, path, p_dir, &uniname);
+ ret = exfat_resolve_path(inode, path, &uniname);
if (ret)
goto out;
@@ -503,7 +498,7 @@ static int exfat_add_entry(struct inode *inode, const char *path,
* the first cluster is not determined yet. (0)
*/
exfat_init_dir_entry(&es, type, start_clu, clu_size, &ts);
- exfat_init_ext_entry(&es, num_entries, &uniname);
+ exfat_init_ext_entry(&es, num_entries, &uniname, NULL, 0);
ret = exfat_put_dentry_set(&es, IS_DIRSYNC(inode));
if (ret)
@@ -602,10 +597,13 @@ static int exfat_find(struct inode *dir, struct qstr *qname,
return -ENOENT;
/* check the validity of directory name in the given pathname */
- ret = exfat_resolve_path_for_lookup(dir, qname->name, &cdir, &uni_name);
+ ret = exfat_resolve_path_for_lookup(dir, qname->name, &uni_name);
if (ret)
return ret;
+ exfat_chain_set(&cdir, ei->start_clu,
+ EXFAT_B_TO_CLU(i_size_read(dir), sbi), ei->flags);
+
/* check the validation of hint_stat and initialize it if required */
if (ei->version != (inode_peek_iversion_raw(dir) & 0xffffffff)) {
ei->hint_stat.clu = cdir.dir;
@@ -788,26 +786,23 @@ static struct dentry *exfat_lookup(struct inode *dir, struct dentry *dentry,
/* remove an entry, BUT don't truncate */
static int exfat_unlink(struct inode *dir, struct dentry *dentry)
{
- struct exfat_chain cdir;
struct super_block *sb = dir->i_sb;
struct inode *inode = dentry->d_inode;
struct exfat_inode_info *ei = EXFAT_I(inode);
struct exfat_entry_set_cache es;
- int entry, err = 0;
+ int err = 0;
if (unlikely(exfat_forced_shutdown(sb)))
return -EIO;
mutex_lock(&EXFAT_SB(sb)->s_lock);
- exfat_chain_dup(&cdir, &ei->dir);
- entry = ei->entry;
if (ei->dir.dir == DIR_DELETED) {
exfat_err(sb, "abnormal access to deleted dentry");
err = -ENOENT;
goto unlock;
}
- err = exfat_get_dentry_set(&es, sb, &cdir, entry, ES_ALL_ENTRIES);
+ err = exfat_get_dentry_set_by_ei(&es, sb, ei);
if (err) {
err = -EIO;
goto unlock;
@@ -816,7 +811,7 @@ static int exfat_unlink(struct inode *dir, struct dentry *dentry)
exfat_set_volume_dirty(sb);
/* update the directory entry */
- exfat_remove_entries(inode, &es, ES_IDX_FILE);
+ exfat_remove_entries(inode, &es, ES_IDX_FILE, true);
err = exfat_put_dentry_set(&es, IS_DIRSYNC(inode));
if (err)
@@ -942,21 +937,18 @@ static int exfat_check_dir_empty(struct super_block *sb,
static int exfat_rmdir(struct inode *dir, struct dentry *dentry)
{
struct inode *inode = dentry->d_inode;
- struct exfat_chain cdir, clu_to_free;
+ struct exfat_chain clu_to_free;
struct super_block *sb = inode->i_sb;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
struct exfat_inode_info *ei = EXFAT_I(inode);
struct exfat_entry_set_cache es;
- int entry, err;
+ int err;
if (unlikely(exfat_forced_shutdown(sb)))
return -EIO;
mutex_lock(&EXFAT_SB(inode->i_sb)->s_lock);
- exfat_chain_dup(&cdir, &ei->dir);
- entry = ei->entry;
-
if (ei->dir.dir == DIR_DELETED) {
exfat_err(sb, "abnormal access to deleted dentry");
err = -ENOENT;
@@ -974,7 +966,7 @@ static int exfat_rmdir(struct inode *dir, struct dentry *dentry)
goto unlock;
}
- err = exfat_get_dentry_set(&es, sb, &cdir, entry, ES_ALL_ENTRIES);
+ err = exfat_get_dentry_set_by_ei(&es, sb, ei);
if (err) {
err = -EIO;
goto unlock;
@@ -982,7 +974,7 @@ static int exfat_rmdir(struct inode *dir, struct dentry *dentry)
exfat_set_volume_dirty(sb);
- exfat_remove_entries(inode, &es, ES_IDX_FILE);
+ exfat_remove_entries(inode, &es, ES_IDX_FILE, true);
err = exfat_put_dentry_set(&es, IS_DIRSYNC(dir));
if (err)
@@ -1009,15 +1001,32 @@ static int exfat_rmdir(struct inode *dir, struct dentry *dentry)
return err;
}
-static int exfat_rename_file(struct inode *inode, struct exfat_chain *p_dir,
- int oldentry, struct exfat_uni_name *p_uniname,
- struct exfat_inode_info *ei)
+/*
+ * Count benign secondary entries beyond the filename entries.
+ * Returns the count, or -EIO if the entry set is inconsistent.
+ */
+static int exfat_count_extra_entries(struct exfat_entry_set_cache *es)
+{
+ struct exfat_dentry *stream;
+ unsigned int name_entries;
+ int extra;
+
+ stream = exfat_get_dentry_cached(es, ES_IDX_STREAM);
+ name_entries = EXFAT_FILENAME_ENTRY_NUM(stream->dentry.stream.name_len);
+ extra = es->num_entries - (ES_IDX_FIRST_FILENAME + name_entries);
+
+ return extra >= 0 ? extra : -EIO;
+}
+
+static int exfat_rename_file(struct inode *parent_inode,
+ struct exfat_uni_name *p_uniname, struct exfat_inode_info *ei)
{
int ret, num_new_entries;
struct exfat_dentry *epold, *epnew;
- struct super_block *sb = inode->i_sb;
+ struct super_block *sb = parent_inode->i_sb;
struct exfat_entry_set_cache old_es, new_es;
- int sync = IS_DIRSYNC(inode);
+ int sync = IS_DIRSYNC(parent_inode);
+ unsigned int num_extra_entries, num_total_entries;
if (unlikely(exfat_forced_shutdown(sb)))
return -EIO;
@@ -1026,19 +1035,24 @@ static int exfat_rename_file(struct inode *inode, struct exfat_chain *p_dir,
if (num_new_entries < 0)
return num_new_entries;
- ret = exfat_get_dentry_set(&old_es, sb, p_dir, oldentry, ES_ALL_ENTRIES);
- if (ret) {
- ret = -EIO;
- return ret;
- }
+ ret = exfat_get_dentry_set_by_ei(&old_es, sb, ei);
+ if (ret)
+ return -EIO;
epold = exfat_get_dentry_cached(&old_es, ES_IDX_FILE);
- if (old_es.num_entries < num_new_entries) {
+ ret = exfat_count_extra_entries(&old_es);
+ if (ret < 0)
+ goto put_old_es;
+ num_extra_entries = ret;
+ num_total_entries = num_new_entries + num_extra_entries;
+
+ if (old_es.num_entries < num_total_entries) {
int newentry;
+ struct exfat_chain dir;
- newentry = exfat_find_empty_entry(inode, p_dir, num_new_entries,
- &new_es);
+ newentry = exfat_find_empty_entry(parent_inode, &dir,
+ num_total_entries, &new_es);
if (newentry < 0) {
ret = newentry; /* -EIO or -ENOSPC */
goto put_old_es;
@@ -1055,14 +1069,24 @@ static int exfat_rename_file(struct inode *inode, struct exfat_chain *p_dir,
epnew = exfat_get_dentry_cached(&new_es, ES_IDX_STREAM);
*epnew = *epold;
- exfat_init_ext_entry(&new_es, num_new_entries, p_uniname);
+ exfat_init_ext_entry(&new_es, num_new_entries, p_uniname,
+ &old_es, num_extra_entries);
ret = exfat_put_dentry_set(&new_es, sync);
- if (ret)
+ if (ret) {
+ /* Best-effort delete to avoid duplicate entries */
+ if (!exfat_get_dentry_set(&new_es, sb,
+ &dir, newentry,
+ ES_ALL_ENTRIES)) {
+ exfat_remove_entries(parent_inode, &new_es,
+ ES_IDX_FILE, false);
+ exfat_put_dentry_set(&new_es, false);
+ }
goto put_old_es;
+ }
- exfat_remove_entries(inode, &old_es, ES_IDX_FILE);
- ei->dir = *p_dir;
+ exfat_remove_entries(parent_inode, &old_es, ES_IDX_FILE, false);
+ ei->dir = dir;
ei->entry = newentry;
} else {
if (exfat_get_entry_type(epold) == TYPE_FILE) {
@@ -1070,8 +1094,8 @@ static int exfat_rename_file(struct inode *inode, struct exfat_chain *p_dir,
ei->attr |= EXFAT_ATTR_ARCHIVE;
}
- exfat_remove_entries(inode, &old_es, ES_IDX_FIRST_FILENAME + 1);
- exfat_init_ext_entry(&old_es, num_new_entries, p_uniname);
+ exfat_init_ext_entry(&old_es, num_new_entries, p_uniname,
+ &old_es, num_extra_entries);
}
return exfat_put_dentry_set(&old_es, sync);
@@ -1080,26 +1104,31 @@ static int exfat_rename_file(struct inode *inode, struct exfat_chain *p_dir,
return ret;
}
-static int exfat_move_file(struct inode *inode, struct exfat_chain *p_olddir,
- int oldentry, struct exfat_chain *p_newdir,
+static int exfat_move_file(struct inode *parent_inode,
struct exfat_uni_name *p_uniname, struct exfat_inode_info *ei)
{
int ret, newentry, num_new_entries;
struct exfat_dentry *epmov, *epnew;
- struct super_block *sb = inode->i_sb;
struct exfat_entry_set_cache mov_es, new_es;
+ struct exfat_chain newdir;
+ unsigned int num_extra_entries, num_total_entries;
num_new_entries = exfat_calc_num_entries(p_uniname);
if (num_new_entries < 0)
return num_new_entries;
- ret = exfat_get_dentry_set(&mov_es, sb, p_olddir, oldentry,
- ES_ALL_ENTRIES);
+ ret = exfat_get_dentry_set_by_ei(&mov_es, parent_inode->i_sb, ei);
if (ret)
return -EIO;
- newentry = exfat_find_empty_entry(inode, p_newdir, num_new_entries,
- &new_es);
+ ret = exfat_count_extra_entries(&mov_es);
+ if (ret < 0)
+ goto put_mov_es;
+ num_extra_entries = ret;
+ num_total_entries = num_new_entries + num_extra_entries;
+
+ newentry = exfat_find_empty_entry(parent_inode, &newdir,
+ num_total_entries, &new_es);
if (newentry < 0) {
ret = newentry; /* -EIO or -ENOSPC */
goto put_mov_es;
@@ -1117,23 +1146,31 @@ static int exfat_move_file(struct inode *inode, struct exfat_chain *p_olddir,
epnew = exfat_get_dentry_cached(&new_es, ES_IDX_STREAM);
*epnew = *epmov;
- exfat_init_ext_entry(&new_es, num_new_entries, p_uniname);
- exfat_remove_entries(inode, &mov_es, ES_IDX_FILE);
+ exfat_init_ext_entry(&new_es, num_new_entries, p_uniname,
+ &mov_es, num_extra_entries);
- exfat_chain_set(&ei->dir, p_newdir->dir, p_newdir->size,
- p_newdir->flags);
+ exfat_remove_entries(parent_inode, &mov_es, ES_IDX_FILE, false);
+ ei->dir = newdir;
ei->entry = newentry;
- ret = exfat_put_dentry_set(&new_es, IS_DIRSYNC(inode));
- if (ret)
+ ret = exfat_put_dentry_set(&new_es, IS_DIRSYNC(parent_inode));
+ if (ret) {
+ /* Best-effort delete to avoid duplicate entries */
+ if (!exfat_get_dentry_set(&new_es, parent_inode->i_sb,
+ &newdir, newentry,
+ ES_ALL_ENTRIES)) {
+ exfat_remove_entries(parent_inode, &new_es,
+ ES_IDX_FILE, false);
+ exfat_put_dentry_set(&new_es, false);
+ }
goto put_mov_es;
+ }
- return exfat_put_dentry_set(&mov_es, IS_DIRSYNC(inode));
+ return exfat_put_dentry_set(&mov_es, IS_DIRSYNC(parent_inode));
put_mov_es:
exfat_put_dentry_set(&mov_es, false);
-
return ret;
}
@@ -1143,19 +1180,12 @@ static int __exfat_rename(struct inode *old_parent_inode,
struct dentry *new_dentry)
{
int ret;
- int dentry;
- struct exfat_chain olddir, newdir;
- struct exfat_chain *p_dir = NULL;
struct exfat_uni_name uni_name;
- struct exfat_dentry *ep;
struct super_block *sb = old_parent_inode->i_sb;
struct exfat_sb_info *sbi = EXFAT_SB(sb);
const unsigned char *new_path = new_dentry->d_name.name;
struct inode *new_inode = new_dentry->d_inode;
struct exfat_inode_info *new_ei = NULL;
- unsigned int new_entry_type = TYPE_UNUSED;
- int new_entry = 0;
- struct buffer_head *new_bh = NULL;
/* check the validity of pointer parameters */
if (new_path == NULL || strlen(new_path) == 0)
@@ -1166,11 +1196,6 @@ static int __exfat_rename(struct inode *old_parent_inode,
return -ENOENT;
}
- exfat_chain_set(&olddir, EXFAT_I(old_parent_inode)->start_clu,
- EXFAT_B_TO_CLU_ROUND_UP(i_size_read(old_parent_inode), sbi),
- EXFAT_I(old_parent_inode)->flags);
- dentry = ei->entry;
-
/* check whether new dir is existing directory and empty */
if (new_inode) {
ret = -EIO;
@@ -1181,17 +1206,8 @@ static int __exfat_rename(struct inode *old_parent_inode,
goto out;
}
- p_dir = &(new_ei->dir);
- new_entry = new_ei->entry;
- ep = exfat_get_dentry(sb, p_dir, new_entry, &new_bh);
- if (!ep)
- goto out;
-
- new_entry_type = exfat_get_entry_type(ep);
- brelse(new_bh);
-
/* if new_inode exists, update ei */
- if (new_entry_type == TYPE_DIR) {
+ if (S_ISDIR(new_inode->i_mode)) {
struct exfat_chain new_clu;
new_clu.dir = new_ei->start_clu;
@@ -1207,39 +1223,35 @@ static int __exfat_rename(struct inode *old_parent_inode,
}
/* check the validity of directory name in the given new pathname */
- ret = exfat_resolve_path(new_parent_inode, new_path, &newdir,
- &uni_name);
+ ret = exfat_resolve_path(new_parent_inode, new_path, &uni_name);
if (ret)
goto out;
exfat_set_volume_dirty(sb);
- if (olddir.dir == newdir.dir)
- ret = exfat_rename_file(new_parent_inode, &olddir, dentry,
- &uni_name, ei);
+ if (new_parent_inode == old_parent_inode)
+ ret = exfat_rename_file(new_parent_inode, &uni_name, ei);
else
- ret = exfat_move_file(new_parent_inode, &olddir, dentry,
- &newdir, &uni_name, ei);
+ ret = exfat_move_file(new_parent_inode, &uni_name, ei);
if (!ret && new_inode) {
struct exfat_entry_set_cache es;
/* delete entries of new_dir */
- ret = exfat_get_dentry_set(&es, sb, p_dir, new_entry,
- ES_ALL_ENTRIES);
+ ret = exfat_get_dentry_set_by_ei(&es, sb, new_ei);
if (ret) {
ret = -EIO;
goto del_out;
}
- exfat_remove_entries(new_inode, &es, ES_IDX_FILE);
+ exfat_remove_entries(new_inode, &es, ES_IDX_FILE, true);
ret = exfat_put_dentry_set(&es, IS_DIRSYNC(new_inode));
if (ret)
goto del_out;
/* Free the clusters if new_inode is a dir(as if exfat_rmdir) */
- if (new_entry_type == TYPE_DIR &&
+ if (S_ISDIR(new_inode->i_mode) &&
new_ei->start_clu != EXFAT_EOF_CLUSTER) {
/* new_ei, new_clu_to_free */
struct exfat_chain new_clu_to_free;
diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index 10b061ac5bc0..908d4849874d 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -262,12 +262,15 @@ static ssize_t ext2_dio_write_iter(struct kiocb *iocb, struct iov_iter *from)
endbyte = pos + status - 1;
ret2 = filemap_write_and_wait_range(inode->i_mapping, pos,
endbyte);
- if (!ret2)
+ if (!ret2) {
invalidate_mapping_pages(inode->i_mapping,
pos >> PAGE_SHIFT,
endbyte >> PAGE_SHIFT);
- if (ret > 0)
- generic_write_sync(iocb, ret);
+ if (ret > 0)
+ ret = generic_write_sync(iocb, ret);
+ } else {
+ ret = ret2;
+ }
}
out_unlock:
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index a0b68b9d9626..de8ad1f90af1 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -1410,7 +1410,8 @@ static int ext4_journalled_write_end(struct file *file,
BUG_ON(!ext4_handle_valid(handle));
- if (ext4_has_inline_data(inode))
+ if (ext4_has_inline_data(inode) &&
+ ext4_test_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA))
return ext4_write_inline_data_end(inode, pos, len, copied,
folio);
diff --git a/fs/ext4/ioctl.c b/fs/ext4/ioctl.c
index 99d185d00f37..a8a024bb62b4 100644
--- a/fs/ext4/ioctl.c
+++ b/fs/ext4/ioctl.c
@@ -826,11 +826,17 @@ int ext4_force_shutdown(struct super_block *sb, u32 flags)
bdev_thaw(sb->s_bdev);
break;
case EXT4_GOING_FLAGS_LOGFLUSH:
+ /*
+ * Call ext4_force_commit() before setting EXT4_FLAGS_SHUTDOWN.
+ * This is because in data=ordered mode, journal commit
+ * triggers data writeback which fails if shutdown is already
+ * set, causing the journal to be aborted prematurely before
+ * the commit succeeds.
+ */
+ (void) ext4_force_commit(sb);
set_bit(EXT4_FLAGS_SHUTDOWN, &sbi->s_ext4_flags);
- if (sbi->s_journal && !is_journal_aborted(sbi->s_journal)) {
- (void) ext4_force_commit(sb);
+ if (sbi->s_journal && !is_journal_aborted(sbi->s_journal))
jbd2_journal_abort(sbi->s_journal, -ESHUTDOWN);
- }
break;
case EXT4_GOING_FLAGS_NOLOGFLUSH:
set_bit(EXT4_FLAGS_SHUTDOWN, &sbi->s_ext4_flags);
@@ -1366,6 +1372,9 @@ static long __ext4_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
goto mext_out;
}
+ if (file_inode(filp)->i_sb != file_inode(fd_file(donor))->i_sb)
+ return -EXDEV;
+
err = mnt_want_write_file(filp);
if (err)
goto mext_out;
diff --git a/fs/fhandle.c b/fs/fhandle.c
index bbb5857c77b9..28d8393f7699 100644
--- a/fs/fhandle.c
+++ b/fs/fhandle.c
@@ -252,7 +252,7 @@ static bool capable_wrt_mount(struct mount *mount)
*/
guard(rcu)();
mnt_ns = READ_ONCE(mount->mnt_ns);
- return ns_capable(mnt_ns->user_ns, CAP_SYS_ADMIN);
+ return mnt_ns && ns_capable(mnt_ns->user_ns, CAP_SYS_ADMIN);
}
/*
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 57a31bd110c0..4e1e0dd16428 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -1218,11 +1218,6 @@ void cgroup_writeback_umount(struct super_block *sb)
* will then drain it.
*/
synchronize_rcu();
- /*
- * Use rcu_barrier() to wait for all pending callbacks to
- * ensure that all in-flight wb switches are in the workqueue.
- */
- rcu_barrier();
flush_workqueue(isw_wq);
}
}
diff --git a/fs/hfs/bfind.c b/fs/hfs/bfind.c
index e46f650b5e9c..245c2b967189 100644
--- a/fs/hfs/bfind.c
+++ b/fs/hfs/bfind.c
@@ -167,7 +167,7 @@ int hfs_brec_find(struct hfs_find_data *fd)
return res;
}
-int hfs_brec_read(struct hfs_find_data *fd, void *rec, int rec_len)
+int hfs_brec_read(struct hfs_find_data *fd, void *rec, u32 rec_len)
{
int res;
diff --git a/fs/hfs/bnode.c b/fs/hfs/bnode.c
index ce2e573a071e..f9db5b31acd1 100644
--- a/fs/hfs/bnode.c
+++ b/fs/hfs/bnode.c
@@ -16,14 +16,14 @@
#include "btree.h"
static inline
-bool is_bnode_offset_valid(struct hfs_bnode *node, int off)
+bool is_bnode_offset_valid(struct hfs_bnode *node, u32 off)
{
bool is_valid = off < node->tree->node_size;
if (!is_valid) {
pr_err("requested invalid offset: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d\n",
+ "node_size %u, offset %u\n",
node->this, node->type, node->height,
node->tree->node_size, off);
}
@@ -32,7 +32,7 @@ bool is_bnode_offset_valid(struct hfs_bnode *node, int off)
}
static inline
-int check_and_correct_requested_length(struct hfs_bnode *node, int off, int len)
+u32 check_and_correct_requested_length(struct hfs_bnode *node, u32 off, u32 len)
{
unsigned int node_size;
@@ -41,13 +41,13 @@ int check_and_correct_requested_length(struct hfs_bnode *node, int off, int len)
node_size = node->tree->node_size;
- if ((off + len) > node_size) {
- int new_len = (int)node_size - off;
+ if ((u64)off + len > node_size) {
+ u32 new_len = node_size - off;
pr_err("requested length has been corrected: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, "
- "requested_len %d, corrected_len %d\n",
+ "node_size %u, offset %u, "
+ "requested_len %u, corrected_len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len, new_len);
@@ -57,12 +57,12 @@ int check_and_correct_requested_length(struct hfs_bnode *node, int off, int len)
return len;
}
-void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len)
+void hfs_bnode_read(struct hfs_bnode *node, void *buf, u32 off, u32 len)
{
struct page *page;
- int pagenum;
- int bytes_read;
- int bytes_to_read;
+ u32 pagenum;
+ u32 bytes_read;
+ u32 bytes_to_read;
memset(buf, 0, len);
@@ -72,7 +72,7 @@ void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len)
if (len == 0) {
pr_err("requested zero length: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, len %d\n",
+ "node_size %u, offset %u, len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len);
return;
@@ -88,7 +88,7 @@ void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len)
if (pagenum >= node->tree->pages_per_bnode)
break;
page = node->page[pagenum];
- bytes_to_read = min_t(int, len - bytes_read, PAGE_SIZE - off);
+ bytes_to_read = min_t(u32, len - bytes_read, PAGE_SIZE - off);
memcpy_from_page(buf + bytes_read, page, off, bytes_to_read);
@@ -97,7 +97,7 @@ void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len)
}
}
-u16 hfs_bnode_read_u16(struct hfs_bnode *node, int off)
+u16 hfs_bnode_read_u16(struct hfs_bnode *node, u32 off)
{
__be16 data;
// optimize later...
@@ -105,7 +105,7 @@ u16 hfs_bnode_read_u16(struct hfs_bnode *node, int off)
return be16_to_cpu(data);
}
-u8 hfs_bnode_read_u8(struct hfs_bnode *node, int off)
+u8 hfs_bnode_read_u8(struct hfs_bnode *node, u32 off)
{
u8 data;
// optimize later...
@@ -113,10 +113,10 @@ u8 hfs_bnode_read_u8(struct hfs_bnode *node, int off)
return data;
}
-void hfs_bnode_read_key(struct hfs_bnode *node, void *key, int off)
+void hfs_bnode_read_key(struct hfs_bnode *node, void *key, u32 off)
{
struct hfs_btree *tree;
- int key_len;
+ u32 key_len;
tree = node->tree;
if (node->type == HFS_NODE_LEAF ||
@@ -127,14 +127,14 @@ void hfs_bnode_read_key(struct hfs_bnode *node, void *key, int off)
if (key_len > sizeof(hfs_btree_key) || key_len < 1) {
memset(key, 0, sizeof(hfs_btree_key));
- pr_err("hfs: Invalid key length: %d\n", key_len);
+ pr_err("hfs: Invalid key length: %u\n", key_len);
return;
}
hfs_bnode_read(node, key, off, key_len);
}
-void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len)
+void hfs_bnode_write(struct hfs_bnode *node, void *buf, u32 off, u32 len)
{
struct page *page;
@@ -144,7 +144,7 @@ void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len)
if (len == 0) {
pr_err("requested zero length: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, len %d\n",
+ "node_size %u, offset %u, len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len);
return;
@@ -159,20 +159,20 @@ void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len)
set_page_dirty(page);
}
-void hfs_bnode_write_u16(struct hfs_bnode *node, int off, u16 data)
+void hfs_bnode_write_u16(struct hfs_bnode *node, u32 off, u16 data)
{
__be16 v = cpu_to_be16(data);
// optimize later...
hfs_bnode_write(node, &v, off, 2);
}
-void hfs_bnode_write_u8(struct hfs_bnode *node, int off, u8 data)
+void hfs_bnode_write_u8(struct hfs_bnode *node, u32 off, u8 data)
{
// optimize later...
hfs_bnode_write(node, &data, off, 1);
}
-void hfs_bnode_clear(struct hfs_bnode *node, int off, int len)
+void hfs_bnode_clear(struct hfs_bnode *node, u32 off, u32 len)
{
struct page *page;
@@ -182,7 +182,7 @@ void hfs_bnode_clear(struct hfs_bnode *node, int off, int len)
if (len == 0) {
pr_err("requested zero length: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, len %d\n",
+ "node_size %u, offset %u, len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len);
return;
@@ -197,8 +197,8 @@ void hfs_bnode_clear(struct hfs_bnode *node, int off, int len)
set_page_dirty(page);
}
-void hfs_bnode_copy(struct hfs_bnode *dst_node, int dst,
- struct hfs_bnode *src_node, int src, int len)
+void hfs_bnode_copy(struct hfs_bnode *dst_node, u32 dst,
+ struct hfs_bnode *src_node, u32 src, u32 len)
{
struct page *src_page, *dst_page;
@@ -218,7 +218,7 @@ void hfs_bnode_copy(struct hfs_bnode *dst_node, int dst,
set_page_dirty(dst_page);
}
-void hfs_bnode_move(struct hfs_bnode *node, int dst, int src, int len)
+void hfs_bnode_move(struct hfs_bnode *node, u32 dst, u32 src, u32 len)
{
struct page *page;
void *ptr;
diff --git a/fs/hfs/brec.c b/fs/hfs/brec.c
index b01db1fae147..a463315316bf 100644
--- a/fs/hfs/brec.c
+++ b/fs/hfs/brec.c
@@ -62,7 +62,7 @@ u16 hfs_brec_keylen(struct hfs_bnode *node, u16 rec)
return retval;
}
-int hfs_brec_insert(struct hfs_find_data *fd, void *entry, int entry_len)
+int hfs_brec_insert(struct hfs_find_data *fd, void *entry, u32 entry_len)
{
struct hfs_btree *tree;
struct hfs_bnode *node, *new_node;
diff --git a/fs/hfs/btree.c b/fs/hfs/btree.c
index e86e1e235658..e748d92d8410 100644
--- a/fs/hfs/btree.c
+++ b/fs/hfs/btree.c
@@ -259,7 +259,7 @@ static struct hfs_bnode *hfs_bmap_new_bmap(struct hfs_bnode *prev, u32 idx)
}
/* Make sure @tree has enough space for the @rsvd_nodes */
-int hfs_bmap_reserve(struct hfs_btree *tree, int rsvd_nodes)
+int hfs_bmap_reserve(struct hfs_btree *tree, u32 rsvd_nodes)
{
struct inode *inode = tree->inode;
u32 count;
diff --git a/fs/hfs/btree.h b/fs/hfs/btree.h
index 0e6baee93245..97f88035b224 100644
--- a/fs/hfs/btree.h
+++ b/fs/hfs/btree.h
@@ -86,48 +86,49 @@ struct hfs_find_data {
/* btree.c */
-extern struct hfs_btree *hfs_btree_open(struct super_block *, u32, btree_keycmp);
-extern void hfs_btree_close(struct hfs_btree *);
-extern void hfs_btree_write(struct hfs_btree *);
-extern int hfs_bmap_reserve(struct hfs_btree *, int);
-extern struct hfs_bnode * hfs_bmap_alloc(struct hfs_btree *);
+extern struct hfs_btree *hfs_btree_open(struct super_block *sb, u32 id,
+ btree_keycmp keycmp);
+extern void hfs_btree_close(struct hfs_btree *tree);
+extern void hfs_btree_write(struct hfs_btree *tree);
+extern int hfs_bmap_reserve(struct hfs_btree *tree, u32 rsvd_nodes);
+extern struct hfs_bnode *hfs_bmap_alloc(struct hfs_btree *tree);
extern void hfs_bmap_free(struct hfs_bnode *node);
/* bnode.c */
-extern void hfs_bnode_read(struct hfs_bnode *, void *, int, int);
-extern u16 hfs_bnode_read_u16(struct hfs_bnode *, int);
-extern u8 hfs_bnode_read_u8(struct hfs_bnode *, int);
-extern void hfs_bnode_read_key(struct hfs_bnode *, void *, int);
-extern void hfs_bnode_write(struct hfs_bnode *, void *, int, int);
-extern void hfs_bnode_write_u16(struct hfs_bnode *, int, u16);
-extern void hfs_bnode_write_u8(struct hfs_bnode *, int, u8);
-extern void hfs_bnode_clear(struct hfs_bnode *, int, int);
-extern void hfs_bnode_copy(struct hfs_bnode *, int,
- struct hfs_bnode *, int, int);
-extern void hfs_bnode_move(struct hfs_bnode *, int, int, int);
-extern void hfs_bnode_dump(struct hfs_bnode *);
-extern void hfs_bnode_unlink(struct hfs_bnode *);
-extern struct hfs_bnode *hfs_bnode_findhash(struct hfs_btree *, u32);
-extern struct hfs_bnode *hfs_bnode_find(struct hfs_btree *, u32);
-extern void hfs_bnode_unhash(struct hfs_bnode *);
-extern void hfs_bnode_free(struct hfs_bnode *);
-extern struct hfs_bnode *hfs_bnode_create(struct hfs_btree *, u32);
-extern void hfs_bnode_get(struct hfs_bnode *);
-extern void hfs_bnode_put(struct hfs_bnode *);
+extern void hfs_bnode_read(struct hfs_bnode *node, void *buf, u32 off, u32 len);
+extern u16 hfs_bnode_read_u16(struct hfs_bnode *node, u32 off);
+extern u8 hfs_bnode_read_u8(struct hfs_bnode *node, u32 off);
+extern void hfs_bnode_read_key(struct hfs_bnode *node, void *key, u32 off);
+extern void hfs_bnode_write(struct hfs_bnode *node, void *buf, u32 off, u32 len);
+extern void hfs_bnode_write_u16(struct hfs_bnode *node, u32 off, u16 data);
+extern void hfs_bnode_write_u8(struct hfs_bnode *node, u32 off, u8 data);
+extern void hfs_bnode_clear(struct hfs_bnode *node, u32 off, u32 len);
+extern void hfs_bnode_copy(struct hfs_bnode *dst_node, u32 dst,
+ struct hfs_bnode *src_node, u32 src, u32 len);
+extern void hfs_bnode_move(struct hfs_bnode *node, u32 dst, u32 src, u32 len);
+extern void hfs_bnode_dump(struct hfs_bnode *node);
+extern void hfs_bnode_unlink(struct hfs_bnode *node);
+extern struct hfs_bnode *hfs_bnode_findhash(struct hfs_btree *tree, u32 cnid);
+extern struct hfs_bnode *hfs_bnode_find(struct hfs_btree *tree, u32 num);
+extern void hfs_bnode_unhash(struct hfs_bnode *node);
+extern void hfs_bnode_free(struct hfs_bnode *node);
+extern struct hfs_bnode *hfs_bnode_create(struct hfs_btree *tree, u32 num);
+extern void hfs_bnode_get(struct hfs_bnode *node);
+extern void hfs_bnode_put(struct hfs_bnode *node);
/* brec.c */
-extern u16 hfs_brec_lenoff(struct hfs_bnode *, u16, u16 *);
-extern u16 hfs_brec_keylen(struct hfs_bnode *, u16);
-extern int hfs_brec_insert(struct hfs_find_data *, void *, int);
-extern int hfs_brec_remove(struct hfs_find_data *);
+extern u16 hfs_brec_lenoff(struct hfs_bnode *node, u16 rec, u16 *off);
+extern u16 hfs_brec_keylen(struct hfs_bnode *node, u16 rec);
+extern int hfs_brec_insert(struct hfs_find_data *fd, void *entry, u32 entry_len);
+extern int hfs_brec_remove(struct hfs_find_data *fd);
/* bfind.c */
-extern int hfs_find_init(struct hfs_btree *, struct hfs_find_data *);
-extern void hfs_find_exit(struct hfs_find_data *);
-extern int __hfs_brec_find(struct hfs_bnode *, struct hfs_find_data *);
-extern int hfs_brec_find(struct hfs_find_data *);
-extern int hfs_brec_read(struct hfs_find_data *, void *, int);
-extern int hfs_brec_goto(struct hfs_find_data *, int);
+extern int hfs_find_init(struct hfs_btree *tree, struct hfs_find_data *fd);
+extern void hfs_find_exit(struct hfs_find_data *fd);
+extern int __hfs_brec_find(struct hfs_bnode *bnode, struct hfs_find_data *fd);
+extern int hfs_brec_find(struct hfs_find_data *fd);
+extern int hfs_brec_read(struct hfs_find_data *fd, void *rec, u32 rec_len);
+extern int hfs_brec_goto(struct hfs_find_data *fd, int cnt);
struct hfs_bnode_desc {
diff --git a/fs/hfs/hfs_fs.h b/fs/hfs/hfs_fs.h
index 732c5c4c7545..5d287f50b43a 100644
--- a/fs/hfs/hfs_fs.h
+++ b/fs/hfs/hfs_fs.h
@@ -171,74 +171,90 @@ struct hfs_sb_info {
#define HFS_FLG_ALT_MDB_DIRTY 2
/* bitmap.c */
-extern u32 hfs_vbm_search_free(struct super_block *, u32, u32 *);
-extern int hfs_clear_vbm_bits(struct super_block *, u16, u16);
+extern u32 hfs_vbm_search_free(struct super_block *sb, u32 goal, u32 *num_bits);
+extern int hfs_clear_vbm_bits(struct super_block *sb, u16 start, u16 count);
/* catalog.c */
-extern int hfs_cat_keycmp(const btree_key *, const btree_key *);
+extern int hfs_cat_keycmp(const btree_key *key1, const btree_key *key2);
struct hfs_find_data;
-extern int hfs_cat_find_brec(struct super_block *, u32, struct hfs_find_data *);
-extern int hfs_cat_create(u32, struct inode *, const struct qstr *, struct inode *);
-extern int hfs_cat_delete(u32, struct inode *, const struct qstr *);
-extern int hfs_cat_move(u32, struct inode *, const struct qstr *,
- struct inode *, const struct qstr *);
-extern void hfs_cat_build_key(struct super_block *, btree_key *, u32, const struct qstr *);
+extern int hfs_cat_find_brec(struct super_block *sb, u32 cnid,
+ struct hfs_find_data *fd);
+extern int hfs_cat_create(u32 cnid, struct inode *dir,
+ const struct qstr *str, struct inode *inode);
+extern int hfs_cat_delete(u32 cnid, struct inode *dir, const struct qstr *str);
+extern int hfs_cat_move(u32 cnid, struct inode *src_dir,
+ const struct qstr *src_name,
+ struct inode *dst_dir,
+ const struct qstr *dst_name);
+extern void hfs_cat_build_key(struct super_block *sb, btree_key *key,
+ u32 parent, const struct qstr *name);
/* dir.c */
extern const struct file_operations hfs_dir_operations;
extern const struct inode_operations hfs_dir_inode_operations;
/* extent.c */
-extern int hfs_ext_keycmp(const btree_key *, const btree_key *);
+extern int hfs_ext_keycmp(const btree_key *key1, const btree_key *key2);
extern u16 hfs_ext_find_block(struct hfs_extent *ext, u16 off);
-extern int hfs_free_fork(struct super_block *, struct hfs_cat_file *, int);
-extern int hfs_ext_write_extent(struct inode *);
-extern int hfs_extend_file(struct inode *);
-extern void hfs_file_truncate(struct inode *);
+extern int hfs_free_fork(struct super_block *sb,
+ struct hfs_cat_file *file, int type);
+extern int hfs_ext_write_extent(struct inode *inode);
+extern int hfs_extend_file(struct inode *inode);
+extern void hfs_file_truncate(struct inode *inode);
-extern int hfs_get_block(struct inode *, sector_t, struct buffer_head *, int);
+extern int hfs_get_block(struct inode *inode, sector_t block,
+ struct buffer_head *bh_result, int create);
/* inode.c */
extern const struct address_space_operations hfs_aops;
extern const struct address_space_operations hfs_btree_aops;
int hfs_write_begin(struct file *file, struct address_space *mapping,
- loff_t pos, unsigned len, struct folio **foliop, void **fsdata);
-extern struct inode *hfs_new_inode(struct inode *, const struct qstr *, umode_t);
-extern void hfs_inode_write_fork(struct inode *, struct hfs_extent *, __be32 *, __be32 *);
-extern int hfs_write_inode(struct inode *, struct writeback_control *);
-extern int hfs_inode_setattr(struct mnt_idmap *, struct dentry *,
- struct iattr *);
+ loff_t pos, unsigned int len, struct folio **foliop,
+ void **fsdata);
+extern struct inode *hfs_new_inode(struct inode *dir, const struct qstr *name,
+ umode_t mode);
+extern void hfs_inode_write_fork(struct inode *inode, struct hfs_extent *ext,
+ __be32 *log_size, __be32 *phys_size);
+extern int hfs_write_inode(struct inode *inode, struct writeback_control *wbc);
+extern int hfs_inode_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
+ struct iattr *attr);
extern void hfs_inode_read_fork(struct inode *inode, struct hfs_extent *ext,
- __be32 log_size, __be32 phys_size, u32 clump_size);
-extern struct inode *hfs_iget(struct super_block *, struct hfs_cat_key *, hfs_cat_rec *);
-extern void hfs_evict_inode(struct inode *);
-extern void hfs_delete_inode(struct inode *);
+ __be32 __log_size, __be32 phys_size,
+ u32 clump_size);
+extern struct inode *hfs_iget(struct super_block *sb, struct hfs_cat_key *key,
+ hfs_cat_rec *rec);
+extern void hfs_evict_inode(struct inode *inode);
+extern void hfs_delete_inode(struct inode *inode);
/* attr.c */
extern const struct xattr_handler * const hfs_xattr_handlers[];
/* mdb.c */
-extern int hfs_mdb_get(struct super_block *);
-extern void hfs_mdb_commit(struct super_block *);
-extern void hfs_mdb_close(struct super_block *);
-extern void hfs_mdb_put(struct super_block *);
+extern int hfs_mdb_get(struct super_block *sb);
+extern void hfs_mdb_commit(struct super_block *sb);
+extern void hfs_mdb_close(struct super_block *sb);
+extern void hfs_mdb_put(struct super_block *sb);
/* part_tbl.c */
-extern int hfs_part_find(struct super_block *, sector_t *, sector_t *);
+extern int hfs_part_find(struct super_block *sb,
+ sector_t *part_start, sector_t *part_size);
/* string.c */
extern const struct dentry_operations hfs_dentry_operations;
-extern int hfs_hash_dentry(const struct dentry *, struct qstr *);
-extern int hfs_strcmp(const unsigned char *, unsigned int,
- const unsigned char *, unsigned int);
+extern int hfs_hash_dentry(const struct dentry *dentry, struct qstr *this);
+extern int hfs_strcmp(const unsigned char *s1, unsigned int len1,
+ const unsigned char *s2, unsigned int len2);
extern int hfs_compare_dentry(const struct dentry *dentry,
- unsigned int len, const char *str, const struct qstr *name);
+ unsigned int len, const char *str,
+ const struct qstr *name);
/* trans.c */
-extern void hfs_asc2mac(struct super_block *, struct hfs_name *, const struct qstr *);
-extern int hfs_mac2asc(struct super_block *, char *, const struct hfs_name *);
+extern void hfs_asc2mac(struct super_block *sb,
+ struct hfs_name *out, const struct qstr *in);
+extern int hfs_mac2asc(struct super_block *sb,
+ char *out, const struct hfs_name *in);
/* super.c */
extern void hfs_mark_mdb_dirty(struct super_block *sb);
diff --git a/fs/hfs/inode.c b/fs/hfs/inode.c
index 451115360f73..483f3f1e209b 100644
--- a/fs/hfs/inode.c
+++ b/fs/hfs/inode.c
@@ -45,7 +45,8 @@ static void hfs_write_failed(struct address_space *mapping, loff_t to)
}
int hfs_write_begin(struct file *file, struct address_space *mapping,
- loff_t pos, unsigned len, struct folio **foliop, void **fsdata)
+ loff_t pos, unsigned int len, struct folio **foliop,
+ void **fsdata)
{
int ret;
diff --git a/fs/hfsplus/bfind.c b/fs/hfsplus/bfind.c
index 41f4f56f90fa..094ce196a4d4 100644
--- a/fs/hfsplus/bfind.c
+++ b/fs/hfsplus/bfind.c
@@ -210,7 +210,7 @@ int hfs_brec_find(struct hfs_find_data *fd, search_strategy_t do_key_compare)
return res;
}
-int hfs_brec_read(struct hfs_find_data *fd, void *rec, int rec_len)
+int hfs_brec_read(struct hfs_find_data *fd, void *rec, u32 rec_len)
{
int res;
diff --git a/fs/hfsplus/bnode.c b/fs/hfsplus/bnode.c
index eb565c0db000..5bd5d3ae40b0 100644
--- a/fs/hfsplus/bnode.c
+++ b/fs/hfsplus/bnode.c
@@ -20,10 +20,10 @@
/* Copy a specified range of bytes from the raw data of a node */
-void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len)
+void hfs_bnode_read(struct hfs_bnode *node, void *buf, u32 off, u32 len)
{
struct page **pagep;
- int l;
+ u32 l;
memset(buf, 0, len);
@@ -33,7 +33,7 @@ void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len)
if (len == 0) {
pr_err("requested zero length: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, len %d\n",
+ "node_size %u, offset %u, len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len);
return;
@@ -45,17 +45,17 @@ void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len)
pagep = node->page + (off >> PAGE_SHIFT);
off &= ~PAGE_MASK;
- l = min_t(int, len, PAGE_SIZE - off);
+ l = min_t(u32, len, PAGE_SIZE - off);
memcpy_from_page(buf, *pagep, off, l);
while ((len -= l) != 0) {
buf += l;
- l = min_t(int, len, PAGE_SIZE);
+ l = min_t(u32, len, PAGE_SIZE);
memcpy_from_page(buf, *++pagep, 0, l);
}
}
-u16 hfs_bnode_read_u16(struct hfs_bnode *node, int off)
+u16 hfs_bnode_read_u16(struct hfs_bnode *node, u32 off)
{
__be16 data;
/* TODO: optimize later... */
@@ -63,7 +63,7 @@ u16 hfs_bnode_read_u16(struct hfs_bnode *node, int off)
return be16_to_cpu(data);
}
-u8 hfs_bnode_read_u8(struct hfs_bnode *node, int off)
+u8 hfs_bnode_read_u8(struct hfs_bnode *node, u32 off)
{
u8 data;
/* TODO: optimize later... */
@@ -71,10 +71,10 @@ u8 hfs_bnode_read_u8(struct hfs_bnode *node, int off)
return data;
}
-void hfs_bnode_read_key(struct hfs_bnode *node, void *key, int off)
+void hfs_bnode_read_key(struct hfs_bnode *node, void *key, u32 off)
{
struct hfs_btree *tree;
- int key_len;
+ u32 key_len;
tree = node->tree;
if (node->type == HFS_NODE_LEAF ||
@@ -86,17 +86,17 @@ void hfs_bnode_read_key(struct hfs_bnode *node, void *key, int off)
if (key_len > sizeof(hfsplus_btree_key) || key_len < 1) {
memset(key, 0, sizeof(hfsplus_btree_key));
- pr_err("hfsplus: Invalid key length: %d\n", key_len);
+ pr_err("hfsplus: Invalid key length: %u\n", key_len);
return;
}
hfs_bnode_read(node, key, off, key_len);
}
-void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len)
+void hfs_bnode_write(struct hfs_bnode *node, void *buf, u32 off, u32 len)
{
struct page **pagep;
- int l;
+ u32 l;
if (!is_bnode_offset_valid(node, off))
return;
@@ -104,7 +104,7 @@ void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len)
if (len == 0) {
pr_err("requested zero length: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, len %d\n",
+ "node_size %u, offset %u, len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len);
return;
@@ -116,29 +116,29 @@ void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len)
pagep = node->page + (off >> PAGE_SHIFT);
off &= ~PAGE_MASK;
- l = min_t(int, len, PAGE_SIZE - off);
+ l = min_t(u32, len, PAGE_SIZE - off);
memcpy_to_page(*pagep, off, buf, l);
set_page_dirty(*pagep);
while ((len -= l) != 0) {
buf += l;
- l = min_t(int, len, PAGE_SIZE);
+ l = min_t(u32, len, PAGE_SIZE);
memcpy_to_page(*++pagep, 0, buf, l);
set_page_dirty(*pagep);
}
}
-void hfs_bnode_write_u16(struct hfs_bnode *node, int off, u16 data)
+void hfs_bnode_write_u16(struct hfs_bnode *node, u32 off, u16 data)
{
__be16 v = cpu_to_be16(data);
/* TODO: optimize later... */
hfs_bnode_write(node, &v, off, 2);
}
-void hfs_bnode_clear(struct hfs_bnode *node, int off, int len)
+void hfs_bnode_clear(struct hfs_bnode *node, u32 off, u32 len)
{
struct page **pagep;
- int l;
+ u32 l;
if (!is_bnode_offset_valid(node, off))
return;
@@ -146,7 +146,7 @@ void hfs_bnode_clear(struct hfs_bnode *node, int off, int len)
if (len == 0) {
pr_err("requested zero length: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, len %d\n",
+ "node_size %u, offset %u, len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len);
return;
@@ -158,22 +158,22 @@ void hfs_bnode_clear(struct hfs_bnode *node, int off, int len)
pagep = node->page + (off >> PAGE_SHIFT);
off &= ~PAGE_MASK;
- l = min_t(int, len, PAGE_SIZE - off);
+ l = min_t(u32, len, PAGE_SIZE - off);
memzero_page(*pagep, off, l);
set_page_dirty(*pagep);
while ((len -= l) != 0) {
- l = min_t(int, len, PAGE_SIZE);
+ l = min_t(u32, len, PAGE_SIZE);
memzero_page(*++pagep, 0, l);
set_page_dirty(*pagep);
}
}
-void hfs_bnode_copy(struct hfs_bnode *dst_node, int dst,
- struct hfs_bnode *src_node, int src, int len)
+void hfs_bnode_copy(struct hfs_bnode *dst_node, u32 dst,
+ struct hfs_bnode *src_node, u32 src, u32 len)
{
struct page **src_page, **dst_page;
- int l;
+ u32 l;
hfs_dbg(BNODE_MOD, "copybytes: %u,%u,%u\n", dst, src, len);
if (!len)
@@ -190,12 +190,12 @@ void hfs_bnode_copy(struct hfs_bnode *dst_node, int dst,
dst &= ~PAGE_MASK;
if (src == dst) {
- l = min_t(int, len, PAGE_SIZE - src);
+ l = min_t(u32, len, PAGE_SIZE - src);
memcpy_page(*dst_page, src, *src_page, src, l);
set_page_dirty(*dst_page);
while ((len -= l) != 0) {
- l = min_t(int, len, PAGE_SIZE);
+ l = min_t(u32, len, PAGE_SIZE);
memcpy_page(*++dst_page, 0, *++src_page, 0, l);
set_page_dirty(*dst_page);
}
@@ -227,11 +227,11 @@ void hfs_bnode_copy(struct hfs_bnode *dst_node, int dst,
}
}
-void hfs_bnode_move(struct hfs_bnode *node, int dst, int src, int len)
+void hfs_bnode_move(struct hfs_bnode *node, u32 dst, u32 src, u32 len)
{
struct page **src_page, **dst_page;
void *src_ptr, *dst_ptr;
- int l;
+ u32 l;
hfs_dbg(BNODE_MOD, "movebytes: %u,%u,%u\n", dst, src, len);
if (!len)
@@ -301,7 +301,7 @@ void hfs_bnode_move(struct hfs_bnode *node, int dst, int src, int len)
dst &= ~PAGE_MASK;
if (src == dst) {
- l = min_t(int, len, PAGE_SIZE - src);
+ l = min_t(u32, len, PAGE_SIZE - src);
dst_ptr = kmap_local_page(*dst_page) + src;
src_ptr = kmap_local_page(*src_page) + src;
@@ -311,7 +311,7 @@ void hfs_bnode_move(struct hfs_bnode *node, int dst, int src, int len)
kunmap_local(dst_ptr);
while ((len -= l) != 0) {
- l = min_t(int, len, PAGE_SIZE);
+ l = min_t(u32, len, PAGE_SIZE);
dst_ptr = kmap_local_page(*++dst_page);
src_ptr = kmap_local_page(*++src_page);
memmove(dst_ptr, src_ptr, l);
diff --git a/fs/hfsplus/brec.c b/fs/hfsplus/brec.c
index 1918544a7871..53e4fa3f608b 100644
--- a/fs/hfsplus/brec.c
+++ b/fs/hfsplus/brec.c
@@ -60,7 +60,7 @@ u16 hfs_brec_keylen(struct hfs_bnode *node, u16 rec)
return retval;
}
-int hfs_brec_insert(struct hfs_find_data *fd, void *entry, int entry_len)
+int hfs_brec_insert(struct hfs_find_data *fd, void *entry, u32 entry_len)
{
struct hfs_btree *tree;
struct hfs_bnode *node, *new_node;
diff --git a/fs/hfsplus/btree.c b/fs/hfsplus/btree.c
index fe6a54c4083c..72c593308bba 100644
--- a/fs/hfsplus/btree.c
+++ b/fs/hfsplus/btree.c
@@ -344,7 +344,7 @@ static struct hfs_bnode *hfs_bmap_new_bmap(struct hfs_bnode *prev, u32 idx)
}
/* Make sure @tree has enough space for the @rsvd_nodes */
-int hfs_bmap_reserve(struct hfs_btree *tree, int rsvd_nodes)
+int hfs_bmap_reserve(struct hfs_btree *tree, u32 rsvd_nodes)
{
struct inode *inode = tree->inode;
struct hfsplus_inode_info *hip = HFSPLUS_I(inode);
diff --git a/fs/hfsplus/hfsplus_fs.h b/fs/hfsplus/hfsplus_fs.h
index af264f8dc536..11cef30803d5 100644
--- a/fs/hfsplus/hfsplus_fs.h
+++ b/fs/hfsplus/hfsplus_fs.h
@@ -389,21 +389,21 @@ u32 hfsplus_calc_btree_clump_size(u32 block_size, u32 node_size, u64 sectors,
struct hfs_btree *hfs_btree_open(struct super_block *sb, u32 id);
void hfs_btree_close(struct hfs_btree *tree);
int hfs_btree_write(struct hfs_btree *tree);
-int hfs_bmap_reserve(struct hfs_btree *tree, int rsvd_nodes);
+int hfs_bmap_reserve(struct hfs_btree *tree, u32 rsvd_nodes);
struct hfs_bnode *hfs_bmap_alloc(struct hfs_btree *tree);
void hfs_bmap_free(struct hfs_bnode *node);
/* bnode.c */
-void hfs_bnode_read(struct hfs_bnode *node, void *buf, int off, int len);
-u16 hfs_bnode_read_u16(struct hfs_bnode *node, int off);
-u8 hfs_bnode_read_u8(struct hfs_bnode *node, int off);
-void hfs_bnode_read_key(struct hfs_bnode *node, void *key, int off);
-void hfs_bnode_write(struct hfs_bnode *node, void *buf, int off, int len);
-void hfs_bnode_write_u16(struct hfs_bnode *node, int off, u16 data);
-void hfs_bnode_clear(struct hfs_bnode *node, int off, int len);
-void hfs_bnode_copy(struct hfs_bnode *dst_node, int dst,
- struct hfs_bnode *src_node, int src, int len);
-void hfs_bnode_move(struct hfs_bnode *node, int dst, int src, int len);
+void hfs_bnode_read(struct hfs_bnode *node, void *buf, u32 off, u32 len);
+u16 hfs_bnode_read_u16(struct hfs_bnode *node, u32 off);
+u8 hfs_bnode_read_u8(struct hfs_bnode *node, u32 off);
+void hfs_bnode_read_key(struct hfs_bnode *node, void *key, u32 off);
+void hfs_bnode_write(struct hfs_bnode *node, void *buf, u32 off, u32 len);
+void hfs_bnode_write_u16(struct hfs_bnode *node, u32 off, u16 data);
+void hfs_bnode_clear(struct hfs_bnode *node, u32 off, u32 len);
+void hfs_bnode_copy(struct hfs_bnode *dst_node, u32 dst,
+ struct hfs_bnode *src_node, u32 src, u32 len);
+void hfs_bnode_move(struct hfs_bnode *node, u32 dst, u32 src, u32 len);
void hfs_bnode_dump(struct hfs_bnode *node);
void hfs_bnode_unlink(struct hfs_bnode *node);
struct hfs_bnode *hfs_bnode_findhash(struct hfs_btree *tree, u32 cnid);
@@ -418,7 +418,7 @@ bool hfs_bnode_need_zeroout(struct hfs_btree *tree);
/* brec.c */
u16 hfs_brec_lenoff(struct hfs_bnode *node, u16 rec, u16 *off);
u16 hfs_brec_keylen(struct hfs_bnode *node, u16 rec);
-int hfs_brec_insert(struct hfs_find_data *fd, void *entry, int entry_len);
+int hfs_brec_insert(struct hfs_find_data *fd, void *entry, u32 entry_len);
int hfs_brec_remove(struct hfs_find_data *fd);
/* bfind.c */
@@ -431,7 +431,7 @@ int hfs_find_rec_by_key(struct hfs_bnode *bnode, struct hfs_find_data *fd,
int __hfs_brec_find(struct hfs_bnode *bnode, struct hfs_find_data *fd,
search_strategy_t rec_found);
int hfs_brec_find(struct hfs_find_data *fd, search_strategy_t do_key_compare);
-int hfs_brec_read(struct hfs_find_data *fd, void *rec, int rec_len);
+int hfs_brec_read(struct hfs_find_data *fd, void *rec, u32 rec_len);
int hfs_brec_goto(struct hfs_find_data *fd, int cnt);
/* catalog.c */
@@ -589,14 +589,14 @@ hfsplus_btree_lock_class(struct hfs_btree *tree)
}
static inline
-bool is_bnode_offset_valid(struct hfs_bnode *node, int off)
+bool is_bnode_offset_valid(struct hfs_bnode *node, u32 off)
{
bool is_valid = off < node->tree->node_size;
if (!is_valid) {
pr_err("requested invalid offset: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d\n",
+ "node_size %u, offset %u\n",
node->this, node->type, node->height,
node->tree->node_size, off);
}
@@ -605,7 +605,7 @@ bool is_bnode_offset_valid(struct hfs_bnode *node, int off)
}
static inline
-int check_and_correct_requested_length(struct hfs_bnode *node, int off, int len)
+u32 check_and_correct_requested_length(struct hfs_bnode *node, u32 off, u32 len)
{
unsigned int node_size;
@@ -614,13 +614,13 @@ int check_and_correct_requested_length(struct hfs_bnode *node, int off, int len)
node_size = node->tree->node_size;
- if ((off + len) > node_size) {
- int new_len = (int)node_size - off;
+ if ((u64)off + len > node_size) {
+ u32 new_len = node_size - off;
pr_err("requested length has been corrected: "
"NODE: id %u, type %#x, height %u, "
- "node_size %u, offset %d, "
- "requested_len %d, corrected_len %d\n",
+ "node_size %u, offset %u, "
+ "requested_len %u, corrected_len %u\n",
node->this, node->type, node->height,
node->tree->node_size, off, len, new_len);
diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c
index e64a7487e315..c7b042c67a75 100644
--- a/fs/jbd2/journal.c
+++ b/fs/jbd2/journal.c
@@ -2273,6 +2273,8 @@ jbd2_journal_initialize_fast_commit(journal_t *journal)
unsigned long long num_fc_blks;
num_fc_blks = jbd2_journal_get_num_fc_blks(sb);
+ if (num_fc_blks > journal->j_last)
+ return -EFSCORRUPTED;
if (journal->j_last - num_fc_blks < JBD2_MIN_JOURNAL_BLOCKS)
return -ENOSPC;
diff --git a/fs/lockd/svcsubs.c b/fs/lockd/svcsubs.c
index 9103896164f6..1a97f69397c0 100644
--- a/fs/lockd/svcsubs.c
+++ b/fs/lockd/svcsubs.c
@@ -123,6 +123,8 @@ nlm_lookup_file(struct svc_rqst *rqstp, struct nlm_file **result,
mutex_lock(&file->f_mutex);
nfserr = nlm_do_fopen(rqstp, file, mode);
mutex_unlock(&file->f_mutex);
+ if (nfserr)
+ goto out_unlock;
goto found;
}
nlm_debug_print_fh("creating file for", &lock->fh);
@@ -139,7 +141,7 @@ nlm_lookup_file(struct svc_rqst *rqstp, struct nlm_file **result,
nfserr = nlm_do_fopen(rqstp, file, mode);
if (nfserr)
- goto out_unlock;
+ goto out_free;
hlist_add_head(&file->f_list, &nlm_files[hash]);
diff --git a/fs/minix/minix.h b/fs/minix/minix.h
index d54273c3c9ff..c81e48badae9 100644
--- a/fs/minix/minix.h
+++ b/fs/minix/minix.h
@@ -91,7 +91,7 @@ static inline struct minix_inode_info *minix_i(struct inode *inode)
static inline unsigned minix_blocks_needed(unsigned bits, unsigned blocksize)
{
- return DIV_ROUND_UP(bits, blocksize * 8);
+ return DIV_ROUND_UP_POW2(bits, blocksize * 8);
}
#if defined(CONFIG_MINIX_FS_NATIVE_ENDIAN) && \
diff --git a/fs/namei.c b/fs/namei.c
index 1eb20cb5e58f..ffa68aa996a2 100644
--- a/fs/namei.c
+++ b/fs/namei.c
@@ -125,6 +125,11 @@
#define EMBEDDED_NAME_MAX (PATH_MAX - offsetof(struct filename, iname))
+/*
+ * Type of the last component on LOOKUP_PARENT
+ */
+enum {LAST_NORM, LAST_ROOT, LAST_DOT, LAST_DOTDOT};
+
struct filename *
getname_flags(const char __user *filename, int flags)
{
@@ -2736,15 +2741,22 @@ EXPORT_SYMBOL(kern_path);
* @flags: lookup flags
* @parent: pointer to struct path to fill
* @last: last component
- * @type: type of the last component
* @root: pointer to struct path of the base directory
*/
int vfs_path_parent_lookup(struct filename *filename, unsigned int flags,
- struct path *parent, struct qstr *last, int *type,
+ struct path *parent, struct qstr *last,
const struct path *root)
{
- return __filename_parentat(AT_FDCWD, filename, flags, parent, last,
- type, root);
+ int type;
+ int err = __filename_parentat(AT_FDCWD, filename, flags, parent, last,
+ &type, root);
+ if (err)
+ return err;
+ if (unlikely(type != LAST_NORM)) {
+ path_put(parent);
+ return -EINVAL;
+ }
+ return 0;
}
EXPORT_SYMBOL(vfs_path_parent_lookup);
@@ -3818,6 +3830,10 @@ int vfs_tmpfile(struct mnt_idmap *idmap,
int error;
int open_flag = file->f_flags;
+ /* A tmpfile is I_LINKABLE, so guard its owner like may_o_create(). */
+ if (!fsuidgid_has_mapping(dir->i_sb, idmap))
+ return -EOVERFLOW;
+
/* we want directory to be writable */
error = inode_permission(idmap, dir, MAY_WRITE | MAY_EXEC);
if (error)
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 3b418fed4602..5f3073ad4208 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -190,14 +190,13 @@ static void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error
{
struct netfs_io_subrequest *subreq = priv;
- if (transferred_or_error < 0) {
- netfs_read_subreq_terminated(subreq, transferred_or_error, was_async);
- return;
- }
-
- if (transferred_or_error > 0)
+ if (transferred_or_error > 0) {
subreq->transferred += transferred_or_error;
- netfs_read_subreq_terminated(subreq, 0, was_async);
+ subreq->error = 0;
+ } else {
+ subreq->error = transferred_or_error;
+ }
+ netfs_read_subreq_terminated(subreq, was_async);
}
/*
@@ -297,7 +296,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
if (slice < 0)
goto prep_iter_failed;
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
- netfs_read_subreq_terminated(subreq, 0, false);
+ subreq->error = 0;
+ netfs_read_subreq_terminated(subreq, false);
goto done;
}
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index 31e388ec6e48..eb8411bd0a74 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -191,7 +191,20 @@ struct netfs_io_subrequest *netfs_alloc_subrequest(struct netfs_io_request *rreq
}
memset(subreq, 0, kmem_cache_size(cache));
- INIT_WORK(&subreq->work, NULL);
+
+ switch (rreq->origin) {
+ case NETFS_READAHEAD:
+ case NETFS_READPAGE:
+ case NETFS_READ_GAPS:
+ case NETFS_READ_FOR_WRITE:
+ case NETFS_DIO_READ:
+ INIT_WORK(&subreq->work, netfs_read_subreq_termination_worker);
+ break;
+ default:
+ INIT_WORK(&subreq->work, NULL);
+ break;
+ }
+
INIT_LIST_HEAD(&subreq->rreq_link);
refcount_set(&subreq->ref, 2);
subreq->rreq = rreq;
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index eae415efae24..2955dc31d3c7 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -449,28 +449,26 @@ EXPORT_SYMBOL(netfs_read_subreq_progress);
/**
* netfs_read_subreq_terminated - Note the termination of an I/O operation.
* @subreq: The I/O request that has terminated.
- * @error: Error code indicating type of completion.
- * @was_async: The termination was asynchronous
+ * @was_async: True if we're in an asynchronous context.
*
* This tells the read helper that a contributory I/O operation has terminated,
* one way or another, and that it should integrate the results.
*
- * The caller indicates the outcome of the operation through @error, supplying
- * 0 to indicate a successful or retryable transfer (if NETFS_SREQ_NEED_RETRY
- * is set) or a negative error code. The helper will look after reissuing I/O
- * operations as appropriate and writing downloaded data to the cache.
+ * The caller indicates the outcome of the operation through @subreq->error,
+ * supplying 0 to indicate a successful or retryable transfer (if
+ * NETFS_SREQ_NEED_RETRY is set) or a negative error code. The helper will
+ * look after reissuing I/O operations as appropriate and writing downloaded
+ * data to the cache.
*
* Before calling, the filesystem should update subreq->transferred to track
* the amount of data copied into the output buffer.
- *
- * If @was_async is true, the caller might be running in softirq or interrupt
- * context and we can't sleep.
*/
-void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq,
- int error, bool was_async)
+void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq, bool was_async)
{
struct netfs_io_request *rreq = subreq->rreq;
+ might_sleep();
+
switch (subreq->source) {
case NETFS_READ_FROM_CACHE:
netfs_stat(&netfs_n_rh_read_done);
@@ -488,7 +486,7 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq,
* If the read completed validly short, then we can clear the
* tail before going on to unlock the folios.
*/
- if (error == 0 && subreq->transferred < subreq->len &&
+ if (subreq->error == 0 && subreq->transferred < subreq->len &&
(test_bit(NETFS_SREQ_HIT_EOF, &subreq->flags) ||
test_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags))) {
netfs_clear_unread(subreq);
@@ -508,7 +506,7 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq,
/* Deal with retry requests, short reads and errors. If we retry
* but don't make progress, we abandon the attempt.
*/
- if (!error && subreq->transferred < subreq->len) {
+ if (!subreq->error && subreq->transferred < subreq->len) {
if (test_bit(NETFS_SREQ_HIT_EOF, &subreq->flags)) {
trace_netfs_sreq(subreq, netfs_sreq_trace_hit_eof);
} else {
@@ -522,16 +520,15 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq,
set_bit(NETFS_RREQ_NEED_RETRY, &rreq->flags);
} else {
__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
- error = -ENODATA;
+ subreq->error = -ENODATA;
}
}
}
- subreq->error = error;
trace_netfs_sreq(subreq, netfs_sreq_trace_terminated);
- if (unlikely(error < 0)) {
- trace_netfs_failure(rreq, subreq, error, netfs_fail_read);
+ if (unlikely(subreq->error < 0)) {
+ trace_netfs_failure(rreq, subreq, subreq->error, netfs_fail_read);
if (subreq->source == NETFS_READ_FROM_CACHE) {
netfs_stat(&netfs_n_rh_read_failed);
} else {
@@ -547,3 +544,19 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq,
netfs_put_subrequest(subreq, was_async, netfs_sreq_trace_put_terminated);
}
EXPORT_SYMBOL(netfs_read_subreq_terminated);
+
+/**
+ * netfs_read_subreq_termination_worker - Workqueue helper for read termination
+ * @work: The subreq->work in the I/O request that has been terminated.
+ *
+ * Helper function to jump to netfs_read_subreq_terminated() from the
+ * subrequest work item.
+ */
+void netfs_read_subreq_termination_worker(struct work_struct *work)
+{
+ struct netfs_io_subrequest *subreq =
+ container_of(work, struct netfs_io_subrequest, work);
+
+ netfs_read_subreq_terminated(subreq, false);
+}
+EXPORT_SYMBOL(netfs_read_subreq_termination_worker);
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 2789ac4c8027..7431c0a4252c 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -574,8 +574,6 @@ int netfs_writepages(struct address_space *mapping,
}
error = netfs_write_folio(wreq, wbc, folio);
- if (error < 0)
- break;
} while ((folio = writeback_iter(mapping, wbc, folio, &error)));
netfs_end_issue_write(wreq);
@@ -587,7 +585,14 @@ int netfs_writepages(struct address_space *mapping,
return error;
couldnt_start:
- netfs_kill_dirty_pages(mapping, wbc, folio);
+ if (error == -ENOMEM) {
+ folio_redirty_for_writepage(wbc, folio);
+ folio_unlock(folio);
+ folio = writeback_iter(mapping, wbc, folio, &error);
+ WARN_ON_ONCE(folio != NULL);
+ } else {
+ netfs_kill_dirty_pages(mapping, wbc, folio);
+ }
out:
mutex_unlock(&ictx->wb_lock);
_leave(" = %d", error);
diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c
index 7832fb0369a1..b70ae09aeac1 100644
--- a/fs/nfs/callback_proc.c
+++ b/fs/nfs/callback_proc.c
@@ -254,6 +254,7 @@ static u32 initiate_file_draining(struct nfs_client *clp,
struct pnfs_layout_hdr *lo;
u32 rv = NFS4ERR_NOMATCHING_LAYOUT;
LIST_HEAD(free_me_list);
+ bool return_range = false;
ino = nfs_layout_find_inode(clp, &args->cbl_fh, &args->cbl_stateid);
if (IS_ERR(ino)) {
@@ -298,13 +299,13 @@ static u32 initiate_file_draining(struct nfs_client *clp,
/* Embrace your forgetfulness! */
rv = NFS4ERR_NOMATCHING_LAYOUT;
- if (NFS_SERVER(ino)->pnfs_curr_ld->return_range) {
- NFS_SERVER(ino)->pnfs_curr_ld->return_range(lo,
- &args->cbl_range);
- }
+ return_range = true;
}
unlock:
spin_unlock(&ino->i_lock);
+ if (return_range && NFS_SERVER(ino)->pnfs_curr_ld->return_range)
+ NFS_SERVER(ino)->pnfs_curr_ld->return_range(lo,
+ &args->cbl_range);
pnfs_free_lseg_list(&free_me_list);
/* Free all lsegs that are attached to commit buckets */
nfs_commit_inode(ino, 0);
diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c
index 8bdbc4dca89c..c1bbc358833d 100644
--- a/fs/nfs/delegation.c
+++ b/fs/nfs/delegation.c
@@ -149,6 +149,7 @@ int nfs4_check_delegation(struct inode *inode, fmode_t type)
static int nfs_delegation_claim_locks(struct nfs4_state *state, const nfs4_stateid *stateid)
{
struct inode *inode = state->inode;
+ struct nfs_inode *nfsi = NFS_I(inode);
struct file_lock *fl;
struct file_lock_context *flctx = locks_inode_context(inode);
struct list_head *list;
@@ -158,6 +159,9 @@ static int nfs_delegation_claim_locks(struct nfs4_state *state, const nfs4_state
goto out;
list = &flctx->flc_posix;
+
+ /* Guard against reclaim and new lock/unlock calls */
+ down_write(&nfsi->rwsem);
spin_lock(&flctx->flc_lock);
restart:
for_each_file_lock(fl, list) {
@@ -165,8 +169,10 @@ static int nfs_delegation_claim_locks(struct nfs4_state *state, const nfs4_state
continue;
spin_unlock(&flctx->flc_lock);
status = nfs4_lock_delegation_recall(fl, state, stateid);
- if (status < 0)
+ if (status < 0) {
+ up_write(&nfsi->rwsem);
goto out;
+ }
spin_lock(&flctx->flc_lock);
}
if (list == &flctx->flc_posix) {
@@ -174,6 +180,7 @@ static int nfs_delegation_claim_locks(struct nfs4_state *state, const nfs4_state
goto restart;
}
spin_unlock(&flctx->flc_lock);
+ up_write(&nfsi->rwsem);
out:
return status;
}
diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c
index d39a1f58e18d..c064433cf6d6 100644
--- a/fs/nfs/filelayout/filelayout.c
+++ b/fs/nfs/filelayout/filelayout.c
@@ -778,6 +778,8 @@ filelayout_alloc_lseg(struct pnfs_layout_hdr *layoutid,
static bool
filelayout_lseg_is_striped(const struct nfs4_filelayout_segment *flseg)
{
+ if (flseg->dsaddr)
+ return flseg->dsaddr->stripe_count > 1;
return flseg->num_fh > 1;
}
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
index 7e1494cc116b..7de5575f4bee 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.c
+++ b/fs/nfs/flexfilelayout/flexfilelayout.c
@@ -574,6 +574,9 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh,
if (!p)
goto out_sort_mirrors;
fls->flags = be32_to_cpup(p);
+ if (fls->flags & FF_FLAGS_NO_IO_THRU_MDS)
+ set_bit(NFS4_FF_HDR_NO_IO_THRU_MDS,
+ &FF_LAYOUT_FROM_HDR(lh)->flags);
p = xdr_inline_decode(&stream, 4);
if (!p)
@@ -1019,6 +1022,16 @@ ff_layout_pg_get_mirror_count_write(struct nfs_pageio_descriptor *pgio,
0, NFS4_MAX_UINT64, IOMODE_RW,
NFS_I(pgio->pg_inode)->layout,
pgio->pg_lseg);
+ if (NFS_I(pgio->pg_inode)->layout &&
+ ff_layout_hdr_no_fallback_to_mds(NFS_I(pgio->pg_inode)->layout)) {
+ /*
+ * FF_FLAGS_NO_IO_THRU_MDS: no current lseg but the server's
+ * policy forbids MDS fallback. Surface -EAGAIN so writeback
+ * retries rather than silently issuing the WRITE via MDS.
+ */
+ pgio->pg_error = -EAGAIN;
+ goto out;
+ }
/* no lseg means that pnfs is not in use, so no mirroring here */
nfs_pageio_reset_write_mds(pgio);
out:
@@ -1937,6 +1950,14 @@ ff_layout_read_pagelist(struct nfs_pgio_header *hdr)
out_failed:
if (ff_layout_avoid_mds_available_ds(lseg) && !ds_fatal_error)
return PNFS_TRY_AGAIN;
+ if (ff_layout_no_fallback_to_mds(lseg)) {
+ /*
+ * FF_FLAGS_NO_IO_THRU_MDS: force fresh LAYOUTGET,
+ * never fall through to MDS I/O.
+ */
+ pnfs_error_mark_layout_for_return(hdr->inode, lseg);
+ return PNFS_TRY_AGAIN;
+ }
trace_pnfs_mds_fallback_read_pagelist(hdr->inode,
hdr->args.offset, hdr->args.count,
IOMODE_READ, NFS_I(hdr->inode)->layout, lseg);
@@ -2017,6 +2038,14 @@ ff_layout_write_pagelist(struct nfs_pgio_header *hdr, int sync)
out_failed:
if (ff_layout_avoid_mds_available_ds(lseg) && !ds_fatal_error)
return PNFS_TRY_AGAIN;
+ if (ff_layout_no_fallback_to_mds(lseg)) {
+ /*
+ * FF_FLAGS_NO_IO_THRU_MDS: force fresh LAYOUTGET,
+ * never fall through to MDS I/O.
+ */
+ pnfs_error_mark_layout_for_return(hdr->inode, lseg);
+ return PNFS_TRY_AGAIN;
+ }
trace_pnfs_mds_fallback_write_pagelist(hdr->inode,
hdr->args.offset, hdr->args.count,
IOMODE_RW, NFS_I(hdr->inode)->layout, lseg);
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h
index cf688889c918..1561e9ceab53 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.h
+++ b/fs/nfs/flexfilelayout/flexfilelayout.h
@@ -109,12 +109,16 @@ struct nfs4_ff_layout_segment {
struct nfs4_ff_layout_mirror *mirror_array[] __counted_by(mirror_array_cnt);
};
+/* nfs4_flexfile_layout::flags bit indices */
+#define NFS4_FF_HDR_NO_IO_THRU_MDS 0 /* any lseg has had FF_FLAGS_NO_IO_THRU_MDS */
+
struct nfs4_flexfile_layout {
struct pnfs_layout_hdr generic_hdr;
struct pnfs_ds_commit_info commit_info;
struct list_head mirrors;
struct list_head error_list; /* nfs4_ff_layout_ds_err */
ktime_t last_report_time; /* Layoutstat report times */
+ unsigned long flags;
};
struct nfs4_flexfile_layoutreturn_args {
@@ -181,6 +185,18 @@ ff_layout_no_fallback_to_mds(struct pnfs_layout_segment *lseg)
return FF_LAYOUT_LSEG(lseg)->flags & FF_FLAGS_NO_IO_THRU_MDS;
}
+/*
+ * Sticky hdr-level mirror of FF_FLAGS_NO_IO_THRU_MDS so callers that have
+ * no current lseg (e.g. between LAYOUTRETURN and the next LAYOUTGET) can
+ * still honor the no-MDS-fallback policy.
+ */
+static inline bool
+ff_layout_hdr_no_fallback_to_mds(struct pnfs_layout_hdr *lo)
+{
+ return test_bit(NFS4_FF_HDR_NO_IO_THRU_MDS,
+ &FF_LAYOUT_FROM_HDR(lo)->flags);
+}
+
static inline bool
ff_layout_no_read_on_rw(struct pnfs_layout_segment *lseg)
{
diff --git a/fs/nfs/fscache.c b/fs/nfs/fscache.c
index d49e4ce27999..be14d30608f6 100644
--- a/fs/nfs/fscache.c
+++ b/fs/nfs/fscache.c
@@ -314,8 +314,10 @@ static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
&nfs_async_read_completion_ops);
netfs = nfs_netfs_alloc(sreq);
- if (!netfs)
- return netfs_read_subreq_terminated(sreq, -ENOMEM, false);
+ if (!netfs) {
+ sreq->error = -ENOMEM;
+ return netfs_read_subreq_terminated(sreq, false);
+ }
pgio.pg_netfs = netfs; /* used in completion */
diff --git a/fs/nfs/fscache.h b/fs/nfs/fscache.h
index 772d485e96d3..1d86f7cc7195 100644
--- a/fs/nfs/fscache.h
+++ b/fs/nfs/fscache.h
@@ -74,7 +74,8 @@ static inline void nfs_netfs_put(struct nfs_netfs_io_data *netfs)
*/
netfs->sreq->transferred = min_t(s64, netfs->sreq->len,
atomic64_read(&netfs->transferred));
- netfs_read_subreq_terminated(netfs->sreq, netfs->error, false);
+ netfs->sreq->error = netfs->error;
+ netfs_read_subreq_terminated(netfs->sreq, false);
kfree(netfs);
}
static inline void nfs_netfs_inode_init(struct nfs_inode *nfsi)
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index b0ab79894544..4dccacf51750 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -839,17 +839,19 @@ void nfs_super_set_maxbytes(struct super_block *sb, __u64 maxfilesize)
}
/*
- * Record the page as unstable (an extra writeback period) and mark its
- * inode as dirty.
+ * Record the request's range as unstable (an extra writeback period) and
+ * mark its inode as dirty.
*/
-static inline void nfs_folio_mark_unstable(struct folio *folio,
+static inline void nfs_folio_mark_unstable(struct nfs_page *req,
struct nfs_commit_info *cinfo)
{
+ struct folio *folio = nfs_page_to_folio(req);
+
if (folio && !cinfo->dreq) {
struct inode *inode = folio->mapping->host;
- long nr = folio_nr_pages(folio);
+ long nr = DIV_ROUND_UP(req->wb_bytes, PAGE_SIZE);
- /* This page is really still in write-back - just that the
+ /* This range is really still in write-back - just that the
* writeback is happening on the server now.
*/
node_stat_mod_folio(folio, NR_WRITEBACK, nr);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index c76acd537be0..840789e182ef 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -7128,7 +7128,6 @@ static void nfs4_locku_done(struct rpc_task *task, void *data)
switch (task->tk_status) {
case 0:
renew_lease(calldata->server, calldata->timestamp);
- locks_lock_inode_wait(calldata->lsp->ls_state->inode, &calldata->fl);
if (nfs4_update_lock_stateid(calldata->lsp,
&calldata->res.stateid))
break;
@@ -7395,11 +7394,6 @@ static void nfs4_lock_done(struct rpc_task *task, void *calldata)
case 0:
renew_lease(NFS_SERVER(d_inode(data->ctx->dentry)),
data->timestamp);
- if (data->arg.new_lock && !data->cancelled) {
- data->fl.c.flc_flags &= ~(FL_SLEEP | FL_ACCESS);
- if (locks_lock_inode_wait(lsp->ls_state->inode, &data->fl) < 0)
- goto out_restart;
- }
if (data->arg.new_lock_owner != 0) {
nfs_confirm_seqid(&lsp->ls_seqid, 0);
nfs4_stateid_copy(&lsp->ls_stateid, &data->res.stateid);
@@ -7509,11 +7503,10 @@ static int _nfs4_do_setlk(struct nfs4_state *state, int cmd, struct file_lock *f
msg.rpc_argp = &data->arg;
msg.rpc_resp = &data->res;
task_setup_data.callback_data = data;
- if (recovery_type > NFS_LOCK_NEW) {
- if (recovery_type == NFS_LOCK_RECLAIM)
- data->arg.reclaim = NFS_LOCK_RECLAIM;
- } else
- data->arg.new_lock = 1;
+
+ if (recovery_type == NFS_LOCK_RECLAIM)
+ data->arg.reclaim = NFS_LOCK_RECLAIM;
+
task = rpc_run_task(&task_setup_data);
if (IS_ERR(task))
return PTR_ERR(task);
@@ -7625,6 +7618,13 @@ static int _nfs4_proc_setlk(struct nfs4_state *state, int cmd, struct file_lock
up_read(&nfsi->rwsem);
mutex_unlock(&sp->so_delegreturn_mutex);
status = _nfs4_do_setlk(state, cmd, request, NFS_LOCK_NEW);
+ if (status)
+ goto out;
+
+ down_read(&nfsi->rwsem);
+ request->c.flc_flags &= ~(FL_SLEEP | FL_ACCESS);
+ status = locks_lock_inode_wait(state->inode, request);
+ up_read(&nfsi->rwsem);
out:
request->c.flc_flags = flags;
return status;
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 5928d1cb1de2..2a6d109551d3 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -1476,8 +1476,6 @@ _pnfs_return_layout(struct inode *ino)
pnfs_clear_layoutcommit(ino, &tmp_list);
pnfs_mark_matching_lsegs_return(lo, &tmp_list, &range, 0);
- if (NFS_SERVER(ino)->pnfs_curr_ld->return_range)
- NFS_SERVER(ino)->pnfs_curr_ld->return_range(lo, &range);
/* Don't send a LAYOUTRETURN if list was initially empty */
if (!test_bit(NFS_LAYOUT_RETURN_REQUESTED, &lo->plh_flags) ||
@@ -1489,6 +1487,8 @@ _pnfs_return_layout(struct inode *ino)
send = pnfs_prepare_layoutreturn(lo, &stateid, &cred, NULL);
spin_unlock(&ino->i_lock);
+ if (NFS_SERVER(ino)->pnfs_curr_ld->return_range)
+ NFS_SERVER(ino)->pnfs_curr_ld->return_range(lo, &range);
if (send)
status = pnfs_send_layoutreturn(lo, &stateid, &cred, IOMODE_ANY,
0);
diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c
index 39507b2e259b..bf6abca0c29d 100644
--- a/fs/nfs/pnfs_nfs.c
+++ b/fs/nfs/pnfs_nfs.c
@@ -1179,7 +1179,7 @@ pnfs_layout_mark_request_commit(struct nfs_page *req,
nfs_request_add_commit_list_locked(req, list, cinfo);
mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex);
- nfs_folio_mark_unstable(nfs_page_to_folio(req), cinfo);
+ nfs_folio_mark_unstable(req, cinfo);
return;
out_resched:
mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex);
diff --git a/fs/nfs/read.c b/fs/nfs/read.c
index 3c1fa320b3f1..fe41e4272994 100644
--- a/fs/nfs/read.c
+++ b/fs/nfs/read.c
@@ -132,10 +132,32 @@ static void nfs_readpage_release(struct nfs_page *req, int error)
static void nfs_page_group_set_uptodate(struct nfs_page *req)
{
- if (nfs_page_group_sync_on_bit(req, PG_UPTODATE))
+ bool uptodate = false;
+
+ nfs_page_group_lock(req);
+ if (!test_bit(PG_READ_FAILED, &req->wb_head->wb_flags) &&
+ nfs_page_group_sync_on_bit_locked(req, PG_UPTODATE))
+ uptodate = true;
+ nfs_page_group_unlock(req);
+
+ if (uptodate)
folio_mark_uptodate(nfs_page_to_folio(req));
}
+static void nfs_page_group_mark_read_failed(struct nfs_page *req)
+{
+ struct nfs_page *tmp;
+
+ nfs_page_group_lock(req);
+ set_bit(PG_READ_FAILED, &req->wb_head->wb_flags);
+ tmp = req;
+ do {
+ clear_bit(PG_UPTODATE, &tmp->wb_flags);
+ tmp = tmp->wb_this_page;
+ } while (tmp != req);
+ nfs_page_group_unlock(req);
+}
+
static void nfs_read_completion(struct nfs_pgio_header *hdr)
{
unsigned long bytes = 0;
@@ -172,6 +194,7 @@ static void nfs_read_completion(struct nfs_pgio_header *hdr)
if (bytes <= hdr->good_bytes)
nfs_page_group_set_uptodate(req);
else {
+ nfs_page_group_mark_read_failed(req);
error = hdr->error;
xchg(&nfs_req_openctx(req)->error, error);
}
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 48a8866220d1..563b93db3554 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -858,7 +858,7 @@ nfs_request_add_commit_list(struct nfs_page *req, struct nfs_commit_info *cinfo)
mutex_lock(&NFS_I(cinfo->inode)->commit_mutex);
nfs_request_add_commit_list_locked(req, &cinfo->mds->list, cinfo);
mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex);
- nfs_folio_mark_unstable(nfs_page_to_folio(req), cinfo);
+ nfs_folio_mark_unstable(req, cinfo);
}
EXPORT_SYMBOL_GPL(nfs_request_add_commit_list);
@@ -917,10 +917,12 @@ nfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg,
nfs_request_add_commit_list(req, cinfo);
}
-static void nfs_folio_clear_commit(struct folio *folio)
+static void nfs_folio_clear_commit(struct nfs_page *req)
{
+ struct folio *folio = nfs_page_to_folio(req);
+
if (folio) {
- long nr = folio_nr_pages(folio);
+ long nr = DIV_ROUND_UP(req->wb_bytes, PAGE_SIZE);
node_stat_mod_folio(folio, NR_WRITEBACK, -nr);
wb_stat_mod(&inode_to_bdi(folio->mapping->host)->wb,
@@ -941,7 +943,7 @@ static void nfs_clear_request_commit(struct nfs_commit_info *cinfo,
nfs_request_remove_commit_list(req, cinfo);
}
mutex_unlock(&NFS_I(inode)->commit_mutex);
- nfs_folio_clear_commit(nfs_page_to_folio(req));
+ nfs_folio_clear_commit(req);
}
}
@@ -1777,7 +1779,7 @@ void nfs_retry_commit(struct list_head *page_list,
req = nfs_list_entry(page_list->next);
nfs_list_remove_request(req);
nfs_mark_request_commit(req, lseg, cinfo, ds_commit_idx);
- nfs_folio_clear_commit(nfs_page_to_folio(req));
+ nfs_folio_clear_commit(req);
nfs_unlock_and_release_request(req);
}
}
@@ -1848,7 +1850,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data)
req = nfs_list_entry(data->pages.next);
nfs_list_remove_request(req);
folio = nfs_page_to_folio(req);
- nfs_folio_clear_commit(folio);
+ nfs_folio_clear_commit(req);
dprintk("NFS: commit (%s/%llu %d@%lld)",
nfs_req_openctx(req)->dentry->d_sb->s_id,
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 8c13fa68c08a..342de8d2c4aa 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1374,7 +1374,8 @@ static void destroy_delegation(struct nfs4_delegation *dp)
* stateid or it's called from a laundromat thread (nfsd4_landromat()) that
* determined that this specific state has expired and needs to be revoked
* (both mark state with the appropriate stid sc_status mode). It is also
- * assumed that a reference was taken on the @dp state.
+ * assumed that a reference was taken on the @dp state. This function
+ * consumes that reference.
*
* If this function finds that the @dp state is SC_STATUS_FREED it means
* that a FREE_STATEID operation for this stateid has been processed and
@@ -1820,6 +1821,10 @@ void nfsd4_revoke_states(struct nfsd_net *nn, struct super_block *sb)
mutex_unlock(&stp->st_mutex);
break;
case SC_TYPE_DELEG:
+ /* Extra reference guards against concurrent
+ * FREE_STATEID; revoke_delegation() consumes
+ * it, otherwise release it directly.
+ */
refcount_inc(&stid->sc_count);
dp = delegstateid(stid);
spin_lock(&state_lock);
@@ -1829,6 +1834,8 @@ void nfsd4_revoke_states(struct nfsd_net *nn, struct super_block *sb)
spin_unlock(&state_lock);
if (dp)
revoke_delegation(dp);
+ else
+ nfs4_put_stid(stid);
break;
case SC_TYPE_LAYOUT:
ls = layoutstateid(stid);
@@ -4851,6 +4858,7 @@ static void nfsd4_drop_revoked_stid(struct nfs4_stid *s)
{
struct nfs4_client *cl = s->sc_client;
LIST_HEAD(reaplist);
+ struct nfs4_layout_stateid *ls;
struct nfs4_ol_stateid *stp;
struct nfs4_delegation *dp;
bool unhashed;
@@ -4876,6 +4884,12 @@ static void nfsd4_drop_revoked_stid(struct nfs4_stid *s)
spin_unlock(&cl->cl_lock);
nfs4_put_stid(s);
break;
+ case SC_TYPE_LAYOUT:
+ ls = layoutstateid(s);
+ list_del_init(&ls->ls_perclnt);
+ spin_unlock(&cl->cl_lock);
+ nfs4_put_stid(s);
+ break;
default:
spin_unlock(&cl->cl_lock);
}
diff --git a/fs/nilfs2/super.c b/fs/nilfs2/super.c
index eca79cca3803..7ed64df82d1b 100644
--- a/fs/nilfs2/super.c
+++ b/fs/nilfs2/super.c
@@ -1062,8 +1062,6 @@ nilfs_fill_super(struct super_block *sb, struct fs_context *fc)
sb->s_time_gran = 1;
sb->s_max_links = NILFS_LINK_MAX;
- sb->s_bdi = bdi_get(sb->s_bdev->bd_disk->bdi);
-
err = load_nilfs(nilfs, sb);
if (err)
goto failed_nilfs;
diff --git a/fs/ntfs3/fslog.c b/fs/ntfs3/fslog.c
index 02f4b39a5a15..a403b037a001 100644
--- a/fs/ntfs3/fslog.c
+++ b/fs/ntfs3/fslog.c
@@ -764,8 +764,19 @@ static bool check_rstbl(const struct RESTART_TABLE *rt, size_t bytes)
/*
* Walk through the list headed by the first entry to make
* sure none of the entries are currently being used.
+ *
+ * Bound traversal by ne (rt->used) to defeat a crafted on-disk
+ * cycle in the free chain. Each entry in a legitimate free
+ * list is unique, so a chain that visits more than ne slots
+ * is malformed. Without this guard, an attacker-controlled
+ * RESTART_TABLE with a self-loop or A->B->A cycle whose
+ * offsets satisfy the existing alignment + in-bounds guards
+ * spins forever at mount time.
*/
- for (off = ff; off;) {
+ for (off = ff, i = 0; off; i++) {
+ if (i > ne)
+ return false;
+
if (off == RESTART_ENTRY_ALLOCATED)
return false;
@@ -1170,7 +1181,7 @@ static int read_log_page(struct ntfs_log *log, u32 vbo,
goto out;
if (page_buf->rhdr.sign != NTFS_FFFF_SIGNATURE)
- ntfs_fix_post_read(&page_buf->rhdr, PAGE_SIZE, false);
+ ntfs_fix_post_read(&page_buf->rhdr, log->page_size, false);
if (page_buf != *buffer)
memcpy(*buffer, Add2Ptr(page_buf, page_off), bytes);
@@ -3313,6 +3324,17 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
nsize = ALIGN(nsize, 8);
data_off = le16_to_cpu(attr->res.data_off);
+ /*
+ * aoff comes from the on-disk lrh->attr_off. Forbid
+ * writes that begin below the resident attribute's
+ * data_off (which would overwrite the resident header),
+ * and forbid aoff + dlen < data_off, which would make
+ * the data_size assignment below underflow to ~4 GiB.
+ */
+ if (aoff < data_off || aoff + dlen < data_off ||
+ aoff + dlen > asize)
+ goto dirty_vol;
+
if (nsize < asize) {
memmove(Add2Ptr(attr, aoff), data, dlen);
data = NULL; // To skip below memmove().
@@ -3364,7 +3386,10 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
memmove(Add2Ptr(attr, aoff), data, dlen);
if (run_get_highest_vcn(le64_to_cpu(attr->nres.svcn),
- attr_run(attr), &t64)) {
+ attr_run(attr),
+ le32_to_cpu(attr->size) -
+ le16_to_cpu(attr->nres.run_off),
+ &t64)) {
goto dirty_vol;
}
@@ -3498,6 +3523,18 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
e = Add2Ptr(attr, le16_to_cpu(lrh->attr_off));
+ /*
+ * e->view.data_off and dlen come from the on-disk
+ * INDEX_ROOT entry / LRH. The neighbouring read sites
+ * (e.g. fs/ntfs3/index.c) check that
+ * view.data_off + view.data_size <= e->size; mirror that
+ * bound here so the memmove cannot reach past the entry.
+ */
+ if (le16_to_cpu(e->view.data_off) > le16_to_cpu(e->size) ||
+ le16_to_cpu(e->view.data_off) + dlen >
+ le16_to_cpu(e->size))
+ goto dirty_vol;
+
memmove(Add2Ptr(e, le16_to_cpu(e->view.data_off)), data, dlen);
mi->dirty = true;
@@ -3571,9 +3608,23 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
}
e1 = Add2Ptr(e, esize);
- nsize = esize;
used = le32_to_cpu(hdr->used);
+ /*
+ * Reject crafted entries whose e->size makes e + esize
+ * point past the INDEX_HDR's used boundary. Without this,
+ * PtrOffset(e1, hdr + used) underflows to a quasi-infinite
+ * size_t when fed to the memmove() below.
+ *
+ * Also reject esize == 0: memmove(e, e, ...) is a no-op and
+ * leaves hdr->used unchanged, masking the crafted entry.
+ */
+ if (!esize || Add2Ptr(e, esize) > Add2Ptr(hdr, used) ||
+ PtrOffset(e1, Add2Ptr(hdr, used)) < esize)
+ goto dirty_vol;
+
+ nsize = esize;
+
memmove(e, e1, PtrOffset(e1, Add2Ptr(hdr, used)));
hdr->used = cpu_to_le32(used - nsize);
@@ -3690,6 +3741,12 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
goto dirty_vol;
}
+ /* See UpdateRecordDataRoot for the rationale. */
+ if (le16_to_cpu(e->view.data_off) > le16_to_cpu(e->size) ||
+ le16_to_cpu(e->view.data_off) + dlen >
+ le16_to_cpu(e->size))
+ goto dirty_vol;
+
memmove(Add2Ptr(e, le16_to_cpu(e->view.data_off)), data, dlen);
a_dirty = true;
@@ -3797,11 +3854,7 @@ int log_replay(struct ntfs_inode *ni, bool *initialized)
log->l_size = log->orig_file_size = ni->vfs_inode.i_size;
/* Get the size of page. NOTE: To replay we can use default page. */
-#if PAGE_SIZE >= DefaultLogPageSize && PAGE_SIZE <= DefaultLogPageSize * 2
log->page_size = norm_file_page(PAGE_SIZE, &log->l_size, true);
-#else
- log->page_size = norm_file_page(PAGE_SIZE, &log->l_size, false);
-#endif
if (!log->page_size) {
err = -EINVAL;
goto out;
@@ -3939,9 +3992,28 @@ int log_replay(struct ntfs_inode *ni, bool *initialized)
*/
t32 = le32_to_cpu(log->rst_info.r_page->sys_page_size);
if (log->page_size != t32) {
+ u32 old_page_size = log->page_size;
+
log->l_size = log->orig_file_size;
log->page_size = norm_file_page(t32, &log->l_size,
t32 == DefaultLogPageSize);
+
+ /*
+ * If the adopted on-disk page size is larger than the size used
+ * to allocate one_page_buf above, grow the scratch buffer so a
+ * later read_log_page() cannot overflow it.
+ */
+ if (log->page_size > old_page_size) {
+ void *buf;
+
+ buf = krealloc(log->one_page_buf, log->page_size,
+ GFP_NOFS);
+ if (!buf) {
+ err = -ENOMEM;
+ goto out;
+ }
+ log->one_page_buf = buf;
+ }
}
if (log->page_size != t32 ||
@@ -4220,13 +4292,26 @@ int log_replay(struct ntfs_inode *ni, bool *initialized)
if (rst->major_ver)
goto end_conv_1; /* reduce tab pressure. */
+ t16 = le16_to_cpu(dptbl->size);
+ if (t16 < sizeof(struct DIR_PAGE_ENTRY)) {
+ log->set_dirty = true;
+ goto out;
+ }
+
+ t32 = (t16 - sizeof(struct DIR_PAGE_ENTRY)) / sizeof(u64);
+
dp = NULL;
while ((dp = enum_rstbl(dptbl, dp))) {
struct DIR_PAGE_ENTRY_32 *dp0 = (struct DIR_PAGE_ENTRY_32 *)dp;
- // NOTE: Danger. Check for of boundary.
+ u32 lcns = le32_to_cpu(dp->lcns_follow);
+
+ if (lcns > t32) {
+ log->set_dirty = true;
+ goto out;
+ }
+
memmove(&dp->vcn, &dp0->vcn_low,
- 2 * sizeof(u64) +
- le32_to_cpu(dp->lcns_follow) * sizeof(u64));
+ 2 * sizeof(u64) + lcns * sizeof(u64));
}
end_conv_1:
@@ -4548,22 +4633,34 @@ int log_replay(struct ntfs_inode *ni, bool *initialized)
* whole routine a loop, case Lcns do not fit below.
*/
t16 = le16_to_cpu(lrh->lcns_follow);
- t32 = le32_to_cpu(dp->lcns_follow);
- if (le64_to_cpu(lrh->target_vcn) < le64_to_cpu(dp->vcn)) {
- err = -EINVAL;
- goto out;
- }
-
- for (i = 0; i < t16; i++) {
- size_t j = (size_t)(le64_to_cpu(lrh->target_vcn) -
- le64_to_cpu(dp->vcn));
- if (j >= t32 || i >= t32 - j) {
- err = -EINVAL;
- goto out;
- }
- dp->page_lcns[j + i] = lrh->page_lcns[i];
- }
+ t32 = le32_to_cpu(dp->lcns_follow);
+ if (le64_to_cpu(lrh->target_vcn) < le64_to_cpu(dp->vcn)) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ /*
+ * find_dp() only validates that target_vcn is the first
+ * cluster covered by dp. The walk through lrh->lcns_follow
+ * further entries must stay within the allocated
+ * dp->page_lcns[] array, which is sized by dp->lcns_follow.
+ */
+ if (le64_to_cpu(lrh->target_vcn) - le64_to_cpu(dp->vcn) + t16 >
+ le32_to_cpu(dp->lcns_follow)) {
+ err = -EINVAL;
+ log->set_dirty = true;
+ goto out;
+ }
+ for (i = 0; i < t16; i++) {
+ size_t j = (size_t)(le64_to_cpu(lrh->target_vcn) -
+ le64_to_cpu(dp->vcn));
+ if (j >= t32 || i >= t32 - j) {
+ err = -EINVAL;
+ goto out;
+ }
+ dp->page_lcns[j + i] = lrh->page_lcns[i];
+ }
goto next_log_record_analyze;
case DeleteDirtyClusters: {
diff --git a/fs/ntfs3/index.c b/fs/ntfs3/index.c
index 050b3709e020..a9c5dcc23a5c 100644
--- a/fs/ntfs3/index.c
+++ b/fs/ntfs3/index.c
@@ -1743,6 +1743,22 @@ static int indx_insert_into_root(struct ntfs_index *indx, struct ntfs_inode *ni,
hdr_used = le32_to_cpu(hdr->used);
hdr_total = le32_to_cpu(hdr->total);
+ /*
+ * The destination INDEX_BUFFER has 'hdr_total' bytes of payload
+ * available after the header, of which 'hdr_used' are already
+ * consumed by the single terminal END entry installed by
+ * indx_new(). A crafted image can present a resident root whose
+ * non-last entries (summing to 'to_move') exceed what fits in
+ * this buffer; copying them unchecked would overrun the
+ * kmalloc(1u << indx->index_bits) allocation backing the new
+ * buffer. Reject the copy in that case.
+ */
+ if (to_move > hdr_total - hdr_used) {
+ err = -EINVAL;
+ ntfs_set_state(sbi, NTFS_DIRTY_ERROR);
+ goto out_put_n;
+ }
+
/* Copy root entries into new buffer. */
hdr_insert_head(hdr, re, to_move);
@@ -1847,6 +1863,20 @@ indx_insert_into_buffer(struct ntfs_index *indx, struct ntfs_inode *ni,
memcpy(up_e, sp, sp_size);
used1 = le32_to_cpu(hdr1->used);
+
+ /*
+ * hdr_find_split does not validate per-entry sizes, so a crafted
+ * NTFS_DE whose le16 size field is out of range can place sp such
+ * that (PtrOffset(hdr1, sp) + sp_size) exceeds used1. Without this
+ * guard the u32 'used = used1 - to_copy - sp_size' underflows and
+ * the subsequent memmove count becomes a near-4-GiB value,
+ * triggering an out-of-bounds kernel write.
+ */
+ if (PtrOffset(hdr1, sp) + sp_size > used1) {
+ err = -EINVAL;
+ goto out;
+ }
+
hdr1_saved = kmemdup(hdr1, used1, GFP_NOFS);
if (!hdr1_saved) {
err = -ENOMEM;
@@ -2014,13 +2044,21 @@ int indx_insert_entry(struct ntfs_index *indx, struct ntfs_inode *ni,
static struct indx_node *indx_find_buffer(struct ntfs_index *indx,
struct ntfs_inode *ni,
const struct INDEX_ROOT *root,
- __le64 vbn, struct indx_node *n)
+ __le64 vbn, struct indx_node *n,
+ int depth)
{
int err;
const struct NTFS_DE *e;
struct indx_node *r;
const struct INDEX_HDR *hdr = n ? &n->index->ihdr : &root->ihdr;
+ /*
+ * Limit recursion depth to prevent stack overflow from crafted
+ * images. Use the same bound as the fnd->nodes array (20).
+ */
+ if (depth > ARRAY_SIZE(((struct ntfs_fnd *)NULL)->nodes))
+ return ERR_PTR(-EINVAL);
+
/* Step 1: Scan one level. */
for (e = hdr_first_de(hdr);; e = hdr_next_de(hdr, e)) {
if (!e)
@@ -2041,7 +2079,8 @@ static struct indx_node *indx_find_buffer(struct ntfs_index *indx,
if (err)
return ERR_PTR(err);
- r = indx_find_buffer(indx, ni, root, vbn, n);
+ r = indx_find_buffer(indx, ni, root, vbn, n,
+ depth + 1);
if (r)
return r;
}
@@ -2446,7 +2485,7 @@ int indx_delete_entry(struct ntfs_index *indx, struct ntfs_inode *ni,
fnd_clear(fnd);
- in = indx_find_buffer(indx, ni, root, sub_vbn, NULL);
+ in = indx_find_buffer(indx, ni, root, sub_vbn, NULL, 0);
if (IS_ERR(in)) {
err = PTR_ERR(in);
goto out;
diff --git a/fs/ntfs3/lznt.c b/fs/ntfs3/lznt.c
index fdc9b2ebf341..f818d9785004 100644
--- a/fs/ntfs3/lznt.c
+++ b/fs/ntfs3/lznt.c
@@ -240,7 +240,7 @@ static inline ssize_t decompress_chunk(u8 *unc, u8 *unc_end, const u8 *cmpr,
if (up - unc > LZNT_CHUNK_SIZE)
return -EINVAL;
/* Correct index */
- while (unc + s_max_off[index] < up)
+ while (index < ARRAY_SIZE(s_max_off) - 1 && unc + s_max_off[index] < up)
index += 1;
/* Check the current flag for zero. */
diff --git a/fs/ntfs3/namei.c b/fs/ntfs3/namei.c
index fa4f7d9f3845..8e787c2f1777 100644
--- a/fs/ntfs3/namei.c
+++ b/fs/ntfs3/namei.c
@@ -324,7 +324,7 @@ static int ntfs_rename(struct mnt_idmap *idmap, struct inode *dir,
ntfs_sync_inode(dir);
if (IS_DIRSYNC(new_dir))
- ntfs_sync_inode(inode);
+ ntfs_sync_inode(new_dir);
}
if (dir_ni != new_dir_ni)
diff --git a/fs/ntfs3/ntfs_fs.h b/fs/ntfs3/ntfs_fs.h
index 7375ea351f0e..919fb3f36af0 100644
--- a/fs/ntfs3/ntfs_fs.h
+++ b/fs/ntfs3/ntfs_fs.h
@@ -822,7 +822,8 @@ int run_unpack_ex(struct runs_tree *run, struct ntfs_sb_info *sbi, CLST ino,
#else
#define run_unpack_ex run_unpack
#endif
-int run_get_highest_vcn(CLST vcn, const u8 *run_buf, u64 *highest_vcn);
+int run_get_highest_vcn(CLST vcn, const u8 *run_buf, size_t run_buf_size,
+ u64 *highest_vcn);
int run_clone(const struct runs_tree *run, struct runs_tree *new_run);
/* Globals from super.c */
diff --git a/fs/ntfs3/run.c b/fs/ntfs3/run.c
index 744c1a317f8e..12f93f5a3d9b 100644
--- a/fs/ntfs3/run.c
+++ b/fs/ntfs3/run.c
@@ -1157,18 +1157,23 @@ int run_unpack_ex(struct runs_tree *run, struct ntfs_sb_info *sbi, CLST ino,
* Return the highest vcn from a mapping pairs array
* it used while replaying log file.
*/
-int run_get_highest_vcn(CLST vcn, const u8 *run_buf, u64 *highest_vcn)
+int run_get_highest_vcn(CLST vcn, const u8 *run_buf, size_t run_buf_size,
+ u64 *highest_vcn)
{
+ const u8 *run_last = run_buf + run_buf_size;
u64 vcn64 = vcn;
u8 size_size;
- while ((size_size = *run_buf & 0xF)) {
+ while (run_buf < run_last && (size_size = *run_buf & 0xF)) {
u8 offset_size = *run_buf++ >> 4;
u64 len;
if (size_size > 8 || offset_size > 8)
return -EINVAL;
+ if (run_buf + size_size + offset_size > run_last)
+ return -EINVAL;
+
len = run_unpack_s64(run_buf, size_size, 0);
if (!len)
return -EINVAL;
diff --git a/fs/ocfs2/alloc.c b/fs/ocfs2/alloc.c
index f8025433ce3b..eb6adb785e45 100644
--- a/fs/ocfs2/alloc.c
+++ b/fs/ocfs2/alloc.c
@@ -7553,7 +7553,7 @@ int ocfs2_trim_mainbm(struct super_block *sb, struct fstrim_range *range)
len = range->len >> osb->s_clustersize_bits;
minlen = range->minlen >> osb->s_clustersize_bits;
- if (minlen >= osb->bitmap_cpg || range->len < sb->s_blocksize)
+ if (minlen >= osb->bitmap_cpg || range->len < osb->s_clustersize)
return -EINVAL;
trace_ocfs2_trim_mainbm(start, len, minlen);
diff --git a/fs/ocfs2/aops.c b/fs/ocfs2/aops.c
index ba58572dfe63..5acae85dfe58 100644
--- a/fs/ocfs2/aops.c
+++ b/fs/ocfs2/aops.c
@@ -2396,6 +2396,15 @@ static int ocfs2_dio_end_io_write(struct inode *inode,
unlock:
up_write(&oi->ip_alloc_sem);
+ if (data_ac) {
+ ocfs2_free_alloc_context(data_ac);
+ data_ac = NULL;
+ }
+ if (meta_ac) {
+ ocfs2_free_alloc_context(meta_ac);
+ meta_ac = NULL;
+ }
+
/* everything looks good, let's start the cleanup */
if (!ret && dwc->dw_orphaned) {
BUG_ON(dwc->dw_writer_pid != task_pid_nr(current));
@@ -2407,10 +2416,6 @@ static int ocfs2_dio_end_io_write(struct inode *inode,
ocfs2_inode_unlock(inode, 1);
brelse(di_bh);
out:
- if (data_ac)
- ocfs2_free_alloc_context(data_ac);
- if (meta_ac)
- ocfs2_free_alloc_context(meta_ac);
ocfs2_run_deallocs(osb, &dealloc);
ocfs2_dio_free_write_ctx(inode, dwc);
diff --git a/fs/ocfs2/buffer_head_io.c b/fs/ocfs2/buffer_head_io.c
index 8f714406528d..e52e202857d1 100644
--- a/fs/ocfs2/buffer_head_io.c
+++ b/fs/ocfs2/buffer_head_io.c
@@ -350,8 +350,6 @@ int ocfs2_read_blocks(struct ocfs2_caching_info *ci, u64 block, int nr,
wait_on_buffer(bh);
put_bh(bh);
bhs[i] = NULL;
- } else if (bh && buffer_uptodate(bh)) {
- clear_buffer_uptodate(bh);
}
continue;
}
@@ -380,8 +378,11 @@ int ocfs2_read_blocks(struct ocfs2_caching_info *ci, u64 block, int nr,
BUG_ON(buffer_jbd(bh));
clear_buffer_needs_validate(bh);
status = validate(sb, bh);
- if (status)
+ if (status) {
+ if (buffer_uptodate(bh))
+ clear_buffer_uptodate(bh);
goto read_failure;
+ }
}
}
diff --git a/fs/ocfs2/dlm/dlmdebug.c b/fs/ocfs2/dlm/dlmdebug.c
index be5e9ed7da8d..e62ed98df7de 100644
--- a/fs/ocfs2/dlm/dlmdebug.c
+++ b/fs/ocfs2/dlm/dlmdebug.c
@@ -612,6 +612,7 @@ static int debug_lockres_open(struct inode *inode, struct file *file)
struct dlm_ctxt *dlm = inode->i_private;
struct debug_lockres *dl;
void *buf;
+ int status = -ENOMEM;
buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!buf)
@@ -624,16 +625,23 @@ static int debug_lockres_open(struct inode *inode, struct file *file)
dl->dl_len = PAGE_SIZE;
dl->dl_buf = buf;
- dlm_grab(dlm);
- dl->dl_ctxt = dlm;
+ /* ->release uses dl_ctxt after open, so it needs a real pin. */
+ dl->dl_ctxt = dlm_grab(dlm);
+ if (!dl->dl_ctxt) {
+ status = -ENOENT;
+ goto bailseq;
+ }
return 0;
+bailseq:
+ seq_release_private(inode, file);
bailfree:
kfree(buf);
bail:
- mlog_errno(-ENOMEM);
- return -ENOMEM;
+ if (status != -ENOENT)
+ mlog_errno(status);
+ return status;
}
static int debug_lockres_release(struct inode *inode, struct file *file)
diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index 764ecbd5ad41..7c8990531b02 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -3135,6 +3135,22 @@ static void *ocfs2_dlm_seq_next(struct seq_file *m, void *v, loff_t *pos)
* - Add last pr/ex unlock times and first lock wait time in usecs
*/
#define OCFS2_DLM_DEBUG_STR_VERSION 4
+
+/*
+ * The debug iterator snapshots lockres by value, so a userspace-stack LVB
+ * pointer copied from the original lockres must be rebased to the copied
+ * lksb before the dump walks the raw bytes.
+ */
+static void ocfs2_dlm_seq_rebase_lvb(struct ocfs2_lock_res *lockres)
+{
+ if (!ocfs2_stack_supports_plocks())
+ return;
+
+ if (lockres->l_lksb.lksb_fsdlm.sb_lvbptr)
+ lockres->l_lksb.lksb_fsdlm.sb_lvbptr =
+ (char *)&lockres->l_lksb + sizeof(struct dlm_lksb);
+}
+
static int ocfs2_dlm_seq_show(struct seq_file *m, void *v)
{
int i;
@@ -3192,6 +3208,7 @@ static int ocfs2_dlm_seq_show(struct seq_file *m, void *v)
lockres->l_blocking);
/* Dump the raw LVB */
+ ocfs2_dlm_seq_rebase_lvb(lockres);
lvb = ocfs2_dlm_lvb(&lockres->l_lksb);
for(i = 0; i < DLM_LVB_LEN; i++)
seq_printf(m, "0x%x\t", lvb[i]);
diff --git a/fs/ocfs2/inode.c b/fs/ocfs2/inode.c
index c29b065f33a7..9a56c57f9ded 100644
--- a/fs/ocfs2/inode.c
+++ b/fs/ocfs2/inode.c
@@ -66,7 +66,40 @@ static int ocfs2_filecheck_read_inode_block_full(struct inode *inode,
static int ocfs2_filecheck_validate_inode_block(struct super_block *sb,
struct buffer_head *bh);
static int ocfs2_filecheck_repair_inode_block(struct super_block *sb,
- struct buffer_head *bh);
+ struct buffer_head *bh);
+
+static bool ocfs2_valid_inode_mode(umode_t mode)
+{
+ return fs_umode_to_ftype(mode) != FT_UNKNOWN;
+}
+
+static bool ocfs2_dinode_has_unexpected_rdev(struct ocfs2_dinode *di)
+{
+ umode_t mode = le16_to_cpu(di->i_mode);
+
+ if (le32_to_cpu(di->i_flags) & OCFS2_SYSTEM_FL)
+ return false;
+
+ return !S_ISCHR(mode) && !S_ISBLK(mode) && di->id1.dev1.i_rdev != 0;
+}
+
+static bool ocfs2_dinode_has_size_without_clusters(struct super_block *sb,
+ struct ocfs2_dinode *di)
+{
+ umode_t mode = le16_to_cpu(di->i_mode);
+
+ if (le32_to_cpu(di->i_flags) & OCFS2_SYSTEM_FL)
+ return false;
+ if (le16_to_cpu(di->i_dyn_features) & OCFS2_INLINE_DATA_FL)
+ return false;
+ if (!le64_to_cpu(di->i_size) || le32_to_cpu(di->i_clusters))
+ return false;
+
+ if (S_ISDIR(mode))
+ return true;
+
+ return !ocfs2_sparse_alloc(OCFS2_SB(sb)) && S_ISREG(mode);
+}
void ocfs2_set_inode_flags(struct inode *inode)
{
@@ -1419,6 +1452,86 @@ int ocfs2_validate_inode_block(struct super_block *sb,
goto bail;
}
+ /*
+ * Reject dinodes whose i_mode does not name one of the seven
+ * canonical POSIX file types. ocfs2_populate_inode() copies
+ * i_mode verbatim into inode->i_mode and then dispatches via
+ * switch (mode & S_IFMT) to file/dir/symlink/special_file iops;
+ * an unrecognised type falls into ocfs2_special_file_iops with
+ * init_special_inode(), which interprets i_rdev. Constrain the
+ * type here so the dispatch only ever sees a value mkfs.ocfs2 /
+ * VFS can produce.
+ */
+ if (!ocfs2_valid_inode_mode(le16_to_cpu(di->i_mode))) {
+ rc = ocfs2_error(sb,
+ "Invalid dinode #%llu: mode 0%o has unknown file type\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(di->i_mode));
+ goto bail;
+ }
+
+ /*
+ * id1.dev1.i_rdev is the device-number arm of the id1 union and
+ * is only meaningful for character and block device inodes. For
+ * any other regular user-visible file type the on-disk value
+ * must be zero. ocfs2_populate_inode() currently runs
+ *
+ * inode->i_rdev = huge_decode_dev(le64_to_cpu(fe->id1.dev1.i_rdev));
+ *
+ * unconditionally, before the S_IFMT switch decides whether the
+ * inode is a special file. As a result, an i_rdev value present
+ * on a non-device inode is silently published into the in-core
+ * inode; a subsequent forced re-read or in-core mode mutation
+ * (cluster peer with raw write access to the shared LUN,
+ * on-disk corruption, or a separately forged dinode) can then
+ * expose the attacker-controlled device number to
+ * init_special_inode() without ever showing an unusual i_mode
+ * at validation time.
+ *
+ * System inodes (OCFS2_SYSTEM_FL) legitimately use the bitmap1
+ * and journal1 arms of the same union (allocator i_used /
+ * i_total counters and the journal ij_flags /
+ * ij_recovery_generation pair); those bytes are not an i_rdev
+ * and must not be checked here. Restrict the cross-check to
+ * non-system inodes, which is the full attacker-controllable
+ * surface.
+ */
+ if (ocfs2_dinode_has_unexpected_rdev(di)) {
+ rc = ocfs2_error(sb,
+ "Invalid dinode #%llu: non-device mode 0%o with i_rdev %llu\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(di->i_mode),
+ (unsigned long long)le64_to_cpu(di->id1.dev1.i_rdev));
+ goto bail;
+ }
+
+ /*
+ * Non-inline directories must not have i_size without allocated
+ * clusters: directory growth adds storage before advancing i_size,
+ * and readdir walks i_size block-by-block. A forged directory
+ * with zero clusters and a huge i_size would repeatedly fault on
+ * holes while advancing through the claimed size.
+ *
+ * Non-inline regular files have the same invariant on non-sparse
+ * volumes. Sparse regular files are different: truncate can
+ * legitimately grow i_size without allocating clusters, so keep
+ * the sparse-alloc carveout for S_IFREG only. System inodes and
+ * inline-data dinodes have their own storage rules.
+ */
+ if (ocfs2_dinode_has_size_without_clusters(sb, di)) {
+ if (S_ISDIR(le16_to_cpu(di->i_mode)))
+ rc = ocfs2_error(sb,
+ "Invalid dinode #%llu: directory i_size %llu with i_clusters 0 and no inline-data flag\n",
+ (unsigned long long)bh->b_blocknr,
+ (unsigned long long)le64_to_cpu(di->i_size));
+ else
+ rc = ocfs2_error(sb,
+ "Invalid dinode #%llu: regular file i_size %llu with i_clusters 0 and no inline-data flag on non-sparse volume\n",
+ (unsigned long long)bh->b_blocknr,
+ (unsigned long long)le64_to_cpu(di->i_size));
+ goto bail;
+ }
+
if (le16_to_cpu(di->i_dyn_features) & OCFS2_INLINE_DATA_FL) {
struct ocfs2_inline_data *data = &di->id2.i_data;
@@ -1515,6 +1628,40 @@ static int ocfs2_filecheck_validate_inode_block(struct super_block *sb,
(unsigned long long)bh->b_blocknr,
le32_to_cpu(di->i_fs_generation));
rc = -OCFS2_FILECHECK_ERR_GENERATION;
+ goto bail;
+ }
+
+ if (!ocfs2_valid_inode_mode(le16_to_cpu(di->i_mode))) {
+ mlog(ML_ERROR,
+ "Filecheck: invalid dinode #%llu: mode 0%o has unknown file type\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(di->i_mode));
+ rc = -OCFS2_FILECHECK_ERR_INVALIDINO;
+ goto bail;
+ }
+
+ if (ocfs2_dinode_has_unexpected_rdev(di)) {
+ mlog(ML_ERROR,
+ "Filecheck: invalid dinode #%llu: non-device mode 0%o with i_rdev %llu\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(di->i_mode),
+ (unsigned long long)le64_to_cpu(di->id1.dev1.i_rdev));
+ rc = -OCFS2_FILECHECK_ERR_INVALIDINO;
+ goto bail;
+ }
+
+ if (ocfs2_dinode_has_size_without_clusters(sb, di)) {
+ if (S_ISDIR(le16_to_cpu(di->i_mode)))
+ mlog(ML_ERROR,
+ "Filecheck: invalid dinode #%llu: directory i_size %llu with i_clusters 0 and no inline-data flag\n",
+ (unsigned long long)bh->b_blocknr,
+ (unsigned long long)le64_to_cpu(di->i_size));
+ else
+ mlog(ML_ERROR,
+ "Filecheck: invalid dinode #%llu: regular file i_size %llu with i_clusters 0 and no inline-data flag on non-sparse volume\n",
+ (unsigned long long)bh->b_blocknr,
+ (unsigned long long)le64_to_cpu(di->i_size));
+ rc = -OCFS2_FILECHECK_ERR_INVALIDINO;
}
bail:
@@ -1692,4 +1839,3 @@ const struct ocfs2_caching_operations ocfs2_inode_caching_ops = {
.co_io_lock = ocfs2_inode_cache_io_lock,
.co_io_unlock = ocfs2_inode_cache_io_unlock,
};
-
diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c
index c2a73bfb16aa..eddd6a4693a6 100644
--- a/fs/ocfs2/journal.c
+++ b/fs/ocfs2/journal.c
@@ -476,8 +476,12 @@ int ocfs2_extend_trans(handle_t *handle, int nblocks)
*/
int ocfs2_assure_trans_credits(handle_t *handle, int nblocks)
{
- int old_nblks = jbd2_handle_buffer_credits(handle);
+ int old_nblks;
+ if (is_handle_aborted(handle))
+ return -EROFS;
+
+ old_nblks = jbd2_handle_buffer_credits(handle);
trace_ocfs2_assure_trans_credits(old_nblks);
if (old_nblks >= nblocks)
return 0;
@@ -1027,11 +1031,8 @@ static int ocfs2_journal_toggle_dirty(struct ocfs2_super *osb,
struct ocfs2_dinode *fe;
fe = (struct ocfs2_dinode *)bh->b_data;
-
- /* The journal bh on the osb always comes from ocfs2_journal_init()
- * and was validated there inside ocfs2_inode_lock_full(). It's a
- * code bug if we mess it up. */
- BUG_ON(!OCFS2_IS_VALID_DINODE(fe));
+ if (WARN_ON(!OCFS2_IS_VALID_DINODE(fe)))
+ return -EIO;
flags = le32_to_cpu(fe->id1.journal1.ij_flags);
if (dirty)
diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h
index 6397170f302f..f8b3b2a3d630 100644
--- a/fs/ocfs2/journal.h
+++ b/fs/ocfs2/journal.h
@@ -196,6 +196,9 @@ static inline void ocfs2_checkpoint_inode(struct inode *inode)
if (ocfs2_mount_local(osb))
return;
+ if (!osb->journal)
+ return;
+
if (!ocfs2_ci_fully_checkpointed(INODE_CACHE(inode))) {
/* WARNING: This only kicks off a single
* checkpoint. If someone races you and adds more
diff --git a/fs/ocfs2/move_extents.c b/fs/ocfs2/move_extents.c
index 6fcaaeece666..decf14ea2147 100644
--- a/fs/ocfs2/move_extents.c
+++ b/fs/ocfs2/move_extents.c
@@ -534,6 +534,8 @@ static void ocfs2_probe_alloc_group(struct inode *inode, struct buffer_head *bh,
u32 base_cpos = ocfs2_blocks_to_clusters(inode->i_sb,
le64_to_cpu(gd->bg_blkno));
+ *phys_cpos = 0;
+
for (i = base_bit; i < le16_to_cpu(gd->bg_bits); i++) {
used = ocfs2_test_bit(i, (unsigned long *)gd->bg_bitmap);
@@ -555,7 +557,7 @@ static void ocfs2_probe_alloc_group(struct inode *inode, struct buffer_head *bh,
last_free_bits++;
if (last_free_bits == move_len) {
- i -= move_len;
+ i = i - move_len + 1;
*goal_bit = i;
*phys_cpos = base_cpos + i;
break;
diff --git a/fs/ocfs2/ocfs2.h b/fs/ocfs2/ocfs2.h
index 6aaa94c554c1..8bdeea60742a 100644
--- a/fs/ocfs2/ocfs2.h
+++ b/fs/ocfs2/ocfs2.h
@@ -494,8 +494,6 @@ struct ocfs2_super
struct rb_root osb_rf_lock_tree;
struct ocfs2_refcount_tree *osb_ref_tree_lru;
- struct mutex system_file_mutex;
-
/*
* OCFS2 needs to schedule several different types of work which
* require cluster locking, disk I/O, recovery waits, etc. Since these
diff --git a/fs/ocfs2/quota_local.c b/fs/ocfs2/quota_local.c
index de7f12858729..bd3eb098097f 100644
--- a/fs/ocfs2/quota_local.c
+++ b/fs/ocfs2/quota_local.c
@@ -302,7 +302,7 @@ static int ocfs2_add_recovery_chunk(struct super_block *sb,
if (!rc)
return -ENOMEM;
rc->rc_chunk = chunk;
- rc->rc_bitmap = kmalloc(sb->s_blocksize, GFP_NOFS);
+ rc->rc_bitmap = kzalloc(sb->s_blocksize, GFP_NOFS);
if (!rc->rc_bitmap) {
kfree(rc);
return -ENOMEM;
diff --git a/fs/ocfs2/stack_user.c b/fs/ocfs2/stack_user.c
index c5236b3ed168..0ec0a934a098 100644
--- a/fs/ocfs2/stack_user.c
+++ b/fs/ocfs2/stack_user.c
@@ -327,18 +327,14 @@ static int ocfs2_control_install_private(struct file *file)
ocfs2_control_this_node = p->op_this_node;
running_proto.pv_major = p->op_proto.pv_major;
running_proto.pv_minor = p->op_proto.pv_minor;
- }
-
-out_unlock:
- mutex_unlock(&ocfs2_control_lock);
-
- if (!rc && set_p) {
- /* We set the global values successfully */
atomic_inc(&ocfs2_control_opened);
ocfs2_control_set_handshake_state(file,
OCFS2_CONTROL_HANDSHAKE_VALID);
}
+out_unlock:
+ mutex_unlock(&ocfs2_control_lock);
+
return rc;
}
diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c
index 868ccdf44738..4ce57996e3d5 100644
--- a/fs/ocfs2/super.c
+++ b/fs/ocfs2/super.c
@@ -2052,8 +2052,6 @@ static int ocfs2_initialize_super(struct super_block *sb,
spin_lock_init(&osb->osb_xattr_lock);
ocfs2_init_steal_slots(osb);
- mutex_init(&osb->system_file_mutex);
-
atomic_set(&osb->alloc_stats.moves, 0);
atomic_set(&osb->alloc_stats.local_data, 0);
atomic_set(&osb->alloc_stats.bitmap_data, 0);
diff --git a/fs/ocfs2/sysfile.c b/fs/ocfs2/sysfile.c
index 53a945da873b..b63af8d64904 100644
--- a/fs/ocfs2/sysfile.c
+++ b/fs/ocfs2/sysfile.c
@@ -98,11 +98,9 @@ struct inode *ocfs2_get_system_file_inode(struct ocfs2_super *osb,
} else
arr = get_local_system_inode(osb, type, slot);
- mutex_lock(&osb->system_file_mutex);
if (arr && ((inode = *arr) != NULL)) {
/* get a ref in addition to the array ref */
inode = igrab(inode);
- mutex_unlock(&osb->system_file_mutex);
BUG_ON(!inode);
return inode;
@@ -112,11 +110,10 @@ struct inode *ocfs2_get_system_file_inode(struct ocfs2_super *osb,
inode = _ocfs2_get_system_file_inode(osb, type, slot);
/* add one more if putting into array for first time */
- if (arr && inode) {
- *arr = igrab(inode);
- BUG_ON(!*arr);
+ if (inode && arr && !*arr && !cmpxchg(&(*arr), NULL, inode)) {
+ inode = igrab(inode);
+ BUG_ON(!inode);
}
- mutex_unlock(&osb->system_file_mutex);
return inode;
}
diff --git a/fs/orangefs/dir.c b/fs/orangefs/dir.c
index 6d1fbeca9d81..53acc281fcd9 100644
--- a/fs/orangefs/dir.c
+++ b/fs/orangefs/dir.c
@@ -190,7 +190,8 @@ static int fill_from_part(struct orangefs_dir_part *part,
{
const int offset = sizeof(struct orangefs_readdir_response_s);
struct orangefs_khandle *khandle;
- __u32 *len, padlen;
+ __u32 *len;
+ u64 padlen;
loff_t i;
char *s;
i = ctx->pos & ~PART_MASK;
@@ -214,8 +215,8 @@ static int fill_from_part(struct orangefs_dir_part *part,
* len is the size of the string itself. padlen is the
* total size of the encoded string.
*/
- padlen = (sizeof *len + *len + 1) +
- (8 - (sizeof *len + *len + 1)%8)%8;
+ padlen = (u64)sizeof *len + *len + 1;
+ padlen += (8 - padlen % 8) % 8;
if (part->len < i + padlen + sizeof *khandle)
goto next;
s = (void *)part + offset + i + sizeof *len;
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index baa54c718bd7..14b06060db4b 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -800,8 +800,8 @@ static const struct address_space_operations ovl_aops = {
*
* This chain is valid:
* - inode->i_rwsem (inode_lock[2])
- * - upper_mnt->mnt_sb->s_writers (ovl_want_write[0])
* - OVL_I(inode)->lock (ovl_inode_lock[2])
+ * - upper_mnt->mnt_sb->s_writers (ovl_want_write[0])
* - OVL_I(lowerinode)->lock (ovl_inode_lock[1])
*
* And this chain is valid:
@@ -814,8 +814,8 @@ static const struct address_space_operations ovl_aops = {
* held, because it is in reverse order of the non-nested case using the same
* upper fs:
* - inode->i_rwsem (inode_lock[1])
- * - upper_mnt->mnt_sb->s_writers (ovl_want_write[0])
* - OVL_I(inode)->lock (ovl_inode_lock[1])
+ * - upper_mnt->mnt_sb->s_writers (ovl_want_write[0])
*/
#define OVL_MAX_NESTING FILESYSTEM_MAX_STACK_DEPTH
diff --git a/fs/proc/array.c b/fs/proc/array.c
index 363d9331216b..e5001b5fc3f0 100644
--- a/fs/proc/array.c
+++ b/fs/proc/array.c
@@ -483,7 +483,11 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
unsigned long flags;
int exit_code = task->exit_code;
struct signal_struct *sig = task->signal;
- unsigned int seq = 1;
+ int ret;
+
+ ret = down_read_killable(&task->signal->exec_update_lock);
+ if (ret)
+ return ret;
state = *get_task_state(task);
vsize = eip = esp = 0;
@@ -540,10 +544,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
if (permitted && (!whole || num_threads < 2))
wchan = !task_is_running(task);
- do {
- seq++; /* 2 on the 1st/lockless path, otherwise odd */
- flags = read_seqbegin_or_lock_irqsave(&sig->stats_lock, &seq);
-
+ scoped_seqlock_read (&sig->stats_lock, ss_lock_irqsave) {
cmin_flt = sig->cmin_flt;
cmaj_flt = sig->cmaj_flt;
cutime = sig->cutime;
@@ -565,8 +566,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
}
rcu_read_unlock();
}
- } while (need_seqretry(&sig->stats_lock, seq));
- done_seqretry_irqrestore(&sig->stats_lock, seq, flags);
+ }
if (whole) {
thread_group_cputime_adjusted(task, &utime, &stime);
@@ -663,6 +663,7 @@ static int do_task_stat(struct seq_file *m, struct pid_namespace *ns,
seq_puts(m, " 0");
seq_putc(m, '\n');
+ up_read(&task->signal->exec_update_lock);
if (mm)
mmput(mm);
return 0;
diff --git a/fs/proc/base.c b/fs/proc/base.c
index 704cf6a0612e..9f0322368497 100644
--- a/fs/proc/base.c
+++ b/fs/proc/base.c
@@ -218,33 +218,24 @@ static int get_task_root(struct task_struct *task, struct path *root)
return result;
}
-static int proc_cwd_link(struct dentry *dentry, struct path *path)
+static int proc_cwd_link(struct dentry *dentry, struct path *path,
+ struct task_struct *task)
{
- struct task_struct *task = get_proc_task(d_inode(dentry));
int result = -ENOENT;
- if (task) {
- task_lock(task);
- if (task->fs) {
- get_fs_pwd(task->fs, path);
- result = 0;
- }
- task_unlock(task);
- put_task_struct(task);
+ task_lock(task);
+ if (task->fs) {
+ get_fs_pwd(task->fs, path);
+ result = 0;
}
+ task_unlock(task);
return result;
}
-static int proc_root_link(struct dentry *dentry, struct path *path)
+static int proc_root_link(struct dentry *dentry, struct path *path,
+ struct task_struct *task)
{
- struct task_struct *task = get_proc_task(d_inode(dentry));
- int result = -ENOENT;
-
- if (task) {
- result = get_task_root(task, path);
- put_task_struct(task);
- }
- return result;
+ return get_task_root(task, path);
}
/*
@@ -423,18 +414,24 @@ static int proc_pid_wchan(struct seq_file *m, struct pid_namespace *ns,
{
unsigned long wchan;
char symname[KSYM_NAME_LEN];
+ int err;
+ err = down_read_killable(&task->signal->exec_update_lock);
+ if (err)
+ return err;
if (!ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS))
goto print0;
wchan = get_wchan(task);
if (wchan && !lookup_symbol_name(wchan, symname)) {
seq_puts(m, symname);
+ up_read(&task->signal->exec_update_lock);
return 0;
}
print0:
seq_putc(m, '0');
+ up_read(&task->signal->exec_update_lock);
return 0;
}
#endif /* CONFIG_KALLSYMS */
@@ -704,24 +701,7 @@ static int proc_pid_syscall(struct seq_file *m, struct pid_namespace *ns,
/* Here the fs part begins */
/************************************************************************/
-/* permission checks */
-static bool proc_fd_access_allowed(struct inode *inode)
-{
- struct task_struct *task;
- bool allowed = false;
- /* Allow access to a task's file descriptors if it is us or we
- * may use ptrace attach to the process and find out that
- * information.
- */
- task = get_proc_task(inode);
- if (task) {
- allowed = ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS);
- put_task_struct(task);
- }
- return allowed;
-}
-
-int proc_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
+int proc_nochmod_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
struct iattr *attr)
{
int error;
@@ -794,7 +774,7 @@ static int proc_pid_permission(struct mnt_idmap *idmap,
static const struct inode_operations proc_def_inode_operations = {
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static int proc_single_show(struct seq_file *m, void *v)
@@ -1778,16 +1758,12 @@ static const struct file_operations proc_pid_set_comm_operations = {
.release = single_release,
};
-static int proc_exe_link(struct dentry *dentry, struct path *exe_path)
+static int proc_exe_link(struct dentry *dentry, struct path *exe_path,
+ struct task_struct *task)
{
- struct task_struct *task;
struct file *exe_file;
- task = get_proc_task(d_inode(dentry));
- if (!task)
- return -ENOENT;
exe_file = get_task_exe_file(task);
- put_task_struct(task);
if (exe_file) {
*exe_path = exe_file->f_path;
path_get(&exe_file->f_path);
@@ -1797,26 +1773,42 @@ static int proc_exe_link(struct dentry *dentry, struct path *exe_path)
return -ENOENT;
}
+static int call_proc_get_link(struct dentry *dentry, struct inode *inode, struct path *path_out)
+{
+ struct task_struct *task;
+ int ret;
+
+ task = get_proc_task(inode);
+ if (!task)
+ return -ENOENT;
+ ret = down_read_killable(&task->signal->exec_update_lock);
+ if (ret)
+ goto out_put_task;
+ if (!ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS)) {
+ ret = -EACCES;
+ goto out;
+ }
+ ret = PROC_I(inode)->op.proc_get_link(dentry, path_out, task);
+
+out:
+ up_read(&task->signal->exec_update_lock);
+out_put_task:
+ put_task_struct(task);
+ return ret;
+}
+
static const char *proc_pid_get_link(struct dentry *dentry,
struct inode *inode,
struct delayed_call *done)
{
struct path path;
- int error = -EACCES;
+ int error;
if (!dentry)
return ERR_PTR(-ECHILD);
-
- /* Are we allowed to snoop on the tasks file descriptors? */
- if (!proc_fd_access_allowed(inode))
- goto out;
-
- error = PROC_I(inode)->op.proc_get_link(dentry, &path);
- if (error)
- goto out;
-
- error = nd_jump_link(&path);
-out:
+ error = call_proc_get_link(dentry, inode, &path);
+ if (!error)
+ error = nd_jump_link(&path);
return ERR_PTR(error);
}
@@ -1850,24 +1842,18 @@ static int proc_pid_readlink(struct dentry * dentry, char __user * buffer, int b
struct inode *inode = d_inode(dentry);
struct path path;
- /* Are we allowed to snoop on the tasks file descriptors? */
- if (!proc_fd_access_allowed(inode))
- goto out;
-
- error = PROC_I(inode)->op.proc_get_link(dentry, &path);
- if (error)
- goto out;
-
- error = do_proc_readlink(&path, buffer, buflen);
- path_put(&path);
-out:
+ error = call_proc_get_link(dentry, inode, &path);
+ if (!error) {
+ error = do_proc_readlink(&path, buffer, buflen);
+ path_put(&path);
+ }
return error;
}
const struct inode_operations proc_pid_link_inode_operations = {
.readlink = proc_pid_readlink,
.get_link = proc_pid_get_link,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
@@ -2246,21 +2232,16 @@ static const struct dentry_operations tid_map_files_dentry_operations = {
.d_delete = pid_delete_dentry,
};
-static int map_files_get_link(struct dentry *dentry, struct path *path)
+static int map_files_get_link(struct dentry *dentry, struct path *path,
+ struct task_struct *task)
{
unsigned long vm_start, vm_end;
struct vm_area_struct *vma;
- struct task_struct *task;
struct mm_struct *mm;
int rc;
rc = -ENOENT;
- task = get_proc_task(d_inode(dentry));
- if (!task)
- goto out;
-
mm = get_task_mm(task);
- put_task_struct(task);
if (!mm)
goto out;
@@ -2315,7 +2296,7 @@ proc_map_files_get_link(struct dentry *dentry,
static const struct inode_operations proc_map_files_link_inode_operations = {
.readlink = proc_pid_readlink,
.get_link = proc_map_files_get_link,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static struct dentry *
@@ -2356,17 +2337,15 @@ static struct dentry *proc_map_files_lookup(struct inode *dir,
if (!task)
goto out;
- result = ERR_PTR(-EACCES);
- if (!ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS))
- goto out_put_task;
-
result = ERR_PTR(-ENOENT);
if (dname_to_vma_addr(dentry, &vm_start, &vm_end))
goto out_put_task;
- mm = get_task_mm(task);
- if (!mm)
+ mm = mm_access(task, PTRACE_MODE_READ_FSCREDS);
+ if (IS_ERR(mm)) {
+ result = ERR_CAST(mm);
goto out_put_task;
+ }
result = ERR_PTR(-EINTR);
if (mmap_read_lock_killable(mm))
@@ -2394,7 +2373,7 @@ static struct dentry *proc_map_files_lookup(struct inode *dir,
static const struct inode_operations proc_map_files_inode_operations = {
.lookup = proc_map_files_lookup,
.permission = proc_fd_permission,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static int
@@ -2416,23 +2395,22 @@ proc_map_files_readdir(struct file *file, struct dir_context *ctx)
if (!task)
goto out;
- ret = -EACCES;
- if (!ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS))
- goto out_put_task;
-
ret = 0;
if (!dir_emit_dots(file, ctx))
goto out_put_task;
- mm = get_task_mm(task);
- if (!mm)
+ mm = mm_access(task, PTRACE_MODE_READ_FSCREDS);
+ if (IS_ERR(mm)) {
+ ret = PTR_ERR(mm);
+ /* if the task has no mm, the directory should just be empty */
+ if (ret == -ESRCH)
+ ret = 0;
goto out_put_task;
+ }
ret = mmap_read_lock_killable(mm);
- if (ret) {
- mmput(mm);
- goto out_put_task;
- }
+ if (ret)
+ goto out_put_mm;
nr_files = 0;
@@ -2458,8 +2436,7 @@ proc_map_files_readdir(struct file *file, struct dir_context *ctx)
if (!p) {
ret = -ENOMEM;
mmap_read_unlock(mm);
- mmput(mm);
- goto out_put_task;
+ goto out_put_mm;
}
p->start = vma->vm_start;
@@ -2467,7 +2444,6 @@ proc_map_files_readdir(struct file *file, struct dir_context *ctx)
p->mode = vma->vm_file->f_mode;
}
mmap_read_unlock(mm);
- mmput(mm);
for (i = 0; i < nr_files; i++) {
char buf[4 * sizeof(long) + 2]; /* max: %lx-%lx\0 */
@@ -2484,6 +2460,8 @@ proc_map_files_readdir(struct file *file, struct dir_context *ctx)
ctx->pos++;
}
+out_put_mm:
+ mmput(mm);
out_put_task:
put_task_struct(task);
out:
@@ -2890,7 +2868,7 @@ static struct dentry *proc_##LSM##_attr_dir_lookup(struct inode *dir, \
static const struct inode_operations proc_##LSM##_attr_dir_inode_ops = { \
.lookup = proc_##LSM##_attr_dir_lookup, \
.getattr = pid_getattr, \
- .setattr = proc_setattr, \
+ .setattr = proc_nochmod_setattr, \
}
#ifdef CONFIG_SECURITY_SMACK
@@ -2949,7 +2927,7 @@ static struct dentry *proc_attr_dir_lookup(struct inode *dir,
static const struct inode_operations proc_attr_dir_inode_operations = {
.lookup = proc_attr_dir_lookup,
.getattr = pid_getattr,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
#endif
@@ -3454,7 +3432,7 @@ static struct dentry *proc_tgid_base_lookup(struct inode *dir, struct dentry *de
static const struct inode_operations proc_tgid_base_inode_operations = {
.lookup = proc_tgid_base_lookup,
.getattr = pid_getattr,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
.permission = proc_pid_permission,
};
@@ -3654,7 +3632,7 @@ static int proc_tid_comm_permission(struct mnt_idmap *idmap,
}
static const struct inode_operations proc_tid_comm_inode_operations = {
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
.permission = proc_tid_comm_permission,
};
@@ -3785,7 +3763,7 @@ static const struct file_operations proc_tid_base_operations = {
static const struct inode_operations proc_tid_base_inode_operations = {
.lookup = proc_tid_base_lookup,
.getattr = pid_getattr,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static struct dentry *proc_task_instantiate(struct dentry *dentry,
@@ -3999,7 +3977,7 @@ static loff_t proc_dir_llseek(struct file *file, loff_t offset, int whence)
static const struct inode_operations proc_task_inode_operations = {
.lookup = proc_task_lookup,
.getattr = proc_task_getattr,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
.permission = proc_pid_permission,
};
diff --git a/fs/proc/fd.c b/fs/proc/fd.c
index 5e391cbca7a3..92f075639411 100644
--- a/fs/proc/fd.c
+++ b/fs/proc/fd.c
@@ -102,7 +102,7 @@ static int proc_fdinfo_permission(struct mnt_idmap *idmap, struct inode *inode,
static const struct inode_operations proc_fdinfo_file_inode_operations = {
.permission = proc_fdinfo_permission,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static const struct file_operations proc_fdinfo_file_operations = {
@@ -172,24 +172,19 @@ static const struct dentry_operations tid_fd_dentry_operations = {
.d_delete = pid_delete_dentry,
};
-static int proc_fd_link(struct dentry *dentry, struct path *path)
+static int proc_fd_link(struct dentry *dentry, struct path *path,
+ struct task_struct *task)
{
- struct task_struct *task;
int ret = -ENOENT;
-
- task = get_proc_task(d_inode(dentry));
- if (task) {
- unsigned int fd = proc_fd(d_inode(dentry));
- struct file *fd_file;
-
- fd_file = fget_task(task, fd);
- if (fd_file) {
- *path = fd_file->f_path;
- path_get(&fd_file->f_path);
- ret = 0;
- fput(fd_file);
- }
- put_task_struct(task);
+ unsigned int fd = proc_fd(d_inode(dentry));
+ struct file *fd_file;
+
+ fd_file = fget_task(task, fd);
+ if (fd_file) {
+ *path = fd_file->f_path;
+ path_get(&fd_file->f_path);
+ ret = 0;
+ fput(fd_file);
}
return ret;
@@ -375,7 +370,7 @@ const struct inode_operations proc_fd_inode_operations = {
.lookup = proc_lookupfd,
.permission = proc_fd_permission,
.getattr = proc_fd_getattr,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static struct dentry *proc_fdinfo_instantiate(struct dentry *dentry,
@@ -416,7 +411,7 @@ static int proc_fdinfo_iterate(struct file *file, struct dir_context *ctx)
const struct inode_operations proc_fdinfo_inode_operations = {
.lookup = proc_lookupfdinfo,
.permission = proc_fdinfo_permission,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
const struct file_operations proc_fdinfo_operations = {
diff --git a/fs/proc/generic.c b/fs/proc/generic.c
index 69150974ad87..c3cc99973ef6 100644
--- a/fs/proc/generic.c
+++ b/fs/proc/generic.c
@@ -112,6 +112,8 @@ static bool pde_subdir_insert(struct proc_dir_entry *dir,
/* Add new node and rebalance tree. */
rb_link_node(&de->subdir_node, parent, new);
rb_insert_color(&de->subdir_node, root);
+ if (S_ISDIR(de->mode))
+ dir->nlink++;
return true;
}
@@ -399,7 +401,6 @@ struct proc_dir_entry *proc_register(struct proc_dir_entry *dir,
write_unlock(&proc_subdir_lock);
goto out_free_inum;
}
- dir->nlink++;
write_unlock(&proc_subdir_lock);
return dp;
@@ -698,6 +699,8 @@ static void pde_erase(struct proc_dir_entry *pde, struct proc_dir_entry *parent)
{
rb_erase(&pde->subdir_node, &parent->subdir);
RB_CLEAR_NODE(&pde->subdir_node);
+ if (S_ISDIR(pde->mode))
+ parent->nlink--;
}
/*
@@ -723,8 +726,6 @@ void remove_proc_entry(const char *name, struct proc_dir_entry *parent)
de = NULL;
} else {
pde_erase(de, parent);
- if (S_ISDIR(de->mode))
- parent->nlink--;
}
}
write_unlock(&proc_subdir_lock);
@@ -783,8 +784,6 @@ int remove_proc_subtree(const char *name, struct proc_dir_entry *parent)
continue;
}
next = de->parent;
- if (S_ISDIR(de->mode))
- next->nlink--;
write_unlock(&proc_subdir_lock);
proc_entry_rundown(de);
diff --git a/fs/proc/internal.h b/fs/proc/internal.h
index edd4eb6fa12a..fa69de181523 100644
--- a/fs/proc/internal.h
+++ b/fs/proc/internal.h
@@ -108,7 +108,7 @@ extern struct kmem_cache *proc_dir_entry_cache;
void pde_free(struct proc_dir_entry *pde);
union proc_op {
- int (*proc_get_link)(struct dentry *, struct path *);
+ int (*proc_get_link)(struct dentry *, struct path *, struct task_struct *);
int (*proc_show)(struct seq_file *m,
struct pid_namespace *ns, struct pid *pid,
struct task_struct *task);
@@ -215,8 +215,8 @@ extern int proc_pid_statm(struct seq_file *, struct pid_namespace *,
extern const struct dentry_operations pid_dentry_operations;
extern int pid_getattr(struct mnt_idmap *, const struct path *,
struct kstat *, u32, unsigned int);
-extern int proc_setattr(struct mnt_idmap *, struct dentry *,
- struct iattr *);
+int proc_nochmod_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
+ struct iattr *attr);
extern void proc_pid_evict_inode(struct proc_inode *);
extern struct inode *proc_pid_make_inode(struct super_block *, struct task_struct *, umode_t);
extern void pid_update_inode(struct task_struct *, struct inode *);
diff --git a/fs/proc/namespaces.c b/fs/proc/namespaces.c
index 8e159fc78c0a..2a3fc96ca622 100644
--- a/fs/proc/namespaces.c
+++ b/fs/proc/namespaces.c
@@ -55,6 +55,10 @@ static const char *proc_ns_get_link(struct dentry *dentry,
if (!task)
return ERR_PTR(-EACCES);
+ error = down_read_killable(&task->signal->exec_update_lock);
+ if (error)
+ goto out_put_task;
+
if (!ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS))
goto out;
@@ -64,6 +68,8 @@ static const char *proc_ns_get_link(struct dentry *dentry,
error = nd_jump_link(&ns_path);
out:
+ up_read(&task->signal->exec_update_lock);
+out_put_task:
put_task_struct(task);
return ERR_PTR(error);
}
@@ -80,11 +86,17 @@ static int proc_ns_readlink(struct dentry *dentry, char __user *buffer, int bufl
if (!task)
return res;
+ res = down_read_killable(&task->signal->exec_update_lock);
+ if (res)
+ goto out_put_task;
+
if (ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS)) {
res = ns_get_name(name, sizeof(name), task, ns_ops);
if (res >= 0)
res = readlink_copy(buffer, buflen, name);
}
+ up_read(&task->signal->exec_update_lock);
+out_put_task:
put_task_struct(task);
return res;
}
@@ -92,7 +104,7 @@ static int proc_ns_readlink(struct dentry *dentry, char __user *buffer, int bufl
static const struct inode_operations proc_ns_link_inode_operations = {
.readlink = proc_ns_readlink,
.get_link = proc_ns_get_link,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static struct dentry *proc_ns_instantiate(struct dentry *dentry,
@@ -179,5 +191,5 @@ static struct dentry *proc_ns_dir_lookup(struct inode *dir,
const struct inode_operations proc_ns_dir_inode_operations = {
.lookup = proc_ns_dir_lookup,
.getattr = pid_getattr,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
diff --git a/fs/proc/proc_net.c b/fs/proc/proc_net.c
index 52f0b75cbce2..184cddeb8215 100644
--- a/fs/proc/proc_net.c
+++ b/fs/proc/proc_net.c
@@ -322,7 +322,7 @@ static int proc_tgid_net_getattr(struct mnt_idmap *idmap,
const struct inode_operations proc_net_inode_operations = {
.lookup = proc_tgid_net_lookup,
.getattr = proc_tgid_net_getattr,
- .setattr = proc_setattr,
+ .setattr = proc_nochmod_setattr,
};
static int proc_tgid_net_readdir(struct file *file, struct dir_context *ctx)
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index fb01a071b83d..673b7d0fcfd8 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -2606,7 +2606,7 @@ static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask,
if (~categories & PAGE_IS_WRITTEN)
goto out_unlock;
- if (end != start + HPAGE_SIZE) {
+ if (end != start + huge_page_size(hstate_vma(vma))) {
/* Partial HugeTLB page WP isn't possible. */
pagemap_scan_backout_range(p, start, end);
p->arg.walk_end = start;
@@ -2623,8 +2623,62 @@ static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask,
return ret;
}
+
+/*
+ * Write-protect the unpopulated hugetlb entries covering [addr, end) by
+ * installing uffd-wp markers inline, exactly as pagemap_scan_hugetlb_entry()
+ * does for populated entries.
+ *
+ * walk_hugetlb_range() currently calls ->pte_hole() once per huge page, so the
+ * loop normally runs a single iteration; it is written to cover the full range
+ * in case the walker ever coalesces adjacent holes.
+ *
+ * The obvious route -- uffd_wp_range() -> hugetlb_change_protection() --
+ * cannot be used here: it takes hugetlb_vma_lock_write(), but the page-table
+ * walker (walk_hugetlb_range()) already holds hugetlb_vma_lock_read() on the
+ * same VMA, so the scanning thread would deadlock against itself. PMD sharing
+ * is disabled on uffd-wp VMAs (hugetlb_unshare_all_pmds() at registration), so
+ * the vma lock guards nothing that matters for these entries anyway.
+ */
+static int pagemap_scan_hugetlb_hole_wp(struct vm_area_struct *vma,
+ unsigned long addr, unsigned long end)
+{
+ struct hstate *h = hstate_vma(vma);
+ unsigned long psize = huge_page_size(h);
+ struct mm_struct *mm = vma->vm_mm;
+ spinlock_t *ptl;
+ pte_t *ptep;
+ pte_t pte;
+
+ for (addr = ALIGN_DOWN(addr, psize); addr < end; addr += psize) {
+ ptep = huge_pte_alloc(mm, vma, addr, psize);
+ if (!ptep)
+ return -ENOMEM;
+
+ i_mmap_lock_write(vma->vm_file->f_mapping);
+ ptl = huge_pte_lock(h, mm, ptep);
+ pte = huge_ptep_get(mm, addr, ptep);
+ make_uffd_wp_huge_pte(vma, addr, ptep, pte);
+ /*
+ * A none entry has no cached translation, so installing the
+ * marker needs no TLB flush. Flush only if a fault populated
+ * the entry between huge_pte_alloc() and the page table lock.
+ */
+ if (!huge_pte_none(pte))
+ flush_hugetlb_tlb_range(vma, addr, addr + psize);
+ spin_unlock(ptl);
+ i_mmap_unlock_write(vma->vm_file->f_mapping);
+ }
+
+ return 0;
+}
#else
#define pagemap_scan_hugetlb_entry NULL
+static int pagemap_scan_hugetlb_hole_wp(struct vm_area_struct *vma,
+ unsigned long addr, unsigned long end)
+{
+ return 0;
+}
#endif
static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
@@ -2644,7 +2698,10 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
if (~p->arg.flags & PM_SCAN_WP_MATCHING)
return ret;
- err = uffd_wp_range(vma, addr, end - addr, true);
+ if (is_vm_hugetlb_page(vma))
+ err = pagemap_scan_hugetlb_hole_wp(vma, addr, end);
+ else
+ err = uffd_wp_range(vma, addr, end - addr, true);
if (err < 0)
ret = err;
diff --git a/fs/smb/client/cifs_swn.c b/fs/smb/client/cifs_swn.c
index 7233c6a7e6d7..693f609512e6 100644
--- a/fs/smb/client/cifs_swn.c
+++ b/fs/smb/client/cifs_swn.c
@@ -28,10 +28,54 @@ struct cifs_swn_reg {
bool net_name_notify;
bool share_name_notify;
bool ip_notify;
+};
- struct cifs_tcon *tcon;
+struct cifs_swn_reg_info {
+ int id;
+ unsigned int ref_count;
+ const char *net_name;
+ const char *share_name;
+ bool net_name_notify;
+ bool share_name_notify;
+ bool ip_notify;
};
+static void cifs_swn_snapshot_reg(struct cifs_swn_reg *swnreg,
+ struct cifs_swn_reg_info *info)
+{
+ info->id = swnreg->id;
+ info->ref_count = kref_read(&swnreg->ref_count);
+ info->net_name = swnreg->net_name;
+ info->share_name = swnreg->share_name;
+ info->net_name_notify = swnreg->net_name_notify;
+ info->share_name_notify = swnreg->share_name_notify;
+ info->ip_notify = swnreg->ip_notify;
+}
+
+static int cifs_swn_dup_reg(struct cifs_swn_reg *swnreg,
+ struct cifs_swn_reg_info *info)
+{
+ cifs_swn_snapshot_reg(swnreg, info);
+
+ info->net_name = kstrdup(swnreg->net_name, GFP_KERNEL);
+ if (!info->net_name)
+ return -ENOMEM;
+
+ info->share_name = kstrdup(swnreg->share_name, GFP_KERNEL);
+ if (!info->share_name) {
+ kfree(info->net_name);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static void cifs_swn_free_reg_info(struct cifs_swn_reg_info *info)
+{
+ kfree(info->net_name);
+ kfree(info->share_name);
+}
+
static int cifs_swn_auth_info_krb(struct cifs_tcon *tcon, struct sk_buff *skb)
{
int ret;
@@ -73,7 +117,8 @@ static int cifs_swn_auth_info_ntlm(struct cifs_tcon *tcon, struct sk_buff *skb)
* The authentication information to connect to the witness service is bundled
* into the message.
*/
-static int cifs_swn_send_register_message(struct cifs_swn_reg *swnreg)
+static int cifs_swn_send_register_message(struct cifs_swn_reg_info *swnreg,
+ struct cifs_tcon *tcon)
{
struct sk_buff *skb;
struct genlmsghdr *hdr;
@@ -111,10 +156,10 @@ static int cifs_swn_send_register_message(struct cifs_swn_reg *swnreg)
* told to switch to it (client move message). In these cases we unregister from the
* server address and register to the new address when we receive the notification.
*/
- if (swnreg->tcon->ses->server->use_swn_dstaddr)
- addr = &swnreg->tcon->ses->server->swn_dstaddr;
+ if (tcon->ses->server->use_swn_dstaddr)
+ addr = &tcon->ses->server->swn_dstaddr;
else
- addr = &swnreg->tcon->ses->server->dstaddr;
+ addr = &tcon->ses->server->dstaddr;
ret = nla_put(skb, CIFS_GENL_ATTR_SWN_IP, sizeof(struct sockaddr_storage), addr);
if (ret < 0)
@@ -138,10 +183,10 @@ static int cifs_swn_send_register_message(struct cifs_swn_reg *swnreg)
goto nlmsg_fail;
}
- authtype = cifs_select_sectype(swnreg->tcon->ses->server, swnreg->tcon->ses->sectype);
+ authtype = cifs_select_sectype(tcon->ses->server, tcon->ses->sectype);
switch (authtype) {
case Kerberos:
- ret = cifs_swn_auth_info_krb(swnreg->tcon, skb);
+ ret = cifs_swn_auth_info_krb(tcon, skb);
if (ret < 0) {
cifs_dbg(VFS, "%s: Failed to get kerberos auth info: %d\n", __func__, ret);
goto nlmsg_fail;
@@ -149,7 +194,7 @@ static int cifs_swn_send_register_message(struct cifs_swn_reg *swnreg)
break;
case NTLMv2:
case RawNTLMSSP:
- ret = cifs_swn_auth_info_ntlm(swnreg->tcon, skb);
+ ret = cifs_swn_auth_info_ntlm(tcon, skb);
if (ret < 0) {
cifs_dbg(VFS, "%s: Failed to get NTLM auth info: %d\n", __func__, ret);
goto nlmsg_fail;
@@ -179,7 +224,8 @@ static int cifs_swn_send_register_message(struct cifs_swn_reg *swnreg)
/*
* Sends an uregister message to the userspace daemon based on the registration
*/
-static int cifs_swn_send_unregister_message(struct cifs_swn_reg *swnreg)
+static int cifs_swn_send_unregister_message(struct cifs_swn_reg_info *swnreg,
+ struct cifs_tcon *tcon)
{
struct sk_buff *skb;
struct genlmsghdr *hdr;
@@ -208,7 +254,7 @@ static int cifs_swn_send_unregister_message(struct cifs_swn_reg *swnreg)
goto nlmsg_fail;
ret = nla_put(skb, CIFS_GENL_ATTR_SWN_IP, sizeof(struct sockaddr_storage),
- &swnreg->tcon->ses->server->dstaddr);
+ &tcon->ses->server->dstaddr);
if (ret < 0)
goto nlmsg_fail;
@@ -244,6 +290,88 @@ static int cifs_swn_send_unregister_message(struct cifs_swn_reg *swnreg)
return ret;
}
+/*
+ * Allocation-free mirror of extract_hostname() + extract_sharename() from
+ * fs/smb/client/unc.c. Those helpers kmalloc(GFP_KERNEL); this runs under
+ * cifs_tcp_ses_lock and tcon->tc_lock, both spinlocks, so we mirror their
+ * parsing in place against the caller's stable net_name/share_name strings.
+ * Keep in sync with unc.c.
+ */
+static bool cifs_swn_tcon_matches(struct cifs_tcon *tcon,
+ const char *net_name,
+ const char *share_name)
+{
+ const char *unc = tcon->tree_name;
+ const char *host, *share, *delim;
+ size_t host_len, share_len;
+
+ if (!tcon->use_witness)
+ return false;
+
+ /* extract_hostname: require strlen(unc) >= 3 */
+ if (strnlen(unc, 3) < 3)
+ return false;
+ /* extract_hostname: skip all leading '\' characters */
+ for (host = unc; *host == '\\'; host++)
+ ;
+ if (!*host)
+ return false;
+ delim = strchr(host, '\\');
+ if (!delim)
+ return false;
+ host_len = delim - host;
+ if (strlen(net_name) != host_len ||
+ strncasecmp(host, net_name, host_len))
+ return false;
+
+ /* extract_sharename: start at unc + 2, then first '\' onward */
+ share = unc + 2;
+ delim = strchr(share, '\\');
+ if (!delim)
+ return false;
+ share = delim + 1;
+ share_len = strlen(share);
+
+ return strlen(share_name) == share_len &&
+ !strncasecmp(share, share_name, share_len);
+}
+
+/*
+ * One SWN registration id represents one net/share name pair. Multiple
+ * mounted tcons can therefore share the id. Pick a live representative at
+ * use time instead of caching the first tcon pointer in the registration.
+ */
+static struct cifs_tcon *cifs_swn_get_tcon(struct cifs_swn_reg_info *swnreg)
+{
+ struct TCP_Server_Info *server;
+ struct cifs_ses *ses;
+ struct cifs_tcon *tcon;
+
+ spin_lock(&cifs_tcp_ses_lock);
+ list_for_each_entry(server, &cifs_tcp_ses_list, tcp_ses_list) {
+ list_for_each_entry(ses, &server->smb_ses_list, smb_ses_list) {
+ list_for_each_entry(tcon, &ses->tcon_list, tcon_list) {
+ spin_lock(&tcon->tc_lock);
+ if (tcon->status == TID_EXITING ||
+ !cifs_swn_tcon_matches(tcon, swnreg->net_name,
+ swnreg->share_name)) {
+ spin_unlock(&tcon->tc_lock);
+ continue;
+ }
+ ++tcon->tc_count;
+ trace_smb3_tcon_ref(tcon->debug_id,
+ tcon->tc_count,
+ netfs_trace_tcon_ref_get_swn_notify);
+ spin_unlock(&tcon->tc_lock);
+ spin_unlock(&cifs_tcp_ses_lock);
+ return tcon;
+ }
+ }
+ }
+ spin_unlock(&cifs_tcp_ses_lock);
+ return NULL;
+}
+
/*
* Try to find a matching registration for the tcon's server name and share name.
* Calls to this function must be protected by cifs_swnreg_idr_mutex.
@@ -313,17 +441,15 @@ static struct cifs_swn_reg *cifs_get_swn_reg(struct cifs_tcon *tcon)
reg = cifs_find_swn_reg(tcon);
if (!IS_ERR(reg)) {
kref_get(®->ref_count);
- mutex_unlock(&cifs_swnreg_idr_mutex);
- return reg;
+ goto unlock;
} else if (PTR_ERR(reg) != -EEXIST) {
- mutex_unlock(&cifs_swnreg_idr_mutex);
- return reg;
+ goto unlock;
}
reg = kmalloc(sizeof(struct cifs_swn_reg), GFP_ATOMIC);
if (reg == NULL) {
- mutex_unlock(&cifs_swnreg_idr_mutex);
- return ERR_PTR(-ENOMEM);
+ ret = -ENOMEM;
+ goto fail_unlock;
}
kref_init(®->ref_count);
@@ -352,9 +478,7 @@ static struct cifs_swn_reg *cifs_get_swn_reg(struct cifs_tcon *tcon)
reg->net_name_notify = true;
reg->share_name_notify = true;
reg->ip_notify = (tcon->capabilities & SMB2_SHARE_CAP_SCALEOUT);
-
- reg->tcon = tcon;
-
+unlock:
mutex_unlock(&cifs_swnreg_idr_mutex);
return reg;
@@ -365,6 +489,7 @@ static struct cifs_swn_reg *cifs_get_swn_reg(struct cifs_tcon *tcon)
idr_remove(&cifs_swnreg_idr, reg->id);
fail:
kfree(reg);
+fail_unlock:
mutex_unlock(&cifs_swnreg_idr_mutex);
return ERR_PTR(ret);
}
@@ -372,11 +497,6 @@ static struct cifs_swn_reg *cifs_get_swn_reg(struct cifs_tcon *tcon)
static void cifs_swn_reg_release(struct kref *ref)
{
struct cifs_swn_reg *swnreg = container_of(ref, struct cifs_swn_reg, ref_count);
- int ret;
-
- ret = cifs_swn_send_unregister_message(swnreg);
- if (ret < 0)
- cifs_dbg(VFS, "%s: Failed to send unregister message: %d\n", __func__, ret);
idr_remove(&cifs_swnreg_idr, swnreg->id);
kfree(swnreg->net_name);
@@ -384,23 +504,33 @@ static void cifs_swn_reg_release(struct kref *ref)
kfree(swnreg);
}
-static void cifs_put_swn_reg(struct cifs_swn_reg *swnreg)
+static void cifs_put_swn_reg_locked(struct cifs_swn_reg *swnreg,
+ struct cifs_tcon *tcon)
{
- mutex_lock(&cifs_swnreg_idr_mutex);
+ if (kref_read(&swnreg->ref_count) == 1) {
+ struct cifs_swn_reg_info swnreg_info;
+ int ret;
+
+ cifs_swn_snapshot_reg(swnreg, &swnreg_info);
+ ret = cifs_swn_send_unregister_message(&swnreg_info, tcon);
+ if (ret < 0)
+ cifs_dbg(VFS, "%s: Failed to send unregister message: %d\n",
+ __func__, ret);
+ }
+
kref_put(&swnreg->ref_count, cifs_swn_reg_release);
- mutex_unlock(&cifs_swnreg_idr_mutex);
}
-static int cifs_swn_resource_state_changed(struct cifs_swn_reg *swnreg, const char *name, int state)
+static int cifs_swn_resource_state_changed(struct cifs_tcon *tcon, const char *name, int state)
{
switch (state) {
case CIFS_SWN_RESOURCE_STATE_UNAVAILABLE:
cifs_dbg(FYI, "%s: resource name '%s' become unavailable\n", __func__, name);
- cifs_signal_cifsd_for_reconnect(swnreg->tcon->ses->server, true);
+ cifs_signal_cifsd_for_reconnect(tcon->ses->server, true);
break;
case CIFS_SWN_RESOURCE_STATE_AVAILABLE:
cifs_dbg(FYI, "%s: resource name '%s' become available\n", __func__, name);
- cifs_signal_cifsd_for_reconnect(swnreg->tcon->ses->server, true);
+ cifs_signal_cifsd_for_reconnect(tcon->ses->server, true);
break;
case CIFS_SWN_RESOURCE_STATE_UNKNOWN:
cifs_dbg(FYI, "%s: resource name '%s' changed to unknown state\n", __func__, name);
@@ -506,7 +636,7 @@ static int cifs_swn_reconnect(struct cifs_tcon *tcon, struct sockaddr_storage *a
return ret;
}
-static int cifs_swn_client_move(struct cifs_swn_reg *swnreg, struct sockaddr_storage *addr)
+static int cifs_swn_client_move(struct cifs_tcon *tcon, struct sockaddr_storage *addr)
{
struct sockaddr_in *ipv4 = (struct sockaddr_in *)addr;
struct sockaddr_in6 *ipv6 = (struct sockaddr_in6 *)addr;
@@ -516,14 +646,17 @@ static int cifs_swn_client_move(struct cifs_swn_reg *swnreg, struct sockaddr_sto
else if (addr->ss_family == AF_INET6)
cifs_dbg(FYI, "%s: move to %pI6\n", __func__, &ipv6->sin6_addr);
- return cifs_swn_reconnect(swnreg->tcon, addr);
+ return cifs_swn_reconnect(tcon, addr);
}
int cifs_swn_notify(struct sk_buff *skb, struct genl_info *info)
{
struct cifs_swn_reg *swnreg;
+ struct cifs_swn_reg_info swnreg_info;
+ struct cifs_tcon *tcon;
char name[256];
int type;
+ int ret = 0;
if (info->attrs[CIFS_GENL_ATTR_SWN_REGISTRATION_ID]) {
int swnreg_id;
@@ -531,21 +664,34 @@ int cifs_swn_notify(struct sk_buff *skb, struct genl_info *info)
swnreg_id = nla_get_u32(info->attrs[CIFS_GENL_ATTR_SWN_REGISTRATION_ID]);
mutex_lock(&cifs_swnreg_idr_mutex);
swnreg = idr_find(&cifs_swnreg_idr, swnreg_id);
- mutex_unlock(&cifs_swnreg_idr_mutex);
if (swnreg == NULL) {
+ mutex_unlock(&cifs_swnreg_idr_mutex);
cifs_dbg(FYI, "%s: registration id %d not found\n", __func__, swnreg_id);
return -EINVAL;
}
+ ret = cifs_swn_dup_reg(swnreg, &swnreg_info);
+ mutex_unlock(&cifs_swnreg_idr_mutex);
+ if (ret)
+ return ret;
} else {
cifs_dbg(FYI, "%s: missing registration id attribute\n", __func__);
return -EINVAL;
}
+ tcon = cifs_swn_get_tcon(&swnreg_info);
+ if (!tcon) {
+ cifs_dbg(FYI, "%s: registration id %d has no live tcon\n",
+ __func__, swnreg_info.id);
+ ret = -ENODEV;
+ goto free_info;
+ }
+
if (info->attrs[CIFS_GENL_ATTR_SWN_NOTIFICATION_TYPE]) {
type = nla_get_u32(info->attrs[CIFS_GENL_ATTR_SWN_NOTIFICATION_TYPE]);
} else {
cifs_dbg(FYI, "%s: missing notification type attribute\n", __func__);
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
switch (type) {
@@ -557,15 +703,18 @@ int cifs_swn_notify(struct sk_buff *skb, struct genl_info *info)
sizeof(name));
} else {
cifs_dbg(FYI, "%s: missing resource name attribute\n", __func__);
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
if (info->attrs[CIFS_GENL_ATTR_SWN_RESOURCE_STATE]) {
state = nla_get_u32(info->attrs[CIFS_GENL_ATTR_SWN_RESOURCE_STATE]);
} else {
cifs_dbg(FYI, "%s: missing resource state attribute\n", __func__);
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
- return cifs_swn_resource_state_changed(swnreg, name, state);
+ ret = cifs_swn_resource_state_changed(tcon, name, state);
+ break;
}
case CIFS_SWN_NOTIFICATION_CLIENT_MOVE: {
struct sockaddr_storage addr;
@@ -574,28 +723,36 @@ int cifs_swn_notify(struct sk_buff *skb, struct genl_info *info)
nla_memcpy(&addr, info->attrs[CIFS_GENL_ATTR_SWN_IP], sizeof(addr));
} else {
cifs_dbg(FYI, "%s: missing IP address attribute\n", __func__);
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
- return cifs_swn_client_move(swnreg, &addr);
+ ret = cifs_swn_client_move(tcon, &addr);
+ break;
}
default:
cifs_dbg(FYI, "%s: unknown notification type %d\n", __func__, type);
break;
}
- return 0;
+out:
+ cifs_put_tcon(tcon, netfs_trace_tcon_ref_put_swn_notify);
+free_info:
+ cifs_swn_free_reg_info(&swnreg_info);
+ return ret;
}
int cifs_swn_register(struct cifs_tcon *tcon)
{
struct cifs_swn_reg *swnreg;
+ struct cifs_swn_reg_info swnreg_info;
int ret;
swnreg = cifs_get_swn_reg(tcon);
if (IS_ERR(swnreg))
return PTR_ERR(swnreg);
- ret = cifs_swn_send_register_message(swnreg);
+ cifs_swn_snapshot_reg(swnreg, &swnreg_info);
+ ret = cifs_swn_send_register_message(&swnreg_info, tcon);
if (ret < 0) {
cifs_dbg(VFS, "%s: Failed to send swn register message: %d\n", __func__, ret);
/* Do not put the swnreg or return error, the echo task will retry */
@@ -616,35 +773,68 @@ int cifs_swn_unregister(struct cifs_tcon *tcon)
return PTR_ERR(swnreg);
}
+ cifs_put_swn_reg_locked(swnreg, tcon);
mutex_unlock(&cifs_swnreg_idr_mutex);
- cifs_put_swn_reg(swnreg);
-
return 0;
}
-void cifs_swn_dump(struct seq_file *m)
+/*
+ * Snapshot one registration under cifs_swnreg_idr_mutex and return. Callers
+ * intentionally do the per-registration network/genlmsg work without the
+ * mutex held, both to keep the critical section short and to avoid nesting
+ * cifs_swnreg_idr_mutex inside the higher tc_lock when a live tcon is then
+ * pinned for the send.
+ */
+static int cifs_swn_get_next_reg_info(int *id, struct cifs_swn_reg_info *info)
{
struct cifs_swn_reg *swnreg;
+ int ret = 0;
+
+ mutex_lock(&cifs_swnreg_idr_mutex);
+ swnreg = idr_get_next(&cifs_swnreg_idr, id);
+ if (swnreg) {
+ ret = cifs_swn_dup_reg(swnreg, info);
+ if (!ret) {
+ *id = swnreg->id + 1;
+ ret = 1;
+ }
+ }
+ mutex_unlock(&cifs_swnreg_idr_mutex);
+
+ return ret;
+}
+
+void cifs_swn_dump(struct seq_file *m)
+{
+ struct cifs_swn_reg_info swnreg_info;
+ struct cifs_tcon *tcon;
struct sockaddr_in *sa;
struct sockaddr_in6 *sa6;
- int id;
+ int id = 0;
+ int ret;
seq_puts(m, "Witness registrations:");
- mutex_lock(&cifs_swnreg_idr_mutex);
- idr_for_each_entry(&cifs_swnreg_idr, swnreg, id) {
+ while ((ret = cifs_swn_get_next_reg_info(&id, &swnreg_info)) > 0) {
seq_printf(m, "\nId: %u Refs: %u Network name: '%s'%s Share name: '%s'%s Ip address: ",
- id, kref_read(&swnreg->ref_count),
- swnreg->net_name, swnreg->net_name_notify ? "(y)" : "(n)",
- swnreg->share_name, swnreg->share_name_notify ? "(y)" : "(n)");
- switch (swnreg->tcon->ses->server->dstaddr.ss_family) {
+ swnreg_info.id, swnreg_info.ref_count,
+ swnreg_info.net_name, swnreg_info.net_name_notify ? "(y)" : "(n)",
+ swnreg_info.share_name, swnreg_info.share_name_notify ? "(y)" : "(n)");
+
+ tcon = cifs_swn_get_tcon(&swnreg_info);
+ if (!tcon) {
+ seq_puts(m, "(no live tcon)");
+ goto next;
+ }
+
+ switch (tcon->ses->server->dstaddr.ss_family) {
case AF_INET:
- sa = (struct sockaddr_in *) &swnreg->tcon->ses->server->dstaddr;
+ sa = (struct sockaddr_in *)&tcon->ses->server->dstaddr;
seq_printf(m, "%pI4", &sa->sin_addr.s_addr);
break;
case AF_INET6:
- sa6 = (struct sockaddr_in6 *) &swnreg->tcon->ses->server->dstaddr;
+ sa6 = (struct sockaddr_in6 *)&tcon->ses->server->dstaddr;
seq_printf(m, "%pI6", &sa6->sin6_addr.s6_addr);
if (sa6->sin6_scope_id)
seq_printf(m, "%%%u", sa6->sin6_scope_id);
@@ -652,23 +842,38 @@ void cifs_swn_dump(struct seq_file *m)
default:
seq_puts(m, "(unknown)");
}
- seq_printf(m, "%s", swnreg->ip_notify ? "(y)" : "(n)");
+ cifs_put_tcon(tcon, netfs_trace_tcon_ref_put_swn_notify);
+next:
+ seq_printf(m, "%s", swnreg_info.ip_notify ? "(y)" : "(n)");
+ cifs_swn_free_reg_info(&swnreg_info);
}
- mutex_unlock(&cifs_swnreg_idr_mutex);
+ if (ret < 0)
+ seq_printf(m, "\nFailed to snapshot witness registration: %d", ret);
seq_puts(m, "\n");
}
void cifs_swn_check(void)
{
- struct cifs_swn_reg *swnreg;
- int id;
+ struct cifs_swn_reg_info swnreg_info;
+ struct cifs_tcon *tcon;
+ int id = 0;
int ret;
- mutex_lock(&cifs_swnreg_idr_mutex);
- idr_for_each_entry(&cifs_swnreg_idr, swnreg, id) {
- ret = cifs_swn_send_register_message(swnreg);
+ while ((ret = cifs_swn_get_next_reg_info(&id, &swnreg_info)) > 0) {
+ tcon = cifs_swn_get_tcon(&swnreg_info);
+ if (!tcon) {
+ cifs_dbg(FYI, "%s: registration id %d has no live tcon\n",
+ __func__, swnreg_info.id);
+ goto free_info;
+ }
+
+ ret = cifs_swn_send_register_message(&swnreg_info, tcon);
if (ret < 0)
cifs_dbg(FYI, "%s: Failed to send register message: %d\n", __func__, ret);
+ cifs_put_tcon(tcon, netfs_trace_tcon_ref_put_swn_notify);
+free_info:
+ cifs_swn_free_reg_info(&swnreg_info);
}
- mutex_unlock(&cifs_swnreg_idr_mutex);
+ if (ret < 0)
+ cifs_dbg(FYI, "%s: Failed to snapshot registration: %d\n", __func__, ret);
}
diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c
index 8a257cb29c95..81a1c116e903 100644
--- a/fs/smb/client/cifssmb.c
+++ b/fs/smb/client/cifssmb.c
@@ -1262,14 +1262,6 @@ CIFS_open(const unsigned int xid, struct cifs_open_parms *oparms, int *oplock,
return rc;
}
-static void cifs_readv_worker(struct work_struct *work)
-{
- struct cifs_io_subrequest *rdata =
- container_of(work, struct cifs_io_subrequest, subreq.work);
-
- netfs_read_subreq_terminated(&rdata->subreq, rdata->result, false);
-}
-
static void
cifs_readv_callback(struct mid_q_entry *mid)
{
@@ -1336,8 +1328,8 @@ cifs_readv_callback(struct mid_q_entry *mid)
}
rdata->credits.value = 0;
+ rdata->subreq.error = rdata->result;
rdata->subreq.transferred += rdata->got_bytes;
- INIT_WORK(&rdata->subreq.work, cifs_readv_worker);
queue_work(cifsiod_wq, &rdata->subreq.work);
release_mid(mid);
add_credits(server, &credits, 0);
diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c
index d801e0288a6d..a8c72b5f4818 100644
--- a/fs/smb/client/connect.c
+++ b/fs/smb/client/connect.c
@@ -3971,6 +3971,9 @@ cifs_umount(struct cifs_sb_info *cifs_sb)
}
spin_unlock(&cifs_sb->tlink_tree_lock);
+ flush_workqueue(serverclose_wq);
+ flush_workqueue(fileinfo_put_wq);
+
kfree(cifs_sb->prepath);
call_rcu(&cifs_sb->rcu, delayed_free);
}
diff --git a/fs/smb/client/file.c b/fs/smb/client/file.c
index 53349f2118cd..70b64df9fb2e 100644
--- a/fs/smb/client/file.c
+++ b/fs/smb/client/file.c
@@ -229,7 +229,9 @@ static void cifs_issue_read(struct netfs_io_subrequest *subreq)
return;
failed:
- netfs_read_subreq_terminated(subreq, rc, false);
+ add_credits_and_wake_if(rdata->server, &rdata->credits, 0);
+ subreq->error = rc;
+ netfs_read_subreq_terminated(subreq, false);
}
/*
diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c
index 4862a9518a32..e7fc8d0aebee 100644
--- a/fs/smb/client/inode.c
+++ b/fs/smb/client/inode.c
@@ -28,6 +28,23 @@
#include "cached_dir.h"
#include "reparse.h"
+static void cifs_invalidate_cached_dir(struct cifs_tcon *tcon,
+ struct dentry *parent)
+{
+ struct cached_fid *parent_cfid = NULL;
+
+ if (!tcon || !parent)
+ return;
+
+ if (!open_cached_dir_by_dentry(tcon, parent, &parent_cfid)) {
+ mutex_lock(&parent_cfid->dirents.de_mutex);
+ parent_cfid->dirents.is_valid = false;
+ parent_cfid->dirents.is_failed = true;
+ mutex_unlock(&parent_cfid->dirents.de_mutex);
+ close_cached_dir(parent_cfid);
+ }
+}
+
/*
* Set parameters for the netfs library
*/
@@ -2026,6 +2043,9 @@ int cifs_unlink(struct inode *dir, struct dentry *dentry)
cifs_set_file_info(inode, attrs, xid, full_path, origattr);
out_reval:
+ if (!rc && dentry->d_parent)
+ cifs_invalidate_cached_dir(tcon, dentry->d_parent);
+
if (inode) {
cifs_inode = CIFS_I(inode);
cifs_inode->time = 0; /* will force revalidate to get info
@@ -2330,13 +2350,14 @@ int cifs_rmdir(struct inode *inode, struct dentry *direntry)
}
rc = server->ops->rmdir(xid, tcon, full_path, cifs_sb);
- cifs_put_tlink(tlink);
if (!rc) {
spin_lock(&d_inode(direntry)->i_lock);
i_size_write(d_inode(direntry), 0);
clear_nlink(d_inode(direntry));
spin_unlock(&d_inode(direntry)->i_lock);
+ if (direntry->d_parent)
+ cifs_invalidate_cached_dir(tcon, direntry->d_parent);
}
cifsInode = CIFS_I(d_inode(direntry));
@@ -2352,6 +2373,7 @@ int cifs_rmdir(struct inode *inode, struct dentry *direntry)
inode_set_ctime_current(d_inode(direntry));
inode_set_mtime_to_ts(inode, inode_set_ctime_current(inode));
+ cifs_put_tlink(tlink);
rmdir_exit:
free_dentry_path(page);
@@ -2576,6 +2598,12 @@ cifs_rename2(struct mnt_idmap *idmap, struct inode *source_dir,
}
/* force revalidate to go get info when needed */
+ if (!rc) {
+ cifs_invalidate_cached_dir(tcon, source_dentry->d_parent);
+ if (target_dentry->d_parent != source_dentry->d_parent)
+ cifs_invalidate_cached_dir(tcon, target_dentry->d_parent);
+ }
+
CIFS_I(source_dir)->time = CIFS_I(target_dir)->time = 0;
cifs_rename_exit:
diff --git a/fs/smb/client/ioctl.c b/fs/smb/client/ioctl.c
index 200af8bad57d..2aef6d7617b7 100644
--- a/fs/smb/client/ioctl.c
+++ b/fs/smb/client/ioctl.c
@@ -396,13 +396,11 @@ long cifs_ioctl(struct file *filep, unsigned int command, unsigned long arg)
}
#endif /* CONFIG_CIFS_ALLOW_INSECURE_LEGACY */
#endif /* CONFIG_CIFS_POSIX */
- rc = 0;
- if (CIFS_I(inode)->cifsAttrs & ATTR_COMPRESSED) {
- /* add in the compressed bit */
- ExtAttrBits = FS_COMPR_FL;
- rc = put_user(ExtAttrBits & FS_FL_USER_VISIBLE,
- (int __user *)arg);
- }
+ if (CIFS_I(inode)->cifsAttrs & FILE_ATTRIBUTE_COMPRESSED)
+ ExtAttrBits |= FS_COMPR_FL;
+
+ rc = put_user(ExtAttrBits & FS_FL_USER_VISIBLE,
+ (int __user *)arg);
break;
case FS_IOC_SETFLAGS:
if (pSMBFile == NULL)
diff --git a/fs/smb/client/misc.c b/fs/smb/client/misc.c
index 6abfa22f4c17..713b95a23611 100644
--- a/fs/smb/client/misc.c
+++ b/fs/smb/client/misc.c
@@ -1029,6 +1029,10 @@ parse_dfs_referrals(struct get_dfs_referral_rsp *rsp, u32 rsp_size,
node->ref_flag = le16_to_cpu(ref->ReferralEntryFlags);
/* copy DfsPath */
+ if (le16_to_cpu(ref->DfsPathOffset) > data_end - (char *)ref) {
+ rc = -EINVAL;
+ goto parse_DFS_referrals_exit;
+ }
temp = (char *)ref + le16_to_cpu(ref->DfsPathOffset);
max_len = data_end - temp;
node->path_name = cifs_strndup_from_utf16(temp, max_len,
@@ -1039,6 +1043,10 @@ parse_dfs_referrals(struct get_dfs_referral_rsp *rsp, u32 rsp_size,
}
/* copy link target UNC */
+ if (le16_to_cpu(ref->NetworkAddressOffset) > data_end - (char *)ref) {
+ rc = -EINVAL;
+ goto parse_DFS_referrals_exit;
+ }
temp = (char *)ref + le16_to_cpu(ref->NetworkAddressOffset);
max_len = data_end - temp;
node->node_name = cifs_strndup_from_utf16(temp, max_len,
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 5acbe606c15b..f63361bc1641 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -1771,8 +1771,8 @@ smb2_ioctl_query_info(const unsigned int xid,
if (le32_to_cpu(io_rsp->OutputCount) < qi.input_buffer_length)
qi.input_buffer_length = le32_to_cpu(io_rsp->OutputCount);
if (qi.input_buffer_length > 0 &&
- le32_to_cpu(io_rsp->OutputOffset) + qi.input_buffer_length
- > rsp_iov[1].iov_len) {
+ size_add(le32_to_cpu(io_rsp->OutputOffset),
+ qi.input_buffer_length) > rsp_iov[1].iov_len) {
rc = -EFAULT;
goto out;
}
@@ -2017,8 +2017,9 @@ smb2_sync_write(const unsigned int xid, struct cifs_fid *pfid,
}
/* Set or clear the SPARSE_FILE attribute based on value passed in setsparse */
-static bool smb2_set_sparse(const unsigned int xid, struct cifs_tcon *tcon,
- struct cifsFileInfo *cfile, struct inode *inode, __u8 setsparse)
+static int smb2_set_sparse(const unsigned int xid, struct cifs_tcon *tcon,
+ struct cifsFileInfo *cfile, struct inode *inode,
+ __u8 setsparse)
{
struct cifsInodeInfo *cifsi;
int rc;
@@ -2027,31 +2028,31 @@ static bool smb2_set_sparse(const unsigned int xid, struct cifs_tcon *tcon,
/* if file already sparse don't bother setting sparse again */
if ((cifsi->cifsAttrs & FILE_ATTRIBUTE_SPARSE_FILE) && setsparse)
- return true; /* already sparse */
+ return 0; /* already sparse */
if (!(cifsi->cifsAttrs & FILE_ATTRIBUTE_SPARSE_FILE) && !setsparse)
- return true; /* already not sparse */
+ return 0; /* already not sparse */
/*
* Can't check for sparse support on share the usual way via the
* FS attribute info (FILE_SUPPORTS_SPARSE_FILES) on the share
* since Samba server doesn't set the flag on the share, yet
* supports the set sparse FSCTL and returns sparse correctly
- * in the file attributes. If we fail setting sparse though we
- * mark that server does not support sparse files for this share
- * to avoid repeatedly sending the unsupported fsctl to server
- * if the file is repeatedly extended.
+ * in the file attributes. If the server returns EOPNOTSUPP, mark
+ * that sparse files are not supported on this share to avoid
+ * repeatedly sending the unsupported FSCTL.
*/
if (tcon->broken_sparse_sup)
- return false;
+ return -EOPNOTSUPP;
rc = SMB2_ioctl(xid, tcon, cfile->fid.persistent_fid,
cfile->fid.volatile_fid, FSCTL_SET_SPARSE,
&setsparse, 1, CIFSMaxBufSize, NULL, NULL);
if (rc) {
- tcon->broken_sparse_sup = true;
+ if (rc == -EOPNOTSUPP)
+ tcon->broken_sparse_sup = true;
cifs_dbg(FYI, "set sparse rc = %d\n", rc);
- return false;
+ return rc;
}
if (setsparse)
@@ -2059,7 +2060,7 @@ static bool smb2_set_sparse(const unsigned int xid, struct cifs_tcon *tcon,
else
cifsi->cifsAttrs &= (~FILE_ATTRIBUTE_SPARSE_FILE);
- return true;
+ return 0;
}
static int
@@ -3387,10 +3388,9 @@ static long smb3_punch_hole(struct file *file, struct cifs_tcon *tcon,
inode_lock(inode);
/* Need to make file sparse, if not already, before freeing range. */
/* Consider adding equivalent for compressed since it could also work */
- if (!smb2_set_sparse(xid, tcon, cfile, inode, set_sparse)) {
- rc = -EOPNOTSUPP;
+ rc = smb2_set_sparse(xid, tcon, cfile, inode, set_sparse);
+ if (rc)
goto out;
- }
filemap_invalidate_lock(inode->i_mapping);
/*
@@ -3496,7 +3496,7 @@ static int smb3_simple_fallocate_range(unsigned int xid,
if (rc)
goto out;
- buf = kzalloc(1024 * 1024, GFP_KERNEL);
+ buf = kvzalloc(1024 * 1024, GFP_KERNEL);
if (buf == NULL) {
rc = -ENOMEM;
goto out;
@@ -3553,7 +3553,7 @@ static int smb3_simple_fallocate_range(unsigned int xid,
out:
kfree(out_data);
- kfree(buf);
+ kvfree(buf);
return rc;
}
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index 7fc588eabb22..5a8a9e4ba36d 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -4536,14 +4536,6 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
return rc;
}
-static void smb2_readv_worker(struct work_struct *work)
-{
- struct cifs_io_subrequest *rdata =
- container_of(work, struct cifs_io_subrequest, subreq.work);
-
- netfs_read_subreq_terminated(&rdata->subreq, rdata->result, false);
-}
-
static void
smb2_readv_callback(struct mid_q_entry *mid)
{
@@ -4656,9 +4648,9 @@ smb2_readv_callback(struct mid_q_entry *mid)
server->credits, server->in_flight,
0, cifs_trace_rw_credits_read_response_clear);
rdata->credits.value = 0;
+ rdata->subreq.error = rdata->result;
rdata->subreq.transferred += rdata->got_bytes;
trace_netfs_sreq(&rdata->subreq, netfs_sreq_trace_io_progress);
- INIT_WORK(&rdata->subreq.work, smb2_readv_worker);
queue_work(cifsiod_wq, &rdata->subreq.work);
release_mid(mid);
trace_smb3_rw_credits(rreq_debug_id, subreq_debug_index, 0,
diff --git a/fs/smb/client/trace.h b/fs/smb/client/trace.h
index 547d1e5d000a..7b3675cfa771 100644
--- a/fs/smb/client/trace.h
+++ b/fs/smb/client/trace.h
@@ -52,6 +52,7 @@
EM(netfs_trace_tcon_ref_get_find, "GET Find ") \
EM(netfs_trace_tcon_ref_get_find_sess_tcon, "GET FndSes") \
EM(netfs_trace_tcon_ref_get_reconnect_server, "GET Reconn") \
+ EM(netfs_trace_tcon_ref_get_swn_notify, "GET SwnNot") \
EM(netfs_trace_tcon_ref_new, "NEW ") \
EM(netfs_trace_tcon_ref_new_ipc, "NEW Ipc ") \
EM(netfs_trace_tcon_ref_new_reconnect_server, "NEW Reconn") \
@@ -63,6 +64,7 @@
EM(netfs_trace_tcon_ref_put_mnt_ctx, "PUT MntCtx") \
EM(netfs_trace_tcon_ref_put_dfs_refer, "PUT DfsRfr") \
EM(netfs_trace_tcon_ref_put_reconnect_server, "PUT Reconn") \
+ EM(netfs_trace_tcon_ref_put_swn_notify, "PUT SwnNot") \
EM(netfs_trace_tcon_ref_put_tlink, "PUT Tlink ") \
EM(netfs_trace_tcon_ref_see_cancelled_close, "SEE Cn-Cls") \
EM(netfs_trace_tcon_ref_see_fscache_collision, "SEE FV-CO!") \
diff --git a/fs/smb/server/connection.c b/fs/smb/server/connection.c
index 1610b4d2fd41..8af6e9621d32 100644
--- a/fs/smb/server/connection.c
+++ b/fs/smb/server/connection.c
@@ -22,6 +22,81 @@ static struct ksmbd_conn_ops default_conn_ops;
DEFINE_HASHTABLE(conn_list, CONN_HASH_BITS);
DECLARE_RWSEM(conn_list_lock);
+static struct workqueue_struct *ksmbd_conn_wq;
+
+int ksmbd_conn_wq_init(void)
+{
+ ksmbd_conn_wq = alloc_workqueue("ksmbd-conn-release",
+ WQ_UNBOUND | WQ_MEM_RECLAIM, 0);
+ if (!ksmbd_conn_wq)
+ return -ENOMEM;
+ return 0;
+}
+
+void ksmbd_conn_wq_destroy(void)
+{
+ if (ksmbd_conn_wq) {
+ destroy_workqueue(ksmbd_conn_wq);
+ ksmbd_conn_wq = NULL;
+ }
+}
+
+/*
+ * __ksmbd_conn_release_work() - perform the final, once-per-struct cleanup
+ * of a ksmbd_conn whose refcount has just dropped to zero.
+ *
+ * This is the common release path used by ksmbd_conn_put() for the embedded
+ * state that outlives the connection thread: async_ida and the attached
+ * transport (which owns the socket and iov for TCP). Called from a workqueue
+ * so that sleep-allowed teardown (sock_release -> tcp_close ->
+ * lock_sock_nested) never runs from an RCU softirq callback (free_opinfo_rcu)
+ * or any other non-sleeping putter context.
+ */
+static void __ksmbd_conn_release_work(struct work_struct *work)
+{
+ struct ksmbd_conn *conn =
+ container_of(work, struct ksmbd_conn, release_work);
+
+ ida_destroy(&conn->async_ida);
+ conn->transport->ops->free_transport(conn->transport);
+ kfree(conn);
+}
+
+/**
+ * ksmbd_conn_get() - take a reference on @conn and return it.
+ *
+ * Returns @conn unchanged so callers can write
+ * "fp->conn = ksmbd_conn_get(work->conn);" in one expression. Returns NULL
+ * if @conn is NULL.
+ */
+struct ksmbd_conn *ksmbd_conn_get(struct ksmbd_conn *conn)
+{
+ if (!conn)
+ return NULL;
+
+ atomic_inc(&conn->refcnt);
+ return conn;
+}
+
+/**
+ * ksmbd_conn_put() - drop a reference and, if it was the last, queue the
+ * release onto ksmbd_conn_wq so it runs from process context.
+ *
+ * Callable from any context including RCU softirq callbacks and non-sleeping
+ * locks; the actual release is deferred to the workqueue. ksmbd_conn_wq is
+ * created in ksmbd_server_init() before any conn can be allocated and is
+ * destroyed in ksmbd_server_exit() after rcu_barrier(), so it is always
+ * non-NULL while a conn reference is held.
+ */
+void ksmbd_conn_put(struct ksmbd_conn *conn)
+{
+ if (!conn)
+ return;
+
+ if (atomic_dec_and_test(&conn->refcnt))
+ queue_work(ksmbd_conn_wq, &conn->release_work);
+}
+
/**
* ksmbd_conn_free() - free resources of the connection instance
*
@@ -36,23 +111,19 @@ void ksmbd_conn_free(struct ksmbd_conn *conn)
hash_del(&conn->hlist);
up_write(&conn_list_lock);
+ /*
+ * request_buf / preauth_info / mechToken are only ever accessed by the
+ * connection handler thread that owns @conn. ksmbd_conn_free() is
+ * called from the transport free_transport() path when that thread is
+ * exiting, so it is safe to release them unconditionally even when
+ * ksmbd_conn_put() below is not the final putter (oplock / ksmbd_file
+ * holders only retain the conn pointer, not these per-thread buffers).
+ */
xa_destroy(&conn->sessions);
kvfree(conn->request_buf);
kfree(conn->preauth_info);
kfree(conn->mechToken);
- if (atomic_dec_and_test(&conn->refcnt)) {
- /*
- * async_ida is embedded in struct ksmbd_conn, so pair
- * ida_destroy() with the final kfree() rather than with
- * the unconditional field teardown above. This keeps
- * the IDA valid for the entire lifetime of the struct,
- * even while other refcount holders (oplock / vfs
- * durable handles) still reference the connection.
- */
- ida_destroy(&conn->async_ida);
- conn->transport->ops->free_transport(conn->transport);
- kfree(conn);
- }
+ ksmbd_conn_put(conn);
}
/**
@@ -79,6 +150,7 @@ struct ksmbd_conn *ksmbd_conn_alloc(void)
conn->um = ERR_PTR(-EOPNOTSUPP);
if (IS_ERR(conn->um))
conn->um = NULL;
+ INIT_WORK(&conn->release_work, __ksmbd_conn_release_work);
atomic_set(&conn->req_running, 0);
atomic_set(&conn->r_count, 0);
atomic_set(&conn->refcnt, 1);
@@ -457,8 +529,7 @@ void ksmbd_conn_r_count_dec(struct ksmbd_conn *conn)
if (!atomic_dec_return(&conn->r_count) && waitqueue_active(&conn->r_count_q))
wake_up(&conn->r_count_q);
- if (atomic_dec_and_test(&conn->refcnt))
- kfree(conn);
+ ksmbd_conn_put(conn);
}
int ksmbd_conn_transport_init(void)
diff --git a/fs/smb/server/connection.h b/fs/smb/server/connection.h
index 47eb3de7f307..26e6204ef93e 100644
--- a/fs/smb/server/connection.h
+++ b/fs/smb/server/connection.h
@@ -15,6 +15,7 @@
#include <linux/kthread.h>
#include <linux/nls.h>
#include <linux/unicode.h>
+#include <linux/workqueue.h>
#include "smb_common.h"
#include "ksmbd_work.h"
@@ -117,6 +118,7 @@ struct ksmbd_conn {
bool binding;
atomic_t refcnt;
bool is_aapl;
+ struct work_struct release_work;
};
struct ksmbd_conn_ops {
@@ -162,6 +164,10 @@ void ksmbd_conn_wait_idle(struct ksmbd_conn *conn);
int ksmbd_conn_wait_idle_sess_id(struct ksmbd_conn *curr_conn, u64 sess_id);
struct ksmbd_conn *ksmbd_conn_alloc(void);
void ksmbd_conn_free(struct ksmbd_conn *conn);
+struct ksmbd_conn *ksmbd_conn_get(struct ksmbd_conn *conn);
+void ksmbd_conn_put(struct ksmbd_conn *conn);
+int ksmbd_conn_wq_init(void);
+void ksmbd_conn_wq_destroy(void);
bool ksmbd_conn_lookup_dialect(struct ksmbd_conn *c);
int ksmbd_conn_write(struct ksmbd_work *work);
int ksmbd_conn_rdma_read(struct ksmbd_conn *conn,
diff --git a/fs/smb/server/misc.c b/fs/smb/server/misc.c
index cb2a11ffb23f..53ddcae31f71 100644
--- a/fs/smb/server/misc.c
+++ b/fs/smb/server/misc.c
@@ -278,39 +278,6 @@ char *ksmbd_extract_sharename(struct unicode_map *um, const char *treename)
return ksmbd_casefold_sharename(um, name);
}
-/**
- * convert_to_unix_name() - convert windows name to unix format
- * @share: ksmbd_share_config pointer
- * @name: file name that is relative to share
- *
- * Return: converted name on success, otherwise NULL
- */
-char *convert_to_unix_name(struct ksmbd_share_config *share, const char *name)
-{
- int no_slash = 0, name_len, path_len;
- char *new_name;
-
- if (name[0] == '/')
- name++;
-
- path_len = share->path_sz;
- name_len = strlen(name);
- new_name = kmalloc(path_len + name_len + 2, KSMBD_DEFAULT_GFP);
- if (!new_name)
- return new_name;
-
- memcpy(new_name, share->path, path_len);
- if (new_name[path_len - 1] != '/') {
- new_name[path_len] = '/';
- no_slash = 1;
- }
-
- memcpy(new_name + path_len + no_slash, name, name_len);
- path_len += name_len + no_slash;
- new_name[path_len] = 0x00;
- return new_name;
-}
-
char *ksmbd_convert_dir_info_name(struct ksmbd_dir_info *d_info,
const struct nls_table *local_nls,
int *conv_len)
diff --git a/fs/smb/server/misc.h b/fs/smb/server/misc.h
index 1facfcd21200..53282a6d2436 100644
--- a/fs/smb/server/misc.h
+++ b/fs/smb/server/misc.h
@@ -22,7 +22,6 @@ void ksmbd_strip_last_slash(char *path);
void ksmbd_conv_path_to_windows(char *path);
char *ksmbd_casefold_sharename(struct unicode_map *um, const char *name);
char *ksmbd_extract_sharename(struct unicode_map *um, const char *treename);
-char *convert_to_unix_name(struct ksmbd_share_config *share, const char *name);
#define KSMBD_DIR_INFO_ALIGNMENT 8
struct ksmbd_dir_info;
diff --git a/fs/smb/server/oplock.c b/fs/smb/server/oplock.c
index 6454c7a4baa4..a019c4496ae4 100644
--- a/fs/smb/server/oplock.c
+++ b/fs/smb/server/oplock.c
@@ -30,7 +30,6 @@ static DEFINE_RWLOCK(lease_list_lock);
static struct oplock_info *alloc_opinfo(struct ksmbd_work *work,
u64 id, __u16 Tid)
{
- struct ksmbd_conn *conn = work->conn;
struct ksmbd_session *sess = work->sess;
struct oplock_info *opinfo;
@@ -39,7 +38,7 @@ static struct oplock_info *alloc_opinfo(struct ksmbd_work *work,
return NULL;
opinfo->sess = sess;
- opinfo->conn = conn;
+ opinfo->conn = ksmbd_conn_get(work->conn);
opinfo->level = SMB2_OPLOCK_LEVEL_NONE;
opinfo->op_state = OPLOCK_STATE_NONE;
opinfo->pending_break = 0;
@@ -50,7 +49,6 @@ static struct oplock_info *alloc_opinfo(struct ksmbd_work *work,
init_waitqueue_head(&opinfo->oplock_brk);
atomic_set(&opinfo->refcount, 1);
atomic_set(&opinfo->breaking_cnt, 0);
- atomic_inc(&opinfo->conn->refcnt);
return opinfo;
}
@@ -132,8 +130,7 @@ static void __free_opinfo(struct oplock_info *opinfo)
{
if (opinfo->is_lease)
free_lease(opinfo);
- if (opinfo->conn && atomic_dec_and_test(&opinfo->conn->refcnt))
- kfree(opinfo->conn);
+ ksmbd_conn_put(opinfo->conn);
kfree(opinfo);
}
@@ -531,7 +528,12 @@ static struct oplock_info *same_client_has_lease(struct ksmbd_inode *ci,
ret = compare_guid_key(opinfo, client_guid, lctx->lease_key);
if (ret) {
+ if (!atomic_inc_not_zero(&opinfo->refcount))
+ continue;
+ if (m_opinfo)
+ opinfo_put(m_opinfo);
m_opinfo = opinfo;
+
/* skip upgrading lease about breaking lease */
if (atomic_read(&opinfo->breaking_cnt))
continue;
@@ -1249,6 +1251,7 @@ int smb_grant_oplock(struct ksmbd_work *work, int req_op_level, u64 pid,
if (atomic_read(&m_opinfo->breaking_cnt))
opinfo->o_lease->flags =
SMB2_LEASE_FLAG_BREAK_IN_PROGRESS_LE;
+ opinfo_put(m_opinfo);
goto out;
}
}
diff --git a/fs/smb/server/server.c b/fs/smb/server/server.c
index 73a07758f97a..4c7924e25417 100644
--- a/fs/smb/server/server.c
+++ b/fs/smb/server/server.c
@@ -587,8 +587,14 @@ static int __init ksmbd_server_init(void)
if (ret)
goto err_crypto_destroy;
+ ret = ksmbd_conn_wq_init();
+ if (ret)
+ goto err_workqueue_destroy;
+
return 0;
+err_workqueue_destroy:
+ ksmbd_workqueue_destroy();
err_crypto_destroy:
ksmbd_crypto_destroy();
err_release_inode_hash:
@@ -614,6 +620,12 @@ static void __exit ksmbd_server_exit(void)
{
ksmbd_server_shutdown();
rcu_barrier();
+ /*
+ * ksmbd_conn_put() defers the final release onto ksmbd_conn_wq,
+ * so drain it after rcu_barrier() has fired any pending RCU
+ * callbacks that may have queued a release.
+ */
+ ksmbd_conn_wq_destroy();
ksmbd_release_inode_hash();
}
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index 2fd1fb03c01e..b7e48e086aaa 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -6311,6 +6311,7 @@ static int set_file_allocation_info(struct ksmbd_work *work,
*/
loff_t alloc_blks;
+ u64 alloc_size;
struct inode *inode;
struct kstat stat;
int rc;
@@ -6323,7 +6324,19 @@ static int set_file_allocation_info(struct ksmbd_work *work,
if (rc)
return rc;
- alloc_blks = (le64_to_cpu(file_alloc_info->AllocationSize) + 511) >> 9;
+ /*
+ * AllocationSize is fully client-controlled (the caller only
+ * validates the fixed 8-byte buffer length). Reject values that
+ * would overflow the "round up to 512-byte blocks" conversion
+ * below instead of silently wrapping it to a tiny block count,
+ * which would truncate the file to a size the client never
+ * asked for.
+ */
+ alloc_size = le64_to_cpu(file_alloc_info->AllocationSize);
+ if (alloc_size > MAX_LFS_FILESIZE - 511)
+ return -EINVAL;
+
+ alloc_blks = (alloc_size + 511) >> 9;
inode = file_inode(fp->filp);
if (alloc_blks > stat.blocks) {
@@ -6526,16 +6539,18 @@ static int smb2_set_info_file(struct ksmbd_work *work, struct ksmbd_file *fp,
}
case FILE_LINK_INFORMATION:
{
- if (!(fp->daccess & FILE_DELETE_LE)) {
- pr_err("no right to delete : 0x%x\n", fp->daccess);
- return -EACCES;
- }
+ struct smb2_file_link_info *file_info;
if (buf_len < sizeof(struct smb2_file_link_info))
return -EINVAL;
- return smb2_create_link(work, work->tcon->share_conf,
- (struct smb2_file_link_info *)buffer,
+ file_info = (struct smb2_file_link_info *)buffer;
+ if (file_info->ReplaceIfExists && !(fp->daccess & FILE_DELETE_LE)) {
+ pr_err("no right to delete : 0x%x\n", fp->daccess);
+ return -EACCES;
+ }
+
+ return smb2_create_link(work, work->tcon->share_conf, file_info,
buf_len, fp->filp,
work->conn->local_nls);
}
@@ -7587,9 +7602,11 @@ int smb2_lock(struct ksmbd_work *work)
nolock = 0;
list_del(&cmp_lock->flist);
list_del(&cmp_lock->clist);
+ cmp_lock->conn = NULL;
spin_unlock(&conn->llist_lock);
up_read(&conn_list_lock);
+ ksmbd_conn_put(conn);
locks_free_lock(cmp_lock->fl);
kfree(cmp_lock);
goto out_check_cl;
@@ -7724,6 +7741,7 @@ int smb2_lock(struct ksmbd_work *work)
goto out2;
} else if (!rc) {
list_add(&smb_lock->llist, &rollback_list);
+ smb_lock->conn = ksmbd_conn_get(work->conn);
spin_lock(&work->conn->llist_lock);
list_add_tail(&smb_lock->clist,
&work->conn->lock_list);
@@ -7778,11 +7796,14 @@ int smb2_lock(struct ksmbd_work *work)
}
list_del(&smb_lock->llist);
- spin_lock(&work->conn->llist_lock);
+ conn = smb_lock->conn;
+ spin_lock(&conn->llist_lock);
if (!list_empty(&smb_lock->flist))
list_del(&smb_lock->flist);
list_del(&smb_lock->clist);
- spin_unlock(&work->conn->llist_lock);
+ smb_lock->conn = NULL;
+ spin_unlock(&conn->llist_lock);
+ ksmbd_conn_put(conn);
locks_free_lock(smb_lock->fl);
if (rlock)
@@ -8189,6 +8210,7 @@ static inline int fsctl_set_sparse(struct ksmbd_work *work, u64 id,
if (fp->f_ci->m_fattr != old_fattr &&
test_share_config_flag(work->tcon->share_conf,
KSMBD_SHARE_FLAG_STORE_DOS_ATTRS)) {
+ const struct cred *saved_cred;
struct xattr_dos_attrib da;
ret = ksmbd_vfs_get_dos_attrib_xattr(idmap,
@@ -8197,9 +8219,11 @@ static inline int fsctl_set_sparse(struct ksmbd_work *work, u64 id,
goto out;
da.attr = le32_to_cpu(fp->f_ci->m_fattr);
+ saved_cred = override_creds(fp->filp->f_cred);
ret = ksmbd_vfs_set_dos_attrib_xattr(idmap,
&fp->filp->f_path,
&da, true);
+ revert_creds(saved_cred);
if (ret)
fp->f_ci->m_fattr = old_fattr;
}
@@ -9140,10 +9164,13 @@ void smb3_preauth_hash_rsp(struct ksmbd_work *work)
WORK_BUFFERS(work, req, rsp);
- if (le16_to_cpu(req->Command) == SMB2_NEGOTIATE_HE &&
- conn->preauth_info)
- ksmbd_gen_preauth_integrity_hash(conn, work->response_buf,
- conn->preauth_info->Preauth_HashValue);
+ if (le16_to_cpu(req->Command) == SMB2_NEGOTIATE_HE) {
+ ksmbd_conn_lock(conn);
+ if (conn->preauth_info)
+ ksmbd_gen_preauth_integrity_hash(conn, work->response_buf,
+ conn->preauth_info->Preauth_HashValue);
+ ksmbd_conn_unlock(conn);
+ }
if (le16_to_cpu(rsp->Command) == SMB2_SESSION_SETUP_HE && sess) {
__u8 *hash_value;
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index f5864dd1dd5f..b09bc8d9389a 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -374,6 +374,7 @@ static void parse_dacl(struct mnt_idmap *idmap,
{
int i, ret;
u16 num_aces = 0;
+ u16 dacl_size;
unsigned int acl_size;
char *acl_base;
struct smb_ace **ppace;
@@ -403,7 +404,11 @@ static void parse_dacl(struct mnt_idmap *idmap,
if (num_aces <= 0)
return;
- if (num_aces > (le16_to_cpu(pdacl->size) - sizeof(struct smb_acl)) /
+ dacl_size = le16_to_cpu(pdacl->size);
+ if (dacl_size < sizeof(struct smb_acl))
+ return;
+
+ if (num_aces > (dacl_size - sizeof(struct smb_acl)) /
(offsetof(struct smb_ace, sid) +
offsetof(struct smb_sid, sub_auth) + sizeof(__le16)))
return;
diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c
index 1f1df7412b56..d77c3e95645d 100644
--- a/fs/smb/server/vfs.c
+++ b/fs/smb/server/vfs.c
@@ -18,7 +18,6 @@
#include <linux/vmalloc.h>
#include <linux/sched/xacct.h>
#include <linux/crc32c.h>
-#include <linux/namei.h>
#include "glob.h"
#include "oplock.h"
@@ -73,7 +72,7 @@ static int ksmbd_vfs_path_lookup_locked(struct ksmbd_share_config *share_conf,
struct qstr last;
struct filename *filename;
struct path *root_share_path = &share_conf->vfs_path;
- int err, type;
+ int err;
struct dentry *d;
if (pathname[0] == '\0') {
@@ -88,19 +87,13 @@ static int ksmbd_vfs_path_lookup_locked(struct ksmbd_share_config *share_conf,
return PTR_ERR(filename);
err = vfs_path_parent_lookup(filename, flags,
- parent_path, &last, &type,
+ parent_path, &last,
root_share_path);
if (err) {
putname(filename);
return err;
}
- if (unlikely(type != LAST_NORM)) {
- path_put(parent_path);
- putname(filename);
- return -ENOENT;
- }
-
err = mnt_want_write(parent_path->mnt);
if (err) {
path_put(parent_path);
@@ -708,22 +701,16 @@ int ksmbd_vfs_rename(struct ksmbd_work *work, const struct path *old_path,
struct filename *to;
struct ksmbd_share_config *share_conf = work->tcon->share_conf;
struct ksmbd_file *parent_fp;
- int new_type;
int err, lookup_flags = LOOKUP_NO_SYMLINKS;
if (ksmbd_override_fsids(work))
return -ENOMEM;
to = getname_kernel(newname);
- if (IS_ERR(to)) {
- err = PTR_ERR(to);
- goto revert_fsids;
- }
retry:
err = vfs_path_parent_lookup(to, lookup_flags | LOOKUP_BENEATH,
- &new_path, &new_last, &new_type,
- &share_conf->vfs_path);
+ &new_path, &new_last, &share_conf->vfs_path);
if (err)
goto out1;
@@ -819,7 +806,6 @@ int ksmbd_vfs_rename(struct ksmbd_work *work, const struct path *old_path,
}
out1:
putname(to);
-revert_fsids:
ksmbd_revert_fsids(work);
return err;
}
@@ -1006,15 +992,21 @@ void ksmbd_vfs_set_fadvise(struct file *filp, __le32 option)
int ksmbd_vfs_zero_data(struct ksmbd_work *work, struct ksmbd_file *fp,
loff_t off, loff_t len)
{
+ const struct cred *saved_cred;
+ int err;
+
smb_break_all_levII_oplock(work, fp, 1);
+ saved_cred = override_creds(fp->filp->f_cred);
if (fp->f_ci->m_fattr & FILE_ATTRIBUTE_SPARSE_FILE_LE)
- return vfs_fallocate(fp->filp,
- FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
- off, len);
-
- return vfs_fallocate(fp->filp,
- FALLOC_FL_ZERO_RANGE | FALLOC_FL_KEEP_SIZE,
- off, len);
+ err = vfs_fallocate(fp->filp,
+ FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
+ off, len);
+ else
+ err = vfs_fallocate(fp->filp,
+ FALLOC_FL_ZERO_RANGE | FALLOC_FL_KEEP_SIZE,
+ off, len);
+ revert_creds(saved_cred);
+ return err;
}
int ksmbd_vfs_fqar_lseek(struct ksmbd_file *fp, loff_t start, loff_t length,
@@ -1333,15 +1325,39 @@ struct dentry *ksmbd_vfs_kern_path_create(struct ksmbd_work *work,
unsigned int flags,
struct path *path)
{
- char *abs_name;
+ struct ksmbd_share_config *share_conf = work->tcon->share_conf;
+ struct qstr last;
+ struct filename *filename;
struct dentry *dent;
+ int err;
+
+ /* resolve the name beneath the share root so ".." cannot escape */
+ filename = getname_kernel(name);
+ if (IS_ERR(filename))
+ return ERR_CAST(filename);
+
+ err = vfs_path_parent_lookup(filename, flags | LOOKUP_BENEATH,
+ path, &last, &share_conf->vfs_path);
+ if (err) {
+ putname(filename);
+ return ERR_PTR(err);
+ }
- abs_name = convert_to_unix_name(work->tcon->share_conf, name);
- if (!abs_name)
- return ERR_PTR(-ENOMEM);
+ err = mnt_want_write(path->mnt);
+ if (err) {
+ path_put(path);
+ putname(filename);
+ return ERR_PTR(err);
+ }
- dent = kern_path_create(AT_FDCWD, abs_name, path, flags);
- kfree(abs_name);
+ inode_lock_nested(path->dentry->d_inode, I_MUTEX_PARENT);
+ dent = lookup_one_qstr_excl(&last, path->dentry, LOOKUP_CREATE);
+ if (IS_ERR(dent)) {
+ inode_unlock(path->dentry->d_inode);
+ mnt_drop_write(path->mnt);
+ path_put(path);
+ }
+ putname(filename);
return dent;
}
diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c
index a1f59a8e16e2..5b6d8bb8edb2 100644
--- a/fs/smb/server/vfs_cache.c
+++ b/fs/smb/server/vfs_cache.c
@@ -391,10 +391,14 @@ static void __ksmbd_close_fd(struct ksmbd_file_table *ft, struct ksmbd_file *fp)
* there are not accesses to fp->lock_list.
*/
list_for_each_entry_safe(smb_lock, tmp_lock, &fp->lock_list, flist) {
- if (!list_empty(&smb_lock->clist) && fp->conn) {
- spin_lock(&fp->conn->llist_lock);
- list_del(&smb_lock->clist);
- spin_unlock(&fp->conn->llist_lock);
+ struct ksmbd_conn *conn = smb_lock->conn;
+
+ if (conn) {
+ spin_lock(&conn->llist_lock);
+ list_del_init(&smb_lock->clist);
+ smb_lock->conn = NULL;
+ spin_unlock(&conn->llist_lock);
+ ksmbd_conn_put(conn);
}
list_del(&smb_lock->flist);
@@ -402,6 +406,17 @@ static void __ksmbd_close_fd(struct ksmbd_file_table *ft, struct ksmbd_file *fp)
kfree(smb_lock);
}
+ /*
+ * Drop fp's strong reference on conn (taken in ksmbd_open_fd() /
+ * ksmbd_reopen_durable_fd()). Durable fps that reached the
+ * scavenger have already had fp->conn cleared by session_fd_check(),
+ * in which case there is nothing to drop here.
+ */
+ if (fp->conn) {
+ ksmbd_conn_put(fp->conn);
+ fp->conn = NULL;
+ }
+
if (ksmbd_stream_fd(fp))
kfree(fp->stream.name);
kfree(fp->owner.name);
@@ -694,7 +709,14 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
atomic_set(&fp->refcount, 1);
fp->filp = filp;
- fp->conn = work->conn;
+ /*
+ * fp owns a strong reference on fp->conn for as long as fp->conn is
+ * non-NULL, so session_fd_check() and __ksmbd_close_fd() never
+ * dereference a dangling pointer. Paired with ksmbd_conn_put() in
+ * session_fd_check() (durable preserve), in __ksmbd_close_fd()
+ * (final close), and on the error paths below.
+ */
+ fp->conn = ksmbd_conn_get(work->conn);
fp->tcon = work->tcon;
fp->volatile_id = KSMBD_NO_FID;
fp->persistent_id = KSMBD_NO_FID;
@@ -716,6 +738,8 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
return fp;
err_out:
+ /* fp->conn was set and refcounted before every branch here. */
+ ksmbd_conn_put(fp->conn);
kmem_cache_free(filp_cache, fp);
return ERR_PTR(ret);
}
@@ -980,16 +1004,21 @@ void ksmbd_stop_durable_scavenger(void)
static int ksmbd_vfs_copy_durable_owner(struct ksmbd_file *fp,
struct ksmbd_user *user)
{
+ char *name;
+
if (!user)
return -EINVAL;
/* Duplicate the user name to ensure identity persistence */
- fp->owner.name = kstrdup(user->name, GFP_KERNEL);
- if (!fp->owner.name)
+ name = kstrdup(user->name, GFP_KERNEL);
+ if (!name)
return -ENOMEM;
+ spin_lock(&fp->f_lock);
fp->owner.uid = user->uid;
fp->owner.gid = user->gid;
+ fp->owner.name = name;
+ spin_unlock(&fp->f_lock);
return 0;
}
@@ -1007,18 +1036,24 @@ static int ksmbd_vfs_copy_durable_owner(struct ksmbd_file *fp,
bool ksmbd_vfs_compare_durable_owner(struct ksmbd_file *fp,
struct ksmbd_user *user)
{
- if (!user || !fp->owner.name)
+ bool ret = false;
+
+ if (!user)
return false;
+ spin_lock(&fp->f_lock);
+ if (!fp->owner.name)
+ goto out;
+
/* Check if the UID and GID match first (fast path) */
if (fp->owner.uid != user->uid || fp->owner.gid != user->gid)
- return false;
+ goto out;
/* Validate the account name to ensure the same SecurityContext */
- if (strcmp(fp->owner.name, user->name))
- return false;
-
- return true;
+ ret = (strcmp(fp->owner.name, user->name) == 0);
+out:
+ spin_unlock(&fp->f_lock);
+ return ret;
}
static bool session_fd_check(struct ksmbd_tree_connect *tcon,
@@ -1032,25 +1067,38 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
if (!is_reconnectable(fp))
return false;
+ if (WARN_ON_ONCE(!fp->conn))
+ return false;
+
if (ksmbd_vfs_copy_durable_owner(fp, user))
return false;
+ /*
+ * fp owns a strong reference on fp->conn (taken in ksmbd_open_fd()
+ * / ksmbd_reopen_durable_fd()), so conn stays valid for the whole
+ * body of this function regardless of any op->conn puts below.
+ */
conn = fp->conn;
ci = fp->f_ci;
down_write(&ci->m_lock);
list_for_each_entry_rcu(op, &ci->m_op_list, op_entry) {
if (op->conn != conn)
continue;
- if (op->conn && atomic_dec_and_test(&op->conn->refcnt))
- kfree(op->conn);
+ ksmbd_conn_put(op->conn);
op->conn = NULL;
}
up_write(&ci->m_lock);
list_for_each_entry_safe(smb_lock, tmp_lock, &fp->lock_list, flist) {
- spin_lock(&fp->conn->llist_lock);
+ struct ksmbd_conn *lock_conn = smb_lock->conn;
+
+ if (!lock_conn)
+ continue;
+ spin_lock(&lock_conn->llist_lock);
list_del_init(&smb_lock->clist);
- spin_unlock(&fp->conn->llist_lock);
+ smb_lock->conn = NULL;
+ spin_unlock(&lock_conn->llist_lock);
+ ksmbd_conn_put(lock_conn);
}
fp->conn = NULL;
@@ -1061,6 +1109,8 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
fp->durable_scavenger_timeout =
jiffies_to_msecs(jiffies) + fp->durable_timeout;
+ /* Drop fp's own reference on conn. */
+ ksmbd_conn_put(conn);
return true;
}
@@ -1135,28 +1185,47 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
struct ksmbd_lock *smb_lock;
unsigned int old_f_state;
+ write_lock(&global_ft.lock);
if (!fp->is_durable || fp->conn || fp->tcon) {
+ write_unlock(&global_ft.lock);
pr_err("Invalid durable fd [%p:%p]\n", fp->conn, fp->tcon);
return -EBADF;
}
if (has_file_id(fp->volatile_id)) {
+ write_unlock(&global_ft.lock);
pr_err("Still in use durable fd: %llu\n", fp->volatile_id);
return -EBADF;
}
+ /*
+ * Initialize fp's connection binding before publishing fp into the
+ * session's file table. If __open_id() is ordered first, a
+ * concurrent teardown that iterates the table can observe a valid
+ * volatile_id with fp->conn == NULL and preserve a
+ * partially-initialized fp. fp owns a strong reference on the new
+ * conn (see ksmbd_open_fd()); undo it on __open_id() failure.
+ */
+ fp->conn = ksmbd_conn_get(conn);
+ fp->tcon = work->tcon;
+ write_unlock(&global_ft.lock);
+
old_f_state = fp->f_state;
fp->f_state = FP_NEW;
+
__open_id(&work->sess->file_table, fp, OPEN_ID_TYPE_VOLATILE_ID);
if (!has_file_id(fp->volatile_id)) {
+ write_lock(&global_ft.lock);
+ fp->conn = NULL;
+ fp->tcon = NULL;
+ write_unlock(&global_ft.lock);
+ ksmbd_conn_put(conn);
fp->f_state = old_f_state;
return -EBADF;
}
- fp->conn = conn;
- fp->tcon = work->tcon;
-
list_for_each_entry(smb_lock, &fp->lock_list, flist) {
+ smb_lock->conn = ksmbd_conn_get(conn);
spin_lock(&conn->llist_lock);
list_add_tail(&smb_lock->clist, &conn->lock_list);
spin_unlock(&conn->llist_lock);
@@ -1167,14 +1236,15 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
list_for_each_entry_rcu(op, &ci->m_op_list, op_entry) {
if (op->conn)
continue;
- op->conn = fp->conn;
- atomic_inc(&op->conn->refcnt);
+ op->conn = ksmbd_conn_get(fp->conn);
}
up_write(&ci->m_lock);
+ spin_lock(&fp->f_lock);
fp->owner.uid = fp->owner.gid = 0;
kfree(fp->owner.name);
fp->owner.name = NULL;
+ spin_unlock(&fp->f_lock);
return 0;
}
diff --git a/fs/smb/server/vfs_cache.h b/fs/smb/server/vfs_cache.h
index 025c2c03b9a9..b658ae8c3b40 100644
--- a/fs/smb/server/vfs_cache.h
+++ b/fs/smb/server/vfs_cache.h
@@ -32,6 +32,7 @@ struct ksmbd_session;
struct ksmbd_lock {
struct file_lock *fl;
+ struct ksmbd_conn *conn;
struct list_head clist;
struct list_head flist;
struct list_head llist;
diff --git a/fs/sysfs/file.c b/fs/sysfs/file.c
index d1995e2d6c94..c76a8b81102b 100644
--- a/fs/sysfs/file.c
+++ b/fs/sysfs/file.c
@@ -114,6 +114,10 @@ static ssize_t sysfs_kf_read(struct kernfs_open_file *of, char *buf,
len = ops->show(kobj, of->kn->priv, buf);
if (len < 0)
return len;
+ if (len >= (ssize_t)PAGE_SIZE) {
+ printk("fill_read_buffer: %pS returned bad count\n", ops->show);
+ len = PAGE_SIZE - 1;
+ }
if (pos) {
if (len <= pos)
return 0;
diff --git a/fs/udf/super.c b/fs/udf/super.c
index 6104fb3594bd..c7c5fd45166b 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -2331,7 +2331,7 @@ static int udf_fill_super(struct super_block *sb, struct fs_context *fc)
error_out:
iput(sbi->s_vat_inode);
- unload_nls(uopt->nls_map);
+ unload_nls(sbi->s_nls_map);
if (lvid_open)
udf_close_lvid(sb);
brelse(sbi->s_lvid_bh);
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index b2e377b7f337..2aa34fcc10a1 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -625,6 +625,8 @@ int acpi_dev_install_notify_handler(struct acpi_device *adev,
void acpi_dev_remove_notify_handler(struct acpi_device *adev,
u32 handler_type,
acpi_notify_handler handler);
+int devm_acpi_install_notify_handler(struct device *dev, u32 handler_type,
+ acpi_notify_handler handler, void *context);
extern int acpi_notifier_call_chain(struct acpi_device *, u32, u32);
extern int register_acpi_notifier(struct notifier_block *);
extern int unregister_acpi_notifier(struct notifier_block *);
diff --git a/include/asm-generic/bitops/lock.h b/include/asm-generic/bitops/lock.h
index 14d4ec8c5152..ffb73b6129e7 100644
--- a/include/asm-generic/bitops/lock.h
+++ b/include/asm-generic/bitops/lock.h
@@ -16,16 +16,16 @@
* It can be used to implement bit locks.
*/
static __always_inline int
-arch_test_and_set_bit_lock(unsigned int nr, volatile unsigned long *p)
+arch_test_and_set_bit_lock(unsigned int nr, volatile unsigned long *addr)
{
long old;
unsigned long mask = BIT_MASK(nr);
- p += BIT_WORD(nr);
- if (READ_ONCE(*p) & mask)
+ addr += BIT_WORD(nr);
+ if (READ_ONCE(*addr) & mask)
return 1;
- old = raw_atomic_long_fetch_or_acquire(mask, (atomic_long_t *)p);
+ old = raw_atomic_long_fetch_or_acquire(mask, (atomic_long_t *)addr);
return !!(old & mask);
}
@@ -38,10 +38,10 @@ arch_test_and_set_bit_lock(unsigned int nr, volatile unsigned long *p)
* This operation is atomic and provides release barrier semantics.
*/
static __always_inline void
-arch_clear_bit_unlock(unsigned int nr, volatile unsigned long *p)
+arch_clear_bit_unlock(unsigned int nr, volatile unsigned long *addr)
{
- p += BIT_WORD(nr);
- raw_atomic_long_fetch_andnot_release(BIT_MASK(nr), (atomic_long_t *)p);
+ addr += BIT_WORD(nr);
+ raw_atomic_long_fetch_andnot_release(BIT_MASK(nr), (atomic_long_t *)addr);
}
/**
@@ -56,14 +56,14 @@ arch_clear_bit_unlock(unsigned int nr, volatile unsigned long *p)
* See for example x86's implementation.
*/
static inline void
-arch___clear_bit_unlock(unsigned int nr, volatile unsigned long *p)
+arch___clear_bit_unlock(unsigned int nr, volatile unsigned long *addr)
{
unsigned long old;
- p += BIT_WORD(nr);
- old = READ_ONCE(*p);
+ addr += BIT_WORD(nr);
+ old = READ_ONCE(*addr);
old &= ~BIT_MASK(nr);
- raw_atomic_long_set_release((atomic_long_t *)p, old);
+ raw_atomic_long_set_release((atomic_long_t *)addr, old);
}
#ifndef arch_xor_unlock_is_negative_byte
diff --git a/include/dt-bindings/power/fsl,imx93-power.h b/include/dt-bindings/power/fsl,imx93-power.h
index 17f9f015bf7d..071221fe5c57 100644
--- a/include/dt-bindings/power/fsl,imx93-power.h
+++ b/include/dt-bindings/power/fsl,imx93-power.h
@@ -11,5 +11,6 @@
#define IMX93_MEDIABLK_PD_PXP 2
#define IMX93_MEDIABLK_PD_LCDIF 3
#define IMX93_MEDIABLK_PD_ISI 4
+#define IMX93_MEDIABLK_PD_MIPI_PHY 5
#endif
diff --git a/include/linux/amba/bus.h b/include/linux/amba/bus.h
index dda2f3ea89cb..4f735883da32 100644
--- a/include/linux/amba/bus.h
+++ b/include/linux/amba/bus.h
@@ -71,11 +71,6 @@ struct amba_device {
unsigned int cid;
struct amba_cs_uci_id uci;
unsigned int irq[AMBA_NR_IRQS];
- /*
- * Driver name to force a match. Do not set directly, because core
- * frees it. Use driver_set_override() to set or clear it.
- */
- const char *driver_override;
};
struct amba_driver {
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e8d9803cc675..9e1731db76d8 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -2216,6 +2216,12 @@ bpf_prog_run_array_uprobe(const struct bpf_prog_array *array,
return ret;
}
+bool bpf_jit_bypass_spec_v1(void);
+bool bpf_jit_bypass_spec_v4(void);
+
+#define bpf_rcu_lock_held() \
+ (rcu_read_lock_held() || rcu_read_lock_trace_held() || rcu_read_lock_bh_held())
+
#ifdef CONFIG_BPF_SYSCALL
DECLARE_PER_CPU(int, bpf_prog_active);
extern struct mutex bpf_stats_enabled_mutex;
@@ -2386,12 +2392,16 @@ static inline bool bpf_allow_uninit_stack(const struct bpf_token *token)
static inline bool bpf_bypass_spec_v1(const struct bpf_token *token)
{
- return cpu_mitigations_off() || bpf_token_capable(token, CAP_PERFMON);
+ return bpf_jit_bypass_spec_v1() ||
+ cpu_mitigations_off() ||
+ bpf_token_capable(token, CAP_PERFMON);
}
static inline bool bpf_bypass_spec_v4(const struct bpf_token *token)
{
- return cpu_mitigations_off() || bpf_token_capable(token, CAP_PERFMON);
+ return bpf_jit_bypass_spec_v4() ||
+ cpu_mitigations_off() ||
+ bpf_token_capable(token, CAP_PERFMON);
}
int bpf_map_new_fd(struct bpf_map *map, int flags);
diff --git a/include/linux/bpf_local_storage.h b/include/linux/bpf_local_storage.h
index dcddb0aef7d8..87836e26ee0f 100644
--- a/include/linux/bpf_local_storage.h
+++ b/include/linux/bpf_local_storage.h
@@ -18,9 +18,6 @@
#define BPF_LOCAL_STORAGE_CACHE_SIZE 16
-#define bpf_rcu_lock_held() \
- (rcu_read_lock_held() || rcu_read_lock_trace_held() || \
- rcu_read_lock_bh_held())
struct bpf_local_storage_map_bucket {
struct hlist_head list;
raw_spinlock_t lock;
diff --git a/include/linux/bpf_lsm.h b/include/linux/bpf_lsm.h
index aefcd6564251..625f4926da53 100644
--- a/include/linux/bpf_lsm.h
+++ b/include/linux/bpf_lsm.h
@@ -14,6 +14,8 @@
#ifdef CONFIG_BPF_LSM
+extern bool bpf_lsm_initialized __ro_after_init;
+
#define LSM_HOOK(RET, DEFAULT, NAME, ...) \
RET bpf_lsm_##NAME(__VA_ARGS__);
#include <linux/lsm_hook_defs.h>
@@ -50,6 +52,8 @@ int bpf_lsm_get_retval_range(const struct bpf_prog *prog,
struct bpf_retval_range *range);
#else /* !CONFIG_BPF_LSM */
+#define bpf_lsm_initialized false
+
static inline bool bpf_lsm_is_sleepable_hook(u32 btf_id)
{
return false;
diff --git a/include/linux/cdx/cdx_bus.h b/include/linux/cdx/cdx_bus.h
index 79bb80e56790..527f8851f0cd 100644
--- a/include/linux/cdx/cdx_bus.h
+++ b/include/linux/cdx/cdx_bus.h
@@ -137,9 +137,6 @@ struct cdx_controller {
* @enabled: is this bus enabled
* @msi_dev_id: MSI Device ID associated with CDX device
* @num_msi: Number of MSI's supported by the device
- * @driver_override: driver name to force a match; do not set directly,
- * because core frees it; use driver_set_override() to
- * set or clear it.
* @irqchip_lock: lock to synchronize irq/msi configuration
* @msi_write_pending: MSI write pending for this device
*/
@@ -165,7 +162,6 @@ struct cdx_device {
bool enabled;
u32 msi_dev_id;
u32 num_msi;
- const char *driver_override;
struct mutex irqchip_lock;
bool msi_write_pending;
};
diff --git a/include/linux/device.h b/include/linux/device.h
index ecbce2530eb7..3463dec1b0c3 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -1023,9 +1023,12 @@ static inline void device_lock_assert(struct device *dev)
static inline bool dev_has_sync_state(struct device *dev)
{
+ struct device_driver *drv;
+
if (!dev)
return false;
- if (dev->driver && dev->driver->sync_state)
+ drv = READ_ONCE(dev->driver);
+ if (drv && drv->sync_state)
return true;
if (dev->bus && dev->bus->sync_state)
return true;
diff --git a/include/linux/dpll.h b/include/linux/dpll.h
index 81f7b623d0ba..3c03b19d9390 100644
--- a/include/linux/dpll.h
+++ b/include/linux/dpll.h
@@ -11,11 +11,13 @@
#include <linux/device.h>
#include <linux/netlink.h>
#include <linux/netdevice.h>
+#include <linux/notifier.h>
#include <linux/rtnetlink.h>
struct dpll_device;
struct dpll_pin;
struct dpll_pin_esync;
+struct fwnode_handle;
struct dpll_device_ops {
int (*mode_get)(const struct dpll_device *dpll, void *dpll_priv,
@@ -91,6 +93,16 @@ struct dpll_pin_ops {
const struct dpll_device *dpll, void *dpll_priv,
struct dpll_pin_esync *esync,
struct netlink_ext_ack *extack);
+ int (*ref_sync_set)(const struct dpll_pin *pin, void *pin_priv,
+ const struct dpll_pin *ref_sync_pin,
+ void *ref_sync_pin_priv,
+ const enum dpll_pin_state state,
+ struct netlink_ext_ack *extack);
+ int (*ref_sync_get)(const struct dpll_pin *pin, void *pin_priv,
+ const struct dpll_pin *ref_sync_pin,
+ void *ref_sync_pin_priv,
+ enum dpll_pin_state *state,
+ struct netlink_ext_ack *extack);
};
struct dpll_pin_frequency {
@@ -137,6 +149,30 @@ struct dpll_pin_properties {
struct dpll_pin_phase_adjust_range phase_range;
};
+#define DPLL_DEVICE_CREATED 1
+#define DPLL_DEVICE_DELETED 2
+#define DPLL_DEVICE_CHANGED 3
+#define DPLL_PIN_CREATED 4
+#define DPLL_PIN_DELETED 5
+#define DPLL_PIN_CHANGED 6
+
+struct dpll_device_notifier_info {
+ struct dpll_device *dpll;
+ u32 id;
+ u32 idx;
+ u64 clock_id;
+ enum dpll_type type;
+};
+
+struct dpll_pin_notifier_info {
+ struct dpll_pin *pin;
+ u32 id;
+ u32 idx;
+ u64 clock_id;
+ const struct fwnode_handle *fwnode;
+ const struct dpll_pin_properties *prop;
+};
+
#if IS_ENABLED(CONFIG_DPLL)
void dpll_netdev_pin_set(struct net_device *dev, struct dpll_pin *dpll_pin);
void dpll_netdev_pin_clear(struct net_device *dev);
@@ -144,6 +180,8 @@ void dpll_netdev_pin_clear(struct net_device *dev);
size_t dpll_netdev_pin_handle_size(const struct net_device *dev);
int dpll_netdev_add_pin_handle(struct sk_buff *msg,
const struct net_device *dev);
+
+struct dpll_pin *fwnode_dpll_pin_find(struct fwnode_handle *fwnode);
#else
static inline void
dpll_netdev_pin_set(struct net_device *dev, struct dpll_pin *dpll_pin) { }
@@ -159,6 +197,12 @@ dpll_netdev_add_pin_handle(struct sk_buff *msg, const struct net_device *dev)
{
return 0;
}
+
+static inline struct dpll_pin *
+fwnode_dpll_pin_find(struct fwnode_handle *fwnode)
+{
+ return NULL;
+}
#endif
struct dpll_device *
@@ -172,6 +216,8 @@ int dpll_device_register(struct dpll_device *dpll, enum dpll_type type,
void dpll_device_unregister(struct dpll_device *dpll,
const struct dpll_device_ops *ops, void *priv);
+#define DPLL_PIN_IDX_UNSPEC U32_MAX
+
struct dpll_pin *
dpll_pin_get(u64 clock_id, u32 dev_driver_id, struct module *module,
const struct dpll_pin_properties *prop);
@@ -184,14 +230,23 @@ void dpll_pin_unregister(struct dpll_device *dpll, struct dpll_pin *pin,
void dpll_pin_put(struct dpll_pin *pin);
+void dpll_pin_fwnode_set(struct dpll_pin *pin, struct fwnode_handle *fwnode);
+
int dpll_pin_on_pin_register(struct dpll_pin *parent, struct dpll_pin *pin,
const struct dpll_pin_ops *ops, void *priv);
void dpll_pin_on_pin_unregister(struct dpll_pin *parent, struct dpll_pin *pin,
const struct dpll_pin_ops *ops, void *priv);
+int dpll_pin_ref_sync_pair_add(struct dpll_pin *pin,
+ struct dpll_pin *ref_sync_pin);
+
int dpll_device_change_ntf(struct dpll_device *dpll);
int dpll_pin_change_ntf(struct dpll_pin *pin);
+int register_dpll_notifier(struct notifier_block *nb);
+
+int unregister_dpll_notifier(struct notifier_block *nb);
+
#endif
diff --git a/include/linux/filter.h b/include/linux/filter.h
index a91f2babf425..b88a213bcbbe 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -22,6 +22,7 @@
#include <linux/vmalloc.h>
#include <linux/sockptr.h>
#include <crypto/sha1.h>
+#include <linux/static_call.h>
#include <linux/u64_stats_sync.h>
#include <net/sch_generic.h>
@@ -1238,6 +1239,15 @@ extern long bpf_jit_limit_max;
typedef void (*bpf_jit_fill_hole_t)(void *area, unsigned int size);
+/*
+ * Flush the indirect branch predictors before reusing JIT memory, so that
+ * indirect jumps into a newly written program don't reuse predictions left
+ * behind by an old program that occupied the same space.
+ */
+void bpf_arch_pred_flush(void);
+DECLARE_STATIC_CALL(bpf_arch_pred_flush, bpf_arch_pred_flush);
+DECLARE_STATIC_KEY_FALSE(bpf_pred_flush_enabled);
+
void bpf_jit_fill_hole_with_zero(void *area, unsigned int size);
struct bpf_binary_header *
@@ -1252,7 +1262,7 @@ void bpf_jit_free(struct bpf_prog *fp);
struct bpf_binary_header *
bpf_jit_binary_pack_hdr(const struct bpf_prog *fp);
-void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns);
+void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool was_classic);
void bpf_prog_pack_free(void *ptr, u32 size);
static inline bool bpf_prog_kallsyms_verify_off(const struct bpf_prog *fp)
@@ -1266,7 +1276,8 @@ bpf_jit_binary_pack_alloc(unsigned int proglen, u8 **ro_image,
unsigned int alignment,
struct bpf_binary_header **rw_hdr,
u8 **rw_image,
- bpf_jit_fill_hole_t bpf_fill_ill_insns);
+ bpf_jit_fill_hole_t bpf_fill_ill_insns,
+ bool was_classic);
int bpf_jit_binary_pack_finalize(struct bpf_binary_header *ro_header,
struct bpf_binary_header *rw_header);
void bpf_jit_binary_pack_free(struct bpf_binary_header *ro_header,
diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index bc59016743fb..c7621a0714c1 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -12,6 +12,10 @@
struct vm_area_struct;
struct mempolicy;
+/* Helper macro to avoid gfp flags if they are the default one */
+#define __default_gfp(a,b,...) b
+#define default_gfp(...) __default_gfp(,##__VA_ARGS__,GFP_KERNEL)
+
/* Convert GFP flags to their corresponding migrate type */
#define GFP_MOVABLE_MASK (__GFP_RECLAIMABLE|__GFP_MOVABLE)
#define GFP_MOVABLE_SHIFT 3
diff --git a/include/linux/hid.h b/include/linux/hid.h
index 7d05b1edacd8..e2734b9d3ee0 100644
--- a/include/linux/hid.h
+++ b/include/linux/hid.h
@@ -154,6 +154,7 @@ struct hid_item {
#define HID_UP_TELEPHONY 0x000b0000
#define HID_UP_CONSUMER 0x000c0000
#define HID_UP_DIGITIZER 0x000d0000
+#define HID_UP_HAPTIC 0x000e0000
#define HID_UP_PID 0x000f0000
#define HID_UP_BATTERY 0x00850000
#define HID_UP_CAMERA 0x00900000
@@ -314,6 +315,28 @@ struct hid_item {
#define HID_DG_TOOLSERIALNUMBER 0x000d005b
#define HID_DG_LATENCYMODE 0x000d0060
+#define HID_HP_SIMPLECONTROLLER 0x000e0001
+#define HID_HP_WAVEFORMLIST 0x000e0010
+#define HID_HP_DURATIONLIST 0x000e0011
+#define HID_HP_AUTOTRIGGER 0x000e0020
+#define HID_HP_MANUALTRIGGER 0x000e0021
+#define HID_HP_AUTOTRIGGERASSOCIATEDCONTROL 0x000e0022
+#define HID_HP_INTENSITY 0x000e0023
+#define HID_HP_REPEATCOUNT 0x000e0024
+#define HID_HP_RETRIGGERPERIOD 0x000e0025
+#define HID_HP_WAVEFORMVENDORPAGE 0x000e0026
+#define HID_HP_WAVEFORMVENDORID 0x000e0027
+#define HID_HP_WAVEFORMCUTOFFTIME 0x000e0028
+#define HID_HP_WAVEFORMNONE 0x000e1001
+#define HID_HP_WAVEFORMSTOP 0x000e1002
+#define HID_HP_WAVEFORMCLICK 0x000e1003
+#define HID_HP_WAVEFORMBUZZCONTINUOUS 0x000e1004
+#define HID_HP_WAVEFORMRUMBLECONTINUOUS 0x000e1005
+#define HID_HP_WAVEFORMPRESS 0x000e1006
+#define HID_HP_WAVEFORMRELEASE 0x000e1007
+#define HID_HP_VENDORWAVEFORMMIN 0x000e2001
+#define HID_HP_VENDORWAVEFORMMAX 0x000e2fff
+
#define HID_BAT_ABSOLUTESTATEOFCHARGE 0x00850065
#define HID_BAT_CHARGING 0x00850044
@@ -419,6 +442,12 @@ struct hid_item {
#define HID_REPORT_PROTOCOL 1
#define HID_BOOT_PROTOCOL 0
+/*
+ * HID units
+ */
+#define HID_UNIT_GRAM 0x0101
+#define HID_UNIT_NEWTON 0xe111
+
/*
* This is the global environment of the parser. This information is
* persistent for main-items. The global environment can be saved and
diff --git a/include/linux/host1x.h b/include/linux/host1x.h
index c4dde3aafcac..c2df9a79b5ce 100644
--- a/include/linux/host1x.h
+++ b/include/linux/host1x.h
@@ -138,6 +138,12 @@ static inline struct host1x_bo_mapping *to_host1x_bo_mapping(struct kref *ref)
return container_of(ref, struct host1x_bo_mapping, ref);
}
+/**
+ * struct host1x_bo_ops - operations implemented by a host1x_bo provider
+ *
+ * @pin: create a DMA mapping. Implementation must not touch the bo's refcount.
+ * @unpin: destroy a DMA mapping. Implementation must not touch the bo's refcount.
+ */
struct host1x_bo_ops {
struct host1x_bo *(*get)(struct host1x_bo *bo);
void (*put)(struct host1x_bo *bo);
@@ -176,6 +182,7 @@ struct host1x_bo_mapping *host1x_bo_pin(struct device *dev, struct host1x_bo *bo
enum dma_data_direction dir,
struct host1x_bo_cache *cache);
void host1x_bo_unpin(struct host1x_bo_mapping *map);
+void host1x_bo_clear_cached_mappings(struct host1x_bo *bo);
static inline void *host1x_bo_mmap(struct host1x_bo *bo)
{
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 44bf8af37901..2f40e2f40aa2 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -1311,11 +1311,6 @@ struct hv_device {
u16 device_id;
struct device device;
- /*
- * Driver name to force a match. Do not set directly, because core
- * frees it. Use driver_set_override() to set or clear it.
- */
- const char *driver_override;
struct vmbus_channel *channel;
struct kset *channels_kset;
diff --git a/include/linux/i3c/master.h b/include/linux/i3c/master.h
index 6e5328c6c6af..414a8acde018 100644
--- a/include/linux/i3c/master.h
+++ b/include/linux/i3c/master.h
@@ -503,7 +503,7 @@ struct i3c_master_controller_ops {
* @boardinfo.i2c: list of I2C boardinfo objects
* @boardinfo: board-level information attached to devices connected on the bus
* @bus: I3C bus exposed by this master
- * @wq: workqueue which can be used by master
+ * @wq: freezable workqueue which can be used by master
* drivers if they need to postpone operations that need to take place
* in a thread context. Typical examples are Hot Join processing which
* requires taking the bus lock in maintenance, which in turn, can only
diff --git a/include/linux/inetdevice.h b/include/linux/inetdevice.h
index 9c0f263e2cd2..d1a51865fc2a 100644
--- a/include/linux/inetdevice.h
+++ b/include/linux/inetdevice.h
@@ -284,6 +284,11 @@ static inline void in_dev_put(struct in_device *idev)
#define __in_dev_put(idev) refcount_dec(&(idev)->refcnt)
#define in_dev_hold(idev) refcount_inc(&(idev)->refcnt)
+static inline bool in_dev_hold_safe(struct in_device *idev)
+{
+ return refcount_inc_not_zero(&idev->refcnt);
+}
+
#endif /* __KERNEL__ */
static __inline__ __be32 inet_make_mask(int logmask)
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 10f7b1df0723..dc7ceb9406ed 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -642,7 +642,7 @@ struct iommu_ops {
struct iommu_domain_ops {
int (*attach_dev)(struct iommu_domain *domain, struct device *dev);
int (*set_dev_pasid)(struct iommu_domain *domain, struct device *dev,
- ioasid_t pasid);
+ ioasid_t pasid, struct iommu_domain *old);
int (*map_pages)(struct iommu_domain *domain, unsigned long iova,
phys_addr_t paddr, size_t pgsize, size_t pgcount,
diff --git a/include/linux/jiffies.h b/include/linux/jiffies.h
index 5d21dacd62bc..91b20788273d 100644
--- a/include/linux/jiffies.h
+++ b/include/linux/jiffies.h
@@ -59,7 +59,7 @@
/* LATCH is used in the interval timer and ftape setup. */
#define LATCH ((CLOCK_TICK_RATE + HZ/2) / HZ) /* For divider */
-extern int register_refined_jiffies(long clock_tick_rate);
+extern void register_refined_jiffies(long clock_tick_rate);
/* TICK_USEC is the time between ticks in usec assuming SHIFTED_HZ */
#define TICK_USEC ((USEC_PER_SEC + HZ/2) / HZ)
@@ -526,6 +526,19 @@ static __always_inline unsigned long msecs_to_jiffies(const unsigned int m)
}
}
+/**
+ * secs_to_jiffies: - convert seconds to jiffies
+ * @_secs: time in seconds
+ *
+ * Conversion is done by simple multiplication with HZ
+ *
+ * secs_to_jiffies() is defined as a macro rather than a static inline
+ * function so it can be used in static initializers.
+ *
+ * Return: jiffies value
+ */
+#define secs_to_jiffies(_secs) (unsigned long)((_secs) * HZ)
+
extern unsigned long __usecs_to_jiffies(const unsigned int u);
#if !(USEC_PER_SEC % HZ)
static inline unsigned long _usecs_to_jiffies(const unsigned int u)
diff --git a/include/linux/namei.h b/include/linux/namei.h
index 8ec8fed3bce8..99b158ca0ce2 100644
--- a/include/linux/namei.h
+++ b/include/linux/namei.h
@@ -12,11 +12,6 @@ enum { MAX_NESTED_LINKS = 8 };
#define MAXSYMLINKS 40
-/*
- * Type of the last component on LOOKUP_PARENT
- */
-enum {LAST_NORM, LAST_ROOT, LAST_DOT, LAST_DOTDOT};
-
/* pathwalk mode */
#define LOOKUP_FOLLOW 0x0001 /* follow links at the end */
#define LOOKUP_DIRECTORY 0x0002 /* require a directory */
@@ -63,7 +58,7 @@ extern void done_path_create(struct path *, struct dentry *);
extern struct dentry *kern_path_locked(const char *, struct path *);
extern struct dentry *user_path_locked_at(int , const char __user *, struct path *);
int vfs_path_parent_lookup(struct filename *filename, unsigned int flags,
- struct path *parent, struct qstr *last, int *type,
+ struct path *parent, struct qstr *last,
const struct path *root);
int vfs_path_lookup(struct dentry *, struct vfsmount *, const char *,
unsigned int, struct path *);
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 83d313718cd5..6bd0dbfe8f0c 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -429,10 +429,9 @@ bool netfs_release_folio(struct folio *folio, gfp_t gfp);
vm_fault_t netfs_page_mkwrite(struct vm_fault *vmf, struct netfs_group *netfs_group);
/* (Sub)request management API. */
-void netfs_read_subreq_progress(struct netfs_io_subrequest *subreq,
- bool was_async);
-void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq,
- int error, bool was_async);
+void netfs_read_subreq_progress(struct netfs_io_subrequest *subreq, bool was_async);
+void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq, bool was_async);
+void netfs_read_subreq_termination_worker(struct work_struct *work);
void netfs_get_subrequest(struct netfs_io_subrequest *subreq,
enum netfs_sreq_ref_trace what);
void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
diff --git a/include/linux/nfs_page.h b/include/linux/nfs_page.h
index 9aed39abc94b..cf085f7f65cd 100644
--- a/include/linux/nfs_page.h
+++ b/include/linux/nfs_page.h
@@ -33,6 +33,7 @@ enum {
PG_TEARDOWN, /* page group sync for destroy */
PG_UNLOCKPAGE, /* page group sync bit in read path */
PG_UPTODATE, /* page group sync bit in read path */
+ PG_READ_FAILED, /* page group saw a read error */
PG_WB_END, /* page group sync bit in write path */
PG_REMOVE, /* page group sync bit in write path */
PG_CONTENDED1, /* Is someone waiting for a lock? */
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index ea751edf247b..b6f948408746 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -576,7 +576,6 @@ struct nfs_lock_args {
struct nfs_lowner lock_owner;
unsigned char block : 1;
unsigned char reclaim : 1;
- unsigned char new_lock : 1;
unsigned char new_lock_owner : 1;
};
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 825e6b3056f1..24e586153f62 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1418,7 +1418,8 @@ static inline int pci_rebar_bytes_to_size(u64 bytes)
}
u32 pci_rebar_get_possible_sizes(struct pci_dev *pdev, int bar);
-int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size);
+int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size,
+ int exclude_bars);
int pci_select_bars(struct pci_dev *dev, unsigned long flags);
bool pci_device_is_present(struct pci_dev *pdev);
void pci_ignore_hotplug(struct pci_dev *dev);
@@ -1488,7 +1489,6 @@ void pci_assign_unassigned_resources(void);
void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge);
void pci_assign_unassigned_bus_resources(struct pci_bus *bus);
void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus);
-int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type);
int pci_enable_resources(struct pci_dev *, int mask);
void pci_assign_irq(struct pci_dev *dev);
struct resource *pci_find_resource(struct pci_dev *dev, struct resource *res);
diff --git a/include/linux/rpmsg.h b/include/linux/rpmsg.h
index 90d8e4475f80..328192936706 100644
--- a/include/linux/rpmsg.h
+++ b/include/linux/rpmsg.h
@@ -41,9 +41,6 @@ struct rpmsg_channel_info {
* rpmsg_device - device that belong to the rpmsg bus
* @dev: the device struct
* @id: device id (used to match between rpmsg drivers and devices)
- * @driver_override: driver name to force a match; do not set directly,
- * because core frees it; use driver_set_override() to
- * set or clear it.
* @src: local address
* @dst: destination address
* @ept: the rpmsg endpoint of this channel
@@ -53,7 +50,6 @@ struct rpmsg_channel_info {
struct rpmsg_device {
struct device dev;
struct rpmsg_device_id id;
- const char *driver_override;
u32 src;
u32 dst;
struct rpmsg_endpoint *ept;
diff --git a/include/linux/rtnetlink.h b/include/linux/rtnetlink.h
index a7da7dfc06a2..f1d8f5964028 100644
--- a/include/linux/rtnetlink.h
+++ b/include/linux/rtnetlink.h
@@ -54,6 +54,10 @@ extern atomic_t dev_unreg_count;
extern struct rw_semaphore pernet_ops_rwsem;
extern struct rw_semaphore net_rwsem;
+#define ASSERT_RTNL() \
+ WARN_ONCE(!rtnl_is_locked(), \
+ "RTNL: assertion failed at %s (%d)\n", __FILE__, __LINE__)
+
#ifdef CONFIG_PROVE_LOCKING
extern bool lockdep_rtnl_is_held(void);
#else
@@ -95,6 +99,61 @@ static inline bool lockdep_rtnl_is_held(void)
#define rcu_replace_pointer_rtnl(rp, p) \
rcu_replace_pointer(rp, p, lockdep_rtnl_is_held())
+#ifdef CONFIG_DEBUG_NET_SMALL_RTNL
+void __rtnl_net_lock(struct net *net);
+void __rtnl_net_unlock(struct net *net);
+void rtnl_net_lock(struct net *net);
+void rtnl_net_unlock(struct net *net);
+int rtnl_net_trylock(struct net *net);
+int rtnl_net_lock_cmp_fn(const struct lockdep_map *a, const struct lockdep_map *b);
+
+bool rtnl_net_is_locked(struct net *net);
+
+#define ASSERT_RTNL_NET(net) \
+ WARN_ONCE(!rtnl_net_is_locked(net), \
+ "RTNL_NET: assertion failed at %s (%d)\n", \
+ __FILE__, __LINE__)
+
+bool lockdep_rtnl_net_is_held(struct net *net);
+
+#define rcu_dereference_rtnl_net(net, p) \
+ rcu_dereference_check(p, lockdep_rtnl_net_is_held(net))
+#define rtnl_net_dereference(net, p) \
+ rcu_dereference_protected(p, lockdep_rtnl_net_is_held(net))
+#define rcu_replace_pointer_rtnl_net(net, rp, p) \
+ rcu_replace_pointer(rp, p, lockdep_rtnl_net_is_held(net))
+#else
+static inline void __rtnl_net_lock(struct net *net) {}
+static inline void __rtnl_net_unlock(struct net *net) {}
+
+static inline void rtnl_net_lock(struct net *net)
+{
+ rtnl_lock();
+}
+
+static inline void rtnl_net_unlock(struct net *net)
+{
+ rtnl_unlock();
+}
+
+static inline int rtnl_net_trylock(struct net *net)
+{
+ return rtnl_trylock();
+}
+
+static inline void ASSERT_RTNL_NET(struct net *net)
+{
+ ASSERT_RTNL();
+}
+
+#define rcu_dereference_rtnl_net(net, p) \
+ rcu_dereference_rtnl(p)
+#define rtnl_net_dereference(net, p) \
+ rtnl_dereference(p)
+#define rcu_replace_pointer_rtnl_net(net, rp, p) \
+ rcu_replace_pointer_rtnl(rp, p)
+#endif
+
static inline struct netdev_queue *dev_ingress_queue(struct net_device *dev)
{
return rtnl_dereference(dev->ingress_queue);
@@ -122,10 +181,6 @@ void rtnetlink_init(void);
void __rtnl_unlock(void);
void rtnl_kfree_skbs(struct sk_buff *head, struct sk_buff *tail);
-#define ASSERT_RTNL() \
- WARN_ONCE(!rtnl_is_locked(), \
- "RTNL: assertion failed at %s (%d)\n", __FILE__, __LINE__)
-
extern int ndo_dflt_fdb_dump(struct sk_buff *skb,
struct netlink_callback *cb,
struct net_device *dev,
diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h
index 5298765d6ca4..20c8645d4f50 100644
--- a/include/linux/seqlock.h
+++ b/include/linux/seqlock.h
@@ -1186,4 +1186,114 @@ done_seqretry_irqrestore(seqlock_t *lock, int seq, unsigned long flags)
if (seq & 1)
read_sequnlock_excl_irqrestore(lock, flags);
}
+
+enum ss_state {
+ ss_done = 0,
+ ss_lock,
+ ss_lock_irqsave,
+ ss_lockless,
+};
+
+struct ss_tmp {
+ enum ss_state state;
+ unsigned long data;
+ spinlock_t *lock;
+ spinlock_t *lock_irqsave;
+};
+
+static inline void __scoped_seqlock_cleanup(struct ss_tmp *sst)
+{
+ if (sst->lock)
+ spin_unlock(sst->lock);
+ if (sst->lock_irqsave)
+ spin_unlock_irqrestore(sst->lock_irqsave, sst->data);
+}
+
+extern void __scoped_seqlock_invalid_target(void);
+
+#if defined(CONFIG_CC_IS_GCC) && CONFIG_GCC_VERSION < 90000
+/*
+ * For some reason some GCC-8 architectures (nios2, alpha) have trouble
+ * determining that the ss_done state is impossible in __scoped_seqlock_next()
+ * below.
+ */
+static inline void __scoped_seqlock_bug(void) { }
+#else
+/*
+ * Canary for compiler optimization -- if the compiler doesn't realize this is
+ * an impossible state, it very likely generates sub-optimal code here.
+ */
+extern void __scoped_seqlock_bug(void);
+#endif
+
+static inline void
+__scoped_seqlock_next(struct ss_tmp *sst, seqlock_t *lock, enum ss_state target)
+{
+ switch (sst->state) {
+ case ss_done:
+ __scoped_seqlock_bug();
+ return;
+
+ case ss_lock:
+ case ss_lock_irqsave:
+ sst->state = ss_done;
+ return;
+
+ case ss_lockless:
+ if (!read_seqretry(lock, sst->data)) {
+ sst->state = ss_done;
+ return;
+ }
+ break;
+ }
+
+ switch (target) {
+ case ss_done:
+ __scoped_seqlock_invalid_target();
+ return;
+
+ case ss_lock:
+ sst->lock = &lock->lock;
+ spin_lock(sst->lock);
+ sst->state = ss_lock;
+ return;
+
+ case ss_lock_irqsave:
+ sst->lock_irqsave = &lock->lock;
+ spin_lock_irqsave(sst->lock_irqsave, sst->data);
+ sst->state = ss_lock_irqsave;
+ return;
+
+ case ss_lockless:
+ sst->data = read_seqbegin(lock);
+ return;
+ }
+}
+
+#define __scoped_seqlock_read(_seqlock, _target, _s) \
+ for (struct ss_tmp _s __cleanup(__scoped_seqlock_cleanup) = \
+ { .state = ss_lockless, .data = read_seqbegin(_seqlock) }; \
+ _s.state != ss_done; \
+ __scoped_seqlock_next(&_s, _seqlock, _target))
+
+/**
+ * scoped_seqlock_read() - execute the read-side critical section
+ * without manual sequence counter handling
+ * or calls to other helpers
+ * @_seqlock: pointer to seqlock_t protecting the data
+ * @_target: an enum ss_state: one of {ss_lock, ss_lock_irqsave, ss_lockless}
+ * indicating the type of critical read section
+ *
+ * Example::
+ *
+ * scoped_seqlock_read (&lock, ss_lock) {
+ * // read-side critical section
+ * }
+ *
+ * Starts with a lockess pass first. If it fails, restarts the critical
+ * section with the lock held.
+ */
+#define scoped_seqlock_read(_seqlock, _target) \
+ __scoped_seqlock_read(_seqlock, _target, __UNIQUE_ID(seqlock))
+
#endif /* __LINUX_SEQLOCK_H */
diff --git a/include/linux/slab.h b/include/linux/slab.h
index b35e2db7eb0e..773843a71960 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -12,6 +12,7 @@
#ifndef _LINUX_SLAB_H
#define _LINUX_SLAB_H
+#include <linux/bug.h>
#include <linux/cache.h>
#include <linux/gfp.h>
#include <linux/overflow.h>
@@ -883,6 +884,105 @@ static __always_inline __alloc_size(1) void *kmalloc_noprof(size_t size, gfp_t f
}
#define kmalloc(...) alloc_hooks(kmalloc_noprof(__VA_ARGS__))
+/**
+ * __alloc_objs - Allocate objects of a given type using
+ * @KMALLOC: which size-based kmalloc wrapper to allocate with.
+ * @GFP: GFP flags for the allocation.
+ * @TYPE: type to allocate space for.
+ * @COUNT: how many @TYPE objects to allocate.
+ *
+ * Returns: Newly allocated pointer to (first) @TYPE of @COUNT-many
+ * allocated @TYPE objects, or NULL on failure.
+ */
+#define __alloc_objs(KMALLOC, GFP, TYPE, COUNT) \
+({ \
+ const size_t __obj_size = size_mul(sizeof(TYPE), COUNT); \
+ (TYPE *)KMALLOC(__obj_size, GFP); \
+})
+
+/**
+ * __alloc_flex - Allocate an object that has a trailing flexible array
+ * @KMALLOC: kmalloc wrapper function to use for allocation.
+ * @GFP: GFP flags for the allocation.
+ * @TYPE: type of structure to allocate space for.
+ * @FAM: The name of the flexible array member of @TYPE structure.
+ * @COUNT: how many @FAM elements to allocate space for.
+ *
+ * Returns: Newly allocated pointer to @TYPE with @COUNT-many trailing
+ * @FAM elements, or NULL on failure or if @COUNT cannot be represented
+ * by the member of @TYPE that counts the @FAM elements (annotated via
+ * __counted_by()).
+ */
+#define __alloc_flex(KMALLOC, GFP, TYPE, FAM, COUNT) \
+({ \
+ const size_t __count = (COUNT); \
+ const size_t __obj_size = struct_size_t(TYPE, FAM, __count); \
+ TYPE *__obj_ptr = KMALLOC(__obj_size, GFP); \
+ __obj_ptr; \
+})
+
+/**
+ * kmalloc_obj - Allocate a single instance of the given type
+ * @VAR_OR_TYPE: Variable or type to allocate.
+ * @...: optional GFP flags for the allocation (GFP_KERNEL when not specified).
+ *
+ * Returns: newly allocated pointer to a @VAR_OR_TYPE on success, or NULL
+ * on failure.
+ */
+#define kmalloc_obj(VAR_OR_TYPE, ...) \
+ __alloc_objs(kmalloc, default_gfp(__VA_ARGS__), typeof(VAR_OR_TYPE), 1)
+
+/**
+ * kmalloc_objs - Allocate an array of the given type
+ * @VAR_OR_TYPE: Variable or type to allocate an array of.
+ * @COUNT: How many elements in the array.
+ * @...: optional GFP flags for the allocation (GFP_KERNEL when not specified).
+ *
+ * Returns: newly allocated pointer to array of @VAR_OR_TYPE on success,
+ * or NULL on failure.
+ */
+#define kmalloc_objs(VAR_OR_TYPE, COUNT, ...) \
+ __alloc_objs(kmalloc, default_gfp(__VA_ARGS__), typeof(VAR_OR_TYPE), COUNT)
+
+/**
+ * kmalloc_flex - Allocate a single instance of the given flexible structure
+ * @VAR_OR_TYPE: Variable or type to allocate (with its flex array).
+ * @FAM: The name of the flexible array member of the structure.
+ * @COUNT: How many flexible array member elements are desired.
+ * @...: optional GFP flags for the allocation (GFP_KERNEL when not specified).
+ *
+ * Returns: newly allocated pointer to @VAR_OR_TYPE on success, NULL on
+ * failure. If @FAM has been annotated with __counted_by(), the allocation
+ * will immediately fail if @COUNT is larger than what the type of the
+ * struct's counter variable can represent.
+ */
+#define kmalloc_flex(VAR_OR_TYPE, FAM, COUNT, ...) \
+ __alloc_flex(kmalloc, default_gfp(__VA_ARGS__), typeof(VAR_OR_TYPE), FAM, COUNT)
+
+/* All kzalloc aliases for kmalloc_(obj|objs|flex). */
+#define kzalloc_obj(P, ...) \
+ __alloc_objs(kzalloc, default_gfp(__VA_ARGS__), typeof(P), 1)
+#define kzalloc_objs(P, COUNT, ...) \
+ __alloc_objs(kzalloc, default_gfp(__VA_ARGS__), typeof(P), COUNT)
+#define kzalloc_flex(P, FAM, COUNT, ...) \
+ __alloc_flex(kzalloc, default_gfp(__VA_ARGS__), typeof(P), FAM, COUNT)
+
+/* All kvmalloc aliases for kmalloc_(obj|objs|flex). */
+#define kvmalloc_obj(P, ...) \
+ __alloc_objs(kvmalloc, default_gfp(__VA_ARGS__), typeof(P), 1)
+#define kvmalloc_objs(P, COUNT, ...) \
+ __alloc_objs(kvmalloc, default_gfp(__VA_ARGS__), typeof(P), COUNT)
+#define kvmalloc_flex(P, FAM, COUNT, ...) \
+ __alloc_flex(kvmalloc, default_gfp(__VA_ARGS__), typeof(P), FAM, COUNT)
+
+/* All kvzalloc aliases for kmalloc_(obj|objs|flex). */
+#define kvzalloc_obj(P, ...) \
+ __alloc_objs(kvzalloc, default_gfp(__VA_ARGS__), typeof(P), 1)
+#define kvzalloc_objs(P, COUNT, ...) \
+ __alloc_objs(kvzalloc, default_gfp(__VA_ARGS__), typeof(P), COUNT)
+#define kvzalloc_flex(P, FAM, COUNT, ...) \
+ __alloc_flex(kvzalloc, default_gfp(__VA_ARGS__), typeof(P), FAM, COUNT)
+
#define kmem_buckets_alloc(_b, _size, _flags) \
alloc_hooks(__kmalloc_node_noprof(PASS_BUCKET_PARAMS(_size, _b), _flags, NUMA_NO_NODE))
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 5321585c778f..bd6ac060450c 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -186,6 +186,7 @@ int rpc_switch_client_transport(struct rpc_clnt *,
const struct rpc_timeout *);
void rpc_shutdown_client(struct rpc_clnt *);
+void rpc_hold_client(struct rpc_clnt *);
void rpc_release_client(struct rpc_clnt *);
void rpc_task_release_transport(struct rpc_task *);
void rpc_task_release_client(struct rpc_task *);
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index 81b952649d35..af35131d8d71 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -389,6 +389,8 @@ struct rpc_xprt * xprt_alloc(struct net *net, size_t size,
unsigned int max_req);
void xprt_free(struct rpc_xprt *);
void xprt_add_backlog(struct rpc_xprt *xprt, struct rpc_task *task);
+void xprt_add_backlog_noncongested(struct rpc_xprt *xprt,
+ struct rpc_task *task);
bool xprt_wake_up_backlog(struct rpc_xprt *xprt, struct rpc_rqst *req);
void xprt_cleanup_ids(void);
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index 1fce4f606770..cb6fd04c6a51 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -245,7 +245,9 @@ static inline bool is_vm_area_hugepages(const void *addr)
* allocated in the vmalloc layer.
*/
#ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC
- return find_vm_area(addr)->page_order > 0;
+ struct vm_struct *area = find_vm_area(addr);
+
+ return area && area->page_order > 0;
#else
return false;
#endif
diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h
index 23642bb1a103..c75327e31e6a 100644
--- a/include/linux/workqueue.h
+++ b/include/linux/workqueue.h
@@ -401,6 +401,7 @@ enum wq_flags {
* http://thread.gmane.org/gmane.linux.kernel/1480396
*/
WQ_POWER_EFFICIENT = 1 << 7,
+ WQ_PERCPU = 1 << 8, /* bound to a specific cpu */
__WQ_DESTROYING = 1 << 15, /* internal: workqueue is destroying */
__WQ_DRAINING = 1 << 16, /* internal: workqueue is draining */
diff --git a/include/net/addrconf.h b/include/net/addrconf.h
index 363dd63babe7..3650adc635ad 100644
--- a/include/net/addrconf.h
+++ b/include/net/addrconf.h
@@ -330,6 +330,11 @@ static inline struct inet6_dev *__in6_dev_get(const struct net_device *dev)
return rcu_dereference_rtnl(dev->ip6_ptr);
}
+static inline struct inet6_dev *__in6_dev_get_rtnl_net(const struct net_device *dev)
+{
+ return rtnl_net_dereference(dev_net(dev), dev->ip6_ptr);
+}
+
/**
* __in6_dev_stats_get - get inet6_dev pointer for stats
* @dev: network device
@@ -341,8 +346,11 @@ static inline struct inet6_dev *__in6_dev_get(const struct net_device *dev)
static inline struct inet6_dev *__in6_dev_stats_get(const struct net_device *dev,
const struct sk_buff *skb)
{
- if (netif_is_l3_master(dev))
+ if (netif_is_l3_master(dev)) {
dev = dev_get_by_index_rcu(dev_net(dev), inet6_iif(skb));
+ if (!dev)
+ return NULL;
+ }
return __in6_dev_get(dev);
}
@@ -415,6 +423,11 @@ static inline void in6_dev_hold(struct inet6_dev *idev)
refcount_inc(&idev->refcnt);
}
+static inline bool in6_dev_hold_safe(struct inet6_dev *idev)
+{
+ return refcount_inc_not_zero(&idev->refcnt);
+}
+
/* called with rcu_read_lock held */
static inline bool ip6_ignore_linkdown(const struct net_device *dev)
{
diff --git a/include/net/bluetooth/hci.h b/include/net/bluetooth/hci.h
index 999ac2705099..4f6490102239 100644
--- a/include/net/bluetooth/hci.h
+++ b/include/net/bluetooth/hci.h
@@ -207,6 +207,13 @@ enum {
*/
HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED,
+ /* When this quirk is set consider Sync Flow Control as supported by
+ * the driver.
+ *
+ * This quirk must be set before hci_register_dev is called.
+ */
+ HCI_QUIRK_SYNC_FLOWCTL_SUPPORTED,
+
/* When this quirk is set, the LE states reported through the
* HCI_LE_READ_SUPPORTED_STATES are invalid/broken.
*
@@ -448,6 +455,7 @@ enum {
HCI_WIDEBAND_SPEECH_ENABLED,
HCI_EVENT_FILTER_CONFIGURED,
HCI_PA_SYNC,
+ HCI_SCO_FLOWCTL,
HCI_DUT_MODE,
HCI_VENDOR_DIAG,
@@ -550,7 +558,8 @@ enum {
#define ESCO_LINK 0x02
/* Low Energy links do not have defined link type. Use invented one */
#define LE_LINK 0x80
-#define ISO_LINK 0x82
+#define CIS_LINK 0x82
+#define BIS_LINK 0x83
#define INVALID_LINK 0xff
/* LMP features */
@@ -1545,6 +1554,11 @@ struct hci_rp_read_tx_power {
__s8 tx_power;
} __packed;
+#define HCI_OP_WRITE_SYNC_FLOWCTL 0x0c2f
+struct hci_cp_write_sync_flowctl {
+ __u8 enable;
+} __packed;
+
#define HCI_OP_READ_PAGE_SCAN_TYPE 0x0c46
struct hci_rp_read_page_scan_type {
__u8 status;
@@ -2958,8 +2972,9 @@ static inline struct hci_iso_hdr *hci_iso_hdr(const struct sk_buff *skb)
#define hci_iso_flags_pack(pb, ts) ((pb & 0x03) | ((ts & 0x01) << 2))
/* ISO data length and flags pack/unpack */
-#define hci_iso_data_len_pack(h, f) ((__u16) ((h) | ((f) << 14)))
-#define hci_iso_data_len(h) ((h) & 0x3fff)
+#define hci_iso_data_len_pack(h, f) ((__u16) (((h) & 0x0fff) | \
+ (((f) & 0x3) << 14)))
+#define hci_iso_data_len(h) ((h) & 0x0fff)
#define hci_iso_data_flags(h) ((h) >> 14)
/* codec transport types */
diff --git a/include/net/bluetooth/hci_core.h b/include/net/bluetooth/hci_core.h
index a0c84a83f25e..c9ec63ab6a6c 100644
--- a/include/net/bluetooth/hci_core.h
+++ b/include/net/bluetooth/hci_core.h
@@ -958,6 +958,7 @@ enum {
HCI_CONN_AUTH_FAILURE,
HCI_CONN_PER_ADV,
HCI_CONN_BIG_CREATED,
+ HCI_CONN_CREATE,
HCI_CONN_CREATE_CIS,
HCI_CONN_CREATE_BIG_SYNC,
HCI_CONN_BIG_SYNC,
@@ -998,7 +999,8 @@ static inline void hci_conn_hash_add(struct hci_dev *hdev, struct hci_conn *c)
case ESCO_LINK:
h->sco_num++;
break;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
h->iso_num++;
break;
}
@@ -1024,7 +1026,8 @@ static inline void hci_conn_hash_del(struct hci_dev *hdev, struct hci_conn *c)
case ESCO_LINK:
h->sco_num--;
break;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
h->iso_num--;
break;
}
@@ -1041,7 +1044,8 @@ static inline unsigned int hci_conn_num(struct hci_dev *hdev, __u8 type)
case SCO_LINK:
case ESCO_LINK:
return h->sco_num;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
return h->iso_num;
default:
return 0;
@@ -1102,7 +1106,7 @@ static inline struct hci_conn *hci_conn_hash_lookup_bis(struct hci_dev *hdev,
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (bacmp(&c->dst, ba) || c->type != ISO_LINK)
+ if (bacmp(&c->dst, ba) || c->type != BIS_LINK)
continue;
if (c->iso_qos.bcast.bis == bis) {
@@ -1124,7 +1128,7 @@ hci_conn_hash_lookup_create_pa_sync(struct hci_dev *hdev)
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != ISO_LINK)
+ if (c->type != BIS_LINK)
continue;
if (!test_bit(HCI_CONN_CREATE_PA_SYNC, &c->flags))
@@ -1150,8 +1154,8 @@ hci_conn_hash_lookup_per_adv_bis(struct hci_dev *hdev,
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (bacmp(&c->dst, ba) || c->type != ISO_LINK ||
- !test_bit(HCI_CONN_PER_ADV, &c->flags))
+ if (bacmp(&c->dst, ba) || c->type != BIS_LINK ||
+ !test_bit(HCI_CONN_PER_ADV, &c->flags))
continue;
if (c->iso_qos.bcast.big == big &&
@@ -1261,7 +1265,7 @@ static inline struct hci_conn *hci_conn_hash_lookup_cis(struct hci_dev *hdev,
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != ISO_LINK || !bacmp(&c->dst, BDADDR_ANY))
+ if (c->type != CIS_LINK)
continue;
/* Match CIG ID if set */
@@ -1293,7 +1297,7 @@ static inline struct hci_conn *hci_conn_hash_lookup_cig(struct hci_dev *hdev,
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != ISO_LINK || !bacmp(&c->dst, BDADDR_ANY))
+ if (c->type != CIS_LINK)
continue;
if (handle == c->iso_qos.ucast.cig) {
@@ -1316,17 +1320,7 @@ static inline struct hci_conn *hci_conn_hash_lookup_big(struct hci_dev *hdev,
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != ISO_LINK)
- continue;
-
- /* An ISO_LINK hcon with BDADDR_ANY as destination
- * address is a Broadcast connection. A Broadcast
- * slave connection is associated with a PA train,
- * so the sync_handle can be used to differentiate
- * from unicast.
- */
- if (bacmp(&c->dst, BDADDR_ANY) &&
- c->sync_handle == HCI_SYNC_HANDLE_INVALID)
+ if (c->type != BIS_LINK)
continue;
if (handle == c->iso_qos.bcast.big) {
@@ -1350,7 +1344,7 @@ hci_conn_hash_lookup_big_sync_pend(struct hci_dev *hdev,
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != ISO_LINK)
+ if (c->type != BIS_LINK)
continue;
if (handle == c->iso_qos.bcast.big && num_bis == c->num_bis) {
@@ -1373,8 +1367,7 @@ hci_conn_hash_lookup_big_state(struct hci_dev *hdev, __u8 handle, __u16 state)
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (bacmp(&c->dst, BDADDR_ANY) || c->type != ISO_LINK ||
- c->state != state)
+ if (c->type != BIS_LINK || c->state != state)
continue;
if (handle == c->iso_qos.bcast.big) {
@@ -1397,8 +1390,8 @@ hci_conn_hash_lookup_pa_sync_big_handle(struct hci_dev *hdev, __u8 big)
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != ISO_LINK ||
- !test_bit(HCI_CONN_PA_SYNC, &c->flags))
+ if (c->type != BIS_LINK ||
+ !test_bit(HCI_CONN_PA_SYNC, &c->flags))
continue;
if (c->iso_qos.bcast.big == big) {
@@ -1420,7 +1413,7 @@ hci_conn_hash_lookup_pa_sync_handle(struct hci_dev *hdev, __u16 sync_handle)
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != ISO_LINK)
+ if (c->type != BIS_LINK)
continue;
/* Ignore the listen hcon, we are looking
@@ -1891,6 +1884,7 @@ void hci_conn_del_sysfs(struct hci_conn *conn);
#define lmp_hold_capable(dev) ((dev)->features[0][0] & LMP_HOLD)
#define lmp_sniff_capable(dev) ((dev)->features[0][0] & LMP_SNIFF)
#define lmp_park_capable(dev) ((dev)->features[0][1] & LMP_PARK)
+#define lmp_sco_capable(dev) ((dev)->features[0][1] & LMP_SCO)
#define lmp_inq_rssi_capable(dev) ((dev)->features[0][3] & LMP_RSSI_INQ)
#define lmp_esco_capable(dev) ((dev)->features[0][3] & LMP_ESCO)
#define lmp_bredr_capable(dev) (!((dev)->features[0][4] & LMP_NO_BREDR))
@@ -2029,7 +2023,8 @@ static inline int hci_proto_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr,
case ESCO_LINK:
return sco_connect_ind(hdev, bdaddr, flags);
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
return iso_connect_ind(hdev, bdaddr, flags);
default:
diff --git a/include/net/bluetooth/l2cap.h b/include/net/bluetooth/l2cap.h
index 00b5d56f369f..8f0df86ee684 100644
--- a/include/net/bluetooth/l2cap.h
+++ b/include/net/bluetooth/l2cap.h
@@ -620,7 +620,8 @@ struct l2cap_chan {
struct l2cap_ops {
char *name;
- struct l2cap_chan *(*new_connection) (struct l2cap_chan *chan);
+ int (*new_connection)(struct l2cap_chan *chan,
+ struct l2cap_chan *new_chan);
int (*recv) (struct l2cap_chan * chan,
struct sk_buff *skb);
void (*teardown) (struct l2cap_chan *chan, int err);
@@ -658,8 +659,8 @@ struct l2cap_conn {
struct sk_buff *rx_skb;
__u32 rx_len;
+ struct ida tx_ida;
__u8 tx_ident;
- struct mutex ident_lock;
struct sk_buff_head pending_rx;
struct work_struct pending_rx_work;
@@ -748,6 +749,7 @@ enum {
FLAG_ECRED_CONN_REQ_SENT,
FLAG_PENDING_SECURITY,
FLAG_HOLD_HCI_CONN,
+ FLAG_DEL,
};
/* Lock nesting levels for L2CAP channels. We need these because lockdep
@@ -884,9 +886,10 @@ static inline __u16 __next_seq(struct l2cap_chan *chan, __u16 seq)
return (seq + 1) % (chan->tx_win_max + 1);
}
-static inline struct l2cap_chan *l2cap_chan_no_new_connection(struct l2cap_chan *chan)
+static inline int l2cap_chan_no_new_connection(struct l2cap_chan *chan,
+ struct l2cap_chan *new_chan)
{
- return NULL;
+ return -EOPNOTSUPP;
}
static inline int l2cap_chan_no_recv(struct l2cap_chan *chan, struct sk_buff *skb)
@@ -962,7 +965,7 @@ int l2cap_chan_send(struct l2cap_chan *chan, struct msghdr *msg, size_t len);
void l2cap_chan_busy(struct l2cap_chan *chan, int busy);
void l2cap_chan_rx_avail(struct l2cap_chan *chan, ssize_t rx_avail);
int l2cap_chan_check_security(struct l2cap_chan *chan, bool initiator);
-void l2cap_chan_set_defaults(struct l2cap_chan *chan);
+void l2cap_chan_set_defaults(struct l2cap_chan *chan, struct l2cap_chan *pchan);
int l2cap_ertm_init(struct l2cap_chan *chan);
void l2cap_chan_add(struct l2cap_conn *conn, struct l2cap_chan *chan);
void __l2cap_chan_add(struct l2cap_conn *conn, struct l2cap_chan *chan);
diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h
index 02e7be19b042..07826c85d31c 100644
--- a/include/net/dropreason-core.h
+++ b/include/net/dropreason-core.h
@@ -6,9 +6,12 @@
#define DEFINE_DROP_REASON(FN, FNe) \
FN(NOT_SPECIFIED) \
FN(NO_SOCKET) \
+ FN(SOCKET_CLOSE) \
+ FN(SOCKET_FILTER) \
+ FN(SOCKET_RCVBUFF) \
+ FN(UNIX_SKIP_OOB) \
FN(PKT_TOO_SMALL) \
FN(TCP_CSUM) \
- FN(SOCKET_FILTER) \
FN(UDP_CSUM) \
FN(NETFILTER_DROP) \
FN(OTHERHOST) \
@@ -18,7 +21,6 @@
FN(UNICAST_IN_L2_MULTICAST) \
FN(XFRM_POLICY) \
FN(IP_NOPROTO) \
- FN(SOCKET_RCVBUFF) \
FN(PROTO_MEM) \
FN(TCP_AUTH_HDR) \
FN(TCP_MD5NOTFOUND) \
@@ -124,12 +126,21 @@ enum skb_drop_reason {
* 3) no valid child socket during 3WHS process
*/
SKB_DROP_REASON_NO_SOCKET,
+ /** @SKB_DROP_REASON_SOCKET_CLOSE: socket is close()d */
+ SKB_DROP_REASON_SOCKET_CLOSE,
+ /** @SKB_DROP_REASON_SOCKET_FILTER: dropped by socket filter */
+ SKB_DROP_REASON_SOCKET_FILTER,
+ /** @SKB_DROP_REASON_SOCKET_RCVBUFF: socket receive buff is full */
+ SKB_DROP_REASON_SOCKET_RCVBUFF,
+ /**
+ * @SKB_DROP_REASON_UNIX_SKIP_OOB: Out-Of-Band data is skipped by
+ * recv() without MSG_OOB so dropped.
+ */
+ SKB_DROP_REASON_UNIX_SKIP_OOB,
/** @SKB_DROP_REASON_PKT_TOO_SMALL: packet size is too small */
SKB_DROP_REASON_PKT_TOO_SMALL,
/** @SKB_DROP_REASON_TCP_CSUM: TCP checksum error */
SKB_DROP_REASON_TCP_CSUM,
- /** @SKB_DROP_REASON_SOCKET_FILTER: dropped by socket filter */
- SKB_DROP_REASON_SOCKET_FILTER,
/** @SKB_DROP_REASON_UDP_CSUM: UDP checksum error */
SKB_DROP_REASON_UDP_CSUM,
/** @SKB_DROP_REASON_NETFILTER_DROP: dropped by netfilter */
@@ -160,8 +171,6 @@ enum skb_drop_reason {
SKB_DROP_REASON_XFRM_POLICY,
/** @SKB_DROP_REASON_IP_NOPROTO: no support for IP protocol */
SKB_DROP_REASON_IP_NOPROTO,
- /** @SKB_DROP_REASON_SOCKET_RCVBUFF: socket receive buff is full */
- SKB_DROP_REASON_SOCKET_RCVBUFF,
/**
* @SKB_DROP_REASON_PROTO_MEM: proto memory limitation, such as
* udp packet drop out of udp_memory_allocated.
diff --git a/include/net/fib_rules.h b/include/net/fib_rules.h
index d17855c52ef9..b5dd8917f56d 100644
--- a/include/net/fib_rules.h
+++ b/include/net/fib_rules.h
@@ -107,6 +107,11 @@ static inline void fib_rule_get(struct fib_rule *rule)
refcount_inc(&rule->refcnt);
}
+static inline bool fib_rule_get_safe(struct fib_rule *rule)
+{
+ return refcount_inc_not_zero(&rule->refcnt);
+}
+
static inline void fib_rule_put(struct fib_rule *rule)
{
if (refcount_dec_and_test(&rule->refcnt))
diff --git a/include/net/gue.h b/include/net/gue.h
index dfca298bec9c..caefd6da8693 100644
--- a/include/net/gue.h
+++ b/include/net/gue.h
@@ -80,7 +80,7 @@ static inline size_t guehdr_flags_len(__be16 flags)
static inline size_t guehdr_priv_flags_len(__be32 flags)
{
- return 0;
+ return (flags & GUE_PFLAG_REMCSUM) ? GUE_PLEN_REMCSUM : 0;
}
/* Validate standard and private flags. Returns non-zero (meaning invalid)
diff --git a/include/net/ip_fib.h b/include/net/ip_fib.h
index 339b92cd5cec..50060441fa93 100644
--- a/include/net/ip_fib.h
+++ b/include/net/ip_fib.h
@@ -373,7 +373,7 @@ static inline int fib_lookup(struct net *net, struct flowi4 *flp,
struct fib_result *res, unsigned int flags)
{
struct fib_table *tb;
- int err = -ENETUNREACH;
+ int err = -EAGAIN;
flags |= FIB_LOOKUP_NOREF;
if (net->ipv4.fib_has_custom_rules)
@@ -387,17 +387,16 @@ static inline int fib_lookup(struct net *net, struct flowi4 *flp,
if (tb)
err = fib_table_lookup(tb, flp, res, flags);
- if (!err)
+ if (err != -EAGAIN)
goto out;
tb = rcu_dereference_rtnl(net->ipv4.fib_default);
if (tb)
err = fib_table_lookup(tb, flp, res, flags);
-out:
if (err == -EAGAIN)
err = -ENETUNREACH;
-
+out:
rcu_read_unlock();
return err;
diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h
index d70268cf1af8..08b90d33acdc 100644
--- a/include/net/ip_vs.h
+++ b/include/net/ip_vs.h
@@ -516,7 +516,8 @@ struct ip_vs_protocol {
void (*state_transition)(struct ip_vs_conn *cp, int direction,
const struct sk_buff *skb,
- struct ip_vs_proto_data *pd);
+ struct ip_vs_proto_data *pd,
+ unsigned int iph_len);
int (*register_app)(struct netns_ipvs *ipvs, struct ip_vs_app *inc);
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index 022ee2fc627c..47181fd749b2 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -189,6 +189,10 @@ struct net {
#if IS_ENABLED(CONFIG_SMC)
struct netns_smc smc;
#endif
+#ifdef CONFIG_DEBUG_NET_SMALL_RTNL
+ /* Move to a better place when the config guard is removed. */
+ struct mutex rtnl_mutex;
+#endif
} __randomize_layout;
#include <linux/seq_file_net.h>
diff --git a/include/net/netfilter/nf_conntrack_extend.h b/include/net/netfilter/nf_conntrack_extend.h
index 0b247248b032..fd5c4dbf72ca 100644
--- a/include/net/netfilter/nf_conntrack_extend.h
+++ b/include/net/netfilter/nf_conntrack_extend.h
@@ -38,7 +38,6 @@ enum nf_ct_ext_id {
struct nf_ct_ext {
u8 offset[NF_CT_EXT_NUM];
u8 len;
- unsigned int gen_id;
char data[] __aligned(8);
};
@@ -52,8 +51,6 @@ static inline bool nf_ct_ext_exist(const struct nf_conn *ct, u8 id)
return (ct->ext && __nf_ct_ext_exist(ct->ext, id));
}
-void *__nf_ct_ext_find(const struct nf_ct_ext *ext, u8 id);
-
static inline void *nf_ct_ext_find(const struct nf_conn *ct, u8 id)
{
struct nf_ct_ext *ext = ct->ext;
@@ -61,19 +58,10 @@ static inline void *nf_ct_ext_find(const struct nf_conn *ct, u8 id)
if (!ext || !__nf_ct_ext_exist(ext, id))
return NULL;
- if (unlikely(ext->gen_id))
- return __nf_ct_ext_find(ext, id);
-
return (void *)ct->ext + ct->ext->offset[id];
}
/* Add this type, returns pointer to data or NULL. */
void *nf_ct_ext_add(struct nf_conn *ct, enum nf_ct_ext_id id, gfp_t gfp);
-/* ext genid. if ext->id != ext_genid, extensions cannot be used
- * anymore unless conntrack has CONFIRMED bit set.
- */
-extern atomic_t nf_conntrack_ext_genid;
-void nf_ct_ext_bump_genid(void);
-
#endif /* _NF_CONNTRACK_EXTEND_H */
diff --git a/include/net/netfilter/nf_queue.h b/include/net/netfilter/nf_queue.h
index 3978c3174cdb..fc3e81c07364 100644
--- a/include/net/netfilter/nf_queue.h
+++ b/include/net/netfilter/nf_queue.h
@@ -18,6 +18,7 @@ struct nf_queue_entry {
unsigned int id;
unsigned int hook_index; /* index in hook_entries->hook[] */
#if IS_ENABLED(CONFIG_BRIDGE_NETFILTER)
+ struct net_device *bridge_dev;
struct net_device *physin;
struct net_device *physout;
#endif
diff --git a/include/net/scm.h b/include/net/scm.h
index 0d35c7c77a74..ea85f7427a19 100644
--- a/include/net/scm.h
+++ b/include/net/scm.h
@@ -69,7 +69,7 @@ static __inline__ void unix_get_peersec_dgram(struct socket *sock, struct scm_co
static __inline__ void scm_set_cred(struct scm_cookie *scm,
struct pid *pid, kuid_t uid, kgid_t gid)
{
- scm->pid = get_pid(pid);
+ scm->pid = get_pid(pid);
scm->creds.pid = pid_vnr(pid);
scm->creds.uid = uid;
scm->creds.gid = gid;
@@ -78,7 +78,7 @@ static __inline__ void scm_set_cred(struct scm_cookie *scm,
static __inline__ void scm_destroy_cred(struct scm_cookie *scm)
{
put_pid(scm->pid);
- scm->pid = NULL;
+ scm->pid = NULL;
}
static __inline__ void scm_destroy(struct scm_cookie *scm)
diff --git a/include/net/sctp/sctp.h b/include/net/sctp/sctp.h
index 84e6b9fd5610..faef88b5dffd 100644
--- a/include/net/sctp/sctp.h
+++ b/include/net/sctp/sctp.h
@@ -112,7 +112,8 @@ int sctp_transport_lookup_process(sctp_callback_t cb, struct net *net,
const union sctp_addr *paddr, void *p, int dif);
int sctp_transport_traverse_process(sctp_callback_t cb, sctp_callback_t cb_done,
struct net *net, int *pos, void *p);
-int sctp_for_each_endpoint(int (*cb)(struct sctp_endpoint *, void *), void *p);
+int sctp_for_each_endpoint(int (*cb)(struct sctp_endpoint *, void *),
+ struct net *net, int *pos, void *p);
int sctp_get_sctp_info(struct sock *sk, struct sctp_association *asoc,
struct sctp_info *info);
diff --git a/include/net/tc_act/tc_pedit.h b/include/net/tc_act/tc_pedit.h
index cb7b82f2cbc7..97754ea0a827 100644
--- a/include/net/tc_act/tc_pedit.h
+++ b/include/net/tc_act/tc_pedit.h
@@ -37,17 +37,15 @@ static inline bool is_tcf_pedit(const struct tc_action *a)
return false;
}
-static inline int tcf_pedit_nkeys(const struct tc_action *a)
+/* Must be called with act->tcfa_lock held to ensure consistency of parallel
+ * reads of the same action's pedit keys (e.g. flow_offload count vs fill).
+ * Note, this is only used for pedit offload.
+ */
+static inline int tcf_pedit_nkeys_locked(const struct tc_action *a)
{
- struct tcf_pedit_parms *parms;
- int nkeys;
-
- rcu_read_lock();
- parms = to_pedit_parms(a);
- nkeys = parms->tcfp_nkeys;
- rcu_read_unlock();
-
- return nkeys;
+ lockdep_assert_held(&a->tcfa_lock);
+ return rcu_dereference_protected(to_pedit(a)->parms,
+ lockdep_is_held(&a->tcfa_lock))->tcfp_nkeys;
}
static inline u32 tcf_pedit_htype(const struct tc_action *a, int index)
diff --git a/include/net/udp_tunnel.h b/include/net/udp_tunnel.h
index 8ed36ec520d7..ce4e7ad28cde 100644
--- a/include/net/udp_tunnel.h
+++ b/include/net/udp_tunnel.h
@@ -130,22 +130,6 @@ void udp_tunnel_drop_rx_port(struct net_device *dev, struct socket *sock,
void udp_tunnel_notify_add_rx_port(struct socket *sock, unsigned short type);
void udp_tunnel_notify_del_rx_port(struct socket *sock, unsigned short type);
-static inline void udp_tunnel_get_rx_info(struct net_device *dev)
-{
- ASSERT_RTNL();
- if (!(dev->features & NETIF_F_RX_UDP_TUNNEL_PORT))
- return;
- call_netdevice_notifiers(NETDEV_UDP_TUNNEL_PUSH_INFO, dev);
-}
-
-static inline void udp_tunnel_drop_rx_info(struct net_device *dev)
-{
- ASSERT_RTNL();
- if (!(dev->features & NETIF_F_RX_UDP_TUNNEL_PORT))
- return;
- call_netdevice_notifiers(NETDEV_UDP_TUNNEL_DROP_INFO, dev);
-}
-
/* Transmit the skb using UDP encapsulation. */
void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb,
__be32 src, __be32 dst, __u8 tos, __u8 ttl,
@@ -206,19 +190,17 @@ static inline void udp_tunnel_encap_enable(struct sock *sk)
#define UDP_TUNNEL_NIC_MAX_TABLES 4
enum udp_tunnel_nic_info_flags {
- /* Device callbacks may sleep */
- UDP_TUNNEL_NIC_INFO_MAY_SLEEP = BIT(0),
/* Device only supports offloads when it's open, all ports
* will be removed before close and re-added after open.
*/
- UDP_TUNNEL_NIC_INFO_OPEN_ONLY = BIT(1),
+ UDP_TUNNEL_NIC_INFO_OPEN_ONLY = BIT(0),
/* Device supports only IPv4 tunnels */
- UDP_TUNNEL_NIC_INFO_IPV4_ONLY = BIT(2),
+ UDP_TUNNEL_NIC_INFO_IPV4_ONLY = BIT(1),
/* Device has hard-coded the IANA VXLAN port (4789) as VXLAN.
* This port must not be counted towards n_entries of any table.
* Driver will not receive any callback associated with port 4789.
*/
- UDP_TUNNEL_NIC_INFO_STATIC_IANA_VXLAN = BIT(3),
+ UDP_TUNNEL_NIC_INFO_STATIC_IANA_VXLAN = BIT(2),
};
struct udp_tunnel_nic;
@@ -309,6 +291,9 @@ struct udp_tunnel_nic_ops {
size_t (*dump_size)(struct net_device *dev, unsigned int table);
int (*dump_write)(struct net_device *dev, unsigned int table,
struct sk_buff *skb);
+ void (*assert_locked)(struct net_device *dev);
+ void (*lock)(struct net_device *dev);
+ void (*unlock)(struct net_device *dev);
};
#ifdef CONFIG_INET
@@ -337,8 +322,28 @@ static inline void
udp_tunnel_nic_set_port_priv(struct net_device *dev, unsigned int table,
unsigned int idx, u8 priv)
{
- if (udp_tunnel_nic_ops)
+ if (udp_tunnel_nic_ops) {
+ udp_tunnel_nic_ops->assert_locked(dev);
udp_tunnel_nic_ops->set_port_priv(dev, table, idx, priv);
+ }
+}
+
+static inline void udp_tunnel_nic_assert_locked(struct net_device *dev)
+{
+ if (udp_tunnel_nic_ops)
+ udp_tunnel_nic_ops->assert_locked(dev);
+}
+
+static inline void udp_tunnel_nic_lock(struct net_device *dev)
+{
+ if (udp_tunnel_nic_ops)
+ udp_tunnel_nic_ops->lock(dev);
+}
+
+static inline void udp_tunnel_nic_unlock(struct net_device *dev)
+{
+ if (udp_tunnel_nic_ops)
+ udp_tunnel_nic_ops->unlock(dev);
}
static inline void
@@ -380,17 +385,50 @@ static inline void udp_tunnel_nic_reset_ntf(struct net_device *dev)
static inline size_t
udp_tunnel_nic_dump_size(struct net_device *dev, unsigned int table)
{
+ size_t ret;
+
if (!udp_tunnel_nic_ops)
return 0;
- return udp_tunnel_nic_ops->dump_size(dev, table);
+
+ udp_tunnel_nic_ops->lock(dev);
+ ret = udp_tunnel_nic_ops->dump_size(dev, table);
+ udp_tunnel_nic_ops->unlock(dev);
+
+ return ret;
}
static inline int
udp_tunnel_nic_dump_write(struct net_device *dev, unsigned int table,
struct sk_buff *skb)
{
+ int ret;
+
if (!udp_tunnel_nic_ops)
return 0;
- return udp_tunnel_nic_ops->dump_write(dev, table, skb);
+
+ udp_tunnel_nic_ops->lock(dev);
+ ret = udp_tunnel_nic_ops->dump_write(dev, table, skb);
+ udp_tunnel_nic_ops->unlock(dev);
+
+ return ret;
+}
+
+static inline void udp_tunnel_get_rx_info(struct net_device *dev)
+{
+ ASSERT_RTNL();
+ if (!(dev->features & NETIF_F_RX_UDP_TUNNEL_PORT))
+ return;
+ udp_tunnel_nic_assert_locked(dev);
+ call_netdevice_notifiers(NETDEV_UDP_TUNNEL_PUSH_INFO, dev);
}
+
+static inline void udp_tunnel_drop_rx_info(struct net_device *dev)
+{
+ ASSERT_RTNL();
+ if (!(dev->features & NETIF_F_RX_UDP_TUNNEL_PORT))
+ return;
+ udp_tunnel_nic_assert_locked(dev);
+ call_netdevice_notifiers(NETDEV_UDP_TUNNEL_DROP_INFO, dev);
+}
+
#endif
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index b51d65cd965e..a20a01dd6f98 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -878,6 +878,9 @@ static inline bool addr_match(const void *token1, const void *token2,
unsigned int pdw;
unsigned int pbi;
+ if (prefixlen > 128)
+ return false;
+
pdw = prefixlen >> 5; /* num of whole u32 in prefix */
pbi = prefixlen & 0x1f; /* num of bits in incomplete u32 in prefix */
@@ -902,6 +905,10 @@ static inline bool addr4_match(__be32 a1, __be32 a2, u8 prefixlen)
/* C99 6.5.7 (3): u32 << 32 is undefined behaviour */
if (sizeof(long) == 4 && prefixlen == 0)
return true;
+
+ if (prefixlen > 32)
+ return false;
+
return !((a1 ^ a2) & htonl(~0UL << (32 - prefixlen)));
}
@@ -1185,8 +1192,8 @@ int __xfrm_policy_check(struct sock *, int dir, struct sk_buff *skb,
static inline bool __xfrm_check_nopolicy(struct net *net, struct sk_buff *skb,
int dir)
{
- if (!net->xfrm.policy_count[dir] && !secpath_exists(skb))
- return net->xfrm.policy_default[dir] == XFRM_USERPOLICY_ACCEPT;
+ if (!READ_ONCE(net->xfrm.policy_count[dir]) && !secpath_exists(skb))
+ return READ_ONCE(net->xfrm.policy_default[dir]) == XFRM_USERPOLICY_ACCEPT;
return false;
}
@@ -1286,8 +1293,8 @@ static inline int xfrm_route_forward(struct sk_buff *skb, unsigned short family)
{
struct net *net = dev_net(skb->dev);
- if (!net->xfrm.policy_count[XFRM_POLICY_OUT] &&
- net->xfrm.policy_default[XFRM_POLICY_OUT] == XFRM_USERPOLICY_ACCEPT)
+ if (!READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_OUT]) &&
+ READ_ONCE(net->xfrm.policy_default[XFRM_POLICY_OUT]) == XFRM_USERPOLICY_ACCEPT)
return true;
return (skb_dst(skb)->flags & DST_NOXFRM) ||
diff --git a/include/trace/events/afs.h b/include/trace/events/afs.h
index 3dddfc6abf0e..4fcfb2411f9d 100644
--- a/include/trace/events/afs.h
+++ b/include/trace/events/afs.h
@@ -170,6 +170,7 @@ enum yfs_cm_operation {
#define afs_cell_traces \
EM(afs_cell_trace_alloc, "ALLOC ") \
EM(afs_cell_trace_free, "FREE ") \
+ EM(afs_cell_trace_get_atcell, "GET atcell") \
EM(afs_cell_trace_get_queue_dns, "GET q-dns ") \
EM(afs_cell_trace_get_queue_manage, "GET q-mng ") \
EM(afs_cell_trace_get_queue_new, "GET q-new ") \
@@ -177,6 +178,7 @@ enum yfs_cm_operation {
EM(afs_cell_trace_get_vol, "GET vol ") \
EM(afs_cell_trace_insert, "INSERT ") \
EM(afs_cell_trace_manage, "MANAGE ") \
+ EM(afs_cell_trace_put_atcell, "PUT atcell") \
EM(afs_cell_trace_put_candidate, "PUT candid") \
EM(afs_cell_trace_put_destroy, "PUT destry") \
EM(afs_cell_trace_put_queue_work, "PUT q-work") \
@@ -188,8 +190,10 @@ enum yfs_cm_operation {
EM(afs_cell_trace_unuse_alias, "UNU alias ") \
EM(afs_cell_trace_unuse_check_alias, "UNU chk-al") \
EM(afs_cell_trace_unuse_delete, "UNU delete") \
+ EM(afs_cell_trace_unuse_dynroot_mntpt, "UNU dyn-mp") \
EM(afs_cell_trace_unuse_fc, "UNU fc ") \
EM(afs_cell_trace_unuse_lookup, "UNU lookup") \
+ EM(afs_cell_trace_unuse_lookup_dynroot, "UNU lu-dyn") \
EM(afs_cell_trace_unuse_mntpt, "UNU mntpt ") \
EM(afs_cell_trace_unuse_no_pin, "UNU no-pin") \
EM(afs_cell_trace_unuse_parse, "UNU parse ") \
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 5a5cdb453935..c8fb6abfeca9 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -6592,6 +6592,7 @@ struct bpf_prog_info {
__u32 verified_insns;
__u32 attach_btf_obj_id;
__u32 attach_btf_id;
+ __u32 :32;
} __attribute__((aligned(8)));
struct bpf_map_info {
diff --git a/include/uapi/linux/dpll.h b/include/uapi/linux/dpll.h
index b0654ade7b7e..7fa641b3a20a 100644
--- a/include/uapi/linux/dpll.h
+++ b/include/uapi/linux/dpll.h
@@ -213,6 +213,7 @@ enum dpll_a_pin {
DPLL_A_PIN_ESYNC_FREQUENCY,
DPLL_A_PIN_ESYNC_FREQUENCY_SUPPORTED,
DPLL_A_PIN_ESYNC_PULSE,
+ DPLL_A_PIN_REFERENCE_SYNC,
__DPLL_A_PIN_MAX,
DPLL_A_PIN_MAX = (__DPLL_A_PIN_MAX - 1)
diff --git a/io_uring/rw.c b/io_uring/rw.c
index ab2c092d82b4..a744cb5d235a 100644
--- a/io_uring/rw.c
+++ b/io_uring/rw.c
@@ -546,16 +546,15 @@ static void io_complete_rw_iopoll(struct kiocb *kiocb, long res)
{
struct io_rw *rw = container_of(kiocb, struct io_rw, kiocb);
struct io_kiocb *req = cmd_to_io_kiocb(rw);
+ int final_res = io_fixup_rw_res(req, res);
if (kiocb->ki_flags & IOCB_WRITE)
io_req_end_write(req);
- if (unlikely(res != req->cqe.res)) {
- if (res == -EAGAIN && io_rw_should_reissue(req)) {
- req->flags |= REQ_F_REISSUE | REQ_F_BL_NO_RECYCLE;
- return;
- }
- req->cqe.res = res;
- }
+
+ if (res == -EAGAIN && io_rw_should_reissue(req))
+ req->flags |= REQ_F_REISSUE | REQ_F_BL_NO_RECYCLE;
+ else if (unlikely(final_res != req->cqe.res))
+ req->cqe.res = final_res;
/* order with io_iopoll_complete() checking ->iopoll_completed */
smp_store_release(&req->iopoll_completed, 1);
diff --git a/kernel/bpf/bpf_inode_storage.c b/kernel/bpf/bpf_inode_storage.c
index e16e79f8cd6d..564ba5f414f5 100644
--- a/kernel/bpf/bpf_inode_storage.c
+++ b/kernel/bpf/bpf_inode_storage.c
@@ -182,6 +182,15 @@ static int notsupp_get_next_key(struct bpf_map *map, void *key,
static struct bpf_map *inode_storage_map_alloc(union bpf_attr *attr)
{
+ /*
+ * Do not allow allocation of BPF_MAP_TYPE_INODE_STORAGE if the BPF LSM
+ * was not initialized by the LSM framework at boot. Without proper
+ * initialization, the BPF inode security blob offset remains unprepared,
+ * causing bpf_inode() to calculate an invalid memory offset and corrupt
+ * inode->i_security.
+ */
+ if (!bpf_lsm_initialized)
+ return ERR_PTR(-EOPNOTSUPP);
return bpf_local_storage_map_alloc(attr, &inode_cache, false);
}
diff --git a/kernel/bpf/bpf_lsm.c b/kernel/bpf/bpf_lsm.c
index 0849453b3617..969a6ca362ec 100644
--- a/kernel/bpf/bpf_lsm.c
+++ b/kernel/bpf/bpf_lsm.c
@@ -50,6 +50,9 @@ BTF_ID(func, bpf_lsm_key_getsecurity)
#ifdef CONFIG_AUDIT
BTF_ID(func, bpf_lsm_audit_rule_match)
#endif
+#ifdef CONFIG_SECURITY_NETWORK_XFRM
+BTF_ID(func, bpf_lsm_xfrm_decode_session)
+#endif
BTF_ID(func, bpf_lsm_ismaclabel)
BTF_SET_END(bpf_lsm_disabled_hooks)
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index 26ed09cdc856..ad054a4c782e 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -769,19 +769,65 @@ static int cgroup_bpf_attach(struct cgroup *cgrp,
return ret;
}
+static int effective_prog_pos(struct cgroup *cgrp,
+ enum cgroup_bpf_attach_type atype,
+ struct bpf_prog_list *target_pl)
+{
+ int cnt = 0, preorder_cnt = 0, fstart, bstart, init_bstart, pos = -1;
+ struct bpf_prog_list *pl;
+ struct cgroup *p = cgrp;
+
+ /* count effective programs to find where the preorder region ends */
+ do {
+ if (cnt == 0 || (p->bpf.flags[atype] & BPF_F_ALLOW_MULTI))
+ cnt += prog_list_length(&p->bpf.progs[atype], &preorder_cnt);
+ p = cgroup_parent(p);
+ } while (p);
+
+ /* replay compute_effective_progs() placement and record target's slot */
+ cnt = 0;
+ p = cgrp;
+ fstart = preorder_cnt;
+ bstart = preorder_cnt - 1;
+ do {
+ if (cnt > 0 && !(p->bpf.flags[atype] & BPF_F_ALLOW_MULTI))
+ continue;
+
+ init_bstart = bstart;
+ hlist_for_each_entry(pl, &p->bpf.progs[atype], node) {
+ if (!prog_list_prog(pl))
+ continue;
+
+ if (pl->flags & BPF_F_PREORDER) {
+ if (pl == target_pl)
+ pos = bstart;
+ bstart--;
+ } else {
+ if (pl == target_pl)
+ pos = fstart;
+ fstart++;
+ }
+ cnt++;
+ }
+
+ /* reverse pre-ordering progs at this cgroup level */
+ if (pos >= bstart + 1 && pos <= init_bstart)
+ pos = bstart + 1 + init_bstart - pos;
+ } while ((p = cgroup_parent(p)));
+
+ return pos;
+}
+
/* Swap updated BPF program for given link in effective program arrays across
* all descendant cgroups. This function is guaranteed to succeed.
*/
static void replace_effective_prog(struct cgroup *cgrp,
enum cgroup_bpf_attach_type atype,
- struct bpf_cgroup_link *link)
+ struct bpf_prog_list *pl)
{
struct bpf_prog_array_item *item;
struct cgroup_subsys_state *css;
struct bpf_prog_array *progs;
- struct bpf_prog_list *pl;
- struct hlist_head *head;
- struct cgroup *cg;
int pos;
css_for_each_descendant_pre(css, &cgrp->self) {
@@ -790,27 +836,15 @@ static void replace_effective_prog(struct cgroup *cgrp,
if (percpu_ref_is_zero(&desc->bpf.refcnt))
continue;
- /* find position of link in effective progs array */
- for (pos = 0, cg = desc; cg; cg = cgroup_parent(cg)) {
- if (pos && !(cg->bpf.flags[atype] & BPF_F_ALLOW_MULTI))
- continue;
+ pos = effective_prog_pos(desc, atype, pl);
+ if (WARN_ON_ONCE(pos < 0))
+ continue;
- head = &cg->bpf.progs[atype];
- hlist_for_each_entry(pl, head, node) {
- if (!prog_list_prog(pl))
- continue;
- if (pl->link == link)
- goto found;
- pos++;
- }
- }
-found:
- BUG_ON(!cg);
progs = rcu_dereference_protected(
desc->bpf.effective[atype],
lockdep_is_held(&cgroup_mutex));
item = &progs->items[pos];
- WRITE_ONCE(item->prog, link->link.prog);
+ WRITE_ONCE(item->prog, pl->link->link.prog);
}
}
@@ -853,7 +887,7 @@ static int __cgroup_bpf_replace(struct cgroup *cgrp,
return -ENOENT;
old_prog = xchg(&link->link.prog, new_prog);
- replace_effective_prog(cgrp, atype, link);
+ replace_effective_prog(cgrp, atype, pl);
bpf_prog_put(old_prog);
return 0;
}
@@ -920,19 +954,14 @@ static struct bpf_prog_list *find_detach_entry(struct hlist_head *progs,
* recomputing the array in place.
*
* @cgrp: The cgroup which descendants to travers
- * @prog: A program to detach or NULL
- * @link: A link to detach or NULL
+ * @pl: The prog_list entry being detached
* @atype: Type of detach operation
*/
-static void purge_effective_progs(struct cgroup *cgrp, struct bpf_prog *prog,
- struct bpf_cgroup_link *link,
+static void purge_effective_progs(struct cgroup *cgrp, struct bpf_prog_list *pl,
enum cgroup_bpf_attach_type atype)
{
struct cgroup_subsys_state *css;
struct bpf_prog_array *progs;
- struct bpf_prog_list *pl;
- struct hlist_head *head;
- struct cgroup *cg;
int pos;
/* recompute effective prog array in place */
@@ -942,24 +971,11 @@ static void purge_effective_progs(struct cgroup *cgrp, struct bpf_prog *prog,
if (percpu_ref_is_zero(&desc->bpf.refcnt))
continue;
- /* find position of link or prog in effective progs array */
- for (pos = 0, cg = desc; cg; cg = cgroup_parent(cg)) {
- if (pos && !(cg->bpf.flags[atype] & BPF_F_ALLOW_MULTI))
- continue;
-
- head = &cg->bpf.progs[atype];
- hlist_for_each_entry(pl, head, node) {
- if (!prog_list_prog(pl))
- continue;
- if (pl->prog == prog && pl->link == link)
- goto found;
- pos++;
- }
- }
-
+ pos = effective_prog_pos(desc, atype, pl);
/* no link or prog match, skip the cgroup of this layer */
- continue;
-found:
+ if (pos < 0)
+ continue;
+
progs = rcu_dereference_protected(
desc->bpf.effective[atype],
lockdep_is_held(&cgroup_mutex));
@@ -1020,7 +1036,7 @@ static int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
/* if update effective array failed replace the prog with a dummy prog*/
pl->prog = old_prog;
pl->link = link;
- purge_effective_progs(cgrp, old_prog, link, atype);
+ purge_effective_progs(cgrp, pl, atype);
}
/* now can actually delete it from this cgroup list */
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 517710c89fa5..6bc3dc998ae1 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -38,6 +38,7 @@
#include <linux/bpf_mem_alloc.h>
#include <linux/memcontrol.h>
#include <linux/execmem.h>
+#include <linux/static_call.h>
#include <asm/barrier.h>
#include <linux/unaligned.h>
@@ -884,6 +885,7 @@ int bpf_jit_add_poke_descriptor(struct bpf_prog *prog,
struct bpf_prog_pack {
struct list_head list;
void *ptr;
+ bool arch_flush_needed;
unsigned long bitmap[];
};
@@ -892,6 +894,15 @@ void bpf_jit_fill_hole_with_zero(void *area, unsigned int size)
memset(area, 0, size);
}
+DEFINE_STATIC_CALL_NULL(bpf_arch_pred_flush, bpf_arch_pred_flush);
+
+/*
+ * Enabled once bpf_arch_pred_flush points at a real flush routine. Lets the
+ * pack allocator test "is a predictor flush wired up at all" with a cheap
+ * static branch instead of repeatedly querying the static call target.
+ */
+DEFINE_STATIC_KEY_FALSE(bpf_pred_flush_enabled);
+
#define BPF_PROG_SIZE_TO_NBITS(size) (round_up(size, BPF_PROG_CHUNK_SIZE) / BPF_PROG_CHUNK_SIZE)
static DEFINE_MUTEX(pack_mutex);
@@ -928,6 +939,8 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins
bpf_fill_ill_insns(pack->ptr, BPF_PROG_PACK_SIZE);
bitmap_zero(pack->bitmap, BPF_PROG_PACK_SIZE / BPF_PROG_CHUNK_SIZE);
+ if (static_branch_unlikely(&bpf_pred_flush_enabled))
+ pack->arch_flush_needed = true;
set_vm_flush_reset_perms(pack->ptr);
err = set_memory_rox((unsigned long)pack->ptr,
BPF_PROG_PACK_SIZE / PAGE_SIZE);
@@ -942,15 +955,23 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins
return NULL;
}
-void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
+void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool was_classic)
{
unsigned int nbits = BPF_PROG_SIZE_TO_NBITS(size);
- struct bpf_prog_pack *pack;
- unsigned long pos;
+ struct bpf_prog_pack *pack, *fallback_pack = NULL;
+ unsigned long pos, fallback_pos = 0;
void *ptr = NULL;
mutex_lock(&pack_mutex);
if (size > BPF_PROG_PACK_SIZE) {
+ /*
+ * Allocations larger than a pack get their own pages, and
+ * predictors are not flushed for such allocation. This is only
+ * safe because cBPF programs (the unprivileged attack surface)
+ * are bounded well below a pack size.
+ */
+ if (was_classic && static_branch_unlikely(&bpf_pred_flush_enabled))
+ pr_warn_once("BPF: Predictors not flushed for allocations greater than BPF_PROG_PACK_SIZE\n");
size = round_up(size, PAGE_SIZE);
ptr = bpf_jit_alloc_exec(size);
if (ptr) {
@@ -970,8 +991,29 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
list_for_each_entry(pack, &pack_list, list) {
pos = bitmap_find_next_zero_area(pack->bitmap, BPF_PROG_CHUNK_COUNT, 0,
nbits, 0);
- if (pos < BPF_PROG_CHUNK_COUNT)
+ if (pos >= BPF_PROG_CHUNK_COUNT)
+ continue;
+ /* Flush not enabled, use any pack */
+ if (!static_branch_unlikely(&bpf_pred_flush_enabled))
+ goto found_free_area;
+ /*
+ * cBPF reuse of a dirty pack triggers a flush, so prefer a
+ * clean pack for cBPF. eBPF never flushes, so steer it to a
+ * dirty pack and keep clean packs free for cBPF.
+ */
+ if (was_classic ^ pack->arch_flush_needed)
goto found_free_area;
+ if (!fallback_pack) {
+ fallback_pack = pack;
+ fallback_pos = pos;
+ }
+ }
+
+ /* No preferred pack found */
+ if (fallback_pack) {
+ pack = fallback_pack;
+ pos = fallback_pos;
+ goto found_free_area;
}
pack = alloc_new_pack(bpf_fill_ill_insns);
@@ -981,6 +1023,16 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
pos = 0;
found_free_area:
+ /* Flush only for cBPF as it may contain a crafted gadget */
+ if (static_branch_unlikely(&bpf_pred_flush_enabled) &&
+ pack->arch_flush_needed &&
+ was_classic) {
+ struct bpf_prog_pack *p;
+
+ static_call_cond(bpf_arch_pred_flush)();
+ list_for_each_entry(p, &pack_list, list)
+ p->arch_flush_needed = false;
+ }
bitmap_set(pack->bitmap, pos, nbits);
ptr = (void *)(pack->ptr) + (pos << BPF_PROG_CHUNK_SHIFT);
@@ -1018,6 +1070,9 @@ void bpf_prog_pack_free(void *ptr, u32 size)
"bpf_prog_pack bug: missing bpf_arch_text_invalidate?\n");
bitmap_clear(pack->bitmap, pos, nbits);
+
+ if (static_branch_unlikely(&bpf_pred_flush_enabled))
+ pack->arch_flush_needed = true;
if (bitmap_find_next_zero_area(pack->bitmap, BPF_PROG_CHUNK_COUNT, 0,
BPF_PROG_CHUNK_COUNT, 0) == 0) {
list_del(&pack->list);
@@ -1140,7 +1195,8 @@ bpf_jit_binary_pack_alloc(unsigned int proglen, u8 **image_ptr,
unsigned int alignment,
struct bpf_binary_header **rw_header,
u8 **rw_image,
- bpf_jit_fill_hole_t bpf_fill_ill_insns)
+ bpf_jit_fill_hole_t bpf_fill_ill_insns,
+ bool was_classic)
{
struct bpf_binary_header *ro_header;
u32 size, hole, start;
@@ -1153,7 +1209,7 @@ bpf_jit_binary_pack_alloc(unsigned int proglen, u8 **image_ptr,
if (bpf_jit_charge_modmem(size))
return NULL;
- ro_header = bpf_prog_pack_alloc(size, bpf_fill_ill_insns);
+ ro_header = bpf_prog_pack_alloc(size, bpf_fill_ill_insns, was_classic);
if (!ro_header) {
bpf_jit_uncharge_modmem(size);
return NULL;
@@ -2377,7 +2433,7 @@ static bool __bpf_prog_map_compatible(struct bpf_map *map,
cookie = aux->cgroup_storage[i] ?
aux->cgroup_storage[i]->cookie : 0;
ret = map->owner->storage_cookie[i] == cookie ||
- !cookie;
+ (!cookie && !aux->tail_call_reachable);
}
if (ret &&
map->owner->attach_func_proto != aux->attach_func_proto) {
@@ -3032,6 +3088,21 @@ bool __weak bpf_jit_needs_zext(void)
return false;
}
+/* By default, enable the verifier's mitigations against Spectre v1 and v4 for
+ * all archs. The value returned must not change at runtime as there is
+ * currently no support for reloading programs that were loaded without
+ * mitigations.
+ */
+bool __weak bpf_jit_bypass_spec_v1(void)
+{
+ return false;
+}
+
+bool __weak bpf_jit_bypass_spec_v4(void)
+{
+ return false;
+}
+
/* Return true if the JIT inlines the call to the helper corresponding to
* the imm.
*
diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c
index c19eb814bbfd..1c64c4cff41e 100644
--- a/kernel/bpf/devmap.c
+++ b/kernel/bpf/devmap.c
@@ -698,6 +698,18 @@ int dev_map_generic_redirect(struct bpf_dtab_netdev *dst, struct sk_buff *skb,
if (unlikely(err))
return err;
+ if (dst->xdp_prog && skb_cloned(skb)) {
+ struct sk_buff *nskb;
+
+ nskb = skb_copy(skb, GFP_ATOMIC);
+ if (!nskb)
+ return -ENOMEM;
+
+ nskb->mac_len = skb->mac_len;
+ consume_skb(skb);
+ skb = nskb;
+ }
+
/* Redirect has already succeeded semantically at this point, so we just
* return 0 even if packet is dropped. Helper below takes care of
* freeing skb.
diff --git a/kernel/bpf/dispatcher.c b/kernel/bpf/dispatcher.c
index b77db7413f8c..ea2d60dc1fee 100644
--- a/kernel/bpf/dispatcher.c
+++ b/kernel/bpf/dispatcher.c
@@ -145,7 +145,7 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
mutex_lock(&d->mutex);
if (!d->image) {
- d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero);
+ d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
if (!d->image)
goto out;
d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 9f9026a619d5..66eaf95f9dbe 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -685,8 +685,7 @@ static void *__htab_map_lookup_elem(struct bpf_map *map, void *key)
struct htab_elem *l;
u32 hash, key_size;
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
key_size = map->key_size;
@@ -1122,8 +1121,7 @@ static long htab_map_update_elem(struct bpf_map *map, void *key, void *value,
/* unknown flags */
return -EINVAL;
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
key_size = map->key_size;
@@ -1240,8 +1238,7 @@ static long htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value
/* unknown flags */
return -EINVAL;
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
key_size = map->key_size;
@@ -1309,8 +1306,7 @@ static long __htab_percpu_map_update_elem(struct bpf_map *map, void *key,
/* unknown flags */
return -EINVAL;
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
key_size = map->key_size;
@@ -1364,8 +1360,7 @@ static long __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key,
/* unknown flags */
return -EINVAL;
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
key_size = map->key_size;
@@ -1442,8 +1437,7 @@ static long htab_map_delete_elem(struct bpf_map *map, void *key)
u32 hash, key_size;
int ret;
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
key_size = map->key_size;
@@ -1478,8 +1472,7 @@ static long htab_lru_map_delete_elem(struct bpf_map *map, void *key)
u32 hash, key_size;
int ret;
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
key_size = map->key_size;
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 8aa7ca9c34c7..3126ec5097f1 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -37,8 +37,7 @@
*/
BPF_CALL_2(bpf_map_lookup_elem, struct bpf_map *, map, void *, key)
{
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
return (unsigned long) map->ops->map_lookup_elem(map, key);
}
@@ -54,8 +53,7 @@ const struct bpf_func_proto bpf_map_lookup_elem_proto = {
BPF_CALL_4(bpf_map_update_elem, struct bpf_map *, map, void *, key,
void *, value, u64, flags)
{
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
return map->ops->map_update_elem(map, key, value, flags);
}
@@ -72,8 +70,7 @@ const struct bpf_func_proto bpf_map_update_elem_proto = {
BPF_CALL_2(bpf_map_delete_elem, struct bpf_map *, map, void *, key)
{
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
return map->ops->map_delete_elem(map, key);
}
@@ -129,8 +126,7 @@ const struct bpf_func_proto bpf_map_peek_elem_proto = {
BPF_CALL_3(bpf_map_lookup_percpu_elem, struct bpf_map *, map, void *, key, u32, cpu)
{
- WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() &&
- !rcu_read_lock_bh_held());
+ WARN_ON_ONCE(!bpf_rcu_lock_held());
return (unsigned long) map->ops->map_lookup_percpu_elem(map, key, cpu);
}
diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c
index 746b5644d9a1..57c2c3196999 100644
--- a/kernel/bpf/inode.c
+++ b/kernel/bpf/inode.c
@@ -779,10 +779,18 @@ static void bpf_destroy_inode(struct inode *inode)
{
enum bpf_type type;
- if (S_ISLNK(inode->i_mode))
- kfree(inode->i_link);
if (!bpf_inode_type(inode, &type))
bpf_any_put(inode->i_private, type);
+}
+
+/*
+ * Called after RCU grace period - safe to free inode and anything
+ * that might be accessed by RCU pathwalk (inode fields, i_link).
+ */
+static void bpf_free_inode(struct inode *inode)
+{
+ if (S_ISLNK(inode->i_mode))
+ kfree(inode->i_link);
free_inode_nonrcu(inode);
}
@@ -791,6 +799,7 @@ const struct super_operations bpf_super_ops = {
.drop_inode = generic_delete_inode,
.show_options = bpf_show_options,
.destroy_inode = bpf_destroy_inode,
+ .free_inode = bpf_free_inode,
};
enum {
diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
index 010e91ac978e..4d373c8279cc 100644
--- a/kernel/bpf/lpm_trie.c
+++ b/kernel/bpf/lpm_trie.c
@@ -15,6 +15,7 @@
#include <net/ipv6.h>
#include <uapi/linux/btf.h>
#include <linux/btf_ids.h>
+#include <linux/bpf_local_storage.h>
/* Intermediate node */
#define LPM_TREE_NODE_FLAG_IM BIT(0)
@@ -244,7 +245,7 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key)
/* Start walking the trie from the root node ... */
- for (node = rcu_dereference_check(trie->root, rcu_read_lock_bh_held());
+ for (node = rcu_dereference_check(trie->root, bpf_rcu_lock_held());
node;) {
unsigned int next_bit;
size_t matchlen;
@@ -278,7 +279,7 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key)
*/
next_bit = extract_bit(key->data, node->prefixlen);
node = rcu_dereference_check(node->child[next_bit],
- rcu_read_lock_bh_held());
+ bpf_rcu_lock_held());
}
if (!found)
@@ -361,8 +362,7 @@ static long trie_update_elem(struct bpf_map *map,
*/
slot = &trie->root;
- while ((node = rcu_dereference_protected(*slot,
- lockdep_is_held(&trie->lock)))) {
+ while ((node = rcu_dereference_protected(*slot, 1))) {
matchlen = longest_prefix_match(trie, node, key);
if (node->prefixlen != matchlen ||
@@ -483,8 +483,7 @@ static long trie_delete_elem(struct bpf_map *map, void *_key)
trim = &trie->root;
trim2 = trim;
parent = NULL;
- while ((node = rcu_dereference_protected(
- *trim, lockdep_is_held(&trie->lock)))) {
+ while ((node = rcu_dereference_protected(*trim, 1))) {
matchlen = longest_prefix_match(trie, node, key);
if (node->prefixlen != matchlen ||
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 04c8755c0b95..3bdfac9b4418 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -4568,10 +4568,11 @@ static int bpf_prog_get_info_by_fd(struct file *file,
u32 info_len = attr->info.info_len;
struct bpf_prog_kstats stats;
char __user *uinsns;
- u32 ulen;
+ u32 ulen, len;
int err;
- err = bpf_check_uarg_tail_zero(USER_BPFPTR(uinfo), sizeof(info), info_len);
+ len = offsetofend(struct bpf_prog_info, attach_btf_id);
+ err = bpf_check_uarg_tail_zero(USER_BPFPTR(uinfo), len, info_len);
if (err)
return err;
info_len = min_t(u32, sizeof(info), info_len);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 64a6ec8eb847..7f1144486a8c 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -4769,7 +4769,8 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env,
bool sanitize = reg && is_spillable_regtype(reg->type);
for (i = 0; i < size; i++) {
- u8 type = state->stack[spi].slot_type[i];
+ u8 type = state->stack[spi].slot_type[(slot - i) %
+ BPF_REG_SIZE];
if (type != STACK_MISC && type != STACK_ZERO) {
sanitize = true;
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 3f0804794b98..4bc1ea983fe7 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2586,7 +2586,12 @@ void cpuset_update_tasks_nodemask(struct cpuset *cs)
migrate = is_memory_migrate(cs);
- mpol_rebind_mm(mm, &cs->mems_allowed);
+ /*
+ * For v1 we can have empty effective_mems, but we cannot
+ * attach any tasks (see cpuset_can_attach_check()). For v2,
+ * effective_mems is guaranteed to not be empty.
+ */
+ mpol_rebind_mm(mm, &cs->effective_mems);
if (migrate)
cpuset_migrate_mm(mm, &cs->old_mems_allowed, &newmems);
else
diff --git a/kernel/cpu.c b/kernel/cpu.c
index cf02a629f990..7047e36092ce 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -174,7 +174,7 @@ static int cpuhp_invoke_callback(unsigned int cpu, enum cpuhp_state state,
struct cpuhp_step *step = cpuhp_get_step(state);
int (*cbm)(unsigned int cpu, struct hlist_node *node);
int (*cb)(unsigned int cpu);
- int ret, cnt;
+ int ret, cnt, rollback_ret;
if (st->fail == state) {
st->fail = CPUHP_INVALID;
@@ -238,12 +238,12 @@ static int cpuhp_invoke_callback(unsigned int cpu, enum cpuhp_state state,
break;
trace_cpuhp_multi_enter(cpu, st->target, state, cbm, node);
- ret = cbm(cpu, node);
- trace_cpuhp_exit(cpu, st->state, state, ret);
+ rollback_ret = cbm(cpu, node);
+ trace_cpuhp_exit(cpu, st->state, state, rollback_ret);
/*
* Rollback must not fail,
*/
- WARN_ON_ONCE(ret);
+ WARN_ON_ONCE(rollback_ret);
}
return ret;
}
@@ -2864,21 +2864,17 @@ static const struct attribute_group cpuhp_cpu_attr_group = {
NULL
};
-static ssize_t states_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t states_show(struct device *dev, struct device_attribute *attr, char *buf)
{
- ssize_t cur, res = 0;
+ ssize_t res = 0;
int i;
mutex_lock(&cpuhp_state_mutex);
for (i = CPUHP_OFFLINE; i <= CPUHP_ONLINE; i++) {
struct cpuhp_step *sp = cpuhp_get_step(i);
- if (sp->name) {
- cur = sprintf(buf, "%3d: %s\n", i, sp->name);
- buf += cur;
- res += cur;
- }
+ if (sp->name)
+ res += sysfs_emit_at(buf, res, "%3d: %s\n", i, sp->name);
}
mutex_unlock(&cpuhp_state_mutex);
return res;
diff --git a/kernel/fork.c b/kernel/fork.c
index c81b45509e8a..6191f1f8bde8 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1213,6 +1213,11 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
tsk->mm_cid_active = 0;
tsk->migrate_from_cpu = -1;
#endif
+
+#ifdef CONFIG_BPF_SYSCALL
+ RCU_INIT_POINTER(tsk->bpf_storage, NULL);
+ tsk->bpf_ctx = NULL;
+#endif
return tsk;
free_stack:
@@ -2368,10 +2373,6 @@ __latent_entropy struct task_struct *copy_process(
p->sequential_io = 0;
p->sequential_io_avg = 0;
#endif
-#ifdef CONFIG_BPF_SYSCALL
- RCU_INIT_POINTER(p->bpf_storage, NULL);
- p->bpf_ctx = NULL;
-#endif
/* Perform scheduler related setup. Assign this task to a CPU. */
retval = sched_fork(clone_flags, p);
diff --git a/kernel/kcov.c b/kernel/kcov.c
index 187ba1b80bda..bbfb4689f81c 100644
--- a/kernel/kcov.c
+++ b/kernel/kcov.c
@@ -1086,10 +1086,10 @@ static void __init selftest(void)
* potentially traced functions in this region.
*/
start = jiffies;
- current->kcov_mode = KCOV_MODE_TRACE_PC;
+ WRITE_ONCE(current->kcov_mode, KCOV_MODE_TRACE_PC);
while ((jiffies - start) * MSEC_PER_SEC / HZ < 300)
;
- current->kcov_mode = 0;
+ WRITE_ONCE(current->kcov_mode, 0);
pr_err("done running self test\n");
}
#endif
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 49dbb52842c9..ea106403abd8 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4705,6 +4705,7 @@ int sched_fork(unsigned long clone_flags, struct task_struct *p)
p->policy = SCHED_NORMAL;
p->static_prio = NICE_TO_PRIO(0);
p->rt_priority = 0;
+ p->timer_slack_ns = p->default_timer_slack_ns;
} else if (PRIO_TO_NICE(p->static_prio) < 0)
p->static_prio = NICE_TO_PRIO(0);
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 7d14e9fa53ac..27dcb5a9be4f 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -378,10 +378,11 @@ static ssize_t sched_fair_server_write(struct file *filp, const char __user *ubu
return -EINVAL;
}
- if (rq->cfs.h_nr_queued) {
- update_rq_clock(rq);
- dl_server_stop(&rq->fair_server);
- }
+ if (!cpu_online(cpu_of(rq)))
+ return -EBUSY;
+
+ update_rq_clock(rq);
+ dl_server_stop(&rq->fair_server);
retval = dl_server_apply_params(&rq->fair_server, runtime, period, 0);
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index f36512892adf..5a09448e22c3 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8101,25 +8101,32 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
static unsigned long
cpu_util(int cpu, struct task_struct *p, int dst_cpu, int boost)
{
+ bool add_task = p && task_cpu(p) != cpu && dst_cpu == cpu;
+ bool sub_task = p && task_cpu(p) == cpu && dst_cpu != cpu;
struct cfs_rq *cfs_rq = &cpu_rq(cpu)->cfs;
unsigned long util = READ_ONCE(cfs_rq->avg.util_avg);
unsigned long runnable;
- if (boost) {
- runnable = READ_ONCE(cfs_rq->avg.runnable_avg);
- util = max(util, runnable);
- }
-
/*
* If @dst_cpu is -1 or @p migrates from @cpu to @dst_cpu remove its
* contribution. If @p migrates from another CPU to @cpu add its
* contribution. In all the other cases @cpu is not impacted by the
* migration so its util_avg is already correct.
*/
- if (p && task_cpu(p) == cpu && dst_cpu != cpu)
- lsub_positive(&util, task_util(p));
- else if (p && task_cpu(p) != cpu && dst_cpu == cpu)
+ if (add_task)
util += task_util(p);
+ else if (sub_task)
+ lsub_positive(&util, task_util(p));
+
+ if (boost) {
+ runnable = READ_ONCE(cfs_rq->avg.runnable_avg);
+ if (add_task)
+ runnable += READ_ONCE(p->se.avg.runnable_avg);
+ else if (sub_task)
+ lsub_positive(&runnable,
+ READ_ONCE(p->se.avg.runnable_avg));
+ util = max(util, runnable);
+ }
if (sched_feat(UTIL_EST)) {
unsigned long util_est;
@@ -10739,7 +10746,7 @@ static inline void update_sg_wakeup_stats(struct sched_domain *sd,
if (sd->flags & SD_ASYM_CPUCAPACITY)
sgs->group_misfit_task_load = 1;
- for_each_cpu(i, sched_group_span(group)) {
+ for_each_cpu_and(i, sched_group_span(group), p->cpus_ptr) {
struct rq *rq = cpu_rq(i);
unsigned int local;
diff --git a/kernel/taskstats.c b/kernel/taskstats.c
index f0702a21a15f..f7ce4d19c053 100644
--- a/kernel/taskstats.c
+++ b/kernel/taskstats.c
@@ -210,13 +210,39 @@ static int fill_stats_for_pid(pid_t pid, struct taskstats *stats)
return 0;
}
+static void tgid_stats_add_task(struct taskstats *stats,
+ struct task_struct *tsk, u64 now_ns)
+{
+ u64 delta, utime, stime;
+
+ /*
+ * Each accounting subsystem calls its functions here to
+ * accumulate its per-task stats for tsk, into the per-tgid structure
+ *
+ * per-task-foo(stats, tsk);
+ */
+ delayacct_add_tsk(stats, tsk);
+
+ /* calculate task elapsed time in nsec */
+ delta = now_ns - tsk->start_time;
+ /* Convert to micro seconds */
+ do_div(delta, NSEC_PER_USEC);
+ stats->ac_etime += delta;
+
+ task_cputime(tsk, &utime, &stime);
+ stats->ac_utime += div_u64(utime, NSEC_PER_USEC);
+ stats->ac_stime += div_u64(stime, NSEC_PER_USEC);
+
+ stats->nvcsw += tsk->nvcsw;
+ stats->nivcsw += tsk->nivcsw;
+}
+
static int fill_stats_for_tgid(pid_t tgid, struct taskstats *stats)
{
struct task_struct *tsk, *first;
unsigned long flags;
int rc = -ESRCH;
- u64 delta, utime, stime;
- u64 start_time;
+ u64 now_ns;
/*
* Add additional stats from live tasks except zombie thread group
@@ -233,30 +259,12 @@ static int fill_stats_for_tgid(pid_t tgid, struct taskstats *stats)
else
memset(stats, 0, sizeof(*stats));
- start_time = ktime_get_ns();
+ now_ns = ktime_get_ns();
for_each_thread(first, tsk) {
if (tsk->exit_state)
continue;
- /*
- * Accounting subsystem can call its functions here to
- * fill in relevant parts of struct taskstsats as follows
- *
- * per-task-foo(stats, tsk);
- */
- delayacct_add_tsk(stats, tsk);
-
- /* calculate task elapsed time in nsec */
- delta = start_time - tsk->start_time;
- /* Convert to micro seconds */
- do_div(delta, NSEC_PER_USEC);
- stats->ac_etime += delta;
- task_cputime(tsk, &utime, &stime);
- stats->ac_utime += div_u64(utime, NSEC_PER_USEC);
- stats->ac_stime += div_u64(stime, NSEC_PER_USEC);
-
- stats->nvcsw += tsk->nvcsw;
- stats->nivcsw += tsk->nivcsw;
+ tgid_stats_add_task(stats, tsk, now_ns);
}
unlock_task_sighand(first, &flags);
@@ -275,18 +283,14 @@ static int fill_stats_for_tgid(pid_t tgid, struct taskstats *stats)
static void fill_tgid_exit(struct task_struct *tsk)
{
unsigned long flags;
+ u64 now_ns;
spin_lock_irqsave(&tsk->sighand->siglock, flags);
if (!tsk->signal->stats)
goto ret;
- /*
- * Each accounting subsystem calls its functions here to
- * accumalate its per-task stats for tsk, into the per-tgid structure
- *
- * per-task-foo(tsk->signal->stats, tsk);
- */
- delayacct_add_tsk(tsk->signal->stats, tsk);
+ now_ns = ktime_get_ns();
+ tgid_stats_add_task(tsk->signal->stats, tsk, now_ns);
ret:
spin_unlock_irqrestore(&tsk->sighand->siglock, flags);
return;
diff --git a/kernel/time/jiffies.c b/kernel/time/jiffies.c
index bc4db9e5ab70..e34a0623e7ad 100644
--- a/kernel/time/jiffies.c
+++ b/kernel/time/jiffies.c
@@ -61,27 +61,24 @@ EXPORT_SYMBOL(get_jiffies_64);
EXPORT_SYMBOL(jiffies);
-static int __init init_jiffies_clocksource(void)
-{
- return __clocksource_register(&clocksource_jiffies);
-}
-
-core_initcall(init_jiffies_clocksource);
+static bool cs_jiffies_registered __initdata;
struct clocksource * __init __weak clocksource_default_clock(void)
{
+ if (!cs_jiffies_registered) {
+ __clocksource_register(&clocksource_jiffies);
+ cs_jiffies_registered = true;
+ }
return &clocksource_jiffies;
}
static struct clocksource refined_jiffies;
-int register_refined_jiffies(long cycles_per_second)
+void __init register_refined_jiffies(long cycles_per_second)
{
u64 nsec_per_tick, shift_hz;
long cycles_per_tick;
-
-
refined_jiffies = clocksource_jiffies;
refined_jiffies.name = "refined-jiffies";
refined_jiffies.rating++;
@@ -100,5 +97,4 @@ int register_refined_jiffies(long cycles_per_second)
refined_jiffies.mult = ((u32)nsec_per_tick) << JIFFIES_SHIFT;
__clocksource_register(&refined_jiffies);
- return 0;
}
diff --git a/kernel/time/posix-cpu-timers.c b/kernel/time/posix-cpu-timers.c
index fdff4f1d5c04..635fac5f5fbd 100644
--- a/kernel/time/posix-cpu-timers.c
+++ b/kernel/time/posix-cpu-timers.c
@@ -41,7 +41,7 @@ void posix_cputimers_group_init(struct posix_cputimers *pct, u64 cpu_limit)
*/
int update_rlimit_cpu(struct task_struct *task, unsigned long rlim_new)
{
- u64 nsecs = rlim_new * NSEC_PER_SEC;
+ u64 nsecs = (u64)rlim_new * NSEC_PER_SEC;
unsigned long irq_fl;
if (!lock_task_sighand(task, &irq_fl))
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index f203f000da1a..88fcd439e93d 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -774,15 +774,16 @@ static u64 get_cpu_sleep_time_us(struct tick_sched *ts, ktime_t *sleeptime,
*last_update_time = ktime_to_us(now);
do {
+ ktime_t delta = 0;
+
seq = read_seqcount_begin(&ts->idle_sleeptime_seq);
if (tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE) && compute_delta) {
- ktime_t delta = ktime_sub(now, ts->idle_entrytime);
-
- idle = ktime_add(*sleeptime, delta);
- } else {
- idle = *sleeptime;
+ if (now > ts->idle_entrytime)
+ delta = ktime_sub(now, ts->idle_entrytime);
}
+
+ idle = ktime_add(*sleeptime, delta);
} while (read_seqcount_retry(&ts->idle_sleeptime_seq, seq));
return ktime_to_us(idle);
diff --git a/kernel/time/time.c b/kernel/time/time.c
index a6261fadb92b..bfb0c1636290 100644
--- a/kernel/time/time.c
+++ b/kernel/time/time.c
@@ -251,7 +251,7 @@ COMPAT_SYSCALL_DEFINE2(settimeofday, struct old_timeval32 __user *, tv,
get_user(new_ts.tv_nsec, &tv->tv_usec))
return -EFAULT;
- if (new_ts.tv_nsec > USEC_PER_SEC || new_ts.tv_nsec < 0)
+ if (new_ts.tv_nsec >= USEC_PER_SEC || new_ts.tv_nsec < 0)
return -EINVAL;
new_ts.tv_nsec *= NSEC_PER_USEC;
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index e3896b2be453..b99171e71e0a 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -1685,13 +1685,14 @@ void __init timekeeping_init(void)
*/
wall_to_mono = timespec64_sub(boot_offset, wall_time);
+ clock = clocksource_default_clock();
+ if (clock->enable)
+ clock->enable(clock);
+
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&tk_core.seq);
ntp_init();
- clock = clocksource_default_clock();
- if (clock->enable)
- clock->enable(clock);
tk_setup_internals(tk, clock);
tk_set_xtime(tk, &wall_time);
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 41c874fbd6fa..e3038cc26f67 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -2559,9 +2559,12 @@ static int copy_user_syms(struct user_syms *us, unsigned long __user *usyms, u32
int err = -ENOMEM;
unsigned int i;
+ if (!access_ok(usyms, cnt * sizeof(*usyms)))
+ return -EFAULT;
+
syms = kvmalloc_array(cnt, sizeof(*syms), GFP_KERNEL);
if (!syms)
- goto error;
+ return -ENOMEM;
buf = kvmalloc_array(cnt, KSYM_NAME_LEN, GFP_KERNEL);
if (!buf)
@@ -2586,10 +2589,8 @@ static int copy_user_syms(struct user_syms *us, unsigned long __user *usyms, u32
return 0;
error:
- if (err) {
- kvfree(syms);
- kvfree(buf);
- }
+ kvfree(syms);
+ kvfree(buf);
return err;
}
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 4deeb613c556..fcd2363d1296 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -277,7 +277,8 @@ unsigned ring_buffer_event_length(struct ring_buffer_event *event)
if (event->type_len > RINGBUF_TYPE_DATA_TYPE_LEN_MAX)
return length;
length -= RB_EVNT_HDR_SIZE;
- if (length > RB_MAX_SMALL_DATA + sizeof(event->array[0]))
+ if (length > RB_MAX_SMALL_DATA + sizeof(event->array[0]) ||
+ RB_FORCE_8BYTE_ALIGNMENT)
length -= sizeof(event->array[0]);
return length;
}
diff --git a/kernel/trace/trace_eprobe.c b/kernel/trace/trace_eprobe.c
index be8be0c1aaf0..216e2390a23d 100644
--- a/kernel/trace/trace_eprobe.c
+++ b/kernel/trace/trace_eprobe.c
@@ -314,7 +314,7 @@ get_event_field(struct fetch_insn *code, void *rec)
val = (unsigned long)addr;
break;
case FILTER_PTR_STRING:
- val = (unsigned long)(*(char *)addr);
+ val = *(unsigned long *)addr;
break;
default:
WARN_ON_ONCE(1);
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index 2a1e3537332d..145070bbcda7 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <linux/kallsyms.h>
#include <linux/security.h>
+#include <linux/seq_buf.h>
#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/stacktrace.h>
@@ -2936,13 +2937,22 @@ find_synthetic_field_var(struct hist_trigger_data *target_hist_data,
{
struct hist_field *event_var;
char *synthetic_name;
+ struct seq_buf s;
synthetic_name = kzalloc(MAX_FILTER_STR_VAL, GFP_KERNEL);
if (!synthetic_name)
return ERR_PTR(-ENOMEM);
- strcpy(synthetic_name, "synthetic_");
- strcat(synthetic_name, field_name);
+ seq_buf_init(&s, synthetic_name, MAX_FILTER_STR_VAL);
+ seq_buf_printf(&s, "synthetic_%s", field_name);
+
+ /* Terminate synthetic_name with a NUL. */
+ seq_buf_str(&s);
+
+ if (seq_buf_has_overflowed(&s)) {
+ kfree(synthetic_name);
+ return ERR_PTR(-E2BIG);
+ }
event_var = find_event_var(target_hist_data, system, event_name, synthetic_name);
@@ -2988,6 +2998,7 @@ create_field_var_hist(struct hist_trigger_data *target_hist_data,
struct hist_field *key_field;
struct hist_field *event_var;
char *saved_filter;
+ struct seq_buf s;
char *cmd;
int ret;
@@ -3032,28 +3043,34 @@ create_field_var_hist(struct hist_trigger_data *target_hist_data,
return ERR_PTR(-ENOMEM);
}
+ seq_buf_init(&s, cmd, MAX_FILTER_STR_VAL);
+
/* Use the same keys as the compatible histogram */
- strcat(cmd, "keys=");
+ seq_buf_puts(&s, "keys=");
for_each_hist_key_field(i, hist_data) {
key_field = hist_data->fields[i];
if (!first)
- strcat(cmd, ",");
- strcat(cmd, key_field->field->name);
+ seq_buf_putc(&s, ',');
+ seq_buf_puts(&s, key_field->field->name);
first = false;
}
/* Create the synthetic field variable specification */
- strcat(cmd, ":synthetic_");
- strcat(cmd, field_name);
- strcat(cmd, "=");
- strcat(cmd, field_name);
+ seq_buf_printf(&s, ":synthetic_%s=%s", field_name, field_name);
/* Use the same filter as the compatible histogram */
saved_filter = find_trigger_filter(hist_data, file);
- if (saved_filter) {
- strcat(cmd, " if ");
- strcat(cmd, saved_filter);
+ if (saved_filter)
+ seq_buf_printf(&s, " if %s", saved_filter);
+
+ /* Terminate cmd with a NUL. */
+ seq_buf_str(&s);
+
+ if (seq_buf_has_overflowed(&s)) {
+ kfree(cmd);
+ kfree(var_hist);
+ return ERR_PTR(-E2BIG);
}
var_hist->cmd = kstrdup(cmd, GFP_KERNEL);
diff --git a/kernel/trace/trace_events_user.c b/kernel/trace/trace_events_user.c
index 42b0d998d103..888c8f9155eb 100644
--- a/kernel/trace/trace_events_user.c
+++ b/kernel/trace/trace_events_user.c
@@ -109,6 +109,9 @@ struct user_event_enabler {
/* Track enable bit, flags, etc. Aligned for bitops. */
unsigned long values;
+
+ /* Defer the event put and enabler free past an RCU grace period. */
+ struct rcu_work put_rwork;
};
/* Bits 0-5 are for the bit to update upon enable/disable (0-63 allowed) */
@@ -396,17 +399,39 @@ static struct user_event_group *user_event_group_create(void)
return NULL;
};
-static void user_event_enabler_destroy(struct user_event_enabler *enabler,
- bool locked)
+static void delayed_user_event_enabler_put(struct work_struct *work)
{
- list_del_rcu(&enabler->mm_enablers_link);
+ struct user_event_enabler *enabler = container_of(to_rcu_work(work),
+ struct user_event_enabler, put_rwork);
/* No longer tracking the event via the enabler */
- user_event_put(enabler->event, locked);
+ user_event_put(enabler->event, false);
+ /* Run from queue_rcu_work(), the RCU grace period has elapsed */
kfree(enabler);
}
+static void user_event_enabler_destroy(struct user_event_enabler *enabler)
+{
+ list_del_rcu(&enabler->mm_enablers_link);
+
+ /*
+ * The enabler is removed from an RCU-traversed list
+ * (user_event_mm_dup() walks mm->enablers under rcu_read_lock() only),
+ * and readers there dereference enabler->event and take a new ref on
+ * it. Both the put of that event reference and the free of the enabler
+ * therefore have to wait for a grace period so no reader can be looking
+ * at the enabler or racing the last put of its event.
+ *
+ * The put itself must not run in RCU context: when it drops the last
+ * reference user_event_put() takes event_mutex, which cannot be taken
+ * from a softirq/RCU callback. Defer both to a work item scheduled
+ * after a grace period via queue_rcu_work().
+ */
+ INIT_RCU_WORK(&enabler->put_rwork, delayed_user_event_enabler_put);
+ queue_rcu_work(system_percpu_wq, &enabler->put_rwork);
+}
+
static int user_event_mm_fault_in(struct user_event_mm *mm, unsigned long uaddr,
int attempt)
{
@@ -464,7 +489,7 @@ static void user_event_enabler_fault_fixup(struct work_struct *work)
/* User asked for enabler to be removed during fault */
if (test_bit(ENABLE_VAL_FREEING_BIT, ENABLE_BITOPS(enabler))) {
- user_event_enabler_destroy(enabler, true);
+ user_event_enabler_destroy(enabler);
goto out;
}
@@ -764,7 +789,7 @@ static void user_event_mm_destroy(struct user_event_mm *mm)
struct user_event_enabler *enabler, *next;
list_for_each_entry_safe(enabler, next, &mm->enablers, mm_enablers_link)
- user_event_enabler_destroy(enabler, false);
+ user_event_enabler_destroy(enabler);
mmdrop(mm->mm);
kfree(mm);
@@ -2655,7 +2680,7 @@ static long user_events_ioctl_unreg(unsigned long uarg)
flags |= enabler->values & ENABLE_VAL_COMPAT_MASK;
if (!test_bit(ENABLE_VAL_FAULTING_BIT, ENABLE_BITOPS(enabler)))
- user_event_enabler_destroy(enabler, true);
+ user_event_enabler_destroy(enabler);
/* Removed at least one */
ret = 0;
diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
index 7d96be13367f..4daedf8debdc 100644
--- a/kernel/trace/trace_osnoise.c
+++ b/kernel/trace/trace_osnoise.c
@@ -159,7 +159,9 @@ static void osnoise_unregister_instance(struct trace_array *tr)
if (!found)
return;
- kvfree_rcu_mightsleep(inst);
+ /* Do a full sync to ensure that tr remains valid, not just inst */
+ synchronize_rcu();
+ kvfree(inst);
}
/*
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index a433912b202b..b96237fc4cb5 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -1105,6 +1105,7 @@ parse_probe_arg(char *arg, const struct fetch_type *type,
code->op = FETCH_OP_FOFFS;
code->immediate = (unsigned long)offset; // imm64?
+ offset = 0;
} else {
/* uprobes don't support symbols */
if (!(ctx->flags & TPARG_FL_KERNEL)) {
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index 3d52d3c4d495..8f6d5fac0992 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -499,7 +499,7 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(NO_RETVAL, "This function returns 'void' type"), \
C(BAD_STACK_NUM, "Invalid stack number"), \
C(BAD_ARG_NUM, "Invalid argument number"), \
- C(BAD_VAR, "Invalid $-valiable specified"), \
+ C(BAD_VAR, "Invalid $-variable specified"), \
C(BAD_REG_NAME, "Invalid register name"), \
C(BAD_MEM_ADDR, "Invalid memory address"), \
C(BAD_IMM, "Invalid immediate value"), \
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 2fdf9174ccec..a91215dc0db4 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -6172,7 +6172,7 @@ EXPORT_SYMBOL_GPL(set_worker_desc);
*/
void print_worker_info(const char *log_lvl, struct task_struct *task)
{
- work_func_t *fn = NULL;
+ work_func_t fn = NULL;
char name[WQ_NAME_LEN] = { };
char desc[WORKER_DESC_LEN] = { };
struct pool_workqueue *pwq = NULL;
diff --git a/lib/test_meminit.c b/lib/test_meminit.c
index 6298f66c964b..d028a6552cd6 100644
--- a/lib/test_meminit.c
+++ b/lib/test_meminit.c
@@ -387,7 +387,7 @@ static int __init test_kmemcache(int *total_failures)
ctor = flags & 1;
rcu = flags & 2;
zero = flags & 4;
- if (ctor & zero)
+ if (ctor && zero)
continue;
num_tests += do_kmem_cache_size(size, ctor, rcu, zero,
&failures);
diff --git a/mm/compaction.c b/mm/compaction.c
index 66032064387e..6fcfbdb121aa 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1894,15 +1894,14 @@ static void compaction_free(struct folio *dst, unsigned long data)
int order = folio_order(dst);
struct page *page = &dst->page;
- if (folio_put_testzero(dst)) {
- free_pages_prepare(page, order);
+ if (folio_put_testzero(dst) && free_pages_prepare(page, order)) {
list_add(&dst->lru, &cc->freepages[order]);
cc->nr_freepages += 1 << order;
}
cc->nr_migratepages += 1 << order;
/*
- * someone else has referenced the page, we cannot take it back to our
- * free list.
+ * someone else has referenced the page or free_pages_prepare() fails,
+ * we cannot take it back to our free list.
*/
}
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 2a6da4981652..7911599c37f2 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -938,10 +938,36 @@ static int damon_commit_target(
return 0;
}
+/*
+ * damon_revert_target_commits() - revert unsuccessful target commits.
+ * @dst: Commit destination context
+ * @failed: Commit failed destination target
+ * @src: Commit source context
+ *
+ * Revert target states that changed by damon_commit_target(), and cannot be
+ * cleaned up by the destination context's ops.cleanup_target().
+ */
+static void damon_revert_target_commits(struct damon_ctx *dst,
+ struct damon_target *failed, struct damon_ctx *src)
+{
+ struct damon_target *target;
+
+ if (!damon_target_has_pid(src))
+ return;
+ if (damon_target_has_pid(dst))
+ return;
+ damon_for_each_target(target, dst) {
+ if (target == failed)
+ return;
+ put_pid(target->pid);
+ }
+}
+
static int damon_commit_targets(
struct damon_ctx *dst, struct damon_ctx *src)
{
struct damon_target *dst_target, *next, *src_target, *new_target;
+ struct damon_target *failed;
int i = 0, j = 0, err;
damon_for_each_target_safe(dst_target, next, dst) {
@@ -950,8 +976,10 @@ static int damon_commit_targets(
err = damon_commit_target(
dst_target, damon_target_has_pid(dst),
src_target, damon_target_has_pid(src));
- if (err)
- return err;
+ if (err) {
+ failed = dst_target;
+ goto out;
+ }
} else {
if (damon_target_has_pid(dst))
put_pid(dst_target->pid);
@@ -959,21 +987,28 @@ static int damon_commit_targets(
}
}
+ failed = NULL;
damon_for_each_target_safe(src_target, next, src) {
if (j++ < i)
continue;
new_target = damon_new_target();
- if (!new_target)
- return -ENOMEM;
+ if (!new_target) {
+ err = -ENOMEM;
+ goto out;
+ }
err = damon_commit_target(new_target, false,
src_target, damon_target_has_pid(src));
if (err) {
damon_destroy_target(new_target);
- return err;
+ goto out;
}
damon_add_target(dst, new_target);
}
return 0;
+
+out:
+ damon_revert_target_commits(dst, failed, src);
+ return err;
}
/**
@@ -1007,8 +1042,10 @@ int damon_commit_ctx(struct damon_ctx *dst, struct damon_ctx *src)
* committing require putting pids).
*/
err = damon_set_attrs(dst, &src->attrs);
- if (err)
+ if (err) {
+ damon_revert_target_commits(dst, NULL, src);
return err;
+ }
dst->ops = src->ops;
return 0;
@@ -1435,7 +1472,7 @@ static void damos_apply_scheme(struct damon_ctx *c, struct damon_target *t,
quota->charged_sz += sz;
if (quota->esz && quota->charged_sz >= quota->esz) {
quota->charge_target_from = t;
- quota->charge_addr_from = r->ar.end + 1;
+ quota->charge_addr_from = r->ar.end;
}
}
if (s->action != DAMOS_STAT)
diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index a84a633e7924..998c7b5b4325 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -1393,22 +1393,19 @@ static int damon_sysfs_access_pattern_add_dirs(
err = damon_sysfs_access_pattern_add_range_dir(access_pattern,
&access_pattern->sz, "sz");
if (err)
- goto put_sz_out;
+ return err;
err = damon_sysfs_access_pattern_add_range_dir(access_pattern,
&access_pattern->nr_accesses, "nr_accesses");
if (err)
- goto put_nr_accesses_sz_out;
+ goto put_sz_out;
err = damon_sysfs_access_pattern_add_range_dir(access_pattern,
&access_pattern->age, "age");
if (err)
- goto put_age_nr_accesses_sz_out;
+ goto put_nr_accesses_sz_out;
return 0;
-put_age_nr_accesses_sz_out:
- kobject_put(&access_pattern->age->kobj);
- access_pattern->age = NULL;
put_nr_accesses_sz_out:
kobject_put(&access_pattern->nr_accesses->kobj);
access_pattern->nr_accesses = NULL;
@@ -1627,12 +1624,12 @@ static int damon_sysfs_scheme_add_dirs(struct damon_sysfs_scheme *scheme)
goto put_filters_watermarks_quotas_access_pattern_out;
err = damon_sysfs_scheme_set_tried_regions(scheme);
if (err)
- goto put_tried_regions_out;
+ goto put_stats_out;
return 0;
-put_tried_regions_out:
- kobject_put(&scheme->tried_regions->kobj);
- scheme->tried_regions = NULL;
+put_stats_out:
+ kobject_put(&scheme->stats->kobj);
+ scheme->stats = NULL;
put_filters_watermarks_quotas_access_pattern_out:
kobject_put(&scheme->filters->kobj);
scheme->filters = NULL;
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 16d58151a531..3e2140891a9d 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1456,7 +1456,7 @@ static int create_altmaps_and_memory_blocks(int nid, struct memory_group *group,
ret = create_memory_block_devices(cur_start, memblock_size,
params.altmap, group);
if (ret) {
- arch_remove_memory(cur_start, memblock_size, NULL);
+ arch_remove_memory(cur_start, memblock_size, params.altmap);
kfree(params.altmap);
goto out;
}
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 624c1f90ce05..1450b2bb98ee 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -1031,10 +1031,17 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn,
* of how the sparse_vmemmap internals handle compound pages in the lack
* of an altmap. See vmemmap_populate_compound_pages().
*/
-static inline unsigned long compound_nr_pages(struct vmem_altmap *altmap,
+static inline unsigned long compound_nr_pages(unsigned long pfn,
+ struct vmem_altmap *altmap,
struct dev_pagemap *pgmap)
{
- if (!vmemmap_can_optimize(altmap, pgmap))
+ /*
+ * If DAX memory is hot-plugged into an unoccupied subsection
+ * of an early section, the unoptimized boot memmap is reused.
+ * See section_activate().
+ */
+ if (early_section(__pfn_to_section(pfn)) ||
+ !vmemmap_can_optimize(altmap, pgmap))
return pgmap_vmemmap_nr(pgmap);
return VMEMMAP_RESERVE_NR * (PAGE_SIZE / sizeof(struct page));
@@ -1102,7 +1109,7 @@ void __ref memmap_init_zone_device(struct zone *zone,
continue;
memmap_init_compound(page, pfn, zone_idx, nid, pgmap,
- compound_nr_pages(altmap, pgmap));
+ compound_nr_pages(pfn, altmap, pgmap));
}
pr_debug("%s initialised %lu pages in %ums\n", __func__,
diff --git a/mm/numa_emulation.c b/mm/numa_emulation.c
index 9d55679d99ce..e22eca812b62 100644
--- a/mm/numa_emulation.c
+++ b/mm/numa_emulation.c
@@ -214,7 +214,7 @@ static u64 uniform_size(u64 max_addr, u64 base, u64 hole, int nr_nodes)
* Sets up fake nodes of `size' interleaved over physical nodes ranging from
* `addr' to `max_addr'.
*
- * Returns zero on success or negative on error.
+ * Returns node ID of the next node on success or negative error code.
*/
static int __init split_nodes_size_interleave_uniform(struct numa_meminfo *ei,
struct numa_meminfo *pi,
@@ -398,7 +398,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
*/
if (strchr(emu_cmdline, 'U')) {
unsigned long n;
- int nid = 0;
+ int nid = 0, nr_created;
n = simple_strtoul(emu_cmdline, &emu_cmdline, 0);
ret = -1;
@@ -416,9 +416,18 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
n, &pi.blk[0], nid);
if (ret < 0)
break;
- if (ret < n) {
+
+ /*
+ * If no memory was found for this physical node,
+ * skip the under-allocation check.
+ */
+ if (ret == nid)
+ continue;
+
+ nr_created = ret - nid;
+ if (nr_created < n) {
pr_info("%s: phys: %d only got %d of %ld nodes, failing\n",
- __func__, i, ret, n);
+ __func__, i, nr_created, n);
ret = -1;
break;
}
diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c
index da1278f0563b..d6f6ae0ef2b3 100644
--- a/mm/swap_cgroup.c
+++ b/mm/swap_cgroup.c
@@ -161,8 +161,14 @@ unsigned short swap_cgroup_record(swp_entry_t ent, unsigned short id,
*/
unsigned short lookup_swap_cgroup_id(swp_entry_t ent)
{
+ struct swap_cgroup_ctrl *ctrl;
+
if (mem_cgroup_disabled())
return 0;
+
+ ctrl = &swap_cgroup_ctrl[swp_type(ent)];
+ if (unlikely(!ctrl->map))
+ return 0;
return lookup_swap_cgroup(ent, NULL)->id;
}
diff --git a/net/9p/client.c b/net/9p/client.c
index 2f6baeb47500..54a05d9082c5 100644
--- a/net/9p/client.c
+++ b/net/9p/client.c
@@ -723,6 +723,8 @@ p9_client_rpc(struct p9_client *c, int8_t type, const char *fmt, ...)
if (err == -ERESTARTSYS && c->status == Connected &&
type == P9_TFLUSH) {
+ if (fatal_signal_pending(current))
+ goto recalc_sigpending;
sigpending = 1;
clear_thread_flag(TIF_SIGPENDING);
goto again;
diff --git a/net/9p/trans_rdma.c b/net/9p/trans_rdma.c
index b84748baf9cb..d9f53f1da2c2 100644
--- a/net/9p/trans_rdma.c
+++ b/net/9p/trans_rdma.c
@@ -240,25 +240,36 @@ p9_cm_event_handler(struct rdma_cm_id *id, struct rdma_cm_event *event)
{
struct p9_client *c = id->context;
struct p9_trans_rdma *rdma = c->trans;
+ unsigned long flags;
+
switch (event->event) {
case RDMA_CM_EVENT_ADDR_RESOLVED:
+ spin_lock_irqsave(&rdma->req_lock, flags);
BUG_ON(rdma->state != P9_RDMA_INIT);
rdma->state = P9_RDMA_ADDR_RESOLVED;
+ spin_unlock_irqrestore(&rdma->req_lock, flags);
break;
case RDMA_CM_EVENT_ROUTE_RESOLVED:
+ spin_lock_irqsave(&rdma->req_lock, flags);
BUG_ON(rdma->state != P9_RDMA_ADDR_RESOLVED);
rdma->state = P9_RDMA_ROUTE_RESOLVED;
+ spin_unlock_irqrestore(&rdma->req_lock, flags);
break;
case RDMA_CM_EVENT_ESTABLISHED:
+ spin_lock_irqsave(&rdma->req_lock, flags);
BUG_ON(rdma->state != P9_RDMA_ROUTE_RESOLVED);
rdma->state = P9_RDMA_CONNECTED;
+ spin_unlock_irqrestore(&rdma->req_lock, flags);
break;
case RDMA_CM_EVENT_DISCONNECTED:
- if (rdma)
+ if (rdma) {
+ spin_lock_irqsave(&rdma->req_lock, flags);
rdma->state = P9_RDMA_CLOSED;
+ spin_unlock_irqrestore(&rdma->req_lock, flags);
+ }
c->status = Disconnected;
break;
@@ -296,6 +307,7 @@ recv_done(struct ib_cq *cq, struct ib_wc *wc)
struct p9_req_t *req;
int err = 0;
int16_t tag;
+ unsigned long flags;
req = NULL;
ib_dma_unmap_single(rdma->cm_id->device, c->busa, client->msize,
@@ -332,7 +344,10 @@ recv_done(struct ib_cq *cq, struct ib_wc *wc)
err_out:
p9_debug(P9_DEBUG_ERROR, "req %p err %d status %d\n",
req, err, wc->status);
- rdma->state = P9_RDMA_FLUSHING;
+ spin_lock_irqsave(&rdma->req_lock, flags);
+ if (rdma->state < P9_RDMA_FLUSHING)
+ rdma->state = P9_RDMA_FLUSHING;
+ spin_unlock_irqrestore(&rdma->req_lock, flags);
client->status = Disconnected;
goto out;
}
diff --git a/net/Kconfig.debug b/net/Kconfig.debug
index 5e3fffe707dd..277fab8c4d77 100644
--- a/net/Kconfig.debug
+++ b/net/Kconfig.debug
@@ -24,3 +24,18 @@ config DEBUG_NET
help
Enable extra sanity checks in networking.
This is mostly used by fuzzers, but is safe to select.
+
+config DEBUG_NET_SMALL_RTNL
+ bool "Add extra per-netns mutex inside RTNL"
+ depends on DEBUG_KERNEL && NET && LOCK_DEBUGGING_SUPPORT
+ select PROVE_LOCKING
+ default n
+ help
+ rtnl_lock() is being replaced with rtnl_net_lock() that
+ acquires the global RTNL and a small per-netns RTNL mutex.
+
+ During the conversion, rtnl_net_lock() just adds an extra
+ mutex in every RTNL scope and slows down the operations.
+
+ Once the conversion completes, rtnl_lock() will be removed
+ and rtnetlink will gain per-netns scalability.
diff --git a/net/atm/common.c b/net/atm/common.c
index 881c7f259dbd..196c36269b34 100644
--- a/net/atm/common.c
+++ b/net/atm/common.c
@@ -720,6 +720,8 @@ static int atm_change_qos(struct atm_vcc *vcc, struct atm_qos *qos)
static int check_tp(const struct atm_trafprm *tp)
{
/* @@@ Should be merged with adjust_tp */
+ if (tp->traffic_class > ATM_ANYCLASS)
+ return -EINVAL;
if (!tp->traffic_class || tp->traffic_class == ATM_ANYCLASS)
return 0;
if (tp->traffic_class != ATM_UBR && !tp->min_pcr && !tp->pcr &&
diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index bd5477263ecd..ceebfa4f0107 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -546,7 +546,7 @@ static bool batadv_is_orig_node_eligible(struct batadv_dat_candidate *res,
* the one with the lowest address
*/
if (tmp_max == max && max_orig_node &&
- batadv_compare_eth(candidate->orig, max_orig_node->orig))
+ memcmp(candidate->orig, max_orig_node->orig, ETH_ALEN) >= 0)
goto out;
ret = true;
@@ -1076,6 +1076,9 @@ static u16 batadv_arp_get_type(struct batadv_priv *bat_priv,
* @skb: the buffer containing the packet to extract the VID from
* @hdr_size: the size of the batman-adv header encapsulating the packet
*
+ * The caller must ensure that at least @hdr_size + ETH_HLEN bytes are
+ * accessible after skb->data.
+ *
* Return: If the packet embedded in the skb is vlan tagged this function
* returns the VID with the BATADV_VLAN_HAS_TAG flag. Otherwise BATADV_NO_FLAGS
* is returned.
@@ -1158,6 +1161,10 @@ bool batadv_dat_snoop_outgoing_arp_request(struct batadv_priv *bat_priv,
if (!atomic_read(&bat_priv->distributed_arp_table))
goto out;
+ /* first, find out the vid. */
+ if (!pskb_may_pull(skb, hdr_size + ETH_HLEN))
+ goto out;
+
vid = batadv_dat_get_vid(skb, &hdr_size);
type = batadv_arp_get_type(bat_priv, skb, hdr_size);
@@ -1253,6 +1260,10 @@ bool batadv_dat_snoop_incoming_arp_request(struct batadv_priv *bat_priv,
if (!atomic_read(&bat_priv->distributed_arp_table))
goto out;
+ /* first, find out the vid. */
+ if (!pskb_may_pull(skb, hdr_size + ETH_HLEN))
+ goto out;
+
vid = batadv_dat_get_vid(skb, &hdr_size);
type = batadv_arp_get_type(bat_priv, skb, hdr_size);
@@ -1315,6 +1326,10 @@ void batadv_dat_snoop_outgoing_arp_reply(struct batadv_priv *bat_priv,
if (!atomic_read(&bat_priv->distributed_arp_table))
return;
+ /* first, find out the vid. */
+ if (!pskb_may_pull(skb, hdr_size + ETH_HLEN))
+ return;
+
vid = batadv_dat_get_vid(skb, &hdr_size);
type = batadv_arp_get_type(bat_priv, skb, hdr_size);
@@ -1363,6 +1378,10 @@ bool batadv_dat_snoop_incoming_arp_reply(struct batadv_priv *bat_priv,
if (!atomic_read(&bat_priv->distributed_arp_table))
goto out;
+ /* first, find out the vid. */
+ if (!pskb_may_pull(skb, hdr_size + ETH_HLEN))
+ goto out;
+
vid = batadv_dat_get_vid(skb, &hdr_size);
type = batadv_arp_get_type(bat_priv, skb, hdr_size);
@@ -1757,6 +1776,7 @@ void batadv_dat_snoop_incoming_dhcp_ack(struct batadv_priv *bat_priv,
struct ethhdr *ethhdr;
__be32 ip_src, yiaddr;
unsigned short vid;
+ int hdr_size_tmp;
__be16 proto;
u8 *hw_src;
@@ -1773,8 +1793,10 @@ void batadv_dat_snoop_incoming_dhcp_ack(struct batadv_priv *bat_priv,
if (!batadv_dat_check_dhcp_ack(skb, proto, &ip_src, chaddr, &yiaddr))
return;
+ hdr_size_tmp = hdr_size;
+ vid = batadv_dat_get_vid(skb, &hdr_size_tmp);
+ ethhdr = (struct ethhdr *)(skb->data + hdr_size);
hw_src = ethhdr->h_source;
- vid = batadv_dat_get_vid(skb, &hdr_size);
batadv_dat_entry_add(bat_priv, yiaddr, chaddr, vid);
batadv_dat_entry_add(bat_priv, ip_src, hw_src, vid);
@@ -1814,6 +1836,10 @@ bool batadv_dat_drop_broadcast_packet(struct batadv_priv *bat_priv,
if (batadv_forw_packet_is_rebroadcast(forw_packet))
goto out;
+ /* first, find out the vid. */
+ if (!pskb_may_pull(forw_packet->skb, hdr_size + ETH_HLEN))
+ goto out;
+
vid = batadv_dat_get_vid(forw_packet->skb, &hdr_size);
type = batadv_arp_get_type(bat_priv, forw_packet->skb, hdr_size);
diff --git a/net/batman-adv/fragmentation.c b/net/batman-adv/fragmentation.c
index 375c94991213..c366d86950a0 100644
--- a/net/batman-adv/fragmentation.c
+++ b/net/batman-adv/fragmentation.c
@@ -516,8 +516,10 @@ int batadv_frag_send_packet(struct sk_buff *skb,
mtu = min_t(unsigned int, mtu, BATADV_FRAG_MAX_FRAG_SIZE);
max_fragment_size = mtu - header_size;
- if (skb->len == 0 || max_fragment_size == 0)
- return -EINVAL;
+ if (skb->len == 0 || max_fragment_size == 0) {
+ ret = -EINVAL;
+ goto free_skb;
+ }
num_fragments = (skb->len - 1) / max_fragment_size + 1;
max_fragment_size = (skb->len - 1) / num_fragments + 1;
@@ -543,7 +545,7 @@ int batadv_frag_send_packet(struct sk_buff *skb,
*/
if (skb_has_frag_list(skb) && __skb_linearize(skb)) {
ret = -ENOMEM;
- goto free_skb;
+ goto put_primary_if;
}
/* Create one header to be copied to all fragments */
diff --git a/net/batman-adv/gateway_client.c b/net/batman-adv/gateway_client.c
index 0c1485f6131d..450a48e09122 100644
--- a/net/batman-adv/gateway_client.c
+++ b/net/batman-adv/gateway_client.c
@@ -693,12 +693,13 @@ bool batadv_gw_out_of_range(struct batadv_priv *bat_priv,
struct batadv_gw_node *gw_node = NULL;
struct batadv_gw_node *curr_gw = NULL;
struct batadv_neigh_ifinfo *curr_ifinfo, *old_ifinfo;
- struct ethhdr *ethhdr = (struct ethhdr *)skb->data;
+ struct ethhdr *ethhdr;
bool out_of_range = false;
u8 curr_tq_avg;
unsigned short vid;
vid = batadv_get_vid(skb, 0);
+ ethhdr = (struct ethhdr *)skb->data;
if (is_multicast_ether_addr(ethhdr->h_dest))
goto out;
diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c
index e989a81084e9..87f798701dab 100644
--- a/net/batman-adv/main.c
+++ b/net/batman-adv/main.c
@@ -259,6 +259,7 @@ int batadv_mesh_init(struct net_device *soft_iface)
void batadv_mesh_free(struct net_device *soft_iface)
{
struct batadv_priv *bat_priv = netdev_priv(soft_iface);
+ struct batadv_softif_vlan *vlan;
atomic_set(&bat_priv->mesh_state, BATADV_MESH_DEACTIVATING);
@@ -274,6 +275,13 @@ void batadv_mesh_free(struct net_device *soft_iface)
batadv_mcast_free(bat_priv);
+ /* destroy the "untagged" VLAN */
+ vlan = batadv_softif_vlan_get(bat_priv, BATADV_NO_FLAGS);
+ if (vlan) {
+ batadv_softif_destroy_vlan(bat_priv, vlan);
+ batadv_softif_vlan_put(vlan);
+ }
+
/* Free the TT and the originator tables only after having terminated
* all the other depending components which may use these structures for
* their purposes.
@@ -376,7 +384,7 @@ void batadv_skb_set_priority(struct sk_buff *skb, int offset)
switch (ethhdr->h_proto) {
case htons(ETH_P_8021Q):
- vhdr = skb_header_pointer(skb, offset + sizeof(*vhdr),
+ vhdr = skb_header_pointer(skb, offset,
sizeof(*vhdr), &vhdr_tmp);
if (!vhdr)
return;
@@ -621,6 +629,9 @@ __be32 batadv_skb_crc32(struct sk_buff *skb, u8 *payload_ptr)
* @skb: the buffer containing the packet
* @header_len: length of the batman header preceding the ethernet header
*
+ * The caller must ensure that at least @header_len + ETH_HLEN bytes are
+ * accessible after skb->data.
+ *
* Return: VID with the BATADV_VLAN_HAS_TAG flag when the packet embedded in the
* skb is vlan tagged. Otherwise BATADV_NO_FLAGS.
*/
diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c
index fafd6ba8c056..6575e7cb2bcf 100644
--- a/net/batman-adv/multicast_forw.c
+++ b/net/batman-adv/multicast_forw.c
@@ -927,11 +927,11 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv,
{
struct batadv_tvlv_mcast_tracker *mcast_tracker;
struct batadv_neigh_node *neigh_node;
- unsigned long offset, num_dests_off;
struct sk_buff *nexthop_skb;
unsigned char *skb_net_hdr;
bool local_recv = false;
unsigned int tvlv_len;
+ unsigned long offset;
bool xmitted = false;
u8 *dest, *next_dest;
u16 num_dests;
@@ -940,9 +940,8 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv,
/* (at least) TVLV part needs to be linearized */
SKB_LINEAR_ASSERT(skb);
- /* check if num_dests is within skb length */
- num_dests_off = offsetof(struct batadv_tvlv_mcast_tracker, num_dests);
- if (num_dests_off > skb_network_header_len(skb))
+ /* check if batadv_tvlv_mcast_tracker header is within skb length */
+ if (sizeof(*mcast_tracker) > skb_network_header_len(skb))
return -EINVAL;
skb_net_hdr = skb_network_header(skb);
diff --git a/net/batman-adv/routing.c b/net/batman-adv/routing.c
index f828d8143f75..28fc95bdf25c 100644
--- a/net/batman-adv/routing.c
+++ b/net/batman-adv/routing.c
@@ -856,8 +856,8 @@ static bool batadv_check_unicast_ttvn(struct batadv_priv *bat_priv,
if (skb_cow(skb, sizeof(*unicast_packet)) < 0)
return false;
- unicast_packet = (struct batadv_unicast_packet *)skb->data;
vid = batadv_get_vid(skb, hdr_len);
+ unicast_packet = (struct batadv_unicast_packet *)skb->data;
ethhdr = (struct ethhdr *)(skb->data + hdr_len);
/* do not reroute multicast frames in a unicast header */
@@ -1036,6 +1036,7 @@ int batadv_recv_unicast_packet(struct sk_buff *skb,
hdr_size);
batadv_orig_node_put(orig_node_gw);
if (is_gw) {
+ orig_addr_gw = eth_hdr(skb)->h_source;
batadv_dbg(BATADV_DBG_BLA, bat_priv,
"%s(): Dropped unicast pkt received from another backbone gw %pM.\n",
__func__, orig_addr_gw);
diff --git a/net/batman-adv/soft-interface.c b/net/batman-adv/soft-interface.c
index f46064333f33..aaebe74c1bc0 100644
--- a/net/batman-adv/soft-interface.c
+++ b/net/batman-adv/soft-interface.c
@@ -206,6 +206,9 @@ static netdev_tx_t batadv_interface_tx(struct sk_buff *skb,
if (atomic_read(&bat_priv->mesh_state) != BATADV_MESH_ACTIVE)
goto dropped;
+ if (!pskb_may_pull(skb, ETH_HLEN))
+ goto dropped;
+
/* reset control block to avoid left overs from previous users */
memset(skb->cb, 0, sizeof(struct batadv_skb_cb));
@@ -445,6 +448,7 @@ void batadv_interface_rx(struct net_device *soft_iface,
if (!pskb_may_pull(skb, VLAN_ETH_HLEN))
goto dropped;
+ ethhdr = eth_hdr(skb);
vhdr = skb_vlan_eth_hdr(skb);
/* drop batman-in-batman packets to prevent loops */
@@ -602,8 +606,8 @@ int batadv_softif_create_vlan(struct batadv_priv *bat_priv, unsigned short vid)
* @bat_priv: the bat priv with all the soft interface information
* @vlan: the object to remove
*/
-static void batadv_softif_destroy_vlan(struct batadv_priv *bat_priv,
- struct batadv_softif_vlan *vlan)
+void batadv_softif_destroy_vlan(struct batadv_priv *bat_priv,
+ struct batadv_softif_vlan *vlan)
{
/* explicitly remove the associated TT local entry because it is marked
* with the NOPURGE flag
@@ -1099,22 +1103,13 @@ static int batadv_softif_newlink(struct net *src_net, struct net_device *dev,
static void batadv_softif_destroy_netlink(struct net_device *soft_iface,
struct list_head *head)
{
- struct batadv_priv *bat_priv = netdev_priv(soft_iface);
struct batadv_hard_iface *hard_iface;
- struct batadv_softif_vlan *vlan;
list_for_each_entry(hard_iface, &batadv_hardif_list, list) {
if (hard_iface->soft_iface == soft_iface)
batadv_hardif_disable_interface(hard_iface);
}
- /* destroy the "untagged" VLAN */
- vlan = batadv_softif_vlan_get(bat_priv, BATADV_NO_FLAGS);
- if (vlan) {
- batadv_softif_destroy_vlan(bat_priv, vlan);
- batadv_softif_vlan_put(vlan);
- }
-
unregister_netdevice_queue(soft_iface, head);
}
diff --git a/net/batman-adv/soft-interface.h b/net/batman-adv/soft-interface.h
index 9f2003f1a497..13709a5d23e3 100644
--- a/net/batman-adv/soft-interface.h
+++ b/net/batman-adv/soft-interface.h
@@ -22,6 +22,8 @@ void batadv_interface_rx(struct net_device *soft_iface,
bool batadv_softif_is_valid(const struct net_device *net_dev);
extern struct rtnl_link_ops batadv_link_ops;
int batadv_softif_create_vlan(struct batadv_priv *bat_priv, unsigned short vid);
+void batadv_softif_destroy_vlan(struct batadv_priv *bat_priv,
+ struct batadv_softif_vlan *vlan);
void batadv_softif_vlan_release(struct kref *ref);
struct batadv_softif_vlan *batadv_softif_vlan_get(struct batadv_priv *bat_priv,
unsigned short vid);
diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c
index 2a7caab5c197..7df77cca52a0 100644
--- a/net/batman-adv/translation-table.c
+++ b/net/batman-adv/translation-table.c
@@ -3032,7 +3032,7 @@ static bool batadv_send_tt_request(struct batadv_priv *bat_priv,
out:
batadv_hardif_put(primary_if);
- if (ret && tt_req_node) {
+ if (!ret && tt_req_node) {
spin_lock_bh(&bat_priv->tt.req_list_lock);
if (!hlist_unhashed(&tt_req_node->list)) {
hlist_del_init(&tt_req_node->list);
@@ -4101,7 +4101,8 @@ static int batadv_tt_tvlv_unicast_handler_v1(struct batadv_priv *bat_priv,
u16 tvlv_value_len)
{
struct batadv_tvlv_tt_data *tt_data;
- u16 tt_vlan_len, tt_num_entries;
+ u16 tt_num_entries;
+ size_t tt_vlan_len;
char tt_flag;
bool ret;
diff --git a/net/bluetooth/6lowpan.c b/net/bluetooth/6lowpan.c
index 03f0b5d27b60..89de21ec62a1 100644
--- a/net/bluetooth/6lowpan.c
+++ b/net/bluetooth/6lowpan.c
@@ -631,7 +631,7 @@ static struct l2cap_chan *chan_create(void)
if (!chan)
return NULL;
- l2cap_chan_set_defaults(chan);
+ l2cap_chan_set_defaults(chan, NULL);
chan->chan_type = L2CAP_CHAN_CONN_ORIENTED;
chan->mode = L2CAP_MODE_LE_FLOWCTL;
@@ -744,19 +744,24 @@ static inline void chan_ready_cb(struct l2cap_chan *chan)
ifup(dev->netdev);
}
-static inline struct l2cap_chan *chan_new_conn_cb(struct l2cap_chan *pchan)
+static void unregister_dev(struct lowpan_btle_dev *dev)
{
- struct l2cap_chan *chan;
-
- chan = chan_create();
- if (!chan)
- return NULL;
-
- chan->ops = pchan->ops;
-
- BT_DBG("chan %p pchan %p", chan, pchan);
-
- return chan;
+ struct hci_dev *hdev = READ_ONCE(dev->hdev);
+
+ /* If netdev holds last reference to hci_dev (its parent device), this
+ * leads to theoretical cyclic locking on lowpan_unregister_netdev:
+ *
+ * rtnl_lock -> put_device(parent) -> hci_release_dev ->
+ * destroy_workqueue -> hci_rx_work -> l2cap_recv_acldata ->
+ * chan_ready_cb -> ifup -> rtnl_lock
+ *
+ * However, hci_rx_work is disabled in hci_unregister_dev, so this
+ * should not occur. Make lockdep happy by postponing hdev release after
+ * netdev put.
+ */
+ hci_dev_hold(hdev);
+ lowpan_unregister_netdev(dev->netdev);
+ hci_dev_put(hdev);
}
static void delete_netdev(struct work_struct *work)
@@ -765,7 +770,7 @@ static void delete_netdev(struct work_struct *work)
struct lowpan_btle_dev,
delete_netdev);
- lowpan_unregister_netdev(entry->netdev);
+ unregister_dev(entry);
/* The entry pointer is deleted by the netdev destructor. */
}
@@ -776,20 +781,10 @@ static void chan_close_cb(struct l2cap_chan *chan)
struct lowpan_btle_dev *dev = NULL;
struct lowpan_peer *peer;
int err = -ENOENT;
- bool last = false, remove = true;
+ bool last = false;
BT_DBG("chan %p conn %p", chan, chan->conn);
- if (chan->conn && chan->conn->hcon) {
- if (!is_bt_6lowpan(chan->conn->hcon))
- return;
-
- /* If conn is set, then the netdev is also there and we should
- * not remove it.
- */
- remove = false;
- }
-
spin_lock(&devices_lock);
list_for_each_entry_rcu(entry, &bt_6lowpan_devices, list) {
@@ -816,10 +811,8 @@ static void chan_close_cb(struct l2cap_chan *chan)
ifdown(dev->netdev);
- if (remove) {
- INIT_WORK(&entry->delete_netdev, delete_netdev);
- schedule_work(&entry->delete_netdev);
- }
+ INIT_WORK(&entry->delete_netdev, delete_netdev);
+ schedule_work(&entry->delete_netdev);
} else {
spin_unlock(&devices_lock);
}
@@ -880,7 +873,6 @@ static long chan_get_sndtimeo_cb(struct l2cap_chan *chan)
static const struct l2cap_ops bt_6lowpan_chan_ops = {
.name = "L2CAP 6LoWPAN channel",
- .new_connection = chan_new_conn_cb,
.recv = chan_recv_cb,
.close = chan_close_cb,
.state_change = chan_state_change_cb,
@@ -1006,16 +998,19 @@ static int get_l2cap_conn(char *buf, bdaddr_t *addr, u8 *addr_type,
hci_dev_lock(hdev);
hcon = hci_conn_hash_lookup_le(hdev, addr, le_addr_type);
- hci_dev_unlock(hdev);
- hci_dev_put(hdev);
-
- if (!hcon)
+ if (!hcon) {
+ hci_dev_unlock(hdev);
+ hci_dev_put(hdev);
return -ENOENT;
+ }
- *conn = (struct l2cap_conn *)hcon->l2cap_data;
+ *conn = l2cap_conn_hold_unless_zero(hcon->l2cap_data);
BT_DBG("conn %p dst %pMR type %u", *conn, &hcon->dst, hcon->dst_type);
+ hci_dev_unlock(hdev);
+ hci_dev_put(hdev);
+
return 0;
}
@@ -1158,18 +1153,22 @@ static ssize_t lowpan_control_write(struct file *fp,
if (conn) {
struct lowpan_peer *peer;
- if (!is_bt_6lowpan(conn->hcon))
+ if (!is_bt_6lowpan(conn->hcon)) {
+ l2cap_conn_put(conn);
return -EINVAL;
+ }
peer = lookup_peer(conn);
if (peer) {
BT_DBG("6LoWPAN connection already exists");
+ l2cap_conn_put(conn);
return -EALREADY;
}
BT_DBG("conn %p dst %pMR type %d user %u", conn,
&conn->hcon->dst, conn->hcon->dst_type,
addr_type);
+ l2cap_conn_put(conn);
}
ret = bt_6lowpan_connect(&addr, addr_type);
@@ -1185,6 +1184,8 @@ static ssize_t lowpan_control_write(struct file *fp,
return ret;
ret = bt_6lowpan_disconnect(conn, addr_type);
+ if (conn)
+ l2cap_conn_put(conn);
if (ret < 0)
return ret;
@@ -1245,6 +1246,7 @@ static void disconnect_devices(void)
break;
new_dev->netdev = entry->netdev;
+ new_dev->hdev = entry->hdev;
INIT_LIST_HEAD(&new_dev->list);
list_add_rcu(&new_dev->list, &devices);
@@ -1256,7 +1258,7 @@ static void disconnect_devices(void)
ifdown(entry->netdev);
BT_DBG("Unregistering netdev %s %p",
entry->netdev->name, entry->netdev);
- lowpan_unregister_netdev(entry->netdev);
+ unregister_dev(entry);
kfree(entry);
}
}
diff --git a/net/bluetooth/eir.c b/net/bluetooth/eir.c
index 3f72111ba651..1de5f9df6eec 100644
--- a/net/bluetooth/eir.c
+++ b/net/bluetooth/eir.c
@@ -283,10 +283,12 @@ u8 eir_create_adv_data(struct hci_dev *hdev, u8 instance, u8 *ptr, u8 size)
if (!flags)
flags |= mgmt_get_adv_discov_flags(hdev);
- /* If flags would still be empty, then there is no need to
- * include the "Flags" AD field".
+ /* Only add the "Flags" if it fits together with the instance
+ * advertising data; drop it rather than overflow the buffer.
*/
- if (flags && (ad_len + eir_precalc_len(1) <= size)) {
+ if (flags &&
+ (ad_len + eir_precalc_len(1) +
+ (adv ? adv->adv_data_len : 0) <= size)) {
ptr[0] = 0x02;
ptr[1] = EIR_FLAGS;
ptr[2] = flags;
diff --git a/net/bluetooth/hci_codec.c b/net/bluetooth/hci_codec.c
index 3cc135bb1d30..5bc5003c387c 100644
--- a/net/bluetooth/hci_codec.c
+++ b/net/bluetooth/hci_codec.c
@@ -100,7 +100,7 @@ static void hci_read_codec_capabilities(struct hci_dev *hdev, __u8 transport,
caps = (void *)skb->data;
if (skb->len < sizeof(*caps))
goto error;
- if (skb->len < caps->len)
+ if (skb->len < sizeof(caps->len) + caps->len)
goto error;
len += sizeof(caps->len) + caps->len;
skb_pull(skb, sizeof(caps->len) + caps->len);
diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c
index d34c66b92fbc..17b0cd1c2b52 100644
--- a/net/bluetooth/hci_conn.c
+++ b/net/bluetooth/hci_conn.c
@@ -785,7 +785,7 @@ static int hci_le_big_terminate(struct hci_dev *hdev, u8 big, struct hci_conn *c
d->sync_handle = conn->sync_handle;
if (test_and_clear_bit(HCI_CONN_PA_SYNC, &conn->flags)) {
- hci_conn_hash_list_flag(hdev, find_bis, ISO_LINK,
+ hci_conn_hash_list_flag(hdev, find_bis, BIS_LINK,
HCI_CONN_PA_SYNC, d);
if (!d->count)
@@ -795,7 +795,7 @@ static int hci_le_big_terminate(struct hci_dev *hdev, u8 big, struct hci_conn *c
}
if (test_and_clear_bit(HCI_CONN_BIG_SYNC, &conn->flags)) {
- hci_conn_hash_list_flag(hdev, find_bis, ISO_LINK,
+ hci_conn_hash_list_flag(hdev, find_bis, BIS_LINK,
HCI_CONN_BIG_SYNC, d);
if (!d->count)
@@ -885,9 +885,11 @@ static void cis_cleanup(struct hci_conn *conn)
/* Check if ISO connection is a CIS and remove CIG if there are
* no other connections using it.
*/
- hci_conn_hash_list_state(hdev, find_cis, ISO_LINK, BT_BOUND, &d);
- hci_conn_hash_list_state(hdev, find_cis, ISO_LINK, BT_CONNECT, &d);
- hci_conn_hash_list_state(hdev, find_cis, ISO_LINK, BT_CONNECTED, &d);
+ hci_conn_hash_list_state(hdev, find_cis, CIS_LINK, BT_BOUND, &d);
+ hci_conn_hash_list_state(hdev, find_cis, CIS_LINK, BT_CONNECT,
+ &d);
+ hci_conn_hash_list_state(hdev, find_cis, CIS_LINK, BT_CONNECTED,
+ &d);
if (d.count)
return;
@@ -910,7 +912,8 @@ static struct hci_conn *__hci_conn_add(struct hci_dev *hdev, int type, bdaddr_t
if (!hdev->acl_mtu)
return ERR_PTR(-ECONNREFUSED);
break;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
if (hdev->iso_mtu)
/* Dedicated ISO Buffer exists */
break;
@@ -975,7 +978,8 @@ static struct hci_conn *__hci_conn_add(struct hci_dev *hdev, int type, bdaddr_t
hci_copy_identity_address(hdev, &conn->src, &conn->src_type);
conn->mtu = hdev->le_mtu ? hdev->le_mtu : hdev->acl_mtu;
break;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
/* conn->src should reflect the local identity address */
hci_copy_identity_address(hdev, &conn->src, &conn->src_type);
@@ -1071,7 +1075,8 @@ static void hci_conn_cleanup_child(struct hci_conn *conn, u8 reason)
if (HCI_CONN_HANDLE_UNSET(conn->handle))
hci_conn_failed(conn, reason);
break;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
if ((conn->state != BT_CONNECTED &&
!test_bit(HCI_CONN_CREATE_CIS, &conn->flags)) ||
test_bit(HCI_CONN_BIG_CREATED, &conn->flags))
@@ -1146,7 +1151,8 @@ void hci_conn_del(struct hci_conn *conn)
hdev->acl_cnt += conn->sent;
} else {
/* Unacked ISO frames */
- if (conn->type == ISO_LINK) {
+ if (conn->type == CIS_LINK ||
+ conn->type == BIS_LINK) {
if (hdev->iso_pkts)
hdev->iso_cnt += conn->sent;
else if (hdev->le_pkts)
@@ -1531,7 +1537,7 @@ static struct hci_conn *hci_add_bis(struct hci_dev *hdev, bdaddr_t *dst,
memcmp(conn->le_per_adv_data, base, base_len)))
return ERR_PTR(-EADDRINUSE);
- conn = hci_conn_add_unset(hdev, ISO_LINK, dst, HCI_ROLE_MASTER);
+ conn = hci_conn_add_unset(hdev, BIS_LINK, dst, HCI_ROLE_MASTER);
if (IS_ERR(conn))
return conn;
@@ -1740,7 +1746,7 @@ static int hci_le_create_big(struct hci_conn *conn, struct bt_iso_qos *qos)
data.count = 0;
/* Create a BIS for each bound connection */
- hci_conn_hash_list_state(hdev, bis_list, ISO_LINK,
+ hci_conn_hash_list_state(hdev, bis_list, BIS_LINK,
BT_BOUND, &data);
cp.handle = qos->bcast.big;
@@ -1835,12 +1841,12 @@ static bool hci_le_set_cig_params(struct hci_conn *conn, struct bt_iso_qos *qos)
for (data.cig = 0x00; data.cig < 0xf0; data.cig++) {
data.count = 0;
- hci_conn_hash_list_state(hdev, find_cis, ISO_LINK,
+ hci_conn_hash_list_state(hdev, find_cis, CIS_LINK,
BT_CONNECT, &data);
if (data.count)
continue;
- hci_conn_hash_list_state(hdev, find_cis, ISO_LINK,
+ hci_conn_hash_list_state(hdev, find_cis, CIS_LINK,
BT_CONNECTED, &data);
if (!data.count)
break;
@@ -1892,7 +1898,8 @@ struct hci_conn *hci_bind_cis(struct hci_dev *hdev, bdaddr_t *dst,
cis = hci_conn_hash_lookup_cis(hdev, dst, dst_type, qos->ucast.cig,
qos->ucast.cis);
if (!cis) {
- cis = hci_conn_add_unset(hdev, ISO_LINK, dst, HCI_ROLE_MASTER);
+ cis = hci_conn_add_unset(hdev, CIS_LINK, dst,
+ HCI_ROLE_MASTER);
if (IS_ERR(cis))
return cis;
cis->cleanup = cis_cleanup;
@@ -1982,7 +1989,7 @@ bool hci_iso_setup_path(struct hci_conn *conn)
int hci_conn_check_create_cis(struct hci_conn *conn)
{
- if (conn->type != ISO_LINK || !bacmp(&conn->dst, BDADDR_ANY))
+ if (conn->type != CIS_LINK)
return -EINVAL;
if (!conn->parent || conn->parent->state != BT_CONNECTED ||
@@ -2082,7 +2089,7 @@ struct hci_conn *hci_pa_create_sync(struct hci_dev *hdev, bdaddr_t *dst,
bt_dev_dbg(hdev, "dst %pMR type %d sid %d", dst, dst_type, sid);
- conn = hci_conn_add_unset(hdev, ISO_LINK, dst, HCI_ROLE_SLAVE);
+ conn = hci_conn_add_unset(hdev, BIS_LINK, dst, HCI_ROLE_SLAVE);
if (IS_ERR(conn))
return conn;
@@ -2259,7 +2266,7 @@ struct hci_conn *hci_connect_bis(struct hci_dev *hdev, bdaddr_t *dst,
* the start periodic advertising and create BIG commands have
* been queued
*/
- hci_conn_hash_list_state(hdev, bis_mark_per_adv, ISO_LINK,
+ hci_conn_hash_list_state(hdev, bis_mark_per_adv, BIS_LINK,
BT_BOUND, &data);
/* Queue start periodic advertising and create BIG */
@@ -2914,26 +2921,11 @@ int hci_abort_conn(struct hci_conn *conn, u8 reason)
conn->abort_reason = reason;
- /* If the connection is pending check the command opcode since that
- * might be blocking on hci_cmd_sync_work while waiting its respective
- * event so we need to hci_cmd_sync_cancel to cancel it.
- *
- * hci_connect_le serializes the connection attempts so only one
- * connection can be in BT_CONNECT at time.
+ /* Cancel the connect attempt. A return of 0 means the create command
+ * was still queued and got dequeued, so there is nothing to disconnect.
*/
- if (conn->state == BT_CONNECT && READ_ONCE(hdev->req_status) == HCI_REQ_PEND) {
- switch (hci_skb_event(hdev->sent_cmd)) {
- case HCI_EV_CONN_COMPLETE:
- case HCI_EV_LE_CONN_COMPLETE:
- case HCI_EV_LE_ENHANCED_CONN_COMPLETE:
- case HCI_EVT_LE_CIS_ESTABLISHED:
- hci_cmd_sync_cancel(hdev, ECANCELED);
- break;
- }
- /* Cancel connect attempt if still queued/pending */
- } else if (!hci_cancel_connect_sync(hdev, conn)) {
+ if (!hci_cancel_connect_sync(hdev, conn))
return 0;
- }
/* Run immediately if on cmd_sync_work since this may be called
* as a result to MGMT_OP_DISCONNECT/MGMT_OP_UNPAIR which does
diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c
index e96ccdd7ef15..fb93ff8f1e73 100644
--- a/net/bluetooth/hci_core.c
+++ b/net/bluetooth/hci_core.c
@@ -2729,6 +2729,8 @@ void hci_unregister_dev(struct hci_dev *hdev)
disable_work_sync(&hdev->tx_work);
disable_work_sync(&hdev->power_on);
disable_work_sync(&hdev->error_reset);
+ disable_delayed_work_sync(&hdev->cmd_timer);
+ disable_delayed_work_sync(&hdev->ncmd_timer);
hci_cmd_sync_clear(hdev);
@@ -2957,12 +2959,13 @@ int hci_recv_frame(struct hci_dev *hdev, struct sk_buff *skb)
break;
case HCI_ACLDATA_PKT:
/* Detect if ISO packet has been sent as ACL */
- if (hci_conn_num(hdev, ISO_LINK)) {
+ if (hci_conn_num(hdev, CIS_LINK) ||
+ hci_conn_num(hdev, BIS_LINK)) {
__u16 handle = __le16_to_cpu(hci_acl_hdr(skb)->handle);
__u8 type;
type = hci_conn_lookup_type(hdev, hci_handle(handle));
- if (type == ISO_LINK)
+ if (type == CIS_LINK || type == BIS_LINK)
hci_skb_pkt_type(skb) = HCI_ISODATA_PKT;
}
break;
@@ -3397,7 +3400,8 @@ static inline void hci_quote_sent(struct hci_conn *conn, int num, int *quote)
case LE_LINK:
cnt = hdev->le_mtu ? hdev->le_cnt : hdev->acl_cnt;
break;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
cnt = hdev->iso_mtu ? hdev->iso_cnt :
hdev->le_mtu ? hdev->le_cnt : hdev->acl_cnt;
break;
@@ -3411,7 +3415,7 @@ static inline void hci_quote_sent(struct hci_conn *conn, int num, int *quote)
}
static struct hci_conn *hci_low_sent(struct hci_dev *hdev, __u8 type,
- int *quote)
+ __u8 type2, int *quote)
{
struct hci_conn_hash *h = &hdev->conn_hash;
struct hci_conn *conn = NULL, *c;
@@ -3423,7 +3427,8 @@ static struct hci_conn *hci_low_sent(struct hci_dev *hdev, __u8 type,
rcu_read_lock();
list_for_each_entry_rcu(c, &h->list, list) {
- if (c->type != type || skb_queue_empty(&c->data_q))
+ if ((c->type != type && c->type != type2) ||
+ skb_queue_empty(&c->data_q))
continue;
if (c->state != BT_CONNECTED && c->state != BT_CONFIG)
@@ -3606,42 +3611,27 @@ static void __check_timeout(struct hci_dev *hdev, unsigned int cnt, u8 type)
}
/* Schedule SCO */
-static void hci_sched_sco(struct hci_dev *hdev)
+static void hci_sched_sco(struct hci_dev *hdev, __u8 type)
{
struct hci_conn *conn;
struct sk_buff *skb;
- int quote;
+ int quote, *cnt;
+ unsigned int pkts = hdev->sco_pkts;
- BT_DBG("%s", hdev->name);
+ bt_dev_dbg(hdev, "type %u", type);
- if (!hci_conn_num(hdev, SCO_LINK))
+ if (!hci_conn_num(hdev, type) || !pkts)
return;
- while (hdev->sco_cnt && (conn = hci_low_sent(hdev, SCO_LINK, "e))) {
- while (quote-- && (skb = skb_dequeue(&conn->data_q))) {
- BT_DBG("skb %p len %d", skb, skb->len);
- hci_send_frame(hdev, skb);
-
- conn->sent++;
- if (conn->sent == ~0)
- conn->sent = 0;
- }
- }
-}
-
-static void hci_sched_esco(struct hci_dev *hdev)
-{
- struct hci_conn *conn;
- struct sk_buff *skb;
- int quote;
-
- BT_DBG("%s", hdev->name);
-
- if (!hci_conn_num(hdev, ESCO_LINK))
- return;
+ /* Use sco_pkts if flow control has not been enabled which will limit
+ * the amount of buffer sent in a row.
+ */
+ if (!hci_dev_test_flag(hdev, HCI_SCO_FLOWCTL))
+ cnt = &pkts;
+ else
+ cnt = &hdev->sco_cnt;
- while (hdev->sco_cnt && (conn = hci_low_sent(hdev, ESCO_LINK,
- "e))) {
+ while (*cnt && (conn = hci_low_sent(hdev, type, type, "e))) {
while (quote-- && (skb = skb_dequeue(&conn->data_q))) {
BT_DBG("skb %p len %d", skb, skb->len);
hci_send_frame(hdev, skb);
@@ -3649,8 +3639,17 @@ static void hci_sched_esco(struct hci_dev *hdev)
conn->sent++;
if (conn->sent == ~0)
conn->sent = 0;
+ (*cnt)--;
}
}
+
+ /* Rescheduled if all packets were sent and flow control is not enabled
+ * as there could be more packets queued that could not be sent and
+ * since no HCI_EV_NUM_COMP_PKTS event will be generated the reschedule
+ * needs to be forced.
+ */
+ if (!pkts && !hci_dev_test_flag(hdev, HCI_SCO_FLOWCTL))
+ queue_work(hdev->workqueue, &hdev->tx_work);
}
static void hci_sched_acl_pkt(struct hci_dev *hdev)
@@ -3686,8 +3685,8 @@ static void hci_sched_acl_pkt(struct hci_dev *hdev)
chan->conn->sent++;
/* Send pending SCO packets right away */
- hci_sched_sco(hdev);
- hci_sched_esco(hdev);
+ hci_sched_sco(hdev, SCO_LINK);
+ hci_sched_sco(hdev, ESCO_LINK);
}
}
@@ -3742,8 +3741,8 @@ static void hci_sched_le(struct hci_dev *hdev)
chan->conn->sent++;
/* Send pending SCO packets right away */
- hci_sched_sco(hdev);
- hci_sched_esco(hdev);
+ hci_sched_sco(hdev, SCO_LINK);
+ hci_sched_sco(hdev, ESCO_LINK);
}
}
@@ -3760,12 +3759,14 @@ static void hci_sched_iso(struct hci_dev *hdev)
BT_DBG("%s", hdev->name);
- if (!hci_conn_num(hdev, ISO_LINK))
+ if (!hci_conn_num(hdev, CIS_LINK) &&
+ !hci_conn_num(hdev, BIS_LINK))
return;
cnt = hdev->iso_pkts ? &hdev->iso_cnt :
hdev->le_pkts ? &hdev->le_cnt : &hdev->acl_cnt;
- while (*cnt && (conn = hci_low_sent(hdev, ISO_LINK, "e))) {
+ while (*cnt && (conn = hci_low_sent(hdev, CIS_LINK, BIS_LINK,
+ "e))) {
while (quote-- && (skb = skb_dequeue(&conn->data_q))) {
BT_DBG("skb %p len %d", skb, skb->len);
hci_send_frame(hdev, skb);
@@ -3788,8 +3789,8 @@ static void hci_tx_work(struct work_struct *work)
if (!hci_dev_test_flag(hdev, HCI_USER_CHANNEL)) {
/* Schedule queues and send stuff to HCI driver */
- hci_sched_sco(hdev);
- hci_sched_esco(hdev);
+ hci_sched_sco(hdev, SCO_LINK);
+ hci_sched_sco(hdev, ESCO_LINK);
hci_sched_iso(hdev);
hci_sched_acl(hdev);
hci_sched_le(hdev);
diff --git a/net/bluetooth/hci_event.c b/net/bluetooth/hci_event.c
index 92270b99ee0f..fbd63e51b493 100644
--- a/net/bluetooth/hci_event.c
+++ b/net/bluetooth/hci_event.c
@@ -42,8 +42,6 @@
#define ZERO_KEY "\x00\x00\x00\x00\x00\x00\x00\x00" \
"\x00\x00\x00\x00\x00\x00\x00\x00"
-#define secs_to_jiffies(_secs) msecs_to_jiffies((_secs) * 1000)
-
/* Handle HCI Event packets */
static void *hci_ev_skb_pull(struct hci_dev *hdev, struct sk_buff *skb,
@@ -3778,7 +3776,7 @@ static void hci_unbound_cis_failed(struct hci_dev *hdev, u8 cig, u8 status)
lockdep_assert_held(&hdev->lock);
list_for_each_entry_safe(conn, tmp, &hdev->conn_hash.list, list) {
- if (conn->type != ISO_LINK || !bacmp(&conn->dst, BDADDR_ANY) ||
+ if (conn->type != CIS_LINK ||
conn->state == BT_OPEN || conn->iso_qos.ucast.cig != cig)
continue;
@@ -4448,12 +4446,15 @@ static void hci_num_comp_pkts_evt(struct hci_dev *hdev, void *data,
break;
case SCO_LINK:
+ case ESCO_LINK:
hdev->sco_cnt += count;
if (hdev->sco_cnt > hdev->sco_pkts)
hdev->sco_cnt = hdev->sco_pkts;
+
break;
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
if (hdev->iso_pkts) {
hdev->iso_cnt += count;
if (hdev->iso_cnt > hdev->iso_pkts)
@@ -6420,7 +6421,8 @@ static void hci_le_pa_sync_estabilished_evt(struct hci_dev *hdev, void *data,
conn->sync_handle = le16_to_cpu(ev->handle);
conn->sid = HCI_SID_INVALID;
- mask |= hci_proto_connect_ind(hdev, &ev->bdaddr, ISO_LINK, &flags);
+ mask |= hci_proto_connect_ind(hdev, &ev->bdaddr, BIS_LINK,
+ &flags);
if (!(mask & HCI_LM_ACCEPT)) {
hci_le_pa_term_sync(hdev, ev->handle);
goto unlock;
@@ -6430,7 +6432,7 @@ static void hci_le_pa_sync_estabilished_evt(struct hci_dev *hdev, void *data,
goto unlock;
/* Add connection to indicate PA sync event */
- pa_sync = hci_conn_add_unset(hdev, ISO_LINK, BDADDR_ANY,
+ pa_sync = hci_conn_add_unset(hdev, BIS_LINK, BDADDR_ANY,
HCI_ROLE_SLAVE);
if (IS_ERR(pa_sync))
@@ -6461,7 +6463,7 @@ static void hci_le_per_adv_report_evt(struct hci_dev *hdev, void *data,
hci_dev_lock(hdev);
- mask |= hci_proto_connect_ind(hdev, BDADDR_ANY, ISO_LINK, &flags);
+ mask |= hci_proto_connect_ind(hdev, BDADDR_ANY, BIS_LINK, &flags);
if (!(mask & HCI_LM_ACCEPT))
goto unlock;
@@ -6752,7 +6754,7 @@ static void hci_le_cis_estabilished_evt(struct hci_dev *hdev, void *data,
goto unlock;
}
- if (conn->type != ISO_LINK) {
+ if (conn->type != CIS_LINK) {
bt_dev_err(hdev,
"Invalid connection link type handle 0x%4.4x",
handle);
@@ -6870,7 +6872,7 @@ static void hci_le_cis_req_evt(struct hci_dev *hdev, void *data,
if (!acl)
goto unlock;
- mask = hci_proto_connect_ind(hdev, &acl->dst, ISO_LINK, &flags);
+ mask = hci_proto_connect_ind(hdev, &acl->dst, CIS_LINK, &flags);
if (!(mask & HCI_LM_ACCEPT)) {
hci_le_reject_cis(hdev, ev->cis_handle);
goto unlock;
@@ -6878,8 +6880,8 @@ static void hci_le_cis_req_evt(struct hci_dev *hdev, void *data,
cis = hci_conn_hash_lookup_handle(hdev, cis_handle);
if (!cis) {
- cis = hci_conn_add(hdev, ISO_LINK, &acl->dst, HCI_ROLE_SLAVE,
- cis_handle);
+ cis = hci_conn_add(hdev, CIS_LINK, &acl->dst,
+ HCI_ROLE_SLAVE, cis_handle);
if (IS_ERR(cis)) {
hci_le_reject_cis(hdev, ev->cis_handle);
goto unlock;
@@ -7017,7 +7019,7 @@ static void hci_le_big_sync_established_evt(struct hci_dev *hdev, void *data,
bt_dev_dbg(hdev, "ignore too large handle %u", handle);
continue;
}
- bis = hci_conn_add(hdev, ISO_LINK, BDADDR_ANY,
+ bis = hci_conn_add(hdev, BIS_LINK, BDADDR_ANY,
HCI_ROLE_SLAVE, handle);
if (IS_ERR(bis))
continue;
@@ -7076,7 +7078,7 @@ static void hci_le_big_info_adv_report_evt(struct hci_dev *hdev, void *data,
hci_dev_lock(hdev);
- mask |= hci_proto_connect_ind(hdev, BDADDR_ANY, ISO_LINK, &flags);
+ mask |= hci_proto_connect_ind(hdev, BDADDR_ANY, BIS_LINK, &flags);
if (!(mask & HCI_LM_ACCEPT))
goto unlock;
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
index 9e5c26c41f6e..661bbf06cf7f 100644
--- a/net/bluetooth/hci_sync.c
+++ b/net/bluetooth/hci_sync.c
@@ -2979,7 +2979,7 @@ static int hci_le_set_ext_scan_param_sync(struct hci_dev *hdev, u8 type,
if (sent) {
struct hci_conn *conn;
- conn = hci_conn_hash_lookup_ba(hdev, ISO_LINK,
+ conn = hci_conn_hash_lookup_ba(hdev, BIS_LINK,
&sent->bdaddr);
if (conn) {
struct bt_iso_qos *qos = &conn->iso_qos;
@@ -3888,6 +3888,28 @@ static int hci_write_ca_timeout_sync(struct hci_dev *hdev)
sizeof(param), ¶m, HCI_CMD_TIMEOUT);
}
+/* Enable SCO flow control if supported */
+static int hci_write_sync_flowctl_sync(struct hci_dev *hdev)
+{
+ struct hci_cp_write_sync_flowctl cp;
+ int err;
+
+ /* Check if the controller supports SCO and HCI_OP_WRITE_SYNC_FLOWCTL */
+ if (!lmp_sco_capable(hdev) || !(hdev->commands[10] & BIT(4)) ||
+ !test_bit(HCI_QUIRK_SYNC_FLOWCTL_SUPPORTED, &hdev->quirks))
+ return 0;
+
+ memset(&cp, 0, sizeof(cp));
+ cp.enable = 0x01;
+
+ err = __hci_cmd_sync_status(hdev, HCI_OP_WRITE_SYNC_FLOWCTL,
+ sizeof(cp), &cp, HCI_CMD_TIMEOUT);
+ if (!err)
+ hci_dev_set_flag(hdev, HCI_SCO_FLOWCTL);
+
+ return err;
+}
+
/* BR Controller init stage 2 command sequence */
static const struct hci_init_stage br_init2[] = {
/* HCI_OP_READ_BUFFER_SIZE */
@@ -3906,6 +3928,8 @@ static const struct hci_init_stage br_init2[] = {
HCI_INIT(hci_clear_event_filter_sync),
/* HCI_OP_WRITE_CA_TIMEOUT */
HCI_INIT(hci_write_ca_timeout_sync),
+ /* HCI_OP_WRITE_SYNC_FLOWCTL */
+ HCI_INIT(hci_write_sync_flowctl_sync),
{}
};
@@ -5519,7 +5543,7 @@ static int hci_disconnect_sync(struct hci_dev *hdev, struct hci_conn *conn,
{
struct hci_cp_disconnect cp;
- if (test_bit(HCI_CONN_BIG_CREATED, &conn->flags)) {
+ if (conn->type == BIS_LINK) {
/* This is a BIS connection, hci_conn_del will
* do the necessary cleanup.
*/
@@ -5572,7 +5596,7 @@ static int hci_connect_cancel_sync(struct hci_dev *hdev, struct hci_conn *conn,
if (conn->type == LE_LINK)
return hci_le_connect_cancel_sync(hdev, conn, reason);
- if (conn->type == ISO_LINK) {
+ if (conn->type == CIS_LINK) {
/* BLUETOOTH CORE SPECIFICATION Version 5.3 | Vol 4, Part E
* page 1857:
*
@@ -5585,9 +5609,10 @@ static int hci_connect_cancel_sync(struct hci_dev *hdev, struct hci_conn *conn,
return hci_disconnect_sync(hdev, conn, reason);
/* CIS with no Create CIS sent have nothing to cancel */
- if (bacmp(&conn->dst, BDADDR_ANY))
- return HCI_ERROR_LOCAL_HOST_TERM;
+ return HCI_ERROR_LOCAL_HOST_TERM;
+ }
+ if (conn->type == BIS_LINK) {
/* There is no way to cancel a BIS without terminating the BIG
* which is done later on connection cleanup.
*/
@@ -5649,9 +5674,12 @@ static int hci_reject_conn_sync(struct hci_dev *hdev, struct hci_conn *conn,
{
struct hci_cp_reject_conn_req cp;
- if (conn->type == ISO_LINK)
+ if (conn->type == CIS_LINK)
return hci_le_reject_cis_sync(hdev, conn, reason);
+ if (conn->type == BIS_LINK)
+ return -EINVAL;
+
if (conn->type == SCO_LINK || conn->type == ESCO_LINK)
return hci_reject_sco_sync(hdev, conn, reason);
@@ -6540,6 +6568,11 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data)
bt_dev_dbg(hdev, "conn %p", conn);
+ /* Hold a reference so conn stays valid for the HCI_CONN_CREATE
+ * clear_bit() at done.
+ */
+ hci_conn_get(conn);
+
clear_bit(HCI_CONN_SCANNING, &conn->flags);
conn->state = BT_CONNECT;
@@ -6552,6 +6585,7 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data)
hdev->le_scan_type == LE_SCAN_ACTIVE &&
!hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES)) {
hci_conn_del(conn);
+ hci_conn_put(conn);
return -EBUSY;
}
@@ -6597,6 +6631,12 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data)
&own_addr_type);
if (err)
goto done;
+
+ /* Mark create connection in flight so hci_cancel_connect_sync() can
+ * cancel it while blocking on the connection complete event.
+ */
+ set_bit(HCI_CONN_CREATE, &conn->flags);
+
/* Send command LE Extended Create Connection if supported */
if (use_ext_conn(hdev)) {
err = hci_le_ext_create_conn_sync(hdev, conn, own_addr_type);
@@ -6632,11 +6672,14 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data)
conn->conn_timeout, NULL);
done:
+ clear_bit(HCI_CONN_CREATE, &conn->flags);
+
if (err == -ETIMEDOUT)
hci_le_connect_cancel_sync(hdev, conn, 0x00);
/* Re-enable advertising after the connection attempt is finished. */
hci_resume_advertising_sync(hdev);
+ hci_conn_put(conn);
return err;
}
@@ -6911,10 +6954,25 @@ static int hci_acl_create_conn_sync(struct hci_dev *hdev, void *data)
else
cp.role_switch = 0x00;
- return __hci_cmd_sync_status_sk(hdev, HCI_OP_CREATE_CONN,
- sizeof(cp), &cp,
- HCI_EV_CONN_COMPLETE,
- conn->conn_timeout, NULL);
+ /* Hold a reference so conn stays valid for the HCI_CONN_CREATE
+ * clear_bit() below.
+ */
+ hci_conn_get(conn);
+
+ /* Mark create connection in flight so hci_cancel_connect_sync() can
+ * cancel it while blocking on the connection complete event.
+ */
+ set_bit(HCI_CONN_CREATE, &conn->flags);
+
+ err = __hci_cmd_sync_status_sk(hdev, HCI_OP_CREATE_CONN,
+ sizeof(cp), &cp,
+ HCI_EV_CONN_COMPLETE,
+ conn->conn_timeout, NULL);
+
+ clear_bit(HCI_CONN_CREATE, &conn->flags);
+ hci_conn_put(conn);
+
+ return err;
}
int hci_connect_acl_sync(struct hci_dev *hdev, struct hci_conn *conn)
@@ -6960,22 +7018,97 @@ int hci_connect_le_sync(struct hci_dev *hdev, struct hci_conn *conn)
create_le_conn_complete);
}
-int hci_cancel_connect_sync(struct hci_dev *hdev, struct hci_conn *conn)
+static int hci_acl_cancel_create_conn_sync(struct hci_dev *hdev,
+ struct hci_conn *conn)
{
- if (conn->state != BT_OPEN)
- return -EINVAL;
+ struct hci_cmd_sync_work_entry *entry;
+ int err = -EBUSY;
+
+ /* cmd_sync_work_lock makes the HCI_CONN_CREATE test and the cancel
+ * atomic against the worker, which takes this lock to dequeue every
+ * entry: while it is held no other command can become pending, so
+ * hci_cmd_sync_cancel() cannot cancel an unrelated command.
+ */
+ mutex_lock(&hdev->cmd_sync_work_lock);
+
+ /* In flight: this connection owns the pending request, cancel it. */
+ if (test_bit(HCI_CONN_CREATE, &conn->flags)) {
+ hci_cmd_sync_cancel(hdev, ECANCELED);
+ goto unlock;
+ }
+ /* Still queued: a successful dequeue means it never started, so there
+ * is nothing to disconnect.
+ */
+ entry = _hci_cmd_sync_lookup_entry(hdev, hci_acl_create_conn_sync, conn,
+ NULL);
+ if (entry) {
+ _hci_cmd_sync_cancel_entry(hdev, entry, -ECANCELED);
+ err = 0;
+ }
+
+unlock:
+ mutex_unlock(&hdev->cmd_sync_work_lock);
+ return err;
+}
+
+static int hci_le_cancel_create_conn_sync(struct hci_dev *hdev,
+ struct hci_conn *conn)
+{
+ struct hci_cmd_sync_work_entry *entry;
+ int err = -EBUSY;
+
+ /* cmd_sync_work_lock keeps the HCI_CONN_CREATE test and the cancel
+ * atomic against the cmd_sync worker.
+ */
+ mutex_lock(&hdev->cmd_sync_work_lock);
+
+ if (test_bit(HCI_CONN_CREATE, &conn->flags)) {
+ hci_cmd_sync_cancel(hdev, ECANCELED);
+ goto unlock;
+ }
+
+ entry = _hci_cmd_sync_lookup_entry(hdev, hci_le_create_conn_sync, conn,
+ create_le_conn_complete);
+ if (entry) {
+ _hci_cmd_sync_cancel_entry(hdev, entry, -ECANCELED);
+ err = 0;
+ }
+
+unlock:
+ mutex_unlock(&hdev->cmd_sync_work_lock);
+ return err;
+}
+
+static int hci_cis_cancel_create_conn_sync(struct hci_dev *hdev,
+ struct hci_conn *conn)
+{
+ /* LE Create CIS is shared by the whole CIG and cannot be dequeued
+ * per-connection, so only an in-flight command can be cancelled.
+ * cmd_sync_work_lock keeps the test and the cancel atomic against the
+ * cmd_sync worker.
+ */
+ mutex_lock(&hdev->cmd_sync_work_lock);
+
+ if (test_bit(HCI_CONN_CREATE_CIS, &conn->flags))
+ hci_cmd_sync_cancel(hdev, ECANCELED);
+
+ mutex_unlock(&hdev->cmd_sync_work_lock);
+ return -EBUSY;
+}
+
+int hci_cancel_connect_sync(struct hci_dev *hdev, struct hci_conn *conn)
+{
switch (conn->type) {
case ACL_LINK:
- return !hci_cmd_sync_dequeue_once(hdev,
- hci_acl_create_conn_sync,
- conn, NULL);
+ return hci_acl_cancel_create_conn_sync(hdev, conn);
case LE_LINK:
- return !hci_cmd_sync_dequeue_once(hdev, hci_le_create_conn_sync,
- conn, create_le_conn_complete);
+ return hci_le_cancel_create_conn_sync(hdev, conn);
+ case CIS_LINK:
+ return hci_cis_cancel_create_conn_sync(hdev, conn);
+ default:
+ return -ENOENT;
}
-
- return -ENOENT;
}
int hci_le_conn_update_sync(struct hci_dev *hdev, struct hci_conn *conn,
@@ -7015,7 +7148,7 @@ static void create_pa_complete(struct hci_dev *hdev, void *data, int err)
goto unlock;
/* Add connection to indicate PA sync error */
- pa_sync = hci_conn_add_unset(hdev, ISO_LINK, BDADDR_ANY,
+ pa_sync = hci_conn_add_unset(hdev, BIS_LINK, BDADDR_ANY,
HCI_ROLE_SLAVE);
if (IS_ERR(pa_sync))
diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c
index 697d613fbdfb..30c100243780 100644
--- a/net/bluetooth/iso.c
+++ b/net/bluetooth/iso.c
@@ -2211,7 +2211,7 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags)
static void iso_connect_cfm(struct hci_conn *hcon, __u8 status)
{
- if (hcon->type != ISO_LINK) {
+ if (hcon->type != CIS_LINK && hcon->type != BIS_LINK) {
if (hcon->type != LE_LINK)
return;
@@ -2252,7 +2252,7 @@ static void iso_connect_cfm(struct hci_conn *hcon, __u8 status)
static void iso_disconn_cfm(struct hci_conn *hcon, __u8 reason)
{
- if (hcon->type != ISO_LINK)
+ if (hcon->type != CIS_LINK && hcon->type != BIS_LINK)
return;
BT_DBG("hcon %p reason %d", hcon, reason);
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index 8a41d4b05b00..56337f164d30 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -411,7 +411,7 @@ static void l2cap_chan_timeout(struct work_struct *work)
BT_DBG("chan %p state %s", chan, state_to_string(chan->state));
- if (!conn) {
+ if (test_bit(FLAG_DEL, &chan->flags)) {
l2cap_chan_put(chan);
return;
}
@@ -422,6 +422,9 @@ static void l2cap_chan_timeout(struct work_struct *work)
*/
l2cap_chan_lock(chan);
+ if (test_bit(FLAG_DEL, &chan->flags))
+ goto unlock;
+
if (chan->state == BT_CONNECTED || chan->state == BT_CONFIG)
reason = ECONNREFUSED;
else if (chan->state == BT_CONNECT &&
@@ -434,10 +437,10 @@ static void l2cap_chan_timeout(struct work_struct *work)
chan->ops->close(chan);
+unlock:
l2cap_chan_unlock(chan);
- l2cap_chan_put(chan);
-
mutex_unlock(&conn->lock);
+ l2cap_chan_put(chan);
}
struct l2cap_chan *l2cap_chan_create(void)
@@ -490,6 +493,9 @@ static void l2cap_chan_destroy(struct kref *kref)
list_del(&chan->global_l);
write_unlock(&chan_list_lock);
+ if (chan->conn)
+ l2cap_conn_put(chan->conn);
+
kfree(chan);
}
@@ -519,7 +525,10 @@ void l2cap_chan_put(struct l2cap_chan *c)
}
EXPORT_SYMBOL_GPL(l2cap_chan_put);
-void l2cap_chan_set_defaults(struct l2cap_chan *chan)
+/* Initialise @chan with default values, inheriting from the parent channel
+ * @pchan when it is given.
+ */
+void l2cap_chan_set_defaults(struct l2cap_chan *chan, struct l2cap_chan *pchan)
{
chan->fcs = L2CAP_FCS_CRC16;
chan->max_tx = L2CAP_DEFAULT_MAX_TX;
@@ -533,6 +542,31 @@ void l2cap_chan_set_defaults(struct l2cap_chan *chan)
chan->retrans_timeout = L2CAP_DEFAULT_RETRANS_TO;
chan->monitor_timeout = L2CAP_DEFAULT_MONITOR_TO;
+ if (pchan) {
+ BT_DBG("chan %p pchan %p", chan, pchan);
+
+ chan->chan_type = pchan->chan_type;
+ chan->imtu = pchan->imtu;
+ chan->omtu = pchan->omtu;
+ chan->mode = pchan->mode;
+ chan->fcs = pchan->fcs;
+ chan->max_tx = pchan->max_tx;
+ chan->tx_win = pchan->tx_win;
+ chan->tx_win_max = pchan->tx_win_max;
+ chan->sec_level = pchan->sec_level;
+ chan->conf_state = pchan->conf_state;
+ chan->flags = pchan->flags;
+ chan->tx_credits = pchan->tx_credits;
+ chan->rx_credits = pchan->rx_credits;
+
+ if (chan->chan_type == L2CAP_CHAN_FIXED) {
+ chan->scid = pchan->scid;
+ chan->dcid = pchan->scid;
+ }
+
+ return;
+ }
+
chan->conf_state = 0;
set_bit(CONF_NOT_COMPLETE, &chan->conf_state);
@@ -593,7 +627,7 @@ void __l2cap_chan_add(struct l2cap_conn *conn, struct l2cap_chan *chan)
conn->disc_reason = HCI_ERROR_REMOTE_USER_TERM;
- chan->conn = conn;
+ chan->conn = l2cap_conn_get(conn);
switch (chan->chan_type) {
case L2CAP_CHAN_CONN_ORIENTED:
@@ -648,30 +682,26 @@ void l2cap_chan_add(struct l2cap_conn *conn, struct l2cap_chan *chan)
void l2cap_chan_del(struct l2cap_chan *chan, int err)
{
- struct l2cap_conn *conn = chan->conn;
-
__clear_chan_timer(chan);
- BT_DBG("chan %p, conn %p, err %d, state %s", chan, conn, err,
+ BT_DBG("chan %p, err %d, state %s", chan, err,
state_to_string(chan->state));
chan->ops->teardown(chan, err);
- if (conn) {
+ if (!test_and_set_bit(FLAG_DEL, &chan->flags)) {
/* Delete from channel list */
list_del(&chan->list);
l2cap_chan_put(chan);
- chan->conn = NULL;
-
/* Reference was only held for non-fixed channels or
* fixed channels that explicitly requested it using the
* FLAG_HOLD_HCI_CONN flag.
*/
if (chan->chan_type != L2CAP_CHAN_FIXED ||
test_bit(FLAG_HOLD_HCI_CONN, &chan->flags))
- hci_conn_drop(conn->hcon);
+ hci_conn_drop(chan->conn->hcon);
}
if (test_bit(CONF_NOT_COMPLETE, &chan->conf_state))
@@ -926,26 +956,41 @@ int l2cap_chan_check_security(struct l2cap_chan *chan, bool initiator)
initiator);
}
-static u8 l2cap_get_ident(struct l2cap_conn *conn)
+static int l2cap_get_ident(struct l2cap_conn *conn)
{
- u8 id;
+ u8 max;
+ int ident;
+ /* LE link does not support tools like l2ping so use the full range */
+ if (conn->hcon->type == LE_LINK)
+ max = 255;
/* Get next available identificator.
* 1 - 128 are used by kernel.
* 129 - 199 are reserved.
* 200 - 254 are used by utilities like l2ping, etc.
*/
+ else
+ max = 128;
+
+ /* Allocate ident using min as last used + 1 (cyclic) */
+ ident = ida_alloc_range(&conn->tx_ida, READ_ONCE(conn->tx_ident) + 1,
+ max, GFP_ATOMIC);
+ /* Force min 1 to start over */
+ if (ident <= 0) {
+ ident = ida_alloc_range(&conn->tx_ida, 1, max, GFP_ATOMIC);
+ if (ident <= 0) {
+ /* If all idents are in use, log an error, this is
+ * extremely unlikely to happen and would indicate a bug
+ * in the code that idents are not being freed properly.
+ */
+ BT_ERR("Unable to allocate ident: %d", ident);
+ return 0;
+ }
+ }
- mutex_lock(&conn->ident_lock);
-
- if (++conn->tx_ident > 128)
- conn->tx_ident = 1;
-
- id = conn->tx_ident;
-
- mutex_unlock(&conn->ident_lock);
+ WRITE_ONCE(conn->tx_ident, ident);
- return id;
+ return ident;
}
static void l2cap_send_acl(struct l2cap_conn *conn, struct sk_buff *skb,
@@ -1761,18 +1806,14 @@ static void l2cap_conn_del(struct hci_conn *hcon, int err)
disable_delayed_work_sync(&conn->info_timer);
disable_delayed_work_sync(&conn->id_addr_timer);
+ cancel_work_sync(&conn->pending_rx_work);
+
mutex_lock(&conn->lock);
kfree_skb(conn->rx_skb);
skb_queue_purge(&conn->pending_rx);
-
- /* We can not call flush_work(&conn->pending_rx_work) here since we
- * might block if we are running on a worker from the same workqueue
- * pending_rx_work is waiting on.
- */
- if (work_pending(&conn->pending_rx_work))
- cancel_work_sync(&conn->pending_rx_work);
+ ida_destroy(&conn->tx_ida);
l2cap_unregister_all_users(conn);
@@ -1886,7 +1927,7 @@ static void l2cap_monitor_timeout(struct work_struct *work)
l2cap_chan_lock(chan);
- if (!chan->conn) {
+ if (test_bit(FLAG_DEL, &chan->flags)) {
l2cap_chan_unlock(chan);
l2cap_chan_put(chan);
return;
@@ -1907,7 +1948,7 @@ static void l2cap_retrans_timeout(struct work_struct *work)
l2cap_chan_lock(chan);
- if (!chan->conn) {
+ if (test_bit(FLAG_DEL, &chan->flags)) {
l2cap_chan_unlock(chan);
l2cap_chan_put(chan);
return;
@@ -2522,7 +2563,7 @@ int l2cap_chan_send(struct l2cap_chan *chan, struct msghdr *msg, size_t len)
int err;
struct sk_buff_head seg_queue;
- if (!chan->conn)
+ if (test_bit(FLAG_DEL, &chan->flags))
return -ENOTCONN;
/* Connectionless channel */
@@ -3119,12 +3160,16 @@ static void l2cap_ack_timeout(struct work_struct *work)
l2cap_chan_lock(chan);
+ if (test_bit(FLAG_DEL, &chan->flags))
+ goto unlock;
+
frames_to_ack = __seq_offset(chan, chan->buffer_seq,
chan->last_acked_seq);
if (frames_to_ack)
l2cap_send_rr_or_rnr(chan, 0);
+unlock:
l2cap_chan_unlock(chan);
l2cap_chan_put(chan);
}
@@ -3975,6 +4020,38 @@ static inline int l2cap_command_rej(struct l2cap_conn *conn,
return 0;
}
+/* Allocate and initialise a channel for an incoming connection.
+ *
+ * The channel inherits its configuration from @pchan and is linked into @conn
+ * before ->new_connection() runs, so the conn list reference keeps it alive if
+ * the callback exposes it (e.g. via the socket accept queue) before this
+ * returns. The l2cap_chan_create() reference is taken over by the subsystem on
+ * success and dropped here on failure.
+ */
+static struct l2cap_chan *l2cap_new_connection(struct l2cap_conn *conn,
+ struct l2cap_chan *pchan)
+{
+ struct l2cap_chan *chan;
+
+ chan = l2cap_chan_create();
+ if (!chan)
+ return NULL;
+
+ l2cap_chan_set_defaults(chan, pchan);
+ chan->ops = pchan->ops;
+
+ __l2cap_chan_add(conn, chan);
+
+ if (pchan->ops->new_connection &&
+ pchan->ops->new_connection(pchan, chan) < 0) {
+ l2cap_chan_del(chan, 0);
+ l2cap_chan_put(chan);
+ return NULL;
+ }
+
+ return chan;
+}
+
static void l2cap_connect(struct l2cap_conn *conn, struct l2cap_cmd_hdr *cmd,
u8 *data, u8 rsp_code)
{
@@ -4021,7 +4098,7 @@ static void l2cap_connect(struct l2cap_conn *conn, struct l2cap_cmd_hdr *cmd,
goto response;
}
- chan = pchan->ops->new_connection(pchan);
+ chan = l2cap_new_connection(conn, pchan);
if (!chan)
goto response;
@@ -4039,8 +4116,6 @@ static void l2cap_connect(struct l2cap_conn *conn, struct l2cap_cmd_hdr *cmd,
chan->psm = psm;
chan->dcid = scid;
- __l2cap_chan_add(conn, chan);
-
dcid = chan->scid;
__set_chan_timer(chan, chan->ops->get_sndtimeo(chan));
@@ -4770,12 +4845,42 @@ static int l2cap_le_connect_rsp(struct l2cap_conn *conn,
return err;
}
+static void l2cap_put_ident(struct l2cap_conn *conn, u8 code, u8 id)
+{
+ int ret;
+
+ switch (code) {
+ case L2CAP_COMMAND_REJ:
+ case L2CAP_CONN_RSP:
+ case L2CAP_CONF_RSP:
+ case L2CAP_DISCONN_RSP:
+ case L2CAP_ECHO_RSP:
+ case L2CAP_INFO_RSP:
+ case L2CAP_CONN_PARAM_UPDATE_RSP:
+ case L2CAP_LE_CONN_RSP:
+ case L2CAP_ECRED_CONN_RSP:
+ case L2CAP_ECRED_RECONF_RSP:
+ /* The remote may send bogus ids that would make ida_free
+ * generate warnings, so only free ids that are actually
+ * allocated: probing the exact id returns -ENOSPC when it
+ * is in use, otherwise the probe allocated it and freeing
+ * is safe either way. Only on -ENOMEM is the id known to
+ * be unallocated and the free must be skipped.
+ */
+ ret = ida_alloc_range(&conn->tx_ida, id, id, GFP_ATOMIC);
+ if (ret >= 0 || ret == -ENOSPC)
+ ida_free(&conn->tx_ida, id);
+ }
+}
+
static inline int l2cap_bredr_sig_cmd(struct l2cap_conn *conn,
struct l2cap_cmd_hdr *cmd, u16 cmd_len,
u8 *data)
{
int err = 0;
+ l2cap_put_ident(conn, cmd->code, cmd->ident);
+
switch (cmd->code) {
case L2CAP_COMMAND_REJ:
l2cap_command_rej(conn, cmd, cmd_len, data);
@@ -4909,7 +5014,7 @@ static int l2cap_le_connect_req(struct l2cap_conn *conn,
goto response_unlock;
}
- chan = pchan->ops->new_connection(pchan);
+ chan = l2cap_new_connection(conn, pchan);
if (!chan) {
result = L2CAP_CR_LE_NO_MEM;
goto response_unlock;
@@ -4924,8 +5029,6 @@ static int l2cap_le_connect_req(struct l2cap_conn *conn,
chan->omtu = mtu;
chan->remote_mps = mps;
- __l2cap_chan_add(conn, chan);
-
l2cap_le_flowctl_init(chan, __le16_to_cpu(req->credits));
dcid = chan->scid;
@@ -5133,7 +5236,7 @@ static inline int l2cap_ecred_conn_req(struct l2cap_conn *conn,
continue;
}
- chan = pchan->ops->new_connection(pchan);
+ chan = l2cap_new_connection(conn, pchan);
if (!chan) {
result = L2CAP_CR_LE_NO_MEM;
continue;
@@ -5148,8 +5251,6 @@ static inline int l2cap_ecred_conn_req(struct l2cap_conn *conn,
chan->omtu = mtu;
chan->remote_mps = mps;
- __l2cap_chan_add(conn, chan);
-
l2cap_ecred_init(chan, __le16_to_cpu(req->credits));
/* Init response */
@@ -5489,6 +5590,8 @@ static inline int l2cap_le_sig_cmd(struct l2cap_conn *conn,
{
int err = 0;
+ l2cap_put_ident(conn, cmd->code, cmd->ident);
+
switch (cmd->code) {
case L2CAP_COMMAND_REJ:
l2cap_le_command_rej(conn, cmd, cmd_len, data);
@@ -6653,6 +6756,7 @@ static void l2cap_chan_le_send_credits(struct l2cap_chan *chan)
struct l2cap_conn *conn = chan->conn;
struct l2cap_le_credits pkt;
u16 return_credits = l2cap_le_rx_credits(chan);
+ int ident;
if (chan->mode != L2CAP_MODE_LE_FLOWCTL &&
chan->mode != L2CAP_MODE_EXT_FLOWCTL)
@@ -6670,9 +6774,18 @@ static void l2cap_chan_le_send_credits(struct l2cap_chan *chan)
pkt.cid = cpu_to_le16(chan->scid);
pkt.credits = cpu_to_le16(return_credits);
- chan->ident = l2cap_get_ident(conn);
+ ident = l2cap_get_ident(conn);
+
+ l2cap_send_cmd(conn, ident, L2CAP_LE_CREDITS, sizeof(pkt), &pkt);
- l2cap_send_cmd(conn, chan->ident, L2CAP_LE_CREDITS, sizeof(pkt), &pkt);
+ /* L2CAP_LE_CREDITS has no response so the ident is never released by
+ * l2cap_put_ident() - release it right away, otherwise the tx_ida
+ * range is exhausted after 254 packets and from then on credits are
+ * sent with the invalid ident 0, which some remote stacks ignore,
+ * stalling the channel.
+ */
+ if (ident > 0)
+ ida_free(&conn->tx_ida, ident);
}
void l2cap_chan_rx_avail(struct l2cap_chan *chan, ssize_t rx_avail)
@@ -7037,13 +7150,13 @@ static struct l2cap_conn *l2cap_conn_add(struct hci_conn *hcon)
hci_dev_test_flag(hcon->hdev, HCI_FORCE_BREDR_SMP)))
conn->local_fixed_chan |= L2CAP_FC_SMP_BREDR;
- mutex_init(&conn->ident_lock);
mutex_init(&conn->lock);
INIT_LIST_HEAD(&conn->chan_l);
INIT_LIST_HEAD(&conn->users);
INIT_DELAYED_WORK(&conn->info_timer, l2cap_info_timeout);
+ ida_init(&conn->tx_ida);
skb_queue_head_init(&conn->pending_rx);
INIT_WORK(&conn->pending_rx_work, process_pending_rx);
@@ -7415,14 +7528,12 @@ static void l2cap_connect_cfm(struct hci_conn *hcon, u8 status)
goto next;
l2cap_chan_lock(pchan);
- chan = pchan->ops->new_connection(pchan);
+ chan = l2cap_new_connection(conn, pchan);
if (chan) {
bacpy(&chan->src, &hcon->src);
bacpy(&chan->dst, &hcon->dst);
chan->src_type = bdaddr_src_type(hcon);
chan->dst_type = dst_type;
-
- __l2cap_chan_add(conn, chan);
}
l2cap_chan_unlock(pchan);
@@ -7639,6 +7750,7 @@ struct l2cap_conn *l2cap_conn_hold_unless_zero(struct l2cap_conn *c)
return c;
}
+EXPORT_SYMBOL(l2cap_conn_hold_unless_zero);
void l2cap_recv_acldata(struct hci_conn *hcon, struct sk_buff *skb, u16 flags)
{
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index 9dd9de8558b9..47ef01e2f7e2 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -45,7 +45,8 @@ static struct bt_sock_list l2cap_sk_list = {
static const struct proto_ops l2cap_sock_ops;
static void l2cap_sock_init(struct sock *sk, struct sock *parent);
static struct sock *l2cap_sock_alloc(struct net *net, struct socket *sock,
- int proto, gfp_t prio, int kern);
+ int proto, gfp_t prio, int kern,
+ struct l2cap_chan *chan);
static void l2cap_sock_cleanup_listen(struct sock *parent);
bool l2cap_is_socket(struct socket *sock)
@@ -1243,6 +1244,23 @@ static int l2cap_sock_recvmsg(struct socket *sock, struct msghdr *msg,
return err;
}
+/* Release the sock's ref on chan and clear the pointer so that the ref is
+ * dropped exactly once even if both l2cap_sock_kill() and
+ * l2cap_sock_destruct() run. Setting chan->data to NULL first stops any other
+ * task from dereferencing the now-dead sock pointer.
+ */
+static void l2cap_sock_put_chan(struct sock *sk)
+{
+ struct l2cap_chan *chan = l2cap_pi(sk)->chan;
+
+ if (!chan)
+ return;
+
+ chan->data = NULL;
+ l2cap_pi(sk)->chan = NULL;
+ l2cap_chan_put(chan);
+}
+
/* Kill socket (only if zapped and orphan)
* Must be called on unlocked socket, with l2cap channel lock.
*/
@@ -1253,13 +1271,9 @@ static void l2cap_sock_kill(struct sock *sk)
BT_DBG("sk %p state %s", sk, state_to_string(sk->sk_state));
- /* Sock is dead, so set chan data to NULL, avoid other task use invalid
- * sock pointer.
- */
- l2cap_pi(sk)->chan->data = NULL;
- /* Kill poor orphan */
+ l2cap_sock_put_chan(sk);
- l2cap_chan_put(l2cap_pi(sk)->chan);
+ /* Kill poor orphan */
sock_set_flag(sk, SOCK_DEAD);
sock_put(sk);
}
@@ -1502,12 +1516,13 @@ static void l2cap_sock_cleanup_listen(struct sock *parent)
}
}
-static struct l2cap_chan *l2cap_sock_new_connection_cb(struct l2cap_chan *chan)
+static int l2cap_sock_new_connection_cb(struct l2cap_chan *chan,
+ struct l2cap_chan *new_chan)
{
struct sock *sk, *parent = chan->data;
if (!parent)
- return NULL;
+ return -EINVAL;
lock_sock(parent);
@@ -1515,25 +1530,28 @@ static struct l2cap_chan *l2cap_sock_new_connection_cb(struct l2cap_chan *chan)
if (sk_acceptq_is_full(parent)) {
BT_DBG("backlog full %d", parent->sk_ack_backlog);
release_sock(parent);
- return NULL;
+ return -ENOBUFS;
}
sk = l2cap_sock_alloc(sock_net(parent), NULL, BTPROTO_L2CAP,
- GFP_ATOMIC, 0);
+ GFP_ATOMIC, 0, new_chan);
if (!sk) {
release_sock(parent);
- return NULL;
- }
+ return -ENOMEM;
+ }
bt_sock_reclassify_lock(sk, BTPROTO_L2CAP);
l2cap_sock_init(sk, parent);
+ /* The conn list reference taken by l2cap_new_connection() keeps new_chan
+ * alive once release_sock() lets another task free this socket.
+ */
bt_accept_enqueue(parent, sk, false);
release_sock(parent);
- return l2cap_pi(sk)->chan;
+ return 0;
}
static int l2cap_sock_recv_cb(struct l2cap_chan *chan, struct sk_buff *skb)
@@ -1831,10 +1849,7 @@ static void l2cap_sock_destruct(struct sock *sk)
BT_DBG("sk %p", sk);
- if (l2cap_pi(sk)->chan) {
- l2cap_pi(sk)->chan->data = NULL;
- l2cap_chan_put(l2cap_pi(sk)->chan);
- }
+ l2cap_sock_put_chan(sk);
list_for_each_entry_safe(rx_busy, next, &l2cap_pi(sk)->rx_busy, list) {
kfree_skb(rx_busy->skb);
@@ -1866,30 +1881,12 @@ static void l2cap_sock_init(struct sock *sk, struct sock *parent)
BT_DBG("sk %p", sk);
if (parent) {
- struct l2cap_chan *pchan = l2cap_pi(parent)->chan;
-
sk->sk_type = parent->sk_type;
bt_sk(sk)->flags = bt_sk(parent)->flags;
- chan->chan_type = pchan->chan_type;
- chan->imtu = pchan->imtu;
- chan->omtu = pchan->omtu;
- chan->conf_state = pchan->conf_state;
- chan->mode = pchan->mode;
- chan->fcs = pchan->fcs;
- chan->max_tx = pchan->max_tx;
- chan->tx_win = pchan->tx_win;
- chan->tx_win_max = pchan->tx_win_max;
- chan->sec_level = pchan->sec_level;
- chan->flags = pchan->flags;
- chan->tx_credits = pchan->tx_credits;
- chan->rx_credits = pchan->rx_credits;
-
- if (chan->chan_type == L2CAP_CHAN_FIXED) {
- chan->scid = pchan->scid;
- chan->dcid = pchan->scid;
- }
-
+ /* Channel configuration is inherited from the parent by
+ * l2cap_new_connection().
+ */
security_sk_clone(parent, sk);
} else {
switch (sk->sk_type) {
@@ -1915,7 +1912,7 @@ static void l2cap_sock_init(struct sock *sk, struct sock *parent)
chan->mode = L2CAP_MODE_BASIC;
}
- l2cap_chan_set_defaults(chan);
+ l2cap_chan_set_defaults(chan, NULL);
}
/* Default config options */
@@ -1934,10 +1931,10 @@ static struct proto l2cap_proto = {
};
static struct sock *l2cap_sock_alloc(struct net *net, struct socket *sock,
- int proto, gfp_t prio, int kern)
+ int proto, gfp_t prio, int kern,
+ struct l2cap_chan *chan)
{
struct sock *sk;
- struct l2cap_chan *chan;
sk = bt_sock_alloc(net, sock, &l2cap_proto, proto, prio, kern);
if (!sk)
@@ -1948,16 +1945,7 @@ static struct sock *l2cap_sock_alloc(struct net *net, struct socket *sock,
INIT_LIST_HEAD(&l2cap_pi(sk)->rx_busy);
- chan = l2cap_chan_create();
- if (!chan) {
- sk_free(sk);
- if (sock)
- sock->sk = NULL;
- return NULL;
- }
-
- l2cap_chan_hold(chan);
-
+ /* The sock takes ownership of the caller's reference on chan. */
l2cap_pi(sk)->chan = chan;
return sk;
@@ -1967,6 +1955,7 @@ static int l2cap_sock_create(struct net *net, struct socket *sock, int protocol,
int kern)
{
struct sock *sk;
+ struct l2cap_chan *chan;
BT_DBG("sock %p", sock);
@@ -1981,10 +1970,16 @@ static int l2cap_sock_create(struct net *net, struct socket *sock, int protocol,
sock->ops = &l2cap_sock_ops;
- sk = l2cap_sock_alloc(net, sock, protocol, GFP_ATOMIC, kern);
- if (!sk)
+ chan = l2cap_chan_create();
+ if (!chan)
return -ENOMEM;
+ sk = l2cap_sock_alloc(net, sock, protocol, GFP_ATOMIC, kern, chan);
+ if (!sk) {
+ l2cap_chan_put(chan);
+ return -ENOMEM;
+ }
+
l2cap_sock_init(sk, NULL);
bt_sock_link(&l2cap_sk_list, sk);
return 0;
diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c
index 06067fc40655..6280fcfcdd0c 100644
--- a/net/bluetooth/mgmt.c
+++ b/net/bluetooth/mgmt.c
@@ -3240,7 +3240,8 @@ static int disconnect(struct sock *sk, struct hci_dev *hdev, void *data,
static u8 link_to_bdaddr(u8 link_type, u8 addr_type)
{
switch (link_type) {
- case ISO_LINK:
+ case CIS_LINK:
+ case BIS_LINK:
case LE_LINK:
switch (addr_type) {
case ADDR_LE_DEV_PUBLIC:
@@ -5355,6 +5356,8 @@ static void mgmt_add_adv_patterns_monitor_complete(struct hci_dev *hdev,
if (monitor->state == ADV_MONITOR_STATE_NOT_REGISTERED)
monitor->state = ADV_MONITOR_STATE_REGISTERED;
hci_update_passive_scan(hdev);
+ } else {
+ hci_free_adv_monitor(hdev, monitor);
}
mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode,
diff --git a/net/bluetooth/msft.c b/net/bluetooth/msft.c
index 5a8ccc491b14..0da28910d00b 100644
--- a/net/bluetooth/msft.c
+++ b/net/bluetooth/msft.c
@@ -291,7 +291,7 @@ static int msft_le_monitor_advertisement_cb(struct hci_dev *hdev, u16 opcode,
monitor->state = ADV_MONITOR_STATE_OFFLOADED;
unlock:
- if (status)
+ if (status && msft->resuming)
hci_free_adv_monitor(hdev, monitor);
hci_dev_unlock(hdev);
diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
index 6383db546570..e5ef90f63478 100644
--- a/net/bluetooth/sco.c
+++ b/net/bluetooth/sco.c
@@ -410,9 +410,13 @@ static struct sock *sco_get_sock_listen(bdaddr_t *src)
sk1 = sk;
}
+ sk = sk ? sk : sk1;
+ if (sk)
+ sock_hold(sk);
+
read_unlock(&sco_sk_list.lock);
- return sk ? sk : sk1;
+ return sk;
}
static void sco_sock_destruct(struct sock *sk)
@@ -452,11 +456,13 @@ static void sco_sock_kill(struct sock *sk)
BT_DBG("sk %p state %d", sk, sk->sk_state);
/* Sock is dead, so set conn->sk to NULL to avoid possible UAF */
+ lock_sock(sk);
if (sco_pi(sk)->conn) {
sco_conn_lock(sco_pi(sk)->conn);
sco_pi(sk)->conn->sk = NULL;
sco_conn_unlock(sco_pi(sk)->conn);
}
+ release_sock(sk);
/* Kill poor orphan */
bt_sock_unlink(&sco_sk_list, sk);
@@ -1305,40 +1311,51 @@ static int sco_sock_release(struct socket *sock)
static void sco_conn_ready(struct sco_conn *conn)
{
- struct sock *parent;
- struct sock *sk = conn->sk;
+ struct sock *parent, *sk;
+
+ sco_conn_lock(conn);
+ sk = sco_sock_hold(conn);
+ sco_conn_unlock(conn);
BT_DBG("conn %p", conn);
if (sk) {
lock_sock(sk);
- sco_sock_clear_timer(sk);
- sk->sk_state = BT_CONNECTED;
- sk->sk_state_change(sk);
+
+ /* conn->sk may have become NULL if racing with sk close, but
+ * due to held hdev->lock, it can't become different sk.
+ */
+ if (conn->sk) {
+ sco_sock_clear_timer(sk);
+ sk->sk_state = BT_CONNECTED;
+ sk->sk_state_change(sk);
+ }
+
release_sock(sk);
+ sock_put(sk);
} else {
- sco_conn_lock(conn);
-
- if (!conn->hcon) {
- sco_conn_unlock(conn);
+ if (!conn->hcon)
return;
- }
+
+ lockdep_assert_held(&conn->hcon->hdev->lock);
parent = sco_get_sock_listen(&conn->hcon->src);
- if (!parent) {
- sco_conn_unlock(conn);
+ if (!parent)
return;
- }
lock_sock(parent);
+ sco_conn_lock(conn);
+
+ /* hdev->lock guarantees conn->sk == NULL still here */
+
+ if (parent->sk_state != BT_LISTEN)
+ goto release;
+
sk = sco_sock_alloc(sock_net(parent), NULL,
BTPROTO_SCO, GFP_ATOMIC, 0);
- if (!sk) {
- release_sock(parent);
- sco_conn_unlock(conn);
- return;
- }
+ if (!sk)
+ goto release;
sco_sock_init(sk, parent);
@@ -1356,9 +1373,10 @@ static void sco_conn_ready(struct sco_conn *conn)
/* Wake up parent */
parent->sk_data_ready(parent);
- release_sock(parent);
-
+release:
sco_conn_unlock(conn);
+ release_sock(parent);
+ sock_put(parent);
}
}
diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c
index d790d791ddd8..9e9e9e293309 100644
--- a/net/bluetooth/smp.c
+++ b/net/bluetooth/smp.c
@@ -3234,34 +3234,19 @@ static const struct l2cap_ops smp_chan_ops = {
.get_sndtimeo = l2cap_chan_no_get_sndtimeo,
};
-static inline struct l2cap_chan *smp_new_conn_cb(struct l2cap_chan *pchan)
+static inline int smp_new_conn_cb(struct l2cap_chan *chan,
+ struct l2cap_chan *new_chan)
{
- struct l2cap_chan *chan;
-
- BT_DBG("pchan %p", pchan);
-
- chan = l2cap_chan_create();
- if (!chan)
- return NULL;
-
- chan->chan_type = pchan->chan_type;
- chan->ops = &smp_chan_ops;
- chan->scid = pchan->scid;
- chan->dcid = chan->scid;
- chan->imtu = pchan->imtu;
- chan->omtu = pchan->omtu;
- chan->mode = pchan->mode;
+ new_chan->ops = &smp_chan_ops;
/* Other L2CAP channels may request SMP routines in order to
* change the security level. This means that the SMP channel
* lock must be considered in its own category to avoid lockdep
* warnings.
*/
- atomic_set(&chan->nesting, L2CAP_NESTING_SMP);
-
- BT_DBG("created chan %p", chan);
+ atomic_set(&new_chan->nesting, L2CAP_NESTING_SMP);
- return chan;
+ return 0;
}
static const struct l2cap_ops smp_root_chan_ops = {
@@ -3332,7 +3317,7 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid)
l2cap_add_scid(chan, cid);
- l2cap_chan_set_defaults(chan);
+ l2cap_chan_set_defaults(chan, NULL);
if (cid == L2CAP_CID_SMP) {
u8 bdaddr_type;
diff --git a/net/bridge/br_cfm.c b/net/bridge/br_cfm.c
index ffa571e38c54..d76a0440e2b8 100644
--- a/net/bridge/br_cfm.c
+++ b/net/bridge/br_cfm.c
@@ -805,6 +805,12 @@ int br_cfm_cc_ccm_tx(struct net_bridge *br, const u32 instance,
goto save;
}
+ if (!interval_to_us(mep->cc_config.exp_interval)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Invalid CCM interval");
+ return -EINVAL;
+ }
+
/* Start delayed work to transmit CCM frames. It is done with zero delay
* to send first frame immediately
*/
diff --git a/net/bridge/br_cfm_netlink.c b/net/bridge/br_cfm_netlink.c
index 2faab44652e7..91b9922dc3f2 100644
--- a/net/bridge/br_cfm_netlink.c
+++ b/net/bridge/br_cfm_netlink.c
@@ -34,7 +34,9 @@ br_cfm_cc_config_policy[IFLA_BRIDGE_CFM_CC_CONFIG_MAX + 1] = {
[IFLA_BRIDGE_CFM_CC_CONFIG_UNSPEC] = { .type = NLA_REJECT },
[IFLA_BRIDGE_CFM_CC_CONFIG_INSTANCE] = { .type = NLA_U32 },
[IFLA_BRIDGE_CFM_CC_CONFIG_ENABLE] = { .type = NLA_U32 },
- [IFLA_BRIDGE_CFM_CC_CONFIG_EXP_INTERVAL] = { .type = NLA_U32 },
+ [IFLA_BRIDGE_CFM_CC_CONFIG_EXP_INTERVAL] =
+ NLA_POLICY_RANGE(NLA_U32, BR_CFM_CCM_INTERVAL_3_3_MS,
+ BR_CFM_CCM_INTERVAL_10_MIN),
[IFLA_BRIDGE_CFM_CC_CONFIG_EXP_MAID] = {
.type = NLA_BINARY, .len = CFM_MAID_LENGTH },
};
diff --git a/net/bridge/br_if.c b/net/bridge/br_if.c
index 6ffc81eedf07..eee6ceb7cdda 100644
--- a/net/bridge/br_if.c
+++ b/net/bridge/br_if.c
@@ -391,6 +391,9 @@ void br_dev_delete(struct net_device *dev, struct list_head *head)
br_fdb_delete_by_port(br, NULL, 0, 1);
+ timer_shutdown_sync(&br->hello_timer);
+ timer_shutdown_sync(&br->topology_change_timer);
+ timer_shutdown_sync(&br->tcn_timer);
cancel_delayed_work_sync(&br->gc_work);
br_sysfs_delbr(br->dev);
diff --git a/net/bridge/br_stp.c b/net/bridge/br_stp.c
index 7d27b2e6038f..4cfebf5fcdf5 100644
--- a/net/bridge/br_stp.c
+++ b/net/bridge/br_stp.c
@@ -371,7 +371,8 @@ void br_topology_change_detection(struct net_bridge *br)
{
int isroot = br_is_root_bridge(br);
- if (br->stp_enabled != BR_KERNEL_STP)
+ if (br->stp_enabled != BR_KERNEL_STP ||
+ !(br->dev->flags & IFF_UP))
return;
br_info(br, "topology change detected, %s\n",
diff --git a/net/bridge/netfilter/ebtables.c b/net/bridge/netfilter/ebtables.c
index ba7a61f0ccd9..ab5eca1c9981 100644
--- a/net/bridge/netfilter/ebtables.c
+++ b/net/bridge/netfilter/ebtables.c
@@ -923,8 +923,7 @@ static int translate_table(struct net *net, const char *name,
* if an error occurs
*/
newinfo->chainstack =
- vmalloc(array_size(nr_cpu_ids,
- sizeof(*(newinfo->chainstack))));
+ vcalloc(nr_cpu_ids, sizeof(*(newinfo->chainstack)));
if (!newinfo->chainstack)
return -ENOMEM;
for_each_possible_cpu(i) {
@@ -941,7 +940,7 @@ static int translate_table(struct net *net, const char *name,
}
}
- cl_s = vmalloc(array_size(udc_cnt, sizeof(*cl_s)));
+ cl_s = vmalloc_array(udc_cnt, sizeof(*cl_s));
if (!cl_s)
return -ENOMEM;
i = 0; /* the i'th udc */
@@ -1021,8 +1020,8 @@ static int do_replace_finish(struct net *net, struct ebt_replace *repl,
* the check on the size is done later, when we have the lock
*/
if (repl->num_counters) {
- unsigned long size = repl->num_counters * sizeof(*counterstmp);
- counterstmp = vmalloc(size);
+ counterstmp = vmalloc_array(repl->num_counters,
+ sizeof(*counterstmp));
if (!counterstmp)
return -ENOMEM;
}
@@ -1389,7 +1388,7 @@ static int do_update_counters(struct net *net, const char *name,
if (num_counters == 0)
return -EINVAL;
- tmp = vmalloc(array_size(num_counters, sizeof(*tmp)));
+ tmp = vmalloc_array(num_counters, sizeof(*tmp));
if (!tmp)
return -ENOMEM;
@@ -1531,7 +1530,7 @@ static int copy_counters_to_user(struct ebt_table *t,
if (num_counters != nentries)
return -EINVAL;
- counterstmp = vmalloc(array_size(nentries, sizeof(*counterstmp)));
+ counterstmp = vmalloc_array(nentries, sizeof(*counterstmp));
if (!counterstmp)
return -ENOMEM;
diff --git a/net/bridge/netfilter/nft_meta_bridge.c b/net/bridge/netfilter/nft_meta_bridge.c
index d12a221366d6..475d7d1709b9 100644
--- a/net/bridge/netfilter/nft_meta_bridge.c
+++ b/net/bridge/netfilter/nft_meta_bridge.c
@@ -44,7 +44,9 @@ static void nft_meta_bridge_get_eval(const struct nft_expr *expr,
if (!br_dev || !br_vlan_enabled(br_dev))
goto err;
- br_vlan_get_pvid_rcu(in, &p_pvid);
+ if (br_vlan_get_pvid_rcu(in, &p_pvid))
+ goto err;
+
nft_reg_store16(dest, p_pvid);
return;
}
diff --git a/net/can/bcm.c b/net/can/bcm.c
index 152cc29e87d7..03626d1ee665 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -58,6 +58,7 @@
#include <linux/can/skb.h>
#include <linux/can/bcm.h>
#include <linux/slab.h>
+#include <linux/workqueue.h>
#include <linux/spinlock.h>
#include <net/sock.h>
#include <net/net_namespace.h>
@@ -91,6 +92,8 @@ MODULE_ALIAS("can-proto-2");
#define BCM_MIN_NAMELEN CAN_REQUIRED_SIZE(struct sockaddr_can, can_ifindex)
+static struct workqueue_struct *bcm_wq;
+
/*
* easy access to the first 64 bit of can(fd)_frame payload. cp->data is
* 64 bit aligned so the offset has to be multiples of 8 which is ensured
@@ -104,6 +107,7 @@ static inline u64 get_u64(const struct canfd_frame *cp, int offset)
struct bcm_op {
struct list_head list;
struct rcu_head rcu;
+ struct work_struct work;
int ifindex;
canid_t can_id;
u32 flags;
@@ -256,7 +260,7 @@ static int bcm_proc_show(struct seq_file *m, void *v)
(reduction == 100) ? "near " : "", reduction);
}
- list_for_each_entry(op, &bo->tx_ops, list) {
+ list_for_each_entry_rcu(op, &bo->tx_ops, list) {
seq_printf(m, "tx_op: %03X %s ", op->can_id,
bcm_proc_getifname(net, ifname, op->ifindex));
@@ -787,9 +791,12 @@ static struct bcm_op *bcm_find_op(struct list_head *ops,
return NULL;
}
-static void bcm_free_op_rcu(struct rcu_head *rcu_head)
+static void bcm_free_op_work(struct work_struct *work)
{
- struct bcm_op *op = container_of(rcu_head, struct bcm_op, rcu);
+ struct bcm_op *op = container_of(work, struct bcm_op, work);
+
+ hrtimer_cancel(&op->timer);
+ hrtimer_cancel(&op->thrtimer);
if ((op->frames) && (op->frames != &op->sframe))
kfree(op->frames);
@@ -797,9 +804,23 @@ static void bcm_free_op_rcu(struct rcu_head *rcu_head)
if ((op->last_frames) && (op->last_frames != &op->last_sframe))
kfree(op->last_frames);
+ /* the last possible access to op->timer/op->thrtimer has now
+ * happened above via hrtimer_cancel() - op->sk is no longer
+ * needed by any pending timer callback, so drop our reference
+ */
+ sock_put(op->sk);
+
kfree(op);
}
+static void bcm_free_op_rcu(struct rcu_head *rcu_head)
+{
+ struct bcm_op *op = container_of(rcu_head, struct bcm_op, rcu);
+
+ INIT_WORK(&op->work, bcm_free_op_work);
+ queue_work(bcm_wq, &op->work);
+}
+
static void bcm_remove_op(struct bcm_op *op)
{
hrtimer_cancel(&op->timer);
@@ -924,6 +945,7 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
struct bcm_sock *bo = bcm_sk(sk);
struct bcm_op *op;
struct canfd_frame *cf;
+ bool add_op_to_list = false;
unsigned int i;
int err;
@@ -1054,6 +1076,7 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
/* bcm_can_tx / bcm_tx_timeout_handler needs this */
op->sk = sk;
+ sock_hold(sk);
op->ifindex = ifindex;
/* initialize uninitialized (kzalloc) structure */
@@ -1065,8 +1088,7 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
hrtimer_init(&op->thrtimer, CLOCK_MONOTONIC,
HRTIMER_MODE_REL_SOFT);
- /* add this bcm_op to the list of the tx_ops */
- list_add(&op->list, &bo->tx_ops);
+ add_op_to_list = true;
} /* if ((op = bcm_find_op(&bo->tx_ops, msg_head->can_id, ifindex))) */
@@ -1088,6 +1110,10 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
op->flags |= TX_ANNOUNCE;
}
+ /* add this bcm_op to the list of the tx_ops? */
+ if (add_op_to_list)
+ list_add_rcu(&op->list, &bo->tx_ops);
+
if (op->flags & TX_ANNOUNCE)
bcm_can_tx(op);
@@ -1216,6 +1242,7 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
/* bcm_can_tx / bcm_tx_timeout_handler needs this */
op->sk = sk;
+ sock_hold(sk);
op->ifindex = ifindex;
/* ifindex for timeout events w/o previous frame reception */
@@ -1230,9 +1257,6 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
HRTIMER_MODE_REL_SOFT);
op->thrtimer.function = bcm_rx_thr_handler;
- /* add this bcm_op to the list of the rx_ops */
- list_add(&op->list, &bo->rx_ops);
-
/* call can_rx_register() */
do_rx_register = 1;
@@ -1298,6 +1322,11 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
op->rx_reg_dev = dev;
dev_put(dev);
+ } else {
+ /* the requested device is gone - do not
+ * silently succeed without registering
+ */
+ err = -ENODEV;
}
} else
@@ -1306,10 +1335,12 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg,
bcm_rx_handler, op, "bcm", sk);
if (err) {
/* this bcm rx op is broken -> remove it */
- list_del_rcu(&op->list);
bcm_remove_op(op);
return err;
}
+
+ /* add this bcm_op to the list of the rx_ops */
+ list_add_rcu(&op->list, &bo->rx_ops);
}
return msg_head->nframes * op->cfsiz + MHSIZ;
@@ -1367,12 +1398,13 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
{
struct sock *sk = sock->sk;
struct bcm_sock *bo = bcm_sk(sk);
- int ifindex = bo->ifindex; /* default ifindex for this bcm_op */
+ int ifindex;
struct bcm_msg_head msg_head;
int cfsiz;
int ret; /* read bytes or error codes as return value */
- if (!bo->bound)
+ /* Lockless fast-path check for bound socket */
+ if (!READ_ONCE(bo->bound))
return -ENOTCONN;
/* check for valid message length from userspace */
@@ -1388,17 +1420,38 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
if ((size - MHSIZ) % cfsiz)
return -EINVAL;
+ lock_sock(sk);
+
+ /* Re-validate under the socket lock: a concurrent bcm_notify()
+ * may have unbound this socket (device removal) after the
+ * lockless fast-path check above. bo->ifindex is only ever
+ * mutated under lock_sock(), so reading it here - instead of
+ * before taking the lock - guarantees it can't be observed
+ * torn against bo->bound.
+ */
+ if (!bo->bound) {
+ ret = -ENOTCONN;
+ goto out_release;
+ }
+
+ /* default ifindex for this bcm_op */
+ ifindex = bo->ifindex;
+
/* check for alternative ifindex for this bcm_op */
if (!ifindex && msg->msg_name) {
/* no bound device as default => check msg_name */
DECLARE_SOCKADDR(struct sockaddr_can *, addr, msg->msg_name);
- if (msg->msg_namelen < BCM_MIN_NAMELEN)
- return -EINVAL;
+ if (msg->msg_namelen < BCM_MIN_NAMELEN) {
+ ret = -EINVAL;
+ goto out_release;
+ }
- if (addr->can_family != AF_CAN)
- return -EINVAL;
+ if (addr->can_family != AF_CAN) {
+ ret = -EINVAL;
+ goto out_release;
+ }
/* ifindex from sendto() */
ifindex = addr->can_ifindex;
@@ -1407,20 +1460,21 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
struct net_device *dev;
dev = dev_get_by_index(sock_net(sk), ifindex);
- if (!dev)
- return -ENODEV;
+ if (!dev) {
+ ret = -ENODEV;
+ goto out_release;
+ }
if (dev->type != ARPHRD_CAN) {
dev_put(dev);
- return -ENODEV;
+ ret = -ENODEV;
+ goto out_release;
}
dev_put(dev);
}
}
- lock_sock(sk);
-
switch (msg_head.opcode) {
case TX_SETUP:
@@ -1470,6 +1524,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
break;
}
+out_release:
release_sock(sk);
return ret;
@@ -1506,7 +1561,12 @@ static void bcm_notify(struct bcm_sock *bo, unsigned long msg,
bo->bcm_proc_read = NULL;
}
#endif
- bo->bound = 0;
+ /* Paired with the lockless fast-path check in
+ * bcm_sendmsg(); bo->ifindex itself is only ever
+ * accessed under lock_sock() so it needs no
+ * annotation.
+ */
+ WRITE_ONCE(bo->bound, 0);
bo->ifindex = 0;
notify_enodev = 1;
}
@@ -1610,8 +1670,10 @@ static int bcm_release(struct socket *sock)
remove_proc_entry(bo->procname, net->can.bcmproc_dir);
#endif /* CONFIG_PROC_FS */
- list_for_each_entry_safe(op, next, &bo->tx_ops, list)
+ list_for_each_entry_safe(op, next, &bo->tx_ops, list) {
+ list_del_rcu(&op->list);
bcm_remove_op(op);
+ }
list_for_each_entry_safe(op, next, &bo->rx_ops, list) {
/*
@@ -1642,12 +1704,14 @@ static int bcm_release(struct socket *sock)
synchronize_rcu();
- list_for_each_entry_safe(op, next, &bo->rx_ops, list)
+ list_for_each_entry_safe(op, next, &bo->rx_ops, list) {
+ list_del_rcu(&op->list);
bcm_remove_op(op);
+ }
/* remove device reference */
if (bo->bound) {
- bo->bound = 0;
+ WRITE_ONCE(bo->bound, 0);
bo->ifindex = 0;
}
@@ -1716,7 +1780,10 @@ static int bcm_connect(struct socket *sock, struct sockaddr *uaddr, int len,
}
#endif /* CONFIG_PROC_FS */
- bo->bound = 1;
+ /* bo->ifindex above is fully assigned before this point; pairs
+ * with the lockless fast-path check in bcm_sendmsg()
+ */
+ WRITE_ONCE(bo->bound, 1);
fail:
release_sock(sk);
@@ -1832,11 +1899,15 @@ static int __init bcm_module_init(void)
{
int err;
+ bcm_wq = alloc_workqueue("can-bcm-wq", WQ_UNBOUND, 0);
+ if (!bcm_wq)
+ return -ENOMEM;
+
pr_info("can: broadcast manager protocol\n");
err = register_pernet_subsys(&canbcm_pernet_ops);
if (err)
- return err;
+ goto register_pernet_failed;
err = register_netdevice_notifier(&canbcm_notifier);
if (err)
@@ -1854,6 +1925,8 @@ static int __init bcm_module_init(void)
unregister_netdevice_notifier(&canbcm_notifier);
register_notifier_failed:
unregister_pernet_subsys(&canbcm_pernet_ops);
+register_pernet_failed:
+ destroy_workqueue(bcm_wq);
return err;
}
@@ -1862,6 +1935,8 @@ static void __exit bcm_module_exit(void)
can_proto_unregister(&bcm_can_proto);
unregister_netdevice_notifier(&canbcm_notifier);
unregister_pernet_subsys(&canbcm_pernet_ops);
+ rcu_barrier();
+ destroy_workqueue(bcm_wq);
}
module_init(bcm_module_init);
diff --git a/net/can/isotp.c b/net/can/isotp.c
index 3815deb5c4e1..176e16b46413 100644
--- a/net/can/isotp.c
+++ b/net/can/isotp.c
@@ -1217,11 +1217,18 @@ static int isotp_release(struct socket *sock)
SINGLE_MASK(so->txid),
isotp_rcv_echo, sk);
dev_put(dev);
- synchronize_rcu();
}
}
}
+ /* Always wait for a grace period before touching the timers below.
+ * A concurrent NETDEV_UNREGISTER may have already unregistered our
+ * filters and cleared so->bound in isotp_notify() without waiting
+ * for in-flight isotp_rcv() callers to finish, so this call must not
+ * be skipped just because so->bound is already 0 here.
+ */
+ synchronize_rcu();
+
hrtimer_cancel(&so->txfrtimer);
hrtimer_cancel(&so->txtimer);
hrtimer_cancel(&so->rxtimer);
diff --git a/net/core/dev.c b/net/core/dev.c
index 8a9a85af17a7..ce5a1ab86ab5 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -10241,12 +10241,14 @@ int __netdev_update_features(struct net_device *dev)
* *before* calling udp_tunnel_get_rx_info,
* but *after* calling udp_tunnel_drop_rx_info.
*/
+ udp_tunnel_nic_lock(dev);
if (features & NETIF_F_RX_UDP_TUNNEL_PORT) {
dev->features = features;
udp_tunnel_get_rx_info(dev);
} else {
udp_tunnel_drop_rx_info(dev);
}
+ udp_tunnel_nic_unlock(dev);
}
if (diff & NETIF_F_HW_VLAN_CTAG_FILTER) {
diff --git a/net/core/fib_rules.c b/net/core/fib_rules.c
index 388ff1d6d86b..35865ce1e2c5 100644
--- a/net/core/fib_rules.c
+++ b/net/core/fib_rules.c
@@ -330,7 +330,7 @@ int fib_rules_lookup(struct fib_rules_ops *ops, struct flowi *fl,
if (err != -EAGAIN) {
if ((arg->flags & FIB_LOOKUP_NOREF) ||
- likely(refcount_inc_not_zero(&rule->refcnt))) {
+ likely(fib_rule_get_safe(rule))) {
arg->rule = rule;
goto out;
}
@@ -388,8 +388,12 @@ int fib_rules_dump(struct net *net, struct notifier_block *nb, int family,
if (!ops)
return -EAFNOSUPPORT;
list_for_each_entry_rcu(rule, &ops->rules_list, list) {
+ if (!fib_rule_get_safe(rule))
+ continue;
+
err = call_fib_rule_notifier(nb, FIB_EVENT_RULE_ADD,
rule, family, extack);
+ fib_rule_put(rule);
if (err)
break;
}
diff --git a/net/core/filter.c b/net/core/filter.c
index b03621405d06..f54e7152d687 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2841,6 +2841,9 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start,
if (!space || (space == 1 && start != offset))
copy = msg->sg.data[i].length;
+ if (unlikely(copy + len < copy))
+ return -EINVAL;
+
page = alloc_pages(__GFP_NOWARN | GFP_ATOMIC | __GFP_COMP,
get_order(copy + len));
if (unlikely(!page))
@@ -3004,8 +3007,8 @@ BPF_CALL_4(bpf_msg_pop_data, struct sk_msg *, msg, u32, start,
u32, len, u64, flags)
{
u32 i = 0, l = 0, space, offset = 0;
- u64 last = start + len;
- int pop;
+ u64 last = (u64)start + len;
+ u32 pop;
if (unlikely(flags))
return -EINVAL;
@@ -4424,7 +4427,7 @@ u32 xdp_master_redirect(struct xdp_buff *xdp)
struct net_device *master, *slave;
master = netdev_master_upper_dev_get_rcu(xdp->rxq->dev);
- if (unlikely(!(master->flags & IFF_UP)))
+ if (unlikely(!master || !(master->flags & IFF_UP)))
return XDP_ABORTED;
slave = master->netdev_ops->ndo_xdp_get_xmit_slave(master, xdp);
if (slave && slave != xdp->rxq->dev) {
@@ -5507,11 +5510,24 @@ static int sol_tcp_sockopt(struct sock *sk, int optname,
KERNEL_SOCKPTR(optval), *optlen);
}
+static bool sk_allows_sol_ip_sockopt(struct sock *sk)
+{
+ switch (sk->sk_family) {
+ case AF_INET:
+ return true;
+ case AF_INET6:
+ /* Allow getting/setting sockopt for possible ipv4-mapped ipv6 socket. */
+ return sk->sk_type != SOCK_RAW && !ipv6_only_sock(sk);
+ default:
+ return false;
+ }
+}
+
static int sol_ip_sockopt(struct sock *sk, int optname,
char *optval, int *optlen,
bool getopt)
{
- if (sk->sk_family != AF_INET)
+ if (!sk_allows_sol_ip_sockopt(sk))
return -EINVAL;
switch (optname) {
@@ -5991,7 +6007,7 @@ static int bpf_ipv4_fib_lookup(struct net *net, struct bpf_fib_lookup *params,
struct neighbour *neigh;
struct net_device *dev;
struct fib_result res;
- struct flowi4 fl4;
+ struct flowi4 fl4 = {};
u32 mtu = 0;
int err;
@@ -6131,7 +6147,7 @@ static int bpf_ipv6_fib_lookup(struct net *net, struct bpf_fib_lookup *params,
struct neighbour *neigh;
struct net_device *dev;
struct inet6_dev *idev;
- struct flowi6 fl6;
+ struct flowi6 fl6 = {};
int strict = 0;
int oif, err;
u32 mtu = 0;
diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c
index 8be2ef8088be..bf63af1bca0d 100644
--- a/net/core/flow_dissector.c
+++ b/net/core/flow_dissector.c
@@ -1173,13 +1173,21 @@ bool __skb_flow_dissect(const struct net *net,
if (dissector_uses_key(flow_dissector,
FLOW_DISSECTOR_KEY_ETH_ADDRS)) {
- struct ethhdr *eth = eth_hdr(skb);
struct flow_dissector_key_eth_addrs *key_eth_addrs;
key_eth_addrs = skb_flow_dissector_target(flow_dissector,
FLOW_DISSECTOR_KEY_ETH_ADDRS,
target_container);
- memcpy(key_eth_addrs, eth, sizeof(*key_eth_addrs));
+ /* TC filter blocks can be shared across devices with
+ * different link types, so we cannot validate this
+ * when the filter is installed -- check at dissect time.
+ */
+ if (skb && skb->dev &&
+ skb->dev->type == ARPHRD_ETHER &&
+ skb_mac_header_was_set(skb))
+ memcpy(key_eth_addrs, eth_hdr(skb), sizeof(*key_eth_addrs));
+ else
+ memset(key_eth_addrs, 0, sizeof(*key_eth_addrs));
}
if (dissector_uses_key(flow_dissector,
diff --git a/net/core/lwt_bpf.c b/net/core/lwt_bpf.c
index 1a14f915b7a4..ab23323314b7 100644
--- a/net/core/lwt_bpf.c
+++ b/net/core/lwt_bpf.c
@@ -592,6 +592,7 @@ static int handle_gso_encap(struct sk_buff *skb, bool ipv4, int encap_len)
int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
{
+ bool is_udp_tunnel;
struct iphdr *iph;
bool ipv4;
int err;
@@ -605,10 +606,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
ipv4 = true;
if (unlikely(len < iph->ihl * 4))
return -EINVAL;
+ is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+ if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+ return -EINVAL;
} else if (iph->version == 6) {
ipv4 = false;
if (unlikely(len < sizeof(struct ipv6hdr)))
return -EINVAL;
+ is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP;
+ if (unlikely(is_udp_tunnel && len < sizeof(struct ipv6hdr) + sizeof(struct udphdr)))
+ return -EINVAL;
} else {
return -EINVAL;
}
@@ -630,6 +637,11 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
if (ingress)
skb_postpush_rcsum(skb, iph, len);
skb_reset_network_header(skb);
+ if (is_udp_tunnel) {
+ size_t iph_sz = ipv4 ? iph->ihl * 4 : sizeof(struct ipv6hdr);
+
+ skb_set_transport_header(skb, skb_network_offset(skb) + iph_sz);
+ }
memcpy(skb_network_header(skb), hdr, len);
bpf_compute_data_pointers(skb);
skb_clear_hash(skb);
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index bb4ae3864ad2..cc03e3fd85fc 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -334,6 +334,12 @@ static __net_init void preinit_net(struct net *net, struct user_namespace *user_
idr_init(&net->netns_ids);
spin_lock_init(&net->nsid_lock);
mutex_init(&net->ipv4.ra_mutex);
+
+#ifdef CONFIG_DEBUG_NET_SMALL_RTNL
+ mutex_init(&net->rtnl_mutex);
+ lock_set_cmp_fn(&net->rtnl_mutex, rtnl_net_lock_cmp_fn, NULL);
+#endif
+
preinit_net_sysctl(net);
}
diff --git a/net/core/netdev-genl.c b/net/core/netdev-genl.c
index 6d4db55e90ed..12c6f7d5add0 100644
--- a/net/core/netdev-genl.c
+++ b/net/core/netdev-genl.c
@@ -2,6 +2,7 @@
#include <linux/netdevice.h>
#include <linux/notifier.h>
+#include <linux/pid_namespace.h>
#include <linux/rtnetlink.h>
#include <net/busy_poll.h>
#include <net/net_namespace.h>
@@ -181,7 +182,8 @@ netdev_nl_napi_fill_one(struct sk_buff *rsp, struct napi_struct *napi,
goto nla_put_failure;
if (napi->thread) {
- pid = task_pid_nr(napi->thread);
+ pid = task_pid_nr_ns(napi->thread,
+ task_active_pid_ns(current));
if (nla_put_u32(rsp, NETDEV_A_NAPI_PID, pid))
goto nla_put_failure;
}
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index 118612325ce9..625f273aeeed 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -179,6 +179,87 @@ bool lockdep_rtnl_is_held(void)
EXPORT_SYMBOL(lockdep_rtnl_is_held);
#endif /* #ifdef CONFIG_PROVE_LOCKING */
+#ifdef CONFIG_DEBUG_NET_SMALL_RTNL
+void __rtnl_net_lock(struct net *net)
+{
+ ASSERT_RTNL();
+
+ mutex_lock(&net->rtnl_mutex);
+}
+EXPORT_SYMBOL(__rtnl_net_lock);
+
+void __rtnl_net_unlock(struct net *net)
+{
+ ASSERT_RTNL();
+
+ mutex_unlock(&net->rtnl_mutex);
+}
+EXPORT_SYMBOL(__rtnl_net_unlock);
+
+void rtnl_net_lock(struct net *net)
+{
+ rtnl_lock();
+ __rtnl_net_lock(net);
+}
+EXPORT_SYMBOL(rtnl_net_lock);
+
+void rtnl_net_unlock(struct net *net)
+{
+ __rtnl_net_unlock(net);
+ rtnl_unlock();
+}
+EXPORT_SYMBOL(rtnl_net_unlock);
+
+int rtnl_net_trylock(struct net *net)
+{
+ int ret = rtnl_trylock();
+
+ if (ret)
+ __rtnl_net_lock(net);
+
+ return ret;
+}
+EXPORT_SYMBOL(rtnl_net_trylock);
+
+static int rtnl_net_cmp_locks(const struct net *net_a, const struct net *net_b)
+{
+ if (net_eq(net_a, net_b))
+ return 0;
+
+ /* always init_net first */
+ if (net_eq(net_a, &init_net))
+ return -1;
+
+ if (net_eq(net_b, &init_net))
+ return 1;
+
+ /* otherwise lock in ascending order */
+ return net_a < net_b ? -1 : 1;
+}
+
+int rtnl_net_lock_cmp_fn(const struct lockdep_map *a, const struct lockdep_map *b)
+{
+ const struct net *net_a, *net_b;
+
+ net_a = container_of(a, struct net, rtnl_mutex.dep_map);
+ net_b = container_of(b, struct net, rtnl_mutex.dep_map);
+
+ return rtnl_net_cmp_locks(net_a, net_b);
+}
+
+bool rtnl_net_is_locked(struct net *net)
+{
+ return rtnl_is_locked() && mutex_is_locked(&net->rtnl_mutex);
+}
+EXPORT_SYMBOL(rtnl_net_is_locked);
+
+bool lockdep_rtnl_net_is_held(struct net *net)
+{
+ return lockdep_rtnl_is_held() && lockdep_is_held(&net->rtnl_mutex);
+}
+EXPORT_SYMBOL(lockdep_rtnl_net_is_held);
+#endif
+
static struct rtnl_link __rcu *__rcu *rtnl_msg_handlers[RTNL_FAMILY_MAX + 1];
static inline int rtm_msgindex(int msgtype)
diff --git a/net/devlink/rate.c b/net/devlink/rate.c
index adb5267d377c..f3b234f356f3 100644
--- a/net/devlink/rate.c
+++ b/net/devlink/rate.c
@@ -597,11 +597,6 @@ devl_rate_node_create(struct devlink *devlink, void *priv, char *node_name,
if (!rate_node)
return ERR_PTR(-ENOMEM);
- if (parent) {
- rate_node->parent = parent;
- refcount_inc(&rate_node->parent->refcnt);
- }
-
rate_node->type = DEVLINK_RATE_TYPE_NODE;
rate_node->devlink = devlink;
rate_node->priv = priv;
@@ -612,6 +607,11 @@ devl_rate_node_create(struct devlink *devlink, void *priv, char *node_name,
return ERR_PTR(-ENOMEM);
}
+ if (parent) {
+ rate_node->parent = parent;
+ refcount_inc(&rate_node->parent->refcnt);
+ }
+
refcount_set(&rate_node->refcnt, 1);
list_add(&rate_node->list, &devlink->rate_list);
devlink_rate_notify(rate_node, DEVLINK_CMD_RATE_NEW);
diff --git a/net/handshake/genl.c b/net/handshake/genl.c
index a5fa8b27f224..7ec805ae3d7d 100644
--- a/net/handshake/genl.c
+++ b/net/handshake/genl.c
@@ -37,7 +37,7 @@ static const struct genl_split_ops handshake_nl_ops[] = {
.doit = handshake_nl_done_doit,
.policy = handshake_done_nl_policy,
.maxattr = HANDSHAKE_A_DONE_REMOTE_AUTH,
- .flags = GENL_CMD_CAP_DO,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
};
diff --git a/net/ieee802154/core.c b/net/ieee802154/core.c
index 88adb04e4072..4a4379c828b9 100644
--- a/net/ieee802154/core.c
+++ b/net/ieee802154/core.c
@@ -351,7 +351,7 @@ static void __net_exit cfg802154_pernet_exit(struct net *net)
rtnl_lock();
list_for_each_entry(rdev, &cfg802154_rdev_list, list) {
if (net_eq(wpan_phy_net(&rdev->wpan_phy), net))
- WARN_ON(cfg802154_switch_netns(rdev, &init_net));
+ cfg802154_switch_netns(rdev, &init_net);
}
rtnl_unlock();
}
diff --git a/net/ieee802154/header_ops.c b/net/ieee802154/header_ops.c
index 41a556be1017..a9f0c8df5ae4 100644
--- a/net/ieee802154/header_ops.c
+++ b/net/ieee802154/header_ops.c
@@ -173,10 +173,13 @@ ieee802154_hdr_get_addr(const u8 *buf, int mode, bool omit_pan,
{
int pos = 0;
- addr->mode = mode;
-
- if (mode == IEEE802154_ADDR_NONE)
+ if (mode == IEEE802154_ADDR_NONE) {
+ memset(addr, 0, sizeof(*addr));
+ addr->mode = IEEE802154_ADDR_NONE;
return 0;
+ }
+
+ addr->mode = mode;
if (!omit_pan) {
memcpy(&addr->pan_id, buf + pos, 2);
diff --git a/net/ieee802154/ieee802154.h b/net/ieee802154/ieee802154.h
index c5d91f78301a..e765adc4b88f 100644
--- a/net/ieee802154/ieee802154.h
+++ b/net/ieee802154/ieee802154.h
@@ -16,6 +16,15 @@ void ieee802154_nl_exit(void);
.flags = GENL_ADMIN_PERM, \
}
+#define IEEE802154_OP_RELAXED(_cmd, _func) \
+ { \
+ .cmd = _cmd, \
+ .doit = _func, \
+ .dumpit = NULL, \
+ .flags = GENL_ADMIN_PERM, \
+ .validate = GENL_DONT_VALIDATE_STRICT,\
+ }
+
#define IEEE802154_DUMP(_cmd, _func, _dump) \
{ \
.cmd = _cmd, \
@@ -23,6 +32,14 @@ void ieee802154_nl_exit(void);
.dumpit = _dump, \
}
+#define IEEE802154_DUMP_PRIV(_cmd, _func, _dump) \
+ { \
+ .cmd = _cmd, \
+ .doit = _func, \
+ .dumpit = _dump, \
+ .flags = GENL_ADMIN_PERM, \
+ }
+
struct genl_info;
struct sk_buff *ieee802154_nl_create(int flags, u8 req);
diff --git a/net/ieee802154/netlink.c b/net/ieee802154/netlink.c
index 7d2de4ee6992..cacad21347ec 100644
--- a/net/ieee802154/netlink.c
+++ b/net/ieee802154/netlink.c
@@ -98,24 +98,24 @@ static const struct genl_small_ops ieee802154_ops[] = {
IEEE802154_OP(IEEE802154_SET_MACPARAMS, ieee802154_set_macparams),
IEEE802154_OP(IEEE802154_LLSEC_GETPARAMS, ieee802154_llsec_getparams),
IEEE802154_OP(IEEE802154_LLSEC_SETPARAMS, ieee802154_llsec_setparams),
- IEEE802154_DUMP(IEEE802154_LLSEC_LIST_KEY, NULL,
- ieee802154_llsec_dump_keys),
- IEEE802154_OP(IEEE802154_LLSEC_ADD_KEY, ieee802154_llsec_add_key),
- IEEE802154_OP(IEEE802154_LLSEC_DEL_KEY, ieee802154_llsec_del_key),
- IEEE802154_DUMP(IEEE802154_LLSEC_LIST_DEV, NULL,
- ieee802154_llsec_dump_devs),
- IEEE802154_OP(IEEE802154_LLSEC_ADD_DEV, ieee802154_llsec_add_dev),
- IEEE802154_OP(IEEE802154_LLSEC_DEL_DEV, ieee802154_llsec_del_dev),
- IEEE802154_DUMP(IEEE802154_LLSEC_LIST_DEVKEY, NULL,
- ieee802154_llsec_dump_devkeys),
- IEEE802154_OP(IEEE802154_LLSEC_ADD_DEVKEY, ieee802154_llsec_add_devkey),
- IEEE802154_OP(IEEE802154_LLSEC_DEL_DEVKEY, ieee802154_llsec_del_devkey),
- IEEE802154_DUMP(IEEE802154_LLSEC_LIST_SECLEVEL, NULL,
- ieee802154_llsec_dump_seclevels),
- IEEE802154_OP(IEEE802154_LLSEC_ADD_SECLEVEL,
- ieee802154_llsec_add_seclevel),
- IEEE802154_OP(IEEE802154_LLSEC_DEL_SECLEVEL,
- ieee802154_llsec_del_seclevel),
+ IEEE802154_DUMP_PRIV(IEEE802154_LLSEC_LIST_KEY, NULL,
+ ieee802154_llsec_dump_keys),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_ADD_KEY, ieee802154_llsec_add_key),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_DEL_KEY, ieee802154_llsec_del_key),
+ IEEE802154_DUMP_PRIV(IEEE802154_LLSEC_LIST_DEV, NULL,
+ ieee802154_llsec_dump_devs),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_ADD_DEV, ieee802154_llsec_add_dev),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_DEL_DEV, ieee802154_llsec_del_dev),
+ IEEE802154_DUMP_PRIV(IEEE802154_LLSEC_LIST_DEVKEY, NULL,
+ ieee802154_llsec_dump_devkeys),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_ADD_DEVKEY, ieee802154_llsec_add_devkey),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_DEL_DEVKEY, ieee802154_llsec_del_devkey),
+ IEEE802154_DUMP_PRIV(IEEE802154_LLSEC_LIST_SECLEVEL, NULL,
+ ieee802154_llsec_dump_seclevels),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_ADD_SECLEVEL,
+ ieee802154_llsec_add_seclevel),
+ IEEE802154_OP_RELAXED(IEEE802154_LLSEC_DEL_SECLEVEL,
+ ieee802154_llsec_del_seclevel),
};
static const struct genl_multicast_group ieee802154_mcgrps[] = {
diff --git a/net/ife/ife.c b/net/ife/ife.c
index be05b690b9ef..7a75947a31e3 100644
--- a/net/ife/ife.c
+++ b/net/ife/ife.c
@@ -79,7 +79,7 @@ void *ife_decode(struct sk_buff *skb, u16 *metalen)
if (unlikely(ifehdrln < 2))
return NULL;
- if (unlikely(!pskb_may_pull(skb, total_pull)))
+ if (unlikely(!pskb_may_pull(skb, total_pull + ETH_HLEN)))
return NULL;
ifehdr = (struct ifeheadr *)(skb->data + skb->dev->hard_header_len);
diff --git a/net/ipv4/igmp.c b/net/ipv4/igmp.c
index cdf146af4f79..3a6e0a371127 100644
--- a/net/ipv4/igmp.c
+++ b/net/ipv4/igmp.c
@@ -232,16 +232,20 @@ static void igmp_gq_start_timer(struct in_device *in_dev)
return;
in_dev->mr_gq_running = 1;
- if (!mod_timer(&in_dev->mr_gq_timer, exp))
- in_dev_hold(in_dev);
+ if (in_dev_hold_safe(in_dev)) {
+ if (mod_timer(&in_dev->mr_gq_timer, exp))
+ in_dev_put(in_dev);
+ }
}
static void igmp_ifc_start_timer(struct in_device *in_dev, int delay)
{
- int tv = get_random_u32_below(delay);
+ if (in_dev_hold_safe(in_dev)) {
+ int tv = get_random_u32_below(delay);
- if (!mod_timer(&in_dev->mr_ifc_timer, jiffies+tv+2))
- in_dev_hold(in_dev);
+ if (mod_timer(&in_dev->mr_ifc_timer, jiffies + tv + 2))
+ in_dev_put(in_dev);
+ }
}
static void igmp_mod_timer(struct ip_mc_list *im, int max_delay)
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index dd39cabb3900..5ff45bc85442 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1186,6 +1186,9 @@ static bool reqsk_queue_hash_req(struct request_sock *req,
/* The timer needs to be setup after a successful insertion. */
timer_setup(&req->rsk_timer, reqsk_timer_handler, TIMER_PINNED);
+
+ preempt_disable_nested();
+
mod_timer(&req->rsk_timer, jiffies + timeout);
/* before letting lookups find us, make sure all req fields
@@ -1193,6 +1196,9 @@ static bool reqsk_queue_hash_req(struct request_sock *req,
*/
smp_wmb();
refcount_set(&req->rsk_refcnt, 2 + 1);
+
+ preempt_enable_nested();
+
return true;
}
diff --git a/net/ipv4/ip_vti.c b/net/ipv4/ip_vti.c
index fc9516166307..37306978e5b4 100644
--- a/net/ipv4/ip_vti.c
+++ b/net/ipv4/ip_vti.c
@@ -594,6 +594,9 @@ static int vti_changelink(struct net_device *dev, struct nlattr *tb[],
struct ip_tunnel_parm_kern p;
__u32 fwmark = t->fwmark;
+ if (!rtnl_dev_link_net_capable(dev, t->net))
+ return -EPERM;
+
vti_netlink_parms(data, &p, &fwmark);
return ip_tunnel_changelink(dev, tb, &p, fwmark);
}
diff --git a/net/ipv4/ipip.c b/net/ipv4/ipip.c
index dc0db5895e0e..f16101dc9f63 100644
--- a/net/ipv4/ipip.c
+++ b/net/ipv4/ipip.c
@@ -466,6 +466,9 @@ static int ipip_changelink(struct net_device *dev, struct nlattr *tb[],
bool collect_md;
__u32 fwmark = t->fwmark;
+ if (!rtnl_dev_link_net_capable(dev, t->net))
+ return -EPERM;
+
if (ip_tunnel_netlink_encap_parms(data, &ipencap)) {
int err = ip_tunnel_encap_setup(t, &ipencap);
diff --git a/net/ipv4/netfilter/nf_reject_ipv4.c b/net/ipv4/netfilter/nf_reject_ipv4.c
index a7a3439fe780..1c739587bacd 100644
--- a/net/ipv4/netfilter/nf_reject_ipv4.c
+++ b/net/ipv4/netfilter/nf_reject_ipv4.c
@@ -80,7 +80,7 @@ static bool nf_skb_is_icmp_unreach(const struct sk_buff *skb)
if (iph->protocol != IPPROTO_ICMP)
return false;
- thoff = skb_network_offset(skb) + sizeof(*iph);
+ thoff = skb_network_offset(skb) + ip_hdrlen(skb);
tp = skb_header_pointer(skb,
thoff + offsetof(struct icmphdr, type),
diff --git a/net/ipv4/udp_tunnel_core.c b/net/ipv4/udp_tunnel_core.c
index 619a53eb672d..7e2f7d9716da 100644
--- a/net/ipv4/udp_tunnel_core.c
+++ b/net/ipv4/udp_tunnel_core.c
@@ -119,15 +119,17 @@ void udp_tunnel_notify_add_rx_port(struct socket *sock, unsigned short type)
struct udp_tunnel_info ti;
struct net_device *dev;
+ ASSERT_RTNL();
+
ti.type = type;
ti.sa_family = sk->sk_family;
ti.port = inet_sk(sk)->inet_sport;
- rcu_read_lock();
- for_each_netdev_rcu(net, dev) {
+ for_each_netdev(net, dev) {
+ udp_tunnel_nic_lock(dev);
udp_tunnel_nic_add_port(dev, &ti);
+ udp_tunnel_nic_unlock(dev);
}
- rcu_read_unlock();
}
EXPORT_SYMBOL_GPL(udp_tunnel_notify_add_rx_port);
@@ -139,15 +141,17 @@ void udp_tunnel_notify_del_rx_port(struct socket *sock, unsigned short type)
struct udp_tunnel_info ti;
struct net_device *dev;
+ ASSERT_RTNL();
+
ti.type = type;
ti.sa_family = sk->sk_family;
ti.port = inet_sk(sk)->inet_sport;
- rcu_read_lock();
- for_each_netdev_rcu(net, dev) {
+ for_each_netdev(net, dev) {
+ udp_tunnel_nic_lock(dev);
udp_tunnel_nic_del_port(dev, &ti);
+ udp_tunnel_nic_unlock(dev);
}
- rcu_read_unlock();
}
EXPORT_SYMBOL_GPL(udp_tunnel_notify_del_rx_port);
diff --git a/net/ipv4/udp_tunnel_nic.c b/net/ipv4/udp_tunnel_nic.c
index 6d13d2f82924..b13e8f7092f4 100644
--- a/net/ipv4/udp_tunnel_nic.c
+++ b/net/ipv4/udp_tunnel_nic.c
@@ -29,6 +29,7 @@ struct udp_tunnel_nic_table_entry {
* struct udp_tunnel_nic - UDP tunnel port offload state
* @work: async work for talking to hardware from process context
* @dev: netdev pointer
+ * @lock: protects all fields
* @need_sync: at least one port start changed
* @need_replay: space was freed, we need a replay of all ports
* @work_pending: @work is currently scheduled
@@ -41,6 +42,8 @@ struct udp_tunnel_nic {
struct net_device *dev;
+ struct mutex lock;
+
u8 need_sync:1;
u8 need_replay:1;
u8 work_pending:1;
@@ -298,22 +301,11 @@ __udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn)
static void
udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn)
{
- const struct udp_tunnel_nic_info *info = dev->udp_tunnel_nic_info;
- bool may_sleep;
-
- if (!utn->need_sync)
+ if (!utn->need_sync || utn->work_pending)
return;
- /* Drivers which sleep in the callback need to update from
- * the workqueue, if we come from the tunnel driver's notification.
- */
- may_sleep = info->flags & UDP_TUNNEL_NIC_INFO_MAY_SLEEP;
- if (!may_sleep)
- __udp_tunnel_nic_device_sync(dev, utn);
- if (may_sleep || utn->need_replay) {
- queue_work(udp_tunnel_nic_workqueue, &utn->work);
- utn->work_pending = 1;
- }
+ queue_work(udp_tunnel_nic_workqueue, &utn->work);
+ utn->work_pending = 1;
}
static bool
@@ -554,12 +546,12 @@ static void __udp_tunnel_nic_reset_ntf(struct net_device *dev)
struct udp_tunnel_nic *utn;
unsigned int i, j;
- ASSERT_RTNL();
-
utn = dev->udp_tunnel_nic;
if (!utn)
return;
+ mutex_lock(&utn->lock);
+
utn->need_sync = false;
for (i = 0; i < utn->n_tables; i++)
for (j = 0; j < info->tables[i].n_entries; j++) {
@@ -569,7 +561,7 @@ static void __udp_tunnel_nic_reset_ntf(struct net_device *dev)
entry->flags &= ~(UDP_TUNNEL_NIC_ENTRY_DEL |
UDP_TUNNEL_NIC_ENTRY_OP_FAIL);
- /* We don't release rtnl across ops */
+ /* We don't release utn lock across ops */
WARN_ON(entry->flags & UDP_TUNNEL_NIC_ENTRY_FROZEN);
if (!entry->use_cnt)
continue;
@@ -579,6 +571,8 @@ static void __udp_tunnel_nic_reset_ntf(struct net_device *dev)
}
__udp_tunnel_nic_device_sync(dev, utn);
+
+ mutex_unlock(&utn->lock);
}
static size_t
@@ -643,6 +637,33 @@ __udp_tunnel_nic_dump_write(struct net_device *dev, unsigned int table,
return -EMSGSIZE;
}
+static void __udp_tunnel_nic_assert_locked(struct net_device *dev)
+{
+ struct udp_tunnel_nic *utn;
+
+ utn = dev->udp_tunnel_nic;
+ if (utn)
+ lockdep_assert_held(&utn->lock);
+}
+
+static void __udp_tunnel_nic_lock(struct net_device *dev)
+{
+ struct udp_tunnel_nic *utn;
+
+ utn = dev->udp_tunnel_nic;
+ if (utn)
+ mutex_lock(&utn->lock);
+}
+
+static void __udp_tunnel_nic_unlock(struct net_device *dev)
+{
+ struct udp_tunnel_nic *utn;
+
+ utn = dev->udp_tunnel_nic;
+ if (utn)
+ mutex_unlock(&utn->lock);
+}
+
static const struct udp_tunnel_nic_ops __udp_tunnel_nic_ops = {
.get_port = __udp_tunnel_nic_get_port,
.set_port_priv = __udp_tunnel_nic_set_port_priv,
@@ -651,6 +672,9 @@ static const struct udp_tunnel_nic_ops __udp_tunnel_nic_ops = {
.reset_ntf = __udp_tunnel_nic_reset_ntf,
.dump_size = __udp_tunnel_nic_dump_size,
.dump_write = __udp_tunnel_nic_dump_write,
+ .assert_locked = __udp_tunnel_nic_assert_locked,
+ .lock = __udp_tunnel_nic_lock,
+ .unlock = __udp_tunnel_nic_unlock,
};
static void
@@ -710,11 +734,15 @@ static void udp_tunnel_nic_device_sync_work(struct work_struct *work)
container_of(work, struct udp_tunnel_nic, work);
rtnl_lock();
+ mutex_lock(&utn->lock);
+
utn->work_pending = 0;
__udp_tunnel_nic_device_sync(utn->dev, utn);
if (utn->need_replay)
udp_tunnel_nic_replay(utn->dev, utn);
+
+ mutex_unlock(&utn->lock);
rtnl_unlock();
}
@@ -730,6 +758,7 @@ udp_tunnel_nic_alloc(const struct udp_tunnel_nic_info *info,
return NULL;
utn->n_tables = n_tables;
INIT_WORK(&utn->work, udp_tunnel_nic_device_sync_work);
+ mutex_init(&utn->lock);
for (i = 0; i < n_tables; i++) {
utn->entries[i] = kcalloc(info->tables[i].n_entries,
@@ -821,8 +850,11 @@ static int udp_tunnel_nic_register(struct net_device *dev)
dev_hold(dev);
dev->udp_tunnel_nic = utn;
- if (!(info->flags & UDP_TUNNEL_NIC_INFO_OPEN_ONLY))
+ if (!(info->flags & UDP_TUNNEL_NIC_INFO_OPEN_ONLY)) {
+ udp_tunnel_nic_lock(dev);
udp_tunnel_get_rx_info(dev);
+ udp_tunnel_nic_unlock(dev);
+ }
return 0;
}
@@ -832,6 +864,8 @@ udp_tunnel_nic_unregister(struct net_device *dev, struct udp_tunnel_nic *utn)
{
const struct udp_tunnel_nic_info *info = dev->udp_tunnel_nic_info;
+ udp_tunnel_nic_lock(dev);
+
/* For a shared table remove this dev from the list of sharing devices
* and if there are other devices just detach.
*/
@@ -841,8 +875,10 @@ udp_tunnel_nic_unregister(struct net_device *dev, struct udp_tunnel_nic *utn)
list_for_each_entry(node, &info->shared->devices, list)
if (node->dev == dev)
break;
- if (list_entry_is_head(node, &info->shared->devices, list))
+ if (list_entry_is_head(node, &info->shared->devices, list)) {
+ udp_tunnel_nic_unlock(dev);
return;
+ }
list_del(&node->list);
kfree(node);
@@ -852,6 +888,7 @@ udp_tunnel_nic_unregister(struct net_device *dev, struct udp_tunnel_nic *utn)
if (first) {
udp_tunnel_drop_rx_info(dev);
utn->dev = first->dev;
+ udp_tunnel_nic_unlock(dev);
goto release_dev;
}
@@ -862,6 +899,7 @@ udp_tunnel_nic_unregister(struct net_device *dev, struct udp_tunnel_nic *utn)
* from the work which we will boot immediately.
*/
udp_tunnel_nic_flush(dev, utn);
+ udp_tunnel_nic_unlock(dev);
/* Wait for the work to be done using the state, netdev core will
* retry unregister until we give up our reference on this device.
@@ -910,12 +948,16 @@ udp_tunnel_nic_netdevice_event(struct notifier_block *unused,
return NOTIFY_DONE;
if (event == NETDEV_UP) {
+ udp_tunnel_nic_lock(dev);
WARN_ON(!udp_tunnel_nic_is_empty(dev, utn));
udp_tunnel_get_rx_info(dev);
+ udp_tunnel_nic_unlock(dev);
return NOTIFY_OK;
}
if (event == NETDEV_GOING_DOWN) {
+ udp_tunnel_nic_lock(dev);
udp_tunnel_nic_flush(dev, utn);
+ udp_tunnel_nic_unlock(dev);
return NOTIFY_OK;
}
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index c6fcdb60dfee..9fe7851029e2 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -852,7 +852,7 @@ static void addrconf_forward_change(struct net *net, __s32 newf)
struct inet6_dev *idev;
for_each_netdev(net, dev) {
- idev = __in6_dev_get(dev);
+ idev = __in6_dev_get_rtnl_net(dev);
if (idev) {
int changed = (!idev->cnf.forwarding) ^ (!newf);
@@ -865,13 +865,12 @@ static void addrconf_forward_change(struct net *net, __s32 newf)
static int addrconf_fixup_forwarding(const struct ctl_table *table, int *p, int newf)
{
- struct net *net;
+ struct net *net = (struct net *)table->extra2;
int old;
- if (!rtnl_trylock())
+ if (!rtnl_net_trylock(net))
return restart_syscall();
- net = (struct net *)table->extra2;
old = *p;
WRITE_ONCE(*p, newf);
@@ -881,7 +880,7 @@ static int addrconf_fixup_forwarding(const struct ctl_table *table, int *p, int
NETCONFA_FORWARDING,
NETCONFA_IFINDEX_DEFAULT,
net->ipv6.devconf_dflt);
- rtnl_unlock();
+ rtnl_net_unlock(net);
return 0;
}
@@ -903,11 +902,11 @@ static int addrconf_fixup_forwarding(const struct ctl_table *table, int *p, int
net->ipv6.devconf_all);
} else if ((!newf) ^ (!old))
dev_forward_change((struct inet6_dev *)table->extra1);
- rtnl_unlock();
+ rtnl_net_unlock(net);
if (newf)
rt6_purge_dflt_routers(net);
- return 1;
+ return 0;
}
static void addrconf_linkdown_change(struct net *net, __s32 newf)
@@ -916,7 +915,7 @@ static void addrconf_linkdown_change(struct net *net, __s32 newf)
struct inet6_dev *idev;
for_each_netdev(net, dev) {
- idev = __in6_dev_get(dev);
+ idev = __in6_dev_get_rtnl_net(dev);
if (idev) {
int changed = (!idev->cnf.ignore_routes_with_linkdown) ^ (!newf);
@@ -933,13 +932,12 @@ static void addrconf_linkdown_change(struct net *net, __s32 newf)
static int addrconf_fixup_linkdown(const struct ctl_table *table, int *p, int newf)
{
- struct net *net;
+ struct net *net = (struct net *)table->extra2;
int old;
- if (!rtnl_trylock())
+ if (!rtnl_net_trylock(net))
return restart_syscall();
- net = (struct net *)table->extra2;
old = *p;
WRITE_ONCE(*p, newf);
@@ -950,11 +948,7 @@ static int addrconf_fixup_linkdown(const struct ctl_table *table, int *p, int ne
NETCONFA_IGNORE_ROUTES_WITH_LINKDOWN,
NETCONFA_IFINDEX_DEFAULT,
net->ipv6.devconf_dflt);
- rtnl_unlock();
- return 0;
- }
-
- if (p == &net->ipv6.devconf_all->ignore_routes_with_linkdown) {
+ } else if (p == &net->ipv6.devconf_all->ignore_routes_with_linkdown) {
WRITE_ONCE(net->ipv6.devconf_dflt->ignore_routes_with_linkdown, newf);
addrconf_linkdown_change(net, newf);
if ((!newf) ^ (!old))
@@ -963,10 +957,21 @@ static int addrconf_fixup_linkdown(const struct ctl_table *table, int *p, int ne
NETCONFA_IGNORE_ROUTES_WITH_LINKDOWN,
NETCONFA_IFINDEX_ALL,
net->ipv6.devconf_all);
+ } else {
+ if (!newf ^ !old) {
+ struct inet6_dev *idev = table->extra1;
+
+ inet6_netconf_notify_devconf(net,
+ RTM_NEWNETCONF,
+ NETCONFA_IGNORE_ROUTES_WITH_LINKDOWN,
+ idev->dev->ifindex,
+ &idev->cnf);
+ }
}
- rtnl_unlock();
- return 1;
+ rtnl_net_unlock(net);
+
+ return 0;
}
#endif
@@ -2162,16 +2167,18 @@ void addrconf_dad_failure(struct sk_buff *skb, struct inet6_ifaddr *ifp)
struct net *net = dev_net(idev->dev);
int max_addresses;
- if (addrconf_dad_end(ifp)) {
+ spin_lock_bh(&ifp->lock);
+
+ if (ifp->state != INET6_IFADDR_STATE_DAD) {
+ spin_unlock_bh(&ifp->lock);
in6_ifa_put(ifp);
return;
}
+ ifp->state = INET6_IFADDR_STATE_POSTDAD;
net_info_ratelimited("%s: IPv6 duplicate address %pI6c used by %pM detected!\n",
ifp->idev->dev->name, &ifp->addr, eth_hdr(skb)->h_source);
- spin_lock_bh(&ifp->lock);
-
if (ifp->flags & IFA_F_STABLE_PRIVACY) {
struct in6_addr new_addr;
struct inet6_ifaddr *ifp2;
@@ -2219,6 +2226,11 @@ void addrconf_dad_failure(struct sk_buff *skb, struct inet6_ifaddr *ifp)
in6_ifa_put(ifp2);
lock_errdad:
spin_lock_bh(&ifp->lock);
+ if (ifp->state != INET6_IFADDR_STATE_POSTDAD) {
+ spin_unlock_bh(&ifp->lock);
+ in6_ifa_put(ifp);
+ return;
+ }
}
errdad:
@@ -6355,6 +6367,8 @@ static int addrconf_sysctl_forward(const struct ctl_table *ctl, int write,
lctl.data = &val;
ret = proc_dointvec(&lctl, write, buffer, lenp, ppos);
+ if (ret)
+ return ret;
if (write)
ret = addrconf_fixup_forwarding(ctl, valp, val);
@@ -6397,7 +6411,7 @@ static void addrconf_disable_change(struct net *net, __s32 newf)
struct inet6_dev *idev;
for_each_netdev(net, dev) {
- idev = __in6_dev_get(dev);
+ idev = __in6_dev_get_rtnl_net(dev);
if (idev) {
int changed = (!idev->cnf.disable_ipv6) ^ (!newf);
@@ -6418,7 +6432,7 @@ static int addrconf_disable_ipv6(const struct ctl_table *table, int *p, int newf
return 0;
}
- if (!rtnl_trylock())
+ if (!rtnl_net_trylock(net))
return restart_syscall();
old = *p;
@@ -6427,10 +6441,11 @@ static int addrconf_disable_ipv6(const struct ctl_table *table, int *p, int newf
if (p == &net->ipv6.devconf_all->disable_ipv6) {
WRITE_ONCE(net->ipv6.devconf_dflt->disable_ipv6, newf);
addrconf_disable_change(net, newf);
- } else if ((!newf) ^ (!old))
+ } else if ((!newf) ^ (!old)) {
dev_disable_change((struct inet6_dev *)table->extra1);
+ }
- rtnl_unlock();
+ rtnl_net_unlock(net);
return 0;
}
@@ -6451,6 +6466,8 @@ static int addrconf_sysctl_disable(const struct ctl_table *ctl, int write,
lctl.data = &val;
ret = proc_dointvec(&lctl, write, buffer, lenp, ppos);
+ if (ret)
+ return ret;
if (write)
ret = addrconf_disable_ipv6(ctl, valp, val);
@@ -6473,20 +6490,20 @@ static int addrconf_sysctl_proxy_ndp(const struct ctl_table *ctl, int write,
if (write && old != new) {
struct net *net = ctl->extra2;
- if (!rtnl_trylock())
+ if (!rtnl_net_trylock(net))
return restart_syscall();
- if (valp == &net->ipv6.devconf_dflt->proxy_ndp)
+ if (valp == &net->ipv6.devconf_dflt->proxy_ndp) {
inet6_netconf_notify_devconf(net, RTM_NEWNETCONF,
NETCONFA_PROXY_NEIGH,
NETCONFA_IFINDEX_DEFAULT,
net->ipv6.devconf_dflt);
- else if (valp == &net->ipv6.devconf_all->proxy_ndp)
+ } else if (valp == &net->ipv6.devconf_all->proxy_ndp) {
inet6_netconf_notify_devconf(net, RTM_NEWNETCONF,
NETCONFA_PROXY_NEIGH,
NETCONFA_IFINDEX_ALL,
net->ipv6.devconf_all);
- else {
+ } else {
struct inet6_dev *idev = ctl->extra1;
inet6_netconf_notify_devconf(net, RTM_NEWNETCONF,
@@ -6494,7 +6511,7 @@ static int addrconf_sysctl_proxy_ndp(const struct ctl_table *ctl, int write,
idev->dev->ifindex,
&idev->cnf);
}
- rtnl_unlock();
+ rtnl_net_unlock(net);
}
return ret;
@@ -6514,7 +6531,7 @@ static int addrconf_sysctl_addr_gen_mode(const struct ctl_table *ctl, int write,
.mode = ctl->mode,
};
- if (!rtnl_trylock())
+ if (!rtnl_net_trylock(net))
return restart_syscall();
new_val = *((u32 *)ctl->data);
@@ -6544,7 +6561,7 @@ static int addrconf_sysctl_addr_gen_mode(const struct ctl_table *ctl, int write,
WRITE_ONCE(net->ipv6.devconf_dflt->addr_gen_mode, new_val);
for_each_netdev(net, dev) {
- idev = __in6_dev_get(dev);
+ idev = __in6_dev_get_rtnl_net(dev);
if (idev &&
idev->cnf.addr_gen_mode != new_val) {
WRITE_ONCE(idev->cnf.addr_gen_mode,
@@ -6558,7 +6575,7 @@ static int addrconf_sysctl_addr_gen_mode(const struct ctl_table *ctl, int write,
}
out:
- rtnl_unlock();
+ rtnl_net_unlock(net);
return ret;
}
@@ -6580,7 +6597,7 @@ static int addrconf_sysctl_stable_secret(const struct ctl_table *ctl, int write,
lctl.maxlen = IPV6_MAX_STRLEN;
lctl.data = str;
- if (!rtnl_trylock())
+ if (!rtnl_net_trylock(net))
return restart_syscall();
if (!write && !secret->initialized) {
@@ -6610,7 +6627,7 @@ static int addrconf_sysctl_stable_secret(const struct ctl_table *ctl, int write,
struct net_device *dev;
for_each_netdev(net, dev) {
- struct inet6_dev *idev = __in6_dev_get(dev);
+ struct inet6_dev *idev = __in6_dev_get_rtnl_net(dev);
if (idev) {
WRITE_ONCE(idev->cnf.addr_gen_mode,
@@ -6625,7 +6642,7 @@ static int addrconf_sysctl_stable_secret(const struct ctl_table *ctl, int write,
}
out:
- rtnl_unlock();
+ rtnl_net_unlock(net);
return err;
}
@@ -6649,6 +6666,8 @@ int addrconf_sysctl_ignore_routes_with_linkdown(const struct ctl_table *ctl,
lctl.data = &val;
ret = proc_dointvec(&lctl, write, buffer, lenp, ppos);
+ if (ret)
+ return ret;
if (write)
ret = addrconf_fixup_linkdown(ctl, valp, val);
@@ -6709,7 +6728,7 @@ int addrconf_disable_policy(const struct ctl_table *ctl, int *valp, int val)
return 0;
}
- if (!rtnl_trylock())
+ if (!rtnl_net_trylock(net))
return restart_syscall();
WRITE_ONCE(*valp, val);
@@ -6718,7 +6737,7 @@ int addrconf_disable_policy(const struct ctl_table *ctl, int *valp, int val)
struct net_device *dev;
for_each_netdev(net, dev) {
- idev = __in6_dev_get(dev);
+ idev = __in6_dev_get_rtnl_net(dev);
if (idev)
addrconf_disable_policy_idev(idev, val);
}
@@ -6727,7 +6746,7 @@ int addrconf_disable_policy(const struct ctl_table *ctl, int *valp, int val)
addrconf_disable_policy_idev(idev, val);
}
- rtnl_unlock();
+ rtnl_net_unlock(net);
return 0;
}
@@ -6743,6 +6762,8 @@ static int addrconf_sysctl_disable_policy(const struct ctl_table *ctl, int write
lctl = *ctl;
lctl.data = &val;
ret = proc_dointvec(&lctl, write, buffer, lenp, ppos);
+ if (ret)
+ return ret;
if (write && (*valp != val))
ret = addrconf_disable_policy(ctl, valp, val);
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 2072c788c912..5b94cb8df8d8 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -629,12 +629,12 @@ static int inet6_dump_fib(struct sk_buff *skb, struct netlink_callback *cb)
};
const struct nlmsghdr *nlh = cb->nlh;
struct net *net = sock_net(skb->sk);
- unsigned int e = 0, s_e;
struct hlist_head *head;
struct fib6_walker *w;
struct fib6_table *tb;
unsigned int h, s_h;
int err = 0;
+ u32 s_id;
rcu_read_lock();
if (cb->strict_check) {
@@ -694,23 +694,22 @@ static int inet6_dump_fib(struct sk_buff *skb, struct netlink_callback *cb)
}
s_h = cb->args[0];
- s_e = cb->args[1];
+ s_id = cb->args[1];
- for (h = s_h; h < FIB6_TABLE_HASHSZ; h++, s_e = 0) {
- e = 0;
+ for (h = s_h; h < FIB6_TABLE_HASHSZ; h++, s_id = 0) {
head = &net->ipv6.fib_table_hash[h];
hlist_for_each_entry_rcu(tb, head, tb6_hlist) {
- if (e < s_e)
- goto next;
+ if (s_id && tb->tb6_id != s_id)
+ continue;
+
+ s_id = 0;
+ cb->args[1] = tb->tb6_id;
err = fib6_dump_table(tb, skb, cb);
if (err != 0)
goto out;
-next:
- e++;
}
}
out:
- cb->args[1] = e;
cb->args[0] = h;
unlock:
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index 93543fcb4c92..f1dc1244ade3 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -2083,6 +2083,9 @@ static int ip6gre_changelink(struct net_device *dev, struct nlattr *tb[],
struct ip6gre_net *ign = net_generic(t->net, ip6gre_net_id);
struct __ip6_tnl_parm p;
+ if (!rtnl_dev_link_net_capable(dev, t->net))
+ return -EPERM;
+
t = ip6gre_changelink_common(dev, tb, data, &p, extack);
if (IS_ERR(t))
return PTR_ERR(t);
@@ -2300,6 +2303,9 @@ static int ip6erspan_changelink(struct net_device *dev, struct nlattr *tb[],
struct __ip6_tnl_parm p;
struct ip6gre_net *ign;
+ if (!rtnl_dev_link_net_capable(dev, t->net))
+ return -EPERM;
+
ign = net_generic(t->net, ip6gre_net_id);
t = ip6gre_changelink_common(dev, tb, data, &p, extack);
if (IS_ERR(t))
diff --git a/net/ipv6/ip6_vti.c b/net/ipv6/ip6_vti.c
index e0e6e67a25e0..e8c60b7fb012 100644
--- a/net/ipv6/ip6_vti.c
+++ b/net/ipv6/ip6_vti.c
@@ -1044,6 +1044,9 @@ static int vti6_changelink(struct net_device *dev, struct nlattr *tb[],
struct __ip6_tnl_parm p;
struct vti6_net *ip6n;
+ if (!rtnl_dev_link_net_capable(dev, net))
+ return -EPERM;
+
ip6n = net_generic(net, vti6_net_id);
if (dev == ip6n->fb_tnl_dev)
return -EINVAL;
diff --git a/net/ipv6/mcast.c b/net/ipv6/mcast.c
index b769e856a068..92f321635e4e 100644
--- a/net/ipv6/mcast.c
+++ b/net/ipv6/mcast.c
@@ -105,9 +105,9 @@ static int __ipv6_dev_mc_inc(struct net_device *dev,
int sysctl_mld_max_msf __read_mostly = IPV6_MLD_MAX_MSF;
int sysctl_mld_qrv __read_mostly = MLD_QRV_DEFAULT;
-/*
- * socket join on multicast group
- */
+#define mc_assert_locked(idev) \
+ lockdep_assert_held(&(idev)->mc_lock)
+
#define mc_dereference(e, idev) \
rcu_dereference_protected(e, lockdep_is_held(&(idev)->mc_lock))
@@ -166,6 +166,9 @@ static int unsolicited_report_interval(struct inet6_dev *idev)
return iv > 0 ? iv : 1;
}
+/*
+ * socket join on multicast group
+ */
static int __ipv6_sock_mc_join(struct sock *sk, int ifindex,
const struct in6_addr *addr, unsigned int mode)
{
@@ -665,12 +668,13 @@ bool inet6_mc_check(const struct sock *sk, const struct in6_addr *mc_addr,
return rv;
}
-/* called with mc_lock */
static void igmp6_group_added(struct ifmcaddr6 *mc)
{
struct net_device *dev = mc->idev->dev;
char buf[MAX_ADDR_LEN];
+ mc_assert_locked(mc->idev);
+
if (IPV6_ADDR_MC_SCOPE(&mc->mca_addr) <
IPV6_ADDR_SCOPE_LINKLOCAL)
return;
@@ -700,12 +704,13 @@ static void igmp6_group_added(struct ifmcaddr6 *mc)
mld_ifc_event(mc->idev);
}
-/* called with mc_lock */
static void igmp6_group_dropped(struct ifmcaddr6 *mc)
{
struct net_device *dev = mc->idev->dev;
char buf[MAX_ADDR_LEN];
+ mc_assert_locked(mc->idev);
+
if (IPV6_ADDR_MC_SCOPE(&mc->mca_addr) <
IPV6_ADDR_SCOPE_LINKLOCAL)
return;
@@ -726,14 +731,13 @@ static void igmp6_group_dropped(struct ifmcaddr6 *mc)
refcount_dec(&mc->mca_refcnt);
}
-/*
- * deleted ifmcaddr6 manipulation
- * called with mc_lock
- */
+/* deleted ifmcaddr6 manipulation */
static void mld_add_delrec(struct inet6_dev *idev, struct ifmcaddr6 *im)
{
struct ifmcaddr6 *pmc;
+ mc_assert_locked(idev);
+
/* this is an "ifmcaddr6" for convenience; only the fields below
* are actually used. In particular, the refcnt and users are not
* used for management of the delete list. Using the same structure
@@ -767,13 +771,14 @@ static void mld_add_delrec(struct inet6_dev *idev, struct ifmcaddr6 *im)
rcu_assign_pointer(idev->mc_tomb, pmc);
}
-/* called with mc_lock */
static void mld_del_delrec(struct inet6_dev *idev, struct ifmcaddr6 *im)
{
struct ip6_sf_list *psf, *sources, *tomb;
struct in6_addr *pmca = &im->mca_addr;
struct ifmcaddr6 *pmc, *pmc_prev;
+ mc_assert_locked(idev);
+
pmc_prev = NULL;
for_each_mc_tomb(idev, pmc) {
if (ipv6_addr_equal(&pmc->mca_addr, pmca))
@@ -810,11 +815,12 @@ static void mld_del_delrec(struct inet6_dev *idev, struct ifmcaddr6 *im)
}
}
-/* called with mc_lock */
static void mld_clear_delrec(struct inet6_dev *idev)
{
struct ifmcaddr6 *pmc, *nextpmc;
+ mc_assert_locked(idev);
+
pmc = mc_dereference(idev->mc_tomb, idev);
RCU_INIT_POINTER(idev->mc_tomb, NULL);
@@ -871,13 +877,14 @@ static void ma_put(struct ifmcaddr6 *mc)
}
}
-/* called with mc_lock */
static struct ifmcaddr6 *mca_alloc(struct inet6_dev *idev,
const struct in6_addr *addr,
unsigned int mode)
{
struct ifmcaddr6 *mc;
+ mc_assert_locked(idev);
+
mc = kzalloc(sizeof(*mc), GFP_KERNEL);
if (!mc)
return NULL;
@@ -1047,48 +1054,59 @@ bool ipv6_chk_mcast_addr(struct net_device *dev, const struct in6_addr *group,
return rv;
}
-/* called with mc_lock */
static void mld_gq_start_work(struct inet6_dev *idev)
{
unsigned long tv = get_random_u32_below(idev->mc_maxdelay);
+ mc_assert_locked(idev);
+
idev->mc_gq_running = 1;
- if (!mod_delayed_work(mld_wq, &idev->mc_gq_work, tv + 2))
- in6_dev_hold(idev);
+ if (in6_dev_hold_safe(idev)) {
+ if (mod_delayed_work(mld_wq, &idev->mc_gq_work, tv + 2))
+ in6_dev_put(idev);
+ }
}
-/* called with mc_lock */
static void mld_gq_stop_work(struct inet6_dev *idev)
{
+ mc_assert_locked(idev);
+
idev->mc_gq_running = 0;
if (cancel_delayed_work(&idev->mc_gq_work))
__in6_dev_put(idev);
}
-/* called with mc_lock */
static void mld_ifc_start_work(struct inet6_dev *idev, unsigned long delay)
{
unsigned long tv = get_random_u32_below(delay);
- if (!mod_delayed_work(mld_wq, &idev->mc_ifc_work, tv + 2))
- in6_dev_hold(idev);
+ mc_assert_locked(idev);
+
+ if (in6_dev_hold_safe(idev)) {
+ if (mod_delayed_work(mld_wq, &idev->mc_ifc_work, tv + 2))
+ in6_dev_put(idev);
+ }
}
-/* called with mc_lock */
static void mld_ifc_stop_work(struct inet6_dev *idev)
{
+ mc_assert_locked(idev);
+
idev->mc_ifc_count = 0;
if (cancel_delayed_work(&idev->mc_ifc_work))
__in6_dev_put(idev);
}
-/* called with mc_lock */
static void mld_dad_start_work(struct inet6_dev *idev, unsigned long delay)
{
unsigned long tv = get_random_u32_below(delay);
- if (!mod_delayed_work(mld_wq, &idev->mc_dad_work, tv + 2))
- in6_dev_hold(idev);
+ mc_assert_locked(idev);
+
+ if (in6_dev_hold_safe(idev)) {
+ if (mod_delayed_work(mld_wq, &idev->mc_dad_work, tv + 2))
+ in6_dev_put(idev);
+ }
}
static void mld_dad_stop_work(struct inet6_dev *idev)
@@ -1111,14 +1129,13 @@ static void mld_report_stop_work(struct inet6_dev *idev)
__in6_dev_put(idev);
}
-/*
- * IGMP handling (alias multicast ICMPv6 messages)
- * called with mc_lock
- */
+/* IGMP handling (alias multicast ICMPv6 messages) */
static void igmp6_group_queried(struct ifmcaddr6 *ma, unsigned long resptime)
{
unsigned long delay = resptime;
+ mc_assert_locked(ma->idev);
+
/* Do not start work for these addresses */
if (ipv6_addr_is_ll_all_nodes(&ma->mca_addr) ||
IPV6_ADDR_MC_SCOPE(&ma->mca_addr) < IPV6_ADDR_SCOPE_LINKLOCAL)
@@ -1137,15 +1154,15 @@ static void igmp6_group_queried(struct ifmcaddr6 *ma, unsigned long resptime)
ma->mca_flags |= MAF_TIMER_RUNNING;
}
-/* mark EXCLUDE-mode sources
- * called with mc_lock
- */
+/* mark EXCLUDE-mode sources */
static bool mld_xmarksources(struct ifmcaddr6 *pmc, int nsrcs,
const struct in6_addr *srcs)
{
struct ip6_sf_list *psf;
int i, scount;
+ mc_assert_locked(pmc->idev);
+
scount = 0;
for_each_psf_mclock(pmc, psf) {
if (scount == nsrcs)
@@ -1168,13 +1185,14 @@ static bool mld_xmarksources(struct ifmcaddr6 *pmc, int nsrcs,
return true;
}
-/* called with mc_lock */
static bool mld_marksources(struct ifmcaddr6 *pmc, int nsrcs,
const struct in6_addr *srcs)
{
struct ip6_sf_list *psf;
int i, scount;
+ mc_assert_locked(pmc->idev);
+
if (pmc->mca_sfmode == MCAST_EXCLUDE)
return mld_xmarksources(pmc, nsrcs, srcs);
@@ -1374,18 +1392,23 @@ static void mld_process_v2(struct inet6_dev *idev, struct mld2_query *mld,
void igmp6_event_query(struct sk_buff *skb)
{
struct inet6_dev *idev = __in6_dev_get(skb->dev);
+ bool put = false;
if (!idev || idev->dead)
goto out;
spin_lock_bh(&idev->mc_query_lock);
- if (skb_queue_len(&idev->mc_query_queue) < MLD_MAX_SKBS) {
+ if (skb_queue_len(&idev->mc_query_queue) < MLD_MAX_SKBS &&
+ in6_dev_hold_safe(idev)) {
__skb_queue_tail(&idev->mc_query_queue, skb);
- if (!mod_delayed_work(mld_wq, &idev->mc_query_work, 0))
- in6_dev_hold(idev);
+ if (mod_delayed_work(mld_wq, &idev->mc_query_work, 0))
+ put = true;
skb = NULL;
}
spin_unlock_bh(&idev->mc_query_lock);
+
+ if (put)
+ in6_dev_put(idev);
out:
kfree_skb(skb);
}
@@ -1542,18 +1565,23 @@ static void mld_query_work(struct work_struct *work)
void igmp6_event_report(struct sk_buff *skb)
{
struct inet6_dev *idev = __in6_dev_get(skb->dev);
+ bool put = false;
if (!idev || idev->dead)
goto out;
spin_lock_bh(&idev->mc_report_lock);
- if (skb_queue_len(&idev->mc_report_queue) < MLD_MAX_SKBS) {
+ if (skb_queue_len(&idev->mc_report_queue) < MLD_MAX_SKBS &&
+ in6_dev_hold_safe(idev)) {
__skb_queue_tail(&idev->mc_report_queue, skb);
- if (!mod_delayed_work(mld_wq, &idev->mc_report_work, 0))
- in6_dev_hold(idev);
+ if (mod_delayed_work(mld_wq, &idev->mc_report_work, 0))
+ put = true;
skb = NULL;
}
spin_unlock_bh(&idev->mc_report_lock);
+
+ if (put)
+ in6_dev_put(idev);
out:
kfree_skb(skb);
}
@@ -1869,7 +1897,6 @@ static struct sk_buff *add_grhead(struct sk_buff *skb, struct ifmcaddr6 *pmc,
#define AVAILABLE(skb) ((skb) ? skb_availroom(skb) : 0)
-/* called with mc_lock */
static struct sk_buff *add_grec(struct sk_buff *skb, struct ifmcaddr6 *pmc,
int type, int gdeleted, int sdeleted,
int crsend)
@@ -1883,6 +1910,8 @@ static struct sk_buff *add_grec(struct sk_buff *skb, struct ifmcaddr6 *pmc,
struct mld2_report *pmr;
unsigned int mtu;
+ mc_assert_locked(idev);
+
if (pmc->mca_flags & MAF_NOREPORT)
return skb;
@@ -2001,12 +2030,13 @@ static struct sk_buff *add_grec(struct sk_buff *skb, struct ifmcaddr6 *pmc,
return skb;
}
-/* called with mc_lock */
static void mld_send_report(struct inet6_dev *idev, struct ifmcaddr6 *pmc)
{
struct sk_buff *skb = NULL;
int type;
+ mc_assert_locked(idev);
+
if (!pmc) {
for_each_mc_mclock(idev, pmc) {
if (pmc->mca_flags & MAF_NOREPORT)
@@ -2028,10 +2058,7 @@ static void mld_send_report(struct inet6_dev *idev, struct ifmcaddr6 *pmc)
mld_sendpack(skb);
}
-/*
- * remove zero-count source records from a source filter list
- * called with mc_lock
- */
+/* remove zero-count source records from a source filter list */
static void mld_clear_zeros(struct ip6_sf_list __rcu **ppsf, struct inet6_dev *idev)
{
struct ip6_sf_list *psf_prev, *psf_next, *psf;
@@ -2055,7 +2082,6 @@ static void mld_clear_zeros(struct ip6_sf_list __rcu **ppsf, struct inet6_dev *i
}
}
-/* called with mc_lock */
static void mld_send_cr(struct inet6_dev *idev)
{
struct ifmcaddr6 *pmc, *pmc_prev, *pmc_next;
@@ -2219,13 +2245,14 @@ static void igmp6_send(struct in6_addr *addr, struct net_device *dev, int type)
goto out;
}
-/* called with mc_lock */
static void mld_send_initial_cr(struct inet6_dev *idev)
{
- struct sk_buff *skb;
struct ifmcaddr6 *pmc;
+ struct sk_buff *skb;
int type;
+ mc_assert_locked(idev);
+
if (mld_in_v1_mode(idev))
return;
@@ -2272,13 +2299,14 @@ static void mld_dad_work(struct work_struct *work)
in6_dev_put(idev);
}
-/* called with mc_lock */
static int ip6_mc_del1_src(struct ifmcaddr6 *pmc, int sfmode,
- const struct in6_addr *psfsrc)
+ const struct in6_addr *psfsrc)
{
struct ip6_sf_list *psf, *psf_prev;
int rv = 0;
+ mc_assert_locked(pmc->idev);
+
psf_prev = NULL;
for_each_psf_mclock(pmc, psf) {
if (ipv6_addr_equal(&psf->sf_addr, psfsrc))
@@ -2315,7 +2343,6 @@ static int ip6_mc_del1_src(struct ifmcaddr6 *pmc, int sfmode,
return rv;
}
-/* called with mc_lock */
static int ip6_mc_del_src(struct inet6_dev *idev, const struct in6_addr *pmca,
int sfmode, int sfcount, const struct in6_addr *psfsrc,
int delta)
@@ -2327,6 +2354,8 @@ static int ip6_mc_del_src(struct inet6_dev *idev, const struct in6_addr *pmca,
if (!idev)
return -ENODEV;
+ mc_assert_locked(idev);
+
for_each_mc_mclock(idev, pmc) {
if (ipv6_addr_equal(pmca, &pmc->mca_addr))
break;
@@ -2368,15 +2397,14 @@ static int ip6_mc_del_src(struct inet6_dev *idev, const struct in6_addr *pmca,
return err;
}
-/*
- * Add multicast single-source filter to the interface list
- * called with mc_lock
- */
+/* Add multicast single-source filter to the interface list */
static int ip6_mc_add1_src(struct ifmcaddr6 *pmc, int sfmode,
- const struct in6_addr *psfsrc)
+ const struct in6_addr *psfsrc)
{
struct ip6_sf_list *psf, *psf_prev;
+ mc_assert_locked(pmc->idev);
+
psf_prev = NULL;
for_each_psf_mclock(pmc, psf) {
if (ipv6_addr_equal(&psf->sf_addr, psfsrc))
@@ -2399,11 +2427,12 @@ static int ip6_mc_add1_src(struct ifmcaddr6 *pmc, int sfmode,
return 0;
}
-/* called with mc_lock */
static void sf_markstate(struct ifmcaddr6 *pmc)
{
- struct ip6_sf_list *psf;
int mca_xcount = pmc->mca_sfcount[MCAST_EXCLUDE];
+ struct ip6_sf_list *psf;
+
+ mc_assert_locked(pmc->idev);
for_each_psf_mclock(pmc, psf) {
if (pmc->mca_sfcount[MCAST_EXCLUDE]) {
@@ -2416,14 +2445,15 @@ static void sf_markstate(struct ifmcaddr6 *pmc)
}
}
-/* called with mc_lock */
static int sf_setstate(struct ifmcaddr6 *pmc)
{
- struct ip6_sf_list *psf, *dpsf;
int mca_xcount = pmc->mca_sfcount[MCAST_EXCLUDE];
+ struct ip6_sf_list *psf, *dpsf;
int qrv = pmc->idev->mc_qrv;
int new_in, rv;
+ mc_assert_locked(pmc->idev);
+
rv = 0;
for_each_psf_mclock(pmc, psf) {
if (pmc->mca_sfcount[MCAST_EXCLUDE]) {
@@ -2482,10 +2512,7 @@ static int sf_setstate(struct ifmcaddr6 *pmc)
return rv;
}
-/*
- * Add multicast source filter list to the interface list
- * called with mc_lock
- */
+/* Add multicast source filter list to the interface list */
static int ip6_mc_add_src(struct inet6_dev *idev, const struct in6_addr *pmca,
int sfmode, int sfcount, const struct in6_addr *psfsrc,
int delta)
@@ -2497,6 +2524,8 @@ static int ip6_mc_add_src(struct inet6_dev *idev, const struct in6_addr *pmca,
if (!idev)
return -ENODEV;
+ mc_assert_locked(idev);
+
for_each_mc_mclock(idev, pmc) {
if (ipv6_addr_equal(pmca, &pmc->mca_addr))
break;
@@ -2542,11 +2571,12 @@ static int ip6_mc_add_src(struct inet6_dev *idev, const struct in6_addr *pmca,
return err;
}
-/* called with mc_lock */
static void ip6_mc_clear_src(struct ifmcaddr6 *pmc)
{
struct ip6_sf_list *psf, *nextpsf;
+ mc_assert_locked(pmc->idev);
+
for (psf = mc_dereference(pmc->mca_tomb, pmc->idev);
psf;
psf = nextpsf) {
@@ -2566,11 +2596,12 @@ static void ip6_mc_clear_src(struct ifmcaddr6 *pmc)
pmc->mca_sfcount[MCAST_EXCLUDE] = 1;
}
-/* called with mc_lock */
static void igmp6_join_group(struct ifmcaddr6 *ma)
{
unsigned long delay;
+ mc_assert_locked(ma->idev);
+
if (ma->mca_flags & MAF_NOREPORT)
return;
@@ -2617,9 +2648,10 @@ static int ip6_mc_leave_src(struct sock *sk, struct ipv6_mc_socklist *iml,
return err;
}
-/* called with mc_lock */
static void igmp6_leave_group(struct ifmcaddr6 *ma)
{
+ mc_assert_locked(ma->idev);
+
if (mld_in_v1_mode(ma->idev)) {
if (ma->mca_flags & MAF_LAST_REPORTER) {
igmp6_send(&ma->mca_addr, ma->idev->dev,
@@ -2664,9 +2696,10 @@ static void mld_ifc_work(struct work_struct *work)
in6_dev_put(idev);
}
-/* called with mc_lock */
static void mld_ifc_event(struct inet6_dev *idev)
{
+ mc_assert_locked(idev);
+
if (mld_in_v1_mode(idev))
return;
diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
index 64bd0cbb67d7..73e2e457e30b 100644
--- a/net/ipv6/ndisc.c
+++ b/net/ipv6/ndisc.c
@@ -976,10 +976,8 @@ static enum skb_drop_reason ndisc_recv_ns(struct sk_buff *skb)
return reason;
}
-static int accept_untracked_na(struct net_device *dev, struct in6_addr *saddr)
+static int accept_untracked_na(struct inet6_dev *idev, struct in6_addr *saddr)
{
- struct inet6_dev *idev = __in6_dev_get(dev);
-
switch (READ_ONCE(idev->cnf.accept_untracked_na)) {
case 0: /* Don't accept untracked na (absent in neighbor cache) */
return 0;
@@ -989,7 +987,7 @@ static int accept_untracked_na(struct net_device *dev, struct in6_addr *saddr)
* same subnet as an address configured on the interface that
* received the na
*/
- return !!ipv6_chk_prefix(saddr, dev);
+ return !!ipv6_chk_prefix(saddr, idev->dev);
default:
return 0;
}
@@ -1088,7 +1086,7 @@ static enum skb_drop_reason ndisc_recv_na(struct sk_buff *skb)
*/
new_state = msg->icmph.icmp6_solicited ? NUD_REACHABLE : NUD_STALE;
if (!neigh && lladdr && idev && READ_ONCE(idev->cnf.forwarding)) {
- if (accept_untracked_na(dev, saddr)) {
+ if (accept_untracked_na(idev, saddr)) {
neigh = neigh_create(&nd_tbl, &msg->target, dev);
new_state = NUD_STALE;
}
diff --git a/net/ipv6/netfilter.c b/net/ipv6/netfilter.c
index 4541836ee3da..2fd2c3e315ee 100644
--- a/net/ipv6/netfilter.c
+++ b/net/ipv6/netfilter.c
@@ -131,7 +131,7 @@ int br_ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb,
ktime_t tstamp = skb->tstamp;
struct ip6_frag_state state;
u8 *prevhdr, nexthdr = 0;
- unsigned int mtu, hlen;
+ unsigned int mtu, hlen, nexthdr_offset;
int hroom, err = 0;
__be32 frag_id;
@@ -140,6 +140,7 @@ int br_ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb,
goto blackhole;
hlen = err;
nexthdr = *prevhdr;
+ nexthdr_offset = prevhdr - skb_network_header(skb);
mtu = skb->dev->mtu;
if (frag_max_size > mtu ||
@@ -158,6 +159,7 @@ int br_ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb,
(err = skb_checksum_help(skb)))
goto blackhole;
+ prevhdr = skb_network_header(skb) + nexthdr_offset;
hroom = LL_RESERVED_SPACE(skb->dev);
if (skb_has_frag_list(skb)) {
unsigned int first_len = skb_pagelen(skb);
diff --git a/net/ipv6/netfilter/ip6t_ah.c b/net/ipv6/netfilter/ip6t_ah.c
index 70da2f2ce064..1258783ed876 100644
--- a/net/ipv6/netfilter/ip6t_ah.c
+++ b/net/ipv6/netfilter/ip6t_ah.c
@@ -56,6 +56,11 @@ static bool ah_mt6(const struct sk_buff *skb, struct xt_action_param *par)
}
hdrlen = ipv6_authlen(ah);
+ if (skb->len - ptr < hdrlen) {
+ /* Packet smaller than its length field */
+ par->hotdrop = true;
+ return false;
+ }
pr_debug("IPv6 AH LEN %u %u ", hdrlen, ah->hdrlen);
pr_debug("RES %04X ", ah->reserved);
diff --git a/net/ipv6/netfilter/ip6t_hbh.c b/net/ipv6/netfilter/ip6t_hbh.c
index 450dd53846a2..6d1a5d2026a6 100644
--- a/net/ipv6/netfilter/ip6t_hbh.c
+++ b/net/ipv6/netfilter/ip6t_hbh.c
@@ -75,6 +75,7 @@ hbh_mt6(const struct sk_buff *skb, struct xt_action_param *par)
hdrlen = ipv6_optlen(oh);
if (skb->len - ptr < hdrlen) {
/* Packet smaller than it's length field */
+ par->hotdrop = true;
return false;
}
diff --git a/net/ipv6/netfilter/ip6t_rt.c b/net/ipv6/netfilter/ip6t_rt.c
index 5561bd9cea81..278b52752f36 100644
--- a/net/ipv6/netfilter/ip6t_rt.c
+++ b/net/ipv6/netfilter/ip6t_rt.c
@@ -56,7 +56,8 @@ static bool rt_mt6(const struct sk_buff *skb, struct xt_action_param *par)
hdrlen = ipv6_optlen(rh);
if (skb->len - ptr < hdrlen) {
- /* Pcket smaller than its length field */
+ /* Packet smaller than its length field */
+ par->hotdrop = true;
return false;
}
diff --git a/net/ipv6/netfilter/nf_conntrack_reasm.c b/net/ipv6/netfilter/nf_conntrack_reasm.c
index bd3058d10a4b..fbcc8af7435c 100644
--- a/net/ipv6/netfilter/nf_conntrack_reasm.c
+++ b/net/ipv6/netfilter/nf_conntrack_reasm.c
@@ -345,7 +345,8 @@ static int nf_ct_frag6_reasm(struct frag_queue *fq, struct sk_buff *skb,
skb_network_header(skb)[fq->nhoffset] = skb_transport_header(skb)[0];
memmove(skb->head + sizeof(struct frag_hdr), skb->head,
(skb->data - skb->head) - sizeof(struct frag_hdr));
- skb->mac_header += sizeof(struct frag_hdr);
+ if (skb_mac_header_was_set(skb))
+ skb->mac_header += sizeof(struct frag_hdr);
skb->network_header += sizeof(struct frag_hdr);
skb_reset_transport_header(skb);
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 9e7470e81544..2ce05303e403 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -3261,11 +3261,11 @@ static unsigned int ip6_default_advmss(const struct dst_entry *dst)
/*
* Maximal non-jumbo IPv6 payload is IPV6_MAXPLEN and
* corresponding MSS is IPV6_MAXPLEN - tcp_header_size.
- * IPV6_MAXPLEN is also valid and means: "any MSS,
- * rely only on pmtu discovery"
+ * Limit the default MSS to GSO_BY_FRAGS - 1 to avoid
+ * collision with the GSO_BY_FRAGS magic value (0xFFFF).
*/
if (mtu > IPV6_MAXPLEN - sizeof(struct tcphdr))
- mtu = IPV6_MAXPLEN;
+ mtu = min_t(unsigned int, IPV6_MAXPLEN, GSO_BY_FRAGS - 1);
return mtu;
}
@@ -4992,6 +4992,9 @@ static int fib6_nh_mtu_change(struct fib6_nh *nh, void *_arg)
struct inet6_dev *idev = __in6_dev_get(arg->dev);
u32 mtu = f6i->fib6_pmtu;
+ if (!idev)
+ return 0;
+
if (mtu >= arg->mtu ||
(mtu < arg->mtu && mtu == idev->cnf.mtu6))
fib6_metric_set(f6i, RTAX_MTU, arg->mtu);
diff --git a/net/ipv6/seg6.c b/net/ipv6/seg6.c
index 180da19c148c..5b57ea4fdda9 100644
--- a/net/ipv6/seg6.c
+++ b/net/ipv6/seg6.c
@@ -29,6 +29,9 @@ bool seg6_validate_srh(struct ipv6_sr_hdr *srh, int len, bool reduced)
int max_last_entry;
int trailing;
+ if (len < sizeof(*srh))
+ return false;
+
if (srh->type != IPV6_SRCRT_TYPE_4)
return false;
diff --git a/net/ipv6/sit.c b/net/ipv6/sit.c
index 5c1982358aca..78e685011382 100644
--- a/net/ipv6/sit.c
+++ b/net/ipv6/sit.c
@@ -1615,6 +1615,9 @@ static int ipip6_changelink(struct net_device *dev, struct nlattr *tb[],
__u32 fwmark = t->fwmark;
int err;
+ if (!rtnl_dev_link_net_capable(dev, net))
+ return -EPERM;
+
if (dev == sitn->fb_tunnel_dev)
return -EINVAL;
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index 4e34311b27b1..5736f06d8143 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -1299,8 +1299,8 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
psock->save_write_space = csk->sk_write_space;
psock->save_state_change = csk->sk_state_change;
csk->sk_user_data = psock;
- csk->sk_data_ready = psock_data_ready;
- csk->sk_write_space = psock_write_space;
+ WRITE_ONCE(csk->sk_data_ready, psock_data_ready);
+ WRITE_ONCE(csk->sk_write_space, psock_write_space);
csk->sk_state_change = psock_state_change;
write_unlock_bh(&csk->sk_callback_lock);
@@ -1376,8 +1376,8 @@ static void kcm_unattach(struct kcm_psock *psock)
*/
write_lock_bh(&csk->sk_callback_lock);
csk->sk_user_data = NULL;
- csk->sk_data_ready = psock->save_data_ready;
- csk->sk_write_space = psock->save_write_space;
+ WRITE_ONCE(csk->sk_data_ready, psock->save_data_ready);
+ WRITE_ONCE(csk->sk_write_space, psock->save_write_space);
csk->sk_state_change = psock->save_state_change;
strp_stop(&psock->strp);
diff --git a/net/llc/af_llc.c b/net/llc/af_llc.c
index cc77ec5769d8..7c90fecfd4bf 100644
--- a/net/llc/af_llc.c
+++ b/net/llc/af_llc.c
@@ -317,6 +317,7 @@ static int llc_ui_autobind(struct socket *sock, struct sockaddr_llc *addr)
/* assign new connection to its SAP */
llc_sap_add_socket(sap, sk);
sock_reset_flag(sk, SOCK_ZAPPED);
+ llc_sap_put(sap);
rc = 0;
out:
dev_put(dev);
diff --git a/net/llc/llc_conn.c b/net/llc/llc_conn.c
index afc6974eafda..dfc451e773c3 100644
--- a/net/llc/llc_conn.c
+++ b/net/llc/llc_conn.c
@@ -773,7 +773,6 @@ static struct sock *llc_create_incoming_sock(struct sock *sk,
newllc->dev = dev;
dev_hold(dev);
llc_sap_add_socket(llc->sap, newsk);
- llc_sap_hold(llc->sap);
out:
return newsk;
}
diff --git a/net/mac80211/main.c b/net/mac80211/main.c
index c745de0aae77..84cc2a21a9a5 100644
--- a/net/mac80211/main.c
+++ b/net/mac80211/main.c
@@ -1568,7 +1568,7 @@ int ieee80211_register_hw(struct ieee80211_hw *hw)
sband = kmemdup(sband, sizeof(*sband), GFP_KERNEL);
if (!sband) {
result = -ENOMEM;
- goto fail_rate;
+ goto fail_band;
}
wiphy_dbg(hw->wiphy, "copying sband (band %d) due to VHT EXT NSS BW flag\n",
@@ -1635,6 +1635,7 @@ int ieee80211_register_hw(struct ieee80211_hw *hw)
#endif
wiphy_unregister(local->hw.wiphy);
fail_wiphy_register:
+ fail_band:
rtnl_lock();
rate_control_deinitialize(local);
ieee80211_remove_interfaces(local);
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index b82c7884a92d..7e23916c9189 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -2578,6 +2578,18 @@ static u16 ieee80211_store_ack_skb(struct ieee80211_local *local,
return info_id;
}
+static void ieee80211_remove_ack_skb(struct ieee80211_local *local, u16 info_id)
+{
+ struct sk_buff *ack_skb;
+ unsigned long flags;
+
+ spin_lock_irqsave(&local->ack_status_lock, flags);
+ ack_skb = idr_remove(&local->ack_status_frames, info_id);
+ spin_unlock_irqrestore(&local->ack_status_lock, flags);
+
+ kfree_skb(ack_skb);
+}
+
/**
* ieee80211_build_hdr - build 802.11 header in the given frame
* @sdata: virtual interface to build the header for
@@ -2935,7 +2947,8 @@ static struct sk_buff *ieee80211_build_hdr(struct ieee80211_sub_if_data *sdata,
if (ieee80211_skb_resize(sdata, skb, head_need, ENCRYPT_DATA)) {
ieee80211_free_txskb(&local->hw, skb);
skb = NULL;
- return ERR_PTR(-ENOMEM);
+ ret = -ENOMEM;
+ goto free;
}
}
@@ -3003,6 +3016,8 @@ static struct sk_buff *ieee80211_build_hdr(struct ieee80211_sub_if_data *sdata,
return skb;
free:
+ if (info_id)
+ ieee80211_remove_ack_skb(local, info_id);
kfree_skb(skb);
return ERR_PTR(ret);
}
diff --git a/net/mac802154/iface.c b/net/mac802154/iface.c
index 000be60d9580..b823720630e7 100644
--- a/net/mac802154/iface.c
+++ b/net/mac802154/iface.c
@@ -703,7 +703,7 @@ void ieee802154_remove_interfaces(struct ieee802154_local *local)
mutex_lock(&local->iflist_mtx);
list_for_each_entry_safe(sdata, tmp, &local->interfaces, list) {
- list_del(&sdata->list);
+ list_del_rcu(&sdata->list);
unregister_netdevice(sdata->dev);
}
diff --git a/net/mac802154/scan.c b/net/mac802154/scan.c
index a6dab3cc3ad8..0eef3b4f5b9b 100644
--- a/net/mac802154/scan.c
+++ b/net/mac802154/scan.c
@@ -594,6 +594,7 @@ int mac802154_perform_association(struct ieee802154_sub_if_data *sdata,
"Negative ASSOC RESP received from %8phC: %s\n", &ceaddr,
local->assoc_status == IEEE802154_PAN_AT_CAPACITY ?
"PAN at capacity" : "access denied");
+ goto clear_assoc;
}
ret = 0;
diff --git a/net/netfilter/ipset/ip_set_core.c b/net/netfilter/ipset/ip_set_core.c
index 94157104d85a..f51a1af31513 100644
--- a/net/netfilter/ipset/ip_set_core.c
+++ b/net/netfilter/ipset/ip_set_core.c
@@ -351,8 +351,8 @@ ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment,
if (unlikely(c)) {
set->ext_size -= sizeof(*c) + strlen(c->str) + 1;
- kfree_rcu(c, rcu);
rcu_assign_pointer(comment->c, NULL);
+ kfree_rcu(c, rcu);
}
if (!len)
return;
@@ -393,8 +393,8 @@ ip_set_comment_free(struct ip_set *set, void *ptr)
if (unlikely(!c))
return;
set->ext_size -= sizeof(*c) + strlen(c->str) + 1;
- kfree_rcu(c, rcu);
rcu_assign_pointer(comment->c, NULL);
+ kfree_rcu(c, rcu);
}
typedef void (*destroyer)(struct ip_set *, void *);
diff --git a/net/netfilter/ipset/ip_set_hash_gen.h b/net/netfilter/ipset/ip_set_hash_gen.h
index 4e56269efef2..0426d012185d 100644
--- a/net/netfilter/ipset/ip_set_hash_gen.h
+++ b/net/netfilter/ipset/ip_set_hash_gen.h
@@ -386,8 +386,9 @@ static void
mtype_ext_cleanup(struct ip_set *set, struct hbucket *n)
{
int i;
+ u8 pos = smp_load_acquire(&n->pos);
- for (i = 0; i < n->pos; i++)
+ for (i = 0; i < pos; i++)
if (test_bit(i, n->used))
ip_set_ext_destroy(set, ahash_data(n, i, set->dsize));
}
@@ -490,7 +491,7 @@ mtype_gc_do(struct ip_set *set, struct htype *h, struct htable *t, u32 r)
#ifdef IP_SET_HASH_WITH_NETS
u8 k;
#endif
- u8 htable_bits = t->htable_bits;
+ u8 pos, htable_bits = t->htable_bits;
spin_lock_bh(&t->hregion[r].lock);
for (i = ahash_bucket_start(r, htable_bits);
@@ -498,7 +499,8 @@ mtype_gc_do(struct ip_set *set, struct htype *h, struct htable *t, u32 r)
n = __ipset_dereference(hbucket(t, i));
if (!n)
continue;
- for (j = 0, d = 0; j < n->pos; j++) {
+ pos = smp_load_acquire(&n->pos);
+ for (j = 0, d = 0; j < pos; j++) {
if (!test_bit(j, n->used)) {
d++;
continue;
@@ -534,7 +536,7 @@ mtype_gc_do(struct ip_set *set, struct htype *h, struct htable *t, u32 r)
/* Still try to delete expired elements. */
continue;
tmp->size = n->size - AHASH_INIT_SIZE;
- for (j = 0, d = 0; j < n->pos; j++) {
+ for (j = 0, d = 0; j < pos; j++) {
if (!test_bit(j, n->used))
continue;
data = ahash_data(n, j, dsize);
@@ -604,7 +606,7 @@ mtype_cancel_gc(struct ip_set *set)
struct htype *h = set->data;
if (SET_WITH_TIMEOUT(set))
- cancel_delayed_work_sync(&h->gc.dwork);
+ disable_delayed_work_sync(&h->gc.dwork);
}
static int
@@ -623,7 +625,7 @@ mtype_resize(struct ip_set *set, bool retried)
{
struct htype *h = set->data;
struct htable *t, *orig;
- u8 htable_bits;
+ u8 pos, htable_bits;
size_t hsize, dsize = set->dsize;
#ifdef IP_SET_HASH_WITH_NETS
u8 flags;
@@ -685,8 +687,9 @@ mtype_resize(struct ip_set *set, bool retried)
n = __ipset_dereference(hbucket(orig, i));
if (!n)
continue;
- for (j = 0; j < n->pos; j++) {
- if (!test_bit(j, n->used))
+ pos = smp_load_acquire(&n->pos);
+ for (j = 0; j < pos; j++) {
+ if (!test_bit_acquire(j, n->used))
continue;
data = ahash_data(n, j, dsize);
if (SET_ELEM_EXPIRED(set, data))
@@ -809,9 +812,10 @@ mtype_ext_size(struct ip_set *set, u32 *elements, size_t *ext_size)
{
struct htype *h = set->data;
const struct htable *t;
- u32 i, j, r;
struct hbucket *n;
struct mtype_elem *data;
+ u32 i, j, r;
+ u8 pos;
t = rcu_dereference_bh(h->table);
for (r = 0; r < ahash_numof_locks(t->htable_bits); r++) {
@@ -820,8 +824,9 @@ mtype_ext_size(struct ip_set *set, u32 *elements, size_t *ext_size)
n = rcu_dereference_bh(hbucket(t, i));
if (!n)
continue;
- for (j = 0; j < n->pos; j++) {
- if (!test_bit(j, n->used))
+ pos = smp_load_acquire(&n->pos);
+ for (j = 0; j < pos; j++) {
+ if (!test_bit_acquire(j, n->used))
continue;
data = ahash_data(n, j, set->dsize);
if (!SET_ELEM_EXPIRED(set, data))
@@ -848,6 +853,7 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext,
bool flag_exist = flags & IPSET_FLAG_EXIST;
bool deleted = false, forceadd = false, reuse = false;
u32 r, key, multi = 0, elements, maxelem;
+ u8 npos = 0;
rcu_read_lock_bh();
t = rcu_dereference_bh(h->table);
@@ -889,7 +895,8 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext,
ext_size(AHASH_INIT_SIZE, set->dsize);
goto copy_elem;
}
- for (i = 0; i < n->pos; i++) {
+ npos = smp_load_acquire(&n->pos);
+ for (i = 0; i < npos; i++) {
if (!test_bit(i, n->used)) {
/* Reuse first deleted entry */
if (j == -1) {
@@ -933,7 +940,7 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext,
if (elements >= maxelem)
goto set_full;
/* Create a new slot */
- if (n->pos >= n->size) {
+ if (npos >= n->size) {
#ifdef IP_SET_HASH_WITH_MULTI
if (h->bucketsize >= AHASH_MAX_TUNED)
goto set_full;
@@ -962,7 +969,7 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext,
}
copy_elem:
- j = n->pos++;
+ j = npos++;
data = ahash_data(n, j, set->dsize);
copy_data:
t->hregion[r].elements++;
@@ -985,6 +992,8 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext,
if (SET_WITH_TIMEOUT(set))
ip_set_timeout_set(ext_timeout(data, set), ext->timeout);
smp_mb__before_atomic();
+ /* Ensure all data writes are visible before updating position */
+ smp_store_release(&n->pos, npos);
set_bit(j, n->used);
if (old != ERR_PTR(-ENOENT)) {
rcu_assign_pointer(hbucket(t, key), n);
@@ -1043,6 +1052,7 @@ mtype_del(struct ip_set *set, void *value, const struct ip_set_ext *ext,
int i, j, k, r, ret = -IPSET_ERR_EXIST;
u32 key, multi = 0;
size_t dsize = set->dsize;
+ u8 pos;
/* Userspace add and resize is excluded by the mutex.
* Kernespace add does not trigger resize.
@@ -1058,7 +1068,8 @@ mtype_del(struct ip_set *set, void *value, const struct ip_set_ext *ext,
n = rcu_dereference_bh(hbucket(t, key));
if (!n)
goto out;
- for (i = 0, k = 0; i < n->pos; i++) {
+ pos = smp_load_acquire(&n->pos);
+ for (i = 0, k = 0; i < pos; i++) {
if (!test_bit(i, n->used)) {
k++;
continue;
@@ -1072,8 +1083,8 @@ mtype_del(struct ip_set *set, void *value, const struct ip_set_ext *ext,
ret = 0;
clear_bit(i, n->used);
smp_mb__after_atomic();
- if (i + 1 == n->pos)
- n->pos--;
+ if (i + 1 == pos)
+ smp_store_release(&n->pos, --pos);
t->hregion[r].elements--;
#ifdef IP_SET_HASH_WITH_NETS
for (j = 0; j < IPSET_NET_COUNT; j++)
@@ -1095,11 +1106,11 @@ mtype_del(struct ip_set *set, void *value, const struct ip_set_ext *ext,
x->flags = flags;
}
}
- for (; i < n->pos; i++) {
+ for (; i < pos; i++) {
if (!test_bit(i, n->used))
k++;
}
- if (k == n->pos) {
+ if (k == pos) {
t->hregion[r].ext_size -= ext_size(n->size, dsize);
rcu_assign_pointer(hbucket(t, key), NULL);
kfree_rcu(n, rcu);
@@ -1110,7 +1121,7 @@ mtype_del(struct ip_set *set, void *value, const struct ip_set_ext *ext,
if (!tmp)
goto out;
tmp->size = n->size - AHASH_INIT_SIZE;
- for (j = 0, k = 0; j < n->pos; j++) {
+ for (j = 0, k = 0; j < pos; j++) {
if (!test_bit(j, n->used))
continue;
data = ahash_data(n, j, dsize);
@@ -1171,6 +1182,7 @@ mtype_test_cidrs(struct ip_set *set, struct mtype_elem *d,
int ret, i, j = 0;
#endif
u32 key, multi = 0;
+ u8 pos;
pr_debug("test by nets\n");
for (; j < NLEN && h->nets[j].cidr[0] && !multi; j++) {
@@ -1188,8 +1200,9 @@ mtype_test_cidrs(struct ip_set *set, struct mtype_elem *d,
n = rcu_dereference_bh(hbucket(t, key));
if (!n)
continue;
- for (i = 0; i < n->pos; i++) {
- if (!test_bit(i, n->used))
+ pos = smp_load_acquire(&n->pos);
+ for (i = 0; i < pos; i++) {
+ if (!test_bit_acquire(i, n->used))
continue;
data = ahash_data(n, i, set->dsize);
if (!mtype_data_equal(data, d, &multi))
@@ -1222,6 +1235,7 @@ mtype_test(struct ip_set *set, void *value, const struct ip_set_ext *ext,
struct mtype_elem *data;
int i, ret = 0;
u32 key, multi = 0;
+ u8 pos;
rcu_read_lock_bh();
t = rcu_dereference_bh(h->table);
@@ -1244,8 +1258,9 @@ mtype_test(struct ip_set *set, void *value, const struct ip_set_ext *ext,
ret = 0;
goto out;
}
- for (i = 0; i < n->pos; i++) {
- if (!test_bit(i, n->used))
+ pos = smp_load_acquire(&n->pos);
+ for (i = 0; i < pos; i++) {
+ if (!test_bit_acquire(i, n->used))
continue;
data = ahash_data(n, i, set->dsize);
if (!mtype_data_equal(data, d, &multi))
@@ -1361,6 +1376,7 @@ mtype_list(const struct ip_set *set,
/* We assume that one hash bucket fills into one page */
void *incomplete;
int i, ret = 0;
+ u8 pos;
atd = nla_nest_start(skb, IPSET_ATTR_ADT);
if (!atd)
@@ -1379,8 +1395,9 @@ mtype_list(const struct ip_set *set,
cb->args[IPSET_CB_ARG0], t, n);
if (!n)
continue;
- for (i = 0; i < n->pos; i++) {
- if (!test_bit(i, n->used))
+ pos = smp_load_acquire(&n->pos);
+ for (i = 0; i < pos; i++) {
+ if (!test_bit_acquire(i, n->used))
continue;
e = ahash_data(n, i, set->dsize);
if (SET_ELEM_EXPIRED(set, e))
diff --git a/net/netfilter/ipvs/ip_vs_app.c b/net/netfilter/ipvs/ip_vs_app.c
index fdacbc3c15be..f9132b359f0c 100644
--- a/net/netfilter/ipvs/ip_vs_app.c
+++ b/net/netfilter/ipvs/ip_vs_app.c
@@ -362,14 +362,13 @@ static inline int app_tcp_pkt_out(struct ip_vs_conn *cp, struct sk_buff *skb,
struct ip_vs_iphdr *ipvsh)
{
int diff;
- const unsigned int tcp_offset = ip_hdrlen(skb);
struct tcphdr *th;
__u32 seq;
- if (skb_ensure_writable(skb, tcp_offset + sizeof(*th)))
+ if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th)))
return 0;
- th = (struct tcphdr *)(skb_network_header(skb) + tcp_offset);
+ th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len);
/*
* Remember seq number in case this pkt gets resized
@@ -439,14 +438,13 @@ static inline int app_tcp_pkt_in(struct ip_vs_conn *cp, struct sk_buff *skb,
struct ip_vs_iphdr *ipvsh)
{
int diff;
- const unsigned int tcp_offset = ip_hdrlen(skb);
struct tcphdr *th;
__u32 seq;
- if (skb_ensure_writable(skb, tcp_offset + sizeof(*th)))
+ if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th)))
return 0;
- th = (struct tcphdr *)(skb_network_header(skb) + tcp_offset);
+ th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len);
/*
* Remember seq number in case this pkt gets resized
diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c
index 327baa17882a..7ab2b482d4bf 100644
--- a/net/netfilter/ipvs/ip_vs_conn.c
+++ b/net/netfilter/ipvs/ip_vs_conn.c
@@ -996,8 +996,8 @@ ip_vs_conn_new(const struct ip_vs_conn_param *p, int dest_af,
cp->app = NULL;
cp->app_data = NULL;
/* reset struct ip_vs_seq */
- cp->in_seq.delta = 0;
- cp->out_seq.delta = 0;
+ memset(&cp->in_seq, 0, sizeof(cp->in_seq));
+ memset(&cp->out_seq, 0, sizeof(cp->out_seq));
atomic_inc(&ipvs->conn_count);
if (flags & IP_VS_CONN_F_NO_CPORT)
diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c
index 5ea7ab8bf4dc..90619453cb6f 100644
--- a/net/netfilter/ipvs/ip_vs_core.c
+++ b/net/netfilter/ipvs/ip_vs_core.c
@@ -220,10 +220,10 @@ ip_vs_conn_stats(struct ip_vs_conn *cp, struct ip_vs_service *svc)
static inline void
ip_vs_set_state(struct ip_vs_conn *cp, int direction,
const struct sk_buff *skb,
- struct ip_vs_proto_data *pd)
+ struct ip_vs_proto_data *pd, unsigned int iph_len)
{
if (likely(pd->pp->state_transition))
- pd->pp->state_transition(cp, direction, skb, pd);
+ pd->pp->state_transition(cp, direction, skb, pd, iph_len);
}
static inline int
@@ -625,7 +625,7 @@ int ip_vs_leave(struct ip_vs_service *svc, struct sk_buff *skb,
ip_vs_in_stats(cp, skb);
/* set state */
- ip_vs_set_state(cp, IP_VS_DIR_INPUT, skb, pd);
+ ip_vs_set_state(cp, IP_VS_DIR_INPUT, skb, pd, iph->len);
/* transmit the first SYN packet */
ret = cp->packet_xmit(skb, cp, pd->pp, iph);
@@ -1041,8 +1041,7 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb,
snet.in6 = ciph.saddr.in6;
offset = ciph.len;
return handle_response_icmp(AF_INET6, skb, &snet, ciph.protocol, cp,
- pp, offset, sizeof(struct ipv6hdr),
- hooknum);
+ pp, offset, ipvsh->len, hooknum);
}
#endif
@@ -1306,7 +1305,7 @@ handle_response(int af, struct sk_buff *skb, struct ip_vs_proto_data *pd,
after_nat:
ip_vs_out_stats(cp, skb);
- ip_vs_set_state(cp, IP_VS_DIR_OUTPUT, skb, pd);
+ ip_vs_set_state(cp, IP_VS_DIR_OUTPUT, skb, pd, iph->len);
skb->ipvs_property = 1;
if (!(cp->flags & IP_VS_CONN_F_NFCT))
ip_vs_notrack(skb);
@@ -1589,6 +1588,8 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
bool tunnel, new_cp = false;
union nf_inet_addr *raddr;
char *outer_proto = "IPIP";
+ unsigned int hlen_ipip;
+ int ulen = 0;
*related = 1;
@@ -1625,9 +1626,10 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
/* Now find the contained IP header */
offset += sizeof(_icmph);
cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph);
- if (cih == NULL)
+ if (!(cih && cih->version == 4 && cih->ihl >= 5))
return NF_ACCEPT; /* The packet looks wrong, ignore */
raddr = (union nf_inet_addr *)&cih->daddr;
+ hlen_ipip = cih->ihl * 4;
/* Special case for errors for IPIP/UDP/GRE tunnel packets */
tunnel = false;
@@ -1643,9 +1645,9 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
/* Only for known tunnel */
if (!dest || dest->tun_type != IP_VS_CONN_F_TUNNEL_TYPE_IPIP)
return NF_ACCEPT;
- offset += cih->ihl * 4;
+ offset += hlen_ipip;
cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph);
- if (cih == NULL)
+ if (!(cih && cih->version == 4 && cih->ihl >= 5))
return NF_ACCEPT; /* The packet looks wrong, ignore */
tunnel = true;
} else if ((cih->protocol == IPPROTO_UDP || /* Can be UDP encap */
@@ -1653,12 +1655,11 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
/* Error for our tunnel must arrive at LOCAL_IN */
(skb_rtable(skb)->rt_flags & RTCF_LOCAL)) {
__u8 iproto;
- int ulen;
/* Non-first fragment has no UDP/GRE header */
if (unlikely(cih->frag_off & htons(IP_OFFSET)))
return NF_ACCEPT;
- offset2 = offset + cih->ihl * 4;
+ offset2 = offset + hlen_ipip;
if (cih->protocol == IPPROTO_UDP) {
ulen = ipvs_udp_decap(ipvs, skb, offset2, AF_INET,
raddr, &iproto);
@@ -1727,6 +1728,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
}
if (tunnel) {
+ unsigned int hlen_orig = cih->ihl * 4;
__be32 info = ic->un.gateway;
__u8 type = ic->type;
__u8 code = ic->code;
@@ -1743,6 +1745,9 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
goto ignore_tunnel;
offset2 -= ihl + sizeof(_icmph);
skb_reset_network_header(skb);
+ /* Ensure the IP header is present in headroom */
+ if (!pskb_may_pull(skb, hlen_ipip))
+ goto ignore_tunnel;
IP_VS_DBG(12, "ICMP for %s %pI4->%pI4: mtu=%u\n",
outer_proto, &ip_hdr(skb)->saddr,
&ip_hdr(skb)->daddr, mtu);
@@ -1758,8 +1763,8 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
if (dest_dst)
mtu = dst_mtu(dest_dst->dst_cache);
}
- if (mtu > 68 + sizeof(struct iphdr))
- mtu -= sizeof(struct iphdr);
+ if (mtu > 68 + hlen_ipip + ulen)
+ mtu -= hlen_ipip + ulen;
info = htonl(mtu);
}
/* Strip outer IP, ICMP and IPIP/UDP/GRE, go to IP header of
@@ -1768,6 +1773,9 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
if (pskb_pull(skb, offset2) == NULL)
goto ignore_tunnel;
skb_reset_network_header(skb);
+ /* Ensure the IP header is present in headroom */
+ if (!pskb_may_pull(skb, hlen_orig))
+ goto ignore_tunnel;
IP_VS_DBG(12, "Sending ICMP for %pI4->%pI4: t=%u, c=%u, i=%u\n",
&ip_hdr(skb)->saddr, &ip_hdr(skb)->daddr,
type, code, ntohl(info));
@@ -2055,7 +2063,7 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state
IP_VS_DBG_PKT(11, af, pp, skb, iph.off, "Incoming packet");
ip_vs_in_stats(cp, skb);
- ip_vs_set_state(cp, IP_VS_DIR_INPUT, skb, pd);
+ ip_vs_set_state(cp, IP_VS_DIR_INPUT, skb, pd, iph.len);
if (cp->packet_xmit)
ret = cp->packet_xmit(skb, cp, pp, &iph);
/* do not touch skb anymore */
diff --git a/net/netfilter/ipvs/ip_vs_proto_sctp.c b/net/netfilter/ipvs/ip_vs_proto_sctp.c
index 63c78a1f3918..c67317be17df 100644
--- a/net/netfilter/ipvs/ip_vs_proto_sctp.c
+++ b/net/netfilter/ipvs/ip_vs_proto_sctp.c
@@ -372,20 +372,15 @@ static const char *sctp_state_name(int state)
static inline void
set_sctp_state(struct ip_vs_proto_data *pd, struct ip_vs_conn *cp,
- int direction, const struct sk_buff *skb)
+ int direction, const struct sk_buff *skb,
+ unsigned int iph_len)
{
struct sctp_chunkhdr _sctpch, *sch;
unsigned char chunk_type;
int event, next_state;
- int ihl, cofs;
+ int cofs;
-#ifdef CONFIG_IP_VS_IPV6
- ihl = cp->af == AF_INET ? ip_hdrlen(skb) : sizeof(struct ipv6hdr);
-#else
- ihl = ip_hdrlen(skb);
-#endif
-
- cofs = ihl + sizeof(struct sctphdr);
+ cofs = iph_len + sizeof(struct sctphdr);
sch = skb_header_pointer(skb, cofs, sizeof(_sctpch), &_sctpch);
if (sch == NULL)
return;
@@ -468,10 +463,11 @@ set_sctp_state(struct ip_vs_proto_data *pd, struct ip_vs_conn *cp,
static void
sctp_state_transition(struct ip_vs_conn *cp, int direction,
- const struct sk_buff *skb, struct ip_vs_proto_data *pd)
+ const struct sk_buff *skb, struct ip_vs_proto_data *pd,
+ unsigned int iph_len)
{
spin_lock_bh(&cp->lock);
- set_sctp_state(pd, cp, direction, skb);
+ set_sctp_state(pd, cp, direction, skb, iph_len);
spin_unlock_bh(&cp->lock);
}
diff --git a/net/netfilter/ipvs/ip_vs_proto_tcp.c b/net/netfilter/ipvs/ip_vs_proto_tcp.c
index ede4fa3b63f5..b382810156b2 100644
--- a/net/netfilter/ipvs/ip_vs_proto_tcp.c
+++ b/net/netfilter/ipvs/ip_vs_proto_tcp.c
@@ -580,17 +580,12 @@ set_tcp_state(struct ip_vs_proto_data *pd, struct ip_vs_conn *cp,
static void
tcp_state_transition(struct ip_vs_conn *cp, int direction,
const struct sk_buff *skb,
- struct ip_vs_proto_data *pd)
+ struct ip_vs_proto_data *pd,
+ unsigned int iph_len)
{
struct tcphdr _tcph, *th;
-#ifdef CONFIG_IP_VS_IPV6
- int ihl = cp->af == AF_INET ? ip_hdrlen(skb) : sizeof(struct ipv6hdr);
-#else
- int ihl = ip_hdrlen(skb);
-#endif
-
- th = skb_header_pointer(skb, ihl, sizeof(_tcph), &_tcph);
+ th = skb_header_pointer(skb, iph_len, sizeof(_tcph), &_tcph);
if (th == NULL)
return;
diff --git a/net/netfilter/ipvs/ip_vs_proto_udp.c b/net/netfilter/ipvs/ip_vs_proto_udp.c
index ffbebda547fc..dbd4155bb075 100644
--- a/net/netfilter/ipvs/ip_vs_proto_udp.c
+++ b/net/netfilter/ipvs/ip_vs_proto_udp.c
@@ -445,7 +445,8 @@ static const char * udp_state_name(int state)
static void
udp_state_transition(struct ip_vs_conn *cp, int direction,
const struct sk_buff *skb,
- struct ip_vs_proto_data *pd)
+ struct ip_vs_proto_data *pd,
+ unsigned int iph_len)
{
if (unlikely(!pd)) {
pr_err("UDP no ns data\n");
diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c
index ed8b2616cf17..fa1b03c3201f 100644
--- a/net/netfilter/ipvs/ip_vs_xmit.c
+++ b/net/netfilter/ipvs/ip_vs_xmit.c
@@ -733,13 +733,11 @@ int
ip_vs_bypass_xmit(struct sk_buff *skb, struct ip_vs_conn *cp,
struct ip_vs_protocol *pp, struct ip_vs_iphdr *ipvsh)
{
- struct iphdr *iph = ip_hdr(skb);
-
- if (__ip_vs_get_out_rt(cp->ipvs, cp->af, skb, NULL, iph->daddr,
+ if (__ip_vs_get_out_rt(cp->ipvs, cp->af, skb, NULL, ip_hdr(skb)->daddr,
IP_VS_RT_MODE_NON_LOCAL, NULL, ipvsh) < 0)
goto tx_error;
- ip_send_check(iph);
+ ip_send_check(ip_hdr(skb));
/* Another hack: avoid icmp_send in ip_fragment */
skb->ignore_df = 1;
diff --git a/net/netfilter/nf_conncount.c b/net/netfilter/nf_conncount.c
index 14e62b3263cd..aca365f9f0ae 100644
--- a/net/netfilter/nf_conncount.c
+++ b/net/netfilter/nf_conncount.c
@@ -179,17 +179,16 @@ static int __nf_conncount_add(struct net *net,
return -ENOENT;
if (ct && nf_ct_is_confirmed(ct)) {
- /* local connections are confirmed in postrouting so confirmation
- * might have happened before hitting connlimit
+ /* Connection is confirmed but might still be in the setup phase.
+ * Only skip the tracking if it is fully assured. This guarantees
+ * that setup packets or retransmissions are properly counted and
+ * deduplicated.
*/
- if (skb->skb_iif != LOOPBACK_IFINDEX) {
+ if (test_bit(IPS_ASSURED_BIT, &ct->status)) {
err = -EEXIST;
goto out_put;
}
- /* this is likely a local connection, skip optimization to avoid
- * adding duplicates from a 'packet train'
- */
goto check_connections;
}
@@ -208,8 +207,8 @@ static int __nf_conncount_add(struct net *net,
/* Not found, but might be about to be confirmed */
if (PTR_ERR(found) == -EAGAIN) {
if (nf_ct_tuple_equal(&conn->tuple, &tuple) &&
- nf_ct_zone_id(&conn->zone, conn->zone.dir) ==
- nf_ct_zone_id(zone, zone->dir))
+ nf_ct_zone_id(&conn->zone, IP_CT_DIR_ORIGINAL) ==
+ nf_ct_zone_id(zone, IP_CT_DIR_ORIGINAL))
goto out_put; /* already exists */
} else {
collect++;
@@ -220,7 +219,7 @@ static int __nf_conncount_add(struct net *net,
found_ct = nf_ct_tuplehash_to_ctrack(found);
if (nf_ct_tuple_equal(&conn->tuple, &tuple) &&
- nf_ct_zone_equal(found_ct, zone, zone->dir)) {
+ nf_ct_zone_equal(found_ct, zone, IP_CT_DIR_ORIGINAL)) {
/*
* We should not see tuples twice unless someone hooks
* this into a table without "-p tcp --syn".
@@ -499,7 +498,7 @@ count_tree(struct net *net,
hash = jhash2(key, data->keylen, conncount_rnd) % CONNCOUNT_SLOTS;
root = &data->root[hash];
- parent = rcu_dereference_raw(root->rb_node);
+ parent = rcu_dereference(root->rb_node);
while (parent) {
int diff;
@@ -507,9 +506,9 @@ count_tree(struct net *net,
diff = key_diff(key, rbconn->key, data->keylen);
if (diff < 0) {
- parent = rcu_dereference_raw(parent->rb_left);
+ parent = rcu_dereference(parent->rb_left);
} else if (diff > 0) {
- parent = rcu_dereference_raw(parent->rb_right);
+ parent = rcu_dereference(parent->rb_right);
} else {
int ret;
diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index f5bde4f13958..423080cf86a4 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -565,9 +565,23 @@ static void destroy_gre_conntrack(struct nf_conn *ct)
{
#ifdef CONFIG_NF_CT_PROTO_GRE
struct nf_conn *master = ct->master;
+ struct nf_conn_help *help;
+
+ if (!master)
+ return;
+
+ help = nfct_help(master);
+ if (help) {
+ struct nf_conntrack_helper *helper;
+
+ rcu_read_lock();
+ helper = rcu_dereference(help->helper);
+ /* Only pptp helper has a destroy callback. */
+ if (helper && helper->destroy)
+ nf_ct_gre_keymap_destroy(master);
- if (master)
- nf_ct_gre_keymap_destroy(master);
+ rcu_read_unlock();
+ }
#endif
}
@@ -836,33 +850,6 @@ static void __nf_conntrack_hash_insert(struct nf_conn *ct,
&nf_conntrack_hash[reply_hash]);
}
-static bool nf_ct_ext_valid_pre(const struct nf_ct_ext *ext)
-{
- /* if ext->gen_id is not equal to nf_conntrack_ext_genid, some extensions
- * may contain stale pointers to e.g. helper that has been removed.
- *
- * The helper can't clear this because the nf_conn object isn't in
- * any hash and synchronize_rcu() isn't enough because associated skb
- * might sit in a queue.
- */
- return !ext || ext->gen_id == atomic_read(&nf_conntrack_ext_genid);
-}
-
-static bool nf_ct_ext_valid_post(struct nf_ct_ext *ext)
-{
- if (!ext)
- return true;
-
- if (ext->gen_id != atomic_read(&nf_conntrack_ext_genid))
- return false;
-
- /* inserted into conntrack table, nf_ct_iterate_cleanup()
- * will find it. Disable nf_ct_ext_find() id check.
- */
- WRITE_ONCE(ext->gen_id, 0);
- return true;
-}
-
int
nf_conntrack_hash_check_insert(struct nf_conn *ct)
{
@@ -878,9 +865,6 @@ nf_conntrack_hash_check_insert(struct nf_conn *ct)
zone = nf_ct_zone(ct);
- if (!nf_ct_ext_valid_pre(ct->ext))
- return -EAGAIN;
-
local_bh_disable();
do {
sequence = read_seqcount_begin(&nf_conntrack_generation);
@@ -914,18 +898,6 @@ nf_conntrack_hash_check_insert(struct nf_conn *ct)
goto chaintoolong;
}
- /* If genid has changed, we can't insert anymore because ct
- * extensions could have stale pointers and nf_ct_iterate_destroy
- * might have completed its table scan already.
- *
- * Increment of the ext genid right after this check is fine:
- * nf_ct_iterate_destroy blocks until locks are released.
- */
- if (!nf_ct_ext_valid_post(ct->ext)) {
- err = -EAGAIN;
- goto out;
- }
-
smp_wmb();
/* The caller holds a reference to this object */
refcount_set(&ct->ct_general.use, 2);
@@ -1253,11 +1225,6 @@ __nf_conntrack_confirm(struct sk_buff *skb)
return NF_DROP;
}
- if (!nf_ct_ext_valid_pre(ct->ext)) {
- NF_CT_STAT_INC(net, insert_failed);
- goto dying;
- }
-
/* We have to check the DYING flag after unlink to prevent
* a race against nf_ct_get_next_corpse() possibly called from
* user context, else we insert an already 'dead' hash, blocking
@@ -1320,16 +1287,6 @@ __nf_conntrack_confirm(struct sk_buff *skb)
nf_conntrack_double_unlock(hash, reply_hash);
local_bh_enable();
- /* ext area is still valid (rcu read lock is held,
- * but will go out of scope soon, we need to remove
- * this conntrack again.
- */
- if (!nf_ct_ext_valid_post(ct->ext)) {
- nf_ct_kill(ct);
- NF_CT_STAT_INC_ATOMIC(net, drop);
- return NF_DROP;
- }
-
help = nfct_help(ct);
if (help && help->helper)
nf_conntrack_event_cache(IPCT_HELPER, ct);
@@ -2219,6 +2176,7 @@ static int nf_confirm_cthelper(struct sk_buff *skb, struct nf_conn *ct,
{
const struct nf_conntrack_helper *helper;
const struct nf_conn_help *help;
+ unsigned int helper_flags;
int protoff;
help = nfct_help(ct);
@@ -2229,7 +2187,8 @@ static int nf_confirm_cthelper(struct sk_buff *skb, struct nf_conn *ct,
if (!helper)
return NF_ACCEPT;
- if (!(helper->flags & NF_CT_HELPER_F_USERSPACE))
+ helper_flags = READ_ONCE(helper->flags);
+ if (!(helper_flags & NF_CT_HELPER_F_USERSPACE))
return NF_ACCEPT;
switch (nf_ct_l3num(ct)) {
@@ -2442,13 +2401,11 @@ nf_ct_iterate_destroy(int (*iter)(struct nf_conn *i, void *data), void *data)
*/
synchronize_net();
- nf_ct_ext_bump_genid();
iter_data.data = data;
nf_ct_iterate_cleanup(iter, &iter_data);
/* Another cpu might be in a rcu read section with
- * rcu protected pointer cleared in iter callback
- * or hidden via nf_ct_ext_bump_genid() above.
+ * rcu protected pointer cleared in iter callback.
*
* Wait until those are done.
*/
diff --git a/net/netfilter/nf_conntrack_ecache.c b/net/netfilter/nf_conntrack_ecache.c
index 6526bdcca580..efabee2063cc 100644
--- a/net/netfilter/nf_conntrack_ecache.c
+++ b/net/netfilter/nf_conntrack_ecache.c
@@ -77,7 +77,7 @@ static enum retry_state ecache_work_evict_list(struct nf_conntrack_net *cnet)
hlist_nulls_del_rcu(&ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode);
hlist_nulls_add_head(&ct->tuplehash[IP_CT_DIR_REPLY].hnnode, &evicted_list);
- if (time_after(stop, jiffies)) {
+ if (time_after(jiffies, stop)) {
ret = STATE_RESTART;
break;
}
diff --git a/net/netfilter/nf_conntrack_extend.c b/net/netfilter/nf_conntrack_extend.c
index dd62cc12e775..0da105e1ded9 100644
--- a/net/netfilter/nf_conntrack_extend.c
+++ b/net/netfilter/nf_conntrack_extend.c
@@ -27,8 +27,6 @@
#define NF_CT_EXT_PREALLOC 128u /* conntrack events are on by default */
-atomic_t nf_conntrack_ext_genid __read_mostly = ATOMIC_INIT(1);
-
static const u8 nf_ct_ext_type_len[NF_CT_EXT_NUM] = {
[NF_CT_EXT_HELPER] = sizeof(struct nf_conn_help),
#if IS_ENABLED(CONFIG_NF_NAT)
@@ -118,10 +116,8 @@ void *nf_ct_ext_add(struct nf_conn *ct, enum nf_ct_ext_id id, gfp_t gfp)
if (!new)
return NULL;
- if (!ct->ext) {
+ if (!ct->ext)
memset(new->offset, 0, sizeof(new->offset));
- new->gen_id = atomic_read(&nf_conntrack_ext_genid);
- }
new->offset[id] = newoff;
new->len = newlen;
@@ -131,29 +127,3 @@ void *nf_ct_ext_add(struct nf_conn *ct, enum nf_ct_ext_id id, gfp_t gfp)
return (void *)new + newoff;
}
EXPORT_SYMBOL(nf_ct_ext_add);
-
-/* Use nf_ct_ext_find wrapper. This is only useful for unconfirmed entries. */
-void *__nf_ct_ext_find(const struct nf_ct_ext *ext, u8 id)
-{
- unsigned int gen_id = atomic_read(&nf_conntrack_ext_genid);
- unsigned int this_id = READ_ONCE(ext->gen_id);
-
- if (!__nf_ct_ext_exist(ext, id))
- return NULL;
-
- if (this_id == 0 || ext->gen_id == gen_id)
- return (void *)ext + ext->offset[id];
-
- return NULL;
-}
-EXPORT_SYMBOL(__nf_ct_ext_find);
-
-void nf_ct_ext_bump_genid(void)
-{
- unsigned int value = atomic_inc_return(&nf_conntrack_ext_genid);
-
- if (value == UINT_MAX)
- atomic_set(&nf_conntrack_ext_genid, 1);
-
- msleep(HZ);
-}
diff --git a/net/netfilter/nf_conntrack_irc.c b/net/netfilter/nf_conntrack_irc.c
index 0f50ea92ced9..454532838870 100644
--- a/net/netfilter/nf_conntrack_irc.c
+++ b/net/netfilter/nf_conntrack_irc.c
@@ -64,7 +64,7 @@ static const char *const dccprotos[] = {
/* tries to get the ip_addr and port out of a dcc command
* return value: -1 on failure, 0 on success
* data pointer to first byte of DCC command data
- * data_end pointer to last byte of dcc command data
+ * data_end one past end of data
* ip returns parsed ip of dcc command
* port returns parsed port of dcc command
* ad_beg_p returns pointer to first byte of addr data
@@ -82,10 +82,10 @@ static int parse_dcc(char *data, const char *data_end, __be32 *ip,
/* Make sure we have a newline character within the packet boundaries
* because simple_strtoul parses until the first invalid character. */
- for (tmp = data; tmp <= data_end; tmp++)
+ for (tmp = data; tmp < data_end; tmp++)
if (*tmp == '\n')
break;
- if (tmp > data_end || *tmp != '\n')
+ if (tmp >= data_end || *tmp != '\n')
return -1;
*ad_beg_p = data;
diff --git a/net/netfilter/nf_conntrack_seqadj.c b/net/netfilter/nf_conntrack_seqadj.c
index 7ab2b25b57bc..b7e99f34dfce 100644
--- a/net/netfilter/nf_conntrack_seqadj.c
+++ b/net/netfilter/nf_conntrack_seqadj.c
@@ -17,12 +17,14 @@ int nf_ct_seqadj_init(struct nf_conn *ct, enum ip_conntrack_info ctinfo,
if (off == 0)
return 0;
+ spin_lock_bh(&ct->lock);
set_bit(IPS_SEQ_ADJUST_BIT, &ct->status);
seqadj = nfct_seqadj(ct);
this_way = &seqadj->seq[dir];
this_way->offset_before = off;
this_way->offset_after = off;
+ spin_unlock_bh(&ct->lock);
return 0;
}
EXPORT_SYMBOL_GPL(nf_ct_seqadj_init);
diff --git a/net/netfilter/nf_nat_core.c b/net/netfilter/nf_nat_core.c
index 6ba7733355df..041ab3733baa 100644
--- a/net/netfilter/nf_nat_core.c
+++ b/net/netfilter/nf_nat_core.c
@@ -1195,6 +1195,16 @@ int nf_nat_register_fn(struct net *net, u8 pf, const struct nf_hook_ops *ops,
struct nf_hook_ops *nat_ops;
int i, ret;
+#ifndef MODULE
+ /* If nf_nat_core is built-in and nf_nat_init() fails, dependent
+ * modules like nft_chain_nat.ko may still call this function.
+ * However, nat_net would be invalid, likely pointing to some other
+ * per-net structure.
+ */
+ if (WARN_ON_ONCE(!nf_nat_hook))
+ return -EOPNOTSUPP;
+#endif
+
if (WARN_ON_ONCE(pf >= ARRAY_SIZE(nat_net->nat_proto_net)))
return -EINVAL;
diff --git a/net/netfilter/nf_nat_sip.c b/net/netfilter/nf_nat_sip.c
index 00838c0cc5bb..f7be30c69b5c 100644
--- a/net/netfilter/nf_nat_sip.c
+++ b/net/netfilter/nf_nat_sip.c
@@ -283,13 +283,24 @@ static unsigned int nf_nat_sip(struct sk_buff *skb, unsigned int protoff,
/* Mangle destination port for Cisco phones, then fix up checksums */
if (dir == IP_CT_DIR_REPLY && ct_sip_info->forced_dport) {
+ int doff = *dptr - (const char *)skb->data;
struct udphdr *uh;
+ if (doff <= 0) {
+ DEBUG_NET_WARN_ON_ONCE(1);
+ return NF_DROP;
+ }
+
+ /* ct_sip_info->forced_dport only expected with UDP */
+ if (nf_ct_protonum(ct) != IPPROTO_UDP)
+ return NF_DROP;
+
if (skb_ensure_writable(skb, skb->len)) {
nf_ct_helper_log(skb, ct, "cannot mangle packet");
return NF_DROP;
}
+ *dptr = skb->data + doff;
uh = (void *)skb->data + protoff;
uh->dest = ct_sip_info->forced_dport;
diff --git a/net/netfilter/nf_queue.c b/net/netfilter/nf_queue.c
index dd416c8532c5..c0c9ea5fc4e1 100644
--- a/net/netfilter/nf_queue.c
+++ b/net/netfilter/nf_queue.c
@@ -67,6 +67,7 @@ static void nf_queue_entry_release_refs(struct nf_queue_entry *entry)
nf_queue_sock_put(state->sk);
#if IS_ENABLED(CONFIG_BRIDGE_NETFILTER)
+ dev_put(entry->bridge_dev);
dev_put(entry->physin);
dev_put(entry->physout);
#endif
@@ -83,6 +84,8 @@ static void __nf_queue_entry_init_physdevs(struct nf_queue_entry *entry)
{
#if IS_ENABLED(CONFIG_BRIDGE_NETFILTER)
const struct sk_buff *skb = entry->skb;
+ struct dst_entry *dst = skb_dst(skb);
+ struct net_device *dev = NULL;
if (nf_bridge_info_exists(skb)) {
entry->physin = nf_bridge_get_physindev(skb, entry->state.net);
@@ -91,6 +94,16 @@ static void __nf_queue_entry_init_physdevs(struct nf_queue_entry *entry)
entry->physin = NULL;
entry->physout = NULL;
}
+
+ if (entry->state.pf == NFPROTO_BRIDGE &&
+ dst && (dst->flags & DST_FAKE_RTABLE))
+ dev = dst_dev_rcu(dst);
+
+ /* Must hold a reference on the bridge device: dst_hold() protects
+ * the dst itself, but the fake rtable is embedded in bridge-private
+ * storage that netdevice teardown can free independently.
+ */
+ entry->bridge_dev = dev;
#endif
}
@@ -107,6 +120,7 @@ bool nf_queue_entry_get_refs(struct nf_queue_entry *entry)
dev_hold(state->out);
#if IS_ENABLED(CONFIG_BRIDGE_NETFILTER)
+ dev_hold(entry->bridge_dev);
dev_hold(entry->physin);
dev_hold(entry->physout);
#endif
diff --git a/net/netfilter/nf_synproxy_core.c b/net/netfilter/nf_synproxy_core.c
index a277b2bd3275..69f84c44801d 100644
--- a/net/netfilter/nf_synproxy_core.c
+++ b/net/netfilter/nf_synproxy_core.c
@@ -183,23 +183,23 @@ synproxy_check_timestamp_cookie(struct synproxy_options *opts)
opts->options |= opts->tsecr & (1 << 5) ? NF_SYNPROXY_OPT_ECN : 0;
}
-static unsigned int
+static bool
synproxy_tstamp_adjust(struct sk_buff *skb, unsigned int protoff,
struct tcphdr *th, struct nf_conn *ct,
enum ip_conntrack_info ctinfo,
const struct nf_conn_synproxy *synproxy)
{
unsigned int optoff, optend;
- __be32 *ptr, old;
+ u32 new, old;
if (synproxy->tsoff == 0)
- return 1;
+ return true;
optoff = protoff + sizeof(struct tcphdr);
optend = protoff + th->doff * 4;
if (skb_ensure_writable(skb, optend))
- return 0;
+ return false;
th = (struct tcphdr *)(skb->data + protoff);
@@ -208,7 +208,7 @@ synproxy_tstamp_adjust(struct sk_buff *skb, unsigned int protoff,
switch (op[0]) {
case TCPOPT_EOL:
- return 1;
+ return true;
case TCPOPT_NOP:
optoff++;
continue;
@@ -216,28 +216,26 @@ synproxy_tstamp_adjust(struct sk_buff *skb, unsigned int protoff,
if (optoff + 1 == optend ||
optoff + op[1] > optend ||
op[1] < 2)
- return 0;
+ return true;
if (op[0] == TCPOPT_TIMESTAMP &&
op[1] == TCPOLEN_TIMESTAMP) {
if (CTINFO2DIR(ctinfo) == IP_CT_DIR_REPLY) {
- ptr = (__be32 *)&op[2];
- old = *ptr;
- *ptr = htonl(ntohl(*ptr) -
- synproxy->tsoff);
+ old = get_unaligned_be32(&op[2]);
+ new = old - synproxy->tsoff;
+ put_unaligned_be32(new, &op[2]);
} else {
- ptr = (__be32 *)&op[6];
- old = *ptr;
- *ptr = htonl(ntohl(*ptr) +
- synproxy->tsoff);
+ old = get_unaligned_be32(&op[6]);
+ new = old + synproxy->tsoff;
+ put_unaligned_be32(new, &op[6]);
}
inet_proto_csum_replace4(&th->check, skb,
- old, *ptr, false);
- return 1;
+ cpu_to_be32(old),
+ cpu_to_be32(new), false);
}
optoff += op[1];
}
}
- return 1;
+ return true;
}
#ifdef CONFIG_PROC_FS
@@ -748,7 +746,9 @@ ipv4_synproxy_hook(void *priv, struct sk_buff *skb,
break;
}
- synproxy_tstamp_adjust(skb, thoff, th, ct, ctinfo, synproxy);
+ if (!synproxy_tstamp_adjust(skb, thoff, th, ct, ctinfo, synproxy))
+ return NF_DROP_REASON(skb, SKB_DROP_REASON_NETFILTER_DROP, ENOMEM);
+
return NF_ACCEPT;
}
EXPORT_SYMBOL_GPL(ipv4_synproxy_hook);
@@ -1176,7 +1176,9 @@ ipv6_synproxy_hook(void *priv, struct sk_buff *skb,
break;
}
- synproxy_tstamp_adjust(skb, thoff, th, ct, ctinfo, synproxy);
+ if (!synproxy_tstamp_adjust(skb, thoff, th, ct, ctinfo, synproxy))
+ return NF_DROP_REASON(skb, SKB_DROP_REASON_NETFILTER_DROP, ENOMEM);
+
return NF_ACCEPT;
}
EXPORT_SYMBOL_GPL(ipv6_synproxy_hook);
diff --git a/net/netfilter/nfnetlink_cthelper.c b/net/netfilter/nfnetlink_cthelper.c
index 71a248cca746..eee6dbebb2b1 100644
--- a/net/netfilter/nfnetlink_cthelper.c
+++ b/net/netfilter/nfnetlink_cthelper.c
@@ -41,8 +41,9 @@ static int
nfnl_userspace_cthelper(struct sk_buff *skb, unsigned int protoff,
struct nf_conn *ct, enum ip_conntrack_info ctinfo)
{
- const struct nf_conn_help *help;
struct nf_conntrack_helper *helper;
+ const struct nf_conn_help *help;
+ unsigned int helper_flags;
help = nfct_help(ct);
if (help == NULL)
@@ -53,8 +54,10 @@ nfnl_userspace_cthelper(struct sk_buff *skb, unsigned int protoff,
if (helper == NULL)
return NF_DROP;
+ helper_flags = READ_ONCE(helper->flags);
+
/* This is a user-space helper not yet configured, skip. */
- if ((helper->flags &
+ if ((helper_flags &
(NF_CT_HELPER_F_USERSPACE | NF_CT_HELPER_F_CONFIGURED)) ==
NF_CT_HELPER_F_USERSPACE)
return NF_ACCEPT;
@@ -348,8 +351,8 @@ static int nfnl_cthelper_update_policy_all(struct nlattr *tb[],
for (i = 0; i < helper->expect_class_max + 1; i++) {
policy = (struct nf_conntrack_expect_policy *)
&helper->expect_policy[i];
- policy->max_expected = new_policy->max_expected;
- policy->timeout = new_policy->timeout;
+ policy->max_expected = new_policy[i].max_expected;
+ policy->timeout = new_policy[i].timeout;
}
err:
@@ -406,10 +409,10 @@ nfnl_cthelper_update(const struct nlattr * const tb[],
switch(status) {
case NFCT_HELPER_STATUS_ENABLED:
- helper->flags |= NF_CT_HELPER_F_CONFIGURED;
+ WRITE_ONCE(helper->flags, helper->flags | NF_CT_HELPER_F_CONFIGURED);
break;
case NFCT_HELPER_STATUS_DISABLED:
- helper->flags &= ~NF_CT_HELPER_F_CONFIGURED;
+ WRITE_ONCE(helper->flags, helper->flags & ~NF_CT_HELPER_F_CONFIGURED);
break;
}
}
@@ -531,8 +534,8 @@ static int
nfnl_cthelper_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type,
int event, struct nf_conntrack_helper *helper)
{
- struct nlmsghdr *nlh;
unsigned int flags = portid ? NLM_F_MULTI : 0;
+ struct nlmsghdr *nlh;
int status;
event = nfnl_msg_type(NFNL_SUBSYS_CTHELPER, event);
@@ -556,7 +559,7 @@ nfnl_cthelper_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type,
if (nla_put_be32(skb, NFCTH_PRIV_DATA_LEN, htonl(helper->data_len)))
goto nla_put_failure;
- if (helper->flags & NF_CT_HELPER_F_CONFIGURED)
+ if (READ_ONCE(helper->flags) & NF_CT_HELPER_F_CONFIGURED)
status = NFCT_HELPER_STATUS_ENABLED;
else
status = NFCT_HELPER_STATUS_DISABLED;
@@ -577,6 +580,7 @@ static int
nfnl_cthelper_dump_table(struct sk_buff *skb, struct netlink_callback *cb)
{
struct nf_conntrack_helper *cur, *last;
+ unsigned int helper_flags;
rcu_read_lock();
last = (struct nf_conntrack_helper *)cb->args[1];
@@ -585,8 +589,10 @@ nfnl_cthelper_dump_table(struct sk_buff *skb, struct netlink_callback *cb)
hlist_for_each_entry_rcu(cur,
&nf_ct_helper_hash[cb->args[0]], hnode) {
+ helper_flags = READ_ONCE(cur->flags);
+
/* skip non-userspace conntrack helpers. */
- if (!(cur->flags & NF_CT_HELPER_F_USERSPACE))
+ if (!(helper_flags & NF_CT_HELPER_F_USERSPACE))
continue;
if (cb->args[1]) {
diff --git a/net/netfilter/nfnetlink_osf.c b/net/netfilter/nfnetlink_osf.c
index c89efb951994..b9a0257fa2be 100644
--- a/net/netfilter/nfnetlink_osf.c
+++ b/net/netfilter/nfnetlink_osf.c
@@ -95,11 +95,7 @@ static bool nf_osf_match_one(const struct sk_buff *skb,
switch (*optp) {
case OSFOPT_MSS:
- mss = optp[3];
- mss <<= 8;
- mss |= optp[2];
-
- mss = ntohs((__force __be16)mss);
+ mss = get_unaligned_be16(&optp[2]);
break;
case OSFOPT_TS:
break;
diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
index 4a4dd947b37b..2e00a07eb545 100644
--- a/net/netfilter/nfnetlink_queue.c
+++ b/net/netfilter/nfnetlink_queue.c
@@ -1251,6 +1251,9 @@ dev_cmp(struct nf_queue_entry *entry, unsigned long ifindex)
if (physinif == ifindex || physoutif == ifindex)
return 1;
+
+ if (entry->bridge_dev && entry->bridge_dev->ifindex == ifindex)
+ return 1;
#endif
if (entry->skb_dev && entry->skb_dev->ifindex == ifindex)
return 1;
diff --git a/net/netfilter/nft_compat.c b/net/netfilter/nft_compat.c
index 08f620311b03..a77ffe2fcea7 100644
--- a/net/netfilter/nft_compat.c
+++ b/net/netfilter/nft_compat.c
@@ -389,6 +389,22 @@ static int nft_target_validate(const struct nft_ctx *ctx,
return 0;
}
+static int nft_target_bridge_validate(const struct nft_ctx *ctx,
+ const struct nft_expr *expr)
+{
+ struct xt_target *target = expr->ops->data;
+
+ /* Do not allow UNSPEC to stand-in for NFPROTO_BRIDGE
+ * targets: they are incompatible. ebtables targets return
+ * EBT_ACCEPT, DROP and so on which are not compatible with
+ * NF_ACCEPT, NF_DROP and so on.
+ */
+ if (target->family != NFPROTO_BRIDGE)
+ return -ENOENT;
+
+ return nft_target_validate(ctx, expr);
+}
+
static void __nft_match_eval(const struct nft_expr *expr,
struct nft_regs *regs,
const struct nft_pktinfo *pkt,
@@ -916,14 +932,16 @@ nft_target_select_ops(const struct nft_ctx *ctx,
ops->init = nft_target_init;
ops->destroy = nft_target_destroy;
ops->dump = nft_target_dump;
- ops->validate = nft_target_validate;
ops->data = target;
ops->reduce = NFT_REDUCE_READONLY;
- if (family == NFPROTO_BRIDGE)
+ if (family == NFPROTO_BRIDGE) {
ops->eval = nft_target_eval_bridge;
- else
+ ops->validate = nft_target_bridge_validate;
+ } else {
ops->eval = nft_target_eval_xt;
+ ops->validate = nft_target_validate;
+ }
return ops;
err:
diff --git a/net/netfilter/nft_lookup.c b/net/netfilter/nft_lookup.c
index 58c5b14889c4..beca9c7705e0 100644
--- a/net/netfilter/nft_lookup.c
+++ b/net/netfilter/nft_lookup.c
@@ -103,13 +103,13 @@ void nft_lookup_eval(const struct nft_expr *expr,
bool found;
ext = nft_set_do_lookup(net, set, ®s->data[priv->sreg]);
+ if (!ext)
+ ext = nft_set_catchall_lookup(net, set);
+
found = !!ext ^ priv->invert;
if (!found) {
- ext = nft_set_catchall_lookup(net, set);
- if (!ext) {
- regs->verdict.code = NFT_BREAK;
- return;
- }
+ regs->verdict.code = NFT_BREAK;
+ return;
}
if (ext) {
diff --git a/net/netfilter/nft_set_pipapo.c b/net/netfilter/nft_set_pipapo.c
index a2dd1212e0f0..9d2cfdc94a52 100644
--- a/net/netfilter/nft_set_pipapo.c
+++ b/net/netfilter/nft_set_pipapo.c
@@ -342,6 +342,8 @@
#include "nft_set_pipapo_avx2.h"
#include "nft_set_pipapo.h"
+static void nft_pipapo_abort(const struct nft_set *set);
+
/**
* pipapo_refill() - For each set bit, set bits from selected mapping table item
* @map: Bitmap to be scanned for set bits
@@ -1284,7 +1286,7 @@ static int nft_pipapo_insert(const struct net *net, const struct nft_set *set,
const u8 *start_p, *end_p;
int i, bsize_max, err = 0;
- if (!m)
+ if (!m || m->state == NFT_PIPAPO_CLONE_ERR)
return -ENOMEM;
if (nft_set_ext_exists(ext, NFT_SET_EXT_KEY_END))
@@ -1355,8 +1357,10 @@ static int nft_pipapo_insert(const struct net *net, const struct nft_set *set,
else
ret = pipapo_expand(f, start, end, f->groups * f->bb);
- if (ret < 0)
- return ret;
+ if (ret < 0) {
+ err = ret;
+ goto abort;
+ }
if (f->bsize > bsize_max)
bsize_max = f->bsize;
@@ -1372,7 +1376,7 @@ static int nft_pipapo_insert(const struct net *net, const struct nft_set *set,
err = pipapo_realloc_scratch(m, bsize_max);
if (err)
- return err;
+ goto abort;
m->bsize_max = bsize_max;
} else {
@@ -1384,7 +1388,26 @@ static int nft_pipapo_insert(const struct net *net, const struct nft_set *set,
pipapo_map(m, rulemap, e);
+ m->state = NFT_PIPAPO_CLONE_MOD;
return 0;
+abort:
+ DEBUG_NET_WARN_ON_ONCE(m->state == NFT_PIPAPO_CLONE_ERR);
+
+ /* Two rollback cases:
+ * 1) no previous changes. nft_pipapo_abort is not
+ * guaranteed to be invoked (there might be no further
+ * add/delete requests coming after this).
+ *
+ * 2) we had previous changes: there are transaction
+ * records pointing to this set. Leave the rollback to
+ * the transaction handling.
+ */
+ if (m->state == NFT_PIPAPO_CLONE_NEW)
+ nft_pipapo_abort(set); /* releases m */
+ else
+ m->state = NFT_PIPAPO_CLONE_ERR;
+
+ return err;
}
/**
@@ -1462,6 +1485,7 @@ static struct nft_pipapo_match *pipapo_clone(struct nft_pipapo_match *old)
dst++;
}
+ new->state = NFT_PIPAPO_CLONE_NEW;
return new;
out_mt:
@@ -1885,7 +1909,7 @@ nft_pipapo_deactivate(const struct net *net, const struct nft_set *set,
/* removal must occur on priv->clone, if we are low on memory
* we have no choice and must fail the removal request.
*/
- if (!m)
+ if (!m || m->state == NFT_PIPAPO_CLONE_ERR)
return NULL;
e = pipapo_get(m, (const u8 *)elem->key.val.data,
diff --git a/net/netfilter/nft_set_pipapo.h b/net/netfilter/nft_set_pipapo.h
index 49000f5510b2..6cf75680295e 100644
--- a/net/netfilter/nft_set_pipapo.h
+++ b/net/netfilter/nft_set_pipapo.h
@@ -134,9 +134,16 @@ struct nft_pipapo_scratch {
unsigned long map[];
};
+enum nft_pipapo_clone_state {
+ NFT_PIPAPO_CLONE_NEW,
+ NFT_PIPAPO_CLONE_MOD,
+ NFT_PIPAPO_CLONE_ERR,
+};
+
/**
* struct nft_pipapo_match - Data used for lookup and matching
* @field_count: Amount of fields in set
+ * @state: add/delete state; used from control plane
* @bsize_max: Maximum lookup table bucket size of all fields, in longs
* @scratch: Preallocated per-CPU maps for partial matching results
* @rcu: Matching data is swapped on commits
@@ -144,6 +151,7 @@ struct nft_pipapo_scratch {
*/
struct nft_pipapo_match {
u8 field_count;
+ enum nft_pipapo_clone_state state:8;
unsigned int bsize_max;
struct nft_pipapo_scratch * __percpu *scratch;
struct rcu_head rcu;
diff --git a/net/netfilter/nft_synproxy.c b/net/netfilter/nft_synproxy.c
index 4d3e5a31b412..ad3e7d27f02e 100644
--- a/net/netfilter/nft_synproxy.c
+++ b/net/netfilter/nft_synproxy.c
@@ -23,14 +23,13 @@ static const struct nla_policy nft_synproxy_policy[NFTA_SYNPROXY_MAX + 1] = {
static void nft_synproxy_tcp_options(struct synproxy_options *opts,
const struct tcphdr *tcp,
struct synproxy_net *snet,
- struct nf_synproxy_info *info,
- const struct nft_synproxy *priv)
+ struct nf_synproxy_info *info)
{
this_cpu_inc(snet->stats->syn_received);
if (tcp->ece && tcp->cwr)
opts->options |= NF_SYNPROXY_OPT_ECN;
- opts->options &= priv->info.options;
+ opts->options &= info->options;
opts->mss_encode = opts->mss_option;
opts->mss_option = info->mss;
if (opts->options & NF_SYNPROXY_OPT_TIMESTAMP)
@@ -55,7 +54,7 @@ static void nft_synproxy_eval_v4(const struct nft_synproxy *priv,
if (tcp->syn) {
/* Initial SYN from client */
- nft_synproxy_tcp_options(opts, tcp, snet, &info, priv);
+ nft_synproxy_tcp_options(opts, tcp, snet, &info);
synproxy_send_client_synack(net, skb, tcp, opts);
consume_skb(skb);
regs->verdict.code = NF_STOLEN;
@@ -86,7 +85,7 @@ static void nft_synproxy_eval_v6(const struct nft_synproxy *priv,
if (tcp->syn) {
/* Initial SYN from client */
- nft_synproxy_tcp_options(opts, tcp, snet, &info, priv);
+ nft_synproxy_tcp_options(opts, tcp, snet, &info);
synproxy_send_client_synack_ipv6(net, skb, tcp, opts);
consume_skb(skb);
regs->verdict.code = NF_STOLEN;
diff --git a/net/netfilter/xt_cluster.c b/net/netfilter/xt_cluster.c
index 908fd5f2c3c8..eaf2511d63f0 100644
--- a/net/netfilter/xt_cluster.c
+++ b/net/netfilter/xt_cluster.c
@@ -107,7 +107,7 @@ xt_cluster_mt(const struct sk_buff *skb, struct xt_action_param *par)
}
ct = nf_ct_get(skb, &ctinfo);
- if (ct == NULL)
+ if (!ct || nf_ct_is_template(ct))
return false;
if (ct->master)
diff --git a/net/netfilter/xt_connmark.c b/net/netfilter/xt_connmark.c
index 4277084de2e7..2cf27f7d59b9 100644
--- a/net/netfilter/xt_connmark.c
+++ b/net/netfilter/xt_connmark.c
@@ -112,6 +112,16 @@ static int connmark_tg_check(const struct xt_tgchk_param *par)
return ret;
}
+static int connmark_tg_check_v2(const struct xt_tgchk_param *par)
+{
+ const struct xt_connmark_tginfo2 *info = par->targinfo;
+
+ if (info->shift_dir > D_SHIFT_RIGHT || info->shift_bits >= 32)
+ return -EINVAL;
+
+ return connmark_tg_check(par);
+}
+
static void connmark_tg_destroy(const struct xt_tgdtor_param *par)
{
nf_ct_netns_put(par->net, par->family);
@@ -162,7 +172,7 @@ static struct xt_target connmark_tg_reg[] __read_mostly = {
.name = "CONNMARK",
.revision = 2,
.family = NFPROTO_IPV4,
- .checkentry = connmark_tg_check,
+ .checkentry = connmark_tg_check_v2,
.target = connmark_tg_v2,
.targetsize = sizeof(struct xt_connmark_tginfo2),
.destroy = connmark_tg_destroy,
@@ -183,7 +193,7 @@ static struct xt_target connmark_tg_reg[] __read_mostly = {
.name = "CONNMARK",
.revision = 2,
.family = NFPROTO_IPV6,
- .checkentry = connmark_tg_check,
+ .checkentry = connmark_tg_check_v2,
.target = connmark_tg_v2,
.targetsize = sizeof(struct xt_connmark_tginfo2),
.destroy = connmark_tg_destroy,
diff --git a/net/netfilter/xt_nat.c b/net/netfilter/xt_nat.c
index b4f7bbc3f3ca..51c7f7ce88d9 100644
--- a/net/netfilter/xt_nat.c
+++ b/net/netfilter/xt_nat.c
@@ -26,6 +26,15 @@ static int xt_nat_checkentry_v0(const struct xt_tgchk_param *par)
static int xt_nat_checkentry(const struct xt_tgchk_param *par)
{
+ switch (par->family) {
+ case NFPROTO_IPV4:
+ case NFPROTO_IPV6:
+ case NFPROTO_INET:
+ break;
+ default:
+ return -EINVAL;
+ }
+
return nf_ct_netns_get(par->net, par->family);
}
diff --git a/net/netfilter/xt_rateest.c b/net/netfilter/xt_rateest.c
index b1d736c15fcb..7c05b6342578 100644
--- a/net/netfilter/xt_rateest.c
+++ b/net/netfilter/xt_rateest.c
@@ -16,7 +16,7 @@ xt_rateest_mt(const struct sk_buff *skb, struct xt_action_param *par)
{
const struct xt_rateest_match_info *info = par->matchinfo;
struct gnet_stats_rate_est64 sample = {0};
- u_int32_t bps1, bps2, pps1, pps2;
+ u64 bps1, bps2, pps1, pps2;
bool ret = true;
gen_estimator_read(&info->est1->rate_est, &sample);
diff --git a/net/netfilter/xt_u32.c b/net/netfilter/xt_u32.c
index 1ae7f926b0df..dabbaa742874 100644
--- a/net/netfilter/xt_u32.c
+++ b/net/netfilter/xt_u32.c
@@ -106,7 +106,7 @@ static int u32_mt_checkentry(const struct xt_mtchk_param *par)
{
const struct xt_u32 *data = par->matchinfo;
const struct xt_u32_test *ct;
- unsigned int i;
+ unsigned int i, j;
if (data->ntests > ARRAY_SIZE(data->tests))
return -EINVAL;
@@ -117,6 +117,16 @@ static int u32_mt_checkentry(const struct xt_mtchk_param *par)
if (ct->nnums > ARRAY_SIZE(ct->location) ||
ct->nvalues > ARRAY_SIZE(ct->value))
return -EINVAL;
+
+ for (j = 1; j < ct->nnums; ++j) {
+ switch (ct->location[j].nextop) {
+ case XT_U32_LEFTSH:
+ case XT_U32_RIGHTSH:
+ if (ct->location[j].number >= 32)
+ return -EINVAL;
+ break;
+ }
+ }
}
return 0;
diff --git a/net/openvswitch/conntrack.c b/net/openvswitch/conntrack.c
index a0811e1fba65..f63c8ac74e4a 100644
--- a/net/openvswitch/conntrack.c
+++ b/net/openvswitch/conntrack.c
@@ -1800,10 +1800,10 @@ static int ovs_ct_limit_get_zone_limit(struct net *net,
} else {
rcu_read_lock();
limit = ct_limit_get(info, zone);
- rcu_read_unlock();
err = __ovs_ct_limit_get_zone_limit(
net, info->data, zone, limit, reply);
+ rcu_read_unlock();
if (err)
return err;
}
diff --git a/net/openvswitch/flow_netlink.c b/net/openvswitch/flow_netlink.c
index 3d2dbe963813..4622e22a70f7 100644
--- a/net/openvswitch/flow_netlink.c
+++ b/net/openvswitch/flow_netlink.c
@@ -2496,13 +2496,56 @@ static inline int add_nested_action_start(struct sw_flow_actions **sfa,
return used;
}
-static inline void add_nested_action_end(struct sw_flow_actions *sfa,
- int st_offset)
+static inline int add_nested_action_end(struct sw_flow_actions *sfa,
+ int st_offset)
{
- struct nlattr *a = (struct nlattr *) ((unsigned char *)sfa->actions +
- st_offset);
+ struct nlattr *a;
+ u32 attr_len;
+
+ if (WARN_ON_ONCE(st_offset < 0 ||
+ (u32)st_offset > sfa->actions_len))
+ return -EINVAL;
+
+ attr_len = sfa->actions_len - (u32)st_offset;
+ if (WARN_ON_ONCE(attr_len < NLA_HDRLEN))
+ return -EINVAL;
- a->nla_len = sfa->actions_len - st_offset;
+ if (attr_len > U16_MAX)
+ return -EMSGSIZE;
+
+ a = (struct nlattr *)((u8 *)sfa->actions + st_offset);
+ a->nla_len = attr_len;
+ return 0;
+}
+
+/* Free the generated action-list tail at @start and truncate it.
+ * If @nested, @start points to its containing nlattr header.
+ */
+static void ovs_nla_trim(struct sw_flow_actions *sfa, int start, bool nested)
+{
+ const struct nlattr *actions;
+ u32 len;
+
+ if (start < 0)
+ return;
+
+ if (WARN_ON_ONCE((u32)start > sfa->actions_len))
+ return;
+
+ actions = (const struct nlattr *)((u8 *)sfa->actions + start);
+ len = sfa->actions_len - (u32)start;
+
+ if (nested) {
+ if (len < NLA_HDRLEN)
+ goto out;
+
+ actions = (const struct nlattr *)((u8 *)actions + NLA_HDRLEN);
+ len -= NLA_HDRLEN;
+ }
+
+ ovs_nla_free_nested_actions(actions, len);
+out:
+ sfa->actions_len = start;
}
static int __ovs_nla_copy_actions(struct net *net, const struct nlattr *attr,
@@ -2522,6 +2565,7 @@ static int validate_and_copy_sample(struct net *net, const struct nlattr *attr,
const struct nlattr *attrs[OVS_SAMPLE_ATTR_MAX + 1];
const struct nlattr *probability, *actions;
const struct nlattr *a;
+ int actions_start;
int rem, start, err;
struct sample_arg arg;
@@ -2565,18 +2609,27 @@ static int validate_and_copy_sample(struct net *net, const struct nlattr *attr,
err = ovs_nla_add_action(sfa, OVS_SAMPLE_ATTR_ARG, &arg, sizeof(arg),
log);
if (err)
- return err;
+ goto err;
+ actions_start = (*sfa)->actions_len;
err = __ovs_nla_copy_actions(net, actions, key, sfa,
eth_type, vlan_tci, mpls_label_count, log,
depth + 1);
if (err)
- return err;
+ goto err_free;
- add_nested_action_end(*sfa, start);
+ err = add_nested_action_end(*sfa, start);
+ if (err)
+ goto err_free;
return 0;
+
+err_free:
+ ovs_nla_trim(*sfa, actions_start, false);
+err:
+ (*sfa)->actions_len = start;
+ return err;
}
static int validate_and_copy_dec_ttl(struct net *net,
@@ -2624,18 +2677,31 @@ static int validate_and_copy_dec_ttl(struct net *net,
return start;
action_start = add_nested_action_start(sfa, OVS_DEC_TTL_ATTR_ACTION, log);
- if (action_start < 0)
- return action_start;
+ if (action_start < 0) {
+ err = action_start;
+ goto err;
+ }
err = __ovs_nla_copy_actions(net, actions, key, sfa, eth_type,
vlan_tci, mpls_label_count, log,
depth + 1);
if (err)
- return err;
+ goto err_free;
+
+ err = add_nested_action_end(*sfa, action_start);
+ if (err)
+ goto err_free;
- add_nested_action_end(*sfa, action_start);
- add_nested_action_end(*sfa, start);
+ err = add_nested_action_end(*sfa, start);
+ if (err)
+ goto err_free;
return 0;
+
+err_free:
+ ovs_nla_trim(*sfa, action_start, true);
+err:
+ (*sfa)->actions_len = start;
+ return err;
}
static int validate_and_copy_clone(struct net *net,
@@ -2646,6 +2712,7 @@ static int validate_and_copy_clone(struct net *net,
u32 mpls_label_count, bool log, bool last,
u32 depth)
{
+ int actions_start;
int start, err;
u32 exec;
@@ -2661,17 +2728,26 @@ static int validate_and_copy_clone(struct net *net,
err = ovs_nla_add_action(sfa, OVS_CLONE_ATTR_EXEC, &exec,
sizeof(exec), log);
if (err)
- return err;
+ goto err;
+ actions_start = (*sfa)->actions_len;
err = __ovs_nla_copy_actions(net, attr, key, sfa,
eth_type, vlan_tci, mpls_label_count, log,
depth + 1);
if (err)
- return err;
+ goto err_free;
- add_nested_action_end(*sfa, start);
+ err = add_nested_action_end(*sfa, start);
+ if (err)
+ goto err_free;
return 0;
+
+err_free:
+ ovs_nla_trim(*sfa, actions_start, false);
+err:
+ (*sfa)->actions_len = start;
+ return err;
}
void ovs_match_init(struct sw_flow_match *match,
@@ -2763,20 +2839,20 @@ static int validate_and_copy_set_tun(const struct nlattr *attr,
tun_dst = metadata_dst_alloc(key.tun_opts_len, METADATA_IP_TUNNEL,
GFP_KERNEL);
- if (!tun_dst)
- return -ENOMEM;
+ if (!tun_dst) {
+ err = -ENOMEM;
+ goto err;
+ }
err = dst_cache_init(&tun_dst->u.tun_info.dst_cache, GFP_KERNEL);
- if (err) {
- dst_release((struct dst_entry *)tun_dst);
- return err;
- }
+ if (err)
+ goto err_free_tun_dst;
a = __add_action(sfa, OVS_KEY_ATTR_TUNNEL_INFO, NULL,
sizeof(*ovs_tun), log);
if (IS_ERR(a)) {
- dst_release((struct dst_entry *)tun_dst);
- return PTR_ERR(a);
+ err = PTR_ERR(a);
+ goto err_free_tun_dst;
}
ovs_tun = nla_data(a);
@@ -2797,8 +2873,16 @@ static int validate_and_copy_set_tun(const struct nlattr *attr,
ip_tunnel_info_opts_set(tun_info,
TUN_METADATA_OPTS(&key, key.tun_opts_len),
key.tun_opts_len, dst_opt_type);
- add_nested_action_end(*sfa, start);
+ err = add_nested_action_end(*sfa, start);
+ if (WARN_ON_ONCE(err))
+ goto err_free_tun_dst;
+
+ return 0;
+err_free_tun_dst:
+ dst_release((struct dst_entry *)tun_dst);
+err:
+ (*sfa)->actions_len = start;
return err;
}
@@ -2971,7 +3055,7 @@ static int validate_set(const struct nlattr *a,
/* Convert non-masked non-tunnel set actions to masked set actions. */
if (!masked && key_type != OVS_KEY_ATTR_TUNNEL) {
- int start, len = key_len * 2;
+ int err, start, len = key_len * 2;
struct nlattr *at;
*skip_copy = true;
@@ -2983,8 +3067,11 @@ static int validate_set(const struct nlattr *a,
return start;
at = __add_action(sfa, key_type, NULL, len, log);
- if (IS_ERR(at))
- return PTR_ERR(at);
+ if (IS_ERR(at)) {
+ err = PTR_ERR(at);
+ (*sfa)->actions_len = start;
+ return err;
+ }
memcpy(nla_data(at), nla_data(ovs_key), key_len); /* Key. */
memset(nla_data(at) + key_len, 0xff, key_len); /* Mask. */
@@ -2994,7 +3081,11 @@ static int validate_set(const struct nlattr *a,
mask->ipv6_label &= htonl(0x000FFFFF);
}
- add_nested_action_end(*sfa, start);
+ err = add_nested_action_end(*sfa, start);
+ if (WARN_ON_ONCE(err)) {
+ (*sfa)->actions_len = start;
+ return err;
+ }
}
return 0;
@@ -3039,7 +3130,8 @@ static int validate_and_copy_check_pkt_len(struct net *net,
const struct nlattr *acts_if_greater, *acts_if_lesser_eq;
struct nlattr *a[OVS_CHECK_PKT_LEN_ATTR_MAX + 1];
struct check_pkt_len_arg arg;
- int nested_acts_start;
+ int greater_acts_start = -1;
+ int lesser_acts_start = -1;
int start, err;
err = nla_parse_deprecated_strict(a, OVS_CHECK_PKT_LEN_ATTR_MAX,
@@ -3074,37 +3166,58 @@ static int validate_and_copy_check_pkt_len(struct net *net,
err = ovs_nla_add_action(sfa, OVS_CHECK_PKT_LEN_ATTR_ARG, &arg,
sizeof(arg), log);
if (err)
- return err;
+ goto err_free;
- nested_acts_start = add_nested_action_start(sfa,
- OVS_CHECK_PKT_LEN_ATTR_ACTIONS_IF_LESS_EQUAL, log);
- if (nested_acts_start < 0)
- return nested_acts_start;
+ lesser_acts_start =
+ add_nested_action_start(sfa,
+ OVS_CHECK_PKT_LEN_ATTR_ACTIONS_IF_LESS_EQUAL,
+ log);
+ if (lesser_acts_start < 0) {
+ err = lesser_acts_start;
+ goto err_free;
+ }
err = __ovs_nla_copy_actions(net, acts_if_lesser_eq, key, sfa,
eth_type, vlan_tci, mpls_label_count, log,
depth + 1);
if (err)
- return err;
+ goto err_free;
- add_nested_action_end(*sfa, nested_acts_start);
+ err = add_nested_action_end(*sfa, lesser_acts_start);
+ if (err)
+ goto err_free;
- nested_acts_start = add_nested_action_start(sfa,
- OVS_CHECK_PKT_LEN_ATTR_ACTIONS_IF_GREATER, log);
- if (nested_acts_start < 0)
- return nested_acts_start;
+ greater_acts_start =
+ add_nested_action_start(sfa,
+ OVS_CHECK_PKT_LEN_ATTR_ACTIONS_IF_GREATER,
+ log);
+ if (greater_acts_start < 0) {
+ err = greater_acts_start;
+ goto err_free;
+ }
err = __ovs_nla_copy_actions(net, acts_if_greater, key, sfa,
eth_type, vlan_tci, mpls_label_count, log,
depth + 1);
if (err)
- return err;
+ goto err_free;
+
+ err = add_nested_action_end(*sfa, greater_acts_start);
+ if (err)
+ goto err_free;
- add_nested_action_end(*sfa, nested_acts_start);
- add_nested_action_end(*sfa, start);
+ err = add_nested_action_end(*sfa, start);
+ if (err)
+ goto err_free;
return 0;
+
+err_free:
+ ovs_nla_trim(*sfa, greater_acts_start, true);
+ ovs_nla_trim(*sfa, lesser_acts_start, true);
+ ovs_nla_trim(*sfa, start, false);
+ return err;
}
static int validate_psample(const struct nlattr *attr)
diff --git a/net/psample/psample.c b/net/psample/psample.c
index 25f92ba0840c..2d3bda59fa0d 100644
--- a/net/psample/psample.c
+++ b/net/psample/psample.c
@@ -476,15 +476,17 @@ void psample_sample_packet(struct psample_group *group,
goto error;
if (data_len) {
- int nla_len = nla_total_size(data_len);
+ int nla_len = nla_attr_size(data_len);
struct nlattr *nla;
nla = skb_put(nl_skb, nla_len);
nla->nla_type = PSAMPLE_ATTR_DATA;
- nla->nla_len = nla_attr_size(data_len);
+ nla->nla_len = nla_len;
if (skb_copy_bits(skb, 0, nla_data(nla), data_len))
goto error;
+
+ skb_put_zero(nl_skb, nla_padlen(data_len));
}
#ifdef CONFIG_INET
diff --git a/net/qrtr/af_qrtr.c b/net/qrtr/af_qrtr.c
index 2c009793f193..305523cebe3b 100644
--- a/net/qrtr/af_qrtr.c
+++ b/net/qrtr/af_qrtr.c
@@ -496,7 +496,7 @@ int qrtr_endpoint_post(struct qrtr_endpoint *ep, const void *data, size_t len)
if (cb->dst_port == QRTR_PORT_CTRL_LEGACY)
cb->dst_port = QRTR_PORT_CTRL;
- if (!size || len != ALIGN(size, 4) + hdrlen)
+ if (!size || size > len || len != ALIGN(size, 4) + hdrlen)
goto err;
if ((cb->type == QRTR_TYPE_NEW_SERVER ||
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index 99efe14eb0c9..4f4094d0c5cc 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -157,7 +157,7 @@ static int rxrpc_verify_data(struct rxrpc_call *call, struct sk_buff *skb)
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
int ret;
- if (sp->len > call->rx_dec_bsize) {
+ if (sp->len > call->rx_dec_bsize || !call->rx_dec_buffer) {
/* Make sure we can hold a 1412-byte jumbo subpacket and make
* sure that the buffer size is aligned to a crypto blocksize.
*/
@@ -429,8 +429,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
if (test_bit(RXRPC_CALL_RELEASED, &call->flags)) {
rxrpc_see_call(call, rxrpc_call_see_already_released);
mutex_unlock(&call->user_mutex);
- if (!(flags & MSG_PEEK))
- rxrpc_put_call(call, rxrpc_call_put_recvmsg);
+ rxrpc_put_call(call, rxrpc_call_put_recvmsg);
goto try_again;
}
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index 7d903f060743..3b7ae280d32e 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -148,10 +148,15 @@ static void offload_action_hw_count_dec(struct tc_action *act,
static unsigned int tcf_offload_act_num_actions_single(struct tc_action *act)
{
- if (is_tcf_pedit(act))
- return tcf_pedit_nkeys(act);
- else
- return 1;
+ unsigned int count;
+
+ if (is_tcf_pedit(act)) {
+ spin_lock_bh(&act->tcfa_lock);
+ count = tcf_pedit_nkeys_locked(act);
+ spin_unlock_bh(&act->tcfa_lock);
+ return count;
+ }
+ return 1;
}
static bool tc_act_skip_hw(u32 flags)
diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c
index c2b5bc19e091..284800b7d3c5 100644
--- a/net/sched/act_bpf.c
+++ b/net/sched/act_bpf.c
@@ -44,7 +44,7 @@ TC_INDIRECT_SCOPE int tcf_bpf_act(struct sk_buff *skb,
tcf_lastuse_update(&prog->tcf_tm);
bstats_update(this_cpu_ptr(prog->common.cpu_bstats), skb);
- filter = rcu_dereference(prog->filter);
+ filter = rcu_dereference_bh(prog->filter);
if (at_ingress) {
__skb_push(skb, skb->mac_len);
filter_res = bpf_prog_run_data_pointers(filter, skb);
diff --git a/net/sched/act_ct.c b/net/sched/act_ct.c
index c82755749211..a7bf52369cae 100644
--- a/net/sched/act_ct.c
+++ b/net/sched/act_ct.c
@@ -842,11 +842,11 @@ static int tcf_ct_handle_fragments(struct net *net, struct sk_buff *skb,
u8 family, u16 zone, bool *defrag)
{
enum ip_conntrack_info ctinfo;
+ struct tc_skb_cb cb;
struct nf_conn *ct;
int err = 0;
bool frag;
u8 proto;
- u16 mru;
/* Previously seen (loopback)? Ignore. */
ct = nf_ct_get(skb, &ctinfo);
@@ -860,12 +860,13 @@ static int tcf_ct_handle_fragments(struct net *net, struct sk_buff *skb,
if (err || !frag)
return err;
- err = nf_ct_handle_fragments(net, skb, zone, family, &proto, &mru);
+ cb = *tc_skb_cb(skb);
+ err = nf_ct_handle_fragments(net, skb, zone, family, &proto, &cb.mru);
if (err)
return err;
*defrag = true;
- tc_skb_cb(skb)->mru = mru;
+ *tc_skb_cb(skb) = cb;
return 0;
}
@@ -1294,7 +1295,8 @@ static int tcf_ct_fill_params(struct net *net,
if (tb[TCA_CT_ZONE]) {
if (!IS_ENABLED(CONFIG_NF_CONNTRACK_ZONES)) {
NL_SET_ERR_MSG_MOD(extack, "Conntrack zones isn't enabled.");
- return -EOPNOTSUPP;
+ err = -EOPNOTSUPP;
+ goto err;
}
tcf_ct_set_key_val(tb,
@@ -1307,7 +1309,8 @@ static int tcf_ct_fill_params(struct net *net,
tmpl = nf_ct_tmpl_alloc(net, &zone, GFP_KERNEL);
if (!tmpl) {
NL_SET_ERR_MSG_MOD(extack, "Failed to allocate conntrack template");
- return -ENOMEM;
+ err = -ENOMEM;
+ goto err;
}
p->tmpl = tmpl;
if (tb[TCA_CT_HELPER_NAME]) {
diff --git a/net/sched/act_pedit.c b/net/sched/act_pedit.c
index c0a5f5d78dac..5aa91fb145a2 100644
--- a/net/sched/act_pedit.c
+++ b/net/sched/act_pedit.c
@@ -563,9 +563,18 @@ static int tcf_pedit_offload_act_setup(struct tc_action *act, void *entry_data,
{
if (bind) {
struct flow_action_entry *entry = entry_data;
+ int nkeys = tcf_pedit_nkeys_locked(act);
int k;
- for (k = 0; k < tcf_pedit_nkeys(act); k++) {
+ /* If the required keys exceed the remaining capacity return
+ * -ENOSPC to abort the offload and fallback to software.
+ */
+ if (nkeys > *index_inc) {
+ NL_SET_ERR_MSG_MOD(extack, "Not enough space to offload all pedit keys");
+ return -ENOSPC;
+ }
+
+ for (k = 0; k < nkeys; k++) {
switch (tcf_pedit_cmd(act, k)) {
case TCA_PEDIT_KEY_EX_CMD_SET:
entry->id = FLOW_ACTION_MANGLE;
@@ -602,7 +611,7 @@ static int tcf_pedit_offload_act_setup(struct tc_action *act, void *entry_data,
return -EOPNOTSUPP;
}
- for (k = 1; k < tcf_pedit_nkeys(act); k++) {
+ for (k = 1; k < tcf_pedit_nkeys_locked(act); k++) {
if (cmd != tcf_pedit_cmd(act, k)) {
NL_SET_ERR_MSG_MOD(extack, "Unsupported pedit command offload");
return -EOPNOTSUPP;
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index 24c1d0480bc5..c5aeff709ca6 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -3876,12 +3876,21 @@ int tc_setup_action(struct flow_action *flow_action,
entry = &flow_action->entries[j];
spin_lock_bh(&act->tcfa_lock);
+
+ /* Abort the offload if we have exhausted the allocated capacity */
+ if (j >= flow_action->num_entries) {
+ NL_SET_ERR_MSG_MOD(extack, "Flow action buffer overflow");
+ err = -ENOSPC;
+ goto err_out_locked;
+ }
+
err = tcf_act_get_user_cookie(entry, act);
if (err)
goto err_out_locked;
- index = 0;
- err = tc_setup_offload_act(act, entry, &index, extack);
+ index = flow_action->num_entries - j;
+ err = tc_setup_offload_act(act, entry, &index,
+ extack);
if (err)
goto err_out_locked;
@@ -3935,10 +3944,13 @@ unsigned int tcf_exts_num_actions(struct tcf_exts *exts)
int i;
tcf_exts_for_each_action(i, act, exts) {
- if (is_tcf_pedit(act))
- num_acts += tcf_pedit_nkeys(act);
- else
+ if (is_tcf_pedit(act)) {
+ spin_lock_bh(&act->tcfa_lock);
+ num_acts += tcf_pedit_nkeys_locked(act);
+ spin_unlock_bh(&act->tcfa_lock);
+ } else {
num_acts++;
+ }
}
return num_acts;
}
@@ -4039,6 +4051,9 @@ struct sk_buff *tcf_qevent_handle(struct tcf_qevent *qe, struct Qdisc *sch, stru
skb_do_redirect(skb);
*ret = __NET_XMIT_STOLEN;
return NULL;
+ case TC_ACT_CONSUMED:
+ *ret = __NET_XMIT_STOLEN;
+ return NULL;
}
return skb;
diff --git a/net/sched/cls_bpf.c b/net/sched/cls_bpf.c
index a32754a2658b..e888d0aa9f21 100644
--- a/net/sched/cls_bpf.c
+++ b/net/sched/cls_bpf.c
@@ -142,7 +142,8 @@ static bool cls_bpf_is_ebpf(const struct cls_bpf_prog *prog)
static int cls_bpf_offload_cmd(struct tcf_proto *tp, struct cls_bpf_prog *prog,
struct cls_bpf_prog *oldprog,
- struct netlink_ext_ack *extack)
+ struct netlink_ext_ack *extack,
+ bool is_rollback)
{
struct tcf_block *block = tp->chain->block;
struct tc_cls_bpf_offload cls_bpf = {};
@@ -177,7 +178,8 @@ static int cls_bpf_offload_cmd(struct tcf_proto *tp, struct cls_bpf_prog *prog,
&oldprog->in_hw_count, true);
if (prog && err) {
- cls_bpf_offload_cmd(tp, oldprog, prog, extack);
+ if (!is_rollback)
+ cls_bpf_offload_cmd(tp, oldprog, prog, extack, true);
return err;
}
@@ -208,7 +210,7 @@ static int cls_bpf_offload(struct tcf_proto *tp, struct cls_bpf_prog *prog,
if (!prog && !oldprog)
return 0;
- return cls_bpf_offload_cmd(tp, prog, oldprog, extack);
+ return cls_bpf_offload_cmd(tp, prog, oldprog, extack, false);
}
static void cls_bpf_stop_offload(struct tcf_proto *tp,
@@ -217,7 +219,7 @@ static void cls_bpf_stop_offload(struct tcf_proto *tp,
{
int err;
- err = cls_bpf_offload_cmd(tp, NULL, prog, extack);
+ err = cls_bpf_offload_cmd(tp, NULL, prog, extack, false);
if (err)
pr_err("Stopping hardware offload failed: %d\n", err);
}
diff --git a/net/sched/cls_flow.c b/net/sched/cls_flow.c
index 7eeead60ec23..a8b900a27d2f 100644
--- a/net/sched/cls_flow.c
+++ b/net/sched/cls_flow.c
@@ -21,6 +21,7 @@
#include <net/inet_sock.h>
#include <net/pkt_cls.h>
+#include <linux/siphash.h>
#include <net/ip.h>
#include <net/route.h>
#include <net/flow_dissector.h>
@@ -57,11 +58,15 @@ struct flow_filter {
struct rcu_work rwork;
};
+static siphash_aligned_key_t flow_keys_secret __read_mostly;
+
static inline u32 addr_fold(void *addr)
{
- unsigned long a = (unsigned long)addr;
-
- return (a & 0xFFFFFFFF) ^ (BITS_PER_LONG > 32 ? a >> 32 : 0);
+#ifdef CONFIG_64BIT
+ return (u32)siphash_1u64((u64)addr, &flow_keys_secret);
+#else
+ return (u32)siphash_1u32((u32)addr, &flow_keys_secret);
+#endif
}
static u32 flow_get_src(const struct sk_buff *skb, const struct flow_keys *flow)
@@ -596,6 +601,7 @@ static int flow_init(struct tcf_proto *tp)
return -ENOBUFS;
INIT_LIST_HEAD(&head->filters);
rcu_assign_pointer(tp->root, head);
+ net_get_random_once(&flow_keys_secret, sizeof(flow_keys_secret));
return 0;
}
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index ba1a5c15e6ca..2f33739036bb 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -1365,10 +1365,7 @@ static u32 cake_calc_overhead(struct cake_sched_data *q, u32 len, u32 off)
if (q->min_netlen > len)
q->min_netlen = len;
- len += q->rate_overhead;
-
- if (len < q->rate_mpu)
- len = q->rate_mpu;
+ len = max((s32)len + q->rate_overhead, (s32)q->rate_mpu);
if (q->atm_mode == CAKE_ATM_ATM) {
len += 47;
diff --git a/net/sched/sch_codel.c b/net/sched/sch_codel.c
index afd9805cb68e..29f5a98a3bab 100644
--- a/net/sched/sch_codel.c
+++ b/net/sched/sch_codel.c
@@ -56,7 +56,7 @@ static void drop_func(struct sk_buff *skb, void *ctx)
qdisc_qstats_drop(sch);
}
-static struct sk_buff *codel_qdisc_dequeue(struct Qdisc *sch)
+static struct sk_buff *__codel_qdisc_dequeue(struct Qdisc *sch)
{
struct codel_sched_data *q = qdisc_priv(sch);
struct sk_buff *skb;
@@ -65,13 +65,51 @@ static struct sk_buff *codel_qdisc_dequeue(struct Qdisc *sch)
&q->stats, qdisc_pkt_len, codel_get_enqueue_time,
drop_func, dequeue_func);
+ if (skb)
+ qdisc_bstats_update(sch, skb);
+ return skb;
+}
+
+static void codel_dequeue_drop(struct Qdisc *sch)
+{
+ struct codel_sched_data *q = qdisc_priv(sch);
+
if (q->stats.drop_count) {
- qdisc_tree_reduce_backlog(sch, q->stats.drop_count, q->stats.drop_len);
+ qdisc_tree_reduce_backlog(sch, q->stats.drop_count,
+ q->stats.drop_len);
q->stats.drop_count = 0;
q->stats.drop_len = 0;
}
- if (skb)
- qdisc_bstats_update(sch, skb);
+}
+
+static struct sk_buff *codel_qdisc_dequeue(struct Qdisc *sch)
+{
+ struct sk_buff *skb;
+
+ skb = __codel_qdisc_dequeue(sch);
+
+ codel_dequeue_drop(sch);
+
+ return skb;
+}
+
+static struct sk_buff *codel_peek(struct Qdisc *sch)
+{
+ struct sk_buff *skb = skb_peek(&sch->gso_skb);
+
+ if (!skb) {
+ skb = __codel_qdisc_dequeue(sch);
+
+ if (skb) {
+ __skb_queue_head(&sch->gso_skb, skb);
+ /* it's still part of the queue */
+ qdisc_qstats_backlog_inc(sch, skb);
+ sch->q.qlen++;
+ }
+
+ codel_dequeue_drop(sch);
+ }
+
return skb;
}
@@ -253,7 +291,7 @@ static struct Qdisc_ops codel_qdisc_ops __read_mostly = {
.enqueue = codel_qdisc_enqueue,
.dequeue = codel_qdisc_dequeue,
- .peek = qdisc_peek_dequeued,
+ .peek = codel_peek,
.init = codel_init,
.reset = codel_reset,
.change = codel_change,
diff --git a/net/sched/sch_drr.c b/net/sched/sch_drr.c
index 9b6d79bd8737..44686fd1c274 100644
--- a/net/sched/sch_drr.c
+++ b/net/sched/sch_drr.c
@@ -270,7 +270,7 @@ static int drr_dump_class_stats(struct Qdisc *sch, unsigned long arg,
memset(&xstats, 0, sizeof(xstats));
if (qlen)
- xstats.deficit = cl->deficit;
+ xstats.deficit = READ_ONCE(cl->deficit);
if (gnet_stats_copy_basic(d, NULL, &cl->bstats, true) < 0 ||
gnet_stats_copy_rate_est(d, &cl->rate_est) < 0 ||
@@ -362,7 +362,7 @@ static int drr_enqueue(struct sk_buff *skb, struct Qdisc *sch,
if (!cl_is_active(cl)) {
list_add_tail(&cl->alist, &q->active);
- cl->deficit = cl->quantum;
+ WRITE_ONCE(cl->deficit, cl->quantum);
}
sch->qstats.backlog += len;
@@ -389,7 +389,7 @@ static struct sk_buff *drr_dequeue(struct Qdisc *sch)
len = qdisc_pkt_len(skb);
if (len <= cl->deficit) {
- cl->deficit -= len;
+ WRITE_ONCE(cl->deficit, cl->deficit - len);
skb = qdisc_dequeue_peeked(cl->qdisc);
if (unlikely(skb == NULL))
goto out;
@@ -403,7 +403,7 @@ static struct sk_buff *drr_dequeue(struct Qdisc *sch)
return skb;
}
- cl->deficit += cl->quantum;
+ WRITE_ONCE(cl->deficit, cl->deficit + cl->quantum);
list_move_tail(&cl->alist, &q->active);
}
out:
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index 3f797ec4b0c2..f445507eff7a 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -278,7 +278,7 @@ static void drop_func(struct sk_buff *skb, void *ctx)
qdisc_qstats_drop(sch);
}
-static struct sk_buff *fq_codel_dequeue(struct Qdisc *sch)
+static struct sk_buff *__fq_codel_dequeue(struct Qdisc *sch)
{
struct fq_codel_sched_data *q = qdisc_priv(sch);
struct sk_buff *skb;
@@ -315,12 +315,49 @@ static struct sk_buff *fq_codel_dequeue(struct Qdisc *sch)
qdisc_bstats_update(sch, skb);
flow->deficit -= qdisc_pkt_len(skb);
+ return skb;
+}
+
+static void fq_codel_dequeue_drop(struct Qdisc *sch)
+{
+ struct fq_codel_sched_data *q = qdisc_priv(sch);
+
if (q->cstats.drop_count) {
qdisc_tree_reduce_backlog(sch, q->cstats.drop_count,
q->cstats.drop_len);
q->cstats.drop_count = 0;
q->cstats.drop_len = 0;
}
+}
+
+static struct sk_buff *fq_codel_dequeue(struct Qdisc *sch)
+{
+ struct sk_buff *skb;
+
+ skb = __fq_codel_dequeue(sch);
+
+ fq_codel_dequeue_drop(sch);
+
+ return skb;
+}
+
+static struct sk_buff *fq_codel_peek(struct Qdisc *sch)
+{
+ struct sk_buff *skb = skb_peek(&sch->gso_skb);
+
+ if (!skb) {
+ skb = __fq_codel_dequeue(sch);
+
+ if (skb) {
+ __skb_queue_head(&sch->gso_skb, skb);
+ /* it's still part of the queue */
+ qdisc_qstats_backlog_inc(sch, skb);
+ sch->q.qlen++;
+ }
+
+ fq_codel_dequeue_drop(sch);
+ }
+
return skb;
}
@@ -720,7 +757,7 @@ static struct Qdisc_ops fq_codel_qdisc_ops __read_mostly = {
.priv_size = sizeof(struct fq_codel_sched_data),
.enqueue = fq_codel_enqueue,
.dequeue = fq_codel_dequeue,
- .peek = qdisc_peek_dequeued,
+ .peek = fq_codel_peek,
.init = fq_codel_init,
.reset = fq_codel_reset,
.destroy = fq_codel_destroy,
diff --git a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c
index 57221522fe56..601f99dd989a 100644
--- a/net/sched/sch_hfsc.c
+++ b/net/sched/sch_hfsc.c
@@ -715,7 +715,7 @@ init_vf(struct hfsc_class *cl, unsigned int len)
rtsc_min(&cl->cl_virtual, &cl->cl_fsc, cl->cl_vt, cl->cl_total);
cl->cl_vtadj = 0;
- cl->cl_vtperiod++; /* increment vt period */
+ WRITE_ONCE(cl->cl_vtperiod, cl->cl_vtperiod + 1); /* increment vt period */
cl->cl_parentperiod = cl->cl_parent->cl_vtperiod;
if (cl->cl_parent->cl_nactive == 0)
cl->cl_parentperiod++;
@@ -753,11 +753,11 @@ update_vf(struct hfsc_class *cl, unsigned int len, u64 cur_time)
u64 f; /* , myf_bound, delta; */
int go_passive = 0;
- if (cl->qdisc->q.qlen == 0 && cl->cl_flags & HFSC_FSC)
+ if (cl->qdisc->q.qlen == 0 && cl->cl_flags & HFSC_FSC && cl->cl_nactive)
go_passive = 1;
for (; cl->cl_parent != NULL; cl = cl->cl_parent) {
- cl->cl_total += len;
+ WRITE_ONCE(cl->cl_total, cl->cl_total + len);
if (!(cl->cl_flags & HFSC_FSC) || cl->cl_nactive == 0)
continue;
@@ -847,7 +847,7 @@ hfsc_adjust_levels(struct hfsc_class *cl)
if (p->level >= level)
level = p->level + 1;
}
- cl->level = level;
+ WRITE_ONCE(cl->level, level);
} while ((cl = cl->cl_parent) != NULL);
}
@@ -1338,10 +1338,10 @@ hfsc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
__u32 qlen;
qdisc_qstats_qlen_backlog(cl->qdisc, &qlen, &cl->qstats.backlog);
- xstats.level = cl->level;
- xstats.period = cl->cl_vtperiod;
- xstats.work = cl->cl_total;
- xstats.rtwork = cl->cl_cumul;
+ xstats.level = READ_ONCE(cl->level);
+ xstats.period = READ_ONCE(cl->cl_vtperiod);
+ xstats.work = READ_ONCE(cl->cl_total);
+ xstats.rtwork = READ_ONCE(cl->cl_cumul);
if (gnet_stats_copy_basic(d, NULL, &cl->bstats, true) < 0 ||
gnet_stats_copy_rate_est(d, &cl->rate_est) < 0 ||
@@ -1452,15 +1452,15 @@ hfsc_change_qdisc(struct Qdisc *sch, struct nlattr *opt,
static void
hfsc_reset_class(struct hfsc_class *cl)
{
- cl->cl_total = 0;
- cl->cl_cumul = 0;
+ WRITE_ONCE(cl->cl_total, 0);
+ WRITE_ONCE(cl->cl_cumul, 0);
cl->cl_d = 0;
cl->cl_e = 0;
cl->cl_vt = 0;
cl->cl_vtadj = 0;
cl->cl_cvtmin = 0;
cl->cl_cvtoff = 0;
- cl->cl_vtperiod = 0;
+ WRITE_ONCE(cl->cl_vtperiod, 0);
cl->cl_parentperiod = 0;
cl->cl_f = 0;
cl->cl_myf = 0;
@@ -1626,7 +1626,7 @@ hfsc_dequeue(struct Qdisc *sch)
bstats_update(&cl->bstats, skb);
update_vf(cl, qdisc_pkt_len(skb), cur_time);
if (realtime)
- cl->cl_cumul += qdisc_pkt_len(skb);
+ WRITE_ONCE(cl->cl_cumul, cl->cl_cumul + qdisc_pkt_len(skb));
if (cl->cl_flags & HFSC_RSC) {
if (cl->qdisc->q.qlen != 0) {
diff --git a/net/sched/sch_hhf.c b/net/sched/sch_hhf.c
index 914e98542774..f9d2cf7533fa 100644
--- a/net/sched/sch_hhf.c
+++ b/net/sched/sch_hhf.c
@@ -461,12 +461,39 @@ static struct sk_buff *hhf_dequeue(struct Qdisc *sch)
return skb;
}
+static void hhf_reset_classifier(struct hhf_sched_data *q)
+{
+ int i;
+
+ if (!q->hh_flows)
+ return;
+
+ for (i = 0; i < HH_FLOWS_CNT; i++) {
+ struct hh_flow_state *flow, *next;
+ struct list_head *head = &q->hh_flows[i];
+
+ list_for_each_entry_safe(flow, next, head, flowchain) {
+ list_del(&flow->flowchain);
+ kfree(flow);
+ }
+ }
+ WRITE_ONCE(q->hh_flows_current_cnt, 0);
+
+ for (i = 0; i < HHF_ARRAYS_CNT; i++) {
+ if (q->hhf_valid_bits[i])
+ bitmap_zero(q->hhf_valid_bits[i], HHF_ARRAYS_LEN);
+ }
+ q->hhf_arrays_reset_timestamp = hhf_time_stamp();
+}
+
static void hhf_reset(struct Qdisc *sch)
{
+ struct hhf_sched_data *q = qdisc_priv(sch);
struct sk_buff *skb;
while ((skb = hhf_dequeue(sch)) != NULL)
rtnl_kfree_skbs(skb, skb);
+ hhf_reset_classifier(q);
}
static void hhf_destroy(struct Qdisc *sch)
diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index 2c13de8bf16f..49f4599be14b 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -568,7 +568,7 @@ htb_change_class_mode(struct htb_sched *q, struct htb_class *cl, s64 *diff)
if (new_mode == HTB_CANT_SEND) {
cl->overlimits++;
- q->overlimits++;
+ WRITE_ONCE(q->overlimits, q->overlimits + 1);
}
if (cl->prio_activity) { /* not necessary: speed optimization */
@@ -628,7 +628,7 @@ static int htb_enqueue(struct sk_buff *skb, struct Qdisc *sch,
/* enqueue to helper queue */
if (q->direct_queue.qlen < q->direct_qlen) {
__qdisc_enqueue_tail(skb, &q->direct_queue);
- q->direct_pkts++;
+ WRITE_ONCE(q->direct_pkts, q->direct_pkts + 1);
} else {
return qdisc_drop(skb, sch, to_free);
}
@@ -1148,6 +1148,7 @@ static int htb_init(struct Qdisc *sch, struct nlattr *opt,
* parts (especially calling ndo_setup_tc) on errors.
*/
q->offload = true;
+ sch->flags |= TCQ_F_OFFLOADED;
return 0;
}
@@ -1208,17 +1209,12 @@ static int htb_dump(struct Qdisc *sch, struct sk_buff *skb)
struct nlattr *nest;
struct tc_htb_glob gopt;
- if (q->offload)
- sch->flags |= TCQ_F_OFFLOADED;
- else
- sch->flags &= ~TCQ_F_OFFLOADED;
-
- sch->qstats.overlimits = q->overlimits;
+ sch->qstats.overlimits = READ_ONCE(q->overlimits);
/* Its safe to not acquire qdisc lock. As we hold RTNL,
* no change can happen on the qdisc parameters.
*/
- gopt.direct_pkts = q->direct_pkts;
+ gopt.direct_pkts = READ_ONCE(q->direct_pkts);
gopt.version = HTB_VER;
gopt.rate2quantum = q->rate2quantum;
gopt.defcls = q->defcls;
diff --git a/net/sched/sch_multiq.c b/net/sched/sch_multiq.c
index 06e03f5cd7ce..e50a37b7cec1 100644
--- a/net/sched/sch_multiq.c
+++ b/net/sched/sch_multiq.c
@@ -103,7 +103,7 @@ static struct sk_buff *multiq_dequeue(struct Qdisc *sch)
if (!netif_xmit_stopped(
netdev_get_tx_queue(qdisc_dev(sch), q->curband))) {
qdisc = q->queues[q->curband];
- skb = qdisc->dequeue(qdisc);
+ skb = qdisc_dequeue_peeked(qdisc);
if (skb) {
qdisc_bstats_update(sch, skb);
sch->q.qlen--;
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index f1709efb5f04..5091ffc3f475 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -748,7 +748,7 @@ static struct sk_buff *taprio_dequeue_from_txq(struct Qdisc *sch, int txq,
return NULL;
skip_peek_checks:
- skb = child->ops->dequeue(child);
+ skb = qdisc_dequeue_peeked(child);
if (unlikely(!skb))
return NULL;
diff --git a/net/sched/sch_teql.c b/net/sched/sch_teql.c
index ec4039a201a2..4d463ba5ce72 100644
--- a/net/sched/sch_teql.c
+++ b/net/sched/sch_teql.c
@@ -52,7 +52,8 @@
struct teql_master {
struct Qdisc_ops qops;
struct net_device *dev;
- struct Qdisc *slaves;
+ struct Qdisc __rcu *slaves;
+ spinlock_t slaves_lock; /* serializes writes to ->slaves */
struct list_head master_list;
unsigned long tx_bytes;
unsigned long tx_packets;
@@ -61,7 +62,7 @@ struct teql_master {
};
struct teql_sched_data {
- struct Qdisc *next;
+ struct Qdisc __rcu *next;
struct teql_master *m;
struct sk_buff_head q;
};
@@ -101,7 +102,9 @@ teql_dequeue(struct Qdisc *sch)
if (skb == NULL) {
struct net_device *m = qdisc_dev(q);
if (m) {
- dat->m->slaves = sch;
+ spin_lock_bh(&dat->m->slaves_lock);
+ rcu_assign_pointer(dat->m->slaves, sch);
+ spin_unlock_bh(&dat->m->slaves_lock);
netif_wake_queue(m);
}
} else {
@@ -132,34 +135,49 @@ teql_destroy(struct Qdisc *sch)
struct Qdisc *q, *prev;
struct teql_sched_data *dat = qdisc_priv(sch);
struct teql_master *master = dat->m;
+ struct netdev_queue *txq = NULL;
+ bool reset_master_queue = false;
if (!master)
return;
- prev = master->slaves;
+ spin_lock_bh(&master->slaves_lock);
+ prev = rcu_dereference_protected(master->slaves,
+ lockdep_is_held(&master->slaves_lock));
if (prev) {
do {
- q = NEXT_SLAVE(prev);
- if (q == sch) {
- NEXT_SLAVE(prev) = NEXT_SLAVE(q);
- if (q == master->slaves) {
- master->slaves = NEXT_SLAVE(q);
- if (q == master->slaves) {
- struct netdev_queue *txq;
-
- txq = netdev_get_tx_queue(master->dev, 0);
- master->slaves = NULL;
-
- dev_reset_queue(master->dev,
- txq, NULL);
- }
- }
- skb_queue_purge(&dat->q);
- break;
+ struct Qdisc *head, *next;
+
+ q = rcu_dereference_protected(NEXT_SLAVE(prev),
+ lockdep_is_held(&master->slaves_lock));
+ if (q != sch) {
+ prev = q;
+ continue;
}
- } while ((prev = q) != master->slaves);
+ next = rcu_dereference_protected(NEXT_SLAVE(q),
+ lockdep_is_held(&master->slaves_lock));
+ rcu_assign_pointer(NEXT_SLAVE(prev), next);
+
+ head = rcu_dereference_protected(master->slaves,
+ lockdep_is_held(&master->slaves_lock));
+ if (q == head) {
+ rcu_assign_pointer(master->slaves, next);
+ if (q == next) {
+ txq = netdev_get_tx_queue(master->dev, 0);
+ rcu_assign_pointer(master->slaves, NULL);
+ reset_master_queue = true;
+ }
+ }
+ skb_queue_purge(&dat->q);
+ break;
+ } while (prev != rcu_dereference_protected(master->slaves,
+ lockdep_is_held(&master->slaves_lock)));
}
+ spin_unlock_bh(&master->slaves_lock);
+
+ if (reset_master_queue)
+ dev_reset_queue(master->dev, txq, NULL);
}
static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
@@ -168,6 +186,7 @@ static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
struct net_device *dev = qdisc_dev(sch);
struct teql_master *m = (struct teql_master *)sch->ops;
struct teql_sched_data *q = qdisc_priv(sch);
+ struct Qdisc *first;
if (dev->hard_header_len > m->dev->hard_header_len)
return -EINVAL;
@@ -184,7 +203,9 @@ static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
skb_queue_head_init(&q->q);
- if (m->slaves) {
+ spin_lock_bh(&m->slaves_lock);
+ first = rcu_dereference_protected(m->slaves, lockdep_is_held(&m->slaves_lock));
+ if (first) {
if (m->dev->flags & IFF_UP) {
if ((m->dev->flags & IFF_POINTOPOINT &&
!(dev->flags & IFF_POINTOPOINT)) ||
@@ -192,8 +213,10 @@ static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
!(dev->flags & IFF_BROADCAST)) ||
(m->dev->flags & IFF_MULTICAST &&
!(dev->flags & IFF_MULTICAST)) ||
- dev->mtu < m->dev->mtu)
+ dev->mtu < m->dev->mtu) {
+ spin_unlock_bh(&m->slaves_lock);
return -EINVAL;
+ }
} else {
if (!(dev->flags&IFF_POINTOPOINT))
m->dev->flags &= ~IFF_POINTOPOINT;
@@ -204,14 +227,17 @@ static int teql_qdisc_init(struct Qdisc *sch, struct nlattr *opt,
if (dev->mtu < m->dev->mtu)
m->dev->mtu = dev->mtu;
}
- q->next = NEXT_SLAVE(m->slaves);
- NEXT_SLAVE(m->slaves) = sch;
+ rcu_assign_pointer(q->next,
+ rcu_dereference_protected(NEXT_SLAVE(first),
+ lockdep_is_held(&m->slaves_lock)));
+ rcu_assign_pointer(NEXT_SLAVE(first), sch);
} else {
- q->next = sch;
- m->slaves = sch;
+ rcu_assign_pointer(q->next, sch);
+ rcu_assign_pointer(m->slaves, sch);
m->dev->mtu = dev->mtu;
m->dev->flags = (m->dev->flags&~FMASK)|(dev->flags&FMASK);
}
+ spin_unlock_bh(&m->slaves_lock);
return 0;
}
@@ -285,12 +311,14 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
int subq = skb_get_queue_mapping(skb);
struct sk_buff *skb_res = NULL;
- start = master->slaves;
-
restart:
nores = 0;
busy = 0;
+ rcu_read_lock();
+
+ start = rcu_dereference(master->slaves);
+
q = start;
if (!q)
goto drop;
@@ -317,10 +345,17 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
netdev_start_xmit(skb, slave, slave_txq, false) ==
NETDEV_TX_OK) {
__netif_tx_unlock(slave_txq);
- master->slaves = NEXT_SLAVE(q);
+ spin_lock(&master->slaves_lock);
+ if (rcu_dereference_protected(master->slaves,
+ lockdep_is_held(&master->slaves_lock)) == q)
+ rcu_assign_pointer(master->slaves,
+ rcu_dereference_protected(NEXT_SLAVE(q),
+ lockdep_is_held(&master->slaves_lock)));
+ spin_unlock(&master->slaves_lock);
netif_wake_queue(dev);
master->tx_packets++;
master->tx_bytes += length;
+ rcu_read_unlock();
return NETDEV_TX_OK;
}
__netif_tx_unlock(slave_txq);
@@ -329,45 +364,56 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
busy = 1;
break;
case 1:
- master->slaves = NEXT_SLAVE(q);
+ spin_lock(&master->slaves_lock);
+ if (rcu_dereference_protected(master->slaves,
+ lockdep_is_held(&master->slaves_lock)) == q)
+ rcu_assign_pointer(master->slaves,
+ rcu_dereference_protected(NEXT_SLAVE(q),
+ lockdep_is_held(&master->slaves_lock)));
+ spin_unlock(&master->slaves_lock);
+ rcu_read_unlock();
return NETDEV_TX_OK;
default:
nores = 1;
break;
}
__skb_pull(skb, skb_network_offset(skb));
- } while ((q = NEXT_SLAVE(q)) != start);
+ } while ((q = rcu_dereference(NEXT_SLAVE(q))) != start);
if (nores && skb_res == NULL) {
skb_res = skb;
+ rcu_read_unlock();
goto restart;
}
if (busy) {
netif_stop_queue(dev);
+ rcu_read_unlock();
return NETDEV_TX_BUSY;
}
master->tx_errors++;
drop:
master->tx_dropped++;
+ rcu_read_unlock();
dev_kfree_skb(skb);
return NETDEV_TX_OK;
}
static int teql_master_open(struct net_device *dev)
{
- struct Qdisc *q;
+ struct Qdisc *q, *first;
struct teql_master *m = netdev_priv(dev);
int mtu = 0xFFFE;
unsigned int flags = IFF_NOARP | IFF_MULTICAST;
- if (m->slaves == NULL)
+ first = rtnl_dereference(m->slaves);
+ if (!first)
return -EUNATCH;
flags = FMASK;
- q = m->slaves;
+ q = first;
do {
struct net_device *slave = qdisc_dev(q);
@@ -389,7 +435,7 @@ static int teql_master_open(struct net_device *dev)
flags &= ~IFF_BROADCAST;
if (!(slave->flags&IFF_MULTICAST))
flags &= ~IFF_MULTICAST;
- } while ((q = NEXT_SLAVE(q)) != m->slaves);
+ } while ((q = rtnl_dereference(NEXT_SLAVE(q))) != first);
m->dev->mtu = mtu;
m->dev->flags = (m->dev->flags&~FMASK) | flags;
@@ -417,14 +463,15 @@ static void teql_master_stats64(struct net_device *dev,
static int teql_master_mtu(struct net_device *dev, int new_mtu)
{
struct teql_master *m = netdev_priv(dev);
- struct Qdisc *q;
+ struct Qdisc *q, *first;
- q = m->slaves;
+ first = rtnl_dereference(m->slaves);
+ q = first;
if (q) {
do {
if (new_mtu > qdisc_dev(q)->mtu)
return -EINVAL;
- } while ((q = NEXT_SLAVE(q)) != m->slaves);
+ } while ((q = rtnl_dereference(NEXT_SLAVE(q))) != first);
}
WRITE_ONCE(dev->mtu, new_mtu);
@@ -444,6 +491,7 @@ static __init void teql_master_setup(struct net_device *dev)
struct teql_master *master = netdev_priv(dev);
struct Qdisc_ops *ops = &master->qops;
+ spin_lock_init(&master->slaves_lock);
master->dev = dev;
ops->priv_size = sizeof(struct teql_sched_data);
diff --git a/net/sctp/diag.c b/net/sctp/diag.c
index ff4f8a679ffc..f34fd6812392 100644
--- a/net/sctp/diag.c
+++ b/net/sctp/diag.c
@@ -92,6 +92,7 @@ static int inet_diag_msg_sctpladdrs_fill(struct sk_buff *skb,
if (!--addrcnt)
break;
}
+ WARN_ON_ONCE(addrcnt);
rcu_read_unlock();
return 0;
@@ -373,42 +374,39 @@ static int sctp_ep_dump(struct sctp_endpoint *ep, void *p)
struct sk_buff *skb = commp->skb;
struct netlink_callback *cb = commp->cb;
const struct inet_diag_req_v2 *r = commp->r;
- struct net *net = sock_net(skb->sk);
struct inet_sock *inet = inet_sk(sk);
int err = 0;
- if (!net_eq(sock_net(sk), net))
+ lock_sock(sk);
+ if (ep->base.dead)
goto out;
- if (cb->args[4] < cb->args[1])
- goto next;
-
- if (!(r->idiag_states & TCPF_LISTEN) && !list_empty(&ep->asocs))
- goto next;
+ /* Skip eps with assocs if non-LISTEN states were requested, since
+ * they'll be dumped by sctp_sock_dump() during assoc traversal.
+ */
+ if ((r->idiag_states & ~(TCPF_LISTEN | TCPF_CLOSE)) &&
+ !list_empty(&ep->asocs))
+ goto out;
if (r->sdiag_family != AF_UNSPEC &&
sk->sk_family != r->sdiag_family)
- goto next;
+ goto out;
if (r->id.idiag_sport != inet->inet_sport &&
r->id.idiag_sport)
- goto next;
+ goto out;
if (r->id.idiag_dport != inet->inet_dport &&
r->id.idiag_dport)
- goto next;
-
- if (inet_sctp_diag_fill(sk, NULL, skb, r,
- sk_user_ns(NETLINK_CB(cb->skb).sk),
- NETLINK_CB(cb->skb).portid,
- cb->nlh->nlmsg_seq, NLM_F_MULTI,
- cb->nlh, commp->net_admin) < 0) {
- err = 2;
goto out;
- }
-next:
- cb->args[4]++;
+
+ err = inet_sctp_diag_fill(sk, NULL, skb, r,
+ sk_user_ns(NETLINK_CB(cb->skb).sk),
+ NETLINK_CB(cb->skb).portid,
+ cb->nlh->nlmsg_seq, NLM_F_MULTI,
+ cb->nlh, commp->net_admin);
out:
+ release_sock(sk);
return err;
}
@@ -479,41 +477,40 @@ static void sctp_diag_dump(struct sk_buff *skb, struct netlink_callback *cb,
.r = r,
.net_admin = netlink_net_capable(cb->skb, CAP_NET_ADMIN),
};
- int pos = cb->args[2];
+ int pos;
/* eps hashtable dumps
* args:
* 0 : if it will traversal listen sock
* 1 : to record the sock pos of this time's traversal
- * 4 : to work as a temporary variable to traversal list
*/
if (cb->args[0] == 0) {
- if (!(idiag_states & TCPF_LISTEN))
- goto skip;
- if (sctp_for_each_endpoint(sctp_ep_dump, &commp))
- goto done;
-skip:
+ if (idiag_states & TCPF_LISTEN) {
+ pos = cb->args[1];
+ if (sctp_for_each_endpoint(sctp_ep_dump, net, &pos,
+ &commp)) {
+ cb->args[1] = pos;
+ return;
+ }
+ }
cb->args[0] = 1;
cb->args[1] = 0;
- cb->args[4] = 0;
}
+ if (!(idiag_states & ~(TCPF_LISTEN | TCPF_CLOSE)))
+ return;
+
/* asocs by transport hashtable dump
* args:
* 1 : to record the assoc pos of this time's traversal
* 2 : to record the transport pos of this time's traversal
* 3 : to mark if we have dumped the ep info of the current asoc
- * 4 : to work as a temporary variable to traversal list
- * 5 : to save the sk we get from travelsing the tsp list.
+ * 4 : to track position within ep->asocs list in sctp_sock_dump()
*/
- if (!(idiag_states & ~(TCPF_LISTEN | TCPF_CLOSE)))
- goto done;
-
+ pos = cb->args[2];
sctp_transport_traverse_process(sctp_sock_filter, sctp_sock_dump,
net, &pos, &commp);
cb->args[2] = pos;
-
-done:
cb->args[1] = cb->args[4];
cb->args[4] = 0;
}
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index 96ca400120e6..b812d5e9c2dd 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -2662,6 +2662,9 @@ static int sctp_process_param(struct sctp_association *asoc,
goto fall_through;
addr_param = param.v + sizeof(struct sctp_addip_param);
+ if (ntohs(addr_param->p.length) >
+ ntohs(param.p->length) - sizeof(struct sctp_addip_param))
+ break;
af = sctp_get_af_specific(param_type2af(addr_param->p.type));
if (!af)
@@ -3060,13 +3063,16 @@ static __be16 sctp_process_asconf_param(struct sctp_association *asoc,
union sctp_addr addr;
struct sctp_af *af;
- addr_param = (void *)asconf_param + sizeof(*asconf_param);
-
if (asconf_param->param_hdr.type != SCTP_PARAM_ADD_IP &&
asconf_param->param_hdr.type != SCTP_PARAM_DEL_IP &&
asconf_param->param_hdr.type != SCTP_PARAM_SET_PRIMARY)
return SCTP_ERROR_UNKNOWN_PARAM;
+ addr_param = (void *)asconf_param + sizeof(*asconf_param);
+ if (ntohs(addr_param->p.length) >
+ ntohs(asconf_param->param_hdr.length) - sizeof(*asconf_param))
+ return SCTP_ERROR_PROTO_VIOLATION;
+
switch (addr_param->p.type) {
case SCTP_PARAM_IPV6_ADDRESS:
if (!asoc->peer.ipv6_address)
diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c
index 613c5c3fa846..96e92719ff4d 100644
--- a/net/sctp/sm_statefuns.c
+++ b/net/sctp/sm_statefuns.c
@@ -74,7 +74,8 @@ static enum sctp_disposition sctp_sf_do_5_2_6_stale(
const struct sctp_association *asoc,
const union sctp_subtype type,
void *arg,
- struct sctp_cmd_seq *commands);
+ struct sctp_cmd_seq *commands,
+ struct sctp_errhdr *err);
static enum sctp_disposition sctp_sf_shut_8_4_5(
struct net *net,
const struct sctp_endpoint *ep,
@@ -2494,9 +2495,15 @@ enum sctp_disposition sctp_sf_cookie_echoed_err(
* errors.
*/
sctp_walk_errors(err, chunk->chunk_hdr) {
- if (SCTP_ERROR_STALE_COOKIE == err->cause)
- return sctp_sf_do_5_2_6_stale(net, ep, asoc, type,
- arg, commands);
+ if (err->cause != SCTP_ERROR_STALE_COOKIE)
+ continue;
+ /* The staleness is only meaningful if the cause is long
+ * enough to hold it; a shorter one is malformed.
+ */
+ if (ntohs(err->length) < sizeof(*err) + sizeof(__be32))
+ break;
+ return sctp_sf_do_5_2_6_stale(net, ep, asoc, type,
+ arg, commands, err);
}
/* It is possible to have malformed error causes, and that
@@ -2538,13 +2545,13 @@ static enum sctp_disposition sctp_sf_do_5_2_6_stale(
const struct sctp_association *asoc,
const union sctp_subtype type,
void *arg,
- struct sctp_cmd_seq *commands)
+ struct sctp_cmd_seq *commands,
+ struct sctp_errhdr *err)
{
int attempts = asoc->init_err_counter + 1;
- struct sctp_chunk *chunk = arg, *reply;
struct sctp_cookie_preserve_param bht;
struct sctp_bind_addr *bp;
- struct sctp_errhdr *err;
+ struct sctp_chunk *reply;
u32 stale;
if (attempts > asoc->max_init_attempts) {
@@ -2555,8 +2562,6 @@ static enum sctp_disposition sctp_sf_do_5_2_6_stale(
return SCTP_DISPOSITION_DELETE_TCB;
}
- err = (struct sctp_errhdr *)(chunk->skb->data);
-
/* When calculating the time extension, an implementation
* SHOULD use the RTT information measured based on the
* previous COOKIE ECHO / ERROR exchange, and should add no
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index 3e80cf4e63ff..4e09c01c7109 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -4111,8 +4111,9 @@ static int sctp_setsockopt_reset_streams(struct sock *sk,
if (optlen < sizeof(*params))
return -EINVAL;
/* srs_number_streams is u16, so optlen can't be bigger than this. */
- optlen = min_t(unsigned int, optlen, USHRT_MAX +
- sizeof(__u16) * sizeof(*params));
+ optlen = min_t(unsigned int, optlen,
+ struct_size_t(struct sctp_reset_streams, srs_stream_list,
+ USHRT_MAX));
if (params->srs_number_streams * sizeof(__u16) >
optlen - sizeof(*params))
@@ -4598,8 +4599,8 @@ static int sctp_setsockopt(struct sock *sk, int level, int optname,
if (optlen > 0) {
/* Trim it to the biggest size sctp sockopt may need if necessary */
optlen = min_t(unsigned int, optlen,
- PAGE_ALIGN(USHRT_MAX +
- sizeof(__u16) * sizeof(struct sctp_reset_streams)));
+ PAGE_ALIGN(struct_size_t(struct sctp_reset_streams,
+ srs_stream_list, USHRT_MAX)));
kopt = memdup_sockptr(optval, optlen);
if (IS_ERR(kopt))
return PTR_ERR(kopt);
@@ -5323,24 +5324,39 @@ struct sctp_transport *sctp_transport_get_idx(struct net *net,
}
int sctp_for_each_endpoint(int (*cb)(struct sctp_endpoint *, void *),
- void *p) {
- int err = 0;
- int hash = 0;
- struct sctp_endpoint *ep;
+ struct net *net, int *pos, void *p) {
+ int err, hash = 0, idx = 0, start;
struct sctp_hashbucket *head;
+ struct sctp_endpoint *ep;
for (head = sctp_ep_hashtable; hash < sctp_ep_hashsize;
hash++, head++) {
+ start = idx;
+again:
read_lock_bh(&head->lock);
sctp_for_each_hentry(ep, &head->chain) {
- err = cb(ep, p);
- if (err)
+ if (sock_net(ep->base.sk) != net)
+ continue;
+ if (idx++ >= *pos) {
+ sctp_endpoint_hold(ep);
break;
+ }
}
read_unlock_bh(&head->lock);
+
+ if (ep) {
+ err = cb(ep, p);
+ sctp_endpoint_put(ep);
+ if (err)
+ return err;
+ (*pos)++;
+
+ idx = start;
+ goto again;
+ }
}
- return err;
+ return 0;
}
EXPORT_SYMBOL_GPL(sctp_for_each_endpoint);
diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c
index 619b3bab3824..32d6d03df321 100644
--- a/net/smc/smc_cdc.c
+++ b/net/smc/smc_cdc.c
@@ -470,9 +470,9 @@ static void smc_cdc_rx_handler(struct ib_wc *wc, void *buf)
{
struct smc_link *link = (struct smc_link *)wc->qp->qp_context;
struct smc_cdc_msg *cdc = buf;
+ struct smc_sock *smc = NULL;
struct smc_connection *conn;
struct smc_link_group *lgr;
- struct smc_sock *smc;
if (wc->byte_len < offsetof(struct smc_cdc_msg, reserved))
return; /* short message */
@@ -483,21 +483,26 @@ static void smc_cdc_rx_handler(struct ib_wc *wc, void *buf)
lgr = smc_get_lgr(link);
read_lock_bh(&lgr->conns_lock);
conn = smc_lgr_find_conn(ntohl(cdc->token), lgr);
- read_unlock_bh(&lgr->conns_lock);
- if (!conn || conn->out_of_sync)
+ if (!conn || conn->out_of_sync) {
+ read_unlock_bh(&lgr->conns_lock);
return;
+ }
smc = container_of(conn, struct smc_sock, conn);
+ sock_hold(&smc->sk);
+ read_unlock_bh(&lgr->conns_lock);
if (cdc->prod_flags.failover_validation) {
smc_cdc_msg_validate(smc, cdc, link);
- return;
+ goto out;
}
if (smc_cdc_before(ntohs(cdc->seqno),
conn->local_rx_ctrl.seqno))
/* received seqno is old */
- return;
+ goto out;
smc_cdc_msg_recv(smc, cdc);
+out:
+ sock_put(&smc->sk);
}
static struct smc_wr_rx_handler smc_cdc_rx_handlers[] = {
diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c
index 85f0b7853b17..0bf9e4075f03 100644
--- a/net/smc/smc_loopback.c
+++ b/net/smc/smc_loopback.c
@@ -245,6 +245,11 @@ static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok,
read_unlock_bh(&ldev->dmb_ht_lock);
return -EINVAL;
}
+ if ((u64)offset + size > rmb_node->len) {
+ read_unlock_bh(&ldev->dmb_ht_lock);
+ return -EINVAL;
+ }
+
memcpy((char *)rmb_node->cpu_addr + offset, data, size);
read_unlock_bh(&ldev->dmb_ht_lock);
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index e492655cb221..e88e21c2ab32 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -1026,8 +1026,23 @@ rpc_free_auth(struct rpc_clnt *clnt)
return NULL;
}
-/*
- * Release reference to the RPC client
+/**
+ * rpc_hold_client - acquire a reference on an rpc_clnt
+ * @clnt: rpc_clnt to pin
+ *
+ * Pairs with rpc_release_client().
+ */
+void rpc_hold_client(struct rpc_clnt *clnt)
+{
+ refcount_inc(&clnt->cl_count);
+}
+
+/**
+ * rpc_release_client - release a reference on an rpc_clnt
+ * @clnt: rpc_clnt to release
+ *
+ * Pairs with rpc_hold_client(). The rpc_clnt's resources are
+ * freed once its reference count drops to zero.
*/
void
rpc_release_client(struct rpc_clnt *clnt)
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 443d8390ebf1..f83f816bb930 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -462,6 +462,7 @@ static void svc_tcp_handshake_done(void *data, int status, key_serial_t peerid)
}
clear_bit(XPT_HANDSHAKE, &xprt->xpt_flags);
complete_all(&svsk->sk_handshake_done);
+ svc_xprt_put(xprt);
}
/**
@@ -485,9 +486,13 @@ static void svc_tcp_handshake(struct svc_xprt *xprt)
clear_bit(XPT_TLS_SESSION, &xprt->xpt_flags);
init_completion(&svsk->sk_handshake_done);
+ /* Pin the transport across the asynchronous handshake callback. */
+ svc_xprt_get(xprt);
+
ret = tls_server_hello_x509(&args, GFP_KERNEL);
if (ret) {
trace_svc_tls_not_started(xprt);
+ svc_xprt_put(xprt);
goto out_failed;
}
@@ -496,8 +501,13 @@ static void svc_tcp_handshake(struct svc_xprt *xprt)
if (ret <= 0) {
if (tls_handshake_cancel(sk)) {
trace_svc_tls_timed_out(xprt);
+ svc_xprt_put(xprt);
goto out_close;
}
+ /* Cancellation lost to handshake_complete(): the
+ * callback is in flight and should finish quickly.
+ */
+ wait_for_completion(&svsk->sk_handshake_done);
}
if (!test_bit(XPT_TLS_SESSION, &xprt->xpt_flags)) {
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index 62e07c330a66..80250af10777 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -180,6 +180,8 @@ unsigned int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size,
unsigned int count = 0;
if (head->iov_len) {
+ if (unlikely(count >= bvec_size))
+ goto bvec_overflow;
bvec_set_virt(bvec++, head->iov_base, head->iov_len);
++count;
}
@@ -193,25 +195,27 @@ unsigned int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size,
while (remaining > 0) {
len = min_t(unsigned int, remaining,
PAGE_SIZE - offset);
+ if (unlikely(count >= bvec_size))
+ goto bvec_overflow;
bvec_set_page(bvec++, *pages++, len, offset);
remaining -= len;
offset = 0;
- if (unlikely(++count > bvec_size))
- goto bvec_overflow;
+ ++count;
}
}
if (tail->iov_len) {
- bvec_set_virt(bvec, tail->iov_base, tail->iov_len);
- if (unlikely(++count > bvec_size))
+ if (unlikely(count >= bvec_size))
goto bvec_overflow;
+ bvec_set_virt(bvec, tail->iov_base, tail->iov_len);
+ ++count;
}
return count;
bvec_overflow:
pr_warn_once("%s: bio_vec array overflow\n", __func__);
- return count - 1;
+ return count;
}
/**
diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
index 09f245cda526..18ec0763ca21 100644
--- a/net/sunrpc/xprt.c
+++ b/net/sunrpc/xprt.c
@@ -1663,6 +1663,22 @@ void xprt_add_backlog(struct rpc_xprt *xprt, struct rpc_task *task)
}
EXPORT_SYMBOL_GPL(xprt_add_backlog);
+/**
+ * xprt_add_backlog_noncongested - queue task on backlog
+ * @xprt: transport whose backlog queue receives the task
+ * @task: task to queue
+ *
+ * Like xprt_add_backlog, but does not set XPRT_CONGESTED.
+ * For transports whose free_slot path does not synchronize
+ * with xprt_throttle_congested via reserve_lock.
+ */
+void xprt_add_backlog_noncongested(struct rpc_xprt *xprt,
+ struct rpc_task *task)
+{
+ rpc_sleep_on(&xprt->backlog, task, xprt_complete_request_init);
+}
+EXPORT_SYMBOL_GPL(xprt_add_backlog_noncongested);
+
static bool __xprt_set_rq(struct rpc_task *task, void *data)
{
struct rpc_rqst *req = data;
diff --git a/net/sunrpc/xprtrdma/backchannel.c b/net/sunrpc/xprtrdma/backchannel.c
index 8c817e755262..02ffe7e1b5ed 100644
--- a/net/sunrpc/xprtrdma/backchannel.c
+++ b/net/sunrpc/xprtrdma/backchannel.c
@@ -158,9 +158,7 @@ void xprt_rdma_bc_free_rqst(struct rpc_rqst *rqst)
rpcrdma_rep_put(&r_xprt->rx_buf, rep);
req->rl_reply = NULL;
- spin_lock(&xprt->bc_pa_lock);
- list_add_tail(&rqst->rq_bc_pa_list, &xprt->bc_pa_list);
- spin_unlock(&xprt->bc_pa_lock);
+ rpcrdma_req_put(req);
xprt_put(xprt);
}
@@ -202,6 +200,7 @@ static struct rpc_rqst *rpcrdma_bc_rqst_get(struct rpcrdma_xprt *r_xprt)
rqst->rq_xprt = xprt;
__set_bit(RPC_BC_PA_IN_USE, &rqst->rq_bc_pa_state);
xdr_buf_init(&rqst->rq_snd_buf, rdmab_data(req->rl_sendbuf), size);
+ kref_init(&req->rl_kref);
return rqst;
}
diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c
index 31434aeb8e29..a82eefe5e6b5 100644
--- a/net/sunrpc/xprtrdma/frwr_ops.c
+++ b/net/sunrpc/xprtrdma/frwr_ops.c
@@ -404,7 +404,7 @@ int frwr_send(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
++num_wrs;
}
- if ((kref_read(&req->rl_kref) > 1) || num_wrs > ep->re_send_count) {
+ if (req->rl_sendctx->sc_unmap_count || num_wrs > ep->re_send_count) {
send_wr->send_flags |= IB_SEND_SIGNALED;
ep->re_send_count = min_t(unsigned int, ep->re_send_batch,
num_wrs - ep->re_send_count);
diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c
index 1478c41c7e9d..e201b37578a7 100644
--- a/net/sunrpc/xprtrdma/rpc_rdma.c
+++ b/net/sunrpc/xprtrdma/rpc_rdma.c
@@ -516,29 +516,11 @@ static int rpcrdma_encode_reply_chunk(struct rpcrdma_xprt *r_xprt,
return 0;
}
-static void rpcrdma_sendctx_done(struct kref *kref)
-{
- struct rpcrdma_req *req =
- container_of(kref, struct rpcrdma_req, rl_kref);
- struct rpcrdma_rep *rep = req->rl_reply;
-
- rpcrdma_complete_rqst(rep);
- rep->rr_rxprt->rx_stats.reply_waits_for_send++;
-}
-
-/**
- * rpcrdma_sendctx_unmap - DMA-unmap Send buffer
- * @sc: sendctx containing SGEs to unmap
- *
- */
-void rpcrdma_sendctx_unmap(struct rpcrdma_sendctx *sc)
+static void rpcrdma_sendctx_dma_unmap(struct rpcrdma_sendctx *sc)
{
struct rpcrdma_regbuf *rb = sc->sc_req->rl_sendbuf;
struct ib_sge *sge;
- if (!sc->sc_unmap_count)
- return;
-
/* The first two SGEs contain the transport header and
* the inline buffer. These are always left mapped so
* they can be cheaply re-used.
@@ -547,8 +529,29 @@ void rpcrdma_sendctx_unmap(struct rpcrdma_sendctx *sc)
++sge, --sc->sc_unmap_count)
ib_dma_unmap_page(rdmab_device(rb), sge->addr, sge->length,
DMA_TO_DEVICE);
+}
- kref_put(&sc->sc_req->rl_kref, rpcrdma_sendctx_done);
+/**
+ * rpcrdma_sendctx_unmap - DMA-unmap Send buffer and release Send owner
+ * @sc: sendctx containing SGEs to unmap
+ *
+ */
+void rpcrdma_sendctx_unmap(struct rpcrdma_sendctx *sc)
+{
+ struct rpcrdma_req *req = sc->sc_req;
+
+ rpcrdma_sendctx_dma_unmap(sc);
+ sc->sc_req = NULL;
+ rpcrdma_req_put(req);
+}
+
+/* No Send was posted. Release DMA mappings prepared for this
+ * sendctx, but leave the request reference count alone.
+ */
+static void rpcrdma_sendctx_cancel(struct rpcrdma_sendctx *sc)
+{
+ rpcrdma_sendctx_dma_unmap(sc);
+ sc->sc_req = NULL;
}
/* Prepare an SGE for the RPC-over-RDMA transport header.
@@ -740,8 +743,6 @@ static bool rpcrdma_prepare_noch_mapped(struct rpcrdma_xprt *r_xprt,
tail->iov_len))
return false;
- if (req->rl_sendctx->sc_unmap_count)
- kref_get(&req->rl_kref);
return true;
}
@@ -771,7 +772,6 @@ static bool rpcrdma_prepare_readch(struct rpcrdma_xprt *r_xprt,
len -= len & 3;
if (!rpcrdma_prepare_tail_iov(req, xdr, page_base, len))
return false;
- kref_get(&req->rl_kref);
}
return true;
@@ -792,6 +792,7 @@ inline int rpcrdma_prepare_send_sges(struct rpcrdma_xprt *r_xprt,
struct xdr_buf *xdr,
enum rpcrdma_chunktype rtype)
{
+ struct rpcrdma_sendctx *sc;
int ret;
ret = -EAGAIN;
@@ -800,7 +801,6 @@ inline int rpcrdma_prepare_send_sges(struct rpcrdma_xprt *r_xprt,
goto out_nosc;
req->rl_sendctx->sc_unmap_count = 0;
req->rl_sendctx->sc_req = req;
- kref_init(&req->rl_kref);
req->rl_wr.wr_cqe = &req->rl_sendctx->sc_cqe;
req->rl_wr.sg_list = req->rl_sendctx->sc_sges;
req->rl_wr.num_sge = 0;
@@ -828,10 +828,16 @@ inline int rpcrdma_prepare_send_sges(struct rpcrdma_xprt *r_xprt,
goto out_unmap;
}
+ /* The Send-side owner releases this reference when the
+ * Send has completed.
+ */
+ kref_get(&req->rl_kref);
return 0;
out_unmap:
- rpcrdma_sendctx_unmap(req->rl_sendctx);
+ sc = req->rl_sendctx;
+ rpcrdma_sendctx_cancel(sc);
+ rpcrdma_sendctx_unget_locked(r_xprt, sc);
out_nosc:
trace_xprtrdma_prepsend_failed(&req->rl_slot, ret);
return ret;
@@ -1130,6 +1136,8 @@ rpcrdma_is_bcall(struct rpcrdma_xprt *r_xprt, struct rpcrdma_rep *rep)
/* Peek at stream contents without advancing. */
p = xdr_inline_decode(xdr, 0);
+ if ((char *)xdr->end - (char *)p < 5 * XDR_UNIT)
+ return false;
/* Chunk lists */
if (xdr_item_is_present(p++))
@@ -1154,7 +1162,7 @@ rpcrdma_is_bcall(struct rpcrdma_xprt *r_xprt, struct rpcrdma_rep *rep)
*/
p = xdr_inline_decode(xdr, 3 * sizeof(*p));
if (unlikely(!p))
- return true;
+ return false;
rpcrdma_bc_receive_call(r_xprt, rep);
return true;
@@ -1378,6 +1386,11 @@ void rpcrdma_complete_rqst(struct rpcrdma_rep *rep)
struct rpc_rqst *rqst = rep->rr_rqst;
int status;
+ /* I3: every registered MR has been invalidated and
+ * ib_dma_unmap_sg()'d before complete_rqst runs.
+ */
+ WARN_ON_ONCE(!list_empty(&rpcr_to_rdmar(rqst)->rl_registered));
+
switch (rep->rr_proc) {
case rdma_msg:
status = rpcrdma_decode_msg(r_xprt, rep, rqst);
@@ -1409,13 +1422,69 @@ void rpcrdma_complete_rqst(struct rpcrdma_rep *rep)
goto out;
}
-static void rpcrdma_reply_done(struct kref *kref)
-{
- struct rpcrdma_req *req =
- container_of(kref, struct rpcrdma_req, rl_kref);
-
- rpcrdma_complete_rqst(req->rl_reply);
-}
+/* Reply-side ownership invariants
+ *
+ * I1 (Receive WR ownership). A struct rpcrdma_rep is owned by the
+ * HCA between ib_post_recv() and the matching Receive completion.
+ * After ib_dma_sync_single_for_cpu() in rpcrdma_wc_receive() it is
+ * owned by the CPU until rpcrdma_rep_put() returns it to
+ * rb_free_reps; a rep on rb_free_reps is not re-posted until
+ * rpcrdma_post_recvs() pulls it off. Asserted: rpcrdma_post_recvs()
+ * WARNs that a pulled rep has rr_rqst == NULL.
+ *
+ * I2 (rep attachment). While req->rl_reply == rep, the rep cannot be
+ * re-posted. rpcrdma_reply_put() NULLs req->rl_reply before handing
+ * the rep to rpcrdma_rep_put(). Asserted: rpcrdma_reply_put() WARNs
+ * that rl_reply is NULL after the put.
+ *
+ * I3 (Registered-MR fence). On entry to rpcrdma_complete_rqst() every
+ * MR that was on req->rl_registered has had its rkey invalidated
+ * (remotely via IB_WC_WITH_INVALIDATE or locally via IB_WR_LOCAL_INV)
+ * and its pages ib_dma_unmap_sg()'d. The LocalInv chain is posted
+ * on a single QP; strong send-queue ordering makes the last
+ * completion (frwr_wc_localinv_done) observe the
+ * ib_dma_unmap_sg() that ran from each earlier completion's
+ * frwr_mr_put() before complete_rqst is called. The inline
+ * frwr_reminv() path unmaps its one MR synchronously before
+ * rpcrdma_reply_handler() reaches complete_rqst. Asserted:
+ * rpcrdma_complete_rqst() WARNs that rl_registered is empty.
+ *
+ * I4 (Send-buffer release). req->rl_kref carries two unconditional
+ * owners while a Send is outstanding: the RPC-layer reference (set
+ * at xprt_rdma_alloc_slot / xprt_rdma_bc_rqst_get / rpcrdma_req_release
+ * pool-entry) and the Send-side reference (kref_get() in
+ * rpcrdma_prepare_send_sges()). rpcrdma_req_release() runs only
+ * after both have dropped, so the req does not return to its free
+ * pool until rpcrdma_sendctx_unmap() has fired -- the HCA has
+ * released the send buffer before the req can be reused. Asserted:
+ * rpcrdma_req_release() WARNs that rl_sendctx is NULL.
+ *
+ * I5 (req lifecycle). A req is owned by the RPC layer between slot
+ * acquisition and the matching xprt_rdma_free_slot() (or, for the
+ * backchannel, xprt_rdma_bc_free_rqst()). While owned, rl_kref >= 1.
+ * The pools (rb_send_bufs, bc_pa_list, backlog wake target) never
+ * contain a req with outstanding Send-side or Reply-side work.
+ *
+ * Non-hazards. The following claims have been raised by adversarial
+ * review and are each closed by the invariants above:
+ *
+ * * "Reply completes the RPC while the HCA still holds the send
+ * buffer" -- excluded by I4. The Send-side kref reference is held
+ * until rpcrdma_sendctx_unmap() runs from Send completion.
+ *
+ * * "Signal-driven release races the in-flight Send" -- same
+ * resolution. xprt_rdma_free() does not touch rl_kref; the
+ * Send-side reference keeps the req out of its pool until Send
+ * completion fires.
+ *
+ * * "Receive completion races rep reuse" -- excluded by I1. A rep
+ * is on rb_free_reps only after rpcrdma_rep_put() has been called
+ * and rpcrdma_post_recvs() owns the next transition back to the HCA.
+ *
+ * * "Pages still DMA-mapped when call_decode reads them" -- excluded
+ * by I3. The matching ib_dma_unmap_sg() for every MR has run on
+ * the same CPU thread that calls rpcrdma_complete_rqst().
+ */
/**
* rpcrdma_reply_handler - Process received RPC/RDMA messages
@@ -1451,6 +1520,14 @@ void rpcrdma_reply_handler(struct rpcrdma_rep *rep)
credits = be32_to_cpu(*p++);
rep->rr_proc = *p++;
+ /* The credit grant from the wire is not trustworthy;
+ * sanitize it before any code path consumes it.
+ */
+ if (credits == 0)
+ credits = 1; /* don't deadlock */
+ else if (credits > r_xprt->rx_ep->re_max_requests)
+ credits = r_xprt->rx_ep->re_max_requests;
+
if (rep->rr_vers != rpcrdma_version)
goto out_badversion;
@@ -1467,11 +1544,6 @@ void rpcrdma_reply_handler(struct rpcrdma_rep *rep)
xprt_pin_rqst(rqst);
spin_unlock(&xprt->queue_lock);
- if (credits == 0)
- credits = 1; /* don't deadlock */
- else if (credits > r_xprt->rx_ep->re_max_requests)
- credits = r_xprt->rx_ep->re_max_requests;
- rpcrdma_post_recvs(r_xprt, credits + (buf->rb_bc_srv_max_requests << 1));
if (buf->rb_credits != credits)
rpcrdma_update_cwnd(r_xprt, credits);
@@ -1489,21 +1561,28 @@ void rpcrdma_reply_handler(struct rpcrdma_rep *rep)
frwr_unmap_async(r_xprt, req);
/* LocalInv completion will complete the RPC */
else
- kref_put(&req->rl_kref, rpcrdma_reply_done);
- return;
+ rpcrdma_complete_rqst(rep);
-out_badversion:
- trace_xprtrdma_reply_vers_err(rep);
- goto out;
+out_post:
+ rpcrdma_post_recvs(r_xprt,
+ credits + (buf->rb_bc_srv_max_requests << 1));
+ return;
out_norqst:
spin_unlock(&xprt->queue_lock);
trace_xprtrdma_reply_rqst_err(rep);
- goto out;
+ rpcrdma_rep_put(buf, rep);
+ goto out_post;
+
+out_badversion:
+ trace_xprtrdma_reply_vers_err(rep);
+ rpcrdma_rep_put(buf, rep);
+ credits = buf->rb_credits;
+ goto out_post;
out_shortreply:
trace_xprtrdma_reply_short_err(rep);
-
-out:
rpcrdma_rep_put(buf, rep);
+ credits = buf->rb_credits;
+ goto out_post;
}
diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c
index 9a8ce5df83ca..5ff8e5126a6c 100644
--- a/net/sunrpc/xprtrdma/transport.c
+++ b/net/sunrpc/xprtrdma/transport.c
@@ -279,6 +279,13 @@ xprt_rdma_destroy(struct rpc_xprt *xprt)
cancel_delayed_work_sync(&r_xprt->rx_connect_worker);
rpcrdma_xprt_disconnect(r_xprt);
+
+ /* The disconnect's sendctx drain can return bc_prealloc reqs
+ * to bc_pa_list after xprt_destroy_backchannel() emptied it.
+ */
+#if defined(CONFIG_SUNRPC_BACKCHANNEL)
+ xprt_rdma_bc_destroy(xprt, 0);
+#endif
rpcrdma_buffer_destroy(&r_xprt->rx_buf);
xprt_rdma_free_addresses(xprt);
@@ -487,6 +494,50 @@ xprt_rdma_connect(struct rpc_xprt *xprt, struct rpc_task *task)
queue_delayed_work(system_long_wq, &r_xprt->rx_connect_worker, delay);
}
+/* rl_kref has two owners while a Send is outstanding: the rpc_rqst
+ * owner and the sendctx. Replies complete the RPC but do not drop
+ * either reference. The req returns to its free pool only after
+ * xprt_rdma_free_slot() or xprt_rdma_bc_free_rqst() has dropped the
+ * RPC-layer reference and rpcrdma_sendctx_unmap() has dropped the
+ * Send-side reference.
+ */
+static void rpcrdma_req_release(struct kref *kref)
+{
+ struct rpcrdma_req *req =
+ container_of(kref, struct rpcrdma_req, rl_kref);
+ struct rpc_rqst *rqst = &req->rl_slot;
+ struct rpc_xprt *xprt = rqst->rq_xprt;
+ struct rpcrdma_xprt *r_xprt;
+
+ /* I4: both the RPC-layer and Send-side owners have dropped,
+ * so rpcrdma_sendctx_unmap() has cleared rl_sendctx.
+ */
+ WARN_ON_ONCE(req->rl_sendctx);
+
+ kref_init(&req->rl_kref);
+
+#if defined(CONFIG_SUNRPC_BACKCHANNEL)
+ if (bc_prealloc(rqst)) {
+ spin_lock(&xprt->bc_pa_lock);
+ list_add_tail(&rqst->rq_bc_pa_list, &xprt->bc_pa_list);
+ spin_unlock(&xprt->bc_pa_lock);
+ return;
+ }
+#endif
+
+ if (xprt_wake_up_backlog(xprt, rqst))
+ return;
+
+ r_xprt = rpcx_to_rdmax(xprt);
+ memset(rqst, 0, sizeof(*rqst));
+ rpcrdma_buffer_put(&r_xprt->rx_buf, req);
+}
+
+void rpcrdma_req_put(struct rpcrdma_req *req)
+{
+ kref_put(&req->rl_kref, rpcrdma_req_release);
+}
+
/**
* xprt_rdma_alloc_slot - allocate an rpc_rqst
* @xprt: controlling RPC transport
@@ -505,13 +556,28 @@ xprt_rdma_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task)
req = rpcrdma_buffer_get(&r_xprt->rx_buf);
if (!req)
goto out_sleep;
+ kref_init(&req->rl_kref);
task->tk_rqstp = &req->rl_slot;
task->tk_status = 0;
return;
out_sleep:
- task->tk_status = -ENOMEM;
- xprt_add_backlog(xprt, task);
+ task->tk_status = -EAGAIN;
+ xprt_add_backlog_noncongested(xprt, task);
+ /* A buffer freed between buffer_get and rpc_sleep_on
+ * goes back to the pool with no waiter to wake.
+ * Re-check after joining the backlog to close that gap.
+ */
+ req = rpcrdma_buffer_get(&r_xprt->rx_buf);
+ if (req) {
+ struct rpc_rqst *rqst = &req->rl_slot;
+
+ kref_init(&req->rl_kref);
+ if (!xprt_wake_up_backlog(xprt, rqst)) {
+ memset(rqst, 0, sizeof(*rqst));
+ rpcrdma_buffer_put(&r_xprt->rx_buf, req);
+ }
+ }
}
/**
@@ -527,10 +593,7 @@ xprt_rdma_free_slot(struct rpc_xprt *xprt, struct rpc_rqst *rqst)
container_of(xprt, struct rpcrdma_xprt, rx_xprt);
rpcrdma_reply_put(&r_xprt->rx_buf, rpcr_to_rdmar(rqst));
- if (!xprt_wake_up_backlog(xprt, rqst)) {
- memset(rqst, 0, sizeof(*rqst));
- rpcrdma_buffer_put(&r_xprt->rx_buf, rpcr_to_rdmar(rqst));
- }
+ rpcrdma_req_put(rpcr_to_rdmar(rqst));
}
static bool rpcrdma_check_regbuf(struct rpcrdma_xprt *r_xprt,
@@ -594,10 +657,10 @@ xprt_rdma_free(struct rpc_task *task)
frwr_unmap_sync(rpcx_to_rdmax(rqst->rq_xprt), req);
}
- /* XXX: If the RPC is completing because of a signal and
- * not because a reply was received, we ought to ensure
- * that the Send completion has fired, so that memory
- * involved with the Send is not still visible to the NIC.
+ /* The Send-side rl_kref owner keeps req out of its free pool
+ * until rpcrdma_sendctx_unmap() has fired -- see I4 above
+ * rpcrdma_reply_handler() -- so signal-driven release here
+ * does not let the HCA touch a recycled send buffer.
*/
}
@@ -703,7 +766,7 @@ void xprt_rdma_print_stats(struct rpc_xprt *xprt, struct seq_file *seq)
r_xprt->rx_stats.mrs_allocated,
r_xprt->rx_stats.local_inv_needed,
r_xprt->rx_stats.empty_sendctx_q,
- r_xprt->rx_stats.reply_waits_for_send);
+ 0LU); /* was reply_waits_for_send; column preserved */
}
static int
diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index 8abbd9c4045a..efd91ed249fa 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -65,6 +65,8 @@
static int rpcrdma_sendctxs_create(struct rpcrdma_xprt *r_xprt);
static void rpcrdma_sendctxs_destroy(struct rpcrdma_xprt *r_xprt);
+static unsigned long rpcrdma_sendctx_next(struct rpcrdma_buffer *buf,
+ unsigned long item);
static void rpcrdma_sendctx_put_locked(struct rpcrdma_xprt *r_xprt,
struct rpcrdma_sendctx *sc);
static int rpcrdma_reqs_setup(struct rpcrdma_xprt *r_xprt);
@@ -79,6 +81,8 @@ rpcrdma_regbuf_alloc_node(size_t size, enum dma_data_direction direction,
int node);
static struct rpcrdma_regbuf *
rpcrdma_regbuf_alloc(size_t size, enum dma_data_direction direction);
+static bool rpcrdma_regbuf_realloc_node(struct rpcrdma_regbuf *rb,
+ size_t size, gfp_t flags, int node);
static void rpcrdma_regbuf_dma_unmap(struct rpcrdma_regbuf *rb);
static void rpcrdma_regbuf_free(struct rpcrdma_regbuf *rb);
@@ -243,8 +247,17 @@ rpcrdma_cm_event_handler(struct rdma_cm_id *id, struct rdma_cm_event *event)
complete(&ep->re_done);
return 0;
case RDMA_CM_EVENT_ADDR_CHANGE:
- ep->re_connect_status = -ENODEV;
- goto disconnected;
+ switch (xchg(&ep->re_connect_status, -ENODEV)) {
+ case 0:
+ goto wake_connect_worker;
+ case 1:
+ /* The later DISCONNECTED event balances the
+ * ESTABLISHED get; do not put here.
+ */
+ rpcrdma_force_disconnect(ep);
+ return 0;
+ }
+ return 0;
case RDMA_CM_EVENT_ESTABLISHED:
rpcrdma_ep_get(ep);
ep->re_connect_status = 1;
@@ -267,7 +280,6 @@ rpcrdma_cm_event_handler(struct rdma_cm_id *id, struct rdma_cm_event *event)
return 0;
case RDMA_CM_EVENT_DISCONNECTED:
ep->re_connect_status = -ECONNABORTED;
-disconnected:
rpcrdma_force_disconnect(ep);
return rpcrdma_ep_put(ep);
default:
@@ -324,6 +336,7 @@ static struct rdma_cm_id *rpcrdma_create_id(struct rpcrdma_xprt *r_xprt,
if (rc)
goto out;
+ ep->re_id = id;
rc = rpcrdma_rn_register(id->device, &ep->re_rn, rpcrdma_ep_removal_done);
if (rc)
goto out;
@@ -396,7 +409,6 @@ static int rpcrdma_ep_create(struct rpcrdma_xprt *r_xprt)
}
__module_get(THIS_MODULE);
device = id->device;
- ep->re_id = id;
reinit_completion(&ep->re_done);
ep->re_max_requests = r_xprt->rx_xprt.max_reqs;
@@ -539,7 +551,17 @@ int rpcrdma_xprt_connect(struct rpcrdma_xprt *r_xprt)
goto out;
}
rpcrdma_mrs_create(r_xprt);
- frwr_wp_create(r_xprt);
+
+ /*
+ * rpcrdma_encode_write_list() dereferences the write-pad
+ * MR with no NULL check, so fail the connect rather than
+ * publish a transport whose write-pad MR is NULL.
+ */
+ rc = frwr_wp_create(r_xprt);
+ if (rc) {
+ rc = -ENOTCONN;
+ goto out;
+ }
out:
trace_xprtrdma_connect(r_xprt, rc);
@@ -571,9 +593,9 @@ void rpcrdma_xprt_disconnect(struct rpcrdma_xprt *r_xprt)
rpcrdma_xprt_drain(r_xprt);
rpcrdma_reps_unmap(r_xprt);
+ rpcrdma_sendctxs_destroy(r_xprt);
rpcrdma_reqs_reset(r_xprt);
rpcrdma_mrs_destroy(r_xprt);
- rpcrdma_sendctxs_destroy(r_xprt);
if (rpcrdma_ep_put(ep))
rdma_destroy_id(id);
@@ -605,6 +627,25 @@ static void rpcrdma_sendctxs_destroy(struct rpcrdma_xprt *r_xprt)
if (!buf->rb_sc_ctxs)
return;
+
+ /* The QP is drained, but the final unsignaled Sends might not
+ * have been walked by a signaled Send completion. Release those
+ * Send owners before request buffers are reset.
+ *
+ * Unlike the completion sweep, this walk can visit slots with
+ * no Send posted: after a partial rpcrdma_sendctxs_create()
+ * failure on reconnect, rb_sc_head and rb_sc_tail are stale,
+ * and slots between them can be NULL or have sc_req clear.
+ */
+ for (i = rpcrdma_sendctx_next(buf, buf->rb_sc_tail);
+ i != rpcrdma_sendctx_next(buf, buf->rb_sc_head);
+ i = rpcrdma_sendctx_next(buf, i)) {
+ struct rpcrdma_sendctx *sc = buf->rb_sc_ctxs[i];
+
+ if (sc && sc->sc_req)
+ rpcrdma_sendctx_unmap(sc);
+ }
+
for (i = 0; i <= buf->rb_sc_last; i++)
kfree(buf->rb_sc_ctxs[i]);
kfree(buf->rb_sc_ctxs);
@@ -667,6 +708,12 @@ static unsigned long rpcrdma_sendctx_next(struct rpcrdma_buffer *buf,
return likely(item < buf->rb_sc_last) ? item + 1 : 0;
}
+static unsigned long rpcrdma_sendctx_prev(struct rpcrdma_buffer *buf,
+ unsigned long item)
+{
+ return item > 0 ? item - 1 : buf->rb_sc_last;
+}
+
/**
* rpcrdma_sendctx_get_locked - Acquire a send context
* @r_xprt: controlling transport instance
@@ -711,6 +758,29 @@ struct rpcrdma_sendctx *rpcrdma_sendctx_get_locked(struct rpcrdma_xprt *r_xprt)
return NULL;
}
+/**
+ * rpcrdma_sendctx_unget_locked - Release an unposted send context
+ * @r_xprt: controlling transport instance
+ * @sc: send context to release
+ *
+ * Usage: Called when no Send is posted for the sendctx most
+ * recently returned by rpcrdma_sendctx_get_locked().
+ *
+ * The caller serializes calls to this function and to
+ * rpcrdma_sendctx_get_locked() (per transport).
+ */
+void rpcrdma_sendctx_unget_locked(struct rpcrdma_xprt *r_xprt,
+ struct rpcrdma_sendctx *sc)
+{
+ struct rpcrdma_buffer *buf = &r_xprt->rx_buf;
+
+ if (WARN_ON_ONCE(buf->rb_sc_ctxs[buf->rb_sc_head] != sc))
+ return;
+
+ buf->rb_sc_head = rpcrdma_sendctx_prev(buf, buf->rb_sc_head);
+ xprt_write_space(&r_xprt->rx_xprt);
+}
+
/**
* rpcrdma_sendctx_put_locked - Release a send context
* @r_xprt: controlling transport instance
@@ -727,15 +797,18 @@ static void rpcrdma_sendctx_put_locked(struct rpcrdma_xprt *r_xprt,
struct rpcrdma_buffer *buf = &r_xprt->rx_buf;
unsigned long next_tail;
- /* Unmap SGEs of previously completed but unsignaled
- * Sends by walking up the queue until @sc is found.
+ /* Release previously completed but unsignaled Sends by walking
+ * up the queue until @sc is found.
*/
next_tail = buf->rb_sc_tail;
do {
+ struct rpcrdma_sendctx *cur;
+
next_tail = rpcrdma_sendctx_next(buf, next_tail);
/* ORDER: item must be accessed _before_ tail is updated */
- rpcrdma_sendctx_unmap(buf->rb_sc_ctxs[next_tail]);
+ cur = buf->rb_sc_ctxs[next_tail];
+ rpcrdma_sendctx_unmap(cur);
} while (buf->rb_sc_ctxs[next_tail] != sc);
@@ -1196,6 +1269,10 @@ void rpcrdma_reply_put(struct rpcrdma_buffer *buffers, struct rpcrdma_req *req)
rpcrdma_rep_put(buffers, req->rl_reply);
req->rl_reply = NULL;
}
+ /* I2: rl_reply NULL after the put closes the
+ * 'rep on rb_free_reps still referenced by req' window.
+ */
+ WARN_ON_ONCE(req->rl_reply);
}
/**
@@ -1276,10 +1353,16 @@ rpcrdma_regbuf_alloc(size_t size, enum dma_data_direction direction)
* returned, @rb is left untouched.
*/
bool rpcrdma_regbuf_realloc(struct rpcrdma_regbuf *rb, size_t size, gfp_t flags)
+{
+ return rpcrdma_regbuf_realloc_node(rb, size, flags, NUMA_NO_NODE);
+}
+
+static bool rpcrdma_regbuf_realloc_node(struct rpcrdma_regbuf *rb,
+ size_t size, gfp_t flags, int node)
{
void *buf;
- buf = kmalloc(size, flags);
+ buf = kmalloc_node(size, flags, node);
if (!buf)
return false;
@@ -1291,6 +1374,23 @@ bool rpcrdma_regbuf_realloc(struct rpcrdma_regbuf *rb, size_t size, gfp_t flags)
return true;
}
+static bool rpcrdma_rep_resize(struct rpcrdma_xprt *r_xprt,
+ struct rpcrdma_rep *rep)
+{
+ struct rpcrdma_regbuf *rb = rep->rr_rdmabuf;
+ struct rpcrdma_ep *ep = r_xprt->rx_ep;
+ size_t size = ep->re_inline_recv;
+
+ if (likely(rdmab_length(rb) >= size))
+ return true;
+ if (!rpcrdma_regbuf_realloc_node(rb, size, XPRTRDMA_GFP_FLAGS,
+ ibdev_to_node(ep->re_id->device)))
+ return false;
+
+ xdr_buf_init(&rep->rr_hdrbuf, rdmab_data(rb), rdmab_length(rb));
+ return true;
+}
+
/**
* __rpcrdma_regbuf_dma_map - DMA-map a regbuf
* @r_xprt: controlling transport instance
@@ -1372,6 +1472,12 @@ void rpcrdma_post_recvs(struct rpcrdma_xprt *r_xprt, int needed)
rep = rpcrdma_rep_create(r_xprt);
if (!rep)
break;
+ /* I1: a rep on rb_free_reps must carry no rqst pointer. */
+ WARN_ON_ONCE(rep->rr_rqst);
+ if (!rpcrdma_rep_resize(r_xprt, rep)) {
+ rpcrdma_rep_put(buf, rep);
+ break;
+ }
if (!rpcrdma_regbuf_dma_map(r_xprt, rep->rr_rdmabuf)) {
rpcrdma_rep_put(buf, rep);
break;
diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index 8147d2b41494..ae14f6a526e6 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -410,7 +410,6 @@ struct rpcrdma_stats {
/* accessed when receiving a reply */
unsigned long long total_rdma_reply;
unsigned long long fixup_copy_count;
- unsigned long reply_waits_for_send;
unsigned long local_inv_needed;
unsigned long nomsg_call_count;
unsigned long bcall_count;
@@ -479,6 +478,8 @@ void rpcrdma_req_destroy(struct rpcrdma_req *req);
int rpcrdma_buffer_create(struct rpcrdma_xprt *);
void rpcrdma_buffer_destroy(struct rpcrdma_buffer *);
struct rpcrdma_sendctx *rpcrdma_sendctx_get_locked(struct rpcrdma_xprt *r_xprt);
+void rpcrdma_sendctx_unget_locked(struct rpcrdma_xprt *r_xprt,
+ struct rpcrdma_sendctx *sc);
struct rpcrdma_mr *rpcrdma_mr_get(struct rpcrdma_xprt *r_xprt);
void rpcrdma_mrs_refresh(struct rpcrdma_xprt *r_xprt);
@@ -488,6 +489,7 @@ void rpcrdma_buffer_put(struct rpcrdma_buffer *buffers,
struct rpcrdma_req *req);
void rpcrdma_rep_put(struct rpcrdma_buffer *buf, struct rpcrdma_rep *rep);
void rpcrdma_reply_put(struct rpcrdma_buffer *buffers, struct rpcrdma_req *req);
+void rpcrdma_req_put(struct rpcrdma_req *req);
bool rpcrdma_regbuf_realloc(struct rpcrdma_regbuf *rb, size_t size,
gfp_t flags);
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index b78f1aae9e80..56819b6a7bde 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -2732,8 +2732,11 @@ static void xs_tcp_tls_setup_socket(struct work_struct *work)
lower_xprt = rcu_dereference(lower_clnt->cl_xprt);
rcu_read_unlock();
- if (wait_on_bit_lock(&lower_xprt->state, XPRT_LOCKED, TASK_KILLABLE))
+ if (wait_on_bit_lock(&lower_xprt->state, XPRT_LOCKED, TASK_KILLABLE)) {
+ /* XPRT_LOCKED was never acquired. */
+ rpc_shutdown_client(lower_clnt);
goto out_unlock;
+ }
status = xs_tls_handshake_sync(lower_xprt, &upper_xprt->xprtsec);
if (status) {
@@ -2767,6 +2770,7 @@ static void xs_tcp_tls_setup_socket(struct work_struct *work)
out_unlock:
current_restore_flags(pflags, PF_MEMALLOC);
upper_transport->clnt = NULL;
+ rpc_release_client(upper_clnt);
xprt_unlock_connect(upper_xprt, upper_transport);
return;
@@ -2814,7 +2818,15 @@ static void xs_connect(struct rpc_xprt *xprt, struct rpc_task *task)
} else
dprintk("RPC: xs_connect scheduled xprt %p\n", xprt);
- transport->clnt = task->tk_client;
+ /*
+ * Only the TLS connect_worker reads transport->clnt; pinning
+ * the upper rpc_clnt unconditionally would form a cycle with
+ * cl_xprt and prevent xprt destruction.
+ */
+ if (xprt->xprtsec.policy != RPC_XPRTSEC_NONE) {
+ rpc_hold_client(task->tk_client);
+ transport->clnt = task->tk_client;
+ }
queue_delayed_work(xprtiod_workqueue,
&transport->connect_worker,
delay);
diff --git a/net/tipc/bearer.c b/net/tipc/bearer.c
index ae1ddbf71853..8866a3d71313 100644
--- a/net/tipc/bearer.c
+++ b/net/tipc/bearer.c
@@ -482,6 +482,7 @@ void tipc_disable_l2_media(struct tipc_bearer *b)
dev = (struct net_device *)rtnl_dereference(b->media_ptr);
dev_remove_pack(&b->pt);
RCU_INIT_POINTER(dev->tipc_ptr, NULL);
+ RCU_INIT_POINTER(b->media_ptr, NULL);
synchronize_net();
dev_put(dev);
}
diff --git a/net/tipc/core.c b/net/tipc/core.c
index 434e70eabe08..1ddecea1df6e 100644
--- a/net/tipc/core.c
+++ b/net/tipc/core.c
@@ -218,6 +218,11 @@ static void __exit tipc_exit(void)
unregister_pernet_device(&tipc_net_ops);
tipc_unregister_sysctl();
+ /* TODO: Wait for all timers that called call_rcu() to finish before
+ * calling rcu_barrier().
+ */
+ rcu_barrier();
+
pr_info("Deactivated\n");
}
diff --git a/net/tipc/discover.c b/net/tipc/discover.c
index 685389d4b245..da60f572dda0 100644
--- a/net/tipc/discover.c
+++ b/net/tipc/discover.c
@@ -58,6 +58,7 @@
* @skb: request message to be (repeatedly) sent
* @timer: timer governing period between requests
* @timer_intv: current interval between requests (in ms)
+ * @rcu: RCU head for deferred freeing
*/
struct tipc_discoverer {
u32 bearer_id;
@@ -69,6 +70,7 @@ struct tipc_discoverer {
struct sk_buff *skb;
struct timer_list timer;
unsigned long timer_intv;
+ struct rcu_head rcu;
};
/**
@@ -382,6 +384,15 @@ int tipc_disc_create(struct net *net, struct tipc_bearer *b,
return 0;
}
+static void tipc_disc_free_rcu(struct rcu_head *rp)
+{
+ struct tipc_discoverer *d = container_of(rp, struct tipc_discoverer,
+ rcu);
+
+ kfree_skb(d->skb);
+ kfree(d);
+}
+
/**
* tipc_disc_delete - destroy object sending periodic link setup requests
* @d: ptr to link dest structure
@@ -389,8 +400,7 @@ int tipc_disc_create(struct net *net, struct tipc_bearer *b,
void tipc_disc_delete(struct tipc_discoverer *d)
{
timer_shutdown_sync(&d->timer);
- kfree_skb(d->skb);
- kfree(d);
+ call_rcu(&d->rcu, tipc_disc_free_rcu);
}
/**
diff --git a/net/tipc/name_distr.c b/net/tipc/name_distr.c
index 190b49c5cbc3..ba4f4906e13b 100644
--- a/net/tipc/name_distr.c
+++ b/net/tipc/name_distr.c
@@ -280,12 +280,21 @@ static bool tipc_update_nametbl(struct net *net, struct distr_item *i,
u32 node, u32 dtype)
{
struct publication *p = NULL;
+ u32 lower = ntohl(i->lower);
+ u32 upper = ntohl(i->upper);
struct tipc_socket_addr sk;
- struct tipc_uaddr ua;
u32 key = ntohl(i->key);
+ struct tipc_uaddr ua;
+
+ /* A peer-advertised binding with lower > upper can never be matched
+ * or withdrawn and would leak the publication; the local bind path
+ * rejects such ranges, so reject ranges learned from the network too.
+ */
+ if (lower > upper)
+ return false;
tipc_uaddr(&ua, TIPC_SERVICE_RANGE, TIPC_CLUSTER_SCOPE,
- ntohl(i->type), ntohl(i->lower), ntohl(i->upper));
+ ntohl(i->type), lower, upper);
sk.ref = ntohl(i->port);
sk.node = node;
diff --git a/net/tipc/netlink.c b/net/tipc/netlink.c
index 1a9a5bdaccf4..8336a9664703 100644
--- a/net/tipc/netlink.c
+++ b/net/tipc/netlink.c
@@ -152,11 +152,13 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_BEARER_DISABLE,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_bearer_disable,
},
{
.cmd = TIPC_NL_BEARER_ENABLE,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_bearer_enable,
},
{
@@ -168,11 +170,13 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_BEARER_ADD,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_bearer_add,
},
{
.cmd = TIPC_NL_BEARER_SET,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_bearer_set,
},
{
@@ -197,11 +201,13 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_LINK_SET,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_node_set_link,
},
{
.cmd = TIPC_NL_LINK_RESET_STATS,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_node_reset_link_stats,
},
{
@@ -213,6 +219,7 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_MEDIA_SET,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_media_set,
},
{
@@ -228,6 +235,7 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_NET_SET,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_net_set,
},
{
@@ -238,6 +246,7 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_MON_SET,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_node_set_monitor,
},
{
@@ -255,6 +264,7 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_PEER_REMOVE,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_peer_rm,
},
#ifdef CONFIG_TIPC_MEDIA_UDP
@@ -269,11 +279,13 @@ static const struct genl_ops tipc_genl_v2_ops[] = {
{
.cmd = TIPC_NL_KEY_SET,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_node_set_key,
},
{
.cmd = TIPC_NL_KEY_FLUSH,
.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+ .flags = GENL_UNS_ADMIN_PERM,
.doit = tipc_nl_node_flush_key,
},
#endif
diff --git a/net/tipc/socket.c b/net/tipc/socket.c
index f6ae4332c070..ed8ddde5088e 100644
--- a/net/tipc/socket.c
+++ b/net/tipc/socket.c
@@ -1362,6 +1362,9 @@ static void tipc_sk_conn_proto_rcv(struct tipc_sock *tsk, struct sk_buff *skb,
__skb_queue_tail(xmitq, skb);
return;
} else if (mtyp == CONN_ACK) {
+ if (tsk->snt_unacked < msg_conn_ack(hdr))
+ goto exit;
+
was_cong = tsk_conn_cong(tsk);
tipc_sk_push_backlog(tsk, msg_nagle_ack(hdr));
tsk->snt_unacked -= msg_conn_ack(hdr);
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index d3eab47b47ea..2b875369d523 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -2391,6 +2391,17 @@ int tls_sw_read_sock(struct sock *sk, read_descriptor_t *desc,
goto read_sock_requeue;
}
+ /* An empty data record (legal in TLS 1.3) gives a zero
+ * read_actor return, indistinguishable from the consumer
+ * stalling; the used <= 0 path would requeue it at the
+ * head of rx_list and block all later records. Consume it
+ * here instead.
+ */
+ if (rxm->full_len == 0) {
+ consume_skb(skb);
+ continue;
+ }
+
used = read_actor(desc, skb, rxm->offset, rxm->full_len);
if (used <= 0) {
if (!copied)
diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
index 4682cc59b7a7..23d94a08e481 100644
--- a/net/unix/af_unix.c
+++ b/net/unix/af_unix.c
@@ -731,8 +731,8 @@ static void unix_release_sock(struct sock *sk, int embrion)
if (state == TCP_LISTEN)
unix_release_sock(skb->sk, 1);
- /* passed fds are erased in the kfree_skb hook */
- kfree_skb(skb);
+ /* passed fds are erased in the kfree_skb hook */
+ kfree_skb_reason(skb, SKB_DROP_REASON_SOCKET_CLOSE);
}
if (path.dentry)
@@ -1884,16 +1884,19 @@ static void unix_peek_fds(struct scm_cookie *scm, struct sk_buff *skb)
static void unix_destruct_scm(struct sk_buff *skb)
{
- struct scm_cookie scm;
+ struct scm_cookie scm = {};
+
+ swap(scm.pid, UNIXCB(skb).pid);
- memset(&scm, 0, sizeof(scm));
- scm.pid = UNIXCB(skb).pid;
if (UNIXCB(skb).fp)
unix_detach_fds(&scm, skb);
- /* Alas, it calls VFS */
- /* So fscking what? fput() had been SMP-safe since the last Summer */
scm_destroy(&scm);
+}
+
+static void unix_wfree(struct sk_buff *skb)
+{
+ unix_destruct_scm(skb);
sock_wfree(skb);
}
@@ -1909,7 +1912,7 @@ static int unix_scm_to_skb(struct scm_cookie *scm, struct sk_buff *skb, bool sen
if (scm->fp && send_fds)
err = unix_attach_fds(scm, skb);
- skb->destructor = unix_destruct_scm;
+ skb->destructor = unix_wfree;
return err;
}
@@ -1970,6 +1973,13 @@ static void scm_stat_del(struct sock *sk, struct sk_buff *skb)
}
}
+static void unix_orphan_scm(struct sock *sk, struct sk_buff *skb)
+{
+ scm_stat_del(sk, skb);
+ unix_destruct_scm(skb);
+ skb->destructor = sock_wfree;
+}
+
/*
* Send AF_UNIX data.
*/
@@ -2463,12 +2473,10 @@ int __unix_dgram_recvmsg(struct sock *sk, struct msghdr *msg, size_t size,
&err, &timeo, last));
if (!skb) { /* implies iolock unlocked */
- unix_state_lock(sk);
/* Signal EOF on disconnected non-blocking SEQPACKET socket. */
if (sk->sk_type == SOCK_SEQPACKET && err == -EAGAIN &&
- (sk->sk_shutdown & RCV_SHUTDOWN))
+ (READ_ONCE(sk->sk_shutdown) & RCV_SHUTDOWN))
err = 0;
- unix_state_unlock(sk);
goto out;
}
@@ -2558,10 +2566,16 @@ static int unix_read_skb(struct sock *sk, skb_read_actor_t recv_actor)
int err;
mutex_lock(&u->iolock);
+
skb = skb_recv_datagram(sk, MSG_DONTWAIT, &err);
- mutex_unlock(&u->iolock);
- if (!skb)
+ if (!skb) {
+ mutex_unlock(&u->iolock);
return err;
+ }
+
+ unix_orphan_scm(sk, skb);
+
+ mutex_unlock(&u->iolock);
return recv_actor(sk, skb);
}
@@ -2714,7 +2728,7 @@ static struct sk_buff *manage_oob(struct sk_buff *skb, struct sock *sk,
spin_unlock(&sk->sk_receive_queue.lock);
consume_skb(read_skb);
- kfree_skb(unread_skb);
+ kfree_skb_reason(unread_skb, SKB_DROP_REASON_UNIX_SKIP_OOB);
return skb;
}
@@ -2722,6 +2736,7 @@ static struct sk_buff *manage_oob(struct sk_buff *skb, struct sock *sk,
static int unix_stream_read_skb(struct sock *sk, skb_read_actor_t recv_actor)
{
+ struct sk_buff_head *queue = &sk->sk_receive_queue;
struct unix_sock *u = unix_sk(sk);
struct sk_buff *skb;
int err;
@@ -2729,40 +2744,37 @@ static int unix_stream_read_skb(struct sock *sk, skb_read_actor_t recv_actor)
if (unlikely(READ_ONCE(sk->sk_state) != TCP_ESTABLISHED))
return -ENOTCONN;
- mutex_lock(&u->iolock);
- skb = skb_recv_datagram(sk, MSG_DONTWAIT, &err);
- mutex_unlock(&u->iolock);
- if (!skb)
+ err = sock_error(sk);
+ if (err)
return err;
-#if IS_ENABLED(CONFIG_AF_UNIX_OOB)
- if (unlikely(skb == READ_ONCE(u->oob_skb))) {
- bool drop = false;
-
- unix_state_lock(sk);
-
- if (sock_flag(sk, SOCK_DEAD)) {
- unix_state_unlock(sk);
- kfree_skb(skb);
- return -ECONNRESET;
- }
+ mutex_lock(&u->iolock);
+ spin_lock(&queue->lock);
- spin_lock(&sk->sk_receive_queue.lock);
- if (likely(skb == u->oob_skb)) {
- WRITE_ONCE(u->oob_skb, NULL);
- drop = true;
- }
- spin_unlock(&sk->sk_receive_queue.lock);
+ skb = __skb_dequeue(queue);
+ if (!skb) {
+ spin_unlock(&queue->lock);
+ mutex_unlock(&u->iolock);
+ return -EAGAIN;
+ }
- unix_state_unlock(sk);
+#if IS_ENABLED(CONFIG_AF_UNIX_OOB)
+ if (skb == u->oob_skb) {
+ WRITE_ONCE(u->oob_skb, NULL);
+ spin_unlock(&queue->lock);
+ mutex_unlock(&u->iolock);
- if (drop) {
- kfree_skb(skb);
- return -EAGAIN;
- }
+ kfree_skb_reason(skb, SKB_DROP_REASON_UNIX_SKIP_OOB);
+ return -EAGAIN;
}
#endif
+ spin_unlock(&queue->lock);
+
+ unix_orphan_scm(sk, skb);
+
+ mutex_unlock(&u->iolock);
+
return recv_actor(sk, skb);
}
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index 95170c7be758..49bd5d12c302 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -72,35 +72,6 @@ static bool virtio_transport_can_zcopy(const struct virtio_transport *t_ops,
return true;
}
-static int virtio_transport_init_zcopy_skb(struct vsock_sock *vsk,
- struct sk_buff *skb,
- struct msghdr *msg,
- bool zerocopy)
-{
- struct ubuf_info *uarg;
-
- if (msg->msg_ubuf) {
- uarg = msg->msg_ubuf;
- net_zcopy_get(uarg);
- } else {
- struct iov_iter *iter = &msg->msg_iter;
- struct ubuf_info_msgzc *uarg_zc;
-
- uarg = msg_zerocopy_realloc(sk_vsock(vsk),
- iter->count,
- NULL);
- if (!uarg)
- return -1;
-
- uarg_zc = uarg_to_msgzc(uarg);
- uarg_zc->zerocopy = zerocopy ? 1 : 0;
- }
-
- skb_zcopy_init(skb, uarg);
-
- return 0;
-}
-
static int virtio_transport_fill_skb(struct sk_buff *skb,
struct virtio_vsock_pkt_info *info,
size_t len,
@@ -237,6 +208,7 @@ static u16 virtio_transport_get_type(struct sock *sk)
static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info,
size_t payload_len,
bool zcopy,
+ struct ubuf_info *uarg,
u32 src_cid,
u32 src_port,
u32 dst_cid,
@@ -277,6 +249,12 @@ static struct sk_buff *virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *
if (info->msg && payload_len > 0) {
int err;
+ /* Bind the zerocopy lifetime before filling frags so error
+ * rollback frees managed fixed-buffer pages through
+ * the uarg-aware path.
+ */
+ skb_zcopy_set(skb, uarg, NULL);
+
err = virtio_transport_fill_skb(skb, info, payload_len, zcopy);
if (err)
goto out;
@@ -321,8 +299,10 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,
u32 src_cid, src_port, dst_cid, dst_port;
const struct virtio_transport *t_ops;
struct virtio_vsock_sock *vvs;
+ struct ubuf_info *uarg = NULL;
u32 pkt_len = info->pkt_len;
bool can_zcopy = false;
+ bool have_uref = false;
u32 rest_len;
int ret;
@@ -364,6 +344,25 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,
if (can_zcopy)
max_skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE,
(MAX_SKB_FRAGS * PAGE_SIZE));
+
+ if (info->msg->msg_flags & MSG_ZEROCOPY &&
+ info->op == VIRTIO_VSOCK_OP_RW) {
+ uarg = info->msg->msg_ubuf;
+
+ if (!uarg) {
+ uarg = msg_zerocopy_realloc(sk_vsock(vsk),
+ pkt_len, NULL);
+ if (!uarg) {
+ virtio_transport_put_credit(vvs, pkt_len);
+ return -ENOMEM;
+ }
+
+ if (!can_zcopy)
+ uarg_to_msgzc(uarg)->zerocopy = 0;
+
+ have_uref = true;
+ }
+ }
}
rest_len = pkt_len;
@@ -375,6 +374,7 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,
skb_len = min(max_skb_len, rest_len);
skb = virtio_transport_alloc_skb(info, skb_len, can_zcopy,
+ uarg,
src_cid, src_port,
dst_cid, dst_port);
if (!skb) {
@@ -382,22 +382,6 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,
break;
}
- /* We process buffer part by part, allocating skb on
- * each iteration. If this is last skb for this buffer
- * and MSG_ZEROCOPY mode is in use - we must allocate
- * completion for the current syscall.
- */
- if (info->msg && info->msg->msg_flags & MSG_ZEROCOPY &&
- skb_len == rest_len && info->op == VIRTIO_VSOCK_OP_RW) {
- if (virtio_transport_init_zcopy_skb(vsk, skb,
- info->msg,
- can_zcopy)) {
- kfree_skb(skb);
- ret = -ENOMEM;
- break;
- }
- }
-
virtio_transport_inc_tx_pkt(vvs, skb);
ret = t_ops->send_pkt(skb);
@@ -420,6 +404,18 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,
virtio_transport_put_credit(vvs, rest_len);
+ /* msg_zerocopy_realloc() initializes the ubuf_info refcnt to 1.
+ * skb_zcopy_set() increases it for each skb, so we can drop that
+ * initial reference to keep it balanced.
+ */
+ if (have_uref) {
+ if (rest_len == pkt_len)
+ /* No data sent, abort the notification. */
+ net_zcopy_put_abort(uarg, true);
+ else
+ net_zcopy_put(uarg);
+ }
+
/* Return number of bytes, if any data has been sent. */
if (rest_len != pkt_len)
ret = pkt_len - rest_len;
@@ -1181,7 +1177,7 @@ static int virtio_transport_reset_no_sock(const struct virtio_transport *t,
if (!t)
return -ENOTCONN;
- reply = virtio_transport_alloc_skb(&info, 0, false,
+ reply = virtio_transport_alloc_skb(&info, 0, false, NULL,
le64_to_cpu(hdr->dst_cid),
le32_to_cpu(hdr->dst_port),
le64_to_cpu(hdr->src_cid),
diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c
index dc862ca02e43..15a8237f79bf 100644
--- a/net/wireless/nl80211.c
+++ b/net/wireless/nl80211.c
@@ -4571,7 +4571,7 @@ static int nl80211_validate_key_link_id(struct genl_info *info,
if (wdev->valid_links) {
if (link_id == -1) {
GENL_SET_ERR_MSG(info,
- "link ID must for MLO group key");
+ "link ID must be set for MLO group key");
return -EINVAL;
}
if (!(wdev->valid_links & BIT(link_id))) {
diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c
index dbf8e71ca201..999d71003f34 100644
--- a/net/xfrm/espintcp.c
+++ b/net/xfrm/espintcp.c
@@ -215,43 +215,23 @@ static int espintcp_sendskmsg_locked(struct sock *sk,
struct sk_msg *skmsg = &emsg->skmsg;
bool more = flags & MSG_MORE;
struct scatterlist *sg;
- int done = 0;
int ret;
- sg = &skmsg->sg.data[skmsg->sg.start];
do {
struct bio_vec bvec;
- size_t size = sg->length - emsg->offset;
- int offset = sg->offset + emsg->offset;
- struct page *p;
-
- emsg->offset = 0;
+ sg = &skmsg->sg.data[skmsg->sg.start];
if (sg_is_last(sg) && !more)
msghdr.msg_flags &= ~MSG_MORE;
- p = sg_page(sg);
-retry:
- bvec_set_page(&bvec, p, size, offset);
- iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size);
- ret = tcp_sendmsg_locked(sk, &msghdr, size);
- if (ret < 0) {
- emsg->offset = offset - sg->offset;
- skmsg->sg.start += done;
+ bvec_set_page(&bvec, sg_page(sg), sg->length, sg->offset);
+ iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, sg->length);
+ ret = tcp_sendmsg_locked(sk, &msghdr, sg->length);
+ if (ret < 0)
return ret;
- }
-
- if (ret != size) {
- offset += ret;
- size -= ret;
- goto retry;
- }
- done++;
- put_page(p);
- sk_mem_uncharge(sk, sg->length);
- sg = sg_next(sg);
- } while (sg);
+ sk_msg_free_partial(sk, skmsg, ret);
+ } while (skmsg->sg.size);
memset(emsg, 0, sizeof(*emsg));
diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c
index c92db6b497d4..be0000448af2 100644
--- a/net/xfrm/xfrm_interface_core.c
+++ b/net/xfrm/xfrm_interface_core.c
@@ -868,6 +868,9 @@ static int xfrmi_changelink(struct net_device *dev, struct nlattr *tb[],
struct net *net = xi->net;
struct xfrm_if_parms p = {};
+ if (!rtnl_dev_link_net_capable(dev, net))
+ return -EPERM;
+
xfrmi_netlink_parms(data, &p);
if (!p.if_id) {
NL_SET_ERR_MSG(extack, "if_id must be non zero");
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 5a7ec72e17b0..f74e45a34b70 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -242,6 +242,9 @@ __xfrm6_selector_match(const struct xfrm_selector *sel, const struct flowi *fl)
bool xfrm_selector_match(const struct xfrm_selector *sel, const struct flowi *fl,
unsigned short family)
{
+ if (family != sel->family && sel->family != AF_UNSPEC)
+ return false;
+
switch (family) {
case AF_INET:
return __xfrm4_selector_match(sel, fl);
@@ -685,7 +688,7 @@ static void xfrm_byidx_resize(struct net *net)
static inline int xfrm_bydst_should_resize(struct net *net, int dir, int *total)
{
- unsigned int cnt = net->xfrm.policy_count[dir];
+ unsigned int cnt = READ_ONCE(net->xfrm.policy_count[dir]);
unsigned int hmask = net->xfrm.policy_bydst[dir].hmask;
if (total)
@@ -711,12 +714,12 @@ static inline int xfrm_byidx_should_resize(struct net *net, int total)
void xfrm_spd_getinfo(struct net *net, struct xfrmk_spdinfo *si)
{
- si->incnt = net->xfrm.policy_count[XFRM_POLICY_IN];
- si->outcnt = net->xfrm.policy_count[XFRM_POLICY_OUT];
- si->fwdcnt = net->xfrm.policy_count[XFRM_POLICY_FWD];
- si->inscnt = net->xfrm.policy_count[XFRM_POLICY_IN+XFRM_POLICY_MAX];
- si->outscnt = net->xfrm.policy_count[XFRM_POLICY_OUT+XFRM_POLICY_MAX];
- si->fwdscnt = net->xfrm.policy_count[XFRM_POLICY_FWD+XFRM_POLICY_MAX];
+ si->incnt = READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_IN]);
+ si->outcnt = READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_OUT]);
+ si->fwdcnt = READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_FWD]);
+ si->inscnt = READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_IN+XFRM_POLICY_MAX]);
+ si->outscnt = READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_OUT+XFRM_POLICY_MAX]);
+ si->fwdscnt = READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_FWD+XFRM_POLICY_MAX]);
si->spdhcnt = net->xfrm.policy_idx_hmask;
si->spdhmcnt = xfrm_policy_hashmax;
}
@@ -2318,7 +2321,7 @@ static void __xfrm_policy_link(struct xfrm_policy *pol, int dir)
}
list_add(&pol->walk.all, &net->xfrm.policy_all);
- net->xfrm.policy_count[dir]++;
+ WRITE_ONCE(net->xfrm.policy_count[dir], net->xfrm.policy_count[dir] + 1);
xfrm_pol_hold(pol);
}
@@ -2337,7 +2340,7 @@ static struct xfrm_policy *__xfrm_policy_unlink(struct xfrm_policy *pol,
}
list_del_init(&pol->walk.all);
- net->xfrm.policy_count[dir]--;
+ WRITE_ONCE(net->xfrm.policy_count[dir], net->xfrm.policy_count[dir] - 1);
return pol;
}
@@ -3217,7 +3220,7 @@ struct dst_entry *xfrm_lookup_with_ifid(struct net *net,
/* To accelerate a bit... */
if (!if_id && ((dst_orig->flags & DST_NOXFRM) ||
- !net->xfrm.policy_count[XFRM_POLICY_OUT]))
+ !READ_ONCE(net->xfrm.policy_count[XFRM_POLICY_OUT])))
goto nopol;
xdst = xfrm_bundle_lookup(net, fl, family, dir, &xflo, if_id);
@@ -3290,7 +3293,7 @@ struct dst_entry *xfrm_lookup_with_ifid(struct net *net,
nopol:
if ((!dst_orig->dev || !(dst_orig->dev->flags & IFF_LOOPBACK)) &&
- net->xfrm.policy_default[dir] == XFRM_USERPOLICY_BLOCK) {
+ READ_ONCE(net->xfrm.policy_default[dir]) == XFRM_USERPOLICY_BLOCK) {
err = -EPERM;
goto error;
}
@@ -3744,7 +3747,7 @@ int __xfrm_policy_check(struct sock *sk, int dir, struct sk_buff *skb,
const bool is_crypto_offload = sp &&
(xfrm_input_state(skb)->xso.type == XFRM_DEV_OFFLOAD_CRYPTO);
- if (net->xfrm.policy_default[dir] == XFRM_USERPOLICY_BLOCK) {
+ if (READ_ONCE(net->xfrm.policy_default[dir]) == XFRM_USERPOLICY_BLOCK) {
XFRM_INC_STATS(net, LINUX_MIB_XFRMINNOPOLS);
return 0;
}
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 04cb20163802..792e5ef9361a 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -1144,9 +1144,11 @@ struct xfrm_state *xfrm_input_state_lookup(struct net *net, u32 mark,
struct hlist_head *state_cache_input;
struct xfrm_state *x = NULL;
+ /* BH is always disabled on the input path. */
+ lockdep_assert_in_softirq();
+
state_cache_input = raw_cpu_ptr(net->xfrm.state_cache_input);
- rcu_read_lock();
hlist_for_each_entry_rcu(x, state_cache_input, state_cache_input) {
if (x->props.family != family ||
x->id.spi != spi ||
@@ -1164,20 +1166,25 @@ struct xfrm_state *xfrm_input_state_lookup(struct net *net, u32 mark,
xfrm_hash_ptrs_get(net, &state_ptrs);
x = __xfrm_state_lookup(&state_ptrs, mark, daddr, spi, proto, family);
-
- if (x && x->km.state == XFRM_STATE_VALID) {
- spin_lock_bh(&net->xfrm.xfrm_state_lock);
- if (hlist_unhashed(&x->state_cache_input)) {
+ if (x) {
+ spin_lock(&net->xfrm.xfrm_state_lock);
+ if (x->km.state != XFRM_STATE_VALID) {
+ /*
+ * The state is about to be destroyed.
+ *
+ * Don't add it to the cache but still
+ * return it to the caller.
+ */
+ } else if (hlist_unhashed(&x->state_cache_input)) {
hlist_add_head_rcu(&x->state_cache_input, state_cache_input);
} else {
hlist_del_rcu(&x->state_cache_input);
hlist_add_head_rcu(&x->state_cache_input, state_cache_input);
}
- spin_unlock_bh(&net->xfrm.xfrm_state_lock);
+ spin_unlock(&net->xfrm.xfrm_state_lock);
}
out:
- rcu_read_unlock();
return x;
}
EXPORT_SYMBOL(xfrm_input_state_lookup);
@@ -1954,6 +1961,8 @@ static struct xfrm_state *xfrm_state_clone(struct xfrm_state *orig,
if (!x->encap)
goto error;
+ x->mapping_maxage = orig->mapping_maxage;
+ x->nat_keepalive_interval = orig->nat_keepalive_interval;
}
if (orig->security)
@@ -1988,7 +1997,6 @@ static struct xfrm_state *xfrm_state_clone(struct xfrm_state *orig,
x->km.seq = orig->km.seq;
x->replay = orig->replay;
x->preplay = orig->preplay;
- x->mapping_maxage = orig->mapping_maxage;
x->lastused = orig->lastused;
x->new_mapping = 0;
x->new_mapping_sport = 0;
@@ -2884,7 +2892,7 @@ int xfrm_user_policy(struct sock *sk, int optname, sockptr_t optval, int optlen)
if (IS_ERR(data))
return PTR_ERR(data);
- if (in_compat_syscall()) {
+ if (IS_ENABLED(CONFIG_COMPAT_FOR_U64_ALIGNMENT) && in_compat_syscall()) {
struct xfrm_translator *xtr = xfrm_get_translator();
if (!xtr) {
diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index 50d916654b52..9da6bb095888 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -2409,9 +2409,9 @@ static int xfrm_notify_userpolicy(struct net *net)
}
up = nlmsg_data(nlh);
- up->in = net->xfrm.policy_default[XFRM_POLICY_IN];
- up->fwd = net->xfrm.policy_default[XFRM_POLICY_FWD];
- up->out = net->xfrm.policy_default[XFRM_POLICY_OUT];
+ up->in = READ_ONCE(net->xfrm.policy_default[XFRM_POLICY_IN]);
+ up->fwd = READ_ONCE(net->xfrm.policy_default[XFRM_POLICY_FWD]);
+ up->out = READ_ONCE(net->xfrm.policy_default[XFRM_POLICY_OUT]);
nlmsg_end(skb, nlh);
@@ -2435,13 +2435,13 @@ static int xfrm_set_default(struct sk_buff *skb, struct nlmsghdr *nlh,
struct xfrm_userpolicy_default *up = nlmsg_data(nlh);
if (xfrm_userpolicy_is_valid(up->in))
- net->xfrm.policy_default[XFRM_POLICY_IN] = up->in;
+ WRITE_ONCE(net->xfrm.policy_default[XFRM_POLICY_IN], up->in);
if (xfrm_userpolicy_is_valid(up->fwd))
- net->xfrm.policy_default[XFRM_POLICY_FWD] = up->fwd;
+ WRITE_ONCE(net->xfrm.policy_default[XFRM_POLICY_FWD], up->fwd);
if (xfrm_userpolicy_is_valid(up->out))
- net->xfrm.policy_default[XFRM_POLICY_OUT] = up->out;
+ WRITE_ONCE(net->xfrm.policy_default[XFRM_POLICY_OUT], up->out);
rt_genid_bump_all(net);
@@ -2471,9 +2471,9 @@ static int xfrm_get_default(struct sk_buff *skb, struct nlmsghdr *nlh,
}
r_up = nlmsg_data(r_nlh);
- r_up->in = net->xfrm.policy_default[XFRM_POLICY_IN];
- r_up->fwd = net->xfrm.policy_default[XFRM_POLICY_FWD];
- r_up->out = net->xfrm.policy_default[XFRM_POLICY_OUT];
+ r_up->in = READ_ONCE(net->xfrm.policy_default[XFRM_POLICY_IN]);
+ r_up->fwd = READ_ONCE(net->xfrm.policy_default[XFRM_POLICY_FWD]);
+ r_up->out = READ_ONCE(net->xfrm.policy_default[XFRM_POLICY_OUT]);
nlmsg_end(r_skb, r_nlh);
return nlmsg_unicast(net->xfrm.nlsk, r_skb, portid);
@@ -3364,7 +3364,7 @@ static int xfrm_user_rcv_msg(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!netlink_net_capable(skb, CAP_NET_ADMIN))
return -EPERM;
- if (in_compat_syscall()) {
+ if (IS_ENABLED(CONFIG_COMPAT_FOR_U64_ALIGNMENT) && in_compat_syscall()) {
struct xfrm_translator *xtr = xfrm_get_translator();
if (!xtr)
diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
index f62204fe563f..df52ca6fd438 100644
--- a/rust/kernel/alloc/kvec.rs
+++ b/rust/kernel/alloc/kvec.rs
@@ -411,7 +411,7 @@ pub fn into_raw_parts(self) -> (*mut T, usize, usize) {
///
/// v.reserve(10, GFP_KERNEL)?;
/// let cap = v.capacity();
- /// assert!(cap >= 10);
+ /// assert!(cap >= v.len() + 10);
///
/// v.reserve(10, GFP_KERNEL)?;
/// let new_cap = v.capacity();
@@ -479,7 +479,7 @@ pub fn extend_with(&mut self, n: usize, value: T, flags: Flags) -> Result<(), Al
spare[n - 1].write(value);
// SAFETY:
- // - `self.len() + n < self.capacity()` due to the call to reserve above,
+ // - `self.len() + n <= self.capacity()` due to the call to reserve above,
// - the loop and the line above initialized the next `n` elements.
unsafe { self.set_len(self.len() + n) };
diff --git a/rust/kernel/block/mq/gen_disk.rs b/rust/kernel/block/mq/gen_disk.rs
index 8cd47ddd1dbb..c40959844440 100644
--- a/rust/kernel/block/mq/gen_disk.rs
+++ b/rust/kernel/block/mq/gen_disk.rs
@@ -6,7 +6,7 @@
//! C header: [`include/linux/blk-mq.h`](srctree/include/linux/blk-mq.h)
use crate::block::mq::{raw_writer::RawWriter, Operations, TagSet};
-use crate::{bindings, error::from_err_ptr, error::Result, sync::Arc};
+use crate::{bindings, error::from_err_ptr, error::Result, sync::Arc, types::ScopeGuard};
use crate::{error, static_lock_class};
use core::fmt::{self, Write};
@@ -139,6 +139,12 @@ pub fn build<T: Operations>(
// SAFETY: `gendisk` is a valid pointer as we initialized it above
unsafe { (*gendisk).fops = &TABLE };
+ let cleanup_failure = ScopeGuard::new_with_data(gendisk, |gendisk| {
+ // SAFETY: `gendisk` came from `__blk_mq_alloc_disk()` above and
+ // has not been added to the VFS on this cleanup path.
+ unsafe { bindings::put_disk(gendisk) };
+ });
+
let mut raw_writer = RawWriter::from_array(
// SAFETY: `gendisk` points to a valid and initialized instance. We
// have exclusive access, since the disk is not added to the VFS
@@ -161,6 +167,8 @@ pub fn build<T: Operations>(
},
)?;
+ cleanup_failure.dismiss();
+
// INVARIANT: `gendisk` was initialized above.
// INVARIANT: `gendisk` was added to the VFS via `device_add_disk` above.
Ok(GenDisk {
@@ -192,5 +200,10 @@ fn drop(&mut self) {
// initialized instance of `struct gendisk`, and it was previously added
// to the VFS.
unsafe { bindings::del_gendisk(self.gendisk) };
+
+ // SAFETY: By type invariant, `self.gendisk` was added to the VFS, so
+ // `put_disk()` must follow `del_gendisk()` to drop the final gendisk
+ // reference and trigger the remaining release path.
+ unsafe { bindings::put_disk(self.gendisk) };
}
}
diff --git a/rust/kernel/block/mq/request.rs b/rust/kernel/block/mq/request.rs
index 7943f43b9575..1c7937653542 100644
--- a/rust/kernel/block/mq/request.rs
+++ b/rust/kernel/block/mq/request.rs
@@ -203,6 +203,7 @@ unsafe impl<T: Operations> Sync for Request<T> {}
/// Store the result of `op(target.load())` in target, returning new value of
/// target.
fn atomic_relaxed_op_return(target: &AtomicU64, op: impl Fn(u64) -> u64) -> u64 {
+ #[allow(deprecated)]
let old = target.fetch_update(Ordering::Relaxed, Ordering::Relaxed, |x| Some(op(x)));
// SAFETY: Because the operation passed to `fetch_update` above always
@@ -215,6 +216,7 @@ fn atomic_relaxed_op_return(target: &AtomicU64, op: impl Fn(u64) -> u64) -> u64
/// Store the result of `op(target.load)` in `target` if `target.load() !=
/// pred`, returning [`true`] if the target was updated.
fn atomic_relaxed_op_unless(target: &AtomicU64, op: impl Fn(u64) -> u64, pred: u64) -> bool {
+ #[allow(deprecated)]
target
.fetch_update(Ordering::Relaxed, Ordering::Relaxed, |x| {
if x == pred {
diff --git a/samples/trace_events/trace-events-sample.c b/samples/trace_events/trace-events-sample.c
index ecc7db237f2e..0b7a6efdb247 100644
--- a/samples/trace_events/trace-events-sample.c
+++ b/samples/trace_events/trace-events-sample.c
@@ -107,6 +107,10 @@ int foo_bar_reg(void)
* for consistency sake, we still take the thread_mutex.
*/
simple_tsk_fn = kthread_run(simple_thread_fn, NULL, "event-sample-fn");
+ if (IS_ERR_OR_NULL(simple_tsk_fn)) {
+ pr_err("Failed to create simple_thread_fn\n");
+ simple_tsk_fn = NULL;
+ }
out:
mutex_unlock(&thread_mutex);
return 0;
diff --git a/scripts/kconfig/conf.c b/scripts/kconfig/conf.c
index 3d7d454c54da..7f4b371ef41f 100644
--- a/scripts/kconfig/conf.c
+++ b/scripts/kconfig/conf.c
@@ -297,9 +297,7 @@ static int conf_askvalue(struct symbol *sym, const char *def)
line[1] = 0;
if (!sym_is_changeable(sym)) {
- printf("%s\n", def);
- line[0] = '\n';
- line[1] = 0;
+ printf("%s\n", def ?: "");
return 0;
}
@@ -307,7 +305,7 @@ static int conf_askvalue(struct symbol *sym, const char *def)
case oldconfig:
case syncconfig:
if (sym_has_value(sym)) {
- printf("%s\n", def);
+ printf("%s\n", def ?: "");
return 0;
}
/* fall through */
diff --git a/security/apparmor/apparmorfs.c b/security/apparmor/apparmorfs.c
index c12406e62e58..c94c7c9f227b 100644
--- a/security/apparmor/apparmorfs.c
+++ b/security/apparmor/apparmorfs.c
@@ -71,10 +71,10 @@
struct rawdata_f_data {
struct aa_loaddata *loaddata;
+ DECLARE_FLEX_ARRAY(char, data);
};
#ifdef CONFIG_SECURITY_APPARMOR_EXPORT_BINARY
-#define RAWDATA_F_DATA_BUF(p) (char *)(p + 1)
static void rawdata_f_data_free(struct rawdata_f_data *private)
{
@@ -174,6 +174,7 @@ static struct aa_proxy *get_proxy_common_ref(struct aa_common_ref *ref)
return NULL;
}
+#ifdef CONFIG_SECURITY_APPARMOR_EXPORT_BINARY
static struct aa_loaddata *get_loaddata_common_ref(struct aa_common_ref *ref)
{
if (ref)
@@ -181,6 +182,7 @@ static struct aa_loaddata *get_loaddata_common_ref(struct aa_common_ref *ref)
count));
return NULL;
}
+#endif
static void aa_put_common_ref(struct aa_common_ref *ref)
{
@@ -1432,7 +1434,7 @@ static ssize_t rawdata_read(struct file *file, char __user *buf, size_t size,
struct rawdata_f_data *private = file->private_data;
return simple_read_from_buffer(buf, size, ppos,
- RAWDATA_F_DATA_BUF(private),
+ private->data,
private->loaddata->size);
}
@@ -1465,8 +1467,7 @@ static int rawdata_open(struct inode *inode, struct file *file)
private->loaddata = loaddata;
error = decompress_zstd(loaddata->data, loaddata->compressed_size,
- RAWDATA_F_DATA_BUF(private),
- loaddata->size);
+ private->data, loaddata->size);
if (error)
goto fail_decompress;
@@ -1750,6 +1751,80 @@ static const struct inode_operations rawdata_link_abi_iops = {
static const struct inode_operations rawdata_link_data_iops = {
.get_link = rawdata_get_link_data,
};
+
+/*
+ * Requires: @profile->ns->lock held
+ */
+void __aa_remove_rawdata_symlink_dents(struct aa_profile *profile)
+{
+ aafs_remove(profile->dents[AAFS_PROF_RAW_HASH]);
+ profile->dents[AAFS_PROF_RAW_HASH] = NULL;
+ aafs_remove(profile->dents[AAFS_PROF_RAW_ABI]);
+ profile->dents[AAFS_PROF_RAW_ABI] = NULL;
+ aafs_remove(profile->dents[AAFS_PROF_RAW_DATA]);
+ profile->dents[AAFS_PROF_RAW_DATA] = NULL;
+}
+
+static inline int create_symlink_dent(struct aa_profile *profile,
+ const char *name,
+ enum aafs_prof_type type,
+ const struct inode_operations *iops)
+{
+ struct dentry *dent = NULL;
+ struct dentry *dir = prof_dir(profile);
+
+ if (profile->dents[type])
+ return 0;
+
+ dent = aafs_create(name, S_IFLNK | 0444, dir,
+ &profile->label.proxy->count, NULL, NULL, iops);
+ if (IS_ERR(dent))
+ return PTR_ERR(dent);
+
+ profile->dents[type] = dent;
+ return 0;
+}
+
+/*
+ * Requires: @profile->ns->lock held
+ */
+int __aa_create_rawdata_symlink_dents(struct aa_profile *profile)
+{
+ int error;
+
+ if (!profile ||
+ (profile->dents[AAFS_PROF_RAW_HASH] &&
+ profile->dents[AAFS_PROF_RAW_ABI] &&
+ profile->dents[AAFS_PROF_RAW_DATA]))
+ return 0;
+
+ if (!profile->rawdata)
+ return 0;
+
+ if (aa_g_hash_policy) {
+ error = create_symlink_dent(profile, "raw_sha256",
+ AAFS_PROF_RAW_HASH,
+ &rawdata_link_sha256_iops);
+ if (error)
+ return error;
+ }
+
+ error = create_symlink_dent(profile, "raw_abi",
+ AAFS_PROF_RAW_ABI,
+ &rawdata_link_abi_iops);
+ if (error)
+ return error;
+
+
+ error = create_symlink_dent(profile, "raw_data",
+ AAFS_PROF_RAW_DATA,
+ &rawdata_link_data_iops);
+ if (error)
+ return error;
+
+ return 0;
+}
+
#endif /* CONFIG_SECURITY_APPARMOR_EXPORT_BINARY */
/*
@@ -1825,31 +1900,9 @@ int __aafs_profile_mkdir(struct aa_profile *profile, struct dentry *parent)
profile->dents[AAFS_PROF_HASH] = dent;
}
-#ifdef CONFIG_SECURITY_APPARMOR_EXPORT_BINARY
- if (profile->rawdata) {
- if (aa_g_hash_policy) {
- dent = aafs_create("raw_sha256", S_IFLNK | 0444, dir,
- &profile->label.proxy->count, NULL,
- NULL, &rawdata_link_sha256_iops);
- if (IS_ERR(dent))
- goto fail;
- profile->dents[AAFS_PROF_RAW_HASH] = dent;
- }
- dent = aafs_create("raw_abi", S_IFLNK | 0444, dir,
- &profile->label.proxy->count, NULL, NULL,
- &rawdata_link_abi_iops);
- if (IS_ERR(dent))
- goto fail;
- profile->dents[AAFS_PROF_RAW_ABI] = dent;
-
- dent = aafs_create("raw_data", S_IFLNK | 0444, dir,
- &profile->label.proxy->count, NULL, NULL,
- &rawdata_link_data_iops);
- if (IS_ERR(dent))
- goto fail;
- profile->dents[AAFS_PROF_RAW_DATA] = dent;
- }
-#endif /*CONFIG_SECURITY_APPARMOR_EXPORT_BINARY */
+ error = __aa_create_rawdata_symlink_dents(profile);
+ if (error)
+ goto fail2;
list_for_each_entry(child, &profile->base.profiles, base.list) {
error = __aafs_profile_mkdir(child, prof_child_dir(profile));
diff --git a/security/apparmor/domain.c b/security/apparmor/domain.c
index fbfb1d48dc88..ced513197ed5 100644
--- a/security/apparmor/domain.c
+++ b/security/apparmor/domain.c
@@ -12,6 +12,7 @@
#include <linux/fs.h>
#include <linux/file.h>
#include <linux/mount.h>
+#include <linux/mutex.h>
#include <linux/syscalls.h>
#include <linux/personality.h>
#include <linux/xattr.h>
@@ -1068,6 +1069,7 @@ static struct aa_label *change_hat(const struct cred *subj_cred,
int count, int flags)
{
struct aa_profile *profile, *root, *hat = NULL;
+ struct aa_ns *ns, *new_ns;
struct aa_label *new;
struct label_it it;
bool sibling = false;
@@ -1078,6 +1080,32 @@ static struct aa_label *change_hat(const struct cred *subj_cred,
AA_BUG(!hats);
AA_BUG(count < 1);
+ /*
+ * Acquire the newest label and then hold the lock until we choose a
+ * hat, so that profile replacement doesn't atomically truncate the
+ * list of potential hats. Because we are getting the namespaces from
+ * the profiles and label, we can rely on the namespaces being live
+ * and avoid incrementing their refcounts while grabbing the lock.
+ */
+ label = aa_get_label(label);
+ ns = labels_ns(label);
+
+retry:
+ mutex_lock_nested(&ns->lock, ns->level);
+ if (label_is_stale(label)) {
+ new = aa_get_newest_label(label);
+ new_ns = labels_ns(new);
+ if (new_ns != ns) {
+ aa_put_label(new);
+ mutex_unlock(&ns->lock);
+ ns = new_ns;
+ label = new;
+ goto retry;
+ }
+ aa_put_label(label);
+ label = new;
+ }
+
if (PROFILE_IS_HAT(labels_profile(label)))
sibling = true;
@@ -1086,7 +1114,7 @@ static struct aa_label *change_hat(const struct cred *subj_cred,
name = hats[i];
label_for_each_in_ns(it, labels_ns(label), label, profile) {
if (sibling && PROFILE_IS_HAT(profile)) {
- root = aa_get_profile_rcu(&profile->parent);
+ root = aa_get_profile(profile->parent);
} else if (!sibling && !PROFILE_IS_HAT(profile)) {
root = aa_get_profile(profile);
} else { /* conflicting change type */
@@ -1146,6 +1174,7 @@ static struct aa_label *change_hat(const struct cred *subj_cred,
GLOBAL_ROOT_UID, info, error);
}
}
+ mutex_unlock(&ns->lock);
return ERR_PTR(error);
build:
@@ -1158,7 +1187,7 @@ static struct aa_label *change_hat(const struct cred *subj_cred,
error = -ENOMEM;
goto fail;
} /* else if (IS_ERR) build_change_hat has logged error so return new */
-
+ mutex_unlock(&ns->lock);
return new;
}
@@ -1471,6 +1500,8 @@ int aa_change_profile(const char *fqname, int flags)
new = fn_label_build_in_ns(label, profile, GFP_KERNEL,
aa_get_label(target),
aa_get_label(&profile->label));
+ if (IS_ERR_OR_NULL(new))
+ goto build_fail;
/*
* no new privs prevents domain transitions that would
* reduce restrictions.
@@ -1488,16 +1519,8 @@ int aa_change_profile(const char *fqname, int flags)
/* only transition profiles in the current ns */
if (stack)
new = aa_label_merge(label, target, GFP_KERNEL);
- if (IS_ERR_OR_NULL(new)) {
- info = "failed to build target label";
- if (!new)
- error = -ENOMEM;
- else
- error = PTR_ERR(new);
- new = NULL;
- perms.allow = 0;
- goto audit;
- }
+ if (IS_ERR_OR_NULL(new))
+ goto build_fail;
error = aa_replace_current_label(new);
} else {
if (new) {
@@ -1509,6 +1532,17 @@ int aa_change_profile(const char *fqname, int flags)
aa_set_current_onexec(target, stack);
}
+ goto audit;
+
+build_fail:
+ info = "failed to build target label";
+ if (!new)
+ error = -ENOMEM;
+ else
+ error = PTR_ERR(new);
+ new = NULL;
+ perms.allow = 0;
+
audit:
error = fn_for_each_in_ns(label, profile,
aa_audit_file(subj_cred,
diff --git a/security/apparmor/include/apparmorfs.h b/security/apparmor/include/apparmorfs.h
index 1e94904f68d9..9224497736fd 100644
--- a/security/apparmor/include/apparmorfs.h
+++ b/security/apparmor/include/apparmorfs.h
@@ -118,6 +118,8 @@ struct aa_loaddata;
#ifdef CONFIG_SECURITY_APPARMOR_EXPORT_BINARY
void __aa_fs_remove_rawdata(struct aa_loaddata *rawdata);
int __aa_fs_create_rawdata(struct aa_ns *ns, struct aa_loaddata *rawdata);
+void __aa_remove_rawdata_symlink_dents(struct aa_profile *profile);
+int __aa_create_rawdata_symlink_dents(struct aa_profile *profile);
#else
static inline void __aa_fs_remove_rawdata(struct aa_loaddata *rawdata)
{
@@ -129,6 +131,16 @@ static inline int __aa_fs_create_rawdata(struct aa_ns *ns,
{
return 0;
}
+
+static inline void __aa_remove_rawdata_symlink_dents(struct aa_profile *profile)
+{
+ /* empty stub */
+}
+
+static inline int __aa_create_rawdata_symlink_dents(struct aa_profile *profile)
+{
+ return 0;
+}
#endif /* CONFIG_SECURITY_APPARMOR_EXPORT_BINARY */
#endif /* __AA_APPARMORFS_H */
diff --git a/security/apparmor/label.c b/security/apparmor/label.c
index 3bcc45437b44..04fe4d16952d 100644
--- a/security/apparmor/label.c
+++ b/security/apparmor/label.c
@@ -454,7 +454,7 @@ struct aa_label *aa_label_alloc(int size, struct aa_proxy *proxy, gfp_t gfp)
return new;
fail:
- kfree(new);
+ aa_label_free(new);
return NULL;
}
diff --git a/security/apparmor/net.c b/security/apparmor/net.c
index b6df6ff6bd9c..6e989184e408 100644
--- a/security/apparmor/net.c
+++ b/security/apparmor/net.c
@@ -217,6 +217,7 @@ static int apparmor_secmark_init(struct aa_secmark *secmark)
return PTR_ERR(label);
secmark->secid = label->secid;
+ aa_put_label(label);
return 0;
}
diff --git a/security/apparmor/policy.c b/security/apparmor/policy.c
index 9140b613c0cf..da39bd937c2a 100644
--- a/security/apparmor/policy.c
+++ b/security/apparmor/policy.c
@@ -1298,6 +1298,16 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
goto skip;
}
+ if (!aa_g_export_binary) {
+ if (ent->old && ent->old->rawdata &&
+ ent->old->dents[AAFS_LOADDATA_DIR]) {
+ /* remove rawdata symlinks because the symlink
+ * target will be removed
+ */
+ __aa_remove_rawdata_symlink_dents(ent->old);
+ }
+ }
+
/*
* TODO: finer dedup based on profile range in data. Load set
* can differ but profile may remain unchanged
@@ -1308,6 +1318,11 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
if (ent->old) {
share_name(ent->old, ent->new);
__replace_profile(ent->old, ent->new);
+ if (aa_g_export_binary) {
+ /* recreate rawdata symlinks */
+ if (!ent->old->rawdata)
+ __aa_create_rawdata_symlink_dents(ent->new);
+ }
} else {
struct list_head *lh;
@@ -1328,12 +1343,15 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
out:
aa_put_ns(ns);
+
+ ssize_t udata_sz = udata->size;
+
aa_put_profile_loaddata(udata);
kfree(ns_name);
if (error)
return error;
- return udata->size;
+ return udata_sz;
fail_lock:
mutex_unlock(&ns->lock);
diff --git a/security/apparmor/procattr.c b/security/apparmor/procattr.c
index e3857e3d7c6c..7ed6e4fe3f55 100644
--- a/security/apparmor/procattr.c
+++ b/security/apparmor/procattr.c
@@ -54,6 +54,8 @@ int aa_getprocattr(struct aa_label *label, char **string, bool newline)
FLAG_SHOW_MODE | FLAG_VIEW_SUBNS |
FLAG_HIDDEN_UNCONFINED);
if (len < 0) {
+ kfree(*string);
+ *string = NULL;
aa_put_ns(current_ns);
return len;
}
diff --git a/security/bpf/hooks.c b/security/bpf/hooks.c
index 3663aec7bcbd..aa1c7915ed22 100644
--- a/security/bpf/hooks.c
+++ b/security/bpf/hooks.c
@@ -7,6 +7,8 @@
#include <linux/bpf_lsm.h>
#include <uapi/linux/lsm.h>
+bool bpf_lsm_initialized __ro_after_init;
+
static struct security_hook_list bpf_lsm_hooks[] __ro_after_init = {
#define LSM_HOOK(RET, DEFAULT, NAME, ...) \
LSM_HOOK_INIT(NAME, bpf_lsm_##NAME),
@@ -25,6 +27,7 @@ static int __init bpf_lsm_init(void)
{
security_add_hooks(bpf_lsm_hooks, ARRAY_SIZE(bpf_lsm_hooks),
&bpf_lsmid);
+ bpf_lsm_initialized = true;
pr_info("LSM support for eBPF active\n");
return 0;
}
diff --git a/security/integrity/evm/evm_secfs.c b/security/integrity/evm/evm_secfs.c
index 9b907c2fee60..07c6982e1fac 100644
--- a/security/integrity/evm/evm_secfs.c
+++ b/security/integrity/evm/evm_secfs.c
@@ -128,8 +128,8 @@ static ssize_t evm_read_xattrs(struct file *filp, char __user *buf,
size_t count, loff_t *ppos)
{
char *temp;
- int offset = 0;
- ssize_t rc, size = 0;
+ size_t offset = 0, size = 0;
+ ssize_t rc;
struct xattr_list *xattr;
if (*ppos != 0)
@@ -152,16 +152,22 @@ static ssize_t evm_read_xattrs(struct file *filp, char __user *buf,
return -ENOMEM;
}
+ temp[size] = '\0';
+
+ /*
+ * No truncation possible: size is computed over the same enabled
+ * xattrs under xattr_list_mutex, so offset never exceeds size.
+ */
list_for_each_entry(xattr, &evm_config_xattrnames, list) {
if (!xattr->enabled)
continue;
- sprintf(temp + offset, "%s\n", xattr->name);
- offset += strlen(xattr->name) + 1;
+ offset += snprintf(temp + offset, size + 1 - offset, "%s\n",
+ xattr->name);
}
mutex_unlock(&xattr_list_mutex);
- rc = simple_read_from_buffer(buf, count, ppos, temp, strlen(temp));
+ rc = simple_read_from_buffer(buf, count, ppos, temp, offset);
kfree(temp);
diff --git a/security/keys/keyring.c b/security/keys/keyring.c
index df2580072cfe..e105349794f2 100644
--- a/security/keys/keyring.c
+++ b/security/keys/keyring.c
@@ -576,7 +576,7 @@ static int keyring_search_iterator(const void *object, void *iterator_data)
struct keyring_search_context *ctx = iterator_data;
const struct key *key = keyring_ptr_to_key(object);
unsigned long kflags = READ_ONCE(key->flags);
- short state = READ_ONCE(key->state);
+ short state = key_read_state(key);
kenter("{%d}", key->serial);
diff --git a/security/landlock/net.c b/security/landlock/net.c
index 53d479893475..9c9608924fbd 100644
--- a/security/landlock/net.c
+++ b/security/landlock/net.c
@@ -48,6 +48,7 @@ static int current_check_access_socket(struct socket *const sock,
const int addrlen,
access_mask_t access_request)
{
+ unsigned short sock_family;
__be16 port;
layer_mask_t layer_masks[LANDLOCK_NUM_ACCESS_NET] = {};
const struct landlock_rule *rule;
@@ -70,6 +71,12 @@ static int current_check_access_socket(struct socket *const sock,
if (addrlen < offsetofend(typeof(*address), sa_family))
return -EINVAL;
+ /*
+ * The socket is not locked, so sk_family can change concurrently due to
+ * e.g. setsockopt(IPV6_ADDRFORM).
+ */
+ sock_family = READ_ONCE(sock->sk->sk_family);
+
switch (address->sa_family) {
case AF_UNSPEC:
if (access_request == LANDLOCK_ACCESS_NET_CONNECT_TCP) {
@@ -106,7 +113,7 @@ static int current_check_access_socket(struct socket *const sock,
* these checks, but it is safer to return a proper
* error and test consistency thanks to kselftest.
*/
- if (sock->sk->__sk_common.skc_family == AF_INET) {
+ if (sock_family == AF_INET) {
const struct sockaddr_in *const sockaddr =
(struct sockaddr_in *)address;
@@ -154,7 +161,7 @@ static int current_check_access_socket(struct socket *const sock,
* check, but it is safer to return a proper error and test
* consistency thanks to kselftest.
*/
- if (address->sa_family != sock->sk->__sk_common.skc_family &&
+ if (address->sa_family != sock_family &&
address->sa_family != AF_UNSPEC)
return -EINVAL;
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
index 1b89c8d5fa2f..cd60ece2fd58 100644
--- a/security/selinux/hooks.c
+++ b/security/selinux/hooks.c
@@ -3804,9 +3804,9 @@ static int selinux_file_ioctl_compat(struct file *file, unsigned int cmd,
static int default_noexec __ro_after_init;
-static int __file_map_prot_check(const struct cred *cred,
- const struct file *file, unsigned long prot,
- bool shared, bool bf_user_file)
+static int __file_map_prot_check(const struct file *file, unsigned long prot,
+ bool shared, bool mounter_check,
+ bool bf_user_file)
{
struct inode *inode = NULL;
bool prot_exec = prot & PROT_EXEC;
@@ -3819,10 +3819,10 @@ static int __file_map_prot_check(const struct cred *cred,
inode = file_inode(file);
}
- if (default_noexec && prot_exec &&
+ if (!mounter_check && default_noexec && prot_exec &&
(!file || IS_PRIVATE(inode) || (!shared && prot_write))) {
int rc;
- u32 sid = cred_sid(cred);
+ u32 sid = current_sid();
/*
* We are making executable an anonymous mapping or a private
@@ -3835,6 +3835,8 @@ static int __file_map_prot_check(const struct cred *cred,
}
if (file) {
+ const struct cred *cred = mounter_check ?
+ file->f_cred : current_cred();
/* "read" always possible, "write" only if shared */
u32 av = FILE__READ;
if (shared && prot_write)
@@ -3848,11 +3850,11 @@ static int __file_map_prot_check(const struct cred *cred,
return 0;
}
-static inline int file_map_prot_check(const struct cred *cred,
- const struct file *file,
- unsigned long prot, bool shared)
+static inline int file_map_prot_check(const struct file *file,
+ unsigned long prot, bool shared,
+ bool mounter_check)
{
- return __file_map_prot_check(cred, file, prot, shared, false);
+ return __file_map_prot_check(file, prot, shared, mounter_check, false);
}
static int selinux_mmap_addr(unsigned long addr)
@@ -3868,12 +3870,14 @@ static int selinux_mmap_addr(unsigned long addr)
return rc;
}
-static int selinux_mmap_file_common(const struct cred *cred, struct file *file,
- unsigned long prot, bool shared)
+static int selinux_mmap_file_common(struct file *file, unsigned long prot,
+ bool shared, bool mounter_check)
{
if (file) {
int rc;
struct common_audit_data ad;
+ const struct cred *cred = mounter_check ?
+ file->f_cred : current_cred();
ad.type = LSM_AUDIT_DATA_FILE;
ad.u.file = file;
@@ -3882,15 +3886,16 @@ static int selinux_mmap_file_common(const struct cred *cred, struct file *file,
return rc;
}
- return file_map_prot_check(cred, file, prot, shared);
+ return file_map_prot_check(file, prot, shared, mounter_check);
}
static int selinux_mmap_file(struct file *file,
unsigned long reqprot __always_unused,
unsigned long prot, unsigned long flags)
{
- return selinux_mmap_file_common(current_cred(), file, prot,
- (flags & MAP_TYPE) == MAP_SHARED);
+ return selinux_mmap_file_common(file, prot,
+ (flags & MAP_TYPE) == MAP_SHARED,
+ false);
}
/**
@@ -3922,8 +3927,9 @@ static int selinux_mmap_backing_file(struct vm_area_struct *vma,
if (vma->vm_flags & VM_EXEC)
prot |= PROT_EXEC;
- return selinux_mmap_file_common(backing_file->f_cred, backing_file,
- prot, vma->vm_flags & VM_SHARED);
+ return selinux_mmap_file_common(backing_file, prot,
+ vma->vm_flags & VM_SHARED,
+ true);
}
static int selinux_file_mprotect(struct vm_area_struct *vma,
@@ -3984,11 +3990,11 @@ static int selinux_file_mprotect(struct vm_area_struct *vma,
}
}
- rc = __file_map_prot_check(cred, file, prot, shared, backing_file);
+ rc = __file_map_prot_check(file, prot, shared, false, backing_file);
if (rc)
return rc;
if (backing_file) {
- rc = file_map_prot_check(file->f_cred, file, prot, shared);
+ rc = file_map_prot_check(file, prot, shared, true);
if (rc)
return rc;
}
@@ -4801,9 +4807,8 @@ static int selinux_socket_socketpair(struct socket *socka,
Need to determine whether we should perform a name_bind
permission check between the socket and the port number. */
-static int selinux_socket_bind(struct socket *sock, struct sockaddr *address, int addrlen)
+static int __selinux_socket_bind(struct sock *sk, struct sockaddr *address, int addrlen)
{
- struct sock *sk = sock->sk;
struct sk_security_struct *sksec = selinux_sock(sk);
u16 family;
int err;
@@ -4937,13 +4942,17 @@ static int selinux_socket_bind(struct socket *sock, struct sockaddr *address, in
return -EAFNOSUPPORT;
}
+static int selinux_socket_bind(struct socket *sock, struct sockaddr *address, int addrlen)
+{
+ return __selinux_socket_bind(sock->sk, address, addrlen);
+}
+
/* This supports connect(2) and SCTP connect services such as sctp_connectx(3)
* and sctp_sendmsg(3) as described in Documentation/security/SCTP.rst
*/
-static int selinux_socket_connect_helper(struct socket *sock,
+static int selinux_socket_connect_helper(struct sock *sk,
struct sockaddr *address, int addrlen)
{
- struct sock *sk = sock->sk;
struct sk_security_struct *sksec = selinux_sock(sk);
int err;
@@ -5036,7 +5045,7 @@ static int selinux_socket_connect(struct socket *sock,
int err;
struct sock *sk = sock->sk;
- err = selinux_socket_connect_helper(sock, address, addrlen);
+ err = selinux_socket_connect_helper(sk, address, addrlen);
if (err)
return err;
@@ -5077,7 +5086,24 @@ static int selinux_socket_accept(struct socket *sock, struct socket *newsock)
static int selinux_socket_sendmsg(struct socket *sock, struct msghdr *msg,
int size)
{
- return sock_has_perm(sock->sk, SOCKET__WRITE);
+ int rc;
+ struct sockaddr *const addr = msg->msg_name;
+ const int addrlen = msg->msg_namelen;
+
+ rc = sock_has_perm(sock->sk, SOCKET__WRITE);
+ if (rc)
+ return rc;
+
+ if (addr && (msg->msg_flags & MSG_FASTOPEN) &&
+ (sk_is_tcp(sock->sk) ||
+ (sk_is_inet(sock->sk) && sock->sk->sk_type == SOCK_STREAM &&
+ sock->sk->sk_protocol == IPPROTO_MPTCP))) {
+ rc = selinux_socket_connect(sock, addr, addrlen);
+ if (rc)
+ return rc;
+ }
+
+ return 0;
}
static int selinux_socket_recvmsg(struct socket *sock, struct msghdr *msg,
@@ -5521,13 +5547,11 @@ static int selinux_sctp_bind_connect(struct sock *sk, int optname,
int len, err = 0, walk_size = 0;
void *addr_buf;
struct sockaddr *addr;
- struct socket *sock;
if (!selinux_policycap_extsockclass())
return 0;
/* Process one or more addresses that may be IPv4 or IPv6 */
- sock = sk->sk_socket;
addr_buf = address;
while (walk_size < addrlen) {
@@ -5556,14 +5580,14 @@ static int selinux_sctp_bind_connect(struct sock *sk, int optname,
case SCTP_PRIMARY_ADDR:
case SCTP_SET_PEER_PRIMARY_ADDR:
case SCTP_SOCKOPT_BINDX_ADD:
- err = selinux_socket_bind(sock, addr, len);
+ err = __selinux_socket_bind(sk, addr, len);
break;
/* Connect checks */
case SCTP_SOCKOPT_CONNECTX:
case SCTP_PARAM_SET_PRIMARY:
case SCTP_PARAM_ADD_IP:
case SCTP_SENDMSG_CONNECT:
- err = selinux_socket_connect_helper(sock, addr, len);
+ err = selinux_socket_connect_helper(sk, addr, len);
if (err)
return err;
diff --git a/sound/aoa/fabrics/layout.c b/sound/aoa/fabrics/layout.c
index e68b4cb4df29..a161dd913d7e 100644
--- a/sound/aoa/fabrics/layout.c
+++ b/sound/aoa/fabrics/layout.c
@@ -948,6 +948,8 @@ static void layout_attached_codec(struct aoa_codec *codec)
if (lineout == 1)
ldev->gpio.methods->set_lineout(codec->gpio, 1);
ctl = snd_ctl_new1(&lineout_ctl, codec->gpio);
+ if (!ctl)
+ return;
if (cc->connected & CC_LINEOUT_LABELLED_HEADPHONE)
strscpy(ctl->id.name,
"Headphone Switch", sizeof(ctl->id.name));
@@ -962,6 +964,8 @@ static void layout_attached_codec(struct aoa_codec *codec)
if (ldev->have_lineout_detect) {
ctl = snd_ctl_new1(&lineout_detect_choice,
ldev);
+ if (!ctl)
+ return;
if (cc->connected & CC_LINEOUT_LABELLED_HEADPHONE)
strscpy(ctl->id.name,
"Headphone Detect Autoswitch",
@@ -969,6 +973,8 @@ static void layout_attached_codec(struct aoa_codec *codec)
aoa_snd_ctl_add(ctl);
ctl = snd_ctl_new1(&lineout_detected,
ldev);
+ if (!ctl)
+ return;
if (cc->connected & CC_LINEOUT_LABELLED_HEADPHONE)
strscpy(ctl->id.name,
"Headphone Detected",
diff --git a/sound/core/init.c b/sound/core/init.c
index 114fb87de990..e0be60981285 100644
--- a/sound/core/init.c
+++ b/sound/core/init.c
@@ -1133,7 +1133,7 @@ EXPORT_SYMBOL(snd_card_file_remove);
* typically around calling control ops.
*
* The caller needs to pull down the refcount via snd_power_unref() later
- * no matter whether the error is returned from this function or not.
+ * when this function returns 0.
*
* Return: Zero if successful, or a negative error code.
*/
@@ -1146,7 +1146,11 @@ int snd_power_ref_and_wait(struct snd_card *card)
card->shutdown ||
snd_power_get_state(card) == SNDRV_CTL_POWER_D0,
snd_power_unref(card), snd_power_ref(card));
- return card->shutdown ? -ENODEV : 0;
+ if (card->shutdown) {
+ snd_power_unref(card);
+ return -ENODEV;
+ }
+ return 0;
}
EXPORT_SYMBOL_GPL(snd_power_ref_and_wait);
@@ -1163,7 +1167,8 @@ int snd_power_wait(struct snd_card *card)
int ret;
ret = snd_power_ref_and_wait(card);
- snd_power_unref(card);
+ if (!ret)
+ snd_power_unref(card);
return ret;
}
EXPORT_SYMBOL(snd_power_wait);
diff --git a/sound/core/seq/seq_clientmgr.c b/sound/core/seq/seq_clientmgr.c
index 0ddf84b36c13..530a7d9be677 100644
--- a/sound/core/seq/seq_clientmgr.c
+++ b/sound/core/seq/seq_clientmgr.c
@@ -488,6 +488,7 @@ static ssize_t snd_seq_read(struct file *file, char __user *buf, size_t count,
memcpy(&tmpev, &cell->event, aligned_size);
tmpev.data.ext.len &= ~SNDRV_SEQ_EXT_MASK;
+ tmpev.data.ext.ptr = NULL;
if (copy_to_user(buf, &tmpev, aligned_size)) {
err = -EFAULT;
break;
@@ -588,16 +589,38 @@ static int bounce_error_event(struct snd_seq_client *client,
/* set up quoted error */
memset(&bounce_ev, 0, sizeof(bounce_ev));
- bounce_ev.type = SNDRV_SEQ_EVENT_KERNEL_ERROR;
- bounce_ev.flags = SNDRV_SEQ_EVENT_LENGTH_FIXED;
+
+ if (client->type == USER_CLIENT) {
+ /*
+ * For user clients, send SNDRV_SEQ_EVENT_BOUNCE with the
+ * original event embedded as variable-length data. This
+ * avoids exposing data.quote.event (a kernel pointer) to
+ * userspace. The variable-length path in snd_seq_event_dup()
+ * copies the event data from data.ext.ptr into chained cells,
+ * and snd_seq_expand_var_event() copies only the data content
+ * -- never the pointer -- to userspace.
+ */
+ bounce_ev.type = SNDRV_SEQ_EVENT_BOUNCE;
+ bounce_ev.flags = SNDRV_SEQ_EVENT_LENGTH_VARIABLE;
+ bounce_ev.data.ext.len = sizeof(struct snd_seq_event);
+ bounce_ev.data.ext.ptr = (char *)event;
+ } else {
+ /*
+ * For kernel clients, quote the event pointer directly.
+ * Kernel consumers can safely dereference the pointer.
+ */
+ bounce_ev.type = SNDRV_SEQ_EVENT_KERNEL_ERROR;
+ bounce_ev.flags = SNDRV_SEQ_EVENT_LENGTH_FIXED;
+ bounce_ev.data.quote.origin = event->dest;
+ bounce_ev.data.quote.event = event;
+ bounce_ev.data.quote.value = -err; /* use positive value */
+ }
+
bounce_ev.queue = SNDRV_SEQ_QUEUE_DIRECT;
bounce_ev.source.client = SNDRV_SEQ_CLIENT_SYSTEM;
bounce_ev.source.port = SNDRV_SEQ_PORT_SYSTEM_ANNOUNCE;
bounce_ev.dest.client = client->number;
bounce_ev.dest.port = event->source.port;
- bounce_ev.data.quote.origin = event->dest;
- bounce_ev.data.quote.event = event;
- bounce_ev.data.quote.value = -err; /* use positive value */
result = snd_seq_deliver_single_event(NULL, &bounce_ev, atomic, hop + 1);
if (result < 0) {
client->event_lost++;
diff --git a/sound/core/seq/seq_memory.c b/sound/core/seq/seq_memory.c
index e7edea10027f..60afe351bbca 100644
--- a/sound/core/seq/seq_memory.c
+++ b/sound/core/seq/seq_memory.c
@@ -211,7 +211,7 @@ int snd_seq_expand_var_event_at(const struct snd_seq_event *event, int count,
len -= offset;
if (len > count)
len = count;
- err = expand_var_event(event, offset, count, buf, true);
+ err = expand_var_event(event, offset, len, buf, true);
if (err < 0)
return err;
return len;
diff --git a/sound/core/seq/seq_midi.c b/sound/core/seq/seq_midi.c
index ba52a77eda38..38cd631223c6 100644
--- a/sound/core/seq/seq_midi.c
+++ b/sound/core/seq/seq_midi.c
@@ -24,6 +24,7 @@ Possible options for midisynth module:
#include <sound/seq_device.h>
#include <sound/seq_midi_event.h>
#include <sound/initval.h>
+#include "seq_lock.h"
MODULE_AUTHOR("Frank van de Pol <fvdpol@coil.demon.nl>, Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Advanced Linux Sound Architecture sequencer MIDI synth.");
@@ -42,6 +43,8 @@ struct seq_midisynth {
int device;
int subdevice;
struct snd_rawmidi_file input_rfile;
+ spinlock_t output_lock; /* protects output_rfile publication */
+ snd_use_lock_t output_use_lock; /* in-flight event_input users */
struct snd_rawmidi_file output_rfile;
int seq_client;
int seq_port;
@@ -125,31 +128,42 @@ static int event_process_midi(struct snd_seq_event *ev, int direct,
struct seq_midisynth *msynth = private_data;
unsigned char msg[10]; /* buffer for constructing midi messages */
struct snd_rawmidi_substream *substream;
+ int err = 0;
int len;
if (snd_BUG_ON(!msynth))
return -EINVAL;
- substream = msynth->output_rfile.output;
- if (substream == NULL)
- return -ENODEV;
+
+ scoped_guard(spinlock_irqsave, &msynth->output_lock) {
+ substream = msynth->output_rfile.output;
+ if (!substream)
+ return -ENODEV;
+ snd_use_lock_use(&msynth->output_use_lock);
+ }
+
if (ev->type == SNDRV_SEQ_EVENT_SYSEX) { /* special case, to save space */
if ((ev->flags & SNDRV_SEQ_EVENT_LENGTH_MASK) != SNDRV_SEQ_EVENT_LENGTH_VARIABLE) {
/* invalid event */
pr_debug("ALSA: seq_midi: invalid sysex event flags = 0x%x\n", ev->flags);
- return 0;
+ goto out;
}
snd_seq_dump_var_event(ev, __dump_midi, substream);
snd_midi_event_reset_decode(msynth->parser);
} else {
- if (msynth->parser == NULL)
- return -EIO;
+ if (!msynth->parser) {
+ err = -EIO;
+ goto out;
+ }
len = snd_midi_event_decode(msynth->parser, msg, sizeof(msg), ev);
if (len < 0)
- return 0;
+ goto out;
if (dump_midi(substream, msg, len) < 0)
snd_midi_event_reset_decode(msynth->parser);
}
- return 0;
+
+out:
+ snd_use_lock_free(&msynth->output_use_lock);
+ return err;
}
@@ -163,6 +177,8 @@ static int snd_seq_midisynth_new(struct seq_midisynth *msynth,
msynth->card = card;
msynth->device = device;
msynth->subdevice = subdevice;
+ spin_lock_init(&msynth->output_lock);
+ snd_use_lock_init(&msynth->output_use_lock);
return 0;
}
@@ -215,12 +231,13 @@ static int midisynth_use(void *private_data, struct snd_seq_port_subscribe *info
{
int err;
struct seq_midisynth *msynth = private_data;
+ struct snd_rawmidi_file rfile = {};
struct snd_rawmidi_params params;
/* open midi port */
err = snd_rawmidi_kernel_open(msynth->rmidi, msynth->subdevice,
SNDRV_RAWMIDI_LFLG_OUTPUT,
- &msynth->output_rfile);
+ &rfile);
if (err < 0) {
pr_debug("ALSA: seq_midi: midi output open failed!!!\n");
return err;
@@ -229,12 +246,14 @@ static int midisynth_use(void *private_data, struct snd_seq_port_subscribe *info
params.avail_min = 1;
params.buffer_size = output_buffer_size;
params.no_active_sensing = 1;
- err = snd_rawmidi_output_params(msynth->output_rfile.output, ¶ms);
+ err = snd_rawmidi_output_params(rfile.output, ¶ms);
if (err < 0) {
- snd_rawmidi_kernel_release(&msynth->output_rfile);
+ snd_rawmidi_kernel_release(&rfile);
return err;
}
snd_midi_event_reset_decode(msynth->parser);
+ scoped_guard(spinlock_irqsave, &msynth->output_lock)
+ msynth->output_rfile = rfile;
return 0;
}
@@ -242,11 +261,19 @@ static int midisynth_use(void *private_data, struct snd_seq_port_subscribe *info
static int midisynth_unuse(void *private_data, struct snd_seq_port_subscribe *info)
{
struct seq_midisynth *msynth = private_data;
+ struct snd_rawmidi_file rfile = {};
- if (snd_BUG_ON(!msynth->output_rfile.output))
+ scoped_guard(spinlock_irqsave, &msynth->output_lock) {
+ rfile = msynth->output_rfile;
+ msynth->output_rfile = (struct snd_rawmidi_file){};
+ }
+
+ if (snd_BUG_ON(!rfile.output))
return -EINVAL;
- snd_rawmidi_drain_output(msynth->output_rfile.output);
- return snd_rawmidi_kernel_release(&msynth->output_rfile);
+
+ snd_use_lock_sync(&msynth->output_use_lock);
+ snd_rawmidi_drain_output(rfile.output);
+ return snd_rawmidi_kernel_release(&rfile);
}
/* delete given midi synth port */
diff --git a/sound/pci/hda/cs35l41_hda.c b/sound/pci/hda/cs35l41_hda.c
index 42c576d9f117..099b8e9582b8 100644
--- a/sound/pci/hda/cs35l41_hda.c
+++ b/sound/pci/hda/cs35l41_hda.c
@@ -1293,6 +1293,43 @@ static int cs35l41_fw_type_ctl_info(struct snd_kcontrol *kcontrol, struct snd_ct
return snd_ctl_enum_info(uinfo, 1, ARRAY_SIZE(hda_cs_dsp_fw_ids), hda_cs_dsp_fw_ids);
}
+static void cs35l41_remove_controls(struct cs35l41_hda *cs35l41)
+{
+ if (!cs35l41->codec)
+ return;
+
+ snd_ctl_remove(cs35l41->codec->card, cs35l41->mute_override_ctl);
+ cs35l41->mute_override_ctl = NULL;
+
+ snd_ctl_remove(cs35l41->codec->card, cs35l41->fw_load_ctl);
+ cs35l41->fw_load_ctl = NULL;
+
+ snd_ctl_remove(cs35l41->codec->card, cs35l41->fw_type_ctl);
+ cs35l41->fw_type_ctl = NULL;
+}
+
+static int cs35l41_add_control(struct cs35l41_hda *cs35l41,
+ struct snd_kcontrol_new *ctl,
+ struct snd_kcontrol **kctl)
+{
+ int ret;
+
+ *kctl = snd_ctl_new1(ctl, cs35l41);
+ if (!*kctl)
+ return -ENOMEM;
+
+ ret = snd_ctl_add(cs35l41->codec->card, *kctl);
+ if (ret) {
+ dev_err(cs35l41->dev, "Failed to add KControl %s = %d\n", ctl->name, ret);
+ *kctl = NULL;
+ return ret;
+ }
+
+ dev_dbg(cs35l41->dev, "Added Control %s\n", ctl->name);
+
+ return 0;
+}
+
static int cs35l41_create_controls(struct cs35l41_hda *cs35l41)
{
char fw_type_ctl_name[SNDRV_CTL_ELEM_ID_NAME_MAXLEN];
@@ -1328,32 +1365,23 @@ static int cs35l41_create_controls(struct cs35l41_hda *cs35l41)
scnprintf(mute_override_ctl_name, SNDRV_CTL_ELEM_ID_NAME_MAXLEN, "%s Forced Mute Status",
cs35l41->amp_name);
- ret = snd_ctl_add(cs35l41->codec->card, snd_ctl_new1(&fw_type_ctl, cs35l41));
- if (ret) {
- dev_err(cs35l41->dev, "Failed to add KControl %s = %d\n", fw_type_ctl.name, ret);
- return ret;
- }
-
- dev_dbg(cs35l41->dev, "Added Control %s\n", fw_type_ctl.name);
-
- ret = snd_ctl_add(cs35l41->codec->card, snd_ctl_new1(&fw_load_ctl, cs35l41));
- if (ret) {
- dev_err(cs35l41->dev, "Failed to add KControl %s = %d\n", fw_load_ctl.name, ret);
- return ret;
- }
-
- dev_dbg(cs35l41->dev, "Added Control %s\n", fw_load_ctl.name);
+ ret = cs35l41_add_control(cs35l41, &fw_type_ctl, &cs35l41->fw_type_ctl);
+ if (ret)
+ goto err;
- ret = snd_ctl_add(cs35l41->codec->card, snd_ctl_new1(&mute_override_ctl, cs35l41));
- if (ret) {
- dev_err(cs35l41->dev, "Failed to add KControl %s = %d\n", mute_override_ctl.name,
- ret);
- return ret;
- }
+ ret = cs35l41_add_control(cs35l41, &fw_load_ctl, &cs35l41->fw_load_ctl);
+ if (ret)
+ goto err;
- dev_dbg(cs35l41->dev, "Added Control %s\n", mute_override_ctl.name);
+ ret = cs35l41_add_control(cs35l41, &mute_override_ctl, &cs35l41->mute_override_ctl);
+ if (ret)
+ goto err;
return 0;
+
+err:
+ cs35l41_remove_controls(cs35l41);
+ return ret;
}
static bool cs35l41_dsm_supported(acpi_handle handle, unsigned int commands)
@@ -1491,6 +1519,10 @@ static void cs35l41_hda_unbind(struct device *dev, struct device *master, void *
device_link_remove(&cs35l41->codec->core.dev, cs35l41->dev);
unlock_system_sleep(sleep_flags);
memset(comp, 0, sizeof(*comp));
+
+ cs35l41_remove_controls(cs35l41);
+ cancel_work_sync(&cs35l41->fw_load_work);
+ cs35l41->codec = NULL;
}
}
@@ -2030,6 +2062,7 @@ void cs35l41_hda_remove(struct device *dev)
struct cs35l41_hda *cs35l41 = dev_get_drvdata(dev);
component_del(cs35l41->dev, &cs35l41_hda_comp_ops);
+ cancel_work_sync(&cs35l41->fw_load_work);
pm_runtime_get_sync(cs35l41->dev);
pm_runtime_dont_use_autosuspend(cs35l41->dev);
diff --git a/sound/pci/hda/cs35l41_hda.h b/sound/pci/hda/cs35l41_hda.h
index c730b3351589..65aed476fd80 100644
--- a/sound/pci/hda/cs35l41_hda.h
+++ b/sound/pci/hda/cs35l41_hda.h
@@ -56,6 +56,8 @@ enum control_bus {
SPI
};
+struct snd_kcontrol;
+
struct cs35l41_hda {
struct device *dev;
struct regmap *regmap;
@@ -74,6 +76,9 @@ struct cs35l41_hda {
int speaker_id;
struct mutex fw_mutex;
struct work_struct fw_load_work;
+ struct snd_kcontrol *fw_type_ctl;
+ struct snd_kcontrol *fw_load_ctl;
+ struct snd_kcontrol *mute_override_ctl;
struct regmap_irq_chip_data *irq_data;
bool firmware_running;
diff --git a/sound/soc/amd/acp/acp-sdw-sof-mach.c b/sound/soc/amd/acp/acp-sdw-sof-mach.c
index 876f0b7fcd3d..e169b33f6571 100644
--- a/sound/soc/amd/acp/acp-sdw-sof-mach.c
+++ b/sound/soc/amd/acp/acp-sdw-sof-mach.c
@@ -260,13 +260,14 @@ static int create_sdw_dailink(struct snd_soc_card *card,
static int create_sdw_dailinks(struct snd_soc_card *card,
struct snd_soc_dai_link **dai_links, int *be_id,
- struct asoc_sdw_dailink *sof_dais,
+ struct asoc_sdw_dailink *sof_dais, int num_dais,
struct snd_soc_codec_conf **codec_conf)
{
+ int i;
int ret;
/* generate DAI links by each sdw link */
- while (sof_dais->initialised) {
+ for (i = 0; i < num_dais && sof_dais->initialised; i++) {
int current_be_id = 0;
ret = create_sdw_dailink(card, sof_dais, dai_links,
@@ -373,7 +374,7 @@ static int sof_card_dai_links_create(struct snd_soc_card *card)
/* SDW */
if (sdw_be_num) {
ret = create_sdw_dailinks(card, &dai_links, &be_id,
- sof_dais, &codec_conf);
+ sof_dais, num_ends, &codec_conf);
if (ret)
goto err_end;
}
diff --git a/sound/soc/codecs/adau1372.c b/sound/soc/codecs/adau1372.c
index 25f123110b5b..7dc0b5c79e39 100644
--- a/sound/soc/codecs/adau1372.c
+++ b/sound/soc/codecs/adau1372.c
@@ -812,6 +812,11 @@ static int adau1372_set_power(struct adau1372 *adau1372, bool enable)
if (adau1372->use_pll) {
ret = adau1372_enable_pll(adau1372);
if (ret) {
+ if (!adau1372->pd_gpio)
+ regmap_update_bits(adau1372->regmap,
+ ADAU1372_REG_CLK_CTRL,
+ ADAU1372_CLK_CTRL_PLL_EN,
+ 0);
regcache_cache_only(adau1372->regmap, true);
if (adau1372->pd_gpio)
gpiod_set_value(adau1372->pd_gpio, 1);
diff --git a/sound/soc/codecs/aw88261.c b/sound/soc/codecs/aw88261.c
index 38f362b022aa..999461885a62 100644
--- a/sound/soc/codecs/aw88261.c
+++ b/sound/soc/codecs/aw88261.c
@@ -283,22 +283,22 @@ static void aw88261_reg_force_set(struct aw88261 *aw88261)
if (aw88261->frcset_en == AW88261_FRCSET_ENABLE) {
/* set FORCE_PWM */
regmap_update_bits(aw88261->regmap, AW88261_BSTCTRL3_REG,
- AW88261_FORCE_PWM_MASK, AW88261_FORCE_PWM_FORCEMINUS_PWM_VALUE);
+ ~AW88261_FORCE_PWM_MASK, AW88261_FORCE_PWM_FORCEMINUS_PWM_VALUE);
/* set BOOST_OS_WIDTH */
regmap_update_bits(aw88261->regmap, AW88261_BSTCTRL5_REG,
- AW88261_BST_OS_WIDTH_MASK, AW88261_BST_OS_WIDTH_50NS_VALUE);
+ ~AW88261_BST_OS_WIDTH_MASK, AW88261_BST_OS_WIDTH_50NS_VALUE);
/* set BURST_LOOPR */
regmap_update_bits(aw88261->regmap, AW88261_BSTCTRL6_REG,
- AW88261_BST_LOOPR_MASK, AW88261_BST_LOOPR_340K_VALUE);
+ ~AW88261_BST_LOOPR_MASK, AW88261_BST_LOOPR_340K_VALUE);
/* set RSQN_DLY */
regmap_update_bits(aw88261->regmap, AW88261_BSTCTRL7_REG,
- AW88261_RSQN_DLY_MASK, AW88261_RSQN_DLY_35NS_VALUE);
+ ~AW88261_RSQN_DLY_MASK, AW88261_RSQN_DLY_35NS_VALUE);
/* set BURST_SSMODE */
regmap_update_bits(aw88261->regmap, AW88261_BSTCTRL8_REG,
- AW88261_BURST_SSMODE_MASK, AW88261_BURST_SSMODE_FAST_VALUE);
+ ~AW88261_BURST_SSMODE_MASK, AW88261_BURST_SSMODE_FAST_VALUE);
/* set BST_BURST */
regmap_update_bits(aw88261->regmap, AW88261_BSTCTRL9_REG,
- AW88261_BST_BURST_MASK, AW88261_BST_BURST_30MA_VALUE);
+ ~AW88261_BST_BURST_MASK, AW88261_BST_BURST_30MA_VALUE);
} else {
dev_dbg(aw88261->aw_pa->dev, "needn't set reg value");
}
diff --git a/sound/soc/codecs/cs35l56-shared.c b/sound/soc/codecs/cs35l56-shared.c
index 9007484b31c7..6a49e01ef098 100644
--- a/sound/soc/codecs/cs35l56-shared.c
+++ b/sound/soc/codecs/cs35l56-shared.c
@@ -363,6 +363,7 @@ static void cs35l56_spi_system_reset(struct cs35l56_base *cs35l56_base)
* The regmap must remain in cache-only until the chip has
* booted, so use a bypassed read.
*/
+ val = 0;
ret = read_poll_timeout(regmap_read_bypassed, read_ret,
(val > 0) && (val < 0xffffffff),
CS35L56_HALO_STATE_POLL_US,
diff --git a/sound/soc/codecs/cs35l56.c b/sound/soc/codecs/cs35l56.c
index 757ade6373ed..c78f7f95f127 100644
--- a/sound/soc/codecs/cs35l56.c
+++ b/sound/soc/codecs/cs35l56.c
@@ -950,6 +950,7 @@ static int __maybe_unused cs35l56_runtime_resume_i2c_spi(struct device *dev)
int cs35l56_system_suspend(struct device *dev)
{
struct cs35l56_private *cs35l56 = dev_get_drvdata(dev);
+ int ret;
dev_dbg(dev, "system_suspend\n");
@@ -965,7 +966,11 @@ int cs35l56_system_suspend(struct device *dev)
if (cs35l56->base.irq)
disable_irq(cs35l56->base.irq);
- return pm_runtime_force_suspend(dev);
+ ret = pm_runtime_force_suspend(dev);
+ if ((ret < 0) && cs35l56->base.irq)
+ enable_irq(cs35l56->base.irq);
+
+ return ret;
}
EXPORT_SYMBOL_GPL(cs35l56_system_suspend);
@@ -1330,11 +1335,14 @@ int cs35l56_common_probe(struct cs35l56_private *cs35l56)
cs35l56_dai, ARRAY_SIZE(cs35l56_dai));
if (ret < 0) {
dev_err_probe(cs35l56->base.dev, ret, "Register codec failed\n");
- goto err;
+ goto err_remove_wm_adsp;
}
return 0;
+err_remove_wm_adsp:
+ wm_adsp2_remove(&cs35l56->dsp);
+
err:
gpiod_set_value_cansleep(cs35l56->base.reset_gpio, 0);
regulator_bulk_disable(ARRAY_SIZE(cs35l56->supplies), cs35l56->supplies);
@@ -1438,6 +1446,8 @@ void cs35l56_remove(struct cs35l56_private *cs35l56)
flush_workqueue(cs35l56->dsp_wq);
destroy_workqueue(cs35l56->dsp_wq);
+ wm_adsp2_remove(&cs35l56->dsp);
+
pm_runtime_dont_use_autosuspend(cs35l56->base.dev);
pm_runtime_suspend(cs35l56->base.dev);
pm_runtime_disable(cs35l56->base.dev);
diff --git a/sound/soc/codecs/hdac_hdmi.c b/sound/soc/codecs/hdac_hdmi.c
index 33c7ba842eee..183c5a50a4e3 100644
--- a/sound/soc/codecs/hdac_hdmi.c
+++ b/sound/soc/codecs/hdac_hdmi.c
@@ -909,12 +909,14 @@ static int hdac_hdmi_set_pin_port_mux(struct snd_kcontrol *kcontrol,
struct hdac_device *hdev = dev_to_hdac_dev(dapm->dev);
struct hdac_hdmi_priv *hdmi = hdev_to_hdmi_priv(hdev);
struct hdac_hdmi_pcm *pcm;
- const char *cvt_name = e->texts[ucontrol->value.enumerated.item[0]];
+ const char *cvt_name;
ret = snd_soc_dapm_put_enum_double(kcontrol, ucontrol);
if (ret < 0)
return ret;
+ cvt_name = e->texts[ucontrol->value.enumerated.item[0]];
+
if (port == NULL)
return -EINVAL;
diff --git a/sound/soc/codecs/lpass-va-macro.c b/sound/soc/codecs/lpass-va-macro.c
index d44b5b0a80a1..8d28e3721ca3 100644
--- a/sound/soc/codecs/lpass-va-macro.c
+++ b/sound/soc/codecs/lpass-va-macro.c
@@ -237,6 +237,11 @@ static const struct va_macro_data sm8250_va_data = {
.version = LPASS_CODEC_VERSION_1_0,
};
+static const struct va_macro_data sc7280_va_data = {
+ .has_swr_master = false,
+ .has_npl_clk = false,
+};
+
static const struct va_macro_data sm8450_va_data = {
.has_swr_master = true,
.has_npl_clk = true,
@@ -1721,7 +1726,8 @@ static const struct dev_pm_ops va_macro_pm_ops = {
};
static const struct of_device_id va_macro_dt_match[] = {
- { .compatible = "qcom,sc7280-lpass-va-macro", .data = &sm8250_va_data },
+ { .compatible = "qcom,sc7280-lpass-va-macro", .data = &sc7280_va_data },
+ { .compatible = "qcom,sm6115-lpass-va-macro", .data = &sm8450_va_data },
{ .compatible = "qcom,sm8250-lpass-va-macro", .data = &sm8250_va_data },
{ .compatible = "qcom,sm8450-lpass-va-macro", .data = &sm8450_va_data },
{ .compatible = "qcom,sm8550-lpass-va-macro", .data = &sm8550_va_data },
diff --git a/sound/soc/codecs/tlv320aic3x.c b/sound/soc/codecs/tlv320aic3x.c
index 591f6c9f9d3a..05eff1ed4e96 100644
--- a/sound/soc/codecs/tlv320aic3x.c
+++ b/sound/soc/codecs/tlv320aic3x.c
@@ -1049,11 +1049,13 @@ static int aic3x_hw_params(struct snd_pcm_substream *substream,
struct snd_pcm_hw_params *params,
struct snd_soc_dai *dai)
{
+ static const u8 dual_rate_q[] = {4, 8, 9, 12, 16};
struct snd_soc_component *component = dai->component;
struct aic3x_priv *aic3x = snd_soc_component_get_drvdata(component);
int codec_clk = 0, bypass_pll = 0, fsref, last_clk = 0;
u8 data, j, r, p, pll_q, pll_p = 1, pll_r = 1, pll_j = 1;
u16 d, pll_d = 1;
+ bool dual_rate;
int clk;
int width = aic3x->slot_width;
@@ -1079,14 +1081,25 @@ static int aic3x_hw_params(struct snd_pcm_substream *substream,
/* Fsref can be 44100 or 48000 */
fsref = (params_rate(params) % 11025 == 0) ? 44100 : 48000;
+ dual_rate = params_rate(params) >= 64000;
/* Try to find a value for Q which allows us to bypass the PLL and
* generate CODEC_CLK directly. */
- for (pll_q = 2; pll_q < 18; pll_q++)
- if (aic3x->sysclk / (128 * pll_q) == fsref) {
- bypass_pll = 1;
- break;
+ if (dual_rate) {
+ for (int i = 0; i < ARRAY_SIZE(dual_rate_q); i++) {
+ pll_q = dual_rate_q[i];
+ if (aic3x->sysclk / (128 * pll_q) == fsref) {
+ bypass_pll = 1;
+ break;
+ }
}
+ } else {
+ for (pll_q = 2; pll_q < 18; pll_q++)
+ if (aic3x->sysclk / (128 * pll_q) == fsref) {
+ bypass_pll = 1;
+ break;
+ }
+ }
if (bypass_pll) {
pll_q &= 0xf;
@@ -1106,13 +1119,13 @@ static int aic3x_hw_params(struct snd_pcm_substream *substream,
* right DAC to right channel input */
data = (LDAC2LCH | RDAC2RCH);
data |= (fsref == 44100) ? FSREF_44100 : FSREF_48000;
- if (params_rate(params) >= 64000)
+ if (dual_rate)
data |= DUAL_RATE_MODE;
snd_soc_component_write(component, AIC3X_CODEC_DATAPATH_REG, data);
/* codec sample rate select */
data = (fsref * 20) / params_rate(params);
- if (params_rate(params) < 64000)
+ if (!dual_rate)
data /= 2;
data /= 5;
data -= 2;
diff --git a/sound/soc/fsl/fsl_asrc_dma.c b/sound/soc/fsl/fsl_asrc_dma.c
index 1bba48318e2d..c96369da6aec 100644
--- a/sound/soc/fsl/fsl_asrc_dma.c
+++ b/sound/soc/fsl/fsl_asrc_dma.c
@@ -288,6 +288,26 @@ static int fsl_asrc_dma_hw_params(struct snd_soc_component *component,
config_be.dst_addr_width = buswidth;
config_be.dst_maxburst = dma_params_be->maxburst;
+ /*
+ * For eDMA, the back-end may report a maxburst size that is not evenly
+ * divisible by the channel count. This causes the DMA transfer length
+ * to misalign with the FIFO boundary, resulting in wrong data and
+ * audible noise. Align maxburst to the nearest valid boundary:
+ * - If maxburst >= channel count, override to the channel count so
+ * each transfer equals exactly one audio frame.
+ * - If maxburst < channel count, override to 1 to avoid partial-frame
+ * transfers.
+ */
+ if (asrc->use_edma && (dma_params_be->maxburst % params_channels(params))) {
+ if (dma_params_be->maxburst >= params_channels(params)) {
+ config_be.src_maxburst = params_channels(params);
+ config_be.dst_maxburst = params_channels(params);
+ } else {
+ config_be.src_maxburst = 1;
+ config_be.dst_maxburst = 1;
+ }
+ }
+
memset(&audio_config, 0, sizeof(audio_config));
config_be.peripheral_config = &audio_config;
config_be.peripheral_size = sizeof(audio_config);
diff --git a/sound/soc/fsl/fsl_audmix.c b/sound/soc/fsl/fsl_audmix.c
index 7981d598ba13..d9b0bd61755d 100644
--- a/sound/soc/fsl/fsl_audmix.c
+++ b/sound/soc/fsl/fsl_audmix.c
@@ -117,6 +117,9 @@ static int fsl_audmix_put_mix_clk_src(struct snd_kcontrol *kcontrol,
unsigned int *item = ucontrol->value.enumerated.item;
unsigned int reg_val, val, mix_clk;
+ if (item[0] >= e->items)
+ return -EINVAL;
+
/* Get current state */
reg_val = snd_soc_component_read(comp, FSL_AUDMIX_CTR);
mix_clk = ((reg_val & FSL_AUDMIX_CTR_MIXCLK_MASK)
@@ -157,6 +160,9 @@ static int fsl_audmix_put_out_src(struct snd_kcontrol *kcontrol,
unsigned int reg_val, val, mask = 0, ctr = 0;
int ret;
+ if (item[0] >= e->items)
+ return -EINVAL;
+
/* Get current state */
reg_val = snd_soc_component_read(comp, FSL_AUDMIX_CTR);
diff --git a/sound/soc/mediatek/mt8183/mt8183-afe-pcm.c b/sound/soc/mediatek/mt8183/mt8183-afe-pcm.c
index 501ea9d92aea..3e14849ebb0e 100644
--- a/sound/soc/mediatek/mt8183/mt8183-afe-pcm.c
+++ b/sound/soc/mediatek/mt8183/mt8183-afe-pcm.c
@@ -1073,6 +1073,11 @@ static const dai_register_cb dai_register_cbs[] = {
mt8183_dai_memif_register,
};
+static void mt8183_afe_release_reserved_mem(void *data)
+{
+ of_reserved_mem_device_release(data);
+}
+
static int mt8183_afe_pcm_dev_probe(struct platform_device *pdev)
{
struct mtk_base_afe *afe;
@@ -1099,6 +1104,12 @@ static int mt8183_afe_pcm_dev_probe(struct platform_device *pdev)
if (ret) {
dev_info(dev, "no reserved memory found, pre-allocating buffers instead\n");
afe->preallocate_buffers = true;
+ } else {
+ ret = devm_add_action_or_reset(dev,
+ mt8183_afe_release_reserved_mem,
+ dev);
+ if (ret)
+ return ret;
}
/* initial audio related clock */
diff --git a/sound/soc/mediatek/mt8192/mt8192-afe-pcm.c b/sound/soc/mediatek/mt8192/mt8192-afe-pcm.c
index 69ed34495d0f..362865227752 100644
--- a/sound/soc/mediatek/mt8192/mt8192-afe-pcm.c
+++ b/sound/soc/mediatek/mt8192/mt8192-afe-pcm.c
@@ -2155,6 +2155,11 @@ static const dai_register_cb dai_register_cbs[] = {
mt8192_dai_memif_register,
};
+static void mt8192_afe_release_reserved_mem(void *data)
+{
+ of_reserved_mem_device_release(data);
+}
+
static int mt8192_afe_pcm_dev_probe(struct platform_device *pdev)
{
struct mtk_base_afe *afe;
@@ -2185,6 +2190,10 @@ static int mt8192_afe_pcm_dev_probe(struct platform_device *pdev)
if (ret) {
dev_info(dev, "no reserved memory found, pre-allocating buffers instead\n");
afe->preallocate_buffers = true;
+ } else {
+ ret = devm_add_action_or_reset(dev, mt8192_afe_release_reserved_mem, dev);
+ if (ret)
+ return ret;
}
/* init audio related clock */
diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c
index 7b447cb50d50..e2ad6dc2dc85 100644
--- a/sound/soc/qcom/qdsp6/q6apm.c
+++ b/sound/soc/qcom/qdsp6/q6apm.c
@@ -533,6 +533,10 @@ static int graph_callback(struct gpr_resp_pkt *data, void *priv, int op)
token = hdr->token & APM_WRITE_TOKEN_MASK;
done = data->payload;
+ if (!graph->rx_data.buf) {
+ mutex_unlock(&graph->lock);
+ break;
+ }
phys = graph->rx_data.buf[token].phys;
mutex_unlock(&graph->lock);
/* token numbering starts at 0 */
@@ -567,6 +571,10 @@ static int graph_callback(struct gpr_resp_pkt *data, void *priv, int op)
client_event = APM_CLIENT_EVENT_DATA_READ_DONE;
mutex_lock(&graph->lock);
rd_done = data->payload;
+ if (!graph->tx_data.buf) {
+ mutex_unlock(&graph->lock);
+ break;
+ }
phys = graph->tx_data.buf[hdr->token].phys;
mutex_unlock(&graph->lock);
/* token numbering starts at 0 */
diff --git a/sound/soc/sh/rcar/rsnd.h b/sound/soc/sh/rcar/rsnd.h
index 3f1100b98cdd..286873effae2 100644
--- a/sound/soc/sh/rcar/rsnd.h
+++ b/sound/soc/sh/rcar/rsnd.h
@@ -624,7 +624,7 @@ struct rsnd_priv {
#define RSND_GEN2 (2 << 0)
#define RSND_GEN3 (3 << 0)
#define RSND_GEN4 (4 << 0)
-#define RSND_SOC_MASK (0xFF << 4)
+#define RSND_SOC_MASK (0xF << 4)
#define RSND_SOC_E (1 << 4) /* E1/E2/E3 */
/*
diff --git a/sound/soc/sof/ipc3-control.c b/sound/soc/sof/ipc3-control.c
index 2b1befad6d5c..d1697401b1da 100644
--- a/sound/soc/sof/ipc3-control.c
+++ b/sound/soc/sof/ipc3-control.c
@@ -315,10 +315,13 @@ static int sof_ipc3_bytes_get(struct snd_sof_control *scontrol,
}
/* be->max has been verified to be >= sizeof(struct sof_abi_hdr) */
- if (data->size > scontrol->max_size - sizeof(*data)) {
+ if (data->size > scontrol->max_size - sizeof(*cdata) -
+ sizeof(*data)) {
dev_err_ratelimited(scomp->dev,
"%u bytes of control data is invalid, max is %zu\n",
- data->size, scontrol->max_size - sizeof(*data));
+ data->size,
+ scontrol->max_size - sizeof(*cdata) -
+ sizeof(*data));
return -EINVAL;
}
@@ -336,6 +339,8 @@ static int sof_ipc3_bytes_put(struct snd_sof_control *scontrol,
struct sof_ipc_ctrl_data *cdata = scontrol->ipc_control_data;
struct snd_soc_component *scomp = scontrol->scomp;
struct sof_abi_hdr *data = cdata->data;
+ const struct sof_abi_hdr *new_hdr =
+ (const struct sof_abi_hdr *)ucontrol->value.bytes.data;
size_t size;
if (scontrol->max_size > sizeof(ucontrol->value.bytes.data)) {
@@ -344,14 +349,18 @@ static int sof_ipc3_bytes_put(struct snd_sof_control *scontrol,
return -EINVAL;
}
- /* scontrol->max_size has been verified to be >= sizeof(struct sof_abi_hdr) */
- if (data->size > scontrol->max_size - sizeof(*data)) {
- dev_err_ratelimited(scomp->dev, "data size too big %u bytes max is %zu\n",
- data->size, scontrol->max_size - sizeof(*data));
+ /* Validate the new data's size, not the old one */
+ if (new_hdr->size > scontrol->max_size - sizeof(*cdata) -
+ sizeof(*new_hdr)) {
+ dev_err_ratelimited(scomp->dev,
+ "data size too big %u bytes max is %zu\n",
+ new_hdr->size,
+ scontrol->max_size - sizeof(*cdata) -
+ sizeof(*new_hdr));
return -EINVAL;
}
- size = data->size + sizeof(*data);
+ size = new_hdr->size + sizeof(*new_hdr);
/* copy from kcontrol */
memcpy(data, ucontrol->value.bytes.data, size);
@@ -389,9 +398,17 @@ static int sof_ipc3_bytes_ext_put(struct snd_sof_control *scontrol,
}
/* be->max is coming from topology */
- if (header.length > scontrol->max_size) {
- dev_err_ratelimited(scomp->dev, "Bytes data size %d exceeds max %zu\n",
- header.length, scontrol->max_size);
+ if (header.length > scontrol->max_size - sizeof(*cdata)) {
+ dev_err_ratelimited(scomp->dev, "Bytes data size %u exceeds max %zu\n",
+ header.length, scontrol->max_size - sizeof(*cdata));
+ return -EINVAL;
+ }
+
+ /* Ensure the data is large enough to contain the ABI header */
+ if (header.length < sizeof(struct sof_abi_hdr)) {
+ dev_err_ratelimited(scomp->dev,
+ "Bytes data size %u less than ABI header %zu\n",
+ header.length, sizeof(struct sof_abi_hdr));
return -EINVAL;
}
@@ -427,7 +444,7 @@ static int sof_ipc3_bytes_ext_put(struct snd_sof_control *scontrol,
}
/* be->max has been verified to be >= sizeof(struct sof_abi_hdr) */
- if (cdata->data->size > scontrol->max_size - sizeof(struct sof_abi_hdr)) {
+ if (cdata->data->size > scontrol->max_size - sizeof(*cdata) - sizeof(struct sof_abi_hdr)) {
dev_err_ratelimited(scomp->dev, "Mismatch in ABI data size (truncated?)\n");
goto err_restore;
}
@@ -443,7 +460,7 @@ static int sof_ipc3_bytes_ext_put(struct snd_sof_control *scontrol,
err_restore:
/* If we have an issue, we restore the old, valid bytes control data */
if (scontrol->old_ipc_control_data) {
- memcpy(cdata->data, scontrol->old_ipc_control_data, scontrol->max_size);
+ memcpy(cdata, scontrol->old_ipc_control_data, scontrol->max_size);
kfree(scontrol->old_ipc_control_data);
scontrol->old_ipc_control_data = NULL;
}
@@ -482,10 +499,13 @@ static int _sof_ipc3_bytes_ext_get(struct snd_sof_control *scontrol,
}
/* check data size doesn't exceed max coming from topology */
- if (cdata->data->size > scontrol->max_size - sizeof(struct sof_abi_hdr)) {
- dev_err_ratelimited(scomp->dev, "User data size %d exceeds max size %zu\n",
+ if (cdata->data->size > scontrol->max_size - sizeof(*cdata) -
+ sizeof(struct sof_abi_hdr)) {
+ dev_err_ratelimited(scomp->dev,
+ "User data size %u exceeds max size %zu\n",
cdata->data->size,
- scontrol->max_size - sizeof(struct sof_abi_hdr));
+ scontrol->max_size - sizeof(*cdata) -
+ sizeof(struct sof_abi_hdr));
return -EINVAL;
}
@@ -535,6 +555,15 @@ static void snd_sof_update_control(struct snd_sof_control *scontrol,
return;
}
+ /* Verify the size fits within the allocation */
+ if (cdata->num_elems > scontrol->max_size - sizeof(*local_cdata) -
+ sizeof(*local_cdata->data)) {
+ dev_err(scomp->dev,
+ "cdata binary size %u exceeds buffer\n",
+ cdata->num_elems);
+ return;
+ }
+
/* copy the new binary data */
memcpy(local_cdata->data, cdata->data, cdata->num_elems);
} else if (cdata->num_elems != scontrol->num_channels) {
@@ -626,16 +655,28 @@ static void sof_ipc3_control_update(struct snd_sof_dev *sdev, void *ipc_control_
return;
}
- expected_size = sizeof(struct sof_ipc_ctrl_data);
switch (cdata->type) {
case SOF_CTRL_TYPE_VALUE_CHAN_GET:
case SOF_CTRL_TYPE_VALUE_CHAN_SET:
- expected_size += cdata->num_elems *
- sizeof(struct sof_ipc_ctrl_value_chan);
+ if (check_mul_overflow((size_t)cdata->num_elems,
+ sizeof(struct sof_ipc_ctrl_value_chan),
+ &expected_size))
+ return;
+ if (check_add_overflow(expected_size,
+ sizeof(struct sof_ipc_ctrl_data),
+ &expected_size))
+ return;
break;
case SOF_CTRL_TYPE_DATA_GET:
case SOF_CTRL_TYPE_DATA_SET:
- expected_size += cdata->num_elems + sizeof(struct sof_abi_hdr);
+ if (check_add_overflow((size_t)cdata->num_elems,
+ sizeof(struct sof_abi_hdr),
+ &expected_size))
+ return;
+ if (check_add_overflow(expected_size,
+ sizeof(struct sof_ipc_ctrl_data),
+ &expected_size))
+ return;
break;
default:
return;
diff --git a/sound/soc/sof/ipc4-control.c b/sound/soc/sof/ipc4-control.c
index 453ed1643b89..e50bc0dad18f 100644
--- a/sound/soc/sof/ipc4-control.c
+++ b/sound/soc/sof/ipc4-control.c
@@ -455,6 +455,8 @@ static int sof_ipc4_bytes_put(struct snd_sof_control *scontrol,
struct snd_soc_component *scomp = scontrol->scomp;
struct snd_sof_dev *sdev = snd_soc_component_get_drvdata(scomp);
struct sof_abi_hdr *data = cdata->data;
+ const struct sof_abi_hdr *new_hdr =
+ (const struct sof_abi_hdr *)ucontrol->value.bytes.data;
size_t size;
int ret;
@@ -465,15 +467,16 @@ static int sof_ipc4_bytes_put(struct snd_sof_control *scontrol,
return -EINVAL;
}
- /* scontrol->max_size has been verified to be >= sizeof(struct sof_abi_hdr) */
- if (data->size > scontrol->max_size - sizeof(*data)) {
+ /* Validate the new data's size, not the old one */
+ if (new_hdr->size > scontrol->max_size - sizeof(*new_hdr)) {
dev_err_ratelimited(scomp->dev,
"data size too big %u bytes max is %zu\n",
- data->size, scontrol->max_size - sizeof(*data));
+ new_hdr->size,
+ scontrol->max_size - sizeof(*new_hdr));
return -EINVAL;
}
- size = data->size + sizeof(*data);
+ size = new_hdr->size + sizeof(*new_hdr);
/* copy from kcontrol */
memcpy(data, ucontrol->value.bytes.data, size);
diff --git a/sound/soc/sof/topology.c b/sound/soc/sof/topology.c
index cdf8e56fd8a0..e6831d5aa15e 100644
--- a/sound/soc/sof/topology.c
+++ b/sound/soc/sof/topology.c
@@ -721,10 +721,13 @@ static int sof_parse_token_sets(struct snd_soc_component *scomp,
int ret;
while (array_size > 0 && total < count * token_instance_num) {
+ if (array_size < (int)sizeof(*array))
+ return -EINVAL;
+
asize = le32_to_cpu(array->size);
/* validate asize */
- if (asize < sizeof(*array)) {
+ if (asize < (int)sizeof(*array)) {
dev_err(scomp->dev, "error: invalid array size 0x%x\n",
asize);
return -EINVAL;
diff --git a/sound/soc/tegra/tegra210_ahub.c b/sound/soc/tegra/tegra210_ahub.c
index 51043e556b3e..65cb699d82e8 100644
--- a/sound/soc/tegra/tegra210_ahub.c
+++ b/sound/soc/tegra/tegra210_ahub.c
@@ -60,13 +60,15 @@ static int tegra_ahub_put_value_enum(struct snd_kcontrol *kctl,
struct soc_enum *e = (struct soc_enum *)kctl->private_value;
struct snd_soc_dapm_update update[TEGRA_XBAR_UPDATE_MAX_REG] = { };
unsigned int *item = uctl->value.enumerated.item;
- unsigned int value = e->values[item[0]];
+ unsigned int value;
unsigned int i, bit_pos, reg_idx = 0, reg_val = 0;
int change = 0;
if (item[0] >= e->items)
return -EINVAL;
+ value = e->values[item[0]];
+
if (value) {
/* Get the register index and value to set */
reg_idx = (value - 1) / (8 * cmpnt->val_bytes);
diff --git a/sound/usb/midi2.c b/sound/usb/midi2.c
index 6f04aa7f685b..95c21c9bf906 100644
--- a/sound/usb/midi2.c
+++ b/sound/usb/midi2.c
@@ -478,6 +478,11 @@ static int create_midi2_endpoint(struct snd_usb_midi2_interface *umidi,
static void free_midi2_endpoint(struct snd_usb_midi2_endpoint *ep)
{
list_del(&ep->list);
+ if (!ep->disconnected) {
+ ep->disconnected = 1;
+ kill_midi_urbs(ep, false);
+ drain_urb_queue(ep);
+ }
free_midi_urbs(ep);
kfree(ep);
}
diff --git a/sound/usb/mixer_scarlett2.c b/sound/usb/mixer_scarlett2.c
index c46c4d41ca0f..28ea0dd5726a 100644
--- a/sound/usb/mixer_scarlett2.c
+++ b/sound/usb/mixer_scarlett2.c
@@ -588,6 +588,20 @@ struct scarlett2_config_set {
const struct scarlett2_config items[SCARLETT2_CONFIG_COUNT];
};
+/* Map firmware versions to config sets per-device.
+ *
+ * Each device lists one or more entries, sorted in ascending order of
+ * from_firmware_version. At probe time the running firmware version
+ * is looked up against this list and the last entry whose
+ * from_firmware_version is <= the running version is selected.
+ *
+ * The list is terminated by a sentinel entry with config_set == NULL.
+ */
+struct scarlett2_config_set_entry {
+ u16 from_firmware_version;
+ const struct scarlett2_config_set *config_set;
+};
+
/* Input gain TLV dB ranges */
static const DECLARE_TLV_DB_MINMAX(
@@ -896,6 +910,63 @@ static const struct scarlett2_config_set scarlett2_config_set_gen4_2i2 = {
}
};
+/* 2i2 Gen 4, firmware version 2417 and above
+ *
+ * Firmware 2417 shifted DIRECT_MONITOR_GAIN by 4 bytes; all other
+ * offsets are unchanged from scarlett2_config_set_gen4_2i2.
+ */
+static const struct scarlett2_config_set scarlett2_config_set_gen4_2i2_2417 = {
+ .notifications = scarlett4_2i2_notifications,
+ .param_buf_addr = 0xfc,
+ .input_gain_tlv = db_scale_gen4_gain,
+ .autogain_status_texts = scarlett2_autogain_status_gen4,
+ .items = {
+ [SCARLETT2_CONFIG_MSD_SWITCH] = {
+ .offset = 0x49, .size = 8, .activate = 4 },
+
+ [SCARLETT2_CONFIG_DIRECT_MONITOR] = {
+ .offset = 0x14a, .size = 8, .activate = 16, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_AUTOGAIN_SWITCH] = {
+ .offset = 0x135, .size = 8, .activate = 10, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_AUTOGAIN_STATUS] = {
+ .offset = 0x137, .size = 8 },
+
+ [SCARLETT2_CONFIG_AG_MEAN_TARGET] = {
+ .offset = 0x131, .size = 8, .activate = 29, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_AG_PEAK_TARGET] = {
+ .offset = 0x132, .size = 8, .activate = 30, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_PHANTOM_SWITCH] = {
+ .offset = 0x48, .size = 8, .activate = 11, .pbuf = 1,
+ .mute = 1 },
+
+ [SCARLETT2_CONFIG_INPUT_GAIN] = {
+ .offset = 0x4b, .size = 8, .activate = 12, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_LEVEL_SWITCH] = {
+ .offset = 0x3c, .size = 8, .activate = 13, .pbuf = 1,
+ .mute = 1 },
+
+ [SCARLETT2_CONFIG_SAFE_SWITCH] = {
+ .offset = 0x147, .size = 8, .activate = 14, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_AIR_SWITCH] = {
+ .offset = 0x3e, .size = 8, .activate = 15, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_INPUT_SELECT_SWITCH] = {
+ .offset = 0x14b, .size = 8, .activate = 17, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_INPUT_LINK_SWITCH] = {
+ .offset = 0x14e, .size = 8, .activate = 18, .pbuf = 1 },
+
+ [SCARLETT2_CONFIG_DIRECT_MONITOR_GAIN] = {
+ .offset = 0x2a4, .size = 16, .activate = 36 }
+ }
+};
+
/* 4i4 Gen 4 */
static const struct scarlett2_config_set scarlett2_config_set_gen4_4i4 = {
.notifications = scarlett4_4i4_notifications,
@@ -1073,8 +1144,8 @@ struct scarlett2_meter_entry {
};
struct scarlett2_device_info {
- /* which set of configuration parameters the device uses */
- const struct scarlett2_config_set *config_set;
+ /* which sets of configuration parameters the device uses */
+ const struct scarlett2_config_set_entry *config_sets;
/* minimum firmware version required */
u16 min_firmware_version;
@@ -1309,7 +1380,10 @@ struct scarlett2_data {
/*** Model-specific data ***/
static const struct scarlett2_device_info s6i6_gen2_info = {
- .config_set = &scarlett2_config_set_gen2a,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen2a },
+ { }
+ },
.level_input_count = 2,
.pad_input_count = 2,
@@ -1359,7 +1433,10 @@ static const struct scarlett2_device_info s6i6_gen2_info = {
};
static const struct scarlett2_device_info s18i8_gen2_info = {
- .config_set = &scarlett2_config_set_gen2a,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen2a },
+ { }
+ },
.level_input_count = 2,
.pad_input_count = 4,
@@ -1412,7 +1489,10 @@ static const struct scarlett2_device_info s18i8_gen2_info = {
};
static const struct scarlett2_device_info s18i20_gen2_info = {
- .config_set = &scarlett2_config_set_gen2b,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen2b },
+ { }
+ },
.line_out_descrs = {
"Monitor L",
@@ -1469,7 +1549,10 @@ static const struct scarlett2_device_info s18i20_gen2_info = {
};
static const struct scarlett2_device_info solo_gen3_info = {
- .config_set = &scarlett2_config_set_gen3a,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen3a },
+ { }
+ },
.level_input_count = 1,
.level_input_first = 1,
.air_input_count = 1,
@@ -1479,7 +1562,10 @@ static const struct scarlett2_device_info solo_gen3_info = {
};
static const struct scarlett2_device_info s2i2_gen3_info = {
- .config_set = &scarlett2_config_set_gen3a,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen3a },
+ { }
+ },
.level_input_count = 2,
.air_input_count = 2,
.phantom_count = 1,
@@ -1488,7 +1574,10 @@ static const struct scarlett2_device_info s2i2_gen3_info = {
};
static const struct scarlett2_device_info s4i4_gen3_info = {
- .config_set = &scarlett2_config_set_gen3b,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen3b },
+ { }
+ },
.level_input_count = 2,
.pad_input_count = 2,
.air_input_count = 2,
@@ -1537,7 +1626,10 @@ static const struct scarlett2_device_info s4i4_gen3_info = {
};
static const struct scarlett2_device_info s8i6_gen3_info = {
- .config_set = &scarlett2_config_set_gen3b,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen3b },
+ { }
+ },
.level_input_count = 2,
.pad_input_count = 2,
.air_input_count = 2,
@@ -1603,7 +1695,10 @@ static const char * const scarlett2_spdif_s18i8_gen3_texts[] = {
};
static const struct scarlett2_device_info s18i8_gen3_info = {
- .config_set = &scarlett2_config_set_gen3c,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen3c },
+ { }
+ },
.has_speaker_switching = 1,
.level_input_count = 2,
.pad_input_count = 4,
@@ -1695,7 +1790,10 @@ static const char * const scarlett2_spdif_s18i20_gen3_texts[] = {
};
static const struct scarlett2_device_info s18i20_gen3_info = {
- .config_set = &scarlett2_config_set_gen3c,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_gen3c },
+ { }
+ },
.has_speaker_switching = 1,
.has_talkback = 1,
.level_input_count = 2,
@@ -1769,7 +1867,10 @@ static const struct scarlett2_device_info s18i20_gen3_info = {
};
static const struct scarlett2_device_info vocaster_one_info = {
- .config_set = &scarlett2_config_set_vocaster,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 1769, &scarlett2_config_set_vocaster },
+ { }
+ },
.min_firmware_version = 1769,
.phantom_count = 1,
@@ -1811,7 +1912,10 @@ static const struct scarlett2_device_info vocaster_one_info = {
};
static const struct scarlett2_device_info vocaster_two_info = {
- .config_set = &scarlett2_config_set_vocaster,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 1769, &scarlett2_config_set_vocaster },
+ { }
+ },
.min_firmware_version = 1769,
.phantom_count = 2,
@@ -1854,7 +1958,10 @@ static const struct scarlett2_device_info vocaster_two_info = {
};
static const struct scarlett2_device_info solo_gen4_info = {
- .config_set = &scarlett2_config_set_gen4_solo,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 2115, &scarlett2_config_set_gen4_solo },
+ { }
+ },
.min_firmware_version = 2115,
.level_input_count = 1,
@@ -1908,7 +2015,11 @@ static const struct scarlett2_device_info solo_gen4_info = {
};
static const struct scarlett2_device_info s2i2_gen4_info = {
- .config_set = &scarlett2_config_set_gen4_2i2,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 2115, &scarlett2_config_set_gen4_2i2 },
+ { 2417, &scarlett2_config_set_gen4_2i2_2417 },
+ { }
+ },
.min_firmware_version = 2115,
.level_input_count = 2,
@@ -1962,7 +2073,10 @@ static const struct scarlett2_device_info s2i2_gen4_info = {
};
static const struct scarlett2_device_info s4i4_gen4_info = {
- .config_set = &scarlett2_config_set_gen4_4i4,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 2089, &scarlett2_config_set_gen4_4i4 },
+ { }
+ },
.min_firmware_version = 2089,
.level_input_count = 2,
@@ -2010,7 +2124,10 @@ static const struct scarlett2_device_info s4i4_gen4_info = {
};
static const struct scarlett2_device_info clarett_2pre_info = {
- .config_set = &scarlett2_config_set_clarett,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_clarett },
+ { }
+ },
.level_input_count = 2,
.air_input_count = 2,
@@ -2066,7 +2183,10 @@ static const char * const scarlett2_spdif_clarett_texts[] = {
};
static const struct scarlett2_device_info clarett_4pre_info = {
- .config_set = &scarlett2_config_set_clarett,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_clarett },
+ { }
+ },
.level_input_count = 2,
.air_input_count = 4,
@@ -2122,7 +2242,10 @@ static const struct scarlett2_device_info clarett_4pre_info = {
};
static const struct scarlett2_device_info clarett_8pre_info = {
- .config_set = &scarlett2_config_set_clarett,
+ .config_sets = (const struct scarlett2_config_set_entry[]) {
+ { 0, &scarlett2_config_set_clarett },
+ { }
+ },
.level_input_count = 2,
.air_input_count = 8,
@@ -8539,10 +8662,32 @@ static void scarlett2_private_suspend(struct usb_mixer_interface *mixer)
/*** Initialisation ***/
+/* Select the config_set matching the running firmware version.
+ *
+ * The device info's config_sets array is ordered by ascending
+ * from_firmware_version; pick the last entry whose version is <= the
+ * running firmware version. If the running firmware is older than the
+ * first entry's from_firmware_version (i.e. older than the driver's
+ * minimum supported version for this device), the first entry's
+ * config_set is selected anyway so firmware updates can still be done
+ * (requires only the ACK handler), but the usual mixer controls
+ * aren't created.
+ */
+static void scarlett2_resolve_config_set(struct scarlett2_data *private)
+{
+ const struct scarlett2_config_set_entry *entry =
+ private->info->config_sets;
+
+ private->config_set = entry->config_set;
+ for (entry++; entry->config_set; entry++)
+ if (entry->from_firmware_version <= private->firmware_version)
+ private->config_set = entry->config_set;
+}
+
static void scarlett2_count_io(struct scarlett2_data *private)
{
const struct scarlett2_device_info *info = private->info;
- const struct scarlett2_config_set *config_set = info->config_set;
+ const struct scarlett2_config_set *config_set = private->config_set;
const int (*port_count)[SCARLETT2_PORT_DIRNS] = info->port_count;
int port_type, srcs = 0, dsts = 0, i;
@@ -8640,9 +8785,14 @@ static int scarlett2_init_private(struct usb_mixer_interface *mixer,
mixer->private_suspend = scarlett2_private_suspend;
private->info = entry->info;
- private->config_set = entry->info->config_set;
+
+ /* Set config_set to the first entry's config_set so the
+ * notify handler has a valid pointer while USB init runs; it
+ * is re-resolved once the firmware version has been read.
+ */
+ private->config_set = entry->info->config_sets[0].config_set;
+
private->series_name = entry->series_name;
- scarlett2_count_io(private);
private->scarlett2_seq = 0;
private->mixer = mixer;
@@ -9046,6 +9196,13 @@ static int snd_scarlett2_controls_create(
if (err < 0)
return err;
+ /* Now that the firmware version is known, pick the matching
+ * config_set
+ */
+ scarlett2_resolve_config_set(private);
+
+ scarlett2_count_io(private);
+
/* Get the upgrade & settings flash segment numbers */
err = scarlett2_get_flash_segment_nums(mixer);
if (err < 0)
diff --git a/tools/bpf/bpftool/cgroup.c b/tools/bpf/bpftool/cgroup.c
index 4189c9d74fb0..f4aa711c7fb1 100644
--- a/tools/bpf/bpftool/cgroup.c
+++ b/tools/bpf/bpftool/cgroup.c
@@ -74,6 +74,13 @@ static unsigned int query_flags;
static struct btf *btf_vmlinux;
static __u32 btf_vmlinux_id;
+static void free_btf_vmlinux(void)
+{
+ btf__free(btf_vmlinux);
+ btf_vmlinux = NULL;
+ btf_vmlinux_id = 0;
+}
+
static enum bpf_attach_type parse_attach_type(const char *str)
{
const char *attach_type_str;
@@ -384,6 +391,8 @@ static int do_show(int argc, char **argv)
if (json_output)
jsonw_end_array(json_wtr);
+ free_btf_vmlinux();
+
exit_cgroup:
close(cgroup_fd);
exit:
@@ -433,7 +442,9 @@ static int do_show_tree_fn(const char *fpath, const struct stat *sb,
printf("%s\n", fpath);
}
- btf_vmlinux = libbpf_find_kernel_btf();
+ if (!btf_vmlinux)
+ btf_vmlinux = libbpf_find_kernel_btf();
+
for (i = 0; i < ARRAY_SIZE(cgroup_attach_types); i++)
show_bpf_progs(cgroup_fd, cgroup_attach_types[i], ftw->level);
@@ -536,6 +547,7 @@ static int do_show_tree(int argc, char **argv)
if (json_output)
jsonw_end_array(json_wtr);
+ free_btf_vmlinux();
free(cgroup_alloced);
return ret;
diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c
index 5a4d3240689e..a89cd4f16312 100644
--- a/tools/bpf/bpftool/gen.c
+++ b/tools/bpf/bpftool/gen.c
@@ -1347,7 +1347,7 @@ static int do_skeleton(int argc, char **argv)
continue;
if (use_loader)
- printf("t\tint %s_fd;\n", ident);
+ printf("\t\tint %s_fd;\n", ident);
else
printf("\t\tstruct bpf_link *%s;\n", ident);
}
diff --git a/tools/bpf/bpftool/net.c b/tools/bpf/bpftool/net.c
index 587403a19af3..89e2adcccf13 100644
--- a/tools/bpf/bpftool/net.c
+++ b/tools/bpf/bpftool/net.c
@@ -603,14 +603,14 @@ static int query_flow_dissector(struct bpf_attach_info *attach_info)
&attach_flags, prog_ids, &prog_cnt);
close(fd);
if (err) {
- if (errno == EINVAL) {
+ if (err == -EINVAL) {
/* Older kernel's don't support querying
* flow dissector programs.
*/
errno = 0;
return 0;
}
- p_err("can't query prog: %s", strerror(errno));
+ p_err("can't query prog: %s", strerror(-err));
return -1;
}
diff --git a/tools/include/linux/args.h b/tools/include/linux/args.h
new file mode 100644
index 000000000000..14b268f2389a
--- /dev/null
+++ b/tools/include/linux/args.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _LINUX_ARGS_H
+#define _LINUX_ARGS_H
+
+/*
+ * How do these macros work?
+ *
+ * In __COUNT_ARGS() _0 to _12 are just placeholders from the start
+ * in order to make sure _n is positioned over the correct number
+ * from 12 to 0 (depending on X, which is a variadic argument list).
+ * They serve no purpose other than occupying a position. Since each
+ * macro parameter must have a distinct identifier, those identifiers
+ * are as good as any.
+ *
+ * In COUNT_ARGS() we use actual integers, so __COUNT_ARGS() returns
+ * that as _n.
+ */
+
+/* This counts to 15. Any more, it will return 16th argument. */
+#define __COUNT_ARGS(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _n, X...) _n
+#define COUNT_ARGS(X...) __COUNT_ARGS(, ##X, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)
+
+/* Concatenate two parameters, but allow them to be expanded beforehand. */
+#ifndef __CONCAT
+#define __CONCAT(a, b) a ## b
+#endif
+#ifndef CONCATENATE
+#define CONCATENATE(a, b) __CONCAT(a, b)
+#endif
+
+#endif /* _LINUX_ARGS_H */
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 5a5cdb453935..c8fb6abfeca9 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -6592,6 +6592,7 @@ struct bpf_prog_info {
__u32 verified_insns;
__u32 attach_btf_obj_id;
__u32 attach_btf_id;
+ __u32 :32;
} __attribute__((aligned(8)));
struct bpf_map_info {
diff --git a/tools/include/uapi/linux/mount.h b/tools/include/uapi/linux/mount.h
new file mode 100644
index 000000000000..7fa67c2031a5
--- /dev/null
+++ b/tools/include/uapi/linux/mount.h
@@ -0,0 +1,235 @@
+#ifndef _UAPI_LINUX_MOUNT_H
+#define _UAPI_LINUX_MOUNT_H
+
+#include <linux/types.h>
+
+/*
+ * These are the fs-independent mount-flags: up to 32 flags are supported
+ *
+ * Usage of these is restricted within the kernel to core mount(2) code and
+ * callers of sys_mount() only. Filesystems should be using the SB_*
+ * equivalent instead.
+ */
+#define MS_RDONLY 1 /* Mount read-only */
+#define MS_NOSUID 2 /* Ignore suid and sgid bits */
+#define MS_NODEV 4 /* Disallow access to device special files */
+#define MS_NOEXEC 8 /* Disallow program execution */
+#define MS_SYNCHRONOUS 16 /* Writes are synced at once */
+#define MS_REMOUNT 32 /* Alter flags of a mounted FS */
+#define MS_MANDLOCK 64 /* Allow mandatory locks on an FS */
+#define MS_DIRSYNC 128 /* Directory modifications are synchronous */
+#define MS_NOSYMFOLLOW 256 /* Do not follow symlinks */
+#define MS_NOATIME 1024 /* Do not update access times. */
+#define MS_NODIRATIME 2048 /* Do not update directory access times */
+#define MS_BIND 4096
+#define MS_MOVE 8192
+#define MS_REC 16384
+#define MS_VERBOSE 32768 /* War is peace. Verbosity is silence.
+ MS_VERBOSE is deprecated. */
+#define MS_SILENT 32768
+#define MS_POSIXACL (1<<16) /* VFS does not apply the umask */
+#define MS_UNBINDABLE (1<<17) /* change to unbindable */
+#define MS_PRIVATE (1<<18) /* change to private */
+#define MS_SLAVE (1<<19) /* change to slave */
+#define MS_SHARED (1<<20) /* change to shared */
+#define MS_RELATIME (1<<21) /* Update atime relative to mtime/ctime. */
+#define MS_KERNMOUNT (1<<22) /* this is a kern_mount call */
+#define MS_I_VERSION (1<<23) /* Update inode I_version field */
+#define MS_STRICTATIME (1<<24) /* Always perform atime updates */
+#define MS_LAZYTIME (1<<25) /* Update the on-disk [acm]times lazily */
+
+/* These sb flags are internal to the kernel */
+#define MS_SUBMOUNT (1<<26)
+#define MS_NOREMOTELOCK (1<<27)
+#define MS_NOSEC (1<<28)
+#define MS_BORN (1<<29)
+#define MS_ACTIVE (1<<30)
+#define MS_NOUSER (1<<31)
+
+/*
+ * Superblock flags that can be altered by MS_REMOUNT
+ */
+#define MS_RMT_MASK (MS_RDONLY|MS_SYNCHRONOUS|MS_MANDLOCK|MS_I_VERSION|\
+ MS_LAZYTIME)
+
+/*
+ * Old magic mount flag and mask
+ */
+#define MS_MGC_VAL 0xC0ED0000
+#define MS_MGC_MSK 0xffff0000
+
+/*
+ * open_tree() flags.
+ */
+#define OPEN_TREE_CLONE 1 /* Clone the target tree and attach the clone */
+#define OPEN_TREE_CLOEXEC O_CLOEXEC /* Close the file on execve() */
+
+/*
+ * move_mount() flags.
+ */
+#define MOVE_MOUNT_F_SYMLINKS 0x00000001 /* Follow symlinks on from path */
+#define MOVE_MOUNT_F_AUTOMOUNTS 0x00000002 /* Follow automounts on from path */
+#define MOVE_MOUNT_F_EMPTY_PATH 0x00000004 /* Empty from path permitted */
+#define MOVE_MOUNT_T_SYMLINKS 0x00000010 /* Follow symlinks on to path */
+#define MOVE_MOUNT_T_AUTOMOUNTS 0x00000020 /* Follow automounts on to path */
+#define MOVE_MOUNT_T_EMPTY_PATH 0x00000040 /* Empty to path permitted */
+#define MOVE_MOUNT_SET_GROUP 0x00000100 /* Set sharing group instead */
+#define MOVE_MOUNT_BENEATH 0x00000200 /* Mount beneath top mount */
+#define MOVE_MOUNT__MASK 0x00000377
+
+/*
+ * fsopen() flags.
+ */
+#define FSOPEN_CLOEXEC 0x00000001
+
+/*
+ * fspick() flags.
+ */
+#define FSPICK_CLOEXEC 0x00000001
+#define FSPICK_SYMLINK_NOFOLLOW 0x00000002
+#define FSPICK_NO_AUTOMOUNT 0x00000004
+#define FSPICK_EMPTY_PATH 0x00000008
+
+/*
+ * The type of fsconfig() call made.
+ */
+enum fsconfig_command {
+ FSCONFIG_SET_FLAG = 0, /* Set parameter, supplying no value */
+ FSCONFIG_SET_STRING = 1, /* Set parameter, supplying a string value */
+ FSCONFIG_SET_BINARY = 2, /* Set parameter, supplying a binary blob value */
+ FSCONFIG_SET_PATH = 3, /* Set parameter, supplying an object by path */
+ FSCONFIG_SET_PATH_EMPTY = 4, /* Set parameter, supplying an object by (empty) path */
+ FSCONFIG_SET_FD = 5, /* Set parameter, supplying an object by fd */
+ FSCONFIG_CMD_CREATE = 6, /* Create new or reuse existing superblock */
+ FSCONFIG_CMD_RECONFIGURE = 7, /* Invoke superblock reconfiguration */
+ FSCONFIG_CMD_CREATE_EXCL = 8, /* Create new superblock, fail if reusing existing superblock */
+};
+
+/*
+ * fsmount() flags.
+ */
+#define FSMOUNT_CLOEXEC 0x00000001
+
+/*
+ * Mount attributes.
+ */
+#define MOUNT_ATTR_RDONLY 0x00000001 /* Mount read-only */
+#define MOUNT_ATTR_NOSUID 0x00000002 /* Ignore suid and sgid bits */
+#define MOUNT_ATTR_NODEV 0x00000004 /* Disallow access to device special files */
+#define MOUNT_ATTR_NOEXEC 0x00000008 /* Disallow program execution */
+#define MOUNT_ATTR__ATIME 0x00000070 /* Setting on how atime should be updated */
+#define MOUNT_ATTR_RELATIME 0x00000000 /* - Update atime relative to mtime/ctime. */
+#define MOUNT_ATTR_NOATIME 0x00000010 /* - Do not update access times. */
+#define MOUNT_ATTR_STRICTATIME 0x00000020 /* - Always perform atime updates */
+#define MOUNT_ATTR_NODIRATIME 0x00000080 /* Do not update directory access times */
+#define MOUNT_ATTR_IDMAP 0x00100000 /* Idmap mount to @userns_fd in struct mount_attr. */
+#define MOUNT_ATTR_NOSYMFOLLOW 0x00200000 /* Do not follow symlinks */
+
+/*
+ * mount_setattr()
+ */
+struct mount_attr {
+ __u64 attr_set;
+ __u64 attr_clr;
+ __u64 propagation;
+ __u64 userns_fd;
+};
+
+/* List of all mount_attr versions. */
+#define MOUNT_ATTR_SIZE_VER0 32 /* sizeof first published struct */
+
+
+/*
+ * Structure for getting mount/superblock/filesystem info with statmount(2).
+ *
+ * The interface is similar to statx(2): individual fields or groups can be
+ * selected with the @mask argument of statmount(). Kernel will set the @mask
+ * field according to the supported fields.
+ *
+ * If string fields are selected, then the caller needs to pass a buffer that
+ * has space after the fixed part of the structure. Nul terminated strings are
+ * copied there and offsets relative to @str are stored in the relevant fields.
+ * If the buffer is too small, then EOVERFLOW is returned. The actually used
+ * size is returned in @size.
+ */
+struct statmount {
+ __u32 size; /* Total size, including strings */
+ __u32 mnt_opts; /* [str] Options (comma separated, escaped) */
+ __u64 mask; /* What results were written */
+ __u32 sb_dev_major; /* Device ID */
+ __u32 sb_dev_minor;
+ __u64 sb_magic; /* ..._SUPER_MAGIC */
+ __u32 sb_flags; /* SB_{RDONLY,SYNCHRONOUS,DIRSYNC,LAZYTIME} */
+ __u32 fs_type; /* [str] Filesystem type */
+ __u64 mnt_id; /* Unique ID of mount */
+ __u64 mnt_parent_id; /* Unique ID of parent (for root == mnt_id) */
+ __u32 mnt_id_old; /* Reused IDs used in proc/.../mountinfo */
+ __u32 mnt_parent_id_old;
+ __u64 mnt_attr; /* MOUNT_ATTR_... */
+ __u64 mnt_propagation; /* MS_{SHARED,SLAVE,PRIVATE,UNBINDABLE} */
+ __u64 mnt_peer_group; /* ID of shared peer group */
+ __u64 mnt_master; /* Mount receives propagation from this ID */
+ __u64 propagate_from; /* Propagation from in current namespace */
+ __u32 mnt_root; /* [str] Root of mount relative to root of fs */
+ __u32 mnt_point; /* [str] Mountpoint relative to current root */
+ __u64 mnt_ns_id; /* ID of the mount namespace */
+ __u32 fs_subtype; /* [str] Subtype of fs_type (if any) */
+ __u32 sb_source; /* [str] Source string of the mount */
+ __u32 opt_num; /* Number of fs options */
+ __u32 opt_array; /* [str] Array of nul terminated fs options */
+ __u32 opt_sec_num; /* Number of security options */
+ __u32 opt_sec_array; /* [str] Array of nul terminated security options */
+ __u64 supported_mask; /* Mask flags that this kernel supports */
+ __u32 mnt_uidmap_num; /* Number of uid mappings */
+ __u32 mnt_uidmap; /* [str] Array of uid mappings (as seen from callers namespace) */
+ __u32 mnt_gidmap_num; /* Number of gid mappings */
+ __u32 mnt_gidmap; /* [str] Array of gid mappings (as seen from callers namespace) */
+ __u64 __spare2[43];
+ char str[]; /* Variable size part containing strings */
+};
+
+/*
+ * Structure for passing mount ID and miscellaneous parameters to statmount(2)
+ * and listmount(2).
+ *
+ * For statmount(2) @param represents the request mask.
+ * For listmount(2) @param represents the last listed mount id (or zero).
+ */
+struct mnt_id_req {
+ __u32 size;
+ __u32 spare;
+ __u64 mnt_id;
+ __u64 param;
+ __u64 mnt_ns_id;
+};
+
+/* List of all mnt_id_req versions. */
+#define MNT_ID_REQ_SIZE_VER0 24 /* sizeof first published struct */
+#define MNT_ID_REQ_SIZE_VER1 32 /* sizeof second published struct */
+
+/*
+ * @mask bits for statmount(2)
+ */
+#define STATMOUNT_SB_BASIC 0x00000001U /* Want/got sb_... */
+#define STATMOUNT_MNT_BASIC 0x00000002U /* Want/got mnt_... */
+#define STATMOUNT_PROPAGATE_FROM 0x00000004U /* Want/got propagate_from */
+#define STATMOUNT_MNT_ROOT 0x00000008U /* Want/got mnt_root */
+#define STATMOUNT_MNT_POINT 0x00000010U /* Want/got mnt_point */
+#define STATMOUNT_FS_TYPE 0x00000020U /* Want/got fs_type */
+#define STATMOUNT_MNT_NS_ID 0x00000040U /* Want/got mnt_ns_id */
+#define STATMOUNT_MNT_OPTS 0x00000080U /* Want/got mnt_opts */
+#define STATMOUNT_FS_SUBTYPE 0x00000100U /* Want/got fs_subtype */
+#define STATMOUNT_SB_SOURCE 0x00000200U /* Want/got sb_source */
+#define STATMOUNT_OPT_ARRAY 0x00000400U /* Want/got opt_... */
+#define STATMOUNT_OPT_SEC_ARRAY 0x00000800U /* Want/got opt_sec... */
+#define STATMOUNT_SUPPORTED_MASK 0x00001000U /* Want/got supported mask flags */
+#define STATMOUNT_MNT_UIDMAP 0x00002000U /* Want/got uidmap... */
+#define STATMOUNT_MNT_GIDMAP 0x00004000U /* Want/got gidmap... */
+
+/*
+ * Special @mnt_id values that can be passed to listmount
+ */
+#define LSMT_ROOT 0xffffffffffffffff /* root mount */
+#define LISTMOUNT_REVERSE (1 << 0) /* List later mounts first */
+
+#endif /* _UAPI_LINUX_MOUNT_H */
diff --git a/tools/include/uapi/linux/nsfs.h b/tools/include/uapi/linux/nsfs.h
new file mode 100644
index 000000000000..34127653fd00
--- /dev/null
+++ b/tools/include/uapi/linux/nsfs.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __LINUX_NSFS_H
+#define __LINUX_NSFS_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+#define NSIO 0xb7
+
+/* Returns a file descriptor that refers to an owning user namespace */
+#define NS_GET_USERNS _IO(NSIO, 0x1)
+/* Returns a file descriptor that refers to a parent namespace */
+#define NS_GET_PARENT _IO(NSIO, 0x2)
+/* Returns the type of namespace (CLONE_NEW* value) referred to by
+ file descriptor */
+#define NS_GET_NSTYPE _IO(NSIO, 0x3)
+/* Get owner UID (in the caller's user namespace) for a user namespace */
+#define NS_GET_OWNER_UID _IO(NSIO, 0x4)
+/* Get the id for a mount namespace */
+#define NS_GET_MNTNS_ID _IOR(NSIO, 0x5, __u64)
+/* Translate pid from target pid namespace into the caller's pid namespace. */
+#define NS_GET_PID_FROM_PIDNS _IOR(NSIO, 0x6, int)
+/* Return thread-group leader id of pid in the callers pid namespace. */
+#define NS_GET_TGID_FROM_PIDNS _IOR(NSIO, 0x7, int)
+/* Translate pid from caller's pid namespace into a target pid namespace. */
+#define NS_GET_PID_IN_PIDNS _IOR(NSIO, 0x8, int)
+/* Return thread-group leader id of pid in the target pid namespace. */
+#define NS_GET_TGID_IN_PIDNS _IOR(NSIO, 0x9, int)
+
+struct mnt_ns_info {
+ __u32 size;
+ __u32 nr_mounts;
+ __u64 mnt_ns_id;
+};
+
+#define MNT_NS_INFO_SIZE_VER0 16 /* size of first published struct */
+
+/* Get information about namespace. */
+#define NS_MNT_GET_INFO _IOR(NSIO, 10, struct mnt_ns_info)
+/* Get next namespace. */
+#define NS_MNT_GET_NEXT _IOR(NSIO, 11, struct mnt_ns_info)
+/* Get previous namespace. */
+#define NS_MNT_GET_PREV _IOR(NSIO, 12, struct mnt_ns_info)
+
+#endif /* __LINUX_NSFS_H */
diff --git a/tools/lib/api/fs/fs.c b/tools/lib/api/fs/fs.c
index 337fde770e45..a3f81fda3e84 100644
--- a/tools/lib/api/fs/fs.c
+++ b/tools/lib/api/fs/fs.c
@@ -261,8 +261,8 @@ static const char *mount_overload(struct fs *fs)
/* "PERF_" + name + "_ENVIRONMENT" + '\0' */
char upper_name[5 + name_len + 12 + 1];
- snprintf(upper_name, name_len, "PERF_%s_ENVIRONMENT", fs->name);
- mem_toupper(upper_name, name_len);
+ snprintf(upper_name, sizeof(upper_name), "PERF_%s_ENVIRONMENT", fs->name);
+ mem_toupper(upper_name, strlen(upper_name));
return getenv(upper_name) ?: *fs->mounts;
}
@@ -294,11 +294,14 @@ int filename__read_int(const char *filename, int *value)
{
char line[64];
int fd = open(filename, O_RDONLY), err = -1;
+ ssize_t n;
if (fd < 0)
return -1;
- if (read(fd, line, sizeof(line)) > 0) {
+ n = read(fd, line, sizeof(line) - 1);
+ if (n > 0) {
+ line[n] = '\0';
*value = atoi(line);
err = 0;
}
@@ -312,11 +315,14 @@ static int filename__read_ull_base(const char *filename,
{
char line[64];
int fd = open(filename, O_RDONLY), err = -1;
+ ssize_t n;
if (fd < 0)
return -1;
- if (read(fd, line, sizeof(line)) > 0) {
+ n = read(fd, line, sizeof(line) - 1);
+ if (n > 0) {
+ line[n] = '\0';
*value = strtoull(line, NULL, base);
if (*value != ULLONG_MAX)
err = 0;
@@ -370,12 +376,13 @@ int filename__write_int(const char *filename, int value)
{
int fd = open(filename, O_WRONLY), err = -1;
char buf[64];
+ int len;
if (fd < 0)
return err;
- sprintf(buf, "%d", value);
- if (write(fd, buf, sizeof(buf)) == sizeof(buf))
+ len = sprintf(buf, "%d", value);
+ if (write(fd, buf, len) == len)
err = 0;
close(fd);
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 7d496f0a9a30..9bbf7a3b87de 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -11626,7 +11626,7 @@ bpf_program__attach_kprobe_multi_opts(const struct bpf_program *prog,
static int attach_kprobe(const struct bpf_program *prog, long cookie, struct bpf_link **link)
{
DECLARE_LIBBPF_OPTS(bpf_kprobe_opts, opts);
- unsigned long offset = 0;
+ long offset = 0;
const char *func_name;
char *func;
int n;
@@ -11648,6 +11648,13 @@ static int attach_kprobe(const struct bpf_program *prog, long cookie, struct bpf
pr_warn("kprobe name is invalid: %s\n", func_name);
return -EINVAL;
}
+
+ if (offset < 0) {
+ free(func);
+ pr_warn("kprobe offset must be a non-negative integer: %li\n", offset);
+ return -EINVAL;
+ }
+
if (opts.retprobe && offset != 0) {
free(func);
pr_warn("kretprobes do not support offset specification\n");
diff --git a/tools/lib/bpf/strset.c b/tools/lib/bpf/strset.c
index 2464bcbd04e0..ace73c6b3d62 100644
--- a/tools/lib/bpf/strset.c
+++ b/tools/lib/bpf/strset.c
@@ -107,6 +107,41 @@ static void *strset_add_str_mem(struct strset *set, size_t add_sz)
set->strs_data_len, set->strs_data_max_len, add_sz);
}
+static long strset_str_append(struct strset *set, const char *s)
+{
+ uintptr_t old_data = (uintptr_t)set->strs_data;
+ size_t old_data_len = set->strs_data_len;
+ uintptr_t old_s = (uintptr_t)s;
+ long len = strlen(s) + 1;
+ void *p;
+
+ /*
+ * Hashmap keys are always offsets within set->strs_data, so to even
+ * look up some string from the "outside", we need to first append it
+ * at the end, so that it can be addressed with an offset. Luckily,
+ * until set->strs_data_len is incremented, that string is just a piece
+ * of garbage for the rest of the code, so no harm, no foul. On the
+ * other hand, if the string is unique, it's already appended and
+ * ready to be used, only a simple set->strs_data_len increment away.
+ */
+ p = strset_add_str_mem(set, len);
+ if (!p)
+ return -ENOMEM;
+
+ /*
+ * The set->strs_data might have reallocated and if 's' pointed
+ * to an internal string within the old buffer, then it became
+ * dangling and needs to be reconstructed before the copy.
+ */
+ if (old_data && old_data != (uintptr_t)set->strs_data &&
+ old_s >= old_data && old_s < old_data + old_data_len)
+ s = set->strs_data + (old_s - old_data);
+
+ memcpy(p, s, len);
+
+ return len;
+}
+
/* Find string offset that corresponds to a given string *s*.
* Returns:
* - >0 offset into string data, if string is found;
@@ -116,16 +151,12 @@ static void *strset_add_str_mem(struct strset *set, size_t add_sz)
int strset__find_str(struct strset *set, const char *s)
{
long old_off, new_off, len;
- void *p;
- /* see strset__add_str() for why we do this */
- len = strlen(s) + 1;
- p = strset_add_str_mem(set, len);
- if (!p)
- return -ENOMEM;
+ len = strset_str_append(set, s);
+ if (len < 0)
+ return len;
new_off = set->strs_data_len;
- memcpy(p, s, len);
if (hashmap__find(set->strs_hash, new_off, &old_off))
return old_off;
@@ -142,24 +173,13 @@ int strset__find_str(struct strset *set, const char *s)
int strset__add_str(struct strset *set, const char *s)
{
long old_off, new_off, len;
- void *p;
int err;
- /* Hashmap keys are always offsets within set->strs_data, so to even
- * look up some string from the "outside", we need to first append it
- * at the end, so that it can be addressed with an offset. Luckily,
- * until set->strs_data_len is incremented, that string is just a piece
- * of garbage for the rest of the code, so no harm, no foul. On the
- * other hand, if the string is unique, it's already appended and
- * ready to be used, only a simple set->strs_data_len increment away.
- */
- len = strlen(s) + 1;
- p = strset_add_str_mem(set, len);
- if (!p)
- return -ENOMEM;
+ len = strset_str_append(set, s);
+ if (len < 0)
+ return len;
new_off = set->strs_data_len;
- memcpy(p, s, len);
/* Now attempt to add the string, but only if the string with the same
* contents doesn't exist already (HASHMAP_ADD strategy). If such
diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c
index 76e7c5bf0cb2..e9ad470e7df6 100644
--- a/tools/lib/bpf/usdt.c
+++ b/tools/lib/bpf/usdt.c
@@ -460,10 +460,10 @@ static int parse_elf_segs(Elf *elf, const char *path, struct elf_seg **segs, siz
static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs, size_t *seg_cnt)
{
- char path[PATH_MAX], line[PATH_MAX], mode[16];
+ char path[PATH_MAX], line[4096], mode[16];
size_t seg_start, seg_end, seg_off;
struct elf_seg *seg;
- int tmp_pid, i, err;
+ int tmp_pid, n, i, err;
FILE *f;
*seg_cnt = 0;
@@ -472,8 +472,13 @@ static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs,
* /proc/<pid>/root/<path>. They will be reported as just /<path> in
* /proc/<pid>/maps.
*/
- if (sscanf(lib_path, "/proc/%d/root%s", &tmp_pid, path) == 2 && pid == tmp_pid)
+ /* %n is not counted in sscanf() return value, so initialize it. */
+ n = 0;
+ if (sscanf(lib_path, "/proc/%d/root%n", &tmp_pid, &n) == 1 &&
+ n > 0 && pid == tmp_pid && lib_path[n] == '/') {
+ libbpf_strlcpy(path, lib_path + n, sizeof(path));
goto proceed;
+ }
if (!realpath(lib_path, path)) {
pr_warn("usdt: failed to get absolute path of '%s' (err %d), using path as is...\n",
@@ -496,8 +501,11 @@ static int parse_vma_segs(int pid, const char *lib_path, struct elf_seg **segs,
* 7f5c6f5d1000-7f5c6f5d3000 rw-p 001c7000 08:04 21238613 /usr/lib64/libc-2.17.so
* 7f5c6f5d3000-7f5c6f5d8000 rw-p 00000000 00:00 0
* 7f5c6f5d8000-7f5c6f5d9000 r-xp 00000000 103:01 362990598 /data/users/andriin/linux/tools/bpf/usdt/libhello_usdt.so
+ *
+ * Some VMA names can be longer than the local buffer. Bound the
+ * writes, but still consume the rest of the line.
*/
- while (fscanf(f, "%zx-%zx %s %zx %*s %*d%[^\n]\n",
+ while (fscanf(f, "%zx-%zx %15s %zx %*s %*d%4095[^\n]%*[^\n]\n",
&seg_start, &seg_end, mode, &seg_off, line) == 5) {
void *tmp;
diff --git a/tools/lib/symbol/kallsyms.c b/tools/lib/symbol/kallsyms.c
index e335ac2b9e19..d64bd9cc82a9 100644
--- a/tools/lib/symbol/kallsyms.c
+++ b/tools/lib/symbol/kallsyms.c
@@ -60,7 +60,7 @@ int kallsyms__parse(const char *filename, void *arg,
read_to_eol(&io);
continue;
}
- for (i = 0; i < sizeof(symbol_name); i++) {
+ for (i = 0; i < KSYM_NAME_LEN; i++) {
ch = io__get_char(&io);
if (ch < 0 || ch == '\n')
break;
@@ -68,6 +68,9 @@ int kallsyms__parse(const char *filename, void *arg,
}
symbol_name[i] = '\0';
+ if (i == KSYM_NAME_LEN)
+ read_to_eol(&io);
+
err = process_symbol(arg, symbol_name, symbol_type, start);
if (err)
break;
diff --git a/tools/power/x86/intel-speed-select/isst-daemon.c b/tools/power/x86/intel-speed-select/isst-daemon.c
index 66df21b2b573..acedb7432849 100644
--- a/tools/power/x86/intel-speed-select/isst-daemon.c
+++ b/tools/power/x86/intel-speed-select/isst-daemon.c
@@ -148,6 +148,7 @@ static void daemonize(char *rundir, char *pidfile)
{
int pid, sid, i;
char str[10];
+ struct stat st;
struct sigaction sig_actions;
sigset_t sig_set;
int ret;
@@ -200,11 +201,17 @@ static void daemonize(char *rundir, char *pidfile)
if (ret == -1)
exit(EXIT_FAILURE);
- pid_file_handle = open(pidfile, O_RDWR | O_CREAT, 0600);
+ pid_file_handle = open(pidfile, O_RDWR | O_CREAT | O_NOFOLLOW, 0600);
if (pid_file_handle == -1) {
/* Couldn't open lock file */
exit(1);
}
+
+ if (fstat(pid_file_handle, &st) == -1)
+ exit(1);
+
+ if (!S_ISREG(st.st_mode))
+ exit(1);
/* Try to lock file */
#ifdef LOCKF_SUPPORT
if (lockf(pid_file_handle, F_TLOCK, 0) == -1) {
diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c
index 0d0c434426e7..b9236c6d30d9 100644
--- a/tools/testing/cxl/test/cxl.c
+++ b/tools/testing/cxl/test/cxl.c
@@ -402,12 +402,16 @@ static void depopulate_all_mock_resources(void)
static struct cxl_mock_res *alloc_mock_res(resource_size_t size, int align)
{
- struct cxl_mock_res *res = kzalloc(sizeof(*res), GFP_KERNEL);
struct genpool_data_align data = {
.align = align,
};
unsigned long phys;
+ struct cxl_mock_res *res __free(kfree) = kzalloc(sizeof(*res),
+ GFP_KERNEL);
+ if (!res)
+ return NULL;
+
INIT_LIST_HEAD(&res->list);
phys = gen_pool_alloc_algo(cxl_mock_pool, size,
gen_pool_first_fit_align, &data);
@@ -422,7 +426,7 @@ static struct cxl_mock_res *alloc_mock_res(resource_size_t size, int align)
list_add(&res->list, &mock_res);
mutex_unlock(&mock_res_lock);
- return res;
+ return no_free_ptr(res);
}
static int populate_cedt(void)
@@ -1448,7 +1452,7 @@ static __init int cxl_test_init(void)
return 0;
err_root:
- platform_device_put(cxl_acpi);
+ platform_device_unregister(cxl_acpi);
err_rch:
cxl_rch_topo_exit();
err_single:
diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c
index 71916e0e1546..7d657ce54d64 100644
--- a/tools/testing/cxl/test/mem.c
+++ b/tools/testing/cxl/test/mem.c
@@ -1015,7 +1015,7 @@ static int mock_get_lsa(struct cxl_mockmem_data *mdata,
return -EINVAL;
offset = le32_to_cpu(get_lsa->offset);
length = le32_to_cpu(get_lsa->length);
- if (offset + length > LSA_SIZE)
+ if (offset > LSA_SIZE || length > LSA_SIZE - offset)
return -EINVAL;
if (length > cmd->size_out)
return -EINVAL;
@@ -1035,7 +1035,7 @@ static int mock_set_lsa(struct cxl_mockmem_data *mdata,
return -EINVAL;
offset = le32_to_cpu(set_lsa->offset);
length = cmd->size_in - sizeof(*set_lsa);
- if (offset + length > LSA_SIZE)
+ if (offset > LSA_SIZE || length > LSA_SIZE - offset)
return -EINVAL;
memcpy(lsa + offset, &set_lsa->data[0], length);
@@ -1484,7 +1484,7 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)
return -ENOMEM;
dev_set_drvdata(dev, mdata);
- mdata->lsa = vmalloc(LSA_SIZE);
+ mdata->lsa = vzalloc(LSA_SIZE);
if (!mdata->lsa)
return -ENOMEM;
mdata->fw = vmalloc(FW_SIZE);
diff --git a/tools/testing/selftests/bpf/bpf_util.h b/tools/testing/selftests/bpf/bpf_util.h
index feff92219e21..33486c460913 100644
--- a/tools/testing/selftests/bpf/bpf_util.h
+++ b/tools/testing/selftests/bpf/bpf_util.h
@@ -8,6 +8,7 @@
#include <errno.h>
#include <syscall.h>
#include <bpf/libbpf.h> /* libbpf_num_possible_cpus */
+#include <linux/args.h>
static inline unsigned int bpf_num_possible_cpus(void)
{
@@ -21,25 +22,43 @@ static inline unsigned int bpf_num_possible_cpus(void)
return possible_cpus;
}
-/* Copy up to sz - 1 bytes from zero-terminated src string and ensure that dst
- * is zero-terminated string no matter what (unless sz == 0, in which case
- * it's a no-op). It's conceptually close to FreeBSD's strlcpy(), but differs
- * in what is returned. Given this is internal helper, it's trivial to extend
- * this, when necessary. Use this instead of strncpy inside libbpf source code.
+/*
+ * Simplified strscpy() implementation. The kernel one is in lib/string.c
*/
-static inline void bpf_strlcpy(char *dst, const char *src, size_t sz)
+static inline ssize_t sized_strscpy(char *dest, const char *src, size_t count)
{
- size_t i;
+ long res = 0;
- if (sz == 0)
- return;
+ if (count == 0)
+ return -E2BIG;
- sz--;
- for (i = 0; i < sz && src[i]; i++)
- dst[i] = src[i];
- dst[i] = '\0';
+ while (count > 1) {
+ char c;
+
+ c = src[res];
+ dest[res] = c;
+ if (!c)
+ return res;
+ res++;
+ count--;
+ }
+
+ /* Force NUL-termination. */
+ dest[res] = '\0';
+
+ /* Return E2BIG if the source didn't stop */
+ return src[res] ? -E2BIG : res;
}
+#define __strscpy0(dst, src, ...) \
+ sized_strscpy(dst, src, sizeof(dst))
+#define __strscpy1(dst, src, size) \
+ sized_strscpy(dst, src, size)
+
+#undef strscpy /* Redefine the placeholder from tools/include/linux/string.h */
+#define strscpy(dst, src, ...) \
+ CONCATENATE(__strscpy, COUNT_ARGS(__VA_ARGS__))(dst, src, __VA_ARGS__)
+
#define __bpf_percpu_val_align __attribute__((__aligned__(8)))
#define BPF_DECLARE_PERCPU(type, name) \
diff --git a/tools/testing/selftests/bpf/cgroup_helpers.c b/tools/testing/selftests/bpf/cgroup_helpers.c
index e4535451322e..23bcf1aca76f 100644
--- a/tools/testing/selftests/bpf/cgroup_helpers.c
+++ b/tools/testing/selftests/bpf/cgroup_helpers.c
@@ -85,7 +85,7 @@ static int __enable_controllers(const char *cgroup_path, const char *controllers
enable[len] = 0;
close(fd);
} else {
- bpf_strlcpy(enable, controllers, sizeof(enable));
+ strscpy(enable, controllers);
}
snprintf(path, sizeof(path), "%s/cgroup.subtree_control", cgroup_path);
diff --git a/tools/testing/selftests/bpf/prog_tests/fill_link_info.c b/tools/testing/selftests/bpf/prog_tests/fill_link_info.c
index e59af2aa6601..d6b50f73bae2 100644
--- a/tools/testing/selftests/bpf/prog_tests/fill_link_info.c
+++ b/tools/testing/selftests/bpf/prog_tests/fill_link_info.c
@@ -455,7 +455,7 @@ verify_umulti_link_info(int fd, bool retprobe, __u64 *offsets,
ASSERT_EQ(info.uprobe_multi.pid, getpid(), "info.uprobe_multi.pid");
ASSERT_EQ(info.uprobe_multi.count, 3, "info.uprobe_multi.count");
- ASSERT_EQ(info.uprobe_multi.flags & BPF_F_KPROBE_MULTI_RETURN,
+ ASSERT_EQ(info.uprobe_multi.flags & BPF_F_UPROBE_MULTI_RETURN,
retprobe, "info.uprobe_multi.flags.retprobe");
ASSERT_EQ(info.uprobe_multi.path_size, strlen(path) + 1, "info.uprobe_multi.path_size");
ASSERT_STREQ(path_buf, path, "info.uprobe_multi.path");
diff --git a/tools/testing/selftests/bpf/prog_tests/verifier_log.c b/tools/testing/selftests/bpf/prog_tests/verifier_log.c
index 8337c6bc5b95..abc473c3741f 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier_log.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier_log.c
@@ -317,6 +317,7 @@ static void verif_btf_log_subtest(bool bad_btf)
res = load_btf(&opts, true);
ASSERT_EQ(res, -ENOSPC, "half_log_fd");
ASSERT_EQ(strlen(logs.buf), 24, "log_fixed_25");
+ strscpy(op_name, "log_fixed", sizeof(op_name));
ASSERT_STRNEQ(logs.buf, logs.reference, 24, op_name);
/* validate rolling verifier log logic: try all variations of log buf
diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_task_vmas.c b/tools/testing/selftests/bpf/progs/bpf_iter_task_vmas.c
index 423b39e60b6f..81390e3a7861 100644
--- a/tools/testing/selftests/bpf/progs/bpf_iter_task_vmas.c
+++ b/tools/testing/selftests/bpf/progs/bpf_iter_task_vmas.c
@@ -52,7 +52,7 @@ SEC("iter/task_vma") int proc_maps(struct bpf_iter__task_vma *ctx)
bpf_d_path(&file->f_path, d_path_buf, D_PATH_BUF_SIZE);
BPF_SEQ_PRINTF(seq, "%08llx ", vma->vm_pgoff << 12);
- BPF_SEQ_PRINTF(seq, "%02x:%02x %u", MAJOR(dev), MINOR(dev),
+ BPF_SEQ_PRINTF(seq, "%02x:%02x %llu", MAJOR(dev), MINOR(dev),
file->f_inode->i_ino);
BPF_SEQ_PRINTF(seq, "\t%s\n", d_path_buf);
} else {
diff --git a/tools/testing/selftests/bpf/progs/test_tunnel_kern.c b/tools/testing/selftests/bpf/progs/test_tunnel_kern.c
index 32127f1cd687..30f1de458669 100644
--- a/tools/testing/selftests/bpf/progs/test_tunnel_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_tunnel_kern.c
@@ -6,6 +6,7 @@
* modify it under the terms of version 2 of the GNU General Public
* License as published by the Free Software Foundation.
*/
+#define BPF_NO_KFUNC_PROTOTYPES
#include "vmlinux.h"
#include <bpf/bpf_core_read.h>
#include <bpf/bpf_helpers.h>
@@ -36,12 +37,10 @@ enum bpf_fou_encap_type___local {
FOU_BPF_ENCAP_GUE___local,
};
-struct bpf_fou_encap;
-
int bpf_skb_set_fou_encap(struct __sk_buff *skb_ctx,
- struct bpf_fou_encap *encap, int type) __ksym;
+ struct bpf_fou_encap___local *encap, int type) __ksym;
int bpf_skb_get_fou_encap(struct __sk_buff *skb_ctx,
- struct bpf_fou_encap *encap) __ksym;
+ struct bpf_fou_encap___local *encap) __ksym;
struct xfrm_state *
bpf_xdp_get_xfrm_state(struct xdp_md *ctx, struct bpf_xfrm_state_opts *opts,
u32 opts__sz) __ksym;
@@ -781,7 +780,7 @@ int ipip_gue_set_tunnel(struct __sk_buff *skb)
encap.sport = 0;
encap.dport = bpf_htons(5555);
- ret = bpf_skb_set_fou_encap(skb, (struct bpf_fou_encap *)&encap,
+ ret = bpf_skb_set_fou_encap(skb, &encap,
bpf_core_enum_value(enum bpf_fou_encap_type___local,
FOU_BPF_ENCAP_GUE___local));
if (ret < 0) {
@@ -820,7 +819,7 @@ int ipip_fou_set_tunnel(struct __sk_buff *skb)
encap.sport = 0;
encap.dport = bpf_htons(5555);
- ret = bpf_skb_set_fou_encap(skb, (struct bpf_fou_encap *)&encap,
+ ret = bpf_skb_set_fou_encap(skb, &encap,
FOU_BPF_ENCAP_FOU___local);
if (ret < 0) {
log_err(ret);
@@ -843,7 +842,7 @@ int ipip_encap_get_tunnel(struct __sk_buff *skb)
return TC_ACT_SHOT;
}
- ret = bpf_skb_get_fou_encap(skb, (struct bpf_fou_encap *)&encap);
+ ret = bpf_skb_get_fou_encap(skb, &encap);
if (ret < 0) {
log_err(ret);
return TC_ACT_SHOT;
diff --git a/tools/testing/selftests/bpf/progs/xdp_flowtable.c b/tools/testing/selftests/bpf/progs/xdp_flowtable.c
index 7fdc7b23ee74..e67daa02749d 100644
--- a/tools/testing/selftests/bpf/progs/xdp_flowtable.c
+++ b/tools/testing/selftests/bpf/progs/xdp_flowtable.c
@@ -15,7 +15,10 @@ struct bpf_flowtable_opts___local {
s32 error;
};
-struct flow_offload_tuple_rhash *
+struct flow_offload_tuple_rhash___local {
+};
+
+struct flow_offload_tuple_rhash___local *
bpf_xdp_flow_lookup(struct xdp_md *, struct bpf_fib_lookup *,
struct bpf_flowtable_opts___local *, u32) __ksym;
@@ -67,7 +70,7 @@ int xdp_flowtable_do_lookup(struct xdp_md *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
struct bpf_flowtable_opts___local opts = {};
- struct flow_offload_tuple_rhash *tuplehash;
+ struct flow_offload_tuple_rhash___local *tuplehash;
struct bpf_fib_lookup tuple = {
.ifindex = ctx->ingress_ifindex,
};
diff --git a/tools/testing/selftests/bpf/vmtest.sh b/tools/testing/selftests/bpf/vmtest.sh
index 79505d294c44..b0aca33d7591 100755
--- a/tools/testing/selftests/bpf/vmtest.sh
+++ b/tools/testing/selftests/bpf/vmtest.sh
@@ -373,7 +373,7 @@ main()
local exit_command="poweroff -f"
local debug_shell="no"
- while getopts ':hskl:id:j:' opt; do
+ while getopts ':hsl:id:j:' opt; do
case ${opt} in
l)
LOCAL_ROOTFS_IMAGE="$OPTARG"
diff --git a/tools/testing/selftests/filesystems/statmount/Makefile b/tools/testing/selftests/filesystems/statmount/Makefile
index 3af3136e35a4..2a4ddfa54834 100644
--- a/tools/testing/selftests/filesystems/statmount/Makefile
+++ b/tools/testing/selftests/filesystems/statmount/Makefile
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-or-later
-CFLAGS += -Wall -O2 -g $(KHDR_INCLUDES)
+CFLAGS += -Wall -O2 -g $(KHDR_INCLUDES) $(TOOLS_INCLUDES)
+
TEST_GEN_PROGS := statmount_test statmount_test_ns
include ../../lib.mk
diff --git a/tools/testing/selftests/filesystems/statmount/statmount.h b/tools/testing/selftests/filesystems/statmount/statmount.h
index f4294bab9d73..8a4084d94d70 100644
--- a/tools/testing/selftests/filesystems/statmount/statmount.h
+++ b/tools/testing/selftests/filesystems/statmount/statmount.h
@@ -7,6 +7,42 @@
#include <linux/mount.h>
#include <asm/unistd.h>
+#ifndef __NR_statmount
+ #if defined __alpha__
+ #define __NR_statmount 567
+ #elif defined _MIPS_SIM
+ #if _MIPS_SIM == _MIPS_SIM_ABI32 /* o32 */
+ #define __NR_statmount 4457
+ #endif
+ #if _MIPS_SIM == _MIPS_SIM_NABI32 /* n32 */
+ #define __NR_statmount 6457
+ #endif
+ #if _MIPS_SIM == _MIPS_SIM_ABI64 /* n64 */
+ #define __NR_statmount 5457
+ #endif
+ #else
+ #define __NR_statmount 457
+ #endif
+#endif
+
+#ifndef __NR_listmount
+ #if defined __alpha__
+ #define __NR_listmount 568
+ #elif defined _MIPS_SIM
+ #if _MIPS_SIM == _MIPS_SIM_ABI32 /* o32 */
+ #define __NR_listmount 4458
+ #endif
+ #if _MIPS_SIM == _MIPS_SIM_NABI32 /* n32 */
+ #define __NR_listmount 6458
+ #endif
+ #if _MIPS_SIM == _MIPS_SIM_ABI64 /* n64 */
+ #define __NR_listmount 5458
+ #endif
+ #else
+ #define __NR_listmount 458
+ #endif
+#endif
+
static inline int statmount(uint64_t mnt_id, uint64_t mnt_ns_id, uint64_t mask,
struct statmount *buf, size_t bufsize,
unsigned int flags)
diff --git a/tools/testing/selftests/ftrace/test.d/00basic/test_ownership.tc b/tools/testing/selftests/ftrace/test.d/00basic/test_ownership.tc
index e71cc3ad0bdf..6d00d3c0f493 100644
--- a/tools/testing/selftests/ftrace/test.d/00basic/test_ownership.tc
+++ b/tools/testing/selftests/ftrace/test.d/00basic/test_ownership.tc
@@ -6,7 +6,7 @@
original_group=`stat -c "%g" .`
original_owner=`stat -c "%u" .`
-local mount_point=$(get_mount_point)
+mount_point=$(get_mount_point)
mount_options=$(get_mnt_options "$mount_point")
diff --git a/tools/testing/selftests/hid/Makefile b/tools/testing/selftests/hid/Makefile
index 662209f5fabc..e50c0624cca2 100644
--- a/tools/testing/selftests/hid/Makefile
+++ b/tools/testing/selftests/hid/Makefile
@@ -24,7 +24,7 @@ CXX ?= $(CROSS_COMPILE)g++
HOSTPKG_CONFIG := pkg-config
-CFLAGS += -g -O0 -rdynamic -Wall -Werror -I$(OUTPUT)
+CFLAGS += -g -O0 -rdynamic -Wall -Werror -I$(OUTPUT) $(KHDR_INCLUDES)
CFLAGS += -I$(OUTPUT)/tools/include
LDLIBS += -lelf -lz -lrt -lpthread
diff --git a/tools/testing/selftests/hid/hid_bpf.c b/tools/testing/selftests/hid/hid_bpf.c
index 86f4d66379f7..73ae0948f0b6 100644
--- a/tools/testing/selftests/hid/hid_bpf.c
+++ b/tools/testing/selftests/hid/hid_bpf.c
@@ -109,6 +109,20 @@ static void load_programs(const struct test_program programs[],
self->skel = hid__open();
ASSERT_OK_PTR(self->skel) TEARDOWN_LOG("Error while calling hid__open");
+ /*
+ * Disable all struct_ops maps by default so libbpf does not autoload
+ * programs referenced by maps that are unrelated to the current test.
+ */
+ bpf_object__for_each_map(iter_map, *self->skel->skeleton->obj) {
+ if (bpf_map__type(iter_map) == BPF_MAP_TYPE_STRUCT_OPS) {
+ err = bpf_map__set_autocreate(iter_map, false);
+ ASSERT_OK(err) TH_LOG("can not disable struct_ops map '%s'",
+ bpf_map__name(iter_map));
+ }
+
+ bpf_map__set_autoattach(iter_map, false);
+ }
+
for (int i = 0; i < progs_count; i++) {
struct bpf_program *prog;
struct bpf_map *map;
@@ -125,6 +139,10 @@ static void load_programs(const struct test_program programs[],
ASSERT_OK_PTR(map) TH_LOG("can not find struct_ops by name '%s'",
programs[i].name + 4);
+ err = bpf_map__set_autocreate(map, true);
+ ASSERT_OK(err) TH_LOG("can not enable struct_ops map '%s'",
+ programs[i].name + 4);
+
/* hid_id is the first field of struct hid_bpf_ops */
ops_hid_id = bpf_map__initial_value(map, NULL);
ASSERT_OK_PTR(ops_hid_id) TH_LOG("unable to retrieve struct_ops data");
@@ -132,13 +150,6 @@ static void load_programs(const struct test_program programs[],
*ops_hid_id = self->hid_id;
}
- /* we disable the auto-attach feature of all maps because we
- * only want the tested one to be manually attached in the next
- * call to bpf_map__attach_struct_ops()
- */
- bpf_object__for_each_map(iter_map, *self->skel->skeleton->obj)
- bpf_map__set_autoattach(iter_map, false);
-
err = hid__load(self->skel);
ASSERT_OK(err) TH_LOG("hid_skel_load failed: %d", err);
@@ -910,6 +921,17 @@ TEST_F(hid_bpf, test_rdesc_fixup)
ASSERT_EQ(rpt_desc.value[4], 0x42);
}
+TEST_F(hid_bpf, test_rdesc_fixup_get_data_overflow)
+{
+ const struct test_program progs[] = {
+ { .name = "hid_rdesc_fixup_get_data_overflow" },
+ };
+
+ LOAD_PROGRAMS(progs);
+
+ ASSERT_EQ(self->skel->bss->get_data_overflow_check, 1);
+}
+
static int libbpf_print_fn(enum libbpf_print_level level,
const char *format, va_list args)
{
diff --git a/tools/testing/selftests/hid/progs/hid.c b/tools/testing/selftests/hid/progs/hid.c
index 5ecc845ef792..b21fbb13c926 100644
--- a/tools/testing/selftests/hid/progs/hid.c
+++ b/tools/testing/selftests/hid/progs/hid.c
@@ -13,6 +13,7 @@ struct attach_prog_args {
__u64 callback_check = 52;
__u64 callback2_check = 52;
+__u64 get_data_overflow_check;
SEC("?struct_ops/hid_device_event")
int BPF_PROG(hid_first_event, struct hid_bpf_ctx *hid_ctx, enum hid_report_type type)
@@ -240,6 +241,20 @@ struct hid_bpf_ops rdesc_fixup = {
.hid_rdesc_fixup = (void *)hid_rdesc_fixup,
};
+SEC("?struct_ops.s/hid_rdesc_fixup")
+int BPF_PROG(hid_rdesc_fixup_get_data_overflow, struct hid_bpf_ctx *hid_ctx)
+{
+ if (!hid_bpf_get_data(hid_ctx, 2 /* offset */, ~0ULL /* size */))
+ get_data_overflow_check = 1;
+
+ return 0;
+}
+
+SEC(".struct_ops.link")
+struct hid_bpf_ops rdesc_fixup_get_data_overflow = {
+ .hid_rdesc_fixup = (void *)hid_rdesc_fixup_get_data_overflow,
+};
+
SEC("?struct_ops/hid_device_event")
int BPF_PROG(hid_test_insert1, struct hid_bpf_ctx *hid_ctx, enum hid_report_type type)
{
diff --git a/tools/testing/selftests/landlock/scoped_signal_test.c b/tools/testing/selftests/landlock/scoped_signal_test.c
index d8bf33417619..2d37d0c06c06 100644
--- a/tools/testing/selftests/landlock/scoped_signal_test.c
+++ b/tools/testing/selftests/landlock/scoped_signal_test.c
@@ -249,12 +249,12 @@ TEST_F(scoped_domains, check_access_signal)
_metadata->exit_code = KSFT_FAIL;
}
-enum thread_return {
- THREAD_INVALID = 0,
- THREAD_SUCCESS = 1,
- THREAD_ERROR = 2,
- THREAD_TEST_FAILED = 3,
-};
+/* clang-format off */
+#define THREAD_INVALID ((void *)0)
+#define THREAD_SUCCESS ((void *)1)
+#define THREAD_ERROR ((void *)2)
+#define THREAD_TEST_FAILED ((void *)3)
+/* clang-format on */
static void *thread_sync(void *arg)
{
@@ -262,15 +262,15 @@ static void *thread_sync(void *arg)
char buf;
if (read(pipe_read, &buf, 1) != 1)
- return (void *)THREAD_ERROR;
+ return THREAD_ERROR;
- return (void *)THREAD_SUCCESS;
+ return THREAD_SUCCESS;
}
TEST(signal_scoping_thread_before)
{
pthread_t no_sandbox_thread;
- enum thread_return ret = THREAD_INVALID;
+ void *ret = THREAD_INVALID;
int thread_pipe[2];
drop_caps(_metadata);
@@ -285,7 +285,7 @@ TEST(signal_scoping_thread_before)
EXPECT_EQ(0, pthread_kill(no_sandbox_thread, 0));
EXPECT_EQ(1, write(thread_pipe[1], ".", 1));
- EXPECT_EQ(0, pthread_join(no_sandbox_thread, (void **)&ret));
+ EXPECT_EQ(0, pthread_join(no_sandbox_thread, &ret));
EXPECT_EQ(THREAD_SUCCESS, ret);
EXPECT_EQ(0, close(thread_pipe[0]));
@@ -295,7 +295,7 @@ TEST(signal_scoping_thread_before)
TEST(signal_scoping_thread_after)
{
pthread_t scoped_thread;
- enum thread_return ret = THREAD_INVALID;
+ void *ret = THREAD_INVALID;
int thread_pipe[2];
drop_caps(_metadata);
@@ -310,7 +310,7 @@ TEST(signal_scoping_thread_after)
EXPECT_EQ(0, pthread_kill(scoped_thread, 0));
EXPECT_EQ(1, write(thread_pipe[1], ".", 1));
- EXPECT_EQ(0, pthread_join(scoped_thread, (void **)&ret));
+ EXPECT_EQ(0, pthread_join(scoped_thread, &ret));
EXPECT_EQ(THREAD_SUCCESS, ret);
EXPECT_EQ(0, close(thread_pipe[0]));
@@ -327,20 +327,20 @@ void *thread_setuid(void *ptr)
char buf;
if (read(arg->pipe_read, &buf, 1) != 1)
- return (void *)THREAD_ERROR;
+ return THREAD_ERROR;
/* libc's setuid() should update all thread's credentials. */
if (getuid() != arg->new_uid)
- return (void *)THREAD_TEST_FAILED;
+ return THREAD_TEST_FAILED;
- return (void *)THREAD_SUCCESS;
+ return THREAD_SUCCESS;
}
TEST(signal_scoping_thread_setuid)
{
struct thread_setuid_args arg;
pthread_t no_sandbox_thread;
- enum thread_return ret = THREAD_INVALID;
+ void *ret = THREAD_INVALID;
int pipe_parent[2];
int prev_uid;
@@ -367,7 +367,7 @@ TEST(signal_scoping_thread_setuid)
EXPECT_EQ(arg.new_uid, getuid());
EXPECT_EQ(1, write(pipe_parent[1], ".", 1));
- EXPECT_EQ(0, pthread_join(no_sandbox_thread, (void **)&ret));
+ EXPECT_EQ(0, pthread_join(no_sandbox_thread, &ret));
EXPECT_EQ(THREAD_SUCCESS, ret);
clear_cap(_metadata, CAP_SETUID);
@@ -400,6 +400,24 @@ static int setup_signal_handler(int signal)
return sigaction(SIGURG, &sa, NULL);
}
+/*
+ * MSG_OOB might be disabled in the kernel via the CONFIG_AF_UNIX_OOB
+ * switch, so this function can be used for probing for its availability.
+ */
+static bool has_af_unix_oob(void)
+{
+ bool available = false;
+ int sp[2];
+
+ if (socketpair(AF_UNIX, SOCK_STREAM, 0, sp) == 0) {
+ available = (send(sp[0], ".", 1, MSG_OOB) == 1);
+ close(sp[0]);
+ close(sp[1]);
+ }
+
+ return available;
+}
+
/* clang-format off */
FIXTURE(fown) {};
/* clang-format on */
@@ -462,6 +480,9 @@ TEST_F(fown, sigurg_socket)
int pipe_parent[2], pipe_child[2];
pid_t child;
+ if (!has_af_unix_oob())
+ SKIP(return, "CONFIG_AF_UNIX_OOB / MSG_OOB not available");
+
memset(&server_address, 0, sizeof(server_address));
set_unix_address(&server_address, 0);
@@ -559,4 +580,186 @@ TEST_F(fown, sigurg_socket)
_metadata->exit_code = KSFT_FAIL;
}
+/*
+ * Checks that LANDLOCK_SCOPE_SIGNAL is enforced on the asynchronous SIGIO
+ * delivery path (fcntl(F_SETOWN)) when the file owner is a process group.
+ *
+ * A sandboxed process sitting at the head of its process group's PID hlist (the
+ * default position right after fork()) used to escape the fcntl(F_SETOWN,
+ * -pgrp) domain recording: pid_task(pgrp, PIDTYPE_PGID) resolved to the process
+ * itself, so the same-thread-group exemption skipped recording its Landlock
+ * domain. At SIGIO time that domain was then unset and the signal fanned out
+ * to every group member, including non-sandboxed processes outside the domain.
+ */
+TEST(sigio_to_pgid_members)
+{
+ int trigger[2], sync_child[2];
+ char buf;
+ pid_t child;
+ int status, i;
+
+ drop_caps(_metadata);
+
+ /*
+ * Isolates the test in its own process group so the SIGIO fan-out stays
+ * bounded to this parent and the child forked below.
+ */
+ ASSERT_EQ(0, setpgid(0, 0));
+
+ /* The non-sandboxed parent is the protected (out-of-domain) target. */
+ ASSERT_EQ(0, setup_signal_handler(SIGURG));
+ signal_received = 0;
+
+ ASSERT_EQ(0, pipe2(trigger, O_CLOEXEC));
+ ASSERT_EQ(0, pipe2(sync_child, O_CLOEXEC));
+
+ child = fork();
+ ASSERT_LE(0, child);
+ if (child == 0) {
+ /*
+ * The child inherits the parent's new process group and, just
+ * attached with hlist_add_head_rcu(), is now the head of the
+ * pgid hlist: this is the case that used to skip the recording.
+ */
+ EXPECT_EQ(0, close(sync_child[0]));
+
+ /* In-domain positive control: the child must be signaled. */
+ ASSERT_EQ(0, setup_signal_handler(SIGURG));
+ signal_received = 0;
+
+ create_scoped_domain(_metadata, LANDLOCK_SCOPE_SIGNAL);
+
+ /* Owns the SIGIO source for the whole process group. */
+ ASSERT_EQ(0, fcntl(trigger[0], F_SETSIG, SIGURG));
+ ASSERT_EQ(0, fcntl(trigger[0], F_SETOWN, -getpgrp()));
+ ASSERT_EQ(0, fcntl(trigger[0], F_SETFL, O_ASYNC));
+
+ /* Fans SIGURG out to every member of the process group. */
+ ASSERT_EQ(1, write(trigger[1], ".", 1));
+
+ /*
+ * The sandboxed child is in its own domain and must always be
+ * signaled: this proves the SIGIO actually fired.
+ */
+ for (i = 0; i < 1000 && !signal_received; i++)
+ usleep(1000);
+ EXPECT_EQ(1, signal_received);
+
+ ASSERT_EQ(1, write(sync_child[1], ".", 1));
+ EXPECT_EQ(0, close(sync_child[1]));
+
+ _exit(_metadata->exit_code);
+ return;
+ }
+ EXPECT_EQ(0, close(sync_child[1]));
+ EXPECT_EQ(0, close(trigger[0]));
+ EXPECT_EQ(0, close(trigger[1]));
+
+ /* Waits for the child to generate the SIGIO. */
+ ASSERT_EQ(1, read(sync_child[0], &buf, 1));
+ EXPECT_EQ(0, close(sync_child[0]));
+
+ /* Lets a delivered-but-pending signal run our handler, if any. */
+ for (i = 0; i < 100 && !signal_received; i++)
+ usleep(1000);
+
+ /*
+ * SCOPE_SIGNAL must block the fan-out to this non-sandboxed parent,
+ * which is outside the child's Landlock domain. Before the fix the
+ * parent was signaled here.
+ */
+ EXPECT_EQ(0, signal_received);
+
+ ASSERT_EQ(child, waitpid(child, &status, 0));
+ if (WIFSIGNALED(status) || !WIFEXITED(status) ||
+ WEXITSTATUS(status) != EXIT_SUCCESS)
+ _metadata->exit_code = KSFT_FAIL;
+}
+
+static void *thread_setown_scoped(void *arg)
+{
+ const int fd = *(int *)arg;
+ int ruleset_fd;
+ const struct landlock_ruleset_attr ruleset_attr = {
+ .scoped = LANDLOCK_SCOPE_SIGNAL,
+ };
+
+ /* Sandboxes only this non-leader thread (no thread syncing). */
+ ruleset_fd =
+ landlock_create_ruleset(&ruleset_attr, sizeof(ruleset_attr), 0);
+ if (ruleset_fd < 0)
+ return THREAD_ERROR;
+ if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) ||
+ landlock_restrict_self(ruleset_fd, 0)) {
+ close(ruleset_fd);
+ return THREAD_ERROR;
+ }
+ close(ruleset_fd);
+
+ /* Makes this process group own the SIGIO source. */
+ if (fcntl(fd, F_SETSIG, SIGURG) || fcntl(fd, F_SETOWN, -getpgrp()) ||
+ fcntl(fd, F_SETFL, O_ASYNC))
+ return THREAD_ERROR;
+
+ return THREAD_SUCCESS;
+}
+
+/*
+ * Checks that the SIGIO fan-out is still delivered to the file owner's own
+ * process when fcntl(F_SETOWN, -pgrp) was issued from a sandboxed non-leader
+ * thread.
+ *
+ * The Landlock domain is recorded for a process-group owner (so out-of-domain
+ * members stay blocked, see sigio_to_pgid_members), but the kernel signals a
+ * process group through its members' thread-group leaders. Here the leader is
+ * not sandboxed and thus has a different domain than the registering thread, so
+ * the registration-time check cannot tell that it belongs to the owner's own
+ * process. hook_file_send_sigiotask() must recognize it through the recorded
+ * thread group and allow the delivery, matching the same-process guarantee of
+ * commit 18eb75f3af40. Without that exemption the leader is wrongly denied and
+ * never signaled.
+ */
+TEST(sigio_to_pgid_self)
+{
+ int trigger[2];
+ pthread_t thread;
+ void *ret = THREAD_INVALID;
+ int i;
+
+ drop_caps(_metadata);
+
+ /* Bounds the SIGIO fan-out to this process. */
+ ASSERT_EQ(0, setpgid(0, 0));
+
+ /* The non-sandboxed thread-group leader is the SIGIO target. */
+ ASSERT_EQ(0, setup_signal_handler(SIGURG));
+ signal_received = 0;
+
+ ASSERT_EQ(0, pipe2(trigger, O_CLOEXEC));
+
+ /*
+ * Registers the process-group fowner from a sibling thread that
+ * sandboxes only itself, so its domain differs from the leader's.
+ */
+ ASSERT_EQ(0, pthread_create(&thread, NULL, thread_setown_scoped,
+ &trigger[0]));
+ ASSERT_EQ(0, pthread_join(thread, &ret));
+ ASSERT_EQ(THREAD_SUCCESS, ret);
+
+ /* Fans SIGURG out to the process group. */
+ ASSERT_EQ(1, write(trigger[1], ".", 1));
+
+ for (i = 0; i < 1000 && !signal_received; i++)
+ usleep(1000);
+
+ /*
+ * Same-process delivery must always be allowed, even though the owner
+ * was registered from a sandboxed sibling thread.
+ */
+ EXPECT_EQ(1, signal_received);
+
+ EXPECT_EQ(0, close(trigger[0]));
+ EXPECT_EQ(0, close(trigger[1]));
+}
+
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/mm/charge_reserved_hugetlb.sh b/tools/testing/selftests/mm/charge_reserved_hugetlb.sh
index fa6713892d82..2b24d703e7ad 100755
--- a/tools/testing/selftests/mm/charge_reserved_hugetlb.sh
+++ b/tools/testing/selftests/mm/charge_reserved_hugetlb.sh
@@ -12,6 +12,7 @@ if [[ $(id -u) -ne 0 ]]; then
fi
nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages)
+trap 'echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages' EXIT INT TERM
fault_limit_file=limit_in_bytes
reservation_limit_file=rsvd.limit_in_bytes
@@ -65,7 +66,6 @@ function cleanup() {
if [[ -e $cgroup_path/hugetlb_cgroup_test2 ]]; then
rmdir $cgroup_path/hugetlb_cgroup_test2
fi
- echo 0 >/proc/sys/vm/nr_hugepages
echo CLEANUP DONE
}
@@ -585,4 +585,3 @@ if [[ $do_umount ]]; then
rmdir $cgroup_path
fi
-echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages
diff --git a/tools/testing/selftests/mm/compaction_test.c b/tools/testing/selftests/mm/compaction_test.c
index 9bc4591c7b16..be226ead936f 100644
--- a/tools/testing/selftests/mm/compaction_test.c
+++ b/tools/testing/selftests/mm/compaction_test.c
@@ -261,6 +261,9 @@ int main(int argc, char **argv)
mem_fragmentable_MB -= MAP_SIZE_MB;
}
+ /* Unmap every other entry in the list to create fragmentation with
+ * locked pages before invoking check_compaction().
+ */
for (entry = list; entry != NULL; entry = entry->next) {
munmap(entry->map, MAP_SIZE);
if (!entry->next)
diff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftests/mm/hmm-tests.c
index a545b5e50b19..722beaa6da1b 100644
--- a/tools/testing/selftests/mm/hmm-tests.c
+++ b/tools/testing/selftests/mm/hmm-tests.c
@@ -1612,8 +1612,8 @@ TEST_F(hmm2, snapshot)
}
/*
- * Test the hmm_range_fault() HMM_PFN_PMD flag for large pages that
- * should be mapped by a large page table entry.
+ * Test the hmm_range_fault() handling of large pages (PMD or PUD)
+ * that should be mapped by a large page table entry.
*/
TEST_F(hmm, compound)
{
@@ -1623,6 +1623,7 @@ TEST_F(hmm, compound)
unsigned long default_hsize;
int *ptr;
unsigned char *m;
+ unsigned char prot;
int ret;
unsigned long i;
@@ -1661,11 +1662,20 @@ TEST_F(hmm, compound)
ASSERT_EQ(ret, 0);
ASSERT_EQ(buffer->cpages, npages);
- /* Check what the device saw. */
+ /*
+ * Check what the device saw. The region is backed by a single huge
+ * page that the device reports either at PMD or at PUD level depending
+ * on the configured default hugepage size. Determine that level from
+ * the first page and require every page in the range to match it
+ * exactly, so that a fragmented mapping mixing levels (or a missing
+ * large-page bit) is still caught and reported with its actual value.
+ */
m = buffer->mirror;
+ prot = HMM_DMIRROR_PROT_WRITE |
+ ((m[0] & HMM_DMIRROR_PROT_PUD) ? HMM_DMIRROR_PROT_PUD :
+ HMM_DMIRROR_PROT_PMD);
for (i = 0; i < npages; ++i)
- ASSERT_EQ(m[i], HMM_DMIRROR_PROT_WRITE |
- HMM_DMIRROR_PROT_PMD);
+ ASSERT_EQ(m[i], prot);
/* Make the region read-only. */
ret = mprotect(buffer->ptr, size, PROT_READ);
@@ -1676,11 +1686,17 @@ TEST_F(hmm, compound)
ASSERT_EQ(ret, 0);
ASSERT_EQ(buffer->cpages, npages);
- /* Check what the device saw. */
+ /*
+ * Check what the device saw after mprotect(PROT_READ). Same
+ * approach as above: determine the mapping level from the first
+ * page and require every page to match it exactly.
+ */
m = buffer->mirror;
+ prot = HMM_DMIRROR_PROT_READ |
+ ((m[0] & HMM_DMIRROR_PROT_PUD) ? HMM_DMIRROR_PROT_PUD :
+ HMM_DMIRROR_PROT_PMD);
for (i = 0; i < npages; ++i)
- ASSERT_EQ(m[i], HMM_DMIRROR_PROT_READ |
- HMM_DMIRROR_PROT_PMD);
+ ASSERT_EQ(m[i], prot);
munmap(buffer->ptr, buffer->size);
buffer->ptr = NULL;
@@ -1880,6 +1896,8 @@ TEST_F(hmm, exclusive_cow)
unsigned long i;
int *ptr;
int ret;
+ pid_t pid;
+ int status;
npages = ALIGN(HMM_BUFFER_SIZE, self->page_size) >> self->page_shift;
ASSERT_NE(npages, 0);
@@ -1908,14 +1926,37 @@ TEST_F(hmm, exclusive_cow)
ASSERT_EQ(ret, 0);
ASSERT_EQ(buffer->cpages, npages);
- fork();
+ pid = fork();
+ if (pid == -1)
+ ASSERT_EQ(pid, 0);
+
+ if (pid == 0) {
+ /*
+ * Child verifies COW independently, then _exit(0)s so it does
+ * not run the test teardown. A failed ASSERT_* here makes the
+ * harness abort() the child, so the parent sees
+ * !WIFEXITED(status) below and fails in turn.
+ */
+ for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i)
+ ASSERT_EQ(ptr[i]++, i);
- /* Fault pages back to system memory and check them. */
+ for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i)
+ ASSERT_EQ(ptr[i], i + 1);
+
+ _exit(0);
+ }
+
+ /* Parent: also increment to verify COW works for both processes. */
for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i)
ASSERT_EQ(ptr[i]++, i);
for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i)
- ASSERT_EQ(ptr[i], i+1);
+ ASSERT_EQ(ptr[i], i + 1);
+
+ /* Parent: wait for child and then free the buffer. */
+ ASSERT_EQ(waitpid(pid, &status, 0), pid);
+ ASSERT_TRUE(WIFEXITED(status));
+ ASSERT_EQ(WEXITSTATUS(status), 0);
hmm_buffer_free(buffer);
}
diff --git a/tools/testing/selftests/mm/hugepage-mremap.c b/tools/testing/selftests/mm/hugepage-mremap.c
index c463d1c09c9b..3070484f9ed0 100644
--- a/tools/testing/selftests/mm/hugepage-mremap.c
+++ b/tools/testing/selftests/mm/hugepage-mremap.c
@@ -31,7 +31,7 @@
#define MB_TO_BYTES(x) (x * 1024 * 1024)
#define PROTECTION (PROT_READ | PROT_WRITE | PROT_EXEC)
-#define FLAGS (MAP_SHARED | MAP_ANONYMOUS)
+#define FLAGS (MAP_HUGETLB | MAP_SHARED)
static void check_bytes(char *addr)
{
@@ -75,25 +75,14 @@ static void register_region_with_uffd(char *addr, size_t len)
if (ioctl(uffd, UFFDIO_API, &uffdio_api) == -1)
ksft_exit_fail_msg("ioctl-UFFDIO_API: %s\n", strerror(errno));
- /* Create a private anonymous mapping. The memory will be
- * demand-zero paged--that is, not yet allocated. When we
- * actually touch the memory, it will be allocated via
- * the userfaultfd.
- */
-
- addr = mmap(NULL, len, PROT_READ | PROT_WRITE,
- MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
- if (addr == MAP_FAILED)
- ksft_exit_fail_msg("mmap: %s\n", strerror(errno));
-
- ksft_print_msg("Address returned by mmap() = %p\n", addr);
-
- /* Register the memory range of the mapping we just created for
- * handling by the userfaultfd object. In mode, we request to track
- * missing pages (i.e., pages that have not yet been faulted in).
+ /* Register the passed memory range for handling by the userfaultfd object.
+ * In mode, we request to track missing pages
+ * (i.e., pages that have not yet been faulted in).
*/
if (uffd_register(uffd, addr, len, true, false, false))
ksft_exit_fail_msg("ioctl-UFFDIO_REGISTER: %s\n", strerror(errno));
+
+ ksft_print_msg("Registered memory at address %p with userfaultfd\n", addr);
}
int main(int argc, char *argv[])
@@ -122,23 +111,20 @@ int main(int argc, char *argv[])
/* mmap to a PUD aligned address to hopefully trigger pmd sharing. */
unsigned long suggested_addr = 0x7eaa40000000;
- void *haddr = mmap((void *)suggested_addr, length, PROTECTION,
- MAP_HUGETLB | MAP_SHARED | MAP_POPULATE, fd, 0);
+ void *haddr = mmap((void *)suggested_addr, length, PROTECTION, FLAGS, fd, 0);
ksft_print_msg("Map haddr: Returned address is %p\n", haddr);
if (haddr == MAP_FAILED)
ksft_exit_fail_msg("mmap1: %s\n", strerror(errno));
/* mmap again to a dummy address to hopefully trigger pmd sharing. */
suggested_addr = 0x7daa40000000;
- void *daddr = mmap((void *)suggested_addr, length, PROTECTION,
- MAP_HUGETLB | MAP_SHARED | MAP_POPULATE, fd, 0);
+ void *daddr = mmap((void *)suggested_addr, length, PROTECTION, FLAGS, fd, 0);
ksft_print_msg("Map daddr: Returned address is %p\n", daddr);
if (daddr == MAP_FAILED)
ksft_exit_fail_msg("mmap3: %s\n", strerror(errno));
suggested_addr = 0x7faa40000000;
- void *vaddr =
- mmap((void *)suggested_addr, length, PROTECTION, FLAGS, -1, 0);
+ void *vaddr = mmap((void *)suggested_addr, length, PROTECTION, FLAGS, fd, 0);
ksft_print_msg("Map vaddr: Returned address is %p\n", vaddr);
if (vaddr == MAP_FAILED)
ksft_exit_fail_msg("mmap2: %s\n", strerror(errno));
diff --git a/tools/testing/selftests/mm/pkey-arm64.h b/tools/testing/selftests/mm/pkey-arm64.h
index 580e1b0bb38e..5583ea47768d 100644
--- a/tools/testing/selftests/mm/pkey-arm64.h
+++ b/tools/testing/selftests/mm/pkey-arm64.h
@@ -129,9 +129,10 @@ static inline u64 get_pkey_bits(u64 reg, int pkey)
static void aarch64_write_signal_pkey(ucontext_t *uctxt, u64 pkey)
{
struct _aarch64_ctx *ctx = GET_UC_RESV_HEAD(uctxt);
+ size_t resv_size = GET_UCP_RESV_SIZE(uctxt);
struct poe_context *poe_ctx =
(struct poe_context *) get_header(ctx, POE_MAGIC,
- sizeof(uctxt->uc_mcontext), NULL);
+ resv_size, NULL);
if (poe_ctx)
poe_ctx->por_el0 = pkey;
}
diff --git a/tools/testing/selftests/net/lib.sh b/tools/testing/selftests/net/lib.sh
index 501615d28530..32230e3508b9 100644
--- a/tools/testing/selftests/net/lib.sh
+++ b/tools/testing/selftests/net/lib.sh
@@ -67,12 +67,33 @@ ksft_exit_status_merge()
$ksft_xfail $ksft_pass $ksft_skip $ksft_fail
}
+timestamp_ms()
+{
+ local now
+ local seconds
+ local nanoseconds
+
+ now=$(date -u +%s:%N) || return
+ seconds=${now%:*}
+ nanoseconds=${now#*:}
+
+ if [[ $nanoseconds =~ ^[0-9]+$ ]]; then
+ nanoseconds=${nanoseconds:0:9}
+ else
+ nanoseconds=0
+ fi
+
+ echo $((seconds * 1000 + 10#$nanoseconds / 1000000))
+}
+
loopy_wait()
{
local sleep_cmd=$1; shift
local timeout_ms=$1; shift
+ local start_time
+ local current_time
- local start_time="$(date -u +%s%3N)"
+ start_time=$(timestamp_ms) || return
while true
do
local out
@@ -81,7 +102,7 @@ loopy_wait()
return 0
fi
- local current_time="$(date -u +%s%3N)"
+ current_time=$(timestamp_ms) || return
if ((current_time - start_time > timeout_ms)); then
echo -n "$out"
return 1
diff --git a/tools/testing/selftests/net/tcp_mmap.c b/tools/testing/selftests/net/tcp_mmap.c
index 4fcce5150850..2544ae35d07a 100644
--- a/tools/testing/selftests/net/tcp_mmap.c
+++ b/tools/testing/selftests/net/tcp_mmap.c
@@ -313,6 +313,8 @@ void *child_thread(void *arg)
tcp_info_get_rcv_mss(fd));
}
error:
+ if (ctx)
+ EVP_MD_CTX_free(ctx);
munmap(buffer, buffer_sz);
close(fd);
if (zflg)
@@ -606,6 +608,8 @@ int main(int argc, char *argv[])
EVP_DigestFinal_ex(ctx, digest, &digest_len);
send(fd, digest, (size_t)SHA256_DIGEST_LENGTH, 0);
}
+ if (ctx)
+ EVP_MD_CTX_free(ctx);
close(fd);
munmap(buffer, buffer_sz);
return 0;
diff --git a/tools/testing/shared/linux/idr.h b/tools/testing/shared/linux/idr.h
index 4e342f2e37cf..676c5564e33f 100644
--- a/tools/testing/shared/linux/idr.h
+++ b/tools/testing/shared/linux/idr.h
@@ -1 +1,5 @@
+/* Avoid duplicate definitions due to system headers. */
+#ifdef __CONCAT
+#undef __CONCAT
+#endif
#include "../../../../include/linux/idr.h"
diff --git a/tools/virtio/vringh_test.c b/tools/virtio/vringh_test.c
index b9591223437a..5ea6d29bc992 100644
--- a/tools/virtio/vringh_test.c
+++ b/tools/virtio/vringh_test.c
@@ -159,7 +159,12 @@ static int parallel_test(u64 features,
/* Parent and child use separate addresses, to check our mapping logic! */
host_map = mmap(NULL, mapsize, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
+ if (host_map == MAP_FAILED)
+ err(1, "mmap host_map");
+
guest_map = mmap(NULL, mapsize, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
+ if (guest_map == MAP_FAILED)
+ err(1, "mmap guest_map");
pipe_ret = pipe(to_guest);
assert(!pipe_ret);
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 84e6b8684ef6..f1351d0ceb30 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -6063,25 +6063,19 @@ struct kvm_io_device *kvm_io_bus_get_dev(struct kvm *kvm, enum kvm_bus bus_idx,
gpa_t addr)
{
struct kvm_io_bus *bus;
- int dev_idx, srcu_idx;
- struct kvm_io_device *iodev = NULL;
+ int dev_idx;
- srcu_idx = srcu_read_lock(&kvm->srcu);
+ lockdep_assert_held(&kvm->srcu);
bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
if (!bus)
- goto out_unlock;
+ return NULL;
dev_idx = kvm_io_bus_get_first_dev(bus, addr, 1);
if (dev_idx < 0)
- goto out_unlock;
-
- iodev = bus->range[dev_idx].dev;
-
-out_unlock:
- srcu_read_unlock(&kvm->srcu, srcu_idx);
+ return NULL;
- return iodev;
+ return bus->range[dev_idx].dev;
}
EXPORT_SYMBOL_GPL(kvm_io_bus_get_dev);
prev parent reply other threads:[~2026-07-24 15:06 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-24 15:05 Linux 6.12.97 Greg Kroah-Hartman
2026-07-24 15:05 ` Greg Kroah-Hartman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2026072452-june-static-438c@gregkh \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=lwn@lwn.net \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.