From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 650F3343887; Mon, 3 Aug 2026 09:35:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785749777; cv=none; b=SOvyC+TNOK6U9SHJIHkr1+Q57Gy+KbIpvDrv6zTuwujllai1KfWAD1ajFilArEOvOGe/znXujZpnHQ1JR59+vq++OJO2zq/78P6A+joWHOg9kUdfUHVaNVUgylkWRcaqA0NwYYVBa4zZNTReuO0OyYEhHi4ho5OmkAttq1VTg88= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785749777; c=relaxed/simple; bh=D0wugflmjvzFk9cgFmpYpXieBIkQ0N7tonyWZUdjZ7g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WUCwqsbpC9oBCS4+fK+eO8lWuwwzZd7jTSehVtmRYlDrKGBgg4XKh7wfjIQccn6KP1w2Qx3MdeEt5YUkXVj4mUxgeG5qgFR41BIG/WlW18XrLBvyFhk7mLJTGmzOCvZcbCth6f7n2qs67aE1IIYmvmeaG00pDMZ7e7dK4L5OSj4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=d1S6FWBK; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="d1S6FWBK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A747F1F00A3E; Mon, 3 Aug 2026 09:35:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1785749703; bh=U4jSYvq82y7hDmBXpoCUh51HjGbEmfUVToBZTR/WrX4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=d1S6FWBKrFCGIHrlGWqE5jP+rZkuICfcGtRgoCBAuvIkdDk3vdoSRxClLadaurXoQ /kxFEAmNU9xkGzVLEeCBBR9gStyOvMCMZUIw2b7DZIgFwDEIBW6qGXxxDPYgmJQC+j gvO271V6GivEBGnaGVeaBOvSMZE0trsIzTQMZeJA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org, stable@vger.kernel.org Cc: lwn@lwn.net, jslaby@suse.cz, Greg Kroah-Hartman Subject: Re: Linux 6.18.42 Date: Mon, 3 Aug 2026 11:34:31 +0200 Message-ID: <2026080331-bobbed-outshoot-bb3d@gregkh> X-Mailer: git-send-email 2.55.0 In-Reply-To: <2026080331-unadvised-spelling-faa3@gregkh> References: <2026080331-unadvised-spelling-faa3@gregkh> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst index d55751cad67c..8e0d31230b29 100644 --- a/Documentation/gpu/drm-mm.rst +++ b/Documentation/gpu/drm-mm.rst @@ -509,8 +509,14 @@ DRM GPUVM Function References DRM Buddy Allocator =================== -DRM Buddy Function References ------------------------------ +Buddy Allocator Function References (GPU buddy) +----------------------------------------------- + +.. kernel-doc:: drivers/gpu/buddy.c + :export: + +DRM Buddy Specific Logging Function References +---------------------------------------------- .. kernel-doc:: drivers/gpu/drm/drm_buddy.c :export: diff --git a/Documentation/netlink/specs/rt-link.yaml b/Documentation/netlink/specs/rt-link.yaml index ae4db7e03277..692b3cd47c31 100644 --- a/Documentation/netlink/specs/rt-link.yaml +++ b/Documentation/netlink/specs/rt-link.yaml @@ -1564,31 +1564,31 @@ attribute-sets: type: u32 - name: mode - type: flag + type: u8 - name: guard - type: flag + type: u8 - name: protect - type: flag + type: u8 - name: fast-leave - type: flag + type: u8 - name: learning - type: flag + type: u8 - name: unicast-flood - type: flag + type: u8 - name: proxyarp - type: flag + type: u8 - name: learning-sync - type: flag + type: u8 - name: proxyarp-wifi - type: flag + type: u8 - name: root-id type: binary @@ -1635,34 +1635,34 @@ attribute-sets: type: pad - name: mcast-flood - type: flag + type: u8 - name: mcast-to-ucast - type: flag + type: u8 - name: vlan-tunnel - type: flag + type: u8 - name: bcast-flood - type: flag + type: u8 - name: group-fwd-mask type: u16 - name: neigh-suppress - type: flag + type: u8 - name: isolated - type: flag + type: u8 - name: backup-port type: u32 - name: mrp-ring-open - type: flag + type: u8 - name: mrp-in-open - type: flag + type: u8 - name: mcast-eht-hosts-limit type: u32 @@ -1671,10 +1671,10 @@ attribute-sets: type: u32 - name: locked - type: flag + type: u8 - name: mab - type: flag + type: u8 - name: mcast-n-groups type: u32 @@ -1683,7 +1683,7 @@ attribute-sets: type: u32 - name: neigh-vlan-suppress - type: flag + type: u8 - name: backup-nhid type: u32 diff --git a/MAINTAINERS b/MAINTAINERS index cbfe2a7dd993..d27beeb0d9b8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8656,6 +8656,19 @@ T: git https://gitlab.freedesktop.org/drm/misc/kernel.git F: drivers/gpu/drm/ttm/ F: include/drm/ttm/ +GPU BUDDY ALLOCATOR +M: Matthew Auld +M: Arun Pravin +R: Christian Koenig +L: dri-devel@lists.freedesktop.org +S: Maintained +T: git https://gitlab.freedesktop.org/drm/misc/kernel.git +F: drivers/gpu/drm_buddy.c +F: drivers/gpu/buddy.c +F: drivers/gpu/tests/gpu_buddy_test.c +F: include/linux/gpu_buddy.h +F: include/drm/drm_buddy.h + DRM AUTOMATED TESTING M: Helen Koike M: Vignesh Raman diff --git a/Makefile b/Makefile index f3c450a0a2fb..05ccceb3a87c 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 6 PATCHLEVEL = 18 -SUBLEVEL = 41 +SUBLEVEL = 42 EXTRAVERSION = NAME = Baby Opossum Posse @@ -470,6 +470,10 @@ KBUILD_USERLDFLAGS := $(USERLDFLAGS) # These flags apply to all Rust code in the tree, including the kernel and # host programs. +# +# `-Aclippy::unwrap_or_default`: the lint is buggy [1] and ignores our +# MSRV. It can trigger depending on the optimization level. +# [1] https://github.com/rust-lang/rust-clippy/issues/17379 export rust_common_flags := --edition=2021 \ -Zbinary_dep_depinfo=y \ -Astable_features \ @@ -497,6 +501,7 @@ export rust_common_flags := --edition=2021 \ -Aclippy::uninlined_format_args \ -Wclippy::unnecessary_safety_comment \ -Wclippy::unnecessary_safety_doc \ + -Aclippy::unwrap_or_default \ -Wrustdoc::missing_crate_level_docs \ -Wrustdoc::unescaped_backticks diff --git a/arch/arm64/boot/dts/nvidia/tegra234.dtsi b/arch/arm64/boot/dts/nvidia/tegra234.dtsi index 5b245977d6a0..345587fa1d9a 100644 --- a/arch/arm64/boot/dts/nvidia/tegra234.dtsi +++ b/arch/arm64/boot/dts/nvidia/tegra234.dtsi @@ -5345,7 +5345,7 @@ cpus { #size-cells = <0>; cpu0_0: cpu@0 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x00000>; @@ -5364,7 +5364,7 @@ cpu0_0: cpu@0 { }; cpu0_1: cpu@100 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x00100>; @@ -5383,7 +5383,7 @@ cpu0_1: cpu@100 { }; cpu0_2: cpu@200 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x00200>; @@ -5402,7 +5402,7 @@ cpu0_2: cpu@200 { }; cpu0_3: cpu@300 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x00300>; @@ -5421,7 +5421,7 @@ cpu0_3: cpu@300 { }; cpu1_0: cpu@10000 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x10000>; @@ -5440,7 +5440,7 @@ cpu1_0: cpu@10000 { }; cpu1_1: cpu@10100 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x10100>; @@ -5459,7 +5459,7 @@ cpu1_1: cpu@10100 { }; cpu1_2: cpu@10200 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x10200>; @@ -5478,7 +5478,7 @@ cpu1_2: cpu@10200 { }; cpu1_3: cpu@10300 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x10300>; @@ -5497,7 +5497,7 @@ cpu1_3: cpu@10300 { }; cpu2_0: cpu@20000 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x20000>; @@ -5516,7 +5516,7 @@ cpu2_0: cpu@20000 { }; cpu2_1: cpu@20100 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x20100>; @@ -5535,7 +5535,7 @@ cpu2_1: cpu@20100 { }; cpu2_2: cpu@20200 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x20200>; @@ -5554,7 +5554,7 @@ cpu2_2: cpu@20200 { }; cpu2_3: cpu@20300 { - compatible = "arm,cortex-a78"; + compatible = "arm,cortex-a78ae"; device_type = "cpu"; reg = <0x20300>; diff --git a/arch/arm64/boot/dts/nvidia/tegra264.dtsi b/arch/arm64/boot/dts/nvidia/tegra264.dtsi index 872a69553e3c..b16d380209a5 100644 --- a/arch/arm64/boot/dts/nvidia/tegra264.dtsi +++ b/arch/arm64/boot/dts/nvidia/tegra264.dtsi @@ -50,7 +50,7 @@ timer@8000000 { }; gpcdma: dma-controller@8400000 { - compatible = "nvidia,tegra264-gpcdma", "nvidia,tegra186-gpcdma"; + compatible = "nvidia,tegra264-gpcdma"; reg = <0x0 0x08400000 0x0 0x210000>; interrupts = , , diff --git a/arch/arm64/boot/dts/qcom/lemans.dtsi b/arch/arm64/boot/dts/qcom/lemans.dtsi index 5c00d2e89fb8..496e319d7b11 100644 --- a/arch/arm64/boot/dts/qcom/lemans.dtsi +++ b/arch/arm64/boot/dts/qcom/lemans.dtsi @@ -5146,8 +5146,8 @@ port@1 { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -5234,8 +5234,8 @@ port@1 { dp1_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sc7180.dtsi b/arch/arm64/boot/dts/qcom/sc7180.dtsi index 892b3d2f1bf9..dc96d3d5e943 100644 --- a/arch/arm64/boot/dts/qcom/sc7180.dtsi +++ b/arch/arm64/boot/dts/qcom/sc7180.dtsi @@ -3452,8 +3452,8 @@ port@1 { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sc7280.dtsi b/arch/arm64/boot/dts/qcom/sc7280.dtsi index ed87bcb0d4b6..0f566dd75c96 100644 --- a/arch/arm64/boot/dts/qcom/sc7280.dtsi +++ b/arch/arm64/boot/dts/qcom/sc7280.dtsi @@ -5196,8 +5196,8 @@ port@1 { edp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -5295,8 +5295,8 @@ mdss_dp_out: endpoint { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sc8180x.dtsi b/arch/arm64/boot/dts/qcom/sc8180x.dtsi index e20a096aac7f..c6d96022c666 100644 --- a/arch/arm64/boot/dts/qcom/sc8180x.dtsi +++ b/arch/arm64/boot/dts/qcom/sc8180x.dtsi @@ -3303,8 +3303,8 @@ mdss_dp0_out: endpoint { dp0_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -3385,8 +3385,8 @@ mdss_dp1_out: endpoint { dp1_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -3461,8 +3461,8 @@ mdss_edp_out: endpoint { edp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi index d89938e17e09..40649fe7a509 100644 --- a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi +++ b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi @@ -4754,8 +4754,8 @@ mdss0_dp0_out: endpoint { mdss0_dp0_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -4836,8 +4836,8 @@ mdss0_dp1_out: endpoint { mdss0_dp1_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -4916,8 +4916,8 @@ mdss0_dp2_out: endpoint { mdss0_dp2_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -4991,8 +4991,8 @@ mdss0_dp3_out: endpoint { mdss0_dp3_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -6110,8 +6110,8 @@ mdss1_dp0_out: endpoint { mdss1_dp0_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -6190,8 +6190,8 @@ mdss1_dp1_out: endpoint { mdss1_dp1_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -6270,8 +6270,8 @@ mdss1_dp2_out: endpoint { mdss1_dp2_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; @@ -6345,8 +6345,8 @@ mdss1_dp3_out: endpoint { mdss1_dp3_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sm6350.dtsi b/arch/arm64/boot/dts/qcom/sm6350.dtsi index 19a7b9f9ea8b..0c6787d46ddc 100644 --- a/arch/arm64/boot/dts/qcom/sm6350.dtsi +++ b/arch/arm64/boot/dts/qcom/sm6350.dtsi @@ -2306,8 +2306,8 @@ mdss_dp_out: endpoint { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sm8150.dtsi b/arch/arm64/boot/dts/qcom/sm8150.dtsi index acdba79612aa..e1f480e3ed0b 100644 --- a/arch/arm64/boot/dts/qcom/sm8150.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8150.dtsi @@ -3939,8 +3939,8 @@ mdss_dp_out: endpoint { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sm8250.dtsi b/arch/arm64/boot/dts/qcom/sm8250.dtsi index adf6b49b52c4..79ba2c0c96b7 100644 --- a/arch/arm64/boot/dts/qcom/sm8250.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8250.dtsi @@ -4824,8 +4824,8 @@ mdss_dp_out: endpoint { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi index fc4ce9d4977e..af892b76322f 100644 --- a/arch/arm64/boot/dts/qcom/sm8350.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi @@ -2925,8 +2925,8 @@ mdss_dp_out: endpoint { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi index 2d49df7b69e4..991f3cbde1c8 100644 --- a/arch/arm64/boot/dts/qcom/sm8450.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi @@ -3483,8 +3483,8 @@ mdss_dp0_out: endpoint { dp_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; + opp-162000000 { + opp-hz = /bits/ 64 <162000000>; required-opps = <&rpmhpd_opp_low_svs>; }; diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi index ee9c12600f95..adb18c10dbd9 100644 --- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi +++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi @@ -5522,18 +5522,18 @@ mdss_dp0_out: endpoint { mdss_dp0_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; - required-opps = <&rpmhpd_opp_low_svs>; - }; - opp-270000000 { opp-hz = /bits/ 64 <270000000>; - required-opps = <&rpmhpd_opp_svs>; + required-opps = <&rpmhpd_opp_low_svs>; }; opp-540000000 { opp-hz = /bits/ 64 <540000000>; + required-opps = <&rpmhpd_opp_svs>; + }; + + opp-594000000 { + opp-hz = /bits/ 64 <594000000>; required-opps = <&rpmhpd_opp_svs_l1>; }; @@ -5574,7 +5574,7 @@ mdss_dp1: displayport-controller@ae98000 { <&usb_1_ss1_qmpphy QMP_USB43DP_DP_VCO_DIV_CLK>, <&usb_1_ss1_qmpphy QMP_USB43DP_DP_VCO_DIV_CLK>; - operating-points-v2 = <&mdss_dp1_opp_table>; + operating-points-v2 = <&mdss_dp0_opp_table>; power-domains = <&rpmhpd RPMHPD_MMCX>; @@ -5606,30 +5606,6 @@ mdss_dp1_out: endpoint { }; }; }; - - mdss_dp1_opp_table: opp-table { - compatible = "operating-points-v2"; - - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; - required-opps = <&rpmhpd_opp_low_svs>; - }; - - opp-270000000 { - opp-hz = /bits/ 64 <270000000>; - required-opps = <&rpmhpd_opp_svs>; - }; - - opp-540000000 { - opp-hz = /bits/ 64 <540000000>; - required-opps = <&rpmhpd_opp_svs_l1>; - }; - - opp-810000000 { - opp-hz = /bits/ 64 <810000000>; - required-opps = <&rpmhpd_opp_nom>; - }; - }; }; mdss_dp2: displayport-controller@ae9a000 { @@ -5662,7 +5638,7 @@ mdss_dp2: displayport-controller@ae9a000 { <&usb_1_ss2_qmpphy QMP_USB43DP_DP_VCO_DIV_CLK>, <&usb_1_ss2_qmpphy QMP_USB43DP_DP_VCO_DIV_CLK>; - operating-points-v2 = <&mdss_dp2_opp_table>; + operating-points-v2 = <&mdss_dp0_opp_table>; power-domains = <&rpmhpd RPMHPD_MMCX>; @@ -5693,30 +5669,6 @@ mdss_dp2_out: endpoint { }; }; }; - - mdss_dp2_opp_table: opp-table { - compatible = "operating-points-v2"; - - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; - required-opps = <&rpmhpd_opp_low_svs>; - }; - - opp-270000000 { - opp-hz = /bits/ 64 <270000000>; - required-opps = <&rpmhpd_opp_svs>; - }; - - opp-540000000 { - opp-hz = /bits/ 64 <540000000>; - required-opps = <&rpmhpd_opp_svs_l1>; - }; - - opp-810000000 { - opp-hz = /bits/ 64 <810000000>; - required-opps = <&rpmhpd_opp_nom>; - }; - }; }; mdss_dp3: displayport-controller@aea0000 { @@ -5779,19 +5731,14 @@ mdss_dp3_out: endpoint { mdss_dp3_opp_table: opp-table { compatible = "operating-points-v2"; - opp-160000000 { - opp-hz = /bits/ 64 <160000000>; - required-opps = <&rpmhpd_opp_low_svs>; - }; - opp-270000000 { opp-hz = /bits/ 64 <270000000>; - required-opps = <&rpmhpd_opp_svs>; + required-opps = <&rpmhpd_opp_low_svs>; }; - opp-540000000 { - opp-hz = /bits/ 64 <540000000>; - required-opps = <&rpmhpd_opp_svs_l1>; + opp-594000000 { + opp-hz = /bits/ 64 <594000000>; + required-opps = <&rpmhpd_opp_svs>; }; opp-810000000 { diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index e1deed824464..31058567eee4 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -130,7 +130,7 @@ * Annoyingly, the negative levels for Address size faults aren't laid out * contiguously (or in the desired order) */ -#define ESR_ELx_FSC_ADDRSZ_nL(n) ((n) == -1 ? 0x25 : 0x2C) +#define ESR_ELx_FSC_ADDRSZ_nL(n) ((n) == -1 ? 0x29 : 0x2C) #define ESR_ELx_FSC_ADDRSZ_L(n) ((n) < 0 ? ESR_ELx_FSC_ADDRSZ_nL(n) : \ (ESR_ELx_FSC_ADDRSZ + (n))) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 1d90a7e75333..fa63c96dbf79 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -91,7 +91,7 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, p4dp = p4d_offset(pgdp, addr); pudp = pud_offset(p4dp, addr); pmdp = pmd_offset(pudp, addr); - if ((pte_t *)pmdp == ptep) { + if ((pte_t *)PTR_ALIGN_DOWN(pmdp, sizeof(*pmdp) * CONT_PMDS) == ptep) { *pgsize = PMD_SIZE; return CONT_PMDS; } diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c index 88ad24d1e729..fb9a167ee7a0 100644 --- a/arch/loongarch/kernel/acpi.c +++ b/arch/loongarch/kernel/acpi.c @@ -201,10 +201,12 @@ static void __init acpi_process_madt(void) } int pptt_enabled; +static int acpi_nr_packages; +static int acpi_package_ids[MAX_PACKAGES]; int __init parse_acpi_topology(void) { - int cpu, topology_id; + int i, cpu, topology_id; for_each_possible_cpu(cpu) { topology_id = find_acpi_cpu_topology(cpu, 0); @@ -222,6 +224,29 @@ int __init parse_acpi_topology(void) cpu_data[cpu].core = topology_id; } + + topology_id = find_acpi_cpu_topology_package(cpu); + if (topology_id < 0) { + pr_warn("Invalid BIOS PPTT\n"); + return -ENOENT; + } + + for (i = 0; i < acpi_nr_packages; i++) + if (acpi_package_ids[i] == topology_id) + break; + + if (i == acpi_nr_packages) + acpi_package_ids[acpi_nr_packages++] = topology_id; + + cpu_data[cpu].package = topology_id; + } + + for_each_possible_cpu(cpu) { + for (i = 0; i < acpi_nr_packages; i++) + if (cpu_data[cpu].package == acpi_package_ids[i]) { + cpu_data[cpu].package = i; /* Canonicalize */ + break; + } } pptt_enabled = 1; diff --git a/arch/loongarch/kernel/kgdb.c b/arch/loongarch/kernel/kgdb.c index 7be5b4c0c900..6728b2d67dd3 100644 --- a/arch/loongarch/kernel/kgdb.c +++ b/arch/loongarch/kernel/kgdb.c @@ -252,7 +252,8 @@ static int kgdb_loongarch_notify(struct notifier_block *self, unsigned long cmd, if (atomic_read(&kgdb_active) != -1) kgdb_nmicallback(smp_processor_id(), regs); - if (kgdb_handle_exception(args->trapnr, args->signr, cmd, regs)) + if (kgdb_handle_exception(regs->csr_era == stepped_address ? 0 : args->trapnr, + args->signr, cmd, regs)) return NOTIFY_DONE; if (atomic_read(&kgdb_setting_breakpoint)) diff --git a/arch/loongarch/kernel/machine_kexec.c b/arch/loongarch/kernel/machine_kexec.c index d7fafda1d541..1883cae93bc3 100644 --- a/arch/loongarch/kernel/machine_kexec.c +++ b/arch/loongarch/kernel/machine_kexec.c @@ -42,6 +42,7 @@ static unsigned long first_ind_entry; int machine_kexec_prepare(struct kimage *kimage) { int i; + char head[8]; char *bootloader = "kexec"; void *cmdline_ptr = (void *)KEXEC_CMDLINE_ADDR; @@ -59,7 +60,9 @@ int machine_kexec_prepare(struct kimage *kimage) } else { /* Find the command line */ for (i = 0; i < kimage->nr_segments; i++) { - if (!strncmp(bootloader, (char __user *)kimage->segment[i].buf, strlen(bootloader))) { + if (copy_from_user(head, kimage->segment[i].buf, strlen(bootloader))) + continue; + if (!strncmp(bootloader, head, strlen(bootloader))) { if (!copy_from_user(cmdline_ptr, kimage->segment[i].buf, COMMAND_LINE_SIZE)) kimage->arch.cmdline_ptr = (unsigned long)cmdline_ptr; break; diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c index 75e3dfa06055..c0b59b813f8e 100644 --- a/arch/loongarch/kernel/setup.c +++ b/arch/loongarch/kernel/setup.c @@ -610,6 +610,7 @@ void __init setup_arch(char **cmdline_p) memblock_init(); pagetable_init(); bootcmdline_init(cmdline_p); + jump_label_init(); /* Initialise the static keys for early params */ parse_early_param(); reserve_initrd_mem(); @@ -617,8 +618,6 @@ void __init setup_arch(char **cmdline_p) arch_mem_init(cmdline_p); resource_init(); - jump_label_init(); /* Initialise the static keys for paravirtualization */ - #ifdef CONFIG_SMP plat_smp_setup(); prefill_possible_map(); diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c index 5d6a60294fb2..fc3817bd1919 100644 --- a/arch/loongarch/kernel/smp.c +++ b/arch/loongarch/kernel/smp.c @@ -412,10 +412,10 @@ void loongson_init_secondary(void) numa_add_cpu(cpu); #endif per_cpu(cpu_state, cpu) = CPU_ONLINE; - cpu_data[cpu].package = - cpu_logical_map(cpu) / loongson_sysconf.cores_per_package; cpu_data[cpu].core = pptt_enabled ? cpu_data[cpu].core : cpu_logical_map(cpu) % loongson_sysconf.cores_per_package; + cpu_data[cpu].package = pptt_enabled ? cpu_data[cpu].package : + cpu_logical_map(cpu) / loongson_sysconf.cores_per_package; cpu_data[cpu].global_id = cpu_logical_map(cpu); } diff --git a/arch/powerpc/kernel/time.c b/arch/powerpc/kernel/time.c index b4472288e0d4..11145c40183d 100644 --- a/arch/powerpc/kernel/time.c +++ b/arch/powerpc/kernel/time.c @@ -376,6 +376,47 @@ void vtime_task_switch(struct task_struct *prev) acct->starttime = acct0->starttime; } } + +/** + * vtime_reset - Fast forward vtime entry clocks + * + * Called from dynticks idle IRQ entry to fast-forward the clocks to current time + * so that the IRQ time is still accounted by vtime while nohz cputime is paused. + */ +void vtime_reset(void) +{ + struct cpu_accounting_data *acct = get_accounting(current); + + acct->starttime = mftb(); +#ifdef CONFIG_ARCH_HAS_SCALED_CPUTIME + acct->startspurr = read_spurr(acct->starttime); +#endif +} + +#ifdef CONFIG_NO_HZ_COMMON +/** + * vtime_dyntick_start - Inform vtime about entry to idle-dynticks + * + * Called when idle enters in dyntick mode. The idle cputime that elapsed so far + * is accumulated and the tick subsystem takes over the idle cputime accounting. + */ +void vtime_dyntick_start(void) +{ + vtime_account_idle(current); +} + +/** + * vtime_dyntick_stop - Inform vtime about exit from idle-dynticks + * + * Called when idle exits from dyntick mode. The vtime entry clocks are + * fast-forward to current time so that idle accounting restarts elapsing from + * now. + */ +void vtime_dyntick_stop(void) +{ + vtime_reset(); +} +#endif /* CONFIG_NO_HZ_COMMON */ #endif /* CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ void __no_kcsan __delay(unsigned long loops) @@ -892,6 +933,7 @@ static void __init set_decrementer_max(void) static void __init init_decrementer_clockevent(void) { register_decrementer_clockevent(smp_processor_id()); + vtime_reset(); } void secondary_cpu_time_init(void) @@ -907,6 +949,7 @@ void secondary_cpu_time_init(void) /* FIME: Should make unrelated change to move snapshot_timebase * call here ! */ register_decrementer_clockevent(smp_processor_id()); + vtime_reset(); } /* diff --git a/arch/powerpc/platforms/85xx/common.c b/arch/powerpc/platforms/85xx/common.c index 757811155587..c11deb2f50ed 100644 --- a/arch/powerpc/platforms/85xx/common.c +++ b/arch/powerpc/platforms/85xx/common.c @@ -42,6 +42,8 @@ static const struct of_device_id mpc85xx_common_ids[] __initconst = { { .compatible = "fsl,qoriq-pcie-v2.3", }, { .compatible = "fsl,qoriq-pcie-v2.2", }, { .compatible = "fsl,fman", }, + /* IFC NAND and NOR controllers */ + { .compatible = "fsl,ifc", }, {}, }; diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h index 4d794573e3db..fd3ba183cd82 100644 --- a/arch/riscv/include/asm/kvm_host.h +++ b/arch/riscv/include/asm/kvm_host.h @@ -220,13 +220,13 @@ struct kvm_vcpu_arch { /* * VCPU interrupts * - * We have a lockless approach for tracking pending VCPU interrupts - * implemented using atomic bitops. The irqs_pending bitmap represent - * pending interrupts whereas irqs_pending_mask represent bits changed - * in irqs_pending. Our approach is modeled around multiple producer - * and single consumer problem where the consumer is the VCPU itself. + * The irqs_pending bitmap represents pending interrupts whereas + * irqs_pending_mask represents bits changed in irqs_pending. Updates + * to these bitmaps are serialized so vcpu interrupt sync/flush cannot + * drop a newly injected interrupt while syncing guest-visible HVIP. */ #define KVM_RISCV_VCPU_NR_IRQS 64 + raw_spinlock_t irqs_pending_lock; DECLARE_BITMAP(irqs_pending, KVM_RISCV_VCPU_NR_IRQS); DECLARE_BITMAP(irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS); diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index 199d13f86f31..460fb297672e 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -409,6 +409,7 @@ static int hwprobe_get_cpus(struct riscv_hwprobe __user *pairs, if (cpusetsize > cpumask_size()) cpusetsize = cpumask_size(); + cpumask_clear(&cpus); ret = copy_from_user(&cpus, cpus_user, cpusetsize); if (ret) return -EFAULT; diff --git a/arch/riscv/kvm/aia.c b/arch/riscv/kvm/aia.c index dad318185660..b8b506082155 100644 --- a/arch/riscv/kvm/aia.c +++ b/arch/riscv/kvm/aia.c @@ -50,12 +50,15 @@ void kvm_riscv_vcpu_aia_flush_interrupts(struct kvm_vcpu *vcpu) struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; unsigned long mask, val; + lockdep_assert_held(&vcpu->arch.irqs_pending_lock); + if (!kvm_riscv_aia_available()) return; - if (READ_ONCE(vcpu->arch.irqs_pending_mask[1])) { - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[1], 0); - val = READ_ONCE(vcpu->arch.irqs_pending[1]) & mask; + mask = vcpu->arch.irqs_pending_mask[1]; + if (mask) { + vcpu->arch.irqs_pending_mask[1] = 0; + val = vcpu->arch.irqs_pending[1] & mask; csr->hviph &= ~mask; csr->hviph |= val; @@ -66,6 +69,8 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu) { struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; + lockdep_assert_held(&vcpu->arch.irqs_pending_lock); + if (kvm_riscv_aia_available()) csr->vsieh = ncsr_read(CSR_VSIEH); } @@ -74,13 +79,22 @@ void kvm_riscv_vcpu_aia_sync_interrupts(struct kvm_vcpu *vcpu) bool kvm_riscv_vcpu_aia_has_interrupts(struct kvm_vcpu *vcpu, u64 mask) { unsigned long seip; +#ifdef CONFIG_32BIT + unsigned long flags; + bool pending; +#endif if (!kvm_riscv_aia_available()) return false; #ifdef CONFIG_32BIT - if (READ_ONCE(vcpu->arch.irqs_pending[1]) & - (vcpu->arch.aia_context.guest_csr.vsieh & upper_32_bits(mask))) + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); + pending = vcpu->arch.irqs_pending[1] & + (vcpu->arch.aia_context.guest_csr.vsieh & + upper_32_bits(mask)); + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags); + + if (pending) return true; #endif @@ -198,6 +212,9 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu, unsigned long val) { struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; +#ifdef CONFIG_32BIT + unsigned long flags; +#endif if (reg_num >= sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long)) return -ENOENT; @@ -206,8 +223,12 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu, ((unsigned long *)csr)[reg_num] = val; #ifdef CONFIG_32BIT - if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) - WRITE_ONCE(vcpu->arch.irqs_pending_mask[1], 0); + if (reg_num == KVM_REG_RISCV_CSR_AIA_REG(siph)) { + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); + vcpu->arch.irqs_pending_mask[1] = 0; + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, + flags); + } #endif } diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c index d26c4967c20e..6f8e9b105da7 100644 --- a/arch/riscv/kvm/vcpu.c +++ b/arch/riscv/kvm/vcpu.c @@ -78,6 +78,7 @@ static void kvm_riscv_vcpu_context_reset(struct kvm_vcpu *vcpu, static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset) { + unsigned long flags; bool loaded; /** @@ -102,8 +103,10 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu, bool kvm_sbi_reset) kvm_riscv_vcpu_aia_reset(vcpu); + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); bitmap_zero(vcpu->arch.irqs_pending, KVM_RISCV_VCPU_NR_IRQS); bitmap_zero(vcpu->arch.irqs_pending_mask, KVM_RISCV_VCPU_NR_IRQS); + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags); kvm_riscv_vcpu_pmu_reset(vcpu); @@ -147,6 +150,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) /* Setup VCPU hfence queue */ spin_lock_init(&vcpu->arch.hfence_lock); + raw_spin_lock_init(&vcpu->arch.irqs_pending_lock); spin_lock_init(&vcpu->arch.reset_state.lock); @@ -348,10 +352,14 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu) { struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; unsigned long mask, val; + unsigned long flags; - if (READ_ONCE(vcpu->arch.irqs_pending_mask[0])) { - mask = xchg_acquire(&vcpu->arch.irqs_pending_mask[0], 0); - val = READ_ONCE(vcpu->arch.irqs_pending[0]) & mask; + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); + + mask = vcpu->arch.irqs_pending_mask[0]; + if (mask) { + vcpu->arch.irqs_pending_mask[0] = 0; + val = vcpu->arch.irqs_pending[0] & mask; csr->hvip &= ~mask; csr->hvip |= val; @@ -359,11 +367,14 @@ void kvm_riscv_vcpu_flush_interrupts(struct kvm_vcpu *vcpu) /* Flush AIA high interrupts */ kvm_riscv_vcpu_aia_flush_interrupts(vcpu); + + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags); } void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu) { unsigned long hvip; + unsigned long flags; struct kvm_vcpu_arch *v = &vcpu->arch; struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; @@ -372,34 +383,41 @@ void kvm_riscv_vcpu_sync_interrupts(struct kvm_vcpu *vcpu) /* Sync-up HVIP.VSSIP bit changes does by Guest */ hvip = ncsr_read(CSR_HVIP); + + raw_spin_lock_irqsave(&v->irqs_pending_lock, flags); + if ((csr->hvip ^ hvip) & (1UL << IRQ_VS_SOFT)) { if (hvip & (1UL << IRQ_VS_SOFT)) { - if (!test_and_set_bit(IRQ_VS_SOFT, - v->irqs_pending_mask)) - set_bit(IRQ_VS_SOFT, v->irqs_pending); + if (!__test_and_set_bit(IRQ_VS_SOFT, + v->irqs_pending_mask)) + __set_bit(IRQ_VS_SOFT, v->irqs_pending); } else { - if (!test_and_set_bit(IRQ_VS_SOFT, - v->irqs_pending_mask)) - clear_bit(IRQ_VS_SOFT, v->irqs_pending); + if (!__test_and_set_bit(IRQ_VS_SOFT, + v->irqs_pending_mask)) + __clear_bit(IRQ_VS_SOFT, v->irqs_pending); } } /* Sync up the HVIP.LCOFIP bit changes (only clear) by the guest */ if ((csr->hvip ^ hvip) & (1UL << IRQ_PMU_OVF)) { if (!(hvip & (1UL << IRQ_PMU_OVF)) && - !test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask)) - clear_bit(IRQ_PMU_OVF, v->irqs_pending); + !__test_and_set_bit(IRQ_PMU_OVF, v->irqs_pending_mask)) + __clear_bit(IRQ_PMU_OVF, v->irqs_pending); } /* Sync-up AIA high interrupts */ kvm_riscv_vcpu_aia_sync_interrupts(vcpu); + raw_spin_unlock_irqrestore(&v->irqs_pending_lock, flags); + /* Sync-up timer CSRs */ kvm_riscv_vcpu_timer_sync(vcpu); } int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq) { + unsigned long flags; + /* * We only allow VS-mode software, timer, and external * interrupts when irq is one of the local interrupts @@ -412,9 +430,10 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq) irq != IRQ_PMU_OVF) return -EINVAL; - set_bit(irq, vcpu->arch.irqs_pending); - smp_mb__before_atomic(); - set_bit(irq, vcpu->arch.irqs_pending_mask); + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); + __set_bit(irq, vcpu->arch.irqs_pending); + __set_bit(irq, vcpu->arch.irqs_pending_mask); + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags); kvm_vcpu_kick(vcpu); @@ -423,6 +442,8 @@ int kvm_riscv_vcpu_set_interrupt(struct kvm_vcpu *vcpu, unsigned int irq) int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq) { + unsigned long flags; + /* * We only allow VS-mode software, timer, counter overflow and external * interrupts when irq is one of the local interrupts @@ -435,26 +456,33 @@ int kvm_riscv_vcpu_unset_interrupt(struct kvm_vcpu *vcpu, unsigned int irq) irq != IRQ_PMU_OVF) return -EINVAL; - clear_bit(irq, vcpu->arch.irqs_pending); - smp_mb__before_atomic(); - set_bit(irq, vcpu->arch.irqs_pending_mask); + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); + __clear_bit(irq, vcpu->arch.irqs_pending); + __set_bit(irq, vcpu->arch.irqs_pending_mask); + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags); return 0; } bool kvm_riscv_vcpu_has_interrupts(struct kvm_vcpu *vcpu, u64 mask) { + unsigned long flags; unsigned long ie; + bool ret; + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); ie = ((vcpu->arch.guest_csr.vsie & VSIP_VALID_MASK) << VSIP_TO_HVIP_SHIFT) & (unsigned long)mask; ie |= vcpu->arch.guest_csr.vsie & ~IRQ_LOCAL_MASK & (unsigned long)mask; - if (READ_ONCE(vcpu->arch.irqs_pending[0]) & ie) - return true; + ret = vcpu->arch.irqs_pending[0] & ie; + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags); /* Check AIA high interrupts */ - return kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask); + if (!ret) + ret = kvm_riscv_vcpu_aia_has_interrupts(vcpu, mask); + + return ret; } void __kvm_riscv_vcpu_power_off(struct kvm_vcpu *vcpu) diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c index 865dae903aa0..3e8f9a283412 100644 --- a/arch/riscv/kvm/vcpu_onereg.c +++ b/arch/riscv/kvm/vcpu_onereg.c @@ -522,6 +522,7 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu, unsigned long reg_val) { struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; + unsigned long flags; if (reg_num >= sizeof(struct kvm_riscv_csr) / sizeof(unsigned long)) return -ENOENT; @@ -533,8 +534,11 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu, ((unsigned long *)csr)[reg_num] = reg_val; - if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) - WRITE_ONCE(vcpu->arch.irqs_pending_mask[0], 0); + if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) { + raw_spin_lock_irqsave(&vcpu->arch.irqs_pending_lock, flags); + vcpu->arch.irqs_pending_mask[0] = 0; + raw_spin_unlock_irqrestore(&vcpu->arch.irqs_pending_lock, flags); + } return 0; } diff --git a/arch/s390/kernel/time.c b/arch/s390/kernel/time.c index 63517b85f4c9..246fc5b936fa 100644 --- a/arch/s390/kernel/time.c +++ b/arch/s390/kernel/time.c @@ -66,6 +66,7 @@ ATOMIC_NOTIFIER_HEAD(s390_epoch_delta_notifier); EXPORT_SYMBOL(s390_epoch_delta_notifier); unsigned char ptff_function_mask[16]; +EXPORT_SYMBOL(ptff_function_mask); static unsigned long lpar_offset; static unsigned long initial_leap_seconds; diff --git a/arch/s390/lib/csum-partial.c b/arch/s390/lib/csum-partial.c index 458abd9bac70..9d74ceff136c 100644 --- a/arch/s390/lib/csum-partial.c +++ b/arch/s390/lib/csum-partial.c @@ -23,7 +23,7 @@ static __always_inline __wsum csum_copy(void *dst, const void *src, int len, __w if (!cpu_has_vx()) { if (copy) memcpy(dst, src, len); - return cksm(dst, len, sum); + return cksm(src, len, sum); } kernel_fpu_begin(&vxstate, KERNEL_VXR_V16V23); fpu_vlvgf(16, (__force u32)sum, 1); diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 2013840d6318..99d85503e509 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -27,6 +27,7 @@ targets := vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma \ KBUILD_CFLAGS := -m$(BITS) -O2 $(CLANG_FLAGS) KBUILD_CFLAGS += -std=gnu11 KBUILD_CFLAGS += -fno-strict-aliasing -fPIE +KBUILD_CFLAGS += -fno-jump-tables KBUILD_CFLAGS += -Wundef KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING cflags-$(CONFIG_X86_32) := -march=i386 diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 79a1f2d5f501..f9025779292a 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -363,7 +363,7 @@ static void ctrl_domain_free(struct rdt_hw_ctrl_domain *hw_dom) kfree(hw_dom); } -static void mon_domain_free(struct rdt_hw_mon_domain *hw_dom) +static void l3_mon_domain_free(struct rdt_hw_l3_mon_domain *hw_dom) { int idx; @@ -396,11 +396,13 @@ static int domain_setup_ctrlval(struct rdt_resource *r, struct rdt_ctrl_domain * } /** - * arch_domain_mbm_alloc() - Allocate arch private storage for the MBM counters + * l3_mon_domain_mbm_alloc() - Allocate arch private storage for the MBM counters * @num_rmid: The size of the MBM counter array * @hw_dom: The domain that owns the allocated arrays + * + * Return: 0 for success, or -ENOMEM. */ -static int arch_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_mon_domain *hw_dom) +static int l3_mon_domain_mbm_alloc(u32 num_rmid, struct rdt_hw_l3_mon_domain *hw_dom) { size_t tsize = sizeof(*hw_dom->arch_mbm_states[0]); enum resctrl_event_id eventid; @@ -499,9 +501,9 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r) { int id = get_domain_id_from_scope(cpu, r->mon_scope); struct list_head *add_pos = NULL; - struct rdt_hw_mon_domain *hw_dom; + struct rdt_hw_l3_mon_domain *hw_dom; struct rdt_domain_hdr *hdr; - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; struct cacheinfo *ci; int err; @@ -517,7 +519,7 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r) if (hdr) { if (WARN_ON_ONCE(hdr->type != RESCTRL_MON_DOMAIN)) return; - d = container_of(hdr, struct rdt_mon_domain, hdr); + d = container_of(hdr, struct rdt_l3_mon_domain, hdr); cpumask_set_cpu(cpu, &d->hdr.cpu_mask); /* Update the mbm_assign_mode state for the CPU if supported */ @@ -536,7 +538,7 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r) ci = get_cpu_cacheinfo_level(cpu, RESCTRL_L3_CACHE); if (!ci) { pr_warn_once("Can't find L3 cache for CPU:%d resource %s\n", cpu, r->name); - mon_domain_free(hw_dom); + l3_mon_domain_free(hw_dom); return; } d->ci_id = ci->id; @@ -548,8 +550,8 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r) arch_mon_domain_online(r, d); - if (arch_domain_mbm_alloc(r->mon.num_rmid, hw_dom)) { - mon_domain_free(hw_dom); + if (l3_mon_domain_mbm_alloc(r->mon.num_rmid, hw_dom)) { + l3_mon_domain_free(hw_dom); return; } @@ -559,7 +561,7 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r) if (err) { list_del_rcu(&d->hdr.list); synchronize_rcu(); - mon_domain_free(hw_dom); + l3_mon_domain_free(hw_dom); } } @@ -620,9 +622,9 @@ static void domain_remove_cpu_ctrl(int cpu, struct rdt_resource *r) static void domain_remove_cpu_mon(int cpu, struct rdt_resource *r) { int id = get_domain_id_from_scope(cpu, r->mon_scope); - struct rdt_hw_mon_domain *hw_dom; + struct rdt_hw_l3_mon_domain *hw_dom; struct rdt_domain_hdr *hdr; - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; lockdep_assert_held(&domain_list_lock); @@ -642,7 +644,7 @@ static void domain_remove_cpu_mon(int cpu, struct rdt_resource *r) if (WARN_ON_ONCE(hdr->type != RESCTRL_MON_DOMAIN)) return; - d = container_of(hdr, struct rdt_mon_domain, hdr); + d = container_of(hdr, struct rdt_l3_mon_domain, hdr); hw_dom = resctrl_to_arch_mon_dom(d); cpumask_clear_cpu(cpu, &d->hdr.cpu_mask); @@ -650,7 +652,7 @@ static void domain_remove_cpu_mon(int cpu, struct rdt_resource *r) resctrl_offline_mon_domain(r, d); list_del_rcu(&d->hdr.list); synchronize_rcu(); - mon_domain_free(hw_dom); + l3_mon_domain_free(hw_dom); return; } @@ -897,7 +899,7 @@ static __init bool get_rdt_mon_resources(void) if (!ret) return false; - return !rdt_get_mon_l3_config(r); + return !rdt_get_l3_mon_config(r); } static __init void __check_quirks_intel(void) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 6da9bd1a188b..6c9e75440c10 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -63,17 +63,17 @@ struct rdt_hw_ctrl_domain { }; /** - * struct rdt_hw_mon_domain - Arch private attributes of a set of CPUs that share - * a resource for a monitor function - * @d_resctrl: Properties exposed to the resctrl file system + * struct rdt_hw_l3_mon_domain - Arch private attributes of a set of CPUs sharing + * RDT_RESOURCE_L3 monitoring + * @d_resctrl: Properties exposed to the resctrl file system * @arch_mbm_states: Per-event pointer to the MBM event's saved state. * An MBM event's state is an array of struct arch_mbm_state * indexed by RMID on x86. * * Members of this structure are accessed via helpers that provide abstraction. */ -struct rdt_hw_mon_domain { - struct rdt_mon_domain d_resctrl; +struct rdt_hw_l3_mon_domain { + struct rdt_l3_mon_domain d_resctrl; struct arch_mbm_state *arch_mbm_states[QOS_NUM_L3_MBM_EVENTS]; }; @@ -82,9 +82,9 @@ static inline struct rdt_hw_ctrl_domain *resctrl_to_arch_ctrl_dom(struct rdt_ctr return container_of(r, struct rdt_hw_ctrl_domain, d_resctrl); } -static inline struct rdt_hw_mon_domain *resctrl_to_arch_mon_dom(struct rdt_mon_domain *r) +static inline struct rdt_hw_l3_mon_domain *resctrl_to_arch_mon_dom(struct rdt_l3_mon_domain *r) { - return container_of(r, struct rdt_hw_mon_domain, d_resctrl); + return container_of(r, struct rdt_hw_l3_mon_domain, d_resctrl); } /** @@ -138,7 +138,7 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *r extern struct rdt_hw_resource rdt_resources_all[]; -void arch_mon_domain_online(struct rdt_resource *r, struct rdt_mon_domain *d); +void arch_mon_domain_online(struct rdt_resource *r, struct rdt_l3_mon_domain *d); /* CPUID.(EAX=10H, ECX=ResID=1).EAX */ union cpuid_0x10_1_eax { @@ -211,7 +211,7 @@ union l3_qos_abmc_cfg { void rdt_ctrl_update(void *arg); -int rdt_get_mon_l3_config(struct rdt_resource *r); +int rdt_get_l3_mon_config(struct rdt_resource *r); bool rdt_cpu_has(int flag); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 4b2b539ddff9..9fc3adb4b232 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -109,7 +109,7 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val) * * In RMID sharing mode there are fewer "logical RMID" values available * to accumulate data ("physical RMIDs" are divided evenly between SNC - * nodes that share an L3 cache). Linux creates an rdt_mon_domain for + * nodes that share an L3 cache). Linux creates an rdt_l3_mon_domain for * each SNC node. * * The value loaded into IA32_PQR_ASSOC is the "logical RMID". @@ -157,7 +157,7 @@ static int __rmid_read_phys(u32 prmid, enum resctrl_event_id eventid, u64 *val) return 0; } -static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_mon_domain *hw_dom, +static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_l3_mon_domain *hw_dom, u32 rmid, enum resctrl_event_id eventid) { @@ -171,11 +171,11 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_mon_domain *hw_do return state ? &state[rmid] : NULL; } -void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d, +void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 unused, u32 rmid, enum resctrl_event_id eventid) { - struct rdt_hw_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); + struct rdt_hw_l3_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); int cpu = cpumask_any(&d->hdr.cpu_mask); struct arch_mbm_state *am; u32 prmid; @@ -194,9 +194,9 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d, * Assumes that hardware counters are also reset and thus that there is * no need to record initial non-zero counts. */ -void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_mon_domain *d) +void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_l3_mon_domain *d) { - struct rdt_hw_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); + struct rdt_hw_l3_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); enum resctrl_event_id eventid; int idx; @@ -217,10 +217,10 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) return chunks >> shift; } -static u64 get_corrected_val(struct rdt_resource *r, struct rdt_mon_domain *d, +static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 rmid, enum resctrl_event_id eventid, u64 msr_val) { - struct rdt_hw_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); + struct rdt_hw_l3_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); struct arch_mbm_state *am; u64 chunks; @@ -238,11 +238,11 @@ static u64 get_corrected_val(struct rdt_resource *r, struct rdt_mon_domain *d, return chunks * hw_res->mon_scale; } -int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_mon_domain *d, +int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 unused, u32 rmid, enum resctrl_event_id eventid, u64 *val, void *ignored) { - struct rdt_hw_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); + struct rdt_hw_l3_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); struct arch_mbm_state *am; u64 msr_val; u32 prmid; @@ -308,11 +308,11 @@ static int __cntr_id_read(u32 cntr_id, u64 *val) return 0; } -void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, +void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 unused, u32 rmid, int cntr_id, enum resctrl_event_id eventid) { - struct rdt_hw_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); + struct rdt_hw_l3_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); struct arch_mbm_state *am; am = get_arch_mbm_state(hw_dom, rmid, eventid); @@ -324,7 +324,7 @@ void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, } } -int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_mon_domain *d, +int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 unused, u32 rmid, int cntr_id, enum resctrl_event_id eventid, u64 *val) { @@ -354,7 +354,7 @@ int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_mon_domain *d, * must adjust RMID counter numbers based on SNC node. See * logical_rmid_to_physical_rmid() for code that does this. */ -void arch_mon_domain_online(struct rdt_resource *r, struct rdt_mon_domain *d) +void arch_mon_domain_online(struct rdt_resource *r, struct rdt_l3_mon_domain *d) { if (snc_nodes_per_l3_cache > 1) msr_clear_bit(MSR_RMID_SNC_CONFIG, 0); @@ -423,7 +423,7 @@ static __init int snc_get_config(void) return ret; } -int __init rdt_get_mon_l3_config(struct rdt_resource *r) +int __init rdt_get_l3_mon_config(struct rdt_resource *r) { unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset; struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); @@ -515,7 +515,7 @@ static void resctrl_abmc_set_one_amd(void *arg) */ static void _resctrl_abmc_enable(struct rdt_resource *r, bool enable) { - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; lockdep_assert_cpus_held(); @@ -554,11 +554,11 @@ static void resctrl_abmc_config_one_amd(void *info) /* * Send an IPI to the domain to assign the counter to RMID, event pair. */ -void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, +void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d, enum resctrl_event_id evtid, u32 rmid, u32 closid, u32 cntr_id, bool assign) { - struct rdt_hw_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); + struct rdt_hw_l3_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d); union l3_qos_abmc_cfg abmc_cfg = { 0 }; struct arch_mbm_state *am; diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index f895019a55e5..e6e747507fd3 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2065,7 +2065,7 @@ static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) if (apic_lvtt_tscdeadline(apic) || ktimer->hv_timer_in_use) ktimer->expired_tscdeadline = ktimer->tscdeadline; - if (!from_timer_fn && apic->apicv_active) { + if (!from_timer_fn && apic->apicv_active && vcpu->wants_to_run) { WARN_ON(kvm_get_running_vcpu() != vcpu); kvm_apic_inject_pending_timer_irqs(apic); return; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index eadeca9786f6..e95d96dccfe4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4806,16 +4806,17 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault if (r != RET_PF_CONTINUE) return r; - r = RET_PF_RETRY; write_lock(&vcpu->kvm->mmu_lock); - if (is_page_fault_stale(vcpu, fault)) - goto out_unlock; - r = make_mmu_pages_available(vcpu); if (r) goto out_unlock; + if (is_page_fault_stale(vcpu, fault)) { + r = RET_PF_RETRY; + goto out_unlock; + } + r = direct_map(vcpu, fault); out_unlock: @@ -7392,7 +7393,9 @@ void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen) static void mmu_destroy_caches(void) { kmem_cache_destroy(pte_list_desc_cache); + pte_list_desc_cache = NULL; kmem_cache_destroy(mmu_page_header_cache); + mmu_page_header_cache = NULL; } static void kvm_wake_nx_recovery_thread(struct kvm *kvm) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index ed762bb4b007..af220c9c8ab8 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -827,15 +827,17 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault } #endif - r = RET_PF_RETRY; write_lock(&vcpu->kvm->mmu_lock); - if (is_page_fault_stale(vcpu, fault)) - goto out_unlock; - r = make_mmu_pages_available(vcpu); if (r) goto out_unlock; + + if (is_page_fault_stale(vcpu, fault)) { + r = RET_PF_RETRY; + goto out_unlock; + } + r = FNAME(fetch)(vcpu, fault, &walker); out_unlock: diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index bb034896bf6d..36fad2f95bd2 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -523,7 +523,12 @@ static int svm_enable_virtualization_cpu(void) return -EBUSY; sd = per_cpu_ptr(&svm_data, me); - sd->asid_generation = 1; + /* + * Bump the current asid_generation value to ensure any vCPU that + * previously ran on this CPU sees a stale generation and is forced + * to acquire a new ASID, preventing a latent ASID collision. + */ + sd->asid_generation++; sd->max_asid = cpuid_ebx(SVM_CPUID_FUNC) - 1; sd->next_asid = sd->max_asid + 1; sd->min_asid = max_sev_asid + 1; diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index a5a525ae8e78..afb4be700f9a 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -333,6 +333,7 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) static void free_nested(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); + struct vmcs *shadow_vmcs; if (WARN_ON_ONCE(vmx->loaded_vmcs != &vmx->vmcs01)) vmx_switch_vmcs(vcpu, &vmx->vmcs01); @@ -350,9 +351,15 @@ static void free_nested(struct kvm_vcpu *vcpu) vmx->nested.current_vmptr = INVALID_GPA; if (enable_shadow_vmcs) { vmx_disable_shadow_vmcs(vmx); - vmcs_clear(vmx->vmcs01.shadow_vmcs); - free_vmcs(vmx->vmcs01.shadow_vmcs); + + /* + * Keep the pointer visible until after VMCLEAR, so migration + * can clear an active shadow VMCS on the old CPU. + */ + shadow_vmcs = vmx->vmcs01.shadow_vmcs; + vmcs_clear(shadow_vmcs); vmx->vmcs01.shadow_vmcs = NULL; + free_vmcs(shadow_vmcs); } kfree(vmx->nested.cached_vmcs12); vmx->nested.cached_vmcs12 = NULL; diff --git a/drivers/accel/amdxdna/amdxdna_ctx.c b/drivers/accel/amdxdna/amdxdna_ctx.c index cfee89681ff3..a49c059e6ed4 100644 --- a/drivers/accel/amdxdna/amdxdna_ctx.c +++ b/drivers/accel/amdxdna/amdxdna_ctx.c @@ -393,6 +393,7 @@ void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job) amdxdna_arg_bos_put(job); amdxdna_gem_put_obj(job->cmd_bo); dma_fence_put(job->fence); + mmdrop(job->mm); } int amdxdna_cmd_submit(struct amdxdna_client *client, @@ -405,6 +406,10 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, int ret, idx; XDNA_DBG(xdna, "Command BO hdl %d, Arg BO count %d", cmd_bo_hdl, arg_bo_cnt); + + if (!xdna->dev_info->ops->cmd_submit) + return -EOPNOTSUPP; + job = kzalloc(struct_size(job, bos, arg_bo_cnt), GFP_KERNEL); if (!job) return -ENOMEM; @@ -443,6 +448,7 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, job->hwctx = hwctx; job->mm = current->mm; + mmgrab(job->mm); job->fence = amdxdna_fence_create(hwctx); if (!job->fence) { @@ -475,6 +481,8 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, cmd_put: amdxdna_gem_put_obj(job->cmd_bo); free_job: + if (job->mm) + mmdrop(job->mm); kfree(job); return ret; } diff --git a/drivers/accel/ivpu/ivpu_hw_btrs.c b/drivers/accel/ivpu/ivpu_hw_btrs.c index aa33f562d29c..e9786204feca 100644 --- a/drivers/accel/ivpu/ivpu_hw_btrs.c +++ b/drivers/accel/ivpu/ivpu_hw_btrs.c @@ -845,7 +845,7 @@ static void diagnose_failure_mtl(struct ivpu_device *vdev) static void diagnose_failure_lnl(struct ivpu_device *vdev) { - u32 reg = REGB_RD32(VPU_HW_BTRS_MTL_INTERRUPT_STAT) & BTRS_LNL_IRQ_MASK; + u32 reg = REGB_RD32(VPU_HW_BTRS_LNL_INTERRUPT_STAT) & BTRS_LNL_IRQ_MASK; if (REG_TEST_FLD(VPU_HW_BTRS_LNL_INTERRUPT_STAT, ATS_ERR, reg)) { ivpu_err(vdev, "ATS_ERR_LOG1 0x%08x ATS_ERR_LOG2 0x%08x\n", diff --git a/drivers/android/binder/thread.rs b/drivers/android/binder/thread.rs index ad8bdf762ddc..93b550e6b60e 100644 --- a/drivers/android/binder/thread.rs +++ b/drivers/android/binder/thread.rs @@ -1268,14 +1268,14 @@ fn transaction(self: &Arc, cmd: u32, reader: &mut UserSliceReader) -> Resu inner.extended_error = ExtendedError::new(info.debug_id as u32, err.reply, source.to_errno()); } - } - pr_warn!( - "{}:{} transaction to {} failed: {err:?}", - info.from_pid, - info.from_tid, - info.to_pid - ); + pr_warn!( + "{}:{} transaction to {} failed: {err:?}", + info.from_pid, + info.from_tid, + info.to_pid + ); + } } } diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c index 22ce6accbc5b..f0753f5e57a8 100644 --- a/drivers/ata/libata-core.c +++ b/drivers/ata/libata-core.c @@ -2833,6 +2833,24 @@ static void ata_dev_config_cpr(struct ata_device *dev) if (!nr_cpr) goto out; + /* + * The device reports the number of CPR descriptors independently of the + * log size, and that count is also used to emit VPD page B9h into the + * fixed-size rbuf. Reject a count larger than what that buffer can hold + * (ATA_DEV_MAX_CPR) or larger than the log the device actually returned. + */ + if (nr_cpr > ATA_DEV_MAX_CPR) { + ata_dev_warn(dev, + "Too many concurrent positioning ranges\n"); + goto out; + } + + if (buf_len < 64 + (size_t)nr_cpr * 32) { + ata_dev_warn(dev, + "Invalid number of concurrent positioning ranges\n"); + goto out; + } + cpr_log = kzalloc(struct_size(cpr_log, cpr, nr_cpr), GFP_KERNEL); if (!cpr_log) goto out; diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c index 46ef1ef6f812..a7916b50cbfc 100644 --- a/drivers/ata/libata-scsi.c +++ b/drivers/ata/libata-scsi.c @@ -37,8 +37,6 @@ #include "libata.h" #include "libata-transport.h" -#define ATA_SCSI_RBUF_SIZE 2048 - static DEFINE_SPINLOCK(ata_scsi_rbuf_lock); static u8 ata_scsi_rbuf[ATA_SCSI_RBUF_SIZE]; diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h index 0002994ddfc9..56c4136b7417 100644 --- a/drivers/ata/libata.h +++ b/drivers/ata/libata.h @@ -144,6 +144,15 @@ static inline void ata_acpi_bind_dev(struct ata_device *dev) {} #endif /* libata-scsi.c */ +#define ATA_SCSI_RBUF_SIZE 2048 + +/* + * Maximum number of concurrent positioning ranges (CPR) supported. The ACS + * specifications allow up to 255, but we limit this to the number of CPR + * descriptors that fit in the rbuf buffer used to emit VPD page B9h. + */ +#define ATA_DEV_MAX_CPR min(255, ((ATA_SCSI_RBUF_SIZE - 64) / 32)) + extern struct ata_device *ata_scsi_find_dev(struct ata_port *ap, const struct scsi_device *scsidev); extern int ata_scsi_add_hosts(struct ata_host *host, diff --git a/drivers/ata/sata_dwc_460ex.c b/drivers/ata/sata_dwc_460ex.c index 7a4f59202156..36cc28b89865 100644 --- a/drivers/ata/sata_dwc_460ex.c +++ b/drivers/ata/sata_dwc_460ex.c @@ -19,7 +19,6 @@ #include #include #include -#include #include #include #include @@ -226,7 +225,6 @@ static int sata_dwc_dma_init_old(struct platform_device *pdev, struct sata_dwc_device *hsdev) { struct device *dev = &pdev->dev; - struct device_node *np = dev->of_node; hsdev->dma = devm_kzalloc(dev, sizeof(*hsdev->dma), GFP_KERNEL); if (!hsdev->dma) @@ -236,11 +234,9 @@ static int sata_dwc_dma_init_old(struct platform_device *pdev, hsdev->dma->id = pdev->id; /* Get SATA DMA interrupt number */ - hsdev->dma->irq = irq_of_parse_and_map(np, 1); - if (!hsdev->dma->irq) { - dev_err(dev, "no SATA DMA irq\n"); - return -ENODEV; - } + hsdev->dma->irq = platform_get_irq(pdev, 1); + if (hsdev->dma->irq < 0) + return hsdev->dma->irq; /* Get physical SATA DMA register base address */ hsdev->dma->regs = devm_platform_ioremap_resource(pdev, 1); @@ -398,8 +394,7 @@ static void clear_serror(struct ata_port *ap) static void clear_interrupt_bit(struct sata_dwc_device *hsdev, u32 bit) { - sata_dwc_writel(&hsdev->sata_dwc_regs->intpr, - sata_dwc_readl(&hsdev->sata_dwc_regs->intpr)); + sata_dwc_writel(&hsdev->sata_dwc_regs->intpr, bit); } static u32 qcmd_tag_to_mask(u8 tag) @@ -612,14 +607,9 @@ static irqreturn_t sata_dwc_isr(int irq, void *dev_instance) status = ap->ops->sff_check_status(ap); dev_dbg(ap->dev, "%s ATA status register=0x%x\n", __func__, status); - tag = 0; while (tag_mask) { - while (!(tag_mask & 0x00000001)) { - tag++; - tag_mask <<= 1; - } - - tag_mask &= (~0x00000001); + tag = __ffs(tag_mask); + tag_mask &= ~(1U << tag); qc = ata_qc_from_tag(ap, tag); if (unlikely(!qc)) { dev_err(ap->dev, "failed to get qc"); @@ -1125,7 +1115,6 @@ static const struct ata_port_info sata_dwc_port_info[] = { static int sata_dwc_probe(struct platform_device *ofdev) { struct device *dev = &ofdev->dev; - struct device_node *np = dev->of_node; struct sata_dwc_device *hsdev; u32 idr, versionr; char *ver = (char *)&versionr; @@ -1168,18 +1157,13 @@ static int sata_dwc_probe(struct platform_device *ofdev) /* Save dev for later use in dev_xxx() routines */ hsdev->dev = dev; - /* Enable SATA Interrupts */ - sata_dwc_enable_interrupts(hsdev); - /* Get SATA interrupt number */ - irq = irq_of_parse_and_map(np, 0); - if (!irq) { - dev_err(dev, "no SATA DMA irq\n"); - return -ENODEV; - } + irq = platform_get_irq(ofdev, 0); + if (irq < 0) + return irq; #ifdef CONFIG_SATA_DWC_OLD_DMA - if (!of_property_present(np, "dmas")) { + if (!of_property_present(dev->of_node, "dmas")) { err = sata_dwc_dma_init_old(ofdev, hsdev); if (err) return err; @@ -1203,6 +1187,8 @@ static int sata_dwc_probe(struct platform_device *ofdev) if (err) dev_err(dev, "failed to activate host"); + /* Enable SATA Interrupts */ + sata_dwc_enable_interrupts(hsdev); return 0; error_out: diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 97cd8cf7bad3..6c802f029f57 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -1957,9 +1957,14 @@ static int rbd_object_map_update_finish(struct rbd_obj_request *obj_req, bool has_current_state; void *p; - if (osd_req->r_result) + if (osd_req->r_result < 0) return osd_req->r_result; + /* + * Writes aren't allowed to return a data payload. + */ + WARN_ON_ONCE(osd_req->r_result > 0); + /* * Nothing to do for a snapshot object map. */ diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index c339222513b0..cb31e96f01cd 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -26,7 +27,6 @@ #include #include #include -#include #include #include #include @@ -230,7 +230,6 @@ struct ublk_device { struct ublk_params params; - struct completion completion; u32 nr_io_ready; bool unprivileged_daemons; struct mutex cancel_mutex; @@ -2150,9 +2149,13 @@ static void ublk_mark_io_ready(struct ublk_device *ub) ub->nr_io_ready++; if (ublk_dev_ready(ub)) { - /* now we are ready for handling ublk io request */ + /* + * now we are ready for handling ublk io request, clear + * device-level canceling flag and wake ublk_dev_ready() + * waiters + */ ublk_reset_io_flags(ub); - complete_all(&ub->completion); + wake_up_var(&ub->nr_io_ready); } } @@ -2829,7 +2832,6 @@ static int ublk_init_queues(struct ublk_device *ub) goto fail; } - init_completion(&ub->completion); return 0; fail: @@ -2966,6 +2968,26 @@ static bool ublk_validate_user_pid(struct ublk_device *ub, pid_t ublksrv_pid) return ub->ublksrv_tgid == ublksrv_pid; } +/* + * Wait until all queues have fetched their I/O commands, and return with + * ub->mutex held and readiness guaranteed: then every queue's ->canceling + * is cleared. Ready may regress between wakeup and mutex_lock() (daemon + * death), so re-check it under the mutex and wait again. + */ +static int ublk_wait_dev_ready_and_lock(struct ublk_device *ub) +{ + while (true) { + if (wait_var_event_interruptible(&ub->nr_io_ready, + ublk_dev_ready(ub))) + return -EINTR; + + mutex_lock(&ub->mutex); + if (ublk_dev_ready(ub)) + return 0; + mutex_unlock(&ub->mutex); + } +} + static int ublk_ctrl_start_dev(struct ublk_device *ub, const struct ublksrv_ctrl_cmd *header) { @@ -3031,13 +3053,13 @@ static int ublk_ctrl_start_dev(struct ublk_device *ub, lim.max_segments = ub->params.seg.max_segments; } - if (wait_for_completion_interruptible(&ub->completion) != 0) + if (ublk_wait_dev_ready_and_lock(ub)) return -EINTR; - if (!ublk_validate_user_pid(ub, ublksrv_pid)) - return -EINVAL; - - mutex_lock(&ub->mutex); + if (!ublk_validate_user_pid(ub, ublksrv_pid)) { + ret = -EINVAL; + goto out_unlock; + } if (ub->dev_info.state == UBLK_S_DEV_LIVE || test_bit(UB_STATE_USED, &ub->state)) { ret = -EEXIST; @@ -3549,7 +3571,6 @@ static int ublk_ctrl_start_recovery(struct ublk_device *ub, goto out_unlock; } pr_devel("%s: start recovery for dev id %d.\n", __func__, header->dev_id); - init_completion(&ub->completion); ret = 0; out_unlock: mutex_unlock(&ub->mutex); @@ -3565,16 +3586,17 @@ static int ublk_ctrl_end_recovery(struct ublk_device *ub, pr_devel("%s: Waiting for all FETCH_REQs, dev id %d...\n", __func__, header->dev_id); - if (wait_for_completion_interruptible(&ub->completion)) + if (ublk_wait_dev_ready_and_lock(ub)) return -EINTR; pr_devel("%s: All FETCH_REQs received, dev id %d\n", __func__, header->dev_id); - if (!ublk_validate_user_pid(ub, ublksrv_pid)) - return -EINVAL; + if (!ublk_validate_user_pid(ub, ublksrv_pid)) { + ret = -EINVAL; + goto out_unlock; + } - mutex_lock(&ub->mutex); if (ublk_nosrv_should_stop_dev(ub)) goto out_unlock; diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c index 86a48d009d1b..afab479ac894 100644 --- a/drivers/bluetooth/btqca.c +++ b/drivers/bluetooth/btqca.c @@ -413,7 +413,7 @@ static int qca_tlv_check_data(struct hci_dev *hdev, idx = 0; data = tlv->data; - while (idx < length - sizeof(struct tlv_type_nvm)) { + while (idx + sizeof(struct tlv_type_nvm) <= length) { tlv_nvm = (struct tlv_type_nvm *)(data + idx); tag_id = le16_to_cpu(tlv_nvm->tag_id); diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c index b9e5adecfed6..fea3cb502a4e 100644 --- a/drivers/bluetooth/btusb.c +++ b/drivers/bluetooth/btusb.c @@ -2716,7 +2716,9 @@ static int btusb_setup_realtek(struct hci_dev *hdev) static int btusb_recv_event_realtek(struct hci_dev *hdev, struct sk_buff *skb) { - if (skb->data[0] == HCI_VENDOR_PKT && skb->data[2] == RTK_SUB_EVENT_CODE_COREDUMP) { + if (skb->len >= HCI_EVENT_HDR_SIZE + 1 && + skb->data[0] == HCI_VENDOR_PKT && + skb->data[2] == RTK_SUB_EVENT_CODE_COREDUMP) { struct rtk_dev_coredump_hdr hdr = { .code = RTK_DEVCOREDUMP_CODE_MEMDUMP, }; diff --git a/drivers/bluetooth/hci_qca.c b/drivers/bluetooth/hci_qca.c index c649a3f702c0..b510b03dae87 100644 --- a/drivers/bluetooth/hci_qca.c +++ b/drivers/bluetooth/hci_qca.c @@ -1083,6 +1083,10 @@ static void qca_controller_memdump(struct work_struct *work) if (!(qca_memdump->ram_dump_size)) { bt_dev_err(hu->hdev, "Rx invalid memdump size"); kfree(qca_memdump); + qca->qca_memdump = NULL; + qca->memdump_state = QCA_MEMDUMP_COLLECTED; + clear_and_wake_up_bit(QCA_MEMDUMP_COLLECTION, &qca->flags); + clear_bit(QCA_IBS_DISABLED, &qca->flags); kfree_skb(skb); mutex_unlock(&qca->hci_memdump_lock); return; diff --git a/drivers/cdrom/cdrom.c b/drivers/cdrom/cdrom.c index e30414f0d456..ce2de543bac3 100644 --- a/drivers/cdrom/cdrom.c +++ b/drivers/cdrom/cdrom.c @@ -3187,6 +3187,7 @@ static noinline int mmc_ioctl_cdrom_volume(struct cdrom_device_info *cdi, /* set volume */ cgc->buffer = buffer + offset - 8; + cgc->buflen -= offset - 8; memset(cgc->buffer, 0, 8); return cdrom_mode_select(cdi, cgc); } diff --git a/drivers/comedi/drivers/comedi_parport.c b/drivers/comedi/drivers/comedi_parport.c index 098738a688fe..db9f58792ab9 100644 --- a/drivers/comedi/drivers/comedi_parport.c +++ b/drivers/comedi/drivers/comedi_parport.c @@ -211,6 +211,13 @@ static irqreturn_t parport_interrupt(int irq, void *d) unsigned int ctrl; unsigned short val = 0; + /* + * Check device is fully attached. Device interrupts should have + * been disabled, but do this in case of bad hardware. + */ + if (!dev->attached) + return IRQ_NONE; + ctrl = inb(dev->iobase + PARPORT_CTRL_REG); if (!(ctrl & PARPORT_CTRL_IRQ_ENA)) return IRQ_NONE; @@ -231,6 +238,9 @@ static int parport_attach(struct comedi_device *dev, if (ret) return ret; + outb(0, dev->iobase + PARPORT_DATA_REG); + outb(0, dev->iobase + PARPORT_CTRL_REG); + if (it->options[1]) { ret = request_irq(it->options[1], parport_interrupt, 0, dev->board_name, dev); @@ -286,9 +296,6 @@ static int parport_attach(struct comedi_device *dev, s->cancel = parport_intr_cancel; } - outb(0, dev->iobase + PARPORT_DATA_REG); - outb(0, dev->iobase + PARPORT_CTRL_REG); - return 0; } diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index a70df77f9b7d..ced96d9e254e 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -2584,6 +2584,9 @@ static void cpufreq_update_pressure(struct cpufreq_policy *policy) cpu = cpumask_first(policy->related_cpus); max_freq = arch_scale_freq_ref(cpu); + if (!max_freq) + max_freq = policy->cpuinfo.max_freq; + capped_freq = policy->max; /* diff --git a/drivers/crypto/tegra/tegra-se-main.c b/drivers/crypto/tegra/tegra-se-main.c index 2755f19ad05e..77b923a6b7c1 100644 --- a/drivers/crypto/tegra/tegra-se-main.c +++ b/drivers/crypto/tegra/tegra-se-main.c @@ -52,7 +52,7 @@ tegra_se_cmdbuf_pin(struct device *dev, struct host1x_bo *bo, enum dma_data_dire return ERR_PTR(-ENOMEM); kref_init(&map->ref); - map->bo = host1x_bo_get(bo); + map->bo = bo; map->direction = direction; map->dev = dev; @@ -93,7 +93,6 @@ static void tegra_se_cmdbuf_unpin(struct host1x_bo_mapping *map) dma_unmap_sgtable(map->dev, map->sgt, map->direction, 0); sg_free_table(map->sgt); kfree(map->sgt); - host1x_bo_put(map->bo); kfree(map); } diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 599e126a18eb..4bdec46b2a6b 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -234,4 +234,8 @@ config CXL_MCE def_bool y depends on X86_MCE && MEMORY_FAILURE +config CXL_RAS + def_bool y + depends on ACPI_APEI_GHES && PCIEAER && CXL_PCI + endif diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 5ad8fef210b5..b2930cc54f8b 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -14,9 +14,9 @@ cxl_core-y += pci.o cxl_core-y += hdm.o cxl_core-y += pmu.o cxl_core-y += cdat.o -cxl_core-y += ras.o cxl_core-$(CONFIG_TRACING) += trace.o cxl_core-$(CONFIG_CXL_REGION) += region.o cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o +cxl_core-$(CONFIG_CXL_RAS) += ras.o diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 1fb66132b777..bc818de87ccc 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -144,8 +144,39 @@ int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c); int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port, struct access_coordinate *c); +#ifdef CONFIG_CXL_RAS int cxl_ras_init(void); void cxl_ras_exit(void); +bool cxl_handle_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base); +void cxl_handle_cor_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base); +#else +static inline int cxl_ras_init(void) +{ + return 0; +} + +static inline void cxl_ras_exit(void) +{ +} + +static inline bool cxl_handle_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base) +{ + return false; +} +static inline void cxl_handle_cor_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base) { } +#endif /* CONFIG_CXL_RAS */ + +/* Restricted CXL Host specific RAS functions */ +#ifdef CONFIG_CXL_RAS +void cxl_dport_map_rch_aer(struct cxl_dport *dport); +void cxl_disable_rch_root_ints(struct cxl_dport *dport); +void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds); +#else +static inline void cxl_dport_map_rch_aer(struct cxl_dport *dport) { } +static inline void cxl_disable_rch_root_ints(struct cxl_dport *dport) { } +static inline void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) { } +#endif /* CONFIG_CXL_RAS */ + int cxl_gpf_port_setup(struct cxl_dport *dport); struct cxl_hdm; diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 18825e1505d6..bd7d7e382d04 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -711,91 +711,8 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, "CXL"); -static void __cxl_handle_cor_ras(struct cxl_dev_state *cxlds, - void __iomem *ras_base) -{ - void __iomem *addr; - u32 status; - - if (!ras_base) - return; - - addr = ras_base + CXL_RAS_CORRECTABLE_STATUS_OFFSET; - status = readl(addr); - if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { - writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); - trace_cxl_aer_correctable_error(cxlds->cxlmd, status); - } -} - -static void cxl_handle_endpoint_cor_ras(struct cxl_dev_state *cxlds) -{ - return __cxl_handle_cor_ras(cxlds, cxlds->regs.ras); -} - -/* CXL spec rev3.0 8.2.4.16.1 */ -static void header_log_copy(void __iomem *ras_base, u32 *log) -{ - void __iomem *addr; - u32 *log_addr; - int i, log_u32_size = CXL_HEADERLOG_SIZE / sizeof(u32); - - addr = ras_base + CXL_RAS_HEADER_LOG_OFFSET; - log_addr = log; - - for (i = 0; i < log_u32_size; i++) { - *log_addr = readl(addr); - log_addr++; - addr += sizeof(u32); - } -} - -/* - * Log the state of the RAS status registers and prepare them to log the - * next error status. Return 1 if reset needed. - */ -static bool __cxl_handle_ras(struct cxl_dev_state *cxlds, - void __iomem *ras_base) -{ - u32 hl[CXL_HEADERLOG_SIZE_U32]; - void __iomem *addr; - u32 status; - u32 fe; - - if (!ras_base) - return false; - - addr = ras_base + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; - status = readl(addr); - if (!(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK)) - return false; - - /* If multiple errors, log header points to first error from ctrl reg */ - if (hweight32(status) > 1) { - void __iomem *rcc_addr = - ras_base + CXL_RAS_CAP_CONTROL_OFFSET; - - fe = BIT(FIELD_GET(CXL_RAS_CAP_CONTROL_FE_MASK, - readl(rcc_addr))); - } else { - fe = status; - } - - header_log_copy(ras_base, hl); - trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, hl); - writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); - - return true; -} - -static bool cxl_handle_endpoint_ras(struct cxl_dev_state *cxlds) -{ - return __cxl_handle_ras(cxlds, cxlds->regs.ras); -} - -#ifdef CONFIG_PCIEAER_CXL - -static void cxl_dport_map_rch_aer(struct cxl_dport *dport) +#ifdef CONFIG_CXL_RAS +void cxl_dport_map_rch_aer(struct cxl_dport *dport) { resource_size_t aer_phys; struct device *host; @@ -810,19 +727,7 @@ static void cxl_dport_map_rch_aer(struct cxl_dport *dport) } } -static void cxl_dport_map_ras(struct cxl_dport *dport) -{ - struct cxl_register_map *map = &dport->reg_map; - struct device *dev = dport->dport_dev; - - if (!map->component_map.ras.valid) - dev_dbg(dev, "RAS registers not found\n"); - else if (cxl_map_component_regs(map, &dport->regs.component, - BIT(CXL_CM_CAP_CAP_ID_RAS))) - dev_dbg(dev, "Failed to map RAS capability.\n"); -} - -static void cxl_disable_rch_root_ints(struct cxl_dport *dport) +void cxl_disable_rch_root_ints(struct cxl_dport *dport) { void __iomem *aer_base = dport->regs.dport_aer; u32 aer_cmd_mask, aer_cmd; @@ -846,40 +751,6 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport) writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); } -/** - * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport - * @dport: the cxl_dport that needs to be initialized - * @host: host device for devm operations - */ -void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) -{ - dport->reg_map.host = host; - cxl_dport_map_ras(dport); - - if (dport->rch) { - struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport->dport_dev); - - if (!host_bridge->native_aer) - return; - - cxl_dport_map_rch_aer(dport); - cxl_disable_rch_root_ints(dport); - } -} -EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); - -static void cxl_handle_rdport_cor_ras(struct cxl_dev_state *cxlds, - struct cxl_dport *dport) -{ - return __cxl_handle_cor_ras(cxlds, dport->regs.ras); -} - -static bool cxl_handle_rdport_ras(struct cxl_dev_state *cxlds, - struct cxl_dport *dport) -{ - return __cxl_handle_ras(cxlds, dport->regs.ras); -} - /* * Copy the AER capability registers using 32 bit read accesses. * This is necessary because RCRB AER capability is MMIO mapped. Clear the @@ -928,7 +799,7 @@ static bool cxl_rch_get_aer_severity(struct aer_capability_regs *aer_regs, return false; } -static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) +void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) { struct pci_dev *pdev = to_pci_dev(cxlds->dev); struct aer_capability_regs aer_regs; @@ -949,86 +820,12 @@ static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) pci_print_aer(pdev, severity, &aer_regs); if (severity == AER_CORRECTABLE) - cxl_handle_rdport_cor_ras(cxlds, dport); + cxl_handle_cor_ras(cxlds, dport->regs.ras); else - cxl_handle_rdport_ras(cxlds, dport); + cxl_handle_ras(cxlds, dport->regs.ras); } - -#else -static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) { } #endif -void cxl_cor_error_detected(struct pci_dev *pdev) -{ - struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); - struct device *dev = &cxlds->cxlmd->dev; - - scoped_guard(device, dev) { - if (!dev->driver) { - dev_warn(&pdev->dev, - "%s: memdev disabled, abort error handling\n", - dev_name(dev)); - return; - } - - if (cxlds->rcd) - cxl_handle_rdport_errors(cxlds); - - cxl_handle_endpoint_cor_ras(cxlds); - } -} -EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, "CXL"); - -pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, - pci_channel_state_t state) -{ - struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); - struct cxl_memdev *cxlmd = cxlds->cxlmd; - struct device *dev = &cxlmd->dev; - bool ue; - - scoped_guard(device, dev) { - if (!dev->driver) { - dev_warn(&pdev->dev, - "%s: memdev disabled, abort error handling\n", - dev_name(dev)); - return PCI_ERS_RESULT_DISCONNECT; - } - - if (cxlds->rcd) - cxl_handle_rdport_errors(cxlds); - /* - * A frozen channel indicates an impending reset which is fatal to - * CXL.mem operation, and will likely crash the system. On the off - * chance the situation is recoverable dump the status of the RAS - * capability registers and bounce the active state of the memdev. - */ - ue = cxl_handle_endpoint_ras(cxlds); - } - - - switch (state) { - case pci_channel_io_normal: - if (ue) { - device_release_driver(dev); - return PCI_ERS_RESULT_NEED_RESET; - } - return PCI_ERS_RESULT_CAN_RECOVER; - case pci_channel_io_frozen: - dev_warn(&pdev->dev, - "%s: frozen state error detected, disable CXL.mem\n", - dev_name(dev)); - device_release_driver(dev); - return PCI_ERS_RESULT_NEED_RESET; - case pci_channel_io_perm_failure: - dev_warn(&pdev->dev, - "failure state error detected, request disconnect\n"); - return PCI_ERS_RESULT_DISCONNECT; - } - return PCI_ERS_RESULT_NEED_RESET; -} -EXPORT_SYMBOL_NS_GPL(cxl_error_detected, "CXL"); - static int cxl_flit_size(struct pci_dev *pdev) { if (cxl_pci_flit_256(pdev)) diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c index 2731ba3a0799..7056a79bff4d 100644 --- a/drivers/cxl/core/ras.c +++ b/drivers/cxl/core/ras.c @@ -5,8 +5,13 @@ #include #include #include +#include #include "trace.h" +/* Check that UCE header definition is maintained to keep ABI intact */ +static_assert(CXL_HEADERLOG_TRACE_SIZE_U32 == 128, + "rasdaemon ABI requires exactly 128 u32s"); + static void cxl_cper_trace_corr_port_prot_err(struct pci_dev *pdev, struct cxl_ras_capability_regs ras_cap) { @@ -18,6 +23,7 @@ static void cxl_cper_trace_corr_port_prot_err(struct pci_dev *pdev, static void cxl_cper_trace_uncorr_port_prot_err(struct pci_dev *pdev, struct cxl_ras_capability_regs ras_cap) { + u32 hl[CXL_HEADERLOG_TRACE_SIZE_U32] = {}; u32 status = ras_cap.uncor_status & ~ras_cap.uncor_mask; u32 fe; @@ -27,8 +33,8 @@ static void cxl_cper_trace_uncorr_port_prot_err(struct pci_dev *pdev, else fe = status; - trace_cxl_port_aer_uncorrectable_error(&pdev->dev, status, fe, - ras_cap.header_log); + memcpy(hl, ras_cap.header_log, CXL_HEADERLOG_SIZE); + trace_cxl_port_aer_uncorrectable_error(&pdev->dev, status, fe, hl); } static void cxl_cper_trace_corr_prot_err(struct cxl_memdev *cxlmd, @@ -43,6 +49,7 @@ static void cxl_cper_trace_uncorr_prot_err(struct cxl_memdev *cxlmd, struct cxl_ras_capability_regs ras_cap) { + u32 hl[CXL_HEADERLOG_TRACE_SIZE_U32] = {}; u32 status = ras_cap.uncor_status & ~ras_cap.uncor_mask; u32 fe; @@ -52,8 +59,15 @@ cxl_cper_trace_uncorr_prot_err(struct cxl_memdev *cxlmd, else fe = status; - trace_cxl_aer_uncorrectable_error(cxlmd, status, fe, - ras_cap.header_log); + /* + * ras_cap.header_log[] holds CXL_HEADERLOG_SIZE_U32 (16) hardware + * dwords. Copy them into the front of a zero-filled + * CXL_HEADERLOG_TRACE_SIZE_U32 (128) u32 staging buffer so the trace + * event memcpy sees a full 512-byte source and the userspace ABI + * (rasdaemon) is preserved. + */ + memcpy(hl, ras_cap.header_log, CXL_HEADERLOG_SIZE); + trace_cxl_aer_uncorrectable_error(cxlmd, status, fe, hl); } static int match_memdev_by_parent(struct device *dev, const void *uport) @@ -124,3 +138,178 @@ void cxl_ras_exit(void) cxl_cper_unregister_prot_err_work(&cxl_cper_prot_err_work); cancel_work_sync(&cxl_cper_prot_err_work); } + +static void cxl_dport_map_ras(struct cxl_dport *dport) +{ + struct cxl_register_map *map = &dport->reg_map; + struct device *dev = dport->dport_dev; + + if (!map->component_map.ras.valid) + dev_dbg(dev, "RAS registers not found\n"); + else if (cxl_map_component_regs(map, &dport->regs.component, + BIT(CXL_CM_CAP_CAP_ID_RAS))) + dev_dbg(dev, "Failed to map RAS capability.\n"); +} + +/** + * cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport + * @dport: the cxl_dport that needs to be initialized + * @host: host device for devm operations + */ +void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) +{ + dport->reg_map.host = host; + cxl_dport_map_ras(dport); + + if (dport->rch) { + struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport->dport_dev); + + if (!host_bridge->native_aer) + return; + + cxl_dport_map_rch_aer(dport); + cxl_disable_rch_root_ints(dport); + } +} +EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL"); + +void cxl_handle_cor_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base) +{ + void __iomem *addr; + u32 status; + + if (!ras_base) + return; + + addr = ras_base + CXL_RAS_CORRECTABLE_STATUS_OFFSET; + status = readl(addr); + if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { + writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); + trace_cxl_aer_correctable_error(cxlds->cxlmd, status); + } +} + +/* CXL spec rev3.0 8.2.4.16.1 */ +static void header_log_copy(void __iomem *ras_base, u32 *log) +{ + void __iomem *addr; + u32 *log_addr; + int i; + + addr = ras_base + CXL_RAS_HEADER_LOG_OFFSET; + log_addr = log; + + for (i = 0; i < CXL_HEADERLOG_SIZE_U32; i++) { + *log_addr = readl(addr); + log_addr++; + addr += sizeof(u32); + } +} + +/* + * Log the state of the RAS status registers and prepare them to log the + * next error status. Return 1 if reset needed. + */ +bool cxl_handle_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base) +{ + u32 hl[CXL_HEADERLOG_TRACE_SIZE_U32] = {}; + void __iomem *addr; + u32 status; + u32 fe; + + if (!ras_base) + return false; + + addr = ras_base + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; + status = readl(addr); + if (!(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK)) + return false; + + /* If multiple errors, log header points to first error from ctrl reg */ + if (hweight32(status) > 1) { + void __iomem *rcc_addr = + ras_base + CXL_RAS_CAP_CONTROL_OFFSET; + + fe = BIT(FIELD_GET(CXL_RAS_CAP_CONTROL_FE_MASK, + readl(rcc_addr))); + } else { + fe = status; + } + + header_log_copy(ras_base, hl); + trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, hl); + writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); + + return true; +} + +void cxl_cor_error_detected(struct pci_dev *pdev) +{ + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); + struct device *dev = &cxlds->cxlmd->dev; + + scoped_guard(device, dev) { + if (!dev->driver) { + dev_warn(&pdev->dev, + "%s: memdev disabled, abort error handling\n", + dev_name(dev)); + return; + } + + if (cxlds->rcd) + cxl_handle_rdport_errors(cxlds); + + cxl_handle_cor_ras(cxlds, cxlds->regs.ras); + } +} +EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, "CXL"); + +pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, + pci_channel_state_t state) +{ + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); + struct cxl_memdev *cxlmd = cxlds->cxlmd; + struct device *dev = &cxlmd->dev; + bool ue; + + scoped_guard(device, dev) { + if (!dev->driver) { + dev_warn(&pdev->dev, + "%s: memdev disabled, abort error handling\n", + dev_name(dev)); + return PCI_ERS_RESULT_DISCONNECT; + } + + if (cxlds->rcd) + cxl_handle_rdport_errors(cxlds); + /* + * A frozen channel indicates an impending reset which is fatal to + * CXL.mem operation, and will likely crash the system. On the off + * chance the situation is recoverable dump the status of the RAS + * capability registers and bounce the active state of the memdev. + */ + ue = cxl_handle_ras(cxlds, cxlds->regs.ras); + } + + + switch (state) { + case pci_channel_io_normal: + if (ue) { + device_release_driver(dev); + return PCI_ERS_RESULT_NEED_RESET; + } + return PCI_ERS_RESULT_CAN_RECOVER; + case pci_channel_io_frozen: + dev_warn(&pdev->dev, + "%s: frozen state error detected, disable CXL.mem\n", + dev_name(dev)); + device_release_driver(dev); + return PCI_ERS_RESULT_NEED_RESET; + case pci_channel_io_perm_failure: + dev_warn(&pdev->dev, + "failure state error detected, request disconnect\n"); + return PCI_ERS_RESULT_DISCONNECT; + } + return PCI_ERS_RESULT_NEED_RESET; +} +EXPORT_SYMBOL_NS_GPL(cxl_error_detected, "CXL"); diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index a972e4ef1936..d37876096dd7 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -56,7 +56,7 @@ TRACE_EVENT(cxl_port_aer_uncorrectable_error, __string(host, dev_name(dev->parent)) __field(u32, status) __field(u32, first_error) - __array(u32, header_log, CXL_HEADERLOG_SIZE_U32) + __array(u32, header_log, CXL_HEADERLOG_TRACE_SIZE_U32) ), TP_fast_assign( __assign_str(device); @@ -64,10 +64,14 @@ TRACE_EVENT(cxl_port_aer_uncorrectable_error, __entry->status = status; __entry->first_error = fe; /* - * Embed the 512B headerlog data for user app retrieval and - * parsing, but no need to print this in the trace buffer. + * Embed headerlog data for user app retrieval and parsing, + * but no need to print in the trace buffer. Only + * CXL_HEADERLOG_SIZE_U32 (16) dwords are hardware data; + * the remaining entries preserve the 512-byte ABI layout + * rasdaemon depends on and are zero-filled by the caller. */ - memcpy(__entry->header_log, hl, CXL_HEADERLOG_SIZE); + memcpy(__entry->header_log, hl, + CXL_HEADERLOG_TRACE_SIZE_U32 * sizeof(u32)); ), TP_printk("device=%s host=%s status: '%s' first_error: '%s'", __get_str(device), __get_str(host), @@ -85,7 +89,7 @@ TRACE_EVENT(cxl_aer_uncorrectable_error, __field(u64, serial) __field(u32, status) __field(u32, first_error) - __array(u32, header_log, CXL_HEADERLOG_SIZE_U32) + __array(u32, header_log, CXL_HEADERLOG_TRACE_SIZE_U32) ), TP_fast_assign( __assign_str(memdev); @@ -94,10 +98,14 @@ TRACE_EVENT(cxl_aer_uncorrectable_error, __entry->status = status; __entry->first_error = fe; /* - * Embed the 512B headerlog data for user app retrieval and - * parsing, but no need to print this in the trace buffer. + * Embed headerlog data for user app retrieval and parsing, + * but no need to print in the trace buffer. Only + * CXL_HEADERLOG_SIZE_U32 (16) dwords are hardware data; + * the remaining entries preserve the 512-byte ABI layout + * rasdaemon depends on and are zero-filled by the caller. */ - memcpy(__entry->header_log, hl, CXL_HEADERLOG_SIZE); + memcpy(__entry->header_log, hl, + CXL_HEADERLOG_TRACE_SIZE_U32 * sizeof(u32)); ), TP_printk("memdev=%s host=%s serial=%lld: status: '%s' first_error: '%s'", __get_str(memdev), __get_str(host), __entry->serial, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 3a794278cc7f..a5b00c3b0444 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -148,8 +148,18 @@ static inline int ways_to_eiw(unsigned int ways, u8 *eiw) #define CXL_RAS_CAP_CONTROL_FE_MASK GENMASK(5, 0) #define CXL_RAS_HEADER_LOG_OFFSET 0x18 #define CXL_RAS_CAPABILITY_LENGTH 0x58 -#define CXL_HEADERLOG_SIZE SZ_512 -#define CXL_HEADERLOG_SIZE_U32 SZ_512 / sizeof(u32) +#define CXL_HEADERLOG_SIZE SZ_64 +#define CXL_HEADERLOG_SIZE_U32 (CXL_HEADERLOG_SIZE / sizeof(u32)) + +/* + * The RAS UCE trace event header array was originally sized at SZ_512/sizeof(u32) + * = 128 u32s due to a bug. Userspace tools (rasdaemon) have grown a dependency + * on that 512-byte layout. Keep the trace array at 128 u32s to preserve the + * ABI; only CXL_HEADERLOG_SIZE_U32 (16) dwords are valid hardware data, the + * remainder are zero-filled. + */ +#define CXL_HEADERLOG_TRACE_SIZE SZ_512 +#define CXL_HEADERLOG_TRACE_SIZE_U32 (CXL_HEADERLOG_TRACE_SIZE / sizeof(u32)) /* CXL 2.0 8.2.8.1 Device Capabilities Array Register */ #define CXLDEV_CAP_ARRAY_OFFSET 0x0 @@ -781,14 +791,6 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, struct device *dport_dev, int port_id, resource_size_t rcrb); -#ifdef CONFIG_PCIEAER_CXL -void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport); -void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host); -#else -static inline void cxl_dport_init_ras_reporting(struct cxl_dport *dport, - struct device *host) { } -#endif - struct cxl_decoder *to_cxl_decoder(struct device *dev); struct cxl_root_decoder *to_cxl_root_decoder(struct device *dev); struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev); diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 7ae621e618e7..970e84cf49e9 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -130,7 +130,23 @@ static inline bool cxl_pci_flit_256(struct pci_dev *pdev) int devm_cxl_port_enumerate_dports(struct cxl_port *port); struct cxl_dev_state; void read_cdat_data(struct cxl_port *port); + +#ifdef CONFIG_CXL_RAS void cxl_cor_error_detected(struct pci_dev *pdev); pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, pci_channel_state_t state); +void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host); +#else +static inline void cxl_cor_error_detected(struct pci_dev *pdev) { } + +static inline pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, + pci_channel_state_t state) +{ + return PCI_ERS_RESULT_NONE; +} + +static inline void cxl_dport_init_ras_reporting(struct cxl_dport *dport, + struct device *host) { } +#endif + #endif /* __CXL_PCI_H__ */ diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index be47d1ff6c50..30b0f0c72e92 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -249,21 +249,22 @@ static int begin_cpu_udmabuf(struct dma_buf *buf, { struct udmabuf *ubuf = buf->priv; struct device *dev = ubuf->device->this_device; - int ret = 0; if (!ubuf->sg) { ubuf->sg = get_sg_table(dev, buf, direction); if (IS_ERR(ubuf->sg)) { + int ret; + ret = PTR_ERR(ubuf->sg); ubuf->sg = NULL; + return ret; } else { ubuf->sg_dir = direction; } - } else { - dma_sync_sgtable_for_cpu(dev, ubuf->sg, direction); } - return ret; + dma_sync_sgtable_for_cpu(dev, ubuf->sg, direction); + return 0; } static int end_cpu_udmabuf(struct dma_buf *buf, diff --git a/drivers/dma/dw-edma/dw-edma-pcie.c b/drivers/dma/dw-edma/dw-edma-pcie.c index 3371e0a76d3c..dc6ffb6941ad 100644 --- a/drivers/dma/dw-edma/dw-edma-pcie.c +++ b/drivers/dma/dw-edma/dw-edma-pcie.c @@ -161,13 +161,16 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev, const struct pci_device_id *pid) { struct dw_edma_pcie_data *pdata = (void *)pid->driver_data; - struct dw_edma_pcie_data *vsec_data __free(kfree) = NULL; struct device *dev = &pdev->dev; struct dw_edma_chip *chip; int err, nr_irqs; int i, mask; - vsec_data = kmalloc(sizeof(*vsec_data), GFP_KERNEL); + if (!pdata) + return -ENODEV; + + struct dw_edma_pcie_data *vsec_data __free(kfree) = + kmalloc(sizeof(*vsec_data), GFP_KERNEL); if (!vsec_data) return -ENOMEM; diff --git a/drivers/dma/sh/rz-dmac.c b/drivers/dma/sh/rz-dmac.c index 818d1ef6f0bf..4db84b83177a 100644 --- a/drivers/dma/sh/rz-dmac.c +++ b/drivers/dma/sh/rz-dmac.c @@ -811,27 +811,6 @@ static int rz_dmac_chan_probe(struct rz_dmac *dmac, channel->index = index; channel->mid_rid = -EINVAL; - /* Request the channel interrupt. */ - scnprintf(pdev_irqname, sizeof(pdev_irqname), "ch%u", index); - channel->irq = platform_get_irq_byname(pdev, pdev_irqname); - if (channel->irq < 0) - return channel->irq; - - irqname = devm_kasprintf(dmac->dev, GFP_KERNEL, "%s:%u", - dev_name(dmac->dev), index); - if (!irqname) - return -ENOMEM; - - ret = devm_request_threaded_irq(dmac->dev, channel->irq, - rz_dmac_irq_handler, - rz_dmac_irq_handler_thread, 0, - irqname, channel); - if (ret) { - dev_err(dmac->dev, "failed to request IRQ %u (%d)\n", - channel->irq, ret); - return ret; - } - /* Set io base address for each channel */ if (index < 8) { channel->ch_base = dmac->base + CHANNEL_0_7_OFFSET + @@ -844,9 +823,9 @@ static int rz_dmac_chan_probe(struct rz_dmac *dmac, } /* Allocate descriptors */ - lmdesc = dma_alloc_coherent(&pdev->dev, - sizeof(struct rz_lmdesc) * DMAC_NR_LMDESC, - &channel->lmdesc.base_dma, GFP_KERNEL); + lmdesc = dmam_alloc_coherent(&pdev->dev, + sizeof(struct rz_lmdesc) * DMAC_NR_LMDESC, + &channel->lmdesc.base_dma, GFP_KERNEL); if (!lmdesc) { dev_err(&pdev->dev, "Can't allocate memory (lmdesc)\n"); return -ENOMEM; @@ -862,7 +841,26 @@ static int rz_dmac_chan_probe(struct rz_dmac *dmac, INIT_LIST_HEAD(&channel->ld_free); INIT_LIST_HEAD(&channel->ld_active); - return 0; + /* Request the channel interrupt. */ + scnprintf(pdev_irqname, sizeof(pdev_irqname), "ch%u", index); + channel->irq = platform_get_irq_byname(pdev, pdev_irqname); + if (channel->irq < 0) + return channel->irq; + + irqname = devm_kasprintf(dmac->dev, GFP_KERNEL, "%s:%u", + dev_name(dmac->dev), index); + if (!irqname) + return -ENOMEM; + + ret = devm_request_threaded_irq(dmac->dev, channel->irq, + rz_dmac_irq_handler, + rz_dmac_irq_handler_thread, 0, + irqname, channel); + if (ret) + dev_err(dmac->dev, "failed to request IRQ %u (%d)\n", + channel->irq, ret); + + return ret; } static void rz_dmac_put_device(void *_dev) @@ -932,7 +930,6 @@ static int rz_dmac_probe(struct platform_device *pdev) const char *irqname = "error"; struct dma_device *engine; struct rz_dmac *dmac; - int channel_num; int ret; int irq; u8 i; @@ -964,19 +961,6 @@ static int rz_dmac_probe(struct platform_device *pdev) return PTR_ERR(dmac->ext_base); } - /* Register interrupt handler for error */ - irq = platform_get_irq_byname(pdev, irqname); - if (irq < 0) - return irq; - - ret = devm_request_irq(&pdev->dev, irq, rz_dmac_irq_handler, 0, - irqname, NULL); - if (ret) { - dev_err(&pdev->dev, "failed to request IRQ %u (%d)\n", - irq, ret); - return ret; - } - /* Initialize the channels. */ INIT_LIST_HEAD(&dmac->engine.channels); @@ -1002,6 +986,21 @@ static int rz_dmac_probe(struct platform_device *pdev) goto err; } + /* Register interrupt handler for error */ + irq = platform_get_irq_byname(pdev, irqname); + if (irq < 0) { + ret = irq; + goto err; + } + + ret = devm_request_irq(&pdev->dev, irq, rz_dmac_irq_handler, 0, + irqname, NULL); + if (ret) { + dev_err(&pdev->dev, "failed to request IRQ %u (%d)\n", + irq, ret); + goto err; + } + /* Register the DMAC as a DMA provider for DT. */ ret = of_dma_controller_register(pdev->dev.of_node, rz_dmac_of_xlate, NULL); @@ -1040,16 +1039,6 @@ static int rz_dmac_probe(struct platform_device *pdev) dma_register_err: of_dma_controller_free(pdev->dev.of_node); err: - channel_num = i ? i - 1 : 0; - for (i = 0; i < channel_num; i++) { - struct rz_dmac_chan *channel = &dmac->channels[i]; - - dma_free_coherent(&pdev->dev, - sizeof(struct rz_lmdesc) * DMAC_NR_LMDESC, - channel->lmdesc.base, - channel->lmdesc.base_dma); - } - reset_control_assert(dmac->rstc); err_pm_runtime_put: pm_runtime_put(&pdev->dev); @@ -1062,18 +1051,9 @@ static int rz_dmac_probe(struct platform_device *pdev) static void rz_dmac_remove(struct platform_device *pdev) { struct rz_dmac *dmac = platform_get_drvdata(pdev); - unsigned int i; dma_async_device_unregister(&dmac->engine); of_dma_controller_free(pdev->dev.of_node); - for (i = 0; i < dmac->n_channels; i++) { - struct rz_dmac_chan *channel = &dmac->channels[i]; - - dma_free_coherent(&pdev->dev, - sizeof(struct rz_lmdesc) * DMAC_NR_LMDESC, - channel->lmdesc.base, - channel->lmdesc.base_dma); - } reset_control_assert(dmac->rstc); pm_runtime_put(&pdev->dev); pm_runtime_disable(&pdev->dev); diff --git a/drivers/dpll/dpll_netlink.c b/drivers/dpll/dpll_netlink.c index de8b065280d1..eb23ee401aae 100644 --- a/drivers/dpll/dpll_netlink.c +++ b/drivers/dpll/dpll_netlink.c @@ -486,6 +486,9 @@ dpll_msg_add_pin_ref_sync(struct sk_buff *msg, struct dpll_pin *pin, if (!dpll_pin_available(ref_sync_pin)) continue; ref_sync_pin_priv = dpll_pin_on_dpll_priv(dpll, ref_sync_pin); + /* Pin may have been unregistered from this dpll already */ + if (!ref_sync_pin_priv) + continue; if (WARN_ON(!ops->ref_sync_get)) return -EOPNOTSUPP; ret = ops->ref_sync_get(pin, pin_priv, ref_sync_pin, diff --git a/drivers/firewire/net.c b/drivers/firewire/net.c index e82945408955..354c81409946 100644 --- a/drivers/firewire/net.c +++ b/drivers/firewire/net.c @@ -298,31 +298,34 @@ static struct fwnet_fragment_info *fwnet_frag_new( if (fi->offset + fi->len == offset) { /* The new fragment can be tacked on to the end */ /* Did the new fragment plug a hole? */ - fi2 = list_entry(fi->fi_link.next, - struct fwnet_fragment_info, fi_link); - if (fi->offset + fi->len == fi2->offset) { - /* glue fragments together */ - fi->len += len + fi2->len; - list_del(&fi2->fi_link); - kfree(fi2); - } else { - fi->len += len; + if (!list_is_last(&fi->fi_link, &pd->fi_list)) { + fi2 = list_next_entry(fi, fi_link); + if (offset + len == fi2->offset) { + /* glue fragments together */ + fi->len += len + fi2->len; + list_del(&fi2->fi_link); + kfree(fi2); + + return fi; + } } + fi->len += len; return fi; } if (offset + len == fi->offset) { /* The new fragment can be tacked on to the beginning */ /* Did the new fragment plug a hole? */ - fi2 = list_entry(fi->fi_link.prev, - struct fwnet_fragment_info, fi_link); - if (fi2->offset + fi2->len == fi->offset) { - /* glue fragments together */ - fi2->len += fi->len + len; - list_del(&fi->fi_link); - kfree(fi); - - return fi2; + if (!list_is_first(&fi->fi_link, &pd->fi_list)) { + fi2 = list_prev_entry(fi, fi_link); + if (fi2->offset + fi2->len == offset) { + /* glue fragments together */ + fi2->len += fi->len + len; + list_del(&fi->fi_link); + kfree(fi); + + return fi2; + } } fi->offset = offset; fi->len += len; diff --git a/drivers/firmware/arm_ffa/driver.c b/drivers/firmware/arm_ffa/driver.c index 3fbc75caf0c5..1d0bd586da4d 100644 --- a/drivers/firmware/arm_ffa/driver.c +++ b/drivers/firmware/arm_ffa/driver.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include #include @@ -55,7 +56,9 @@ (FIELD_PREP(SENDER_ID_MASK, (s)) | FIELD_PREP(RECEIVER_ID_MASK, (r))) #define RXTX_MAP_MIN_BUFSZ_MASK GENMASK(1, 0) -#define RXTX_MAP_MIN_BUFSZ(x) ((x) & RXTX_MAP_MIN_BUFSZ_MASK) +#define RXTX_MAP_MAX_BUFSZ_MASK GENMASK(31, 16) +#define RXTX_MAP_MIN_BUFSZ(x) (FIELD_GET(RXTX_MAP_MIN_BUFSZ_MASK, (x))) +#define RXTX_MAP_MAX_BUFSZ(x) (FIELD_GET(RXTX_MAP_MAX_BUFSZ_MASK, (x))) #define FFA_MAX_NOTIFICATIONS 64 @@ -704,30 +707,39 @@ ffa_setup_and_transmit(u32 func_id, void *buffer, u32 max_fragsize, struct ffa_composite_mem_region *composite; struct ffa_mem_region_addr_range *constituents; struct ffa_mem_region_attributes *ep_mem_access; - u32 idx, frag_len, length, buf_sz = 0, num_entries = sg_nents(args->sg); + u32 idx, frag_len, length, buf_sz = 0, num_entries = sg_nents(args->sg), ep_offset; + u32 emad_end, emad_size = ffa_emad_size_get(drv_info->version); mem_region->tag = args->tag; mem_region->flags = args->flags; mem_region->sender_id = drv_info->vm_id; mem_region->attributes = ffa_memory_attributes_get(func_id); + + ffa_mem_region_additional_setup(drv_info->version, mem_region); composite_offset = ffa_mem_desc_offset(buffer, args->nattrs, drv_info->version); + if (composite_offset + sizeof(*composite) > max_fragsize) + return -ENXIO; for (idx = 0; idx < args->nattrs; idx++) { - ep_mem_access = buffer + - ffa_mem_desc_offset(buffer, idx, drv_info->version); + ep_offset = ffa_mem_desc_offset(buffer, idx, drv_info->version); + if (check_add_overflow(ep_offset, emad_size, &emad_end)) + return -ENXIO; + + if (emad_end > max_fragsize) + return -ENXIO; + + ep_mem_access = buffer + ep_offset; + memset(ep_mem_access, 0, emad_size); ep_mem_access->receiver = args->attrs[idx].receiver; ep_mem_access->attrs = args->attrs[idx].attrs; ep_mem_access->composite_off = composite_offset; - ep_mem_access->flag = 0; - ep_mem_access->reserved = 0; ffa_emad_impdef_value_init(drv_info->version, ep_mem_access->impdef_val, args->attrs[idx].impdef_val); } mem_region->handle = 0; mem_region->ep_count = args->nattrs; - ffa_mem_region_additional_setup(drv_info->version, mem_region); composite = buffer + composite_offset; composite->total_pg_cnt = ffa_get_num_pages_sg(args->sg); @@ -760,7 +772,7 @@ ffa_setup_and_transmit(u32 func_id, void *buffer, u32 max_fragsize, constituents = buffer; } - if ((void *)constituents - buffer > max_fragsize) { + if ((void *)constituents + sizeof(*constituents) - buffer > max_fragsize) { pr_err("Memory Region Fragment > Tx Buffer size\n"); return -EFAULT; } @@ -769,7 +781,7 @@ ffa_setup_and_transmit(u32 func_id, void *buffer, u32 max_fragsize, constituents->pg_cnt = args->sg->length / FFA_PAGE_SIZE; constituents->reserved = 0; constituents++; - frag_len += sizeof(struct ffa_mem_region_addr_range); + frag_len += sizeof(*constituents); } while ((args->sg = sg_next(args->sg))); return ffa_transmit_fragment(func_id, addr, buf_sz, frag_len, @@ -1130,7 +1142,7 @@ static int ffa_partition_info_get(const char *uuid_str, uuid_t uuid; struct ffa_partition_info *pbuf; - if (uuid_parse(uuid_str, &uuid)) { + if (!uuid_str || uuid_parse(uuid_str, &uuid)) { pr_err("invalid uuid (%s)\n", uuid_str); return -ENODEV; } @@ -2088,7 +2100,7 @@ static int __init ffa_init(void) { int ret; u32 buf_sz; - size_t rxtx_bufsz = SZ_4K; + size_t rxtx_min_bufsz = SZ_4K, rxtx_max_bufsz = 0, rxtx_bufsz; ret = ffa_transport_init(&invoke_ffa_fn); if (ret) @@ -2111,15 +2123,18 @@ static int __init ffa_init(void) ret = ffa_features(FFA_FN_NATIVE(RXTX_MAP), 0, &buf_sz, NULL); if (!ret) { if (RXTX_MAP_MIN_BUFSZ(buf_sz) == 1) - rxtx_bufsz = SZ_64K; + rxtx_min_bufsz = SZ_64K; else if (RXTX_MAP_MIN_BUFSZ(buf_sz) == 2) - rxtx_bufsz = SZ_16K; + rxtx_min_bufsz = SZ_16K; else - rxtx_bufsz = SZ_4K; + rxtx_min_bufsz = SZ_4K; + + rxtx_max_bufsz = RXTX_MAP_MAX_BUFSZ(buf_sz) * SZ_4K; + if (rxtx_max_bufsz != 0 && rxtx_max_bufsz < rxtx_min_bufsz) + rxtx_max_bufsz = rxtx_min_bufsz; } - rxtx_bufsz = PAGE_ALIGN(rxtx_bufsz); - drv_info->rxtx_bufsz = rxtx_bufsz; + rxtx_bufsz = min_not_zero(PAGE_ALIGN(rxtx_min_bufsz), rxtx_max_bufsz); drv_info->rx_buffer = alloc_pages_exact(rxtx_bufsz, GFP_KERNEL); if (!drv_info->rx_buffer) { ret = -ENOMEM; @@ -2135,10 +2150,17 @@ static int __init ffa_init(void) ret = ffa_rxtx_map(virt_to_phys(drv_info->tx_buffer), virt_to_phys(drv_info->rx_buffer), rxtx_bufsz / FFA_PAGE_SIZE); + if (ret == -EINVAL && !rxtx_max_bufsz && rxtx_min_bufsz < rxtx_bufsz) { + rxtx_bufsz = rxtx_min_bufsz; + ret = ffa_rxtx_map(virt_to_phys(drv_info->tx_buffer), + virt_to_phys(drv_info->rx_buffer), + rxtx_bufsz / FFA_PAGE_SIZE); + } if (ret) { pr_err("failed to register FFA RxTx buffers\n"); goto free_pages; } + drv_info->rxtx_bufsz = rxtx_bufsz; mutex_init(&drv_info->rx_lock); mutex_init(&drv_info->tx_lock); diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c index 2047edbdc5f6..9bf7f43ab868 100644 --- a/drivers/firmware/arm_scmi/notify.c +++ b/drivers/firmware/arm_scmi/notify.c @@ -600,9 +600,9 @@ int scmi_notify(const struct scmi_handle *handle, u8 proto_id, u8 evt_id, return -EINVAL; } if (kfifo_avail(&r_evt->proto->equeue.kfifo) < sizeof(eh) + len) { - dev_warn(handle->dev, - "queue full, dropping proto_id:%d evt_id:%d ts:%lld\n", - proto_id, evt_id, ktime_to_ns(ts)); + dev_warn_ratelimited(handle->dev, + "queue full, dropping proto_id:%d evt_id:%d ts:%lld\n", + proto_id, evt_id, ktime_to_ns(ts)); return -ENOMEM; } diff --git a/drivers/firmware/stratix10-svc.c b/drivers/firmware/stratix10-svc.c index 1d24e82f3d09..ba715c6eb32b 100644 --- a/drivers/firmware/stratix10-svc.c +++ b/drivers/firmware/stratix10-svc.c @@ -1080,14 +1080,16 @@ void *stratix10_svc_allocate_memory(struct stratix10_svc_chan *chan, struct gen_pool *genpool = chan->ctrl->genpool; size_t s = roundup(size, 1 << genpool->min_alloc_order); - pmem = devm_kzalloc(chan->ctrl->dev, sizeof(*pmem), GFP_KERNEL); + pmem = kzalloc_obj(*pmem); if (!pmem) return ERR_PTR(-ENOMEM); guard(mutex)(&svc_mem_lock); va = gen_pool_alloc(genpool, s); - if (!va) + if (!va) { + kfree(pmem); return ERR_PTR(-ENOMEM); + } memset((void *)va, 0, s); pa = gen_pool_virt_to_phys(genpool, va); @@ -1113,6 +1115,7 @@ EXPORT_SYMBOL_GPL(stratix10_svc_allocate_memory); void stratix10_svc_free_memory(struct stratix10_svc_chan *chan, void *kaddr) { struct stratix10_svc_data_mem *pmem; + guard(mutex)(&svc_mem_lock); list_for_each_entry(pmem, &svc_data_mem, node) @@ -1121,10 +1124,9 @@ void stratix10_svc_free_memory(struct stratix10_svc_chan *chan, void *kaddr) (unsigned long)kaddr, pmem->size); pmem->vaddr = NULL; list_del(&pmem->node); + kfree(pmem); return; } - - list_del(&svc_data_mem); } EXPORT_SYMBOL_GPL(stratix10_svc_free_memory); diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c index 02da3e48bc8f..dc9c5d32108d 100644 --- a/drivers/firmware/xilinx/zynqmp.c +++ b/drivers/firmware/xilinx/zynqmp.c @@ -461,6 +461,7 @@ int zynqmp_pm_invoke_fn(u32 pm_api_id, u32 *ret_payload, u32 num_args, ...) return do_fw_call(ret_payload, 8, smc_arg[0], smc_arg[1], smc_arg[2], smc_arg[3], smc_arg[4], smc_arg[5], smc_arg[6], smc_arg[7]); } +EXPORT_SYMBOL_GPL(zynqmp_pm_invoke_fn); static u32 pm_api_version; static u32 pm_tz_version; diff --git a/drivers/gpu/Kconfig b/drivers/gpu/Kconfig new file mode 100644 index 000000000000..ebb2ad4b7ea0 --- /dev/null +++ b/drivers/gpu/Kconfig @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0 + +config GPU_BUDDY + bool + help + A page based buddy allocator for GPU memory. + +config GPU_BUDDY_KUNIT_TEST + tristate "KUnit tests for GPU buddy allocator" if !KUNIT_ALL_TESTS + depends on GPU_BUDDY && KUNIT + default KUNIT_ALL_TESTS + help + KUnit tests for the GPU buddy allocator. diff --git a/drivers/gpu/Makefile b/drivers/gpu/Makefile index 36a54d456630..b4e5e338efa2 100644 --- a/drivers/gpu/Makefile +++ b/drivers/gpu/Makefile @@ -2,7 +2,9 @@ # drm/tegra depends on host1x, so if both drivers are built-in care must be # taken to initialize them in the correct order. Link order is the only way # to ensure this currently. -obj-y += host1x/ drm/ vga/ +# Similarly, buddy must come first since it is used by other drivers. +obj-$(CONFIG_GPU_BUDDY) += buddy.o +obj-y += host1x/ drm/ vga/ tests/ obj-$(CONFIG_IMX_IPUV3_CORE) += ipu-v3/ obj-$(CONFIG_TRACE_GPU_MEM) += trace/ obj-$(CONFIG_NOVA_CORE) += nova-core/ diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c new file mode 100644 index 000000000000..bf5920bd2258 --- /dev/null +++ b/drivers/gpu/buddy.c @@ -0,0 +1,1347 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2021 Intel Corporation + */ + +#include + +#include +#include +#include +#include + +#include + +static struct kmem_cache *slab_blocks; + +static struct gpu_buddy_block *gpu_block_alloc(struct gpu_buddy *mm, + struct gpu_buddy_block *parent, + unsigned int order, + u64 offset) +{ + struct gpu_buddy_block *block; + + BUG_ON(order > GPU_BUDDY_MAX_ORDER); + + block = kmem_cache_zalloc(slab_blocks, GFP_KERNEL); + if (!block) + return NULL; + + block->header = offset; + block->header |= order; + block->parent = parent; + + RB_CLEAR_NODE(&block->rb); + + BUG_ON(block->header & GPU_BUDDY_HEADER_UNUSED); + return block; +} + +static void gpu_block_free(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + kmem_cache_free(slab_blocks, block); +} + +static enum gpu_buddy_free_tree +get_block_tree(struct gpu_buddy_block *block) +{ + return gpu_buddy_block_is_clear(block) ? + GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE; +} + +static struct gpu_buddy_block * +rbtree_get_free_block(const struct rb_node *node) +{ + return node ? rb_entry(node, struct gpu_buddy_block, rb) : NULL; +} + +static struct gpu_buddy_block * +rbtree_last_free_block(struct rb_root *root) +{ + return rbtree_get_free_block(rb_last(root)); +} + +static bool rbtree_is_empty(struct rb_root *root) +{ + return RB_EMPTY_ROOT(root); +} + +static bool gpu_buddy_block_offset_less(const struct gpu_buddy_block *block, + const struct gpu_buddy_block *node) +{ + return gpu_buddy_block_offset(block) < gpu_buddy_block_offset(node); +} + +static bool rbtree_block_offset_less(struct rb_node *block, + const struct rb_node *node) +{ + return gpu_buddy_block_offset_less(rbtree_get_free_block(block), + rbtree_get_free_block(node)); +} + +static void rbtree_insert(struct gpu_buddy *mm, + struct gpu_buddy_block *block, + enum gpu_buddy_free_tree tree) +{ + rb_add(&block->rb, + &mm->free_trees[tree][gpu_buddy_block_order(block)], + rbtree_block_offset_less); +} + +static void rbtree_remove(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + unsigned int order = gpu_buddy_block_order(block); + enum gpu_buddy_free_tree tree; + struct rb_root *root; + + tree = get_block_tree(block); + root = &mm->free_trees[tree][order]; + + rb_erase(&block->rb, root); + RB_CLEAR_NODE(&block->rb); +} + +static void clear_reset(struct gpu_buddy_block *block) +{ + block->header &= ~GPU_BUDDY_HEADER_CLEAR; +} + +static void mark_cleared(struct gpu_buddy_block *block) +{ + block->header |= GPU_BUDDY_HEADER_CLEAR; +} + +static void mark_allocated(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + block->header &= ~GPU_BUDDY_HEADER_STATE; + block->header |= GPU_BUDDY_ALLOCATED; + + rbtree_remove(mm, block); +} + +static void mark_free(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + enum gpu_buddy_free_tree tree; + + block->header &= ~GPU_BUDDY_HEADER_STATE; + block->header |= GPU_BUDDY_FREE; + + tree = get_block_tree(block); + rbtree_insert(mm, block, tree); +} + +static void mark_split(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + block->header &= ~GPU_BUDDY_HEADER_STATE; + block->header |= GPU_BUDDY_SPLIT; + + rbtree_remove(mm, block); +} + +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= e2 && e1 >= s2; +} + +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) +{ + return s1 <= s2 && e1 >= e2; +} + +static struct gpu_buddy_block * +__get_buddy(struct gpu_buddy_block *block) +{ + struct gpu_buddy_block *parent; + + parent = block->parent; + if (!parent) + return NULL; + + if (parent->left == block) + return parent->right; + + return parent->left; +} + +static unsigned int __gpu_buddy_free(struct gpu_buddy *mm, + struct gpu_buddy_block *block, + bool force_merge) +{ + struct gpu_buddy_block *parent; + unsigned int order; + + while ((parent = block->parent)) { + struct gpu_buddy_block *buddy; + + buddy = __get_buddy(block); + + if (!gpu_buddy_block_is_free(buddy)) + break; + + if (!force_merge) { + /* + * Check the block and its buddy clear state and exit + * the loop if they both have the dissimilar state. + */ + if (gpu_buddy_block_is_clear(block) != + gpu_buddy_block_is_clear(buddy)) + break; + + if (gpu_buddy_block_is_clear(block)) + mark_cleared(parent); + } + + rbtree_remove(mm, buddy); + if (force_merge && gpu_buddy_block_is_clear(buddy)) + mm->clear_avail -= gpu_buddy_block_size(mm, buddy); + + gpu_block_free(mm, block); + gpu_block_free(mm, buddy); + + block = parent; + } + + order = gpu_buddy_block_order(block); + mark_free(mm, block); + + return order; +} + +static int __force_merge(struct gpu_buddy *mm, + u64 start, + u64 end, + unsigned int min_order) +{ + unsigned int tree, order; + int i; + + if (!min_order) + return -ENOMEM; + + if (min_order > mm->max_order) + return -EINVAL; + + for_each_free_tree(tree) { + for (i = min_order - 1; i >= 0; i--) { + struct rb_node *iter = rb_last(&mm->free_trees[tree][i]); + + while (iter) { + struct gpu_buddy_block *block, *buddy; + u64 block_start, block_end; + + block = rbtree_get_free_block(iter); + iter = rb_prev(iter); + + if (!block || !block->parent) + continue; + + block_start = gpu_buddy_block_offset(block); + block_end = block_start + gpu_buddy_block_size(mm, block) - 1; + + if (!contains(start, end, block_start, block_end)) + continue; + + buddy = __get_buddy(block); + if (!gpu_buddy_block_is_free(buddy)) + continue; + + WARN_ON(gpu_buddy_block_is_clear(block) == + gpu_buddy_block_is_clear(buddy)); + + /* + * Advance to the next node when the current node is the buddy, + * as freeing the block will also remove its buddy from the tree. + */ + if (iter == &buddy->rb) + iter = rb_prev(iter); + + rbtree_remove(mm, block); + if (gpu_buddy_block_is_clear(block)) + mm->clear_avail -= gpu_buddy_block_size(mm, block); + + order = __gpu_buddy_free(mm, block, true); + if (order >= min_order) + return 0; + } + } + } + + return -ENOMEM; +} + +/** + * gpu_buddy_init - init memory manager + * + * @mm: GPU buddy manager to initialize + * @size: size in bytes to manage + * @chunk_size: minimum page size in bytes for our allocations + * + * Initializes the memory manager and its resources. + * + * Returns: + * 0 on success, error code on failure. + */ +int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size) +{ + unsigned int i, j, root_count = 0; + u64 offset = 0; + + if (size < chunk_size) + return -EINVAL; + + if (chunk_size < SZ_4K) + return -EINVAL; + + if (!is_power_of_2(chunk_size)) + return -EINVAL; + + size = round_down(size, chunk_size); + + mm->size = size; + mm->avail = size; + mm->clear_avail = 0; + mm->chunk_size = chunk_size; + mm->max_order = ilog2(size) - ilog2(chunk_size); + + BUG_ON(mm->max_order > GPU_BUDDY_MAX_ORDER); + + mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES, + sizeof(*mm->free_trees), + GFP_KERNEL); + if (!mm->free_trees) + return -ENOMEM; + + for_each_free_tree(i) { + mm->free_trees[i] = kmalloc_array(mm->max_order + 1, + sizeof(struct rb_root), + GFP_KERNEL); + if (!mm->free_trees[i]) + goto out_free_tree; + + for (j = 0; j <= mm->max_order; ++j) + mm->free_trees[i][j] = RB_ROOT; + } + + mm->n_roots = hweight64(size); + + mm->roots = kmalloc_array(mm->n_roots, + sizeof(struct gpu_buddy_block *), + GFP_KERNEL); + if (!mm->roots) + goto out_free_tree; + + /* + * Split into power-of-two blocks, in case we are given a size that is + * not itself a power-of-two. + */ + do { + struct gpu_buddy_block *root; + unsigned int order; + u64 root_size; + + order = ilog2(size) - ilog2(chunk_size); + root_size = chunk_size << order; + + root = gpu_block_alloc(mm, NULL, order, offset); + if (!root) + goto out_free_roots; + + mark_free(mm, root); + + BUG_ON(root_count > mm->max_order); + BUG_ON(gpu_buddy_block_size(mm, root) < chunk_size); + + mm->roots[root_count] = root; + + offset += root_size; + size -= root_size; + root_count++; + } while (size); + + return 0; + +out_free_roots: + while (root_count--) + gpu_block_free(mm, mm->roots[root_count]); + kfree(mm->roots); +out_free_tree: + while (i--) + kfree(mm->free_trees[i]); + kfree(mm->free_trees); + return -ENOMEM; +} +EXPORT_SYMBOL(gpu_buddy_init); + +/** + * gpu_buddy_fini - tear down the memory manager + * + * @mm: GPU buddy manager to free + * + * Cleanup memory manager resources and the freetree + */ +void gpu_buddy_fini(struct gpu_buddy *mm) +{ + u64 root_size, size, start; + unsigned int order; + int i; + + size = mm->size; + + for (i = 0; i < mm->n_roots; ++i) { + order = ilog2(size) - ilog2(mm->chunk_size); + start = gpu_buddy_block_offset(mm->roots[i]); + __force_merge(mm, start, start + size, order); + + if (WARN_ON(!gpu_buddy_block_is_free(mm->roots[i]))) + kunit_fail_current_test("buddy_fini() root"); + + gpu_block_free(mm, mm->roots[i]); + + root_size = mm->chunk_size << order; + size -= root_size; + } + + WARN_ON(mm->avail != mm->size); + + for_each_free_tree(i) + kfree(mm->free_trees[i]); + kfree(mm->free_trees); + kfree(mm->roots); +} +EXPORT_SYMBOL(gpu_buddy_fini); + +static int split_block(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + unsigned int block_order = gpu_buddy_block_order(block) - 1; + u64 offset = gpu_buddy_block_offset(block); + + BUG_ON(!gpu_buddy_block_is_free(block)); + BUG_ON(!gpu_buddy_block_order(block)); + + block->left = gpu_block_alloc(mm, block, block_order, offset); + if (!block->left) + return -ENOMEM; + + block->right = gpu_block_alloc(mm, block, block_order, + offset + (mm->chunk_size << block_order)); + if (!block->right) { + gpu_block_free(mm, block->left); + return -ENOMEM; + } + + mark_split(mm, block); + + if (gpu_buddy_block_is_clear(block)) { + mark_cleared(block->left); + mark_cleared(block->right); + clear_reset(block); + } + + mark_free(mm, block->left); + mark_free(mm, block->right); + + return 0; +} + +/** + * gpu_get_buddy - get buddy address + * + * @block: GPU buddy block + * + * Returns the corresponding buddy block for @block, or NULL + * if this is a root block and can't be merged further. + * Requires some kind of locking to protect against + * any concurrent allocate and free operations. + */ +struct gpu_buddy_block * +gpu_get_buddy(struct gpu_buddy_block *block) +{ + return __get_buddy(block); +} +EXPORT_SYMBOL(gpu_get_buddy); + +/** + * gpu_buddy_reset_clear - reset blocks clear state + * + * @mm: GPU buddy manager + * @is_clear: blocks clear state + * + * Reset the clear state based on @is_clear value for each block + * in the freetree. + */ +void gpu_buddy_reset_clear(struct gpu_buddy *mm, bool is_clear) +{ + enum gpu_buddy_free_tree src_tree, dst_tree; + u64 root_size, size, start; + unsigned int order; + int i; + + size = mm->size; + for (i = 0; i < mm->n_roots; ++i) { + order = ilog2(size) - ilog2(mm->chunk_size); + start = gpu_buddy_block_offset(mm->roots[i]); + __force_merge(mm, start, start + size, order); + + root_size = mm->chunk_size << order; + size -= root_size; + } + + src_tree = is_clear ? GPU_BUDDY_DIRTY_TREE : GPU_BUDDY_CLEAR_TREE; + dst_tree = is_clear ? GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE; + + for (i = 0; i <= mm->max_order; ++i) { + struct rb_root *root = &mm->free_trees[src_tree][i]; + struct gpu_buddy_block *block, *tmp; + + rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) { + rbtree_remove(mm, block); + if (is_clear) { + mark_cleared(block); + mm->clear_avail += gpu_buddy_block_size(mm, block); + } else { + clear_reset(block); + mm->clear_avail -= gpu_buddy_block_size(mm, block); + } + + rbtree_insert(mm, block, dst_tree); + } + } +} +EXPORT_SYMBOL(gpu_buddy_reset_clear); + +/** + * gpu_buddy_free_block - free a block + * + * @mm: GPU buddy manager + * @block: block to be freed + */ +void gpu_buddy_free_block(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + BUG_ON(!gpu_buddy_block_is_allocated(block)); + mm->avail += gpu_buddy_block_size(mm, block); + if (gpu_buddy_block_is_clear(block)) + mm->clear_avail += gpu_buddy_block_size(mm, block); + + __gpu_buddy_free(mm, block, false); +} +EXPORT_SYMBOL(gpu_buddy_free_block); + +static void __gpu_buddy_free_list(struct gpu_buddy *mm, + struct list_head *objects, + bool mark_clear, + bool mark_dirty) +{ + struct gpu_buddy_block *block, *on; + + WARN_ON(mark_dirty && mark_clear); + + list_for_each_entry_safe(block, on, objects, link) { + if (mark_clear) + mark_cleared(block); + else if (mark_dirty) + clear_reset(block); + gpu_buddy_free_block(mm, block); + cond_resched(); + } + INIT_LIST_HEAD(objects); +} + +static void gpu_buddy_free_list_internal(struct gpu_buddy *mm, + struct list_head *objects) +{ + /* + * Don't touch the clear/dirty bit, since allocation is still internal + * at this point. For example we might have just failed part of the + * allocation. + */ + __gpu_buddy_free_list(mm, objects, false, false); +} + +/** + * gpu_buddy_free_list - free blocks + * + * @mm: GPU buddy manager + * @objects: input list head to free blocks + * @flags: optional flags like GPU_BUDDY_CLEARED + */ +void gpu_buddy_free_list(struct gpu_buddy *mm, + struct list_head *objects, + unsigned int flags) +{ + bool mark_clear = flags & GPU_BUDDY_CLEARED; + + __gpu_buddy_free_list(mm, objects, mark_clear, !mark_clear); +} +EXPORT_SYMBOL(gpu_buddy_free_list); + +static bool block_incompatible(struct gpu_buddy_block *block, unsigned int flags) +{ + bool needs_clear = flags & GPU_BUDDY_CLEAR_ALLOCATION; + + return needs_clear != gpu_buddy_block_is_clear(block); +} + +static struct gpu_buddy_block * +__alloc_range_bias(struct gpu_buddy *mm, + u64 start, u64 end, + unsigned int order, + unsigned long flags, + bool fallback) +{ + u64 req_size = mm->chunk_size << order; + struct gpu_buddy_block *block; + struct gpu_buddy_block *buddy; + LIST_HEAD(dfs); + int err; + int i; + + end = end - 1; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(&dfs, + struct gpu_buddy_block, + tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + if (gpu_buddy_block_order(block) < order) + continue; + + block_start = gpu_buddy_block_offset(block); + block_end = block_start + gpu_buddy_block_size(mm, block) - 1; + + if (!overlaps(start, end, block_start, block_end)) + continue; + + if (gpu_buddy_block_is_allocated(block)) + continue; + + if (block_start < start || block_end > end) { + u64 adjusted_start = max(block_start, start); + u64 adjusted_end = min(block_end, end); + + if (round_down(adjusted_end + 1, req_size) <= + round_up(adjusted_start, req_size)) + continue; + } + + if (!fallback && block_incompatible(block, flags)) + continue; + + if (contains(start, end, block_start, block_end) && + order == gpu_buddy_block_order(block)) { + /* + * Find the free block within the range. + */ + if (gpu_buddy_block_is_free(block)) + return block; + + continue; + } + + if (!gpu_buddy_block_is_split(block)) { + err = split_block(mm, block); + if (unlikely(err)) + goto err_undo; + } + + list_add(&block->right->tmp_link, &dfs); + list_add(&block->left->tmp_link, &dfs); + } while (1); + + return ERR_PTR(-ENOSPC); + +err_undo: + /* + * We really don't want to leave around a bunch of split blocks, since + * bigger is better, so make sure we merge everything back before we + * free the allocated blocks. + */ + buddy = __get_buddy(block); + if (buddy && + (gpu_buddy_block_is_free(block) && + gpu_buddy_block_is_free(buddy))) + __gpu_buddy_free(mm, block, false); + return ERR_PTR(err); +} + +static struct gpu_buddy_block * +__gpu_buddy_alloc_range_bias(struct gpu_buddy *mm, + u64 start, u64 end, + unsigned int order, + unsigned long flags) +{ + struct gpu_buddy_block *block; + bool fallback = false; + + block = __alloc_range_bias(mm, start, end, order, + flags, fallback); + if (IS_ERR(block)) + return __alloc_range_bias(mm, start, end, order, + flags, !fallback); + + return block; +} + +static struct gpu_buddy_block * +get_maxblock(struct gpu_buddy *mm, + unsigned int order, + enum gpu_buddy_free_tree tree) +{ + struct gpu_buddy_block *max_block = NULL, *block = NULL; + struct rb_root *root; + unsigned int i; + + for (i = order; i <= mm->max_order; ++i) { + root = &mm->free_trees[tree][i]; + block = rbtree_last_free_block(root); + if (!block) + continue; + + if (!max_block) { + max_block = block; + continue; + } + + if (gpu_buddy_block_offset(block) > + gpu_buddy_block_offset(max_block)) { + max_block = block; + } + } + + return max_block; +} + +static struct gpu_buddy_block * +alloc_from_freetree(struct gpu_buddy *mm, + unsigned int order, + unsigned long flags) +{ + struct gpu_buddy_block *block = NULL; + struct rb_root *root; + enum gpu_buddy_free_tree tree; + unsigned int tmp; + int err; + + tree = (flags & GPU_BUDDY_CLEAR_ALLOCATION) ? + GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE; + + if (flags & GPU_BUDDY_TOPDOWN_ALLOCATION) { + block = get_maxblock(mm, order, tree); + if (block) + /* Store the obtained block order */ + tmp = gpu_buddy_block_order(block); + } else { + for (tmp = order; tmp <= mm->max_order; ++tmp) { + /* Get RB tree root for this order and tree */ + root = &mm->free_trees[tree][tmp]; + block = rbtree_last_free_block(root); + if (block) + break; + } + } + + if (!block) { + /* Try allocating from the other tree */ + tree = (tree == GPU_BUDDY_CLEAR_TREE) ? + GPU_BUDDY_DIRTY_TREE : GPU_BUDDY_CLEAR_TREE; + + for (tmp = order; tmp <= mm->max_order; ++tmp) { + root = &mm->free_trees[tree][tmp]; + block = rbtree_last_free_block(root); + if (block) + break; + } + + if (!block) + return ERR_PTR(-ENOSPC); + } + + BUG_ON(!gpu_buddy_block_is_free(block)); + + while (tmp != order) { + err = split_block(mm, block); + if (unlikely(err)) + goto err_undo; + + block = block->right; + tmp--; + } + return block; + +err_undo: + if (tmp != order) + __gpu_buddy_free(mm, block, false); + return ERR_PTR(err); +} + +static int __alloc_range(struct gpu_buddy *mm, + struct list_head *dfs, + u64 start, u64 size, + struct list_head *blocks, + u64 *total_allocated_on_err) +{ + struct gpu_buddy_block *block; + struct gpu_buddy_block *buddy; + u64 total_allocated = 0; + LIST_HEAD(allocated); + u64 end; + int err; + + end = start + size - 1; + + do { + u64 block_start; + u64 block_end; + + block = list_first_entry_or_null(dfs, + struct gpu_buddy_block, + tmp_link); + if (!block) + break; + + list_del(&block->tmp_link); + + block_start = gpu_buddy_block_offset(block); + block_end = block_start + gpu_buddy_block_size(mm, block) - 1; + + if (!overlaps(start, end, block_start, block_end)) + continue; + + if (gpu_buddy_block_is_allocated(block)) { + err = -ENOSPC; + goto err_free; + } + + if (contains(start, end, block_start, block_end)) { + if (gpu_buddy_block_is_free(block)) { + mark_allocated(mm, block); + total_allocated += gpu_buddy_block_size(mm, block); + mm->avail -= gpu_buddy_block_size(mm, block); + if (gpu_buddy_block_is_clear(block)) + mm->clear_avail -= gpu_buddy_block_size(mm, block); + list_add_tail(&block->link, &allocated); + continue; + } else if (!mm->clear_avail) { + err = -ENOSPC; + goto err_free; + } + } + + if (!gpu_buddy_block_is_split(block)) { + err = split_block(mm, block); + if (unlikely(err)) + goto err_undo; + } + + list_add(&block->right->tmp_link, dfs); + list_add(&block->left->tmp_link, dfs); + } while (1); + + if (total_allocated < size) { + err = -ENOSPC; + goto err_free; + } + + list_splice_tail(&allocated, blocks); + + return 0; + +err_undo: + /* + * We really don't want to leave around a bunch of split blocks, since + * bigger is better, so make sure we merge everything back before we + * free the allocated blocks. + */ + buddy = __get_buddy(block); + if (buddy && + (gpu_buddy_block_is_free(block) && + gpu_buddy_block_is_free(buddy))) + __gpu_buddy_free(mm, block, false); + +err_free: + if (err == -ENOSPC && total_allocated_on_err) { + list_splice_tail(&allocated, blocks); + *total_allocated_on_err = total_allocated; + } else { + gpu_buddy_free_list_internal(mm, &allocated); + } + + return err; +} + +static int __gpu_buddy_alloc_range(struct gpu_buddy *mm, + u64 start, + u64 size, + u64 *total_allocated_on_err, + struct list_head *blocks) +{ + LIST_HEAD(dfs); + int i; + + for (i = 0; i < mm->n_roots; ++i) + list_add_tail(&mm->roots[i]->tmp_link, &dfs); + + return __alloc_range(mm, &dfs, start, size, + blocks, total_allocated_on_err); +} + +static int __alloc_contig_aligned_retry(struct gpu_buddy *mm, + u64 unaligned_offset, + u64 size, + u64 min_block_size, + struct list_head *blocks) +{ + u64 aligned_offset = round_down(unaligned_offset, min_block_size); + + return __gpu_buddy_alloc_range(mm, aligned_offset, size, NULL, blocks); +} + +static int __alloc_contig_try_harder(struct gpu_buddy *mm, + u64 size, + u64 min_block_size, + struct list_head *blocks) +{ + u64 rhs_offset, lhs_offset, filled; + struct gpu_buddy_block *block; + unsigned int tree, order; + u64 modify_size; + int err; + + modify_size = rounddown_pow_of_two(size); + order = ilog2(modify_size) - ilog2(mm->chunk_size); + if (order == 0) + return -ENOSPC; + + for_each_free_tree(tree) { + struct rb_root *root; + struct rb_node *iter; + + root = &mm->free_trees[tree][order]; + if (rbtree_is_empty(root)) + continue; + + iter = rb_last(root); + while (iter) { + block = rbtree_get_free_block(iter); + + rhs_offset = gpu_buddy_block_offset(block); + + /* Allocate blocks traversing RHS */ + err = __gpu_buddy_alloc_range(mm, rhs_offset, size, + &filled, blocks); + if (err && err != -ENOSPC) + return err; + if (!err && IS_ALIGNED(rhs_offset, min_block_size)) + return 0; + if (!err) { + /* Allocate the unaligned RHS offset using round_down */ + gpu_buddy_free_list_internal(mm, blocks); + err = __alloc_contig_aligned_retry(mm, rhs_offset, + size, + min_block_size, + blocks); + if (!err) + return 0; + if (err != -ENOSPC) { + gpu_buddy_free_list_internal(mm, blocks); + return err; + } + goto next; + } + + if (size - filled > rhs_offset) + goto next; + + lhs_offset = rhs_offset - (size - filled); + + /* Allocate the unaligned LHS offset using round_down */ + gpu_buddy_free_list_internal(mm, blocks); + err = __alloc_contig_aligned_retry(mm, lhs_offset, size, + min_block_size, blocks); + if (!err) + return 0; + if (err != -ENOSPC) { + gpu_buddy_free_list_internal(mm, blocks); + return err; + } +next: + gpu_buddy_free_list_internal(mm, blocks); + iter = rb_prev(iter); + } + } + + return -ENOSPC; +} + +/** + * gpu_buddy_block_trim - free unused pages + * + * @mm: GPU buddy manager + * @start: start address to begin the trimming. + * @new_size: original size requested + * @blocks: Input and output list of allocated blocks. + * MUST contain single block as input to be trimmed. + * On success will contain the newly allocated blocks + * making up the @new_size. Blocks always appear in + * ascending order + * + * For contiguous allocation, we round up the size to the nearest + * power of two value, drivers consume *actual* size, so remaining + * portions are unused and can be optionally freed with this function + * + * Returns: + * 0 on success, error code on failure. + */ +int gpu_buddy_block_trim(struct gpu_buddy *mm, + u64 *start, + u64 new_size, + struct list_head *blocks) +{ + struct gpu_buddy_block *parent; + struct gpu_buddy_block *block; + u64 block_start, block_end; + LIST_HEAD(dfs); + u64 new_start; + int err; + + if (!list_is_singular(blocks)) + return -EINVAL; + + block = list_first_entry(blocks, + struct gpu_buddy_block, + link); + + block_start = gpu_buddy_block_offset(block); + block_end = block_start + gpu_buddy_block_size(mm, block); + + if (WARN_ON(!gpu_buddy_block_is_allocated(block))) + return -EINVAL; + + if (new_size > gpu_buddy_block_size(mm, block)) + return -EINVAL; + + if (!new_size || !IS_ALIGNED(new_size, mm->chunk_size)) + return -EINVAL; + + if (new_size == gpu_buddy_block_size(mm, block)) + return 0; + + new_start = block_start; + if (start) { + new_start = *start; + + if (new_start < block_start) + return -EINVAL; + + if (!IS_ALIGNED(new_start, mm->chunk_size)) + return -EINVAL; + + if (range_overflows(new_start, new_size, block_end)) + return -EINVAL; + } + + list_del(&block->link); + mark_free(mm, block); + mm->avail += gpu_buddy_block_size(mm, block); + if (gpu_buddy_block_is_clear(block)) + mm->clear_avail += gpu_buddy_block_size(mm, block); + + /* Prevent recursively freeing this node */ + parent = block->parent; + block->parent = NULL; + + list_add(&block->tmp_link, &dfs); + err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); + if (err) { + mark_allocated(mm, block); + mm->avail -= gpu_buddy_block_size(mm, block); + if (gpu_buddy_block_is_clear(block)) + mm->clear_avail -= gpu_buddy_block_size(mm, block); + list_add(&block->link, blocks); + } + + block->parent = parent; + return err; +} +EXPORT_SYMBOL(gpu_buddy_block_trim); + +static struct gpu_buddy_block * +__gpu_buddy_alloc_blocks(struct gpu_buddy *mm, + u64 start, u64 end, + unsigned int order, + unsigned long flags) +{ + if (flags & GPU_BUDDY_RANGE_ALLOCATION) + /* Allocate traversing within the range */ + return __gpu_buddy_alloc_range_bias(mm, start, end, + order, flags); + else + /* Allocate from freetree */ + return alloc_from_freetree(mm, order, flags); +} + +/** + * gpu_buddy_alloc_blocks - allocate power-of-two blocks + * + * @mm: GPU buddy manager to allocate from + * @start: start of the allowed range for this block + * @end: end of the allowed range for this block + * @size: size of the allocation in bytes + * @min_block_size: alignment of the allocation + * @blocks: output list head to add allocated blocks + * @flags: GPU_BUDDY_*_ALLOCATION flags + * + * alloc_range_bias() called on range limitations, which traverses + * the tree and returns the desired block. + * + * alloc_from_freetree() called when *no* range restrictions + * are enforced, which picks the block from the freetree. + * + * Returns: + * 0 on success, error code on failure. + */ +int gpu_buddy_alloc_blocks(struct gpu_buddy *mm, + u64 start, u64 end, u64 size, + u64 min_block_size, + struct list_head *blocks, + unsigned long flags) +{ + struct gpu_buddy_block *block = NULL; + u64 original_size, original_min_size; + unsigned int min_order, order; + LIST_HEAD(allocated); + unsigned long pages; + int err; + + if (size < mm->chunk_size) + return -EINVAL; + + if (min_block_size < mm->chunk_size) + return -EINVAL; + + if (!is_power_of_2(min_block_size)) + return -EINVAL; + + if (!IS_ALIGNED(start | end | size, mm->chunk_size)) + return -EINVAL; + + if (end > mm->size) + return -EINVAL; + + if (range_overflows(start, size, mm->size)) + return -EINVAL; + + /* Actual range allocation */ + if (start + size == end) { + if (!IS_ALIGNED(start | end, min_block_size)) + return -EINVAL; + + return __gpu_buddy_alloc_range(mm, start, size, NULL, blocks); + } + + original_size = size; + original_min_size = min_block_size; + + /* Roundup the size to power of 2 */ + if (flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION) { + size = roundup_pow_of_two(size); + min_block_size = size; + /* Align size value to min_block_size */ + } else if (!IS_ALIGNED(size, min_block_size)) { + size = round_up(size, min_block_size); + } + + pages = size >> ilog2(mm->chunk_size); + order = fls(pages) - 1; + min_order = ilog2(min_block_size) - ilog2(mm->chunk_size); + + if (order > mm->max_order || size > mm->size) { + if ((flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION) && + !(flags & GPU_BUDDY_RANGE_ALLOCATION)) + return __alloc_contig_try_harder(mm, original_size, + original_min_size, blocks); + + return -EINVAL; + } + + do { + order = min(order, (unsigned int)fls(pages) - 1); + BUG_ON(order > mm->max_order); + BUG_ON(order < min_order); + + do { + block = __gpu_buddy_alloc_blocks(mm, start, + end, + order, + flags); + if (!IS_ERR(block)) + break; + + if (order-- == min_order) { + /* Try allocation through force merge method */ + if (mm->clear_avail && + !__force_merge(mm, start, end, min_order)) { + block = __gpu_buddy_alloc_blocks(mm, start, + end, + min_order, + flags); + if (!IS_ERR(block)) { + order = min_order; + break; + } + } + + /* + * Try contiguous block allocation through + * try harder method. + */ + if (flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION && + !(flags & GPU_BUDDY_RANGE_ALLOCATION)) + return __alloc_contig_try_harder(mm, + original_size, + original_min_size, + blocks); + err = -ENOSPC; + goto err_free; + } + } while (1); + + mark_allocated(mm, block); + mm->avail -= gpu_buddy_block_size(mm, block); + if (gpu_buddy_block_is_clear(block)) + mm->clear_avail -= gpu_buddy_block_size(mm, block); + kmemleak_update_trace(block); + list_add_tail(&block->link, &allocated); + + pages -= BIT(order); + + if (!pages) + break; + } while (1); + + /* Trim the allocated block to the required size */ + if (!(flags & GPU_BUDDY_TRIM_DISABLE) && + original_size != size) { + struct list_head *trim_list; + LIST_HEAD(temp); + u64 trim_size; + + trim_list = &allocated; + trim_size = original_size; + + if (!list_is_singular(&allocated)) { + block = list_last_entry(&allocated, typeof(*block), link); + list_move(&block->link, &temp); + trim_list = &temp; + trim_size = gpu_buddy_block_size(mm, block) - + (size - original_size); + } + + gpu_buddy_block_trim(mm, + NULL, + trim_size, + trim_list); + + if (!list_empty(&temp)) + list_splice_tail(trim_list, &allocated); + } + + list_splice_tail(&allocated, blocks); + return 0; + +err_free: + gpu_buddy_free_list_internal(mm, &allocated); + return err; +} +EXPORT_SYMBOL(gpu_buddy_alloc_blocks); + +/** + * gpu_buddy_block_print - print block information + * + * @mm: GPU buddy manager + * @block: GPU buddy block + */ +void gpu_buddy_block_print(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + u64 start = gpu_buddy_block_offset(block); + u64 size = gpu_buddy_block_size(mm, block); + + pr_info("%#018llx-%#018llx: %llu\n", start, start + size, size); +} +EXPORT_SYMBOL(gpu_buddy_block_print); + +/** + * gpu_buddy_print - print allocator state + * + * @mm: GPU buddy manager + * @p: GPU printer to use + */ +void gpu_buddy_print(struct gpu_buddy *mm) +{ + int order; + + pr_info("chunk_size: %lluKiB, total: %lluMiB, free: %lluMiB, clear_free: %lluMiB\n", + mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20); + + for (order = mm->max_order; order >= 0; order--) { + struct gpu_buddy_block *block, *tmp; + struct rb_root *root; + u64 count = 0, free; + unsigned int tree; + + for_each_free_tree(tree) { + root = &mm->free_trees[tree][order]; + + rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) { + BUG_ON(!gpu_buddy_block_is_free(block)); + count++; + } + } + + free = count * (mm->chunk_size << order); + if (free < SZ_1M) + pr_info("order-%2d free: %8llu KiB, blocks: %llu\n", + order, free >> 10, count); + else + pr_info("order-%2d free: %8llu MiB, blocks: %llu\n", + order, free >> 20, count); + } +} +EXPORT_SYMBOL(gpu_buddy_print); + +static void gpu_buddy_module_exit(void) +{ + kmem_cache_destroy(slab_blocks); +} + +static int __init gpu_buddy_module_init(void) +{ + slab_blocks = KMEM_CACHE(gpu_buddy_block, 0); + if (!slab_blocks) + return -ENOMEM; + + return 0; +} + +module_init(gpu_buddy_module_init); +module_exit(gpu_buddy_module_exit); + +MODULE_DESCRIPTION("GPU Buddy Allocator"); +MODULE_LICENSE("Dual MIT/GPL"); diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index ed85d0ceee3b..4b0228c88f49 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -220,6 +220,7 @@ config DRM_GPUSVM config DRM_BUDDY tristate depends on DRM + select GPU_BUDDY help A page based buddy allocator @@ -416,10 +417,6 @@ config DRM_HYPERV config DRM_PANEL_BACKLIGHT_QUIRKS tristate -config DRM_LIB_RANDOM - bool - default n - config DRM_PRIVACY_SCREEN bool default n diff --git a/drivers/gpu/drm/Kconfig.debug b/drivers/gpu/drm/Kconfig.debug index 05dc43c0b8c5..3b7886865335 100644 --- a/drivers/gpu/drm/Kconfig.debug +++ b/drivers/gpu/drm/Kconfig.debug @@ -69,7 +69,6 @@ config DRM_KUNIT_TEST select DRM_EXPORT_FOR_TESTS if m select DRM_GEM_SHMEM_HELPER select DRM_KUNIT_TEST_HELPERS - select DRM_LIB_RANDOM select DRM_SYSFB_HELPER select PRIME_NUMBERS default KUNIT_ALL_TESTS diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile index b248e64587ed..1b3a7ab6df58 100644 --- a/drivers/gpu/drm/Makefile +++ b/drivers/gpu/drm/Makefile @@ -77,7 +77,6 @@ drm-$(CONFIG_DRM_CLIENT) += \ drm_client.o \ drm_client_event.o \ drm_client_modeset.o -drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o drm-$(CONFIG_COMPAT) += drm_ioc32.o drm-$(CONFIG_DRM_PANEL) += drm_panel.o drm-$(CONFIG_OF) += drm_of.o @@ -239,7 +238,7 @@ obj-$(CONFIG_DRM_POWERVR) += imagination/ # Ensure drm headers are self-contained and pass kernel-doc hdrtest-files := \ $(shell cd $(src) && find . -maxdepth 1 -name 'drm_*.h') \ - $(shell cd $(src) && find display lib -name '*.h') + $(shell cd $(src) && find display -name '*.h') always-$(CONFIG_DRM_HEADER_TEST) += \ $(patsubst %.h,%.hdrtest, $(hdrtest-files)) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c index 4926996f94da..fe39b150ee53 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c @@ -505,6 +505,7 @@ static int acp_hw_fini(struct amdgpu_ip_block *ip_block) u32 val = 0; u32 count = 0; struct amdgpu_device *adev = ip_block->adev; + int ret = 0; /* return early if no ACP */ if (!adev->acp.acp_genpd) { @@ -526,7 +527,8 @@ static int acp_hw_fini(struct amdgpu_ip_block *ip_block) break; if (--count == 0) { dev_err(&adev->pdev->dev, "Failed to reset ACP\n"); - return -ETIMEDOUT; + ret = -ETIMEDOUT; + goto out; } udelay(100); } @@ -543,20 +545,23 @@ static int acp_hw_fini(struct amdgpu_ip_block *ip_block) break; if (--count == 0) { dev_err(&adev->pdev->dev, "Failed to reset ACP\n"); - return -ETIMEDOUT; + ret = -ETIMEDOUT; + goto out; } udelay(100); } - +out: device_for_each_child(adev->acp.parent, NULL, acp_genpd_remove_device); mfd_remove_devices(adev->acp.parent); kfree(adev->acp.acp_res); + pm_genpd_remove(&adev->acp.acp_genpd->gpd); kfree(adev->acp.acp_genpd); + adev->acp.acp_genpd = NULL; kfree(adev->acp.acp_cell); - return 0; + return ret; } static int acp_suspend(struct amdgpu_ip_block *ip_block) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c index 00e96419fcda..4e60753e16e9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c @@ -355,19 +355,59 @@ static bool amdgpu_read_disabled_bios(struct amdgpu_device *adev) } #ifdef CONFIG_ACPI +/** + * amdgpu_acpi_vfct_match() - Check if a VFCT entry matches the device + * @adev: AMDGPU device + * @vhdr: VFCT image header to check + * + * VFCT entries contain the PCI bus number as recorded during BIOS POST. + * On systems where the kernel renumbers PCI buses (e.g. pci=realloc or + * resource conflicts), the runtime bus number may differ from the POST + * value. Match by device identity (vendor + device + function) and use + * the bus number as a preference: exact bus match is preferred, but when + * the bus numbers disagree we accept the entry if the device identity + * matches. + * + * Returns: 0 on match, -ENODEV on no match + */ +static int amdgpu_acpi_vfct_match(struct amdgpu_device *adev, + VFCT_IMAGE_HEADER *vhdr) +{ + /* Vendor and device IDs must always match */ + if (vhdr->VendorID != adev->pdev->vendor || + vhdr->DeviceID != adev->pdev->device) + return -ENODEV; + + if (vhdr->PCIDevice != PCI_SLOT(adev->pdev->devfn) || + vhdr->PCIFunction != PCI_FUNC(adev->pdev->devfn)) + return -ENODEV; + + /* Exact bus number match - preferred */ + if (vhdr->PCIBus == adev->pdev->bus->number) + return 0; + + /* Bus mismatch but device identity matches (PCI renumbering case) */ + dev_notice(adev->dev, + "VFCT bus number mismatch: table %u != runtime %u, matching by device identity (vendor 0x%04x device 0x%04x)\n", + vhdr->PCIBus, adev->pdev->bus->number, + adev->pdev->vendor, adev->pdev->device); + return 0; +} + static bool amdgpu_acpi_vfct_bios(struct amdgpu_device *adev) { struct acpi_table_header *hdr; acpi_size tbl_size; UEFI_ACPI_VFCT *vfct; unsigned int offset; + bool r = false; if (!ACPI_SUCCESS(acpi_get_table("VFCT", 1, &hdr))) return false; tbl_size = hdr->length; if (tbl_size < sizeof(UEFI_ACPI_VFCT)) { dev_info(adev->dev, "ACPI VFCT table present but broken (too short #1),skipping\n"); - return false; + goto out; } vfct = (UEFI_ACPI_VFCT *)hdr; @@ -380,36 +420,36 @@ static bool amdgpu_acpi_vfct_bios(struct amdgpu_device *adev) offset += sizeof(VFCT_IMAGE_HEADER); if (offset > tbl_size) { dev_info(adev->dev, "ACPI VFCT image header truncated,skipping\n"); - return false; + goto out; } offset += vhdr->ImageLength; if (offset > tbl_size) { dev_info(adev->dev, "ACPI VFCT image truncated,skipping\n"); - return false; + goto out; } if (vhdr->ImageLength && - vhdr->PCIBus == adev->pdev->bus->number && - vhdr->PCIDevice == PCI_SLOT(adev->pdev->devfn) && - vhdr->PCIFunction == PCI_FUNC(adev->pdev->devfn) && - vhdr->VendorID == adev->pdev->vendor && - vhdr->DeviceID == adev->pdev->device) { + !amdgpu_acpi_vfct_match(adev, vhdr)) { adev->bios = kmemdup(&vbios->VbiosContent, vhdr->ImageLength, GFP_KERNEL); if (!check_atom_bios(adev, vhdr->ImageLength)) { amdgpu_bios_release(adev); - return false; + goto out; } adev->bios_size = vhdr->ImageLength; - return true; + r = true; + goto out; } } dev_info(adev->dev, "ACPI VFCT table present but broken (too short #2),skipping\n"); - return false; + +out: + acpi_put_table(hdr); + return r; } #else static inline bool amdgpu_acpi_vfct_bios(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 8669e2fd6eff..4b449b58d907 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -260,13 +260,17 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, goto free_partial_kdata; break; + case AMDGPU_CHUNK_ID_CP_GFX_SHADOW: + if (size < sizeof(struct drm_amdgpu_cs_chunk_cp_gfx_shadow)) + goto free_partial_kdata; + break; + case AMDGPU_CHUNK_ID_DEPENDENCIES: case AMDGPU_CHUNK_ID_SYNCOBJ_IN: case AMDGPU_CHUNK_ID_SYNCOBJ_OUT: case AMDGPU_CHUNK_ID_SCHEDULED_DEPENDENCIES: case AMDGPU_CHUNK_ID_SYNCOBJ_TIMELINE_WAIT: case AMDGPU_CHUNK_ID_SYNCOBJ_TIMELINE_SIGNAL: - case AMDGPU_CHUNK_ID_CP_GFX_SHADOW: break; default: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index d0cb300cd382..2c55925895f9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -1878,6 +1878,15 @@ static bool amdgpu_device_pcie_dynamic_switching_supported(struct amdgpu_device if (c->x86_vendor == X86_VENDOR_INTEL) return false; + + /* + * AMD Ryzen Pinnacle Ridge (Zen+, family 0x17 model 0x08) CPUs don't + * support PCIe dynamic speed switching. + * https://gitlab.freedesktop.org/drm/amd/-/work_items/5436 + */ + if (c->x86_vendor == X86_VENDOR_AMD && c->x86 == 0x17 && + c->x86_model == 0x08) + return false; #endif return true; } @@ -1888,7 +1897,8 @@ static bool amdgpu_device_aspm_support_quirk(struct amdgpu_device *adev) * It's unclear if this is a platform-specific or GPU-specific issue. * Disable ASPM on SI for the time being. */ - if (adev->family == AMDGPU_FAMILY_SI) + if (adev->family == AMDGPU_FAMILY_SI || + (!(adev->pm.pp_feature & PP_PCIE_DPM_MASK) && adev->family == AMDGPU_FAMILY_VI)) return true; #if IS_ENABLED(CONFIG_X86) @@ -4936,8 +4946,6 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) iounmap(adev->rmmio); adev->rmmio = NULL; - if (adev->mman.aper_base_kaddr) - iounmap(adev->mman.aper_base_kaddr); adev->mman.aper_base_kaddr = NULL; /* Memory manager related */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index cd52ddd27c58..5fc9a6b1ec72 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -523,6 +523,7 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data, bo = gem_to_amdgpu_bo(gobj); bo->preferred_domains = AMDGPU_GEM_DOMAIN_GTT; bo->allowed_domains = AMDGPU_GEM_DOMAIN_GTT; + bo->parent = amdgpu_bo_ref(fpriv->vm.root.bo); r = amdgpu_ttm_tt_set_userptr(&bo->tbo, args->addr, args->flags); if (r) goto release_object; @@ -1075,6 +1076,11 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data, * If that number is larger than the size of the array, the ioctl must * be retried. */ + if (!bo_va) { + r = -ENOENT; + goto out_exec; + } + if (args->num_entries > INT_MAX / sizeof(*vm_entries)) { r = -EINVAL; goto out_exec; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c index ebe2b4c68b0f..40e748298069 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c @@ -1456,12 +1456,13 @@ static int amdgpu_gfx_run_cleaner_shader_job(struct amdgpu_ring *ring) struct amdgpu_device *adev = ring->adev; struct drm_gpu_scheduler *sched = &ring->sched; struct drm_sched_entity entity; + unsigned int ib_size_dw = 16; static atomic_t counter; struct dma_fence *f; struct amdgpu_job *job; struct amdgpu_ib *ib; void *owner; - int i, r; + int r; /* Initialize the scheduler entity */ r = drm_sched_entity_init(&entity, DRM_SCHED_PRIORITY_NORMAL, @@ -1479,7 +1480,7 @@ static int amdgpu_gfx_run_cleaner_shader_job(struct amdgpu_ring *ring) owner = (void *)(unsigned long)atomic_inc_return(&counter); r = amdgpu_job_alloc_with_ib(ring->adev, &entity, owner, - 64, 0, &job, + ib_size_dw * sizeof(uint32_t), 0, &job, AMDGPU_KERNEL_JOB_ID_CLEANER_SHADER); if (r) goto err; @@ -1489,9 +1490,8 @@ static int amdgpu_gfx_run_cleaner_shader_job(struct amdgpu_ring *ring) job->run_cleaner_shader = true; ib = &job->ibs[0]; - for (i = 0; i <= ring->funcs->align_mask; ++i) - ib->ptr[i] = ring->funcs->nop; - ib->length_dw = ring->funcs->align_mask + 1; + memset32(ib->ptr, ring->funcs->nop, ib_size_dw); + ib->length_dw = ib_size_dw; f = amdgpu_job_submit(job); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c index ccda3b58a455..f60f26470af0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.c @@ -69,7 +69,6 @@ static bool amdgpu_hmm_invalidate_gfx(struct mmu_interval_notifier *mni, { struct amdgpu_bo *bo = container_of(mni, struct amdgpu_bo, notifier); struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); - struct amdgpu_bo *vm_root = bo->vm_bo->vm->root.bo; long r; if (!mmu_notifier_range_blockable(range)) @@ -80,7 +79,7 @@ static bool amdgpu_hmm_invalidate_gfx(struct mmu_interval_notifier *mni, mmu_interval_set_seq(mni, cur_seq); amdgpu_vm_bo_invalidate(bo, false); - r = dma_resv_wait_timeout(vm_root->tbo.base.resv, + r = dma_resv_wait_timeout(bo->parent->tbo.base.resv, DMA_RESV_USAGE_BOOKKEEP, false, MAX_SCHEDULE_TIMEOUT); mutex_unlock(&adev->notifier_lock); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index e08f58de4b17..cdab2c098cfa 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -284,10 +284,12 @@ int amdgpu_bo_create_reserved(struct amdgpu_device *adev, goto error_free; } - r = amdgpu_bo_pin(*bo_ptr, domain); - if (r) { - dev_err(adev->dev, "(%d) kernel bo pin failed\n", r); - goto error_unreserve; + if (free) { + r = amdgpu_bo_pin(*bo_ptr, domain); + if (r) { + dev_err(adev->dev, "(%d) kernel bo pin failed\n", r); + goto error_unreserve; + } } r = amdgpu_ttm_alloc_gart(&(*bo_ptr)->tbo); @@ -310,7 +312,8 @@ int amdgpu_bo_create_reserved(struct amdgpu_device *adev, return 0; error_unpin: - amdgpu_bo_unpin(*bo_ptr); + if (free) + amdgpu_bo_unpin(*bo_ptr); error_unreserve: amdgpu_bo_unreserve(*bo_ptr); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index e9ed1a3b135c..9df691364a18 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -5416,7 +5416,7 @@ int amdgpu_ras_add_critical_region(struct amdgpu_device *adev, struct amdgpu_ras *con = amdgpu_ras_get_context(adev); struct amdgpu_vram_mgr_resource *vres; struct ras_critical_region *region; - struct drm_buddy_block *block; + struct gpu_buddy_block *block; int ret = 0; if (!bo || !bo->tbo.resource) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h index be2e56ce1355..8908d9e08a30 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h @@ -55,7 +55,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res, uint64_t start, uint64_t size, struct amdgpu_res_cursor *cur) { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; struct list_head *head, *next; struct drm_mm_node *node; @@ -71,7 +71,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res, head = &to_amdgpu_vram_mgr_resource(res)->blocks; block = list_first_entry_or_null(head, - struct drm_buddy_block, + struct gpu_buddy_block, link); if (!block) goto fallback; @@ -81,7 +81,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res, next = block->link.next; if (next != head) - block = list_entry(next, struct drm_buddy_block, link); + block = list_entry(next, struct gpu_buddy_block, link); } cur->start = amdgpu_vram_mgr_block_start(block) + start; @@ -125,7 +125,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res, */ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size) { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; struct drm_mm_node *node; struct list_head *next; @@ -146,7 +146,7 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size) block = cur->node; next = block->link.next; - block = list_entry(next, struct drm_buddy_block, link); + block = list_entry(next, struct gpu_buddy_block, link); cur->node = block; cur->start = amdgpu_vram_mgr_block_start(block); @@ -175,7 +175,7 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size) */ static inline bool amdgpu_res_cleared(struct amdgpu_res_cursor *cur) { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; switch (cur->mem_type) { case TTM_PL_VRAM: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index d629c5f73bf5..a4b5fe35c3cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -505,6 +505,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo, bool evict, if (new_mem->mem_type == TTM_PL_TT || new_mem->mem_type == AMDGPU_PL_PREEMPT) { + if (old_mem && (old_mem->mem_type == TTM_PL_TT || + old_mem->mem_type == AMDGPU_PL_PREEMPT)) { + r = ttm_bo_wait_ctx(bo, ctx); + if (r) + return r; + + amdgpu_ttm_backend_unbind(bo->bdev, bo->ttm); + } + r = amdgpu_ttm_backend_bind(bo->bdev, bo->ttm, new_mem); if (r) return r; @@ -539,6 +548,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo, bool evict, ttm_bo_assign_mem(bo, new_mem); return 0; } + if ((old_mem->mem_type == TTM_PL_TT || + old_mem->mem_type == AMDGPU_PL_PREEMPT) && + (new_mem->mem_type == TTM_PL_TT || + new_mem->mem_type == AMDGPU_PL_PREEMPT)) { + amdgpu_bo_move_notify(bo, evict, new_mem); + ttm_resource_free(bo, &bo->resource); + ttm_bo_assign_mem(bo, new_mem); + return 0; + } if (old_mem->mem_type == AMDGPU_PL_GDS || old_mem->mem_type == AMDGPU_PL_GWS || @@ -1962,18 +1980,23 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) /* Change the size here instead of the init above so only lpfn is affected */ amdgpu_ttm_set_buffer_funcs_status(adev, false); #ifdef CONFIG_64BIT -#ifdef CONFIG_X86 - if (adev->gmc.xgmi.connected_to_cpu) - adev->mman.aper_base_kaddr = ioremap_cache(adev->gmc.aper_base, - adev->gmc.visible_vram_size); - - else if (adev->gmc.is_app_apu) + if (adev->gmc.xgmi.connected_to_cpu) { + void *kaddr = devm_memremap(adev->dev, adev->gmc.aper_base, + adev->gmc.visible_vram_size, + MEMREMAP_WB); + if (IS_ERR(kaddr)) + return PTR_ERR(kaddr); + adev->mman.aper_base_kaddr = (__force void __iomem *)kaddr; + } else if (adev->gmc.is_app_apu) { DRM_DEBUG_DRIVER( "No need to ioremap when real vram size is 0\n"); - else -#endif - adev->mman.aper_base_kaddr = ioremap_wc(adev->gmc.aper_base, - adev->gmc.visible_vram_size); + } else { + adev->mman.aper_base_kaddr = devm_ioremap_wc(adev->dev, + adev->gmc.aper_base, + adev->gmc.visible_vram_size); + if (!adev->mman.aper_base_kaddr) + return -ENOMEM; + } #endif /* @@ -2134,8 +2157,6 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) */ void amdgpu_ttm_fini(struct amdgpu_device *adev) { - int idx; - if (!adev->mman.initialized) return; @@ -2162,14 +2183,7 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) amdgpu_ttm_fw_reserve_vram_fini(adev); amdgpu_ttm_drv_reserve_vram_fini(adev); - if (drm_dev_enter(adev_to_drm(adev), &idx)) { - - if (adev->mman.aper_base_kaddr) - iounmap(adev->mman.aper_base_kaddr); - adev->mman.aper_base_kaddr = NULL; - - drm_dev_exit(idx); - } + adev->mman.aper_base_kaddr = NULL; if (!adev->gmc.is_app_apu) amdgpu_vram_mgr_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index 5c38f0d30c87..5f1199b592b0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -135,7 +135,7 @@ MODULE_FIRMWARE(FIRMWARE_VEGA12); MODULE_FIRMWARE(FIRMWARE_VEGA20); static void amdgpu_uvd_idle_work_handler(struct work_struct *work); -static void amdgpu_uvd_force_into_uvd_segment(struct amdgpu_bo *abo); +static void amdgpu_uvd_force_into_vcpu_segment(struct amdgpu_bo *abo); static int amdgpu_uvd_create_msg_bo_helper(struct amdgpu_device *adev, uint32_t size, @@ -158,7 +158,7 @@ static int amdgpu_uvd_create_msg_bo_helper(struct amdgpu_device *adev, amdgpu_bo_kunmap(bo); amdgpu_bo_unpin(bo); amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_VRAM); - amdgpu_uvd_force_into_uvd_segment(bo); + amdgpu_uvd_force_into_vcpu_segment(bo); r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); if (r) goto err; @@ -188,6 +188,7 @@ int amdgpu_uvd_sw_init(struct amdgpu_device *adev) const struct common_firmware_header *hdr; unsigned int family_id; int i, j, r; + u32 vcpu_bo_domain; INIT_DELAYED_WORK(&adev->uvd.idle_work, amdgpu_uvd_idle_work_handler); @@ -319,12 +320,20 @@ int amdgpu_uvd_sw_init(struct amdgpu_device *adev) if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) bo_size += AMDGPU_GPU_PAGE_ALIGN(le32_to_cpu(hdr->ucode_size_bytes) + 8); + /* UVD 5.0 and newer HW can use 64 bit addressing. */ + adev->uvd.address_64_bit = + !amdgpu_device_ip_block_version_cmp(adev, AMD_IP_BLOCK_TYPE_UVD, 5, 0); + + vcpu_bo_domain = AMDGPU_GEM_DOMAIN_VRAM; + if (adev->uvd.address_64_bit) + vcpu_bo_domain |= AMDGPU_GEM_DOMAIN_GTT; + for (j = 0; j < adev->uvd.num_uvd_inst; j++) { if (adev->uvd.harvest_config & (1 << j)) continue; + r = amdgpu_bo_create_kernel(adev, bo_size, PAGE_SIZE, - AMDGPU_GEM_DOMAIN_VRAM | - AMDGPU_GEM_DOMAIN_GTT, + vcpu_bo_domain, &adev->uvd.inst[j].vcpu_bo, &adev->uvd.inst[j].gpu_addr, &adev->uvd.inst[j].cpu_addr); @@ -339,10 +348,6 @@ int amdgpu_uvd_sw_init(struct amdgpu_device *adev) adev->uvd.filp[i] = NULL; } - /* from uvd v5.0 HW addressing capacity increased to 64 bits */ - if (!amdgpu_device_ip_block_version_cmp(adev, AMD_IP_BLOCK_TYPE_UVD, 5, 0)) - adev->uvd.address_64_bit = true; - r = amdgpu_uvd_create_msg_bo_helper(adev, 128 << 10, &adev->uvd.ib_bo); if (r) return r; @@ -544,6 +549,24 @@ void amdgpu_uvd_free_handles(struct amdgpu_device *adev, struct drm_file *filp) } } +static void amdgpu_uvd_force_into_vcpu_segment(struct amdgpu_bo *bo) +{ + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); + struct amdgpu_bo *vcpu_bo = adev->uvd.inst[0].vcpu_bo; + struct amdgpu_res_cursor vcpu_cur; + + amdgpu_res_first(vcpu_bo->tbo.resource, 0, + amdgpu_bo_size(vcpu_bo), &vcpu_cur); + + bo->placement.num_placement = 1; + bo->placement.placement = &bo->placements[0]; + bo->placements[0].fpfn = ALIGN_DOWN(vcpu_cur.start, SZ_256M) >> PAGE_SHIFT; + bo->placements[0].lpfn = bo->placements[0].fpfn + (SZ_256M >> PAGE_SHIFT); + bo->placements[0].mem_type = vcpu_bo->tbo.resource->mem_type; + if (bo->placements[0].mem_type == TTM_PL_VRAM) + bo->placements[0].flags |= TTM_PL_FLAG_CONTIGUOUS; +} + static void amdgpu_uvd_force_into_uvd_segment(struct amdgpu_bo *abo) { int i; @@ -594,13 +617,10 @@ static int amdgpu_uvd_cs_pass1(struct amdgpu_uvd_cs_ctx *ctx) if (!ctx->parser->adev->uvd.address_64_bit) { /* check if it's a message or feedback command */ cmd = amdgpu_ib_get_value(ctx->ib, ctx->idx) >> 1; - if (cmd == 0x0 || cmd == 0x3) { - /* yes, force it into VRAM */ - uint32_t domain = AMDGPU_GEM_DOMAIN_VRAM; - - amdgpu_bo_placement_from_domain(bo, domain); - } - amdgpu_uvd_force_into_uvd_segment(bo); + if (cmd == 0x0 || cmd == 0x3) + amdgpu_uvd_force_into_vcpu_segment(bo); + else + amdgpu_uvd_force_into_uvd_segment(bo); r = ttm_bo_validate(&bo->tbo, &bo->placement, &tctx); } @@ -634,6 +654,14 @@ static int amdgpu_uvd_cs_msg_decode(struct amdgpu_device *adev, uint32_t *msg, unsigned int image_size, tmp, min_dpb_size, num_dpb_buffer; unsigned int min_ctx_size = ~0; + /* Reject invalid dimensions to prevent division by zero */ + if (width < 16 || height < 16) { + dev_WARN_ONCE(adev->dev, 1, + "Invalid UVD decoding dimensions (%dx%d)!\n", + width, height); + return -EINVAL; + } + image_size = width * height; image_size += image_size / 2; image_size = ALIGN(image_size, 1024); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index 3b9a9010ed35..3dc2b7f91e28 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -853,9 +853,20 @@ int amdgpu_vce_ring_parse_cs(struct amdgpu_cs_parser *p, goto out; } - *size = amdgpu_ib_get_value(ib, idx + 8) * - amdgpu_ib_get_value(ib, idx + 10) * - 8 * 3 / 2; + uint32_t width, height; + width = amdgpu_ib_get_value(ib, idx + 8); + height = amdgpu_ib_get_value(ib, idx + 10); + + if (width == 0 || height == 0 || + width > 4096 || height > 2304) { + DRM_ERROR("invalid VCE image size: %ux%u\n", + width, height); + r = -EINVAL; + goto out; + } + + *size = width * height * 8 * 3 / 2; + break; case 0x04000001: /* config extension */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index ffbae605caaa..fd80571a1ace 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -2453,19 +2453,6 @@ static void amdgpu_vm_destroy_task_info(struct kref *kref) kfree(ti); } -static inline struct amdgpu_vm * -amdgpu_vm_get_vm_from_pasid(struct amdgpu_device *adev, u32 pasid) -{ - struct amdgpu_vm *vm; - unsigned long flags; - - xa_lock_irqsave(&adev->vm_manager.pasids, flags); - vm = xa_load(&adev->vm_manager.pasids, pasid); - xa_unlock_irqrestore(&adev->vm_manager.pasids, flags); - - return vm; -} - /** * amdgpu_vm_put_task_info - reference down the vm task_info ptr * @@ -2512,8 +2499,16 @@ amdgpu_vm_get_task_info_vm(struct amdgpu_vm *vm) struct amdgpu_task_info * amdgpu_vm_get_task_info_pasid(struct amdgpu_device *adev, u32 pasid) { - return amdgpu_vm_get_task_info_vm( - amdgpu_vm_get_vm_from_pasid(adev, pasid)); + struct amdgpu_task_info *ti; + struct amdgpu_vm *vm; + unsigned long flags; + + xa_lock_irqsave(&adev->vm_manager.pasids, flags); + vm = xa_load(&adev->vm_manager.pasids, pasid); + ti = amdgpu_vm_get_task_info_vm(vm); + xa_unlock_irqrestore(&adev->vm_manager.pasids, flags); + + return ti; } static int amdgpu_vm_create_task_info(struct amdgpu_vm *vm) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 9d934c07fa6b..cd94f6efb7cb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -25,6 +25,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_vm.h" @@ -52,15 +53,15 @@ to_amdgpu_device(struct amdgpu_vram_mgr *mgr) return container_of(mgr, struct amdgpu_device, mman.vram_mgr); } -static inline struct drm_buddy_block * +static inline struct gpu_buddy_block * amdgpu_vram_mgr_first_block(struct list_head *list) { - return list_first_entry_or_null(list, struct drm_buddy_block, link); + return list_first_entry_or_null(list, struct gpu_buddy_block, link); } static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head) { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; u64 start, size; block = amdgpu_vram_mgr_first_block(head); @@ -71,7 +72,7 @@ static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head) start = amdgpu_vram_mgr_block_start(block); size = amdgpu_vram_mgr_block_size(block); - block = list_entry(block->link.next, struct drm_buddy_block, link); + block = list_entry(block->link.next, struct gpu_buddy_block, link); if (start + size != amdgpu_vram_mgr_block_start(block)) return false; } @@ -81,7 +82,7 @@ static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head) static inline u64 amdgpu_vram_mgr_blocks_size(struct list_head *head) { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; u64 size = 0; list_for_each_entry(block, head, link) @@ -254,7 +255,7 @@ const struct attribute_group amdgpu_vram_mgr_attr_group = { * Calculate how many bytes of the DRM BUDDY block are inside visible VRAM */ static u64 amdgpu_vram_mgr_vis_size(struct amdgpu_device *adev, - struct drm_buddy_block *block) + struct gpu_buddy_block *block) { u64 start = amdgpu_vram_mgr_block_start(block); u64 end = start + amdgpu_vram_mgr_block_size(block); @@ -279,7 +280,7 @@ u64 amdgpu_vram_mgr_bo_visible_size(struct amdgpu_bo *bo) struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); struct ttm_resource *res = bo->tbo.resource; struct amdgpu_vram_mgr_resource *vres = to_amdgpu_vram_mgr_resource(res); - struct drm_buddy_block *block; + struct gpu_buddy_block *block; u64 usage = 0; if (amdgpu_gmc_vram_full_visible(&adev->gmc)) @@ -299,15 +300,15 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man) { struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); struct amdgpu_device *adev = to_amdgpu_device(mgr); - struct drm_buddy *mm = &mgr->mm; + struct gpu_buddy *mm = &mgr->mm; struct amdgpu_vram_reservation *rsv, *temp; - struct drm_buddy_block *block; + struct gpu_buddy_block *block; uint64_t vis_usage; list_for_each_entry_safe(rsv, temp, &mgr->reservations_pending, blocks) { - if (drm_buddy_alloc_blocks(mm, rsv->start, rsv->start + rsv->size, + if (gpu_buddy_alloc_blocks(mm, rsv->start, rsv->start + rsv->size, rsv->size, mm->chunk_size, &rsv->allocated, - DRM_BUDDY_RANGE_ALLOCATION)) + GPU_BUDDY_RANGE_ALLOCATION)) continue; block = amdgpu_vram_mgr_first_block(&rsv->allocated); @@ -403,7 +404,7 @@ int amdgpu_vram_mgr_query_address_block_info(struct amdgpu_vram_mgr *mgr, uint64_t address, struct amdgpu_vram_block_info *info) { struct amdgpu_vram_mgr_resource *vres; - struct drm_buddy_block *block; + struct gpu_buddy_block *block; u64 start, size; int ret = -ENOENT; @@ -450,8 +451,8 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, struct amdgpu_vram_mgr_resource *vres; u64 size, remaining_size, lpfn, fpfn; unsigned int adjust_dcc_size = 0; - struct drm_buddy *mm = &mgr->mm; - struct drm_buddy_block *block; + struct gpu_buddy *mm = &mgr->mm; + struct gpu_buddy_block *block; unsigned long pages_per_block; int r; @@ -493,17 +494,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, INIT_LIST_HEAD(&vres->blocks); if (place->flags & TTM_PL_FLAG_TOPDOWN) - vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + vres->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION; if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS) - vres->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION; + vres->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION; if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CLEARED) - vres->flags |= DRM_BUDDY_CLEAR_ALLOCATION; + vres->flags |= GPU_BUDDY_CLEAR_ALLOCATION; if (fpfn || lpfn != mgr->mm.size) /* Allocate blocks in desired range */ - vres->flags |= DRM_BUDDY_RANGE_ALLOCATION; + vres->flags |= GPU_BUDDY_RANGE_ALLOCATION; if (bo->flags & AMDGPU_GEM_CREATE_GFX12_DCC && adev->gmc.gmc_funcs->get_dcc_alignment) @@ -516,7 +517,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, dcc_size = roundup_pow_of_two(vres->base.size + adjust_dcc_size); remaining_size = (u64)dcc_size; - vres->flags |= DRM_BUDDY_TRIM_DISABLE; + vres->flags |= GPU_BUDDY_TRIM_DISABLE; } mutex_lock(&mgr->lock); @@ -536,7 +537,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, BUG_ON(min_block_size < mm->chunk_size); - r = drm_buddy_alloc_blocks(mm, fpfn, + r = gpu_buddy_alloc_blocks(mm, fpfn, lpfn, size, min_block_size, @@ -545,7 +546,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, if (unlikely(r == -ENOSPC) && pages_per_block == ~0ul && !(place->flags & TTM_PL_FLAG_CONTIGUOUS)) { - vres->flags &= ~DRM_BUDDY_CONTIGUOUS_ALLOCATION; + vres->flags &= ~GPU_BUDDY_CONTIGUOUS_ALLOCATION; pages_per_block = max_t(u32, 2UL << (20UL - PAGE_SHIFT), tbo->page_alignment); @@ -566,7 +567,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, list_add_tail(&vres->vres_node, &mgr->allocated_vres_list); if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS && adjust_dcc_size) { - struct drm_buddy_block *dcc_block; + struct gpu_buddy_block *dcc_block; unsigned long dcc_start; u64 trim_start; @@ -576,7 +577,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, roundup((unsigned long)amdgpu_vram_mgr_block_start(dcc_block), adjust_dcc_size); trim_start = (u64)dcc_start; - drm_buddy_block_trim(mm, &trim_start, + gpu_buddy_block_trim(mm, &trim_start, (u64)vres->base.size, &vres->blocks); } @@ -614,7 +615,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &vres->blocks, 0); + gpu_buddy_free_list(mm, &vres->blocks, 0); mutex_unlock(&mgr->lock); error_fini: ttm_resource_fini(man, &vres->base); @@ -637,8 +638,8 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, struct amdgpu_vram_mgr_resource *vres = to_amdgpu_vram_mgr_resource(res); struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); struct amdgpu_device *adev = to_amdgpu_device(mgr); - struct drm_buddy *mm = &mgr->mm; - struct drm_buddy_block *block; + struct gpu_buddy *mm = &mgr->mm; + struct gpu_buddy_block *block; uint64_t vis_usage = 0; mutex_lock(&mgr->lock); @@ -649,7 +650,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, list_for_each_entry(block, &vres->blocks, link) vis_usage += amdgpu_vram_mgr_vis_size(adev, block); - drm_buddy_free_list(mm, &vres->blocks, vres->flags); + gpu_buddy_free_list(mm, &vres->blocks, vres->flags); amdgpu_vram_mgr_do_reserve(man); mutex_unlock(&mgr->lock); @@ -688,7 +689,7 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev, if (!*sgt) return -ENOMEM; - /* Determine the number of DRM_BUDDY blocks to export */ + /* Determine the number of GPU_BUDDY blocks to export */ amdgpu_res_first(res, offset, length, &cursor); while (cursor.remaining) { num_entries++; @@ -704,10 +705,10 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev, sg->length = 0; /* - * Walk down DRM_BUDDY blocks to populate scatterlist nodes - * @note: Use iterator api to get first the DRM_BUDDY block + * Walk down GPU_BUDDY blocks to populate scatterlist nodes + * @note: Use iterator api to get first the GPU_BUDDY block * and the number of bytes from it. Access the following - * DRM_BUDDY block(s) if more buffer needs to exported + * GPU_BUDDY block(s) if more buffer needs to exported */ amdgpu_res_first(res, offset, length, &cursor); for_each_sgtable_sg((*sgt), sg, i) { @@ -792,10 +793,10 @@ uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr) void amdgpu_vram_mgr_clear_reset_blocks(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; - struct drm_buddy *mm = &mgr->mm; + struct gpu_buddy *mm = &mgr->mm; mutex_lock(&mgr->lock); - drm_buddy_reset_clear(mm, false); + gpu_buddy_reset_clear(mm, false); mutex_unlock(&mgr->lock); } @@ -815,7 +816,7 @@ static bool amdgpu_vram_mgr_intersects(struct ttm_resource_manager *man, size_t size) { struct amdgpu_vram_mgr_resource *mgr = to_amdgpu_vram_mgr_resource(res); - struct drm_buddy_block *block; + struct gpu_buddy_block *block; /* Check each drm buddy block individually */ list_for_each_entry(block, &mgr->blocks, link) { @@ -848,7 +849,7 @@ static bool amdgpu_vram_mgr_compatible(struct ttm_resource_manager *man, size_t size) { struct amdgpu_vram_mgr_resource *mgr = to_amdgpu_vram_mgr_resource(res); - struct drm_buddy_block *block; + struct gpu_buddy_block *block; /* Check each drm buddy block individually */ list_for_each_entry(block, &mgr->blocks, link) { @@ -877,7 +878,7 @@ static void amdgpu_vram_mgr_debug(struct ttm_resource_manager *man, struct drm_printer *printer) { struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); - struct drm_buddy *mm = &mgr->mm; + struct gpu_buddy *mm = &mgr->mm; struct amdgpu_vram_reservation *rsv; drm_printf(printer, " vis usage:%llu\n", @@ -930,7 +931,7 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) mgr->default_page_size = PAGE_SIZE; man->func = &amdgpu_vram_mgr_func; - err = drm_buddy_init(&mgr->mm, man->size, PAGE_SIZE); + err = gpu_buddy_init(&mgr->mm, man->size, PAGE_SIZE); if (err) return err; @@ -965,11 +966,11 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) kfree(rsv); list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) { - drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0); + gpu_buddy_free_list(&mgr->mm, &rsv->allocated, 0); kfree(rsv); } if (!adev->gmc.is_app_apu) - drm_buddy_fini(&mgr->mm); + gpu_buddy_fini(&mgr->mm); mutex_unlock(&mgr->lock); ttm_resource_manager_cleanup(man); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h index 5f5fd9a911c2..429a21a2e9b2 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h @@ -24,11 +24,11 @@ #ifndef __AMDGPU_VRAM_MGR_H__ #define __AMDGPU_VRAM_MGR_H__ -#include +#include struct amdgpu_vram_mgr { struct ttm_resource_manager manager; - struct drm_buddy mm; + struct gpu_buddy mm; /* protects access to buffer objects */ struct mutex lock; struct list_head reservations_pending; @@ -57,19 +57,19 @@ struct amdgpu_vram_mgr_resource { struct amdgpu_vres_task task; }; -static inline u64 amdgpu_vram_mgr_block_start(struct drm_buddy_block *block) +static inline u64 amdgpu_vram_mgr_block_start(struct gpu_buddy_block *block) { - return drm_buddy_block_offset(block); + return gpu_buddy_block_offset(block); } -static inline u64 amdgpu_vram_mgr_block_size(struct drm_buddy_block *block) +static inline u64 amdgpu_vram_mgr_block_size(struct gpu_buddy_block *block) { - return (u64)PAGE_SIZE << drm_buddy_block_order(block); + return (u64)PAGE_SIZE << gpu_buddy_block_order(block); } -static inline bool amdgpu_vram_mgr_is_cleared(struct drm_buddy_block *block) +static inline bool amdgpu_vram_mgr_is_cleared(struct gpu_buddy_block *block) { - return drm_buddy_block_is_clear(block); + return gpu_buddy_block_is_clear(block); } static inline struct amdgpu_vram_mgr_resource * @@ -82,8 +82,8 @@ static inline void amdgpu_vram_mgr_set_cleared(struct ttm_resource *res) { struct amdgpu_vram_mgr_resource *ares = to_amdgpu_vram_mgr_resource(res); - WARN_ON(ares->flags & DRM_BUDDY_CLEARED); - ares->flags |= DRM_BUDDY_CLEARED; + WARN_ON(ares->flags & GPU_BUDDY_CLEARED); + ares->flags |= GPU_BUDDY_CLEARED; } int amdgpu_vram_mgr_query_address_block_info(struct amdgpu_vram_mgr *mgr, diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c index 234753a10361..b8cc213b347c 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c @@ -4022,7 +4022,7 @@ static void gfx_v10_0_wait_reg_mem(struct amdgpu_ring *ring, int eng_sel, WAIT_REG_MEM_ENGINE(eng_sel))); if (mem_space) - BUG_ON(addr0 & 0x3); /* Dword align */ + WARN_ON(addr0 & 0x3); /* Dword align */ amdgpu_ring_write(ring, addr0); amdgpu_ring_write(ring, addr1); amdgpu_ring_write(ring, ref); @@ -5349,6 +5349,15 @@ static void gfx_v10_0_constants_init(struct amdgpu_device *adev) gfx_v10_0_get_tcc_info(adev); adev->gfx.config.pa_sc_tile_steering_override = gfx_v10_0_init_pa_sc_tile_steering_override(adev); + /* Program DB_RING_CONTROL for multiple GFX pipes + * Default power up value is 1. + * Possible values: + * 0 - split occlusion counters between gfx pipes + * 1 - all occlusion counters to pipe 0 + * 2 - all occlusion counters to pipe 1 + */ + WREG32_FIELD15(GC, 0, DB_RING_CONTROL, COUNTER_CONTROL, + (adev->gfx.me.num_pipe_per_me > 1) ? 0 : 1); /* XXX SH_MEM regs */ /* where to put LDS, scratch, GPUVM in FSA64 space */ @@ -8665,7 +8674,7 @@ static void gfx_v10_0_ring_emit_ib_gfx(struct amdgpu_ring *ring, } amdgpu_ring_write(ring, header); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -8700,7 +8709,7 @@ static void gfx_v10_0_ring_emit_ib_compute(struct amdgpu_ring *ring, } amdgpu_ring_write(ring, PACKET3(PACKET3_INDIRECT_BUFFER, 2)); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -8733,9 +8742,9 @@ static void gfx_v10_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, * aligned if only send 32bit data low (discard data high) */ if (write64bit) - BUG_ON(addr & 0x7); + WARN_ON(addr & 0x7); else - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -8783,9 +8792,6 @@ static void gfx_v10_0_ring_emit_fence_kiq(struct amdgpu_ring *ring, u64 addr, { struct amdgpu_device *adev = ring->adev; - /* we only allocate 32bit for each seq wb address */ - BUG_ON(flags & AMDGPU_FENCE_FLAG_64BIT); - /* write fence seq to the "addr" */ amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c index cf23b5da6dbb..f428b652db33 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c @@ -537,7 +537,7 @@ static void gfx_v11_0_wait_reg_mem(struct amdgpu_ring *ring, int eng_sel, WAIT_REG_MEM_ENGINE(eng_sel))); if (mem_space) - BUG_ON(addr0 & 0x3); /* Dword align */ + WARN_ON(addr0 & 0x3); /* Dword align */ amdgpu_ring_write(ring, addr0); amdgpu_ring_write(ring, addr1); amdgpu_ring_write(ring, ref); @@ -5901,7 +5901,7 @@ static void gfx_v11_0_ring_emit_ib_gfx(struct amdgpu_ring *ring, } amdgpu_ring_write(ring, header); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -5936,7 +5936,7 @@ static void gfx_v11_0_ring_emit_ib_compute(struct amdgpu_ring *ring, } amdgpu_ring_write(ring, PACKET3(PACKET3_INDIRECT_BUFFER, 2)); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -5969,9 +5969,9 @@ static void gfx_v11_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, * aligned if only send 32bit data low (discard data high) */ if (write64bit) - BUG_ON(addr & 0x7); + WARN_ON(addr & 0x7); else - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -6025,9 +6025,6 @@ static void gfx_v11_0_ring_emit_fence_kiq(struct amdgpu_ring *ring, u64 addr, { struct amdgpu_device *adev = ring->adev; - /* we only allocate 32bit for each seq wb address */ - BUG_ON(flags & AMDGPU_FENCE_FLAG_64BIT); - /* write fence seq to the "addr" */ amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c index 50e39b9d9df6..b91e81276877 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c @@ -439,7 +439,7 @@ static void gfx_v12_0_wait_reg_mem(struct amdgpu_ring *ring, int eng_sel, WAIT_REG_MEM_ENGINE(eng_sel))); if (mem_space) - BUG_ON(addr0 & 0x3); /* Dword align */ + WARN_ON(addr0 & 0x3); /* Dword align */ amdgpu_ring_write(ring, addr0); amdgpu_ring_write(ring, addr1); amdgpu_ring_write(ring, ref); @@ -4424,7 +4424,7 @@ static void gfx_v12_0_ring_emit_ib_gfx(struct amdgpu_ring *ring, control |= ib->length_dw | (vmid << 24); amdgpu_ring_write(ring, header); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -4443,7 +4443,7 @@ static void gfx_v12_0_ring_emit_ib_compute(struct amdgpu_ring *ring, u32 control = INDIRECT_BUFFER_VALID | ib->length_dw | (vmid << 24); amdgpu_ring_write(ring, PACKET3(PACKET3_INDIRECT_BUFFER, 2)); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -4474,9 +4474,9 @@ static void gfx_v12_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, * aligned if only send 32bit data low (discard data high) */ if (write64bit) - BUG_ON(addr & 0x7); + WARN_ON(addr & 0x7); else - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -4524,9 +4524,6 @@ static void gfx_v12_0_ring_emit_fence_kiq(struct amdgpu_ring *ring, u64 addr, { struct amdgpu_device *adev = ring->adev; - /* we only allocate 32bit for each seq wb address */ - BUG_ON(flags & AMDGPU_FENCE_FLAG_64BIT); - /* write fence seq to the "addr" */ amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c index 8a81713d97aa..690128ad3094 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c @@ -6226,9 +6226,6 @@ static void gfx_v8_0_ring_emit_fence_compute(struct amdgpu_ring *ring, static void gfx_v8_0_ring_emit_fence_kiq(struct amdgpu_ring *ring, u64 addr, u64 seq, unsigned int flags) { - /* we only allocate 32bit for each seq wb address */ - BUG_ON(flags & AMDGPU_FENCE_FLAG_64BIT); - /* write fence seq to the "addr" */ amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c index f589457f6018..c5549a5abcd4 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c @@ -1183,7 +1183,7 @@ static void gfx_v9_0_wait_reg_mem(struct amdgpu_ring *ring, int eng_sel, WAIT_REG_MEM_ENGINE(eng_sel))); if (mem_space) - BUG_ON(addr0 & 0x3); /* Dword align */ + WARN_ON(addr0 & 0x3); /* Dword align */ amdgpu_ring_write(ring, addr0); amdgpu_ring_write(ring, addr1); amdgpu_ring_write(ring, ref); @@ -4045,6 +4045,41 @@ static int gfx_v9_0_hw_init(struct amdgpu_ip_block *ip_block) return r; } +static void gfx_v9_0_deactivate_kcq_hqd(struct amdgpu_device *adev) +{ + amdgpu_gfx_rlc_enter_safe_mode(adev, 0); + for (int i = 0; i < adev->gfx.num_compute_rings; i++) { + u32 tmp; + struct amdgpu_ring *ring = &adev->gfx.compute_ring[i]; + + mutex_lock(&adev->srbm_mutex); + soc15_grbm_select(adev, ring->me, ring->pipe, ring->queue, 0, 0); + tmp = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); + /* disable the queue if it's active */ + if (tmp & CP_HQD_ACTIVE__ACTIVE_MASK) { + int j; + + WREG32_SOC15(GC, 0, mmCP_HQD_DEQUEUE_REQUEST, 1); + for (j = 0; j < adev->usec_timeout; j++) { + tmp = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); + if (!(tmp & CP_HQD_ACTIVE__ACTIVE_MASK)) + break; + udelay(1); + } + if (j == AMDGPU_MAX_USEC_TIMEOUT) { + DRM_DEBUG("comp_%u_%u_%u dequeue request failed.\n", + ring->me, ring->pipe, ring->queue); + /* Manual disable if dequeue request times out */ + WREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE, 0); + } + WREG32_SOC15(GC, 0, mmCP_HQD_DEQUEUE_REQUEST, 0); + } + soc15_grbm_select(adev, 0, 0, 0, 0, 0); + mutex_unlock(&adev->srbm_mutex); + } + amdgpu_gfx_rlc_exit_safe_mode(adev, 0); +} + static int gfx_v9_0_hw_fini(struct amdgpu_ip_block *ip_block) { struct amdgpu_device *adev = ip_block->adev; @@ -4071,6 +4106,10 @@ static int gfx_v9_0_hw_fini(struct amdgpu_ip_block *ip_block) return 0; } + if ((adev->flags & AMD_IS_APU) && amdgpu_in_reset(adev) && + amdgpu_asic_reset_method(adev) == AMD_RESET_METHOD_MODE2) + gfx_v9_0_deactivate_kcq_hqd(adev); + /* Use deinitialize sequence from CAIL when unbinding device from driver, * otherwise KIQ is hanging when binding back */ @@ -5434,7 +5473,7 @@ static void gfx_v9_0_ring_emit_ib_gfx(struct amdgpu_ring *ring, } amdgpu_ring_write(ring, header); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -5530,7 +5569,7 @@ static void gfx_v9_0_ring_emit_ib_compute(struct amdgpu_ring *ring, } amdgpu_ring_write(ring, PACKET3(PACKET3_INDIRECT_BUFFER, 2)); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -5571,9 +5610,9 @@ static void gfx_v9_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, * aligned if only send 32bit data low (discard data high) */ if (write64bit) - BUG_ON(addr & 0x7); + WARN_ON(addr & 0x7); else - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c index a4ebb6c5af55..314916bd2368 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_3.c @@ -405,7 +405,7 @@ static void gfx_v9_4_3_wait_reg_mem(struct amdgpu_ring *ring, int eng_sel, WAIT_REG_MEM_ENGINE(eng_sel))); if (mem_space) - BUG_ON(addr0 & 0x3); /* Dword align */ + WARN_ON(addr0 & 0x3); /* Dword align */ amdgpu_ring_write(ring, addr0); amdgpu_ring_write(ring, addr1); amdgpu_ring_write(ring, ref); @@ -2871,7 +2871,7 @@ static void gfx_v9_4_3_ring_emit_ib_compute(struct amdgpu_ring *ring, } amdgpu_ring_write(ring, PACKET3(PACKET3_INDIRECT_BUFFER, 2)); - BUG_ON(ib->gpu_addr & 0x3); /* Dword align */ + WARN_ON(ib->gpu_addr & 0x3); /* Dword align */ amdgpu_ring_write(ring, #ifdef __BIG_ENDIAN (2 << 0) | @@ -2905,9 +2905,9 @@ static void gfx_v9_4_3_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, * aligned if only send 32bit data low (discard data high) */ if (write64bit) - BUG_ON(addr & 0x7); + WARN_ON(addr & 0x7); else - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -2967,9 +2967,6 @@ static void gfx_v9_4_3_ring_emit_fence_kiq(struct amdgpu_ring *ring, u64 addr, { struct amdgpu_device *adev = ring->adev; - /* we only allocate 32bit for each seq wb address */ - BUG_ON(flags & AMDGPU_FENCE_FLAG_64BIT); - /* write fence seq to the "addr" */ amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3)); amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(0) | diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c index 99e6dabc36ee..452146550668 100644 --- a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c +++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_3.c @@ -1010,7 +1010,7 @@ void jpeg_v4_0_3_dec_ring_nop(struct amdgpu_ring *ring, uint32_t count) static bool jpeg_v4_0_3_is_idle(struct amdgpu_ip_block *ip_block) { struct amdgpu_device *adev = ip_block->adev; - bool ret = false; + bool ret = true; int i, j; for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) { diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_1.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_1.c index 1fb2386352cc..de9ca229479e 100644 --- a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_1.c +++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_1.c @@ -657,7 +657,7 @@ static void jpeg_v5_0_1_dec_ring_set_wptr(struct amdgpu_ring *ring) static bool jpeg_v5_0_1_is_idle(struct amdgpu_ip_block *ip_block) { struct amdgpu_device *adev = ip_block->adev; - bool ret = false; + bool ret = true; int i, j; for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) { diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c index a1443990d5c6..c8b4fda59016 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c @@ -458,7 +458,7 @@ static void sdma_v4_4_2_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 /* write the fence */ amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -468,7 +468,7 @@ static void sdma_v4_4_2_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 addr += 4; amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE)); /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(seq)); diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c index 45e2933214a8..e1d9cea98a2d 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c @@ -528,7 +528,7 @@ static void sdma_v5_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* Ucached(UC) */ /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -539,7 +539,7 @@ static void sdma_v5_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(seq)); diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c index 5b982cc91af3..48c672c1b74d 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c @@ -378,7 +378,7 @@ static void sdma_v5_2_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* Ucached(UC) */ /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -389,7 +389,7 @@ static void sdma_v5_2_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(seq)); diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c index 6809c6d4be5b..ef411bd02237 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c @@ -360,7 +360,7 @@ static void sdma_v6_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_COPY_LINEAR_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* Ucached(UC) */ /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -371,7 +371,7 @@ static void sdma_v6_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_COPY_LINEAR_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(seq)); diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c index 2b81344dcd66..d1318fb8060f 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c @@ -364,7 +364,7 @@ static void sdma_v7_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_COPY_LINEAR_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* Ucached(UC) */ /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, lower_32_bits(seq)); @@ -375,7 +375,7 @@ static void sdma_v7_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 se amdgpu_ring_write(ring, SDMA_PKT_COPY_LINEAR_HEADER_OP(SDMA_OP_FENCE) | SDMA_PKT_FENCE_HEADER_MTYPE(0x3)); /* zero in first two bits */ - BUG_ON(addr & 0x3); + WARN_ON(addr & 0x3); amdgpu_ring_write(ring, lower_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(addr)); amdgpu_ring_write(ring, upper_32_bits(seq)); diff --git a/drivers/gpu/drm/amd/amdgpu/soc24.c b/drivers/gpu/drm/amd/amdgpu/soc24.c index 972b449ab89f..999d9da6756d 100644 --- a/drivers/gpu/drm/amd/amdgpu/soc24.c +++ b/drivers/gpu/drm/amd/amdgpu/soc24.c @@ -526,8 +526,36 @@ static int soc24_common_suspend(struct amdgpu_ip_block *ip_block) return soc24_common_hw_fini(ip_block); } +static bool soc24_need_reset_on_resume(struct amdgpu_device *adev) +{ + u32 sol_reg1, sol_reg2; + + /* Will reset for the following suspend abort cases. + * 1) Only reset dGPU side. + * 2) S3 suspend got aborted and TOS is active. + * As for dGPU suspend abort cases the SOL value + * will be kept as zero at this resume point. + */ + if (!(adev->flags & AMD_IS_APU) && adev->in_s3) { + sol_reg1 = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_81); + msleep(100); + sol_reg2 = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_81); + + return (sol_reg1 != sol_reg2); + } + + return false; +} + static int soc24_common_resume(struct amdgpu_ip_block *ip_block) { + struct amdgpu_device *adev = ip_block->adev; + + if (soc24_need_reset_on_resume(adev)) { + dev_info(adev->dev, "S3 suspend aborted, resetting..."); + soc24_asic_reset(adev); + } + return soc24_common_hw_init(ip_block); } diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c index 21e40eb2078d..ad74edc1bbfc 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c @@ -1926,14 +1926,17 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job, #define RENCODE_IB_PARAM_SESSION_INIT 0x00000003 /* return the offset in ib if id is found, -1 otherwise */ -static int vcn_v4_0_enc_find_ib_param(struct amdgpu_ib *ib, uint32_t id, int start) +static int vcn_v4_0_enc_find_ib_param(struct amdgpu_ib *ib, uint32_t id, int start, uint32_t *length) { int i; uint32_t len; for (i = start; (len = amdgpu_ib_get_value(ib, i)) >= 8; i += len / 4) { - if (amdgpu_ib_get_value(ib, i + 1) == id) + if (amdgpu_ib_get_value(ib, i + 1) == id) { + if (length) + *length = len; return i; + } } return -1; } @@ -1943,14 +1946,14 @@ static int vcn_v4_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p, struct amdgpu_ib *ib) { struct amdgpu_ring *ring = amdgpu_job_ring(job); - uint32_t val; + uint32_t val, len; int idx = 0, sidx; /* The first instance can decode anything */ if (!ring->me) return 0; - while ((idx = vcn_v4_0_enc_find_ib_param(ib, RADEON_VCN_ENGINE_INFO, idx)) >= 0) { + while ((idx = vcn_v4_0_enc_find_ib_param(ib, RADEON_VCN_ENGINE_INFO, idx, &len)) >= 0) { val = amdgpu_ib_get_value(ib, idx + 2); /* RADEON_VCN_ENGINE_TYPE */ if (val == RADEON_VCN_ENGINE_TYPE_DECODE) { uint32_t valid_buf_flag = amdgpu_ib_get_value(ib, idx + 6); @@ -1963,12 +1966,12 @@ static int vcn_v4_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p, amdgpu_ib_get_value(ib, idx + 8); return vcn_v4_0_dec_msg(p, job, msg_buffer_addr); } else if (val == RADEON_VCN_ENGINE_TYPE_ENCODE) { - sidx = vcn_v4_0_enc_find_ib_param(ib, RENCODE_IB_PARAM_SESSION_INIT, idx); + sidx = vcn_v4_0_enc_find_ib_param(ib, RENCODE_IB_PARAM_SESSION_INIT, idx, NULL); if (sidx >= 0 && amdgpu_ib_get_value(ib, sidx + 2) == RENCODE_ENCODE_STANDARD_AV1) return vcn_v4_0_limit_sched(p, job); } - idx += amdgpu_ib_get_value(ib, idx) / 4; + idx += len / 4; } return 0; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index c2100fd78b52..c4fd83ae1762 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -1790,13 +1790,13 @@ static int criu_checkpoint_devices(struct kfd_process *p, struct kfd_criu_device_bucket *device_buckets = NULL; int ret = 0, i; - device_buckets = kvzalloc(num_devices * sizeof(*device_buckets), GFP_KERNEL); + device_buckets = kvcalloc(num_devices, sizeof(*device_buckets), GFP_KERNEL); if (!device_buckets) { ret = -ENOMEM; goto exit; } - device_priv = kvzalloc(num_devices * sizeof(*device_priv), GFP_KERNEL); + device_priv = kvcalloc(num_devices, sizeof(*device_priv), GFP_KERNEL); if (!device_priv) { ret = -ENOMEM; goto exit; @@ -1916,17 +1916,17 @@ static int criu_checkpoint_bos(struct kfd_process *p, int ret = 0, pdd_index, bo_index = 0, id; void *mem; - bo_buckets = kvzalloc(num_bos * sizeof(*bo_buckets), GFP_KERNEL); + bo_buckets = kvcalloc(num_bos, sizeof(*bo_buckets), GFP_KERNEL); if (!bo_buckets) return -ENOMEM; - bo_privs = kvzalloc(num_bos * sizeof(*bo_privs), GFP_KERNEL); + bo_privs = kvcalloc(num_bos, sizeof(*bo_privs), GFP_KERNEL); if (!bo_privs) { ret = -ENOMEM; goto exit; } - files = kvzalloc(num_bos * sizeof(struct file *), GFP_KERNEL); + files = kvcalloc(num_bos, sizeof(struct file *), GFP_KERNEL); if (!files) { ret = -ENOMEM; goto exit; @@ -2463,7 +2463,7 @@ static int criu_restore_bos(struct kfd_process *p, if (!bo_buckets) return -ENOMEM; - files = kvzalloc(args->num_bos * sizeof(struct file *), GFP_KERNEL); + files = kvcalloc(args->num_bos, sizeof(struct file *), GFP_KERNEL); if (!files) { ret = -ENOMEM; goto exit; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c index 18d03f33c620..c650ffc3e66e 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c @@ -3628,6 +3628,12 @@ bool kfd_dqm_is_queue_in_process(struct device_queue_manager *dqm, dqm_unlock(dqm); return r; } + +size_t mqd_size_from_queue_type(struct device_queue_manager *dqm, enum kfd_queue_type type) +{ + return dqm->mqd_mgrs[get_mqd_type_from_queue_type(type)]->mqd_size; +} + #if defined(CONFIG_DEBUG_FS) static void seq_reg_dump(struct seq_file *m, diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h index 74a61b5b2f0b..46c6849df13f 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h @@ -327,6 +327,8 @@ int debug_refresh_runlist(struct device_queue_manager *dqm); bool kfd_dqm_is_queue_in_process(struct device_queue_manager *dqm, struct qcm_process_device *qpd, int doorbell_off, u32 *queue_format); +size_t mqd_size_from_queue_type(struct device_queue_manager *dqm, + enum kfd_queue_type type); static inline unsigned int get_sh_mem_bases_32(struct kfd_process_device *pdd) { diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_events.c index 63039035b194..2fdee34ae543 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_events.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_events.c @@ -107,6 +107,9 @@ static int allocate_event_notification_slot(struct kfd_process *p, } if (restore_id) { + if (*restore_id >= KFD_SIGNAL_EVENT_LIMIT) + return -EINVAL; + id = idr_alloc(&p->event_idr, ev, *restore_id, *restore_id + 1, GFP_KERNEL); } else { diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index 24e85440473a..d7e915040242 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -436,7 +436,8 @@ enum kfd_queue_type { KFD_QUEUE_TYPE_HIQ, KFD_QUEUE_TYPE_DIQ, KFD_QUEUE_TYPE_SDMA_XGMI, - KFD_QUEUE_TYPE_SDMA_BY_ENG_ID + KFD_QUEUE_TYPE_SDMA_BY_ENG_ID, + KFD_QUEUE_TYPE_MAX, }; enum kfd_queue_format { diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c index 7bf712032c52..0bfe64cdabe6 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c @@ -1015,6 +1015,23 @@ int kfd_criu_restore_queue(struct kfd_process *p, goto exit; } + pdd = kfd_process_device_data_by_id(p, q_data->gpu_id); + if (!pdd) { + pr_err("Failed to get pdd\n"); + ret = -EINVAL; + goto exit; + } + + if (q_data->type >= KFD_QUEUE_TYPE_MAX) { + ret = -EINVAL; + goto exit; + } + + if (q_data->mqd_size != mqd_size_from_queue_type(pdd->dev->dqm, q_data->type)) { + ret = -EINVAL; + goto exit; + } + *priv_data_offset += sizeof(*q_data); q_extra_data_size = (uint64_t)q_data->ctl_stack_size + q_data->mqd_size; @@ -1037,13 +1054,6 @@ int kfd_criu_restore_queue(struct kfd_process *p, *priv_data_offset += q_extra_data_size; - pdd = kfd_process_device_data_by_id(p, q_data->gpu_id); - if (!pdd) { - pr_err("Failed to get pdd\n"); - ret = -EINVAL; - goto exit; - } - /* * data stored in this order: * mqd[xcc0], mqd[xcc1],..., ctl_stack[xcc0], ctl_stack[xcc1]... diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c index b97f4a51db6e..8e74e2bae590 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c @@ -23,6 +23,7 @@ */ #include +#include #include "kfd_priv.h" #include "kfd_topology.h" #include "kfd_svm.h" @@ -235,7 +236,7 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope struct kfd_topology_device *topo_dev; u64 expected_queue_size; struct amdgpu_vm *vm; - u32 total_cwsr_size; + u64 total_cwsr_size; int err; topo_dev = kfd_topology_device_by_id(pdd->dev->id); @@ -305,8 +306,14 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope goto out_err_unreserve; } - total_cwsr_size = (properties->ctx_save_restore_area_size + - topo_dev->node_props.debug_memory_size) * NUM_XCC(pdd->dev->xcc_mask); + total_cwsr_size = (u64)properties->ctx_save_restore_area_size + + topo_dev->node_props.debug_memory_size; + if (check_mul_overflow(total_cwsr_size, + NUM_XCC(pdd->dev->xcc_mask), + &total_cwsr_size)) { + err = -EINVAL; + goto out_err_unreserve; + } total_cwsr_size = ALIGN(total_cwsr_size, PAGE_SIZE); err = kfd_queue_buffer_get(vm, (void *)properties->ctx_save_restore_area_address, @@ -341,7 +348,7 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope int kfd_queue_release_buffers(struct kfd_process_device *pdd, struct queue_properties *properties) { struct kfd_topology_device *topo_dev; - u32 total_cwsr_size; + u64 total_cwsr_size; kfd_queue_buffer_put(&properties->wptr_bo); kfd_queue_buffer_put(&properties->rptr_bo); @@ -352,8 +359,12 @@ int kfd_queue_release_buffers(struct kfd_process_device *pdd, struct queue_prope topo_dev = kfd_topology_device_by_id(pdd->dev->id); if (!topo_dev) return -EINVAL; - total_cwsr_size = (properties->ctx_save_restore_area_size + - topo_dev->node_props.debug_memory_size) * NUM_XCC(pdd->dev->xcc_mask); + total_cwsr_size = (u64)properties->ctx_save_restore_area_size + + topo_dev->node_props.debug_memory_size; + if (check_mul_overflow(total_cwsr_size, + NUM_XCC(pdd->dev->xcc_mask), + &total_cwsr_size)) + return -EINVAL; total_cwsr_size = ALIGN(total_cwsr_size, PAGE_SIZE); kfd_queue_buffer_svm_put(pdd, properties->ctx_save_restore_area_address, total_cwsr_size); diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index a2ca1de6b43c..99759559cc0f 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -575,89 +575,25 @@ static void schedule_dc_vmin_vmax(struct amdgpu_device *adev, queue_work(system_wq, &offload_work->work); } -static void dm_vupdate_high_irq(void *interrupt_params) -{ - struct common_irq_params *irq_params = interrupt_params; - struct amdgpu_device *adev = irq_params->adev; - struct amdgpu_crtc *acrtc; - struct drm_device *drm_dev; - struct drm_vblank_crtc *vblank; - ktime_t frame_duration_ns, previous_timestamp; - unsigned long flags; - int vrr_active; - - acrtc = get_crtc_by_otg_inst(adev, irq_params->irq_src - IRQ_TYPE_VUPDATE); - - if (acrtc) { - vrr_active = amdgpu_dm_crtc_vrr_active_irq(acrtc); - drm_dev = acrtc->base.dev; - vblank = drm_crtc_vblank_crtc(&acrtc->base); - previous_timestamp = atomic64_read(&irq_params->previous_timestamp); - frame_duration_ns = vblank->time - previous_timestamp; - - if (frame_duration_ns > 0) { - trace_amdgpu_refresh_rate_track(acrtc->base.index, - frame_duration_ns, - ktime_divns(NSEC_PER_SEC, frame_duration_ns)); - atomic64_set(&irq_params->previous_timestamp, vblank->time); - } - - drm_dbg_vbl(drm_dev, - "crtc:%d, vupdate-vrr:%d\n", acrtc->crtc_id, - vrr_active); - - /* Core vblank handling is done here after end of front-porch in - * vrr mode, as vblank timestamping will give valid results - * while now done after front-porch. This will also deliver - * page-flip completion events that have been queued to us - * if a pageflip happened inside front-porch. - */ - if (vrr_active && acrtc->dm_irq_params.stream) { - bool replay_en = acrtc->dm_irq_params.stream->link->replay_settings.replay_feature_enabled; - bool psr_en = acrtc->dm_irq_params.stream->link->psr_settings.psr_feature_enabled; - bool fs_active_var_en = acrtc->dm_irq_params.freesync_config.state - == VRR_STATE_ACTIVE_VARIABLE; - - amdgpu_dm_crtc_handle_vblank(acrtc); - - /* BTR processing for pre-DCE12 ASICs */ - if (adev->family < AMDGPU_FAMILY_AI) { - spin_lock_irqsave(&adev_to_drm(adev)->event_lock, flags); - mod_freesync_handle_v_update( - adev->dm.freesync_module, - acrtc->dm_irq_params.stream, - &acrtc->dm_irq_params.vrr_params); - - if (fs_active_var_en || (!fs_active_var_en && !replay_en && !psr_en)) { - schedule_dc_vmin_vmax(adev, - acrtc->dm_irq_params.stream, - &acrtc->dm_irq_params.vrr_params.adjust); - } - spin_unlock_irqrestore(&adev_to_drm(adev)->event_lock, flags); - } - } - } -} - /** - * dm_crtc_high_irq() - Handles CRTC interrupt - * @interrupt_params: used for determining the CRTC instance + * dm_crtc_high_irq_handler() - Common OTG vblank/flip event handling + * @adev: amdgpu device + * @acrtc: the CRTC to service * - * Handles the CRTC/VSYNC interrupt by notfying DRM's VBLANK - * event handler. + * Performs writeback completion, vblank event handling, CRC processing, VRR BTR + * updates and pageflip completion delivery. + * + * On DCN this is driven by VUPDATE_NO_LOCK (the register latch point) from + * dm_vupdate_high_irq(); on DCE it is driven by VLINE0 at the start of vblank + * from dm_crtc_high_irq(). */ -static void dm_crtc_high_irq(void *interrupt_params) +static void dm_crtc_high_irq_handler(struct amdgpu_device *adev, + struct amdgpu_crtc *acrtc) { - struct common_irq_params *irq_params = interrupt_params; - struct amdgpu_device *adev = irq_params->adev; struct drm_writeback_job *job; - struct amdgpu_crtc *acrtc; unsigned long flags; int vrr_active; - - acrtc = get_crtc_by_otg_inst(adev, irq_params->irq_src - IRQ_TYPE_VBLANK); - if (!acrtc) - return; + bool is_dcn = amdgpu_ip_version(adev, DCE_HWIP, 0) != 0; if (acrtc->wb_conn) { spin_lock_irqsave(&acrtc->wb_conn->job_lock, flags); @@ -694,12 +630,17 @@ static void dm_crtc_high_irq(void *interrupt_params) vrr_active, acrtc->dm_irq_params.active_planes); /** - * Core vblank handling at start of front-porch is only possible - * in non-vrr mode, as only there vblank timestamping will give - * valid results while done in front-porch. Otherwise defer it - * to dm_vupdate_high_irq after end of front-porch. + * Core vblank handling. + * + * On DCN this handler runs at VUPDATE_NO_LOCK, the register latch + * point, which is the correct place to timestamp both VRR and non-VRR + * vblanks. + * + * On DCE this handler runs at the start of front-porch, where only + * non-VRR timestamping is valid; VRR vblank is deferred to + * dm_vupdate_high_irq() after end of front-porch. */ - if (!vrr_active) + if (is_dcn || !vrr_active) amdgpu_dm_crtc_handle_vblank(acrtc); /** @@ -732,18 +673,16 @@ static void dm_crtc_high_irq(void *interrupt_params) } /* - * If there aren't any active_planes then DCH HUBP may be clock-gated. - * In that case, pageflip completion interrupts won't fire and pageflip - * completion events won't get delivered. Prevent this by sending - * pending pageflip events from here if a flip is still pending. + * Deliver pageflip completion events (DCN only). * - * If any planes are enabled, use dm_pflip_high_irq() instead, to - * avoid race conditions between flip programming and completion, - * which could cause too early flip completion events. + * Since GRPH_PFLIP is not used, VUPDATE_NO_LOCK is the flip latch + * point. Deliver any pending pageflip completion event from here. + * + * NOTE: This can deliver an event for a flip that was armed but not yet + * programmed into HW; that race is closed in a follow-up change by + * checking the programmed flip status. */ - if (adev->family >= AMDGPU_FAMILY_RV && - acrtc->pflip_status == AMDGPU_FLIP_SUBMITTED && - acrtc->dm_irq_params.active_planes == 0) { + if (is_dcn && acrtc->pflip_status == AMDGPU_FLIP_SUBMITTED) { if (acrtc->event) { drm_crtc_send_vblank_event(&acrtc->base, acrtc->event); acrtc->event = NULL; @@ -755,6 +694,104 @@ static void dm_crtc_high_irq(void *interrupt_params) spin_unlock_irqrestore(&adev_to_drm(adev)->event_lock, flags); } +static void dm_vupdate_high_irq(void *interrupt_params) +{ + struct common_irq_params *irq_params = interrupt_params; + struct amdgpu_device *adev = irq_params->adev; + struct amdgpu_crtc *acrtc; + struct drm_device *drm_dev; + struct drm_vblank_crtc *vblank; + ktime_t frame_duration_ns, previous_timestamp; + unsigned long flags; + int vrr_active; + + acrtc = get_crtc_by_otg_inst(adev, irq_params->irq_src - IRQ_TYPE_VUPDATE); + if (!acrtc) + return; + + vrr_active = amdgpu_dm_crtc_vrr_active_irq(acrtc); + drm_dev = acrtc->base.dev; + vblank = drm_crtc_vblank_crtc(&acrtc->base); + previous_timestamp = atomic64_read(&irq_params->previous_timestamp); + frame_duration_ns = vblank->time - previous_timestamp; + + if (frame_duration_ns > 0) { + trace_amdgpu_refresh_rate_track(acrtc->base.index, + frame_duration_ns, + ktime_divns(NSEC_PER_SEC, frame_duration_ns)); + atomic64_set(&irq_params->previous_timestamp, vblank->time); + } + + drm_dbg_vbl(drm_dev, + "crtc:%d, vupdate-vrr:%d\n", acrtc->crtc_id, + vrr_active); + + /* + * On DCN, VUPDATE_NO_LOCK is the single OTG interrupt used to deliver + * vblank and pageflip completion events; VSTARTUP and GRPH_PFLIP are + * not used. Run the full handler here. + */ + if (amdgpu_ip_version(adev, DCE_HWIP, 0) != 0) { + dm_crtc_high_irq_handler(adev, acrtc); + return; + } + + /* DCE only below. */ + + /* Core vblank handling is done here after end of front-porch in + * vrr mode, as vblank timestamping will give valid results + * while now done after front-porch. This will also deliver + * page-flip completion events that have been queued to us + * if a pageflip happened inside front-porch. + */ + if (vrr_active && acrtc->dm_irq_params.stream) { + bool replay_en = acrtc->dm_irq_params.stream->link->replay_settings.replay_feature_enabled; + bool psr_en = acrtc->dm_irq_params.stream->link->psr_settings.psr_feature_enabled; + bool fs_active_var_en = acrtc->dm_irq_params.freesync_config.state + == VRR_STATE_ACTIVE_VARIABLE; + + amdgpu_dm_crtc_handle_vblank(acrtc); + + /* BTR processing for pre-DCE12 ASICs */ + if (adev->family < AMDGPU_FAMILY_AI) { + spin_lock_irqsave(&adev_to_drm(adev)->event_lock, flags); + mod_freesync_handle_v_update( + adev->dm.freesync_module, + acrtc->dm_irq_params.stream, + &acrtc->dm_irq_params.vrr_params); + + if (fs_active_var_en || (!fs_active_var_en && !replay_en && !psr_en)) { + schedule_dc_vmin_vmax(adev, + acrtc->dm_irq_params.stream, + &acrtc->dm_irq_params.vrr_params.adjust); + } + spin_unlock_irqrestore(&adev_to_drm(adev)->event_lock, flags); + } + } +} + +/** + * dm_crtc_high_irq() - Handles CRTC interrupt + * @interrupt_params: used for determining the CRTC instance + * + * Handles the CRTC/VSYNC interrupt by notifying DRM's VBLANK event handler. + * + * Used on DCE (VLINE0, set to vblank start). On DCN the equivalent handling is + * driven by VUPDATE_NO_LOCK in dm_vupdate_high_irq(). + */ +static void dm_crtc_high_irq(void *interrupt_params) +{ + struct common_irq_params *irq_params = interrupt_params; + struct amdgpu_device *adev = irq_params->adev; + struct amdgpu_crtc *acrtc; + + acrtc = get_crtc_by_otg_inst(adev, irq_params->irq_src - IRQ_TYPE_VBLANK); + if (!acrtc) + return; + + dm_crtc_high_irq_handler(adev, acrtc); +} + #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) /** * dm_dcn_vertical_interrupt0_high_irq() - Handles OTG Vertical interrupt0 for @@ -3031,7 +3068,8 @@ static void dm_gpureset_toggle_interrupts(struct amdgpu_device *adev, acrtc = get_crtc_by_otg_inst( adev, state->stream_status[i].primary_otg_inst); - if (acrtc && state->stream_status[i].plane_count != 0) { + if (acrtc && state->stream_status[i].plane_count != 0 && + amdgpu_ip_version(adev, DCE_HWIP, 0) == 0) { irq_source = IRQ_TYPE_PFLIP + acrtc->otg_inst; rc = dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY; if (rc) @@ -3059,6 +3097,13 @@ static void dm_gpureset_toggle_interrupts(struct amdgpu_device *adev, */ if (!dc_interrupt_set(adev->dm.dc, irq_source, enable)) drm_warn(adev_to_drm(adev), "Failed to %sable vblank interrupt\n", enable ? "en" : "dis"); + + } else if (acrtc && state->stream_status[i].plane_count != 0) { + /* DCN only needs to toggle VUPDATE_NO_LOCK */ + rc = amdgpu_dm_crtc_set_vupdate_irq(&acrtc->base, enable); + if (rc) + drm_warn(adev_to_drm(adev), "Failed to %sable vupdate interrupt\n", + enable ? "en" : "dis"); } } @@ -3680,6 +3725,8 @@ static void update_connector_ext_caps(struct amdgpu_dm_connector *aconnector) caps->ext_caps = &aconnector->dc_link->dpcd_sink_ext_caps; caps->aux_support = false; + panel_backlight_quirk = drm_get_panel_backlight_quirk(aconnector->drm_edid); + if (caps->ext_caps->bits.oled == 1 /* * || @@ -3692,6 +3739,9 @@ static void update_connector_ext_caps(struct amdgpu_dm_connector *aconnector) caps->aux_support = false; else if (amdgpu_backlight == 1) caps->aux_support = true; + else if (!IS_ERR_OR_NULL(panel_backlight_quirk) && + panel_backlight_quirk->force_pwm) + caps->aux_support = false; if (caps->aux_support) aconnector->dc_link->backlight_control_type = BACKLIGHT_CONTROL_AMD_AUX; @@ -3707,8 +3757,6 @@ static void update_connector_ext_caps(struct amdgpu_dm_connector *aconnector) else caps->aux_min_input_signal = 1; - panel_backlight_quirk = - drm_get_panel_backlight_quirk(aconnector->drm_edid); if (!IS_ERR_OR_NULL(panel_backlight_quirk)) { if (panel_backlight_quirk->min_brightness) { caps->min_input_signal = @@ -4558,38 +4606,6 @@ static int dcn10_register_irq_handlers(struct amdgpu_device *adev) * for acknowledging and handling. */ - /* Use VSTARTUP interrupt */ - for (i = DCN_1_0__SRCID__DC_D1_OTG_VSTARTUP; - i <= DCN_1_0__SRCID__DC_D1_OTG_VSTARTUP + adev->mode_info.num_crtc - 1; - i++) { - r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE, i, &adev->crtc_irq); - - if (r) { - drm_err(adev_to_drm(adev), "Failed to add crtc irq id!\n"); - return r; - } - - int_params.int_context = INTERRUPT_HIGH_IRQ_CONTEXT; - int_params.irq_source = - dc_interrupt_to_irq_source(dc, i, 0); - - if (int_params.irq_source == DC_IRQ_SOURCE_INVALID || - int_params.irq_source < DC_IRQ_SOURCE_VBLANK1 || - int_params.irq_source > DC_IRQ_SOURCE_VBLANK6) { - drm_err(adev_to_drm(adev), "Failed to register vblank irq!\n"); - return -EINVAL; - } - - c_irq_params = &adev->dm.vblank_params[int_params.irq_source - DC_IRQ_SOURCE_VBLANK1]; - - c_irq_params->adev = adev; - c_irq_params->irq_src = int_params.irq_source; - - if (!amdgpu_dm_irq_register_interrupt(adev, &int_params, - dm_crtc_high_irq, c_irq_params)) - return -ENOMEM; - } - /* Use otg vertical line interrupt */ #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) for (i = 0; i <= adev->mode_info.num_crtc - 1; i++) { @@ -4661,37 +4677,6 @@ static int dcn10_register_irq_handlers(struct amdgpu_device *adev) return -ENOMEM; } - /* Use GRPH_PFLIP interrupt */ - for (i = DCN_1_0__SRCID__HUBP0_FLIP_INTERRUPT; - i <= DCN_1_0__SRCID__HUBP0_FLIP_INTERRUPT + dc->caps.max_otg_num - 1; - i++) { - r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE, i, &adev->pageflip_irq); - if (r) { - drm_err(adev_to_drm(adev), "Failed to add page flip irq id!\n"); - return r; - } - - int_params.int_context = INTERRUPT_HIGH_IRQ_CONTEXT; - int_params.irq_source = - dc_interrupt_to_irq_source(dc, i, 0); - - if (int_params.irq_source == DC_IRQ_SOURCE_INVALID || - int_params.irq_source < DC_IRQ_SOURCE_PFLIP_FIRST || - int_params.irq_source > DC_IRQ_SOURCE_PFLIP_LAST) { - drm_err(adev_to_drm(adev), "Failed to register pflip irq!\n"); - return -EINVAL; - } - - c_irq_params = &adev->dm.pflip_params[int_params.irq_source - DC_IRQ_SOURCE_PFLIP_FIRST]; - - c_irq_params->adev = adev; - c_irq_params->irq_src = int_params.irq_source; - - if (!amdgpu_dm_irq_register_interrupt(adev, &int_params, - dm_pflip_high_irq, c_irq_params)) - return -ENOMEM; - } - /* HPD */ r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE, DCN_1_0__SRCID__DC_HPD1_INT, &adev->hpd_irq); @@ -5220,11 +5205,11 @@ amdgpu_dm_register_backlight_device(struct amdgpu_dm_connector *aconnector) caps = &dm->backlight_caps[aconnector->bl_idx]; if (get_brightness_range(caps, &min, &max)) { if (power_supply_is_system_supplied() > 0) - props.brightness = DIV_ROUND_CLOSEST((max - min) * caps->ac_level, 100); + props.brightness = DIV_ROUND_CLOSEST(max * caps->ac_level, 100); else - props.brightness = DIV_ROUND_CLOSEST((max - min) * caps->dc_level, 100); + props.brightness = DIV_ROUND_CLOSEST(max * caps->dc_level, 100); /* min is zero, so max needs to be adjusted */ - props.max_brightness = max - min; + props.max_brightness = max; drm_dbg(drm, "Backlight caps: min: %d, max: %d, ac %d, dc %d\n", min, max, caps->ac_level, caps->dc_level); } else @@ -6240,8 +6225,8 @@ static void fill_dc_dirty_rects(struct drm_plane *plane, { struct dm_crtc_state *dm_crtc_state = to_dm_crtc_state(crtc_state); struct rect *dirty_rects = flip_addrs->dirty_rects; - u32 num_clips; - struct drm_mode_rect *clips; + u32 num_clips = 0; + struct drm_mode_rect *clips = NULL; bool bb_changed; bool fb_changed; u32 i = 0; @@ -6257,8 +6242,10 @@ static void fill_dc_dirty_rects(struct drm_plane *plane, if (new_plane_state->rotation != DRM_MODE_ROTATE_0) goto ffu; - num_clips = drm_plane_get_damage_clips_count(new_plane_state); - clips = drm_plane_get_damage_clips(new_plane_state); + if (!new_plane_state->ignore_damage_clips) { + num_clips = drm_plane_get_damage_clips_count(new_plane_state); + clips = drm_plane_get_damage_clips(new_plane_state); + } if (num_clips && (!amdgpu_damage_clips || (amdgpu_damage_clips < 0 && is_psr_su))) @@ -9116,14 +9103,22 @@ static void manage_dm_interrupts(struct amdgpu_device *adev, drm_crtc_vblank_on_config(&acrtc->base, &config); - /* Allow RX6xxx, RX7700, RX7800 GPUs to call amdgpu_irq_get.*/ + /* + * Since pflip_high_irq is no longer registered for DCN, grab an + * extra reference to vupdate irq instead to workaround this + * issue: + * https://gitlab.freedesktop.org/drm/amd/-/work_items/3936 + * + * The callbacks to drm_vblank_on/off should really take care of + * this though. + */ switch (amdgpu_ip_version(adev, DCE_HWIP, 0)) { case IP_VERSION(3, 0, 0): case IP_VERSION(3, 0, 2): case IP_VERSION(3, 0, 3): case IP_VERSION(3, 2, 0): - if (amdgpu_irq_get(adev, &adev->pageflip_irq, irq_type)) - drm_err(dev, "DM_IRQ: Cannot get pageflip irq!\n"); + if (amdgpu_irq_get(adev, &adev->vupdate_irq, irq_type)) + drm_err(dev, "DM_IRQ: Cannot get vupdate irq!\n"); #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) if (amdgpu_irq_get(adev, &adev->vline0_irq, irq_type)) drm_err(dev, "DM_IRQ: Cannot get vline0 irq!\n"); @@ -9141,8 +9136,8 @@ static void manage_dm_interrupts(struct amdgpu_device *adev, if (amdgpu_irq_put(adev, &adev->vline0_irq, irq_type)) drm_err(dev, "DM_IRQ: Cannot put vline0 irq!\n"); #endif - if (amdgpu_irq_put(adev, &adev->pageflip_irq, irq_type)) - drm_err(dev, "DM_IRQ: Cannot put pageflip irq!\n"); + if (amdgpu_irq_put(adev, &adev->vupdate_irq, irq_type)) + drm_err(dev, "DM_IRQ: Cannot put vupdate irq!\n"); } drm_crtc_vblank_off(&acrtc->base); @@ -9155,6 +9150,10 @@ static void dm_update_pflip_irq_state(struct amdgpu_device *adev, int irq_type = amdgpu_display_crtc_idx_to_irq_type(adev, acrtc->crtc_id); + /* GRPH_PFLIP is not used on DCN; nothing to reapply. */ + if (amdgpu_ip_version(adev, DCE_HWIP, 0) != 0) + return; + /** * This reads the current state for the IRQ and force reapplies * the setting to hardware. @@ -9486,9 +9485,13 @@ static void update_stream_irq_parameters( static void amdgpu_dm_handle_vrr_transition(struct dm_crtc_state *old_state, struct dm_crtc_state *new_state) { + struct amdgpu_device *adev = drm_to_adev(new_state->base.crtc->dev); bool old_vrr_active = amdgpu_dm_crtc_vrr_active(old_state); bool new_vrr_active = amdgpu_dm_crtc_vrr_active(new_state); + /* Only DCE gates vupdate on VRR, keep it enabled for DCN */ + bool vrr_gates_vupdate = amdgpu_ip_version(adev, DCE_HWIP, 0) == 0; + if (!old_vrr_active && new_vrr_active) { /* Transition VRR inactive -> active: * While VRR is active, we must not disable vblank irq, as a @@ -9498,7 +9501,8 @@ static void amdgpu_dm_handle_vrr_transition(struct dm_crtc_state *old_state, * We also need vupdate irq for the actual core vblank handling * at end of vblank. */ - WARN_ON(amdgpu_dm_crtc_set_vupdate_irq(new_state->base.crtc, true) != 0); + if (vrr_gates_vupdate) + WARN_ON(amdgpu_dm_crtc_set_vupdate_irq(new_state->base.crtc, true) != 0); WARN_ON(drm_crtc_vblank_get(new_state->base.crtc) != 0); drm_dbg_driver(new_state->base.crtc->dev, "%s: crtc=%u VRR off->on: Get vblank ref\n", __func__, new_state->base.crtc->base.id); @@ -9506,7 +9510,8 @@ static void amdgpu_dm_handle_vrr_transition(struct dm_crtc_state *old_state, /* Transition VRR active -> inactive: * Allow vblank irq disable again for fixed refresh rate. */ - WARN_ON(amdgpu_dm_crtc_set_vupdate_irq(new_state->base.crtc, false) != 0); + if (vrr_gates_vupdate) + WARN_ON(amdgpu_dm_crtc_set_vupdate_irq(new_state->base.crtc, false) != 0); drm_crtc_vblank_put(new_state->base.crtc); drm_dbg_driver(new_state->base.crtc->dev, "%s: crtc=%u VRR on->off: Drop vblank ref\n", __func__, new_state->base.crtc->base.id); @@ -9681,6 +9686,7 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, bool vrr_active = amdgpu_dm_crtc_vrr_active(acrtc_state); bool cursor_update = false; bool pflip_present = false; + bool immediate_flip = false; bool dirty_rects_changed = false; bool updated_planes_and_streams = false; struct { @@ -9846,6 +9852,8 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, acrtc_state->update_type == UPDATE_TYPE_FAST && get_mem_type(old_plane_state->fb) == get_mem_type(fb); + immediate_flip |= bundle->flip_addrs[planes_count].flip_immediate; + timestamp_ns = ktime_get_ns(); bundle->flip_addrs[planes_count].flip_timestamp_in_us = div_u64(timestamp_ns, 1000); bundle->surface_updates[planes_count].flip_addr = &bundle->flip_addrs[planes_count]; @@ -10047,6 +10055,29 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, acrtc_state->cursor_mode == DM_CURSOR_NATIVE_MODE) amdgpu_dm_commit_cursors(state); + /* + * On DCN, flip completion is normally delivered from VUPDATE_NO_LOCK. + * However, an immediate (tearing / async) flip is latched by HW right + * away and does not wait for the next vupdate, so deliver its + * completion event here after programming. + * + * On DCE, GRPH_PFLIP already fires immediately for immediate flips, so + * this is DCN-only. + */ + if (immediate_flip && amdgpu_ip_version(dm->adev, DCE_HWIP, 0) != 0) { + spin_lock_irqsave(&pcrtc->dev->event_lock, flags); + if (acrtc_attach->pflip_status == AMDGPU_FLIP_SUBMITTED && + acrtc_attach->event) { + drm_crtc_accurate_vblank_count(&acrtc_attach->base); + drm_crtc_send_vblank_event(&acrtc_attach->base, + acrtc_attach->event); + acrtc_attach->event = NULL; + drm_crtc_vblank_put(&acrtc_attach->base); + acrtc_attach->pflip_status = AMDGPU_FLIP_NONE; + } + spin_unlock_irqrestore(&pcrtc->dev->event_lock, flags); + } + cleanup: kfree(bundle); } @@ -11381,6 +11412,7 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm, /* Release extra reference */ if (new_stream) dc_stream_release(new_stream); + new_stream = NULL; /* * We want to do dc stream updates that do not require a diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c index 2e7ee77c010e..cc23d45815c3 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c @@ -327,9 +327,25 @@ static inline int amdgpu_dm_crtc_set_vblank(struct drm_crtc *crtc, bool enable) drm_crtc_vblank_restore(crtc); } - if (dc_supports_vrr(dm->dc->ctx->dce_version)) { + /* + * On DCN, VUPDATE_NO_LOCK is the single OTG interrupt used to deliver + * vblank and pageflip completion events, so enable it whenever vblank + * is enabled. On DCE, vupdate is only needed in VRR mode. + */ + if (amdgpu_ip_version(adev, DCE_HWIP, 0) != 0) { if (enable) { - /* vblank irq on -> Only need vupdate irq in vrr mode */ + rc = amdgpu_irq_get(adev, &adev->vupdate_irq, irq_type); + drm_dbg_vbl(crtc->dev, "Get vupdate_irq ret=%d\n", rc); + } else { + rc = amdgpu_irq_put(adev, &adev->vupdate_irq, irq_type); + drm_dbg_vbl(crtc->dev, "Put vupdate_irq ret=%d\n", rc); + } + } else if (dc_supports_vrr(dm->dc->ctx->dce_version)) { + if (enable) { + /* vblank irq on -> Only need vupdate irq in vrr mode + * Not ref-counted since we need explicit enable/disable + * for DCE VRR handling + */ if (amdgpu_dm_crtc_vrr_active(acrtc_state)) rc = amdgpu_dm_crtc_set_vupdate_irq(crtc, true); } else { @@ -338,39 +354,46 @@ static inline int amdgpu_dm_crtc_set_vblank(struct drm_crtc *crtc, bool enable) } } - if (rc) - return rc; - - /* crtc vblank or vstartup interrupt */ - if (enable) { - rc = amdgpu_irq_get(adev, &adev->crtc_irq, irq_type); - drm_dbg_vbl(crtc->dev, "Get crtc_irq ret=%d\n", rc); - } else { - rc = amdgpu_irq_put(adev, &adev->crtc_irq, irq_type); - drm_dbg_vbl(crtc->dev, "Put crtc_irq ret=%d\n", rc); - } - if (rc) return rc; /* - * hubp surface flip interrupt - * - * We have no guarantee that the frontend index maps to the same - * backend index - some even map to more than one. - * - * TODO: Use a different interrupt or check DC itself for the mapping. + * VLINE0 (crtc_irq) and GRPH_PFLIP (pageflip_irq) are only used on + * DCE. On DCN, vblank and pageflip completion are delivered from + * VUPDATE_NO_LOCK (enabled above), so don't touch them here. */ - if (enable) { - rc = amdgpu_irq_get(adev, &adev->pageflip_irq, irq_type); - drm_dbg_vbl(crtc->dev, "Get pageflip_irq ret=%d\n", rc); - } else { - rc = amdgpu_irq_put(adev, &adev->pageflip_irq, irq_type); - drm_dbg_vbl(crtc->dev, "Put pageflip_irq ret=%d\n", rc); - } + if (amdgpu_ip_version(adev, DCE_HWIP, 0) == 0) { + /* crtc vblank or vstartup interrupt */ + if (enable) { + rc = amdgpu_irq_get(adev, &adev->crtc_irq, irq_type); + drm_dbg_vbl(crtc->dev, "Get crtc_irq ret=%d\n", rc); + } else { + rc = amdgpu_irq_put(adev, &adev->crtc_irq, irq_type); + drm_dbg_vbl(crtc->dev, "Put crtc_irq ret=%d\n", rc); + } - if (rc) - return rc; + if (rc) + return rc; + + /* + * hubp surface flip interrupt + * + * We have no guarantee that the frontend index maps to the same + * backend index - some even map to more than one. + * + * TODO: Use a different interrupt or check DC itself for the mapping. + */ + if (enable) { + rc = amdgpu_irq_get(adev, &adev->pageflip_irq, irq_type); + drm_dbg_vbl(crtc->dev, "Get pageflip_irq ret=%d\n", rc); + } else { + rc = amdgpu_irq_put(adev, &adev->pageflip_irq, irq_type); + drm_dbg_vbl(crtc->dev, "Put pageflip_irq ret=%d\n", rc); + } + + if (rc) + return rc; + } #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) /* crtc vline0 interrupt, only available on DCN+ */ diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c index 7da7b41bd092..b98d946a8d2e 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn32/dcn32_clk_mgr.c @@ -276,13 +276,20 @@ static void dcn32_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr, struct dtbclk_dto_params dto_params = {0}; /* use mask to program DTO once per tg */ - if (pipe_ctx->stream_res.tg && + if (pipe_ctx->stream && pipe_ctx->stream_res.tg && !(tg_mask & (1 << pipe_ctx->stream_res.tg->inst))) { tg_mask |= (1 << pipe_ctx->stream_res.tg->inst); dto_params.otg_inst = pipe_ctx->stream_res.tg->inst; dto_params.ref_dtbclk_khz = ref_dtbclk_khz; + if (dccg->ctx->dc->link_srv->dp_is_128b_132b_signal(pipe_ctx)) + dto_params.pixclk_khz = pipe_ctx->stream->timing.pix_clk_100hz / 10; + + if (dc_is_hdmi_signal(pipe_ctx->stream->signal) || + dc_is_dvi_signal(pipe_ctx->stream->signal)) + dto_params.is_hdmi = true; + dccg->funcs->set_dtbclk_dto(clk_mgr->dccg, &dto_params); //dccg->funcs->set_audio_dtbclk_dto(clk_mgr->dccg, &dto_params); } diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn35/dcn35_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn35/dcn35_clk_mgr.c index 817a0253d10e..d1785a5c7f85 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn35/dcn35_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn35/dcn35_clk_mgr.c @@ -264,13 +264,20 @@ static void dcn35_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr, struct dtbclk_dto_params dto_params = {0}; /* use mask to program DTO once per tg */ - if (pipe_ctx->stream_res.tg && + if (pipe_ctx->stream && pipe_ctx->stream_res.tg && !(tg_mask & (1 << pipe_ctx->stream_res.tg->inst))) { tg_mask |= (1 << pipe_ctx->stream_res.tg->inst); dto_params.otg_inst = pipe_ctx->stream_res.tg->inst; dto_params.ref_dtbclk_khz = ref_dtbclk_khz; + if (dccg->ctx->dc->link_srv->dp_is_128b_132b_signal(pipe_ctx)) + dto_params.pixclk_khz = pipe_ctx->stream->timing.pix_clk_100hz / 10; + + if (dc_is_hdmi_signal(pipe_ctx->stream->signal) || + dc_is_dvi_signal(pipe_ctx->stream->signal)) + dto_params.is_hdmi = true; + dccg->funcs->set_dtbclk_dto(clk_mgr->dccg, &dto_params); //dccg->funcs->set_audio_dtbclk_dto(clk_mgr->dccg, &dto_params); } diff --git a/drivers/gpu/drm/amd/display/dc/link/link_detection.c b/drivers/gpu/drm/amd/display/dc/link/link_detection.c index 31b404e9b9ae..37ca3bb734c3 100644 --- a/drivers/gpu/drm/amd/display/dc/link/link_detection.c +++ b/drivers/gpu/drm/amd/display/dc/link/link_detection.c @@ -979,8 +979,11 @@ static bool detect_link_and_local_sink(struct dc_link *link, link->link_enc->features.flags.bits.DP_IS_USB_C == 1) { /* if alt mode times out, return false */ - if (!wait_for_entering_dp_alt_mode(link)) + if (!wait_for_entering_dp_alt_mode(link)) { + if (prev_sink) + dc_sink_release(prev_sink); return false; + } } if (!detect_dp(link, &sink_caps, reason)) { diff --git a/drivers/gpu/drm/amd/display/dc/resource/dce100/dce100_resource.c b/drivers/gpu/drm/amd/display/dc/resource/dce100/dce100_resource.c index ef146f253577..91fd73f4a96c 100644 --- a/drivers/gpu/drm/amd/display/dc/resource/dce100/dce100_resource.c +++ b/drivers/gpu/drm/amd/display/dc/resource/dce100/dce100_resource.c @@ -957,6 +957,11 @@ struct stream_encoder *dce100_find_first_free_match_stream_enc_for_link( for (i = 0; i < pool->stream_enc_count; i++) { if (!res_ctx->is_stream_enc_acquired[i] && pool->stream_enc[i]) { + /* DP/MST needs a digital encoder; skip analog/no-DP encoders */ + if (dc_is_dp_signal(stream->signal) && + (!pool->stream_enc[i]->funcs || + !pool->stream_enc[i]->funcs->dp_set_stream_attribute)) + continue; /* Store first available for MST second display * in daisy chain use case */ @@ -980,7 +985,7 @@ struct stream_encoder *dce100_find_first_free_match_stream_enc_for_link( * required for non DP connectors. */ - if (j >= 0 && link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT) + if (j >= 0 && dc_is_dp_signal(stream->signal)) return pool->stream_enc[j]; return NULL; diff --git a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c index ae858a43c35f..b3d55cac3569 100644 --- a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c +++ b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c @@ -447,8 +447,6 @@ void mod_build_vsc_infopacket(const struct dc_stream_state *stream, * * @stream: contains data we may need to construct VSIF (i.e. timing_3d_format, etc.) * @info_packet: output structure where to store VSIF - * @ALLMEnabled: indicates whether ALLM HF-VSIF should be generated - * @ALLMValue: ALLM bit value to advertise in HF-VSIF */ void mod_build_hf_vsif_infopacket(const struct dc_stream_state *stream, struct dc_info_packet *info_packet) diff --git a/drivers/gpu/drm/amd/pm/amdgpu_pm.c b/drivers/gpu/drm/amd/pm/amdgpu_pm.c index 0d72118a12b4..7c9e2003e2a0 100644 --- a/drivers/gpu/drm/amd/pm/amdgpu_pm.c +++ b/drivers/gpu/drm/amd/pm/amdgpu_pm.c @@ -40,6 +40,8 @@ #define DEVICE_ATTR_IS(_name) (attr_id == device_attr_id__##_name) +#define power_2_mwatt(power) (((power) >> 8) * 1000 + ((power) & 0xff)) + struct od_attribute { struct kobj_attribute attribute; struct list_head entry; @@ -2617,6 +2619,11 @@ static int default_attr_update(struct amdgpu_device *adev, struct amdgpu_device_ gc_ver != IP_VERSION(9, 4, 3)) || gc_ver < IP_VERSION(9, 0, 0)) *states = ATTR_STATE_UNSUPPORTED; + + if (adev->scpm_enabled) { + dev_attr->attr.mode &= ~S_IWUGO; + dev_attr->store = NULL; + } } else if (DEVICE_ATTR_IS(gpu_metrics)) { if (gc_ver < IP_VERSION(9, 1, 0)) *states = ATTR_STATE_UNSUPPORTED; @@ -3261,7 +3268,6 @@ static int amdgpu_hwmon_get_power(struct device *dev, enum amd_pp_sensors sensor) { struct amdgpu_device *adev = dev_get_drvdata(dev); - unsigned int uw; u32 query = 0; int r; @@ -3270,9 +3276,7 @@ static int amdgpu_hwmon_get_power(struct device *dev, return r; /* convert to microwatts */ - uw = (query >> 8) * 1000000 + (query & 0xff) * 1000; - - return uw; + return power_2_mwatt(query) * 1000; } static ssize_t amdgpu_hwmon_show_power_avg(struct device *dev, @@ -4804,7 +4808,7 @@ static int amdgpu_debugfs_pm_info_pp(struct seq_file *m, struct amdgpu_device *a { uint32_t mp1_ver = amdgpu_ip_version(adev, MP1_HWIP, 0); uint32_t gc_ver = amdgpu_ip_version(adev, GC_HWIP, 0); - uint32_t value; + uint32_t value, mwatt, centiwatt; uint64_t value64 = 0; uint32_t query = 0; int size; @@ -4829,17 +4833,21 @@ static int amdgpu_debugfs_pm_info_pp(struct seq_file *m, struct amdgpu_device *a seq_printf(m, "\t%u mV (VDDNB)\n", value); size = sizeof(uint32_t); if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_AVG_POWER, (void *)&query, &size)) { + mwatt = power_2_mwatt(query); + centiwatt = DIV_ROUND_CLOSEST(mwatt, 10); if (adev->flags & AMD_IS_APU) - seq_printf(m, "\t%u.%02u W (average SoC including CPU)\n", query >> 8, query & 0xff); + seq_printf(m, "\t%u.%02u W (average SoC including CPU)\n", centiwatt / 100, centiwatt % 100); else - seq_printf(m, "\t%u.%02u W (average SoC)\n", query >> 8, query & 0xff); + seq_printf(m, "\t%u.%02u W (average SoC)\n", centiwatt / 100, centiwatt % 100); } size = sizeof(uint32_t); if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_INPUT_POWER, (void *)&query, &size)) { + mwatt = power_2_mwatt(query); + centiwatt = DIV_ROUND_CLOSEST(mwatt, 10); if (adev->flags & AMD_IS_APU) - seq_printf(m, "\t%u.%02u W (current SoC including CPU)\n", query >> 8, query & 0xff); + seq_printf(m, "\t%u.%02u W (current SoC including CPU)\n", centiwatt / 100, centiwatt % 100); else - seq_printf(m, "\t%u.%02u W (current SoC)\n", query >> 8, query & 0xff); + seq_printf(m, "\t%u.%02u W (current SoC)\n", centiwatt / 100, centiwatt % 100); } size = sizeof(value); seq_printf(m, "\n"); diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c index 1d6e30269d56..4d553be56396 100644 --- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c +++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/hwmgr.c @@ -106,11 +106,8 @@ int hwmgr_early_init(struct pp_hwmgr *hwmgr) hwmgr->od_enabled = false; switch (hwmgr->chip_id) { case CHIP_BONAIRE: - /* R9 M380 in iMac 2015: SMU hangs when enabling MCLK DPM - * R7 260X cards with old MC ucode: MCLK DPM is unstable - */ - if (adev->pdev->subsystem_vendor == 0x106B || - adev->pdev->device == 0x6658) { + /* R9 M380 in iMac 2015: SMU hangs when enabling MCLK DPM */ + if (adev->pdev->subsystem_vendor == 0x106B) { dev_info(adev->dev, "disabling MCLK DPM on quirky ASIC"); adev->pm.pp_feature &= ~PP_MCLK_DPM_MASK; hwmgr->feature_mask &= ~PP_MCLK_DPM_MASK; diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c index 96a2b5ab87d8..f5c20f232062 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c @@ -2481,11 +2481,14 @@ static int smu_v13_0_0_get_power_limit(struct smu_context *smu, uint32_t pp_limit = smu->adev->pm.ac_power ? skutable->SocketPowerLimitAc[PPT_THROTTLER_PPT0] : skutable->SocketPowerLimitDc[PPT_THROTTLER_PPT0]; - uint32_t power_limit = 0, od_percent_upper = 0, od_percent_lower = 0; + uint32_t msg_limit = skutable->MsgLimits.Power[PPT_THROTTLER_PPT0][POWER_SOURCE_AC]; + uint32_t min_limit = min_t(uint32_t, pp_limit, msg_limit); + uint32_t max_limit = max_t(uint32_t, pp_limit, msg_limit); + uint32_t od_percent_upper = 0, od_percent_lower = 0; int ret; if (current_power_limit) { - ret = smu_v13_0_get_current_power_limit(smu, &power_limit); + ret = smu_v13_0_get_current_power_limit(smu, current_power_limit); if (ret) *current_power_limit = pp_limit; } @@ -2508,12 +2511,12 @@ static int smu_v13_0_0_get_power_limit(struct smu_context *smu, od_percent_upper, od_percent_lower, pp_limit); if (max_power_limit) { - *max_power_limit = pp_limit * (100 + od_percent_upper); + *max_power_limit = max_limit * (100 + od_percent_upper); *max_power_limit /= 100; } if (min_power_limit) { - *min_power_limit = pp_limit * (100 - od_percent_lower); + *min_power_limit = min_limit * (100 - od_percent_lower); *min_power_limit /= 100; } diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c index 0843fa0f5e2a..427f3569f1d3 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c @@ -2442,15 +2442,16 @@ static int smu_v13_0_7_get_power_limit(struct smu_context *smu, uint32_t pp_limit = smu->adev->pm.ac_power ? skutable->SocketPowerLimitAc[PPT_THROTTLER_PPT0] : skutable->SocketPowerLimitDc[PPT_THROTTLER_PPT0]; - uint32_t power_limit = 0, od_percent_upper = 0, od_percent_lower = 0; + uint32_t msg_limit = skutable->MsgLimits.Power[PPT_THROTTLER_PPT0][POWER_SOURCE_AC]; + uint32_t min_limit = min_t(uint32_t, pp_limit, msg_limit); + uint32_t max_limit = max_t(uint32_t, pp_limit, msg_limit); + uint32_t od_percent_upper = 0, od_percent_lower = 0; int ret; if (current_power_limit) { - ret = smu_v13_0_get_current_power_limit(smu, &power_limit); + ret = smu_v13_0_get_current_power_limit(smu, current_power_limit); if (ret) - power_limit = pp_limit; - - *current_power_limit = power_limit; + *current_power_limit = pp_limit; } if (default_power_limit) @@ -2471,12 +2472,12 @@ static int smu_v13_0_7_get_power_limit(struct smu_context *smu, od_percent_upper, od_percent_lower, pp_limit); if (max_power_limit) { - *max_power_limit = pp_limit * (100 + od_percent_upper); + *max_power_limit = max_limit * (100 + od_percent_upper); *max_power_limit /= 100; } if (min_power_limit) { - *min_power_limit = pp_limit * (100 - od_percent_lower); + *min_power_limit = min_limit * (100 - od_percent_lower); *min_power_limit /= 100; } diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c index 969be5ad0681..9e42cc8dd133 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c @@ -1701,19 +1701,23 @@ static int smu_v14_0_2_get_power_limit(struct smu_context *smu, table_context->power_play_table; PPTable_t *pptable = table_context->driver_pptable; CustomSkuTable_t *skutable = &pptable->CustomSkuTable; - int16_t od_percent_upper = 0, od_percent_lower = 0; + uint32_t pp_limit = smu->adev->pm.ac_power ? + skutable->SocketPowerLimitAc[PPT_THROTTLER_PPT0] : + skutable->SocketPowerLimitDc[PPT_THROTTLER_PPT0]; uint32_t msg_limit = pptable->SkuTable.MsgLimits.Power[PPT_THROTTLER_PPT0][POWER_SOURCE_AC]; - uint32_t power_limit; + uint32_t min_limit = min_t(uint32_t, pp_limit, msg_limit); + uint32_t max_limit = max_t(uint32_t, pp_limit, msg_limit); + int16_t od_percent_upper = 0, od_percent_lower = 0; + int ret; - if (smu_v14_0_get_current_power_limit(smu, &power_limit)) - power_limit = smu->adev->pm.ac_power ? - skutable->SocketPowerLimitAc[PPT_THROTTLER_PPT0] : - skutable->SocketPowerLimitDc[PPT_THROTTLER_PPT0]; + if (current_power_limit) { + ret = smu_v14_0_get_current_power_limit(smu, current_power_limit); + if (ret) + *current_power_limit = pp_limit; + } - if (current_power_limit) - *current_power_limit = power_limit; if (default_power_limit) - *default_power_limit = power_limit; + *default_power_limit = pp_limit; if (powerplay_table) { if (smu->od_enabled && @@ -1727,15 +1731,15 @@ static int smu_v14_0_2_get_power_limit(struct smu_context *smu, } dev_dbg(smu->adev->dev, "od percent upper:%d, od percent lower:%d (default power: %d)\n", - od_percent_upper, od_percent_lower, power_limit); + od_percent_upper, od_percent_lower, pp_limit); if (max_power_limit) { - *max_power_limit = msg_limit * (100 + od_percent_upper); + *max_power_limit = max_limit * (100 + od_percent_upper); *max_power_limit /= 100; } if (min_power_limit) { - *min_power_limit = power_limit * (100 + od_percent_lower); + *min_power_limit = min_limit * (100 + od_percent_lower); *min_power_limit /= 100; } diff --git a/drivers/gpu/drm/bridge/cadence/cdns-dsi-core.c b/drivers/gpu/drm/bridge/cadence/cdns-dsi-core.c index 09b289f0fcbf..de2a32112913 100644 --- a/drivers/gpu/drm/bridge/cadence/cdns-dsi-core.c +++ b/drivers/gpu/drm/bridge/cadence/cdns-dsi-core.c @@ -1230,7 +1230,7 @@ static const struct mipi_dsi_host_ops cdns_dsi_ops = { .transfer = cdns_dsi_transfer, }; -static int __maybe_unused cdns_dsi_resume(struct device *dev) +static int cdns_dsi_resume(struct device *dev) { struct cdns_dsi *dsi = dev_get_drvdata(dev); @@ -1241,7 +1241,7 @@ static int __maybe_unused cdns_dsi_resume(struct device *dev) return 0; } -static int __maybe_unused cdns_dsi_suspend(struct device *dev) +static int cdns_dsi_suspend(struct device *dev) { struct cdns_dsi *dsi = dev_get_drvdata(dev); @@ -1251,8 +1251,9 @@ static int __maybe_unused cdns_dsi_suspend(struct device *dev) return 0; } -static UNIVERSAL_DEV_PM_OPS(cdns_dsi_pm_ops, cdns_dsi_suspend, cdns_dsi_resume, - NULL); +static const struct dev_pm_ops cdns_dsi_pm_ops = { + RUNTIME_PM_OPS(cdns_dsi_suspend, cdns_dsi_resume, NULL) +}; static int cdns_dsi_drm_probe(struct platform_device *pdev) { @@ -1399,7 +1400,7 @@ static struct platform_driver cdns_dsi_platform_driver = { .driver = { .name = "cdns-dsi", .of_match_table = cdns_dsi_of_match, - .pm = &cdns_dsi_pm_ops, + .pm = pm_ptr(&cdns_dsi_pm_ops), }, }; module_platform_driver(cdns_dsi_platform_driver); diff --git a/drivers/gpu/drm/display/drm_dp_mst_topology.c b/drivers/gpu/drm/display/drm_dp_mst_topology.c index be749dcad3b5..383cfd99f955 100644 --- a/drivers/gpu/drm/display/drm_dp_mst_topology.c +++ b/drivers/gpu/drm/display/drm_dp_mst_topology.c @@ -789,6 +789,12 @@ static bool drm_dp_sideband_append_payload(struct drm_dp_sideband_msg_rx *msg, { u8 crc4; + /* curchunk_len must be >= 1 (min 1 CRC byte) and fit in chunk[] */ + if (!msg->curchunk_len || + msg->curchunk_len > ARRAY_SIZE(msg->chunk) || + msg->curchunk_idx + replybuflen > ARRAY_SIZE(msg->chunk)) + return false; + memcpy(&msg->chunk[msg->curchunk_idx], replybuf, replybuflen); msg->curchunk_idx += replybuflen; @@ -799,6 +805,9 @@ static bool drm_dp_sideband_append_payload(struct drm_dp_sideband_msg_rx *msg, print_hex_dump(KERN_DEBUG, "wrong crc", DUMP_PREFIX_NONE, 16, 1, msg->chunk, msg->curchunk_len, false); + /* Guard against accumulated msg[] overflow */ + if (msg->curlen + msg->curchunk_len - 1 > ARRAY_SIZE(msg->msg)) + return false; /* copy chunk into bigger msg */ memcpy(&msg->msg[msg->curlen], msg->chunk, msg->curchunk_len - 1); msg->curlen += msg->curchunk_len - 1; @@ -871,7 +880,7 @@ static bool drm_dp_sideband_parse_remote_dpcd_read(struct drm_dp_sideband_msg_rx goto fail_len; repmsg->u.remote_dpcd_read_ack.num_bytes = raw->msg[idx]; idx++; - if (idx > raw->curlen) + if (idx + repmsg->u.remote_dpcd_read_ack.num_bytes > raw->curlen) goto fail_len; memcpy(repmsg->u.remote_dpcd_read_ack.bytes, &raw->msg[idx], repmsg->u.remote_dpcd_read_ack.num_bytes); @@ -907,7 +916,9 @@ static bool drm_dp_sideband_parse_remote_i2c_read_ack(struct drm_dp_sideband_msg goto fail_len; repmsg->u.remote_i2c_read_ack.num_bytes = raw->msg[idx]; idx++; - /* TODO check */ + if (idx + repmsg->u.remote_i2c_read_ack.num_bytes > raw->curlen) + goto fail_len; + memcpy(repmsg->u.remote_i2c_read_ack.bytes, &raw->msg[idx], repmsg->u.remote_i2c_read_ack.num_bytes); return true; fail_len: @@ -923,16 +934,13 @@ static bool drm_dp_sideband_parse_enum_path_resources_ack(struct drm_dp_sideband repmsg->u.path_resources.port_number = (raw->msg[idx] >> 4) & 0xf; repmsg->u.path_resources.fec_capable = raw->msg[idx] & 0x1; idx++; - if (idx > raw->curlen) + if (idx + 2 > raw->curlen) goto fail_len; repmsg->u.path_resources.full_payload_bw_number = (raw->msg[idx] << 8) | (raw->msg[idx+1]); idx += 2; - if (idx > raw->curlen) + if (idx + 2 > raw->curlen) goto fail_len; repmsg->u.path_resources.avail_payload_bw_number = (raw->msg[idx] << 8) | (raw->msg[idx+1]); - idx += 2; - if (idx > raw->curlen) - goto fail_len; return true; fail_len: DRM_DEBUG_KMS("enum resource parse length fail %d %d\n", idx, raw->curlen); @@ -950,12 +958,9 @@ static bool drm_dp_sideband_parse_allocate_payload_ack(struct drm_dp_sideband_ms goto fail_len; repmsg->u.allocate_payload.vcpi = raw->msg[idx]; idx++; - if (idx > raw->curlen) + if (idx + 2 > raw->curlen) goto fail_len; repmsg->u.allocate_payload.allocated_pbn = (raw->msg[idx] << 8) | (raw->msg[idx+1]); - idx += 2; - if (idx > raw->curlen) - goto fail_len; return true; fail_len: DRM_DEBUG_KMS("allocate payload parse length fail %d %d\n", idx, raw->curlen); @@ -969,12 +974,9 @@ static bool drm_dp_sideband_parse_query_payload_ack(struct drm_dp_sideband_msg_r repmsg->u.query_payload.port_number = (raw->msg[idx] >> 4) & 0xf; idx++; - if (idx > raw->curlen) + if (idx + 2 > raw->curlen) goto fail_len; repmsg->u.query_payload.allocated_pbn = (raw->msg[idx] << 8) | (raw->msg[idx + 1]); - idx += 2; - if (idx > raw->curlen) - goto fail_len; return true; fail_len: DRM_DEBUG_KMS("query payload parse length fail %d %d\n", idx, raw->curlen); @@ -3738,8 +3740,10 @@ void drm_dp_mst_topology_queue_probe(struct drm_dp_mst_topology_mgr *mgr) { mutex_lock(&mgr->lock); - if (drm_WARN_ON(mgr->dev, !mgr->mst_state || !mgr->mst_primary)) + if (!mgr->mst_state || !mgr->mst_primary) { + drm_dbg_kms(mgr->dev, "queue_probe skipped: topology torn down\n"); goto out_unlock; + } drm_dp_mst_topology_mgr_invalidate_mstb(mgr->mst_primary); drm_dp_mst_queue_probe_work(mgr); diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 640d93070bb7..841f3de5f307 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -10,1250 +10,9 @@ #include #include +#include #include - -enum drm_buddy_free_tree { - DRM_BUDDY_CLEAR_TREE = 0, - DRM_BUDDY_DIRTY_TREE, - DRM_BUDDY_MAX_FREE_TREES, -}; - -static struct kmem_cache *slab_blocks; - -#define for_each_free_tree(tree) \ - for ((tree) = 0; (tree) < DRM_BUDDY_MAX_FREE_TREES; (tree)++) - -static struct drm_buddy_block *drm_block_alloc(struct drm_buddy *mm, - struct drm_buddy_block *parent, - unsigned int order, - u64 offset) -{ - struct drm_buddy_block *block; - - BUG_ON(order > DRM_BUDDY_MAX_ORDER); - - block = kmem_cache_zalloc(slab_blocks, GFP_KERNEL); - if (!block) - return NULL; - - block->header = offset; - block->header |= order; - block->parent = parent; - - RB_CLEAR_NODE(&block->rb); - - BUG_ON(block->header & DRM_BUDDY_HEADER_UNUSED); - return block; -} - -static void drm_block_free(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - kmem_cache_free(slab_blocks, block); -} - -static enum drm_buddy_free_tree -get_block_tree(struct drm_buddy_block *block) -{ - return drm_buddy_block_is_clear(block) ? - DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE; -} - -static struct drm_buddy_block * -rbtree_get_free_block(const struct rb_node *node) -{ - return node ? rb_entry(node, struct drm_buddy_block, rb) : NULL; -} - -static struct drm_buddy_block * -rbtree_last_free_block(struct rb_root *root) -{ - return rbtree_get_free_block(rb_last(root)); -} - -static bool rbtree_is_empty(struct rb_root *root) -{ - return RB_EMPTY_ROOT(root); -} - -static bool drm_buddy_block_offset_less(const struct drm_buddy_block *block, - const struct drm_buddy_block *node) -{ - return drm_buddy_block_offset(block) < drm_buddy_block_offset(node); -} - -static bool rbtree_block_offset_less(struct rb_node *block, - const struct rb_node *node) -{ - return drm_buddy_block_offset_less(rbtree_get_free_block(block), - rbtree_get_free_block(node)); -} - -static void rbtree_insert(struct drm_buddy *mm, - struct drm_buddy_block *block, - enum drm_buddy_free_tree tree) -{ - rb_add(&block->rb, - &mm->free_trees[tree][drm_buddy_block_order(block)], - rbtree_block_offset_less); -} - -static void rbtree_remove(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - unsigned int order = drm_buddy_block_order(block); - enum drm_buddy_free_tree tree; - struct rb_root *root; - - tree = get_block_tree(block); - root = &mm->free_trees[tree][order]; - - rb_erase(&block->rb, root); - RB_CLEAR_NODE(&block->rb); -} - -static void clear_reset(struct drm_buddy_block *block) -{ - block->header &= ~DRM_BUDDY_HEADER_CLEAR; -} - -static void mark_cleared(struct drm_buddy_block *block) -{ - block->header |= DRM_BUDDY_HEADER_CLEAR; -} - -static void mark_allocated(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - block->header &= ~DRM_BUDDY_HEADER_STATE; - block->header |= DRM_BUDDY_ALLOCATED; - - rbtree_remove(mm, block); -} - -static void mark_free(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - enum drm_buddy_free_tree tree; - - block->header &= ~DRM_BUDDY_HEADER_STATE; - block->header |= DRM_BUDDY_FREE; - - tree = get_block_tree(block); - rbtree_insert(mm, block, tree); -} - -static void mark_split(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - block->header &= ~DRM_BUDDY_HEADER_STATE; - block->header |= DRM_BUDDY_SPLIT; - - rbtree_remove(mm, block); -} - -static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2) -{ - return s1 <= e2 && e1 >= s2; -} - -static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2) -{ - return s1 <= s2 && e1 >= e2; -} - -static struct drm_buddy_block * -__get_buddy(struct drm_buddy_block *block) -{ - struct drm_buddy_block *parent; - - parent = block->parent; - if (!parent) - return NULL; - - if (parent->left == block) - return parent->right; - - return parent->left; -} - -static unsigned int __drm_buddy_free(struct drm_buddy *mm, - struct drm_buddy_block *block, - bool force_merge) -{ - struct drm_buddy_block *parent; - unsigned int order; - - while ((parent = block->parent)) { - struct drm_buddy_block *buddy; - - buddy = __get_buddy(block); - - if (!drm_buddy_block_is_free(buddy)) - break; - - if (!force_merge) { - /* - * Check the block and its buddy clear state and exit - * the loop if they both have the dissimilar state. - */ - if (drm_buddy_block_is_clear(block) != - drm_buddy_block_is_clear(buddy)) - break; - - if (drm_buddy_block_is_clear(block)) - mark_cleared(parent); - } - - rbtree_remove(mm, buddy); - if (force_merge && drm_buddy_block_is_clear(buddy)) - mm->clear_avail -= drm_buddy_block_size(mm, buddy); - - drm_block_free(mm, block); - drm_block_free(mm, buddy); - - block = parent; - } - - order = drm_buddy_block_order(block); - mark_free(mm, block); - - return order; -} - -static int __force_merge(struct drm_buddy *mm, - u64 start, - u64 end, - unsigned int min_order) -{ - unsigned int tree, order; - int i; - - if (!min_order) - return -ENOMEM; - - if (min_order > mm->max_order) - return -EINVAL; - - for_each_free_tree(tree) { - for (i = min_order - 1; i >= 0; i--) { - struct rb_node *iter = rb_last(&mm->free_trees[tree][i]); - - while (iter) { - struct drm_buddy_block *block, *buddy; - u64 block_start, block_end; - - block = rbtree_get_free_block(iter); - iter = rb_prev(iter); - - if (!block || !block->parent) - continue; - - block_start = drm_buddy_block_offset(block); - block_end = block_start + drm_buddy_block_size(mm, block) - 1; - - if (!contains(start, end, block_start, block_end)) - continue; - - buddy = __get_buddy(block); - if (!drm_buddy_block_is_free(buddy)) - continue; - - WARN_ON(drm_buddy_block_is_clear(block) == - drm_buddy_block_is_clear(buddy)); - - /* - * Advance to the next node when the current node is the buddy, - * as freeing the block will also remove its buddy from the tree. - */ - if (iter == &buddy->rb) - iter = rb_prev(iter); - - rbtree_remove(mm, block); - if (drm_buddy_block_is_clear(block)) - mm->clear_avail -= drm_buddy_block_size(mm, block); - - order = __drm_buddy_free(mm, block, true); - if (order >= min_order) - return 0; - } - } - } - - return -ENOMEM; -} - -/** - * drm_buddy_init - init memory manager - * - * @mm: DRM buddy manager to initialize - * @size: size in bytes to manage - * @chunk_size: minimum page size in bytes for our allocations - * - * Initializes the memory manager and its resources. - * - * Returns: - * 0 on success, error code on failure. - */ -int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size) -{ - unsigned int i, j, root_count = 0; - u64 offset = 0; - - if (size < chunk_size) - return -EINVAL; - - if (chunk_size < SZ_4K) - return -EINVAL; - - if (!is_power_of_2(chunk_size)) - return -EINVAL; - - size = round_down(size, chunk_size); - - mm->size = size; - mm->avail = size; - mm->clear_avail = 0; - mm->chunk_size = chunk_size; - mm->max_order = ilog2(size) - ilog2(chunk_size); - - BUG_ON(mm->max_order > DRM_BUDDY_MAX_ORDER); - - mm->free_trees = kmalloc_array(DRM_BUDDY_MAX_FREE_TREES, - sizeof(*mm->free_trees), - GFP_KERNEL); - if (!mm->free_trees) - return -ENOMEM; - - for_each_free_tree(i) { - mm->free_trees[i] = kmalloc_array(mm->max_order + 1, - sizeof(struct rb_root), - GFP_KERNEL); - if (!mm->free_trees[i]) - goto out_free_tree; - - for (j = 0; j <= mm->max_order; ++j) - mm->free_trees[i][j] = RB_ROOT; - } - - mm->n_roots = hweight64(size); - - mm->roots = kmalloc_array(mm->n_roots, - sizeof(struct drm_buddy_block *), - GFP_KERNEL); - if (!mm->roots) - goto out_free_tree; - - /* - * Split into power-of-two blocks, in case we are given a size that is - * not itself a power-of-two. - */ - do { - struct drm_buddy_block *root; - unsigned int order; - u64 root_size; - - order = ilog2(size) - ilog2(chunk_size); - root_size = chunk_size << order; - - root = drm_block_alloc(mm, NULL, order, offset); - if (!root) - goto out_free_roots; - - mark_free(mm, root); - - BUG_ON(root_count > mm->max_order); - BUG_ON(drm_buddy_block_size(mm, root) < chunk_size); - - mm->roots[root_count] = root; - - offset += root_size; - size -= root_size; - root_count++; - } while (size); - - return 0; - -out_free_roots: - while (root_count--) - drm_block_free(mm, mm->roots[root_count]); - kfree(mm->roots); -out_free_tree: - while (i--) - kfree(mm->free_trees[i]); - kfree(mm->free_trees); - return -ENOMEM; -} -EXPORT_SYMBOL(drm_buddy_init); - -/** - * drm_buddy_fini - tear down the memory manager - * - * @mm: DRM buddy manager to free - * - * Cleanup memory manager resources and the freetree - */ -void drm_buddy_fini(struct drm_buddy *mm) -{ - u64 root_size, size, start; - unsigned int order; - int i; - - size = mm->size; - - for (i = 0; i < mm->n_roots; ++i) { - order = ilog2(size) - ilog2(mm->chunk_size); - start = drm_buddy_block_offset(mm->roots[i]); - __force_merge(mm, start, start + size, order); - - if (WARN_ON(!drm_buddy_block_is_free(mm->roots[i]))) - kunit_fail_current_test("buddy_fini() root"); - - drm_block_free(mm, mm->roots[i]); - - root_size = mm->chunk_size << order; - size -= root_size; - } - - WARN_ON(mm->avail != mm->size); - - for_each_free_tree(i) - kfree(mm->free_trees[i]); - kfree(mm->free_trees); - kfree(mm->roots); -} -EXPORT_SYMBOL(drm_buddy_fini); - -static int split_block(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - unsigned int block_order = drm_buddy_block_order(block) - 1; - u64 offset = drm_buddy_block_offset(block); - - BUG_ON(!drm_buddy_block_is_free(block)); - BUG_ON(!drm_buddy_block_order(block)); - - block->left = drm_block_alloc(mm, block, block_order, offset); - if (!block->left) - return -ENOMEM; - - block->right = drm_block_alloc(mm, block, block_order, - offset + (mm->chunk_size << block_order)); - if (!block->right) { - drm_block_free(mm, block->left); - return -ENOMEM; - } - - mark_split(mm, block); - - if (drm_buddy_block_is_clear(block)) { - mark_cleared(block->left); - mark_cleared(block->right); - clear_reset(block); - } - - mark_free(mm, block->left); - mark_free(mm, block->right); - - return 0; -} - -/** - * drm_get_buddy - get buddy address - * - * @block: DRM buddy block - * - * Returns the corresponding buddy block for @block, or NULL - * if this is a root block and can't be merged further. - * Requires some kind of locking to protect against - * any concurrent allocate and free operations. - */ -struct drm_buddy_block * -drm_get_buddy(struct drm_buddy_block *block) -{ - return __get_buddy(block); -} -EXPORT_SYMBOL(drm_get_buddy); - -/** - * drm_buddy_reset_clear - reset blocks clear state - * - * @mm: DRM buddy manager - * @is_clear: blocks clear state - * - * Reset the clear state based on @is_clear value for each block - * in the freetree. - */ -void drm_buddy_reset_clear(struct drm_buddy *mm, bool is_clear) -{ - enum drm_buddy_free_tree src_tree, dst_tree; - u64 root_size, size, start; - unsigned int order; - int i; - - size = mm->size; - for (i = 0; i < mm->n_roots; ++i) { - order = ilog2(size) - ilog2(mm->chunk_size); - start = drm_buddy_block_offset(mm->roots[i]); - __force_merge(mm, start, start + size, order); - - root_size = mm->chunk_size << order; - size -= root_size; - } - - src_tree = is_clear ? DRM_BUDDY_DIRTY_TREE : DRM_BUDDY_CLEAR_TREE; - dst_tree = is_clear ? DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE; - - for (i = 0; i <= mm->max_order; ++i) { - struct rb_root *root = &mm->free_trees[src_tree][i]; - struct drm_buddy_block *block, *tmp; - - rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) { - rbtree_remove(mm, block); - if (is_clear) { - mark_cleared(block); - mm->clear_avail += drm_buddy_block_size(mm, block); - } else { - clear_reset(block); - mm->clear_avail -= drm_buddy_block_size(mm, block); - } - - rbtree_insert(mm, block, dst_tree); - } - } -} -EXPORT_SYMBOL(drm_buddy_reset_clear); - -/** - * drm_buddy_free_block - free a block - * - * @mm: DRM buddy manager - * @block: block to be freed - */ -void drm_buddy_free_block(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - BUG_ON(!drm_buddy_block_is_allocated(block)); - mm->avail += drm_buddy_block_size(mm, block); - if (drm_buddy_block_is_clear(block)) - mm->clear_avail += drm_buddy_block_size(mm, block); - - __drm_buddy_free(mm, block, false); -} -EXPORT_SYMBOL(drm_buddy_free_block); - -static void __drm_buddy_free_list(struct drm_buddy *mm, - struct list_head *objects, - bool mark_clear, - bool mark_dirty) -{ - struct drm_buddy_block *block, *on; - - WARN_ON(mark_dirty && mark_clear); - - list_for_each_entry_safe(block, on, objects, link) { - if (mark_clear) - mark_cleared(block); - else if (mark_dirty) - clear_reset(block); - drm_buddy_free_block(mm, block); - cond_resched(); - } - INIT_LIST_HEAD(objects); -} - -static void drm_buddy_free_list_internal(struct drm_buddy *mm, - struct list_head *objects) -{ - /* - * Don't touch the clear/dirty bit, since allocation is still internal - * at this point. For example we might have just failed part of the - * allocation. - */ - __drm_buddy_free_list(mm, objects, false, false); -} - -/** - * drm_buddy_free_list - free blocks - * - * @mm: DRM buddy manager - * @objects: input list head to free blocks - * @flags: optional flags like DRM_BUDDY_CLEARED - */ -void drm_buddy_free_list(struct drm_buddy *mm, - struct list_head *objects, - unsigned int flags) -{ - bool mark_clear = flags & DRM_BUDDY_CLEARED; - - __drm_buddy_free_list(mm, objects, mark_clear, !mark_clear); -} -EXPORT_SYMBOL(drm_buddy_free_list); - -static bool block_incompatible(struct drm_buddy_block *block, unsigned int flags) -{ - bool needs_clear = flags & DRM_BUDDY_CLEAR_ALLOCATION; - - return needs_clear != drm_buddy_block_is_clear(block); -} - -static struct drm_buddy_block * -__alloc_range_bias(struct drm_buddy *mm, - u64 start, u64 end, - unsigned int order, - unsigned long flags, - bool fallback) -{ - u64 req_size = mm->chunk_size << order; - struct drm_buddy_block *block; - struct drm_buddy_block *buddy; - LIST_HEAD(dfs); - int err; - int i; - - end = end - 1; - - for (i = 0; i < mm->n_roots; ++i) - list_add_tail(&mm->roots[i]->tmp_link, &dfs); - - do { - u64 block_start; - u64 block_end; - - block = list_first_entry_or_null(&dfs, - struct drm_buddy_block, - tmp_link); - if (!block) - break; - - list_del(&block->tmp_link); - - if (drm_buddy_block_order(block) < order) - continue; - - block_start = drm_buddy_block_offset(block); - block_end = block_start + drm_buddy_block_size(mm, block) - 1; - - if (!overlaps(start, end, block_start, block_end)) - continue; - - if (drm_buddy_block_is_allocated(block)) - continue; - - if (block_start < start || block_end > end) { - u64 adjusted_start = max(block_start, start); - u64 adjusted_end = min(block_end, end); - - if (round_down(adjusted_end + 1, req_size) <= - round_up(adjusted_start, req_size)) - continue; - } - - if (!fallback && block_incompatible(block, flags)) - continue; - - if (contains(start, end, block_start, block_end) && - order == drm_buddy_block_order(block)) { - /* - * Find the free block within the range. - */ - if (drm_buddy_block_is_free(block)) - return block; - - continue; - } - - if (!drm_buddy_block_is_split(block)) { - err = split_block(mm, block); - if (unlikely(err)) - goto err_undo; - } - - list_add(&block->right->tmp_link, &dfs); - list_add(&block->left->tmp_link, &dfs); - } while (1); - - return ERR_PTR(-ENOSPC); - -err_undo: - /* - * We really don't want to leave around a bunch of split blocks, since - * bigger is better, so make sure we merge everything back before we - * free the allocated blocks. - */ - buddy = __get_buddy(block); - if (buddy && - (drm_buddy_block_is_free(block) && - drm_buddy_block_is_free(buddy))) - __drm_buddy_free(mm, block, false); - return ERR_PTR(err); -} - -static struct drm_buddy_block * -__drm_buddy_alloc_range_bias(struct drm_buddy *mm, - u64 start, u64 end, - unsigned int order, - unsigned long flags) -{ - struct drm_buddy_block *block; - bool fallback = false; - - block = __alloc_range_bias(mm, start, end, order, - flags, fallback); - if (IS_ERR(block)) - return __alloc_range_bias(mm, start, end, order, - flags, !fallback); - - return block; -} - -static struct drm_buddy_block * -get_maxblock(struct drm_buddy *mm, - unsigned int order, - enum drm_buddy_free_tree tree) -{ - struct drm_buddy_block *max_block = NULL, *block = NULL; - struct rb_root *root; - unsigned int i; - - for (i = order; i <= mm->max_order; ++i) { - root = &mm->free_trees[tree][i]; - block = rbtree_last_free_block(root); - if (!block) - continue; - - if (!max_block) { - max_block = block; - continue; - } - - if (drm_buddy_block_offset(block) > - drm_buddy_block_offset(max_block)) { - max_block = block; - } - } - - return max_block; -} - -static struct drm_buddy_block * -alloc_from_freetree(struct drm_buddy *mm, - unsigned int order, - unsigned long flags) -{ - struct drm_buddy_block *block = NULL; - struct rb_root *root; - enum drm_buddy_free_tree tree; - unsigned int tmp; - int err; - - tree = (flags & DRM_BUDDY_CLEAR_ALLOCATION) ? - DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE; - - if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) { - block = get_maxblock(mm, order, tree); - if (block) - /* Store the obtained block order */ - tmp = drm_buddy_block_order(block); - } else { - for (tmp = order; tmp <= mm->max_order; ++tmp) { - /* Get RB tree root for this order and tree */ - root = &mm->free_trees[tree][tmp]; - block = rbtree_last_free_block(root); - if (block) - break; - } - } - - if (!block) { - /* Try allocating from the other tree */ - tree = (tree == DRM_BUDDY_CLEAR_TREE) ? - DRM_BUDDY_DIRTY_TREE : DRM_BUDDY_CLEAR_TREE; - - for (tmp = order; tmp <= mm->max_order; ++tmp) { - root = &mm->free_trees[tree][tmp]; - block = rbtree_last_free_block(root); - if (block) - break; - } - - if (!block) - return ERR_PTR(-ENOSPC); - } - - BUG_ON(!drm_buddy_block_is_free(block)); - - while (tmp != order) { - err = split_block(mm, block); - if (unlikely(err)) - goto err_undo; - - block = block->right; - tmp--; - } - return block; - -err_undo: - if (tmp != order) - __drm_buddy_free(mm, block, false); - return ERR_PTR(err); -} - -static int __alloc_range(struct drm_buddy *mm, - struct list_head *dfs, - u64 start, u64 size, - struct list_head *blocks, - u64 *total_allocated_on_err) -{ - struct drm_buddy_block *block; - struct drm_buddy_block *buddy; - u64 total_allocated = 0; - LIST_HEAD(allocated); - u64 end; - int err; - - end = start + size - 1; - - do { - u64 block_start; - u64 block_end; - - block = list_first_entry_or_null(dfs, - struct drm_buddy_block, - tmp_link); - if (!block) - break; - - list_del(&block->tmp_link); - - block_start = drm_buddy_block_offset(block); - block_end = block_start + drm_buddy_block_size(mm, block) - 1; - - if (!overlaps(start, end, block_start, block_end)) - continue; - - if (drm_buddy_block_is_allocated(block)) { - err = -ENOSPC; - goto err_free; - } - - if (contains(start, end, block_start, block_end)) { - if (drm_buddy_block_is_free(block)) { - mark_allocated(mm, block); - total_allocated += drm_buddy_block_size(mm, block); - mm->avail -= drm_buddy_block_size(mm, block); - if (drm_buddy_block_is_clear(block)) - mm->clear_avail -= drm_buddy_block_size(mm, block); - list_add_tail(&block->link, &allocated); - continue; - } else if (!mm->clear_avail) { - err = -ENOSPC; - goto err_free; - } - } - - if (!drm_buddy_block_is_split(block)) { - err = split_block(mm, block); - if (unlikely(err)) - goto err_undo; - } - - list_add(&block->right->tmp_link, dfs); - list_add(&block->left->tmp_link, dfs); - } while (1); - - if (total_allocated < size) { - err = -ENOSPC; - goto err_free; - } - - list_splice_tail(&allocated, blocks); - - return 0; - -err_undo: - /* - * We really don't want to leave around a bunch of split blocks, since - * bigger is better, so make sure we merge everything back before we - * free the allocated blocks. - */ - buddy = __get_buddy(block); - if (buddy && - (drm_buddy_block_is_free(block) && - drm_buddy_block_is_free(buddy))) - __drm_buddy_free(mm, block, false); - -err_free: - if (err == -ENOSPC && total_allocated_on_err) { - list_splice_tail(&allocated, blocks); - *total_allocated_on_err = total_allocated; - } else { - drm_buddy_free_list_internal(mm, &allocated); - } - - return err; -} - -static int __drm_buddy_alloc_range(struct drm_buddy *mm, - u64 start, - u64 size, - u64 *total_allocated_on_err, - struct list_head *blocks) -{ - LIST_HEAD(dfs); - int i; - - for (i = 0; i < mm->n_roots; ++i) - list_add_tail(&mm->roots[i]->tmp_link, &dfs); - - return __alloc_range(mm, &dfs, start, size, - blocks, total_allocated_on_err); -} - -static int __alloc_contig_try_harder(struct drm_buddy *mm, - u64 size, - u64 min_block_size, - struct list_head *blocks) -{ - u64 rhs_offset, lhs_offset, lhs_size, filled; - struct drm_buddy_block *block; - unsigned int tree, order; - LIST_HEAD(blocks_lhs); - unsigned long pages; - u64 modify_size; - int err; - - modify_size = rounddown_pow_of_two(size); - pages = modify_size >> ilog2(mm->chunk_size); - order = fls(pages) - 1; - if (order == 0) - return -ENOSPC; - - for_each_free_tree(tree) { - struct rb_root *root; - struct rb_node *iter; - - root = &mm->free_trees[tree][order]; - if (rbtree_is_empty(root)) - continue; - - iter = rb_last(root); - while (iter) { - block = rbtree_get_free_block(iter); - - /* Allocate blocks traversing RHS */ - rhs_offset = drm_buddy_block_offset(block); - err = __drm_buddy_alloc_range(mm, rhs_offset, size, - &filled, blocks); - if (!err || err != -ENOSPC) - return err; - - lhs_size = max((size - filled), min_block_size); - if (!IS_ALIGNED(lhs_size, min_block_size)) - lhs_size = round_up(lhs_size, min_block_size); - - /* Allocate blocks traversing LHS */ - lhs_offset = drm_buddy_block_offset(block) - lhs_size; - err = __drm_buddy_alloc_range(mm, lhs_offset, lhs_size, - NULL, &blocks_lhs); - if (!err) { - list_splice(&blocks_lhs, blocks); - return 0; - } else if (err != -ENOSPC) { - drm_buddy_free_list_internal(mm, blocks); - return err; - } - /* Free blocks for the next iteration */ - drm_buddy_free_list_internal(mm, blocks); - - iter = rb_prev(iter); - } - } - - return -ENOSPC; -} - -/** - * drm_buddy_block_trim - free unused pages - * - * @mm: DRM buddy manager - * @start: start address to begin the trimming. - * @new_size: original size requested - * @blocks: Input and output list of allocated blocks. - * MUST contain single block as input to be trimmed. - * On success will contain the newly allocated blocks - * making up the @new_size. Blocks always appear in - * ascending order - * - * For contiguous allocation, we round up the size to the nearest - * power of two value, drivers consume *actual* size, so remaining - * portions are unused and can be optionally freed with this function - * - * Returns: - * 0 on success, error code on failure. - */ -int drm_buddy_block_trim(struct drm_buddy *mm, - u64 *start, - u64 new_size, - struct list_head *blocks) -{ - struct drm_buddy_block *parent; - struct drm_buddy_block *block; - u64 block_start, block_end; - LIST_HEAD(dfs); - u64 new_start; - int err; - - if (!list_is_singular(blocks)) - return -EINVAL; - - block = list_first_entry(blocks, - struct drm_buddy_block, - link); - - block_start = drm_buddy_block_offset(block); - block_end = block_start + drm_buddy_block_size(mm, block); - - if (WARN_ON(!drm_buddy_block_is_allocated(block))) - return -EINVAL; - - if (new_size > drm_buddy_block_size(mm, block)) - return -EINVAL; - - if (!new_size || !IS_ALIGNED(new_size, mm->chunk_size)) - return -EINVAL; - - if (new_size == drm_buddy_block_size(mm, block)) - return 0; - - new_start = block_start; - if (start) { - new_start = *start; - - if (new_start < block_start) - return -EINVAL; - - if (!IS_ALIGNED(new_start, mm->chunk_size)) - return -EINVAL; - - if (range_overflows(new_start, new_size, block_end)) - return -EINVAL; - } - - list_del(&block->link); - mark_free(mm, block); - mm->avail += drm_buddy_block_size(mm, block); - if (drm_buddy_block_is_clear(block)) - mm->clear_avail += drm_buddy_block_size(mm, block); - - /* Prevent recursively freeing this node */ - parent = block->parent; - block->parent = NULL; - - list_add(&block->tmp_link, &dfs); - err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); - if (err) { - mark_allocated(mm, block); - mm->avail -= drm_buddy_block_size(mm, block); - if (drm_buddy_block_is_clear(block)) - mm->clear_avail -= drm_buddy_block_size(mm, block); - list_add(&block->link, blocks); - } - - block->parent = parent; - return err; -} -EXPORT_SYMBOL(drm_buddy_block_trim); - -static struct drm_buddy_block * -__drm_buddy_alloc_blocks(struct drm_buddy *mm, - u64 start, u64 end, - unsigned int order, - unsigned long flags) -{ - if (flags & DRM_BUDDY_RANGE_ALLOCATION) - /* Allocate traversing within the range */ - return __drm_buddy_alloc_range_bias(mm, start, end, - order, flags); - else - /* Allocate from freetree */ - return alloc_from_freetree(mm, order, flags); -} - -/** - * drm_buddy_alloc_blocks - allocate power-of-two blocks - * - * @mm: DRM buddy manager to allocate from - * @start: start of the allowed range for this block - * @end: end of the allowed range for this block - * @size: size of the allocation in bytes - * @min_block_size: alignment of the allocation - * @blocks: output list head to add allocated blocks - * @flags: DRM_BUDDY_*_ALLOCATION flags - * - * alloc_range_bias() called on range limitations, which traverses - * the tree and returns the desired block. - * - * alloc_from_freetree() called when *no* range restrictions - * are enforced, which picks the block from the freetree. - * - * Returns: - * 0 on success, error code on failure. - */ -int drm_buddy_alloc_blocks(struct drm_buddy *mm, - u64 start, u64 end, u64 size, - u64 min_block_size, - struct list_head *blocks, - unsigned long flags) -{ - struct drm_buddy_block *block = NULL; - u64 original_size, original_min_size; - unsigned int min_order, order; - LIST_HEAD(allocated); - unsigned long pages; - int err; - - if (size < mm->chunk_size) - return -EINVAL; - - if (min_block_size < mm->chunk_size) - return -EINVAL; - - if (!is_power_of_2(min_block_size)) - return -EINVAL; - - if (!IS_ALIGNED(start | end | size, mm->chunk_size)) - return -EINVAL; - - if (end > mm->size) - return -EINVAL; - - if (range_overflows(start, size, mm->size)) - return -EINVAL; - - /* Actual range allocation */ - if (start + size == end) { - if (!IS_ALIGNED(start | end, min_block_size)) - return -EINVAL; - - return __drm_buddy_alloc_range(mm, start, size, NULL, blocks); - } - - original_size = size; - original_min_size = min_block_size; - - /* Roundup the size to power of 2 */ - if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION) { - size = roundup_pow_of_two(size); - min_block_size = size; - /* Align size value to min_block_size */ - } else if (!IS_ALIGNED(size, min_block_size)) { - size = round_up(size, min_block_size); - } - - pages = size >> ilog2(mm->chunk_size); - order = fls(pages) - 1; - min_order = ilog2(min_block_size) - ilog2(mm->chunk_size); - - if (order > mm->max_order || size > mm->size) { - if ((flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION) && - !(flags & DRM_BUDDY_RANGE_ALLOCATION)) - return __alloc_contig_try_harder(mm, original_size, - original_min_size, blocks); - - return -EINVAL; - } - - do { - order = min(order, (unsigned int)fls(pages) - 1); - BUG_ON(order > mm->max_order); - BUG_ON(order < min_order); - - do { - block = __drm_buddy_alloc_blocks(mm, start, - end, - order, - flags); - if (!IS_ERR(block)) - break; - - if (order-- == min_order) { - /* Try allocation through force merge method */ - if (mm->clear_avail && - !__force_merge(mm, start, end, min_order)) { - block = __drm_buddy_alloc_blocks(mm, start, - end, - min_order, - flags); - if (!IS_ERR(block)) { - order = min_order; - break; - } - } - - /* - * Try contiguous block allocation through - * try harder method. - */ - if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION && - !(flags & DRM_BUDDY_RANGE_ALLOCATION)) - return __alloc_contig_try_harder(mm, - original_size, - original_min_size, - blocks); - err = -ENOSPC; - goto err_free; - } - } while (1); - - mark_allocated(mm, block); - mm->avail -= drm_buddy_block_size(mm, block); - if (drm_buddy_block_is_clear(block)) - mm->clear_avail -= drm_buddy_block_size(mm, block); - kmemleak_update_trace(block); - list_add_tail(&block->link, &allocated); - - pages -= BIT(order); - - if (!pages) - break; - } while (1); - - /* Trim the allocated block to the required size */ - if (!(flags & DRM_BUDDY_TRIM_DISABLE) && - original_size != size) { - struct list_head *trim_list; - LIST_HEAD(temp); - u64 trim_size; - - trim_list = &allocated; - trim_size = original_size; - - if (!list_is_singular(&allocated)) { - block = list_last_entry(&allocated, typeof(*block), link); - list_move(&block->link, &temp); - trim_list = &temp; - trim_size = drm_buddy_block_size(mm, block) - - (size - original_size); - } - - drm_buddy_block_trim(mm, - NULL, - trim_size, - trim_list); - - if (!list_empty(&temp)) - list_splice_tail(trim_list, &allocated); - } - - list_splice_tail(&allocated, blocks); - return 0; - -err_free: - drm_buddy_free_list_internal(mm, &allocated); - return err; -} -EXPORT_SYMBOL(drm_buddy_alloc_blocks); +#include /** * drm_buddy_block_print - print block information @@ -1262,12 +21,12 @@ EXPORT_SYMBOL(drm_buddy_alloc_blocks); * @block: DRM buddy block * @p: DRM printer to use */ -void drm_buddy_block_print(struct drm_buddy *mm, - struct drm_buddy_block *block, +void drm_buddy_block_print(struct gpu_buddy *mm, + struct gpu_buddy_block *block, struct drm_printer *p) { - u64 start = drm_buddy_block_offset(block); - u64 size = drm_buddy_block_size(mm, block); + u64 start = gpu_buddy_block_offset(block); + u64 size = gpu_buddy_block_size(mm, block); drm_printf(p, "%#018llx-%#018llx: %llu\n", start, start + size, size); } @@ -1279,7 +38,7 @@ EXPORT_SYMBOL(drm_buddy_block_print); * @mm: DRM buddy manager * @p: DRM printer to use */ -void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p) +void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p) { int order; @@ -1287,7 +46,7 @@ void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p) mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20); for (order = mm->max_order; order >= 0; order--) { - struct drm_buddy_block *block, *tmp; + struct gpu_buddy_block *block, *tmp; struct rb_root *root; u64 count = 0, free; unsigned int tree; @@ -1296,7 +55,7 @@ void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p) root = &mm->free_trees[tree][order]; rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) { - BUG_ON(!drm_buddy_block_is_free(block)); + BUG_ON(!gpu_buddy_block_is_free(block)); count++; } } @@ -1314,22 +73,5 @@ void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p) } EXPORT_SYMBOL(drm_buddy_print); -static void drm_buddy_module_exit(void) -{ - kmem_cache_destroy(slab_blocks); -} - -static int __init drm_buddy_module_init(void) -{ - slab_blocks = KMEM_CACHE(drm_buddy_block, 0); - if (!slab_blocks) - return -ENOMEM; - - return 0; -} - -module_init(drm_buddy_module_init); -module_exit(drm_buddy_module_exit); - -MODULE_DESCRIPTION("DRM Buddy Allocator"); +MODULE_DESCRIPTION("DRM-specific GPU Buddy Allocator Print Helpers"); MODULE_LICENSE("Dual MIT/GPL"); diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c index 272d6254ea47..0326b9adb5f1 100644 --- a/drivers/gpu/drm/drm_connector.c +++ b/drivers/gpu/drm/drm_connector.c @@ -3554,7 +3554,7 @@ EXPORT_SYMBOL(drm_mode_put_tile_group); /** * drm_mode_get_tile_group - get a reference to an existing tile group * @dev: DRM device - * @topology: 8-bytes unique per monitor. + * @topology_id: 9-byte unique ID per monitor. * * Use the unique bytes to get a reference to an existing tile group. * @@ -3562,14 +3562,14 @@ EXPORT_SYMBOL(drm_mode_put_tile_group); * tile group or NULL if not found. */ struct drm_tile_group *drm_mode_get_tile_group(struct drm_device *dev, - const char topology[8]) + const char topology_id[9]) { struct drm_tile_group *tg; int id; mutex_lock(&dev->mode_config.idr_mutex); idr_for_each_entry(&dev->mode_config.tile_idr, tg, id) { - if (!memcmp(tg->group_data, topology, 8)) { + if (!memcmp(tg->group_data, topology_id, sizeof(tg->group_data))) { if (!kref_get_unless_zero(&tg->refcount)) tg = NULL; mutex_unlock(&dev->mode_config.idr_mutex); @@ -3584,7 +3584,7 @@ EXPORT_SYMBOL(drm_mode_get_tile_group); /** * drm_mode_create_tile_group - create a tile group from a displayid description * @dev: DRM device - * @topology: 8-bytes unique per monitor. + * @topology_id: 9-byte unique ID per monitor. * * Create a tile group for the unique monitor, and get a unique * identifier for the tile group. @@ -3593,7 +3593,7 @@ EXPORT_SYMBOL(drm_mode_get_tile_group); * new tile group or NULL. */ struct drm_tile_group *drm_mode_create_tile_group(struct drm_device *dev, - const char topology[8]) + const char topology_id[9]) { struct drm_tile_group *tg; int ret; @@ -3603,7 +3603,7 @@ struct drm_tile_group *drm_mode_create_tile_group(struct drm_device *dev, return NULL; kref_init(&tg->refcount); - memcpy(tg->group_data, topology, 8); + memcpy(tg->group_data, topology_id, sizeof(tg->group_data)); tg->dev = dev; mutex_lock(&dev->mode_config.idr_mutex); diff --git a/drivers/gpu/drm/drm_displayid_internal.h b/drivers/gpu/drm/drm_displayid_internal.h index 5b1b32f73516..4590d6a3d821 100644 --- a/drivers/gpu/drm/drm_displayid_internal.h +++ b/drivers/gpu/drm/drm_displayid_internal.h @@ -109,7 +109,7 @@ struct displayid_tiled_block { u8 topo[3]; u8 tile_size[4]; u8 tile_pixel_bezel[5]; - u8 topology_id[8]; + u8 topology_id[9]; } __packed; struct displayid_detailed_timings_1 { diff --git a/drivers/gpu/drm/drm_gpusvm.c b/drivers/gpu/drm/drm_gpusvm.c index 4cde5bf4aedd..781cd4e5b704 100644 --- a/drivers/gpu/drm/drm_gpusvm.c +++ b/drivers/gpu/drm/drm_gpusvm.c @@ -1566,8 +1566,10 @@ int drm_gpusvm_range_evict(struct drm_gpusvm *gpusvm, return -EFAULT; pfns = kvmalloc_array(npages, sizeof(*pfns), GFP_KERNEL); - if (!pfns) + if (!pfns) { + mmput(mm); return -ENOMEM; + } hmm_range.hmm_pfns = pfns; while (!time_after(jiffies, timeout)) { diff --git a/drivers/gpu/drm/drm_panel_backlight_quirks.c b/drivers/gpu/drm/drm_panel_backlight_quirks.c index 537dc6dd0534..2d0238382ebd 100644 --- a/drivers/gpu/drm/drm_panel_backlight_quirks.c +++ b/drivers/gpu/drm/drm_panel_backlight_quirks.c @@ -21,6 +21,15 @@ struct drm_get_panel_backlight_quirk { }; static const struct drm_get_panel_backlight_quirk drm_panel_min_backlight_quirks[] = { + /* Lenovo Legion 5 15ARH05, AUX backlight non-functional, force PWM */ + { + .dmi_match.field = DMI_SYS_VENDOR, + .dmi_match.value = "LENOVO", + .dmi_match_other.field = DMI_PRODUCT_VERSION, + .dmi_match_other.value = "Lenovo Legion 5 15ARH05", + .ident.panel_id = drm_edid_encode_panel_id('B', 'O', 'E', 0x08df), + .quirk = { .force_pwm = true, }, + }, /* 13 inch matte panel */ { .dmi_match.field = DMI_BOARD_VENDOR, diff --git a/drivers/gpu/drm/i915/display/intel_bios.c b/drivers/gpu/drm/i915/display/intel_bios.c index 3596dce84c28..a2b6f40b4b0c 100644 --- a/drivers/gpu/drm/i915/display/intel_bios.c +++ b/drivers/gpu/drm/i915/display/intel_bios.c @@ -624,6 +624,21 @@ get_lfp_data_tail(const struct bdb_lfp_data *data, return NULL; } +static bool is_panel_type_valid(int panel_type) +{ + return panel_type >= 0 && panel_type < 16; +} + +static bool is_panel_type_pnp(int panel_type) +{ + return panel_type == 0xff; +} + +static bool is_panel_type_valid_or_pnp(int panel_type) +{ + return is_panel_type_valid(panel_type) || is_panel_type_pnp(panel_type); +} + static int opregion_get_panel_type(struct intel_display *display, const struct intel_bios_encoder_data *devdata, const struct drm_edid *drm_edid, bool use_fallback) @@ -641,15 +656,21 @@ static int vbt_get_panel_type(struct intel_display *display, if (!lfp_options) return -1; - if (lfp_options->panel_type > 0xf && - lfp_options->panel_type != 0xff) { + if (!is_panel_type_valid_or_pnp(lfp_options->panel_type)) { drm_dbg_kms(display->drm, "Invalid VBT panel type 0x%x\n", lfp_options->panel_type); return -1; } - if (devdata && devdata->child.handle == DEVICE_HANDLE_LFP2) + if (devdata && devdata->child.handle == DEVICE_HANDLE_LFP2) { + if (!is_panel_type_valid_or_pnp(lfp_options->panel_type2)) { + drm_dbg_kms(display->drm, "Invalid VBT panel type 2 0x%x\n", + lfp_options->panel_type2); + return -1; + } + return lfp_options->panel_type2; + } drm_WARN_ON(display->drm, devdata && devdata->child.handle != DEVICE_HANDLE_LFP1); @@ -763,13 +784,12 @@ static int get_panel_type(struct intel_display *display, panel_types[i].name, panel_types[i].panel_type); } - if (panel_types[PANEL_TYPE_OPREGION].panel_type >= 0) + if (is_panel_type_valid(panel_types[PANEL_TYPE_OPREGION].panel_type)) i = PANEL_TYPE_OPREGION; - else if (panel_types[PANEL_TYPE_VBT].panel_type == 0xff && - panel_types[PANEL_TYPE_PNPID].panel_type >= 0) + else if (is_panel_type_pnp(panel_types[PANEL_TYPE_VBT].panel_type) && + is_panel_type_valid(panel_types[PANEL_TYPE_PNPID].panel_type)) i = PANEL_TYPE_PNPID; - else if (panel_types[PANEL_TYPE_VBT].panel_type != 0xff && - panel_types[PANEL_TYPE_VBT].panel_type >= 0) + else if (is_panel_type_valid(panel_types[PANEL_TYPE_VBT].panel_type)) i = PANEL_TYPE_VBT; else i = PANEL_TYPE_FALLBACK; diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c index c7886b364770..2a0750acf205 100644 --- a/drivers/gpu/drm/i915/display/intel_dp.c +++ b/drivers/gpu/drm/i915/display/intel_dp.c @@ -5203,8 +5203,9 @@ intel_dp_check_mst_status(struct intel_dp *intel_dp) struct intel_encoder *encoder = &dig_port->base; bool link_ok = true; bool reprobe_needed = false; + int tries = 33; - for (;;) { + while (--tries) { u8 esi[4] = {}; u8 ack[4] = {}; @@ -5247,6 +5248,11 @@ intel_dp_check_mst_status(struct intel_dp *intel_dp) if (!link_ok || intel_dp->link.force_retrain) intel_encoder_link_check_queue_work(encoder, 0); + if (!tries) { + drm_dbg_kms(display->drm, "DPRX ESI not clearing, device may be stuck\n"); + reprobe_needed = true; + } + return !reprobe_needed; } diff --git a/drivers/gpu/drm/i915/display/intel_hdcp.c b/drivers/gpu/drm/i915/display/intel_hdcp.c index 531ee122bf82..68503a3f945e 100644 --- a/drivers/gpu/drm/i915/display/intel_hdcp.c +++ b/drivers/gpu/drm/i915/display/intel_hdcp.c @@ -140,6 +140,9 @@ intel_hdcp_required_content_stream(struct intel_atomic_state *state, if (conn_dig_port != dig_port) continue; + if (drm_WARN_ON(display->drm, data->k >= INTEL_NUM_PIPES(display))) + return -EINVAL; + data->streams[data->k].stream_id = intel_conn_to_vcpi(state, connector); data->k++; @@ -150,7 +153,7 @@ intel_hdcp_required_content_stream(struct intel_atomic_state *state, } drm_connector_list_iter_end(&conn_iter); - if (drm_WARN_ON(display->drm, data->k > INTEL_NUM_PIPES(display) || data->k == 0)) + if (drm_WARN_ON(display->drm, !data->k)) return -EINVAL; /* @@ -1794,9 +1797,10 @@ int hdcp2_authenticate_repeater_topology(struct intel_connector *connector) return -EINVAL; } - if (seq_num_v < hdcp->seq_num_v) { - /* Roll over of the seq_num_v from repeater. Reauthenticate. */ - drm_dbg_kms(display->drm, "Seq_num_v roll over.\n"); + if (hdcp->hdcp2_encrypted && seq_num_v <= hdcp->seq_num_v) { + /* Reauthenticate on Seq_num_v repeat or rollover */ + drm_dbg_kms(display->drm, "Seq_num_v %s\n", + seq_num_v == hdcp->seq_num_v ? "repeat" : "rollover"); return -EINVAL; } diff --git a/drivers/gpu/drm/i915/display/skl_watermark.c b/drivers/gpu/drm/i915/display/skl_watermark.c index a5862998bf14..eb54fffc6988 100644 --- a/drivers/gpu/drm/i915/display/skl_watermark.c +++ b/drivers/gpu/drm/i915/display/skl_watermark.c @@ -3815,7 +3815,7 @@ void skl_wm_plane_disable_noatomic(struct intel_crtc *crtc, return; skl_ddb_entry_init(&crtc_state->wm.skl.plane_ddb[plane->id], 0, 0); - skl_ddb_entry_init(&crtc_state->wm.skl.plane_ddb[plane->id], 0, 0); + skl_ddb_entry_init(&crtc_state->wm.skl.plane_ddb_y[plane->id], 0, 0); crtc_state->wm.skl.plane_min_ddb[plane->id] = 0; crtc_state->wm.skl.plane_interim_ddb[plane->id] = 0; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index ed6599694835..ae5194625508 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -612,6 +612,7 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base, return -EINVAL; } + slot = array_index_nospec(slot, set->num_engines); if (set->engines[slot].type != I915_GEM_ENGINE_TYPE_INVALID) { drm_dbg(&i915->drm, "Invalid placement[%d], already occupied\n", slot); @@ -769,8 +770,8 @@ static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv, struct intel_engine_cs *engine; if (copy_from_user(&ci, &user->engines[n], sizeof(ci))) { - kfree(set.engines); - return -EFAULT; + err = -EFAULT; + goto err; } memset(&set.engines[n], 0, sizeof(set.engines[n])); @@ -786,8 +787,8 @@ static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv, drm_dbg(&i915->drm, "Invalid engine[%d]: { class:%d, instance:%d }\n", n, ci.engine_class, ci.engine_instance); - kfree(set.engines); - return -ENOENT; + err = -ENOENT; + goto err; } set.engines[n].type = I915_GEM_ENGINE_TYPE_PHYSICAL; @@ -800,15 +801,21 @@ static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv, set_proto_ctx_engines_extensions, ARRAY_SIZE(set_proto_ctx_engines_extensions), &set); - if (err) { - kfree(set.engines); - return err; - } + if (err) + goto err_extensions; pc->num_user_engines = set.num_engines; pc->user_engines = set.engines; return 0; + +err_extensions: + for (n = 0; n < set.num_engines; n++) + kfree(set.engines[n].siblings); +err: + kfree(set.engines); + + return err; } static int set_proto_ctx_sseu(struct drm_i915_file_private *fpriv, @@ -850,7 +857,7 @@ static int set_proto_ctx_sseu(struct drm_i915_file_private *fpriv, pe = &pc->user_engines[idx]; /* Only render engine supports RPCS configuration. */ - if (pe->engine->class != RENDER_CLASS) + if (!pe->engine || pe->engine->class != RENDER_CLASS) return -EINVAL; sseu = &pe->sseu; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 3d53c54eb48f..d17695ae0a31 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -5,9 +5,10 @@ #include +#include +#include #include #include -#include #include "i915_drv.h" #include "i915_ttm_buddy_manager.h" diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c index 833987015b8b..ed42b9d05a07 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_user.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c @@ -257,7 +257,7 @@ void intel_engines_driver_register(struct drm_i915_private *i915) p = &prev->rb_right; } - if (IS_ENABLED(CONFIG_DRM_I915_SELFTESTS) && + if (IS_ENABLED(CONFIG_DRM_I915_SELFTEST) && IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)) { struct intel_engine_cs *engine; unsigned int isolation; diff --git a/drivers/gpu/drm/i915/gt/selftest_gt_pm.c b/drivers/gpu/drm/i915/gt/selftest_gt_pm.c index 33351deeea4f..07eaf71955c4 100644 --- a/drivers/gpu/drm/i915/gt/selftest_gt_pm.c +++ b/drivers/gpu/drm/i915/gt/selftest_gt_pm.c @@ -16,9 +16,9 @@ static int cmp_u64(const void *A, const void *B) { const u64 *a = A, *b = B; - if (a < b) + if (*a < *b) return -1; - else if (a > b) + else if (*a > *b) return 1; else return 0; @@ -28,9 +28,9 @@ static int cmp_u32(const void *A, const void *B) { const u32 *a = A, *b = B; - if (a < b) + if (*a < *b) return -1; - else if (a > b) + else if (*a > *b) return 1; else return 0; diff --git a/drivers/gpu/drm/i915/i915_active.c b/drivers/gpu/drm/i915/i915_active.c index 6b0c1162505a..bbcfdf23c0a7 100644 --- a/drivers/gpu/drm/i915/i915_active.c +++ b/drivers/gpu/drm/i915/i915_active.c @@ -318,7 +318,7 @@ active_instance(struct i915_active *ref, u64 idx) */ node = kmem_cache_alloc(slab_cache, GFP_ATOMIC); if (!node) - goto out; + goto err; __i915_active_fence_init(&node->base, NULL, node_retire); node->ref = ref; @@ -332,6 +332,11 @@ active_instance(struct i915_active *ref, u64 idx) spin_unlock_irq(&ref->tree_lock); return &node->base; + +err: + spin_unlock_irq(&ref->tree_lock); + + return NULL; } void __i915_active_init(struct i915_active *ref, diff --git a/drivers/gpu/drm/i915/i915_scatterlist.c b/drivers/gpu/drm/i915/i915_scatterlist.c index 4d830740946d..6a34dae13769 100644 --- a/drivers/gpu/drm/i915/i915_scatterlist.c +++ b/drivers/gpu/drm/i915/i915_scatterlist.c @@ -7,7 +7,7 @@ #include "i915_scatterlist.h" #include "i915_ttm_buddy_manager.h" -#include +#include #include #include @@ -167,9 +167,9 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res, struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res); const u64 size = res->size; const u32 max_segment = round_down(UINT_MAX, page_alignment); - struct drm_buddy *mm = bman_res->mm; + struct gpu_buddy *mm = bman_res->mm; struct list_head *blocks = &bman_res->blocks; - struct drm_buddy_block *block; + struct gpu_buddy_block *block; struct i915_refct_sgt *rsgt; struct scatterlist *sg; struct sg_table *st; @@ -202,8 +202,8 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res, list_for_each_entry(block, blocks, link) { u64 block_size, offset; - block_size = min_t(u64, size, drm_buddy_block_size(mm, block)); - offset = drm_buddy_block_offset(block); + block_size = min_t(u64, size, gpu_buddy_block_size(mm, block)); + offset = gpu_buddy_block_offset(block); while (block_size) { u64 len; diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 942345548bc3..c5ca90088705 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -5,18 +5,19 @@ #include +#include +#include +#include #include #include -#include - #include "i915_ttm_buddy_manager.h" #include "i915_gem.h" struct i915_ttm_buddy_manager { struct ttm_resource_manager manager; - struct drm_buddy mm; + struct gpu_buddy mm; struct list_head reserved; struct mutex lock; unsigned long visible_size; @@ -38,7 +39,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, { struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); struct i915_ttm_buddy_resource *bman_res; - struct drm_buddy *mm = &bman->mm; + struct gpu_buddy *mm = &bman->mm; unsigned long n_pages, lpfn; u64 min_page_size; u64 size; @@ -57,13 +58,13 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, bman_res->mm = mm; if (place->flags & TTM_PL_FLAG_TOPDOWN) - bman_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + bman_res->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION; if (place->flags & TTM_PL_FLAG_CONTIGUOUS) - bman_res->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION; + bman_res->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION; if (place->fpfn || lpfn != man->size) - bman_res->flags |= DRM_BUDDY_RANGE_ALLOCATION; + bman_res->flags |= GPU_BUDDY_RANGE_ALLOCATION; GEM_BUG_ON(!bman_res->base.size); size = bman_res->base.size; @@ -89,7 +90,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, goto err_free_res; } - err = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT, + err = gpu_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT, (u64)lpfn << PAGE_SHIFT, (u64)n_pages << PAGE_SHIFT, min_page_size, @@ -101,15 +102,15 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, if (lpfn <= bman->visible_size) { bman_res->used_visible_size = PFN_UP(bman_res->base.size); } else { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; list_for_each_entry(block, &bman_res->blocks, link) { unsigned long start = - drm_buddy_block_offset(block) >> PAGE_SHIFT; + gpu_buddy_block_offset(block) >> PAGE_SHIFT; if (start < bman->visible_size) { unsigned long end = start + - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT); bman_res->used_visible_size += min(end, bman->visible_size) - start; @@ -126,7 +127,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man, return 0; err_free_blocks: - drm_buddy_free_list(mm, &bman_res->blocks, 0); + gpu_buddy_free_list(mm, &bman_res->blocks, 0); mutex_unlock(&bman->lock); err_free_res: ttm_resource_fini(man, &bman_res->base); @@ -141,7 +142,7 @@ static void i915_ttm_buddy_man_free(struct ttm_resource_manager *man, struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); mutex_lock(&bman->lock); - drm_buddy_free_list(&bman->mm, &bman_res->blocks, 0); + gpu_buddy_free_list(&bman->mm, &bman_res->blocks, 0); bman->visible_avail += bman_res->used_visible_size; mutex_unlock(&bman->lock); @@ -156,8 +157,8 @@ static bool i915_ttm_buddy_man_intersects(struct ttm_resource_manager *man, { struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res); struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); - struct drm_buddy *mm = &bman->mm; - struct drm_buddy_block *block; + struct gpu_buddy *mm = &bman->mm; + struct gpu_buddy_block *block; if (!place->fpfn && !place->lpfn) return true; @@ -176,9 +177,9 @@ static bool i915_ttm_buddy_man_intersects(struct ttm_resource_manager *man, /* Check each drm buddy block individually */ list_for_each_entry(block, &bman_res->blocks, link) { unsigned long fpfn = - drm_buddy_block_offset(block) >> PAGE_SHIFT; + gpu_buddy_block_offset(block) >> PAGE_SHIFT; unsigned long lpfn = fpfn + - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT); if (place->fpfn < lpfn && place->lpfn > fpfn) return true; @@ -194,8 +195,8 @@ static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man, { struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res); struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); - struct drm_buddy *mm = &bman->mm; - struct drm_buddy_block *block; + struct gpu_buddy *mm = &bman->mm; + struct gpu_buddy_block *block; if (!place->fpfn && !place->lpfn) return true; @@ -209,9 +210,9 @@ static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man, /* Check each drm buddy block individually */ list_for_each_entry(block, &bman_res->blocks, link) { unsigned long fpfn = - drm_buddy_block_offset(block) >> PAGE_SHIFT; + gpu_buddy_block_offset(block) >> PAGE_SHIFT; unsigned long lpfn = fpfn + - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT); if (fpfn < place->fpfn || lpfn > place->lpfn) return false; @@ -224,7 +225,7 @@ static void i915_ttm_buddy_man_debug(struct ttm_resource_manager *man, struct drm_printer *printer) { struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); - struct drm_buddy_block *block; + struct gpu_buddy_block *block; mutex_lock(&bman->lock); drm_printf(printer, "default_page_size: %lluKiB\n", @@ -293,7 +294,7 @@ int i915_ttm_buddy_man_init(struct ttm_device *bdev, if (!bman) return -ENOMEM; - err = drm_buddy_init(&bman->mm, size, chunk_size); + err = gpu_buddy_init(&bman->mm, size, chunk_size); if (err) goto err_free_bman; @@ -333,7 +334,7 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type) { struct ttm_resource_manager *man = ttm_manager_type(bdev, type); struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); - struct drm_buddy *mm = &bman->mm; + struct gpu_buddy *mm = &bman->mm; int ret; ttm_resource_manager_set_used(man, false); @@ -345,8 +346,8 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type) ttm_set_driver_manager(bdev, type, NULL); mutex_lock(&bman->lock); - drm_buddy_free_list(mm, &bman->reserved, 0); - drm_buddy_fini(mm); + gpu_buddy_free_list(mm, &bman->reserved, 0); + gpu_buddy_fini(mm); bman->visible_avail += bman->visible_reserved; WARN_ON_ONCE(bman->visible_avail != bman->visible_size); mutex_unlock(&bman->lock); @@ -371,15 +372,15 @@ int i915_ttm_buddy_man_reserve(struct ttm_resource_manager *man, u64 start, u64 size) { struct i915_ttm_buddy_manager *bman = to_buddy_manager(man); - struct drm_buddy *mm = &bman->mm; + struct gpu_buddy *mm = &bman->mm; unsigned long fpfn = start >> PAGE_SHIFT; unsigned long flags = 0; int ret; - flags |= DRM_BUDDY_RANGE_ALLOCATION; + flags |= GPU_BUDDY_RANGE_ALLOCATION; mutex_lock(&bman->lock); - ret = drm_buddy_alloc_blocks(mm, start, + ret = gpu_buddy_alloc_blocks(mm, start, start + size, size, mm->chunk_size, &bman->reserved, diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h index d64620712830..1cff018c1689 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h @@ -13,7 +13,7 @@ struct ttm_device; struct ttm_resource_manager; -struct drm_buddy; +struct gpu_buddy; /** * struct i915_ttm_buddy_resource @@ -33,7 +33,7 @@ struct i915_ttm_buddy_resource { struct list_head blocks; unsigned long flags; unsigned long used_visible_size; - struct drm_buddy *mm; + struct gpu_buddy *mm; }; /** diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c index 7b856b5090f9..8307390943a2 100644 --- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c +++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c @@ -6,7 +6,7 @@ #include #include -#include +#include #include "../i915_selftest.h" @@ -371,7 +371,7 @@ static int igt_mock_splintered_region(void *arg) struct drm_i915_private *i915 = mem->i915; struct i915_ttm_buddy_resource *res; struct drm_i915_gem_object *obj; - struct drm_buddy *mm; + struct gpu_buddy *mm; unsigned int expected_order; LIST_HEAD(objects); u64 size; @@ -447,8 +447,8 @@ static int igt_mock_max_segment(void *arg) struct drm_i915_private *i915 = mem->i915; struct i915_ttm_buddy_resource *res; struct drm_i915_gem_object *obj; - struct drm_buddy_block *block; - struct drm_buddy *mm; + struct gpu_buddy_block *block; + struct gpu_buddy *mm; struct list_head *blocks; struct scatterlist *sg; I915_RND_STATE(prng); @@ -487,8 +487,8 @@ static int igt_mock_max_segment(void *arg) mm = res->mm; size = 0; list_for_each_entry(block, blocks, link) { - if (drm_buddy_block_size(mm, block) > size) - size = drm_buddy_block_size(mm, block); + if (gpu_buddy_block_size(mm, block) > size) + size = gpu_buddy_block_size(mm, block); } if (size < max_segment) { pr_err("%s: Failed to create a huge contiguous block [> %u], largest block %lld\n", @@ -527,14 +527,14 @@ static u64 igt_object_mappable_total(struct drm_i915_gem_object *obj) struct intel_memory_region *mr = obj->mm.region; struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(obj->mm.res); - struct drm_buddy *mm = bman_res->mm; - struct drm_buddy_block *block; + struct gpu_buddy *mm = bman_res->mm; + struct gpu_buddy_block *block; u64 total; total = 0; list_for_each_entry(block, &bman_res->blocks, link) { - u64 start = drm_buddy_block_offset(block); - u64 end = start + drm_buddy_block_size(mm, block); + u64 start = gpu_buddy_block_offset(block); + u64 end = start + gpu_buddy_block_size(mm, block); if (start < resource_size(&mr->io)) total += min_t(u64, end, resource_size(&mr->io)) - start; diff --git a/drivers/gpu/drm/imagination/pvr_context.c b/drivers/gpu/drm/imagination/pvr_context.c index 5edc3c01af72..d9066a4fefcf 100644 --- a/drivers/gpu/drm/imagination/pvr_context.c +++ b/drivers/gpu/drm/imagination/pvr_context.c @@ -161,22 +161,24 @@ ctx_fw_data_init(void *cpu_ptr, void *priv) /** * pvr_context_destroy_queues() - Destroy all queues attached to a context. * @ctx: Context to destroy queues on. + * @cleanup_queue_entity: Whether to cleanup the queue entity e.g. context + * creation failure path. * * Should be called when the last reference to a context object is dropped. * It releases all resources attached to the queues bound to this context. */ -static void pvr_context_destroy_queues(struct pvr_context *ctx) +static void pvr_context_destroy_queues(struct pvr_context *ctx, bool cleanup_queue_entity) { switch (ctx->type) { case DRM_PVR_CTX_TYPE_RENDER: - pvr_queue_destroy(ctx->queues.fragment); - pvr_queue_destroy(ctx->queues.geometry); + pvr_queue_destroy(ctx->queues.fragment, cleanup_queue_entity); + pvr_queue_destroy(ctx->queues.geometry, cleanup_queue_entity); break; case DRM_PVR_CTX_TYPE_COMPUTE: - pvr_queue_destroy(ctx->queues.compute); + pvr_queue_destroy(ctx->queues.compute, cleanup_queue_entity); break; case DRM_PVR_CTX_TYPE_TRANSFER_FRAG: - pvr_queue_destroy(ctx->queues.transfer); + pvr_queue_destroy(ctx->queues.transfer, cleanup_queue_entity); break; } } @@ -240,7 +242,7 @@ static int pvr_context_create_queues(struct pvr_context *ctx, return -EINVAL; err_destroy_queues: - pvr_context_destroy_queues(ctx); + pvr_context_destroy_queues(ctx, true); return err; } @@ -307,8 +309,8 @@ int pvr_context_create(struct pvr_file *pvr_file, struct drm_pvr_ioctl_create_co goto err_free_ctx; ctx->vm_ctx = pvr_vm_context_lookup(pvr_file, args->vm_context_handle); - if (IS_ERR(ctx->vm_ctx)) { - err = PTR_ERR(ctx->vm_ctx); + if (!ctx->vm_ctx) { + err = -EINVAL; goto err_free_ctx; } @@ -356,7 +358,7 @@ int pvr_context_create(struct pvr_file *pvr_file, struct drm_pvr_ioctl_create_co pvr_fw_object_destroy(ctx->fw_obj); err_destroy_queues: - pvr_context_destroy_queues(ctx); + pvr_context_destroy_queues(ctx, true); err_free_ctx_data: kfree(ctx->data); @@ -382,7 +384,7 @@ pvr_context_release(struct kref *ref_count) spin_unlock(&pvr_dev->ctx_list_lock); xa_erase(&pvr_dev->ctx_ids, ctx->ctx_id); - pvr_context_destroy_queues(ctx); + pvr_context_destroy_queues(ctx, false); pvr_fw_object_destroy(ctx->fw_obj); kfree(ctx->data); pvr_vm_context_put(ctx->vm_ctx); diff --git a/drivers/gpu/drm/imagination/pvr_drv.c b/drivers/gpu/drm/imagination/pvr_drv.c index 916b40ced7eb..e403411cf963 100644 --- a/drivers/gpu/drm/imagination/pvr_drv.c +++ b/drivers/gpu/drm/imagination/pvr_drv.c @@ -1252,14 +1252,13 @@ pvr_set_uobj_array(const struct drm_pvr_obj_array *out, u32 min_stride, u32 obj_ if (copy_to_user(out_ptr, in_ptr, cpy_elem_size)) return -EFAULT; - out_ptr += obj_size; - in_ptr += out->stride; - } + if (out->stride > obj_size && + clear_user(out_ptr + cpy_elem_size, out->stride - obj_size)) { + return -EFAULT; + } - if (out->stride > obj_size && - clear_user(u64_to_user_ptr(out->array + obj_size), - out->stride - obj_size)) { - return -EFAULT; + out_ptr += out->stride; + in_ptr += obj_size; } } diff --git a/drivers/gpu/drm/imagination/pvr_queue.c b/drivers/gpu/drm/imagination/pvr_queue.c index fc415dd0d7a7..bb5835ede666 100644 --- a/drivers/gpu/drm/imagination/pvr_queue.c +++ b/drivers/gpu/drm/imagination/pvr_queue.c @@ -179,7 +179,7 @@ static const struct dma_fence_ops pvr_queue_job_fence_ops = { /** * to_pvr_queue_job_fence() - Return a pvr_queue_fence object if the fence is - * backed by a UFO. + * already backed by a UFO. * @f: The dma_fence to turn into a pvr_queue_fence. * * Return: @@ -356,6 +356,15 @@ static u32 job_cmds_size(struct pvr_job *job, u32 ufo_wait_count) pvr_cccb_get_size_of_cmd_with_hdr(job->cmd_len); } +static bool +is_paired_job_fence(struct dma_fence *fence, struct pvr_job *job) +{ + /* This assumes "fence" is one of "job"'s drm_sched_job::dependencies */ + return job->type == DRM_PVR_JOB_TYPE_FRAGMENT && + job->paired_job && + &job->paired_job->base.s_fence->scheduled == fence; +} + /** * job_count_remaining_native_deps() - Count the number of non-signaled native dependencies. * @job: Job to operate on. @@ -371,6 +380,17 @@ static unsigned long job_count_remaining_native_deps(struct pvr_job *job) xa_for_each(&job->base.dependencies, index, fence) { struct pvr_queue_fence *jfence; + if (is_paired_job_fence(fence, job)) { + /* + * A fence between paired jobs won't resolve to a pvr_queue_fence (i.e. + * be backed by a UFO) until the jobs have been submitted, together. + * The submitting code will insert a partial render fence command for this. + */ + WARN_ON(dma_fence_is_signaled(fence)); + remaining_count++; + continue; + } + jfence = to_pvr_queue_job_fence(fence); if (!jfence) continue; @@ -468,10 +488,11 @@ pvr_queue_get_job_kccb_fence(struct pvr_queue *queue, struct pvr_job *job) } static struct dma_fence * -pvr_queue_get_paired_frag_job_dep(struct pvr_queue *queue, struct pvr_job *job) +pvr_queue_get_paired_frag_job_dep(struct pvr_job *job) { struct pvr_job *frag_job = job->type == DRM_PVR_JOB_TYPE_GEOMETRY ? job->paired_job : NULL; + struct pvr_queue *frag_queue = frag_job ? frag_job->ctx->queues.fragment : NULL; struct dma_fence *f; unsigned long index; @@ -490,7 +511,10 @@ pvr_queue_get_paired_frag_job_dep(struct pvr_queue *queue, struct pvr_job *job) return dma_fence_get(f); } - return frag_job->base.sched->ops->prepare_job(&frag_job->base, &queue->entity); + /* Initialize the paired fragment job's done_fence, so we can signal it. */ + pvr_queue_job_fence_init(frag_job->done_fence, frag_queue); + + return pvr_queue_get_job_cccb_fence(frag_queue, frag_job); } /** @@ -509,11 +533,6 @@ pvr_queue_prepare_job(struct drm_sched_job *sched_job, struct pvr_queue *queue = container_of(s_entity, struct pvr_queue, entity); struct dma_fence *internal_dep = NULL; - /* - * Initialize the done_fence, so we can signal it. This must be done - * here because otherwise by the time of run_job() the job will end up - * in the pending list without a valid fence. - */ if (job->type == DRM_PVR_JOB_TYPE_FRAGMENT && job->paired_job) { /* * This will be called on a paired fragment job after being @@ -523,18 +542,15 @@ pvr_queue_prepare_job(struct drm_sched_job *sched_job, */ if (job->paired_job->has_pm_ref) return NULL; - - /* - * In this case we need to use the job's own ctx to initialise - * the done_fence. The other steps are done in the ctx of the - * paired geometry job. - */ - pvr_queue_job_fence_init(job->done_fence, - job->ctx->queues.fragment); - } else { - pvr_queue_job_fence_init(job->done_fence, queue); } + /* + * Initialize the done_fence, so we can signal it. This must be done + * here because otherwise by the time of run_job() the job will end up + * in the pending list without a valid fence. + */ + pvr_queue_job_fence_init(job->done_fence, queue); + /* CCCB fence is used to make sure we have enough space in the CCCB to * submit our commands. */ @@ -555,7 +571,7 @@ pvr_queue_prepare_job(struct drm_sched_job *sched_job, /* The paired job fence should come last, when everything else is ready. */ if (!internal_dep) - internal_dep = pvr_queue_get_paired_frag_job_dep(queue, job); + internal_dep = pvr_queue_get_paired_frag_job_dep(job); return internal_dep; } @@ -630,9 +646,8 @@ static void pvr_queue_submit_job_to_cccb(struct pvr_job *job) if (!jfence) continue; - /* Skip the partial render fence, we will place it at the end. */ - if (job->type == DRM_PVR_JOB_TYPE_FRAGMENT && job->paired_job && - &job->paired_job->base.s_fence->scheduled == fence) + /* This fence will be placed last, as partial render fence. */ + if (is_paired_job_fence(fence, job)) continue; if (dma_fence_is_signaled(&jfence->base)) @@ -1386,11 +1401,12 @@ void pvr_queue_kill(struct pvr_queue *queue) /** * pvr_queue_destroy() - Destroy a queue. * @queue: The queue to destroy. + * @cleanup_queue_entity: Whether to cleanup the queue entity. * * Cleanup the queue and free the resources attached to it. Should be * called from the context release function. */ -void pvr_queue_destroy(struct pvr_queue *queue) +void pvr_queue_destroy(struct pvr_queue *queue, bool cleanup_queue_entity) { if (!queue) return; @@ -1400,7 +1416,8 @@ void pvr_queue_destroy(struct pvr_queue *queue) mutex_unlock(&queue->ctx->pvr_dev->queues.lock); drm_sched_fini(&queue->scheduler); - drm_sched_entity_fini(&queue->entity); + if (cleanup_queue_entity) + drm_sched_entity_fini(&queue->entity); if (WARN_ON(queue->last_queued_job_scheduled_fence)) dma_fence_put(queue->last_queued_job_scheduled_fence); diff --git a/drivers/gpu/drm/imagination/pvr_queue.h b/drivers/gpu/drm/imagination/pvr_queue.h index fc1986d73fc8..a205e29437f4 100644 --- a/drivers/gpu/drm/imagination/pvr_queue.h +++ b/drivers/gpu/drm/imagination/pvr_queue.h @@ -158,7 +158,7 @@ struct pvr_queue *pvr_queue_create(struct pvr_context *ctx, void pvr_queue_kill(struct pvr_queue *queue); -void pvr_queue_destroy(struct pvr_queue *queue); +void pvr_queue_destroy(struct pvr_queue *queue, bool cleanup_queue_entity); void pvr_queue_process(struct pvr_queue *queue); diff --git a/drivers/gpu/drm/imagination/pvr_vm.c b/drivers/gpu/drm/imagination/pvr_vm.c index 3d97990170bf..b2f8e89dafb7 100644 --- a/drivers/gpu/drm/imagination/pvr_vm.c +++ b/drivers/gpu/drm/imagination/pvr_vm.c @@ -746,6 +746,7 @@ pvr_vm_map(struct pvr_vm_context *vm_ctx, struct pvr_gem_object *pvr_obj, pvr_gem_object_get(pvr_obj); + mutex_lock(&vm_ctx->lock); err = drm_gpuvm_exec_lock(&vm_exec); if (err) goto err_cleanup; @@ -755,6 +756,7 @@ pvr_vm_map(struct pvr_vm_context *vm_ctx, struct pvr_gem_object *pvr_obj, drm_gpuvm_exec_unlock(&vm_exec); err_cleanup: + mutex_unlock(&vm_ctx->lock); pvr_vm_bind_op_fini(&bind_op); return err; diff --git a/drivers/gpu/drm/lib/drm_random.c b/drivers/gpu/drm/lib/drm_random.c deleted file mode 100644 index 0e9dba1ef4af..000000000000 --- a/drivers/gpu/drm/lib/drm_random.c +++ /dev/null @@ -1,44 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -#include -#include -#include -#include -#include -#include - -#include "drm_random.h" - -u32 drm_prandom_u32_max_state(u32 ep_ro, struct rnd_state *state) -{ - return upper_32_bits((u64)prandom_u32_state(state) * ep_ro); -} -EXPORT_SYMBOL(drm_prandom_u32_max_state); - -void drm_random_reorder(unsigned int *order, unsigned int count, - struct rnd_state *state) -{ - unsigned int i, j; - - for (i = 0; i < count; ++i) { - BUILD_BUG_ON(sizeof(unsigned int) > sizeof(u32)); - j = drm_prandom_u32_max_state(count, state); - swap(order[i], order[j]); - } -} -EXPORT_SYMBOL(drm_random_reorder); - -unsigned int *drm_random_order(unsigned int count, struct rnd_state *state) -{ - unsigned int *order, i; - - order = kmalloc_array(count, sizeof(*order), GFP_KERNEL); - if (!order) - return order; - - for (i = 0; i < count; i++) - order[i] = i; - - drm_random_reorder(order, count, state); - return order; -} -EXPORT_SYMBOL(drm_random_order); diff --git a/drivers/gpu/drm/lib/drm_random.h b/drivers/gpu/drm/lib/drm_random.h deleted file mode 100644 index 9f827260a89d..000000000000 --- a/drivers/gpu/drm/lib/drm_random.h +++ /dev/null @@ -1,28 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __DRM_RANDOM_H__ -#define __DRM_RANDOM_H__ - -/* This is a temporary home for a couple of utility functions that should - * be transposed to lib/ at the earliest convenience. - */ - -#include - -#define DRM_RND_STATE_INITIALIZER(seed__) ({ \ - struct rnd_state state__; \ - prandom_seed_state(&state__, (seed__)); \ - state__; \ -}) - -#define DRM_RND_STATE(name__, seed__) \ - struct rnd_state name__ = DRM_RND_STATE_INITIALIZER(seed__) - -unsigned int *drm_random_order(unsigned int count, - struct rnd_state *state); -void drm_random_reorder(unsigned int *order, - unsigned int count, - struct rnd_state *state); -u32 drm_prandom_u32_max_state(u32 ep_ro, - struct rnd_state *state); - -#endif /* !__DRM_RANDOM_H__ */ diff --git a/drivers/gpu/drm/nouveau/nouveau_exec.c b/drivers/gpu/drm/nouveau/nouveau_exec.c index c4949e815eb3..6c31093be1c7 100644 --- a/drivers/gpu/drm/nouveau/nouveau_exec.c +++ b/drivers/gpu/drm/nouveau/nouveau_exec.c @@ -331,10 +331,10 @@ nouveau_exec_ucopy(struct nouveau_exec_job_args *args, return 0; -err_free_pushs: - u_free(args->push.s); err_free_ins: u_free(args->in_sync.s); +err_free_pushs: + u_free(args->push.s); return ret; } diff --git a/drivers/gpu/drm/nouveau/nouveau_uvmm.c b/drivers/gpu/drm/nouveau/nouveau_uvmm.c index 79eefdfd08a2..fd518e3f1809 100644 --- a/drivers/gpu/drm/nouveau/nouveau_uvmm.c +++ b/drivers/gpu/drm/nouveau/nouveau_uvmm.c @@ -1711,10 +1711,10 @@ nouveau_uvmm_vm_bind_ucopy(struct nouveau_uvmm_bind_job_args *args, return 0; -err_free_ops: - u_free(args->op.s); err_free_ins: u_free(args->in_sync.s); +err_free_ops: + u_free(args->op.s); return ret; } diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c index 13d829593180..8b6274b945b0 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.c @@ -315,6 +315,7 @@ nvkm_acr_oneinit(struct nvkm_subdev *subdev) i, us, fw); } } + nvkm_done(acr->wpr); return -EINVAL; } nvkm_done(acr->wpr); diff --git a/drivers/gpu/drm/panel/Kconfig b/drivers/gpu/drm/panel/Kconfig index 1516c6fa265d..7354dc85a6aa 100644 --- a/drivers/gpu/drm/panel/Kconfig +++ b/drivers/gpu/drm/panel/Kconfig @@ -814,6 +814,7 @@ config DRM_PANEL_SAMSUNG_S6E3HA8 depends on OF depends on DRM_MIPI_DSI depends on BACKLIGHT_CLASS_DEVICE + select DRM_DISPLAY_HELPER select DRM_DISPLAY_DSC_HELPER help Say Y or M here if you want to enable support for the diff --git a/drivers/gpu/drm/panthor/panthor_device.c b/drivers/gpu/drm/panthor/panthor_device.c index 962a10e00848..cdf82d83de7a 100644 --- a/drivers/gpu/drm/panthor/panthor_device.c +++ b/drivers/gpu/drm/panthor/panthor_device.c @@ -182,7 +182,10 @@ int panthor_device_init(struct panthor_device *ptdev) return ret; #ifdef CONFIG_DEBUG_FS - drmm_mutex_init(&ptdev->base, &ptdev->gems.lock); + ret = drmm_mutex_init(&ptdev->base, &ptdev->gems.lock); + if (ret) + return ret; + INIT_LIST_HEAD(&ptdev->gems.node); #endif diff --git a/drivers/gpu/drm/panthor/panthor_fw.c b/drivers/gpu/drm/panthor/panthor_fw.c index df767e82148a..a8a5a8e26024 100644 --- a/drivers/gpu/drm/panthor/panthor_fw.c +++ b/drivers/gpu/drm/panthor/panthor_fw.c @@ -777,6 +777,7 @@ static int panthor_fw_load(struct panthor_device *ptdev) } if (hdr.size > iter.size) { + ret = -EINVAL; drm_err(&ptdev->base, "Firmware image is truncated\n"); goto out; } diff --git a/drivers/gpu/drm/radeon/r100.c b/drivers/gpu/drm/radeon/r100.c index 07a9c523a17a..59ea0b94932d 100644 --- a/drivers/gpu/drm/radeon/r100.c +++ b/drivers/gpu/drm/radeon/r100.c @@ -906,6 +906,7 @@ struct radeon_fence *r100_copy_blit(struct radeon_device *rdev, { struct radeon_ring *ring = &rdev->ring[RADEON_RING_TYPE_GFX_INDEX]; struct radeon_fence *fence; + uint64_t cur_src_offset, cur_dst_offset; uint32_t cur_pages; uint32_t stride_bytes = RADEON_GPU_PAGE_SIZE; uint32_t pitch; @@ -934,6 +935,10 @@ struct radeon_fence *r100_copy_blit(struct radeon_device *rdev, cur_pages = 8191; } num_gpu_pages -= cur_pages; + cur_src_offset = src_offset + + (uint64_t)num_gpu_pages * RADEON_GPU_PAGE_SIZE; + cur_dst_offset = dst_offset + + (uint64_t)num_gpu_pages * RADEON_GPU_PAGE_SIZE; /* pages are in Y direction - height page width in X direction - width */ @@ -950,13 +955,13 @@ struct radeon_fence *r100_copy_blit(struct radeon_device *rdev, RADEON_DP_SRC_SOURCE_MEMORY | RADEON_GMC_CLR_CMP_CNTL_DIS | RADEON_GMC_WR_MSK_DIS); - radeon_ring_write(ring, (pitch << 22) | (src_offset >> 10)); - radeon_ring_write(ring, (pitch << 22) | (dst_offset >> 10)); + radeon_ring_write(ring, (pitch << 22) | (cur_src_offset >> 10)); + radeon_ring_write(ring, (pitch << 22) | (cur_dst_offset >> 10)); radeon_ring_write(ring, (0x1fff) | (0x1fff << 16)); radeon_ring_write(ring, 0); radeon_ring_write(ring, (0x1fff) | (0x1fff << 16)); - radeon_ring_write(ring, num_gpu_pages); - radeon_ring_write(ring, num_gpu_pages); + radeon_ring_write(ring, 0); + radeon_ring_write(ring, 0); radeon_ring_write(ring, cur_pages | (stride_pixels << 16)); } radeon_ring_write(ring, PACKET0(RADEON_DSTCACHE_CTLSTAT, 0)); diff --git a/drivers/gpu/drm/renesas/rz-du/rzg2l_mipi_dsi.c b/drivers/gpu/drm/renesas/rz-du/rzg2l_mipi_dsi.c index cbd898a182b5..04a24ff9dde1 100644 --- a/drivers/gpu/drm/renesas/rz-du/rzg2l_mipi_dsi.c +++ b/drivers/gpu/drm/renesas/rz-du/rzg2l_mipi_dsi.c @@ -272,7 +272,7 @@ static int rzg2l_mipi_dsi_dphy_init(struct rzg2l_mipi_dsi *dsi, if (ret < 0) return ret; - udelay(1); + fsleep(1000); return 0; } @@ -604,29 +604,33 @@ static void rzg2l_mipi_dsi_atomic_pre_enable(struct drm_bridge *bridge, const struct drm_display_mode *mode; struct drm_connector *connector; struct drm_crtc *crtc; - int ret; connector = drm_atomic_get_new_connector_for_encoder(state, bridge->encoder); crtc = drm_atomic_get_new_connector_state(state, connector)->crtc; mode = &drm_atomic_get_new_crtc_state(state, crtc)->adjusted_mode; - ret = rzg2l_mipi_dsi_startup(dsi, mode); - if (ret < 0) - return; - - rzg2l_mipi_dsi_set_display_timing(dsi, mode); + rzg2l_mipi_dsi_startup(dsi, mode); } static void rzg2l_mipi_dsi_atomic_enable(struct drm_bridge *bridge, struct drm_atomic_state *state) { struct rzg2l_mipi_dsi *dsi = bridge_to_rzg2l_mipi_dsi(bridge); + const struct drm_display_mode *mode; + struct drm_connector *connector; + struct drm_crtc *crtc; int ret; ret = rzg2l_mipi_dsi_start_hs_clock(dsi); if (ret < 0) goto err_stop; + connector = drm_atomic_get_new_connector_for_encoder(state, bridge->encoder); + crtc = drm_atomic_get_new_connector_state(state, connector)->crtc; + mode = &drm_atomic_get_new_crtc_state(state, crtc)->adjusted_mode; + + rzg2l_mipi_dsi_set_display_timing(dsi, mode); + ret = rzg2l_mipi_dsi_start_video(dsi); if (ret < 0) goto err_stop_clock; diff --git a/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c b/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c index d30f0983a53a..b905e28afa07 100644 --- a/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c +++ b/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c @@ -469,6 +469,8 @@ static int rockchip_dp_probe(struct platform_device *pdev) return -ENOMEM; res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) + return -EINVAL; i = 0; while (dp_data[i].reg) { diff --git a/drivers/gpu/drm/rockchip/cdn-dp-reg.c b/drivers/gpu/drm/rockchip/cdn-dp-reg.c index 924fb1d3ece2..c0866ca10543 100644 --- a/drivers/gpu/drm/rockchip/cdn-dp-reg.c +++ b/drivers/gpu/drm/rockchip/cdn-dp-reg.c @@ -683,6 +683,8 @@ int cdn_dp_config_video(struct cdn_dp_device *dp) val = div_u64(8 * (symbol + 1), bit_per_pix) - val; val += 2; ret = cdn_dp_reg_write(dp, DP_VC_TABLE(15), val); + if (ret) + goto err_config_video; switch (video->color_depth) { case 6: diff --git a/drivers/gpu/drm/sysfb/drm_sysfb_helper.h b/drivers/gpu/drm/sysfb/drm_sysfb_helper.h index 265a1b8a9349..6caf87ab1f15 100644 --- a/drivers/gpu/drm/sysfb/drm_sysfb_helper.h +++ b/drivers/gpu/drm/sysfb/drm_sysfb_helper.h @@ -39,7 +39,7 @@ struct resource *drm_sysfb_get_memory_si(struct drm_device *dev, int drm_sysfb_get_stride_si(struct drm_device *dev, const struct screen_info *si, const struct drm_format_info *format, unsigned int width, unsigned int height, u64 size); -u64 drm_sysfb_get_visible_size_si(struct drm_device *dev, const struct screen_info *si, +s64 drm_sysfb_get_visible_size_si(struct drm_device *dev, const struct screen_info *si, unsigned int height, unsigned int stride, u64 size); const struct drm_format_info *drm_sysfb_get_format_si(struct drm_device *dev, const struct drm_sysfb_format *formats, diff --git a/drivers/gpu/drm/sysfb/drm_sysfb_screen_info.c b/drivers/gpu/drm/sysfb/drm_sysfb_screen_info.c index 885864168c54..a250bea7b19c 100644 --- a/drivers/gpu/drm/sysfb/drm_sysfb_screen_info.c +++ b/drivers/gpu/drm/sysfb/drm_sysfb_screen_info.c @@ -2,6 +2,7 @@ #include #include +#include #include #include @@ -56,18 +57,24 @@ int drm_sysfb_get_stride_si(struct drm_device *dev, const struct screen_info *si unsigned int width, unsigned int height, u64 size) { u64 lfb_linelength = si->lfb_linelength; + s64 stride; if (!lfb_linelength) lfb_linelength = drm_format_info_min_pitch(format, 0, width); - return drm_sysfb_get_validated_int0(dev, "stride", lfb_linelength, div64_u64(size, height)); + stride = drm_sysfb_get_validated_size0(dev, "stride", lfb_linelength, + div64_u64(size, height)); + if (stride < INT_MIN || stride > INT_MAX) + return -EINVAL; + + return (int)stride; /* stride or negative errno code */ } EXPORT_SYMBOL(drm_sysfb_get_stride_si); -u64 drm_sysfb_get_visible_size_si(struct drm_device *dev, const struct screen_info *si, +s64 drm_sysfb_get_visible_size_si(struct drm_device *dev, const struct screen_info *si, unsigned int height, unsigned int stride, u64 size) { - u64 vsize = PAGE_ALIGN(height * stride); + u64 vsize = mul_u32_u32(height, stride); return drm_sysfb_get_validated_size0(dev, "visible size", vsize, size); } diff --git a/drivers/gpu/drm/sysfb/efidrm.c b/drivers/gpu/drm/sysfb/efidrm.c index 97a3711e7933..844d1e8b0ab3 100644 --- a/drivers/gpu/drm/sysfb/efidrm.c +++ b/drivers/gpu/drm/sysfb/efidrm.c @@ -143,7 +143,8 @@ static struct efidrm_device *efidrm_device_create(struct drm_driver *drv, const struct screen_info *si; const struct drm_format_info *format; int width, height, stride; - u64 vsize, mem_flags; + s64 vsize; + u64 mem_flags; struct resource resbuf; struct resource *res; struct efidrm_device *efi; @@ -195,8 +196,8 @@ static struct efidrm_device *efidrm_device_create(struct drm_driver *drv, if (stride < 0) return ERR_PTR(stride); vsize = drm_sysfb_get_visible_size_si(dev, si, height, stride, resource_size(res)); - if (!vsize) - return ERR_PTR(-EINVAL); + if (vsize < 0) + return ERR_PTR(vsize); drm_dbg(dev, "framebuffer format=%p4cc, size=%dx%d, stride=%d bytes\n", &format->format, width, height, stride); diff --git a/drivers/gpu/drm/sysfb/vesadrm.c b/drivers/gpu/drm/sysfb/vesadrm.c index 16a4b52d45c6..37123976e1fb 100644 --- a/drivers/gpu/drm/sysfb/vesadrm.c +++ b/drivers/gpu/drm/sysfb/vesadrm.c @@ -392,7 +392,7 @@ static struct vesadrm_device *vesadrm_device_create(struct drm_driver *drv, const struct screen_info *si; const struct drm_format_info *format; int width, height, stride; - u64 vsize; + s64 vsize; struct resource resbuf; struct resource *res; struct vesadrm_device *vesa; @@ -445,8 +445,8 @@ static struct vesadrm_device *vesadrm_device_create(struct drm_driver *drv, if (stride < 0) return ERR_PTR(stride); vsize = drm_sysfb_get_visible_size_si(dev, si, height, stride, resource_size(res)); - if (!vsize) - return ERR_PTR(-EINVAL); + if (vsize < 0) + return ERR_PTR(vsize); drm_dbg(dev, "framebuffer format=%p4cc, size=%dx%d, stride=%d bytes\n", &format->format, width, height, stride); diff --git a/drivers/gpu/drm/tests/Makefile b/drivers/gpu/drm/tests/Makefile index c0e952293ad0..524310241e16 100644 --- a/drivers/gpu/drm/tests/Makefile +++ b/drivers/gpu/drm/tests/Makefile @@ -7,7 +7,6 @@ obj-$(CONFIG_DRM_KUNIT_TEST) += \ drm_atomic_test.o \ drm_atomic_state_test.o \ drm_bridge_test.o \ - drm_buddy_test.o \ drm_cmdline_parser_test.o \ drm_connector_test.o \ drm_damage_helper_test.o \ diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c deleted file mode 100644 index 7a0e523651f0..000000000000 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ /dev/null @@ -1,788 +0,0 @@ -// SPDX-License-Identifier: MIT -/* - * Copyright © 2019 Intel Corporation - * Copyright © 2022 Maíra Canal - */ - -#include - -#include -#include -#include - -#include - -#include "../lib/drm_random.h" - -static unsigned int random_seed; - -static inline u64 get_size(int order, u64 chunk_size) -{ - return (1 << order) * chunk_size; -} - -static void drm_test_buddy_alloc_range_bias(struct kunit *test) -{ - u32 mm_size, size, ps, bias_size, bias_start, bias_end, bias_rem; - DRM_RND_STATE(prng, random_seed); - unsigned int i, count, *order; - struct drm_buddy_block *block; - unsigned long flags; - struct drm_buddy mm; - LIST_HEAD(allocated); - - bias_size = SZ_1M; - ps = roundup_pow_of_two(prandom_u32_state(&prng) % bias_size); - ps = max(SZ_4K, ps); - mm_size = (SZ_8M-1) & ~(ps-1); /* Multiple roots */ - - kunit_info(test, "mm_size=%u, ps=%u\n", mm_size, ps); - - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps), - "buddy_init failed\n"); - - count = mm_size / bias_size; - order = drm_random_order(count, &prng); - KUNIT_EXPECT_TRUE(test, order); - - /* - * Idea is to split the address space into uniform bias ranges, and then - * in some random order allocate within each bias, using various - * patterns within. This should detect if allocations leak out from a - * given bias, for example. - */ - - for (i = 0; i < count; i++) { - LIST_HEAD(tmp); - u32 size; - - bias_start = order[i] * bias_size; - bias_end = bias_start + bias_size; - bias_rem = bias_size; - - /* internal round_up too big */ - KUNIT_ASSERT_TRUE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, bias_size + ps, bias_size, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, bias_size, bias_size); - - /* size too big */ - KUNIT_ASSERT_TRUE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, bias_size + ps, ps, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, bias_size + ps, ps); - - /* bias range too small for size */ - KUNIT_ASSERT_TRUE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start + ps, - bias_end, bias_size, ps, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n", - bias_start + ps, bias_end, bias_size, ps); - - /* bias misaligned */ - KUNIT_ASSERT_TRUE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start + ps, - bias_end - ps, - bias_size >> 1, bias_size >> 1, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc h didn't fail with bias(%x-%x), size=%u, ps=%u\n", - bias_start + ps, bias_end - ps, bias_size >> 1, bias_size >> 1); - - /* single big page */ - KUNIT_ASSERT_FALSE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, bias_size, bias_size, - &tmp, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc i failed with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, bias_size, bias_size); - drm_buddy_free_list(&mm, &tmp, 0); - - /* single page with internal round_up */ - KUNIT_ASSERT_FALSE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, ps, bias_size, - &tmp, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, ps, bias_size); - drm_buddy_free_list(&mm, &tmp, 0); - - /* random size within */ - size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); - if (size) - KUNIT_ASSERT_FALSE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, size, ps, - &tmp, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, size, ps); - - bias_rem -= size; - /* too big for current avail */ - KUNIT_ASSERT_TRUE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, bias_rem + ps, ps, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, bias_rem + ps, ps); - - if (bias_rem) { - /* random fill of the remainder */ - size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); - size = max(size, ps); - - KUNIT_ASSERT_FALSE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, size, ps, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, size, ps); - /* - * Intentionally allow some space to be left - * unallocated, and ideally not always on the bias - * boundaries. - */ - drm_buddy_free_list(&mm, &tmp, 0); - } else { - list_splice_tail(&tmp, &allocated); - } - } - - kfree(order); - drm_buddy_free_list(&mm, &allocated, 0); - drm_buddy_fini(&mm); - - /* - * Something more free-form. Idea is to pick a random starting bias - * range within the address space and then start filling it up. Also - * randomly grow the bias range in both directions as we go along. This - * should give us bias start/end which is not always uniform like above, - * and in some cases will require the allocator to jump over already - * allocated nodes in the middle of the address space. - */ - - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps), - "buddy_init failed\n"); - - bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps); - bias_end = round_up(bias_start + prandom_u32_state(&prng) % (mm_size - bias_start), ps); - bias_end = max(bias_end, bias_start + ps); - bias_rem = bias_end - bias_start; - - do { - u32 size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); - - KUNIT_ASSERT_FALSE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, size, ps, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, size, ps); - bias_rem -= size; - - /* - * Try to randomly grow the bias range in both directions, or - * only one, or perhaps don't grow at all. - */ - do { - u32 old_bias_start = bias_start; - u32 old_bias_end = bias_end; - - if (bias_start) - bias_start -= round_up(prandom_u32_state(&prng) % bias_start, ps); - if (bias_end != mm_size) - bias_end += round_up(prandom_u32_state(&prng) % (mm_size - bias_end), ps); - - bias_rem += old_bias_start - bias_start; - bias_rem += bias_end - old_bias_end; - } while (!bias_rem && (bias_start || bias_end != mm_size)); - } while (bias_rem); - - KUNIT_ASSERT_EQ(test, bias_start, 0); - KUNIT_ASSERT_EQ(test, bias_end, mm_size); - KUNIT_ASSERT_TRUE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, bias_end, - ps, ps, - &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc passed with bias(%x-%x), size=%u\n", - bias_start, bias_end, ps); - - drm_buddy_free_list(&mm, &allocated, 0); - drm_buddy_fini(&mm); - - /* - * Allocate cleared blocks in the bias range when the DRM buddy's clear avail is - * zero. This will validate the bias range allocation in scenarios like system boot - * when no cleared blocks are available and exercise the fallback path too. The resulting - * blocks should always be dirty. - */ - - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps), - "buddy_init failed\n"); - - bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps); - bias_end = round_up(bias_start + prandom_u32_state(&prng) % (mm_size - bias_start), ps); - bias_end = max(bias_end, bias_start + ps); - bias_rem = bias_end - bias_start; - - flags = DRM_BUDDY_CLEAR_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION; - size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); - - KUNIT_ASSERT_FALSE_MSG(test, - drm_buddy_alloc_blocks(&mm, bias_start, - bias_end, size, ps, - &allocated, - flags), - "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", - bias_start, bias_end, size, ps); - - list_for_each_entry(block, &allocated, link) - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false); - - drm_buddy_free_list(&mm, &allocated, 0); - drm_buddy_fini(&mm); -} - -static void drm_test_buddy_alloc_clear(struct kunit *test) -{ - unsigned long n_pages, total, i = 0; - const unsigned long ps = SZ_4K; - struct drm_buddy_block *block; - const int max_order = 12; - LIST_HEAD(allocated); - struct drm_buddy mm; - unsigned int order; - u32 mm_size, size; - LIST_HEAD(dirty); - LIST_HEAD(clean); - - mm_size = SZ_4K << max_order; - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); - - KUNIT_EXPECT_EQ(test, mm.max_order, max_order); - - /* - * Idea is to allocate and free some random portion of the address space, - * returning those pages as non-dirty and randomly alternate between - * requesting dirty and non-dirty pages (not going over the limit - * we freed as non-dirty), putting that into two separate lists. - * Loop over both lists at the end checking that the dirty list - * is indeed all dirty pages and vice versa. Free it all again, - * keeping the dirty/clear status. - */ - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 5 * ps, ps, &allocated, - DRM_BUDDY_TOPDOWN_ALLOCATION), - "buddy_alloc hit an error size=%lu\n", 5 * ps); - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED); - - n_pages = 10; - do { - unsigned long flags; - struct list_head *list; - int slot = i % 2; - - if (slot == 0) { - list = &dirty; - flags = 0; - } else { - list = &clean; - flags = DRM_BUDDY_CLEAR_ALLOCATION; - } - - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - ps, ps, list, - flags), - "buddy_alloc hit an error size=%lu\n", ps); - } while (++i < n_pages); - - list_for_each_entry(block, &clean, link) - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), true); - - list_for_each_entry(block, &dirty, link) - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false); - - drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED); - - /* - * Trying to go over the clear limit for some allocation. - * The allocation should never fail with reasonable page-size. - */ - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 10 * ps, ps, &clean, - DRM_BUDDY_CLEAR_ALLOCATION), - "buddy_alloc hit an error size=%lu\n", 10 * ps); - - drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED); - drm_buddy_free_list(&mm, &dirty, 0); - drm_buddy_fini(&mm); - - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); - - /* - * Create a new mm. Intentionally fragment the address space by creating - * two alternating lists. Free both lists, one as dirty the other as clean. - * Try to allocate double the previous size with matching min_page_size. The - * allocation should never fail as it calls the force_merge. Also check that - * the page is always dirty after force_merge. Free the page as dirty, then - * repeat the whole thing, increment the order until we hit the max_order. - */ - - i = 0; - n_pages = mm_size / ps; - do { - struct list_head *list; - int slot = i % 2; - - if (slot == 0) - list = &dirty; - else - list = &clean; - - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - ps, ps, list, 0), - "buddy_alloc hit an error size=%lu\n", ps); - } while (++i < n_pages); - - drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED); - drm_buddy_free_list(&mm, &dirty, 0); - - order = 1; - do { - size = SZ_4K << order; - - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - size, size, &allocated, - DRM_BUDDY_CLEAR_ALLOCATION), - "buddy_alloc hit an error size=%u\n", size); - total = 0; - list_for_each_entry(block, &allocated, link) { - if (size != mm_size) - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false); - total += drm_buddy_block_size(&mm, block); - } - KUNIT_EXPECT_EQ(test, total, size); - - drm_buddy_free_list(&mm, &allocated, 0); - } while (++order <= max_order); - - drm_buddy_fini(&mm); - - /* - * Create a new mm with a non power-of-two size. Allocate a random size from each - * root, free as cleared and then call fini. This will ensure the multi-root - * force merge during fini. - */ - mm_size = (SZ_4K << max_order) + (SZ_4K << (max_order - 2)); - - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); - KUNIT_EXPECT_EQ(test, mm.max_order, max_order); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order, - 4 * ps, ps, &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc hit an error size=%lu\n", 4 * ps); - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order, - 2 * ps, ps, &allocated, - DRM_BUDDY_CLEAR_ALLOCATION), - "buddy_alloc hit an error size=%lu\n", 2 * ps); - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, SZ_4K << max_order, mm_size, - ps, ps, &allocated, - DRM_BUDDY_RANGE_ALLOCATION), - "buddy_alloc hit an error size=%lu\n", ps); - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED); - drm_buddy_fini(&mm); -} - -static void drm_test_buddy_alloc_contiguous(struct kunit *test) -{ - const unsigned long ps = SZ_4K, mm_size = 16 * 3 * SZ_4K; - unsigned long i, n_pages, total; - struct drm_buddy_block *block; - struct drm_buddy mm; - LIST_HEAD(left); - LIST_HEAD(middle); - LIST_HEAD(right); - LIST_HEAD(allocated); - - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps)); - - /* - * Idea is to fragment the address space by alternating block - * allocations between three different lists; one for left, middle and - * right. We can then free a list to simulate fragmentation. In - * particular we want to exercise the DRM_BUDDY_CONTIGUOUS_ALLOCATION, - * including the try_harder path. - */ - - i = 0; - n_pages = mm_size / ps; - do { - struct list_head *list; - int slot = i % 3; - - if (slot == 0) - list = &left; - else if (slot == 1) - list = &middle; - else - list = &right; - KUNIT_ASSERT_FALSE_MSG(test, - drm_buddy_alloc_blocks(&mm, 0, mm_size, - ps, ps, list, 0), - "buddy_alloc hit an error size=%lu\n", - ps); - } while (++i < n_pages); - - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 3 * ps, ps, &allocated, - DRM_BUDDY_CONTIGUOUS_ALLOCATION), - "buddy_alloc didn't error size=%lu\n", 3 * ps); - - drm_buddy_free_list(&mm, &middle, 0); - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 3 * ps, ps, &allocated, - DRM_BUDDY_CONTIGUOUS_ALLOCATION), - "buddy_alloc didn't error size=%lu\n", 3 * ps); - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 2 * ps, ps, &allocated, - DRM_BUDDY_CONTIGUOUS_ALLOCATION), - "buddy_alloc didn't error size=%lu\n", 2 * ps); - - drm_buddy_free_list(&mm, &right, 0); - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 3 * ps, ps, &allocated, - DRM_BUDDY_CONTIGUOUS_ALLOCATION), - "buddy_alloc didn't error size=%lu\n", 3 * ps); - /* - * At this point we should have enough contiguous space for 2 blocks, - * however they are never buddies (since we freed middle and right) so - * will require the try_harder logic to find them. - */ - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 2 * ps, ps, &allocated, - DRM_BUDDY_CONTIGUOUS_ALLOCATION), - "buddy_alloc hit an error size=%lu\n", 2 * ps); - - drm_buddy_free_list(&mm, &left, 0); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, - 3 * ps, ps, &allocated, - DRM_BUDDY_CONTIGUOUS_ALLOCATION), - "buddy_alloc hit an error size=%lu\n", 3 * ps); - - total = 0; - list_for_each_entry(block, &allocated, link) - total += drm_buddy_block_size(&mm, block); - - KUNIT_ASSERT_EQ(test, total, ps * 2 + ps * 3); - - drm_buddy_free_list(&mm, &allocated, 0); - drm_buddy_fini(&mm); -} - -static void drm_test_buddy_alloc_pathological(struct kunit *test) -{ - u64 mm_size, size, start = 0; - struct drm_buddy_block *block; - const int max_order = 3; - unsigned long flags = 0; - int order, top; - struct drm_buddy mm; - LIST_HEAD(blocks); - LIST_HEAD(holes); - LIST_HEAD(tmp); - - /* - * Create a pot-sized mm, then allocate one of each possible - * order within. This should leave the mm with exactly one - * page left. Free the largest block, then whittle down again. - * Eventually we will have a fully 50% fragmented mm. - */ - - mm_size = SZ_4K << max_order; - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K), - "buddy_init failed\n"); - - KUNIT_EXPECT_EQ(test, mm.max_order, max_order); - - for (top = max_order; top; top--) { - /* Make room by freeing the largest allocated block */ - block = list_first_entry_or_null(&blocks, typeof(*block), link); - if (block) { - list_del(&block->link); - drm_buddy_free_block(&mm, block); - } - - for (order = top; order--;) { - size = get_size(order, mm.chunk_size); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, - mm_size, size, size, - &tmp, flags), - "buddy_alloc hit -ENOMEM with order=%d, top=%d\n", - order, top); - - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link); - KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); - - list_move_tail(&block->link, &blocks); - } - - /* There should be one final page for this sub-allocation */ - size = get_size(0, mm.chunk_size); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc hit -ENOMEM for hole\n"); - - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link); - KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); - - list_move_tail(&block->link, &holes); - - size = get_size(top, mm.chunk_size); - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc unexpectedly succeeded at top-order %d/%d, it should be full!", - top, max_order); - } - - drm_buddy_free_list(&mm, &holes, 0); - - /* Nothing larger than blocks of chunk_size now available */ - for (order = 1; order <= max_order; order++) { - size = get_size(order, mm.chunk_size); - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc unexpectedly succeeded at order %d, it should be full!", - order); - } - - list_splice_tail(&holes, &blocks); - drm_buddy_free_list(&mm, &blocks, 0); - drm_buddy_fini(&mm); -} - -static void drm_test_buddy_alloc_pessimistic(struct kunit *test) -{ - u64 mm_size, size, start = 0; - struct drm_buddy_block *block, *bn; - const unsigned int max_order = 16; - unsigned long flags = 0; - struct drm_buddy mm; - unsigned int order; - LIST_HEAD(blocks); - LIST_HEAD(tmp); - - /* - * Create a pot-sized mm, then allocate one of each possible - * order within. This should leave the mm with exactly one - * page left. - */ - - mm_size = SZ_4K << max_order; - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K), - "buddy_init failed\n"); - - KUNIT_EXPECT_EQ(test, mm.max_order, max_order); - - for (order = 0; order < max_order; order++) { - size = get_size(order, mm.chunk_size); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc hit -ENOMEM with order=%d\n", - order); - - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link); - KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); - - list_move_tail(&block->link, &blocks); - } - - /* And now the last remaining block available */ - size = get_size(0, mm.chunk_size); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc hit -ENOMEM on final alloc\n"); - - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link); - KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); - - list_move_tail(&block->link, &blocks); - - /* Should be completely full! */ - for (order = max_order; order--;) { - size = get_size(order, mm.chunk_size); - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc unexpectedly succeeded, it should be full!"); - } - - block = list_last_entry(&blocks, typeof(*block), link); - list_del(&block->link); - drm_buddy_free_block(&mm, block); - - /* As we free in increasing size, we make available larger blocks */ - order = 1; - list_for_each_entry_safe(block, bn, &blocks, link) { - list_del(&block->link); - drm_buddy_free_block(&mm, block); - - size = get_size(order, mm.chunk_size); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc hit -ENOMEM with order=%d\n", - order); - - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link); - KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); - - list_del(&block->link); - drm_buddy_free_block(&mm, block); - order++; - } - - /* To confirm, now the whole mm should be available */ - size = get_size(max_order, mm.chunk_size); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc (realloc) hit -ENOMEM with order=%d\n", - max_order); - - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link); - KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); - - list_del(&block->link); - drm_buddy_free_block(&mm, block); - drm_buddy_free_list(&mm, &blocks, 0); - drm_buddy_fini(&mm); -} - -static void drm_test_buddy_alloc_optimistic(struct kunit *test) -{ - u64 mm_size, size, start = 0; - struct drm_buddy_block *block; - unsigned long flags = 0; - const int max_order = 16; - struct drm_buddy mm; - LIST_HEAD(blocks); - LIST_HEAD(tmp); - int order; - - /* - * Create a mm with one block of each order available, and - * try to allocate them all. - */ - - mm_size = SZ_4K * ((1 << (max_order + 1)) - 1); - - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K), - "buddy_init failed\n"); - - KUNIT_EXPECT_EQ(test, mm.max_order, max_order); - - for (order = 0; order <= max_order; order++) { - size = get_size(order, mm.chunk_size); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc hit -ENOMEM with order=%d\n", - order); - - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link); - KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); - - list_move_tail(&block->link, &blocks); - } - - /* Should be completely full! */ - size = get_size(0, mm.chunk_size); - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, - size, size, &tmp, flags), - "buddy_alloc unexpectedly succeeded, it should be full!"); - - drm_buddy_free_list(&mm, &blocks, 0); - drm_buddy_fini(&mm); -} - -static void drm_test_buddy_alloc_limit(struct kunit *test) -{ - u64 size = U64_MAX, start = 0; - struct drm_buddy_block *block; - unsigned long flags = 0; - LIST_HEAD(allocated); - struct drm_buddy mm; - - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, size, SZ_4K)); - - KUNIT_EXPECT_EQ_MSG(test, mm.max_order, DRM_BUDDY_MAX_ORDER, - "mm.max_order(%d) != %d\n", mm.max_order, - DRM_BUDDY_MAX_ORDER); - - size = mm.chunk_size << mm.max_order; - KUNIT_EXPECT_FALSE(test, drm_buddy_alloc_blocks(&mm, start, size, size, - mm.chunk_size, &allocated, flags)); - - block = list_first_entry_or_null(&allocated, struct drm_buddy_block, link); - KUNIT_EXPECT_TRUE(test, block); - - KUNIT_EXPECT_EQ_MSG(test, drm_buddy_block_order(block), mm.max_order, - "block order(%d) != %d\n", - drm_buddy_block_order(block), mm.max_order); - - KUNIT_EXPECT_EQ_MSG(test, drm_buddy_block_size(&mm, block), - BIT_ULL(mm.max_order) * mm.chunk_size, - "block size(%llu) != %llu\n", - drm_buddy_block_size(&mm, block), - BIT_ULL(mm.max_order) * mm.chunk_size); - - drm_buddy_free_list(&mm, &allocated, 0); - drm_buddy_fini(&mm); -} - -static int drm_buddy_suite_init(struct kunit_suite *suite) -{ - while (!random_seed) - random_seed = get_random_u32(); - - kunit_info(suite, "Testing DRM buddy manager, with random_seed=0x%x\n", - random_seed); - - return 0; -} - -static struct kunit_case drm_buddy_tests[] = { - KUNIT_CASE(drm_test_buddy_alloc_limit), - KUNIT_CASE(drm_test_buddy_alloc_optimistic), - KUNIT_CASE(drm_test_buddy_alloc_pessimistic), - KUNIT_CASE(drm_test_buddy_alloc_pathological), - KUNIT_CASE(drm_test_buddy_alloc_contiguous), - KUNIT_CASE(drm_test_buddy_alloc_clear), - KUNIT_CASE(drm_test_buddy_alloc_range_bias), - {} -}; - -static struct kunit_suite drm_buddy_test_suite = { - .name = "drm_buddy", - .suite_init = drm_buddy_suite_init, - .test_cases = drm_buddy_tests, -}; - -kunit_test_suite(drm_buddy_test_suite); - -MODULE_AUTHOR("Intel Corporation"); -MODULE_DESCRIPTION("Kunit test for drm_buddy functions"); -MODULE_LICENSE("GPL"); diff --git a/drivers/gpu/drm/tests/drm_exec_test.c b/drivers/gpu/drm/tests/drm_exec_test.c index 3a20c788c51f..2fc47f3b463b 100644 --- a/drivers/gpu/drm/tests/drm_exec_test.c +++ b/drivers/gpu/drm/tests/drm_exec_test.c @@ -16,8 +16,6 @@ #include #include -#include "../lib/drm_random.h" - struct drm_exec_priv { struct device *dev; struct drm_device *drm; diff --git a/drivers/gpu/drm/tests/drm_gem_shmem_test.c b/drivers/gpu/drm/tests/drm_gem_shmem_test.c index 4b459f21acfd..b1fa287892fb 100644 --- a/drivers/gpu/drm/tests/drm_gem_shmem_test.c +++ b/drivers/gpu/drm/tests/drm_gem_shmem_test.c @@ -95,13 +95,9 @@ static void drm_gem_shmem_test_obj_create_private(struct kunit *test) sg_init_one(sgt->sgl, buf, TEST_SIZE); /* - * Set the DMA mask to 64-bits and map the sgtables - * otherwise drm_gem_shmem_free will cause a warning - * on debug kernels. + * Map the sgtables otherwise drm_gem_shmem_free will cause a warning on + * debug kernels. */ - ret = dma_set_mask(drm_dev->dev, DMA_BIT_MASK(64)); - KUNIT_ASSERT_EQ(test, ret, 0); - ret = dma_map_sgtable(drm_dev->dev, sgt, DMA_BIDIRECTIONAL, 0); KUNIT_ASSERT_EQ(test, ret, 0); @@ -352,11 +348,19 @@ static int drm_gem_shmem_test_init(struct kunit *test) { struct device *dev; struct drm_device *drm_dev; + int ret; /* Allocate a parent device */ dev = drm_kunit_helper_alloc_device(test); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, dev); + /* + * Set the DMA mask to 64-bits to avoid intermittent failures calling + * drm_gem_shmem_get_pages_sgt(). + */ + ret = dma_set_mask(dev, DMA_BIT_MASK(64)); + KUNIT_ASSERT_EQ(test, ret, 0); + /* * The DRM core will automatically initialize the GEM core and create * a DRM Memory Manager object which provides an address space pool diff --git a/drivers/gpu/drm/tests/drm_mm_test.c b/drivers/gpu/drm/tests/drm_mm_test.c index 6174d0929020..99744322ed1a 100644 --- a/drivers/gpu/drm/tests/drm_mm_test.c +++ b/drivers/gpu/drm/tests/drm_mm_test.c @@ -15,8 +15,6 @@ #include -#include "../lib/drm_random.h" - enum { BEST, BOTTOMUP, diff --git a/drivers/gpu/drm/tidss/tidss_encoder.c b/drivers/gpu/drm/tidss/tidss_encoder.c index 81a04f767770..db467bbcdb77 100644 --- a/drivers/gpu/drm/tidss/tidss_encoder.c +++ b/drivers/gpu/drm/tidss/tidss_encoder.c @@ -106,6 +106,8 @@ int tidss_encoder_create(struct tidss_device *tidss, enc = &t_enc->encoder; enc->possible_crtcs = possible_crtcs; + devm_drm_bridge_add(tidss->dev, &t_enc->bridge); + /* Attaching first bridge to the encoder */ ret = drm_bridge_attach(enc, &t_enc->bridge, NULL, DRM_BRIDGE_ATTACH_NO_CONNECTOR); diff --git a/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c b/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c index 1bcc67977f48..3cbdc8101b7c 100644 --- a/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c +++ b/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c @@ -251,7 +251,7 @@ static void ttm_bo_validate_basic(struct kunit *test) NULL, &dummy_ttm_bo_destroy); KUNIT_EXPECT_EQ(test, err, 0); - snd_place = ttm_place_kunit_init(test, snd_mem, DRM_BUDDY_TOPDOWN_ALLOCATION); + snd_place = ttm_place_kunit_init(test, snd_mem, GPU_BUDDY_TOPDOWN_ALLOCATION); snd_placement = ttm_placement_kunit_init(test, snd_place, 1); err = ttm_bo_validate(bo, snd_placement, &ctx_val); @@ -263,7 +263,7 @@ static void ttm_bo_validate_basic(struct kunit *test) KUNIT_EXPECT_TRUE(test, ttm_tt_is_populated(bo->ttm)); KUNIT_EXPECT_EQ(test, bo->resource->mem_type, snd_mem); KUNIT_EXPECT_EQ(test, bo->resource->placement, - DRM_BUDDY_TOPDOWN_ALLOCATION); + GPU_BUDDY_TOPDOWN_ALLOCATION); ttm_bo_put(bo); ttm_mock_manager_fini(priv->ttm_dev, snd_mem); diff --git a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c index d7eb6471f2ed..a9c233f982f6 100644 --- a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c +++ b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c @@ -30,7 +30,7 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man, { struct ttm_mock_manager *manager = to_mock_mgr(man); struct ttm_mock_resource *mock_res; - struct drm_buddy *mm = &manager->mm; + struct gpu_buddy *mm = &manager->mm; u64 lpfn, fpfn, alloc_size; int err; @@ -46,14 +46,14 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man, INIT_LIST_HEAD(&mock_res->blocks); if (place->flags & TTM_PL_FLAG_TOPDOWN) - mock_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + mock_res->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION; if (place->flags & TTM_PL_FLAG_CONTIGUOUS) - mock_res->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION; + mock_res->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION; alloc_size = (uint64_t)mock_res->base.size; mutex_lock(&manager->lock); - err = drm_buddy_alloc_blocks(mm, fpfn, lpfn, alloc_size, + err = gpu_buddy_alloc_blocks(mm, fpfn, lpfn, alloc_size, manager->default_page_size, &mock_res->blocks, mock_res->flags); @@ -66,7 +66,7 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man, return 0; error_free_blocks: - drm_buddy_free_list(mm, &mock_res->blocks, 0); + gpu_buddy_free_list(mm, &mock_res->blocks, 0); ttm_resource_fini(man, &mock_res->base); mutex_unlock(&manager->lock); @@ -78,10 +78,10 @@ static void ttm_mock_manager_free(struct ttm_resource_manager *man, { struct ttm_mock_manager *manager = to_mock_mgr(man); struct ttm_mock_resource *mock_res = to_mock_mgr_resource(res); - struct drm_buddy *mm = &manager->mm; + struct gpu_buddy *mm = &manager->mm; mutex_lock(&manager->lock); - drm_buddy_free_list(mm, &mock_res->blocks, 0); + gpu_buddy_free_list(mm, &mock_res->blocks, 0); mutex_unlock(&manager->lock); ttm_resource_fini(man, res); @@ -105,7 +105,7 @@ int ttm_mock_manager_init(struct ttm_device *bdev, u32 mem_type, u32 size) mutex_init(&manager->lock); - err = drm_buddy_init(&manager->mm, size, PAGE_SIZE); + err = gpu_buddy_init(&manager->mm, size, PAGE_SIZE); if (err) { kfree(manager); @@ -141,7 +141,7 @@ void ttm_mock_manager_fini(struct ttm_device *bdev, u32 mem_type) ttm_resource_manager_set_used(man, false); mutex_lock(&mock_man->lock); - drm_buddy_fini(&mock_man->mm); + gpu_buddy_fini(&mock_man->mm); mutex_unlock(&mock_man->lock); ttm_set_driver_manager(bdev, mem_type, NULL); diff --git a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h index e4c95f86a467..08710756fd8e 100644 --- a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h +++ b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h @@ -5,11 +5,11 @@ #ifndef TTM_MOCK_MANAGER_H #define TTM_MOCK_MANAGER_H -#include +#include struct ttm_mock_manager { struct ttm_resource_manager man; - struct drm_buddy mm; + struct gpu_buddy mm; u64 default_page_size; /* protects allocations of mock buffer objects */ struct mutex lock; diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c index baf27c70a419..01083720e4e7 100644 --- a/drivers/gpu/drm/ttm/ttm_pool.c +++ b/drivers/gpu/drm/ttm/ttm_pool.c @@ -981,9 +981,31 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *tt, return -EBUSY; #ifdef CONFIG_X86 - /* Anything returned to the system needs to be cached. */ - if (tt->caching != ttm_cached) - set_pages_array_wb(tt->pages, tt->num_pages); + /* Anything returned to the system needs to be cached. Walk allocations + * skipping NULL pages and issue set_pages_array_wb() per contiguous run. + */ + if (tt->caching != ttm_cached) { + pgoff_t run_start = 0, run_count = 0; + + for (i = 0; i < tt->num_pages; i += num_pages) { + page = tt->pages[i]; + if (unlikely(!page || ttm_backup_page_ptr_is_handle(page))) { + if (run_count) { + set_pages_array_wb(&tt->pages[run_start], + run_count); + run_count = 0; + } + num_pages = 1; + continue; + } + num_pages = 1UL << ttm_pool_page_order(pool, page); + if (!run_count) + run_start = i; + run_count += num_pages; + } + if (run_count) + set_pages_array_wb(&tt->pages[run_start], run_count); + } #endif if (tt->dma_address || flags->purge) { @@ -991,7 +1013,7 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *tt, unsigned int order; page = tt->pages[i]; - if (unlikely(!page)) { + if (unlikely(!page || ttm_backup_page_ptr_is_handle(page))) { num_pages = 1; continue; } @@ -1034,6 +1056,10 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *tt, if (unlikely(!page)) continue; + /* Already-handled entry from a previous attempt. */ + if (unlikely(ttm_backup_page_ptr_is_handle(page))) + continue; + ttm_pool_split_for_swap(pool, page); shandle = ttm_backup_backup_page(backup, page, flags->writeback, i, diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c index ab29dde22361..b499f6635cd2 100644 --- a/drivers/gpu/drm/v3d/v3d_gem.c +++ b/drivers/gpu/drm/v3d/v3d_gem.c @@ -45,6 +45,18 @@ v3d_init_hw_state(struct v3d_dev *v3d) static void v3d_idle_axi(struct v3d_dev *v3d, int core) { + if (v3d->ver >= V3D_GEN_71) { + V3D_WRITE(V3D_GMP_CFG(v3d->ver), V3D_GMP_CFG_STOP_REQ); + + if (wait_for((V3D_READ(V3D_GMP_STATUS(v3d->ver)) & + (V3D_GMP_STATUS_RD_COUNT_MASK | + V3D_GMP_STATUS_WR_COUNT_MASK | + V3D_GMP_STATUS_CFG_BUSY)) == 0, 100)) { + drm_err(&v3d->drm, "Failed to wait for safe GMP shutdown\n"); + } + return; + } + V3D_CORE_WRITE(core, V3D_GMP_CFG(v3d->ver), V3D_GMP_CFG_STOP_REQ); if (wait_for((V3D_CORE_READ(core, V3D_GMP_STATUS(v3d->ver)) & diff --git a/drivers/gpu/drm/vc4/vc4_bo.c b/drivers/gpu/drm/vc4/vc4_bo.c index a1efda9c39f9..cd7996592db3 100644 --- a/drivers/gpu/drm/vc4/vc4_bo.c +++ b/drivers/gpu/drm/vc4/vc4_bo.c @@ -733,9 +733,13 @@ static int vc4_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct { struct vc4_bo *bo = to_vc4_bo(obj); - if (bo->validated_shader && (vma->vm_flags & VM_WRITE)) { - DRM_DEBUG("mmapping of shader BOs for writing not allowed.\n"); - return -EINVAL; + if (bo->validated_shader) { + if (vma->vm_flags & VM_WRITE) { + DRM_DEBUG("mmapping of shader BOs for writing not allowed.\n"); + return -EINVAL; + } + + vm_flags_clear(vma, VM_MAYWRITE); } mutex_lock(&bo->madv_lock); @@ -1046,7 +1050,7 @@ static void vc4_bo_cache_destroy(struct drm_device *dev, void *unused) struct vc4_dev *vc4 = to_vc4_dev(dev); int i; - timer_delete(&vc4->bo_cache.time_timer); + timer_shutdown_sync(&vc4->bo_cache.time_timer); cancel_work_sync(&vc4->bo_cache.time_work); vc4_bo_cache_purge(dev); diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c index 015b5debd745..7174086119c5 100644 --- a/drivers/gpu/drm/virtio/virtgpu_gem.c +++ b/drivers/gpu/drm/virtio/virtgpu_gem.c @@ -139,13 +139,15 @@ void virtio_gpu_gem_object_close(struct drm_gem_object *obj, if (!vgdev->has_virgl_3d) return; - objs = virtio_gpu_array_alloc(1); - if (!objs) - return; - virtio_gpu_array_add_obj(objs, obj); + if (vfpriv->context_created) { + objs = virtio_gpu_array_alloc(1); + if (!objs) + return; + virtio_gpu_array_add_obj(objs, obj); - virtio_gpu_cmd_context_detach_resource(vgdev, vfpriv->ctx_id, - objs); + virtio_gpu_cmd_context_detach_resource(vgdev, vfpriv->ctx_id, + objs); + } virtio_gpu_notify(vgdev); } diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c index 1c15cbf326b7..4a25347734fd 100644 --- a/drivers/gpu/drm/virtio/virtgpu_kms.c +++ b/drivers/gpu/drm/virtio/virtgpu_kms.c @@ -48,7 +48,10 @@ static void virtio_gpu_config_changed_work_func(struct work_struct *work) virtio_gpu_cmd_get_edids(vgdev); virtio_gpu_cmd_get_display_info(vgdev); virtio_gpu_notify(vgdev); - drm_helper_hpd_irq_event(vgdev->ddev); + wait_event_timeout(vgdev->resp_wq, + !vgdev->display_info_pending, + 5 * HZ); + drm_kms_helper_hotplug_event(vgdev->ddev); } events_clear |= VIRTIO_GPU_EVENT_DISPLAY; } diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c index 8181b22b9b46..83d46a8edf79 100644 --- a/drivers/gpu/drm/virtio/virtgpu_vq.c +++ b/drivers/gpu/drm/virtio/virtgpu_vq.c @@ -839,9 +839,6 @@ static void virtio_gpu_cmd_get_display_info_cb(struct virtio_gpu_device *vgdev, vgdev->display_info_pending = false; spin_unlock(&vgdev->display_info_lock); wake_up(&vgdev->resp_wq); - - if (!drm_helper_hpd_irq_event(vgdev->ddev)) - drm_kms_helper_hotplug_event(vgdev->ddev); } static void virtio_gpu_cmd_get_capset_info_cb(struct virtio_gpu_device *vgdev, @@ -896,7 +893,8 @@ static int virtio_get_edid_block(void *data, u8 *buf, struct virtio_gpu_resp_edid *resp = data; size_t start = block * EDID_LENGTH; - if (start + len > le32_to_cpu(resp->size)) + if (start + len > le32_to_cpu(resp->size) || + start + len > sizeof(resp->edid)) return -EINVAL; memcpy(buf, resp->edid + start, len); return 0; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c index 7e281c3c6bc5..8e01e13eebf4 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c @@ -77,7 +77,7 @@ static int vmw_gb_surface_unbind(struct vmw_resource *res, static int vmw_gb_surface_destroy(struct vmw_resource *res); static int vmw_gb_surface_define_internal(struct drm_device *dev, - struct drm_vmw_gb_surface_create_ext_req *req, + const struct drm_vmw_gb_surface_create_ext_req *req, struct drm_vmw_gb_surface_create_rep *rep, struct drm_file *file_priv); static int @@ -1503,7 +1503,7 @@ int vmw_gb_surface_reference_ext_ioctl(struct drm_device *dev, void *data, */ static int vmw_gb_surface_define_internal(struct drm_device *dev, - struct drm_vmw_gb_surface_create_ext_req *req, + const struct drm_vmw_gb_surface_create_ext_req *req, struct drm_vmw_gb_surface_create_rep *rep, struct drm_file *file_priv) { @@ -1521,9 +1521,21 @@ vmw_gb_surface_define_internal(struct drm_device *dev, req->base.svga3d_flags); /* array_size must be null for non-GL3 host. */ - if (req->base.array_size > 0 && !has_sm4_context(dev_priv)) { - VMW_DEBUG_USER("SM4 surface not supported.\n"); - return -EINVAL; + if (req->base.array_size > 0) { + if (has_sm5_context(dev_priv)) { + if (req->base.array_size > SVGA3D_SM5_MAX_SURFACE_ARRAYSIZE) { + VMW_DEBUG_USER("Invalid Surface Array Size.\n"); + return -EINVAL; + } + } else if (has_sm4_context(dev_priv)) { + if (req->base.array_size > SVGA3D_SM4_MAX_SURFACE_ARRAYSIZE) { + VMW_DEBUG_USER("Invalid Surface Array Size.\n"); + return -EINVAL; + } + } else { + VMW_DEBUG_USER("SM4+ surface not supported.\n"); + return -EINVAL; + } } if (!has_sm4_1_context(dev_priv)) { diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 0dabe8535139..5c531b8d9211 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -930,6 +930,21 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, xe_pm_runtime_get_noresume(xe); } + /* + * Attach CCS BBs before submitting the copy job below so a VF + * migration racing the copy sees valid, up to date attach state. + */ + if (IS_VF_CCS_READY(xe) && + ((move_lacks_source && new_mem->mem_type == XE_PL_TT) || + (old_mem_type == XE_PL_SYSTEM && new_mem->mem_type == XE_PL_TT)) && + handle_system_ccs) { + ret = xe_sriov_vf_ccs_attach_bo(bo, new_mem); + if (ret) { + xe_pm_runtime_put(xe); + goto out; + } + } + if (move_lacks_source) { u32 flags = 0; @@ -967,22 +982,19 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm_bo_move_null(ttm_bo, new_mem); } - dma_fence_put(fence); - xe_pm_runtime_put(xe); - /* - * CCS meta data is migrated from TT -> SMEM. So, let us detach the - * BBs from BO as it is no longer needed. + * Detach must wait for the copy above to complete: a VF migration + * racing an in-flight copy must still see valid CCS BBs, so don't + * tear them down until the copy fence has signaled. */ if (IS_VF_CCS_READY(xe) && old_mem_type == XE_PL_TT && - new_mem->mem_type == XE_PL_SYSTEM) + new_mem->mem_type == XE_PL_SYSTEM) { + dma_fence_wait(fence, false); xe_sriov_vf_ccs_detach_bo(bo); + } - if (IS_VF_CCS_READY(xe) && - ((move_lacks_source && new_mem->mem_type == XE_PL_TT) || - (old_mem_type == XE_PL_SYSTEM && new_mem->mem_type == XE_PL_TT)) && - handle_system_ccs) - ret = xe_sriov_vf_ccs_attach_bo(bo); + dma_fence_put(fence); + xe_pm_runtime_put(xe); out: if ((!ttm_bo->resource || ttm_bo->resource->mem_type == XE_PL_SYSTEM) && @@ -1174,7 +1186,7 @@ int xe_bo_notifier_prepare_pinned(struct xe_bo *bo) backup = xe_bo_init_locked(xe, NULL, NULL, bo->ttm.base.resv, NULL, xe_bo_size(bo), DRM_XE_GEM_CPU_CACHING_WB, ttm_bo_type_kernel, XE_BO_FLAG_SYSTEM | XE_BO_FLAG_NEEDS_CPU_ACCESS | - XE_BO_FLAG_PINNED, &exec); + XE_BO_FLAG_PINNED, NULL, &exec); if (IS_ERR(backup)) { drm_exec_retry_on_contention(&exec); ret = PTR_ERR(backup); @@ -1315,7 +1327,7 @@ int xe_bo_evict_pinned(struct xe_bo *bo) xe_bo_size(bo), DRM_XE_GEM_CPU_CACHING_WB, ttm_bo_type_kernel, XE_BO_FLAG_SYSTEM | XE_BO_FLAG_NEEDS_CPU_ACCESS | - XE_BO_FLAG_PINNED, &exec); + XE_BO_FLAG_PINNED, NULL, &exec); if (IS_ERR(backup)) { drm_exec_retry_on_contention(&exec); ret = PTR_ERR(backup); @@ -1663,6 +1675,8 @@ static void xe_ttm_bo_destroy(struct ttm_buffer_object *ttm_bo) if (bo->ttm.base.import_attach) drm_prime_gem_destroy(&bo->ttm.base, NULL); + if (bo->dma_buf) + dma_buf_put(bo->dma_buf); drm_gem_object_release(&bo->ttm.base); xe_assert(xe, list_empty(&ttm_bo->base.gpuva.list)); @@ -2080,6 +2094,8 @@ void xe_bo_free(struct xe_bo *bo) * @cpu_caching: The cpu caching used for system memory backing store. * @type: The TTM buffer object type. * @flags: XE_BO_FLAG_ flags. + * @dma_buf: The dma-buf to reference for the BO lifetime (imported BOs), + * or NULL. * @exec: The drm_exec transaction to use for exhaustive eviction. * * Initialize or create an xe buffer object. On failure, any allocated buffer @@ -2091,7 +2107,8 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, struct xe_tile *tile, struct dma_resv *resv, struct ttm_lru_bulk_move *bulk, size_t size, u16 cpu_caching, enum ttm_bo_type type, - u32 flags, struct drm_exec *exec) + u32 flags, struct dma_buf *dma_buf, + struct drm_exec *exec) { struct ttm_operation_ctx ctx = { .interruptible = true, @@ -2177,6 +2194,17 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, placement = (type == ttm_bo_type_sg || bo->flags & XE_BO_FLAG_DEFER_BACKING) ? &sys_placement : &bo->placement; + + /* + * For imported BOs, keep the exporter dma-buf alive for the BO + * lifetime. Taken before ttm_bo_init_reserved() to also cover a + * creation failure there. Released in xe_ttm_bo_destroy(). + */ + if (dma_buf) { + get_dma_buf(dma_buf); + bo->dma_buf = dma_buf; + } + err = ttm_bo_init_reserved(&xe->ttm, &bo->ttm, type, placement, alignment, &ctx, NULL, resv, xe_ttm_bo_destroy); @@ -2291,7 +2319,7 @@ __xe_bo_create_locked(struct xe_device *xe, vm && !xe_vm_in_fault_mode(vm) && flags & XE_BO_FLAG_USER ? &vm->lru_bulk_move : NULL, size, - cpu_caching, type, flags, exec); + cpu_caching, type, flags, NULL, exec); if (IS_ERR(bo)) return bo; diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h index a77af42b5f9e..9c1ba37983d9 100644 --- a/drivers/gpu/drm/xe/xe_bo.h +++ b/drivers/gpu/drm/xe/xe_bo.h @@ -93,7 +93,8 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, struct xe_tile *tile, struct dma_resv *resv, struct ttm_lru_bulk_move *bulk, size_t size, u16 cpu_caching, enum ttm_bo_type type, - u32 flags, struct drm_exec *exec); + u32 flags, struct dma_buf *dma_buf, + struct drm_exec *exec); struct xe_bo *xe_bo_create_locked(struct xe_device *xe, struct xe_tile *tile, struct xe_vm *vm, size_t size, enum ttm_bo_type type, u32 flags, diff --git a/drivers/gpu/drm/xe/xe_bo_types.h b/drivers/gpu/drm/xe/xe_bo_types.h index d4fe3c8dca5b..88e3d8b33c53 100644 --- a/drivers/gpu/drm/xe/xe_bo_types.h +++ b/drivers/gpu/drm/xe/xe_bo_types.h @@ -35,6 +35,8 @@ struct xe_bo { struct xe_bo *backup_obj; /** @parent_obj: Ref to parent bo if this a backup_obj */ struct xe_bo *parent_obj; + /** @dma_buf: Imported dma-buf ref to keep its resv alive. */ + struct dma_buf *dma_buf; /** @flags: flags for this buffer object */ u32 flags; /** @vm: VM this BO is attached to, for extobj this will be NULL */ diff --git a/drivers/gpu/drm/xe/xe_dma_buf.c b/drivers/gpu/drm/xe/xe_dma_buf.c index eeea2a6e15b5..07010f097e58 100644 --- a/drivers/gpu/drm/xe/xe_dma_buf.c +++ b/drivers/gpu/drm/xe/xe_dma_buf.c @@ -251,7 +251,7 @@ xe_dma_buf_create_obj(struct drm_device *dev, struct dma_buf *dma_buf) bo = xe_bo_init_locked(xe, NULL, NULL, resv, NULL, dma_buf->size, 0, /* Will require 1way or 2way for vm_bind */ - ttm_bo_type_sg, XE_BO_FLAG_SYSTEM, &exec); + ttm_bo_type_sg, XE_BO_FLAG_SYSTEM, dma_buf, &exec); drm_exec_retry_on_contention(&exec); if (IS_ERR(bo)) { ret = PTR_ERR(bo); diff --git a/drivers/gpu/drm/xe/xe_guc_buf.c b/drivers/gpu/drm/xe/xe_guc_buf.c index 502ca3a4ee60..3de6377ae2d5 100644 --- a/drivers/gpu/drm/xe/xe_guc_buf.c +++ b/drivers/gpu/drm/xe/xe_guc_buf.c @@ -13,6 +13,8 @@ #include "xe_guc_buf.h" #include "xe_sa.h" +#define XE_GUC_BUF_CACHE_DEFAULT_SIZE SZ_8K + static struct xe_guc *cache_to_guc(struct xe_guc_buf_cache *cache) { return container_of(cache, struct xe_guc, buf); @@ -23,21 +25,12 @@ static struct xe_gt *cache_to_gt(struct xe_guc_buf_cache *cache) return guc_to_gt(cache_to_guc(cache)); } -/** - * xe_guc_buf_cache_init() - Initialize the GuC Buffer Cache. - * @cache: the &xe_guc_buf_cache to initialize - * - * The Buffer Cache allows to obtain a reusable buffer that can be used to pass - * indirect H2G data to GuC without a need to create a ad-hoc allocation. - * - * Return: 0 on success or a negative error code on failure. - */ -int xe_guc_buf_cache_init(struct xe_guc_buf_cache *cache) +static int guc_buf_cache_init(struct xe_guc_buf_cache *cache, u32 size) { struct xe_gt *gt = cache_to_gt(cache); struct xe_sa_manager *sam; - sam = __xe_sa_bo_manager_init(gt_to_tile(gt), SZ_8K, 0, sizeof(u32)); + sam = __xe_sa_bo_manager_init(gt_to_tile(gt), size, 0, sizeof(u32), 0); if (IS_ERR(sam)) return PTR_ERR(sam); cache->sam = sam; @@ -48,6 +41,35 @@ int xe_guc_buf_cache_init(struct xe_guc_buf_cache *cache) return 0; } +/** + * xe_guc_buf_cache_init() - Initialize the GuC Buffer Cache. + * @cache: the &xe_guc_buf_cache to initialize + * + * The Buffer Cache allows to obtain a reusable buffer that can be used to pass + * data to GuC or read data from GuC without a need to create a ad-hoc allocation. + * + * Return: 0 on success or a negative error code on failure. + */ +int xe_guc_buf_cache_init(struct xe_guc_buf_cache *cache) +{ + return guc_buf_cache_init(cache, XE_GUC_BUF_CACHE_DEFAULT_SIZE); +} + +/** + * xe_guc_buf_cache_init_with_size() - Initialize the GuC Buffer Cache. + * @cache: the &xe_guc_buf_cache to initialize + * @size: size in bytes + * + * Like xe_guc_buf_cache_init(), except it allows the caller to make the cache + * buffer larger, allowing to accommodate larger objects. + * + * Return: 0 on success or a negative error code on failure. + */ +int xe_guc_buf_cache_init_with_size(struct xe_guc_buf_cache *cache, u32 size) +{ + return guc_buf_cache_init(cache, max(XE_GUC_BUF_CACHE_DEFAULT_SIZE, size)); +} + /** * xe_guc_buf_cache_dwords() - Number of dwords the GuC Buffer Cache supports. * @cache: the &xe_guc_buf_cache to query diff --git a/drivers/gpu/drm/xe/xe_guc_buf.h b/drivers/gpu/drm/xe/xe_guc_buf.h index 0d67604d96bd..8204e5896250 100644 --- a/drivers/gpu/drm/xe/xe_guc_buf.h +++ b/drivers/gpu/drm/xe/xe_guc_buf.h @@ -12,6 +12,7 @@ #include "xe_guc_buf_types.h" int xe_guc_buf_cache_init(struct xe_guc_buf_cache *cache); +int xe_guc_buf_cache_init_with_size(struct xe_guc_buf_cache *cache, u32 size); u32 xe_guc_buf_cache_dwords(struct xe_guc_buf_cache *cache); struct xe_guc_buf xe_guc_buf_reserve(struct xe_guc_buf_cache *cache, u32 dwords); struct xe_guc_buf xe_guc_buf_from_data(struct xe_guc_buf_cache *cache, diff --git a/drivers/gpu/drm/xe/xe_i2c.c b/drivers/gpu/drm/xe/xe_i2c.c index 48dfcb41fa08..2788dc6510d6 100644 --- a/drivers/gpu/drm/xe/xe_i2c.c +++ b/drivers/gpu/drm/xe/xe_i2c.c @@ -94,18 +94,21 @@ static int xe_i2c_register_adapter(struct xe_i2c *i2c) struct platform_device *pdev; struct fwnode_handle *fwnode; int ret; + u32 id; fwnode = fwnode_create_software_node(xe_i2c_adapter_properties, NULL); if (IS_ERR(fwnode)) return PTR_ERR(fwnode); + id = (pci_domain_nr(pci->bus) << 16) | pci_dev_id(pci); + /* * Not using platform_device_register_full() here because we don't have * a handle to the platform_device before it returns. xe_i2c_notifier() * uses that handle, but it may be called before * platform_device_register_full() is done. */ - pdev = platform_device_alloc(adapter_name, pci_dev_id(pci)); + pdev = platform_device_alloc(adapter_name, id); if (!pdev) { ret = -ENOMEM; goto err_fwnode_remove; diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c index b6905f35d6c8..e0a14360d312 100644 --- a/drivers/gpu/drm/xe/xe_migrate.c +++ b/drivers/gpu/drm/xe/xe_migrate.c @@ -33,6 +33,7 @@ #include "xe_res_cursor.h" #include "xe_sa.h" #include "xe_sched_job.h" +#include "xe_sriov_vf_ccs.h" #include "xe_sync.h" #include "xe_trace_bo.h" #include "xe_validation.h" @@ -1000,6 +1001,8 @@ static int emit_flush_invalidate(struct xe_exec_queue *q, u32 *dw, int i, * @tile: Tile whose migration context to be used. * @q : Execution to be used along with migration context. * @src_bo: The buffer object @src is currently bound to. + * @new_mem: The (not yet committed) destination resource @src_bo is being + * moved into; src_bo->ttm.resource is still the old resource. * @read_write : Creates BB commands for CCS read/write. * * Creates batch buffer instructions to copy CCS metadata from CCS pool to @@ -1011,23 +1014,28 @@ static int emit_flush_invalidate(struct xe_exec_queue *q, u32 *dw, int i, */ int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q, struct xe_bo *src_bo, + struct ttm_resource *new_mem, enum xe_sriov_vf_ccs_rw_ctxs read_write) { bool src_is_pltt = read_write == XE_SRIOV_VF_CCS_READ_CTX; bool dst_is_pltt = read_write == XE_SRIOV_VF_CCS_WRITE_CTX; - struct ttm_resource *src = src_bo->ttm.resource; + struct ttm_resource *src = new_mem; struct xe_migrate *m = tile->migrate; struct xe_gt *gt = tile->primary_gt; u32 batch_size, batch_size_allocated; struct xe_device *xe = gt_to_xe(gt); struct xe_res_cursor src_it, ccs_it; + struct xe_sriov_vf_ccs_ctx *ctx; + struct xe_sa_manager *bb_pool; u64 size = xe_bo_size(src_bo); struct xe_bb *bb = NULL; u64 src_L0, src_L0_ofs; u32 src_L0_pt; int err; + ctx = &xe->sriov.vf.ccs.contexts[read_write]; + xe_res_first_sg(xe_bo_sg(src_bo), 0, size, &src_it); xe_res_first_sg(xe_bo_sg(src_bo), xe_bo_ccs_pages_start(src_bo), @@ -1060,11 +1068,15 @@ int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q, size -= src_L0; } + bb_pool = ctx->mem.ccs_bb_pool; + guard(mutex) (xe_sa_bo_swap_guard(bb_pool)); + xe_sa_bo_swap_shadow(bb_pool); + bb = xe_bb_ccs_new(gt, batch_size, read_write); if (IS_ERR(bb)) { drm_err(&xe->drm, "BB allocation failed.\n"); err = PTR_ERR(bb); - goto err_ret; + return err; } batch_size_allocated = batch_size; @@ -1113,10 +1125,52 @@ int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q, xe_assert(xe, (batch_size_allocated == bb->len)); src_bo->bb_ccs[read_write] = bb; + xe_sriov_vf_ccs_rw_update_bb_addr(ctx); + xe_sa_bo_sync_shadow(bb->bo); return 0; +} -err_ret: - return err; +/** + * xe_migrate_ccs_rw_copy_clear() - Clear the CCS read/write batch buffer + * content. + * @src_bo: The buffer object @src is currently bound to. + * @read_write : Creates BB commands for CCS read/write. + * + * Directly clearing the BB lacks atomicity and can lead to undefined + * behavior if the vCPU is halted mid-operation during the clearing + * process. To avoid this issue, we use a shadow buffer object approach. + * + * First swap the SA BO address with the shadow BO, perform the clearing + * operation on the BB, update the shadow BO in the ring buffer, then + * sync the shadow and the actual buffer to maintain consistency. + * + * Returns: None. + */ +void xe_migrate_ccs_rw_copy_clear(struct xe_bo *src_bo, + enum xe_sriov_vf_ccs_rw_ctxs read_write) +{ + struct xe_bb *bb = src_bo->bb_ccs[read_write]; + struct xe_device *xe = xe_bo_device(src_bo); + struct xe_sriov_vf_ccs_ctx *ctx; + struct xe_sa_manager *bb_pool; + u32 *cs; + + xe_assert(xe, IS_SRIOV_VF(xe)); + + ctx = &xe->sriov.vf.ccs.contexts[read_write]; + bb_pool = ctx->mem.ccs_bb_pool; + + guard(mutex) (xe_sa_bo_swap_guard(bb_pool)); + xe_sa_bo_swap_shadow(bb_pool); + + cs = xe_sa_bo_cpu_addr(bb->bo); + memset(cs, MI_NOOP, bb->len * sizeof(u32)); + xe_sriov_vf_ccs_rw_update_bb_addr(ctx); + + xe_sa_bo_sync_shadow(bb->bo); + + xe_bb_free(bb, NULL); + src_bo->bb_ccs[read_write] = NULL; } /** diff --git a/drivers/gpu/drm/xe/xe_migrate.h b/drivers/gpu/drm/xe/xe_migrate.h index bc55f650204b..8bac669685ee 100644 --- a/drivers/gpu/drm/xe/xe_migrate.h +++ b/drivers/gpu/drm/xe/xe_migrate.h @@ -129,8 +129,12 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q, struct xe_bo *src_bo, + struct ttm_resource *new_mem, enum xe_sriov_vf_ccs_rw_ctxs read_write); +void xe_migrate_ccs_rw_copy_clear(struct xe_bo *src_bo, + enum xe_sriov_vf_ccs_rw_ctxs read_write); + struct xe_lrc *xe_migrate_lrc(struct xe_migrate *migrate); struct xe_exec_queue *xe_migrate_exec_queue(struct xe_migrate *migrate); int xe_migrate_access_memory(struct xe_migrate *m, struct xe_bo *bo, diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c index 1f0262adc3ae..de0d104fc8b6 100644 --- a/drivers/gpu/drm/xe/xe_pt.c +++ b/drivers/gpu/drm/xe/xe_pt.c @@ -999,12 +999,22 @@ xe_vm_populate_pgtable(struct xe_migrate_pt_update *pt_update, struct xe_tile *t u64 *ptr = data; u32 i; + /* + * @qword_ofs is the absolute entry offset within the page table, while + * @ptes is indexed relative to @update->ofs (its first entry). The GPU + * path (write_pgtable) splits a single update into MAX_PTE_PER_SDI-sized + * chunks, calling this with an advancing @qword_ofs but a fresh @data + * pointer per chunk, so translate back into a @ptes index rather than + * assuming the chunk starts at ptes[0]. + */ for (i = 0; i < num_qwords; i++) { + u32 idx = qword_ofs - update->ofs + i; + if (map) xe_map_wr(tile_to_xe(tile), map, (qword_ofs + i) * - sizeof(u64), u64, ptes[i].pte); + sizeof(u64), u64, ptes[idx].pte); else - ptr[i] = ptes[i].pte; + ptr[i] = ptes[idx].pte; } } diff --git a/drivers/gpu/drm/xe/xe_res_cursor.h b/drivers/gpu/drm/xe/xe_res_cursor.h index 4e00008b7081..5f4ab08c0686 100644 --- a/drivers/gpu/drm/xe/xe_res_cursor.h +++ b/drivers/gpu/drm/xe/xe_res_cursor.h @@ -58,7 +58,7 @@ struct xe_res_cursor { /** @dma_addr: Current element in a struct drm_pagemap_addr array */ const struct drm_pagemap_addr *dma_addr; /** @mm: Buddy allocator for VRAM cursor */ - struct drm_buddy *mm; + struct gpu_buddy *mm; /** * @dma_start: DMA start address for the current segment. * This may be different to @dma_addr.addr since elements in @@ -69,7 +69,7 @@ struct xe_res_cursor { u64 dma_seg_size; }; -static struct drm_buddy *xe_res_get_buddy(struct ttm_resource *res) +static struct gpu_buddy *xe_res_get_buddy(struct ttm_resource *res) { struct ttm_resource_manager *mgr; @@ -104,30 +104,30 @@ static inline void xe_res_first(struct ttm_resource *res, case XE_PL_STOLEN: case XE_PL_VRAM0: case XE_PL_VRAM1: { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; struct list_head *head, *next; - struct drm_buddy *mm = xe_res_get_buddy(res); + struct gpu_buddy *mm = xe_res_get_buddy(res); head = &to_xe_ttm_vram_mgr_resource(res)->blocks; block = list_first_entry_or_null(head, - struct drm_buddy_block, + struct gpu_buddy_block, link); if (!block) goto fallback; - while (start >= drm_buddy_block_size(mm, block)) { - start -= drm_buddy_block_size(mm, block); + while (start >= gpu_buddy_block_size(mm, block)) { + start -= gpu_buddy_block_size(mm, block); next = block->link.next; if (next != head) - block = list_entry(next, struct drm_buddy_block, + block = list_entry(next, struct gpu_buddy_block, link); } cur->mm = mm; - cur->start = drm_buddy_block_offset(block) + start; - cur->size = min(drm_buddy_block_size(mm, block) - start, + cur->start = gpu_buddy_block_offset(block) + start; + cur->size = min(gpu_buddy_block_size(mm, block) - start, size); cur->remaining = size; cur->node = block; @@ -259,7 +259,7 @@ static inline void xe_res_first_dma(const struct drm_pagemap_addr *dma_addr, */ static inline void xe_res_next(struct xe_res_cursor *cur, u64 size) { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; struct list_head *next; u64 start; @@ -295,18 +295,18 @@ static inline void xe_res_next(struct xe_res_cursor *cur, u64 size) block = cur->node; next = block->link.next; - block = list_entry(next, struct drm_buddy_block, link); + block = list_entry(next, struct gpu_buddy_block, link); - while (start >= drm_buddy_block_size(cur->mm, block)) { - start -= drm_buddy_block_size(cur->mm, block); + while (start >= gpu_buddy_block_size(cur->mm, block)) { + start -= gpu_buddy_block_size(cur->mm, block); next = block->link.next; - block = list_entry(next, struct drm_buddy_block, link); + block = list_entry(next, struct gpu_buddy_block, link); } - cur->start = drm_buddy_block_offset(block) + start; - cur->size = min(drm_buddy_block_size(cur->mm, block) - start, + cur->start = gpu_buddy_block_offset(block) + start; + cur->size = min(gpu_buddy_block_size(cur->mm, block) - start, cur->remaining); cur->node = block; break; diff --git a/drivers/gpu/drm/xe/xe_sa.c b/drivers/gpu/drm/xe/xe_sa.c index fedd017d6dd3..d9e0d30f8135 100644 --- a/drivers/gpu/drm/xe/xe_sa.c +++ b/drivers/gpu/drm/xe/xe_sa.c @@ -29,6 +29,7 @@ static void xe_sa_bo_manager_fini(struct drm_device *drm, void *arg) kvfree(sa_manager->cpu_ptr); sa_manager->bo = NULL; + sa_manager->shadow = NULL; } /** @@ -37,12 +38,14 @@ static void xe_sa_bo_manager_fini(struct drm_device *drm, void *arg) * @size: number of bytes to allocate * @guard: number of bytes to exclude from suballocations * @align: alignment for each suballocated chunk + * @flags: flags for suballocator * * Prepares the suballocation manager for suballocations. * * Return: a pointer to the &xe_sa_manager or an ERR_PTR on failure. */ -struct xe_sa_manager *__xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, u32 guard, u32 align) +struct xe_sa_manager *__xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, + u32 guard, u32 align, u32 flags) { struct xe_device *xe = tile_to_xe(tile); struct xe_sa_manager *sa_manager; @@ -79,6 +82,26 @@ struct xe_sa_manager *__xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, u3 memset(sa_manager->cpu_ptr, 0, bo->ttm.base.size); } + if (flags & XE_SA_BO_MANAGER_FLAG_SHADOW) { + struct xe_bo *shadow; + + ret = drmm_mutex_init(&xe->drm, &sa_manager->swap_guard); + if (ret) + return ERR_PTR(ret); + + shadow = xe_managed_bo_create_pin_map(xe, tile, size, + XE_BO_FLAG_VRAM_IF_DGFX(tile) | + XE_BO_FLAG_GGTT | + XE_BO_FLAG_GGTT_INVALIDATE | + XE_BO_FLAG_PINNED_NORESTORE); + if (IS_ERR(shadow)) { + drm_err(&xe->drm, "Failed to prepare %uKiB BO for SA manager (%pe)\n", + size / SZ_1K, shadow); + return ERR_CAST(shadow); + } + sa_manager->shadow = shadow; + } + drm_suballoc_manager_init(&sa_manager->base, managed_size, align); ret = drmm_add_action_or_reset(&xe->drm, xe_sa_bo_manager_fini, sa_manager); @@ -88,6 +111,48 @@ struct xe_sa_manager *__xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, u3 return sa_manager; } +/** + * xe_sa_bo_swap_shadow() - Swap the SA BO with shadow BO. + * @sa_manager: the XE sub allocator manager + * + * Swaps the sub-allocator primary buffer object with shadow buffer object. + * + * Return: None. + */ +void xe_sa_bo_swap_shadow(struct xe_sa_manager *sa_manager) +{ + struct xe_device *xe = tile_to_xe(sa_manager->bo->tile); + + xe_assert(xe, sa_manager->shadow); + lockdep_assert_held(&sa_manager->swap_guard); + + swap(sa_manager->bo, sa_manager->shadow); + if (!sa_manager->bo->vmap.is_iomem) + sa_manager->cpu_ptr = sa_manager->bo->vmap.vaddr; +} + +/** + * xe_sa_bo_sync_shadow() - Sync the SA Shadow BO with primary BO. + * @sa_bo: the sub-allocator buffer object. + * + * Synchronize sub-allocator shadow buffer object with primary buffer object. + * + * Return: None. + */ +void xe_sa_bo_sync_shadow(struct drm_suballoc *sa_bo) +{ + struct xe_sa_manager *sa_manager = to_xe_sa_manager(sa_bo->manager); + struct xe_device *xe = tile_to_xe(sa_manager->bo->tile); + + xe_assert(xe, sa_manager->shadow); + lockdep_assert_held(&sa_manager->swap_guard); + + xe_map_memcpy_to(xe, &sa_manager->shadow->vmap, + drm_suballoc_soffset(sa_bo), + xe_sa_bo_cpu_addr(sa_bo), + drm_suballoc_size(sa_bo)); +} + /** * __xe_sa_bo_new() - Make a suballocation but use custom gfp flags. * @sa_manager: the &xe_sa_manager diff --git a/drivers/gpu/drm/xe/xe_sa.h b/drivers/gpu/drm/xe/xe_sa.h index 99dbf0eea540..90ffa93a000c 100644 --- a/drivers/gpu/drm/xe/xe_sa.h +++ b/drivers/gpu/drm/xe/xe_sa.h @@ -14,12 +14,14 @@ struct dma_fence; struct xe_tile; -struct xe_sa_manager *__xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, u32 guard, u32 align); +#define XE_SA_BO_MANAGER_FLAG_SHADOW BIT(0) +struct xe_sa_manager *__xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, + u32 guard, u32 align, u32 flags); struct drm_suballoc *__xe_sa_bo_new(struct xe_sa_manager *sa_manager, u32 size, gfp_t gfp); static inline struct xe_sa_manager *xe_sa_bo_manager_init(struct xe_tile *tile, u32 size, u32 align) { - return __xe_sa_bo_manager_init(tile, size, SZ_4K, align); + return __xe_sa_bo_manager_init(tile, size, SZ_4K, align, 0); } /** @@ -68,4 +70,18 @@ static inline void *xe_sa_bo_cpu_addr(struct drm_suballoc *sa) drm_suballoc_soffset(sa); } +void xe_sa_bo_swap_shadow(struct xe_sa_manager *sa_manager); +void xe_sa_bo_sync_shadow(struct drm_suballoc *sa_bo); + +/** + * xe_sa_bo_swap_guard() - Retrieve the SA BO swap guard within sub-allocator. + * @sa_manager: the &xe_sa_manager + * + * Return: Sub alloctor swap guard mutex. + */ +static inline struct mutex *xe_sa_bo_swap_guard(struct xe_sa_manager *sa_manager) +{ + return &sa_manager->swap_guard; +} + #endif diff --git a/drivers/gpu/drm/xe/xe_sa_types.h b/drivers/gpu/drm/xe/xe_sa_types.h index cb7238799dcb..1085c9c37d6b 100644 --- a/drivers/gpu/drm/xe/xe_sa_types.h +++ b/drivers/gpu/drm/xe/xe_sa_types.h @@ -12,6 +12,9 @@ struct xe_bo; struct xe_sa_manager { struct drm_suballoc_manager base; struct xe_bo *bo; + struct xe_bo *shadow; + /** @swap_guard: Timeline guard updating @bo and @shadow */ + struct mutex swap_guard; void *cpu_ptr; bool is_iomem; }; diff --git a/drivers/gpu/drm/xe/xe_sriov_vf_ccs.c b/drivers/gpu/drm/xe/xe_sriov_vf_ccs.c index 739a3eb180b5..e03714ef7ae9 100644 --- a/drivers/gpu/drm/xe/xe_sriov_vf_ccs.c +++ b/drivers/gpu/drm/xe/xe_sriov_vf_ccs.c @@ -148,7 +148,8 @@ static int alloc_bb_pool(struct xe_tile *tile, struct xe_sriov_vf_ccs_ctx *ctx) xe_sriov_info(xe, "Allocating %s CCS BB pool size = %lldMB\n", ctx->ctx_id ? "Restore" : "Save", bb_pool_size / SZ_1M); - sa_manager = xe_sa_bo_manager_init(tile, bb_pool_size, SZ_16); + sa_manager = __xe_sa_bo_manager_init(tile, bb_pool_size, SZ_4K, SZ_16, + XE_SA_BO_MANAGER_FLAG_SHADOW); if (IS_ERR(sa_manager)) { xe_sriov_err(xe, "Suballocator init failed with error: %pe\n", @@ -160,9 +161,12 @@ static int alloc_bb_pool(struct xe_tile *tile, struct xe_sriov_vf_ccs_ctx *ctx) offset = 0; xe_map_memset(xe, &sa_manager->bo->vmap, offset, MI_NOOP, bb_pool_size); + xe_map_memset(xe, &sa_manager->shadow->vmap, offset, MI_NOOP, + bb_pool_size); offset = bb_pool_size - sizeof(u32); xe_map_wr(xe, &sa_manager->bo->vmap, offset, u32, MI_BATCH_BUFFER_END); + xe_map_wr(xe, &sa_manager->shadow->vmap, offset, u32, MI_BATCH_BUFFER_END); ctx->mem.ccs_bb_pool = sa_manager; @@ -311,9 +315,23 @@ int xe_sriov_vf_ccs_init(struct xe_device *xe) return err; } +#define XE_SRIOV_VF_CCS_RW_BB_ADDR_OFFSET (2 * sizeof(u32)) +void xe_sriov_vf_ccs_rw_update_bb_addr(struct xe_sriov_vf_ccs_ctx *ctx) +{ + u64 addr = xe_sa_manager_gpu_addr(ctx->mem.ccs_bb_pool); + struct xe_lrc *lrc = xe_exec_queue_lrc(ctx->mig_q); + struct xe_device *xe = gt_to_xe(ctx->mig_q->gt); + + xe_device_wmb(xe); + xe_map_wr(xe, &lrc->bo->vmap, XE_SRIOV_VF_CCS_RW_BB_ADDR_OFFSET, u32, addr); + xe_device_wmb(xe); +} + /** * xe_sriov_vf_ccs_attach_bo - Insert CCS read write commands in the BO. * @bo: the &buffer object to which batch buffer commands will be added. + * @new_mem: the (not yet committed) destination resource @bo is being moved + * into; bo->ttm.resource is still the old resource at this point. * * This function shall be called only by VF. It inserts the PTEs and copy * command instructions in the BO by calling xe_migrate_ccs_rw_copy() @@ -321,7 +339,7 @@ int xe_sriov_vf_ccs_init(struct xe_device *xe) * * Returns: 0 if successful, negative error code on failure. */ -int xe_sriov_vf_ccs_attach_bo(struct xe_bo *bo) +int xe_sriov_vf_ccs_attach_bo(struct xe_bo *bo, struct ttm_resource *new_mem) { struct xe_device *xe = xe_bo_device(bo); enum xe_sriov_vf_ccs_rw_ctxs ctx_id; @@ -340,7 +358,21 @@ int xe_sriov_vf_ccs_attach_bo(struct xe_bo *bo) xe_assert(xe, !bb); ctx = &xe->sriov.vf.ccs.contexts[ctx_id]; - err = xe_migrate_ccs_rw_copy(tile, ctx->mig_q, bo, ctx_id); + err = xe_migrate_ccs_rw_copy(tile, ctx->mig_q, bo, new_mem, ctx_id); + if (err) + goto err_unwind; + } + return 0; + +err_unwind: + /* + * Clean up any contexts already attached. Can't reuse + * xe_sriov_vf_ccs_detach_bo() here as it requires both contexts + * attached before cleaning up either one. + */ + for_each_ccs_rw_ctx(ctx_id) { + if (bo->bb_ccs[ctx_id]) + xe_migrate_ccs_rw_copy_clear(bo, ctx_id); } return err; } @@ -371,9 +403,7 @@ int xe_sriov_vf_ccs_detach_bo(struct xe_bo *bo) if (!bb) continue; - memset(bb->cs, MI_NOOP, bb->len * sizeof(u32)); - xe_bb_free(bb, NULL); - bo->bb_ccs[ctx_id] = NULL; + xe_migrate_ccs_rw_copy_clear(bo, ctx_id); } return 0; } diff --git a/drivers/gpu/drm/xe/xe_sriov_vf_ccs.h b/drivers/gpu/drm/xe/xe_sriov_vf_ccs.h index 0745c0ff0228..00d2c9dbe00d 100644 --- a/drivers/gpu/drm/xe/xe_sriov_vf_ccs.h +++ b/drivers/gpu/drm/xe/xe_sriov_vf_ccs.h @@ -11,14 +11,16 @@ #include "xe_sriov_vf_ccs_types.h" struct drm_printer; +struct ttm_resource; struct xe_device; struct xe_bo; int xe_sriov_vf_ccs_init(struct xe_device *xe); -int xe_sriov_vf_ccs_attach_bo(struct xe_bo *bo); +int xe_sriov_vf_ccs_attach_bo(struct xe_bo *bo, struct ttm_resource *new_mem); int xe_sriov_vf_ccs_detach_bo(struct xe_bo *bo); int xe_sriov_vf_ccs_register_context(struct xe_device *xe); void xe_sriov_vf_ccs_print(struct xe_device *xe, struct drm_printer *p); +void xe_sriov_vf_ccs_rw_update_bb_addr(struct xe_sriov_vf_ccs_ctx *ctx); static inline bool xe_sriov_vf_ccs_ready(struct xe_device *xe) { diff --git a/drivers/gpu/drm/xe/xe_svm.c b/drivers/gpu/drm/xe/xe_svm.c index 65452c20609e..e1b8030b6bcf 100644 --- a/drivers/gpu/drm/xe/xe_svm.c +++ b/drivers/gpu/drm/xe/xe_svm.c @@ -689,7 +689,7 @@ static u64 block_offset_to_pfn(struct xe_vram_region *vr, u64 offset) return PHYS_PFN(offset + vr->hpa_base); } -static struct drm_buddy *vram_to_buddy(struct xe_vram_region *vram) +static struct gpu_buddy *vram_to_buddy(struct xe_vram_region *vram) { return &vram->ttm.mm; } @@ -700,16 +700,16 @@ static int xe_svm_populate_devmem_pfn(struct drm_pagemap_devmem *devmem_allocati struct xe_bo *bo = to_xe_bo(devmem_allocation); struct ttm_resource *res = bo->ttm.resource; struct list_head *blocks = &to_xe_ttm_vram_mgr_resource(res)->blocks; - struct drm_buddy_block *block; + struct gpu_buddy_block *block; int j = 0; list_for_each_entry(block, blocks, link) { struct xe_vram_region *vr = block->private; - struct drm_buddy *buddy = vram_to_buddy(vr); - u64 block_pfn = block_offset_to_pfn(vr, drm_buddy_block_offset(block)); + struct gpu_buddy *buddy = vram_to_buddy(vr); + u64 block_pfn = block_offset_to_pfn(vr, gpu_buddy_block_offset(block)); int i; - for (i = 0; i < drm_buddy_block_size(buddy, block) >> PAGE_SHIFT; ++i) + for (i = 0; i < gpu_buddy_block_size(buddy, block) >> PAGE_SHIFT; ++i) pfn[j++] = block_pfn + i; } @@ -877,7 +877,7 @@ static int xe_drm_pagemap_populate_mm(struct drm_pagemap *dpagemap, struct dma_fence *pre_migrate_fence = NULL; struct xe_device *xe = vr->xe; struct device *dev = xe->drm.dev; - struct drm_buddy_block *block; + struct gpu_buddy_block *block; struct xe_validation_ctx vctx; struct list_head *blocks; struct drm_exec exec; @@ -1074,10 +1074,8 @@ static int __xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma, xe_svm_range_fault_count_stats_incr(gt, range); - if (ctx.devmem_only && !range->base.pages.flags.migrate_devmem) { - err = -EACCES; - goto out; - } + if (ctx.devmem_only && !range->base.pages.flags.migrate_devmem) + return -EACCES; if (xe_svm_range_is_valid(range, tile, ctx.devmem_only)) { xe_svm_range_valid_fault_count_stats_incr(gt, range); diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c index 9175b4a2214b..f3d72773835c 100644 --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c @@ -6,6 +6,7 @@ #include #include +#include #include #include @@ -17,16 +18,16 @@ #include "xe_ttm_vram_mgr.h" #include "xe_vram_types.h" -static inline struct drm_buddy_block * +static inline struct gpu_buddy_block * xe_ttm_vram_mgr_first_block(struct list_head *list) { - return list_first_entry_or_null(list, struct drm_buddy_block, link); + return list_first_entry_or_null(list, struct gpu_buddy_block, link); } -static inline bool xe_is_vram_mgr_blocks_contiguous(struct drm_buddy *mm, +static inline bool xe_is_vram_mgr_blocks_contiguous(struct gpu_buddy *mm, struct list_head *head) { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; u64 start, size; block = xe_ttm_vram_mgr_first_block(head); @@ -34,12 +35,12 @@ static inline bool xe_is_vram_mgr_blocks_contiguous(struct drm_buddy *mm, return false; while (head != block->link.next) { - start = drm_buddy_block_offset(block); - size = drm_buddy_block_size(mm, block); + start = gpu_buddy_block_offset(block); + size = gpu_buddy_block_size(mm, block); - block = list_entry(block->link.next, struct drm_buddy_block, + block = list_entry(block->link.next, struct gpu_buddy_block, link); - if (start + size != drm_buddy_block_offset(block)) + if (start + size != gpu_buddy_block_offset(block)) return false; } @@ -53,7 +54,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, { struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man); struct xe_ttm_vram_mgr_resource *vres; - struct drm_buddy *mm = &mgr->mm; + struct gpu_buddy *mm = &mgr->mm; u64 size, min_page_size; unsigned long lpfn; int err; @@ -80,10 +81,10 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, INIT_LIST_HEAD(&vres->blocks); if (place->flags & TTM_PL_FLAG_TOPDOWN) - vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION; + vres->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION; if (place->fpfn || lpfn != man->size >> PAGE_SHIFT) - vres->flags |= DRM_BUDDY_RANGE_ALLOCATION; + vres->flags |= GPU_BUDDY_RANGE_ALLOCATION; if (WARN_ON(!vres->base.size)) { err = -EINVAL; @@ -119,27 +120,27 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, lpfn = max_t(unsigned long, place->fpfn + (size >> PAGE_SHIFT), lpfn); } - err = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT, + err = gpu_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT, (u64)lpfn << PAGE_SHIFT, size, min_page_size, &vres->blocks, vres->flags); if (err) goto error_unlock; if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { - if (!drm_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks)) + if (!gpu_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks)) size = vres->base.size; } if (lpfn <= mgr->visible_size >> PAGE_SHIFT) { vres->used_visible_size = size; } else { - struct drm_buddy_block *block; + struct gpu_buddy_block *block; list_for_each_entry(block, &vres->blocks, link) { - u64 start = drm_buddy_block_offset(block); + u64 start = gpu_buddy_block_offset(block); if (start < mgr->visible_size) { - u64 end = start + drm_buddy_block_size(mm, block); + u64 end = start + gpu_buddy_block_size(mm, block); vres->used_visible_size += min(end, mgr->visible_size) - start; @@ -159,11 +160,11 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, * the object. */ if (vres->base.placement & TTM_PL_FLAG_CONTIGUOUS) { - struct drm_buddy_block *block = list_first_entry(&vres->blocks, + struct gpu_buddy_block *block = list_first_entry(&vres->blocks, typeof(*block), link); - vres->base.start = drm_buddy_block_offset(block) >> PAGE_SHIFT; + vres->base.start = gpu_buddy_block_offset(block) >> PAGE_SHIFT; } else { vres->base.start = XE_BO_INVALID_OFFSET; } @@ -185,10 +186,10 @@ static void xe_ttm_vram_mgr_del(struct ttm_resource_manager *man, struct xe_ttm_vram_mgr_resource *vres = to_xe_ttm_vram_mgr_resource(res); struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man); - struct drm_buddy *mm = &mgr->mm; + struct gpu_buddy *mm = &mgr->mm; mutex_lock(&mgr->lock); - drm_buddy_free_list(mm, &vres->blocks, 0); + gpu_buddy_free_list(mm, &vres->blocks, 0); mgr->visible_avail += vres->used_visible_size; mutex_unlock(&mgr->lock); @@ -201,7 +202,7 @@ static void xe_ttm_vram_mgr_debug(struct ttm_resource_manager *man, struct drm_printer *printer) { struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man); - struct drm_buddy *mm = &mgr->mm; + struct gpu_buddy *mm = &mgr->mm; mutex_lock(&mgr->lock); drm_printf(printer, "default_page_size: %lluKiB\n", @@ -224,8 +225,8 @@ static bool xe_ttm_vram_mgr_intersects(struct ttm_resource_manager *man, struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man); struct xe_ttm_vram_mgr_resource *vres = to_xe_ttm_vram_mgr_resource(res); - struct drm_buddy *mm = &mgr->mm; - struct drm_buddy_block *block; + struct gpu_buddy *mm = &mgr->mm; + struct gpu_buddy_block *block; if (!place->fpfn && !place->lpfn) return true; @@ -235,9 +236,9 @@ static bool xe_ttm_vram_mgr_intersects(struct ttm_resource_manager *man, list_for_each_entry(block, &vres->blocks, link) { unsigned long fpfn = - drm_buddy_block_offset(block) >> PAGE_SHIFT; + gpu_buddy_block_offset(block) >> PAGE_SHIFT; unsigned long lpfn = fpfn + - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT); if (place->fpfn < lpfn && place->lpfn > fpfn) return true; @@ -254,8 +255,8 @@ static bool xe_ttm_vram_mgr_compatible(struct ttm_resource_manager *man, struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man); struct xe_ttm_vram_mgr_resource *vres = to_xe_ttm_vram_mgr_resource(res); - struct drm_buddy *mm = &mgr->mm; - struct drm_buddy_block *block; + struct gpu_buddy *mm = &mgr->mm; + struct gpu_buddy_block *block; if (!place->fpfn && !place->lpfn) return true; @@ -265,9 +266,9 @@ static bool xe_ttm_vram_mgr_compatible(struct ttm_resource_manager *man, list_for_each_entry(block, &vres->blocks, link) { unsigned long fpfn = - drm_buddy_block_offset(block) >> PAGE_SHIFT; + gpu_buddy_block_offset(block) >> PAGE_SHIFT; unsigned long lpfn = fpfn + - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT); + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT); if (fpfn < place->fpfn || lpfn > place->lpfn) return false; @@ -297,7 +298,7 @@ static void ttm_vram_mgr_fini(struct drm_device *dev, void *arg) WARN_ON_ONCE(mgr->visible_avail != mgr->visible_size); - drm_buddy_fini(&mgr->mm); + gpu_buddy_fini(&mgr->mm); ttm_resource_manager_cleanup(&mgr->manager); @@ -328,7 +329,7 @@ int __xe_ttm_vram_mgr_init(struct xe_device *xe, struct xe_ttm_vram_mgr *mgr, mgr->visible_avail = io_size; ttm_resource_manager_init(man, &xe->ttm, size); - err = drm_buddy_init(&mgr->mm, man->size, default_page_size); + err = gpu_buddy_init(&mgr->mm, man->size, default_page_size); if (err) return err; @@ -376,7 +377,7 @@ int xe_ttm_vram_mgr_alloc_sgt(struct xe_device *xe, if (!*sgt) return -ENOMEM; - /* Determine the number of DRM_BUDDY blocks to export */ + /* Determine the number of GPU_BUDDY blocks to export */ xe_res_first(res, offset, length, &cursor); while (cursor.remaining) { num_entries++; @@ -393,10 +394,10 @@ int xe_ttm_vram_mgr_alloc_sgt(struct xe_device *xe, sg->length = 0; /* - * Walk down DRM_BUDDY blocks to populate scatterlist nodes - * @note: Use iterator api to get first the DRM_BUDDY block + * Walk down GPU_BUDDY blocks to populate scatterlist nodes + * @note: Use iterator api to get first the GPU_BUDDY block * and the number of bytes from it. Access the following - * DRM_BUDDY block(s) if more buffer needs to exported + * GPU_BUDDY block(s) if more buffer needs to exported */ xe_res_first(res, offset, length, &cursor); for_each_sgtable_sg((*sgt), sg, i) { diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h index 1144f9232ebb..674a846f9986 100644 --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h @@ -6,7 +6,7 @@ #ifndef _XE_TTM_VRAM_MGR_TYPES_H_ #define _XE_TTM_VRAM_MGR_TYPES_H_ -#include +#include #include /** @@ -18,7 +18,7 @@ struct xe_ttm_vram_mgr { /** @manager: Base TTM resource manager */ struct ttm_resource_manager manager; /** @mm: DRM buddy allocator which manages the VRAM */ - struct drm_buddy mm; + struct gpu_buddy mm; /** @visible_size: Proped size of the CPU visible portion */ u64 visible_size; /** @visible_avail: CPU visible portion still unallocated */ diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index 8f7b8f2da06b..ed0c8acd0fe2 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -1626,10 +1626,10 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags, struct xe_file *xef) return ERR_PTR(err); err_svm_fini: - if (flags & XE_VM_FLAG_FAULT_MODE) { - vm->size = 0; /* close the vm */ - xe_svm_fini(vm); - } + vm->size = 0; /* close the vm */ + if (flags & XE_VM_FLAG_FAULT_MODE) + xe_svm_close(vm); + xe_svm_fini(vm); err_no_resv: mutex_destroy(&vm->snap_mutex); for_each_tile(tile, xe, id) @@ -3393,6 +3393,8 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe, struct xe_vm *vm, op == DRM_XE_VM_BIND_OP_MAP_USERPTR) || XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE && op == DRM_XE_VM_BIND_OP_MAP_USERPTR) || + XE_IOCTL_DBG(xe, !IS_DGFX(xe) && coh_mode == XE_COH_NONE && + is_cpu_addr_mirror) || XE_IOCTL_DBG(xe, op == DRM_XE_VM_BIND_OP_MAP_USERPTR && !IS_ENABLED(CONFIG_DRM_GPUSVM)) || XE_IOCTL_DBG(xe, obj && diff --git a/drivers/gpu/drm/xe/xe_wopcm.c b/drivers/gpu/drm/xe/xe_wopcm.c index ada0d0aa6b74..2fd2a82be89f 100644 --- a/drivers/gpu/drm/xe/xe_wopcm.c +++ b/drivers/gpu/drm/xe/xe_wopcm.c @@ -49,9 +49,9 @@ */ /* Default WOPCM size is 2MB from Gen11, 1MB on previous platforms */ -/* FIXME: Larger size require for 2 tile PVC, do a proper probe sooner or later */ +/* FIXME: Larger size require for some platforms, do a proper probe sooner or later */ #define DGFX_WOPCM_SIZE SZ_4M -/* FIXME: Larger size require for MTL, do a proper probe sooner or later */ +#define LNL_WOPCM_SIZE SZ_8M #define MTL_WOPCM_SIZE SZ_4M #define WOPCM_SIZE SZ_2M @@ -181,9 +181,14 @@ static int __wopcm_init_regs(struct xe_device *xe, struct xe_gt *gt, u32 xe_wopcm_size(struct xe_device *xe) { - return IS_DGFX(xe) ? DGFX_WOPCM_SIZE : - xe->info.platform == XE_METEORLAKE ? MTL_WOPCM_SIZE : - WOPCM_SIZE; + if (xe->info.platform >= XE_LUNARLAKE) + return LNL_WOPCM_SIZE; + else if (IS_DGFX(xe)) + return DGFX_WOPCM_SIZE; + else if (xe->info.platform == XE_METEORLAKE) + return MTL_WOPCM_SIZE; + else + return WOPCM_SIZE; } /** diff --git a/drivers/gpu/host1x/bus.c b/drivers/gpu/host1x/bus.c index 3801eab63282..3313ead8bfcc 100644 --- a/drivers/gpu/host1x/bus.c +++ b/drivers/gpu/host1x/bus.c @@ -1001,10 +1001,10 @@ void host1x_bo_clear_cached_mappings(struct host1x_bo *bo) if (WARN_ON(!cache)) continue; - mutex_lock(&mapping->cache->lock); + mutex_lock(&cache->lock); WARN_ON(kref_read(&mapping->ref) != 1); __host1x_bo_unpin(&mapping->ref); - mutex_unlock(&mapping->cache->lock); + mutex_unlock(&cache->lock); } } EXPORT_SYMBOL(host1x_bo_clear_cached_mappings); diff --git a/drivers/gpu/tests/Makefile b/drivers/gpu/tests/Makefile new file mode 100644 index 000000000000..4183e6e2de45 --- /dev/null +++ b/drivers/gpu/tests/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 + +gpu_buddy_tests-y = gpu_buddy_test.o gpu_random.o +obj-$(CONFIG_GPU_BUDDY_KUNIT_TEST) += gpu_buddy_tests.o diff --git a/drivers/gpu/tests/gpu_buddy_test.c b/drivers/gpu/tests/gpu_buddy_test.c new file mode 100644 index 000000000000..3f689e1f3581 --- /dev/null +++ b/drivers/gpu/tests/gpu_buddy_test.c @@ -0,0 +1,788 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2019 Intel Corporation + * Copyright © 2022 Maíra Canal + */ + +#include + +#include +#include +#include + +#include + +#include "gpu_random.h" + +static unsigned int random_seed; + +static inline u64 get_size(int order, u64 chunk_size) +{ + return (1 << order) * chunk_size; +} + +static void gpu_test_buddy_alloc_range_bias(struct kunit *test) +{ + u32 mm_size, size, ps, bias_size, bias_start, bias_end, bias_rem; + GPU_RND_STATE(prng, random_seed); + unsigned int i, count, *order; + struct gpu_buddy_block *block; + unsigned long flags; + struct gpu_buddy mm; + LIST_HEAD(allocated); + + bias_size = SZ_1M; + ps = roundup_pow_of_two(prandom_u32_state(&prng) % bias_size); + ps = max(SZ_4K, ps); + mm_size = (SZ_8M-1) & ~(ps-1); /* Multiple roots */ + + kunit_info(test, "mm_size=%u, ps=%u\n", mm_size, ps); + + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps), + "buddy_init failed\n"); + + count = mm_size / bias_size; + order = gpu_random_order(count, &prng); + KUNIT_EXPECT_TRUE(test, order); + + /* + * Idea is to split the address space into uniform bias ranges, and then + * in some random order allocate within each bias, using various + * patterns within. This should detect if allocations leak out from a + * given bias, for example. + */ + + for (i = 0; i < count; i++) { + LIST_HEAD(tmp); + u32 size; + + bias_start = order[i] * bias_size; + bias_end = bias_start + bias_size; + bias_rem = bias_size; + + /* internal round_up too big */ + KUNIT_ASSERT_TRUE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, bias_size + ps, bias_size, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, bias_size, bias_size); + + /* size too big */ + KUNIT_ASSERT_TRUE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, bias_size + ps, ps, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, bias_size + ps, ps); + + /* bias range too small for size */ + KUNIT_ASSERT_TRUE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start + ps, + bias_end, bias_size, ps, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n", + bias_start + ps, bias_end, bias_size, ps); + + /* bias misaligned */ + KUNIT_ASSERT_TRUE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start + ps, + bias_end - ps, + bias_size >> 1, bias_size >> 1, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc h didn't fail with bias(%x-%x), size=%u, ps=%u\n", + bias_start + ps, bias_end - ps, bias_size >> 1, bias_size >> 1); + + /* single big page */ + KUNIT_ASSERT_FALSE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, bias_size, bias_size, + &tmp, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc i failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, bias_size, bias_size); + gpu_buddy_free_list(&mm, &tmp, 0); + + /* single page with internal round_up */ + KUNIT_ASSERT_FALSE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, ps, bias_size, + &tmp, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, ps, bias_size); + gpu_buddy_free_list(&mm, &tmp, 0); + + /* random size within */ + size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); + if (size) + KUNIT_ASSERT_FALSE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, size, ps, + &tmp, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, size, ps); + + bias_rem -= size; + /* too big for current avail */ + KUNIT_ASSERT_TRUE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, bias_rem + ps, ps, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, bias_rem + ps, ps); + + if (bias_rem) { + /* random fill of the remainder */ + size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); + size = max(size, ps); + + KUNIT_ASSERT_FALSE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, size, ps, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, size, ps); + /* + * Intentionally allow some space to be left + * unallocated, and ideally not always on the bias + * boundaries. + */ + gpu_buddy_free_list(&mm, &tmp, 0); + } else { + list_splice_tail(&tmp, &allocated); + } + } + + kfree(order); + gpu_buddy_free_list(&mm, &allocated, 0); + gpu_buddy_fini(&mm); + + /* + * Something more free-form. Idea is to pick a random starting bias + * range within the address space and then start filling it up. Also + * randomly grow the bias range in both directions as we go along. This + * should give us bias start/end which is not always uniform like above, + * and in some cases will require the allocator to jump over already + * allocated nodes in the middle of the address space. + */ + + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps), + "buddy_init failed\n"); + + bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps); + bias_end = round_up(bias_start + prandom_u32_state(&prng) % (mm_size - bias_start), ps); + bias_end = max(bias_end, bias_start + ps); + bias_rem = bias_end - bias_start; + + do { + u32 size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); + + KUNIT_ASSERT_FALSE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, size, ps, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, size, ps); + bias_rem -= size; + + /* + * Try to randomly grow the bias range in both directions, or + * only one, or perhaps don't grow at all. + */ + do { + u32 old_bias_start = bias_start; + u32 old_bias_end = bias_end; + + if (bias_start) + bias_start -= round_up(prandom_u32_state(&prng) % bias_start, ps); + if (bias_end != mm_size) + bias_end += round_up(prandom_u32_state(&prng) % (mm_size - bias_end), ps); + + bias_rem += old_bias_start - bias_start; + bias_rem += bias_end - old_bias_end; + } while (!bias_rem && (bias_start || bias_end != mm_size)); + } while (bias_rem); + + KUNIT_ASSERT_EQ(test, bias_start, 0); + KUNIT_ASSERT_EQ(test, bias_end, mm_size); + KUNIT_ASSERT_TRUE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, bias_end, + ps, ps, + &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc passed with bias(%x-%x), size=%u\n", + bias_start, bias_end, ps); + + gpu_buddy_free_list(&mm, &allocated, 0); + gpu_buddy_fini(&mm); + + /* + * Allocate cleared blocks in the bias range when the GPU buddy's clear avail is + * zero. This will validate the bias range allocation in scenarios like system boot + * when no cleared blocks are available and exercise the fallback path too. The resulting + * blocks should always be dirty. + */ + + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps), + "buddy_init failed\n"); + + bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps); + bias_end = round_up(bias_start + prandom_u32_state(&prng) % (mm_size - bias_start), ps); + bias_end = max(bias_end, bias_start + ps); + bias_rem = bias_end - bias_start; + + flags = GPU_BUDDY_CLEAR_ALLOCATION | GPU_BUDDY_RANGE_ALLOCATION; + size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps); + + KUNIT_ASSERT_FALSE_MSG(test, + gpu_buddy_alloc_blocks(&mm, bias_start, + bias_end, size, ps, + &allocated, + flags), + "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n", + bias_start, bias_end, size, ps); + + list_for_each_entry(block, &allocated, link) + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false); + + gpu_buddy_free_list(&mm, &allocated, 0); + gpu_buddy_fini(&mm); +} + +static void gpu_test_buddy_alloc_clear(struct kunit *test) +{ + unsigned long n_pages, total, i = 0; + const unsigned long ps = SZ_4K; + struct gpu_buddy_block *block; + const int max_order = 12; + LIST_HEAD(allocated); + struct gpu_buddy mm; + unsigned int order; + u32 mm_size, size; + LIST_HEAD(dirty); + LIST_HEAD(clean); + + mm_size = SZ_4K << max_order; + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps)); + + KUNIT_EXPECT_EQ(test, mm.max_order, max_order); + + /* + * Idea is to allocate and free some random portion of the address space, + * returning those pages as non-dirty and randomly alternate between + * requesting dirty and non-dirty pages (not going over the limit + * we freed as non-dirty), putting that into two separate lists. + * Loop over both lists at the end checking that the dirty list + * is indeed all dirty pages and vice versa. Free it all again, + * keeping the dirty/clear status. + */ + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 5 * ps, ps, &allocated, + GPU_BUDDY_TOPDOWN_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 5 * ps); + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED); + + n_pages = 10; + do { + unsigned long flags; + struct list_head *list; + int slot = i % 2; + + if (slot == 0) { + list = &dirty; + flags = 0; + } else { + list = &clean; + flags = GPU_BUDDY_CLEAR_ALLOCATION; + } + + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + ps, ps, list, + flags), + "buddy_alloc hit an error size=%lu\n", ps); + } while (++i < n_pages); + + list_for_each_entry(block, &clean, link) + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), true); + + list_for_each_entry(block, &dirty, link) + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false); + + gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED); + + /* + * Trying to go over the clear limit for some allocation. + * The allocation should never fail with reasonable page-size. + */ + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 10 * ps, ps, &clean, + GPU_BUDDY_CLEAR_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 10 * ps); + + gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED); + gpu_buddy_free_list(&mm, &dirty, 0); + gpu_buddy_fini(&mm); + + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps)); + + /* + * Create a new mm. Intentionally fragment the address space by creating + * two alternating lists. Free both lists, one as dirty the other as clean. + * Try to allocate double the previous size with matching min_page_size. The + * allocation should never fail as it calls the force_merge. Also check that + * the page is always dirty after force_merge. Free the page as dirty, then + * repeat the whole thing, increment the order until we hit the max_order. + */ + + i = 0; + n_pages = mm_size / ps; + do { + struct list_head *list; + int slot = i % 2; + + if (slot == 0) + list = &dirty; + else + list = &clean; + + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + ps, ps, list, 0), + "buddy_alloc hit an error size=%lu\n", ps); + } while (++i < n_pages); + + gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED); + gpu_buddy_free_list(&mm, &dirty, 0); + + order = 1; + do { + size = SZ_4K << order; + + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + size, size, &allocated, + GPU_BUDDY_CLEAR_ALLOCATION), + "buddy_alloc hit an error size=%u\n", size); + total = 0; + list_for_each_entry(block, &allocated, link) { + if (size != mm_size) + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false); + total += gpu_buddy_block_size(&mm, block); + } + KUNIT_EXPECT_EQ(test, total, size); + + gpu_buddy_free_list(&mm, &allocated, 0); + } while (++order <= max_order); + + gpu_buddy_fini(&mm); + + /* + * Create a new mm with a non power-of-two size. Allocate a random size from each + * root, free as cleared and then call fini. This will ensure the multi-root + * force merge during fini. + */ + mm_size = (SZ_4K << max_order) + (SZ_4K << (max_order - 2)); + + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps)); + KUNIT_EXPECT_EQ(test, mm.max_order, max_order); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order, + 4 * ps, ps, &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 4 * ps); + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order, + 2 * ps, ps, &allocated, + GPU_BUDDY_CLEAR_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 2 * ps); + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, SZ_4K << max_order, mm_size, + ps, ps, &allocated, + GPU_BUDDY_RANGE_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", ps); + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED); + gpu_buddy_fini(&mm); +} + +static void gpu_test_buddy_alloc_contiguous(struct kunit *test) +{ + const unsigned long ps = SZ_4K, mm_size = 16 * 3 * SZ_4K; + unsigned long i, n_pages, total; + struct gpu_buddy_block *block; + struct gpu_buddy mm; + LIST_HEAD(left); + LIST_HEAD(middle); + LIST_HEAD(right); + LIST_HEAD(allocated); + + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps)); + + /* + * Idea is to fragment the address space by alternating block + * allocations between three different lists; one for left, middle and + * right. We can then free a list to simulate fragmentation. In + * particular we want to exercise the GPU_BUDDY_CONTIGUOUS_ALLOCATION, + * including the try_harder path. + */ + + i = 0; + n_pages = mm_size / ps; + do { + struct list_head *list; + int slot = i % 3; + + if (slot == 0) + list = &left; + else if (slot == 1) + list = &middle; + else + list = &right; + KUNIT_ASSERT_FALSE_MSG(test, + gpu_buddy_alloc_blocks(&mm, 0, mm_size, + ps, ps, list, 0), + "buddy_alloc hit an error size=%lu\n", + ps); + } while (++i < n_pages); + + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + GPU_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%lu\n", 3 * ps); + + gpu_buddy_free_list(&mm, &middle, 0); + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + GPU_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%lu\n", 3 * ps); + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 2 * ps, ps, &allocated, + GPU_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%lu\n", 2 * ps); + + gpu_buddy_free_list(&mm, &right, 0); + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + GPU_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc didn't error size=%lu\n", 3 * ps); + /* + * At this point we should have enough contiguous space for 2 blocks, + * however they are never buddies (since we freed middle and right) so + * will require the try_harder logic to find them. + */ + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 2 * ps, ps, &allocated, + GPU_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 2 * ps); + + gpu_buddy_free_list(&mm, &left, 0); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, + 3 * ps, ps, &allocated, + GPU_BUDDY_CONTIGUOUS_ALLOCATION), + "buddy_alloc hit an error size=%lu\n", 3 * ps); + + total = 0; + list_for_each_entry(block, &allocated, link) + total += gpu_buddy_block_size(&mm, block); + + KUNIT_ASSERT_EQ(test, total, ps * 2 + ps * 3); + + gpu_buddy_free_list(&mm, &allocated, 0); + gpu_buddy_fini(&mm); +} + +static void gpu_test_buddy_alloc_pathological(struct kunit *test) +{ + u64 mm_size, size, start = 0; + struct gpu_buddy_block *block; + const int max_order = 3; + unsigned long flags = 0; + int order, top; + struct gpu_buddy mm; + LIST_HEAD(blocks); + LIST_HEAD(holes); + LIST_HEAD(tmp); + + /* + * Create a pot-sized mm, then allocate one of each possible + * order within. This should leave the mm with exactly one + * page left. Free the largest block, then whittle down again. + * Eventually we will have a fully 50% fragmented mm. + */ + + mm_size = SZ_4K << max_order; + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K), + "buddy_init failed\n"); + + KUNIT_EXPECT_EQ(test, mm.max_order, max_order); + + for (top = max_order; top; top--) { + /* Make room by freeing the largest allocated block */ + block = list_first_entry_or_null(&blocks, typeof(*block), link); + if (block) { + list_del(&block->link); + gpu_buddy_free_block(&mm, block); + } + + for (order = top; order--;) { + size = get_size(order, mm.chunk_size); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, + mm_size, size, size, + &tmp, flags), + "buddy_alloc hit -ENOMEM with order=%d, top=%d\n", + order, top); + + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link); + KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); + + list_move_tail(&block->link, &blocks); + } + + /* There should be one final page for this sub-allocation */ + size = get_size(0, mm.chunk_size); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc hit -ENOMEM for hole\n"); + + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link); + KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); + + list_move_tail(&block->link, &holes); + + size = get_size(top, mm.chunk_size); + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc unexpectedly succeeded at top-order %d/%d, it should be full!", + top, max_order); + } + + gpu_buddy_free_list(&mm, &holes, 0); + + /* Nothing larger than blocks of chunk_size now available */ + for (order = 1; order <= max_order; order++) { + size = get_size(order, mm.chunk_size); + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc unexpectedly succeeded at order %d, it should be full!", + order); + } + + list_splice_tail(&holes, &blocks); + gpu_buddy_free_list(&mm, &blocks, 0); + gpu_buddy_fini(&mm); +} + +static void gpu_test_buddy_alloc_pessimistic(struct kunit *test) +{ + u64 mm_size, size, start = 0; + struct gpu_buddy_block *block, *bn; + const unsigned int max_order = 16; + unsigned long flags = 0; + struct gpu_buddy mm; + unsigned int order; + LIST_HEAD(blocks); + LIST_HEAD(tmp); + + /* + * Create a pot-sized mm, then allocate one of each possible + * order within. This should leave the mm with exactly one + * page left. + */ + + mm_size = SZ_4K << max_order; + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K), + "buddy_init failed\n"); + + KUNIT_EXPECT_EQ(test, mm.max_order, max_order); + + for (order = 0; order < max_order; order++) { + size = get_size(order, mm.chunk_size); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc hit -ENOMEM with order=%d\n", + order); + + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link); + KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); + + list_move_tail(&block->link, &blocks); + } + + /* And now the last remaining block available */ + size = get_size(0, mm.chunk_size); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc hit -ENOMEM on final alloc\n"); + + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link); + KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); + + list_move_tail(&block->link, &blocks); + + /* Should be completely full! */ + for (order = max_order; order--;) { + size = get_size(order, mm.chunk_size); + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc unexpectedly succeeded, it should be full!"); + } + + block = list_last_entry(&blocks, typeof(*block), link); + list_del(&block->link); + gpu_buddy_free_block(&mm, block); + + /* As we free in increasing size, we make available larger blocks */ + order = 1; + list_for_each_entry_safe(block, bn, &blocks, link) { + list_del(&block->link); + gpu_buddy_free_block(&mm, block); + + size = get_size(order, mm.chunk_size); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc hit -ENOMEM with order=%d\n", + order); + + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link); + KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); + + list_del(&block->link); + gpu_buddy_free_block(&mm, block); + order++; + } + + /* To confirm, now the whole mm should be available */ + size = get_size(max_order, mm.chunk_size); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc (realloc) hit -ENOMEM with order=%d\n", + max_order); + + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link); + KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); + + list_del(&block->link); + gpu_buddy_free_block(&mm, block); + gpu_buddy_free_list(&mm, &blocks, 0); + gpu_buddy_fini(&mm); +} + +static void gpu_test_buddy_alloc_optimistic(struct kunit *test) +{ + u64 mm_size, size, start = 0; + struct gpu_buddy_block *block; + unsigned long flags = 0; + const int max_order = 16; + struct gpu_buddy mm; + LIST_HEAD(blocks); + LIST_HEAD(tmp); + int order; + + /* + * Create a mm with one block of each order available, and + * try to allocate them all. + */ + + mm_size = SZ_4K * ((1 << (max_order + 1)) - 1); + + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K), + "buddy_init failed\n"); + + KUNIT_EXPECT_EQ(test, mm.max_order, max_order); + + for (order = 0; order <= max_order; order++) { + size = get_size(order, mm.chunk_size); + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc hit -ENOMEM with order=%d\n", + order); + + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link); + KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n"); + + list_move_tail(&block->link, &blocks); + } + + /* Should be completely full! */ + size = get_size(0, mm.chunk_size); + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size, + size, size, &tmp, flags), + "buddy_alloc unexpectedly succeeded, it should be full!"); + + gpu_buddy_free_list(&mm, &blocks, 0); + gpu_buddy_fini(&mm); +} + +static void gpu_test_buddy_alloc_limit(struct kunit *test) +{ + u64 size = U64_MAX, start = 0; + struct gpu_buddy_block *block; + unsigned long flags = 0; + LIST_HEAD(allocated); + struct gpu_buddy mm; + + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, size, SZ_4K)); + + KUNIT_EXPECT_EQ_MSG(test, mm.max_order, GPU_BUDDY_MAX_ORDER, + "mm.max_order(%d) != %d\n", mm.max_order, + GPU_BUDDY_MAX_ORDER); + + size = mm.chunk_size << mm.max_order; + KUNIT_EXPECT_FALSE(test, gpu_buddy_alloc_blocks(&mm, start, size, size, + mm.chunk_size, &allocated, flags)); + + block = list_first_entry_or_null(&allocated, struct gpu_buddy_block, link); + KUNIT_EXPECT_TRUE(test, block); + + KUNIT_EXPECT_EQ_MSG(test, gpu_buddy_block_order(block), mm.max_order, + "block order(%d) != %d\n", + gpu_buddy_block_order(block), mm.max_order); + + KUNIT_EXPECT_EQ_MSG(test, gpu_buddy_block_size(&mm, block), + BIT_ULL(mm.max_order) * mm.chunk_size, + "block size(%llu) != %llu\n", + gpu_buddy_block_size(&mm, block), + BIT_ULL(mm.max_order) * mm.chunk_size); + + gpu_buddy_free_list(&mm, &allocated, 0); + gpu_buddy_fini(&mm); +} + +static int gpu_buddy_suite_init(struct kunit_suite *suite) +{ + while (!random_seed) + random_seed = get_random_u32(); + + kunit_info(suite, "Testing GPU buddy manager, with random_seed=0x%x\n", + random_seed); + + return 0; +} + +static struct kunit_case gpu_buddy_tests[] = { + KUNIT_CASE(gpu_test_buddy_alloc_limit), + KUNIT_CASE(gpu_test_buddy_alloc_optimistic), + KUNIT_CASE(gpu_test_buddy_alloc_pessimistic), + KUNIT_CASE(gpu_test_buddy_alloc_pathological), + KUNIT_CASE(gpu_test_buddy_alloc_contiguous), + KUNIT_CASE(gpu_test_buddy_alloc_clear), + KUNIT_CASE(gpu_test_buddy_alloc_range_bias), + {} +}; + +static struct kunit_suite gpu_buddy_test_suite = { + .name = "gpu_buddy", + .suite_init = gpu_buddy_suite_init, + .test_cases = gpu_buddy_tests, +}; + +kunit_test_suite(gpu_buddy_test_suite); + +MODULE_AUTHOR("Intel Corporation"); +MODULE_DESCRIPTION("Kunit test for gpu_buddy functions"); +MODULE_LICENSE("GPL"); diff --git a/drivers/gpu/tests/gpu_random.c b/drivers/gpu/tests/gpu_random.c new file mode 100644 index 000000000000..6356372f7e52 --- /dev/null +++ b/drivers/gpu/tests/gpu_random.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include + +#include "gpu_random.h" + +u32 gpu_prandom_u32_max_state(u32 ep_ro, struct rnd_state *state) +{ + return upper_32_bits((u64)prandom_u32_state(state) * ep_ro); +} +EXPORT_SYMBOL(gpu_prandom_u32_max_state); + +void gpu_random_reorder(unsigned int *order, unsigned int count, + struct rnd_state *state) +{ + unsigned int i, j; + + for (i = 0; i < count; ++i) { + BUILD_BUG_ON(sizeof(unsigned int) > sizeof(u32)); + j = gpu_prandom_u32_max_state(count, state); + swap(order[i], order[j]); + } +} +EXPORT_SYMBOL(gpu_random_reorder); + +unsigned int *gpu_random_order(unsigned int count, struct rnd_state *state) +{ + unsigned int *order, i; + + order = kmalloc_array(count, sizeof(*order), GFP_KERNEL); + if (!order) + return order; + + for (i = 0; i < count; i++) + order[i] = i; + + gpu_random_reorder(order, count, state); + return order; +} +EXPORT_SYMBOL(gpu_random_order); diff --git a/drivers/gpu/tests/gpu_random.h b/drivers/gpu/tests/gpu_random.h new file mode 100644 index 000000000000..b68cf3448264 --- /dev/null +++ b/drivers/gpu/tests/gpu_random.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __GPU_RANDOM_H__ +#define __GPU_RANDOM_H__ + +/* This is a temporary home for a couple of utility functions that should + * be transposed to lib/ at the earliest convenience. + */ + +#include + +#define GPU_RND_STATE_INITIALIZER(seed__) ({ \ + struct rnd_state state__; \ + prandom_seed_state(&state__, (seed__)); \ + state__; \ +}) + +#define GPU_RND_STATE(name__, seed__) \ + struct rnd_state name__ = GPU_RND_STATE_INITIALIZER(seed__) + +unsigned int *gpu_random_order(unsigned int count, + struct rnd_state *state); +void gpu_random_reorder(unsigned int *order, + unsigned int count, + struct rnd_state *state); +u32 gpu_prandom_u32_max_state(u32 ep_ro, + struct rnd_state *state); + +#endif /* !__GPU_RANDOM_H__ */ diff --git a/drivers/hwmon/asus-ec-sensors.c b/drivers/hwmon/asus-ec-sensors.c index 95c50d3a788c..13fd212ad404 100644 --- a/drivers/hwmon/asus-ec-sensors.c +++ b/drivers/hwmon/asus-ec-sensors.c @@ -870,7 +870,7 @@ struct ec_sensors_data { /* sorted list of unique register banks */ u8 banks[ASUS_EC_MAX_BANK + 1]; /* in jiffies */ - unsigned long last_updated; + u64 next_update; struct lock_data lock_data; /* number of board EC sensors */ u8 nr_sensors; @@ -1050,7 +1050,7 @@ static int asus_ec_block_read(const struct device *dev, } for (ireg = 0; ireg < ec->nr_registers; ireg++) { reg_bank = register_bank(ec->registers[ireg]); - if (reg_bank < bank) { + if (reg_bank != bank) { continue; } ec_read(register_index(ec->registers[ireg]), @@ -1139,13 +1139,12 @@ static int get_cached_value_or_update(const struct device *dev, int sensor_index, struct ec_sensors_data *state, s32 *value) { - if (time_after(jiffies, state->last_updated + HZ)) { + if (time_after64(get_jiffies_64(), state->next_update)) { if (update_ec_sensors(dev, state)) { dev_err(dev, "update_ec_sensors() failure\n"); return -EIO; } - - state->last_updated = jiffies; + state->next_update = get_jiffies_64() + HZ; } *value = state->sensors[sensor_index].cached_value; @@ -1263,6 +1262,7 @@ static int asus_ec_probe(struct platform_device *pdev) if (!ec_data) return -ENOMEM; + ec_data->next_update = INITIAL_JIFFIES; dev_set_drvdata(dev, ec_data); ec_data->board_info = pboard_info; @@ -1353,9 +1353,11 @@ static int asus_ec_probe(struct platform_device *pdev) if (!nr_count[type]) continue; - asus_ec_hwmon_add_chan_info(asus_ec_hwmon_chan, dev, - nr_count[type], type, - hwmon_attributes[type]); + status = asus_ec_hwmon_add_chan_info(asus_ec_hwmon_chan, dev, + nr_count[type], type, + hwmon_attributes[type]); + if (status) + return status; *ptr_asus_ec_ci++ = asus_ec_hwmon_chan++; } diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c index b7b911f8359c..71e48b3bba60 100644 --- a/drivers/hwmon/corsair-cpro.c +++ b/drivers/hwmon/corsair-cpro.c @@ -645,6 +645,7 @@ static int ccp_probe(struct hid_device *hdev, const struct hid_device_id *id) out_hw_close: hid_hw_close(hdev); + hid_device_io_stop(hdev); out_hw_stop: hid_hw_stop(hdev); return ret; diff --git a/drivers/hwmon/corsair-psu.c b/drivers/hwmon/corsair-psu.c index ea28c9219507..4681af684359 100644 --- a/drivers/hwmon/corsair-psu.c +++ b/drivers/hwmon/corsair-psu.c @@ -831,6 +831,7 @@ static int corsairpsu_probe(struct hid_device *hdev, const struct hid_device_id fail_and_close: hid_hw_close(hdev); + hid_device_io_stop(hdev); fail_and_stop: hid_hw_stop(hdev); return ret; diff --git a/drivers/hwmon/gigabyte_waterforce.c b/drivers/hwmon/gigabyte_waterforce.c index 27487e215bdd..4eea05f8b569 100644 --- a/drivers/hwmon/gigabyte_waterforce.c +++ b/drivers/hwmon/gigabyte_waterforce.c @@ -371,13 +371,15 @@ static int waterforce_probe(struct hid_device *hdev, const struct hid_device_id if (IS_ERR(priv->hwmon_dev)) { ret = PTR_ERR(priv->hwmon_dev); hid_err(hdev, "hwmon registration failed with %d\n", ret); - goto fail_and_close; + goto fail_and_io_stop; } waterforce_debugfs_init(priv); return 0; +fail_and_io_stop: + hid_device_io_stop(hdev); fail_and_close: hid_hw_close(hdev); fail_and_stop: diff --git a/drivers/hwmon/nzxt-kraken3.c b/drivers/hwmon/nzxt-kraken3.c index d00409bcab93..05525406c5fb 100644 --- a/drivers/hwmon/nzxt-kraken3.c +++ b/drivers/hwmon/nzxt-kraken3.c @@ -948,7 +948,7 @@ static int kraken3_probe(struct hid_device *hdev, const struct hid_device_id *id ret = kraken3_init_device(hdev); if (ret < 0) { hid_err(hdev, "device init failed with %d\n", ret); - goto fail_and_close; + goto fail_and_stop_io; } ret = kraken3_get_fw_ver(hdev); @@ -960,13 +960,15 @@ static int kraken3_probe(struct hid_device *hdev, const struct hid_device_id *id if (IS_ERR(priv->hwmon_dev)) { ret = PTR_ERR(priv->hwmon_dev); hid_err(hdev, "hwmon registration failed with %d\n", ret); - goto fail_and_close; + goto fail_and_stop_io; } kraken3_debugfs_init(priv, device_name); return 0; +fail_and_stop_io: + hid_device_io_stop(hdev); fail_and_close: hid_hw_close(hdev); fail_and_stop: diff --git a/drivers/hwmon/nzxt-smart2.c b/drivers/hwmon/nzxt-smart2.c index 58ef9fa0184b..e2316c46629d 100644 --- a/drivers/hwmon/nzxt-smart2.c +++ b/drivers/hwmon/nzxt-smart2.c @@ -768,7 +768,7 @@ static int nzxt_smart2_hid_probe(struct hid_device *hdev, out_hw_close: hid_hw_close(hdev); - + hid_device_io_stop(hdev); out_hw_stop: hid_hw_stop(hdev); return ret; diff --git a/drivers/hwmon/occ/common.c b/drivers/hwmon/occ/common.c index e18e80e832fd..175208d712b0 100644 --- a/drivers/hwmon/occ/common.c +++ b/drivers/hwmon/occ/common.c @@ -1052,32 +1052,49 @@ static int occ_setup_sensor_attrs(struct occ *occ) } /* only need to do this once at startup, as OCC won't change sensors on us */ -static void occ_parse_poll_response(struct occ *occ) +static int occ_parse_poll_response(struct occ *occ) { unsigned int i, old_offset, offset = 0, size = 0; + u16 data_length; struct occ_sensor *sensor; - struct occ_sensors *sensors = &occ->sensors; + struct occ_sensors parsed = {}; + struct occ_sensors *sensors = &parsed; struct occ_response *resp = &occ->resp; struct occ_poll_response *poll = (struct occ_poll_response *)&resp->data[0]; struct occ_poll_response_header *header = &poll->header; struct occ_sensor_data_block *block = &poll->block; + data_length = get_unaligned_be16(&resp->data_length); + if (data_length < sizeof(*header) || data_length > OCC_RESP_DATA_BYTES) { + dev_err(occ->bus_dev, "invalid OCC poll response length %u\n", + data_length); + return -EMSGSIZE; + } + dev_info(occ->bus_dev, "OCC found, code level: %.16s\n", header->occ_code_level); for (i = 0; i < header->num_sensor_data_blocks; ++i) { block = (struct occ_sensor_data_block *)((u8 *)block + offset); + if (size + sizeof(*header) + sizeof(block->header) > + data_length) { + dev_err(occ->bus_dev, + "truncated OCC sensor block header\n"); + return -EMSGSIZE; + } + old_offset = offset; offset = (block->header.num_sensors * block->header.sensor_length) + sizeof(block->header); - size += offset; /* validate all the length/size fields */ - if ((size + sizeof(*header)) >= OCC_RESP_DATA_BYTES) { - dev_warn(occ->bus_dev, "exceeded response buffer\n"); - return; + if (size + sizeof(*header) + offset > data_length) { + dev_err(occ->bus_dev, + "exceeded OCC poll response length\n"); + return -EMSGSIZE; } + size += offset; dev_dbg(occ->bus_dev, " %04x..%04x: %.4s (%d sensors)\n", old_offset, offset - 1, block->header.eye_catcher, @@ -1107,6 +1124,9 @@ static void occ_parse_poll_response(struct occ *occ) dev_dbg(occ->bus_dev, "Max resp size: %u+%zd=%zd\n", size, sizeof(*header), size + sizeof(*header)); + occ->sensors = parsed; + + return 0; } int occ_active(struct occ *occ, bool active) @@ -1138,10 +1158,12 @@ int occ_active(struct occ *occ, bool active) goto unlock; } - occ->active = true; occ->next_update = jiffies + OCC_UPDATE_FREQUENCY; - occ_parse_poll_response(occ); + rc = occ_parse_poll_response(occ); + if (rc) + goto unlock; + occ->active = true; rc = occ_setup_sensor_attrs(occ); if (rc) { dev_err(occ->bus_dev, diff --git a/drivers/hwtracing/intel_th/core.c b/drivers/hwtracing/intel_th/core.c index e3a7ab112ea9..dc888a11c7a0 100644 --- a/drivers/hwtracing/intel_th/core.c +++ b/drivers/hwtracing/intel_th/core.c @@ -843,18 +843,8 @@ static int intel_th_output_open(struct inode *inode, struct file *file) return err; } -static int intel_th_output_release(struct inode *inode, struct file *file) -{ - struct intel_th_device *thdev = file->private_data; - - put_device(&thdev->dev); - - return 0; -} - static const struct file_operations intel_th_output_fops = { .open = intel_th_output_open, - .release = intel_th_output_release, .llseek = noop_llseek, }; diff --git a/drivers/hwtracing/intel_th/msu.c b/drivers/hwtracing/intel_th/msu.c index f3a13b300835..d3170b1476de 100644 --- a/drivers/hwtracing/intel_th/msu.c +++ b/drivers/hwtracing/intel_th/msu.c @@ -1490,8 +1490,10 @@ static int intel_th_msc_release(struct inode *inode, struct file *file) { struct msc_iter *iter = file->private_data; struct msc *msc = iter->msc; + struct intel_th_device *thdev = msc->thdev; msc_iter_remove(iter, msc); + put_device(&thdev->dev); return 0; } diff --git a/drivers/i3c/master/mipi-i3c-hci/core.c b/drivers/i3c/master/mipi-i3c-hci/core.c index 75e6610cfd81..1eeab337e5e5 100644 --- a/drivers/i3c/master/mipi-i3c-hci/core.c +++ b/drivers/i3c/master/mipi-i3c-hci/core.c @@ -147,7 +147,10 @@ static int i3c_hci_bus_init(struct i3c_master_controller *m) if (hci->quirks & HCI_QUIRK_RESP_BUF_THLD) amd_set_resp_buf_thld(hci); - reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE); + WRITE_ONCE(hci->irq_inactive, false); + + /* Enable bus with Hot-Join disabled */ + reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE | HC_CONTROL_HOT_JOIN_CTRL); dev_dbg(&hci->master.dev, "HC_CONTROL = %#x", reg_read(HC_CONTROL)); return 0; @@ -536,6 +539,15 @@ static irqreturn_t i3c_hci_irq_handler(int irq, void *dev_id) irqreturn_t result = IRQ_NONE; u32 val; + /* + * The IRQ can be shared, so the handler may be called when the IRQ is + * due to a different device. That could happen before the controller + * has been initialized, so exit immediately if IRQs are not expected + * for this device. + */ + if (READ_ONCE(hci->irq_inactive)) + return IRQ_NONE; + val = reg_read(INTR_STATUS); reg_write(INTR_STATUS, val); dev_dbg(&hci->master.dev, "INTR_STATUS %#x", val); @@ -775,6 +787,8 @@ static int i3c_hci_probe(struct platform_device *pdev) if (ret) return ret; + WRITE_ONCE(hci->irq_inactive, true); + irq = platform_get_irq(pdev, 0); ret = devm_request_irq(&pdev->dev, irq, i3c_hci_irq_handler, IRQF_SHARED, NULL, hci); diff --git a/drivers/i3c/master/mipi-i3c-hci/hci.h b/drivers/i3c/master/mipi-i3c-hci/hci.h index 32c8aecde9f7..5502ab0a3535 100644 --- a/drivers/i3c/master/mipi-i3c-hci/hci.h +++ b/drivers/i3c/master/mipi-i3c-hci/hci.h @@ -46,6 +46,7 @@ struct i3c_hci { void *io_data; const struct hci_cmd_ops *cmd; spinlock_t lock; + bool irq_inactive; struct mutex control_mutex; atomic_t next_cmd_tid; u32 caps; diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index ce511800b056..648acc2222cf 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -5237,7 +5237,7 @@ static int cma_netevent_callback(struct notifier_block *self, list_for_each_entry(current_id, &ips_node->id_list, id_list_entry) { if (!memcmp(current_id->id.route.addr.dev_addr.dst_dev_addr, - neigh->ha, ETH_ALEN)) + neigh->ha, neigh->dev->addr_len)) continue; cma_id_get(current_id); if (!queue_work(cma_wq, ¤t_id->id.net_work)) diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c index 8f26bfb69586..5150cd53d443 100644 --- a/drivers/infiniband/core/mad.c +++ b/drivers/infiniband/core/mad.c @@ -2031,6 +2031,24 @@ void ib_mark_mad_done(struct ib_mad_send_wr_private *mad_send_wr) change_mad_state(mad_send_wr, IB_MAD_STATE_EARLY_RESP); } +static bool is_kernel_rmpp_data_response(struct ib_mad_agent_private *agent, + struct ib_mad_recv_wc *mad_recv_wc) +{ + const struct ib_mad_hdr *mad_hdr = &mad_recv_wc->recv_buf.mad->mad_hdr; + struct ib_rmpp_mad *rmpp_mad; + + if (!ib_mad_kernel_rmpp_agent(&agent->agent) || + !ib_response_mad(mad_hdr) || + !ib_is_mad_class_rmpp(mad_hdr->mgmt_class)) + return false; + + rmpp_mad = (struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad; + + return (ib_get_rmpp_flags(&rmpp_mad->rmpp_hdr) & + IB_MGMT_RMPP_FLAG_ACTIVE) && + rmpp_mad->rmpp_hdr.rmpp_type == IB_MGMT_RMPP_TYPE_DATA; +} + static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, struct ib_mad_recv_wc *mad_recv_wc) { @@ -2050,6 +2068,18 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, } list_add(&mad_recv_wc->recv_buf.list, &mad_recv_wc->rmpp_list); + if (is_kernel_rmpp_data_response(mad_agent_priv, mad_recv_wc)) { + spin_lock_irqsave(&mad_agent_priv->lock, flags); + mad_send_wr = ib_find_send_mad(mad_agent_priv, mad_recv_wc); + spin_unlock_irqrestore(&mad_agent_priv->lock, flags); + + if (!mad_send_wr) { + ib_free_recv_mad(mad_recv_wc); + deref_mad_agent(mad_agent_priv); + return; + } + } + if (ib_mad_kernel_rmpp_agent(&mad_agent_priv->agent)) { mad_recv_wc = ib_process_rmpp_recv_wc(mad_agent_priv, mad_recv_wc); diff --git a/drivers/infiniband/core/umem_dmabuf.c b/drivers/infiniband/core/umem_dmabuf.c index b239ac09ec29..4fc73f53342b 100644 --- a/drivers/infiniband/core/umem_dmabuf.c +++ b/drivers/infiniband/core/umem_dmabuf.c @@ -206,6 +206,10 @@ static void ib_umem_dmabuf_revoke_locked(struct dma_buf_attachment *attach) if (umem_dmabuf->revoked) return; + + if (umem_dmabuf->pinned_revoke) + umem_dmabuf->pinned_revoke(umem_dmabuf->private); + ib_umem_dmabuf_unmap_pages(umem_dmabuf); if (umem_dmabuf->pinned) { dma_buf_unpin(umem_dmabuf->attach); @@ -214,6 +218,11 @@ static void ib_umem_dmabuf_revoke_locked(struct dma_buf_attachment *attach) umem_dmabuf->revoked = 1; } +static struct dma_buf_attach_ops ib_umem_dmabuf_attach_pinned_revocable_ops = { + .allow_peer2peer = true, + .move_notify = ib_umem_dmabuf_revoke_locked, +}; + static struct ib_umem_dmabuf * ib_umem_dmabuf_get_pinned_and_lock(struct ib_device *device, struct device *dma_device, @@ -266,6 +275,58 @@ ib_umem_dmabuf_get_pinned_with_dma_device(struct ib_device *device, } EXPORT_SYMBOL(ib_umem_dmabuf_get_pinned_with_dma_device); +/** + * ib_umem_dmabuf_get_pinned_revocable_and_lock - Map & pin a revocable dmabuf + * @device: IB device. + * @offset: Start offset. + * @size: Length. + * @fd: dmabuf fd. + * @access: Access flags. + * + * Obtains a umem from a dmabuf for drivers/devices that can support revocation. + * + * Returns with dma_resv_lock held upon success. The driver must set the revoke + * callback prior to unlock by calling ib_umem_dmabuf_set_revoke_locked(). + * + * When a revocation occurs, the revoke callback will be called. The driver must + * ensure that the region is no longer accessed when the callback returns. Any + * subsequent access attempts should also probably cause an AE for MRs. + * + * If the umem is used for an MR, the driver must ensure that the key remains in + * use such that it cannot be obtained by a new region until this region is + * fully deregistered (i.e., ibv_dereg_mr). If a driver needs to serialize with + * revoke calls, it can use dma_resv_lock. + * + * If successful, then the revoke callback may be called at any time and will + * also be called automatically upon ib_umem_release (serialized). The revoke + * callback will be called one time at most. + * + * Return: A pointer to ib_umem_dmabuf on success, or an ERR_PTR on failure. + */ +struct ib_umem_dmabuf * +ib_umem_dmabuf_get_pinned_revocable_and_lock(struct ib_device *device, + unsigned long offset, size_t size, + int fd, int access) +{ + const struct dma_buf_attach_ops *ops = + &ib_umem_dmabuf_attach_pinned_revocable_ops; + + return ib_umem_dmabuf_get_pinned_and_lock(device, device->dma_device, + offset, size, fd, access, + ops); +} +EXPORT_SYMBOL(ib_umem_dmabuf_get_pinned_revocable_and_lock); + +void ib_umem_dmabuf_set_revoke_locked(struct ib_umem_dmabuf *umem_dmabuf, + void (*revoke)(void *priv), void *priv) +{ + dma_resv_assert_held(umem_dmabuf->attach->dmabuf->resv); + + umem_dmabuf->pinned_revoke = revoke; + umem_dmabuf->private = priv; +} +EXPORT_SYMBOL(ib_umem_dmabuf_set_revoke_locked); + struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned(struct ib_device *device, unsigned long offset, size_t size, int fd, diff --git a/drivers/infiniband/hw/erdma/erdma_qp.c b/drivers/infiniband/hw/erdma/erdma_qp.c index 25f6c49aec77..e002343832f7 100644 --- a/drivers/infiniband/hw/erdma/erdma_qp.c +++ b/drivers/infiniband/hw/erdma/erdma_qp.c @@ -734,7 +734,7 @@ int erdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *recv_wr, const struct ib_recv_wr *wr = recv_wr; struct erdma_qp *qp = to_eqp(ibqp); unsigned long flags; - int ret; + int ret = 0; spin_lock_irqsave(&qp->lock, flags); diff --git a/drivers/infiniband/hw/hns/hns_roce_hem.c b/drivers/infiniband/hw/hns/hns_roce_hem.c index 1680d0ac071e..ae7a3939124f 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hem.c +++ b/drivers/infiniband/hw/hns/hns_roce_hem.c @@ -842,7 +842,7 @@ static void hns_roce_cleanup_mhop_hem_table(struct hns_roce_dev *hr_dev, mhop.bt_chunk_size; for (i = 0; i < table->num_hem; ++i) { - obj = i * buf_chunk_size / table->obj_size; + obj = (u64)i * buf_chunk_size / table->obj_size; if (table->hem[i]) hns_roce_table_mhop_put(hr_dev, table, obj, 0); } diff --git a/drivers/infiniband/hw/irdma/uk.c b/drivers/infiniband/hw/irdma/uk.c index 4cc81d61be7f..7aab00761966 100644 --- a/drivers/infiniband/hw/irdma/uk.c +++ b/drivers/infiniband/hw/irdma/uk.c @@ -1540,15 +1540,12 @@ static const struct irdma_wqe_uk_ops iw_wqe_uk_ops_gen_1 = { * irdma_setup_connection_wqes - setup WQEs necessary to complete * connection. * @qp: hw qp (user and kernel) - * @info: qp initialization info */ -static void irdma_setup_connection_wqes(struct irdma_qp_uk *qp, - struct irdma_qp_uk_init_info *info) +static void irdma_setup_connection_wqes(struct irdma_qp_uk *qp) { u16 move_cnt = 1; - if (!info->legacy_mode && - (qp->uk_attrs->feature_flags & IRDMA_FEATURE_RTS_AE)) + if (qp->uk_attrs->feature_flags & IRDMA_FEATURE_RTS_AE) move_cnt = 3; qp->conn_wqes = move_cnt; @@ -1699,7 +1696,7 @@ int irdma_uk_qp_init(struct irdma_qp_uk *qp, struct irdma_qp_uk_init_info *info) sq_ring_size = qp->sq_size << info->sq_shift; IRDMA_RING_INIT(qp->sq_ring, sq_ring_size); if (info->first_sq_wq) { - irdma_setup_connection_wqes(qp, info); + irdma_setup_connection_wqes(qp); qp->swqe_polarity = 1; qp->first_sq_wq = true; } else { diff --git a/drivers/infiniband/hw/irdma/user.h b/drivers/infiniband/hw/irdma/user.h index aeebf768174a..a0b409a895f0 100644 --- a/drivers/infiniband/hw/irdma/user.h +++ b/drivers/infiniband/hw/irdma/user.h @@ -560,7 +560,6 @@ struct irdma_qp_uk_init_info { u8 sq_shift; u8 rq_shift; int abi_ver; - bool legacy_mode; struct irdma_srq_uk *srq_uk; }; diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index 2b85e32134a5..4084168d0194 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -631,18 +631,16 @@ static int irdma_setup_umode_qp(struct ib_udata *udata, iwqp->ctx_info.qp_compl_ctx = req.user_compl_ctx; iwqp->user_mode = 1; - if (req.user_wqe_bufs) { - info->qp_uk_init_info.legacy_mode = ucontext->legacy_mode; - spin_lock_irqsave(&ucontext->qp_reg_mem_list_lock, flags); - iwqp->iwpbl = irdma_get_pbl((unsigned long)req.user_wqe_bufs, - &ucontext->qp_reg_mem_list); - spin_unlock_irqrestore(&ucontext->qp_reg_mem_list_lock, flags); - if (!iwqp->iwpbl) { - ret = -ENODATA; - ibdev_dbg(&iwdev->ibdev, "VERBS: no pbl info\n"); - return ret; - } + spin_lock_irqsave(&ucontext->qp_reg_mem_list_lock, flags); + iwqp->iwpbl = irdma_get_pbl((unsigned long)req.user_wqe_bufs, + &ucontext->qp_reg_mem_list); + spin_unlock_irqrestore(&ucontext->qp_reg_mem_list_lock, flags); + + if (!iwqp->iwpbl) { + ret = -ENODATA; + ibdev_dbg(&iwdev->ibdev, "VERBS: no pbl info\n"); + return ret; } if (!ucontext->use_raw_attrs) { @@ -2068,10 +2066,6 @@ static int irdma_resize_cq(struct ib_cq *ibcq, int entries, rdma_udata_to_drv_context(udata, struct irdma_ucontext, ibucontext); - /* CQ resize not supported with legacy GEN_1 libi40iw */ - if (ucontext->legacy_mode) - return -EOPNOTSUPP; - if (ib_copy_from_udata(&req, udata, min(sizeof(req), udata->inlen))) return -EINVAL; @@ -2550,7 +2544,7 @@ static int irdma_create_cq(struct ib_cq *ibcq, cqmr = &iwpbl->cq_mr; if (rf->sc_dev.hw_attrs.uk_attrs.feature_flags & - IRDMA_FEATURE_CQ_RESIZE && !ucontext->legacy_mode) { + IRDMA_FEATURE_CQ_RESIZE) { spin_lock_irqsave(&ucontext->cq_reg_mem_list_lock, flags); iwpbl_shadow = irdma_get_pbl( (unsigned long)req.user_shadow_area, @@ -2806,7 +2800,7 @@ static bool irdma_check_mem_contiguous(u64 *arr, u32 npages, u32 pg_size) u32 pg_idx; for (pg_idx = 0; pg_idx < npages; pg_idx++) { - if ((*arr + (pg_size * pg_idx)) != arr[pg_idx]) + if ((*arr + ((u64)pg_size * pg_idx)) != arr[pg_idx]) return false; } @@ -2839,7 +2833,7 @@ static bool irdma_check_mr_contiguous(struct irdma_pble_alloc *palloc, for (i = 0; i < lvl2->leaf_cnt; i++, leaf++) { arr = leaf->addr; - if ((*start_addr + (i * pg_size * PBLE_PER_PAGE)) != *arr) + if ((*start_addr + ((u64)i * pg_size * PBLE_PER_PAGE)) != *arr) return false; ret = irdma_check_mem_contiguous(arr, leaf->cnt, pg_size); if (!ret) @@ -3756,6 +3750,9 @@ static struct ib_mr *irdma_rereg_user_mr(struct ib_mr *ib_mr, int flags, if (flags & ~(IB_MR_REREG_TRANS | IB_MR_REREG_PD | IB_MR_REREG_ACCESS)) return ERR_PTR(-EOPNOTSUPP); + if (iwmr->type != IRDMA_MEMREG_TYPE_MEM) + return ERR_PTR(-EINVAL); + ret = ib_umem_check_rereg(iwmr->region, flags, new_access); if (ret) return ERR_PTR(ret); diff --git a/drivers/infiniband/hw/mana/wr.c b/drivers/infiniband/hw/mana/wr.c index 1813567d3b16..36a1d506f08f 100644 --- a/drivers/infiniband/hw/mana/wr.c +++ b/drivers/infiniband/hw/mana/wr.c @@ -144,7 +144,7 @@ static int mana_ib_post_send_ud(struct mana_ib_qp *qp, const struct ib_ud_wr *wr int mana_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, const struct ib_send_wr **bad_wr) { - int err; + int err = 0; struct mana_ib_qp *qp = container_of(ibqp, struct mana_ib_qp, ibqp); for (; wr; wr = wr->next) { diff --git a/drivers/infiniband/sw/siw/siw_verbs.c b/drivers/infiniband/sw/siw/siw_verbs.c index efa2f097b582..9fbade0b8571 100644 --- a/drivers/infiniband/sw/siw/siw_verbs.c +++ b/drivers/infiniband/sw/siw/siw_verbs.c @@ -316,6 +316,7 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, struct siw_ucontext *uctx = rdma_udata_to_drv_context(udata, struct siw_ucontext, base_ucontext); + struct siw_uresp_create_qp uresp = {}; unsigned long flags; int num_sqe, num_rqe, rv = 0; size_t length; @@ -369,11 +370,6 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, spin_lock_init(&qp->rq_lock); spin_lock_init(&qp->orq_lock); - rv = siw_qp_add(sdev, qp); - if (rv) - goto err_atomic; - - /* All queue indices are derived from modulo operations * on a free running 'get' (consumer) and 'put' (producer) * unsigned counter. Having queue sizes at power of two @@ -391,14 +387,14 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, if (qp->sendq == NULL) { rv = -ENOMEM; - goto err_out_xa; + goto err_out; } if (attrs->sq_sig_type != IB_SIGNAL_REQ_WR) { if (attrs->sq_sig_type == IB_SIGNAL_ALL_WR) qp->attrs.flags |= SIW_SIGNAL_ALL_WR; else { rv = -EINVAL; - goto err_out_xa; + goto err_out; } } qp->pd = pd; @@ -424,7 +420,7 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, if (qp->recvq == NULL) { rv = -ENOMEM; - goto err_out_xa; + goto err_out; } qp->attrs.rq_size = num_rqe; } @@ -439,11 +435,8 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, qp->attrs.state = SIW_QP_STATE_IDLE; if (udata) { - struct siw_uresp_create_qp uresp = {}; - uresp.num_sqe = num_sqe; uresp.num_rqe = num_rqe; - uresp.qp_id = qp_id(qp); if (qp->sendq) { length = num_sqe * sizeof(struct siw_sqe); @@ -452,7 +445,7 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, length, &uresp.sq_key); if (!qp->sq_entry) { rv = -ENOMEM; - goto err_out_xa; + goto err_out; } } @@ -464,9 +457,23 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, if (!qp->rq_entry) { uresp.sq_key = SIW_INVAL_UOBJ_KEY; rv = -ENOMEM; - goto err_out_xa; + goto err_out; } } + } + qp->tx_cpu = siw_get_tx_cpu(sdev); + if (qp->tx_cpu < 0) { + rv = -EINVAL; + goto err_out; + } + init_completion(&qp->qp_free); + + rv = siw_qp_add(sdev, qp); + if (rv) + goto err_out_tx; + + if (udata) { + uresp.qp_id = qp_id(qp); if (udata->outlen < sizeof(uresp)) { rv = -EINVAL; @@ -476,22 +483,19 @@ int siw_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs, if (rv) goto err_out_xa; } - qp->tx_cpu = siw_get_tx_cpu(sdev); - if (qp->tx_cpu < 0) { - rv = -EINVAL; - goto err_out_xa; - } + INIT_LIST_HEAD(&qp->devq); spin_lock_irqsave(&sdev->lock, flags); list_add_tail(&qp->devq, &sdev->qp_list); spin_unlock_irqrestore(&sdev->lock, flags); - init_completion(&qp->qp_free); - return 0; err_out_xa: xa_erase(&sdev->qp_xa, qp_id(qp)); +err_out_tx: + siw_put_tx_cpu(qp->tx_cpu); +err_out: if (uctx) { rdma_user_mmap_entry_remove(qp->sq_entry); rdma_user_mmap_entry_remove(qp->rq_entry); diff --git a/drivers/input/misc/ims-pcu.c b/drivers/input/misc/ims-pcu.c index 94113e14bf59..168662a2cc89 100644 --- a/drivers/input/misc/ims-pcu.c +++ b/drivers/input/misc/ims-pcu.c @@ -448,6 +448,14 @@ static void ims_pcu_handle_response(struct ims_pcu *pcu) } } +static void ims_pcu_reset_packet(struct ims_pcu *pcu) +{ + pcu->have_stx = false; + pcu->have_dle = false; + pcu->read_pos = 0; + pcu->check_sum = 0; +} + static void ims_pcu_process_data(struct ims_pcu *pcu, struct urb *urb) { int i; @@ -460,6 +468,14 @@ static void ims_pcu_process_data(struct ims_pcu *pcu, struct urb *urb) continue; if (pcu->have_dle) { + if (pcu->read_pos >= IMS_PCU_BUF_SIZE) { + dev_warn(pcu->dev, + "Packet too long (%d bytes), discarding\n", + pcu->read_pos); + ims_pcu_reset_packet(pcu); + continue; + } + pcu->have_dle = false; pcu->read_buf[pcu->read_pos++] = data; pcu->check_sum += data; @@ -472,10 +488,8 @@ static void ims_pcu_process_data(struct ims_pcu *pcu, struct urb *urb) dev_warn(pcu->dev, "Unexpected STX at byte %d, discarding old data\n", pcu->read_pos); + ims_pcu_reset_packet(pcu); pcu->have_stx = true; - pcu->have_dle = false; - pcu->read_pos = 0; - pcu->check_sum = 0; break; case IMS_PCU_PROTOCOL_DLE: @@ -495,12 +509,18 @@ static void ims_pcu_process_data(struct ims_pcu *pcu, struct urb *urb) ims_pcu_handle_response(pcu); } - pcu->have_stx = false; - pcu->have_dle = false; - pcu->read_pos = 0; + ims_pcu_reset_packet(pcu); break; default: + if (pcu->read_pos >= IMS_PCU_BUF_SIZE) { + dev_warn(pcu->dev, + "Packet too long (%d bytes), discarding\n", + pcu->read_pos); + ims_pcu_reset_packet(pcu); + continue; + } + pcu->read_buf[pcu->read_pos++] = data; pcu->check_sum += data; break; diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 32175daa3dd8..69170146d442 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -3860,6 +3860,12 @@ static int __init parse_ivrs_acpihid(char *str) return 1; found: + if (early_acpihid_map_size == EARLY_MAP_SIZE) { + pr_err("Early ACPI HID map overflow - ignoring ivrs_acpihid%s\n", + str); + return 1; + } + p = acpiid; hid = strsep(&p, ":"); uid = p; diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index c458eec8a5bb..12302647e267 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1408,11 +1408,23 @@ static int iommu_completion_wait(struct amd_iommu *iommu) int ret; u64 data; - if (!iommu->need_sync) - return 0; - raw_spin_lock_irqsave(&iommu->lock, flags); + if (!iommu->need_sync) { + /* + * No command has been queued since the last completion-wait. + * A concurrent CPU may have already queued that CWAIT and + * cleared need_sync; need_sync == false only means a covering + * CWAIT is queued, not that all prior commands have completed. + * Wait for the last allocated sequence number so that any + * command queued before this call (possibly on another CPU) + * is guaranteed to have completed before returning. + */ + data = iommu->cmd_sem_val; + raw_spin_unlock_irqrestore(&iommu->lock, flags); + return wait_on_sem(iommu, data); + } + data = get_cmdsem_val(iommu); build_completion_wait(&cmd, iommu, data); @@ -1422,9 +1434,7 @@ static int iommu_completion_wait(struct amd_iommu *iommu) if (ret) return ret; - ret = wait_on_sem(iommu, data); - - return ret; + return wait_on_sem(iommu, data); } static void domain_flush_complete(struct protection_domain *domain) diff --git a/drivers/iommu/intel/perf.c b/drivers/iommu/intel/perf.c index dceeadc3ee7c..6afb55073e9f 100644 --- a/drivers/iommu/intel/perf.c +++ b/drivers/iommu/intel/perf.c @@ -63,7 +63,7 @@ void dmar_latency_disable(struct intel_iommu *iommu, enum latency_type type) return; spin_lock_irqsave(&latency_lock, flags); - memset(&lstat[type], 0, sizeof(*lstat) * DMAR_LATENCY_NUM); + memset(&lstat[type], 0, sizeof(*lstat)); spin_unlock_irqrestore(&latency_lock, flags); } diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index e147f71f91b7..6c275c767945 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -27,7 +27,7 @@ void intel_svm_check(struct intel_iommu *iommu) { - if (!pasid_supported(iommu)) + if (!pasid_supported(iommu) || !ecap_smpwc(iommu->ecap)) return; if (cpu_feature_enabled(X86_FEATURE_GBPAGES) && diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 03e59b85132a..d1e4fbc5a21d 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -39,36 +39,6 @@ static inline u64 fec_interleave(struct dm_verity *v, u64 offset) return offset + mod * (v->fec->rounds << v->data_dev_block_bits); } -/* - * Read error-correcting codes for the requested RS block. Returns a pointer - * to the data block. Caller is responsible for releasing buf. - */ -static u8 *fec_read_parity(struct dm_verity *v, u64 rsb, int index, - unsigned int *offset, unsigned int par_buf_offset, - struct dm_buffer **buf, unsigned short ioprio) -{ - u64 position, block, rem; - u8 *res; - - /* We have already part of parity bytes read, skip to the next block */ - if (par_buf_offset) - index++; - - position = (index + rsb) * v->fec->roots; - block = div64_u64_rem(position, v->fec->io_size, &rem); - *offset = par_buf_offset ? 0 : (unsigned int)rem; - - res = dm_bufio_read_with_ioprio(v->fec->bufio, block, buf, ioprio); - if (IS_ERR(res)) { - DMERR("%s: FEC %llu: parity read failed (block %llu): %ld", - v->data_dev->name, (unsigned long long)rsb, - (unsigned long long)block, PTR_ERR(res)); - *buf = NULL; - } - - return res; -} - /* Loop over each preallocated buffer slot. */ #define fec_for_each_prealloc_buffer(__i) \ for (__i = 0; __i < DM_VERITY_FEC_BUF_PREALLOC; __i++) @@ -116,15 +86,29 @@ static int fec_decode_bufs(struct dm_verity *v, struct dm_verity_io *io, { int r, corrected = 0, res; struct dm_buffer *buf; - unsigned int n, i, j, offset, par_buf_offset = 0; - uint16_t par_buf[DM_VERITY_FEC_RSM - DM_VERITY_FEC_MIN_RSN]; + unsigned int n, i, j, parity_pos, to_copy; + uint16_t par_buf[DM_VERITY_FEC_MAX_ROOTS]; u8 *par, *block; + u64 parity_block; struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); - par = fec_read_parity(v, rsb, block_offset, &offset, - par_buf_offset, &buf, bio->bi_ioprio); - if (IS_ERR(par)) + /* + * Compute the index of the first parity block that will be needed and + * the starting position in that block. Then read that block. + * + * io_size is always a power of 2, but roots might not be. Note that + * when it's not, a codeword's parity bytes can span a block boundary. + */ + parity_block = (rsb + block_offset) * v->fec->roots; + parity_pos = parity_block & (v->fec->io_size - 1); + parity_block >>= v->data_dev_block_bits; + par = dm_bufio_read_with_ioprio(v->fec->bufio, parity_block, &buf, + bio->bi_ioprio); + if (IS_ERR(par)) { + DMERR("%s: FEC %llu: parity read failed (block %llu): %ld", + v->data_dev->name, rsb, parity_block, PTR_ERR(par)); return PTR_ERR(par); + } /* * Decode the RS blocks we have in bufs. Each RS block results in @@ -132,8 +116,32 @@ static int fec_decode_bufs(struct dm_verity *v, struct dm_verity_io *io, */ fec_for_each_buffer_rs_block(fio, n, i) { block = fec_buffer_rs_block(v, fio, n, i); - for (j = 0; j < v->fec->roots - par_buf_offset; j++) - par_buf[par_buf_offset + j] = par[offset + j]; + + /* + * Copy the next 'roots' parity bytes to 'par_buf', reading + * another parity block if needed. + */ + to_copy = min(v->fec->io_size - parity_pos, v->fec->roots); + for (j = 0; j < to_copy; j++) + par_buf[j] = par[parity_pos++]; + if (to_copy < v->fec->roots) { + parity_block++; + parity_pos = 0; + + dm_bufio_release(buf); + par = dm_bufio_read_with_ioprio(v->fec->bufio, + parity_block, &buf, + bio->bi_ioprio); + if (IS_ERR(par)) { + DMERR("%s: FEC %llu: parity read failed (block %llu): %ld", + v->data_dev->name, rsb, parity_block, + PTR_ERR(par)); + return PTR_ERR(par); + } + for (; j < v->fec->roots; j++) + par_buf[j] = par[parity_pos++]; + } + /* Decode an RS block using Reed-Solomon */ res = decode_rs8(fio->rs, block, par_buf, v->fec->rsn, NULL, neras, fio->erasures, 0, NULL); @@ -148,26 +156,6 @@ static int fec_decode_bufs(struct dm_verity *v, struct dm_verity_io *io, block_offset++; if (block_offset >= 1 << v->data_dev_block_bits) goto done; - - /* Read the next block when we run out of parity bytes */ - offset += (v->fec->roots - par_buf_offset); - /* Check if parity bytes are split between blocks */ - if (offset < v->fec->io_size && (offset + v->fec->roots) > v->fec->io_size) { - par_buf_offset = v->fec->io_size - offset; - for (j = 0; j < par_buf_offset; j++) - par_buf[j] = par[offset + j]; - offset += par_buf_offset; - } else - par_buf_offset = 0; - - if (offset >= v->fec->io_size) { - dm_bufio_release(buf); - - par = fec_read_parity(v, rsb, block_offset, &offset, - par_buf_offset, &buf, bio->bi_ioprio); - if (IS_ERR(par)) - return PTR_ERR(par); - } } done: r = corrected; @@ -262,7 +250,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, (unsigned long long)block, PTR_ERR(bbuf)); /* assume the block is corrupted */ - if (neras && *neras <= v->fec->roots) + if (neras && *neras < v->fec->roots) fio->erasures[(*neras)++] = i; continue; @@ -280,7 +268,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, * skip if we have already found the theoretical * maximum number (i.e. fec->roots) of erasures */ - if (neras && *neras <= v->fec->roots && + if (neras && *neras < v->fec->roots && fec_is_erasure(v, io, want_digest, bbuf)) fio->erasures[(*neras)++] = i; } @@ -617,8 +605,8 @@ int verity_fec_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v, } else if (!strcasecmp(arg_name, DM_VERITY_OPT_FEC_ROOTS)) { if (sscanf(arg_value, "%hhu%c", &num_c, &dummy) != 1 || !num_c || - num_c < (DM_VERITY_FEC_RSM - DM_VERITY_FEC_MAX_RSN) || - num_c > (DM_VERITY_FEC_RSM - DM_VERITY_FEC_MIN_RSN)) { + num_c < DM_VERITY_FEC_MIN_ROOTS || + num_c > DM_VERITY_FEC_MAX_ROOTS) { ti->error = "Invalid " DM_VERITY_OPT_FEC_ROOTS; return -EINVAL; } diff --git a/drivers/md/dm-verity-fec.h b/drivers/md/dm-verity-fec.h index ec37e607cb3f..8552b5d3c915 100644 --- a/drivers/md/dm-verity-fec.h +++ b/drivers/md/dm-verity-fec.h @@ -13,8 +13,8 @@ /* Reed-Solomon(M, N) parameters */ #define DM_VERITY_FEC_RSM 255 -#define DM_VERITY_FEC_MAX_RSN 253 -#define DM_VERITY_FEC_MIN_RSN 231 /* ~10% space overhead */ +#define DM_VERITY_FEC_MIN_ROOTS 2 /* RS(255, 253): ~0.8% space overhead */ +#define DM_VERITY_FEC_MAX_ROOTS 24 /* RS(255, 231): ~10% space overhead */ /* buffers for deinterleaving and decoding */ #define DM_VERITY_FEC_BUF_PREALLOC 1 /* buffers to preallocate */ @@ -50,7 +50,7 @@ struct dm_verity_fec { /* per-bio data */ struct dm_verity_fec_io { struct rs_control *rs; /* Reed-Solomon state */ - int erasures[DM_VERITY_FEC_MAX_RSN]; /* erasures for decode_rs8 */ + int erasures[DM_VERITY_FEC_MAX_ROOTS]; /* erasures for decode_rs8 */ u8 *bufs[DM_VERITY_FEC_BUF_MAX]; /* bufs for deinterleaving */ unsigned int nbufs; /* number of buffers allocated */ u8 *output; /* buffer for corrected output */ diff --git a/drivers/md/dm.c b/drivers/md/dm.c index fb6bcd598c5b..28d825cc69c5 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -735,7 +735,16 @@ static struct table_device *open_table_device(struct mapped_device *md, return ERR_PTR(-ENOMEM); refcount_set(&td->count, 1); - bdev_file = bdev_file_open_by_dev(dev, mode, _dm_claim_ptr, NULL); + /* + * Open the backing device with kernel rather than caller + * credentials. Otherwise the caller's credentials would be + * pinned in bdev_file->f_cred until the table device is closed. + * That would keep the caller's thread keyring alive long beyond the + * lifetime of the caller, breaking userspace expectation (e.g. + * cryptsetup(8) leaking the LUKS volume key). + */ + scoped_with_kernel_creds() + bdev_file = bdev_file_open_by_dev(dev, mode, _dm_claim_ptr, NULL); if (IS_ERR(bdev_file)) { r = PTR_ERR(bdev_file); goto out_free_td; diff --git a/drivers/media/cec/platform/seco/seco-cec.c b/drivers/media/cec/platform/seco/seco-cec.c index b7bb49f02395..97ed9654c78a 100644 --- a/drivers/media/cec/platform/seco/seco-cec.c +++ b/drivers/media/cec/platform/seco/seco-cec.c @@ -649,7 +649,7 @@ static int secocec_probe(struct platform_device *pdev) ret = secocec_ir_probe(secocec); if (ret) - goto err_notifier; + goto err_unregister_adapter; platform_set_drvdata(pdev, secocec); @@ -657,6 +657,10 @@ static int secocec_probe(struct platform_device *pdev) return ret; +err_unregister_adapter: + cec_notifier_cec_adap_unregister(secocec->notifier, secocec->cec_adap); + cec_unregister_adapter(secocec->cec_adap); + goto err; err_notifier: cec_notifier_cec_adap_unregister(secocec->notifier, secocec->cec_adap); err_delete_adapter: diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c index 2df566f409b6..28313e284bb7 100644 --- a/drivers/media/common/videobuf2/videobuf2-core.c +++ b/drivers/media/common/videobuf2/videobuf2-core.c @@ -3006,8 +3006,8 @@ static int __vb2_cleanup_fileio(struct vb2_queue *q) * @nonblock: mode selector (1 means blocking calls, 0 means nonblocking) * @read: access mode selector (1 means read, 0 means write) */ -static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_t count, - loff_t *ppos, int nonblock, int read) +static ssize_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_t count, + loff_t *ppos, int nonblock, int read) { struct vb2_fileio_data *fileio; struct vb2_fileio_buf *buf; @@ -3170,15 +3170,15 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_ return ret; } -size_t vb2_read(struct vb2_queue *q, char __user *data, size_t count, - loff_t *ppos, int nonblocking) +ssize_t vb2_read(struct vb2_queue *q, char __user *data, size_t count, + loff_t *ppos, int nonblocking) { return __vb2_perform_fileio(q, data, count, ppos, nonblocking, 1); } EXPORT_SYMBOL_GPL(vb2_read); -size_t vb2_write(struct vb2_queue *q, const char __user *data, size_t count, - loff_t *ppos, int nonblocking) +ssize_t vb2_write(struct vb2_queue *q, const char __user *data, size_t count, + loff_t *ppos, int nonblocking) { return __vb2_perform_fileio(q, (char __user *) data, count, ppos, nonblocking, 0); diff --git a/drivers/media/dvb-frontends/rtl2832.c b/drivers/media/dvb-frontends/rtl2832.c index 3b4e46dac1bf..16f5f7e43092 100644 --- a/drivers/media/dvb-frontends/rtl2832.c +++ b/drivers/media/dvb-frontends/rtl2832.c @@ -1115,10 +1115,10 @@ static void rtl2832_remove(struct i2c_client *client) dev_dbg(&client->dev, "\n"); - cancel_delayed_work_sync(&dev->i2c_gate_work); - i2c_mux_del_adapters(dev->muxc); + cancel_delayed_work_sync(&dev->i2c_gate_work); + regmap_exit(dev->regmap); kfree(dev); diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.c b/drivers/media/dvb-frontends/rtl2832_sdr.c index 0357624968f1..787e2ef202a5 100644 --- a/drivers/media/dvb-frontends/rtl2832_sdr.c +++ b/drivers/media/dvb-frontends/rtl2832_sdr.c @@ -399,7 +399,8 @@ static int rtl2832_sdr_alloc_urbs(struct rtl2832_sdr_dev *dev) } /* Must be called with vb_queue_lock hold */ -static void rtl2832_sdr_cleanup_queued_bufs(struct rtl2832_sdr_dev *dev) +static void rtl2832_sdr_cleanup_queued_bufs(struct rtl2832_sdr_dev *dev, + enum vb2_buffer_state state) { struct platform_device *pdev = dev->pdev; unsigned long flags; @@ -413,7 +414,7 @@ static void rtl2832_sdr_cleanup_queued_bufs(struct rtl2832_sdr_dev *dev) buf = list_entry(dev->queued_bufs.next, struct rtl2832_sdr_frame_buf, list); list_del(&buf->list); - vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); + vb2_buffer_done(&buf->vb.vb2_buf, state); } spin_unlock_irqrestore(&dev->queued_bufs_lock, flags); } @@ -855,11 +856,15 @@ static int rtl2832_sdr_start_streaming(struct vb2_queue *vq, unsigned int count) dev_dbg(&pdev->dev, "\n"); - if (!dev->udev) + if (!dev->udev) { + rtl2832_sdr_cleanup_queued_bufs(dev, VB2_BUF_STATE_QUEUED); return -ENODEV; + } - if (mutex_lock_interruptible(&dev->v4l2_lock)) + if (mutex_lock_interruptible(&dev->v4l2_lock)) { + rtl2832_sdr_cleanup_queued_bufs(dev, VB2_BUF_STATE_QUEUED); return -ERESTARTSYS; + } if (d->props->power_ctrl) d->props->power_ctrl(d, 1); @@ -900,7 +905,11 @@ static int rtl2832_sdr_start_streaming(struct vb2_queue *vq, unsigned int count) if (ret) goto err; + mutex_unlock(&dev->v4l2_lock); + return 0; + err: + rtl2832_sdr_cleanup_queued_bufs(dev, VB2_BUF_STATE_QUEUED); mutex_unlock(&dev->v4l2_lock); return ret; @@ -920,7 +929,7 @@ static void rtl2832_sdr_stop_streaming(struct vb2_queue *vq) rtl2832_sdr_kill_urbs(dev); rtl2832_sdr_free_urbs(dev); rtl2832_sdr_free_stream_bufs(dev); - rtl2832_sdr_cleanup_queued_bufs(dev); + rtl2832_sdr_cleanup_queued_bufs(dev, VB2_BUF_STATE_ERROR); rtl2832_sdr_unset_adc(dev); /* sleep tuner */ diff --git a/drivers/media/i2c/alvium-csi2.c b/drivers/media/i2c/alvium-csi2.c index 1f088acecf36..8ea5e9951de0 100644 --- a/drivers/media/i2c/alvium-csi2.c +++ b/drivers/media/i2c/alvium-csi2.c @@ -2100,20 +2100,21 @@ static int alvium_ctrl_init(struct alvium_dev *alvium) V4L2_CID_PIXEL_RATE, 0, ALVIUM_DEFAULT_PIXEL_RATE_MHZ, 1, ALVIUM_DEFAULT_PIXEL_RATE_MHZ); - ctrls->pixel_rate->flags |= V4L2_CTRL_FLAG_READ_ONLY; /* Link freq is fixed */ ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, ops, V4L2_CID_LINK_FREQ, 0, 0, &alvium->link_freq); - ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY; + if (ctrls->link_freq) + ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY; /* Auto/manual white balance */ if (alvium->avail_ft.auto_whiteb) { ctrls->auto_wb = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTO_WHITE_BALANCE, 0, 1, 1, 1); - v4l2_ctrl_auto_cluster(3, &ctrls->auto_wb, 0, false); + if (ctrls->auto_wb) + v4l2_ctrl_auto_cluster(3, &ctrls->auto_wb, 0, false); } ctrls->blue_balance = v4l2_ctrl_new_std(hdl, ops, @@ -2122,6 +2123,7 @@ static int alvium_ctrl_init(struct alvium_dev *alvium) alvium->max_bbalance, alvium->inc_bbalance, alvium->dft_bbalance); + ctrls->red_balance = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_RED_BALANCE, alvium->min_rbalance, @@ -2136,7 +2138,9 @@ static int alvium_ctrl_init(struct alvium_dev *alvium) V4L2_CID_EXPOSURE_AUTO, V4L2_EXPOSURE_MANUAL, 0, V4L2_EXPOSURE_AUTO); - v4l2_ctrl_auto_cluster(2, &ctrls->auto_exp, 1, true); + if (ctrls->auto_exp) + v4l2_ctrl_auto_cluster(2, &ctrls->auto_exp, + V4L2_EXPOSURE_MANUAL, true); } ctrls->exposure = v4l2_ctrl_new_std(hdl, ops, @@ -2145,14 +2149,16 @@ static int alvium_ctrl_init(struct alvium_dev *alvium) alvium->max_exp, alvium->inc_exp, alvium->dft_exp); - ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE; + if (ctrls->exposure) + ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE; /* Auto/manual gain */ if (alvium->avail_ft.auto_gain) { ctrls->auto_gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTOGAIN, 0, 1, 1, 1); - v4l2_ctrl_auto_cluster(2, &ctrls->auto_gain, 0, true); + if (ctrls->auto_gain) + v4l2_ctrl_auto_cluster(2, &ctrls->auto_gain, 0, true); } if (alvium->avail_ft.gain) { @@ -2162,7 +2168,8 @@ static int alvium_ctrl_init(struct alvium_dev *alvium) alvium->max_gain, alvium->inc_gain, alvium->dft_gain); - ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE; + if (ctrls->gain) + ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE; } if (alvium->avail_ft.sat) diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c index 4734c7b4d37b..00aec91e3d49 100644 --- a/drivers/media/i2c/imx219.c +++ b/drivers/media/i2c/imx219.c @@ -71,7 +71,7 @@ /* V_TIMING internal */ #define IMX219_REG_FRM_LENGTH_A CCI_REG16(0x0160) -#define IMX219_FLL_MAX 0xffff +#define IMX219_FLL_MAX 0xfffe #define IMX219_VBLANK_MIN 32 #define IMX219_REG_LINE_LENGTH_A CCI_REG16(0x0162) #define IMX219_LLP_MIN 0x0d78 diff --git a/drivers/media/pci/cx23885/cx23885-core.c b/drivers/media/pci/cx23885/cx23885-core.c index a39f445ce22a..43d658a90f08 100644 --- a/drivers/media/pci/cx23885/cx23885-core.c +++ b/drivers/media/pci/cx23885/cx23885-core.c @@ -990,8 +990,12 @@ static int cx23885_dev_setup(struct cx23885_dev *dev) } /* PCIe stuff */ - dev->lmmio = ioremap(pci_resource_start(dev->pci, 0), - pci_resource_len(dev->pci, 0)); + dev->lmmio = pci_ioremap_bar(dev->pci, 0); + if (!dev->lmmio) { + dev_err(&dev->pci->dev, "CORE %s: can't ioremap MMIO memory\n", + dev->name); + goto err_release_region; + } dev->bmmio = (u8 __iomem *)dev->lmmio; @@ -1096,6 +1100,12 @@ static int cx23885_dev_setup(struct cx23885_dev *dev) } return 0; + +err_release_region: + release_mem_region(pci_resource_start(dev->pci, 0), + pci_resource_len(dev->pci, 0)); + cx23885_devcount--; + return -ENODEV; } static void cx23885_dev_unregister(struct cx23885_dev *dev) diff --git a/drivers/media/pci/dm1105/dm1105.c b/drivers/media/pci/dm1105/dm1105.c index e1185aa669f4..c1d55b983674 100644 --- a/drivers/media/pci/dm1105/dm1105.c +++ b/drivers/media/pci/dm1105/dm1105.c @@ -1194,6 +1194,7 @@ static void dm1105_remove(struct pci_dev *pdev) dm1105_hw_exit(dev); free_irq(pdev->irq, dev); + destroy_workqueue(dev->wq); pci_iounmap(pdev, dev->io_mem); pci_release_regions(pdev); pci_disable_device(pdev); diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-dwc-phy.c b/drivers/media/pci/intel/ipu6/ipu6-isys-dwc-phy.c index db2874843453..237906cb131a 100644 --- a/drivers/media/pci/intel/ipu6/ipu6-isys-dwc-phy.c +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-dwc-phy.c @@ -288,15 +288,27 @@ static const struct dwc_dphy_freq_range freqranges[DPHY_FREQ_RANGE_NUM] = { static u16 get_hsfreq_by_mbps(u32 mbps) { - unsigned int i = DPHY_FREQ_RANGE_NUM; - - while (i--) { - if (freqranges[i].default_mbps == mbps || - (mbps >= freqranges[i].min && mbps <= freqranges[i].max)) - return i; + u16 best = DPHY_FREQ_RANGE_INVALID_INDEX; + unsigned int i; + + for (i = 0; i < DPHY_FREQ_RANGE_NUM; i++) { + if (mbps > freqranges[i].max) + continue; + + if (mbps < freqranges[i].min) + break; + + if (best == DPHY_FREQ_RANGE_INVALID_INDEX || + freqranges[i].osc_freq_target > + freqranges[best].osc_freq_target || + (freqranges[i].osc_freq_target == + freqranges[best].osc_freq_target && + abs((int)mbps - (int)freqranges[i].default_mbps) < + abs((int)mbps - (int)freqranges[best].default_mbps))) + best = i; } - return DPHY_FREQ_RANGE_INVALID_INDEX; + return best; } static int ipu6_isys_dwc_phy_config(struct ipu6_isys *isys, diff --git a/drivers/media/pci/saa7134/saa7134-video.c b/drivers/media/pci/saa7134/saa7134-video.c index 4a51b873e47a..2b1672737d84 100644 --- a/drivers/media/pci/saa7134/saa7134-video.c +++ b/drivers/media/pci/saa7134/saa7134-video.c @@ -1714,8 +1714,10 @@ int saa7134_video_init1(struct saa7134_dev *dev) q->dev = &dev->pci->dev; ret = vb2_queue_init(q); if (ret) - return ret; - saa7134_pgtable_alloc(dev->pci, &dev->video_q.pt); + goto err_free_ctrl; + ret = saa7134_pgtable_alloc(dev->pci, &dev->video_q.pt); + if (ret) + goto err_free_ctrl; q = &dev->vbi_vbq; q->type = V4L2_BUF_TYPE_VBI_CAPTURE; @@ -1732,11 +1734,24 @@ int saa7134_video_init1(struct saa7134_dev *dev) q->lock = &dev->lock; q->dev = &dev->pci->dev; ret = vb2_queue_init(q); - if (ret) - return ret; - saa7134_pgtable_alloc(dev->pci, &dev->vbi_q.pt); + if (ret) { + saa7134_pgtable_free(dev->pci, &dev->video_q.pt); + goto err_free_ctrl; + } + + ret = saa7134_pgtable_alloc(dev->pci, &dev->vbi_q.pt); + if (ret) { + saa7134_pgtable_free(dev->pci, &dev->video_q.pt); + goto err_free_ctrl; + } return 0; + +err_free_ctrl: + v4l2_ctrl_handler_free(&dev->ctrl_handler); + if (card_has_radio(dev)) + v4l2_ctrl_handler_free(&dev->radio_ctrl_handler); + return ret; } void saa7134_video_fini(struct saa7134_dev *dev) diff --git a/drivers/media/platform/amlogic/c3/isp/c3-isp-params.c b/drivers/media/platform/amlogic/c3/isp/c3-isp-params.c index c80667dd7662..ec25299e52ab 100644 --- a/drivers/media/platform/amlogic/c3/isp/c3-isp-params.c +++ b/drivers/media/platform/amlogic/c3/isp/c3-isp-params.c @@ -107,6 +107,8 @@ static void c3_isp_params_awb_wt(struct c3_isp_device *isp, c3_isp_write(isp, ISP_AWB_BLK_WT_ADDR, 0); zones_num = cfg->horiz_zones_num * cfg->vert_zones_num; + if (zones_num > C3_ISP_AWB_MAX_ZONES) + zones_num = C3_ISP_AWB_MAX_ZONES; /* Need to write 8 weights at once */ for (i = 0; i < zones_num / 8; i++) { @@ -223,6 +225,8 @@ static void c3_isp_params_ae_wt(struct c3_isp_device *isp, c3_isp_write(isp, ISP_AE_BLK_WT_ADDR, 0); zones_num = cfg->horiz_zones_num * cfg->vert_zones_num; + if (zones_num > C3_ISP_AE_MAX_ZONES) + zones_num = C3_ISP_AE_MAX_ZONES; /* Need to write 8 weights at once */ for (i = 0; i < zones_num / 8; i++) { diff --git a/drivers/media/platform/aspeed/aspeed-video.c b/drivers/media/platform/aspeed/aspeed-video.c index b83e43245277..88ccd2f37469 100644 --- a/drivers/media/platform/aspeed/aspeed-video.c +++ b/drivers/media/platform/aspeed/aspeed-video.c @@ -2327,6 +2327,7 @@ static int aspeed_video_probe(struct platform_device *pdev) rc = aspeed_video_setup_video(video); if (rc) { aspeed_video_free_buf(video, &video->jpeg); + of_reserved_mem_device_release(&pdev->dev); clk_unprepare(video->vclk); clk_unprepare(video->eclk); return rc; diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c index 24fc0d0d3f4a..50cba21fdcdc 100644 --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c @@ -226,13 +226,6 @@ static int start_encode(struct vpu_instance *inst, u32 *fail_res) } else { dev_dbg(inst->dev->dev, "%s: wave5_vpu_enc_start_one_frame success\n", __func__); - /* - * Remove the source buffer from the ready-queue now and finish - * it in the videobuf2 framework once the index is returned by the - * firmware in finish_encode - */ - if (src_buf) - v4l2_m2m_src_buf_remove_by_idx(m2m_ctx, src_buf->vb2_buf.index); } return 0; @@ -259,27 +252,13 @@ static void wave5_vpu_enc_finish_encode(struct vpu_instance *inst) __func__, enc_output_info.pic_type, enc_output_info.recon_frame_index, enc_output_info.enc_src_idx, enc_output_info.enc_pic_byte, enc_output_info.pts); - /* - * The source buffer will not be found in the ready-queue as it has been - * dropped after sending of the encode firmware command, locate it in - * the videobuf2 queue directly - */ if (enc_output_info.enc_src_idx >= 0) { - struct vb2_buffer *vb = vb2_get_buffer(v4l2_m2m_get_src_vq(m2m_ctx), - enc_output_info.enc_src_idx); - if (vb->state != VB2_BUF_STATE_ACTIVE) - dev_warn(inst->dev->dev, - "%s: encoded buffer (%d) was not in ready queue %i.", - __func__, enc_output_info.enc_src_idx, vb->state); - else - src_buf = to_vb2_v4l2_buffer(vb); - - if (src_buf) { + src_buf = v4l2_m2m_src_buf_remove_by_idx(m2m_ctx, enc_output_info.enc_src_idx); + if (!src_buf) { + dev_warn(inst->dev->dev, "%s: no source buffer found\n", __func__); + } else { inst->timestamp = src_buf->vb2_buf.timestamp; v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE); - } else { - dev_warn(inst->dev->dev, "%s: no source buffer with index: %d found\n", - __func__, enc_output_info.enc_src_idx); } } diff --git a/drivers/media/platform/marvell/cafe-driver.c b/drivers/media/platform/marvell/cafe-driver.c index f9796de92aa7..39967ec5e7ee 100644 --- a/drivers/media/platform/marvell/cafe-driver.c +++ b/drivers/media/platform/marvell/cafe-driver.c @@ -609,6 +609,7 @@ static void cafe_pci_remove(struct pci_dev *pdev) return; } cafe_shutdown(cam); + pci_disable_device(pdev); kfree(cam); } diff --git a/drivers/media/platform/nuvoton/npcm-video.c b/drivers/media/platform/nuvoton/npcm-video.c index 44e904e61801..c4c5c318f786 100644 --- a/drivers/media/platform/nuvoton/npcm-video.c +++ b/drivers/media/platform/nuvoton/npcm-video.c @@ -1720,10 +1720,12 @@ static int npcm_video_init(struct npcm_video *video) if (rc) { dev_err(dev, "Failed to set DMA mask\n"); of_reserved_mem_device_release(dev); + return rc; } rc = npcm_video_ece_init(video); if (rc) { + of_reserved_mem_device_release(dev); dev_err(dev, "Failed to initialize ECE\n"); return rc; } @@ -1748,42 +1750,55 @@ static int npcm_video_probe(struct platform_device *pdev) regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(regs)) { dev_err(&pdev->dev, "Failed to parse VCD reg in DTS\n"); - return PTR_ERR(regs); + rc = PTR_ERR(regs); + goto err_free; } video->vcd_regmap = devm_regmap_init_mmio(&pdev->dev, regs, &npcm_video_regmap_cfg); if (IS_ERR(video->vcd_regmap)) { dev_err(&pdev->dev, "Failed to initialize VCD regmap\n"); - return PTR_ERR(video->vcd_regmap); + rc = PTR_ERR(video->vcd_regmap); + goto err_free; } video->reset = devm_reset_control_get(&pdev->dev, NULL); if (IS_ERR(video->reset)) { dev_err(&pdev->dev, "Failed to get VCD reset control in DTS\n"); - return PTR_ERR(video->reset); + rc = PTR_ERR(video->reset); + goto err_free; } video->gcr_regmap = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, "nuvoton,sysgcr"); - if (IS_ERR(video->gcr_regmap)) - return PTR_ERR(video->gcr_regmap); + if (IS_ERR(video->gcr_regmap)) { + rc = PTR_ERR(video->gcr_regmap); + goto err_free; + } video->gfx_regmap = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, "nuvoton,sysgfxi"); - if (IS_ERR(video->gfx_regmap)) - return PTR_ERR(video->gfx_regmap); + if (IS_ERR(video->gfx_regmap)) { + rc = PTR_ERR(video->gfx_regmap); + goto err_free; + } rc = npcm_video_init(video); if (rc) - return rc; + goto err_free; rc = npcm_video_setup_video(video); if (rc) - return rc; + goto err_release_mem; dev_info(video->dev, "NPCM video driver probed\n"); return 0; + +err_release_mem: + of_reserved_mem_device_release(&pdev->dev); +err_free: + kfree(video); + return rc; } static void npcm_video_remove(struct platform_device *pdev) @@ -1798,6 +1813,7 @@ static void npcm_video_remove(struct platform_device *pdev) v4l2_device_unregister(v4l2_dev); if (video->ece.enable) npcm_video_ece_stop(video); + kfree(video); of_reserved_mem_device_release(dev); } diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c index 60fed2fe200d..a6ed1d124472 100644 --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.c @@ -517,6 +517,8 @@ static int mxc_isi_probe(struct platform_device *pdev) return 0; err_xbar: + while (i--) + mxc_isi_pipe_cleanup(&isi->pipes[i]); mxc_isi_crossbar_cleanup(&isi->crossbar); return ret; diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h b/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h index e84af5127e4e..681a05045cd2 100644 --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h @@ -11,6 +11,7 @@ #define __MXC_ISI_CORE_H__ #include +#include #include #include #include @@ -412,4 +413,19 @@ static inline void mxc_isi_debug_cleanup(struct mxc_isi_dev *isi) } #endif +/* + * ISI scaling engine works in two parts: it performs pre-decimation of + * the image followed by bilinear filtering to achieve the desired + * downscaling factor. + * + * The decimation filter provides a maximum downscaling factor of 8, and + * the subsequent bilinear filter provides a maximum downscaling factor + * of 2. Combined, the maximum scaling factor can be up to 16. + */ +static inline unsigned int +mxc_isi_clamp_downscale_16(unsigned int val, unsigned int max_val) +{ + return clamp(val, max(1U, DIV_ROUND_UP(max_val, 16)), max_val); +} + #endif /* __MXC_ISI_CORE_H__ */ diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c index ede6cc74c023..3562c9f5ae2e 100644 --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c @@ -492,6 +492,7 @@ int mxc_isi_crossbar_init(struct mxc_isi_dev *isi) void mxc_isi_crossbar_cleanup(struct mxc_isi_crossbar *xbar) { + v4l2_subdev_cleanup(&xbar->sd); media_entity_cleanup(&xbar->sd.entity); kfree(xbar->pads); kfree(xbar->inputs); diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-hw.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-hw.c index 9225a7ac1c3e..0bfd9b8f5f10 100644 --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-hw.c +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-hw.c @@ -112,7 +112,14 @@ static u32 mxc_isi_channel_scaling_ratio(unsigned int from, unsigned int to, else *dec = 8; - return min_t(u32, from * 0x1000 / (to * *dec), ISI_DOWNSCALE_THRESHOLD); + /* + * The ISI rounds output dimensions up to the next integer (i.MX93 RM + * section 57.7.8). Calculate the scale factor such that the theoretical + * output (input / scale_factor) rounds up to exactly the desired + * output. + */ + return min_t(u32, DIV_ROUND_UP(from * 0x1000, to * *dec), + ISI_DOWNSCALE_THRESHOLD); } static void mxc_isi_channel_set_scaling(struct mxc_isi_pipe *pipe, diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-m2m.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-m2m.c index 00afcbfbdde4..f2a73f0f1ac7 100644 --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-m2m.c +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-m2m.c @@ -509,9 +509,14 @@ __mxc_isi_m2m_try_fmt_vid(struct mxc_isi_m2m_ctx *ctx, const enum mxc_isi_video_type type) { if (type == MXC_ISI_VIDEO_M2M_CAP) { - /* Downscaling only */ - pix->width = min(pix->width, ctx->queues.out.format.width); - pix->height = min(pix->height, ctx->queues.out.format.height); + const struct v4l2_pix_format_mplane *format = + &ctx->queues.out.format; + + /* Downscaling only, by up to 16. */ + pix->width = mxc_isi_clamp_downscale_16(pix->width, + format->width); + pix->height = mxc_isi_clamp_downscale_16(pix->height, + format->height); } return mxc_isi_format_try(ctx->m2m->pipe, pix, type); diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c index a41c51dd9ce0..2d0843c86534 100644 --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c @@ -641,16 +641,19 @@ static int mxc_isi_pipe_set_selection(struct v4l2_subdev *sd, /* Composing is supported on the sink only. */ return -EINVAL; - /* The sink crop is bound by the sink format downscaling only). */ + /* + * The ISI supports downscaling only, with a factor up to 16. + * Clamp the compose rectangle size accordingly. + */ format = mxc_isi_pipe_get_pad_format(pipe, state, MXC_ISI_PIPE_PAD_SINK); sel->r.left = 0; sel->r.top = 0; - sel->r.width = clamp(sel->r.width, MXC_ISI_MIN_WIDTH, - format->width); - sel->r.height = clamp(sel->r.height, MXC_ISI_MIN_HEIGHT, - format->height); + sel->r.width = mxc_isi_clamp_downscale_16(sel->r.width, + format->width); + sel->r.height = mxc_isi_clamp_downscale_16(sel->r.height, + format->height); rect = mxc_isi_pipe_get_pad_compose(pipe, state, MXC_ISI_PIPE_PAD_SINK); @@ -796,18 +799,20 @@ int mxc_isi_pipe_init(struct mxc_isi_dev *isi, unsigned int id) irq = platform_get_irq(to_platform_device(isi->dev), id); if (irq < 0) { ret = irq; - goto error; + goto error_subdev; } ret = devm_request_irq(isi->dev, irq, mxc_isi_pipe_irq_handler, 0, dev_name(isi->dev), pipe); if (ret < 0) { dev_err(isi->dev, "failed to request IRQ (%d)\n", ret); - goto error; + goto error_subdev; } return 0; +error_subdev: + v4l2_subdev_cleanup(sd); error: media_entity_cleanup(&sd->entity); mutex_destroy(&pipe->lock); @@ -819,6 +824,7 @@ void mxc_isi_pipe_cleanup(struct mxc_isi_pipe *pipe) { struct v4l2_subdev *sd = &pipe->sd; + v4l2_subdev_cleanup(sd); media_entity_cleanup(&sd->entity); mutex_destroy(&pipe->lock); } diff --git a/drivers/media/platform/qcom/camss/camss-csid-680.c b/drivers/media/platform/qcom/camss/camss-csid-680.c index 3ad3a174bcfb..edf01ba79907 100644 --- a/drivers/media/platform/qcom/camss/camss-csid-680.c +++ b/drivers/media/platform/qcom/camss/camss-csid-680.c @@ -219,9 +219,9 @@ static void __csid_configure_top(struct csid_device *csid) CSID_TOP_IO_PATH_CFG0(csid->id)); } -static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 vc) +static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 port, u8 vc) { - struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + vc]; + struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + port]; const struct csid_format_info *format = csid_get_fmt_entry(csid->res->formats->formats, csid->res->formats->nformats, input_format->code); @@ -233,28 +233,28 @@ static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 lane_cnt = 4; val = 0; - writel(val, csid->base + CSID_RDI_FRM_DROP_PERIOD(vc)); + writel(val, csid->base + CSID_RDI_FRM_DROP_PERIOD(port)); /* * DT_ID is a two bit bitfield that is concatenated with * the four least significant bits of the five bit VC * bitfield to generate an internal CID value. * - * CSID_RDI_CFG0(vc) + * CSID_RDI_CFG0(port) * DT_ID : 28:27 * VC : 26:22 * DT : 21:16 * * CID : VC 3:0 << 2 | DT_ID 1:0 */ - dt_id = vc & 0x03; + dt_id = port & 0x03; /* note: for non-RDI path, this should be format->decode_format */ val |= DECODE_FORMAT_PAYLOAD_ONLY << RDI_CFG0_DECODE_FORMAT; val |= format->data_type << RDI_CFG0_DATA_TYPE; val |= vc << RDI_CFG0_VIRTUAL_CHANNEL; val |= dt_id << RDI_CFG0_DT_ID; - writel(val, csid->base + CSID_RDI_CFG0(vc)); + writel(val, csid->base + CSID_RDI_CFG0(port)); val = RDI_CFG1_TIMESTAMP_STB_FRAME; val |= RDI_CFG1_BYTE_CNTR_EN; @@ -265,23 +265,23 @@ static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 val |= RDI_CFG1_CROP_V_EN; val |= RDI_CFG1_PACKING_MIPI; - writel(val, csid->base + CSID_RDI_CFG1(vc)); + writel(val, csid->base + CSID_RDI_CFG1(port)); val = 0; - writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PERIOD(vc)); + writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PERIOD(port)); val = 1; - writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PATTERN(vc)); + writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PATTERN(port)); val = 0; - writel(val, csid->base + CSID_RDI_CTRL(vc)); + writel(val, csid->base + CSID_RDI_CTRL(port)); - val = readl(csid->base + CSID_RDI_CFG0(vc)); + val = readl(csid->base + CSID_RDI_CFG0(port)); if (enable) val |= RDI_CFG0_ENABLE; else val &= ~RDI_CFG0_ENABLE; - writel(val, csid->base + CSID_RDI_CFG0(vc)); + writel(val, csid->base + CSID_RDI_CFG0(port)); } static void csid_configure_stream(struct csid_device *csid, u8 enable) @@ -290,11 +290,11 @@ static void csid_configure_stream(struct csid_device *csid, u8 enable) __csid_configure_top(csid); - /* Loop through all enabled VCs and configure stream for each */ + /* Loop through all enabled ports and configure a stream for each */ for (i = 0; i < MSM_CSID_MAX_SRC_STREAMS; i++) { if (csid->phy.en_vc & BIT(i)) { - __csid_configure_rdi_stream(csid, enable, i); - __csid_configure_rx(csid, &csid->phy, i); + __csid_configure_rdi_stream(csid, enable, i, 0); + __csid_configure_rx(csid, &csid->phy, 0); __csid_ctrl_rdi(csid, enable, i); } } diff --git a/drivers/media/platform/qcom/camss/camss-csid-gen2.c b/drivers/media/platform/qcom/camss/camss-csid-gen2.c index 2a1746dcc1c5..eadcb2f7e3aa 100644 --- a/drivers/media/platform/qcom/camss/camss-csid-gen2.c +++ b/drivers/media/platform/qcom/camss/camss-csid-gen2.c @@ -203,10 +203,10 @@ static void __csid_ctrl_rdi(struct csid_device *csid, int enable, u8 rdi) writel_relaxed(val, csid->base + CSID_RDI_CTRL(rdi)); } -static void __csid_configure_testgen(struct csid_device *csid, u8 enable, u8 vc) +static void __csid_configure_testgen(struct csid_device *csid, u8 enable, u8 port, u8 vc) { struct csid_testgen_config *tg = &csid->testgen; - struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + vc]; + struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + port]; const struct csid_format_info *format = csid_get_fmt_entry(csid->res->formats->formats, csid->res->formats->nformats, input_format->code); @@ -253,10 +253,10 @@ static void __csid_configure_testgen(struct csid_device *csid, u8 enable, u8 vc) writel_relaxed(val, csid->base + CSID_TPG_CTRL); } -static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 vc) +static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 port, u8 vc) { /* Source pads matching RDI channels on hardware. Pad 1 -> RDI0, Pad 2 -> RDI1, etc. */ - struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + vc]; + struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + port]; const struct csid_format_info *format = csid_get_fmt_entry(csid->res->formats->formats, csid->res->formats->nformats, input_format->code); @@ -267,14 +267,14 @@ static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 * the four least significant bits of the five bit VC * bitfield to generate an internal CID value. * - * CSID_RDI_CFG0(vc) + * CSID_RDI_CFG0(port) * DT_ID : 28:27 * VC : 26:22 * DT : 21:16 * * CID : VC 3:0 << 2 | DT_ID 1:0 */ - u8 dt_id = vc & 0x03; + u8 dt_id = port & 0x03; val = 1 << RDI_CFG0_BYTE_CNTR_EN; val |= 1 << RDI_CFG0_FORMAT_MEASURE_EN; @@ -284,56 +284,57 @@ static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 val |= format->data_type << RDI_CFG0_DATA_TYPE; val |= vc << RDI_CFG0_VIRTUAL_CHANNEL; val |= dt_id << RDI_CFG0_DT_ID; - writel_relaxed(val, csid->base + CSID_RDI_CFG0(vc)); + writel_relaxed(val, csid->base + CSID_RDI_CFG0(port)); /* CSID_TIMESTAMP_STB_POST_IRQ */ val = 2 << RDI_CFG1_TIMESTAMP_STB_SEL; - writel_relaxed(val, csid->base + CSID_RDI_CFG1(vc)); + writel_relaxed(val, csid->base + CSID_RDI_CFG1(port)); val = 1; - writel_relaxed(val, csid->base + CSID_RDI_FRM_DROP_PERIOD(vc)); + writel_relaxed(val, csid->base + CSID_RDI_FRM_DROP_PERIOD(port)); val = 0; - writel_relaxed(val, csid->base + CSID_RDI_FRM_DROP_PATTERN(vc)); + writel_relaxed(val, csid->base + CSID_RDI_FRM_DROP_PATTERN(port)); val = 1; - writel_relaxed(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PERIOD(vc)); + writel_relaxed(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PERIOD(port)); val = 0; - writel_relaxed(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PATTERN(vc)); + writel_relaxed(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PATTERN(port)); val = 1; - writel_relaxed(val, csid->base + CSID_RDI_RPP_PIX_DROP_PERIOD(vc)); + writel_relaxed(val, csid->base + CSID_RDI_RPP_PIX_DROP_PERIOD(port)); val = 0; - writel_relaxed(val, csid->base + CSID_RDI_RPP_PIX_DROP_PATTERN(vc)); + writel_relaxed(val, csid->base + CSID_RDI_RPP_PIX_DROP_PATTERN(port)); val = 1; - writel_relaxed(val, csid->base + CSID_RDI_RPP_LINE_DROP_PERIOD(vc)); + writel_relaxed(val, csid->base + CSID_RDI_RPP_LINE_DROP_PERIOD(port)); val = 0; - writel_relaxed(val, csid->base + CSID_RDI_RPP_LINE_DROP_PATTERN(vc)); + writel_relaxed(val, csid->base + CSID_RDI_RPP_LINE_DROP_PATTERN(port)); val = 0; - writel_relaxed(val, csid->base + CSID_RDI_CTRL(vc)); + writel_relaxed(val, csid->base + CSID_RDI_CTRL(port)); - val = readl_relaxed(csid->base + CSID_RDI_CFG0(vc)); + val = readl_relaxed(csid->base + CSID_RDI_CFG0(port)); val |= enable << RDI_CFG0_ENABLE; - writel_relaxed(val, csid->base + CSID_RDI_CFG0(vc)); + writel_relaxed(val, csid->base + CSID_RDI_CFG0(port)); } static void csid_configure_stream(struct csid_device *csid, u8 enable) { struct csid_testgen_config *tg = &csid->testgen; u8 i; - /* Loop through all enabled VCs and configure stream for each */ + + /* Loop through all enabled ports and configure a stream for each */ for (i = 0; i < MSM_CSID_MAX_SRC_STREAMS; i++) if (csid->phy.en_vc & BIT(i)) { if (tg->enabled) - __csid_configure_testgen(csid, enable, i); + __csid_configure_testgen(csid, enable, i, 0); - __csid_configure_rdi_stream(csid, enable, i); - __csid_configure_rx(csid, &csid->phy, i); + __csid_configure_rdi_stream(csid, enable, i, 0); + __csid_configure_rx(csid, &csid->phy, 0); __csid_ctrl_rdi(csid, enable, i); } } diff --git a/drivers/media/platform/qcom/camss/camss-csid-gen3.c b/drivers/media/platform/qcom/camss/camss-csid-gen3.c index bd059243790e..ed5c5766efd3 100644 --- a/drivers/media/platform/qcom/camss/camss-csid-gen3.c +++ b/drivers/media/platform/qcom/camss/camss-csid-gen3.c @@ -145,12 +145,12 @@ static void __csid_configure_wrapper(struct csid_device *csid) writel(val, csid->camss->csid_wrapper_base + CSID_IO_PATH_CFG0(csid->id)); } -static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 vc) +static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 port, u8 vc) { u32 val; u8 lane_cnt = csid->phy.lane_cnt; /* Source pads matching RDI channels on hardware. Pad 1 -> RDI0, Pad 2 -> RDI1, etc. */ - struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + vc]; + struct v4l2_mbus_framefmt *input_format = &csid->fmt[MSM_CSID_PAD_FIRST_SRC + port]; const struct csid_format_info *format = csid_get_fmt_entry(csid->res->formats->formats, csid->res->formats->nformats, input_format->code); @@ -163,14 +163,14 @@ static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 * the four least significant bits of the five bit VC * bitfield to generate an internal CID value. * - * CSID_RDI_CFG0(vc) + * CSID_RDI_CFG0(port) * DT_ID : 28:27 * VC : 26:22 * DT : 21:16 * * CID : VC 3:0 << 2 | DT_ID 1:0 */ - u8 dt_id = vc & 0x03; + u8 dt_id = port & 0x03; val = RDI_CFG0_TIMESTAMP_EN; val |= RDI_CFG0_TIMESTAMP_STB_SEL; @@ -180,7 +180,7 @@ static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 val |= format->data_type << RDI_CFG0_DT; val |= dt_id << RDI_CFG0_DT_ID; - writel(val, csid->base + CSID_RDI_CFG0(vc)); + writel(val, csid->base + CSID_RDI_CFG0(port)); val = RDI_CFG1_PACKING_FORMAT_MIPI; val |= RDI_CFG1_PIX_STORE; @@ -189,22 +189,22 @@ static void __csid_configure_rdi_stream(struct csid_device *csid, u8 enable, u8 val |= RDI_CFG1_CROP_H_EN; val |= RDI_CFG1_CROP_V_EN; - writel(val, csid->base + CSID_RDI_CFG1(vc)); + writel(val, csid->base + CSID_RDI_CFG1(port)); val = 0; - writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PERIOD(vc)); + writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PERIOD(port)); val = 1; - writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PATTERN(vc)); + writel(val, csid->base + CSID_RDI_IRQ_SUBSAMPLE_PATTERN(port)); val = 0; - writel(val, csid->base + CSID_RDI_CTRL(vc)); + writel(val, csid->base + CSID_RDI_CTRL(port)); - val = readl(csid->base + CSID_RDI_CFG0(vc)); + val = readl(csid->base + CSID_RDI_CFG0(port)); if (enable) val |= RDI_CFG0_EN; - writel(val, csid->base + CSID_RDI_CFG0(vc)); + writel(val, csid->base + CSID_RDI_CFG0(port)); } static void csid_configure_stream(struct csid_device *csid, u8 enable) @@ -213,11 +213,11 @@ static void csid_configure_stream(struct csid_device *csid, u8 enable) __csid_configure_wrapper(csid); - /* Loop through all enabled VCs and configure stream for each */ + /* Loop through all enabled ports and configure a stream for each */ for (i = 0; i < MSM_CSID_MAX_SRC_STREAMS; i++) if (csid->phy.en_vc & BIT(i)) { - __csid_configure_rdi_stream(csid, enable, i); - __csid_configure_rx(csid, &csid->phy, i); + __csid_configure_rdi_stream(csid, enable, i, 0); + __csid_configure_rx(csid, &csid->phy, 0); __csid_ctrl_rdi(csid, enable, i); } } diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index a197833bac61..cdeb78ba605a 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -256,12 +256,12 @@ static int iris_probe(struct platform_device *pdev) core->iris_platform_data = of_device_get_match_data(core->dev); ret = devm_request_threaded_irq(core->dev, core->irq, iris_hfi_isr, - iris_hfi_isr_handler, IRQF_TRIGGER_HIGH, "iris", core); + iris_hfi_isr_handler, + IRQF_TRIGGER_HIGH | IRQF_NO_AUTOEN, + "iris", core); if (ret) return ret; - disable_irq_nosync(core->irq); - iris_init_ops(core); core->iris_platform_data->init_hfi_command_ops(core); core->iris_platform_data->init_hfi_response_ops(core); diff --git a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-cru-regs.h b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-cru-regs.h index a5a57369ef0e..10e62f2646d0 100644 --- a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-cru-regs.h +++ b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-cru-regs.h @@ -60,6 +60,7 @@ #define ICnMC_CSCTHR BIT(5) #define ICnMC_INF(x) ((x) << 16) #define ICnMC_VCSEL(x) ((x) << 22) +#define ICnMC_VCSEL_MASK GENMASK(23, 22) #define ICnMC_INF_MASK GENMASK(21, 16) #define ICnMS_IA BIT(2) diff --git a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-video.c b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-video.c index 162e2ace6931..6aea7c244df1 100644 --- a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-video.c +++ b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-video.c @@ -262,19 +262,24 @@ static void rzg2l_cru_csi2_setup(struct rzg2l_cru_dev *cru, u8 csi_vc) { const struct rzg2l_cru_info *info = cru->info; - u32 icnmc = ICnMC_INF(ip_fmt->datatype); + u32 icnmc = rzg2l_cru_read(cru, info->image_conv) & ~(ICnMC_INF_MASK | + ICnMC_VCSEL_MASK); + icnmc |= ICnMC_INF(ip_fmt->datatype); + /* + * VC filtering goes through SVC register on G3E/V2H. + * + * FIXME: virtual channel filtering is likely broken and only VC=0 + * works. + */ if (cru->info->regs[ICnSVC]) { rzg2l_cru_write(cru, ICnSVCNUM, csi_vc); rzg2l_cru_write(cru, ICnSVC, ICnSVC_SVC0(0) | ICnSVC_SVC1(1) | ICnSVC_SVC2(2) | ICnSVC_SVC3(3)); + } else { + icnmc |= ICnMC_VCSEL(csi_vc); } - icnmc |= rzg2l_cru_read(cru, info->image_conv) & ~ICnMC_INF_MASK; - - /* Set virtual channel CSI2 */ - icnmc |= ICnMC_VCSEL(csi_vc); - rzg2l_cru_write(cru, info->image_conv, icnmc); } diff --git a/drivers/media/platform/st/stm32/stm32-dcmi.c b/drivers/media/platform/st/stm32/stm32-dcmi.c index 13762861b769..200b49812791 100644 --- a/drivers/media/platform/st/stm32/stm32-dcmi.c +++ b/drivers/media/platform/st/stm32/stm32-dcmi.c @@ -2063,6 +2063,7 @@ static int dcmi_probe(struct platform_device *pdev) return 0; err_cleanup: + v4l2_async_nf_unregister(&dcmi->notifier); v4l2_async_nf_cleanup(&dcmi->notifier); err_media_entity_cleanup: media_entity_cleanup(&dcmi->vdev->entity); diff --git a/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-bytecap.c b/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-bytecap.c index b18e273ef4a3..e9cab99775bc 100644 --- a/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-bytecap.c +++ b/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-bytecap.c @@ -398,8 +398,10 @@ static int dcmipp_bytecap_start_streaming(struct vb2_queue *vq, */ if (!vcap->s_subdev) { pad = media_pad_remote_pad_first(&vcap->vdev.entity.pads[0]); - if (!pad || !is_media_entity_v4l2_subdev(pad->entity)) - return -EINVAL; + if (!pad || !is_media_entity_v4l2_subdev(pad->entity)) { + ret = -EINVAL; + goto err_buffer_done; + } vcap->s_subdev = media_entity_to_v4l2_subdev(pad->entity); vcap->s_subdev_pad_nb = pad->index; } diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c index e911c7f7acc5..4781db21c205 100644 --- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c +++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_dma.c @@ -234,8 +234,10 @@ static int sun4i_csi_start_streaming(struct vb2_queue *vq, unsigned int count) int ret; csi_fmt = sun4i_csi_find_format(&csi->fmt.pixelformat, NULL); - if (!csi_fmt) - return -EINVAL; + if (!csi_fmt) { + ret = -EINVAL; + goto err_clear_dma_queue; + } dev_dbg(csi->dev, "Starting capture\n"); diff --git a/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c b/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c index b7d278b3889f..8e6f452f76c7 100644 --- a/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c +++ b/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c @@ -504,9 +504,9 @@ static void hdmirx_hpd_ctrl(struct snps_hdmirx_dev *hdmirx_dev, bool en) hdmirx_writel(hdmirx_dev, CORE_CONFIG, hdmirx_dev->hpd_trigger_level_high ? en : !en); - /* 100ms delay as per HDMI spec */ + /* 100ms delay as per HDMI spec + extra 50ms to cover internal delay */ if (!en) - msleep(100); + msleep(100 + 50); } static void hdmirx_write_edid_data(struct snps_hdmirx_dev *hdmirx_dev, diff --git a/drivers/media/platform/ti/davinci/vpif_capture.c b/drivers/media/platform/ti/davinci/vpif_capture.c index 243c6196b024..9fbef2402574 100644 --- a/drivers/media/platform/ti/davinci/vpif_capture.c +++ b/drivers/media/platform/ti/davinci/vpif_capture.c @@ -1499,7 +1499,7 @@ vpif_capture_get_pdata(struct platform_device *pdev, * video ports & endpoints data. */ if (pdev->dev.parent && pdev->dev.parent->of_node) - pdev->dev.of_node = pdev->dev.parent->of_node; + device_set_of_node_from_dev(&pdev->dev, pdev->dev.parent); if (!IS_ENABLED(CONFIG_OF) || !pdev->dev.of_node) return pdev->dev.platform_data; diff --git a/drivers/media/platform/ti/vpe/vpe.c b/drivers/media/platform/ti/vpe/vpe.c index 6029d4e8e0bd..e6c3aee946c1 100644 --- a/drivers/media/platform/ti/vpe/vpe.c +++ b/drivers/media/platform/ti/vpe/vpe.c @@ -2546,7 +2546,8 @@ static int vpe_probe(struct platform_device *pdev) "vpe_top"); if (!dev->res) { dev_err(&pdev->dev, "missing 'vpe_top' resources data\n"); - return -ENODEV; + ret = -ENODEV; + goto v4l2_dev_unreg; } /* diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c index fcf3bd9bcda2..83af9fa1ce94 100644 --- a/drivers/media/platform/verisilicon/hantro_v4l2.c +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c @@ -222,6 +222,7 @@ static int vidioc_enum_fmt(struct file *file, void *priv, unsigned int num_fmts, i, j = 0; bool skip_mode_none, enum_all_formats; u32 index = f->index & ~V4L2_FMTDESC_FLAG_ENUM_ALL; + bool need_postproc = ctx->need_postproc; /* * If the V4L2_FMTDESC_FLAG_ENUM_ALL flag is set, we want to enumerate all @@ -230,6 +231,9 @@ static int vidioc_enum_fmt(struct file *file, void *priv, enum_all_formats = !!(f->index & V4L2_FMTDESC_FLAG_ENUM_ALL); f->index = index; + if (enum_all_formats) + need_postproc = HANTRO_AUTO_POSTPROC; + /* * When dealing with an encoder: * - on the capture side we want to filter out all MODE_NONE formats. @@ -242,7 +246,7 @@ static int vidioc_enum_fmt(struct file *file, void *priv, */ skip_mode_none = capture == ctx->is_encoder; - formats = hantro_get_formats(ctx, &num_fmts, HANTRO_AUTO_POSTPROC); + formats = hantro_get_formats(ctx, &num_fmts, need_postproc); for (i = 0; i < num_fmts; i++) { bool mode_none = formats[i].codec_mode == HANTRO_MODE_NONE; fmt = &formats[i]; diff --git a/drivers/media/radio/radio-si476x.c b/drivers/media/radio/radio-si476x.c index 9980346cb5ea..bfe89782dce4 100644 --- a/drivers/media/radio/radio-si476x.c +++ b/drivers/media/radio/radio-si476x.c @@ -1493,6 +1493,7 @@ static int si476x_radio_probe(struct platform_device *pdev) return 0; exit: v4l2_ctrl_handler_free(radio->videodev.ctrl_handler); + v4l2_device_unregister(&radio->v4l2dev); return rval; } diff --git a/drivers/media/test-drivers/vidtv/vidtv_bridge.c b/drivers/media/test-drivers/vidtv/vidtv_bridge.c index 52b2abe16dcf..8a8e097d2ac9 100644 --- a/drivers/media/test-drivers/vidtv/vidtv_bridge.c +++ b/drivers/media/test-drivers/vidtv/vidtv_bridge.c @@ -594,8 +594,10 @@ static int __init vidtv_bridge_init(void) int ret; ret = platform_device_register(&vidtv_bridge_dev); - if (ret) + if (ret) { + platform_device_put(&vidtv_bridge_dev); return ret; + } ret = platform_driver_register(&vidtv_bridge_driver); if (ret) diff --git a/drivers/media/test-drivers/vimc/vimc-core.c b/drivers/media/test-drivers/vimc/vimc-core.c index f632c77e52f5..bc0474d8059a 100644 --- a/drivers/media/test-drivers/vimc/vimc-core.c +++ b/drivers/media/test-drivers/vimc/vimc-core.c @@ -422,6 +422,7 @@ static int __init vimc_init(void) if (ret) { dev_err(&vimc_pdev.dev, "platform device registration failed (err=%d)\n", ret); + platform_device_put(&vimc_pdev); return ret; } diff --git a/drivers/media/test-drivers/vivid/vivid-core.c b/drivers/media/test-drivers/vivid/vivid-core.c index 86506be36acb..f44930502bda 100644 --- a/drivers/media/test-drivers/vivid/vivid-core.c +++ b/drivers/media/test-drivers/vivid/vivid-core.c @@ -2289,8 +2289,10 @@ static int __init vivid_init(void) } } ret = platform_device_register(&vivid_pdev); - if (ret) + if (ret) { + platform_device_put(&vivid_pdev); goto free_output_strings; + } ret = platform_driver_register(&vivid_pdrv); if (ret) goto unreg_device; @@ -2311,7 +2313,7 @@ static int __init vivid_init(void) destroy_hdmi_wq: destroy_workqueue(update_hdmi_ctrls_workqueue); unreg_driver: - platform_driver_register(&vivid_pdrv); + platform_driver_unregister(&vivid_pdrv); unreg_device: platform_device_unregister(&vivid_pdev); free_output_strings: diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c b/drivers/media/test-drivers/vivid/vivid-ctrls.c index f94c15ff84f7..a8a134b36720 100644 --- a/drivers/media/test-drivers/vivid/vivid-ctrls.c +++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c @@ -609,17 +609,24 @@ static int vivid_vid_cap_s_ctrl(struct v4l2_ctrl *ctrl) break; case VIVID_CID_REDUCED_FPS: dev->reduced_fps = ctrl->val; - vivid_update_format_cap(dev, true); + if (dev->input_type[dev->input] == HDMI) + vivid_update_reduced_fps(dev); break; case VIVID_CID_HAS_CROP_CAP: + if (vb2_is_busy(&dev->vb_vid_cap_q)) + return -EBUSY; dev->has_crop_cap = ctrl->val; vivid_update_format_cap(dev, true); break; case VIVID_CID_HAS_COMPOSE_CAP: + if (vb2_is_busy(&dev->vb_vid_cap_q)) + return -EBUSY; dev->has_compose_cap = ctrl->val; vivid_update_format_cap(dev, true); break; case VIVID_CID_HAS_SCALER_CAP: + if (vb2_is_busy(&dev->vb_vid_cap_q)) + return -EBUSY; dev->has_scaler_cap = ctrl->val; vivid_update_format_cap(dev, true); break; @@ -1116,14 +1123,20 @@ static int vivid_vid_out_s_ctrl(struct v4l2_ctrl *ctrl) switch (ctrl->id) { case VIVID_CID_HAS_CROP_OUT: + if (vb2_is_busy(&dev->vb_vid_out_q)) + return -EBUSY; dev->has_crop_out = ctrl->val; vivid_update_format_out(dev); break; case VIVID_CID_HAS_COMPOSE_OUT: + if (vb2_is_busy(&dev->vb_vid_out_q)) + return -EBUSY; dev->has_compose_out = ctrl->val; vivid_update_format_out(dev); break; case VIVID_CID_HAS_SCALER_OUT: + if (vb2_is_busy(&dev->vb_vid_out_q)) + return -EBUSY; dev->has_scaler_out = ctrl->val; vivid_update_format_out(dev); break; diff --git a/drivers/media/test-drivers/vivid/vivid-vid-cap.c b/drivers/media/test-drivers/vivid/vivid-vid-cap.c index 8b3162e82032..d1c09bdb2f3b 100644 --- a/drivers/media/test-drivers/vivid/vivid-vid-cap.c +++ b/drivers/media/test-drivers/vivid/vivid-vid-cap.c @@ -362,6 +362,24 @@ static enum tpg_pixel_aspect vivid_get_pixel_aspect(const struct vivid_dev *dev) return TPG_PIXEL_ASPECT_SQUARE; } +void vivid_update_reduced_fps(struct vivid_dev *dev) +{ + struct v4l2_bt_timings *bt = &dev->dv_timings_cap[dev->input].bt; + unsigned int size = V4L2_DV_BT_FRAME_WIDTH(bt) * V4L2_DV_BT_FRAME_HEIGHT(bt); + u64 pixelclock; + + if (dev->reduced_fps && can_reduce_fps(bt)) { + pixelclock = div_u64(bt->pixelclock * 1000, 1001); + bt->flags |= V4L2_DV_FL_REDUCED_FPS; + } else { + pixelclock = bt->pixelclock; + bt->flags &= ~V4L2_DV_FL_REDUCED_FPS; + } + dev->timeperframe_vid_cap = (struct v4l2_fract) { + size / 100, (u32)pixelclock / 100 + }; +} + /* * Called whenever the format has to be reset which can occur when * changing inputs, standard, timings, etc. @@ -370,8 +388,12 @@ void vivid_update_format_cap(struct vivid_dev *dev, bool keep_controls) { struct v4l2_bt_timings *bt = &dev->dv_timings_cap[dev->input].bt; u32 dims[V4L2_CTRL_MAX_DIMS] = {}; - unsigned size; - u64 pixelclock; + + /* + * This resets the format, so must never be called while vb2_is_busy(). + */ + if (WARN_ON(vb2_is_busy(&dev->vb_vid_cap_q))) + return; switch (dev->input_type[dev->input]) { case WEBCAM: @@ -400,17 +422,7 @@ void vivid_update_format_cap(struct vivid_dev *dev, bool keep_controls) case HDMI: dev->src_rect.width = bt->width; dev->src_rect.height = bt->height; - size = V4L2_DV_BT_FRAME_WIDTH(bt) * V4L2_DV_BT_FRAME_HEIGHT(bt); - if (dev->reduced_fps && can_reduce_fps(bt)) { - pixelclock = div_u64(bt->pixelclock * 1000, 1001); - bt->flags |= V4L2_DV_FL_REDUCED_FPS; - } else { - pixelclock = bt->pixelclock; - bt->flags &= ~V4L2_DV_FL_REDUCED_FPS; - } - dev->timeperframe_vid_cap = (struct v4l2_fract) { - size / 100, (u32)pixelclock / 100 - }; + vivid_update_reduced_fps(dev); if (bt->interlaced) dev->field_cap = V4L2_FIELD_ALTERNATE; else diff --git a/drivers/media/test-drivers/vivid/vivid-vid-cap.h b/drivers/media/test-drivers/vivid/vivid-vid-cap.h index 38a99f7e038e..d08a85927510 100644 --- a/drivers/media/test-drivers/vivid/vivid-vid-cap.h +++ b/drivers/media/test-drivers/vivid/vivid-vid-cap.h @@ -9,6 +9,7 @@ #define _VIVID_VID_CAP_H_ void vivid_update_quality(struct vivid_dev *dev); +void vivid_update_reduced_fps(struct vivid_dev *dev); void vivid_update_format_cap(struct vivid_dev *dev, bool keep_controls); void vivid_update_outputs(struct vivid_dev *dev); void vivid_update_connected_outputs(struct vivid_dev *dev); diff --git a/drivers/media/test-drivers/vivid/vivid-vid-out.c b/drivers/media/test-drivers/vivid/vivid-vid-out.c index 8c037b90833e..23e1d5a189ee 100644 --- a/drivers/media/test-drivers/vivid/vivid-vid-out.c +++ b/drivers/media/test-drivers/vivid/vivid-vid-out.c @@ -214,6 +214,12 @@ void vivid_update_format_out(struct vivid_dev *dev) unsigned size, p; u64 pixelclock; + /* + * This resets the format, so must never be called while vb2_is_busy(). + */ + if (WARN_ON(vb2_is_busy(&dev->vb_vid_out_q))) + return; + switch (dev->output_type[dev->output]) { case SVID: default: diff --git a/drivers/media/usb/airspy/airspy.c b/drivers/media/usb/airspy/airspy.c index 08f0920cf6ca..7bad5d8ea186 100644 --- a/drivers/media/usb/airspy/airspy.c +++ b/drivers/media/usb/airspy/airspy.c @@ -522,11 +522,13 @@ static int airspy_start_streaming(struct vb2_queue *vq, unsigned int count) dev_dbg(s->dev, "\n"); - if (!s->udev) - return -ENODEV; - mutex_lock(&s->v4l2_lock); + if (!s->udev) { + ret = -ENODEV; + goto err_clear_bit; + } + s->sequence = 0; set_bit(POWER_ON, &s->flags); diff --git a/drivers/media/usb/cx231xx/cx231xx-cards.c b/drivers/media/usb/cx231xx/cx231xx-cards.c index 691f073892b3..d96b9ab276b7 100644 --- a/drivers/media/usb/cx231xx/cx231xx-cards.c +++ b/drivers/media/usb/cx231xx/cx231xx-cards.c @@ -1575,7 +1575,8 @@ static int cx231xx_init_v4l2(struct cx231xx *dev, dev->video_mode.end_point_addr, dev->video_mode.num_alt); - dev->video_mode.alt_max_pkt_size = devm_kmalloc_array(&udev->dev, 32, dev->video_mode.num_alt, GFP_KERNEL); + dev->video_mode.alt_max_pkt_size = devm_kmalloc_array(&interface->dev, 32, + dev->video_mode.num_alt, GFP_KERNEL); if (dev->video_mode.alt_max_pkt_size == NULL) return -ENOMEM; @@ -1616,7 +1617,8 @@ static int cx231xx_init_v4l2(struct cx231xx *dev, dev->vbi_mode.num_alt); /* compute alternate max packet sizes for vbi */ - dev->vbi_mode.alt_max_pkt_size = devm_kmalloc_array(&udev->dev, 32, dev->vbi_mode.num_alt, GFP_KERNEL); + dev->vbi_mode.alt_max_pkt_size = devm_kmalloc_array(&interface->dev, 32, + dev->vbi_mode.num_alt, GFP_KERNEL); if (dev->vbi_mode.alt_max_pkt_size == NULL) return -ENOMEM; @@ -1658,7 +1660,9 @@ static int cx231xx_init_v4l2(struct cx231xx *dev, "sliced CC EndPoint Addr 0x%x, Alternate settings: %i\n", dev->sliced_cc_mode.end_point_addr, dev->sliced_cc_mode.num_alt); - dev->sliced_cc_mode.alt_max_pkt_size = devm_kmalloc_array(&udev->dev, 32, dev->sliced_cc_mode.num_alt, GFP_KERNEL); + dev->sliced_cc_mode.alt_max_pkt_size = devm_kmalloc_array(&interface->dev, 32, + dev->sliced_cc_mode.num_alt, + GFP_KERNEL); if (dev->sliced_cc_mode.alt_max_pkt_size == NULL) return -ENOMEM; @@ -1722,7 +1726,7 @@ static int cx231xx_usb_probe(struct usb_interface *interface, udev = usb_get_dev(interface_to_usbdev(interface)); /* allocate memory for our device state and initialize it */ - dev = devm_kzalloc(&udev->dev, sizeof(*dev), GFP_KERNEL); + dev = devm_kzalloc(&interface->dev, sizeof(*dev), GFP_KERNEL); if (dev == NULL) { retval = -ENOMEM; goto err_if; @@ -1852,7 +1856,9 @@ static int cx231xx_usb_probe(struct usb_interface *interface, dev->ts1_mode.end_point_addr, dev->ts1_mode.num_alt); - dev->ts1_mode.alt_max_pkt_size = devm_kmalloc_array(&udev->dev, 32, dev->ts1_mode.num_alt, GFP_KERNEL); + dev->ts1_mode.alt_max_pkt_size = devm_kmalloc_array(&interface->dev, 32, + dev->ts1_mode.num_alt, + GFP_KERNEL); if (dev->ts1_mode.alt_max_pkt_size == NULL) { retval = -ENOMEM; goto err_video_alt; diff --git a/drivers/media/usb/msi2500/msi2500.c b/drivers/media/usb/msi2500/msi2500.c index 33099f39146a..235b685f8536 100644 --- a/drivers/media/usb/msi2500/msi2500.c +++ b/drivers/media/usb/msi2500/msi2500.c @@ -541,7 +541,8 @@ static int msi2500_isoc_init(struct msi2500_dev *dev) } /* Must be called with vb_queue_lock hold */ -static void msi2500_cleanup_queued_bufs(struct msi2500_dev *dev) +static void msi2500_cleanup_queued_bufs(struct msi2500_dev *dev, + enum vb2_buffer_state state) { unsigned long flags; @@ -554,7 +555,7 @@ static void msi2500_cleanup_queued_bufs(struct msi2500_dev *dev) buf = list_entry(dev->queued_bufs.next, struct msi2500_frame_buf, list); list_del(&buf->list); - vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); + vb2_buffer_done(&buf->vb.vb2_buf, state); } spin_unlock_irqrestore(&dev->queued_bufs_lock, flags); } @@ -830,25 +831,40 @@ static int msi2500_start_streaming(struct vb2_queue *vq, unsigned int count) dev_dbg(dev->dev, "\n"); - if (!dev->udev) - return -ENODEV; + if (!dev->udev) { + ret = -ENODEV; + goto err_cleanup; + } - if (mutex_lock_interruptible(&dev->v4l2_lock)) - return -ERESTARTSYS; + if (mutex_lock_interruptible(&dev->v4l2_lock)) { + ret = -ERESTARTSYS; + goto err_cleanup; + } /* wake-up tuner */ v4l2_subdev_call(dev->v4l2_subdev, core, s_power, 1); ret = msi2500_set_usb_adc(dev); + if (ret) + goto err_unlock_cleanup; ret = msi2500_isoc_init(dev); if (ret) - msi2500_cleanup_queued_bufs(dev); + goto err_unlock_cleanup; ret = msi2500_ctrl_msg(dev, CMD_START_STREAMING, 0); + if (ret) + goto err_isoc_cleanup; mutex_unlock(&dev->v4l2_lock); + return 0; +err_isoc_cleanup: + msi2500_isoc_cleanup(dev); +err_unlock_cleanup: + mutex_unlock(&dev->v4l2_lock); +err_cleanup: + msi2500_cleanup_queued_bufs(dev, VB2_BUF_STATE_QUEUED); return ret; } @@ -863,7 +879,7 @@ static void msi2500_stop_streaming(struct vb2_queue *vq) if (dev->udev) msi2500_isoc_cleanup(dev); - msi2500_cleanup_queued_bufs(dev); + msi2500_cleanup_queued_bufs(dev, VB2_BUF_STATE_ERROR); /* according to tests, at least 700us delay is required */ msleep(20); diff --git a/drivers/media/usb/pwc/pwc-if.c b/drivers/media/usb/pwc/pwc-if.c index c6e5d031f068..4cc278d711ae 100644 --- a/drivers/media/usb/pwc/pwc-if.c +++ b/drivers/media/usb/pwc/pwc-if.c @@ -710,11 +710,15 @@ static int start_streaming(struct vb2_queue *vq, unsigned int count) struct pwc_device *pdev = vb2_get_drv_priv(vq); int r; - if (!pdev->udev) + if (!pdev->udev) { + pwc_cleanup_queued_bufs(pdev, VB2_BUF_STATE_QUEUED); return -ENODEV; + } - if (mutex_lock_interruptible(&pdev->v4l2_lock)) + if (mutex_lock_interruptible(&pdev->v4l2_lock)) { + pwc_cleanup_queued_bufs(pdev, VB2_BUF_STATE_QUEUED); return -ERESTARTSYS; + } /* Turn on camera and set LEDS on */ pwc_camera_power(pdev, 1); pwc_set_leds(pdev, leds[0], leds[1]); @@ -726,6 +730,11 @@ static int start_streaming(struct vb2_queue *vq, unsigned int count) pwc_camera_power(pdev, 0); /* And cleanup any queued bufs!! */ pwc_cleanup_queued_bufs(pdev, VB2_BUF_STATE_QUEUED); + if (pdev->fill_buf) { + vb2_buffer_done(&pdev->fill_buf->vb.vb2_buf, + VB2_BUF_STATE_QUEUED); + pdev->fill_buf = NULL; + } } mutex_unlock(&pdev->v4l2_lock); diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c index 85d07ef44f62..838e9aef703c 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c @@ -882,6 +882,7 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, struct v4l2_ctrl_h264_decode_params *p_h264_dec_params; struct v4l2_ctrl_hevc_sps *p_hevc_sps; struct v4l2_ctrl_hevc_pps *p_hevc_pps; + struct v4l2_ctrl_hevc_slice_params *p_hevc_slice_params; struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering; struct v4l2_ctrl_hevc_decode_params *p_hevc_decode_params; struct v4l2_area *area; @@ -1171,6 +1172,18 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, break; case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS: + p_hevc_slice_params = p; + + if (p_hevc_slice_params->num_ref_idx_l0_active_minus1 >= + V4L2_HEVC_DPB_ENTRIES_NUM_MAX) + return -EINVAL; + + if (p_hevc_slice_params->slice_type != V4L2_HEVC_SLICE_TYPE_B) + break; + + if (p_hevc_slice_params->num_ref_idx_l1_active_minus1 >= + V4L2_HEVC_DPB_ENTRIES_NUM_MAX) + return -EINVAL; break; case V4L2_CTRL_TYPE_HDR10_CLL_INFO: diff --git a/drivers/media/v4l2-core/v4l2-ctrls-request.c b/drivers/media/v4l2-core/v4l2-ctrls-request.c index e77f722b36a4..aa5566a8fb7d 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-request.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-request.c @@ -348,13 +348,12 @@ void v4l2_ctrl_request_complete(struct media_request *req, ret = v4l2_ctrl_handler_init(hdl, (main_hdl->nr_of_buckets - 1) * 8); if (!ret) ret = v4l2_ctrl_request_bind(req, hdl, main_hdl); - if (ret) { - v4l2_ctrl_handler_free(hdl); - kfree(hdl); - return; - } + if (ret) + goto error; hdl->request_is_queued = true; obj = media_request_object_find(req, &req_ops, main_hdl); + if (!obj) + goto error; } hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj); @@ -389,6 +388,11 @@ void v4l2_ctrl_request_complete(struct media_request *req, mutex_unlock(main_hdl->lock); media_request_object_complete(obj); media_request_object_put(obj); + return; + +error: + v4l2_ctrl_handler_free(hdl); + kfree(hdl); } EXPORT_SYMBOL(v4l2_ctrl_request_complete); diff --git a/drivers/media/v4l2-core/v4l2-fwnode.c b/drivers/media/v4l2-core/v4l2-fwnode.c index cb153ce42c45..a8becf88b753 100644 --- a/drivers/media/v4l2-core/v4l2-fwnode.c +++ b/drivers/media/v4l2-core/v4l2-fwnode.c @@ -1246,7 +1246,7 @@ v4l2_async_nf_parse_fwnode_sensor(struct device *dev, return 0; } -int v4l2_async_register_subdev_sensor(struct v4l2_subdev *sd) +int __v4l2_async_register_subdev_sensor(struct v4l2_subdev *sd, struct module *module) { struct v4l2_async_notifier *notifier; int ret; @@ -1272,7 +1272,7 @@ int v4l2_async_register_subdev_sensor(struct v4l2_subdev *sd) if (ret < 0) goto out_cleanup; - ret = v4l2_async_register_subdev(sd); + ret = __v4l2_async_register_subdev(sd, module); if (ret < 0) goto out_unregister; @@ -1290,7 +1290,7 @@ int v4l2_async_register_subdev_sensor(struct v4l2_subdev *sd) return ret; } -EXPORT_SYMBOL_GPL(v4l2_async_register_subdev_sensor); +EXPORT_SYMBOL_GPL(__v4l2_async_register_subdev_sensor); MODULE_DESCRIPTION("V4L2 fwnode binding parsing library"); MODULE_LICENSE("GPL"); diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 25e66bf18f5f..349022153b18 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -2506,6 +2506,10 @@ int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable) u64 source_mask = 0; int pad_index = -1; + if (WARN_ON(!v4l2_subdev_has_op(sd, pad, enable_streams) || + !v4l2_subdev_has_op(sd, pad, disable_streams))) + return -ENOIOCTLCMD; + /* * Find the source pad. This helper is meant for subdevs that have a * single source pad, so failures shouldn't happen, but catch them diff --git a/drivers/misc/mei/bus.c b/drivers/misc/mei/bus.c index 2c810ab12e62..a6f0bc150d77 100644 --- a/drivers/misc/mei/bus.c +++ b/drivers/misc/mei/bus.c @@ -4,6 +4,7 @@ * Intel Management Engine Interface (Intel MEI) Linux driver */ +#include #include #include #include @@ -1317,15 +1318,16 @@ static void mei_dev_bus_put(struct mei_device *bus) static void mei_cl_bus_dev_release(struct device *dev) { struct mei_cl_device *cldev = to_mei_cl_device(dev); - struct mei_device *mdev = cldev->cl->dev; + struct mei_device *bus = cldev->bus; struct mei_cl *cl; - mei_cl_flush_queues(cldev->cl, NULL); - mei_me_cl_put(cldev->me_cl); - mei_dev_bus_put(cldev->bus); - - list_for_each_entry(cl, &mdev->file_list, link) - WARN_ON(cl == cldev->cl); + scoped_guard(mutex, &bus->device_lock) { + mei_cl_flush_queues(cldev->cl, NULL); + mei_me_cl_put(cldev->me_cl); + list_for_each_entry(cl, &bus->file_list, link) + WARN_ON(cl == cldev->cl); + } + mei_dev_bus_put(bus); kfree(cldev->cl); kfree(cldev); diff --git a/drivers/misc/nsm.c b/drivers/misc/nsm.c index ef7b32742340..3960506eb7ab 100644 --- a/drivers/misc/nsm.c +++ b/drivers/misc/nsm.c @@ -367,7 +367,7 @@ static long nsm_dev_ioctl(struct file *file, unsigned int cmd, /* Copy user argument struct to kernel argument struct */ r = -EFAULT; if (copy_from_user(&raw, argp, _IOC_SIZE(cmd))) - goto out; + return r; mutex_lock(&nsm->lock); @@ -413,6 +413,7 @@ static int nsm_device_init_vq(struct virtio_device *vdev) } static const struct file_operations nsm_dev_fops = { + .owner = THIS_MODULE, .unlocked_ioctl = nsm_dev_ioctl, .compat_ioctl = compat_ptr_ioctl, }; diff --git a/drivers/mmc/host/vub300.c b/drivers/mmc/host/vub300.c index 93c4e145009e..1614e9995ee6 100644 --- a/drivers/mmc/host/vub300.c +++ b/drivers/mmc/host/vub300.c @@ -2115,19 +2115,19 @@ static int vub300_probe(struct usb_interface *interface, command_out_urb = usb_alloc_urb(0, GFP_KERNEL); if (!command_out_urb) { retval = -ENOMEM; - goto error0; + goto err_put_udev; } command_res_urb = usb_alloc_urb(0, GFP_KERNEL); if (!command_res_urb) { retval = -ENOMEM; - goto error1; + goto err_free_out_urb; } /* this also allocates memory for our VUB300 mmc host device */ mmc = mmc_alloc_host(sizeof(*vub300), &udev->dev); if (!mmc) { retval = -ENOMEM; dev_err(&udev->dev, "not enough memory for the mmc_host\n"); - goto error4; + goto err_free_res_urb; } /* MMC core transfer sizes tunable parameters */ mmc->caps = 0; @@ -2344,23 +2344,29 @@ static int vub300_probe(struct usb_interface *interface, interface_to_InterfaceNumber(interface)); retval = mmc_add_host(mmc); if (retval) - goto error6; + goto err_stop_io; return 0; -error6: - timer_delete_sync(&vub300->inactivity_timer); + +err_stop_io: + vub300->interface = NULL; + kref_put(&vub300->kref, vub300_delete); + + return retval; + err_free_host: mmc_free_host(mmc); /* * and hence also frees vub300 * which is contained at the end of struct mmc */ -error4: +err_free_res_urb: usb_free_urb(command_res_urb); -error1: +err_free_out_urb: usb_free_urb(command_out_urb); -error0: +err_put_udev: usb_put_dev(udev); + return retval; } diff --git a/drivers/mtd/maps/vmu-flash.c b/drivers/mtd/maps/vmu-flash.c index 3667faefe6c5..e992f4d1417e 100644 --- a/drivers/mtd/maps/vmu-flash.c +++ b/drivers/mtd/maps/vmu-flash.c @@ -610,7 +610,7 @@ static int vmu_connect(struct maple_device *mdev) basic_flash_data = be32_to_cpu(mdev->devinfo.function_data[c - 1]); - card = kmalloc(sizeof(struct memcard), GFP_KERNEL); + card = kzalloc_obj(struct memcard); if (!card) { error = -ENOMEM; goto fail_nomem; @@ -628,15 +628,13 @@ static int vmu_connect(struct maple_device *mdev) * Not sure there are actually any multi-partition devices in the * real world, but the hardware supports them, so, so will we */ - card->parts = kmalloc_array(card->partitions, sizeof(struct vmupart), - GFP_KERNEL); + card->parts = kzalloc_objs(struct vmupart, card->partitions); if (!card->parts) { error = -ENOMEM; goto fail_partitions; } - card->mtd = kmalloc_array(card->partitions, sizeof(struct mtd_info), - GFP_KERNEL); + card->mtd = kzalloc_objs(struct mtd_info, card->partitions); if (!card->mtd) { error = -ENOMEM; goto fail_mtd_info; diff --git a/drivers/mtd/mtdcore.c b/drivers/mtd/mtdcore.c index 64808493b4f5..42cabd3c1efa 100644 --- a/drivers/mtd/mtdcore.c +++ b/drivers/mtd/mtdcore.c @@ -104,6 +104,15 @@ static void mtd_release(struct device *dev) device_destroy(&mtd_class, index + 1); } +/* + * No-op device release used in add_mtd_device() error paths. + * Prevents mtd_release() from being called via device_release(), + * which would free the mtd_info that the caller still manages. + */ +static void mtd_dev_release_nop(struct device *dev) +{ +} + static void mtd_device_release(struct kref *kref) { struct mtd_info *mtd = container_of(kref, struct mtd_info, refcnt); @@ -798,10 +807,8 @@ int add_mtd_device(struct mtd_info *mtd) mtd_check_of_node(mtd); of_node_get(mtd_get_of_node(mtd)); error = device_register(&mtd->dev); - if (error) { - put_device(&mtd->dev); + if (error) goto fail_added; - } /* Add the nvmem provider */ error = mtd_nvmem_add(mtd); @@ -839,8 +846,16 @@ int add_mtd_device(struct mtd_info *mtd) return 0; fail_nvmem_add: - device_unregister(&mtd->dev); + device_del(&mtd->dev); fail_added: + /* + * Clear type and set nop release to prevent mtd_release() -> + * release_mtd_partition() -> free_partition() from freeing mtd. + * The caller handles cleanup on failure. + */ + mtd->dev.type = NULL; + mtd->dev.release = mtd_dev_release_nop; + put_device(&mtd->dev); of_node_put(mtd_get_of_node(mtd)); fail_devname: idr_remove(&mtd_idr, i); diff --git a/drivers/mtd/mtdswap.c b/drivers/mtd/mtdswap.c index d8f2e5be2d31..42ff7deace80 100644 --- a/drivers/mtd/mtdswap.c +++ b/drivers/mtd/mtdswap.c @@ -125,6 +125,7 @@ struct mtdswap_dev { char *page_buf; char *oob_buf; + struct dentry *debugfs_stats; }; struct mtdswap_oobdata { @@ -1262,7 +1263,8 @@ static int mtdswap_add_debugfs(struct mtdswap_dev *d) if (IS_ERR_OR_NULL(root)) return -1; - debugfs_create_file("mtdswap_stats", S_IRUSR, root, d, &mtdswap_fops); + d->debugfs_stats = debugfs_create_file("mtdswap_stats", 0400, root, + d, &mtdswap_fops); return 0; } @@ -1463,6 +1465,7 @@ static void mtdswap_remove_dev(struct mtd_blktrans_dev *dev) { struct mtdswap_dev *d = MTDSWAP_MBD_TO_MTDSWAP(dev); + debugfs_remove(d->debugfs_stats); del_mtd_blktrans_dev(dev); mtdswap_cleanup(d); kfree(d); diff --git a/drivers/mtd/nand/ecc-mtk.c b/drivers/mtd/nand/ecc-mtk.c index c75bb8b80cc1..96703f0a418e 100644 --- a/drivers/mtd/nand/ecc-mtk.c +++ b/drivers/mtd/nand/ecc-mtk.c @@ -123,8 +123,8 @@ static int mt7622_ecc_regs[] = { [ECC_DECIRQ_STA] = 0x144, }; -static inline void mtk_ecc_wait_idle(struct mtk_ecc *ecc, - enum mtk_ecc_operation op) +static inline int mtk_ecc_wait_idle(struct mtk_ecc *ecc, + enum mtk_ecc_operation op) { struct device *dev = ecc->dev; u32 val; @@ -136,6 +136,8 @@ static inline void mtk_ecc_wait_idle(struct mtk_ecc *ecc, if (ret) dev_warn(dev, "%s NOT idle\n", op == ECC_ENCODE ? "encoder" : "decoder"); + + return ret; } static irqreturn_t mtk_ecc_irq(int irq, void *id) @@ -312,7 +314,11 @@ int mtk_ecc_enable(struct mtk_ecc *ecc, struct mtk_ecc_config *config) return ret; } - mtk_ecc_wait_idle(ecc, op); + ret = mtk_ecc_wait_idle(ecc, op); + if (ret) { + mutex_unlock(&ecc->lock); + return ret; + } ret = mtk_ecc_config(ecc, config); if (ret) { @@ -412,7 +418,9 @@ int mtk_ecc_encode(struct mtk_ecc *ecc, struct mtk_ecc_config *config, if (ret) goto timeout; - mtk_ecc_wait_idle(ecc, ECC_ENCODE); + ret = mtk_ecc_wait_idle(ecc, ECC_ENCODE); + if (ret) + goto timeout; /* Program ECC bytes to OOB: per sector oob = FDM + ECC + SPARE */ len = (config->strength * ecc->caps->parity_bits + 7) >> 3; diff --git a/drivers/net/amt.c b/drivers/net/amt.c index e4d1aa26d7bb..2bc8781f07c2 100644 --- a/drivers/net/amt.c +++ b/drivers/net/amt.c @@ -1211,7 +1211,7 @@ static netdev_tx_t amt_dev_xmit(struct sk_buff *skb, struct net_device *dev) data = true; } v6 = false; - group.ip4 = iph->daddr; + group.ip4 = ip_hdr(skb)->daddr; #if IS_ENABLED(CONFIG_IPV6) } else if (iph->version == 6) { ip6h = ipv6_hdr(skb); @@ -1235,7 +1235,7 @@ static netdev_tx_t amt_dev_xmit(struct sk_buff *skb, struct net_device *dev) data = true; } v6 = true; - group.ip6 = ip6h->daddr; + group.ip6 = ipv6_hdr(skb)->daddr; #endif } else { dev->stats.tx_errors++; @@ -1278,12 +1278,12 @@ static netdev_tx_t amt_dev_xmit(struct sk_buff *skb, struct net_device *dev) hlist_for_each_entry_rcu(gnode, &tunnel->groups[hash], node) { if (!v6) { - if (gnode->group_addr.ip4 == iph->daddr) + if (gnode->group_addr.ip4 == group.ip4) goto found; #if IS_ENABLED(CONFIG_IPV6) } else { if (ipv6_addr_equal(&gnode->group_addr.ip6, - &ip6h->daddr)) + &group.ip6)) goto found; #endif } @@ -2000,14 +2000,18 @@ static void amt_igmpv3_report_handler(struct amt_dev *amt, struct sk_buff *skb, struct igmpv3_report *ihrv3 = igmpv3_report_hdr(skb); int len = skb_transport_offset(skb) + sizeof(*ihrv3); void *zero_grec = (void *)&igmpv3_zero_grec; - struct iphdr *iph = ip_hdr(skb); struct amt_group_node *gnode; union amt_addr group, host; struct igmpv3_grec *grec; + __be32 saddr; u16 nsrcs; + u16 ngrec; int i; - for (i = 0; i < ntohs(ihrv3->ngrec); i++) { + saddr = ip_hdr(skb)->saddr; + ngrec = ntohs(ihrv3->ngrec); + + for (i = 0; i < ngrec; i++) { len += sizeof(*grec); if (!ip_mc_may_pull(skb, len)) break; @@ -2019,10 +2023,13 @@ static void amt_igmpv3_report_handler(struct amt_dev *amt, struct sk_buff *skb, if (!ip_mc_may_pull(skb, len)) break; + grec = (void *)(skb->data + len - sizeof(*grec) - + nsrcs * sizeof(__be32)); + memset(&group, 0, sizeof(union amt_addr)); group.ip4 = grec->grec_mca; memset(&host, 0, sizeof(union amt_addr)); - host.ip4 = iph->saddr; + host.ip4 = saddr; gnode = amt_lookup_group(tunnel, &group, &host, false); if (!gnode) { gnode = amt_add_group(amt, tunnel, &group, &host, @@ -2162,14 +2169,18 @@ static void amt_mldv2_report_handler(struct amt_dev *amt, struct sk_buff *skb, struct mld2_report *mld2r = (struct mld2_report *)icmp6_hdr(skb); int len = skb_transport_offset(skb) + sizeof(*mld2r); void *zero_grec = (void *)&mldv2_zero_grec; - struct ipv6hdr *ip6h = ipv6_hdr(skb); struct amt_group_node *gnode; union amt_addr group, host; struct mld2_grec *grec; + struct in6_addr saddr; u16 nsrcs; + u16 ngrec; int i; - for (i = 0; i < ntohs(mld2r->mld2r_ngrec); i++) { + saddr = ipv6_hdr(skb)->saddr; + ngrec = ntohs(mld2r->mld2r_ngrec); + + for (i = 0; i < ngrec; i++) { len += sizeof(*grec); if (!ipv6_mc_may_pull(skb, len)) break; @@ -2181,10 +2192,13 @@ static void amt_mldv2_report_handler(struct amt_dev *amt, struct sk_buff *skb, if (!ipv6_mc_may_pull(skb, len)) break; + grec = (void *)(skb->data + len - sizeof(*grec) - + nsrcs * sizeof(struct in6_addr)); + memset(&group, 0, sizeof(union amt_addr)); group.ip6 = grec->grec_mca; memset(&host, 0, sizeof(union amt_addr)); - host.ip6 = ip6h->saddr; + host.ip6 = saddr; gnode = amt_lookup_group(tunnel, &group, &host, true); if (!gnode) { gnode = amt_add_group(amt, tunnel, &group, &host, @@ -2305,7 +2319,9 @@ static bool amt_multicast_data_handler(struct amt_dev *amt, struct sk_buff *skb) skb_push(skb, sizeof(*eth)); skb_reset_mac_header(skb); skb_pull(skb, sizeof(*eth)); - eth = eth_hdr(skb); + + if (skb_cow_head(skb, 0)) + return true; if (!pskb_may_pull(skb, sizeof(*iph))) return true; @@ -2315,6 +2331,7 @@ static bool amt_multicast_data_handler(struct amt_dev *amt, struct sk_buff *skb) if (!ipv4_is_multicast(iph->daddr)) return true; skb->protocol = htons(ETH_P_IP); + eth = eth_hdr(skb); eth->h_proto = htons(ETH_P_IP); ip_eth_mc_map(iph->daddr, eth->h_dest); #if IS_ENABLED(CONFIG_IPV6) @@ -2328,6 +2345,7 @@ static bool amt_multicast_data_handler(struct amt_dev *amt, struct sk_buff *skb) if (!ipv6_addr_is_multicast(&ip6h->daddr)) return true; skb->protocol = htons(ETH_P_IPV6); + eth = eth_hdr(skb); eth->h_proto = htons(ETH_P_IPV6); ipv6_eth_mc_map(&ip6h->daddr, eth->h_dest); #endif @@ -2351,10 +2369,12 @@ static bool amt_membership_query_handler(struct amt_dev *amt, struct sk_buff *skb) { struct amt_header_membership_query *amtmq; - struct igmpv3_query *ihv3; struct ethhdr *eth, *oeth; + struct igmpv3_query *ihv3; + u8 h_source[ETH_ALEN]; struct iphdr *iph; int hdr_size, len; + u64 response_mac; hdr_size = sizeof(*amtmq) + sizeof(struct udphdr); if (!pskb_may_pull(skb, hdr_size)) @@ -2367,6 +2387,8 @@ static bool amt_membership_query_handler(struct amt_dev *amt, if (amtmq->nonce != amt->nonce) return true; + response_mac = amtmq->response_mac; + hdr_size -= sizeof(*eth); if (iptunnel_pull_header(skb, hdr_size, htons(ETH_P_TEB), false)) return true; @@ -2376,6 +2398,9 @@ static bool amt_membership_query_handler(struct amt_dev *amt, skb_pull(skb, sizeof(*eth)); skb_reset_network_header(skb); eth = eth_hdr(skb); + ether_addr_copy(h_source, oeth->h_source); + if (skb_cow_head(skb, 0)) + return true; if (!pskb_may_pull(skb, sizeof(*iph))) return true; @@ -2388,6 +2413,7 @@ static bool amt_membership_query_handler(struct amt_dev *amt, sizeof(*ihv3))) return true; + iph = ip_hdr(skb); if (!ipv4_is_multicast(iph->daddr)) return true; @@ -2395,10 +2421,11 @@ static bool amt_membership_query_handler(struct amt_dev *amt, skb_reset_transport_header(skb); skb_push(skb, sizeof(*iph) + AMT_IPHDR_OPTS); WRITE_ONCE(amt->ready4, true); - amt->mac = amtmq->response_mac; + amt->mac = response_mac; amt->req_cnt = 0; amt->qi = ihv3->qqic; skb->protocol = htons(ETH_P_IP); + eth = eth_hdr(skb); eth->h_proto = htons(ETH_P_IP); ip_eth_mc_map(iph->daddr, eth->h_dest); #if IS_ENABLED(CONFIG_IPV6) @@ -2421,10 +2448,11 @@ static bool amt_membership_query_handler(struct amt_dev *amt, skb_reset_transport_header(skb); skb_push(skb, sizeof(*ip6h) + AMT_IP6HDR_OPTS); WRITE_ONCE(amt->ready6, true); - amt->mac = amtmq->response_mac; + amt->mac = response_mac; amt->req_cnt = 0; amt->qi = mld2q->mld2q_qqic; skb->protocol = htons(ETH_P_IPV6); + eth = eth_hdr(skb); eth->h_proto = htons(ETH_P_IPV6); ipv6_eth_mc_map(&ip6h->daddr, eth->h_dest); #endif @@ -2432,7 +2460,7 @@ static bool amt_membership_query_handler(struct amt_dev *amt, return true; } - ether_addr_copy(eth->h_source, oeth->h_source); + ether_addr_copy(eth->h_source, h_source); skb->pkt_type = PACKET_MULTICAST; skb->ip_summed = CHECKSUM_NONE; len = skb->len; @@ -2455,8 +2483,11 @@ static bool amt_update_handler(struct amt_dev *amt, struct sk_buff *skb) struct ethhdr *eth; struct iphdr *iph; int len, hdr_size; + u64 response_mac; + __be32 saddr; + __be32 nonce; - iph = ip_hdr(skb); + saddr = ip_hdr(skb)->saddr; hdr_size = sizeof(*amtmu) + sizeof(struct udphdr); if (!pskb_may_pull(skb, hdr_size)) @@ -2466,15 +2497,18 @@ static bool amt_update_handler(struct amt_dev *amt, struct sk_buff *skb) if (amtmu->reserved || amtmu->version) return true; + nonce = amtmu->nonce; + response_mac = amtmu->response_mac; + if (iptunnel_pull_header(skb, hdr_size, skb->protocol, false)) return true; skb_reset_network_header(skb); list_for_each_entry_rcu(tunnel, &amt->tunnel_list, list) { - if (tunnel->ip4 == iph->saddr) { - if ((amtmu->nonce == tunnel->nonce && - amtmu->response_mac == tunnel->mac)) { + if (tunnel->ip4 == saddr) { + if ((nonce == tunnel->nonce && + response_mac == tunnel->mac)) { mod_delayed_work(amt_wq, &tunnel->gc_wq, msecs_to_jiffies(amt_gmi(amt)) * 3); @@ -2492,6 +2526,9 @@ static bool amt_update_handler(struct amt_dev *amt, struct sk_buff *skb) if (!pskb_may_pull(skb, sizeof(*iph))) return true; + if (skb_cow_head(skb, 0)) + return true; + iph = ip_hdr(skb); if (iph->version == 4) { if (ip_mc_check_igmp(skb)) { @@ -2508,6 +2545,7 @@ static bool amt_update_handler(struct amt_dev *amt, struct sk_buff *skb) eth = eth_hdr(skb); skb->protocol = htons(ETH_P_IP); eth->h_proto = htons(ETH_P_IP); + iph = ip_hdr(skb); ip_eth_mc_map(iph->daddr, eth->h_dest); #if IS_ENABLED(CONFIG_IPV6) } else if (iph->version == 6) { @@ -2527,6 +2565,7 @@ static bool amt_update_handler(struct amt_dev *amt, struct sk_buff *skb) eth = eth_hdr(skb); skb->protocol = htons(ETH_P_IPV6); eth->h_proto = htons(ETH_P_IPV6); + ip6h = ipv6_hdr(skb); ipv6_eth_mc_map(&ip6h->daddr, eth->h_dest); #endif } else { @@ -2772,7 +2811,7 @@ static void amt_gw_rcv(struct amt_dev *amt, struct sk_buff *skb) static int amt_rcv(struct sock *sk, struct sk_buff *skb) { struct amt_dev *amt; - struct iphdr *iph; + __be32 saddr; int type; bool err; @@ -2785,7 +2824,7 @@ static int amt_rcv(struct sock *sk, struct sk_buff *skb) } skb->dev = amt->dev; - iph = ip_hdr(skb); + saddr = ip_hdr(skb)->saddr; type = amt_parse_type(skb); if (type == -1) { err = true; @@ -2795,7 +2834,7 @@ static int amt_rcv(struct sock *sk, struct sk_buff *skb) if (amt->mode == AMT_MODE_GATEWAY) { switch (type) { case AMT_MSG_ADVERTISEMENT: - if (iph->saddr != amt->discovery_ip) { + if (saddr != amt->discovery_ip) { netdev_dbg(amt->dev, "Invalid Relay IP\n"); err = true; goto drop; @@ -2807,7 +2846,7 @@ static int amt_rcv(struct sock *sk, struct sk_buff *skb) } goto out; case AMT_MSG_MULTICAST_DATA: - if (iph->saddr != amt->remote_ip) { + if (saddr != amt->remote_ip) { netdev_dbg(amt->dev, "Invalid Relay IP\n"); err = true; goto drop; @@ -2818,7 +2857,7 @@ static int amt_rcv(struct sock *sk, struct sk_buff *skb) else goto out; case AMT_MSG_MEMBERSHIP_QUERY: - if (iph->saddr != amt->remote_ip) { + if (saddr != amt->remote_ip) { netdev_dbg(amt->dev, "Invalid Relay IP\n"); err = true; goto drop; @@ -2995,9 +3034,15 @@ static int amt_dev_open(struct net_device *dev) amt->event_idx = 0; amt->nr_events = 0; + enable_delayed_work(&amt->discovery_wq); + enable_delayed_work(&amt->req_wq); + err = amt_socket_create(amt); - if (err) + if (err) { + disable_delayed_work(&amt->req_wq); + disable_delayed_work(&amt->discovery_wq); return err; + } amt->req_cnt = 0; amt->remote_ip = 0; @@ -3023,8 +3068,8 @@ static int amt_dev_stop(struct net_device *dev) struct sk_buff *skb; int i; - cancel_delayed_work_sync(&amt->req_wq); - cancel_delayed_work_sync(&amt->discovery_wq); + disable_delayed_work_sync(&amt->req_wq); + disable_delayed_work_sync(&amt->discovery_wq); cancel_delayed_work_sync(&amt->secret_wq); /* shutdown */ @@ -3278,6 +3323,8 @@ static int amt_newlink(struct net_device *dev, INIT_DELAYED_WORK(&amt->req_wq, amt_req_work); INIT_DELAYED_WORK(&amt->secret_wq, amt_secret_work); INIT_WORK(&amt->event_wq, amt_event_work); + disable_delayed_work(&amt->req_wq); + disable_delayed_work(&amt->discovery_wq); INIT_LIST_HEAD(&amt->tunnel_list); return 0; err: diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index e38cae22fe02..52af663e12e0 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -3422,7 +3422,8 @@ static void bond_send_validate(struct bonding *bond, struct slave *slave) { bond_arp_send_all(bond, slave); #if IS_ENABLED(CONFIG_IPV6) - bond_ns_send_all(bond, slave); + if (likely(ipv6_mod_enabled())) + bond_ns_send_all(bond, slave); #endif } diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c index 5efd03c2deea..64ab34e37c36 100644 --- a/drivers/net/ethernet/airoha/airoha_eth.c +++ b/drivers/net/ethernet/airoha/airoha_eth.c @@ -2326,8 +2326,7 @@ static int airoha_tc_setup_qdisc_ets(struct airoha_gdm_port *port, if (opt->parent == TC_H_ROOT) return -EINVAL; - channel = TC_H_MAJ(opt->handle) >> 16; - channel = channel % AIROHA_NUM_QOS_CHANNELS; + channel = TC_H_MIN(opt->parent) % AIROHA_NUM_QOS_CHANNELS; switch (opt->command) { case TC_ETS_REPLACE: diff --git a/drivers/net/ethernet/airoha/airoha_npu.c b/drivers/net/ethernet/airoha/airoha_npu.c index 8c883f2b2d36..d6547834d3c7 100644 --- a/drivers/net/ethernet/airoha/airoha_npu.c +++ b/drivers/net/ethernet/airoha/airoha_npu.c @@ -154,7 +154,7 @@ static int airoha_npu_send_msg(struct airoha_npu *npu, int func_id, dma_addr_t dma_addr; int ret; - dma_addr = dma_map_single(npu->dev, p, size, DMA_TO_DEVICE); + dma_addr = dma_map_single(npu->dev, p, size, DMA_BIDIRECTIONAL); ret = dma_mapping_error(npu->dev, dma_addr); if (ret) return ret; @@ -177,7 +177,7 @@ static int airoha_npu_send_msg(struct airoha_npu *npu, int func_id, spin_unlock_bh(&npu->cores[core].lock); - dma_unmap_single(npu->dev, dma_addr, size, DMA_TO_DEVICE); + dma_unmap_single(npu->dev, dma_addr, size, DMA_BIDIRECTIONAL); return ret; } diff --git a/drivers/net/ethernet/airoha/airoha_ppe.c b/drivers/net/ethernet/airoha/airoha_ppe.c index 798839aa1010..8defd194568f 100644 --- a/drivers/net/ethernet/airoha/airoha_ppe.c +++ b/drivers/net/ethernet/airoha/airoha_ppe.c @@ -1575,6 +1575,7 @@ void airoha_ppe_deinit(struct airoha_eth *eth) npu = rcu_replace_pointer(eth->npu, NULL, lockdep_is_held(&flow_offload_mutex)); if (npu) { + synchronize_rcu(); npu->ops.ppe_deinit(npu); airoha_npu_put(npu); } diff --git a/drivers/net/ethernet/amd/pds_core/adminq.c b/drivers/net/ethernet/amd/pds_core/adminq.c index 097bb092bdb8..eadb4b604fbe 100644 --- a/drivers/net/ethernet/amd/pds_core/adminq.c +++ b/drivers/net/ethernet/amd/pds_core/adminq.c @@ -18,7 +18,13 @@ static int pdsc_process_notifyq(struct pdsc_qcq *qcq) comp = cq_info->comp; eid = le64_to_cpu(comp->event.eid); while (eid > pdsc->last_eid) { - u16 ecode = le16_to_cpu(comp->event.ecode); + u16 ecode; + + /* Order the payload read after the event id, the field the + * driver uses to detect a new completion. + */ + dma_rmb(); + ecode = le16_to_cpu(comp->event.ecode); switch (ecode) { case PDS_EVENT_LINK_CHANGE: @@ -101,6 +107,10 @@ void pdsc_process_adminq(struct pdsc_qcq *qcq) spin_lock_irqsave(&pdsc->adminq_lock, irqflags); comp = cq->info[cq->tail_idx].comp; while (pdsc_color_match(comp->color, cq->done_color)) { + /* Order the payload reads after the color bit, the field the + * driver uses to detect a new completion. + */ + dma_rmb(); q_info = &q->info[q->tail_idx]; q->tail_idx = (q->tail_idx + 1) & (q->num_descs - 1); diff --git a/drivers/net/ethernet/amd/pds_core/auxbus.c b/drivers/net/ethernet/amd/pds_core/auxbus.c index 92f359f2b449..874812bafc1d 100644 --- a/drivers/net/ethernet/amd/pds_core/auxbus.c +++ b/drivers/net/ethernet/amd/pds_core/auxbus.c @@ -177,17 +177,21 @@ void pdsc_auxbus_dev_del(struct pdsc *cf, struct pdsc *pf, { struct pds_auxiliary_dev *padev; - if (!*pd_ptr) - return; - mutex_lock(&pf->config_lock); + /* A concurrent del may have already torn this device down and + * cleared it. + */ padev = *pd_ptr; + if (!padev) + goto out_unlock; + pds_client_unregister(pf, padev->client_id); auxiliary_device_delete(&padev->aux_dev); auxiliary_device_uninit(&padev->aux_dev); *pd_ptr = NULL; +out_unlock: mutex_unlock(&pf->config_lock); } @@ -210,6 +214,13 @@ int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf, mutex_lock(&pf->config_lock); + /* Nothing to do if the aux device is already present. This also + * guards against a second add overwriting *pd_ptr and leaking the + * first, symmetric with the check in pdsc_auxbus_dev_del(). + */ + if (*pd_ptr) + goto out_unlock; + mask = BIT_ULL(PDSC_S_FW_DEAD) | BIT_ULL(PDSC_S_STOPPING_DRIVER); if (cf->state & mask) { diff --git a/drivers/net/ethernet/amd/pds_core/core.c b/drivers/net/ethernet/amd/pds_core/core.c index 076dfe2910c7..d02e096a2c5f 100644 --- a/drivers/net/ethernet/amd/pds_core/core.c +++ b/drivers/net/ethernet/amd/pds_core/core.c @@ -110,7 +110,6 @@ static void pdsc_qcq_intr_free(struct pdsc *pdsc, struct pdsc_qcq *qcq) return; pdsc_intr_free(pdsc, qcq->intx); - qcq->intx = PDS_CORE_INTR_INDEX_NOT_ASSIGNED; } static int pdsc_qcq_intr_alloc(struct pdsc *pdsc, struct pdsc_qcq *qcq) @@ -145,6 +144,12 @@ void pdsc_qcq_free(struct pdsc *pdsc, struct pdsc_qcq *qcq) pdsc_qcq_intr_free(pdsc, qcq); + /* Drain any work queued by ISR before it was freed above */ + if (qcq->work.func) + cancel_work_sync(&qcq->work); + + qcq->intx = PDS_CORE_INTR_INDEX_NOT_ASSIGNED; + if (qcq->q_base) dma_free_coherent(dev, qcq->q_size, qcq->q_base, qcq->q_base_pa); @@ -304,8 +309,11 @@ int pdsc_qcq_alloc(struct pdsc *pdsc, unsigned int type, unsigned int index, static void pdsc_core_uninit(struct pdsc *pdsc) { - pdsc_qcq_free(pdsc, &pdsc->notifyqcq); + /* Free adminqcq first: its work accesses notifyqcq, so we must + * disable its IRQ and drain its work before freeing notifyqcq. + */ pdsc_qcq_free(pdsc, &pdsc->adminqcq); + pdsc_qcq_free(pdsc, &pdsc->notifyqcq); if (pdsc->kern_dbpage) { iounmap(pdsc->kern_dbpage); @@ -478,8 +486,6 @@ void pdsc_teardown(struct pdsc *pdsc, bool removing) { if (!pdsc->pdev->is_virtfn) pdsc_devcmd_reset(pdsc); - if (pdsc->adminqcq.work.func) - cancel_work_sync(&pdsc->adminqcq.work); pdsc_core_uninit(pdsc); @@ -530,6 +536,7 @@ static void pdsc_adminq_wait_and_dec_once_unused(struct pdsc *pdsc) dev_dbg_ratelimited(pdsc->dev, "%s: adminq in use\n", __func__); cpu_relax(); + cond_resched(); } } @@ -603,9 +610,10 @@ void pdsc_pci_reset_thread(struct work_struct *work) struct pdsc *pdsc = container_of(work, struct pdsc, pci_reset_work); struct pci_dev *pdev = pdsc->pdev; - pci_dev_get(pdev); - pci_reset_function(pdev); - pci_dev_put(pdev); + /* Use try variant to avoid deadlock with pdsc_remove(). + * If lock is contended, the watchdog timer will retry. + */ + pci_try_reset_function(pdev); } static void pdsc_check_pci_health(struct pdsc *pdsc) diff --git a/drivers/net/ethernet/amd/pds_core/devlink.c b/drivers/net/ethernet/amd/pds_core/devlink.c index 621791a3c543..e35572f099c8 100644 --- a/drivers/net/ethernet/amd/pds_core/devlink.c +++ b/drivers/net/ethernet/amd/pds_core/devlink.c @@ -89,6 +89,12 @@ int pdsc_dl_flash_update(struct devlink *dl, { struct pdsc *pdsc = devlink_priv(dl); + if (params->component) { + NL_SET_ERR_MSG_MOD(extack, + "Component update not supported by this device"); + return -EOPNOTSUPP; + } + return pdsc_firmware_update(pdsc, params->fw, extack); } diff --git a/drivers/net/ethernet/amd/pds_core/main.c b/drivers/net/ethernet/amd/pds_core/main.c index c7a2eff57632..a4962dba3140 100644 --- a/drivers/net/ethernet/amd/pds_core/main.c +++ b/drivers/net/ethernet/amd/pds_core/main.c @@ -239,6 +239,10 @@ static int pdsc_init_pf(struct pdsc *pdsc) /* General workqueue and timer, but don't start timer yet */ snprintf(wq_name, sizeof(wq_name), "%s.%d", PDS_CORE_DRV_NAME, pdsc->uid); pdsc->wq = create_singlethread_workqueue(wq_name); + if (!pdsc->wq) { + err = -ENOMEM; + goto err_out_unmap_bars; + } INIT_WORK(&pdsc->health_work, pdsc_health_thread); INIT_WORK(&pdsc->pci_reset_work, pdsc_pci_reset_thread); timer_setup(&pdsc->wdtimer, pdsc_wdtimer_cb, 0); @@ -254,7 +258,7 @@ static int pdsc_init_pf(struct pdsc *pdsc) err = pdsc_setup(pdsc, PDSC_SETUP_INIT); if (err) { mutex_unlock(&pdsc->config_lock); - goto err_out_unmap_bars; + goto err_out_shutdown_timer; } err = pdsc_start(pdsc); @@ -306,13 +310,14 @@ static int pdsc_init_pf(struct pdsc *pdsc) pdsc_stop(pdsc); err_out_teardown: pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING); -err_out_unmap_bars: +err_out_shutdown_timer: timer_shutdown_sync(&pdsc->wdtimer); if (pdsc->wq) destroy_workqueue(pdsc->wq); mutex_destroy(&pdsc->config_lock); mutex_destroy(&pdsc->devcmd_lock); pci_free_irq_vectors(pdsc->pdev); +err_out_unmap_bars: pdsc_unmap_bars(pdsc); err_out_release_regions: pci_release_regions(pdsc->pdev); @@ -436,8 +441,6 @@ static void pdsc_remove(struct pci_dev *pdev) pdsc_auxbus_dev_del(pdsc, pdsc, &pdsc->padev); timer_shutdown_sync(&pdsc->wdtimer); - if (pdsc->wq) - destroy_workqueue(pdsc->wq); mutex_lock(&pdsc->config_lock); set_bit(PDSC_S_STOPPING_DRIVER, &pdsc->state); @@ -445,6 +448,9 @@ static void pdsc_remove(struct pci_dev *pdev) pdsc_stop(pdsc); pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING); mutex_unlock(&pdsc->config_lock); + + if (pdsc->wq) + destroy_workqueue(pdsc->wq); mutex_destroy(&pdsc->config_lock); mutex_destroy(&pdsc->devcmd_lock); diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c b/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c index 7675bb98f029..c2d22fdb6c18 100644 --- a/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c +++ b/drivers/net/ethernet/amd/xgbe/xgbe-mdio.c @@ -267,9 +267,14 @@ static void xgbe_an37_set(struct xgbe_prv_data *pdata, bool enable, XMDIO_WRITE(pdata, MDIO_MMD_VEND2, MDIO_CTRL1, reg); - reg = XMDIO_READ(pdata, MDIO_MMD_VEND2, MDIO_PCS_DIG_CTRL); - reg |= XGBE_VEND2_MAC_AUTO_SW; - XMDIO_WRITE(pdata, MDIO_MMD_VEND2, MDIO_PCS_DIG_CTRL, reg); + if (pdata->an_mode == XGBE_AN_MODE_CL37_SGMII) { + reg = XMDIO_READ(pdata, MDIO_MMD_VEND2, MDIO_PCS_DIG_CTRL); + if (enable) + reg |= XGBE_VEND2_MAC_AUTO_SW; + else + reg &= ~XGBE_VEND2_MAC_AUTO_SW; + XMDIO_WRITE(pdata, MDIO_MMD_VEND2, MDIO_PCS_DIG_CTRL, reg); + } } static void xgbe_an37_restart(struct xgbe_prv_data *pdata) diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index 18d86badd6ea..6fd478572ef8 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -4732,6 +4732,7 @@ static void dpaa2_eth_disconnect_mac(struct dpaa2_eth_priv *priv) dpaa2_mac_disconnect(mac); dpaa2_mac_close(mac); + put_device(&mac->mc_dev->dev); kfree(mac); } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c index 789a46f6938e..846300b77e8a 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c @@ -1511,6 +1511,7 @@ static void dpaa2_switch_port_disconnect_mac(struct ethsw_port_priv *port_priv) dpaa2_mac_disconnect(mac); dpaa2_mac_close(mac); + put_device(&mac->mc_dev->dev); kfree(mac); } diff --git a/drivers/net/ethernet/freescale/fman/fman_dtsec.c b/drivers/net/ethernet/freescale/fman/fman_dtsec.c index 51402dff72c5..52f13b61bc69 100644 --- a/drivers/net/ethernet/freescale/fman/fman_dtsec.c +++ b/drivers/net/ethernet/freescale/fman/fman_dtsec.c @@ -900,22 +900,28 @@ static void dtsec_mac_config(struct phylink_config *config, unsigned int mode, { struct mac_device *mac_dev = fman_config_to_mac(config); struct dtsec_regs __iomem *regs = mac_dev->fman_mac->regs; - u32 tmp; + u32 ecntrl, maccfg2; + + maccfg2 = ioread32be(®s->maccfg2); + maccfg2 &= ~(MACCFG2_NIBBLE_MODE | MACCFG2_BYTE_MODE); switch (state->interface) { case PHY_INTERFACE_MODE_RMII: - tmp = DTSEC_ECNTRL_RMM; + ecntrl = DTSEC_ECNTRL_RMM; + maccfg2 |= MACCFG2_NIBBLE_MODE; break; case PHY_INTERFACE_MODE_RGMII: case PHY_INTERFACE_MODE_RGMII_ID: case PHY_INTERFACE_MODE_RGMII_RXID: case PHY_INTERFACE_MODE_RGMII_TXID: - tmp = DTSEC_ECNTRL_GMIIM | DTSEC_ECNTRL_RPM; + ecntrl = DTSEC_ECNTRL_GMIIM | DTSEC_ECNTRL_RPM; + maccfg2 |= MACCFG2_BYTE_MODE; break; case PHY_INTERFACE_MODE_SGMII: case PHY_INTERFACE_MODE_1000BASEX: case PHY_INTERFACE_MODE_2500BASEX: - tmp = DTSEC_ECNTRL_TBIM | DTSEC_ECNTRL_SGMIIM; + ecntrl = DTSEC_ECNTRL_TBIM | DTSEC_ECNTRL_SGMIIM; + maccfg2 |= MACCFG2_BYTE_MODE; break; default: dev_warn(mac_dev->dev, "cannot configure dTSEC for %s\n", @@ -923,7 +929,8 @@ static void dtsec_mac_config(struct phylink_config *config, unsigned int mode, return; } - iowrite32be(tmp, ®s->ecntrl); + iowrite32be(ecntrl, ®s->ecntrl); + iowrite32be(maccfg2, ®s->maccfg2); } static void dtsec_link_up(struct phylink_config *config, struct phy_device *phy, diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index 4cc6dcbfd367..9e7d6860b83f 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -41,6 +42,7 @@ /* Interval to schedule a stats report update, 20000ms. */ #define GVE_STATS_REPORT_TIMER_PERIOD 20000 +#define GVE_RX_NAPI_RESCHED_MS 20 /* msecs */ /* Numbers of NIC tx/rx stats in stats report. */ #define NIC_TX_STATS_REPORT_NUM 0 @@ -338,6 +340,7 @@ struct gve_rx_ring { struct xdp_rxq_info xdp_rxq; struct xsk_buff_pool *xsk_pool; struct page_frag_cache page_cache; /* Page cache to allocate XDP frames */ + struct timer_list starvation_timer; /* for queue starvation recovery */ }; /* A TX desc ring entry */ diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c index 6a0430a93ed7..fc0c7991c3a2 100644 --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c @@ -18,6 +18,16 @@ #include #include +static void gve_rx_starvation_timer(struct timer_list *t) +{ + struct gve_rx_ring *rx = timer_container_of(rx, t, starvation_timer); + struct gve_priv *priv = rx->gve; + struct gve_notify_block *block; + + block = &priv->ntfy_blocks[rx->ntfy_id]; + napi_schedule(&block->napi); +} + static void gve_rx_free_hdr_bufs(struct gve_priv *priv, struct gve_rx_ring *rx) { struct device *hdev = &priv->pdev->dev; @@ -120,6 +130,7 @@ void gve_rx_stop_ring_dqo(struct gve_priv *priv, int idx) if (rx->dqo.page_pool) page_pool_disable_direct_recycling(rx->dqo.page_pool); + timer_shutdown_sync(&rx->starvation_timer); gve_remove_napi(priv, ntfy_idx); gve_rx_remove_from_block(priv, idx); gve_rx_reset_ring_dqo(priv, idx); @@ -208,8 +219,10 @@ static int gve_rx_alloc_hdr_bufs(struct gve_priv *priv, struct gve_rx_ring *rx, void gve_rx_start_ring_dqo(struct gve_priv *priv, int idx) { int ntfy_idx = gve_rx_idx_to_ntfy(priv, idx); + struct gve_rx_ring *rx = &priv->rx[idx]; gve_rx_add_to_block(priv, idx); + timer_setup(&rx->starvation_timer, gve_rx_starvation_timer, 0); gve_add_napi(priv, ntfy_idx, gve_napi_poll_dqo); } @@ -363,6 +376,7 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx) struct gve_rx_compl_queue_dqo *complq = &rx->dqo.complq; struct gve_rx_buf_queue_dqo *bufq = &rx->dqo.bufq; struct gve_priv *priv = rx->gve; + u32 num_bufs_avail_to_hw; u32 num_avail_slots; u32 num_full_slots; u32 num_posted = 0; @@ -398,6 +412,26 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx) } rx->fill_cnt += num_posted; + + /* If the queue has fewer than GVE_RX_BUF_THRESH_DQO descriptors + * visible to the hardware, the hardware is in danger of starving + * and cannot trigger interrupts. + * + * We use a threshold of 32 because a single maximum-sized RSC + * packet can consume up to 19 descriptors in the Rx path. Lower + * thresholds (e.g., 8 or 16) would be unsafe as they could cause + * the device to drop/stall on a maximum-sized RSC packet. + * + * Start the timer to periodically reschedule NAPI and recover. + */ + num_bufs_avail_to_hw = + ((bufq->tail & ~(GVE_RX_BUF_THRESH_DQO - 1)) - + bufq->head) & bufq->mask; + + if (num_bufs_avail_to_hw < GVE_RX_BUF_THRESH_DQO) { + mod_timer(&rx->starvation_timer, + jiffies + msecs_to_jiffies(GVE_RX_NAPI_RESCHED_MS)); + } } static void gve_rx_skb_csum(struct sk_buff *skb, diff --git a/drivers/net/ethernet/hisilicon/hip04_eth.c b/drivers/net/ethernet/hisilicon/hip04_eth.c index 18376bcc718a..fc2c47dcfaab 100644 --- a/drivers/net/ethernet/hisilicon/hip04_eth.c +++ b/drivers/net/ethernet/hisilicon/hip04_eth.c @@ -594,7 +594,11 @@ static int hip04_rx_poll(struct napi_struct *napi, int budget) skb = build_skb(buf, priv->rx_buf_size); if (unlikely(!skb)) { net_dbg_ratelimited("build_skb failed\n"); - goto refill; + /* Retain the slot; return budget so NAPI retries this + * buffer. Refill would overwrite rx_buf[]/rx_phys[] + * and leak them. + */ + return budget; } dma_unmap_single(priv->dev, priv->rx_phys[priv->rx_head], @@ -622,14 +626,15 @@ static int hip04_rx_poll(struct napi_struct *napi, int budget) rx++; } -refill: buf = netdev_alloc_frag(priv->rx_buf_size); if (!buf) goto done; phys = dma_map_single(priv->dev, buf, RX_BUF_SIZE, DMA_FROM_DEVICE); - if (dma_mapping_error(priv->dev, phys)) + if (dma_mapping_error(priv->dev, phys)) { + skb_free_frag(buf); goto done; + } priv->rx_buf[priv->rx_head] = buf; priv->rx_phys[priv->rx_head] = phys; hip04_set_recv_desc(priv, phys); diff --git a/drivers/net/ethernet/huawei/hinic/hinic_dev.h b/drivers/net/ethernet/huawei/hinic/hinic_dev.h index 52ea97c818b8..d9ab94910a2a 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_dev.h +++ b/drivers/net/ethernet/huawei/hinic/hinic_dev.h @@ -104,8 +104,6 @@ struct hinic_dev { u16 num_rss; u16 rss_limit; struct hinic_rss_type rss_type; - u8 *rss_hkey_user; - s32 *rss_indir_user; struct hinic_intr_coal_info *rx_intr_coalesce; struct hinic_intr_coal_info *tx_intr_coalesce; struct hinic_sriov_info sriov_info; diff --git a/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c b/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c index e9f338e9dbe7..cd4c295ec7ae 100644 --- a/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c +++ b/drivers/net/ethernet/huawei/hinic/hinic_ethtool.c @@ -1064,17 +1064,6 @@ static int __set_rss_rxfh(struct net_device *netdev, int err; if (indir) { - if (!nic_dev->rss_indir_user) { - nic_dev->rss_indir_user = - kzalloc(sizeof(u32) * HINIC_RSS_INDIR_SIZE, - GFP_KERNEL); - if (!nic_dev->rss_indir_user) - return -ENOMEM; - } - - memcpy(nic_dev->rss_indir_user, indir, - sizeof(u32) * HINIC_RSS_INDIR_SIZE); - err = hinic_rss_set_indir_tbl(nic_dev, nic_dev->rss_tmpl_idx, indir); if (err) @@ -1082,16 +1071,6 @@ static int __set_rss_rxfh(struct net_device *netdev, } if (key) { - if (!nic_dev->rss_hkey_user) { - nic_dev->rss_hkey_user = - kzalloc(HINIC_RSS_KEY_SIZE * 2, GFP_KERNEL); - - if (!nic_dev->rss_hkey_user) - return -ENOMEM; - } - - memcpy(nic_dev->rss_hkey_user, key, HINIC_RSS_KEY_SIZE); - err = hinic_rss_set_template_tbl(nic_dev, nic_dev->rss_tmpl_idx, key); if (err) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index 688618804376..fc4eae0a00cd 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -765,6 +765,9 @@ static inline bool ice_is_txtime_ena(const struct ice_tx_ring *ring) struct ice_vsi *vsi = ring->vsi; struct ice_pf *pf = vsi->back; + if (vsi->type != ICE_VSI_PF) + return false; + return test_bit(ring->q_index, pf->txtime_txqs); } diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.c b/drivers/net/ethernet/intel/ice/ice_eswitch.c index c30e27bbfe6e..b069e6c514fb 100644 --- a/drivers/net/ethernet/intel/ice/ice_eswitch.c +++ b/drivers/net/ethernet/intel/ice/ice_eswitch.c @@ -512,9 +512,6 @@ int ice_eswitch_attach_vf(struct ice_pf *pf, struct ice_vf *vf) struct ice_repr *repr; int err; - if (!ice_is_eswitch_mode_switchdev(pf)) - return 0; - repr = ice_repr_create_vf(vf); if (IS_ERR(repr)) return PTR_ERR(repr); diff --git a/drivers/net/ethernet/intel/ice/ice_lag.c b/drivers/net/ethernet/intel/ice/ice_lag.c index aebf8e08a297..e8ab36d0f11d 100644 --- a/drivers/net/ethernet/intel/ice/ice_lag.c +++ b/drivers/net/ethernet/intel/ice/ice_lag.c @@ -2624,7 +2624,7 @@ int ice_init_lag(struct ice_pf *pf) goto free_lport_res; /* associate recipes to profiles */ - for (n = 0; n < ICE_PROFID_IPV6_GTPU_IPV6_TCP_INNER; n++) { + for (n = 0; n < ICE_MAX_NUM_PROFILES; n++) { err = ice_aq_get_recipe_to_profile(&pf->hw, n, &recipe_bits, NULL); if (err) diff --git a/drivers/net/ethernet/intel/ice/ice_parser.c b/drivers/net/ethernet/intel/ice/ice_parser.c index 664beb64f557..0c240e90a54e 100644 --- a/drivers/net/ethernet/intel/ice/ice_parser.c +++ b/drivers/net/ethernet/intel/ice/ice_parser.c @@ -2368,6 +2368,9 @@ int ice_parser_profile_init(struct ice_parser_result *rslt, u16 proto_off = 0; u16 off; + if (rslt->ptype >= ICE_FLOW_PTYPE_MAX) + return -EINVAL; + memset(prof, 0, sizeof(*prof)); set_bit(rslt->ptype, prof->ptypes); if (blk == ICE_BLK_SW) { diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c index 86eb3d0315a2..30a34ed2ae60 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp.c +++ b/drivers/net/ethernet/intel/ice/ice_ptp.c @@ -350,7 +350,7 @@ static u64 ice_ptp_extend_40b_ts(struct ice_pf *pf, u64 in_tstamp) return 0; } - return ice_ptp_extend_32b_ts(pf->ptp.cached_phc_time, + return ice_ptp_extend_32b_ts(READ_ONCE(pf->ptp.cached_phc_time), (in_tstamp >> 8) & mask); } @@ -2994,6 +2994,11 @@ void ice_ptp_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type) struct ice_ptp *ptp = &pf->ptp; int err; + if (ptp->state == ICE_PTP_UNINIT) { + dev_dbg(ice_pf_to_dev(pf), "PTP was not initialized, skipping rebuild\n"); + return; + } + if (ptp->state == ICE_PTP_READY) { ice_ptp_prepare_for_reset(pf, reset_type); } else if (ptp->state != ICE_PTP_RESETTING) { diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c index 843e82fd3bf9..6a0b724e46f9 100644 --- a/drivers/net/ethernet/intel/ice/ice_sriov.c +++ b/drivers/net/ethernet/intel/ice/ice_sriov.c @@ -484,12 +484,14 @@ static int ice_start_vfs(struct ice_pf *pf) goto teardown; } - retval = ice_eswitch_attach_vf(pf, vf); - if (retval) { - dev_err(ice_pf_to_dev(pf), "Failed to attach VF %d to eswitch, error %d", - vf->vf_id, retval); - ice_vf_vsi_release(vf); - goto teardown; + if (ice_is_eswitch_mode_switchdev(pf)) { + retval = ice_eswitch_attach_vf(pf, vf); + if (retval) { + dev_err(ice_pf_to_dev(pf), "Failed to attach VF %d to eswitch, error %d", + vf->vf_id, retval); + ice_vf_vsi_release(vf); + goto teardown; + } } set_bit(ICE_VF_STATE_INIT, vf->vf_states); diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c index 4f86412a9c0c..bab968b05540 100644 --- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c @@ -812,7 +812,8 @@ void ice_reset_all_vfs(struct ice_pf *pf) } ice_vf_post_vsi_rebuild(vf); - ice_eswitch_attach_vf(pf, vf); + if (ice_is_eswitch_mode_switchdev(pf)) + ice_eswitch_attach_vf(pf, vf); mutex_unlock(&vf->cfg_lock); } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c index ad733f4da3c1..019bd49015c6 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -435,7 +435,7 @@ struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc) return &rvu->pf[rvu_get_pf(rvu->pdev, pcifunc)]; } -static bool is_pf_func_valid(struct rvu *rvu, u16 pcifunc) +bool is_pf_func_valid(struct rvu *rvu, u16 pcifunc) { int pf, vf, nvfs; u64 cfg; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h index 914ba2b691ec..c9ed5276690d 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -900,6 +900,7 @@ u16 rvu_get_rsrc_mapcount(struct rvu_pfvf *pfvf, int blkaddr); struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc); void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf); bool is_block_implemented(struct rvu_hwinfo *hw, int blkaddr); +bool is_pf_func_valid(struct rvu *rvu, u16 pcifunc); bool is_pffunc_map_valid(struct rvu *rvu, u16 pcifunc, int blktype); int rvu_get_lf(struct rvu *rvu, struct rvu_block *block, u16 pcifunc, u16 slot); int rvu_lf_reset(struct rvu *rvu, struct rvu_block *block, int lf); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c index d2163da28d18..fa4ea1258d29 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c @@ -178,6 +178,15 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu, * pcifunc (will be the one who is calling this mailbox). */ if (req->base_pcifunc) { + /* A VF is untrusted and must not redirect its LMTLINE to + * another PF's region, so confine VF callers to their own PF. + */ + if (is_vf(req->hdr.pcifunc) && + (!is_pf_func_valid(rvu, req->base_pcifunc) || + rvu_get_pf(rvu->pdev, req->hdr.pcifunc) != + rvu_get_pf(rvu->pdev, req->base_pcifunc))) + return -EPERM; + /* Calculating the LMT table index equivalent to primary * pcifunc. */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c index 4415d0ce9aef..d43c034b0264 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c @@ -97,6 +97,14 @@ int rvu_mbox_handler_rep_event_notify(struct rvu *rvu, struct rep_event *req, { struct rep_evtq_ent *qentry; + /* The mailbox dispatcher normalises only the header pcifunc; the + * nested struct rep_event::pcifunc body field is sender-controlled + * and is later used by rvu_rep_up_notify() to index rvu->pf[] / + * rvu->hwvf[]. Reject out-of-range body selectors before queueing. + */ + if (!is_pf_func_valid(rvu, req->pcifunc)) + return -EINVAL; + qentry = kmalloc(sizeof(*qentry), GFP_ATOMIC); if (!qentry) return -ENOMEM; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c index 64c6d9162ef6..17e8f6e51808 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c @@ -146,6 +146,7 @@ int otx2_alloc_mcam_entries(struct otx2_nic *pfvf, u16 count) if (allocated) { pfvf->flags |= OTX2_FLAG_MCAM_ENTRIES_ALLOC; pfvf->flags |= OTX2_FLAG_NTUPLE_SUPPORT; + pfvf->flags |= OTX2_FLAG_TC_FLOWER_SUPPORT; } if (allocated != count) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c index 26a08d2cfbb1..f4906c67d5ee 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c @@ -30,6 +30,7 @@ #define OTX2_UNSUPP_LSE_DEPTH GENMASK(6, 4) #define MCAST_INVALID_GRP (-1U) +#define RATE_MANTISSA_BITS 8 struct otx2_tc_flow_stats { u64 bytes; @@ -90,28 +91,30 @@ static void otx2_get_egress_burst_cfg(struct otx2_nic *nic, u32 burst, static void otx2_get_egress_rate_cfg(u64 maxrate, u32 *exp, u32 *mantissa, u32 *div_exp) { - u64 tmp; - /* Rate calculation by hardware * * PIR_ADD = ((256 + mantissa) << exp) / 256 * rate = (2 * PIR_ADD) / ( 1 << div_exp) * The resultant rate is in Mbps. + * + * Use div_exp = 0 and compute exp/mantissa for maxrate / 2; the + * leading factor of two yields the full rate. Rates below 2 Mbps + * are floored to the smallest step (exp = 0, mantissa = 0). */ - /* 2Mbps to 100Gbps can be expressed with div_exp = 0. - * Setting this to '0' will ease the calculation of - * exponent and mantissa. - */ *div_exp = 0; - if (maxrate) { - *exp = ilog2(maxrate) ? ilog2(maxrate) - 1 : 0; - tmp = maxrate - rounddown_pow_of_two(maxrate); - if (maxrate < MAX_RATE_MANTISSA) - *mantissa = tmp * 2; - else - *mantissa = tmp / (1ULL << (*exp - 7)); + maxrate = maxrate / 2; + if (!maxrate) { + /* Rates below 2 Mbps map to the smallest step */ + *exp = 0; + *mantissa = 0; + } else { + *exp = ilog2(maxrate); + /* Clear MSB and derive fractional bits */ + maxrate &= ~BIT(*exp); + *mantissa = (maxrate << RATE_MANTISSA_BITS) >> *exp; + } } else { /* Instead of disabling rate limiting, set all values to max */ *exp = MAX_RATE_EXPONENT; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c b/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c index cf8f14ce4cd5..20db77552545 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c @@ -158,6 +158,13 @@ static int mlx5e_dcbnl_ieee_getets(struct net_device *netdev, } memcpy(ets->tc_tsa, priv->dcbx.tc_tsa, sizeof(ets->tc_tsa)); + /* Report 0 for non ETS TSA */ + for (i = 0; i < ets->ets_cap; i++) { + if (ets->tc_tx_bw[i] == MLX5E_MAX_BW_ALLOC && + priv->dcbx.tc_tsa[i] != IEEE_8021QAZ_TSA_ETS) + ets->tc_tx_bw[i] = 0; + } + return err; } @@ -302,6 +309,14 @@ static int mlx5e_dbcnl_validate_ets(struct net_device *netdev, } } + for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) { + if (ets->tc_tsa[i] == IEEE_8021QAZ_TSA_CB_SHAPER) { + netdev_err(netdev, + "Failed to validate ETS: CB Shaper is not supported\n"); + return -EOPNOTSUPP; + } + } + /* Validate Bandwidth Sum */ for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) { if (ets->tc_tsa[i] == IEEE_8021QAZ_TSA_ETS) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c index 893e1380a7c9..8a07b9e8dc38 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c @@ -2027,7 +2027,7 @@ static int mlx5e_get_module_info(struct net_device *netdev, int size_read = 0; u8 data[4] = {0}; - size_read = mlx5_query_module_eeprom(dev, 0, 2, data); + size_read = mlx5_query_module_eeprom(dev, 0, 2, data, NULL); if (size_read < 2) return -EIO; @@ -2069,6 +2069,7 @@ static int mlx5e_get_module_eeprom(struct net_device *netdev, struct mlx5_core_dev *mdev = priv->mdev; int offset = ee->offset; int size_read; + u8 status = 0; int i = 0; if (!ee->len) @@ -2078,15 +2079,15 @@ static int mlx5e_get_module_eeprom(struct net_device *netdev, while (i < ee->len) { size_read = mlx5_query_module_eeprom(mdev, offset, ee->len - i, - data + i); - + data + i, &status); if (!size_read) /* Done reading */ return 0; if (size_read < 0) { - netdev_err(priv->netdev, "%s: mlx5_query_eeprom failed:0x%x\n", - __func__, size_read); + netdev_err(netdev, + "%s: mlx5_query_eeprom failed:0x%x, status %u\n", + __func__, size_read, status); return size_read; } @@ -2106,6 +2107,7 @@ static int mlx5e_get_module_eeprom_by_page(struct net_device *netdev, struct mlx5_core_dev *mdev = priv->mdev; u8 *data = page_data->data; int size_read; + u8 status = 0; int i = 0; if (!page_data->length) @@ -2119,7 +2121,8 @@ static int mlx5e_get_module_eeprom_by_page(struct net_device *netdev, query.page = page_data->page; while (i < page_data->length) { query.size = page_data->length - i; - size_read = mlx5_query_module_eeprom_by_page(mdev, &query, data + i); + size_read = mlx5_query_module_eeprom_by_page(mdev, &query, + data + i, &status); /* Done reading, return how many bytes was read */ if (!size_read) @@ -2128,8 +2131,8 @@ static int mlx5e_get_module_eeprom_by_page(struct net_device *netdev, if (size_read < 0) { NL_SET_ERR_MSG_FMT_MOD( extack, - "Query module eeprom by page failed, read %u bytes, err %d", - i, size_read); + "Query module eeprom by page failed, read %u bytes, err %d, status %u", + i, size_read, status); return size_read; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index f660ef37b0a6..7e754b72f889 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -211,11 +211,11 @@ static void mlx5e_disable_async_events(struct mlx5e_priv *priv) static int mlx5e_devcom_event_mpv(int event, void *my_data, void *event_data) { - struct mlx5e_priv *slave_priv = my_data; + struct mlx5e_priv *master_priv = event_data; switch (event) { case MPV_DEVCOM_MASTER_UP: - mlx5_devcom_comp_set_ready(slave_priv->devcom, true); + mlx5_devcom_comp_set_ready(master_priv->devcom, true); break; case MPV_DEVCOM_MASTER_DOWN: /* no need for comp set ready false since we unregister after diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/helper.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/helper.c index 3ce455c2535c..53f064ebd1c1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/helper.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/helper.c @@ -71,7 +71,7 @@ int esw_egress_acl_vlan_create(struct mlx5_eswitch *esw, flow_act.action = flow_action; vport->egress.allowed_vlan = mlx5_add_flow_rules(vport->egress.acl, spec, - &flow_act, fwd_dest, 0); + &flow_act, fwd_dest, fwd_dest ? 1 : 0); if (IS_ERR(vport->egress.allowed_vlan)) { err = PTR_ERR(vport->egress.allowed_vlan); esw_warn(esw->dev, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index 09c544bdf70d..e08903d002a1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -358,11 +358,11 @@ int mlx5_set_port_fcs(struct mlx5_core_dev *mdev, u8 enable); void mlx5_query_port_fcs(struct mlx5_core_dev *mdev, bool *supported, bool *enabled); int mlx5_query_module_eeprom(struct mlx5_core_dev *dev, - u16 offset, u16 size, u8 *data); + u16 offset, u16 size, u8 *data, u8 *status); int mlx5_query_module_eeprom_by_page(struct mlx5_core_dev *dev, struct mlx5_module_eeprom_query_params *params, - u8 *data); + u8 *data, u8 *status); int mlx5_query_port_dcbx_param(struct mlx5_core_dev *mdev, u32 *out); int mlx5_set_port_dcbx_param(struct mlx5_core_dev *mdev, u32 *in); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/port.c b/drivers/net/ethernet/mellanox/mlx5/core/port.c index 876e648c91ba..23796fb353e4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/port.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/port.c @@ -289,11 +289,11 @@ int mlx5_query_module_num(struct mlx5_core_dev *dev, int *module_num) } static int mlx5_query_module_id(struct mlx5_core_dev *dev, int module_num, - u8 *module_id) + u8 *module_id, u8 *status) { u32 in[MLX5_ST_SZ_DW(mcia_reg)] = {}; u32 out[MLX5_ST_SZ_DW(mcia_reg)]; - int err, status; + int err; u8 *ptr; MLX5_SET(mcia_reg, in, i2c_device_address, MLX5_I2C_ADDR_LOW); @@ -308,13 +308,13 @@ static int mlx5_query_module_id(struct mlx5_core_dev *dev, int module_num, if (err) return err; - status = MLX5_GET(mcia_reg, out, status); - if (status) { - mlx5_core_err(dev, "query_mcia_reg failed: status: 0x%x\n", - status); + if (MLX5_GET(mcia_reg, out, status)) { + if (status) + *status = MLX5_GET(mcia_reg, out, status); return -EIO; } - ptr = MLX5_ADDR_OF(mcia_reg, out, dword_0); + + ptr = MLX5_ADDR_OF(mcia_reg, out, dwords); *module_id = ptr[0]; @@ -370,13 +370,14 @@ static int mlx5_mcia_max_bytes(struct mlx5_core_dev *dev) } static int mlx5_query_mcia(struct mlx5_core_dev *dev, - struct mlx5_module_eeprom_query_params *params, u8 *data) + struct mlx5_module_eeprom_query_params *params, + u8 *data, u8 *status) { u32 in[MLX5_ST_SZ_DW(mcia_reg)] = {}; u32 out[MLX5_ST_SZ_DW(mcia_reg)]; - int status, err; void *ptr; u16 size; + int err; size = min_t(int, params->size, mlx5_mcia_max_bytes(dev)); @@ -392,21 +393,20 @@ static int mlx5_query_mcia(struct mlx5_core_dev *dev, if (err) return err; - status = MLX5_GET(mcia_reg, out, status); - if (status) { - mlx5_core_err(dev, "query_mcia_reg failed: status: 0x%x\n", - status); + if (MLX5_GET(mcia_reg, out, status)) { + if (status) + *status = MLX5_GET(mcia_reg, out, status); return -EIO; } - ptr = MLX5_ADDR_OF(mcia_reg, out, dword_0); + ptr = MLX5_ADDR_OF(mcia_reg, out, dwords); memcpy(data, ptr, size); return size; } int mlx5_query_module_eeprom(struct mlx5_core_dev *dev, - u16 offset, u16 size, u8 *data) + u16 offset, u16 size, u8 *data, u8 *status) { struct mlx5_module_eeprom_query_params query = {0}; u8 module_id; @@ -416,7 +416,8 @@ int mlx5_query_module_eeprom(struct mlx5_core_dev *dev, if (err) return err; - err = mlx5_query_module_id(dev, query.module_number, &module_id); + err = mlx5_query_module_id(dev, query.module_number, &module_id, + status); if (err) return err; @@ -442,12 +443,12 @@ int mlx5_query_module_eeprom(struct mlx5_core_dev *dev, query.size = size; query.offset = offset; - return mlx5_query_mcia(dev, &query, data); + return mlx5_query_mcia(dev, &query, data, status); } int mlx5_query_module_eeprom_by_page(struct mlx5_core_dev *dev, struct mlx5_module_eeprom_query_params *params, - u8 *data) + u8 *data, u8 *status) { int err; @@ -461,7 +462,7 @@ int mlx5_query_module_eeprom_by_page(struct mlx5_core_dev *dev, return -EINVAL; } - return mlx5_query_mcia(dev, params, data); + return mlx5_query_mcia(dev, params, data, status); } static int mlx5_query_port_pvlc(struct mlx5_core_dev *dev, u32 *pvlc, diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c index 243f8d11be68..81cbc47de0c6 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -125,6 +125,8 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) } else { /* If dynamic allocation is enabled we have already allocated * hwc msi + * Also, we make sure in this case the following is always true + * (num_msix_usable - 1 HWC) <= num_online_cpus() */ gc->num_msix_usable = min(resp.max_msix, num_online_cpus() + 1); } @@ -1587,8 +1589,8 @@ void mana_gd_free_res_map(struct gdma_resource *r) * do the same thing. */ -static int irq_setup(unsigned int *irqs, unsigned int len, int node, - bool skip_first_cpu) +static int mana_irq_setup_numa_aware(unsigned int *irqs, unsigned int len, + int node, bool skip_first_cpu) { const struct cpumask *next, *prev = cpu_none_mask; cpumask_var_t cpus __free(free_cpumask_var); @@ -1624,11 +1626,24 @@ static int irq_setup(unsigned int *irqs, unsigned int len, int node, return 0; } +/* must be called with cpus_read_lock() held */ +static void mana_irq_setup_linear(unsigned int *irqs, unsigned int len) +{ + int cpu; + + for_each_online_cpu(cpu) { + if (len == 0) + break; + + irq_set_affinity_and_hint(*irqs++, cpumask_of(cpu)); + len--; + } +} + static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) { struct gdma_context *gc = pci_get_drvdata(pdev); struct gdma_irq_context *gic; - bool skip_first_cpu = false; int *irqs, irq, err, i; irqs = kmalloc_array(nvec, sizeof(int), GFP_KERNEL); @@ -1636,10 +1651,12 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) return -ENOMEM; /* + * In this function, num_msix_usable = HWC IRQ + Queue IRQ. + * nvec is only Queue IRQ (HWC already setup). * While processing the next pci irq vector, we start with index 1, * as IRQ vector at index 0 is already processed for HWC. * However, the population of irqs array starts with index 0, to be - * further used in irq_setup() + * further used in mana_irq_setup_numa_aware() */ for (i = 1; i <= nvec; i++) { gic = kzalloc(sizeof(*gic), GFP_KERNEL); @@ -1669,18 +1686,51 @@ static int mana_gd_setup_dyn_irqs(struct pci_dev *pdev, int nvec) } /* - * When calling irq_setup() for dynamically added IRQs, if number of - * CPUs is more than or equal to allocated MSI-X, we need to skip the - * first CPU sibling group since they are already affinitized to HWC IRQ + * When calling mana_irq_setup_numa_aware() for dynamically added IRQs, + * if number of CPUs is more than or equal to allocated MSI-X, we need to + * skip the first CPU sibling group since they are already affinitized to + * HWC IRQ */ cpus_read_lock(); - if (gc->num_msix_usable <= num_online_cpus()) - skip_first_cpu = true; + if (gc->num_msix_usable <= num_online_cpus()) { + err = mana_irq_setup_numa_aware(irqs, nvec, gc->numa_node, + true); + if (err) { + cpus_read_unlock(); + goto free_irq; + } + } else { + /* + * When num_msix_usable are more than num_online_cpus, our + * queue IRQs should be equal to num of online vCPUs. + * We try to make sure queue IRQs spread across all vCPUs. + * In such a case NUMA or CPU core affinity does not matter. + * Note: in this case the total mana IRQ should always be + * num_online_cpus + 1. The first HWC IRQ is already handled + * in HWC setup calls + * However, if CPUs went offline since num_msix_usable was + * computed, queue IRQs will be more than num_online_cpus(). + * In such cases remaining extra IRQs will retain their default + * affinity. + */ + int first_unassigned = num_online_cpus(); - err = irq_setup(irqs, nvec, gc->numa_node, skip_first_cpu); - if (err) { - cpus_read_unlock(); - goto free_irq; + if (nvec > first_unassigned) { + char buf[32]; + + if (first_unassigned == nvec - 1) + snprintf(buf, sizeof(buf), "%d", + first_unassigned); + else + snprintf(buf, sizeof(buf), "%d-%d", + first_unassigned, nvec - 1); + + dev_dbg(&pdev->dev, + "MANA IRQ indices #%s will retain the default CPU affinity\n", + buf); + } + + mana_irq_setup_linear(irqs, nvec); } cpus_read_unlock(); @@ -1766,7 +1816,7 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev, int nvec) nvec -= 1; } - err = irq_setup(irqs, nvec, gc->numa_node, false); + err = mana_irq_setup_numa_aware(irqs, nvec, gc->numa_node, false); if (err) { cpus_read_unlock(); goto free_irq; diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c index 62418f164dba..adf25acb0d46 100644 --- a/drivers/net/ethernet/microsoft/mana/mana_en.c +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c @@ -2160,6 +2160,19 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq, rxbuf_oob = &rxq->rx_oobs[curr]; WARN_ON_ONCE(rxbuf_oob->wqe_inf.wqe_size_in_bu != 1); + if (unlikely(pktlen > rxq->datasize)) { + /* Increase it even if mana_rx_skb() isn't called. */ + rxq->rx_cq.work_done++; + + ++ndev->stats.rx_dropped; + netdev_warn_once(ndev, + "Dropped oversized RX packet: len=%u, datasize=%u\n", + pktlen, rxq->datasize); + + /* Reuse the RX buffer since rxbuf_oob is unchanged. */ + goto drop; + } + mana_refill_rx_oob(dev, rxq, rxbuf_oob, &old_buf, &old_fp); /* Unsuccessful refill will have old_buf == NULL. diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c index 279ea0b56955..55525f45e447 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c @@ -96,6 +96,9 @@ static int nfp_cpp_resource_find(struct nfp_cpp *cpp, struct nfp_resource *res) res->mutex = nfp_cpp_mutex_alloc(cpp, NFP_RESOURCE_TBL_TARGET, addr, key); + if (!res->mutex) + return -ENOMEM; + res->cpp_id = NFP_CPP_ID(entry.region.cpp_target, entry.region.cpp_action, entry.region.cpp_token); diff --git a/drivers/net/ethernet/realtek/rtase/rtase.h b/drivers/net/ethernet/realtek/rtase/rtase.h index b9209eb6ea73..43087b17a0ff 100644 --- a/drivers/net/ethernet/realtek/rtase/rtase.h +++ b/drivers/net/ethernet/realtek/rtase/rtase.h @@ -188,6 +188,12 @@ enum rtase_sw_flag_content { RTASE_SWF_MSIX_ENABLED = BIT(2), }; +enum rtase_parse_result { + RTASE_PARSE_OK, + RTASE_PARSE_SKIP, + RTASE_PARSE_DROP, +}; + #define RSVD_MASK 0x3FFFC000 struct rtase_tx_desc { @@ -359,4 +365,6 @@ struct rtase_private { #define RTASE_MSS_MASK GENMASK(28, 18) +#define RTASE_MIN_PAD_LEN 47 + #endif /* RTASE_H */ diff --git a/drivers/net/ethernet/realtek/rtase/rtase_main.c b/drivers/net/ethernet/realtek/rtase/rtase_main.c index 55105d34bc79..a57a525327a3 100644 --- a/drivers/net/ethernet/realtek/rtase/rtase_main.c +++ b/drivers/net/ethernet/realtek/rtase/rtase_main.c @@ -61,6 +61,7 @@ #include #include #include +#include #include #include #include @@ -1249,6 +1250,199 @@ static u32 rtase_tx_csum(struct sk_buff *skb, const struct net_device *dev) return csum_cmd; } +static enum rtase_parse_result rtase_get_l3_proto(struct sk_buff *skb, + __be16 *proto, + u32 *network_offset) +{ + struct vlan_hdr *vh, _vh; + struct ethhdr *eh, _eh; + u32 offset = ETH_HLEN; + + eh = skb_header_pointer(skb, 0, sizeof(_eh), &_eh); + if (!eh) + return RTASE_PARSE_DROP; + + *proto = eh->h_proto; + + while (eth_type_vlan(*proto)) { + vh = skb_header_pointer(skb, offset, sizeof(_vh), &_vh); + if (!vh) + return RTASE_PARSE_DROP; + + *proto = vh->h_vlan_encapsulated_proto; + offset += VLAN_HLEN; + } + + *network_offset = offset; + + return RTASE_PARSE_OK; +} + +static bool rtase_pad_to_transport_len(struct sk_buff *skb, + u32 transport_offset, + u32 pad_to_len) +{ + u32 trans_data_len; + u32 pad_len; + + trans_data_len = skb->len - transport_offset; + if (trans_data_len >= pad_to_len) + return true; + + if (skb_is_nonlinear(skb)) { + if (skb_linearize(skb)) + return false; + } + + pad_len = pad_to_len - trans_data_len; + if (__skb_put_padto(skb, skb->len + pad_len, false)) + return false; + + return true; +} + +static enum rtase_parse_result rtase_get_transport_offset(struct sk_buff *skb, + u32 *transport_offset, + u8 *transport_proto, + u32 *pad_to_len) +{ + enum rtase_parse_result ret; + struct ipv6hdr *i6h, _i6h; + struct iphdr *ih, _ih; + bool non_first_frag; + __be16 proto; + u32 offset; + u32 no; + + ret = rtase_get_l3_proto(skb, &proto, &no); + if (ret != RTASE_PARSE_OK) + return ret; + + switch (proto) { + case htons(ETH_P_IP): + ih = skb_header_pointer(skb, no, sizeof(_ih), &_ih); + if (!ih) + return RTASE_PARSE_DROP; + + if (ih->ihl < 5) + return RTASE_PARSE_DROP; + + offset = no + ih->ihl * 4; + if (offset > skb->len) + return RTASE_PARSE_DROP; + + non_first_frag = ntohs(ih->frag_off) & IP_OFFSET; + + if (ih->protocol == IPPROTO_TCP) { + if (skb->len - offset < sizeof(struct tcphdr)) { + if (non_first_frag) { + *transport_offset = offset; + *transport_proto = IPPROTO_TCP; + *pad_to_len = sizeof(struct tcphdr); + + return RTASE_PARSE_OK; + } + + return RTASE_PARSE_DROP; + } + + return RTASE_PARSE_SKIP; + } + + if (ih->protocol != IPPROTO_UDP) + return RTASE_PARSE_SKIP; + + *transport_offset = offset; + *transport_proto = IPPROTO_UDP; + + if (skb->len - offset < sizeof(struct udphdr)) { + if (non_first_frag) { + *pad_to_len = sizeof(struct udphdr); + + return RTASE_PARSE_OK; + } + + return RTASE_PARSE_DROP; + } + + return RTASE_PARSE_OK; + + case htons(ETH_P_IPV6): + i6h = skb_header_pointer(skb, no, sizeof(_i6h), &_i6h); + if (!i6h) + return RTASE_PARSE_DROP; + + offset = no + sizeof(*i6h); + + if (i6h->nexthdr == IPPROTO_TCP) { + if (skb->len - offset < sizeof(struct tcphdr)) + return RTASE_PARSE_DROP; + + return RTASE_PARSE_SKIP; + } + + if (i6h->nexthdr != IPPROTO_UDP) + return RTASE_PARSE_SKIP; + + if (skb->len - offset < sizeof(struct udphdr)) + return RTASE_PARSE_DROP; + + *transport_offset = offset; + *transport_proto = IPPROTO_UDP; + + return RTASE_PARSE_OK; + + default: + return RTASE_PARSE_SKIP; + } +} + +static bool rtase_skb_pad(struct sk_buff *skb) +{ + enum rtase_parse_result ret; + u32 transport_offset; + __be16 *dest, _dest; + u32 trans_data_len; + u32 pad_to_len = 0; + u8 transport_proto; + u16 dest_port; + + ret = rtase_get_transport_offset(skb, &transport_offset, + &transport_proto, &pad_to_len); + if (ret == RTASE_PARSE_SKIP) { + return true; + } else if (ret == RTASE_PARSE_DROP) { + netdev_dbg(skb->dev, "drop malformed packet\n"); + return false; + } + + if (pad_to_len && + !rtase_pad_to_transport_len(skb, transport_offset, pad_to_len)) + return false; + + if (transport_proto != IPPROTO_UDP) + return true; + + trans_data_len = skb->len - transport_offset; + if (trans_data_len < offsetof(struct udphdr, len) || + trans_data_len >= RTASE_MIN_PAD_LEN) + return true; + + dest = skb_header_pointer(skb, + transport_offset + + offsetof(struct udphdr, dest), + sizeof(_dest), &_dest); + if (!dest) + return true; + + dest_port = ntohs(*dest); + if (dest_port != PTP_EV_PORT && dest_port != PTP_GEN_PORT) + return true; + + return rtase_pad_to_transport_len(skb, transport_offset, + RTASE_MIN_PAD_LEN); +} + static int rtase_xmit_frags(struct rtase_ring *ring, struct sk_buff *skb, u32 opts1, u32 opts2) { @@ -1362,6 +1556,9 @@ static netdev_tx_t rtase_start_xmit(struct sk_buff *skb, opts2 |= rtase_tx_csum(skb, dev); } + if (!rtase_skb_pad(skb)) + goto err_dma_0; + frags = rtase_xmit_frags(ring, skb, opts1, opts2); if (unlikely(frags < 0)) goto err_dma_0; diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h index acd7719506b6..83929477b058 100644 --- a/drivers/net/ethernet/stmicro/stmmac/common.h +++ b/drivers/net/ethernet/stmicro/stmmac/common.h @@ -197,9 +197,6 @@ struct stmmac_extra_stats { unsigned long irq_pcs_ane_n; unsigned long irq_pcs_link_n; unsigned long irq_rgmii_n; - unsigned long pcs_link; - unsigned long pcs_duplex; - unsigned long pcs_speed; /* debug register */ unsigned long mtl_tx_status_fifo_full; unsigned long mtl_tx_fifo_not_empty; diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c index 47fda982d6b1..a54bd924032c 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c @@ -524,6 +524,32 @@ static int intel_set_reg_access(const struct pmc_serdes_regs *regs, int max_regs return ret; } +/* + * Return true if the SerDes lane rate must change to serve @interface. + * If the current rate cannot be determined, reconfigure as before. + */ +static bool intel_serdes_needs_reconfig(struct stmmac_priv *priv, + struct intel_priv_data *intel_priv, + phy_interface_t interface) +{ + u32 cur_rate, want_rate; + int data; + + if (!intel_priv->mdio_adhoc_addr) + return true; + + data = mdiobus_read(priv->mii, intel_priv->mdio_adhoc_addr, + SERDES_GCR0); + if (data < 0) + return true; + + cur_rate = (data & SERDES_RATE_MASK) >> SERDES_RATE_PCIE_SHIFT; + want_rate = interface == PHY_INTERFACE_MODE_2500BASEX ? + SERDES_RATE_PCIE_GEN2 : SERDES_RATE_PCIE_GEN1; + + return cur_rate != want_rate; +} + static int intel_mac_finish(struct net_device *ndev, void *intel_data, unsigned int mode, @@ -535,6 +561,11 @@ static int intel_mac_finish(struct net_device *ndev, int max_regs = 0; int ret = 0; + if (!intel_serdes_needs_reconfig(priv, intel_priv, interface)) { + priv->plat->phy_interface = interface; + return 0; + } + ret = intel_tsn_lane_is_available(ndev, intel_priv); if (ret < 0) { netdev_info(priv->dev, "No TSN lane available to set the registers.\n"); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-loongson.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-loongson.c index ab431bf9b25f..e6bce3a433a7 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-loongson.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-loongson.c @@ -209,9 +209,8 @@ static void loongson_dwmac_dma_init_channel(struct stmmac_priv *priv, value |= DMA_BUS_MODE_MAXPBL; value |= DMA_BUS_MODE_USP; - value &= ~(DMA_BUS_MODE_PBL_MASK | DMA_BUS_MODE_RPBL_MASK); - value |= (txpbl << DMA_BUS_MODE_PBL_SHIFT); - value |= (rxpbl << DMA_BUS_MODE_RPBL_SHIFT); + value = u32_replace_bits(value, txpbl, DMA_BUS_MODE_PBL_MASK); + value = u32_replace_bits(value, rxpbl, DMA_BUS_MODE_RPBL_MASK); /* Set the Fixed burst mode */ if (dma_cfg->fixed_burst) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c index 2ff5db6d41ca..e40d0d696303 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c @@ -5,6 +5,7 @@ */ #include +#include #include #include #include @@ -15,8 +16,10 @@ #include #include +#include "dwxgmac2.h" #include "stmmac.h" #include "stmmac_platform.h" +#include "stmmac_ptp.h" #define SYSMGR_EMACGRP_CTRL_PHYSEL_ENUM_GMII_MII 0x0 #define SYSMGR_EMACGRP_CTRL_PHYSEL_ENUM_RGMII 0x1 @@ -41,9 +44,17 @@ #define SGMII_ADAPTER_ENABLE 0x0000 #define SGMII_ADAPTER_DISABLE 0x0001 +#define SMTG_MDIO_ADDR 0x15 +#define SMTG_TSC_WORD0 0xC +#define SMTG_TSC_WORD1 0xD +#define SMTG_TSC_WORD2 0xE +#define SMTG_TSC_WORD3 0xF +#define SMTG_TSC_SHIFT 16 + struct socfpga_dwmac; struct socfpga_dwmac_ops { int (*set_phy_mode)(struct socfpga_dwmac *dwmac_priv); + void (*setup_plat_dat)(struct socfpga_dwmac *dwmac_priv); }; struct socfpga_dwmac { @@ -268,6 +279,111 @@ static int socfpga_set_phy_mode_common(int phymode, u32 *val) return 0; } +static void get_smtgtime(struct mii_bus *mii, int smtg_addr, u64 *smtg_time) +{ + u64 ns; + + ns = mdiobus_read(mii, smtg_addr, SMTG_TSC_WORD3); + ns <<= SMTG_TSC_SHIFT; + ns |= mdiobus_read(mii, smtg_addr, SMTG_TSC_WORD2); + ns <<= SMTG_TSC_SHIFT; + ns |= mdiobus_read(mii, smtg_addr, SMTG_TSC_WORD1); + ns <<= SMTG_TSC_SHIFT; + ns |= mdiobus_read(mii, smtg_addr, SMTG_TSC_WORD0); + + *smtg_time = ns; +} + +static int smtg_crosststamp(ktime_t *device, struct system_counterval_t *system, + void *ctx) +{ + struct stmmac_priv *priv = (struct stmmac_priv *)ctx; + u32 num_snapshot, gpio_value, acr_value; + void __iomem *ptpaddr = priv->ptpaddr; + void __iomem *ioaddr = priv->hw->pcsr; + unsigned long flags; + u64 smtg_time = 0; + u64 ptp_time = 0; + int i, ret; + u32 v; + + /* Both internal crosstimestamping and external triggered event + * timestamping cannot be run concurrently. + */ + if (priv->plat->flags & STMMAC_FLAG_EXT_SNAPSHOT_EN) + return -EBUSY; + + mutex_lock(&priv->aux_ts_lock); + /* Enable Internal snapshot trigger */ + acr_value = readl(ptpaddr + PTP_ACR); + acr_value &= ~PTP_ACR_MASK; + switch (priv->plat->int_snapshot_num) { + case AUX_SNAPSHOT0: + acr_value |= PTP_ACR_ATSEN0; + break; + case AUX_SNAPSHOT1: + acr_value |= PTP_ACR_ATSEN1; + break; + case AUX_SNAPSHOT2: + acr_value |= PTP_ACR_ATSEN2; + break; + case AUX_SNAPSHOT3: + acr_value |= PTP_ACR_ATSEN3; + break; + default: + mutex_unlock(&priv->aux_ts_lock); + return -EINVAL; + } + writel(acr_value, ptpaddr + PTP_ACR); + + /* Clear FIFO */ + acr_value = readl(ptpaddr + PTP_ACR); + acr_value |= PTP_ACR_ATSFC; + writel(acr_value, ptpaddr + PTP_ACR); + /* Release the mutex */ + mutex_unlock(&priv->aux_ts_lock); + + /* Trigger Internal snapshot signal. Create a rising edge by just toggle + * the GPO0 to low and back to high. + */ + gpio_value = readl(ioaddr + XGMAC_GPIO_STATUS); + gpio_value &= ~XGMAC_GPIO_GPO0; + writel(gpio_value, ioaddr + XGMAC_GPIO_STATUS); + gpio_value |= XGMAC_GPIO_GPO0; + writel(gpio_value, ioaddr + XGMAC_GPIO_STATUS); + + /* Poll for time sync operation done */ + ret = readl_poll_timeout(priv->ioaddr + XGMAC_INT_STATUS, v, + (v & XGMAC_INT_TSIS), 100, 10000); + if (ret) { + netdev_err(priv->dev, "%s: Wait for time sync operation timeout\n", + __func__); + return ret; + } + + *system = (struct system_counterval_t) { + .cycles = 0, + .cs_id = CSID_ARM_ARCH_COUNTER, + .use_nsecs = false, + }; + + num_snapshot = FIELD_GET(XGMAC_TIMESTAMP_ATSNS_MASK, + readl(ioaddr + XGMAC_TIMESTAMP_STATUS)); + + /* Repeat until the timestamps are from the FIFO last segment */ + for (i = 0; i < num_snapshot; i++) { + read_lock_irqsave(&priv->ptp_lock, flags); + stmmac_get_ptptime(priv, ptpaddr, &ptp_time); + *device = ns_to_ktime(ptp_time); + read_unlock_irqrestore(&priv->ptp_lock, flags); + } + + get_smtgtime(priv->mii, SMTG_MDIO_ADDR, &smtg_time); + system->cycles = smtg_time; + + return 0; +} + static int socfpga_gen5_set_phy_mode(struct socfpga_dwmac *dwmac) { struct regmap *sys_mgr_base_addr = dwmac->sys_mgr_base_addr; @@ -441,6 +557,40 @@ static int socfpga_dwmac_init(struct platform_device *pdev, void *bsp_priv) return dwmac->ops->set_phy_mode(dwmac); } +static void socfpga_gen5_setup_plat_dat(struct socfpga_dwmac *dwmac) +{ + struct plat_stmmacenet_data *plat_dat = dwmac->plat_dat; + + plat_dat->core_type = DWMAC_CORE_GMAC; + + /* Rx watchdog timer in dwmac is buggy in this hw */ + plat_dat->riwt_off = 1; +} + +static void socfpga_agilex5_setup_plat_dat(struct socfpga_dwmac *dwmac) +{ + struct plat_stmmacenet_data *plat_dat = dwmac->plat_dat; + + plat_dat->core_type = DWMAC_CORE_XGMAC; + + /* Enable TBS */ + switch (plat_dat->tx_queues_to_use) { + case 8: + plat_dat->tx_queues_cfg[7].tbs_en = true; + fallthrough; + case 7: + plat_dat->tx_queues_cfg[6].tbs_en = true; + break; + default: + /* Tx Queues 0 - 5 doesn't support TBS on Agilex5 */ + break; + } + + /* Hw supported cross-timestamp */ + plat_dat->int_snapshot_num = AUX_SNAPSHOT0; + plat_dat->crosststamp = smtg_crosststamp; +} + static int socfpga_dwmac_probe(struct platform_device *pdev) { struct plat_stmmacenet_data *plat_dat; @@ -497,25 +647,31 @@ static int socfpga_dwmac_probe(struct platform_device *pdev) plat_dat->pcs_init = socfpga_dwmac_pcs_init; plat_dat->pcs_exit = socfpga_dwmac_pcs_exit; plat_dat->select_pcs = socfpga_dwmac_select_pcs; - plat_dat->core_type = DWMAC_CORE_GMAC; - plat_dat->riwt_off = 1; + ops->setup_plat_dat(dwmac); return devm_stmmac_pltfr_probe(pdev, plat_dat, &stmmac_res); } static const struct socfpga_dwmac_ops socfpga_gen5_ops = { .set_phy_mode = socfpga_gen5_set_phy_mode, + .setup_plat_dat = socfpga_gen5_setup_plat_dat, }; static const struct socfpga_dwmac_ops socfpga_gen10_ops = { .set_phy_mode = socfpga_gen10_set_phy_mode, + .setup_plat_dat = socfpga_gen5_setup_plat_dat, +}; + +static const struct socfpga_dwmac_ops socfpga_agilex5_ops = { + .set_phy_mode = socfpga_gen10_set_phy_mode, + .setup_plat_dat = socfpga_agilex5_setup_plat_dat, }; static const struct of_device_id socfpga_dwmac_match[] = { { .compatible = "altr,socfpga-stmmac", .data = &socfpga_gen5_ops }, { .compatible = "altr,socfpga-stmmac-a10-s10", .data = &socfpga_gen10_ops }, - { .compatible = "altr,socfpga-stmmac-agilex5", .data = &socfpga_gen10_ops }, + { .compatible = "altr,socfpga-stmmac-agilex5", .data = &socfpga_agilex5_ops }, { } }; MODULE_DEVICE_TABLE(of, socfpga_dwmac_match); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100.h b/drivers/net/ethernet/stmicro/stmmac/dwmac100.h index 7ab791c8d355..eae929955ad7 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac100.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100.h @@ -59,8 +59,7 @@ #define MAC_CORE_INIT (MAC_CONTROL_HBD) /* MAC FLOW CTRL defines */ -#define MAC_FLOW_CTRL_PT_MASK 0xffff0000 /* Pause Time Mask */ -#define MAC_FLOW_CTRL_PT_SHIFT 16 +#define MAC_FLOW_CTRL_PT_MASK GENMASK(31, 16) /* Pause Time Mask */ #define MAC_FLOW_CTRL_PASS 0x00000004 /* Pass Control Frames */ #define MAC_FLOW_CTRL_ENABLE 0x00000002 /* Flow Control Enable */ #define MAC_FLOW_CTRL_PAUSE 0x00000001 /* Flow Control Busy ... */ @@ -76,10 +75,8 @@ /* DMA Bus Mode register defines */ #define DMA_BUS_MODE_DBO 0x00100000 /* Descriptor Byte Ordering */ #define DMA_BUS_MODE_BLE 0x00000080 /* Big Endian/Little Endian */ -#define DMA_BUS_MODE_PBL_MASK 0x00003f00 /* Programmable Burst Len */ -#define DMA_BUS_MODE_PBL_SHIFT 8 -#define DMA_BUS_MODE_DSL_MASK 0x0000007c /* Descriptor Skip Length */ -#define DMA_BUS_MODE_DSL_SHIFT 2 /* (in DWORDS) */ +#define DMA_BUS_MODE_PBL_MASK GENMASK(13, 8) /* Programmable Burst Len */ +#define DMA_BUS_MODE_DSL_MASK GENMASK(6, 2) /* Descriptor Skip Length */ #define DMA_BUS_MODE_BAR_BUS 0x00000002 /* Bar-Bus Arbitration */ #define DMA_BUS_MODE_DEFAULT 0x00000000 diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h b/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h index 0c011a47d5a3..12c82235c1d7 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h @@ -77,7 +77,6 @@ enum power_event { /* SGMII/RGMII status register */ #define GMAC_RGSMIIIS_LNKMODE BIT(0) #define GMAC_RGSMIIIS_SPEED GENMASK(2, 1) -#define GMAC_RGSMIIIS_SPEED_SHIFT 1 #define GMAC_RGSMIIIS_LNKSTS BIT(3) #define GMAC_RGSMIIIS_JABTO BIT(4) #define GMAC_RGSMIIIS_FALSECARDET BIT(5) @@ -134,8 +133,7 @@ enum inter_frame_gap { #define GMAC_MII_ADDR_WRITE 0x00000002 /* MII Write */ #define GMAC_MII_ADDR_BUSY 0x00000001 /* MII Busy */ /* GMAC FLOW CTRL defines */ -#define GMAC_FLOW_CTRL_PT_MASK 0xffff0000 /* Pause Time Mask */ -#define GMAC_FLOW_CTRL_PT_SHIFT 16 +#define GMAC_FLOW_CTRL_PT_MASK GENMASK(31, 16) /* Pause Time Mask */ #define GMAC_FLOW_CTRL_UP 0x00000008 /* Unicast pause frame enable */ #define GMAC_FLOW_CTRL_RFE 0x00000004 /* Rx Flow Control Enable */ #define GMAC_FLOW_CTRL_TFE 0x00000002 /* Tx Flow Control Enable */ @@ -148,7 +146,6 @@ enum inter_frame_gap { #define GMAC_DEBUG_TWCSTS BIT(22) /* MTL Tx FIFO Write Controller */ /* MTL Tx FIFO Read Controller Status */ #define GMAC_DEBUG_TRCSTS_MASK GENMASK(21, 20) -#define GMAC_DEBUG_TRCSTS_SHIFT 20 #define GMAC_DEBUG_TRCSTS_IDLE 0 #define GMAC_DEBUG_TRCSTS_READ 1 #define GMAC_DEBUG_TRCSTS_TXW 2 @@ -156,7 +153,6 @@ enum inter_frame_gap { #define GMAC_DEBUG_TXPAUSED BIT(19) /* MAC Transmitter in PAUSE */ /* MAC Transmit Frame Controller Status */ #define GMAC_DEBUG_TFCSTS_MASK GENMASK(18, 17) -#define GMAC_DEBUG_TFCSTS_SHIFT 17 #define GMAC_DEBUG_TFCSTS_IDLE 0 #define GMAC_DEBUG_TFCSTS_WAIT 1 #define GMAC_DEBUG_TFCSTS_GEN_PAUSE 2 @@ -164,13 +160,11 @@ enum inter_frame_gap { /* MAC GMII or MII Transmit Protocol Engine Status */ #define GMAC_DEBUG_TPESTS BIT(16) #define GMAC_DEBUG_RXFSTS_MASK GENMASK(9, 8) /* MTL Rx FIFO Fill-level */ -#define GMAC_DEBUG_RXFSTS_SHIFT 8 #define GMAC_DEBUG_RXFSTS_EMPTY 0 #define GMAC_DEBUG_RXFSTS_BT 1 #define GMAC_DEBUG_RXFSTS_AT 2 #define GMAC_DEBUG_RXFSTS_FULL 3 #define GMAC_DEBUG_RRCSTS_MASK GENMASK(6, 5) /* MTL Rx FIFO Read Controller */ -#define GMAC_DEBUG_RRCSTS_SHIFT 5 #define GMAC_DEBUG_RRCSTS_IDLE 0 #define GMAC_DEBUG_RRCSTS_RDATA 1 #define GMAC_DEBUG_RRCSTS_RSTAT 2 @@ -178,7 +172,6 @@ enum inter_frame_gap { #define GMAC_DEBUG_RWCSTS BIT(4) /* MTL Rx FIFO Write Controller Active */ /* MAC Receive Frame Controller FIFO Status */ #define GMAC_DEBUG_RFCFCSTS_MASK GENMASK(2, 1) -#define GMAC_DEBUG_RFCFCSTS_SHIFT 1 /* MAC GMII or MII Receive Protocol Engine Status */ #define GMAC_DEBUG_RPESTS BIT(0) @@ -188,8 +181,7 @@ enum inter_frame_gap { #define DMA_BUS_MODE_DSL_MASK 0x0000007c /* Descriptor Skip Length */ #define DMA_BUS_MODE_DSL_SHIFT 2 /* (in DWORDS) */ /* Programmable burst length (passed thorugh platform)*/ -#define DMA_BUS_MODE_PBL_MASK 0x00003f00 /* Programmable Burst Len */ -#define DMA_BUS_MODE_PBL_SHIFT 8 +#define DMA_BUS_MODE_PBL_MASK GENMASK(13, 8) /* Programmable Burst Len */ #define DMA_BUS_MODE_ATDS 0x00000080 /* Alternate Descriptor Size */ enum rx_tx_priority_ratio { @@ -200,8 +192,7 @@ enum rx_tx_priority_ratio { #define DMA_BUS_MODE_FB 0x00010000 /* Fixed burst */ #define DMA_BUS_MODE_MB 0x04000000 /* Mixed burst */ -#define DMA_BUS_MODE_RPBL_MASK 0x007e0000 /* Rx-Programmable Burst Len */ -#define DMA_BUS_MODE_RPBL_SHIFT 17 +#define DMA_BUS_MODE_RPBL_MASK GENMASK(22, 17) /* Rx-Programmable Burst Len */ #define DMA_BUS_MODE_USP 0x00800000 #define DMA_BUS_MODE_MAXPBL 0x01000000 #define DMA_BUS_MODE_AAL 0x02000000 @@ -321,7 +312,6 @@ enum rtc_control { /* PTP and timestamping registers */ #define GMAC3_X_ATSNS GENMASK(29, 25) -#define GMAC3_X_ATSNS_SHIFT 25 #define GMAC_PTP_TCR_ATSFC BIT(24) #define GMAC_PTP_TCR_ATSEN0 BIT(25) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c index fe776ddf6889..5bf717188c92 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c @@ -240,7 +240,7 @@ static void dwmac1000_flow_ctrl(struct mac_device_info *hw, unsigned int duplex, if (duplex) { pr_debug("\tduplex mode: PAUSE %d\n", pause_time); - flow |= (pause_time << GMAC_FLOW_CTRL_PT_SHIFT); + flow |= FIELD_PREP(GMAC_FLOW_CTRL_PT_MASK, pause_time); } writel(flow, ioaddr + GMAC_FLOW_CTRL); @@ -266,34 +266,8 @@ static void dwmac1000_pmt(struct mac_device_info *hw, unsigned long mode) /* RGMII or SMII interface */ static void dwmac1000_rgsmii(void __iomem *ioaddr, struct stmmac_extra_stats *x) { - u32 status; - - status = readl(ioaddr + GMAC_RGSMIIIS); + readl(ioaddr + GMAC_RGSMIIIS); x->irq_rgmii_n++; - - /* Check the link status */ - if (status & GMAC_RGSMIIIS_LNKSTS) { - int speed_value; - - x->pcs_link = 1; - - speed_value = ((status & GMAC_RGSMIIIS_SPEED) >> - GMAC_RGSMIIIS_SPEED_SHIFT); - if (speed_value == GMAC_RGSMIIIS_SPEED_125) - x->pcs_speed = SPEED_1000; - else if (speed_value == GMAC_RGSMIIIS_SPEED_25) - x->pcs_speed = SPEED_100; - else - x->pcs_speed = SPEED_10; - - x->pcs_duplex = (status & GMAC_RGSMIIIS_LNKMOD_MASK); - - pr_info("Link is Up - %d/%s\n", (int)x->pcs_speed, - x->pcs_duplex ? "Full" : "Half"); - } else { - x->pcs_link = 0; - pr_info("Link is Down\n"); - } } static int dwmac1000_irq_status(struct mac_device_info *hw, @@ -412,8 +386,8 @@ static void dwmac1000_debug(struct stmmac_priv *priv, void __iomem *ioaddr, if (value & GMAC_DEBUG_TWCSTS) x->mmtl_fifo_ctrl++; if (value & GMAC_DEBUG_TRCSTS_MASK) { - u32 trcsts = (value & GMAC_DEBUG_TRCSTS_MASK) - >> GMAC_DEBUG_TRCSTS_SHIFT; + u32 trcsts = FIELD_GET(GMAC_DEBUG_TRCSTS_MASK, value); + if (trcsts == GMAC_DEBUG_TRCSTS_WRITE) x->mtl_tx_fifo_read_ctrl_write++; else if (trcsts == GMAC_DEBUG_TRCSTS_TXW) @@ -426,8 +400,7 @@ static void dwmac1000_debug(struct stmmac_priv *priv, void __iomem *ioaddr, if (value & GMAC_DEBUG_TXPAUSED) x->mac_tx_in_pause++; if (value & GMAC_DEBUG_TFCSTS_MASK) { - u32 tfcsts = (value & GMAC_DEBUG_TFCSTS_MASK) - >> GMAC_DEBUG_TFCSTS_SHIFT; + u32 tfcsts = FIELD_GET(GMAC_DEBUG_TFCSTS_MASK, value); if (tfcsts == GMAC_DEBUG_TFCSTS_XFER) x->mac_tx_frame_ctrl_xfer++; @@ -441,8 +414,7 @@ static void dwmac1000_debug(struct stmmac_priv *priv, void __iomem *ioaddr, if (value & GMAC_DEBUG_TPESTS) x->mac_gmii_tx_proto_engine++; if (value & GMAC_DEBUG_RXFSTS_MASK) { - u32 rxfsts = (value & GMAC_DEBUG_RXFSTS_MASK) - >> GMAC_DEBUG_RRCSTS_SHIFT; + u32 rxfsts = FIELD_GET(GMAC_DEBUG_RXFSTS_MASK, value); if (rxfsts == GMAC_DEBUG_RXFSTS_FULL) x->mtl_rx_fifo_fill_level_full++; @@ -454,8 +426,7 @@ static void dwmac1000_debug(struct stmmac_priv *priv, void __iomem *ioaddr, x->mtl_rx_fifo_fill_level_empty++; } if (value & GMAC_DEBUG_RRCSTS_MASK) { - u32 rrcsts = (value & GMAC_DEBUG_RRCSTS_MASK) >> - GMAC_DEBUG_RRCSTS_SHIFT; + u32 rrcsts = FIELD_GET(GMAC_DEBUG_RRCSTS_MASK, value); if (rrcsts == GMAC_DEBUG_RRCSTS_FLUSH) x->mtl_rx_fifo_read_ctrl_flush++; @@ -469,8 +440,8 @@ static void dwmac1000_debug(struct stmmac_priv *priv, void __iomem *ioaddr, if (value & GMAC_DEBUG_RWCSTS) x->mtl_rx_fifo_ctrl_active++; if (value & GMAC_DEBUG_RFCFCSTS_MASK) - x->mac_rx_frame_ctrl_fifo = (value & GMAC_DEBUG_RFCFCSTS_MASK) - >> GMAC_DEBUG_RFCFCSTS_SHIFT; + x->mac_rx_frame_ctrl_fifo = FIELD_GET(GMAC_DEBUG_RFCFCSTS_MASK, + value); if (value & GMAC_DEBUG_RPESTS) x->mac_gmii_rx_proto_engine++; } @@ -566,7 +537,7 @@ void dwmac1000_timestamp_interrupt(struct stmmac_priv *priv) if (!(priv->plat->flags & STMMAC_FLAG_EXT_SNAPSHOT_EN)) return; - num_snapshot = (ts_status & GMAC3_X_ATSNS) >> GMAC3_X_ATSNS_SHIFT; + num_snapshot = FIELD_GET(GMAC3_X_ATSNS, ts_status); for (i = 0; i < num_snapshot; i++) { read_lock_irqsave(&priv->ptp_lock, flags); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c index 118a22406a2e..cacbf6d4365c 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c @@ -29,13 +29,10 @@ static void dwmac1000_dma_axi(void __iomem *ioaddr, struct stmmac_axi *axi) if (axi->axi_xit_frm) value |= DMA_AXI_LPI_XIT_FRM; - value &= ~DMA_AXI_WR_OSR_LMT; - value |= (axi->axi_wr_osr_lmt & DMA_AXI_WR_OSR_LMT_MASK) << - DMA_AXI_WR_OSR_LMT_SHIFT; - - value &= ~DMA_AXI_RD_OSR_LMT; - value |= (axi->axi_rd_osr_lmt & DMA_AXI_RD_OSR_LMT_MASK) << - DMA_AXI_RD_OSR_LMT_SHIFT; + value = u32_replace_bits(value, axi->axi_wr_osr_lmt, + DMA_AXI_WR_OSR_LMT); + value = u32_replace_bits(value, axi->axi_rd_osr_lmt, + DMA_AXI_RD_OSR_LMT); /* Depending on the UNDEF bit the Master AXI will perform any burst * length according to the BLEN programmed (by default all BLEN are @@ -88,9 +85,8 @@ static void dwmac1000_dma_init_channel(struct stmmac_priv *priv, if (dma_cfg->pblx8) value |= DMA_BUS_MODE_MAXPBL; value |= DMA_BUS_MODE_USP; - value &= ~(DMA_BUS_MODE_PBL_MASK | DMA_BUS_MODE_RPBL_MASK); - value |= (txpbl << DMA_BUS_MODE_PBL_SHIFT); - value |= (rxpbl << DMA_BUS_MODE_RPBL_SHIFT); + value = u32_replace_bits(value, txpbl, DMA_BUS_MODE_PBL_MASK); + value = u32_replace_bits(value, rxpbl, DMA_BUS_MODE_RPBL_MASK); /* Set the Fixed burst mode */ if (dma_cfg->fixed_burst) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c index 14e847c0e1a9..dbc0c1019ed5 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c @@ -132,7 +132,7 @@ static void dwmac100_flow_ctrl(struct mac_device_info *hw, unsigned int duplex, unsigned int flow = MAC_FLOW_CTRL_ENABLE; if (duplex) - flow |= (pause_time << MAC_FLOW_CTRL_PT_SHIFT); + flow |= FIELD_PREP(MAC_FLOW_CTRL_PT_MASK, pause_time); writel(flow, ioaddr + MAC_FLOW_CTRL); } diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c index 82957db47c99..12b2bf2d739a 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c @@ -22,7 +22,8 @@ static void dwmac100_dma_init(void __iomem *ioaddr, struct stmmac_dma_cfg *dma_cfg) { /* Enable Application Access by writing to DMA CSR0 */ - writel(DMA_BUS_MODE_DEFAULT | (dma_cfg->pbl << DMA_BUS_MODE_PBL_SHIFT), + writel(DMA_BUS_MODE_DEFAULT | + FIELD_PREP(DMA_BUS_MODE_PBL_MASK, dma_cfg->pbl), ioaddr + DMA_BUS_MODE); /* Mask interrupts by writing to CSR7 */ diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4.h b/drivers/net/ethernet/stmicro/stmmac/dwmac4.h index 3dec1a264cf6..7a6609185fd0 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4.h @@ -95,7 +95,7 @@ /* MAC Flow Control TX */ #define GMAC_TX_FLOW_CTRL_TFE BIT(1) -#define GMAC_TX_FLOW_CTRL_PT_SHIFT 16 +#define GMAC_TX_FLOW_CTRL_PT_MASK GENMASK(31, 16) /* MAC Interrupt bitmap*/ #define GMAC_INT_RGSMIIS BIT(0) @@ -145,23 +145,19 @@ enum power_event { /* MAC Debug bitmap */ #define GMAC_DEBUG_TFCSTS_MASK GENMASK(18, 17) -#define GMAC_DEBUG_TFCSTS_SHIFT 17 #define GMAC_DEBUG_TFCSTS_IDLE 0 #define GMAC_DEBUG_TFCSTS_WAIT 1 #define GMAC_DEBUG_TFCSTS_GEN_PAUSE 2 #define GMAC_DEBUG_TFCSTS_XFER 3 #define GMAC_DEBUG_TPESTS BIT(16) #define GMAC_DEBUG_RFCFCSTS_MASK GENMASK(2, 1) -#define GMAC_DEBUG_RFCFCSTS_SHIFT 1 #define GMAC_DEBUG_RPESTS BIT(0) /* MAC config */ #define GMAC_CONFIG_ARPEN BIT(31) #define GMAC_CONFIG_SARC GENMASK(30, 28) -#define GMAC_CONFIG_SARC_SHIFT 28 #define GMAC_CONFIG_IPC BIT(27) #define GMAC_CONFIG_IPG GENMASK(26, 24) -#define GMAC_CONFIG_IPG_SHIFT 24 #define GMAC_CONFIG_2K BIT(22) #define GMAC_CONFIG_ACS BIT(20) #define GMAC_CONFIG_BE BIT(18) @@ -169,7 +165,6 @@ enum power_event { #define GMAC_CONFIG_JE BIT(16) #define GMAC_CONFIG_PS BIT(15) #define GMAC_CONFIG_FES BIT(14) -#define GMAC_CONFIG_FES_SHIFT 14 #define GMAC_CONFIG_DM BIT(13) #define GMAC_CONFIG_LM BIT(12) #define GMAC_CONFIG_DCRS BIT(9) @@ -178,11 +173,9 @@ enum power_event { /* MAC extended config */ #define GMAC_CONFIG_EIPG GENMASK(29, 25) -#define GMAC_CONFIG_EIPG_SHIFT 25 #define GMAC_CONFIG_EIPG_EN BIT(24) #define GMAC_CONFIG_HDSMS GENMASK(22, 20) -#define GMAC_CONFIG_HDSMS_SHIFT 20 -#define GMAC_CONFIG_HDSMS_256 (0x2 << GMAC_CONFIG_HDSMS_SHIFT) +#define GMAC_CONFIG_HDSMS_256 FIELD_PREP_CONST(GMAC_CONFIG_HDSMS, 0x2) /* MAC HW features0 bitmap */ #define GMAC_HW_FEAT_SAVLANINS BIT(27) @@ -245,7 +238,6 @@ enum power_event { /* MAC HW ADDR regs */ #define GMAC_HI_DCS GENMASK(18, 16) -#define GMAC_HI_DCS_SHIFT 16 #define GMAC_HI_REG_AE BIT(31) /* L3/L4 Filters regs */ @@ -260,7 +252,6 @@ enum power_event { #define GMAC_L3SAM0 BIT(2) #define GMAC_L3PEN0 BIT(0) #define GMAC_L4DP0 GENMASK(31, 16) -#define GMAC_L4DP0_SHIFT 16 #define GMAC_L4SP0 GENMASK(15, 0) /* MAC Timestamp Status */ @@ -317,39 +308,32 @@ static inline u32 mtl_chanx_base_addr(const struct dwmac4_addrs *addrs, #define MTL_OP_MODE_TSF BIT(1) #define MTL_OP_MODE_TQS_MASK GENMASK(24, 16) -#define MTL_OP_MODE_TQS_SHIFT 16 -#define MTL_OP_MODE_TTC_MASK 0x70 -#define MTL_OP_MODE_TTC_SHIFT 4 - -#define MTL_OP_MODE_TTC_32 0 -#define MTL_OP_MODE_TTC_64 (1 << MTL_OP_MODE_TTC_SHIFT) -#define MTL_OP_MODE_TTC_96 (2 << MTL_OP_MODE_TTC_SHIFT) -#define MTL_OP_MODE_TTC_128 (3 << MTL_OP_MODE_TTC_SHIFT) -#define MTL_OP_MODE_TTC_192 (4 << MTL_OP_MODE_TTC_SHIFT) -#define MTL_OP_MODE_TTC_256 (5 << MTL_OP_MODE_TTC_SHIFT) -#define MTL_OP_MODE_TTC_384 (6 << MTL_OP_MODE_TTC_SHIFT) -#define MTL_OP_MODE_TTC_512 (7 << MTL_OP_MODE_TTC_SHIFT) +#define MTL_OP_MODE_TTC_MASK GENMASK(6, 4) +#define MTL_OP_MODE_TTC_32 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 0) +#define MTL_OP_MODE_TTC_64 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 1) +#define MTL_OP_MODE_TTC_96 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 2) +#define MTL_OP_MODE_TTC_128 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 3) +#define MTL_OP_MODE_TTC_192 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 4) +#define MTL_OP_MODE_TTC_256 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 5) +#define MTL_OP_MODE_TTC_384 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 6) +#define MTL_OP_MODE_TTC_512 FIELD_PREP(MTL_OP_MODE_TTC_MASK, 7) #define MTL_OP_MODE_RQS_MASK GENMASK(29, 20) -#define MTL_OP_MODE_RQS_SHIFT 20 #define MTL_OP_MODE_RFD_MASK GENMASK(19, 14) -#define MTL_OP_MODE_RFD_SHIFT 14 #define MTL_OP_MODE_RFA_MASK GENMASK(13, 8) -#define MTL_OP_MODE_RFA_SHIFT 8 #define MTL_OP_MODE_EHFC BIT(7) #define MTL_OP_MODE_DIS_TCP_EF BIT(6) #define MTL_OP_MODE_RTC_MASK GENMASK(1, 0) -#define MTL_OP_MODE_RTC_SHIFT 0 -#define MTL_OP_MODE_RTC_32 (1 << MTL_OP_MODE_RTC_SHIFT) -#define MTL_OP_MODE_RTC_64 0 -#define MTL_OP_MODE_RTC_96 (2 << MTL_OP_MODE_RTC_SHIFT) -#define MTL_OP_MODE_RTC_128 (3 << MTL_OP_MODE_RTC_SHIFT) +#define MTL_OP_MODE_RTC_32 FIELD_PREP(MTL_OP_MODE_RTC_MASK, 1) +#define MTL_OP_MODE_RTC_64 FIELD_PREP(MTL_OP_MODE_RTC_MASK, 0) +#define MTL_OP_MODE_RTC_96 FIELD_PREP(MTL_OP_MODE_RTC_MASK, 2) +#define MTL_OP_MODE_RTC_128 FIELD_PREP(MTL_OP_MODE_RTC_MASK, 3) /* MTL ETS Control register */ #define MTL_ETS_CTRL_BASE_ADDR 0x00000d10 @@ -454,7 +438,6 @@ static inline u32 mtl_low_credx_base_addr(const struct dwmac4_addrs *addrs, /* MTL debug: Tx FIFO Read Controller Status */ #define MTL_DEBUG_TRCSTS_MASK GENMASK(2, 1) -#define MTL_DEBUG_TRCSTS_SHIFT 1 #define MTL_DEBUG_TRCSTS_IDLE 0 #define MTL_DEBUG_TRCSTS_READ 1 #define MTL_DEBUG_TRCSTS_TXW 2 @@ -469,7 +452,6 @@ static inline u32 mtl_low_credx_base_addr(const struct dwmac4_addrs *addrs, #define MTL_DEBUG_RXFSTS_AT 2 #define MTL_DEBUG_RXFSTS_FULL 3 #define MTL_DEBUG_RRCSTS_MASK GENMASK(2, 1) -#define MTL_DEBUG_RRCSTS_SHIFT 1 #define MTL_DEBUG_RRCSTS_IDLE 0 #define MTL_DEBUG_RRCSTS_RDATA 1 #define MTL_DEBUG_RRCSTS_RSTAT 2 @@ -523,7 +505,6 @@ static inline u32 mtl_low_credx_base_addr(const struct dwmac4_addrs *addrs, #define GMAC_PHYIF_CTRLSTATUS_SMIDRXS BIT(4) #define GMAC_PHYIF_CTRLSTATUS_LNKMOD BIT(16) #define GMAC_PHYIF_CTRLSTATUS_SPEED GENMASK(18, 17) -#define GMAC_PHYIF_CTRLSTATUS_SPEED_SHIFT 17 #define GMAC_PHYIF_CTRLSTATUS_LNKSTS BIT(19) #define GMAC_PHYIF_CTRLSTATUS_JABTO BIT(20) #define GMAC_PHYIF_CTRLSTATUS_FALSECARDET BIT(21) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c index d85bc0bb5c3c..1f61177ece45 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c @@ -572,8 +572,8 @@ static void dwmac4_flow_ctrl(struct mac_device_info *hw, unsigned int duplex, flow = GMAC_TX_FLOW_CTRL_TFE; if (duplex) - flow |= - (pause_time << GMAC_TX_FLOW_CTRL_PT_SHIFT); + flow |= FIELD_PREP(GMAC_TX_FLOW_CTRL_PT_MASK, + pause_time); writel(flow, ioaddr + GMAC_QX_TX_FLOW_CTRL(queue)); } @@ -592,34 +592,8 @@ static void dwmac4_ctrl_ane(struct stmmac_priv *priv, bool ane, bool srgmi_ral, /* RGMII or SMII interface */ static void dwmac4_phystatus(void __iomem *ioaddr, struct stmmac_extra_stats *x) { - u32 status; - - status = readl(ioaddr + GMAC_PHYIF_CONTROL_STATUS); + readl(ioaddr + GMAC_PHYIF_CONTROL_STATUS); x->irq_rgmii_n++; - - /* Check the link status */ - if (status & GMAC_PHYIF_CTRLSTATUS_LNKSTS) { - int speed_value; - - x->pcs_link = 1; - - speed_value = ((status & GMAC_PHYIF_CTRLSTATUS_SPEED) >> - GMAC_PHYIF_CTRLSTATUS_SPEED_SHIFT); - if (speed_value == GMAC_PHYIF_CTRLSTATUS_SPEED_125) - x->pcs_speed = SPEED_1000; - else if (speed_value == GMAC_PHYIF_CTRLSTATUS_SPEED_25) - x->pcs_speed = SPEED_100; - else - x->pcs_speed = SPEED_10; - - x->pcs_duplex = (status & GMAC_PHYIF_CTRLSTATUS_LNKMOD); - - pr_info("Link is Up - %d/%s\n", (int)x->pcs_speed, - x->pcs_duplex ? "Full" : "Half"); - } else { - x->pcs_link = 0; - pr_info("Link is Down\n"); - } } static int dwmac4_irq_mtl_status(struct stmmac_priv *priv, @@ -717,8 +691,8 @@ static void dwmac4_debug(struct stmmac_priv *priv, void __iomem *ioaddr, if (value & MTL_DEBUG_TWCSTS) x->mmtl_fifo_ctrl++; if (value & MTL_DEBUG_TRCSTS_MASK) { - u32 trcsts = (value & MTL_DEBUG_TRCSTS_MASK) - >> MTL_DEBUG_TRCSTS_SHIFT; + u32 trcsts = FIELD_GET(MTL_DEBUG_TRCSTS_MASK, value); + if (trcsts == MTL_DEBUG_TRCSTS_WRITE) x->mtl_tx_fifo_read_ctrl_write++; else if (trcsts == MTL_DEBUG_TRCSTS_TXW) @@ -749,8 +723,7 @@ static void dwmac4_debug(struct stmmac_priv *priv, void __iomem *ioaddr, x->mtl_rx_fifo_fill_level_empty++; } if (value & MTL_DEBUG_RRCSTS_MASK) { - u32 rrcsts = (value & MTL_DEBUG_RRCSTS_MASK) >> - MTL_DEBUG_RRCSTS_SHIFT; + u32 rrcsts = FIELD_GET(MTL_DEBUG_RRCSTS_MASK, value); if (rrcsts == MTL_DEBUG_RRCSTS_FLUSH) x->mtl_rx_fifo_read_ctrl_flush++; @@ -769,8 +742,7 @@ static void dwmac4_debug(struct stmmac_priv *priv, void __iomem *ioaddr, value = readl(ioaddr + GMAC_DEBUG); if (value & GMAC_DEBUG_TFCSTS_MASK) { - u32 tfcsts = (value & GMAC_DEBUG_TFCSTS_MASK) - >> GMAC_DEBUG_TFCSTS_SHIFT; + u32 tfcsts = FIELD_GET(GMAC_DEBUG_TFCSTS_MASK, value); if (tfcsts == GMAC_DEBUG_TFCSTS_XFER) x->mac_tx_frame_ctrl_xfer++; @@ -784,8 +756,8 @@ static void dwmac4_debug(struct stmmac_priv *priv, void __iomem *ioaddr, if (value & GMAC_DEBUG_TPESTS) x->mac_gmii_tx_proto_engine++; if (value & GMAC_DEBUG_RFCFCSTS_MASK) - x->mac_rx_frame_ctrl_fifo = (value & GMAC_DEBUG_RFCFCSTS_MASK) - >> GMAC_DEBUG_RFCFCSTS_SHIFT; + x->mac_rx_frame_ctrl_fifo = FIELD_GET(GMAC_DEBUG_RFCFCSTS_MASK, + value); if (value & GMAC_DEBUG_RPESTS) x->mac_gmii_rx_proto_engine++; } @@ -806,8 +778,7 @@ static void dwmac4_sarc_configure(void __iomem *ioaddr, int val) { u32 value = readl(ioaddr + GMAC_CONFIG); - value &= ~GMAC_CONFIG_SARC; - value |= val << GMAC_CONFIG_SARC_SHIFT; + value = u32_replace_bits(value, val, GMAC_CONFIG_SARC); writel(value, ioaddr + GMAC_CONFIG); } @@ -915,9 +886,9 @@ static int dwmac4_config_l4_filter(struct mac_device_info *hw, u32 filter_no, writel(value, ioaddr + GMAC_L3L4_CTRL(filter_no)); if (sa) { - value = match & GMAC_L4SP0; + value = FIELD_PREP(GMAC_L4SP0, match); } else { - value = (match << GMAC_L4DP0_SHIFT) & GMAC_L4DP0; + value = FIELD_PREP(GMAC_L4DP0, match); } writel(value, ioaddr + GMAC_L4_ADDR(filter_no)); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c index d87a8b595e6a..617e55c3711e 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c @@ -28,13 +28,10 @@ static void dwmac4_dma_axi(void __iomem *ioaddr, struct stmmac_axi *axi) if (axi->axi_xit_frm) value |= DMA_AXI_LPI_XIT_FRM; - value &= ~DMA_AXI_WR_OSR_LMT; - value |= (axi->axi_wr_osr_lmt & DMA_AXI_OSR_MAX) << - DMA_AXI_WR_OSR_LMT_SHIFT; - - value &= ~DMA_AXI_RD_OSR_LMT; - value |= (axi->axi_rd_osr_lmt & DMA_AXI_OSR_MAX) << - DMA_AXI_RD_OSR_LMT_SHIFT; + value = u32_replace_bits(value, axi->axi_wr_osr_lmt, + DMA_AXI_WR_OSR_LMT); + value = u32_replace_bits(value, axi->axi_rd_osr_lmt, + DMA_AXI_RD_OSR_LMT); /* Depending on the UNDEF bit the Master AXI will perform any burst * length according to the BLEN programmed (by default all BLEN are @@ -79,7 +76,7 @@ static void dwmac4_dma_init_rx_chan(struct stmmac_priv *priv, u32 rxpbl = dma_cfg->rxpbl ?: dma_cfg->pbl; value = readl(ioaddr + DMA_CHAN_RX_CONTROL(dwmac4_addrs, chan)); - value = value | (rxpbl << DMA_BUS_MODE_RPBL_SHIFT); + value = value | FIELD_PREP(DMA_CHAN_RX_CTRL_RXPBL_MASK, rxpbl); writel(value, ioaddr + DMA_CHAN_RX_CONTROL(dwmac4_addrs, chan)); if (IS_ENABLED(CONFIG_ARCH_DMA_ADDR_T_64BIT) && likely(dma_cfg->eame)) @@ -100,7 +97,7 @@ static void dwmac4_dma_init_tx_chan(struct stmmac_priv *priv, u32 txpbl = dma_cfg->txpbl ?: dma_cfg->pbl; value = readl(ioaddr + DMA_CHAN_TX_CONTROL(dwmac4_addrs, chan)); - value = value | (txpbl << DMA_BUS_MODE_PBL_SHIFT); + value = value | FIELD_PREP(DMA_CHAN_TX_CTRL_TXPBL_MASK, txpbl); /* Enable OSP to get best performance */ value |= DMA_CONTROL_OSP; @@ -125,7 +122,7 @@ static void dwmac4_dma_init_channel(struct stmmac_priv *priv, /* common channel control register config */ value = readl(ioaddr + DMA_CHAN_CONTROL(dwmac4_addrs, chan)); if (dma_cfg->pblx8) - value = value | DMA_BUS_MODE_PBL; + value = value | DMA_CHAN_CTRL_PBLX8; writel(value, ioaddr + DMA_CHAN_CONTROL(dwmac4_addrs, chan)); /* Mask interrupts by writing to CSR7 */ @@ -143,7 +140,7 @@ static void dwmac410_dma_init_channel(struct stmmac_priv *priv, /* common channel control register config */ value = readl(ioaddr + DMA_CHAN_CONTROL(dwmac4_addrs, chan)); if (dma_cfg->pblx8) - value = value | DMA_BUS_MODE_PBL; + value = value | DMA_CHAN_CTRL_PBLX8; writel(value, ioaddr + DMA_CHAN_CONTROL(dwmac4_addrs, chan)); @@ -175,10 +172,9 @@ static void dwmac4_dma_init(void __iomem *ioaddr, value = readl(ioaddr + DMA_BUS_MODE); - if (dma_cfg->multi_msi_en) { - value &= ~DMA_BUS_MODE_INTM_MASK; - value |= (DMA_BUS_MODE_INTM_MODE1 << DMA_BUS_MODE_INTM_SHIFT); - } + if (dma_cfg->multi_msi_en) + value = u32_replace_bits(value, DMA_BUS_MODE_INTM_MODE1, + DMA_BUS_MODE_INTM_MASK); if (dma_cfg->dche) value |= DMA_BUS_MODE_DCHE; @@ -288,7 +284,7 @@ static void dwmac4_dma_rx_chan_op_mode(struct stmmac_priv *priv, } mtl_rx_op &= ~MTL_OP_MODE_RQS_MASK; - mtl_rx_op |= rqs << MTL_OP_MODE_RQS_SHIFT; + mtl_rx_op |= FIELD_PREP(MTL_OP_MODE_RQS_MASK, rqs); /* Enable flow control only if each channel gets 4 KiB or more FIFO and * only if channel is not an AVB channel. @@ -319,11 +315,10 @@ static void dwmac4_dma_rx_chan_op_mode(struct stmmac_priv *priv, break; } - mtl_rx_op &= ~MTL_OP_MODE_RFD_MASK; - mtl_rx_op |= rfd << MTL_OP_MODE_RFD_SHIFT; - - mtl_rx_op &= ~MTL_OP_MODE_RFA_MASK; - mtl_rx_op |= rfa << MTL_OP_MODE_RFA_SHIFT; + mtl_rx_op = u32_replace_bits(mtl_rx_op, rfd, + MTL_OP_MODE_RFD_MASK); + mtl_rx_op = u32_replace_bits(mtl_rx_op, rfa, + MTL_OP_MODE_RFA_MASK); } writel(mtl_rx_op, ioaddr + MTL_CHAN_RX_OP_MODE(dwmac4_addrs, channel)); @@ -378,8 +373,8 @@ static void dwmac4_dma_tx_chan_op_mode(struct stmmac_priv *priv, mtl_tx_op |= MTL_OP_MODE_TXQEN; else mtl_tx_op |= MTL_OP_MODE_TXQEN_AV; - mtl_tx_op &= ~MTL_OP_MODE_TQS_MASK; - mtl_tx_op |= tqs << MTL_OP_MODE_TQS_SHIFT; + + mtl_tx_op = u32_replace_bits(mtl_tx_op, tqs, MTL_OP_MODE_TQS_MASK); writel(mtl_tx_op, ioaddr + MTL_CHAN_TX_OP_MODE(dwmac4_addrs, channel)); } @@ -520,8 +515,7 @@ static void dwmac4_set_bfsize(struct stmmac_priv *priv, void __iomem *ioaddr, const struct dwmac4_addrs *dwmac4_addrs = priv->plat->dwmac4_addrs; u32 value = readl(ioaddr + DMA_CHAN_RX_CONTROL(dwmac4_addrs, chan)); - value &= ~DMA_RBSZ_MASK; - value |= (bfsize << DMA_RBSZ_SHIFT) & DMA_RBSZ_MASK; + value = u32_replace_bits(value, bfsize, DMA_RBSZ_MASK); writel(value, ioaddr + DMA_CHAN_RX_CONTROL(dwmac4_addrs, chan)); } diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.h b/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.h index 4f980dcd3958..a019eb07afcf 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.h @@ -27,15 +27,11 @@ /* DMA Bus Mode bitmap */ #define DMA_BUS_MODE_DCHE BIT(19) #define DMA_BUS_MODE_INTM_MASK GENMASK(17, 16) -#define DMA_BUS_MODE_INTM_SHIFT 16 #define DMA_BUS_MODE_INTM_MODE1 0x1 #define DMA_BUS_MODE_SFT_RESET BIT(0) /* DMA SYS Bus Mode bitmap */ #define DMA_BUS_MODE_SPH BIT(24) -#define DMA_BUS_MODE_PBL BIT(16) -#define DMA_BUS_MODE_PBL_SHIFT 16 -#define DMA_BUS_MODE_RPBL_SHIFT 16 #define DMA_BUS_MODE_MB BIT(14) #define DMA_BUS_MODE_FB BIT(0) @@ -59,13 +55,7 @@ #define DMA_AXI_EN_LPI BIT(31) #define DMA_AXI_LPI_XIT_FRM BIT(30) #define DMA_AXI_WR_OSR_LMT GENMASK(27, 24) -#define DMA_AXI_WR_OSR_LMT_SHIFT 24 #define DMA_AXI_RD_OSR_LMT GENMASK(19, 16) -#define DMA_AXI_RD_OSR_LMT_SHIFT 16 - -#define DMA_AXI_OSR_MAX 0xf -#define DMA_AXI_MAX_OSR_LIMIT ((DMA_AXI_OSR_MAX << DMA_AXI_WR_OSR_LMT_SHIFT) | \ - (DMA_AXI_OSR_MAX << DMA_AXI_RD_OSR_LMT_SHIFT)) #define DMA_SYS_BUS_MB BIT(14) #define DMA_AXI_1KBBE BIT(13) @@ -134,19 +124,21 @@ static inline u32 dma_chanx_base_addr(const struct dwmac4_addrs *addrs, #define DMA_CHAN_STATUS(addrs, x) (dma_chanx_base_addr(addrs, x) + 0x60) /* DMA Control X */ +#define DMA_CHAN_CTRL_PBLX8 BIT(16) #define DMA_CONTROL_SPH BIT(24) #define DMA_CONTROL_MSS_MASK GENMASK(13, 0) /* DMA Tx Channel X Control register defines */ #define DMA_CONTROL_EDSE BIT(28) +#define DMA_CHAN_TX_CTRL_TXPBL_MASK GENMASK(21, 16) #define DMA_CONTROL_TSE BIT(12) #define DMA_CONTROL_OSP BIT(4) #define DMA_CONTROL_ST BIT(0) /* DMA Rx Channel X Control register defines */ -#define DMA_CONTROL_SR BIT(0) +#define DMA_CHAN_RX_CTRL_RXPBL_MASK GENMASK(21, 16) #define DMA_RBSZ_MASK GENMASK(14, 1) -#define DMA_RBSZ_SHIFT 1 +#define DMA_CONTROL_SR BIT(0) /* Interrupt status per channel */ #define DMA_CHAN_STATUS_REB GENMASK(21, 19) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c index 57c03d491774..c098047a3bff 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c @@ -234,7 +234,7 @@ void stmmac_dwmac4_set_mac_addr(void __iomem *ioaddr, const u8 addr[6], * bit that has no effect on the High Reg 0 where the bit 31 (MO) * is RO. */ - data |= (STMMAC_CHAN0 << GMAC_HI_DCS_SHIFT); + data |= FIELD_PREP(GMAC_HI_DCS, STMMAC_CHAN0); writel(data | GMAC_HI_REG_AE, ioaddr + high); data = (addr[3] << 24) | (addr[2] << 16) | (addr[1] << 8) | addr[0]; writel(data, ioaddr + low); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h b/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h index 5d9c18f5bbf5..14cfe0de3327 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h @@ -59,11 +59,7 @@ static inline u32 dma_chan_base_addr(u32 base, u32 chan) #define DMA_AXI_EN_LPI BIT(31) #define DMA_AXI_LPI_XIT_FRM BIT(30) #define DMA_AXI_WR_OSR_LMT GENMASK(23, 20) -#define DMA_AXI_WR_OSR_LMT_SHIFT 20 -#define DMA_AXI_WR_OSR_LMT_MASK 0xf #define DMA_AXI_RD_OSR_LMT GENMASK(19, 16) -#define DMA_AXI_RD_OSR_LMT_SHIFT 16 -#define DMA_AXI_RD_OSR_LMT_MASK 0xf #define DMA_AXI_OSR_MAX 0xf #define DMA_AXI_MAX_OSR_LIMIT ((DMA_AXI_OSR_MAX << DMA_AXI_WR_OSR_LMT_SHIFT) | \ @@ -132,10 +128,8 @@ static inline u32 dma_chan_base_addr(u32 base, u32 chan) #define DMA_STATUS_EB_MASK 0x00380000 /* Error Bits Mask */ #define DMA_STATUS_EB_TX_ABORT 0x00080000 /* Error Bits - TX Abort */ #define DMA_STATUS_EB_RX_ABORT 0x00100000 /* Error Bits - RX Abort */ -#define DMA_STATUS_TS_MASK 0x00700000 /* Transmit Process State */ -#define DMA_STATUS_TS_SHIFT 20 -#define DMA_STATUS_RS_MASK 0x000e0000 /* Receive Process State */ -#define DMA_STATUS_RS_SHIFT 17 +#define DMA_STATUS_TS_MASK GENMASK(22, 20) /* Transmit Process State */ +#define DMA_STATUS_RS_MASK GENMASK(19, 17) /* Receive Process State */ #define DMA_STATUS_NIS 0x00010000 /* Normal Interrupt Summary */ #define DMA_STATUS_AIS 0x00008000 /* Abnormal Interrupt Summary */ #define DMA_STATUS_ERI 0x00004000 /* Early Receive Interrupt */ diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c b/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c index 467f1a05747e..2e979f075665 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c @@ -92,10 +92,7 @@ void dwmac_dma_stop_rx(struct stmmac_priv *priv, void __iomem *ioaddr, u32 chan) #ifdef DWMAC_DMA_DEBUG static void show_tx_process_state(unsigned int status) { - unsigned int state; - state = (status & DMA_STATUS_TS_MASK) >> DMA_STATUS_TS_SHIFT; - - switch (state) { + switch (FIELD_GET(DMA_STATUS_TS_MASK, status)) { case 0: pr_debug("- TX (Stopped): Reset or Stop command\n"); break; @@ -123,10 +120,7 @@ static void show_tx_process_state(unsigned int status) static void show_rx_process_state(unsigned int status) { - unsigned int state; - state = (status & DMA_STATUS_RS_MASK) >> DMA_STATUS_RS_SHIFT; - - switch (state) { + switch (FIELD_GET(DMA_STATUS_RS_MASK, status)) { case 0: pr_debug("- RX (Stopped): Reset or Stop command\n"); break; diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h index 0d408ee17f33..67e2d539c338 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h @@ -24,17 +24,15 @@ #define XGMAC_CONFIG_SS_2500 (0x6 << XGMAC_CONFIG_SS_OFF) #define XGMAC_CONFIG_SS_10_MII (0x7 << XGMAC_CONFIG_SS_OFF) #define XGMAC_CONFIG_SARC GENMASK(22, 20) -#define XGMAC_CONFIG_SARC_SHIFT 20 #define XGMAC_CONFIG_JD BIT(16) #define XGMAC_CONFIG_TE BIT(0) #define XGMAC_CORE_INIT_TX (XGMAC_CONFIG_JD) #define XGMAC_RX_CONFIG 0x00000004 #define XGMAC_CONFIG_ARPEN BIT(31) #define XGMAC_CONFIG_GPSL GENMASK(29, 16) -#define XGMAC_CONFIG_GPSL_SHIFT 16 #define XGMAC_CONFIG_HDSMS GENMASK(14, 12) #define XGMAC_CONFIG_HDSMS_SHIFT 12 -#define XGMAC_CONFIG_HDSMS_256 (0x2 << XGMAC_CONFIG_HDSMS_SHIFT) +#define XGMAC_CONFIG_HDSMS_256 FIELD_PREP(XGMAC_CONFIG_HDSMS, 0x2) #define XGMAC_CONFIG_S2KP BIT(11) #define XGMAC_CONFIG_LM BIT(10) #define XGMAC_CONFIG_IPC BIT(9) @@ -44,8 +42,10 @@ #define XGMAC_CONFIG_CST BIT(2) #define XGMAC_CONFIG_ACS BIT(1) #define XGMAC_CONFIG_RE BIT(0) -#define XGMAC_CORE_INIT_RX (XGMAC_CONFIG_GPSLCE | XGMAC_CONFIG_WD | \ - (XGMAC_JUMBO_LEN << XGMAC_CONFIG_GPSL_SHIFT)) +#define XGMAC_CORE_INIT_RX (XGMAC_CONFIG_GPSLCE | \ + XGMAC_CONFIG_WD | \ + FIELD_PREP(XGMAC_CONFIG_GPSL, \ + XGMAC_JUMBO_LEN)) #define XGMAC_PACKET_FILTER 0x00000008 #define XGMAC_FILTER_RA BIT(31) #define XGMAC_FILTER_IPFE BIT(20) @@ -79,6 +79,7 @@ #define XGMAC_PSRQ(x) GENMASK((x) * 8 + 7, (x) * 8) #define XGMAC_PSRQ_SHIFT(x) ((x) * 8) #define XGMAC_INT_STATUS 0x000000b0 +#define XGMAC_INT_TSIS BIT(12) #define XGMAC_LPIIS BIT(5) #define XGMAC_PMTIS BIT(4) #define XGMAC_INT_EN 0x000000b4 @@ -89,7 +90,6 @@ #define XGMAC_INT_DEFAULT_EN (XGMAC_LPIIE | XGMAC_PMTIE) #define XGMAC_Qx_TX_FLOW_CTRL(x) (0x00000070 + (x) * 4) #define XGMAC_PT GENMASK(31, 16) -#define XGMAC_PT_SHIFT 16 #define XGMAC_TFE BIT(1) #define XGMAC_RX_FLOW_CTRL 0x00000090 #define XGMAC_RFE BIT(0) @@ -173,16 +173,17 @@ #define XGMAC_MDIO_ADDR 0x00000200 #define XGMAC_MDIO_DATA 0x00000204 #define XGMAC_MDIO_C22P 0x00000220 +#define XGMAC_GPIO_STATUS 0x0000027c +#define XGMAC_GPIO_GPO0 BIT(16) #define XGMAC_ADDRx_HIGH(x) (0x00000300 + (x) * 0x8) #define XGMAC_ADDR_MAX 32 #define XGMAC_AE BIT(31) #define XGMAC_DCS GENMASK(19, 16) -#define XGMAC_DCS_SHIFT 16 #define XGMAC_ADDRx_LOW(x) (0x00000304 + (x) * 0x8) #define XGMAC_L3L4_ADDR_CTRL 0x00000c00 #define XGMAC_IDDR GENMASK(16, 8) -#define XGMAC_IDDR_SHIFT 8 -#define XGMAC_IDDR_FNUM 4 +#define XGMAC_IDDR_FNUM_MASK GENMASK(7, 4) /* FNUM within IDDR */ +#define XGMAC_IDDR_REG_MASK GENMASK(3, 0) /* REG within IDDR */ #define XGMAC_TT BIT(1) #define XGMAC_XB BIT(0) #define XGMAC_L3L4_DATA 0x00000c04 @@ -201,7 +202,6 @@ #define XGMAC_L3PEN0 BIT(0) #define XGMAC_L4_ADDR 0x1 #define XGMAC_L4DP0 GENMASK(31, 16) -#define XGMAC_L4DP0_SHIFT 16 #define XGMAC_L4SP0 GENMASK(15, 0) #define XGMAC_L3_ADDR0 0x4 #define XGMAC_L3_ADDR1 0x5 @@ -220,6 +220,7 @@ #define XGMAC_OB BIT(0) #define XGMAC_RSS_DATA 0x00000c8c #define XGMAC_TIMESTAMP_STATUS 0x00000d20 +#define XGMAC_TIMESTAMP_ATSNS_MASK GENMASK(29, 25) #define XGMAC_TXTSC BIT(15) #define XGMAC_TXTIMESTAMP_NSEC 0x00000d30 #define XGMAC_TXTSSTSLO GENMASK(30, 0) @@ -285,13 +286,9 @@ #define XGMAC_DPP_DISABLE BIT(0) #define XGMAC_MTL_TXQ_OPMODE(x) (0x00001100 + (0x80 * (x))) #define XGMAC_TQS GENMASK(25, 16) -#define XGMAC_TQS_SHIFT 16 #define XGMAC_Q2TCMAP GENMASK(10, 8) -#define XGMAC_Q2TCMAP_SHIFT 8 #define XGMAC_TTC GENMASK(6, 4) -#define XGMAC_TTC_SHIFT 4 #define XGMAC_TXQEN GENMASK(3, 2) -#define XGMAC_TXQEN_SHIFT 2 #define XGMAC_TSF BIT(1) #define XGMAC_MTL_TCx_ETS_CONTROL(x) (0x00001110 + (0x80 * (x))) #define XGMAC_MTL_TCx_QUANTUM_WEIGHT(x) (0x00001118 + (0x80 * (x))) @@ -305,16 +302,12 @@ #define XGMAC_ETS (0x2 << 0) #define XGMAC_MTL_RXQ_OPMODE(x) (0x00001140 + (0x80 * (x))) #define XGMAC_RQS GENMASK(25, 16) -#define XGMAC_RQS_SHIFT 16 #define XGMAC_EHFC BIT(7) #define XGMAC_RSF BIT(5) #define XGMAC_RTC GENMASK(1, 0) -#define XGMAC_RTC_SHIFT 0 #define XGMAC_MTL_RXQ_FLOW_CONTROL(x) (0x00001150 + (0x80 * (x))) #define XGMAC_RFD GENMASK(31, 17) -#define XGMAC_RFD_SHIFT 17 #define XGMAC_RFA GENMASK(15, 1) -#define XGMAC_RFA_SHIFT 1 #define XGMAC_MTL_QINTEN(x) (0x00001170 + (0x80 * (x))) #define XGMAC_RXOIE BIT(16) #define XGMAC_MTL_QINT_STATUS(x) (0x00001174 + (0x80 * (x))) @@ -328,9 +321,7 @@ #define XGMAC_SWR BIT(0) #define XGMAC_DMA_SYSBUS_MODE 0x00003004 #define XGMAC_WR_OSR_LMT GENMASK(29, 24) -#define XGMAC_WR_OSR_LMT_SHIFT 24 #define XGMAC_RD_OSR_LMT GENMASK(21, 16) -#define XGMAC_RD_OSR_LMT_SHIFT 16 #define XGMAC_EN_LPI BIT(15) #define XGMAC_LPI_XIT_PKT BIT(14) #define XGMAC_AAL BIT(12) @@ -372,15 +363,12 @@ #define XGMAC_DMA_CH_TX_CONTROL(x) (0x00003104 + (0x80 * (x))) #define XGMAC_EDSE BIT(28) #define XGMAC_TxPBL GENMASK(21, 16) -#define XGMAC_TxPBL_SHIFT 16 #define XGMAC_TSE BIT(12) #define XGMAC_OSP BIT(4) #define XGMAC_TXST BIT(0) #define XGMAC_DMA_CH_RX_CONTROL(x) (0x00003108 + (0x80 * (x))) #define XGMAC_RxPBL GENMASK(21, 16) -#define XGMAC_RxPBL_SHIFT 16 #define XGMAC_RBSZ GENMASK(14, 1) -#define XGMAC_RBSZ_SHIFT 1 #define XGMAC_RXST BIT(0) #define XGMAC_DMA_CH_TxDESC_HADDR(x) (0x00003110 + (0x80 * (x))) #define XGMAC_DMA_CH_TxDESC_LADDR(x) (0x00003114 + (0x80 * (x))) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c index 00e929bf280b..ee41c77426f1 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c @@ -376,7 +376,7 @@ static void dwxgmac2_flow_ctrl(struct mac_device_info *hw, unsigned int duplex, u32 value = XGMAC_TFE; if (duplex) - value |= pause_time << XGMAC_PT_SHIFT; + value |= FIELD_PREP(XGMAC_PT, pause_time); writel(value, ioaddr + XGMAC_Qx_TX_FLOW_CTRL(i)); } @@ -1233,8 +1233,7 @@ static void dwxgmac2_sarc_configure(void __iomem *ioaddr, int val) { u32 value = readl(ioaddr + XGMAC_TX_CONFIG); - value &= ~XGMAC_CONFIG_SARC; - value |= val << XGMAC_CONFIG_SARC_SHIFT; + value = u32_replace_bits(value, val, XGMAC_CONFIG_SARC); writel(value, ioaddr + XGMAC_TX_CONFIG); } @@ -1254,14 +1253,16 @@ static int dwxgmac2_filter_read(struct mac_device_info *hw, u32 filter_no, u8 reg, u32 *data) { void __iomem *ioaddr = hw->pcsr; - u32 value; + u32 value, iddr; int ret; ret = dwxgmac2_filter_wait(hw); if (ret) return ret; - value = ((filter_no << XGMAC_IDDR_FNUM) | reg) << XGMAC_IDDR_SHIFT; + iddr = FIELD_PREP(XGMAC_IDDR_FNUM_MASK, filter_no) | + FIELD_PREP(XGMAC_IDDR_REG_MASK, reg); + value = FIELD_PREP(XGMAC_IDDR, iddr); value |= XGMAC_TT | XGMAC_XB; writel(value, ioaddr + XGMAC_L3L4_ADDR_CTRL); @@ -1277,7 +1278,7 @@ static int dwxgmac2_filter_write(struct mac_device_info *hw, u32 filter_no, u8 reg, u32 data) { void __iomem *ioaddr = hw->pcsr; - u32 value; + u32 value, iddr; int ret; ret = dwxgmac2_filter_wait(hw); @@ -1286,7 +1287,9 @@ static int dwxgmac2_filter_write(struct mac_device_info *hw, u32 filter_no, writel(data, ioaddr + XGMAC_L3L4_DATA); - value = ((filter_no << XGMAC_IDDR_FNUM) | reg) << XGMAC_IDDR_SHIFT; + iddr = FIELD_PREP(XGMAC_IDDR_FNUM_MASK, filter_no) | + FIELD_PREP(XGMAC_IDDR_REG_MASK, reg); + value = FIELD_PREP(XGMAC_IDDR, iddr); value |= XGMAC_XB; writel(value, ioaddr + XGMAC_L3L4_ADDR_CTRL); @@ -1378,36 +1381,40 @@ static int dwxgmac2_config_l4_filter(struct mac_device_info *hw, u32 filter_no, value &= ~XGMAC_L4PEN0; } - value &= ~(XGMAC_L4SPM0 | XGMAC_L4SPIM0); - value &= ~(XGMAC_L4DPM0 | XGMAC_L4DPIM0); if (sa) { value |= XGMAC_L4SPM0; if (inv) value |= XGMAC_L4SPIM0; + else + value &= ~XGMAC_L4SPIM0; } else { value |= XGMAC_L4DPM0; if (inv) value |= XGMAC_L4DPIM0; + else + value &= ~XGMAC_L4DPIM0; } ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L3L4_CTRL, value); if (ret) return ret; - if (sa) { - value = match & XGMAC_L4SP0; + ret = dwxgmac2_filter_read(hw, filter_no, XGMAC_L4_ADDR, &value); + if (ret) + return ret; - ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L4_ADDR, value); - if (ret) - return ret; + if (sa) { + value &= ~XGMAC_L4SP0; + value |= FIELD_PREP(XGMAC_L4SP0, match); } else { - value = (match << XGMAC_L4DP0_SHIFT) & XGMAC_L4DP0; - - ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L4_ADDR, value); - if (ret) - return ret; + value &= ~XGMAC_L4DP0; + value |= FIELD_PREP(XGMAC_L4DP0, match); } + ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L4_ADDR, value); + if (ret) + return ret; + if (!en) return dwxgmac2_filter_write(hw, filter_no, XGMAC_L3L4_CTRL, 0); diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c index 4d6bb995d8d8..964ce2d73316 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c @@ -55,8 +55,7 @@ static void dwxgmac2_dma_init_rx_chan(struct stmmac_priv *priv, u32 value; value = readl(ioaddr + XGMAC_DMA_CH_RX_CONTROL(chan)); - value &= ~XGMAC_RxPBL; - value |= (rxpbl << XGMAC_RxPBL_SHIFT) & XGMAC_RxPBL; + value = u32_replace_bits(value, rxpbl, XGMAC_RxPBL); writel(value, ioaddr + XGMAC_DMA_CH_RX_CONTROL(chan)); writel(upper_32_bits(phy), ioaddr + XGMAC_DMA_CH_RxDESC_HADDR(chan)); @@ -72,9 +71,7 @@ static void dwxgmac2_dma_init_tx_chan(struct stmmac_priv *priv, u32 value; value = readl(ioaddr + XGMAC_DMA_CH_TX_CONTROL(chan)); - value &= ~XGMAC_TxPBL; - value |= (txpbl << XGMAC_TxPBL_SHIFT) & XGMAC_TxPBL; - value |= XGMAC_OSP; + value = u32_replace_bits(value, txpbl, XGMAC_TxPBL); writel(value, ioaddr + XGMAC_DMA_CH_TX_CONTROL(chan)); writel(upper_32_bits(phy), ioaddr + XGMAC_DMA_CH_TxDESC_HADDR(chan)); @@ -91,13 +88,8 @@ static void dwxgmac2_dma_axi(void __iomem *ioaddr, struct stmmac_axi *axi) if (axi->axi_xit_frm) value |= XGMAC_LPI_XIT_PKT; - value &= ~XGMAC_WR_OSR_LMT; - value |= (axi->axi_wr_osr_lmt << XGMAC_WR_OSR_LMT_SHIFT) & - XGMAC_WR_OSR_LMT; - - value &= ~XGMAC_RD_OSR_LMT; - value |= (axi->axi_rd_osr_lmt << XGMAC_RD_OSR_LMT_SHIFT) & - XGMAC_RD_OSR_LMT; + value = u32_replace_bits(value, axi->axi_wr_osr_lmt, XGMAC_WR_OSR_LMT); + value = u32_replace_bits(value, axi->axi_rd_osr_lmt, XGMAC_RD_OSR_LMT); if (!axi->axi_fb) value |= XGMAC_UNDEF; @@ -148,23 +140,24 @@ static void dwxgmac2_dma_rx_mode(struct stmmac_priv *priv, void __iomem *ioaddr, { u32 value = readl(ioaddr + XGMAC_MTL_RXQ_OPMODE(channel)); unsigned int rqs = fifosz / 256 - 1; + unsigned int rtc; if (mode == SF_DMA_MODE) { value |= XGMAC_RSF; } else { value &= ~XGMAC_RSF; - value &= ~XGMAC_RTC; if (mode <= 64) - value |= 0x0 << XGMAC_RTC_SHIFT; + rtc = 0x0; else if (mode <= 96) - value |= 0x2 << XGMAC_RTC_SHIFT; + rtc = 0x2; else - value |= 0x3 << XGMAC_RTC_SHIFT; + rtc = 0x3; + + value = u32_replace_bits(value, rtc, XGMAC_RTC); } - value &= ~XGMAC_RQS; - value |= (rqs << XGMAC_RQS_SHIFT) & XGMAC_RQS; + value = u32_replace_bits(value, rqs, XGMAC_RQS); if ((fifosz >= 4096) && (qmode != MTL_QUEUE_AVB)) { u32 flow = readl(ioaddr + XGMAC_MTL_RXQ_FLOW_CONTROL(channel)); @@ -193,11 +186,8 @@ static void dwxgmac2_dma_rx_mode(struct stmmac_priv *priv, void __iomem *ioaddr, break; } - flow &= ~XGMAC_RFD; - flow |= rfd << XGMAC_RFD_SHIFT; - - flow &= ~XGMAC_RFA; - flow |= rfa << XGMAC_RFA_SHIFT; + flow = u32_replace_bits(flow, rfd, XGMAC_RFD); + flow = u32_replace_bits(flow, rfa, XGMAC_RFA); writel(flow, ioaddr + XGMAC_MTL_RXQ_FLOW_CONTROL(channel)); } @@ -210,40 +200,41 @@ static void dwxgmac2_dma_tx_mode(struct stmmac_priv *priv, void __iomem *ioaddr, { u32 value = readl(ioaddr + XGMAC_MTL_TXQ_OPMODE(channel)); unsigned int tqs = fifosz / 256 - 1; + unsigned int ttc, txqen; if (mode == SF_DMA_MODE) { value |= XGMAC_TSF; } else { value &= ~XGMAC_TSF; - value &= ~XGMAC_TTC; if (mode <= 64) - value |= 0x0 << XGMAC_TTC_SHIFT; + ttc = 0x0; else if (mode <= 96) - value |= 0x2 << XGMAC_TTC_SHIFT; + ttc = 0x2; else if (mode <= 128) - value |= 0x3 << XGMAC_TTC_SHIFT; + ttc = 0x3; else if (mode <= 192) - value |= 0x4 << XGMAC_TTC_SHIFT; + ttc = 0x4; else if (mode <= 256) - value |= 0x5 << XGMAC_TTC_SHIFT; + ttc = 0x5; else if (mode <= 384) - value |= 0x6 << XGMAC_TTC_SHIFT; + ttc = 0x6; else - value |= 0x7 << XGMAC_TTC_SHIFT; + ttc = 0x7; + + value = u32_replace_bits(value, ttc, XGMAC_TTC); } /* Use static TC to Queue mapping */ - value |= (channel << XGMAC_Q2TCMAP_SHIFT) & XGMAC_Q2TCMAP; + value |= FIELD_PREP(XGMAC_Q2TCMAP, channel); - value &= ~XGMAC_TXQEN; if (qmode != MTL_QUEUE_AVB) - value |= 0x2 << XGMAC_TXQEN_SHIFT; + txqen = 0x2; else - value |= 0x1 << XGMAC_TXQEN_SHIFT; + txqen = 0x1; - value &= ~XGMAC_TQS; - value |= (tqs << XGMAC_TQS_SHIFT) & XGMAC_TQS; + value = u32_replace_bits(value, txqen, XGMAC_TXQEN); + value = u32_replace_bits(value, tqs, XGMAC_TQS); writel(value, ioaddr + XGMAC_MTL_TXQ_OPMODE(channel)); } @@ -547,16 +538,17 @@ static void dwxgmac2_qmode(struct stmmac_priv *priv, void __iomem *ioaddr, { u32 value = readl(ioaddr + XGMAC_MTL_TXQ_OPMODE(channel)); u32 flow = readl(ioaddr + XGMAC_RX_FLOW_CTRL); + unsigned int txqen; - value &= ~XGMAC_TXQEN; if (qmode != MTL_QUEUE_AVB) { - value |= 0x2 << XGMAC_TXQEN_SHIFT; + txqen = 0x2; writel(0, ioaddr + XGMAC_MTL_TCx_ETS_CONTROL(channel)); } else { - value |= 0x1 << XGMAC_TXQEN_SHIFT; + txqen = 0x1; writel(flow & (~XGMAC_RFE), ioaddr + XGMAC_RX_FLOW_CTRL); } + value = u32_replace_bits(value, txqen, XGMAC_TXQEN); writel(value, ioaddr + XGMAC_MTL_TXQ_OPMODE(channel)); } @@ -566,8 +558,7 @@ static void dwxgmac2_set_bfsize(struct stmmac_priv *priv, void __iomem *ioaddr, u32 value; value = readl(ioaddr + XGMAC_DMA_CH_RX_CONTROL(chan)); - value &= ~XGMAC_RBSZ; - value |= bfsize << XGMAC_RBSZ_SHIFT; + value = u32_replace_bits(value, bfsize, XGMAC_RBSZ); writel(value, ioaddr + XGMAC_DMA_CH_RX_CONTROL(chan)); } diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 1ceedd74e429..3be0b7953249 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -892,63 +892,45 @@ static void stmmac_mac_link_up(struct phylink_config *config, old_ctrl = readl(priv->ioaddr + MAC_CTRL_REG); ctrl = old_ctrl & ~priv->hw->link.speed_mask; - if (interface == PHY_INTERFACE_MODE_USXGMII) { - switch (speed) { - case SPEED_10000: - ctrl |= priv->hw->link.xgmii.speed10000; - break; - case SPEED_5000: - ctrl |= priv->hw->link.xgmii.speed5000; - break; - case SPEED_2500: + switch (speed) { + case SPEED_100000: + ctrl |= priv->hw->link.xlgmii.speed100000; + break; + case SPEED_50000: + ctrl |= priv->hw->link.xlgmii.speed50000; + break; + case SPEED_40000: + ctrl |= priv->hw->link.xlgmii.speed40000; + break; + case SPEED_25000: + ctrl |= priv->hw->link.xlgmii.speed25000; + break; + case SPEED_10000: + ctrl |= priv->hw->link.xgmii.speed10000; + break; + case SPEED_5000: + ctrl |= priv->hw->link.xgmii.speed5000; + break; + case SPEED_2500: + if (interface == PHY_INTERFACE_MODE_USXGMII) ctrl |= priv->hw->link.xgmii.speed2500; - break; - default: - return; - } - } else if (interface == PHY_INTERFACE_MODE_XLGMII) { - switch (speed) { - case SPEED_100000: - ctrl |= priv->hw->link.xlgmii.speed100000; - break; - case SPEED_50000: - ctrl |= priv->hw->link.xlgmii.speed50000; - break; - case SPEED_40000: - ctrl |= priv->hw->link.xlgmii.speed40000; - break; - case SPEED_25000: - ctrl |= priv->hw->link.xlgmii.speed25000; - break; - case SPEED_10000: - ctrl |= priv->hw->link.xgmii.speed10000; - break; - case SPEED_2500: - ctrl |= priv->hw->link.speed2500; - break; - case SPEED_1000: - ctrl |= priv->hw->link.speed1000; - break; - default: - return; - } - } else { - switch (speed) { - case SPEED_2500: + else ctrl |= priv->hw->link.speed2500; - break; - case SPEED_1000: - ctrl |= priv->hw->link.speed1000; - break; - case SPEED_100: - ctrl |= priv->hw->link.speed100; - break; - case SPEED_10: - ctrl |= priv->hw->link.speed10; - break; - default: - return; - } + break; + case SPEED_1000: + ctrl |= priv->hw->link.speed1000; + break; + case SPEED_100: + ctrl |= priv->hw->link.speed100; + break; + case SPEED_10: + ctrl |= priv->hw->link.speed10; + break; + default: + netdev_err(priv->dev, + "unsupported speed %s on %s, leaving the MAC disabled\n", + phy_speed_to_str(speed), phy_modes(interface)); + return; } if (priv->plat->fix_mac_speed) diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c index 3b4d4696afe9..0119b4b89fc0 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c @@ -446,6 +446,7 @@ static int tc_parse_flow_actions(struct stmmac_priv *priv, } #define ETHER_TYPE_FULL_MASK cpu_to_be16(~0) +#define IP_PROTO_FULL_MASK 0xFF static int tc_add_basic_flow(struct stmmac_priv *priv, struct flow_cls_offload *cls, @@ -461,6 +462,37 @@ static int tc_add_basic_flow(struct stmmac_priv *priv, flow_rule_match_basic(rule, &match); + /* Both network proto and transport proto not present in the key */ + if (!match.mask || !(match.mask->n_proto || match.mask->ip_proto)) { + NL_SET_ERR_MSG_MOD(cls->common.extack, + "filter must specify network or transport protocol"); + return -EOPNOTSUPP; + } + + /* If the proto is present in the key and is not full mask */ + if ((match.mask->n_proto && match.mask->n_proto != ETHER_TYPE_FULL_MASK) || + (match.mask->ip_proto && match.mask->ip_proto != IP_PROTO_FULL_MASK)) { + NL_SET_ERR_MSG_MOD(cls->common.extack, + "only full protocol mask is supported"); + return -EOPNOTSUPP; + } + + /* Network proto is present in the key and is not IPv4 */ + if (match.mask->n_proto && match.key->n_proto != cpu_to_be16(ETH_P_IP)) { + NL_SET_ERR_MSG_MOD(cls->common.extack, + "only IPv4 network protocol is supported"); + return -EOPNOTSUPP; + } + + /* Transport proto is present in the key and is not TCP or UDP */ + if (match.mask->ip_proto && + match.key->ip_proto != IPPROTO_TCP && + match.key->ip_proto != IPPROTO_UDP) { + NL_SET_ERR_MSG_MOD(cls->common.extack, + "only TCP and UDP transport protocols are supported"); + return -EOPNOTSUPP; + } + entry->ip_proto = match.key->ip_proto; return 0; } @@ -598,6 +630,8 @@ static int tc_add_flow(struct stmmac_priv *priv, ret = tc_flow_parsers[i].fn(priv, cls, entry); if (!ret) entry->in_use = true; + else if (ret == -EOPNOTSUPP) + return ret; } if (!entry->in_use) @@ -627,6 +661,7 @@ static int tc_del_flow(struct stmmac_priv *priv, entry->in_use = false; entry->cookie = 0; entry->is_l4 = false; + entry->action = 0; return ret; } diff --git a/drivers/net/ethernet/wangxun/txgbe/txgbe_main.c b/drivers/net/ethernet/wangxun/txgbe/txgbe_main.c index 4d20b178af23..76d2940a49fa 100644 --- a/drivers/net/ethernet/wangxun/txgbe/txgbe_main.c +++ b/drivers/net/ethernet/wangxun/txgbe/txgbe_main.c @@ -956,6 +956,7 @@ static void txgbe_remove(struct pci_dev *pdev) netdev = wx->netdev; wx_disable_sriov(wx); unregister_netdev(netdev); + txgbe_fdir_filter_exit(wx); txgbe_remove_phy(txgbe); wx_free_isb_resources(wx); diff --git a/drivers/net/geneve.c b/drivers/net/geneve.c index 77b0c3d52041..73e020270b98 100644 --- a/drivers/net/geneve.c +++ b/drivers/net/geneve.c @@ -1736,6 +1736,9 @@ static int geneve_changelink(struct net_device *dev, struct nlattr *tb[], struct geneve_config cfg; int err; + if (!rtnl_dev_link_net_capable(dev, geneve->net)) + return -EPERM; + /* If the geneve device is configured for metadata (or externally * controlled, for example, OVS), then nothing can be changed. */ diff --git a/drivers/net/gtp.c b/drivers/net/gtp.c index 2f8626c3824d..98614d0b390b 100644 --- a/drivers/net/gtp.c +++ b/drivers/net/gtp.c @@ -669,8 +669,9 @@ static int gtp1u_send_echo_resp(struct gtp_dev *gtp, struct sk_buff *skb) return -1; /* pull GTP and UDP headers */ - skb_pull_data(skb, - sizeof(struct gtp1_header_long) + sizeof(struct udphdr)); + if (!skb_pull_data(skb, sizeof(struct gtp1_header_long) + + sizeof(struct udphdr))) + return -1; gtp_pkt = skb_push(skb, sizeof(struct gtp1u_packet)); memset(gtp_pkt, 0, sizeof(struct gtp1u_packet)); @@ -826,13 +827,17 @@ static int gtp1u_udp_encap_recv(struct gtp_dev *gtp, struct sk_buff *skb) if (!pskb_may_pull(skb, hdrlen)) return -1; + gtp1 = (struct gtp1_header *)(skb->data + sizeof(struct udphdr)); + + if (gtp1->flags & GTP1_F_EXTHDR && + gtp_parse_exthdrs(skb, &hdrlen) < 0) + return -1; + if (gtp_inner_proto(skb, hdrlen, &inner_proto) < 0) { netdev_dbg(gtp->dev, "GTP packet does not encapsulate an IP packet\n"); return -1; } - gtp1 = (struct gtp1_header *)(skb->data + sizeof(struct udphdr)); - pctx = gtp1_pdp_find(gtp, ntohl(gtp1->tid), gtp_proto_to_family(inner_proto)); if (!pctx) { @@ -840,10 +845,6 @@ static int gtp1u_udp_encap_recv(struct gtp_dev *gtp, struct sk_buff *skb) return 1; } - if (gtp1->flags & GTP1_F_EXTHDR && - gtp_parse_exthdrs(skb, &hdrlen) < 0) - return -1; - return gtp_rx(pctx, skb, hdrlen, gtp->role, inner_proto); } diff --git a/drivers/net/ipa/ipa_smp2p.c b/drivers/net/ipa/ipa_smp2p.c index fcaadd111a8a..0c184f6f91c6 100644 --- a/drivers/net/ipa/ipa_smp2p.c +++ b/drivers/net/ipa/ipa_smp2p.c @@ -234,19 +234,27 @@ ipa_smp2p_init(struct ipa *ipa, struct platform_device *pdev, bool modem_init) &valid_bit); if (IS_ERR(valid_state)) return PTR_ERR(valid_state); - if (valid_bit >= 32) /* BITS_PER_U32 */ - return -EINVAL; + if (valid_bit >= 32) { /* BITS_PER_U32 */ + ret = -EINVAL; + goto err_valid_state_put; + } enabled_state = qcom_smem_state_get(dev, "ipa-clock-enabled", &enabled_bit); - if (IS_ERR(enabled_state)) - return PTR_ERR(enabled_state); - if (enabled_bit >= 32) /* BITS_PER_U32 */ - return -EINVAL; + if (IS_ERR(enabled_state)) { + ret = PTR_ERR(enabled_state); + goto err_valid_state_put; + } + if (enabled_bit >= 32) { /* BITS_PER_U32 */ + ret = -EINVAL; + goto err_enabled_state_put; + } smp2p = kzalloc(sizeof(*smp2p), GFP_KERNEL); - if (!smp2p) - return -ENOMEM; + if (!smp2p) { + ret = -ENOMEM; + goto err_enabled_state_put; + } smp2p->ipa = ipa; @@ -291,6 +299,10 @@ ipa_smp2p_init(struct ipa *ipa, struct platform_device *pdev, bool modem_init) ipa->smp2p = NULL; mutex_destroy(&smp2p->mutex); kfree(smp2p); +err_enabled_state_put: + qcom_smem_state_put(enabled_state); +err_valid_state_put: + qcom_smem_state_put(valid_state); return ret; } @@ -307,6 +319,8 @@ void ipa_smp2p_exit(struct ipa *ipa) ipa_smp2p_power_release(ipa); ipa->smp2p = NULL; mutex_destroy(&smp2p->mutex); + qcom_smem_state_put(smp2p->enabled_state); + qcom_smem_state_put(smp2p->valid_state); kfree(smp2p); } diff --git a/drivers/net/mctp/mctp-i3c.c b/drivers/net/mctp/mctp-i3c.c index c678f79aa356..c5d0bc517fea 100644 --- a/drivers/net/mctp/mctp-i3c.c +++ b/drivers/net/mctp/mctp-i3c.c @@ -731,18 +731,21 @@ static __init int mctp_i3c_mod_init(void) int rc; rc = i3c_register_notifier(&mctp_i3c_notifier); - if (rc < 0) { - i3c_driver_unregister(&mctp_i3c_driver); + if (rc < 0) return rc; - } i3c_for_each_bus_locked(mctp_i3c_bus_add_new, NULL); rc = i3c_driver_register(&mctp_i3c_driver); if (rc < 0) - return rc; + goto err_unregister_notifier; return 0; + +err_unregister_notifier: + i3c_unregister_notifier(&mctp_i3c_notifier); + mctp_i3c_bus_remove_all(); + return rc; } static __exit void mctp_i3c_mod_exit(void) diff --git a/drivers/net/mctp/mctp-serial.c b/drivers/net/mctp/mctp-serial.c index 26c9a33fd636..a5070ffa9a95 100644 --- a/drivers/net/mctp/mctp-serial.c +++ b/drivers/net/mctp/mctp-serial.c @@ -318,7 +318,7 @@ static void mctp_serial_push_header(struct mctp_serial *dev, u8 c) } else { dev->rxlen = c; dev->rxpos = 0; - dev->rxstate = STATE_DATA; + dev->rxstate = c > 0 ? STATE_DATA : STATE_TRAILER; dev->rxfcs = crc_ccitt_byte(dev->rxfcs, c); } break; diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index c03e58e28a86..0008a3d30c19 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -137,7 +137,7 @@ void ovpn_decrypt_post(void *data, int ret) } /* keep track of last received authenticated packet for keepalive */ - WRITE_ONCE(peer->last_recv, ktime_get_real_seconds()); + WRITE_ONCE(peer->last_recv, ktime_get_boottime_seconds()); rcu_read_lock(); sock = rcu_dereference(peer->sock); @@ -291,7 +291,7 @@ void ovpn_encrypt_post(void *data, int ret) ovpn_peer_stats_increment_tx(&peer->link_stats, orig_len); /* keep track of last sent packet for keepalive */ - WRITE_ONCE(peer->last_sent, ktime_get_real_seconds()); + WRITE_ONCE(peer->last_sent, ktime_get_boottime_seconds()); /* skb passed down the stack - don't free it */ skb = NULL; err_unlock: diff --git a/drivers/net/ovpn/peer.c b/drivers/net/ovpn/peer.c index 87a83321f1dd..85fa9bd475f4 100644 --- a/drivers/net/ovpn/peer.c +++ b/drivers/net/ovpn/peer.c @@ -26,11 +26,12 @@ static void unlock_ovpn(struct ovpn_priv *ovpn, struct llist_head *release_list) __releases(&ovpn->lock) { - struct ovpn_peer *peer; + struct ovpn_peer *peer, *next; spin_unlock_bh(&ovpn->lock); - llist_for_each_entry(peer, release_list->first, release_entry) { + llist_for_each_entry_safe(peer, next, release_list->first, + release_entry) { ovpn_socket_release(peer); ovpn_peer_put(peer); } @@ -44,7 +45,7 @@ static void unlock_ovpn(struct ovpn_priv *ovpn, */ void ovpn_peer_keepalive_set(struct ovpn_peer *peer, u32 interval, u32 timeout) { - time64_t now = ktime_get_real_seconds(); + time64_t now = ktime_get_boottime_seconds(); netdev_dbg(peer->ovpn->dev, "scheduling keepalive for peer %u: interval=%u timeout=%u\n", @@ -1162,7 +1163,6 @@ static void ovpn_peer_release_p2p(struct ovpn_priv *ovpn, struct sock *sk, ovpn_sock = rcu_access_pointer(peer->sock); if (!ovpn_sock || ovpn_sock->sk != sk) { spin_unlock_bh(&ovpn->lock); - ovpn_peer_put(peer); return; } } @@ -1280,8 +1280,10 @@ static time64_t ovpn_peer_keepalive_work_single(struct ovpn_peer *peer, netdev_dbg(peer->ovpn->dev, "sending keepalive to peer %u\n", peer->id); - if (schedule_work(&peer->keepalive_work)) - ovpn_peer_hold(peer); + if (WARN_ON(!ovpn_peer_hold(peer))) + return 0; + if (!schedule_work(&peer->keepalive_work)) + ovpn_peer_put(peer); } if (next_run1 < next_run2) @@ -1352,7 +1354,7 @@ void ovpn_peer_keepalive_work(struct work_struct *work) { struct ovpn_priv *ovpn = container_of(work, struct ovpn_priv, keepalive_work.work); - time64_t next_run = 0, now = ktime_get_real_seconds(); + time64_t next_run = 0, now = ktime_get_boottime_seconds(); LLIST_HEAD(release_list); spin_lock_bh(&ovpn->lock); diff --git a/drivers/net/ovpn/tcp.c b/drivers/net/ovpn/tcp.c index 433bd07a4f1b..0af14055c39a 100644 --- a/drivers/net/ovpn/tcp.c +++ b/drivers/net/ovpn/tcp.c @@ -151,7 +151,8 @@ static void ovpn_tcp_rcv(struct strparser *strp, struct sk_buff *skb) /* take reference for deferred peer deletion. should never fail */ if (WARN_ON(!ovpn_peer_hold(peer))) goto err_nopeer; - schedule_work(&peer->tcp.defer_del_work); + if (!schedule_work(&peer->tcp.defer_del_work)) + ovpn_peer_put(peer); ovpn_dev_dstats_rx_dropped(peer->ovpn->dev); err_nopeer: kfree_skb(skb); @@ -283,7 +284,8 @@ static void ovpn_tcp_send_sock(struct ovpn_peer *peer, struct sock *sk) * stream therefore we abort the connection */ ovpn_peer_hold(peer); - schedule_work(&peer->tcp.defer_del_work); + if (!schedule_work(&peer->tcp.defer_del_work)) + ovpn_peer_put(peer); /* we bail out immediately and keep tx_in_progress set * to true. This way we prevent more TX attempts diff --git a/drivers/net/pcs/pcs-xpcs.c b/drivers/net/pcs/pcs-xpcs.c index 3d1bd5aac093..14c4b10fee74 100644 --- a/drivers/net/pcs/pcs-xpcs.c +++ b/drivers/net/pcs/pcs-xpcs.c @@ -958,6 +958,7 @@ static int xpcs_get_state_c37_sgmii(struct dw_xpcs *xpcs, /* Reset link_state */ state->link = false; + state->an_complete = false; state->speed = SPEED_UNKNOWN; state->duplex = DUPLEX_UNKNOWN; state->pause = 0; @@ -969,6 +970,8 @@ static int xpcs_get_state_c37_sgmii(struct dw_xpcs *xpcs, if (ret < 0) return ret; + state->an_complete = ret & DW_VR_MII_AN_STS_C37_ANCMPLT_INTR; + if (ret & DW_VR_MII_C37_ANSGM_SP_LNKSTS) { int speed_value; @@ -986,34 +989,13 @@ static int xpcs_get_state_c37_sgmii(struct dw_xpcs *xpcs, state->duplex = DUPLEX_FULL; else state->duplex = DUPLEX_HALF; - } else if (ret == DW_VR_MII_AN_STS_C37_ANCMPLT_INTR) { - int speed, duplex; - - state->link = true; - - speed = xpcs_read(xpcs, MDIO_MMD_VEND2, MII_BMCR); - if (speed < 0) - return speed; - - speed &= BMCR_SPEED100 | BMCR_SPEED1000; - if (speed == BMCR_SPEED1000) - state->speed = SPEED_1000; - else if (speed == BMCR_SPEED100) - state->speed = SPEED_100; - else if (speed == 0) - state->speed = SPEED_10; - - duplex = xpcs_read(xpcs, MDIO_MMD_VEND2, MII_ADVERTISE); - if (duplex < 0) - return duplex; - if (duplex & ADVERTISE_1000XFULL) - state->duplex = DUPLEX_FULL; - else if (duplex & ADVERTISE_1000XHALF) - state->duplex = DUPLEX_HALF; + return 0; + } + /* Clear AN complete status or interrupt */ + if (state->an_complete) xpcs_write(xpcs, MDIO_MMD_VEND2, DW_VR_MII_AN_INTR_STS, 0); - } return 0; } diff --git a/drivers/net/phy/marvell.c b/drivers/net/phy/marvell.c index c248c90510ae..f191b8c43218 100644 --- a/drivers/net/phy/marvell.c +++ b/drivers/net/phy/marvell.c @@ -753,7 +753,7 @@ static int m88e1111_config_inband(struct phy_device *phydev, unsigned int modes) err = phy_modify(phydev, MII_M1111_PHY_EXT_SR, MII_M1111_HWCFG_SERIAL_AN_BYPASS, extsr); if (err < 0) - return extsr; + return err; return phy_modify_paged(phydev, MII_MARVELL_FIBER_PAGE, MII_BMCR, BMCR_ANENABLE, bmcr); diff --git a/drivers/net/ppp/ppp_generic.c b/drivers/net/ppp/ppp_generic.c index 507d216256c0..14fa87a8e083 100644 --- a/drivers/net/ppp/ppp_generic.c +++ b/drivers/net/ppp/ppp_generic.c @@ -134,7 +134,6 @@ struct ppp { int debug; /* debug flags 70 */ struct slcompress *vj; /* state for VJ header compression */ enum NPmode npmode[NUM_NP]; /* what to do with each net proto 78 */ - struct sk_buff *xmit_pending; /* a packet ready to go out 88 */ struct compressor *xcomp; /* transmit packet compressor 8c */ void *xc_state; /* its internal state 90 */ struct compressor *rcomp; /* receive decompressor 94 */ @@ -185,6 +184,7 @@ struct channel { struct list_head clist; /* link in list of channels per unit */ spinlock_t upl; /* protects `ppp' and 'bridge' */ struct channel __rcu *bridge; /* "bridged" ppp channel */ + struct rcu_head rcu; /* for RCU-deferred free of the channel */ #ifdef CONFIG_PPP_MULTILINK u8 avail; /* flag used in multilink stuff */ u8 had_frag; /* >= 1 fragments have been sent */ @@ -264,8 +264,8 @@ struct ppp_net { static int ppp_unattached_ioctl(struct net *net, struct ppp_file *pf, struct file *file, unsigned int cmd, unsigned long arg); static void ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb); -static void ppp_send_frame(struct ppp *ppp, struct sk_buff *skb); -static void ppp_push(struct ppp *ppp); +static int ppp_prepare_tx_skb(struct ppp *ppp, struct sk_buff **pskb); +static int ppp_push(struct ppp *ppp, struct sk_buff *skb); static void ppp_channel_push(struct channel *pch); static void ppp_receive_frame(struct ppp *ppp, struct sk_buff *skb, struct channel *pch); @@ -818,7 +818,9 @@ static long ppp_ioctl(struct file *file, unsigned int cmd, unsigned long arg) case PPPIOCSMRU: if (get_user(val, p)) break; + ppp_recv_lock(ppp); ppp->mru = val; + ppp_recv_unlock(ppp); err = 0; break; @@ -839,7 +841,9 @@ static long ppp_ioctl(struct file *file, unsigned int cmd, unsigned long arg) break; case PPPIOCGFLAGS: + ppp_lock(ppp); val = ppp->flags | ppp->xstate | ppp->rstate; + ppp_unlock(ppp); if (put_user(val, p)) break; err = 0; @@ -863,7 +867,7 @@ static long ppp_ioctl(struct file *file, unsigned int cmd, unsigned long arg) case PPPIOCSDEBUG: if (get_user(val, p)) break; - ppp->debug = val; + WRITE_ONCE(ppp->debug, val); err = 0; break; @@ -874,16 +878,16 @@ static long ppp_ioctl(struct file *file, unsigned int cmd, unsigned long arg) break; case PPPIOCGIDLE32: - idle32.xmit_idle = (jiffies - ppp->last_xmit) / HZ; - idle32.recv_idle = (jiffies - ppp->last_recv) / HZ; - if (copy_to_user(argp, &idle32, sizeof(idle32))) + idle32.xmit_idle = max(0L, (long)(jiffies - READ_ONCE(ppp->last_xmit))) / HZ; + idle32.recv_idle = max(0L, (long)(jiffies - READ_ONCE(ppp->last_recv))) / HZ; + if (copy_to_user(argp, &idle32, sizeof(idle32))) break; err = 0; break; case PPPIOCGIDLE64: - idle64.xmit_idle = (jiffies - ppp->last_xmit) / HZ; - idle64.recv_idle = (jiffies - ppp->last_recv) / HZ; + idle64.xmit_idle = max(0L, (long)(jiffies - READ_ONCE(ppp->last_xmit))) / HZ; + idle64.recv_idle = max(0L, (long)(jiffies - READ_ONCE(ppp->last_recv))) / HZ; if (copy_to_user(argp, &idle64, sizeof(idle64))) break; err = 0; @@ -924,7 +928,7 @@ static long ppp_ioctl(struct file *file, unsigned int cmd, unsigned long arg) if (copy_to_user(argp, &npi, sizeof(npi))) break; } else { - ppp->npmode[i] = npi.mode; + WRITE_ONCE(ppp->npmode[i], npi.mode); /* we may be able to transmit more packets now (??) */ netif_wake_queue(ppp->dev); } @@ -1462,7 +1466,7 @@ ppp_start_xmit(struct sk_buff *skb, struct net_device *dev) goto outf; /* Drop, accept or reject the packet */ - switch (ppp->npmode[npi]) { + switch (READ_ONCE(ppp->npmode[npi])) { case NPMODE_PASS: break; case NPMODE_QUEUE: @@ -1490,7 +1494,7 @@ ppp_start_xmit(struct sk_buff *skb, struct net_device *dev) outf: kfree_skb(skb); - ++dev->stats.tx_dropped; + DEV_STATS_INC(dev, tx_dropped); return NETDEV_TX_OK; } @@ -1540,11 +1544,11 @@ ppp_net_siocdevprivate(struct net_device *dev, struct ifreq *ifr, static void ppp_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats64) { - stats64->rx_errors = dev->stats.rx_errors; - stats64->tx_errors = dev->stats.tx_errors; - stats64->rx_dropped = dev->stats.rx_dropped; - stats64->tx_dropped = dev->stats.tx_dropped; - stats64->rx_length_errors = dev->stats.rx_length_errors; + stats64->rx_errors = DEV_STATS_READ(dev, rx_errors); + stats64->tx_errors = DEV_STATS_READ(dev, tx_errors); + stats64->rx_dropped = DEV_STATS_READ(dev, rx_dropped); + stats64->tx_dropped = DEV_STATS_READ(dev, tx_dropped); + stats64->rx_length_errors = DEV_STATS_READ(dev, rx_length_errors); dev_fetch_sw_netstats(stats64, dev->tstats); } @@ -1644,6 +1648,8 @@ static void ppp_setup(struct net_device *dev) dev->flags = IFF_POINTOPOINT | IFF_NOARP | IFF_MULTICAST; dev->priv_destructor = ppp_dev_priv_destructor; dev->pcpu_stat_type = NETDEV_PCPU_STAT_TSTATS; + dev->features = NETIF_F_SG | NETIF_F_FRAGLIST; + dev->hw_features = dev->features; netif_keep_dst(dev); } @@ -1652,26 +1658,44 @@ static void ppp_setup(struct net_device *dev) */ /* Called to do any work queued up on the transmit side that can now be done */ +static void ppp_xmit_flush(struct ppp *ppp) +{ + struct sk_buff *skb; + + while ((skb = skb_dequeue(&ppp->file.xq))) { + if (unlikely(!ppp_push(ppp, skb))) { + skb_queue_head(&ppp->file.xq, skb); + return; + } + } + /* If there's no work left to do, tell the core net code that we can + * accept some more. + */ + netif_wake_queue(ppp->dev); +} + static void __ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb) { ppp_xmit_lock(ppp); - if (!ppp->closing) { - ppp_push(ppp); - - if (skb) + if (unlikely(ppp->closing)) { + kfree_skb(skb); + goto out; + } + if (unlikely(ppp_prepare_tx_skb(ppp, &skb))) + goto out; + /* Fastpath: No backlog, just send the new skb. */ + if (likely(skb_queue_empty(&ppp->file.xq))) { + if (unlikely(!ppp_push(ppp, skb))) { skb_queue_tail(&ppp->file.xq, skb); - while (!ppp->xmit_pending && - (skb = skb_dequeue(&ppp->file.xq))) - ppp_send_frame(ppp, skb); - /* If there's no work left to do, tell the core net - code that we can accept some more. */ - if (!ppp->xmit_pending && !skb_peek(&ppp->file.xq)) - netif_wake_queue(ppp->dev); - else netif_stop_queue(ppp->dev); - } else { - kfree_skb(skb); + } + goto out; } + + /* Slowpath: Enqueue the new skb and process backlog */ + skb_queue_tail(&ppp->file.xq, skb); + ppp_xmit_flush(ppp); +out: ppp_xmit_unlock(ppp); } @@ -1713,6 +1737,10 @@ pad_compress_skb(struct ppp *ppp, struct sk_buff *skb) ppp->xcomp->comp_extra + ppp->dev->hard_header_len; int compressor_skb_size = ppp->dev->mtu + ppp->xcomp->comp_extra + PPP_HDRLEN; + + if (skb_linearize(skb)) + return NULL; + new_skb = alloc_skb(new_skb_size, GFP_ATOMIC); if (!new_skb) { if (net_ratelimit()) @@ -1754,13 +1782,15 @@ pad_compress_skb(struct ppp *ppp, struct sk_buff *skb) } /* - * Compress and send a frame. - * The caller should have locked the xmit path, - * and xmit_pending should be 0. + * Compress and prepare to send a frame. + * The caller should have locked the xmit path. + * Returns 1 if the skb was consumed, 0 if it can be passed to ppp_push(). + * @pskb is updated if a compressor is in use. */ -static void -ppp_send_frame(struct ppp *ppp, struct sk_buff *skb) +static int +ppp_prepare_tx_skb(struct ppp *ppp, struct sk_buff **pskb) { + struct sk_buff *skb = *pskb; int proto = PPP_PROTO(skb); struct sk_buff *new_skb; int len; @@ -1776,21 +1806,21 @@ ppp_send_frame(struct ppp *ppp, struct sk_buff *skb) *(__be16 *)skb_push(skb, 2) = htons(PPP_FILTER_OUTBOUND_TAG); if (ppp->pass_filter && bpf_prog_run(ppp->pass_filter, skb) == 0) { - if (ppp->debug & 1) + if (READ_ONCE(ppp->debug) & 1) netdev_printk(KERN_DEBUG, ppp->dev, "PPP: outbound frame " "not passed\n"); kfree_skb(skb); - return; + return 1; } /* if this packet passes the active filter, record the time */ if (!(ppp->active_filter && bpf_prog_run(ppp->active_filter, skb) == 0)) - ppp->last_xmit = jiffies; + WRITE_ONCE(ppp->last_xmit, jiffies); skb_pull(skb, 2); #else /* for data packets, record the time */ - ppp->last_xmit = jiffies; + WRITE_ONCE(ppp->last_xmit, jiffies); #endif /* CONFIG_PPP_FILTER */ } @@ -1800,6 +1830,10 @@ ppp_send_frame(struct ppp *ppp, struct sk_buff *skb) case PPP_IP: if (!ppp->vj || (ppp->flags & SC_COMP_TCP) == 0) break; + + if (skb_linearize(skb)) + goto drop; + /* try to do VJ TCP header compression */ new_skb = alloc_skb(skb->len + ppp->dev->hard_header_len - 2, GFP_ATOMIC); @@ -1825,6 +1859,7 @@ ppp_send_frame(struct ppp *ppp, struct sk_buff *skb) } consume_skb(skb); skb = new_skb; + *pskb = skb; cp = skb_put(skb, len + 2); cp[0] = 0; cp[1] = proto; @@ -1851,6 +1886,7 @@ ppp_send_frame(struct ppp *ppp, struct sk_buff *skb) if (!new_skb) goto drop; skb = new_skb; + *pskb = skb; } /* @@ -1862,67 +1898,69 @@ ppp_send_frame(struct ppp *ppp, struct sk_buff *skb) goto drop; skb_queue_tail(&ppp->file.rq, skb); wake_up_interruptible(&ppp->file.rwait); - return; + return 1; } - ppp->xmit_pending = skb; - ppp_push(ppp); - return; + return 0; drop: kfree_skb(skb); - ++ppp->dev->stats.tx_errors; + DEV_STATS_INC(ppp->dev, tx_errors); + return 1; } /* - * Try to send the frame in xmit_pending. + * Try to send the frame. * The caller should have the xmit path locked. + * Returns 1 if the skb was consumed, 0 if not. */ -static void -ppp_push(struct ppp *ppp) +static int +ppp_push(struct ppp *ppp, struct sk_buff *skb) { struct list_head *list; struct channel *pch; - struct sk_buff *skb = ppp->xmit_pending; - - if (!skb) - return; list = &ppp->channels; if (list_empty(list)) { /* nowhere to send the packet, just drop it */ - ppp->xmit_pending = NULL; kfree_skb(skb); - return; + return 1; } if ((ppp->flags & SC_MULTILINK) == 0) { + struct ppp_channel *chan; + int ret; /* not doing multilink: send it down the first channel */ list = list->next; pch = list_entry(list, struct channel, clist); spin_lock(&pch->downl); - if (pch->chan) { - if (pch->chan->ops->start_xmit(pch->chan, skb)) - ppp->xmit_pending = NULL; - } else { - /* channel got unregistered */ + chan = pch->chan; + if (unlikely(!chan || (!chan->direct_xmit && skb_linearize(skb)))) { + /* channel got unregistered, or it requires a linear + * skb but linearization failed + */ kfree_skb(skb); - ppp->xmit_pending = NULL; + ret = 1; + goto out; } + + ret = chan->ops->start_xmit(chan, skb); + +out: spin_unlock(&pch->downl); - return; + return ret; } #ifdef CONFIG_PPP_MULTILINK /* Multilink: fragment the packet over as many links as can take the packet at the moment. */ if (!ppp_mp_explode(ppp, skb)) - return; + return 0; #endif /* CONFIG_PPP_MULTILINK */ - ppp->xmit_pending = NULL; kfree_skb(skb); + return 1; } #ifdef CONFIG_PPP_MULTILINK @@ -1991,9 +2029,11 @@ static int ppp_mp_explode(struct ppp *ppp, struct sk_buff *skb) * performance if we have a lot of channels. */ if (nfree == 0 || nfree < navail / 2) - return 0; /* can't take now, leave it in xmit_pending */ + return 0; /* can't take now, leave it in transmit queue */ /* Do protocol field compression */ + if (skb_linearize(skb)) + goto err_linearize; p = skb->data; len = skb->len; if (*p == 0 && mp_protocol_compress) { @@ -2152,9 +2192,10 @@ static int ppp_mp_explode(struct ppp *ppp, struct sk_buff *skb) noskb: spin_unlock(&pch->downl); - if (ppp->debug & 1) + err_linearize: + if (READ_ONCE(ppp->debug) & 1) netdev_err(ppp->dev, "PPP: no memory (fragment)\n"); - ++ppp->dev->stats.tx_errors; + DEV_STATS_INC(ppp->dev, tx_errors); ++ppp->nxseq; return 1; /* abandon the frame */ } @@ -2182,8 +2223,12 @@ static void __ppp_channel_push(struct channel *pch, struct ppp *ppp) spin_unlock(&pch->downl); /* see if there is anything from the attached unit to be sent */ if (skb_queue_empty(&pch->file.xq)) { - if (ppp) - __ppp_xmit_process(ppp, NULL); + if (ppp) { + ppp_xmit_lock(ppp); + if (!ppp->closing) + ppp_xmit_flush(ppp); + ppp_xmit_unlock(ppp); + } } } @@ -2323,7 +2368,7 @@ ppp_input(struct ppp_channel *chan, struct sk_buff *skb) if (!ppp_decompress_proto(skb)) { kfree_skb(skb); if (ppp) { - ++ppp->dev->stats.rx_length_errors; + DEV_STATS_INC(ppp->dev, rx_length_errors); ppp_receive_error(ppp); } goto done; @@ -2396,7 +2441,7 @@ ppp_receive_frame(struct ppp *ppp, struct sk_buff *skb, struct channel *pch) static void ppp_receive_error(struct ppp *ppp) { - ++ppp->dev->stats.rx_errors; + DEV_STATS_INC(ppp->dev, rx_errors); if (ppp->vj) slhc_toss(ppp->vj); } @@ -2507,7 +2552,7 @@ ppp_receive_nonmp_frame(struct ppp *ppp, struct sk_buff *skb) *(__be16 *)skb_push(skb, 2) = htons(PPP_FILTER_INBOUND_TAG); if (ppp->pass_filter && bpf_prog_run(ppp->pass_filter, skb) == 0) { - if (ppp->debug & 1) + if (READ_ONCE(ppp->debug) & 1) netdev_printk(KERN_DEBUG, ppp->dev, "PPP: inbound frame " "not passed\n"); @@ -2516,14 +2561,14 @@ ppp_receive_nonmp_frame(struct ppp *ppp, struct sk_buff *skb) } if (!(ppp->active_filter && bpf_prog_run(ppp->active_filter, skb) == 0)) - ppp->last_recv = jiffies; + WRITE_ONCE(ppp->last_recv, jiffies); __skb_pull(skb, 2); } else #endif /* CONFIG_PPP_FILTER */ - ppp->last_recv = jiffies; + WRITE_ONCE(ppp->last_recv, jiffies); if ((ppp->dev->flags & IFF_UP) == 0 || - ppp->npmode[npi] != NPMODE_PASS) { + READ_ONCE(ppp->npmode[npi]) != NPMODE_PASS) { kfree_skb(skb); } else { /* chop off protocol */ @@ -2663,7 +2708,7 @@ ppp_receive_mp_frame(struct ppp *ppp, struct sk_buff *skb, struct channel *pch) */ if (seq_before(seq, ppp->nextseq)) { kfree_skb(skb); - ++ppp->dev->stats.rx_dropped; + DEV_STATS_INC(ppp->dev, rx_dropped); ppp_receive_error(ppp); return; } @@ -2699,7 +2744,7 @@ ppp_receive_mp_frame(struct ppp *ppp, struct sk_buff *skb, struct channel *pch) if (pskb_may_pull(skb, 2)) ppp_receive_nonmp_frame(ppp, skb); else { - ++ppp->dev->stats.rx_length_errors; + DEV_STATS_INC(ppp->dev, rx_length_errors); kfree_skb(skb); ppp_receive_error(ppp); } @@ -2776,7 +2821,7 @@ ppp_mp_reconstruct(struct ppp *ppp) seq = seq_before(minseq, PPP_MP_CB(p)->sequence)? minseq + 1: PPP_MP_CB(p)->sequence; - if (ppp->debug & 1) + if (READ_ONCE(ppp->debug) & 1) netdev_printk(KERN_DEBUG, ppp->dev, "lost frag %u..%u\n", oldseq, seq-1); @@ -2805,7 +2850,7 @@ ppp_mp_reconstruct(struct ppp *ppp) if (lost == 0 && (PPP_MP_CB(p)->BEbits & E) && (PPP_MP_CB(head)->BEbits & B)) { if (len > ppp->mrru + 2) { - ++ppp->dev->stats.rx_length_errors; + DEV_STATS_INC(ppp->dev, rx_length_errors); netdev_printk(KERN_DEBUG, ppp->dev, "PPP: reconstructed packet" " is too long (%d)\n", len); @@ -2825,7 +2870,7 @@ ppp_mp_reconstruct(struct ppp *ppp) struct sk_buff *tmp2; skb_queue_reverse_walk_from_safe(list, p, tmp2) { - if (ppp->debug & 1) + if (READ_ONCE(ppp->debug) & 1) netdev_printk(KERN_DEBUG, ppp->dev, "discarding frag %u\n", PPP_MP_CB(p)->sequence); @@ -2847,7 +2892,7 @@ ppp_mp_reconstruct(struct ppp *ppp) skb_queue_walk_safe(list, p, tmp) { if (p == head) break; - if (ppp->debug & 1) + if (READ_ONCE(ppp->debug) & 1) netdev_printk(KERN_DEBUG, ppp->dev, "discarding frag %u\n", PPP_MP_CB(p)->sequence); @@ -2855,12 +2900,12 @@ ppp_mp_reconstruct(struct ppp *ppp) kfree_skb(p); } - if (ppp->debug & 1) + if (READ_ONCE(ppp->debug) & 1) netdev_printk(KERN_DEBUG, ppp->dev, " missed pkts %u..%u\n", ppp->nextseq, PPP_MP_CB(head)->sequence-1); - ++ppp->dev->stats.rx_dropped; + DEV_STATS_INC(ppp->dev, rx_dropped); ppp_receive_error(ppp); } @@ -3168,7 +3213,8 @@ ppp_ccp_peek(struct ppp *ppp, struct sk_buff *skb, int inbound) if (!ppp->rc_state) break; if (ppp->rcomp->decomp_init(ppp->rc_state, dp, len, - ppp->file.index, 0, ppp->mru, ppp->debug)) { + ppp->file.index, 0, ppp->mru, + READ_ONCE(ppp->debug))) { ppp->rstate |= SC_DECOMP_RUN; ppp->rstate &= ~(SC_DC_ERROR | SC_DC_FERROR); } @@ -3177,7 +3223,8 @@ ppp_ccp_peek(struct ppp *ppp, struct sk_buff *skb, int inbound) if (!ppp->xc_state) break; if (ppp->xcomp->comp_init(ppp->xc_state, dp, len, - ppp->file.index, 0, ppp->debug)) + ppp->file.index, 0, + READ_ONCE(ppp->debug))) ppp->xstate |= SC_COMP_RUN; } break; @@ -3329,8 +3376,8 @@ ppp_get_stats(struct ppp *ppp, struct ppp_stats *st) st->p.ppp_opackets += tx_packets; st->p.ppp_obytes += tx_bytes; } - st->p.ppp_ierrors = ppp->dev->stats.rx_errors; - st->p.ppp_oerrors = ppp->dev->stats.tx_errors; + st->p.ppp_ierrors = DEV_STATS_READ(ppp->dev, rx_errors); + st->p.ppp_oerrors = DEV_STATS_READ(ppp->dev, tx_errors); if (!vj) return; st->vj.vjs_packets = vj->sls_o_compressed + vj->sls_o_uncompressed; @@ -3443,7 +3490,6 @@ static void ppp_destroy_interface(struct ppp *ppp) } #endif /* CONFIG_PPP_FILTER */ - kfree_skb(ppp->xmit_pending); free_percpu(ppp->xmit_recursion); free_netdev(ppp->dev); @@ -3570,6 +3616,18 @@ ppp_disconnect_channel(struct channel *pch) return err; } +/* Purge after the grace period: a late ppp_input() may still queue an + * skb on pch->file.rq before the last RCU reader drains. + */ +static void ppp_release_channel_free(struct rcu_head *rcu) +{ + struct channel *pch = container_of(rcu, struct channel, rcu); + + skb_queue_purge(&pch->file.xq); + skb_queue_purge(&pch->file.rq); + kfree(pch); +} + /* * Free up the resources used by a ppp channel. */ @@ -3585,9 +3643,7 @@ static void ppp_destroy_channel(struct channel *pch) pr_err("ppp: destroying undead channel %p !\n", pch); return; } - skb_queue_purge(&pch->file.xq); - skb_queue_purge(&pch->file.rq); - kfree(pch); + call_rcu(&pch->rcu, ppp_release_channel_free); } static void __exit ppp_cleanup(void) @@ -3600,6 +3656,7 @@ static void __exit ppp_cleanup(void) device_destroy(&ppp_class, MKDEV(PPP_MAJOR, 0)); class_unregister(&ppp_class); unregister_pernet_device(&ppp_net_ops); + rcu_barrier(); /* wait for RCU callbacks before module unload */ } /* diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c index dc6d13943238..7ee63051a161 100644 --- a/drivers/net/ppp/pppoe.c +++ b/drivers/net/ppp/pppoe.c @@ -903,6 +903,7 @@ static int pppoe_sendmsg(struct socket *sock, struct msghdr *m, dev_hard_header(skb, dev, ETH_P_PPP_SES, po->pppoe_pa.remote, NULL, total_len); + ph = pppoe_hdr(skb); memcpy(ph, &hdr, sizeof(struct pppoe_hdr)); ph->length = htons(total_len); diff --git a/drivers/net/slip/slip.c b/drivers/net/slip/slip.c index c889fb374703..bc84e2cc0502 100644 --- a/drivers/net/slip/slip.c +++ b/drivers/net/slip/slip.c @@ -693,6 +693,8 @@ static void slip_receive_buf(struct tty_struct *tty, const u8 *cp, const u8 *fp, if (!sl || sl->magic != SLIP_MAGIC || !netif_running(sl->dev)) return; + spin_lock_bh(&sl->lock); + /* Read the characters out of the buffer */ while (count--) { if (fp && *fp++) { @@ -708,6 +710,8 @@ static void slip_receive_buf(struct tty_struct *tty, const u8 *cp, const u8 *fp, #endif slip_unesc(sl, *cp++); } + + spin_unlock_bh(&sl->lock); } /************************************ diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c index 0572f6a9bdb6..679208d587f8 100644 --- a/drivers/net/vmxnet3/vmxnet3_drv.c +++ b/drivers/net/vmxnet3/vmxnet3_drv.c @@ -1530,7 +1530,11 @@ vmxnet3_get_hdr_len(struct vmxnet3_adapter *adapter, struct sk_buff *skb, struct ipv6hdr *ipv6; struct tcphdr *tcp; } hdr; - BUG_ON(gdesc->rcd.tcp == 0); + + /* v4/v6/tcp then describe the inner header, which we can't locate. */ + if ((le32_to_cpu(gdesc->dword[0]) & (1UL << VMXNET3_RCD_HDR_INNER_SHIFT)) || + gdesc->rcd.tcp == 0) + return 0; maplen = skb_headlen(skb); if (unlikely(sizeof(struct iphdr) + sizeof(struct tcphdr) > maplen)) @@ -1544,15 +1548,21 @@ vmxnet3_get_hdr_len(struct vmxnet3_adapter *adapter, struct sk_buff *skb, hdr.eth = eth_hdr(skb); if (gdesc->rcd.v4) { - BUG_ON(hdr.eth->h_proto != htons(ETH_P_IP) && - hdr.veth->h_vlan_encapsulated_proto != htons(ETH_P_IP)); + if (hdr.eth->h_proto != htons(ETH_P_IP) && + hdr.veth->h_vlan_encapsulated_proto != htons(ETH_P_IP)) + return 0; + hdr.ptr += hlen; - BUG_ON(hdr.ipv4->protocol != IPPROTO_TCP); + if (hdr.ipv4->protocol != IPPROTO_TCP) + return 0; + hlen = hdr.ipv4->ihl << 2; hdr.ptr += hdr.ipv4->ihl << 2; } else if (gdesc->rcd.v6) { - BUG_ON(hdr.eth->h_proto != htons(ETH_P_IPV6) && - hdr.veth->h_vlan_encapsulated_proto != htons(ETH_P_IPV6)); + if (hdr.eth->h_proto != htons(ETH_P_IPV6) && + hdr.veth->h_vlan_encapsulated_proto != htons(ETH_P_IPV6)) + return 0; + hdr.ptr += hlen; /* Use an estimated value, since we also need to handle * TSO case. diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index b706e1777354..cb7b01e492fa 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -4415,6 +4415,9 @@ static int vxlan_changelink(struct net_device *dev, struct nlattr *tb[], struct vxlan_rdst *dst; int err; + if (!rtnl_dev_link_net_capable(dev, vxlan->net)) + return -EPERM; + dst = &vxlan->default_dst; err = vxlan_nl2conf(tb, data, dev, &conf, true, extack); if (err) diff --git a/drivers/net/vxlan/vxlan_mdb.c b/drivers/net/vxlan/vxlan_mdb.c index 816ab1aa0526..4e02a669d045 100644 --- a/drivers/net/vxlan/vxlan_mdb.c +++ b/drivers/net/vxlan/vxlan_mdb.c @@ -42,6 +42,7 @@ struct vxlan_mdb_remote { }; #define VXLAN_SGRP_F_DELETE BIT(0) +#define VXLAN_SGRP_F_NEW BIT(1) struct vxlan_mdb_src_entry { struct hlist_node node; @@ -844,6 +845,7 @@ vxlan_mdb_remote_src_add(const struct vxlan_mdb_config *cfg, ent = vxlan_mdb_remote_src_entry_add(remote, &src->addr); if (!ent) return -ENOMEM; + ent->flags |= VXLAN_SGRP_F_NEW; } else if (!(cfg->nlflags & NLM_F_REPLACE)) { NL_SET_ERR_MSG_MOD(extack, "Source entry already exists"); return -EEXIST; @@ -853,15 +855,16 @@ vxlan_mdb_remote_src_add(const struct vxlan_mdb_config *cfg, if (err) goto err_src_del; - /* Clear flags in case source entry was marked for deletion as part of - * replace flow. + /* Clear the deletion mark so the entry survives the replace sweep. + * The new mark is retained until the whole operation succeeds. */ - ent->flags = 0; + ent->flags &= ~VXLAN_SGRP_F_DELETE; return 0; err_src_del: - vxlan_mdb_remote_src_entry_del(ent); + if (ent->flags & VXLAN_SGRP_F_NEW) + vxlan_mdb_remote_src_entry_del(ent); return err; } @@ -889,11 +892,19 @@ static int vxlan_mdb_remote_srcs_add(const struct vxlan_mdb_config *cfg, goto err_src_del; } + hlist_for_each_entry(ent, &remote->src_list, node) + ent->flags &= ~VXLAN_SGRP_F_NEW; + return 0; err_src_del: - hlist_for_each_entry_safe(ent, tmp, &remote->src_list, node) - vxlan_mdb_remote_src_del(cfg->vxlan, &cfg->group, remote, ent); + hlist_for_each_entry_safe(ent, tmp, &remote->src_list, node) { + if (ent->flags & VXLAN_SGRP_F_NEW) + vxlan_mdb_remote_src_del(cfg->vxlan, &cfg->group, remote, + ent); + else + ent->flags &= ~VXLAN_SGRP_F_DELETE; + } return err; } @@ -1069,7 +1080,7 @@ vxlan_mdb_remote_srcs_replace(const struct vxlan_mdb_config *cfg, err = vxlan_mdb_remote_srcs_add(cfg, remote, extack); if (err) - goto err_clear_delete; + return err; hlist_for_each_entry_safe(ent, tmp, &remote->src_list, node) { if (ent->flags & VXLAN_SGRP_F_DELETE) @@ -1078,11 +1089,6 @@ vxlan_mdb_remote_srcs_replace(const struct vxlan_mdb_config *cfg, } return 0; - -err_clear_delete: - hlist_for_each_entry(ent, &remote->src_list, node) - ent->flags &= ~VXLAN_SGRP_F_DELETE; - return err; } static int vxlan_mdb_remote_replace(const struct vxlan_mdb_config *cfg, diff --git a/drivers/net/wan/wanxl.c b/drivers/net/wan/wanxl.c index 5a9e262188ef..c38dd741401e 100644 --- a/drivers/net/wan/wanxl.c +++ b/drivers/net/wan/wanxl.c @@ -514,7 +514,8 @@ static void wanxl_pci_remove_one(struct pci_dev *pdev) if (card->irq) free_irq(card->irq, card); - wanxl_reset(card); + if (card->plx) + wanxl_reset(card); for (i = 0; i < RX_QUEUE_LENGTH; i++) if (card->rx_skbs[i]) { diff --git a/drivers/net/wireless/ath/ath11k/ahb.c b/drivers/net/wireless/ath/ath11k/ahb.c index 8dfe9b40c126..b8437e23e519 100644 --- a/drivers/net/wireless/ath/ath11k/ahb.c +++ b/drivers/net/wireless/ath/ath11k/ahb.c @@ -998,6 +998,7 @@ static int ath11k_ahb_fw_resources_init(struct ath11k_base *ab) ret = ath11k_ahb_setup_msa_resources(ab); if (ret) { ath11k_err(ab, "failed to setup msa resources\n"); + of_node_put(node); return ret; } diff --git a/drivers/net/wireless/ath/ath11k/dp_rx.c b/drivers/net/wireless/ath/ath11k/dp_rx.c index 5666f6647445..330446f279cd 100644 --- a/drivers/net/wireless/ath/ath11k/dp_rx.c +++ b/drivers/net/wireless/ath/ath11k/dp_rx.c @@ -4609,6 +4609,9 @@ static void ath11k_hal_rx_msdu_list_get(struct ath11k *ar, msdu_details = &msdu_link->msdu_link[0]; for (i = 0; i < HAL_RX_NUM_MSDU_DESC; i++) { + if (!i && FIELD_GET(BUFFER_ADDR_INFO0_ADDR, + msdu_details[i].buf_addr_info.info0) == 0) + break; if (FIELD_GET(BUFFER_ADDR_INFO0_ADDR, msdu_details[i].buf_addr_info.info0) == 0) { msdu_desc_info = &msdu_details[i - 1].rx_msdu_info; diff --git a/drivers/net/wireless/ath/ath11k/pci.c b/drivers/net/wireless/ath/ath11k/pci.c index 7114eca8810d..c7048219f3e0 100644 --- a/drivers/net/wireless/ath/ath11k/pci.c +++ b/drivers/net/wireless/ath/ath11k/pci.c @@ -199,6 +199,8 @@ static void ath11k_pci_soc_global_reset(struct ath11k_base *ab) val |= PCIE_SOC_GLOBAL_RESET_V; ath11k_pcic_write32(ab, PCIE_SOC_GLOBAL_RESET, val); + /* Flush the posted write to the device */ + ath11k_pcic_read32(ab, PCIE_SOC_GLOBAL_RESET); /* TODO: exact time to sleep is uncertain */ delay = 10; @@ -208,6 +210,8 @@ static void ath11k_pci_soc_global_reset(struct ath11k_base *ab) val &= ~PCIE_SOC_GLOBAL_RESET_V; ath11k_pcic_write32(ab, PCIE_SOC_GLOBAL_RESET, val); + /* Flush the posted write to the device */ + ath11k_pcic_read32(ab, PCIE_SOC_GLOBAL_RESET); mdelay(delay); diff --git a/drivers/net/wireless/ath/ath11k/qmi.c b/drivers/net/wireless/ath/ath11k/qmi.c index aea56c38bf8f..f23d75c8ad67 100644 --- a/drivers/net/wireless/ath/ath11k/qmi.c +++ b/drivers/net/wireless/ath/ath11k/qmi.c @@ -3295,9 +3295,14 @@ static void ath11k_qmi_driver_event_work(struct work_struct *work) clear_bit(ATH11K_FLAG_CRASH_FLUSH, &ab->dev_flags); clear_bit(ATH11K_FLAG_RECOVERY, &ab->dev_flags); - ath11k_core_qmi_firmware_ready(ab); - set_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags); - + if (!test_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags)) { + ret = ath11k_core_qmi_firmware_ready(ab); + if (ret) { + set_bit(ATH11K_FLAG_QMI_FAIL, &ab->dev_flags); + break; + } + set_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags); + } break; case ATH11K_QMI_EVENT_COLD_BOOT_CAL_DONE: break; diff --git a/drivers/net/wireless/ath/ath12k/pci.c b/drivers/net/wireless/ath/ath12k/pci.c index 60b8f7361b7f..37f41635d3b0 100644 --- a/drivers/net/wireless/ath/ath12k/pci.c +++ b/drivers/net/wireless/ath/ath12k/pci.c @@ -240,6 +240,8 @@ static void ath12k_pci_soc_global_reset(struct ath12k_base *ab) val |= PCIE_SOC_GLOBAL_RESET_V; ath12k_pci_write32(ab, PCIE_SOC_GLOBAL_RESET, val); + /* Flush the posted write to the device */ + ath12k_pci_read32(ab, PCIE_SOC_GLOBAL_RESET); /* TODO: exact time to sleep is uncertain */ delay = 10; @@ -249,6 +251,8 @@ static void ath12k_pci_soc_global_reset(struct ath12k_base *ab) val &= ~PCIE_SOC_GLOBAL_RESET_V; ath12k_pci_write32(ab, PCIE_SOC_GLOBAL_RESET, val); + /* Flush the posted write to the device */ + ath12k_pci_read32(ab, PCIE_SOC_GLOBAL_RESET); mdelay(delay); diff --git a/drivers/net/wireless/ath/ath6kl/txrx.c b/drivers/net/wireless/ath/ath6kl/txrx.c index c3b06b515c4f..5fd9cfdb5671 100644 --- a/drivers/net/wireless/ath/ath6kl/txrx.c +++ b/drivers/net/wireless/ath/ath6kl/txrx.c @@ -1723,13 +1723,15 @@ void aggr_recv_addba_req_evt(struct ath6kl_vif *vif, u8 tid_mux, u16 seq_no, rxtid = &aggr_conn->rx_tid[tid]; - if (win_sz < AGGR_WIN_SZ_MIN || win_sz > AGGR_WIN_SZ_MAX) - ath6kl_dbg(ATH6KL_DBG_WLAN_RX, "%s: win_sz %d, tid %d\n", - __func__, win_sz, tid); - if (rxtid->aggr) aggr_delete_tid_state(aggr_conn, tid); + if (win_sz < AGGR_WIN_SZ_MIN || win_sz > AGGR_WIN_SZ_MAX) { + ath6kl_dbg(ATH6KL_DBG_WLAN_RX, "%s: win_sz %d, tid %d\n", + __func__, win_sz, tid); + return; + } + rxtid->seq_next = seq_no; hold_q_size = TID_WINDOW_SZ(win_sz) * sizeof(struct skb_hold_q); rxtid->hold_q = kzalloc(hold_q_size, GFP_KERNEL); @@ -1828,7 +1830,7 @@ void aggr_reset_state(struct aggr_info_conn *aggr_conn) return; if (aggr_conn->timer_scheduled) { - timer_delete(&aggr_conn->timer); + timer_delete_sync(&aggr_conn->timer); aggr_conn->timer_scheduled = false; } diff --git a/drivers/net/wireless/ath/ath6kl/wmi.c b/drivers/net/wireless/ath/ath6kl/wmi.c index 08a154bce139..0cdcbc3c7796 100644 --- a/drivers/net/wireless/ath/ath6kl/wmi.c +++ b/drivers/net/wireless/ath/ath6kl/wmi.c @@ -484,6 +484,18 @@ static int ath6kl_wmi_tx_complete_event_rx(u8 *datap, int len) evt = (struct wmi_tx_complete_event *) datap; + if (len < sizeof(*evt)) { + ath6kl_dbg(ATH6KL_DBG_WMI, "tx complete: invalid len %d\n", + len); + return -EINVAL; + } + + if (len < sizeof(*evt) + evt->num_msg * sizeof(struct tx_complete_msg_v1)) { + ath6kl_dbg(ATH6KL_DBG_WMI, "tx complete: invalid len %d for %u msgs\n", + len, evt->num_msg); + return -EINVAL; + } + ath6kl_dbg(ATH6KL_DBG_WMI, "comp: %d %d %d\n", evt->num_msg, evt->msg_len, evt->msg_type); @@ -862,6 +874,14 @@ static int ath6kl_wmi_connect_event_rx(struct wmi *wmi, u8 *datap, int len, ev = (struct wmi_connect_event *) datap; + if (len < sizeof(*ev) + ev->beacon_ie_len + + ev->assoc_req_len + ev->assoc_resp_len) { + ath6kl_dbg(ATH6KL_DBG_WMI, + "connect event: IE lengths %u+%u+%u exceed buffer %d\n", + ev->beacon_ie_len, ev->assoc_req_len, + ev->assoc_resp_len, len); + return -EINVAL; + } if (vif->nw_type == AP_NETWORK) { /* AP mode start/STA connected event */ struct net_device *dev = vif->ndev; diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.c b/drivers/net/wireless/ath/ath9k/hif_usb.c index fe9abe8cd268..0a3d2190b9be 100644 --- a/drivers/net/wireless/ath/ath9k/hif_usb.c +++ b/drivers/net/wireless/ath/ath9k/hif_usb.c @@ -1225,15 +1225,10 @@ static int ath9k_hif_request_firmware(struct hif_device_usb *hif_dev, ret = request_firmware_nowait(THIS_MODULE, true, hif_dev->fw_name, &hif_dev->udev->dev, GFP_KERNEL, hif_dev, ath9k_hif_usb_firmware_cb); - if (ret) { + if (ret) dev_err(&hif_dev->udev->dev, "ath9k_htc: Async request for firmware %s failed\n", hif_dev->fw_name); - return ret; - } - - dev_info(&hif_dev->udev->dev, "ath9k_htc: Firmware %s requested\n", - hif_dev->fw_name); return ret; } diff --git a/drivers/net/wireless/ath/carl9170/rx.c b/drivers/net/wireless/ath/carl9170/rx.c index 6833430130f4..0383d5c9698b 100644 --- a/drivers/net/wireless/ath/carl9170/rx.c +++ b/drivers/net/wireless/ath/carl9170/rx.c @@ -150,7 +150,8 @@ static void carl9170_cmd_callback(struct ar9170 *ar, u32 len, void *buffer) spin_lock(&ar->cmd_lock); if (ar->readbuf) { if (len >= 4) - memcpy(ar->readbuf, buffer + 4, len - 4); + memcpy(ar->readbuf, buffer + 4, + min_t(u32, len - 4, ar->readlen)); ar->readbuf = NULL; } @@ -917,7 +918,9 @@ static void carl9170_rx_stream(struct ar9170 *ar, void *buf, unsigned int len) } } - skb_put_data(ar->rx_failover, tbuf, tlen); + skb_put_data(ar->rx_failover, tbuf, + min_t(unsigned int, tlen, + ar->rx_failover_missing)); ar->rx_failover_missing -= tlen; if (ar->rx_failover_missing <= 0) { diff --git a/drivers/net/wireless/ath/carl9170/tx.c b/drivers/net/wireless/ath/carl9170/tx.c index b7717f9e1e9b..b0eee8f892af 100644 --- a/drivers/net/wireless/ath/carl9170/tx.c +++ b/drivers/net/wireless/ath/carl9170/tx.c @@ -692,7 +692,7 @@ void carl9170_tx_process_status(struct ar9170 *ar, unsigned int i; for (i = 0; i < cmd->hdr.ext; i++) { - if (WARN_ON(i > ((cmd->hdr.len / 2) + 1))) { + if (WARN_ON(i >= (cmd->hdr.len / 2))) { print_hex_dump_bytes("UU:", DUMP_PREFIX_NONE, (void *) cmd, cmd->hdr.len + 4); break; diff --git a/drivers/net/wireless/atmel/at76c50x-usb.c b/drivers/net/wireless/atmel/at76c50x-usb.c index aa683eacaf38..b983b51ffee7 100644 --- a/drivers/net/wireless/atmel/at76c50x-usb.c +++ b/drivers/net/wireless/atmel/at76c50x-usb.c @@ -1521,13 +1521,16 @@ static inline int at76_guess_freq(struct at76_priv *priv) if (ieee80211_is_probe_resp(hdr->frame_control)) { el_off = offsetof(struct ieee80211_mgmt, u.probe_resp.variable); - el = ((struct ieee80211_mgmt *)hdr)->u.probe_resp.variable; } else if (ieee80211_is_beacon(hdr->frame_control)) { el_off = offsetof(struct ieee80211_mgmt, u.beacon.variable); - el = ((struct ieee80211_mgmt *)hdr)->u.beacon.variable; } else { goto exit; } + + if (len < el_off) + goto exit; + + el = priv->rx_skb->data + el_off; len -= el_off; el = cfg80211_find_ie(WLAN_EID_DS_PARAMS, el, len); diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c index 13952dfeb3e3..0beb38047df5 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c @@ -911,6 +911,7 @@ int brcmf_sdiod_probe(struct brcmf_sdio_dev *sdiodev) return ret; } switch (sdiodev->func2->device) { + case SDIO_DEVICE_ID_BROADCOM_43752: case SDIO_DEVICE_ID_BROADCOM_CYPRESS_4373: f2_blksz = SDIO_4373_FUNC2_BLOCKSIZE; break; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c index bb96b87b2a6e..1fc9ecd86d5d 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c @@ -2146,7 +2146,7 @@ brcmf_set_key_mgmt(struct net_device *ndev, struct cfg80211_connect_params *sme) sme->crypto.akm_suites[0]); return -EINVAL; } - } else if (val & (WPA2_AUTH_PSK | WPA2_AUTH_UNSPECIFIED)) { + } else if (val & (WPA2_AUTH_PSK | WPA2_AUTH_UNSPECIFIED | WPA2_AUTH_1X_SHA256)) { switch (sme->crypto.akm_suites[0]) { case WLAN_AKM_SUITE_8021X: val = WPA2_AUTH_UNSPECIFIED; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c index 6327f4eca500..fe4f23decc70 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c @@ -1383,16 +1383,20 @@ static int brcmf_pcie_init_ringbuffers(struct brcmf_pciedev_info *devinfo) static void brcmf_pcie_release_scratchbuffers(struct brcmf_pciedev_info *devinfo) { - if (devinfo->shared.scratch) + if (devinfo->shared.scratch) { dma_free_coherent(&devinfo->pdev->dev, BRCMF_DMA_D2H_SCRATCH_BUF_LEN, devinfo->shared.scratch, devinfo->shared.scratch_dmahandle); - if (devinfo->shared.ringupd) + devinfo->shared.scratch = NULL; + } + if (devinfo->shared.ringupd) { dma_free_coherent(&devinfo->pdev->dev, BRCMF_DMA_D2H_RINGUPD_BUF_LEN, devinfo->shared.ringupd, devinfo->shared.ringupd_dmahandle); + devinfo->shared.ringupd = NULL; + } } static int brcmf_pcie_init_scratchbuffers(struct brcmf_pciedev_info *devinfo) diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c index a0e88dbaaebe..9ab62a2667c7 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c @@ -4465,6 +4465,7 @@ int brcmf_sdio_probe(struct brcmf_sdio_dev *sdiodev) bus->sdiodev = sdiodev; sdiodev->bus = bus; skb_queue_head_init(&bus->glom); + INIT_WORK(&bus->datawork, brcmf_sdio_dataworker); bus->txbound = BRCMF_TXBOUND; bus->rxbound = BRCMF_RXBOUND; bus->txminmax = BRCMF_TXMINMAX; @@ -4479,7 +4480,6 @@ int brcmf_sdio_probe(struct brcmf_sdio_dev *sdiodev) goto fail; } brcmf_sdiod_freezer_count(sdiodev); - INIT_WORK(&bus->datawork, brcmf_sdio_dataworker); bus->brcmf_wq = wq; /* attempt to attach to the dongle */ diff --git a/drivers/net/wireless/intel/ipw2x00/ipw2100.c b/drivers/net/wireless/intel/ipw2x00/ipw2100.c index 215814861cbd..0808e69c3c17 100644 --- a/drivers/net/wireless/intel/ipw2x00/ipw2100.c +++ b/drivers/net/wireless/intel/ipw2x00/ipw2100.c @@ -6162,6 +6162,8 @@ static int ipw2100_pci_init_one(struct pci_dev *pci_dev, if (err) { printk(KERN_WARNING DRV_NAME "Error calling pci_enable_device.\n"); + free_libipw(dev, 0); + pci_iounmap(pci_dev, ioaddr); return err; } @@ -6174,16 +6176,14 @@ static int ipw2100_pci_init_one(struct pci_dev *pci_dev, if (err) { printk(KERN_WARNING DRV_NAME "Error calling pci_set_dma_mask.\n"); - pci_disable_device(pci_dev); - return err; + goto fail; } err = pci_request_regions(pci_dev, DRV_NAME); if (err) { printk(KERN_WARNING DRV_NAME "Error calling pci_request_regions.\n"); - pci_disable_device(pci_dev); - return err; + goto fail; } /* We disable the RETRY_TIMEOUT register (0x41) to keep diff --git a/drivers/net/wireless/intel/iwlwifi/fw/pnvm.c b/drivers/net/wireless/intel/iwlwifi/fw/pnvm.c index f297e82d63d2..5de3a9c0b3ef 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/pnvm.c +++ b/drivers/net/wireless/intel/iwlwifi/fw/pnvm.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* - * Copyright(c) 2020-2025 Intel Corporation + * Copyright(c) 2020-2026 Intel Corporation */ #include "iwl-drv.h" @@ -12,6 +12,7 @@ #include "fw/api/alive.h" #include "fw/uefi.h" #include "fw/img.h" +#include "fw/dbg.h" #define IWL_PNVM_REDUCED_CAP_BIT BIT(25) @@ -26,6 +27,12 @@ static bool iwl_pnvm_complete_fn(struct iwl_notif_wait_data *notif_wait, struct iwl_trans *trans = (struct iwl_trans *)data; struct iwl_pnvm_init_complete_ntfy *pnvm_ntf = (void *)pkt->data; + if (IWL_FW_CHECK(trans, + iwl_rx_packet_payload_len(pkt) < sizeof(*pnvm_ntf), + "Bad notif len: %d\n", + iwl_rx_packet_payload_len(pkt))) + return true; + IWL_DEBUG_FW(trans, "PNVM complete notification received with status 0x%0x\n", le32_to_cpu(pnvm_ntf->status)); diff --git a/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c b/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c index e1f28b053253..818eb1c4b158 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c +++ b/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c @@ -483,7 +483,7 @@ bool iwl_add_mcc_to_tas_block_list(u16 *list, u8 *size, u16 mcc) if (*size >= IWL_WTAS_BLACK_LIST_MAX) return false; - list[*size++] = mcc; + list[(*size)++] = mcc; return true; } IWL_EXPORT_SYMBOL(iwl_add_mcc_to_tas_block_list); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c index 11afe373961f..46e95e25dcd4 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c @@ -2783,7 +2783,7 @@ static int iwl_mvm_wowlan_store_wake_pkt(struct iwl_mvm *mvm, struct iwl_wowlan_status_data *status, u32 len) { - u32 data_size, packet_len = le32_to_cpu(notif->wake_packet_length); + u32 data_size, packet_len; if (len < sizeof(*notif)) { IWL_ERR(mvm, "Invalid WoWLAN wake packet notification!\n"); @@ -2802,6 +2802,7 @@ static int iwl_mvm_wowlan_store_wake_pkt(struct iwl_mvm *mvm, return -EIO; } + packet_len = le32_to_cpu(notif->wake_packet_length); data_size = len - offsetof(struct iwl_wowlan_wake_pkt_notif, wake_packet); /* data_size got the padding from the notification, remove it. */ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c index aa517978fc7a..cfca5a3ee78e 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c @@ -955,12 +955,22 @@ int iwl_mvm_get_sar_geo_profile(struct iwl_mvm *mvm) return ret; } + if (IWL_FW_CHECK(mvm, + iwl_rx_packet_payload_len(cmd.resp_pkt) != + sizeof(*resp), + "Wrong size for iwl_geo_tx_power_profiles_resp: %d\n", + iwl_rx_packet_payload_len(cmd.resp_pkt))) { + ret = -EIO; + goto out; + } + resp = (void *)cmd.resp_pkt->data; ret = le32_to_cpu(resp->profile_idx); if (WARN_ON(ret > BIOS_GEO_MAX_PROFILE_NUM)) ret = -EIO; +out: iwl_free_resp(&cmd); return ret; } diff --git a/drivers/net/wireless/intersil/p54/txrx.c b/drivers/net/wireless/intersil/p54/txrx.c index 1294a1d6528e..9f491334c8d0 100644 --- a/drivers/net/wireless/intersil/p54/txrx.c +++ b/drivers/net/wireless/intersil/p54/txrx.c @@ -499,11 +499,19 @@ static void p54_rx_eeprom_readback(struct p54_common *priv, if (le16_to_cpu(eeprom->v2.len) != priv->eeprom_slice_size) return; + if (eeprom->v2.data + priv->eeprom_slice_size > + skb_tail_pointer(skb)) + return; + memcpy(priv->eeprom, eeprom->v2.data, priv->eeprom_slice_size); } else { if (le16_to_cpu(eeprom->v1.len) != priv->eeprom_slice_size) return; + if (eeprom->v1.data + priv->eeprom_slice_size > + skb_tail_pointer(skb)) + return; + memcpy(priv->eeprom, eeprom->v1.data, priv->eeprom_slice_size); } diff --git a/drivers/net/wireless/marvell/libertas/firmware.c b/drivers/net/wireless/marvell/libertas/firmware.c index f124110944b7..9bf7d4c207b9 100644 --- a/drivers/net/wireless/marvell/libertas/firmware.c +++ b/drivers/net/wireless/marvell/libertas/firmware.c @@ -78,6 +78,7 @@ static void helper_firmware_cb(const struct firmware *firmware, void *context) } else { /* No main firmware needed for this helper --> success! */ lbs_fw_loaded(priv, 0, firmware, NULL); + release_firmware(firmware); } } diff --git a/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c b/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c index ef6722ffdc74..358de94eeb5e 100644 --- a/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c +++ b/drivers/net/wireless/marvell/mwifiex/sta_ioctl.c @@ -196,6 +196,7 @@ static int mwifiex_request_rgpower_table(struct mwifiex_private *priv) struct mwifiex_adapter *adapter = priv->adapter; char rgpower_table_name[30]; char country_code[3]; + int ret; strscpy(country_code, domain_info->country_code, sizeof(country_code)); @@ -214,16 +215,17 @@ static int mwifiex_request_rgpower_table(struct mwifiex_private *priv) adapter->rgpower_data = NULL; } - if ((request_firmware(&adapter->rgpower_data, rgpower_table_name, - adapter->dev))) { + ret = request_firmware_direct(&adapter->rgpower_data, rgpower_table_name, + adapter->dev); + + if (ret) { mwifiex_dbg( adapter, INFO, - "info: %s: failed to request regulatory power table\n", - __func__); - return -EIO; + "info: %s: failed to request regulatory power table: %d\n", + __func__, ret); } - return 0; + return ret; } static int mwifiex_dnld_rgpower_table(struct mwifiex_private *priv) diff --git a/drivers/net/wireless/marvell/mwifiex/tdls.c b/drivers/net/wireless/marvell/mwifiex/tdls.c index 77a9a6de636d..b94ca13086d6 100644 --- a/drivers/net/wireless/marvell/mwifiex/tdls.c +++ b/drivers/net/wireless/marvell/mwifiex/tdls.c @@ -215,7 +215,7 @@ mwifiex_tdls_add_ht_oper(struct mwifiex_private *priv, const u8 *mac, /* follow AP's channel bandwidth */ if (ISSUPP_CHANWIDTH40(priv->adapter->hw_dot_11n_dev_cap) && - bss_desc->bcn_ht_cap && + bss_desc->bcn_ht_oper && ISALLOWED_CHANWIDTH40(bss_desc->bcn_ht_oper->ht_param)) ht_oper->ht_param = bss_desc->bcn_ht_oper->ht_param; diff --git a/drivers/net/wireless/marvell/mwifiex/uap_event.c b/drivers/net/wireless/marvell/mwifiex/uap_event.c index 245cb99a3daa..fc9da9006a78 100644 --- a/drivers/net/wireless/marvell/mwifiex/uap_event.c +++ b/drivers/net/wireless/marvell/mwifiex/uap_event.c @@ -123,11 +123,31 @@ int mwifiex_process_uap_event(struct mwifiex_private *priv) len = ETH_ALEN; if (len != -1) { + u16 evt_len = le16_to_cpu(event->len); + sinfo->assoc_req_ies = &event->data[len]; len = (u8 *)sinfo->assoc_req_ies - (u8 *)&event->frame_control; - sinfo->assoc_req_ies_len = - le16_to_cpu(event->len) - (u16)len; + + /* + * event->len is reported by the device firmware + * and is not otherwise validated. Reject a + * length that underflows the header, or that + * would place the association request IEs + * outside the fixed-size event_body[] buffer the + * event was copied into; otherwise the IE walk + * in mwifiex_set_sta_ht_cap() reads past + * event_body and out of the adapter slab object. + */ + if (evt_len < len || + (u8 *)&event->frame_control + evt_len > + adapter->event_body + MAX_EVENT_SIZE) { + mwifiex_dbg(adapter, ERROR, + "invalid STA assoc event length\n"); + kfree(sinfo); + return -1; + } + sinfo->assoc_req_ies_len = evt_len - (u16)len; } } cfg80211_new_sta(priv->netdev, event->sta_addr, sinfo, diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c index 13c9dbeaee84..6c7debe439b6 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c @@ -1601,6 +1601,8 @@ bool mt7615_rx_check(struct mt76_dev *mdev, void *data, int len) switch (type) { case PKT_TYPE_TXRX_NOTIFY: + if (!mt76_is_mmio(mdev)) + return false; mt7615_mac_tx_free(dev, data, len); return false; case PKT_TYPE_TXS: @@ -1634,6 +1636,10 @@ void mt7615_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, dev_kfree_skb(skb); break; case PKT_TYPE_TXRX_NOTIFY: + if (!mt76_is_mmio(mdev)) { + dev_kfree_skb(skb); + break; + } mt7615_mac_tx_free(dev, skb->data, skb->len); dev_kfree_skb(skb); break; diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c index fc3e6728fcfb..2aa7b711c774 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c @@ -1441,6 +1441,8 @@ mt76_connac_mcu_uni_bss_he_tlv(struct mt76_phy *phy, struct ieee80211_vif *vif, struct bss_info_uni_he *he; cap = mt76_connac_get_he_phy_cap(phy, vif); + if (!cap) + return; he = (struct bss_info_uni_he *)tlv; he->he_pe_duration = vif->bss_conf.htc_trig_based_pkt_ext; diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c index 79e021ac0bdb..f60e2eaea2af 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c @@ -595,6 +595,8 @@ mt7915_mcu_bss_he_tlv(struct sk_buff *skb, struct ieee80211_vif *vif, struct tlv *tlv; cap = mt76_connac_get_he_phy_cap(phy->mt76, vif); + if (!cap) + return; tlv = mt76_connac_mcu_add_tlv(skb, BSS_INFO_HE_BASIC, sizeof(*he)); @@ -1177,13 +1179,12 @@ mt7915_mcu_sta_bfer_vht(struct ieee80211_sta *sta, struct mt7915_phy *phy, } static void -mt7915_mcu_sta_bfer_he(struct ieee80211_sta *sta, struct ieee80211_vif *vif, - struct mt7915_phy *phy, struct sta_rec_bf *bf) +mt7915_mcu_sta_bfer_he(struct ieee80211_sta *sta, + const struct ieee80211_sta_he_cap *vc, + struct sta_rec_bf *bf) { struct ieee80211_sta_he_cap *pc = &sta->deflink.he_cap; struct ieee80211_he_cap_elem *pe = &pc->he_cap_elem; - const struct ieee80211_sta_he_cap *vc = - mt76_connac_get_he_phy_cap(phy->mt76, vif); const struct ieee80211_he_cap_elem *ve = &vc->he_cap_elem; u16 mcs_map = le16_to_cpu(pc->he_mcs_nss_supp.rx_mcs_80); u8 nss_mcs = mt7915_mcu_get_sta_nss(mcs_map); @@ -1242,6 +1243,7 @@ mt7915_mcu_sta_bfer_tlv(struct mt7915_dev *dev, struct sk_buff *skb, { struct mt7915_vif *mvif = (struct mt7915_vif *)vif->drv_priv; struct mt7915_phy *phy = mvif->phy; + const struct ieee80211_sta_he_cap *vc = NULL; int tx_ant = hweight8(phy->mt76->chainmask) - 1; struct sta_rec_bf *bf; struct tlv *tlv; @@ -1260,6 +1262,12 @@ mt7915_mcu_sta_bfer_tlv(struct mt7915_dev *dev, struct sk_buff *skb, if (!ebf && !dev->ibf) return; + if (sta->deflink.he_cap.has_he && ebf) { + vc = mt76_connac_get_he_phy_cap(phy->mt76, vif); + if (!vc) + return; + } + tlv = mt76_connac_mcu_add_tlv(skb, STA_REC_BF, sizeof(*bf)); bf = (struct sta_rec_bf *)tlv; @@ -1268,7 +1276,7 @@ mt7915_mcu_sta_bfer_tlv(struct mt7915_dev *dev, struct sk_buff *skb, * ht: iBF only, since mac80211 lacks of eBF support */ if (sta->deflink.he_cap.has_he && ebf) - mt7915_mcu_sta_bfer_he(sta, vif, phy, bf); + mt7915_mcu_sta_bfer_he(sta, vc, bf); else if (sta->deflink.vht_cap.vht_supported) mt7915_mcu_sta_bfer_vht(sta, phy, bf, ebf); else if (sta->deflink.ht_cap.ht_supported) diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c index d2731e9953bf..251a9a1f664d 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c @@ -569,8 +569,9 @@ bool mt7921_rx_check(struct mt76_dev *mdev, void *data, int len) switch (type) { case PKT_TYPE_TXRX_NOTIFY: - /* PKT_TYPE_TXRX_NOTIFY can be received only by mmio devices */ - mt7921_mac_tx_free(dev, data, len); /* mmio */ + if (!mt76_is_mmio(mdev)) + return false; + mt7921_mac_tx_free(dev, data, len); return false; case PKT_TYPE_TXS: for (rxd += 2; rxd + 8 <= end; rxd += 8) @@ -599,7 +600,10 @@ void mt7921_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, switch (type) { case PKT_TYPE_TXRX_NOTIFY: - /* PKT_TYPE_TXRX_NOTIFY can be received only by mmio devices */ + if (!mt76_is_mmio(mdev)) { + napi_consume_skb(skb, 1); + break; + } mt7921_mac_tx_free(dev, skb->data, skb->len); napi_consume_skb(skb, 1); break; diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c index d951a46e9d48..6a7477943091 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c @@ -1193,8 +1193,9 @@ bool mt7925_rx_check(struct mt76_dev *mdev, void *data, int len) switch (type) { case PKT_TYPE_TXRX_NOTIFY: - /* PKT_TYPE_TXRX_NOTIFY can be received only by mmio devices */ - mt7925_mac_tx_free(dev, data, len); /* mmio */ + if (!mt76_is_mmio(mdev)) + return false; + mt7925_mac_tx_free(dev, data, len); return false; case PKT_TYPE_TXS: for (rxd += 4; rxd + 12 <= end; rxd += 12) @@ -1230,7 +1231,10 @@ void mt7925_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q, switch (type) { case PKT_TYPE_TXRX_NOTIFY: - /* PKT_TYPE_TXRX_NOTIFY can be received only by mmio devices */ + if (!mt76_is_mmio(mdev)) { + napi_consume_skb(skb, 1); + break; + } mt7925_mac_tx_free(dev, skb->data, skb->len); napi_consume_skb(skb, 1); break; @@ -1274,6 +1278,9 @@ mt7925_vif_connect_iter(void *priv, u8 *mac, for_each_set_bit(i, &valid, IEEE80211_MLD_MAX_NUM_LINKS) { bss_conf = mt792x_vif_to_bss_conf(vif, i); + if (!bss_conf) + continue; + mconf = mt792x_vif_to_link(mvif, i); mt76_connac_mcu_uni_add_dev(&dev->mphy, bss_conf, &mconf->mt76, diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c index 6c0dc72efc39..910713e62e14 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c @@ -1573,6 +1573,9 @@ static void mt7925_sta_set_decap_offload(struct ieee80211_hw *hw, mlink = mt792x_sta_to_link(msta, i); + if (!mlink) + continue; + if (enabled) set_bit(MT_WCID_FLAG_HDR_TRANS, &mlink->wcid.flags); else diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c index d887aa9a3dff..647efa963db4 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c @@ -2706,6 +2706,8 @@ mt7925_mcu_bss_he_tlv(struct sk_buff *skb, struct ieee80211_bss_conf *link_conf, struct tlv *tlv; cap = mt76_connac_get_he_phy_cap(phy->mt76, link_conf->vif); + if (!cap) + return; tlv = mt76_connac_mcu_add_tlv(skb, UNI_BSS_INFO_HE_BASIC, sizeof(*he)); diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c index a3f813be107d..72851399575f 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c +++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c @@ -860,6 +860,8 @@ mt7996_mcu_bss_he_tlv(struct sk_buff *skb, struct ieee80211_vif *vif, struct tlv *tlv; cap = mt76_connac_get_he_phy_cap(phy->mt76, vif); + if (!cap) + return; tlv = mt7996_mcu_add_uni_tlv(skb, UNI_BSS_INFO_HE_BASIC, sizeof(*he)); @@ -1659,17 +1661,18 @@ mt7996_mcu_sta_bfer_he(struct ieee80211_link_sta *link_sta, { struct ieee80211_sta_he_cap *pc = &link_sta->he_cap; struct ieee80211_he_cap_elem *pe = &pc->he_cap_elem; - const struct ieee80211_sta_he_cap *vc = - mt76_connac_get_he_phy_cap(phy->mt76, vif); - const struct ieee80211_he_cap_elem *ve = &vc->he_cap_elem; u16 mcs_map = le16_to_cpu(pc->he_mcs_nss_supp.rx_mcs_80); u8 nss_mcs = mt7996_mcu_get_sta_nss(mcs_map); + const struct ieee80211_he_cap_elem *ve; + const struct ieee80211_sta_he_cap *vc; u8 snd_dim, sts; + vc = mt76_connac_get_he_phy_cap(phy->mt76, vif); if (!vc) return; bf->tx_mode = MT_PHY_TYPE_HE_SU; + ve = &vc->he_cap_elem; mt7996_mcu_sta_sounding_rate(bf, phy); @@ -1725,14 +1728,18 @@ mt7996_mcu_sta_bfer_eht(struct ieee80211_link_sta *link_sta, struct ieee80211_sta_eht_cap *pc = &link_sta->eht_cap; struct ieee80211_eht_cap_elem_fixed *pe = &pc->eht_cap_elem; struct ieee80211_eht_mcs_nss_supp *eht_nss = &pc->eht_mcs_nss_supp; - const struct ieee80211_sta_eht_cap *vc = - mt76_connac_get_eht_phy_cap(phy->mt76, vif); - const struct ieee80211_eht_cap_elem_fixed *ve = &vc->eht_cap_elem; u8 nss_mcs = u8_get_bits(eht_nss->bw._80.rx_tx_mcs9_max_nss, IEEE80211_EHT_MCS_NSS_RX) - 1; + const struct ieee80211_eht_cap_elem_fixed *ve; + const struct ieee80211_sta_eht_cap *vc; u8 snd_dim, sts; + vc = mt76_connac_get_eht_phy_cap(phy->mt76, vif); + if (!vc) + return; + bf->tx_mode = MT_PHY_TYPE_EHT_MU; + ve = &vc->eht_cap_elem; mt7996_mcu_sta_sounding_rate(bf, phy); diff --git a/drivers/net/wireless/microchip/wilc1000/hif.c b/drivers/net/wireless/microchip/wilc1000/hif.c index d795ca05729a..f08426ca5113 100644 --- a/drivers/net/wireless/microchip/wilc1000/hif.c +++ b/drivers/net/wireless/microchip/wilc1000/hif.c @@ -600,6 +600,11 @@ static s32 wilc_parse_assoc_resp_info(u8 *buffer, u32 buffer_len, u16 ies_len; struct wilc_assoc_resp *res = (struct wilc_assoc_resp *)buffer; + if (buffer_len < sizeof(*res)) { + ret_conn_info->status = WLAN_STATUS_UNSPECIFIED_FAILURE; + return -EINVAL; + } + ret_conn_info->status = le16_to_cpu(res->status_code); if (ret_conn_info->status == WLAN_STATUS_SUCCESS) { ies = &buffer[sizeof(*res)]; diff --git a/drivers/net/wireless/virtual/mac80211_hwsim.c b/drivers/net/wireless/virtual/mac80211_hwsim.c index 20815fdc9d37..a0724e1c5307 100644 --- a/drivers/net/wireless/virtual/mac80211_hwsim.c +++ b/drivers/net/wireless/virtual/mac80211_hwsim.c @@ -6898,6 +6898,7 @@ static void hwsim_virtio_rx_work(struct work_struct *work) skb->data = skb->head; skb_reset_tail_pointer(skb); + len = min(len, skb_end_offset(skb)); skb_put(skb, len); hwsim_virtio_handle_cmd(skb); diff --git a/drivers/pinctrl/renesas/pinctrl-rzg2l.c b/drivers/pinctrl/renesas/pinctrl-rzg2l.c index aaa783c79e6d..e8cc83029db3 100644 --- a/drivers/pinctrl/renesas/pinctrl-rzg2l.c +++ b/drivers/pinctrl/renesas/pinctrl-rzg2l.c @@ -1097,7 +1097,7 @@ static int rzg2l_read_oen(struct rzg2l_pinctrl *pctrl, unsigned int _pin) int bit; if (!pctrl->data->pin_to_oen_bit) - return -EOPNOTSUPP; + return -ENOTSUPP; bit = pctrl->data->pin_to_oen_bit(pctrl, _pin); if (bit < 0) @@ -1115,7 +1115,7 @@ static int rzg2l_write_oen(struct rzg2l_pinctrl *pctrl, unsigned int _pin, u8 oe int bit; if (!pctrl->data->pin_to_oen_bit) - return -EOPNOTSUPP; + return -ENOTSUPP; bit = pctrl->data->pin_to_oen_bit(pctrl, _pin); if (bit < 0) @@ -1552,7 +1552,7 @@ static int rzg2l_pinctrl_pinconf_set(struct pinctrl_dev *pctldev, break; default: - return -EOPNOTSUPP; + return -ENOTSUPP; } } @@ -1634,7 +1634,7 @@ static int rzg2l_pinctrl_pinconf_group_get(struct pinctrl_dev *pctldev, /* Check config matching between to pin */ if (i && prev_config != *config) - return -EOPNOTSUPP; + return -ENOTSUPP; prev_config = *config; } diff --git a/drivers/platform/loongarch/loongson-laptop.c b/drivers/platform/loongarch/loongson-laptop.c index 61b18ac206c9..dc8142f4fa95 100644 --- a/drivers/platform/loongarch/loongson-laptop.c +++ b/drivers/platform/loongarch/loongson-laptop.c @@ -189,6 +189,7 @@ static int __init setup_acpi_notify(struct generic_sub_driver *sub_driver) static int loongson_hotkey_suspend(struct device *dev) { + bl_powered = false; return 0; } diff --git a/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c b/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c index 65897fae17df..6091b617480f 100644 --- a/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c +++ b/drivers/platform/x86/intel/uncore-frequency/uncore-frequency-common.c @@ -274,15 +274,20 @@ int uncore_freq_add_entry(struct uncore_data *data, int cpu) sprintf(data->name, "package_%02d_die_%02d", data->package_id, data->die_id); } + /* + * Set the control CPU before any read path so entry recreation after CPU + * hotplug can populate read-only attributes from the new online CPU. + */ + data->control_cpu = cpu; uncore_read(data, &data->initial_min_freq_khz, UNCORE_INDEX_MIN_FREQ); uncore_read(data, &data->initial_max_freq_khz, UNCORE_INDEX_MAX_FREQ); ret = create_attr_group(data, data->name); if (ret) { + data->control_cpu = -1; if (data->domain_id != UNCORE_DOMAIN_ID_INVALID) ida_free(&intel_uncore_ida, data->instance_id); } else { - data->control_cpu = cpu; data->valid = true; } diff --git a/drivers/pmdomain/imx/imx93-blk-ctrl.c b/drivers/pmdomain/imx/imx93-blk-ctrl.c index e094fe5a42bf..28688ca46517 100644 --- a/drivers/pmdomain/imx/imx93-blk-ctrl.c +++ b/drivers/pmdomain/imx/imx93-blk-ctrl.c @@ -47,6 +47,8 @@ #define PRIO(X) (X) +#define BLK_CTRL_NO_PARENT UINT_MAX + struct imx93_blk_ctrl_domain; struct imx93_blk_ctrl { @@ -67,12 +69,18 @@ struct imx93_blk_ctrl_qos { u32 cfg_prio; }; +struct imx93_blk_ctrl_subdomain_link { + struct generic_pm_domain *parent; + struct generic_pm_domain *subdomain; +}; + struct imx93_blk_ctrl_domain_data { const char *name; const char * const *clk_names; int num_clks; u32 rst_mask; u32 clk_mask; + u32 parent; int num_qos; struct imx93_blk_ctrl_qos qos[DOMAIN_MAX_QOS]; }; @@ -188,6 +196,27 @@ static int imx93_blk_ctrl_power_off(struct generic_pm_domain *genpd) return 0; } +static void imx93_release_genpd_provider(void *data) +{ + struct device_node *of_node = data; + + of_genpd_del_provider(of_node); +} + +static void imx93_release_pm_genpd(void *data) +{ + struct generic_pm_domain *genpd = data; + + pm_genpd_remove(genpd); +} + +static void imx93_release_subdomain(void *data) +{ + struct imx93_blk_ctrl_subdomain_link *link = data; + + pm_genpd_remove_subdomain(link->parent, link->subdomain); +} + static struct lock_class_key blk_ctrl_genpd_lock_class; static int imx93_blk_ctrl_probe(struct platform_device *pdev) @@ -258,10 +287,8 @@ static int imx93_blk_ctrl_probe(struct platform_device *pdev) domain->clks[j].id = data->clk_names[j]; ret = devm_clk_bulk_get(dev, data->num_clks, domain->clks); - if (ret) { - dev_err_probe(dev, ret, "failed to get clock\n"); - goto cleanup_pds; - } + if (ret) + return dev_err_probe(dev, ret, "failed to get clock\n"); domain->genpd.name = data->name; domain->genpd.power_on = imx93_blk_ctrl_power_on; @@ -269,11 +296,12 @@ static int imx93_blk_ctrl_probe(struct platform_device *pdev) domain->bc = bc; ret = pm_genpd_init(&domain->genpd, NULL, true); - if (ret) { - dev_err_probe(dev, ret, "failed to init power domain\n"); - goto cleanup_pds; - } + if (ret) + return dev_err_probe(dev, ret, "failed to init power domain\n"); + ret = devm_add_action_or_reset(dev, imx93_release_pm_genpd, &domain->genpd); + if (ret) + return dev_err_probe(dev, ret, "failed to add pm_genpd release callback\n"); /* * We use runtime PM to trigger power on/off of the upstream GPC * domain, as a strict hierarchical parent/child power domain @@ -290,39 +318,47 @@ static int imx93_blk_ctrl_probe(struct platform_device *pdev) bc->onecell_data.domains[i] = &domain->genpd; } - pm_runtime_enable(dev); - - ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); - if (ret) { - dev_err_probe(dev, ret, "failed to add power domain provider\n"); - goto cleanup_pds; - } + for (i = 0; i < bc_data->num_domains; i++) { + struct imx93_blk_ctrl_domain *domain = &bc->domains[i]; + const struct imx93_blk_ctrl_domain_data *data = domain->data; + struct imx93_blk_ctrl_subdomain_link *link; - dev_set_drvdata(dev, bc); + if (bc_data->skip_mask & BIT(i) || + data->parent == BLK_CTRL_NO_PARENT) + continue; - return 0; + link = devm_kzalloc(dev, sizeof(*link), GFP_KERNEL); + if (!link) + return -ENOMEM; -cleanup_pds: - for (i--; i >= 0; i--) - pm_genpd_remove(&bc->domains[i].genpd); + link->parent = &bc->domains[data->parent].genpd; + link->subdomain = &domain->genpd; - return ret; -} + ret = pm_genpd_add_subdomain(&bc->domains[data->parent].genpd, + &domain->genpd); + if (ret) + return dev_err_probe(dev, ret, "failed to add subdomain %s\n", + domain->genpd.name); -static void imx93_blk_ctrl_remove(struct platform_device *pdev) -{ - struct imx93_blk_ctrl *bc = dev_get_drvdata(&pdev->dev); - int i; + ret = devm_add_action_or_reset(dev, imx93_release_subdomain, link); + if (ret) + return dev_err_probe(dev, ret, + "failed to add subdomain release callback\n"); + } - of_genpd_del_provider(pdev->dev.of_node); + ret = devm_pm_runtime_enable(dev); + if (ret) + return dev_err_probe(dev, ret, "failed to enable pm-runtime\n"); - pm_runtime_disable(&pdev->dev); + ret = of_genpd_add_provider_onecell(dev->of_node, &bc->onecell_data); + if (ret) + return dev_err_probe(dev, ret, "failed to add power domain provider\n"); - for (i = 0; i < bc->onecell_data.num_domains; i++) { - struct imx93_blk_ctrl_domain *domain = &bc->domains[i]; + ret = devm_add_action_or_reset(dev, imx93_release_genpd_provider, dev->of_node); + if (ret) + return dev_err_probe(dev, ret, "failed to add genpd_provider release callback\n"); - pm_genpd_remove(&domain->genpd); - } + return 0; } static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] = { @@ -330,8 +366,9 @@ static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] .name = "mediablk-mipi-dsi", .clk_names = (const char *[]){ "dsi" }, .num_clks = 1, - .rst_mask = BIT(11) | BIT(12), - .clk_mask = BIT(11) | BIT(12), + .rst_mask = BIT(11), + .clk_mask = BIT(11), + .parent = IMX93_MEDIABLK_PD_MIPI_PHY, }, [IMX93_MEDIABLK_PD_MIPI_CSI] = { .name = "mediablk-mipi-csi", @@ -339,6 +376,7 @@ static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] .num_clks = 2, .rst_mask = BIT(9) | BIT(10), .clk_mask = BIT(9) | BIT(10), + .parent = IMX93_MEDIABLK_PD_MIPI_PHY, }, [IMX93_MEDIABLK_PD_PXP] = { .name = "mediablk-pxp", @@ -346,6 +384,7 @@ static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] .num_clks = 1, .rst_mask = BIT(7) | BIT(8), .clk_mask = BIT(7) | BIT(8), + .parent = BLK_CTRL_NO_PARENT, .num_qos = 2, .qos = { { @@ -367,6 +406,7 @@ static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] .num_clks = 2, .rst_mask = BIT(4) | BIT(5) | BIT(6), .clk_mask = BIT(4) | BIT(5) | BIT(6), + .parent = BLK_CTRL_NO_PARENT, .num_qos = 1, .qos = { { @@ -383,6 +423,7 @@ static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] .num_clks = 1, .rst_mask = BIT(2) | BIT(3), .clk_mask = BIT(2) | BIT(3), + .parent = BLK_CTRL_NO_PARENT, .num_qos = 4, .qos = { { @@ -408,6 +449,14 @@ static const struct imx93_blk_ctrl_domain_data imx93_media_blk_ctl_domain_data[] } } }, + [IMX93_MEDIABLK_PD_MIPI_PHY] = { + .name = "mediablk-mipi-phy", + .clk_names = NULL, + .num_clks = 0, + .rst_mask = BIT(12), + .clk_mask = BIT(12), + .parent = BLK_CTRL_NO_PARENT, + }, }; static const struct regmap_range imx93_media_blk_ctl_yes_ranges[] = { @@ -457,7 +506,6 @@ MODULE_DEVICE_TABLE(of, imx93_blk_ctrl_of_match); static struct platform_driver imx93_blk_ctrl_driver = { .probe = imx93_blk_ctrl_probe, - .remove = imx93_blk_ctrl_remove, .driver = { .name = "imx93-blk-ctrl", .of_match_table = imx93_blk_ctrl_of_match, diff --git a/drivers/ptp/ptp_netc.c b/drivers/ptp/ptp_netc.c index 94e952ee6990..5e381c354d74 100644 --- a/drivers/ptp/ptp_netc.c +++ b/drivers/ptp/ptp_netc.c @@ -779,6 +779,7 @@ static void netc_timer_init(struct netc_timer *priv) netc_timer_wr(priv, NETC_TMR_FIPER_CTRL, fiper_ctrl); netc_timer_wr(priv, NETC_TMR_ECTRL, NETC_TMR_DEFAULT_ETTF_THR); + netc_timer_offset_write(priv, 0); ktime_get_real_ts64(&now); ns = timespec64_to_ns(&now); netc_timer_cnt_write(priv, ns); diff --git a/drivers/ptp/ptp_s390.c b/drivers/ptp/ptp_s390.c index 29618eb9bf44..02d624d89a0a 100644 --- a/drivers/ptp/ptp_s390.c +++ b/drivers/ptp/ptp_s390.c @@ -107,6 +107,9 @@ static __init int ptp_s390_init(void) if (IS_ERR(ptp_stcke_clock)) return PTR_ERR(ptp_stcke_clock); + if (!test_facility(28) || !ptff_query(PTFF_QPT)) + return 0; + ptp_qpt_clock = ptp_clock_register(&ptp_s390_qpt_info, NULL); if (IS_ERR(ptp_qpt_clock)) { ptp_clock_unregister(ptp_stcke_clock); @@ -117,7 +120,8 @@ static __init int ptp_s390_init(void) static __exit void ptp_s390_exit(void) { - ptp_clock_unregister(ptp_qpt_clock); + if (ptp_qpt_clock) + ptp_clock_unregister(ptp_qpt_clock); ptp_clock_unregister(ptp_stcke_clock); } diff --git a/drivers/regulator/mt6358-regulator.c b/drivers/regulator/mt6358-regulator.c index e4745f616cea..44bef0bdfdbe 100644 --- a/drivers/regulator/mt6358-regulator.c +++ b/drivers/regulator/mt6358-regulator.c @@ -492,7 +492,7 @@ static const struct regulator_ops mt6358_volt_fixed_ops = { .list_voltage = regulator_list_voltage_linear, .map_voltage = regulator_map_voltage_linear, .set_voltage_sel = regulator_set_voltage_sel_regmap, - .get_voltage_sel = mt6358_get_buck_voltage_sel, + .get_voltage_sel = regulator_get_voltage_sel_regmap, .set_voltage_time_sel = regulator_set_voltage_time_sel, .enable = regulator_enable_regmap, .disable = regulator_disable_regmap, diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c index 5ef586fd37bc..a238b45bf2f2 100644 --- a/drivers/remoteproc/xlnx_r5_remoteproc.c +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c @@ -959,16 +959,6 @@ static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev) goto free_rproc; } - /* - * If firmware is already available in the memory then move rproc state - * to DETACHED. Firmware can be preloaded via debugger or by any other - * agent (processors) in the system. - * If firmware isn't available in the memory and resource table isn't - * found, then rproc state remains OFFLINE. - */ - if (!zynqmp_r5_get_rsc_table_va(r5_core)) - r5_rproc->state = RPROC_DETACHED; - r5_core->rproc = r5_rproc; return r5_core; @@ -1221,6 +1211,7 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster, { struct device *dev = cluster->dev; struct zynqmp_r5_core *r5_core; + u32 payload[PAYLOAD_ARG_CNT]; int ret = -EINVAL, i; r5_core = cluster->r5_cores[0]; @@ -1266,6 +1257,45 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster, ret = zynqmp_r5_get_sram_banks(r5_core); if (ret) return ret; + + /* + * It is possible that firmware is loaded into the memory, but + * RPU (remote) is not running. In such case, RPU state will be + * moved to RPROC_DETACHED wrongfully. To avoid it first make + * sure RPU is power-on and out of reset before parsing for the + * resource table. + */ + ret = zynqmp_pm_feature(PM_GET_NODE_STATUS); + if (ret < PM_API_VERSION_2) + ret = -EOPNOTSUPP; + else + ret = zynqmp_pm_invoke_fn(PM_GET_NODE_STATUS, payload, 1, + r5_core->pm_domain_id); + if (ret) { + dev_warn(r5_core->dev, + "failed to get rpu node status, err %d\n", ret); + continue; + } + + /* + * If RPU state is power on and out of reset i.e. running, then + * assign RPROC_DETACHED state. If the RPU is not out of reset + * then do not attempt to attach to the remote processor. + */ + if (payload[1] == PM_NODE_RUNNING) { + /* + * Not all the firmware that is running on the remote + * core is expected to have the resource table. The + * firmware might not use RPMsg at all, and in that case + * resource table becomes irrelevant. However, we still + * need to make sure that running core is not reported + * as offline. so do not decide remote core state based + * on the resource table availability + */ + if (zynqmp_r5_get_rsc_table_va(r5_core)) + dev_dbg(r5_core->dev, "rsc tbl not found\n"); + r5_core->rproc->state = RPROC_DETACHED; + } } return 0; diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c index 17173239301e..a1055fe4922f 100644 --- a/drivers/scsi/hosts.c +++ b/drivers/scsi/hosts.c @@ -343,6 +343,7 @@ static void scsi_host_dev_release(struct device *dev) /* Wait for functions invoked through call_rcu(&scmd->rcu, ...) */ rcu_barrier(); + cancel_work_sync(&shost->eh_work); if (shost->tmf_work_q) destroy_workqueue(shost->tmf_work_q); if (shost->ehandler) @@ -408,6 +409,7 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv INIT_LIST_HEAD(&shost->starved_list); init_waitqueue_head(&shost->host_wait); mutex_init(&shost->scan_mutex); + INIT_WORK(&shost->eh_work, scsi_rcu_eh_wakeup); index = ida_alloc(&host_index_ida, GFP_KERNEL); if (index < 0) { diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c index 2fd4ca96b308..13d46b6e0359 100644 --- a/drivers/scsi/scsi_error.c +++ b/drivers/scsi/scsi_error.c @@ -73,6 +73,26 @@ void scsi_eh_wakeup(struct Scsi_Host *shost, unsigned int busy) } } +void scsi_rcu_eh_wakeup(struct work_struct *work) +{ + struct Scsi_Host *shost = container_of(work, struct Scsi_Host, eh_work); + unsigned long flags; + unsigned int busy; + + /* + * Ensure any running scsi_dec_host_busy has completed its rcu section + * so changes to host state and host_eh_scheduled are visible to all + * future calls of scsi_dec_host_busy + */ + synchronize_rcu(); + + busy = scsi_host_busy(shost); + + spin_lock_irqsave(shost->host_lock, flags); + scsi_eh_wakeup(shost, busy); + spin_unlock_irqrestore(shost->host_lock, flags); +} + /** * scsi_schedule_eh - schedule EH for SCSI host * @shost: SCSI host to invoke error handling on. @@ -88,7 +108,7 @@ void scsi_schedule_eh(struct Scsi_Host *shost) if (scsi_host_set_state(shost, SHOST_RECOVERY) == 0 || scsi_host_set_state(shost, SHOST_CANCEL_RECOVERY) == 0) { shost->host_eh_scheduled++; - scsi_eh_wakeup(shost, scsi_host_busy(shost)); + queue_work(shost->tmf_work_q, &shost->eh_work); } spin_unlock_irqrestore(shost->host_lock, flags); diff --git a/drivers/scsi/scsi_priv.h b/drivers/scsi/scsi_priv.h index 5b2b19f5e8ec..68866b18a600 100644 --- a/drivers/scsi/scsi_priv.h +++ b/drivers/scsi/scsi_priv.h @@ -91,6 +91,7 @@ extern enum blk_eh_timer_return scsi_timeout(struct request *req); extern int scsi_error_handler(void *host); extern enum scsi_disposition scsi_decide_disposition(struct scsi_cmnd *cmd); extern void scsi_eh_wakeup(struct Scsi_Host *shost, unsigned int busy); +extern void scsi_rcu_eh_wakeup(struct work_struct *work); extern void scsi_eh_scmd_add(struct scsi_cmnd *); void scsi_eh_ready_devs(struct Scsi_Host *shost, struct list_head *work_q, diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c index 49e497a32973..8545d62683eb 100644 --- a/drivers/staging/media/meson/vdec/vdec.c +++ b/drivers/staging/media/meson/vdec/vdec.c @@ -889,7 +889,7 @@ static int vdec_open(struct file *file) ret = vdec_init_ctrls(sess); if (ret) - goto err_m2m_release; + goto err_m2m_ctx_release; sess->pixfmt_cap = formats[0].pixfmts_cap[0]; sess->fmt_out = &formats[0]; @@ -913,6 +913,8 @@ static int vdec_open(struct file *file) return 0; +err_m2m_ctx_release: + v4l2_m2m_ctx_release(sess->m2m_ctx); err_m2m_release: v4l2_m2m_release(sess->m2m_dev); err_free_sess: diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index 1c7191bb0d8f..6de345b4ed17 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -391,6 +391,7 @@ static int cedrus_open(struct file *file) err_m2m_release: v4l2_m2m_ctx_release(ctx->fh.m2m_ctx); err_free: + v4l2_fh_exit(&ctx->fh); kfree(ctx); mutex_unlock(&dev->dev_mutex); @@ -507,7 +508,7 @@ static int cedrus_probe(struct platform_device *pdev) ret = video_register_device(vfd, VFL_TYPE_VIDEO, 0); if (ret) { v4l2_err(&dev->v4l2_dev, "Failed to register video device\n"); - goto err_m2m; + goto err_media; } v4l2_info(&dev->v4l2_dev, @@ -533,7 +534,8 @@ static int cedrus_probe(struct platform_device *pdev) v4l2_m2m_unregister_media_controller(dev->m2m_dev); err_video: video_unregister_device(&dev->vfd); -err_m2m: +err_media: + media_device_cleanup(&dev->mdev); v4l2_m2m_release(dev->m2m_dev); err_v4l2: v4l2_device_unregister(&dev->v4l2_dev); diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index 3e2843ef6cce..fc54d993b11f 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -210,6 +210,9 @@ static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, u8 dpb_idx; dpb_idx = ref_list[i].index; + if (dpb_idx >= V4L2_H264_NUM_DPB_ENTRIES) + continue; + dpb = &decode->dpb[dpb_idx]; if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) diff --git a/drivers/staging/media/tegra-video/vi.c b/drivers/staging/media/tegra-video/vi.c index 14b327afe045..46f169346782 100644 --- a/drivers/staging/media/tegra-video/vi.c +++ b/drivers/staging/media/tegra-video/vi.c @@ -80,8 +80,8 @@ static int tegra_get_format_idx_by_code(struct tegra_vi *vi, static u32 tegra_get_format_fourcc_by_idx(struct tegra_vi *vi, unsigned int index) { - if (index >= vi->soc->nformats) - return -EINVAL; + if (WARN_ON_ONCE(index >= vi->soc->nformats)) + return vi->soc->video_formats[0].fourcc; return vi->soc->video_formats[index].fourcc; } diff --git a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c index 01ab7fb5e5bc..e61bc9c6bb21 100644 --- a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c +++ b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c @@ -679,7 +679,14 @@ u8 *rtw_get_wps_ie(u8 *in_ie, uint in_len, u8 *wps_ie, uint *wps_ielen) while (cnt < in_len) { eid = in_ie[cnt]; - if ((eid == WLAN_EID_VENDOR_SPECIFIC) && (!memcmp(&in_ie[cnt + 2], wps_oui, 4))) { + if (cnt + 2 > in_len) + break; + + if (in_ie[cnt + 1] + 2 > in_len - cnt) + break; + + if ((eid == WLAN_EID_VENDOR_SPECIFIC) && (in_ie[cnt + 1] >= 4) && + (!memcmp(&in_ie[cnt + 2], wps_oui, 4))) { wpsie_ptr = &in_ie[cnt]; if (wps_ie) diff --git a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c index 73fc1d4f92e6..1bc60be309dd 100644 --- a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c +++ b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c @@ -1958,7 +1958,7 @@ static u8 rtw_get_chan_type(struct adapter *adapter) else return NL80211_CHAN_NO_HT; case CHANNEL_WIDTH_40: - if (mlme_ext->cur_ch_offset == HAL_PRIME_CHNL_OFFSET_UPPER) + if (mlme_ext->cur_ch_offset == HAL_PRIME_CHNL_OFFSET_LOWER) return NL80211_CHAN_HT40PLUS; else return NL80211_CHAN_HT40MINUS; diff --git a/drivers/tty/serial/8250/8250_mid.c b/drivers/tty/serial/8250/8250_mid.c index f88809ff370b..82656645b8a6 100644 --- a/drivers/tty/serial/8250/8250_mid.c +++ b/drivers/tty/serial/8250/8250_mid.c @@ -318,9 +318,11 @@ static int mid8250_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (!uart.port.membase) return -ENOMEM; - ret = mid->board->setup(mid, &uart.port); - if (ret) - return ret; + if (mid->board->setup) { + ret = mid->board->setup(mid, &uart.port); + if (ret) + return ret; + } ret = mid8250_dma_setup(mid, &uart); if (ret) @@ -336,7 +338,8 @@ static int mid8250_probe(struct pci_dev *pdev, const struct pci_device_id *id) return 0; err: - mid->board->exit(mid); + if (mid->board->exit) + mid->board->exit(mid); return ret; } @@ -346,7 +349,8 @@ static void mid8250_remove(struct pci_dev *pdev) serial8250_unregister_port(mid->line); - mid->board->exit(mid); + if (mid->board->exit) + mid->board->exit(mid); } static const struct mid8250_board pnw_board = { diff --git a/drivers/tty/serial/sc16is7xx.c b/drivers/tty/serial/sc16is7xx.c index c7435595dce1..13a86f0c2d9b 100644 --- a/drivers/tty/serial/sc16is7xx.c +++ b/drivers/tty/serial/sc16is7xx.c @@ -1328,6 +1328,17 @@ static int sc16is7xx_gpio_set(struct gpio_chip *chip, unsigned int offset, return 0; } +static int sc16is7xx_gpio_get_direction(struct gpio_chip *chip, unsigned int offset) +{ + struct sc16is7xx_port *s = gpiochip_get_data(chip); + struct uart_port *port = &s->p[0].port; + unsigned int val; + + val = sc16is7xx_port_read(port, SC16IS7XX_IODIR_REG); + + return val & BIT(offset) ? GPIO_LINE_DIRECTION_OUT : GPIO_LINE_DIRECTION_IN; +} + static int sc16is7xx_gpio_direction_input(struct gpio_chip *chip, unsigned offset) { @@ -1405,6 +1416,7 @@ static int sc16is7xx_setup_gpio_chip(struct sc16is7xx_port *s) s->gpio.parent = dev; s->gpio.label = dev_name(dev); s->gpio.init_valid_mask = sc16is7xx_gpio_init_valid_mask; + s->gpio.get_direction = sc16is7xx_gpio_get_direction; s->gpio.direction_input = sc16is7xx_gpio_direction_input; s->gpio.get = sc16is7xx_gpio_get; s->gpio.direction_output = sc16is7xx_gpio_direction_output; diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c index 3f8e2e27697f..c47d46334a7c 100644 --- a/drivers/uio/uio_hv_generic.c +++ b/drivers/uio/uio_hv_generic.c @@ -395,9 +395,15 @@ hv_uio_remove(struct hv_device *dev) vmbus_free_ring(dev->channel); } +static const struct hv_vmbus_device_id hv_uio_id_table[] = { + { HV_FCOPY_GUID }, + {} +}; +MODULE_DEVICE_TABLE(vmbus, hv_uio_id_table); + static struct hv_driver hv_uio_drv = { .name = "uio_hv_generic", - .id_table = NULL, /* only dynamic id's */ + .id_table = hv_uio_id_table, .probe = hv_uio_probe, .remove = hv_uio_remove, }; diff --git a/drivers/usb/atm/ueagle-atm.c b/drivers/usb/atm/ueagle-atm.c index a943a5257a59..f1f1fc16a471 100644 --- a/drivers/usb/atm/ueagle-atm.c +++ b/drivers/usb/atm/ueagle-atm.c @@ -2550,6 +2550,7 @@ static struct usbatm_driver uea_usbatm_driver = { static int uea_probe(struct usb_interface *intf, const struct usb_device_id *id) { struct usb_device *usb = interface_to_usbdev(intf); + bool single_iface = usb->config->desc.bNumInterfaces == 1; int ret; uea_dbg(usb, "ADSL device found with vid (%#X) pid (%#X) Rev (%#X): %s\n", @@ -2558,6 +2559,22 @@ static int uea_probe(struct usb_interface *intf, const struct usb_device_id *id) le16_to_cpu(usb->descriptor.bcdDevice), chip_name[UEA_CHIP_VERSION(id)]); + /* + * uea_probe() decides between the pre-firmware and post-firmware case + * from the USB id and stores a different object as interface data in + * each case: a struct completion for a pre-firmware device, a struct + * usbatm_data for a post-firmware one. uea_disconnect() instead tells + * the two apart by the number of interfaces (a pre-firmware device + * exposes a single interface, ADI930 has 2 and eagle has 3). A crafted + * device advertising a pre-firmware id together with a multi-interface + * descriptor (or the other way around) makes the two disagree, so that + * usbatm_usb_disconnect() treats the small completion object as a + * struct usbatm_data and reads out of bounds. Reject such inconsistent + * descriptors so both paths make the same decision. + */ + if (UEA_IS_PREFIRM(id) != single_iface) + return -ENODEV; + usb_reset_device(usb); if (UEA_IS_PREFIRM(id)) { diff --git a/drivers/usb/chipidea/core.c b/drivers/usb/chipidea/core.c index 4d3a89b9d386..628b83099764 100644 --- a/drivers/usb/chipidea/core.c +++ b/drivers/usb/chipidea/core.c @@ -1265,6 +1265,7 @@ static void ci_hdrc_remove(struct platform_device *pdev) usb_role_switch_unregister(ci->role_switch); if (ci->supports_runtime_pm) { + pm_runtime_dont_use_autosuspend(&pdev->dev); pm_runtime_get_sync(&pdev->dev); pm_runtime_disable(&pdev->dev); pm_runtime_put_noidle(&pdev->dev); diff --git a/drivers/usb/core/port.c b/drivers/usb/core/port.c index a47df5d32f7c..7cae19484355 100644 --- a/drivers/usb/core/port.c +++ b/drivers/usb/core/port.c @@ -726,6 +726,8 @@ static void connector_unbind(struct device *dev, struct device *connector, void sysfs_remove_link(&connector->kobj, dev_name(dev)); sysfs_remove_link(&dev->kobj, "connector"); + if (port_dev->child) + typec_deattach(port_dev->connector, &port_dev->child->dev); port_dev->connector = NULL; } diff --git a/drivers/usb/core/sysfs.c b/drivers/usb/core/sysfs.c index a07866f1060c..d22dc78457d7 100644 --- a/drivers/usb/core/sysfs.c +++ b/drivers/usb/core/sysfs.c @@ -899,10 +899,15 @@ bos_descriptors_read(struct file *filp, struct kobject *kobj, { struct device *dev = kobj_to_dev(kobj); struct usb_device *udev = to_usb_device(dev); - struct usb_host_bos *bos = udev->bos; + struct usb_host_bos *bos; struct usb_bos_descriptor *desc; size_t desclen, n = 0; + int rc; + rc = usb_lock_device_interruptible(udev); + if (rc < 0) + return -EINTR; + bos = udev->bos; if (bos) { desc = bos->desc; desclen = le16_to_cpu(desc->wTotalLength); @@ -911,6 +916,7 @@ bos_descriptors_read(struct file *filp, struct kobject *kobj, memcpy(buf, (void *) desc + off, n); } } + usb_unlock_device(udev); return n; } static const BIN_ATTR_RO(bos_descriptors, 65535); /* max-size BOS */ diff --git a/drivers/usb/gadget/function/f_midi.c b/drivers/usb/gadget/function/f_midi.c index da82598fcef8..28eca6698ada 100644 --- a/drivers/usb/gadget/function/f_midi.c +++ b/drivers/usb/gadget/function/f_midi.c @@ -1302,6 +1302,7 @@ static void f_midi_free(struct usb_function *f) opts = container_of(f->fi, struct f_midi_opts, func_inst); mutex_lock(&opts->lock); if (!--midi->free_ref) { + cancel_work_sync(&midi->work); kfree(midi->id); kfifo_free(&midi->in_req_fifo); kfree(midi); diff --git a/drivers/usb/gadget/function/f_ncm.c b/drivers/usb/gadget/function/f_ncm.c index 04be7047aa3d..fb57d57781dd 100644 --- a/drivers/usb/gadget/function/f_ncm.c +++ b/drivers/usb/gadget/function/f_ncm.c @@ -1189,6 +1189,10 @@ static int ncm_unwrap_ntb(struct gether *port, frame_max = ncm_opts->max_segment_size; parse_ntb: + if (to_process < (int)opts->nth_size) { + INFO(port->func.config->cdev, "Packet too small for headers\n"); + goto err; + } tmp = (__le16 *)ntb_ptr; /* dwSignature */ @@ -1209,8 +1213,12 @@ static int ncm_unwrap_ntb(struct gether *port, tmp++; /* skip wSequence */ block_len = get_ncm(&tmp, opts->block_length); + if (block_len == 0) + block_len = to_process; + /* (d)wBlockLength */ - if ((block_len < opts->nth_size + opts->ndp_size) || (block_len > ntb_max)) { + if ((block_len < opts->nth_size + opts->ndp_size) || (block_len > ntb_max) || + (block_len > to_process)) { INFO(port->func.config->cdev, "Bad block length: %#X\n", block_len); goto err; } @@ -1273,7 +1281,7 @@ static int ncm_unwrap_ntb(struct gether *port, index = index2; /* wDatagramIndex[0] */ if ((index < opts->nth_size) || - (index > block_len - opts->dpe_size)) { + (index > block_len)) { INFO(port->func.config->cdev, "Bad index: %#X\n", index); goto err; @@ -1285,7 +1293,8 @@ static int ncm_unwrap_ntb(struct gether *port, * ethernet hdr + crc or larger than max frame size */ if ((dg_len < 14 + crc_len) || - (dg_len > frame_max)) { + (dg_len > frame_max) || + (dg_len > block_len - index)) { INFO(port->func.config->cdev, "Bad dgram length: %#X\n", dg_len); goto err; @@ -1310,7 +1319,7 @@ static int ncm_unwrap_ntb(struct gether *port, dg_len2 = get_ncm(&tmp, opts->dgram_item_len); /* wDatagramIndex[1] */ - if (index2 > block_len - opts->dpe_size) { + if (index2 > block_len) { INFO(port->func.config->cdev, "Bad index: %#X\n", index2); goto err; diff --git a/drivers/usb/gadget/function/f_printer.c b/drivers/usb/gadget/function/f_printer.c index a3884afa42b4..3db07d0fba89 100644 --- a/drivers/usb/gadget/function/f_printer.c +++ b/drivers/usb/gadget/function/f_printer.c @@ -431,7 +431,7 @@ printer_read(struct file *fd, char __user *buf, size_t len, loff_t *ptr) { struct printer_dev *dev = fd->private_data; unsigned long flags; - size_t size; + size_t size, not_copied, copied; size_t bytes_copied; struct usb_request *req; /* This is a pointer to the current USB rx request. */ @@ -524,10 +524,12 @@ printer_read(struct file *fd, char __user *buf, size_t len, loff_t *ptr) else size = len; - size -= copy_to_user(buf, current_rx_buf, size); - bytes_copied += size; - len -= size; - buf += size; + not_copied = copy_to_user(buf, current_rx_buf, size); + copied = size - not_copied; + + bytes_copied += copied; + len -= copied; + buf += copied; spin_lock_irqsave(&dev->lock, flags); @@ -542,6 +544,17 @@ printer_read(struct file *fd, char __user *buf, size_t len, loff_t *ptr) if (dev->interface < 0) goto out_disabled; + if (!copied) { + dev->current_rx_req = current_rx_req; + dev->current_rx_bytes = current_rx_bytes; + dev->current_rx_buf = current_rx_buf; + spin_unlock_irqrestore(&dev->lock, flags); + mutex_unlock(&dev->lock_printer_io); + return bytes_copied ? bytes_copied : -EFAULT; + } + + size = copied; + /* If we not returning all the data left in this RX request * buffer then adjust the amount of data left in the buffer. * Othewise if we are done with this RX request buffer then diff --git a/drivers/usb/gadget/function/f_tcm.c b/drivers/usb/gadget/function/f_tcm.c index 7b27f8082ace..5753c83f5578 100644 --- a/drivers/usb/gadget/function/f_tcm.c +++ b/drivers/usb/gadget/function/f_tcm.c @@ -2361,31 +2361,158 @@ static int tcm_bind(struct usb_configuration *c, struct usb_function *f) return -ENOTSUPP; } -struct guas_setup_wq { - struct work_struct work; - struct f_uas *fu; - unsigned int alt; -}; +static void tcm_cleanup_old_alt(struct f_uas *fu) +{ + if (fu->flags & USBG_IS_UAS) + uasp_cleanup_old_alt(fu); + else if (fu->flags & USBG_IS_BOT) + bot_cleanup_old_alt(fu); + fu->flags = 0; +} + +static void tcm_delayed_set_alt_done(struct f_uas *fu) +{ + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); +} + +static bool tcm_delayed_set_alt_cancelled(struct f_uas *fu) +{ + bool cancelled; + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + cancelled = fu->delayed_set_alt_cancel; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + return cancelled; +} + +static bool tcm_complete_delayed_status(struct f_uas *fu) +{ + struct usb_composite_dev *cdev = fu->function.config->cdev; + struct usb_request *req = cdev->req; + unsigned long cdev_flags; + bool cancelled; + int ret; + + spin_lock_irqsave(&cdev->lock, cdev_flags); + spin_lock(&fu->delayed_set_alt_lock); + cancelled = fu->delayed_set_alt_cancel; + if (!cancelled) { + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + } + spin_unlock(&fu->delayed_set_alt_lock); + + if (cancelled) { + spin_unlock_irqrestore(&cdev->lock, cdev_flags); + return false; + } + + if (cdev->delayed_status == 0) { + WARN(cdev, "%s: Unexpected call\n", __func__); + } else if (--cdev->delayed_status == 0) { + req->length = 0; + req->context = cdev; + ret = usb_ep_queue(cdev->gadget->ep0, req, GFP_ATOMIC); + if (ret == 0) { + cdev->setup_pending = true; + } else { + req->status = 0; + req->complete(cdev->gadget->ep0, req); + } + } + + spin_unlock_irqrestore(&cdev->lock, cdev_flags); + + return true; +} + +static bool tcm_cancel_delayed_set_alt(struct f_uas *fu) +{ + bool cleanup = false; + bool cancel = false; + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + switch (fu->delayed_set_alt_state) { + case USBG_DELAYED_SET_ALT_IDLE: + cleanup = true; + break; + case USBG_DELAYED_SET_ALT_QUEUED: + case USBG_DELAYED_SET_ALT_RUNNING: + fu->delayed_set_alt_cancel = true; + cancel = true; + break; + } + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + if (cancel && cancel_work(&fu->delayed_set_alt)) { + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state == USBG_DELAYED_SET_ALT_QUEUED) { + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + cleanup = true; + } + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + } + + return cleanup; +} + +static void tcm_cancel_delayed_set_alt_sync(struct f_uas *fu) +{ + unsigned long flags; + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state != USBG_DELAYED_SET_ALT_IDLE) + fu->delayed_set_alt_cancel = true; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + cancel_work_sync(&fu->delayed_set_alt); + + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_IDLE; + fu->delayed_set_alt_cancel = false; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); +} static void tcm_delayed_set_alt(struct work_struct *wq) { - struct guas_setup_wq *work = container_of(wq, struct guas_setup_wq, - work); - struct f_uas *fu = work->fu; - int alt = work->alt; + struct f_uas *fu = container_of(wq, struct f_uas, delayed_set_alt); + unsigned long flags; + unsigned int alt; - kfree(work); + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state != USBG_DELAYED_SET_ALT_QUEUED) { + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + return; + } + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_RUNNING; + alt = fu->delayed_alt; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); - if (fu->flags & USBG_IS_BOT) - bot_cleanup_old_alt(fu); - if (fu->flags & USBG_IS_UAS) - uasp_cleanup_old_alt(fu); + tcm_cleanup_old_alt(fu); + + if (tcm_delayed_set_alt_cancelled(fu)) + goto out_done; if (alt == USB_G_ALT_INT_BBB) bot_set_alt(fu); else if (alt == USB_G_ALT_INT_UAS) uasp_set_alt(fu); - usb_composite_setup_continue(fu->function.config->cdev); + + if (tcm_complete_delayed_status(fu)) + return; + + tcm_cleanup_old_alt(fu); +out_done: + tcm_delayed_set_alt_done(fu); } static int tcm_get_alt(struct usb_function *f, unsigned intf) @@ -2411,15 +2538,20 @@ static int tcm_set_alt(struct usb_function *f, unsigned intf, unsigned alt) return -EOPNOTSUPP; if ((alt == USB_G_ALT_INT_BBB) || (alt == USB_G_ALT_INT_UAS)) { - struct guas_setup_wq *work; + unsigned long flags; - work = kmalloc(sizeof(*work), GFP_ATOMIC); - if (!work) - return -ENOMEM; - INIT_WORK(&work->work, tcm_delayed_set_alt); - work->fu = fu; - work->alt = alt; - schedule_work(&work->work); + spin_lock_irqsave(&fu->delayed_set_alt_lock, flags); + if (fu->delayed_set_alt_state != USBG_DELAYED_SET_ALT_IDLE) { + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, + flags); + return -EBUSY; + } + fu->delayed_alt = alt; + fu->delayed_set_alt_cancel = false; + fu->delayed_set_alt_state = USBG_DELAYED_SET_ALT_QUEUED; + spin_unlock_irqrestore(&fu->delayed_set_alt_lock, flags); + + schedule_work(&fu->delayed_set_alt); return USB_GADGET_DELAYED_STATUS; } return -EOPNOTSUPP; @@ -2429,11 +2561,8 @@ static void tcm_disable(struct usb_function *f) { struct f_uas *fu = to_f_uas(f); - if (fu->flags & USBG_IS_UAS) - uasp_cleanup_old_alt(fu); - else if (fu->flags & USBG_IS_BOT) - bot_cleanup_old_alt(fu); - fu->flags = 0; + if (tcm_cancel_delayed_set_alt(fu)) + tcm_cleanup_old_alt(fu); } static int tcm_setup(struct usb_function *f, @@ -2581,11 +2710,16 @@ static void tcm_free(struct usb_function *f) { struct f_uas *tcm = to_f_uas(f); + tcm_cancel_delayed_set_alt_sync(tcm); kfree(tcm); } static void tcm_unbind(struct usb_configuration *c, struct usb_function *f) { + struct f_uas *fu = to_f_uas(f); + + tcm_cancel_delayed_set_alt_sync(fu); + tcm_cleanup_old_alt(fu); usb_free_all_descriptors(f); } @@ -2618,6 +2752,8 @@ static struct usb_function *tcm_alloc(struct usb_function_instance *fi) fu->function.disable = tcm_disable; fu->function.free_func = tcm_free; fu->tpg = tpg_instances[i].tpg; + INIT_WORK(&fu->delayed_set_alt, tcm_delayed_set_alt); + spin_lock_init(&fu->delayed_set_alt_lock); hash_init(fu->stream_hash); mutex_unlock(&tpg_instances_lock); diff --git a/drivers/usb/gadget/function/tcm.h b/drivers/usb/gadget/function/tcm.h index 009974d81d66..e1d5a9391612 100644 --- a/drivers/usb/gadget/function/tcm.h +++ b/drivers/usb/gadget/function/tcm.h @@ -3,6 +3,7 @@ #define __TARGET_USB_GADGET_H__ #include +#include /* #include */ #include #include @@ -29,6 +30,12 @@ enum { #define USB_G_DEFAULT_SESSION_TAGS USBG_NUM_CMDS +enum { + USBG_DELAYED_SET_ALT_IDLE = 0, + USBG_DELAYED_SET_ALT_QUEUED, + USBG_DELAYED_SET_ALT_RUNNING, +}; + struct tcm_usbg_nexus { struct se_session *tvn_se_sess; }; @@ -132,6 +139,12 @@ struct f_uas { #define USBG_BOT_CMD_PEND (1 << 4) #define USBG_BOT_WEDGED (1 << 5) + struct work_struct delayed_set_alt; + spinlock_t delayed_set_alt_lock; /* protects delayed_set_alt_* */ + unsigned int delayed_alt; + unsigned int delayed_set_alt_state; + bool delayed_set_alt_cancel; + struct usbg_cdb cmd[USBG_NUM_CMDS]; struct usb_ep *ep_in; struct usb_ep *ep_out; diff --git a/drivers/usb/gadget/function/uvc_v4l2.c b/drivers/usb/gadget/function/uvc_v4l2.c index 23bafb07133f..b5384c88b84c 100644 --- a/drivers/usb/gadget/function/uvc_v4l2.c +++ b/drivers/usb/gadget/function/uvc_v4l2.c @@ -200,6 +200,8 @@ uvc_send_response(struct uvc_device *uvc, struct uvc_request_data *data) return usb_ep_set_halt(cdev->gadget->ep0); req->length = min_t(unsigned int, uvc->event_length, data->length); + if (req->length > sizeof(data->data)) + req->length = sizeof(data->data); req->zero = data->length < uvc->event_length; memcpy(req->buf, data->data, req->length); diff --git a/drivers/usb/gadget/udc/bdc/bdc_core.c b/drivers/usb/gadget/udc/bdc/bdc_core.c index f47aac078f6b..de44cda273fb 100644 --- a/drivers/usb/gadget/udc/bdc/bdc_core.c +++ b/drivers/usb/gadget/udc/bdc/bdc_core.c @@ -586,9 +586,29 @@ static int bdc_probe(struct platform_device *pdev) static void bdc_remove(struct platform_device *pdev) { struct bdc *bdc; + unsigned long flags; + u32 temp; bdc = platform_get_drvdata(pdev); dev_dbg(bdc->dev, "%s ()\n", __func__); + /* + * Disable the device interrupt source before freeing the IRQ: + * clear BDC_GIE so the controller stops asserting interrupts, + * then free_irq drains any in-flight handler. + */ + spin_lock_irqsave(&bdc->lock, flags); + temp = bdc_readl(bdc->regs, BDC_BDCSC); + temp &= ~BDC_GIE; + bdc_writel(bdc->regs, BDC_BDCSC, temp); + spin_unlock_irqrestore(&bdc->lock, flags); + free_irq(bdc->irq, bdc); + /* + * Drain func_wake_notify after free_irq: the IRQ handler arms this + * delayed_work via bdc_sr_uspc -> handle_link_state_change -> + * schedule_delayed_work (self-rearmed in bdc_func_wake_timer), so + * the IRQ must be released first to prevent re-arm after cancel. + */ + cancel_delayed_work_sync(&bdc->func_wake_notify); bdc_udc_exit(bdc); bdc_hw_exit(bdc); bdc_phy_exit(bdc); diff --git a/drivers/usb/gadget/udc/bdc/bdc_udc.c b/drivers/usb/gadget/udc/bdc/bdc_udc.c index 23826fd7a8e6..7a12219edac6 100644 --- a/drivers/usb/gadget/udc/bdc/bdc_udc.c +++ b/drivers/usb/gadget/udc/bdc/bdc_udc.c @@ -530,8 +530,8 @@ int bdc_udc_init(struct bdc *bdc) bdc->gadget.name = BRCM_BDC_NAME; - ret = devm_request_irq(bdc->dev, bdc->irq, bdc_udc_interrupt, - IRQF_SHARED, BRCM_BDC_NAME, bdc); + ret = request_irq(bdc->irq, bdc_udc_interrupt, IRQF_SHARED, + BRCM_BDC_NAME, bdc); if (ret) { dev_err(bdc->dev, "failed to request irq #%d %d\n", @@ -542,7 +542,7 @@ int bdc_udc_init(struct bdc *bdc) ret = bdc_init_ep(bdc); if (ret) { dev_err(bdc->dev, "bdc init ep fail: %d\n", ret); - return ret; + goto err0; } ret = usb_add_gadget_udc(bdc->dev, &bdc->gadget); @@ -571,6 +571,7 @@ int bdc_udc_init(struct bdc *bdc) err1: usb_del_gadget_udc(&bdc->gadget); err0: + free_irq(bdc->irq, bdc); bdc_free_ep(bdc); return ret; diff --git a/drivers/usb/gadget/udc/dummy_hcd.c b/drivers/usb/gadget/udc/dummy_hcd.c index 4a78536644ba..5e3cbe0338bf 100644 --- a/drivers/usb/gadget/udc/dummy_hcd.c +++ b/drivers/usb/gadget/udc/dummy_hcd.c @@ -278,6 +278,7 @@ struct dummy { unsigned ints_enabled:1; unsigned udc_suspended:1; unsigned pullup:1; + unsigned fifo_req_busy:1; /* * HOST side support @@ -329,6 +330,26 @@ static inline struct dummy *gadget_dev_to_dummy(struct device *dev) /* DEVICE/GADGET SIDE UTILITY ROUTINES */ +/* + * Give back a gadget request with dum->lock dropped around the callback. + * If @req is the shared fifo_req, clear fifo_req_busy afterward: the flag + * was set in dummy_queue() when the shared request was taken and must stay + * set until its completion callback has returned; list_del_init() alone + * makes the request look idle while the callback is still running. + * Caller holds dum->lock and has already done list_del_init() + status. + */ +static void dummy_giveback(struct dummy *dum, struct usb_ep *_ep, + struct dummy_request *req) +{ + bool fifo = req == &dum->fifo_req; + + spin_unlock(&dum->lock); + usb_gadget_giveback_request(_ep, &req->req); + spin_lock(&dum->lock); + if (fifo) + dum->fifo_req_busy = 0; +} + /* called with spinlock held */ static void nuke(struct dummy *dum, struct dummy_ep *ep) { @@ -339,9 +360,7 @@ static void nuke(struct dummy *dum, struct dummy_ep *ep) list_del_init(&req->queue); req->req.status = -ESHUTDOWN; - spin_unlock(&dum->lock); - usb_gadget_giveback_request(&ep->ep, &req->req); - spin_lock(&dum->lock); + dummy_giveback(dum, &ep->ep, req); } } @@ -728,10 +747,11 @@ static int dummy_queue(struct usb_ep *_ep, struct usb_request *_req, /* implement an emulated single-request FIFO */ if (ep->desc && (ep->desc->bEndpointAddress & USB_DIR_IN) && - list_empty(&dum->fifo_req.queue) && + !dum->fifo_req_busy && list_empty(&ep->queue) && _req->length <= FIFO_SIZE) { req = &dum->fifo_req; + dum->fifo_req_busy = 1; req->req = *_req; req->req.buf = dum->fifo_buf; memcpy(dum->fifo_buf, _req->buf, _req->length); @@ -785,9 +805,7 @@ static int dummy_dequeue(struct usb_ep *_ep, struct usb_request *_req) dev_dbg(udc_dev(dum), "dequeued req %p from %s, len %d buf %p\n", req, _ep->name, _req->length, _req->buf); - spin_unlock(&dum->lock); - usb_gadget_giveback_request(_ep, _req); - spin_lock(&dum->lock); + dummy_giveback(dum, _ep, req); } spin_unlock_irqrestore(&dum->lock, flags); return retval; @@ -1523,9 +1541,7 @@ static int transfer(struct dummy_hcd *dum_hcd, struct urb *urb, if (req->req.status != -EINPROGRESS) { list_del_init(&req->queue); - spin_unlock(&dum->lock); - usb_gadget_giveback_request(&ep->ep, &req->req); - spin_lock(&dum->lock); + dummy_giveback(dum, &ep->ep, req); /* requests might have been unlinked... */ rescan = 1; @@ -1910,9 +1926,7 @@ static enum hrtimer_restart dummy_timer(struct hrtimer *t) dev_dbg(udc_dev(dum), "stale req = %p\n", req); - spin_unlock(&dum->lock); - usb_gadget_giveback_request(&ep->ep, &req->req); - spin_lock(&dum->lock); + dummy_giveback(dum, &ep->ep, req); ep->already_seen = 0; goto restart; } diff --git a/drivers/usb/gadget/udc/fsl_udc_core.c b/drivers/usb/gadget/udc/fsl_udc_core.c index 4dea8bc30cf6..ed311f0d74b5 100644 --- a/drivers/usb/gadget/udc/fsl_udc_core.c +++ b/drivers/usb/gadget/udc/fsl_udc_core.c @@ -184,7 +184,7 @@ __acquires(ep->udc->lock) usb_gadget_unmap_request(&ep->udc->gadget, &req->req, ep_is_in(ep)); if (status && (status != -ESHUTDOWN)) - dev_vdbg(&udc->gadget.dev, "complete %s req %p stat %d len %u/%u\n", + dev_vdbg(udc->dev, "complete %s req %p stat %d len %u/%u\n", ep->ep.name, &req->req, status, req->req.actual, req->req.length); @@ -286,7 +286,7 @@ static int dr_controller_setup(struct fsl_udc *udc) timeout = jiffies + FSL_UDC_RESET_TIMEOUT; while (fsl_readl(&dr_regs->usbcmd) & USB_CMD_CTRL_RESET) { if (time_after(jiffies, timeout)) { - dev_err(&udc->gadget.dev, "udc reset timeout!\n"); + dev_err(udc->dev, "udc reset timeout!\n"); return -ETIMEDOUT; } cpu_relax(); @@ -309,7 +309,7 @@ static int dr_controller_setup(struct fsl_udc *udc) tmp &= USB_EP_LIST_ADDRESS_MASK; fsl_writel(tmp, &dr_regs->endpointlistaddr); - dev_vdbg(&udc->gadget.dev, + dev_vdbg(udc->dev, "vir[qh_base] is %p phy[qh_base] is 0x%8x reg is 0x%8x\n", udc->ep_qh, (int)tmp, fsl_readl(&dr_regs->endpointlistaddr)); @@ -500,7 +500,7 @@ static void struct_ep_qh_setup(struct fsl_udc *udc, unsigned char ep_num, tmp = max_pkt_len << EP_QUEUE_HEAD_MAX_PKT_LEN_POS; break; default: - dev_vdbg(&udc->gadget.dev, "error ep type is %d\n", ep_type); + dev_vdbg(udc->dev, "error ep type is %d\n", ep_type); return; } if (zlt) @@ -613,7 +613,7 @@ static int fsl_ep_enable(struct usb_ep *_ep, spin_unlock_irqrestore(&udc->lock, flags); retval = 0; - dev_vdbg(&udc->gadget.dev, "enabled %s (ep%d%s) maxpacket %d\n", + dev_vdbg(udc->dev, "enabled %s (ep%d%s) maxpacket %d\n", ep->ep.name, ep->ep.desc->bEndpointAddress & 0x0f, (desc->bEndpointAddress & USB_DIR_IN) ? "in" : "out", max); @@ -634,13 +634,8 @@ static int fsl_ep_disable(struct usb_ep *_ep) int ep_num; ep = container_of(_ep, struct fsl_ep, ep); - if (!_ep || !ep->ep.desc) { - /* - * dev_vdbg(&udc->gadget.dev, "%s not enabled\n", - * _ep ? ep->ep.name : NULL); - */ + if (!_ep || !ep->ep.desc) return -EINVAL; - } /* disable ep on controller */ ep_num = ep_index(ep); @@ -664,7 +659,7 @@ static int fsl_ep_disable(struct usb_ep *_ep) ep->stopped = 1; spin_unlock_irqrestore(&udc->lock, flags); - dev_vdbg(&udc->gadget.dev, "disabled %s OK\n", _ep->name); + dev_vdbg(udc->dev, "disabled %s OK\n", _ep->name); return 0; } @@ -724,9 +719,6 @@ static void fsl_queue_td(struct fsl_ep *ep, struct fsl_req *req) { u32 temp, bitmask, tmp_stat; - /* dev_vdbg(&udc->gadget.dev, "QH addr Register 0x%8x\n", dr_regs->endpointlistaddr); - dev_vdbg(&udc->gadget.dev, "ep_qh[%d] addr is 0x%8x\n", i, (u32)&(ep->udc->ep_qh[i])); */ - bitmask = ep_is_in(ep) ? (1 << (ep_index(ep) + 16)) : (1 << (ep_index(ep))); @@ -813,7 +805,7 @@ static struct ep_td_struct *fsl_build_dtd(struct fsl_req *req, unsigned *length, *is_last = 0; if ((*is_last) == 0) - dev_vdbg(&udc_controller->gadget.dev, "multi-dtd request!\n"); + dev_vdbg(udc_controller->dev, "multi-dtd request!\n"); /* Fill in the transfer size; set active bit */ swap_temp = ((*length << DTD_LENGTH_BIT_POS) | DTD_STATUS_ACTIVE); @@ -825,7 +817,7 @@ static struct ep_td_struct *fsl_build_dtd(struct fsl_req *req, unsigned *length, mb(); - dev_vdbg(&udc_controller->gadget.dev, "length = %d address= 0x%x\n", *length, (int)*dma); + dev_vdbg(udc_controller->dev, "length = %d address= 0x%x\n", *length, (int)*dma); return dtd; } @@ -876,11 +868,11 @@ fsl_ep_queue(struct usb_ep *_ep, struct usb_request *_req, gfp_t gfp_flags) /* catch various bogus parameters */ if (!_req || !req->req.complete || !req->req.buf || !list_empty(&req->queue)) { - dev_vdbg(&udc->gadget.dev, "%s, bad params\n", __func__); + dev_vdbg(udc->dev, "%s, bad params\n", __func__); return -EINVAL; } if (unlikely(!ep->ep.desc)) { - dev_vdbg(&udc->gadget.dev, "%s, bad ep\n", __func__); + dev_vdbg(udc->dev, "%s, bad ep\n", __func__); return -EINVAL; } if (usb_endpoint_xfer_isoc(ep->ep.desc)) { @@ -1040,7 +1032,7 @@ static int fsl_ep_set_halt(struct usb_ep *_ep, int value) udc->ep0_dir = 0; } out: - dev_vdbg(&udc->gadget.dev, "%s %s halt stat %d\n", ep->ep.name, + dev_vdbg(udc->dev, "%s %s halt stat %d\n", ep->ep.name, value ? "set" : "clear", status); return status; @@ -1109,7 +1101,7 @@ static void fsl_ep_fifo_flush(struct usb_ep *_ep) /* Wait until flush complete */ while (fsl_readl(&dr_regs->endptflush)) { if (time_after(jiffies, timeout)) { - dev_err(&udc_controller->gadget.dev, + dev_err(udc_controller->dev, "ep flush timeout\n"); return; } @@ -1182,7 +1174,7 @@ static int fsl_vbus_session(struct usb_gadget *gadget, int is_active) udc = container_of(gadget, struct fsl_udc, gadget); spin_lock_irqsave(&udc->lock, flags); - dev_vdbg(&gadget->dev, "VBUS %s\n", str_on_off(is_active)); + dev_vdbg(udc->dev, "VBUS %s\n", str_on_off(is_active)); udc->vbus_active = (is_active != 0); if (can_pullup(udc)) fsl_writel((fsl_readl(&dr_regs->usbcmd) | USB_CMD_RUN_STOP), @@ -1548,7 +1540,7 @@ static void ep0_req_complete(struct fsl_udc *udc, struct fsl_ep *ep0, udc->ep0_state = WAIT_FOR_SETUP; break; case WAIT_FOR_SETUP: - dev_err(&udc->gadget.dev, "Unexpected ep0 packets\n"); + dev_err(udc->dev, "Unexpected ep0 packets\n"); break; default: ep0stall(udc); @@ -1617,7 +1609,7 @@ static int process_ep_req(struct fsl_udc *udc, int pipe, errors = hc32_to_cpu(curr_td->size_ioc_sts); if (errors & DTD_ERROR_MASK) { if (errors & DTD_STATUS_HALTED) { - dev_err(&udc->gadget.dev, "dTD error %08x QH=%d\n", errors, pipe); + dev_err(udc->dev, "dTD error %08x QH=%d\n", errors, pipe); /* Clear the errors and Halt condition */ tmp = hc32_to_cpu(curr_qh->size_ioc_int_sts); tmp &= ~errors; @@ -1628,26 +1620,26 @@ static int process_ep_req(struct fsl_udc *udc, int pipe, break; } if (errors & DTD_STATUS_DATA_BUFF_ERR) { - dev_vdbg(&udc->gadget.dev, "Transfer overflow\n"); + dev_vdbg(udc->dev, "Transfer overflow\n"); status = -EPROTO; break; } else if (errors & DTD_STATUS_TRANSACTION_ERR) { - dev_vdbg(&udc->gadget.dev, "ISO error\n"); + dev_vdbg(udc->dev, "ISO error\n"); status = -EILSEQ; break; } else - dev_err(&udc->gadget.dev, + dev_err(udc->dev, "Unknown error has occurred (0x%x)!\n", errors); } else if (hc32_to_cpu(curr_td->size_ioc_sts) & DTD_STATUS_ACTIVE) { - dev_vdbg(&udc->gadget.dev, "Request not complete\n"); + dev_vdbg(udc->dev, "Request not complete\n"); status = REQ_UNCOMPLETE; return status; } else if (remaining_length) { if (direction) { - dev_vdbg(&udc->gadget.dev, + dev_vdbg(udc->dev, "Transmit dTD remaining length not zero\n"); status = -EPROTO; break; @@ -1655,8 +1647,7 @@ static int process_ep_req(struct fsl_udc *udc, int pipe, break; } } else { - dev_vdbg(&udc->gadget.dev, - "dTD transmitted successful\n"); + dev_vdbg(udc->dev, "dTD transmitted successful\n"); } if (j != curr_req->dtd_count - 1) @@ -1699,7 +1690,7 @@ static void dtd_complete_irq(struct fsl_udc *udc) /* If the ep is configured */ if (!curr_ep->ep.name) { - dev_warn(&udc->gadget.dev, "Invalid EP?\n"); + dev_warn(udc->dev, "Invalid EP?\n"); continue; } @@ -1708,7 +1699,7 @@ static void dtd_complete_irq(struct fsl_udc *udc) queue) { status = process_ep_req(udc, i, curr_req); - dev_vdbg(&udc->gadget.dev, + dev_vdbg(udc->dev, "status of process_ep_req= %d, ep = %d\n", status, ep_num); if (status == REQ_UNCOMPLETE) @@ -1829,7 +1820,7 @@ static void reset_irq(struct fsl_udc *udc) while (fsl_readl(&dr_regs->endpointprime)) { /* Wait until all endptprime bits cleared */ if (time_after(jiffies, timeout)) { - dev_err(&udc->gadget.dev, "Timeout for reset\n"); + dev_err(udc->dev, "Timeout for reset\n"); break; } cpu_relax(); @@ -1839,7 +1830,7 @@ static void reset_irq(struct fsl_udc *udc) fsl_writel(0xffffffff, &dr_regs->endptflush); if (fsl_readl(&dr_regs->portsc1) & PORTSCX_PORT_RESET) { - dev_vdbg(&udc->gadget.dev, "Bus reset\n"); + dev_vdbg(udc->dev, "Bus reset\n"); /* Bus is reseting */ udc->bus_reset = 1; /* Reset all the queues, include XD, dTD, EP queue @@ -1847,7 +1838,7 @@ static void reset_irq(struct fsl_udc *udc) reset_queues(udc, true); udc->usb_state = USB_STATE_DEFAULT; } else { - dev_vdbg(&udc->gadget.dev, "Controller reset\n"); + dev_vdbg(udc->dev, "Controller reset\n"); /* initialize usb hw reg except for regs for EP, not * touch usbintr reg */ dr_controller_setup(udc); @@ -1881,7 +1872,7 @@ static irqreturn_t fsl_udc_irq(int irq, void *_udc) /* Clear notification bits */ fsl_writel(irq_src, &dr_regs->usbsts); - /* dev_vdbg(&udc->gadget.dev, "irq_src [0x%8x]", irq_src); */ + /* dev_vdbg(udc->dev, "irq_src [0x%8x]", irq_src); */ /* Need to resume? */ if (udc->usb_state == USB_STATE_SUSPENDED) @@ -1890,7 +1881,7 @@ static irqreturn_t fsl_udc_irq(int irq, void *_udc) /* USB Interrupt */ if (irq_src & USB_STS_INT) { - dev_vdbg(&udc->gadget.dev, "Packet int\n"); + dev_vdbg(udc->dev, "Packet int\n"); /* Setup package, we only support ep0 as control ep */ if (fsl_readl(&dr_regs->endptsetupstat) & EP_SETUP_STATUS_EP0) { tripwire_handler(udc, 0, @@ -1919,7 +1910,7 @@ static irqreturn_t fsl_udc_irq(int irq, void *_udc) /* Reset Received */ if (irq_src & USB_STS_RESET) { - dev_vdbg(&udc->gadget.dev, "reset int\n"); + dev_vdbg(udc->dev, "reset int\n"); reset_irq(udc); status = IRQ_HANDLED; } @@ -1931,7 +1922,7 @@ static irqreturn_t fsl_udc_irq(int irq, void *_udc) } if (irq_src & (USB_STS_ERR | USB_STS_SYS_ERR)) { - dev_vdbg(&udc->gadget.dev, "Error IRQ %x\n", irq_src); + dev_vdbg(udc->dev, "Error IRQ %x\n", irq_src); } spin_unlock_irqrestore(&udc->lock, flags); @@ -1967,7 +1958,7 @@ static int fsl_udc_start(struct usb_gadget *g, udc_controller->transceiver->otg, &udc_controller->gadget); if (retval < 0) { - dev_err(&udc_controller->gadget.dev, "can't bind to transceiver\n"); + dev_err(udc_controller->dev, "can't bind to transceiver\n"); udc_controller->driver = NULL; return retval; } @@ -2252,7 +2243,7 @@ static int struct_udc_setup(struct fsl_udc *udc, udc->eps = kcalloc(udc->max_ep, sizeof(struct fsl_ep), GFP_KERNEL); if (!udc->eps) { - dev_err(&udc->gadget.dev, "kmalloc udc endpoint status failed\n"); + dev_err(udc->dev, "kmalloc udc endpoint status failed\n"); goto eps_alloc_failed; } @@ -2267,7 +2258,7 @@ static int struct_udc_setup(struct fsl_udc *udc, udc->ep_qh = dma_alloc_coherent(&pdev->dev, size, &udc->ep_qh_dma, GFP_KERNEL); if (!udc->ep_qh) { - dev_err(&udc->gadget.dev, "malloc QHs for udc failed\n"); + dev_err(udc->dev, "malloc QHs for udc failed\n"); goto ep_queue_alloc_failed; } @@ -2278,14 +2269,14 @@ static int struct_udc_setup(struct fsl_udc *udc, udc->status_req = container_of(fsl_alloc_request(NULL, GFP_KERNEL), struct fsl_req, req); if (!udc->status_req) { - dev_err(&udc->gadget.dev, "kzalloc for udc status request failed\n"); + dev_err(udc->dev, "kzalloc for udc status request failed\n"); goto udc_status_alloc_failed; } /* allocate a small amount of memory to get valid address */ udc->status_req->req.buf = kmalloc(8, GFP_KERNEL); if (!udc->status_req->req.buf) { - dev_err(&udc->gadget.dev, "kzalloc for udc request buffer failed\n"); + dev_err(udc->dev, "kzalloc for udc request buffer failed\n"); goto udc_req_buf_alloc_failed; } @@ -2373,6 +2364,7 @@ static int fsl_udc_probe(struct platform_device *pdev) if (udc_controller == NULL) return -ENOMEM; + udc_controller->dev = &pdev->dev; pdata = dev_get_platdata(&pdev->dev); udc_controller->pdata = pdata; spin_lock_init(&udc_controller->lock); @@ -2382,7 +2374,7 @@ static int fsl_udc_probe(struct platform_device *pdev) if (pdata->operating_mode == FSL_USB2_DR_OTG) { udc_controller->transceiver = usb_get_phy(USB_PHY_TYPE_USB2); if (IS_ERR_OR_NULL(udc_controller->transceiver)) { - dev_err(&udc_controller->gadget.dev, "Can't find OTG driver!\n"); + dev_err(&pdev->dev, "Can't find OTG driver!\n"); ret = -ENODEV; goto err_kfree; } @@ -2398,7 +2390,7 @@ static int fsl_udc_probe(struct platform_device *pdev) if (pdata->operating_mode == FSL_USB2_DR_DEVICE) { if (!request_mem_region(res->start, resource_size(res), driver_name)) { - dev_err(&udc_controller->gadget.dev, "request mem region for %s failed\n", pdev->name); + dev_err(&pdev->dev, "failed to request mem region\n"); ret = -EBUSY; goto err_kfree; } @@ -2429,7 +2421,7 @@ static int fsl_udc_probe(struct platform_device *pdev) /* Read Device Controller Capability Parameters register */ dccparams = fsl_readl(&dr_regs->dccparams); if (!(dccparams & DCCPARAMS_DC)) { - dev_err(&udc_controller->gadget.dev, "This SOC doesn't support device role\n"); + dev_err(&pdev->dev, "This SOC doesn't support device role\n"); ret = -ENODEV; goto err_exit; } @@ -2447,14 +2439,14 @@ static int fsl_udc_probe(struct platform_device *pdev) ret = request_irq(udc_controller->irq, fsl_udc_irq, IRQF_SHARED, driver_name, udc_controller); if (ret != 0) { - dev_err(&udc_controller->gadget.dev, "cannot request irq %d err %d\n", + dev_err(&pdev->dev, "cannot request irq %d err %d\n", udc_controller->irq, ret); goto err_exit; } /* Initialize the udc structure including QH member and other member */ if (struct_udc_setup(udc_controller, pdev)) { - dev_err(&udc_controller->gadget.dev, "Can't initialize udc data structure\n"); + dev_err(&pdev->dev, "Can't initialize udc data structure\n"); ret = -ENOMEM; goto err_free_irq; } @@ -2474,7 +2466,6 @@ static int fsl_udc_probe(struct platform_device *pdev) udc_controller->gadget.name = driver_name; /* Setup gadget.dev and register with kernel */ - dev_set_name(&udc_controller->gadget.dev, "gadget"); udc_controller->gadget.dev.of_node = pdev->dev.of_node; if (!IS_ERR_OR_NULL(udc_controller->transceiver)) diff --git a/drivers/usb/gadget/udc/fsl_usb2_udc.h b/drivers/usb/gadget/udc/fsl_usb2_udc.h index cc1756f3e89d..53922bc58ca0 100644 --- a/drivers/usb/gadget/udc/fsl_usb2_udc.h +++ b/drivers/usb/gadget/udc/fsl_usb2_udc.h @@ -470,6 +470,7 @@ struct fsl_ep { #define EP_DIR_OUT 0 struct fsl_udc { + struct device *dev; struct usb_gadget gadget; struct usb_gadget_driver *driver; struct fsl_usb2_platform_data *pdata; diff --git a/drivers/usb/gadget/udc/snps_udc_core.c b/drivers/usb/gadget/udc/snps_udc_core.c index 373942ceb076..2f790cc65257 100644 --- a/drivers/usb/gadget/udc/snps_udc_core.c +++ b/drivers/usb/gadget/udc/snps_udc_core.c @@ -3133,7 +3133,6 @@ int udc_probe(struct udc *dev) /* device struct setup */ dev->gadget.ops = &udc_ops; - dev_set_name(&dev->gadget.dev, "gadget"); dev->gadget.name = name; dev->gadget.max_speed = USB_SPEED_HIGH; diff --git a/drivers/usb/host/xhci-pci.c b/drivers/usb/host/xhci-pci.c index f67a4d956204..734f7e741583 100644 --- a/drivers/usb/host/xhci-pci.c +++ b/drivers/usb/host/xhci-pci.c @@ -448,6 +448,7 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci) if (pdev->vendor == PCI_VENDOR_ID_VIA && pdev->device == PCI_DEVICE_ID_VIA_VL805) { xhci->quirks |= XHCI_LPM_SUPPORT; xhci->quirks |= XHCI_TRB_OVERFETCH; + xhci->dma_mask_bits = 36; } if (pdev->vendor == PCI_VENDOR_ID_ASMEDIA && diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c index a06758addbf8..23b104c2956c 100644 --- a/drivers/usb/host/xhci.c +++ b/drivers/usb/host/xhci.c @@ -5459,6 +5459,7 @@ int xhci_gen_setup(struct usb_hcd *hcd, xhci_get_quirks_t get_quirks) if (xhci->hci_version > 0x100) xhci->hcc_params2 = readl(&xhci->cap_regs->hcc_params2); + xhci->dma_mask_bits = 64; /* xhci-plat or xhci-pci might have set max_interrupters already */ if ((!xhci->max_interrupters) || xhci->max_interrupters > HCS_MAX_INTRS(xhci->hcs_params1)) @@ -5505,12 +5506,16 @@ int xhci_gen_setup(struct usb_hcd *hcd, xhci_get_quirks_t get_quirks) if (xhci->quirks & XHCI_NO_64BIT_SUPPORT) xhci->hcc_params &= ~BIT(0); - /* Set dma_mask and coherent_dma_mask to 64-bits, - * if xHC supports 64-bit addressing */ + /* + * Set dma_mask and coherent_dma_mask to 64-bits if xHC supports + * 64-bit addressing, unless a controller-specific quirk callback + * limits the usable address width. + */ if (HCC_64BIT_ADDR(xhci->hcc_params) && - !dma_set_mask(dev, DMA_BIT_MASK(64))) { - xhci_dbg(xhci, "Enabling 64-bit DMA addresses.\n"); - dma_set_coherent_mask(dev, DMA_BIT_MASK(64)); + !dma_set_mask(dev, DMA_BIT_MASK(xhci->dma_mask_bits))) { + xhci_dbg(xhci, "Enabling %u-bit DMA addresses.\n", + xhci->dma_mask_bits); + dma_set_coherent_mask(dev, DMA_BIT_MASK(xhci->dma_mask_bits)); } else { /* * This is to avoid error in cases where a 32-bit USB diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h index 58a51f09cceb..4cd4cb0e431d 100644 --- a/drivers/usb/host/xhci.h +++ b/drivers/usb/host/xhci.h @@ -1524,6 +1524,7 @@ struct xhci_hcd { /* imod_interval in ns (I * 250ns) */ u32 imod_interval; u32 page_size; + unsigned int dma_mask_bits; /* MSI-X/MSI vectors */ int nvecs; /* optional clocks */ diff --git a/drivers/usb/musb/omap2430.c b/drivers/usb/musb/omap2430.c index aadabc415145..77133ee77063 100644 --- a/drivers/usb/musb/omap2430.c +++ b/drivers/usb/musb/omap2430.c @@ -455,7 +455,6 @@ static int omap2430_probe(struct platform_device *pdev) dev_err(&pdev->dev, "failed to register musb device\n"); goto err_disable_rpm; } - of_node_put(np); return 0; @@ -465,7 +464,6 @@ static int omap2430_probe(struct platform_device *pdev) if (!IS_ERR(glue->control_otghs)) put_device(glue->control_otghs); err_put_musb: - of_node_put(np); platform_device_put(musb); return ret; diff --git a/drivers/usb/serial/ftdi_sio.c b/drivers/usb/serial/ftdi_sio.c index ceddfb1852e1..8eb50640e8b7 100644 --- a/drivers/usb/serial/ftdi_sio.c +++ b/drivers/usb/serial/ftdi_sio.c @@ -1075,6 +1075,8 @@ static const struct usb_device_id id_table_combined[] = { { USB_DEVICE_INTERFACE_NUMBER(ALTERA_VID, ALTERA_UB3_602E_PID, 3) }, /* Abacus Electrics */ { USB_DEVICE(FTDI_VID, ABACUS_OPTICAL_PROBE_PID) }, + /* Endress+Hauser AG devices */ + { USB_DEVICE(FTDI_VID, FTDI_EH_FXA291_PID) }, { } /* Terminating entry */ }; diff --git a/drivers/usb/serial/ftdi_sio_ids.h b/drivers/usb/serial/ftdi_sio_ids.h index 6c76cfebfd0e..9c83c17853c8 100644 --- a/drivers/usb/serial/ftdi_sio_ids.h +++ b/drivers/usb/serial/ftdi_sio_ids.h @@ -313,6 +313,11 @@ #define FTDI_ELV_UDF77_PID 0xFB5E /* USB DCF Funkuhr (UDF 77) */ #define FTDI_ELV_UIO88_PID 0xFB5F /* USB-I/O Interface (UIO 88) */ +/* + * Endress+Hauser AG product ids (FTDI_VID) + */ +#define FTDI_EH_FXA291_PID 0xE510 + /* * EVER Eco Pro UPS (http://www.ever.com.pl/) */ diff --git a/drivers/usb/serial/io_edgeport.c b/drivers/usb/serial/io_edgeport.c index ad73040b30c8..9b8fb8467b53 100644 --- a/drivers/usb/serial/io_edgeport.c +++ b/drivers/usb/serial/io_edgeport.c @@ -646,7 +646,8 @@ static void edge_interrupt_callback(struct urb *urb) if (edge_port && edge_port->open) { spin_lock_irqsave(&edge_port->ep_lock, flags); - edge_port->txCredits += txCredits; + edge_port->txCredits = min(edge_port->txCredits + txCredits, + edge_port->maxTxCredits); spin_unlock_irqrestore(&edge_port->ep_lock, flags); dev_dbg(dev, "%s - txcredits for port%d = %d\n", diff --git a/drivers/usb/serial/keyspan_pda.c b/drivers/usb/serial/keyspan_pda.c index f800bf5ac1de..e500ffd15085 100644 --- a/drivers/usb/serial/keyspan_pda.c +++ b/drivers/usb/serial/keyspan_pda.c @@ -35,6 +35,8 @@ struct keyspan_pda_private { struct work_struct unthrottle_work; struct usb_serial *serial; struct usb_serial_port *port; + bool throttled; + bool throttle_req; }; static int keyspan_pda_write_start(struct usb_serial_port *port); @@ -150,6 +152,7 @@ static void keyspan_pda_rx_interrupt(struct urb *urb) int retval; int status = urb->status; struct keyspan_pda_private *priv; + bool throttled = false; unsigned long flags; priv = usb_get_serial_port_data(port); @@ -211,16 +214,24 @@ static void keyspan_pda_rx_interrupt(struct urb *urb) } exit: - retval = usb_submit_urb(urb, GFP_ATOMIC); - if (retval) - dev_err(&port->dev, - "%s - usb_submit_urb failed with result %d\n", - __func__, retval); + spin_lock_irqsave(&port->lock, flags); + if (priv->throttle_req) { + priv->throttled = true; + throttled = true; + } + spin_unlock_irqrestore(&port->lock, flags); + + if (!throttled) { + retval = usb_submit_urb(urb, GFP_ATOMIC); + if (retval) + dev_err(&port->dev, "failed to resubmit in urb: %d\n", retval); + } } static void keyspan_pda_rx_throttle(struct tty_struct *tty) { struct usb_serial_port *port = tty->driver_data; + struct keyspan_pda_private *priv = usb_get_serial_port_data(port); /* * Stop receiving characters. We just turn off the URB request, and @@ -230,16 +241,29 @@ static void keyspan_pda_rx_throttle(struct tty_struct *tty) * send an XOFF, although it might make sense to foist that off upon * the device too. */ - usb_kill_urb(port->interrupt_in_urb); + spin_lock_irq(&port->lock); + priv->throttle_req = true; + spin_unlock_irq(&port->lock); } static void keyspan_pda_rx_unthrottle(struct tty_struct *tty) { struct usb_serial_port *port = tty->driver_data; + struct keyspan_pda_private *priv = usb_get_serial_port_data(port); + bool throttled; + int ret; - /* just restart the receive interrupt URB */ - if (usb_submit_urb(port->interrupt_in_urb, GFP_KERNEL)) - dev_dbg(&port->dev, "usb_submit_urb(read urb) failed\n"); + spin_lock_irq(&port->lock); + throttled = priv->throttled; + priv->throttled = false; + priv->throttle_req = false; + spin_unlock_irq(&port->lock); + + if (throttled) { + ret = usb_submit_urb(port->interrupt_in_urb, GFP_KERNEL); + if (ret) + dev_err(&port->dev, "failed to submit in urb: %d\n", ret); + } } static speed_t keyspan_pda_setbaud(struct usb_serial *serial, speed_t baud) @@ -579,6 +603,8 @@ static int keyspan_pda_open(struct tty_struct *tty, spin_lock_irq(&port->lock); priv->tx_room = rc; + priv->throttled = false; + priv->throttle_req = false; spin_unlock_irq(&port->lock); rc = usb_submit_urb(port->interrupt_in_urb, GFP_KERNEL); diff --git a/drivers/usb/serial/option.c b/drivers/usb/serial/option.c index b4b58e48e225..89624f38fec2 100644 --- a/drivers/usb/serial/option.c +++ b/drivers/usb/serial/option.c @@ -2497,6 +2497,7 @@ static const struct usb_device_id option_ids[] = { .driver_info = RSVD(5) }, { USB_DEVICE_INTERFACE_CLASS(0x33f8, 0x1003, 0xff), /* Rolling RW135R-GL (laptop MBIM) */ .driver_info = RSVD(5) }, + { USB_DEVICE_INTERFACE_CLASS(0x3466, 0x3301, 0xff) }, /* TDTECH MT5710-CN */ { USB_DEVICE_AND_INTERFACE_INFO(0x3731, 0x0100, 0xff, 0xff, 0x30) }, /* NetPrisma LCUK54-WWD for Global */ { USB_DEVICE_AND_INTERFACE_INFO(0x3731, 0x0100, 0xff, 0x00, 0x40) }, { USB_DEVICE_AND_INTERFACE_INFO(0x3731, 0x0100, 0xff, 0xff, 0x40) }, diff --git a/drivers/usb/storage/unusual_devs.h b/drivers/usb/storage/unusual_devs.h index 255968f9ca42..ac22fa318734 100644 --- a/drivers/usb/storage/unusual_devs.h +++ b/drivers/usb/storage/unusual_devs.h @@ -395,6 +395,13 @@ UNUSUAL_DEV( 0x04b3, 0x4001, 0x0110, 0x0110, USB_SC_DEVICE, USB_PR_CB, NULL, US_FL_MAX_SECTORS_MIN), +/* Reported by Ai Chao */ +UNUSUAL_DEV( 0x04b4, 0xb708, 0x0000, 0xffff, + "Longmai Technologies", + "USB Key", + USB_SC_SCSI, USB_PR_BULK, NULL, + US_FL_NO_ATA_1X), + /* * Reported by Simon Levitt * This entry needs Sub and Proto fields diff --git a/drivers/usb/typec/ucsi/ucsi.c b/drivers/usb/typec/ucsi/ucsi.c index b4b0c46d2fa4..01a15ab5a734 100644 --- a/drivers/usb/typec/ucsi/ucsi.c +++ b/drivers/usb/typec/ucsi/ucsi.c @@ -487,6 +487,129 @@ static int ucsi_register_altmode(struct ucsi_connector *con, return ret; } +static void ucsi_dump_duplicate_altmode(struct ucsi_connector *con, + u8 recipient, u16 svid, + u32 existing_vdo, u32 new_vdo, + int offset) +{ + static const char * const recipient_names[] = { + [UCSI_RECIPIENT_CON] = "port", + [UCSI_RECIPIENT_SOP] = "partner", + [UCSI_RECIPIENT_SOP_P] = "plug", + [UCSI_RECIPIENT_SOP_PP] = "cable plug prime", + }; + + dev_warn(con->ucsi->dev, + "con%d: Firmware bug: duplicate %s altmode SVID 0x%04x at offset %d, ignoring but please contact the BIOS vendor to fix this issue.\n", + con->num, recipient_names[recipient], svid, offset); + + if (existing_vdo != new_vdo) + dev_warn(con->ucsi->dev, + "con%d: VDO mismatch: 0x%08x vs 0x%08x\n", + con->num, existing_vdo, new_vdo); +} + +/* Count altmodes in @altmodes that advertise @svid. */ +static int ucsi_altmode_count_svid(struct typec_altmode **altmodes, u16 svid) +{ + int count = 0; + int k; + + for (k = 0; k < UCSI_MAX_ALTMODES; k++) { + if (!altmodes[k]) + break; + if (altmodes[k]->svid == svid) + count++; + } + + return count; +} + +/* + * Check if an altmode is a duplicate. Some firmware implementations + * incorrectly return the same altmode multiple times, causing sysfs errors. + * Returns true if the altmode should be skipped. + * + * The matching rules differ by recipient: + * + * - UCSI_RECIPIENT_CON (port) and UCSI_RECIPIENT_SOP_P (plug): + * Two altmodes with identical SVID and VDO are byte-for-byte duplicates + * and the second has no observable function. Drop them. + * + * - UCSI_RECIPIENT_SOP (partner): + * The typec class binds each partner altmode to a port altmode of the + * same SVID via altmode_match()/device_find_child(), which returns the + * first port altmode with a matching SVID. If the partner advertises + * more altmodes for SVID X than the port advertises, the surplus + * partner altmode(s) collapse onto an already-paired port altmode and + * trigger a "duplicate filename .../partner" sysfs error during + * typec_altmode_create_links(). Use the port-side altmode count for + * SVID X as the authoritative cap and reject any partner altmode that + * would exceed it. This preserves legitimate multi-Mode partner + * altmodes (e.g. vendor SVIDs that the port really does advertise + * twice) while filtering the firmware-generated duplicates that have + * no port counterpart. + */ +static bool ucsi_altmode_is_duplicate(struct ucsi_connector *con, u8 recipient, + const struct ucsi_altmode *alt_batch, int batch_idx, + u16 svid, u32 vdo, int offset) +{ + struct typec_altmode **altmodes; + int port_count, partner_count; + int k; + + /* Check for duplicates within the current batch first */ + for (k = 0; k < batch_idx; k++) { + if (alt_batch[k].svid == svid && alt_batch[k].mid == vdo) { + ucsi_dump_duplicate_altmode(con, recipient, svid, + vdo, vdo, offset); + return true; + } + } + + switch (recipient) { + case UCSI_RECIPIENT_SOP: + /* + * Cap partner altmodes per SVID by the port-side count: + * any further partner altmode for that SVID would alias an + * already-paired port altmode and break typec sysfs. + */ + port_count = ucsi_altmode_count_svid(con->port_altmode, svid); + partner_count = ucsi_altmode_count_svid(con->partner_altmode, + svid); + if (port_count && partner_count >= port_count) { + ucsi_dump_duplicate_altmode(con, recipient, svid, + con->partner_altmode[partner_count - 1]->vdo, + vdo, offset); + return true; + } + return false; + case UCSI_RECIPIENT_CON: + altmodes = con->port_altmode; + break; + case UCSI_RECIPIENT_SOP_P: + altmodes = con->plug_altmode; + break; + default: + return false; + } + + /* CON and SOP_P: drop only exact SVID+VDO duplicates. */ + for (k = 0; k < UCSI_MAX_ALTMODES; k++) { + if (!altmodes[k]) + break; + + if (altmodes[k]->svid != svid || altmodes[k]->vdo != vdo) + continue; + + ucsi_dump_duplicate_altmode(con, recipient, svid, + altmodes[k]->vdo, vdo, offset); + return true; + } + + return false; +} + static int ucsi_register_altmodes_nvidia(struct ucsi_connector *con, u8 recipient) { @@ -541,19 +664,25 @@ ucsi_register_altmodes_nvidia(struct ucsi_connector *con, u8 recipient) /* now register altmodes */ for (i = 0; i < max_altmodes; i++) { - memset(&desc, 0, sizeof(desc)); - if (multi_dp) { - desc.svid = updated[i].svid; - desc.vdo = updated[i].mid; - } else { - desc.svid = orig[i].svid; - desc.vdo = orig[i].mid; - } - desc.roles = TYPEC_PORT_DRD; + struct ucsi_altmode *altmode_array = multi_dp ? updated : orig; - if (!desc.svid) + if (!altmode_array[i].svid) return 0; + /* + * Check for duplicates in current array and already + * registered altmodes. Skip if duplicate found. + */ + if (ucsi_altmode_is_duplicate(con, recipient, altmode_array, i, + altmode_array[i].svid, + altmode_array[i].mid, i)) + continue; + + memset(&desc, 0, sizeof(desc)); + desc.svid = altmode_array[i].svid; + desc.vdo = altmode_array[i].mid; + desc.roles = TYPEC_PORT_DRD; + ret = ucsi_register_altmode(con, &desc, recipient); if (ret) return ret; @@ -611,6 +740,15 @@ static int ucsi_register_altmodes(struct ucsi_connector *con, u8 recipient) if (!alt[j].svid) return 0; + /* + * Check for duplicates in current batch and already + * registered altmodes. Skip if duplicate found. + */ + if (ucsi_altmode_is_duplicate(con, recipient, alt, j, + alt[j].svid, alt[j].mid, + i - num + j)) + continue; + memset(&desc, 0, sizeof(desc)); desc.vdo = alt[j].mid; desc.svid = alt[j].svid; diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c index 4352b5cf74f0..ec743bed361c 100644 --- a/drivers/vdpa/vdpa_user/iova_domain.c +++ b/drivers/vdpa/vdpa_user/iova_domain.c @@ -124,7 +124,7 @@ static int vduse_domain_map_bounce_page(struct vduse_iova_domain *domain, if (!map->bounce_page) { head_map = &domain->bounce_maps[(iova & PAGE_MASK) >> BOUNCE_MAP_SHIFT]; if (!head_map->bounce_page) { - tmp_page = alloc_page(GFP_ATOMIC); + tmp_page = alloc_page(GFP_ATOMIC | __GFP_ZERO); if (!tmp_page) return -ENOMEM; if (cmpxchg(&head_map->bounce_page, NULL, tmp_page)) @@ -493,17 +493,15 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain, vduse_domain_free_iova(iovad, dma_addr, size); } -void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, - size_t size, dma_addr_t *dma_addr, - gfp_t flag) +dma_addr_t vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, + size_t size, void *orig) { struct iova_domain *iovad = &domain->consistent_iovad; unsigned long limit = domain->iova_limit; dma_addr_t iova = vduse_domain_alloc_iova(iovad, size, limit); - void *orig = alloc_pages_exact(size, flag); - if (!iova || !orig) - goto err; + if (!iova) + return DMA_MAPPING_ERROR; spin_lock(&domain->iotlb_lock); if (vduse_iotlb_add_range(domain, (u64)iova, (u64)iova + size - 1, @@ -514,27 +512,20 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, } spin_unlock(&domain->iotlb_lock); - *dma_addr = iova; + return iova; - return orig; err: - *dma_addr = DMA_MAPPING_ERROR; - if (orig) - free_pages_exact(orig, size); - if (iova) - vduse_domain_free_iova(iovad, iova, size); + vduse_domain_free_iova(iovad, iova, size); - return NULL; + return DMA_MAPPING_ERROR; } void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, - void *vaddr, dma_addr_t dma_addr, - unsigned long attrs) + dma_addr_t dma_addr, unsigned long attrs) { struct iova_domain *iovad = &domain->consistent_iovad; struct vhost_iotlb_map *map; struct vdpa_map_file *map_file; - phys_addr_t pa; spin_lock(&domain->iotlb_lock); map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr, @@ -546,12 +537,10 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, map_file = (struct vdpa_map_file *)map->opaque; fput(map_file->file); kfree(map_file); - pa = map->addr; vhost_iotlb_map_free(domain->iotlb, map); spin_unlock(&domain->iotlb_lock); vduse_domain_free_iova(iovad, dma_addr, size); - free_pages_exact(phys_to_virt(pa), size); } static vm_fault_t vduse_domain_mmap_fault(struct vm_fault *vmf) diff --git a/drivers/vdpa/vdpa_user/iova_domain.h b/drivers/vdpa/vdpa_user/iova_domain.h index 775cad5238f3..ead5ab477079 100644 --- a/drivers/vdpa/vdpa_user/iova_domain.h +++ b/drivers/vdpa/vdpa_user/iova_domain.h @@ -67,13 +67,11 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs); -void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, - size_t size, dma_addr_t *dma_addr, - gfp_t flag); +dma_addr_t vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, + size_t size, void *orig); void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, - void *vaddr, dma_addr_t dma_addr, - unsigned long attrs); + dma_addr_t dma_addr, unsigned long attrs); void vduse_domain_reset_bounce_map(struct vduse_iova_domain *domain); diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index db7af4b3fd4a..1cc826742ac9 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -83,6 +84,10 @@ struct vduse_umem { struct mm_struct *mm; }; +struct vduse_vq_group { + struct vduse_dev *dev; +}; + struct vduse_dev { struct vduse_vdpa *vdev; struct device *dev; @@ -115,6 +120,7 @@ struct vduse_dev { u32 vq_num; u32 vq_align; struct vduse_umem *umem; + struct vduse_vq_group *groups; struct mutex mem_lock; unsigned int bounce_size; struct mutex domain_lock; @@ -615,6 +621,16 @@ static int vduse_vdpa_set_vq_state(struct vdpa_device *vdpa, u16 idx, return 0; } +static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx) +{ + struct vduse_dev *dev = vdpa_to_vduse(vdpa); + union virtio_map ret = { + .group = &dev->groups[0], + }; + + return ret; +} + static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx, struct vdpa_vq_state *state) { @@ -834,6 +850,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = { .get_vq_affinity = vduse_vdpa_get_vq_affinity, .reset = vduse_vdpa_reset, .set_map = vduse_vdpa_set_map, + .get_vq_map = vduse_get_vq_map, .free = vduse_vdpa_free, }; @@ -841,7 +858,14 @@ static void vduse_dev_sync_single_for_device(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; + + if (!token.group) + return; + + vdev = token.group->dev; + domain = vdev->domain; vduse_domain_sync_single_for_device(domain, dma_addr, size, dir); } @@ -850,7 +874,14 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; + + if (!token.group) + return; + + vdev = token.group->dev; + domain = vdev->domain; vduse_domain_sync_single_for_cpu(domain, dma_addr, size, dir); } @@ -860,7 +891,14 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; + + if (!token.group) + return DMA_MAPPING_ERROR; + + vdev = token.group->dev; + domain = vdev->domain; return vduse_domain_map_page(domain, page, offset, size, dir, attrs); } @@ -869,7 +907,14 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; + + if (!token.group) + return; + + vdev = token.group->dev; + domain = vdev->domain; return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs); } @@ -877,33 +922,58 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr, static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, dma_addr_t *dma_addr, gfp_t flag) { - struct vduse_iova_domain *domain = token.iova_domain; - unsigned long iova; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; void *addr; *dma_addr = DMA_MAPPING_ERROR; - addr = vduse_domain_alloc_coherent(domain, size, - (dma_addr_t *)&iova, flag); + if (!token.group) + return NULL; + + addr = alloc_pages_exact(size, flag | __GFP_ZERO); if (!addr) return NULL; - *dma_addr = (dma_addr_t)iova; + vdev = token.group->dev; + domain = vdev->domain; + *dma_addr = vduse_domain_alloc_coherent(domain, size, addr); + if (*dma_addr == DMA_MAPPING_ERROR) + goto err; return addr; + +err: + free_pages_exact(addr, size); + return NULL; } static void vduse_dev_free_coherent(union virtio_map token, size_t size, void *vaddr, dma_addr_t dma_addr, unsigned long attrs) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; + + if (!token.group) + return; + + vdev = token.group->dev; + domain = vdev->domain; - vduse_domain_free_coherent(domain, size, vaddr, dma_addr, attrs); + vduse_domain_free_coherent(domain, size, dma_addr, attrs); + free_pages_exact(vaddr, size); } static bool vduse_dev_need_sync(union virtio_map token, dma_addr_t dma_addr) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; + + if (!token.group) + return false; + + vdev = token.group->dev; + domain = vdev->domain; return dma_addr < domain->bounce_size; } @@ -917,7 +987,14 @@ static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr) static size_t vduse_dev_max_mapping_size(union virtio_map token) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; + struct vduse_iova_domain *domain; + + if (!token.group) + return 0; + + vdev = token.group->dev; + domain = vdev->domain; return domain->bounce_size; } @@ -1716,6 +1793,7 @@ static int vduse_destroy_dev(char *name) if (dev->domain) vduse_domain_destroy(dev->domain); kfree(dev->name); + kfree(dev->groups); vduse_dev_destroy(dev); module_put(THIS_MODULE); @@ -1874,6 +1952,11 @@ static int vduse_create_dev(struct vduse_dev_config *config, dev->device_features = config->features; dev->device_id = config->device_id; dev->vendor_id = config->vendor_id; + dev->groups = kcalloc(1, sizeof(dev->groups[0]), GFP_KERNEL); + if (!dev->groups) + goto err_vq_groups; + dev->groups[0].dev = dev; + dev->name = kstrdup(config->name, GFP_KERNEL); if (!dev->name) goto err_str; @@ -1910,6 +1993,8 @@ static int vduse_create_dev(struct vduse_dev_config *config, err_idr: kfree(dev->name); err_str: + kfree(dev->groups); +err_vq_groups: vduse_dev_destroy(dev); err: return ret; @@ -2071,7 +2156,6 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, return -ENOMEM; } - dev->vdev->vdpa.vmap.iova_domain = dev->domain; ret = _vdpa_register_device(&dev->vdev->vdpa, dev->vq_num); if (ret) { put_device(&dev->vdev->vdpa.dev); diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index b2b90732b1d5..89d5438ba7af 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -724,10 +724,12 @@ static int vhost_net_build_xdp(struct vhost_net_virtqueue *nvq, goto err; } - gso = buf + pad - sock_hlen; - - if (!sock_hlen) + if (!sock_hlen) { memset(buf, 0, pad); + gso = buf; + } else { + gso = buf + pad - sock_hlen; + } if ((gso->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) && vhost16_to_cpu(vq, gso->csum_start) + diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig index d51777df12d1..0adb1e2fa533 100644 --- a/drivers/video/Kconfig +++ b/drivers/video/Kconfig @@ -37,6 +37,7 @@ source "drivers/char/agp/Kconfig" source "drivers/gpu/vga/Kconfig" +source "drivers/gpu/Kconfig" source "drivers/gpu/host1x/Kconfig" source "drivers/gpu/ipu-v3/Kconfig" source "drivers/gpu/nova-core/Kconfig" diff --git a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c b/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c index 4230b817a80b..d66291def0f4 100644 --- a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c +++ b/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c @@ -82,8 +82,8 @@ static int mmio_guard_ioremap_hook(phys_addr_t phys, size_t size, if (protval != PROT_DEVICE_nGnRE && protval != PROT_DEVICE_nGnRnE) return 0; + end = PAGE_ALIGN(phys + size); phys = PAGE_ALIGN_DOWN(phys); - end = phys + PAGE_ALIGN(size); while (phys < end) { const int func_id = ARM_SMCCC_VENDOR_HYP_KVM_MMIO_GUARD_FUNC_ID; diff --git a/drivers/watchdog/airoha_wdt.c b/drivers/watchdog/airoha_wdt.c index dc8ca11c14d8..4bd333189b87 100644 --- a/drivers/watchdog/airoha_wdt.c +++ b/drivers/watchdog/airoha_wdt.c @@ -147,6 +147,9 @@ static int airoha_wdt_probe(struct platform_device *pdev) /* Watchdog ticks at half the bus rate */ airoha_wdt->wdt_freq = clk_get_rate(bus_clk) / 2; + if (!airoha_wdt->wdt_freq) + return dev_err_probe(dev, -EINVAL, + "invalid clock frequency\n"); /* Initialize struct watchdog device */ wdog_dev = &airoha_wdt->wdog_dev; diff --git a/drivers/watchdog/s32g_wdt.c b/drivers/watchdog/s32g_wdt.c index ad55063060af..6422a694fc65 100644 --- a/drivers/watchdog/s32g_wdt.c +++ b/drivers/watchdog/s32g_wdt.c @@ -56,8 +56,7 @@ MODULE_PARM_DESC(early_enable, static const struct watchdog_info s32g_wdt_info = { .identity = "s32g watchdog", - .options = WDIOF_KEEPALIVEPING | WDIOF_SETTIMEOUT | WDIOF_MAGICCLOSE | - WDIOC_GETTIMEOUT | WDIOC_GETTIMELEFT, + .options = WDIOF_KEEPALIVEPING | WDIOF_SETTIMEOUT | WDIOF_MAGICCLOSE, }; static struct s32g_wdt_device *wdd_to_s32g_wdt(struct watchdog_device *wdd) diff --git a/drivers/watchdog/watchdog_pretimeout.c b/drivers/watchdog/watchdog_pretimeout.c index 2526436dc74d..ea05f84f07e1 100644 --- a/drivers/watchdog/watchdog_pretimeout.c +++ b/drivers/watchdog/watchdog_pretimeout.c @@ -167,6 +167,8 @@ void watchdog_unregister_governor(struct watchdog_governor *gov) } spin_lock_irq(&pretimeout_lock); + if (default_gov == gov) + default_gov = NULL; list_for_each_entry(p, &pretimeout_list, entry) if (p->wdd->gov == gov) p->wdd->gov = default_gov; diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c index fd3aa9f97ce6..3ead36a07048 100644 --- a/fs/afs/dir_edit.c +++ b/fs/afs/dir_edit.c @@ -415,7 +415,7 @@ void afs_edit_dir_remove(struct afs_vnode *vnode, if (!afs_dir_init_iter(&iter, name)) return; - meta = afs_dir_find_block(&iter, 0); + meta = afs_dir_get_block(&iter, 0); if (!meta) return; diff --git a/fs/binfmt_elf_fdpic.c b/fs/binfmt_elf_fdpic.c index a3d4e6973b29..7834ca77877d 100644 --- a/fs/binfmt_elf_fdpic.c +++ b/fs/binfmt_elf_fdpic.c @@ -231,6 +231,10 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm) for (i = 0; i < exec_params.hdr.e_phnum; i++, phdr++) { switch (phdr->p_type) { case PT_INTERP: + /* elf ABI allows only one interpreter */ + if (interpreter_name) + continue; + retval = -ENOMEM; if (phdr->p_filesz > PATH_MAX) goto error; diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c index a8b1d79e4af0..d44529d5ecf5 100644 --- a/fs/binfmt_misc.c +++ b/fs/binfmt_misc.c @@ -227,9 +227,6 @@ static int load_misc_binary(struct linux_binprm *bprm) goto ret; } - if (fmt->flags & MISC_FMT_OPEN_BINARY) - bprm->have_execfd = 1; - /* make argv[1] be the path to the binary */ retval = copy_string_kernel(bprm->interp, bprm); if (retval < 0) @@ -259,6 +256,8 @@ static int load_misc_binary(struct linux_binprm *bprm) goto ret; bprm->interpreter = interp_file; + if (fmt->flags & MISC_FMT_OPEN_BINARY) + bprm->have_execfd = 1; if (fmt->flags & MISC_FMT_CREDENTIALS) bprm->execfd_creds = 1; diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c index e050d0938dc4..0c47b4570a7b 100644 --- a/fs/btrfs/backref.c +++ b/fs/btrfs/backref.c @@ -1408,12 +1408,12 @@ static int find_parent_nodes(struct btrfs_backref_walk_ctx *ctx, if (!path) return -ENOMEM; if (!ctx->trans) { - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; } if (ctx->time_seq == BTRFS_SEQ_LAST) - path->skip_locking = 1; + path->skip_locking = true; again: head = NULL; @@ -1560,7 +1560,7 @@ static int find_parent_nodes(struct btrfs_backref_walk_ctx *ctx, btrfs_release_path(path); - ret = add_missing_keys(ctx->fs_info, &preftrees, path->skip_locking == 0); + ret = add_missing_keys(ctx->fs_info, &preftrees, !path->skip_locking); if (ret) goto out; @@ -2833,8 +2833,8 @@ struct btrfs_backref_iter *btrfs_backref_iter_alloc(struct btrfs_fs_info *fs_inf } /* Current backref iterator only supports iteration in commit root */ - ret->path->search_commit_root = 1; - ret->path->skip_locking = 1; + ret->path->search_commit_root = true; + ret->path->skip_locking = true; ret->fs_info = fs_info; return ret; @@ -3307,8 +3307,8 @@ static int handle_indirect_tree_backref(struct btrfs_trans_handle *trans, level = cur->level + 1; /* Search the tree to find parent blocks referring to the block */ - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; path->lowest_level = level; ret = btrfs_search_slot(NULL, root, tree_key, path, 0, 0); path->lowest_level = 0; diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c index 1e57f7d04c47..e6bac34ebe83 100644 --- a/fs/btrfs/block-group.c +++ b/fs/btrfs/block-group.c @@ -613,8 +613,8 @@ static int sample_block_group_extent_item(struct btrfs_caching_control *caching_ extent_root = btrfs_extent_root(fs_info, max_t(u64, block_group->start, BTRFS_SUPER_INFO_OFFSET)); - path->skip_locking = 1; - path->search_commit_root = 1; + path->skip_locking = true; + path->search_commit_root = true; path->reada = READA_FORWARD; search_offset = index * div_u64(block_group->length, max_index); @@ -744,8 +744,8 @@ static int load_extent_tree_free(struct btrfs_caching_control *caching_ctl) * root to add free space. So we skip locking and search the commit * root, since its read-only */ - path->skip_locking = 1; - path->search_commit_root = 1; + path->skip_locking = true; + path->search_commit_root = true; path->reada = READA_FORWARD; key.objectid = last; diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index 27e2adc2ee71..0f2e9f33dd0c 100644 --- a/fs/btrfs/ctree.c +++ b/fs/btrfs/ctree.c @@ -1709,9 +1709,9 @@ static struct extent_buffer *btrfs_search_slot_get_root(struct btrfs_root *root, level = btrfs_header_level(b); /* * Ensure that all callers have set skip_locking when - * p->search_commit_root = 1. + * p->search_commit_root is true. */ - ASSERT(p->skip_locking == 1); + ASSERT(p->skip_locking); goto out; } @@ -3858,10 +3858,10 @@ static noinline int setup_leaf_for_split(struct btrfs_trans_handle *trans, } btrfs_release_path(path); - path->keep_locks = 1; - path->search_for_split = 1; + path->keep_locks = true; + path->search_for_split = true; ret = btrfs_search_slot(trans, root, &key, path, 0, 1); - path->search_for_split = 0; + path->search_for_split = false; if (ret > 0) ret = -EAGAIN; if (ret < 0) @@ -3888,11 +3888,11 @@ static noinline int setup_leaf_for_split(struct btrfs_trans_handle *trans, if (ret) goto err; - path->keep_locks = 0; + path->keep_locks = false; btrfs_unlock_up_safe(path, 1); return 0; err: - path->keep_locks = 0; + path->keep_locks = false; return ret; } @@ -4610,11 +4610,11 @@ int btrfs_search_forward(struct btrfs_root *root, struct btrfs_key *min_key, u32 nritems; int level; int ret = 1; - int keep_locks = path->keep_locks; + const bool keep_locks = path->keep_locks; ASSERT(!path->nowait); ASSERT(path->lowest_level == 0); - path->keep_locks = 1; + path->keep_locks = true; again: cur = btrfs_read_lock_root_node(root); level = btrfs_header_level(cur); @@ -4704,7 +4704,7 @@ int btrfs_search_forward(struct btrfs_root *root, struct btrfs_key *min_key, * 0 is returned if another key is found, < 0 if there are any errors * and 1 is returned if there are no higher keys in the tree * - * path->keep_locks should be set to 1 on the search made before + * path->keep_locks should be set to true on the search made before * calling this function. */ int btrfs_find_next_key(struct btrfs_root *root, struct btrfs_path *path, @@ -4803,13 +4803,13 @@ int btrfs_next_old_leaf(struct btrfs_root *root, struct btrfs_path *path, next = NULL; btrfs_release_path(path); - path->keep_locks = 1; + path->keep_locks = true; if (time_seq) { ret = btrfs_search_old_slot(root, &key, path, time_seq); } else { if (path->need_commit_sem) { - path->need_commit_sem = 0; + path->need_commit_sem = false; need_commit_sem = true; if (path->nowait) { if (!down_read_trylock(&fs_info->commit_root_sem)) { @@ -4822,7 +4822,7 @@ int btrfs_next_old_leaf(struct btrfs_root *root, struct btrfs_path *path, } ret = btrfs_search_slot(NULL, root, &key, path, 0, 0); } - path->keep_locks = 0; + path->keep_locks = false; if (ret < 0) goto done; @@ -4961,7 +4961,7 @@ int btrfs_next_old_leaf(struct btrfs_root *root, struct btrfs_path *path, if (need_commit_sem) { int ret2; - path->need_commit_sem = 1; + path->need_commit_sem = true; ret2 = finish_need_commit_sem_search(path); up_read(&fs_info->commit_root_sem); if (ret2) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 16dd11c48531..692370fc07b2 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -65,21 +65,21 @@ struct btrfs_path { * set by btrfs_split_item, tells search_slot to keep all locks * and to force calls to keep space in the nodes */ - unsigned int search_for_split:1; + bool search_for_split:1; /* Keep some upper locks as we walk down. */ - unsigned int keep_locks:1; - unsigned int skip_locking:1; - unsigned int search_commit_root:1; - unsigned int need_commit_sem:1; - unsigned int skip_release_on_error:1; + bool keep_locks:1; + bool skip_locking:1; + bool search_commit_root:1; + bool need_commit_sem:1; + bool skip_release_on_error:1; /* * Indicate that new item (btrfs_search_slot) is extending already * existing item and ins_len contains only the data size and not item * header (ie. sizeof(struct btrfs_item) is not included). */ - unsigned int search_for_extension:1; + bool search_for_extension:1; /* Stop search if any locks need to be taken (for read) */ - unsigned int nowait:1; + bool nowait:1; }; #define BTRFS_PATH_AUTO_FREE(path_name) \ diff --git a/fs/btrfs/defrag.c b/fs/btrfs/defrag.c index a4cc1bc63562..2e3c011d410a 100644 --- a/fs/btrfs/defrag.c +++ b/fs/btrfs/defrag.c @@ -472,7 +472,7 @@ static int btrfs_defrag_leaves(struct btrfs_trans_handle *trans, memcpy(&key, &root->defrag_progress, sizeof(key)); } - path->keep_locks = 1; + path->keep_locks = true; ret = btrfs_search_forward(root, &key, path, BTRFS_OLDEST_GENERATION); if (ret < 0) @@ -515,7 +515,7 @@ static int btrfs_defrag_leaves(struct btrfs_trans_handle *trans, /* * Now that we reallocated the node we can find the next key. Note that * btrfs_find_next_key() can release our path and do another search - * without COWing, this is because even with path->keep_locks = 1, + * without COWing, this is because even with path->keep_locks == true, * btrfs_search_slot() / ctree.c:unlock_up() does not keeps a lock on a * node when path->slots[node_level - 1] does not point to the last * item or a slot beyond the last item (ctree.c:unlock_up()). Therefore diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index a4eaef60549e..b6c7da8e1bc8 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -489,8 +489,8 @@ static int mark_block_group_to_copy(struct btrfs_fs_info *fs_info, } path->reada = READA_FORWARD; - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; key.objectid = src_dev->devid; key.type = BTRFS_DEV_EXTENT_KEY; diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 6ef7cc116bfc..d2d96f366dac 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -788,7 +788,7 @@ int lookup_inline_extent_backref(struct btrfs_trans_handle *trans, want = extent_ref_type(parent, owner); if (insert) { extra_size = btrfs_extent_inline_ref_size(want); - path->search_for_extension = 1; + path->search_for_extension = true; } else extra_size = -1; @@ -954,7 +954,7 @@ int lookup_inline_extent_backref(struct btrfs_trans_handle *trans, if (!path->keep_locks) { btrfs_release_path(path); - path->keep_locks = 1; + path->keep_locks = true; goto again; } @@ -975,11 +975,11 @@ int lookup_inline_extent_backref(struct btrfs_trans_handle *trans, *ref_ret = (struct btrfs_extent_inline_ref *)ptr; out: if (path->keep_locks) { - path->keep_locks = 0; + path->keep_locks = false; btrfs_unlock_up_safe(path, 1); } if (insert) - path->search_for_extension = 0; + path->search_for_extension = false; return ret; } diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index f955b4003d42..80fc2d59bfeb 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1949,7 +1949,7 @@ static noinline_for_stack bool lock_extent_buffer_for_io(struct extent_buffer *e btrfs_set_header_flag(eb, BTRFS_HEADER_FLAG_WRITTEN); percpu_counter_add_batch(&fs_info->dirty_metadata_bytes, - -eb->len, + -(s64)eb->len, fs_info->dirty_metadata_batch); ret = true; } else { @@ -3731,7 +3731,7 @@ void btrfs_clear_buffer_dirty(struct btrfs_trans_handle *trans, return; buffer_tree_clear_mark(eb, PAGECACHE_TAG_DIRTY); - percpu_counter_add_batch(&fs_info->dirty_metadata_bytes, -eb->len, + percpu_counter_add_batch(&fs_info->dirty_metadata_bytes, -(s64)eb->len, fs_info->dirty_metadata_batch); for (int i = 0; i < num_extent_folios(eb); i++) { diff --git a/fs/btrfs/extent_map.c b/fs/btrfs/extent_map.c index 7e38c23a0c1c..21eb104ee2ae 100644 --- a/fs/btrfs/extent_map.c +++ b/fs/btrfs/extent_map.c @@ -854,13 +854,13 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end, goto next; } - flags = em->flags; /* * In case we split the extent map, we want to preserve the * EXTENT_FLAG_LOGGING flag on our extent map, but we don't want * it on the new extent maps. */ - em->flags &= ~(EXTENT_FLAG_PINNED | EXTENT_FLAG_LOGGING); + flags = em->flags & ~EXTENT_FLAG_LOGGING; + em->flags &= ~EXTENT_FLAG_PINNED; modified = !list_empty(&em->list); /* diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c index 4b7c40f05e8f..9b19b33e85b6 100644 --- a/fs/btrfs/file-item.c +++ b/fs/btrfs/file-item.c @@ -350,6 +350,7 @@ int btrfs_lookup_bio_sums(struct btrfs_bio *bbio) const unsigned int nblocks = orig_len >> fs_info->sectorsize_bits; int ret = 0; u32 bio_offset = 0; + bool using_commit_root = false; if ((inode->flags & BTRFS_INODE_NODATASUM) || test_bit(BTRFS_FS_STATE_NO_DATA_CSUMS, &fs_info->fs_state)) @@ -394,8 +395,8 @@ int btrfs_lookup_bio_sums(struct btrfs_bio *bbio) * between reading the free space cache and updating the csum tree. */ if (btrfs_is_free_space_inode(inode)) { - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; } /* @@ -423,8 +424,9 @@ int btrfs_lookup_bio_sums(struct btrfs_bio *bbio) * from across transactions. */ if (bbio->csum_search_commit_root) { - path->search_commit_root = 1; - path->skip_locking = 1; + using_commit_root = true; + path->search_commit_root = true; + path->skip_locking = true; down_read(&fs_info->commit_root_sem); } @@ -455,6 +457,28 @@ int btrfs_lookup_bio_sums(struct btrfs_bio *bbio) * assume this is the case. */ if (count == 0) { + /* + * If an extent is relocated in the current transaction + * then relocation writes a new csum without updating + * the extent map generation. Until the next commit, we + * will see a hole in that case, so we need to fallback + * to searching the transaction csum root. + * + * Note that a commit root lookup of a referenced extent can + * only miss, not return a stale csum. A freed extent's csum + * is deleted in the same transaction and its bytenr is not + * reusable until that transaction has committed and the + * extent is unpinned. + */ + if (using_commit_root) { + up_read(&fs_info->commit_root_sem); + using_commit_root = false; + path->search_commit_root = false; + path->skip_locking = false; + btrfs_release_path(path); + continue; + } + memset(csum_dst, 0, csum_size); count = 1; @@ -473,7 +497,7 @@ int btrfs_lookup_bio_sums(struct btrfs_bio *bbio) bio_offset += count * sectorsize; } - if (bbio->csum_search_commit_root) + if (using_commit_root) up_read(&fs_info->commit_root_sem); return ret; } @@ -1168,10 +1192,10 @@ int btrfs_csum_file_blocks(struct btrfs_trans_handle *trans, } btrfs_release_path(path); - path->search_for_extension = 1; + path->search_for_extension = true; ret = btrfs_search_slot(trans, root, &file_key, path, csum_size, 1); - path->search_for_extension = 0; + path->search_for_extension = false; if (ret < 0) goto out; diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index ab873bd67192..7ee3d8f05c1a 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -554,6 +554,9 @@ static int io_ctl_check_crc(struct btrfs_io_ctl *io_ctl, int index) u32 crc = ~(u32)0; unsigned offset = 0; + if (index >= io_ctl->num_pages) + return -EIO; + if (index == 0) offset = sizeof(u32) * io_ctl->num_pages; @@ -968,8 +971,8 @@ int load_free_space_cache(struct btrfs_block_group *block_group) path = btrfs_alloc_path(); if (!path) return 0; - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; /* * We must pass a path with search_commit_root set to btrfs_iget in diff --git a/fs/btrfs/free-space-tree.c b/fs/btrfs/free-space-tree.c index c3734892d654..f9ffb1c8988d 100644 --- a/fs/btrfs/free-space-tree.c +++ b/fs/btrfs/free-space-tree.c @@ -1699,8 +1699,8 @@ int btrfs_load_free_space_tree(struct btrfs_caching_control *caching_ctl) * Just like caching_thread() doesn't want to deadlock on the extent * tree, we don't want to deadlock on the free space tree. */ - path->skip_locking = 1; - path->search_commit_root = 1; + path->skip_locking = true; + path->search_commit_root = true; path->reada = READA_FORWARD; info = btrfs_search_free_space_info(NULL, block_group, path, 0); diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c index 7e14e1bbcf38..b73e1dd97208 100644 --- a/fs/btrfs/inode-item.c +++ b/fs/btrfs/inode-item.c @@ -312,7 +312,7 @@ int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans, if (!path) return -ENOMEM; - path->skip_release_on_error = 1; + path->skip_release_on_error = true; ret = btrfs_insert_empty_item(trans, root, path, &key, ins_len); if (ret == -EEXIST) { diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 30a34cd905ba..36f75c6a8344 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -7170,8 +7170,8 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode, * point the commit_root has everything we need. */ if (btrfs_is_free_space_inode(inode)) { - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; } ret = btrfs_lookup_file_extent(NULL, root, path, objectid, start, 0); diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 261aa6501920..b4c6cbbcd168 100644 --- a/fs/btrfs/qgroup.c +++ b/fs/btrfs/qgroup.c @@ -3882,8 +3882,8 @@ static void btrfs_qgroup_rescan_worker(struct btrfs_work *work) * Rescan should only search for commit root, and any later difference * should be recorded by qgroup */ - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; while (!ret && !(stopped = rescan_should_stop(fs_info))) { trans = btrfs_start_transaction(fs_info->fs_root, 0); diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c index cc6f6095cc9f..f6f02463dd1a 100644 --- a/fs/btrfs/raid-stripe-tree.c +++ b/fs/btrfs/raid-stripe-tree.c @@ -394,8 +394,8 @@ int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info, return -ENOMEM; if (stripe->rst_search_commit_root) { - path->skip_locking = 1; - path->search_commit_root = 1; + path->skip_locking = true; + path->search_commit_root = true; } ret = btrfs_search_slot(NULL, stripe_root, &stripe_key, path, 0, 0); diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index fc76013b1a3e..80e5de6736e8 100644 --- a/fs/btrfs/relocation.c +++ b/fs/btrfs/relocation.c @@ -496,6 +496,7 @@ static int __add_reloc_root(struct btrfs_root *root) btrfs_err(fs_info, "Duplicate root found for start=%llu while inserting into relocation tree", node->bytenr); + kfree(node); return -EEXIST; } @@ -1857,6 +1858,7 @@ void merge_reloc_roots(struct reloc_control *rc) * corruption, e.g. bad reloc tree key offset. */ ret = -EINVAL; + btrfs_put_root(root); goto out; } ret = merge_reloc_root(rc, root); @@ -3175,8 +3177,8 @@ static int __add_tree_block(struct reloc_control *rc, key.offset = blocksize; } - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; ret = btrfs_search_slot(NULL, rc->extent_root, &key, path, 0, 0); if (ret < 0) return ret; @@ -3368,8 +3370,8 @@ int find_next_extent(struct reloc_control *rc, struct btrfs_path *path, key.type = BTRFS_EXTENT_ITEM_KEY; key.offset = 0; - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; ret = btrfs_search_slot(NULL, rc->extent_root, &key, path, 0, 0); if (ret < 0) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 16936d17166e..368f1d444493 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -463,10 +463,10 @@ static noinline_for_stack struct scrub_ctx *scrub_setup_ctx( refcount_set(&sctx->refs, 1); sctx->is_dev_replace = is_dev_replace; sctx->fs_info = fs_info; - sctx->extent_path.search_commit_root = 1; - sctx->extent_path.skip_locking = 1; - sctx->csum_path.search_commit_root = 1; - sctx->csum_path.skip_locking = 1; + sctx->extent_path.search_commit_root = true; + sctx->extent_path.skip_locking = true; + sctx->csum_path.search_commit_root = true; + sctx->csum_path.skip_locking = true; for (i = 0; i < SCRUB_TOTAL_STRIPES; i++) { int ret; @@ -2103,10 +2103,10 @@ static int scrub_raid56_parity_stripe(struct scrub_ctx *sctx, * as the data stripe bytenr may be smaller than previous extent. Thus * we have to use our own extent/csum paths. */ - extent_path.search_commit_root = 1; - extent_path.skip_locking = 1; - csum_path.search_commit_root = 1; - csum_path.skip_locking = 1; + extent_path.search_commit_root = true; + extent_path.skip_locking = true; + csum_path.search_commit_root = true; + csum_path.skip_locking = true; for (int i = 0; i < data_stripes; i++) { int stripe_index; @@ -2630,8 +2630,8 @@ int scrub_enumerate_chunks(struct scrub_ctx *sctx, return -ENOMEM; path->reada = READA_FORWARD; - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; key.objectid = scrub_dev->devid; key.type = BTRFS_DEV_EXTENT_KEY; diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 04473387ee8b..34a57f11e4bb 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -633,9 +633,9 @@ static struct btrfs_path *alloc_path_for_send(void) path = btrfs_alloc_path(); if (!path) return NULL; - path->search_commit_root = 1; - path->skip_locking = 1; - path->need_commit_sem = 1; + path->search_commit_root = true; + path->skip_locking = true; + path->need_commit_sem = true; return path; } @@ -7643,10 +7643,10 @@ static int btrfs_compare_trees(struct btrfs_root *left_root, goto out; } - left_path->search_commit_root = 1; - left_path->skip_locking = 1; - right_path->search_commit_root = 1; - right_path->skip_locking = 1; + left_path->search_commit_root = true; + left_path->skip_locking = true; + right_path->search_commit_root = true; + right_path->skip_locking = true; /* * Strategy: Go to the first items of both trees. Then do diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index a0e12b4fb956..f84065b69464 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -602,9 +602,9 @@ static int overwrite_item(struct walk_control *wc) insert: btrfs_release_path(wc->subvol_path); /* try to insert the key into the destination tree */ - wc->subvol_path->skip_release_on_error = 1; + wc->subvol_path->skip_release_on_error = true; ret = btrfs_insert_empty_item(trans, root, wc->subvol_path, &wc->log_key, item_size); - wc->subvol_path->skip_release_on_error = 0; + wc->subvol_path->skip_release_on_error = false; dst_eb = wc->subvol_path->nodes[0]; dst_slot = wc->subvol_path->slots[0]; @@ -5734,8 +5734,8 @@ static int btrfs_check_ref_name_override(struct extent_buffer *eb, search_path = btrfs_alloc_path(); if (!search_path) return -ENOMEM; - search_path->search_commit_root = 1; - search_path->skip_locking = 1; + search_path->search_commit_root = true; + search_path->skip_locking = true; while (cur_offset < item_size) { u64 parent; @@ -6052,8 +6052,8 @@ static int conflicting_inode_is_dir(struct btrfs_root *root, u64 ino, key.type = BTRFS_INODE_ITEM_KEY; key.offset = 0; - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; ret = btrfs_search_slot(NULL, root, &key, path, 0, 0); if (WARN_ON_ONCE(ret > 0)) { @@ -6073,8 +6073,8 @@ static int conflicting_inode_is_dir(struct btrfs_root *root, u64 ino, } btrfs_release_path(path); - path->search_commit_root = 0; - path->skip_locking = 0; + path->search_commit_root = false; + path->skip_locking = false; return ret; } @@ -7232,8 +7232,8 @@ static int btrfs_log_all_parents(struct btrfs_trans_handle *trans, path = btrfs_alloc_path(); if (!path) return -ENOMEM; - path->skip_locking = 1; - path->search_commit_root = 1; + path->skip_locking = true; + path->search_commit_root = true; key.objectid = ino; key.type = BTRFS_INODE_REF_KEY; diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 630fb5885692..863f69b9cf14 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -1839,8 +1839,8 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes, } path->reada = READA_FORWARD; - path->search_commit_root = 1; - path->skip_locking = 1; + path->search_commit_root = true; + path->skip_locking = true; key.objectid = device->devid; key.type = BTRFS_DEV_EXTENT_KEY; @@ -7627,7 +7627,7 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info) * chunk tree, to keep it simple, just skip locking on the chunk tree. */ ASSERT(!test_bit(BTRFS_FS_OPEN, &fs_info->flags)); - path->skip_locking = 1; + path->skip_locking = true; /* * Read all device items, and then all the chunk items. All diff --git a/fs/btrfs/xattr.c b/fs/btrfs/xattr.c index b6f01d6c79e7..b6e91e8fb7e3 100644 --- a/fs/btrfs/xattr.c +++ b/fs/btrfs/xattr.c @@ -97,7 +97,7 @@ int btrfs_setxattr(struct btrfs_trans_handle *trans, struct inode *inode, path = btrfs_alloc_path(); if (!path) return -ENOMEM; - path->skip_release_on_error = 1; + path->skip_release_on_error = true; if (!value) { di = btrfs_lookup_xattr(trans, root, path, diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index 3af6795cb3c1..ea31c892a1fb 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -785,6 +785,9 @@ static int write_folio_nounlock(struct folio *folio, ceph_wbc.truncate_size, true); if (IS_ERR(req)) { folio_redirty_for_writepage(wbc, folio); + if (atomic_long_dec_return(&fsc->writeback_count) < + CONGESTION_OFF_THRESH(fsc->mount_options->congestion_kb)) + fsc->write_congested = false; return PTR_ERR(req); } @@ -804,6 +807,9 @@ static int write_folio_nounlock(struct folio *folio, folio_redirty_for_writepage(wbc, folio); folio_end_writeback(folio); ceph_osdc_put_request(req); + if (atomic_long_dec_return(&fsc->writeback_count) < + CONGESTION_OFF_THRESH(fsc->mount_options->congestion_kb)) + fsc->write_congested = false; return PTR_ERR(bounce_page); } } @@ -838,6 +844,9 @@ static int write_folio_nounlock(struct folio *folio, ceph_vinop(inode), folio); folio_redirty_for_writepage(wbc, folio); folio_end_writeback(folio); + if (atomic_long_dec_return(&fsc->writeback_count) < + CONGESTION_OFF_THRESH(fsc->mount_options->congestion_kb)) + fsc->write_congested = false; return err; } if (err == -EBLOCKLISTED) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index b1a8ff612c41..d9924ef55f4a 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -4364,6 +4364,7 @@ void ceph_handle_caps(struct ceph_mds_session *session, snaptrace = h + 1; snaptrace_len = le32_to_cpu(h->snap_trace_len); + ceph_decode_need(&snaptrace, end, snaptrace_len, bad); p = snaptrace + snaptrace_len; if (msg_version >= 2) { diff --git a/fs/ceph/dir.c b/fs/ceph/dir.c index 66e592c47e57..c30510bdae3a 100644 --- a/fs/ceph/dir.c +++ b/fs/ceph/dir.c @@ -546,11 +546,16 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) pr_warn_client(cl, "%p %llx.%llx rde->offset 0x%llx ctx->pos 0x%llx\n", inode, ceph_vinop(inode), rde->offset, ctx->pos); + ceph_mdsc_put_request(dfi->last_readdir); + dfi->last_readdir = NULL; return -EIO; } - if (WARN_ON_ONCE(!rde->inode.in)) + if (WARN_ON_ONCE(!rde->inode.in)) { + ceph_mdsc_put_request(dfi->last_readdir); + dfi->last_readdir = NULL; return -EIO; + } ctx->pos = rde->offset; doutc(cl, "%p %llx.%llx (%d/%d) -> %llx '%.*s' %p\n", inode, diff --git a/fs/crypto/inline_crypt.c b/fs/crypto/inline_crypt.c index 645cc4936072..500397ca8a26 100644 --- a/fs/crypto/inline_crypt.c +++ b/fs/crypto/inline_crypt.c @@ -22,22 +22,14 @@ #include "fscrypt_private.h" -static struct block_device **fscrypt_get_devices(struct super_block *sb, - unsigned int *num_devs) +static unsigned int +fscrypt_get_devices(struct super_block *sb, + struct block_device *devs[FSCRYPT_MAX_DEVICES]) { - struct block_device **devs; - - if (sb->s_cop->get_devices) { - devs = sb->s_cop->get_devices(sb, num_devs); - if (devs) - return devs; - } - devs = kmalloc(sizeof(*devs), GFP_KERNEL); - if (!devs) - return ERR_PTR(-ENOMEM); + if (sb->s_cop->get_devices) + return sb->s_cop->get_devices(sb, devs); devs[0] = sb->s_bdev; - *num_devs = 1; - return devs; + return 1; } static unsigned int fscrypt_get_dun_bytes(const struct fscrypt_inode_info *ci) @@ -96,7 +88,7 @@ int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci, const struct inode *inode = ci->ci_inode; struct super_block *sb = inode->i_sb; struct blk_crypto_config crypto_cfg; - struct block_device **devs; + struct block_device *devs[FSCRYPT_MAX_DEVICES]; unsigned int num_devs; unsigned int i; @@ -135,20 +127,15 @@ int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci, crypto_cfg.key_type = is_hw_wrapped_key ? BLK_CRYPTO_KEY_TYPE_HW_WRAPPED : BLK_CRYPTO_KEY_TYPE_RAW; - devs = fscrypt_get_devices(sb, &num_devs); - if (IS_ERR(devs)) - return PTR_ERR(devs); - + num_devs = fscrypt_get_devices(sb, devs); for (i = 0; i < num_devs; i++) { if (!blk_crypto_config_supported(devs[i], &crypto_cfg)) - goto out_free_devs; + return 0; } fscrypt_log_blk_crypto_impl(ci->ci_mode, devs, num_devs, &crypto_cfg); ci->ci_inlinecrypt = true; -out_free_devs: - kfree(devs); return 0; } @@ -164,7 +151,7 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, enum blk_crypto_key_type key_type = is_hw_wrapped ? BLK_CRYPTO_KEY_TYPE_HW_WRAPPED : BLK_CRYPTO_KEY_TYPE_RAW; struct blk_crypto_key *blk_key; - struct block_device **devs; + struct block_device *devs[FSCRYPT_MAX_DEVICES]; unsigned int num_devs; unsigned int i; int err; @@ -182,17 +169,12 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, } /* Start using blk-crypto on all the filesystem's block devices. */ - devs = fscrypt_get_devices(sb, &num_devs); - if (IS_ERR(devs)) { - err = PTR_ERR(devs); - goto fail; - } + num_devs = fscrypt_get_devices(sb, devs); for (i = 0; i < num_devs; i++) { err = blk_crypto_start_using_key(devs[i], blk_key); if (err) break; } - kfree(devs); if (err) { fscrypt_err(inode, "error %d starting to use blk-crypto", err); goto fail; @@ -210,20 +192,21 @@ void fscrypt_destroy_inline_crypt_key(struct super_block *sb, struct fscrypt_prepared_key *prep_key) { struct blk_crypto_key *blk_key = prep_key->blk_key; - struct block_device **devs; + struct block_device *devs[FSCRYPT_MAX_DEVICES]; unsigned int num_devs; unsigned int i; if (!blk_key) return; - /* Evict the key from all the filesystem's block devices. */ - devs = fscrypt_get_devices(sb, &num_devs); - if (!IS_ERR(devs)) { - for (i = 0; i < num_devs; i++) - blk_crypto_evict_key(devs[i], blk_key); - kfree(devs); - } + /* + * Evict the key from all the filesystem's block devices. + * This *must* be done before the key is freed. + */ + num_devs = fscrypt_get_devices(sb, devs); + for (i = 0; i < num_devs; i++) + blk_crypto_evict_key(devs[i], blk_key); + kfree_sensitive(blk_key); } diff --git a/fs/crypto/keysetup_v1.c b/fs/crypto/keysetup_v1.c index c4d05168522b..7b2ee3c14d31 100644 --- a/fs/crypto/keysetup_v1.c +++ b/fs/crypto/keysetup_v1.c @@ -190,13 +190,19 @@ find_or_insert_direct_key(struct fscrypt_direct_key *to_insert, if (memcmp(ci->ci_policy.v1.master_key_descriptor, dk->dk_descriptor, FSCRYPT_KEY_DESCRIPTOR_SIZE) != 0) continue; + /* The sb is used at eviction time, so it must be the same. */ + if (ci->ci_inode->i_sb != dk->dk_sb) + continue; if (ci->ci_mode != dk->dk_mode) continue; if (!fscrypt_is_key_prepared(&dk->dk_key, ci)) continue; if (crypto_memneq(raw_key, dk->dk_raw, ci->ci_mode->keysize)) continue; - /* using existing tfm with same (descriptor, mode, raw_key) */ + /* + * Use an existing prepared key with the same (descriptor, sb, + * mode, inlinecrypt, raw_key) combination. + */ refcount_inc(&dk->dk_refcount); spin_unlock(&fscrypt_direct_keys_lock); free_direct_key(to_insert); diff --git a/fs/exec.c b/fs/exec.c index 4298e7e08d5d..6eb1c8da2c7d 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -736,7 +736,7 @@ int transfer_args_to_stack(struct linux_binprm *bprm, stop = bprm->p >> PAGE_SHIFT; sp = *sp_location; - for (index = MAX_ARG_PAGES - 1; index >= stop; index--) { + for (index = MAX_ARG_PAGES; index-- > stop; ) { unsigned int offset = index == stop ? bprm->p & ~PAGE_MASK : 0; char *src = kmap_local_page(bprm->page[index]) + offset; sp -= PAGE_SIZE - offset; diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index f6b75ce11d1c..6f787cbeefd0 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -3659,24 +3659,27 @@ static bool f2fs_has_stable_inodes(struct super_block *sb) return true; } -static struct block_device **f2fs_get_devices(struct super_block *sb, - unsigned int *num_devs) +static unsigned int +f2fs_get_devices(struct super_block *sb, + struct block_device *devs[FSCRYPT_MAX_DEVICES]) { struct f2fs_sb_info *sbi = F2FS_SB(sb); - struct block_device **devs; + int ndevs; int i; - if (!f2fs_is_multi_device(sbi)) - return NULL; + static_assert(MAX_DEVICES <= FSCRYPT_MAX_DEVICES); - devs = kmalloc_array(sbi->s_ndevs, sizeof(*devs), GFP_KERNEL); - if (!devs) - return ERR_PTR(-ENOMEM); + if (!f2fs_is_multi_device(sbi)) { + devs[0] = sb->s_bdev; + return 1; + } + ndevs = sbi->s_ndevs; + if (WARN_ON_ONCE(ndevs > FSCRYPT_MAX_DEVICES)) + ndevs = FSCRYPT_MAX_DEVICES; - for (i = 0; i < sbi->s_ndevs; i++) + for (i = 0; i < ndevs; i++) devs[i] = FDEV(i).bdev; - *num_devs = sbi->s_ndevs; - return devs; + return ndevs; } static const struct fscrypt_operations f2fs_cryptops = { diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c index 42c82b547659..6f74c345080f 100644 --- a/fs/fuse/dev_uring.c +++ b/fs/fuse/dev_uring.c @@ -998,15 +998,26 @@ static bool is_ring_ready(struct fuse_ring *ring, int current_qid) /* * fuse_uring_req_fetch command handling */ -static void fuse_uring_do_register(struct fuse_ring_ent *ent, - struct io_uring_cmd *cmd, - unsigned int issue_flags) +static int fuse_uring_do_register(struct fuse_ring_ent *ent, + struct io_uring_cmd *cmd, + unsigned int issue_flags) { struct fuse_ring_queue *queue = ent->queue; struct fuse_ring *ring = queue->ring; struct fuse_conn *fc = ring->fc; struct fuse_iqueue *fiq = &fc->iq; + spin_lock(&fc->lock); + /* abort teardown path is running or has run */ + if (!fc->connected) { + spin_unlock(&fc->lock); + if (atomic_dec_and_test(&ring->queue_refs)) + wake_up_all(&ring->stop_waitq); + kfree(ent); + return -ECONNABORTED; + } + spin_unlock(&fc->lock); + fuse_uring_prepare_cancel(cmd, issue_flags, ent); spin_lock(&queue->lock); @@ -1023,6 +1034,7 @@ static void fuse_uring_do_register(struct fuse_ring_ent *ent, wake_up_all(&fc->blocked_waitq); } } + return 0; } /* @@ -1138,9 +1150,7 @@ static int fuse_uring_register(struct io_uring_cmd *cmd, if (IS_ERR(ent)) return PTR_ERR(ent); - fuse_uring_do_register(ent, cmd, issue_flags); - - return 0; + return fuse_uring_do_register(ent, cmd, issue_flags); } /* diff --git a/fs/fuse/file.c b/fs/fuse/file.c index 79d1b502e731..585dd90361b6 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -2140,7 +2140,10 @@ static bool fuse_writepage_need_send(struct fuse_conn *fc, loff_t pos, WARN_ON(!ap->num_folios); - /* Reached max pages */ + /* Reached max pages or max folio slots */ + if (ap->num_folios >= fc->max_pages) + return true; + if ((bytes + PAGE_SIZE - 1) >> PAGE_SHIFT > fc->max_pages) return true; diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index c0fa6acf1937..c9ece378687b 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -47,11 +47,13 @@ static bool ifs_set_range_uptodate(struct folio *folio, struct iomap_folio_state *ifs, size_t off, size_t len) { struct inode *inode = folio->mapping->host; - unsigned int first_blk = off >> inode->i_blkbits; - unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; - unsigned int nr_blks = last_blk - first_blk + 1; + unsigned int first_blk, last_blk; - bitmap_set(ifs->state, first_blk, nr_blks); + if (len) { + first_blk = off >> inode->i_blkbits; + last_blk = (off + len - 1) >> inode->i_blkbits; + bitmap_set(ifs->state, first_blk, last_blk - first_blk + 1); + } return ifs_is_fully_uptodate(folio, ifs); } @@ -127,13 +129,17 @@ static void ifs_clear_range_dirty(struct folio *folio, { struct inode *inode = folio->mapping->host; unsigned int blks_per_folio = i_blocks_per_folio(inode, folio); - unsigned int first_blk = (off >> inode->i_blkbits); - unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; - unsigned int nr_blks = last_blk - first_blk + 1; + unsigned int first_blk = round_up(off, i_blocksize(inode)) >> + inode->i_blkbits; + unsigned int last_blk = (off + len) >> inode->i_blkbits; unsigned long flags; + if (first_blk >= last_blk) + return; + spin_lock_irqsave(&ifs->state_lock, flags); - bitmap_clear(ifs->state, first_blk + blks_per_folio, nr_blks); + bitmap_clear(ifs->state, first_blk + blks_per_folio, + last_blk - first_blk); spin_unlock_irqrestore(&ifs->state_lock, flags); } @@ -150,13 +156,17 @@ static void ifs_set_range_dirty(struct folio *folio, { struct inode *inode = folio->mapping->host; unsigned int blks_per_folio = i_blocks_per_folio(inode, folio); - unsigned int first_blk = (off >> inode->i_blkbits); - unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; - unsigned int nr_blks = last_blk - first_blk + 1; + unsigned int first_blk, last_blk; unsigned long flags; + if (!len) + return; + + first_blk = off >> inode->i_blkbits; + last_blk = (off + len - 1) >> inode->i_blkbits; spin_lock_irqsave(&ifs->state_lock, flags); - bitmap_set(ifs->state, first_blk + blks_per_folio, nr_blks); + bitmap_set(ifs->state, first_blk + blks_per_folio, + last_blk - first_blk + 1); spin_unlock_irqrestore(&ifs->state_lock, flags); } diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index a96029c967e6..1e158881c1af 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -4478,7 +4478,7 @@ static __be32 nfsd4_encode_splice_read( static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, struct nfsd4_read *read, - struct file *file, unsigned long maxcount) + unsigned long maxcount) { struct xdr_stream *xdr = resp->xdr; unsigned int base = xdr->buf->page_len & ~PAGE_MASK; @@ -4489,7 +4489,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, if (xdr_reserve_space_vec(xdr, maxcount) < 0) return nfserr_resource; - nfserr = nfsd_iter_read(resp->rqstp, read->rd_fhp, file, + nfserr = nfsd_iter_read(resp->rqstp, read->rd_fhp, read->rd_nf, read->rd_offset, &maxcount, base, &read->rd_eof); read->rd_length = maxcount; @@ -4536,7 +4536,7 @@ nfsd4_encode_read(struct nfsd4_compoundres *resp, __be32 nfserr, if (file->f_op->splice_read && splice_ok) nfserr = nfsd4_encode_splice_read(resp, read, file, maxcount); else - nfserr = nfsd4_encode_readv(resp, read, file, maxcount); + nfserr = nfsd4_encode_readv(resp, read, maxcount); if (nfserr) { xdr_truncate_encode(xdr, eof_offset); return nfserr; @@ -5432,7 +5432,7 @@ nfsd4_encode_read_plus_data(struct nfsd4_compoundres *resp, if (file->f_op->splice_read && splice_ok) nfserr = nfsd4_encode_splice_read(resp, read, file, maxcount); else - nfserr = nfsd4_encode_readv(resp, read, file, maxcount); + nfserr = nfsd4_encode_readv(resp, read, maxcount); if (nfserr) return nfserr; diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 203303723878..ba882a4ea9cb 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -1078,7 +1078,7 @@ __be32 nfsd_splice_read(struct svc_rqst *rqstp, struct svc_fh *fhp, * nfsd_iter_read - Perform a VFS read using an iterator * @rqstp: RPC transaction context * @fhp: file handle of file to be read - * @file: opened struct file of file to be read + * @nf: opened struct nfsd_file of file to be read * @offset: starting byte offset * @count: IN: requested number of bytes; OUT: number of bytes read * @base: offset in first page of read buffer @@ -1091,9 +1091,10 @@ __be32 nfsd_splice_read(struct svc_rqst *rqstp, struct svc_fh *fhp, * returned. */ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp, - struct file *file, loff_t offset, unsigned long *count, + struct nfsd_file *nf, loff_t offset, unsigned long *count, unsigned int base, u32 *eof) { + struct file *file = nf->nf_file; unsigned long v, total; struct iov_iter iter; struct kiocb kiocb; @@ -1202,7 +1203,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, unsigned long exp_op_flags = 0; unsigned int pflags = current->flags; bool restore_flags = false; - unsigned int nvecs; + int nvecs; trace_nfsd_write_opened(rqstp, fhp, offset, *cnt); @@ -1242,7 +1243,13 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, } nvecs = xdr_buf_to_bvec(rqstp->rq_bvec, rqstp->rq_maxpages, payload); + if (nvecs < 0) { + host_err = nvecs; + goto out_nfserr; + } + iov_iter_bvec(&iter, ITER_SOURCE, rqstp->rq_bvec, nvecs, *cnt); + since = READ_ONCE(file->f_wb_err); if (verf) nfsd_copy_write_verifier(verf, nn); @@ -1346,7 +1353,7 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp, if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp)) err = nfsd_splice_read(rqstp, fhp, file, offset, count, eof); else - err = nfsd_iter_read(rqstp, fhp, file, offset, count, 0, eof); + err = nfsd_iter_read(rqstp, fhp, nf, offset, count, 0, eof); nfsd_file_put(nf); trace_nfsd_read_done(rqstp, fhp, offset, *count); diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h index 9652c1de53ec..1dd3ae3ceb3a 100644 --- a/fs/nfsd/vfs.h +++ b/fs/nfsd/vfs.h @@ -122,7 +122,7 @@ __be32 nfsd_splice_read(struct svc_rqst *rqstp, struct svc_fh *fhp, unsigned long *count, u32 *eof); __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp, - struct file *file, loff_t offset, + struct nfsd_file *nf, loff_t offset, unsigned long *count, unsigned int base, u32 *eof); bool nfsd_read_splice_ok(struct svc_rqst *rqstp); diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c index 97f373fd9879..89317e1e7361 100644 --- a/fs/overlayfs/copy_up.c +++ b/fs/overlayfs/copy_up.c @@ -864,7 +864,7 @@ static int ovl_copy_up_tmpfile(struct ovl_copy_up_ctx *c) { struct ovl_fs *ofs = OVL_FS(c->dentry->d_sb); struct inode *udir = d_inode(c->destdir); - struct dentry *temp, *upper; + struct dentry *temp, *upper, *newdentry = NULL; struct file *tmpfile; struct ovl_cu_creds cc; int err; @@ -901,6 +901,14 @@ static int ovl_copy_up_tmpfile(struct ovl_copy_up_ctx *c) err = PTR_ERR(upper); if (!IS_ERR(upper)) { err = ovl_do_link(ofs, temp, udir, upper); + if (!err) { + /* + * Record the linked dentry -- not the disconnected + * O_TMPFILE dentry -- so that ->d_revalidate() on + * the upper fs sees the real parent/name. + */ + newdentry = dget(upper); + } dput(upper); } inode_unlock(udir); @@ -916,7 +924,7 @@ static int ovl_copy_up_tmpfile(struct ovl_copy_up_ctx *c) if (!c->metacopy) ovl_set_upperdata(d_inode(c->dentry)); - ovl_inode_update(d_inode(c->dentry), dget(temp)); + ovl_inode_update(d_inode(c->dentry), newdentry); out: ovl_end_write(c->dentry); diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 3fedfdddfa75..a857ce377b0c 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -541,9 +541,9 @@ static loff_t ovl_copyfile(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, loff_t len, unsigned int flags, enum ovl_copyop op) { + struct inode *inode_in = file_inode(file_in); struct inode *inode_out = file_inode(file_out); struct file *realfile_in, *realfile_out; - const struct cred *old_cred; loff_t ret; inode_lock(inode_out); @@ -565,25 +565,38 @@ static loff_t ovl_copyfile(struct file *file_in, loff_t pos_in, if (IS_ERR(realfile_in)) goto out_unlock; - old_cred = ovl_override_creds(file_inode(file_out)->i_sb); - switch (op) { - case OVL_COPY: - ret = vfs_copy_file_range(realfile_in, pos_in, - realfile_out, pos_out, len, flags); - break; - - case OVL_CLONE: - ret = vfs_clone_file_range(realfile_in, pos_in, - realfile_out, pos_out, len, flags); - break; - - case OVL_DEDUPE: - ret = vfs_dedupe_file_range_one(realfile_in, pos_in, - realfile_out, pos_out, len, - flags); - break; + /* + * For cross-sb copy, vfs_copy_file_range() will verify read access with + * the mounter creds of the dest fs mounter, so we need to explicitly + * verify read access with the source mounter creds. + */ + if (unlikely(inode_in->i_sb != inode_out->i_sb)) { + with_ovl_creds(inode_in->i_sb) { + ret = rw_verify_area(READ, realfile_in, &pos_in, len); + if (unlikely(ret)) + goto out_unlock; + } + } + + with_ovl_creds(inode_out->i_sb) { + switch (op) { + case OVL_COPY: + ret = vfs_copy_file_range(realfile_in, pos_in, + realfile_out, pos_out, len, flags); + break; + + case OVL_CLONE: + ret = vfs_clone_file_range(realfile_in, pos_in, + realfile_out, pos_out, len, flags); + break; + + case OVL_DEDUPE: + ret = vfs_dedupe_file_range_one(realfile_in, pos_in, + realfile_out, pos_out, len, + flags); + break; + } } - ovl_revert_creds(old_cred); /* Update size */ ovl_file_modified(file_out); diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h index d1eb1cbe7a45..696b13d9e1f1 100644 --- a/fs/overlayfs/overlayfs.h +++ b/fs/overlayfs/overlayfs.h @@ -445,6 +445,11 @@ struct dentry *ovl_workdir(struct dentry *dentry); const struct cred *ovl_override_creds(struct super_block *sb); void ovl_revert_creds(const struct cred *old_cred); +EXTEND_CLASS(override_creds, _ovl, ovl_override_creds(sb), struct super_block *sb) + +#define with_ovl_creds(sb) \ + scoped_class(override_creds_ovl, __UNIQUE_ID(label), sb) + static inline const struct cred *ovl_creds(struct super_block *sb) { return OVL_FS(sb)->creator_cred; diff --git a/fs/overlayfs/xattrs.c b/fs/overlayfs/xattrs.c index 88055deca936..c86f1a6c8b93 100644 --- a/fs/overlayfs/xattrs.c +++ b/fs/overlayfs/xattrs.c @@ -13,7 +13,7 @@ static bool ovl_is_escaped_xattr(struct super_block *sb, const char *name) OVL_XATTR_ESCAPE_USER_PREFIX_LEN) == 0; else return strncmp(name, OVL_XATTR_ESCAPE_TRUSTED_PREFIX, - OVL_XATTR_ESCAPE_TRUSTED_PREFIX_LEN - 1) == 0; + OVL_XATTR_ESCAPE_TRUSTED_PREFIX_LEN) == 0; } static bool ovl_is_own_xattr(struct super_block *sb, const char *name) diff --git a/fs/posix_acl.c b/fs/posix_acl.c index 4050942ab52f..0d92d6a631e1 100644 --- a/fs/posix_acl.c +++ b/fs/posix_acl.c @@ -93,6 +93,13 @@ static void __forget_cached_acl(struct posix_acl **p) { struct posix_acl *old; + /* + * ACL_DONT_CACHE is expected to be a "const" value and xchg it with + * ACL_NOT_CACHED would enable acl caching for the inode - + * clearly not what the caller has intended. + */ + if (READ_ONCE(*p) == ACL_DONT_CACHE) + return; old = xchg(p, ACL_NOT_CACHED); if (!is_uncached_acl(old)) posix_acl_release(old); diff --git a/fs/proc/namespaces.c b/fs/proc/namespaces.c index 2f46f1396744..ea6ec61a0430 100644 --- a/fs/proc/namespaces.c +++ b/fs/proc/namespaces.c @@ -46,7 +46,7 @@ static const char *proc_ns_get_link(struct dentry *dentry, const struct proc_ns_operations *ns_ops = PROC_I(inode)->ns_ops; struct task_struct *task; struct path ns_path; - int error = -EACCES; + int error; if (!dentry) return ERR_PTR(-ECHILD); @@ -59,6 +59,7 @@ static const char *proc_ns_get_link(struct dentry *dentry, if (error) goto out_put_task; + error = -EACCES; if (!ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS)) goto out; @@ -90,6 +91,7 @@ static int proc_ns_readlink(struct dentry *dentry, char __user *buffer, int bufl if (res) goto out_put_task; + res = -EACCES; if (ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS)) { res = ns_get_name(name, sizeof(name), task, ns_ops); if (res >= 0) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 38b9d47426ca..858b11009c26 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2485,15 +2485,19 @@ static void make_uffd_wp_huge_pte(struct vm_area_struct *vma, psize = huge_page_size(hstate_vma(vma)); - if (is_hugetlb_entry_migration(ptent)) + if (is_hugetlb_entry_migration(ptent)) { set_huge_pte_at(vma->vm_mm, addr, ptep, pte_swp_mkuffd_wp(ptent), psize); - else if (!huge_pte_none(ptent)) - huge_ptep_modify_prot_commit(vma, addr, ptep, ptent, - huge_pte_mkuffd_wp(ptent)); - else + } else if (!huge_pte_none(ptent)) { + pte_t old_pte, new_pte; + + old_pte = huge_ptep_modify_prot_start(vma, addr, ptep); + new_pte = huge_pte_mkuffd_wp(old_pte); + huge_ptep_modify_prot_commit(vma, addr, ptep, old_pte, new_pte); + } else { set_huge_pte_at(vma->vm_mm, addr, ptep, make_pte_marker(PTE_MARKER_UFFD_WP), psize); + } } #endif /* CONFIG_HUGETLB_PAGE */ diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c index 0d0ef54fc4de..eeb12ec594c7 100644 --- a/fs/resctrl/ctrlmondata.c +++ b/fs/resctrl/ctrlmondata.c @@ -547,7 +547,7 @@ struct rdt_domain_hdr *resctrl_find_domain(struct list_head *h, int id, } void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, - struct rdt_mon_domain *d, struct rdtgroup *rdtgrp, + struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, cpumask_t *cpumask, int evtid, int first) { int cpu; @@ -596,9 +596,9 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) struct kernfs_open_file *of = m->private; enum resctrl_res_level resid; enum resctrl_event_id evtid; + struct rdt_l3_mon_domain *d; struct rdt_domain_hdr *hdr; struct rmid_read rr = {0}; - struct rdt_mon_domain *d; struct rdtgroup *rdtgrp; int domid, cpu, ret = 0; struct rdt_resource *r; @@ -653,7 +653,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) ret = -ENOENT; goto out; } - d = container_of(hdr, struct rdt_mon_domain, hdr); + d = container_of(hdr, struct rdt_l3_mon_domain, hdr); mon_event_read(&rr, r, d, rdtgrp, &d->hdr.cpu_mask, evtid, false); } diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h index cf1fd82dc5a9..77f6e531e261 100644 --- a/fs/resctrl/internal.h +++ b/fs/resctrl/internal.h @@ -123,7 +123,7 @@ struct mon_data { struct rmid_read { struct rdtgroup *rgrp; struct rdt_resource *r; - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; enum resctrl_event_id evtid; bool first; struct cacheinfo *ci; @@ -351,23 +351,27 @@ int closids_supported(void); void closid_free(int closid); +int setup_rmid_lru_list(void); + +void free_rmid_lru_list(void); + int alloc_rmid(u32 closid); void free_rmid(u32 closid, u32 rmid); -void resctrl_mon_resource_exit(void); +int resctrl_l3_mon_resource_init(void); + +void resctrl_l3_mon_resource_exit(void); void mon_event_count(void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, - struct rdt_mon_domain *d, struct rdtgroup *rdtgrp, + struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, cpumask_t *cpumask, int evtid, int first); -int resctrl_mon_resource_init(void); - -void mbm_setup_overflow_handler(struct rdt_mon_domain *dom, +void mbm_setup_overflow_handler(struct rdt_l3_mon_domain *dom, unsigned long delay_ms, int exclude_cpu); @@ -375,14 +379,14 @@ void mbm_handle_overflow(struct work_struct *work); bool is_mba_sc(struct rdt_resource *r); -void cqm_setup_limbo_handler(struct rdt_mon_domain *dom, unsigned long delay_ms, +void cqm_setup_limbo_handler(struct rdt_l3_mon_domain *dom, unsigned long delay_ms, int exclude_cpu); void cqm_handle_limbo(struct work_struct *work); -bool has_busy_rmid(struct rdt_mon_domain *d); +bool has_busy_rmid(struct rdt_l3_mon_domain *d); -void __check_limbo(struct rdt_mon_domain *d, bool force_free); +void __check_limbo(struct rdt_l3_mon_domain *d, bool force_free); void resctrl_file_fflags_init(const char *config, unsigned long fflags); diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c index 7012a75814ac..55db47e969ce 100644 --- a/fs/resctrl/monitor.c +++ b/fs/resctrl/monitor.c @@ -130,7 +130,7 @@ static void limbo_release_entry(struct rmid_entry *entry) * decrement the count. If the busy count gets to zero on an RMID, we * free the RMID */ -void __check_limbo(struct rdt_mon_domain *d, bool force_free) +void __check_limbo(struct rdt_l3_mon_domain *d, bool force_free) { struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3); u32 idx_limit = resctrl_arch_system_num_rmid_idx(); @@ -193,7 +193,7 @@ void __check_limbo(struct rdt_mon_domain *d, bool force_free) resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx); } -bool has_busy_rmid(struct rdt_mon_domain *d) +bool has_busy_rmid(struct rdt_l3_mon_domain *d) { u32 idx_limit = resctrl_arch_system_num_rmid_idx(); @@ -294,7 +294,7 @@ int alloc_rmid(u32 closid) static void add_rmid_to_limbo(struct rmid_entry *entry) { struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3); - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; u32 idx; lockdep_assert_held(&rdtgroup_mutex); @@ -347,7 +347,7 @@ void free_rmid(u32 closid, u32 rmid) list_add_tail(&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state(struct rdt_mon_domain *d, u32 closid, +static struct mbm_state *get_mbm_state(struct rdt_l3_mon_domain *d, u32 closid, u32 rmid, enum resctrl_event_id evtid) { u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); @@ -367,7 +367,7 @@ static struct mbm_state *get_mbm_state(struct rdt_mon_domain *d, u32 closid, * Return: * Valid counter ID on success, or -ENOENT on failure. */ -static int mbm_cntr_get(struct rdt_resource *r, struct rdt_mon_domain *d, +static int mbm_cntr_get(struct rdt_resource *r, struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, enum resctrl_event_id evtid) { int cntr_id; @@ -394,7 +394,7 @@ static int mbm_cntr_get(struct rdt_resource *r, struct rdt_mon_domain *d, * Return: * Valid counter ID on success, or -ENOSPC on failure. */ -static int mbm_cntr_alloc(struct rdt_resource *r, struct rdt_mon_domain *d, +static int mbm_cntr_alloc(struct rdt_resource *r, struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, enum resctrl_event_id evtid) { int cntr_id; @@ -413,17 +413,17 @@ static int mbm_cntr_alloc(struct rdt_resource *r, struct rdt_mon_domain *d, /* * mbm_cntr_free() - Clear the counter ID configuration details in the domain @d. */ -static void mbm_cntr_free(struct rdt_mon_domain *d, int cntr_id) +static void mbm_cntr_free(struct rdt_l3_mon_domain *d, int cntr_id) { memset(&d->cntr_cfg[cntr_id], 0, sizeof(*d->cntr_cfg)); } -static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr) +static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr) { int cpu = smp_processor_id(); u32 closid = rdtgrp->closid; u32 rmid = rdtgrp->mon.rmid; - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; int cntr_id = -ENOENT; struct mbm_state *m; int err, ret; @@ -499,6 +499,17 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr) return ret; } +static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr) +{ + switch (rr->r->rid) { + case RDT_RESOURCE_L3: + return __l3_mon_event_count(rdtgrp, rr); + default: + rr->err = -EINVAL; + return -EINVAL; + } +} + /* * mbm_bw_count() - Update bw count from values previously read by * __mon_event_count(). @@ -617,7 +628,7 @@ static struct rdt_ctrl_domain *get_ctrl_domain_from_cpu(int cpu, * throttle MSRs already have low percentage values. To avoid * unnecessarily restricting such rdtgroups, we also increase the bandwidth. */ -static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_mon_domain *dom_mbm) +static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_l3_mon_domain *dom_mbm) { u32 closid, rmid, cur_msr_val, new_msr_val; struct mbm_state *pmbm_data, *cmbm_data; @@ -685,7 +696,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_mon_domain *dom_mbm) resctrl_arch_update_one(r_mba, dom_mba, closid, CDP_NONE, new_msr_val); } -static void mbm_update_one_event(struct rdt_resource *r, struct rdt_mon_domain *d, +static void mbm_update_one_event(struct rdt_resource *r, struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, enum resctrl_event_id evtid) { struct rmid_read rr = {0}; @@ -717,7 +728,7 @@ static void mbm_update_one_event(struct rdt_resource *r, struct rdt_mon_domain * resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx); } -static void mbm_update(struct rdt_resource *r, struct rdt_mon_domain *d, +static void mbm_update(struct rdt_resource *r, struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp) { /* @@ -738,12 +749,12 @@ static void mbm_update(struct rdt_resource *r, struct rdt_mon_domain *d, void cqm_handle_limbo(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL); - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; cpus_read_lock(); mutex_lock(&rdtgroup_mutex); - d = container_of(work, struct rdt_mon_domain, cqm_limbo.work); + d = container_of(work, struct rdt_l3_mon_domain, cqm_limbo.work); __check_limbo(d, false); @@ -766,7 +777,7 @@ void cqm_handle_limbo(struct work_struct *work) * @exclude_cpu: Which CPU the handler should not run on, * RESCTRL_PICK_ANY_CPU to pick any CPU. */ -void cqm_setup_limbo_handler(struct rdt_mon_domain *dom, unsigned long delay_ms, +void cqm_setup_limbo_handler(struct rdt_l3_mon_domain *dom, unsigned long delay_ms, int exclude_cpu) { unsigned long delay = msecs_to_jiffies(delay_ms); @@ -783,7 +794,7 @@ void mbm_handle_overflow(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL); struct rdtgroup *prgrp, *crgrp; - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; struct list_head *head; struct rdt_resource *r; @@ -798,7 +809,7 @@ void mbm_handle_overflow(struct work_struct *work) goto out_unlock; r = resctrl_arch_get_resource(RDT_RESOURCE_L3); - d = container_of(work, struct rdt_mon_domain, mbm_over.work); + d = container_of(work, struct rdt_l3_mon_domain, mbm_over.work); list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) { mbm_update(r, d, prgrp); @@ -832,7 +843,7 @@ void mbm_handle_overflow(struct work_struct *work) * @exclude_cpu: Which CPU the handler should not run on, * RESCTRL_PICK_ANY_CPU to pick any CPU. */ -void mbm_setup_overflow_handler(struct rdt_mon_domain *dom, unsigned long delay_ms, +void mbm_setup_overflow_handler(struct rdt_l3_mon_domain *dom, unsigned long delay_ms, int exclude_cpu) { unsigned long delay = msecs_to_jiffies(delay_ms); @@ -851,42 +862,29 @@ void mbm_setup_overflow_handler(struct rdt_mon_domain *dom, unsigned long delay_ schedule_delayed_work_on(cpu, &dom->mbm_over, delay); } -static int dom_data_init(struct rdt_resource *r) +int setup_rmid_lru_list(void) { - u32 idx_limit = resctrl_arch_system_num_rmid_idx(); - u32 num_closid = resctrl_arch_get_num_closid(r); struct rmid_entry *entry = NULL; - int err = 0, i; + u32 idx_limit; u32 idx; + int i; - mutex_lock(&rdtgroup_mutex); - if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { - u32 *tmp; - - /* - * If the architecture hasn't provided a sanitised value here, - * this may result in larger arrays than necessary. Resctrl will - * use a smaller system wide value based on the resources in - * use. - */ - tmp = kcalloc(num_closid, sizeof(*tmp), GFP_KERNEL); - if (!tmp) { - err = -ENOMEM; - goto out_unlock; - } + if (!resctrl_arch_mon_capable()) + return 0; - closid_num_dirty_rmid = tmp; - } + /* + * Called on every mount, but the number of RMIDs cannot change + * after the first mount, so keep using the same set of rmid_ptrs[] + * until resctrl_exit(). Note that the limbo handler continues to + * access rmid_ptrs[] after resctrl is unmounted. + */ + if (rmid_ptrs) + return 0; + idx_limit = resctrl_arch_system_num_rmid_idx(); rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL); - if (!rmid_ptrs) { - if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { - kfree(closid_num_dirty_rmid); - closid_num_dirty_rmid = NULL; - } - err = -ENOMEM; - goto out_unlock; - } + if (!rmid_ptrs) + return -ENOMEM; for (i = 0; i < idx_limit; i++) { entry = &rmid_ptrs[i]; @@ -899,35 +897,24 @@ static int dom_data_init(struct rdt_resource *r) /* * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and * are always allocated. These are used for the rdtgroup_default - * control group, which will be setup later in resctrl_init(). + * control group, which was setup earlier in rdtgroup_setup_default(). */ idx = resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID, RESCTRL_RESERVED_RMID); entry = __rmid_entry(idx); list_del(&entry->list); -out_unlock: - mutex_unlock(&rdtgroup_mutex); - - return err; + return 0; } -static void dom_data_exit(struct rdt_resource *r) +void free_rmid_lru_list(void) { - mutex_lock(&rdtgroup_mutex); - - if (!r->mon_capable) - goto out_unlock; - - if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { - kfree(closid_num_dirty_rmid); - closid_num_dirty_rmid = NULL; - } + if (!resctrl_arch_mon_capable()) + return; + mutex_lock(&rdtgroup_mutex); kfree(rmid_ptrs); rmid_ptrs = NULL; - -out_unlock: mutex_unlock(&rdtgroup_mutex); } @@ -1087,7 +1074,7 @@ ssize_t resctrl_mbm_assign_on_mkdir_write(struct kernfs_open_file *of, char *buf * mbm_cntr_free_all() - Clear all the counter ID configuration details in the * domain @d. Called when mbm_assign_mode is changed. */ -static void mbm_cntr_free_all(struct rdt_resource *r, struct rdt_mon_domain *d) +static void mbm_cntr_free_all(struct rdt_resource *r, struct rdt_l3_mon_domain *d) { memset(d->cntr_cfg, 0, sizeof(*d->cntr_cfg) * r->mon.num_mbm_cntrs); } @@ -1096,7 +1083,7 @@ static void mbm_cntr_free_all(struct rdt_resource *r, struct rdt_mon_domain *d) * resctrl_reset_rmid_all() - Reset all non-architecture states for all the * supported RMIDs. */ -static void resctrl_reset_rmid_all(struct rdt_resource *r, struct rdt_mon_domain *d) +static void resctrl_reset_rmid_all(struct rdt_resource *r, struct rdt_l3_mon_domain *d) { u32 idx_limit = resctrl_arch_system_num_rmid_idx(); enum resctrl_event_id evt; @@ -1117,7 +1104,7 @@ static void resctrl_reset_rmid_all(struct rdt_resource *r, struct rdt_mon_domain * Assign the counter if @assign is true else unassign the counter. Reset the * associated non-architectural state. */ -static void rdtgroup_assign_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, +static void rdtgroup_assign_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d, enum resctrl_event_id evtid, u32 rmid, u32 closid, u32 cntr_id, bool assign) { @@ -1137,7 +1124,7 @@ static void rdtgroup_assign_cntr(struct rdt_resource *r, struct rdt_mon_domain * * Return: * 0 on success, < 0 on failure. */ -static int rdtgroup_alloc_assign_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, +static int rdtgroup_alloc_assign_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, struct mon_evt *mevt) { int cntr_id; @@ -1172,7 +1159,7 @@ static int rdtgroup_alloc_assign_cntr(struct rdt_resource *r, struct rdt_mon_dom * Return: * 0 on success, < 0 on failure. */ -static int rdtgroup_assign_cntr_event(struct rdt_mon_domain *d, struct rdtgroup *rdtgrp, +static int rdtgroup_assign_cntr_event(struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, struct mon_evt *mevt) { struct rdt_resource *r = resctrl_arch_get_resource(mevt->rid); @@ -1222,7 +1209,7 @@ void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp) * rdtgroup_free_unassign_cntr() - Unassign and reset the counter ID configuration * for the event pointed to by @mevt within the domain @d and resctrl group @rdtgrp. */ -static void rdtgroup_free_unassign_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, +static void rdtgroup_free_unassign_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, struct mon_evt *mevt) { int cntr_id; @@ -1243,7 +1230,7 @@ static void rdtgroup_free_unassign_cntr(struct rdt_resource *r, struct rdt_mon_d * the event structure @mevt from the domain @d and the group @rdtgrp. Unassign * the counters from all the domains if @d is NULL else unassign from @d. */ -static void rdtgroup_unassign_cntr_event(struct rdt_mon_domain *d, struct rdtgroup *rdtgrp, +static void rdtgroup_unassign_cntr_event(struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, struct mon_evt *mevt) { struct rdt_resource *r = resctrl_arch_get_resource(mevt->rid); @@ -1318,7 +1305,7 @@ static int resctrl_parse_mem_transactions(char *tok, u32 *val) static void rdtgroup_update_cntr_event(struct rdt_resource *r, struct rdtgroup *rdtgrp, enum resctrl_event_id evtid) { - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; int cntr_id; list_for_each_entry(d, &r->mon_domains, hdr.list) { @@ -1424,7 +1411,7 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off) { struct rdt_resource *r = rdt_kn_parent_priv(of->kn); - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; int ret = 0; bool enable; @@ -1497,7 +1484,7 @@ int resctrl_num_mbm_cntrs_show(struct kernfs_open_file *of, struct seq_file *s, void *v) { struct rdt_resource *r = rdt_kn_parent_priv(of->kn); - struct rdt_mon_domain *dom; + struct rdt_l3_mon_domain *dom; bool sep = false; cpus_read_lock(); @@ -1521,7 +1508,7 @@ int resctrl_available_mbm_cntrs_show(struct kernfs_open_file *of, struct seq_file *s, void *v) { struct rdt_resource *r = rdt_kn_parent_priv(of->kn); - struct rdt_mon_domain *dom; + struct rdt_l3_mon_domain *dom; bool sep = false; u32 cntrs, i; int ret = 0; @@ -1562,7 +1549,7 @@ int resctrl_available_mbm_cntrs_show(struct kernfs_open_file *of, int mbm_L3_assignments_show(struct kernfs_open_file *of, struct seq_file *s, void *v) { struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3); - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; struct rdtgroup *rdtgrp; struct mon_evt *mevt; int ret = 0; @@ -1625,7 +1612,7 @@ static struct mon_evt *mbm_get_mon_event_by_name(struct rdt_resource *r, char *n return NULL; } -static int rdtgroup_modify_assign_state(char *assign, struct rdt_mon_domain *d, +static int rdtgroup_modify_assign_state(char *assign, struct rdt_l3_mon_domain *d, struct rdtgroup *rdtgrp, struct mon_evt *mevt) { int ret = 0; @@ -1651,7 +1638,7 @@ static int rdtgroup_modify_assign_state(char *assign, struct rdt_mon_domain *d, static int resctrl_parse_mbm_assignment(struct rdt_resource *r, struct rdtgroup *rdtgrp, char *event, char *tok) { - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; unsigned long dom_id = 0; char *dom_str, *id_str; struct mon_evt *mevt; @@ -1746,19 +1733,59 @@ ssize_t mbm_L3_assignments_write(struct kernfs_open_file *of, char *buf, return ret ?: nbytes; } +static int closid_num_dirty_rmid_alloc(struct rdt_resource *r) +{ + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { + u32 num_closid = resctrl_arch_get_num_closid(r); + u32 *tmp; + + /* For ARM memory ordering access to closid_num_dirty_rmid */ + mutex_lock(&rdtgroup_mutex); + + /* + * If the architecture hasn't provided a sanitised value here, + * this may result in larger arrays than necessary. Resctrl will + * use a smaller system wide value based on the resources in + * use. + */ + tmp = kcalloc(num_closid, sizeof(*tmp), GFP_KERNEL); + if (!tmp) { + mutex_unlock(&rdtgroup_mutex); + return -ENOMEM; + } + + closid_num_dirty_rmid = tmp; + + mutex_unlock(&rdtgroup_mutex); + } + + return 0; +} + +static void closid_num_dirty_rmid_free(void) +{ + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { + mutex_lock(&rdtgroup_mutex); + kfree(closid_num_dirty_rmid); + closid_num_dirty_rmid = NULL; + mutex_unlock(&rdtgroup_mutex); + } +} + /** - * resctrl_mon_resource_init() - Initialise global monitoring structures. + * resctrl_l3_mon_resource_init() - Initialise global monitoring structures. * * Allocate and initialise global monitor resources that do not belong to a - * specific domain. i.e. the rmid_ptrs[] used for the limbo and free lists. + * specific domain. i.e. the closid_num_dirty_rmid[] used to find the CLOSID + * with the cleanest set of RMIDs. * Called once during boot after the struct rdt_resource's have been configured * but before the filesystem is mounted. * Resctrl's cpuhp callbacks may be called before this point to bring a domain * online. * - * Returns 0 for success, or -ENOMEM. + * Return: 0 for success, or -ENOMEM. */ -int resctrl_mon_resource_init(void) +int resctrl_l3_mon_resource_init(void) { struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3); int ret; @@ -1766,7 +1793,7 @@ int resctrl_mon_resource_init(void) if (!r->mon_capable) return 0; - ret = dom_data_init(r); + ret = closid_num_dirty_rmid_alloc(r); if (ret) return ret; @@ -1808,9 +1835,12 @@ int resctrl_mon_resource_init(void) return 0; } -void resctrl_mon_resource_exit(void) +void resctrl_l3_mon_resource_exit(void) { struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3); - dom_data_exit(r); + if (!r->mon_capable) + return; + + closid_num_dirty_rmid_free(); } diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c index 7cfb0086451c..08e26c0c9fb9 100644 --- a/fs/resctrl/rdtgroup.c +++ b/fs/resctrl/rdtgroup.c @@ -581,14 +581,20 @@ static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of, * * On resource group creation via a mkdir, an extra kernfs_node reference is * taken to ensure that the rdtgroup structure remains accessible for the - * rdtgroup_kn_unlock() calls where it is removed. + * rdtgroup_kn_unlock() calls where it is removed. The default group is + * statically allocated: it does not have an extra reference but will have + * RDT_DELETED set on unmount to support safe access to its associated files + * via rdtgroup_kn_lock_live/rdtgroup_kn_unlock(). * - * Drop the extra reference here, then free the rdtgroup structure. + * For all but the default group: drop the extra reference, then free the + * rdtgroup structure. * * Return: void */ static void rdtgroup_remove(struct rdtgroup *rdtgrp) { + if (rdtgrp == &rdtgroup_default) + return; kernfs_put(rdtgrp->kn); kfree(rdtgrp); } @@ -1620,7 +1626,7 @@ static void mondata_config_read(struct resctrl_mon_config_info *mon_info) static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid) { struct resctrl_mon_config_info mon_info; - struct rdt_mon_domain *dom; + struct rdt_l3_mon_domain *dom; bool sep = false; cpus_read_lock(); @@ -1668,7 +1674,7 @@ static int mbm_local_bytes_config_show(struct kernfs_open_file *of, } static void mbm_config_write_domain(struct rdt_resource *r, - struct rdt_mon_domain *d, u32 evtid, u32 val) + struct rdt_l3_mon_domain *d, u32 evtid, u32 val) { struct resctrl_mon_config_info mon_info = {0}; @@ -1710,7 +1716,7 @@ static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid) { char *dom_str = NULL, *id_str; unsigned long dom_id, val; - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; /* Walking r->domains, ensure it can't race with cpuhp */ lockdep_assert_cpus_held(); @@ -2718,7 +2724,7 @@ static int rdt_get_tree(struct fs_context *fc) { struct rdt_fs_context *ctx = rdt_fc2context(fc); unsigned long flags = RFTYPE_CTRL_BASE; - struct rdt_mon_domain *dom; + struct rdt_l3_mon_domain *dom; struct rdt_resource *r; int ret; @@ -2732,6 +2738,16 @@ static int rdt_get_tree(struct fs_context *fc) goto out; } + /* Avoid races from pending operations from a previous mount */ + if (atomic_read(&rdtgroup_default.waitcount) != 0) { + ret = -EBUSY; + goto out; + } + + ret = setup_rmid_lru_list(); + if (ret) + goto out; + ret = rdtgroup_setup_root(ctx); if (ret) goto out; @@ -3090,6 +3106,7 @@ static void resctrl_fs_teardown(void) mon_put_kn_priv(); rdt_pseudo_lock_release(); rdtgroup_default.mode = RDT_MODE_SHAREABLE; + rdtgroup_default.flags = RDT_DELETED; closid_exit(); schemata_list_destroy(); rdtgroup_destroy_root(); @@ -3169,7 +3186,7 @@ static void mon_rmdir_one_subdir(struct kernfs_node *pkn, char *name, char *subn * when last domain being summed is removed. */ static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r, - struct rdt_mon_domain *d) + struct rdt_l3_mon_domain *d) { struct rdtgroup *prgrp, *crgrp; char subname[32]; @@ -3189,7 +3206,7 @@ static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r, } } -static int mon_add_all_files(struct kernfs_node *kn, struct rdt_mon_domain *d, +static int mon_add_all_files(struct kernfs_node *kn, struct rdt_l3_mon_domain *d, struct rdt_resource *r, struct rdtgroup *prgrp, bool do_sum) { @@ -3218,7 +3235,7 @@ static int mon_add_all_files(struct kernfs_node *kn, struct rdt_mon_domain *d, } static int mkdir_mondata_subdir(struct kernfs_node *parent_kn, - struct rdt_mon_domain *d, + struct rdt_l3_mon_domain *d, struct rdt_resource *r, struct rdtgroup *prgrp) { struct kernfs_node *kn, *ckn; @@ -3280,7 +3297,7 @@ static int mkdir_mondata_subdir(struct kernfs_node *parent_kn, * and "monitor" groups with given domain id. */ static void mkdir_mondata_subdir_allrdtgrp(struct rdt_resource *r, - struct rdt_mon_domain *d) + struct rdt_l3_mon_domain *d) { struct kernfs_node *parent_kn; struct rdtgroup *prgrp, *crgrp; @@ -3302,7 +3319,7 @@ static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn, struct rdt_resource *r, struct rdtgroup *prgrp) { - struct rdt_mon_domain *dom; + struct rdt_l3_mon_domain *dom; int ret; /* Walking r->domains, ensure it can't race with cpuhp */ @@ -4144,6 +4161,7 @@ static int rdtgroup_setup_root(struct rdt_fs_context *ctx) ctx->kfc.root = rdt_root; rdtgroup_default.kn = kernfs_root_to_node(rdt_root); + rdtgroup_default.flags = 0; return 0; } @@ -4170,7 +4188,7 @@ static void rdtgroup_setup_default(void) mutex_unlock(&rdtgroup_mutex); } -static void domain_destroy_mon_state(struct rdt_mon_domain *d) +static void domain_destroy_l3_mon_state(struct rdt_l3_mon_domain *d) { int idx; @@ -4192,7 +4210,7 @@ void resctrl_offline_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain mutex_unlock(&rdtgroup_mutex); } -void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d) +void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_l3_mon_domain *d) { mutex_lock(&rdtgroup_mutex); @@ -4218,13 +4236,13 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d cancel_delayed_work(&d->cqm_limbo); } - domain_destroy_mon_state(d); + domain_destroy_l3_mon_state(d); mutex_unlock(&rdtgroup_mutex); } /** - * domain_setup_mon_state() - Initialise domain monitoring structures. + * domain_setup_l3_mon_state() - Initialise domain monitoring structures. * @r: The resource for the newly online domain. * @d: The newly online domain. * @@ -4232,11 +4250,11 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d * Called when the first CPU of a domain comes online, regardless of whether * the filesystem is mounted. * During boot this may be called before global allocations have been made by - * resctrl_mon_resource_init(). + * resctrl_l3_mon_resource_init(). * - * Returns 0 for success, or -ENOMEM. + * Return: 0 for success, or -ENOMEM. */ -static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_mon_domain *d) +static int domain_setup_l3_mon_state(struct rdt_resource *r, struct rdt_l3_mon_domain *d) { u32 idx_limit = resctrl_arch_system_num_rmid_idx(); size_t tsize = sizeof(*d->mbm_states[0]); @@ -4292,13 +4310,13 @@ int resctrl_online_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain *d return err; } -int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d) +int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_l3_mon_domain *d) { int err; mutex_lock(&rdtgroup_mutex); - err = domain_setup_mon_state(r, d); + err = domain_setup_l3_mon_state(r, d); if (err) goto out_unlock; @@ -4344,10 +4362,10 @@ static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) } } -static struct rdt_mon_domain *get_mon_domain_from_cpu(int cpu, +static struct rdt_l3_mon_domain *get_mon_domain_from_cpu(int cpu, struct rdt_resource *r) { - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; lockdep_assert_cpus_held(); @@ -4363,7 +4381,7 @@ static struct rdt_mon_domain *get_mon_domain_from_cpu(int cpu, void resctrl_offline_cpu(unsigned int cpu) { struct rdt_resource *l3 = resctrl_arch_get_resource(RDT_RESOURCE_L3); - struct rdt_mon_domain *d; + struct rdt_l3_mon_domain *d; struct rdtgroup *rdtgrp; mutex_lock(&rdtgroup_mutex); @@ -4413,13 +4431,13 @@ int resctrl_init(void) thread_throttle_mode_init(); - ret = resctrl_mon_resource_init(); + ret = resctrl_l3_mon_resource_init(); if (ret) return ret; ret = sysfs_create_mount_point(fs_kobj, "resctrl"); if (ret) { - resctrl_mon_resource_exit(); + resctrl_l3_mon_resource_exit(); return ret; } @@ -4454,7 +4472,7 @@ int resctrl_init(void) cleanup_mountpoint: sysfs_remove_mount_point(fs_kobj, "resctrl"); - resctrl_mon_resource_exit(); + resctrl_l3_mon_resource_exit(); return ret; } @@ -4490,7 +4508,7 @@ static bool resctrl_online_domains_exist(void) * When called by the architecture code, all CPUs and resctrl domains must be * offline. This ensures the limbo and overflow handlers are not scheduled to * run, meaning the data structures they access can be freed by - * resctrl_mon_resource_exit(). + * resctrl_l3_mon_resource_exit(). * * After resctrl_exit() returns, the architecture code should return an * error from all resctrl_arch_ functions that can do this. @@ -4517,5 +4535,6 @@ void resctrl_exit(void) * it can be used to umount resctrl. */ - resctrl_mon_resource_exit(); + resctrl_l3_mon_resource_exit(); + free_rmid_lru_list(); } diff --git a/fs/smb/client/misc.c b/fs/smb/client/misc.c index f37843711312..d9771d224527 100644 --- a/fs/smb/client/misc.c +++ b/fs/smb/client/misc.c @@ -957,6 +957,8 @@ parse_dfs_referrals(struct get_dfs_referral_rsp *rsp, u32 rsp_size, int i, rc = 0; char *data_end; struct dfs_referral_level_3 *ref; + unsigned int path_consumed; + size_t search_name_len; if (rsp_size < sizeof(*rsp)) { cifs_dbg(VFS | ONCE, @@ -1004,6 +1006,7 @@ parse_dfs_referrals(struct get_dfs_referral_rsp *rsp, u32 rsp_size, rc = -ENOMEM; goto parse_DFS_referrals_exit; } + search_name_len = strlen(searchName); /* collect necessary data from referrals */ for (i = 0; i < *num_of_nodes; i++) { @@ -1012,21 +1015,34 @@ parse_dfs_referrals(struct get_dfs_referral_rsp *rsp, u32 rsp_size, struct dfs_info3_param *node = (*target_nodes)+i; node->flags = le32_to_cpu(rsp->DFSFlags); + path_consumed = le16_to_cpu(rsp->PathConsumed); if (is_unicode) { - __le16 *tmp = kmalloc(strlen(searchName)*2 + 2, - GFP_KERNEL); - if (tmp == NULL) { + size_t search_name_utf16_len = search_name_len * 2 + 2; + __le16 *tmp; + + if (path_consumed > search_name_utf16_len) { + rc = -EINVAL; + goto parse_DFS_referrals_exit; + } + + tmp = kmalloc(search_name_utf16_len, GFP_KERNEL); + if (!tmp) { rc = -ENOMEM; goto parse_DFS_referrals_exit; } - cifsConvertToUTF16((__le16 *) tmp, searchName, + cifsConvertToUTF16((__le16 *)tmp, searchName, PATH_MAX, nls_codepage, remap); - node->path_consumed = cifs_utf16_bytes(tmp, - le16_to_cpu(rsp->PathConsumed), - nls_codepage); + node->path_consumed = cifs_utf16_bytes(tmp, path_consumed, + nls_codepage); kfree(tmp); - } else - node->path_consumed = le16_to_cpu(rsp->PathConsumed); + } else { + if (path_consumed > search_name_len) { + rc = -EINVAL; + goto parse_DFS_referrals_exit; + } + + node->path_consumed = path_consumed; + } node->server_type = le16_to_cpu(ref->ServerType); node->ref_flag = le16_to_cpu(ref->ReferralEntryFlags); diff --git a/fs/smb/client/smb2file.c b/fs/smb/client/smb2file.c index cc1ff2461b00..9bab3c57b3a4 100644 --- a/fs/smb/client/smb2file.c +++ b/fs/smb/client/smb2file.c @@ -30,6 +30,19 @@ static struct smb2_symlink_err_rsp *symlink_data(const struct kvec *iov) u8 *end = (u8 *)err + iov->iov_len; u32 len; + /* + * Per [MS-SMB2] section 2.2.2, a STATUS_STOPPED_ON_SYMLINK response has to + * carry a Symbolic Link Error Response, so ByteCount cannot be zero. Some + * servers (e.g. the macOS built-in SMB server) violate this and return an + * empty error response, with both ErrorContextCount and ByteCount set to + * zero, i.e. without the symlink target. Detect this and return -ENODATA + * so that callers can tell "server did not send the target" apart from a + * malformed response, and retrieve the target with FSCTL_GET_REPARSE_POINT + * instead. + */ + if (!err->ErrorContextCount && !le32_to_cpu(err->ByteCount)) + return ERR_PTR(-ENODATA); + if (err->ErrorContextCount) { struct smb2_error_context_rsp *p; @@ -200,6 +213,14 @@ int smb2_open_file(const unsigned int xid, struct cifs_open_parms *oparms, __u32 rc = smb2_parse_symlink_response(oparms->cifs_sb, &err_iov, oparms->path, &data->symlink_target); + /* + * If smb2_parse_symlink_response returned -ENODATA then the + * symlink_target was not sent. Treat this as if the SMB2_open() + * failed with STATUS_IO_REPARSE_TAG_NOT_HANDLED status, which is + * indicated by the -EIO errno. + */ + if (rc == -ENODATA) + rc = -EIO; if (!rc) { memset(smb2_data, 0, sizeof(*smb2_data)); oparms->create_options |= OPEN_REPARSE_POINT; diff --git a/fs/smb/client/smb2inode.c b/fs/smb/client/smb2inode.c index e13bc8a97c44..f1de84990aed 100644 --- a/fs/smb/client/smb2inode.c +++ b/fs/smb/client/smb2inode.c @@ -896,9 +896,19 @@ static int parse_create_response(struct cifs_open_info_data *data, rc = smb2_parse_symlink_response(cifs_sb, iov, full_path, &data->symlink_target); - if (rc) + if (rc != 0 && rc != -ENODATA) return rc; - tag = IO_REPARSE_TAG_SYMLINK; + /* + * -ENODATA means that the response was parsed but did not contain + * the symlink target at all (see symlink_data()). Treat it like + * STATUS_IO_REPARSE_TAG_NOT_HANDLED, which does not contain it + * either: leave the tag unset and clear rc, so that the caller + * retrieves the target with SMB2_OP_GET_REPARSE. + */ + if (rc == -ENODATA) + rc = 0; + else + tag = IO_REPARSE_TAG_SYMLINK; reparse_point = true; break; case STATUS_SUCCESS: @@ -1091,7 +1101,14 @@ int smb2_query_path_info(const unsigned int xid, rc = -EOPNOTSUPP; } - if (data->reparse.tag == IO_REPARSE_TAG_SYMLINK && !rc) { + /* + * If the symlink was already parsed in create response then it is needed to fix + * its type now (after the second call with OPEN_REPARSE_POINT which filled the + * data->fi.Attributes). If the symlink was not parsed in create response then + * the data->symlink_target was not filled yet and then the type will be fixed + * later after data->symlink_target is filled. + */ + if (data->reparse.tag == IO_REPARSE_TAG_SYMLINK && !rc && data->symlink_target) { bool directory = le32_to_cpu(data->fi.Attributes) & ATTR_DIRECTORY; rc = smb2_fix_symlink_target_type(&data->symlink_target, directory, cifs_sb); } diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c index 786a7e963f1f..5bbe98dc0529 100644 --- a/fs/smb/client/smb2ops.c +++ b/fs/smb/client/smb2ops.c @@ -3546,6 +3546,7 @@ static int smb3_simple_fallocate_range(unsigned int xid, struct file_allocated_range_buffer in_data, *out_data = NULL, *tmp_data; u32 out_data_len; char *buf = NULL; + u64 range_start, range_len, range_end; loff_t l; int rc; @@ -3582,13 +3583,21 @@ static int smb3_simple_fallocate_range(unsigned int xid, goto out; } - if (off < le64_to_cpu(tmp_data->file_offset)) { + range_start = le64_to_cpu(tmp_data->file_offset); + range_len = le64_to_cpu(tmp_data->length); + if (check_add_overflow(range_start, range_len, &range_end) || + range_end > S64_MAX) { + rc = -EINVAL; + goto out; + } + + if (off < range_start) { /* * We are at a hole. Write until the end of the region * or until the next allocated data, * whichever comes next. */ - l = le64_to_cpu(tmp_data->file_offset) - off; + l = range_start - off; if (len < l) l = len; rc = smb3_simple_fallocate_write_range(xid, tcon, @@ -3605,11 +3614,13 @@ static int smb3_simple_fallocate_range(unsigned int xid, * until the end of the data or the end of the region * we are supposed to fallocate, whichever comes first. */ - l = le64_to_cpu(tmp_data->length); - if (len < l) - l = len; - off += l; - len -= l; + if (off < range_end) { + l = range_end - off; + if (len < l) + l = len; + off += l; + len -= l; + } tmp_data = &tmp_data[1]; out_data_len -= sizeof(struct file_allocated_range_buffer); diff --git a/fs/smb/server/oplock.c b/fs/smb/server/oplock.c index a019c4496ae4..ff0dbf3f8cbf 100644 --- a/fs/smb/server/oplock.c +++ b/fs/smb/server/oplock.c @@ -705,6 +705,7 @@ static void __smb2_oplock_break_noti(struct work_struct *wk) out: ksmbd_free_work_struct(work); ksmbd_conn_r_count_dec(conn); + ksmbd_conn_put(conn); } /** @@ -740,7 +741,7 @@ static int smb2_oplock_break_noti(struct oplock_info *opinfo) br_info->open_trunc = opinfo->open_trunc; work->request_buf = (char *)br_info; - work->conn = conn; + work->conn = ksmbd_conn_get(conn); work->sess = opinfo->sess; ksmbd_conn_r_count_inc(conn); @@ -814,6 +815,7 @@ static void __smb2_lease_break_noti(struct work_struct *wk) out: ksmbd_free_work_struct(work); ksmbd_conn_r_count_dec(conn); + ksmbd_conn_put(conn); } /** @@ -853,7 +855,7 @@ static int smb2_lease_break_noti(struct oplock_info *opinfo) memcpy(br_info->lease_key, lease->lease_key, SMB2_LEASE_KEY_SIZE); work->request_buf = (char *)br_info; - work->conn = conn; + work->conn = ksmbd_conn_get(conn); work->sess = opinfo->sess; ksmbd_conn_r_count_inc(conn); diff --git a/fs/smb/server/smb2misc.c b/fs/smb/server/smb2misc.c index 67a2d7a793f6..b11d854d3fcf 100644 --- a/fs/smb/server/smb2misc.c +++ b/fs/smb/server/smb2misc.c @@ -400,6 +400,11 @@ int ksmbd_smb2_check_message(struct ksmbd_work *work) return 1; } + if (len < __SMB2_HEADER_STRUCTURE_SIZE + sizeof(__le16)) { + ksmbd_debug(SMB, "Message is too small for StructureSize2\n"); + return 1; + } + if (smb2_req_struct_sizes[command] != pdu->StructureSize2) { if (!(command == SMB2_OPLOCK_BREAK_HE && (le16_to_cpu(pdu->StructureSize2) == OP_BREAK_STRUCT_SIZE_20 || diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c index a69bc75c85ad..64a1cfc4e809 100644 --- a/fs/smb/server/smb2pdu.c +++ b/fs/smb/server/smb2pdu.c @@ -1492,11 +1492,6 @@ static int ntlm_authenticate(struct ksmbd_work *work, return -EPERM; } - /* Check for previous session */ - prev_id = le64_to_cpu(req->PreviousSessionId); - if (prev_id && prev_id != sess->id) - destroy_previous_session(conn, user, prev_id); - if (sess->state == SMB2_SESSION_VALID) { /* * Reuse session if anonymous try to connect @@ -1534,6 +1529,10 @@ static int ntlm_authenticate(struct ksmbd_work *work, } } + prev_id = le64_to_cpu(req->PreviousSessionId); + if (prev_id && prev_id != sess->id) + destroy_previous_session(conn, sess->user, prev_id); + /* * If session state is SMB2_SESSION_VALID, We can assume * that it is reauthentication. And the user/password diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c index b09bc8d9389a..d3d0a22620f9 100644 --- a/fs/smb/server/smbacl.c +++ b/fs/smb/server/smbacl.c @@ -595,7 +595,8 @@ static void parse_dacl(struct mnt_idmap *idmap, static void set_posix_acl_entries_dacl(struct mnt_idmap *idmap, struct smb_ace *pndace, struct smb_fattr *fattr, u16 *num_aces, - u16 *size, u32 nt_aces_num) + u16 *size, u16 existing_nt_aces, + bool had_nt_aces) { struct posix_acl_entry *pace; struct smb_sid *sid; @@ -627,14 +628,14 @@ static void set_posix_acl_entries_dacl(struct mnt_idmap *idmap, gid = posix_acl_gid_translate(idmap, pace); id_to_sid(gid, SIDUNIX_GROUP, sid); - } else if (pace->e_tag == ACL_OTHER && !nt_aces_num) { + } else if (pace->e_tag == ACL_OTHER && !had_nt_aces) { smb_copy_sid(sid, &sid_everyone); } else { kfree(sid); continue; } ntace = pndace; - for (j = 0; j < nt_aces_num; j++) { + for (j = 0; j < existing_nt_aces; j++) { if (ntace->sid.sub_auth[ntace->sid.num_subauth - 1] == sid->sub_auth[sid->num_subauth - 1]) goto pass_same_sid; @@ -649,6 +650,7 @@ static void set_posix_acl_entries_dacl(struct mnt_idmap *idmap, ace_sz = fill_ace_for_sid(ntace, sid, ACCESS_ALLOWED, flags, pace->e_perm, 0777); if (check_add_overflow(*size, ace_sz, size)) { + *size -= ace_sz; kfree(sid); break; } @@ -663,6 +665,7 @@ static void set_posix_acl_entries_dacl(struct mnt_idmap *idmap, ace_sz = fill_ace_for_sid(ntace, sid, ACCESS_ALLOWED, 0x03, pace->e_perm, 0777); if (check_add_overflow(*size, ace_sz, size)) { + *size -= ace_sz; kfree(sid); break; } @@ -676,7 +679,7 @@ static void set_posix_acl_entries_dacl(struct mnt_idmap *idmap, kfree(sid); } - if (nt_aces_num) + if (had_nt_aces) return; posix_default_acl: @@ -708,6 +711,7 @@ static void set_posix_acl_entries_dacl(struct mnt_idmap *idmap, ace_sz = fill_ace_for_sid(ntace, sid, ACCESS_ALLOWED, 0x0b, pace->e_perm, 0777); if (check_add_overflow(*size, ace_sz, size)) { + *size -= ace_sz; kfree(sid); break; } @@ -729,6 +733,7 @@ static void set_ntacl_dacl(struct mnt_idmap *idmap, { struct smb_ace *ntace, *pndace; u16 nt_num_aces = le16_to_cpu(nt_dacl->num_aces), num_aces = 0; + u16 copied_nt_aces; unsigned short size = 0; int i; @@ -738,24 +743,41 @@ static void set_ntacl_dacl(struct mnt_idmap *idmap, for (i = 0; i < nt_num_aces; i++) { unsigned short nt_ace_size; - if (offsetof(struct smb_ace, access_req) > aces_size) + if (aces_size < offsetof(struct smb_ace, sid) + + CIFS_SID_BASE_SIZE) break; nt_ace_size = le16_to_cpu(ntace->size); - if (nt_ace_size > aces_size) + if (nt_ace_size > aces_size || + nt_ace_size < offsetof(struct smb_ace, sid) + + CIFS_SID_BASE_SIZE) break; + if (ntace->sid.num_subauth == 0 || + ntace->sid.num_subauth > SID_MAX_SUB_AUTHORITIES || + nt_ace_size < offsetof(struct smb_ace, sid) + + CIFS_SID_BASE_SIZE + + sizeof(__le32) * + ntace->sid.num_subauth) + goto next_ace; + memcpy((char *)pndace + size, ntace, nt_ace_size); - if (check_add_overflow(size, nt_ace_size, &size)) + if (check_add_overflow(size, nt_ace_size, &size)) { + size -= nt_ace_size; break; + } + num_aces++; + +next_ace: aces_size -= nt_ace_size; ntace = (struct smb_ace *)((char *)ntace + nt_ace_size); - num_aces++; } } + copied_nt_aces = num_aces; set_posix_acl_entries_dacl(idmap, pndace, fattr, - &num_aces, &size, nt_num_aces); + &num_aces, &size, copied_nt_aces, + nt_num_aces != 0); pndacl->num_aces = cpu_to_le16(num_aces); pndacl->size = cpu_to_le16(le16_to_cpu(pndacl->size) + size); } @@ -773,7 +795,7 @@ static void set_mode_dacl(struct mnt_idmap *idmap, if (fattr->cf_acls) { set_posix_acl_entries_dacl(idmap, pndace, fattr, - &num_aces, &size, num_aces); + &num_aces, &size, num_aces, false); goto out; } diff --git a/fs/super.c b/fs/super.c index ce6e5b4ff3c5..8af2ee9395bc 100644 --- a/fs/super.c +++ b/fs/super.c @@ -1139,18 +1139,35 @@ void emergency_remount(void) } } +static inline bool get_active_super(struct super_block *sb) +{ + bool active = false; + + if (super_lock_excl(sb)) { + active = atomic_inc_not_zero(&sb->s_active); + super_unlock_excl(sb); + } + return active; +} + static void do_thaw_all_callback(struct super_block *sb, void *unused) { + if (!get_active_super(sb)) + return; + + /* fs_bdev_thaw() acquires s_umount so it must not be held here */ if (IS_ENABLED(CONFIG_BLOCK)) while (sb->s_bdev && !bdev_thaw(sb->s_bdev)) pr_warn("Emergency Thaw on %pg\n", sb->s_bdev); - thaw_super_locked(sb, FREEZE_HOLDER_USERSPACE, NULL); - return; + + if (super_lock_excl(sb)) + thaw_super_locked(sb, FREEZE_HOLDER_USERSPACE, NULL); + deactivate_super(sb); } static void do_thaw_all(struct work_struct *work) { - __iterate_supers(do_thaw_all_callback, NULL, SUPER_ITER_EXCL); + __iterate_supers(do_thaw_all_callback, NULL, SUPER_ITER_UNLOCKED); kfree(work); printk(KERN_WARNING "Emergency Thaw complete\n"); } @@ -1171,17 +1188,6 @@ void emergency_thaw_all(void) } } -static inline bool get_active_super(struct super_block *sb) -{ - bool active = false; - - if (super_lock_excl(sb)) { - active = atomic_inc_not_zero(&sb->s_active); - super_unlock_excl(sb); - } - return active; -} - static const char *filesystems_freeze_ptr = "filesystems_freeze"; static void filesystems_freeze_callback(struct super_block *sb, void *freeze_all_ptr) diff --git a/fs/xfs/scrub/cow_repair.c b/fs/xfs/scrub/cow_repair.c index fc70fd5b975a..87627ab8943c 100644 --- a/fs/xfs/scrub/cow_repair.c +++ b/fs/xfs/scrub/cow_repair.c @@ -80,12 +80,6 @@ struct xrep_cow { unsigned int next_bno; }; -/* CoW staging extent. */ -struct xrep_cow_extent { - xfs_fsblock_t fsbno; - xfs_extlen_t len; -}; - /* * Mark the part of the file range that corresponds to the given physical * space. Caller must ensure that the physical range is within xc->irec. @@ -401,22 +395,21 @@ xrep_cow_find_bad_rt( STATIC int xrep_cow_alloc( struct xfs_scrub *sc, - xfs_extlen_t maxlen, - struct xrep_cow_extent *repl) + struct xfs_bmbt_irec *del) { struct xfs_alloc_arg args = { .tp = sc->tp, .mp = sc->mp, .oinfo = XFS_RMAP_OINFO_SKIP_UPDATE, .minlen = 1, - .maxlen = maxlen, + .maxlen = del->br_blockcount, .prod = 1, .resv = XFS_AG_RESV_NONE, .datatype = XFS_ALLOC_USERDATA, }; int error; - error = xfs_trans_reserve_more(sc->tp, maxlen, 0); + error = xfs_trans_reserve_more(sc->tp, del->br_blockcount, 0); if (error) return error; @@ -429,8 +422,8 @@ xrep_cow_alloc( xfs_refcount_alloc_cow_extent(sc->tp, false, args.fsbno, args.len); - repl->fsbno = args.fsbno; - repl->len = args.len; + del->br_startblock = args.fsbno; + del->br_blockcount = args.len; return 0; } @@ -441,10 +434,12 @@ xrep_cow_alloc( STATIC int xrep_cow_alloc_rt( struct xfs_scrub *sc, - xfs_extlen_t maxlen, - struct xrep_cow_extent *repl) + struct xfs_bmbt_irec *del) { - xfs_rtxlen_t maxrtx = xfs_rtb_to_rtx(sc->mp, maxlen); + xfs_fsblock_t fsbno; + xfs_rtxlen_t maxrtx = + min(U32_MAX, xfs_blen_to_rtbxlen(sc->mp, del->br_blockcount)); + xfs_extlen_t len; int error; error = xfs_trans_reserve_more(sc->tp, 0, maxrtx); @@ -452,11 +447,14 @@ xrep_cow_alloc_rt( return error; error = xfs_rtallocate_rtgs(sc->tp, NULLRTBLOCK, 1, maxrtx, 1, false, - false, &repl->fsbno, &repl->len); + false, &fsbno, &len); if (error) return error; - xfs_refcount_alloc_cow_extent(sc->tp, true, repl->fsbno, repl->len); + xfs_refcount_alloc_cow_extent(sc->tp, true, fsbno, len); + + del->br_startblock = fsbno; + del->br_blockcount = len; return 0; } @@ -470,19 +468,19 @@ static inline int xrep_cow_find_mapping( struct xrep_cow *xc, struct xfs_iext_cursor *icur, - xfs_fileoff_t startoff, - struct xfs_bmbt_irec *got) + xfs_fileoff_t badoff, + xfs_extlen_t badlen, + struct xfs_bmbt_irec *got, + struct xfs_bmbt_irec *rep) { struct xfs_inode *ip = xc->sc->ip; struct xfs_ifork *ifp = xfs_ifork_ptr(ip, XFS_COW_FORK); - if (!xfs_iext_lookup_extent(ip, ifp, startoff, icur, got)) + if (!xfs_iext_lookup_extent(ip, ifp, badoff, icur, got)) goto bad; + memcpy(rep, got, sizeof(*rep)); - if (got->br_startoff > startoff) - goto bad; - - if (got->br_blockcount == 0) + if (got->br_startoff > badoff) goto bad; if (isnullstartblock(got->br_startblock)) @@ -491,6 +489,24 @@ xrep_cow_find_mapping( if (xfs_bmap_is_written_extent(got)) goto bad; + if (got->br_startoff < badoff) { + const int64_t delta = badoff - got->br_startoff; + + rep->br_blockcount -= delta; + rep->br_startoff += delta; + rep->br_startblock += delta; + } + + if (got->br_startoff + got->br_blockcount > badoff + badlen) { + const int64_t delta = (got->br_startoff + got->br_blockcount) - + (badoff + badlen); + + rep->br_blockcount -= delta; + } + + if (got->br_blockcount == 0) + goto bad; + return 0; bad: ASSERT(0); @@ -501,46 +517,92 @@ xrep_cow_find_mapping( #define REPLACE_RIGHT_SIDE (1U << 1) /* - * Given a CoW fork mapping @got and a replacement mapping @repl, remap the - * beginning of @got with the space described by @rep. + * Given a CoW fork mapping @got and a replacement mapping @rep, map the space + * described by @rep into the cow fork, pushing aside @got as necessary. @icur + * must point to iext tree leaf containing @got. */ static inline void xrep_cow_replace_mapping( - struct xfs_inode *ip, - struct xfs_iext_cursor *icur, - const struct xfs_bmbt_irec *got, - const struct xrep_cow_extent *repl) + struct xfs_inode *ip, + struct xfs_iext_cursor *icur, + struct xfs_bmbt_irec *got, + struct xfs_bmbt_irec *rep) { - struct xfs_bmbt_irec new = *got; /* struct copy */ + struct xfs_ifork *ifp = xfs_ifork_ptr(ip, XFS_COW_FORK); + xfs_fileoff_t rep_endoff = + rep->br_startoff + rep->br_blockcount; + xfs_fileoff_t got_endoff = + got->br_startoff + got->br_blockcount; + uint32_t state = BMAP_COWFORK; - ASSERT(repl->len > 0); + ASSERT(rep->br_blockcount > 0); ASSERT(!isnullstartblock(got->br_startblock)); + ASSERT(got->br_startoff <= rep->br_startoff); + ASSERT(got_endoff >= rep_endoff); + + trace_xrep_cow_replace_mapping(ip, got, rep); - trace_xrep_cow_replace_mapping(ip, got, repl->fsbno, repl->len); + if (got->br_startoff == rep->br_startoff) + state |= BMAP_LEFT_FILLING; + if (got_endoff == rep_endoff) + state |= BMAP_RIGHT_FILLING; - if (got->br_blockcount == repl->len) { + switch (state & (BMAP_LEFT_FILLING | BMAP_RIGHT_FILLING)) { + case BMAP_LEFT_FILLING | BMAP_RIGHT_FILLING: /* - * The new extent is a complete replacement for the existing - * extent. Update the COW fork record. + * Replacement matches the whole mapping, update the record. */ - new.br_startblock = repl->fsbno; - xfs_iext_update_extent(ip, BMAP_COWFORK, icur, &new); - return; - } + xfs_iext_update_extent(ip, state, icur, rep); + break; + case BMAP_LEFT_FILLING: + /* + * Replace the first part of the mapping: Update the cursor + * position with the new mapping, then add a record with the + * tail of the old mapping. + */ + got->br_startoff = rep_endoff; + got->br_blockcount -= rep->br_blockcount; + got->br_startblock += rep->br_blockcount; + + xfs_iext_update_extent(ip, state, icur, rep); + xfs_iext_next(ifp, icur); + xfs_iext_insert(ip, icur, got, state); + break; + case BMAP_RIGHT_FILLING: + /* + * Replacing the last part of the mapping. Shorten the current + * mapping then add a record with the new mapping. + */ + got->br_blockcount -= rep->br_blockcount; - /* - * The new extent can replace the beginning of the COW fork record. - * Move the left side of @got upwards, then insert the new record. - */ - new.br_startoff += repl->len; - new.br_startblock += repl->len; - new.br_blockcount -= repl->len; - xfs_iext_update_extent(ip, BMAP_COWFORK, icur, &new); - - new.br_startoff = got->br_startoff; - new.br_startblock = repl->fsbno; - new.br_blockcount = repl->len; - xfs_iext_insert(ip, icur, &new, BMAP_COWFORK); + xfs_iext_update_extent(ip, state, icur, got); + xfs_iext_next(ifp, icur); + xfs_iext_insert(ip, icur, rep, state); + break; + case 0: + /* + * Replacing the middle of the extent. Shorten the current + * mapping, add a new record with the new mapping, and add a + * second new record with the tail of the old mapping. + */ + got->br_blockcount = rep->br_startoff - got->br_startoff; + + struct xfs_bmbt_irec new = { + .br_startoff = rep_endoff, + .br_blockcount = got_endoff - rep_endoff, + .br_state = got->br_state, + .br_startblock = got->br_startblock + + rep->br_blockcount + + got->br_blockcount, + }; + + xfs_iext_update_extent(ip, state, icur, got); + xfs_iext_next(ifp, icur); + xfs_iext_insert(ip, icur, rep, state); + xfs_iext_next(ifp, icur); + xfs_iext_insert(ip, icur, &new, state); + break; + } } /* @@ -554,33 +616,30 @@ xrep_cow_replace_range( xfs_extlen_t *blockcount) { struct xfs_iext_cursor icur; - struct xrep_cow_extent repl; - struct xfs_bmbt_irec got; + struct xfs_bmbt_irec got, rep; struct xfs_scrub *sc = xc->sc; - xfs_fileoff_t nextoff; - xfs_extlen_t alloc_len; + xfs_fsblock_t old_fsbno; int error; /* - * Put the existing CoW fork mapping in @got. If @got ends before - * @rep, truncate @rep so we only replace one extent mapping at a time. + * Put the existing CoW fork mapping in @got, and put in @rep the + * contents of @got trimmed to @startoff/@blockcount. We only want + * to replace the bad region, and only one mapping at a time. */ - error = xrep_cow_find_mapping(xc, &icur, startoff, &got); + error = xrep_cow_find_mapping(xc, &icur, startoff, *blockcount, &got, + &rep); if (error) return error; - nextoff = min(startoff + *blockcount, - got.br_startoff + got.br_blockcount); + old_fsbno = rep.br_startblock; /* * Allocate a replacement extent. If we don't fill all the blocks, * shorten the quantity that will be deleted in this step. */ - alloc_len = min_t(xfs_fileoff_t, XFS_MAX_BMBT_EXTLEN, - nextoff - startoff); if (XFS_IS_REALTIME_INODE(sc->ip)) - error = xrep_cow_alloc_rt(sc, alloc_len, &repl); + error = xrep_cow_alloc_rt(sc, &rep); else - error = xrep_cow_alloc(sc, alloc_len, &repl); + error = xrep_cow_alloc(sc, &rep); if (error) return error; @@ -588,7 +647,7 @@ xrep_cow_replace_range( * Replace the old mapping with the new one, and commit the metadata * changes made so far. */ - xrep_cow_replace_mapping(sc->ip, &icur, &got, &repl); + xrep_cow_replace_mapping(sc->ip, &icur, &got, &rep); xfs_inode_set_cowblocks_tag(sc->ip); error = xfs_defer_finish(&sc->tp); @@ -597,15 +656,15 @@ xrep_cow_replace_range( /* Note the old CoW staging extents; we'll reap them all later. */ if (XFS_IS_REALTIME_INODE(sc->ip)) - error = xrtb_bitmap_set(&xc->old_cowfork_rtblocks, - got.br_startblock, repl.len); + error = xrtb_bitmap_set(&xc->old_cowfork_rtblocks, old_fsbno, + rep.br_blockcount); else - error = xfsb_bitmap_set(&xc->old_cowfork_fsblocks, - got.br_startblock, repl.len); + error = xfsb_bitmap_set(&xc->old_cowfork_fsblocks, old_fsbno, + rep.br_blockcount); if (error) return error; - *blockcount = repl.len; + *blockcount = rep.br_blockcount; return 0; } diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h index 286c5f5e0544..fab3ce323e4c 100644 --- a/fs/xfs/scrub/trace.h +++ b/fs/xfs/scrub/trace.h @@ -2672,9 +2672,9 @@ TRACE_EVENT(xrep_cow_mark_file_range, ); TRACE_EVENT(xrep_cow_replace_mapping, - TP_PROTO(struct xfs_inode *ip, const struct xfs_bmbt_irec *irec, - xfs_fsblock_t new_startblock, xfs_extlen_t new_blockcount), - TP_ARGS(ip, irec, new_startblock, new_blockcount), + TP_PROTO(struct xfs_inode *ip, const struct xfs_bmbt_irec *got, + const struct xfs_bmbt_irec *rep), + TP_ARGS(ip, got, rep), TP_STRUCT__entry( __field(dev_t, dev) __field(xfs_ino_t, ino) @@ -2682,28 +2682,34 @@ TRACE_EVENT(xrep_cow_replace_mapping, __field(xfs_fileoff_t, startoff) __field(xfs_filblks_t, blockcount) __field(xfs_exntst_t, state) + __field(xfs_fileoff_t, new_startoff) __field(xfs_fsblock_t, new_startblock) __field(xfs_extlen_t, new_blockcount) + __field(xfs_exntst_t, new_state) ), TP_fast_assign( __entry->dev = ip->i_mount->m_super->s_dev; __entry->ino = ip->i_ino; - __entry->startoff = irec->br_startoff; - __entry->startblock = irec->br_startblock; - __entry->blockcount = irec->br_blockcount; - __entry->state = irec->br_state; - __entry->new_startblock = new_startblock; - __entry->new_blockcount = new_blockcount; + __entry->startoff = got->br_startoff; + __entry->startblock = got->br_startblock; + __entry->blockcount = got->br_blockcount; + __entry->state = got->br_state; + __entry->new_startoff = rep->br_startoff; + __entry->new_startblock = rep->br_startblock; + __entry->new_blockcount = rep->br_blockcount; + __entry->new_state = rep->br_state; ), - TP_printk("dev %d:%d ino 0x%llx startoff 0x%llx startblock 0x%llx fsbcount 0x%llx state 0x%x new_startblock 0x%llx new_fsbcount 0x%x", + TP_printk("dev %d:%d ino 0x%llx startoff 0x%llx startblock 0x%llx fsbcount 0x%llx state 0x%x new_startoff 0x%llx new_startblock 0x%llx new_fsbcount 0x%x new_state 0x%x", MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino, __entry->startoff, __entry->startblock, __entry->blockcount, __entry->state, + __entry->new_startoff, __entry->new_startblock, - __entry->new_blockcount) + __entry->new_blockcount, + __entry->new_state) ); TRACE_EVENT(xrep_cow_free_staging, diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index c2e05a281252..3054369bebff 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -6,166 +6,13 @@ #ifndef __DRM_BUDDY_H__ #define __DRM_BUDDY_H__ -#include -#include -#include -#include -#include +#include -#include +struct drm_printer; -#define DRM_BUDDY_RANGE_ALLOCATION BIT(0) -#define DRM_BUDDY_TOPDOWN_ALLOCATION BIT(1) -#define DRM_BUDDY_CONTIGUOUS_ALLOCATION BIT(2) -#define DRM_BUDDY_CLEAR_ALLOCATION BIT(3) -#define DRM_BUDDY_CLEARED BIT(4) -#define DRM_BUDDY_TRIM_DISABLE BIT(5) - -struct drm_buddy_block { -#define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12) -#define DRM_BUDDY_HEADER_STATE GENMASK_ULL(11, 10) -#define DRM_BUDDY_ALLOCATED (1 << 10) -#define DRM_BUDDY_FREE (2 << 10) -#define DRM_BUDDY_SPLIT (3 << 10) -#define DRM_BUDDY_HEADER_CLEAR GENMASK_ULL(9, 9) -/* Free to be used, if needed in the future */ -#define DRM_BUDDY_HEADER_UNUSED GENMASK_ULL(8, 6) -#define DRM_BUDDY_HEADER_ORDER GENMASK_ULL(5, 0) - u64 header; - - struct drm_buddy_block *left; - struct drm_buddy_block *right; - struct drm_buddy_block *parent; - - void *private; /* owned by creator */ - - /* - * While the block is allocated by the user through drm_buddy_alloc*, - * the user has ownership of the link, for example to maintain within - * a list, if so desired. As soon as the block is freed with - * drm_buddy_free* ownership is given back to the mm. - */ - union { - struct rb_node rb; - struct list_head link; - }; - - struct list_head tmp_link; -}; - -/* Order-zero must be at least SZ_4K */ -#define DRM_BUDDY_MAX_ORDER (63 - 12) - -/* - * Binary Buddy System. - * - * Locking should be handled by the user, a simple mutex around - * drm_buddy_alloc* and drm_buddy_free* should suffice. - */ -struct drm_buddy { - /* Maintain a free list for each order. */ - struct rb_root **free_trees; - - /* - * Maintain explicit binary tree(s) to track the allocation of the - * address space. This gives us a simple way of finding a buddy block - * and performing the potentially recursive merge step when freeing a - * block. Nodes are either allocated or free, in which case they will - * also exist on the respective free list. - */ - struct drm_buddy_block **roots; - - /* - * Anything from here is public, and remains static for the lifetime of - * the mm. Everything above is considered do-not-touch. - */ - unsigned int n_roots; - unsigned int max_order; - - /* Must be at least SZ_4K */ - u64 chunk_size; - u64 size; - u64 avail; - u64 clear_avail; -}; - -static inline u64 -drm_buddy_block_offset(const struct drm_buddy_block *block) -{ - return block->header & DRM_BUDDY_HEADER_OFFSET; -} - -static inline unsigned int -drm_buddy_block_order(struct drm_buddy_block *block) -{ - return block->header & DRM_BUDDY_HEADER_ORDER; -} - -static inline unsigned int -drm_buddy_block_state(struct drm_buddy_block *block) -{ - return block->header & DRM_BUDDY_HEADER_STATE; -} - -static inline bool -drm_buddy_block_is_allocated(struct drm_buddy_block *block) -{ - return drm_buddy_block_state(block) == DRM_BUDDY_ALLOCATED; -} - -static inline bool -drm_buddy_block_is_clear(struct drm_buddy_block *block) -{ - return block->header & DRM_BUDDY_HEADER_CLEAR; -} - -static inline bool -drm_buddy_block_is_free(struct drm_buddy_block *block) -{ - return drm_buddy_block_state(block) == DRM_BUDDY_FREE; -} - -static inline bool -drm_buddy_block_is_split(struct drm_buddy_block *block) -{ - return drm_buddy_block_state(block) == DRM_BUDDY_SPLIT; -} - -static inline u64 -drm_buddy_block_size(struct drm_buddy *mm, - struct drm_buddy_block *block) -{ - return mm->chunk_size << drm_buddy_block_order(block); -} - -int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size); - -void drm_buddy_fini(struct drm_buddy *mm); - -struct drm_buddy_block * -drm_get_buddy(struct drm_buddy_block *block); - -int drm_buddy_alloc_blocks(struct drm_buddy *mm, - u64 start, u64 end, u64 size, - u64 min_page_size, - struct list_head *blocks, - unsigned long flags); - -int drm_buddy_block_trim(struct drm_buddy *mm, - u64 *start, - u64 new_size, - struct list_head *blocks); - -void drm_buddy_reset_clear(struct drm_buddy *mm, bool is_clear); - -void drm_buddy_free_block(struct drm_buddy *mm, struct drm_buddy_block *block); - -void drm_buddy_free_list(struct drm_buddy *mm, - struct list_head *objects, - unsigned int flags); - -void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p); -void drm_buddy_block_print(struct drm_buddy *mm, - struct drm_buddy_block *block, +/* DRM-specific GPU Buddy Allocator print helpers */ +void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p); +void drm_buddy_block_print(struct gpu_buddy *mm, + struct gpu_buddy_block *block, struct drm_printer *p); #endif diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h index 8f34f4b8183d..d5e6f4b832a7 100644 --- a/include/drm/drm_connector.h +++ b/include/drm/drm_connector.h @@ -2502,13 +2502,13 @@ struct drm_tile_group { struct kref refcount; struct drm_device *dev; int id; - u8 group_data[8]; + u8 group_data[9]; }; struct drm_tile_group *drm_mode_create_tile_group(struct drm_device *dev, - const char topology[8]); + const char topology_id[9]); struct drm_tile_group *drm_mode_get_tile_group(struct drm_device *dev, - const char topology[8]); + const char topology_id[9]); void drm_mode_put_tile_group(struct drm_device *dev, struct drm_tile_group *tg); diff --git a/include/drm/drm_utils.h b/include/drm/drm_utils.h index 6a46f755daba..7e077484c5bb 100644 --- a/include/drm/drm_utils.h +++ b/include/drm/drm_utils.h @@ -19,6 +19,7 @@ int drm_get_panel_orientation_quirk(int width, int height); struct drm_panel_backlight_quirk { u16 min_brightness; u32 brightness_mask; + bool force_pwm; }; const struct drm_panel_backlight_quirk * diff --git a/include/linux/arm_ffa.h b/include/linux/arm_ffa.h index 81e603839c4a..62d67dae8b70 100644 --- a/include/linux/arm_ffa.h +++ b/include/linux/arm_ffa.h @@ -445,7 +445,7 @@ ffa_mem_desc_offset(struct ffa_mem_region *buf, int count, u32 ffa_version) if (!FFA_MEM_REGION_HAS_EP_MEM_OFFSET(ffa_version)) offset += offsetof(struct ffa_mem_region, ep_mem_offset); else - offset += sizeof(struct ffa_mem_region); + offset += buf->ep_mem_offset; return offset; } diff --git a/include/linux/audit.h b/include/linux/audit.h index b8d8029c6c48..09d06684174b 100644 --- a/include/linux/audit.h +++ b/include/linux/audit.h @@ -16,7 +16,7 @@ #include #include -#define AUDIT_INO_UNSET ((unsigned long)-1) +#define AUDIT_INO_UNSET ((u64)-1) #define AUDIT_DEV_UNSET ((dev_t)-1) struct audit_sig_info { @@ -125,8 +125,8 @@ enum audit_nfcfgop { AUDIT_NFT_OP_INVALID, }; -extern int __init audit_register_class(int class, unsigned *list); -extern int audit_classify_syscall(int abi, unsigned syscall); +extern int __init audit_register_class(int class, unsigned int *list); +extern int audit_classify_syscall(int abi, unsigned int syscall); extern int audit_classify_arch(int arch); /* audit_names->type values */ diff --git a/include/linux/audit_arch.h b/include/linux/audit_arch.h index 2b8153791e6a..a35069a6c15d 100644 --- a/include/linux/audit_arch.h +++ b/include/linux/audit_arch.h @@ -21,13 +21,13 @@ enum auditsc_class_t { AUDITSC_NVALS /* count */ }; -extern int audit_classify_compat_syscall(int abi, unsigned syscall); +extern int audit_classify_compat_syscall(int abi, unsigned int syscall); /* only for compat system calls */ -extern unsigned compat_write_class[]; -extern unsigned compat_read_class[]; -extern unsigned compat_dir_class[]; -extern unsigned compat_chattr_class[]; -extern unsigned compat_signal_class[]; +extern unsigned int compat_write_class[]; +extern unsigned int compat_read_class[]; +extern unsigned int compat_dir_class[]; +extern unsigned int compat_chattr_class[]; +extern unsigned int compat_signal_class[]; #endif diff --git a/include/linux/bootconfig.h b/include/linux/bootconfig.h index 25df9260d206..1f6e9c54a2a5 100644 --- a/include/linux/bootconfig.h +++ b/include/linux/bootconfig.h @@ -265,6 +265,9 @@ static inline struct xbc_node * __init xbc_node_get_subkey(struct xbc_node *node int __init xbc_node_compose_key_after(struct xbc_node *root, struct xbc_node *node, char *buf, size_t size); +/* Render key/value pairs under @root as a flat cmdline string */ +int __init xbc_snprint_cmdline(char *buf, size_t size, struct xbc_node *root); + /** * xbc_node_compose_key() - Compose full key string of the XBC node * @node: An XBC node. diff --git a/include/linux/cred.h b/include/linux/cred.h index 89ae50ad2ace..be2cd07b174c 100644 --- a/include/linux/cred.h +++ b/include/linux/cred.h @@ -20,6 +20,8 @@ struct cred; struct inode; +extern struct task_struct init_task; + /* * COW Supplementary groups list */ @@ -156,6 +158,11 @@ extern struct cred *prepare_exec_creds(void); extern int commit_creds(struct cred *); extern void abort_creds(struct cred *); extern struct cred *prepare_kernel_cred(struct task_struct *); +static inline const struct cred *kernel_cred(void) +{ + /* shut up sparse */ + return rcu_dereference_raw(init_task.cred); +} extern int set_security_override(struct cred *, u32); extern int set_security_override_from_ctx(struct cred *, const char *); extern int set_create_files_as(struct cred *, struct inode *); @@ -180,6 +187,14 @@ static inline const struct cred *revert_creds(const struct cred *revert_cred) return rcu_replace_pointer(current->cred, revert_cred, 1); } +DEFINE_CLASS(override_creds, + const struct cred *, + revert_creds(_T), + override_creds(override_cred), const struct cred *override_cred) + +#define scoped_with_kernel_creds() \ + scoped_class(override_creds, __UNIQUE_ID(cred), kernel_cred()) + /** * get_cred_many - Get references on a set of credentials * @cred: The credentials to reference diff --git a/include/linux/firmware/xlnx-zynqmp.h b/include/linux/firmware/xlnx-zynqmp.h index ae48d619c4e0..2f1c71f0cea7 100644 --- a/include/linux/firmware/xlnx-zynqmp.h +++ b/include/linux/firmware/xlnx-zynqmp.h @@ -164,6 +164,7 @@ enum pm_api_cb_id { enum pm_api_id { PM_API_FEATURES = 0, PM_GET_API_VERSION = 1, + PM_GET_NODE_STATUS = 3, PM_REGISTER_NOTIFIER = 5, PM_FORCE_POWERDOWN = 8, PM_REQUEST_WAKEUP = 10, @@ -544,6 +545,18 @@ enum pm_gem_config_type { GEM_CONFIG_FIXED = 2, }; +/** + * enum pm_node_status - Device node status provided by xilpm fw + * @PM_NODE_UNUSED: Device is not used + * @PM_NODE_RUNNING: Device is power-on and out of reset + * @PM_NODE_HALT: Device is power-on but in the reset state + */ +enum pm_node_status { + PM_NODE_UNUSED = 0, + PM_NODE_RUNNING = 1, + PM_NODE_HALT = 12, +}; + /** * struct zynqmp_pm_query_data - PM query data * @qid: query ID diff --git a/include/linux/fscrypt.h b/include/linux/fscrypt.h index 516aba5b858b..c009d4afe918 100644 --- a/include/linux/fscrypt.h +++ b/include/linux/fscrypt.h @@ -57,6 +57,9 @@ struct fscrypt_name { /* Maximum value for the third parameter of fscrypt_operations.set_context(). */ #define FSCRYPT_SET_CONTEXT_MAX_SIZE 40 +/* Maximum supported number of block devices per filesystem */ +#define FSCRYPT_MAX_DEVICES 8 + #ifdef CONFIG_FS_ENCRYPTION /* Crypto operations for filesystems */ @@ -181,21 +184,20 @@ struct fscrypt_operations { bool (*has_stable_inodes)(struct super_block *sb); /* - * Return an array of pointers to the block devices to which the - * filesystem may write encrypted file contents, NULL if the filesystem - * only has a single such block device, or an ERR_PTR() on error. + * Retrieve the list of block devices to which the filesystem may write + * encrypted file contents. * - * On successful non-NULL return, *num_devs is set to the number of - * devices in the returned array. The caller must free the returned - * array using kfree(). + * This writes the block_device pointers to @devs and returns the count + * (between 1 and FSCRYPT_MAX_DEVICES inclusively). * * If the filesystem can use multiple block devices (other than block * devices that aren't used for encrypted file contents, such as * external journal devices), and wants to support inline encryption, * then it must implement this function. Otherwise it's not needed. */ - struct block_device **(*get_devices)(struct super_block *sb, - unsigned int *num_devs); + unsigned int (*get_devices)( + struct super_block *sb, + struct block_device *devs[FSCRYPT_MAX_DEVICES]); }; int fscrypt_d_revalidate(struct inode *dir, const struct qstr *name, diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h new file mode 100644 index 000000000000..07ac65db6d2e --- /dev/null +++ b/include/linux/gpu_buddy.h @@ -0,0 +1,177 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2021 Intel Corporation + */ + +#ifndef __GPU_BUDDY_H__ +#define __GPU_BUDDY_H__ + +#include +#include +#include +#include +#include + +#define GPU_BUDDY_RANGE_ALLOCATION BIT(0) +#define GPU_BUDDY_TOPDOWN_ALLOCATION BIT(1) +#define GPU_BUDDY_CONTIGUOUS_ALLOCATION BIT(2) +#define GPU_BUDDY_CLEAR_ALLOCATION BIT(3) +#define GPU_BUDDY_CLEARED BIT(4) +#define GPU_BUDDY_TRIM_DISABLE BIT(5) + +enum gpu_buddy_free_tree { + GPU_BUDDY_CLEAR_TREE = 0, + GPU_BUDDY_DIRTY_TREE, + GPU_BUDDY_MAX_FREE_TREES, +}; + +#define for_each_free_tree(tree) \ + for ((tree) = 0; (tree) < GPU_BUDDY_MAX_FREE_TREES; (tree)++) + +struct gpu_buddy_block { +#define GPU_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12) +#define GPU_BUDDY_HEADER_STATE GENMASK_ULL(11, 10) +#define GPU_BUDDY_ALLOCATED (1 << 10) +#define GPU_BUDDY_FREE (2 << 10) +#define GPU_BUDDY_SPLIT (3 << 10) +#define GPU_BUDDY_HEADER_CLEAR GENMASK_ULL(9, 9) +/* Free to be used, if needed in the future */ +#define GPU_BUDDY_HEADER_UNUSED GENMASK_ULL(8, 6) +#define GPU_BUDDY_HEADER_ORDER GENMASK_ULL(5, 0) + u64 header; + + struct gpu_buddy_block *left; + struct gpu_buddy_block *right; + struct gpu_buddy_block *parent; + + void *private; /* owned by creator */ + + /* + * While the block is allocated by the user through gpu_buddy_alloc*, + * the user has ownership of the link, for example to maintain within + * a list, if so desired. As soon as the block is freed with + * gpu_buddy_free* ownership is given back to the mm. + */ + union { + struct rb_node rb; + struct list_head link; + }; + + struct list_head tmp_link; +}; + +/* Order-zero must be at least SZ_4K */ +#define GPU_BUDDY_MAX_ORDER (63 - 12) + +/* + * Binary Buddy System. + * + * Locking should be handled by the user, a simple mutex around + * gpu_buddy_alloc* and gpu_buddy_free* should suffice. + */ +struct gpu_buddy { + /* Maintain a free list for each order. */ + struct rb_root **free_trees; + + /* + * Maintain explicit binary tree(s) to track the allocation of the + * address space. This gives us a simple way of finding a buddy block + * and performing the potentially recursive merge step when freeing a + * block. Nodes are either allocated or free, in which case they will + * also exist on the respective free list. + */ + struct gpu_buddy_block **roots; + + /* + * Anything from here is public, and remains static for the lifetime of + * the mm. Everything above is considered do-not-touch. + */ + unsigned int n_roots; + unsigned int max_order; + + /* Must be at least SZ_4K */ + u64 chunk_size; + u64 size; + u64 avail; + u64 clear_avail; +}; + +static inline u64 +gpu_buddy_block_offset(const struct gpu_buddy_block *block) +{ + return block->header & GPU_BUDDY_HEADER_OFFSET; +} + +static inline unsigned int +gpu_buddy_block_order(struct gpu_buddy_block *block) +{ + return block->header & GPU_BUDDY_HEADER_ORDER; +} + +static inline unsigned int +gpu_buddy_block_state(struct gpu_buddy_block *block) +{ + return block->header & GPU_BUDDY_HEADER_STATE; +} + +static inline bool +gpu_buddy_block_is_allocated(struct gpu_buddy_block *block) +{ + return gpu_buddy_block_state(block) == GPU_BUDDY_ALLOCATED; +} + +static inline bool +gpu_buddy_block_is_clear(struct gpu_buddy_block *block) +{ + return block->header & GPU_BUDDY_HEADER_CLEAR; +} + +static inline bool +gpu_buddy_block_is_free(struct gpu_buddy_block *block) +{ + return gpu_buddy_block_state(block) == GPU_BUDDY_FREE; +} + +static inline bool +gpu_buddy_block_is_split(struct gpu_buddy_block *block) +{ + return gpu_buddy_block_state(block) == GPU_BUDDY_SPLIT; +} + +static inline u64 +gpu_buddy_block_size(struct gpu_buddy *mm, + struct gpu_buddy_block *block) +{ + return mm->chunk_size << gpu_buddy_block_order(block); +} + +int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size); + +void gpu_buddy_fini(struct gpu_buddy *mm); + +struct gpu_buddy_block * +gpu_get_buddy(struct gpu_buddy_block *block); + +int gpu_buddy_alloc_blocks(struct gpu_buddy *mm, + u64 start, u64 end, u64 size, + u64 min_page_size, + struct list_head *blocks, + unsigned long flags); + +int gpu_buddy_block_trim(struct gpu_buddy *mm, + u64 *start, + u64 new_size, + struct list_head *blocks); + +void gpu_buddy_reset_clear(struct gpu_buddy *mm, bool is_clear); + +void gpu_buddy_free_block(struct gpu_buddy *mm, struct gpu_buddy_block *block); + +void gpu_buddy_free_list(struct gpu_buddy *mm, + struct list_head *objects, + unsigned int flags); + +void gpu_buddy_print(struct gpu_buddy *mm); +void gpu_buddy_block_print(struct gpu_buddy *mm, + struct gpu_buddy_block *block); +#endif diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 1b0b36aa2a76..61257b866fb2 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -12141,18 +12141,7 @@ struct mlx5_ifc_mcia_reg_bits { u8 reserved_at_60[0x20]; - u8 dword_0[0x20]; - u8 dword_1[0x20]; - u8 dword_2[0x20]; - u8 dword_3[0x20]; - u8 dword_4[0x20]; - u8 dword_5[0x20]; - u8 dword_6[0x20]; - u8 dword_7[0x20]; - u8 dword_8[0x20]; - u8 dword_9[0x20]; - u8 dword_10[0x20]; - u8 dword_11[0x20]; + u8 dwords[0x400]; }; struct mlx5_ifc_dcbx_param_bits { diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index a7d92718b653..f9bed457fc5f 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -169,7 +169,7 @@ struct mbm_cntr_cfg { }; /** - * struct rdt_mon_domain - group of CPUs sharing a resctrl monitor resource + * struct rdt_l3_mon_domain - group of CPUs sharing RDT_RESOURCE_L3 monitoring * @hdr: common header for different domain types * @ci_id: cache info id for this domain * @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold @@ -183,7 +183,7 @@ struct mbm_cntr_cfg { * @cntr_cfg: array of assignable counters' configuration (indexed * by counter ID) */ -struct rdt_mon_domain { +struct rdt_l3_mon_domain { struct rdt_domain_hdr hdr; unsigned int ci_id; unsigned long *rmid_busy_llc; @@ -355,10 +355,10 @@ struct resctrl_cpu_defaults { }; struct resctrl_mon_config_info { - struct rdt_resource *r; - struct rdt_mon_domain *d; - u32 evtid; - u32 mon_config; + struct rdt_resource *r; + struct rdt_l3_mon_domain *d; + u32 evtid; + u32 mon_config; }; /** @@ -495,9 +495,9 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d, u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d, u32 closid, enum resctrl_conf_type type); int resctrl_online_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain *d); -int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d); +int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_l3_mon_domain *d); void resctrl_offline_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain *d); -void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d); +void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_l3_mon_domain *d); void resctrl_online_cpu(unsigned int cpu); void resctrl_offline_cpu(unsigned int cpu); @@ -526,7 +526,7 @@ void resctrl_offline_cpu(unsigned int cpu); * Return: * 0 on success, or -EIO, -EINVAL etc on error. */ -int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_mon_domain *d, +int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid, u64 *val, void *arch_mon_ctx); @@ -573,7 +573,7 @@ struct rdt_domain_hdr *resctrl_find_domain(struct list_head *h, int id, * * This can be called from any CPU. */ -void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d, +void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid); @@ -586,7 +586,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_mon_domain *d, * * This can be called from any CPU. */ -void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_mon_domain *d); +void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_l3_mon_domain *d); /** * resctrl_arch_reset_all_ctrls() - Reset the control for each CLOSID to its @@ -612,7 +612,7 @@ void resctrl_arch_reset_all_ctrls(struct rdt_resource *r); * * This can be called from any CPU. */ -void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, +void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d, enum resctrl_event_id evtid, u32 rmid, u32 closid, u32 cntr_id, bool assign); @@ -635,7 +635,7 @@ void resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, * Return: * 0 on success, or -EIO, -EINVAL etc on error. */ -int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_mon_domain *d, +int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 closid, u32 rmid, int cntr_id, enum resctrl_event_id eventid, u64 *val); @@ -650,7 +650,7 @@ int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_mon_domain *d, * * This can be called from any CPU. */ -void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_mon_domain *d, +void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 closid, u32 rmid, int cntr_id, enum resctrl_event_id eventid); diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 6ab7924f33f5..6729245aa154 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -1224,7 +1224,7 @@ struct ss_tmp { spinlock_t *lock_irqsave; }; -static inline void __scoped_seqlock_cleanup(struct ss_tmp *sst) +static __always_inline void __scoped_seqlock_cleanup(struct ss_tmp *sst) { if (sst->lock) spin_unlock(sst->lock); @@ -1234,11 +1234,15 @@ static inline void __scoped_seqlock_cleanup(struct ss_tmp *sst) extern void __scoped_seqlock_invalid_target(void); -#if defined(CONFIG_CC_IS_GCC) && CONFIG_GCC_VERSION < 90000 +#if (defined(CONFIG_CC_IS_GCC) && CONFIG_GCC_VERSION < 90000) || \ + defined(CONFIG_KASAN) || defined(CONFIG_UBSAN_ALIGNMENT) /* * For some reason some GCC-8 architectures (nios2, alpha) have trouble * determining that the ss_done state is impossible in __scoped_seqlock_next() * below. + * + * Similarly KASAN and UBSAN_ALIGNMENT are known to confuse compilers enough + * to break this. But we don't care about code quality for such builds anyway. */ static inline void __scoped_seqlock_bug(void) { } #else @@ -1249,7 +1253,7 @@ static inline void __scoped_seqlock_bug(void) { } extern void __scoped_seqlock_bug(void); #endif -static inline void +static __always_inline void __scoped_seqlock_next(struct ss_tmp *sst, seqlock_t *lock, enum ss_state target) { switch (sst->state) { diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 963bbe251e52..de37069aba90 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -26,6 +26,9 @@ struct svc_sock { void (*sk_odata)(struct sock *); void (*sk_owspace)(struct sock *); + /* For sends (protected by xpt_mutex) */ + struct bio_vec *sk_bvec; + /* private TCP part */ /* On-the-wire fragment header: */ __be32 sk_marker; diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h index 152597750f55..656e0ad323af 100644 --- a/include/linux/sunrpc/xdr.h +++ b/include/linux/sunrpc/xdr.h @@ -138,8 +138,23 @@ void xdr_terminate_string(const struct xdr_buf *, const u32); size_t xdr_buf_pagecount(const struct xdr_buf *buf); int xdr_alloc_bvec(struct xdr_buf *buf, gfp_t gfp); void xdr_free_bvec(struct xdr_buf *buf); -unsigned int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size, - const struct xdr_buf *xdr); +int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size, + const struct xdr_buf *xdr); +int xdr_buf_to_sg(const struct xdr_buf *buf, unsigned int offset, + unsigned int len, struct scatterlist *sg, unsigned int nsg); +int xdr_buf_to_sg_alloc(const struct xdr_buf *buf, unsigned int offset, + unsigned int len, struct scatterlist *sg_head, + unsigned int sg_head_nents, + struct scatterlist **sg_overflow, gfp_t gfp); + +/* + * Inline scatterlist entries for xdr_buf_to_sg_alloc(). Sized to cover the + * head kvec, tail kvec, and a few page fragments without any heap allocation. + */ +enum { + XDR_BUF_TO_SG_NENTS = 8, +}; + static inline __be32 *xdr_encode_array(__be32 *p, const void *s, unsigned int len) { diff --git a/include/linux/virtio.h b/include/linux/virtio.h index 132a474e5914..d3f723b875a7 100644 --- a/include/linux/virtio.h +++ b/include/linux/virtio.h @@ -41,13 +41,13 @@ struct virtqueue { void *priv; }; -struct vduse_iova_domain; +struct vduse_vq_group; union virtio_map { /* Device that performs DMA */ struct device *dma_dev; - /* VDUSE specific mapping data */ - struct vduse_iova_domain *iova_domain; + /* VDUSE specific virtqueue group for doing map */ + struct vduse_vq_group *group; }; int virtqueue_add_outbuf(struct virtqueue *vq, diff --git a/include/linux/vtime.h b/include/linux/vtime.h index 29dd5b91dd7d..3fc04b849e4e 100644 --- a/include/linux/vtime.h +++ b/include/linux/vtime.h @@ -32,11 +32,17 @@ extern void vtime_account_irq(struct task_struct *tsk, unsigned int offset); extern void vtime_account_softirq(struct task_struct *tsk); extern void vtime_account_hardirq(struct task_struct *tsk); extern void vtime_flush(struct task_struct *tsk); +extern void vtime_reset(void); +extern void vtime_dyntick_start(void); +extern void vtime_dyntick_stop(void); #else /* !CONFIG_VIRT_CPU_ACCOUNTING_NATIVE */ static inline void vtime_account_irq(struct task_struct *tsk, unsigned int offset) { } static inline void vtime_account_softirq(struct task_struct *tsk) { } static inline void vtime_account_hardirq(struct task_struct *tsk) { } static inline void vtime_flush(struct task_struct *tsk) { } +static inline void vtime_reset(void) { } +static inline void vtime_dyntick_start(void) { } +static inline void vtime_dyntick_stop(void) { } #endif /* diff --git a/include/media/v4l2-async.h b/include/media/v4l2-async.h index f26c323e9c96..54a2d9620ed5 100644 --- a/include/media/v4l2-async.h +++ b/include/media/v4l2-async.h @@ -333,8 +333,10 @@ int __v4l2_async_register_subdev(struct v4l2_subdev *sd, struct module *module); * An error is returned if the module is no longer loaded on any attempts * to register it. */ +#define v4l2_async_register_subdev_sensor(sd) \ + __v4l2_async_register_subdev_sensor(sd, THIS_MODULE) int __must_check -v4l2_async_register_subdev_sensor(struct v4l2_subdev *sd); +__v4l2_async_register_subdev_sensor(struct v4l2_subdev *sd, struct module *module); /** * v4l2_async_unregister_subdev - unregisters a sub-device to the asynchronous diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index 9b02aeba4108..b86258b98bd2 100644 --- a/include/media/videobuf2-core.h +++ b/include/media/videobuf2-core.h @@ -1106,8 +1106,8 @@ __poll_t vb2_core_poll(struct vb2_queue *q, struct file *file, * @ppos: file handle position tracking pointer * @nonblock: mode selector (1 means blocking calls, 0 means nonblocking) */ -size_t vb2_read(struct vb2_queue *q, char __user *data, size_t count, - loff_t *ppos, int nonblock); +ssize_t vb2_read(struct vb2_queue *q, char __user *data, size_t count, + loff_t *ppos, int nonblock); /** * vb2_write() - implements write() syscall logic. * @q: pointer to &struct vb2_queue with videobuf2 queue. @@ -1116,8 +1116,8 @@ size_t vb2_read(struct vb2_queue *q, char __user *data, size_t count, * @ppos: file handle position tracking pointer * @nonblock: mode selector (1 means blocking calls, 0 means nonblocking) */ -size_t vb2_write(struct vb2_queue *q, const char __user *data, size_t count, - loff_t *ppos, int nonblock); +ssize_t vb2_write(struct vb2_queue *q, const char __user *data, size_t count, + loff_t *ppos, int nonblock); /** * typedef vb2_thread_fnc - callback function for use with vb2_thread. diff --git a/include/net/bluetooth/rfcomm.h b/include/net/bluetooth/rfcomm.h index c05882476900..7e40d3c5f5a9 100644 --- a/include/net/bluetooth/rfcomm.h +++ b/include/net/bluetooth/rfcomm.h @@ -229,6 +229,9 @@ int rfcomm_send_rpn(struct rfcomm_session *s, int cr, u8 dlci, u8 bit_rate, u8 data_bits, u8 stop_bits, u8 parity, u8 flow_ctrl_settings, u8 xon_char, u8 xoff_char, u16 param_mask); +int rfcomm_dlc_send_rpn(struct rfcomm_dlc *d, u8 bit_rate, u8 data_bits, + u8 stop_bits, u8 parity, u8 flow_ctrl_settings, + u8 xon_char, u8 xoff_char, u16 param_mask); /* ---- RFCOMM DLCs (channels) ---- */ struct rfcomm_dlc *rfcomm_dlc_alloc(gfp_t prio); diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h index 820e299f06b5..1509be85139e 100644 --- a/include/net/cfg80211.h +++ b/include/net/cfg80211.h @@ -6740,7 +6740,7 @@ struct wireless_dev { struct list_head pmsr_list; spinlock_t pmsr_lock; - struct work_struct pmsr_free_wk; + struct wiphy_work pmsr_free_wk; unsigned long unprot_beacon_reported; diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h index 58d91ccc56e0..de74e537b4ed 100644 --- a/include/net/dropreason-core.h +++ b/include/net/dropreason-core.h @@ -129,6 +129,7 @@ FN(DUALPI2_STEP_DROP) \ FN(PSP_INPUT) \ FN(PSP_OUTPUT) \ + FN(RECURSION_LIMIT) \ FNe(MAX) /** @@ -616,6 +617,8 @@ enum skb_drop_reason { SKB_DROP_REASON_PSP_INPUT, /** @SKB_DROP_REASON_PSP_OUTPUT: PSP output checks failed */ SKB_DROP_REASON_PSP_OUTPUT, + /** @SKB_DROP_REASON_RECURSION_LIMIT: Dead loop on virtual device. */ + SKB_DROP_REASON_RECURSION_LIMIT, /** * @SKB_DROP_REASON_MAX: the maximum of core drop reasons, which * shouldn't be used as a real 'reason' - only for tracing code gen diff --git a/include/net/ip6_tunnel.h b/include/net/ip6_tunnel.h index 359b595f1df9..b99805ee2fd1 100644 --- a/include/net/ip6_tunnel.h +++ b/include/net/ip6_tunnel.h @@ -162,7 +162,7 @@ static inline void ip6tunnel_xmit(struct sock *sk, struct sk_buff *skb, dev->name); DEV_STATS_INC(dev, tx_errors); } - kfree_skb(skb); + kfree_skb_reason(skb, SKB_DROP_REASON_RECURSION_LIMIT); return; } diff --git a/include/net/netfilter/nf_tables.h b/include/net/netfilter/nf_tables.h index b35e8c02fadc..fda5bc82dd50 100644 --- a/include/net/netfilter/nf_tables.h +++ b/include/net/netfilter/nf_tables.h @@ -123,17 +123,6 @@ struct nft_regs { }; }; -struct nft_regs_track { - struct { - const struct nft_expr *selector; - const struct nft_expr *bitwise; - u8 num_reg; - } regs[NFT_REG32_NUM]; - - const struct nft_expr *cur; - const struct nft_expr *last; -}; - /* Store/load an u8, u16 or u64 integer to/from the u32 data register. * * Note, when using concatenations, register allocation happens at 32-bit @@ -433,8 +422,6 @@ int nft_expr_clone(struct nft_expr *dst, struct nft_expr *src, gfp_t gfp); void nft_expr_destroy(const struct nft_ctx *ctx, struct nft_expr *expr); int nft_expr_dump(struct sk_buff *skb, unsigned int attr, const struct nft_expr *expr, bool reset); -bool nft_expr_reduce_bitwise(struct nft_regs_track *track, - const struct nft_expr *expr); struct nft_set_ext; @@ -949,7 +936,6 @@ struct nft_offload_ctx; * @destroy_clone: destruction clone function * @dump: function to dump parameters * @validate: validate expression, called during loop detection - * @reduce: reduce expression * @gc: garbage collection expression * @offload: hardware offload expression * @offload_action: function to report true/false to allocate one slot or not in the flow @@ -983,8 +969,6 @@ struct nft_expr_ops { bool reset); int (*validate)(const struct nft_ctx *ctx, const struct nft_expr *expr); - bool (*reduce)(struct nft_regs_track *track, - const struct nft_expr *expr); bool (*gc)(struct net *net, const struct nft_expr *expr); int (*offload)(struct nft_offload_ctx *ctx, @@ -1967,20 +1951,4 @@ static inline u64 nft_net_tstamp(const struct net *net) return nft_pernet(net)->tstamp; } -#define __NFT_REDUCE_READONLY 1UL -#define NFT_REDUCE_READONLY (void *)__NFT_REDUCE_READONLY - -void nft_reg_track_update(struct nft_regs_track *track, - const struct nft_expr *expr, u8 dreg, u8 len); -void nft_reg_track_cancel(struct nft_regs_track *track, u8 dreg, u8 len); -void __nft_reg_track_cancel(struct nft_regs_track *track, u8 dreg); - -static inline bool nft_reg_track_cmp(struct nft_regs_track *track, - const struct nft_expr *expr, u8 dreg) -{ - return track->regs[dreg].selector && - track->regs[dreg].selector->ops == expr->ops && - track->regs[dreg].num_reg == 0; -} - #endif /* _NET_NF_TABLES_H */ diff --git a/include/net/netfilter/nft_fib.h b/include/net/netfilter/nft_fib.h index 7370fba844ef..e0422456f27b 100644 --- a/include/net/netfilter/nft_fib.h +++ b/include/net/netfilter/nft_fib.h @@ -66,6 +66,4 @@ void nft_fib6_eval(const struct nft_expr *expr, struct nft_regs *regs, void nft_fib_store_result(void *reg, const struct nft_fib *priv, const struct net_device *dev); -bool nft_fib_reduce(struct nft_regs_track *track, - const struct nft_expr *expr); #endif diff --git a/include/net/netfilter/nft_meta.h b/include/net/netfilter/nft_meta.h index fab3e6658e91..6cf1d910bbf8 100644 --- a/include/net/netfilter/nft_meta.h +++ b/include/net/netfilter/nft_meta.h @@ -45,9 +45,6 @@ int nft_meta_get_validate(const struct nft_ctx *ctx, int nft_meta_set_validate(const struct nft_ctx *ctx, const struct nft_expr *expr); -bool nft_meta_get_reduce(struct nft_regs_track *track, - const struct nft_expr *expr); - struct nft_inner_tun_ctx; void nft_meta_inner_eval(const struct nft_expr *expr, struct nft_regs *regs, const struct nft_pktinfo *pkt, diff --git a/include/net/sctp/structs.h b/include/net/sctp/structs.h index 2ae390219efd..4bbba29f7341 100644 --- a/include/net/sctp/structs.h +++ b/include/net/sctp/structs.h @@ -316,7 +316,8 @@ struct sctp_cookie { __u8 auth_random[sizeof(struct sctp_paramhdr) + SCTP_AUTH_RANDOM_LENGTH]; - __u8 auth_hmacs[SCTP_AUTH_NUM_HMACS * sizeof(__u16) + 2]; + __u8 auth_hmacs[sizeof(struct sctp_paramhdr) + + SCTP_AUTH_NUM_HMACS * sizeof(__u16)]; __u8 auth_chunks[sizeof(struct sctp_paramhdr) + SCTP_AUTH_MAX_CHUNKS]; /* This is a shim for my peer's INIT packet, followed by diff --git a/include/net/tcp.h b/include/net/tcp.h index 31ac90230b12..c9439e7f3e7b 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -1878,6 +1878,8 @@ static inline void tcp_fast_path_check(struct sock *sk) bool tcp_oow_rate_limited(struct net *net, const struct sk_buff *skb, int mib_idx, u32 *last_oow_ack_time); +void tcp_reqsk_send_challenge_ack(struct sock *sk, struct sk_buff *skb, + struct request_sock *req); static inline void tcp_mib_init(struct net *net) { diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h index 5122f7846767..90b1ea19a90a 100644 --- a/include/rdma/ib_umem.h +++ b/include/rdma/ib_umem.h @@ -36,6 +36,7 @@ struct ib_umem_dmabuf { struct scatterlist *last_sg; unsigned long first_sg_offset; unsigned long last_sg_trim; + void (*pinned_revoke)(void *priv); void *private; u8 pinned : 1; u8 revoked : 1; @@ -141,6 +142,12 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get_pinned(struct ib_device *device, size_t size, int fd, int access); struct ib_umem_dmabuf * +ib_umem_dmabuf_get_pinned_revocable_and_lock(struct ib_device *device, + unsigned long offset, size_t size, + int fd, int access); +void ib_umem_dmabuf_set_revoke_locked(struct ib_umem_dmabuf *umem_dmabuf, + void (*revoke)(void *priv), void *priv); +struct ib_umem_dmabuf * ib_umem_dmabuf_get_pinned_with_dma_device(struct ib_device *device, struct device *dma_device, unsigned long offset, size_t size, @@ -201,6 +208,18 @@ ib_umem_dmabuf_get_pinned(struct ib_device *device, unsigned long offset, return ERR_PTR(-EOPNOTSUPP); } +static inline struct ib_umem_dmabuf * +ib_umem_dmabuf_get_pinned_revocable_and_lock(struct ib_device *device, + unsigned long offset, size_t size, + int fd, int access) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static inline void +ib_umem_dmabuf_set_revoke_locked(struct ib_umem_dmabuf *umem_dmabuf, + void (*revoke)(void *priv), void *priv) {} + static inline struct ib_umem_dmabuf * ib_umem_dmabuf_get_pinned_with_dma_device(struct ib_device *device, struct device *dma_device, diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index f5a243261236..936403c15e53 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -714,6 +714,9 @@ struct Scsi_Host { */ struct device *dma_dev; + /* Used for an rcu-synchronizing eh wakeup */ + struct work_struct eh_work; + /* Delay for runtime autosuspend */ int rpm_autosuspend_delay; diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index 8e710bbb688e..fb89c5c37eb0 100644 --- a/include/uapi/linux/btrfs.h +++ b/include/uapi/linux/btrfs.h @@ -597,7 +597,7 @@ struct btrfs_ioctl_search_args_v2 { __u64 buf_size; /* in - size of buffer * out - on EOVERFLOW: needed size * to store item */ - __u64 buf[]; /* out - found items */ + __u8 buf[]; /* out - found items */ }; /* With a @src_length of zero, the range from @src_offset->EOF is cloned! */ diff --git a/init/Kconfig b/init/Kconfig index 0c25e78e10f4..6ce315af48c1 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -162,6 +162,9 @@ config RUSTC_HAS_FILE_WITH_NUL config RUSTC_HAS_FILE_AS_C_STR def_bool RUSTC_VERSION >= 109100 +config RUSTC_HAS_SUSPICIOUS_RUNTIME_SYMBOL_DEFINITIONS + def_bool RUSTC_VERSION >= 109800 + config PAHOLE_VERSION int default $(shell,$(srctree)/scripts/pahole-version.sh $(PAHOLE)) diff --git a/init/main.c b/init/main.c index 048a62538242..e3ef1eb34aa3 100644 --- a/init/main.c +++ b/init/main.c @@ -320,51 +320,6 @@ static void * __init get_boot_config_from_initrd(size_t *_size) #ifdef CONFIG_BOOT_CONFIG -static char xbc_namebuf[XBC_KEYLEN_MAX] __initdata; - -#define rest(dst, end) ((end) > (dst) ? (end) - (dst) : 0) - -static int __init xbc_snprint_cmdline(char *buf, size_t size, - struct xbc_node *root) -{ - struct xbc_node *knode, *vnode; - char *end = buf + size; - const char *val, *q; - int ret; - - xbc_node_for_each_key_value(root, knode, val) { - ret = xbc_node_compose_key_after(root, knode, - xbc_namebuf, XBC_KEYLEN_MAX); - if (ret < 0) - return ret; - - vnode = xbc_node_get_child(knode); - if (!vnode) { - ret = snprintf(buf, rest(buf, end), "%s ", xbc_namebuf); - if (ret < 0) - return ret; - buf += ret; - continue; - } - xbc_array_for_each_value(vnode, val) { - /* - * For prettier and more readable /proc/cmdline, only - * quote the value when necessary, i.e. when it contains - * whitespace. - */ - q = strpbrk(val, " \t\r\n") ? "\"" : ""; - ret = snprintf(buf, rest(buf, end), "%s=%s%s%s ", - xbc_namebuf, q, val, q); - if (ret < 0) - return ret; - buf += ret; - } - } - - return buf - (end - size); -} -#undef rest - /* Make an extra command line under given key word */ static char * __init xbc_make_cmdline(const char *key) { diff --git a/io_uring/rw.c b/io_uring/rw.c index 1a5dee8fe619..3285a58409df 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -625,6 +625,24 @@ static void io_complete_rw_iopoll(struct kiocb *kiocb, long res) smp_store_release(&req->iopoll_completed, 1); } +static inline ssize_t io_fixup_restart_res(ssize_t ret) +{ + switch (ret) { + case -ERESTARTSYS: + case -ERESTARTNOINTR: + case -ERESTARTNOHAND: + case -ERESTART_RESTARTBLOCK: + /* + * We can't just restart the syscall, since previously + * submitted sqes may already be in progress. Just fail + * this IO with EINTR. + */ + return -EINTR; + default: + return ret; + } +} + static inline void io_rw_done(struct io_kiocb *req, ssize_t ret) { struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw); @@ -634,21 +652,8 @@ static inline void io_rw_done(struct io_kiocb *req, ssize_t ret) return; /* transform internal restart error codes */ - if (unlikely(ret < 0)) { - switch (ret) { - case -ERESTARTSYS: - case -ERESTARTNOINTR: - case -ERESTARTNOHAND: - case -ERESTART_RESTARTBLOCK: - /* - * We can't just restart the syscall, since previously - * submitted sqes may already be in progress. Just fail - * this IO with EINTR. - */ - ret = -EINTR; - break; - } - } + if (unlikely(ret < 0)) + ret = io_fixup_restart_res(ret); if (req->ctx->flags & IORING_SETUP_IOPOLL) io_complete_rw_iopoll(&rw->kiocb, ret); @@ -1044,7 +1049,8 @@ int io_read(struct io_kiocb *req, unsigned int issue_flags) if (req->flags & REQ_F_BUFFERS_COMMIT) io_kbuf_recycle(req, sel.buf_list, issue_flags); - return ret; + + return io_fixup_restart_res(ret); } int io_read_mshot(struct io_kiocb *req, unsigned int issue_flags) @@ -1078,8 +1084,10 @@ int io_read_mshot(struct io_kiocb *req, unsigned int issue_flags) return IOU_RETRY; } else if (ret <= 0) { io_kbuf_recycle(req, sel.buf_list, issue_flags); - if (ret < 0) + if (ret < 0) { + ret = io_fixup_restart_res(ret); req_set_fail(req); + } } else if (!(req->flags & REQ_F_APOLL_MULTISHOT)) { cflags = io_put_kbuf(req, ret, sel.buf_list); } else { diff --git a/kernel/audit.c b/kernel/audit.c index db7868945a87..e175fce41ef9 100644 --- a/kernel/audit.c +++ b/kernel/audit.c @@ -2030,7 +2030,7 @@ void audit_log_vformat(struct audit_buffer *ab, const char *fmt, va_list args) * here and AUDIT_BUFSIZ is at least 1024, then we can * log everything that printk could have logged. */ avail = audit_expand(ab, - max_t(unsigned, AUDIT_BUFSIZ, 1+len-avail)); + max_t(unsigned int, AUDIT_BUFSIZ, 1+len-avail)); if (!avail) goto out_va_end; len = vsnprintf(skb_tail_pointer(skb), avail, fmt, args2); diff --git a/kernel/audit.h b/kernel/audit.h index 0f05933a173b..08bd61f431b2 100644 --- a/kernel/audit.h +++ b/kernel/audit.h @@ -76,7 +76,7 @@ struct audit_names { int name_len; /* number of chars to log */ bool hidden; /* don't log this record */ - unsigned long ino; + u64 ino; dev_t dev; umode_t mode; kuid_t uid; @@ -225,15 +225,15 @@ extern int auditd_test_task(struct task_struct *task); #define AUDIT_INODE_BUCKETS 32 extern struct list_head audit_inode_hash[AUDIT_INODE_BUCKETS]; -static inline int audit_hash_ino(u32 ino) +static inline int audit_hash_ino(u64 ino) { - return (ino & (AUDIT_INODE_BUCKETS-1)); + return ((u32)ino & (AUDIT_INODE_BUCKETS-1)); } /* Indicates that audit should log the full pathname. */ #define AUDIT_NAME_FULL -1 -extern int audit_match_class(int class, unsigned syscall); +extern int audit_match_class(int class, unsigned int syscall); extern int audit_comparator(const u32 left, const u32 op, const u32 right); extern int audit_uid_comparator(kuid_t left, u32 op, kuid_t right); extern int audit_gid_comparator(kgid_t left, u32 op, kgid_t right); @@ -256,8 +256,13 @@ extern int audit_del_rule(struct audit_entry *entry); extern void audit_free_rule_rcu(struct rcu_head *head); extern struct list_head audit_filter_list[]; -extern struct audit_entry *audit_dupe_rule(struct audit_krule *old); +struct audit_watch_ctx { + struct inode *dir; + struct inode *child; +}; +extern struct audit_entry *audit_dupe_rule(struct audit_krule *old, + struct audit_watch_ctx *ctx); extern void audit_log_d_path_exe(struct audit_buffer *ab, struct mm_struct *mm); @@ -277,17 +282,18 @@ extern int audit_to_watch(struct audit_krule *krule, char *path, int len, extern int audit_add_watch(struct audit_krule *krule, struct list_head **list); extern void audit_remove_watch_rule(struct audit_krule *krule); extern char *audit_watch_path(struct audit_watch *watch); -extern int audit_watch_compare(struct audit_watch *watch, unsigned long ino, - dev_t dev); +extern int audit_watch_compare(struct audit_watch *watch, u64 ino, dev_t dev); extern struct audit_fsnotify_mark *audit_alloc_mark(struct audit_krule *krule, - char *pathname, int len); + char *pathname, int len, + struct audit_watch_ctx *ctx); extern char *audit_mark_path(struct audit_fsnotify_mark *mark); extern void audit_remove_mark(struct audit_fsnotify_mark *audit_mark); extern void audit_remove_mark_rule(struct audit_krule *krule); -extern int audit_mark_compare(struct audit_fsnotify_mark *mark, - unsigned long ino, dev_t dev); -extern int audit_dupe_exe(struct audit_krule *new, struct audit_krule *old); +extern int audit_mark_compare(struct audit_fsnotify_mark *mark, u64 ino, + dev_t dev); +extern int audit_dupe_exe(struct audit_krule *new, struct audit_krule *old, + struct audit_watch_ctx *ctx); extern int audit_exe_compare(struct task_struct *tsk, struct audit_fsnotify_mark *mark); @@ -318,13 +324,13 @@ extern struct list_head *audit_killed_trees(void); #define audit_watch_path(w) "" #define audit_watch_compare(w, i, d) 0 -#define audit_alloc_mark(k, p, l) (ERR_PTR(-EINVAL)) +#define audit_alloc_mark(k, p, l, c) (ERR_PTR(-EINVAL)) #define audit_mark_path(m) "" #define audit_remove_mark(m) do { } while (0) #define audit_remove_mark_rule(k) do { } while (0) #define audit_mark_compare(m, i, d) 0 #define audit_exe_compare(t, m) (-EINVAL) -#define audit_dupe_exe(n, o) (-EINVAL) +#define audit_dupe_exe(n, o, c) (-EINVAL) #define audit_remove_tree_rule(rule) BUG() #define audit_add_tree_rule(rule) -EINVAL diff --git a/kernel/audit_fsnotify.c b/kernel/audit_fsnotify.c index 0046f93ae914..e62066bf0b0a 100644 --- a/kernel/audit_fsnotify.c +++ b/kernel/audit_fsnotify.c @@ -25,7 +25,7 @@ */ struct audit_fsnotify_mark { dev_t dev; /* associated superblock device */ - unsigned long ino; /* associated inode number */ + u64 ino; /* associated inode number */ char *path; /* insertion path */ struct fsnotify_mark mark; /* fsnotify mark on the inode */ struct audit_krule *rule; @@ -57,7 +57,7 @@ char *audit_mark_path(struct audit_fsnotify_mark *mark) return mark->path; } -int audit_mark_compare(struct audit_fsnotify_mark *mark, unsigned long ino, dev_t dev) +int audit_mark_compare(struct audit_fsnotify_mark *mark, u64 ino, dev_t dev) { if (mark->ino == AUDIT_INO_UNSET) return 0; @@ -71,19 +71,30 @@ static void audit_update_mark(struct audit_fsnotify_mark *audit_mark, audit_mark->ino = inode ? inode->i_ino : AUDIT_INO_UNSET; } -struct audit_fsnotify_mark *audit_alloc_mark(struct audit_krule *krule, char *pathname, int len) +struct audit_fsnotify_mark *audit_alloc_mark(struct audit_krule *krule, char *pathname, + int len, struct audit_watch_ctx *ctx) { struct audit_fsnotify_mark *audit_mark; struct path path; struct dentry *dentry; - int ret; + struct inode *dir, *child; + int ret, allow_dups; if (pathname[0] != '/' || pathname[len-1] == '/') return ERR_PTR(-EINVAL); - dentry = kern_path_parent(pathname, &path); - if (IS_ERR(dentry)) - return ERR_CAST(dentry); /* returning an error */ + if (!ctx) { + dentry = kern_path_parent(pathname, &path); + if (IS_ERR(dentry)) + return ERR_CAST(dentry); /* returning an error */ + dir = d_inode(path.dentry); + child = d_inode(dentry); + allow_dups = 0; + } else { + dir = ctx->dir; + child = ctx->child; + allow_dups = 1; + } audit_mark = kzalloc(sizeof(*audit_mark), GFP_KERNEL); if (unlikely(!audit_mark)) { @@ -94,18 +105,21 @@ struct audit_fsnotify_mark *audit_alloc_mark(struct audit_krule *krule, char *pa fsnotify_init_mark(&audit_mark->mark, audit_fsnotify_group); audit_mark->mark.mask = AUDIT_FS_EVENTS; audit_mark->path = pathname; - audit_update_mark(audit_mark, dentry->d_inode); audit_mark->rule = krule; - ret = fsnotify_add_inode_mark(&audit_mark->mark, path.dentry->d_inode, 0); + audit_update_mark(audit_mark, child); + ret = fsnotify_add_inode_mark(&audit_mark->mark, dir, allow_dups); + if (ret < 0) { audit_mark->path = NULL; fsnotify_put_mark(&audit_mark->mark); audit_mark = ERR_PTR(ret); } out: - dput(dentry); - path_put(&path); + if (!ctx) { + dput(dentry); + path_put(&path); + } return audit_mark; } diff --git a/kernel/audit_tree.c b/kernel/audit_tree.c index fda6beb041e0..f2c13a62bfb6 100644 --- a/kernel/audit_tree.c +++ b/kernel/audit_tree.c @@ -33,7 +33,7 @@ struct audit_chunk { struct audit_node { struct list_head list; struct audit_tree *owner; - unsigned index; /* index; upper bit indicates 'will prune' */ + unsigned int index; /* index; upper bit indicates 'will prune' */ } owners[] __counted_by(count); }; diff --git a/kernel/audit_watch.c b/kernel/audit_watch.c index a700e3c8925f..ec88f38ee685 100644 --- a/kernel/audit_watch.c +++ b/kernel/audit_watch.c @@ -37,7 +37,7 @@ struct audit_watch { refcount_t count; /* reference count */ dev_t dev; /* associated superblock device */ char *path; /* insertion path */ - unsigned long ino; /* associated inode number */ + u64 ino; /* associated inode number */ struct audit_parent *parent; /* associated parent */ struct list_head wlist; /* entry in parent->watches list */ struct list_head rules; /* anchor for krule->rlist */ @@ -125,7 +125,7 @@ char *audit_watch_path(struct audit_watch *watch) return watch->path; } -int audit_watch_compare(struct audit_watch *watch, unsigned long ino, dev_t dev) +int audit_watch_compare(struct audit_watch *watch, u64 ino, dev_t dev) { return (watch->ino != AUDIT_INO_UNSET) && (watch->ino == ino) && @@ -244,7 +244,8 @@ static void audit_watch_log_rule_change(struct audit_krule *r, struct audit_watc /* Update inode info in audit rules based on filesystem event. */ static void audit_update_watch(struct audit_parent *parent, const struct qstr *dname, dev_t dev, - unsigned long ino, unsigned invalidating) + u64 ino, unsigned int invalidating, + struct audit_watch_ctx *ctx) { struct audit_watch *owatch, *nwatch, *nextw; struct audit_krule *r, *nextr; @@ -280,12 +281,12 @@ static void audit_update_watch(struct audit_parent *parent, list_del(&oentry->rule.rlist); list_del_rcu(&oentry->list); - nentry = audit_dupe_rule(&oentry->rule); + nentry = audit_dupe_rule(&oentry->rule, ctx); if (IS_ERR(nentry)) { list_del(&oentry->rule.list); audit_panic("error updating watch, removing"); } else { - int h = audit_hash_ino((u32)ino); + int h = audit_hash_ino(ino); /* * nentry->rule.watch == oentry->rule.watch so @@ -439,7 +440,7 @@ int audit_add_watch(struct audit_krule *krule, struct list_head **list) audit_add_to_parent(krule, parent); - h = audit_hash_ino((u32)watch->ino); + h = audit_hash_ino(watch->ino); *list = &audit_inode_hash[h]; error: path_put(&parent_path); @@ -479,10 +480,17 @@ static int audit_watch_handle_event(struct fsnotify_mark *inode_mark, u32 mask, if (WARN_ON_ONCE(inode_mark->group != audit_watch_group)) return 0; - if (mask & (FS_CREATE|FS_MOVED_TO) && inode) - audit_update_watch(parent, dname, inode->i_sb->s_dev, inode->i_ino, 0); - else if (mask & (FS_DELETE|FS_MOVED_FROM)) - audit_update_watch(parent, dname, AUDIT_DEV_UNSET, AUDIT_INO_UNSET, 1); + if (mask & (FS_CREATE|FS_MOVED_TO) && inode) { + struct audit_watch_ctx ctx = { .dir = dir, .child = inode }; + + audit_update_watch(parent, dname, inode->i_sb->s_dev, inode->i_ino, 0, + &ctx); + } else if (mask & (FS_DELETE|FS_MOVED_FROM)) { + struct audit_watch_ctx ctx = { .dir = dir, .child = NULL }; + + audit_update_watch(parent, dname, AUDIT_DEV_UNSET, AUDIT_INO_UNSET, 1, + &ctx); + } else if (mask & (FS_DELETE_SELF|FS_UNMOUNT|FS_MOVE_SELF)) audit_remove_parent_watches(parent); @@ -505,7 +513,8 @@ static int __init audit_watch_init(void) } device_initcall(audit_watch_init); -int audit_dupe_exe(struct audit_krule *new, struct audit_krule *old) +int audit_dupe_exe(struct audit_krule *new, struct audit_krule *old, + struct audit_watch_ctx *ctx) { struct audit_fsnotify_mark *audit_mark; char *pathname; @@ -514,7 +523,7 @@ int audit_dupe_exe(struct audit_krule *new, struct audit_krule *old) if (!pathname) return -ENOMEM; - audit_mark = audit_alloc_mark(new, pathname, strlen(pathname)); + audit_mark = audit_alloc_mark(new, pathname, strlen(pathname), ctx); if (IS_ERR(audit_mark)) { kfree(pathname); return PTR_ERR(audit_mark); @@ -527,7 +536,7 @@ int audit_dupe_exe(struct audit_krule *new, struct audit_krule *old) int audit_exe_compare(struct task_struct *tsk, struct audit_fsnotify_mark *mark) { struct file *exe_file; - unsigned long ino; + u64 ino; dev_t dev; /* only do exe filtering if we are recording @current events/records */ diff --git a/kernel/auditfilter.c b/kernel/auditfilter.c index c401082d9b25..d0bd6e549634 100644 --- a/kernel/auditfilter.c +++ b/kernel/auditfilter.c @@ -165,13 +165,13 @@ static inline int audit_to_inode(struct audit_krule *krule, static __u32 *classes[AUDIT_SYSCALL_CLASSES]; -int __init audit_register_class(int class, unsigned *list) +int __init audit_register_class(int class, unsigned int *list) { __u32 *p = kcalloc(AUDIT_BITMASK_SIZE, sizeof(__u32), GFP_KERNEL); if (!p) return -ENOMEM; while (*list != ~0U) { - unsigned n = *list++; + unsigned int n = *list++; if (n >= AUDIT_BITMASK_SIZE * 32 - AUDIT_SYSCALL_CLASSES) { kfree(p); return -EINVAL; @@ -186,7 +186,7 @@ int __init audit_register_class(int class, unsigned *list) return 0; } -int audit_match_class(int class, unsigned syscall) +int audit_match_class(int class, unsigned int syscall) { if (unlikely(syscall >= AUDIT_BITMASK_SIZE * 32)) return 0; @@ -237,7 +237,7 @@ static int audit_match_signal(struct audit_entry *entry) /* Common user-space to kernel rule translation. */ static inline struct audit_entry *audit_to_entry_common(struct audit_rule_data *rule) { - unsigned listnr; + unsigned int listnr; struct audit_entry *entry; int i, err; @@ -590,7 +590,7 @@ static struct audit_entry *audit_data_to_entry(struct audit_rule_data *data, err = PTR_ERR(str); goto exit_free; } - audit_mark = audit_alloc_mark(&entry->rule, str, f_val); + audit_mark = audit_alloc_mark(&entry->rule, str, f_val, NULL); if (IS_ERR(audit_mark)) { kfree(str); err = PTR_ERR(audit_mark); @@ -818,7 +818,8 @@ static inline int audit_dupe_lsm_field(struct audit_field *df, * rule with the new rule in the filterlist, then free the old rule. * The rlist element is undefined; list manipulations are handled apart from * the initial copy. */ -struct audit_entry *audit_dupe_rule(struct audit_krule *old) +struct audit_entry *audit_dupe_rule(struct audit_krule *old, + struct audit_watch_ctx *ctx) { u32 fcount = old->field_count; struct audit_entry *entry; @@ -877,7 +878,7 @@ struct audit_entry *audit_dupe_rule(struct audit_krule *old) new->filterkey = fk; break; case AUDIT_EXE: - err = audit_dupe_exe(new, old); + err = audit_dupe_exe(new, old, ctx); break; } if (err) { @@ -1416,7 +1417,7 @@ static int update_lsm_rule(struct audit_krule *r) if (!security_audit_rule_known(r)) return 0; - nentry = audit_dupe_rule(r); + nentry = audit_dupe_rule(r, NULL); if (entry->rule.exe) audit_remove_mark(entry->rule.exe); if (IS_ERR(nentry)) { diff --git a/kernel/auditsc.c b/kernel/auditsc.c index 3e42d32ebb28..c547add13028 100644 --- a/kernel/auditsc.c +++ b/kernel/auditsc.c @@ -150,7 +150,7 @@ static const struct audit_nfcfgop_tab audit_nfcfgs[] = { static int audit_match_perm(struct audit_context *ctx, int mask) { - unsigned n; + unsigned int n; if (unlikely(!ctx)) return 0; @@ -886,7 +886,7 @@ static int audit_filter_inode_name(struct task_struct *tsk, struct audit_names *n, struct audit_context *ctx) { - int h = audit_hash_ino((u32)n->ino); + int h = audit_hash_ino(n->ino); struct list_head *list = &audit_inode_hash[h]; return __audit_filter_op(tsk, ctx, list, n, ctx->major); @@ -1534,7 +1534,7 @@ static void audit_log_name(struct audit_context *context, struct audit_names *n, audit_log_format(ab, " name=(null)"); if (n->ino != AUDIT_INO_UNSET) - audit_log_format(ab, " inode=%lu dev=%02x:%02x mode=%#ho ouid=%u ogid=%u rdev=%02x:%02x", + audit_log_format(ab, " inode=%llu dev=%02x:%02x mode=%#ho ouid=%u ogid=%u rdev=%02x:%02x", n->ino, MAJOR(n->dev), MINOR(n->dev), diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d1c51d812f3b..463455180a8e 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -17402,6 +17402,23 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn) mark_reg_unknown(env, regs, BPF_REG_0); /* ld_abs load up to 32-bit skb data. */ regs[BPF_REG_0].subreg_def = env->insn_idx + 1; + /* + * See bpf_gen_ld_abs() which emits a hidden BPF_EXIT with r0=0 + * which must be explored by the verifier when in a subprog. + */ + if (env->cur_state->curframe) { + struct bpf_verifier_state *branch; + + mark_reg_scratched(env, BPF_REG_0); + branch = push_stack(env, env->insn_idx + 1, env->insn_idx, false); + if (!branch) + return -EFAULT; + mark_reg_known_zero(env, regs, BPF_REG_0); + err = prepare_func_exit(env, &env->insn_idx); + if (err) + return err; + env->insn_idx--; + } return 0; } diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index c13b99f7c989..e1a06b4fc17f 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -562,20 +562,30 @@ early_param("kho_scratch", kho_parse_scratch_size); static void __init scratch_size_update(void) { - phys_addr_t size; + /* + * If fixed sizes are not provided via command line, calculate them + * now. + */ + if (scratch_scale) { + phys_addr_t size; - if (!scratch_scale) - return; + size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT, + NUMA_NO_NODE); + size = size * scratch_scale / 100; + scratch_size_lowmem = size; - size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT, - NUMA_NO_NODE); - size = size * scratch_scale / 100; - scratch_size_lowmem = round_up(size, CMA_MIN_ALIGNMENT_BYTES); + size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE, + NUMA_NO_NODE); + size = size * scratch_scale / 100 - scratch_size_lowmem; + scratch_size_global = size; + } - size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE, - NUMA_NO_NODE); - size = size * scratch_scale / 100 - scratch_size_lowmem; - scratch_size_global = round_up(size, CMA_MIN_ALIGNMENT_BYTES); + /* + * Scratch areas are released as MIGRATE_CMA. Round them up to the right + * size. + */ + scratch_size_lowmem = round_up(scratch_size_lowmem, CMA_MIN_ALIGNMENT_BYTES); + scratch_size_global = round_up(scratch_size_global, CMA_MIN_ALIGNMENT_BYTES); } static phys_addr_t __init scratch_size_node(int nid) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index d8280f874433..83baf2708888 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -2395,9 +2395,14 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p, * sched_class, %SCX_OPS_ENQ_LAST must be set. Tell * ops.enqueue() that @p is the only one available for this cpu, * which should trigger an explicit follow-up scheduling event. + * + * Core scheduling can force this CPU idle while @p stays + * runnable. @p's cookie then won't match the core's, so skip + * the warning in that case. */ - if (sched_class_above(&ext_sched_class, next->sched_class)) { - WARN_ON_ONCE(!(sch->ops.flags & SCX_OPS_ENQ_LAST)); + if (next && sched_class_above(&ext_sched_class, next->sched_class)) { + WARN_ON_ONCE(sched_cpu_cookie_match(rq, p) && + !(sch->ops.flags & SCX_OPS_ENQ_LAST)); do_enqueue_task(rq, p, SCX_ENQ_LAST, -1); } else { do_enqueue_task(rq, p, 0, -1); diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 905f4d167955..d451926bc55d 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -1072,6 +1072,12 @@ struct ftrace_ops global_ops = { FTRACE_OPS_FL_PID, }; +/* + * parser_lock - Protects trace_parser state against concurrent operations. + * Held across trace_get_user() and subsequent buffer parsing to prevent races. + */ +static DEFINE_MUTEX(parser_lock); + /* * Used by the stack unwinder to know about dynamic ftrace trampolines. */ @@ -5783,6 +5789,8 @@ ftrace_regex_write(struct file *file, const char __user *ubuf, /* iter->hash is a local copy, so we don't need regex_lock */ parser = &iter->parser; + + guard(mutex)(&parser_lock); read = trace_get_user(parser, ubuf, cnt, ppos); if (read >= 0 && trace_parser_loaded(parser) && @@ -6551,12 +6559,14 @@ int ftrace_regex_release(struct inode *inode, struct file *file) iter = file->private_data; parser = &iter->parser; + mutex_lock(&parser_lock); if (trace_parser_loaded(parser)) { int enable = !(iter->flags & FTRACE_ITER_NOTRACE); ftrace_process_regex(iter, parser->buffer, parser->idx, enable); } + mutex_unlock(&parser_lock); trace_parser_put(parser); @@ -6888,10 +6898,12 @@ ftrace_graph_release(struct inode *inode, struct file *file) parser = &fgd->parser; + mutex_lock(&parser_lock); if (trace_parser_loaded((parser))) { ret = ftrace_graph_set_hash(fgd->new_hash, parser->buffer); } + mutex_unlock(&parser_lock); trace_parser_put(parser); @@ -7004,6 +7016,7 @@ ftrace_graph_write(struct file *file, const char __user *ubuf, parser = &fgd->parser; + guard(mutex)(&parser_lock); read = trace_get_user(parser, ubuf, cnt, ppos); if (read >= 0 && trace_parser_loaded(parser) && diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 5b1693d58af0..914d6baa1bf9 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -7325,7 +7325,7 @@ static char *trace_user_fault_read(struct trace_user_buf_info *tinfo, { int cpu = smp_processor_id(); char *buffer = per_cpu_ptr(tinfo->tbuf, cpu)->buf; - unsigned int cnt; + unsigned long long cnt; int trys = 0; int ret; diff --git a/kernel/trace/trace_eprobe.c b/kernel/trace/trace_eprobe.c index 7697ce22337b..0a7960641947 100644 --- a/kernel/trace/trace_eprobe.c +++ b/kernel/trace/trace_eprobe.c @@ -169,7 +169,8 @@ static bool eprobe_dyn_event_match(const char *system, const char *event, if (!slash) return false; - if (strncmp(ep->event_system, argv[0], slash - argv[0])) + if (strncmp(ep->event_system, argv[0], slash - argv[0]) || + ep->event_system[slash - argv[0]] != '\0') return false; if (strcmp(ep->event_name, slash + 1)) return false; diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index b86f128ff99f..b0842df4d6e9 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -1333,7 +1333,9 @@ __ftrace_set_clr_event_nolock(struct trace_array *tr, const char *match, call = file->event_call; /* If a module is specified, skip events that are not that module */ - if (module && (!call->module || strcmp(module_name(call->module), module))) + if (module && + ((call->flags & TRACE_EVENT_FL_DYNAMIC) || + !call->module || strcmp(module_name(call->module), module))) continue; name = trace_event_name(call); diff --git a/kernel/trace/trace_mmiotrace.c b/kernel/trace/trace_mmiotrace.c index c706544be60c..dc8a1eafbe57 100644 --- a/kernel/trace/trace_mmiotrace.c +++ b/kernel/trace/trace_mmiotrace.c @@ -109,7 +109,6 @@ static void mmio_pipe_open(struct trace_iterator *iter) iter->private = hiter; } -/* XXX: This is not called when the pipe is closed! */ static void mmio_close(struct trace_iterator *iter) { struct header_iter *hiter = iter->private; @@ -146,7 +145,7 @@ static ssize_t mmio_read(struct trace_iterator *iter, struct file *filp, goto print_out; } - if (!hiter) + if (!hiter || !hiter->dev) return 0; mmio_print_pcidev(s, hiter->dev); @@ -279,6 +278,7 @@ static struct tracer mmio_tracer __read_mostly = .start = mmio_trace_start, .pipe_open = mmio_pipe_open, .close = mmio_close, + .pipe_close = mmio_close, .read = mmio_read, .print_line = mmio_print_line, .noboot = true, diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c index 2333e5446c9f..918fafc5d45c 100644 --- a/kernel/trace/trace_probe.c +++ b/kernel/trace/trace_probe.c @@ -187,7 +187,7 @@ void __trace_probe_log_err(int offset, int err_type) lockdep_assert_held(&dyn_event_ops_mutex); - if (!trace_probe_log.argv) + if (!trace_probe_log.argv || !trace_probe_log.argc) return; /* Recalculate the length and allocate buffer */ @@ -2018,7 +2018,7 @@ int traceprobe_update_arg(struct probe_arg *arg) } /* When len=0, we just calculate the needed length */ -#define LEN_OR_ZERO (len ? len - pos : 0) +#define LEN_OR_ZERO (len > pos ? len - pos : 0) static int __set_print_fmt(struct trace_probe *tp, char *buf, int len, enum probe_print_type ptype) { @@ -2344,16 +2344,17 @@ int trace_probe_compare_arg_type(struct trace_probe *a, struct trace_probe *b) bool trace_probe_match_command_args(struct trace_probe *tp, int argc, const char **argv) { - char buf[MAX_ARGSTR_LEN + 1]; int i; if (tp->nr_args < argc) return false; for (i = 0; i < argc; i++) { - snprintf(buf, sizeof(buf), "%s=%s", - tp->args[i].name, tp->args[i].comm); - if (strcmp(buf, argv[i])) + int len = strlen(tp->args[i].name); + + if (strncmp(argv[i], tp->args[i].name, len) || + argv[i][len] != '=' || + strcmp(argv[i] + len + 1, tp->args[i].comm)) return false; } return true; diff --git a/lib/bootconfig.c b/lib/bootconfig.c index 5d3802eba52a..b4e8e17e2915 100644 --- a/lib/bootconfig.c +++ b/lib/bootconfig.c @@ -405,6 +405,71 @@ const char * __init xbc_node_find_next_key_value(struct xbc_node *root, return ""; /* No value key */ } +static char xbc_namebuf[XBC_KEYLEN_MAX] __initdata; + +#define rest(dst, end) ((end) > (dst) ? (end) - (dst) : 0) + +/** + * xbc_snprint_cmdline() - Render bootconfig keys under @root as a cmdline string + * @buf: Destination buffer (may be NULL when @size is 0 to query the length) + * @size: Size of @buf in bytes + * @root: Subtree root whose key=value pairs should be rendered + * + * Walk all key/value pairs under @root and emit them as a space-separated + * cmdline string into @buf. Values containing whitespace are quoted with + * double quotes. Returns the number of bytes that would be written if @buf + * were large enough (matching snprintf semantics), or a negative errno on + * failure. + */ +int __init xbc_snprint_cmdline(char *buf, size_t size, struct xbc_node *root) +{ + struct xbc_node *knode, *vnode; + const char *val, *q; + size_t len = 0; + int ret; + + /* + * Track the running written length rather than advancing @buf, so we + * never form "buf + size" or "buf += ret" while @buf is NULL (the + * size-probe call passes buf=NULL, size=0). NULL pointer arithmetic + * is undefined behavior and trips host UBSan / FORTIFY_SOURCE when + * this renderer runs at kernel build time. snprintf(NULL, 0, ...) + * itself is well defined and returns the would-be length. + */ + xbc_node_for_each_key_value(root, knode, val) { + ret = xbc_node_compose_key_after(root, knode, + xbc_namebuf, XBC_KEYLEN_MAX); + if (ret < 0) + return ret; + + vnode = xbc_node_get_child(knode); + if (!vnode) { + ret = snprintf(buf ? buf + len : NULL, rest(len, size), + "%s ", xbc_namebuf); + if (ret < 0) + return ret; + len += ret; + continue; + } + xbc_array_for_each_value(vnode, val) { + /* + * For prettier and more readable /proc/cmdline, only + * quote the value when necessary, i.e. when it contains + * whitespace. + */ + q = strpbrk(val, " \t\r\n") ? "\"" : ""; + ret = snprintf(buf ? buf + len : NULL, rest(len, size), + "%s=%s%s%s ", xbc_namebuf, q, val, q); + if (ret < 0) + return ret; + len += ret; + } + } + + return len; +} +#undef rest + /* XBC parse and tree build */ static int __init xbc_init_node(struct xbc_node *node, char *data, uint32_t flag) diff --git a/lib/compat_audit.c b/lib/compat_audit.c index 3d6b8996f027..fee1dfccd116 100644 --- a/lib/compat_audit.c +++ b/lib/compat_audit.c @@ -4,32 +4,32 @@ #include #include -unsigned compat_dir_class[] = { +unsigned int compat_dir_class[] = { #include ~0U }; -unsigned compat_read_class[] = { +unsigned int compat_read_class[] = { #include ~0U }; -unsigned compat_write_class[] = { +unsigned int compat_write_class[] = { #include ~0U }; -unsigned compat_chattr_class[] = { +unsigned int compat_chattr_class[] = { #include ~0U }; -unsigned compat_signal_class[] = { +unsigned int compat_signal_class[] = { #include ~0U }; -int audit_classify_compat_syscall(int abi, unsigned syscall) +int audit_classify_compat_syscall(int abi, unsigned int syscall) { switch (syscall) { #ifdef __NR_open diff --git a/lib/rhashtable.c b/lib/rhashtable.c index fde0f0e556f8..fcc7bd22d86e 100644 --- a/lib/rhashtable.c +++ b/lib/rhashtable.c @@ -742,6 +742,7 @@ int rhashtable_walk_start_check(struct rhashtable_iter *iter) iter->walker.tbl = rht_dereference_rcu(ht->tbl, ht); iter->slot = 0; iter->skip = 0; + iter->p = NULL; return -EAGAIN; } diff --git a/mm/damon/core.c b/mm/damon/core.c index 3e221c888e30..70ac1f08753d 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -213,8 +213,21 @@ int damon_set_regions(struct damon_target *t, struct damon_addr_range *ranges, { struct damon_region *r, *next; unsigned int i; + unsigned long last_end; int err; + for (i = 0; i < nr_ranges; i++) { + unsigned long start, end; + + start = ALIGN_DOWN(ranges[i].start, min_sz_region); + end = ALIGN(ranges[i].end, min_sz_region); + if (start >= end) + return -EINVAL; + if (i > 0 && last_end > start) + return -EINVAL; + last_end = end; + } + /* Remove regions which are not in the new ranges */ damon_for_each_region_safe(r, next, t) { for (i = 0; i < nr_ranges; i++) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b90be9f16b49..c08aa3db9643 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3344,10 +3344,6 @@ static void __split_folio_to_order(struct folio *folio, int old_order, (1L << PG_dirty) | LRU_GEN_MASK | LRU_REFS_MASK)); - if (handle_hwpoison && - page_range_has_hwpoisoned(new_head, new_nr_pages)) - folio_set_has_hwpoisoned(new_folio); - new_folio->mapping = folio->mapping; new_folio->index = folio->index + i; @@ -3378,6 +3374,14 @@ static void __split_folio_to_order(struct folio *folio, int old_order, folio_set_large_rmappable(new_folio); } + /* + * PG_has_hwpoisoned is on the 2nd page, so set it after + * the compound head is prepped. + */ + if (handle_hwpoison && + page_range_has_hwpoisoned(new_head, new_nr_pages)) + folio_set_has_hwpoisoned(new_folio); + if (folio_test_young(folio)) folio_set_young(new_folio); if (folio_test_idle(folio)) diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 1ac56ceb29b6..415b154fdf38 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -676,7 +676,7 @@ static struct kmemleak_object *__alloc_object(gfp_t gfp) atomic_set(&object->use_count, 1); object->excess_ref = 0; object->count = 0; /* white color initially */ - object->checksum = 0; + object->checksum = ~0; object->del_state = 0; /* task information */ @@ -972,7 +972,7 @@ static void reset_checksum(unsigned long ptr) } raw_spin_lock_irqsave(&object->lock, flags); - object->checksum = 0; + object->checksum = ~0; raw_spin_unlock_irqrestore(&object->lock, flags); put_object(object); } @@ -1401,7 +1401,8 @@ static bool update_checksum(struct kmemleak_object *object) for_each_possible_cpu(cpu) { void *ptr = per_cpu_ptr((void __percpu *)object->pointer, cpu); - object->checksum ^= crc32(0, kasan_reset_tag((void *)ptr), object->size); + object->checksum = crc32(object->checksum, + kasan_reset_tag((void *)ptr), object->size); } } else { object->checksum = crc32(0, kasan_reset_tag((void *)object->pointer), object->size); diff --git a/mm/sparse.c b/mm/sparse.c index 17c50a6415c2..bda92a084681 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -670,7 +670,12 @@ static struct page * __meminit populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { - return __populate_section_memmap(pfn, nr_pages, nid, altmap, pgmap); + struct page *page = __populate_section_memmap(pfn, nr_pages, nid, altmap, + pgmap); + + memmap_pages_add(DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE)); + + return page; } static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages, @@ -679,13 +684,17 @@ static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages, unsigned long start = (unsigned long) pfn_to_page(pfn); unsigned long end = start + nr_pages * sizeof(struct page); + memmap_pages_add(-1L * (DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE))); vmemmap_free(start, end, altmap); } + static void free_map_bootmem(struct page *memmap) { unsigned long start = (unsigned long)memmap; unsigned long end = (unsigned long)(memmap + PAGES_PER_SECTION); + memmap_boot_pages_add(-1L * (DIV_ROUND_UP(PAGES_PER_SECTION * sizeof(struct page), + PAGE_SIZE))); vmemmap_free(start, end, NULL); } @@ -742,13 +751,18 @@ static struct page * __meminit populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { - return kvmalloc_node(array_size(sizeof(struct page), - PAGES_PER_SECTION), GFP_KERNEL, nid); + struct page *page = kvmalloc_node(array_size(sizeof(struct page), + PAGES_PER_SECTION), GFP_KERNEL, nid); + + memmap_pages_add(DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE)); + + return page; } static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages, struct vmem_altmap *altmap) { + memmap_pages_add(-1L * (DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE))); kvfree(pfn_to_page(pfn)); } @@ -761,6 +775,9 @@ static void free_map_bootmem(struct page *memmap) nr_pages = PAGE_ALIGN(PAGES_PER_SECTION * sizeof(struct page)) >> PAGE_SHIFT; + memmap_boot_pages_add(-1L * (DIV_ROUND_UP(PAGES_PER_SECTION * sizeof(struct page), + PAGE_SIZE))); + for (i = 0; i < nr_pages; i++, page++) { type = bootmem_type(page); @@ -854,14 +871,10 @@ static void section_deactivate(unsigned long pfn, unsigned long nr_pages, * The memmap of early sections is always fully populated. See * section_activate() and pfn_valid() . */ - if (!section_is_early) { - memmap_pages_add(-1L * (DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE))); + if (!section_is_early) depopulate_section_memmap(pfn, nr_pages, altmap); - } else if (memmap) { - memmap_boot_pages_add(-1L * (DIV_ROUND_UP(nr_pages * sizeof(struct page), - PAGE_SIZE))); + else if (memmap) free_map_bootmem(memmap); - } if (empty) ms->section_mem_map = (unsigned long)NULL; @@ -906,7 +919,6 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn, section_deactivate(pfn, nr_pages, altmap); return ERR_PTR(-ENOMEM); } - memmap_pages_add(DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE)); return memmap; } diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c index a8404027724d..65c8a490c555 100644 --- a/net/bluetooth/hci_sync.c +++ b/net/bluetooth/hci_sync.c @@ -929,12 +929,16 @@ int hci_update_eir_sync(struct hci_dev *hdev) memset(&cp, 0, sizeof(cp)); + hci_dev_lock(hdev); eir_create(hdev, cp.data); - if (memcmp(cp.data, hdev->eir, sizeof(cp.data)) == 0) + if (memcmp(cp.data, hdev->eir, sizeof(cp.data)) == 0) { + hci_dev_unlock(hdev); return 0; + } memcpy(hdev->eir, cp.data, sizeof(cp.data)); + hci_dev_unlock(hdev); return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_EIR, sizeof(cp), &cp, HCI_CMD_TIMEOUT); @@ -966,6 +970,7 @@ int hci_update_class_sync(struct hci_dev *hdev) if (hci_dev_test_flag(hdev, HCI_SERVICE_CACHE)) return 0; + hci_dev_lock(hdev); cod[0] = hdev->minor_class; cod[1] = hdev->major_class; cod[2] = get_service_classes(hdev); @@ -973,8 +978,12 @@ int hci_update_class_sync(struct hci_dev *hdev) if (hci_dev_test_flag(hdev, HCI_LIMITED_DISCOVERABLE)) cod[1] |= 0x20; - if (memcmp(cod, hdev->dev_class, 3) == 0) + if (memcmp(cod, hdev->dev_class, 3) == 0) { + hci_dev_unlock(hdev); return 0; + } + + hci_dev_unlock(hdev); return __hci_cmd_sync_status(hdev, HCI_OP_WRITE_CLASS_OF_DEV, sizeof(cod), cod, HCI_CMD_TIMEOUT); @@ -1054,14 +1063,19 @@ static int hci_set_random_addr_sync(struct hci_dev *hdev, bdaddr_t *rpa) * In this kind of scenario skip the update and let the random * address be updated at the next cycle. */ + rcu_read_lock(); + if (bacmp(&hdev->random_addr, BDADDR_ANY) && (hci_dev_test_flag(hdev, HCI_LE_ADV) || hci_lookup_le_connect(hdev))) { bt_dev_dbg(hdev, "Deferring random address update"); hci_dev_set_flag(hdev, HCI_RPA_EXPIRED); + rcu_read_unlock(); return 0; } + rcu_read_unlock(); + return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_RANDOM_ADDR, 6, rpa, HCI_CMD_TIMEOUT); } @@ -2647,12 +2661,17 @@ static int hci_pause_addr_resolution(struct hci_dev *hdev) /* Cannot disable addr resolution if scanning is enabled or * when initiating an LE connection. */ + rcu_read_lock(); + if (hci_dev_test_flag(hdev, HCI_LE_SCAN) || hci_lookup_le_connect(hdev)) { + rcu_read_unlock(); bt_dev_err(hdev, "Command not allowed when scan/LE connect"); return -EPERM; } + rcu_read_unlock(); + /* Cannot disable addr resolution if advertising is enabled. */ err = hci_pause_advertising_sync(hdev); if (err) { @@ -2790,6 +2809,8 @@ static u8 hci_update_accept_list_sync(struct hci_dev *hdev) if (hci_dev_test_flag(hdev, HCI_PA_SYNC)) { struct hci_conn *conn; + rcu_read_lock(); + conn = hci_conn_hash_lookup_create_pa_sync(hdev); if (conn) { struct conn_params pa; @@ -2799,6 +2820,8 @@ static u8 hci_update_accept_list_sync(struct hci_dev *hdev) bacpy(&pa.addr, &conn->dst); pa.addr_type = conn->dst_type; + rcu_read_unlock(); + /* Clear first since there could be addresses left * behind. */ @@ -2808,6 +2831,8 @@ static u8 hci_update_accept_list_sync(struct hci_dev *hdev) err = hci_le_add_accept_list_sync(hdev, &pa, &num_entries); goto done; + } else { + rcu_read_unlock(); } } @@ -2818,10 +2843,13 @@ static u8 hci_update_accept_list_sync(struct hci_dev *hdev) * the controller. */ list_for_each_entry_safe(b, t, &hdev->le_accept_list, list) { - if (hci_conn_hash_lookup_le(hdev, &b->bdaddr, b->bdaddr_type)) + rcu_read_lock(); + + if (hci_conn_hash_lookup_le(hdev, &b->bdaddr, b->bdaddr_type)) { + rcu_read_unlock(); continue; + } - /* Pointers not dereferenced, no locks needed */ pend_conn = hci_pend_le_action_lookup(&hdev->pend_le_conns, &b->bdaddr, b->bdaddr_type); @@ -2829,6 +2857,8 @@ static u8 hci_update_accept_list_sync(struct hci_dev *hdev) &b->bdaddr, b->bdaddr_type); + rcu_read_unlock(); + /* If the device is not likely to connect or report, * remove it from the acceptlist. */ @@ -2955,6 +2985,8 @@ static int hci_le_set_ext_scan_param_sync(struct hci_dev *hdev, u8 type, if (sent) { struct hci_conn *conn; + rcu_read_lock(); + conn = hci_conn_hash_lookup_ba(hdev, PA_LINK, &sent->bdaddr); if (conn) { @@ -2979,8 +3011,12 @@ static int hci_le_set_ext_scan_param_sync(struct hci_dev *hdev, u8 type, phy++; } + rcu_read_unlock(); + if (num_phy) goto done; + } else { + rcu_read_unlock(); } } } @@ -3231,12 +3267,16 @@ int hci_update_passive_scan_sync(struct hci_dev *hdev) /* If there is at least one pending LE connection, we should * keep the background scan running. */ + bool exists; /* If controller is connecting, we should not start scanning * since some controllers are not able to scan and connect at * the same time. */ - if (hci_lookup_le_connect(hdev)) + rcu_read_lock(); + exists = hci_lookup_le_connect(hdev); + rcu_read_unlock(); + if (exists) return 0; bt_dev_dbg(hdev, "start background scanning"); @@ -3451,6 +3491,7 @@ int hci_write_fast_connectable_sync(struct hci_dev *hdev, bool enable) } static bool disconnected_accept_list_entries(struct hci_dev *hdev) + __must_hold(&hdev->lock) { struct bdaddr_list *b; @@ -3491,12 +3532,16 @@ int hci_update_scan_sync(struct hci_dev *hdev) if (hdev->scanning_paused) return 0; + hci_dev_lock(hdev); + if (hci_dev_test_flag(hdev, HCI_CONNECTABLE) || disconnected_accept_list_entries(hdev)) scan = SCAN_PAGE; else scan = SCAN_DISABLED; + hci_dev_unlock(hdev); + if (hci_dev_test_flag(hdev, HCI_DISCOVERABLE)) scan |= SCAN_INQUIRY; diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c index 410910f64c1c..79eb605be280 100644 --- a/net/bluetooth/mgmt.c +++ b/net/bluetooth/mgmt.c @@ -3082,6 +3082,8 @@ static int unpair_device_sync(struct hci_dev *hdev, void *data) struct mgmt_cp_unpair_device *cp = cmd->param; struct hci_conn *conn; + hci_dev_lock(hdev); + if (cp->addr.type == BDADDR_BREDR) conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &cp->addr.bdaddr); @@ -3089,6 +3091,11 @@ static int unpair_device_sync(struct hci_dev *hdev, void *data) conn = hci_conn_hash_lookup_le(hdev, &cp->addr.bdaddr, le_addr_type(cp->addr.type)); + if (conn) + hci_conn_get(conn); + + hci_dev_unlock(hdev); + if (!conn) return 0; @@ -3096,6 +3103,7 @@ static int unpair_device_sync(struct hci_dev *hdev, void *data) * will clean up the connection no matter the error. */ hci_abort_conn(conn, HCI_ERROR_REMOTE_USER_TERM); + hci_conn_put(conn); return 0; } @@ -3243,6 +3251,8 @@ static int disconnect_sync(struct hci_dev *hdev, void *data) struct mgmt_cp_disconnect *cp = cmd->param; struct hci_conn *conn; + hci_dev_lock(hdev); + if (cp->addr.type == BDADDR_BREDR) conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &cp->addr.bdaddr); @@ -3250,6 +3260,11 @@ static int disconnect_sync(struct hci_dev *hdev, void *data) conn = hci_conn_hash_lookup_le(hdev, &cp->addr.bdaddr, le_addr_type(cp->addr.type)); + if (conn) + hci_conn_get(conn); + + hci_dev_unlock(hdev); + if (!conn) return -ENOTCONN; @@ -3257,6 +3272,7 @@ static int disconnect_sync(struct hci_dev *hdev, void *data) * will clean up the connection no matter the error. */ hci_abort_conn(conn, HCI_ERROR_REMOTE_USER_TERM); + hci_conn_put(conn); return 0; } @@ -7309,6 +7325,9 @@ static void get_conn_info_complete(struct hci_dev *hdev, void *data, int err) rp.max_tx_power = HCI_TX_POWER_INVALID; } + if (conn) + hci_conn_put(conn); + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, MGMT_OP_GET_CONN_INFO, status, &rp, sizeof(rp)); @@ -7323,6 +7342,8 @@ static int get_conn_info_sync(struct hci_dev *hdev, void *data) int err; __le16 handle; + hci_dev_lock(hdev); + /* Make sure we are still connected */ if (cp->addr.type == BDADDR_BREDR) conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, @@ -7330,12 +7351,16 @@ static int get_conn_info_sync(struct hci_dev *hdev, void *data) else conn = hci_conn_hash_lookup_ba(hdev, LE_LINK, &cp->addr.bdaddr); - if (!conn || conn->state != BT_CONNECTED) + if (!conn || conn->state != BT_CONNECTED) { + hci_dev_unlock(hdev); return MGMT_STATUS_NOT_CONNECTED; + } - cmd->user_data = conn; + cmd->user_data = hci_conn_get(conn); handle = cpu_to_le16(conn->handle); + hci_dev_unlock(hdev); + /* Refresh RSSI each time */ err = hci_read_rssi_sync(hdev, handle); @@ -7469,6 +7494,9 @@ static void get_clock_info_complete(struct hci_dev *hdev, void *data, int err) } complete: + if (conn) + hci_conn_put(conn); + mgmt_cmd_complete(cmd->sk, cmd->hdev->id, cmd->opcode, status, &rp, sizeof(rp)); @@ -7485,15 +7513,21 @@ static int get_clock_info_sync(struct hci_dev *hdev, void *data) memset(&hci_cp, 0, sizeof(hci_cp)); hci_read_clock_sync(hdev, &hci_cp); + hci_dev_lock(hdev); + /* Make sure connection still exists */ conn = hci_conn_hash_lookup_ba(hdev, ACL_LINK, &cp->addr.bdaddr); - if (!conn || conn->state != BT_CONNECTED) + if (!conn || conn->state != BT_CONNECTED) { + hci_dev_unlock(hdev); return MGMT_STATUS_NOT_CONNECTED; + } - cmd->user_data = conn; + cmd->user_data = hci_conn_get(conn); hci_cp.handle = cpu_to_le16(conn->handle); hci_cp.which = 0x01; /* Piconet clock */ + hci_dev_unlock(hdev); + return hci_read_clock_sync(hdev, &hci_cp); } @@ -7924,14 +7958,36 @@ static int remove_device(struct sock *sk, struct hci_dev *hdev, static int conn_update_sync(struct hci_dev *hdev, void *data) { - struct hci_conn_params *params = data; - struct hci_conn *conn; + struct hci_conn *conn = data; + struct hci_conn_params *params; + struct hci_conn_params local = {}; - conn = hci_conn_hash_lookup_le(hdev, ¶ms->addr, params->addr_type); - if (!conn) - return -ECANCELED; + hci_dev_lock(hdev); + + if (!hci_conn_valid(hdev, conn) || conn->role != HCI_ROLE_MASTER) + goto cancel; - return hci_le_conn_update_sync(hdev, conn, params); + params = hci_conn_params_lookup(hdev, &conn->dst, conn->dst_type); + if (!params) + goto cancel; + + local.conn_min_interval = params->conn_min_interval; + local.conn_max_interval = params->conn_max_interval; + local.conn_latency = params->conn_latency; + local.supervision_timeout = params->supervision_timeout; + + hci_dev_unlock(hdev); + + return hci_le_conn_update_sync(hdev, conn, &local); + +cancel: + hci_dev_unlock(hdev); + return -ECANCELED; +} + +static void conn_update_sync_destroy(struct hci_dev *hdev, void *data, int err) +{ + hci_conn_put(data); } static int load_conn_param(struct sock *sk, struct hci_dev *hdev, void *data, @@ -8041,9 +8097,13 @@ static int load_conn_param(struct sock *sk, struct hci_dev *hdev, void *data, (conn->le_conn_min_interval != min || conn->le_conn_max_interval != max || conn->le_conn_latency != latency || - conn->le_supv_timeout != timeout)) - hci_cmd_sync_queue(hdev, conn_update_sync, - hci_param, NULL); + conn->le_supv_timeout != timeout)) { + hci_conn_get(conn); + if (hci_cmd_sync_queue(hdev, conn_update_sync, + conn, + conn_update_sync_destroy) < 0) + hci_conn_put(conn); + } } } diff --git a/net/bluetooth/rfcomm/core.c b/net/bluetooth/rfcomm/core.c index dfade1933fa7..d2c869e8286b 100644 --- a/net/bluetooth/rfcomm/core.c +++ b/net/bluetooth/rfcomm/core.c @@ -1031,6 +1031,23 @@ int rfcomm_send_rpn(struct rfcomm_session *s, int cr, u8 dlci, return rfcomm_send_frame(s, buf, ptr - buf); } +int rfcomm_dlc_send_rpn(struct rfcomm_dlc *d, u8 bit_rate, u8 data_bits, + u8 stop_bits, u8 parity, u8 flow_ctrl_settings, + u8 xon_char, u8 xoff_char, u16 param_mask) +{ + int err = -ENOTCONN; + + rfcomm_lock(); + if (d->session) + err = rfcomm_send_rpn(d->session, 1, d->dlci, bit_rate, + data_bits, stop_bits, parity, + flow_ctrl_settings, xon_char, xoff_char, + param_mask); + rfcomm_unlock(); + + return err; +} + static int rfcomm_send_rls(struct rfcomm_session *s, int cr, u8 dlci, u8 status) { struct rfcomm_hdr *hdr; diff --git a/net/bluetooth/rfcomm/tty.c b/net/bluetooth/rfcomm/tty.c index b783526ab588..63f208abbd51 100644 --- a/net/bluetooth/rfcomm/tty.c +++ b/net/bluetooth/rfcomm/tty.c @@ -861,7 +861,7 @@ static void rfcomm_tty_set_termios(struct tty_struct *tty, BT_DBG("tty %p termios %p", tty, old); - if (!dev || !dev->dlc || !dev->dlc->session) + if (!dev || !dev->dlc) return; /* Handle turning off CRTSCTS */ @@ -982,9 +982,8 @@ static void rfcomm_tty_set_termios(struct tty_struct *tty, } if (changes) - rfcomm_send_rpn(dev->dlc->session, 1, dev->dlc->dlci, baud, - data_bits, stop_bits, parity, - RFCOMM_RPN_FLOW_NONE, x_on, x_off, changes); + rfcomm_dlc_send_rpn(dev->dlc, baud, data_bits, stop_bits, parity, + RFCOMM_RPN_FLOW_NONE, x_on, x_off, changes); } static void rfcomm_tty_throttle(struct tty_struct *tty) diff --git a/net/bridge/br_netlink_tunnel.c b/net/bridge/br_netlink_tunnel.c index 71a12da30004..a713668ea34f 100644 --- a/net/bridge/br_netlink_tunnel.c +++ b/net/bridge/br_netlink_tunnel.c @@ -271,7 +271,8 @@ static void __vlan_tunnel_handle_range(const struct net_bridge_port *p, if (!*v_start) goto out_init; - if (v && curr_change && br_vlan_can_enter_range(v, *v_end)) { + if (v && curr_change && + br_vlan_can_enter_range(v, *v_end, br_get_pvid(vg))) { *v_end = v; return; } diff --git a/net/bridge/br_private.h b/net/bridge/br_private.h index b9b2981c4841..2b451f3f72a6 100644 --- a/net/bridge/br_private.h +++ b/net/bridge/br_private.h @@ -1613,7 +1613,8 @@ void br_vlan_notify(const struct net_bridge *br, u16 vid, u16 vid_range, int cmd); bool br_vlan_can_enter_range(const struct net_bridge_vlan *v_curr, - const struct net_bridge_vlan *range_end); + const struct net_bridge_vlan *range_end, + u16 pvid); void br_vlan_fill_forward_path_pvid(struct net_bridge *br, struct net_device_path_ctx *ctx, @@ -1860,7 +1861,8 @@ static inline void br_vlan_notify(const struct net_bridge *br, } static inline bool br_vlan_can_enter_range(const struct net_bridge_vlan *v_curr, - const struct net_bridge_vlan *range_end) + const struct net_bridge_vlan *range_end, + u16 pvid) { return true; } diff --git a/net/bridge/br_vlan.c b/net/bridge/br_vlan.c index ce72b837ff8e..a4ba725ad1df 100644 --- a/net/bridge/br_vlan.c +++ b/net/bridge/br_vlan.c @@ -1977,9 +1977,11 @@ void br_vlan_notify(const struct net_bridge *br, /* check if v_curr can enter a range ending in range_end */ bool br_vlan_can_enter_range(const struct net_bridge_vlan *v_curr, - const struct net_bridge_vlan *range_end) + const struct net_bridge_vlan *range_end, + u16 pvid) { - return v_curr->vid - range_end->vid == 1 && + return v_curr->vid != pvid && range_end->vid != pvid && + v_curr->vid - range_end->vid == 1 && range_end->flags == v_curr->flags && br_vlan_opts_eq_range(v_curr, range_end); } @@ -2061,8 +2063,8 @@ static int br_vlan_dump_dev(const struct net_device *dev, idx += range_end->vid - range_start->vid + 1; range_start = v; - } else if (dump_stats || v->vid == pvid || - !br_vlan_can_enter_range(v, range_end)) { + } else if (dump_stats || + !br_vlan_can_enter_range(v, range_end, pvid)) { u16 vlan_flags = br_vlan_flags(range_start, pvid); if (!br_vlan_fill_vids(skb, range_start->vid, diff --git a/net/bridge/br_vlan_options.c b/net/bridge/br_vlan_options.c index 8fa89b04ee94..4a736e005bad 100644 --- a/net/bridge/br_vlan_options.c +++ b/net/bridge/br_vlan_options.c @@ -310,8 +310,7 @@ int br_vlan_process_options(const struct net_bridge *br, continue; } - if (v->vid == pvid || - !br_vlan_can_enter_range(v, curr_end)) { + if (!br_vlan_can_enter_range(v, curr_end, pvid)) { br_vlan_notify(br, p, curr_start->vid, curr_end->vid, RTM_NEWVLAN); curr_start = v; diff --git a/net/bridge/netfilter/nft_meta_bridge.c b/net/bridge/netfilter/nft_meta_bridge.c index a3d0c8a82d86..e4c9aa1f64e2 100644 --- a/net/bridge/netfilter/nft_meta_bridge.c +++ b/net/bridge/netfilter/nft_meta_bridge.c @@ -134,7 +134,6 @@ static const struct nft_expr_ops nft_meta_bridge_get_ops = { .init = nft_meta_bridge_get_init, .validate = nft_meta_bridge_get_validate, .dump = nft_meta_get_dump, - .reduce = nft_meta_get_reduce, }; static void nft_meta_bridge_set_eval(const struct nft_expr *expr, @@ -181,24 +180,6 @@ static int nft_meta_bridge_set_init(const struct nft_ctx *ctx, return 0; } -static bool nft_meta_bridge_set_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - int i; - - for (i = 0; i < NFT_REG32_NUM; i++) { - if (!track->regs[i].selector) - continue; - - if (track->regs[i].selector->ops != &nft_meta_bridge_get_ops) - continue; - - __nft_reg_track_cancel(track, i); - } - - return false; -} - static int nft_meta_bridge_set_validate(const struct nft_ctx *ctx, const struct nft_expr *expr) { @@ -223,7 +204,6 @@ static const struct nft_expr_ops nft_meta_bridge_set_ops = { .init = nft_meta_bridge_set_init, .destroy = nft_meta_set_destroy, .dump = nft_meta_set_dump, - .reduce = nft_meta_bridge_set_reduce, .validate = nft_meta_bridge_set_validate, }; diff --git a/net/bridge/netfilter/nft_reject_bridge.c b/net/bridge/netfilter/nft_reject_bridge.c index 1cb5c16e97b7..cd2b04236a99 100644 --- a/net/bridge/netfilter/nft_reject_bridge.c +++ b/net/bridge/netfilter/nft_reject_bridge.c @@ -184,7 +184,6 @@ static const struct nft_expr_ops nft_reject_bridge_ops = { .init = nft_reject_init, .dump = nft_reject_dump, .validate = nft_reject_bridge_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_reject_bridge_type __read_mostly = { diff --git a/net/can/bcm.c b/net/can/bcm.c index a40519638a26..76bc7f727425 100644 --- a/net/can/bcm.c +++ b/net/can/bcm.c @@ -111,11 +111,12 @@ struct bcm_op { int ifindex; canid_t can_id; u32 flags; - unsigned long frames_abs, frames_filtered; + atomic_long_t frames_abs, frames_filtered; struct bcm_timeval ival1, ival2; struct hrtimer timer, thrtimer; ktime_t rx_stamp, kt_ival1, kt_ival2, kt_lastmsg; int rx_ifindex; + int if_detected; /* first received ifindex in ANYDEV rx_op mode */ int cfsiz; u32 count; u32 nframes; @@ -127,7 +128,8 @@ struct bcm_op { struct canfd_frame last_sframe; struct sock *sk; struct net_device *rx_reg_dev; - spinlock_t bcm_tx_lock; /* protect currframe/count in runtime updates */ + spinlock_t bcm_tx_lock; /* protect tx data and timer updates */ + spinlock_t bcm_rx_update_lock; /* protect filter/timer data updates */ }; struct bcm_sock { @@ -227,10 +229,13 @@ static int bcm_proc_show(struct seq_file *m, void *v) list_for_each_entry_rcu(op, &bo->rx_ops, list) { - unsigned long reduction; + long reduction, frames_filtered, frames_abs; + + frames_filtered = atomic_long_read(&op->frames_filtered); + frames_abs = atomic_long_read(&op->frames_abs); /* print only active entries & prevent division by zero */ - if (!op->frames_abs) + if (!frames_abs) continue; seq_printf(m, "rx_op: %03X %-5s ", op->can_id, @@ -252,9 +257,9 @@ static int bcm_proc_show(struct seq_file *m, void *v) (long long)ktime_to_us(op->kt_ival2)); seq_printf(m, "# recv %ld (%ld) => reduction: ", - op->frames_filtered, op->frames_abs); + frames_filtered, frames_abs); - reduction = 100 - (op->frames_filtered * 100) / op->frames_abs; + reduction = 100 - (frames_filtered * 100) / frames_abs; seq_printf(m, "%s%ld%%\n", (reduction == 100) ? "near " : "", reduction); @@ -278,7 +283,8 @@ static int bcm_proc_show(struct seq_file *m, void *v) seq_printf(m, "t2=%lld ", (long long)ktime_to_us(op->kt_ival2)); - seq_printf(m, "# sent %ld\n", op->frames_abs); + seq_printf(m, "# sent %ld\n", + atomic_long_read(&op->frames_abs)); } seq_putc(m, '\n'); @@ -288,25 +294,49 @@ static int bcm_proc_show(struct seq_file *m, void *v) } #endif /* CONFIG_PROC_FS */ +static void bcm_update_rx_stats(struct bcm_op *op) +{ + /* prevent overflow of the reduction% calculation in bcm_proc_show() */ + if (atomic_long_inc_return(&op->frames_abs) > LONG_MAX / 100) { + atomic_long_set(&op->frames_filtered, 0); + atomic_long_set(&op->frames_abs, 0); + } +} + +static void bcm_update_tx_stats(struct bcm_op *op) +{ + /* tx_op has no reduction% calculation - use the full range and + * just keep the displayed counter non-negative on overflow + */ + if (atomic_long_inc_return(&op->frames_abs) == LONG_MAX) + atomic_long_set(&op->frames_abs, 0); +} + /* * bcm_can_tx - send the (next) CAN frame to the appropriate CAN interface * of the given bcm tx op */ -static void bcm_can_tx(struct bcm_op *op) +static void bcm_can_tx(struct bcm_op *op, struct canfd_frame *cf) { struct sk_buff *skb; struct net_device *dev; - struct canfd_frame *cf; + struct canfd_frame cframe; + bool cyclic = !cf; + unsigned int idx = 0; int err; /* no target device? => exit */ if (!op->ifindex) return; - /* read currframe under lock protection */ - spin_lock_bh(&op->bcm_tx_lock); - cf = op->frames + op->cfsiz * op->currframe; - spin_unlock_bh(&op->bcm_tx_lock); + if (cyclic) { + /* read currframe under lock protection */ + spin_lock_bh(&op->bcm_tx_lock); + idx = op->currframe; + memcpy(&cframe, op->frames + op->cfsiz * idx, op->cfsiz); + cf = &cframe; + spin_unlock_bh(&op->bcm_tx_lock); + } dev = dev_get_by_index(sock_net(op->sk), op->ifindex); if (!dev) { @@ -333,16 +363,22 @@ static void bcm_can_tx(struct bcm_op *op) spin_lock_bh(&op->bcm_tx_lock); if (!err) - op->frames_abs++; + bcm_update_tx_stats(op); - op->currframe++; + /* only advance the cyclic sequence if nothing reset currframe while + * we were sending - a concurrent TX_RESET_MULTI_IDX means this + * frame's bookkeeping belongs to a sequence that no longer exists + */ + if (!cyclic || op->currframe == idx) { + op->currframe++; - /* reached last frame? */ - if (op->currframe >= op->nframes) - op->currframe = 0; + /* reached last frame? */ + if (op->currframe >= op->nframes) + op->currframe = 0; - if (op->count > 0) - op->count--; + if (op->count > 0) + op->count--; + } spin_unlock_bh(&op->bcm_tx_lock); out: @@ -432,12 +468,18 @@ static bool bcm_tx_set_expiry(struct bcm_op *op, struct hrtimer *hrt) { ktime_t ival; + spin_lock_bh(&op->bcm_tx_lock); + if (op->kt_ival1 && op->count) ival = op->kt_ival1; - else if (op->kt_ival2) + else if (op->kt_ival2) { ival = op->kt_ival2; - else + } else { + spin_unlock_bh(&op->bcm_tx_lock); return false; + } + + spin_unlock_bh(&op->bcm_tx_lock); hrtimer_set_expires(hrt, ktime_add(ktime_get(), ival)); return true; @@ -454,26 +496,48 @@ static enum hrtimer_restart bcm_tx_timeout_handler(struct hrtimer *hrtimer) { struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer); struct bcm_msg_head msg_head; + bool tx_ival1, tx_ival2; + + /* snapshot kt_ival1/kt_ival2/count under lock to avoid torn + * ktime_t reads racing with concurrent bcm_tx_setup() updates + */ + spin_lock_bh(&op->bcm_tx_lock); + tx_ival1 = op->kt_ival1 && (op->count > 0); + tx_ival2 = !!op->kt_ival2; + spin_unlock_bh(&op->bcm_tx_lock); + + if (tx_ival1) { + u32 flags, count; + struct bcm_timeval ival1, ival2; - if (op->kt_ival1 && (op->count > 0)) { - bcm_can_tx(op); - if (!op->count && (op->flags & TX_COUNTEVT)) { + bcm_can_tx(op, NULL); + /* snapshot variables under lock to avoid torn reads racing + * with concurrent bcm_tx_setup() updates + */ + spin_lock_bh(&op->bcm_tx_lock); + flags = op->flags; + count = op->count; + ival1 = op->ival1; + ival2 = op->ival2; + spin_unlock_bh(&op->bcm_tx_lock); + + if (!count && (flags & TX_COUNTEVT)) { /* create notification to user */ memset(&msg_head, 0, sizeof(msg_head)); msg_head.opcode = TX_EXPIRED; - msg_head.flags = op->flags; - msg_head.count = op->count; - msg_head.ival1 = op->ival1; - msg_head.ival2 = op->ival2; + msg_head.flags = flags; + msg_head.count = count; + msg_head.ival1 = ival1; + msg_head.ival2 = ival2; msg_head.can_id = op->can_id; msg_head.nframes = 0; bcm_send_to_user(op, &msg_head, NULL, 0); } - } else if (op->kt_ival2) { - bcm_can_tx(op); + } else if (tx_ival2) { + bcm_can_tx(op, NULL); } return bcm_tx_set_expiry(op, &op->timer) ? @@ -487,12 +551,9 @@ static void bcm_rx_changed(struct bcm_op *op, struct canfd_frame *data) { struct bcm_msg_head head; - /* update statistics */ - op->frames_filtered++; - - /* prevent statistics overflow */ - if (op->frames_filtered > ULONG_MAX/100) - op->frames_filtered = op->frames_abs = 0; + /* update statistics (frames_filtered <= frames_abs) */ + if (atomic_long_read(&op->frames_abs)) + atomic_long_inc(&op->frames_filtered); /* this element is not throttled anymore */ data->flags &= ~RX_THR; @@ -617,6 +678,8 @@ static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer) struct bcm_op *op = container_of(hrtimer, struct bcm_op, timer); struct bcm_msg_head msg_head; + spin_lock_bh(&op->bcm_rx_update_lock); + /* if user wants to be informed, when cyclic CAN-Messages come back */ if ((op->flags & RX_ANNOUNCE_RESUME) && op->last_frames) { /* clear received CAN frames to indicate 'nothing received' */ @@ -633,6 +696,8 @@ static enum hrtimer_restart bcm_rx_timeout_handler(struct hrtimer *hrtimer) msg_head.can_id = op->can_id; msg_head.nframes = 0; + spin_unlock_bh(&op->bcm_rx_update_lock); + bcm_send_to_user(op, &msg_head, NULL, 0); return HRTIMER_NORESTART; @@ -681,15 +746,26 @@ static int bcm_rx_thr_flush(struct bcm_op *op) static enum hrtimer_restart bcm_rx_thr_handler(struct hrtimer *hrtimer) { struct bcm_op *op = container_of(hrtimer, struct bcm_op, thrtimer); + enum hrtimer_restart ret; + + spin_lock_bh(&op->bcm_rx_update_lock); - if (bcm_rx_thr_flush(op)) { + /* kt_ival2 may have been concurrently cleared by bcm_rx_setup() + * before it cancels this timer - never forward with a zero + * interval in that case. + */ + if (bcm_rx_thr_flush(op) && op->kt_ival2) { hrtimer_forward_now(hrtimer, op->kt_ival2); - return HRTIMER_RESTART; + ret = HRTIMER_RESTART; } else { /* rearm throttle handling */ op->kt_lastmsg = 0; - return HRTIMER_NORESTART; + ret = HRTIMER_NORESTART; } + + spin_unlock_bh(&op->bcm_rx_update_lock); + + return ret; } /* @@ -699,8 +775,10 @@ static void bcm_rx_handler(struct sk_buff *skb, void *data) { struct bcm_op *op = (struct bcm_op *)data; const struct canfd_frame *rxframe = (struct canfd_frame *)skb->data; + struct canfd_frame rtrframe; unsigned int i; unsigned char traffic_flags; + bool rtr_frame; if (op->can_id != rxframe->can_id) return; @@ -714,22 +792,59 @@ static void bcm_rx_handler(struct sk_buff *skb, void *data) return; } + /* An ANYDEV op with an active RX timeout and/or throttle timer + * tracks a single source interface: claim the first interface that + * delivers a matching frame and reject frames from any other one, + * before hrtimer_cancel() below can touch op->timer - this avoids + * racing bcm_rx_timeout_handler() across concurrent interfaces. + * RX_RTR_FRAME ops are excluded, as kt_ival1/kt_ival2 may briefly + * hold a stale value from an earlier non-RTR configuration. + */ + if (!op->ifindex) { + spin_lock_bh(&op->bcm_rx_update_lock); + + if (!(op->flags & RX_RTR_FRAME) && + (op->kt_ival1 || op->kt_ival2)) { + /* don't claim to vanishing interface */ + if (!op->if_detected && + skb->dev->reg_state == NETREG_REGISTERED) + op->if_detected = skb->dev->ifindex; + + if (op->if_detected != skb->dev->ifindex) { + spin_unlock_bh(&op->bcm_rx_update_lock); + return; + } + } + + spin_unlock_bh(&op->bcm_rx_update_lock); + } + /* disable timeout */ hrtimer_cancel(&op->timer); - /* save rx timestamp */ - op->rx_stamp = skb->tstamp; - /* save originator for recvfrom() */ - op->rx_ifindex = skb->dev->ifindex; - /* update statistics */ - op->frames_abs++; + /* op->flags/op->frames may be updated concurrently by bcm_rx_setup() */ + spin_lock_bh(&op->bcm_rx_update_lock); + + rtr_frame = op->flags & RX_RTR_FRAME; + if (rtr_frame) { + bcm_update_rx_stats(op); + /* snapshot RTR content under lock */ + memcpy(&rtrframe, op->frames, op->cfsiz); + spin_unlock_bh(&op->bcm_rx_update_lock); - if (op->flags & RX_RTR_FRAME) { /* send reply for RTR-request (placed in op->frames[0]) */ - bcm_can_tx(op); + bcm_can_tx(op, &rtrframe); return; } + /* update statistics in the same critical section as bcm_rx_changed() + * below: frames_filtered must never be checked/incremented against a + * frames_abs snapshot from a concurrent bcm_rx_handler() call on + * another CPU for the same (wildcard) op, or frames_filtered can end + * up larger than frames_abs. + */ + bcm_update_rx_stats(op); + /* compute flags to distinguish between own/local/remote CAN traffic */ traffic_flags = 0; if (skb->sk) { @@ -738,6 +853,13 @@ static void bcm_rx_handler(struct sk_buff *skb, void *data) traffic_flags |= RX_OWN; } + /* save rx timestamp and originator for recvfrom() under lock: an + * ANYDEV op without an active timer can still run concurrently on + * different CPUs, so content and meta data must be bundled here. + */ + op->rx_stamp = skb->tstamp; + op->rx_ifindex = skb->dev->ifindex; + if (op->flags & RX_FILTER_ID) { /* the easiest case */ bcm_rx_update_and_send(op, op->last_frames, rxframe, @@ -773,6 +895,8 @@ static void bcm_rx_handler(struct sk_buff *skb, void *data) rx_starttimer: bcm_rx_starttimer(op); + + spin_unlock_bh(&op->bcm_rx_update_lock); } /* @@ -838,6 +962,7 @@ static void bcm_rx_unreg(struct net_device *dev, struct bcm_op *op) /* mark as removed subscription */ op->rx_reg_dev = NULL; + dev_put(dev); } else printk(KERN_ERR "can-bcm: bcm_rx_unreg: registered device " "mismatch %p %p\n", op->rx_reg_dev, dev); @@ -868,17 +993,14 @@ static int bcm_delete_rx_op(struct list_head *ops, struct bcm_msg_head *mh, * Only remove subscriptions that had not * been removed due to NETDEV_UNREGISTER * in bcm_notifier() + * + * op->rx_reg_dev is a tracked reference taken + * when the subscription was registered, so it + * stays valid here even if a concurrent + * NETDEV_UNREGISTER already unlisted the dev. */ - if (op->rx_reg_dev) { - struct net_device *dev; - - dev = dev_get_by_index(sock_net(op->sk), - op->ifindex); - if (dev) { - bcm_rx_unreg(dev, op); - dev_put(dev); - } - } + if (op->rx_reg_dev) + bcm_rx_unreg(op->rx_reg_dev, op); } else can_rx_unregister(sock_net(op->sk), NULL, op->can_id, @@ -965,6 +1087,8 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, /* check the given can_id */ op = bcm_find_op(&bo->tx_ops, msg_head, ifindex); if (op) { + void *new_frames; + /* update existing BCM operation */ /* @@ -975,11 +1099,23 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, if (msg_head->nframes > op->nframes) return -E2BIG; - /* update CAN frames content */ + /* get new CAN frames content into a staging buffer before + * locking: validate and normalize the frames there so that + * bcm_can_tx() / bcm_tx_timeout_handler() never observe a + * partially updated or unvalidated frame in op->frames + */ + new_frames = kmalloc(msg_head->nframes * op->cfsiz, GFP_KERNEL); + if (!new_frames) + return -ENOMEM; + for (i = 0; i < msg_head->nframes; i++) { - cf = op->frames + op->cfsiz * i; + cf = new_frames + op->cfsiz * i; err = memcpy_from_msg((u8 *)cf, msg, op->cfsiz); + if (err < 0) { + kfree(new_frames); + return err; + } if (op->flags & CAN_FD_FRAME) { if (cf->len > 64) @@ -989,36 +1125,38 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, err = -EINVAL; } - if (err < 0) + if (err < 0) { + kfree(new_frames); return err; + } if (msg_head->flags & TX_CP_CAN_ID) { /* copy can_id into frame */ cf->can_id = msg_head->can_id; } } + + spin_lock_bh(&op->bcm_tx_lock); + + /* update CAN frames content */ + memcpy(op->frames, new_frames, msg_head->nframes * op->cfsiz); + op->flags = msg_head->flags; - /* only lock for unlikely count/nframes/currframe changes */ if (op->nframes != msg_head->nframes || - op->flags & TX_RESET_MULTI_IDX || - op->flags & SETTIMER) { - - spin_lock_bh(&op->bcm_tx_lock); + op->flags & TX_RESET_MULTI_IDX) { + /* potentially update changed nframes */ + op->nframes = msg_head->nframes; + /* restart multiple frame transmission */ + op->currframe = 0; + } - if (op->nframes != msg_head->nframes || - op->flags & TX_RESET_MULTI_IDX) { - /* potentially update changed nframes */ - op->nframes = msg_head->nframes; - /* restart multiple frame transmission */ - op->currframe = 0; - } + if (op->flags & SETTIMER) + op->count = msg_head->count; - if (op->flags & SETTIMER) - op->count = msg_head->count; + spin_unlock_bh(&op->bcm_tx_lock); - spin_unlock_bh(&op->bcm_tx_lock); - } + kfree(new_frames); } else { /* insert new BCM operation for the given can_id */ @@ -1094,10 +1232,12 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, if (op->flags & SETTIMER) { /* set timer values */ + spin_lock_bh(&op->bcm_tx_lock); op->ival1 = msg_head->ival1; op->ival2 = msg_head->ival2; op->kt_ival1 = bcm_timeval_to_ktime(msg_head->ival1); op->kt_ival2 = bcm_timeval_to_ktime(msg_head->ival2); + spin_unlock_bh(&op->bcm_tx_lock); /* disable an active timer due to zero values? */ if (!op->kt_ival1 && !op->kt_ival2) @@ -1115,7 +1255,7 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, list_add_rcu(&op->list, &bo->tx_ops); if (op->flags & TX_ANNOUNCE) - bcm_can_tx(op); + bcm_can_tx(op, NULL); if (op->flags & STARTTIMER) bcm_tx_start_timer(op); @@ -1129,6 +1269,39 @@ static int bcm_tx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, return err; } +static int bcm_rx_setup_rtr_check(struct bcm_msg_head *msg_head, + struct bcm_op *op, void *new_frames) +{ + struct canfd_frame *frame0 = new_frames; + + if (!(msg_head->flags & RX_RTR_FRAME)) + return 0; + + /* this frame is sent out as-is by bcm_can_tx() whenever a matching + * remote request is received, so validate its length the same way + * bcm_tx_setup() validates TX_SETUP frames before installing it + */ + if (msg_head->flags & CAN_FD_FRAME) { + if (frame0->len > 64) + return -EINVAL; + } else { + if (frame0->len > 8) + return -EINVAL; + } + + /* funny feature in RX(!)_SETUP only for RTR-mode: + * copy can_id into frame BUT without RTR-flag to + * prevent a full-load-loopback-test ... ;-] + * normalize this on the staged buffer, before it is + * ever installed into op->frames. + */ + if ((msg_head->flags & TX_CP_CAN_ID) || + frame0->can_id == op->can_id) + frame0->can_id = op->can_id & ~CAN_RTR_FLAG; + + return 0; +} + /* * bcm_rx_setup - create or update a bcm rx op (for bcm_sendmsg) */ @@ -1138,6 +1311,7 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, struct bcm_sock *bo = bcm_sk(sk); struct bcm_op *op; int do_rx_register; + int new_op = 0; int err = 0; if ((msg_head->flags & RX_FILTER_ID) || (!(msg_head->nframes))) { @@ -1163,6 +1337,8 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, /* check the given can_id */ op = bcm_find_op(&bo->rx_ops, msg_head, ifindex); if (op) { + void *new_frames = NULL; + /* update existing BCM operation */ /* @@ -1174,21 +1350,62 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, return -E2BIG; if (msg_head->nframes) { - /* update CAN frames content */ - err = memcpy_from_msg(op->frames, msg, + /* get new CAN frames content before locking */ + new_frames = kmalloc(msg_head->nframes * op->cfsiz, + GFP_KERNEL); + if (!new_frames) + return -ENOMEM; + + err = memcpy_from_msg(new_frames, msg, msg_head->nframes * op->cfsiz); - if (err < 0) + if (err < 0) { + kfree(new_frames); return err; + } - /* clear last_frames to indicate 'nothing received' */ - memset(op->last_frames, 0, msg_head->nframes * op->cfsiz); + err = bcm_rx_setup_rtr_check(msg_head, op, new_frames); + if (err < 0) { + kfree(new_frames); + return err; + } } + spin_lock_bh(&op->bcm_rx_update_lock); op->nframes = msg_head->nframes; op->flags = msg_head->flags; - /* Only an update -> do not call can_rx_register() */ - do_rx_register = 0; + if (msg_head->nframes) { + /* update CAN frames content */ + memcpy(op->frames, new_frames, + msg_head->nframes * op->cfsiz); + + /* clear last_frames to indicate 'nothing received' */ + memset(op->last_frames, 0, + msg_head->nframes * op->cfsiz); + } + + if (msg_head->flags & SETTIMER) { + op->ival1 = msg_head->ival1; + op->ival2 = msg_head->ival2; + op->kt_ival1 = bcm_timeval_to_ktime(msg_head->ival1); + op->kt_ival2 = bcm_timeval_to_ktime(msg_head->ival2); + op->kt_lastmsg = 0; + op->if_detected = 0; /* reclaim ifindex in ANYDEV mode */ + } + spin_unlock_bh(&op->bcm_rx_update_lock); + + /* free temporary frames / kfree(NULL) is safe */ + kfree(new_frames); + + /* Don't register a new CAN filter for the rx_op update unless + * a concurrent NETDEV_UNREGISTER notifier already tore down + * the previous registration. In this case the receiver needs + * to be re-registered here so that this update doesn't + * silently stop delivering frames for the given ifindex. + * Ops with ifindex = 0 (all CAN interfaces) never carry a + * tracked rx_reg_dev and stay registered as-is. + */ + do_rx_register = (ifindex && !op->rx_reg_dev) ? 1 : 0; } else { /* insert new BCM operation for the given can_id */ @@ -1197,6 +1414,7 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, return -ENOMEM; spin_lock_init(&op->bcm_tx_lock); + spin_lock_init(&op->bcm_rx_update_lock); op->can_id = msg_head->can_id; op->nframes = msg_head->nframes; op->cfsiz = CFSIZ(msg_head->flags); @@ -1230,14 +1448,12 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, if (msg_head->nframes) { err = memcpy_from_msg(op->frames, msg, msg_head->nframes * op->cfsiz); - if (err < 0) { - if (op->frames != &op->sframe) - kfree(op->frames); - if (op->last_frames != &op->last_sframe) - kfree(op->last_frames); - kfree(op); - return err; - } + if (err < 0) + goto free_op; + + err = bcm_rx_setup_rtr_check(msg_head, op, op->frames); + if (err < 0) + goto free_op; } /* bcm_can_tx / bcm_tx_timeout_handler needs this */ @@ -1256,35 +1472,29 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, /* call can_rx_register() */ do_rx_register = 1; + new_op = 1; } /* if ((op = bcm_find_op(&bo->rx_ops, msg_head->can_id, ifindex))) */ /* check flags */ if (op->flags & RX_RTR_FRAME) { - struct canfd_frame *frame0 = op->frames; - /* no timers in RTR-mode */ hrtimer_cancel(&op->thrtimer); hrtimer_cancel(&op->timer); - - /* - * funny feature in RX(!)_SETUP only for RTR-mode: - * copy can_id into frame BUT without RTR-flag to - * prevent a full-load-loopback-test ... ;-] - */ - if ((op->flags & TX_CP_CAN_ID) || - (frame0->can_id == op->can_id)) - frame0->can_id = op->can_id & ~CAN_RTR_FLAG; - } else { if (op->flags & SETTIMER) { - /* set timer value */ - op->ival1 = msg_head->ival1; - op->ival2 = msg_head->ival2; - op->kt_ival1 = bcm_timeval_to_ktime(msg_head->ival1); - op->kt_ival2 = bcm_timeval_to_ktime(msg_head->ival2); + /* set timers (locked) for newly created op */ + if (new_op) { + spin_lock_bh(&op->bcm_rx_update_lock); + op->ival1 = msg_head->ival1; + op->ival2 = msg_head->ival2; + op->kt_ival1 = bcm_timeval_to_ktime(msg_head->ival1); + op->kt_ival2 = bcm_timeval_to_ktime(msg_head->ival2); + op->kt_lastmsg = 0; + spin_unlock_bh(&op->bcm_rx_update_lock); + } /* disable an active timer due to zero value? */ if (!op->kt_ival1) @@ -1294,9 +1504,11 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, * In any case cancel the throttle timer, flush * potentially blocked msgs and reset throttle handling */ - op->kt_lastmsg = 0; hrtimer_cancel(&op->thrtimer); + + spin_lock_bh(&op->bcm_rx_update_lock); bcm_rx_thr_flush(op); + spin_unlock_bh(&op->bcm_rx_update_lock); } if ((op->flags & STARTTIMER) && op->kt_ival1) @@ -1304,7 +1516,10 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, HRTIMER_MODE_REL_SOFT); } - /* now we can register for can_ids, if we added a new bcm_op */ + /* now we can register for can_ids, if we added a new bcm_op + * or need to re-register after a NETDEV_UNREGISTER tore down + * the previous registration of an existing op + */ if (do_rx_register) { if (ifindex) { struct net_device *dev; @@ -1317,7 +1532,15 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, bcm_rx_handler, op, "bcm", sk); - op->rx_reg_dev = dev; + /* keep a reference so that a later + * unregister can safely reach the device even + * if a concurrent NETDEV_UNREGISTER has + * already unlisted it by ifindex + */ + if (!err) { + op->rx_reg_dev = dev; + dev_hold(dev); + } dev_put(dev); } else { /* the requested device is gone - do not @@ -1326,21 +1549,43 @@ static int bcm_rx_setup(struct bcm_msg_head *msg_head, struct msghdr *msg, err = -ENODEV; } - } else + } else { err = can_rx_register(sock_net(sk), NULL, op->can_id, REGMASK(op->can_id), bcm_rx_handler, op, "bcm", sk); + } + if (err) { - /* this bcm rx op is broken -> remove it */ - bcm_remove_op(op); + /* newly created bcm rx op is broken -> remove it */ + if (new_op) { + bcm_remove_op(op); + return err; + } + + /* an existing op just stays unregistered. + * Cancel op->timer and (defensively) op->thrtimer. + * Other settings can't be reached until the next + * successful RX_SETUP. + */ + hrtimer_cancel(&op->timer); + hrtimer_cancel(&op->thrtimer); return err; } - /* add this bcm_op to the list of the rx_ops */ - list_add_rcu(&op->list, &bo->rx_ops); + /* add a new bcm_op to the list of the rx_ops */ + if (new_op) + list_add_rcu(&op->list, &bo->rx_ops); } return msg_head->nframes * op->cfsiz + MHSIZ; + +free_op: + if (op->frames != &op->sframe) + kfree(op->frames); + if (op->last_frames != &op->last_sframe) + kfree(op->last_frames); + kfree(op); + return err; } /* @@ -1545,11 +1790,30 @@ static void bcm_notify(struct bcm_sock *bo, unsigned long msg, case NETDEV_UNREGISTER: lock_sock(sk); - /* remove device specific receive entries */ - list_for_each_entry(op, &bo->rx_ops, list) + /* rx_ops: remove device specific receive entries */ + list_for_each_entry(op, &bo->rx_ops, list) { if (op->rx_reg_dev == dev) bcm_rx_unreg(dev, op); + /* release an ANYDEV op's claim (see bcm_rx_handler()) + * on this now confirmed-gone interface. + */ + if (!op->ifindex) { + spin_lock_bh(&op->bcm_rx_update_lock); + if (op->if_detected == dev->ifindex) + op->if_detected = 0; + spin_unlock_bh(&op->bcm_rx_update_lock); + } + } + + /* tx_ops: stop device specific cyclic transmissions on the + * vanishing ifindex. Cancelling the timer is enough to stop + * cyclic bcm_can_tx() calls as there is no re-arming. + */ + list_for_each_entry(op, &bo->tx_ops, list) + if (op->ifindex == dev->ifindex) + hrtimer_cancel(&op->timer); + /* remove device reference, if this is our bound device */ if (bo->bound && bo->ifindex == dev->ifindex) { #if IS_ENABLED(CONFIG_PROC_FS) @@ -1682,16 +1946,14 @@ static int bcm_release(struct socket *sock) * Only remove subscriptions that had not * been removed due to NETDEV_UNREGISTER * in bcm_notifier() + * + * op->rx_reg_dev is a tracked reference taken + * when the subscription was registered, so it + * stays valid here even if a concurrent + * NETDEV_UNREGISTER already unlisted the device. */ - if (op->rx_reg_dev) { - struct net_device *dev; - - dev = dev_get_by_index(net, op->ifindex); - if (dev) { - bcm_rx_unreg(dev, op); - dev_put(dev); - } - } + if (op->rx_reg_dev) + bcm_rx_unreg(op->rx_reg_dev, op); } else can_rx_unregister(net, NULL, op->can_id, REGMASK(op->can_id), diff --git a/net/can/j1939/transport.c b/net/can/j1939/transport.c index 8656ab388c83..9ec929bc0c0d 100644 --- a/net/can/j1939/transport.c +++ b/net/can/j1939/transport.c @@ -350,6 +350,18 @@ static void j1939_session_skb_drop_old(struct j1939_session *session) } } +static bool j1939_address_is_local(struct j1939_priv *priv, u8 addr) +{ + bool local = false; + + read_lock_bh(&priv->lock); + if (j1939_address_is_unicast(addr) && priv->ents[addr].nusers) + local = true; + read_unlock_bh(&priv->lock); + + return local; +} + void j1939_session_skb_queue(struct j1939_session *session, struct sk_buff *skb) { @@ -358,8 +370,7 @@ void j1939_session_skb_queue(struct j1939_session *session, j1939_ac_fixup(priv, skb); - if (j1939_address_is_unicast(skcb->addr.da) && - priv->ents[skcb->addr.da].nusers) + if (j1939_address_is_local(priv, skcb->addr.da)) skcb->flags |= J1939_ECU_LOCAL_DST; skcb->flags |= J1939_ECU_LOCAL_SRC; @@ -2017,8 +2028,7 @@ struct j1939_session *j1939_tp_send(struct j1939_priv *priv, return ERR_PTR(ret); /* fix DST flags, it may be used there soon */ - if (j1939_address_is_unicast(skcb->addr.da) && - priv->ents[skcb->addr.da].nusers) + if (j1939_address_is_local(priv, skcb->addr.da)) skcb->flags |= J1939_ECU_LOCAL_DST; /* src is always local, I'm sending ... */ diff --git a/net/can/raw.c b/net/can/raw.c index 263e7167d2f5..365559d19dc3 100644 --- a/net/can/raw.c +++ b/net/can/raw.c @@ -560,8 +560,8 @@ static int raw_getname(struct socket *sock, struct sockaddr *uaddr, return RAW_MIN_NAMELEN; } -static int raw_setsockopt(struct socket *sock, int level, int optname, - sockptr_t optval, unsigned int optlen) +static int raw_setsockopt_locked(struct socket *sock, int optname, + sockptr_t optval, unsigned int optlen) { struct sock *sk = sock->sk; struct raw_sock *ro = raw_sk(sk); @@ -573,9 +573,6 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, int flag; int err = 0; - if (level != SOL_CAN_RAW) - return -EINVAL; - switch (optname) { case CAN_RAW_FILTER: if (optlen % sizeof(struct can_filter) != 0) @@ -596,17 +593,11 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, return -EFAULT; } - rtnl_lock(); - lock_sock(sk); - dev = ro->dev; - if (ro->bound && dev) { - if (dev->reg_state != NETREG_REGISTERED) { - if (count > 1) - kfree(filter); - err = -ENODEV; - goto out_fil; - } + if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED) { + if (count > 1) + kfree(filter); + return -ENODEV; } if (ro->bound) { @@ -620,7 +611,7 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, if (err) { if (count > 1) kfree(filter); - goto out_fil; + return err; } /* remove old filter registrations */ @@ -640,11 +631,6 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, } ro->filter = filter; ro->count = count; - - out_fil: - release_sock(sk); - rtnl_unlock(); - break; case CAN_RAW_ERR_FILTER: @@ -656,16 +642,9 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, err_mask &= CAN_ERR_MASK; - rtnl_lock(); - lock_sock(sk); - dev = ro->dev; - if (ro->bound && dev) { - if (dev->reg_state != NETREG_REGISTERED) { - err = -ENODEV; - goto out_err; - } - } + if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED) + return -ENODEV; /* remove current error mask */ if (ro->bound) { @@ -674,7 +653,7 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, err_mask); if (err) - goto out_err; + return err; /* remove old err_mask registration */ raw_disable_errfilter(sock_net(sk), dev, sk, @@ -683,11 +662,6 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, /* link new err_mask to the socket */ ro->err_mask = err_mask; - - out_err: - release_sock(sk); - rtnl_unlock(); - break; case CAN_RAW_LOOPBACK: @@ -767,6 +741,26 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, return err; } +static int raw_setsockopt(struct socket *sock, int level, int optname, + sockptr_t optval, unsigned int optlen) +{ + struct sock *sk = sock->sk; + int err; + + if (level != SOL_CAN_RAW) + return -EINVAL; + + rtnl_lock(); + lock_sock(sk); + + err = raw_setsockopt_locked(sock, optname, optval, optlen); + + release_sock(sk); + rtnl_unlock(); + + return err; +} + static int raw_getsockopt(struct socket *sock, int level, int optname, char __user *optval, int __user *optlen) { diff --git a/net/ceph/auth_x.c b/net/ceph/auth_x.c index a21c157daf7d..c83559f015e4 100644 --- a/net/ceph/auth_x.c +++ b/net/ceph/auth_x.c @@ -781,9 +781,16 @@ static int ceph_x_update_authorizer( au = (struct ceph_x_authorizer *)auth->authorizer; if (au->secret_id < th->secret_id) { + int ret; + dout("ceph_x_update_authorizer service %u secret %llu < %llu\n", au->service, au->secret_id, th->secret_id); - return ceph_x_build_authorizer(ac, th, au); + ret = ceph_x_build_authorizer(ac, th, au); + if (ret) + return ret; + + auth->authorizer_buf = au->buf->vec.iov_base; + auth->authorizer_buf_len = au->buf->vec.iov_len; } return 0; } diff --git a/net/ceph/ceph_common.c b/net/ceph/ceph_common.c index e734e57be083..5974c23347dc 100644 --- a/net/ceph/ceph_common.c +++ b/net/ceph/ceph_common.c @@ -763,13 +763,13 @@ void ceph_destroy_client(struct ceph_client *client) atomic_set(&client->msgr.stopping, 1); + ceph_debugfs_client_cleanup(client); + /* unmount */ ceph_osdc_stop(&client->osdc); ceph_monc_stop(&client->monc); ceph_messenger_fini(&client->msgr); - ceph_debugfs_client_cleanup(client); - ceph_destroy_options(client->options); kfree(client); diff --git a/net/ceph/mon_client.c b/net/ceph/mon_client.c index ff4311d5764b..96c9c4d0a375 100644 --- a/net/ceph/mon_client.c +++ b/net/ceph/mon_client.c @@ -114,7 +114,7 @@ static struct ceph_monmap *ceph_monmap_decode(void **p, void *end, bool msgr2) dout("%s fsid %pU epoch %u num_mon %u\n", __func__, &fsid, epoch, num_mon); - if (num_mon > CEPH_MAX_MON) + if (num_mon == 0 || num_mon > CEPH_MAX_MON) goto e_inval; monmap = kmalloc(struct_size(monmap, mon_inst, num_mon), GFP_NOIO); @@ -821,7 +821,7 @@ static void handle_get_version_reply(struct ceph_mon_client *monc, struct ceph_mon_generic_request *req; u64 tid = le64_to_cpu(msg->hdr.tid); void *p = msg->front.iov_base; - void *end = p + msg->front_alloc_len; + void *const end = p + msg->front.iov_len; u64 handle; dout("%s msg %p tid %llu\n", __func__, msg, tid); diff --git a/net/ceph/osdmap.c b/net/ceph/osdmap.c index c34a5bf86831..30d75970be44 100644 --- a/net/ceph/osdmap.c +++ b/net/ceph/osdmap.c @@ -520,6 +520,8 @@ static struct crush_map *crush_decode(void *pbyval, void *end) ceph_decode_need(p, end, 4*sizeof(u32), bad); b->id = ceph_decode_32(p); b->type = ceph_decode_16(p); + if (b->type == 0) + goto bad; b->alg = ceph_decode_8(p); if (b->alg != alg) { b->alg = 0; @@ -1844,6 +1846,8 @@ static int decode_new_up_state_weight(void **p, void *end, u8 struct_v, void *new_up_client; void *new_state; void *new_weight_end; + const u32 new_state_item_size = + sizeof(u32) + (struct_v >= 5 ? sizeof(u32) : sizeof(u8)); u32 len; int ret; int i; @@ -1864,7 +1868,8 @@ static int decode_new_up_state_weight(void **p, void *end, u8 struct_v, new_state = *p; ceph_decode_32_safe(p, end, len, e_inval); - len *= sizeof(u32) + (struct_v >= 5 ? sizeof(u32) : sizeof(u8)); + if (check_mul_overflow(len, new_state_item_size, &len)) + goto e_inval; ceph_decode_need(p, end, len, e_inval); *p += len; @@ -3057,8 +3062,11 @@ static int get_immediate_parent(struct crush_map *c, int id, if (b->items[j] != id) continue; - *parent_type_id = b->type; type_cn = lookup_crush_name(&c->type_names, b->type); + if (WARN_ON_ONCE(!type_cn)) + continue; + + *parent_type_id = b->type; parent_loc->cl_type_name = type_cn->cn_name; parent_loc->cl_name = cn->cn_name; return b->id; diff --git a/net/core/dev.c b/net/core/dev.c index 8dee703a0916..a83083e8761b 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4005,6 +4005,9 @@ static struct sk_buff *validate_xmit_unreadable_skb(struct sk_buff *skb, return NULL; } +/* Returns the skb on success, NULL if dropped, or ERR_PTR(-EINPROGRESS) + * if stolen by async xfrm crypto (delivered via xfrm_dev_resume()). + */ static struct sk_buff *validate_xmit_skb(struct sk_buff *skb, struct net_device *dev, bool *again) { netdev_features_t features; @@ -4076,7 +4079,7 @@ struct sk_buff *validate_xmit_skb_list(struct sk_buff *skb, struct net_device *d skb->prev = skb; skb = validate_xmit_skb(skb, dev, again); - if (!skb) + if (IS_ERR_OR_NULL(skb)) continue; if (!head) @@ -4702,9 +4705,10 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) { struct net_device *dev = skb->dev; struct netdev_queue *txq = NULL; - struct Qdisc *q; - int rc = -ENOMEM; + enum skb_drop_reason reason; + int cpu, rc = -ENOMEM; bool again = false; + struct Qdisc *q; skb_reset_mac_header(skb); skb_assert_len(skb); @@ -4773,59 +4777,64 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) * Check this and shot the lock. It is not prone from deadlocks. *Either shot noqueue qdisc, it is even simpler 8) */ - if (dev->flags & IFF_UP) { - int cpu = smp_processor_id(); /* ok because BHs are off */ + if (unlikely(!(dev->flags & IFF_UP))) { + reason = SKB_DROP_REASON_DEV_READY; + goto drop; + } - if (!netif_tx_owned(txq, cpu)) { - bool is_list = false; + cpu = smp_processor_id(); /* ok because BHs are off */ - if (dev_xmit_recursion()) - goto recursion_alert; + if (likely(!netif_tx_owned(txq, cpu))) { + bool is_list = false; - skb = validate_xmit_skb(skb, dev, &again); - if (!skb) - goto out; + if (dev_xmit_recursion()) + goto recursion_alert; - HARD_TX_LOCK(dev, txq, cpu); + skb = validate_xmit_skb(skb, dev, &again); + if (IS_ERR_OR_NULL(skb)) { + if (PTR_ERR(skb) == -EINPROGRESS) + rc = NET_XMIT_SUCCESS; + goto out; + } - if (!netif_xmit_stopped(txq)) { - is_list = !!skb->next; + HARD_TX_LOCK(dev, txq, cpu); - dev_xmit_recursion_inc(); - skb = dev_hard_start_xmit(skb, dev, txq, &rc); - dev_xmit_recursion_dec(); + if (!netif_xmit_stopped(txq)) { + is_list = !!skb->next; - /* GSO segments a single SKB into - * a list of frames. TCP expects error - * to mean none of the data was sent. - */ - if (is_list) - rc = NETDEV_TX_OK; - } - HARD_TX_UNLOCK(dev, txq); - if (!skb) /* xmit completed */ - goto out; + dev_xmit_recursion_inc(); + skb = dev_hard_start_xmit(skb, dev, txq, &rc); + dev_xmit_recursion_dec(); - net_crit_ratelimited("Virtual device %s asks to queue packet!\n", - dev->name); - /* NETDEV_TX_BUSY or queue was stopped */ - if (!is_list) - rc = -ENETDOWN; - } else { - /* Recursion is detected! It is possible, - * unfortunately + /* GSO segments a single SKB into a list of frames. + * TCP expects error to mean none of the data was sent. */ -recursion_alert: - net_crit_ratelimited("Dead loop on virtual device %s, fix it urgently!\n", - dev->name); - rc = -ENETDOWN; + if (is_list) + rc = NETDEV_TX_OK; } + HARD_TX_UNLOCK(dev, txq); + if (!skb) /* xmit completed */ + goto out; + + net_crit_ratelimited("Virtual device %s asks to queue packet!\n", + dev->name); + /* NETDEV_TX_BUSY or queue was stopped */ + if (!is_list) + rc = -ENETDOWN; + } else { + /* Recursion is detected! It is possible unfortunately. */ +recursion_alert: + net_crit_ratelimited("Dead loop on virtual device %s, fix it urgently!\n", + dev->name); + rc = -ENETDOWN; } + reason = SKB_DROP_REASON_RECURSION_LIMIT; +drop: rcu_read_unlock_bh(); dev_core_stats_tx_dropped_inc(dev); - kfree_skb_list(skb); + kfree_skb_list_reason(skb, reason); return rc; out: rcu_read_unlock_bh(); diff --git a/net/core/filter.c b/net/core/filter.c index 42385e790431..332f4986376f 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -10539,10 +10539,11 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, si->dst_reg, si->dst_reg, \ offsetof(OBJ, OBJ_FIELD)); \ if (si->dst_reg == si->src_reg) { \ - *insn++ = BPF_JMP_A(1); \ + *insn++ = BPF_JMP_A(2); \ *insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg, \ offsetof(struct bpf_sock_ops_kern, \ temp)); \ + *insn++ = BPF_MOV64_IMM(si->dst_reg, 0); \ } \ } while (0) @@ -10576,10 +10577,11 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, si->dst_reg, si->src_reg, \ offsetof(struct bpf_sock_ops_kern, sk));\ if (si->dst_reg == si->src_reg) { \ - *insn++ = BPF_JMP_A(1); \ + *insn++ = BPF_JMP_A(2); \ *insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg, \ offsetof(struct bpf_sock_ops_kern, \ temp)); \ + *insn++ = BPF_MOV64_IMM(si->dst_reg, 0); \ } \ } while (0) diff --git a/net/core/gro.c b/net/core/gro.c index 9ec8a46b30bb..70770f9f9c45 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -231,7 +231,9 @@ int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb) int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb) { - if (unlikely(p->len + skb->len >= 65536)) + /* make sure to check flush flag and to not merge */ + if (unlikely(p->len + skb->len >= 65536 || + NAPI_GRO_CB(skb)->flush)) return -E2BIG; if (!pskb_may_pull(skb, skb_gro_offset(skb))) { diff --git a/net/core/sock.c b/net/core/sock.c index c52cec0bf942..5a658606c50e 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2525,6 +2525,11 @@ struct sock *sk_clone_lock(const struct sock *sk, const gfp_t priority) cgroup_sk_clone(&newsk->sk_cgrp_data); + RCU_INIT_POINTER(newsk->sk_reuseport_cb, NULL); + + if (sock_needs_netstamp(sk) && newsk->sk_flags & SK_FLAGS_TIMESTAMP) + net_enable_timestamp(); + rcu_read_lock(); filter = rcu_dereference(sk->sk_filter); if (filter != NULL) @@ -2547,8 +2552,6 @@ struct sock *sk_clone_lock(const struct sock *sk, const gfp_t priority) goto free; } - RCU_INIT_POINTER(newsk->sk_reuseport_cb, NULL); - if (bpf_sk_storage_clone(sk, newsk)) goto free; @@ -2575,9 +2578,6 @@ struct sock *sk_clone_lock(const struct sock *sk, const gfp_t priority) if (newsk->sk_prot->sockets_allocated) sk_sockets_allocated_inc(newsk); - - if (sock_needs_netstamp(sk) && newsk->sk_flags & SK_FLAGS_TIMESTAMP) - net_enable_timestamp(); out: return newsk; free: diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 5947b38e4f8b..70bbc78fb079 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -542,6 +542,8 @@ static bool sock_map_sk_state_allowed(const struct sock *sk) { if (sk_is_tcp(sk)) return (1 << sk->sk_state) & (TCPF_ESTABLISHED | TCPF_LISTEN); + if (sk_is_udp(sk)) + return sk_hashed(sk); if (sk_is_stream_unix(sk)) return (1 << sk->sk_state) & TCPF_ESTABLISHED; if (sk_is_vsock(sk) && diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c index afe06ba00ea4..474f44f1ee88 100644 --- a/net/hsr/hsr_slave.c +++ b/net/hsr/hsr_slave.c @@ -242,6 +242,8 @@ void hsr_del_port(struct hsr_port *port) netdev_rx_handler_unregister(port->dev); if (!port->hsr->fwd_offloaded) dev_set_promiscuity(port->dev, -1); + if (port->type == HSR_PT_SLAVE_A || port->type == HSR_PT_SLAVE_B) + vlan_vids_del_by_dev(port->dev, master->dev); netdev_upper_dev_unlink(port->dev, master->dev); eth_hw_addr_set(port->dev, port->original_macaddress); } diff --git a/net/ipv4/datagram.c b/net/ipv4/datagram.c index c2b2cda1a7e5..7861615b34fc 100644 --- a/net/ipv4/datagram.c +++ b/net/ipv4/datagram.c @@ -63,12 +63,12 @@ int __ip4_datagram_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len } /* Update addresses before rehashing */ - inet->inet_daddr = fl4->daddr; + WRITE_ONCE(inet->inet_daddr, fl4->daddr); inet->inet_dport = usin->sin_port; if (!inet->inet_saddr) inet->inet_saddr = fl4->saddr; if (!inet->inet_rcv_saddr) { - inet->inet_rcv_saddr = fl4->saddr; + WRITE_ONCE(inet->inet_rcv_saddr, fl4->saddr); if (sk->sk_prot->rehash) sk->sk_prot->rehash(sk); } diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c index 1308213791f1..2dc87e2156d3 100644 --- a/net/ipv4/fib_trie.c +++ b/net/ipv4/fib_trie.c @@ -1385,7 +1385,7 @@ int fib_table_insert(struct net *net, struct fib_table *tb, out_remove_new_fa: fib_remove_alias(t, tp, l, new_fa); out_free_new_fa: - kmem_cache_free(fn_alias_kmem, new_fa); + alias_free_mem_rcu(new_fa); out: fib_release_info(fi); err: diff --git a/net/ipv4/icmp.c b/net/ipv4/icmp.c index fc0a93f43313..691d7b076c62 100644 --- a/net/ipv4/icmp.c +++ b/net/ipv4/icmp.c @@ -539,11 +539,23 @@ static struct rtable *icmp_route_lookup(struct net *net, struct flowi4 *fl4, if (IS_ERR(rt2)) err = PTR_ERR(rt2); } else { - struct flowi4 fl4_2 = {}; + struct flowi4 fl4_2 = fl4_dec; unsigned long orefdst; - fl4_2.daddr = fl4_dec.saddr; - rt2 = ip_route_output_key(net, &fl4_2); + swap(fl4_2.daddr, fl4_2.saddr); + switch (fl4_2.flowi4_proto) { + case IPPROTO_TCP: + case IPPROTO_UDP: + case IPPROTO_SCTP: + case IPPROTO_DCCP: + swap(fl4_2.fl4_sport, fl4_2.fl4_dport); + break; + } + + fl4_2.flowi4_oif = l3mdev_master_ifindex(route_lookup_dev); + fl4_2.flowi4_flags |= FLOWI_FLAG_ANYSRC; + + rt2 = __ip_route_output_key(net, &fl4_2); if (IS_ERR(rt2)) { err = PTR_ERR(rt2); goto relookup_failed; diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c index 879d37c557fa..300b802125db 100644 --- a/net/ipv4/ip_gre.c +++ b/net/ipv4/ip_gre.c @@ -1017,6 +1017,8 @@ static void __gre_tunnel_init(struct net_device *dev) dev->features |= GRE_FEATURES; dev->hw_features |= GRE_FEATURES; + dev->lltx = true; + /* TCP offload with GRE SEQ is not supported, nor can we support 2 * levels of outer headers requiring an update. */ @@ -1028,8 +1030,6 @@ static void __gre_tunnel_init(struct net_device *dev) dev->features |= NETIF_F_GSO_SOFTWARE; dev->hw_features |= NETIF_F_GSO_SOFTWARE; - - dev->lltx = true; } static int ipgre_tunnel_init(struct net_device *dev) diff --git a/net/ipv4/ip_tunnel_core.c b/net/ipv4/ip_tunnel_core.c index 4b5fd4b13722..fc993c78cbcc 100644 --- a/net/ipv4/ip_tunnel_core.c +++ b/net/ipv4/ip_tunnel_core.c @@ -65,7 +65,7 @@ void iptunnel_xmit(struct sock *sk, struct rtable *rt, struct sk_buff *skb, DEV_STATS_INC(dev, tx_errors); } ip_rt_put(rt); - kfree_skb(skb); + kfree_skb_reason(skb, SKB_DROP_REASON_RECURSION_LIMIT); return; } diff --git a/net/ipv4/netfilter/nft_dup_ipv4.c b/net/ipv4/netfilter/nft_dup_ipv4.c index ef5dd88107dd..d53a65ddbd7b 100644 --- a/net/ipv4/netfilter/nft_dup_ipv4.c +++ b/net/ipv4/netfilter/nft_dup_ipv4.c @@ -76,7 +76,6 @@ static const struct nft_expr_ops nft_dup_ipv4_ops = { .eval = nft_dup_ipv4_eval, .init = nft_dup_ipv4_init, .dump = nft_dup_ipv4_dump, - .reduce = NFT_REDUCE_READONLY, }; static const struct nla_policy nft_dup_ipv4_policy[NFTA_DUP_MAX + 1] = { diff --git a/net/ipv4/netfilter/nft_fib_ipv4.c b/net/ipv4/netfilter/nft_fib_ipv4.c index e695283aeb2d..177d738825b4 100644 --- a/net/ipv4/netfilter/nft_fib_ipv4.c +++ b/net/ipv4/netfilter/nft_fib_ipv4.c @@ -163,7 +163,6 @@ static const struct nft_expr_ops nft_fib4_type_ops = { .init = nft_fib_init, .dump = nft_fib_dump, .validate = nft_fib_validate, - .reduce = nft_fib_reduce, }; static const struct nft_expr_ops nft_fib4_ops = { @@ -173,7 +172,6 @@ static const struct nft_expr_ops nft_fib4_ops = { .init = nft_fib_init, .dump = nft_fib_dump, .validate = nft_fib_validate, - .reduce = nft_fib_reduce, }; static const struct nft_expr_ops * diff --git a/net/ipv4/netfilter/nft_reject_ipv4.c b/net/ipv4/netfilter/nft_reject_ipv4.c index 6cb213bb7256..55fc23a8f7a7 100644 --- a/net/ipv4/netfilter/nft_reject_ipv4.c +++ b/net/ipv4/netfilter/nft_reject_ipv4.c @@ -45,7 +45,6 @@ static const struct nft_expr_ops nft_reject_ipv4_ops = { .init = nft_reject_init, .dump = nft_reject_dump, .validate = nft_reject_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_reject_ipv4_type __read_mostly = { diff --git a/net/ipv4/nexthop.c b/net/ipv4/nexthop.c index 5a95b64b61c5..a8bf1ec54899 100644 --- a/net/ipv4/nexthop.c +++ b/net/ipv4/nexthop.c @@ -1791,8 +1791,8 @@ static bool nh_res_bucket_migrate(struct nh_res_table *res_table, bool notify_nl, bool force) { struct nh_res_bucket *bucket = &res_table->nh_buckets[bucket_index]; + struct netlink_ext_ack extack = {}; struct nh_grp_entry *new_nhge; - struct netlink_ext_ack extack; int err; new_nhge = list_first_entry_or_null(&res_table->uw_nh_entries, diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c index 26468a2ec63d..c10cf41a365d 100644 --- a/net/ipv4/raw.c +++ b/net/ipv4/raw.c @@ -120,13 +120,21 @@ bool raw_v4_match(struct net *net, const struct sock *sk, unsigned short num, __be32 raddr, __be32 laddr, int dif, int sdif) { const struct inet_sock *inet = inet_sk(sk); + __be32 daddr, rcv_saddr; - if (net_eq(sock_net(sk), net) && inet->inet_num == num && - !(inet->inet_daddr && inet->inet_daddr != raddr) && - !(inet->inet_rcv_saddr && inet->inet_rcv_saddr != laddr) && - raw_sk_bound_dev_eq(net, sk->sk_bound_dev_if, dif, sdif)) - return true; - return false; + if (!net_eq(sock_net(sk), net) || inet->inet_num != num) + return false; + + daddr = READ_ONCE(inet->inet_daddr); + if (daddr && daddr != raddr) + return false; + + rcv_saddr = READ_ONCE(inet->inet_rcv_saddr); + if (rcv_saddr && rcv_saddr != laddr) + return false; + + return raw_sk_bound_dev_eq(net, READ_ONCE(sk->sk_bound_dev_if), + dif, sdif); } EXPORT_SYMBOL_GPL(raw_v4_match); @@ -721,7 +729,8 @@ static int raw_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len) chk_addr_ret)) goto out; - inet->inet_rcv_saddr = inet->inet_saddr = addr->sin_addr.s_addr; + inet->inet_saddr = addr->sin_addr.s_addr; + WRITE_ONCE(inet->inet_rcv_saddr, addr->sin_addr.s_addr); if (chk_addr_ret == RTN_MULTICAST || chk_addr_ret == RTN_BROADCAST) inet->inet_saddr = 0; /* Use device */ sk_dst_reset(sk); diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 7a3f0b244c8f..c27e797eff27 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -589,7 +589,7 @@ static int tcp_bpf_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) wait_for_memory: err = sk_stream_wait_memory(sk, &timeo); if (err) { - if (msg_tx && msg_tx != psock->cork) + if (msg_tx == &tmp) sk_msg_free(sk, msg_tx); goto out_err; } diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index c650398e9199..8c74fbeb771d 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -3820,24 +3820,17 @@ static void tcp_send_ack_reflect_ect(struct sock *sk, bool accecn_reflector) __tcp_send_ack(sk, tp->rcv_nxt, flags); } -/* RFC 5961 7 [ACK Throttling] */ -static void tcp_send_challenge_ack(struct sock *sk, bool accecn_reflector) +/* Consume one slot from the per-netns RFC 5961 challenge ACK quota. + * Returns true if a challenge ACK may be sent. + */ +static bool tcp_challenge_ack_allowed(struct net *net) { - struct tcp_sock *tp = tcp_sk(sk); - struct net *net = sock_net(sk); u32 count, now, ack_limit; - /* First check our per-socket dupack rate limit. */ - if (__tcp_oow_rate_limited(net, - LINUX_MIB_TCPACKSKIPPEDCHALLENGE, - &tp->last_oow_ack_time)) - return; - ack_limit = READ_ONCE(net->ipv4.sysctl_tcp_challenge_ack_limit); if (ack_limit == INT_MAX) - goto send_ack; + return true; - /* Then check host-wide RFC 5961 rate limit. */ now = jiffies / HZ; if (now != READ_ONCE(net->ipv4.tcp_challenge_timestamp)) { u32 half = (ack_limit + 1) >> 1; @@ -3849,12 +3842,49 @@ static void tcp_send_challenge_ack(struct sock *sk, bool accecn_reflector) count = READ_ONCE(net->ipv4.tcp_challenge_count); if (count > 0) { WRITE_ONCE(net->ipv4.tcp_challenge_count, count - 1); -send_ack: + return true; + } + return false; +} + +/* RFC 5961 7 [ACK Throttling] */ +static void tcp_send_challenge_ack(struct sock *sk, bool accecn_reflector) +{ + struct tcp_sock *tp = tcp_sk(sk); + struct net *net = sock_net(sk); + + /* First check our per-socket dupack rate limit. */ + if (__tcp_oow_rate_limited(net, + LINUX_MIB_TCPACKSKIPPEDCHALLENGE, + &tp->last_oow_ack_time)) + return; + + /* Then check the per-netns RFC 5961 rate limit. */ + if (tcp_challenge_ack_allowed(net)) { NET_INC_STATS(net, LINUX_MIB_TCPCHALLENGEACK); tcp_send_ack_reflect_ect(sk, accecn_reflector); } } +/* Send a challenge ACK from a SYN-RECEIVED request socket. Uses + * __tcp_oow_rate_limited() directly so that an RST carrying payload + * cannot bypass the per-request rate limit. + */ +void tcp_reqsk_send_challenge_ack(struct sock *sk, struct sk_buff *skb, + struct request_sock *req) +{ + struct net *net = sock_net(sk); + + if (__tcp_oow_rate_limited(net, LINUX_MIB_TCPACKSKIPPEDCHALLENGE, + &tcp_rsk(req)->last_oow_ack_time)) + return; + + if (tcp_challenge_ack_allowed(net)) { + NET_INC_STATS(net, LINUX_MIB_TCPCHALLENGEACK); + req->rsk_ops->send_ack(sk, skb, req); + } +} + static void tcp_store_ts_recent(struct tcp_sock *tp) { tp->rx_opt.ts_recent = tp->rx_opt.rcv_tsval; diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index f0106c2be1cf..3463e5b964a9 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -998,6 +998,9 @@ static void tcp_v4_send_ack(const struct sock *sk, key->rcv_next); arg.iov[0].iov_len += tcp_ao_len_aligned(key->ao_key); rep.th.doff = arg.iov[0].iov_len / 4; + memset((u8 *)&rep.opt[offset] + tcp_ao_maclen(key->ao_key), + TCPOPT_NOP, tcp_ao_len_aligned(key->ao_key) - + tcp_ao_len(key->ao_key)); tcp_ao_hash_hdr(AF_INET, (char *)&rep.opt[offset], key->ao_key, key->traffic_key, @@ -2452,8 +2455,10 @@ int tcp_v4_rcv(struct sk_buff *skb) } drop_reason = psp_twsk_rx_policy_check(inet_twsk(sk), skb); - if (drop_reason) - break; + if (drop_reason) { + inet_twsk_put(inet_twsk(sk)); + goto discard_it; + } } /* to ACK */ fallthrough; @@ -3282,8 +3287,11 @@ static struct sock *bpf_iter_tcp_batch(struct seq_file *seq) bpf_iter_tcp_put_batch(iter); err = bpf_iter_tcp_realloc_batch(iter, expected * 3 / 2, GFP_USER); - if (err) + if (err) { + iter->cur_sk = 0; + iter->end_sk = 0; return ERR_PTR(err); + } sk = bpf_iter_tcp_resume(seq); if (!sk) diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index c70c29a3a090..4ac30573f233 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -822,7 +822,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb, * elsewhere and is checked directly against the child socket rather * than req because user data may have been sent out. */ - if ((flg & TCP_FLAG_ACK) && !fastopen && + if ((flg & TCP_FLAG_ACK) && !(flg & TCP_FLAG_RST) && !fastopen && (TCP_SKB_CB(skb)->ack_seq != tcp_rsk(req)->snt_isn + 1)) return sk; @@ -861,6 +861,16 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb, flg &= ~TCP_FLAG_SYN; } + /* RFC 5961 section 3.2, as clarified by RFC 9293 section + * 3.10.7.4, requires a challenge ACK for a non-exact + * in-window RST in SYN-RECEIVED. + */ + if ((flg & TCP_FLAG_RST) && + TCP_SKB_CB(skb)->seq != tcp_rsk(req)->rcv_nxt) { + tcp_reqsk_send_challenge_ack(sk, skb, req); + return NULL; + } + /* RFC793: "second check the RST bit" and * "fourth, check the SYN bit" */ diff --git a/net/ipv6/ila/ila_common.c b/net/ipv6/ila/ila_common.c index b8d43ed4689d..ca54a227fc2f 100644 --- a/net/ipv6/ila/ila_common.c +++ b/net/ipv6/ila/ila_common.c @@ -84,6 +84,7 @@ static void ila_csum_adjust_transport(struct sk_buff *skb, struct tcphdr *th = (struct tcphdr *) (skb_network_header(skb) + nhoff); + ip6h = ipv6_hdr(skb); diff = get_csum_diff(ip6h, p); inet_proto_csum_replace_by_diff(&th->check, skb, diff, true, true); @@ -95,6 +96,7 @@ static void ila_csum_adjust_transport(struct sk_buff *skb, (skb_network_header(skb) + nhoff); if (uh->check || skb->ip_summed == CHECKSUM_PARTIAL) { + ip6h = ipv6_hdr(skb); diff = get_csum_diff(ip6h, p); inet_proto_csum_replace_by_diff(&uh->check, skb, diff, true, true); @@ -109,6 +111,7 @@ static void ila_csum_adjust_transport(struct sk_buff *skb, struct icmp6hdr *ih = (struct icmp6hdr *) (skb_network_header(skb) + nhoff); + ip6h = ipv6_hdr(skb); diff = get_csum_diff(ip6h, p); inet_proto_csum_replace_by_diff(&ih->icmp6_cksum, skb, diff, true, true); @@ -126,6 +129,15 @@ void ila_update_ipv6_locator(struct sk_buff *skb, struct ila_params *p, switch (p->csum_mode) { case ILA_CSUM_ADJUST_TRANSPORT: ila_csum_adjust_transport(skb, p); + /* + * ila_csum_adjust_transport() calls pskb_may_pull(), which can + * reallocate the skb head and leave ip6h (and the iaddr derived + * from it) dangling; reload both before the write below. The + * other csum modes do not pull, so their cached pointers stay + * valid. + */ + ip6h = ipv6_hdr(skb); + iaddr = ila_a2i(&ip6h->daddr); break; case ILA_CSUM_NEUTRAL_MAP: if (sir2ila) { diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c index 75961c4ebcdd..58b473c157c8 100644 --- a/net/ipv6/ip6_gre.c +++ b/net/ipv6/ip6_gre.c @@ -1454,6 +1454,8 @@ static void ip6gre_tnl_init_features(struct net_device *dev) dev->features |= GRE6_FEATURES; dev->hw_features |= GRE6_FEATURES; + dev->lltx = true; + /* TCP offload with GRE SEQ is not supported, nor can we support 2 * levels of outer headers requiring an update. */ @@ -1465,8 +1467,6 @@ static void ip6gre_tnl_init_features(struct net_device *dev) dev->features |= NETIF_F_GSO_SOFTWARE; dev->hw_features |= NETIF_F_GSO_SOFTWARE; - - dev->lltx = true; } static int ip6gre_tunnel_init_common(struct net_device *dev) diff --git a/net/ipv6/netfilter/nft_dup_ipv6.c b/net/ipv6/netfilter/nft_dup_ipv6.c index 492a811828a7..95ec27b3971c 100644 --- a/net/ipv6/netfilter/nft_dup_ipv6.c +++ b/net/ipv6/netfilter/nft_dup_ipv6.c @@ -74,7 +74,6 @@ static const struct nft_expr_ops nft_dup_ipv6_ops = { .eval = nft_dup_ipv6_eval, .init = nft_dup_ipv6_init, .dump = nft_dup_ipv6_dump, - .reduce = NFT_REDUCE_READONLY, }; static const struct nla_policy nft_dup_ipv6_policy[NFTA_DUP_MAX + 1] = { diff --git a/net/ipv6/netfilter/nft_fib_ipv6.c b/net/ipv6/netfilter/nft_fib_ipv6.c index 3005dfbca615..6c04d0a80b26 100644 --- a/net/ipv6/netfilter/nft_fib_ipv6.c +++ b/net/ipv6/netfilter/nft_fib_ipv6.c @@ -225,7 +225,6 @@ static const struct nft_expr_ops nft_fib6_type_ops = { .init = nft_fib_init, .dump = nft_fib_dump, .validate = nft_fib_validate, - .reduce = nft_fib_reduce, }; static const struct nft_expr_ops nft_fib6_ops = { @@ -235,7 +234,6 @@ static const struct nft_expr_ops nft_fib6_ops = { .init = nft_fib_init, .dump = nft_fib_dump, .validate = nft_fib_validate, - .reduce = nft_fib_reduce, }; static const struct nft_expr_ops * diff --git a/net/ipv6/netfilter/nft_reject_ipv6.c b/net/ipv6/netfilter/nft_reject_ipv6.c index 5c61294f410e..ed69c768797e 100644 --- a/net/ipv6/netfilter/nft_reject_ipv6.c +++ b/net/ipv6/netfilter/nft_reject_ipv6.c @@ -46,7 +46,6 @@ static const struct nft_expr_ops nft_reject_ipv6_ops = { .init = nft_reject_init, .dump = nft_reject_dump, .validate = nft_reject_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_reject_ipv6_type __read_mostly = { diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c index c534848933ee..6cb8e32a3061 100644 --- a/net/ipv6/raw.c +++ b/net/ipv6/raw.c @@ -348,7 +348,7 @@ void raw6_icmp_error(struct sk_buff *skb, int nexthdr, const struct ipv6hdr *ip6h = (const struct ipv6hdr *)skb->data; if (!raw_v6_match(net, sk, nexthdr, &ip6h->saddr, &ip6h->daddr, - inet6_iif(skb), inet6_iif(skb))) + inet6_iif(skb), inet6_sdif(skb))) continue; rawv6_err(sk, skb, type, code, inner_offset, info); } diff --git a/net/ipv6/route.c b/net/ipv6/route.c index ef66a3c86feb..a45747bfb31a 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -4011,7 +4011,7 @@ static int __ip6_del_rt_siblings(struct fib6_info *rt, struct fib6_config *cfg) struct fib6_node *fn; /* prefer to send a single notification with all hops */ - skb = nlmsg_new(rt6_nlmsg_size(rt), gfp_any()); + skb = nlmsg_new(rt6_nlmsg_size(rt), GFP_ATOMIC); if (skb) { u32 seq = info->nlh ? info->nlh->nlmsg_seq : 0; @@ -4067,7 +4067,7 @@ static int __ip6_del_rt_siblings(struct fib6_info *rt, struct fib6_config *cfg) if (skb) { rtnl_notify(skb, net, info->portid, RTNLGRP_IPV6_ROUTE, - info->nlh, gfp_any()); + info->nlh, GFP_ATOMIC); } return err; } diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index 59b5900dd42b..952bf49efbc6 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -941,6 +941,8 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32 (tcp_ao_len(key->ao_key) << 16) | (key->ao_key->sndid << 8) | (key->rcv_next)); + memset((u8 *)topt + tcp_ao_maclen(key->ao_key), TCPOPT_NOP, + tcp_ao_len_aligned(key->ao_key) - tcp_ao_len(key->ao_key)); tcp_ao_hash_hdr(AF_INET6, (char *)topt, key->ao_key, key->traffic_key, @@ -1984,8 +1986,10 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb) } drop_reason = psp_twsk_rx_policy_check(inet_twsk(sk), skb); - if (drop_reason) - break; + if (drop_reason) { + inet_twsk_put(inet_twsk(sk)); + goto discard_it; + } } /* to ACK */ fallthrough; diff --git a/net/ipv6/xfrm6_policy.c b/net/ipv6/xfrm6_policy.c index 125ea9a5b8a0..3b749475f6ed 100644 --- a/net/ipv6/xfrm6_policy.c +++ b/net/ipv6/xfrm6_policy.c @@ -88,6 +88,7 @@ static int xfrm6_fill_dst(struct xfrm_dst *xdst, struct net_device *dev, xdst->u.rt6.rt6i_idev = in6_dev_get(dev); if (!xdst->u.rt6.rt6i_idev) { netdev_put(dev, &xdst->u.dst.dev_tracker); + xdst->u.dst.dev = NULL; return -ENODEV; } diff --git a/net/iucv/af_iucv.c b/net/iucv/af_iucv.c index c66b90c912e7..7f5a5aa0cfdc 100644 --- a/net/iucv/af_iucv.c +++ b/net/iucv/af_iucv.c @@ -335,6 +335,7 @@ static void iucv_sever_path(struct sock *sk, int with_user_data) unsigned char user_data[16]; struct iucv_sock *iucv = iucv_sk(sk); struct iucv_path *path = iucv->path; + struct sock_msg_q *p, *n; /* Whoever resets the path pointer, must sever and free it. */ if (xchg(&iucv->path, NULL)) { @@ -346,6 +347,19 @@ static void iucv_sever_path(struct sock *sk, int with_user_data) } else pr_iucv->path_sever(path, NULL); iucv_path_free(path); + + /* + * Message notifications queued on message_q still reference + * the now freed path; drop them, otherwise a later recvmsg() + * would pass the freed iucv_path to message_receive() via + * iucv_process_message_q(). + */ + spin_lock_bh(&iucv->message_q.lock); + list_for_each_entry_safe(p, n, &iucv->message_q.list, list) { + list_del(&p->list); + kfree(p); + } + spin_unlock_bh(&iucv->message_q.lock); } } @@ -1872,7 +1886,8 @@ static int afiucv_hs_callback_syn(struct sock *sk, struct sk_buff *skb) afiucv_swap_src_dest(skb); trans_hdr->flags = AF_IUCV_FLAG_SYN | AF_IUCV_FLAG_FIN; err = dev_queue_xmit(skb); - iucv_sock_kill(nsk); + if (nsk) + iucv_sock_kill(nsk); bh_unlock_sock(sk); goto out; } @@ -2089,6 +2104,8 @@ static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev, } } } + if (sk) + sock_hold(sk); read_unlock(&iucv_sk_list.lock); if (!iucv) sk = NULL; @@ -2138,6 +2155,8 @@ static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev, kfree_skb(skb); } + if (sk) + sock_put(sk); return err; } diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c index d32eacbb7517..f6232948bc0f 100644 --- a/net/mac80211/cfg.c +++ b/net/mac80211/cfg.c @@ -1124,9 +1124,6 @@ static int ieee80211_set_fils_discovery(struct ieee80211_sub_if_data *sdata, fd->max_interval = params->max_interval; old = sdata_dereference(link->u.ap.fils_discovery, sdata); - if (old) - kfree_rcu(old, rcu_head); - if (params->tmpl && params->tmpl_len) { new = kzalloc(sizeof(*new) + params->tmpl_len, GFP_KERNEL); if (!new) @@ -1138,6 +1135,9 @@ static int ieee80211_set_fils_discovery(struct ieee80211_sub_if_data *sdata, RCU_INIT_POINTER(link->u.ap.fils_discovery, NULL); } + if (old) + kfree_rcu(old, rcu_head); + *changed |= BSS_CHANGED_FILS_DISCOVERY; return 0; } @@ -1157,8 +1157,6 @@ ieee80211_set_unsol_bcast_probe_resp(struct ieee80211_sub_if_data *sdata, link_conf->unsol_bcast_probe_resp_interval = params->interval; old = sdata_dereference(link->u.ap.unsol_bcast_probe_resp, sdata); - if (old) - kfree_rcu(old, rcu_head); if (params->tmpl && params->tmpl_len) { new = kzalloc(sizeof(*new) + params->tmpl_len, GFP_KERNEL); @@ -1171,6 +1169,9 @@ ieee80211_set_unsol_bcast_probe_resp(struct ieee80211_sub_if_data *sdata, RCU_INIT_POINTER(link->u.ap.unsol_bcast_probe_resp, NULL); } + if (old) + kfree_rcu(old, rcu_head); + *changed |= BSS_CHANGED_UNSOL_BCAST_PROBE_RESP; return 0; } diff --git a/net/mac80211/iface.c b/net/mac80211/iface.c index 72c129478da0..499126317a91 100644 --- a/net/mac80211/iface.c +++ b/net/mac80211/iface.c @@ -581,6 +581,7 @@ static void ieee80211_do_stop(struct ieee80211_sub_if_data *sdata, bool going_do WARN_ON(!list_empty(&sdata->u.ap.vlans)); } else if (sdata->vif.type == NL80211_IFTYPE_AP_VLAN) { /* remove all packets in parent bc_buf pointing to this dev */ + __skb_queue_head_init(&freeq); ps = &sdata->bss->ps; spin_lock_irqsave(&ps->bc_buf.lock, flags); @@ -588,10 +589,15 @@ static void ieee80211_do_stop(struct ieee80211_sub_if_data *sdata, bool going_do if (skb->dev == sdata->dev) { __skb_unlink(skb, &ps->bc_buf); local->total_ps_buffered--; - ieee80211_free_txskb(&local->hw, skb); + __skb_queue_tail(&freeq, skb); } } spin_unlock_irqrestore(&ps->bc_buf.lock, flags); + + skb_queue_walk_safe(&freeq, skb, tmp) { + __skb_unlink(skb, &freeq); + ieee80211_free_txskb(&local->hw, skb); + } } if (going_down) diff --git a/net/mac80211/link.c b/net/mac80211/link.c index 05b0472bda40..235e370c2b59 100644 --- a/net/mac80211/link.c +++ b/net/mac80211/link.c @@ -382,6 +382,10 @@ static int ieee80211_vif_update_links(struct ieee80211_sub_if_data *sdata, memcpy(sdata->link, old_data, sizeof(old_data)); memcpy(sdata->vif.link_conf, old, sizeof(old)); ieee80211_set_vif_links_bitmaps(sdata, old_links, dormant_links); + for_each_set_bit(link_id, &add, IEEE80211_MLD_MAX_NUM_LINKS) { + ieee80211_link_debugfs_remove(&links[link_id]->data); + ieee80211_link_stop(&links[link_id]->data); + } /* and free (only) the newly allocated links */ memset(to_free, 0, sizeof(links)); goto free; diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c index 8854e47b0567..9ec4125c06d1 100644 --- a/net/mac80211/mlme.c +++ b/net/mac80211/mlme.c @@ -6476,7 +6476,7 @@ static void ieee80211_rx_mgmt_assoc_resp(struct ieee80211_sub_if_data *sdata, { struct ieee80211_if_managed *ifmgd = &sdata->u.mgd; struct ieee80211_mgd_assoc_data *assoc_data = ifmgd->assoc_data; - u16 capab_info, status_code, aid; + u16 capab_info, status_code, aid = 0; struct ieee80211_elems_parse_params parse_params = { .bss = NULL, .link_id = -1, @@ -6553,8 +6553,10 @@ static void ieee80211_rx_mgmt_assoc_resp(struct ieee80211_sub_if_data *sdata, if (elems->aid_resp) aid = le16_to_cpu(elems->aid_resp->aid); - else + else if (!assoc_data->s1g) aid = le16_to_cpu(mgmt->u.assoc_resp.aid); + else if (status_code == WLAN_STATUS_SUCCESS) + goto abandon_assoc; /* * The 5 MSB of the AID field are reserved for a non-S1G STA. For diff --git a/net/mac80211/rx.c b/net/mac80211/rx.c index 0a986607e659..13b7432d732f 100644 --- a/net/mac80211/rx.c +++ b/net/mac80211/rx.c @@ -1612,6 +1612,8 @@ static void sta_ps_start(struct sta_info *sta) else clear_bit(tid, &sta->txq_buffered_tids); } + + sta_info_recalc_tim(sta); } static void sta_ps_end(struct sta_info *sta) diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index b0d9bb830f29..54a6b20a4c5c 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -355,6 +355,15 @@ static void sta_info_free_link(struct link_sta_info *link_sta) free_percpu(link_sta->pcpu_rx_stats); } +static void sta_link_free_rcu(struct rcu_head *head) +{ + struct sta_link_alloc *alloc = + container_of(head, struct sta_link_alloc, rcu_head); + + sta_info_free_link(&alloc->info); + kfree(alloc); +} + static void sta_accumulate_removed_link_stats(struct sta_info *sta, int link_id) { struct link_sta_info *link_sta = wiphy_dereference(sta->local->hw.wiphy, @@ -427,10 +436,8 @@ static void sta_remove_link(struct sta_info *sta, unsigned int link_id, RCU_INIT_POINTER(sta->link[link_id], NULL); RCU_INIT_POINTER(sta->sta.link[link_id], NULL); - if (alloc) { - sta_info_free_link(&alloc->info); - kfree_rcu(alloc, rcu_head); - } + if (alloc) + call_rcu(&alloc->rcu_head, sta_link_free_rcu); ieee80211_sta_recalc_aggregates(&sta->sta); } diff --git a/net/mac802154/llsec.c b/net/mac802154/llsec.c index 09a47104b577..a55fa5017e93 100644 --- a/net/mac802154/llsec.c +++ b/net/mac802154/llsec.c @@ -891,6 +891,11 @@ llsec_do_decrypt_auth(struct sk_buff *skb, const struct mac802154_llsec *sec, data = skb_mac_header(skb) + skb->mac_len; datalen = skb_tail_pointer(skb) - data; + if (datalen < authlen) { + kfree_sensitive(req); + return -EBADMSG; + } + sg_init_one(&sg, skb_mac_header(skb), assoclen + datalen); if (!(hdr->sec.level & IEEE802154_SCF_SECLEVEL_ENC)) { diff --git a/net/mac802154/scan.c b/net/mac802154/scan.c index 0eef3b4f5b9b..775809ee0206 100644 --- a/net/mac802154/scan.c +++ b/net/mac802154/scan.c @@ -179,6 +179,7 @@ void mac802154_scan_worker(struct work_struct *work) enum nl802154_scan_types scan_req_type; struct ieee802154_sub_if_data *sdata; unsigned int scan_duration = 0; + netdevice_tracker dev_tracker; struct wpan_phy *wpan_phy; u8 scan_req_duration; u8 page, channel; @@ -209,6 +210,14 @@ void mac802154_scan_worker(struct work_struct *work) return; } + /* + * sdata->dev is dereferenced below after rcu_read_unlock() and outside + * the rtnl, and a concurrent DEL_INTERFACE / PHY teardown can free it + * asynchronously from netdev_run_todo(). Pin it with a reference taken + * while the RCU read lock is still held, and drop it at every exit. + */ + netdev_hold(sdata->dev, &dev_tracker, GFP_ATOMIC); + wpan_phy = scan_req->wpan_phy; scan_req_type = scan_req->type; scan_req_duration = scan_req->duration; @@ -262,12 +271,14 @@ void mac802154_scan_worker(struct work_struct *work) "Scan page %u channel %u for %ums\n", page, channel, jiffies_to_msecs(scan_duration)); queue_delayed_work(local->mac_wq, &local->scan_work, scan_duration); + netdev_put(sdata->dev, &dev_tracker); return; end_scan: rtnl_lock(); mac802154_scan_cleanup_locked(local, sdata, false); rtnl_unlock(); + netdev_put(sdata->dev, &dev_tracker); } int mac802154_trigger_scan_locked(struct ieee802154_sub_if_data *sdata, diff --git a/net/mctp/device.c b/net/mctp/device.c index 04c5570bacff..7861d83aaad2 100644 --- a/net/mctp/device.c +++ b/net/mctp/device.c @@ -536,7 +536,9 @@ int __init mctp_device_init(void) { int err; - register_netdevice_notifier(&mctp_dev_nb); + err = register_netdevice_notifier(&mctp_dev_nb); + if (err) + return err; err = rtnl_af_register(&mctp_af_ops); if (err) diff --git a/net/mpls/af_mpls.c b/net/mpls/af_mpls.c index 1c70cb26e7ba..f7f60025d042 100644 --- a/net/mpls/af_mpls.c +++ b/net/mpls/af_mpls.c @@ -2123,6 +2123,9 @@ static int mpls_valid_fib_dump_req(struct net *net, const struct nlmsghdr *nlh, int ifindex; if (i == RTA_OIF) { + if (!tb[i]) + continue; + ifindex = nla_get_u32(tb[i]); filter->dev = __dev_get_by_index(net, ifindex); if (!filter->dev) diff --git a/net/mptcp/options.c b/net/mptcp/options.c index 845860311ea6..cbece69dc5f0 100644 --- a/net/mptcp/options.c +++ b/net/mptcp/options.c @@ -157,17 +157,11 @@ static void mptcp_parse_option(const struct sk_buff *skb, ptr++; flags = (*ptr++) & MPTCP_DSS_FLAG_MASK; - mp_opt->data_fin = (flags & MPTCP_DSS_DATA_FIN) != 0; mp_opt->dsn64 = (flags & MPTCP_DSS_DSN64) != 0; mp_opt->use_map = (flags & MPTCP_DSS_HAS_MAP) != 0; mp_opt->ack64 = (flags & MPTCP_DSS_ACK64) != 0; mp_opt->use_ack = (flags & MPTCP_DSS_HAS_ACK); - pr_debug("data_fin=%d dsn64=%d use_map=%d ack64=%d use_ack=%d\n", - mp_opt->data_fin, mp_opt->dsn64, - mp_opt->use_map, mp_opt->ack64, - mp_opt->use_ack); - expected_opsize = TCPOLEN_MPTCP_DSS_BASE; if (mp_opt->use_ack) { @@ -178,12 +172,18 @@ static void mptcp_parse_option(const struct sk_buff *skb, } if (mp_opt->use_map) { + mp_opt->data_fin = (flags & MPTCP_DSS_DATA_FIN) != 0; if (mp_opt->dsn64) expected_opsize += TCPOLEN_MPTCP_DSS_MAP64; else expected_opsize += TCPOLEN_MPTCP_DSS_MAP32; } + pr_debug("data_fin=%d dsn64=%d use_map=%d ack64=%d use_ack=%d\n", + mp_opt->data_fin, mp_opt->dsn64, + mp_opt->use_map, mp_opt->ack64, + mp_opt->use_ack); + /* Always parse any csum presence combination, we will enforce * RFC 8684 Section 3.3.0 checks later in subflow_data_ready */ diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c index 0d3a95e676f1..db5693262c8c 100644 --- a/net/mptcp/pm_userspace.c +++ b/net/mptcp/pm_userspace.c @@ -132,12 +132,15 @@ int mptcp_userspace_pm_get_local_id(struct mptcp_sock *msk, __be16 msk_sport = ((struct inet_sock *) inet_sk((struct sock *)msk))->inet_sport; struct mptcp_pm_addr_entry *entry; + int id; spin_lock_bh(&msk->pm.lock); entry = mptcp_userspace_pm_lookup_addr(msk, &skc->addr); + id = entry ? entry->addr.id : -1; spin_unlock_bh(&msk->pm.lock); - if (entry) - return entry->addr.id; + + if (id != -1) + return id; if (skc->addr.port == msk_sport) skc->addr.port = 0; diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 72a909bc0fca..9e381408d58a 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -3758,6 +3758,7 @@ bool mptcp_finish_join(struct sock *ssk) mptcp_data_unlock(parent); if (!ret) { + mptcp_pm_close_subflow(msk); err_prohibited: subflow->reset_reason = MPTCP_RST_EPROHIBIT; return false; diff --git a/net/netfilter/nf_conntrack_sip.c b/net/netfilter/nf_conntrack_sip.c index 81534213f00f..147043f19521 100644 --- a/net/netfilter/nf_conntrack_sip.c +++ b/net/netfilter/nf_conntrack_sip.c @@ -947,10 +947,8 @@ static int set_expected_rtp_rtcp(struct sk_buff *skb, unsigned int protoff, return NF_ACCEPT; saddr = &ct->tuplehash[!dir].tuple.src.u3; } else if (sip_external_media) { - struct net_device *dev = skb_dst(skb)->dev; - struct net *net = dev_net(dev); - struct flowi fl; struct dst_entry *dst = NULL; + struct flowi fl; memset(&fl, 0, sizeof(fl)); @@ -970,7 +968,11 @@ static int set_expected_rtp_rtcp(struct sk_buff *skb, unsigned int protoff, * through the same interface as the signalling peer. */ if (dst) { - bool external_media = (dst->dev == dev); + const struct dst_entry *this_dst = skb_dst(skb); + bool external_media = false; + + if (this_dst && dst->dev == this_dst->dev) + external_media = true; dst_release(dst); if (external_media) diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c index 0842916c19f5..51d3d4d44918 100644 --- a/net/netfilter/nf_tables_api.c +++ b/net/netfilter/nf_tables_api.c @@ -939,58 +939,6 @@ static int nft_delflowtable(struct nft_ctx *ctx, return 0; } -static void __nft_reg_track_clobber(struct nft_regs_track *track, u8 dreg) -{ - int i; - - for (i = track->regs[dreg].num_reg; i > 0; i--) - __nft_reg_track_cancel(track, dreg - i); -} - -static void __nft_reg_track_update(struct nft_regs_track *track, - const struct nft_expr *expr, - u8 dreg, u8 num_reg) -{ - track->regs[dreg].selector = expr; - track->regs[dreg].bitwise = NULL; - track->regs[dreg].num_reg = num_reg; -} - -void nft_reg_track_update(struct nft_regs_track *track, - const struct nft_expr *expr, u8 dreg, u8 len) -{ - unsigned int regcount; - int i; - - __nft_reg_track_clobber(track, dreg); - - regcount = DIV_ROUND_UP(len, NFT_REG32_SIZE); - for (i = 0; i < regcount; i++, dreg++) - __nft_reg_track_update(track, expr, dreg, i); -} -EXPORT_SYMBOL_GPL(nft_reg_track_update); - -void nft_reg_track_cancel(struct nft_regs_track *track, u8 dreg, u8 len) -{ - unsigned int regcount; - int i; - - __nft_reg_track_clobber(track, dreg); - - regcount = DIV_ROUND_UP(len, NFT_REG32_SIZE); - for (i = 0; i < regcount; i++, dreg++) - __nft_reg_track_cancel(track, dreg); -} -EXPORT_SYMBOL_GPL(nft_reg_track_cancel); - -void __nft_reg_track_cancel(struct nft_regs_track *track, u8 dreg) -{ - track->regs[dreg].selector = NULL; - track->regs[dreg].bitwise = NULL; - track->regs[dreg].num_reg = 0; -} -EXPORT_SYMBOL_GPL(__nft_reg_track_cancel); - /* * Tables */ @@ -3908,23 +3856,6 @@ static int nf_tables_dump_rules(struct sk_buff *skb, return skb->len; } -static int nf_tables_dumpreset_rules(struct sk_buff *skb, - struct netlink_callback *cb) -{ - struct nftables_pernet *nft_net = nft_pernet(sock_net(skb->sk)); - int ret; - - /* Mutex is held is to prevent that two concurrent dump-and-reset calls - * do not underrun counters and quotas. The commit_mutex is used for - * the lack a better lock, this is not transaction path. - */ - mutex_lock(&nft_net->commit_mutex); - ret = nf_tables_dump_rules(skb, cb); - mutex_unlock(&nft_net->commit_mutex); - - return ret; -} - static int nf_tables_dump_rules_start(struct netlink_callback *cb) { struct nft_rule_dump_ctx *ctx = (void *)cb->ctx; @@ -3944,16 +3875,10 @@ static int nf_tables_dump_rules_start(struct netlink_callback *cb) return -ENOMEM; } } - return 0; -} - -static int nf_tables_dumpreset_rules_start(struct netlink_callback *cb) -{ - struct nft_rule_dump_ctx *ctx = (void *)cb->ctx; + if (NFNL_MSG_TYPE(cb->nlh->nlmsg_type) == NFT_MSG_GETRULE_RESET) + ctx->reset = true; - ctx->reset = true; - - return nf_tables_dump_rules_start(cb); + return 0; } static int nf_tables_dump_rules_done(struct netlink_callback *cb) @@ -4019,6 +3944,8 @@ static int nf_tables_getrule(struct sk_buff *skb, const struct nfnl_info *info, u32 portid = NETLINK_CB(skb).portid; struct net *net = info->net; struct sk_buff *skb2; + bool reset = false; + char *buf; if (info->nlh->nlmsg_flags & NLM_F_DUMP) { struct netlink_dump_control c = { @@ -4032,47 +3959,16 @@ static int nf_tables_getrule(struct sk_buff *skb, const struct nfnl_info *info, return nft_netlink_dump_start_rcu(info->sk, skb, info->nlh, &c); } - skb2 = nf_tables_getrule_single(portid, info, nla, false); - if (IS_ERR(skb2)) - return PTR_ERR(skb2); - - return nfnetlink_unicast(skb2, net, portid); -} - -static int nf_tables_getrule_reset(struct sk_buff *skb, - const struct nfnl_info *info, - const struct nlattr * const nla[]) -{ - struct nftables_pernet *nft_net = nft_pernet(info->net); - u32 portid = NETLINK_CB(skb).portid; - struct net *net = info->net; - struct sk_buff *skb2; - char *buf; - - if (info->nlh->nlmsg_flags & NLM_F_DUMP) { - struct netlink_dump_control c = { - .start= nf_tables_dumpreset_rules_start, - .dump = nf_tables_dumpreset_rules, - .done = nf_tables_dump_rules_done, - .module = THIS_MODULE, - .data = (void *)nla, - }; - - return nft_netlink_dump_start_rcu(info->sk, skb, info->nlh, &c); - } - - if (!try_module_get(THIS_MODULE)) - return -EINVAL; - rcu_read_unlock(); - mutex_lock(&nft_net->commit_mutex); - skb2 = nf_tables_getrule_single(portid, info, nla, true); - mutex_unlock(&nft_net->commit_mutex); - rcu_read_lock(); - module_put(THIS_MODULE); + if (NFNL_MSG_TYPE(info->nlh->nlmsg_type) == NFT_MSG_GETRULE_RESET) + reset = true; + skb2 = nf_tables_getrule_single(portid, info, nla, reset); if (IS_ERR(skb2)) return PTR_ERR(skb2); + if (!reset) + return nfnetlink_unicast(skb2, net, portid); + buf = kasprintf(GFP_ATOMIC, "%.*s:%u", nla_len(nla[NFTA_RULE_TABLE]), (char *)nla_data(nla[NFTA_RULE_TABLE]), @@ -6332,6 +6228,10 @@ static int nf_tables_dump_set(struct sk_buff *skb, struct netlink_callback *cb) nla_nest_end(skb, nest); nlmsg_end(skb, nlh); + if (dump_ctx->reset && args.iter.count > args.iter.skip) + audit_log_nft_set_reset(table, cb->seq, + args.iter.count - args.iter.skip); + rcu_read_unlock(); if (args.iter.err && args.iter.err != -EMSGSIZE) @@ -6347,26 +6247,6 @@ static int nf_tables_dump_set(struct sk_buff *skb, struct netlink_callback *cb) return -ENOSPC; } -static int nf_tables_dumpreset_set(struct sk_buff *skb, - struct netlink_callback *cb) -{ - struct nftables_pernet *nft_net = nft_pernet(sock_net(skb->sk)); - struct nft_set_dump_ctx *dump_ctx = cb->data; - int ret, skip = cb->args[0]; - - mutex_lock(&nft_net->commit_mutex); - - ret = nf_tables_dump_set(skb, cb); - - if (cb->args[0] > skip) - audit_log_nft_set_reset(dump_ctx->ctx.table, cb->seq, - cb->args[0] - skip); - - mutex_unlock(&nft_net->commit_mutex); - - return ret; -} - static int nf_tables_dump_set_start(struct netlink_callback *cb) { struct nft_set_dump_ctx *dump_ctx = cb->data; @@ -6613,8 +6493,13 @@ static int nf_tables_getsetelem(struct sk_buff *skb, { struct netlink_ext_ack *extack = info->extack; struct nft_set_dump_ctx dump_ctx; + int rem, err = 0, nelems = 0; + struct net *net = info->net; struct nlattr *attr; - int rem, err = 0; + bool reset = false; + + if (NFNL_MSG_TYPE(info->nlh->nlmsg_type) == NFT_MSG_GETSETELEM_RESET) + reset = true; if (info->nlh->nlmsg_flags & NLM_F_DUMP) { struct netlink_dump_control c = { @@ -6624,7 +6509,7 @@ static int nf_tables_getsetelem(struct sk_buff *skb, .module = THIS_MODULE, }; - err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, false); + err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, reset); if (err) return err; @@ -6635,75 +6520,21 @@ static int nf_tables_getsetelem(struct sk_buff *skb, if (!nla[NFTA_SET_ELEM_LIST_ELEMENTS]) return -EINVAL; - err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, false); + err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, reset); if (err) return err; nla_for_each_nested(attr, nla[NFTA_SET_ELEM_LIST_ELEMENTS], rem) { - err = nft_get_set_elem(&dump_ctx.ctx, dump_ctx.set, attr, false); - if (err < 0) { - NL_SET_BAD_ATTR(extack, attr); - break; - } - } - - return err; -} - -static int nf_tables_getsetelem_reset(struct sk_buff *skb, - const struct nfnl_info *info, - const struct nlattr * const nla[]) -{ - struct nftables_pernet *nft_net = nft_pernet(info->net); - struct netlink_ext_ack *extack = info->extack; - struct nft_set_dump_ctx dump_ctx; - int rem, err = 0, nelems = 0; - struct nlattr *attr; - - if (info->nlh->nlmsg_flags & NLM_F_DUMP) { - struct netlink_dump_control c = { - .start = nf_tables_dump_set_start, - .dump = nf_tables_dumpreset_set, - .done = nf_tables_dump_set_done, - .module = THIS_MODULE, - }; - - err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, true); - if (err) - return err; - - c.data = &dump_ctx; - return nft_netlink_dump_start_rcu(info->sk, skb, info->nlh, &c); - } - - if (!nla[NFTA_SET_ELEM_LIST_ELEMENTS]) - return -EINVAL; - - if (!try_module_get(THIS_MODULE)) - return -EINVAL; - rcu_read_unlock(); - mutex_lock(&nft_net->commit_mutex); - rcu_read_lock(); - - err = nft_set_dump_ctx_init(&dump_ctx, skb, info, nla, true); - if (err) - goto out_unlock; - - nla_for_each_nested(attr, nla[NFTA_SET_ELEM_LIST_ELEMENTS], rem) { - err = nft_get_set_elem(&dump_ctx.ctx, dump_ctx.set, attr, true); + err = nft_get_set_elem(&dump_ctx.ctx, dump_ctx.set, attr, reset); if (err < 0) { NL_SET_BAD_ATTR(extack, attr); break; } nelems++; } - audit_log_nft_set_reset(dump_ctx.ctx.table, nft_base_seq(info->net), nelems); - -out_unlock: - rcu_read_unlock(); - mutex_unlock(&nft_net->commit_mutex); - rcu_read_lock(); - module_put(THIS_MODULE); + if (reset) + audit_log_nft_set_reset(dump_ctx.ctx.table, nft_base_seq(net), + nelems); return err; } @@ -8567,19 +8398,6 @@ static int nf_tables_dump_obj(struct sk_buff *skb, struct netlink_callback *cb) return skb->len; } -static int nf_tables_dumpreset_obj(struct sk_buff *skb, - struct netlink_callback *cb) -{ - struct nftables_pernet *nft_net = nft_pernet(sock_net(skb->sk)); - int ret; - - mutex_lock(&nft_net->commit_mutex); - ret = nf_tables_dump_obj(skb, cb); - mutex_unlock(&nft_net->commit_mutex); - - return ret; -} - static int nf_tables_dump_obj_start(struct netlink_callback *cb) { struct nft_obj_dump_ctx *ctx = (void *)cb->ctx; @@ -8596,16 +8414,10 @@ static int nf_tables_dump_obj_start(struct netlink_callback *cb) if (nla[NFTA_OBJ_TYPE]) ctx->type = ntohl(nla_get_be32(nla[NFTA_OBJ_TYPE])); - return 0; -} + if (NFNL_MSG_TYPE(cb->nlh->nlmsg_type) == NFT_MSG_GETOBJ_RESET) + ctx->reset = true; -static int nf_tables_dumpreset_obj_start(struct netlink_callback *cb) -{ - struct nft_obj_dump_ctx *ctx = (void *)cb->ctx; - - ctx->reset = true; - - return nf_tables_dump_obj_start(cb); + return 0; } static int nf_tables_dump_obj_done(struct netlink_callback *cb) @@ -8667,42 +8479,16 @@ nf_tables_getobj_single(u32 portid, const struct nfnl_info *info, static int nf_tables_getobj(struct sk_buff *skb, const struct nfnl_info *info, const struct nlattr * const nla[]) { - u32 portid = NETLINK_CB(skb).portid; - struct sk_buff *skb2; - - if (info->nlh->nlmsg_flags & NLM_F_DUMP) { - struct netlink_dump_control c = { - .start = nf_tables_dump_obj_start, - .dump = nf_tables_dump_obj, - .done = nf_tables_dump_obj_done, - .module = THIS_MODULE, - .data = (void *)nla, - }; - - return nft_netlink_dump_start_rcu(info->sk, skb, info->nlh, &c); - } - - skb2 = nf_tables_getobj_single(portid, info, nla, false); - if (IS_ERR(skb2)) - return PTR_ERR(skb2); - - return nfnetlink_unicast(skb2, info->net, portid); -} - -static int nf_tables_getobj_reset(struct sk_buff *skb, - const struct nfnl_info *info, - const struct nlattr * const nla[]) -{ - struct nftables_pernet *nft_net = nft_pernet(info->net); u32 portid = NETLINK_CB(skb).portid; struct net *net = info->net; struct sk_buff *skb2; + bool reset = false; char *buf; if (info->nlh->nlmsg_flags & NLM_F_DUMP) { struct netlink_dump_control c = { - .start = nf_tables_dumpreset_obj_start, - .dump = nf_tables_dumpreset_obj, + .start = nf_tables_dump_obj_start, + .dump = nf_tables_dump_obj, .done = nf_tables_dump_obj_done, .module = THIS_MODULE, .data = (void *)nla, @@ -8711,18 +8497,16 @@ static int nf_tables_getobj_reset(struct sk_buff *skb, return nft_netlink_dump_start_rcu(info->sk, skb, info->nlh, &c); } - if (!try_module_get(THIS_MODULE)) - return -EINVAL; - rcu_read_unlock(); - mutex_lock(&nft_net->commit_mutex); - skb2 = nf_tables_getobj_single(portid, info, nla, true); - mutex_unlock(&nft_net->commit_mutex); - rcu_read_lock(); - module_put(THIS_MODULE); + if (NFNL_MSG_TYPE(info->nlh->nlmsg_type) == NFT_MSG_GETOBJ_RESET) + reset = true; + skb2 = nf_tables_getobj_single(portid, info, nla, reset); if (IS_ERR(skb2)) return PTR_ERR(skb2); + if (!reset) + return nfnetlink_unicast(skb2, net, NETLINK_CB(skb).portid); + buf = kasprintf(GFP_ATOMIC, "%.*s:%u", nla_len(nla[NFTA_OBJ_TABLE]), (char *)nla_data(nla[NFTA_OBJ_TABLE]), @@ -10030,7 +9814,7 @@ static const struct nfnl_callback nf_tables_cb[NFT_MSG_MAX] = { .policy = nft_rule_policy, }, [NFT_MSG_GETRULE_RESET] = { - .call = nf_tables_getrule_reset, + .call = nf_tables_getrule, .type = NFNL_CB_RCU, .attr_count = NFTA_RULE_MAX, .policy = nft_rule_policy, @@ -10084,7 +9868,7 @@ static const struct nfnl_callback nf_tables_cb[NFT_MSG_MAX] = { .policy = nft_set_elem_list_policy, }, [NFT_MSG_GETSETELEM_RESET] = { - .call = nf_tables_getsetelem_reset, + .call = nf_tables_getsetelem, .type = NFNL_CB_RCU, .attr_count = NFTA_SET_ELEM_LIST_MAX, .policy = nft_set_elem_list_policy, @@ -10130,7 +9914,7 @@ static const struct nfnl_callback nf_tables_cb[NFT_MSG_MAX] = { .policy = nft_obj_policy, }, [NFT_MSG_GETOBJ_RESET] = { - .call = nf_tables_getobj_reset, + .call = nf_tables_getobj, .type = NFNL_CB_RCU, .attr_count = NFTA_OBJ_MAX, .policy = nft_obj_policy, @@ -10337,16 +10121,9 @@ void nf_tables_trans_destroy_flush_work(struct net *net) } EXPORT_SYMBOL_GPL(nf_tables_trans_destroy_flush_work); -static bool nft_expr_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - return false; -} - static int nf_tables_commit_chain_prepare(struct net *net, struct nft_chain *chain) { const struct nft_expr *expr, *last; - struct nft_regs_track track = {}; unsigned int size, data_size; void *data, *data_boundary; struct nft_rule_dp *prule; @@ -10383,15 +10160,7 @@ static int nf_tables_commit_chain_prepare(struct net *net, struct nft_chain *cha return -ENOMEM; size = 0; - track.last = nft_expr_last(rule); nft_rule_for_each_expr(expr, last, rule) { - track.cur = expr; - - if (nft_expr_reduce(&track, expr)) { - expr = track.cur; - continue; - } - if (WARN_ON_ONCE(data + size + expr->ops->size > data_boundary)) return -ENOMEM; diff --git a/net/netfilter/nft_bitwise.c b/net/netfilter/nft_bitwise.c index 1afb36fb5994..e0a6703dc0e4 100644 --- a/net/netfilter/nft_bitwise.c +++ b/net/netfilter/nft_bitwise.c @@ -402,61 +402,12 @@ static int nft_bitwise_offload(struct nft_offload_ctx *ctx, return 0; } -static bool nft_bitwise_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_bitwise *priv = nft_expr_priv(expr); - const struct nft_bitwise *bitwise; - unsigned int regcount; - u8 dreg; - int i; - - if (!track->regs[priv->sreg].selector) - return false; - - bitwise = nft_expr_priv(track->regs[priv->dreg].selector); - if (track->regs[priv->sreg].selector == track->regs[priv->dreg].selector && - track->regs[priv->sreg].num_reg == 0 && - track->regs[priv->dreg].bitwise && - track->regs[priv->dreg].bitwise->ops == expr->ops && - priv->sreg == bitwise->sreg && - priv->sreg2 == bitwise->sreg2 && - priv->dreg == bitwise->dreg && - priv->op == bitwise->op && - priv->len == bitwise->len && - !memcmp(&priv->mask, &bitwise->mask, sizeof(priv->mask)) && - !memcmp(&priv->xor, &bitwise->xor, sizeof(priv->xor)) && - !memcmp(&priv->data, &bitwise->data, sizeof(priv->data))) { - track->cur = expr; - return true; - } - - if (track->regs[priv->sreg].bitwise || - track->regs[priv->sreg].num_reg != 0) { - nft_reg_track_cancel(track, priv->dreg, priv->len); - return false; - } - - if (priv->sreg != priv->dreg) { - nft_reg_track_update(track, track->regs[priv->sreg].selector, - priv->dreg, priv->len); - } - - dreg = priv->dreg; - regcount = DIV_ROUND_UP(priv->len, NFT_REG32_SIZE); - for (i = 0; i < regcount; i++, dreg++) - track->regs[dreg].bitwise = expr; - - return false; -} - static const struct nft_expr_ops nft_bitwise_ops = { .type = &nft_bitwise_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_bitwise)), .eval = nft_bitwise_eval, .init = nft_bitwise_init, .dump = nft_bitwise_dump, - .reduce = nft_bitwise_reduce, .offload = nft_bitwise_offload, }; @@ -559,48 +510,12 @@ static int nft_bitwise_fast_offload(struct nft_offload_ctx *ctx, return 0; } -static bool nft_bitwise_fast_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_bitwise_fast_expr *priv = nft_expr_priv(expr); - const struct nft_bitwise_fast_expr *bitwise; - - if (!track->regs[priv->sreg].selector) - return false; - - bitwise = nft_expr_priv(track->regs[priv->dreg].selector); - if (track->regs[priv->sreg].selector == track->regs[priv->dreg].selector && - track->regs[priv->dreg].bitwise && - track->regs[priv->dreg].bitwise->ops == expr->ops && - priv->sreg == bitwise->sreg && - priv->dreg == bitwise->dreg && - priv->mask == bitwise->mask && - priv->xor == bitwise->xor) { - track->cur = expr; - return true; - } - - if (track->regs[priv->sreg].bitwise) { - nft_reg_track_cancel(track, priv->dreg, NFT_REG32_SIZE); - return false; - } - - if (priv->sreg != priv->dreg) { - track->regs[priv->dreg].selector = - track->regs[priv->sreg].selector; - } - track->regs[priv->dreg].bitwise = expr; - - return false; -} - const struct nft_expr_ops nft_bitwise_fast_ops = { .type = &nft_bitwise_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_bitwise_fast_expr)), .eval = NULL, /* inlined */ .init = nft_bitwise_fast_init, .dump = nft_bitwise_fast_dump, - .reduce = nft_bitwise_fast_reduce, .offload = nft_bitwise_fast_offload, }; @@ -637,22 +552,3 @@ struct nft_expr_type nft_bitwise_type __read_mostly = { .maxattr = NFTA_BITWISE_MAX, .owner = THIS_MODULE, }; - -bool nft_expr_reduce_bitwise(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_expr *last = track->last; - const struct nft_expr *next; - - if (expr == last) - return false; - - next = nft_expr_next(expr); - if (next->ops == &nft_bitwise_ops) - return nft_bitwise_reduce(track, next); - else if (next->ops == &nft_bitwise_fast_ops) - return nft_bitwise_fast_reduce(track, next); - - return false; -} -EXPORT_SYMBOL_GPL(nft_expr_reduce_bitwise); diff --git a/net/netfilter/nft_byteorder.c b/net/netfilter/nft_byteorder.c index 5e7a7841b789..74548b4c114b 100644 --- a/net/netfilter/nft_byteorder.c +++ b/net/netfilter/nft_byteorder.c @@ -177,23 +177,12 @@ static int nft_byteorder_dump(struct sk_buff *skb, return -1; } -static bool nft_byteorder_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - struct nft_byteorder *priv = nft_expr_priv(expr); - - nft_reg_track_cancel(track, priv->dreg, priv->len); - - return false; -} - static const struct nft_expr_ops nft_byteorder_ops = { .type = &nft_byteorder_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_byteorder)), .eval = nft_byteorder_eval, .init = nft_byteorder_init, .dump = nft_byteorder_dump, - .reduce = nft_byteorder_reduce, }; struct nft_expr_type nft_byteorder_type __read_mostly = { diff --git a/net/netfilter/nft_cmp.c b/net/netfilter/nft_cmp.c index 2605f43737bc..b61dc9c3383e 100644 --- a/net/netfilter/nft_cmp.c +++ b/net/netfilter/nft_cmp.c @@ -190,7 +190,6 @@ static const struct nft_expr_ops nft_cmp_ops = { .eval = nft_cmp_eval, .init = nft_cmp_init, .dump = nft_cmp_dump, - .reduce = NFT_REDUCE_READONLY, .offload = nft_cmp_offload, }; @@ -282,7 +281,6 @@ const struct nft_expr_ops nft_cmp_fast_ops = { .eval = NULL, /* inlined */ .init = nft_cmp_fast_init, .dump = nft_cmp_fast_dump, - .reduce = NFT_REDUCE_READONLY, .offload = nft_cmp_fast_offload, }; @@ -376,7 +374,6 @@ const struct nft_expr_ops nft_cmp16_fast_ops = { .eval = NULL, /* inlined */ .init = nft_cmp16_fast_init, .dump = nft_cmp16_fast_dump, - .reduce = NFT_REDUCE_READONLY, .offload = nft_cmp16_fast_offload, }; diff --git a/net/netfilter/nft_compat.c b/net/netfilter/nft_compat.c index a77ffe2fcea7..ba4d304a3014 100644 --- a/net/netfilter/nft_compat.c +++ b/net/netfilter/nft_compat.c @@ -794,14 +794,6 @@ static const struct nfnetlink_subsystem nfnl_compat_subsys = { static struct nft_expr_type nft_match_type; -static bool nft_match_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct xt_match *match = expr->ops->data; - - return strcmp(match->name, "comment") == 0; -} - static const struct nft_expr_ops * nft_match_select_ops(const struct nft_ctx *ctx, const struct nlattr * const tb[]) @@ -844,7 +836,6 @@ nft_match_select_ops(const struct nft_ctx *ctx, ops->dump = nft_match_dump; ops->validate = nft_match_validate; ops->data = match; - ops->reduce = nft_match_reduce; matchsize = NFT_EXPR_SIZE(XT_ALIGN(match->matchsize)); if (matchsize > NFT_MATCH_LARGE_THRESH) { @@ -933,7 +924,6 @@ nft_target_select_ops(const struct nft_ctx *ctx, ops->destroy = nft_target_destroy; ops->dump = nft_target_dump; ops->data = target; - ops->reduce = NFT_REDUCE_READONLY; if (family == NFPROTO_BRIDGE) { ops->eval = nft_target_eval_bridge; diff --git a/net/netfilter/nft_connlimit.c b/net/netfilter/nft_connlimit.c index 714a59485935..99079afcc5ba 100644 --- a/net/netfilter/nft_connlimit.c +++ b/net/netfilter/nft_connlimit.c @@ -247,7 +247,6 @@ static const struct nft_expr_ops nft_connlimit_ops = { .destroy_clone = nft_connlimit_destroy_clone, .dump = nft_connlimit_dump, .gc = nft_connlimit_gc, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_connlimit_type __read_mostly = { diff --git a/net/netfilter/nft_counter.c b/net/netfilter/nft_counter.c index 0d70325280cc..3fa6369790f4 100644 --- a/net/netfilter/nft_counter.c +++ b/net/netfilter/nft_counter.c @@ -32,6 +32,9 @@ struct nft_counter_percpu_priv { static DEFINE_PER_CPU(struct u64_stats_sync, nft_counter_sync); +/* control plane only: sync fetch+reset */ +static DEFINE_SPINLOCK(nft_counter_lock); + static inline void nft_counter_do_eval(struct nft_counter_percpu_priv *priv, struct nft_regs *regs, const struct nft_pktinfo *pkt) @@ -148,13 +151,25 @@ static void nft_counter_fetch(struct nft_counter_percpu_priv *priv, } } +static void nft_counter_fetch_and_reset(struct nft_counter_percpu_priv *priv, + struct nft_counter_tot *total) +{ + spin_lock(&nft_counter_lock); + nft_counter_fetch(priv, total); + nft_counter_reset(priv, total); + spin_unlock(&nft_counter_lock); +} + static int nft_counter_do_dump(struct sk_buff *skb, struct nft_counter_percpu_priv *priv, bool reset) { struct nft_counter_tot total; - nft_counter_fetch(priv, &total); + if (unlikely(reset)) + nft_counter_fetch_and_reset(priv, &total); + else + nft_counter_fetch(priv, &total); if (nla_put_be64(skb, NFTA_COUNTER_BYTES, cpu_to_be64(total.bytes), NFTA_COUNTER_PAD) || @@ -162,9 +177,6 @@ static int nft_counter_do_dump(struct sk_buff *skb, NFTA_COUNTER_PAD)) goto nla_put_failure; - if (reset) - nft_counter_reset(priv, &total); - return 0; nla_put_failure: @@ -301,7 +313,6 @@ static const struct nft_expr_ops nft_counter_ops = { .destroy_clone = nft_counter_destroy, .dump = nft_counter_dump, .clone = nft_counter_clone, - .reduce = NFT_REDUCE_READONLY, .offload = nft_counter_offload, .offload_stats = nft_counter_offload_stats, }; diff --git a/net/netfilter/nft_ct.c b/net/netfilter/nft_ct.c index c776eb38f1db..b29ff555979b 100644 --- a/net/netfilter/nft_ct.c +++ b/net/netfilter/nft_ct.c @@ -698,29 +698,6 @@ static int nft_ct_get_dump(struct sk_buff *skb, return -1; } -static bool nft_ct_get_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_ct *priv = nft_expr_priv(expr); - const struct nft_ct *ct; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - ct = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->key != ct->key) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return nft_expr_reduce_bitwise(track, expr); -} - static int nft_ct_set_dump(struct sk_buff *skb, const struct nft_expr *expr, bool reset) { @@ -755,27 +732,8 @@ static const struct nft_expr_ops nft_ct_get_ops = { .init = nft_ct_get_init, .destroy = nft_ct_get_destroy, .dump = nft_ct_get_dump, - .reduce = nft_ct_get_reduce, }; -static bool nft_ct_set_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - int i; - - for (i = 0; i < NFT_REG32_NUM; i++) { - if (!track->regs[i].selector) - continue; - - if (track->regs[i].selector->ops != &nft_ct_get_ops) - continue; - - __nft_reg_track_cancel(track, i); - } - - return false; -} - #ifdef CONFIG_MITIGATION_RETPOLINE static const struct nft_expr_ops nft_ct_get_fast_ops = { .type = &nft_ct_type, @@ -784,7 +742,6 @@ static const struct nft_expr_ops nft_ct_get_fast_ops = { .init = nft_ct_get_init, .destroy = nft_ct_get_destroy, .dump = nft_ct_get_dump, - .reduce = nft_ct_set_reduce, }; #endif @@ -795,7 +752,6 @@ static const struct nft_expr_ops nft_ct_set_ops = { .init = nft_ct_set_init, .destroy = nft_ct_set_destroy, .dump = nft_ct_set_dump, - .reduce = nft_ct_set_reduce, }; #ifdef CONFIG_NF_CONNTRACK_ZONES @@ -806,7 +762,6 @@ static const struct nft_expr_ops nft_ct_set_zone_ops = { .init = nft_ct_set_init, .destroy = nft_ct_set_destroy, .dump = nft_ct_set_dump, - .reduce = nft_ct_set_reduce, }; #endif @@ -876,7 +831,6 @@ static const struct nft_expr_ops nft_notrack_ops = { .type = &nft_notrack_type, .size = NFT_EXPR_SIZE(0), .eval = nft_notrack_eval, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_notrack_type __read_mostly = { diff --git a/net/netfilter/nft_dup_netdev.c b/net/netfilter/nft_dup_netdev.c index 0573f96ce079..06866799e946 100644 --- a/net/netfilter/nft_dup_netdev.c +++ b/net/netfilter/nft_dup_netdev.c @@ -80,7 +80,6 @@ static const struct nft_expr_ops nft_dup_netdev_ops = { .eval = nft_dup_netdev_eval, .init = nft_dup_netdev_init, .dump = nft_dup_netdev_dump, - .reduce = NFT_REDUCE_READONLY, .offload = nft_dup_netdev_offload, .offload_action = nft_dup_netdev_offload_action, }; diff --git a/net/netfilter/nft_dynset.c b/net/netfilter/nft_dynset.c index 9123277be03c..57bf94ae8724 100644 --- a/net/netfilter/nft_dynset.c +++ b/net/netfilter/nft_dynset.c @@ -429,7 +429,6 @@ static const struct nft_expr_ops nft_dynset_ops = { .activate = nft_dynset_activate, .deactivate = nft_dynset_deactivate, .dump = nft_dynset_dump, - .reduce = NFT_REDUCE_READONLY, }; struct nft_expr_type nft_dynset_type __read_mostly = { diff --git a/net/netfilter/nft_exthdr.c b/net/netfilter/nft_exthdr.c index 9471328802d3..cee93149dca7 100644 --- a/net/netfilter/nft_exthdr.c +++ b/net/netfilter/nft_exthdr.c @@ -705,40 +705,12 @@ static int nft_exthdr_dump_strip(struct sk_buff *skb, return nft_exthdr_dump_common(skb, priv); } -static bool nft_exthdr_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_exthdr *priv = nft_expr_priv(expr); - const struct nft_exthdr *exthdr; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - exthdr = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->type != exthdr->type || - priv->op != exthdr->op || - priv->flags != exthdr->flags || - priv->offset != exthdr->offset || - priv->len != exthdr->len) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return nft_expr_reduce_bitwise(track, expr); -} - static const struct nft_expr_ops nft_exthdr_ipv6_ops = { .type = &nft_exthdr_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_exthdr)), .eval = nft_exthdr_ipv6_eval, .init = nft_exthdr_init, .dump = nft_exthdr_dump, - .reduce = nft_exthdr_reduce, }; static const struct nft_expr_ops nft_exthdr_ipv4_ops = { @@ -747,7 +719,6 @@ static const struct nft_expr_ops nft_exthdr_ipv4_ops = { .eval = nft_exthdr_ipv4_eval, .init = nft_exthdr_ipv4_init, .dump = nft_exthdr_dump, - .reduce = nft_exthdr_reduce, }; static const struct nft_expr_ops nft_exthdr_tcp_ops = { @@ -756,7 +727,6 @@ static const struct nft_expr_ops nft_exthdr_tcp_ops = { .eval = nft_exthdr_tcp_eval, .init = nft_exthdr_init, .dump = nft_exthdr_dump, - .reduce = nft_exthdr_reduce, }; static const struct nft_expr_ops nft_exthdr_tcp_set_ops = { @@ -765,7 +735,6 @@ static const struct nft_expr_ops nft_exthdr_tcp_set_ops = { .eval = nft_exthdr_tcp_set_eval, .init = nft_exthdr_tcp_set_init, .dump = nft_exthdr_dump_set, - .reduce = NFT_REDUCE_READONLY, }; static const struct nft_expr_ops nft_exthdr_tcp_strip_ops = { @@ -774,7 +743,6 @@ static const struct nft_expr_ops nft_exthdr_tcp_strip_ops = { .eval = nft_exthdr_tcp_strip_eval, .init = nft_exthdr_tcp_strip_init, .dump = nft_exthdr_dump_strip, - .reduce = NFT_REDUCE_READONLY, }; static const struct nft_expr_ops nft_exthdr_sctp_ops = { @@ -783,7 +751,6 @@ static const struct nft_expr_ops nft_exthdr_sctp_ops = { .eval = nft_exthdr_sctp_eval, .init = nft_exthdr_init, .dump = nft_exthdr_dump, - .reduce = nft_exthdr_reduce, }; #ifdef CONFIG_NFT_EXTHDR_DCCP @@ -793,7 +760,6 @@ static const struct nft_expr_ops nft_exthdr_dccp_ops = { .eval = nft_exthdr_dccp_eval, .init = nft_exthdr_dccp_init, .dump = nft_exthdr_dump, - .reduce = nft_exthdr_reduce, }; #endif diff --git a/net/netfilter/nft_fib.c b/net/netfilter/nft_fib.c index 22846136c754..7b2a0a031c4b 100644 --- a/net/netfilter/nft_fib.c +++ b/net/netfilter/nft_fib.c @@ -31,6 +31,15 @@ int nft_fib_validate(const struct nft_ctx *ctx, const struct nft_expr *expr) const struct nft_fib *priv = nft_expr_priv(expr); unsigned int hooks; + switch (ctx->family) { + case NFPROTO_IPV4: + case NFPROTO_IPV6: + case NFPROTO_INET: + break; + default: + return -EOPNOTSUPP; + } + switch (priv->result) { case NFT_FIB_RESULT_OIF: case NFT_FIB_RESULT_OIFNAME: @@ -168,48 +177,6 @@ void nft_fib_store_result(void *reg, const struct nft_fib *priv, } EXPORT_SYMBOL_GPL(nft_fib_store_result); -bool nft_fib_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_fib *priv = nft_expr_priv(expr); - unsigned int len = NFT_REG32_SIZE; - const struct nft_fib *fib; - - switch (priv->result) { - case NFT_FIB_RESULT_OIF: - break; - case NFT_FIB_RESULT_OIFNAME: - if (priv->flags & NFTA_FIB_F_PRESENT) - len = NFT_REG32_SIZE; - else - len = IFNAMSIZ; - break; - case NFT_FIB_RESULT_ADDRTYPE: - break; - default: - WARN_ON_ONCE(1); - break; - } - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, len); - return false; - } - - fib = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->result != fib->result || - priv->flags != fib->flags) { - nft_reg_track_update(track, expr, priv->dreg, len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return false; -} -EXPORT_SYMBOL_GPL(nft_fib_reduce); - MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Query routing table from nftables"); MODULE_AUTHOR("Florian Westphal "); diff --git a/net/netfilter/nft_fib_inet.c b/net/netfilter/nft_fib_inet.c index 666a3741d20b..a88d44e163d1 100644 --- a/net/netfilter/nft_fib_inet.c +++ b/net/netfilter/nft_fib_inet.c @@ -49,7 +49,6 @@ static const struct nft_expr_ops nft_fib_inet_ops = { .init = nft_fib_init, .dump = nft_fib_dump, .validate = nft_fib_validate, - .reduce = nft_fib_reduce, }; static struct nft_expr_type nft_fib_inet_type __read_mostly = { diff --git a/net/netfilter/nft_fib_netdev.c b/net/netfilter/nft_fib_netdev.c index 9121ec64e918..5774a7544027 100644 --- a/net/netfilter/nft_fib_netdev.c +++ b/net/netfilter/nft_fib_netdev.c @@ -50,6 +50,33 @@ static void nft_fib_netdev_eval(const struct nft_expr *expr, regs->verdict.code = NFT_BREAK; } +static int nft_fib_netdev_validate(const struct nft_ctx *ctx, + const struct nft_expr *expr) +{ + const struct nft_fib *priv = nft_expr_priv(expr); + unsigned int hooks; + + switch (priv->result) { + case NFT_FIB_RESULT_OIF: + case NFT_FIB_RESULT_OIFNAME: + hooks = (1 << NF_NETDEV_INGRESS); + break; + case NFT_FIB_RESULT_ADDRTYPE: + if (priv->flags & NFTA_FIB_F_IIF) + hooks = (1 << NF_NETDEV_INGRESS); + else if (priv->flags & NFTA_FIB_F_OIF) + hooks = (1 << NF_NETDEV_EGRESS); + else + hooks = (1 << NF_NETDEV_INGRESS) | + (1 << NF_NETDEV_EGRESS); + break; + default: + return -EINVAL; + } + + return nft_chain_validate_hooks(ctx->chain, hooks); +} + static struct nft_expr_type nft_fib_netdev_type; static const struct nft_expr_ops nft_fib_netdev_ops = { .type = &nft_fib_netdev_type, @@ -57,8 +84,7 @@ static const struct nft_expr_ops nft_fib_netdev_ops = { .eval = nft_fib_netdev_eval, .init = nft_fib_init, .dump = nft_fib_dump, - .validate = nft_fib_validate, - .reduce = nft_fib_reduce, + .validate = nft_fib_netdev_validate, }; static struct nft_expr_type nft_fib_netdev_type __read_mostly = { diff --git a/net/netfilter/nft_flow_offload.c b/net/netfilter/nft_flow_offload.c index b8f76c9057fd..3e1d6c6295b9 100644 --- a/net/netfilter/nft_flow_offload.c +++ b/net/netfilter/nft_flow_offload.c @@ -224,7 +224,6 @@ static const struct nft_expr_ops nft_flow_offload_ops = { .destroy = nft_flow_offload_destroy, .validate = nft_flow_offload_validate, .dump = nft_flow_offload_dump, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_flow_offload_type __read_mostly = { diff --git a/net/netfilter/nft_fwd_netdev.c b/net/netfilter/nft_fwd_netdev.c index 256e832f1bb9..4bce36c3a6a0 100644 --- a/net/netfilter/nft_fwd_netdev.c +++ b/net/netfilter/nft_fwd_netdev.c @@ -228,7 +228,6 @@ static const struct nft_expr_ops nft_fwd_neigh_netdev_ops = { .init = nft_fwd_neigh_init, .dump = nft_fwd_neigh_dump, .validate = nft_fwd_validate, - .reduce = NFT_REDUCE_READONLY, }; static const struct nft_expr_ops nft_fwd_netdev_ops = { @@ -238,7 +237,6 @@ static const struct nft_expr_ops nft_fwd_netdev_ops = { .init = nft_fwd_netdev_init, .dump = nft_fwd_netdev_dump, .validate = nft_fwd_validate, - .reduce = NFT_REDUCE_READONLY, .offload = nft_fwd_netdev_offload, .offload_action = nft_fwd_netdev_offload_action, }; diff --git a/net/netfilter/nft_hash.c b/net/netfilter/nft_hash.c index 5d034bbb6913..1cf41e0a0e0c 100644 --- a/net/netfilter/nft_hash.c +++ b/net/netfilter/nft_hash.c @@ -166,16 +166,6 @@ static int nft_jhash_dump(struct sk_buff *skb, return -1; } -static bool nft_jhash_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_jhash *priv = nft_expr_priv(expr); - - nft_reg_track_cancel(track, priv->dreg, sizeof(u32)); - - return false; -} - static int nft_symhash_dump(struct sk_buff *skb, const struct nft_expr *expr, bool reset) { @@ -196,30 +186,6 @@ static int nft_symhash_dump(struct sk_buff *skb, return -1; } -static bool nft_symhash_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - struct nft_symhash *priv = nft_expr_priv(expr); - struct nft_symhash *symhash; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, sizeof(u32)); - return false; - } - - symhash = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->offset != symhash->offset || - priv->modulus != symhash->modulus) { - nft_reg_track_update(track, expr, priv->dreg, sizeof(u32)); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return false; -} - static struct nft_expr_type nft_hash_type; static const struct nft_expr_ops nft_jhash_ops = { .type = &nft_hash_type, @@ -227,7 +193,6 @@ static const struct nft_expr_ops nft_jhash_ops = { .eval = nft_jhash_eval, .init = nft_jhash_init, .dump = nft_jhash_dump, - .reduce = nft_jhash_reduce, }; static const struct nft_expr_ops nft_symhash_ops = { @@ -236,7 +201,6 @@ static const struct nft_expr_ops nft_symhash_ops = { .eval = nft_symhash_eval, .init = nft_symhash_init, .dump = nft_symhash_dump, - .reduce = nft_symhash_reduce, }; static const struct nft_expr_ops * diff --git a/net/netfilter/nft_immediate.c b/net/netfilter/nft_immediate.c index 02ee5fb69871..37c29947b380 100644 --- a/net/netfilter/nft_immediate.c +++ b/net/netfilter/nft_immediate.c @@ -320,17 +320,6 @@ static bool nft_immediate_offload_action(const struct nft_expr *expr) return false; } -static bool nft_immediate_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_immediate_expr *priv = nft_expr_priv(expr); - - if (priv->dreg != NFT_REG_VERDICT) - nft_reg_track_cancel(track, priv->dreg, priv->dlen); - - return false; -} - static const struct nft_expr_ops nft_imm_ops = { .type = &nft_imm_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_immediate_expr)), @@ -341,7 +330,6 @@ static const struct nft_expr_ops nft_imm_ops = { .destroy = nft_immediate_destroy, .dump = nft_immediate_dump, .validate = nft_immediate_validate, - .reduce = nft_immediate_reduce, .offload = nft_immediate_offload, .offload_action = nft_immediate_offload_action, }; diff --git a/net/netfilter/nft_last.c b/net/netfilter/nft_last.c index de1b6066bfa8..e845779268d3 100644 --- a/net/netfilter/nft_last.c +++ b/net/netfilter/nft_last.c @@ -125,7 +125,6 @@ static const struct nft_expr_ops nft_last_ops = { .destroy = nft_last_destroy, .clone = nft_last_clone, .dump = nft_last_dump, - .reduce = NFT_REDUCE_READONLY, }; struct nft_expr_type nft_last_type __read_mostly = { diff --git a/net/netfilter/nft_limit.c b/net/netfilter/nft_limit.c index 21d26b79b460..0daeb0b23c20 100644 --- a/net/netfilter/nft_limit.c +++ b/net/netfilter/nft_limit.c @@ -243,7 +243,6 @@ static const struct nft_expr_ops nft_limit_pkts_ops = { .destroy = nft_limit_pkts_destroy, .clone = nft_limit_pkts_clone, .dump = nft_limit_pkts_dump, - .reduce = NFT_REDUCE_READONLY, }; static void nft_limit_bytes_eval(const struct nft_expr *expr, @@ -299,7 +298,6 @@ static const struct nft_expr_ops nft_limit_bytes_ops = { .dump = nft_limit_bytes_dump, .clone = nft_limit_bytes_clone, .destroy = nft_limit_bytes_destroy, - .reduce = NFT_REDUCE_READONLY, }; static const struct nft_expr_ops * diff --git a/net/netfilter/nft_log.c b/net/netfilter/nft_log.c index e35588137995..575021f6ee33 100644 --- a/net/netfilter/nft_log.c +++ b/net/netfilter/nft_log.c @@ -291,7 +291,6 @@ static const struct nft_expr_ops nft_log_ops = { .init = nft_log_init, .destroy = nft_log_destroy, .dump = nft_log_dump, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_log_type __read_mostly = { diff --git a/net/netfilter/nft_lookup.c b/net/netfilter/nft_lookup.c index beca9c7705e0..699254cb3ecd 100644 --- a/net/netfilter/nft_lookup.c +++ b/net/netfilter/nft_lookup.c @@ -269,17 +269,6 @@ static int nft_lookup_validate(const struct nft_ctx *ctx, return 0; } -static bool nft_lookup_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_lookup *priv = nft_expr_priv(expr); - - if (priv->set->flags & NFT_SET_MAP) - nft_reg_track_cancel(track, priv->dreg, priv->set->dlen); - - return false; -} - static const struct nft_expr_ops nft_lookup_ops = { .type = &nft_lookup_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_lookup)), @@ -290,7 +279,6 @@ static const struct nft_expr_ops nft_lookup_ops = { .destroy = nft_lookup_destroy, .dump = nft_lookup_dump, .validate = nft_lookup_validate, - .reduce = nft_lookup_reduce, }; struct nft_expr_type nft_lookup_type __read_mostly = { diff --git a/net/netfilter/nft_masq.c b/net/netfilter/nft_masq.c index 868bd4d73555..2b01128737a3 100644 --- a/net/netfilter/nft_masq.c +++ b/net/netfilter/nft_masq.c @@ -143,7 +143,6 @@ static const struct nft_expr_ops nft_masq_ipv4_ops = { .destroy = nft_masq_ipv4_destroy, .dump = nft_masq_dump, .validate = nft_masq_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_masq_ipv4_type __read_mostly = { @@ -171,7 +170,6 @@ static const struct nft_expr_ops nft_masq_ipv6_ops = { .destroy = nft_masq_ipv6_destroy, .dump = nft_masq_dump, .validate = nft_masq_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_masq_ipv6_type __read_mostly = { @@ -213,7 +211,6 @@ static const struct nft_expr_ops nft_masq_inet_ops = { .destroy = nft_masq_inet_destroy, .dump = nft_masq_dump, .validate = nft_masq_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_masq_inet_type __read_mostly = { diff --git a/net/netfilter/nft_meta.c b/net/netfilter/nft_meta.c index d3b0efec036a..19e6d1c2436a 100644 --- a/net/netfilter/nft_meta.c +++ b/net/netfilter/nft_meta.c @@ -743,60 +743,16 @@ static int nft_meta_get_offload(struct nft_offload_ctx *ctx, return 0; } -bool nft_meta_get_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_meta *priv = nft_expr_priv(expr); - const struct nft_meta *meta; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - meta = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->key != meta->key || - priv->dreg != meta->dreg) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return nft_expr_reduce_bitwise(track, expr); -} -EXPORT_SYMBOL_GPL(nft_meta_get_reduce); - static const struct nft_expr_ops nft_meta_get_ops = { .type = &nft_meta_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_meta)), .eval = nft_meta_get_eval, .init = nft_meta_get_init, .dump = nft_meta_get_dump, - .reduce = nft_meta_get_reduce, .validate = nft_meta_get_validate, .offload = nft_meta_get_offload, }; -static bool nft_meta_set_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - int i; - - for (i = 0; i < NFT_REG32_NUM; i++) { - if (!track->regs[i].selector) - continue; - - if (track->regs[i].selector->ops != &nft_meta_get_ops) - continue; - - __nft_reg_track_cancel(track, i); - } - - return false; -} - static const struct nft_expr_ops nft_meta_set_ops = { .type = &nft_meta_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_meta)), @@ -804,7 +760,6 @@ static const struct nft_expr_ops nft_meta_set_ops = { .init = nft_meta_set_init, .destroy = nft_meta_set_destroy, .dump = nft_meta_set_dump, - .reduce = nft_meta_set_reduce, .validate = nft_meta_set_validate, }; diff --git a/net/netfilter/nft_nat.c b/net/netfilter/nft_nat.c index 6e21f72c5b57..e32cd9fbc7c2 100644 --- a/net/netfilter/nft_nat.c +++ b/net/netfilter/nft_nat.c @@ -320,7 +320,6 @@ static const struct nft_expr_ops nft_nat_ops = { .destroy = nft_nat_destroy, .dump = nft_nat_dump, .validate = nft_nat_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_nat_type __read_mostly = { @@ -351,7 +350,6 @@ static const struct nft_expr_ops nft_nat_inet_ops = { .destroy = nft_nat_destroy, .dump = nft_nat_dump, .validate = nft_nat_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_inet_nat_type __read_mostly = { diff --git a/net/netfilter/nft_numgen.c b/net/netfilter/nft_numgen.c index bd058babfc82..06e87dfd76e7 100644 --- a/net/netfilter/nft_numgen.c +++ b/net/netfilter/nft_numgen.c @@ -84,16 +84,6 @@ static int nft_ng_inc_init(const struct nft_ctx *ctx, return err; } -static bool nft_ng_inc_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_ng_inc *priv = nft_expr_priv(expr); - - nft_reg_track_cancel(track, priv->dreg, NFT_REG32_SIZE); - - return false; -} - static int nft_ng_dump(struct sk_buff *skb, enum nft_registers dreg, u32 modulus, enum nft_ng_types type, u32 offset) { @@ -178,16 +168,6 @@ static int nft_ng_random_dump(struct sk_buff *skb, priv->offset); } -static bool nft_ng_random_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_ng_random *priv = nft_expr_priv(expr); - - nft_reg_track_cancel(track, priv->dreg, NFT_REG32_SIZE); - - return false; -} - static struct nft_expr_type nft_ng_type; static const struct nft_expr_ops nft_ng_inc_ops = { .type = &nft_ng_type, @@ -196,7 +176,6 @@ static const struct nft_expr_ops nft_ng_inc_ops = { .init = nft_ng_inc_init, .destroy = nft_ng_inc_destroy, .dump = nft_ng_inc_dump, - .reduce = nft_ng_inc_reduce, }; static const struct nft_expr_ops nft_ng_random_ops = { @@ -205,7 +184,6 @@ static const struct nft_expr_ops nft_ng_random_ops = { .eval = nft_ng_random_eval, .init = nft_ng_random_init, .dump = nft_ng_random_dump, - .reduce = nft_ng_random_reduce, }; static const struct nft_expr_ops * diff --git a/net/netfilter/nft_objref.c b/net/netfilter/nft_objref.c index 1a62e384766a..633cce69568f 100644 --- a/net/netfilter/nft_objref.c +++ b/net/netfilter/nft_objref.c @@ -123,7 +123,6 @@ static const struct nft_expr_ops nft_objref_ops = { .deactivate = nft_objref_deactivate, .dump = nft_objref_dump, .validate = nft_objref_validate, - .reduce = NFT_REDUCE_READONLY, }; struct nft_objref_map { @@ -245,7 +244,6 @@ static const struct nft_expr_ops nft_objref_map_ops = { .destroy = nft_objref_map_destroy, .dump = nft_objref_map_dump, .validate = nft_objref_map_validate, - .reduce = NFT_REDUCE_READONLY, }; static const struct nft_expr_ops * diff --git a/net/netfilter/nft_osf.c b/net/netfilter/nft_osf.c index bdc2f6c90e2f..8d84ef3fb487 100644 --- a/net/netfilter/nft_osf.c +++ b/net/netfilter/nft_osf.c @@ -131,30 +131,6 @@ static int nft_osf_validate(const struct nft_ctx *ctx, return nft_chain_validate_hooks(ctx->chain, hooks); } -static bool nft_osf_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - struct nft_osf *priv = nft_expr_priv(expr); - struct nft_osf *osf; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, NFT_OSF_MAXGENRELEN); - return false; - } - - osf = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->flags != osf->flags || - priv->ttl != osf->ttl) { - nft_reg_track_update(track, expr, priv->dreg, NFT_OSF_MAXGENRELEN); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return false; -} - static struct nft_expr_type nft_osf_type; static const struct nft_expr_ops nft_osf_op = { .eval = nft_osf_eval, @@ -163,7 +139,6 @@ static const struct nft_expr_ops nft_osf_op = { .dump = nft_osf_dump, .type = &nft_osf_type, .validate = nft_osf_validate, - .reduce = nft_osf_reduce, }; static struct nft_expr_type nft_osf_type __read_mostly = { diff --git a/net/netfilter/nft_payload.c b/net/netfilter/nft_payload.c index da1c6bf0dc57..86a21c2b954f 100644 --- a/net/netfilter/nft_payload.c +++ b/net/netfilter/nft_payload.c @@ -256,31 +256,6 @@ static int nft_payload_dump(struct sk_buff *skb, return -1; } -static bool nft_payload_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_payload *priv = nft_expr_priv(expr); - const struct nft_payload *payload; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - payload = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->base != payload->base || - priv->offset != payload->offset || - priv->len != payload->len) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return nft_expr_reduce_bitwise(track, expr); -} - static bool nft_payload_offload_mask(struct nft_offload_reg *reg, u32 priv_len, u32 field_len) { @@ -584,7 +559,6 @@ static const struct nft_expr_ops nft_payload_ops = { .eval = nft_payload_eval, .init = nft_payload_init, .dump = nft_payload_dump, - .reduce = nft_payload_reduce, .offload = nft_payload_offload, }; @@ -594,7 +568,6 @@ const struct nft_expr_ops nft_payload_fast_ops = { .eval = nft_payload_eval, .init = nft_payload_init, .dump = nft_payload_dump, - .reduce = nft_payload_reduce, .offload = nft_payload_offload, }; @@ -1022,32 +995,12 @@ static int nft_payload_set_dump(struct sk_buff *skb, return -1; } -static bool nft_payload_set_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - int i; - - for (i = 0; i < NFT_REG32_NUM; i++) { - if (!track->regs[i].selector) - continue; - - if (track->regs[i].selector->ops != &nft_payload_ops && - track->regs[i].selector->ops != &nft_payload_fast_ops) - continue; - - __nft_reg_track_cancel(track, i); - } - - return false; -} - static const struct nft_expr_ops nft_payload_set_ops = { .type = &nft_payload_type, .size = NFT_EXPR_SIZE(sizeof(struct nft_payload_set)), .eval = nft_payload_set_eval, .init = nft_payload_set_init, .dump = nft_payload_set_dump, - .reduce = nft_payload_set_reduce, }; static const struct nft_expr_ops * diff --git a/net/netfilter/nft_queue.c b/net/netfilter/nft_queue.c index 344fe311878f..8eb13a02942e 100644 --- a/net/netfilter/nft_queue.c +++ b/net/netfilter/nft_queue.c @@ -191,7 +191,6 @@ static const struct nft_expr_ops nft_queue_ops = { .init = nft_queue_init, .dump = nft_queue_dump, .validate = nft_queue_validate, - .reduce = NFT_REDUCE_READONLY, }; static const struct nft_expr_ops nft_queue_sreg_ops = { @@ -201,7 +200,6 @@ static const struct nft_expr_ops nft_queue_sreg_ops = { .init = nft_queue_sreg_init, .dump = nft_queue_sreg_dump, .validate = nft_queue_validate, - .reduce = NFT_REDUCE_READONLY, }; static const struct nft_expr_ops * diff --git a/net/netfilter/nft_quota.c b/net/netfilter/nft_quota.c index df0798da2329..bb3cf3d16e79 100644 --- a/net/netfilter/nft_quota.c +++ b/net/netfilter/nft_quota.c @@ -140,11 +140,16 @@ static int nft_quota_do_dump(struct sk_buff *skb, struct nft_quota *priv, u64 consumed, consumed_cap, quota; u32 flags = priv->flags; - /* Since we inconditionally increment consumed quota for each packet + /* Since we unconditionally increment consumed quota for each packet * that we see, don't go over the quota boundary in what we send to * userspace. */ - consumed = atomic64_read(priv->consumed); + if (reset) { + consumed = atomic64_xchg(priv->consumed, 0); + clear_bit(NFT_QUOTA_DEPLETED_BIT, &priv->flags); + } else { + consumed = atomic64_read(priv->consumed); + } quota = atomic64_read(&priv->quota); if (consumed >= quota) { consumed_cap = quota; @@ -160,10 +165,6 @@ static int nft_quota_do_dump(struct sk_buff *skb, struct nft_quota *priv, nla_put_be32(skb, NFTA_QUOTA_FLAGS, htonl(flags))) goto nla_put_failure; - if (reset) { - atomic64_sub(consumed, priv->consumed); - clear_bit(NFT_QUOTA_DEPLETED_BIT, &priv->flags); - } return 0; nla_put_failure: @@ -265,7 +266,6 @@ static const struct nft_expr_ops nft_quota_ops = { .destroy = nft_quota_destroy, .clone = nft_quota_clone, .dump = nft_quota_dump, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_quota_type __read_mostly = { diff --git a/net/netfilter/nft_range.c b/net/netfilter/nft_range.c index ea382f7bbd78..cbb02644b836 100644 --- a/net/netfilter/nft_range.c +++ b/net/netfilter/nft_range.c @@ -138,7 +138,6 @@ static const struct nft_expr_ops nft_range_ops = { .eval = nft_range_eval, .init = nft_range_init, .dump = nft_range_dump, - .reduce = NFT_REDUCE_READONLY, }; struct nft_expr_type nft_range_type __read_mostly = { diff --git a/net/netfilter/nft_redir.c b/net/netfilter/nft_redir.c index 95eedad85c83..58ae802db8f5 100644 --- a/net/netfilter/nft_redir.c +++ b/net/netfilter/nft_redir.c @@ -146,7 +146,6 @@ static const struct nft_expr_ops nft_redir_ipv4_ops = { .destroy = nft_redir_ipv4_destroy, .dump = nft_redir_dump, .validate = nft_redir_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_redir_ipv4_type __read_mostly = { @@ -174,7 +173,6 @@ static const struct nft_expr_ops nft_redir_ipv6_ops = { .destroy = nft_redir_ipv6_destroy, .dump = nft_redir_dump, .validate = nft_redir_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_redir_ipv6_type __read_mostly = { @@ -203,7 +201,6 @@ static const struct nft_expr_ops nft_redir_inet_ops = { .destroy = nft_redir_inet_destroy, .dump = nft_redir_dump, .validate = nft_redir_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_redir_inet_type __read_mostly = { diff --git a/net/netfilter/nft_reject_inet.c b/net/netfilter/nft_reject_inet.c index 49020e67304a..dcae83ddc32e 100644 --- a/net/netfilter/nft_reject_inet.c +++ b/net/netfilter/nft_reject_inet.c @@ -79,7 +79,6 @@ static const struct nft_expr_ops nft_reject_inet_ops = { .init = nft_reject_init, .dump = nft_reject_dump, .validate = nft_reject_inet_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_reject_inet_type __read_mostly = { diff --git a/net/netfilter/nft_reject_netdev.c b/net/netfilter/nft_reject_netdev.c index 2558ce1505d9..b53e81e4ca75 100644 --- a/net/netfilter/nft_reject_netdev.c +++ b/net/netfilter/nft_reject_netdev.c @@ -158,7 +158,6 @@ static const struct nft_expr_ops nft_reject_netdev_ops = { .init = nft_reject_init, .dump = nft_reject_dump, .validate = nft_reject_netdev_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_reject_netdev_type __read_mostly = { diff --git a/net/netfilter/nft_rt.c b/net/netfilter/nft_rt.c index dc50b9a5bd68..ad527f3596c0 100644 --- a/net/netfilter/nft_rt.c +++ b/net/netfilter/nft_rt.c @@ -195,7 +195,6 @@ static const struct nft_expr_ops nft_rt_get_ops = { .init = nft_rt_get_init, .dump = nft_rt_get_dump, .validate = nft_rt_validate, - .reduce = NFT_REDUCE_READONLY, }; struct nft_expr_type nft_rt_type __read_mostly = { diff --git a/net/netfilter/nft_socket.c b/net/netfilter/nft_socket.c index 36affbb697c2..c55a1310226a 100644 --- a/net/netfilter/nft_socket.c +++ b/net/netfilter/nft_socket.c @@ -249,31 +249,6 @@ static int nft_socket_dump(struct sk_buff *skb, return 0; } -static bool nft_socket_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_socket *priv = nft_expr_priv(expr); - const struct nft_socket *socket; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - socket = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->key != socket->key || - priv->dreg != socket->dreg || - priv->level != socket->level) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return nft_expr_reduce_bitwise(track, expr); -} - static int nft_socket_validate(const struct nft_ctx *ctx, const struct nft_expr *expr) { @@ -296,7 +271,6 @@ static const struct nft_expr_ops nft_socket_ops = { .init = nft_socket_init, .dump = nft_socket_dump, .validate = nft_socket_validate, - .reduce = nft_socket_reduce, }; static struct nft_expr_type nft_socket_type __read_mostly = { diff --git a/net/netfilter/nft_synproxy.c b/net/netfilter/nft_synproxy.c index ad3e7d27f02e..a80bdce38546 100644 --- a/net/netfilter/nft_synproxy.c +++ b/net/netfilter/nft_synproxy.c @@ -290,7 +290,6 @@ static const struct nft_expr_ops nft_synproxy_ops = { .dump = nft_synproxy_dump, .type = &nft_synproxy_type, .validate = nft_synproxy_validate, - .reduce = NFT_REDUCE_READONLY, }; static struct nft_expr_type nft_synproxy_type __read_mostly = { diff --git a/net/netfilter/nft_tproxy.c b/net/netfilter/nft_tproxy.c index 50481280abd2..f2101af8c867 100644 --- a/net/netfilter/nft_tproxy.c +++ b/net/netfilter/nft_tproxy.c @@ -331,7 +331,6 @@ static const struct nft_expr_ops nft_tproxy_ops = { .init = nft_tproxy_init, .destroy = nft_tproxy_destroy, .dump = nft_tproxy_dump, - .reduce = NFT_REDUCE_READONLY, .validate = nft_tproxy_validate, }; diff --git a/net/netfilter/nft_tunnel.c b/net/netfilter/nft_tunnel.c index db183e43941a..78d47f550355 100644 --- a/net/netfilter/nft_tunnel.c +++ b/net/netfilter/nft_tunnel.c @@ -124,31 +124,6 @@ static int nft_tunnel_get_dump(struct sk_buff *skb, return -1; } -static bool nft_tunnel_get_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_tunnel *priv = nft_expr_priv(expr); - const struct nft_tunnel *tunnel; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - tunnel = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->key != tunnel->key || - priv->dreg != tunnel->dreg || - priv->mode != tunnel->mode) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return false; -} - static struct nft_expr_type nft_tunnel_type; static const struct nft_expr_ops nft_tunnel_get_ops = { .type = &nft_tunnel_type, @@ -156,7 +131,6 @@ static const struct nft_expr_ops nft_tunnel_get_ops = { .eval = nft_tunnel_get_eval, .init = nft_tunnel_get_init, .dump = nft_tunnel_get_dump, - .reduce = nft_tunnel_get_reduce, }; static struct nft_expr_type nft_tunnel_type __read_mostly = { diff --git a/net/netfilter/nft_xfrm.c b/net/netfilter/nft_xfrm.c index 3210cfc966ab..7ffe6a2690d1 100644 --- a/net/netfilter/nft_xfrm.c +++ b/net/netfilter/nft_xfrm.c @@ -259,32 +259,6 @@ static int nft_xfrm_validate(const struct nft_ctx *ctx, const struct nft_expr *e return nft_chain_validate_hooks(ctx->chain, hooks); } -static bool nft_xfrm_reduce(struct nft_regs_track *track, - const struct nft_expr *expr) -{ - const struct nft_xfrm *priv = nft_expr_priv(expr); - const struct nft_xfrm *xfrm; - - if (!nft_reg_track_cmp(track, expr, priv->dreg)) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - xfrm = nft_expr_priv(track->regs[priv->dreg].selector); - if (priv->key != xfrm->key || - priv->dreg != xfrm->dreg || - priv->dir != xfrm->dir || - priv->spnum != xfrm->spnum) { - nft_reg_track_update(track, expr, priv->dreg, priv->len); - return false; - } - - if (!track->regs[priv->dreg].bitwise) - return true; - - return nft_expr_reduce_bitwise(track, expr); -} - static struct nft_expr_type nft_xfrm_type; static const struct nft_expr_ops nft_xfrm_get_ops = { .type = &nft_xfrm_type, @@ -293,7 +267,6 @@ static const struct nft_expr_ops nft_xfrm_get_ops = { .init = nft_xfrm_get_init, .dump = nft_xfrm_get_dump, .validate = nft_xfrm_validate, - .reduce = nft_xfrm_reduce, }; static struct nft_expr_type nft_xfrm_type __read_mostly = { diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions.c index 792ca44a461d..56ea40d23e28 100644 --- a/net/openvswitch/actions.c +++ b/net/openvswitch/actions.c @@ -836,12 +836,8 @@ static void do_output(struct datapath *dp, struct sk_buff *skb, int out_port, u16 mru = OVS_CB(skb)->mru; u32 cutlen = OVS_CB(skb)->cutlen; - if (unlikely(cutlen > 0)) { - if (skb->len - cutlen > ovs_mac_header_len(key)) - pskb_trim(skb, skb->len - cutlen); - else - pskb_trim(skb, ovs_mac_header_len(key)); - } + if (unlikely(cutlen < skb->len)) + pskb_trim(skb, max(cutlen, ovs_mac_header_len(key))); if (likely(!mru || (skb->len <= mru + vport->dev->hard_header_len))) { @@ -1233,7 +1229,7 @@ static void execute_psample(struct datapath *dp, struct sk_buff *skb, psample_group.net = ovs_dp_get_net(dp); md.in_ifindex = OVS_CB(skb)->input_vport->dev->ifindex; - md.trunc_size = skb->len - OVS_CB(skb)->cutlen; + md.trunc_size = min(skb->len, OVS_CB(skb)->cutlen); md.rate_as_probability = 1; rate = OVS_CB(skb)->probability ? OVS_CB(skb)->probability : U32_MAX; @@ -1283,22 +1279,21 @@ static int do_execute_actions(struct datapath *dp, struct sk_buff *skb, clone = skb_clone(skb, GFP_ATOMIC); if (clone) do_output(dp, clone, port, key); - OVS_CB(skb)->cutlen = 0; + OVS_CB(skb)->cutlen = U32_MAX; break; } case OVS_ACTION_ATTR_TRUNC: { struct ovs_action_trunc *trunc = nla_data(a); - if (skb->len > trunc->max_len) - OVS_CB(skb)->cutlen = skb->len - trunc->max_len; + OVS_CB(skb)->cutlen = trunc->max_len; break; } case OVS_ACTION_ATTR_USERSPACE: output_userspace(dp, skb, key, a, attr, len, OVS_CB(skb)->cutlen); - OVS_CB(skb)->cutlen = 0; + OVS_CB(skb)->cutlen = U32_MAX; if (nla_is_last(a, rem)) { consume_skb(skb); return 0; @@ -1452,7 +1447,7 @@ static int do_execute_actions(struct datapath *dp, struct sk_buff *skb, case OVS_ACTION_ATTR_PSAMPLE: execute_psample(dp, skb, a); - OVS_CB(skb)->cutlen = 0; + OVS_CB(skb)->cutlen = U32_MAX; if (nla_is_last(a, rem)) { consume_skb(skb); return 0; diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c index 56c744e1e14c..e3d89ce7df9f 100644 --- a/net/openvswitch/datapath.c +++ b/net/openvswitch/datapath.c @@ -276,7 +276,7 @@ void ovs_dp_process_packet(struct sk_buff *skb, struct sw_flow_key *key) upcall.portid = ovs_vport_find_upcall_portid(p, skb); upcall.mru = OVS_CB(skb)->mru; - error = ovs_dp_upcall(dp, skb, key, &upcall, 0); + error = ovs_dp_upcall(dp, skb, key, &upcall, U32_MAX); switch (error) { case 0: case -EAGAIN: @@ -457,7 +457,8 @@ static int queue_userspace_packet(struct datapath *dp, struct sk_buff *skb, struct sk_buff *nskb = NULL; struct sk_buff *user_skb = NULL; /* to be queued to userspace */ struct nlattr *nla; - size_t len; + size_t msg_size; + size_t skb_len; unsigned int hlen; int err, dp_ifindex; u64 hash; @@ -478,7 +479,8 @@ static int queue_userspace_packet(struct datapath *dp, struct sk_buff *skb, skb = nskb; } - if (nla_attr_size(skb->len) > USHRT_MAX) { + skb_len = min(skb->len, cutlen); + if (nla_attr_size(skb_len) > USHRT_MAX) { err = -EFBIG; goto out; } @@ -493,13 +495,13 @@ static int queue_userspace_packet(struct datapath *dp, struct sk_buff *skb, * padding logic. Only perform zerocopy if padding is not required. */ if (dp->user_features & OVS_DP_F_UNALIGNED) - hlen = skb_zerocopy_headlen(skb); + hlen = min(skb_zerocopy_headlen(skb), cutlen); else - hlen = skb->len; + hlen = skb_len; - len = upcall_msg_size(upcall_info, hlen - cutlen, - OVS_CB(skb)->acts_origlen); - user_skb = genlmsg_new(len, GFP_ATOMIC); + msg_size = upcall_msg_size(upcall_info, hlen, + OVS_CB(skb)->acts_origlen); + user_skb = genlmsg_new(msg_size, GFP_ATOMIC); if (!user_skb) { err = -ENOMEM; goto out; @@ -560,7 +562,7 @@ static int queue_userspace_packet(struct datapath *dp, struct sk_buff *skb, } /* Add OVS_PACKET_ATTR_LEN when packet is truncated */ - if (cutlen > 0 && + if (skb_len < skb->len && nla_put_u32(user_skb, OVS_PACKET_ATTR_LEN, skb->len)) { err = -ENOBUFS; goto out; @@ -585,9 +587,9 @@ static int queue_userspace_packet(struct datapath *dp, struct sk_buff *skb, err = -ENOBUFS; goto out; } - nla->nla_len = nla_attr_size(skb->len - cutlen); + nla->nla_len = nla_attr_size(skb_len); - err = skb_zerocopy(user_skb, skb, skb->len - cutlen, hlen); + err = skb_zerocopy(user_skb, skb, skb_len, hlen); if (err) goto out; @@ -644,6 +646,7 @@ static int ovs_packet_cmd_execute(struct sk_buff *skb, struct genl_info *info) packet->ignore_df = 1; } OVS_CB(packet)->mru = mru; + OVS_CB(packet)->cutlen = U32_MAX; if (a[OVS_PACKET_ATTR_HASH]) { hash = nla_get_u64(a[OVS_PACKET_ATTR_HASH]); diff --git a/net/openvswitch/datapath.h b/net/openvswitch/datapath.h index db0c3e69d66c..696640e88fa7 100644 --- a/net/openvswitch/datapath.h +++ b/net/openvswitch/datapath.h @@ -118,7 +118,7 @@ struct datapath { * @mru: The maximum received fragement size; 0 if the packet is not * fragmented. * @acts_origlen: The netlink size of the flow actions applied to this skb. - * @cutlen: The number of bytes from the packet end to be removed. + * @cutlen: The number of bytes in the packet to preserve on output. * @probability: The sampling probability that was applied to this skb; 0 means * no sampling has occurred; U32_MAX means 100% probability. * @upcall_pid: Netlink socket PID to use for sending this packet to userspace; diff --git a/net/openvswitch/vport.c b/net/openvswitch/vport.c index 1f0c86a9f43b..7470f43d31ea 100644 --- a/net/openvswitch/vport.c +++ b/net/openvswitch/vport.c @@ -503,7 +503,7 @@ int ovs_vport_receive(struct vport *vport, struct sk_buff *skb, OVS_CB(skb)->input_vport = vport; OVS_CB(skb)->mru = 0; - OVS_CB(skb)->cutlen = 0; + OVS_CB(skb)->cutlen = U32_MAX; OVS_CB(skb)->probability = 0; OVS_CB(skb)->upcall_pid = 0; if (unlikely(dev_net(skb->dev) != ovs_dp_get_net(vport->dp))) { diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index 8b3961fba750..2989f1481aef 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -4561,7 +4561,11 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u, spin_lock(&po->bind_lock); WRITE_ONCE(po->num, num); - if (was_running) + /* + * NETDEV_UNREGISTER may have invalidated the binding while bind_lock + * was dropped above. Do not re-add a fanout hook to a dead device. + */ + if (was_running && READ_ONCE(po->ifindex) != -1) register_prot_hook(sk); spin_unlock(&po->bind_lock); diff --git a/net/phonet/pep.c b/net/phonet/pep.c index 7c695f3afc28..5910dcd26555 100644 --- a/net/phonet/pep.c +++ b/net/phonet/pep.c @@ -55,6 +55,8 @@ static unsigned char *pep_get_sb(struct sk_buff *skb, u8 *ptype, u8 *plen, ph = skb_header_pointer(skb, 0, 2, &h); if (ph == NULL || ph->sb_len < 2 || !pskb_may_pull(skb, ph->sb_len)) return NULL; + /* pskb_may_pull() may have reallocated the head; refetch ph. */ + ph = skb_header_pointer(skb, 0, 2, &h); ph->sb_len -= 2; *ptype = ph->sb_type; *plen = ph->sb_len; diff --git a/net/qrtr/af_qrtr.c b/net/qrtr/af_qrtr.c index 305523cebe3b..fcd24a7df3f0 100644 --- a/net/qrtr/af_qrtr.c +++ b/net/qrtr/af_qrtr.c @@ -1261,6 +1261,14 @@ static int qrtr_create(struct net *net, struct socket *sock, if (sock->type != SOCK_DGRAM) return -EPROTOTYPE; + /* QRTR keeps its port and node state in module-global variables that + * are not partitioned per network namespace, and the in-kernel name + * service only operates in init_net. Confine the family to init_net so + * a socket in another namespace cannot reach the global control plane. + */ + if (!net_eq(net, &init_net)) + return -EAFNOSUPPORT; + sk = sk_alloc(net, AF_QIPCRTR, GFP_KERNEL, &qrtr_proto, kern); if (!sk) return -ENOMEM; diff --git a/net/qrtr/ns.c b/net/qrtr/ns.c index a10e8de4f7e1..e8e3ba1b3b82 100644 --- a/net/qrtr/ns.c +++ b/net/qrtr/ns.c @@ -85,9 +85,9 @@ struct qrtr_node { /* Max nodes limit is chosen based on the current platform requirements. * If the requirement changes in the future, this value can be increased. */ -#define QRTR_NS_MAX_NODES 64 +#define QRTR_NS_MAX_NODES 512 -static u8 node_count; +static u16 node_count; static struct qrtr_node *node_get(unsigned int node_id) { diff --git a/net/rds/recv.c b/net/rds/recv.c index 66205d6924bf..4b12fa4f1e78 100644 --- a/net/rds/recv.c +++ b/net/rds/recv.c @@ -366,6 +366,21 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr, goto out; } + /* + * rds_find_bound() uses a global (netns-agnostic) hash table. + * An RDS connection created in netns A can match a socket bound + * in the init netns, delivering inc cross-netns with inc->i_conn + * pointing into netns A. When cleanup_net() then frees that conn, + * any subsequent dereference of inc->i_conn is a use-after-free. + * Drop the inc if the receiving socket lives in a different netns. + */ + if (!net_eq(sock_net(rds_rs_to_sk(rs)), rds_conn_net(conn))) { + rds_stats_inc(s_recv_drop_no_sock); + rds_sock_put(rs); + rs = NULL; + goto out; + } + /* Process extension headers */ rds_recv_incoming_exthdrs(inc, rs); diff --git a/net/rds/tcp.c b/net/rds/tcp.c index b66dfcc3efaa..06a2d8d48bba 100644 --- a/net/rds/tcp.c +++ b/net/rds/tcp.c @@ -637,13 +637,13 @@ static void __net_exit rds_tcp_exit_net(struct net *net) { struct rds_tcp_net *rtn = net_generic(net, rds_tcp_netid); - rds_tcp_kill_sock(net); - if (rtn->rds_tcp_sysctl) unregister_net_sysctl_table(rtn->rds_tcp_sysctl); if (net != &init_net) kfree(rtn->ctl_table); + + rds_tcp_kill_sock(net); } static struct pernet_operations rds_tcp_net_ops = { diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h index d2b31d15851b..38b0997ce0cb 100644 --- a/net/rxrpc/ar-internal.h +++ b/net/rxrpc/ar-internal.h @@ -1266,9 +1266,11 @@ int rxrpc_io_thread(void *data); void rxrpc_post_response(struct rxrpc_connection *conn, struct sk_buff *skb); static inline void rxrpc_wake_up_io_thread(struct rxrpc_local *local) { - if (!local->io_thread) + struct task_struct *io_thread = READ_ONCE(local->io_thread); + + if (!io_thread) return; - wake_up_process(READ_ONCE(local->io_thread)); + wake_up_process(io_thread); } static inline bool rxrpc_protocol_error(struct sk_buff *skb, enum rxrpc_abort_reason why) diff --git a/net/sched/act_tunnel_key.c b/net/sched/act_tunnel_key.c index 876b30c5709e..b14807761d82 100644 --- a/net/sched/act_tunnel_key.c +++ b/net/sched/act_tunnel_key.c @@ -342,14 +342,20 @@ static const struct nla_policy tunnel_key_policy[TCA_TUNNEL_KEY_MAX + 1] = { [TCA_TUNNEL_KEY_ENC_TTL] = { .type = NLA_U8 }, }; -static void tunnel_key_release_params(struct tcf_tunnel_key_params *p) +static void tunnel_key_release_params_rcu(struct rcu_head *head) { - if (!p) - return; + struct tcf_tunnel_key_params *p = container_of(head, typeof(*p), rcu); + if (p->tcft_action == TCA_TUNNEL_KEY_ACT_SET) dst_release(&p->tcft_enc_metadata->dst); + kfree(p); +} - kfree_rcu(p, rcu); +static void tunnel_key_release_params(struct tcf_tunnel_key_params *p) +{ + if (!p) + return; + call_rcu(&p->rcu, tunnel_key_release_params_rcu); } static int tunnel_key_init(struct net *net, struct nlattr *nla, diff --git a/net/sctp/auth.c b/net/sctp/auth.c index 82aad477590e..fbbfa0c27955 100644 --- a/net/sctp/auth.c +++ b/net/sctp/auth.c @@ -672,7 +672,7 @@ int sctp_auth_ep_add_chunkid(struct sctp_endpoint *ep, __u8 chunk_id) /* Check if we can add this chunk to the array */ param_len = ntohs(p->param_hdr.length); nchunks = param_len - sizeof(struct sctp_paramhdr); - if (nchunks == SCTP_NUM_CHUNK_TYPES) + if (nchunks == SCTP_AUTH_MAX_CHUNKS) return -EINVAL; p->chunks[nchunks] = chunk_id; diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c index bbeff25185b1..5718328cb8a7 100644 --- a/net/sctp/protocol.c +++ b/net/sctp/protocol.c @@ -1410,10 +1410,6 @@ static int __net_init sctp_defaults_init(struct net *net) net->sctp.l3mdev_accept = 1; #endif - status = sctp_sysctl_net_register(net); - if (status) - goto err_sysctl_register; - /* Allocate and initialise sctp mibs. */ status = init_sctp_mibs(net); if (status) @@ -1447,8 +1443,6 @@ static int __net_init sctp_defaults_init(struct net *net) cleanup_sctp_mibs(net); #endif err_init_mibs: - sctp_sysctl_net_unregister(net); -err_sysctl_register: return status; } @@ -1463,7 +1457,6 @@ static void __net_exit sctp_defaults_exit(struct net *net) net->sctp.proc_net_sctp = NULL; #endif cleanup_sctp_mibs(net); - sctp_sysctl_net_unregister(net); } static struct pernet_operations sctp_defaults_ops = { @@ -1477,16 +1470,28 @@ static int __net_init sctp_ctrlsock_init(struct net *net) /* Initialize the control inode/socket for handling OOTB packets. */ status = sctp_ctl_sock_init(net); - if (status) + if (status) { pr_err("Failed to initialize the SCTP control sock\n"); + return status; + } + + status = sctp_sysctl_net_register(net); + if (status) { + inet_ctl_sock_destroy(net->sctp.ctl_sock); + net->sctp.ctl_sock = NULL; + } return status; } static void __net_exit sctp_ctrlsock_exit(struct net *net) { + sctp_sysctl_net_unregister(net); + sctp_udp_sock_stop(net); + /* Free the control endpoint. */ inet_ctl_sock_destroy(net->sctp.ctl_sock); + net->sctp.ctl_sock = NULL; } static struct pernet_operations sctp_ctrlsock_ops = { diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c index a7804e47d382..1f8a1e6069b3 100644 --- a/net/sctp/sm_make_chunk.c +++ b/net/sctp/sm_make_chunk.c @@ -3153,6 +3153,12 @@ static __be16 sctp_process_asconf_param(struct sctp_association *asoc, if (!peer) return SCTP_ERROR_DNS_FAILED; + /* Don't free asconf->transport; a later wildcard DEL-IP + * parameter reuses it. + */ + if (peer == asconf->transport) + return SCTP_ERROR_REQ_REFUSED; + sctp_assoc_rm_peer(asoc, peer); break; case SCTP_PARAM_SET_PRIMARY: diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c index abfd72d1f20c..4b8d48b6706c 100644 --- a/net/sctp/sm_statefuns.c +++ b/net/sctp/sm_statefuns.c @@ -641,7 +641,7 @@ static bool sctp_auth_chunk_verify(struct net *net, struct sctp_chunk *chunk, struct sctp_chunk auth; if (!chunk->auth_chunk) - return true; + return !sctp_auth_recv_cid(chunk->chunk_hdr->type, asoc); /* SCTP-AUTH: auth_chunk pointer is only set when the cookie-echo * is supposed to be authenticated and we have to do delayed diff --git a/net/sctp/stream.c b/net/sctp/stream.c index 39b8f5e8ce35..9addcfad467f 100644 --- a/net/sctp/stream.c +++ b/net/sctp/stream.c @@ -308,7 +308,8 @@ int sctp_send_reset_streams(struct sctp_association *asoc, goto out; param_len += str_nums * sizeof(__u16) + - sizeof(struct sctp_strreset_inreq); + (out ? sizeof(struct sctp_strreset_inreq) + : sizeof(struct sctp_strreset_outreq)); } if (param_len > SCTP_MAX_CHUNK_LEN - @@ -639,6 +640,9 @@ struct sctp_chunk *sctp_process_strreset_inreq( nums = (ntohs(param.p->length) - sizeof(*inreq)) / sizeof(__u16); str_p = inreq->list_of_streams; + if (nums * sizeof(__u16) + sizeof(struct sctp_strreset_outreq) > + SCTP_MAX_CHUNK_LEN - sizeof(struct sctp_reconf_chunk)) + goto out; for (i = 0; i < nums; i++) { if (ntohs(str_p[i]) >= stream->outcnt) { result = SCTP_STRRESET_ERR_WRONG_SSN; diff --git a/net/sctp/sysctl.c b/net/sctp/sysctl.c index 15e7db9a3ab2..fca840484ebf 100644 --- a/net/sctp/sysctl.c +++ b/net/sctp/sysctl.c @@ -615,11 +615,16 @@ int sctp_sysctl_net_register(struct net *net) void sctp_sysctl_net_unregister(struct net *net) { + struct ctl_table_header *header = net->sctp.sysctl_header; const struct ctl_table *table; - table = net->sctp.sysctl_header->ctl_table_arg; - unregister_net_sysctl_table(net->sctp.sysctl_header); + if (!header) + return; + + table = header->ctl_table_arg; + unregister_net_sysctl_table(header); kfree(table); + net->sctp.sysctl_header = NULL; } static struct ctl_table_header *sctp_sysctl_header; diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 9c14558bc552..0ed8d5637471 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -68,6 +68,17 @@ #define RPCDBG_FACILITY RPCDBG_SVCXPRT +/* + * For UDP: + * 1 for header page + * enough pages for RPCSVC_MAXPAYLOAD_UDP + * 1 in case payload is not aligned + * 1 for tail page + */ +enum { + SUNRPC_MAX_UDP_SENDPAGES = 1 + RPCSVC_MAXPAYLOAD_UDP / PAGE_SIZE + 1 + 1 +}; + /* To-do: to avoid tying up an nfsd thread while waiting for a * handshake request, the request could instead be deferred. */ @@ -737,7 +748,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) .msg_flags = MSG_SPLICE_PAGES, .msg_controllen = sizeof(buffer), }; - unsigned int count; + int count; int err; svc_udp_release_ctxt(xprt, rqstp->rq_xprt_ctxt); @@ -750,18 +761,23 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) if (svc_xprt_is_dead(xprt)) goto out_notconn; - count = xdr_buf_to_bvec(rqstp->rq_bvec, rqstp->rq_maxpages, xdr); + count = xdr_buf_to_bvec(svsk->sk_bvec, SUNRPC_MAX_UDP_SENDPAGES, xdr); + if (count < 0) { + err = count; + goto out_trace; + } - iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, rqstp->rq_bvec, + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, svsk->sk_bvec, count, rqstp->rq_res.len); err = sock_sendmsg(svsk->sk_sock, &msg); if (err == -ECONNREFUSED) { /* ICMP error on earlier request. */ - iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, rqstp->rq_bvec, + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, svsk->sk_bvec, count, rqstp->rq_res.len); err = sock_sendmsg(svsk->sk_sock, &msg); } +out_trace: trace_svcsock_udp_send(xprt, err); mutex_unlock(&xprt->xpt_mutex); @@ -1240,26 +1256,31 @@ static int svc_tcp_sendmsg(struct svc_sock *svsk, struct svc_rqst *rqstp, struct msghdr msg = { .msg_flags = MSG_SPLICE_PAGES, }; - unsigned int count; + int count; void *buf; int ret; /* The stream record marker is copied into a temporary page - * fragment buffer so that it can be included in rq_bvec. + * fragment buffer so that it can be included in sk_bvec. */ buf = page_frag_alloc(&svsk->sk_frag_cache, sizeof(marker), GFP_KERNEL); if (!buf) return -ENOMEM; memcpy(buf, &marker, sizeof(marker)); - bvec_set_virt(rqstp->rq_bvec, buf, sizeof(marker)); + bvec_set_virt(svsk->sk_bvec, buf, sizeof(marker)); - count = xdr_buf_to_bvec(rqstp->rq_bvec + 1, rqstp->rq_maxpages, + count = xdr_buf_to_bvec(svsk->sk_bvec + 1, rqstp->rq_maxpages, &rqstp->rq_res); + if (count < 0) { + ret = count; + goto out; + } - iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, rqstp->rq_bvec, + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, svsk->sk_bvec, 1 + count, sizeof(marker) + rqstp->rq_res.len); ret = sock_sendmsg(svsk->sk_sock, &msg); +out: page_frag_free(buf); return ret; } @@ -1402,6 +1423,20 @@ void svc_sock_update_bufs(struct svc_serv *serv) spin_unlock_bh(&serv->sv_lock); } +static int svc_sock_sendpages(struct svc_serv *serv, struct socket *sock, int flags) +{ + switch (sock->type) { + case SOCK_STREAM: + /* +1 for TCP record marker */ + if (flags & SVC_SOCK_TEMPORARY) + return svc_serv_maxpages(serv) + 1; + return 0; + case SOCK_DGRAM: + return SUNRPC_MAX_UDP_SENDPAGES; + } + return -EINVAL; +} + /* * Initialize socket for RPC use and create svc_sock struct */ @@ -1412,12 +1447,26 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv, struct svc_sock *svsk; struct sock *inet; int pmap_register = !(flags & SVC_SOCK_ANONYMOUS); + int sendpages; unsigned long pages; + sendpages = svc_sock_sendpages(serv, sock, flags); + if (sendpages < 0) + return ERR_PTR(sendpages); + pages = svc_serv_maxpages(serv); svsk = kzalloc(struct_size(svsk, sk_pages, pages), GFP_KERNEL); if (!svsk) return ERR_PTR(-ENOMEM); + + if (sendpages) { + svsk->sk_bvec = kcalloc(sendpages, sizeof(*svsk->sk_bvec), GFP_KERNEL); + if (!svsk->sk_bvec) { + kfree(svsk); + return ERR_PTR(-ENOMEM); + } + } + svsk->sk_maxpages = pages; inet = sock->sk; @@ -1429,6 +1478,7 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv, inet->sk_protocol, ntohs(inet_sk(inet)->inet_sport)); if (err < 0) { + kfree(svsk->sk_bvec); kfree(svsk); return ERR_PTR(err); } @@ -1646,5 +1696,6 @@ static void svc_sock_free(struct svc_xprt *xprt) sock_release(sock); page_frag_cache_drain(&svsk->sk_frag_cache); + kfree(svsk->sk_bvec); kfree(svsk); } diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index fe17a6e8372b..fb4dab207000 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -139,13 +139,14 @@ xdr_free_bvec(struct xdr_buf *buf) /** * xdr_buf_to_bvec - Copy components of an xdr_buf into a bio_vec array * @bvec: bio_vec array to populate - * @bvec_size: element count of @bio_vec + * @bvec_size: element count of @bvec * @xdr: xdr_buf to be copied * - * Returns the number of entries consumed in @bvec. + * Returns the number of entries consumed in @bvec on success, or + * -ESERVERFAULT when @xdr does not fit within @bvec_size entries. */ -unsigned int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size, - const struct xdr_buf *xdr) +int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size, + const struct xdr_buf *xdr) { const struct kvec *head = xdr->head; const struct kvec *tail = xdr->tail; @@ -187,10 +188,209 @@ unsigned int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size, bvec_overflow: pr_warn_once("%s: bio_vec array overflow\n", __func__); - return count; + return -ESERVERFAULT; } EXPORT_SYMBOL_GPL(xdr_buf_to_bvec); +/** + * xdr_buf_to_sg - Populate a scatterlist from an xdr_buf range + * @buf: xdr_buf to map + * @offset: starting byte offset within @buf + * @len: number of bytes to cover + * @sg: scatterlist array initialized with sg_init_table() + * @nsg: number of entries available in @sg + * + * @sg is traversed with sg_next(), so callers may pass a list + * assembled with sg_chain(). + * + * Return: on success, the number of scatterlist entries used; the + * last used entry is marked with sg_mark_end(). On failure, a + * negative errno. + */ +int xdr_buf_to_sg(const struct xdr_buf *buf, unsigned int offset, + unsigned int len, struct scatterlist *sg, unsigned int nsg) +{ + unsigned int page_len, thislen, page_offset; + struct scatterlist *cur = sg, *prev = NULL; + int nents = 0; + int i; + + if (len == 0) + return 0; + + if (offset >= buf->head[0].iov_len) { + offset -= buf->head[0].iov_len; + } else { + thislen = min_t(unsigned int, + buf->head[0].iov_len - offset, len); + if (nents >= nsg) + return -ENOSPC; + sg_set_buf(cur, buf->head[0].iov_base + offset, + thislen); + prev = cur; + cur = sg_next(cur); + nents++; + len -= thislen; + offset = 0; + } + if (len == 0) + goto done; + + if (offset >= buf->page_len) { + offset -= buf->page_len; + } else { + page_len = min(buf->page_len - offset, len); + len -= page_len; + page_offset = (offset + buf->page_base) & (PAGE_SIZE - 1); + i = (offset + buf->page_base) >> PAGE_SHIFT; + thislen = PAGE_SIZE - page_offset; + do { + if (thislen > page_len) + thislen = page_len; + if (nents >= nsg) + return -ENOSPC; + sg_set_page(cur, buf->pages[i], + thislen, page_offset); + prev = cur; + cur = sg_next(cur); + nents++; + page_len -= thislen; + i++; + page_offset = 0; + thislen = PAGE_SIZE; + } while (page_len != 0); + offset = 0; + } + if (len == 0) + goto done; + + if (offset < buf->tail[0].iov_len) { + thislen = min_t(unsigned int, + buf->tail[0].iov_len - offset, len); + if (nents >= nsg) + return -ENOSPC; + sg_set_buf(cur, buf->tail[0].iov_base + offset, + thislen); + prev = cur; + nents++; + len -= thislen; + } + if (len != 0) + return -EINVAL; + +done: + if (prev) + sg_mark_end(prev); + return nents; +} +EXPORT_SYMBOL_GPL(xdr_buf_to_sg); + +/* + * Count the scatterlist entries needed to cover [offset, offset + len) + * within @buf. Mirrors the walk in xdr_buf_to_sg() so the caller can + * size an allocation that matches the requested sub-range rather than + * the full xdr_buf. + */ +static unsigned int xdr_buf_sg_nents(const struct xdr_buf *buf, + unsigned int offset, unsigned int len) +{ + unsigned int nsg = 0, thislen, page_offset; + + if (len == 0) + return 0; + + if (offset < buf->head[0].iov_len) { + thislen = min_t(unsigned int, + buf->head[0].iov_len - offset, len); + nsg++; + len -= thislen; + offset = 0; + } else { + offset -= buf->head[0].iov_len; + } + if (len == 0) + return nsg; + + if (offset < buf->page_len) { + thislen = min(buf->page_len - offset, len); + page_offset = (offset + buf->page_base) & (PAGE_SIZE - 1); + nsg += DIV_ROUND_UP(page_offset + thislen, PAGE_SIZE); + len -= thislen; + offset = 0; + } else { + offset -= buf->page_len; + } + if (len == 0) + return nsg; + + if (offset < buf->tail[0].iov_len) + nsg++; + return nsg; +} + +/** + * xdr_buf_to_sg_alloc - Populate a scatterlist for an xdr_buf range + * @buf: xdr_buf to map + * @offset: starting byte offset within @buf + * @len: number of bytes to cover + * @sg_head: caller-provided scatterlist array (typically stack-allocated) + * @sg_head_nents: number of entries in @sg_head + * @sg_overflow: OUT: chained extension, or NULL when @sg_head sufficed + * @gfp: memory allocation flags for overflow + * + * Populates @sg_head directly when the xdr_buf fits. When more + * entries are needed, an overflow scatterlist is allocated and + * chained from @sg_head so that the result is traversable with + * sg_next(). + * + * Return: on success, the number of populated scatterlist entries + * (counting only data entries, not chain entries). @sg_head is + * the head of the resulting list. Caller must kfree @sg_overflow + * when done. On failure, a negative errno. + */ +int xdr_buf_to_sg_alloc(const struct xdr_buf *buf, unsigned int offset, + unsigned int len, struct scatterlist *sg_head, + unsigned int sg_head_nents, + struct scatterlist **sg_overflow, gfp_t gfp) +{ + unsigned int nsg; + int ret; + + *sg_overflow = NULL; + if (len == 0) + return 0; + + nsg = xdr_buf_sg_nents(buf, offset, len); + if (nsg == 0) + return -EINVAL; + + if (nsg <= sg_head_nents) { + sg_init_table(sg_head, nsg); + } else { + /* +1 replaces the slot sg_chain() consumes as the link. */ + unsigned int overflow_nents = nsg - sg_head_nents + 1; + struct scatterlist *overflow; + + overflow = kmalloc_array(overflow_nents, sizeof(*overflow), + gfp); + if (!overflow) + return -ENOMEM; + + sg_init_table(sg_head, sg_head_nents); + sg_init_table(overflow, overflow_nents); + sg_chain(sg_head, sg_head_nents, overflow); + *sg_overflow = overflow; + } + + ret = xdr_buf_to_sg(buf, offset, len, sg_head, nsg); + if (ret < 0) { + kfree(*sg_overflow); + *sg_overflow = NULL; + } + return ret; +} +EXPORT_SYMBOL_GPL(xdr_buf_to_sg_alloc); + /** * xdr_inline_pages - Prepare receive buffer for a large reply * @xdr: xdr_buf into which reply will be placed diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index 5d1f96bffd74..e2694fb03ba1 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -542,6 +542,7 @@ void rpcrdma_sendctx_unmap(struct rpcrdma_sendctx *sc) rpcrdma_sendctx_dma_unmap(sc); sc->sc_req = NULL; + req->rl_sendctx = NULL; rpcrdma_req_put(req); } @@ -550,8 +551,11 @@ void rpcrdma_sendctx_unmap(struct rpcrdma_sendctx *sc) */ static void rpcrdma_sendctx_cancel(struct rpcrdma_sendctx *sc) { + struct rpcrdma_req *req = sc->sc_req; + rpcrdma_sendctx_dma_unmap(sc); sc->sc_req = NULL; + req->rl_sendctx = NULL; } /* Prepare an SGE for the RPC-over-RDMA transport header. diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index efd91ed249fa..e2d3cee825cd 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -1080,9 +1080,15 @@ static struct rpcrdma_rep *rpcrdma_rep_get_locked(struct rpcrdma_buffer *buf) * @buf: buffer pool * @rep: rep to release * + * The rep's transient association with an rpc_rqst, established + * by rpcrdma_reply_handler() and torn down here, must not survive + * onto rb_free_reps: rpcrdma_post_recvs() pulls reps from the free + * list to re-post them, and a non-NULL rr_rqst on a free-listed rep + * would imply the rep is still referenced by a req. */ void rpcrdma_rep_put(struct rpcrdma_buffer *buf, struct rpcrdma_rep *rep) { + rep->rr_rqst = NULL; llist_add(&rep->rr_node, &buf->rb_free_reps); } @@ -1265,9 +1271,11 @@ rpcrdma_mr_get(struct rpcrdma_xprt *r_xprt) */ void rpcrdma_reply_put(struct rpcrdma_buffer *buffers, struct rpcrdma_req *req) { - if (req->rl_reply) { - rpcrdma_rep_put(buffers, req->rl_reply); + struct rpcrdma_rep *rep = req->rl_reply; + + if (rep) { req->rl_reply = NULL; + rpcrdma_rep_put(buffers, rep); } /* I2: rl_reply NULL after the put closes the * 'rep on rb_free_reps still referenced by req' window. diff --git a/net/tipc/netlink.c b/net/tipc/netlink.c index 8336a9664703..1307dd1a9613 100644 --- a/net/tipc/netlink.c +++ b/net/tipc/netlink.c @@ -113,12 +113,16 @@ const struct nla_policy tipc_nl_node_policy[TIPC_NLA_NODE_MAX + 1] = { }; /* Properties valid for media, bearer and link */ +static const struct netlink_range_validation tipc_nl_mtu_range = { + .max = U16_MAX, +}; + const struct nla_policy tipc_nl_prop_policy[TIPC_NLA_PROP_MAX + 1] = { [TIPC_NLA_PROP_UNSPEC] = { .type = NLA_UNSPEC }, [TIPC_NLA_PROP_PRIO] = { .type = NLA_U32 }, [TIPC_NLA_PROP_TOL] = { .type = NLA_U32 }, [TIPC_NLA_PROP_WIN] = { .type = NLA_U32 }, - [TIPC_NLA_PROP_MTU] = { .type = NLA_U32 }, + [TIPC_NLA_PROP_MTU] = NLA_POLICY_FULL_RANGE(NLA_U32, &tipc_nl_mtu_range), [TIPC_NLA_PROP_BROADCAST] = { .type = NLA_U32 }, [TIPC_NLA_PROP_BROADCAST_RATIO] = { .type = NLA_U32 } }; diff --git a/net/tipc/netlink_compat.c b/net/tipc/netlink_compat.c index 079aebb16ed8..30e37c05a3f3 100644 --- a/net/tipc/netlink_compat.c +++ b/net/tipc/netlink_compat.c @@ -222,6 +222,10 @@ static int __tipc_nl_compat_dumpit(struct tipc_nl_compat_cmd_dump *cmd, int rem; len = (*cmd->dumpit)(buf, &cb); + if (len < 0) { + err = len; + goto err_out; + } nlmsg_for_each_msg(nlmsg, nlmsg_hdr(buf), len, rem) { err = nlmsg_parse_deprecated(nlmsg, GENL_HDRLEN, diff --git a/net/tipc/socket.c b/net/tipc/socket.c index 677e17ec2939..24013455fbf0 100644 --- a/net/tipc/socket.c +++ b/net/tipc/socket.c @@ -501,6 +501,7 @@ static int tipc_sk_create(struct net *net, struct socket *sock, tipc_set_sk_state(sk, TIPC_OPEN); if (tipc_sk_insert(tsk)) { sk_free(sk); + sock->sk = NULL; pr_warn("Socket create failed; port number exhausted\n"); return -EINVAL; } diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 71734411ff4c..d2e3870590c8 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -594,13 +594,15 @@ void tls_device_splice_eof(struct socket *sock) struct tls_context *tls_ctx = tls_get_ctx(sk); struct iov_iter iter = {}; - if (!tls_is_partially_sent_record(tls_ctx)) + if (!tls_is_partially_sent_record(tls_ctx) && + !tls_is_pending_open_record(tls_ctx)) return; mutex_lock(&tls_ctx->tx_lock); lock_sock(sk); - if (tls_is_partially_sent_record(tls_ctx)) { + if (tls_is_partially_sent_record(tls_ctx) || + tls_is_pending_open_record(tls_ctx)) { iov_iter_bvec(&iter, ITER_SOURCE, NULL, 0, 0); tls_push_data(sk, &iter, 0, 0, TLS_RECORD_TYPE_DATA); } diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index d93735dbe3a2..518ef9fca33d 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -26,6 +26,13 @@ /* Threshold for detecting small packets to copy */ #define GOOD_COPY_LEN 128 +/* Max payload that can be collapsed into a single linear skb, using the same + * allocation threshold as virtio_vsock_alloc_skb() to avoid adding pressure + * on the page allocator. + */ +#define MAX_COLLAPSE_LEN \ + SKB_MAX_ORDER(VIRTIO_VSOCK_SKB_HEADROOM, PAGE_ALLOC_COSTLY_ORDER) + static void virtio_transport_cancel_close_work(struct vsock_sock *vsk, bool cancel_timeout); static s64 virtio_transport_has_space(struct virtio_vsock_sock *vvs); @@ -422,6 +429,145 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, return ret; } +static bool virtio_transport_can_collapse(struct sk_buff *skb) +{ + /* skbs that are partially consumed, mark a SEQPACKET message boundary, + * or are already large enough should not be collapsed: they either + * need special accounting, carry protocol state, or already have a + * good data-to-overhead ratio. + */ + if (VIRTIO_VSOCK_SKB_CB(skb)->offset) + return false; + if (le32_to_cpu(virtio_vsock_hdr(skb)->flags) & VIRTIO_VSOCK_SEQ_EOM) + return false; + if (skb->len >= MAX_COLLAPSE_LEN) + return false; + return true; +} + +/* Iterate through the packets in the queue starting from the current skb to + * count the number of bytes we can collapse. + */ +static unsigned int +virtio_transport_collapse_size(struct sk_buff *skb, struct sk_buff_head *queue) +{ + unsigned int target = skb->len - VIRTIO_VSOCK_SKB_CB(skb)->offset; + + while ((skb = skb_peek_next(skb, queue)) && + virtio_transport_can_collapse(skb)) { + unsigned int len = skb->len - VIRTIO_VSOCK_SKB_CB(skb)->offset; + + if (len > MAX_COLLAPSE_LEN - target) + return target; + + target += len; + } + + return target; +} + +/* Called under lock_sock to compact the receive queue by merging small skbs. + * @min_to_free: minimum number of skbs to eliminate from the queue. May free + * more to fill each collapsed skb to capacity. + */ +static void +virtio_transport_collapse_rx_queue(struct virtio_vsock_sock *vvs, + u32 min_to_free) +{ + struct sk_buff *skb, *next_skb, *new_skb = NULL; + struct sk_buff_head new_queue; + u32 saved = 0; + + __skb_queue_head_init(&new_queue); + + skb_queue_walk_safe(&vvs->rx_queue, skb, next_skb) { + struct virtio_vsock_hdr *hdr = virtio_vsock_hdr(skb); + u32 src_off = VIRTIO_VSOCK_SKB_CB(skb)->offset; + u32 src_len = skb->len - src_off; + bool keep; + + keep = !virtio_transport_can_collapse(skb); + if (keep) { + /* Finalize pending collapsed skb to preserve packet + * ordering. + */ + if (new_skb) { + __skb_queue_tail(&new_queue, new_skb); + new_skb = NULL; + saved--; + } + goto next; + } + + /* Finalize if this packet won't fit in the remaining tailroom, + * so we can allocate a right-sized new_skb. + */ + if (new_skb && src_len > skb_tailroom(new_skb)) { + __skb_queue_tail(&new_queue, new_skb); + new_skb = NULL; + saved--; + } + + if (!new_skb) { + unsigned int alloc_size; + + /* Check after finalizing to opportunistically fill + * each collapsed skb to capacity, merging more skbs + * than strictly required. + */ + if (saved >= min_to_free) + break; + + alloc_size = virtio_transport_collapse_size(skb, &vvs->rx_queue); + + /* Only this skb's data is eligible, nothing to merge + * with. Keep as-is. + */ + if (alloc_size <= src_len) { + keep = true; + goto next; + } + + new_skb = virtio_vsock_alloc_linear_skb(alloc_size + + VIRTIO_VSOCK_SKB_HEADROOM, GFP_KERNEL); + if (!new_skb) + break; + + memcpy(virtio_vsock_hdr(new_skb), hdr, + sizeof(struct virtio_vsock_hdr)); + virtio_vsock_hdr(new_skb)->len = 0; + } + + /* Cannot fail since src_off/src_len are within bounds, but if + * it does, discard new_skb to avoid queuing corrupted data. + */ + if (WARN_ON_ONCE(skb_copy_bits(skb, src_off, + skb_put(new_skb, src_len), + src_len))) { + kfree_skb(new_skb); + new_skb = NULL; + break; + } + + le32_add_cpu(&virtio_vsock_hdr(new_skb)->len, src_len); + virtio_vsock_hdr(new_skb)->flags |= hdr->flags; + +next: + __skb_unlink(skb, &vvs->rx_queue); + if (keep) { + __skb_queue_tail(&new_queue, skb); + } else { + consume_skb(skb); + saved++; + } + } + + if (new_skb) + __skb_queue_tail(&new_queue, new_skb); + + skb_queue_splice(&new_queue, &vvs->rx_queue); +} + static bool virtio_transport_inc_rx_pkt(struct virtio_vsock_sock *vvs, u32 len) { @@ -1341,12 +1487,29 @@ virtio_transport_recv_enqueue(struct vsock_sock *vsk, { struct virtio_vsock_sock *vvs = vsk->trans; bool can_enqueue, free_pkt = false; + u32 len, queue_max, queue_len; struct virtio_vsock_hdr *hdr; - u32 len; hdr = virtio_vsock_hdr(skb); len = le32_to_cpu(hdr->len); + /* virtio_transport_inc_rx_pkt() rejects packets when the per-skb + * overhead (skb_queue_len * SKB_TRUESIZE(0)) exceeds buf_alloc. + * Proactively collapse the queue before that happens. + * No rx_lock needed: lock_sock is held by caller, preventing + * concurrent enqueue or dequeue. + */ + queue_max = vvs->buf_alloc / SKB_TRUESIZE(0); + queue_len = skb_queue_len(&vvs->rx_queue); + if (queue_len >= queue_max) { + /* Walking a large queue may take a significant amount of time + * and cache misses, causing traffic burstiness. Limit the + * collapse to freeing room for this packet and the next one. + * It may free more to fill each collapsed skb to capacity. + */ + virtio_transport_collapse_rx_queue(vvs, queue_len + 2 - queue_max); + } + spin_lock_bh(&vvs->rx_lock); can_enqueue = virtio_transport_inc_rx_pkt(vvs, len); diff --git a/net/wireless/core.c b/net/wireless/core.c index 6b204ca91a14..efbf7b304634 100644 --- a/net/wireless/core.c +++ b/net/wireless/core.c @@ -1195,6 +1195,7 @@ void wiphy_unregister(struct wiphy *wiphy) /* this has nothing to do now but make sure it's gone */ cancel_work_sync(&rdev->wiphy_work); + cancel_work_sync(&rdev->sched_scan_res_wk); cancel_work_sync(&rdev->rfkill_block); cancel_work_sync(&rdev->conn_work); flush_work(&rdev->event_work); @@ -1443,7 +1444,7 @@ void cfg80211_init_wdev(struct wireless_dev *wdev) INIT_LIST_HEAD(&wdev->mgmt_registrations); INIT_LIST_HEAD(&wdev->pmsr_list); spin_lock_init(&wdev->pmsr_lock); - INIT_WORK(&wdev->pmsr_free_wk, cfg80211_pmsr_free_wk); + wiphy_work_init(&wdev->pmsr_free_wk, cfg80211_pmsr_free_wk); #ifdef CONFIG_CFG80211_WEXT wdev->wext.default_key = -1; @@ -1578,7 +1579,6 @@ static int cfg80211_netdev_notifier_call(struct notifier_block *nb, } /* since we just did cfg80211_leave() nothing to do there */ cancel_work_sync(&wdev->disconnect_wk); - cancel_work_sync(&wdev->pmsr_free_wk); break; case NETDEV_DOWN: wiphy_lock(&rdev->wiphy); diff --git a/net/wireless/core.h b/net/wireless/core.h index d5d78752227a..9bfd39af1742 100644 --- a/net/wireless/core.h +++ b/net/wireless/core.h @@ -566,7 +566,7 @@ cfg80211_get_6ghz_power_type(const u8 *elems, size_t elems_len); void cfg80211_release_pmsr(struct wireless_dev *wdev, u32 portid); void cfg80211_pmsr_wdev_down(struct wireless_dev *wdev); -void cfg80211_pmsr_free_wk(struct work_struct *work); +void cfg80211_pmsr_free_wk(struct wiphy *wiphy, struct wiphy_work *work); void cfg80211_remove_link(struct wireless_dev *wdev, unsigned int link_id); void cfg80211_remove_links(struct wireless_dev *wdev); diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c index 6e41a4269e1c..776dcf8835d7 100644 --- a/net/wireless/nl80211.c +++ b/net/wireless/nl80211.c @@ -347,7 +347,9 @@ nl80211_ftm_responder_policy[NL80211_FTM_RESP_ATTR_MAX + 1] = { static const struct nla_policy nl80211_pmsr_ftm_req_attr_policy[NL80211_PMSR_FTM_REQ_ATTR_MAX + 1] = { [NL80211_PMSR_FTM_REQ_ATTR_ASAP] = { .type = NLA_FLAG }, - [NL80211_PMSR_FTM_REQ_ATTR_PREAMBLE] = { .type = NLA_U32 }, + [NL80211_PMSR_FTM_REQ_ATTR_PREAMBLE] = + NLA_POLICY_RANGE(NLA_U32, NL80211_PREAMBLE_LEGACY, + NL80211_PREAMBLE_HE), [NL80211_PMSR_FTM_REQ_ATTR_NUM_BURSTS_EXP] = NLA_POLICY_MAX(NLA_U8, 15), [NL80211_PMSR_FTM_REQ_ATTR_BURST_PERIOD] = { .type = NLA_U16 }, @@ -501,7 +503,7 @@ nl80211_mbssid_config_policy[NL80211_MBSSID_CONFIG_ATTR_MAX + 1] = { [NL80211_MBSSID_CONFIG_ATTR_TX_IFINDEX] = { .type = NLA_U32 }, [NL80211_MBSSID_CONFIG_ATTR_EMA] = { .type = NLA_FLAG }, [NL80211_MBSSID_CONFIG_ATTR_TX_LINK_ID] = - NLA_POLICY_MAX(NLA_U8, IEEE80211_MLD_MAX_NUM_LINKS), + NLA_POLICY_RANGE(NLA_U8, 0, IEEE80211_MLD_MAX_NUM_LINKS - 1), }; static const struct nla_policy @@ -6034,7 +6036,8 @@ static int nl80211_parse_mbssid_config(struct wiphy *wiphy, } static struct cfg80211_mbssid_elems * -nl80211_parse_mbssid_elems(struct wiphy *wiphy, struct nlattr *attrs) +nl80211_parse_mbssid_elems(struct wiphy *wiphy, struct nlattr *attrs, + struct netlink_ext_ack *extack) { struct nlattr *nl_elems; struct cfg80211_mbssid_elems *elems; @@ -6045,6 +6048,12 @@ nl80211_parse_mbssid_elems(struct wiphy *wiphy, struct nlattr *attrs) return ERR_PTR(-EINVAL); nla_for_each_nested(nl_elems, attrs, rem_elems) { + int ret; + + ret = validate_ie_attr(nl_elems, extack); + if (ret) + return ERR_PTR(ret); + if (num_elems >= 255) return ERR_PTR(-EINVAL); num_elems++; @@ -6216,7 +6225,8 @@ static int nl80211_parse_beacon(struct cfg80211_registered_device *rdev, if (attrs[NL80211_ATTR_MBSSID_ELEMS]) { struct cfg80211_mbssid_elems *mbssid = nl80211_parse_mbssid_elems(&rdev->wiphy, - attrs[NL80211_ATTR_MBSSID_ELEMS]); + attrs[NL80211_ATTR_MBSSID_ELEMS], + extack); if (IS_ERR(mbssid)) return PTR_ERR(mbssid); @@ -6232,8 +6242,10 @@ static int nl80211_parse_beacon(struct cfg80211_registered_device *rdev, if (IS_ERR(rnr)) return PTR_ERR(rnr); - if (rnr && rnr->cnt < bcn->mbssid_ies->cnt) + if (rnr && rnr->cnt < bcn->mbssid_ies->cnt) { + kfree(rnr); return -EINVAL; + } bcn->rnr_ies = rnr; } diff --git a/net/wireless/pmsr.c b/net/wireless/pmsr.c index 13801cf35e9f..06563f608487 100644 --- a/net/wireless/pmsr.c +++ b/net/wireless/pmsr.c @@ -114,6 +114,7 @@ static int pmsr_parse_ftm(struct cfg80211_registered_device *rdev, NL_SET_ERR_MSG_ATTR(info->extack, tb[NL80211_PMSR_FTM_REQ_ATTR_REQUEST_LCI], "FTM: LCI request not supported"); + return -EOPNOTSUPP; } out->ftm.request_civicloc = @@ -122,6 +123,7 @@ static int pmsr_parse_ftm(struct cfg80211_registered_device *rdev, NL_SET_ERR_MSG_ATTR(info->extack, tb[NL80211_PMSR_FTM_REQ_ATTR_REQUEST_CIVICLOC], "FTM: civic location request not supported"); + return -EOPNOTSUPP; } out->ftm.trigger_based = @@ -196,6 +198,7 @@ static int pmsr_parse_peer(struct cfg80211_registered_device *rdev, { struct nlattr *tb[NL80211_PMSR_PEER_ATTR_MAX + 1]; struct nlattr *req[NL80211_PMSR_REQ_ATTR_MAX + 1]; + bool have_measurement_type = false; struct nlattr *treq; int err, rem; @@ -248,6 +251,14 @@ static int pmsr_parse_peer(struct cfg80211_registered_device *rdev, } nla_for_each_nested(treq, req[NL80211_PMSR_REQ_ATTR_DATA], rem) { + if (have_measurement_type) { + NL_SET_ERR_MSG_ATTR(info->extack, treq, + "multiple measurement types in request data"); + return -EINVAL; + } + + have_measurement_type = true; + switch (nla_type(treq)) { case NL80211_PMSR_TYPE_FTM: err = pmsr_parse_ftm(rdev, treq, out, info); @@ -257,10 +268,16 @@ static int pmsr_parse_peer(struct cfg80211_registered_device *rdev, "unsupported measurement type"); err = -EINVAL; } + if (err) + return err; } - if (err) - return err; + if (!have_measurement_type) { + NL_SET_ERR_MSG_ATTR(info->extack, + req[NL80211_PMSR_REQ_ATTR_DATA], + "missing measurement type in request data"); + return -EINVAL; + } return 0; } @@ -625,13 +642,11 @@ static void cfg80211_pmsr_process_abort(struct wireless_dev *wdev) } } -void cfg80211_pmsr_free_wk(struct work_struct *work) +void cfg80211_pmsr_free_wk(struct wiphy *wiphy, struct wiphy_work *work) { struct wireless_dev *wdev = container_of(work, struct wireless_dev, pmsr_free_wk); - guard(wiphy)(wdev->wiphy); - cfg80211_pmsr_process_abort(wdev); } @@ -647,7 +662,7 @@ void cfg80211_pmsr_wdev_down(struct wireless_dev *wdev) } spin_unlock_bh(&wdev->pmsr_lock); - cancel_work_sync(&wdev->pmsr_free_wk); + wiphy_work_cancel(wdev->wiphy, &wdev->pmsr_free_wk); if (found) cfg80211_pmsr_process_abort(wdev); @@ -662,7 +677,7 @@ void cfg80211_release_pmsr(struct wireless_dev *wdev, u32 portid) list_for_each_entry(req, &wdev->pmsr_list, list) { if (req->nl_portid == portid) { req->nl_portid = 0; - schedule_work(&wdev->pmsr_free_wk); + wiphy_work_queue(wdev->wiphy, &wdev->pmsr_free_wk); } } spin_unlock_bh(&wdev->pmsr_lock); diff --git a/net/wireless/scan.c b/net/wireless/scan.c index 2f2104bb0d52..11f7a27bb704 100644 --- a/net/wireless/scan.c +++ b/net/wireless/scan.c @@ -205,7 +205,7 @@ bool cfg80211_is_element_inherited(const struct element *elem, return true; if (elem->id == WLAN_EID_EXTENSION) { - if (!ext_id_len) + if (!ext_id_len || !elem->datalen) return true; loop_len = ext_id_len; list = &non_inherit_elem->data[3 + id_len]; @@ -3326,14 +3326,15 @@ cfg80211_inform_bss_frame_data(struct wiphy *wiphy, bssid = ext->u.s1g_beacon.sa; capability = le16_to_cpu(compat->compat_info); beacon_interval = le16_to_cpu(compat->beacon_int); + tsf = le32_to_cpu(ext->u.s1g_beacon.timestamp); + tsf |= (u64)le32_to_cpu(compat->tsf_completion) << 32; } else { bssid = mgmt->bssid; beacon_interval = le16_to_cpu(mgmt->u.probe_resp.beacon_int); capability = le16_to_cpu(mgmt->u.probe_resp.capab_info); + tsf = le64_to_cpu(mgmt->u.probe_resp.timestamp); } - tsf = le64_to_cpu(mgmt->u.probe_resp.timestamp); - if (ieee80211_is_probe_resp(mgmt->frame_control)) ftype = CFG80211_BSS_FTYPE_PRESP; else if (ext) @@ -3627,8 +3628,10 @@ int cfg80211_wext_siwscan(struct net_device *dev, /* translate "Scan for SSID" request */ if (wreq) { if (wrqu->data.flags & IW_SCAN_THIS_ESSID) { - if (wreq->essid_len > IEEE80211_MAX_SSID_LEN) - return -EINVAL; + if (wreq->essid_len > IEEE80211_MAX_SSID_LEN) { + err = -EINVAL; + goto out; + } memcpy(creq->req.ssids[0].ssid, wreq->essid, wreq->essid_len); creq->req.ssids[0].ssid_len = wreq->essid_len; diff --git a/net/x25/af_x25.c b/net/x25/af_x25.c index 655d1e0ae25f..a1b093308ccb 100644 --- a/net/x25/af_x25.c +++ b/net/x25/af_x25.c @@ -1772,15 +1772,19 @@ void x25_kill_by_neigh(struct x25_neigh *nb) { struct sock *s; +again: write_lock_bh(&x25_list_lock); sk_for_each(s, &x25_list) { if (x25_sk(s)->neighbour == nb) { + sock_hold(s); write_unlock_bh(&x25_list_lock); lock_sock(s); - x25_disconnect(s, ENETUNREACH, 0, 0); + if (x25_sk(s)->neighbour == nb) + x25_disconnect(s, ENETUNREACH, 0, 0); release_sock(s); - write_lock_bh(&x25_list_lock); + sock_put(s); + goto again; } } write_unlock_bh(&x25_list_lock); diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c index 550457e4c4f0..4719aba331cd 100644 --- a/net/xfrm/xfrm_device.c +++ b/net/xfrm/xfrm_device.c @@ -182,7 +182,7 @@ struct sk_buff *validate_xmit_xfrm(struct sk_buff *skb, netdev_features_t featur err = x->type_offload->xmit(x, skb, esp_features); if (err) { if (err == -EINPROGRESS) - return NULL; + return ERR_PTR(-EINPROGRESS); XFRM_INC_STATS(xs_net(x), LINUX_MIB_XFRMOUTSTATEPROTOERROR); kfree_skb(skb); @@ -224,7 +224,15 @@ struct sk_buff *validate_xmit_xfrm(struct sk_buff *skb, netdev_features_t featur pskb = skb2; } - return skb; + /* skb_gso_segment() set skb->prev to the last segment, but async + * crypto may have stolen it above without updating ->prev. Repoint + * it at the last retained segment so validate_xmit_skb_list() does + * not chain onto a segment now owned by the crypto engine. + */ + if (skb) + skb->prev = pskb; + + return skb ? skb : ERR_PTR(-EINPROGRESS); } EXPORT_SYMBOL_GPL(validate_xmit_xfrm); diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c index fe8e4f21b328..d39253b06c4e 100644 --- a/net/xfrm/xfrm_iptfs.c +++ b/net/xfrm/xfrm_iptfs.c @@ -480,6 +480,7 @@ static int iptfs_skb_add_frags(struct sk_buff *skb, } __skb_frag_ref(tofrag); shinfo->nr_frags++; + shinfo->flags |= SKBFL_SHARED_FRAG; /* see if we are done */ fraglen = tofrag->len; diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c index 4dad1f817228..25f9cbd3ce6a 100644 --- a/net/xfrm/xfrm_policy.c +++ b/net/xfrm/xfrm_policy.c @@ -1329,8 +1329,8 @@ static void xfrm_hash_rebuild(struct work_struct *work) } } - if (policy->selector.prefixlen_d < dbits || - policy->selector.prefixlen_s < sbits) + if (policy->selector.prefixlen_d >= dbits && + policy->selector.prefixlen_s >= sbits) continue; bin = xfrm_policy_inexact_alloc_bin(policy, dir); diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c index 89daad5db10c..b9049c2297bd 100644 --- a/net/xfrm/xfrm_state.c +++ b/net/xfrm/xfrm_state.c @@ -2070,8 +2070,11 @@ static struct xfrm_state *xfrm_state_clone_and_setup(struct xfrm_state *orig, x->mode_cbs = orig->mode_cbs; if (x->mode_cbs && x->mode_cbs->clone_state) { - if (x->mode_cbs->clone_state(x, orig)) + if (x->mode_cbs->clone_state(x, orig)) { + if (!x->mode_data) + x->mode_cbs = NULL; goto error; + } } @@ -3253,6 +3256,8 @@ int __xfrm_init_state(struct xfrm_state *x, struct netlink_ext_ack *extack) if (x->mode_cbs->init_state) err = x->mode_cbs->init_state(x); module_put(x->mode_cbs->owner); + if (err && !x->mode_data) + x->mode_cbs = NULL; } error: return err; diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c index 9ad6619888c7..3641ccccbc41 100644 --- a/net/xfrm/xfrm_user.c +++ b/net/xfrm/xfrm_user.c @@ -2068,13 +2068,12 @@ static int validate_tmpl(int nr, struct xfrm_user_tmpl *ut, u16 family, switch (ut[i].mode) { case XFRM_MODE_TUNNEL: case XFRM_MODE_BEET: + case XFRM_MODE_IPTFS: if (ut[i].optional && dir == XFRM_POLICY_OUT) { NL_SET_ERR_MSG(extack, "Mode in optional template not allowed in outbound policy"); return -EINVAL; } break; - case XFRM_MODE_IPTFS: - break; default: if (ut[i].family != prev_family) { NL_SET_ERR_MSG(extack, "Mode in template doesn't support a family change"); diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs index 0c57cf9b4004..5e208064d46c 100644 --- a/rust/bindings/lib.rs +++ b/rust/bindings/lib.rs @@ -30,6 +30,10 @@ #[allow(clippy::ref_as_ptr)] #[allow(clippy::undocumented_unsafe_blocks)] #[cfg_attr(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES, allow(unnecessary_transmutes))] +#[cfg_attr( + CONFIG_RUSTC_HAS_SUSPICIOUS_RUNTIME_SYMBOL_DEFINITIONS, + allow(suspicious_runtime_symbol_definitions) +)] mod bindings_raw { use pin_init::{MaybeZeroable, Zeroable}; diff --git a/rust/kernel/device.rs b/rust/kernel/device.rs index 176531f54ed3..a82b1bde1fa8 100644 --- a/rust/kernel/device.rs +++ b/rust/kernel/device.rs @@ -625,7 +625,7 @@ macro_rules! dev_printk { /// ``` #[macro_export] macro_rules! dev_emerg { - ($($f:tt)*) => { $crate::dev_printk!(pr_emerg, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_emerg, $($f)*) } } /// Prints an alert-level message (level 1) prefixed with device information. @@ -651,7 +651,7 @@ macro_rules! dev_emerg { /// ``` #[macro_export] macro_rules! dev_alert { - ($($f:tt)*) => { $crate::dev_printk!(pr_alert, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_alert, $($f)*) } } /// Prints a critical-level message (level 2) prefixed with device information. @@ -677,7 +677,7 @@ macro_rules! dev_alert { /// ``` #[macro_export] macro_rules! dev_crit { - ($($f:tt)*) => { $crate::dev_printk!(pr_crit, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_crit, $($f)*) } } /// Prints an error-level message (level 3) prefixed with device information. @@ -703,7 +703,7 @@ macro_rules! dev_crit { /// ``` #[macro_export] macro_rules! dev_err { - ($($f:tt)*) => { $crate::dev_printk!(pr_err, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_err, $($f)*) } } /// Prints a warning-level message (level 4) prefixed with device information. @@ -729,7 +729,7 @@ macro_rules! dev_err { /// ``` #[macro_export] macro_rules! dev_warn { - ($($f:tt)*) => { $crate::dev_printk!(pr_warn, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_warn, $($f)*) } } /// Prints a notice-level message (level 5) prefixed with device information. @@ -755,7 +755,7 @@ macro_rules! dev_warn { /// ``` #[macro_export] macro_rules! dev_notice { - ($($f:tt)*) => { $crate::dev_printk!(pr_notice, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_notice, $($f)*) } } /// Prints an info-level message (level 6) prefixed with device information. @@ -781,7 +781,7 @@ macro_rules! dev_notice { /// ``` #[macro_export] macro_rules! dev_info { - ($($f:tt)*) => { $crate::dev_printk!(pr_info, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_info, $($f)*) } } /// Prints a debug-level message (level 7) prefixed with device information. @@ -807,5 +807,5 @@ macro_rules! dev_info { /// ``` #[macro_export] macro_rules! dev_dbg { - ($($f:tt)*) => { $crate::dev_printk!(pr_dbg, $($f)*); } + ($($f:tt)*) => { $crate::dev_printk!(pr_dbg, $($f)*) } } diff --git a/rust/kernel/time.rs b/rust/kernel/time.rs index 6ea98dfcd027..c1e83aed2b46 100644 --- a/rust/kernel/time.rs +++ b/rust/kernel/time.rs @@ -421,15 +421,22 @@ pub const fn as_nanos(self) -> i64 { /// to the value in the [`Delta`]. #[inline] pub fn as_micros_ceil(self) -> i64 { + let n = self.as_nanos(); + let n = if n >= 0 { + n.saturating_add(NSEC_PER_USEC - 1) + } else { + n + }; + #[cfg(CONFIG_64BIT)] { - self.as_nanos().saturating_add(NSEC_PER_USEC - 1) / NSEC_PER_USEC + n / NSEC_PER_USEC } #[cfg(not(CONFIG_64BIT))] // SAFETY: It is always safe to call `ktime_to_us()` with any value. unsafe { - bindings::ktime_to_us(self.as_nanos().saturating_add(NSEC_PER_USEC - 1)) + bindings::ktime_to_us(n) } } diff --git a/rust/uapi/lib.rs b/rust/uapi/lib.rs index 1d5fd9efb93e..bec80b1dba23 100644 --- a/rust/uapi/lib.rs +++ b/rust/uapi/lib.rs @@ -28,6 +28,10 @@ unsafe_op_in_unsafe_fn )] #![cfg_attr(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES, allow(unnecessary_transmutes))] +#![cfg_attr( + CONFIG_RUSTC_HAS_SUSPICIOUS_RUNTIME_SYMBOL_DEFINITIONS, + allow(suspicious_runtime_symbol_definitions) +)] // Manual definition of blocklisted types. type __kernel_size_t = usize; diff --git a/security/landlock/domain.c b/security/landlock/domain.c index a647b68e8d06..74fbe8de1fc5 100644 --- a/security/landlock/domain.c +++ b/security/landlock/domain.c @@ -90,12 +90,12 @@ static struct landlock_details *get_current_details(void) return ERR_CAST(buffer); /* - * Create the new details according to the path's length. Do not - * allocate with GFP_KERNEL_ACCOUNT because it is independent from the - * caller. + * Create the new details according to the path's length. Account to + * the calling task's memcg, like the other Landlock per-domain + * allocations, even if it may not control the related size. */ details = - kzalloc(struct_size(details, exe_path, path_size), GFP_KERNEL); + kzalloc_flex(*details, exe_path, path_size, GFP_KERNEL_ACCOUNT); if (!details) return ERR_PTR(-ENOMEM); diff --git a/security/landlock/domain.h b/security/landlock/domain.h index 7fb70b25f85a..e3525da27b93 100644 --- a/security/landlock/domain.h +++ b/security/landlock/domain.h @@ -33,10 +33,7 @@ enum landlock_log_status { * Rarely accessed, mainly when logging the first domain's denial. * * The contained pointers are initialized at the domain creation time and never - * changed again. Contrary to most other Landlock object types, this one is - * not allocated with GFP_KERNEL_ACCOUNT because its size may not be under the - * caller's control (e.g. unknown exe_path) and the data is not explicitly - * requested nor used by tasks. + * changed again. */ struct landlock_details { /** diff --git a/security/landlock/ruleset.c b/security/landlock/ruleset.c index ce7940efea51..2a1e57ac0c53 100644 --- a/security/landlock/ruleset.c +++ b/security/landlock/ruleset.c @@ -33,9 +33,8 @@ static struct landlock_ruleset *create_ruleset(const u32 num_layers) { struct landlock_ruleset *new_ruleset; - new_ruleset = - kzalloc(struct_size(new_ruleset, access_masks, num_layers), - GFP_KERNEL_ACCOUNT); + new_ruleset = kzalloc_flex(*new_ruleset, access_masks, num_layers, + GFP_KERNEL_ACCOUNT); if (!new_ruleset) return ERR_PTR(-ENOMEM); refcount_set(&new_ruleset->usage, 1); @@ -553,7 +552,7 @@ landlock_merge_ruleset(struct landlock_ruleset *const parent, return new_dom; new_dom->hierarchy = - kzalloc(sizeof(*new_dom->hierarchy), GFP_KERNEL_ACCOUNT); + kzalloc_obj(*new_dom->hierarchy, GFP_KERNEL_ACCOUNT); if (!new_dom->hierarchy) return ERR_PTR(-ENOMEM); diff --git a/sound/core/seq/seq_timer.c b/sound/core/seq/seq_timer.c index 29b018a212fc..0256caa61c29 100644 --- a/sound/core/seq/seq_timer.c +++ b/sound/core/seq/seq_timer.c @@ -61,12 +61,23 @@ struct snd_seq_timer *snd_seq_timer_new(void) void snd_seq_timer_delete(struct snd_seq_timer **tmr) { struct snd_seq_timer *t = *tmr; - *tmr = NULL; + struct snd_timer_instance *ti; if (t == NULL) { pr_debug("ALSA: seq: snd_seq_timer_delete() called with NULL timer\n"); return; } + + scoped_guard(spinlock_irq, &t->lock) { + ti = t->timeri; + t->timeri = NULL; + } + if (ti) { + snd_timer_close(ti); + snd_timer_instance_free(ti); + } + + *tmr = NULL; t->running = 0; /* reset time */ diff --git a/sound/core/timer.c b/sound/core/timer.c index 35651c273b2f..a2c344b4cc73 100644 --- a/sound/core/timer.c +++ b/sound/core/timer.c @@ -437,10 +437,20 @@ static void snd_timer_close_locked(struct snd_timer_instance *timeri, snd_timer_stop(timeri); if (timer) { + struct snd_timer_instance *slave; + bool busy; + timer->num_instances--; - /* wait, until the active callback is finished */ + /* unqueue then drain the slaves' callbacks before remove_slave_links() severs them */ spin_lock_irq(&timer->lock); - while (timeri->flags & SNDRV_TIMER_IFLG_CALLBACK) { + list_for_each_entry(slave, &timeri->slave_list_head, open_list) + list_del_init(&slave->ack_list); + for (;;) { + busy = timeri->flags & SNDRV_TIMER_IFLG_CALLBACK; + list_for_each_entry(slave, &timeri->slave_list_head, open_list) + busy |= slave->flags & SNDRV_TIMER_IFLG_CALLBACK; + if (!busy) + break; spin_unlock_irq(&timer->lock); udelay(10); spin_lock_irq(&timer->lock); @@ -873,12 +883,15 @@ void snd_timer_interrupt(struct snd_timer * timer, unsigned long ticks_left) ack_list_head = &timer->ack_list_head; else ack_list_head = &timer->sack_list_head; - if (list_empty(&ti->ack_list)) + /* don't requeue an instance whose callback is still running */ + if (list_empty(&ti->ack_list) && + !(ti->flags & SNDRV_TIMER_IFLG_CALLBACK)) list_add_tail(&ti->ack_list, ack_list_head); list_for_each_entry(ts, &ti->slave_active_head, active_list) { ts->pticks = ti->pticks; ts->resolution = resolution; - if (list_empty(&ts->ack_list)) + if (list_empty(&ts->ack_list) && + !(ts->flags & SNDRV_TIMER_IFLG_CALLBACK)) list_add_tail(&ts->ack_list, ack_list_head); } } diff --git a/sound/hda/codecs/hdmi/hdmi.c b/sound/hda/codecs/hdmi/hdmi.c index 1036d08d1431..f427d04176e9 100644 --- a/sound/hda/codecs/hdmi/hdmi.c +++ b/sound/hda/codecs/hdmi/hdmi.c @@ -1688,6 +1688,9 @@ int snd_hda_hdmi_generic_pcm_prepare(struct hda_pcm_stream *hinfo, per_pin->channels = substream->runtime->channels; per_pin->setup = true; + if (spec->ops.prepare) + spec->ops.prepare(codec, per_pin); + if (get_wcaps(codec, cvt_nid) & AC_WCAP_STRIPE) { stripe = snd_hdac_get_stream_stripe_ctl(&codec->bus->core, substream); diff --git a/sound/hda/codecs/hdmi/hdmi_local.h b/sound/hda/codecs/hdmi/hdmi_local.h index 548241ad3fa9..dce9a335cff2 100644 --- a/sound/hda/codecs/hdmi/hdmi_local.h +++ b/sound/hda/codecs/hdmi/hdmi_local.h @@ -74,6 +74,15 @@ struct hdmi_ops { hda_nid_t pin_nid, int dev_id, u32 stream_tag, int format); + /* + * Optional hook invoked at the beginning of the PCM prepare + * sequence, before the audio infoframe and stream format are + * (re)programmed. Used to disable keep-alive / silent stream so + * that the format change is not done while keep-alive is active. + */ + void (*prepare)(struct hda_codec *codec, + struct hdmi_spec_per_pin *per_pin); + void (*pin_cvt_fixup)(struct hda_codec *codec, struct hdmi_spec_per_pin *per_pin, hda_nid_t cvt_nid); diff --git a/sound/hda/codecs/hdmi/intelhdmi.c b/sound/hda/codecs/hdmi/intelhdmi.c index 23237d527430..3a942343bc98 100644 --- a/sound/hda/codecs/hdmi/intelhdmi.c +++ b/sound/hda/codecs/hdmi/intelhdmi.c @@ -418,6 +418,28 @@ static void intel_not_share_assigned_cvt_nid(struct hda_codec *codec, intel_not_share_assigned_cvt(codec, pin_nid, dev_id, mux_idx); } +/* + * prepare ops override for HSW+ + * + * Disable keep-alive before the converter format and audio infoframe are + * reprogrammed by the PCM prepare sequence. Changing the audio format (e.g. + * the channel count when switching to multichannel PCM) while a keep-alive + * stream is active is not safe, so release keep-alive here, early in the + * sequence. It is re-enabled once the new stream has been set up, in + * i915_hsw_setup_stream(). + */ +static void i915_hsw_prepare(struct hda_codec *codec, + struct hdmi_spec_per_pin *per_pin) +{ + struct hdmi_spec *spec = codec->spec; + + if (spec->silent_stream_type == SILENT_STREAM_KAE && per_pin->silent_stream) { + silent_stream_set_kae(codec, per_pin, false); + /* wait for pending transfers in codec to clear */ + usleep_range(100, 200); + } +} + /* setup_stream ops override for HSW+ */ static int i915_hsw_setup_stream(struct hda_codec *codec, hda_nid_t cvt_nid, hda_nid_t pin_nid, int dev_id, u32 stream_tag, @@ -435,15 +457,16 @@ static int i915_hsw_setup_stream(struct hda_codec *codec, hda_nid_t cvt_nid, haswell_verify_D0(codec, cvt_nid, pin_nid); - if (spec->silent_stream_type == SILENT_STREAM_KAE && per_pin && per_pin->silent_stream) { - silent_stream_set_kae(codec, per_pin, false); - /* wait for pending transfers in codec to clear */ - usleep_range(100, 200); - } - res = snd_hda_hdmi_setup_stream(codec, cvt_nid, pin_nid, dev_id, stream_tag, format); + /* + * Keep-alive was disabled in i915_hsw_prepare(), re-enable it now. + * The pin lookup above resolves to the same per_pin that prepare + * used (pin_nid comes from that per_pin), so this stays balanced; a + * NULL per_pin only occurs on a lookup failure that also implies no + * active keep-alive stream to restore. + */ if (spec->silent_stream_type == SILENT_STREAM_KAE && per_pin && per_pin->silent_stream) { usleep_range(100, 200); silent_stream_set_kae(codec, per_pin, true); @@ -607,6 +630,7 @@ static int intel_hsw_common_init(struct hda_codec *codec, hda_nid_t vendor_nid, codec->depop_delay = 0; codec->auto_runtime_pm = 1; + spec->ops.prepare = i915_hsw_prepare; spec->ops.setup_stream = i915_hsw_setup_stream; spec->ops.pin_cvt_fixup = i915_pin_cvt_fixup; spec->ops.silent_stream = i915_set_silent_stream; diff --git a/sound/hda/codecs/realtek/alc269.c b/sound/hda/codecs/realtek/alc269.c index 01483eb545cd..40a75d72f712 100644 --- a/sound/hda/codecs/realtek/alc269.c +++ b/sound/hda/codecs/realtek/alc269.c @@ -3742,6 +3742,7 @@ enum { ALC269_FIXUP_DMIC_THINKPAD_ACPI, ALC269VB_FIXUP_INFINIX_ZERO_BOOK_13, ALC269VC_FIXUP_INFINIX_Y4_MAX, + ALC269VC_FIXUP_LUNNEN_GROUND_14, ALC269VB_FIXUP_CHUWI_COREBOOK_XPRO, ALC255_FIXUP_ACER_MIC_NO_PRESENCE, ALC255_FIXUP_ASUS_MIC_NO_PRESENCE, @@ -4183,6 +4184,13 @@ static const struct hda_fixup alc269_fixups[] = { .chained = true, .chain_id = ALC269_FIXUP_LIMIT_INT_MIC_BOOST }, + [ALC269VC_FIXUP_LUNNEN_GROUND_14] = { + .type = HDA_FIXUP_PINS, + .v.pins = (const struct hda_pintbl[]) { + { 0x1b, 0x90170150 }, /* internal speaker */ + { } + }, + }, [ALC269VB_FIXUP_CHUWI_COREBOOK_XPRO] = { .type = HDA_FIXUP_PINS, .v.pins = (const struct hda_pintbl[]) { @@ -7573,6 +7581,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x2782, 0x1705, "MEDION E15433", ALC269VC_FIXUP_INFINIX_Y4_MAX), SND_PCI_QUIRK(0x2782, 0x1707, "Vaio VJFE-ADL", ALC298_FIXUP_SPK_VOLUME), SND_PCI_QUIRK(0x2782, 0x4900, "MEDION E15443", ALC233_FIXUP_MEDION_MTL_SPK), + SND_PCI_QUIRK(0x2782, 0xa212, "Lunnen Ground 14", ALC269VC_FIXUP_LUNNEN_GROUND_14), SND_PCI_QUIRK(0x7017, 0x2014, "Star Labs StarFighter", ALC233_FIXUP_STARLABS_STARFIGHTER), SND_PCI_QUIRK(0x8086, 0x2074, "Intel NUC 8", ALC233_FIXUP_INTEL_NUC8_DMIC), SND_PCI_QUIRK(0x8086, 0x2080, "Intel NUC 8 Rugged", ALC256_FIXUP_INTEL_NUC8_RUGGED), diff --git a/sound/hda/codecs/side-codecs/cs35l41_hda.c b/sound/hda/codecs/side-codecs/cs35l41_hda.c index 438a19bd979b..de2fd6471f05 100644 --- a/sound/hda/codecs/side-codecs/cs35l41_hda.c +++ b/sound/hda/codecs/side-codecs/cs35l41_hda.c @@ -1437,10 +1437,19 @@ static int cs35l41_get_acpi_mute_state(struct cs35l41_hda *cs35l41, acpi_handle guid_parse(CS35L41_UUID, &guid); if (cs35l41_dsm_supported(handle, CS35L41_DSM_GET_MUTE)) { - ret = acpi_evaluate_dsm(handle, &guid, 0, CS35L41_DSM_GET_MUTE, NULL); + ret = acpi_evaluate_dsm_typed(handle, &guid, 0, + CS35L41_DSM_GET_MUTE, NULL, + ACPI_TYPE_BUFFER); + if (!ret) return -EINVAL; + if (!ret->buffer.length || !ret->buffer.pointer) { + ACPI_FREE(ret); + return -EINVAL; + } + mute = *ret->buffer.pointer; + ACPI_FREE(ret); dev_dbg(cs35l41->dev, "CS35L41_DSM_GET_MUTE: %d\n", mute); } diff --git a/sound/soc/amd/ps/pci-ps.c b/sound/soc/amd/ps/pci-ps.c index c62299b29204..cf06d4ae299b 100644 --- a/sound/soc/amd/ps/pci-ps.c +++ b/sound/soc/amd/ps/pci-ps.c @@ -248,7 +248,7 @@ static irqreturn_t acp63_irq_handler(int irq, void *dev_id) if (sdw_dma_irq_flag) return IRQ_WAKE_THREAD; - if (irq_flag | wake_irq_flag) + if (irq_flag || wake_irq_flag) return IRQ_HANDLED; else return IRQ_NONE; @@ -595,7 +595,7 @@ static int snd_acp63_probe(struct pci_dev *pci, return -ENODEV; } - ret = pci_request_regions(pci, "AMD ACP6.2 audio"); + ret = pci_request_regions(pci, "AMD ACP6.3 audio"); if (ret < 0) { dev_err(&pci->dev, "pci_request_regions failed\n"); goto disable_pci; @@ -683,8 +683,37 @@ static int snd_acp_runtime_resume(struct device *dev) return acp_hw_runtime_resume(dev); } +static void acp_disable_msi_on_resume(struct pci_dev *pdev) +{ + u16 control; + + if (!pdev->msi_cap) + return; + + pci_read_config_word(pdev, pdev->msi_cap + PCI_MSI_FLAGS, &control); + if (control & PCI_MSI_FLAGS_ENABLE) { + dev_warn(&pdev->dev, + "ACP: MSI unexpectedly enabled after resume (flags=0x%04x), disabling\n", + control); + control &= ~PCI_MSI_FLAGS_ENABLE; + pci_write_config_word(pdev, pdev->msi_cap + PCI_MSI_FLAGS, control); + } +} + static int snd_acp_resume(struct device *dev) { + struct pci_dev *pdev = to_pci_dev(dev); + + /* + * BIOS/firmware may re-enable MSI in PCI config space during + * system resume even though this driver only uses legacy INTx + * interrupts. If MSI is left enabled with stale address/data + * registers, the device will write interrupts to a bogus address + * causing IOMMU IO_PAGE_FAULT and interrupt delivery failure. + * Explicitly clear the MSI Enable bit before reinitializing + * the ACP hardware. + */ + acp_disable_msi_on_resume(pdev); return acp_hw_resume(dev); } diff --git a/sound/soc/codecs/bt-sco.c b/sound/soc/codecs/bt-sco.c index 3afcef2dfa35..c0bf45b76cb8 100644 --- a/sound/soc/codecs/bt-sco.c +++ b/sound/soc/codecs/bt-sco.c @@ -17,11 +17,17 @@ static const struct snd_soc_dapm_widget bt_sco_widgets[] = { SND_SOC_NOPM, 0, 0), SND_SOC_DAPM_AIF_OUT("BT_SCO_TX", "Capture", 0, SND_SOC_NOPM, 0, 0), + SND_SOC_DAPM_AIF_IN("BT_SCO_RX_WB", "WB Playback", 0, + SND_SOC_NOPM, 0, 0), + SND_SOC_DAPM_AIF_OUT("BT_SCO_TX_WB", "WB Capture", 0, + SND_SOC_NOPM, 0, 0), }; static const struct snd_soc_dapm_route bt_sco_routes[] = { { "BT_SCO_TX", NULL, "RX" }, { "TX", NULL, "BT_SCO_RX" }, + { "BT_SCO_TX_WB", NULL, "RX" }, + { "TX", NULL, "BT_SCO_RX_WB" }, }; static struct snd_soc_dai_driver bt_sco_dai[] = { @@ -45,14 +51,14 @@ static struct snd_soc_dai_driver bt_sco_dai[] = { { .name = "bt-sco-pcm-wb", .playback = { - .stream_name = "Playback", + .stream_name = "WB Playback", .channels_min = 1, .channels_max = 1, .rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_16000, .formats = SNDRV_PCM_FMTBIT_S16_LE, }, .capture = { - .stream_name = "Capture", + .stream_name = "WB Capture", .channels_min = 1, .channels_max = 1, .rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_16000, diff --git a/sound/soc/codecs/cs35l56-i2c.c b/sound/soc/codecs/cs35l56-i2c.c index 0492ddc4102d..8259714d8b26 100644 --- a/sound/soc/codecs/cs35l56-i2c.c +++ b/sound/soc/codecs/cs35l56-i2c.c @@ -55,9 +55,7 @@ static int cs35l56_i2c_probe(struct i2c_client *client) if (ret != 0) return ret; - ret = cs35l56_init(cs35l56); - if (ret == 0) - ret = cs35l56_irq_request(&cs35l56->base, client->irq); + ret = cs35l56_irq_request(&cs35l56->base, client->irq); if (ret < 0) cs35l56_remove(cs35l56); diff --git a/sound/soc/codecs/cs35l56-spi.c b/sound/soc/codecs/cs35l56-spi.c index 9bc9b7c98390..b1eb924a5b6c 100644 --- a/sound/soc/codecs/cs35l56-spi.c +++ b/sound/soc/codecs/cs35l56-spi.c @@ -44,9 +44,7 @@ static int cs35l56_spi_probe(struct spi_device *spi) if (ret != 0) return ret; - ret = cs35l56_init(cs35l56); - if (ret == 0) - ret = cs35l56_irq_request(&cs35l56->base, spi->irq); + ret = cs35l56_irq_request(&cs35l56->base, spi->irq); if (ret < 0) cs35l56_remove(cs35l56); diff --git a/sound/soc/codecs/cs35l56.c b/sound/soc/codecs/cs35l56.c index 7b57ef2336ea..e3334b2d8453 100644 --- a/sound/soc/codecs/cs35l56.c +++ b/sound/soc/codecs/cs35l56.c @@ -1437,9 +1437,19 @@ int cs35l56_common_probe(struct cs35l56_private *cs35l56) goto err; } - ret = devm_snd_soc_register_component(cs35l56->base.dev, - &soc_component_dev_cs35l56, - cs35l56_dai, ARRAY_SIZE(cs35l56_dai)); + /* + * On SoundWire the cs35l56_init() cannot be run until after the + * device has been enumerated by the SoundWire core. + */ + if (!cs35l56->sdw_peripheral) { + ret = cs35l56_init(cs35l56); + if (ret) + goto err_remove_wm_adsp; + } + + ret = snd_soc_register_component(cs35l56->base.dev, + &soc_component_dev_cs35l56, + cs35l56_dai, ARRAY_SIZE(cs35l56_dai)); if (ret < 0) { dev_err_probe(cs35l56->base.dev, ret, "Register codec failed\n"); goto err_remove_wm_adsp; @@ -1451,6 +1461,11 @@ int cs35l56_common_probe(struct cs35l56_private *cs35l56) wm_adsp2_remove(&cs35l56->dsp); err: + if (pm_runtime_enabled(cs35l56->base.dev)) { + pm_runtime_dont_use_autosuspend(cs35l56->base.dev); + pm_runtime_disable(cs35l56->base.dev); + } + gpiod_set_value_cansleep(cs35l56->base.reset_gpio, 0); regulator_bulk_disable(ARRAY_SIZE(cs35l56->supplies), cs35l56->supplies); @@ -1533,7 +1548,7 @@ int cs35l56_init(struct cs35l56_private *cs35l56) return dev_err_probe(cs35l56->base.dev, ret, "Failed to write ASP1_CONTROL3\n"); cs35l56->base.init_done = true; - complete(&cs35l56->init_completion); + complete_all(&cs35l56->init_completion); return 0; } @@ -1541,6 +1556,8 @@ EXPORT_SYMBOL_NS_GPL(cs35l56_init, "SND_SOC_CS35L56_CORE"); void cs35l56_remove(struct cs35l56_private *cs35l56) { + snd_soc_unregister_component(cs35l56->base.dev); + cs35l56->base.init_done = false; /* diff --git a/sound/soc/codecs/cs42l43-jack.c b/sound/soc/codecs/cs42l43-jack.c index ecba6c795238..36fb67059e06 100644 --- a/sound/soc/codecs/cs42l43-jack.c +++ b/sound/soc/codecs/cs42l43-jack.c @@ -310,6 +310,7 @@ irqreturn_t cs42l43_bias_detect_clamp(int irq, void *data) #define CS42L43_JACK_ABSENT 0x0 #define CS42L43_JACK_OPTICAL (SND_JACK_MECHANICAL | SND_JACK_AVOUT) +#define CS42L43_JACK_MICROPHONE (SND_JACK_MECHANICAL | SND_JACK_MICROPHONE) #define CS42L43_JACK_HEADPHONE (SND_JACK_MECHANICAL | SND_JACK_HEADPHONE) #define CS42L43_JACK_HEADSET (SND_JACK_MECHANICAL | SND_JACK_HEADSET) #define CS42L43_JACK_LINEOUT (SND_JACK_MECHANICAL | SND_JACK_LINEOUT) @@ -871,7 +872,7 @@ static const struct cs42l43_jack_override_mode { .hsdet_mode = CS42L43_JACK_3_POLE_SWITCHES, .mic_ctrl = (0x3 << CS42L43_JACK_STEREO_CONFIG_SHIFT) | CS42L43_HS1_BIAS_EN_MASK | CS42L43_HS2_BIAS_EN_MASK, - .report = CS42L43_JACK_LINEIN, + .report = CS42L43_JACK_MICROPHONE, }, [CS42L43_JACK_RAW_OPTICAL] = { .hsdet_mode = CS42L43_JACK_3_POLE_SWITCHES, diff --git a/sound/soc/codecs/tas2562.c b/sound/soc/codecs/tas2562.c index 8e00dcc09d0c..795c66cb4713 100644 --- a/sound/soc/codecs/tas2562.c +++ b/sound/soc/codecs/tas2562.c @@ -675,11 +675,12 @@ static int tas2562_parse_dt(struct tas2562_data *tas2562) if (tas2562->sdz_gpio == NULL) { tas2562->sdz_gpio = devm_gpiod_get_optional(dev, "shut-down", GPIOD_OUT_HIGH); - if (IS_ERR(tas2562->sdz_gpio)) + if (IS_ERR(tas2562->sdz_gpio)) { if (PTR_ERR(tas2562->sdz_gpio) == -EPROBE_DEFER) return -EPROBE_DEFER; - tas2562->sdz_gpio = NULL; + tas2562->sdz_gpio = NULL; + } } if (tas2562->model_id == TAS2110) diff --git a/sound/soc/codecs/tas2781-fmwlib.c b/sound/soc/codecs/tas2781-fmwlib.c index 78fd0a5dc6f2..2f6522f76df9 100644 --- a/sound/soc/codecs/tas2781-fmwlib.c +++ b/sound/soc/codecs/tas2781-fmwlib.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -1082,13 +1083,42 @@ static int tasdevice_load_block_kernel( return 0; } +static int tasdevice_fw_strnlen(const struct firmware *fmw, int offset) +{ + const u8 *start; + const u8 *nul; + size_t remaining; + size_t len; + + if (offset < 0 || offset >= fmw->size) + return -EINVAL; + + start = fmw->data + offset; + remaining = fmw->size - offset; + nul = memchr(start, '\0', remaining); + if (!nul) + return -EINVAL; + + len = nul - start; + if (len > INT_MAX) + return -EOVERFLOW; + + return len; +} + static int fw_parse_variable_hdr(struct tasdevice_priv *tas_priv, struct tasdevice_dspfw_hdr *fw_hdr, const struct firmware *fmw, int offset) { const unsigned char *buf = fmw->data; - int len = strlen((char *)&buf[offset]); + int len; + len = tasdevice_fw_strnlen(fmw, offset); + if (len < 0) { + dev_err(tas_priv->dev, "%s: Description error\n", __func__); + offset = len; + goto out; + } len++; if (offset + len + 8 > fmw->size) { @@ -1220,7 +1250,12 @@ static int fw_parse_data(struct tasdevice_fw *tas_fmw, memcpy(img_data->name, &data[offset], 64); offset += 64; - n = strlen((char *)&data[offset]); + n = tasdevice_fw_strnlen(fmw, offset); + if (n < 0) { + dev_err(tas_fmw->dev, "%s: Description error\n", __func__); + offset = n; + goto out; + } n++; if (offset + n + 2 > fmw->size) { dev_err(tas_fmw->dev, "%s: Description error\n", __func__); @@ -1293,7 +1328,12 @@ static int fw_parse_program_data(struct tasdevice_priv *tas_priv, } offset += 64; - n = strlen((char *)&buf[offset]); + n = tasdevice_fw_strnlen(fmw, offset); + if (n < 0) { + dev_err(tas_priv->dev, "Description err\n"); + offset = n; + goto out; + } /* skip '\0' and 5 unused bytes */ n += 6; if (offset + n > fmw->size) { @@ -1356,7 +1396,12 @@ static int fw_parse_configuration_data( memcpy(config->name, &data[offset], 64); offset += 64; - n = strlen((char *)&data[offset]); + n = tasdevice_fw_strnlen(fmw, offset); + if (n < 0) { + dev_err(tas_priv->dev, "Description err\n"); + offset = n; + goto out; + } n += 15; if (offset + n > fmw->size) { dev_err(tas_priv->dev, "Description err\n"); @@ -2038,7 +2083,8 @@ static int fw_parse_calibration_data(struct tasdevice_priv *tas_priv, { struct tasdevice_calibration *calibration; unsigned char *data = (unsigned char *)fmw->data; - unsigned int i, n; + unsigned int i; + int n; if (offset + 2 > fmw->size) { dev_err(tas_priv->dev, "%s: Calibrations error\n", __func__); @@ -2070,7 +2116,12 @@ static int fw_parse_calibration_data(struct tasdevice_priv *tas_priv, calibration = &(tas_fmw->calibrations[i]); offset += 64; - n = strlen((char *)&data[offset]); + n = tasdevice_fw_strnlen(fmw, offset); + if (n < 0) { + dev_err(tas_priv->dev, "Description err\n"); + offset = n; + goto out; + } /* skip '\0' and 2 unused bytes */ n += 3; if (offset + n > fmw->size) { diff --git a/sound/soc/fsl/fsl_sai.c b/sound/soc/fsl/fsl_sai.c index d48f72e5d49b..03107feb9eca 100644 --- a/sound/soc/fsl/fsl_sai.c +++ b/sound/soc/fsl/fsl_sai.c @@ -757,6 +757,8 @@ static int fsl_sai_hw_free(struct snd_pcm_substream *substream, struct fsl_sai *sai = snd_soc_dai_get_drvdata(cpu_dai); bool tx = substream->stream == SNDRV_PCM_STREAM_PLAYBACK; unsigned int ofs = sai->soc_data->reg_offset; + int adir = tx ? RX : TX; + int dir = tx ? TX : RX; /* Clear xMR to avoid channel swap with mclk_with_tere enabled case */ regmap_write(sai->regmap, FSL_SAI_xMR(tx), 0); @@ -764,10 +766,29 @@ static int fsl_sai_hw_free(struct snd_pcm_substream *substream, regmap_update_bits(sai->regmap, FSL_SAI_xCR3(tx, ofs), FSL_SAI_CR3_TRCE_MASK, 0); - if (!sai->is_consumer_mode[tx] && - sai->mclk_streams & BIT(substream->stream)) { - clk_disable_unprepare(sai->mclk_clk[sai->mclk_id[tx]]); - sai->mclk_streams &= ~BIT(substream->stream); + if (!sai->is_consumer_mode[tx]) { + bool adir_active = !!(sai->mclk_streams & BIT(!substream->stream)); + /* + * If opposite stream provides clocks for synchronous mode and + * it is inactive, Clear BYP and BCI + */ + if (fsl_sai_dir_is_synced(sai, adir) && !adir_active) + regmap_update_bits(sai->regmap, FSL_SAI_xCR2(!tx, ofs), + FSL_SAI_CR2_BCI | FSL_SAI_CR2_BYP, 0); + /* + * Clear BYP and BCI of current stream if either of: + * 1. current stream doesn't provide clocks for synchronous mode + * 2. current stream provides clocks for synchronous mode but no + * more stream is active. + */ + if (!fsl_sai_dir_is_synced(sai, dir) || !adir_active) + regmap_update_bits(sai->regmap, FSL_SAI_xCR2(tx, ofs), + FSL_SAI_CR2_BCI | FSL_SAI_CR2_BYP, 0); + + if (sai->mclk_streams & BIT(substream->stream)) { + clk_disable_unprepare(sai->mclk_clk[sai->mclk_id[tx]]); + sai->mclk_streams &= ~BIT(substream->stream); + } } return 0; diff --git a/sound/soc/fsl/imx-card.c b/sound/soc/fsl/imx-card.c index a4518fefad69..43438af1e1c6 100644 --- a/sound/soc/fsl/imx-card.c +++ b/sound/soc/fsl/imx-card.c @@ -496,11 +496,15 @@ static void imx_aif_shutdown(struct snd_pcm_substream *substream) struct snd_soc_dai *codec_dai; int i; - for_each_rtd_cpu_dais(rtd, i, cpu_dai) - snd_soc_dai_set_sysclk(cpu_dai, 0, 0, SND_SOC_CLOCK_OUT); + for_each_rtd_cpu_dais(rtd, i, cpu_dai) { + if (!snd_soc_dai_active(cpu_dai)) + snd_soc_dai_set_sysclk(cpu_dai, 0, 0, SND_SOC_CLOCK_OUT); + } - for_each_rtd_codec_dais(rtd, i, codec_dai) - snd_soc_dai_set_sysclk(codec_dai, 0, 0, SND_SOC_CLOCK_IN); + for_each_rtd_codec_dais(rtd, i, codec_dai) { + if (!snd_soc_dai_active(codec_dai)) + snd_soc_dai_set_sysclk(codec_dai, 0, 0, SND_SOC_CLOCK_IN); + } } static const struct snd_soc_ops imx_aif_ops = { diff --git a/sound/soc/meson/aiu-fifo-spdif.c b/sound/soc/meson/aiu-fifo-spdif.c index e0e00ec026dc..a9861c5d6637 100644 --- a/sound/soc/meson/aiu-fifo-spdif.c +++ b/sound/soc/meson/aiu-fifo-spdif.c @@ -24,6 +24,7 @@ #define AIU_MEM_IEC958_CONTROL_MODE_16BIT BIT(7) #define AIU_MEM_IEC958_CONTROL_MODE_LINEAR BIT(8) #define AIU_MEM_IEC958_BUF_CNTL_INIT BIT(0) +#define AIU_RST_SOFT_958_FAST BIT(2) #define AIU_FIFO_SPDIF_BLOCK 8 @@ -68,11 +69,15 @@ static int fifo_spdif_trigger(struct snd_pcm_substream *substream, int cmd, case SNDRV_PCM_TRIGGER_START: case SNDRV_PCM_TRIGGER_RESUME: case SNDRV_PCM_TRIGGER_PAUSE_RELEASE: + snd_soc_component_write(component, AIU_RST_SOFT, + AIU_RST_SOFT_958_FAST); fifo_spdif_dcu_enable(component, true); break; case SNDRV_PCM_TRIGGER_SUSPEND: case SNDRV_PCM_TRIGGER_PAUSE_PUSH: case SNDRV_PCM_TRIGGER_STOP: + snd_soc_component_write(component, AIU_RST_SOFT, + AIU_RST_SOFT_958_FAST); fifo_spdif_dcu_enable(component, false); break; default: diff --git a/sound/soc/sunxi/sun4i-codec.c b/sound/soc/sunxi/sun4i-codec.c index 93733ff2e32a..3b92318e18ba 100644 --- a/sound/soc/sunxi/sun4i-codec.c +++ b/sound/soc/sunxi/sun4i-codec.c @@ -2236,6 +2236,7 @@ static const struct sun4i_codec_quirks sun50i_h616_codec_quirks = { .reg_dac_fifoc = REG_FIELD(SUN50I_H616_CODEC_DAC_FIFOC, 0, 31), .reg_dac_txdata = SUN8I_H3_CODEC_DAC_TXDATA, .has_reset = true, + .playback_only = true, .dma_max_burst = SUN4I_DMA_MAX_BURST, }; diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c index 1b4cfc2b68f9..3d81b003a510 100644 --- a/sound/usb/quirks.c +++ b/sound/usb/quirks.c @@ -2380,6 +2380,8 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { QUIRK_FLAG_DSD_RAW), DEVICE_FLG(0x2708, 0x0002, /* Audient iD14 */ QUIRK_FLAG_IGNORE_CTL_ERROR), + DEVICE_FLG(0x2772, 0x0502, /* Musical Fidelity M6s DAC */ + 0), /* for avoiding QUIRK_FLAG_DSD_RAW with vendor match */ DEVICE_FLG(0x2912, 0x30c8, /* Audioengine D1 */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x2a70, 0x1881, /* OnePlus Technology (Shenzhen) Co., Ltd. BE02T */ diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 75b6592afcc2..c7d64ab2ea4c 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -228,6 +228,7 @@ static bool is_rust_noreturn(const struct symbol *func) str_ends_with(func->name, "_4core9panicking18panic_nounwind_fmt") || str_ends_with(func->name, "_4core9panicking19assert_failed_inner") || str_ends_with(func->name, "_4core9panicking30panic_null_pointer_dereference") || + str_ends_with(func->name, "_4core9panicking32panic_null_reference_constructed") || str_ends_with(func->name, "_4core9panicking36panic_misaligned_pointer_dereference") || str_ends_with(func->name, "_7___rustc17rust_begin_unwind") || strstr(func->name, "_4core9panicking13assert_failed") || diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index 0d5ce4b74b9f..927fbb6c061f 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -58,12 +58,12 @@ cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o -cxl_core-y += $(CXL_CORE_SRC)/ras.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o +cxl_core-$(CONFIG_CXL_RAS) += $(CXL_CORE_SRC)/ras.o cxl_core-y += config_check.o cxl_core-y += cxl_core_test.o cxl_core-y += cxl_core_exports.o diff --git a/tools/testing/selftests/alsa/mixer-test.c b/tools/testing/selftests/alsa/mixer-test.c index e113dafa5c24..cfb7968fbac0 100644 --- a/tools/testing/selftests/alsa/mixer-test.c +++ b/tools/testing/selftests/alsa/mixer-test.c @@ -84,6 +84,7 @@ static void find_controls(void) if (err < 0) { ksft_print_msg("Failed to get hctl for card %d: %s\n", card, snd_strerror(err)); + free(card_data); goto next_card; } diff --git a/tools/testing/selftests/bpf/progs/verifier_ld_ind.c b/tools/testing/selftests/bpf/progs/verifier_ld_ind.c index c925ba9a2e74..09e81b99eecb 100644 --- a/tools/testing/selftests/bpf/progs/verifier_ld_ind.c +++ b/tools/testing/selftests/bpf/progs/verifier_ld_ind.c @@ -107,4 +107,146 @@ __naked void ind_check_calling_conv_r7(void) : __clobber_all); } +/* + * ld_{abs,ind} subprog that always sets r0=1 on the success path. + * bpf_gen_ld_abs() emits a hidden exit with r0=0 when the load helper + * fails. The verifier must model this failure return so that callers + * account for r0=0 as a possible return value. + */ +__naked __noinline __used +static int ldabs_subprog(void) +{ + asm volatile ( + "r6 = r1;" + ".8byte %[ld_abs];" + "r0 = 1;" + "exit;" + : + : __imm_insn(ld_abs, BPF_LD_ABS(BPF_W, 0)) + : __clobber_all); +} + +__naked __noinline __used +static int ldind_subprog(void) +{ + asm volatile ( + "r6 = r1;" + "r7 = 0;" + ".8byte %[ld_ind];" + "r0 = 1;" + "exit;" + : + : __imm_insn(ld_ind, BPF_LD_IND(BPF_W, BPF_REG_7, 0)) + : __clobber_all); +} + +SEC("socket") +__description("ld_abs: subprog early exit on ld_abs failure") +__failure __msg("R9 !read_ok") +__naked void ld_abs_subprog_early_exit(void) +{ + asm volatile ( + "call ldabs_subprog;" + "if r0 != 0 goto l_exit_%=;" + "r0 = r9;" + "l_exit_%=:" + "r0 = 0;" + "exit;" + ::: __clobber_all); +} + +SEC("socket") +__description("ld_ind: subprog early exit on ld_ind failure") +__failure __msg("R9 !read_ok") +__naked void ld_ind_subprog_early_exit(void) +{ + asm volatile ( + "call ldind_subprog;" + "if r0 != 0 goto l_exit_%=;" + "r0 = r9;" + "l_exit_%=:" + "r0 = 0;" + "exit;" + ::: __clobber_all); +} + +SEC("socket") +__description("ld_abs: subprog with both paths safe") +__success +__naked void ld_abs_subprog_both_paths_safe(void) +{ + asm volatile ( + "call ldabs_subprog;" + "r0 = 0;" + "exit;" + ::: __clobber_all); +} + +SEC("socket") +__description("ld_ind: subprog with both paths safe") +__success +__naked void ld_ind_subprog_both_paths_safe(void) +{ + asm volatile ( + "call ldind_subprog;" + "r0 = 0;" + "exit;" + ::: __clobber_all); +} + +/* + * ld_{abs,ind} in subprogs require scalar (int) return type in BTF. + * A test with void return must be rejected. + */ +__naked __noinline __used +static void ldabs_void_subprog(void) +{ + asm volatile ( + "r6 = r1;" + ".8byte %[ld_abs];" + "r0 = 1;" + "exit;" + : + : __imm_insn(ld_abs, BPF_LD_ABS(BPF_W, 0)) + : __clobber_all); +} + +SEC("socket") +__description("ld_abs: reject void return subprog") +__failure __msg("LD_ABS is only allowed in functions that return 'int'") +__naked void ld_abs_void_subprog_reject(void) +{ + asm volatile ( + "call ldabs_void_subprog;" + "r0 = 0;" + "exit;" + ::: __clobber_all); +} + +__naked __noinline __used +static void ldind_void_subprog(void) +{ + asm volatile ( + "r6 = r1;" + "r7 = 0;" + ".8byte %[ld_ind];" + "r0 = 1;" + "exit;" + : + : __imm_insn(ld_ind, BPF_LD_IND(BPF_W, BPF_REG_7, 0)) + : __clobber_all); +} + +SEC("socket") +__description("ld_ind: reject void return subprog") +__failure __msg("LD_ABS is only allowed in functions that return 'int'") +__naked void ld_ind_void_subprog_reject(void) +{ + asm volatile ( + "call ldind_void_subprog;" + "r0 = 0;" + "exit;" + ::: __clobber_all); +} + char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/verifier_map_ptr.c b/tools/testing/selftests/bpf/progs/verifier_map_ptr.c index e2767d27d8aa..166193659870 100644 --- a/tools/testing/selftests/bpf/progs/verifier_map_ptr.c +++ b/tools/testing/selftests/bpf/progs/verifier_map_ptr.c @@ -70,17 +70,45 @@ __naked void bpf_map_ptr_write_rejected(void) : __clobber_all); } -/* The first element of struct bpf_map is a SHA256 hash of 32 bytes, accessing - * into this array is valid. The opts field is now at offset 33. +/* + * struct bpf_map starts with the SHA256 hash sha[32] at offset 0 (a readable + * byte array), the u32 excl field at offset 32, and the ops pointer at offset + * 40. Reading a u32 at offset 41 reaches into the middle of the ops pointer, + * i.e. a partial pointer access, which is rejected. */ SEC("socket") __description("bpf_map_ptr: read non-existent field rejected") __failure -__msg("cannot access ptr member ops with moff 32 in struct bpf_map with off 33 size 4") +__msg("cannot access ptr member ops with moff 40 in struct bpf_map with off 41 size 4") __failure_unpriv __msg_unpriv("access is allowed only to CAP_PERFMON and CAP_SYS_ADMIN") __flag(BPF_F_ANY_ALIGNMENT) __naked void read_non_existent_field_rejected(void) +{ + asm volatile (" \ + r6 = 0; \ + r1 = %[map_array_48b] ll; \ + r6 = *(u32*)(r1 + 41); \ + r0 = 1; \ + exit; \ +" : + : __imm_addr(map_array_48b) + : __clobber_all); +} + +/* + * The u32 excl field spans offsets 32..35 (mend 36). Reading a u32 at offset + * 33 starts inside excl but extends past its end, which the verifier rejects + * as an out-of-bounds scalar access. + */ +SEC("socket") +__description("bpf_map_ptr: read beyond excl field rejected") +__failure +__msg("access beyond the end of member excl (mend:36) in struct bpf_map with off 33 size 4") +__failure_unpriv +__msg_unpriv("access is allowed only to CAP_PERFMON and CAP_SYS_ADMIN") +__flag(BPF_F_ANY_ALIGNMENT) +__naked void read_beyond_excl_field_rejected(void) { asm volatile (" \ r6 = 0; \ @@ -103,7 +131,7 @@ __naked void ptr_read_ops_field_accepted(void) asm volatile (" \ r6 = 0; \ r1 = %[map_array_48b] ll; \ - r6 = *(u64*)(r1 + 0); \ + r6 = *(u64*)(r1 + 40); \ r0 = 1; \ exit; \ " : diff --git a/tools/testing/selftests/drivers/net/settings b/tools/testing/selftests/drivers/net/settings new file mode 100644 index 000000000000..eef533824a3c --- /dev/null +++ b/tools/testing/selftests/drivers/net/settings @@ -0,0 +1 @@ +timeout=360 diff --git a/tools/testing/selftests/drivers/ntsync/config b/tools/testing/selftests/drivers/ntsync/config index 60539c826d06..0aa68de147af 100644 --- a/tools/testing/selftests/drivers/ntsync/config +++ b/tools/testing/selftests/drivers/ntsync/config @@ -1 +1 @@ -CONFIG_WINESYNC=y +CONFIG_NTSYNC=y diff --git a/tools/testing/selftests/net/af_unix/config b/tools/testing/selftests/net/af_unix/config index b5429c15a53c..41dbb03c747e 100644 --- a/tools/testing/selftests/net/af_unix/config +++ b/tools/testing/selftests/net/af_unix/config @@ -1,3 +1,4 @@ CONFIG_AF_UNIX_OOB=y CONFIG_UNIX=y CONFIG_UNIX_DIAG=m +CONFIG_USER_NS=y diff --git a/tools/testing/selftests/net/openvswitch/config b/tools/testing/selftests/net/openvswitch/config new file mode 100644 index 000000000000..c659749cd086 --- /dev/null +++ b/tools/testing/selftests/net/openvswitch/config @@ -0,0 +1,16 @@ +CONFIG_GENEVE=m +CONFIG_INET_DIAG=y +CONFIG_IPV6=y +CONFIG_NETFILTER=y +CONFIG_NET_IPGRE=m +CONFIG_NET_IPGRE_DEMUX=m +CONFIG_NF_CONNTRACK=m +CONFIG_NF_CONNTRACK_OVS=y +CONFIG_OPENVSWITCH=m +CONFIG_OPENVSWITCH_GENEVE=m +CONFIG_OPENVSWITCH_GRE=m +CONFIG_OPENVSWITCH_VXLAN=m +CONFIG_PSAMPLE=m +CONFIG_VETH=y +CONFIG_VLAN_8021Q=y +CONFIG_VXLAN=m diff --git a/tools/testing/selftests/net/ovpn/config b/tools/testing/selftests/net/ovpn/config index 42699740936d..c0a21a43018a 100644 --- a/tools/testing/selftests/net/ovpn/config +++ b/tools/testing/selftests/net/ovpn/config @@ -4,7 +4,9 @@ CONFIG_CRYPTO_CHACHA20POLY1305=y CONFIG_CRYPTO_GCM=y CONFIG_DST_CACHE=y CONFIG_INET=y +CONFIG_IPV6=y CONFIG_NET=y CONFIG_NET_UDP_TUNNEL=y CONFIG_OVPN=m CONFIG_STREAM_PARSER=y +CONFIG_VETH=y diff --git a/tools/testing/selftests/net/ovpn/ovpn-cli.c b/tools/testing/selftests/net/ovpn/ovpn-cli.c index 0a5226196a2e..44418118f6ea 100644 --- a/tools/testing/selftests/net/ovpn/ovpn-cli.c +++ b/tools/testing/selftests/net/ovpn/ovpn-cli.c @@ -1746,7 +1746,7 @@ static int ovpn_parse_remote(struct ovpn_ctx *ovpn, const char *host, const char *service, const char *vpnip) { int ret; - struct addrinfo *result; + struct addrinfo *result = NULL; struct addrinfo hints = { .ai_family = ovpn->sa_family, .ai_socktype = SOCK_DGRAM, @@ -1770,6 +1770,8 @@ static int ovpn_parse_remote(struct ovpn_ctx *ovpn, const char *host, } memcpy(&ovpn->remote, result->ai_addr, result->ai_addrlen); + freeaddrinfo(result); + result = NULL; } if (vpnip) { diff --git a/tools/testing/selftests/net/ovpn/settings b/tools/testing/selftests/net/ovpn/settings new file mode 100644 index 000000000000..ba4d85f74cd6 --- /dev/null +++ b/tools/testing/selftests/net/ovpn/settings @@ -0,0 +1 @@ +timeout=90